Log file Viewer

[00:39:20] <cpu> I've managed to pull up 7 shards on same machine with ssd and 30 cores (doc size is average < 200byte). One secondary index. Insertion time reaches 10 seconds for 16000 docs, it's unbearably slow. Have I reached a top speed of some sort?

[00:53:17] <nyov> what unit does Object.bsonsize() return? is that bits or bytes, kbyte or KByte?

[00:59:03] <nyov> nvm, found it

[02:20:56] <tg2> @ cpu

[02:20:58] <tg2> what was it doing before?

[02:21:02] <tg2> with a single shard/instance?

[03:03:09] <t0th_-> is a good idea use mongo and sphinx?

[03:21:17] <tg2> yeah

[03:21:27] <tg2> for fulltext nothing beats sphinx

[03:21:43] <tg2> new version uses push too so when you insert a record into the db or upate

[03:21:49] <tg2> you upate sphinx simultaneously

[03:21:53] <tg2> so the index is always fresh

[03:22:15] <tg2> using sphinx isntead of mongo indexes

[03:22:20] <tg2> for searching data

[03:22:21] <tg2> is great

[03:22:36] <tg2> craigslist indexes like 4bn records

[03:22:39] <tg2> using sphinx

[03:28:52] <t0th_-> i am using mysql and sphinx

[03:29:34] <t0th_-> so i don't need use mongo?

[03:30:16] <t0th_-> sphinx is easer to sql database?

[03:38:47] <t0th_-> http://d.pr/i/sI5j

[03:38:48] <t0th_-> why?

[03:44:17] <tg2> sphinx works as your indexing

[03:44:23] <tg2> it doesn't care what is behind it

[03:44:28] <tg2> you are doing your updates via cron

[03:44:31] <tg2> by full indexer --rotate

[03:44:38] <tg2> but you can also push fresh data into sphinx

[03:44:41] <tg2> as you get it

[03:44:50] <tg2> so this way you don't have to do a full db dump every x minutes to have a sphinx index up to date

[03:44:51] <tg2> ;)

[03:45:03] <tg2> mysql is fast for index sorting/searching

[03:45:07] <tg2> but slow for fulltext

[03:45:10] <tg2> so is mongo

[03:45:15] <tg2> sphinx is lightening fast

[09:47:08] <cpu> @tg2 It's worse than before

[09:47:29] <cpu> 40 seconds average instead of ~35

[15:55:59] <amacgregor> Hi

[15:57:37] <amacgregor> I'm trying to get a subdocument query working but I'm not getting anything back; my document looks like this

[15:57:38] <amacgregor> http://hastebin.com/yuwafugogo.coffee

[15:57:53] <amacgregor> and my query looks like the following

[15:57:54] <amacgregor> db.record.find({trackerEvent: {type: "page_info"}})

[16:00:26] <kali> amacgregor: db.record.find({"trackerEvent.type": "page_info"})

[16:00:59] <kali> amacgregor: what you write means trackedEvent is exactly { type: "page_info"}

[16:01:31] <amacgregor> interesting, that works :) kali, thank you

[16:24:15] <cpu> Can someone help me troubleshoot bad performance, I can't seem to get inserts in a normal speed

[16:24:27] <cpu> I've been working on it for two days now

[16:26:04] <kizzx2> cpu: start with commands like iostat, mongostat

[16:26:22] <kizzx2> cpu: has it been "degraded" or is it just not fast enough for you?

[16:29:06] <cpu> It had not been degraded, I can't seem to make it as fast as I think It should be no matter how hardware I throw in

[16:29:24] <cpu> I have all the specs down the the finest I think but still writes are slow

[16:29:27] <kizzx2> what numbers do you get, what do you expect?

[16:29:40] <kizzx2> how do you benchmark it?

[16:30:13] <cpu> My doc size is 200byte average, and I'm doing one insert command of 16,000 docs (should be a few MBs total).

[16:30:24] <cpu> I have SSD, 10 shards of mongo on 10 different CPUs

[16:30:53] <cpu> @kizzx2 did you get my last two lines? (you just went out came back)

[16:31:02] <kizzx2> what's your setup? do you do sharding? do you run your benchmark program single threaded? from how many nodes?

[16:31:11] <kizzx2> cpu: no i just disconnected :P

[16:31:27] <cpu> can I send you in private, I don't want to flood the main channel

[16:32:49] <kizzx2> ok

[16:57:55] <kesor> can someone point me to an article explaining how I can migrate a large mongodb cluster (shards, replicasets, config servers) to new hardware and keeping all the data? preferably without days worth of downtime… about 3-4 terrabyte data total in collections.

[20:09:53] <caitp-> okay, mongoose unique sparse indexes are driving me crazy =___=

[20:10:04] <caitp-> it's like it's completely ignoring the "sparse" factor

[20:23:17] <klj613> mongodb 10gen yum repo packages not signed? :( lol

[22:07:08] <ukd1> Just made a dockerfile for mongo: https://github.com/mongodb/mongo/pull/456

[23:19:27] <w3pm> seems like the erlang driver isn't being maintained, any plans on addressing that?