Log file Viewer
#mongodb logs for Saturday the 5th of January, 2013
[00:24:05] <fommil> hi all – if I do an update on a document, but don't specify some of the fields, will they be deleted?
[00:25:50] <drjeats> fommil: by default, yes. use $set (I believe that's the one, check the docs) to add/remove
[00:41:04] <fommil> if I do a sorted search and go through the cursor, and this takes say 5 minutes or so – what will happen if the collection is edited in that time?
[00:45:41] <fommil> also, does the timeout work for the whole search, or just for each next() call of the cursor?
[02:02:49] <Dr{Who}> anyone know what a bson type of 120 is? I am storing what I think is a copy of a bson oid type in a BSONElement but it seems to change from 7 to 120 I am thiking its not a copy but a pointer to the real record? so what is 120 and what is a safe way to copy an BSONElement
[02:04:58] <Dr{Who}> meh. ya just saying BSONElement A = B does not actually copy the data looks to be by ref as soon as i do a c.next() it changes to 120
[02:12:22] <Dr{Who}> ok i see part of the issue. I do copy the BSONObj using .copy but then extract to an element the _id once I change my copy it breaks. So I need a real copy of my field in my case _id something I can keep around a bit.
[02:41:34] <Guest_1448> is it possible to group documents by a field and return a result with the documents keyed by field?
[02:43:50] <Guest_1448> e.g. if I have [{a: 'b', foo: 'bar'}, {a: 'b', y: 'z'}, {a: 'c', blah: 'blah'}] I want mongo to group by 'a' field and return {'b': [{a: 'b', foo: 'bar'}, {a: 'b', y: 'z'}], 'c': [{a: 'c', blah: 'blah'}]}
[03:37:39] <Guest_1448> what's better - .group(..) with a function to count or .aggregate(..) with count: {$sum: 1} ?
[03:56:37] <Guest_1448> and regarding my original question, I managed to do that using .group() + reduce function
[04:34:57] <Dr{Who}> collection.stats does not seem to include anything that is clear to be a unique id of the collection. does such a best exist?
[07:22:49] <dawra> i read this, but i cannot figure out how to do a query like WHERE field != '0' AND field > 0
[11:57:46] <hever> Hello, as mongo db provides using Array.unique in Map/Reduce, I'm asking myself what kind of JavaScript Version or JavaScript Frameworks works serverside because Array.unique is not a "standard" JS function.
[12:06:20] <hever_> Hello, as mongo db provides using Array.unique in Map/Reduce, I'm asking myself what kind of JavaScript Version or JavaScript Frameworks works serverside because Array.unique is not a "standard" JS function.
[13:09:37] <hever_> Can I somehow achieve that data that have just one value are ignored and not part of the result ?
[13:11:35] <hever_> I'm using map and reduce and if there are data with just one value there's nothing todo for reduce (and it's not passed to reduce) so in finalize I've to change the data but I'm going to skip the data there from the result or by some flag perhaps...
[13:12:43] <kali> hever_: there is something fichy here. reduce should not change the format of the data, just aggregate it
[13:13:39] <kali> hever_: so basically, the idea is, if one key is emitted only once, it's value as output of map should be in the final state
[13:17:38] <kali> hever_: also, if you can achieve whatever you're doing with the aggregation framework, it will be way faster
[13:19:41] <kali> hever_: also, if you can achieve whatever you're doing with the aggregation framework, it will be way faster
[15:29:10] <bizzle> Does anyone know how in java, I can not have an insert attempt throw and exception on a duplicate key error? I would like to just handle the application by doing commandResult.getLastError.ok()
[15:38:59] <Guest_1448> is it possible to find out the total amount of documents that the query would return IF I hadn't applied the limit?
[15:40:27] <Guest_1448> can I do foo=db.coll.find({...}); foo.count() then continue calling .limit() etc on the 'foo' cursor?
[15:51:34] <skot> yes, count does not change foo, but simply copies stuff out of it to run the count command.
[15:56:11] <Guest_1448> is there a way to 'apply' the options to a cursor later then? like .find(..).options({sort:'x', limit: 200}) instead of .find(..., options)
[16:23:04] <skot> anyway, the cursor is just a holder for fields which will be sent to the server before you start getting results.
[16:28:06] <Guest_1448> I want to do a .find(query), get the count then apply the rest of the options/paging stuff and carry on
[16:42:22] <igotux> hey guys.. am getting this on a slave node :- "[replslave] all sources dead: data too stale halted replication, sleeping for 5 seconds"
[17:01:50] <igotux> currently our master is small.. so thought of porting this data to a bigger node first and then we will make this slave node master and start replica set
[17:04:47] <igotux> it synces almost all of the data to new system and around at the end, it started showing this error and stops replication...
[17:05:26] <kali> so you had one master and one slave, but you outgrowed them, so you're trying to substitute the slave by a bigger one, and plan to use this slave as a seed to a replica set ?
[17:06:50] <igotux> our DB is growing pretty fast and now we need to convert it to a RS ... so we are using a big node and made as slave of current master.. while current slave is also syncing from current master...
[17:09:44] <igotux> does this help :- http://pastie.org/pastes/5627196/text?key=wgf5lm3cza2yithp9mcbba
[17:09:56] <kali> master / slave is so old and deprecated, most people with live experience are dead and burried :)
[17:10:39] <kali> i think i would try to degrade the current master/slave to standalone, and seed the replica set from there
[17:12:19] <kali> mmm yeah: 1/ make the current master a standalone node 2/ check everything is fine 3/ restart the standalone node with a rs name, add a node, and hope the sync will be fast enough
[17:13:17] <kali> igotux: but i've never done that: i don't know how "broken" is a master slave if you try to "downgrade" it to standalone
[17:14:16] <igotux> there is no change on master side if you make it a standalone na.. as per my understanding, a master node is a standalone node as well..correct ?
[17:15:29] <kali> but i have some on RS, so at least, i might help you once we're on this terrain :)
[17:16:24] <igotux> kali: in the pasteit, "Sat Jan 5 08:32:12 [replslave] repl: nextOpTime Jan 5 06:26:37 50e7c79d:4 > syncedTo Jan 5 05:26:56 50e7b9a0:1c"
[17:24:51] <kali> yes, it synced to a state 4 or 5 hours stale, and the oplog is not deep enough to close the gap
[17:24:53] <igotux> can you explain the relation ship of oplog < 2 hours and sync getting broken ?..can you pls explain
[17:25:53] <kali> i think so. make the oplog 128G, assuming you have the disk space, and restart the sync
[17:26:48] <igotux> or what about , we sync the data manually from the backup mongodump to the new node and make the new node as slave..
[17:29:13] <kali> but just checking the documentation, it does not says you can perform a resync with a dump
[17:29:37] <kali> and honestly, if the initial syn took 5 hours, i doubt you can perform a dump and a restore in less than two hours
[22:17:30] <timah> i've thoroughly combed the inter webs for an answer, some clues, anything really.
[22:21:25] <timah> my event documents look something like this: { _id: { g: 123, o: Object() }, a: 0, b: 1, c: 2, d: 3 }
[22:24:55] <timah> that's the new design… the previous design (created by someone other than myself) :) looked something like this: { _id: ObjectId(), properties: { some_key: 'some_static_option_value', some_other_long_key: 'some_other_long_static_option_value', etc: 'etc_etc' } }
[22:27:49] <timah> my new design includes the application layer handling the on-the-fly transformation (compacting, really) of both keys and values for all of this static data.
[22:31:56] <timah> by utilizing multiple collections (g.1.e - events, g.1.c - counts/aggregates), as well as including the second level grouping id in the document._id itself i've exponentially reduced the overall size of the data (dataSize?), as well as improved performance.
[22:33:22] <timah> however… it seems as though either the preallocation or nssize is forcing my uber tiny documents to still pad much more than necessary.
[22:34:23] <timah> in size that is… if i'm not being clear i apologize… it's not really all that complicated… i'm really just trying to figure out why the size on disk doesn't seem to change much, even though the size of my documents has changed alot.
[22:37:29] <Dr{Who}> timah: you may try exporting all the bson data and then re importing or just doing a repair on the collection to see if it changes.
[22:43:50] <Dr{Who}> timah be warned btree indexes in general on any database tend to fall apart for insert speed when you get into the millions or billions of records.
[22:46:35] <Dr{Who}> a recent downside I am finding is collections dont seem to have any unique id so it can be hard to manage the shards.
[22:47:43] <cad> Are pymongo lists atomic? For ex. if I append something to a list in an existing document. Than save() it. Will this be atomic ?
[22:49:20] <timah> publishers > merchants (o:m), merchants > consumers (m:m), consumers > events (m:m)
[22:53:39] <Dr{Who}> cad: until you save afaik no communication happens so the original document is not touched.