Log file Viewer
#mongodb logs for Sunday the 21st of April, 2013
[04:39:53] <nemothekid> (I want to write 10 million records to an empty sharded collection, but I don't want all the data to hit a single shard)
[07:36:15] <k610> if i for my collection i want to specify the "_id" myself, should i { "_id" => "mystring" } or { "_id" => BSON::ObjectId("mystring")}
[13:15:42] <efazati> some thing like this -> db.books.find().forEach(function(obj) { obj.update({$set: {'circulation':sum}}) })
[13:40:32] <Bilge> Didn't I read somewhere that it is possible to run mongodb in a mode where it will reject all queries that are not indexed?
[13:41:30] <kali> Bilge: http://docs.mongodb.org/manual/reference/mongod/#cmdoption-mongod--notablescan
[15:13:23] <fomatin> but it ended up: doc1._id != doc2._id; parseInt(doc1._id) == parseInt(doc2._id);
[15:27:14] <OliverJAsh> I have a model called "users" which needs to know how many "posts" that user has. Would it be a good idea to create a property in the "users" collection which represents a "posts" count, and update that every time a new post comes in? How would you do it?
[15:29:15] <kali> OliverJAsh: it's a perfectly valid idea, but you have to do it yourself, in your application code.
[15:30:05] <OliverJAsh> kali: i wasn't sure if it would be best to create a property on the schema, or just count the posts every time a "user" is requested — so it it sent along to the API but it doesn't persist. what's better?
[15:31:36] <kali> OliverJAsh: as i said, it makes sense to have a post counter field on the user document
[16:57:36] <joshua> Anyone familiar with flushRouterConfig command for mongos? Any issue running that periodically?
[17:48:30] <scoates> Given this http://paste.roguecoders.com/p/c11acea40731540ae4da79f38799a675.txt , does anyone know why 1) indexOnly is false, and 2) without the hint(), it uses a different index that has `owner` but not the other fields… ?
[17:56:09] <scoates> first, the hint: it's a new index (last night), so maybe the statistical optimizer hasn't noticed yet?
[17:57:13] <scoates> really? I thought query plans were based on trial+error + best plan…? is that old? or imaginary?
[17:57:40] <scoates> I added the index with the offline build approach (RS). Maybe that has something to do with it?
[17:58:48] <Derick> "indexOnly" only shows up for when your "projection" (select field1, field as opposed to select *) can be satisfied by data coming just from the index
[17:59:36] <Derick> so you need to add { _id: 0, owner: 1, backup: 1, date: 1 } to your query (As 2nd argument)
[18:00:02] <scoates> right. I actually need more data. I just misunderstood indexOnly (thought it had to do with scanning)
[18:00:29] <joshua> Ah so it still has to pull the results from non-indexed data unless you add the projection to limit it. Hmm
[18:02:12] <joshua> Are hints something available from the API to put in code or for the shell, and will it still work if the index doesn't exist?
[18:03:15] <joshua> I guess python too http://api.mongodb.org/python/current/api/pymongo/cursor.html
[18:03:46] <joshua> " Hinting will not do anything if the corresponding index does not exist." that answers my other question
[18:04:02] <scoates> Derick: in production, this query has a limit(), too. I wonder if that affects the `best` logging in the query planner
[18:05:00] <scoates> so, I think my #1 question is resolved (I just misunderstood indexOnly), but #2 about hinting is still confusing.
[18:05:35] <Derick> depending on the case, the index that you think works best is not the one that hits 101 items first
[18:06:35] <joshua> There was a section on the online course where they detailed the order of find operations like sort/limit etc.
[18:06:49] <scoates> Derick: "millis is a number that reflects the time in milliseconds to complete the query." <-- that's the time it takes to complete the actual query, not the time it takes to explain(), right?
[18:08:00] <scoates> I guess that's kind of what I'm asking. I don't know if/how explain()'s inputs/outputs might be cached.
[18:08:51] <scoates> the cursor might be generated differently (as it is for .limit()) if there's an explain() before a read, too (but I hope not)
[18:09:45] <joshua> That data is there if you ask for it or not I thought, cause the profiler will give you some of it for slow queries
[18:10:37] <scoates> joshua: limit() was a bad example there. sort() definitely affects a query between find and read
[18:11:10] <scoates> Derick: anyway, this is what I get (without hint) now: http://paste.roguecoders.com/p/6b47429ea920fdb9fe24c508c2817bd6.txt … thanks for the help
[18:11:42] <scoates> I just need to adjust the app to use `from_backup: {$in: [true, undefined]}` instead of the current `from_backup: {$ne: [false]}`
[18:19:15] <scoates> right. I meant I'm confused by why my hint works better than the query planner.
[18:23:46] <scoates> yeah. that's on the list. there are a bunch. annoying I have to do them offline, though
[18:29:21] <joshua> I have a database with too many indexes at the moment, and I don't write the app. Going to have to go through and determine what needs to be removed. Fun
[18:31:49] <scoates> Derick: "however, when trying the indexes, the first one that has 101 matches, "wins"" <-- even if the n and nscanned are very different, and even if millis is much longer than others returning 101?
[18:34:22] <scoates> joshua: I'd started writing a tool for that (based on the profiler data), but never got it to useful-to-share state. /-:
[18:36:27] <scoates> joshua this might help: http://paste.roguecoders.com/p/05ae5ea9da6b1237bc362b3aaea129f0.txt
[18:36:34] <joshua> There is this one https://github.com/wfreeman/indexalizer but the problem is in production so I don't know if I can turn on profiling. Its annoying not having control over everything
[18:37:53] <joshua> I'll have to go back to the app dev and the person who did the load testing (they seem to keep track of it).
[18:39:00] <scoates> (IIRC to use that script, you need to copy the profiler collection into a not-capped collection (because of the updates))
[20:19:52] <Snii> Hi, I have a selection like {{id:a, rev:0},{id:a, rev:1},{id:b, rev:0},{id:b, rev:1},{id:b, rev:2}} - How can i make a query to get only the highest rev of each id? like {a, 1} and {b, 2}?
[20:32:56] <Snii> Can it be done by a normal find(), or do I need an aggregation or group or something similiar?
[21:38:43] <preinhei_> Didn't work: 'host' => "127.0.0.1:27017,sanantonio.wonderproxy.com:27017, montreal.wonderproxy.com:27017",
[21:41:39] <preinhei_> $mongo_host = "washington.wonderproxy.com:27017,192.168.42.3:27017,sanantonio.wonderproxy.com:27017,montreal.wonderproxy.com:27017";
[21:44:29] <Derick> because the driver will talk to 192.168 and see it wants voltron, and will create another connection to that from the replset config
[21:47:38] <preinhei_> so, i want $mongo_host = "washington.wonderproxy.com:27017,mongo.voltron.wondernetwork.com:27017,sanantonio.wonderproxy.com:27017,montreal.wonderproxy.com:27017";
[21:47:43] <Snii> I'll try once more before going to sleep: I have a selection like {{id:a, rev:0},{id:a, rev:1},{id:b, rev:0},{id:b, rev:1},{id:b, rev:2}} - How can i make a query to get only the highest rev of each id? like {a, 1} and {b, 2}?
[21:51:33] <preinhei_> this paste from earlier, it talks about connecting to washington, then says it hasn't found a server http://paste.roguecoders.com/p/11745a23d2514e57578344f84f128112.txt
[21:56:25] <nemothekid> I am downgrading my cluster to 2.2.4 from 2.4.2. Should I downgrade the config servers as well?
[22:10:04] <Derick> ismaster: the server name (washington.wonderproxy.com:27017) did not match with what we thought it'd be (127.0.0.1:27017)
[22:13:19] <fl0w> I'm doing some reading on MongoDB, and I'm thinking about using it for an experimental project of mine. If I understood things right (comming from a traditional relational DB background), to truly utilize the power of mongo, I shouldn't normalise my data - but say I have a blog posts created by user "John Doe", I shouldn't link it (again, to gain the power of mongo) - but what if the user John changes his name, and I need to update all the records
[22:13:19] <fl0w> based on this name? What if I have multiple John Doe's (though still different users)?
[22:16:08] <Derick> preinhei_: hmm, as long as you have the same names as in your repl set config, and all the members can access them under the same name and IP, it should work
[22:19:46] <preinhei_> fl0w: I'm making it up as I go along, but I'd store a bit on the user there (their name, id and anything else you need to display when showing that blog post), and yes update if a user changes his name
[22:20:26] <preinhei_> at the expense of a larger operation on the rare instance that someone changes their name
[22:20:42] <fl0w> preinhei_: I should however keep a "foreign key"-like identifier though, because otherwise, I'm extremely confused.
[22:21:22] <preinhei_> I'm not storing a blog, but in that similar circumstance I am keeping a foreign key like element