Log file Viewer
#mongodb logs for Monday the 12th of November, 2012
[00:27:06] <Bilge> I have an embedded document called emails.address and I have an index on emails.address but when I do db.User.find({ 'emails.address': "an@example.com" }).explain() I find that "indexOnly" is false
[00:33:18] <crudson> Bilge: what does db.User.find({ 'emails.address': "an@example.com" }).explain().cursor say?
[00:50:37] <crudson> indexOnly needs to work better, but .cursor is showing correctly that it's being used
[01:11:40] <therealkoopa> I'm struggling with the $match on an aggregation. If I put { $match: { 'people.inactive' : false, _id: id }}, it doesn't return anything if there is a document with all inactive people, but I want it to always return the document, but have the people filtered.
[03:55:01] <ckd> mrpro: https://jira.mongodb.org/browse/SERVER-1240 it's mentioned somewhere near the top that it's scheduled for 2.4, but i'm looking for more concrete info
[03:56:45] <ckd> mrpro: i should walk that back, http://www.mongodb.org/display/DOCS/How+does+concurrency+work mentions it's not necessarily going into 2.4
[04:00:43] <ckd> is it impractical for you to try that suggestion about 2.2 and 1 collection per db?
[05:24:22] <mrpro> but writing to log causes more IO thus slowing down even more incoming transactions
[08:06:33] <samurai2> Hi there, If I use compound keys as my shard key and I use query only using one of them, will that query will be done in parallel? thanks :)
[09:25:01] <Lujeni> Hello - query with the findOne operator works well. However , when i use *find* operator, mongo don't return the cursor ( collection = ~200docs ). Someone have an idea pls?
[10:11:43] <Derick> it has a lot of really nice features that make things like socket work a lot easier
[10:13:43] <bigmeow> Derick: just kidding, the main reason is that i am not root on that machine, i have to build it from source:( what a hell, it's painful:(
[10:15:45] <Derick> bigmeow: what you can do, is download the RPM source and compile that - it should have all the required patches in there
[10:16:22] <bigmeow> Derick: i have only tried to install rpm using rpm -ivh xx.rpm, and it will install that package to place where root priviledge is needed:(
[10:17:12] <bigmeow> Derick: dude, do you use debian? then how to install mongodb to my home directory on debian?
[10:19:29] <bigmeow> Derick: and each database maybe stored in one file, where i can find the format of that file?
[10:21:00] <Derick> http://www.slideshare.net/mdirolf/inside-mongodb-the-internals-of-an-opensource-database is probably your best bet
[10:35:29] <roxlu> I've been asking on the google groups and in this channel for a way to set meta data for files, when using gridfs and the c-driver. When using buffer writes it seems there is no way to add meta data? (when doing a normal gridfs insert you can pass an bson object as meta data).
[10:43:31] <roxlu> https://groups.google.com/forum/?fromgroups=#!searchin/mongodb-user/meta/mongodb-user/k7S9goWVKq4/CjZZIaZufvUJ
[10:51:43] <nico__> Hi, I'm using pymongo and I'm testing the ReplicaSetConnection with read_preference. However with a small python script which lists a collection, the find() seems to be frozen at the end of the loop. If I run the same script with Connection (Single Node) instead of ReplicaSetConnection, the find() terminates normally...
[11:05:51] <bigmeow> Derick: dude, what is this? http://www.slideshare.net/pengwynn/mongodb-ruby-document-store-that-doesnt-rhyme-with-ouch
[11:46:25] <Gargoyle> Derick: Last night when I updated my blog, I am running RC1 ad home (and now RC2) on the server and the result from aggregate() was different. Is that something specific to the aggregate() function or results in general?
[11:54:17] <Gargoyle> Derick: Thanks. Just wanted to check another project wasn't going to need a bunch of updates to other queries.
[12:11:50] <rickogden> what I'm attempting to do is store some data from OSM which include a combination of nodes and buildings
[12:18:28] <rickogden> ooh while I'm here, are there any recommend practices for tracking revision history of a MongoDB document?
[15:01:42] <therealkoopa> Is it possible to use an aggregation filter subdocuments based on position? I'd like to find a document that has an array of subdocuments, but only return subdocuments that are before one of the subdocument's id.
[15:05:56] <therealkoopa> I don't think I need slice. I would like something "All subdocuments that are before the subdocument with an id of 10", but I odn't know if that's possible
[15:06:59] <Derick> what you probably can do, is $unwind, filter out id < 10, and then group again on the $unwinded tag
[15:09:52] <therealkoopa> This seems to work: db.histories.aggregate( { $unwind: '$changesets' }, {$match: {"changesets._id": {"$gte": ObjectId("50a10a77fe69c6dee5000007")} }} , { $project: {changesets: 1}})
[15:41:39] <therealkoopa> I'm able to get my query to work within the mongo command line, but the node mongo driver doesn't return results when I have the match in there: { $match: { 'changesets._id': { $gte: mongoose.Schema.ObjectId("50a110dafe69c6dee500000c") }}}. Any ideas how I can figure out why the match doesn't return anything from within the node mongo driver?
[16:21:11] <alexnixon> hi all. I have two collections, c1 and c2, in the same database which have almost identical documents, except in c1 all documents have a field "x" which is not present in c2. I would like to copy the field "f" from each document in c1 into the corresponding document in c2. I have written the following: https://www.refheap.com/paste/6579 however it's incredibly slow - some updates are taking upwards of 30 seconds. Does anyone have ideas for how I could a
[16:22:42] <alexnixon> I have two collections, c1 and c2, in the same database which have almost identical documents, except in c1 all documents have a field "x" which is not present in c2
[16:22:48] <alexnixon> I would like to copy the field "f" from each document in c1 into the corresponding document in c2
[16:22:52] <alexnixon> I have written the following: https://www.refheap.com/paste/6579 however it's incredibly slow - some updates are taking upwards of 30 seconds
[16:22:57] <alexnixon> Does anyone have ideas for how I could achieve the same effect, but more quickly?
[16:24:49] <kchodorow> db.c1.find().forEach(function(olddoc) { db.entitiessold.update({title:olddoc.title}, {$set:{f:olddoc.f}}); }
[16:25:41] <kchodorow> db.c1.find().forEach(function(olddoc) { db.entitiessold.update({title:olddoc.title}, {$set:{f:olddoc.f}}, false, true); }
[16:26:22] <kchodorow> alexnixon: you're doing an extra findOne and rewriting the whole doc each time (instead of jut adding a field)
[16:26:57] <kchodorow> yeah, it probably still won't be super fast, because it has to grow every doc in your collection
[16:27:47] <kali> i can't see how an update can take 30s. you're sure you have an index on c2.title ?
[16:27:59] <alexnixon> yeah I've read about the 'shuffling' of docs when you grow them, and suspected that's what I'm running in to
[16:28:58] <alexnixon> yes there's definitely an index, and the collection has around 4 million rows
[16:29:54] <alexnixon> it's not all inserts that are taking that long - it's just that when I hammer the console with db.currentOp(), I often see long running inserts
[16:31:59] <alexnixon> is there a general solution to the "collection migration" problem - i.e. you want update a whole bunch of documents before the universe expires?
[16:32:51] <alexnixon> I've hit this problem before and wasn't too sure what to do. Perhaps a mongodump/mongorestore, with some flag telling mongo to leave some extra padding for each doc when importing?
[16:39:36] <kchodorow> alexnixon: you could try compacting the collection first, you can tell compact to leave padding on each document (see paddingFactor: http://docs.mongodb.org/manual/reference/command/compact/)
[16:43:11] <alexnixon> kchodorow: thanks for that - it could well be quicker than waiting for the updates...worth a try at least
[16:43:46] <kchodorow> alexnixon: compact does block everything else from running, so don't do it on a live system
[16:44:44] <alexnixon> kchodorow: cool that's not a problem for me right now as I'm doing this manipulation off-line, and then will dump/restore to the live system when it's finished.
[16:46:52] <neiz> I am updating a collection by using Collection.Update.PushWrapped("Item", item) to push my object in as an array. Once inserted as an array, how do I access individual elements? (10gen c# driver)
[17:16:16] <kali> hays: not really, it has some knowledge of what is in the document, so you can have indexes, server side filtering, selective retrieval, atomic updates
[17:36:13] <doxavore> never mind - i'm an idiot. forgot to add coll.count(:query => my_opts) in my code. hooray mongodb.
[19:08:37] <srp_> Hello, I have this BSON in my database.. http://paste.kde.org/604988/ I want to update this by removing {"50a13bec21966423837a02cd" : 0 } . how do i do this? I'm a newbie and tried this(http://paste.kde.org/605000/) query but i think its wrong! i'd be glad if someone helped me! :)
[19:09:02] <totic> wereHamster: if you want to save space you dont use strings.. if you want to make it human readable you dont use numbers .. the combo is enums
[19:10:16] <totic> weeb1e_: http://mongoengine-odm.readthedocs.org/en/latest/upgrade.html#choice-options
[19:10:26] <totic> wereHamster: http://mongoengine-odm.readthedocs.org/en/latest/upgrade.html#choice-options
[19:11:02] <wereHamster> totic: well, that is specific to your ODM. Next time mention what language/driver/mapper you are using
[19:13:07] <srp_> wereHamster: I tried it.. not getting the query right :( this is what i tried.. db.sim.update({"type" : "note"}, {"similarity" : {"$pull" : [{"50a13bec21966423837a02cd": 1}]}}) but it said "uncaught exception: field names cannot start with $ [$pull]"
[19:16:50] <srp_> wereHamster: ah.. saw it.. i understood that i cannot use $pull here because i cannot know the value in advance. I only know the key in the example i showed! how do i do that?
[19:17:50] <srp_> wereHamster: is there a wildcard in mongodb which matches anything in the place of value.. ?
[19:18:38] <srp_> like.. db.sim.update({"type" : "note"}, {"similarity" : {"$pull" : {"50a13bec21966423837a02cd": WILDCARD_HERE}}})
[19:20:55] <srp_> wereHamster: no problem.. thats fine.. i can store those numbers as strings as i have very less number of these floating numbers .. how can i use wildcard on strings?
[20:03:30] <ee99ee> hi... I'm having trouble with mongo and PHP, getting this simple query to work: http://pastebin.com/E51XQ44d
[20:03:40] <ee99ee> anyone have any ideas? it returns the right data at the command line, but not within PHP
[20:06:15] <Gargoyle> ee99ee: Your array looks OK. But I have never used new MongoCollection(). Do you get the same result if you use $this->mongodb->users->find() ?
[20:16:25] <Derick> ee99ee: instead of: new MongoCollection($this->mongodb, 'users'); you can just do $this->mongodb->users
[20:16:28] <Gargoyle> ee99ee: You are deffo connecting to all the right db's / collections. Not using a test db by mistake (Not that I have ever done that and run myself round in circles for hours!! ;)
[20:17:40] <ee99ee> Derick: yeah, I realize that now... I'm changing my code to that... but I still have the same issue going on
[20:18:36] <wereHamster> gsamat: google.coim, enter 'mongodb index' into the search field, then press the button
[20:26:25] <Gargoyle> ee99ee: Can you pastebin the output from the shell for db.users.findOne() or is it sensitive data?
[20:30:40] <Gargoyle> ee99ee: In your screenshot for Derick you are passing a real session id, in that pastebin its "abc123"
[20:31:11] <Derick> ee99ee: the output of findOne() should *not* be empty if you have data in your database and collection
[20:33:11] <Nicolas_Leonidas> Hi, I want to move the docs from one collection to another, this is how I'm doing it
[20:33:12] <Nicolas_Leonidas> db.smalldaily.find().forEach(function(doc){db.largedaily.save(doc);db.smalldaily.remove(doc);});
[20:33:46] <Nicolas_Leonidas> This changes the "_id" attribute of docs, correct? is there a way to preserve the id that rows have i n smalldaily, when they are moved to largedaily?
[20:35:06] <Gargoyle> I would have thought the _id would be preserved. Mongo only creates one if there isn't one in the doc you are saving.
[20:37:13] <Gargoyle> On code that had been running for about two weeks - just started segfaulting this weekend.
[20:39:24] <Nicolas_Leonidas> is there a place I can turn an ObjectId into human readable values just to see what exactly is the date and stuff?
[20:44:34] <Gargoyle> Derick: client server 1 segfaulted 3 times on the run. I realised this isn't the one I installed valgrind on, so switched to the other box - and the script runs fine!
[21:06:18] <Derick> Easy enough for you to create a one-script-one-request solution that includes the data in the the script?
[21:07:22] <Gargoyle> Errr. Possibly. I can run my script isolated to the one set of records. Might take me a bit longer to dig up the single doc.
[21:08:06] <Gargoyle> But by the logic of those diffs, if I install the debug stuff on the other server, it'll work! :S
[21:16:42] <roxlu> Ah got i: db.images.files.find( { "_id" : ObjectId("50A162D07AA35C1300000271") } ).toArray()
[21:17:07] <Derick> Nicolas_Leonidas: without knowing what your variables are, I've no idea what to say
[21:26:50] <Nicolas_Leonidas> it's empty I know the problem is with the query I'm sending to mongodb
[21:30:13] <Nicolas_Leonidas> ok, I json_encoded them too {"$and":{"date":{"$gt":"2010-01-01","$lte":"2010-01-02"},"lid":6209}}
[21:32:06] <Derick> and no need for the $and (you didn't have that in your first example!) as that is defauly
[21:33:28] <Gargoyle> Derick: I have isolated it to two iterations of a loop and pulled out the code.
[21:33:30] <roxlu> hi guys, when I'm using gridfs, and have a collection: "images.files", how can I update the meta data when I've got an oid ?
[21:33:36] <Nicolas_Leonidas> when I do this in shell => db.largedaily.find({"$and":{"date":{"$gt":"2010-01-01","$lte":"2010-01-02"},"lid":6209}});
[21:33:45] <Nicolas_Leonidas> it says error: { "$err" : "$and expression must be a nonempty array", "code" : 14816 }
[21:34:46] <Gargoyle> Derick: Line 30 is the one segfaulting. But ONLY when the other commands have run. If I remove lines 3 to 29 which is essentially what the previous loop iteration is doing, it does not crash.
[21:37:40] <Gargoyle> the other server hasn't been rebooted since we were last debugging, so USE_ZEND_ALLOC is still exported on the working machine.
[21:37:57] <Derick> Gargoyle: I'm getting tired here thoug,h need some down time - are you around tomorrow morning UTC?
[21:38:08] <Gargoyle> So I should be able to get this to crash on the other machine by setting that var to 1?
[22:04:32] <Nicolas_Leonidas> Derick: I asked my question with more details here: http://stackoverflow.com/questions/13352302/mongodb-search-using-a-date-range-and-another-condition
[22:09:38] <Nicolas_Leonidas> Derick: that actually works, but returns no results let me make sure there is data for that
[22:39:42] <navu> I'm quite new to mongodb and wanted to find out some information. I'm planning to deploy a mongodb project to Azure and would like to take advantage of the replica sets
[22:57:30] <timhaines> Guys - a newb question. When does it make sense to put many collections inside one DB, and when should collections have their own DBs? I read the write lock is now set at DB level. Does this mean it's beneficial for each collection to have their own db?
[23:10:31] <Freso> What's the "definitive guide" book for MongoDB? I see both O'Reilly and Apress having one.
[23:13:56] <jin_> Hey guys, I have a question regarding setting up replicasets. Currently, I'm setting up the mongod instances on three separate VMs using a chef recipe. After it is setup, one of the VM executes a recipe that initializes it by itself, then gathers the other nodes' fqdn to reconfigure the replicaset. The first node has a configuration file setup as it should, but the other nodes are not initializing, although they are being contacted by
[23:40:55] <ckd> timhaines: that's an excellent example of when it's great to have one collection per DB
[23:43:07] <ckd> Freso: Probably not :) but hers is def a great book, but factor in that both books are already a bit out of date
[23:44:37] <Freso> ckd: Yeah. I'm compiling a list of books for my birthday if anyone wants to make a donation for a project I'm working on (which will eventually utilise MongoDB).
[23:45:29] <ckd> timhaines: that's a question for somebody more competent than myself… believe it or not, for what I use Mongo for, I don't use map reduce or aggregation, so i don't' know if either of those are limited in cross-database stuff