PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 11th of June, 2012

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:08:19] <Baribal> So how do you save i.e. the data coming in streaming from i.e. a camera? Or should the camera-program accumulate frames and insert them, not using GridFS?
[00:26:41] <lonnylot> does anyone know why you can't use a URL as a bson object id?
[00:27:01] <lonnylot> the mongodb docs say an object id can be anything
[00:45:54] <chretienmn> Hey a simple question: how to do this: db.collection.find({ "field.a" : { $gt: "field.b" } } );
[00:49:03] <chretienmn> oops guess its simply a $where
[00:49:11] <Baribal> chretienmn, off the top of my head, $where, but ... yeah.
[00:54:10] <henrykim> is there any articles to share ways for better index structures under random read/writes environments?
[00:54:29] <chretienmn> Baribal: guess it's the easiest way. was looking for something native without using JS VM
[00:54:37] <henrykim> I mean that it should resolve page-faults problems.
[01:52:48] <henrykim> I need a script to get averages per each minute. How can I do with mongo? columns are | times(second by second) | value |
[04:01:00] <supe> hi
[04:01:00] <supe> What's an efficient way of serialising a large list into a database? - http://stackoverflow.com/q/10973974
[04:11:40] <wereHamster> supe: the more important question you want answered is: is mongodb fast enough when using that data set?
[04:11:53] <wereHamster> also, what about using redis?
[04:24:18] <gigo1980> is there an mailinlist for mongodb ?
[04:25:43] <freezey> hey does it matter if a replica set member has a different version of mongo?
[04:26:56] <wereHamster> gigo1980: http://bit.ly/KeOU4y
[06:22:58] <supe> wereHamster: Redis would be better for this problem?
[06:39:48] <wereHamster> supe: I don't know the problem, so I can't tell
[06:43:14] <ranman> clear
[06:43:18] <ranman> woops sorry
[07:35:24] <[AD]Turbo> hola
[09:19:58] <Killerguy> §j #wmfs
[09:20:19] <Zelest> i refuse!
[09:55:45] <timkuijsten> can mongo read passwords from dot files?
[10:41:25] <suprball> Is there any way to provide a $hint to my $match in the New Aggregation Framework?
[11:44:52] <ro_st> so, i have an array of {id, created, modified} in a document. how do i query the containing document's collection with a value for id in the list i just typed out, such that i get the document back when id is present, and null if not?
[11:45:22] <ro_st> i know that arrays of primitives 'just work'; simply pass the test value in as though the collection were a normal field
[11:45:57] <ro_st> find( { sub-coll.id : id-value } ) ?
[11:47:19] <NodeX> you cant get null if not
[11:47:24] <NodeX> it will just not return
[11:47:53] <ro_st> sorry, that's unimportant. i'm after the correct syntax for the find
[11:47:57] <NodeX> but you can do .... id : { $in : ["foo",null] }
[11:48:35] <ro_st> can i go array-field-name.field-name-of-something-in-the-array ?
[11:48:43] <NodeX> you can do this ....
[11:48:56] <NodeX> find({"foo.id":"bar"});
[11:49:19] <NodeX> where your document looks like this. .... foo : { id :"bar", name:"xyz"} ...
[11:50:29] <ro_st> i have document { foo : 1, bar : [ { id: 1 }, { id: 2 } ] }
[11:50:42] <ro_st> how do i match against the id field in the items present in the bar array?
[11:50:45] <ro_st> bar.id ?
[11:51:00] <NodeX> yeh
[11:51:19] <ro_st> wow. that's powerful
[11:51:32] <NodeX> you can also number it
[11:51:36] <ro_st> i'm actually doing this via monger, the clojure library. just works!
[11:51:41] <NodeX> foo.0.id=
[11:52:06] <ro_st> so a reference to an array will expand out to a test against each element in that array, it appears
[11:52:49] <NodeX> correct
[11:53:06] <NodeX> but .. just rememeber it will return the whole document
[11:53:30] <NodeX> if you need to just return the part of the array that matched in "foo" you have to do that client side
[11:53:40] <ro_st> i actually just want to test whether it returns anything at all. so i'll restrict the returned fields to _id
[12:33:04] <suprball> Is there any way to provide a $hint to my $match in the New Aggregation Framework?
[12:47:30] <souza> Hello everybody!
[12:48:33] <ovaillancourt> hi
[12:49:24] <souza> Guys, i'm having a great problem, using MongoDB and C, we have some code that users ZeroMQ api (in C), and i must to include my mongoC code in this file, but when i try to compile this code, i got a conflit
[12:49:41] <souza> confict between ZeroMQ and MongoDB flags
[13:10:11] <wereHamster> which flags
[13:53:53] <Joshsz> Hey all, quick question, what's the largest single collection you've heard of (in data size)? Some terabytes? petabytes? :)
[13:54:53] <Joshsz> fwiw our biggest is at 500Gb now
[13:56:40] <ro_st> that's a lot of cat pictures
[14:01:21] <NodeX> lol
[14:01:32] <NodeX> craigslist is pretty big
[14:01:39] <NodeX> in the TB's iirc
[14:01:54] <NodeX> and wordnik
[14:07:46] <ro_st> that's a lot of cat pictures in HD
[14:07:56] <ro_st> lemme guess, retina support?
[14:08:34] <NodeX> lolol
[14:08:53] <ranman> anyone @ wwdc?
[14:16:01] <Joshsz> 3tb? nice
[14:16:02] <Joshsz> thanks :)
[14:22:51] <algernon> ro_st: there are a few ascii cats hidden in that 3tb, yes.
[14:22:59] <algernon> I wouldn't call them 'HD', though.
[14:23:35] <ro_st> -grin-
[14:24:35] <SLNP> Can anyone point me in the direction of some reading materials about how the decision process works for deciding whether or not reads are directed to a slave?
[14:25:44] <ar_blues> after 36 hours, achieved the feat of aggregation of collection containing nested documents using inline map reduce, over spring data for mongodb
[14:26:00] <ar_blues> man feels like I have a grown a beard in that time
[14:33:15] <ranman> http://www.youtube.com/watch?v=jyx8iP5tfCI
[14:33:39] <rick446> SNLP: it's up to the driver to decide where to direct reads; probably your best bet is to search for ReadPreference in your driver docs
[14:33:47] <ranman> ar_blues: maybe not a beard but definitely a need to sleep.
[14:37:43] <ninja_p> anyone any experience with how the PHP driver handles directing reads?
[14:37:58] <NodeX> you can force it to read from a specific slave iirc
[14:38:49] <ninja_p> and what about with no preference
[14:38:49] <NodeX> but I coulvd'e dreamt it
[14:38:57] <NodeX> just checking to see if I did lol
[14:39:02] <ninja_p> awesome thanks
[14:39:48] <NodeX> sorry, no you can't
[14:39:58] <NodeX> you can only get information on it... not tell it which one to use...
[14:40:11] <NodeX> $mongo->getSlave();
[14:40:26] <ninja_p> http://php.net/manual/en/mongo.switchslave.php looks related perhaps
[14:40:46] <ninja_p> but that looks to choose a random one (which is fine)
[14:40:54] <NodeX> you can switch but it looks random
[14:41:05] <NodeX> ^^
[14:41:25] <ninja_p> ha, great timing
[14:41:32] <ninja_p> thanks for the heads up
[14:43:14] <NodeX> I suppose there is no reason you can't getHosts and keep switching in your app
[14:43:24] <NodeX> until you find one that's suitible that is
[14:43:36] <NodeX> it's a bit inefficient though!
[14:48:18] <ninja_p> ok, that really helped NodeX
[14:48:26] <ninja_p> after reading a bit more
[14:48:52] <ninja_p> i see that setSlaveOk actually forces all reads to hit any non Primary box
[14:49:10] <ninja_p> determined the the non primary node with the lowest ping time
[14:50:36] <NodeX> kewl
[14:51:49] <carsten> this dumb bot is still there?
[14:53:18] <ninja_p> On an unrelated topic: is there any performance difference with field : { $in : "string" } Vs field : "string"
[14:54:19] <NodeX> in ["string"] ?
[14:54:49] <NodeX> and no, they are both indexed
[14:56:34] <carsten> only if they are indexed
[15:02:47] <ninja_p> yeah
[15:03:03] <ninja_p> i just wasn't sure if there was any difference in terms of how Mongo actually made the comparison
[15:24:27] <waheedi> i have a collection with an embedded array
[15:24:39] <waheedi> i'm doing a lot of update $push on it
[15:24:59] <waheedi> i've indexed the embedded array itself
[15:25:10] <waheedi> the writes are very good, but the update are very slow
[15:25:18] <waheedi> on average 50 ms per update
[15:25:23] <waheedi> any suggestions
[15:27:48] <dgottlieb> waheedi: pure speculation, but you could be growing the document beyond the size allocated for it which would require some more work than a standard insert
[15:29:01] <waheedi> actually not dgottlieb
[15:29:15] <waheedi> the size of the documents is normal
[15:35:23] <carsten> what means "Normal"?
[15:38:59] <waheedi> normal means they are not the cause of slow update $push mr carsten
[15:45:05] <waheedi> this really sucks
[15:59:05] <souza> Hi all
[16:01:56] <souza> guys, i'm using MongoDB and C language, them i was inserting and retrieving data from Mongo, but now the API has changed, in last week, and the official webpage of mongodb and C has a poor documentation about this, and anyone example, someone had used this API with the last modifications?
[16:26:31] <wesc> we've got a secondary that's refusing to catch up with the primary (actually, all secondaries are stuck with the exact same optimeDate). Any thoughts?
[16:30:42] <skot1> Can you post your rs.status() from each of them to pastie/gist/etc?
[16:31:20] <skot1> Also, what does mongostat look like on the primary?
[16:36:38] <wesc> skot1: https://gist.github.com/3ccf30a80b4c82e1d9ec
[16:36:46] <wesc> (yes we know the arbiter situation is unusual)
[16:37:14] <wesc> what you'll see there is a single secondary. We did previously have multiple secondaries, and they were all blocked at the same optime too
[16:37:53] <wesc> we reconstructed the replica set from scratch, using a new replset name, deleted oplog contents, and got the same result
[16:39:39] <skot1> What does rs.config() look like?
[16:39:39] <skot1> The same result being some constant delay?
[16:42:14] <wesc> skot1: rs.config here https://gist.github.com/3ccf30a80b4c82e1d9ec
[16:44:58] <skot1> k, with queued writes like that it will cause replication to stall
[16:45:10] <skot1> It will cause read starvation.
[16:48:17] <wesc> oh interesting, so replication won't happen if there's a large number of insert/update/deletes?
[16:48:58] <skot1> yes, it is a query just like any other and writes take priority
[16:50:19] <ron> so morphia is pretty dead, eh?
[16:58:05] <rhqq> hello. what should i do after i change nssize in .conf file? i've restarted mongo, done db.repairDatabase() and i couldnt create more collections anyway.
[17:25:22] <linsys> rhqq: never done it but the docs say to do exactly what you did.
[17:25:35] <rhqq> and it didnt work for me :D
[17:25:41] <rhqq> thats why im asking :D
[17:29:28] <wesc> skot1: the bulk of our writes are going to the oplog apparently, we're trying to figure out how it could dominate everything else in the instance by 2 orders of magnitude.
[17:29:41] <wesc> skot1: we see a lot of oplog entries of the form found at the start of the gist https://gist.github.com/3ccf30a80b4c82e1d9ec
[17:30:03] <wesc> ie, ns: songza.system.indexes
[17:30:13] <wesc> what does that entry mean?
[17:30:43] <wesc> is something attempting to rebuild indexes over and over again?
[18:58:29] <jkevinburton> hello everyone. Im wanting to use MongoDB with NodeJS because it seems to be the "thing" to use when using Node. I have a collection of users, and then I have a collection of "checkins" each checkin has a user id and a location id. I also have a collection of followers (user 1 can have many user ids) - would you suggest keeping with MongoDB - or rather a relational database. If you say M
[18:58:29] <jkevinburton> ongoDB, what is the proper way about getting the location and the user from the checkin document (in SQL i would do a join..) .. im a n00b, but very interested in the MongoDB way of things
[19:01:34] <mediocretes> if you can predict what data you will need in the most common case, you can denormalize there
[19:01:52] <mediocretes> otherwise, at least to start, you should probably just do the obvious thing and query both
[19:02:55] <jkevinburton> cheers
[19:48:12] <bluesnow> Hi, I'm wondering if advanced queries are available in PyMongo
[19:48:54] <bluesnow> I'm looking for a way to do collection.find({"key": [list of possible values]"})
[19:56:37] <ranman> collection.find({'$in': [ObjectId('...'), ... ]})
[19:56:40] <ranman> bluesnow
[20:02:08] <z0netti> is there a way to use distinct like where()?
[20:04:20] <ranman> z0netti: some of the drivers have that functionality if I'm understanding you correctly
[20:07:24] <z0netti> ranman, my problem is that I do a bunch of verifications to use skip(), sort(), where()... and then populate()... but now I need to distinct the results by a certain field, but I can't call run() after distinct =/
[20:08:13] <ranman> yikes that's tricky… let me see if I can figure it out.
[20:10:54] <z0netti> ranman, I found this example: "Category.all(:id => Deal.distinct(:category_id))"
[20:10:56] <z0netti> I'll try
[20:11:15] <ranman> ruby driver?
[20:11:28] <z0netti> ranman, no.. mongoojs here, for node
[20:11:32] <z0netti> mongoose*
[20:14:28] <z0netti> fail =/
[20:15:28] <aheckmann> z0netti: distinct must be the last method call in the chain, it executes the query model.where().distinct('field', callback)
[20:19:03] <z0netti> aheckmann, but when I do that, I got only the objectids... and not the whole docs that I want =/
[20:19:13] <z0netti> I'll try again in a different way
[20:20:59] <aheckmann> z0netti: distinct doesn't return the docs: http://www.mongodb.org/display/DOCS/Aggregation#Aggregation-Distinct
[20:22:12] <z0netti> aheckmann, ya... apparently I can't do that.. =/
[20:22:26] <z0netti> but thanks anyway ranman and aheckmann
[20:22:38] <aheckmann> :)
[20:55:21] <hadees> anyone have any suggestions for a good general php orm for mongo? I know there is a list of them on the website but i've found other ones and i'm just wondering if there are more.
[21:13:38] <edvorkin> I am getting ReferenceError when querying my collection named activity-events
[21:14:17] <edvorkin> PRIMARY> db.activity-events.find()
[21:14:19] <edvorkin> Mon Jun 11 21:09:19 ReferenceError: events is not defined (shell):1
[21:14:20] <edvorkin> Is events a reserved word in mongoDB? I am sure the collection exists
[21:16:09] <mediocretes> if you do db['activity-events'] does it work?
[21:18:34] <edvorkin> yes. That works
[21:18:35] <edvorkin> PRIMARY> db['activity-events']
[21:18:37] <edvorkin> cloud-cp.activity-events
[21:19:32] <mediocretes> so it thinks that you want it to subtract events from activity
[21:20:50] <edvorkin> I see. Can I rename sharded collection. No data exists yet
[21:21:28] <mediocretes> should be pretty easy to just drop it and recreate it then, eh?
[21:26:06] <edvorkin> yes. thanks