PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 22nd of November, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[04:52:29] <ThatTreeOverTher> I'm trying to model friendship in MongoDB but it seems I can't without doubly linking the User model
[04:53:07] <ThatTreeOverTher> Is there a way to do it besides containing a list of friends that are just a reference to the User model and link them twice?
[08:53:52] <omid8bimo> hey guys, how can i get the oplog record date that secondary has picked up from primary upon starting initial sync?
[08:54:26] <omid8bimo> to compare it to primary and see if its already too much behind
[12:45:46] <omid8bimo> yeah
[12:45:50] <omid8bimo> still cloning data
[12:45:57] <omid8bimo> 400GB so far after 3 days
[12:47:02] <omid8bimo> also, is it a good method to sync from a secondary instead of primary? maybe this will speed up the process since nothing happening on secondary
[12:47:43] <joannac> um
[12:48:02] <joannac> whatever writes are happening on your primary also happens on your secondary
[12:49:00] <joannac> the only difference is load, i guess that might make a difference
[12:49:25] <joannac> you'd need to look at your nodes and see if it makes sense for you
[12:52:58] <omid8bimo> well, according to mongostat, there is a huge difference between "query" on primary and secondary
[12:53:24] <omid8bimo> by the way, how can i force the new member to replicate from secondary upon start instead of primary?
[12:54:40] <joannac> http://docs.mongodb.org/manual/tutorial/configure-replica-set-secondary-sync-target/
[14:01:58] <mickeypick> Guys, Can anybody tell me why mongodb doesn't enable authorization by default..? Would I not be needing it in most cases?
[14:09:33] <cheeser> mickeypick: because for dev you don't normally need it and mongodb's experience has been optimized for the ease of the developer
[14:10:29] <mickeypick> oh! and if I do username:password@mymongurl.com.. will it work?
[14:10:38] <mickeypick> cheeser: ^
[14:10:54] <mickeypick> in the standard mongodb drivers
[14:14:33] <cheeser> it should, yeah
[15:43:32] <winem_> hey guys, why does this error "errmsg" : "not authorized on admin to execute command { listDatabases: 1.0 }" appear even when I try to connect with the admin user and password
[15:43:48] <winem_> no difference if I connect to the mongos or one of the shard-members
[15:47:25] <winem_> all I can do is a switch of the database. not even the usersInfo command works. I guess I have a stupid error but I don't know where I did it..
[15:48:57] <winem_> the admin user has this roles: "'readWriteAnyDatabase', 'userAdminAnyDatabase', 'dbAdminAnyDatabase', 'clusterAdmin'"
[16:12:02] <winem_> ok fixed.. was a very stupid mistake
[20:54:23] <drorh> Hello again. Is there a way of running custom JavaScript routines on specificcollection.find() and have mongodb choose which to run based on some value in the document or subdoc?
[20:55:12] <cheeser> no
[20:55:45] <drorh> ok.. thanks
[22:53:01] <m3ng> I have a database table of million rows. I want to do something to each of these rows, while deleting the rows.
[22:53:02] <m3ng> So this is my "operation": Grab the first row from the table. Immediately remove it from the database. Do my stuff with it.
[22:53:02] <m3ng> My question is this: If I run many of these "operations" from different programs, or the same program from different threads, can I somehow guarantee that no two "operations" will accidentally get the same row?
[22:53:21] <m3ng> This is a pretty noob question, I'm sure this is possible. Just not sure what to look for.
[22:57:43] <GothAlice> m3ng: In the end, will the collection be empty?
[22:58:11] <m3ng> GothAlice: yes "in the end" it will, but for all practical purposes, we can assume that million is virtually infinite
[22:58:15] <m3ng> it won't happen any time soon
[22:58:17] <m3ng> assume that
[22:58:41] <GothAlice> Okay, so if you want multiple "workers" running across that data processing it, you could do thusly:
[22:59:16] <GothAlice> Add a field to all of your records ($set) for "processed", defaulting to false. This is needed to prevent two workers processing the same record.
[23:00:44] <GothAlice> Write your worker code so that it find()s records with processed: false and iterates them. Before doing any work on the retrieved record, update it using the "update-if-not-modified" semantic. .update({_id: record._id, processed: false}, {$set: {processed: true}}) — check the number of updated records, if not 1, skip that record since another worker already got it.
[23:01:21] <GothAlice> Are records being continually added?
[23:01:35] <m3ng> GothAlice: nah i made those rows once and for all
[23:01:42] <GothAlice> Perfect.
[23:01:43] <m3ng> should last me a lifetime, so to speak
[23:02:18] <GothAlice> So, you won't reclaim any on-disk space even if you issue .remove() calls. This is due to the way MongoDB allocates space on-disk (like a disk image would).
[23:02:31] <GothAlice> So, once there are no more records to process: just delete the collection. :)
[23:02:37] <m3ng> GothAlice: Are you sure your mechanism is thread-safe?
[23:02:42] <GothAlice> Yup!
[23:02:46] <GothAlice> Guaranteed to be atomic.
[23:02:52] <m3ng> interesting
[23:03:09] <GothAlice> m3ng: Further reading: https://gist.github.com/amcgregor/4207375
[23:06:12] <m3ng> internet troubles today
[23:06:15] <GothAlice> m3ng: Further reading: https://gist.github.com/amcgregor/4207375
[23:06:19] <m3ng> Yeah, thanks GothAlice
[23:06:25] <m3ng> I'll look into this
[23:06:52] <m3ng> I imagine they cover this in the MongoDB docs, or any decent mongodb book?
[23:07:31] <GothAlice> We do this at work quite a bit; even performing this "get record, update record lock, deserialize function, execute function, update record with return value, insert notification" repeatedly, that architecture handled 1.9 million RPC round-trips per second per host at lest benchmark.
[23:07:37] <GothAlice> Aye; it's called atomic locking.
[23:07:45] <m3ng> Cool, thanks
[23:08:14] <GothAlice> http://docs.mongodb.org/manual/tutorial/isolate-sequence-of-operations/
[23:09:28] <m3ng> thanks
[23:21:40] <m3ng> If the rows for a particular column take say 50 MB of data
[23:21:55] <m3ng> 50 bytes for each row, million rows.
[23:22:12] <m3ng> Can we estimate roughly how much disk space it will take to index this column?
[23:22:15] <m3ng> Just roughly
[23:22:26] <m3ng> Will it be another 50 MB, or 100 MB, or 1 GB ?