PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 9th of January, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[01:30:24] <bobbobbins> hey, I'm trying to do a read from a mongo db but I'm getting an error about it not being the master...and i see that I'm supposed to run slaveOk to fix it, but I'd it to affect all connections...is there a way to do that?
[05:16:28] <Alex474> Hi! Looking for help with $addToSet - works not as expected: http://stackoverflow.com/questions/21011773/addtoset-works-like-push
[06:33:40] <Ravenslock> Hi. There are 3 mongo extensions to choose from mongoyii, directmongosuite, yiimongodbsuite. Which one do I use?
[07:02:22] <Ravenslock> Hi. There are 3 mongo extensions to choose from mongoyii, directmongosuite, yiimongodbsuite. Which one do I use?
[07:16:26] <Ravenslock> There are 3 mongo extensions to choose from mongoyii, directmongosuite, yiimongodbsuite. Which one do I use?
[07:50:06] <axprv> hi, is anyone in here willing to assist me in designing a basic schema in exchange for $50?
[08:07:16] <belak> What's the best way to dump a mongo database so it can be copied to another server?
[08:14:37] <kali> belak: mongodump. but you can also copy the db files if you can stop the source server
[08:14:45] <kali> belak: or do a filesystem snapshot
[08:15:31] <belak> Thanks
[08:15:38] <belak> mongodump looks perfect
[08:47:40] <ns5> g.mongo_db.thread.find({'source_ids':source_id}, {'_id':1}) is this equivalent to g.mongo_db.thread.find({'source_ids':source_id})?
[09:54:53] <ns5> threads = db.thread.find({'source_ids':source_id}, {'_id':1}).skip(count).limit(10), this always find all the threads in db, it seems that skip() and limit() are not working, any idea?
[09:56:00] <ron> that's when you run the query how?
[09:56:54] <ns5> ron: ?
[09:57:14] <ron> ns5: what's not clear about my question?
[09:57:15] <ns5> threads.count() is always 15
[09:58:19] <ns5> ron: I have db.find().skip().limit() in a while loop, but the loop goes forever because it always finds all the threads
[09:58:55] <ron> you run it in a shell? using python? php? nodejs? java? c#? ruby?
[09:59:28] <ns5> ron: it's in python, I'm using pymongo
[09:59:48] <ron> okay, and when you try running the same query from the shell, what do you get?
[10:02:50] <ns5> ron: > db.thread.find({'source_ids':ObjectId("52cd13cae8bfec03b2ceb385")}).skip(0).limit(10).count()
[10:02:50] <ns5> 15
[10:03:23] <ron> which version of mongo?
[10:04:22] <ns5> ron: mongodb 2.4.8-1
[10:04:38] <ron> odd. :)
[10:04:50] <ron> what if you remove the skip?
[10:05:31] <ns5> ron: the result is still 15
[10:05:46] <ron> what if you remove the limit?
[10:06:42] <ns5> ron: it's still 15 no matter how many I skip
[10:07:02] <ron> but if you don't use skip and limit, what do you get? 15?
[10:08:27] <ns5> ron: yes, that's the total number of threads
[10:10:44] <ns5> ron: I updated mongodb to 2.4.8-2, still facing the same issue
[10:11:44] <ron> ns5: http://docs.mongodb.org/manual/reference/method/cursor.count/#cursor.count read :)
[10:11:52] <ron> check the parameter.
[10:13:41] <ns5> ron: thanks for the hint, cursor.size() works well, thanks
[10:15:08] <ron> (y)
[10:26:26] <ns5> ron: it seems that pymongo does not support cursor.size()
[10:27:02] <ron> that I can't help with. but maybe it supports .count() taking the extra parameter?
[10:27:58] <ns5> ron: yes cursor.count(True) works
[10:28:06] <ron> there you go.
[11:23:29] <kas84> hi
[11:25:44] <kas84> if I store multiple kinds of elements in a collection multiple times (with a date field)
[11:26:09] <kas84> how can I retrieve only the latest objects of every different kind of element?
[11:28:58] <Derick> example?
[11:33:59] <kas84> Derick http://hastebin.com/nefesivoni.md
[11:34:46] <kas84> I want to retrieve all the objects with different id but only the latest (based on date field which is a unix timestamp)
[11:36:10] <kas84> btw the ones with id 2_1_2 are a copy of 2_1_1 so don't take into account ObjectId
[11:38:57] <Derick> kas84: ah
[11:39:03] <Derick> you need some aggregation!
[11:41:08] <Derick> trying to come up with how
[11:43:49] <kas84> hehe
[11:44:03] <kas84> I am a newbie in mongo :/
[11:45:06] <ppetermann> i've yet to find a problem where i need the aggregation framework, so far map/reduces allways seemed the better solution, but that might just be usecase related
[11:45:26] <Derick> ppetermann: M/R is always slower and can't use indexes. Be warned.
[11:45:41] <Derick> ppetermann: your goal should always be to use A/F first if you can
[11:45:42] <ppetermann> Derick: m/r i can do incrementally in the background
[11:47:24] <kas84> Derick: any ideas?
[11:47:43] <Derick> kas84: still on it
[11:50:41] <Derick> kas84: db.test.aggregate( [ { $sort: { date: -1 } }, { $project: { _id: 1, id: 1, sub: { type: '$type', value: '$value', level: '$level', status: '$status', _id: '$_id' } } }, { $group: { _id: '$id', latest: { $first: '$sub' } } } ] );
[11:52:55] <kas84> awesome Derick!!
[11:53:00] <kas84> thank you very much!!!
[11:53:19] <Derick> kas84: try to understand what it does though
[11:54:32] <kas84> yep, that's what I'm trying to do now :D
[13:03:16] <jalkstal> My mongorc.js file seems to be cached or something. If I remove it it will still be loaded and if I change its contents the changes are not seen.
[13:03:30] <qoottaa> hello all!
[13:03:34] <jalkstal> Any help on how I can remove the mongorc settings?
[13:03:56] <qoottaa> where i can find advanced server conf?
[13:07:08] <jalkstal> How can I "reset" the shell so that my mongorc.js file configurations are not loaded?
[15:36:54] <MatToufoutu> Hi there
[15:37:53] <Derick> hi
[15:38:07] <MatToufoutu> I am having an issue with my mongo server, it is running and listening on the right port (netstat confirms), but I'm still unable to connect to it (be it using a library or the mongo client), getting a connection timeout
[15:40:01] <MatToufoutu> and I get no error in the logs
[15:41:31] <MatToufoutu> I remember having had a similar issue on another machine in the past, but can't remember how I solved it, so any help would be greatly appreciated :)
[15:43:44] <Joeskyyy> Are you trying to connect to it remotely? Or on the same machine the mongod is running on?
[15:43:57] <MatToufoutu> on the same machine
[15:44:26] <Derick> selinux?
[15:44:26] <Joeskyyy> And what command are you using to connect to it?
[15:45:37] <MatToufoutu> simply "mongo"
[15:46:20] <Joeskyyy> So your mongod is infact running on port 27017?
[15:46:30] <MatToufoutu> yep
[15:46:39] <Derick> can you telnet to port 27017?
[15:46:44] <Joeskyyy> +1 ^
[15:47:14] <MatToufoutu> times out too
[15:47:45] <Derick> telnet localhost 27017 ?
[15:47:48] <MatToufoutu> yes
[15:47:55] <Derick> can you paste the output of : netstat -a -n ?
[15:47:58] <Derick> pastebin*
[15:48:26] <Joeskyyy> dat bind addr
[15:48:44] <MatToufoutu> err, ok, just let me a few minutes to do it and get the output on that machine (different from the one mongo is running on)
[15:49:41] <Derick> uh
[15:49:50] <Joeskyyy> wat
[15:49:52] <Derick> mongo is running on a different machine than "mongod" ?
[15:49:59] <MatToufoutu> no, my irc client
[15:50:10] <Derick> oh :D
[15:50:11] <Joeskyyy> Ah
[15:50:19] <Derick> MatToufoutu: use pastebin.com
[15:53:08] <MatToufoutu> https://gist.github.com/mattoufoutu/165913fbfb86825da957
[15:54:04] <MatToufoutu> Derick: no offense here, just used to use gist ^^
[15:54:19] <Derick> that's ok too
[15:54:23] <Derick> just don't paste it in here :P
[15:54:34] <MatToufoutu> "too late"? :)
[15:54:38] <Derick> heh
[15:54:39] <Derick> ok
[15:54:40] <MatToufoutu> ah, the output
[15:54:41] <Derick> one more:
[15:54:49] <MatToufoutu> no, I would have never pasted all these lines here :)
[15:54:50] <Derick> strace -o /tmp/strace.log mongo
[15:54:58] <Derick> MatToufoutu: other people have done
[15:55:01] <MatToufoutu> k, 1sec
[15:56:45] <MatToufoutu> ok, just added the strace log to the previously linked gist
[15:58:17] <Derick> MatToufoutu: it is threading
[15:58:21] <MatToufoutu> (if that helps, the mongo server is running on CentOS (not my choice ^^))
[15:58:45] <MatToufoutu> (or maybe RHEL, I never recall which one)
[15:58:47] <Derick> you need to ad -f before the -o in the strace command
[15:59:07] <MatToufoutu> ok, I redo it with -f so
[16:00:02] <Joeskyyy> Might wanna go back to the SELinux suggestion from earlier if it's Cent/RHEL
[16:02:10] <Derick> yup
[16:02:14] <Derick> selinux is the devil
[16:02:35] <MatToufoutu> gist updated
[16:03:11] <Derick> connect(3, {sa_family=AF_INET, sin_port=htons(27017), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNRESET (Connection reset by peer)
[16:03:15] <Derick> something kills the connection
[16:03:27] <Derick> my hunch is still selinux
[16:03:30] <MatToufoutu> hmmm
[16:03:30] <Derick> can you turn it off?
[16:03:48] <Derick> echo 0 >/selinux/enforce
[16:04:39] <MatToufoutu> is there a service to restart for this to be effective?
[16:04:46] <Derick> not to try it with
[16:05:00] <Derick> http://www.crypt.gen.nz/selinux/disable_selinux.html
[16:05:03] <Derick> has the full docs
[16:05:11] <Derick> echo 0 >/selinux/enforce should be enough as a test
[16:06:22] <MatToufoutu> still timesout after that
[16:06:29] <Derick> hm
[16:06:32] <Derick> firewalls?
[16:06:37] <Derick> iptables -v -L
[16:06:44] <MatToufoutu> iptables alllows loopback connections
[16:07:02] <Derick> you can firewall that off too
[16:07:17] <MatToufoutu> no, I can't turn off iptables on that machine
[16:08:26] <MatToufoutu> hmmm, would someone have messed with my iptables config....
[16:08:55] <Joeskyyy> It'd be a bit silly to add loopback rules to iptables lol
[16:09:21] <Derick> MatToufoutu: were you root when turning off selinux?
[16:09:28] <MatToufoutu> Derick: yep
[16:14:11] <MatToufoutu> ok, I'm gonna find who I must kill now, someone did something with my iptables config....
[16:14:16] <MatToufoutu> sorry for the time waste guys
[16:14:37] <MatToufoutu> (and thanks!)
[16:15:10] <Joeskyyy> :D
[16:18:40] <Derick> MatToufoutu: :-) glad it works now
[17:25:08] <Kaim> hi all
[17:25:17] <Kaim> I just add a new shard in my shard cluster
[17:25:17] <Joeskyyy> hello
[17:25:30] <Kaim> how can I make chunk migration fast to the new shard?
[17:29:08] <Joeskyyy> Er…. I mean you can't really force it to move faster? The balancer is going to do its job in the most efficient manner it can
[17:39:05] <Kaim> Joeskyyy, yes but it take hours ^^
[18:57:09] <ghostbar> So, If I have an embedded doc, and want to update one of them based on the id. How do I do? db.collection.update({"_id": ObjectId}, {embeddoc: .. ??
[19:00:22] <locojay> hi my disk has a 95% io utilization but mem only 6% utilication. how can i dig why mongo is using that much io
[19:02:24] <ron> ghostbar: http://docs.mongodb.org/manual/reference/method/db.collection.update/#update-specific-fields-in-subdocuments
[19:03:46] <jergason> does turning on slow query logging affect performance?
[19:04:04] <ghostbar> db.user.update({"_id": ObjectId("52ceed2bb76111e596e1d143"), "random._id": "hola"}, {$set: {"random.$": {_id: "hola", "a": "y", "b": "z"} }})
[19:04:07] <ghostbar> ron: nice! That's easier than the one I just discovered. Thanks! Was using
[19:04:44] <ron> ghostbar: well, reading is fundamental. ;)
[19:04:52] <ghostbar> ron: definitely
[19:05:08] <jergason> also is there a way to monitor when new things get inserted in to the slow query log? I would love to create a dashboard of some kind to let me know how often we are running slow queries
[19:05:16] <cheeser> unless you work with perl. then it's just write write write
[19:05:35] <cheeser> jergason: mms can show you your slow queries
[19:05:43] <jergason> wellllllllllll now
[19:05:45] <jergason> ill check that out
[19:05:58] <ghostbar> ron: actually I will need the field.$ because is an array of embedded documents
[19:06:11] <ron> ghostbar: the answer still lies there.
[19:10:48] <jergason> is there support for exporting data from mms?
[19:13:27] <saml> wht's mms?
[19:13:31] <saml> mms:// ?
[19:13:38] <cheeser> you want export the performance metric numbers?
[19:14:41] <cheeser> saml: https://www.mongodb.com/products/mongodb-management-service
[19:14:58] <saml> oh thanks
[19:50:52] <jergason> saml: it is the mongodb monitoring service
[19:51:01] <jergason> basically dashboard/metrics for your hosts
[19:51:05] <jergason> err databases
[19:51:10] <jergason> oh doh
[19:51:12] <jergason> beated to the punch
[19:51:48] <jergason> cheeser: i'd love to have a graph of incidence of slow queries somehow
[19:52:04] <jergason> that is the end goal, so i can see if they jump up suddenly on a graph instead of digging through table rows
[20:07:50] <NaN> hi there
[20:08:19] <NaN> should I need to start the mongod with --rest to use the simple rest interface, or where do I need to put that --rest line?
[20:08:37] <cheeser> how do you start mongod?
[20:10:30] <NaN> cheeser: chkconfig mongod on
[20:10:50] <NaN> (after sudo service mongod start)
[20:11:07] <NaN> I mean before
[20:11:25] <cheeser> ubuntu?
[20:11:30] <NaN> fedora
[20:14:23] <cheeser> put rest=true in /etc/mongodb.conf
[20:14:34] <cheeser> http://docs.mongodb.org/manual/reference/configuration-options/#rest
[20:16:07] <NaN> cheeser: done, now when I try to restart the service it says FAILED but if I go to the URL the server seems running
[20:16:25] <NaN> is it possible to have another mongod instances runing? how can I check it?
[20:17:11] <cheeser> try to stop your service. then run mongo.
[20:17:19] <cheeser> if you can connect still, you have another process running.
[20:18:32] <NaN> cheeser: yes I have another process running =/
[20:39:31] <locojay> any easy way to list unused indexes
[22:10:20] <spec__> hey all. I'm a research scientist at NVIDIA where I work on parallel algorithms. I'm getting interested in data mining or analytics as a research direction. Does anyone want to discuss the concerns that are at the frontier of data science right now, so we have some better idea what to investigate?
[22:11:56] <spec__> I'm looking at frequent itemset mining and classification algorithms specifically, but would like some input from NoSQL users