PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 19th of September, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[11:37:37] <duncanhp> any mongo core dev’s here
[11:37:47] <duncanhp> we’ve just lost production for an issue i created a couple days ago
[11:37:47] <duncanhp> https://jira.mongodb.org/browse/SERVER-15265
[11:37:58] <duncanhp> would really appreciate some help in getting to the bottom of it
[11:38:30] <SmitySmiter> hello guys, is it possible to query a collection which would return documents, for distinct values of a particular field?
[11:39:28] <SmitySmiter> hello?
[11:41:17] <tscanausa> SmitySmiter: do you have an example?
[11:41:38] <SmitySmiter> I'll prepare one and post on bpaste
[11:41:40] <SmitySmiter> give me a min please
[11:45:17] <SmitySmiter> tscanausa, here's a sample https://bpaste.net/show/987373e17b6b
[11:49:41] <tscanausa> Simply put no your desire is not possible. distincts generally only return the distinct values of that field.
[11:49:56] <SmitySmiter> yes
[11:50:04] <SmitySmiter> is there any way I can return the whole document?
[11:50:13] <tscanausa> again no
[11:50:25] <SmitySmiter> hmm, so I'll have to query the whole collection and prepare the result from that?
[11:50:37] <SmitySmiter> that's going to be a heavy toll, right?
[11:51:09] <tscanausa> yes. if you explained your use case a better solution might arise.
[11:51:16] <SmitySmiter> hmm
[11:53:26] <SmitySmiter> all right, I'll give you the exact use-case
[12:13:04] <Ephexeve> hey guys, mongodb vs pymondo. Which?
[12:13:10] <Ephexeve> *pymongo
[12:18:13] <tscanausa> Ephexeve: it depends. I have no direct experience with either but some times the mongodb ones are not so great. I would recommend trying them out.
[12:26:51] <jordana> Thoughts on the need for RAID1 on shards if they're setup as 3+ replic sets?
[12:27:26] <tscanausa> jordana: I personally dont think it is needed.
[12:27:33] <jordana> tscanausa: that's what I'm thinking.
[12:27:44] <jordana> tscanausa, by the way, the sync finished successfully this morning :)
[12:28:10] <tscanausa> jordana: hah awesome, what was wrong?
[12:28:17] <tscanausa> are you cloud or dedicated?
[12:28:26] <jordana> dedicated, hosted in house.
[12:29:00] <jordana> I think it might have been something to do with having to readd it rs.remove rs.add
[12:29:13] <jordana> after doing that, seemed to work fine.
[12:29:39] <tscanausa> interesting
[12:30:05] <jordana> I completely wiped the system, went from CentOS 6 to 7
[12:30:23] <jordana> not 100% sure if that had some factors
[12:30:49] <jordana> now I have another 3 shards to go through :P
[12:30:54] <tscanausa> interesting. if it is in house you are probably good. if they are not all living together
[12:31:09] <tscanausa> jordana: only do one node in each shard at a time
[12:31:21] <jordana> tscanausa, yeah, I am.
[12:31:38] <jordana> they are living together but we're going to stream the oplog over to a secondary data center
[12:32:00] <jordana> for DR
[12:32:16] <tscanausa> what i ment is not in a vm on the same host
[12:32:27] <jordana> Ahh no, they're all bare metal.
[12:32:30] <jordana> each shard
[12:32:58] <tscanausa> what about each member in the replica set?
[12:33:04] <jordana> Each member
[12:33:10] <jordana> each mongod is bare metal sorry
[12:33:18] <jordana> the configs are VMs
[12:33:21] <jordana> and the routers
[12:33:47] <jordana> so i'm thinking getting rid of RAID1 on them
[12:34:37] <tscanausa> awesome ok, I would take the risk. if they were vms then i would hope the host is raid 1
[12:34:58] <jordana> the VM hosts have a combination of RAID1 and RAID5 on different volumes.
[12:36:31] <tscanausa> I would go raid 0 or jbod then
[12:37:06] <jordana> JBOD is fater
[12:37:08] <jordana> faster*
[12:37:10] <jordana> :)
[12:37:33] <tscanausa> I would actually to lvm, but that is a linux thing
[12:38:42] <jordana> LVM isn't as fast
[12:39:02] <tscanausa> adminstatively it is nicer
[12:39:17] <jordana> I agree
[12:51:07] <Strekerud> Hi! Is there anyone with experience with MongoSessionManager who has some spare time to help me out?
[12:54:10] <Strekerud> I'm trying to persist web forms using MongoSessionManager. Persistence of the same page works with JDBCSessionManager, but it's way too slow. The sessions stored in the mongodb doesn't seem to update at all as I click through the form, nor does it persist during failover.
[12:59:47] <Tajt> if I run db.dropdatabase on a mongodb and then recreate it from a previous dump, giving it the same name, is there any risk of old data laying around still or is everything guaranteed to be removed even if I do not run any repair command?
[13:00:53] <tscanausa> Tajt: technically you should be good but there is a race condition if you do it too fast
[13:01:32] <Tajt> ok, thanks
[13:37:44] <Strekerud> Anyone with any experience using MongoSessionManager?
[14:00:04] <Diplomat> Hey guys! Any ideas why I can't use "FindOne" method here's what I have http://pastebin.com/pmu3Ac2y (error and code that creates it)
[14:00:46] <Diplomat> and this "using" list is what I'm using
[14:01:29] <Diplomat> I'm following this info here: http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-csharp-driver/
[14:17:25] <blarghmatey> Does anyone happen to know what the output of rs.status or rs.config are while a replica set is still initializing?
[14:17:46] <blarghmatey> I need to know so that I can add some flow control into a script to prevent performing writes during initialization.
[16:02:06] <postitnote> Anyone here using macports mongodb on osx mavericks?
[16:02:59] <luca3m> why not homebrew?
[16:04:25] <postitnote> luca3m: I've been using macports since forever, I guess.
[16:05:09] <luca3m> ok, I suggest you migrating to homebrew, it is on another level
[16:05:16] <postitnote> If I'm not using macports I'm usually using Vagrant.
[16:05:45] <postitnote> For me, switching to homebrew will kill all of today heh.
[16:06:13] <postitnote> If mongodb admin was all I did for a living, I'd do it. But, this is just for a silly node app.
[16:15:17] <ehershey_> hehe
[16:15:23] <ehershey_> homebrew is on another level
[16:15:40] <ehershey> well said
[16:34:45] <postitnote> Do you guys know what to make of this? ERROR: child process failed, exited with error number 1?
[16:42:10] <postitnote> Ugh, this is killing me. Damn you macports.
[16:44:29] <freeone3000> postitnote: POSIX error code 1 is EPERM, "Permission Denied".
[16:44:56] <postitnote> can I just chmod everything 777? We're all adults here.
[16:45:47] <postitnote> What would one enable permissions on to fix this problem?
[16:56:44] <ianblenke> I'm having a difficult time re-establishing a replication set on a new mongo cluster that is a restore from a production mongo cluster
[16:58:37] <ianblenke> I've removed the replSet argument, done a "use local" and "db.dropDatabase()" while in stand-alone mode, "use admin" followed by "db.shutdownServer()" then restarted mongo with the replSet argument and did an rs.initiate() with the json arguments of that node.
[16:59:23] <ianblenke> it looks like a healthy master. but when I try to add an empty mongo server running with the same replSet, it just keeps saying "replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG)" and the master doesn't seem able to do an rs.add() on it.
[17:12:20] <freeone3000> ianblenke: Does it have the same replSet in its config? Do you use authentication?
[17:21:50] <yopp> hey
[17:22:04] <yopp> what the simpliest way to convers bson to mongo shell compatible format?
[17:22:07] <yopp> •convert
[17:28:36] <ianblenke> figured out my problem. AWS security groups. port was blocked. doh.
[17:36:25] <tscanausa> yopp why do you need that?
[17:36:41] <yopp> tscanausa, for better logging
[17:37:02] <yopp> current moped (ruby driver) logging is shitty
[17:37:08] <yopp> database=fire_development collection=users selector={"$query"=>{}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.6960ms
[17:37:24] <yopp> There no way to use it in shell
[17:37:49] <tscanausa> why would you use it in the shell? can you explain your use case?
[17:38:00] <yopp> tscanausa, uh?
[17:39:04] <yopp> it's as simple as it's sounds: I need to see queries in the same way as they are described in docs.
[17:39:12] <tscanausa> why are you trying to do what you are doing?
[17:39:47] <yopp> And in docs they are described in "shell commands" format
[17:40:05] <yopp> So my need is to simple copy & paste queries from logs to shell.
[17:40:38] <yopp> For example, in ActiveRecord with *SQL backed, I can copy query and then execute it in db shell
[17:40:52] <yopp> There billions of use cases when you need this.
[17:41:24] <yopp> My only probles is that in json representation of bson there few "hacks"
[17:41:45] <yopp> For few types: objectid, time and so on
[17:41:55] <tscanausa> do you have an example?
[17:42:01] <yopp> Of what?
[17:42:25] <tscanausa> one of the log lines that is not compatable.
[17:42:37] <yopp> 34:00] <yopp> database=fire_development collection=users selector={"$query"=>{}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.6960ms
[17:42:53] <yopp> (it's simple one)
[17:43:07] <tscanausa> ya that is not even json
[17:43:11] <tscanausa> or bson
[17:43:17] <yopp> yep. Now I have json
[17:43:33] <tscanausa> how did you get the json?
[17:43:35] <yopp> I've patched driver and now I'm getting this in logs:
[17:43:36] <yopp> db.users.find({}).sort({"_id":1}).limit(-1).skip(0)
[17:43:39] <yopp> oops
[17:44:21] <tscanausa> ya, I dont think you are going to be able to support this feature by pathcing a driver
[17:49:42] <yopp> battery died
[17:50:05] <yopp> I'm dumping query to json and now I'm getting: db.users.find({"_id":{"$oid":"5412e874d458dad8da2284c2"}}).limit(0).skip(0)
[17:53:14] <yopp> and seem like mongo shell is not happy about "$oid"
[18:03:38] <Lingo___> yopp: not sure if this is what you need, but pymongo installs a bson module that can convert bson to json
[18:26:51] <loincloth> heyo
[19:20:18] <yopp> they all have same issue: it's JSON, not JavaScript :)
[19:20:31] <yopp> okay then, I'll dump em by myself
[19:52:22] <jblancett> can anybody link me to a good guide on creating a single read replica of my mongodb server?
[19:54:44] <decompiled> hello
[20:06:09] <dgarstang> I need some help with mongo credentials. Is there a better doc than this? http://docs.mongodb.org/manual/tutorial/enable-authentication/ ?
[20:06:30] <dgarstang> because, if you follow that, and then authenticate as that user, you cant' even to a sh.status()
[20:07:11] <freeone3000> dgarstang: userAdminAnyDatabase?
[20:07:24] <dgarstang> freeone3000: yes...
[20:07:43] <dgarstang> Specifically a sh.status() returns "not authorized for query on config.version"
[20:08:12] <dgarstang> Those docs are pretty ... slim. So, the admin user isn't really an admin user as defined by that doc?
[20:09:34] <dgarstang> and the guys that are trying to use this are saying "de (unable to review cluster state, insert something, review config db, etc.).." ... no auth for those. So, the admin user, as defined by that doc, apparently can't do any db operations
[20:13:07] <BlueDo> Is anyone able to get Mongo working with XAMPP?
[20:15:14] <dgarstang> omg mongo. why do I have to write a damn json blob to run a command?
[20:16:59] <tjbiddle> Hey all - I'm trying to configure a hidden replica set member; however I'm not overly experienced in mongo. I'm looking at http://docs.mongodb.org/manual/tutorial/configure-a-hidden-replica-set-member/ and it doesn't explain what file I need to be editing
[20:28:35] <Almindor> hey, does mongodb have a bug between mongoexport and mongoimport when date is ISODate("1970-01-01T00:00:00Z") ?
[20:28:52] <Almindor> it's giving me error BSON representation of supplied JSON is too large: code BadValue: BadValue: Year out of range: 1969
[20:29:12] <Almindor> note: same mongodb version (2.6.1) with the data being just piped from mongoexport to mongoimport
[20:29:37] <Almindor> also if I put that isodate in a variable and do .getYear() it gives me 1969, which I guess is what's the problem here (local vs iso?)
[20:36:27] <decompiled> Has anyone seen a lot of splitvector queries that take a long time (30+ mins) on a sharded cluster?
[20:39:38] <cheeser> yes. let me try to remember the reasoning...
[20:42:03] <cheeser> hrm. can't find it.
[20:46:49] <BlueDo> part
[20:55:12] <kamoricks> I would like to add a member to a replica set that will never become primary, doesn't vote, and is only hit by queries when specifically mentioned. What is the word for this?
[20:55:21] <loincloth> hey all... is anyone here familiar with the Mongoid library for Ruby?
[20:56:26] <loincloth> or the official MongoDB Ruby driver?
[21:02:07] <kali> loincloth: just ask
[21:03:22] <loincloth> kali: sure, i was just treading lightly because i wasn't sure it would be on topic
[21:03:35] <loincloth> i am unclear about how some caching is working out
[21:03:52] <loincloth> in a test environment, i wanted to track when embedded documents were created
[21:04:15] <loincloth> so i had something like: criteria = user.rankings.some_scope
[21:04:31] <loincloth> and then i would do assertions against things like criteria.count and criteria.last's attributes
[21:04:51] <loincloth> this worked up until i expected those value to change and i struggled for a while until i eventually just repeated the expression i used to assign criteria to begin with
[21:04:59] <loincloth> after the point it should have changed
[21:05:15] <loincloth> and then i was getting expected results, whereas the criteria variable object was still returning cached results
[21:06:31] <loincloth> i've started looking into it and it's not clear right away who is responsible for this(mongoid? moped? mongodb ruby driver? mongodb itself?) so i was hoping to take a shortcut via some helpful humans
[21:07:19] <loincloth> even a tip about who is responsible would be helpful and i get look into the details
[21:15:11] <kali> loincloth: i think this is in mongoid scope
[21:16:19] <kali> loincloth: mongodb ruby driver does not do anything like that, but as far as i know, mongoid no longer uses it
[21:16:29] <kali> loincloth: depending on your mongoi version of course
[21:18:16] <kali> loincloth: unfortunately, mongoid is huge and complex contraption.
[21:19:43] <kali> loincloth: http://rdoc.info/github/mongoid/mongoid/Mongoid/QueryCache maybe this can be a relevant starting point
[21:25:02] <loincloth> kali: my hunch is you are right about the scope, but i'm not clear on some of the rest there... like http://mongoid.org/en/mongoid/docs/extras.html#caching suggests the official ruby driver is still in play
[21:25:08] <loincloth> but the docs could of course be out of date
[21:27:32] <kali> mmmmm
[21:27:35] <loincloth> kali: is that for Mongoid 4.0.0? i am on 3.x
[21:28:00] <kali> well, if you're on 3.x, i guess you should refer to the /v3/ docs pages
[21:28:14] <kali> but... they say the same thing about the driver :)
[21:28:32] <loincloth> yeah i tried to make sure i was on the right docs
[21:28:45] <loincloth> let me actually make sure i'm on 3.x too just in case...
[21:28:58] <loincloth> yeah 3.0.23
[21:30:02] <loincloth> i might be able to go to 4... i didn't really realize 4 even existed... i installed mongoid without a version requirement to start and then locked it to the one i got which was 3.0.23... at least i think that's what happened now i dunno anything anymore :P
[21:30:22] <kali> loincloth: honestly, don't rush into this :)
[21:30:47] <loincloth> oh i'm not gonna was just thinking that i have no hard requirement to be on 3.x that i'm aware of
[21:31:01] <loincloth> but i realize my rambling is complicating the train of thought here, sorry :P
[21:32:47] <kali> loincloth: check what cache? do on your criteria maybe
[21:33:01] <kali> or cached? actually
[21:35:30] <loincloth> kali: heh... well... what i am seeing is criteria.cached? returns false always and differs from the value of repeating the expression which created criteria
[21:36:33] <kali> great :/
[21:36:55] <loincloth> kali: yeah... fwiw glancing at the source for 3.0.23 does seem to suggest the official driver is no longer in play which seems noteworthy
[21:37:15] <loincloth> i was vaguely aware of that, i would say, and just read that extras section and assumed it knew what it was talking about... oops
[21:37:24] <loincloth> but i had seen various references to Moped already of course
[21:37:25] <kali> loincloth: yeah, i was pretty sure of this because this was the reason why we stayed on 2.x
[21:37:48] <loincloth> mmkay... guess i'll dig around with Moped for a little while
[21:38:39] <kali> yeah. if you want to convince you that the server side is not behaving badly (i'm convinced here) you can run a mongosniff or set the profiler at level 2
[21:39:36] <kali> maybe the log can tell you that too... but the log will be generated by moped too
[21:47:00] <loincloth> kali: thx a lot for the help and effort
[21:57:34] <loincloth> kali: btw, do you have a preference for alternative to mongoid?
[21:58:20] <loincloth> i was following lead of a team member who i don't think has a ton of experience to the point of necessarily demanding mongoid... i have even less so no opinions yet other than creeping issues with mongoid not behaving as expected and difficulty understanding why
[21:59:04] <loincloth> i don't have a strong sense of what the Ruby community at large uses, etc.
[21:59:41] <kali> loincloth: i am not a big fan, as you have guessed, but i haven't had a chance to try anything else. i think it's probably the "safe" choice, as in "most projects will use it"
[22:01:10] <loincloth> kali: cool, thx again, i'm out the door... have a good one
[22:14:38] <freeone3000> I would like to add a member to a replica set that will never become primary, doesn't vote, and is only hit by queries when specifically mentioned. What is the word for this?
[22:15:24] <freeone3000> Non-voting arbiter doesn't make sense because it doesn't hold data. priority-zero non-voting members still get hit by queries.