PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 31st of May, 2013

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[03:06:57] <nlognbob> where is a good place to discuss mongodb schema design & rest. here?
[03:09:17] <nlognbob> say I have customer record containing email address; customer can edit address, but I need to keep history of addresses [addr, start, end]. how to do rest with this? one document, customer can only edit a portion (makes rest difficult?). or 2 docs, one for history and one for current cust record
[03:48:44] <sinusss> hi
[03:48:55] <sinusss> how do I query field_name: field-1 ?
[03:50:18] <sinusss> woops
[03:50:29] <sinusss> how do I query field_name: field-1 in http://pastie.org/7987112#8
[03:54:12] <sinusss> 'form_elements.field_name' : 'field-1' got it
[04:54:01] <nlognbob> exit
[08:38:29] <MilosStanic> hello everyone
[08:38:49] <MilosStanic> I have pecl mongo 1.4 installed on my Ubuntu
[08:39:15] <MilosStanic> but I keep getting error "class MongoClient not found"
[08:39:50] <MilosStanic> any ideas? similar experiences?
[08:42:34] <MilosStanic> wow, I didn't know Mongoers are such a silent bunch :)
[08:50:44] <ncls> MilosStanic sorry I don't use it with PHP ..
[08:51:04] <MilosStanic> ncls, thanks anyway
[08:52:14] <ncls> MilosStanic did you restart your server after installing the module ?
[08:52:45] <MilosStanic> of course, rebooted the machine, and did phpinfo(), and it shows mongo 1.4.0 driver installed
[08:54:59] <Nodex> MilosStanic : how are you instantiating it?
[08:55:20] <MilosStanic> $somevar = new MongoClient();
[08:55:56] <MilosStanic> sorry, here it is:
[08:55:58] <MilosStanic> $connection = new MongoClient($connectionString);
[08:56:27] <Nodex> are you sure it's 1.4 you have installed?
[08:56:34] <MilosStanic> Error Output: PHP Fatal error: Class 'MongoClient' not found
[08:56:38] <MilosStanic> well..
[08:56:57] <MilosStanic> MongoDB Support enabled
[08:56:57] <MilosStanic> Version 1.4.0
[08:57:04] <MilosStanic> this is paste from phpinfo
[08:57:20] <Nodex> can you try "$db=new Mongo();" and see what happens?
[08:57:36] <MilosStanic> sorry, have to go, thanks guys (taking my daughter to kindergarten ) :)
[08:57:42] <MilosStanic> will be back in one hour
[10:05:57] <joannac> Yay netsplits
[10:28:09] <Nodex> netsplits ftw
[10:29:46] <MilosStanic> I'm back Nodex
[10:29:57] <MilosStanic> I just tried Mongo instead of MongoClient
[10:30:10] <MilosStanic> again I get error, but somewhat different
[10:30:26] <MilosStanic> Error Output: PHP Fatal error: Class 'Zizaco\Mongolid\Mongo' not found
[10:30:55] <MilosStanic> now it tries to find Mongo class in the namespace of the custom library it seems
[10:30:55] <Nodex> are you using some sort of framework or somehting
[10:31:01] <MilosStanic> yes, Laravel 4
[10:31:22] <Nodex> please try on a page that doesn't include your framework
[10:31:42] <MilosStanic> ok
[10:37:10] <gyre007> anyone here uses MOngoid/Moped driver in Rails ?
[10:37:30] <shylent> hello, how would I do the following: insert a document into collection only if there's no document with the same _id as the document, that is being inserted, otherwise do nothing
[10:37:53] <kali> call insert, and catch or ignore the exception
[10:40:12] <MilosStanic> Nodex, MongoClient works outside of the framework... now I'm baffled, is this some kind of namespace conflict or what?
[10:43:08] <Nodex> must be
[10:43:37] <Nodex> loop through the loaded classes in a script and see if it's being loaded by the framework
[10:45:23] <MilosStanic> Nodex, thanks a bunch!!!
[10:46:14] <Nodex> no probs
[10:47:27] <shylent> kali: no, wait, if I call insert, don't I replace the existing document if there's one?
[10:48:57] <Nodex> that's "upsert"
[10:58:18] <MilosStanic> Nodex, the problem was in that I was running script from CLI and PHP CLI has a different php.ini file than the Apache
[10:58:31] <MilosStanic> so there was no extension=mongo.so in the CLI's php.ini
[11:07:20] <Nodex> oops
[11:12:59] <richthegeek> what is the best way to copy a collection (possibly with a query) on the same database+server, but with a different name?
[11:13:24] <Nodex> I normaly export it and rename it but you can copyCollection() iirc
[11:13:29] <richthegeek> doing a find and insert via a program seems like it incurrs a lot of costs that it shouldn't
[11:13:45] <richthegeek> http://docs.mongodb.org/manual/reference/command/cloneCollection/#dbcmd.cloneCollection - seems to be only for remote > local ?
[11:15:20] <Nodex> where does it say that?
[11:17:13] <richthegeek> "copies a collection from a remote server to the server where you run the command"
[11:17:24] <Nodex> mongoexport -d $DATABASE -c $SOURCE_COLLECTION | mongoimport -d $DATABASE -c $TARGET_COLLECTION
[11:17:35] <richthegeek> there is no "as" option to change the target name
[11:17:52] <richthegeek> leaving out the "from" does *something* (I think append to self, which reuslts in no change due to ID conflicts)
[11:18:18] <Nodex> mongoexport -d $DATABASE -c $SOURCE_COLLECTION | mongoimport -d $DATABASE -c $TARGET_COLLECTION
[11:19:04] <richthegeek> does that allow a query? I suppose a remove once the import is done would achieve roughly the same
[11:20:18] <Nodex> -q adds a query
[11:21:01] <Nodex> mongoexport -q '{foo:"bar"}' -d $DATABASE -c $SOURCE_COLLECTION | mongoimport -d $DATABASE -c $TARGET_COLLECTION
[11:21:42] <richthegeek> thanks!
[11:31:11] <r04r> https://jira.mongodb.org/browse/PYTHON-532
[11:34:47] <Nodex> LOL, the database is webscale, the driver maybe not so much
[11:35:49] <Nodex> gotta love butt hurt kids who love to nerd rage because somehting they did failed due to no testing
[11:36:11] <double_p> point in case, even automatic tests would find such foobar
[11:37:15] <Nodex> dbrefs are also a convention, they promise nothing
[11:37:35] <double_p> uh, dont reason a bug ;)
[11:54:28] <richthegeek> that post is gold
[11:54:33] <richthegeek> clearly his ineptitude is your fault
[13:32:46] <marcqualie> I'm having an issue with .remove() on a sharded collection. When I run this with a query, it seems to delete all data in the entire collection. Viewing db.currentOp() shows the command with the query parameters specified. Could it be to do with the fact I don't have an index on the field? Surely that shouldn't matter though, just make it much slower
[14:02:31] <Nodex> seems groupon have been breached :/
[14:03:16] <richthegeek> lolgroupon
[14:07:27] <starfly> marcqualie: it seems more likely that your query isn't limiting as you might expect; have you used the same query on a find().count() and received the expected result? Whether there's an index or not should not affect what's removed.
[14:11:26] <marcqualie> starfly: I think it may have been a bug with the application rather than mongodb itself, the numbers did seem weird to me at first
[14:40:22] <dberry> in mongodb 1.8 if I have a cluster running(primary,secondary,arbiter) and I want to add a hidden member for backups will it cause a renegotiation of the primary?
[14:41:57] <kali> hidden on 1.8 ? that was already ?
[14:46:07] <dberry> it would be a new member but I do not want it to vote, etc
[14:46:14] <richthegeek> how do I use stored javascript with the JS mongo driver?
[14:48:09] <kali> hidden nodes take part in votes. they just don't get queries from clients
[14:48:34] <kali> but i'm not sure hidden nodes where implemented in 1.8
[14:49:52] <dberry> if I add it with votes: 0, will it kick off a re-election? Basically I want to add a new server into production for backups without interruption
[14:54:07] <richthegeek> ah, figured it out - have to wrap the function in a call to mongodb.Code
[14:58:42] <starfly> dberry: looks like (from quick 1.8 doc check) that your primary will stay primary if all secondaries (including new one) are set to priority 0. It looks like hidden secondary was supported for 1.8 too.
[14:59:15] <dberry> do you have a link for the 1.8 docs(all I see is 2.4/2.2)?
[14:59:29] <starfly> dberry: http://docs.mongodb.org/manual/release-notes/1.8/
[15:09:40] <dberry> so if the default priority on my cluster is 1, instead of 0, it will force an election when I add a hidden non-voting secondary?
[15:10:08] <dberry> I never set a priority on the original servers(primary/secondary) so it defaults to 1, I assume
[15:12:28] <starfly> dberry: each member of a replica set has a priority (not a cluster-wise thing), you can login to your primary and run rs.conf() to see the current replica set configuration
[15:15:41] <dberry> yeah, priority is not listed when I run rs.conf() since I never set it.
[15:16:43] <aricg> Hi, when I ran rs.initiate() it took my machines short hostname, not the fqdn, how do I change the name? the other machines in the replica set cannot resolve the shortname
[15:17:10] <starfly> dberry: then presumably (your mileage may vary, this is free advice) you can reset the replica set config to provide a priority of 1 to the primary and 0 to secondaries...
[15:18:15] <aricg> conf.members[0].host = "foo.com:27017" ?
[15:18:36] <aricg> and then.. um restart or something?
[15:21:25] <aricg> or.. rs.reconfig(config, {force: true})
[15:24:42] <grouch-dev> that command executes from the mongodb shell (mongodb 2.4.3) then fails with a socketexception error
[15:24:46] <grouch-dev> in the log file I see:
[15:25:39] <grouch-dev> this /usr/local/mongodb/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdced21]
[15:28:17] <grouch-dev> This only occurs in an environment where replica-set is geographically separated
[15:30:23] <grouch-dev> 2 servers in one location, 2 servers in another location, and an arbitor in a 3rd location.
[15:30:42] <grouch-dev> this is our production environment
[15:31:22] <grouch-dev> our test environment consists of 3 servers in a single location, and the error does not occur there with a backup of the production database
[15:31:44] <double_p> timeout?
[15:35:35] <grouch-dev> How would I know if its a timeout error? There is no indication. Our network monitoring shows that there are no connectivity issues between the environments
[15:36:41] <aricg> okay, I figured it out, https://gist.github.com/Aricg/5685819
[15:36:45] <double_p> good question, i only see that from the app log. but see rs.status()?
[15:37:18] <grouch-dev> rs.status shows everything is fine after this error occurs
[15:38:07] <double_p> but in the log something happens?
[15:38:07] <grouch-dev> sorry for spamming, but this is the entire stacktrace
[15:38:23] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdced21]
[15:38:33] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x9b) [0xd903eb]
[15:38:42] <grouch-dev> . /usr/local/mongodb/bin/mongod [0xa67473]
[15:38:52] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pbb+0x49) [0xa64739]
[15:39:00] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo13deleteObjectsEPKcNS_7BSONObjEbbbPNS_11RemoveSaverE+0x10d4) [0xa767a4]
[15:39:11] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo14receivedDeleteERNS_7MessageERNS_5CurOpE+0x3a1) [0x9ef5e1]
[15:39:23] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xae6) [0x9f56b6]
[15:39:35] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x90) [0x6f1640]
[15:39:47] <grouch-dev> . /usr/local/mongodb/bin/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x42e) [0xdbb53e]
[15:40:25] <double_p> _ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x42e - hmm?
[15:41:13] <grouch-dev> I assume that is from the shell
[15:41:44] <grouch-dev> ie: shell executes the query via socket, socket msg received, being evaluated, then error occurs
[15:42:01] <grouch-dev> the query runs for a very long time before this error occurs
[15:42:14] <grouch-dev> like 5 minutes or so
[15:42:28] <grouch-dev> its trying to delete approx 300k records
[15:42:45] <grouch-dev> db only has around 9million
[15:45:59] <grouch-dev> and there is an index covering the query
[18:10:49] <kfox1111> sh.shardCollection( "pacifica_db.tmds_collection", { _id: "hashed" } )
[18:10:57] <kfox1111> "errmsg" : "exception: SyncClusterConnection::udpate prepare failed: 9001 socket exception [5] server [m0:27019] m0:27019:{}"
[18:11:10] <kfox1111> Any ideas?
[18:11:27] <kfox1111> the machine in question seems to be up and working.
[18:12:56] <kfox1111> In fact, I see a message from m0 when I issue the command.
[18:16:08] <kfox1111> ah... for some reason its trying to contact m0:27019 and m0 is only listening on 27017 and 28017. Why would mongos try and talk to it on 27019?
[18:17:05] <kfox1111> OH.... its trying to talk to the config server, not the database. sorry for the noise.
[18:18:21] <leifw> hey, does anyone know how to compare to ISODates in javascript?
[18:18:23] <crudson> kfox1111: there is always someone watching that will learn from a problem being solved
[18:18:51] <skram> hey crudson, how's it going?
[18:19:30] <crudson> skram: fabulous. How's the aggregate thingy working out?
[18:19:50] <skram> good to hear. perfectly fine!
[18:19:51] <leifw> this is the best I've got so far, but it doesn't look great: http://pastebin.com/81inmNGj
[18:21:39] <kfox1111> crudson: True. :)
[18:21:41] <cmacmurray> Hi yall, im trying to automate reading of the web status url on port 28017 for mongo, and ive tried using requests, and urllib2, no matter what I get a 401
[18:21:46] <cmacmurray> has anybody else seen this
[18:21:53] <cmacmurray> im using correct username /password
[18:25:41] <skram> leifw: I dont know much about javascript.
[18:25:57] <skram> But, I think I found a solution. Just tested.
[18:27:03] <skram> leifw: http://pastebin.com/PutySCFx
[18:27:08] <skram> Is that what you're aiming for?
[18:27:08] <crudson> leifw: d1.getTime() == d2.getTime() should be true for two separate Date/ISODate instances that represent the same date
[18:27:36] <skram> ah, crudson. we're on the same page :)
[18:27:40] <crudson> heheh
[18:34:09] <skram> If a write request is made on a secondary, does it forward it to the primary? or does it just die?
[19:42:24] <jstream> I've worked through this example, but I can't figure out how to return the results from a .find to a variable. Example: http://howtonode.org/node-js-and-mongodb-getting-started-with-mongojs
[19:43:11] <jstream> var result = db.users.find(); doesn't seem to work
[19:58:11] <harenson> jstream: explain yourself
[19:58:39] <jstream> eh, I think I figured it out. You have to use a callback
[20:19:27] <starfly> skram: write attempts to secondaries will just fail
[20:20:11] <skram> starfly: i see. I have a master-slave setup on 2 servers, im paying for a load balancer but not using it because not sure how to handle that situation.
[20:21:07] <skram> if traffic is directed to the server where the secondary resides, the requests will fail.
[20:29:27] <starfly> skram: typically for replica set configurations, leaving the sharding topic and mongos aside, you want to direct your application mostly-write connections to the one primary/master and read-only connections to one or more secondaries. Maybe you should think in terms of having multiple secondaries and load-balance between them for reads instead of your current config
[20:35:05] <skram> starfly: i see, makes sense.
[20:36:30] <starfly> skram: of course, what is primary one minute can become secondary the next, so it isn't simply a matter of directing connections to one particular mongod on one server for writes, etc.
[20:37:43] <starfly> skram: drivers typically sort it out, so connection setup is the key
[20:37:59] <skram> ok i see
[20:51:04] <starfly> skram: good doc for this topic: http://docs.mongodb.org/manual/core/read-preference/
[20:51:35] <AlanBell> evening all, I am just starting out with mongodb and I am looking at security and authentication, should a web application just use one user for the application itself or create a user in the database for each user of the application?
[20:52:02] <AlanBell> I can't really see how to do user level control of data with the roles that I am reading about
[20:55:40] <crudson> AlanBell: do the former. your application should handle user authentication to itself and guard what actions a user can perform.
[20:58:27] <AlanBell> OK, thanks
[21:03:13] <harenson> AlanBell: Cork
[21:03:34] <AlanBell> hmm?
[21:04:11] <harenson> AlanBell: sorry, are you using Python + MongoDB?
[21:05:32] <AlanBell> yes, python at the moment, not completely decided on the language though
[21:06:09] <AlanBell> it is a two part thing, it receives emails and puts them in the database, that bit is certainly python, the web front end will probably be python, but I am open to other suggestions
[21:07:20] <AlanBell> ah, cork is an authentication framework
[21:17:02] <harenson> AlanBell: yes
[21:17:05] <harenson> AlanBell: http://cork.firelet.net/
[21:30:14] <AlanBell> thanks, that looks quite handy
[21:38:43] <AlanBell> yay, I have data arriving in my database
[21:46:16] <harenson> AlanBell: :D