Log file Viewer
#mongodb logs for Thursday the 2nd of August, 2012
[01:17:14] <Almindor> our replica set restore is taking too much time and resources, is it possible to just copy files from the master (the whole directory contents) when mongod is turned off and sync this way or would the metadata prevent this from working? (I noticed this method is mentioned on the web)
[01:17:28] <Almindor> I wasn't sure what "all node data" means if all files in the node dir, or just some
[02:51:00] <Almindor> can someone confirm if copying all node data (that is all contents of the node's data dir) to a replica node is the same as doing a full resync? (when mongod is off of course)
[04:51:43] <adiabatic> I have a document that looks like {'quests': {'kill_ten_rats': {'status': 'unstarted', 'progress_max': 10}}}. How can I change the status of the kill-ten-rats quest to 'started' without accidentally erasing the progress_max key/value pair?
[04:58:03] <crudson> adiabatic: update with something like {$set:{'quests.kill_ten_rats.status':'completed'}}
[05:00:08] <crudson> adiabatic: the key points are 1) using $set so the rest of the document is unaffected 2) using dot notation to reference and embedded attribute
[08:53:00] <NodeX> but as it has spaces you probably want .. var foo = 'Vatten & Avlopp'; db.foo.drop()
[08:55:14] <stefancrs> > var mycoll = "Vatten & Avlopp"; db.mycoll.drop(); false > db.getCollectionNames(); [ "Vatten & Avlopp", "accounts", "issues", "system.indexes" ]
[08:55:41] <Bartzy> I mean, if it's not per request - Where the connections are saved, where are they initiated ?
[08:55:52] <Bartzy> And if it's per request - we still create a new connection per page load - so what's the point ?
[08:59:45] <Bartzy> so the connection is saved on the mongodb server - and then the clients only need to open a TCP socket and ask for a connection ?
[09:00:04] <Bartzy> NodeX: Can you provide a link to the docs? I didn't find anything that explains how it works there.
[09:01:16] <NodeX> pass, I really don't care for the internals as they don't effect my apps ergo I don't know where the docs are on how mongodb handles memory with connections
[11:47:56] <atul> hi, can anyone help. i am trying to use casbah with scala 2.9.1 , what should be the sbt dependecy...
[12:26:57] <cenuij> zakg: if there's something missing, patch the package yourself via your open build service account and submitrequest it back to the development project ;)
[13:36:15] <Datapata> Hiya. I have massive problems with my replica set. I have one node on AWS and two in the office, one of the machines in the office is trying to sync and the other machine in the office keeps dying. Since sync uses the closest machine to sync it uses the dying machine.
[13:37:09] <Datapata> I have a mongodump from yesterday witch I took with --oplog, can I use that on the node that needs to be resynced and then let it joing the replica-set?
[13:57:56] <stefancrs> when implementing a "full text search" in mongodb I usually split any stored text up into its separate words and store those as search keywords. but if one also want to be able to search with quotation marks, like "word1 word2" and only get hits with word2 following word1 in the stored text, how would one go about doing that?
[13:59:49] <kali> you can store both the split words and the actual text. on a query, you filter the result given by the words by looking for the right sequence on the text
[14:06:04] <NodeX> stefancrs : you can also use $in to search your separate words if you store them as an array
[14:17:10] <stefancrs> NodeX: yeah, which is why "always use a prefix" caught me off guard, since it was after I said "no regex will eb used, or maybe just ^keywrd" :)
[14:19:07] <akaIDIOT> i remember one of the 10gen guys mentioning something about the mongos process needing all keys in memory, creating a limit on the total size of things
[14:22:22] <akaIDIOT> last two comments on the wiki page indicate the limit is per collection, though does not point to any resource stating why and the like
[14:32:30] <paultag> Hey mongo'ers. I'm interested to know if I can search by finding a subset of a given array. Let's say I have a "capabilities" array for nodes, and in a document, it has a "needed_capabilities" (I'm just making this up as I go on, work with me here) -- is there any way to search for all documents who's required_capabilities are a subset of capabilities (array) ?
[14:37:24] <paultag> If I had a build-farm and I wanted to test based on constraints - let's say my builders have a capabilities array -- [ 'gcc', 'debian', 'sid' ] -- and a build job -- needed_capabilities: [ 'gcc' ] & another with [ 'clang'] -- how can (given the first array, of the builder) I query for all jobs that have a subset of my array (e.g. return gcc, but not clang)
[14:37:37] <paultag> let me write some python to explain this better. Sorry, this isn't my strong suit.
[14:38:38] <NodeX> http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24elemMatch
[14:40:07] <paultag> I'd just like to make sure [ 'joe', 'bob' ] matches ['joe'], ['bob'], ['joe', 'bob'], ['bob', 'joe'], but not ['joe', 'jose'] or ['billy', 'rex']
[14:42:39] <paultag> NodeX: how would you construct the find spec? I'm still a bit lost, I'm new to Mongo.
[14:53:34] <paultag> NodeX: Still around? I could use some help with this, if you could spare some moments for me.
[14:56:51] <NodeX> bleh: [{name:"baz",name:"foo",name:"bar"}] .. then you can do foo.find({"bleh.name":"baz","bleh.name":"foo"});
[14:56:56] <paultag> Well, the thing is that the array will be dynamic, I don't know this before the fact
[14:58:08] <paultag> and also, that's not exactly the right logic - I want to define the superset and find all records with a subset or matching set
[14:58:15] <NodeX> if you can garuntee the order then you can use .... bleh: ['foo','bar','baz'] ... {'bleh.0':foo, 'bleh.1':bar}
[15:02:48] <NodeX> http://stackoverflow.com/questions/6165121/mongodb-query-an-array-for-an-exact-element-match-but-may-be-out-of-order/7938453#comment9706014_7938453
[15:06:43] <paultag> Ah, yeah, I see. I'd like to have defined users: [ '2' ] & gotten back all things with users: '2' in it, as with [ '2', '3' ] getting back all with both 2 and 3 in there
[15:07:12] <paultag> I'd like to have defined users: [ '2' ] & gotten back all things with users: '2' in it, as with [ '2', '3' ] getting back all with both 2 and 3 in there
[15:07:36] <paultag> I get the feeling Mongo can't handle this sort of expression, and I should do this outside mongo
[15:07:48] <NodeX> [15:58:24] <paultag> since that'd allow ['foo', 'bar'] to match ['foo', 'bar', 'baz']
[15:10:25] <paultag> NodeX: OK, I need to sit down with this. I think you've given me enough to work with :)
[15:26:50] <Almindor> could someone please confirm if it's possible to restore a replica node by deleting all it's data and then copying all the data from working master node (including journal etc. all in the folder)?
[15:27:11] <Almindor> I tried a full resync by deleting the node but it's taking ages and I think it actually stalled out of memory o.O
[15:30:21] <kali> Almindor: http://www.mongodb.org/display/DOCS/Resyncing+a+Very+Stale+Replica+Set+Member, last chapter, second option
[15:37:26] <Almindor> I just don't want to kill this stalled resync when I don't know if method #2 will work
[15:38:33] <Almindor> is it normal that a resync of ~300gb can eat 8gb or RAM and 16gb of swap and just stall?
[15:41:26] <Almindor> kali: it's doing this % thing but it's stuck on 94% and I see about only half the files in the slave node dir compared to master
[15:43:29] <Almindor> I think there's no point in letting it hang the server now tho it had the whole night and didn't move anywhere
[15:43:50] <jgornick> Hey guys, I'm trying to use map/reduce to produce a count of documents that will be uncategorized if I remove a category and all it's child categories. I have a sample dataset of my category collection found https://gist.github.com/c1dc2da8fdfb89a9ee69 Any help would be greatly appreciated!
[15:59:25] <niram> hi, i have a question about MongoGridFS::storeBytes in php driver (http://php.net/manual/en/mongogridfs.storebytes.php)
[16:00:23] <niram> what exactly does the "safe" option do? Does it ensure it was writen to the journal (as it should) or does it do fsync (which would sucks)
[16:34:03] <Almindor> just for others info it works if you copy everything from one replset node to another to resync
[17:52:32] <drummerp> Hi, I'm having trouble using the Java connector for MongoDB. I'm using a function which retrieves a single document by the field objectId, as you can see here: http://pastebin.com/vM3BBLwn I have already connected to the database and initialised the `collection` variable. The issue is that when I call cursor.count(), I get an IllegalArgumentException at runtime with the message "'ok' should never be null..."
[18:06:24] <drummerp> I apologize, would it be better for you if I provided an SSCCE instead of just a code sample from a single function?
[18:06:29] <alexyz> anyone has a pointer/link to an explanation of why *mongos* role isn't part of a normal mongo server (why does it need to be a separated process)?
[18:14:52] <Vile1> Hi, does the sharding and replicaset lives together? I want to get something like http://vile.su/pics/is-it-20120802-191338.png that, but not sure if that is how it actually works
[18:21:30] <drummerp> This is very strange. I just created an SSCCE from my project, and the exception went away, and it properly retrieved the document and printed it accurately.
[18:21:44] <drummerp> I suppose it must be something to do with the way I'm integrating it with Spring.
[18:27:06] <Vile1> jY: do you know if mongo compresses the transferred data? If not, then I can probably introduce compression on VPN between DCs
[18:38:13] <thedahv> Howdy. I have a question about a good strategy to search on "full name" fields on a document if I'm currently storing "first_name" and "last_name" separately
[18:38:45] <thedahv> I'm using mongodb through the mongoose library. It's easy enough to define a virtual field so my application-level stuff can get the full_name without much fuss
[18:39:53] <drummerp> I believe I may have found the issue. I believe it to be a scope issue relating to Spring MVC which I have overlooked.
[18:42:25] <thedahv> That does sound nice. I'm not yet at the point where there would be another keyword search other than full_name
[18:42:57] <thedahv> But at the user-level, they don't care about that. They just want to start typing and get the result they want
[18:43:35] <aheckmann> thedahv: so that module lets you specify which document fields to keywordize, e.g. auto-add to the keywords array when saving etc
[18:47:32] <aheckmann> thedahv: the same way you search anything else in mongo. you can treat arrays like normal props, find({ keywords: /name/ })
[18:48:27] <thedahv> So if my query is something like "George Washington" and keywords is ['first_name', 'last_name'], it will match?
[18:49:22] <aheckmann> thedahv: yeah playing around with it quick (take a peek at the tests) will show you more quickly than i can type
[19:23:33] <houms> i have installed mongo on 3 servers and one of them i cannot access the web interface. even though lsof shows the port is listening. it seems it is listening on localhost instead of the LAN ip
[19:28:25] <houms> slava what i am not clear on is the servers were setup the exact same way using the same mongod conf file which does not define bind_ip. just not sure why this one server is doing that. thought i missed a step somewhere
[19:37:43] <jonwage> my secondaries replication is stopped due to this error "errmsg" : "syncTail: 10142 Cannot apply $pull/$pullAll modifier to non-array, syncing: { ts: Timestamp 1343931549000|54, h: 6544226030691754726, op: \"u\", ns: \"opensky_devo.identities\", o2: { _id: ObjectId('501ac24c1c0ba0563500007d') }, o: { $pull: { cart.quoteItems:"
[19:58:14] <thedahv> So the reason I started exploring this is I'm implementing a typeahead search on an input field
[19:59:01] <nofxx> Just wondering.. is it normal a 'local.oplog.rs' of > 1200ms ? All my collections show there with 3ms, 50ms tops...
[19:59:10] <thedahv> This keyword search approach works, but only when the entire first name is entered
[20:20:40] <elux> im trying to figure out how to query if a field is an empty array .. any suggestions?
[20:34:36] <thedahv> aheckmann: I got a decent working solution for implementing typeahead searching on a 'full name' field
[20:35:04] <thedahv> Using the custom function on that keywordize library, I was able to get the union of the character arrays from the first_name and last_name fields
[20:35:51] <thedahv> Then in my query, I can do something like "Person.where('keywords').in(searchChars)"
[20:59:16] <jn> hi, is there a way to get better formatting from the mongo shell? this isn't working for me http://xn--bl-wia.se/bd23f5ed98.png
[20:59:52] <grallan> wingy: I don't think there is from the command line. If you use an ORM, there might be one built for you. Like in mongoose for node.
[21:03:29] <aheckmann> jn: add "DBQuery.prototype._prettyShell = true" to your .mongorc.js file to have it pretty by default
[21:04:38] <aheckmann> MANCHUCK: you create it your self. mongo picks it up if present. not sure about win but should be same AFAIK
[22:37:09] <chubz> Is there a way I can find out which process id is being used by whichever mongo node?
[23:32:01] <rossdm> any way to track progress of db.copyDatabase()? It just seems to hang with no apparent network traffic