Log file Viewer
#mongodb logs for Tuesday the 19th of June, 2012
[02:34:45] <personal> Hey guys, I was wondering if anyone here could potentially help me find an alternative for a SQL query -- I have no clue how to do this on Mongo.
[04:01:09] <richardraseley> Just to clarify my understanding on Mongo - a collection contains one or more documents and those documents contain key, value pairs called "field" and "value" respectively?
[04:06:45] <richardraseley> mrpro: I am trying to visualize what you are saying. If I were storing a list of employees - name : john smith, address : 123 any street, etc.
[04:07:14] <richardraseley> If the key is "name" and the value is "john smith" - where does the "element" come into play?
[04:07:46] <richardraseley> mrpro: Or are you just talking more generally about an element of an array?
[04:10:17] <richardraseley> So, the element is just the name for the key : value group - the element isn't something that has a structure of its own.
[04:11:57] <richardraseley> vs. an "element" being an actual identifier distinct from the key:value pair it contains.
[04:13:04] <richardraseley> I think I was thinking of it in terms of (for example) a row containing a single column in cassandra
[04:13:15] <stefancrs> how do I change a specific field in all documents in a collection? I need to change all "punchIn" : 2 to 0 :)
[04:13:15] <richardraseley> There is the row key (which has a value) and then the column (which is a key value pair).
[04:15:45] <advorak> stefancrs, http://www.mongodb.org/display/DOCS/Updating ; db.collection_name.update( criteria, objNew, upsert, multi )
[04:17:35] <advorak> stefancrs, upset will create a document if the matching criteria """Upsert means "update the document if present; insert (a single document) if missing"."""
[04:20:39] <stefancrs> in my specific case: db.videos.update(true, {$set : {"punchIn" : 0}}, false, true); ?
[04:21:02] <advorak> db.vehicles.update( {category: "Cars"} , { $set: { discount: 0.25 } } , false, true)
[04:21:23] <advorak> will update the vehicles collection matching the category Cars and set the discount field in the result to be 0.25 (multi is true)
[04:21:34] <advorak> http://www.mongodb.org/display/DOCS/Atomic+Operations#AtomicOperations-FindandModify%28orRemove%29
[04:22:06] <stefancrs> advorak: and if one want to match all documents, having "true" as a criteria works?
[04:23:26] <advorak> stefancrs, are you wanting it to update if the field merely if it EXISTS in a document? or do you want to update it if the value of x is true?
[04:24:02] <stefancrs> advorak: the field exists in all documents in the collection. I want to set its value to 0 on all documents.
[05:29:24] <omid8bimo> im trying to run repair on my database but i get this error: "errmsg" : "clone failed for kookoja with error: query failed kookoja.system.namespaces"
[06:39:20] <omid8bimo> im trying to run repair on my database but i get this error: "errmsg" : "clone failed for kookoja with error: query failed kookoja.system.namespaces"
[06:39:50] <Guest42665> why don't you use the mailinglist instead of waiting here for hours for a response?
[06:40:59] <omid8bimo> i prefer live conversation. but probably knows the issue and can respond here faster
[06:47:32] <Guest42665> yes, you can wait for hours and days or get a more fast response on the list
[06:49:16] <justdit> hi, I've a problem with mongo everytime I restart my os. I get Error: couldn't connect to server 127.0.0.1 shell/mongo.js:84
[06:51:57] <Guest42665> justdit: because you mongod is obviously not started during the boot phase...
[07:02:16] <justdit> Guest42665: starting manually doesn't help, I call --repair each time after rebooting
[08:21:08] <skot> Guest42665: you should not call —repair without a good reason. It will be time consuming to copy all the database files and rebuild the indexes.
[09:59:48] <guest_34672> Hello. I have a question regarding the java driver. I have a class "MyClass" that extends BasicDBObject. If I call "(MyClass) collection.find().limit(1).next()" everything works fine. It is correctly casted to MyClass. Now I want to use findAndRemove. When I call "(MyClass)collection.findAndRemove(new BasicDBObject())" i get the following Exception: java.lang.ClassCastException: com.mongodb.BasicDBObject cannot be cast to org.example
[10:15:14] <skot> not really, there is a thing called quotaFiles which limit the number of files create for any database (each database)
[10:17:15] <gigo1980> at the moment i have in my shard and also replication the following naming convention "mongo01" mongo02… now i will set the full quallified domain name for the nodes, how can i do this on production ?
[11:32:29] <omid8bimo> im trying to run repair on my database but i get this error: "errmsg" : "clone failed for kookoja with error: query failed kookoja.system.namespaces"
[11:51:30] <gigo1980> if i have a problem with a node. i stop the node, delete the hdd and than start the node again
[13:05:03] <Progster> I don't understand. I'm connecting to my db using my user name and password, and yet when I execute show dbs I get a message same listDatabases failed, "need to login"
[13:48:11] <infinitiguy> currently I have 3 1gb journal files - what causes this to grow and is there any way to limit it?
[13:49:31] <skot> no, it only has 1-3 files at a time and roll over as new ones are created and deleted.
[13:49:39] <dcrosta> infinitiguy: the journal files are rotated as they age out. in general you'll have 3 or 4 gigs of journal files, but possibly a bit more under heavy write load
[13:54:15] <dcrosta> infinitiguy: I'd have to check the code to be exactly sure, but I believe the case is when lots of data is written between main data file flushes. journal files will be kept around until all the data they represent has been flushed to the main files.
[13:54:47] <infinitiguy> one more question: what is the order that a sharded environment should be started? config server - mongos router - mongo databases?
[13:55:00] <infinitiguy> or does it not matter much and they'll converge once everything is online eventually?
[13:58:19] <infinitiguy> actually - one more after that - is there a way to see how much whitespace is allocated compared to actual data within a database? Our mongo has about 6.5-7GB of DB files allocated but looking in the collections it only says about 1.5gb of data
[14:13:12] <infinitiguy> db.createCollection("results", {capped:true, size:20769803776}) for example
[14:13:32] <dcrosta> infinitiguy: both, or rather, whichever is hit first. there's an implicit limit of 2**31-1 documents if you don't specify "max"
[14:14:15] <dcrosta> see http://www.mongodb.org/display/DOCS/Capped+Collections#CappedCollections-Options
[14:14:21] <infinitiguy> if we have a collection specified for 20gb how could the mongo DB grow to 100gb?
[14:16:45] <remonvv> Yes but a) How did you conclude your data should actually take 20gb of space and b) why do you think the 100Gb reserved for data files is wrong?
[14:17:02] <dcrosta> infinitiguy: could be other databases (the "local" databse can be large if you have a lot of free space and are using replica sets), or other collections in your "main" database
[14:18:20] <infinitiguy> i guess those are good questions - I'm not familiar with how this environment was setup - just throwing a question from over the wall here in the office
[14:18:30] <remonvv> If you want to store X bytes of data in MongoDB it will not result in X bytes of disk space used. Every document includes all field names as well so that alone may double or triple space used. Then there's things like index files, oplog, journaling, preallocation of data files, padding, etc.
[14:19:13] <remonvv> I can give a more detailed description but most of it is pretty well documented.
[14:20:23] <remonvv> mongos is a router process, it needs to figure out cluster topology from a config server
[14:21:34] <infinitiguy> the only difference was today I initially tried to start it using an init script (copied and changed from the default mongo init) and I got that error so I tried my old way.
[14:24:32] <infinitiguy> Tue Jun 19 10:14:50 [websvr] admin web console waiting for connections on port 31000
[14:24:32] <infinitiguy> Tue Jun 19 10:14:50 [Balancer] config servers and shards contacted successfully
[14:26:47] <rydgel> Hey guys, just to be sure. If I have to repair a sharded database, I would have to do a db.repairDatabase() on each shard, right?
[14:27:27] <remonvv> rydgel, it'll invoke it automatically on the shards that are required to know about the repair
[14:28:04] <remonvv> rydgel, then do it on that shard :) it's usually better to just do it through mongos though. It's pretty quick if you have journaling enabled.
[14:29:45] <remonvv> rydgel, shards don't know they're part of a sharded setup so all operations that do not affect cluster topology in some way can be invoked on a local shard.
[14:31:58] <rydgel> remonvv: ok I see. In fact I’ve got a problem because my chunks do not move anymore. I've got a chunk which is somewhat unmovable and mongos try to move it forever
[14:32:32] <rydgel> remonvv: so I guess something is broken on the source shard, that's why I'm trying to repair it
[14:39:41] <spillere> to add another photo, would I add an update,save? how would be the new query to add a new item to photos?
[14:48:18] <remonvv> spillere, db.col.update({username:<YOUR USERNAME>}, {$push:{photos:<YOUR PHOTO URL>}})
[14:54:10] <spillere> i'm trying db.dataz.update({'username':cookie_usr}, {'$push':{'photos': { 'filename': filename_out, 'caption': caption, 'created': date}}})
[14:56:14] <remonvv> if it doesn't work it's because {'username':cookie_usr} doesn't match with any data
[14:56:21] <infinitiguy> in a sharded environment are all deletes done through the mongos service now?
[14:56:39] <infinitiguy> in other words - if I have a collection and im trying to delete all where somenumber=391
[14:58:04] <remonvv> doing things through mongos means the whole business of talking to a sharded cluster rather than a single mongod is hidden
[14:58:38] <remonvv> there are a few edge cases where it might be more efficient but that generally doesn't apply to deletes.
[14:58:46] <infinitiguy> and is the process to do something like this db.collectionname.remove({idname:391});
[15:05:18] <rydgel> remonvv: ok, here is the log from the error I got. Very strange I don’t understand. http://pastebin.com/1YXSmkGF
[15:06:50] <remonvv> rydgel, your data header is malformed. Repair is your only possible route but might fail. Alternative is to find the broken object and delete it.
[15:14:32] <rydgel> remonvv: there is a mention of ObjectId('4f49803dbf30e2a54f000100') but I cant find it with find()
[15:18:49] <Scyllinice> What's the right way to create a unique index on embedded documents? I want a user to have only unique "likes" (category_name, and name) but I want other users to be able to like the same things. So I want the unique index to apply only within the context of the user itself, not the entire user collection. I'm not sure how to go about it.
[15:22:43] <skot> make the query such the if the user has it already it doesn't return any docs for the update
[15:23:10] <skot> If you provide a sample doc in gist/pastebin/etc I can help you fashion the query.
[15:28:08] <Scyllinice> skot: Scrubbed it to contain only the bits I important: https://gist.github.com/34c8b054b3bbd3c2a3dd
[15:36:25] <remonvv> Scyllinice, why would you want to do that with an index? Just only add it when it's not already there.
[16:02:29] <remonvv> rydgel, not an official feature but you can remove the chunk from config.chunks collection
[16:03:19] <remonvv> try repairing the db, if that doesn't solve it report the issue and wait for a 10gen response
[16:03:39] <remonvv> if that doesn't work try exporting the entire database, set up a new cluster and import it again
[16:04:27] <remonvv> you can have a look around to get a better understanding of how things work but i'd not update anything there
[16:23:30] <Goopyo> Q: I want to write a ranking algorithm that calculates P/L on trades for all users/trades. If I have say 50,000 trades and new trades come in regularly and I want read all 50,000 documents to python process them. I have 2 ways of doing this:
[17:16:21] <richardraseley> So, is Mongo's query language officially called anything (e.g. Mongo Query Language, MQL) or is that just what people refer to it as?
[18:04:33] <alnewkirk> I can tell you that MongoDB is a company also and you can apply for a job there
[19:16:19] <doug> just so i don't have to worry about those dependencies when porting this stuff to a new architecture
[19:17:13] <Progster> I'm using mongo as a lookup for a set of values (i.e. distinct). It seems wasteful to create an objectid for each item since each item is already guaranteed to be unique. But when I try to set the ObjectId to my string, I get a mesage saying, "Illegal ObjectId format". Is what I'm doing a bad idea in general? Or am I just misunderstanding something about ObjectIds?
[19:18:20] <doug> so i don't have to worry about those dependencies when porting this stuff to a new architecture
[20:07:44] <Vile1> Guys, mongo 1.8.4 replca set switches primary node every few minutes. Is there a way to lock one one as primary?
[21:29:38] <christopherbull> I'm trying to setup a db to load with --replset and --rest options, but it's currently being managed by a service script on AWS linux (redhat variant), can anyone point me to where I can set these config options either in the init script or in the config file?
[23:23:46] <christopherbull> I'm getting this error when I try to add a replica set node on ec2 "need most members up to reconfigure, not ok ". I've init'd the rs on the main node and just trying to add one more, can anyone shed any light?