Log file Viewer
#mongodb logs for Tuesday the 8th of October, 2013
[00:48:06] <datafirm> is the 16MB limit on the aggregation pipeline for each document returned as a max size, or the collection response as a whole?
[00:52:18] <Lobosque> Hey guys, I'm new to mongo. I have three collections: users, items, and quotes. items references users and quotes references items. What is the best way to get all quotes for items of a given user?
[01:24:12] <rafaelhbarros> cheeser: I figured out. I've been facing a bunch of interesting problems with mongo & pymongo this week
[02:00:30] <adizam> Using mongoose & nodejs.. mongoose attempts to call 'collStats' quite a bit.. and looking at my mongo logs, I'm seeing: DateTime [conn#] command denied: { collStats: "collection_name" }
[02:27:14] <adizam> It is nodetime.. initiating a new (non admin) connection for every collection I have.. and attempting to run db.collection.stats(), among other things.. does it every X seconds
[02:27:44] <adizam> I'm not sure if map reduce does a collection scan.. and I know I have a few map reduces in my codebase.. so maybe that is another instance.. but I have confirmed this is the case for me at least. Thanks for the feedback tho :)
[03:01:59] <defaultro> hey guys, is it possible to implement somewhat similar to triggers? I want mongodb to react when a field gets updated. What it would need to do is connect to our nodejs server
[03:03:39] <adizam> defaultro: http://stackoverflow.com/questions/9691316/how-to-listen-for-changes-to-a-mongodb-collection
[04:10:31] <ruphos> I'm running into an issue using the MongoDB perl driver. I'm trying to page through a large number of documents (125M) to move an array field to its own collection. When I set a batch to be much over 100 docs, I get "can't get db response, not connected"
[04:11:08] <ruphos> it looks like an older issue (https://jira.mongodb.org/browse/PERL-196), but I'm using the latest version of the driver so that should be fixed.
[05:51:27] <ctppppp> Hi everyone, was wondering if when upgrading a mongo replica set from 2.0 if I need to upgrade from 2.0 to 2.2 or I can just go 2.0 straight to 2.4 by following the upgrade procedure in the manual?
[06:59:56] <johnnode> Hello. I did try to apply this case study (http://docs.mongodb.org/ecosystem/use-cases/storing-comments/#hybrid-schema-design) with this test : http://pastebin.com/rUQheti9. But I got inconsistent data for parent & child. The results of parent & child are not consistent. Did I miss some write/update concern in the code? Thanks a lot for your help.
[07:18:58] <johnnode> Hello. I did try to apply this case study (http://docs.mongodb.org/ecosystem/use-cases/storing-comments/#hybrid-schema-design) with this test : http://pastebin.com/rUQheti9. But I got inconsistent data for parent & child. The results of parent & child are not consistent. Did I miss some write/update concern in the code? Thanks a lot for your help.
[08:28:28] <hello_kitty> can mongodb (or a lightweight likeness) be built into an offline desktop app and packaged/configured in an installer?
[09:14:56] <quattr8> I'm getting really weird behaviour from mongodb :/ I have a session tracking collection were i do findOne's only on the _id (=shard key) before getting the session data I get the site data also with findOne, i cache the site data in memcached, this way the tracking findOne takes about 0.04 seconds to fetch, but when I disable memcached on the site data and force it to get the data from mongodb the tracking findOne takes 0.001 seconds and t
[09:17:21] <quattr8> how can querying the tracking collection suddenly be fast when querying another collection before?
[09:38:59] <KamZou> Hi, i've the following command : mongodump -d stats -c $CUR_COLLECTION --query '{"_id.d": '$YDAY_TIMESTAMP'}' -o ... <<< It's working with integers but not with strings, how could i do please ?
[12:18:32] <phrearch> is there some example populate function i can use to for instance populate userids with the appropriate user objects from another query? i know about mongoose, but cant use that in my case
[13:16:49] <Kim^J> I need some help with $elemMatch. According to http://docs.mongodb.org/manual/reference/projection/positional/#proj._S_ and http://docs.mongodb.org/manual/reference/method/db.collection.find/#db.collection.find and http://docs.mongodb.org/manual/reference/projection/elemMatch/ I'm using them right. But my result is a bit odd. Instead of selecting the matching array element it selects every ...
[13:50:22] <birdy111> I am facing an issue with mongodb.... during continuous writes to mongodb ... if I drop some older database, the write performance drastically drops....
[13:52:20] <birdy111> We want maximum 365 databases to exists at a time... so as soon as 366th database comes in we want the first one to be dropped....
[13:54:18] <birdy111> For example today's db is 08-10-13... so the database of 08-10-12 should be removed....
[13:56:49] <Kim^J> birdy111: If you let each database exist on different drives it shouldn't happen.
[13:57:13] <Kim^J> birdy111: I think the performance drops because it now writes twice the amount of data.
[13:59:14] <birdy111> we remove database by issuing "db.dropDatabase()" from mongos... after running "use 08-10-12"
[14:01:54] <birdy111> For drop database, mongodb has to just remove the database file and clean config db, which should finish within a second... according to some experts... But in my case it takes 4-5 minutes on each shard
[14:24:26] <Number6> Are both servers showing up on rs.status()? If so, you may need to add another server (or arbiter) to balance out the votes
[14:28:40] <talbott> it's weird that when i start the mongo command line on the primary though, that the prompt is :SECONDARY!
[14:31:21] <Number6> Ok - does your rs.status() show two nodes? If so, you'll need to add in another one, for a majority to take place
[14:32:02] <talbott> "errmsg" : "not electing self, not all members up and we have been up less than 5 minutes",
[14:32:18] <defaultro> hey guys, I'm following this guy's howto. I tried inserting a data but nothing happens on my php page. http://jwage.com/post/30490196727/mongodb-tailable-cursors
[14:33:53] <Number6> Your cluster is in, essentially, failsafe mode. This kicks in to prevent split brain senarios, such as having two primaries.
[14:36:58] <defaultro> btw, when I ran show dbs earlier, I didn't see test. But now, it's showing it. Why?
[14:40:27] <defaultro> hey guys, how do we create a db? I know how to create the collection but not the db. What is the command?
[14:45:49] <bmcgee> And on a related note, I need some help with tailable cursors, mine aren't behaving as I would expect
[14:48:45] <bmcgee> I think my issue is down to my key. I know that capped collections maintain the insertion order. If I am using a capped collection as a kind of oplog and want to restart the cursor from a known point, am I able to do something like find({ _id: { $gt: "bla" }}).sort({ $natural: 1
[14:49:13] <bmcgee> soz hit enter early find({ _id: { $gt: "bla" }}).sort({ $natural: 1}).addOption(Bytes.QUERYOPTION_TAILABLE)
[15:27:35] <bakis> hey guys i'm using mongohq on heroku, is there a way to dump the data without the user credentials? like the heroku username/password etc? or restore the data in a database while skipping that step?
[15:30:59] <appleboy> also, if a map reduce job errors out on insert due to duplicate key, will it stil try to enter everything else?
[15:41:02] <remonvv> chaotic_good: By making sure you don't store more data than fits on a drive I suppose.
[15:49:43] <remonvv> It's rather like asking how you can prevent a bucket from overflowing if you put too much water into it. The cause is you putting too much water in it, so prevent that from happening.
[15:55:56] <remonvv> Right, but you don't want to hit that point really. If you want to do a write but you can't because you ran out of diskspace you need more diskspace or less data. There's no "Oh we'v reached 20Gb, I'm going to reject writes from now on" sort of functionality.
[15:57:00] <chaotic_good> if they are simply storing files......is the mongo gridFS a better way to go?
[15:57:02] <remonvv> chaotic_good: That's a specific type of collection that is size limited, and thus loses data (FIFO) if it becomes full.
[15:57:31] <remonvv> Err. Well. Depends who you ask. I wouldn't really store files in MongoDB usually.
[15:57:48] <remonvv> Yes, that's very specific to capped collections. It's the collection type used for mongo's oplog for example.
[15:59:31] <remonvv> Whichever, some sort of storage solution optimized for file storage and serving. In my humble opinion GridFS (and databases in general) aren't exactly the sweetspot for that sort of thing.
[16:10:53] <defaultro> this is the output of the cursor though, how can I use this? HASH(0x7fe9397068b8)
[16:14:31] <quickdry21> When migrating the primary of a replica set to a new machine, does the balancer need to be disabled?
[16:14:48] <ruphos> defaultro: http://search.cpan.org/~friedo/MongoDB-0.702.2/lib/MongoDB/Cursor.pm#METHODS
[16:16:55] <ruphos> I'm running into an issue using the MongoDB perl driver. I'm trying to page through a large number of documents (125M) to move an array field to its own collection. When I set a batch to be much over 100 docs, I get "can't get db response, not connected" or "missed the response we wanted, please try again"
[16:17:06] <ruphos> it looks like an older issue (https://jira.mongodb.org/browse/PERL-196), but I'm using the latest version of the driver so that should be fixed.
[16:18:39] <ruphos> it seems like an issue with the perl driver, or maybe how the master/slaves are set up
[16:26:34] <remonvv> The only relevant context is capped collections, which are size limited collections with FIFO semantics.
[16:34:24] <remonvv> There might also be a way to just have mongodb log oplog entries but I've never used or tried that.
[16:47:16] <drag> I have two documents, A and B. B references A through a field. I want to delete A and, when that happens, I want to delete all B that references A as well. Is there a way for me to do it via aggregation framework?
[16:53:02] <remonvv> No. Your only option is embedding if it needs to be an atomic removal or execute two deletes if you do not have that requirement.
[16:53:52] <remonvv> Of course, if B's can no longer be references once the A is gone you can opt for removing the orphaned B's at a later time.
[17:25:40] <clarkk> can someone please tell me what the difference is between the _id and the obj.parent[0]._id in this database? http://pastebin.com/raw.php?i=qEFpSH9i
[17:29:40] <cheeser> welll, _id is the id of that document. and the other seems to be more of a foreign key.
[17:30:56] <clarkk> cheeser: what is the difference between what looks like a simple string for the _id and the ObjectID('xxxxx') of the sub document?
[17:33:11] <clarkk> ruphos: the _id is supposedly the originating unique id, that the subdocument references
[17:36:05] <ruphos> e.g., http://stackoverflow.com/questions/7764781/store-id-as-object-or-string-in-mongodb
[17:36:39] <clarkk> I have tried all day to find something that will allow me to load fixtures (json data) from a file, where subdocuments reference the first-class objects. Does anyone know how I can do this, please?
[17:37:08] <clarkk> I've tried two node packages - pow-fixtures and mongoose-fixtures, but neither do it correctly
[18:04:01] <neeky> at the application level, is having default values for all document fields in my schema a good practice?
[18:08:59] <chaotic_good> at the app level you should strive to keep blobs in disk and logic in ram, including the string key to the blob location
[18:25:37] <defaultro> hey guys, if we update(NOT insert) a value in the collection, will oplog see it?
[19:03:19] <squeakytoy> Hey all. Is this a typicle situation where a relationship database is better? I have a collection of Videos where each Video can have a number of Tags. Here is the killer requirement; I should be able to find Videos by Tags.
[19:04:10] <cheeser> depends on how you want to model tags, but you could just embed the array of tags directly on the video's document. mongodb lets you query against those, too.
[19:06:01] <defaultro> I would like to try this, http://danielweberonline.wordpress.com/tag/tailable-cursors/
[19:06:54] <defaultro> do I need another MongoDB or can my existing one be converted to a replicaset?
[19:11:09] <defaultro> however, on our existing work setup, data is big. When I make a new replica, do I still need to reconfigure our existing mongodb?
[19:14:03] <defaultro> since I'm on the experimtation part, what would you suggest to me if what I'm doing is tailing the oplog to monitor updates/inserts/deletes? Should I convert to replicaset or bring up another mongodb server?
[19:15:33] <defaultro> in your first sentence, does it mean I should convert my standalone as replicaset?
[19:22:38] <defaultro> reading this now, http://stackoverflow.com/questions/13603096/starting-over-with-replica-configuration-in-mongodb
[19:30:46] <defaultro> should this be done on the same machine or should it be done on another mongodb machine?
[19:31:12] <cheeser> that other instance will need to be brought up using the same --replSet option, though.
[19:35:18] <defaultro> now, I need to figure out the perl code for tailing it. I have a code for tailing the regular tailable cursor
[19:36:28] <dllama> hi, i'm using mongoid in a rails4, would this be the wrong room to get some info on eager loading?
[19:36:45] <defaultro> oh, I started this mongodb with --replSet rs0. Should I kill and restart it without this parameters?
[19:41:28] <dllama> guys, anyone? eager loading? https://gist.github.com/mvoloz/3f920a722b233e37f3ea — this definitely does not seem the least bit efficient when fetching records through an association
[19:42:30] <defaultro> ruphos: I do, this is my code from my first one but it's not query oplog.rs, --replSet rs0
[19:47:53] <ruphos> I think you just need to use 'rs.oplog' as collection instead of 'my_collection'
[19:48:32] <defaultro> do I need to restart mongodb since it's currently running with --replSet rs0?
[20:01:14] <dllama> I can't seem to get an answer anywhere to my eager loading issue, and its quiet frustrating
[20:02:37] <defaultro> do I need to add "_id" : ObjectId("525404579a368e6906e98c5f"), to my update command?
[20:05:56] <defaultro> now, I can install http modules in perl so I can write a code that will connect to NodeJS when there is an update in mongodb :D
[20:18:59] <defaultro> i'm having some minor issues with the field lenght. The lastname field is not accepting more than 7 letters. failing update: objects in a capped ns cannot grow
[20:44:24] <diegows> is it safe to remove all the files from a DB from the file system? I need a repair, don't have enough space and I don't need one of the DB anymore
[20:58:51] <cheeser> defaultro: the oplog does its thing regardless of what kind of collection you create
[20:59:18] <defaultro> so if I make a non capped collection, there will be no limit on string lenght?
[21:03:13] <defaultro> db.my_collection.update( { "name" : "ron" }, { $set: { name: "ron ron ron ron ron ron ron ron" } } )
[21:04:15] <davi1015> Can anyone tell me what moveChunk.commit is doing in the change log, sometimes i am seeing 1 hour pass from that point to the point moveChunk.from shows up ( doing a loop to moveChunk's manually so thats actually the time commit took to run)
[22:01:42] <cordoval> and i now just need to get the mapping lines for when inheriting a supperclass mapping
[22:36:58] <dllama> hey guys, i'm getting a "too much data for sort() with no index" error. am not really sure how to fix it. i've tried running db:create_indexes but that didn't solve it