Log file Viewer
#mongodb logs for Tuesday the 21st of January, 2014
[00:08:29] <donski> Hi, is it possible to have nested groupings as part of the mongo aggregation pipeline? I'm stuck on an issue where I want to have a nested grouping as the final output. I've detailed it here: http://pastie.org/8652193 if anyone could take a look and advise me
[01:33:11] <arthurnn> see this http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/
[01:34:37] <arthurnn> i see... django might have something to do that for you.. i wrote a gem do that behind the scenes on ruby
[01:35:26] <george2> unfortunately, MongoEngine's Django docs are rather... lacking, so I'm pretty much making stuff up as I go.
[01:36:04] <preaction> george2: iirc mongoengine has something to do it automatically. let me check my code quick
[03:05:38] <SaintMoriarty> I am currently writing a test app to get used to NoSQL and mongo. I am using the MEAN stack and wanted to see what is the best way to take a update post in nodes and apply it to the model?
[03:15:03] <kveroneau> Hello everyone. I have a scary question about MongoDB. Although this video on YouTube was uploaded 2 years ago, I really need to know if this is still a fact, or was just FUD.
[03:15:35] <kveroneau> It says something about MongoDB not writing/flushing to disk often enough, making data loss more of a possibility.
[03:16:42] <kveroneau> Is this FUD? Please say it is so, I really like MongoDB, but some things said there are sort of... well... scary for a DBA or someone who makes choices on what DB to use in an app.
[03:24:17] <kveroneau> retran, never said I am a DBA. However, I am currently in a position where I need to confirm that MongoDB will work for my client's web application. I am doing some search, and have tested MongoDB with Python and seems all too good to be true.
[03:27:42] <kveroneau> retran, yes I have used MySQL for quite awhile. I use transactions to make sure my changes are atomic.
[03:27:52] <retran> are you aware, for example, that "flush tables" in mysql doesn't actually literally gaurantee the data is written once the command is done
[03:28:31] <retran> in mysql using innodb have tons of data that is not yet flushed to disk at any given time
[03:29:16] <retran> it has 0 things to do with the server, and everything to do with the nature of transactional processing
[03:30:07] <retran> if you want to see heartbreaking data loss, talk to people who've done important things with Mysql
[03:32:59] <kveroneau> Hmm, this has been most of the debate about SQL vs NoSQL, each party states that their DB has data integrity and the other doesn't.
[03:34:07] <kveroneau> In the #django channel, they mentioned "risking your data integrity", whereas in this channel, you say there is data integrity.
[03:38:04] <kveroneau> I already talked to Eliza recently about my personal drama, so I don't think that's it.
[03:41:54] <kveroneau> cheeser, thank you. That's what I wanted to hear. I am sure there are lots of developers and use in production to report solid statistics.
[03:46:10] <kveroneau> Just awful, a search for "mongodb case studies" isn't really that helpful, with of course the top results from mongodb's website.
[03:49:13] <kveroneau> The only information to reasure cheeser's claim is the company list on the MongoDB website, which shows lots of large companies with a usually high volume of traffic using MongoDB. For example eBay wouldn't use a DB without proper data integrity would it?
[04:59:26] <btclovernick> Mon Jan 20 23:11:54.002 [initandlisten] Index: (1/3) External Sort Progress: 2772400/4305650 64%
[04:59:26] <btclovernick> Mon Jan 20 23:12:04.001 [initandlisten] Index: (1/3) External Sort Progress: 2815000/4305650 65%
[05:21:07] <Mallanaga> I'm trying to learn this stuff... and I'm using magic cards as a tool... what's a good way to denormalize a JSON file into several documents?
[05:22:33] <btclovernick> Mon Jan 20 23:34:55.009 [initandlisten] Index: (1/3) External Sort Progress: 3366500/4305650 78%
[05:22:34] <btclovernick> Mon Jan 20 23:35:05.013 [initandlisten] Index: (1/3) External Sort Progress: 3409100/4305650 79%
[05:40:20] <cheeser> no idea. are they actively getting errors now? if the data is there but no indexes, it'll just be slow
[05:41:50] <cheeser> well, it's starting. whether or not it's responding to requests is what i'm asking.
[05:51:11] <Mallanaga> is the idea to to have all the information for an instance of the model in the one document?
[09:07:11] <Nodex> perhaps you can try this : http://c-mobberley.com/wordpress/index.php/2013/10/14/raspberry-pi-mongodb-installation-the-working-guide/
[09:08:16] <crashev> Nodex: ok, I saw that, I was expecting some way that I can redirect it to file from the console/mongo shell itself
[09:52:15] <r1pp3rj4ck> it seems that this tutorial is wrong: http://docs.mongodb.org/manual/tutorial/return-text-queries-using-only-text-index/
[09:53:24] <r1pp3rj4ck> not saying i'm not wrong of course, that's why i came here to find someone who can correct me :)
[10:01:12] <theblackbox> hello all, I'm just looking for a way to start the mongod as a service - it currently locks the terminal instance that inits it and this is very annoying when trying to script. I'm pretty damn sure I can't be the only one that's tackled this, but I can't seem to find any useful info
[10:03:42] <theblackbox> locks/owns/binds - you know what I mean where the instance is bound to the running program?
[10:05:36] <megawolt> for example when you try to execute mongod in command prompt. It works but you cannot write anything
[10:06:19] <theblackbox> but this cannot be the desired method of instantiating a MongoDB _daemon_ ... can it
[10:09:57] <theblackbox> but I'd like to script the starting and stopping of the DB as part of my deployment, so it would be desirable to swoop in on SSH wings and --restart the mongos instance but given the way the mongod "service" behaves this breaks my deployment script - there is no way I can automate the starting/stopping of mongod
[10:12:14] <theblackbox> hah, that's the doo-hicky I was looking for... I knew there was some way to force this
[10:12:24] <megawolt> @theblackbox also use can use db.shutdownServer() for close... But restart can be little comlicated :d
[10:13:13] <theblackbox> megawolt: yeah I'm working on restart. I think if I keep tabs on a pid file I might save myself some pain. We'll see
[10:13:45] <theblackbox> Nodex: the mongod service is running on Debian, the deployment script is running on OS X with Bamboo installed
[10:14:57] <theblackbox> Nodex: correct, but I think it was simply a matter of forgetting the trailing & on the init command. Once I've put that in place the rest should fall together
[10:18:37] <r1pp3rj4ck> if you use the init script, restarting mongodb should be as simple as /etc/init.d/mongodb restart
[10:26:28] <r1pp3rj4ck> can i somehow sort the results by a specified integer instead of the score when running text command?
[10:26:52] <r1pp3rj4ck> it doesn't work as this tutorial suggests: http://docs.mongodb.org/manual/tutorial/return-text-queries-using-only-text-index/
[10:43:49] <abhishek> Hi everyone, Is there a way to get whole collection and using distinct in mongo.
[10:46:12] <abhishek> kali: For eg i have documents in collection with multiple email, i can get unique one using db.collection.distinct('email') but this returns only email values, is it possible to fetch other values.
[12:42:53] <kali> ha. that just freeze one member... basically the idea of repairing a replica set is: run repair on each secondary (in turn), then switchover and run on the former primary
[12:49:56] <kali> scristian: then you need a failover. you can either: run repair on the primary and it will let the other secondary elect one of them to primary, or: trigger the failover yourself by freeze()ing the primary, or altering the configuration, or just stopping it
[12:55:02] <kali> scristian: you're aware you need twice the database size on disk for the repair to run ?
[12:58:14] <kali> scristian: because another option is to just delete everything from the former primary disk, and let it pull the dataset from one of the two other nodes
[13:03:13] <chronos> hello guys, is anyone here working with Delphi and Delphi Mongo Driver i need some help with an find command?
[14:02:23] <bdiu> Anyone interested in a few hours paid consulting? I'm trying to come up with the ideal strategy for map/reduce for ongoing data analysis of our data set...
[14:11:16] <Nodex> bdiu : perhaps you can just ask the questions and someone will be able to help you anyway
[14:16:40] <Nodex> probably not going to get a good consultant who knows what their talking about unless you're willing to spend the money tbh
[14:17:04] <ron> yeah, like, I'm willing to do it for half that, but I don't know what I'm talking about.
[14:17:29] <bdiu> I think the $100/hr area is still very reasonable for a professional... just not an agency rate
[14:34:34] <lgp171188> How do I connect to the default test database as an admin user? I have enabled auth and added a user to the admin database. Using that user account and credentials I am able to connect to the admin database, but I want to be able to connect to the test database and all the databases. How do I do it?
[15:02:58] <Razz_> I'm having an issue with Mongo that I've been banging my head against for the better part of two days:
[15:04:19] <Razz_> I store an array of longs using the C++ API, which works just fine. However, when I try to retrieve the data again one of the documents (not all, just one) throws an assertion error 10320:BSONElement: Bad type 110 / 50 / etc
[15:04:57] <Razz_> so it seems that it goes wrong in the BSONElement::size() function, which is incapable of finding the type for that document, even though at insertion time they are all the same (NumberLong)
[15:05:26] <Razz_> google is trying to tell me the DB is corrupt, however if I set the value to 42 everywhere it works again -.-
[15:06:12] <Razz_> The odd thing is though that printing the document as JSON shows a correct JSON object and using the Mongo CLI also finds and shows the document just fine
[15:11:07] <Joeskyyy> Quick glance at the c++ docs shows that BSONElement::size returns an int (if I'm reading this correctly)
[15:11:11] <Joeskyyy> http://api.mongodb.org/cplusplus/current/classmongo_1_1_b_s_o_n_element.html#a1d46b39f5c5a2235a1c5c73f22528974
[15:11:51] <Razz_> Joeskyyy: no, it seems to get into the 'default' clause of the switch inside size()
[15:14:01] <Razz_> calling 'valid()' on the BSONObj also returns false, so the object is genuinely broken, just what about it is broken is a mystery to me
[15:16:50] <theblackbox> can I set a log level? I would like to see what command is being issued to the server from my node app
[15:21:02] <Joeskyyy> Last I heard the way to do that was to set the profiler correct, and the "slow queries" to a really low value
[15:21:56] <richthegeek> can anyone give me a brief summary on the issue with excessive data usage? I've got a DB that's 2gb now with only ~270mb of data+indexes... the database is mostly append-only so it just seems insane?
[15:23:41] <Joeskyyy> richthegeek: Has to do with record padding most likely, mongo likes to use extra data to avoid moves on disk
[16:03:33] <oceanx> hello, I've created a replicaSet, two members (one primary and one secondary) and an arbiter, i did shut off mongo before (i converted a standalone mongo to a replicaset) and copied all the content over ssh to the secondary, then started all the mongod instances configured the replicaSet on the primary and added the secondary and the arbiter
[16:04:46] <oceanx> now what's strange is that i only had a db called "applicationdb" (90GB), and now I can see there's also a db called "local" which weights almost the same on both the secondary and the primary
[16:06:33] <oceanx> in fact i could only find the oplog and a few other informations inside, is there a way to be sure it doesn't weight too much? or is it normal that it takes that much (right now I don't have problems not having enough space since it resides on a 2tb xfs partition)
[17:06:16] <tiller> Is it normal that with the Java's driver, I can't serialize a class into a DBObject? Even if this class implements Serializable ?
[17:15:19] <cheeser> though that's unlikely the cause of a serialization bug it'll definitely hit on the other end.
[17:29:11] <bdiu> Sorry to repeat my earlier query, but... anyone have any interest in a few hours of paid consulting to help with some architectural/query/map-reduce goals? If not, anyone have any recommendations for other individuals or companies that do this? (Not 10gen as their rate is way to high for me $450/hr... ouch!)
[17:56:58] <bdiu> Nodex: good idea... might head that way... just wary about the quality of assistance
[20:01:03] <darkk^> I see quite an interesting behavior: mongodb can run long read (getmore) query for minutes even if the client is already disconnected and the socket is in CLOSE_WAIT state. I see the issue on 2.0.x branch, but I've not found any relevant bugs in jira. Have anyone seen behavior like that?
[20:05:00] <paulovap> is there a way to make text search ignore punctuation like "é" and recognize it for as "e"
[20:14:49] <BlakeRG> Hello, been banging my head against the wall on this for about an hour now, i just need to remove a single value from an array in a document (PHP) https://gist.github.com/BlakeGardner/615f9ff20959d4ab969a
[20:35:52] <kali> BlakeRG: avoid using variable stuff as a docuemnt key... prefer { name: "About", values: ["", "" ] } or you'll regret it sooner or later
[20:40:36] <treaves> When I use appendBinData() to create an entry on a BSONObj, what is the correct way to retrieve that data back off of the BSONValue?
[20:41:20] <BlakeRG> kali: will take that into consideration, i am just writing some scripts to remove data from an existing MongoDB that i didn't design
[20:52:10] <generic_nick> i have a process that does some backing up of old data to long term storage
[20:52:21] <generic_nick> i am continually seeing this as it fails and restarts: shard global version for collection is higher than trying to set to
[20:53:00] <generic_nick> there are multiple processes using that mongos, but only the one reading that specific collection is having issues
[21:43:45] <hugod> I'm trying to set up MMS, but it keeps picking up an unqualified hostname, which doesn't resolve. The replica set is configured using IP addresses. Can we configure MMS to force it to use ip's?
[21:45:29] <hugod> (I have the MMS agent running on the same node as the arbiter, and seed it with `localhost`)
[22:46:00] <proteneer> my document looks like { "foo": { "someList": { "array_1": [1234] } } }, and i want to do an $push on the "array_1"
[23:50:26] <proteneer> Joeskyyy, so find_one returns the entire document, is there a way to query for only a field within the document? so in the above case, I only want to display the 'array_1' list
[23:54:32] <cheeser> not sure about the driver API you're using but find takes an optional projection
[23:54:58] <justen_> What's the best way to handle saving changes to documents with populated fields? Right now I'm going through each field for each schema and changing the populated fields to _Id's before saving, but it doesn't really feel like the right solution.
[23:56:39] <justen_> ha ok. I have a document, and for display purposes on the client side I populate some fields in that document.