Log file Viewer
#mongodb logs for Tuesday the 9th of December, 2014
[00:04:05] <d-snp> hi, I wasn't paying attention and let a shard take 97% diskspace, I've reduced the dataSize, but I'd like to reclaim some diskspace to have some breathing room for other services
[00:04:49] <d-snp> it's replicated, can I stop the shard, remove the database, and then start it again? will it automatically reassemble from scratch?
[00:06:16] <d-snp> in the replicaset tutorials it only says how to perform maintenance, not what to do when for example a disk failed or something, perhaps I'm looking at the wrong place
[00:09:26] <d-snp> "Restart the mongod with an empty data directory and let MongoDB’s normal initial syncing feature restore the data."
[00:13:43] <Boomtime> d-snp: when you say "stop the shard, remove the database", are you talking about a single member of a replica-set? or are you talking about the replica-set of a single shard?
[00:15:02] <d-snp> and I'll copy at least ones data to a hard-disk so I won't lose anything if it's a disastrous failure :P
[00:16:16] <d-snp> cool, thanks guys I'll start the procedure as soon as the cleaning process is done (it's removing 100's of gb's of data now :P)
[00:52:16] <kenalex> i am new to mongodb and want to know what type of applications it is suitable for ?
[01:06:42] <hahuang65> does that mean it didn't restore all the objects? Why do the numbers not match?
[01:44:55] <armcor> if I had two servers one 2GB mem and 2 cores and the other 4 GB and 4 cores would i tmake sense to would it make sense to put my mongo instance on the larger one and my node.js app which doesnt do much other than query my mongo store on the smaller?
[01:45:37] <armcor> it seems like it would but under load I cant seem to see mongo appear anywhere near the the node instances in the output of top
[07:30:12] <Ravenheart> is there a way to tell mongo to ignore properties it can't find in the database?
[08:02:11] <armcor> is there any way I can confirm that mongo is running with the recommended ulimit settings? I installed in the following manor http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ and can see the startup script with n open files 64000 etc but that woudl be under the mongdb user right?
[08:06:32] <armcor> you should be able to insert anything into any collection assuming you didnt create an index
[08:08:47] <armcor> right there must be some kind of validation going on with C#, Im not familiar with how C# does things though
[08:23:11] <joannac> Ravenheart: http://docs.mongodb.org/ecosystem/tutorial/serialize-documents-with-the-csharp-driver/#ignoring-extra-elements
[13:03:20] <kenalex> i am new to mongodb and want to know what type of applications it is suitable for ?
[13:24:21] <jonasliljestrand> hey, im just curious and looking into http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/ and came up with a question, is it possible to store getNextSequence like a stored procedure?
[13:35:54] <kali> check out this: http://docs.mongodb.org/manual/tutorial/store-javascript-function-on-server/
[14:20:25] <jonasliljestrand> kali: thanks, one more question. Are these functions available for drivers etc?
[14:27:45] <kali> jonasliljestrand: you really want to do that ? it's actually not a good idea, imho
[14:29:00] <jonasliljestrand> just considering options, so i guess premutation is best for performance
[14:31:03] <kali> jonasliljestrand: well, let me say that: autoincrement are generally not a good idea, as it will lead you two peform two round trips for each insertion
[14:31:23] <kali> jonasliljestrand: even if you store the autoincrement logic server-side, you still have to do two round trips
[14:32:59] <kali> jonasliljestrand: nope, same latency, you can not make mongodb run the "stored procedure" and use its result in the same call, so there really is no point
[14:34:41] <kali> jonasliljestrand: the database subsystem is the part of the architecture which is harder to scale up. so anything that can be done application-side should be done there
[16:08:21] <whyhankee> hi, i'm trying to install mongo 2.4 (yes, we need to upgrade) on amazon linux. The mongo-10gen should be installing 2.4.12 but is installing 2.6.x. How do i get 2.4.12 installed?
[16:26:19] <salty-horse> hey. I'd like to restart a shard server (no replica set). If I shut it down gracefully and start it again, will the config servers handle it nicely, or should I restart them too?
[17:06:48] <blizzow> I dropped some databases from my sharded cluster. when I do show dbs, the databases are gone. When I do an sh.status(), a couple(but not all) of the databases are still listed. Anyone here know how to truly drop them?
[17:17:49] <till_varoquaux> I am sure this comes up semi regularly but is there any unofficial documentation on mongo 2.6's wire protocol? I tried reading PyMongo's source code but this looks like it would take me a fair amout of time.
[19:32:35] <GothAlice> till_varoquaux: For the most part it's variations on a theme: http://bsonspec.org/spec.html
[19:34:10] <GothAlice> till_varoquaux: https://github.com/mongodb/mongo-python-driver/blob/master/pymongo/helpers.py#L76-L119 is the magic that handles server responses. https://github.com/mongodb/mongo-python-driver/blob/master/pymongo/message.py is the message packing code.
[19:34:47] <GothAlice> Alas, because this is a binary wire protocol, you're going to want to be intimately familiar with Python's struct module. (The "simple" interface to packing/unpacking C structs.)
[19:36:04] <GothAlice> till_varoquaux: http://docs.mongodb.org/meta-driver/latest/tutorial/ — the links on this page direct you to a few resources (wiki, doc page, and video) on how to "write your own driver".
[19:41:08] <harttho> error creating initial database config information :: caused by :: can't have 2 databases that just differ on case
[19:47:08] <GothAlice> Indeed, then this is very confusing. Is it viable to attempt to --repair your dataset? (I.e. do you have free space equal to your real data size + ~2GB?)
[19:47:45] <GothAlice> Sounds like MongoDB has allocated a namespace for it, but didn't get far enough to actually allocate the stripes. :/
[19:52:29] <GothAlice> Aye, so lesson learned: database names (and collection names!) are treated as case-insensitive for the purpose of collision detection.
[19:52:56] <GothAlice> Purposefully having dbs/collections that differ only in case will lead to madness, thus it's a no-no. ;)
[19:53:10] <mike_edmr> hey GothAlice, question: is there any "best practice" for running GIANT queries (which modify or delete large numbers of documents) in a way that doesn't slow down the normal database load? I.e. some kind of low-priority query execution?
[19:53:58] <mike_edmr> I often have changes I need to make that could be done in a single query, but for performance reasons i need to build scripts that do it programmatically at a speed that doesn't slow the rest of the database requests down
[19:54:08] <GothAlice> mike_edmr: I generally chop those types of updates up into discrete chunks that can be trickled out, allowing things to get time between write locks.
[19:55:03] <GothAlice> Don't worry, the new back-end engine coming out Real Soon Now™ should nearly eliminate lock contention—it has per-document locks instead of collection locks.
[19:55:07] <mike_edmr> I have been talking to my PO about the need for some kind of generic low-priority ad hoc query interface
[19:56:39] <mike_edmr> we tried upgrading once and performance went through the floor... tons of queries were waiting on the write lock
[19:57:02] <GothAlice> It pains me when I see people running old versions. Esp. when it's so easy to look on JIRA and see: "since the version we're running X security bugs have been fixed, Y performance optimizations were implemented, and Z data-eating bugs were squashed".
[19:57:51] <GothAlice> mike_edmr: One typical culprit in sudden performance tanking is in the changes to how MongoDB's query analyzer examines indexes for selection. Often after an upgrade from 2.4 to 2.6 one needs to begin "hinting" some queries.
[22:05:19] <mbc> got a very simple query on a large collection that's taking forever under mongn 2.6.5
[22:06:09] <mbc> the query is db.history_events.find({account_id: ObjectId(...)}).sort({created_at: -1})
[22:06:39] <mbc> what would be the possible reasons for something this simple completely blocking the read lock of the db and getting stuck forever?
[22:20:31] <till_varoquaux> GothAlice: Thank you. I had that part implement I just wasn't sure how to implement the new 2.6 bulk commands but I think that I hit the jackpot:
[22:21:06] <till_varoquaux> Between that and reading pymongo's implementation of getLastError I think that I am good.
[22:56:02] <bpap> if I use a hashed shard key and a 1GB chunk size, will latest mongo allow me to ENABLE sharding on an 8TB collection?
[22:56:41] <bpap> the docs give an example of 200 byte shard key size having a 5.3TB limit, but i can't find any specific formula.
[23:04:46] <JamesHarrison> mongo seems to be picking the wrong index given two composite indexes a: 1, b: 1 and a: 1, b: -1 (eg, for queries sorting by b -1, it's using the 1 and nscanned is consequently a tad high) - anything I'm doing wrong here?
[23:06:30] <Boomtime> sort order does not matter on the final field of an index, it can go in either direction and be exactly as efficient