Log file Viewer
#mongodb logs for Tuesday the 14th of October, 2014
[00:29:06] <Synt4x`> back for a question, if my id: {name, score, date} , and then I do $first : 'OtherOption' , it will grab the first OtherOption per unique {name, score, date} pair correct?
[00:37:59] <Synt4x`> my thoughts on $first were correct, it will grab the OtherOption field from the first {name, score, date} pair it comes across
[00:40:46] <diegoaguilar> I have a question about auth setting up, I followed this tutorial straight forward http://docs.mongodb.org/manual/tutorial/enable-authentication-without-bypass/
[00:40:53] <Boomtime> the first that "it comes across" is exactly right, unless you have previously specifically a sort order (in a previous aggregation stage for example) you won't know which document happens to be first
[00:42:20] <diegoaguilar> but I still can log in to mongo by simpy typing "mongo" from the linux shell
[00:43:53] <diegoaguilar> so my question now is, the admin user with the userAdminAnyDatabase can actually do anything on any DB
[00:44:27] <diegoaguilar> if I want to add any user I have to auth as that user and then create a new user for a certain database with the proper roles
[00:53:32] <diegoaguilar> Im having problems with the auth thing :( I logged to mongo then typed "use admin" "db.auth(user,password)"
[00:54:03] <diegoaguilar> when trying to switch to other DB and show collections it'll fail was I have no permission, but it's the userAdminAnyDatabase user!
[00:54:50] <joannac> it does not inherently give you privileges to read/write, however you can make yourself a user with whatever privileges you want
[01:07:58] <skaag> is there a .config thing to set the default mongo port for when I run the mongo shell?
[01:11:16] <Boomtime> that is an interesting question.. the shell has some startup scripts but i'm not sure if you can change the default port via that mechanism - what OS are you using?
[01:22:44] <joannac> I already told you how to do it in the mongo shell. you can use any other unix epoch timestamp -> normal date converter too
[02:07:17] <morenoh149> anyone know how amazon's dynamodb (or any nosql) has the ability to scale horizontally without rebalancing?
[02:18:10] <locojay> hi i have been deleteting data from gridfs using python delete. now i have allot of orphan chunks
[02:20:43] <Boomtime> when you say "using python delete", do you mean the delete method of a gridfs object, or something else?
[02:20:47] <Boomtime> i.e do you mean this: http://api.mongodb.org/python/current/api/gridfs/#gridfs.GridFS.delete
[02:24:53] <Boomtime> "locojay: es the delete method" <- this confirms nothing, both options are a 'delete' method
[02:25:21] <Boomtime> you will not see the size going down in "show dbs" no matter how many deletes you do
[02:26:26] <Boomtime> it basically only goes up, unless you actively do something to reclaim disk space
[02:26:56] <Boomtime> it is telling you the disk space usage, and it's only a guess anyway - you should use db.stats() to get some accurate stats
[02:27:25] <locojay> been doing manually a fs.files.findOne({"_id":someval}) where someval is files_id of fs.chunks and found some orphan. are these really orphans? can i just delete them?
[02:30:26] <Boomtime> check in that to see if what you think is true - gridfs is just a wrapper to those collections so manipulating them manually is doable, just not advisable
[03:08:17] <morenoh149> it's called consistent hashing http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf
[05:13:53] <eric_lagergren> Okay, I'm stuck again. I'm using Mongoengine to query my database. But right now if I use `MyDoc.objects.get(state='WA')` I get an error nothing was found, even though every entry in my db has a "state" property, and half are "WA"
[05:15:20] <eric_lagergren> Where would I start debugging this? I can't find any results, even though I easily insert data earlier and I see the data using the CLI application.
[05:18:37] <eric_lagergren> joannac: yeah, I can run `db.collection.find({state:'WA'});` and it gets results.
[05:19:56] <eric_lagergren> I'm writing a portion of a larger codebase -- how would I double check this?
[05:21:38] <eric_lagergren> I was also wondering if there were any constraints on objects.get(). I couldn't find any in mongoengine's docs, so I'm assuming there isn't
[07:43:48] <dbousamra> I have Events. They have snapshots inside of them. I want to add a snapshot to the correct event based on whether that event was no longer than 5 mins ago, and the events last snapshot was less than 1 min ago. Otherwise, I want to create a NEW event. Here is my query: https://gist.github.com/dbousamra/85f2053fab77f3eb1cee
[07:48:47] <Boomtime> dbousamra: the action of findAndModify is always atomic, however, there are multiple ways your command can complete successfully without ever breaking the atomicity
[07:54:09] <Boomtime> if you can guarantee the find conditions match one and only one document then your command will behave the way you want
[07:57:21] <Boomtime> it is important that the result of the update still matches the find conditions, if not then the same findAndModify command racing from a concurrent thread would invoke the upsert path instead
[08:10:29] <foofoobar> Hi. I’m using mongodb with php. I can insert to collection „foobar“ with php without any problem. Now I have this command: $ret = $collection->update($crit, $structure, array('upset' => true, 'w' => true));
[08:10:56] <foofoobar> $crit does not match because the collection does not exist yet, $structure is just some data
[10:28:24] <arussel> joannac: we already got solid cloudformation script, db running in production without problem, tested backup scrit, not sure I want to move all this somewhere else
[10:35:10] <joannac> i would say memory usage, cpu, disk usage, opcounters, lock%, queues, bgflush, replication stats
[11:22:48] <ssarah> hei guys, basic question i think. So i load a document into a var x, i alter some of x fields, and now would like to update the collection with my altered version of the document.
[11:23:50] <cofeineSunshine> there is difference between updating whole document and updating fields with $set operator
[11:28:53] <ssarah> cofeineSunshine: I could use set, yeh. But can't I just use my altered doc and replace the old one?
[11:35:22] <ssarah> db.products.update(db.products.findOne( { _id : ObjectId("507d95d5719dbef170f15c00") } ),x) <- like so, hei?
[11:50:45] <kakashiA1> I dont understand why require('../models/person'); stands alone and why I need it
[11:50:47] <kakashiA1> require() need modules.export but in my schema file there is ony mongoose.model()
[12:16:43] <ssarah> ty. How about if i just want to find all that have a certain field? like limits.voice.units:*
[13:45:22] <ssarah> hmmm, db.products.update( { for: "ac3" } , { $inc: { "price": 3 } } ) only increases the price of the first object, why is this? (for is an index)
[13:49:22] <skot> see the docs: http://docs.mongodb.org/manual/reference/method/db.collection.update/
[15:05:32] <hydrajump> I have 3 members in a replica set. Looking at this link http://docs.mongodb.org/v2.4/tutorial/backup-with-filesystem-snapshots/ if I want to make backups do I run `db.fsyncLock();` on one of the secondary members?
[15:06:36] <tscanausa1> saml: i am trying to find definitive proof but show dbs, I think show the on disk size not the actual size
[15:11:55] <tscanausa1> and objects are not put in the same spot on disk with 2 different servers. think if it like disk fragmentation
[15:26:01] <hydrajump> so it looks like I need to add a 4th member to the replicaset that is "hidden" and "priority=0" and use this one for backups, e.g. mongo shell `db.fsyncLock();`, make EBS snapshot, `db.fsyncUnLock();`. Can someone please confirm if this is the way to do it?
[15:29:36] <hydrajump> the following in the docs is not clear to me "Hidden members do vote in replica set elections. If you stop a hidden member, ensure that the set has an active majority or the primary will step down.
[15:29:38] <hydrajump> For the purposes of backups, you can avoid stopping a hidden member with the db.fsyncLock() and db.fsyncUnlock() operations to flush all writes and lock the mongod instance for the duration of the backup operation.
[15:30:54] <hydrajump> so if the 4th member is hidden and p=0 than it can still vote which is an issue with an even number of members, right?
[15:31:14] <minioak> Hi - question about HA clusters. What is the typical reason why you wouldn't want a member in a secondary data center to become a primary?
[17:20:41] <culthero> I have a sharded collection across 4 shards that has a TTL index, but the TTL index is not expiring data. Any suggestions as to why this would happen?
[17:55:38] <edrocks> is it better to store arrays on a user document then to just store the items in their own collection? They would never outgrow the user document btw
[18:20:56] <hydrajump> in a 3 member replicaset. Do I need a 4th "hidden, p=0" member to execute db.fsyncLock() for EBS snapshots?
[19:42:04] <desmondmorris> Is it possible to pull specifc versions of a document using its version key (__v)?
[19:44:23] <hydrajump> when you run db.fsyncLock() on a replicaset member does that mean that the PRIMARY is not able to receive reads/writes or is it the SECONDARY that won't replicate during the Lock?
[21:45:08] <James1x0> Anyone want to take a stab at a mongoose population issue? https://stackoverflow.com/questions/26370673/mongoose-population-myproperty-mymodel
[21:52:55] <zacaj> using nodejs and the regular mongo driver, is there any way to query for a single document and just get back that document that doesn't involve callbacks
[21:54:15] <zacaj> it's really annoying having to do a giant db.collection('name',{options},function(err,collection){collection.find({query},function(err,item){look I've got the item})})}); and indent all my code even further every time I want to look up a single thing
[21:56:27] <Boomtime> have you considered using a different language? because what you describe is just Node.js
[21:59:24] <Boomtime> http://docs.mongodb.org/manual/core/backups/#mongodb-management-service-mms-cloud-backup
[22:00:29] <hydrajump> Boomtime: "MMS continually backs up MongoDB replica sets and sharded clusters by reading the oplog data from your MongoDB deployment.
[22:03:27] <hydrajump> cheeser: ok. I'm just trying to understand how it is done. Can't use MMS because of compliance so have to do it ourselves
[22:06:03] <Boomtime> hydrajump: http://docs.mongodb.org/manual/core/backups/#mongodb-management-service-mms-on-prem-backup-software
[22:10:26] <hydrajump> sure. I'm interested in doing it without purchasing the enterprise subscription for now. Whatever the MMS backup agent is doing must be possible with the mongo tools or?
[22:15:39] <hydrajump> ok so let me ask a different question if I'm not trying to replicate MMS backup.
[22:16:47] <Boomtime> do you really need point-in-time restore or is that just a dream you have? for example, are 24 hourly backups sufficient?
[22:17:38] <hydrajump> I have many dreams one of which is to lose the minimum amount of data in a catastrophe ;)
[22:18:21] <hydrajump> I've been reading the docs on backups/restore and it's not entirely clear to me what impact there is on a 3 member replicaset when db.fsyncLock(); id run.
[22:20:08] <hydrajump> Boomtime: so if I run that on a SECONDARY that member won't replicate during the lock, but the PRIMARY will still be able to receive reads/writes and the 2nd SECONDARY will replicate?
[22:20:52] <hydrajump> and mongodump effects a whole replicaset or it works in the same way but without requiring locking?
[22:22:00] <Boomtime> mongodump is basically a client that connects to a database and extracts everything that you ask for
[22:22:59] <Boomtime> mongodump requires no locking because it's just a client, but equally it does not produce a point-in-time
[22:24:30] <hydrajump> Boomtime: ok so when it connects to a database member in a replicaset, any reads/writes to the PRIMARY and replication are not effected by the dump, they will continue it's just that they won't be part of the dump unless --oplog specified?
[22:25:59] <Terabyte> hey, suppose mongodb goes down, and comes back up again, is there anyway to get the clients to retry connecting? using java (spring)
[22:26:19] <hydrajump> ok and I assume that mongodump as it connects as a client impacts performance versus looking a SECONDARY and doing a snapshot. That's why snaphsots are preferred?
[22:28:36] <Terabyte> "There is no replacement for this method. Use the connectTimeout property to control connection timeout." not clear on how connectTimeout should be used to be equivalent to autoConnectRetry..
[22:30:34] <joannac> Terabyte: http://docs.spring.io/spring-data/mongodb/docs/current/reference/html/
[23:00:09] <heath> anyone in here aware of a way to convert ObjectId('<24bitencoding') into just a string with pymongo?
[23:06:53] <Boomtime> http://api.mongodb.org/python/1.7/api/pymongo/objectid.html#pymongo.objectid.ObjectId
[23:07:58] <hydrajump> before running db.fsyncLock() on a SECONDARY should there be a check to see that that SECONDARY has replicated all the data from the PRIMARY or that implied?
[23:09:10] <Boomtime> hydrajump: i would certainly recommend checking that the secondary you intend to copy from is not too far behind
[23:16:32] <hydrajump> Boomtime: is rs.status() and comparing the optime between the PRIMARY and SECONDARY the way to do this or is there a quicker way?
[23:18:36] <heath> https://gist.githubusercontent.com/heath/1abdcff045f992e9b888/raw/47cf1952d140d3b96e26c37872017849f1118470/gistfile2.txt
[23:18:39] <Boomtime> hydrajump: that is fine, however, if the difference is large, you should wait a moment and check it again
[23:20:22] <heath> i keep reading http://api.mongodb.org/python/1.7/api/pymongo/objectid.html#pymongo.objectid.ObjectId as bson.objectid.ObjectId.str
[23:36:26] <Boomtime> if you happen to query right after the primary accepts a write (after many minutes of nothing) then the secondary might appear to be lagging behind a long way (even though it is only behind by 1 write)
[23:39:39] <hydrajump> Boomtime: ok I understand. I'm looking in the docs now for how to script this from another server.
[23:40:39] <hydrajump> Boomtime: regarding verification. If I do hourly snapshots would running verification once a day, e.g. running db.collection.validate(true)
[23:41:57] <hydrajump> on the last days snapshot ok or would I need to run it hourly? I don't know if that's the only way to be truly confident that the snapshots are ok.
[23:45:39] <Boomtime> hydrajump: ultimately, running validate(true) is the most assured way of knowing that a snapshot is fully functional - it walks the entire dataset - unless you do this you can't know for absolute certain
[23:47:23] <Boomtime> with that said, obviously you may decide that your confidence levels will be high enough by checking every Nth backup rather than every single one
[23:48:32] <hydrajump> Boomtime: yeah but if I only verify the Nth backup and I need to restore maybe the N-1th than I won't know if that one is fully functional until I try.
[23:49:58] <hydrajump> so you've pretty much answered that verifying each hourly backup is the only way.
[23:54:10] <Boomtime> hydrajump: correct, if you need guaranteed 0% risk then you should check every backup