Log file Viewer
#mongodb logs for Tuesday the 13th of January, 2015
[00:35:56] <jrua_> so nobody passed C100DBA here ? What a pity :S, not sure if register for the exam
[00:39:58] <cheeser> for me, the value would be in learning the material not so much padding the resume
[00:56:17] <lqez> https://github.com/mongodb/mongo/blob/master/src/mongo/db/storage/mmap_v1/data_file.cpp#L71-81
[00:57:23] <lqez> https://github.com/mongodb/mongo/blob/master/src/mongo/db/storage/mmap_v1/extent_manager.cpp#L60-80
[02:13:50] <FFForever> What is the best use-case example for mongodb compared to a traditional k/v database?
[02:17:33] <dimon222> basically its mongo is not a great opponent of k/v databases, its just serves its purpose, and does it great
[02:19:26] <FFForever> Would you recommend using mongo for a full stack backend or using it as more of a in-ram cache service?
[02:20:46] <dimon222> mongo is not supposed to be used as in-ram cache service. It mostly relies on hard drive speeds since it usually stores huge amounts of data (when RAM is obviously not enough)
[02:22:25] <dimon222> that may be rough, but i would call Mongo is some kind of solution for non-relational data warehouse
[02:30:25] <FFForever> So if it's for non-relational data I would still be using it in conjunction with postgres or miriadb, right?
[02:33:53] <FFForever> No. It was just trying to figure out what was a proper use case. One moment, let me write up a sample json.
[03:23:04] <pikapp> I can’t seem to connect to mongodb remotely. I am pretty sure my firewall is open is on port 27017. What else could I be doing wrong?
[03:25:18] <cheeser> pikapp: make sure you're binding to either 0.0.0.0 or your public IP and not localhost.
[03:25:22] <Boomtime> pikapp: check if you have bindIp set: http://docs.mongodb.org/manual/reference/configuration-options/#net.bindIp
[03:25:22] <bmillham> And make sure mongo isn't bound to 127.0.0.1, it has to be bound to your actual IP
[03:26:28] <pikapp> Why would it be port 37017? Says here the default port is 27017 http://docs.mongodb.org/manual/reference/default-mongodb-port/
[03:30:09] <pikapp> which is odd because the default is 127.0.0.1 when installing from yum on centos, so you would think one of the 3 tutorials i am using would mention it as a setup step
[03:33:13] <pikapp> it was just leading me down the iptables road and that wouldn’t have been good, so glad I asked here first
[09:11:52] <winem_> good morning. I'm a bit confused and need some help. I downloaded mongodb 2.6.6. tar.gz for linux from the website and tried to add an user with createUser but it says it's not a function. but the deprecated addUser works
[09:40:31] <hmsimha> if you have a field you want to either increment or set to a value if upserted, can that be done atomically?
[09:42:05] <hmsimha> for example if a user is given $100 then I want to either increment that user's piggybank field by 100 or set it to 100 if that user doesn't already have a document
[13:21:54] <swaagie> Are there issues with mongodb 2.6.6 and debian with geoNear queries? Got a script that runs fine on my local ubuntu machine but fails on some debian server claiming the special index: 2d is missing, while my debug statement after ensureIndex returns: Ensured index ran, result: location_2dsphere (no errors
[13:24:42] <swaagie> the query I'm running is just a basic: `{ location: { $near: { $maxDistance: 5E4, $geometry: { type: 'Point', coordinates: [lng, lat] }}}` as its 2dsphere I don't get why it complains about missing 2d indexes in the first place
[13:57:56] <jiffe> how can I tell what each replica set member is synced up to with respect to one another?
[14:01:07] <jiffe> can that be manupulated? it seems that after starting a replica member it crashes soon after it starts to sync. I found a related issue where someone had the same condition and it was based on detecting data corruption
[14:02:17] <jiffe> although they did a full resync to detect the problem which suggests the corruption was on the side of the member that was crashing
[14:03:18] <jiffe> a full resync is really a last ditch option in my case though because there's just too much data and these machines should be mostly in sync already
[14:04:34] <jiffe> I don't know if the comment that mongodb shuts itself down to prevent further damage is correct either
[14:07:05] <jiffe> the optime on the working member is two operations higher than the non-working member
[14:28:11] <stava> Doing db.collection.update({...}, {$pull: {myArray: 'foo'}, $addToSet: {myArray: 'bar'}}) I get the error "Cannot update 'myArray' and 'myArray' at the same time". But how then can I remove "foo" and add "bar" to myArray in a single update()?
[14:31:02] <stava> I thought that I atleast would be able to nest some $-operators to accomplish something like this
[14:35:22] <swaagie> Would anyone know if there are issues with mongodb 2.6.6 and debian with geoNear queries? Got a script that runs fine on my local ubuntu machine but fails on some debian server claiming the special index: 2d is missing, the query I'm running is just a basic: `{ location: { $near: { $maxDistance: 5E4, $geometry: { type: 'Point', coordinates: [lng, lat] }}}`
[14:35:22] <swaagie> as its 2dsphere I don't get why it complains about missing a 2d index in the first place
[14:43:01] <swaagie> would raising the loglevel of mongodb provide more insight? or would that just blur the issue
[14:46:44] <StephenLynx> if you wish to just remove ONE element and add ONE element, this could word
[14:52:23] <StephenLynx> but the overall impression I am getting from searching is that indeed is a mongo deficiency, you can't perform multiple updates on sub arrays in a single query.
[14:54:20] <swaagie> cheeser: got it working, thanks for the help, silly me, appearantly the reinstall did not kill the current older running mongod, somehow the client did connect like it was working normally/regularly tricking me into thinking 2.6.6 was running
[14:55:10] <jiffe> well my crashing does seem to be due to rsyncing, if I shut the other replica member down then the member that crashes doesn't crash until I start the other member up again
[14:55:41] <StephenLynx> swaagie probably it happens because from what I heard linux keeps the files in memory. unlike windows, it won't lock files that are in use by the system
[14:56:47] <StephenLynx> lesson learned: makes sure stuff is shutdown for good when uninstalling/reinstalling :v
[14:57:39] <jiffe> files aren't removed from disk until there are 0 references, they are just unlinked from the filesystem
[15:00:20] <jiffe> if you're talking something like fopen, fread, the read doesn't have anything to do with referencing but the open does
[15:00:38] <jiffe> so while the file is open it is still referenced, but as soon as you close it, that reference is removed
[15:02:09] <FunnyLookinHat> I'm not sure what I should be googling for - feel free to suggest the keywords I'm missing... but I want to create a report that looks for a nested key/value pair within a collection... what would that be called?
[15:02:47] <FunnyLookinHat> i.e. { 'someObj': { 'someKey': 'someVal' } } - I want to look through data on 'someKey' - if it exists.
[15:06:13] <FunnyLookinHat> I wasn't sure if I should be generating a view or something... ( totally new to NoSQL )
[15:08:34] <FunnyLookinHat> StephenLynx, Yeah - this field is for arbitrary data that's specific to the use case... it's a one-off compared to the other top-level fields I have
[15:09:04] <FunnyLookinHat> StephenLynx, i.e. https://github.com/funnylookinhat/tetryon/blob/master/spec/particles.json
[15:11:08] <StephenLynx> I don't try and format my docs. https://gitlab.com/mrseth/lynxchan/blob/master/doc/Model.txt
[15:12:51] <FunnyLookinHat> StephenLynx, better? :) https://github.com/funnylookinhat/tetryon/blob/master/spec/particles.txt
[15:13:29] <StephenLynx> but could be better, the comments seems to be isolated from what they should define
[15:15:54] <StephenLynx> and you are declaring the collection is an array. that should be pretty obvious, so is redundant.
[15:17:10] <FunnyLookinHat> beam_id referenences https://github.com/funnylookinhat/tetryon/blob/master/spec/beams.txt
[15:17:34] <FunnyLookinHat> I can't use the built-in ID as it's not unique enough for my use case ( has to be client generated and nearly-guaranteed unique )
[15:21:31] <StephenLynx> and as I said, I would remove the data field and just add the fields to the object.
[15:23:10] <FunnyLookinHat> StephenLynx, it doesn't help to keep things organized at all? these will be user-customizable - so I was thinking it would just muddy the collections
[15:33:24] <mrx1> I will be storing data logs from different devices (~100 devices) and there will be many data points (let's assume 10M in total). I would like to be able to quickly retrive contents of last data point for each device. How to do it nicely? I could store last record of each device in devices collection in a separate field, but this would require updating 2 fields when inserting records - so it wouldn't be atomic and I would have to mimic transactions. It's p
[15:34:44] <mrx1> "this would require updating 2 fields" --> "this would require updating 2 documents"
[15:35:52] <StephenLynx> then you just get the last element of B where the document matches the device ID you are looking for
[15:36:40] <mrx1> that's my plan currently, but will it be fast enough ? I would prefer some constant-time per device solution
[15:41:12] <mrx1> Not last element of the array, but last element for each device. One of that elements can be actually last in whole collection, but most of them won't be last :)
[15:43:28] <StephenLynx> if you have 1 device, you will read 1 element, if you have 10000 device you will read 10000 elements
[15:43:46] <mrx1> it can when there would be some mechanism to store log for each device during inserting that log
[15:45:38] <mrx1> I can't hold logs of device in one document, because there can be lot of logs per device (more than 16 MB)
[15:46:37] <StephenLynx> yeah, you can have that, but that would be even more expensive, IMO because then you will have to splice and group.
[15:54:02] <StephenLynx> yeah, that seens to be the best solution. use the device id for the _id when grouping
[16:38:54] <StephenLynx> db's are just a really different and constrained way to lay out instructions
[16:39:36] <StephenLynx> when it fails but looks like it it's working only to crumble later and you have absolutely no idea what the problem is then you will say "this is hard"
[17:02:08] <rodd> Hi I'm new to mongo, I'd like to setup a document for countries with lat and long, can I user whatever geospatial index I choose or would it be better to use a specific one, like 2dsphere?
[17:06:13] <keso_> anybody used morphia for java before? I have a question about the UpdateOperations class
[17:10:42] <StephenLynx> rodd you can use anything for index, you just set it as unique. but personally I would use something human friendly
[17:12:12] <rodd> StephenLynx: oh right, but I'd like to be able to detect a user's position through lat and long
[17:12:54] <StephenLynx> if you want to have that everytime you read the user, then yes, I would use that.
[17:13:51] <StephenLynx> if you are ok with making occasional additional queries for that, and want to show something in the GUI regarding the country, then I would use something else.
[17:18:07] <ac360> HI all, I'm a bit new to MongoDB and database infrastructure and I have a fundamentals question. In general, if I have an application running online on a server, I want my Mongo database to be hosted on the same server so that both could communicate quickly without any network latency, correct?
[17:19:31] <ac360> I'm just curious how this works in AWS EC2. If I spin up an EC2 instance with a web application, and then use a managed-mongoDB-provider like MongoLab who claims they can also host the database on AWS to "host the database where my app is".
[17:20:44] <ac360> (Sorry for the vague language. I just don't know a lot about the connection protocol through which apps connect to the databases)
[17:21:14] <ac360> Amazon Web Services EC2 is a cloud computing service which lets you spin up computing instances you can use for launching web applications
[17:21:18] <kali> ac360: they will allow you to pick an ec2 zone, iirc. so the database will not run on the same server, but it will not be very far
[17:21:46] <kali> ac360: it's not that common to have the DB and the app on the same hardware actually
[17:21:53] <StephenLynx> that is very specific to their services. you would get more people to help you contacting them or in their forums
[17:22:10] <ac360> kali: Thanks. So Every query to the database is actually a network request and is subject to network latency?
[17:23:39] <kali> ac360: i'm not sure they allow you to pick a zone or region, but don't worry too much about that
[17:24:03] <kali> i mean. it's either a zone or a region. but definitely one of them. just can't remember which :)
[17:25:32] <ac360> I ask because I'm intrigued by AWS's new Lambda service. It allows me to simply upload javascript functions to the cloud and uses the most minimal amount of computing resources to process it. I'd like to write javascript that connects to a Mongo Database, but I don't know the best way to set up the infrastructure to be as fast as possible.
[17:27:09] <kali> as far as i understand it, lambda is not for low latency problems (like answering http request), more for high bandwith problem (async jobs, generating stats, thumbs, etc.)
[17:27:22] <StephenLynx> ac360 if performance is an issue maybe you could rent servers and handle them via SSH
[17:30:39] <ac360> StephenLynx: That's what I do, but I'm looking to stop paying monthly fees for server infrastructure and instead pay only for the precise infrastructure required when my code is actually run. This is what AWS Lambda represents.
[18:42:34] <jcox1> hey everyone. I'm looking for some advice on configuring replica set members. The docs seem ambiguous regarding what I want to do so I figured I'd check here to see if anyone has experience
[18:43:07] <mrx1> Is it OK to use username as object id in Users collection? I will have only 1 user with any name and I won't change usernames. But that username will be less random then autogenerated ID. Is it okay or will it introduce some performance penalty or something ?
[18:43:41] <cheeser> your ID can be whatever you need it to be. but mongo won't let you change the _id value of a document.
[18:48:49] <jcox1> I have replica sets with three members (1 prim., 2 sec). I want to add a node that I am solely going to use for dumping data into our data warehouse. I'm going to configure this member to be hidden (and priority:0) so it doesn't receive any read traffic. But now I have 4 voting members in the replica set. So I either need to make the new member non-voting or add an arbiter. Ideally I'd just make it non-voting, but the documentation seems to discourage changing
[18:54:17] <jcox1> @cheeser are there disadvantages to having the new member be non-voting? I'll be deploying this setup to many many replica sets across many mongo clusters. So if there's no disadvantage, it'd be cheaper to have the warehouse nodes not vote.
[20:04:46] <dman777_alter> much...not sure how this happened but I have a duplicate collection. In otherwards, I have 2 foos...one is (empty) bytes and the other is 2 gigs
[21:06:42] <digicyc> I can't even figure out how to add a collection with the same exact name of an existing collection. o_O
[21:15:41] <digicyc> http://icephoenix.us/stuff/mongodb-deleting-collection-with-a-space-in-the-name/
[22:18:48] <liamkeily> In my app, there would be some relations. Like a user would own some objects. Is this something you would/could use mongo for.
[22:22:25] <liamkeily> So a query on a collection of tweets for example. Could be queried by user_id in a similar way to mysql
[22:23:20] <liamkeily> cool :) i've always had a bad impression of document databases so i've avoided them for a while lol
[22:29:08] <Torkable> say I want to create a new user, and I need to insert the data in more than one collection, and if one write fails, roll back
[22:32:59] <Torkable> joannac, yea I'm looking at that page now, can that pattern be applied to multiple collections
[22:35:02] <joannac> No, I'm just not sure why you think it's not possible to do across multiple collections
[22:35:34] <Torkable> looked like it was making use of bulk inserts, can't bulk insert on multiple collections
[23:27:54] <Torkable> http://edgystuff.tumblr.com/post/93523827905/how-to-implement-robust-and-scalable-transactions