Log file Viewer
#mongodb logs for Tuesday the 7th of October, 2014
[03:16:34] <donCams> hi! say I have a collection with the following data {skills: [{name: 'foo', rating: 9}, {name: 'bar', rating: 8}]} and I want to query all documents where skill.name = 'foo' and its rating is greater than 5
[03:37:44] <jkretz> i've just watched a webinar on dealing with time-based data in mongodb, and wanted to try to wrap my head around a few concepts if anyone is game
[03:39:12] <jkretz> essentially, the gist of the webinar was that inserts are expensive, updates are cheap - so instead of having a document per time-based record, it is smarter to have a document that spans some amount of time, and then update that document as new data comes in
[03:40:17] <jkretz> the example in the webinar setup a document that was storing hourly data - they had the data nested based on minutes 0 through 59 within the document
[03:40:50] <jkretz> i'm working with data that is very sparse, but i don't want to inherently lose time resolution if there were dense data in the system as well
[03:41:55] <jkretz> so i was thinking of designing my documents to be daily, based on hours, with each hour then having nested minutes, and each minute having seconds - but this seems like an absurd amount of overhead
[03:42:30] <jkretz> i guess i'm just curious if there is a known model that gracefully handles both dense and sparse time-based data
[03:44:03] <Boomtime> "inserts are expensive, updates are cheap (if the document does not grow in size nor change any index entires)"
[03:44:48] <Boomtime> if your document grows larger as a result of the update then it's disk location might require a move, that's a penalty that insert always takes
[03:45:03] <Boomtime> if your document changes index entries then that's another penalty that insert always takes
[03:45:35] <jkretz> hrmmm, pardon me as this is very much my first experience with mongodb - when you say the document grows, what do you mean? i'm appending data to the thing so i would assume it would grow...
[03:45:44] <Boomtime> if your update causes it to grow larger AND change index entries then it's basically an insert
[03:47:55] <jkretz> so is there anything i should be doing to design around the problem of the document moving?
[03:48:15] <Boomtime> if your document moves on disk it is possible for the penalty of the move to exceed that of an insert - though usually negligible difference
[03:51:04] <Boomtime> ok, powerOf2Sizes is enabled by default then, document resizing will affect you less
[03:51:34] <Boomtime> however, if your documents change size significantly, as a percentage of their total size, then document moves still matter
[03:52:18] <Boomtime> one thing you can do is initialize your documents at a fair size of their eventual total, by creating all the fields/subdocuments you'll ever have
[03:53:11] <jkretz> so essentially if i create the document with placeholder data for everything, i don't have to sweat the moves
[03:54:37] <Boomtime> if your placeholder data produces a document exactly the same size as a document will all the data filled in, then it will never move
[03:55:19] <jkretz> i should be able to do that - this is sensor data that is regularly the same size within any given time window
[03:56:49] <Boomtime> since the space that is not in use as a result of the document reducing in size is not recycled until the document is deleted
[03:57:12] <jkretz> so, say i'm expecting 24 records within a day (hourly reporting) - i prepopulate the document with dummy data of the same size - but only receive 12 updates
[03:57:33] <jkretz> at the end of that window, removing the 12 original records of dummy data from the document is ok?
[03:58:05] <jkretz> ok - cool - thanks again, just trying to wrap my head around the fundamentals here
[03:59:44] <jkretz> databases are not my area of expertise, i knew just enough to be a functional idiot in with sql - which i'm assuming will lead me astray more than once in the mongo world
[04:00:06] <jkretz> want to get started doing things the mongo way, rather than just translating sql and crying about bad performance
[04:03:45] <william_du> how to obtain individual elements of an array (elements are embedded document) ?
[04:10:40] <joannac> actually, maybe the answer is http://docs.mongodb.org/manual/reference/operator/projection/slice/
[04:13:04] <Boomtime> william_du: do you want to know how to query for the presence of such a document based on the content of the array, or to return just an item from an array?
[04:17:18] <william_du> yes , the $elemMatch & $slice what is tried, but the result is expected for me
[04:19:53] <Boomtime> also: "and conditions is (people_id : 1 & privilege : 0)" <- typo? should be privilege 5 right?
[04:56:50] <arussel> is there a way to check for the presence of an property that does not match a value ?
[05:11:01] <Boomtime> @arussel: no, you can't really do that - it would require a query on field names, rather than values - if you find yourself needing this, you probably have a schema problem
[05:11:38] <Boomtime> you could achieve what you want in the aggregation framework, if you don't mind a collection scan
[05:12:08] <Boomtime> (there is no index that could help you do what you want, you need to check every document directly)
[05:21:38] <no_mind> I am using Mongoose but when I ry to search in an array of objectId I do not get anresponse. Bu the same query on terminal gives correct response. What could be wrong ?
[05:22:52] <Boomtime> sounds mongoose specific, if it works in the shell (which is an excellent way to test by the way)
[05:52:42] <arussel> Boomtime: thanks, I'll just use a function (I'm migrating data, this is not a 'normal' call)
[06:48:45] <kakashiAL> hey guys, I am very new in mongoDB, installed my server and created a new collection with an object in it
[07:30:03] <kakashiAL> mongod is the service, but if you mean there is an editor called like this, I will see what I cna find
[07:30:51] <Boomtime> mongod is the reader/writer of these files, modifying them externally is fraught with danger
[07:31:44] <Boomtime> maybe if you say what it is you want to do, we can recommend a client/gui program that does what you want
[07:34:01] <Boomtime> so, robomongo is just a client, it connects to a mongod and provides you the access you a GUI like you see in screenshots here: http://robomongo.org/
[07:56:16] <krion> for a two nodes replicaSet, what's the best way to force a node to become primary ?
[09:09:48] <krion> looks like there is plenty of options available as i read, you could directly rsync data
[11:33:07] <krion> i answer to myself: Hidden members, delayed members, and arbiters all have priority set to 0.
[12:01:37] <testerbit> I'm getting a couldn't connect to server error with mongo and mongotop even after a system reboot in ubuntu. How can I start mongodb or troubleshoot this issue?
[12:26:37] <ssarah> hei guys, im trying to start a: var rst = new ReplSetTest( { name: 'rsTest', nodes: 3 } ); rst.startSet()
[12:36:29] <quattr8> I’m trying to create an index with TTL but getting this error since i upgraded mongodb to 2.6.4 -> bad index key pattern { 0: "created" }: Unknown index plugin 'created' on the PHP driver
[13:10:14] <ssarah> Guys, what I am founding is that you run "mongod --parameters" in one shell, then connect to that from another shell, define the rs and initiate it.
[13:10:44] <ssarah> but this way they all die when the shell dies. This is probably an easy thing, can ya get me a reference in the manual? =P
[14:03:35] <ssarah> cheeser: if i start it with mongo & then i cant issue commands inside it, i'll have to connect remotely, right?
[14:24:49] <berkley> I'm very new to Mongo, and I'm curious: if I'm building a database of archeological artifacts including loads of metadata plus pictures and/or 3D scans, do I store those pictures/3D scans as BSON, or do I store a reference to where they are stored on disk?
[15:46:24] <dgarstang> Is it possible to connect to mongos locally as the admin user without a password, even after one has been set?
[15:48:20] <dgarstang> i want a local script to add users, which runs againat localhost, but I have to put the admin password somewhere. Duh mongo
[18:35:45] <flatroze> Hello. I'm having trouble iterating through collections with native mongodb npm (not mongoose). I have a list of collections to go through and a `for()` loop with the find() and toArray() within it's block; but the collection_name variable behaves very strange with .toArray()
[21:09:42] <mst1228> hey, i have a model and one of it's fields is an ObjectId relating it to another model. how can i use the $nin operator with ObjectIds?
[23:28:52] <hydrajump> I have a 3 member replica set and I want to migrate the data to 3 new instances. If I add 3 new members to the existing replica set, waiting for replication to complete and than promote a new one to primary and remove the old 3 members is this the best approach without resorting to downtime?
[23:35:21] <Boomtime> "beat approach" is a subjective term, the method you describe is the easiest and probably safest
[23:39:10] <hydrajump> Boomtime: how can I ensure that the new members have been replicated successfully and that the data is consistent with the old 3 members before I demote them and stop them?
[23:40:51] <Boomtime> the new members will only enter SECONDARY status once they have replicated fully and closely caught up
[23:41:21] <Boomtime> once the new members have alkl entered SECONDARY status, set one of them to a higher priority (reconfig)
[23:41:52] <Boomtime> the replica-set will elect the higher priority member to be PRIMARY once it is able to
[23:49:52] <hydrajump> Boomtime: ok and if the old replica set is on 2.4.4 which I was told has replication issues how will I know that the data will replicate successfully to the new 2.4.11 members? Is there any comparison commands I can run between old and new?
[23:51:35] <Boomtime> for that version of mongodb you should probably add one new member at a time and wait for it to transition into SECONDARY
[23:52:43] <Boomtime> replication will be fine, it works in 2.4.4 as it ever did, but there some problems surrounding elections and when members can vote etc
[23:53:04] <hydrajump> ok and the mongod.conf needs to be identical between all 6 members, e.g. dbpath, log path etc?
[23:54:56] <Boomtime> network speed and latency, hard-drive speed, ongoing system load, number of indexes...