Log file Viewer
#mongodb logs for Friday the 24th of May, 2013
[00:03:35] <iNick> redsand: understood. so you're not sure where the replset info is physically stored. no problem :) I can recreate the replset, and import the main db
[06:32:52] <TTilus> a db design consideration: say ive got book, which has embedded chapters and i now want to add themes, which would reference book and have many-to-many with pages
[07:32:59] <groovemonkey> I have a question about deleting a sub-object nested in an array -- could someone help me?
[07:38:41] <resting> how would i find all documents where packets[0] = 0? with data structure packets: { "0" : "0", "1":"0", "2":"0"….}
[07:39:17] <Gargoyle> Do you need the $ when using $pull. Eg, can you use questions.text instead of questions.$.text
[07:52:30] <Nodex> if you match it on _id for example and the array is an object {} rather than array [] then you dont need it else you do
[08:51:38] <remonvv> Anyone opinions about the ideal amount of (max) connecties from app servers to mongos's?
[08:52:24] <remonvv> We're seeing a rather spectacular degradation in performance when configuring a high (100) number of connection to mongos
[08:55:54] <TTilus> a db design consideration: say ive got book, which has embedded chapters and i now want to add themes, which would reference book and have many-to-many with pages
[09:02:09] <TTilus> coming from rdbms world my first reaction would be to extract chapters from book and introduce many-to-many between chapters and themes
[09:05:55] <TTilus> i would never need to look up for themes or chapters, but always find a book and a chapter inside it and then themes of that chapter
[09:10:17] <crudson> TTilus: if you're dealing with books and chapters, it sounds like separate entities may be worthwhile due to size. Your themes seem to be independent of books, keep them separate too if they can apply to many books. Not totally sure I understand your datamodel...
[09:12:11] <crudson> Gargoyle: sure, but you want it to be efficient across the wire too without having to babysit every query.
[09:12:47] <TTilus> Nodex: in this context you can think of it being "things relevant to particular audience"
[09:13:12] <Gargoyle> crudson: That comes down to app. I would hope that it goes without saying, that if you want a list of just titles, you don't load everything from every doc!
[09:34:15] <jgorset> MongoDB is not supposed to allow two keys by the same name in a single document, right?
[09:36:19] <jgorset> I see, okay. I've found two keys by the same name in my document, and it's wreaking havoc on my ORM (Mongoid).
[09:37:01] <kali> yeah. with ruby, there was an issue where you could write { :a => 12, "a" => 42 } and save this
[09:37:31] <kali> the driver was converting symbols to strings, and so storing a document with two key named "a'
[09:37:31] <jgorset> I'm not sure how they came to be, though, because I can't seem to be able to create a document with duplicate keys in the mongo shell.
[09:54:11] <jgorset> I can't find anything about that in the MongoDB or Moped release notes, kali, but it makes a lot of sense.
[10:00:51] <TTilus> Gargoyle, crudson, Nodex: i inherited a codebase which already has books and chapters embedded inside them
[10:02:13] <TTilus> what i need is "themes" or "tags" (which judging by guidance in http://docs.mongodb.org/manual/core/data-modeling/ should be embedded inside book too) which would have many-to-many with chapters
[10:03:53] <TTilus> having a themes field (with list of theme ids) in chapter document would do the job for me
[10:07:09] <Nodex> if each book is a single document with embedded chapters and each "theme" relates to that chapter then you should probably embedd the theme or tag with the chapter
[10:07:31] <Nodex> if you need a list of all "themes" then you should keep a separate document with an array in it that has all the chapters
[10:08:45] <Gargoyle> Although, you can also very quickly build such a list in "application land" if it makes the DB easier to manage.
[10:14:21] <majoh> hey, I'm trying to do a aggregate on 2.2.3, I have an array that I want to project, but only the first item in that array. I've tried {$project: {"interesting": "$thearray.0"}} etc... any ideas?
[11:35:32] <remonvv> NodeX, sorry, was afk. It's both degraded and drivers occasionally complain about not being able to establish connections.
[11:36:50] <remonvv> It seems to be that the more connections you have to mongos (and thus the shard mongod's) the worse performance gets. Might be a Java driver issue as well since the network code in that won't win any prizes either.
[11:54:08] <Nodex> `3rdEden : New laptop = don't have a choice as I am not paying for another winblows licence
[13:23:16] <richthegeek> I'm having something of an issue with read speeds - I have a collection with ~200k rows in it, and I want to know the ID of all rows in the set. Unfortunately, the list is only building it about ~500 rows per second. This seems very slow?
[13:24:14] <richthegeek> the bit inside is just adding the result to an array, so it seems to be a read issue
[13:27:29] <richthegeek> ok, just used the aggregation framework ({$group: {_id: null, _ids: {$push: '$_id'}}}) and it got the result in ~300ms
[13:30:24] <richthegeek> so... what is the /disadvantage/ of using the aggregation framework? seems like it's hella faster?
[13:33:01] <richthegeek> it's off a 7200rpm HDD with an i7 and 8gb of ram so it's not exactly server spec but nor is it slow
[13:36:32] <richthegeek> performance seems to have degraded to about half of what it was earlier in the week :s
[13:44:23] <richthegeek> which is 1382 (read from orders, do some processing, write to orders_big, write to track) per second
[13:54:24] <ThePrimeMedian> morning everyone. i've googled myself out - thought I would ask someone in here: Anyone done an activity collection? (like facebook's feed) or something of the sort? I am having a brain-fart and cannot wrap myself around the db design (like do I enter raw data or references, etc)
[13:57:22] <ThePrimeMedian> Nodex: what if the person's name changes? or something important in the data changes?
[13:58:46] <ThePrimeMedian> my thought would be make a template field "mark follows %s" and then have a data field
[14:24:41] <theRoUS> i have a mongodb instance with dbpath=/srv/mongo and directoryperdb=true. /srv/mongo is a mounted filesystem, so it has a 'lost+found' directory. mongod is crapping out on startup because it's trying to treat that as a database directory.
[14:25:12] <theRoUS> is there any way to tell mongod to ignore lost+found? or do i need to go to directoryperdb=false (suboptimal) ?
[14:47:08] <starfly> kfox1111: you could use a "driver" collection that the tailers check regularly and persist which collection they should use. When you want them to switch to a new capped collection, update the driver collection to indicate that.
[14:49:34] <theRoUS> is there any way to tell mongod to ignore lost+found? i mean, having its dbpath be the top of a filesystem can't be that uncommon..
[14:50:37] <starfly> theRoUS: can't you mkdir -p /srv/mongo/data and use that instead for the mongod dbpath?
[14:52:24] <orospakr> data model question: in order to avoid classic N+1 problems, is doing aggregation of (similar in notion to a SQL join) the related documents together by means of a incremental map/reduce to a "join" collection reasonable? eg., in couchdb, I'd do it with a view in my design document.
[15:15:08] <starfly> theRoUS: maybe just create a symbolic link to such a directory, e.g. mkdir -p /srv/mongo/data (as mongo user); cd /; ln -s /srv/mongo/data mongodata (as root user) -- then use /mongodata as dbpath and change puppet once, when/if you need to relocate your MongoDB data directory in the future, take an outage, move MongoDB files to the new location, modify the symlink, and restart mongod.
[15:16:47] <theRoUS> double_p: it's assuming that every directory in dbpath represents a database, because of directoryperdb
[15:17:37] <double_p> yeah, figures. but, is that only a startup error (fatal) or is just croaking and the moving along?
[15:18:48] <double_p> subdir and/or jira ticket. i mean, almost noone would name his database lost+found? :D
[17:01:45] <maxamillion> why would commands entered into the mongo shell that work throw a syntax error when I try to run them from a .js file?
[17:16:48] <kali> maxamillion: there used to be a page that explain the differences between the interactive shell and the scripting mode
[17:17:20] <maxamillion> kali: ok, so what is supposed to be in it's place? ... or how do I script that?
[17:17:39] <kali> maxamillion: http://docs.mongodb.org/manual/tutorial/write-scripts-for-the-mongo-shell/
[17:41:52] <leifw> bhangm: http://en.wikipedia.org/wiki/De_Morgan%27s_laws#Negation_of_a_conjunction
[17:44:41] <bhangm> hmmm... I suppose you're right, it would only use 1 index in the case of the $and, correct ?
[17:46:57] <kali> i think the optimiser would give up on the $not anyway, and start scanning the collection
[17:53:18] <kali> this does not mean you don't have options. can you show me one example docuemnt and the query you're trying to optimize ?
[18:15:49] <bhangm> kali: The query itself is not the problem, with an index on each of the $or expressions it's able to use them in parallel and comes back pretty quickly
[18:16:22] <bhangm> however, tacking a sort onto that kills the performance since it's almost essentially doing a full scan
[19:20:12] <baniir> I'm getting "WARNING: Readahead for /home/mongod/data is set to 1024KB" though I'm not sure why as I followed the steps at http://docs.mongodb.org/ecosystem/tutorial/install-mongodb-on-amazon-ec2/
[19:56:58] <baniir> yes, as described here: http://docs.mongodb.org/ecosystem/tutorial/install-mongodb-on-amazon-ec2/
[20:16:38] <baniir> blockdev --setra 128 /dev/md0 && blockdev --setra 128 /dev/xvdh1 && blockdev --setra 128 /dev/xvdh2 && blockdev --setra 128 /dev/xvdh3 && blockdev --setra 128 /dev/xvdh4
[20:17:57] <baniir> these are a part of a shell script i pass as userdata to the ec2 instance when it is created
[20:19:12] <baniir> i'm not sure if there were errors, but i've successfully used the same script on other boxes running the same os, ubuntu 12.04
[20:27:11] <baniir> they don't directly map to /home/mongod/data as i'm using lvm as described in the previous link
[23:12:22] <llakey> is there a way to connect to mongo somehow and see all of the commands/queries that are being issued against mongo, something similar to `redis-cli monitor`
[23:37:54] <federated_life> possible to force initial sync from a specific member of a replica set ?