Log file Viewer
#mongodb logs for Thursday the 12th of February, 2015
[00:39:47] <jayjo> Why is it that my mongo daemon is now starting on port 43980, when I've ran service mongod stop and restarted
[02:38:36] <hackel> Why does the positional $ operator only return one result? How can I filter the contents of an array and include *all* the matching elements?
[02:41:03] <joannac> if you need multiple results from a single array, your array elements are better suited to be top level documents
[02:42:40] <hackel> joannac: I am attempting to filter out subdocuments in an array when deleted_at is not null. I do want them to remain subdocuments, though.
[02:45:52] <hackel> Yes, I could make everything a separate collection, but in that case I might as well use SQL and not have to deal with all this headache to accomplish simple tasks.
[02:55:48] <haole> where do I get a list of the support events in MongoDB's Node.js driver? like 'fullsetup' and 'close'
[03:21:31] <morenoh149> make sure you use the right docs. I got really pissed off when I started out doing mongod from the node driver. I was looking at the 2.0 docs the whole time -.-
[04:40:03] <benson> is it possible to install just mongodump? I want to periodically backup a remote database but dont want to install mongodb on the server doing the backing up
[04:44:17] <Boomtime> but i don't think there is a way to install a single particular tool from the set
[04:45:41] <Boomtime> there might be, that would depend on the status of your destination machine - if you want to be sure you can do the do the fake install trick and determine if it would pull in other packages
[10:30:57] <brokenpipe> I just installed mongoDb and configured /etc/mongo.config file with correct permissions and when I start app does not read this file
[11:14:04] <guest9999> hello. just wondering. why dont mongodb have different yum repositories for previous releases? e.g. 2.4. so when i create and bootstrap new server it doesnt automatically got to 2.6?
[11:14:57] <guest9999> e.g. i can install different php versions like: `yum install -y php53u` with php, php53u, php54, php55u, php56u
[12:37:38] <elshaa> Is there a variable I can check from mongo client to know about the logging and verbosity options ?
[12:46:54] <haole> morenoh149: couldn't find the events - only "events" in the sense of people getting together to talk about mongodb lol
[12:46:56] <haole> where do I get a list of the support events in MongoDB's Node.js driver? like 'fullsetup' and 'close'
[13:02:55] <flyingkiwi> mongod is preallocating disk space (the said 3gb) - so 915mb free disk space is insuffient
[13:53:49] <Sticky> re mongo preallocation, can mongo give any indication of how full its preallocated files are and when it is likely/close to growing?
[13:58:39] <Sticky> mongos inability to defrag when you remove records without doing a full resync is quite annoying. A few times when we have run out of disk space on servers then cleaned the db it was an issue
[15:27:51] <jiffe> cheeser: that is never going to happen and the software ultimately gets blamed for it
[15:31:35] <StephenLynx> the default configurations make is secure so only local connections are accepted.
[15:31:50] <StephenLynx> and they didn't even tested for all those databases to see if they could actually access them.
[15:32:09] <StephenLynx> they were just "oh, it is open, lets assume it is open to anyone to access it"
[15:33:50] <jiffe> the only barrier to stop that access is auth and I'm willing to bet most of them don't run auth
[15:34:38] <StephenLynx> that and not taking external connections. which is turned on by default on install.
[15:35:28] <jiffe> but all those 40000 were accepting those connections, I'm sure they tried to access the db from another machine and when they couldn't they listened on all interfaces and then everything was working so no need to go further
[15:36:19] <StephenLynx> did their test just checked for an open port or it actually tried to connect?
[15:42:18] <Tausen> Hey! I'm a bit puzzled with some timing and hope someone can help me shed some light. I'm using aggregate through pymongo and filtering with $match to find documents where a field is in a list of strings. If I for example have 3 entries in the list and a lot of data in the collection, but no data matching those 3 entries, it is *much* faster to do three separate requests with only one element in the list than doing one request with all three in the list. Can
[15:42:18] <Tausen> mongodb not use the index I have on the field as efficiently in this case or something?
[15:43:23] <Sticky> having a sane default that non-localhost connections require auth, would probably have prevented a huge number of those misconfigurations. And should not be terribly difficult to implement
[15:54:13] <Sticky> the argument that it is safe since it does not bind the public port is a bit off, people are used to existing db's that by default are secured (atleast the dbs I have used). Similarly web servers when asked to bind a public port will not expose their admin consoles unauthed to the world
[15:55:46] <Sticky> there is an expectation of safe by default by more than just binding the right ip, breaking that expectation is exposing your users to risk
[16:02:26] <StephenLynx> doesn't change the fact the user has to screw up to make its db not secured, though.
[16:05:22] <Sticky> yeah, but for very little extra effort and inconvinience you could protect the majority of users. If you want to add an enableunauthedPublicAccess config param fine, then the few people who do want it have to explicitly request it for little effort
[16:07:40] <StephenLynx> again, a good argument and I'm sure they had a reason to not do that. You could try and open a ticket on jigra about it.
[16:09:33] <Sticky> tbh the fact that mongo does not make it easy to obtain an ssl'ed mongo is almost as bad as this issue as well
[16:12:16] <Sticky> I thought your "not intentional." ment they are not intentionally shipping ssl'ed mongo
[16:24:39] <AnnaGrey> Im trying out mongodb for the first time and im wondering if im in the right spot with this schema http://pastie.org/9942231
[16:55:14] <hmsimha> I've been finding the documentation on TTL indexes a bit incomplete. If a TTL is set, say, for 3600 seconds (1 hour) on a 'lastUpdated' field that may get updated with some frequency, does that TTL countdown reset every time the field is updated?
[17:34:08] <hmsimha> If I'm getting documents from a collection in chunks of 1000 (`.limit(1000)`) and in between performing `collection.find().limit(1000)` and `collection.find().limit(1000).skip(1000)` one of the documents returned in the first query is deleted from the db, how can I prevent skipping over a document?
[17:51:52] <jiffe> hmsimha: instead of skipping you could sort by an ascending field and add a $gt filter
[17:54:02] <appledash> Hello... Is there any way to have a "Standalone" MongoDB server? Perhaps I am using the wrong term, but what I want to do is have an application that uses MongoDB as a database, but it spins up its own internal copy of MongoDB to use just for itself, with the data being storeed in a subfolder of the dir the application is in. Is this possible?
[17:55:29] <hmsimha> jiffe: thanks, but lets say I have 10000 documents numbered 1-10000 and I request `collection.find({someField: {$gte: 0}}).limit(1000)` to start off with, I guess on the server I need to store the last value of someField somewhere?
[17:56:11] <cheeser> mostly, yes. but it wouldn't be embedded in the sense of running in process with your app
[17:57:19] <jiffe> hmsimha: yes each lookup will have to know the last value, similar to if you were to use skip you would have to skip(1000 * n)
[18:15:54] <hmsimha> the docs list an example for index creation of a compound index: `db.products.ensureIndex( { item: 1, quantity: -1 } )`. How does index creation preserve the order of the fields if they're passed as object keys?
[18:20:47] <StephenLynx> if you get to print an index name, you will see it is something like field_1_anotherfield_1 or something. I don't remember too well. I know It would be in the error.err or something when you try to insert something that disrespects the compound index.
[18:23:50] <hmsimha> StephenLynx: http://stackoverflow.com/questions/5525795/does-javascript-guarantee-object-property-order
[18:26:05] <hmsimha> ah, found the answer: http://stackoverflow.com/questions/18514188/how-can-you-specify-the-order-of-properties-in-a-javascript-object-for-a-mongodb
[18:48:16] <ezakimak> I just did db.coll1.copyTo(db.coll2) and now when i show collections there's an entry "[object Object]", and the coll2 is still empty
[18:49:38] <cheeser> use this instead: http://docs.mongodb.org/manual/reference/method/db.cloneCollection/#db.cloneCollection
[18:57:03] <ezakimak> ok, so schema question, i have on Person collection, and each person can have 0 or more roles. I implement the roles as subdocuments. would it possibly be better to split the roles out into their own collection? (classic collection vs subdocument question)
[18:58:06] <StephenLynx> I have something similar in two projects of mine, I use a subarray with strings for the roles.
[18:59:07] <ezakimak> in mine the roles are specific, eg Person: { _id: 4, name: "joe", admin: { admin role stuff } }
[19:02:49] <ezakimak> but, i have one role, student, which is very large, and the central focus of the application, that subdocument is complex
[19:03:29] <ezakimak> i didn't want two mechanism for representing roles, but didn't want to make a new collection for each role (because eventually i want to allow custom roles)
[19:05:22] <ezakimak> well, i think i just answer my own question. I can't do searches easily if they are split into different collections
[19:15:45] <ezakimak> just thinking through if it would be better to split them out, decided against it
[19:17:29] <ezakimak> a teacher will have data like state id, qualifications, etc. a student role will have all the student record data
[19:20:48] <fewknow> okay you said a role = https://dash.metamarkets.com/magnetic_audience/explore_audience_searches#e=2015-01-19&p=custom&s=2015-01-12&zz=3
[19:21:00] <fewknow> role = a teacher will have data like state id, qualifications, etc. a student role will have all the student record data
[19:21:59] <ezakimak> if they have the role, it has the subdocument, otherwise it doesn't even have the key
[19:23:55] <StephenLynx> again, we can't really help you on schema design if you don't show us the schema.
[19:27:28] <ezakimak> but then you'd have to do a second query to get the rest of the teacher data, right?
[19:30:06] <ezakimak> Person: { _id: 2, name: "John", teacher: { grades: [7,10], classes: [4,22,97], ... }, user: { passphrase_sha1: 'xyz', ... } }
[19:31:08] <ezakimak> no, because there could be other roles that also have grades and for different reasons
[19:31:36] <ezakimak> i am using the terms from my ubiquitous language from the business model in my design
[19:32:18] <ezakimak> that is my original question, how much worse or better is it to leave these as subdocuments vs splitting them out into their own collection at the expense of now having to do joins and complicating searches
[19:32:22] <fewknow> Person: { _id: 2, name: "John", role : ['teacher','user'] , teacher_grades: [7,10]}
[19:33:29] <fewknow> every role in the collection, no matter what role with have the corresponding _id to the PERSON document
[19:34:45] <ezakimak> my "role" is a subset of the person's profile, with data specific to that role, if they have it, it's *still* part of the profile
[19:37:11] <ezakimak> a) keep it as one collection with subdocuments, b) split out each role into it's own collection, c) your idea of lumping all roles into a 2nd "roles" collection
[19:37:50] <MacWinner> i'm planning on moving away from using gluster for scalable storage to GridFS
[19:38:14] <MacWinner> i was wondering, how do you handle multiple filenames for the same blob of data with the same md5 hash?
[19:38:56] <MacWinner> in gluster I'm appending the filename to the blob of data, and doing md5 hash on that..
[19:41:13] <ezakimak> fewknow, so in your idea (c), you'd have to encode the role type into each document
[19:41:59] <ezakimak> use the same _ids from the Person collection, then also index the type field also
[19:44:02] <ezakimak> I think i'll leave things as they are for now, maybe later i'll revisit this if i move to an ES achitecture
[20:08:51] <ezakimak> so "bucketing" on them simply means lumping them all together in the same document
[22:12:02] <timmytool> We recently upgraded our mongo servers from 2.4.10 to 2.6.7. We have been running fine for almost a year on the older version. After the upgrade we are having some serious performance problems. Based on what I can tell by looking at the currentOp statistcs, we have some queries that are taking a long time to run. The queries are fully indexed, I have verified this by doing an explain. Looking at the ops they have up to an hour with a
[22:12:03] <timmytool> timelockedMicro. The num yields is also very high for these queries. Is there any way I can get more information about what is going on? Thanks
[22:14:52] <timmytool> fewknow: thanks. I’ve done an explain on the query and it is using an index. It is the same query that is taking a long time. There are many instances of it running with.
[22:15:42] <timmytool> Is there some way for me to tell why mongo is yielding the locks on these queries. I’m assuming that that is the problem.
[22:17:27] <fewknow> are you sure it wasn't some code that was released that is inserting something that is causing the lock?
[22:17:39] <timmytool> fewknow: Thanks again. How do I determine replicat set performance from the output of mongo stat.
[22:19:15] <timmytool> I’m thinking that something in the newer version of mongo is yielding those locks more easily than the old versions.
[23:05:39] <jiffe> if you run a replicated or shareded setup on separate machines then you need to configure mongod to bind on an interface other than localhost right?