Log file Viewer
#mongodb logs for Friday the 10th of January, 2014
[00:37:38] <nemothekid> Hi is there way to perform an upsert, when the queries _id is complex? For example coll.update({"_id":{"$in":["a", "b"]}, {"$set":{"x":"y"}}, {upsert:true})
[00:46:16] <nemothekid> Admittedly I didn't think it all the way throught, I was expecting _id to be "a"
[00:52:14] <joannac_> It's just because of the complex query, if it's simple we would set it on upsert
[00:52:41] <joannac_> but because it's complicated, you can't tell mongod "hey use this one" because it's the _id
[01:20:13] <swak> I want to make sure my logic is right for this: Node server connects to MongoDB, server performs it's operations, server ends connection. Is this the correct way of handling things?
[02:24:28] <ShortWave> I hate that, when I type the question and figure out the answer as I'm typing it.
[09:50:01] <intellix> hey, I'm pasting a script into mongo shell and it's complaining about syntax errors, which worked fine yesterday :P I think to do with new lines
[09:50:28] <intellix> SyntaxError: syntax error (shell):0..... I'm pasting in something like: var things = ["one", "two"];
[09:54:43] <intellix> thought it would accept new lines. it's worked perfectly fine until today, just today it stopped working :P
[09:56:07] <intellix> yeah, I literally copy and pasted a large script into shell on multiple servers and it worked perfectly fine. The only difference is today I connected to a REPLICA set, pasted and it didn't work. connected to PRIMARY to paste the script and it didn't work again.. like the REPLICA pasting broke it or something :P
[09:56:30] <intellix> ultimately, I guess it should be imported via a JS file instead, which would work for sure.. but weird that it just stopped working
[09:58:53] <intellix> from JIRA actually.... so maybe it is to do with different new lines or something
[10:01:31] <intellix> actually. it works on another box (shell 2.4.5)..... and not on (shell 1.6.5); I'm sure this worked yesterday
[12:10:37] <NaN> the faq says "Do not use MongoDB for systems that require SQL, joins, and multi-object transactions." that means mongodb doesn't support fk?
[12:12:59] <NaN> Derick: nested and "shared" (kind of fk) documents? or nested and duplicated across my documents?
[12:14:01] <NaN> that means if I need to change 1 value in a shared document, I need to change it in all the ocurrences?
[12:14:23] <ppetermann> if you have it in several documents, you need to change it in several documents, that is correct
[12:15:19] <Derick> all data in a document belongs to that document, whether it is "nested" or "embedded" (the same thing)
[12:20:05] <ppetermann> say, i write two letters, and sign both with my name, is my signature a duplicated value?
[12:21:35] <NaN> so your signature will be on 2 documents, if you signature where fk you can have it only once
[12:22:36] <ppetermann> since my signature in this example is on a sheet of paper, there are no fk, also my signature is my signature, but its not identical - its both a version that each blongs to a specific document
[12:23:43] <NaN> ok ok I was thought in a signature like the final message in mail, exactly the same data
[12:25:23] <NaN> ok the thing is that I need to do a "mail like" app where you can create, receive, revew, re-send "mails" (like private messages), do you recommend me to use mongo?
[12:26:16] <ppetermann> that depends on a lot more factors than a "mail like app", including what mails, how much writing, how many data storage you have, etc.
[12:27:13] <ppetermann> personally i'm using a fork of mongo (tokumx), to store a database of so-called killmails which are documents that represent kills within the game eve online
[12:27:37] <ppetermann> and since the data is very simmilar in most of those, the choice is not too obvious
[14:05:56] <ppetermann> if you use 2 collections and do manual relations, you most likely have a broken db layout
[14:07:23] <ron> NaN: there's no fk in mongo. you can emulate it. but it *may* lead to consistency issues since there are no transactions.
[14:07:53] <ron> one direction is fine, most of the time. just make sure you pick the right direction :)
[14:11:53] <NaN> oh well, I will do one-direction only, then it's ok to emulate fk with one-direction? :)
[14:29:53] <tresbailey> Anyone on that can comment on Java Mongo Driver, specifically for GridFS support?
[14:35:02] <tresbailey> using spring-data-mongo(1.3.2) and mongo-java(2.11.3) I can save the file but when writing the outputstream for the GridFSInputFile to a byte[], i get NullPointerException in GridFSInputFile:285 - iterates through this line a few times before failing with the _messageDigester attr as null
[14:37:14] <tresbailey> *for more info, specific line of my code: ((GridFSInputFile) gridFsTemplate.store(new ByteArrayInputStream(inputtedByteArr), fileName.toStringMongod(), "application/pdf")).getOutputStream().write(outputtedByteArr)
[15:56:07] <mayanks43> I am trying to set the _id value to a field value minus the surrounding ObjectId
[16:19:45] <Joeskyyy> mayanks43: I know you can do it in the mongo shell doing something like ObjectId("blahblahblah").str
[16:36:54] <includex> Hi guys, I've lost admin pass from my cluster... I have a repset with 1primary, 1secondary and 1 arbiter. To solve the password thing without affecting anything I just need to delete admin.* under the primary; remove the auth from conf and set the admin pass again?
[16:39:29] <Joeskyyy> includex: The issue is that you'll have to bring the primary down, which will result in a new primary being elected.
[16:39:47] <Joeskyyy> So you'd need to stop all the replicas so a re-sync doesn't happen across the board.
[16:41:17] <Joeskyyy> I'm not familiar with a way of doing it without a small portion of downtime at least.
[16:41:17] <includex> Joeskyyy autch. so I need to stop every thing first. remove the admin.* db files on every node and setup it on the master?
[16:41:45] <Joeskyyy> Cause other wise, once you delete it on the original primary, and put it back online, it'll get added to the repl set
[16:42:36] <includex> Joeskyyy my fear is about 'what can go wrong with this admin.* db files removal'
[16:44:07] <includex> Joeskyyy another question (sorry to take your time) I+m using keyfile to auth nodes. I don't need to touch this to rebuild the admin user? just cmment the auth=true
[16:45:45] <includex> Joeskyyy well service is running flawless, I just can't login as admin through mongo cmd
[16:46:31] <Joeskyyy> I'd imagine that can be a bit limiting if you need to do anything to your db haha
[16:50:01] <Joeskyyy> Well, at least you didn't have it on a sticky note at your desk. *sigh* Oh how often I've seen that.
[16:50:39] <therealkoopa> I'm stumped on something. I have a query: db.people.find({_id: 'abc123', 'jobs._id': new ObjectId('someobjectid'), 'jobs.location.nickname': null}), that returns results differently between local env and a server. the nickname is NOT there on either server. According to the docs, this should find results that are missing the nickname field, or if it's null, correct?
[16:53:18] <therealkoopa> http://docs.mongodb.org/manual/faq/developers/#how-do-i-query-for-fields-that-have-null-values
[16:54:28] <Joeskyyy> therealkoopa: Huh… yeah it is supposed to. $exists wouldn't match values set to null
[16:56:35] <therealkoopa> According to the docs, it should return results if th enickname is NOT there.
[17:01:40] <therealkoopa> Yea, it works for me on a bunch of machines, including 2.2.0, 2.2.2, 2.4.6, 2.4.8, but there's something bizarre on this ubuntu box. I don't know if it's a driver, or something insanly silly (most likely) that is wrong.
[17:02:05] <therealkoopa> I noticed this because there's a bug in some code... And I dropped to the mongo shell and noticed this difference.
[17:06:43] <therealkoopa> I'm using mongoose. Part of me wonders if it's tied to the minimize: false in the schema.
[17:12:53] <therealkoopa> Yea, I tried iwth some other objects, and it's matching! I wonder if it's something with a null/undefined/empty business
[17:16:44] <Joeskyyy> It's gotta be, something in the way it's returning in mongoose I'd estimate yeah.
[18:34:53] <ekristen> well I was planning on running with a 3 node replicaset to start and use mongos to connect, and then once I’ve ironed out how I want to do the sharding then start doing that
[18:48:44] <ekristen> for development and testing can my primary mongodb server also act as a config server?
[18:58:37] <ekristen> can I configure a mongod instance as a configsvr just by using the config file or does —configsvr have to be used?
[19:25:57] <pasichnyk> hey guys, i have implemented something with use case similar to the blog about how MMS was built (a ton if small $inc upserts). Master takes the writes without breaking a sweat, but the replicas lag a ton. Any key settings that might be causing issues here? the boxes are colocated together, so latency is not a concern.
[19:39:51] <therealkoopa> Is there a way to find nested values where some field has been explicitly set to undefined?
[19:42:37] <therealkoopa> Joeskyyy: Yea, it's driving me insane. I feel like the data jobs.location.nickname was explicitly set to undefined, which would explain why the query doesn't return... I think
[19:51:00] <therealkoopa> Joeskyyy: I have a few documents, where I cannot figure out what is saved: Is it truly undefined meaning the field isn't there, was it saved with null, or was it saved with a value of undefined.
[19:52:50] <Joeskyyy> Where I'm bit at a loss, when you say "value of undefined" how do you mean? Would it be like the last example I posted?
[19:57:17] <Joeskyyy> I'm not 100% familiar with mongoose, but go ahead. I'll do my best to look at it (:
[19:58:49] <therealkoopa> With that... It generates this: https://gist.github.com/anonymous/c8640d0b34c39edf1445
[19:59:13] <therealkoopa> And this query doesn't return anything: db.dashboards.find({'wobs.widget.someThing': null})
[20:02:39] <Joeskyyy> In order for that query to return something, wobs.widget.someThing would either need to be "null" or the field not exist what so ever.
[20:04:16] <Joeskyyy> It's in an array though, so I'm not sure how that's affecting the behaviour you're expecting though, now I see a potential issue, sec.
[20:07:08] <therealkoopa> I think I have some old data where that "someThing" field was explicitly saved as undefined.
[20:09:48] <therealkoopa> If I remove the field from the first wob, the query still doesn't match, so I don't know now.
[20:10:18] <Joeskyyy> Yeah, I think the fact that it's an array is affecting how the query results are actually interpreted.
[20:11:49] <Joeskyyy> In mine, you'd ideally like line 8 to identify right (obviously it's not undefined but similar idea)
[20:14:51] <Joeskyyy> I'd try and open a ticket with the folks at mongo, or maybe one of them happens to be lurking here today
[21:20:55] <ytrezq> Hello, How I can use the active directory for Mongodb users... (this is a web site database; I need to use a framework which only support MongoDB).
[21:21:06] <ytrezq> I'm speaking of Active Directory, for a Samba4 server (I mean the server isn't running windows).
[21:22:27] <ytrezq> Hello, How I can use the active directory for Mongodb users... (this is a web site database; I need to use a framework which only support MongoDB).
[21:47:00] <kamal_> I won't consider using RethinkDB in production until they implement auto failover
[21:48:51] <ytrezq> Hello, How I can use the active directory for Mongodb users... (this is a web site database; I need to use a framework which only support MongoDB).
[21:49:42] <goog> kamal_: does that mean if failover is supported, you will switch without any doubt?
[23:01:13] <intangible> I have a question about shard key selection: I read that it must be a field in every document in the collection, so for my collections I only have "_id" and "name" for users so I was going to use name, but the docs seem to say you can't update() the value of a shard key? is that correct or does it just mean the original value of the shard key is where it will be sharded to?
[23:06:48] <intangible> yeah, I read about hashed keys and it sounds great, but I don't think objectrocket supports that version of mongo yet :-/
[23:07:53] <Joeskyyy> intangible: Pretty sure they support hashing, I work for rackspace on a team that works closely with them
[23:08:30] <intangible> Joeskyyy, thanks, I'll reach out to them, their UI doesn't seem to offer it, but maybe I missed it, or I can do it via a direct command
[23:09:45] <intangible> I think hashed _id makes the most sense for most of our collections :-)... growing data size kindof forced my hand to define the key now rather than wait anymore though
[23:11:17] <Joeskyyy> Yeah, hased _ids make a lot of sense for most use cases, because it's pretty random, so it makes a nice distrobution :D
[23:12:17] <harttho> hashed_id's will give you good write performance, but you won't really benefit from reads
[23:13:57] <intangible> our two biggest collections are organized by location (geo spatial), so that was the original plan for those keys keeping reads fast and in a single shard
[23:14:40] <intangible> but most of the other collections aren't as high read or write usage, so hashed _id should work for them
[23:15:47] <intangible> the shard keys being set in stone without big collection migration makes me skittish :-D
[23:16:14] <Joeskyyy> harttho: Isn't it also true that it's typically more beneficial to hash at the application level, that way your mongos isn't tied up doing any hashing?
[23:17:17] <harttho> We use our own hash values we actually query by (don't query by _id) so we get more than just write performance
[23:17:23] <Joeskyyy> I coulda sworn that the rule of thumb was, use mongo for ensured persistent hashing, use your application for performance boosting
[23:18:18] <harttho> Haha sounds interesting though, I'll take a look. We stopped using ID hashing a few months ago
[23:22:05] <zfaith> hey guys, I'm trying to use mongo's aggregate function to group things by 3 minutes
[23:35:12] <Joeskyyy> So if the minute is divisible by 5, group it on that minute. You'd need to sum everything up between those though
[23:35:31] <zfaith> like this {"_id": {"year": "$year", "month": "$month", "day": "$day", "hour": "$hour", "$minute": {"$mode": [5, 0]} }?
[23:35:59] <Joeskyyy> Right, but think about where that would get you, that would just give you results that are divisible by 5 evenly.
[23:36:27] <zfaith> I found this: http://www.kamsky.org/1/post/2013/03/stupid-date-tricks-with-aggregation-framework.html
[23:36:58] <Joeskyyy> http://www.nrg-media.de/2013/10/mongodb-aggregation-group-by-any-time-interval/
[23:38:30] <Joeskyyy> No problem. Glad I found that. The embedding in my brain was hurting this late in the afternoon.
[23:48:21] <judges119> I'm running a Node.js script, is it okay practice to open a connection to the database when the script loads and leave that open during it's runtime (only thing accessing the DB, hosted on the same server) or should I open a new connection to the DB every time I want to perform an interaction?
[23:50:41] <Joeskyyy> Cache connections where it makes sense. Leaving open connections to mongoDB can cause a lot of headaches when you run out of em
[23:52:33] <Joeskyyy> It's worth it to have the very very very very split second connection effort, then to deal with your app crashing from too many connections being opened (:
[23:52:36] <judges119> Oh, I'm only making one connection to the database initially, events that require DB all use that same connection? At least I think that's what's happening.
[23:53:28] <Joeskyyy> Most client/server style applications however, open up a new connection per user session.
[23:55:09] <judges119> Ahh, in my case the app opens the DB connection when it's initialised then inside that connects to Twitters streaming API. All incoming tweets/users/data that interact with the database just use that same open connection.
[23:55:34] <Joeskyyy> Gotcha, yeah if you're just data collecting from a stream, you should be safe.
[23:56:35] <judges119> Okay, thank you, I just wanted to make sure I wasn't breaking some cardinal law of MongoDB and would be cast back out into the RDBMS wastelands.