Log file Viewer
#mongodb logs for Thursday the 22nd of May, 2014
[04:36:51] <XeonCore> We have a an API that is built on top of mongodb. We are currently adding some extra network locations (like 6 new ones) that we would like to host the full API on. To make life simpler and to keep the API in sync we were thinking we would run one MongoDB server in our main DC that was written to. We would then setup a mongodb instance in each location that
[04:36:51] <XeonCore> is set to replicate from the master. Everywhere all the documents say "Always make sure you have the majority of members in a single location". We want a single primary and the rest priority 0 secondaries. We don't care if the primary MongoDB instance goes down for a short period as the API will be perfectly happy running on stale data. Am I correct in
[04:36:51] <XeonCore> thinking that any reads to a priority 0 secondary will still work whilst the primary is down?
[05:43:52] <sigurding> any aggregation framework pros here? I have the following dataset: https://gist.github.com/jhiemer/f48edb4f69b0b160fbd9 and aggregation.
[05:44:41] <sigurding> The aggregation should return the avg values of values.data. My ideal idea would be, that the aggregation return a complete entity, but that doesn’t seem to work
[07:45:55] <mylord> How do I update score, if newScore>oldScore, and in case score was stored as string, compare it as an int? http://hastebin.com/toludehiro
[08:04:23] <Industrial> Hi. I'm doing a col.find().stream() on about 5 streams in parallel from node.js
[08:11:31] <Industrial> I cannot find the documentation for .find().stream() at http://mongodb.github.io/node-mongodb-native/api-generated/collection.html#find
[08:11:51] <Industrial> at least, nothing tells me it's actually returning a Cursor and that the callback is optional
[08:23:24] <mylord> how can i do this? db.turnys.update( { "users.u1.score":8 }, { users.u1.score:11 } );
[08:25:48] <mylord> { "pn" : "floppy.bird", "seats" : 10, "start" : ISODate("2014-05-21T19:13:14.550Z"), "price" : 1, "end" : ISODate("2014-05-21T19:15:14.550Z"), "rMin" : 800, "rMax" : 900, "users" : { "u1" : { "score" : 8 }, "u2" : { "score" : 9 } }, "_id" : ObjectId("537cfa8e27b73caefddf82df") }
[08:32:04] <mylord> how can I add a date to the score, like this? db.turnys.update( { "users.u1.score":{$lt:11} }, { $set : { "users.u1.score":11, "users.u1.score.date":new Date() } });
[08:33:32] <mylord> or, i’ll probably want to $set using a score document from my scores collection, which already has a date
[08:33:43] <Warfront1> how can i make it use counter my javascript variable, instead of it thinking it is a fieldname
[08:43:46] <mylord> kali: what if users.u1, or even users, doesn’t exist at the time. How can I do this most painlessly?
[08:46:31] <mylord> something using exists, like if users.u1.score !exist, create it.. but how? (i’m mongo newb)
[08:53:12] <mylord> dumb way i was doing was assuming i’d have users.u1, which i was enforcing by adding it on turny join.. actually that works fine.. i just need to invent an initial score of 0
[09:05:45] <Industrial> then I look at the 4 values and decide to 'pause' 2 and process 2, reinvoking the same method to get 2 new objects from the resumed cursors to make the same decision egain
[09:10:36] <Industrial> "The convenience methods each and toArray call nextObject until the cursor is exhausted."
[11:16:56] <mylord> node.js driver: how can I substitute “u1” in “users.u1.score” as a variable (my user id)? collection.update({'_id':new BSON.ObjectID(tid), "users.u1.score":{"$lt":score}}, {"$set":{"users.u1.score":score}}, {safe:true}, function(err, result) {
[11:19:52] <mylord> I tried substituing the string “users.id.score” with a var called uscore, like this: var uscore = "users."+uid+".score"; //didn’t work
[11:22:14] <kali> the javascript syntax { key: value } does not evaluate key. you need to prepare your query with something like that: var set_clause = { $set: {} } ; set_clause["$set"]["users." + uid + ".score"] = score
[11:25:22] <kali> i use scala and ruby, among other. i happen to hate javascript, but many people use it very succesfully with mongodb or anything, so my opinion does not matter that much
[11:30:17] <mylord> are there db’s you like better than mongodb, generally speaking, because of its use of json/javascript-ish-ness?
[11:34:37] <kali> well, same disclaimer than before, but mongodb is my go-to database and have been so for a few years
[11:38:43] <mylord> kali, you’re syntax worked! so awesome! now I’m trying to replace the 2nd part of my query cluase, "users.u1.score":{"$lt":score}}, with this: var uscore = "users."+uid+".score"; var qu_clause = uscore: {} ; qu_clause[ uscore ]["$lt"] = score;
[11:46:06] <khinester> hello, is there a client interface to connect to a remote mongodb for freebsd?
[11:47:29] <ajph> is it possible to return the last N documents in a capped collection (natural order) -- but return them newest first -- for example: `db.events.find().sort({"$natural":-1}).limit(3)` gives me the last 3 events - but the latest event is first -- how can i get that same output but reversed?
[11:47:57] <ajph> EDIT: is it possible to return the last N documents in a capped collection (natural order) -- but return them oldest first -- for example: `db.events.find().sort({"$natural":-1}).limit(3)` gives me the last 3 events - but the latest event is first -- how can i get that same output but reversed?
[11:58:27] <inad922> Could someone tell me if I do a raw_output on a query and I have a documentFIeld on my model why is that field missing from the result?
[12:12:10] <mylord> any help with the prob on line 3? https://gist.github.com/anonymous/e5b6422c925f08a3c483
[12:20:15] <mylord> I’ve remade like this: var uscore = {users:{uid:score}}; var qu_clause = { uscore: {} };
[12:23:48] <CJNE> you tried to set a property of a proprty that didn't exist, if that makes sense :)
[12:25:03] <mylord> CJNE: thx! still printing uscore as a string, see here: https://gist.github.com/anonymous/5e1fd73c810c3a05f9ec
[12:29:03] <hglattergotz> running really old instance of mongo 2.0.2, built a zipcode radius search tool and it seems to work ok for a radius less than 106 miles, but after that the max result set size is 16837. Does that number ring a bell with anyone?
[12:32:10] <CJNE> mylord: i'm not quite sure what you're trying to do there, what field do you ultimately want to do the $lte check against?
[12:34:39] <mylord> CJNE: ultimately I’m trying to replace “u1” here as my variable user id: collection.update({'_id':new BSON.ObjectID(tid), "users.u1.score":{"$lt":score}},
[12:35:31] <mylord> for the users with id of “u1” in the hard coded case that works, but I want to pass in the uid
[12:42:24] <CJNE> i have a question about if large skips and nscanned, it seems like the number of nscanned increases when i increase skip, is there any way around that? here's a gist: https://gist.github.com/CJNE/6a8b8ed80cd372b2fb58
[12:44:50] <kali> CJNE: nope, that's basically how skip is implemented. if it's a problem, you should consider using the last value from the previous page to move directly to the right place in the index
[12:45:34] <rspijker> CJNE: well… I don’t know much about index intersection, since I’ve hardly used 2.6 myself
[12:47:45] <kali> CJNE: that can only work if you use a small subset of the document field, but it looks like you're paginating a result list, so it might be the case
[12:47:48] <CJNE> rspijker, i've hardly used mongo so i wouldn't know either, i'm considering switching to mongo from mysql, lot's of new stuff to learn here :)
[12:48:49] <kali> CJNE: http://docs.mongodb.org/manual/tutorial/create-indexes-to-support-queries/#create-indexes-that-support-covered-queries
[12:49:15] <rspijker> just from a logical point of view, I’d say if you do skip on a query that’s not indexed, larger skips will *always* need a larger scan...
[12:50:07] <CJNE> kali: i need to use a lot of fields from the result so i don't think it would work with index only fields... one option might be to query for the first matching id in one query and then use that information to skip to the right place in the next (real) query?
[12:53:43] <CJNE> i also have problems with the correspoinding count query for this, it's slower than the find query (since it scans more i guess
[12:56:14] <kali> CJNE: an index with categories + the four other fields may help for both queries, actually
[12:56:45] <mylord> so, if i’m trying to generate the 2nd query part here, should it look the same when I print it out with stringify, or what should it look like when I’m debugging it that way? //collection.update({'_id':new BSON.ObjectID(tid), "users.u1.score":{"$lt":score}}
[12:58:05] <mylord> I’m talking about this part: "users.u1.score":{"$lt":score}} //if I stringify that, it should look like this, right? : “users”: { “u1” : { “score”: { “$lt”:789 } } } ;
[12:58:08] <kali> CJNE: yeah, i'm not surprised. the index can not discriminate the docuemnt alone as the categories are not present. the optimizer choose to pull the documents themselves rather than using the categories index (i'm not sure what are the optimizer capabilities in multi index situation with 2.6, not enough experience built with that yet)
[13:02:50] <CJNE> kali: adding categories to the index helped a little (gist updated), nscanned went down to 33224 from 41708
[13:11:42] <CJNE> while index rebuilds, i modified the query to use { postedAt: { $lt: lastDateOfPreviousPagedResult }} and that performed very well
[13:12:04] <mylord> now, I think, I just need to get the 2 parts of the query, q1, q2, to be presented corrently in the combined q_clause used on line 23, to make it work like line 24
[13:12:19] <kali> CJNE: yeah, that's because mongodb can go straight to the right place in the index and start scanning there
[13:19:07] <CJNE> thanks for the input kali and rspijker, looks like i can solve it now, probably going to skip using skip for pagination
[13:27:12] <CJNE> mylord, not sure if i missed anything but try replace line 6-15 with: var q_clause = {'_id':new BSON.ObjectID(tid), "users."+uid+".score":{"$lt":score}};
[13:29:23] <mylord> btw, all is working, if i only just q1 for the query, but I need both parts q1 and q2.. so maybe i’ll just form them as 1 object from the start.. since i don’t know how the node driver, i guess, wants me to pass in the 2 items, separated by “,”, if not in-place
[13:31:57] <mylord> but i guess i’d need an array to have 2 parts for the update ( { “1st part”, “2nd part” }, … // that’s the problem now.. I can’t figure out how I should pass a 2-part “objec” to the node driver update call
[13:47:17] <mylord> CJNE: thx so much for your help! It should update score from 20 to 22, but it doesn’t: https://gist.github.com/anonymous/19573cf60dffd82c8ca1
[13:59:36] <mylord> CJNE: no, I get this error: https://gist.github.com/anonymous/05d741df518d2749f0f2
[14:04:32] <CJNE> mylord: that looks like some express error, not related to the mongo query, it would be easier for you to test this using the mongo shell
[14:06:02] <mylord> ie, this works for me in the shell, but i need to do it in node, which works, but not if i’m trying to use variables to construct the update call: db.turnys.update( { "users.u1.score":{$lt:22} }, { $set : { "users.u1.score":22 } });
[15:07:33] <mylord> have this: collection.update( q1, { "users": { "$elemMatch" : q2 } }, set_clause, {safe:true}, function(err, result) {
[15:10:41] <IndianaJoe> anyone around to help with a php / mongodb / replset and authentication problem?
[15:17:18] <burhan> I'm new to mongodb - I have a system generating 6 CSV files daily; I would like to dump these into mongodb to build a search and query dashboard. All the CSV files have one common field (a customer identifier). If I create one collection for each type of log, can I query across all using the same key?
[15:18:54] <burhan> By query I mean have the related documents retrieved, based on a date range. So the query is "find me all records from 20/5 to 21/5 for customer #7896", and this number (7896) appears in multiple documents across different collections.
[15:27:40] <mylord> burhan, i’m newb to mongodb, but i think that yes, you can do it, but the way i know to do it involves multiple queries - 1 per collection
[15:28:28] <burhan> yeah this is what I assumed; but I'm not sure if its the best design to have different collections if all the documents have a common key. Should I just stick them in one collection, and have each CSV file as a sub-document?
[15:31:46] <burhan> but I am not sure what would be the best design - multiple collections, or one collection with a document with multiple sub-documents.
[15:40:31] <trench_> I am using mongodb 2.6 with pymongo under python2. I am working with an aggregation
[16:04:30] <Trench2> False is what we have set throughout our system. I'm not entirely sure about that, but I know that part is working.
[16:05:04] <Trench2> Anytime I try to throw messages.timestamp (subdocument) into the mix, either I return an error or I return no results.
[16:19:03] <Trench2> Should I be concerned with the json response coming with this printed in console (I'm throwing a print function to check what is coming through)
[16:23:57] <Trench2> I figured out what is happening... It is returning the thread as long as it has a timestamp of less than the provided timestamp.
[16:24:34] <Trench2> Since it's a subdocument, it's showing everything from the document as long as it has one subdocument with that timestamp.
[16:28:57] <modcure> is there a way to view all the queries that are currently running in the database system?
[17:14:19] <doug1> my boss keeps asking if mongo has a way to auto discover nodes.... what can I say to shut him up?
[17:16:15] <IndianaJoe> I can't see that it does. Each individual server needs to be specified for replSets
[17:18:53] <IndianaJoe> however, I am sure a script could be put together to do the auto-discovery, and to add new servers to the list.
[17:19:45] <Trench2> I am returning no results now, however, the before param is within the needed range
[18:20:33] <mylord> how can I do this? db.turnys.update( { "_id" : ObjectId("537ddaf87fe20000007cda18") }, { "users": { "$elemMatch" : { "u1":"score" } } }, {"$set":{"users.u1.score":26}} )
[18:21:31] <mylord> I’ve found another way to do this update, but this is the way that I might be able to do it from the node.js driver, so I want to do it using $elemMatch (or some other way that will work from node
[18:22:09] <mylord> for background info: this works from mongo, but not from node: //collection.update({'_id':new BSON.ObjectID(tid), "users.u1.score":{"$lt":score}}, {"$set":{"users.u1.score":score}}, {safe:true}, function(err, result) {
[18:22:52] <mylord> I just want to update a subdoc’s (users’s score), if the new score is higher than the current.
[18:23:23] <mylord> the problem from node has been when I try to replace “u1” with a variable, so I want to try the elemMatch way
[18:29:46] <mylord> how do you pass multiple variable values in a selector to the node mongodb driver? e.g., { myObj } // where myObj contains 2 properties intended to restrict results to both
[18:36:49] <Trench2> Every example I have read says I am doing this correctly... but I get nothing or errors!
[19:07:16] <newmongodbuser> Hi, I'm hoping somebody here can answer my stackoverflow question! https://stackoverflow.com/questions/23813680/mongodb-setintesection-on-array-of-arrays
[19:07:43] <clayzermk1> Any Node.js driver performance gurus about? I'm digging in to the various drivers and their insert performance. C++ and Python seem to insert in order, Node.js seems to use a stack.
[19:21:31] <clayzermk1> Any Node.js driver performance gurus about? I'm digging in to the various drivers and their insert performance. C++ and Python seem to insert in order, Node.js seems to use a stack.
[20:24:26] <Trench2> Does anyone happen to know if I can do an $lt against a value/type being returned from mongodb reading like so:
[20:47:22] <mylord> addToSet does it but creates an array element instead of just an object, and I don’t want that.
[21:24:07] <saml> what's difference between db.docs.find({date: {$lt: {$date:1400775813833}}}) and db.docs.find({date: {$lt: ISODate('2014-05-22T12:23:33.833-04:00')}})
[22:07:29] <thewiz> shell.db.User.findOneAndUpdate ({ username: targetname }, { $set: { $roles: "banned" } }, function (err) { // what am I doing wrong?
[22:19:42] <thewiz> joannac: how can I search a database for a subdocument (username) in the collection "User" and update its roles array to "banned"?
[22:49:47] <thewiz> why doesnt this work? db.User.update( { username: options.username }, { $set: {$roles: ["banned"]}});
[22:52:39] <thewiz> nvm db.User.update( { username: "johndoe2" }, { $set: {roles: ["banned"]}}); works
[23:08:45] <hahuang65> guys, I have a giant object in mongo shell, any chance I can write that out to a file, or must I rewrite this as a script and run it? query took a very long time
[23:45:50] <mikehelmick> then doing a $group at the bototm and my values i am specifiying in $project aren't being returned :(