Log file Viewer
#mongodb logs for Thursday the 27th of March, 2014
[01:16:24] <Leech> hi, I have a map reduce that needs to run every day, and my main search engine uses the collection generated by the mr. What's the best way to avoid any search problems while I run the mr?
[01:17:08] <Leech> I was thinking in runnnig the mr to another collection and then delete search collection and rename the new one, but not sure about delete+rename timing
[09:28:37] <c3l> How do I find() several documents in a given order? That is, I have an array of ObjectIds and want to fetch documents in exactly this order. Right now I need to either make each DB request separately or manually sort the result afterwards...
[09:54:46] <kAworu> If I want every document where the array property has a size bigger than three: db.foo.find({array: {$size: {$gt: 3}}})
[09:55:57] <kAworu> Nodex: is using $where equivalent ? I guess the counter method is way more performant though.
[09:56:02] <Nodex> http://docs.mongodb.org/manual/reference/operator/query/where/ <--- doesn't use indexes
[09:56:26] <Nodex> c3l : are you sure it's not returned in the order you're asking for with the $in ?
[09:56:42] <kAworu> Nodex: yes sorry by equivalent I wanted to ask if it could also be achieved using $where
[11:04:41] <Nodex> I think there is a write up bout Mongodb on aws and what's recommended in the docs
[11:39:52] <greybrd> can't I modify the _id field of a document in mongodb? when tried it says "cannot change _id of a document". what to do?
[11:42:47] <greybrd> Nodex: requirement :-) . have added a new field in a map-reduce job which runs every hour. but still the existing documents need to be updated.
[12:09:58] <Leech> hi, I have a map reduce that needs to run every day, and my main search engine uses the collection generated by the mr. What's the best way to avoid any search problems while I run the mr?
[12:10:02] <Leech> I was thinking in runnnig the mr to another collection and then delete search collection and rename the new one, but not sure about delete+rename timing
[13:35:58] <mylord> performance: does it matter how long my document item names are? ie, packageName : “com.mysite.app” vs pn:”com.mysite.app”?
[13:37:35] <mylord> should i typically use, e..g “pn” instead of “packageName” where I, and would-be other views, can pretty easily figure out/remember what pn is? for performance/storage, or does the server remap those names to some optimized bit-mapping anyhow?
[13:43:41] <mylord> k… should my rest paths typically be, e.g., /something/pn, or something/packageName ?
[13:56:33] <Indium> I'm doing a findandmodify to get a value and set a field to "processing" so the next time the same findandmodify will return the another row
[14:24:32] <LoneSoldier728> Story.find({followers: {$nin: [user]}}, "_id name", {$sort: {date: -1}, skip: skipCount, limit: limit}, function(err, results) { I thought I was making a correct query here but for some reason it still shows stories that have the userId in it
[14:37:41] <LoneSoldier728> http://stackoverflow.com/questions/22690856/how-to-query-all-documents-in-mongodb-that-do-not-have-a-specific-id
[14:41:56] <cheeser> http://docs.mongodb.org/manual/reference/operator/query/in/#use-the-in-operator-to-match-values-in-an-array
[14:53:25] <LoneSoldier728> http://stackoverflow.com/questions/22690856/how-to-query-all-documents-in-mongodb-that-do-not-have-a-specific-id
[14:54:17] <wc-> hi all, i have a list of embedded docs, is it possible to query for a field on one of those embedded docs and return only that embedded doc instead of the whole list?
[14:55:20] <LoneSoldier728> the issue I am having is that the document has an array and it will have to check the whole array to make sure that objectId is not in it
[15:00:36] <LoneSoldier728> and all the documents in the db get printed for example I have that being returned in results
[15:04:54] <LoneSoldier728> I just converted the string to an objectid var user = mongoose.Types.ObjectId(req.query.userid);
[15:05:48] <Nodex> unfortunatley you're going to have to pastebin a typical document and pastebin the actual query (the data) being sent to the server for debuggin
[15:12:06] <LoneSoldier728> I added the GET call, then that is the query I have been using and the bottom is grabbed from my db
[15:22:21] <Nodex> can you do this .... var a={followers: {$nin: [user]}}, "_id title", {$sort: {date: -1}, skip: 0, limit: 100}}; console.log(a);
[15:39:06] <Nodex> _conditions: { followers: { '$nin': [user] } }, <----- it's sending it as a string it would seem
[15:39:36] <LoneSoldier728> maybe this model schema can be affecting it... I am adding it to the new pastebin http://pastebin.com/DKjwan0A
[15:59:01] <LoneSoldier728> is it inefficient to grab the results and just sort through them on the server side?
[16:20:31] <madalynn> mongo keeps attempting to connect my my local machine, when inside the js the params for the remote server exist
[16:21:52] <LoneSoldier728> Nodex because I figured that getting that mongo is much quicker in scanning for it rather than grabbing all results then looping through each array
[16:22:35] <Nodex> LoneSoldier728 : your query should be using $nin which just uses a btree to remove anything inside the array
[16:23:41] <LoneSoldier728> right that is what I am doing right now, but I was just saying it is better to have mongo remove the unnecessary documents than getting all results then doing it serverside
[16:36:08] <rybnik> madalynn yes, but you've specified an invalid namespace (I believe). Anyway, to pass along your credentials use the -u and the -p flag on the command line to specify the required credentials
[16:36:43] <rybnik> madalynn mongo -u myUser -p myPassword myhost/myDatabase myscript.js > myoutputfile
[16:39:17] <madalynn> rybnik: it just says in the file "connecting to: myserver.com:9999/mydatabase"
[16:40:07] <rybnik> madalynn care to send a pastie with the following: The command line invocation (please omit user credentials) and the script that you want to run
[16:46:30] <LoneSoldier728> so Nodex instead of doing it in the db I should be doing it on my server side?
[17:08:35] <thearchitectnlo1> how can i export to csv from mysql to be able to work with —headerlines ?
[17:30:32] <LoneSoldier728> Nodex right that is what I thought, I am just confused why the query does not return the results
[18:23:57] <proteneer> i keep getting 2014-03-27T11:19:59.962-0700 [conn3] ERROR: SSL peer certificate validation failed:certificate not trusted
[18:24:41] <kesroesweyth> In v2.4.9, is there a way to update a document with a partial object from PHP and not blow away other object attributes that existed previously but weren't defined in the new object?
[18:25:50] <kesroesweyth> Like if my collection contained {field1: "value1,field2: "value2"} and I wanted to add {field2: "new value2",field3: "value 3"} without clobbering field1, if that makes sense.
[18:42:52] <kesroesweyth> The blog says that partial object updates are possible. I need to basically merge an object with an existing one in Mongo within a single document. Is this possible?
[18:51:58] <bluefoxxx> i.e. if I have a document package={ 'name': 'handyman', 'discount': 0.15, 'bundle': [ ObjectId("11773355aabbccdd"), ObjectId("11559922ffaaccbb") ] }
[18:52:35] <bluefoxxx> and I want to pull a document that has 'bundle' set to an array of documents instead of ObjectId()
[19:11:27] <bluefoxxx> Somebody decided that some of those fields should be able to reference other stuff and we should be able to return the full content of that stuff, so ... many-to-many.
[19:12:20] <bluefoxxx> We've considered embedding the data, but it's single-data: if you update it, you wind up updating it in 300 places, not all of which you may be able to find
[19:33:09] <bluefoxxx> Nodex, can I get either aggregate() or mapReduce() to emit a document in full plus values?
[19:33:34] <h0rnet> seems silly to auto discover the host... but not be able to do anything with it... and make adding the host an extra step
[19:34:40] <bluefoxxx> i.e. d = { field1 : ObjectId(a), field2 : ObjectId(b), field3 : 7 } , but mapReduce() to get back r = {field1 : ObjectId(a), field2 : ObjectId(b), field3 : 7, references : [ObjectId(a), ObjectId(b)] }
[19:57:07] <_Andres> Hi, a quick question: How do I persist additional data about the relation with has_and_belongs_to_many ?
[19:58:01] <_Andres> Mongoid creates an array field of 'blabla_ids', how do I add more data to each relation?
[20:34:08] <ramfjord> if creative_id is null, then that should be the only stat for (line_item_id, date)
[21:12:52] <jello`> anyone around for some nooby advice on unexpected data returned from mongoose find() or findOne()?
[22:51:30] <_Andres> h0bbit, Mongodb is like a siren. It sings a beautiful song that just sounds so irresistible. Getting started and storing your objects is so simple. Yet as time goes by and you start developing more complex relations between these objects.. it eats you alive.
[22:53:11] <h0bbit> my production server is failing, throwing more hardware at it doesn't work, mongodb docs are ambiguous, and there is no "support" to speak of.
[22:54:15] <_Andres> The docs are somewhat funny indeed, by the looks of it a lot of time has spent on it. Yet it lacks clear and concise examples
[22:54:15] <h0bbit> we've had trouble with mongodb before, but nothing of the kind I'm seeing today.
[22:54:49] <lorgio> Does anyone know how MongoDB's MapReduce decides what chunks are processed in what order?
[22:55:07] <_Andres> It doesn't help that every mapper and driver has a different DSL in addition to MongoDB's own..
[23:05:57] <joannac> lorgio: i thought the entire point of mapreduce was that you couldn't pick an order that the results come back. There is a query and a sort so you can in theory, control the order things come in?
[23:06:57] <h0bbit> MMS graphs show me that a background flush has started on the system, and it has now been nearly an hour since any read on that machine has succeeded.
[23:07:25] <h0bbit> I want to pull the secondary out of the replica set, but everything in the docs seems to suggest that this will lead to a re-election
[23:08:56] <joannac> as long as you have majority even with the seconbdary dead, there will be no re-election
[23:10:01] <h0bbit> I don't think every read on that machine is failing, but a significant number of them are.
[23:10:25] <lorgio> joannac: no it's not that the results come back in...what i'm asking is how it picks the chunks of values to reduce and re-reduce
[23:12:01] <joannac> I did this yesterday with a 3 member set. pulled a secondary out. no re-election.
[23:12:28] <joannac> your logs will get a bit sad. the replica set will keep trying to ping the secondary and reporting it's not there
[23:13:42] <h0bbit> full disclosure, since we were in this situation an hour ago, we went ahead and pulled the machine out of the replica set.
[23:14:35] <h0bbit> I don't have the logs in front of me right now, but I do have the error emails. I'll get the logs in 5-10 minutes once my immediate problems have cooled down
[23:14:56] <h0bbit> also, what was even worse, the problem just started occuring on a different secondary
[23:16:00] <h0bbit> here are the docs : http://docs.mongodb.org/manual/tutorial/remove-replica-set-member/
[23:19:08] <h0bbit> btw, we have seen this problem (background flush leading to read operations failing) once before as well. inevitably happens under peak load.
[23:19:34] <h0bbit> I don't know if the background flush is responsible, correlation != causation etc.
[23:20:14] <h0bbit> the last time, we were able to resolve it by forcing secondarypreferred on some of our app servers for a short while.
[23:34:45] <lorgio> If I have a sort in my map reduce call, Can I assume that the emit values are sorted in that order?