Log file Viewer
#mongodb logs for Saturday the 15th of August, 2015
[00:35:21] <diegoaguilar> joannac, I need to make massive updates on a production database, basically they will be 25k inserts and 30k updates
[00:35:50] <diegoaguilar> whats are best techniques to do a healthy import AND/OR have minimal downtime
[01:27:42] <diegoaguilar> Hello I'm trying to do this to get the "update if found, insert if not found" behaviour, but its not working properly as I get a cannot update lfPoints and lfPoints at the same time
[01:32:36] <Boomtime> diegoaguilar: it is a known problem: https://jira.mongodb.org/browse/SERVER-10711
[01:34:03] <Boomtime> the easiest way around it is to not require initializing a field (which you want to increment in an upsert) to anything other than zero
[01:36:18] <diegoaguilar> db.findAndModify("mydb.counters", {_id: "myCounter"}, {$inc: {n: 1}, $setInsert: {n, 0}}, {upsert: true, new: true})
[01:38:15] <Boomtime> diegoaguilar: two things; 1. that line is the same as an update, 2. you don't need setOnInsert at all in that variant
[01:38:59] <Boomtime> all fields initialize automatically to the equivalent of null/zero as soon as you don't anything with tham - $inc assumes a field to be zero if it doesn't already exist
[04:08:45] <wiltors42> Since the entries are not in any particular order, how do I step through all of them in a loop?
[04:57:05] <movedx> MongoDB newbie here. Creating replication sets is a good way of offering data redundancy, backup, and enabling you to create read-only slaves so you can offload read operations to another system. What does sharding mainly introduce? Performance?
[05:20:30] <Boomtime> movedx: http://docs.mongodb.org/manual/core/sharding-introduction/#purpose-of-sharding
[05:36:36] <leptone> i have a collection of cafes with: {name, address, speedTestList} how can i use db.cafes.update() to append a new object to the speedTestList attribute
[05:39:30] <Boomtime> leptone: assuming speedTestList is an array: http://docs.mongodb.org/manual/reference/operator/update/push/
[06:19:26] <Boomtime> leptone: that isn't valid JSON, but if you need examples there are at least 3 on that page i linked
[06:40:52] <Boomtime> 1. http://docs.mongodb.org/manual/reference/operator/update/push/#append-a-value-to-an-array
[06:41:00] <Boomtime> http://docs.mongodb.org/manual/reference/operator/update/push/#use-push-operator-with-multiple-modifiers
[06:41:16] <Boomtime> http://docs.mongodb.org/manual/reference/operator/update/push/#append-multiple-values-to-an-array
[09:16:01] <Boomtime> the docs example uses toArray which is crap because if you have 10,000 docs to process it will grab them all in memory
[09:17:12] <Boomtime> forEach is iterative, it returns to the server in batches and only keeps that number in memory at a time
[09:17:51] <Axy> I need to add an "index" field to my documents, and every time I add it I increatse a counter in a "counters" collection
[09:19:44] <Boomtime> any code which worked with hasNext() and next() must, by definition, work with forEach
[09:21:01] <Axy> and I keep a document, for the "final index" of the original collection I want to add indexes to
[09:21:29] <Axy> so I need to go one by one, and every time the new index is added I need to increase the one at counters
[09:21:45] <Axy> I just tried with foreach it seems to be getting the same index for large number of documents
[09:22:22] <Boomtime> then it would do so for the next() and hasNext() loop as well, or you are breaking the rules of NodeJS - which seems likely at this point
[09:23:03] <Boomtime> next() gets more than one from the server at a time, it merely reveals one to you at a time, exactly the same way that forEach does
[09:25:14] <Boomtime> what is the point of your counters document if it can give out the same value twice?
[09:25:39] <Axy> well this is only to add an auto incremental field to a whole collection filled with documents
[09:27:16] <Boomtime> you'll probably need to resort of doing the batching the updates yourself - use toArray so you can control the order of things, but do the find in batches of a few thousand or something
[09:29:48] <Axy> and then I get responses like "if you need an auto inc. field maybe mongo is not what you should use"
[09:30:26] <Axy> There is no way I could pull this off with a relational db - all of my documents hass differnet field structures
[17:06:12] <coderman1> shouldnt this return me just the leadid field? db.leads.find({'leadid': 1}).sort({leadid:-1}).pretty()
[18:22:05] <coderman1> is there a way to get the mongo cli to show how long a query took after it ran it?
[18:23:23] <coderman1> and that appears to not run the query, just show you the plan that it would run
[20:23:23] <asteele> hi all, i am new to using mongo in production and just yesterday started seeing issues in our application where certain endpoints of the api are very consistently taking a very long time (400 - 1200 ms) to return. I am trying now to find out where the issue is, if it is with mongo or some of the code, i am using mongotop and mongostat and according to those two it shows everything happening very fast, never anything above 1-5 ms, is that
[20:23:24] <asteele> telling me the issue is definitely not with mongo - or are there other places to look
[20:24:42] <asteele> ill be idling so if you see this and have any opinions, your time is very much appreciated <3
[20:30:25] <asteele> are there some known issues i should look for there? i am not doing tons of complex work but give me one second and ill try my best to explain whats going on
[20:32:51] <asteele> lol :( fair enough, but part of me thinks a bigger issue might lie somewhere. everything was working fine for many weeks but just recently there is a very noticable delay in lots of the api calls - reading from the db seems to still be quick
[20:33:17] <asteele> i get your concern though i think, its hard to diagnose slowness when you have mongoose in front of it all
[20:34:00] <StephenLynx> you cannot expect to use it and have a good performance with large datasets.
[20:34:30] <StephenLynx> using mongoose is the most common error people make when using mongodb with node/io
[20:35:31] <asteele> but like 500 ms slow? my data set is really not all that big. We have < 2500 users and most of what they are doing is continually updating one document. I AM using like a populate() query and stuff during some of these calls, but like i said to me the weird part is, up until yesterday all of this stuff ran very fast and fine, even using mongoose
[20:36:06] <StephenLynx> make a prototype performing the same operation with the native database and test it.
[20:37:01] <asteele> okay brb a few minutes while i test some stuff on local, thank you btw StephenLynx