pmxbot IRC Log Viewer

[00:22:38] <oky> bros: exactly 73ms

[00:22:45] <bros> oky: are you serious or no?

[00:22:57] <oky> bros: how long did it take you?

[00:22:59] <bros> My query is taking over 2.5s

[00:23:49] <oky> bros: are you reading it via cursor or all in one go?

[00:23:54] <bros> I tried both.

[00:23:56] <bros> cursor is better.

[00:24:03] <oky> bros: are you running JSON.stringify on it?

[00:24:08] <bros> No.

[00:24:18] <bros> How can I have the DB tell me how long it takes to exhaust a cursor?

[00:24:24] <bros> I'm going off of console.time right now

[00:24:28] <bros> exhaust a cursor/run a query

[00:25:03] <bros> db.items.find( { account_id: ObjectId('561bffa148deeaf021a0ef5d') } ).explain("executionStats") says 74ms to execute

[00:25:10] <oky> bros: what did i tell you?

[00:25:20] <bros> Is that the execution

[00:25:21] <bros> or the stream?

[00:25:29] <bros> How did you know that?

[00:26:14] <oky> bros: it said 74ms to run the query planner? or planned amount of time to run the query?

[00:26:31] <bros> executionStats.executionTImeMillis = 74

[00:26:46] <bros> how can I tell how long the query would take?

[00:26:49] <bros> or does take

[00:27:02] <oky> https://docs.mongodb.com/manual/reference/explain-results/

[00:27:04] <oky> looks about right to me

[00:27:15] <bros> it is pulling 10mb from DB in 74ms?

[00:27:32] <oky> bros: is there any field that's particularly heavy that you can leave off via exclude projection?

[00:27:40] <bros> No. This is optimized.

[00:27:46] <bros> How an I get it closer to 74ms and not 2500ms?

[00:27:56] <oky> bros: how fast is your HD? 10MB in 70ms doesn't sound wrong to me

[00:28:19] <bros> What could be causing 2,500ms response time (best case scenario) then?

[00:28:21] <oky> paste your code, maybe someone will look at it

[00:28:40] <bros> https://gist.github.com/brandonros/e15bffc089d51ad870cfaba6c5e3749a

[00:28:40] <oky> just keep adding instrumentation to account for time

[00:30:08] <bros> doing absolutely nothing other than opening the cursor and waiting for it to close takes 2700ms

[00:31:12] <oky> bros: yeah... what's your batchSize?

[00:31:23] <bros> how cna i tell

[00:31:41] <bros> what's the default?

[00:32:05] <oky> bros: do you actually get any docs in your code?

[00:32:09] <bros> yes. 12k

[00:32:31] <bros> but i can't prove that the 74ms query

[00:32:31] <bros> db.items.find( { account_id: ObjectId('561bffa148deeaf021a0ef5d') } ).explain("executionStats")

[00:32:33] <bros> returns results

[00:33:52] <oky> bros: i need to refresh my knowledge, but don't you have to use cursor methods to iterate the cursor?

[00:34:03] <oky> what is .on('data') API from?

[00:34:30] <bros> what should I make the batchSize?

[00:34:39] <bros> .toArray() isn't any faster

[00:34:43] <oky> bros: huge, dude

[00:34:48] <bros> tell me

[00:34:49] <bros> how big

[00:35:05] <oky> try 1,000

[00:35:17] <bros> 2770ms

[00:35:54] <bros> but in Robomongo, 51ms execution time

[00:36:00] <bros> so it's the javascript driver?...

[00:36:24] <oky> bros: you are iterating doc by doc, still?

[00:36:31] <oky> bros: i don't know - but congrats on finding a culprit

[00:43:46] <bros> i have no idea

[00:43:48] <bros> how can i prove

[00:43:53] <bros> that this 51ms execution time

[00:43:56] <bros> is able to return all docs

[00:44:00] <bros> i feel like that's jsut opening the cursor

[00:49:03] <bros> oky: how long should .toArray() take?

[01:01:21] <bros> oky: i got it down to 1000ms

[01:01:27] <bros> is there any more wiggle room?

[01:06:00] <oky> bros: yes

[01:06:04] <oky> how did you get it to 1,000ms?

[01:06:17] <oky> bros: get it down to < 200ms, IMO

[01:06:24] <bros> tell me how

[01:06:25] <bros> i like

[01:06:28] <bros> desperately need this

[01:06:42] <bros> i got it to 1s by not running on external network SSH'd in

[01:06:56] <bros> i'm on 1s now private network

[01:08:31] <oky> bros: lol... what? you were doing your cursor over a real network?

[01:08:43] <oky> bros: were you sending 10MB over a network connection?

[01:08:46] <bros> yes

[01:08:47] <bros> lol

[01:08:54] <oky> are you still sending 10MB over a network?

[01:08:56] <bros> no

[01:09:05] <bros> well

[01:09:07] <oky> i'm not sure i believe you

[01:09:07] <bros> like

[01:09:11] <bros> eth1 inet addr:10.128.20.56

[01:09:24] <bros> they are on the same network

[01:09:32] <oky> alright

[01:09:43] <bros> tell me waht to run ont he mongo command line

[01:09:46] <bros> to show how long this query is taking

[01:09:54] <bros> and by query i mean get all of the documents

[01:10:03] <oky> bros: i don't know - i'm just trying to be helpful - i think you've given enough info for someone else to step in

[01:10:26] <bros> how can i exhaust the cursor in one go? .toArray()

[01:10:46] <oky> bros: yeah, but thats not really what you want. can you verify that your docs are streaming smoothly over that 1s?

[01:12:04] <bros> oky: https://gist.github.com/brandonros/8a37c7fdc6dee839e12b101c203c3289

[01:19:11] <bros> rl0:PRIMARY> var before = new Date(); var docs = db.items.find( { account_id: ObjectId('561bffa148deeaf021a0ef5d') }).batchSize(1000).toArray(); var after = new Date(); after - before;

[01:19:11] <bros> 295

[01:27:12] <oky> bros: what'd you change?

[01:27:18] <bros> that's the command line in mongo

[01:27:22] <bros> so it has to be the javascript driver?...

[01:27:29] <bros> it's not consistent though

[01:27:58] <bros> 781, 595, 510, 295, 320, 427, 353, 493

[02:39:29] <idioglossia> Is MongoClient.connect() considered expensive? Should I try and preserve the connection throughout an app by passing it around?

[02:39:38] <idioglossia> Or should I reconnect for any disjointed queries

[02:39:46] <idioglossia> as opposed to connecting once at the start of the app

[15:41:13] <Sagar> { "_id": 1, "data": { "new": { "other": 1 } } } i have this mongodb data, i want to increment it's value DB::connection("mongodb")->collection("developers")->where("uid", $id)->increment("data['new']['other']", 1); but nothing happens, is my query right? Using Laravel 5 on php 7

[16:34:01] <idioglossia> Should each module in node have its own connection to mongo?

[16:34:57] <idioglossia> The connection isn't established until the callback succeeds, so I could either block until the callback indicates the connection was successful, or I could just create a new connection every time I need to do a new set of queries

[16:35:32] <idioglossia> hmm actually I could just start the node app in the callback of the connection. That way it's established the whole time

[16:40:15] <idioglossia> yup that works :)

[17:28:10] <seeit> hello what is the location of the mongodb files and how do I move them to a new location and configure them?

[20:52:49] <SubMind> why is MongoDB better than postegreSQL ? am I stupid to ask this question ?

Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 4th of June, 2016