Log file Viewer

[00:38:27] <jiggliemon> Hey guys. I can't stop thinking in relational terms.

[02:10:46] <andholt> quick question if anyone has time to help - running a query with a $where statement like so: $where : "hex_md5(this.email) == '" + md5_hash + "'"

[02:11:01] <andholt> and getting the following error: rror { [MongoError: Error: hex_md5 takes a single string argument -- hex_md5(string) near 'his.email) == 'd1ac2903d2edabc5f2673a9a13' ] name: 'MongoError' }

[02:11:14] <andholt> this works locally on Mongo 2.6, but not remotely on our servers which are running 2.4

[02:15:06] <joannac> did you type that error or copy and paste?

[02:15:17] <joannac> how did this.email turn into his.email?

[02:15:22] <andholt> Copy and paste

[02:15:53] <andholt> not sure...

[02:17:02] <joannac> turn that entire query into a string, and print it out

[02:17:09] <joannac> and see if it says what you think it says

[02:20:16] <andholt> ok, so I'm using Mongoose as my ODM, this is the output in debug mode:

[02:20:18] <andholt> users.findOne({ '$where': 'hex_md5(this.email) == \'0b668ca989d2567af68d624dceb86e18\'' })

[02:25:04] <joannac> andholt: try it in the mongo shell connected to 2your 2.4.x instance?

[02:25:44] <andholt> good idea, one sec

[02:32:26] <andholt> seems to run fine in the console, though it's extraordinarily slow (hasn't returned a result in a few minutes)

[02:32:55] <andholt> this is through a Compose.io interface so I'm not sure if its perhaps just timing out

[02:32:59] <andholt> but no errors are thrown

[04:29:04] <]random[> i'm trying to write a sidekick service to check if mongodb is alive or not. is there a way to check it without having to install the mongo client?

[04:32:28] <]random[> end goal is for sidekick to check if it's up and update etcd accordingly for service discovery by apps and replicas

[04:35:33] <]random[> nvm, i see they actually have a rest api interface.

[04:41:27] <]random[> can't figure out the rest api :(

[05:39:53] <hahuang62> joannac: hey there

[05:39:59] <hahuang62> joannac: I had a question for you

[05:46:09] <joannac> hahuang62: ask your question

[05:47:37] <hahuang62> joannac: we successfully upgraded from 2.0.4 to 2.6.5 yesterday and we're seeing some different query performances. In our mongod logs, we see a TON of queries not hitting an index (or seemingly using the _id index), but when we try .explain() on it, it'll hit the appropriate index and return in 30ms instead of 70k ms like in the mongod logs...

[05:47:39] <hahuang62> joannac: https://gist.github.com/hahuang65/8e5761375806c3d048a0

[05:47:45] <hahuang62> joannac: have yous een anything like that?

[05:48:48] <joannac> hahuang62: upgrade to the latest release 2.6.7

[05:49:27] <hahuang62> oh... was it a bug with 2.6.5?

[05:50:53] <joannac> no idea, but upgrading to the latest release at least assures me it's not a bug already fixed

[05:52:05] <hahuang62> joannac: I searched around, and found: https://jira.mongodb.org/browse/SERVER-13866

[05:52:11] <hahuang62> but that claims it was fixed in 2.6.2

[05:55:41] <joannac> hahuang62: isthe wrong index thing reproducible?

[05:55:48] <joannac> hahuang62: which shard is the mongod.log from?

[05:56:25] <hahuang62> joannac: it's shard 1 always, and we cannot reproduce it in mongo console, but a TON of these show up in the logs, enough to triple our average response time.

[05:56:39] <hahuang62> all the other queries look to be performing about the same as pre-upgrade

[05:57:10] <joannac> hahuang62: turn up your logging and try and catch one

[05:57:35] <hahuang62> joannac: I'm not sure I understand what that means? Aren't I already catching these int he log?

[05:58:05] <joannac> no, 1 line is not enough to diagnose

[05:59:17] <hahuang62> okay, lemme see what I can do

[05:59:39] <joannac> http://pastebin.com/37H3rSFi

[05:59:47] <joannac> warning: will generate a LOT of logging

[06:00:17] <joannac> your issue needs to be reproducible, otherwise you're just going to get a lot of noise

[06:01:01] <hahuang62> joannac: okay, let's try this.

[06:01:06] <joannac> actually, out of curiousity, what happens if you try that query directly against the primary of shard1?

[06:03:33] <hahuang62> lemme try, but I've got a giant log here also

[06:03:36] <hahuang62> what should I grab from it

[06:47:02] <henryC> Is there anybody that can advise me with a situation that is similar, but not the same, as this? https://stackoverflow.com/questions/10726178/how-to-findandmodify-a-value-in-a-nested-array

[06:47:39] <joannac> what's wrong with the answer in the SO ?

[06:48:21] <joannac> how is your question different?

[06:48:41] <henryC> My situation is similar but I don't have unique keys in the array like that. I need to match based on _id first, then select the key based on some string (which is not unique across documents).

[06:49:07] <joannac> ? that makes no sense

[06:49:17] <joannac> pastebin/gist an example

[06:49:25] <henryC> ok one sec

[06:52:50] <henryC> hm I thought it would be easiest to copy an example doc out of chrome dev tools but that didn't come out too pretty.

[06:53:43] <henryC> one sec I'm cleaning it up

[06:56:01] <henryC> sorry if this not the prettiest, but in my callback I have _id and I have the symbol (eg. "VT"), and I want to update the "shares" in the array. http://pastebin.com/EfCVNF9i

[06:56:42] <henryC> VT will exist in other documents

[06:59:10] <joannac> so why can't you do db.foo.update({_id: ..., "tickers.symbol" : "VT"}, {$set: {"tickets.$.symbol" :"AB"}})

[07:00:48] <henryC> ah, thanks, I'll try that. Sorry this is all so new to me coming from SQL.

[07:08:13] <henryC> joannac: thanks. I think that will work but I can't see it work because I ran into an issue relating to meteor's allow/deny rules I think.

[07:10:55] <henryC> joannac: I guess Meteor will only allow me to query based on _id and a user may have several of these documents, so this is a separate problem I have to work out. Thanks for getting me straightened out on the mongo part.

[07:58:26] <jasperr> Searching for awaitData example in ruby. Anybody did this? Thx.

[09:13:01] <joy> hello can anybody sujjest a good database design to store my data's actually i am designing a taxi app and i want to store all data's into two dbs one relational and the other one is mongo db

[09:14:19] <joy> is relational db really needed here for my app design or just any nosql db is enough

[09:14:51] <joy> can any body answer my query

[10:21:19] <hahuang62> Any mongodb employees here right now? We just had some huge perf issues and our cluster is down, would like to see if we can pay for some support

[11:26:42] <hahuang62> Derick: hello?

[11:26:46] <hahuang62> ehershey: hello?

[11:26:50] <hahuang62> ranman: hello?

[12:57:57] <joannac> hahuang62: go talk to your sales rep

[13:03:34] <joannac> alternatively https://www.mongodb.com/products/production-support#buynow

[16:07:31] <hahuang62> joannac: we purchased online actually, but they gave us a generic phone number

[20:12:03] <nfo> Hi everyone. I upgraded from MongoDB 2.4.9 to 2.6.7. Since the update, all requests (upserts) are written in the log file. I kept the configuration file from 2.4. Logging has the default values, which should be "not verbose".

[20:12:04] <nfo> # Verbose logging output.

[20:12:05] <nfo> #verbose = true

[20:12:07] <nfo> # Set oplogging level where n is

[20:12:08] <nfo> # 0=off (default)

[20:12:10] <nfo> # 1=W

[20:12:11] <nfo> # 2=R

[20:12:13] <nfo> # 3=both

[20:12:14] <nfo> # 7=W+some reads

[20:12:15] <nfo> #oplog = 0

[20:12:16] <nfo> #diaglog = 0

[20:12:17] <nfo> (ho, sorry, I thought it would be a single message)

[20:12:48] <nfo> I tried to explicitely set "verbose = false" and "diaglog = 0", and mongod did not even start

[20:15:20] <nfo> yeah, I confirm. If I set "verbose = false" in the configuration file, mongod does not start

[20:23:04] <nfo> no error message in the logs

[20:24:11] <nfo> mongod starts well with "verbose = true". i think I recall that "verbose = false" is an impossible value.

[20:27:24] <nfo> Profiling seems disabled:

[20:27:25] <nfo> rum_a:PRIMARY> db.getProfilingLevel()

[20:27:26] <nfo> 0

[20:27:27] <nfo> rum_a:PRIMARY> db.getProfilingStatus()

[20:27:28] <nfo> { "was" : 0, "slowms" : 100 }

[20:28:42] <joannac> nfo: are all your upserts taking longer than 100ms?

[20:29:50] <nfo> joannac: yes they are. I can't find where profiling is enabled

[20:30:14] <joannac> profiling is not enabled?

[20:30:24] <joannac> by default anything that takes longer than 100ms gets logged

[20:30:26] <nfo> see above

[20:30:37] <joannac> or whatever your slowms is set to

[20:30:51] <nfo> profiling status is 0, I thought it meant "no profiling"

[20:30:56] <joannac> it does

[20:31:02] <joannac> profiling is not the same as logging

[20:31:29] <joannac> the slowms value is shared though

[20:31:38] <nfo> ho ok

[20:32:45] <nfo> ok I get it "mongod writes the output of the database profiler to the system.profile collection."

[20:32:53] <nfo> "mongod prints information about queries that take longer than the slowOpThresholdMs to the log even when the database profiler is not active."

[20:34:14] <nfo> joannac: Can I change this threshold from the configuration file ? Or I have to call "db.setProfilingLevel()" from the mongo shell ?

[20:34:56] <nfo> sorry, it's here: http://docs.mongodb.org/v2.4/reference/configuration-options/#slowms

[20:37:07] <nfo> If mongod 2.4.9 behaved the same way, it worries me a bit that a lot more upserts take more than 100ms (about 200ms apparently) in 2.6.7. It fills gigabytes of log files in seconds.

[20:37:28] <nfo> (*minutes)

[20:40:26] <joannac> pastebin some log lines

[20:41:01] <nfo> I do big upserts :)

[20:41:32] <nfo> and I usually get "warning: log line attempted (11k) over max size (10k)".

[20:47:04] <nfo> joannac: I can't really share the log lines, sensitive data (or I can share in a private message). But as I wrote, my upsert queries are quite big. I can share the mongostat output, if you'd like.

[21:04:00] <nfo> joannac: thanks a lot for your help.

[21:06:58] <joannac> share in PM if you want

[21:07:09] <joannac> I'll have a look a bit later