Log file Viewer

[06:08:10] <george2> I'm planning on doing some benchmarks, but maybe someone here already knows - When serving files from GridFS using Apache/nginks modules, is this generally faster or slower than just serving the files from disk?

[06:08:43] <george2> This would just be for static images, if that matters.

[06:26:44] <george2> s/nginks/nginx/ :)

[08:21:49] <omid8bimo> hello, i asked this yesterday but i still have issues. i need help. my secondary mongo server crashed and i had to do a brand new install and sync from master in replicaSet

[08:22:10] <omid8bimo> but when i start the new mongo on the new server, either its not syncing the data or its too slow

[08:22:14] <omid8bimo> here is the log - http://paste.debian.net/78254/

[08:22:21] <omid8bimo> can anybody tell me what is going on?

[09:06:26] <Gargoyle> omid8bimo: Define too slow!

[09:06:52] <omid8bimo> Gargoyle: hey dude, thanks for your help yesterday :) but i need a little more assist :)

[09:07:28] <omid8bimo> Gargoyle: too slow like i have 180 GB data on master and after 28 hours, only 3 GB has been coppied to the secondary

[09:08:13] <omid8bimo> the log that i posted above says "replSet initial sync pending"

[09:08:16] <omid8bimo> why is that?

[09:10:53] <Gargoyle> probably just while the server starts up and allocates space, etc.

[09:12:59] <omid8bimo> Gargoyle: would you mind taking a look at my log output and tell me if something is wrong?

[09:13:13] <Gargoyle> omid8bimo: You might be best trying to investigate what's going on on line 230. I'm not sure what that means but google / someone else here might.

[09:14:14] <omid8bimo> Gargoyle: ok, so problem might be a collection or a document?

[09:14:38] <omid8bimo> also on the line 228 says "fastsync: skipping database clone" ! this mean the entire data replication was cancelled?

[09:14:49] <Gargoyle> Could be. It's a bit beyond me.

[09:18:53] <omid8bimo> :(

[09:33:40] <kali> omid8bimo: what happens next ?

[09:33:43] <kali> nothing ?

[09:33:46] <kali> more of the same ?

[09:36:21] <kali> iirc fastsync relates to the pre-replica-set era

[09:37:40] <kali> i don't think it is relevant

[09:44:24] <omid8bimo> kali: ok but the rest of the logs are the same as what i pasted in pastebin. and more of "[rsSync] replication batch size is 1"

[09:44:30] <omid8bimo> which i have no idea what that is

[09:56:48] <kali> omid8bimo: https://jira.mongodb.org/browse/SERVER-9791

[09:56:58] <kali> omid8bimo: i don't think you should wotty too much about this either

[09:58:49] <kali> omid8bimo: from what i see, it just look like your replica is busy syncing

[10:06:23] <Gargoyle> omid8bimo: Have you tested the speed of your network?

[10:06:32] <omid8bimo> kali: if its syncing, why its so slow?

[10:07:01] <omid8bimo> Gargoyle: well from graphs i do see sometimes throughput of 40MBps

[10:07:07] <omid8bimo> but now, only 2/3 MBps

[10:08:14] <kali> omid8bimo: maybe the primary is loaded ?

[10:09:31] <omid8bimo> kali: maybe. i have 10gen's MMS on the primary. any graph i should pay attention?

[10:11:31] <kali> not sure, i'm not using mms

[10:11:46] <kali> i would check out iostats on the primary

[10:11:57] <kali> and mongostats

[10:13:46] <omid8bimo> kali: mongostat sounds good. any specific?

[10:28:51] <NyB> kali: I finally managed to multiplex two cursors from two different threads... the results were disappointing, although I do not yet know why...

[10:30:31] <NyB> I do have a few suspicions though...

[10:57:59] <crashev> hello, the installation on Debian 7.3 seems to be broken -> http://pastebin.com/4LmTf9V6 [packages from http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen]

[11:00:17] <NyB> is it possible to use the $mod operator with multiple remainder values somehow? E.g. to select documents where the remainder of a field divided by 5 is 1,2 or 3?

[11:01:01] <NyB> or would I have to resort to complex queries with logical operators?

[11:04:02] <kali> NyB: you'll need a $or of $mod

[11:05:52] <NyB> kali: thanks...

[11:06:01] <NyB> pity though...

[11:49:33] <omid8bimo> can somebody tell me how does mongo's replicaSet replication based on oplog works? (a link or something would be useful)

[11:50:15] <omid8bimo> i mean like when a secondary stops working, how master knows how much data must be sent to secondary? and how to know since which transation?

[11:50:43] <Gargoyle> omid8bimo: http://docs.mongodb.org/

[12:03:43] <omid8bimo> Gargoyle: very general :)

[12:05:23] <Gargoyle> omid8bimo: Pretty sure you'll get there if you keep digging. :)

[12:06:11] <Gargoyle> And unless you are planning on actually developing mongodb, why do you want to know?

[12:09:28] <Gargoyle> omid8bimo: But from what I've interpreted so far. When the secondary comes back online, it will request oplog data from its last know position.

[12:09:57] <Gargoyle> If that position is still in the primary oplog, then replication starts.

[12:10:17] <omid8bimo> Gargoyle: since i came from MySQL background, i just need to know how the replication based on oplog works. like if i unplug the secondary and according to master's "db.printSlaveReplicationInfo()", secondary falls behind for like 4000 seconds, how does it know from where to begin?

[12:10:31] <Gargoyle> otherwise, a full database resync is required.

[12:10:52] <omid8bimo> Gargoyle: ohom

[12:12:24] <Gargoyle> omid8bimo: Have you used the new replication in mysql 5.6 ?

[12:12:53] <omid8bimo> Gargoyle: nope. currently i have mysql 5.5 servers. why? lots of changes?

[12:13:30] <Gargoyle> omid8bimo: They made it so you don't have to specify the binlog file and position when bringing a secondary online.

[12:14:02] <Gargoyle> It figures it out, and assuming you still have binlog position 0 on the master, it will fully sync.

[12:14:18] <Gargoyle> otherwise you have to do a mysqldump and restore.

[12:14:36] <kali> omid8bimo: it stores its last synced timestemp in some collection in local

[12:14:43] <Gargoyle> think of mongo doing that, but mongo handles it all for you, so you can sit back and have a brew! :P

[12:15:42] <omid8bimo> Gargoyle: fair enough :)

[12:15:51] <omid8bimo> kali: in local of the master, correct?

[12:16:22] <kali> nope, each secondary stores its latest oplog timestamp in its own local database

[12:16:28] <kali> local is not replicated

[12:17:24] <omid8bimo> kali: really? then how will secondary know where to begin the catch up if the local files are corrupted?

[12:18:51] <Gargoyle> Right, after 18 months away from mongo setup, I have a test RS up and running! :D Time for some shopping.

[12:19:04] <kali> if the local files are corrupted, you want a full resync anyway. so the secondary look at the time, starting pulling data from the primary (or a running secondary) and apply oplog starting at the point where it started fetching data

[12:33:12] <omid8bimo> kali: ok thats a good information

[13:24:58] <Gargoyle> Ughhhh. I picket the wrong time to goto the shops...