Log file Viewer
#mongodb logs for Tuesday the 13th of September, 2016
[00:08:16] <cheeser> it's pretty new and experimental. i don't think anyone's mentioned using it here.
[00:54:00] <jstream> i tried using mongodump and mongorestore to move an application and its database to another, soon-to-be production server. except surprisingly, the application on the production server is behaving as though there are multiple copies of the same obj stored in the db.
[00:54:41] <jstream> I put on a bandaid by checking to see if an obj had already been read before adding it, and that fixed the problem
[00:56:10] <jstream> curiously, when I use mongo on the production server, i cant find any duplicate copies of objects -- at least by searching for _id's
[00:57:23] <jstream> I wonder if this might be happening because when i moved the database, i did NOT use the point-in-time option
[01:07:29] <cheeser> with mongorestore you can pass --dump to remove the existing collection before restoring the dumped data
[07:27:04] <anubhaw> https://stackoverflow.com/questions/30882629/does-performing-a-partial-update-on-a-mongodb-document-in-wiredtiger-provide-any
[07:29:19] <anubhaw> if you don't want to follow the link then question is this. "Does performing a partial update on a MongoDb document in WiredTiger provide any advantage over a full document update?"
[08:32:31] <catphish> hi, i have just added a node to my replica set, but it seems to be writing to disk a lot more than any other nodes in the cluster, my primary is writing about 20 operations per second, the new secondary is writing about 500, is there any obvious reason for the disparity?
[08:34:29] <ranman> it could be building indexes? I can't remember if repl sets build the indexes after sync or before
[08:35:27] <ranman> the simplest way would be https://docs.mongodb.com/manual/reference/method/db.currentOp/ I think
[08:37:47] <catphish> unless it's learned about it from the primary and is using it, i would not expect that to be the case
[08:38:43] <catphish> this is the currentOp(): http://paste.codebasehq.com/pastes/7gze842p1qty3fqz6p
[08:43:02] <catphish> these are the io stats across the cluster http://paste.codebasehq.com/pastes/glv3muysv3jgruu2xn
[09:41:30] <catphish> so, is there any way i might be able to trace what operation is causing my huge number of disk writes?
[09:46:17] <catphish> i cant work out if this is some kind of one-off operation, or just extremely inefficient ongoing replication
[10:03:27] <catphish> Derick: sorry, i guess you missed my original question, i have a primary and a new secondary, the primary is (afaik) handling all the clients, and is doing about 20 disk writes per second, the secondary (which is new, but has progressed to secondary state), should be doing nothing but replicating the primary, but is running hundreds of writes per second
[10:09:03] <catphish> here's montostat http://paste.codebasehq.com/pastes/a8edlt6ia11irbjlnp and iostat http://paste.codebasehq.com/pastes/7j2bcz22u1i0ivrtdy
[10:16:12] <catphish> i'm not really sure what information you're asking for when you say "what sort of updates"
[10:18:08] <catphish> so either the secondary is running the same updates in a less efficient manner, or its doing something else i'm not aware of
[10:36:48] <catphish> i just noticed my replication is from mmapv1 to wiredTiger, with the latter being the problematic host
[12:55:51] <kees_> hey all, does anyone know why the mongo-php-driver connecting to dead members of a replicaset / arbiters?
[12:56:28] <kees_> i had to reboot an arbiter, and suddenly my error logs were full with connectionTimeoutExceptions
[12:57:14] <cheeser> kees_: https://www.mongodb.com/blog/post/server-discovery-and-monitoring-next-generation-mongodb-drivers
[13:01:53] <kees_> hm, ok that explains it a bit, but i still see no need to connect to arbiters or nodes the cluster already deemed dead
[13:04:51] <cheeser> what if they come back up? they still need to be monitored because they're listed in the topology
[13:06:51] <kees_> my php app isn't really happy with this behaviour tho, as it reconnects all the time
[13:08:47] <kees_> are there any settings to tune this behaviour? i can't really reconfigure the replicaset every time a host goes down and have my php app wait for 300ms to connect to a server not in the seed list
[13:09:35] <kees_> i know discovery and all can have advantages, but if it is killing my performance i'd rather have a less flexibel but more performant setup
[13:10:25] <cheeser> arguably, your cluster is the problem if you have nodes dying. fixing that will fix your perf problems.
[13:11:58] <kees_> it was a planned reboot this time, but i guess i'll just have to reconfigure the cluster next time before i reboot
[13:13:55] <cheeser> there was at one point an SDAM daemon for drivers like the PHP driver that couldn't do the SDAM work in a thread. did that one escape the lab?
[13:16:37] <kees_> i guess i'll just need to rework my mongo architecture to include some sort of mongodb proxy so my (short lived) php apps can connect to localhost and dont have to check the entire cluster for almost every request
[13:26:02] <kees_> would be nice to have a middleway, still have the ability to detect primary/secondary, but without connecting to all the members
[13:26:42] <cheeser> primary can change. you need to track the other nodes so you can find it if it does.
[14:16:57] <ashp> In this case it does a long scan of 1.2M entries rather than using the index that seems to exist
[14:18:02] <ashp> https://gist.github.com/apenney/b86ee2447777b61564467c033f4dcc46 updated - not sure why it has the word test in it now
[14:18:47] <ashp> all the info I have really is: "query db.employer_email query: { employerId: ObjectId('576865fe9b221298341f16ae') } "
[14:18:57] <ashp> so I assume it's literally just passing in an employerId and hoping to get the whole thing back
[14:47:29] <jadew> I'm afraid that manual rollback could also fail, in which case you're left with an unknown state
[14:49:59] <jadew> the recovery scenario is just too complex for something that doesn't expect this to happen
[14:52:31] <jadew> well, I guess I'll just make it try to manually rollback, crash and prevent furthern instances from running
[18:41:56] <AlmightyOatmeal> i'm using Python to cache ElasticSearch data in MongoDB but i was wondering if there was a bulk operation that wouldn't throw an exception if a duplicate key exists but either silently ignore it (and continue processing the remainder of the batch) or update the existing object if it already exists?
[18:43:15] <StephenLynx> did you try bulkwrite? I don't remember if it would stop everything when hitting a duplicate unique index.
[18:43:53] <GothAlice> AlmightyOatmeal: If you want to insert, or update if existing, you want the "upsert" option.
[18:44:31] <GothAlice> If you're using bulk operations, https://docs.mongodb.com/manual/reference/method/Bulk.find.upsert/ or your driver's equivalent may be useful. Otherwise it's just an option to the .update() call.
[18:45:07] <GothAlice> https://docs.mongodb.com/manual/reference/method/db.collection.update/#upsert-behavior
[19:23:33] <catphish> so, i don't understand memory mapped files very well, but it seems to me that mongo maps as much as possible, which rather upsets my free memory monitoring mechanism, is there some way to find out how much memory is mapped (but actually available to other processes)?
[19:25:51] <txm> Hi. AES encryption at rest, https://docs.mongodb.com/manual/core/security-encryption-at-rest/, can this be done with the open source version or enterprise only? I note “Available for the WiredTiger Storage Engine only.”
[19:26:37] <GothAlice> txm: Right above the warning mentioning it's available for WiredTiger is another warning: Available in MongoDB Enterprise only.
[19:36:34] <AlmightyOatmeal> StephenLynx: i didn't see your response earlier, my apologies. afaik all bulk/batch operations will fail if a duplicate key is found but there is a method that would allow me to essentially do update calls, instead of insert, but that would mean i would have to iterate over the list of data i want to send to build a new array of data i want to send which introduces additional overhead :(
[20:04:43] <Doyle> Doesn't this: "Use CNAMEs to identify your config servers to the cluster so that you can rename and renumber your config servers wi1thout downtime." conflict with: "If you start a mongos instance with a string that does not exactly match the string used by the other mongos instances in the cluster, the mongos instance returns a Config Database String Error error and refuses to start."?
[20:04:46] <Doyle> as seen here: https://docs.mongodb.com/v3.0/core/sharded-cluster-config-servers/, https://docs.mongodb.com/v3.0/tutorial/deploy-shard-cluster/
[20:11:59] <kuku1g> Is it possible to have a single node in two replica sets? for example, i want node1 to be in rs0 and rs1
[20:13:16] <kuku1g> @cheeser I have 5 nodes and want to split up my data evenly but have data only replicated once. How do achieve it then? I wanted to have 5 replica sets but this won't work then
[20:17:19] <kuku1g> @cheeser When I create a replica set with 3 nodes and one with 2 nodes, data is only sharded two times. I want to benchmark Cassandra vs MongoDB for my usecase. I think the setup is not equal.
[20:18:17] <Doyle> Nevermind. I found the line I remembered reading. "These records make it possible to change the IP address or rename config servers without changing the connection string and without having to restart the entire cluster."
[20:25:40] <catphish> GothAlice: thanks, i was actually mistaken, my memory monitoring works fine, and only looks at rss :)
[20:27:45] <kuku1g> @cheeser or do i misunderstand the concept of a sharded collection? can I have one replica set of 5 nodes and one sharded collection that will distribute all data across the nodes evenly?
[20:28:21] <kuku1g> it doesnt work that way right? one replica set will cause all nodes in the RS to have the same data?
[20:35:45] <kuku1g> does that mean that I can have a collection sharded on my 5 nodes without needing the replica sets?
[20:36:28] <kuku1g> as in "the replica sets are just there to maintain availability in case of hardware failure"?
[20:41:14] <cheeser> so a sharded cluster with 5 shards will be at least 15 machines: 5 replica sets of 3 members each
[20:41:26] <kuku1g> When a replica set involves faster reads because data is deployed on more machines, this is not equivalent to my cassandra setup!
[20:41:42] <cheeser> you don't *have* to have a 3 member replSet but for production that's the lowest recommendation
[20:42:23] <kuku1g> I see. that's for production though. I want to benchmark the system performance for now
[20:43:22] <kuku1g> I think the best solution I can come up with is a sharded collection without a replica set for mongodb so data gets evenly distributed on the 5 nodes. as for cassandra, i would also use no replication.
[20:44:56] <kuku1g> do you have a rule of thumb on how much a replica set increases the read performance for mongodb?
[20:48:00] <kuku1g> I know that it will not affect write performance because only the primary member will be used to upsert data
[20:58:57] <kuku1g> @cheeser the same approach has been used by Datastax for a YCSB benchmark here: http://www.datastax.com/wp-content/themes/datastax-2014-08/files/NoSQL_Benchmarks_EndPoint.pdf
[21:41:36] <Robbilie> i am running into a mongoerror the past week and i cant really identify the source of it
[21:42:30] <Robbilie> would be greate if anyone could advise me on finding the source of this error :)
[21:47:22] <crazyphil> if I have a sharded 2.6 setup, and I need to turn auth on (mongodb-cr specifically), do I need to set every mongod instance for --auth, or only my config servers, or everything?
[22:20:37] <joannac> It's for questions, so if no one has questions about MongoDB then yes, it is quiet
[22:30:16] <joannac> crazyphil: need to turn on auth for everything. https://docs.mongodb.com/v3.2/tutorial/enforce-keyfile-access-control-in-existing-sharded-cluster/
[22:34:04] <Robbilie> the error is not instant, happens after some time, with both calling .stream() or .each() on the cursor
[22:39:24] <joannac> but I haven't seen that message for anything other than maxTimeMS, so it's weird that you're getting it without setting maxTimeMS
[22:48:53] <Robbilie> joannac: well looking at mongos server source its mentioned with some deadline or replication stuff so maybe also related to that…