Log file Viewer
#mongodb logs for Wednesday the 20th of July, 2016
[00:51:56] <starseed> I'm looking for a way to determine the age or creation date of a cursor. Anyone know a way to do this?
[02:01:00] <starseed> I'm looking for a way to determine the age or creation date of a cursor. Anyone know a way to do this?
[02:15:32] <starseed> so we have these zombie cursors popping up occasionally which block sharding chunk migrations
[02:16:39] <starseed> I am scraping the Cursor IDs out of our logs, then I need to figure out how long each cursor has existed and kill them if they are older than x hours
[02:55:08] <starseed> you can specify notimeout, which means they live until killed via command or you restart the mongoD process
[02:56:06] <starseed> we have some very very long running queries. Due to shard distribution, sometimes we will open a query and access some data...then it'll be more than ten minutes until that query needs data located on the shard again. So the cursor would timeout if set to default and the query fails
[02:57:24] <starseed> our devs 'workaround' was to specify notimeout on the cursors...which frankly wouldn't be the end of the world, except that open cursors block chunk migrations.
[07:14:39] <skoude> hmm. .I would need a unique key for every single mondo document.. Should I use ObjedID or should I generate my own one? Any best practices for this?
[10:01:56] <jokke> my disk space ran out.. i deleted old time series data and wanted to reclaim the space with db.repairDatabase() but i got the following error: Cannot repair database energybox having size: 478605737984 (bytes) because free disk space is: 2851266560 (bytes)
[10:39:59] <kedare> One MongoDB 3.2 feature we demoed at MongoDB World in June was $lookup. $lookup is an aggregation pipeline stage that lets you insert data from a different collection into your pipeline. This is effectively a left outer join. $lookup will only be included in MongoDB Enterprise Server"
[11:41:52] <gzoo> On Linux, will a `service mongod stop` shutdown gracefully like `mongod --shutdown`?
[11:44:11] <gzoo> Derick, I find it uneasy not being mentioned on the official docs page for shutting down the server. Maybe I'm just being paranoid
[12:30:36] <atbe> Hey guys, for the logging done on the COMMAND component, how should we interpret the message?
[12:30:56] <atbe> I COMMAND [conn409274] command config.$cmd command: update { update: "mongos", updates: [ { q: { _id: "mdb32-rh7-s1-r:27017" }, u: { $set: { _id: "mdb32-rh7-s1-r:27017", ping: new Date(1469016660500), up: 148264, waiting: true, mongoVersion: "3.2.8" } }, multi: false, upsert: true } ], writeConcern: { w: "majority", wtimeout: 15000 }, maxTimeMS: 30000 }
[12:30:57] <atbe> keyUpdates:0 writeConflicts:0 numYields:0 reslen:386 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, Database: { acquireCount: { w: 2 } }, Collection: { acquireCount: { w: 1 } }, Metadata: { acquireCount: { w: 1 } }, oplog: { acquireCount: { w: 1 } } } protocol:op_command 106ms
[12:53:41] <atbe> I'm seeing some network discovery going on before I started to use my MongoDriver, is this by design? The mongo code is packaged in with some other code. But the mongo code had not been executed and it already started checking the network for mongo instances
[12:58:50] <cheeser> that's likely the driver discovering the cluster topology. i.e., trying to find the primary
[13:07:47] <atbe> I can clone the driver and grep the source code, any keywords to help narrow the search?
[13:20:04] <cheeser> CustosLimen: https://docs.mongodb.com/manual/core/shell-types/#check-types-in-shell
[13:20:19] <atbe> cheeser: So even before I use `new MongoClient()`, a `MongoClient` instanced is already created as long as I include the library?
[13:22:04] <atbe> So, is it safe to say that the Mongo Driver has some network discovery code included that runs as long as you include the library?
[13:23:03] <cheeser> there's no way for the driver to know what machines to talk to until you give at least one host in a constructor call on MongoClient
[13:48:43] <cheeser> i think that's as close as you're going to get. why does the type number matter?
[13:53:22] <jokke> i synced the db to a new secondary, swiched off the primary and want to elect the secondary to primary again
[13:54:01] <cheeser> you can't force a specific machine to become the primary but you can ask the current primary to step down
[14:01:11] <jokke> now my log is full of these messages: replSet couldn't elect self, only received 1 votes, but needed at least 2
[14:06:57] <nikitosiusis> https://docs.mongodb.com/manual/faq/concurrency/#how-does-concurrency-affect-secondaries this says mongo applies replication oplog in batches and doesn't allow reads while that. can I tune this somehow? When I apply many writes to primary, I get significant read time increase on secondary
[14:17:17] <nikitosiusis> reading from secondary is best practice as written in doc, that's why we are doing this
[14:22:38] <nikitosiusis> ok here is that https://docs.mongodb.com/manual/reference/parameters/#replication-parameters
[14:38:19] <nikitosiusis> but it doesn't seem to work errmsg" : "can't set replApplyBatchSize on a non-slave machine"
[14:54:16] <atbe> cool, for a standalone, will things like queries and commands be included in the log?
[15:00:25] <atbe> http://webcache.googleusercontent.com/search?q=cache:cxhFg7Wq57AJ:stackoverflow.com/questions/15204341/mongodb-logging-all-queries+&cd=5&hl=en&ct=clnk&gl=us
[15:14:14] <AAA_awright> Hey, I popped in here a while back reporting troubles with MongoDB eating 110% CPU sitting idle doing nothing, no I/O, no network, empty data
[15:14:37] <AAA_awright> It appears it was caused by a leapsecond bug fixed with a `sudo date` operation
[15:15:02] <AAA_awright> https://www.mongodb.com/blog/post/mongodb-and-leap-seconds claims MongoDB is leapsecond-safe, is this really the case?
[15:15:43] <AAA_awright> This was bothering me for weeks and I just gave up until recently. A restart may have also helped, but I can't easily do that since it's a production machine
[18:46:50] <boutell> Hi. I’m getting an “Overflow sort stage buffered data usage of 33559541 bytes exceeds internal limit of 33554432 bytes” error. I know this means I’m trying to do a sort() on something that isn’t indexed. And I’m gazing at the mongodb logs to figure out what. But while they tell me exactly what the query is, they don’t seem to tell me what the sort is at all. ): This is 2.6. Any thoughts on how to find out what s
[18:47:51] <boutell> Looks like I’m stuck increasing the limit in the meantime, but I know that’s bad practice.
[19:18:59] <cjhackerz> i tried to start the service as root also by using sudo non of them are working
[19:22:31] <cjhackerz> 2016-07-21T00:43:04.225+0530 W NETWORK [thread1] Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused
[19:22:31] <cjhackerz> 2016-07-21T00:43:04.275+0530 E QUERY [thread1] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed :
[20:19:40] <cheeser> this is not my area, but i would expect that the optimizer would attempt to move that $match before the $lookup. but if you're trying to $match on the values of the $lookup result, it couldn't do that of course.
[20:24:38] <xmad> "If you place a $match at the very beginning of a pipeline, the query can take advantage of indexes"
[20:24:55] <cheeser> it's entirely possible the optimizer is smart enough to detect that filter and apply it to the lookup but i doubt it.
[20:27:07] <xmad> Thanks so much. I think it's fairly safe to assume that $match doesn't use indexes if it's not at the beginning (given the wording from the doc). Maybe $match can be optimized for $lookup on later versions
[20:28:28] <JFlash> F CONTROL Failed global initialization: FileNotOpen Failed to open "/var/log/mongodb/mongod.log"
[20:33:05] <cheeser> JFlash: permissions problem on that file. you're not starting mongod as the correct user.
[20:34:57] <JFlash> Failed to unlink socket file /tmp/mongodb-27017.sock errno:1 Operation not permitted
[20:57:01] <sivli> Hi all, I have a quick question about aggregation and $out if anyone has a moment.
[21:02:11] <sivli> Just wondering what the best way to aggregate data to a constant collection w/o deleting existing docs.
[21:03:53] <sivli> I could just have it returned to my app server and then bulk insert but that seems a needless usage of network resources.
[21:04:42] <AndrewYoung> There's no easy way for mongo to know if it can cleanly insert all those records or not. If there are _id or other unique index constraints on the existing collection it would cause problems with the insert.
[21:06:50] <sivli> So what I am wondering is there a way to cleanly do a collection to collection copy or do I really need to send the results over the wire both ways?
[21:07:14] <sivli> The col to col option likely has the same issues as $out (assuming that is the reason for the limitation)
[21:08:44] <AndrewYoung> It would probably be better to do it the "naive" way by implementing it in code on your app server.
[21:09:21] <sivli> True but my mongo host (Atlas) likely does not have a way to do in locally. So I would have to do it over the wire.
[21:10:05] <sivli> Which I can... but is feels like such a waste. Thus here I am asking for other options :)
[21:12:34] <sivli> Ah well, will stick around for a while in case anyone has other input. Thanks for listening AndrewYoung :)
[21:13:13] <AndrewYoung> The only other thing I can think of is using mongoexport/mongorestore, but that will still go over the network in your case.
[21:47:19] <warp0x00> Nevertheless, systems running MongoDB do not need swap for routine operation. Database files are memory-mapped and should constitute most of your MongoDB memory use. Therefore, it is unlikely that mongod will ever use any swap space in normal operation.
[21:48:00] <warp0x00> but clearly that's wrong because linux doesn't swap memory-mapped files only anonymous memory
[21:48:20] <AndrewYoung> That bit of the documentation is talking specifically about the MMAPv1 storage engine.
[21:49:22] <AndrewYoung> However, the default storage engine for the version you're using is WiredTiger.
[21:53:50] <AndrewYoung> Changed in version 3.2: Starting in MongoDB 3.2, the WiredTiger internal cache, by default, will use the larger of either:
[21:56:36] <AndrewYoung> warp0x00: You might just need to adjust the WiredTiger cache size, but that's just a guess. https://docs.mongodb.com/manual/faq/storage/#wiredtiger-storage-engine
[21:57:10] <AndrewYoung> Yeah, if you put an array of tokens into the document instead of a pure text field, that would make your problem easier.
[21:57:53] <warp0x00> AndrewYoung, that fixed it. However, the documentation is out of date esp. re: "why is mongo using so much memory"
[21:58:12] <JFlash> how can I use aggregate to search for the most common items in a given array... do I have to unwind?
[22:01:31] <AndrewYoung> warp0x00: Is the wording here useful at all? (I'm asking about the documentation quality, specifically) https://docs.mongodb.com/manual/administration/production-notes/?_ga=1.27790261.866869998.1468255429#hardware-considerations
[22:02:38] <AndrewYoung> That would give you a count of how many times that token shows up in the list for that document.
[22:02:56] <AndrewYoung> No problem. I'm still relatively new to MongoDB, so take what I say with a grain of salt. ;)
[22:03:37] <JFlash> yeah.. see I dont need a number of x words per documment , I need a general number in all documents
[22:03:53] <warp0x00> AndrewYoung, yeah that's good. But, all of the answers on stackoverflow seem to predate WiredTiger and 3.2
[22:08:58] <warp0x00> if someone made a new SO/Severfault question it would likely just get marked as duplicate. I wonder if someone with lots of stack overflow points can go fix them or if there is even a mechanism of fixing answers that are wrong because they're out of date on SO