Log file Viewer
#mongodb logs for Tuesday the 12th of January, 2016
[04:51:30] <rk18> my problem is: i use the find method to retrieve a particular value from the database, and when i convert the cursor to an array and pass a callback function, i try to save the result in a global variable
[04:52:01] <rk18> however, when i console.log the global variable, the value of the variable that i assigned a value from the database to shows up as undefined.
[04:52:25] <rk18> any suggestions on what could be wrong? I console log outside of the .find method callback function
[04:55:15] <Boomtime> @rk18: this is in node.js? it sounds like you have a concurrency problem - the find probably hasn't completed at the time you check your global variable
[04:56:35] <rk18> i noticed the console of the global variables value occurs first before the console logs inside the callback, even though i check the value of the variable outside of the callback
[04:57:36] <rk18> _nextCounterVal = _counters.find({name:"main"}).toArray(function(err,db) { if (err) throw err; console.log('here is the damn d.b', db,db.value,db[0].value); res.send(db[0].value); }); console.log("counter val", _nextCounterVal);
[04:58:52] <Boomtime> @rk18: node is event driven, you don't issue commands directly, you 'stack up' things you want done and the callback function is called when that task completes - what would be the point of the callback if the operation was complete immediately?
[04:59:24] <Boomtime> please note, this isn't mongodb related - you are learning node.js by the solunds of it
[08:17:44] <m3t4lukas> hey guys :) I wanted to ask whether anyone of you already did a mongodb connector for postfix and dovecot?
[08:23:47] <m3t4lukas> it seems to be more complicated since db_common is not really made for working with anything else but SQL.
[08:29:34] <m3t4lukas> but what needs to be done needs to be done. It can't be that there is no such thing like no mails stored in mongodb :P
[10:53:01] <themroc> hi, i guess you already know that the packages signatures are invalid on the debian repository at http://repo.mongodb.org/apt/debian/dists/wheezy/mongodb-org/stable/ ?
[11:04:07] <themroc> ah yes that's https://jira.mongodb.org/browse/SERVER-22144 ; packages signatures were fine yesterday before the release of 3.2.1, so i guess this is related to the release of the new version.
[11:47:11] <aps> I'm really disappointed with recent MongoDB releases. Normally, one would hope that upgrading a package would make it better. It gets worse in this case. I upgraded my prod mongo servers from 3.0.8 to 3.2 and all hell broke loose.
[11:47:56] <aps> In one replica-set, mongo 3.2 keeps crashing due to OOM every few hours - https://jira.mongodb.org/browse/SERVER-22000
[11:48:44] <StephenLynx> afaik, unless you manually move your db to WT, it keeps using the previous engine it was using.
[11:50:26] <aps> In other replica-set, I just added a secondary of version 3.2 to a replica set running 3.0.8. It sky-rocketed the "number of connections".
[11:51:42] <aps> Right most is Mongo 3.2. Second one is primary member. https://usercontent.irccloud-cdn.com/file/RHaLSR04/Screen%20Shot%202016-01-12%20at%205.14.17%20PM.png
[12:05:56] <lipiec> 2016-01-12T10:25:41.264+0100 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongod_cache/diagnostic.data'
[12:06:50] <StephenLynx> is it coincidence this many people upgraded to 3.2 recently or something happened?
[12:11:54] <aps> I was asked to upload my diagnostic.data to the ticket I filed. I find it weird that I couldn't even see exactly what data I am uploading.
[12:12:37] <lipiec> StephenLynx: But main reason to upgrade was ability to do partial indexes (lots of space can be saved with that feature).
[12:13:16] <lipiec> aps: maybe there is some tool from mongo-org-tools or command in mongo shell to read this?
[12:14:34] <aps> lipiec: I asked around for that, but I was told there's no such publicly available tool. Again weird.
[12:14:55] <Derick> lipiec: https://docs.mongodb.org/manual/release-notes/3.2/#diagnostic-data-capture
[12:17:11] <lipiec> Derick: thanks thanks for that, my fault not to search by search on docs site but only with google...
[12:19:35] <lipiec> Derick: strange, maybe it's because of i'm using polish version of google? https://www.google.pl/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=mongodb+%22diagnostic.data%22
[12:31:32] <lipiec> aps: what i've found by myself that those files are bson valid documents, you can read them with bsondump utility
[12:32:06] <lipiec> aps: but i can't read what's inside those objects... cause they are some like encrypted...
[15:22:11] <saira_123> hi boys i want to do a research on mongodb 3.0 performance to write a research paper . can someone guide me what metrics should i use for performance ? any research paper i can refer to?
[15:43:50] <metame> saira_123: https://docs.mongodb.org/v3.0/administration/analyzing-mongodb-performance/
[15:44:55] <metame> saira_123: also just as a friendly aside, it may not be taken well addressing an anonymous group of tech professionals as 'boys'
[16:10:47] <saira_123> Waitak thanks , actually i am looking for an academic research instead of search
[16:12:18] <jetole> Hi guys. I am using db.grantRolesToUser() and than db.getUser("userName") and after I see the permissions in getUser() and than a moment later when I re-run it, the permissions are gone. Can anyone please help?
[16:14:59] <Waitak> saira_123: One nice technique is to find an earlier article that's in the ballpark, and to look for others that cite it.
[16:15:53] <saira_123> Waitak thanks i am checking "Performance Evaluations of Document-Oriented Databases Using GPU and Cache Structure"
[16:16:15] <saira_123> Waitak my problem is the machines i am going to evaluate performance on has Solaris and ZFS filesystem
[16:17:29] <Waitak> saira_123: Well, if you're in new territory, the answer might just be to design the experiment yourself.
[16:19:32] <Waitak> saira_123: My pleasure. If your focus is Solaris and ZFS, perhaps you could start with some of the literature on other sorts of performance testing on that platform, and see if you can apply it to testing MongoDB.
[16:21:48] <Waitak> saira_123: Yes, but performance in a database is all about minimizing disk accesses - particularly avoiding seeks. Performance testing in *any* database, relational or not, boils down to how well you're managing disk access.
[16:22:43] <saira_123> Waitak correct and with mongo this means how well we are managing different types of disk cache like arc cache and page cache
[16:22:50] <Waitak> saira_123: If you have a clear understanding of the idiosyncrasies of ZFS on Solaris regarding general IO-centric performance, you ought to be able to apply that to MongoDB.
[16:25:57] <Waitak> saira_123: I'd focus on that first. (As an aside, I haven't had many conversations like this since I was in academia, which has been a while. This is quite enjoyable.)
[16:56:07] <the_voice_> Am I right in my understanding that dot notation with the $ is only for the update query
[16:56:23] <the_voice_> and that you can have regular dot notation in the find(when you are looking for objects inside an array)
[17:00:22] <saira_123> Hi Technical professionals if someone is testing mongodb performance , is there any free or 10gen provided server resources where one can test performance freely?
[17:14:45] <magicantler> when i --fork mongos, i'm finding that shell control is not being returned. is there a good workaround?
[17:39:04] <saira_123> hi professionals , i just finished reading a whitepaper on mongo website ,Comparing Couchbase Server 3.0.2 with MongoDB 3.0
[17:39:06] <saira_123> I feel author did not fairly compared with Mongo, with 9 active Couchbase nodes he only used 3 active Mongod primaries
[17:44:31] <saira_123> I feel i can improve this white paper by using different mongo architecture, but i dont have server (hardware) resources
[18:07:37] <blizzow> I have three mongo config servers in a sharded replicated cluster. When one of my configs goes down (for upgrade), many of my mongos servers start to wig out and take forever to connect to mongo. I'm assuming mongos is somehow setting a preference to which config server it should connect to. A) am I correct in this assumption? B) How do I get mongos to start querying a different config server faster?
[18:09:35] <saira_123> blizzow the other mongos must be just complaining about the absence of 1 config server but should be performing faster as before
[18:09:46] <idd2d> If I have the following schema, how would I query for the user IDs within a group? http://hastebin.com/avahuhizid.json
[18:47:04] <mapc> when I try it, I get an error "already in use", so I figure mongo uses no internal locking for cursors
[18:52:25] <mapc> I just started around ~100 process trying to read from the same cursor and a couple of them got data
[18:52:42] <mapc> but now I am unsure whether this is horribly unsupported and whether my app will break in two weeks :)
[18:53:05] <cheeser> so it's not about sharing between clients. it's about sharing a client/cursor among threads.
[18:53:47] <mapc> cheeser: well I am in python and the goal is to use multi threading, so I can't use threads
[18:54:40] <cheeser> can't help you there. It'd be trivial in java if you feel like rewriting your entire application. :D
[18:55:02] <mapc> So I actually need to opem multiple mongo connections, but of course I still can do locking to ensure only one client at a time does access
[18:55:29] <mapc> cheeser: The first stage of the processing I do wasin C++ and that seems outright simple :D
[18:56:02] <mapc> but yeah, is it defined behaviour to access the cursor from multiple clients, doing my own locking?
[18:59:06] <cheeser> you want one cursor and then serialize access to that cursor between threads/workers
[19:03:04] <mapc> well, I am using mongoengine; the hacking required stuff a different cursor ID into Mongoengine queries is probably less than replacing the entire serialization layer
[19:34:39] <sweettea> i put in the default homedir created when installing via yum install mongodb-org when using repos from mongo:
[19:59:54] <mparm920> Hi I am trying to repair a corrupted database. Upon starting the database I am getting this error 'WiredTiger.turtle: encountered an illegal file format or internal value' I have tried mongod --repair a few different ways and wiretiger wt salvage and help would be apprciated
[20:10:56] <mapc> are interested: python serializing & deserializing actually needs more CPU than the processing I am doing. Thanks pythong :D
[21:06:36] <GothAlice> https://jira.mongodb.org/browse/SERVER-15815 T_T Keeps getting bumped. So sads. 2.6 → 2.9 → Backlog'd → 3.3 → Reschedule.
[21:08:40] <GothAlice> mapc: If you're careful in your BSON document construction, you can map BSON blobs directly to packed C structures and avoid deserialization completely. :P
[21:10:31] <GothAlice> Er, not advised, but technically possible. Just avoid variable-sized types, or only have the last element variable sized. :P
[21:11:24] <mapc> GothAlice: Interestingly, my application is conversion from Dbase, so I can in general arange a fixed size, because my input data is entirely fixed size
[21:11:51] <mapc> any idea whether this is possible the other way around? My application spends most of it's time encodeing bson
[21:12:44] <GothAlice> Ah, MongoDB doesn't have a CHAR type, only variable-length (often Pascal-style length prefixed) UTF-8 strings. If any of your data is text, fixed sizes across the board becomes much harder to manage without explicit padding.
[21:14:13] <mapc> GothAlice: true. I used rapidjson for some time, but then I moved to mongocxx. My applicating suddenly took 10x as much time :(
[22:38:15] <Waheedi> metame: https://docs.mongodb.org/manual/tutorial/build-indexes-on-replica-sets/#build-the-index
[22:38:35] <cbx33> Hi all, need a little help with aggregate, I have a number of records that have identical "name" fields, and they have a "status" field. What I want to return is the latest, there is a "finish_time" field, record for each unique "name"
[22:39:41] <cbx33> si imagine two records {'name': 'a', 'status': 'failed', 'finish_time': '3'} {'name': 'b', 'status': 'failed', 'finish_time': '4'}
[22:41:06] <metame> Waheedi: also remember ensureIndex was deprecated in 3.0. Should be using createIndex if >3.0
[22:48:21] <cbx33> metame, i wondered......but i need it to return the latest object with ALL it's fields
[23:00:08] <metame> unless you have an index on finish time you should just skip the sort and use max
[23:04:20] <metame> as in you're aggregation is going to end up with multiple fields taking the max/min of different fields from your original schema
[23:04:51] <cbx33> which is why I wondered if there was a way to return the "first/last" object as a whole
[23:08:16] <cbx33> that's not right.....the only field that should be used to "sort" is the finish_time
[23:12:29] <metame> you're doing a $match, then $sort, then $group and grabbing all the fields for the $first doc in that group
[23:28:24] <cbx33> metame, that {"$group": {"_id": "$name", "finish_time": {"$first": "$finish_time"}}} will group by the "name" field right?