Log file Viewer
#mongodb logs for Wednesday the 13th of August, 2014
[00:05:39] <Boomtime> ah.. i see, he didn't get an answer to his issue inside of 6 minutes, fair enough
[01:00:10] <cygnet> where each hospital using our system will have their own mongod instance in their on-site servers
[01:06:36] <cygnet> would the database connection be a very slow setup? would it take up many resources on the client/server side?
[01:09:00] <cygnet> so what exactly would be the difference between creating many shards and creating a separate db per location?
[01:09:33] <cygnet> would that mean that information pertaining to that location is not necessarily stored at said location?
[01:10:26] <cheeser> if they're not going to be on shared infrastructure then clearly 1 db per "site" is better
[01:10:27] <cygnet> ex: all of a hospital's patient information, would it be saved only at that hospital with sharding, or spread in most convenient manner across all shards?
[01:12:35] <cygnet> to have a network of institutions, working on a multi-instance network, their data being saved only client-side, on their own instance
[01:15:45] <Boomtime> ok, so seperate instances is looking like a better option.. but there are more considerations
[01:16:05] <Boomtime> have you thought about data resiliency? do you want patient information to be copied real-time to a remote location?
[01:19:03] <Boomtime> sharding is unlikely to be useful to you in your case. you would only use it if you wanted a cohesively accessible data-set that spanned all hospitals in the network, since you don't want that, you're probably better off with simpler deployments
[01:19:16] <cygnet> not real-time, because of the issue of reliable connectivity in some hospitals. During offline periods, data is stored locally, then transferred to the hospital's instance when connection is reestablished
[01:26:45] <cygnet> but the issue we're concerned about is how creating a new instance will affect the rest of the system
[01:27:12] <cygnet> would it make sense, for example, to create a database per department, or per hospital?
[01:36:12] <cygnet> so the biggest concern when creating a new hospital's database is ensuring it would have enough hard drive space
[01:42:20] <cygnet> is it expensive in these regards, or is the expense of RAN and bandwidth negligible?
[01:42:52] <cygnet> actually, more importantly, would mongo use these resources more than another db?
[02:01:01] <themime> using java (and console), im having a hard time getting a filter on a single day working, can i not just put a year-month-day as a string in there? do i have to compare gte le of the day i want and the later day?
[02:02:20] <cygnet> @joannac: it's highly variant; some hospitals currently have a team of ~50 doctors, with ~1600 patients, will have ~200 doctors with ~8000 patients, some will have an entire hospital (as high as 500 doctors)
[02:19:29] <kyle_l2> Howdy, folks. I have a simple question. Are write commands sent to a secondary member of a replica automatically routed to the primary? Forgive the potential obviousness of this question; I have little experience with Mongo.
[02:28:42] <lambdta> How can I use mongoimport to import a csv file and select the columns which I want to import?
[03:05:31] <Boomtime> @kyle_l2: "write commands sent to a secondary member of a replica ..." -> are rejected, you cannot write to a secondary
[03:16:08] <themime> anyone know why { "createdOn" : { "$gte" : "2014-08-12 04:00:00" , "$lt" : "2014-08-13 04:00:00"}} does not match date "2014-08-13 03:09:41"
[03:22:38] <joannac> SahanH: http://docs.mongodb.org/manual/reference/operator/aggregation/last/#grp._S_last
[03:52:20] <themime> gabbles: new to mongo but in general for dbs it depends on what data youre storing and why
[07:02:48] <ezonno> Hi I am seaching for a solution for a problem, I have this data https://gist.github.com/ezonno/11ee956df7705dc2588a I want to make the documents containing the gpsPosition subdocuments of the documents containing the obdEot value but the ts (timestamp) of the subdocument should be between the parent ts - duration
[07:03:36] <ezonno> Can I achieve this with mapreduce or will it also be possible to do this via the aggregation framework?
[08:35:44] <jillesme> Hey guys, is there a difference between name: { type: String } and name: String?
[09:00:17] <rspijker> in the first case, name is a field containing a document, in the second case name is a filed containing a value
[09:14:55] <jillesme> rspijker: well if you look at this article http://blog.modulus.io/getting-started-with-mongoose
[09:41:57] <beebeeep> hello folks, i have some question about how mongodb stores sharding metadata, here is my code http://pastie.org/9469498
[09:44:57] <beebeeep> can anybody explain what is correct way to find out the chunk which stores particular document?
[09:47:12] <BurtyB> beebeeep, assuming you didn't typo your server names/shards (like I did a while ago).. you can end up with orphan data if a migration fails. iirc there's some javascript to help on the mongod site but last time I just wrote a script so I could check things was on both sides before deleting the orphan
[09:59:27] <beebeeep> rspijker: i have 2.4.9 and i'm terrified by thougths that someday i will have to update it :)
[10:01:46] <rspijker> this is probably the script that BurtyB was talking about: https://github.com/mongodb/support-tools/tree/master/orphanage
[12:43:13] <remonvv> Which is about as effective as standing in a forrest and list my objections but hey.
[12:45:15] <rspijker> well… I can see why you would care if embedding would become an “under the hood” reference
[12:46:21] <remonvv> rspijker: Consider the impact it would have on things like the query language, the amount of development resources it would divert from what I consider higher priority issues, the fact that it would basically move MongoDB to RDBMS space.
[12:48:20] <rspijker> remonvv: I agree with the diverting of resources part, the rest, not so much. Would depend very heavily on development choices made
[12:48:25] <remonvv> cheeser: Was that one of those "better get used to the idea we're already on it" sort of you never knows?
[12:55:29] <remonvv> rspijker: I recogize it's a highly subjective discussion but catering to the "I could do this in MySQL but not in MongoDB, fixplz" crowd seems like a dodgy route. I don't think there are as many valid arguments for adding proper relational support to something like MongoDB as people assume there are.
[13:02:47] <kali> i'm more or less convinced it what make the relational system collapse under its own weight, more than relations themselves
[13:05:41] <kali> mmm... distributed fine/coarsed grain locking all over the place in the pretense of achieving ACIDity
[13:06:23] <remonvv> Hm, well, they'd have to put various constraints on which entities can participate in a specific transaction or live with the fact that it's mutually exclusive with linear scalability.
[13:07:23] <remonvv> cheeser: As in, practically speaking what could you do with that that you cannot do already with about as much effort.
[13:08:32] <remonvv> cheeser: If you can live with non-sharded MongoDB you can live without transactions in the vast majority of cases.
[14:14:00] <uehtesham90> mongo course --eval "var c = db.track.find({'event_type' : {'$regex' : '^/courses/progress'}});while(c.hasNext()) {printjson(c.next())}" >> progress_event.json
[14:30:42] <elux> what kind of index would be appropriate for a query like: db.content.find({ $query: { site_id: ObjectId('000388030000fdf6cab527c9'), $or: [ { src_url: "http://www.inc.com/anita-newton/customer-development-the-biggest-lies-are-the-ones-we-tell-ourselves.html" }, { src_uid: "237624761" } ] }, $orderby: { _id: 1 } }).limit(1)
[14:31:27] <elux> i have indexes {site_id:1, src_url:1}, {site_id:1, src_uid:-1}, {src_url:1}, {src_uid:-1} .. and it doesnt seem to be doing any good
[14:31:47] <elux> the mongodb log is showing planSummary: IXSCAN { _id: 1 } ntoskip:0 nscanned:3619716 nscannedObjects:3619716
[14:34:46] <elux> running this in the mongo cli, it looks fine.. and .explain() true shows it using the right indexes
[14:35:14] <BurtyB> uehtesham90, I'd normally switch the quotes around (so '' outside and "" inside)
[14:36:43] <uehtesham90> BurtyB: lol thats weird....its working now....but why is that so? is it a javascript this?
[14:38:11] <rspijker> elux: mongo uses all plans at some point, to determine which is best… so depending on where you;re reading that exactly, it might be deceiving...
[14:57:58] <Shakyj> can I give mongod an arg to locate it's config file? I have it in /etc/mongod.conf but it's not picking it up
[15:12:37] <dmitchell> I put in a bug report (https://jira.mongodb.org/browse/DOCS-3888) on Monday where cursor.count() was returning a non-zero number but the resultset is empty. It's miscategorized b/c it's against 2.2 and that isn't one of the options. idk if anyone will have a chance to verify it, but I thought at a min it may be useful as a "known problem" if you fi
[15:21:10] <borkdox> I am getting an uncaughtException in my NodeJS app using mongodb 2.6.4 with 1.4.8 driver. The error originates from the mongodb driver. The error thrown doesn't contain any stacktrace or message, it's just an empty object. I had to use stackup to trace the error.... Here it is: http://pastebin.com/S0TjvkXN Any suggestions regarding this? Submit bug report?
[15:50:03] <uehtesham90> i want to look for a certain field in the database that matches one of a given list of regexes....i cannot use $in with $regex....is there another way to do it?
[15:51:34] <uehtesham90> instead of doing an or of all 5 elements i want to just a list in my pymongo code
[15:52:59] <uehtesham90> rspijker: ur suggestion gave the following error: Can't canonicalize query: BadValue $regex has to be a string
[15:53:40] <rspijker> uehtesham90: this works for me: db.coll.find({"field":{$regex: [/^9/, /^1/]}})
[15:54:38] <uehtesham90> u will most probably get this error: Can't canonicalize query: BadValue $regex has to be a string
[15:58:33] <uehtesham90> this is my query: db.tracking.find({event_type : {$regex : ['^/courses/info$','^/courses/progress$']}}).pretty()
[16:00:41] <uehtesham90> how can i rewrite it to the /../ notation bcuz i have forward slashes in my regex experession too as u can see
[16:01:37] <uehtesham90> but in the /../ notation, how to i take care of forward slash in my expression?
[16:05:20] <uehtesham90> the /../ notation is working with the $in operator in find....without the regex operator
[16:20:12] <uehtesham90> i dont know if u have ever used pymongo, but do u know how to change a string to raw literal so that i can pass it to my in operator?
[16:20:41] <uehtesham90> as in, if i have ['test/qa', 'test/qb'], i want the raw literal of each string
[16:45:42] <Kenjin> Looking a this setup: http://pastie.org/private/eyc4jsmfaahegqdmzncqvq , how would I accomplish searching for a Directory by name for instance and getting back the directories together with their files and the symbolic links that point to those files?
[16:57:12] <DubLo7> 2014-08-13T12:47:02.653-0400 { "ok" : 0, "errmsg" : "unauthorized" } at src/mongo/shell/db.js:402
[18:40:29] <putty174> Anyone know why I get Default constructor cannot handle exception when I try to make a new MongoClient?
[18:43:14] <cheeser> yeah. the MongoClient constructor throws an exception and you have to handle that.
[18:43:19] <wavded> I'm getting this error "BSONObj size: 39925696 (0x26137C0) is invalid. Size must be between 0 and 16793600(16MB) First element: type: "KEEP_MUTATIONS" when performing searches. My googling told me to repair the database. I did that, it's back. I'm running a replica set. Any ideas?
[18:54:38] <starlord> Is MongoDB appropriate when I want 99% of the time I'm doing queries, but occasionally (every few minutes) update some count-keeping records or user-account records?
[18:55:10] <starlord> It's essentially for a web store that has a few products, a concept of a user account, orders, carts, and statistics about how often web-products are downloaded.
[18:59:40] <starlord> kali: We moved to Datomic because I wanted to take advantage of their "for free" caching. But in reality we just need to make fewer redundant DB calls per web request.
[19:01:25] <kali> starlord: well, it depends exactly what data you want to have in mongodb. but dealing with complex "transactions" (as in buying something, marking it paid, marking it available, etc) is tricky, because mongodb does not offer atomic multi document updates
[19:02:12] <kali> starlord: so you have to do build on findAndModify and the atomic operators to make it happen. it is exactly what a rdbms does behind the scene, but in that case, it's your app that will have to do it
[19:02:26] <wavded> kali: i just noticed the primary switched to another member of the set that hadn't repaired the database on, do these issues you'll infect the whole replica set?
[19:05:40] <wavded> kali: yeah, basically i'm doing a "join" finding on one collection and $in by id on another collection, if the ids become too huge could that cause the issue? the id property is indexed
[19:13:31] <kali> wavded: is it strictly the same query that was broken then working and then broken ? or just queries of the same species ?
[19:13:32] <wavded> that's doable on the application side, not ideal but doable. does mongodb then have a 16mb limit for "queries"?
[19:16:47] <wavded> kali: that same query has broken before, but variations on the theme have broken as well
[19:20:24] <kali> unless i'm mistaken, 255k... id is 12 bytes, plus the inefficient array encoding... 1 byte for type, 6+1 for the string, 255*(12+1+7) that's about 5MB
[19:57:20] <ngoyal> in mongodb 2.4 when inserting an array of documents, if an error occurs, is there a way to determine what documents were successfully inserted as a result of that operation?
[19:58:59] <wavded> kali: same query after i repaired works, so donno, all have been repaired (only 3 in the set) so maybe good
[19:59:28] <ngoyal> that is what i gathered as well. and this feels obvious, but maybe im wrong. it's probably more performant to do a single bulk insert rather than issuing a lot of single inserts in parallel correct?
[20:03:18] <pratikbothra> Hey guys, how do you handle updating populated objects in mongodb? Take the use case of likes in post (likes: [{type: Schema.ObjectId,ref: 'User'}])
[20:03:18] <pratikbothra> While retrieving, I use populate and get the user name, link along with their id's. But on updates back to mongodb, I'm facing a lot of trouble as mongodb is expecting an array of id's, while I'm sending it populated objects back. What is the best way to handle this?
[20:05:13] <pratikbothra> I faced a similar problem when users have to join groups as well. The way to handle it is just not clean, and I don't want to resort to hacks at this stage. Any suggestions would be great?
[20:13:55] <joer> If I have a collection within which each document stores an array of its child documents (an array of document ids of other documents), is there a way to fetch an array of all children of a particular document. All children being children of children etc too
[20:19:00] <pratikbothra> joer, can't you just do flatten if you don't care whose parent the child belongs to?
[20:22:11] <pratikbothra> I mean a javascript equivalent of the flatten function. Just fetch all the children (includes children of children etc) of the required document, and then flatten it with javascript.
[20:23:34] <pratikbothra> Of course, this won't work, if the child parent relationship is also required. Note: Its late, here and groggy. Ignoring me, might be a good thing too. :-)
[20:25:24] <joer> Might work out more efficiently to be honest than querying it and constructing it every time. Thanks
[20:27:43] <pratikbothra> I'm too tired, and leaving for the day....Would someone take a quick peek, at my question posted above, and tell me how they handle it?
[20:29:33] <pratikbothra> Retreiving an array of populated object is great....Saving/Updating it back a nightmare. Mongodb want's only ids while saving it....Grrrr
[20:33:58] <ngoyal> pratikbothra: i dont know of any easy solution other than to map your object array to _ids..
[20:39:47] <pratikbothra> ngoyal: On post update, I do something like this var post = req.post; which is followed with _.extend(post, req.body); ....Now imagine handling post.likes (both likes being added and people removing their likes)
[20:47:11] <jonyfive> is there any way to obtain the current user's username from within .mongorc.js
[21:36:18] <daveops> jonyfive: db.runCommand({connectionStatus: 1}).authInfo.authenticatedUsers[0]['user']
[21:44:27] <jonyfive> daveops: thanks, that just results in: Cannot read property 'user' of undefined
[21:53:54] <ngoyal> what is the release schedule like for 2.6 patches - i.e. is there a regular schedule of updates for 2.6.5, 2.6.6, etc? or is it as needed?
[23:15:25] <salakalaka> joannac: what about running a javascript function on each record before it is inserted?