Log file Viewer
#mongodb logs for Tuesday the 7th of July, 2015
[02:18:03] <topwobble> how can I tell if a secondary is up to date with the primary? I just had it offline for 3 hrs and need to know when it is synced
[02:19:10] <topwobble> it seems like it is serving read requests, which I am worried about since i know it cant be up to date
[04:06:05] <cphoover> Hello I'm working with the node.js mongo driver. I am operating on a cursor. What I want to do is skip an arbitrary number of documents in a collection then read the next object and then skip a number of documents then read the next object... When I attempt to do this I get the error "MongoError: Cursor is closed" maybe I'm misunderstanding 'skip' but can it not be used after the cursor has been moved?
[04:11:45] <Boomtime> cphoover: yeah, that isn't very clear but skip() is actually part of the command that went to the server - you only get to use it at the start, once you start consuming the results, that method is not available
[04:12:10] <Jonno_FTW> I have a problem, I can connect to a local instance of mongodb, but I can't connect to it remotely even though I used the same command line params
[04:12:19] <Boomtime> treat skip() as an addendum to the query (because it is), you can only issue it once
[04:13:28] <Boomtime> heh, yeah :p , 'auth failure' not connect failure... might be the localhost exception which cares not about auth attempts with no users present
[04:15:00] <Jonno_FTW> Failed to authenticate penny@pennyParking with mechanism MONGODB-CR: AuthenticationFailed MONGODB-CR credentials missing in the user document
[04:15:54] <cphoover> Boomtime: Ok thanks for the quick reply. So is there no way to do that? Or perhaps I just need to issue several distinct queries
[04:16:50] <Boomtime> you could knock the server down to using MONGODB-CR but you're better off moving into the future than into the past
[04:17:39] <Boomtime> i'm not sure.. just check for SCRAM-SHA1 support - it's the only thing that will be a roadblock
[04:18:10] <Boomtime> pylua, you issue a remove on the collection, specify the _id of the doc you want removed
[04:37:17] <limbera> would anyone here know if parse.com (which uses mongodb) are encrypting their BSON files?
[04:38:16] <cheeser> to my knowledge they aren't. they're mostly on mmapv1 with about 25% on MongoDB/RocksDB
[04:38:37] <cheeser> not sure about the rocks file storage system but mmapv1 doesn't support encryption at rest.
[04:39:58] <cheeser> check the website, though. i only have a passing knowledge of the rocksdb storage system
[04:39:59] <limbera> how is relying on OS/files system auth/security any different from doing the same with an SQL variant
[04:40:14] <limbera> yeah i've scoured the internet and no mention of encryption so i think you're right
[07:21:26] <nickenchuggets> having a problem with mongodb, I gathered some details from syslog, https://gist.github.com/brlafreniere/cd7b291ac9ece484e48f
[07:25:36] <nickenchuggets> that... kinda sucks, the apt repo that mongodb provides appears to be compatible only with Debian 7
[07:37:17] <nickenchuggets> I tried to install mongodb via the 3rd party apt repos, which is said to be for debian 7 wheezy, yet I'm using debian 8 jessie... and I'm getting all kinds of errors and stuff, so I'm guessing that's not gonna work...
[08:03:43] <Boomtime> tejasmanohar: what is not recommended? if you have a value in your document structure that is unique and already exists, you should totally use it as the _id field
[08:04:44] <Boomtime> if you don't supply it (or use the value that is supplied for you), then there's a whole indexed field, of 12 bytes per document that exists for no reason
[08:06:28] <the_voice_> users.update({column6: { $elemMatch: { _id: id } } }, {$set: { "friends.$.name" : name } })
[08:06:53] <the_voice_> users.update({friends: { $elemMatch: { _id: id } } }, {$set: { "friends.$.name" : name } })
[08:07:13] <the_voice_> I am new to mongo and trying to understnad if searching a subdocument like that would have any performance implications
[08:08:12] <Boomtime> the_voice_: nothing is slow if it has a covering index, the further from being covered by an index that a query gets, the slower it gets
[08:09:47] <the_voice_> I just realized though it;s only updating one row and not another row where that element exists in an array :(
[08:11:01] <the_voice_> it is not updating the second document as it has more than one element in the array
[08:11:53] <Boomtime> only one entry of an array is updated - the $ operator can only address a single match
[08:12:48] <bogn> so it's simply updating the first embedded document, how many levels is that deep?
[08:14:11] <Boomtime> your document must look something like this: { friends: [ { _id: ... }, { _id: ... } ] }
[08:14:49] <Boomtime> the sub-document which caused that document to be matched in the predicate, is the one that will get addressed by $
[08:17:01] <the_voice_> if you index an array hten it automatically indexes both columns of the subdocument or you index both the array and the _id part of the subdocument
[08:18:25] <Boomtime> when you index an array, the precise values of that array are what get indexed
[08:19:17] <the_voice_> as one of the columns, but Iw ould want to index that array and be able to search by _id
[08:19:36] <the_voice_> I haven't yet I am working in dev so none of my tables have more than 10 entries right now :) I want to know how I should index that
[08:20:19] <Boomtime> if you want to search by the _id field inside the subdocuments, you need to index that -> ensureIndex( { "friends._id" : 1 } )
[08:47:49] <synthmeat> (i'm trying to predict my memory requirements for mongo, presuming known concurrent load and document sizes)
[12:10:53] <the_voice_> Is there a way to set a value of a field in a document as the result of an aggregate of one of the subdcuments?
[12:12:36] <the_voice_> so for instance if I had a document user= {_id:1, name: bob, friends: [{_id:2, name:dave},{_id:3, name:jo}]}
[12:13:14] <the_voice_> is there a way I could do an update that would create a third column from the subdocuments that was called friendsList: "dave,jo"
[12:15:00] <StephenLynx> "is there a way I could do an update that would create a third column from the subdocuments that was called friendsList: "dave,jo"'
[12:20:28] <the_voice_> anyways for typeahead I want to be able for them to start with parent then the next one etc...
[12:20:55] <StephenLynx> when displaying the categories, I would perform a single query with the array of ids
[12:24:03] <StephenLynx> so I don't have to do even that. But I reckon that is not a viable solution when you may have multiple objects with the same readable name.
[12:32:02] <StephenLynx> and thinking about that, I was wrong about it being reasonable for two in's too.
[12:33:55] <adsf> but as a noob you can always twist what the doco says into something not all that good :
[12:52:14] <leporello> Hello. Who's using mongoose? How to make a query inside of pre('save') trigger?
[13:06:00] <cheeser> i'm not sure (i'm not a python guy) but you should just be able to put a python date in your document and the driver should convert it for you
[13:07:13] <pylua> cheeser:I usually need to do query at mongo shell command line ,so I need to use ISO date instead
[13:07:59] <cheeser> in java, we just use java.util.Date and the driver converts. pymongo should do the same.
[13:27:51] <jecran> Hi guys. How can I wait for all my mongo processes to finish so I can exit my program? It currently just hangs, and I dont have an option to manually close, because I have no idea when its done lol
[13:28:35] <jecran> I have read that node closes the mongodb connections manually. But mine are not closing, and my node application does not exit when done
[13:29:24] <jecran> how about just mongo, is there maybe something that can tell me when the connections are closed, or when they are done connecting online?
[13:36:23] <StephenLynx> but when I close the connection, io realizes it and shuts down. assuming nothing else is hanging.
[13:37:30] <jecran> When I try to close my connection, I miss out on about 5-10 records out of 100 .. even using callbacks
[13:41:49] <jecran> 2 ---- the tricky one . I can tell when I am done reading the rss feeds, at which point mongodb is still a few records back, and I don't know how to tell when its down
[13:43:10] <jecran> I use wtfnode currently, which lets me know that I have the connections still open. Handy little tool
[13:43:15] <basiclaser> I'm making a dictionary web application. It serves a list of words, based on a string search, and the word definitions along with synonyms. I want the user to be able to request 'tiger' and then also recieve 50 words before and after that word, alphabetically. The user can request two different versions of the dictionary, one with certain words removed.
[13:43:16] <basiclaser> Which, if any, database and distribution system could/should I use in the backend express app to store and distribute the dictionaries? Is there an optimal way of loading subsections of a list of words like this? I currently have the words stored in files in javascript arrays.
[13:43:44] <StephenLynx> then I would have all my incoming requests and outgoing responses funnel through a single place, put a simple counter of active requests, if the flag of shut down is true and no more pending requests are pending, shut the worker down.
[13:44:53] <StephenLynx> one for when it is killed before response and another for when you finish it.
[13:54:52] <jecran> StephenLynx: my simple solution is: check the error, if the error tells me the sockets are closed, reopen the connection. Not efficient, but no counting and only used on the last few results
[14:21:46] <puppeh> does this info still stands for the version 2 ruby driver? https://github.com/mongodb/mongo-ruby-driver/wiki/Replica-Sets#recovery
[14:49:14] <makzen> Hello, I have a last_active date and a created date in my collection documents. I want to find all entries where last_active and created are not on the same day. Can somebody give me a hint?
[16:37:15] <lmatteis> hi all. i have inherited a codebase which seems to run aggregate functions every day or so
[16:37:46] <lmatteis> like for example in couchdb, all you need is the view, and it updates automatically when the data changes
[16:40:59] <cheeser> well, you run your pipelines and write out the results... cron that job and you're done.
[16:43:21] <joelkelly> anyone have any experience with populate on mongoose? Trying to figure out if it needs to be established when the data is passed in or if it can be used on existing documents
[17:05:15] <symbol> I'm still trying to wrap my head around proper use cases and schema design in Mongo...it seems to be that Mongo would not be a good choice for something like twitter, correct?
[17:05:50] <symbol> Followers/following probably can't be embeded since the size can grow to large numbers at times.
[17:06:48] <StephenLynx> the linked people in tweets is dynamic, probably because it uses a join or something.
[17:11:19] <symbol> I feel like every resource I've been studying talks about mongo and its speed with big data...so it seems that it's mostly good with big data that can be embedded.
[17:16:07] <StephenLynx> no, I just keep the array of owned boards in the user and update it when it changes.
[17:19:14] <StephenLynx> in my forums you can't change a board uri or an account login, for example.
[17:22:33] <StephenLynx> so performing these tasks, even if they are really slow, it wouldn't matter.
[17:22:50] <StephenLynx> either because your load is small or because its so large that it doesn't matter in face of how often you do it.
[18:14:22] <ab2qik> Hi, can a search be done to find all collections in a db? e.g. use demo. Then do db.find(). Ofc this does not work?
[18:15:42] <StephenLynx> if you need that on your application, you are probably using dynamic collections, am I right?
[18:48:17] <a|i> in a one-to-many relationship where an array of references to ‘many’ is stored on the ‘one’ side, how would pagination work? keep deserializing the reference array from db and pick the next page ids?
[18:49:42] <StephenLynx> it is much saner to keep the reference to the "one" as a field in the "many"
[18:50:45] <a|i> StephenLynx: http://blog.mongodb.org/post/87200945828/6-rules-of-thumb-for-mongodb-schema-design-part-1 the ‘one-to-many’ option.
[18:51:49] <a|i> the ‘many’ side would be of a few hundred only, but it needs to be paginated by 20 or 30.
[18:53:25] <a|i> StephenLynx: the article suggest to have the children on the parent when the cardinality is of a few 100.
[18:55:09] <StephenLynx> but usually its better to reference the parent on the children and rely on that.
[18:56:47] <a|i> I mean some of the replicas are not updated during the query time and querying for a parent may not return all children.
[18:58:42] <StephenLynx> I am not experienced with replicas and clusters, but yes, if it takes some time for that change to be replicated on the children, that would be a possibility.
[19:00:04] <StephenLynx> notice in the second example in that blog where the author suggests what I suggested.
[19:00:28] <StephenLynx> if you have few children that rarely change, its better to use the first method. otherwise, the second.
[19:02:20] <cheeser> well, queries should generally hit a primary so that's not a really big problem.
[19:12:30] <ab2qik> StephenLynx: Hi, getting, "TypeError: undefined is not a function" at ln29 - http://pastebin.com/yMFiNRgp
[19:27:15] <akoustik> unimportant question: anyone know why the "old" config format is the one used in /etc/mongod.conf in fresh installs, instead of the yaml format? backwards compatibility?
[19:38:31] <akoustik> probably. just interesting that i've seen it in 4 different distros. (don't ask why i have to use 4 different distros, it's a sad story :)
[19:41:10] <EXetoC> so the server sometimes replies with the key "ok" where the value is of type double instead of int? odd
[20:06:35] <dbclk_> hey guys..need some assistance on an issue i'm trying to figure out with my mongo clusters -> http://pastie.org/10278260
[20:08:21] <akoustik> dbclk_: what exactly is the issue? can't get a replica set running at all, or what?
[20:11:54] <akoustik> i assume you're talking about the lines with "replSet info hostdb3.tester.com:xxxxx is down", is that right?
[20:45:09] <acidjazz> hey all.. lets say in a document i have a key=>value object, say { objs: { ornage: {}, black: {}, blue: {} } }, what would be the ideal query to pull documents where objs.orange exist?
[21:32:23] <symbol> Would a many-to-many relationship with references be the one exception to now allowing arrays to be mutable and unbounded?
[21:45:33] <acidjazz> and i want to return documents that anything in that array matches a given array
[21:58:17] <greyTEO> GothAlice, I am trying to startup a worker.py and an receiving this error "got an unexpected keyword argument 'slave_ok'". Any hints?
[22:00:01] <akoustik> if so, what happens if you replace that param with "read_preference=ReadPreference.SECONDARY"
[22:00:32] <akoustik> or rather "read_preference=pymongo.ReadPreference.SECONDARY" if you haven't imported ReadPreference explicitly
[22:03:50] <GothAlice> greyTEO: Yeah, sounds like you're running the latest Pymongo. You need to pin your dependency <3.0, currently, until MongoEngine is updated.
[23:49:20] <ab2qik> Hi, get this error after npm install of mongoose, express: Cannot find module '../build/Release/bson'