Log file Viewer
#mongodb logs for Sunday the 9th of August, 2015
[00:22:32] <Axy> I tried to do toArray and foreach there, then inserd(modifieddocs,..) but I'm getting buffer errors
[00:22:44] <Axy> and it makes sense because I'm modifying the whole collection in memory and trying to push in
[00:25:53] <Axy> I need "index" values to all my existing documents, I already have a method to add index to new documents but I needed to add this to old ones
[00:29:58] <Axy> because it tries to keep everything in an array, I think my collection is too big to do so
[00:34:03] <Axy> can I just do .next and bulkwrite in a for loop, would they work in the exact order
[00:36:44] <StephenLynx> function B calls next and check if anything arrived. if it did, it adds the operation to the operations array and calls function B again passing the cursor and the operations array
[00:37:14] <StephenLynx> if nothing arrives on B's next, it calls function C passing the operations it assembled
[00:38:31] <Axy> If I could hold the whole collection in memory via toArray, could I just do, like, a for lop?
[00:38:38] <StephenLynx> once you become more concerned with scope rather than with sequence of written code
[00:40:36] <Axy> I mean I thought calling the documents with an index numver would be possible anyway
[00:46:49] <Axy> I can't believe I can't pull an already indexed field with an order/index number though
[00:47:07] <Axy> I mean okay the object id is unique and useful but I don't get why otherwise is not possible
[00:48:58] <StephenLynx> so why don`t you just skip the right amount in your collection using the sort method you have?
[00:52:05] <Axy> but I'm just unable to imagine a solution, for a (example) university database where there are only 3 classes, maths, physics, and history --- and all students (seperate documents) have scores for all three fields
[00:52:33] <Axy> I was imagining this to work like, indexing all fields, and somehow getting the n'th item, easily
[00:54:19] <Axy> so this is a sample document: {name:"lazykid",scores:{history:12,maths21,physics:67}}
[00:56:13] <StephenLynx> because instead of generating the page on every request, you generate it once
[00:57:30] <Axy> so yeah, we already have indexes, we can sort things by a specific order, but we can't reach them by index number
[00:58:46] <StephenLynx> now the page that opens the element don`t need to use its index, because the index is already being used on the page that links to the element.
[01:01:01] <StephenLynx> imagine if each entry had a page of it's own and there would be links to it
[01:01:54] <StephenLynx> and each board have a link of it's own that uses not the order, but a unique field
[01:05:09] <StephenLynx> a system aimed to please above anything else is pretty much doomed, in my personal experience.
[01:05:37] <Axy> but the thing is, I really don2t get why I2m not able to get something that's already indexed, via index number
[01:09:14] <Axy> If I wanted to create the same thing via fields it would take incredible abmounts of time, not sure if all is even possible
[01:09:27] <Axy> No it's just an imgeboard thing where people can paste links to websites and I save them there
[01:10:55] <Axy> that way without having access to the board index or anyting like that people can just try differnt url's and hunt for cool stuff
[01:11:57] <Axy> maybe it's the right way to do this, I just want to do this dynamically. For two reasons -- one is tht I really love dynamic stuff and two is that this is not really a commercial project or something, this is just me learning nodejs and mongodb that's all
[01:12:59] <Axy> Yes I understand, her is what I build for the counter thing https://gist.github.com/anonymous/6d6119f5c45f82e64767
[01:43:23] <FailBit> confession: I have no idea how to actually run mongodb, I'm not responsible for that
[01:51:32] <FailBit> what if it were this? db.tag_changes.createIndex({tag_id: 1}, {background: true})
[01:55:52] <cheeser> creating an index is a blocking operation but it will yield to other operations periodically so you don't lock the entire time.
[02:06:43] <FailBit> I'm going to pull down the latest dump of tag_changes and see how long it takes to index locally...
[02:07:45] <cheeser> though keep in mind hardware differences and activity on the machine will vary the results
[02:14:48] <FailBit> I'd like to smack whomever thought it was a good idea to use the ID field to store tag names.
[02:16:05] <FailBit> tag_changes.$tag_id_1 3339 { : \"but-its-worth-trying-10-name-anonymous-2010-10-06-09-27-try-opening-image-in-new-tab-11-name-fuck-2010-10-19-08-50-shits-still-not-working-12-name-ver...\" }
[02:17:02] <FailBit> what's the mongod switch that lets you turn off failure on key too large to index
[02:41:50] <akp> is it possible to use limit_execute with 2 backends? ie something like limit_execute data="db outbound carrier limit redis tn-limit tn limit2" >
[02:44:21] <akp> is it possible to use limit_execute like this - <action application="limit_execute" data="db realm resource1 limit1 redis realm resource2 limit2 " />
[02:49:29] <akp> ok, then change it to <action application="limit_execute" data="db realm resource1 limit1 db realm resource2 limit2 " />
[06:46:38] <steamio> hi all, I have a quick question about replication set - The scenario is that I have a large single instance and Im creating a replication set. Other than restarting the main inctance for replication support, will there be any downtime while the secondary is syncing?
[08:05:59] <hemmi> Hey there. If I have a schema where posts are embeded in users how do I grab all posts for all users?
[10:03:43] <avner> Would here be a good place for Qs related to mongo driver behavior in a replicaset?
[12:20:30] <moksa> its possible to search all objects that contain a specific list element on that element?
[13:52:43] <lmatteis> hi guys. so i have a collection with over 11 thousand documents, each representing a log item that recorded the status of my HTTP server
[13:53:00] <lmatteis> now i'd like to present these results to a user as a webpage, but of course i can't send all 11k records to the user
[13:53:30] <lmatteis> what could be a nice query which would show perhaps the average maybe for every month?
[13:56:25] <StephenLynx> pre-aggregate these in a separate collection so it would reduce the amount of records drastically
[13:57:23] <lmatteis> StephenLynx: right but how do i perform the aggregate query in the first hand?
[14:00:38] <StephenLynx> and query the collection holding the crushed data so you handle less documents
[14:24:25] <l403> hello, I've noticed this rather large (16M) file fix exectuable permissions /var/lib/mongodb/local.ns I was wondering what it is
[14:26:39] <lmatteis> StephenLynx: thanks. aggregate queries are read-only right? i don't want to screw things up in this db
[14:27:48] <StephenLynx> by default yes. and even then, you can only write it to a separate collection.
[14:28:53] <cheeser> i think you *can* write back to the same collection, actually, so you need to be really careful because it'll clobber whatever's in that collection
[14:56:04] <mantovani> mongoexport --db BigData --collection TwitterStreaming --fields="user.screen_name"
[15:00:34] <MalteJ> I am new to mongodb and have a problem: I have a document "user". Every user is allowed to have a maximum of let's say 5 documents of type "virtualmachine". How can I realize this, so I don't have problems with concurrency (e.g. when reading all virtualmachines of a user, counting them and then add a new one, if the count is <5)
[15:11:23] <JamesHarrison> ACID isn't what makes your DB reliable it's just a convenient way to describe a particular pattern for consistency management, it's not the only way
[15:11:38] <MalteJ> JamesHarrison: I could use the update method and query for the user and a timestamp of last change, right?
[15:13:09] <JamesHarrison> http://docs.mongodb.org/manual/core/write-operations-atomicity/ is worth a read
[15:13:48] <JamesHarrison> you can do two-phase commits, yes, or you can use update if current pattern or similar
[15:14:50] <JamesHarrison> mantovani is correct that if you're implementing from scratch, this doesn't sound like something mongodb is likely to be hugely suited for
[15:16:06] <JamesHarrison> mantovani: it's not that useful since it fails in sharded use-cases though
[15:16:49] <MalteJ> I have to synchronize my data with the vm hypervisor anyway. So an RDBMS with strong consistency does not really help, when the DB and there real world are not consistent.
[15:16:51] <mantovani> so this operations just can be used if you just has your data at one instance ?
[15:17:55] <JamesHarrison> MalteJ: basically what I'd do would be to store an updated_at timestamp or version field on the object, query for that version/timestamp, and then findAndModify using the ID _and_ version to insert the document and update the version/timestamp
[15:17:58] <mantovani> 12:14 < JamesHarrison> mantovani: it's not that useful since it fails in sharded use-cases though
[15:19:32] <mantovani> but whatever you agree with me to do what MalteJ want postgres would fet better.
[15:19:50] <JamesHarrison> assuming nothing about the rest of his use-case and deployment approach yes
[15:20:07] <JamesHarrison> but I'm assuming there's a compelling reason for using mongo over pgsql in his case :)
[15:22:56] <JamesHarrison> but realistically I've not seen a mongo cluster with better reliability than a well-engineered single postgres instance
[15:25:18] <JamesHarrison> if you need real HA (and most people don't) then you can do it quite readily these days without any additional software, just core postgresql
[15:26:10] <JamesHarrison> one of those lovely bits of software that Just Works, and keeps Just Working
[15:27:14] <JamesHarrison> native uuids are nice, and you can do a lot of the json-style data manipulation/querying that mongo does in postgres now
[15:27:33] <mantovani> so when you compare both fields postgres won't even compare because 'infinity' is always higher
[15:32:31] <JamesHarrison> and most ORMs will get out of your way and let you poke the DB direct with a stick if you need to
[15:34:41] <JamesHarrison> brave new world - I'd argue stored procedures have very limited use cases, and the overhead of database agnosticism is outweighed heavily by the improvement in developer productivity and reduced lock-in risk
[15:34:55] <JamesHarrison> there are still places where stored procedures/triggers etc make sense, sure
[15:35:51] <JamesHarrison> but for 99% of apps where you just need a reliable datastore, even a performant one, ORMs will be just fine
[15:37:06] <MalteJ> the question is, what is cheaper? coding, testing, debugging for the super-hyper proprietary rdbms functions or just adding more server hardware?
[15:37:55] <JamesHarrison> most of the time unless you're dealing with very tight performance requirements or very large amount of data, the latter is usually the best option
[15:41:12] <mantovani> MalteJ: actually when you use a rdbms since it do a lot for you, you will save code, test and debug
[15:41:35] <mantovani> nif you need to implement what it does for you manually so you will have more code, test and debug
[15:42:31] <MalteJ> the worst thing I have tried was cassandra. all this denormalized stuff. you write everything 5 times :D
[15:44:23] <JamesHarrison> I've used it in graph database use cases with apache titan, works fantastically
[15:47:59] <mantovani> I'm using mongodb because it fit perfect, I take tweets (which are json) and store it there