Log file Viewer
#mongodb logs for Friday the 9th of November, 2012
[00:05:15] <PedjaM> it have moved most of those 88 chunks during that period when all the background jobs were down
[00:06:02] <PedjaM> i was thinking that if i move down chunk size, it would not need that much locking and it can move faster
[00:06:54] <PedjaM> it seams that i will need o make a maintenance window , stop everything and hope that it will move data around fast enough
[02:04:36] <Baribal> MWAHAHAHA, I'm generating 10 million records with random numbers, just to see the effects of indexing!
[02:57:25] <frozenlock> I want to keep an history of every changes that occured in say {"key1" data "key2" data}. If key2's data doesn't change really often, is it possible to make it refer to the last known change?
[02:58:12] <frozenlock> For example: {"key1": 100 "key2" "abc"}, {"key1": 101 "key2" "abc"}, {"key1": 105 "key2" "abc"}. Could it be something like {"key1": 100 "key2" "abc"}, {"key1": 101 "key2" _}, {"key1": 105 "key2" _}, where _ will return "abc"?
[03:01:33] <phira> frozenlock: you're probably just asking for trouble, I'd suggest just making a complete copy of the document and storing it in a history collection
[03:06:12] <frozenlock> Baribal: Hmm could work, but in my case I have mostly 1 value that's changing quite often and a bunch a others that remain 'mostly' static. Adding a revision for each would eat up any saved storage :(
[03:06:53] <frozenlock> phira: As it is that's exaclty what I'm doing, I was just wondering if there was a more efficient way of doing it. My DB is eating diskspace :)
[03:07:04] <Baribal> I just scaled up from 1 to 10 million documents, a query that took 0.5 seconds now takes 3.5 seconds. I thought indexes would scale better?
[03:08:08] <frozenlock> Yeah, I don't want to do that myself... I'm obviously going to mess up somewhere
[03:10:56] <phira> typically it's a failure even if you use a timestamp. The problem is that a new version of a document is typically derived from an old one, which means you actually have a concurrency conflict anyway - imagine two people changing a word doc and saving a new version, each has made a change based on version A, resulting in versions B and C, but C, even though it's the last one, is not derived from B
[03:11:26] <phira> the solution to that is either to declare that this is a feature, or involve some kind of locking, or do changes as a transform.
[03:14:29] <phira> yeah, anything with an atomic update has the basic tools necessary to generate a lock
[03:15:23] <phira> an example of that is simply generating a big random number for yourself, then doing update() on the document assigning owner_task: $number, and then checking to see if the number on the document matches yours.
[03:16:13] <phira> transactions are a much more difficult problem, but they're not necessary for this problem.
[03:16:47] <_m> Baribal: Basically, if you use set safe to true for a connection, you'll lock during an update. More-or-less.
[03:19:17] <phira> no, safe simply means that the operation will check for errors rather than returning as soon as it has handed off the data to mongo
[03:19:55] <phira> slower, but avoids thinking mongo accepted the data when it actually decided you were a loon
[05:37:01] <the-erm> How do you make the results list longer inside the mongo client? I can scroll I'd like more.
[05:56:06] <crudson> the-erm: add it to your .mongorc file if you want it to persist across mongo instances
[06:38:37] <the-erm> Is there a better way to get distinct keys in a collection than this? http://stackoverflow.com/questions/2298870/mongodb-get-names-of-all-keys-in-collection
[06:42:47] <crudson> the-erm: seems fine to me, but with questions like this I tend to ask the reason why from a document structure point of view. It's often easier to have known keys and variable values rather than keys being "freeform", if you get where I am coming from. This can manifest itself in many problematic ways down the road.
[06:45:04] <crudson> the-erm: in that case a map-reduce would be my approach too. emit property values as keys and 'nothing' as value, unless you also want to count the frequency of the keys.
[06:46:41] <samurai2> hi there, how to do something like db.collection.find().sort({ts:-1}).limit(1) in java driver? thanks :)
[06:50:37] <crudson> samurai2: http://api.mongodb.org/java/current/com/mongodb/DBCursor.html - sort and limit are right there on the cursor object
[08:58:57] <crudson> chovy: not *really* - if you use the default _id you could maybe in a non-sharded environment use it but if you want to do this either use a capped collection or add some field that represents insert_date
[09:06:14] <IAD> chovy: you can use this: http://docs.mongodb.org/manual/reference/command/findAndModify/
[10:08:16] <fatninja> I have a collection that has a ISODate() field, and an integer value field. I would like to get all these dates by days, with the total of all the integer value field. Any good approaches on this one ?
[10:41:44] <Baribal> fatninja, You mean you prodive a date (range) and want to get the sum of the corresponding integers?
[10:42:19] <Baribal> I think aggregation would be the hippest way to do that. If you want intermediate results, mapreduce.
[12:11:46] <remonvv> visof, your question is roughly similar to asking "Can a truck do what a Ferrari can?" and NodeX asking "Depends, do you want to move people or go 200mph"
[12:12:07] <remonvv> In other words, tell us what you want to do with it. Obviously a NoSQL database isn't going to be a drop in replacement for an RDBMS.
[12:14:15] <NodeX> well certainly mitigating from an exploiter brute forcing aside the obvious number of attmepts
[12:15:21] <NodeX> but even with token generation an exploiter can always generate said token in another shell and pass that information around their system to send to yours
[12:51:21] <jdevelop> Hi all! I have collection os projects and collection of permission for the project, permissions for project holds documents with project id and user id, and some ACLs. Now I want to find all projects which have specific ACL. How do I use map-reduce in order to join 2 collections? Do I need to follow http://tebros.com/2011/07/using-mongodb-mapreduce-to-join-2-collections/ ?
[12:53:06] <Gargoyle> jdevelop: It's not relational. Get a list of the project_id's from your permissions colleciton and then use those id's to fetch the projects.
[13:01:09] <Gargoyle> jdevelop: Well. It sounds like you have a relational like schema in a non relational db, so you are going to struggle. I think map/reduce might help, but you'll still run into memory issues!
[13:05:41] <NodeX> i/e in the first place I didn't use uid's to assign things to users which made it scalable
[15:00:21] <meghan> fyi, the office office hours for M101 course (mongodb for developers) happening at https://plus.google.com/u/1/101024085748034940765/posts/BukcLAgXWhi
[15:09:11] <NodeX> quick tip for anyone using Eway as a payment provider - they are about to go into liquidation so you may want to cancel your accounts and find another provider
[15:23:18] <P-Alex> instead if i use -> db.videos.update({_id: 1975660}, {$set: {rate.likes: 1}}) everything works
[15:29:09] <wiseguysonly> I've managed to master group and sump, but now I am trying to do something for which I have no clue how to start. I want the sum of a field for each of the last 7 days.
[15:32:08] <makin> Hi people I'm very confused with this error: http://stackoverflow.com/questions/13310300/doctrine-mongo-group-query Anyone can help me?
[15:32:45] <stefancrs> sorry, I know this question is slightly off-topic, but I'm developing a restful API with mongodb as the storage. I want to be able to do queries through query parameters but can't really think of a good way to specify values in an embedded object. For example /api/users?location.city=brooklyn&age=34 or whatever. I understand I need to take another aproach, but I'm out of ideas... :)
[15:35:48] <stefancrs> I _could_ of course just send the actual string for the query, like /api/users?query={"$and" : ["location.city" : "brooklyn", "age" : 34]}, is that the best way forward? I'd prefer if it wasn't THAT mongodb'ified on the API level... :)
[15:39:07] <Baribal> stefancrs, that sounds interesting, could you repeat the beginning of the question?
[15:39:28] <stefancrs> sorry, I know this question is slightly off-topic, but I'm developing a restful API with mongodb as the storage. I want to be able to do queries through query parameters but can't really think of a good way to specify values in an embedded object. For example /api/users?location.city=brooklyn&age=34 or whatever. I understand I need to take another aproach, but I'm out of ideas... :)
[15:41:44] <NodeX> I've built a few API's atop mongo and the best way I found was to recieve the payload as json and parse it out into my own language
[15:45:09] <Baribal> stalled, I think you're going at it the wrong way, the "What can I provide?". First ask yourself what you want or need to provide, then expose that as API and glue it to mongo internally.
[15:46:07] <stefancrs> I _know_ I need to be able to do finds on basically any embedded document value
[15:55:41] <Derick> stefancrs: be very careful with that, you can do all kinds of strange things then
[15:56:16] <stefancrs> Derick: how can I do more strange things like that than if it's a json payload with an array in it?
[15:57:28] <stefancrs> yeah, but understand that authorization is required and you can only read data you're allowed to read
[16:00:03] <nledon> Hello all. Anyone know why I have a node who's stat is UNKNOWN? I have one PRIMARY who's been initiated and added Mr.UNknown just fine. But it won't become secondary.
[16:04:47] <wiseguysonly> I'm doing the following: http://pastebin.com/YF1apX08 but it seems to bring back the sum of all the docs rather than just for the id specified
[16:16:30] <jtomasrl> if i have ID's inside objects in an array how can i take those id's and make a search on another collection?
[19:29:56] <ckd> and actually, depending on how nested that doc is, you probably shouldn't have _object in there either
[19:30:05] <JBJB> array(5) { ["updatedExisting"]=> bool(false) ["n"]=> int(0) ["connectionId"]=> int(956) ["err"]=> NULL ["ok"]=> float(1) }
[19:37:51] <cedrichurst> geospatial question, i know it's possible to get the list of documents inside a specified polygon
[19:38:21] <cedrichurst> but if my documents contain polygons, is there a way to get the list of documents with polygons surrounding a specified point?
[19:38:58] <cedrichurst> and the results would be all documents with a polygon that the provided point is inside
[20:00:55] <fumduq> I'm having some trouble with replication in which replicas do not catch up to their masters. the LVM module (that supports the mongo directory) crashes in the kernel, and when brought back to life, mongo just gets more and more out of sync
[20:02:40] <fumduq> is there a way for me to determine which operation is causing replication to stop?
[21:08:27] <doxavore> Does the Mongo ruby driver 1.7.0 not release connections back to the pool in JRuby on Rails?