Log file Viewer
#mongodb logs for Tuesday the 25th of September, 2012
[01:38:37] <Max-P> Hi, I need to query for the last element of an array in a document, how can I do that? I need to find the immediate childs in a tree (all parents are in an ordered array)
[02:40:34] <niriven> so a long story short, i get an assignment to import 151 million events, and 1075 users into mongodb. the event may or may not 'relate' to a user (4% of the events acutally relate.) so i start by inserting all events related to a user in in event_a, and all events not related to a user in event_u, and all user into user. great, works fine, until have to do some really complicated queries (eg. find users that have events of type a b
[02:41:44] <niriven> so i have to join in code, fair enough, but then things no longer become optimized even though things are indexed. I have to query id's up front that match critera a, and ids tha tmatch criteria b, then query mongo and do some logic in code. so i figure, why not reprepsent everything as a user object? {user {events}}, great, until indexing crashes!
[02:43:21] <niriven> creating indexes crashed mongo, sometimes it did, sometimes it didnt, thought i ran out of memory, grabbed an ec2 instance with 16 gigs of memory instead of 8 gigs, and high io, another crash.
[06:38:34] <sunfmin> db.bbb.update({"name": "Felix"}, {"$pull": {"apples": {"$where": "this.name=='App2'"}}}) ?
[07:22:04] <coalado> I'd like to do a querry like 'find all documents where doc.myField contains "abc" doc.myField contains text.
[08:38:18] <Signum> I start to like the new aggregation feature in 2.1/2.2. But has anyone understood how to group on nested data? i.e. http://pastebin.com/9DsetKSf
[08:50:24] <Signum> Seems like the new-style aggregation helps avoid map/reduce for some quite common cases. Plus I like the UNIX-style pipelines. :)
[09:02:25] <Neptu__> hej just checking this for a while now.... how i can add an array to an array in c++ driver...
[09:45:12] <Neptu__> { one: 6, two: 7, RESULTS: { 0: [ "bar", "baz", "10" ], 1: [ "bar", "baz", "20" ], 2: [ "bar", "baz", "30" ] } }
[10:22:05] <newbie_> have tried some already but far from impressed. Monjadb my latest experiment which is also not exactly okay
[10:25:36] <meghan> newbie_ I have heard good things about genghis and mongovue so i might start with those
[10:29:36] <newbie_> meghan: seems i tried it long time ago, had minor issues with the non-free part, and some other thing i cant remember. but thanks.
[10:30:09] <newbie_> need to make my mongo secure. it isnt secure by default and i can connect over it remotely easily without any security user/password asked
[10:49:32] <Cubud> NodeX : I think I have thought of a way to ensure my video ratings are up to date and accurate
[10:50:14] <Cubud> I have 2 members on "Video" called "StatisticsCalculatedUntil" and "LastStaticalChangeOn"
[10:50:32] <Cubud> When I add a rating I update video.LastStatisticalChangeOn and then insert the rating
[10:51:13] <Cubud> That way if the rating doesn't get saved a process will later run to select ratings between StatisticsCalculatedUntil and LastStatisticalChangeOn and simply find nothing to update
[10:52:14] <Cubud> If it does find data then I can do an atomic update on the Video updating both the statistics + StatisticsCalculatedUntil
[14:05:11] <solars> hey, I (or my boss) came across http://blog.engineering.kiip.me/post/20988881092/a-year-with-mongodb - is there any discussion going on about this? I've recently read this anonymous blah article putting mongodb into bad light so I'm a bit curious
[14:12:20] <NodeX> the only thing that puts mongo in a mad light recently is idiotic users who dont know what they're doing
[14:14:13] <NodeX> the majority of "The bad" in that post have been addressed or are being addressed. Mongo cannot be held responsible for a developers poor choice of schema design and bad coding practices
[14:15:01] <rcmachado> I think the biggest error is using MongoDB as if it was a traditional database (like MySQL or Postgres).
[14:19:29] <solars> NodeX, I'm referring to this: http://www.readwriteweb.com/cloud/2011/11/hacker-news-and-the-damage-don.php
[14:20:43] <trepidacious> Anyone know whether it is possible to read/write plain JSON strings from/to a document?
[14:22:20] <trepidacious> But all the examples I can see involve working with a representation of a JSON object in memory, then writing that
[14:23:04] <NodeX> I dont know about your specific driver, I store Json fine using the php driver - I just have to escape it
[14:24:13] <fitzagard> solars: I believe the root of any proper architecture discussion is "does this thing fit our needs". If the answer is "maybe" then you experiment, test, verify, rinse and repeat the process.
[14:24:52] <NodeX> solars : Mongo never claims(ed) to write data deterministicly, it never claimed to do half of the stuff in that post lol
[14:25:24] <NodeX> what it does -correctly- do (in my opinion) is take all of that important stuff and leave it in the hands of the developer
[14:26:15] <http402> well except "global" lock is now down to "database" level lock i believe (not database, not collection) and i believe it's still a global write lock, just no longer global read locks which screw with replication
[14:27:20] <NodeX> the developer is mad because he didn't think his dataset through first and had to migrate away
[14:27:42] <solars> the second link is basically about another post, that made up stuff to put mongo into bad light
[14:27:47] <http402> NodeX: ive met a lot of developers that actually got mad because they Mongo doesn't do a good job of describing which queries don't work well dynamically
[14:28:01] <solars> that is why I posted the first link as I'm not sure about these 'facts' and wanted to ask if there is already a discussion or comments
[14:29:09] <NodeX> I've been using Mongo for a very long time in production, I have never had a problem (apart from early geo spatial queries) that I could not get around
[14:29:14] <http402> also, make sure that you keep your data set in ram as much as possible, it doesn't swap well, but thats not mongo-specific
[14:30:27] <http402> nothing is mongo :-D not saying otherwise, im saying what alternatives fit other use cases and the choices
[14:30:53] <http402> if youre data fits nicely into the mysql model, you'll probably be happier sticking with mysql
[14:31:40] <NodeX> there is a reason I have not touched an RDBMS for close to 2 years - because I dont need to
[14:33:22] <aboudreault> NodeX, does he explain his use case in its blog? haven't get time to read it yet.
[14:45:44] <topriddy1> NodeX: i;m not an expert in these topic. apologies already. but have been vast in databases (specifically Oracle in the pas)
[14:47:51] <topriddy1> NodeX: my app db isnt exactly critical, so i'm experimenting with it for now. i just have few entities which don have major risk.
[14:48:49] <NodeX> if you're after transactions think again - this will never fully happen unless you bake it in on the app level
[14:49:09] <topriddy1> NodeX: quick one, if i do db.users.save({name: "Paul"}); maybe i could still do: db.users.save({name: 32});
[14:49:44] <topriddy1> NodeX: yeah you getting my point. MongoDB sorts of trust the application guy to do things right.
[14:50:22] <NodeX> unless .... you have safe writes on in which case it will write to X nodes / Disks before returning
[14:51:29] <topriddy1> NodeX: well i am using one node. i am working on a chat app, where i need to store some info on the user in a db (mongo), also i post locations periodically. thats about it. can survive without a traditional db.
[14:51:37] <NodeX> [15:47:37] <topriddy1> NodeX: quick one, if i do db.users.save({name: "Paul"}); maybe i could still do: db.users.save({name: 32}); <--- I dont see the problem
[14:53:33] <topriddy1> NodeX: you cant store a string in a number datatype. it should throw an Exception. Mongo seems to accept this.
[14:54:01] <topriddy1> NodeX: also if you make a mistake with column-name, Mongo simply just creates a new column. (bug introduced this way)
[14:55:03] <topriddy1> NodeX: unfortunately so. Mongo doesnt have this info in itself about "datatypes"
[14:55:42] <topriddy1> NodeX: i'm also scared of what might happen if i mistakenly do a db.users.remove();// not sure yet if i have a rollback mechanism. i dont know if i can even do transactions, etc.
[14:56:33] <http402> topriddy1: one safety mechanism other people have used is delayed replica set - keep one replica 15 minutes behind the others and you have a window of recovery
[14:58:07] <NodeX> http://www.boneheadbeats.co.uk/radio <---- 60 minutes form start to finish on that chat room
[14:58:59] <topriddy1> NodeX: i'm using xmpp. mobile based. just put certain info on mongo. java+morphia at the moment. things working fine.
[15:08:22] <mdeboard> Hi, when I added a member to a replset, that new member was suddenly primary. Now my former primary is in "rollback" status. My intent was to add this new member and start replicating the data but now ... am I losing all my data or what?
[16:42:35] <timeturne> does anyone periodically determine if a user is considered "active" or not and save it i. the user's document?
[16:49:07] <timeturne> that's actually what I'm trying to figure out. basically the scenario is that user A creates a team. Users C, D, E, and F join that team. User A is not the team captain in real life, he/she just happened to be the first one to create the team. Some time later, User G shows up and requests User A to become the team captain because User G is the real team captain. If User A accepts then User A becomes just a team member or a
[16:49:08] <timeturne> co-captain. but I'mtrying to think about what should happen if User A doesn't respond to User G's request. thatwould basically mean that some other team member has to take User A's place and accept or decline User G's request.
[16:51:38] <NodeX> I do something similar in one of my apps - checking that a user has "Activated" - I use workers to occasaionaly check my user collection(s) for it
[16:53:02] <NodeX> ^^ I also (for different reasons) update tiemstamp every time someone does somethign to the record
[16:54:02] <_m> ^This. I think a fair standard are: created_at, updated_at, last_active_at, last_login_at
[17:32:22] <Turicas> I can't load CSS of api.mongodb.org/wiki/current/... (I've tried with and without proxy). Is there any place where I can fill a bug?
[17:39:31] <Turicas> timeturne, but site.css have some error messages in there, like "/* Could not locate resource: /includes/css/master.css */"
[17:41:25] <shadow_fox> hello m new to mongodb actually i know nothing i just downloaded the 32bit tgz file for my linuxmint13 OS
[17:51:55] <NodeX> Need a name for our quaint site persona ... anyone care to input ? -> http://www.jobbasket.co.uk/maintenance
[18:05:41] <Turicas> timeturne_, do you know who is the maintainer of these docs? Probably there is an error on Sphinx doc compilation.
[18:47:54] <rcmachado> I want to test the performance of some drivers (asyncmongo), comparing it to the pymongo
[18:49:09] <rcmachado> Basically, to see (if|how much) the asynchronous really improves the performance
[18:56:04] <mids> https://github.com/ajdavis/chirp is implemented with a bunch of different mongodb drivers
[18:56:27] <rcmachado> mids: but here where I work some guy is working on an implementation, more like asyncmongo
[19:22:01] <aster1sk_> Hey mongodb friends. I'm playing with the new aggregate framework and struggling to work with arbitrary keys. I have a heavily nested document and need some direction.
[19:27:56] <aster1sk_> Say I had a.b.c.123.c (the 123 is a mysql id) - I'd like to aggregate group / aggregate on 'c'.
[19:38:31] <dstorrs> morning all. In a sharded cluster, I'm connected to a config server and I want to ask "are the config servers currently read-only?" what is the correct command?
[19:38:45] <crudson> aster1sk_: I assume the second 'c'; there are two. Can you create a paste of your document(s) and what you are executing?
[19:43:26] <whitaker> Howdy. Rails/Mongoid question here. I've recently upgraded my mongodb instance to v2.2.0, and my mongoid gem (under Rails v3.2.8) to v3.1.0. I have a model which includes Mongoid::Document, and defines a field as type Array. Before, attempts to assign this field with scalar values would properly throw a Mongoid::Errors::InvalidType error, but that behavior seems to have gone away. Now, type checking seems to have been disabled o
[19:43:27] <whitaker> fields declared as Array, such that anything can be assigned to the field. I've searched in vain for workarounds to restore type check validation; anyone else dealt with this behavior?
[20:08:48] <kchodorow> dstorrs: like, "is one of you down and therefore the rest of you are readonly?"
[20:09:25] <dstorrs> or should I just look in a mongos log and see if the distrib lock pinger is responding?
[20:13:41] <kchodorow> i think the lock pinger is the best you're going to do, they don't "really" become read-only, mongos just stops trying to write to them
[20:17:10] <elliot-w> hi mongoids. We're maxing out file descriptors in our nodejs/mongodb(replicaset+gridfs) QA environment. My questions -- will these open descriptors be recycled when the offending process is restarted? Other than boosting the open descriptor limit in the OS to fix the issue... is there any way to catch this before it happens? I'm concerned that this will come up again in production.
[20:18:14] <whitaker> (Shorter version of question I asked before): has type check validation of Array field type in Mongoid::Document gem been recently disabled?
[20:18:35] <cyberd0m> Hi, I've asked in #node.js but maybe it's more pertinent to ask here. Basically, I can connect to mongo with the 'mongo' shell.. but not from node. I really have no idea what to try.. this should work.
[20:31:13] <cyberd0m> I was trying to use it from mongoose.. i.e. require('mongoose').connect('mongodb://localhost/test').. but couldn't get it to work, so I've tried with 'mongodb' with no more success.
[20:49:33] <aster1sk> There is little documentation I can find about this, I'm hoping it's possible -- I spent weeks on the data model.
[20:49:40] <cyberd0m> Because, here: http://mongoosejs.com, it says: mongoose.createConnection('localhost', 'test).. but just that is failing
[20:49:45] <DaveDev> is there a way to rebuild all the indexes from the command line (not in mongo shell)?
[20:50:04] <cyberd0m> but mongodb is clearly working as I've now connected to it from three different ways :-/
[20:53:01] <bhosie> cyberd0m: your error on .connect says you're missing a callback function. have you tried resolving that?
[21:13:38] <cyberd0m> bhosie: I think I've figured it outi.. it was something to do with my hosts file
[21:43:56] <SeyelentEco> Hi all, anyone know of an easy way to test if queries are being sent to replication servers? I'm running PHP, I have 4 replica sets (1 offsite), setup a seperate environment on offsite box, but application is slow... seems as though it's querying primary.
[21:56:17] <timeturner> is it worth it to shorten field names as much as possible in documents? possibly to single letters?
[22:02:18] <SeyelentEco> I've heard of people doing it (one presenter at MongoDB Seattle said his company did it)
[22:02:30] <wereHamster> timeturner: if the size of the keys contributes significantly to the overall data size, then you might consider it
[22:03:47] <wereHamster> shorten the fields, while still letting you access them by their long names.
[22:06:30] <dbe> Hey everyone. I have records with a date field. I want to group based on "week buckets" which would be one group for every 7 days going backwards from today. Any idea on how to do this with the aggregation framework?
[22:19:43] <timeturner> wereHamster: any opinions on this https://github.com/tblobaum/mongoose-troop
[22:55:41] <nemmeviu1> hi guys, you can help-me with a aggregate group by date, where my date has been written in a ISODate ?