Log file Viewer
#mongodb logs for Wednesday the 5th of June, 2013
[00:01:09] <bjori> Aartsie: are you just running mongodb on localhost? then thats perfectly safe yes
[00:05:21] <ackspony> Aartisie, if you have replication impelmented, dont just do an apt-get upgrade
[00:18:54] <bjori> it should be fine with replcasets, just start on the secondaries and execute stopdown on the primary before doing apt-get upgrade on the primary
[07:57:36] <platonic> i seem to remember there used to be some sort of mongo iops testing tool somewhere - i just can't find it. anyone got a pointer?
[07:58:01] <Kosch> Hm. I've got a replica set with a master and a slave. I stopped the master and rs.status() on the secondary node shows me, that it is recognized, that the master is gone, but secondary does not become primary. Inside log I see "replSet can't see a majority, will not try to elect self". So the 2ndary never becomes primary?
[07:59:00] <platonic> doesn't directly relate to mongo (i.e. doesn't actually need a mongodb to install). but actually, i'll try to dig deep in those to see if i can't figure out what's going on
[07:59:33] <double_p> Kosch: you need an uneven amount of replSet members (at least 3) to vote. you can use "empty" mongod instances to fulfill that
[08:01:48] <platonic> anyone care to see if they can point me in the right direction with my issue? it's a really simple one, one that i'm sure you've never heard before.. 'my mongo is actually running insanely slow and i don't know why'
[08:02:25] <platonic> fresh rhel6 install, nothing else running on it, 12gb ram, fast 15k sas disks...
[08:03:49] <Kosch> double_p: another question: Do I need to configure all members of a replica set at client configuration?
[08:04:21] <platonic> essentially, right now i'm hammering it as much as i can with updates (upserts).
[08:04:47] <platonic> refresh rate is per second on that default mongostat so i presume that's actually the performance i'm getting
[08:05:00] <double_p> Kosch: i think it depends on the driver. but usually the replset members will be known to the client. if you leave some out in the configuration, you must ensure that the ones in the configuration are alive at client startup
[08:05:37] <double_p> like, if you add members to the replset while client is running, at some point (refresh whatever) they'll be contacted by the clients
[08:06:08] <platonic> one thing that concerns me though is that i don't see mongo actually taking any memory
[08:07:12] <Kosch> double_p: ic. What I thought. I use the NodeJS MongoDB driver... but after shutdown of the primary, the application fails. Can this be caused by not having a primary at the moment?
[08:08:31] <platonic> ulimits for the pid if anyone's interested: http://pastie.org/private/mnrjmn8tqo9rtso3zpcniw
[08:09:27] <platonic> nodex: if i wipe it now and restart my insertion, any tips on how to look at it? can't use mms unfortunately (firewall issues for now.. :/)
[08:11:26] <platonic> havent tried reading except through robomongo, and that feels pretty responsiv
[08:12:24] <platonic> i've really flooded mongo with, well, about a million update requests with writeconcern 0 at the moment just to see
[08:13:11] <platonic> i'm kindof hoping it might be the disks being slow for some reason (hence why i tried to find that iops thing)
[08:14:09] <platonic> i presume though that upserts (update with upsert:true) are generally slower than inserts - that's fine enough.
[08:19:50] <platonic> just to make sure i'm not expecting too much here though - 10 upserts per second IS slow right?
[08:47:40] <ggherdov> Hi all. Does anybody happens to knwo what the name of the ubuntu package for the C++ mongodb drivers ?
[08:48:20] <ggherdov> also: is such a package provided at the 10gen ubuntu repo, http://downloads-distro.mongodb.org/repo/ubuntu-upstart
[09:07:38] <ggherdov> HI again. Getting a bunch of errors while compiling the tutorial for C++ drivers. Here the code, the compilation command and the errors: http://bpaste.net/show/vXN0IXjthUUC64FUVDog/ what am I doing wrong ?
[09:09:57] <remonvv> Does anyone know where I can find a detailed spec about what happens and what guarantees are given during removeShard/addShard?
[09:49:56] <dsds__> Hi all! I'm using nodejs + mongodb (and the native driver). Should I create a new MongoClient for every request I receive in the server? Right now I'm using an unique instance of the MongoClient shared by every request to communicate with mongo, and it works quite well actually, but I'm used to MySQL where you must end connections as soon as possible, and something seems off...
[10:03:25] <remonvv> dsds__, I have no experience with node.js but the answer is almost certainly no. Never do heavy lifting on a per-request basis.
[10:07:37] <dsds__> remonvv, have you got experience with any mongo driver? I suspect that behavior must be alike in other drivers
[10:19:40] <Nodex> i/e you call connect and it either uses an emtpy connection or it makes a new one
[10:42:32] <dsds__> well, it seems that actually you only open the client once, and then reuse the db in each request (nodejs driver writer dixit), so I guess my approach is OK, thanks for the answers
[11:08:19] <remonvv> khushildep, what kind of help are you expecting? There's zero information in that paste other than that something's crashing.
[11:37:20] <dirk_> Hi, anyone here that could possibly help me out with a mapreduce issue on a sharded collection?
[11:42:03] <khushildep> yeah - I'm basically trying to start and it's just dumping out - it's a really minimal config - just a database path and a log path.
[11:57:08] <dirk_> Ok here goes. I am doing a map reduce on a collection with out-reduce set to a large sharded collection. But it seems like the data never gets to the shards looking at postProcessCounts.
[11:57:41] <dirk_> If I do the same against a small collection with the same indexes and sharded as well postProcessCounts shows that it has written the new data.
[11:58:36] <dirk_> The only difference I can come up with is that one of the collections is small and the other is large.
[12:07:08] <dirk_> It somehow just doesn't get to the reduce state when the out: reduce options points to a large collection. I am at a loss as how to fix this.
[12:33:06] <ggherdov> Hi all, problems installing pymongo on Ubuntu. I can "import pymongo", but not "from pymongo import MongoClient" look http://bpaste.net/show/6Y9mHc9z2gdaRQDlcV40/ is the client class in another package? what is my mistake ?
[12:33:06] <ggherdov__> Hi all, problems installing pymongo on Ubuntu. I can "import pymongo", but not "from pymongo import MongoClient" look http://bpaste.net/show/6Y9mHc9z2gdaRQDlcV40/ is the client class in another package? what is my mistake ?
[13:03:23] <harenson> ggherdov__: import pymongo; then; connection = pymongo.MongoClient('mongodb://localhost')
[13:09:02] <ggherdov__> harenson: via apt-get . and I am on ubuntu 13.04 , which ships pymongo 2.2-4 . and the file mongo_client.py https://github.com/mongodb/mongo-python-driver/blob/master/pymongo/mongo_client.py
[13:12:03] <ggherdov__> harenson: I just finished this report that I hoped to hand to 10gen somehow to get some hint http://bpaste.net/show/wMp96gy8I66Ix1tX9OEp/
[13:13:26] <harenson> ggherdov__: you have to exit from the actual ipython, bpython or whatever you're using
[13:15:23] <ggherdov__> harenson: done that but no luck. But this is NOT surprising: the file mongo_client.py is *not* in my system. The MongoClient class is in that file. no file, no class.
[13:41:17] <harenson> ggherdov__: create new .py file with this content http://pastebin.com/gj2vZYYG
[13:53:32] <pinvok3> Hey there, I'm pretty new with mongodb and I come from a long time SQL development, so it's hard to get the whole syntax. I have this layout: http://pastebin.com/U7a10gcu I want to count the errors.key.message in this array. I have two times the same message. How should the aggregation look like? I just don't get it. Thanks in advance
[14:01:20] <pinvok3> harenson: http://pastebin.com/M3zhbXzP the 2048 and the 8 key is the Id of an PHP error. I think it was E_WARNING or so.. Its a simple error log and I want to have some statistics about the most occouring errors and warnings
[14:05:04] <pinvok3> Nodex: Is it a "you should" or a "its important that you don't use numbers"? It's easier to work with those numbers in php when Ive fetched the data
[14:07:35] <Nodex> they will work but when it comes to adjusting objects/arrays you will run into problems
[14:15:32] <ggherdov__> harenson: thank you for your support; I ended up re-installing pymongo with pip this time, got a newer version (2.5.something) and it went all fine. Ubuntu just packages some obsolete crap
[14:15:33] <pinvok3> Nodex: What other approach would you suggest? Save every message in a seperate collection and link an id?
[14:33:39] <platonic> lookup:{receiver:{fullAddress:[{name: xx, addr1: yyy, zip: 242, place: nnn},{...}]}}
[14:34:17] <Nodex> pinvok3 : how I learned Mongo quickly was forget everything I knew about SQL and treated objects like PHP arrays and or JSON objects
[14:36:15] <platonic> yeah, see, that's the thing - thought that the index would help - but doesn't seem to at the moment
[14:41:25] <platonic> just making sure i actually have shot myself in the foot with the current document structure
[14:41:42] <platonic> there are two usecases for this - one is easy - using services.xxxx.active and updateClass as keys
[14:42:11] <platonic> nodex: so you're saying i can actually index exactly inside lookup.receiver.fullAddress.fullname for example?
[14:42:34] <platonic> here i thought that i could only index up until the array and not inside of it
[14:45:28] <Nodex> large keys are a huge waste of space, mongo doens't compress them (at the moment)
[14:46:04] <platonic> bwahahahahaha.... 0.002 seconds for the find that previously took 1.5 seconds.
[14:46:28] <platonic> "lookup.receiver.fullAddress.fullname_1_lookup.receiver.fullAddress.adr1_1" : 93059232
[14:48:05] <platonic> think i actually got my head mixed up from some old typical walkthroughs.. like indexing arrays of tags
[14:51:35] <platonic> yup, i'll rebuild before we start heavy development anyway - probably wise to trim the keys regardless
[14:53:48] <Nodex> I wrote a map for mine, I can feed into it what I want and it give me back the corresponding mongo key / solr key / redis key w/e
[14:56:38] <hectron> Asking once more since I got a lag spike earlier: I am going to be participating in a hackathon tomorrow and wanted to leverage MongoDB. I wanted to add functionality to an existing website, and include a Javascript file that collects user information and saves it to my db. Do I need Node.js?
[15:03:26] <harenson> hectron: you could use ajax + your server side programming language of preference
[15:03:57] <hectron> I see. I was hoping of having like a drop-in solution, which the reason that I was leaning towards javascript.
[15:05:00] <hectron> The applications that I would be implementing this would be on C# websites and potential Python/Django websites.
[15:11:52] <hectron> Pardon the noobery, but this is my first time actually creating something rather than following or maintaining.
[15:13:59] <harenson> hectron: #IknowThatFeelBro http://th03.deviantart.net/fs70/PRE/f/2011/288/7/a/i_know_that_feel_bro_by_rober_raik-d4cxn5a.png
[15:25:27] <hectron> My registration icon for the event is the feels guy. So I think that it's definitely relevant.
[15:27:38] <kurtis> Hey guys, I am looking for a good "caching scheme". We're doing 'real-time' analysis and I'd like to use existing analysis results (cached) combined with the later results. The primary place I'm running into trouble is determining whether or not it's safe to save a "date" (or objectid) and simply grab all of the objects that came after-wards. Any suggestions?
[15:31:40] <kurtis> Nodex, Well -- I'll definitely be using redis. But, I'm trying to identify a sane methodology for separating "what's already been processed" and what's "new and needs to be processed". As a simple example -- we have millions of MongoDB documents. I need to take a single field, count the number of occurences, and sort it, and return it as a list. The more data we have, the slower this operation becomes