Log file Viewer
#mongodb logs for Friday the 2nd of November, 2012
[00:04:23] <chronidd> I have a composite _id field (i.e. my documents look like {"_id":{"foo":1, "bar"2}, "value":{"v1":1, "v2":2}}, and in my aggregations, I can't seem to group by a sub-component of my _id, just the entire thing. Anybody know if that's supposed to work?
[00:06:01] <chronidd> i.e. { $group : { _id : "$_id.foo" , v : {$sum: "$value.v1" } } produces null for the group's key. BUT if I just specify "$_id" in the groups id, it includes the entire id as the id of the group.
[00:10:50] <chronidd> Just found out that if I add a $project step before my group, I can pull the items I want out of the composite id, so that's an okay workaround for now, I just wonder why I can't include subfields of the id in my aggregation.
[00:30:54] <opus_> http://stackoverflow.com/questions/13187486/mongoose-nested-document-update-failing Hey guys I'm having this problem
[01:03:20] <chronidd> For those following along at home, found the bug that was tripping me up -- https://jira.mongodb.org/browse/SERVER-7491
[03:19:09] <mariooo> but mongo's document IDs are not numeric, and incremental index numbers don't seem to be encouraged, so unsure what the go would be here...?
[03:20:15] <mariooo> a solution is to just have a separate function that brute forces a unique code based on random numbers, but was wondering if mongodb had something elegant built that would be cleaner
[03:36:34] <crudson> mariooo: document IDs can be whatever you want, but even so you could use any other field. Performing atomic number incrementing (ala SQL sequence) is simple in mongodb and is explained in the core docs. I'd also suggest base 36 as upper and lower can sometimes be hard to determine between certain characters.
[03:45:08] <mariooo> crudson: not imposing a limit no, have just settled with generating a base62 encoded random large integer as the default code
[03:45:40] <mariooo> crudson: I figure that gives me just shy of a billion combinations, plenty fine for a little exercise =)
[03:51:14] <crudson> mariooo: you'd have by definition ~2billion on a 32bit arch, 2^63 - 1 on 64bit before mongo starts doing double conversions
[05:29:47] <reza> hi i'm learning more about mongo but was wondering why my brand new empty db has about 200mb allocated
[05:58:54] <reza> so that article i linked to ... he was creating multiple db instances and that's why he ran out of space?
[07:34:20] <crudson> fleetfox: right, have a computed sort field that does it, or...with aggregation you can calculate a new field in the pipeline then use that to sort
[07:35:44] <crudson> fleetfox: if it's a relatively simple mathematical expression you can do it with aggregation: http://docs.mongodb.org/manual/reference/aggregation/
[07:36:49] <crudson> fleetfox: so first $project a new field that is the result of your expression, then $sort on that new field
[08:17:57] <Guest88699> My question is that if I use @Embedded to make a List of Child Java Objects, then what is morphia's default fetching behavior. when I get the parent object will everything embedded also get loaded? or is this some sort of a lazy load?
[08:34:38] <Gargoyle> Odd. I'm getting a non utf error string when trying to write data… but the data is coming from mongo! o_O
[09:02:09] <Gargoyle> NodeX: This is data coming from mongo, I am doing a mb_strtolower() and trying to put it back (with $set) and getting non utf-8 errors. If it's non utf-8 how the hell did I get it in there in the first place! :S
[09:03:08] <kali> Gargoyle: the ascii character is a quote, not an apostrophe, so many people will use the more accurate unicode characters
[09:08:35] <Gargoyle> I just passed them through iconv('ASCII', 'UTF-8//IGNORE//TRANSLIT', $searchname), and afterwards, mb_detect_encoding() thought even more of them were ASCII!
[09:12:06] <Gargoyle> Thanks NodeX, That seems to be working even though mb_detect_encoding tells me its ASCII
[09:13:58] <NodeX> Gargoyle : I add that code into my wrapper and clean everything that goes into the DB - else I found sometimes I had problems
[09:18:50] <Gargoyle> mb_strtolower(mb_convert_encoding($doc['name'],'UTF-8','auto')) = not-utf-8 exception from mongo
[09:19:24] <Gargoyle> mb_convert_encoding(mb_strtolower($doc['name']),'UTF-8','auto') = Mashed £ signs!
[09:32:54] <Gargoyle> Right. Final combo that seems to have worked… mb_strtolower(mb_convert_encoding($doc['name'],'UTF-8','auto'), 'UTF-8');
[09:35:46] <NodeX> ron : did you want to ask me about that domain stuff or were you trolling yesterday?
[09:36:26] <ron> NodeX: no, I really did. I did a little more research. didn't want to disturb you late at night.
[09:38:22] <NodeX> slightly off topic but does anyone know a deamon that will accept any stream I throw at it, check some authentication and pass onto another service
[09:40:31] <NodeX> basically a firewall but with username/password authentication I suppose is the correct terminology
[09:42:24] <Gargoyle> On OSX under the network tab, just about every proxy option has a checkbox for "Proxy server requires auth"
[09:48:55] <NodeX> ok, that's not going to work, looks like i'll have to write one in node and redis :/
[11:41:26] <Gargoyle> Was A bit expensive, but my previous samsung has done 10 years hard time - so I don't really mind! :)
[11:42:38] <mamalujo> hi! Are there any instructions, or pitfalls you might know of, in running mongo on top of a distributed filesystem?
[11:48:17] <mamalujo> Deric, rather than dedicating storage resources for mongo machines, and fiddling w sharding and replication in the mongo layer, and since we already have a cluster (w moosefs) running, we though to use it, w each mongo server storing in some directories in the distributed fs.
[11:48:39] <Derick> mongodb is mostly limited by memory, so distributing the filesystem doesn't help really
[11:54:58] <mamalujo> Derick: a related q then, if you'd indulge me - provided we do shard the dataset on the mongo machines, we could still alleviate the need for replication if the storage layer were already on a replicating dfs, no?
[11:55:53] <Derick> mamalujo: I would leave the replication to mongodb. Having each of the replica nodes use the same files seems like a receipe to disaster as nothing would be locked on the FS level causing lots of corruption
[11:58:28] <mamalujo> I thought, w separate dirs there for each machine, each buffering a part of the data, hopefully that wouldn't include accessing the same files, but anyhow, y, I see your recommendation is still - do it in mongo; which is sufficient response, since we were mostly wondering about the recommended practices.
[12:00:30] <dingens> Is there a limitation for the aggregation framework's ability to match? My example collection has documents where the _id field is a dictionary (e.g., _id : { key: <somevalue>, bucket: <some-othervalue> }). When collection.aggregate is called with { $match: { _id : { key: <somevalue>, bucket: <some-othervalue> } } } no result is returned although documents with these values for _id are in the database. Is this some kind of general l
[15:31:50] <Pego> i have an issue with lock, if anyone can help it would be great :) : i use mongodb 2.0.6 when i lock a secondary with { fsync: 1, lock: true } it's getting stuck and even login after that. the primary is ok. but the mongos is stuck/slow. any idea please ?
[17:10:05] <denom> Hey all I'm using mongoose (node.js) to connect to mongodb and I'm noticing that 'open' event is never being fired on my connection. How can I debug this?
[17:16:43] <tystr> is comparing optime of members in a replica set a good way to monitor the status of the replication?
[17:33:25] <mrgreenfur> I'm finding some weirdo behavior with mongoid; can someone double check my thought process please?
[17:36:01] <mrgreenfur> hdm: thanks much. I'm trying to use find_or_create_by with a bunch of params
[17:36:21] <mrgreenfur> and it seems to work in the rails console, but the objects aren't actually being created in the db
[17:37:25] <hdm> which is basically a find clause followed by commands to update, and you can use things like $set or $inc or $push
[17:38:06] <hdm> my guess is your save() fails for some reason though in the create, check the return value or add a !, but consider using an upsert instead
[17:39:08] <mrgreenfur> hdm: interesting. Where do I put the ! ? Foo.find_or_create_by(bar:=>…)! returns a syntax error. Is it Foo.find_or_create_by!(params)?
[17:57:16] <mrgreenfur> hdm: I'm using a polymorphic association and I noticed a "_field" attribute, any idea what it does?
[17:57:21] <aheckmann> denom: you can listen to the connections error event. how are you connecting?
[18:28:20] <jaimef> should swappiness=0 be set for mongodb servers? or is it bad given the mmap nature of mongodb?
[19:05:46] <hdm> you save, the drive says go, but then mongo deletes it because your unique index is violated
[19:06:09] <hdm> you will see ghost inserts like that - they get saved, but then are purged as mongo processes them
[19:58:11] <wting> I'm trying to get mongo running on a Debian server following the instructions here: http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian/
[20:01:38] <Guest56110> If you didn't set up a config file, you need to pass --dbpath parameter when you run it
[20:05:23] <wting> There we go, have log files now. Not sure why mongodb doesn't create directory if missing considering I'm starting/restarting it as root.
[20:06:02] <doxavore> I think I ended up just changing my config to /var/log/mongodb.log because I was tired of forgetting to change that. :)
[20:07:57] <wting> error: [initandlisten] exception in initAndListen std::exception: boost::filesystem::exists: Permission denied: "/home/wting/mongodb", terminating
[20:09:20] <Ephexeve> Hey guys, question, I am using mongodb with Python, but I wonder, I have a function where the user add a srctype and a link (http://bpaste.net/show/AuCi2Bwq0stDWfpwWfck/) and another function which I am trying to implement which gets the data according to the src type, somthing like "SELECT * from tablename where srctype="hello"), but I wonder how this would be.. find_one(srctype) ?!
[20:11:15] <Ephexeve> hdm: find_one is the function.. Hmm will try.. a bit confused here.. still used to sql..
[20:15:22] <aranea> Hi there. I have an enhancement proposal for your code, but I'm too lazy to create an account on your JIRA.
[20:16:10] <aranea> in src/mongo/util/version.cpp, line 161, the following call is done: if (boost::filesystem::exists("/sys/devices/system/node/node1")){
[20:16:43] <ron> you weren't lazy to make the proposal but you're too lazy to create an account on JIRA? seriously?
[20:17:22] <aranea> on my system, /sys/devices/system/node isn't readable for the account under which mongo is running, that leads to a complete failure of mongo
[20:18:06] <aranea> imho it would be better if this case was just treated the same as if /sys/devices/system/node/node1 wasn't there
[20:18:27] <Ephexeve> since the user says when inserting the source type is called "something": "www.google.com" -> if I try to find({srctype:"something"}) doens't work
[20:21:45] <aranea> crudson: If I'd create an account for each software I find bugs in, that would be /a lot/ of accounts, so I'm trying to avoid that...
[20:25:44] <aranea> crudson: I never understood how open source projects like Mongo can use stuff like JIRA
[20:26:42] <Ephexeve> hdm: well, I would like to do something like: In SQL: a Column called srctype, and another column called url
[20:44:51] <Ephexeve> aranea: No worries, when I finish writting some code I will do that, 30min or so, are you sure this is a bug? What title ?
[20:55:58] <aranea> Ephexeve: What about "Mongod fails if it doesn't have read permissions on /sys/devices/system/node/"?
[20:56:36] <Ephexeve> column called srctype, another called url, where when calling the first row of scrtype, gives me the first item in the row of url
[21:07:12] <aranea> Ephexeve: Could you just write an email or a memo when you're done? I've got to go now...
[21:11:43] <meghan> btw if any of you are planning to come to mongosv (silicon valley conf) today is the last chance for early bird (see mongosv.com)
[23:11:32] <silverfix> I'm starting with MongoDB. I've got a collection and I'd want to retrieve one item of this collection, do something with it (with python/flask) and then update one of its attribute. The point is: how can I use e.g. findOne for retrieve a different item at once ?
[23:50:06] <denom> aheckmann, I cant reproduce it with a simple test script. It's a problem with my code :/