Log file Viewer
#mongodb logs for Tuesday the 6th of November, 2012
[00:00:00] <Nerp> Due to the memory mapped nature of Mongo, is it normal for a mongod process to take up 80% of a systems 8 gigs of ram?
[10:01:07] <Zelest> When it comes to filesystems and operatingsystems, what should I pick and why? I'm currently looking at FreeBSD (UFS2), OpenBSD (FFS) and Linux (EXT4) ..
[11:00:22] <moian> Hi, using mainly aggregation framework, I try to keep all my collections in the RAM, but is the difference really relevant ?
[11:00:35] <moian> I mean, doing this, I need to use sharding with lots of clusters. My question is: Can I do something better by using replica set instead of sharding ?
[11:10:10] <phrearch> im trying to create a reference between two schemas using mongoose, but it errors with a duplicate key error index
[11:12:21] <phrearch> the reference user has an email field, but its supposed to only store an ObjectId to the user
[11:14:44] <moian> NodeX: I use aggregation framework on big collections, I think about keeping the full database in the RAM to keep it as fast as possible, but by doing this, I need to use sharding with lots of clusters. I'm asking if doing this will really increase performances, or if simply using replica set (to increase read/write capacity) could be as good or even better.
[11:14:52] <NodeX> your error is a dupe key either on "_id" or on something else that's been set as "unique"
[11:15:40] <phrearch> bit confusing, since i dont want to store the user again, but only the reference to a user in another collection
[11:21:31] <phrearch> ok thanks. wont that mean duplicating a lot of fields or having huge collections?
[11:23:04] <NodeX> but performance does not come cheap in terms of disk space, but luckily disks are not expensive
[11:24:00] <NodeX> let's take a users forename/surname/email - this doesn't get updated very often if ever
[11:24:04] <moian> NodeX: ok thank you ! that's what I thought.. Do you know what is best to do about security, when we use sharding ?
[11:26:54] <phrearch> NodeX: thanks, ill try to figure out how to model my data as efficient as possible for nosql
[11:27:29] <phrearch> the duplicate error is gone now. seems like i did an update to the model after one was saved to the db. removing the existing entries fixed the problem
[11:42:58] <moian> On my clusters, do using password to access mongo is enough, or is it better to set firewall and/or something else ?
[11:53:37] <ppetermann> you might want to ask your question instead of asking for someone who knows whatever
[11:55:51] <SLNP> Hi has anyone ever some across an issue where indexes don't replicate across to new replica set nodes?
[11:57:44] <jamma> what's the best way to maintain relations between collections ? I actually try to put everything into each document, but it is very slow, i have huge amount of data
[12:54:41] <roxlu> when I have a gridfs in my database called "tweet" and a fs prefix of "images", how can I use mongofiles to list all files in the fs?
[13:15:31] <timroes> Hi, is there any possibility to use the Java driver to store a BSONObject into MongoDB?
[13:25:48] <HongKilDong> Executing this code in mongo shell for(var i=0;i<=10;i++) {echo 'it does not work';}; I get error "SyntaxError: missing ; before statement (shell):1" What's wrong with it ?"
[13:29:43] <roxlu> how do I set metadata of a gridfile when using the c-driver and when I write chunks using gridfile_write_buffer ?
[14:05:32] <eka> is there any reason why inline map/reduce temporal collections are always indexed? that's taking a lot of time
[14:30:52] <roxlu> someone using the c-driver? (looks like on of the few?) Can't find any information on how to store meta data when writing chunks with write_buffer().
[15:31:39] <dorong> is there a way to do a backup for a single server mongod, running on a raid 10 ?
[15:34:57] <doxavore> mongoexport doesn't lock on its own, but i'm not sure how consistent your backup would be.
[15:36:05] <NodeX> you wont get a consistent backup without a lock - that's the reason for a lock lol
[15:55:28] <dorong> what would you say will be the locking time for a db with 8 collection, ~42200 objects, average object size: 111, data size: 4-5mb, storage size: ~14mb, index size: ~5mb, file size: ~200mb?
[15:56:09] <ron> dorong: probably the same amount of time it'd take to count grains of sand in a bottle.
[15:56:42] <ron> dorong: in other words, there's no real way to answer that. does it run on a 486DX2?
[15:57:23] <snizzo> I'm building a little script for importing txt data into mongodb in php. I build the array in php but, once imported without errors mongo shows me empty fields :( like name : ""
[15:57:29] <dorong> well, saying there's no real way to answer that is an answer. all the other is less... but thanks.
[16:00:59] <ron> dorong: there there. at least you now have a good reason to start using a replica set.
[16:11:12] <baptistem> I would like to know if mongodb is able to add a primary key to an existing table ( == not restarting the table from scratch and add all data) ?
[16:23:39] <baptistem> Derick: on a Mysql db I found advice on internet that say : 'create a table with all good columns including your primary key columns, copy data from your old table and drop the old one'
[16:26:59] <baptistem> yes sure, my question was more is mongodb able to do better than mysql with this solution
[16:27:32] <roxlu> Derick: do you maybe know more about the c-driver? (It seems there aren't a lot of people using this) I'm trying to find out how to add meta data to a gridfile incombination with gridfile_write_buffer? (the gridfile_writer_init has no meta data parameter)
[16:28:37] <Derick> baptistem: Indexing in MongoDB can be tricky, so I would suggest you read up on a few articles and tutorials and perhaps presentations about them.
[16:29:23] <baptistem> I saw some article on hackers news I will deep read them :) thx for time/advice
[16:42:50] <MatheusOl> I mean, it would use an index even if I want, let's say, 99% of the collection?
[16:44:53] <Derick> no statistics are kept, but it does remember "the best index" for a specific query type
[16:45:25] <Derick> MatheusOl: for some information how indexes are picked and what is remembered, please read http://emptysquare.net/blog/optimizing-mongodb-compound-indexes/#optimizer
[18:48:25] <atlantaman> hi folks, trying to set a username and password using simple ruby code, so auth i still have as false
[18:49:35] <atlantaman> i make the connection and do this - add_admin = db.add_user("super","password")
[18:52:08] <atlantaman> i can do it with a shell command and --eval, but think there is a cleaner way of doing it
[19:17:53] <JakePee> smsfail: http://blog.mongodb.org/post/33837586050/reactivemongo-for-scala-unleashing-mongodb-streaming
[20:02:25] <ckd> Has anybody had problems dealing with multiple connections in the 1.3RC PHP driver?
[21:18:30] <ckd> Interestingly, I did try the other, "proper" syntax, and it appears to work correctly
[21:29:15] <mrobben> hi, i'm hitting a bug I'd like someone to comment on. When I call db.command{'convertToCapped'…} on a collection that doesn't exist, I get an error output (which is fine), but the error is returned in the result. Err is null. Gist is here: https://gist.github.com/4027649
[21:29:31] <mrobben> is this expected behavior? Should I be parsing the result field for error messages?
[21:39:33] <crudson> mrobben: I have found a couple of operations that the server recognizes as an error but the client isn't notified as such. e.g. https://jira.mongodb.org/browse/RUBY-492 - I would report it
[21:41:18] <mrobben> crudson: thx. How did you work around your issue? did you check the 'ok' field coming back from the server in the message body?
[21:42:40] <bjori> ckd: yeah, so what is happening is we are actually picking a random connection.. as if you were looking for a random secondary on a replicaset :P
[21:43:19] <crudson> I was just goofing around trying to determine what mongo does to determine a "system" collection, so I tried it. Browsing the server source code there is a IsSystem() (or similar) method that just checks the name, there is nothing else tagged for it to be a system collection, so I tried to save to
[21:43:43] <crudson> so it wasn't really an issue for me, just something I found out whilst experimenting
[21:45:35] <crudson> I would probably parse the return message for now with an obvious TODO/NOTE tag in code that this will change.
[22:01:24] <epicheals> I currently have a single mongodb server. Is it possible to "add" storage to it by setting up a new mongodb server or will it be limited to the amount of storage of the smallest instance?
[22:06:31] <ckd> epicheals: are you talking about preallocation, or are just concerned that you're limited by the size of your instance's disk?
[22:06:50] <Xorlev> Hey guys, so here's the situation: We had an issue where our secondary was catching up from an overnight event. When it was ~7200s behind, the primary fell over and the secondary (7200s behind) took over while we diagnosed the server. We brought it back up, the old primary went into ROLLBACK and then "replSet too much data to roll back" ... that's okay. How
[22:06:50] <Xorlev> would I force the old primary in as the source of truth given that it's stuck in ROLLBACK mode? I'm guessing it won't be elected primary even if I go and do a rs.stepDown()
[22:07:18] <Xorlev> To be clear, I don't really care too much about what's happened in the last few hours, I'm okay that the rollback failed.
[22:07:58] <Xorlev> I'd just like to get the old primary back as primary (it's on better hardware) and then worry about resyncing the secondary.
[22:09:51] <ckd> xorlev: Does this help? http://dba.stackexchange.com/questions/18020/mongodb-replica-set-secondary-stuck-in-rollback-state
[22:12:55] <Xorlev> Not really. I'm in the situation where I don't want to hose the server in the ROLLBACK state and instead just want it to become the source of truth
[22:14:27] <Xorlev> epicheals: Through sharding, you can get more space. Or just run everything on lvm if you're really worried about it then you can add new disks (preferably RAID pairs) to expand the space
[22:15:34] <epicheals> xorlev: cool ty :) I unfortunately can't add more hard drives, but I can learn sharding
[22:37:33] <epicheals> I'm confused about shard keys. "Shard keys that have a high correlation with insert time are poor choices for this reason; however, shard keys that have higher “randomness” satisfy this requirement better" then in the next sentence "will make it possible for the http://docs.mongodb.org/manual/reference/config-database/#mongos to return most query operations directly from a single specific mongod instance. Your shard key should b
[22:41:25] <epicheals> I'm guessing a good shard key would be something that's multiple columns with those columns being the most commonly selected columns?
[22:54:27] <Judson> db.addUser("newuser", "password") -> uncaught exception: error { "$err" : "assertion db/key.cpp:409" }
[22:57:32] <PedjaM> Records have only two fields:_id and an array { :_id => 12345, :pp => [ stuff, stuff... ] }
[22:58:14] <PedjaM> I want sharding key to be: _id%2 (because user_ids are linear, and I want both servers equally balanced on writing)
[22:58:42] <PedjaM> when I trieed, all the writing went to one server, after some time that would be balanced
[23:12:18] <PedjaM> It is kinda strange how sharding works, it would be nice if you can chose how something is sharded (% number of servers would be ideal for me)
[23:13:44] <jaha> Question: I cant seem to wrap my head around how to create this query. How would I go about taking 1 keyword/regex and search for the same keyword in 3 different fields and return a doc that matches in at least 1 of the fields?
[23:30:34] <FrenkyNet> hi all, just read that article about MongoDB best practices that was just tweeted. Is safe => true always the way to go when not explicitly set to false?
[23:33:00] <FrenkyNet> ttdevelop: http://weblogs.asp.net/andresv/archive/2012/10/24/mongodb-usage-best-practices.aspx
[23:34:35] <FrenkyNet> btw, Derander / mstearn : might be a good hint to tweet about pecl install mongo-beta so people can install the RC of php 1.3, searched my ass off trying to figure out how to get the RC
[23:35:33] <FrenkyNet> the PHP community is catching up I think, I hear more and more people are using it
[23:36:47] <ttdevelop> FrenkyNet: it is certainly nice to never have to jump through the loops to make a JSON return from server
[23:38:58] <FrenkyNet> wouldn't know, I just test the platform again scenario's and see what helps me the most
[23:46:01] <ckd> FrenkyNet: it's not necessarily their intent, but it seems to me like a good idea to have only people that can figure out how to install beta packages try out the RC
[23:46:59] <ckd> FrenkyNet: neither am I… it took me a few mins to sort it out when I first tried the 1.3 releases
[23:49:54] <ckd> FrenkyNet: I actually didn't have any problems except for a bug in the RC that the 10gen boys tracked down quickly
[23:50:53] <ckd> FrenkyNet: my particular setup uses two separate sets, one for normal shit, one for logging