Log file Viewer
#mongodb logs for Thursday the 18th of October, 2012
[00:02:36] <Notionwork> And I'd love to get mongodb up and running so I can do some awesome app development
[00:21:38] <_m> Quick google gives me this: http://stackoverflow.com/questions/2394532/apache-is-unable-to-initialize-module-because-of-modules-and-phps-api-dont
[06:57:14] <finalspy> hi everybody, does anybody knows of a way to add a replica set to a mongo instance which hasn't been started with -replSet and without restarting it ? In other words create a replica set on running instances of mongodb ?
[07:49:16] <tunnuz> Hi there, does anyone know if I can restore some databases from some .ns, .0, .1, … files?
[10:36:44] <Bartzy> But bigger ones are seen as strings inside NumberLong: NumberLong("100001112653593")
[10:38:37] <plow> hi guys, I've got heterogenous data, ranging from say 1kb to 4mb per document, should I use gridfs or just a normal collection generally speaking?
[10:49:47] <plow> will there be much of a performance cost using gridfs over a document for smaller docs?
[10:53:15] <samurai2> this is really confusing, it's said to do db.fsyncLock() before shutdown the mongod and do the backup, but if we do that, we can shutdown the mongod
[10:55:08] <samurai2> actually this is the first time I do the backup on mongo sharded and replicated clusters
[10:55:38] <NodeX> shitting down the mongod will stop the mongodump from running as it connects to it
[10:56:15] <NodeX> "mongodump cannot be used in this locked mode. If this is a problem consider one of the other methods."
[10:57:50] <samurai2> so what is the proper method to do the backup for sharded and replicated clusters? using snapshot?
[10:58:31] <NodeX> the method described in that doc allows you t lock the DB and use LVM to snapshot it
[10:59:36] <samurai2> so first after I stop the balancer on mongos, I do mongodump --db config on one of config server machine, then after that
[11:00:35] <samurai2> then do what you're saying before, which is lock the db and use lvm to snapshot only to this secondary machine, is that correct?
[11:00:44] <NodeX> you'd be better off waiting for someone who does backups and things on shards - perhaps kali can advise if he is alive
[11:03:34] <kali> i don't care too much about consistency, my whole stack is designed to fail gracefully when something's missing
[12:05:45] <kristaps> Is there a way to show the status of every member of a cluster (replicasets as shards) when connected to an appropriate mongos?
[13:29:56] <mcarroll> if I want to force our PHP application to connect to one specific mongo server that's part of a replica set, do you have to specify the replica set? or is it transparent to the application whether or not it's part of the replica set?
[13:34:16] <mcarroll> my problem is that we're getting a "master not found" error. while we're fixing the replication, I want to point directly to the master (instead of the replica set)
[13:38:28] <aster1sk> This is a huge blocker for my company https://jira.mongodb.org/browse/PHP-535#comment-176220
[13:39:25] <Derick> aster1sk: something we will be looking at, but at least we need to get 1.3 out first
[13:39:47] <Derick> however, you can ofcourse directly connect to a secondary (without marking the connection as replicaset)
[13:40:14] <Derick> aster1sk: 1.3.0 first - we're hoping (fingers crossed) for rc1 tomorrow for that
[13:42:33] <aster1sk> True, and that works beautifully, however unfortunately not with aggregate()->slaveOkay()
[13:45:30] <aster1sk> Derick: You lost me.. But I think for now I'll just use the find() and aggregate the result in PHP (ughh it feels yucky).
[13:47:22] <aster1sk> NodeX: you're saying omit the array('replicaSet' => true) in the connection options and just supply the secondary host?
[13:48:42] <aster1sk> NodeX: I'll give it a shot, but I'm pretty sure the driver will figure out who master is and php_mongo_connection_force_primary()
[13:51:39] <[raymond]> I'm trying to export a collection using: "mongoexport --db website --collection users --csv --out /tmp/users.csv" but I'm getting the error "assertion: 9998 you need to specify fields"
[13:51:46] <[raymond]> From the documentation it looks like I shouldsn't need to provide a field list, am I reading it incorrectly?
[14:24:44] <aster1sk> I'm considering writing a web based mongodb data model creation tool complete with query builder.
[14:29:10] <aster1sk> Derick: I have come across it in my research, yes - but it doesn't play nicely with our spaghetti framework.
[14:31:53] <NodeX> I dot see why someone would choose mongo say for performance then bloat the app out and slow it back down with doctrine or similar
[14:32:10] <aster1sk> ron: don't care for PHP? The output would be JSON so doesn't matter much what the tool uses to generate that.
[14:34:05] <ron> aster1sk: well, I php is the only 'language' I don't respect.. but it's quite off topic for the channel so let's leave it be.
[14:35:12] <remonvv> The point I was trying to make is that it's not relevant what it's used for. The problem with PHP is PHP, the language.
[14:35:23] <ron> it's compiled to c/c++ and they did it because php is crap, but again, let's leave it at that.
[14:36:00] <remonvv> Not really. There's not a single thing PHP does better than a suitable alternative. PHP is legacy, so is Java for example to a large degree.
[14:36:41] <NodeX> I think in the next week or 2 10gen are releasing via their PR people a press release about my company so I'll be sure to include lots about PHP in there just for ron!
[14:37:09] <aster1sk> :) I wrote this tool in PHP to build my current project http://stats.dev.riboflav.in/app/issue_unique_visit
[14:38:55] <aster1sk> We're extremely write heavy and only Chuck Norris can upsert on positional, so I used the MySQL ID's as keys.
[14:39:04] <ppetermann> NodeX btw, hiphop is not really php.. its php code transformed to c++ and then compiled ;)
[14:39:44] <remonvv> ppetermann, I use Java but that has similar issues of being somewhat vebose/outdated. I'd like to use something like Scala, Ruby, etc.
[14:39:59] <remonvv> Preferably something JVM based because I feel that's easily the most mature platform at this point.
[14:40:14] <remonvv> I like language discussions. You can take either side and have valid arguments ;)
[14:40:45] <Aim> remonvv: yeah indeed, someones taste is also important in choosing a language to code in :)
[14:42:15] <remonvv> It's almost a religious argument. That comparison holds up because facts aren't a huge part of the discussion ;)
[14:43:38] <remonvv> ppetermann. you don't understand internet discussion. We all won and we all lost. It's a matter of perspective.
[14:44:51] <ron> http://cdn.buzznet.com/assets/users10/funbobby/default/arguing-internet--large-msg-114407728999-2.jpg
[14:48:59] <Vile> i mean fulltext search engine that can search hierarchy of documents not as a flat list
[14:51:27] <Vile> task is to find a document based on a number of search terms, which can appear in the documnet itself or in the any of the folders above
[14:52:31] <ppetermann> so make your argument why it is bad, but please dont copy that horrible article that everyone is linking in the last half year when talking about php, cause half of it is factual wrong
[14:53:33] <ron> fine. you forced me. http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/
[14:53:56] <Vile> i.e. i can represent it as a flat list, which will include info from all the folders above for each of the documents and search that
[14:55:54] <Vile> NodeX: imagine you have 1 000 000 documents, and the top folder updated every few seconds
[14:56:00] <NodeX> Unlike (literally!) every other language with a similar operator, ?: is left associative. So this: <---- in no language should you EVER stack tenery operators - this guy can't even write good code
[14:57:45] <aster1sk> I must add git to that stack as well, I've a openvz CT I build gitlab on top - srsly the most awesome.
[15:00:02] <Vile> but if i search using two terms and one of those terms in the document and another in the folder that contains the document, this is considered a match.
[15:22:23] <telmich> how do I add / remove nodes from a replica set that used to have three members, of which two are turned off?
[15:25:55] <telmich> the rs.reconfig() command does not work, because the last node is a secondary - same with rs.add()
[15:48:09] <Derick> if you only want a specific nested field, you could do that with the aggregation framework, but it's probably not worth doing that
[15:49:11] <NodeX> select z from things where x="john" -> db.things.find( { x : "john" }, { z : 1 } );
[15:50:59] <ppetermann> its not that weird that it expects an array - what else would you want to give it?
[15:54:21] <ppetermann> 2011-07-03 allow to pull only certain.... <-- thats when i added $fields as array in my framework
[15:59:52] <esiegel> I'd just like to make sure I'm doing this right: $ mongodum p --host mongodb1.oursite.com --out ~/oursite/ --db database
[16:00:15] <esiegel> Basically, I'd like to take the database from our server and back it up to a folder on my computer.
[16:04:12] <esiegel> NodeX, thanks. I'm gonna give it a try and home that everything doesn't explode!
[16:16:33] <mongodb> Can anyone recommend a great mongodb hosting platform? I am currently looking at object rocket, mongoHQ, and mongolab
[16:17:01] <mongodb> Crap.. I jacked up my username.. while adding myself to the channel. Hold on guys.
[16:17:33] <Bartzy> NodeX: Hey, sorry, was away. About NumberLong("34253245325") - the quotes are supposed to be there? Because sometimes they are there and for smaller numbers they are not there.
[16:18:15] <NodeX> it's to do with 64bit int's Bartzy ... if you ask Derick he can explain better than me
[16:18:15] <wereHamster> Bartzy: my guess would be that they are there if the number can't be represented by a javascript number.
[16:19:01] <NodeX> from what I remember the driver will cast it back to an int for you on the way out
[16:20:11] <Slaytorson> Now that I got my name straightend out. Does any know of a great mongodb platform? I have been leaning towards ObjectRocket.
[16:22:59] <Slaytorson> wereHamster: I am wanting to know if anyone has any positive and/or negative experiences with any of them. I am looking for one for mission critical business applications. Obviously I found 3 of them.
[16:26:25] <Slaytorson> Ron: wereHamster: I have googled and done plenty of research. I figured I would ask people specifically in this channel because, well it is the mongodb channel. If you haven't had experience with any of them then you shouldn't be duesh bags about it and you should keep your mouth shut.
[16:27:02] <ron> Slaytorson: I fear that the channel isn't active enough to have people with relevant opinions, but you'd have no way of knowing that unless you're a regular.
[16:38:26] <aster1sk> I'm evaluating it, really dreading OPS on mongod -- would prefer someone else do that bit.
[16:39:00] <aster1sk> Yeah, they must have a lot of something... Can't imagine how much traffic they get... I could see that being the bottleneck here.
[16:39:19] <aster1sk> I just changed my connection string and it was functioning immediately without any BS
[16:40:01] <aster1sk> Is there a tool / algorithm / calculation to guage how big your data set will be?
[16:40:43] <aster1sk> Yeah, I'mm aggregating on MySQL 600 Million rows, so will be significantly smaller.
[16:41:39] <aster1sk> Well depends how much time you'll spend fiddling with mongod ops, backups etc.
[16:42:16] <aster1sk> If CTO spends even two full days mucking with issues / backups etc that's more than the service costs.
[16:42:34] <aster1sk> Also it's third party liability -- therefor if sh1t breaks you have a finger to point.
[16:43:43] <aster1sk> Consider hardware costs, liability, sustainability, uptime... it could get very expensive.
[16:43:57] <algernon> you have to buy the hardware, hire someone to do it, and if it all goes down the drain, that's a lot of money wasted
[16:44:01] <NodeX> one of my clients pays me a weekly retainer to make sure all his stuff is running fine
[17:16:53] <awc737> is it a bad approach to add a new a new global "key" with a default value on thousands of users at once?
[17:19:51] <wereHamster> awc737: if it's a default value, same for every user, then there is no need to store it in the database.
[17:23:46] <awc737> but wereHamster, I need to be able to pick and choose individual users to receive a field when I create it globally.
[17:24:06] <awc737> won't I need to store something on that user document that says they have that field?
[17:25:48] <aster1sk> Heh so with Heroku / MongoHQ I was able to deply my URL shortener in < 7 minutes : http://5kb.us
[17:26:21] <wereHamster> awc737: whenever you load the document from the database, and the field is not set, you assume the default value. If the field is set, you take whatever it's set to.
[17:26:38] <awc737> wereHamster, you said theres no need to store it in the database. but I do need to store who gets that field on their profile
[17:26:54] <NodeX> awc737 : [18:23:51] <NodeX> db.foo.update({query},{$set:{bar:'baz'}},false,true);
[17:27:38] <zane> So... I did something stupid and modeled a collection in Mongoose with the name "stats". Needless to say, I can't find a way to interact with this collection from the shell... eg, db.stats.renameCollection "is not a function"
[17:31:06] <NodeX> if it's index bound then it could be deemed expensive - as wereHamster says depends on the number of docs
[17:34:19] <joec> I have a secondary server running 30 minutes behind and the web app that accesses it is coded to allow secondary, is there anything I can do server side to force only reads from the primary?
[17:38:02] <dblado> hi all -- found a post online about saving a deref() function to the server to get referenced documents but not sure how to run the custom js function with the find() -- anyone point me in the right direction?
[17:38:13] <Bartzy> wereHamster: Never thought about that - not saving default value for most records on the DB, and setting it when I load the model...
[17:38:27] <wereHamster> joec: http://docs.mongodb.org/manual/reference/glossary/#term-hidden-member
[17:38:46] <wereHamster> joec: http://docs.mongodb.org/manual/administration/replica-sets/#replica-set-hidden-members
[17:39:52] <wereHamster> I just use: field: { type: String, default: 'teh default value' } and I'm done.
[17:41:15] <wereHamster> I just use: doc.field = 'whatever'; doc.save(); or: doc.field = undefined; doc.save()
[17:41:25] <NodeX> wereHamster : what happens if you want all of your documents to have that value in the DB?
[17:42:34] <Bartzy> wereHamster: And you're doing that to save space in the DB right? Space/index size etc ?
[17:43:16] <NodeX> [18:37:40] <wereHamster> I just use: field: { type: String, default: 'teh default value' } and I'm done.
[17:43:49] <jawr> http://api.mongodb.org/cplusplus/current/classmongo_1_1_b_s_o_n_obj.html#afdb218ee1301ab651692f96bec95d0d9 once i get an ObjectID, how do i extract the creation date?
[17:43:54] <wereHamster> NodeX: then I'm not understanding your question. Do you mean if I want to set the default value on all documents in the database?
[17:45:24] <NodeX> no, you said that you just do "field: { type: String, default: 'teh default value' } and I'm done." if you use an ODB
[17:46:11] <wereHamster> NodeX: all it does is: after loading the document from the database, if the field is not set, set it to the default value. So in my application the field is always there, set to a non-null value.
[17:46:40] <NodeX> yes but that's not really a help if you want it in the database and dont want to wait for the ODM to update everything
[17:47:10] <wereHamster> if you want it in the database then you have to $set it, there is no way around that.
[18:01:16] <NodeX> if you want to get your head round Mongo then forget everything you know about SQL
[18:04:14] <awc737> so in mysql I will have a users table, and a users_attributes table. users_attributes will just have id / key / value / ordering
[18:04:54] <awc737> if the admin adds new user_attribute, such as 'website', and only selects one user group, it will then add that key / value pair to each user document in that group
[18:06:20] <wereHamster> awc737: isn't your question basically how to map your mysql model to a mongodb model?
[18:07:24] <NodeX> because modeling an app to work in both will be a headache because Mongo is so flexible
[18:07:27] <awc737> wereHamster, my reason is that I don't know what key / values a product might have. i might define one globally for all products, and I might add one to one product
[18:09:58] <NodeX> some people do use mongo for certian things, it sounds like you're trying to have your cake and eat it
[18:10:16] <tystr_> so we're getting random load spikes on our mongodb primary, and we suspect it's due to disk io latency during the fsync op
[18:10:26] <NodeX> the people that tend to use mongo and SQL in the same world dont tend to mix tables and try to model the same thing in both sysems
[18:11:55] <awc737> trying to decide which is best. My company uses SOLR becuase they think they need "fast indexing for search queries", and I thought I should probably look for something comparable
[18:12:56] <awc737> right, but I will probably use elasticsearch, in which case having my products with an attributes key/value, would probably play nice in sending that data to ES
[18:13:26] <NodeX> ES is a little green and buggy for my taste but I know plenty of people who do use it
[18:13:43] <awc737> so I was thinking let me admin define fields in SQL, select which groups or users receives that field, and that stores the attribute / default value into the product document
[18:15:05] <awc737> the concept here, (also relevant for Mongo), is what leads me to believe I *should* use NoSQL: http://neilmiddleton.com/the-wonders-of-hstore/
[18:15:34] <wereHamster> well, if he already has an existing sql database, and wants to keep using it, then adding yet another database is unnecessary overhead
[18:16:13] <NodeX> if you have the need for flexible schema's that your app decides on then I suggest mongo
[18:18:04] <awc737> well I was thinking both would be nice because my admin could define a global attribute, and I could use SQL relationship to determing which products had the attribute
[18:18:24] <awc737> and IF the attribute value is changed for a specific product, then that changes the product's document value
[18:37:41] <Goopyo> Hey what would the use cases for asynchronous mongodb? Is that just for instances where you dont really care about the response?
[18:39:31] <konr_trab> I've got three tables, foo, bar and baz. I'd like to concatenate them into another table, FBB: how can I do so?
[18:40:30] <Goopyo> http://stackoverflow.com/questions/5681851/mongodb-combine-data-from-multiple-collections-in-to-one-how
[18:49:55] <konr_trab> I thank you, dear friends, for your help and wonderful DB, and so I say goodbye and wish you a merry day!
[19:01:01] <tystr_> is it possible to have secondaries in a replica set only replicate certain collections?
[19:40:25] <tystr_> Question: Is there a limit to how many connections can tail a capped collection?
[20:13:21] <dominikd> i have a collection i'd like to expose as an API, is there an easy way or should i whip something up in python+flask?
[22:27:37] <_m> For those of you who don't get the email blasts, there are upcoming free online MongoDB courses
[22:31:04] <opus_> I'm pulling results from my collection, which contains rows of rows. I want to modify one of those rows inside the result, do I really have to do forEach and then find it then modify it?
[22:32:42] <opus_> like, this is sort of what I want to do: db.company.find({name:"bob"}).find({store:"washington"})
[22:34:07] <_m> You're trying to modify the row in place, or do you need to manipulate it in your stack?
[22:37:20] <_m> So you want to find the company with a specified 'name', and a 'store' nested document within said company's document, correct?
[22:37:36] <opus_> i'd be nice if I could just db("collection.company").find("bob").("stores").find("washington").set("status=closed") (obviously psuedo code)
[22:42:10] <_m> Also see: http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24elemMatch
[22:42:10] <opus_> ok, any other keywords I could use to google an answer for? i'm mostly using node.js
[22:46:33] <jtomasrl> how should i find the table with number = 1? [{"name": "foo", "tables": [{"number": 1}, {"number": 2}]}]
[22:49:28] <_m> If that field is an array within a document, you'll want to prepend the container field's name to that query. EG: db.foo.find({ "bar.tables.number" : 1 })
[23:03:48] <_m> You can do the filtering client-side, or move the tables to a separate collection. By http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields - only the tables field - this should be reasonably fast.
[23:23:00] <opus_> But it seems to return all of the nest, and just the column that I wanted to match on