Log file Viewer
#mongodb logs for Thursday the 31st of October, 2013
[03:32:18] <joannac> SteveDC: Don't ask you ask; you'll get much better results if you ask your question outright
[03:32:38] <joannac> LoneSoldier728: That doesn't parse. How much space a single document takes? A collection?
[03:37:40] <SteveDC> I have installed mongodb on centos 6.4 from the 10gen and 10gen-server i start the service and it starts with no problem. I then attempt to connect to mongo trys to connect on 127.0.0.1:27017 returns with exception failed to connect. I have shut it down and removed the mongo.lock and done a repair then started mongo attempted to connect again and got the same error might anyone know why this
[03:43:46] <joannac> SteveDC: does the mongod.log say anything? ps -ef | grep mongod shows it running?
[03:50:11] <SteveDC> No there is nothing in the mongod.log at all except for the start or restarts. Nothing showing for failed connections
[04:22:18] <SteveDC> LoneSoldier: are you hosting it on your machine? or did you purchase it on a server
[04:25:57] <SteveDC> 27017 is the default mongo port, but documentation also says 28017 can be the default port to
[04:28:16] <flyankur> If I have ObjectId, Is it possible to fetch its document without knowing its collection ?
[04:29:27] <flyankur> Lets say If i have a "product" collection, which has a field reference to another collections - "store" & "category"( like foreign key ) - I am using node.js & native mongodb driver. While iterating If i have ObjectID("52710f953564d744923a2f64"), how do I fetch its document, without knowing to which collection it belongs ?
[05:00:20] <LoneSoldier728> anyone know how to run this correctly I am having trouble with the last part
[08:54:54] <flyankur> If I have ObjectId, Is it possible to fetch its document without knowing its collection ?
[09:30:21] <flyankur> NodeX: Please have a look at this example of 2 documents from related collections - https://gist.github.com/flyankur/7244345 , If while iterating through product, I want to get each ObjectID's details too - How should I do, as I don't know what collection to this ObjectID is from.
[09:32:43] <flyankur> Like, Django has an ORM for RDBMs, If I query product model, I can get product.store.name & the whole store model also , but in NoSql, I am getting product, product.store._id only, not the whole store document.
[09:39:11] <flyankur> jonaswouters: check this - http://easeyourshop.com:8082/db/eysapp/product/52710f953564d744923a3005
[09:39:47] <flyankur> can you see ObjectID everywhere where there is a reference to a document from other collection.
[09:40:14] <flyankur> Instead of ObjectId("234234234234"), I want to display the name of may be make it clickable.
[09:41:12] <flyankur> I am ok with that, but unable to figure out. Tried brute force, looped over all collections & did findone. Have a feeling thats not the right ways.
[09:44:20] <joannac> if(key == "store_id") {mystore = db.store.find({"_id": value})} else if (key == "category_id") {...}
[09:44:33] <flyankur> joannac: So, I am right then, with just the Bson ( which includes ObjectID ) while iterating through it, Either I have to store collection type along with it or use DBref ?
[09:45:35] <flyankur> In case of DBref, I dont need to hardcode in the code the to figure out collection ?
[09:46:21] <flyankur> This approach, if(key == "store_id") {mystore = db.store.find({"_id": value})} else if (key == "category_id") {...} - Does not look scalable in my case.
[09:48:21] <flyankur> Imagine If I am creating PhpMyadmin Kind of a tool for MongoDb, I will need DBref.
[09:50:30] <flyankur> joannac: I am building a generic Admin tool/ UI for MongoDb which provides form to edit any document, rather that just plan document Bson - which is not easy to edit / use for a non-tech content team
[09:51:37] <flyankur> joannac: yeah, I read that , I guess that is why there is no Admin tool for Mongodb which provides better/usable CRUD interface
[10:08:03] <flyankur> NodeX: So, current challenge I am facing in a project is - I need to give UI to the content team who is using this app I am building on Node.js & Mongo ( PHPMYADMIN type ) - where the content team can just see records & when they want to edit it - it opens up in a more intuitive interface - a Form
[10:09:35] <flyankur> NodeX: I am using https://github.com/flyankur/mongo-express as my boilerplate, have been able to Edit doc UI to - http://easeyourshop.com:8082/db/eysapp/product/52710f953564d744923a3005
[10:13:12] <flyankur> currently the problem I was facing was, all the reference field are coming as ObjectID(52710f943564d744923a2f59),etc - which is again not human readable - So I want to replace that with its lets say - category.name or store.name - because I want to make it generic, I don't want to hardcode in Node.js app - like if(key == "store_id") {mystore = db.store.find({"_id": value})} else if (key == "category_id") {...}
[10:17:37] <flyankur> Ok, so there is something called embedding too.. I did not knew that ( I am sorry, Its my 8th day with Mongodb ) - Looking into it quickly & will get back to you
[10:20:33] <flyankur> NodeX: as far as I read just now, are you suggesting I should not have multiple collections ( eg. category, sub-category ), rather just embedded.. I want to keep category, stores as seperate collection as well
[10:21:10] <NodeX> you keep a "category" collection but embed the name of the cateogry rather than its ObjectId
[10:22:48] <flyankur> NodeX: On the similar note - how does Joins type data handling work then - for me to achieve that I will have to embed the whole document ?
[10:27:54] <flyankur> NodeX: you mean store/embedd as much data in one document - does not matter if its duplicated ?
[10:32:50] <NodeX> flyankur : the basic concept is this... if you can make your page load with 1 query this is optimal. However, there is administration which has to be done to pages / data but it's accessed less frequenctly
[10:33:18] <flyankur> NodeX: there are long list of cases where joins are very much required. Specially in an E-commerce app ( should I query - may be using Elasticsearch or something ), if so RDBMS has an advantage in e-commerce use-case , where I need to generate all sort of reports , apply filters
[10:33:39] <NodeX> for example. If you had a document with a users "country" - some people (lunatics) would probably relate that to another table/collection but lets be honest how often does the name of a country change
[10:35:12] <chukaman> hi. i have a question. if i'm running on ubuntu 10.04.4 lts and i use the official 10gen "ubuntu-upstart" repo should i be weary about upgrading a cluster comprising of 2x mongos, 3x conf servers and 4x shard servers with a replica set spanned across all of those shard servers from mongodb version 2.0.7 to version 2.4.7?
[10:35:15] <NodeX> it really does depend what your end goal is for using mongodb and why you're using it
[10:35:17] <flyankur> NodeX: what if I want list of all countries , or country with maximum number of lunatics :)
[10:36:02] <NodeX> and secondly for the maximum you can aggrgate that quite easily without a join :)
[10:36:35] <NodeX> performance carries a penalty, in this case it's duplication of data and more work on administrating that data
[10:38:58] <NodeX> For me disks are cheap so data duplication is not an issue and I have been working with mongo for so long it's second nature to write code in a way that makes manegemtn easier
[10:39:40] <joannac> chukaman: you'll need to upgrade to 2.2, then 2.4. Take backups. Take downtime. Go to 2.4.6, there's a problem with config servers on 2.4.7 which is being fixed
[10:40:16] <NodeX> flyankur : almost anything you can think of. CMS, CRM, Job Board, Social Network, Dating sites
[10:40:48] <NodeX> I don't use relational databases at all except when I need to do a transaction that might need a rollback which is not often at all
[10:41:14] <flyankur> NodeX: I have felt the breeze of less complexity too, I also use mongo with Django - and now there are not more migrations & what not :)
[10:41:49] <NodeX> I am finding that I can do things now that really are not possible with relational systems
[10:43:17] <NodeX> we're hoping to open source it one day - when it gets a clean up!! - I write messy but efficient code haha
[10:48:14] <NodeX> you mentioned ElsaticSearch for searching. Personaly we use solr, I find it a lot more mature as a project, nothing against elastic search though
[10:50:13] <flyankur> NodeX: I also use Both, Solr is definitely more mature ( I use it here - thefingo.com ), But in some particular usecases I figured elasticsearch might just be better.. still experimenting though
[10:51:33] <NodeX> good news, it's the mongodb installation, bad news, I don't have a clue how to fix it
[10:54:04] <NodeX> and when it does start it causes some weird error with the PHP driver by giving out jibberish as the db host name or somehting
[10:54:26] <NodeX> yet I can connect via the shell fine and it ingests data (it's a slave) with no problems
[10:56:14] <NodeX> 3 other slaves on the same kernel work fine, this one doesn't. All installed in the exact same way too
[10:58:32] <Derick> NodeX: As a hunch - looks like issues with linking against boost - perhaps linked against an older version of the lib, or perhaps with the wrong header files
[11:01:12] <NodeX> I've googled the error and a few other people have had it. It seems it's something to do with the kernel
[11:01:28] <NodeX> no the other machines are fine, this is the weird thing. they all have the same kernel image on them too
[11:39:56] <benjamin___> I have a dataset consisting of addresses and geojson Points. I'd like to be able to do a query to find addresses near a particular address. I've got as far as being able to specify coordinates to search near, but not a single query. In SQL I'd use a join; is this something I can achieve with a compound index?
[11:43:48] <Derick> benjamin___: you need to do two queries. The first to find the address that you want to search near, and the second one to do the near query that you've already described
[11:44:52] <benjamin___> OK Derick, thanks, I was just wondering if there was a specific way. I'll continue to do it in that way then, thank you for clearing this up
[12:33:54] <amitprakash> Hi, is it possible to ask mongodump to dump a database for which records created are a day or less newer/
[12:34:11] <amitprakash> since ObjectIds are based on timestamp, this should be a doable query for all collections
[12:37:12] <amitprakash> Right.. but queries target a single document, I want mongo to query every single collection
[12:39:00] <amitprakash> NodeX, okay, and how do I specify a query to check that ObjectId to some gt datetime?
[12:42:01] <amitprakash> NodeX, one last clarification.. mongodump -d DB -q "{_id: {\$gt: <RelevantOidForDatetime>}}" will dump all collections in db after executing that query against each collection?
[12:42:58] <amitprakash> I am going to, but on the prod servers so don't want to task the servers heavily :D
[12:43:13] <NodeX> I have never tried it that way, I have always loop the collections and add the query to -c
[12:45:32] <NodeX> For example you might have a backups collection. You create an OID now and insert it as a job, next time you can take that OID as a base
[12:46:31] <NodeX> infact you would have to add the new OID before starting the export as you might have inserted data between the two that wont get backed up
[12:48:00] <NodeX> it's a pretty enat way of creating incremntal backups BUT it doesn't backup changes that might pre-date the oid
[14:11:59] <eldub> when I create a new database ona primary in a replica set, shouldn't I see the same db created on the 2 secondaries?
[14:13:24] <eldub> My other confusion is when I do a show dbs -- I have a "local" that is 10GB -- which I never created
[14:42:05] <kali> eldub: http://docs.mongodb.org/manual/faq/storage/#why-are-the-files-in-my-data-directory-larger-than-the-data-in-my-database start there
[14:57:35] <dasmith91> Hi everyone, I'm getting started with Mongo and MongoEngine and was wondering if someone could advise me on how to structure data? (Still thinking like I'm using SQL)
[14:58:27] <dasmith91> I've got a setup like this: https://gist.github.com/dasmith91/b432f1440d24e690f9da
[14:59:05] <dasmith91> This crashes as User isn't defined on line 11, however switching the classes results in the script crashing as Post is then not defined.
[15:12:27] <rickogden> NodeX: yeah, I was thinking that, an opening and closing time for each day. Just wondered if there was a good way of storing it so I could write a query for shops open at a specific datetime?
[15:14:12] <cheeser> though you might be able to use aggregations to apply time functions to those dates.
[15:21:49] <eldub> when I do an rs.status() I noticed that my db1's name is it's IP address and the other 2 have their host name
[15:31:41] <eldub> http://docs.mongodb.org/manual/tutorial/change-hostnames-in-a-replica-set/ -- in case anyone else was curious
[15:37:40] <eldub> LoneSoldier728 I see. I'd be able to help with a backup / restore but never with your method
[16:42:47] <acalbaza> real quick question... how does the client shell know about hosts in a cluster? if i test failover, it works as expected, but when i do the same thing in an application, it does not.
[16:43:16] <Derick> the shell, as well as all the drivers (providing you use the correct mode) will all detect failovers
[16:44:18] <acalbaza> ok.. i'm working with spring-data + mongo so i need to understand how to configure this client.
[16:59:37] <acalbaza> wondering if auto-connect-retry is what i need to supply in order to get this behavior
[17:50:55] <tbjers> astropirate: what's in your package.json, what node.js mongodb module are you using?
[17:51:55] <tbjers> @Number6: if he's using the raw node.js driver, yes, he could have mongoose installed, what do I know?
[17:52:54] <tbjers> @Number6: I guess that's technically not a driver, but I meant what module was he using
[19:07:13] <tkeith> I set up a replica set using hostnames instead of localhost, but I need localhost. It seems like the simplest solution would be to complete remove the replica set config from mongodb and create it again. I don't have any data I need. How can I completely reset mongodb? Can I just clear /var/lib/mongodb, or will that have bad side effects?
[19:08:36] <revoohc> How does mongodb deal with timezones? Is it safe to move replicate to a server running UTC when the current master is EST/EDT?
[19:10:11] <kali> revoohc: unixes servers do not have timezone, they deal with UTC internally, so no problem
[19:41:23] <umi> I'm trying to perform an update inside a forEach loop. The KEY used in the update is part of the foreach object.
[19:42:32] <umi> i.e. forEach(function(cat) { db.stats.update({id: cat.id}, {$set:{cat.type: cat.mouse}})}
[19:43:09] <umi> I'm trying to refer to part of the earlier object, and not use a period in the key name, yet it still errors out on me.
[19:43:57] <umi> I only pasted a snippet of my full query to show the relevant part. If I replace cat.type with a string like 'test', it works, but I don't get the data I want.
[19:57:58] <kali> umi: var setter = {}; setter[cat["type"]] = cat.mouse; db.foo.update({}, { $set: setter})
[20:02:30] <umi> @kali found out if I create an object then I can do that 'var obj = {}; obj[cat.type] = cat.mouse;' then use obj in the set
[20:39:29] <l0st0ne> I have an odd issue, wondering if anyone has some thoughts…. I have a couple thousand missing documents in a sharded/replicated db.collection. When I query a mongos instance using an indexed query (not using fields in the shard key) the documents are not returned. If I run the same query on one of the db instances I find the documents. Anyone seen anything like this? We are on 2.4.3
[20:44:21] <kali> l0st0ne: i have had one like these at some point. i was using a hash-base sharding, and one mongos had been forgotten in a 2.2.x version, so it was writing some documents on the wrong shard
[20:46:41] <l0st0ne> Kali: thanks, we are not using a hash-based shard key, but I was thinking that maybe it was left behind in chunk migration
[20:58:42] <eldub> anyone know how to edit the init.d of mongo 2.4.6 so it starts up with interleavinga ll
[21:18:44] <eldub> this seemed promising but is completely different than my init.d http://stackoverflow.com/questions/11161308/starting-mongodb-via-numactl-as-daemon