Log file Viewer
#mongodb logs for Wednesday the 12th of March, 2014
[00:51:42] <tomas> hi I'm very new to nosql world, can anyone please tell me what is the proper way of referencing to linked document from the document I've queried? For example I've got: http://pastebin.com/ARzsTd7A
[00:57:52] <joannac> tomas: to get the note from the author document, you would need to do another find with the objectid in your author document
[01:01:17] <hoverbear> joannac: I need to interpolate X programmatically, but JS doesn't like logic in keys
[01:27:43] <hoverbear> joannac: Either that or if I could iterate over to match some value, there's less then 12 of the X
[01:29:05] <hoverbear> joannac: I'm basically having a data race issue, I need some way of detecting when my values are stale.
[03:41:32] <l_s> after importing a csv file using mongoimport, any operations i do with the collection will return a syntax error, even with db.collection.drop(). what could be the problem?
[03:54:58] <joannac> I'd say it's more likely to be something like this http://stackoverflow.com/questions/12719859/syntaxerror-unexpected-token-illegal
[04:22:32] <hoverbear> joannac: I can't find any mention of a $pos operator. Is there a way to make it check each item in the array?
[06:10:58] <Mothership> is there any place i can download precompiled mongodb for arm architecture? don't have to be latest
[08:17:52] <ron__> when I try to retrieve the records matching the date rage (createdDate $gte), it works fine except the date to check is not Sunday
[08:20:18] <ron__> for example db.record.find({"createdDate":{"$gte":"Fri Feb 14 16:25:36 GMT 2014"}}) ---> this works
[08:20:42] <ron__> db.record.find({"createdDate":{"$gte":"Sun Feb 16 16:25:36 GMT 2014"}}) ---> this does not works
[09:16:50] <Diplomat> Array ( [project_id] => 12 [site_id] => 10 [click_time] => Array ( [$gt] => 1393970400 [$lt] => 1394661599 ) )
[09:16:58] <Diplomat> Array ( [project_id] => 12 [site_id] => 10 [click_time] => Array ( [$gt] => 1393970400 )
[09:38:24] <Nodex> also in your query, if you're adding an index for it, make sure "click_time" is the last part of the index as it's a range
[09:39:05] <Nodex> and I think you gain some APC performance when casting as it doens't have to check things all the time
[10:14:36] <Nodex> if you want garunteed latest document you must do find({...}).limit(1).sort({_id:-1});
[10:53:40] <abishekk92> Hi, one of our secondaries stopped playing the oplogs, as the disk space was maxed out. So I add more space in the partition containing the data directory, and I check db.printReplicationSlaveInfo on the Primary and ot says it was synced 1 sec ago, however, the new data doesn't appear on the secondary. Any idea why this is happening?
[11:16:51] <csrgxtu> and this object contains Username, Password, how can i retrive the Username directly
[11:17:44] <kali> csrgxtu: DBObject extends BSONObject, so its get("Username"). see http://api.mongodb.org/java/2.6/org/bson/BSONObject.html#get(java.lang.String)
[11:19:25] <smargareto> It processes a file that has 1M records, accumulates the data in 6 ways and then inserts (upserts) in mongo
[11:20:42] <smargareto> the problem I have is that when I insert in only 1 table of the 6, I get good speed (~2s of "pure" db work)
[11:21:13] <smargareto> but when I start adding more tables, the times of the other tables are worse and worse
[11:23:16] <smargareto> the interesting numbers are the last ones: 3666ms to upsert 414610 records, and 1465ms to upsert 261749
[11:32:12] <Gr1> and I got a message that replication should not be enabled on a config server when I try to start the daemon as mongod --configsvr --dbpath /var/lib/mongodb/ --port 27019 --config /etc/mongodb.conf
[11:32:41] <Gr1> Does this mean that the config server would hold only meta data and not the actual db data?
[11:35:24] <kali> Gr1: and even if your three config server "mirror" each other, they are not a replica set, it's a totally different protocol
[11:36:30] <kali> Gr1: so your typical sharding setup should have 3 config servers somewhere, at least one 3-node replica set somewhere else (even if two or more makes more sense), and some mongos somewhere else
[11:36:30] <Gr1> Thank you Kali. I see. So if I am setting up an environment where I need data replication, and sharding, what all services should I run?
[11:38:46] <Diplomat> it says this Array ( [updatedExisting] => [n] => 0 [connectionId] => 17 [err] => [ok] => 1 )
[11:39:53] <kali> Diplomat: man, is this unreadable. i suggest you try manually your query in the mongodb shell :)
[11:43:16] <Derick> Diplomat: can you please read: http://derickrethans.nl/mongodb-type-juggling.html
[11:46:15] <ajph> is it possible to group by regexp in an aggregation query? for example, $group: { "test": { "$sum": { "$regexp": "^a" } } }
[11:47:50] <kali> ajph: i doubt this $group will produce anything interesting even if it valid. can you show us an example document and explain what you're trying to do ?
[11:51:44] <ajph> kali: i'm looking to sum all fields that begin with "a" : http://pastie.org/8911174
[11:51:47] <smargareto> The dictionaries that accumulate the data are flushed to mongo each 5000 lines
[11:52:20] <smargareto> and they are processed in the reverse order of the log (WebAccumulationCategory..WebNavigationUri)
[11:53:17] <ajph> Nodex: hmm, now i see the problem with not posting an exact use-case. i can't, no. let me get a better example
[11:54:00] <smargareto> it's like the connection gets tired because the times of the first table do not degrade
[11:55:53] <kali> ajph: you should consider changing your schema, actually. if you have to reason on key fields, many different things in mongo will get in the way
[11:56:42] <kali> ajph: consider moving from { key1 : value1, key2 : valueé2 } to [{ key: key1, value: value1}, { key: key2, value: value2 }]
[11:57:30] <ajph> kali: really i'm looking to sum all fields that have the format test.(.*).clicks where (.*) is unknown. i don't use arrays because i need to upsert.
[12:41:56] <Gr1> If I have config server and mongod instances running on the same server with different port, which one should I connect to, to make a query?
[12:43:32] <Gr1> Derick: Please correct me if I am wrong, but I seems to be able to connect to mongos only when I start the config server. So isn't mongos the config server instance?
[13:04:28] <amitprakash> for example combine two of db[collection].update(filter_criteria, update_dict)
[13:05:37] <amitprakash> the issue is with combining update_dicts, the first update_dict could be {'$addToSet': {'key': value}} and the second update_dict would be {'$addToSet': {'key_alt': value_alt}}
[13:14:35] <amitprakash> Something like db.collection.update({'$and': [filter1, filter2]}, {'$and': [update1, update2]})
[13:21:43] <Nodex> you can use multiple addToSet's in one query as long as they don't change or attempt to change the same value
[13:29:53] <muraliv> hi. is there a way to hard-code a key in map-reduce emit function? in a composite keys situation.
[13:30:03] <muraliv> hi. is there a way to hard-code a key in map-reduce emit function? in a composite keys situation.
[13:59:19] <amitprakash> Nodex, point wasnt multiple add to sets, the point was two dicts containing addToSet and merging them somehow
[14:04:36] <Sidd> I am having trouble with connection string, or so I think. I wrote a test application that works with connecting to my mongod instances, but I am trying to test connecting to my Mongos instance. I have sharding and replication set up. Everything appears to be working fine.
[14:05:05] <Sidd> I am using the official C# driver, the message I get is "{"Unable to connect to server nc-dev-api:27018: Configuration system failed to initialize."}"
[14:05:59] <kali> Gr1: it's not supposed to happen. if you see this message, you're just not connected on the primary (or there is a terrible bug, but i doubt it at this stage)
[14:07:24] <Gr1> All I did was rs.slaveOk() but this error came a couple of times afterwards. then it disappeared.
[14:10:50] <kali> mmmm i think you're confused. it's a mongodb shell settings and it only affects the current session
[14:14:20] <smargareto> I have posted it in stackoverflow: http://stackoverflow.com/questions/22354246/mongo-upsert-performance-c
[14:16:19] <Sidd> If someone can help me with my issue I would appreciate it, I have been unable to find anything with google searching, I am sure Im not asking the right question, but I posted on stack overflow http://stackoverflow.com/questions/22354298/unable-to-connect-to-mongos-in-mongodb
[14:19:24] <eagen> You want to connect to the mongos host and port, not to one of the shards directly.
[14:20:12] <Sidd> however i just fixed it....some local data file was corrupt and was causing the problem
[14:23:09] <Sidd> Haha, I don't know why I didn't look at the inner exception sooner. It was having trouble reading the xml from the app data file. I don't think this is the first time this file has messed up on me either.
[14:24:05] <saml> something like pymongo http://api.mongodb.org/python/current/api/pymongo/mongo_replica_set_client.html
[14:24:54] <kali> saml: MongoClient is supposed to be the one and unique, i think it's the case for casbah
[14:25:14] <Sidd> I would think you still need to specify all mongo host in the mongo uri for the connection string
[14:27:56] <kali> saml: it's deprecated in favor of MongoClient, and all drivers are supposed to converge as much as possible to the same API
[14:28:29] <saml> kali, so, a good practice is to use MongoClient and specify all possible mongohosts in mongo uri
[14:30:06] <saml> http://api.mongodb.org/python/current/api/pymongo/mongo_replica_set_client.html#pymongo.mongo_replica_set_client.MongoReplicaSetClient no mention of deprecation
[14:31:01] <saml> http://api.mongodb.org/c/current/api/annotated.html i don't know how to find MongoClient here
[14:31:17] <kali> saml: well, i don't know about pymongo specifics. my reference would be that http://derickrethans.nl/mongoclient.html and various blog entries/discussions indicating a wish to make all APIs similar
[15:13:34] <devdazed> i have about 1400 dead hosts in the "Mongos" section and from what i see, i can't remove them
[15:15:14] <jarjar_> I'm having some trouble running my script on nodejs with mongoose... keep having this error when I launch it:
[15:15:16] <jarjar_> TypeError: 'undefined' is not a function (evaluating 'inherits(InsertCommand, BaseCommand)') /var/node/suchmaschine/node_modules/mongoose/node_modules/mongodb/lib/mongodb/commands/insert_command.js:38
[15:23:43] <kali> Derick: you may want to confirm or infirm what i was saying about the drivers and the client classes
[15:45:33] <KamZou> Hi, i've a strange comportment in my MongoDB : when i type : show dbs i get 200GB for a specific database. And when i use this db and type a db.mycollection.stats i've 100gb+ difference. Any idea please ?
[15:49:05] <kali> KamZou: http://docs.mongodb.org/manual/faq/storage/#why-are-the-files-in-my-data-directory-larger-than-the-data-in-my-database
[15:50:47] <KamZou> kali, so the following command : "show dbs" is listing the room user on my disk ??
[17:16:55] <Wil> Hey, I was hoping someone wouldn't mind taking a few minutes and looking over an aggregation pipeline for inefficencies I'm sure exist. Trying to optimize a database that shouldn't be so bogged down with the small amount of traffic it's getting.
[17:18:35] <Wil> The collection itself is just a time-series database where each document is the address, createdAt time, the values at that time. Index on both the address and the date.
[17:22:30] <Wil> I'm using mongoose with node, and I marked both as an index, but I don't think it's a joint index.
[17:23:57] <Wil> This is my first time making use of mongodb. That aggregation pipeline is being used by ~5000 people every 5 minutes to every 24 hours. Depends on the granularity.
[17:25:03] <Wil> But, again, I feel like these numbers are extremely slow to be causing such a load.
[17:30:47] <Nodex> can you run an explain() on your match (just do a find({address:....,createdAt:...}).explain();
[17:47:16] <Wil> Nodex: Without the single address index, even if queries are run without the createdAt field, does it still make use of the compound index?
[19:21:21] <Diplomat> can you please explain me why mysql fanboys are saying that mongodb is web scale
[19:21:57] <Diplomat> http://www.mongodb-is-web-scale.com/ i read this.. and it just sounds like one cocky tech guy talks with a normal dude
[19:22:45] <Diplomat> most of the DBAs are just sad and lonely , that's why they talk like bastards -.- lol
[19:26:52] <simpsond> "..... If you were stupid enough to totally ignore durability just to get benchmarks, I suggest you pipe your data to /dev/null. It will be very fast.", love that quote from the transcript
[19:39:42] <rob_1234> hello, I have a question: if I have a collection where the _id is a subdocument, { _id: { first_name: "", last_name: "" } }, can I add a field to this subdocument later (as in: db.collection.update( {}, { $set: { '_id.middle_name': '' } }, { multi: true } ); ) ?
[19:45:17] <jrdn> So, we're using pre-aggregated data and the aggregation framework for our reporting (with pivots).. When we add a lot of data, the browser will crash.. so, with that said, we need pagination.. Does anyone know a good place to start for paginating aggregation framework queries effectively?
[20:27:30] <samgranger> Hey Mongo guys - I'm fairly new with Mongo, and have a few small questions - anyone here able to answer a few?
[20:29:26] <samgranger> Well, first of all - lets say I have a few records in a table - if I select a random row and delete it straight away after selecting,is there a chance, even a tiny tiny chance, that another request manages to select the same record?
[20:30:22] <samgranger> And if so, is there a different secure way I can accomplish this? I want an option to only allow records to be read once basically
[20:33:36] <samgranger> Might be a weird question but I'm building a web app with certain unique keys in mongo - and let's say someone requests a key simultaneously, I don't want it to be possible that they receive the same one
[20:36:37] <Joeskyyy> only way that could happen would be if you're in a replset and you delete, but you're reading from a secondary
[20:40:10] <Joeskyyy> Essentially you can tell an operation to not respond back until it's done replicating
[20:40:27] <Joeskyyy> Then you can avoid it there as well. OF course there's a performance cut there, but take that as you will
[20:44:49] <Joeskyyy> sistema nao pode encontrar o caminho especificado = system ….. something specification? Mind translating
[20:45:33] <LucasTT> sistema nao pode encontrar o caminho especificado = system couldn't find the especified path
[20:46:51] <Joeskyyy> it probably created those, but somehow in the configs its trying to reference it's missing out on the char
[20:47:38] <Joeskyyy> http://windows.microsoft.com/en-us/windows/create-user-account#create-user-account=windows-8
[20:48:09] <Joeskyyy> http://www.howtogeek.com/howto/5261/beginner-geek-add-a-new-user-account-in-windows-7/
[21:47:51] <mu> Hello. Can I ask a question about using mongodb here or is this a development channel?
[21:49:19] <mu> I have a collection in which we store a bunch of documents with an array of tags... We want to be able to optimize a query for all documents with the same tag, and it looks like it's done through: ensureIndex("tags.Id": 1) and then just querying on that tag. Is that correct or is there more to it?
[21:52:16] <mu> in the constructor we just do: @Document = mongoose.model('Document', DocumentSchema)... I'm thinking below that is we add an ensureIndex like I described?
[21:55:47] <Derick> in general, show it how mongo does things - not with some framework/ODM on top of it
[23:02:24] <evilC> Hi all, new to mongo here, and having a little trouple making the conceptual leap to mongo. Any help? My data structure is like this:
[23:34:39] <unholycrab> is it considered bad form to run the mongo config servers on the same instances as the mongod instances?
[23:39:30] <unholycrab> the documentation for sharded clusters says "Three config servers. Each config server must be on separate machines."
[23:40:00] <synth_> i understand the return value of pymongo.ASCENDING or DESCENDING will be a -1 or 1; I'm just curious if the php driver has a variable I can call versus using an integer
[23:40:08] <unholycrab> its a little ambiguous. does it mean at least three clusters? does it mean i can put my config servers on my mongod instances?
[23:40:27] <unholycrab> or is it stating that i need exactly three config servers, each on a dedicated machine
[23:50:10] <ElephantHunter> The server's responding with "Max number of collections exceeded" every time I attempt to add a collection
[23:50:33] <ElephantHunter> even when I'm running in completely new instances (incognito window) that have no existing collections