Log file Viewer
#mongodb logs for Friday the 10th of May, 2013
[00:33:49] <uptownhr> db.searches.runCommand( "text", { search: '"testing\' testing"',filter:{type_id:2}, limit: 2 } );
[01:22:57] <scoutz> I'm trying to convert my replica set to replica shard on mongos I'm adding the first replica set and getting "not authorized for command: addShard on database admin"
[08:07:15] <rhalff> odd, I'm looking at the findOne() function, why is there a test whether it returns more than one result in it?
[08:12:07] <kali> sh.shardCollection( "some.collection", {_id: "hashed" }) tells me { "ok" : 0, "errmsg" : "shard keys must all be ascending" } . anybody knows what's wrong ?
[08:14:01] <rhalff> kali, anyway I'm just following the first chapter of a book, I've not done anything yet with mongodb :-)
[08:14:07] <kali> rhalff: this is just client side. tbh, i don't think the server has a findOne, even if every driver will recreate one
[08:51:06] <kali> Nodex: i'm actually using icinga, and we have a 10-line ruby script that connects and calls .database_names
[09:25:41] <bcave> I am trying to build a mongo query in java. i am trying to query by Object ID (I have the ID in string format), and now trying to build a query in java along the lines of "{ _id : { $in : [ ObjectID('1'), ObjectID('2')]}}". http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-java-driver/ isn't giving me the info I need to get this query right and i'm not finding results on google...
[09:33:11] <rhalff> why is node-mongodb-native written in javascript and not in C++, or is the compiled javascript just as fast? e.g. https://github.com/rhalff/node-imagemagick-native/blob/master/src/imagemagick.cc
[09:40:12] <bcave> Nodex: thanks. it was to do with my query syntax. I was trying "ObjectId('333')" for the OID, but the syntax for an object ID in java is "{ $oid : \"3434\" }"
[09:44:16] <kali> bcave: building queries in json string is not recommended. you should consider using the java native DBObject / DBList /... instead
[09:50:31] <bcave> kali: it's being implemented in a framework that has limitations around that. Talend.
[09:51:23] <bcave> it's not for a webapp, internal manual use only. exposure is minimal, so I doubt there'll be any room for exploits ;)
[09:59:16] <bcave> so insight into my next issue, now that my query is working.... "BSONDecoder doesn't understand type : 115 name : stName"
[09:59:40] <bcave> so it looks like I'm getting the tail end of a fieldname back from the response. some sort of truncation happening
[11:15:13] <chandru_in> Is a background index always updated in the background or does only creation happen in the background?
[12:37:23] <newbsduser> mongo --eval "printjson(db.serverStatus())" "127.0.0.1:27017/?connectTimeoutMS=5" <--- does it work correctly for 5 seconds connection timeout? to 127.0.0.1 port 27017 ?
[12:44:35] <newbsduser> mongo --eval "printjson(db.serverStatus())" "127.0.0.1:27017/?connectTimeoutMS=5" <---- it gives me valid output
[12:47:43] <newbsduser> mongo --eval "printjson(db.serverStatus())" "127.0.0.1:27017" <---- some times this command hangs.... it says "Connecting..." and wait for a hour
[12:48:23] <newbsduser> Nodex, and actually mongodb instance is up... and iam sure this problem is related with logging system
[12:49:12] <newbsduser> because when i delete all logs /service/mongodb27017/* ( <----- iam using daemontools to control mongodb instance) session resumes and working well
[13:15:35] <Nodex> sounds to me like it's a problem with whatever that is ^^ seeing as it probably works fine without it
[14:06:00] <yfeldblum> where can i find a reference manual of which javascript array methods are available in mongodb v2.2?
[14:11:25] <yfeldblum> kali, for example, Array.prototype.reduce seems to be present in v2.4 but absent in v2.2; i'm looking for a reference manual on what exactly is available in the server-side javascript
[14:12:42] <kali> yfeldblum: i'm not sure there is such a thing, but 2.2 use spidermonkey, while 2.4 use v8. most of the differences are listed there: http://docs.mongodb.org/manual/release-notes/2.4-javascript/
[14:16:39] <yfeldblum> kali, any idea what version of javascript was shipped with the mongodb-v2.2 spikermonkey?
[14:17:02] <yfeldblum> kali, e.g. js 1.6? js 1.8? i'm looking at this page: https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Array
[14:22:40] <kali> trupheenix: just setup a replica set and add it to your sharded setup like there: http://docs.mongodb.org/manual/tutorial/deploy-shard-cluster/#sharding-setup-add-shards
[14:48:21] <starfly> kali: I thought that we use the aggregation framework to set up a replicated shard? ;)
[14:49:01] <phrearch> im wondering if with mongoose its possible to query polymorphic models with a common object, like query all vehicles, whether that are cars or busses
[14:57:41] <Nodex> phrearch : if you had a key called "vehicles" then you can just query vehicles:{$exists:true}
[15:54:49] <Nodex> "why mongo sucks" - Answer = "Because I don't know how to use it and expect it to do everything that I want out of the box with no learning curve and must be CAP and ACID complient else it must suck
[15:55:11] <rhalff> kali, ok nevermind, I just spend 12 hours learning about node.js and mongodb etc. So I'm sorry if I show troll behaviour :-)
[15:58:16] <Nodex> "Schema vs Schema-less is just a different version of dynamic typing vs. static typing" - dumbest thing I have ever heard and I talk to myself LOL
[16:00:08] <Nodex> the guy knew all these things he's poiinting out regarding Joins/transaction before he started to use it... so why rant about it lol
[16:01:34] <Nodex> page 83 - when do people ever use ":" or "*" or "|" in table/collection names anyway .. wtf
[16:10:35] <Nodex> new nodex site inbound soon - not bad for a days work http://www.stage.nodex.co.uk/ :D
[16:15:32] <rhalff> Nodex, neh it's cooler to have @1996-2013 offcourse you can update it automagically
[16:16:34] <rhalff> anyway, you shouldn't lie about when you started, that's not cool ofcourse and verifyable. :-)
[16:17:00] <Nodex> the company was formed in 2008 in actual fact but only registered LTD in this year
[16:40:43] <kida78> mongodb using the OS cache is awesome, but how the heck can you take real measurements when tweaking a query if you can't compare apples to apples
[16:45:31] <bean__> kida78: i believe, on most linux boxen, you can do: echo 3 > /proc/sys/vm/drop_caches
[16:49:50] <Number6> Fetching a soda from the fridge is like using memory. Driving 4 hours and getting soda in a shop, then driving home to drink it would be the time frame you're looking at when hitting disk
[16:51:49] <kida78> if I ensure a new index on a 30m record collection, would this stop the presses?
[16:52:28] <Number6> kida78: You can use touch and a find().explain() to help preload data into RAM from disk
[16:55:36] <Number6> No, after restarting / Starting mongoDB, run a touch and a find and explain. This will load your documents into memory - think of it like warming up the database
[17:23:03] <Nodex> @Number6 : that's a good tip.... does everything need to be touched - i/e all dataabases?
[17:43:39] <mgriffin> was looking to see if mongod could run o_direct for that drop caches question a bit ago and found this
[18:07:47] <kali> it's a frequent use case, as it is one of the way to implement join, and it is optimized
[18:15:27] <kida78> otherwise, clearing the cache locally is going to require a system restart every single time
[18:18:30] <mgriffin> kida78: i read about that a bit in http://snmaynard.com/2012/10/17/things-i-wish-i-knew-about-mongodb-a-year-ago/ yesterday
[18:21:10] <kali> kida78: you have to be aware that your use case is not typical: mmongo is not design for table scan but for short, fast, small queries
[19:00:55] <devvy> Hi all. Not sure if this is the right place for this sort of question, but I'm looking at using MongoDB for building a dynamic database feature where users can essentially create their own databases for keeping track of data in their own format. The user would essentially be able to create what would be synonymous to a table, then pick and choose their columns/datatypes, the order of each, and possibly some sort of validation rules. They would
[19:00:56] <devvy> be able to record and view data with some basic searching and sorting functionality on each column. There's a possible future requirement to be able to associate records with records from other custom tables. I've looked at using an EAV model with something like MySQL, but further reading seems to indicate a schema-less solution like MongoDB may be more apt. What are your thoughts? Is MongoDB viable tool for the job?
[19:03:49] <Gargoyle> So you want to create a relational DB using a non-relational one! Are you mad?
[19:07:12] <devvy> Sort of. The only relational element is the possible association of records between models. I suppose it's a tradeoff between the relational advantages of an RDBMS vs the unstructured/schemaless benefits of a document oriented DB.
[19:35:21] <yeukhon> I use mongoengine in my web project. and i am getting this strange error: OperationFailure: command SON([('filemd5', ObjectId('518d4a904b8aa409bdf9fafa')), ('root', u'fs')]) failed: need an index on { files_id : 1 , n : 1 }
[19:35:29] <yeukhon> here is the log from mongodb. https://gist.github.com/yeukhon/8fbe2a2fd6454e264ab8
[19:35:57] <yeukhon> i googled a bit but i am not sure why the error is complaining about the index.
[20:29:33] <someprimetime1> Can someone help me figure out how to sort by hour, day, year with this query? http://pastie.org/7828346 I know basically I need to do something like var now = new Date(); and like one day would be now.getDate() - 1 but what method would I use?
[20:45:02] <novochar> after doing "sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10"
[20:45:07] <novochar> then "echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list"
[20:45:25] <novochar> apt-get update failed because "W: GPG error: http://downloads-distro.mongodb.org dist Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 9ECBEC467F0CEB10"
[20:45:37] <novochar> i'm following along with http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/
[20:57:59] <novochar> i wasn't able to verify the key, and i would like to do that still, but i was able to install mongodb nevertheless
[21:04:01] <sudobangbang> i'm getting RangeError: Maximum call stack size exceeded when i try to make a mongoose.connect( <connections_string> )
[21:04:59] <novochar> it might have been that i ran the "sudo apt-key adv..." command while already being logged in as root?
[21:10:55] <leostera> a13x212: not that I'm aware of, make sure you are dumping in a clean folder and you should be fine
[21:11:57] <sudobangbang> leostera: thanks for the look...still getting the error with this code...forget what i said before
[21:13:26] <leostera> a13x212: can't talk about such a big db, worked fine with 70gigs but never tested it to 400gb
[21:16:18] <leostera> a13x212: so *you* have to make sure it is the desired snapshot for backup/restoring
[21:17:27] <leostera> a13x212: I figure that for 400gb it will generate quite a big load so best to do it when not in high peak hours
[21:17:58] <leostera> a13x212: if anything, take a look at this: http://docs.mongodb.org/manual/core/backups/
[21:19:25] <leostera> sudobangbang: it might be possible that connect doesn't establish a connection by the time you try to register a schema/model
[21:19:49] <leostera> sudobangbang: having the on error, on open and on close callbacks log out the status is useful for this situations
[21:20:28] <sudobangbang> so instead of callbacks i can declare events like with the nodejs postgres stuff?
[21:20:33] <leostera> sudobangbang: try using the connect callback to define the schema and well see
[21:21:14] <leostera> sudobangbang: but it shouldn't be that hard, just add a subscriber to a db:connected event and publish it from within the connect callback
[21:29:28] <sudobangbang> leostera: that fixed the immediate issue but it also uncovered more "RangeError: Maximum call stack size exceeded" regrading node event module...fuck me
[21:30:39] <novochar> still reading the tutorial, what's the relationship of a database and a collection?
[21:31:28] <novochar> it's not immediately obvious that test_database contains test_collection right now, if that's the case
[21:33:39] <leostera> sudobangbang: for someone who claims to use a node module for its beauty you write pretty distasteful code :)
[21:36:39] <leostera> sudobangbang: well, for starters your indentation is inconsistent. Sometimes you leave a new line between lines, sometimes you don't. Your variable notation is somewhat consistent. You should mess less with prototype as well…specially in the built-in objects, you might break some internals (counters mostly).
[21:37:58] <kida78> so i'm running an aggregation where i want the end result to be { a: frank, b: tony, rows : 4 }
[21:38:38] <leostera> sudobangbang: data-id is not a valid variable name, nor is location-hash. So car_instance.data-id…and you get my point.
[21:39:34] <kida78> my aggregation is a match on a = frank, a projection on the fields, and a group by B
[21:49:44] <kida78> i'm thinking of only doing the first match,project,group, and then doing the rest in the code.
[21:53:20] <locojay> hi running 2- replicaset. when adding auth=true in boath conf i get or using --auth i get replset error could not reach/authenticate against any members
[21:58:40] <TommyCox> hm, the second doc isn't valid json. maybe you accidentally mucked it when cleaning?
[22:06:14] <locojay> all fine when auth=true is commented out. but adding auth to either of the 2 conf generates replset error could not reach/authenticate against any members. any idea?
[22:14:37] <kida78> TommyCox: did the last one work or was it still busted? what are you trying out?
[22:15:05] <TommyCox> I am just trying to run the match and project to see the structure of what you're grouping
[22:24:55] <TommyCox> Whew, okie dokie. So to clarify, you just want the count of the rows where a has b?
[22:25:54] <kida78> the first 3, i believe gives you the count of date rows, and drink rows, as separate results for the same person
[22:28:26] <kida78> do you see { _id : { b : "samantha", "type" : "DRINKS" }, "sumIntermediate" : 1, "count": 2, "drinks_count" : 1, "dates_count" : 0 } for example
[22:37:11] <TommyCox> well, you wouldn't have to project and regroup if the dates and drinks were in separate collections.
[22:40:43] <sudobangbang> using mongoose, i can .find() an object in the database and in the callback print it out
[22:40:45] <sudobangbang> but i when i try to access one of it's properties ( it's obviously there ) it tells me it's undefined
[22:42:05] <kida78> if i do a match on a, then sort by B, wouldn't it make the group function faster?
[22:42:20] <TommyCox> That sounds like the way to go. While elegant, that query produces more overhead than needed i feel
[22:47:06] <TommyCox> Well, sort can benefit from an index before grouping, so theoretically it would improve performance
[22:49:31] <TommyCox> yeah, sorry i couldn't be of more assistance. If i had more time i'd do a little testing myself
[22:50:14] <TommyCox> this might help as well http://www.10gen.com/presentations/mongosv-2012/mongodb-performance-tuning