Log file Viewer
#mongodb logs for Tuesday the 23rd of July, 2013
[01:09:23] <naegling> anybody have a clue why these aggregates would return three different numbers of clicks? ---> http://pastebin.com/2GmFT4rn
[05:22:41] <dgaffney> and I'm just wondering when the light at the tunnel will appear as the commands its running (from looking at the log) look remarkably similar from the hour before
[06:37:10] <amitev> i have a requirement to generate sequential ids and have to guarantee that there will be no records with the same id and there will be no skipped ids in the sequence
[07:33:13] <kali> amitev: locking is not very mongo-ish, you'll be on your own: you can implement a mutual exclusion with atomic operators on a well known document
[07:33:59] <kali> amitev: for the "working" case, it's pretty straight-forward, but the devil is in the details and the error code path :)
[07:36:11] <kali> _pash: mongodb will enforce its uniqueness on one collection. the rest is up to you
[07:36:47] <_pash> how do i make sure that the _id parameter is completely unique accross all my databases?
[07:43:40] <_pash> "MongoDB uses ObjectIds as the default value for the _id field if the _id field is not specified" so if i dont specify the _id parameter when I am passing the data, then mongo uses ObjectId(), which is completely unique. Correct? kali
[07:45:58] <_pash> Nodex: I am reading it, slowly as you see I am quoting them :) As I said, I cant read all the docs in one night.
[07:49:04] <_pash> Nodex: you might think its easy and self explanatory, but as you have said: "I have been doing this for a very very long time". Knowledge = Time and Practice
[07:52:07] <amitev> kali: i guess that i need something like this - http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/
[07:53:45] <Nodex> _pash : you couldn't even be bothered to read the docs on ensureIndex last night, it explains it better than we can.
[07:54:55] <_pash> Nodex: reading and understading are two different things friend. You cant know if I have read them or not. In fact I have read them.
[07:59:43] <Nodex> I know you didn;t read it because the question you asked made NO sense if you had read it friend
[08:01:04] <kali> amitev: yeah, that will ensure uniqueness, but if for some reason (bug, crash) you can't insert the document after having consumed an int, your sequence will have hole
[08:01:10] <Nodex> exactly the same as if you had read about ObjectId's you wouldv'e known the were unique
[08:07:45] <_pash> Nodex: here is the thing, nowadays, reading docs, searching the web, etc, can give any one person an answer to any question. So then what is the point of irc i ask you? To get a real time, live reply, right? Not to get bashed when you come to ask for one. I understand I have to actually start looking at the docs before coming in here, and asking the question first. I shall start that, but it is a subconsious
[08:23:47] <Nodex> I am not bashing you, I am asking you to read the docs before you ask a question. Both questions i have seen you ask in 2 days are both covered in the first 5 ilnes of the respective pages in the docs.
[08:53:42] <spicewiesel> hi all, I could need some help on deploying a sharded cluster. The first step was the creation of the first replicaset, I did this with auth enabled, so my first question: Do I have to add an admin user one one/all replicaset members or do I have to add this admin user via mongos later.
[09:01:25] <remonvv> http://docs.mongodb.org/manual/tutorial/enable-authentication-in-sharded-cluster/
[09:14:52] <spicewiesel> that's what I did, but I did it while creating the cluster, not as a change later
[09:17:58] <remonvv> spicewiesel, try it and see. I never use auth on dev or production environments.
[09:18:57] <spicewiesel> I added an admin user later, as described in the tutorial, after logging in with that user, I cannot add shards or see the sh.status. I'm not authorized
[09:27:55] <elon> Is there any way to insert a document only if it doesn't already exist, apart from find()ing the document and insert when find() returns nothing?
[09:29:46] <elon> remonv:"The update() method can either replace the existing document with the new document or update specific fields in the existing document."
[09:30:15] <remonvv> elon: Yes, but you can either not update anything or update fields to what you know would be the correct value.
[09:30:29] <remonvv> elon: This is one of those things where it's easier to tell us what you want to do than how you want to solve the problem.
[09:32:44] <elon> remonvv: I get the idea, and in practice there's no real difference(updating a document with exactly the same values if it exists vs checking if the document exists, and if it doesn't creating one), but I thought there might be a dedicated method that doesn't overwrite anything.
[09:33:51] <remonvv> elon: Hm, there is up to a point. You can use upsert in combination with $setOnInsert
[09:38:23] <elon> remonvv: I see. Although I could add every field to $setOnInsert, I doubt there's a significant performance gain(if any!), and it would make my code much less clear. Thanks for looking that up, though.
[09:44:10] <remonvv> elon: There isn't any performance benefit if you're going to insert the entire document, no. And no problem ;)
[11:24:42] <pithagorians> hey folks. i'm new to mongo. i setup a 3 members replication and i kill the mongod process on primary member than start it again and i get in logs recover create file /var/lib/mongodb/local.25 2047MB
[12:09:36] <spicewiesel> I have 3 clean mongod instacnes, 3 clean configservers and the mongos. I want to set up ohne replicaset now and add it as a shard later. But I do not understand when and where I have to set the adminuser. Do I have to create the replicaset without user? do I have to create the admin user on every single mongod instance? Whould be great if anyone could help me with that.
[12:39:01] <spicewiesel> I tried some things now and everytime I ended up in an "not authorized". Other users told me that the userAdminAnyDatabase role isn't enough, but the official tutorial tells to use this one for the admin
[12:40:11] <spicewiesel> I was able to add the admin user on my first mongod instance and it was synced in the rs00, as I was able to connect from the other instances with that credentials. But after that, I was not allowed to to rs.conf() with the admin user...
[12:43:27] <rspijker> spicewiesel: ah, okay. Well, I haven't used it myself in a long time, but this is what I remember
[12:43:48] <rspijker> you can just set it on the primary (after or before, shouldn't matter) and it will sync to the others
[12:43:54] <spicewiesel> and, if servers are started wit replSet option, I am not allowed to create an auser ("not master and slaveOk=false"), So I first have to create the rs, then I could create the user. Dunno if that is correct :)
[12:44:55] <spicewiesel> rspijker: Ok, then it should be ok if I first create the replSet (using the localhost bypass to have the rights) and then I could create the adminsuer on instance1
[12:45:50] <spicewiesel> I think, with enabled replSet you have to create that set before creating users
[12:48:18] <spicewiesel> ok. I did rs.initiate and rs.config now, the replSet is now syncing. After that I could create the admin user, not?
[12:54:16] <spicewiesel> The tutorial tells userAdminAnyDatabase, users yesterday said it's not enough, and an other howto adds the admin without any specific roles, just with "readOnly" : false
[12:56:02] <rspijker> so just do exactly this: http://docs.mongodb.org/manual/tutorial/add-user-administrator/
[12:57:24] <spicewiesel> but, should I do that on my mongos, or on the mongod instance (f.g. PRIMARY)
[13:03:38] <Michae___> Hi channel, just looking on db.config.collections and wondering what the the key "unique" is telling me
[13:03:48] <spicewiesel> narf... I switched to db admin and created the user: admin, role "userAdminAnyDatabase". Now I'm no more allowed to use rs.conf()
[13:15:11] <Michae___> I know that the underlying index pId is unique, but db.config.collections says unique: false
[13:29:18] <Mr_O> i'm using javascript shell script to do updates on a collection. The db.collection.update does not return any status. How do i get to know if update applied or when somehting goes wrong ?
[13:30:35] <spicewiesel> the mongos knows from the confgdb parameter, where the configservers are running, but where is the connection to the mongod/replSet? I should auth against admin db at mongos to be able to setup my shard, but the auth fails. I think mognos/configsvrs just don't know about the replSet, where my admin user is configured now.
[13:54:17] <Nodex> there are those of us that can code and those of us that just make drinks for people ron :)
[13:55:06] <Nodex> http://motherboard.vice.com/blog/this-is-what-wi-fi-would-look-like-if-we-could-see-it
[13:55:28] <spicewiesel> rspijker: I'm still trying. if my admin user got that role I'm not allowed to do replicaset stuff. I created the admin without a specific role now, and now I was able to set the replSet. After that I started my configservers, all fine there. Now I started mongos and added my rs00 as shard, that's done, too.
[13:56:00] <spicewiesel> I think I'm still confused. I cannot auth against db admin on mongos with my admin user.
[13:57:06] <rspijker> so adding it to your primary will add it to the secondary (it syncs) but not to the mongos
[13:57:38] <spicewiesel> ok, so how to I get that connection? The WebApp on the mongos server should be able to use the database I will create later.
[13:58:55] <spicewiesel> ok, than should I see the user management on the mongos completely seperated from the mongod/rs?
[14:02:33] <rspijker> spicewiesel: Come to think of it… I think you need key files for authentication on shared clusters...
[14:03:05] <spicewiesel> yes, I'm provided the key file to every component, mongod, mongo-configsvr and mongos
[14:05:52] <spicewiesel> ok, so if I add a cluster admin on mongos, it will be stored on the configserver. That's fine, because my next question was "where does mongos store it credentials as it's not using a local data directory?" :)
[14:14:26] <spicewiesel> I have now an admin user for my rs00, and a clusterAdmin, created on mongos.
[14:17:17] <spicewiesel> now I have to solve the next problem, the new clusterAdmin is not allowed to execute sh.status(), as it's "not authorized for query on config.version". Same with show dbs and other commands :(
[14:20:44] <spicewiesel> both seems to be fine, then I tried to get the status of my shard added earlier.
[14:22:25] <spicewiesel> userAdminAnyDatabase, as told here http://docs.mongodb.org/manual/tutorial/add-user-administrator/
[14:23:28] <spicewiesel> I swtiched to db admin and executed: db.addUser( { user: "clusterAdmin", pwd: "secret", roles: [ "userAdminAnyDatabase" ] } )
[14:26:30] <spicewiesel> I would like to habe a admin user, later I need a database and a user the app could use to work with that new db
[14:42:55] <spicewiesel> the last step would be to find the correct roles for the application user, should be readWrite on it's database, right?
[14:44:24] <spicewiesel> "By the way, one can still sign up for the course at this point so tell your friends!" from todays newsletter :)
[14:52:05] <DestinyAwaits> spicewiesel: no.. it's ok friend... I should have told you what Course I have enrolled for.. :)
[15:09:53] <Dreamer3> ERROR: BSON::InvalidKeyName:key Dealer.com must not contain '.' - can anyone point me to the commit that started causing this?
[15:10:15] <Dreamer3> we were using mongo / bson 1.6.2 ruby gems now that we're on 1.8.6 this is cropping up, but i can't find the commit that changed it
[15:18:34] <rspijker> Like I said, some drivers used to allow it. not 100% sure but I think it's still not enforced by mongos, only the driver
[15:18:55] <Dreamer3> right, we didn't change mongo versions, so that can't be the problem… mongo definitely allows it
[15:23:00] <rspijker> https://jira.mongodb.org/browse/RUBY-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
[15:28:59] <Dreamer3> https://github.com/mongodb/mongo-ruby-driver/commit/4cc201a9dd2a6e67738a3803a8954acd06bd1496
[16:04:57] <mdeboard> How do I deal with results returned from Casbah (Scala client)? It's just returning plain objects, and nested properties are quite difficult to work with, requiring manual finagling
[16:05:31] <mdeboard> I don't mind rolling my own solution but I'd like to make sure I'm not missing something obvious
[16:07:43] <mdeboard> http://mongodb.github.io/casbah/guide/querying.html#query-operations as well, should I just give up and go read the Java driver docs?
[16:11:04] <rspijker> that should probably be your best resource though if you want to stick with it
[17:22:17] <__ls> hi everyone; i'm having an "BSONObj size is invalid" assertion error; i tried db.repairDatabase() but the issue persists. here's the offending code: http://pastie.org/8168046, and here's the error log: http://pastie.org/8168038
[17:23:46] <__ls> the error doesn't occur if i request 1700 instead of 1800 documents, but I'm not sure it's strictly a size issue; it might just be that somewhere in the final 100 there's a corrupted document in there (maybe?)
[17:24:54] <__ls> intuitively, the log "First element: ns: "esri.block_group"" would speak for something non-document specific (I've found other logs from people where the "first element" was an actual document), but then again, this is way beyond my mongodb knowledge ...
[17:27:41] <Kanara> can anyone point a newbie to info on why save() & insert() in node.js would complete happily but not store any data?
[17:42:00] <zachrab> so im saving a Time.now stamp in ruby to a document in mongodb and it converts it to a ISODate
[19:10:31] <Kanara> its really strange, no errors or feedback of any kind, just runs through the script quite happily but nothing gets inserted
[19:17:23] <beniwzb> Hi all, there's something I'm missing about the docs. I have created an admin user, which has the following privileges: "userAdminAnyDatabase", "userAdmin", "readWriteAnyDatabase", "dbAdminAnyDatabase", "clusterAdmin" , however, if I connect with the mongo command line client I get: connecting to: test authentication fails (code 18) anything I'm missing here?
[19:18:15] <beniwzb> Also for completeness: If i us "admin" as the database on the command line, the client connects fine and I can see all databases
[19:35:41] <Kanara> ok, the mystery deepens. if i connect to "database1" which has been accessed by the node app I cant insert but if i connect to a new database i can, until the node app connects then the db becomes permenantly useless
[20:53:34] <jcromartie> are there any problems with using collection names that contain dots, hyphens, etc?
[20:54:16] <jcromartie> in my simple tests it works fine but I just want to make sure there isn't some edge case where it fails
[20:57:44] <jcromartie> I'm primarily using this from Clojure where hyphens and dots in symbols and keywords are not a problem
[21:00:29] <kali> so it's explicitely forbidden (but mongodb may let you shoot yourself in the foot)
[21:42:40] <VarunVijayarghav> When doing an endureIndex(), is there a difference between doing ensureIndex({timestamp: 1}) vs ensureIndex({timestamp : -1})
[21:52:23] <jcromartie> cheeser: yeah, but I don't care too much about field names except for the hard limitations
[22:16:43] <giraffeslacks> hi. i'm building a ticketing system and am fairly unfamiliar with the mongodb paradigm. generally tickets get a unique sequential ID. mongodb doesn't do it without workarounds and to the detriment of scalability (according to the research I've done.)
[22:17:13] <giraffeslacks> so i'd like to do this a more appropriate way from the start. any suggestions on how to use _id to represent/generate a human-readable ID?
[22:17:46] <giraffeslacks> i was thinking of displaying just the last 5-6 digits of the _id... but that seems like a plan for failure (even if a collision might be extremely unlikely.)
[22:18:05] <preaction> giraffeslacks: the ORM i'm using does it for me, but it still has the _id. keep the _id, add a number field
[22:19:46] <giraffeslacks> cheeser: yeah. i didn't want to write my own (even though the mongo docs explain it pretty well.)