PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 16th of May, 2015

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:05:36] <fructose_> StephenLynx: I think I'd have to omit the _id then. Is there a value for _id I can pass that would be the equivalent of omitting it?
[00:06:16] <StephenLynx> just ignore it.
[00:06:17] <StephenLynx> completely.
[00:06:17] <StephenLynx> and mongo will handle it.
[00:07:53] <fructose_> StephenLynx: I'm trying not write two different things to do insert and update. That seems to be the purpose of upsert. So if I'm passed an ID and I don't know if it exists or not already, what do you suggest?
[00:08:23] <StephenLynx> ah.
[00:08:38] <StephenLynx> for one, I ignore _id as an application unique id.
[00:08:40] <StephenLynx> index*
[00:08:57] <StephenLynx> I just declare my own unique index and use it.
[00:09:49] <StephenLynx> when I used upsert I didn't had that problem because of that, I didn't relied on _id for the update, but on a different field.
[00:10:10] <StephenLynx> lets see what you could do...
[00:10:37] <StephenLynx> you could use application code to add the _id field on the query and projection blocks depending if the _id was informed.
[00:11:23] <StephenLynx> or try and see if you pass the _id as null will make mongo realize what you are trying to do and just create a new _id on its own.
[00:21:14] <fructose_> Can anyone tell me why I'm getting no error, doc is 0, and nothing is being inserted here: http://pastebin.com/id7KWcsP
[00:23:16] <StephenLynx> may I give you several suggestions?
[00:24:00] <fructose_> StephenLynx: As long as they're not to hold me hostage before you tell me what's wrong :)
[00:25:09] <StephenLynx> don't use express, don't use mongo. use just the runtime environment and the node.js driver.
[00:25:14] <StephenLynx> and don't nest callbacks.
[00:25:50] <StephenLynx> when you are inside a callback, use conditions and write nothing but a single line inside of each conditions, each line proceeding the logic on a separate function.
[00:26:31] <StephenLynx> and try to check what you are getting on "doc"
[00:26:38] <fructose_> It's 0
[00:26:47] <StephenLynx> and error is null?
[00:26:51] <fructose_> Correct
[00:26:52] <StephenLynx> or undefined?
[00:26:55] <fructose_> NULL
[00:27:01] <StephenLynx> yeah, try with just the driver.
[00:27:41] <fructose_> Does that mean it looks correct and you don't know why it's not working?
[00:31:08] <StephenLynx> that means I couldn't understand jack because of the ODM that requires the knowledge of a sub community and the bad formatting.
[00:32:12] <StephenLynx> ODM and framework*
[00:33:10] <fructose_> Are you referring to monk? Because I don't see how anything else is even relevant
[00:33:27] <StephenLynx> yes, monk.
[00:34:24] <fructose_> What are you suggesting I use instead? You said "don't use mongo". Did you mean monk?
[00:34:45] <StephenLynx> let me look back what I said
[00:34:46] <st34lth> so I can't connect to db because of auth schema
[00:35:00] <StephenLynx> yeah, I meant "don't use monk"
[00:35:05] <StephenLynx> typo :v
[00:35:17] <StephenLynx> stealth, auth schema?
[00:35:30] <StephenLynx> mongo doesn't have schemas. what exactly is giving you this error?
[00:35:34] <StephenLynx> are you using an ODM?
[00:36:07] <st34lth> how can I create MONGODB-CR instead of SCRAM-SHA-1
[00:36:30] <StephenLynx> ah, hold on
[00:36:36] <StephenLynx> I think I am starting to get it.
[00:36:49] <StephenLynx> yeah, I remember that you can tell mongo how you are authenticating.
[00:37:10] <st34lth> StephenLynx: curious case, right, I create a user and can log in from command line, but none of the client will auth using user name and password, that I can verify works in the cli
[00:37:18] <fructose_> StephenLynx: Is there anything wrong with mongoose or mongoskin?
[00:37:54] <StephenLynx> froctose_ mongoose is cancer, almost any issue with people using mongo with node/io is because of mongo being bad.
[00:38:01] <StephenLynx> and I assume mongoskin is not much better.
[00:38:05] <st34lth> in the log I can clearly tell 2015-05-15T20:29:30.121-0400 I ACCESS [conn2] Failed to authenticate dbadmin@admin with mechanism MONGODB-CR: AuthenticationFailed MONGODB-CR credentials missing in the user document
[00:38:44] <StephenLynx> IMO any tool that does anything you can already do, but does different is bad.
[00:38:53] <StephenLynx> like ODM's and web frameworks.
[00:39:21] <StephenLynx> stealth, show me your code that you are using to connect.
[00:39:46] <StephenLynx> I managed to connect to instances with auth on by simply formatting the connect string.
[00:39:53] <st34lth> mongo --port 27017 -u dbadmin -p --authenticationDatabase admin
[00:40:07] <StephenLynx> that is in your application?
[00:40:19] <StephenLynx> if you provide a lone -p you can't provide the password.
[00:40:38] <st34lth> it makes me type the password after providing -p
[00:41:01] <fructose_> StephenLynx: Is there a tutorial you'd recommend for the native node.js driver?
[00:42:09] <st34lth> I'm fine with the cli, but I need to show the data to other folks, and I automatically taught of a client, postgres, mysql, etc all have clients like that, so i looked into robomongo and mongohub.
[00:42:24] <st34lth> *thought
[00:42:49] <st34lth> and their connection is failing
[00:43:23] <StephenLynx> what about your application?
[00:43:23] <StephenLynx> from what I understood you are able to connect using the terminal.
[00:43:38] <StephenLynx> aaah
[00:43:47] <StephenLynx> I dunno, I don't use any other client than the terminal.
[00:44:02] <StephenLynx> fructose_ the official documentation.
[00:44:41] <StephenLynx> http://mongodb.github.io/node-mongodb-native/2.0/
[00:45:53] <st34lth> I'm not even creating data yet at the application level because I wasn't sure if application will also require MONGODB-CR
[00:46:17] <st34lth> also help me understand this right
[00:46:19] <st34lth> sudo /usr/local/bin/mongod --auth
[00:46:22] <StephenLynx> in my application code I just don't specify the auth mode.
[00:46:40] <StephenLynx> I suggest using the mongod.conf file to specify if auth is on or off.
[00:47:03] <StephenLynx> so a reboot won't change it
[00:47:51] <st34lth> alright, I was following the docs, http://docs.mongodb.org/manual/tutorial/enable-authentication-without-bypass/
[00:48:13] <st34lth> so best practice for mongo is to just install and use no credentials?
[00:48:50] <StephenLynx> if you only connect locally or is a server not meant to be actually used.
[00:49:01] <StephenLynx> because the default settings disable external connections
[00:49:09] <StephenLynx> so you don't need auth if you don't use external connection.
[00:49:34] <st34lth> I was not intending to use an external connection, but it's good to know
[00:49:37] <StephenLynx> I have a server running with just the default settings, no auth.
[00:53:03] <st34lth> I took your advice, connected on first try without creds in mongohub and robomongo StephenLynx
[00:53:44] <st34lth> I went on a tangent trying to fix this auth thing because I thought that will be a standard. next time, I just stick to book. lol
[00:53:55] <StephenLynx> :v
[03:45:57] <erobit> does anyone know the chance of collision for object uuids in mongoose driver across collections / within same db?
[03:48:39] <erobit> I'm using mongoose-version for versioning my schemas and was wondering if its safe to apply it as a plugin to multiple schemas - it uses refId -> objectid as the key to store versions of a document as it changes
[03:50:23] <Boomtime> http://docs.mongodb.org/manual/reference/object-id/
[03:50:53] <Boomtime> objectid, per host that is generating them, can generate 16 million unique values per second before collision becomes imminent
[03:51:32] <Boomtime> that may extend to individual process (not just per host) depending on the driver
[03:51:48] <erobit> Boomtime: thx - I just wasn't sure if it was implemented per driver.
[03:51:54] <Boomtime> it is
[03:51:58] <Boomtime> but that's the spec
[03:52:07] <Boomtime> anything less is a bug
[03:52:26] <erobit> I think I'll be alright, cause each schema that is being versioned is its own collection as well - so those ids should be unique.. ok I shouldn't really worry about it then
[03:53:03] <erobit> it's not like I'll be inserting more than 20 records per second - not an app that needs to support thousands of users either. awesome thnx again
[12:32:49] <pEYEd> what is a decent intro to MongoDB for mysql immigrants?
[12:33:10] <pEYEd> JSON management is my primary forcus.
[12:34:43] <pEYEd> some form of an express howto for not so dummies? :)
[14:16:54] <nalum> hello all, can anyone tell me why members of a replica set would be stuck in the stateStr STARTUP2?
[16:03:36] <Zerith> hey. I'm using pymongo and a local MongoDB server. I'm iteratively calling insert_one() and find() on many collections, this worked for some entries, but now I can see that my insert_one() calls aren't inserting any data (and I can't see anything in the mongodb server logs either) does anyone know what might be the problem?
[16:04:26] <StephenLynx> GothAlice is quite experience with mongo and python
[16:04:35] <StephenLynx> experienced*
[16:12:25] <Zerith> Nevermind! It was a mistake in my search query.
[16:38:32] <Zerith> What is a simple and lightweight MongoDB document viewer that I can use?
[16:38:40] <GothAlice> "mongo"
[16:39:14] <Zerith> well, and is not CLI
[16:39:18] <GothAlice> db.collectionName.findOne({_id: ObjectId("…")})
[16:39:52] <Zerith> I want to show some people the database and I don't think the CLI will be that convenient
[16:40:28] <GothAlice> I have that as a shortcut, with replacements. (I type *fo<tab>, the name of the collection, tab, and paste in an ID.) I have a number of shortcuts. (TextExpander) I prefer to use ipython or bpython for their syntax-highlighting; they're Python, though, so pymongo.
[16:40:39] <GothAlice> (Which is very similar.)
[17:34:59] <RedWraith> Hey guys
[17:35:54] <RedWraith> Is there a way to search a document and modify every found element in some way that uses the original value as a parameter?
[17:36:17] <cheeser> with the aggregation framework you can do that...
[17:41:39] <StephenLynx> you can reference the document's fields by using '$fieldName'
[17:43:42] <RedWraith> StephenLynx, awesome, I'll just do that inside of an update then. :)
[17:44:46] <StephenLynx> depending on what you need, there are even update operators, like the multiply one.
[17:44:53] <StephenLynx> so you won't need to reference any field
[17:45:00] <RedWraith> StephenLynx, I'm just doing an _.shuffle
[17:45:06] <RedWraith> So I doubt there's an operator for it
[17:46:58] <StephenLynx> there is one thing that bummed me out with $out on aggregate, if the collection exists, it will flush the existing values.
[17:51:27] <RedWraith> StephenLynx, could you give me an example of how you might use $fieldname for an update? I tried it and I think I did it incorrectly - in CoffeeScript: Circles.update {}, {data: _.shuffle $data}
[17:54:08] <StephenLynx> i don't know anything about coffeescript.
[17:54:23] <StephenLynx> people really recommend to avoid it.
[17:55:07] <RedWraith> StephenLynx, sorry, I'll convert it to JavaScript then: Circles.update({}, {data: _.shuffle($data)});
[17:55:28] <StephenLynx> what is _?
[17:55:45] <mike_edmr> underscore library
[17:55:52] <mike_edmr> or lodash
[17:56:01] <mike_edmr> VERY common js lib
[17:56:01] <RedWraith> underscore library, all it does it shuffle elements
[17:56:16] <StephenLynx> and there one thing, you can't just give mongo your functions to run on your application.
[17:56:49] <StephenLynx> mongo won't know what to do with _.shuffle
[17:57:12] <mike_edmr> yeah try var dataForMongo = _.shuffle($data) then pass in dataForMongo
[17:57:27] <StephenLynx> then he would have to update one by one.
[17:57:35] <RedWraith> exactly
[17:57:50] <RedWraith> There is an update function that lets you pass in arbitrary functions
[17:57:51] <StephenLynx> which was what I ended up doing on my application when I found an issue like this.
[17:58:02] <RedWraith> but I don't see one that lets you use the doc as a reference in the process
[17:58:24] <StephenLynx> I couldn't figure out map reduce so I just went with a regular find and iterated the documents.
[17:58:48] <RedWraith> It's frustrating because I've done this before in Lync
[17:58:51] <mike_edmr> i usually prefer small updates to big ones, the big ones seem to cause bigger interruptions for other concurrent work
[17:59:11] <mike_edmr> in fact i have a lot of code that "trickles" in updates to mongo to avoid overloading it
[17:59:23] <RedWraith> mike_edmr, there's a point to that, but isn't it less efficient generally?
[17:59:26] <StephenLynx> what is your platform for the application?
[17:59:39] <RedWraith> Meteor (Node + JS)
[17:59:44] <StephenLynx> not you.
[17:59:55] <mike_edmr> we have a big replicaset and no sharding
[18:00:01] <StephenLynx> no, the application.
[18:00:14] <mike_edmr> me? nodejs
[18:00:27] <StephenLynx> since node and io are non IO-blocking
[18:00:35] <StephenLynx> it doesn't matter how long your queries take.
[18:00:44] <mike_edmr> yes it does in practice
[18:00:45] <StephenLynx> because your application won't be stuck waiting for them to finish.
[18:01:05] <mike_edmr> large updates make the database unresponsive to the massive amount of reads happening
[18:01:12] <StephenLynx> ah
[18:01:16] <mike_edmr> their latency spikes high
[18:01:24] <StephenLynx> I see.
[18:01:24] <mike_edmr> unless we trickle in the updates
[18:01:31] <StephenLynx> it was purely on the database.
[18:01:34] <mike_edmr> yep
[18:01:37] <StephenLynx> I understood something else, nvm
[18:03:46] <mike_edmr> it'd be awesome from my POV if there were some kind of low prio update
[18:03:59] <mike_edmr> like "do this huge update but prefer answering other queries"
[23:54:59] <android6011> I'm using mongojs to extract some data from postgresql and insert to a mongodb. I am attaching a a mongojs.ObjectId() to each object before sending. I keep track on the app side of how many diff ones have been seen, for testing I only get 1, but when I look in mongo I have several 'distinct' values for that field
[23:55:41] <android6011> including several nulls