PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 16th of April, 2015

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:55:29] <Gevox_> can someone can confirm whether or not this is a good way of detecting document existence? http://pastebin.com/s7P9RFck
[01:00:16] <joannac> Gevox_: why not count instead?
[01:00:20] <joannac> and why not _id?
[01:01:06] <Gevox_> joannac: Isn't _id is randomly generated for every new object?
[01:01:18] <joannac> yes, but it's unique
[01:01:22] <joannac> is your name unique?
[01:01:38] <Gevox_> title is supposed to be, as you cant create two events with the same title.
[01:01:42] <joannac> s/name/title/
[01:02:10] <joannac> as long as you ensure that, then fine
[01:02:26] <joannac> but count() will be faster because you don't have to actually fetch the document
[01:02:37] <joannac> unless you need the record
[01:05:10] <Gevox_> joannac> : Btw, the method i just showed always returns false although the object exists in the DB. Do you see anything wrong related to it?
[01:06:22] <joannac> yes, I don't understand the logic
[01:07:13] <joannac> please explain how your code achieves what you want
[01:07:21] <joannac> try it out in the mongo shell
[01:07:29] <Gevox_> try my method?
[01:07:59] <joannac> well, try the sequence of operations
[01:09:06] <Gevox_> can i do it through the same shell which now has the running db console?
[01:09:38] <Gevox_> I cant execute further commands at this shell http://i.imgdady.com/bjSRsn.jpg
[01:09:50] <Gevox_> is it convient to start a new one? Sorry i'm beginner.
[01:09:52] <joannac> that's not a shell
[01:09:56] <joannac> that's your running mongod
[01:10:00] <joannac> open a mongo shell
[01:10:53] <joannac> http://docs.mongodb.org/manual/tutorial/install-mongodb-on-windows/#connect-to-mongodb
[01:12:16] <Gevox_> http://i.imgdady.com/dCQzm.jpg
[01:12:26] <Gevox_> I want to use another db, and i'm not allowed to write commands
[01:12:36] <joannac> so authenticate?
[01:12:48] <Gevox_> i'm not allowed to run any commands there, how would i ?
[01:13:03] <joannac> I'm confused
[01:13:09] <joannac> is this your mongod or not?
[01:13:31] <Gevox_> test is not the database i use in my java app
[01:13:32] <joannac> what does the other window say?
[01:13:38] <joannac> why can't you connect to it?
[01:13:56] <Gevox_> OK i connected now
[01:13:58] <Gevox_> it took time
[01:18:35] <Gevox_> jonnac: there's no even collections in the DB http://i.imgdady.com/pRqcu.jpg
[01:18:50] <Gevox_> although my code supposed to be created one
[01:19:15] <Gevox_> nvm wrong db
[03:12:51] <dunkel2> hello
[03:13:40] <dunkel2> im new trying to use mongodb, i willl install it but i see i have two options with and without openssl, which one should i use, my webapp domain will be ssl, should i install it then with ssl?
[03:14:02] <dunkel2> or that ssl is for a direct url to connect to the mongodb itself?
[03:50:21] <Kaylors> How would you denormalize a complex mongodb document using mongoose? For example I have a document with multiple arrays of subdocuments that in have references in them that need to be populated during a read of the parent document.
[03:56:14] <arussel> I've tried to restart my mongo instance using '/etc/init.d/mongod restart', but start failed. So now I'm trying 'mongo -f myconfile', but I get:
[03:56:31] <arussel> RROR: child process failed, exited with error number 1
[03:56:37] <arussel> how can I get more info ?
[03:56:49] <joannac> check the logs
[03:57:42] <arussel> there is nothing in the log
[03:58:12] <joannac> I find that very unlikely
[03:58:16] <joannac> what's the dbpath?
[03:58:26] <joannac> mongo --dbpath /path/to/dbpath
[03:58:35] <joannac> oops, mongod --dbpath /path/to/dbpath
[03:58:40] <joannac> and see what that gives youo
[03:59:35] <arussel> it refuses because there is a lock
[03:59:54] <joannac> there you go
[04:01:51] <arussel> no, removing the lock and starting mongo gives the same error
[04:02:06] <joannac> erm, okay
[04:02:19] <joannac> pastebin your command, full output, and ls -la /path/to/dbpath
[04:02:42] <arussel> which command ? '/etc/init.d/mongod start'
[04:02:50] <joannac> no, what I told you to run
[04:02:59] <joannac> mongod --dbpath /path/to/dbpath
[04:03:09] <arussel> how can I set the user ? I don't want it to be run as root
[04:04:19] <arussel> and a normal user won't be able to create the lock file in the data dir
[04:04:32] <joannac> then fix your permissions?
[04:05:39] <arussel> I'm very happy with my permissions, only mongo user can write to data, I just need that mongod uses that user.
[04:09:00] <joannac> su USER mongod ....
[04:09:15] <joannac> wait
[04:09:24] <joannac> su - USER -c "mongod ...."
[04:12:16] <arussel> I'm doing something, no output: 'su - mongod -c "mongod --dbpath /data"'
[04:14:38] <joannac> arussel: no output at all?
[04:15:24] <arussel> yep
[04:15:57] <joannac> sudo su -s /bin/bash USER -c "mongod --dbpath /data"
[04:16:40] <arussel> joannac: ok, this starts a mongo properlly
[04:17:56] <arussel> so the problem is in my conf file
[04:25:49] <arussel> when I try to run it with my config file, it creates a mongod.log file but then exist with an error and without removing the lock
[04:25:51] <arussel> file
[04:37:15] <arussel> when I start with -f and fork=true, I get the error: 'Cannot write pid file to /var/run/mongodb/mongod.pid'. Considering the dir belongs to mongod, I'm assuming that fork is not running as mongod. Is there an option to make run fork as a user ?
[04:37:46] <arussel> I'm using 2.6
[04:41:10] <arussel> my problem seems very close to: http://stackoverflow.com/questions/23086655/mongodb-service-will-not-start-after-initial-setup
[04:44:52] <arussel> I really don't understand, I get the error: ERROR: Cannot write pid file to /var/run/mongodb/mongod.pid: Success, but when I check, the file was created
[04:55:58] <arussel> and this has nothing to do with the directory, as changign the dir to /tmp gives the same error
[05:01:22] <arussel> removing the pidfilepath from my config file fixes the problem
[05:02:22] <arussel> joannac: thanks for your help this was helpful to find the root of the problem
[06:00:38] <Kaylors> How would you denormalize a complex mongodb document using mongoose? For example I have a document with multiple arrays of subdocuments that in have references in them that need to be populated during a read of the parent document.
[06:11:28] <joannac> Kaylors: pick a new schema
[06:11:52] <joannac> I'm not sure what mongoose has to do with this though
[06:13:33] <Kaylors> joannac: I'm using mongoose as my ODM.
[06:17:05] <joannac> Kaylors: that doesn't really answer my question. Why do you think you need to "use Mongoose" to denormalise?
[06:17:33] <joannac> what's your definition of "denormalise"?
[06:23:24] <Kaylors> joannac: As in, that was my assumption that you'd need to do it at the ODM level. If it can be done by base mongodb queries, I'd be okay with that. I guess I might be using denormalize wrongly. What I want is for every reference in every part of the document to be populated.
[06:25:08] <joannac> http://mongoosejs.com/docs/populate.html
[06:36:56] <Kaylors> joannac: It doesn't populate references within arrays of subdocuments.
[07:57:35] <pagios> hi all, can i output from mongo string instead of BSON?
[07:57:37] <pagios> ERROR TypeError: can't convert BSON::Document into String
[08:19:05] <pagios> TypeError: can't convert BSON::Document into String
[08:19:08] <pagios> how cani solve this
[09:49:35] <leporello> Hi. I want to create tree with categories as leafs and subcategories as nodes. How can i do it if mongoose schema can only have regular structure?
[09:50:40] <leporello> So it's about storing different schemas in one collection.
[09:58:50] <grepwood> hi
[09:59:03] <grepwood> I'm trying to set up a mongos router
[09:59:27] <grepwood> I wanted to have it run from the daemon, with all args fed through the config
[09:59:34] <grepwood> so I added "sharding:"
[09:59:54] <grepwood> " configDB = a.a.a.a:port,b.b.b.b:port,c.c.c.c:port"
[10:00:01] <grepwood> to the config file
[10:00:11] <grepwood> apparently this is not recognized by mongod
[10:00:27] <grepwood> which is the opposite of what http://docs.mongodb.org/v2.6/reference/configuration-options/#sharding.configDB says
[10:00:42] <grepwood> can someone please help me?
[10:23:32] <Kota> here's an interesting conundrum for you guys... i've been having issues lately with how long its been taking for my application to retrieve messages from mongo. i've been systematically going through all the possible bottlenecks, and have concluded that str() is a bottleneck.
[10:24:10] <Kota> the query itself takes around 12ms, and preparing that stuff into a dictionary takes probably less time
[10:24:35] <Kota> so i decided to test this in my loop, by simply printing the stringified oids to the console
[10:24:48] <Kota> it took 4.8 seconds to convert 30 object ids into strings
[10:24:53] <Kota> THIRTY>
[10:24:58] <Kota> now why on earth would that be
[10:25:50] <Kota> i haven't even changed this function for a long time, it used to return in around .3 seconds, but one day it started getting randomly longer
[10:26:18] <Kota> it was up to around 10 seconds, to execute, and I cannot explain why turning objectids into strings would be so dynamically time consuming
[10:55:38] <Kota> well that's a heisenbug wtf... I copied my code around, did tests individually, and when i reverted the code its back to taking 60ms
[10:55:52] <Kota> python what are you doing
[11:43:06] <leporello> Hi again. How can i create document inside of static method? I use model = this, then this.create({some: property}) and then nothing. Collection is empty.
[12:02:53] <arussel> is there a way to setup a single node replica set in the config file of that node ?
[12:03:21] <arussel> I'm starting the instance on aws and don't want to have to connect to the server to set the config
[13:59:30] <derbie> hi
[14:04:02] <derbie> So if i am going to have a collection `users` with attributes ( email, password, role ) and another collection `drivers` with attributes ( licenseid , ssn } how should i relate the two?
[14:04:32] <derbie> Should i add an id field in the users schema and create a foreign key rule for drivers in the code?
[14:04:48] <derbie> I am coming from mysql as you can probably tell
[14:19:08] <arussel> completeNoob: it depends :-)
[14:19:21] <arussel> you can put drivers inside users
[14:19:36] <arussel> you can put user._id in driver
[14:19:47] <arussel> or driver._id in users
[14:29:37] <synthotronic> "To upgrade an existing MongoDB deployment to 3.0, you must be running 2.6" - why, what happens if you upgrade to 3.0 from 2.2?
[14:32:37] <cheeser> there are certain upgrades that get applied at each step
[14:32:48] <cheeser> index formats/optimizations, etc.
[14:34:34] <synthotronic> So you will break the database? Bit worrying then that the CentOS packages give you the option to upgrade directly from 2.2 to 3.0.
[14:36:12] <cheeser> centos has no notion of the upgrade needs of any of its packages so you can skip versions on any package you want. centos doesn't know or care.
[14:40:44] <synthotronic> The mongodb.org packages for CentOS I mean, not the ones provided by CentOS.
[14:58:05] <oznt> How can I use the command copydb? I have added a user with the following roles: [ {role: "dbOwner", db: "company"} , {role: "dbAdmin", db: "company"}, { role: "read", db:"company", collection: "system.namespaces" } ], but mongodb is still complaining "not authorized for query on audible.system.indexes.namespaces"
[14:58:11] <oznt> can some one help me here?
[15:10:37] <snowcode> I've an error executing my $geoIntersects query: Can't use $geoIntersects with String
[15:11:21] <snowcode> called method is pretty simple:
[15:11:22] <snowcode> http://pastebin.com/tvg4rPPq
[16:12:23] <benben421> Hi, i'm trying to implement a "version_number" property in a document that would be atomically incremented on every update. I'm thinking of using redis right now for the lock and autoincrement, but i wondered if mongodb didn't have that feature.
[16:13:03] <benben421> The problem is ensuring that both the update and the autoincrement are performed, or none at all
[16:15:07] <benben421> Another remark is that i don't want to keep the history of my document versions. I just need the version number
[16:17:15] <StephenLynx> what is the update in your case?
[16:17:26] <StephenLynx> ah
[16:17:28] <StephenLynx> I see.
[16:17:35] <StephenLynx> you can perform both on the same operation.
[16:17:37] <GothAlice> benben421: Update-if-not-different is the pattern you're looking for, here. db.collection.update({_id: ObjectId(…document to update…), version: 12}, {$inc: {version: 1}, $set: {field: value}})
[16:18:04] <GothAlice> Set the field and bump the version for the record with the given _id, but only if the version is what we expect: 12.
[16:18:26] <StephenLynx> that. the block with stuff to update can have multiple keys. in your case $set and $inc.
[16:19:35] <snowcode> anyone know what's the right format to store GeoJSON data? I get this "Error: MongoError: insertDocument :: caused by :: 16804 location object expected, location array not in correct format" calling newAlarmObj.loc = {type : "Polygon" , coordinates : polygonList}; where polygonList is a simple array in this form: [[10.371094,42.391009]]
[16:19:44] <snowcode> this thing is driving me crazy
[16:23:48] <benben421> @GothAlice : does this update returns true or false if it found something to update ?
[16:24:17] <StephenLynx> afaik, the default is to return information about what was updated.
[16:24:24] <benben421> ok, cool
[16:24:26] <StephenLynx> what platform are you using to develop?
[16:24:29] <GothAlice> benben421: Neither. As always, the documentation is your friend: http://docs.mongodb.org/manual/reference/method/db.collection.update/ (look for WriteResult)
[16:24:47] <benben421> mongodb 2.6
[16:24:51] <StephenLynx> no, the driver.
[16:25:45] <benben421> i'm using mongoose on node.js
[16:25:50] <StephenLynx> ugh
[16:25:50] <benben421> not sure about the driver version
[16:26:06] <StephenLynx> I suggest migrating to io.js.
[16:26:13] <StephenLynx> it is everything node is but better.
[16:26:25] <StephenLynx> second, do you have a reason to use 2.6? it is pretty old.
[16:27:14] <benben421> yeah the project started some time ago. Does it make any difference with regard to my issue ?
[16:27:18] <StephenLynx> and third, if you were not using mongoose (which is awful if you ask me) you could just use the second parameter of the callback to get the operation result.
[16:27:46] <StephenLynx> I don't know. Probably not since I don't think something like this would change since 2.6.
[16:28:11] <StephenLynx> but I still suggest you try and update to 3.0 if the project is not in production.
[16:28:39] <GothAlice> https://jira.mongodb.org/browse/SERVER-17826?filter=17503 < 1,754 improvements and fixes in the 3.x series so far, FYI.
[16:29:07] <StephenLynx> and mongo is not one of those projects that change your basic workflow on major versions.
[16:29:19] <StephenLynx> like python 3 or SDL 2.
[16:29:33] <StephenLynx> so upgrading is pretty safe.
[16:29:50] <GothAlice> StephenLynx: Let me introduce you to my little polyglot friends…
[16:30:01] <StephenLynx> what :v
[16:30:16] <benben421> Ok, thanks very much to all
[16:32:08] <GothAlice> StephenLynx: (https://travis-ci.org/marrow/package — there's a tendency in my code to not be crap, and work without modification on every available runtime, regardless of 2.x or 3.x versioning. Even when the packages do as low-level things as this one. The "waah, Python 3 broke my stuffs" argument is silly these days.)
[16:32:48] <GothAlice> (Polyglot = "speaking many languages" = doesn't have two cares to rub together about which it runs on.)
[16:33:36] <GothAlice> And if you're lazy, there's always: https://warehouse.python.org/project/six/ which does the work for you.
[16:34:17] <StephenLynx> I was not attacking it for once. Still I am pretty sure python DID changed a lot of basic stuff and lots of python 2 code doesn't run on python 3.
[16:35:13] <GothAlice> Can't blame abandoned packages for not being updated to support it. (It really can be as simple as slapping six into a project and altering a few imports to pull from there instead of stdlib.)
[16:36:18] <GothAlice> A classic example is Paste; a suite of web development tools. That thing is a steaming pile, and likely will never be updated for Python 3. Doesn't matter, the good bits have been superseded by discrete packages that _do_ support Python 3, like WebOb. (Request/response classes.)
[16:47:31] <snowcode> anyone used GeoJSON in mongodb/mongoose?
[16:55:56] <snowcode> If anyone can help me: http://stackoverflow.com/questions/29681304/mongoose-mongodb-polygon-data-type-and-validation-error
[17:27:55] <fontanon> Hi everybody! It seems since mongo3.0.x there's no --filter in mongorestore ... so ... how can I skip some documents to be restored now ?
[17:30:06] <fontanon> In fact, I'm looking for some ways to resume a cutted mongorestore operation.
[19:29:35] <grazfather> $1$.*+
[19:29:37] <grazfather> Hey guys, how do I use $unset on a dict subfield? e.g. a document has a{ b:"", c"" } and I just want to unset b. a { "$unset":{"b":""}) doesn't work and "$unset":{"a":{"b":""}} removes all of a
[19:30:49] <GothAlice> grazfather: {$unset: {"a.b": ""}}
[19:31:06] <GothAlice> In the query syntax, nested values use dot-notation, not actual nesting.
[19:32:05] <grazfather> Thanks
[19:32:16] <grazfather> How does it distinguish from a field that has a dot in it?
[19:32:49] <GothAlice> It doesn't. In fact, never do that.
[19:32:52] <GothAlice> :)
[19:34:14] <GothAlice> (It's generally advisable to keep your database, collection, and field names within the realm of [a-zA-Z_][a-zA-Z0-9_]* — i.e. what would be a viable variable name in the widest range of languages.
[19:39:25] <grazfather> Got it. Thanks again!
[19:51:17] <ericmj> Is the 2.6 or 3.0 protocol documented anywhere?
[19:51:29] <ericmj> I can only find documentation for the "legacy" protocol?
[19:52:04] <cheeser> http://docs.mongodb.org/meta-driver/latest/
[19:53:16] <ericmj> cheeser: there are no docs there
[20:01:24] <cheeser> it's the closer i've found
[20:04:40] <ericmj> so it's not documented at all?
[20:04:50] <ericmj> am i supposed to reverse engineer the protocol?
[20:05:34] <GothAlice> ericmj: The high level protocol encapsulation is BSON, AFIK.
[20:05:49] <GothAlice> Once you have a BSON deserializer for Wireshark, you're good to go in terms of figuring out what's going on. ;)
[20:06:13] <ericmj> so reverse engineer it is i guess
[20:06:14] <GothAlice> See also: http://www.mongodb.com/presentations/how-build-mongodb-driver
[20:06:42] <GothAlice> And the many examples of native drivers that already exist. Source code diving FTW.
[20:09:41] <ericmj> GothAlice: that presentation seems to be on the 2.4 protocol
[20:10:08] <ericmj> it's a shame that the protocol isn't even documented
[20:10:31] <GothAlice> Many believe source code to be a form of documentation, and often it's more up-to-date than any prose can be.
[20:11:41] <ericmj> source code is bad documentation, why even have any documentation in that case
[20:11:48] <ericmj> mongodb is open source
[20:13:03] <GothAlice> ericmj: I find the blog to be the best documentation of how MongoDB works internally: http://www.mongodb.com/blog/post/server-selection-next-generation-mongodb-drivers
[20:13:09] <GothAlice> Also http://www.mongodb.com/blog/post/server-discovery-and-monitoring-next-generation-mongodb-drivers and http://www.mongodb.com/blog/post/consistent-crud-api-next-generation-mongodb-drivers
[20:17:56] <GothAlice> ericmj: And, as you say, MongoDB is open source. If in your investigations you get a clearer picture of how things work, feel free to issue a pull request to improve the docs. :)
[20:20:48] <GothAlice> ericmj: See also: https://github.com/mongodb/specifications/tree/master/source/server-selection https://github.com/mongodb/specifications/blob/master/source/auth/auth.rst and the rest of that "specifications" repository.
[20:21:11] <ericmj> GothAlice: nice, that seems to be exactly what i was looking for
[20:21:24] <GothAlice> Linked to on the blog articles. ;)
[20:21:24] <ericmj> i wonder why it's so hidden
[20:21:28] <ericmj> GothAlice: thanks a lot
[20:21:46] <GothAlice> http://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb < the bullet points in the first section of this article linked there. Does seem to be a tiny bit buried.
[22:10:32] <jr3> can I use mongorestore to "replicate" a local mongodb to an aws instance
[22:23:48] <leporello> Good night. Why I can't add array of values to the document with $push?
[22:25:45] <GothAlice> leporello: You can using $each: http://docs.mongodb.org/manual/reference/operator/update/each/#up._S_each
[22:26:54] <leporello> GothAlice, doesn't work too
[22:27:28] <joannac> leporello: pastebin your update
[22:27:31] <leporello> Category.findByIdAndUpdate(result._id, { $push: {leaves: {$each: results}}})
[22:27:34] <leporello> oops
[22:28:03] <joannac> what's the error?
[22:28:14] <leporello> just an empty leaves
[22:32:05] <leporello> joannac, it's madness!
[22:32:19] <leporello> I've added an empty function at the end and it works!
[22:33:18] <leporello> I've spent half of the day on it.
[22:33:39] <GothAlice> leporello: Which DAO/driver are you using?
[22:34:07] <leporello> GothAlice, mongoose, if it's DAO :)
[22:35:42] <GothAlice> Yes, well. Some would consider that to be a Data Inaccess Objects implementation…
[22:38:01] <leporello> it must be findByIdAndUpdate(id, update, someFunc), not just findByIdAndUpdate(id, update), even if I don't want to work with errors or results
[22:38:24] <GothAlice> "Find and update" in general is a bit of an anti-pattern.
[22:38:44] <GothAlice> Notably because in most cases you'll get the document you're updating… prior to the update being applied.
[22:39:54] <leporello> GothAlice, you mean that result inside someFunc?
[22:40:42] <leporello> It could be non-actual?
[22:40:45] <GothAlice> http://docs.mongodb.org/manual/reference/method/db.collection.findAndModify/#comparisons-with-the-update-method
[22:41:01] <GothAlice> I deal with docs, not functions, since issues like this are cross-driver.
[22:41:51] <leporello> GothAlice, what a lucky man.
[22:42:24] <leporello> Sync RDBMs to async NoSQL blows my mind.
[22:42:35] <jr3> mongoose.connect('mongodb://54.3.23.122:27017/db'); I'm getting invalid URI "url" I dont see any issues
[22:42:48] <GothAlice> (No write concern support, must remember to always specify the "new" option for sanity sake, and that third bullet point contains some doozies relating to multiple matches.
[23:20:39] <sudomarize> I'm trying to set up a replica set, but i get this error when trying to initialize the replica set: https://gist.github.com/sudomarize/952ca4b5955d3258bb34
[23:20:46] <sudomarize> any ideas what i'm doing wrong?
[23:26:47] <DeliriumTremens> using pymongo3 what is the suggested way to insert a subdocument within a subdocument? I can $push to an array in a subdocument, but when i try that with a subdocument it just overwrites
[23:27:02] <DeliriumTremens> using db.coll.update_one
[23:27:47] <GothAlice> What's the exact line, DeliriumTremens?
[23:27:56] <GothAlice> (Gist/pastebin if more than one line.)
[23:28:32] <sudomarize> dw
[23:30:56] <DeliriumTremens> GothAlice: http://pastebin.com/BkgPqAqf
[23:31:02] <DeliriumTremens> be gentle >.>
[23:31:28] <GothAlice> It'll actually be a few minutes before I will have a chance to look at that. ¬_¬
[23:31:35] <DeliriumTremens> no worries
[23:41:34] <GothAlice> All right.
[23:42:44] <GothAlice> Oh, you're trying to do some form of bizarre $set, not $push. 'replies' is a mapping/dict, yes?
[23:42:48] <GothAlice> DeliriumTremens: ^
[23:43:38] <GothAlice> Note that there are severe disadvantages in that approach. Using a real list is better. To do this, instead of having {reply.id: {…}}, have {id: reply.id, …}
[23:44:04] <GothAlice> The latter approach can benefit from indexes, and is much easier to query in general. (Far fewer machine-generated complex $or trees.)
[23:45:18] <DeliriumTremens> yeah, replies are dicts
[23:46:40] <DeliriumTremens> well, replies are generators
[23:54:10] <DeliriumTremens> fwiw, i'm doing the list currently, but you've answered my question! thanks GothAlice