Log file Viewer
#mongodb logs for Thursday the 25th of February, 2016
[00:30:44] <bros> Do I write schemas.account.index({ '_id': 1, 'users._id': 1 }); or schemas.account.index({ 'users._id': 1 });
[07:01:56] <fandi> my version mongo is 3.2.3, when i put replSet on /etc/mongod.conf .. and try start
[10:15:04] <mylord> how to I use update to only update fields I provide, and not replace whole object?
[10:54:44] <Lope> any ideas what happened to the other find params, such as projection? http://mongodb.github.io/node-mongodb-native/2.0/api/Collection.html#find
[11:14:20] <Lope> Boomtime: thanks, sorry I forgot to say I saw that. I was a bit confused because I was previously using the mongojs implementation.
[11:48:11] <jhalliday> https://jira.mongodb.org/browse/HADOOP-143 - anyone know why this is marked fixed even though it isn't?
[12:22:16] <cheeser> it was marked fixed because it was believed to be fixed. if you're finding that's not the case, please file a new ticket.
[14:22:33] <cheeser> only if you state the actual problem. right now all anyone can do is guess. so save everyone some time and explain the problem. if someone knows a solution, perhaps they'll speak up.
[14:27:52] <Fadey> I have an old data (huge) from another machine (centos 6.4) so I took it's disk and attached it to the centos 7 machine
[14:29:03] <Fadey> 2016-02-25T14:08:44.200+0000 [initandlisten] exception in initAndListen std::exception: boost::filesystem::status: Permission denied: "/var/lib/mongo", terminating
[14:39:03] <Fadey> you can find my question also on stackoverflow: http://stackoverflow.com/questions/35630148/mongodb-2-6-11-wont-start-after-changing-dbpath-centos-7
[14:44:35] <Fadey> cheeser that's true :) but that's a poc for like 20 machines that needs to migrate
[14:47:39] <foxnesn> i have a record local:{email:"blah@blah.com",password:"123456"} how on earth do i do a findOne for it?
[15:12:00] <Fadey> did [root@ip-10-0-0-149 lib]# setenforce 0 [root@ip-10-0-0-149 lib]# sestatus SELinux status: enabled SELinuxfs mount: /sys/fs/selinux SELinux root directory: /etc/selinux Loaded policy name: targeted Current mode: permissive Mode from config file: enforcing Policy MLS status: enabled Policy deny_unknown status: allowed Max kernel policy vers
[18:37:27] <Trinity> i've tried a couple of experiments but haven't been able to inject operators into my own code
[18:41:15] <StephenLynx> lets say you want an account that matches the login and token you are receiving
[18:42:09] <StephenLynx> and then if, lets say, token is not a string, but an object with an operator that will always match or just be discarded
[18:42:21] <StephenLynx> then it won't be taken in account and all it will take is the login to match.
[18:43:24] <StephenLynx> so now all the attacker has to know is the login of an account he wishes to be granted authorization.
[18:45:38] <kurushiyama> StephenLynx: Which – correct me if I am wrong here, too – input sanitation?
[18:46:50] <Trinity> so is this the only case scenario? or are there other vectors to be aware of as well>?
[18:49:30] <Trinity> okay, so make sure input is not an object and that's it? of course also testing for acceptable characters would be even better but is there anything else to be aware of?
[18:50:14] <StephenLynx> and yeah, pretty much if the input is not an object, it won't be able to cause anything.
[19:32:23] <diggitydane> I'm signing up users through passport and I want them to only edit their own resources/documents. Is adding a user to the document and then always finding by document._id and user._id the right way to do this? I'm new to mongo, and it feels very RDB, but not sure how else to do it
[19:36:07] <foxnesn> like if i have a users collection which contains email and a products collection how can i relate them?
[19:37:44] <diggitydane> foxnesn, yeah, I can't find any good examples for users only editing their stuff. All the searches I do reference the mongo Users and Roles, which appears different than "end users"
[19:38:58] <diggitydane> I made a UserSchema, and then in the document I do: user: { type: ObjectId, ref: 'UserSchema' }. So it allows me to find anything by that user._id, but again, not sure if that's the right way
[19:42:49] <diggitydane> (If it matters, I'm using node, express, mongoose) https://gist.github.com/anonymous/90077844b450f4e2fed2
[19:46:32] <StephenLynx> the biggest issue with mongoose is that is tries too hard to deny the non-relational nature of mongo
[19:46:49] <StephenLynx> so it ended up being nearly 6x slower than mongo because of all the hacks it has to do
[19:49:45] <diggitydane> sweet, I'll check that out. so is the reference to user appropriate or is that not the mongo way?
[19:50:35] <StephenLynx> all it does is to query both collections and perform a join on your application.
[19:52:05] <diggitydane> so how do you only pull a users documents and not allow them to edit others documents? I'm having a disconnect there and can't find a good resource that explains it
[19:54:28] <diggitydane> the user info is just coming from facebook, and I want them to login and add recipes
[19:55:44] <StephenLynx> all web frameworks are awful by design, but express takes the cake, since it has been abandoned.
[19:57:30] <diggitydane> a user can add many recipes. guess I'm not sure what the question is. I'm using the user ref in recipes to know who added what. So, mongo way would be add all recipes into user document?
[19:57:39] <StephenLynx> when you use web frameworks, the actual runtime environment becomes a black box.
[19:57:59] <StephenLynx> you are on auto pilot and won't be able to dodge a crash when you try to land that plane.
[19:59:18] <StephenLynx> foxnesn, and thats what separates the code monkeys from the valuable developers.
[20:00:56] <diggitydane> which model? and I guess to clear confusion, what are you referring to when you say model? recipe model? It's defined in RecipeSchema in the first snippet
[20:01:47] <StephenLynx> how is your data organized? what are its constraints? what does each part of your data means?
[20:02:43] <StephenLynx> what you showed me doesn't even explain the collection you are asking how to organize.
[20:03:57] <diggitydane> hmm, that's all there is. recipe has name, description, tags (array) and "foreign key" to user. User is facebook info used by passport. that's it. I'm assuming the mongo model definition is close to the mongoose schema definition.
[20:13:39] <diggitydane> I guess that's my question. I'm just trying to build something simple that I can wrap my head around mongo. Is it better embeded in user, or it's own with a relation? From here, I'm going to add ingredients to a recipe. Is that it's own model, or is that embeded in a recipe as an array, etc.
[20:15:32] <diggitydane> yeah, would that just be added into recipes or it's own collection. then I want to search, and give me all recipes that have chicken
[20:16:09] <StephenLynx> if your embedded array elements are going to have more embedded arrays, then you might as well keep them on separate documents.
[20:17:23] <diggitydane> users have recipes, recipes have ingredients. should I have users collection, recipes collection, and ingredients collection? user case scenario would be, what recipes have ingredients X?
[20:18:59] <diggitydane> foxnesn, I'm just playing with the ingredients now. trying it as embeded in recipes, and then as it's own collection
[20:19:56] <foxnesn> i thought the whole point of document oriented database was to not relate things
[20:20:02] <diggitydane> foxnesn, it just feels like everything I'm trying is "relational" in nature, so I'm assuming I'm doing it incorrectly for mongo :) but trying to learn how to do it without
[20:20:32] <StephenLynx> it gets to a point where its worth to have relations because the embedded data is too complex.
[20:25:02] <diggitydane> so, in this case, is it appropriate to have the user ref on recipes, and then when I'm interacting with mongo add in user like .... db.recipes.find({ _id: ObjectId('56cf53a63fa3c061115f01d4'), user: ObjectId('56cc9a28cc22244c05f9b7f3') })
[20:26:00] <diggitydane> otherwise, on angular app that uses a resource like ('/api/recipes/:recipeId'), someone can just type in the id and edit it
[20:28:07] <diggitydane> exactly, so when you query for the id of the recipe, how do you know the owner?
[20:30:45] <StephenLynx> any operation that requires authentication uses that and uses the obtained user to perform authorization
[21:01:17] <bros> I have a really weird bug where MongoDB 3.2 isn't "ANDing" the query fields together for a subdocument.
[21:02:07] <bros> db.items.find({ 'listings.store_id': ObjectId("56560297e4ee9436592ce4a6"), 'listings.item_id': '2583', 'listings.variant_id': null }) returns https://gist.github.com/brandonros/543aec4c0f1bf2349d16
[21:10:00] <kurushiyama> bros: Everybody does. As long as you learn from them. I did Java for half a decade ;)
[22:16:24] <bros> E11000 duplicate key error index: db.items.$listings.variant_id_1_listings.item_id_1_listings.store_id_1 dup key: { : null, : null, : null }
[22:16:27] <bros> Yet db.items.find({ 'listings.variant_id': null, 'listings.item_id': null, 'listings.store_id': null }) yields 0 results.
[22:19:48] <kurushiyama> bros: In general, yes. But keep in mind that for purposes of a unqiue index, non-existing values also are null
[22:21:30] <kurushiyama> bros: Sorry. Yes, you can use a sparse index. But it heavily depends on your use case. Actually some people insert dummy docs with null values to prevent further null values to be enterd
[22:22:26] <kurushiyama> bros: Albeit I am not sure if an explicit null would trigger the unique constraint anyway...
[22:23:29] <kurushiyama> bros: If it is aempty, the corresponding fields of the subdocuments store in the array evaluate to null.
[22:25:24] <kurushiyama> bros: Well, how should I know? If you dont know your use cases – who does? Instead of a unique index, you might want to give document validation a try. Although I am a big fan of keeping logic where it really belongs to: your code.
[22:29:32] <kurushiyama> bros: Unique indices have their uses, of course. Let's take usernames for a webapp, for example. You most likely will check via AJAX while the user types their wanted username. Does a unique index still make sense? Of course, in case somebody tries to bypass you validation logic and simply sends a POST request with a known username and a new password. You are doing an insert on a unique constraint then, causing an e
[22:31:17] <kurushiyama> bros: Of the top of my head – I think so, though I am still not sure about explicit null values.
[22:33:12] <kurushiyama> bros: Well, a partial index might be better here, though admittedly I ahvent worked with them, yet.
[22:33:56] <Hitsnorth> Having some issues setting up authorization in 3.2 its crashing with no message besides Signal 15
[22:37:03] <kurushiyama> Hitsnorth: And you are absolutely positively sure that MongoDB did not complain about something before that?
[22:38:47] <Hitsnorth> If I take that out of the conf it boots up just fine (But of course with no security)
[22:39:33] <kurushiyama> Hitsnorth: Well. I am not really convinced ;) Since you seem to use YAML format: Are you sure you need the quotes? Are you sure you indented correctly?
[22:39:57] <Hitsnorth> Im not sure I need the quotes but thats hwo I saw it (I initially didnt use the quotes).
[22:45:59] <Spynxic> I'm having a problem grabbing the value of an element in a collection, I use the follow statement to call for the _id value: db.collectionname.containers[index]["_id"] but it returns Collection(Database(MongoClient('localhost', 27017)u'graphs'), u'collectionname.containers.index._id')
[22:47:19] <bros> Does this sound right: Profiling is showing that a query I have an index for took 400ms.
[22:48:41] <Hitsnorth> Not sure its ignoring it if i put 2 spaces in front and authorization is not enabled
[22:55:06] <kurushiyama> bros: If the index exists, you should be able to find it via db.yourcoll.getIndices()
[23:01:08] <Hitsnorth> Kurushiyama I have it like that but no keyfile is that required? When I put that part in it crashes again
[23:03:27] <kurushiyama> Hitsnorth: Do you have a standalone instance or a replset/sharded cluster? For the former, you don't need it, otherwise: https://docs.mongodb.org/manual/tutorial/enable-internal-authentication/#enable-internal-authentication
[23:04:34] <kurushiyama> Hitsnorth: And it doesnt crash. It shuts down because of misconfiguration ;) YAML – Yet Another Mistake Loop
[23:04:47] <dreamreal> I'm trying to query an object that looks like { "_id":"blah", "foo":"bar", "emails":[{"address":"foo@bar.com"}]} such that I can find that record given foo@bar.com
[23:08:48] <Doyle> Hey. mmapv1 logged index creation, but it seems WT does not. What's the option to set that?
[23:11:48] <kurushiyama> dreamreal: db.coll.find({"emails.address":"foo@bar.com"}). https://docs.mongodb.org/manual/reference/glossary/#term-dot-notation
[23:14:42] <dreamreal> kurushiyama: I managed with $elemMatch, the dot notation didn't work... although I didn't try it like that. Hmm.
[23:16:15] <Spynxic> Is there a name attributed to the index of an element in an array, for example db.coll.find({"array.key":0}) to find the first element
[23:17:24] <kurushiyama> Hitsnorth: In a lot of environments an exclamation marks has special meanings, so you might want to quote it...
[23:49:54] <Doyle> Index question: When building an index on a sharded collection, should the operation show up on each shard?
[23:52:06] <Doyle> kurushiyama, and if it doesn't? db.currentOp( { msg:/Index/ } ) only shows a result against the first shard
[23:54:40] <kurushiyama> Doyle: maybe already finished on the others? Is the shard key involved? Is it a sharded collection, or is it unsharded and the first shard is the rpimary shard of the DB?
[23:55:26] <Spynxic> Are all the contents of a cursor loaded in memory (ram) when its created? (similar to a dict)
[23:56:30] <kurushiyama> Spynxic: Not too sure about pymongo, but usually not. In general, a cursor is a server side iterator.
[23:57:19] <Doyle> kurushiyama, AH, yes! The index operations went sideways on one and pegged the IO on the index volume at 100%, so I killed them and restarted the index creation