Log file Viewer
#mongodb logs for Sunday the 10th of April, 2016
[12:43:14] <blz> Hello, I'm trying to seed a database with mongoimport, but I'm getting an exception wrt the bson object size being too large. This is surprising, as none of my objects are above teh 16mb limit. https://paste.ubuntu.com/15734094/
[12:52:46] <blz> kurushiyama: I know, right? I'm surely doing something stupid. Here's my import command: mongoimport --host mongodb --collection item --db antidox --file /tmp/example.json
[12:55:12] <kurushiyama> Hm, there was a problem when the max object size was applied to the original JSON docs instead of the resulting BSON docs.
[12:59:59] <blz> kurushiyama: i'm evaluating the presence of my items with `use <dbname>` and `db.<collectionname>.count()` There's no mistake there, right?
[13:01:10] <kurushiyama> blz It depends on wether you have a sharded environment or not. There is a bug which _may_ cause count to return a wrong document count in sharded environments.
[13:04:45] <kurushiyama> blz could you leave out the --file param and just use the path as a positional parameter?
[13:11:35] <blz> And I'm about 99% sure it was exported with default options. My colleague is currently asleep (EST time zone) so I can't confirm :/
[13:12:46] <blz> ah, extract/transfer/load. Again I'm 99% sure that it was done with default settings. I guess that's something I should clarify
[13:30:13] <Siegfried> kurushiyama: I'm wondering why it adds an s to create a collection name when I use mongoose.model ?
[13:30:59] <Siegfried> I have to imagine there are thousands of edge cases where the language is weird
[13:31:01] <kurushiyama> Siegfried Well, it should not matter to you in case you stay within the mongoose tracks, should it?
[13:31:39] <kurushiyama> Siegfried A collection holding users might even be called "pizzaToppings" as long as mongoose gets it sorted out, no?
[13:31:59] <Siegfried> kurushiyama: For sure, I'm just wondering because mongoose automatically creates the collection name based on a string you give it?
[13:32:24] <Siegfried> to me this is bizarre: mongoose.model('user', userSchema) creates a collection called 'users'
[13:32:35] <Siegfried> Why not just have the first function parameter be the name of the collection in the first place?
[13:32:39] <kurushiyama> Siegfried Well, a design decision, and not the best one, imho. Can't you name them manually?
[13:33:31] <kurushiyama> Siegfried But there are more subtle problems with a much larger impact, imho
[13:33:45] <Siegfried> kurushiyama: I wouldn't know - literally started reading about mongoose an hour ago
[13:34:21] <kurushiyama> Siegfried The problem is that mongoose treats MongoDB as sort of an object store and sort of enforces this approach on you.
[13:35:50] <kurushiyama> Siegfried Well, BSON is not JSON (which is a less subtle difference than people think). And MongoDB is a document oriented database.
[13:36:49] <kurushiyama> While it is true that JSON represents objects, treating MongoDB as an object store takes away many of its advantages.
[13:38:21] <kurushiyama> Siegfried When treating MongoDB as an object store, you'd have followings either stored on the followed user or the user following, right?
[13:39:22] <kurushiyama> Siegfried Well, and here is where the problem starts. But let us take a step further and add "likes", "posts" and alike into the mix.
[13:40:21] <kurushiyama> Siegfried When you include all that into a user model (as an object store would dictate), you will run into several problems, not the least being the 16MB size limit of BSON docs.
[13:41:32] <kurushiyama> Siegfried I did not do much with mongoose, just playing around with it, but that was only the _first_ problem I had with it – and the last, admittedly.
[13:42:52] <kurushiyama> Siegfried And, as far as I have seen it so far, mongoose actually encourages overembedding: http://blog.mahlberg.io/blog/2015/11/05/data-modelling-for-mongodb/
[13:44:35] <kurushiyama> Siegfried https://docs.mongodb.org/manual/applications/data-models-tree-structures/
[13:44:42] <Siegfried> Main problem is I am following a coursera course and they only did a brief mongo introduction before focusing on mongoose
[13:45:39] <kurushiyama> Siegfried Plus the basic crud operations. That should do it for starters ;)
[13:47:37] <kurushiyama> Siegfried Hm, how to put it polite. If an introductionary course relies on an ODM considered flawed even by you, that should tell you something about wether it is worth to follow this class ;)
[13:48:26] <Siegfried> kurushiyama: That is a good point, and will definitely do a lot of self reading. I just like to follow courses because they are a way for me to make sure I am making steady progress
[16:19:13] <zsoc> If I can reiterate my question. I'm having this odd issue where i can only login with a user/pass with my application running on the device the user/pass was registered on. The db is shared and accessed remotely. I'm using passport/mongoose/express4/nodejs, as a'local' strategy for passport.
[16:19:59] <zsoc> So If i run the application locally and create user1, and then run the application on the server where the db is located physically... and create user 2 - I can only successfully login with user 1 while running local and user 2 while running remote... even tho the DB clearly shows both users with their hashes/salts.
[16:23:20] <zsoc> Does this have something to do with how hashes/salts work in reference to the env the application is running on? Maybe this is intended behavior...
[16:24:26] <zsoc> I know you can adjust things like salt length, iterations, and key length.. but these are all default. Unless there's some randomization in the way the libs are installed through npm?
[16:30:30] <zsoc> Alright, I guess if the versions of passport/mongoose or their wrappers were different, than the digest algos might be different. That would cause the issue.
[16:48:59] <leolove> Hi. Never tried mongodb myself but have some questions for ops guys. Is it easy to maintain a cluster of mongodb with automatic shard, automatic replication, multi-master, high availability, etc?
[16:58:53] <kurushiyama> StephenLynx You might want to have a look in the channel logs. I am not a JS type, so my explanation why Mongoose is a bad choice to Siegfried might need an improvement.
[16:59:46] <kurushiyama> leolove Well, setting up one sure is not. But I make a living from helping people to utilize and run it properly. Go figure. ;)
[17:00:09] <StephenLynx> second: it turns _id into a string instead of using an actual ObjectId type to handle it
[17:00:39] <kurushiyama> StephenLynx My argument was that it leads one to treat MongoDB as an object store, leading to known problems as overembedding and stuff.
[17:01:38] <kurushiyama> Whereas your third point seems like a total killer from the technical side.
[17:15:39] <Cryptite> I'm connecting to my RS via the java driver; I have two members of my RS right now, both with their hostnames as their public hostname
[17:16:07] <Cryptite> In said Java application, I'm connecting to the replica set via localhost, but it's eventually resolving to the public hostname connection
[17:16:24] <Cryptite> So I'm not 100% but I think it's going through the public ip, rather than using a localhost connection for all its ops
[17:17:57] <Cryptite> I suppos more specifically I get a message that goes like this: "Canonical address the.address.com:27017 does not match server address. Removing localhost:27017 from client view of cluster"
[17:20:19] <kurushiyama> Cryptite Either access them via public adresses or localhost. Do not mix that. And especially do not mix during replset setup.
[17:21:15] <Cryptite> FWIW pinging both localhost and the public ip get the same timings, so it may not matter
[17:21:39] <kurushiyama> With public adresses meaning "resolvable to an ip adress by all nodes of the replset"
[17:22:05] <Cryptite> well that's the concern, right? I don't think I can add the primary as localhost, otherwise other nodes won't see it, because localhost
[17:22:42] <Cryptite> I think this may all be moot, it may be no consequence that it's resolving to the public ip; it may be in name only, and I may not be suffering any performance
[17:22:45] <kurushiyama> Cryptite https://docs.mongodb.org/manual/tutorial/deploy-replica-set/#connectivity
[17:24:20] <Cryptite> may just say the public ip but i'm still getting connections to the db in like 3-6ms so
[17:24:22] <kurushiyama> Cryptite "Removing localhost:27017 from client view of cluster"? I'd call that a problem.
[17:25:09] <Cryptite> client* view, but immediately afterwards, it still picks up the same server in the RS list
[17:30:12] <kurushiyama> Cryptite That is exactly the point. It is on your local machine, but not localhost.
[17:30:46] <Cryptite> right, I think the lesson learned is connect to the proper set ip as defined in the RS
[17:31:08] <Cryptite> it figures it out anyway, but i was just trying to be fancy and don't need to be in this case
[17:31:34] <kurushiyama> Cryptite Aye. And here comes the reason: It is a Very Bad Idea™ to run your application on a mongodb node.
[17:32:44] <Cryptite> I haven't seemingly had issues with it thus far, but that may be due to the fact that the bandwidth to it is pretty low
[17:35:25] <kurushiyama> Cryptite MongoDB (depending on the version) will take up to 85% - 90% of the available RAM. running Your application and MongoDB on the same machine will make them battle for resources when you need performance the most: during high load times.
[17:36:25] <kurushiyama> Cryptite Well, MongoDB keeps a so called working set in RAM. Simplified, the indices and LRU data.
[17:37:41] <kurushiyama> Cryptite Wiredtiger even holds 2 copies of the same data in RAM in case of write operations for a certain amount of time.
[17:41:13] <Cryptite> but that's good to look out for i guess, i could test during more intense data ops
[17:44:00] <kurushiyama> Cryptite With default settings, I assume. So you have a compression on almost every important part, at the expense of CPU which... ...right, the application and mongod would battle for that, too.
[19:01:43] <mylord> what’s this mean? { lastErrorObject: { updatedExisting: false, n: 0 }, value: null, ok: 1 }
[19:14:43] <rickardo1> Is is possible to have the document returned as an assoc array in php? Now I got object
[19:30:52] <zsoc> I don't disagree mongoose may or may not be bad, but I don't see how using it would affect the authentication issue i'm having with passport
[20:24:41] <kurushiyama> Cryptite Sorry, was having dinner. The journal has a fixed size, iirc. 5% of the available disk space of the partition, iirc.
[20:29:54] <edrocks> was gonna switch to wiredtiger but i dont think it's safe to do so yet. plus it's a pain to switch
[20:32:27] <kurushiyama> edrocks And if you have a replset, changing the storage engine is straightforward.
[20:33:45] <kurushiyama> edrocks Bad is not even starting to describe it, especially given the cost of an arbiter.
[20:36:21] <kurushiyama> edrocks But you are right, in that case changing the storage engine will be a pita
[20:37:23] <edrocks> doesn't sound fun and perf isn't an issue right now. I've got 48cores and plenty of ram
[20:37:26] <kurushiyama> edrocks Which should not be much of a problem. But with a 2 member replset, it becomes unavailable, basically.
[20:38:47] <edrocks> kurushiyama: I'm getting a new computer tomorrow so I'll probably through an arbiter on my old dev machine
[20:38:50] <kurushiyama> edrocks Well, the point is that you get better performance for free. You could scale down a bit and add an arbiter ;)
[20:40:30] <edrocks> kurushiyama: did you switch over to wired tiger yet? or just using for new projects?
[20:41:03] <kurushiyama> edrocks I did several migrations to WT, and use it for all new projects and/or deployments.
[20:45:14] <kurushiyama> edrocks What you should consider in case you migrate to WT is wether to have compression or not. Personally, I tend to compress the data files, and leave the indices and journal uncompressed.
[22:55:21] <devdvd> Hi all, I'm setting up my mongo cluster to use ssl for server > server communication. I think I have it setup but I'd like to verify it through mongo that it's using SSL. Is there any way to do that?