Log file Viewer
#mongodb logs for Sunday the 29th of May, 2016
[01:00:44] <gugugaga> is it better idea to use gridFS to serve up picture files to users? or is it better to just sotre a reference to each picture in a db and letting nodejs fs take up the pictuer?
[05:40:26] <Lonesoldier728> Hey does it make sense if I am using an $addToSet operator that duplicate objects still get through... an obj like this {field1: "a", field2: null, field3: "b"}
[09:16:08] <Zelest> Derick, yeah, saw.. lacking documentation as well, which made it harder to test out :)
[09:16:24] <Zelest> Derick, thought I'd give it a try later on and see if I can help debug/test it :)
[11:53:15] <loa> hello! is there way to limit memory consumption of mongodb? can i do this with cgroups? i tried memory.limit_in_bytes and looks like mongodb just crashed.
[12:15:00] <kurushiyama> loa: Yes, you can do it with cgroups. Pesonally, I use docker to limit ressource consumption. Is it an good idea to do so? Not at all. There is only one thing which can replace RAM for MongoDB: more RAM.
[12:15:38] <kurushiyama> loa: For a wt instance with journaling, I'd suggest at least 4GB for any halfway serious deployment.
[12:16:55] <loa> kurushiyama, i explain why i think about this, i have mongodb locally, i use it for different scripts which i run and sometimes i have oom condition.
[13:09:18] <oky> kurushiyama: when 90+% RAM hits, OS tends to throw stuff in swap and then it pages in and out of swap, causing system to freeze
[13:09:48] <oky> kurushiyama: you are probably familiar - i was under the impression most servers don't have swap for that reason
[13:11:49] <kurushiyama> oky: Uhm... I am a sysadmin with this or that year of experience... Yes, systems tend to get slower on swap. But with swap, the OOM killer does not kick in as soon as a critical level of physical RAM requested is reached.
[13:12:40] <kurushiyama> oky: I would rather prefer my service running slowly (and my monitoring system notify me of the exhaustion of physical RAM) than not at all.
[13:12:43] <oky> ("with no swap, OOM killer comes in and does its work just fine" = system doesn't freeze, but when swap is enabled, there'll be more opportunity for freezing as progs are pushed in and out of swap)
[13:13:36] <kurushiyama> oky: Yes. But Freeze? Again – they are slower, but _far_ away from freezing.
[13:14:50] <oky> it's running glacially slow - not fully frozen, but for practical purposes, too much delay to do anything - you wouldn't be able to type inside a shell or anything
[13:15:42] <kurushiyama> oky: Well, I have seen systems BRUTALLY swapping, and I still was able to log in.
[13:16:09] <oky> http://askubuntu.com/questions/41778/computer-freezing-on-almost-full-ram-possibly-disk-cache-problem
[13:16:31] <oky> it's usually a combination of kernel settings that causes it - i understand that you may not have seen it (as you configure your machines properly)
[13:19:31] <kurushiyama> oky: Maybe I just use proper dimensions for my machines, but for my life, I have seen MySQL instances swapping like hell, and yes, they were awkwardly slow, even to the point that connections were dropped, but still far away from freezing (as in SIGSTOP).
[13:21:06] <oky> i don't think it would be a SIGSTOP/SIGCONT style freeze - it is a mem starvation freeze, all the programs are still alive (and some of them are chewing up CPU) - at that point, it becomes starvation of disk IO and CPU as the disk keeps swapping in and out and the programs can't execute to save its life
[13:21:59] <oky> (obviously depends on circumstasnces - but that's how i imagine it's playing out most of the time)
[13:22:30] <kurushiyama> o.O For that to happen, machines would have to be _vastly_ underdimensioned...
[13:23:18] <oky> kurushiyama: all i can say is that i watch my mem (it's at 42% now) religiously when i'm dev-ing, if i see it spike up quickly, i try as fast as i can to kill its process or i lose my machine
[15:42:37] <kurushiyama> kenalex: Redundant data might actually be exactly what you want in certain situations.
[15:43:18] <kurushiyama> kenalex: http://stackoverflow.com/questions/32409635/mongodb-query-comments-along-with-user-information/32411030#32411030
[15:48:12] <kenalex> based on the stack overflow article. it is basically saying mongodb is like any technology where the pros can cons have to be considered based on the applied use case
[15:49:05] <kenalex> for my use case it reduced alot of complexity from using an ORM with a relational database
[15:49:28] <kurushiyama> The fundamental difference with MongoDB is that you can – and should – reverse the modelling process you are use to when coming from Java, for example.
[15:52:21] <kurushiyama> Classic way: Define your ERM, hope your ORM manages to answer the questions derived from your user stories, bang your head against the wall for a few hours to get the upper left beyond JOINS right if not, and live (un)happily ever after.
[15:54:58] <kurushiyama> kenalex: The way I suggest with MongoDB: Get your user stories right, derive the questions your data needs to answer from that, create data models optimized for the most common use cases.
[15:55:56] <kenalex> so basically let the user requirements drive the development process instead of the technology
[15:57:51] <kurushiyama> kenalex: And by optimized I mean _highly_ optimized. All the dirty tricks you can find. Be _very_ careful with embedding, though: http://blog.mahlberg.io/blog/2015/11/05/data-modelling-for-mongodb/
[21:02:18] <Nyterax> is there a shorter/faster way to get a random document than this? let docs = []; Col.find().forEach(function(doc) { docs.push(uni); }); return _.sample(docs);
[21:07:51] <kurushiyama> Nyterax: You might want to look into https://university.mongodb.com/courses/M101JS/about
[21:10:53] <kurushiyama> Nyterax: Shipping products without knowing what you do? Data modelling works different (totally!), administration works different, and using MongoDB without using the aggregation framework is a bit like being in Italy without having Italian food.
[21:12:55] <Nyterax> i use Meteor btw so a bit of the complexity is abstracted away in its Mongo interactions I guess
[21:14:50] <kurushiyama> Nyterax: Hm. I disagree. The key point to successful MongoDB projects is data modelling. If you do it wrong, it will be a pita in the best case scenario, and simply make your application impossible to scale in the worst case.
[21:16:28] <kurushiyama> Nyterax: And there is not tool which can do the data modelling for you. Quite the contrary. Some tools push you in directions which are problematic, to say the least. But it is your call.
[21:16:46] <Nyterax> i'm not focusing too much on the theory really, in Meteor I do use schemas for the collections if that's related to what you mean
[21:17:30] <kurushiyama> Nyterax: Yep. And most likely exactly on the path I just described. But, as said, your call. You have been warned ;P
[21:19:06] <Nyterax> i do study sources of opensource projects sometimes and learn a lot in the process
[21:22:14] <kurushiyama> Nyterax: Well. What can I say? You made up your mind. I disagree. One of us has experience in MongoDB. Your call.
[21:24:20] <Nyterax> well once i get the hang of graphql i might find value in spending time on better theoretical understanding in modeling etc., you're not wrong
[21:26:33] <kurushiyama> Nyterax: Well, you might find the answer in M101JS, hence I suggested it. ;) Good luck, anyways ;)