Log file Viewer

[15:03:50] <jeffreylevesque> Anyone here?

[15:04:16] <Derick> plenty of people here, about 180

[15:04:50] <jeffreylevesque> Wondering if I can use the 'finalize' without the reducer? For example, if the mapper uses a constant key, or if each record has a unique key

[15:06:17] <Derick> you'll probably have to write a really simple one

[15:06:59] <jeffreylevesque> The documentation says the finalize uses arguments from the reducer - https://docs.mongodb.com/manual/reference/command/mapReduce/#mapreduce-finalize-cmd

[15:07:41] <jeffreylevesque> But, in my case, all the keys are unique - https://github.com/jeff1evesque/ist-664/blob/master/data/reddit-2005-12

[15:08:14] <jeffreylevesque> I want to use the map + finalize, to group instances having two similar fields

[15:09:24] <jeffreylevesque> My current map reduce with a constant key doesn't return all results - https://github.com/jeff1evesque/ist-664/blob/bfefcafbebf3d4522cb6809dd1523204db738cd0/app/select.py

[15:09:36] <jeffreylevesque> It returns only the last 2 results

[15:11:05] <jeffreylevesque> So, I was thinking of replacing the constant key on line 26 with 'this.id'. But, since all the records are unique, I was going to use the finalize to check if 'id' and 'parent_id' are equal between any documents in a single collection

[15:12:09] <jeffreylevesque> Is that a possible approach?

[15:13:19] <jeffreylevesque> If I change line 26 to 'this.id', I would likely move the code in the reducer, with some adjustments into the 'finalize'

[15:16:04] <Squarism> What benefits does mongo offer other than efficient storage of json and search of it?

[15:17:01] <jeffreylevesque> Not sure, I hear Postgres is faster.

[15:17:11] <jeffreylevesque> But, I've already started with mongo

[15:17:22] <jeffreylevesque> And map reduce can be used on other systems

[15:18:13] <jeffreylevesque> Except finalize is native to mongodb

[15:20:25] <Derick> jeffreylevesque: I am not sure what you're map/reducing, but have you looked at the Aggregation Framework? Usually much more efficient

[15:20:35] <Derick> Squarism: easy scalability

[15:21:36] <jeffreylevesque> I want to use map reduce, since my proof of concept will discuss sharding mongodb

[15:23:02] <jeffreylevesque> I heard MR is better than aggregation for sharded case

[15:29:44] <jeffreylevesque> Derick: if it's required to create a simple reducer to trigger the 'finalize', is it possible to create a trivial reducer if all the keys are unique each having just one instance?

[15:30:26] <jeffreylevesque> Can I trigger a reducer somehow, just to get the 'finalize' to run given my dataset?