pmxbot IRC Log Viewer

[01:54:32] <symbol> I'm using the node mongodb native driver...can't seem to figure out how to create a mongodb connection in a DRY way. Can anyone recommend an open source app to look at?

[01:55:16] <symbol> With Postgres I just passed a query to the client so one block of code handled all CRUD.

[01:58:45] <StephenLynx> what runtime environment?

[01:59:01] <symbol> Node

[01:59:01] <StephenLynx> symbol

[01:59:03] <StephenLynx> ah

[01:59:07] <StephenLynx> let me link my project then

[01:59:15] <symbol> I can just export each query function?

[01:59:21] <symbol> Found this -

[01:59:22] <symbol> http://coenraets.org/blog/2012/10/creating-a-rest-api-using-node-js-express-and-mongodb/

[01:59:54] <StephenLynx> express is a pile of crap

[02:00:17] <StephenLynx> https://gitlab.com/mrseth/LynxChan/blob/master/src/be/db.js

[02:00:37] <StephenLynx> https://gitlab.com/mrseth/LynxChan/blob/master/src/be/db.js#L569 its where I open the connection.

[02:00:48] <symbol> I used it for a hack spike on this project...definitely not a fan of it but it'll do. In the meantime, I can at least move us off postgres :D

[02:00:51] <StephenLynx> then I assert indexes and get pointers to the collections

[02:01:07] <StephenLynx> and use these pointers.

[02:01:21] <symbol> Nice, thanks for the links

[02:01:24] <StephenLynx> np

[02:15:56] <doc_tuna> symbol: what did you achieve by moving off postgres?

[02:16:36] <StephenLynx> better clustering and cache?

[02:16:44] <StephenLynx> not needing a schema

[02:17:58] <doc_tuna> its not better -- its different

[02:19:32] <StephenLynx> depends on the use case.

[02:19:41] <doc_tuna> right, just curious about that

[02:20:07] <doc_tuna> i like hearing what people/learned accomplished

[02:20:26] <doc_tuna> err why is that / there

[02:20:34] <doc_tuna> and not one word to the right

[02:21:54] <StephenLynx> for one I know that installing postgre is a pain in the balls

[02:22:11] <StephenLynx> I don`t know how good are the clustering tools for it either

[02:22:17] <StephenLynx> or if it allows to store files.

[02:22:25] <doc_tuna> personally i get irate over pgadmin

[02:22:45] <doc_tuna> it crashes like mad and there is a lack of good alternatives

[02:22:52] <StephenLynx> hue

[02:23:05] <StephenLynx> yeah, running relationaldatabases without a GUI tool is hard

[02:23:06] <doc_tuna> no newschool interfaces for postgres

[02:23:18] <StephenLynx> managing mongo in the terminal is a piece of cake

[02:24:04] <doc_tuna> mongo with robomongo is okay

[02:24:17] <doc_tuna> but that has crashing problems too

[02:24:41] <doc_tuna> even cli tools can have better or worse ux

[02:24:52] <doc_tuna> and postgres cli is awkward

[02:26:58] <doc_tuna> probably all that backwards compatibilty with old dbadmin nonsense

[02:27:24] <doc_tuna> as in, adversion to teaching old dogs new tricks

[02:27:32] <doc_tuna> even thoug its a better way

[02:27:36] <doc_tuna> *though

[04:05:22] <tp43_> can I manually upload a collection from one db to another?

[04:15:20] <tp43_> mongodump/mongorestor

[17:07:19] <ronoc> hi there

[17:07:52] <ronoc> yesterday i was trying to get a simple mongo cluster up and going on aws in its own vpc with a NAT in front of it

[17:08:17] <ronoc> I used the basic template provided by aws through the cloud formation tools on aws

[17:08:30] <ronoc> of course this template only half sets things up AFAICT

[17:08:54] <ronoc> I had to manually ssh into each node (its a 3 node replica set)

[17:09:48] <ronoc> and create the primary, secondary and replica node

[17:10:16] <ronoc> all three seem to be working well

[17:10:28] <ronoc> but I can't for the life of me figure out how to setup the nat

[17:10:48] <ronoc> i'm presuming it is not setup properly

[17:10:58] <ronoc> Nothing in the docs about how i actually connect to mongo

[17:11:19] <ronoc> i can ssh into the NAT and from there ssh to the nodes that are on the private subnet

[17:11:37] <ronoc> but I'm clueless at this point as to what I'm supposed to do

[17:11:56] <ronoc> seriously considering just reverting to a big fat ec2 instance that i can ssh into and control

[17:12:59] <ronoc> open up ports and lock down control to one lp

[17:15:39] <ronoc> this is what I followed

[17:15:41] <ronoc> btw

[17:15:48] <ronoc> https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=AWS-MongoDB-Infrastructure&templateURL=https:%2F%2Fs3.amazonaws.com%2Fquickstart-reference%2Fmongodb%2Flatest%2Ftemplates%2FMongoDB-VPC.template

[17:23:29] <ronoc> @joannac any ideas ^

[22:19:47] <janitto> Hello. Is mongo still a good fit for a "conventional" CRUD web app?

[22:20:48] <cheeser> sure.

[22:26:48] <janitto> is there anything to be aware of when using it this way? i've read a bunch of stuff about data loss and unreliability but i'm beginning to think it's been exaggerated

[22:27:06] <janitto> also heard some advice not to use it as a primary data store

[22:31:25] <StephenLynx> all bullshit.

[22:31:36] <StephenLynx> however I do have some word of advice:

[22:31:47] <StephenLynx> big skips are slow

[22:32:03] <StephenLynx> if you need to relate too much data, mongo might not be a good choise

[22:32:06] <StephenLynx> choice*

[22:32:42] <StephenLynx> about primary data store: that would be redis, from what I heard. from what I read, once your dataset exceeds your RAM, you are boned.

[22:33:07] <StephenLynx> mongo does have its own memcache, but as long as your indexes fit RAM, you are fine.

[22:37:12] <janitto> StephenLynx: thanks for the advice. i definitely have some relations but nothing too crazy i hope

[22:37:31] <StephenLynx> yeah, some relations here and there are not bad if you don`t abuse them.

[22:37:41] <StephenLynx> I do have some too on my systems.

[22:38:15] <StephenLynx> https://gitlab.com/mrseth/LynxChan/blob/master/doc/Model.txt

[22:39:16] <janitto> are those all separate collections?

[22:39:25] <janitto> or are you embedding a lot of them?

[22:39:49] <StephenLynx> separate.

[22:39:52] <StephenLynx> the relations are pointed.

[22:40:08] <StephenLynx> embedded data is pointed too.

[22:40:30] <StephenLynx> and you will notice indentation.

[22:40:36] <janitto> why not store posts in users for example? isn't that what mongo encourages?

[22:40:41] <StephenLynx> hue

[22:40:47] <StephenLynx> ok

[22:40:53] <StephenLynx> first: posts are anonymous

[22:41:11] <StephenLynx> second: posts are related to threads and boards

[22:41:45] <StephenLynx> third: there can be A LOT of posts in a board and even in a thread, and documents have a hard limit of 16mb

[22:41:59] <StephenLynx> fourth: handling complex documents can be hard.

[22:42:14] <StephenLynx> and limits what kind of queries and projection you can perform.

[22:42:32] <StephenLynx> thats why I rather split posts in their own collection.

[22:42:57] <StephenLynx> I do embed stuff, however.

[22:43:07] <StephenLynx> like files are embedded in posts and threads

[22:43:37] <StephenLynx> arrays of settings are embedded too

[22:43:56] <StephenLynx> board volunteers are embedded in the board

[22:44:40] <janitto> boy, there sure are a lot of decisions about where to put things

[22:44:52] <StephenLynx> yes.

[22:45:07] <StephenLynx> nosql is not as straightforward as relational databases.

[22:45:44] <StephenLynx> with great powers, come great responsibility.

[22:45:46] <StephenLynx> :v

[22:48:42] <janitto> i have publishers who have sites and zones within those sites, and advertisers who have campaigns and flights within campaigns, and creatives that can be applied to flights. those flights can be applied to specific zones. is that doing too much with relations for mongo?

[22:49:28] <StephenLynx> hm

[22:49:38] <StephenLynx> don`t think so.

[22:49:52] <StephenLynx> depends on how you intend to query, though.

[22:50:25] <StephenLynx> and how much the reading will cascade until you get the final information you wish fo.

[22:50:28] <StephenLynx> for*

[22:50:29] <janitto> yeah, i see what you mean. how it should be structured totally depends on what my app does

[22:51:01] <StephenLynx> and even then, you can always pre-aggregate data.

[22:51:07] <StephenLynx> for example:

[22:51:15] <StephenLynx> see how my boards have owners

[22:51:22] <StephenLynx> but users also have an array of owned boards.

[22:51:57] <StephenLynx> that way I don`t have to look for all boards and see which ones match the user as owner when I want to list the user's owned boards.

[22:52:04] <StephenLynx> or how threads have an array of latest posts

[22:52:22] <StephenLynx> so I don't have to look for all posts, see which ones match the thread, sort by date and limit.

[22:52:31] <StephenLynx> I already have that array of the latest X ones.

[22:53:49] <StephenLynx> or just take a look at the board stats collectiom.

[22:53:52] <StephenLynx> collection*

[22:54:18] <StephenLynx> instead of reading the data real-time for stats, I just store the data every hour

Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 5th of September, 2015