pmxbot IRC Log Viewer

[00:09:12] <janitto> StephenLynx: what about billing related stuff? could i get in trouble if mongo doesn't have transactions? is it a good idea to be playing around with money related stuff?

[00:09:37] <StephenLynx> hmmm

[00:10:48] <StephenLynx> that is a good question, but from I heard, mongo is not that inconsistent to cause issues regarding to that.

[00:17:33] <janitto> StephenLynx: i could just boost up my write concern and it's not a problem anymore right?

[00:17:55] <StephenLynx> I have never handled big deploys, so I don't know.

[00:18:12] <janitto> fair enough

[00:18:29] <StephenLynx> lynxhub.com is the biggest system I ever had running with mongo.

[00:18:51] <janitto> i would probably be using mongolabs or another hosted service, i'm hoping that a lot of that stuff will be avoided by having people who know what they're doing involved

[00:19:36] <StephenLynx> meh

[00:19:53] <StephenLynx> I am not paying more instead of just RTFM' ing

[00:20:12] <StephenLynx> plus if that goes south I don't care.

[00:20:19] <StephenLynx> I can just wipe everything.

[07:33:58] <Rou> Does mongodb work for lubuntu 15.04?

[08:55:22] <frenchiie> can someone explain to me what parameters i should put for the pymongo function find_one_and_update() if i want to update a document with it's name stored in var name and the key of the value being var key and the value being "0"?

[08:55:31] <frenchiie> http://api.mongodb.org/python/current/api/pymongo/collection.html#pymongo.collection.Collection.find_one_and_update

[09:01:10] <frenchiie> going to log for now but will check the irclogger later, i appreciate any answer.

[12:18:04] <hkariti> hi all. i have a quick question about config servers - does it make sense to put them in different dcs? I was thinking of putting one with the backup replicas of all the shards, for DR purposes.

[15:40:22] <repxl> Why does this not work http://pastebin.com/avZQFwej i want to make sure the user_id is allways matched + or search tags or string

[15:42:30] <repxl> or how i can search for tags including normal name in one query _?

[15:43:17] <repxl> so when i write #fish tunaspecial it find me the tag #fish and the name "tunaspacial" also the user must match

[15:43:44] <repxl> and when you write only #fish it leaves the normal regex name search

[19:56:35] <frenchiie> so...

[19:56:42] <frenchiie> do people actually answer questions here

[19:56:55] <StephenLynx> yes.

[19:57:30] <frenchiie> i looked at the irclogger to see if anyone answered mine and nope. Also the next two people who asked questions didn't get an answer...

[19:57:43] <StephenLynx> happens.

[19:57:52] <StephenLynx> have you ever tried asking stuff over linux or systemD channels?

[19:58:11] <frenchiie> nope

[19:58:28] <StephenLynx> its even worse, I can tell you.

[19:58:32] <StephenLynx> what did you asked before?

[19:58:50] <frenchiie> i usually go on the python, django, javascript, and webdev ones

[19:58:55] <StephenLynx> heh

[19:58:58] <frenchiie> usually get an answer right away

[19:59:14] <StephenLynx> because those are a completely different kind of community.

[19:59:25] <StephenLynx> with people dying to please others.

[19:59:33] <frenchiie> lol

[19:59:38] <frenchiie> anyways i asked this:

[19:59:44] <frenchiie> can someone explain to me what parameters i should put for the pymongo function find_one_and_update() if i want to update a document with it's name stored in var name and the key of the value being var key and the value being "0"?

[19:59:45] <StephenLynx> on the other side of the spectrum you get stuff like "yeah, I dunno, educate yourself"

[19:59:50] <StephenLynx> like on the systemD channel

[19:59:51] <frenchiie> http://api.mongodb.org/python/current/api/pymongo/collection.html#pymongo.collection.Collection.find_one_and_update

[20:00:14] <StephenLynx> GothAlice is very experienced with python, but she isnt here.

[20:00:21] <StephenLynx> I know jack of python :v

[20:00:22] <frenchiie> ah

[20:00:30] <frenchiie> that's probably why then

[20:00:41] <frenchiie> is the API that different from language to language?

[20:00:42] <StephenLynx> this is probably why no one answered you: its a more specific question. is not exactly inherent to mongo.

[20:00:49] <StephenLynx> yes, each driver is different.

[20:01:01] <frenchiie> hm i see

[20:01:22] <frenchiie> maybe i'll try the /python

[20:01:26] <frenchiie> see if someone uses mongo

[20:01:35] <StephenLynx> yeah, if someone there read the doc they should be able to assist you.

[20:02:50] <mprelude> Hi, I'm trying to understand the localhost exception. I thought it was that I can connect until I've defined a user. I went ahead and added a user to the 'admin' database (as per the docs) and the login works, but when I go to my 'test' database, I can still connect, view and add collections & documents without authenticating. How can I prevent this?

[20:03:10] <StephenLynx> have you enabled authentication?

[20:03:30] <StephenLynx> afaik its disabled by default.

[20:03:44] <StephenLynx> you have to both have users and have it enabled so mongo will apply permissions.

[20:03:50] <StephenLynx> on the localhost.

[20:03:51] <mprelude> StephenLynx: I'm not sure, but if I specify the username and type the wrong password it fails, so I *assume* so.

[20:04:09] <StephenLynx> hm, I don't think thats enough.

[20:04:28] <StephenLynx> I forgot where you have to set it, but look for how you turn auth on/off

[20:04:31] <mprelude> http://paste2.org/6OPen7EZ

[20:49:24] <JFlash> hi, how should I model pages and user roles in a social app?

[20:49:37] <JFlash> should I add permissions to the user object document

[20:49:57] <JFlash> or should I keep a list of staff roles in the page document ... or both?

[20:50:00] <JFlash> help!

[20:51:22] <StephenLynx> I wouldn't store a list of possible staff roles.

[20:51:41] <StephenLynx> how I do for global staffs is to have a field with the role of the user in the global staff

[20:51:56] <StephenLynx> for local staffs I usually have a list of users in the object representing the local.

[20:52:18] <JFlash> gee.. so to get a list of staff for my page I have to query the complete database one billion gazillion social network users??

[20:52:56] <StephenLynx> yeah, I can see how that would be an issue on a system that mandates user accounts.

[20:53:01] <JFlash> sorry waht is the global staff?

[20:53:12] <StephenLynx> for example

[20:53:13] <JFlash> there's staff for each page

[20:53:26] <StephenLynx> a global staff would be the staff that moderates the whole site

[20:53:29] <StephenLynx> and not just a page.

[20:53:40] <JFlash> theres no such thing, it s a social network

[20:53:46] <JFlash> anyone can create pages

[20:53:53] <JFlash> so there can be millions of pages

[20:53:57] <StephenLynx> what if a page is about killing people?

[20:54:12] <StephenLynx> or child porn?

[20:54:19] <StephenLynx> who moderates that?

[20:54:22] <JFlash> thats an implementation detail

[20:54:36] <JFlash> (could be subject to review etc)

[20:54:38] <StephenLynx> of course thats an implementation detail.

[20:54:47] <StephenLynx> everything is.

[20:54:58] <JFlash> thats for market ppl and layers

[20:55:02] <JFlash> lawyers

[20:55:14] <StephenLynx> so you are just going to wait for some kind of law suit?

[20:55:19] <JFlash> right now I just need to find out how to model this and run to the back to get my money

[20:55:24] <StephenLynx> instead of having people with powers to keep track of that?

[20:55:28] <JFlash> i'm not liable

[20:55:43] <StephenLynx> anyway, I already told you what I would do on local staffs.

[20:55:55] <StephenLynx> I would store a list with the page staff on the document representing the page.

[20:56:03] <JFlash> ah ok

[20:56:07] <JFlash> that's what I thought, thanks

[20:56:10] <StephenLynx> np

[20:56:40] <JFlash> but I suppose I have to have some reference to the page in the users document?

[20:56:48] <JFlash> suppose my boss comes to me and says

[20:57:18] <StephenLynx> on my latest project I just keep track of the boards the user owns.

[20:57:21] <JFlash> ok, now if johny is staff on page xxx , there needs to be a badge next to his profile photo

[20:57:40] <StephenLynx> no problem

[20:57:53] <StephenLynx> because picking the page`s staff when he posts is not hard.

[20:58:00] <JFlash> so I dont want to query for 1 million pages to find out on the fly if johny boy is staff in any of them

[20:58:23] <StephenLynx> you will have already picked the page`s document when creating a post to it.

[20:58:51] <JFlash> so how do I know about the page?

[20:59:06] <JFlash> how do I know he is in anyways related to the page.. that's my modeling question

[20:59:18] <StephenLynx> when someone posts to a page

[20:59:22] <JFlash> do I keep a general list of pages he is connected to

[20:59:25] <StephenLynx> you have to check if the page exists, right?

[20:59:32] <StephenLynx> if the page is locked?

[20:59:34] <StephenLynx> or anything?

[20:59:35] <JFlash> or do I keep a separate reference for pages where he is staff

[20:59:59] <JFlash> no this can be someone visiting his profile, not related to posting on the page at all

[21:00:11] <JFlash> a page on this profile photo on this timeline

[21:00:18] <StephenLynx> hm

[21:00:33] <StephenLynx> I am having a feeling your system might fit a RDB better.

[21:00:39] <StephenLynx> too many relations from everywhere.

[21:00:47] <JFlash> noooooooooo... (dies)

[21:00:58] <StephenLynx> social media in general fits RDBs better, IMO.

[21:01:04] <JFlash> wtf

[21:01:15] <JFlash> I thought mongo was the best db in the world... now this?

[21:01:26] <StephenLynx> there are no silver bullets.

[21:01:36] <StephenLynx> each kind of database fits a scenario better.

[21:01:50] <StephenLynx> RDBs, noSQL, graph dbs

[21:02:15] <JFlash> check this out

[21:02:16] <JFlash> https://github.com/mongodb-labs/socialite

[21:02:40] <JFlash> this is by mongo labs and they tested it agains over 10 million uses and it ran as a breeze

[21:02:45] <StephenLynx> ok

[21:03:12] <StephenLynx> I dont know what mongo labs is though.

[21:03:51] <JFlash> ok maybe I messed up the name but it's some developer team inside Mongo foundation, company

[21:04:42] <StephenLynx> hm

[21:05:11] <JFlash> they wanted to test it against a test load of 100 million users but then they get capped by money restrictions

[21:05:37] <StephenLynx> again, I will give you my opinion.

[21:05:48] <JFlash> but they say there's nothing on the implementation that hints of the system breaking apart beyond that number. the scaling has been proving to be horizontal

[21:06:03] <StephenLynx> tools do can be forced to operate on scenarios where they usually don't fit well, but there will always be compromises and limitations.

[21:06:04] <JFlash> proven*

[21:06:12] <StephenLynx> keep that in mind.

[21:06:25] <StephenLynx> the fact is:

[21:06:40] <StephenLynx> social media REQUIRES lots of relations and mongo DOESN`T even implement relations.

[21:06:50] <JFlash> dammit

[21:07:04] <StephenLynx> even dbrefs operate on a driver level.

[21:07:13] <StephenLynx> and just masquerade multiple queries.

[21:07:22] <JFlash> I need a second opinion on this!

[21:07:27] <StephenLynx> yes.

[21:07:49] <StephenLynx> I have just over a year of experience with mongo, don't take my words as final.

[21:08:11] <StephenLynx> but don't believe any database is the best for every single case.

[21:08:32] <StephenLynx> you will pick the wrong one eventually and get rekt.

[21:09:11] <JFlash> here's what I can tell you

[21:10:10] <JFlash> 95% of the db load in a social network type of app is nothing more than content consuption.. aka... ppl scrolling timelines as crazy

[21:10:40] <JFlash> this translates a shitload more reads than writes

[21:11:17] <JFlash> and it turns out..(correct me if I'm wrong) that such data can be aggregated into documents..

[21:11:31] <JFlash> ... for blazing fast consume

[21:11:40] <JFlash> do you agree?

[21:24:43] <StephenLynx> yes.

[21:24:56] <StephenLynx> that part you got right.

[21:25:56] <StephenLynx> you could use a relational database as a main storage and then store the pre-aggregated most consumed data on mongodb

[21:27:21] <JFlash> dammit

[21:27:34] <JFlash> i dont have the resources for that. I need to use only one

[21:27:50] <JFlash> http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb

[21:27:53] <StephenLynx> welp, then you do could use mongodb for everything

[21:27:56] <StephenLynx> yeah, thats bullshit

[21:28:14] <StephenLynx> but it wouldn't be kosher with that many relations

[21:28:30] <JFlash> this guy talks about how ppl who chose mongo for their social network project ended up loosing all the work

[21:28:36] <StephenLynx> yeah.

[21:28:46] <StephenLynx> the diaspora guys I guess.

[21:29:03] <StephenLynx> again.

[21:29:48] <StephenLynx> using mongo is possible, but also hard to now screw up.

[21:29:52] <StephenLynx> in this scenario.

[21:30:00] <StephenLynx> to not screw*

[21:30:55] <StephenLynx> heh, yeah, diaspora

[21:31:05] <StephenLynx> they did an epic faceplant

[21:32:01] <JFlash> havent had the article, just skimmed

[21:32:09] <StephenLynx> tl,dr;

[21:32:27] <JFlash> but looks like it could be a an implementation problem first and foremost

[21:32:30] <StephenLynx> guy didnt had experience with mongo, bought the hype, used it wrong, got rekt

[21:32:46] <JFlash> also, mongo has been adding new features ever since, which where not available at the time

[21:33:41] <StephenLynx> <JFlash> I thought mongo was the best db in the world... now this?

[21:33:50] <StephenLynx> they thought exact the same thing.

[21:33:54] <mprelude> Probably not the place to ask, but has anyone used mongo with common lisp?

[21:34:51] <mprelude> http://paste2.org/81bMJwnB

[21:34:58] <mprelude> Perplexing behaviors

[21:35:32] <StephenLynx> thats lisp for you

[21:35:56] <mprelude> I think it's an issue with the lib, can't think of anything else.

[21:36:06] <JFlash> mprelude, are you a mongo experiencied user?

[21:36:07] <StephenLynx> yeah, its in the driver.

[21:36:11] <mprelude> JFlash: No.

[21:36:17] <JFlash> ok

[21:36:26] <mprelude> JFlash: I've used for a month or so with ruby.

[21:36:27] <StephenLynx> funny for it to give you that error in specificy.

[21:36:33] <JFlash> guys gotta eat. tty later

[21:36:34] <StephenLynx> specific*

[21:37:33] <mprelude> Yeah, suggests the auth isn't being passed to the mongod right

[21:38:04] <StephenLynx> aye

[21:38:16] <StephenLynx> but I think you don't even have authentication enabled.

[21:38:23] <mprelude> I do. Sec

[21:39:10] <mprelude> http://paste2.org/8AYcjhty

[21:40:04] <StephenLynx> ah

[21:40:26] <StephenLynx> so thats whats wrong, you didn't authenticated the collection you open with your lisp code.

[21:44:38] <mprelude> StephenLynx: I appear to be on the db I'm trying to auth for

[21:47:30] <StephenLynx> connect != auth

[21:48:04] <mprelude> * (cl-mongo:db.auth "aurumreadwrite" "testing")

[21:52:34] <mprelude> StephenLynx: I think I have to create a CLOS object.

[21:52:47] <StephenLynx> :v

[21:52:50] <mprelude> Sorry, the lib has no docs, I'm reading the source code. Heh.

[21:52:50] <StephenLynx> no idea whats that

[21:52:54] <StephenLynx> hueeeeee

[21:53:17] <mprelude> CLOS = Common Lisp Object System

[22:01:31] <mprelude> Nah, still didn't work, I'm stumped, heh.

[22:02:35] <StephenLynx> tried using something maintained?

[22:02:47] <mprelude> There is nothing maintained :D

[22:03:00] <StephenLynx> try outside lisp

[22:03:16] <mprelude> Might be problematic. Probably easier for me to fork the lib and fix it.

[22:03:50] <StephenLynx> you know NOTHING but lisp?

[22:04:06] <mprelude> I know lots of langs, but I have 40 kloc in lisp on this project. :)

[22:04:12] <StephenLynx> :v

[22:04:14] <StephenLynx> welp

[22:04:31] <mprelude> And I'd imagine making auth work isn't so hard, rest of the lib appears to function.

[22:04:40] <StephenLynx> the problem is not having any docs.

[22:04:53] <StephenLynx> as long as you have TCP tools it is possible.

[22:05:06] <mprelude> Well, I read the source, I think the issue is that the lib isn't sending the auth right

[22:05:46] <mprelude> https://github.com/fons/cl-mongo/blob/bb1f807a17c960dc81bebd5e8a2df5d0886d422a/src/db.lisp db.auth is the last func

[22:11:18] <mprelude> StephenLynx: At a glance, this function is using MD5, and the API tells me to use SHA-1

[22:11:24] <mprelude> So I think it's just an outdated lib.

[22:11:42] <StephenLynx> you could write a secondary software to handle the database

[22:11:46] <StephenLynx> and pipe the perl one into it.

[22:16:12] <mprelude> It looks like the lib uses the old MONGODB-CR authentication method, StephenLynx.

[22:25:20] <damoncasale> Afternoon/evening.

[22:25:54] <damoncasale> So, senior developer here with a long history of working with SQL-based databases, attempting to learn Mongo.

[22:27:44] <mprelude> Is anyone here familiar with SCRAM-SHA1 or have a good ref for implementing it? Going to fix up this lib.

[22:29:10] <JFlash> damoncasale, I'm also learning. would you use mongodb for a social network project?

[22:29:33] <StephenLynx> :v

[22:29:34] <StephenLynx> v:

[22:29:46] <damoncasale> I have a use case that I'd like to try to find the "right" Mongo architecture for. Let's say I have a one-to-many relationship that I'd like to define. Let's say I'm attempting to define the relationships between posts and comments. Each post is owned by a user, and each comment is owned by a user. I want to be able to retrieve all of the posts that have comments by me, a specific user.

[22:30:11] <StephenLynx> I would keep posts in their own collections

[22:30:20] <StephenLynx> with a field having an unique identifier of the user.

[22:30:23] <JFlash> damoncasale, what kind of app are you creating?

[22:30:55] <StephenLynx> and preferably a user-friendly identifier of the user, like reddit does with your login.

[22:31:12] <StephenLynx> so you don;t have to fetch the user nor duplicate his display name.

[22:32:28] <damoncasale> JFlash: One sec.

[22:34:57] <StephenLynx> I am designing my third forumish system with mongo, let me show the design of the current one

[22:35:21] <StephenLynx> https://gitlab.com/mrseth/LynxChan/blob/master/doc/Model.txt

[22:35:48] <StephenLynx> I prefer to not embed complex objects

[22:36:22] <StephenLynx> because it limits what you can do regarding queries and projection. and depending on the case, you could hit the 16mb limit cap on documents.

[22:37:23] <StephenLynx> so there are even more pitfalls for arrays that you expect to expand indefinitely, like posts.

[22:44:42] <JFlash> StephenLynx, the other alternative I was considering was using neo4j :(

[22:45:05] <StephenLynx> never heard about it,.

[22:45:17] <JFlash> its the most popular graph database

[22:46:04] <JFlash> however the community should still be tiny compared to the mongo comunity... so I fear i'd be left to the flies with my questions

[22:46:08] <StephenLynx> I don't know much about graph dbs, but I don't think it fits you scenario better than a RDB.

[22:46:34] <StephenLynx> you don't run relations with an unlimited depth.

[22:46:46] <StephenLynx> that is what I THINK graph dbs are for.

[22:46:58] <StephenLynx> but I have never actually used or read much about them.

[22:47:30] <JFlash> same here

[22:47:35] <mprelude> Wow this new auth system is incredibly complicated

[23:02:46] <damoncasale> That was weird...

[23:03:19] <damoncasale> I have flaky internet and I couldn't change my nick after my 'net reconnected. -_-

[23:04:04] <JFlash> damoncasale, would you use mongodb for a social network project?

[23:04:10] <damoncasale> Anyway, my issue is that I don't know how to query for all of the posts with comments by comment_user. I want a unique list of posts, too.

[23:04:25] <JFlash> damoncasale, what kind of app are you creating?

[23:04:36] <damoncasale> It's not a social network, it's something else.

[23:04:43] <damoncasale> I'm just using that as an example.

[23:04:56] <JFlash> no I am the one createin the social network :)

[23:05:05] <damoncasale> Ah.

[23:05:18] <damoncasale> I'm not the one to ask, then. Total noob as far as Mongo goes. ^^;;

[23:05:28] <JFlash> lol. ok, no worries

[23:05:36] <JFlash> same here

[23:15:34] <StephenLynx> damoncasale its because this channel requires you to auth to talk

[23:15:53] <damoncasale> Yeah, just annoying that I couldn't auth while in the channel.

[23:18:21] <StephenLynx> you can auth

[23:18:27] <StephenLynx> you can't change nicks though\

[23:18:32] <damoncasale> Right.

[23:18:43] <StephenLynx> and if you store your posts like I said

[23:18:51] <StephenLynx> you can retrieve all posts by an user easily

[23:19:10] <damoncasale> I think I missed what you said. Lost my chat log when I had to part the channel.

[23:24:38] <StephenLynx> http://pastebin.com/b7SSptss

[23:24:40] <StephenLynx> damoncasale

[23:24:53] <StephenLynx> in their own collection*

[23:25:55] <damoncasale> Yes, that doesn't quite help me.

[23:26:06] <damoncasale> comment_user is not the same as post_user.

[23:26:11] <StephenLynx> ah

[23:26:22] <StephenLynx> comments on posts, you say?

[23:26:29] <JFlash> StephenLynx, what hour can I find more ppl chating here?

[23:26:34] <StephenLynx> dunno

[23:26:43] <damoncasale> Let's say I have three posts with five comments each. Each comment has a different comment_user. I want to get all posts that have comments where comment_user=X.

[23:26:44] <StephenLynx> I just hang around when I am working

[23:27:04] <JFlash> StephenLynx, what language do you use at work?

[23:27:06] <StephenLynx> you could keep comments on their own collection too.

[23:27:14] <StephenLynx> js and C mostly

[23:27:21] <StephenLynx> its been a while since I lasted used java.

[23:27:35] <StephenLynx> damoncasale you could also keep comments on their own collection.

[23:27:42] <damoncasale> Yes but I don't see how that helps me.

[23:27:44] <StephenLynx> and indicate which post they belong to.

[23:27:45] <JFlash> dammit , why dont you use Python?

[23:27:51] <StephenLynx> because it sucks.

[23:28:17] <JFlash> that way you could teach me how to use python with mongo

[23:28:26] <JFlash> it doesnt suck

[23:28:57] <StephenLynx> duck typing, forced indentation, slow, broke retro compatibility, awful syntax, bad community.

[23:29:06] <StephenLynx> you asked me why.

[23:31:11] <damoncasale> Okay, so basically I'd need to do a two-step query, with the first step being getting all post_ids with comment_user=X?

[23:31:20] <damoncasale> And then getting all posts with that list of post_ids?

[23:32:02] <damoncasale> I think I can figure out how to do that. (Using Mongoose btw.)

[23:33:16] <StephenLynx> hm

[23:33:23] <StephenLynx> I strongly suggest you dont use mongoose.

[23:33:43] <StephenLynx> among its issues, its behaviour is inconsistent its extremely slow.

[23:34:00] <StephenLynx> and you don't need to perform two queries.

[23:34:13] <StephenLynx> just query all comments where user_id equals your user.

Log file Viewer

Help | Karma | Search:

#mongodb logs for Sunday the 6th of September, 2015