pmxbot IRC Log Viewer

[07:33:21] <MasterGberry> hi. i have a quick question about indexes. do they behave simularly as in mysql? and do you force on creation of a collection? of during insert time?

[07:40:41] <MasterGberry> guess everyone is asleep. ill check logs in morning if someone could leave a response. thanks

[08:31:09] <kali> MasterGberry: yes, mongodb index are roughly what you would expect coming from sql. creating an index on a non existing collection will create the collection, but creating the index after the collection is seed works too

[08:32:40] <kali> MasterGberry: you want to check out the "background" option, because by default index creation locks everything (not an issue if the collection is empty)

[13:25:34] <msucci> has anyone had success combining text search with geospatial queries / is it even possible

[16:08:44] <MasterGberry> kali: hey are you still around?

[16:11:31] <MasterGberry> How do you create a collection in advanced? From what i understood the collection gets created when you insert a document for the first time.

[16:11:50] <MasterGberry> And what types of indexes are there?

[16:13:01] <MasterGberry> Lastly, there are no joins in mysql. Let's say i'm trying to build a basic private messaging system. I want to have 2 collections (tables). One that lists the participants in a conversation, a unique id for the conversation, and another collection for all of the messages for that conversation. In mongodb is it wiser to nest the second collectino into the first as a sub-column such as "messages"?

[16:13:28] <MasterGberry> Or am i better off keeping them separate? Is it possible to update a subcolumn with new rows? let me link a quick exmaple of what i am talking about

[16:16:19] <MasterGberry> http://pastebin.com/gzKDNwnU

[16:16:37] <MasterGberry> I have to log off again for an hour. But I will be back and then would love to talk this out with someone :)

[16:16:41] <MasterGberry> Thanks!

[16:16:57] <MasterGberry> Trying to get the hang of mongodb so I can use it today to start writing an application.

[17:36:27] <MasterGberry> Hey, I am back if anyone wants to take a few minutes to help me understand the questions i mentioned above :)

[17:45:42] <hitul> Anyone have review about mongodb new version ?

[17:51:42] <aandy> hi, this is probably really stupid, but: i'm trying to follow the deployment guide for replication sets, using a (for now) two host setup. both are reachable from each other (as per "connectivity test") but the states don't make sense to me. on box A: rs.initiate(); rs.status() -> stateStr: PRIMARY, rs.add(boxB). box B

[17:52:19] <aandy> whops., cont: .. box B, rs.status(), shows just itself, and supposedly as PRIMARY.

[17:52:55] <aandy> both are part of same rs, but it seems like only one box can see the other, or am i reading it wrong?

[17:53:10] <aandy> shouldn't i be able to see the health of both members from both boxes?

[17:57:07] <aandy> http://pastie.org/private/kpnh2qyy6gw59aik25u6a here's conf + status after initiate and add

[17:57:34] <aandy> it's box b rs.conf output which concerns me

[18:12:02] <sw99> For a new forum webapp, can you give me some reasons for why i should choose mongodb?

[18:12:17] <sw99> it's basically mongodb vs mysql for me

[18:12:20] <sw99> hard to decide :(

[18:15:13] <aandy> sw99: you mention two non-related database types. you should always choose the db which fits the project's needs (what kind of data to be stored, what type of queries, which demands to backup, bigdata?, etc) http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

[18:17:56] <sw99> well, for a forum i have posts, replies, users

[18:17:57] <sw99> etc.

[18:18:09] <sw99> i guess the search functionality would need to search for a lot of things

[18:18:21] <sw99> not sure i have to worry about big data..

[18:18:32] <sw99> aandy: ^

[18:23:18] <aandy> i'm not the guy to ask. you could structure that in both sql and mongo easily. so a quick off the top of my head, you might prefer mongo because you can set up backup, replication, failure and HA fairly easy, and search options are typically more broad out of the box than mysql (postgres is another case, which has fairly solid fulltext search builtin)

[18:23:38] <sw99> ok

[18:24:23] <sw99> i guess i won't develop a feel for it until i develop a database i mongo

[18:24:33] <sw99> then I will "feel" what i can or can't do

[18:24:39] <sw99> and where mysql would have been nicer

[18:29:08] <aandy> a slight adjustment to your structure: you have posts, users. some posts are replies, and so i'd define them as such (e.g. { replyTo: <some other post id> }, with index on dates and replyTo for quick pull of a "thread" (and sort)

[18:29:26] <aandy> anyway, just my two cents

[19:20:19] <mastergberry> What do people normally use when trying to query by a _id as far as the public can see?

[19:20:37] <mastergberry> Is it safe to let the user query via the uuid? aka www.example.com/some/post/{uuid_goes_here}

[19:40:14] <ExxKA> Using Mongoose: When I save a document, and later change it, then save it again - there is no return value the second time. Is this something you have experienced before?

[19:51:45] <sinisa> hi all... should i use distinct on multiple fields to get something like "facets" per phrase with aggregation .. or should i use solr or something.. anyone with experiance?

[19:56:48] <ExxKA> sinisa, what are you trying to achieve?

[19:57:46] <sinisa> im working on some app that should filter things.. like for exact values in multiple fields, and some ranges (prices,dates)

[19:58:19] <sinisa> also depending on query, i should get count on matched values

[19:59:04] <sinisa> like match:query, and return distinct on fields.

[19:59:12] <ExxKA> you can easily do that with mongoDB

[19:59:24] <sinisa> how its performance

[19:59:32] <ExxKA> mongoDB also supports full text search now, so you can do that aswell, but obviously solr will be more mature

[19:59:34] <sinisa> i will put index on everything

[19:59:44] <sinisa> this is not searching, its filtering

[19:59:57] <ExxKA> It's the same thing.

[20:00:06] <sinisa> i dont think so, but ok

[20:00:24] <ExxKA> Technically there is no difference, it is all searching a data set

[20:00:41] <sinisa> you filter by exact value, searching is searching for terms or exact values

[20:01:21] <ExxKA> Right, that is the difference between normal queries and what is known as full text searching.

[20:01:53] <sinisa> however.. on 1 mil documents, and 20 fields to perform distinct.. how would that work , in time speed

[20:02:21] <ExxKA> To answer your question on performance, it will be as good as the index complexity. So if you put indexes on everything it will not necesarrily be faster because you will have a lot of indexes at the same time, that are not needed.

[20:03:09] <ExxKA> in computer science you measure performance a little bit different that seconds, minutes of hours, because that is all dependent on hardware and operating systems.

[20:03:14] <sinisa> hmm.. if you perform distinct on fields.. isnt index required ?

[20:03:25] <ExxKA> I will find you the article you need on wikipedia, just a sec.

[20:04:33] <ExxKA> http://en.wikipedia.org/wiki/Analysis_of_algorithms

[20:04:47] <ExxKA> This introduces performance and complexity.

[20:05:19] <sinisa> thats too much for me :)

[20:05:43] <kali> oO

[20:05:45] <ExxKA> So the best answer anyone can give you, is try it out. MongoDB will be better than anything you can write yourself, and most likely quicker than relational databases for what you are trying to do

[20:06:16] <sinisa> im long time mongo user.. i do everything in it.. but never have to do something like this

[20:06:37] <ExxKA> sinisa, if you are not willing to put effort into it yourself, you won't find the right answer. Nobody knows your problem like you do yourself, so no one will be able to answer you 100% correctly

[20:07:04] <ExxKA> Right... Maybe I have misunderstood your question.

[20:07:06] <sinisa> i know.. but people test all kind of things.. thats why i ask it

[20:07:12] <ExxKA> Will you rephrase it please?

[20:07:39] <sinisa> aggregation framework.. i have 20 fields, have to perform distinct on them..

[20:09:09] <sinisa> first match:query, and from matched perform distinct on 20 fields

[20:09:58] <ExxKA> Okay, I couldnt answer that, I have no experience with a query like that. Maybe someone else can chip in?

[20:11:58] <mastergberry> What do people normally use with mongodb when it comes to looking up a specific row?

[20:12:07] <mastergberry> I assume people don't have the uuid publicly visible in HTML?

[20:12:22] <mastergberry> aka http://www.example.com/forum/post/{some_id_here}

[20:12:39] <sinisa> master.. whatever you like

[20:13:15] <sinisa> make counter collection.. put myids:0... on insert make $inc:1.. so you can have /forum/post/1-slug

[20:13:25] <ExxKA> mastergberry, just put an index on the field you are using to lookup.

[20:13:38] <sinisa> i think he wants to be nice :)

[20:14:26] <mastergberry> I read in the mongodb documentation that it is not recommended to use regular ints because of conflictions?

[20:14:36] <sinisa> conflictions?

[20:14:37] <mastergberry> I assume I shouldn't worry about this now though since I don't have a large scaled system

[20:14:43] <mastergberry> Race conditions I think

[20:14:55] <sinisa> race conditions on what

[20:15:04] <mastergberry> Who gets the next incremented int

[20:15:06] <sinisa> $inc is atomic

[20:15:14] <sinisa> no conditions on that

[20:15:49] <mastergberry> Errr, so u r saying to make a collection that just stores int counters?

[20:15:55] <sinisa> sure

[20:16:10] <mastergberry> How do you map the counters back to the original documents in the other collections then?

[20:16:17] <sinisa> db.counters.save({"formposts":0})

[20:16:30] <sinisa> map?

[20:16:57] <sinisa> i see

[20:17:22] <ExxKA> mastergberry, the best solution will be to use the uuids, but if you want to create your own primary keys you can, just make sure that there are no collisions. There is documentation on how to do your own primary keys, but It would probably be easier for you to just use something like the email address or title of the blog post as the identifier, if you are doing a web app.

[20:17:27] <sinisa> you update it with findandmodify

[20:17:41] <sinisa> it will return you value

[20:18:06] <sinisa> he is making forum , so no email for id ;)

[20:18:15] <mastergberry> Not a forum, but was just an example

[20:18:31] <sinisa> everything is forum and everything needs some id.. or slug unique

[20:18:32] <mastergberry> Trying to build a mini-chat service in javascript/mongodb

[20:18:41] <ExxKA> Cool

[20:18:41] <mastergberry> so need an id for the chat rooms

[20:18:43] <sinisa> master.. use node for that

[20:18:56] <ExxKA> Take a look at a framework called Meteor :)

[20:19:02] <sinisa> or meteor :)

[20:19:34] <sinisa> node, mongodb/redis, socket.io .. done :)

[20:19:37] <ExxKA> It will wrap mongodb into a client side library, and handle most of the server side for you out of the box

[20:19:45] <ExxKA> So you can focus on the client side yourself

[20:20:22] <sinisa> master.. if its chat rooms.. make a slug on it..

[20:20:55] <mastergberry> a what?

[20:21:04] <sinisa> hm.. if rooms are created automatic.. no :)

[20:21:17] <sinisa> slug.. /chats/myroom1

[20:21:29] <sinisa> ... /chats/mongotalks

[20:21:31] <mastergberry> At the moment, it would be 1 chat room per team league

[20:21:45] <mastergberry> so i guess i could just make the names of the chat rooms unique

[20:21:48] <mastergberry> and then query with that

[20:21:51] <sinisa> good .. /chats/leagename

[20:22:31] <mastergberry> As far as all of these new javascript frameworks

[20:22:40] <sinisa> maybe you want to read about capped collections and tailed cursor

[20:22:51] <mastergberry> i was planning on just using pymongo with some basic async calls via javascript

[20:23:22] <sinisa> chat should work with socket or some transport..

[20:23:36] <sinisa> you cant beat node,redis and socket.io in that field

[20:23:46] <sinisa> that just works

[20:24:27] <mastergberry> Well the thing is I already have a website using the python framework already

[20:24:36] <mastergberry> can't have them both listening on port 80 :/

[20:25:09] <sinisa> put nginx in front of it .. proxy it :)

[20:25:13] <ExxKA> mastergberry, use a reverse proxy then.

[20:25:18] <ExxKA> Like sinisa said ^

[20:25:28] <sinisa> or make node listen on some other port.. why not

[20:25:42] <sinisa> socket.io js client will connect to it.. game over :)

[20:27:01] <mastergberry> meh, i don't think its in my best interested to start playing around with javascript frameworks yet

[20:27:11] <mastergberry> too much wasted time when i am trying to push this feature out fast

[20:27:21] <mastergberry> can probably port what i build onto it later though

[20:28:16] <sinisa> that really depends what kind of chat is that, for how much clients

[20:28:34] <sinisa> you can make it with page refresh ;)

[20:28:49] <mastergberry> It wouldn't be that many clients. ALso trying to build some form of a live drafting/picking system

[20:28:58] <mastergberry> but again i can just do a bunch of setInterval JS calls I suppose

[20:29:07] <mastergberry> to have the clients ask the server what is going on every X min

[20:29:47] <sinisa> every x min .. you will be fine with mongo and everything you have to do :)

[20:30:45] <sinisa> ok, where are the mongo gurus.. can someone help me with aggregation and distinct enourmous amount of fields? :)

[20:31:41] <mastergberry> sinisa, how should i force secondary index on the slugs?

[20:32:54] <sinisa> force?

[20:33:02] <sinisa> ensureIndex

[20:33:56] <sinisa> please dont force anything , that never ends good :)

[20:35:55] <mastergberry> does mongodb do a good job at indexing things nautrally?

[20:36:18] <mastergberry> like mysql sucks if it doesn't have indexes on columns that are queried a lot

[20:37:03] <sinisa> maybe to read about indexes and mongo ?

[21:09:18] <bluepnume> Is it possible to store a web socket connection in a mongodb database?

[21:42:37] <ixti> hi all

[21:43:20] <ixti> does reduce function in map/reduce should return same amount of fields in result object or result should just match same type map emitted?

[21:44:11] <ixti> stupid question i guess. so no answer needed

[21:45:20] <ixti> in fact it's said in docs: the type of the return object must be identical to the type of the value emitted by the map function

[21:45:29] <ixti> so i guess onlty this matters then

[21:58:30] <aandy> any way to figure out which version the mongo-dev files are?

[23:24:57] <ixti> aandy: /quit

[23:25:02] <ixti> sorry

[23:25:04] <ixti> :D

[23:25:23] <ixti> wanted to reply that it's possible to get ref of commit

[23:25:41] <ixti> but then realized that i never used mongo dev so don't know anything about

[23:25:48] <ixti> and then decided to quit :D

[23:36:43] <Wardrop> With the ruby driver, is it possible to set the write concern option after instantiating the connection? It looks like the write concern can only be given on object instantiation?

Log file Viewer

Help | Karma | Search:

#mongodb logs for Sunday the 2nd of June, 2013