Log file Viewer

[04:32:53] <Logicgate> hey guys

[04:33:12] <Logicgate> I need to aggregate date to output like this {videoID: 1, event1: 1, event2: 1, event3: 3},{videoID: 2, event1: 1, event2: 1, event3: 3}, etc...

[04:33:25] <Logicgate> {type: 'event2', userID: 1},{type: 'event3', userID: 1},etc

[04:33:28] <Logicgate> that's the sample data.

[04:35:24] <Logicgate> I need to aggregate by value

[04:38:05] <Logicgate> anybody?

[05:15:45] <staykov> hey using morphia

[05:15:56] <staykov> saw an old users group post say that its safe to use from multiple threads

[05:16:05] <staykov> the database object that is

[05:16:30] <staykov> still feels odd to use it in a webserver like that. is that safe?

[05:20:31] <staykov> ahh nvm saw it on the java drive page

[07:50:09] <Soothsayer> Is there an equivalent of the redis ZADD in Mongo? I am looking to implement the functionality - "If a specified member is already a member of the sorted set, the score is updated and the element reinserted at the right position to ensure the correct ordering."

[07:50:48] <Soothsayer> I explored the $sort operator for an array, but it does not seem to remove the older element with the previous score..

[11:09:40] <Ben`> what is the default number type of mongo? IEEE 754 double-precision floats?

[11:11:00] <ron> Ben`: http://bsonspec.org/

[11:20:49] <Ben`> thanks

[12:10:00] <cm0s> Hi there, I'd like to get your opinion on a specific topic (REST API with mongoDB).

[12:10:28] <cm0s> Let say I have a user which have some comments (embedded)

[12:11:33] <cm0s> I see two way to retrieve it with a rest uri : /api/users/me which will deliver a json response with all the user info and the comments embedded

[12:11:55] <cm0s> or use /api/users/me/comments which will return only the comments

[12:12:42] <cm0s> With the first solution the advantage is : there is only one http request needed to display the user info and its comments

[12:13:34] <cm0s> But the client which consume the REST api have to know he has to search inside the user object to find his comment

[12:14:17] <Soothsayer> cm0s: what do you mean by "find his comment" ?

[12:14:21] <cm0s> with the second solution, we have a cleaner REST api. But we have to run two http request. One to get the user information and a second to get his comment

[12:16:10] <cm0s> @Soothsayer, it's not really finding his comment, bu knowing the comment are in the user object and must not be retrieve with another rest request

[12:16:15] <Soothsayer> cm0s: your decision depends on your use-case.. think about - "how likely is that somebody will ask for the users/me" request and NOT need the comments within the next second..?

[12:16:46] <Soothsayer> If you are seeing a behaviour where those two API's will in most-cases be called back-to-back, then combining it makes sense.

[12:16:55] <Soothsayer> If not, then don't.

[12:17:15] <Soothsayer> You also have the option of keeping both the choices (if somebody wants to query user comments without getting all the other user info)

[12:17:49] <Soothsayer> And you can also keep a parameter called ?fields=a,b,c which decides what fields to send across via REST (so if the user doesn't want the comments.. he can skip it)

[12:19:10] <cm0s> those are great advices, thx Soothsayer.

[12:20:36] <cm0s> But I'm always struggling with the fact we need to choose between clarity (better REST api) or performance

[12:21:04] <cm0s> However, adding the two possibilities (as you propose) seems a good solution.

[12:22:08] <Soothsayer> cm0s: if you're starting off, choose the one that's easier to code and maintain (the first one)

[12:24:01] <cm0s> Yep, I can always improve it later when it will be really necessary

[12:24:32] <cm0s> thx for your help!

[13:21:00] <b0c1> anyone use mongodb from scala (or java)?

[13:22:56] <ron> b0c1: better just ask your question

[13:25:07] <b0c1> ok, what is the prefered way to connection handling?

[13:25:17] <b0c1> I mean I using casbah in my scala app...

[13:25:39] <b0c1> MongoClient is a pool, but what happened if I connect with MongoConnect?

[15:01:18] <alex_1_2> hey guys

[15:01:36] <alex_1_2> can I enforce an unique index on the filename property for GridFS ?

[15:01:53] <alex_1_2> I've tried doing this but it won't work: db.fs.files.ensureIndex({ filename: 1 }, { unique: true })

[15:02:05] <alex_1_2> I want the filename to be unique

[16:28:26] <Ben`> does ObjectId() always create a unique ID, or should I check to see if the ID already exists?

[16:29:38] <ron> http://docs.mongodb.org/manual/reference/object-id/

[16:33:19] <Ben`> thanks

[17:31:30] <wiherek> hi are there any proper CMS for MongoDB based apps?

[17:32:04] <wiherek> I basically need something to administer the documents inside mongo, not a CMS for an app.

[17:32:48] <wiherek> (or not a CMS in the sense of Drupal / Wordpress / Typo, just something to edit the documents)... but I'd be happy to take a look at anything.

[17:38:23] <BurtyB> wiherek, I use phpmoadmin if that's the sort of thing you mean?

[18:01:46] <toothrot> wiherek, i use robomongo

[18:01:54] <toothrot> (it's not web based)

[18:03:11] <toothrot> it's cross platform though, which is nice

[18:36:31] <meltingwax> is the second argument to the reduce() function in a MapReduce passed an iterator or a normal array?

[19:35:26] <ctp> hi folks. i'm running now a replica set consisting of 3 replica nodes (1 primary and 2 secondaries) and an arbiter. all works fine. now i wanna reconfigure that to a replicated sharded cluster. i've added another 3 servers: the config servers. but how does it works exactely in term of: how many physical nodes do i need to make this transition 3-mongod's-in-replica-set to a 3-shard-cluster? is every shard a 3-node-replica-set? so, do i need 9 nodes for this?

[19:37:12] <kali> ctp: that's more or less the spirit, yes. each shard is a replica set

[19:37:23] <kali> ctp: why do you want three shards ?

[19:37:40] <kali> ctp: why do you have an arbiter as you have an off number of replica ?

[19:37:59] <kali> ctp: why have you put the config server on separate servers ?

[19:40:24] <ctp> kali: we are using mongodb for a very high load app. so sharding is a must. 3 nodes shard is the minimum that makes sense. arbiter: we need a completely reliable system that will grow and grow (i expect something like 1-2 additional servers every month). config servers -> same requirement: reliability. thus they are even running in a ha openstack cluster

[19:41:16] <kali> ctp: seriously, you don't want arbiter if you have three nodes

[19:41:29] <kali> ctp: it does not improve reliability

[19:41:37] <ctp> the only point i don't really undestand is: to combine sharding and replication is the product in terms of n replica nodes multiplied by m shards?

[19:41:48] <kali> ctp: yes

[19:42:09] <ctp> kali: but what happens, when a 4th node is added? "... something like 1-2 additional servers every month"

[19:42:19] <ctp> i mean the arbiter thing

[19:42:52] <kali> ctp: you will scale by adding replica to replica set but by adding more shards

[19:43:23] <kali> ctp: sorry. s/will/wont/

[19:43:48] <ctp> i mean 4th replica server for scaling read performance

[19:43:49] <kali> ctp: you can't have more than 7 or 9 replica in a given set anyway

[19:44:39] <ctp> doesn't it make sense to add more replicas as well as more shards to improve read performance and just shards to improve write performance?

[19:45:01] <kali> shards will improves read performance too

[19:46:32] <kali> scaling read by adding a replica makes sense as long as it allow to dodge sharding. but one you've taken the sharding road, it does not make much sense to add replica, imho

[19:46:38] <ctp> oh, sure, sorry. btw, i'm documenting all the stuff in detail to provide a complete kind of walk-through for ppl working on same

[19:47:32] <ctp> 'll publish that next week or so. complex stuff :)

[19:48:40] <kali> seriously, take down your arbiters, there doing more harm than good

[19:49:04] <ctp> kali: btw, with all the replication/sharding running, does it make sense to switch all the stuff to raid-0? i mean when i don't have to care about faults etc :)

[19:49:41] <ctp> kali: ok, i'll disable the arbiter stuff. thanks for learing. i just thought, it always makes sense to add replicas as well

[19:49:57] <kali> what is raid-0 already ? mirror ?

[19:50:13] <ctp> raid-0: more capacity and _maybe_(?) faster data access?

[19:50:20] <ctp> raid-0: striping

[19:50:24] <kali> so that's stripping

[19:50:28] <kali> yeah

[19:50:54] <kali> it can help

[19:52:02] <ctp> sure, the probability for disk failures increases but as far as i can see now, it doesnt matter with 3 by 3 replicated shard

[19:52:23] <ctp> it'll be a long night :) thanks a lot kali

[20:00:15] <ctp> kali: btw, the last one: how does mongodb manage different nodes in terms of hdd's sharding data? i mean the equal distribution of shards per node. what happens when one shard has e.g. 1tb the other 4tb?

[20:02:07] <kali> ctp: apart from the primary shard, which keeps hosting all unsharded data, the shards get a roughly balanced volume of data

[20:05:29] <ctp> kali: balanced == in terms of bytesize balanced or numbers of chunks balanced? the question is: to get all the available space used on servers with e.g. 1tb as well as the 4tb boxes, does it make sense to step down the chunk size to e.g. 16mb on the 1tb server and leave the default 64mb on the 4tb server? you know: 1000 chunks on the one server would correspond to the 1000 chunks on the bigger box, both would have the same hdd "byte-utilization"

[20:08:00] <kali> don't do that :)

[20:08:13] <kali> just have similar boxes

[20:09:33] <ctp> hmhm

[20:09:57] <ctp> however, thanks a lot for your help!

[20:22:30] <ashley_w> trying to build git version of perl driver, but it fails (dzil build && cd MongoDB-0.702.2 && perl Makefile.PL && make)

[20:22:32] <ashley_w> ./perl_mongo.h:22:10: fatal error: 'bson.h' file not found

[20:52:34] <allaire> Anybody use MMS and ever received this error: Detected the possibility of another agent running. Waiting a few minutes to make sure I'm the only agent

[22:08:23] <master_op> hello, how can i install mongodb 2.5.5 under ubuntu

[22:19:13] <ashley_w> master_op: as that's a dev version, i'd be surprised if there was an ubuntu build for it

[22:19:50] <master_op> it's ok, downlod in progress http://docs.mongodb.org/master/tutorial/install-mongodb-on-linux/

[22:31:54] <master_op> hello, can anyone tell how to implement full text search in db.coll.find query instead of runCommand

[22:31:56] <master_op> ??

[22:32:01] <master_op> in mongo 2.5.5

[22:43:02] <rpcesar> first time working with the aggragation pipeline in mongo (used map/reduce in the past). anyways, im running into what seems to be some weird results. im using it to implement a keyword relationship driven search against pre-scored data. $sorting seems to not function in the way i think it would, I am sorting on the score, but it seems to not be gving me them in the right order anyways (but yet it does have SOME impact on the result

[22:44:25] <rpcesar> then I remove the final group for diagnostics, and it shows only one result. which brings me to the unwind. why when i unwind with an array of say 1 element. get back only one element from the unwind if i leave it at the end of the aggragator pipeline. but then when i group it (by using $addToSet) I get all of them

[22:46:25] <rpcesar> the way I read it and understood it was that if i had {a:1 b: [ 1, 2, 3]), and I unwound be, i would get [{a:1 b:1}{a:1 b:2}{a:1, b:3}] but instead if I stop their in the pipeline I get one result {a:1 b:1} which seems useless

[22:47:05] <rpcesar> however if I $group after this, with an $addToSet, it will "add" the 3 elements. even though if you stop before that, it only returns the first. what am I getting screwed up on?

[23:00:43] <rpcesar> actually, maybe im approching this wrong. i had turned to stackoverflow to get this approach. i think my issue is with group losing any semblance of the prior sort. I figured it would be piped into the grouper in order thus the addToSet would be in order. I don't think this is the case. so lets say i have a results array which is built from a group, how do I then sort the subdocuments of it?

[23:01:27] <schreber> If by chance I have a database set up with 2 columns (ex: title & author) how difficult would it be to add a 3rd (format) ?

[23:14:56] <rpcesar> in case anyone was reading my earlier question. i think it was just a mixup with $addToSet when I should have been using $push

[23:44:44] <swak> what's the best online db service to use?