PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 3rd of January, 2015

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:12:28] <mack_> Hi! Can a Mongo node (machine) be a member of multiple replica sets? Like can it be a secondary on one set and a primary on another set?
[00:15:11] <cheeser> no. just one.
[00:15:24] <mack_> Agh
[00:15:39] <mack_> What was the decision behind this?
[00:16:08] <cheeser> well, i wasn't there but probably there are complexity concerns at work here.
[00:18:40] <mack_> Hmm, because I'm thinking, if one were to deploy two Mongo machines for geographically distributed replication (east coast / west coast) then they might as well also be used to reduce latency for homed in users who are closer to them. Since Mongo doesn't have multiple masters, I was thinking of creating two shards (replica sets) and have each machine host one of the sets as a master and host another as the slave.
[00:19:16] <cheeser> you can do sharding like that, yes. each shard would/should the be primary of a replica set.
[00:19:29] <cheeser> e.g., http://docs.mongodb.org/manual/core/tag-aware-sharding/
[00:20:18] <mack_> Yes but then, as you mentioned (maybe I misundersood) one mongo node (machine) can only be a part of one replica set, not more
[00:20:47] <cheeser> that's correct. but there'd be 2 replica sets: one backing each shard
[00:22:18] <mack_> Hmm, so a machine (node) can only be a primary on one replica set, but it may be a secondary on as many as you'd want?
[00:22:49] <cheeser> no. it can only be a member, primary or not, in one replica set.
[00:23:29] <cheeser> http://docs.mongodb.org/manual/core/sharded-cluster-components/
[00:24:21] <mack_> So then, one machine would host one shard of the data but not be a master of its replica set, then the other machine would host the replica set (one instance/backup) of the other machine's shard?
[00:24:47] <mack_> So then, one machine would host one shard of the data but not be a master of its replica set, then the other machine would host the replica set (one instance/backup) of the first machine's shard?
[00:25:12] <cheeser> no. each shard is/can be the primary of its own replica set.
[00:25:43] <cheeser> http://docs.mongodb.org/manual/sharding/
[00:28:42] <mack_> Thanks for the links cheeser, if you have two machines, how can you then create two shards (primaries of replica sets) on each of the machines and have the other machine be a secondary ("backup" as you put it) on the other's shard?
[00:29:00] <cheeser> read the links.
[00:29:27] <mack_> Wouldn't this seem to contradict the rule you mentioned that each machine (node) can only be a part of one replica set?
[00:29:36] <cheeser> not a bit.
[00:29:49] <cheeser> each one would only be a member of one replica set.
[00:30:03] <cheeser> read the links.
[00:30:58] <mack_> ok, thanks
[03:22:40] <haggen> hey fellas, I need a little help with an aggregation
[03:24:48] <haggen> i have a history of buying and selling orders -- fields are item, price and a boolean that's true for buying and false for selling, i'd like to output 2 average prices per item: buying-average and selling-average
[03:25:02] <haggen> has anyone got any ideas ?
[03:29:18] <haggen> so 100+ users online in this channel and everybody's afk =/
[03:29:23] <Nilium> Yes.
[03:29:28] <haggen> damn
[03:31:23] <Nilium> And I'm currently drinking and am kind of bad with mongodb since my entire experience with it has been trying to eek what little performance I can out of someone else's garbage code.
[03:33:24] <haggen> it's ok, enjoy your drinking night
[03:33:53] <Nilium> I probably won't be wasted tonight unless I somehow manage to down this entire wine bottle, so I'm probably fine
[03:34:19] <haggen> order a pizza, pizza goes well with wine
[03:34:39] <Nilium> Anyway, you probably want to look into what you could do with either mapReduce or aggregate (not sure about the actual usage of either -- I haven't touched them)
[03:34:44] <haggen> not pizzahut or dominos , a more italian kind of pizza
[03:35:29] <Nilium> Could probably just use $avg and filter on the bool though
[03:35:43] <haggen> thanks :) i probably could find a fix with mapReduce but i was trying to stay away of it, you know, when you've got a hammer everying's kinda looking the same
[03:35:43] <Nilium> With aggregate
[03:36:28] <haggen> wait wait, the part about "filter with the bool" what's that ?
[03:37:25] <Nilium> Use the buying/selling bool as a selector to only match those documents, then average over them
[03:37:25] <haggen> i was trying with aggregate right, i manage to average all the prices of an item, buying or selling, but I want to split the averages in two fields
[03:37:50] <haggen> but then I would have two results wouldn't I ?
[03:38:01] <Nilium> Sure, but you can just combine them programmatically.
[03:38:02] <haggen> i'd have to merge them or something like this
[03:38:19] <haggen> oohhh I don't think so =/
[03:38:24] <Nilium> How so?
[03:38:37] <haggen> there's too many rows, like millions
[03:39:03] <Nilium> Yeah, I just mean combining the averages after the fact.
[03:39:29] <Nilium> Alternatively, group them as part of the aggregation
[03:39:51] <haggen> by "programmaticaly" you didn't mean outside the mongodb, did you
[03:40:07] <haggen> I'm kind getting your point but I have no idea how to do that
[03:40:33] <Nilium> i.e., db.coll.aggregate [ { }, $group: { buyOrSell: "$someName", avg: { $avg: "$amount" } } ]
[03:40:47] <Nilium> I may be missing brackets, I don't know
[03:40:52] <haggen> it's ok
[03:40:58] <haggen> I can manage the brackets hahah
[03:41:37] <Nilium> Yeah, there should've been a curly brace before $group
[03:41:37] <haggen> ok then how do I merge them into a single document with two avg fields ?
[03:43:02] <Nilium> Also, I should've written $group: { _id: "$buyOrSell"…
[03:43:24] <Nilium> Beside that, merge them programmatically once you get the two documents back from the aggregate
[03:43:37] <Nilium> No real point trying to do everything in the DB.
[03:43:45] <haggen> hmmm, I think you really helped me here
[03:44:01] <haggen> thank you!
[03:44:29] <Nilium> Well, you'll know whether or not if you manage to get it working, I'm just throwing out ideas for things I've not actually done
[03:46:29] <Nilium> Also, this raspberry wine was not the best use of $12.
[04:42:50] <barque> So if I want to communicate with a MongoDB instance running on a local server, I need the C++ drivers I presume... and for a production environment you guys would recommend 26compat? Also, I'm not getting this wrong... right? I can read to and write from a local MongoDB instance...
[04:43:18] <barque> also I need to actually communicate from a program written in C++ :)
[04:43:44] <barque> Basically a game server... that has a NodeJS/MongoDB side for the web 'side' of thing
[04:45:23] <cheeser> node.js *and* a c++ server?
[04:46:52] <barque> a C++ *game* server
[04:46:59] <barque> that needs to write to MongoDB
[04:47:22] <barque> basically imagine a game like Quake 3 Arena or something... and the stats are going to a MongoDB server
[04:47:38] <barque> and they'll be served from NodeJS for when the user logs in to their profiles online :)
[04:48:15] <barque> well it's not Quake 3 Arena,... but to that point it's an Multiplayer Online First Person Shooter
[04:48:41] <barque> I've decided not to opt-in for something classic like ngin-x/MySQL
[04:49:25] <cheeser> i see.
[04:49:26] <barque> the MMO portal is entirely HTML5 Canvas + Javascript... so I'm actually willing to throw my lot in with a full (J/B)SON NoSQL solution + JavaScript backend
[04:50:00] <barque> I've heard that NodeJS and MongoDB together have come quite a long way
[04:50:11] <barque> and looking forward I'm willing to bet they'll even get better
[04:50:20] <barque> and I'm also willing to bet they already scale fairly
[04:50:23] <barque> well
[04:50:27] <barque> if I'm not mistaken :/
[04:51:00] <barque> also players may buy stuff online or in-game... and the game needs to go back to the MongoDB server to write that stuff
[04:51:22] <barque> also the game may look at their new chat messages on their BSONs in case another player has sent a friend request or a chat message
[04:51:33] <barque> so the game server has to go back and forth with the MongoDB instance
[04:52:17] <barque> (in the given example, the game server will relay chat messages/friend requests to their clients for in-game display)
[04:52:33] <barque> all-in-all is 26compat the way to go ?
[04:52:42] <barque> it says it's production stable
[04:52:42] <cheeser> i don't know what that is.
[04:52:57] <barque> https://github.com/mongodb/mongo-cxx-driver
[04:53:03] <barque> the C++ driver
[04:53:14] <barque> there's legacy/26compat/master
[04:53:38] <barque> my question is, is 26compat going to work with the latest version of MongoDB ?
[04:53:50] <barque> as in... is that what I should be looking at?
[04:53:58] <cheeser> it should, yes.
[04:54:20] <cheeser> "If you are looking for a production-ready release of the C++ driver you should use the 26compat branch until our RC testing is complete."
[04:54:29] <barque> Right I know it says that :D
[04:54:35] <barque> but I'm looking for a confirmation here
[04:54:38] <barque> from experts :P
[04:54:40] <barque> preferrably
[04:54:55] <barque> oh shnap
[04:55:06] <cheeser> (though i'm a java guy)
[04:55:09] <barque> so why did you say you 'don't know what that is'
[04:55:20] <barque> oh you just didn't know 26compat? :)
[04:55:42] <cheeser> right.
[04:55:48] <barque> I did JEE5+Spring at the government for over a year
[04:55:51] <barque> worst time of my life
[04:56:00] <barque> anyway
[04:56:04] <barque> putting that aside
[04:56:16] <cheeser> all you need is Java EE. putting spring in there is dumb. :D
[04:56:31] <barque> I hope you're excluding EJBs too
[04:56:49] <barque> Oh god don't you even dare bring that up...
[04:57:05] <barque> I'm gonna wrap you in an entity bean and shoot you to Mars XD
[04:57:42] <barque> anyway...
[04:57:46] <cheeser> i've actually never used EJBs. unless you count EJB3/hibernate
[04:57:52] <barque> mmmmmm
[04:57:55] <barque> fiiine
[04:58:08] <barque> EJB3 .. Spring 2, all the same to me
[04:58:13] <barque> Hibernate is ok
[04:58:18] <barque> actually
[04:58:19] <barque> no
[04:58:28] <barque> I was about to rag on relational databases period
[04:58:32] <barque> damn it
[04:58:42] <barque> Hibernate is not ok
[04:58:44] <barque> not even damn close
[04:58:53] <barque> thank god for non-relational DBs
[04:59:07] <barque> seriously, I'm actually getting sick of them
[04:59:17] <cheeser> i'm actually working on a morphia extension right now.
[04:59:19] <barque> it's the universal golden hammer
[04:59:23] <barque> seriously
[04:59:36] <barque> this crap from the 70's has become the global golden hammer
[04:59:48] <barque> it's the reason we invented the statement 'impedence mismatch'
[05:00:04] <cheeser> it's not actually. that's an old electronics term.
[05:00:20] <barque> lol yes I did finish a Comp Eng degree
[05:00:22] <barque> you're right
[05:00:32] <barque> but it's the reason it leaked into DB talk
[05:01:21] <barque> not long ago I was doing a TomCat based backend for a full-JS-front-end WebApp
[05:01:35] <barque> and the dumb ass kept insisting on using relational DBs and Hibernate
[05:01:55] <barque> especially when the front-end had extremely flexible notions of object structures and used mix-ins heavily
[05:02:15] <barque> I had to sit the guy down and explain to him why he was orchestrating a disaster
[05:03:06] <barque> I was using Gson on the backend just to fiddle with the incoming data structures
[07:36:59] <sahilsk1> hi
[07:37:26] <sahilsk1> Hi guys, I"ve a query related to authentication in mobile app. I"ve one webservice and want one mobileApp to grant access to my api. There are two things to grant here: First mobile app itself , second user login on that app. any expert advise on how to do it, will be great..
[09:50:34] <laurensvanpoucke> Hello, assume that I have an existing document like { name:"test", urls: [ 'http://firsturl.com' ]} and I want to update it later to { name:"test", urls: [ 'http://firsturl.com', 'http://anotherurl.com' ]}, how can I achieve this ? update an existing document value, and if it has already a value, 'merge' it
[09:50:40] <laurensvanpoucke> so not override it
[09:52:07] <laurensvanpoucke> I'm using mongoose
[11:20:38] <laurensvanpoucke> Can I do an update on an instance of a model ? I know the syntax is Place.update(where statement, {}) but can I do var place = new Place(); place.update ???
[13:12:35] <gisto> hi there
[13:12:40] <gisto> anybody around?
[13:13:03] <gisto> I have a quick question on bulk updates
[15:17:31] <mordof> question involving the geospatial indexes.. i'd like to store polygons in a collection (recording x,y locations of a box) - and then query a single coordinate to find which boxes contain that coordinate. it's not using earth lat,lng though. is this supported?
[15:18:16] <mordof> from what i can tell in the mongo documentation, the geospatial indexes all calculate using the assumption that it's on the surface of a sphere - so wouldn't that mess up a flat surface check?
[15:25:15] <mordof> am i just coming in at bad times? or is my question too obscure?... whenever i come in here to try and get help on this topic it seems dead / nobody will respond to me
[15:25:16] <mordof> :/
[15:58:59] <Guest45337> hi
[16:25:21] <Patteh> i'm pushing data to a single db collection, and I want to set a limit of 10 documents from each different value of a single selector
[16:25:43] <Patteh> i'm slicing the data before insert so a total or 10 of each is ever presented
[16:26:01] <Patteh> but that wont account for what is already in the db
[16:26:11] <Patteh> is it bad practice to wipe the db before each round of inserts?
[16:26:26] <Patteh> is there a max i can set that will effectively 'push' the other docs out?
[19:05:55] <voidDotClass> In Mongo, is where an equivalent to sql's IN(...) feature, i.e Select where id in (1, 2, 3)
[19:06:02] <voidDotClass> is there*
[19:06:25] <voidDotClass> i.e to get all rows whose id is within a set
[19:10:56] <LesTR> voidDotClass: http://docs.mongodb.org/manual/reference/operator/query/in/
[19:11:40] <voidDotClass> LesTR: thanks, how would i use that with the java driver?
[19:12:47] <LesTR> same like in another language. Doc is your friend
[19:12:49] <voidDotClass> nvm
[19:13:08] <voidDotClass> yeah its not really that user friendly / pretty piss poor , http://stackoverflow.com/a/11653460/49153
[19:13:21] <bmillham> Yes voidDotClass $in
[20:32:01] <proteneer> how do you negate elemMatch
[20:33:03] <proteneer> never mind
[20:33:07] <proteneer> need to use $nt
[20:33:08] <proteneer> $not*
[21:46:22] <alexi5> hello
[21:46:55] <barque> is there any other way for me to use the Windows C++ drivers other than using SCons?
[21:47:02] <alexi5> I have heard that objects have a less impedance missmatch with document databases in camprison to relational databases. But with ORMs like entity framework that can model complex relationships, why is there a need for document databases ?
[21:47:46] <barque> alexi5... imagine the amount of overhead required for something like Hibernate to serialize deserialize
[21:48:19] <barque> also 'JSON objects' more than 'Objects' (let's not get ahead of ourselves... you can't probably use it well for C++ objects.)
[21:48:59] <barque> (with like 3 levels of inheritence point to random locations on heap and containing 3rd party API handles :)
[21:49:18] <barque> s/point to/that point to
[21:49:59] <barque> Also BSONs have their limits too... I mean 100 levels of nesting max.... it's pretty big but nevertheless a limit
[21:50:06] <barque> also 16 MB maximum size of each document
[21:50:32] <barque> also replace 'a less' with 'they just freakin' don't'
[21:54:06] <barque> Basically to put it this way... if you're using a crapload of JavaScript on the frontend... you will itch to use JavaScript on the bankend (NodeJS anyone?)... and if you're using it on the backend... then you'll itch to store your JSONs as they are... in which case MongoDB or CouchDB will kick in
[21:55:10] <alexi5> so ii guess using ORM and RDBMS to store objects with complex relationships is like putting a square in a round hole
[21:55:21] <barque> yep
[21:56:11] <barque> And from what I've see, you really have to be able to boil it down to primitive data types
[21:56:40] <barque> And the same goes for relational DBs too anyway
[21:57:23] <alexi5> i was trying to decide between a RDBMS and mongo fo an application that will catalog various equipment with different attributes , and from your explanaition it looks like i may shoot myself in the foot by using an RDBMS
[21:58:12] <barque> alexi5, I spent over a year at the government doing JEE5+Spring (JSF/RichFaces/MyFaces/Spring/MyBATIS/iBATIS)
[21:58:23] <barque> it was like watching a lot of grown men pour money in a pit and set it on fire
[21:58:41] <barque> WAS7 load balanced with IHS + Oracle 10g for persistence
[21:59:03] <barque> Such a slow pile of dog ****
[21:59:16] <barque> buggy, bloated, costly, painful, old and junk
[21:59:36] <barque> littered with cargo cult programming
[21:59:52] <barque> loads and loads of design patterns with no real purposes
[21:59:59] <barque> not to mention full of ****ing spelling errors
[22:00:01] <barque> Fasade
[22:00:08] <barque> Warpper
[22:00:21] <barque> Tire (as in Trie/radix-tree)
[22:00:39] <alexi5> it look like they when to town with the design patterns :)
[22:00:50] <barque> Jesus H Christ in a hand basket, don't remind me
[22:01:15] <barque> I work at a bank now... sadly not much has changed (WAS, EJB3s, Portlets)
[22:01:28] <barque> I think they also use Struts?
[22:01:34] <barque> ****ed if I know
[22:01:41] <barque> I work in the mobile department
[22:03:15] <barque> The more time I spend in the I.T. industry, the funnier it becomes
[22:03:22] <barque> people are lost ..... everywhere .....
[22:03:47] <barque> they're trying to one-up each other in using the trendiest crap out there.... instead of really stopping for 2 seconds to evaluate what they're doing
[22:03:57] <barque> and they make up the most ridiculous arsnine excuses to go with it
[22:04:03] <barque> not to mention politics rules the day
[22:04:12] <barque> it's a ****ing 3-ring circus no matter where you go
[22:04:24] <cheeser> this really belongs in a blog and not in channel
[22:04:30] <barque> good point sorry :D
[22:05:56] <alexi5> barque: thanks for sharing
[22:06:05] <barque> yw
[22:06:08] <barque> but yea I went on a rant
[22:06:10] <barque> sorry about that
[22:23:28] <alexi5> i wonder what I.T companies are like in the UK
[23:21:51] <barque> if I have a binary on the same machine as a mongod instance... and I'm communicating with the mongod through mongo command line with redirecting .js files instead of stdin... is that efficient?
[23:22:02] <barque> like
[23:22:10] <barque> mongo --quiet < somefile.js
[23:37:19] <joannac> barque: sure, why would that not be efficient?