PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Tuesday the 28th of October, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:49:33] <mgeorge> is there a best practice configdb dir for rhel? the default selinux blocks mongodb from accessing /data/configdb/
[01:59:59] <mgeorge> so apparently to fix 2.6.5 on rhel7 to use /data/configdb you need to change the context of /data to "mongod_var_lib_t" using the semanage fcontext -a -t mongod_var_lib_t '/data(/.*)?'
[02:16:57] <mgeorge> or just use "semanage permissive -a mongod_t" which i guess is a little easier till dev's create a context "mongod_configdb_t"
[02:28:43] <Streemo> does anyone know why when I do db.posts.insert({title:'blah'}), the _id field is populated with something like ObjectId("ejlksfhskljr32kj4wjasd")
[02:29:02] <Streemo> Why is it wrapping the id in the ObjectId function?
[02:29:14] <GothAlice> http://docs.mongodb.org/manual/reference/object-id/ — technically, it's a type.
[02:29:24] <GothAlice> Like Date(…)
[02:29:57] <Streemo> I know
[02:30:06] <Streemo> but why then are other _id's of the form "adfsjghajkehq3"
[02:30:29] <Streemo> some of my objects have _id's without the ObjectId portion, others do.
[02:30:35] <GothAlice> That's not good at all.
[02:30:43] <Streemo> wait, what?
[02:30:51] <GothAlice> Ones that aren't wrapped aren't actually ObjectIds, and won't sort properly.
[02:30:57] <Streemo> i mean im not doing it on purpose
[02:32:56] <GothAlice> I suspect you have an upsert that specifies _id… as a string.
[02:33:20] <Streemo> the objects i created from a third party
[02:33:25] <Streemo> have _id as a string.
[02:33:30] <GothAlice> …
[02:33:41] <Streemo> auto-assigned
[02:33:45] <Streemo> i dont specify id
[02:33:47] <Streemo> at all
[02:34:05] <Streemo> but i call something like db.insert(...) without _id and then my db gets populated with an object with a string id
[02:37:02] <Streemo> hold on i i'll reset the database
[02:37:06] <Streemo> and see if thats true
[02:37:34] <GothAlice> AFIK inserting without specifying an _id will *always* result in a true ObjectId as the value. Only if you explicitly insert a value for _id (or accidentally set it in an upsert) will it differ from this.
[02:37:57] <GothAlice> (There are very good reasons to primary key on ObjectId _ids, FYI.)
[02:46:18] <Streemo> GothAlice: when i call Framework.collection.insert(...) (from the JS framework) the resulting _id is a string. When i do it straight in mongo it's an ObjectId.
[02:46:52] <Streemo> the Framework must be wrapping some wierd thing arround that vanilla insert
[02:47:02] <Streemo> to make the _id come out as a string
[02:47:11] <GothAlice> A haaa. Your JS driver must only expose them in their encoded string form. Are results in the shell sane?
[02:48:45] <Streemo> like in mongo shell?
[02:48:48] <GothAlice> Aye.
[02:49:01] <GothAlice> db.collection.find({}, {_id: 1}) # < i.e.
[02:49:09] <Streemo> accessing documents in mongo shell which were created in the driver results in _id being a string type
[02:49:16] <GothAlice> …
[02:49:21] <Streemo> o.o
[02:49:54] <GothAlice> That's very not right. Can you pastebin one of the insert() lines from your code?
[02:50:03] <GothAlice> And which driver, specifically, are you using?
[02:50:05] <Streemo> from my JS driver?
[02:50:08] <GothAlice> Aye.
[02:50:10] <Streemo> framework called Meteor
[02:50:19] <Streemo> inclues its own interface to mongo
[02:50:54] <Streemo> Posts = new Meteor.Collection('posts') ===> creates a 'posts' colleciton in mongo (db.posts)
[02:51:33] <Streemo> yeah one sec
[02:51:56] <GothAlice> What version of Meteor? https://github.com/meteor/meteor/issues/61 < are you doing something like this?
[02:53:21] <Streemo> https://gist.github.com/Streemo/09fd2456fc1095390c32
[02:53:35] <GothAlice> Streemo: A ha: http://stackoverflow.com/questions/15041128/meteor-0-5-7-how-to-handle-use-meteor-collection-objectid — new Meteor.Collection('posts', {idGeneration: 'MONGO'})
[02:54:16] <GothAlice> http://docs.meteor.com/#mongo_collection
[02:54:16] <Streemo> o-o
[02:54:20] <Streemo> LEMME TRY IT!!!!!!
[02:54:22] <Streemo> one sec
[02:55:08] <GothAlice> Note that what this documentation specifies ("random Mongo.ObjectID values") is also wholly incorrect. ObjectIds are anything but random.
[02:55:40] <Streemo> some super ultra secret mathematical algorithm?
[02:55:52] <Streemo> which for all effective purposes for the layman == random?
[02:55:57] <GothAlice> http://docs.mongodb.org/manual/reference/object-id/
[02:55:58] <GothAlice> Nope.
[02:56:04] <Streemo> btw
[02:56:08] <GothAlice> Saves having an extra creation timestamp field for the laymen.
[02:56:08] <Streemo> how can it be random?
[02:56:15] <Streemo> they have
[02:56:18] <Streemo> 90% overlap
[02:56:23] <Streemo> or more jsut by looking at it
[02:56:30] <Streemo> asdfghjK vs asdfghjL
[02:56:44] <Streemo> thats some super rigged random
[02:56:48] <Streemo> :>
[02:56:59] <GothAlice> Also why I stated that the Meteor documentation is false.
[02:57:10] <Streemo> and yeah i read that when i was reading the mongo docs
[02:57:13] <Streemo> that is wrong indeed
[02:57:59] <Streemo> its funny though that they are said to be random
[03:00:06] <Streemo> how many hex chars are there, 16? (1/16) ^ what, 24 characters? thats superrrrrrrr improb. to have say 22 / 24 to be the same
[03:00:31] <GothAlice> Streemo: No legitimate source anywhere on the planet will call an ObjectId random.
[03:01:47] <Streemo> ahhh
[03:01:51] <Streemo> theyre not calling it random
[03:01:54] <Streemo> "Create a Mongo-style ObjectID. If you don't specify a hexString, the ObjectID will generated randomly (not using MongoDB's ID construction rules)."
[03:02:10] <GothAlice> What?!
[03:02:14] <Streemo> theyre noting the order to mongo's generation method
[03:02:25] <Streemo> and saying they do it differently
[03:02:29] <GothAlice> "If you *don't* specify a hexString"
[03:02:39] <Streemo> wait im gonna tyr this, that sseems kinda sketch
[03:02:50] <GothAlice> Nothing about that particular quote of yours makes any sense whatsoever.
[03:03:30] <GothAlice> Truly byte-random ObjectIds can cause obscure replication failures.
[03:03:55] <Streemo> yeah isee the problem, im wondering why they choose that
[03:03:59] <GothAlice> (And will likely be rejected by mongod if object validation is enabled.)
[03:04:53] <GothAlice> What exactly does the hexString you *must* specify represent?
[03:06:57] <Streemo> damn, they really are random
[03:06:59] <GothAlice> Streemo: Meteor Considered Harmful™ — breaks expected usage and data validation standards conformed to by every other driver/wrapper I have ever seen for every language ever.
[03:07:50] <GothAlice> I like to reinvent the wheel; but at least my wheels still conform to the expectation of being round.
[03:07:54] <GothAlice> ;P
[03:09:09] <nateritter> get only records where a float field is a positive number?
[03:09:19] <GothAlice> nateritter: Including zero?
[03:09:33] <nateritter> zeros don’t matter in this case…
[03:09:36] <nateritter> so either way
[03:09:38] <GothAlice> nateritter: db.collection.find({someFloatField: {$gt: 0}})
[03:09:57] <nateritter> Awesome. Thanks GothAlice
[03:09:58] <GothAlice> $gte, actually.
[03:10:05] <GothAlice> 'Cause 0.1 is a positive number. XD
[03:10:21] <GothAlice> Ugh; sorry. I clearly work with too many split decimal fields.
[03:10:28] <GothAlice> Doesn't matter in your case.
[03:10:35] <nateritter> GothAlice: No worries. Thanks for the tip. I appreciate it.
[03:10:44] <Streemo> 0.000...1 where the limit of the number of zeroes is taken to go to infinty
[03:10:56] <Streemo> == 0
[03:10:58] <GothAlice> Yeah. And that's why you don't store financial values as floats.
[03:11:07] <lpghatguy> ^
[03:11:27] <nateritter> GothAlice: So, what do you store them as?
[03:11:30] <GothAlice> 0.1 + 0.1 (ten times total) = 0.9(rep) Which mathematically is exactly 1, but not to a computer…
[03:11:32] <Streemo> pennies
[03:11:55] <lpghatguy> Most RDMS I've used implement a decimal type. Otherwise, using two integer fields (one for the integer part and one for the decimal part) is okay
[03:12:01] <Streemo> sigh, discretism.
[03:12:05] <GothAlice> nateritter: I store them as carefully formatted strings with some adapter glue to translate seamlessly to Python's Decimal type while still supporting querying.
[03:12:21] <GothAlice> Streemo: Pennies are not accurate enough. ;)
[03:12:30] <lpghatguy> that's not a bad idea
[03:12:35] <lpghatguy> Store your currency as a multiple of the smallest unit
[03:13:13] <GothAlice> https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/fields.py#L279-L307
[03:13:19] <GothAlice> I also need explicit rounding control.
[03:14:14] <nateritter> kinda lame currency is still something we have to deal with on a regular basis
[03:14:26] <Streemo> yah everything should be free
[03:14:31] <GothAlice> Heh; this world isn't Star Trek yet.
[03:14:44] <nateritter> haha..
[03:15:10] <lpghatguy> Handling Bitcoin is what's silly since it divides so far
[03:16:16] <GothAlice> lpghatguy: Pro tip: Store the whole part of a BTC value as an integer, the decimal part as a string. ;)
[03:17:37] <lpghatguy> sounds like it'd be a pain to do numeric operations on it without a proper decimal implementation in the end
[03:18:01] <GothAlice> lpghatguy: There are some tricks and shortcuts you can use with a split representation like that. Does come down to regexen a bit, though.
[03:18:38] <GothAlice> (But the integer whole portion can be use for to limit the number of strings needing to be matched.)
[03:18:42] <nateritter> GothAlice: Why the decimal part as a string? Oh, the leading zeros… interesting
[03:18:51] <GothAlice> nateritter: Exactly.
[03:18:53] <lpghatguy> ah, damn, forgot about those
[03:18:56] <GothAlice> nateritter: Sorts nicely, too.
[03:19:04] <nateritter> GothAlice: good point
[03:19:41] <GothAlice> :) Strings. The universal arbitrary precision math format.
[03:20:07] <nateritter> haha
[03:20:25] <nateritter> GothAlice: that needs to be on a t-shirt… #nerdcred
[03:20:25] <GothAlice> (Python's Decimal type is string-based, internally.)
[03:21:11] <lpghatguy> All the recommendations I see just say to use cents as your currency unit instead of dollars
[03:21:39] <GothAlice> More like #hipstercode over #nerdcred. XD
[03:22:19] <GothAlice> lpghatguy: That works if you don't need to deal with smaller fractions. Just gets confusing if you start mixing cents and other subdivisions.
[03:23:25] <nateritter> GothAlice: So, back to my original question, I have this… https://gist.github.com/nateritter/7c4d025a7f30280376a4 … and I’d like to add the condition that we only aggregate those values that are greater than or equal to zero… how would I modify this?
[03:23:53] <nateritter> GothAlice: Sorry for the newb question. Took me forever to get to this point.
[03:24:16] <GothAlice> Uhm… don't try to stuff absolutely everything into a $group… that way lies madness and unexpectedly failing queries.
[03:24:37] <GothAlice> $match to select only those accounts with an amount > 0.
[03:24:38] <nateritter> heh.. ok… what’s a better route then?
[03:24:44] <GothAlice> *then* $group
[03:25:22] <GothAlice> Rather, $match on the account you want *and* amount > 0.
[03:25:40] <GothAlice> It'll also run a lot faster as the first $match in an aggregate can make use of indexes.
[03:25:46] <Boomtime> with aggregation, unless the first operation is $match the rest of the pipeline has to trawl the entire database
[03:25:58] <nateritter> good points. thank you guys
[03:26:39] <GothAlice> nateritter: This query ($match on .account and .amount) would benefit most strongly on a compound [account, amount] index. :)
[03:28:08] <Streemo> GothAlice i looked at the meteor/issues/61 thing. its said that they want to focus less on mongodb and make meteor db agnostic
[03:28:23] <Streemo> but this inconsistency is kind of a pain
[03:28:27] <GothAlice> Streemo: Added to my list of packages never to use. XD
[03:28:41] <Streemo> what would you suggest other wiseR?
[03:28:55] <Streemo> BTW meteor handles their own id generation
[03:28:56] <GothAlice> Alas, I don't Node, so I can't really help in selecting anything better.
[03:29:01] <Streemo> in strings
[03:29:25] <GothAlice> Streemo: What upsets me about that is that even if they did absolutely nothing, they'd get ObjectIds back automatically.
[03:29:35] <lpghatguy> Mongoose is pretty much the go-to MongoDB interface
[03:29:46] <GothAlice> T_T
[03:29:56] <Streemo> well i am not sure if minimongo supports object IDs
[03:30:32] <lpghatguy> (if what I said didn't make sense just ignore me, I'm not sure I'm quite up-to-speed in this conversation)
[03:30:57] <GothAlice> lpghatguy: It's more that what you said saddened me. :P
[03:31:30] <lpghatguy> GothAlice, ?
[03:31:48] <GothAlice> That it'd be a defacto standard, and yet so avoiding MongoDB conventions.
[03:32:43] <lpghatguy> It looked to be the official standard judging by the mongodb docs when I was getting started with node and MongoDB
[03:35:24] <Streemo> lol... the importance of convention...
[03:35:33] <Streemo> thats a thing
[03:37:31] <Streemo> GothAlice, I am just gona stick with their string IDs
[03:38:56] <Streemo> is there any reason why stringIds in mongodb is fatal?
[03:39:31] <GothAlice> Fatal, no.
[03:39:39] <Streemo> inconvenient o-o
[03:39:46] <GothAlice> You'll effectively be locked into only ever using the layer to interact with your DB, at least in terms of inserting records or risk mixing and matching which won't work as expected.
[03:39:58] <Boomtime> you can use anything for a _id
[03:40:00] <GothAlice> You lose useful diagnostic information.
[03:40:12] <GothAlice> If you care about record creation times, you suddenly need to add an extra field for that.
[03:41:09] <Streemo> OK. lets make it one step better, let's say I use actualy mongodb objectId's using the {idGeneration: 'mongo'} (ofc theyll have no meaning other than being a unique id, cause they lack the generation order)
[03:41:26] <Streemo> that still wouldnt fix the time problem
[03:41:44] <cheeser> they *do* have generation order, actually.
[03:41:53] <cheeser> one component of an ObjectId is the time
[03:41:56] <Streemo> no they dont
[03:42:02] <Streemo> in Meteor ?
[03:42:07] <cheeser> in mongodb
[03:42:08] <GothAlice> cheeser: Meteor throws the standard out.
[03:42:19] <Streemo> were tlaking about osmething else
[03:42:20] <cheeser> well, fuck meteor then. don't use broken crap.
[03:42:27] <Streemo> its so stupiddddddd
[03:42:34] <GothAlice> cheeser: My opinion; I was having difficulty expressing that politely. ;)
[03:42:40] <cheeser> :)
[03:42:47] <Streemo> you dont need to be polite
[03:42:51] <cheeser> it's too late in the evening for subtlety
[03:42:52] <Streemo> it is over rated.
[03:43:29] <lpghatguy> "don't use broken crap" is great advice in general
[03:43:38] <GothAlice> Streemo: Comes down to "I'd recommend a dead equine over that framework", really.
[03:43:49] <cheeser> i'd beat that!
[03:43:52] <GothAlice> :)
[03:45:26] <Streemo> geez, just say horse. why so obscure
[03:46:18] <GothAlice> darkblue_b: If you're just catching up, we discovered to our joy that Meteor generates truly random ObjectIds instead of following the spec, and defaults to random strings for _ids. That's weird.
[03:46:43] <Boomtime> erk
[03:46:50] <Boomtime> really?
[03:46:52] <darkblue_b> I got distracted by string search indexes and havent been moving fwd on mongodb
[03:46:56] <GothAlice> It seems so, Boomtime.
[03:47:00] <Boomtime> :(
[03:47:05] <lpghatguy> does Mongoose do that too or am I imagining things?
[03:47:18] <darkblue_b> but, still considering a two-machine demo
[03:47:56] <GothAlice> http://mongoosejs.com/docs/api.html#schema-objectid-js — doesn't mention "random", which is a good sign.
[03:48:27] <Streemo> GothAlice what's bad about a random objectID other than the time stamping?
[03:48:59] <GothAlice> It breaks insert-order sorting.
[03:49:26] <Streemo> like, where to insert an element relative to others?
[03:49:39] <Boomtime> Mongoose usually just uses the Node.js driver when possible right?
[03:49:46] <lpghatguy> fairly sure it does
[03:49:55] <lpghatguy> either that or I have both installed in my project for no reason
[03:50:12] <GothAlice> Streemo: More to do with retrieval. Natural order in non-capped collections != insert order, AFIK.
[03:50:34] <GothAlice> Streemo: For sorting.
[03:51:17] <GothAlice> Looks like Mongoose does the Right Thing™.
[03:51:21] <lpghatguy> yay
[03:51:28] <lpghatguy> Thank god, I like using it
[03:52:32] <darkblue_b> π
[03:52:58] <Streemo> you ever heard of the feynman point in pi?
[03:53:22] <lpghatguy> the bunch of 9s?
[03:53:27] <Streemo> ya
[03:54:31] <Streemo> he said he wanted to memorize pi up to that place (maybe in the 700's if i remember right?) to recite it to people and trick them into thinking its rational
[03:54:49] <Streemo> 99999...
[03:54:57] <lpghatguy> "999999 and so on"
[03:55:02] <Streemo> yea
[03:55:36] <GothAlice> 𝛕
[03:56:38] <GothAlice> τ > π But let's not digress from the topic at hand. :3
[03:57:04] <lpghatguy> just as much as 2 > 1
[03:57:43] <Streemo> dont get GothAlice started on tau
[03:57:49] <Streemo> .-.
[04:08:49] <Streemo> GothAlice, as usual thanks for help. :P
[04:08:57] <GothAlice> No worries. :(
[04:08:59] <GothAlice> :) I mean.
[04:09:01] <GothAlice> Darn keys!
[04:10:03] <Streemo> :( maybe you're sad that I'm still using meteor :>
[04:10:06] <GothAlice> XD rbrt jsbr pmr pg yjpdr fsud (all keys translated one to the right)
[04:10:13] <Streemo> -_-
[04:10:19] <Streemo> EVER
[04:10:38] <Streemo> eh maybe ill jsut write a program to do it
[04:11:12] <joannac> ever have one of those days
[04:11:19] <Streemo> hmm
[04:11:59] <GothAlice> Law 57: If something is worth doing twice, it's worth building a tool to do it.
[04:12:15] <joannac> depends how long it takes to do twice :p
[04:12:21] <Streemo> do you actually have >=57 laws?
[04:12:21] <GothAlice> XP
[04:12:34] <GothAlice> An ironic 127 at current count, actually.
[04:12:43] <Streemo> and you remember alo f them
[04:12:53] <Streemo> all of
[04:13:16] <GothAlice> #101: That which does not kill you has made a tactical error.
[04:13:28] <Streemo> shesss a maaaaddddwomann
[04:14:05] <Streemo> alright? QUICK WHATS 104
[04:14:18] <GothAlice> Just because it's easy for you doesn't mean it can't be hard on your clients.
[04:14:33] <Streemo> nice
[04:14:35] <GothAlice> (You can see why I like giving support. ;)
[04:14:38] <Streemo> you know
[04:15:01] <Streemo> even if yo udidnt remember all 127 if you were able to conjure 127 slightly profound statements on the fly id be impressed
[04:15:10] <GothAlice> https://gist.github.com/amcgregor/9f5c0e7b30b7fc042d81 — Collected from various sources; avoided skippy's list references with extreme difficulty. :P
[04:15:29] <GothAlice> Not very well organized, but there are several consistent themes. I need to tag them individually or something. ¬_¬
[04:15:37] <Streemo> whatttt why are the on git hub!?
[04:15:53] <GothAlice> Streemo: https://gist.github.com/amcgregor/9f5c0e7b30b7fc042d81/revisions < Revision tracking, of course.
[04:15:53] <Streemo> ohhh ok its secret
[04:16:12] <GothAlice> I edit gists locally in checked out repos.
[04:16:40] <Streemo> 118
[04:17:18] <Streemo> this is cray i used to come up with things like this, didnt know that was a common thing.
[04:17:26] <Streemo> did you think of these xD?
[04:18:07] <GothAlice> A few of them, not many. Zen of Python is included wholesale, plus my common addendum. Various common project management phrases, plus a bunch of "Milspec+" quotes.
[04:19:11] <Streemo> 64
[04:19:14] <Streemo> haha nice
[04:19:29] <GothAlice> That one's mine. :3
[04:19:51] <Streemo> yepp i can see the python ones too
[04:19:55] <Streemo> 40
[04:20:58] <Streemo> gotta say you're weirder than me and i didnt know that was possible
[04:21:57] <GothAlice> Styled after Lefler's 102 Laws to Live By from ST:TNG episode The Game: http://en.memory-alpha.org/wiki/Robin_Lefler
[04:21:58] <GothAlice> :P
[04:22:11] <GothAlice> I had weird role models growing up. ¬_¬
[04:23:34] <Streemo> weird is what drives the planet
[04:23:52] <Streemo> id be flattered
[04:23:55] <Streemo> if that was 128
[04:23:59] <Streemo> ;)
[04:25:16] <GothAlice> Streemo: I kicked that up a notch.
[04:25:30] <Streemo> YESSSSSSSSSSSSSS
[04:25:33] <Streemo> brilliant
[04:26:12] <GothAlice> Anyways. MongoDB! :P
[04:26:37] <Streemo> indeed
[04:26:49] <Streemo> well, back to my things.
[04:30:17] <darkblue_b> is there a "string distance" routine in mongodb somewhere? take two strings and return dissimlarity by some measure..
[04:30:56] <GothAlice> Not AFIK, darkblue_b.
[04:31:20] <GothAlice> If you need something like that, you can pre-store soundex data (and such) numerical representations of the text for easier in-database comparison.
[04:49:11] <Streemo> thats an interesting question darkblue_b i wonder if there are ways to compute a "norm" of a string via some measure - you could probably define one. maybe even vectorize them and compute inner products for 'dissimilarity' measure.
[04:49:56] <darkblue_b> .. oh more than that.. thats how "datamining" is done.. there are many specific variations on that..
[04:50:46] <darkblue_b> I am just trying some basics on equipment I have, to get a sense of the whole
[04:50:52] <Streemo> i never knew what datamining referred to, but im sure the math behind it isnt too bad
[04:51:14] <Streemo> yeah itd be nice for a string distance thing. Write a basic one
[04:58:49] <nateritter> Where should I put a config variable that I want accessible only on the server side, and how would I access it?
[05:15:05] <Boomtime> nateritter: not sure what you mean, can you provide an intended usage of such a variable?
[05:15:45] <nateritter> no worries., figured it out…
[05:16:01] <nateritter> Boomtime: spent too much time on PHP frameworks… this JS stuff is too easy… haha
[08:41:00] <Yoda-BZH> Hi
[08:42:25] <Yoda-BZH> I'm looking for the necessary roles to monitor a mongodb instance
[08:43:03] <Yoda-BZH> I suppose: readAnyDatabase, dbAdminAnyDatabase and clusterAdmin
[08:43:19] <Yoda-BZH> aren't they too open ? or am I missing a role ?
[08:49:14] <joannac> Yoda-BZH: MMS monitoring, or your own monitoring?
[08:49:35] <Yoda-BZH> own monitoring
[08:49:53] <Yoda-BZH> well, access to the web interface
[08:49:59] <joannac> decide what the user needs to be able to read / run
[08:51:32] <joannac> access to the web interface?
[08:51:38] <joannac> do you mean the REST interface?
[08:53:29] <Yoda-BZH> on the port 28017
[08:53:43] <joannac> yes
[08:53:49] <joannac> http://docs.mongodb.org/manual/core/security-interface/#rest-api
[08:54:26] <joannac> sorry, http://docs.mongodb.org/manual/core/security-interface/#http-status-interface
[08:55:44] <Yoda-BZH> thanks, but the necessary roles aren't clearly stated :s
[08:57:09] <joannac> there are no necessary roles
[08:57:26] <joannac> you do not need to authenticate to use the HTTP interface
[09:02:43] <joannac> Yoda-BZH: I have no idea if that answered your question or not
[09:23:50] <Yoda-BZH> hmm
[09:24:18] <Yoda-BZH> I miss-thought that a particular user with the rights roles could access the monitoring web interface
[09:24:33] <Yoda-BZH> not a lambda user
[10:01:16] <Luser> Hi guys. I am currently working on ruby script and woud like to search through the database for one phrase, but am not sure how to do that. I've seen that method: session.with(database: "db_name").command() but am not sure what to insert into command
[10:01:44] <Luser> http://mongoid.org/en/moped/docs/driver.html i am referencing to that
[11:54:16] <yopp> hey
[11:55:38] <yopp> I have ancestry based tree collection. What the best way to check if there subchildrens of current tree leaf?
[11:56:07] <yopp> Like: a; a/b; a/b/c; a/d, a/e; a/f; a/f/e
[11:56:31] <yopp> if I'm querying a, I need to get a/b and a/f only
[13:24:39] <Luser> Is anyone here?
[14:02:12] <GothAlice> yopp: Tree structures are one of those things that MongoDB does strangely. (Then again, nested set adjacency lists in relational databases are kinda nuts, too.)
[14:02:18] <GothAlice> yopp: Let me dig up a link for you.
[14:02:47] <GothAlice> yopp: https://github.com/marrow/contentment/blob/develop/web/extras/contentment/components/asset/model/__init__.py#L58-L63
[14:04:05] <GothAlice> Two ways of doing "descendants" searching, by string: https://github.com/marrow/contentment/blob/develop/web/extras/contentment/components/asset/model/__init__.py#L269 or by parents list membership. (I.e. asset X is in the parents list of all children at all levels below asset X)
[14:05:20] <GothAlice> "children" is a list of child ObjectIds (to preserve order) whereas "contents" is your "single-level of children below X" query.
[14:08:41] <yopp> GothAlice, in fact I need to make a aggregation operaion for regexp. For example, in this case for $1 in ^(/a\/.*)\/(:?[^\/]+)$
[14:09:05] <yopp> but I'm trying with map/reduce right now, it might be a bit faster
[14:09:50] <yopp> But I'm not sure how to mapreduce on index only, because I don't need a full document
[14:10:09] <cheeser> m/r will almost certainly not be faster
[14:10:32] <GothAlice> yopp: Well, you didn't give an example document in your structure. Storage of coalesced string paths is unusual and inefficient; my Asset document class uses it primarily for direct lookup. (I.e. given a URL like /foo/bar look up /, /foo, and /foo/bar in reverse order, take the first, most specific element to use as the handler.)
[14:13:00] <GothAlice> yopp: In the case where ^/a/.* is your query, I'd recommend the parents list approach.
[14:13:10] <yopp> nope
[14:13:11] <GothAlice> Then it's "object ID X in list" query.
[14:13:24] <yopp> on the root nodes it will be inefficent.
[14:14:00] <yopp> at least ^/a/.*/[^/]+/
[14:14:35] <yopp> but still, if we have 100 root nodes, and each node has like 100 childrens, we will have 10k nodes in result.
[14:14:41] <GothAlice> yopp: So your paths aren't even normalized. (May contain an unlimited nubmer of additional empty separators.)
[14:15:13] <yopp> what?
[14:15:15] <GothAlice> yopp: The "complex" structure I linked you currently contains 24TiB of data in my MongoDB cluster at home. ;) It's quite efficient.
[14:17:16] <GothAlice> yopp: ^/a/.*/[^/]+/ -> [start of string] "/a/" [one or more of any character including slash] "/" [one or more character that isn't slash] "/" -> /a///////////////27/ is valid.
[14:17:23] <GothAlice> ;P
[14:17:36] <yopp> what?
[14:17:52] <yopp> have you ever heard about quantification rules?
[14:18:38] <yopp> http://rubular.com/r/j5FCkjuhNG
[14:19:27] <yopp> in my case path name is normalized, and there no way that path will have more than one consequential slash
[14:19:48] <yopp> so even so, ^a\/[^\/]\/[^\/]+ will be enough
[14:20:03] <GothAlice> yopp: I'm suggesting that such a method of storage is inefficient and unnessicariy in a system that provides rich list types. What does that string represent, exactly?
[14:20:16] <GothAlice> yopp: That string represents a series of parent element references in the form of a path.
[14:20:59] <yopp> Anyway, there no way to change structure of document
[14:21:15] <yopp> If this will make things simplier for you. I have what I have :)
[14:23:21] <GothAlice> yopp: It makes things more complex, difficult, inefficient, and painful, actually. It is the wrong representation for the data based on how you wish to query it. (Which is what MongoDB *actually* cares about.) Unfortunately I need to transition networks in a moment. Be back in a bit.
[14:26:10] <yopp> sure
[16:06:53] <ryanwmedlin> hey, how in the heck do you do group by when dealign with timezones
[16:07:07] <ryanwmedlin> i do nto see a convert_utc fiunction similar to SQL
[16:07:19] <ryanwmedlin> do you do somethign in an aggregation pipeline to solve for this?
[16:07:20] <GothAlice> ryanwmedlin: Always, always, always, always store your date/time objects in canonical UTC form.
[16:07:35] <ryanwmedlin> we do
[16:07:47] <ryanwmedlin> but lets say a user is on west coast
[16:07:57] <ryanwmedlin> and want to see data grouped by day
[16:08:01] <ryanwmedlin> for their timezone
[16:08:23] <GothAlice> You'll have to work out the translation when building the query.
[16:08:45] <ryanwmedlin> yeah thats what i am asking for help with
[16:08:52] <ryanwmedlin> translate the group by how?
[16:08:54] <GothAlice> No, I mean in your host application.
[16:09:15] <ryanwmedlin> run the dat field through an aggregation pipeline to convert the date fielkd to that timezone then group on that?
[16:09:28] <GothAlice> ryanwmedlin: No… see: https://gist.github.com/amcgregor/1ca13e5a74b2ac318017
[16:09:38] <ryanwmedlin> in sql you run it througha convertUTC function as part of the query and do a group by day on that
[16:10:14] <GothAlice> ryanwmedlin: MongoDB doesn't provide functions like that. Instead of passing in localtime references to the query and hoping MongoDB has a way to make those UTC, *make those values UTC in your code*, not MongoDB.
[16:10:37] <GothAlice> I.e. never pass anything to MongoDB that isn't in UTC.
[16:11:19] <GothAlice> ryanwmedlin: Note from my link that I needed to work with ISO DOTW, but MongoDB doesn't use the standard ISO representation, so I converted them *outside* MongoDB.
[16:11:25] <ryanwmedlin> so get the raw data then group by on the server for their timezone?
[16:13:12] <GothAlice> ryanwmedlin: Show me (pastebin) your current query.
[16:16:00] <GothAlice> ryanwmedlin: (P-code is good too; it doesn't have to work, I just need a better picture of exactly what you're trying to do. SQL would be good, too, if you only have that.)
[16:19:29] <GothAlice> http://docs.mongodb.org/manual/reference/operator/aggregation-date/ are the sum total of currently available operations on dates.
[16:19:52] <GothAlice> (Timezones are effectively unsupported at this level.)
[16:21:24] <ryanwmedlin> so if your ont he west coast.. want ot group by day by using dayOfMonth.. your basically getting 7 hours of data you dont want? because mongo will group by the UTC data in the db?
[16:21:37] <ryanwmedlin> im trying to pull a query now thanks for the replies btw
[16:29:32] <niczak> This may sound crazy, but what if I make a secondary replica set the primary w/o running the touch command? Will things just load slowly at first until they find their way into memory or will there be other side effects?
[16:31:17] <GothAlice> niczak: Any time you swap primaries there is the possibility of completely rebuilding the new secondaries (i.e. having each pull a complete copy of the dataset from the new primary). This can hose your network a bit while things re-settle.
[16:31:48] <niczak> Goth: Ahh, okay, defintiely want to avoid that. So running the touch command will ensure that everything is "warmed up" and ready to go?
[16:31:51] <GothAlice> (Usually only happens if the oplog becomes so out-of-sync that there is no commonality between them.)
[16:31:58] <niczak> So I can do the touch stuff, then swap primaries?
[16:54:38] <niczak> Dataset can definitely fit into RAM but I am thinking about letting the application warm up the database by just running it's most typical queries.
[17:11:30] <joaovagner> Hi guys
[17:12:03] <joaovagner> https://gist.github.com/JoaoVagner/41b13ba09328735c7e0b how I can perform a search using the dates between this array?
[18:07:26] <GothAlice> joaovagner: db.collection.find({date: {$gt: someDate, $lt: someOtherDate}})
[18:08:09] <GothAlice> Where "date" is the name of the field. If this is embedded under "ticket_answers" the solution becomes: db.collection.find({"ticket_answers.date": {$gt: someDate, $lt: someOtherDate}})
[18:09:21] <joaovagner> GothAlice: https://gist.github.com/JoaoVagner/8e538242ddb018f445dd
[18:10:36] <joaovagner> however it still fails, I do not know how to turn that into something that your query PHP understands. :(
[18:10:42] <GothAlice> joaovagner: Oh. If you only want the nested records that match back (i.e. not the whole top-level document) you're going to have to turn that into an aggregate query with $match, $unwind, and $match stages.
[18:11:17] <GothAlice> Unfortunately I don't PHP. (And feel nobody else should either.) So I can't really help you with the specific syntax.
[18:11:24] <GothAlice> :(
[18:11:38] <joaovagner> GothAlice: hummm I understand, thanks for the clarification
[18:11:52] <joaovagner> :)
[18:12:05] <GothAlice> (Not being able to help with the syntax comes down to not knowing the syntax. Personal preference is just a nice bonus. ;)
[18:13:18] <GothAlice> I'm sure you've seen http://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ by now. :3 Back in the day I ran into some hilarious issues with the str_parse function being exactly the same as register globals. ^_^;
[18:14:11] <GothAlice> (Oh, and that was parse_str, not str_parse, illustrating one of the many points from that article. ;^)
[18:14:17] <cheeser> such a great post
[18:14:24] <GothAlice> Yeah. ^__^
[18:14:35] <cheeser> we use it in ##java whenever someone starts nattering on about php
[18:15:11] <GothAlice> I'm freely willing to admit the JVM is a work of art. The rest of the language, though… less so. ;^P
[18:15:34] <cheeser> i like the language, actually, though I flirt with other JVM languages fairly regularly.
[18:15:44] <cheeser> i always come home to java at night, though. ;)
[18:16:37] <joaovagner> GothAlice: PHP-FPM and PHP-NG has brought great advances in language that is adopted today the largest in the world :)
[18:17:26] <joaovagner> hhvm (facebook), tested in a client's ecommerce stores, and unfortunately it is far away from reality for a good developer.
[18:17:38] <mike_edmr> largest on the world wide web maybe
[18:18:41] <GothAlice> http://hippyvm.com
[18:19:09] <mike_edmr> php was the first/best at some things and has since grown large but without fixing the issues that make it a poor basis for what it is today
[18:19:43] <cheeser> i don't know that it was the first/best at anything really
[18:19:46] <GothAlice> I <3 Python so much in that it can produce better interpreters for other languages than the other languages' native runtimes.
[18:19:51] <joaovagner> mike_edmr: at all scales. PHP for command line tools composes, (dependencie manager), web, and other things. Despite being in love with Python, who pays the bills is PHP :)
[18:20:00] <GothAlice> PHP took the C global namespace and exposed it through a template language. That's it, and really, all PHP is.
[18:20:11] <cheeser> funny. that's how I feel about python. :)
[18:20:12] <mike_edmr> php shouldnt be used for new projects
[18:20:14] <GothAlice> joaovagner: Yeesh, you use PHP for command-line scripts?!
[18:20:27] <mike_edmr> its "legacy" in my book
[18:20:53] <GothAlice> cheeser: from concurrent.futures import ThreadPoolExecutor; with ThreadPoolExecutor(10) as e: e.submit(print, "Hello world!") — BAM, multithreading.
[18:20:56] <joaovagner> Você quis dizer: O PHP para o cliente fecha a conta de custo x benefícios.
[18:20:56] <joaovagner> PHP for the customer closes the account of cost vs. benefits.
[18:20:57] <darkblue_b> I am a senior editor for a linux "distribution" with over 50 FOSS projects.. php is used in a small number, and that number is getting smaller
[18:21:28] <darkblue_b> several language groups are strong, very much including python
[18:21:41] <joaovagner> GothAlice: php-cli yes, including those famous crontab tasks ... I have great applications integrating with the API ingresse.com site and Amazon Web Services
[18:21:45] <GothAlice> (Replace ThreadPoolExecutor with ProcessPoolExecutor and you have split-process deferred execution.)
[18:22:34] <joaovagner> GothAlice: The consultancy work which manages the Route53 all customers via panel with AWS;
[18:23:06] <joaovagner> However we have applications in other languages ​​like java / python doing work BigData.
[18:23:11] <GothAlice> joaovagner: Let me introduce you to Boto… ;)
[18:23:21] <darkblue_b> work BigData ++
[18:23:45] <joaovagner> darkblue_b: sciencie date is the craziest thing I've ever seen.
[18:23:48] <joaovagner> heheheh ;)
[18:23:54] <GothAlice> joaovagner: http://docs.pythonboto.org/en/latest/
[18:24:20] <GothAlice> (This library is how we automate the AWS infrastructure we use.)
[18:24:43] <joaovagner> GothAlice: The biggest problem here in Brazil is keeping these things, because the enormous difficulty of keeping a team in Python. The canonical came and stole our best :(
[18:25:13] <GothAlice> I have the benefit of living in the Python capitol of Canada.
[18:25:21] <joaovagner> PHP community in Brazil is great and is not hard to keep the project with a team keeping. There is no difficulty in keeping projects in PHP due to cost x benefit. In python costs outweigh benefits.
[18:25:26] <cheeser> Regina? :D
[18:25:33] <GothAlice> cheeser: Montréal
[18:25:41] <cheeser> ah. i was close.
[18:25:59] <GothAlice> (Last PyCon was here, next PyCon will be here, prior to that several years straight of Confoo, etc., etc. Quite the community here.)
[18:26:10] <joaovagner> cheeser: Regina's own name in the Portuguese language
[18:26:12] <cheeser> i was at ConFoo last year
[18:26:13] <joaovagner> hahahahahahhaha
[18:26:26] <GothAlice> cheeser: Darn; I presented the two years prior. ;)
[18:26:36] <joaovagner> I have an aunt named Regina
[18:26:43] <stefandxm> Regina Lund
[18:26:43] <cheeser> a terrible time to visit Montreal
[18:26:56] <cheeser> a moose once bit my sister
[18:27:15] <GothAlice> cheeser: Confoo is almost always cross-scheduled with inclement weather. It's terrible. XD
[18:27:19] <joaovagner> hahahahah
[18:28:48] <GothAlice> cheeser: “I don’t trust your coffee. Had the last batch sent off to a lab and the results came back: congratulations, your moose is pregnant.”
[18:28:55] <joaovagner> Here in Brazil, the summer heat had a hug :D
[18:30:08] <GothAlice> joaovagner: Lived in Panama for a while; it's not the heat that gets me, it's the constant 95% humidity.
[18:31:28] <joaovagner> GothAlice: it's hard to live like this hehehehehe
[18:31:45] <GothAlice> Ciesta makes up for it. ;^P
[18:32:39] <joaovagner> kkkkkkkkk ciesta who did was my grandfather, of Italian descent. I always thought it a lazy!
[18:33:33] <GothAlice> Science agrees that a nap in the afternoon can actually increase productivity through improved alertness and problem solving abilities. Recharging is good for you.
[18:36:12] <joaovagner> GothAlice: I awake to 05am, I'll run and go to the gym, according to my wife, I take the kids to school, it all before 09am. Most people who know the awakened 09am. I'll have plenty of time to rest when you die. hhahahhahaha
[18:36:45] <GothAlice> That didn't translate well, but I think I got the gist of it. ;)
[18:37:32] <joaovagner> GothAlice: speak spanish?
[18:38:05] <joaovagner> is a language closest to Portuguese <my natural language)
[18:38:06] <unholycrab> heavy write load on the primary. one secondary gets replication lagged, the other secondary doesn't
[18:38:07] <GothAlice> No hablo español, amigo. ;)
[18:38:09] <unholycrab> what?
[18:38:31] <GothAlice> unholycrab: Is there any difference in the hardware the two secondaries are running on?
[18:39:16] <joaovagner> unholycrab: date and time in yours servers, or your router not are not sharing the load
[18:39:39] <GothAlice> unholycrab: There are all sorts of reasons lag might affect only one of the pair.
[18:39:52] <joaovagner> Check communications within the machines, they are getting access and permission required.
[18:40:14] <GothAlice> unholycrab: Make sure ntpd is running on all machines and that they are properly synchronized, time-wise.
[18:40:16] <joaovagner> unholycrab: and check your mongo log :)
[18:40:25] <unholycrab> cool
[18:40:27] <GothAlice> :)
[18:40:41] <unholycrab> well, the one was only falling behind while there was heavy write load to the primary
[18:40:43] <joaovagner> the log most of the time, contains details of the problems
[18:40:46] <unholycrab> it caught up almost immeidately after
[18:41:50] <joaovagner> GothAlice: how long works with Mongo?
[18:42:05] <GothAlice> joaovagner: I've been using it for a number of years now.
[18:42:58] <GothAlice> joaovagner: amcgregor@chomolungma ~ $ uptime # 11:41:43 up 956 days < maybe a year or two longer than that (this is the uptime from the primary of my larger cluster)
[18:43:47] <joaovagner> Nice!
[18:44:39] <joaovagner> I wrote only three applications with Mongo, taking the experience yet. And breaking my head :)
[18:45:14] <GothAlice> MongoDB does open your mind to new possibilities for data design, which can be somewhat mind-blowing when you realize the things you can do. :D
[18:47:44] <unholycrab> uptime # 11:41:43 up 956 days
[18:47:47] <joaovagner> GothAlice: indeed, the concept of breaking paradigms NoSQL, did a project a while ago with Java and Neo4j. It's wonderful are old and new concepts while
[18:47:47] <unholycrab> jesus man
[18:47:55] <unholycrab> you're not supposed to do that
[18:48:13] <joaovagner> $: sudo reboot
[18:48:15] <joaovagner> hahahahaha
[18:48:21] <unholycrab> :P
[18:48:22] <xissburg> eh
[18:48:36] <unholycrab> sudo apt-get install kernel upgrade
[18:48:40] <GothAlice> unholycrab: You may be inferring that the kernel has never been upgraded, but it has. Total downtime due to hot upgrades year-to-date of about 15 seconds service level interruption.
[18:48:54] <hexsprite> hey if I want to sort by multiple keys then what's the syntax? for example I want to sort first by "order" and then by "list"... I tried {sort: {order:1, list:1}} but it seems to sort by list first and then order....
[18:49:00] <unholycrab> GothAlice: oh, wow
[18:49:14] <unholycrab> GothAlice: where does the uptime figure come from then?
[18:49:37] <joaovagner> print-screen now kkkk
[18:49:40] <GothAlice> hexsprite: Depending on your language, your dictionary/mapping/object type might not preserve the order of keys, which would mess up your sort.
[18:50:38] <hexsprite> GothAlice: thanks, i just realized this is probably a meteor specific issue because they wrap the mongodb api... asking on #meteor instead ;)
[18:50:44] <GothAlice> unholycrab: Uninterrupted powered operation. In-place kernel updates don't nuke the allocated stats structure.
[18:52:45] <unholycrab> just looked it up. cool GothAlice
[18:53:27] <GothAlice> unholycrab: There are a lot of edge cases to an approach like this, though. Like every counter in the system eventually rolling back around to zero. (Notably network packet and bandwidth counters.)
[18:58:34] <joaovagner> GothAlice: https://gist.github.com/JoaoVagner/65d0dfe3bf799c934a8a solution
[18:58:51] <joaovagner> thanks for the contribution
[18:58:52] <joaovagner> :D
[19:02:51] <GothAlice> joaovagner: It never hurts to help; I'm surprised where_range didn't work.
[19:03:26] <GothAlice> (As I'd expect it to simply combine the where_gte and where_lte clauses.)
[19:10:54] <joaovagner> GothAlice: You said it is the canadar, right?
[19:11:37] <GothAlice> joaovagner: I did not mention calendars, so now I'm confused. XD
[19:12:07] <joaovagner> GothAlice: Country Canada
[19:12:21] <GothAlice> ^_^ Yes, I'm currently in Canada.
[19:12:42] <joaovagner> in portuguese, Canada is "Canadar", sorry :)
[19:13:15] <joaovagner> GothAlice: Tran
[19:13:25] <joaovagner> GothAlice: I saw a blurb about jobs in this region, as is the economic situation?
[19:14:14] <joaovagner> my only fear of accepting a job to go to north america is cold. :(
[19:14:52] <cheeser> depends on the area. Costa Rica is quite warm most of the time. ;)
[19:17:15] <joaovagner> cheeser: Quebec?
[19:18:08] <kali> mmmm... i was in montreal last week, and i'm glad to be back before november happens :)
[19:18:26] <GothAlice> kali: Yeah. It's going to get nasty this winter.
[19:18:40] <kali> winter is coming </stark>
[19:19:01] <GothAlice> Montréal gets the extremes of heat in the summer, cold in the winter. I prefer British Columbia; sure, you get rain like it's foggy old England, but it's never too hot and never too cold.
[19:19:47] <kali> yeah, BC is like canada's california :)
[19:19:57] <GothAlice> And Alberta is Canada's Texas.
[19:19:58] <cheeser> joaovagner: very cold parts of the year
[19:20:20] <cheeser> full of rednecks?
[19:20:30] <kali> cheeser: fly-over state
[19:20:34] <GothAlice> And very flat. With cowboy hats in full prominence.
[19:20:37] <cheeser> i hate that term
[19:21:11] <kali> cheeser: i actually went there, landed in calgary and drive through the rockies to vancouver
[19:21:29] <kali> in august.
[19:21:44] <GothAlice> kali: Nice drive. Jasper park is beautiful, if you went that route.
[19:22:03] <kali> yep. that's the one. the only one, actually :)
[19:22:25] <GothAlice> Erm… I've taken marked highways up there that were dirt paths. You can avoid it if you want to. ;^P
[19:22:56] <kali> GothAlice: yeah, a couple of friend did that. they nearly died :)
[19:23:35] <joaovagner> cheeser: full
[19:23:58] <cheeser> hrm?
[19:24:03] <joaovagner> cheeser: My wife said that the children would not adapt to the cold.
[19:24:13] <cheeser> bah. of course they would!
[19:24:20] <cheeser> *she* might not want to, though. ;)
[19:24:31] <kali> children are resilient, that's an excuse :)
[19:24:47] <joaovagner> hahahaha
[19:25:04] <joaovagner> what is the average salary for a DevOps?
[19:25:16] <kali> where ?
[19:25:35] <joaovagner> yes, in Cadana (quebec)
[19:25:46] <joaovagner> oror other regions of Canada
[19:26:51] <GothAlice> I became so used to the cold just below the arctic circle (Dawson Creek, BC, no relation to Dawsons Creek, the TV show) that at -60°C I was standing outside, barefoot in PJs to smoke. Didn't even notice how cold it was.
[19:26:53] <ehershey> /1
[19:27:00] <joaovagner> Devops Senior, 9 years experience in web (php, javascript, api's rest, etc), and infra-estructure (aws, load balancer's, etc)
[19:27:11] <GothAlice> joaovagner: http://www.payscale.com/research/CA/Job=Web_Developer/Salary/4e4b559e/Montréal-QC and http://www.glassdoor.com/Salaries/montreal-software-developer-salary-SRCH_IL.0,8_IM990_KO9,27.htm
[19:27:32] <GothAlice> (There are so many developers in Montréal it lowers the median.)
[19:28:01] <joaovagner> my country in the recent elections elected a communist government that is destroying our economy. I intend to leave the country for a period escape it. :(
[21:03:37] <bpap> what is the difference between mongodump and mongoexport? more specifically, i want to dump a 7G collection as quickly as possible (consistency not needed). which tool should i use?
[21:04:33] <cheeser> mongodump will output the bson. mongoexport would write to json/tsv/csv
[21:05:08] <bpap> ah, so the only difference is the output format? seems like they could have been combined into one tool, no?
[21:08:05] <niczak> I have a replica set with two nodes which are completely in sync. My goal is to make the current secondary the primary and then shutdown the current primary node because we are retiring that server. Following this guide: http://docs.mongodb.org/manual/tutorial/force-member-to-be-primary/ it not working for me, can someone tell me the best/easiest way to make a secondary the primary and remove any references (or just hide) the original primary?
[21:09:21] <niczak> It seems like this should be a pretty common use case.
[21:13:25] <kali> niczak: just ask the primary to stepDown(), the secondary will get elected primary.
[21:13:31] <kali> niczak: do you have an arbiter ?
[21:24:54] <Trindaz> MySQL is to SQLite as MongoDB is to ___
[21:25:05] <Trindaz> or:
[21:25:07] <Xe> flat file json
[21:25:08] <GothAlice> SQLite
[21:25:23] <GothAlice> Considering that SQLite is also effectively schema-free…
[21:25:37] <Trindaz> How do I write unit tests for code that accesses a MongoDB instance without needing a running MongoDB instance to run the unit tests
[21:25:51] <GothAlice> Trindaz: You run a MongoDB instance for the life of the tests.
[21:27:10] <GothAlice> Trindaz: Consider https://github.com/marrow/marrow.task/blob/develop/.travis/install.sh which is part of my overall Travis CI integration: https://github.com/marrow/marrow.task/blob/develop/.travis.yml#L10
[21:27:33] <niczak> Kali: Oh wow, that's easy! I actually just changed priorities, removed primary from rs and then secondary became primary and all i well.
[21:27:37] <niczak> God Mongo is magical sometimes.
[21:27:37] <Trindaz> Thankyou GothAlice
[21:27:40] <GothAlice> Trindaz: Locally I use Vagrant to run my tests under a VM (with local mongod).
[21:28:17] <GothAlice> niczak: It's explicitly documented that candidates for election are first ordered by priority; as long as there exists one alive node with a priority of 10, no lower priority nodes will even be considered.
[21:28:26] <GothAlice> (As an example.)
[21:28:52] <niczak> Makes perfect sense.
[21:29:16] <GothAlice> Any sufficiently advanced technology is indistinguishable from magic. :)
[21:34:13] <Trindaz> what a bummer. I was really hoping there'd be a file-system only node.js library that would let me do this. (https://github.com/openhoat/mongodb-fs looked promising but it broke pretty badly for me)
[21:35:03] <GothAlice> Trindaz: In order to properly mock MongoDB you would have to reimplement nearly all of MongoDB, sadly. It's easier to just spin up mongod on a random local port, expose the port number as an environment variable, and have your tests use it.
[21:36:52] <darkblue_b> environment variable ?
[21:37:26] <GothAlice> darkblue_b: export MONGOD_TEST_PORT=64624
[21:37:35] <darkblue_b> thx
[21:37:41] <GothAlice> Where 64624 is the result of generating a random number.
[21:38:28] <GothAlice> It's especially easy if your test runner also has good fixture support: https://github.com/marrow/marrow.task/blob/develop/test/model/conftest.py#L9-L14 (with a different scope, I could have this spin up and automatically shut down mongod processes… this just nukes the test db itself between tests.)
[21:39:04] <darkblue_b> oof - more homework :-/
[21:42:22] <kali> GothAlice: and any technology distringuisable from magic is insufficiently advanced.
[21:42:29] <GothAlice> https://github.com/mongodb/mongo-python-driver/blob/master/test/__init__.py — pymongo's own tests use environment variables to pass in the test server info.
[21:42:38] <niczak> Have to jet but as always, thanks for the insight all.
[21:42:39] <niczak> Cheers.
[21:51:08] <pluffsy> Hello!
[21:51:35] <pluffsy> Very newbie question, sorry: I would like to just pass json objects as querys to mongodb. That works fine with simple key value pairs {"x": "y"}, but can I somehow express the $operators as valid json?
[21:54:08] <Boomtime> The $ operators are already valid JSON, perhaps provide an example of what you want to do
[21:54:12] <GothAlice> pluffsy: Valid JSON requires that you quote the strings used as keys, this means "$gt" and such are completely valid keys.
[21:55:07] <GothAlice> pluffsy: E.g. {"age": {"$gte": 18, "$lt": 30}}
[21:55:28] <pluffsy> ahh, sorry. that explains why I didn’t find anything on google. I was so sure it was invalid, should have tested of course. thanks a lot GothAlice and Boomtime
[21:55:54] <GothAlice> pluffsy: No worries; it never hurts to help. :)
[21:56:01] <pluffsy> :)
[22:45:55] <darkblue_b> in mongodb terminology, what might a 'database' be..
[22:46:44] <darkblue_b> I have one server running.. it has one port open.. /etc/mongod.conf shows one dir for storage
[22:47:20] <Boomtime> according to this: docs.mongodb.org/manual/reference/glossary/
[22:47:29] <Boomtime> Database:
[22:47:31] <Boomtime> A physical container for collections. Each database gets its own set of files on the file system. A single MongoDB server typically has multiple databases.
[22:47:51] <darkblue_b> ah glosary thx
[22:48:01] <Boomtime> :D