Log file Viewer

[00:07:23] <FatalNIX> So how is mongodb?

[00:38:41] <kurushiyama> FatalNIX Sorry, my little boy needed me. Well, MongoDB is suitable for some use cases and unsuitable for others. The question actually is: what do you want to _do_ ? ;)

[00:40:44] <croberts> does db.repairDatabase() actually delete any data

[00:40:52] <croberts> i know it frees up space

[00:41:28] <cheeser> kurushiyama: Crush my enemies. See them driven before my. Hear the lamentations of their women.

[00:41:57] <croberts> just wondering where that free space is coming from

[00:42:20] <kurushiyama> croberts It _may_ delete data.

[00:42:41] <croberts> ah

[00:42:48] <kurushiyama> croberts As written in the docs. The only thing repair guarantees is that your db will be in a usable state

[00:43:13] <croberts> kurushiyama: good times, thank you for the info

[00:43:43] <kurushiyama> croberts So if your datafiles were (partially) corrupted, and if it was only one field in one doc, said doc may have vanished.

[00:43:54] <FatalNIX> kurushiyama, I started working on my own database engine which is exactly the way I like for a game I am developing (A MUD in fact) Because Redis didn't suit me, I designed an interface that provides a key value store which is accessed by a Forth interpreter I wrote..

[00:44:08] <kurushiyama> croberts Before using _any_ administrative command, you should read the docs. Just saying.

[00:44:25] <FatalNIX> but when it came to how to store information on disk somebody told me, maybe I could use mongodb for a backend, and just interface it so I dopn't have to worry about storage

[00:44:25] <croberts> kurushiyama: i will start going through them thank you

[00:44:36] <FatalNIX> I was curious if Mongodb would be good for that

[00:45:22] <FatalNIX> my interface has a lot of nifty features that already work such as conditional publish / subscribe and such

[00:45:26] <kurushiyama> FatalNIX Hard to say in particular

[00:45:45] <FatalNIX> and complex data structures that can pretty much be infinite in depth

[00:47:08] <kurushiyama> FatalNIX PubSub can be sort of emulated. Infinite data structures? I am not aware of ANY database providing arbitrary depth data structures. But for an event based system (which a MUD is) I have serious doubts about the modelling, right from what you tell.

[00:49:19] <kurushiyama> FatalNIX to be more specific: MongoDB has a size limit for documents of 16MB. Hardcoded.

[00:53:14] <kurushiyama> Anyway, g2g. Good night!

[00:55:05] <FatalNIX> kurushiyama, I'm using Perl for the hash tables to handle the depth and things at least on this side

[00:55:13] <FatalNIX> yeah i heard about that

[00:55:18] <kurushiyama> cheeser One of the descendants? You name isn't Prosser, is it? ;) Goond Night!

[00:55:21] <FatalNIX> I don't think thatl be a problem for a MUD

[00:55:26] <FatalNIX> 1MB is VERY HUGE

[00:55:40] <FatalNIX> for text

[00:58:36] <kurushiyama> FatalNIX May well be, but what if the scale exceeds your expectations. Better to model properly and to be able to scale (almost) indefinetly. I once calculated the maximum datasize of a MongoDB cluster around 8.2 exabytes. Why would you want impose artificial limits by wrong ceoncepts and bad (and almost always) sub-performing data models? Just something to think about ;) Ok, need to hit the hay, 18h day.

[01:02:44] <FatalNIX> My game will only use probably 5 depths at most

[01:02:54] <FatalNIX> mostly 3

[01:03:10] <FatalNIX> ah, have fun

[02:39:06] <FuzzySockets> Hey all, I'm unable to connect to an ec2 instance with mongo installed: Error: connect ECONNREFUSED is the error I'm getting. The instance has a security group on it, allowing incoming connections from my ip address on tcp port range 27017 - 27030

[02:42:15] <Boomtime> apparently it doesn't - ECONNREFUSED is socket layer, the stack itself said no, so either you hit the wrong machine, mongodb isn't running, or routing isn't permitted

[02:44:20] <FuzzySockets> Boomtime: hmm

[02:54:52] <FuzzySockets> Boomtime: You sounded so sure of yourself. Turns out a line in mongod.conf "bindIp: 127.0.0.1" was causing the issue. Commenting this line out fixed it.

[02:58:44] <Boomtime> heh, that'll do it - mongodb isn't listening - pretty much equivalent to not running

[02:59:04] <Boomtime> but yep, nice find

[03:47:20] <tambu> Any ssl/replica set guys here, I'm having trouble finding clear documentation on how to handle replicaset / client ssl certs with a single CAFile on the mongod server

[04:17:15] <macwinne_> how do you see teh current running storage engine?

[04:18:58] <macwinne_> nm.. db.serverStatus().storageEngine

[04:19:47] <Boomtime> db.serverStatus().storageEngine.name

[04:19:49] <Boomtime> snap

[04:20:19] <macwinne_> lol

[04:20:20] <Boomtime> heh, couldn't remember and had to copy from a shell - you win @macwinne_

[04:20:43] <macwinne_> i'm so pleased with how easy it has been to maintain mongo

[04:21:03] <macwinne_> i'm almost complete updating cluster to 3.2 and migrating to wiredtiger

[04:22:53] <macwinne_> with wired tiger are indexes automatically compressed with prefix compression? or is this a 3.2 feature regardless of which storage engine you use?

[04:24:35] <tambu> so far the only thing causing me headache in mongo is the SSL setup with replicaSets, but I"m not sure if that isn't an issue with my certificates

[04:26:43] <macwinne_> what error do you see in the logs? i'm pretty good with SSL related troubleshooting, though I've never set it up with mongo

[04:28:04] <tambu> I "think" I might have figured it out.. I think the problem was my cert was invalid.. I used "openssl verify mongodb_rootCA.pem mongodb_client.pem and it rejected it

[04:28:22] <tambu> let me test this.. or I definitely might hit up this.. it's been a couple hours of learning for me sorting this out

[04:28:59] <tambu> One quest.. the CAFile option in mongod.conf can you store "multiple" rootCA crts? or just one

[04:29:28] <macwinne_> it's usually not well documented with most things, but i've found that storing multiple root and intermediate certs in the same file works

[04:29:36] <macwinne_> though sometimes I need to reverse the ordering in the file

[04:30:30] <tambu> hrm. thanks.. yes they have the basic ssl stuff documented but when it comes to replica sets and things the information seems a bit more lacking. That being said I must say the mongodb docs are some of the best I've ever read.. :) so not bashing

[04:31:25] <macwinne_> i'm just reading the docs now to get familiar

[04:31:52] <macwinne_> are you using your own self signed certs?

[04:31:57] <macwinne_> or your own CA

[04:32:43] <tambu> WOOT victory.. My own CA it's surprising how many major companies still use self-signed certs..

[04:33:22] <macwinne_> what was the issue?

[04:33:25] <tambu> Do you recommend having I've created a server.pem and client.pem.. technically I'm using the same server.pem on all the replicaSets,

[04:33:58] <tambu> so apparently when creating my server crt and filling the country,state,city,company,orgunit values I used different ones from the rootCA values

[04:33:58] <macwinne_> sorry, didn't understand you

[04:34:19] <tambu> but when creating the client crt I used the EXACT same values as I did on my rootCA and that seemed to cause the client CRT to be invalid

[04:34:37] <tambu> I have a replicaSet 2 server atm but eventually 4+1arbiter

[04:35:08] <tambu> should I use the same PEMKeyFile: /usr/local/var/mongodb/mongodb_server.pem

[04:35:08] <tambu> for all of them? Or should I have like server1.pem server2.pem?

[04:35:15] <macwinne_> oh.. maybe because the common name for teh cert was the same

[04:35:39] <tambu> macwinne_ yes it was.. I was using my laptop "oberon" for all of them

[04:35:43] <macwinne_> i would just have the same one for ease of maintanance.. i don't think there is much of a security gain..

[04:36:23] <tambu> .. actually though can you the common name would be different on each server.. just thought of that.. mine are the same as i'm running both replica sets from the same laptop(diff ports)

[04:36:24] <macwinne_> like when you have a cluster of HTTP servers, you have the same cert and private key on all of them

[04:37:06] <macwinne_> you are using client certificate authentication?

[04:37:27] <tambu> yes I have CAFile: set in the mongod.conf file

[04:38:13] <macwinne_> cool. glad you got it working. i need to do that eventually for my replica set as well

[04:39:35] <tambu> sorry, so you mentioned using the same crt for each server, which I agree would be easier, but when each server has a different FQDN the common names won't match so I'm not certain they would work.

[04:48:42] <macwinne_> true.. but i'm not sure how common validation works within the mongo client to mongo server connection

[04:48:55] <macwinne_> would need to test it..

[04:49:06] <tambu> thanks, yeah I am already on it.. :)

[04:49:30] <macwinne_> the common name validation in HTTP is done by the client web browser.. probably a setting to disable the check

[05:19:17] <macwinne_> are there any good opensource mongo cluster management tools? something that helps to provision sharded clusters

[05:20:00] <Boomtime> 'opensource' = you mean free

[05:21:40] <macwinne_> yep

[05:22:13] <macwinne_> i like mms.. don't mind paying for it, but too pricey right now for our startup

[05:22:30] <tambu> macwinne_ Incase you were interested if you are using client authentication on SSL you will need a separate SSL certificate for each replica server, I did find out you can use *.domain.com wildcard ssl and they work. Or you can net.ssl.allowInvalidHostnames but at that point whats the point of authenticating :)

[05:23:29] <macwinne_> tambu: cool.. good to know, thanks!

[05:24:05] <tambu> i realize it's a little on the paranoid side to have that level of encryption plus authentication, but figure while I'm here might as well get familiar with all the options.

[05:25:33] <macwinne_> if you are not on a secure network, then you need it to prevent MITM attacks

[05:26:27] <tambu> true, enough

[05:26:39] <tambu> Thanks for the help macwinne_ I'm off to bed. GL with your efforts

[05:26:50] <macwinne_> you too.. gnite

[05:43:54] <Boomtime> net.ssl.allowInvalidHostnames doesn't destroy all the security if you have configured your client to only accept certs signed by a specific CA - hostname validation is actually not even part of the SSL spec, it's a stupid ad-hoc check that was implemented by browsers because online banking wants uses weak single-certificate mode rather than real two-way SSL

[05:44:43] <Boomtime> that's right; real SSL is way better than what your online banking uses

[06:19:15] <macwinne_> what's the best way to see if some of your collection indexes are being used if you don't have access to the client code.. i have a bunch of indexes defined on my collection from a long time ago and I suspect they are not being used anymore

[06:19:31] <macwinne_> would like to clean them if they are no longer used

[06:19:53] <macwinne_> mongo 3.2

[06:34:00] <macwinne_> whoa. index size savings because of prefix compression is much more than my wiretiger storage savings

[06:34:10] <macwinne_> 6200MB of index size down to 1200mb