PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 15th of November, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:25:22] <sillyquestion1> <assuming I know that this is under any normal conditions extremely undesirable> How can I run mongod as root instead of mongodb?
[00:28:30] <sillyquestion1> <boot2docker is reeking havoc on my mapped drives that work perfectly well in production, necessitating drastic changes to the developer backend vm(only ever used for testing)>
[00:28:48] <sillyquestion1> anyone?
[00:35:25] <joannac> sillyquestion1: um, okay?
[00:35:51] <sillyquestion1> ?
[00:35:54] <joannac> sillyquestion1: I'm not clear on the link to mongodb
[00:36:00] <joannac> have I missed something?
[00:36:50] <sillyquestion1> it's not a mongodb problem specifically, it's just that in the current version of boot2docker, the directories I mount are only writable by root.
[00:37:01] <sillyquestion1> using -v
[00:37:19] <joannac> that sounds like a docker problem
[00:37:23] <sillyquestion1> so, I need to run mongod as root to mitigate
[00:37:34] <joannac> I wouldn't recommend running mongodb as root
[00:37:41] <sillyquestion1> I know
[00:37:46] <sillyquestion1> hence my caveat
[00:37:48] <joannac> why don't you just change the permissions?
[00:37:48] <sillyquestion1> but
[00:37:58] <sillyquestion1> I would if it were possible
[00:38:08] <sillyquestion1> sadly it's not an option
[00:38:20] <sillyquestion1> in production, it's working fine
[00:38:25] <sillyquestion1> with mongodb as the user
[00:38:28] <sillyquestion1> so no worries there
[00:38:44] <sillyquestion1> just on the dev boxes, I've created a very small backend VM
[00:38:52] <sillyquestion1> with a stripped down version of the db
[00:39:00] <sillyquestion1> and for this one, solitary use case
[00:39:05] <sillyquestion1> I need to run mongod as root
[00:39:34] <sillyquestion1> (for the devs who insist on using macOS instead of a real OS)
[00:39:41] <joannac> okay, then sudo mongod....
[00:39:42] <GothAlice> What?
[00:39:43] <GothAlice> No…
[00:40:12] <sillyquestion1> hrm, I thought that would work as well, but it doesn't.
[00:40:14] <GothAlice> sillyquestion1: homebrew installed mongod runs as a user…
[00:40:36] <sillyquestion1> GothAlice: What does that have to do with what we're talking about?
[00:41:06] <GothAlice> "I need to run mongod as root \n (for the dev who insist on using macOS instead of a real OS)"
[00:41:30] <sillyquestion1> and what does that have to do with homebrew?
[00:41:53] <sillyquestion1> This is to address a boot2docker problem specific to macOS
[00:42:01] <sillyquestion1> https://github.com/docker/docker/issues/4023
[00:42:42] <sillyquestion1> joannac: thanks for the reply. I thought that should work as well.
[00:42:53] <sillyquestion1> *shakesfist*
[00:42:59] <sillyquestion1> this just works in linux
[00:43:05] <GothAlice> If you use a package manager, like homebrew, it'll "do the right thing" and provide a launchd hook that runs mongod automatically.
[00:43:32] <sillyquestion1> GothAlice: I'm using a docker backend vm
[00:43:54] <sillyquestion1> that works in Linux for the sane devs who use a linux box.
[00:44:06] <sillyquestion1> the others are using macOS
[00:44:34] <sillyquestion1> and when they load the backend VM, due to a documented issue, the mapped db dir becomes read-only to all but root
[00:44:36] <joannac> sillyquestion1: it doesn't? error?
[00:44:55] <sillyquestion1> it does.
[00:45:06] <sillyquestion1> predictably, it returns a
[00:45:09] <sillyquestion1> Assertion: 13651:Couldn't fsync directory '/data/db': errno:22 Invalid argument
[00:45:37] <sillyquestion1> it's world-readable, but only writable by root.
[00:46:01] <sillyquestion1> and chown is utterly useless from within the container
[00:46:17] <sillyquestion1> sudo doesn't seem to help
[00:46:51] <sillyquestion1> in the init scripts used by most *nix distros is a --chuid flag
[00:47:15] <sillyquestion1> I had hoped there was a similar thing I could use in a config file to verify this was the problem
[00:47:28] <sillyquestion1> (again, a very narrow use case, I know)
[00:47:36] <sillyquestion1> (not a mongo problem, I know)
[00:47:37] <sillyquestion1> lol
[00:47:52] <GothAlice> … also, is your data really in /data/db?
[00:48:15] <GothAlice> (I ask because that's mongod's default location, and that's usually a sign of not passing any configuration to mongod at all… it doesn't read config files by default.)
[00:48:25] <sillyquestion1> GothAlice: yes. it's mounted there in Docker
[00:48:47] <sillyquestion1> there is no config file in use at all atm
[00:48:53] <joannac> sillyquestion1: I'm not sure what you're asking
[00:49:02] <joannac> it does work if you run sudo mongod ... ?
[00:49:04] <joannac> or not?
[00:49:08] <sillyquestion1> it does not.
[00:49:21] <sillyquestion1> I was hoping there is a --chuid flag for mongod at cli
[00:50:05] <sillyquestion1> GothAlice: it's a Docker thing. This works perfectly in production, and in development on linux)
[00:50:07] <joannac> sillyquestion1: I am super confused
[00:50:13] <joannac> you are starting as root
[00:50:23] <sillyquestion1> yes, but it's launching as mongodb
[00:50:27] <sillyquestion1> when I sudo
[00:50:33] <sillyquestion1> not as root
[00:51:01] <joannac> sillyquestion1: tell me exactly what you're running
[00:51:12] <sillyquestion1> #mongod
[00:51:20] <joannac> where's the sudo?
[00:51:25] <sillyquestion1> (the hash indicating I'm root)
[00:51:30] <sillyquestion1> also #sudo mongod
[00:51:41] <GothAlice> No point sudo'ing if you're already root.
[00:51:41] <joannac> okay
[00:51:55] <joannac> and when you run as root, you get the fsync error?
[00:52:05] <sillyquestion1> yes
[00:52:22] <sillyquestion1> it's a permissions thing on that folder
[00:52:32] <GothAlice> errno:22 Invalid argument — no, actually
[00:52:52] <GothAlice> The underlying block device is saying NO to operations that potentially write data.
[00:52:52] <sillyquestion1> ???
[00:52:57] <joannac> i don't agree
[00:53:11] <joannac> it sounds like the osx docker thing doesn't support the fsync call
[00:53:15] <joannac> fsync system call*
[00:53:23] <sillyquestion1> that's my hypothesis as well.
[00:54:05] <joannac> that's not exactly a permissions problem
[00:54:15] <sillyquestion1> screw it, they're just going to have to run Linux like adults if they want to dev off the Docker infrastructure
[00:54:37] <sillyquestion1> this boot2docker thing is not going to work.
[00:54:41] <joannac> yeah, I agree
[00:54:43] <sillyquestion1> Thank you guys for your help.
[00:54:52] <sillyquestion1> I know it's a silly question
[00:54:55] <sillyquestion1> ;)
[00:54:57] <GothAlice> sillyquestion1: Before you write off an entire UNIX operating system, understand that architectural decisions lead to it. :/
[00:54:59] <joannac> if there's no support for fsync I'm not sure what we can do
[00:55:19] <GothAlice> (Which sucks that the bug exists.)
[00:55:27] <sillyquestion1> GothAlice: I manage 2,000 servers and a team of 100 internal users
[00:55:41] <sillyquestion1> I understand the architectural decisions.
[00:56:08] <sillyquestion1> MacOS is great for photo organization, web browsing, and okay as a thin client
[00:56:24] <sillyquestion1> actually, so so for web browsing
[00:56:25] <sillyquestion1> lol
[00:56:47] <sillyquestion1> but the implementations of various, common utilities are often deeply flawed
[00:56:57] <sillyquestion1> (docker as an example)
[00:57:22] <sillyquestion1> Anyhow, end rant
[00:57:23] <sillyquestion1> lol
[00:57:41] <sillyquestion1> thank you for your help and I apologize for being an angsty IRC'er today
[00:58:30] <sillyquestion1> I've been banging my head against this for a couple of hours now and I'm sure my manners are not in great working order.
[00:58:41] <sillyquestion1> *hugs* Later
[06:39:46] <tucker_> planning to store data for millions of users, what is the best approach to store each user's data: is it a collection per user, database per user, or I can store all users data in a single collection, probably to be able to make aggregation operations on
[06:40:36] <tucker_> users data may include files, will that change any thing
[06:42:37] <tucker_> I'm new to Mongo, so hopefully I'm asking the right questions
[06:59:57] <tucker__> I'm tucker_
[07:00:11] <tucker__> anyone can help with the above question?
[13:25:45] <Cruelio> Hi there, im all newe to mongodb. I have created and saved a funtion to system.js the function is fetching a result from a collection, and if i run the function from db.eval, everyting turns out perfect. However if i use the function in a $where i get a "error: { "$err" : "ReferenceError: db is not defined", "code" : 16722 }"
[13:26:10] <Cruelio> Ill be thankfull for anyhelp
[14:42:37] <teckays> hi there, I have a problem setting the maxIncommingConnections for my mongodb process, I ran ulimit -n 1000000 and restarted mongo but the last ping on MMS shows that connections.current + connections.available is always (default) 819, any ideas?
[15:01:13] <kexmex> hi
[15:01:13] <kexmex> so mongo writes warnings to logfile when an update takes too long
[15:01:26] <kexmex> that in itself slows everything down, as it's writing contents of update to disk
[15:01:41] <kexmex> which in itself causes more warnings.....and logs more, and slows eveyrhting down more
[15:01:41] <kexmex> wtf
[15:01:45] <kexmex> how do i turn that off? :)
[15:06:16] <bazineta> By finding what's slow and fixing it.
[15:06:36] <bazineta> Almost always, it means you have a bad or missing index.
[15:06:56] <bazineta> The dex tool is a good way to find them.
[15:07:25] <bazineta> http://blog.mongolab.com/2012/06/introducing-dex-the-index-bot/
[15:10:29] <user123321> In replication, when a slave DB becomes the master, how do clients know the new IP address of the master DB? Does it happen automatically?
[15:11:31] <bazineta> They get that info communicated to them frequently, the initial connext string is just a seed. When a new master is elected, they see the change automatically.
[15:13:17] <user123321> Suppose the clients connect to the DB with IP 192.168.1.2:27017, and the replication DB as 192.168.1.3:27017. When the 2nd DB becomes the master, it's IP is still 192.168.1.3, right? What would happen when the clients continue to connect to 192.168.1.2?
[15:13:23] <user123321> bazineta, ^
[15:14:04] <bazineta> They get informed that .3 is the master
[15:14:52] <user123321> bazineta, The clients get informed? Gosh, does it mean that my client apps need to change the code? They're all configured to connect to a particular IP address :/
[15:15:09] <bazineta> That's fine.
[15:15:42] <bazineta> Here, think of it like this. When you go into the mongo shell and do 'rs.status()', it tells you who the master and secondaries are, right?
[15:15:45] <teckays> hi there, I have a problem setting the maxIncommingConnections for my mongodb process, I ran ulimit -n 1000000 and restarted mongo but the last ping on MMS shows that connections.current + connections.available is always (default) 819, any ideas?
[15:15:57] <bazineta> So do you thinkg for some reason it's keeping that information a secret from the clients? ;)
[15:16:28] <user123321> bazineta, Oh I see, I was wondering if the clients would get "cannot connect to 192.168.1.2"
[15:16:54] <user123321> So they will end up connecting to 192.168.1.3 anyway, right?
[15:17:47] <cheeser> when you give MongoClient an address (or a list of addresses) that's the just the seed address. once it's connected, it can ask the cluster about it's topology and identify the current primary
[15:18:11] <bazineta> user123321 They'd get that if that was the only IP you provided them with, and the node was down. Typically in your connect string you give it all the IP addresses of the replica set, or at least a representative sample of nodes where at least one will be up. The clients get the RS info from any node they can connect to -- the connect string just 'seeds' them with ones to go
[15:18:11] <bazineta> ask first, after which the clients monitor the state of the RS themselves, automatically.
[15:19:32] <user123321> bazineta, Cool, I've missed giving all the IP's :p
[15:19:58] <bazineta> user123321 Then I feel that Good has been done here this day. Booyah!
[15:21:56] <user123321> bazineta, cheeser thanks :D
[18:34:05] <peterp> Hi everyone, is it possible to have an object literal in your schema? Eg: availability: { day: 04, month: 03, year: 2014 }
[18:48:31] <bazineta> Curious if anyone with data in AWS has investigated I/O tuning with the new I/O size factors for PIOPS and SSD volumes. That is, I think that the canned AWS server instances were tuned for 16K I/O sizes, but now AWS does 256K as the default. http://aws.amazon.com/blogs/aws/enhanced-ebs-throughput/
[19:14:17] <bazineta> GothAlice Goodness yes, MMS is a bargain at any price. We use both the monitoring and backup.
[19:14:51] <GothAlice> bazineta: :D (These days it can even set up and manage your cluster for you, which is Extra Nifty™.)
[19:16:35] <bazineta> GothAlice Yes, recently activated for our account, haven't looked into it much. The SalesDroid did indicate that a rebuild was required to take advantage of it.
[19:17:04] <GothAlice> Yeah; automation always has certain assumptions that need to be met.
[19:17:20] <bazineta> GothAlice which I presumed to be the usual add new replicas and remove old ones.
[22:00:40] <HardyK_> Can anyone point me to any resource explaining the difference between Safe Mode and different levels of Write Concern in MongoDB?
[22:01:29] <Boomtime> http://docs.mongodb.org/manual/core/write-concern/#write-concern-levels
[22:02:17] <kali> Boomtime: safe mode is the old wording
[22:03:13] <kali> Boomtime: iirc, it is know equivalent to the default "acknowledged" level
[22:05:22] <HardyK_> kali: it makes sense now. I'm reading docs of some MongoDB drivers and they use the old wording. Seems that "safe mode on" means { w: 1 } and "safe mode off" means { w: 0 }
[22:07:31] <HardyK_> Thanks, Boomtime and kali!
[22:09:20] <Boomtime> HardyK_: I believe the old "safe mode" is equivalent to journaled which is not quite the same as w:1, but close