Log file Viewer
#mongodb logs for Monday the 18th of April, 2016
[12:48:34] <kurushiyama> Now that's double strange. Maybe we need to get down to verifying the image checksums with the main repo.
[12:53:08] <StephenLynx> if it works, ill get to mongodb dudes and tell them they dun goofed with 3.2.5
[12:58:49] <kurushiyama> StephenLynx: I guess that there are integration tests before release. I know I had some.
[13:00:39] <kurushiyama> StephenLynx: Can well be. I have noticed very strange behavior, though only in non-standalones.
[13:01:27] <kurushiyama> StephenLynx: I guess the problem is with centos7.2 then, because I have 3.2.3 running on CentOS7
[13:04:55] <kurushiyama> My resources are limited, so it might be a problem there. Will retry, but I think we have sth major here.
[13:06:26] <StephenLynx> I just asked if anyone running mongo had issues like mine, even said I don't think its centos fault
[13:07:10] <kurushiyama> StephenLynx: They a re OT... well, you know the guys from my home country with the brown shirts and such.
[13:09:31] <kurushiyama> c) Conclusion: It is sort of a permission or resource limitation problem only applying to user mongodb.
[13:11:40] <StephenLynx> pretty sure there is, I don't think an update would remove it. but ill check it
[13:24:00] <StephenLynx> Derick, cheeser GothAlice anyone have any idea on what might be happening here?
[13:27:26] <pokEarl> If I do System.Exit(0) or whatever in a Java app using the mongoDB Driver, does the mongoclient close connections safely on its own or do I need to do MongoClient.close() ?
[13:58:33] <R1ck> hi. A customer is asking us why a aggragate command is running slower on his server than when running on his virtualbox vm, with the same indexes. Are there any specific settings I could look at?
[14:00:28] <zeld> hi, there exists a simple image that show the difference between a sql database and nosql databases?
[14:02:59] <zeld> so... maybe a n image which show a table and a grapoh with node and key value can be a good example?
[14:07:03] <zeld> i'm developing an application that use big datas... and i would like to use a database nosql to manage data
[14:09:28] <zeld> yes... i was one time to a presentation about mongodb and i was really impressed by its scalability, flexibility and so on.
[14:24:35] <kurushiyama> cheeser: Yeah, but it's like "von hinten durch die brust ins auge" as we say in German. This roughly translates to "tyring to stab somebody from the back through the chest into the eye" ;)
[14:43:24] <kurushiyama> Did a diff over the contents of the 3.2.4 and 3.2.5 rpms. The only major difference seems to be binary.
[14:59:05] <kurushiyama> StephenLynx: Yeah, I get that. But a service file is a service file. And a SysV init script for a systemd distribution is... not optimal.
[14:59:32] <StephenLynx> I agree there is room for improvement, but systemD is designed to run sysvinit files without issues.
[15:04:32] <kurushiyama> The freezing is... astonishin. And the fact that the init file does not seem to be run properly is rather disturbing. The worst being that I can not seem to find a reason.
[15:51:50] <kurushiyama> StephenLynx: The good news are: I have installed and run 3.2.4 and 3.2.5 successfully on CentOS 7.2
[15:55:23] <kurushiyama> What drives me nuts about that is that it is reproducible for you on various versions of MongoDB
[15:57:29] <kurushiyama> Hm, just an idea: Can you export the "broken" VM and run it on a different machine? Just to rule out it's sth with the virtualization. Not very likely, but I am running out of ideas.
[16:05:04] <kurushiyama> StephenLynx: can you try to start mongod with "sudo -u mongod /usr/bin/mongod --dbpath …" ?
[16:09:35] <kurushiyama> Well, startung it at root might have changed some permissions. Please doublecheck. Other than that, it is getting weirder and weirder...
[16:10:34] <StephenLynx> it still doesn't explain why it starts failing before being started as root
[16:12:21] <kurushiyama> StephenLynx: Agreed. Hence I wanted to start it as mongod, but in foreground without a log file to find out whats going on.
[16:19:47] <StephenLynx> Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details.
[16:25:01] <StephenLynx> https://bbs.archlinux.org/viewtopic.php?id=202302 if there is any similarity to this
[16:40:08] <kurushiyama> Because if that was the issue, starting it as mongod with the config would have a problem...
[16:40:41] <StephenLynx> so what's in the init script besides the settings file that is making it fail?
[16:43:56] <kurushiyama> the "chown mongod.mongod -R /var/lib/mongodb && chown mongod.mongod -R /var/log/mongodb/"
[16:51:05] <kurushiyama> "chown mongod.mongod -R /var/lib/mongo && chown mongod.mongod -R /var/log/mongodb/"
[16:55:38] <StephenLynx> Apr 18 13:45:22 localhost.localdomain systemd[1]: Unit mongod.service entered fa
[16:55:39] <StephenLynx> Apr 18 13:45:22 localhost.localdomain polkitd[620]: Unregistered Authentication
[16:56:10] <StephenLynx> Apr 18 13:45:22 localhost.localdomain polkitd[620]: Unregistered Authentication Agent for unix-process:1077:42583 (system bus name :1.17, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
[16:57:00] <kurushiyama> Not sure, tbh. All I can tell you is that it seems to be the only difference.
[16:58:31] <kurushiyama> Aye. I come more and more to the conclusion that there is some problem with dropping privileges
[17:03:09] <kurushiyama> For quite a while. As soon as I got the SELinux stuff fixed, it worked like a charm
[17:04:47] <kurushiyama> Well, docs state that if you try to bind to anything else then 127.0.0.1, you do. ;)
[17:06:49] <kurushiyama> Well, try it. Set SELINUX in /etc/sysconfig/selinux to "permissive" (no quotes) reboot and try the service _as root_. That way, we should have rules out any related problem, at least.
[17:20:11] <kurushiyama> Nah, I doubt that. with enforced and semanage port -a -t mongod_port_t -p tcp 27017, It should run again. My _assumption_ is that we had security context problems, maybe, because you started it as root. Or screwed permissions. Or sth
[17:22:36] <StephenLynx> either that or the command on https://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat/ is wrong
[17:23:16] <kurushiyama> Iirc, you need to have a perm to bind to any IP other than 127.0.0.1 as well. But I'll check that
[17:32:37] <kurushiyama> StephenLynx: Not really. Docker images are lighter in weight and have less overhead. But lets stick to this. I can confirm that SELinux="enforce" + "semanage port -a -t mongod_port_t -p tcp 27017" does _not_ work.
[17:34:00] <kurushiyama> StephenLynx: So, as far as I can see, it seems to me that there is a problem with mongodb-org-server's selinux settings.
[17:38:40] <kurushiyama> Correct. I think it _may_ be something which is a side effect of running the init.d file.
[17:39:20] <kurushiyama> StephenLynx: Since I can rule out it is mongod's selinux context. I can start it manually as user mongod
[17:40:52] <kurushiyama> StephenLynx: If it is a side effect of the interaction between mongod, systemd and SELinux, it can be anything. I just start to track it down, now that I can reproduce it again.
[17:48:28] <kurushiyama> As far as I can see, the user mongod is denied access to ld.so.cache under systemd's runuser
[17:51:56] <kurushiyama> line 1 is where is set the policy from "enforce" to permissive, activating the audit logging.
[17:52:02] <StephenLynx> type=AVC msg=audit(1460997506.961:34): avc: denied { execute } for pid=843 comm="mongod" path="/etc/ld.so.cache" dev="dm-0" ino=17001639 scontext=system_u:system_r:mongod_t:s0 tcontext=unconfined_u:object_r:ld_so_cache_t:s0 tclass=file
[17:59:00] <aguilbau> hi, is their a way to match the last element of an array ? like 'field.$.value', but for the last element.
[18:03:55] <aguilbau> lets say i do this: 'db.tests.insert( { a: [ { b: 1 }, { b: 2 }, { b: 3 }, { b: 4 } ] } )'
[18:05:12] <aguilbau> now can i find all documents where all last values of 'a' have a field set to b = 4
[18:20:25] <kurushiyama> aguilbau: kurushiyama's Rule of thumb: If you depend on the order of an array, there are good chances there is something wrong with your data model.
[19:19:49] <jr3> Mongo writelocks updates by default correct? If I have a document where I update to add to a set if length is < 20, there should be any kinda of "race" to where I actually end up with 21 elements in a set right?
[20:11:12] <nalum> hello, I'm trying to set up tls connections on my replicaset. I have the certs I need for the mongod, but I'm confused as to what certs I need for the cloud manager agents (automation, monitor and backup)?
[20:18:35] <kurushiyama> nalum: Do you use an installed version of CM or do you use The cloud version?
[20:23:22] <kurushiyama> cheeser: You probably understand better what's going on. I am totally puzzled.
[20:24:05] <kurushiyama> nalum: Usually, the agents connect to CM, which has cert from a CA known by the agents.
[20:25:09] <kurushiyama> cheeser: Quick one: unless sth has changed _considerably_, agents dial CM to push data (monitoring, backup) and pull orders (automation), right?
[20:27:01] <kurushiyama> nalum: Well, sorry, not exactly. You need to give the paths to the cert and keyfile for the server so that the server can utilize SSL.
[20:27:44] <kurushiyama> nalum: Think of the cert file as the servers passport, and the key file as the servers keyring.
[20:32:17] <nalum> kurushiyama: Thanks, so the client will look for the relavent CA files on the local machines?
[20:39:50] <nalum> Okay, so I've setup with allowSSL, when I using the mongo command line tool to connect with mongo --host host.com --ssl it fails with: Failed global initialization: BadValue need to either provide sslCAFile or specify sslAllowInvalidCertificates
[20:42:38] <nalum> Okay, is there a way to have it just look for the CAFile as it is a public CA and not self signed? Maybe I should not have the cloud manager set up with the sslCAFile?
[20:49:46] <kurushiyama> nalum: No, it is not. And it does not make sense to do any validation, either.
[20:51:06] <nalum> Why does it not make sense? Is that not part of what TLS certs are for, verifying that I am connecting to the server I think I am?
[20:52:25] <nalum> So sslAllowInvalidCertificates is for verifying the issuer and the clients should continue to verify the host even with that set?
[20:53:15] <kurushiyama> nalum: No. it validates the certificate the server identifies itself with against well known certification authorities.
[20:53:56] <kurushiyama> nalum: The thing is this: A CA is not only ther for encryption, but to validate the certificat holder. Say your bank, for example.
[20:54:51] <kurushiyama> nalum: the CA of the bank's certificate guarantees you (to a varying extend), that they made sure that the ceritificate holder is who he claims to be.
[20:56:17] <kurushiyama> nalum: Since let's encrypt does only verify a host, you _really_ do not want them to be trusted by default.
[20:57:45] <nalum> kurushiyama: Thanks for the info, much appreciated. So would you recommend not using them in production systems?
[20:58:29] <kurushiyama> nalum: For business? I would not consider it for a second, especially given the prices for wildcard certificates nowadays.
[20:59:29] <kurushiyama> nalum: As for secured communication between known hosts, it might be ok. But then, there is no reason to validate it, really.
[21:00:31] <kurushiyama> nalum: As per communication with the outside world? I'd run from a business which is that sloppy with its digital identity...
[21:12:06] <nalum> Okay, so last question for today. With cloud manager is it possible to have it use custom domains rather than mongodbdns.com or should I CNAME or ALIAS the domain name that it generates?
[21:13:47] <nalum> When you use cloud manager to bring up a Machine Instance in AWS it assigs it a domain name.
[21:19:38] <nalum> I've not seen anything in the cloud manager, but maybe I'm looking in the wrong places. Is it possible to change that to use my own domain name or should I create a CNAME or an ALIAS DNS entry for the mongodbdns.com that is created?
[21:31:31] <zsoc> It appears that while doing a Model.create(arr), the process stops after it hits the first error. Maybe not, but even while trying to use the .then() promise it appears that way. I'd like to be able to catch all the error 11000's because they shouldn't be fatal.
[21:37:37] <cslcm> Hi folks. I'm trying to get my head round MongoDB sharding. I think I've successfully created a three-host cluster following the documentation, which is good, however I didn't see any way to configure how much redundancy there is. For example, does each host contain a full copy of the data, or is it split into three?
[21:40:37] <nalum> cslcm: Assuming that when you say 3, you mean 3 nodes? I think it's split across the 3 nodes, but for redundancy you would have those 3 nodes actually be replica sets. So 3 shards of 3 node replica sets.
[21:42:17] <cslcm> ah i see, at the moment I have a single replica set with the three servers as members
[21:42:58] <nalum> If the 3 servers are part of the same replica set then they are not sharded, if I understand it correctly
[21:45:24] <nalum> I don't know if that can be done, but I've not looked too deeply into sharding yet
[21:48:28] <nalum> My understanding would be, you have 3 shards, each shard is a replica set with 3 nodes each node is on it's own server. This then allows for single nodes to fail but keeps the system available. obviously the actual number of nodes is up to you and for production systems 5 nodes in a replica set is recommended.
[21:51:17] <nalum> cslcm: In your suggested setup of 3 servers, you could have 3 nodes per replica set, one on each machine, so the whole replica set is not on one machine, meaning each machine has 100% of the data.
[21:53:54] <cslcm> nalum: The problem is (and this is one of the problems which is supposed to be tackled by sharding, according to the mongo docs) - my data cannot fit on a single machine. There's too much data. Hence why I only want 66% on each machine
[21:57:53] <kurushiyama> cslcm: A sharded cluster _can_ consist of replsets. But it does not _have_ to.
[21:58:18] <cslcm> i think I understand, so if I have host A,B,C.. I can create three replication sets, one with A+B, one with B+C and one with A+C
[22:00:15] <kurushiyama> cslcm: First of all, the minimal replset consists of 3 nodes. 2 data bearing nodes + 1 arbiter.
[22:01:53] <kurushiyama> Because there are usually 2 factors who force you to shard: IO and disk space.
[22:03:38] <cslcm> i'm more than happy to run a single instance on each node, I'm not suggesting otherwise
[22:04:00] <kurushiyama> cslcm: > i think I understand, so if I have host A,B,C.. I can create three replication sets, one with A+B, one with B+C and one with A+C
[22:04:56] <cslcm> oh, I see what you mean, an additional replication set would require another node
[22:07:43] <kurushiyama> cslcm: So, your minimal setup would be 3 config servers, 4 data bearing nodes and 2 arbiters. The latter 6 would each form a replset of 2 data nodes + an arbiter.
[22:08:34] <kurushiyama> Now for the good news: the 3 config servers and the arbiters can be relatively cheapo VPS (though they ofc should be on different machines)
[22:09:53] <cslcm> It sounds like MongoDB can't be used for my application. I only have three physical servers to run this on
[22:09:59] <kurushiyama> cslcm: I can not suggest to run a sharded cluster on standalones or (while technically possible) only 1 config server.
[22:10:32] <kurushiyama> cslcm: Oh, it could. But your requiremnets are a bit like "I want to have the cake _and_ eat it."
[22:13:42] <cslcm> What I'm asking for is very achievable in principle... split the database into three chunks, 1 2 3, one server has 12, one server has 23, one server has 13
[22:13:51] <kurushiyama> cslcm: Scale up and put more space into them. Plus: Which DBMS do you expect to behave better? We are not talking of implementation, but hard facts and logic.
[22:15:28] <kurushiyama> cslcm: If you thinks so. If you ask wether it is technically possible, the answer is yes.
[22:17:05] <kurushiyama> cslcm: Without an arbiter, if one node goes down, the wholw replset goes down.
[22:17:33] <kurushiyama> since 1 node alone can not decide wether it is the most up-to-date. It might just be cut off by a network patitioning.
[22:20:20] <kurushiyama> oh, put this down as a note: you have to set WT's cache to < 1/4 of your physical RAM, and you need tons of swap space.
[22:21:24] <kurushiyama> each instance will a) try to use 50% of the available physical RAM as WT cache
[22:22:23] <kurushiyama> If you are going to run that stunt, it most likely will be best to run the individual instances as docker images for resource limitation.
[22:23:02] <kurushiyama> Do not even bother to think about HDDs, and give each instance a dedicated LVM partiion.
[22:23:56] <kurushiyama> And you _really_, _really_ should have a deployment diagram handy in case you need helpl.
[22:26:19] <cslcm> In the long term I will expand and add more nodes, and then it can become more sane
[22:27:08] <cslcm> Just to confirm though, by default, a replication set splits the data evenly over all the nodes?
[22:32:48] <Freman> in other news, might finally do something with that domain related to twitter I bought years ago lol
[23:29:57] <cslcm> Question: The docs make very clear that the config servers must be backed up. Do I need to back up ALL of them, or will just one be enough?