pmxbot IRC Log Viewer

[03:18:44] <dman777_alter> in a people collection I have "guests" in each document....some documents with "guests: "0"" some with "guests: "2"", and so on. How do I aggregate all the documents and get the total amount of guests?

[03:26:10] <joannac> dman777_alter: $group and $sum

[03:26:59] <dman777_alter> db.guests.aggregate([ { $group: { total: { $sum: "$guests" } } } ])? sorry, collection is also called guests

[03:29:11] <dman777_alter> I did this but got 0 for guests key field values

[03:29:13] <dman777_alter> db.guests.aggregate([ { $group: { _id: '', total: { $sum: "$guests" } } } ])

[03:29:43] <dman777_alter> is it because the numbers are strings?

[03:29:49] <dman777_alter> "guests: "0""

[03:40:10] <dman777_alter> yep

[03:49:41] <dman777_alter> thanks

[03:49:43] <dman777_alter> btw

[08:06:06] <mick27> Hey folks, any idea what can cause a big cpu spike when switching a secondary to primary ?

[09:03:37] <Duikb00t> o/

[09:42:15] <kurushiyama> mick27: Clients opening new connections for the connection pool, incoming data with the need of processing and compression, cache heating... Stuff like that

[12:43:05] <temuccio> Hello

[12:43:33] <temuccio> How I can monitoring mongo automatic operation (for example expired docs deleted)

[12:43:43] <temuccio> In mongo.log I don't have this information

[12:46:15] <temuccio> It is possible?

[13:08:23] <Industrial> Hi. How do I negate `.find({ enrichedWith: { $in: ['x', 'y'] } })` ?

[13:09:49] <Industrial> just add $not ?

[13:21:11] <fiatjaf> how can I force mongod to use mmapv1?

[13:24:38] <cheeser> what version?

[13:25:35] <fiatjaf> 3.26

[13:25:50] <cheeser> https://docs.mongodb.com/manual/reference/configuration-options/#storage-options

[13:26:28] <cheeser> https://docs.mongodb.com/manual/reference/configuration-options/#storage.engine

[13:26:29] <fiatjaf> mongod is being started by another application, but it says (in the logs) wiredtiger is not supported, then closes.

[13:27:06] <fiatjaf> this is set.

[13:27:16] <fiatjaf> already.

[13:27:57] <fiatjaf> do I have to set storage.mmapv1 options?

[13:29:08] <cheeser> only if you want something off of the defaults

[13:34:35] <fiatjaf> the error still happens.

[13:34:51] <fiatjaf> could it be that the other application is requiring wiredTiger?

[13:35:07] <fiatjaf> I don't think that is the case, so probably my config file is wrong? how can I be sure?

[13:36:23] <cheeser> start it manually and point mongod at your config file

[13:41:31] <kurushiyama> fiatjaf: Is it possible that you use a 32-bit version on Windows?

[13:43:56] <fiatjaf> good idea.

[13:44:50] <fiatjaf> now I can say it doesn't work.

[13:45:18] <fiatjaf> I've tried running it with --config /etc/mongod.conf, but it still fails with the wiredtiger thing

[13:45:27] <fiatjaf> tried -f /etc/mongod.conf also

[13:45:33] <fiatjaf> what am I doing wrong?

[13:46:27] <fiatjaf> http://pastie.org/pastes/10843915/text my mongodb.conf

[13:46:57] <fiatjaf> (it is mongodb.conf, not mongod.conf, the path is correct, just double-checked)

[13:47:44] <dragonmantank> I'm trying to connect to a remote MongoDB server using the Mongo PHP extension (not the new one), but keep getting "SASL Authentication failed on database 'admin': Authentication failed". I've tried specifying a database both in the connection string, and as an option, but it seems to keep trying to connect to 'admin'. Any ideas?

[13:48:08] <Derick> hm

[13:48:21] <Derick> sasl is afaik always against admin

[13:48:30] <Derick> can you show the connection string?

[13:49:24] <cheeser> fiatjaf: "fails with the wiredtiger thing" ?

[13:49:34] <cheeser> the server comes up successfully?

[13:49:58] <dragonmantank> Derick: mongodb://iad1-mongos1.domain.com:16229 is the server name being used, passing an array of [w=>1, db=> mydb, password=>mypass, username=>myuser, connect=>false]

[13:50:05] <fiatjaf> no!

[13:50:29] <cheeser> so let's try sharing the actual error message. put that up on a pastebin somewhere.

[13:50:59] <dragonmantank> Derick: Might be this SaaS we're trying to test then. The users don't get access to admin, just the individual databases

[13:51:36] <fiatjaf> well, it doesn't work even if I set --storageEngine=mmapv1 manually in the invocation command.

[13:51:49] <fiatjaf> so the problem is a different one and the error logs aren't helping me.

[13:52:47] <fiatjaf> http://pastie.org/pastes/10843931/text the error logs

[13:54:16] <cheeser> how are you starting mongod?

[13:55:15] <Derick> dragonmantank: username and pass should go into the connection string

[13:55:23] <Derick> and connect = false doesn't exist in the new driver

[13:55:34] <dragonmantank> We're not using the new driver yet :/

[13:55:58] <Derick> dragonmantank: but can you share the bit of code for connecting?

[13:56:58] <fiatjaf> sudo ./bin/mongod --dbpath /usr/lib/unifi/data/db --port 27117 --logappend --logpath logs/mongod.log --nohttpinterface --bind_ip 127.0.0.1 -f /etc/mongodb.conf --storageEngine=mmapv1

[13:58:46] <dragonmantank> Derick: https://gist.github.com/dragonmantank/f43c037451f5695020895aee34917708

[14:01:52] <Derick> dragonmantank: for SASL auth, you need to do a bit more

[14:02:05] <dragonmantank> Derick: And changing the server string to mongodb://myuser:mypass@iad1-mongos1.domain.com:16229 or mongodb://myuser:mypass@iad1-mongos1.domain.com:16229/mydb returns the same error

[14:02:18] <Derick> 'server' => 'mongodb://iad1-mongos1.domain.com:16229/$external?authMechanism=PLAIN',

[14:05:01] <dragonmantank> Interesting. PLAIN isn't supported, just SCRAM-SHA-1, MongoDB-CR and MONGODB-X509

[14:06:10] <Derick> you need sasl support compiled in - do you have that?

[14:06:22] <cheeser> fiatjaf: ah. there are dbs in this dbpath already?

[14:06:46] <Derick> dragonmantank: phpinfo() will tell

[14:08:32] <dragonmantank> Derick: PLAIN and Kerberos are disabled

[14:08:45] <Derick> you need to recompile with sasl support then

[14:08:53] <fiatjaf> no

[14:09:01] <Industrial> Hi.

[14:09:03] <fiatjaf> it should be, but I have removed it

[14:09:22] <fiatjaf> just to be sure, tried now with dbpath=~/db. it has failed again.

[14:09:28] <Industrial> Is there a query for removing all duplicate documents? e.g. the ones with the keys and values that are identical except the _id ?

[14:09:34] <dragonmantank> Derick: I'll look into that then. Looks like we just used pickle to install it with defaults

[14:09:44] <Derick> I don

[14:09:50] <Derick> I don't know what pickle is

[14:09:50] <cheeser> fiatjaf: does this dump the error out in the terminal or in the logs?

[14:10:12] <dragonmantank> A newer extension installer: https://github.com/FriendsOfPHP/pickle

[14:10:26] <Derick> we can't really support random stuff

[14:10:51] <Derick> once that becomes the official installer, we'll have a look

[14:11:02] <dragonmantank> And that's fine. I'll change our install script to make sure we turn on SASL

[14:12:43] <fiatjaf> for some reason it is still trying to open the same db path

[14:12:57] <fiatjaf> ok, I give up.

[14:13:10] <cheeser> well, ~/db is not valid in the config file.

[14:13:25] <fiatjaf> not in the config file, but on the command

[14:13:36] <cheeser> ok

[14:14:57] <fiatjaf> ok?

[14:15:07] <cheeser> yes.

[14:54:31] <StephenLynx> >FriendsofPHP

[14:54:32] <StephenLynx> kek

[14:56:38] <cheeser> https://youtu.be/wi-H6ohY37k?t=10s

[15:34:28] <Derick> dragonmantank: made it work?

[15:34:35] <devloko> hi everyone!

[15:39:29] <kurushiyama> devloko: Hi, devloko!

[15:42:11] <dragonmantank> Derick: Yes and no. Got SASL installed, but can't connect to the SSL port that the SaaS says to use

[15:43:06] <devloko> its a pleasure know this channel. I am begin to learn mongobd, and I want migrate my software to use this database.

[15:51:47] <Derick> dragonmantank: oh - is that a driver problem, or?

[15:55:45] <yopp> cheeser, "friends with of P." (epic facepalm)

[15:55:53] <yopp> •with P

[15:56:05] <yopp> 2009

[15:56:13] <yopp> ah, even 2005

[15:56:15] <yopp> awesome

[15:56:59] <mehwork> i've heard that mongo can lose data and isn't good to use if data integrity is paramount, but is that really true and if it is how so?

[15:57:38] <yopp> mehwork, everything can lose a data.

[15:57:54] <starfly> mehwork: If data integrity is paramount, you might want to look for ACID support

[15:58:46] <starfly> P.S. in a relational database

[15:59:35] <Derick> mehwork: I am not sure where you heard that, but that's sortof true for every information system

[16:00:05] <mehwork> Derick: random internet articles, but i don't automatically trust bloggers either. But they seem to go after mongo particularly in this area

[16:00:17] <Derick> mehwork: it should certainly not mean you can't use MongoDB for anything mission critical

[16:00:20] <Derick> many people do

[16:00:44] <mehwork> i think the arguments were mainly about schemalessness being less durable

[16:00:53] <dragonmantank> Derick: Not sure. I'm putting in a support ticket with the SaaS.

[16:01:12] <Derick> mehwork: "less durable" - I don't know what that has to do with schemalessness

[16:01:14] <mehwork> but i take it that's similar to saying dynamically typed languages can't be used for mission critical apps, which isn't true either. It's just maybe requires a little more effort

[16:01:21] <dragonmantank> I should have all my ducks in a row, so hopefully it's just them or me not understanding something they are doing

[16:01:42] <Derick> mehwork: right

[16:01:55] <mehwork> alright, t hanks

[16:03:15] <cheeser> yopp: a classic!

[16:04:11] <yopp> cheeser, and so true this days ;) If you are not a friends with P, you are in jail :D

[16:48:10] <hardwire> oh hey.. I guess I got banned

[16:49:09] <cheeser> doesn't look that way

[18:02:48] <dragonmantank> Derick: so the hosted mongo SaaS we are trying had to add my user to the `admin` database, and everything worked after that. Thanks for the help wth the SASL stuff though

[18:03:07] <Derick> you're welcome!

[18:29:37] <hardwire> cheeser: yeh it was weird. I think my network flapped.. but I was banned in #mongodb as hardwire_

[18:29:41] <hardwire> weird

[18:32:19] <cheeser> i don't see any such ban, fwiw

[18:32:28] <cheeser> oh, well.

[18:33:34] <hardwire> no worried.. it was just odd.

[18:33:36] <hardwire> 08:39 -!- hardwire #mongodb Cannot change nickname while banned on channel

[18:34:05] <cheeser> i've gotten that before and it's usually something odd about netsplit or somesuch

[18:34:09] <hardwire> ok

[18:34:24] <hardwire> I was more concerned that I had been quitting and joining for some reason

[20:03:18] <dfinn> is this an appropriate place to ask about pymongo? i noticed there is no channel specific to pymongo, at least not by that name

[20:03:42] <kurushiyama> dfinn: Simply shoot ;)

[20:04:05] <dfinn> i'm trying to use this functionality via pymongo : https://docs.mongodb.com/manual/reference/operator/aggregation/sample/

[20:04:32] <dfinn> trying to use it like so : http://pastebin.com/9fhSKQLs

[20:04:36] <dfinn> and I'm getting invalid syntax

[20:05:35] <kurushiyama> dfinn: What version of (py)mongo are you using?

[20:05:45] <dfinn> how can I see that?

[20:06:35] <dfinn> INSTALLED: 3.2.2

[20:07:22] <dfinn> mongo is 3.2.6

[20:07:58] <kurushiyama> dfinn: Ok, just wanted to make sure.

[20:08:53] <kurushiyama> dfinn: https://api.mongodb.com/python/current/examples/aggregation.html#aggregation-framework

[20:09:17] <kurushiyama> dfinn: If I am not mistaken, you are missing square brackets.

[20:09:22] <dfinn> i actually came across that page in my searching last night but I don't see anything there...

[20:09:29] <dfinn> i did try square brackets and got the same error

[20:10:37] <kurushiyama> dfinn: Then, I am out. I do not use Python any more and hence no pymongo ;)

[20:10:59] <dfinn> ok, thanks for looking

[20:12:54] <Doyle> Hey. What are people doing these days to get RA to stay in place after restarts?

[20:13:19] <Derick> RA?

[20:13:55] <Doyle> I put this little guy in the mongod init, and it works, I think it's getting reset after the service starts on boot. RA="$(blockdev --report | grep xvdb | awk '{print $2}')" ; if [ $RA -eq 32 ] ; then : ; else blockdev --setra 32 /dev/xvdb ; fi

[20:13:58] <Doyle> read ahead

[20:16:27] <kurushiyama> dfinn: ip = collection.aggregate([{"$sample":{"size":1}}]) wfm

[20:17:02] <kurushiyama> Doyle: /etc/rc.local?

[20:23:47] <dfinn> that does seem to work, thanks kurushiyama, I must have had my square brackets in the wrong place last night when I tried that

[20:24:05] <kurushiyama> dfinn: Nope. quotes.

[20:25:21] <deathanchor> Tell Buzz, I had fun picking his brain on the conf call. :)

[20:25:43] <kurushiyama> deathanchor: False window?

[20:27:07] <Doyle> kurushiyama, thanks, but rc.local is no longer executed after other services

[20:27:46] <kurushiyama> Doyle: Huh? Since when? You need to make it exec for systemd

[20:28:12] <Doyle> # In contrast to previous versions due to parallel execution during boot

[20:28:12] <Doyle> # this script will NOT be run after all other services.

[20:28:15] <Doyle> In centos7

[20:28:20] <kurushiyama> Doyle: if it wasn't, all my RA settings would not work ;)

[20:28:25] <Doyle> lol

[20:28:32] <Doyle> Yea, it threw me for a loop also

[20:28:42] <Doyle> adding a sleep 10 at the top of the init script fixed it.

[20:29:31] <Doyle> #UGLY

[20:29:54] <RxDx> hi.. I develop APIs using Rails, but Im reading about NodeJS and its frameworks like MEAN.. I am thinking if mongo is a good choice for building common crud (with some business logic) APIs, because most of the data are relational data (like Product has an User).. if I become using this relationships I will lost the power of mongo, right?

[20:30:13] <kurushiyama> Uh, 7. Hmm. Have to look, but I guess sysctl.conf or sth should be there.

[20:30:35] <kurushiyama> RxDx: Wrong. Seriously wrong.

[20:31:12] <kurushiyama> RxDx: You model relations differently, and optimize for use cases. But if relations were a serious problem, nobody would use MongoDB.

[20:31:42] <kurushiyama> RxDx: The power of MongoDb is that you actually _can_ highly optimize relations.

[20:32:46] <RxDx> hmm... my API data are basically relational.. but some cases like Product has many Comments.. is bad using SQL

[20:33:03] <RxDx> can I use both?

[20:33:17] <kurushiyama> 2 data sources usually re a pita

[20:33:48] <kurushiyama> RxDx: I am off for today, but we can discuss that further tomorrow, if you wish.

[20:33:53] <RxDx> hmm.. I will read more about it.. thanks :)

[20:33:59] <RxDx> sure.. thanks

[20:39:59] <dfinn> kurushiyama: one more question which may show off my python ignorance. the data that is returned from that command looks like a dict yet I get errors when I try to use it like a dict. do I need to do something to get that data in a usable format?

[20:53:16] <mehwork> RxDx: in my current opinion, mongo is mainly used by people who don't know sql but already know js. I'm sure there are some intelligent people who have other reasons for using it, but that's what i've noticed so far

[20:53:49] <mehwork> because it takes a lot away from you in terms of what relational db's give you, but gives you other things

[20:54:33] <RxDx> mehwork, I think postgresql is good for me, but can I implement a chat or comments using it? the table will be extremely big!

[20:55:30] <mehwork> depends what engine you use, 'postgres' and 'mysql' are vague terms that don't specify what engines you're using with them

[20:56:57] <mehwork> most of them support json stores and other things that are more 'nosql'

[20:57:12] <mehwork> but then it's more an argument of nosql vs sql still

[20:58:12] <RxDx> hmm.. ok.. ill read more about it.. thanks :)

[20:58:22] <mehwork> if you don't need sql then it probably doesn't matter

[21:01:49] <mehwork> but we also live in a data driven world where analytics is just as important as web app convenience. I'm too afraid to give up sql but still explore nosql stuff to learn about it so i'm not an expert on it

[21:17:42] <JoWie> RxDx: mongo would probably cost you more time to build your application with for such a use case (assuming you know both postgres and mongo well). So I think it would be more of a question of is this premature optimization, and is that needed (prematurely)?

[23:23:07] <kurushiyama> mehwork: I dare to object. I know RDBMS up and down. I could argue that people still using RDBMS need rules to get their data modelling halfway done, instead of being able to use their own judgement. However, that arguing is abdsurd, either way. The usage scenarios for RDBMS and MongoDB might overlap, but that does in no way mean they are identical, much less interchangeable.

Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 19th of May, 2016