pmxbot IRC Log Viewer

[02:14:27] <Doyle> Hey. Is journaling enabled by default on a single host MongoDB instance?

[02:21:51] <Boomtime> Doyle: journaling is on by default, regardless (since version 2.0)

[02:21:52] <Boomtime> http://docs.mongodb.org/master/tutorial/manage-journaling/#enable-journaling

[02:21:58] <Doyle> tyvm

[02:22:16] <Doyle> Got a stuck host, there a good chance that it'll come up happy after a shutdown?

[02:22:26] <Boomtime> is it 64 bit?

[02:22:29] <Doyle> with journaling I think so, but I like to bounce things around

[02:22:31] <Doyle> yea, 64

[02:23:13] <Boomtime> you should still shut it down gracefully rather than rely on journaling to save you

[02:23:31] <Doyle> Can't access it. SSH is offline

[02:23:38] <Doyle> ping is up, but can't access

[02:29:02] <Doyle> Seeing if I can get a mongo shell to connect, but I doubt it

[05:11:36] <aps> Doing repeated db.fsyncLock( ) renders Mongo 2.6.x unlockable? Is this a known thing?

[06:05:52] <mbtamuli12> i installed mongodb from the source in fedora22

[06:06:07] <mbtamuli12> How do I make it start on every reboot?

[07:25:08] <Doyle> mbtamuli12, https://docs.fedoraproject.org/en-US/Fedora/22/html/System_Administrators_Guide/ch-Services_and_Daemons.html#s1-services-configuring

[10:08:41] <Constg> Hello, I have a question about hardware. I have a database about, according to MMS, 213 GB andit's growing. As I need to replace the machine, should I take more RAM 128 -> 256 or should I keep the same RAM but increase the proc (16 cores -> 20 cores but I loose 0.2 Ghz)?

[11:39:19] <jdo_dk> Can i combine $inc and $set in a single query ?

[12:16:08] <d-snp> hi guys, the performance of wiredtigercollection.drop() is significantly lower than that of the old style, is there some documentation on this or something I could do to hasten it?

[12:44:00] <d-snp> our config servers are in an inconsistent state again

[12:44:05] <d-snp> seems like a weekly occurrence :(

[12:45:00] <deathanchor> d-snp: only the logs should tell you what's gone wrong

[12:45:50] <d-snp> I think it crashes during a particularily long collection remove operation

[12:54:23] <deathanchor> the configsvr crashes?

[12:54:42] <deathanchor> logs don't say what's up?

[13:08:36] <d-snp> deathanchor: I put my colleague on figuring it out, if he finds anything I'll give an update here :P

[13:09:23] <deathanchor> yeah also check the logs of the mongos where the remove was done.

[13:21:47] <d-snp> deathanchor: wiredtiger seems to lock the database for as long as a collection.drop() call runs, that seems a bit unnecessary, do you know if it would be easy to fix that?

[13:22:10] <d-snp> or do collections have some relation to eachother in wt? I remembed drops being near instant on mmap

[13:22:37] <d-snp> the whole reason we moved to a multi-collection architecture was that drops were instant on mmap

[13:23:06] <StephenLynx> why do you need to drop collections often?

[13:23:47] <d-snp> it's data with a fixed retention

[13:24:34] <d-snp> we throw away about 60gb per day at the moment I think

[13:27:05] <d-snp> I think a few weeks ago it was established that our setup was nonsensical :P

[13:27:13] <d-snp> (in this channel)

[13:27:37] <d-snp> but it solved a bunch of problems for us, and it was relatively easy to implement on our app side

[13:28:52] <StephenLynx> ok, so let me get this straight

[13:29:00] <StephenLynx> you have an ass-backwards system

[13:29:10] <StephenLynx> and somehow its WT fault for not playing nice with it?

[13:29:27] <StephenLynx> why don't you use TTL instead?

[13:29:27] <d-snp> eh, I'm not faulting wt for anything

[13:29:41] <StephenLynx> why don't you just NOT save all this data?

[13:29:55] <StephenLynx> "deathanchor: wiredtiger seems to lock the database for as long as a collection.drop() call runs, that seems a bit unnecessary"

[13:29:57] <d-snp> I'm just asking whether there's a reason wt's drop takes so long

[13:30:12] <StephenLynx> go figure, is not even the default engine yet, afaik.

[13:30:50] <d-snp> true, but it's very nice it saved us a bunch of space

[13:40:41] <deathanchor> s3ccum86

[13:41:19] <Kobbb> Hello guys, I have some trouble creating MongoDB users/roles

[13:42:06] <Kobbb> here is my commands : http://pastebin.com/mFMAtmNr, but when I run the second one, it tells me : "2015-09-08T13:36:44.593+0000 E QUERY Error: couldn't add user: No role named basicUser@toto

[13:42:06] <Kobbb> at Error (<anonymous>)

[13:42:06] <Kobbb> at DB.createUser (src/mongo/shell/db.js:1096:11)

[13:42:07] <Kobbb> at (shell eval):1:14 at src/mongo/shell/db.js:1096"

[13:43:02] <Kobbb> any thoughts?

[14:07:32] <d-snp> deathanchor: hopefully that's not your password? :P

[14:08:22] <deathanchor> only to my laptop

[14:08:31] <deathanchor> keepass for everything else

[14:08:38] <deathanchor> which of course has another password :)

[14:10:36] <deathanchor> I have 5 passwords I know, and 300+ generated. never the same username/pw used.

[14:28:22] <saml> one node is in RECOVERING state and 90 hours behind primary. it's never catching up

[14:29:43] <Kosch> heyho. I'm compiling mongodb from source and I get pretty large executables ~350MB each. My options are --64 --ssl --no-glibc-check --release all. According to ldd, the executable is not static linked. Tried different versions of gcc with same result. Any ideas what I missed?

[14:34:27] <Kosch> ok, seems the binaries are not stripped

[15:19:36] <symbol> Looking for a sanity check here...with proper data modeling and utilizing embedded documents, would MongoDB be a viable option for a basic e-commerce site?

[15:19:57] <StephenLynx> yes

[15:20:09] <symbol> I feel like it is but I just got ransacked with critique.

[15:20:31] <symbol> Seems a lot of people think that MongoDB is only good for logs an unrelated data.

[15:20:59] <kali> symbol: i would use a sql database for the transaction part

[15:21:06] <symbol> *logs and

[15:21:38] <symbol> Ah, polygot persistence.

[15:21:42] <saml> what's a basic e-commerce site?

[15:21:59] <kali> symbol: you can implement transaction support on top of mongodb, but it's tricky to get it right.

[15:22:18] <symbol> I've seen some of the solutions but you're right it does get tricky.

[15:22:23] <Kosch> symbol: you've to put more logic into your application to keep the data consistent.

[15:22:41] <symbol> It's a loaded question - "basic" is hard to define. I'm just thinking the most basic CRUD with some payment integration.

[15:23:22] <symbol> So it'd take a lot more work to get things "right".

[15:23:36] <saml> do you have to implement credit card processing yourself?

[15:24:33] <symbol> Nope - I imagine we'd use an external service.

[15:24:54] <kali> and availibility ?

[15:25:06] <saml> mongodb is fine

[15:25:59] <symbol> I imaigne availability is improtant

[15:26:03] <symbol> *important

[15:26:15] <saml> just reserve some time to learn about mongodb operations. backup, replication, etc

[15:26:21] <Kosch> .oO( incremental backups too ;-)

[15:27:09] <symbol> Cool - I appreciate the sanity check.

[15:28:36] <jpfarias> hey guys

[15:29:11] <jpfarias> on a python application, is it better to keep one connection open to mongodb and reuse it for all requests or is it better to open a new connection for each request?

[15:33:09] <saml> jpfarias, one

[15:33:19] <saml> jpfarias, flask?

[15:34:13] <jpfarias> pyramid

[15:34:47] <jpfarias> I was just reading the pymongo MongoClient documentation and yeah, they recommend 1 MongoClient per application

[15:35:19] <jpfarias> saml: the reason I ask is my application is eating a lot of memory over time to the point the OS kills it

[15:35:53] <jpfarias> and I was suspecting MongoClient() was not releasing memory or whatever

[15:35:56] <saml> are you instantiating MongoClient on every http request?

[15:36:01] <jpfarias> saml: no

[15:36:06] <Kobbb> Hey guys, can anyone enlight me about Mongo users ? I'm trying to create a role and to apply it to my user. Also, the 'authenticationDatabase' is 'admin'. My problem is that I don't understand how to be able to set the rights to authenticate with the 'admin' base and to apply roles on my 'real' database to perform actions

[15:36:07] <jpfarias> right now it is just one mongoclient

[15:36:29] <d-snp> symbol: imo mongo is great for storing stuff like products, I'd outsource customer/order management though

[15:36:43] <Kobbb> I'm able to set rights on 'admin' or my

[15:36:57] <Kobbb> 'real' db, but I don't understand how to do both, restricting roles

[15:37:07] <symbol> d-snp: Doesn't that complexity start to create some overhead?

[15:39:14] <d-snp> what sort of complexity?

[15:43:05] <symbol> I figured by outsource you meant use a relational db to handle customer/order management

[15:51:27] <d-snp> symbol: ah no more like using stripe or a service like that, I myself don't like dealing with that, it's sensitive and mistakes cost money :P

[15:52:15] <d-snp> that said, I probably would not build an e-commerce site from scratch unless my business was to offer e-commerce as a service

[15:52:33] <StephenLynx> yeah

[15:52:53] <StephenLynx> theres a reason most places use 3rd party services for payments and stuff

[15:57:15] <symbol> Hmm - yeah, and I can't really think of anywhere else I'd need transactions.

[16:36:02] <atomicb0mb> hello, what does the "!" stands for here? $set: {checked: ! this.checked}

[16:36:44] <kali> atomicb0mb: it's a "not" operator

[16:37:08] <kali> atomicb0mb: it probably flips the "checked" boolean

[16:37:43] <atomicb0mb> kali, its doing the flip. yes. But i just can't understand the logic when you set the value "checked" to a "not this.checked"

[16:39:02] <kali> atomicb0mb: "this" comes from the context. it looks like it is a previously loaded version of the document you're updating

[16:40:03] <atomicb0mb> yeah, i think i got it. It checks the opposite value

[16:40:05] <atomicb0mb> =)

[17:57:11] <troypayne> What’s the simplest way to take a prod database and copy it as a dev_database… (P.S. they live on two completely different machines)

[17:59:00] <preaction> mongodump/mongorestore maybe?

[17:59:58] <kali> dump/restore, stop/copy_all_the_files/start

[18:01:20] <troypayne> preaction: when i do the mongodump, is it possible to rename he db? so it’ll be called dev_prod

[18:01:24] <troypayne> the*

[18:04:49] <troypayne> also i don’t need to stop the db for out of sync reasons

[18:05:08] <troypayne> it’s ok if it’s not in sync, i’d rather just keep the thing going if possible

[18:05:10] <preaction> kali was giving two different options

[18:05:26] <preaction> read the mongorestore docs on how to say what db to load into

[18:07:34] <troypayne> ok

[18:47:03] <saml> i have a collection where each doc has an array field. I want to get distribution of the array length

[18:47:35] <saml> should i scan the collection myself?

[18:47:51] <saml> something like 20% of docs have the array of length 1000

[19:18:51] <symbol> Are there any up-to-date speed comparisons between psql and mongodb? I can only find http://www.enterprisedb.com/postgres-plus-edb-blog/marc-linster/postgres-outperforms-mongodb-and-ushers-new-developer-reality which is pre 3.X

[19:20:45] <StephenLynx> even by then those benchmarks were bullshit

[19:21:10] <symbol> I gathered that...it's been hard to find anything reasonable.

[19:22:13] <mprelude> There are three types of lies: lies, damned lies and benchmarks.

[19:22:18] <symbol> :D

[19:24:39] <BadCodSmell> How do I wait for the mongod to start up?

[19:25:58] <cheeser> hrm?

[19:26:03] <BadCodSmell> all my stuff is screwing up because mongo daemonises then loads data then binds to a socket

[19:26:19] <BadCodSmell> What's the actually supported way to wait until mongodb has started?

[19:26:23] <BadCodSmell> actual*

[19:26:47] <cheeser> start mongod when your server comes up.

[19:26:54] <cheeser> then when you app starts, it'll be ready.

[19:27:31] <BadCodSmell> You mean /etc/init.d/ź̡͎̠̥̮̞̠̖ͅa̶͎͖͔͔͈̼͖͟͢͠ĺ̴̡̬͉̣̞͖͔͔̣͖͓̀g̵̷͖̩͇̣̮̦̺̟̙͔̟̰̯̹̖͘͠o͏̢͚̠͈̪̞̥͕̜̰̀d̵͍̞̳̬̟͉̩ start?

[19:27:49] <BadCodSmell> You mean I have to do sleep 666 behind every process that depends on zalgod?

[19:28:45] <BadCodSmell> What happens on a really bad day when Zalgo takes 667 seconds to get out of bed?

[19:29:07] <cheeser> whoa. garbled text on those last messages.

[19:30:19] <BadCodSmell> I suppose I'm stuck with while command to check for port bind on 27017 usleep 50000 if time > 60 seconds die timeout waiting for mongo.

[19:31:02] <cheeser> you shouldn't have to do that, no.

[19:31:12] <BadCodSmell> I obviously do.

[19:31:19] <cheeser> wth is zalgo?

[19:31:34] <BadCodSmell> mongo has a process of daemonise -> load data, etc -> bind to port

[19:32:15] <cheeser> i've always just started mongod on start up. then start my app. works fine.

[19:32:17] <BadCodSmell> The god of insanity inducing collapse of reality.

[19:32:50] <BadCodSmell> Hmm then perhaps the init script is doing it (wrapping in daemonistation).

[19:33:29] <cheeser> i don't know what this "demonistation" bit is all about. i've never heard of the script/process/step and have never needed it.

[19:34:04] <BadCodSmell> start-stop-daemon --background

[19:34:31] <preaction> daemonization is how a program detatches from its parent to ensure it doesn't die if/when its parent does (like a shell or script for example)

[19:34:46] <preaction> that's the "d" in "mongod"

[19:35:02] <cheeser> i know what a daemon is but there isn't some extra process you need for mongod to start that way.

[19:35:06] <BadCodSmell> Damn, all debian packages from mongo its self are using --background

[19:35:15] <BadCodSmell> That's what I'm wondering

[19:35:32] <BadCodSmell> I'm surprised that the way the init scripts are set up in debian are as if it can't daemonise its self

[19:35:40] <cheeser> there'sa "fork" option in the config file. start it via init.d using the scripts that come from the distro repos

[19:36:37] <BadCodSmell> rhel uses daemon() {} too

[19:36:44] <BadCodSmell> cheeser: That's what I'm using

[19:36:50] <BadCodSmell> distro init scripts

[19:37:41] <preaction> starting it via init.d requires daemonization, otherwise the init script (and the shell interpreting it) keeps running. yes, passing --fork and/or --background is how to daemonize the mongod

[19:38:05] <BadCodSmell> Basically the init.d script are doing it wrong?

[19:38:29] <cheeser> yes. that part i know. but he was making it sound like an extra script that just doesn't exist to my knowledge

[19:38:49] <cheeser> BadCodSmell: no, the script isn't doing it wrong (at least if you're using the official packages)

[19:38:54] <BadCodSmell> I know why

[19:39:02] <BadCodSmell> probably fork isn't handing the pid stuff, etc

[19:39:08] <cheeser> what?

[19:39:20] <BadCodSmell> I'm using official packages and they use --background

[19:39:29] <BadCodSmell> Which externally forks mongod

[19:39:34] <cheeser> that's correct

[19:39:50] <BadCodSmell> Which means it's completely unaware of when mongod has actually initialized.

[19:40:09] <BadCodSmell> Hence returning prematurally and continuing, starting other services or running other scripts before mongo is ready

[19:40:14] <cheeser> what's unaware?

[19:40:30] <BadCodSmell> whatever is triggering mongod to start

[19:40:40] <BadCodSmell> It has no idea that it has started, that being the init script

[19:40:54] <cheeser> the init script starts it and moves on

[19:41:10] <cheeser> whether or not mongod is ready for requests is a different story.

[19:41:11] <BadCodSmell> so if I do service mongod start;echo 'db.etc()' | mongo localhost/db < fail

[19:41:30] <BadCodSmell> and that's a problem

[19:41:55] <cheeser> is this on a desktop or something? because on a server this shouldn't be a problem.

[19:42:10] <cheeser> how much data do you have in your db?

[19:42:28] <BadCodSmell> 10KB on an ssd

[19:42:36] <cheeser> pffft. not a problem.

[19:42:55] <BadCodSmell> well whatever mongod is doing, it's too slow

[19:43:21] <cheeser> turn up the logging on mongod and see. but i'd be surprised if mongo's the problem here.

[19:43:53] <BadCodSmell> IT just takes ages to boot/start up because it's mongo

[19:44:55] <preaction> i just started it in 450ms, so ... no?

[19:45:24] <BadCodSmell> What if you have a batch script to init a system such as a docker image or ec2 image?

[19:45:32] <T-Sourcemaker> is there any way to speed up mongodump? I have a 16 GB collections and it's very slow...

[19:45:36] <BadCodSmell> There's it's not server versus desktop, it's just highly inconvenient

[19:45:49] <BadCodSmell> Andway I might as well just put in the kludge and get on with it.

[19:45:53] <BadCodSmell> Thanks.

[19:46:20] <cheeser> T-Sourcemaker: not really

[19:46:34] <cheeser> well, if docker's in the mix all bets are off.

[19:46:35] <BadCodSmell> T-Sourcemaker: copying the db files, if you know what you are doing

[19:46:56] <BadCodSmell> cheeser: any kind of script like that

[19:46:57] <T-Sourcemaker> BadCodSmell: I would like to upgrade my database to WiredTiger

[19:47:19] <T-Sourcemaker> and in the upgrade guide there is a chapter... mongodump and restore

[19:48:11] <BadCodSmell> ouch

[19:48:25] <cheeser> pretty much the only way outside of a replica set.

[19:49:32] <T-Sourcemaker> BadCodSmell: can I do a mongoimport from the current database folder?

[20:02:57] <joshua> Can you reconfigure a replica set to use a new name? Do I have to do something like start each server without replset enabled and then update the database entry? I hate when search on google and I get stackoverflow discussions from 2012

[20:07:12] <cheeser> you might be able to changed the name in the rs config doc. (retrieved via 'rs.config()') and then pass that to 'rs.reconfig()' but you'd still have to restart each member with the new name.

[20:07:28] <cheeser> and i'm just guessing there so back up your data blahblahblah :)

[20:11:38] <joshua> I'll probably just test it out with a replset on my local machine first anyway, Maybe it will work but it being the _id field kinda has my wondering. heh

[20:13:18] <troypayne> I’m trying to copy a database from one server to another. I’m confused on what my host user and its roles should be, and on what my source user and its roles should be. Thanks

[20:16:08] <troypayne> i keep getting not authorized for query on system.users\", code: 13

[20:46:35] <christo_m> is there anyway to clone from local to remote directly?

[20:46:45] <christo_m> i dont want to log into the remote and copy from there and have to open ports etc to access my machine here

[20:58:41] <repxxl> Hello, guys why this mongo query works only on the 'user_id' and 'name' but the other it ignores. http://pastebin.com/Tgpx9rRx

[20:58:50] <repxxl> php ver

[21:12:52] <repxxl> Only find uses indexes?

[21:12:58] <repxxl> or create ,remove too ?

[21:13:07] <cheeser> any query will

[21:13:14] <cheeser> that includes updates and removes.

[21:13:30] <cheeser> create will *update* an index with the new document info

[21:15:41] <repxxl> cheeser i had a idea to find my db records by 'name' but in that case i need to use regex like a search bar for records on my website, but i did read regex don't use indexes which will result in slow queries later on ... what do you suggest ? do the searching on the application side in my case (PHP)

[21:20:00] <cheeser> repxxl: regexs with fixed matches on the front and can. e.g., "blah.*"

[21:52:04] <T-Sourcemaker> is WiredTiger usable for production?

[22:14:44] <Derick> mprelude: making some progress?

[22:16:00] <mprelude> Derick: Not really. I'm getting a totally different response to the one in the example, and because there's no example with intermediate values shown, I can't see *where* I'm going wrong.

[22:16:26] <Derick> what sort of intermediate values?

[22:16:42] <mprelude> Derick: Value of the variables in the generation of the client final message.

[22:16:45] <Derick> i am wondering how to provide that for you

[22:17:20] <mprelude> Derick: If you could dump each of the vars in that step (step2, I think you called it), that would be huge

[22:17:42] <Derick> the C driver's?

[22:18:00] <mprelude> finalMessageBare, saltedPassword, clientKey, storedKey, authMessage, clientSignature, clientProof, serverKey, serverSignature, finalMessage

[22:18:08] <mprelude> As specified by the RFC

[22:18:13] <Derick> « doesn't know scram well

[22:18:19] <mprelude> I'm using username 'user' & password 'pencil' like the RFC.

[22:19:01] <Derick> mprelude: which timezone are you in?

[22:19:08] <mprelude> Derick: GMT

[22:19:18] <Derick> handy

[22:19:24] <Derick> you mean BST though? :)

[22:19:31] <mprelude> Derick: Probably, London. ;)

[22:19:35] <Derick> or are you in Iceland? :P

[22:19:37] <Derick> oh

[22:19:49] <Derick> I'm in London too

[22:20:01] <mprelude> I'm in Greenwich, I'm about as close to Greenwich Mean Time as one can get :)

[22:20:03] <Derick> we used to run office hours so you could just drop by, but not sure if we still do that

[22:20:24] <Derick> i reckon this is a hobby thing?

[22:20:51] <mprelude> It's for commercial purposes, but the cl-scram lib will be free software.

[22:22:32] <Derick> mprelude: see PM

[22:40:53] <Doyle> How accurate is the response in this ticket? https://jira.mongodb.org/browse/SERVER-19261

[22:42:15] <cheeser> pretty accurate: support questions should go to the mailing list and not the issue tracker.

[22:42:24] <Boomtime> that error shows an issue with data written during an initial sync, there aren't many reasons for that

[23:15:09] <mprelude> http://stackoverflow.com/questions/32468600/implementing-scram-sha1-client-getting-it-wrong-somewhere Upvotes appreciated

[23:35:11] <thomasreggi> Hey Mongo people! I'm in need of some help here: http://stackoverflow.com/questions/32468395/minimize-long-mongo-statment

[23:39:41] <cheeser> what do you mean by "conditionally upsert?"

[23:48:22] <thomasreggi> @cheeser if you take a look at my example

[23:48:38] <thomasreggi> @cheeser I don't want to push the name if it already exists in the array

[23:59:11] <curlhelp> Hey quick question here if anyone wants some free SO rep: http://stackoverflow.com/questions/32468889/mongodb-parent-with-child-ids-or-child-with-singular-parent-id

Log file Viewer

Help | Karma | Search:

#mongodb logs for Tuesday the 8th of September, 2015