PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 7th of July, 2016

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:33:04] <edrocks_> any using mgo(golang driver) bulk api to do inserts expierence many documents not being inserted?
[06:23:33] <bambanx> good good
[06:23:48] <bambanx> sup guys
[06:25:25] <Boomtime> hi there
[06:29:40] <bambanx> hello Boomtime
[06:31:19] <bambanx> guys how i can keep running mongodb, i am using webfaction is shared hosting, i used fork method but am not sure if i am doing well
[06:33:08] <bambanx> that looks like this /home/myuser/web/mongo/mongodb-linux-x86_64-3.2.7/bin/mongod --auth --dbpath /home/myuser/web/mongo/data/ --port 34012 --fork --logpath /home/myuser/logs/user/mongo_db.log
[06:33:53] <bambanx> and for stop the only way i founded is : killall -15 mongodb, kill pid is not working at all
[06:36:55] <bambanx> Boomtime, can you give me a hand ?
[06:38:17] <bambanx> also wich free gui for mongo are using guys?
[06:39:04] <Boomtime> does kill -15 pid work?
[06:39:14] <bambanx> yes
[06:39:18] <Boomtime> kill default is 3 i think.. but it's very similar to 15
[06:39:19] <bambanx> not sure what means -15
[06:39:26] <Boomtime> SIGTERM
[06:39:31] <bambanx> i founded on stackoverflow
[06:39:40] <bambanx> i am running looks ok
[06:39:42] <Boomtime> 3 is SIGQUIT and i think it's default - but maybe 15 is default
[06:39:48] <bambanx> but not sure if am doing well
[06:40:03] <Boomtime> whatever, mongodb will quit either way if the signal is actually sent
[06:40:22] <Boomtime> check the mongod.log to see if the signal is received - it prints all signals received
[06:40:29] <bambanx> and the fork i posted above what you think?
[06:40:32] <Boomtime> then does whatever it should
[06:40:36] <Boomtime> yeah
[06:47:10] <bambanx> Boomtime, do you use a gui ?
[06:47:39] <Boomtime> nope
[06:48:10] <Boomtime> but if i did it'd be compass - which doesn't meet your criteria of 'free'
[06:48:13] <bambanx> when you have thoundsans of data how you check if all is fine?
[06:48:18] <bambanx> testing?
[06:48:26] <Boomtime> how is a UI going to help with that anyway?
[06:48:32] <bambanx> dunno
[06:48:37] <Boomtime> and .. like, wut
[06:49:10] <bambanx> how is the name of a state of a database if is good or wrong? "integrity?
[06:49:20] <bambanx> if yes how you can see that?
[06:49:22] <Boomtime> i use my olfactory sense for checking data, i can smell danger
[06:49:37] <bambanx> what do you mean?
[06:49:47] <bambanx> you use like some mental powers?
[06:50:08] <Boomtime> mongodb stores data exactly as it receives it, there is nothing in the world that can tell you if it's "correct" because only you decide what is correct
[06:50:33] <Boomtime> if you can query your data, pretty much at all, then mongodb is doing it's job just fine
[06:50:44] <bambanx> should be tools for check what's going on with your data i think
[06:50:56] <Boomtime> but things like monitoring load are another ball game
[06:51:07] <Boomtime> use cloud manager -> cloud.mongodb.com
[06:51:18] <bambanx> is free?
[06:51:46] <Boomtime> @bambanx: that is a logistically impossible request - "check your data" is meaningless - what do you want to check? be specific
[06:52:21] <Boomtime> if you just want to know if stuff is accessible and general stats like size and counts etc, are there then the answer is cloud manager
[06:53:41] <Boomtime> so be specifc: what do you want to check?
[06:53:59] <bambanx> if i have an app for store : title and post for example, and for some unknown events , my database is saving data in a wrong way or the relations betwen user and posts are wrong, all that shpuld be tested on the app itself so no related to database?
[06:54:04] <bambanx> sorry for my bad english
[06:54:40] <Boomtime> the database doesn't know what your app intends to do with the data - how could it?
[06:54:54] <Boomtime> it's like spelling your name wrong - the database isn't going to know that
[06:55:16] <Boomtime> if you want to check that all the data is readable then you can run validate -> https://docs.mongodb.com/manual/reference/command/validate/#dbcmd.validate
[06:55:36] <Boomtime> but 99.999% of the time it is a completely pointless exercise
[06:56:34] <bambanx> ok thanks you Boomtime
[06:57:30] <bambanx> Boomtime, do you use cron for keep running the database?
[07:01:26] <Boomtime> er.. i suppose you could use cron.. not sure why you would
[07:01:46] <Boomtime> you know there are service start scripts for all the linux distros right?
[07:04:32] <bambanx> i dont have sudo
[07:05:49] <Boomtime> O_o
[07:05:54] <bambanx> as i know i cant run as a service without sudo
[07:06:00] <bambanx> is shared
[07:06:11] <bambanx> for now is running with fork
[07:06:28] <bambanx> just asked if is good idea add a cron for check it time by time
[07:08:34] <Boomtime> i'm sorry... are you a time traveler?
[07:09:16] <Boomtime> ok, yes, you could use cron to check it occasionally
[07:09:59] <Boomtime> there really shouldn't be any reason for the process to exit though - also, you asked about ways to terminate the server, you could also use the shutdownServer command from the mongo shell with appropriate permissions
[07:11:01] <bambanx> how do you know i am time traveler Boomtime ? are you john? john titor?
[07:11:08] <Boomtime> :p
[07:11:47] <bambanx> skynet will drop all the fire if i dont secure this database john
[07:13:49] <bambanx> Boomtime, compass does not have price on the page, do you know how much cost?
[08:59:47] <sandstrom> I've got issues with stacked fsynclock making it impossible to unlock using `db.fsyncUnlock()` Has anyone had this issue?
[09:46:48] <krion> hi guys, how can i investigate a chunk migration that result in a "data transfer error" ?
[10:01:40] <krion> how is the balancer host determined ?
[11:11:54] <Mmike> when using mongo cli - is there a way NOT to display some field from some document? I have a document that has some 'log' field which holds like 10k documents
[11:12:06] <Mmike> and printing that on screen is cumbersome, to say the least
[11:20:55] <deathanchor> Mmike: db.collection.find({ search : "forthis" }, { log : 0 })
[11:34:59] <Mmike> deathanchor: thnx
[11:35:09] <Mmike> deathanchor: and if that log is par of some sub-document?
[11:36:38] <deathanchor> Mmike: db.collection.find({ search : "forthis" }, { "sub.sub2.log" : 0 })
[11:36:57] <Mmike> deathanchor++
[11:36:57] <Mmike> thnx
[11:37:34] <deathanchor> Mmike: if you just want specific stuff you can do the inclusive way
[11:38:09] <deathanchor> Mmike: db.collection.find({ search : "forthis" }, { "want.this.only" : 1 })
[11:39:35] <Mmike> deathanchor: thnx - the first one was what I needed
[11:39:49] <Mmike> now just to figure out how to do it when I have this field in array of documents :)
[11:40:08] <deathanchor> Mmike: db.collection.find({ search : "forthis" }, { "sub[2].log" : 0 })
[11:40:22] <Mmike> you make all of that very simple :)
[11:40:29] <deathanchor> actually
[11:40:36] <deathanchor> just ignore the array part
[11:41:23] <deathanchor> suppose you have sub : [ { log : "longthing" }, { log : "otherlong" } ], just do this
[11:41:29] <deathanchor> Mmike: db.collection.find({ search : "forthis" }, { "sub.log" : 0 })
[11:58:17] <Bengt__> What kind of websites/ web apps benefits from using noSQL rather then a SQL database?
[12:02:50] <deathanchor> Bengt__: depends on your application design and preferred design methods
[12:03:38] <deathanchor> GridFS is the thing boasted about mostly with noSQL.
[12:14:41] <Bengt__> So for most normal uses sql is better to use?
[12:47:12] <mroman> Can you count in {foo: [8, 7]},{foo: [9,1]},{foo: [8,4} how many times each number occurs without doing an unwind?
[12:48:02] <mroman> Currently I do an unwind {$unwind : foo},{$group : {_id : '$foo', count: {$sum : 1}}}
[13:39:45] <cslcm> hi folks, what's the difference between a multikey index and a compound multikey index?
[13:40:55] <Derick> multikey means multiple values in an index entry
[13:41:09] <Derick> if you have { a: [ 1, 43, 5 ] } and an index on "a", it's a multikey
[13:41:27] <cslcm> right, and what does compound mean?
[13:41:37] <cheeser> compound index means indexing more than one field at once
[13:41:42] <Derick> if you have { a: 1, b: 5 } , and an index on a and b
[13:41:42] <cslcm> oh right, ok
[13:43:04] <cslcm> If I wanted to search for documents with A = hashed index OR A in multikey index ordered by a date index, is that doable in an efficent way?
[14:48:56] <cslcm> Does the $in operator use indexes?
[14:49:30] <Derick> it should - but use explain to make sure it does
[15:02:51] <cslcm> Derick: I'm quite new so the explain is rather tricky for me to read :/
[15:09:49] <deathanchor> cslcm: you can pastebin the explain, but the important stuff is the nscanned stuff, also look at the explain() docs online
[15:15:00] <cslcm> nti@87.253.171.209) has left #mongodb
[15:15:02] <cslcm> er
[15:15:03] <cslcm> wtf
[15:15:07] <cslcm> http://pastebin.com/raw/LLu9Jm6w
[15:15:25] <cslcm> i'm guessing that the sort i'm using is eliminating the use of keys
[15:15:28] <cslcm> indexes
[15:15:50] <cslcm> since i don't see anything to do with nscanned in there
[15:16:49] <deathanchor> oh this is 3.2 mongo? and looks like on a sharded cluster
[15:17:29] <cslcm> mongodb-3.2.5 and yes, sharded cluster
[15:18:31] <deathanchor> hmm... I haven't done much troubleshooting queries in 3.2 yet. mostly working in 2.x world.
[15:19:19] <deathanchor> is the shardkey on entityUsers?
[15:21:00] <deathanchor> cslcm: if you query through mongos and it spans multiple shards, the sort will happen on the mongos side instead of on the mongod server.
[15:21:21] <deathanchor> basically it will query the mongod servers, get results, then merge and sort on the mongos.
[15:22:54] <deathanchor> this becomes super painful when you do stuff like skip() with a sort() on mongos as it will query mongod, sort on mongos, and then mongos skips to wherever
[15:23:41] <cslcm> right, will it do an index assisted search?
[15:23:42] <cslcm> er
[15:23:43] <cslcm> sorry
[15:23:46] <cslcm> right, will it do an index assisted sort?
[15:23:49] <deathanchor> I don't know if 3.2 fixes the problem but in 2.x it would have mongos do the sort/skipping even if your query had the shardkey only hit data on one mongod.
[15:24:06] <cslcm> and no the shardkey is on objectKey, not used in this query
[15:24:12] <deathanchor> cslcm: not sure, mongos is a bit of a blackbox to me.
[15:24:31] <deathanchor> it might not use an index
[15:24:34] <cslcm> ok, well thanks for your help anyway :) it's appreciated
[15:25:30] <deathanchor> it might do the sort on mongod server, but then has to merge sorted lists on the mongos
[15:25:59] <spleen> Hello all
[15:26:49] <cslcm> hiya
[15:26:55] <spleen> I would like to check if my mongo are sync via optimeDate value. Do you know a good way ? a nagios check to do that ?
[15:27:06] <deathanchor> it's funny, mongos is described as lightweight, but sometimes it does hog up some ram for these cross cluster sort/merges.
[15:27:28] <deathanchor> spleen: yes let me grab a link
[15:28:02] <deathanchor> spleen: https://github.com/mzupan/nagios-plugin-mongodb
[15:28:18] <deathanchor> has lots of fancy checks you can setup
[15:28:23] <deathanchor> fairly simple
[15:29:51] <spleen> deathanchor, cool. Thanks !
[15:31:44] <spleen> deathanchor, arr.. i have to install python on my prod servers.. is there a bash check to do that ?
[15:35:18] <deathanchor> spleen: you don't need to install it on your prd servers
[15:35:42] <deathanchor> you can install python on your monitor server and have it connect via mongo to your prd servers
[15:36:29] <deathanchor> there is host/port options for the command
[15:37:01] <spleen> deathanchor, ok ! i have just to install python on my monitoring server
[15:37:35] <deathanchor> yeah then configure your check to run the command local on the montior server with the -H -P options to connect to where it should
[15:37:47] <spleen> question : what is the difference between optime & optimeDate via rs.status() command in Mongo ?
[15:41:10] <GothAlice> deathanchor: On your earlier comment about light-weightness; all database services can be reasonably expected to consume all available resources on their respective hosts. They don't play well with others, and this is a good thing, to keep data in RAM cache as much as possible. One of the difficulties with sorts/merges is that they can't easily be streamed, you must gather the results first, and that takes RAM.
[15:48:52] <spleen> deathanchor, on my monitoring server i need wish paquage just for connect to mongo ? mongo-cli something like that ?
[16:02:22] <Mmike> Hi, again! :) How do I copy documents across collections? I tried this, but it won't work: db.collectionB.insert(db.collectionA.find({"_id":"myNeededId"}))
[16:02:31] <Mmike> but I'm getting: can't save a DBQuery object at src/mongo/shell/collection.js:134
[16:04:52] <cheeser> you'll have to query, iterate, insert
[16:05:26] <Mmike> like, .find.forEach(function(doc_
[16:05:28] <Mmike> ... ?
[16:05:42] <cheeser> if you're in the shell, yeah.
[16:05:45] <kurushiyama> Mmike http://stackoverflow.com/questions/27039083/mongodb-move-documents-from-one-collection-to-another-collection/27041518#27041518
[16:05:50] <kurushiyama> Mmike Skip the remove part
[16:06:31] <Mmike> thnx, kurushiyama , cheeser
[16:11:43] <spleen> The best information to know if mongo rsm are sync or not according to you, is optime -> ts when rs.status() or timeDiff when db.getReplicationInfo() ?
[16:16:11] <kurushiyama> spleen Compare the optimes of rs.Status()
[16:18:20] <spleen> kurushiyama, optime is the same thing of OptimeDate but in timestamp mode (then i exec rs.status())?
[16:18:46] <kurushiyama> spleen Aye
[19:13:27] <Mmike> Hola, again :) I need to somehow prettyprint mongo collection
[19:13:38] <Mmike> it's just three documents but one of the fields in one docs is huge
[19:13:58] <Mmike> I tried several tools but they all claim that my document is flawed: http://jebo.me/pas/1g
[19:14:30] <Mmike> I created stateServers.json using bsondump agains the .bson file created by mongodump
[19:18:50] <jayjo> I have asked a similar question here before - is there a way to have a user defined function or VIEW in mongodb? My very particular use case right now is to return a result (like views, or clicks) for the last 10 days at any given day. Can I have a stored query with a dynamic aspect somehow, or is it entirely dependent on the api I access the DB with?
[19:19:51] <jayjo> Or could I automatically create a collection that is just the last 10 days, etc?
[19:19:57] <jayjo> SO I could just query the collection?
[19:20:25] <cheeser> 3.4 will have views.
[19:23:01] <Doyle> Mmike, mongo --eval "printjson(rs.status()orwhatever)" ?
[19:23:24] <Doyle> you can pipe the printed json to jq
[19:26:21] <jayjo> cheeser: so with 3.2 there really isn't a way to acheive something like that?
[19:26:27] <jayjo> Other than a cron job that creates a new collection?
[19:28:35] <Doyle> jayjo, no, that doesn't sound like a db function. That sounds like a 3rd party analysis tool
[19:29:45] <cheeser> jayjo: not in 3.2, no
[19:30:26] <cheeser> you could always dump these documents in to a new collection with a TTL index...
[19:32:05] <Mmike> Doyle, that wont't work for some reason - I dod: --eval "printjson(db.mycollection.find())"
[19:32:22] <Mmike> I got a bunch of javascription functions and code
[19:32:25] <Mmike> instead of my document
[19:36:15] <Doyle> For .find() operations, you need to call an operation on the result object to print the documents, such as toArray() or shellPrint(). e.g., mongo userdb --eval "printjson(db.users.find().toArray())" - http://stackoverflow.com/questions/4837673/how-to-execute-mongo-commands-through-shell-scripts
[19:37:06] <Mmike> Doyle, excellent, thnx!
[19:37:15] <Doyle> np, enjoy
[19:57:29] <jayjo> cheeser: would that not just kill the index and not the document?
[20:04:27] <jayjo> is that right - the _id field doesn't not support TTL indexes?
[20:36:56] <deathanchor> jayjo: I believe so. TTL index cannot be primary index.
[20:54:38] <cschneid_> I'm looking at the drivers in ruby - is moped deprecated / for older versions - and mongo-ruby-driver newer? How are they related/different?
[23:03:13] <bambanx> Hello guys
[23:47:03] <dwn> hi is this the pokemon go channel