pmxbot IRC Log Viewer

[02:57:58] <cppking> hello guys, I'm using mongodb3.2.0 replset, and use pymong-3.2.2 client, I found this problem , I can't always get same output https://bpaste.net/show/891ff15b1220

[03:09:32] <cppking> any help?

[07:54:39] <ren0v0> Hi, how long until 16.04 is officially supported?

[07:54:45] <ren0v0> i mean, official builds

[08:13:25] <joannac> ren0v0: https://jira.mongodb.org/browse/SERVER-23043

[08:14:21] <joannac> ren0v0: within a couple of months (was said 1 month ago)

[08:24:27] <ren0v0> joannac, thankyou

[10:56:37] <qliam> hi

[10:56:48] <qliam> i have a problem with my mongodb server on aws

[10:56:56] <qliam> i have a very high cpu usage

[10:57:12] <qliam> my iops on data & journal are good

[10:57:17] <qliam> not overhead

[10:57:54] <qliam> anybody has an idea ?

[10:57:57] <Boomtime> can you provide some real numbers?

[10:58:07] <qliam> yes

[10:58:17] <qliam> it's a c4.8xlarge

[10:58:30] <qliam> i provided 3000iops on data

[10:58:43] <qliam> i have currently 400 connections opened

[10:58:53] <qliam> and my cpu is at 100%

[10:59:08] <qliam> my entire database contains ~5millions objects

[10:59:19] <Boomtime> the c4.8xlarge has 36 cpus yes?

[10:59:32] <qliam> yes

[10:59:35] <Boomtime> can you provide a top output?

[10:59:42] <qliam> all of them are 100*

[10:59:44] <qliam> all of them are 100%

[10:59:48] <Boomtime> also a serverStatus from mongodb

[11:00:00] <qliam> top - 10:51:39 up 2:23, 2 users, load average: 61.78, 58.69, 63.43 Tasks: 310 total, 1 running, 309 sleeping, 0 stopped, 0 zombie Cpu(s): 98.6%us, 0.7%sy, 0.0%ni, 0.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 61836664k total, 3547424k used, 58289240k free, 50932k buffers Swap: 0k total, 0k used, 0k free, 2769584k cached

[11:00:02] <Boomtime> is it functional? like, is mongodb even working?

[11:00:20] <chris|> when reconfiguring a replica set with replSetReconfig, is it sufficient to apply the config to all members or is the primary sufficient?

[11:00:52] <qliam> it's functionnal but i have latency

[11:01:06] <qliam> yes chris

[11:01:22] <Boomtime> @qliam: in a gist; full top output, also db.serverStatus() and db.currentOp()

[11:01:38] <qliam> ok

[11:02:09] <jsheely> Hey all, Anyone know how to $addToSet but only if part of the object exists?

[11:02:20] <Boomtime> @chris|: on the primary only - it will pass it to all

[11:02:37] <jsheely> or rather, not $addToSet

[11:02:50] <Boomtime> @jsheely: huh? can you give an example?

[11:03:01] <jsheely> Sure

[11:03:05] <chris|> Boomtime: alright, thanks

[11:04:12] <jsheely> document with an array. Array contains {key: 1, value: 'foo'}. I want to to $addToSet but only if the key doesn't exist.

[11:04:46] <qliam> db.serverStatus : https://gist.github.com/qliam/9654c8381cf9ffb179bab84b8a87fe45

[11:04:52] <jsheely> But $addToSet considers the whole object. And I don't know what the value is

[11:07:00] <chris|> btw. is there a way to rebuild a config server from scratch if only the shards are available?

[11:09:08] <joannac> chris|: no

[11:09:13] <qliam> db.currentOp() : https://gist.github.com/qliam/2bf13d42212a88c4f6004dc18d224da3

[11:09:16] <qliam> thx

[11:10:05] <joannac> you still have all of your data, chris|, but it might be a bit tricky to piece it all back.

[11:10:13] <joannac> chris|: how did you lose all your config servers?

[11:10:36] <chris|> joannac: I didn't, it's a hypothetical question

[11:11:05] <joannac> oh

[11:11:09] <joannac> well, there you go

[11:11:57] <qliam> @Boomtime, top : https://gist.github.com/qliam/0e7fddbbfb3cc4e67ffb0d903b4554f8

[11:12:04] <chris|> since there is no way to backup all shards and all configs, there is bound to be a difference between the backups, and rebuilding something partially is basically the same as rebuilding it completely

[11:15:32] <joannac> chris|: um what? https://docs.mongodb.com/manual/administration/backup-sharded-clusters/

[11:17:07] <qliam> Boomtime have you enough informations ?

[11:17:44] <Boomtime> i didn't go though everything, but you have a lot of collscans going on - every operation is a collscan in fact

[11:18:00] <joannac> Also sorting in memory, looks like

[11:18:20] <Boomtime> collscans require the documents to be decompressed to be read

[11:18:28] <Boomtime> and sorting is also cpu

[11:19:27] <qliam> ok

[11:19:31] <Boomtime> @qliam: https://docs.mongodb.com/manual/indexes/

[11:19:43] <joannac> chris|: Cloud Manager also does a lot to try and get consistent backups of a sharded cluster.

[11:20:09] <qliam> so i have to create more index ?

[11:22:48] <chris|> joannac: if for some reason (hypothetical disaster) all config servers die bit shards are still there, the last backup of the config will be outdated compared to the shards. Since shard data is much larger as config data, I would like to not have to rely on restoring the shards from backup and then throw the oplog on top if I could avoid it. Hopefully thats clearer

[11:24:05] <e419> Hi, everyone :)

[11:24:05] <e419> I have faced strange issue while trying to use mongodb mapReduce http://dba.stackexchange.com/questions/140012/error-13141-in-mongos-shell-while-trying-to-write-to-a-result-of-the-mapreduce-i

[11:24:37] <e419> Sadly stackexchange did noting to help me to identify the cause.

[11:25:14] <e419> Please, help me to dig into this one. http://dba.stackexchange.com/questions/140012/error-13141-in-mongos-shell-while-trying-to-write-to-a-result-of-the-mapreduce-i

[11:27:38] <qliam> Boomtime, is it possible de create index on sorting fields ?

[11:28:10] <Boomtime> @qliam: https://docs.mongodb.com/manual/tutorial/sort-results-with-indexes/

[11:28:18] <qliam> thx :)

[11:29:34] <e419> @qliam this affects my issue too?

[11:30:16] <joannac> chris|: rebuilding the config servers will almost certainly take much longer than that

[11:34:17] <e419> Guys, if anyone knows, could you please tell us why we can get "Chunk map pointed to incorrect chunk" with code 13141 when we try to write (out->replace) the result of mapReduce into a separate collection with sharded: true?

[11:35:27] <Exander> this issue is described in more detail on dba.stackexchange, but as of now, no one's answered yet, and it's a high-priority blocker for us...

[11:36:30] <e419> Collective unconscious please help

[11:36:37] <Boomtime> do you have support? you should raise a ticket if you do

[11:37:03] <Boomtime> otherwise that sounds quite complicated

[11:37:33] <Exander> no, we don't we have a devops responsible for the mongodb, and he can't find the reason, either

[11:39:12] <e419> Can you give us a mailing list to write to and ask, what should we do to get support and possibility to raise tickets?

[11:41:03] <joannac> e419: Contact someone in sales to discuss https://www.mongodb.com/contact

[11:41:22] <e419> Thank you

[11:43:03] <yopp> is there official docker image for mms around?

[12:18:01] <seprob> Hi.

[12:53:37] <Ange7> Hello all

[12:54:02] <Ange7> someone can help me how to optimize my aggregate ? http://pastebin.com/y9i0Kr4p it's very very very slow...

[13:39:51] <Ange7> someone can help me to optimize my aggregate function ?

[13:40:02] <Ange7> aggregate pieline*

[13:57:44] <StephenLynx> ['$group' => ['_id' => ['name' => '$name'],

[13:57:47] <StephenLynx> why that?

[13:58:08] <StephenLynx> can't you just use '$name' directly instead of making it an object with a key called name?

[13:58:42] <StephenLynx> the match stage doesn't seem to have any issues.

[13:59:12] <StephenLynx> how many entries per appcode you have?

[13:59:37] <StephenLynx> you could use an index on date too if you have too many.

[15:03:35] <Ange7> stephen i have index.

[15:03:45] <Ange7> StephenLynx: *

[15:04:01] <StephenLynx> on date as well?

[15:04:07] <Ange7> Yes

[15:04:23] <StephenLynx> how many documents are you aggregating at once?

[15:04:31] <Ange7> 100.000.000

[15:04:40] <StephenLynx> how much time does it take?

[15:05:02] <Ange7> i never go at end (too long)

[15:05:17] <StephenLynx> "name" : "date_1_appCode_1",

[15:05:20] <StephenLynx> are you SURE

[15:05:23] <StephenLynx> this index is right?

[15:06:28] <Ange7> i have index on [date: 1, appCode: 1] and [date: 1]

[15:06:40] <StephenLynx> why is the date index compound?

[15:07:09] <Derick> Ange7: if you have [date: 1, appCode: 1] you don't need [date: 1]

[15:07:22] <Ange7> oh... ok.

[15:07:41] <Ange7> cause i filter on date

[15:08:21] <StephenLynx> I don't think you have a performance problem, I just think you are trying to aggregate too much at once.

[15:15:28] <tantamount> Is it possible to use {$inc: -1} with $max?

[15:15:38] <tantamount> i.e. to limit the amount to which a field can be decremented?

[15:15:53] <tantamount> I tried: db.test.update({_id: 1}, {$inc: {foo: -1}, $max: {foo: 0}})

[15:15:59] <tantamount> But I get: "errmsg" : "Cannot update 'foo' and 'foo' at the same time"

[15:16:30] <Derick> i think you can only use one field once

[15:16:49] <tantamount> That certainly seems to be the implication, but how can I achieve what I want to do?

[15:17:16] <tantamount> I am trying to implement a concurrency counter that is incremented when an action needs to be performed and decremented once it is performed

[15:17:22] <StephenLynx> i dont think so.

[15:17:28] <StephenLynx> that its possible, that is.

[15:17:47] <tantamount> However, we must ensure the counter does not drop below zero in case the action is performed by two processors in parallel

[15:18:34] <StephenLynx> I had a similar issue, to be honest.

[15:18:45] <StephenLynx> I ended up just doing a full reaggregation after the operation.

[15:18:57] <StephenLynx> not as efficient, but reliable.

[15:34:40] <shoconinja> any reason when trying to add a node to a replica set it can't find the external user?

[15:34:42] <shoconinja> i'm using x509

[15:36:32] <shoconinja> https://gist.github.com/thebenwaters/a89fc037ac51faca3ba88b26f816a9e3

[15:36:40] <shoconinja> i can see the user in the main node with the rs.initiate

[16:02:43] <Ange7> what is the problem with this update query : http://pastebin.com/LQQjctxj ? i don't understand why i can't insert new line ? (with upsert option to TRUE)

[16:06:31] <ksan> hi guys

[16:07:30] <ksan> who can help me with replicaset integrity check?

[16:08:06] <ksan> i want to know do i have equal amount of data o primary and secondary

[16:08:51] <ksan> because when i'm trying to do rs.stepdown() - i have no response from my mongos

[16:17:37] <Ange7> quit

[16:26:51] <StephenLynx> kek

[16:27:02] <StephenLynx> he tried made a pardoxical query

[16:34:36] <chuy> Does $elemMatch stops looking for elements when it finds one element that matches the criteria? In other words, does $elemMatch behave like javascript's Array.some?

[16:35:10] <StephenLynx> yes, it matches only the first match on a projection.

[16:36:08] <chuy> ah, thanks !

[16:39:11] <StephenLynx> however, I think it behaves differently on match stages.

[16:39:23] <StephenLynx> I remember it being a little ambiguous, so I suggest you read on it.

[16:42:57] <chuy> The doc isn't detailed on that behavior. It says something along the lines of: 'matches documents where at least one array element passes the query criteria', but doesn't specify if it stops looking once there is a match

[16:50:07] <poz2k4444> Hi guys, I'm using mongo-connector to sync my mongo collection to elasticsearch but I cannot make mongo-connector to update the index when a document in the collection was updated, can anybody help me with this

[16:50:08] <ksan> guys, who can help me with replica set?

[16:54:02] <ankurjoveo> Hi, can anyone help me out with the recommended prod settings for the scala client.

[17:23:26] <renlo> whats the fastest way to serialize data from mongo?

[17:23:42] <saml> bson

[17:25:05] <renlo> so are you saying that the documents should be stored as bson?

[17:26:12] <saml> what

[17:26:36] <saml> if you're reading from mongo, use bson

[17:26:59] <renlo> are you referring to the library or the format

[17:27:49] <saml> i'm not sure what you're asking. mongodb stores bson

[17:27:57] <saml> so reading bson is fast

[17:28:32] <renlo> ahh

[18:09:49] <ankurjoveo> Anyone know anything about the scala client?

[18:09:54] <ankurjoveo> or used it before

[18:10:48] <saml> which one?

[18:10:57] <saml> reactmongo?

[20:28:08] <fakingfantastic> I’ve got a db called “api”, how to I copy all of the collections to a db on the same server called “api_development"

[21:16:16] <Ben_1> hi

[21:16:54] <Ben_1> how is mongoDB communicating between its async java driver and the server? is it message based communication or just asynchronous rpc?

[21:21:25] <cheeser> the wire protocol is the same as the regular driver.

[21:35:57] <Ben_1> thx cheeser :)

Log file Viewer

Help | Karma | Search:

#mongodb logs for Wednesday the 1st of June, 2016