pmxbot IRC Log Viewer

[02:36:28] <jiffe> so I'm trying to run db.runCommand({cleanupOrphaned: "mail.message_data"}) and I'm getting "errmsg" : "Error encountered while deleting range: nsmail.message_data from { message_identifier ... cause by: :: caused by :: 10334 BSONObj size: 1542706610 (0x5BF3D5B2) is invalid. Size must be between 0 and 16793600(16MB)

[03:49:42] <Waheedi> is there a way to make a client interact with one specific node only. that node is obviously part of a replicaset

[03:55:55] <Waheedi> I wish i had two primaries :D

[04:04:37] <Boomtime> @Waheedi: yes, you can create a direct connection - every driver supports this, but if the node is secondary you won't be able to write to it (only read)

[04:05:01] <Waheedi> and thats the problem :)

[04:06:53] <Waheedi> Boomtime: I think its better to have a single mode/ rs mode kind hybrid

[04:07:19] <Waheedi> fly

[04:08:04] <Boomtime> what problem are you actually trying to solve?

[04:08:16] <Waheedi> many

[04:08:29] <Waheedi> not really sure want to start that convo

[04:08:42] <Boomtime> the only common reason to talk to a specific member is to get operational stats/control of it

[04:09:00] <Waheedi> i actually have servers across different physical places

[04:09:06] <Boomtime> otherwise you should be using one of the built-in mechanisms for describing your intent

[04:09:44] <Waheedi> and nearest option in read preference does not consider primary to be a nearest node even if it was 0ms ping away

[04:09:50] <Waheedi> thats one problem only

[04:10:24] <Waheedi> maybe that was the case only in 2.4.1

[04:10:27] <Boomtime> ok; it sounds like you have multiple sites and want to constrain reads occurring in one site to members in that site - is that a fair description?

[04:10:35] <Waheedi> but have not tried the 3.x yet

[04:10:53] <Waheedi> Boomtime: yeah might be close

[04:11:13] <Boomtime> https://docs.mongodb.org/manual/tutorial/configure-replica-set-tag-sets/#configure-tag-sets-for-functional-segregation-of-read-and-write-operations

[04:31:40] <jiffe> when I try to start a couple nodes I'm getting [repl writer worker 15] writer worker caught exception: :: caused by :: 10334 BSONObj size: 1542706610 (0x5BF3D5B2) is invalid

[04:31:47] <jiffe> is there a way I can just skip that?

[04:34:42] <jiffe> this used to happen with mysql where replication would fail due to a problem with a record and you could set the skip counter, replication would skip the record and go on its merry way

[04:35:46] <Waheedi> jiffe: i used to find these records and delete them from primary

[04:36:25] <Waheedi> maybe take a backup of that record

[04:38:14] <jiffe> so when I start up these nodes they're not trying to read this out of a local log, its pulling all the replication data from the master?

[04:39:35] <Waheedi> your question has two parts :)

[04:39:42] <Waheedi> i can't answer :D

[04:42:16] <jiffe> well it is either reading replication data from a local log or from the primary

[04:44:09] <jiffe> both nodes that are down are failing on different records, neither of which is in the primary

[04:44:40] <jiffe> I was doing a cleanOrphan on the primary so it could be from that

[04:45:16] <jiffe> suppose it would be in the primary oplog

[09:23:58] <mementomori> hi

[09:25:06] <mementomori> I'm attending m101p class and I need a little help understanding an homework I've already failed.

[09:26:39] <mementomori> the homework wants me to answer true or false on a set of 5 questions

[09:27:27] <mementomori> hw title: Which of the following statements are true about choosing and using a shard key?

[09:27:52] <mementomori> 1) There must be a index on the collection that starts with the shard key.

[09:28:09] <mementomori> here I answered True

[09:28:27] <mementomori> 2) MongoDB can not enforce unique indexes on a sharded collection other than the shard key itself, or indexes prefixed by the shard key.

[09:30:26] <mementomori> here I asnwered False because it could be possibile that data is stored on different shards and mongo can't enforce uniqueness

[09:30:59] <mementomori> 3) Any update that does not contain the shard key will be sent to all shards

[09:32:31] <kali> 2/ was true

[09:33:06] <mementomori> here I answered True. it is true if "multi" is true... The question doesnt tell anything about "multi" so I was in doubt

[09:33:22] <mementomori> kali: why?

[09:33:28] <kali> 3/ was true too

[09:33:38] <kali> even without multi

[09:34:08] <mementomori> kali: so "multi" is optional?

[09:34:39] <mementomori> 4) You can change the shard key on a collection if you desire.

[09:34:45] <mementomori> this is False

[09:34:49] <kali> for 2) if two values are identical and contains the shard key, then the shardkey is identical and the values go to the same shard, so mongodb can ensure they are unique

[09:36:07] <mementomori> kali: 2) also says "or indexes prefixed by the shard key."

[09:36:41] <mementomori> and that could cause data to be in different shards

[09:37:32] <mementomori> in my understanding

[09:37:39] <kali> well, you're wrong :)

[09:38:08] <mementomori> lol. Sure... in fact I failed the hw

[09:38:11] <kali> :)

[09:38:17] <kali> ok

[09:38:32] <kali> how to phrase that

[09:38:57] <kali> if you have shard key { a:1 }

[09:39:09] <kali> and want uniqueness on { a:1, b:1 }

[09:39:28] <kali> we have a unique index which is prefixed by the shard key, right ?

[09:39:32] <mementomori> sure

[09:41:01] <kali> so if two documents have the same a and b (which is what you want to forbid), then they necessarily have the same a, right ?

[09:41:35] <mementomori> sure

[09:42:00] <kali> so they go to the same shard

[09:42:10] <kali> so the unique index will work

[09:42:11] <mementomori> I'm not sure about that

[09:42:23] <mementomori> I could have a:1 b:1 on s0 and a:1, b:2 on s1

[09:42:23] <kali> hon you can be sure about taht

[09:43:03] <mementomori> becouse "a" is the shard key

[09:43:29] <kali> so what ? you want uniqueness on both a,b

[09:43:33] <mementomori> do you see what I mean?

[09:43:52] <kali> wait

[09:44:00] <kali> no.

[09:44:09] <kali> "I could have a:1 b:1 on s0 and a:1, b:2 on s1" <- this is wrong.

[09:44:28] <kali> two documents with the same shard key are going to the same shard. period.

[09:44:30] <mementomori> ok. this is my misunderstand

[09:44:54] <mementomori> d'oh!

[09:45:08] <mementomori> you are right

[09:45:19] <mementomori> now it's clear also to me

[09:45:48] <mementomori> 5) The shard key must be unique

[09:46:00] <mementomori> here I answered true

[09:46:16] <kali> it's false

[09:46:47] <mementomori> I can have a single shard key for colletions do it is unique by definition...

[09:47:04] <mementomori> I can have a single shard key for a colletion so it is unique by definition...

[09:47:12] <kali> i think it is meant as in "unique index"

[09:47:58] <mementomori> ok.

[09:48:46] <mementomori> so the right answers are: true, true, true, false, false ?

[09:49:02] <kali> i'm not 100% sure about the update thingy

[09:49:24] <mementomori> kali: because of "multi" ?

[09:49:53] <kali> no. the multi case is actually the trivial one. you want all records updated, so you send the query everywhere

[09:50:05] <kali> the one that is bothering me is the non-multi update

[09:50:19] <kali> because the mongos will not know where to send the query

[09:50:46] <kali> and if it sends it to all the nodes, we may actually have multiple updates (each node would perform one)

[09:50:47] <mementomori> I suppose it will send the query to every shard

[09:50:54] <kali> so i'm not sure all that is supposed to work

[09:51:02] <kali> s/all/how/

[09:51:21] <mementomori> ok

[09:51:31] <kali> https://docs.mongodb.org/manual/reference/method/db.collection.update/#sharded-collections

[09:51:35] <mementomori> thank you very much for your support

[09:51:42] <kali> ok, that's the answer

[09:52:11] <mementomori> so multi is not optional

[09:52:56] <kali> well, you need either multi, either the the shard key in the selector, either the _id

[09:52:59] <kali> or else you get an error

[09:53:21] <kali> so 3/ is true

[09:53:21] <mementomori> ok so the answer for 2) is False

[09:53:31] <mementomori> ops

[09:53:36] <mementomori> ok so the answer for 3) is False

[09:53:50] <kali> no, it's actually true

[09:54:26] <mementomori> 3) Any update that does not contain the shard key will be sent to all shards.

[09:54:32] <kali> yeah.

[09:54:40] <mementomori> the answer is that I get an error

[09:55:06] <kali> well it depends :)

[09:55:25] <mementomori> the update to works needs multi or _id

[09:55:40] <kali> yeah. either multi, or _id, or the shard key

[09:55:43] <mementomori> but it asked for "any"

[09:55:58] <kali> mmmm yeah

[09:56:11] <kali> ok

[09:56:29] <mementomori> I think that this hw is someway confusing.... :)

[09:56:29] <kali> i would assume it means "any working update", but... well, i'm not sure

[09:56:47] <kali> kinda

[09:57:40] <mementomori> anyway... thank you for your explaination about the unique index

[09:57:54] <mementomori> there I was totally wrong

[09:58:12] <kali> you're welcome

[09:58:35] <mementomori> have a nice week end

[09:58:38] <mementomori> bye!

[09:58:40] <kali> bye

[14:07:21] <jiffe> when I try to start a couple nodes I'm getting [repl writer worker 15] writer worker caught exception: :: caused by :: 10334 BSONObj size: 1542706610 (0x5BF3D5B2) is invalid, is there a way to skip it?

[14:43:19] <jiffe> so do slaves copy objects from the master's oplog to the local oplog before applying?

[15:07:21] <jiffe> I searched for db.oplog.rs.find({h: 8979387659673550781}) on both the primary and secondary and it is not found in either, so where is this coming from?

[16:43:46] <jiffe> ok so I found the oplog record in the master

[16:44:55] <jiffe> there doesn't seem to be anything I can do about it, I can't remove it from the oplog nor can I remove the referenced record from the recovering node

[16:46:24] <kali> jiffe: i would fix the database on the master (by discarding or reducing the document too big) and then resync

[16:47:52] <jiffe> kali: resyncing these boxes takes about a week and this seems to happen frequently so I'm looking for a quicker route

[16:48:03] <Zelest> how can I decide what fields I wish to return using the ruby driver?

[16:52:41] <Zelest> oh, nevermind! .projection() is what I was looking for. :)

[16:54:58] <jiffe> maybe I can modify the master oplog record to delete a record that doesn't exist instead of the one its having a problem with

[17:22:34] <jiffe> trying to update { "ts" : Timestamp(1455116189, 1), "h" : NumberLong("8979387659673550781"), "v" : 2, "op" : "d", "ns" : "mail.message_data", "fromMigrate" : true, "o" : { "_id" : ObjectId("54a622d255a8275d4b000b0f") } }

[17:22:43] <jiffe> using db.oplog.rs.update({h: NumberLong("8979387659673550781")}, { $set: { o: { _id: ObjectId("54a622d255a8275d4b000b0e") } } })

[17:22:53] <jiffe> this causes "errmsg" : "failing update: objects in a capped ns cannot grow", why would it grow?

[17:55:59] <Zelest> How can I use the $slice operator on a find using the ruby driver? :S

[18:01:24] <jiffe> db.oplog.rs.update({h: NumberLong("8979387659673550781")}, { $set: { o: { _id: ObjectId("54a622d255a8275d4b000b0f") } } })

[18:01:30] <jiffe> that is trying to set it back to its same value

[18:01:32] <jiffe> I think something is broke'

[18:01:42] <jiffe> "errmsg" : "failing update: objects in a capped ns cannot grow"

[19:06:25] <Zelest> does the field names take up diskspace? e.g, if I have a LOT of documents, will I save diskspace if I use shorter field names or does wiredtiger handle this automagically?

[19:06:41] <Zelest> (mongodb 3.2.1)

[19:09:42] <Zelest> Nevermind, some rtfm answered that. :)

[19:10:03] <Zelest> (if anyone is curious, the answer is yes, it takes up diskspace and wiredtiger doesn't solve that)

[19:19:44] <jiffe> so I got my nodes up, one is secondary, the other seems to be stuck on recovering

[19:19:59] <jiffe> "optime" : Timestamp(1455333087, 6) isn't changing

[19:28:05] <jiffe> actually the secondary isn't catching up either, its optime is 3 days old and not moving

[20:25:26] <jiffe> I was able to fix my replication issue with db.oplog.rs.update({h: NumberLong("8979387659673550781")}, { $set: { op: "n" } })

[21:23:47] <Zelest> Is it possible to sort an array inside a document.. E.g, get the array sorted by an element inside the array?

[21:25:25] <Zelest> Ah, aggregation framework

[23:22:07] <kryo_> hey guys i'm getting this error: "Btree::insert: key too large to index"

[23:24:43] <kryo_> the indexed field is an Object

[23:24:51] <kryo_> really i only need to index the "id" field

[23:25:04] <kryo_> but i'm not sure how to do this without writing a schema for the whole object

Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 13th of February, 2016