Log file Viewer
#mongodb logs for Monday the 3rd of August, 2015
[09:22:42] <fontanon> Hi everybody, I got something wrong with the balancer ... perhaps somebody could help .... Failed with error 'could not acquire collection lock for podsystem.pwcdr to migrate chunk [{ : MinKey },{ : MaxKey }) :: caused by :: Lock for migrating chunk [{ : MinKey }, { : MaxKey }) in podsystem.pwcdr is taken.', from shard0001 to shard0000
[10:19:30] <fontanon> Hi everybody, I got something wrong with the balancer ... perhaps somebody could help .... Failed with error 'could not acquire collection lock for podsystem.pwcdr to migrate chunk [{ : MinKey },{ : MaxKey }) :: caused by :: Lock for migrating chunk [{ : MinKey }, { : MaxKey }) in podsystem.pwcdr is taken.', from shard0001 to shard0000
[11:07:01] <fontanon> 1 : Failed with error 'could not acquire collection lock for podsystem.pwcdr to migrate chunk [{ : MinKey },{ : MaxKey }) :: caused by :: Lock for migrating chunk [{ : MinKey }, { : MaxKey }) in podsystem.pwcdr is taken.', from shard0001 to shard0000
[11:07:01] <fontanon> 1082 : Failed with error 'moveChunk failed to engage TO-shard in the data transfer: can't accept new chunks because there are still 1 deletes from previous migration', from shard0001 to shard0000
[12:05:28] <fontanon> sorry joannac I dropped my irc connection. what do you mean with a healthy sharding ?
[12:11:37] <fontanon> joannac, i've put a sh.status() on the following pastebin http://pastebin.com/rdB4SssW
[12:42:45] <StephenLynx> when I need to text complext stuff I write stand-alone io.js scripts and run them with io.
[12:48:38] <jayjo> is there a better way to stream server-event http data straight to my mongo instance?
[13:55:26] <vagelis> Hello, i use python and pymongo. Is it possible to store data or the mongo database in memory(or something like that) so unit tests can run faster? Im not familiar with this i learn it 5' ago cause my tests take too long and i asked if there was a way to make them run faster.
[13:57:24] <StephenLynx> you can always store data in RAM, but then I don't think your tests will represent the actual scenario they should be testing in the first place.
[14:01:15] <vagelis> The other guy told me that he uses postgres and in every test he resets the database so he doesnt create it every time. Does mongo/pymongo have something like that? :S
[14:10:06] <ekristen> I had a collection that had grown over the months to be millions of entries, since then we’ve pruned it back down to around 10k entries, but in that time the database went from 50gb to 860gb, can I run compact to gain some of that space back?
[14:19:30] <fontanon> joannac, sorry for bothering ... did you have some time to have a look at my sharding status here? http://pastebin.com/rdB4SssW
[14:24:18] <bigblind> Can someone tell me what the documentation here means by sub-collection? http://api.mongodb.org/python/current/api/pymongo/collection.html#pymongo.collection.Collection
[14:25:22] <bigblind> is that the collection of al the embedded documents for the given key in each document in the collection?
[14:25:45] <cheeser> the wording is weird but i think it's saying just the name of the collection rather than its fully qualified name of <database>.<collection>
[14:29:31] <bigblind> cheeser: Sorry, I'm not talking about the constructor, but the c[name] (or c.name) definition below that.
[14:30:22] <StephenLynx> collections and databases are created automatically once you write anything to them.
[14:30:55] <vagelis> StephenLynx yea i completely forgot about removing everything from colections after every test(teardown) but now im searching how to create the database only once :|
[14:32:19] <vagelis> how could i create the database only once? I mean i know the setup,teardown methods but then what? :S
[14:32:40] <ekristen> question, if I have an index that has had millions of entries and on disk has grown to 860gb, but it is currentl only 100mb, will compact give that back to me, or would it just be easier to backup the documents that are currently in it, delete the collection, and re-add the collection and restore the documents? it only has a few thousand right now in it ...
[14:42:17] <GothAlice> ekristen: Compact won't free allocated disk space, it'll just rearrange the data within that disk space to have fewer "holes" (depending on how often you update and grow record sizes, delete records, etc.)
[14:42:37] <GothAlice> ekristen: A full repair, which writes out new data files, then moved them in place over the originals, _will_ free on-disk stripes.
[14:44:03] <ekristen> GothAlice: in a replicaset, is the best way to do a full repair by doing it on a secondaries first?
[14:44:21] <StephenLynx> it constantly has to add new hard drives to the system to accommodate static files.
[14:44:26] <GothAlice> Depending on your tasks, it's generally a good idea to keep file stripes for the same file together, but that'll depend on usage. If random seeking in very large files is desireable (i.e. to support multiple streaming users) then spreading the chunks around may become a better idea.
[14:45:12] <lqez> ekristen: http://docs.mongodb.org/manual/reference/command/compact/#disk-space says as like as GothAlice
[14:45:20] <lqez> compact requires up to 2 gigabytes of additional disk space while running. Unlike repairDatabase, compact does not free space on the file system.
[14:45:28] <GothAlice> ekristen: Doing it on a secondary first is generally a good idea both as a "test run" and because the secondary will become primary when you bring the primary down to --repair it, so it'll save some time going back and forth if you do the secondaries first.
[15:24:56] <csd_> What's the easiest way to update one field for every document within a collection?
[17:34:53] <LauraG> Hello, I've a question, let's say I have semi-huge collection with an indexed INT field, if I want to start using "strings" instead, do I have to reindex my collection? And do I have to make separate queries to fetch the documents with the "int" field and the ones with the "string" field?
[17:44:00] <LauraG> Thanks cheeser!, so technically I could use something like this for my query: {"$or": [{"field": 1}, {"field": "1"}]}, and still get an indexed query
[17:45:04] <crazydip> from what I've read, WiredTiger in MongoDB 3.0.5 is not supported in PPC64, but what about PPC64LE?
[17:49:20] <LauraG> cheeser: is there an "update" operator to transform my data (ints to string) or do I have to do this manually, as in write a forEach function instead that updates the type.
[17:51:01] <cheeser> search using $type on that field then in forEach() $set them to the new converted type.
[18:06:21] <RWOverdijk> Something I've never quite understood about mongo is how to manage associated data (joins). I know mongo doesn't have joins, that's the confusing part. Say I have addresses and activities. I want to be able to query both. I also want to be able to fetch activities, and also get their associated addresses. Would I have to query address for every activity?
[18:10:00] <RWOverdijk> Or could I C- Use mongo to retrieve IDs and then query my relational database for the actual data?
[18:10:07] <StephenLynx> usually if you can perform just a single additional query for each collection involved, it isn't thaaaaaat bad.
[18:10:38] <StephenLynx> if your critical data is not on the relational database, mongo can still be relevant on that setup.
[18:11:11] <RWOverdijk> I'd use elasticsearch in stead of mongo, but that requires me to actually sync data.
[18:17:38] <StephenLynx> why don't you just create the index on the database itself in the first place?
[18:20:06] <RWOverdijk> It sounds like a horrible argument, but it has proven to be very useful to me
[18:25:56] <RWOverdijk> So I'm assuming you're trying to create a user with role userAdminAnyDatabase
[18:26:35] <ksmtk> db.createUser({ user: "myadmin", pwd: "mypass", roles: [{role: "userAdminAnyDatabase", db: "mydb"}] })
[18:30:08] <RWOverdijk> StephenLynx, One small extra question... Do you think that having 30 extra finds (on ID) with mongo would be a shame?
[18:32:14] <RWOverdijk> Simple: Activity has an address property, which is the ID of the document from Address
[19:11:13] <shortdudey123> i am using the mongodb 3.0.4 rpm install from the mongo repo on CentOS 7.1, when i start the mongo service, it complains about soft process limits being 4096. The default init file that the package drops has
[19:17:00] <RWOverdijk> StephenLynx, Another idea! Fetch all activities, then fetch all addresses where id in []. Then loop the activities and replace the ids with the address objects
[19:18:02] <RWOverdijk> btw if you want me to stop asking you directly, let me know. I can imagine it's getting a bit annoying.
[20:38:32] <mrmccrac> trying to setup RBAC within a sharded environment, turned on authorization on my config servers and my shard, when i restart everything mongos has the following error:
[20:38:43] <mrmccrac> 2015-08-03T20:33:35.141+0000 E SHARDING [mongosMain] could not verify that config servers are in sync :: caused by :: no config servers successfully contacted :: caused by :: not authorized on config to execute command { dbhash: 1, collections: [ "chunks", "databases", "collections", "shards", "version" ] }
[20:39:30] <mrmccrac> i think i need to tell what credentials mongos needs to use when talking to the config servers..?
[21:21:52] <shortdudey123> i am using the mongodb 3.0.4 rpm install from the mongo repo on CentOS 7.1, when i start the mongo service, it complains about soft process limits being 4096. The default init file that the package drops has ulimit -u 64000. Anyone seen this happen before?