Log file Viewer
#mongodb logs for Saturday the 14th of May, 2016
[00:00:18] <Boomtime> probably less - mmap is stored as uncompressed bson, that's what a mongodump is
[00:00:58] <Boomtime> i.e their format is very nearly identical - the dump doesn't have indexes or any fragmentation though, so it will always be smaller, just a question of how much
[00:03:20] <Boomtime> you could try removing the orphans first, but i suspect you're in a very tenuous place
[00:04:14] <Boomtime> you have an unquantified amount of corruption in all your members and the corruption is not the same - because disks just randomly layout to all intent...
[00:04:36] <Boomtime> this means that if you perform a write on the primary that succeeds, but hits a corrupt bit on a secondary = you lose that secondary
[00:05:05] <Boomtime> meanwhile, the other corruption that exists elsewhere on the primary will prevent a new sync
[00:05:46] <Boomtime> this isn't mongodb you're fighting - you are fighting the fact your hard disks have shattered the data store
[00:07:35] <jiffe> and maybe thats another thing to take into account, maybe mongodb isn't the best storage backend for this application
[00:07:45] <Boomtime> what do you expect it to do? if the data files cannot be read, what can it possibly do?
[00:08:43] <Boomtime> you expect that corrupting the filesystem should be recoverable and just kind of plow on in some magical way restoring the data that the hard-disk, the actual storage medium, lost?
[00:08:50] <jiffe> yeah, this is my first experience with data of this size, I've worked with dbs that work around corruption, but there was also corruption it couldn't work around requiring a rebuild which is what we're dealing with here
[00:09:22] <Boomtime> it *is* working around the corruption - and i've given you lots of ways to do it, you're just not willing
[00:09:33] <jiffe> I guess the real problem stems from the fact that we had to start using rsyncs rather than mongodb syncs, that would have probably alleviated a lot of this
[00:10:01] <Boomtime> your real issue is that you have corrupted hard disks in all your replica members
[00:11:26] <jiffe> yeah, which is why I said the mongodb resync would have alleviated a lot of this rather than the rsyncs we had to do
[00:12:55] <Boomtime> again, now you're expecting a feature that is expressly designed to prevent divergence to allow you to diverge - your situation is not the fault of the feature - you should concentrate on what you CAN do
[00:13:49] <jiffe> no, what I was saying was if the mongodb resync was working before we did any rsyncs then the corruption would have been stopped by that resync rather than spread by the rsync
[00:14:45] <jiffe> which is still of concern because of I rebuild this I still don't know if the mongodb resync will work even if it was working with non-corrupt data
[00:32:01] <ganeshaditya1> Hi, I have a bunch of documents in my mongodb collection. They have various attributes like votes, time posted etc., I am thinking of making a ranking scheme to uses these attributes to assign a rank to each of my documents. I wanna be able to retrieve my documents in a sorted order of this rank which I need to calculate spotaneously when I run my query. How do I go about this?
[00:44:52] <Boomtime> ganeshaditya1: read up on this -> https://docs.mongodb.com/manual/aggregation/#aggregation-pipeline
[00:46:58] <Boomtime> with that said, you'd likely be better off calculating the ranks periodically, rather than on-demand, since almost all systems are read heavy - so you want to reduce work in the read cycle
[00:47:31] <zsoc> Heyyy so I was doing some stuff with a mongodb in a vagrant the other day and now it /appears/ to be sort of.. gone. I spun it back up and tried to check it out with mongo shell and it tells me my user doesn't exist. But it also says connecting to "test".. Is there a possibility I had another db .. or something somewhere? I set everything up rather in an automated fashion.
[00:48:33] <Boomtime> mongo shell connects to the 'test' database by default - you can select databases in the shell with "use", like "use foo" to select the foo db
[00:49:32] <zsoc> Ok.. I mean I see the db name there. It says (empty)... so the data is just wapoof? Or is it because I'm not logged in as a user? Or does just 'mongo' log in as admin user?
[00:50:50] <Boomtime> show dbs works? if show dbs works and you didn't log in then auth is not enabled - this isn't the db you expect it is, or something else has changed
[00:52:11] <zsoc> The reason I think it's a different db is when I go to login as the user i had set up (admin user I think?) it tells me the user doesn't exist. But obviously everything wasn't completely reset because it shows the empty db name which i created. Very confusing. Maybe there's a different /data/ folder somewhere...
[00:52:12] <Boomtime> somewhat crazily no, auth is not enabled by default, though there is a startup warning in that case - and it should show up in the shell when you connect too
[00:52:51] <Boomtime> right, even with auth disabled, if you attempt an auth it will actually verify that what you tried "would succeed" if auth was enabled
[00:53:47] <Boomtime> however, auth'ing occurs against the database you have selected - so you'll need to check that too - but it just sounds to me like you're looking at the wrong instance entirely
[00:54:17] <zsoc> So then.. maybe i started mongod in a different way. I don't know if it sudo service mongod start'd or just sudo mongod'd or if it would even make a difference
[00:59:39] <Boomtime> well that depends on your app - how does ranking work? is it just based on some sort of score? do you keep a tally of something that rolls forward?
[01:00:16] <Boomtime> if it's vageuly complex, maybe stick a sample document in a pastebin/gist and a comment showing the calc you'd do to determine rank
[01:01:18] <zsoc> oooooooook wait i think it's working... i manually started mongod using --config /etc/mongod.conf
[01:05:42] <zsoc> It might be. It was like /lib something.... not /data/db.. but also i think it's my exploding harddrive that's half dead might be part of the issue. Have disk read problems where this vagrant storage is
[01:10:03] <zsoc> Ok, well now it's working. Maybe not for very long lol. Got any good data recovery/drive swap resources? :p
[01:47:00] <AxD79> Hi all.. I was wondering if anyone in here has experience with node.js or goLang? If not, is there channel you can point me to?
[01:49:12] <Boomtime> many people frequent here using Node - goLang is common but i think much less so
[01:51:29] <AxD79> My question is not related to Mongodb that is why I'm looking for other channels. I'm trying to determine which server side scripting language to go for on my next project.
[01:53:42] <Boomtime> ah, well that's a little trickier, and is basically more personal pref than anything - can you do what you need in both languages? be rigorous, find out what you'd need to do to be successful in each language
[01:54:31] <Boomtime> if either one lets you down on some point, consider if that point is essential - if you're missing some critical piece of infrastructure, a library or interface implemtnation then that language is out
[02:58:35] <Boomtime> that link seems to suggest that a comparison would find such a document, since a missing field is considered null by default - i don't see any caveat that would prevent it so not sure why it doesn't show up
[02:59:05] <zsoc> Not the first odd thing I've ran into in mongo... won't be the last. It's a bit cowboy out here.
[03:00:47] <zsoc> Huh, so uh... if I try to add some random field during document creation that wasn't modeled... I mean doesn't it sort of just add anyway? Or at least throw an error? It seems like it just rather ignored that field nicely.
[03:00:56] <Boomtime> $lt is certainly intended to compare real values, rather than defaults, but it does seem odd
[03:01:18] <zsoc> Yeah I mean I understand the logic, but in theory it would work that way naturally given how that doc is worded.
[03:01:46] <Boomtime> interestingly, $max works the way you want; https://docs.mongodb.com/manual/reference/operator/meta/max/
[03:02:09] <Boomtime> but that is a sort-of internal operator that directly manipulates against an index, so i wouldn't advise using it
[03:04:41] <Boomtime> yeah, just because the database is schemaless doesn't mean your app gets to be - it's your app that needs to make sense of things
[03:05:10] <Boomtime> but you'll never need to worry about expansion, or even changing the meaning or type of a field - whatever your app can handle
[11:10:19] <Secret> So they say don't normalize mongodb database if you don't need in other parts of your app, but then again I can't query deeply nested arrays...
[11:11:36] <Secret> So they say don't normalize mongodb database if you don't need in other parts of your app, but then again I can't query deeply nested arrays...
[14:10:07] <wspider> mongdb default ID might be too big for sending through ajax, what do you guys think :P?
[15:42:20] <sexytime> i am trying to check the licensing. but i have a question about it. can the community edition be used commercially?
[17:07:01] <ptx0> I'm trying to drop a collection named simply 154, I can drop a collection that starts with a letter but not this one
[17:11:45] <ptx0> now I am even more confused because I've dropped all the collections and my application still sees data ;)
[17:14:13] <kurushiyama> ptx0: PLEASE, for you own sake: Learn the basics _before_ you work on a production system.
[18:03:59] <ptx0> Uncaught exception 'MongoResultException' with message 'localhost:27017: Invalid modifier specified: $pop'
[18:32:54] <konst> What is the best way to create a single reusable instance of the MongoClient in Java?
[23:38:35] <sneumann_> Hi, I am running a mongodb in a docker container. I added some collections, and can see them if I docker exec into the conatainer
[23:58:04] <sneumann_> and I use the identical mongo 172.17.0.5/predret to connect from inside and outside of the container.