Log file Viewer

[09:54:22] <Lokie> hello, have a question about mongodump / mongorestore. I dumped a 2.4GB db as shown by show databases; and the dump is ~ 6GB. Restoring that it still keeps the 6GB size. the --gzip option obviously affects only the dump size as storage

[09:55:43] <Lokie> the mongodb is standalone and pretty old namely 3.2.11

[09:59:01] <Lokie> is that expected behavior and is there a way to avoid it or make the restored DB go back at it's original size?

[09:59:28] <Lokie> I get the same for a 4 an 8GB db, the dumps are ~ 3x larger

[10:47:03] <synthmeat> Lokie: what are your index sizes? i.e. what do you exactly mean by "a 2.4GB db"?

[10:48:00] <synthmeat> for me, pure dumps take ~100GB, and storage size reported on db is ~90GB

[10:49:05] <guardian> hello, is there a way to achieve the following: $currentDate gets applied only when inserting and not updating

[10:51:48] <synthmeat> guardian: i wasn't aware $currentDate works for insertion, unless you consider upserting an insertion

[10:53:17] <Lokie> I am pretty new on mongodb so synthmeat give me a sec and I 'll get that info

[10:53:51] <Lokie> so far I used the 'show databases'; from which I get the size

[10:55:14] <synthmeat> yeah, that doesn't include indexes. db.stats() (per each db) has more data.

[10:55:53] <synthmeat> (pro tip - mongodb compass is a bit sluggish, but ok interface for starting out with mongodb. shows a lot of relevant data)

[10:58:07] <Lokie> hmm

[10:58:44] <Lokie> https://pastebin.com/xe4nF9DP

[10:59:10] <Lokie> that looks very 6Gbish :p

[11:00:53] <synthmeat> yeah

[11:01:03] <Lokie> show databases; shows 2.455GB

[11:01:14] <Lokie> none of those numbers seem to match that

[11:01:24] <Lokie> any idea which value show databases used?

[11:02:02] <synthmeat> show databases shows storageSize + indexSize, apparently

[11:02:14] <synthmeat> https://stackoverflow.com/questions/34054780/how-can-mongodb-datasize-be-larger-than-storagesize

[11:03:46] <synthmeat> so, wiredtiger is compressing your data to look like it's not 6 gigs

[11:04:52] <synthmeat> and mongo doesn't dump it like that, so it can be restored on mongo possibly not using wiredtiger. i assume, anyways.

[11:05:15] <Lokie> yea was reading that, thanks

[11:06:22] <Lokie> bear with me a sec

[11:06:28] <synthmeat> lol, sure

[11:06:44] <Lokie> we also have mongobd cluster and that shows the data as 6GB when I restore that DB there

[11:07:11] <synthmeat> is that cluster using wiredtiger with snappy compression?

[11:07:12] <Lokie> it's sharded for some reason so I assume I can't just log to a node and run the same commands

[11:07:36] <synthmeat> 6gb, and it's sharded? :) big plans, anyone?

[11:07:47] <Lokie> will have to check, exactly what I am trying to see is whether it leaves them as 6GB or compresses them since we are charged per GB of storage I presume

[11:07:59] <Lokie> yea whoever build it thought it needed sharded

[11:08:03] <synthmeat> yeah, i don't have experience with shards, only replicas

[11:08:07] <Lokie> I know sharded is for 1TB +

[11:08:13] <Lokie> but :p

[11:08:51] <Lokie> so have to find what engine is used and whether the restored data will get compressed on disk

[11:08:58] <Lokie> as that translates to $$

[11:09:21] <synthmeat> yeah, it's all in the config, probably. or command line arguments for systemd unit, rc script, whatever

[11:09:48] <Lokie> hopefully dunno how mongo cluster works especially with shards

[11:10:16] <Lokie> thanks for the help synthmeat you helped me learn some basic stuff and steer me towards the right direction :)

[11:10:32] <synthmeat> sure. there's better people than me in this chan though

[11:10:57] <Lokie> I 'll idle in here if anyone wants to add something

[13:46:20] <Lokie> forgot to mention synthmeat, I ssh-ed on one of the shard's nodes and the db.stats are similar with the standalone so there is compression in place

[13:46:52] <Lokie> the cluster UI shows the datasize as size so will be interesting to see if they charge per GiB of the datasize or storagesize

[13:50:37] <synthmeat> Lokie: what's that, atlas? i only self-host.

[13:50:47] <synthmeat> "only"

[13:56:35] <Lokie> mongodbs cloud manager

[13:56:41] <Lokie> pre atlas :p

[13:59:52] <synthmeat> i just don't wanna pay for cross-datacenter juice

[19:29:03] <acecile2> hi, is this a suitable channel for asking help building a mongo query ?