Log file Viewer

[01:36:56] <hdm> jhaddad: totally without more than a N*1 dataset, its a complete mess

[01:37:12] <hdm> keep your indexes in memory or pay the performance gods

[01:37:34] <hdm> but as previously stated "memory" isnt just ps aux

[01:37:46] <hdm> it means taking the index size, not the resident process size

[01:37:47] <max1> hi is anyone here familiar with mongoengine?

[03:34:47] <ra21vi> can someone make me understand what does "Memory Mapped" files mean exactly in MongoDB

[03:35:48] <ra21vi> if I have 2 GB RAM, and total mongodb database size 10GB with index size of 1 GB, will this make mongo slow as data size is more than memory avaiable oon system

[03:40:00] <ra21vi> is this room has any active people? I can see everyone is asking question and then leaves :)

[05:18:55] <DigitalKiwi> ra21vi: do you have 64 bit os?

[05:19:33] <DigitalKiwi> this explains memory mapped files better than I could http://en.wikipedia.org/wiki/Memory-mapped_file

[05:20:09] <DigitalKiwi> but if you have 2 gig of ram you probably have a 32 bit os and you can't have more than like 2.4 GB database or slt

[05:21:20] <DigitalKiwi> * NOTE: when using MongoDB 32 bit, you are limited to about 2 gigabytes of data

[05:21:21] <DigitalKiwi> ** see http://blog.mongodb.org/post/137788967/32-bit-limitations

[05:21:21] <DigitalKiwi> ** with --dur, the limit is lower

[06:02:49] <ra21vi> DigitalKiwi: I understand 32-bit cannt address beyond 3 GB (4gb with PAE). And each application is restricted to 2 GB. So this means, on 64 bit, the db can be more than 2GB.

[06:03:50] <ra21vi> But I want to know, if I have 2 GB RAM on 64 BIT (I am concerned about hosting on EC2 with 1.7 GB RAM). In this case, if I have index 1 GB and data 10 GB, will it effect performance?

[06:04:37] <ra21vi> DigitalKiwi: and what if I have index size more than RAM (like 4 GB index and 20 GB Data) on 2 GB RAM / 64 Bit.

[06:06:24] <DigitalKiwi> I do not think you will have an index of 1GB for only 10GB of data

[06:06:37] <DigitalKiwi> (could be wrong though)

[06:07:08] <DigitalKiwi> and if you don't have enough ram for an index it will just not live all in ram and will be slower than if it was as it'll be on disk

[06:10:27] <ra21vi> DigitalKiwi: thanks, got it :)

[06:14:47] <deoxxa> ra21vi: mongodb uses mmap

[06:14:58] <deoxxa> ra21vi: so the operating system will decide what needs to be in/out of memory

[06:15:18] <deoxxa> mongodb just says "hey operating system, you jerk, give me 50 billion gigabytes of memory"

[06:15:28] <deoxxa> and the operating system says "lol ok here you go"

[06:15:51] <deoxxa> then when mongodb wants to access a part of the mapped memory region that isn't in memory, the operating system will load it in

[06:16:15] <deoxxa> that's the short version anyway

[07:10:29] <ra21vi> deoxxa: got it. So if the index if bigger than RAM or available RAM, it goes into SWAP and thus HIT Ratio drops.

[07:17:45] <deoxxa> bingo

[10:19:24] <amitprakash> Hi.. is it possible for me to store records to a temporary collection before they are updated to the permanent one ? [ i.e. I want to create transient records R, which can be bulk inserted into collection C, or scrapped ]

[10:19:31] <amitprakash> What would be the best way to do this?

[10:56:12] <NodeX> jjust create a temp collection lol

[10:56:27] <NodeX> use it, abuse it, then trash it - just like a woman haha

[11:07:40] <Ditegen> hello

[11:07:45] <Ditegen> is there a way to get BSON as a query result?

[11:09:03] <wereHamster> yes!

[11:09:32] <wereHamster> the protocol uses BSON, so just talk to your database with netcat

[11:12:28] <Ditegen> so you mean to connect directly, bypassing driver layer

[11:13:20] <wereHamster> yes!

[11:13:40] <Ditegen> going to try, thank you!

[11:14:45] <NodeX> haha

[11:20:32] <Ditegen> of course i'm going to have fun

[12:07:45] <wereHamster> undoubtedly..

[16:12:05] <Zelest> If I don't have enough RAM to keep my entire dataset in memory.. it will try to keep the indexes there at least.. If I don't have enough for the indexes either, will it try to keep the most frequently used indexes there?

[16:23:18] <skot> yes

[16:23:59] <skot> It is up to the operating system to manage what is in memory and it will use a most recently used scheme to keep stuff in memory

[16:43:01] <Zelest> skot, Ah, cheers.

[17:32:13] <hvzh> hallo. is there way to set default now() for the date field?

[18:58:02] <wubino> how good are the geo abilities of Mongo?

[21:27:58] <xing> any ETA for perl driver supporting ensure_index({...}, {expireAfterSeconds=>XXX})? trying to test mongo 2.2 TTL

[21:29:00] <xing> this gives the currently-supprted ensure_indexes: http://api.mongodb.org/perl/current/MongoDB/Indexing.html#Options

[21:36:00] <mids> try using the alternative of db.system.indexes.insert

[22:31:13] <xing> mids - thanks.. will play with that approach