Blog post content
This article describes some details of memory allocation by MongoDB.
MongoDB uses memory mapped files for file management, so in this component, it relies a lot on the OS side. But let’s look at it based on Linux OS. MongoDB maps files into memory using standard mmap call from glibc.
How it works
On start, MongoDB maps all its data storage files into memory. RAM is not used at all, this process only reserves the address space. This is reflected as virtual memory usage by mongod process. They can be found either in the top (VIRT) or in MongoDB monitor (Memory Virt). Basically, they reflect the data + indexes size. Res/RSS memory usage doesn’t reflect anything relevant at all.
So in a sense, all data and indexes are in memory, split by small memory pages. MongoDB has pointers to each page. But they are not in RAM yet.
Now when an actual query is made, MongoDB finds the page that contains the required data and reads that data. It reads it from memory in general, and it doesn’t care if the data is in RAM or not. It can’t know that it just sends the request to the address in memory.
The rest is up to the OS. It initializes those pages in RAM on their first usage. Such initialized pages are known as cache pages, their total size can be seen with the command ‘free’ (column ‘cached’). The problem is that OS expires them after some time. This expiration is inevitable no matter how much RAM is available. It is an OS-wide process based on recency of usage. There is a kswapd process that runs periodically and checks whether the page has been accessed since the previous run. If it has, it does nothing. If it hasn’t, it divides the age by two. When the age reaches zero, this page becomes a candidate for eviction. (Well, it never becomes zero, probably just close to it enough, I don’t know exact math behind it).
This mechanism, file mmap’ing, is used for every file in the system. Log files are mmap’ed as well, so the more logs, the more RAM used by cache pages.
There is some dependency on RAM usage: the higher is the ‘active’ RAM usage (see vmstat -s), the more frequently kswapd runs and the faster is the aging process. MongoDB reports the actual size of data in RAM and the maximal age of that data. So we could see that in case of memory management mongodb are pretty much rely on the OS side and this could cause some downside effects like cache expiration and purging of data from RAM just because OS think that data should be expired already.
Cassandra uses the same mmap thing BTW.
You will achieve much better result if will concentrate more on your product and team, instead of solving technology and infrastructure problems. Centaurea’s team has an awesome experience with MongoDB and we are ready to offer you high professional consulting or development services. We are looking forward to start working with you!
P.S. We are glad to receive feedback from you. If you have noticed any mistakes in the article or have suggestions about it please let us know.
