09 Sep 2014

45250

0

Memory Manager Development

The legacy memory manager in ReactOS has many idiosyncrasies and architectural quirks, issues that have dogged the project for many years now. The ARM team began a rewrite, colorfully named Another Rewrite of the Memory Management Module (ARM3), that partially supplanted the old memory manager but a big chunk of responsibility still lie with it, namely in the form of MEMORY_AREA. Simply put, MEMORY_AREA was the means by which memory address ranges were tracked, tracking which is necessary for things like handling paging. Presently there are still chunks of memory whose address range is stored in MEMORY_AREA and not in ARM3's data structures. This is because at present ARM3 is not able to bring in data from physical storage, so anything that requires physical backing such as reading from the pagefile or memory mapping files would either not work or eventually cause a failure. Adding support for interacting with physical storage to ARM3 has been preoccupying Jérôme Gardou for some time now and once achieved will be a major milestone in modernizing ReactOS' core components. Note that the following assumes you have a basic understanding of how virtual memory works. There are enough references for those that I am not going to bother writing one of my own. If you need a refresher, wikipedia has a decent enough writeup.
As part of his effort Jérôme has written a very limited balancing system to determine when to page out memory to the pagefile stored on the harddrive. At present his system only works with the paged pool, a memory resource that is already managed by ARM3 and ARM3 alone instead of the somewhat Frankenstein half and half a lot of other chunks of memory go through. Note that despite being called the paged pool, because ARM3 lacks actual support for paging out to physical storage the paged pool effectively acts like a non-paged pool. The obvious point one should note is that if a situation arises where the pool runs out of available pages to service requests, things start failing, hard. The balancer relies on two counters, one that counts up based on how long ago the address ranged backed up by a page was last accessed and another that counts down based on how many existing references there are to the page, decreasing every time the age counter hits a certain threshold. Once the reference counter hits zero the page in question is paged out and the memory location is marked as available for recycling. If something tries to access memory range backed by the page again before it is recycled a soft fault is generated and the page is simply marked valid again. If something tries to access the memory range after the page has been recycled, a new page will need to be allocated and the data read back in from the pagefile. This is the point where Jérôme is at now, but the biggest challenge comes next.
There are different type of memory resources available that the memory manager must support. One of those is something called sections, which describe chunks of memory that can be shared across processes. ARM3's support for sections is very limited, meaning they are for the most part the purview of the legacy memory manager. The really tricky and painful part about sections is that they can be used to memory map files that exist on disk. Such file backed sections are also what the common cache deals with and those that have followed the project over a sufficiently lengthy amount of time will recall that the common cache is one of the big missing pieces in ReactOS and is a major blocker for not only a proper memory manager but also for supporting more modern filesystems. So in order to properly implement sections support, Jérôme also needs to get some kind of common cache implementation going. Yeah, he has his work cut out for him.

Discussion: https://www.reactos.org/forum/viewtopic.php?f=2&t=13619

This blog post represents the personal opinion of the author and is not representative of the position of the ReactOS Project.