What is Out Of Core Processing

In my previous post on Swap and why to avoid it I mention out of core methods (OOC).  OOC used to be much more common than it is currently as memory sizes have increased and price per GB has fallen.  I still see traces of it in many engineering codes that have been with us a long time.  Many applications do OOC without realizing it.

OOC processing is the assumption that the working memory of the system, RAM, is less than that needed for the working set.  Thus the application is written such that data is read and written from disk to make up for the lacking space.  This is different from swap because the application does the read() and write() calls rather than the operating system doing this transparently for you.

While relying on the operating system is easy, the operating system is really making just educated guesses and tends to writes data in small blocks rather than large sequential reads and writes which are the best for performance.  I noted in my previous post that swapping hard drives maintain about 10-20MB/s, while they stream large reads and writes at 100MB+.   If you write your own OOC method to write all the data to disk,you will read later in a large block and then read the previously written large block in you will have much better performance than that provided by swapping.

A good example of this is iterating on a large array of values much larger than the RAM of the system.  The application would read a chuck of data into memory, do the calculation it could on that chuck, put it back down to disk and read the next chunk etc.  Obviously this is still much slower and much more complicated to code than fitting the entire application into RAM if available.

Thus OOC is not recommended if avoidable, but it is a better option if you know you will never have hardware with enough RAM to fit your data.  Lastly, the case of fitting all data into RAM is called in core.