XMM-Newton SAS Home Page
XMM-Newton Science Analysis System

dal (dal-1.198) [xmmsas_20211130_0941-20.0.0]

Memory Models

There are currently two memory models available; the High Memory and the High/Low Models (The Low Memory Mode, is only partially prototyped and is unlikely to be released in the foreseable future. ).

The memory model may be selected by setting the environment variable SAS_MEMORY_MODEL. The allowed settings are:

When a dataset is opened, with the High Memory Model mode (HMM), it is loaded into memory in its entirety. All subsequent operations are performed on the memory-loaded dataset. When the dataset is closed, the memory is flushed back to disk. The High Memory Mode gives high performance but is memory expensive. With the HMM mode enabled, it is more probable that the operating swapping mechanism will be invoked, giving rise to unacceptibly poor performance.

In the HighLow Memory Model (LMM) mode, when the dataset is opened, only the attributes are loaded into memory. Only when the data is accessed is it loaded into memory. When the data is no longer required it may be released, in which case it is flushed to disk before the memory is freed.

In principle, the LMM mode is capable of using almost no memory, or as much as the high memory mode. A task programmer should take the steps necessary to ensure that the memory consumption is kept to a minimum, and that the task is both high and low memory compatible. The guidelines for this are given later in this document.

The DAL supports the notion of subranges, which essentailly is a convenient mechanism of restricting access to all or only part of the data in a column or an array.

In the HMM mode, a subrange amounts to manipulating memory offsets to the data.

In the LMM mode, however, subranges become very significant because only the data specified in a subrange will be loaded into memory.

XMM-Newton SOC -- 2021-11-30