37 Storage medium

The LSS accesses its storage through the IRAS random-access-store interface. An IRAS is a linear byte-addressed store with 64-bit addresses. It supports aligned memory allocation, reading, writing, querying the current size, and truncating the store.

The caller may invoke RAS operations from multiple threads without using an external mutex. The IRAS implementation is responsible for making those calls thread-safe and for serialising them, so two operations on the same RAS do not execute concurrently. As with other explicitly closed interfaces, the caller must ensure that no operation is in progress, and that no new operation can begin, when it calls Close() or destroys the RAS.


using RASAddress = int64;
using RASSize = RASAddress;

struct IRAS
{
    virtual ~IRAS() {}

    virtual int GetDiskSectorSize() const = 0;

    virtual void* AllocateMemoryBlock(int size) = 0;
    virtual void FreeMemoryBlock(void* p) = 0;

    virtual void Close() = 0;

    virtual void Read(void* buffer, RASAddress offset, int numBytes) = 0;
    virtual void Write(const void* buffer, RASAddress offset, int numBytes) = 0;

    virtual RASSize GetSize() const = 0;
    virtual void TruncateSize(RASSize newSize) = 0;

    virtual void StopWriting() = 0;
};

AllocateMemoryBlock() returns a buffer suitable for the implementation's Read() and Write() operations. A write can extend the store. The supplied implementations return zeroes for the portion of a read beyond the current size. StopWriting() prevents subsequent writes.

Source: Ceda/cxLss/IRAS.h

The supplied implementations store the LSS in a file on Windows or Linux, or in a PagedBuffer using MemRAS. The abstraction could represent another linear medium, but raw-partition support has not been implemented. It is recorded as a proposal in Other.

Segments and addressing

The storage medium contains fixed-size segments. The segment size is chosen when an LSS is created and cannot subsequently be changed for that store. The default is 512 kB. A segment is identified by a positive 32-bit SegId; zero is reserved to represent a null segment reference.

For a file-backed LSS, the 64 kB root block occupies the start of the file and the segments follow it. Segment identifiers begin at one, so the byte position of a segment is calculated as:


ROOT_BLOCK_SIZE + (segid - 1) * segmentSize

Writes automatically grow the underlying RAS as necessary. At a checkpoint, the LSS finds the highest utilised segment and truncates the RAS immediately after it. The lazy cleaner does not give special preference to segments at the end of the file: it selects sufficiently underutilised segments by utilisation. The file therefore becomes shorter only when all segments beyond the highest utilised segment have become free.

File fragmentation

Growing a file in many small steps can contribute to filesystem fragmentation. Each extension may be allocated from the free space available at that moment, and allocations for other files can occupy the adjacent space between extensions. The file can consequently be represented by many separate extents rather than a small number of contiguous extents.

The file RAS implementations try to reduce this effect by extending the file in 512 kB increments rather than extending it by only the size of each write. Larger extensions can reduce the number of filesystem allocation and metadata operations and can improve the opportunity for contiguous allocation, but they do not guarantee it.

There is also a distinction between changing a file's logical length and reserving physical storage. In particular, ftruncate() on Linux can extend the logical file without immediately allocating all of its data blocks. Those blocks can still be allocated separately as they are written. A filesystem preallocation operation such as fallocate() expresses the intention to reserve an extent more directly, but LinuxFileRAS does not currently use it.

Fragmentation is most directly harmful on a rotating disk, where accessing separated extents can require additional head movement. It is generally less costly on an SSD because there is no mechanical seek, although a large number of extents can still increase filesystem metadata and I/O overhead. The benefit of the 512 kB extension policy therefore depends on the filesystem, storage device, available free space, and other allocation activity.

Windows file: FileRAS

FileRAS stores the LSS in a single Windows file and uses Win32 operations with 64-bit file positions. File buffering is disabled by default using FILE_FLAG_NO_BUFFERING, because the LSS maintains its own segment cache. Buffers are allocated with VirtualAlloc() so they satisfy the alignment expected by the current implementation. File buffering can instead be enabled through LssSettings.

FILE_FLAG_WRITE_THROUGH can be requested separately. Reads and writes are serialised by a mutex. When a write extends the file, FileRAS rounds the new file size up to the next 512 kB boundary to reduce filesystem fragmentation. This extension granularity is independent of the configurable LSS segment size. A checkpoint can subsequently truncate the file to the exact end of the highest utilised segment. The implementation currently reports a hard-coded 512-byte sector size rather than querying the alignment requirements of the file or volume.

Sources: Ceda/cxLss/src/FileRAS.h and Ceda/cxLss/src/FileRAS.cpp

Linux file: LinuxFileRAS

LinuxFileRAS stores the LSS in a single file using open(), read(), write(), lseek(), and ftruncate(). It uses Linux open-file-description locks to prevent conflicting access by another process. Shared-read mode takes a read lock; other open modes take a write lock. Reads and writes within the process are serialised by a mutex.

As on Windows, a write that extends the file rounds its new size up to the next 512 kB boundary, while a checkpoint can truncate it to the exact end of the highest utilised segment. Although LinuxFileRAS::Open() accepts the file-buffering and write-through settings, it currently does not use them: the file is opened without O_DIRECT or O_SYNC. The implementation therefore uses buffered file I/O while still reporting a hard-coded 512-byte sector size and enforcing 512-byte offsets and transfer lengths.

Sources: Ceda/cxLss/src/LinuxFileRAS.h and Ceda/cxLss/src/LinuxFileRAS.cpp

Memory: MemRAS

MemRAS stores the LSS in a caller-owned PagedBuffer. It is useful for tests and transient stores small enough to keep in memory. Close() has no work to do, and truncation changes the size of the paged buffer. Like the file implementations, access is serialised and reads beyond the current size are completed with zeroes.

For consistency with the file implementations, MemRAS reports a 512-byte sector size and requires buffers, offsets, and transfer sizes to be aligned to 512 bytes. On Windows it allocates buffers with VirtualAlloc(); on other platforms it uses malloc().

Sources: Ceda/cxLss/src/MemRAS.h and Ceda/cxLss/src/MemRAS.cpp

See RootBlock for the implications of the historical alignment assumptions shared by these implementations.

Previous36 Implementation overview Next38 RootBlock