72 Segment Cache

Purpose

The Version 2 SegmentCache will be similar to the current segment cache in its management of resident segment buffers, access counts and eviction. Its boundary will be narrower: it manages segments in memory, but it does not perform RAS I/O. Loading and writing are asynchronous workflows coordinated above the cache.

This separation is important in the event-driven LSS. A cache lookup is a synchronous memory operation. A RAS read is an asynchronous operation which may complete later and generate an event. Combining them in one call would conceal a state transition and give a function which sometimes returns immediately and sometimes starts an unrelated operation.

No upward reference to LSS

Neither SegmentCache nor Segment retains an LSS&. The LSS is the composition root and coordinator; its lower-level cache and segment objects must not use it as a service locator to reach the RAS, settings, error state or other LSS modules.

The cache can instead be constructed from the values it actually needs, such as the segment size and maximum number of resident segments. A segment needs its SegId, its buffer and the small amount of state required for cache residency. This makes both classes independently understandable and testable.

class SegmentCache
{
public:
    SegmentCache(std::size_t segmentSize, std::size_t maximumResidentSegments);
    // ...
};

class Segment
{
public:
    SegId GetSegId() const;
    std::span<octet_t> GetBuffer();
    std::span<const octet_t> GetBuffer() const;
    // ...
};

The signatures are illustrative. In particular, the buffer may use an internal view type rather than std::span. The architectural requirement is that the segment owns or refers to memory; it does not know how that memory is transferred to or from persistent storage.

Segment does not perform I/O

The following methods on the current Segment will not exist:

void ReadFromRAS();
void WriteToRAS();
void WriteSectionToRAS(int i1, int i2);

A segment is an in-memory object, not an active storage object. Removing these methods also removes its need to find the RAS through LSS&. Code coordinating a write obtains a view of the relevant bytes from the segment and submits that view to the RAS. Code coordinating a read arranges for the RAS to fill the final cache buffer directly.

Cache-only lookup

SegmentCache::GetSegment(SegId segid) only looks for the segment in the cache. It never reads from the RAS, starts a RAS request, waits for I/O or arranges for another module to load the segment. If the requested segment is not resident and available, the call reports a cache miss by returning an empty SegmentAccessor.

SegmentAccessor SegmentCache::GetSegment(SegId segid);

It may be preferable to name this operation FindSegment() or TryGetResidentSegment(), because those names make the possibility of a cache miss explicit. Regardless of its final name, lookup has cache-only semantics.

A successful lookup returns an accessor which pins the segment so it cannot be evicted while the caller uses it.

SegmentAccessor

SegmentAccessor represents a pin giving access to a resident segment. It is a move-only RAII type: constructing it acquires a pin, destroying it releases that pin, and moving it transfers responsibility for the pin to the destination. Copy construction and copy assignment are disabled so an accidental copy cannot silently create or duplicate a pin.

class SegmentAccessor
{
public:
    SegmentAccessor() = default;
    ~SegmentAccessor() { Reset(); }

    SegmentAccessor(const SegmentAccessor&) = delete;
    SegmentAccessor& operator=(const SegmentAccessor&) = delete;

    SegmentAccessor(SegmentAccessor&& other) noexcept;
    SegmentAccessor& operator=(SegmentAccessor&& other) noexcept;

    explicit operator bool() const { return segment_ != nullptr; }
    Segment& operator*() const;
    Segment* operator->() const;
    void Reset();

private:
    friend class SegmentCache;
    SegmentAccessor(SegmentCache& cache, Segment& segment);

    SegmentCache* cache_ = nullptr;
    Segment* segment_ = nullptr;
};

An empty accessor represents a cache miss. A non-empty accessor can be passed between functions by move without changing the underlying segment address or releasing its pin:

auto segment = cache.GetSegment(segid);
if (!segment)
    StartSegmentLoad(segid);
else
    ContinueRead(std::move(segment));

The moved-from accessor becomes empty. Exactly one accessor remains responsible for releasing that particular pin. If a second independent pin is needed, it must be acquired explicitly from the cache or through an explicitly named operation such as Clone(); ordinary copy syntax must not conceal the additional cache access count.

Use by asynchronous operations

An operation may take ownership of a SegmentAccessor and retain it while native I/O refers to the segment buffer. For example, a segment-write operation keeps the accessor until the RAS reports completion:

struct SegmentWriteOperation
{
    SegmentAccessor segment;
    RASWriteRequest request;
};

This makes the buffer-lifetime rule explicit and prevents eviction or recycling while the operation is outstanding. The accessor controls only the lifetime of resident memory; it does not initiate I/O and does not provide a route back to the complete LSS.

Lifetime and access rules

The SegmentCache must outlive every accessor which refers to it. If that ordering cannot be guaranteed structurally, accessors and the cache must share a separate control block whose lifetime extends until the final accessor is destroyed. Move operations should be noexcept so accessors can be stored and transferred reliably.

If segments can have concurrent readers but mutation requires exclusive access, the implementation should distinguish read-only and mutable access rather than allowing every accessor to return an unrestricted mutable reference. This can use separate accessor types or explicit operations on the cache. Pinning prevents eviction; it does not by itself provide synchronisation for concurrent access to the bytes.

Loading a segment

A cache miss is handled by an LSS-level read state machine or a dedicated segment-loading operation, not by GetSegment(). The normal sequence is:

1. Look for an available segment in the cache.
2. On a miss, reserve a cache entry and its final segment buffer.
3. Submit a RAS read which targets that buffer directly.
4. Return control while the RAS read is outstanding.
5. On completion, validate the segment and publish it as available in the cache.
6. Continue the logical operations waiting for that segment.

Reading directly into the reserved cache buffer avoids a staging copy. The segment must not be returned by ordinary cache lookup while its buffer is only partially filled or has not yet been validated.

Concurrent requests for the same segment

The implementation must prevent several cache misses for the same SegId from causing duplicate RAS reads. Reserving an entry for a load therefore records that the segment is being loaded. A later request can distinguish an available segment, an existing outstanding load and the need to start a new load.

enum class SegmentLookupResult
{
    Available,
    Loading,
    NeedToLoad,
    CacheFull
};

This state need not be exposed through the simple cache-only GetSegment() operation. It can be part of a separate atomic reservation API used by the LSS read coordinator. Logical reads which encounter Loading attach themselves to the existing load operation and are continued when its completion event is processed.

Writing segments

Writing is also coordinated outside Segment and SegmentCache. The writer determines the RAS offset and byte range, obtains a view of the segment buffer and submits the asynchronous write:

auto bytes = segment.GetBuffer();
ras.Write(offset, bytes, completion);

A section write uses a subview of the buffer rather than a method which lets the segment call the RAS. The segment buffer must remain alive and unchanged for as long as the native operation may reference it. The write operation therefore owns a cache pin or another lifetime handle until its completion is reported. Eviction cannot recycle that segment buffer while the write is outstanding.

I/O and integrity errors

There is no recovery from a segment read failure within the cache or loading state machine. Any RAS read or write error puts the open LSS into its zombie state, as does an integrity failure discovered while validating a segment. The LSS stops logical processing and does not retry the read, substitute another segment or resume waiting operations.

Native operations which were already submitted must still retain their operation records and buffers until the platform reports completion or cancellation. This lifetime requirement does not imply logical recovery: completions are drained only to finish the safe shutdown of outstanding native work after the LSS has become a zombie.

Responsibilities

The resulting division of responsibilities is:

• Segment represents a segment identifier, its memory and the small amount of state needed for cache residency.
• SegmentCache manages lookup, allocation, pinning, access counts, capacity and eviction of segment buffers.
• The LSS read and write state machines decide when I/O is required, coalesce operations and react to RAS completion events.
• The RAS transfers bytes asynchronously between persistent storage and buffers supplied by the LSS.

This boundary keeps synchronous memory management separate from asynchronous storage operations and removes hidden coupling from the lowest-level segment classes back to the complete LSS.