43.1 Segment

A segment doesn't know about packets or log records. All it knows about is reading/writing a contiguous block of memory from/to disk.

A segment is able to keep track of the current write position, and allow for flushing of more and more data as it is added to the segment.

Forward Linked List for Recovery Scan

The segments on disk from the last valid check point onwards form a forward-linked list, allowing a forward scan during recovery when the store is opened. Other segments are not regarded as part of the forward-linked list, regardless of what their flush unit headers record. For example, they may be in the free segment stack.

Flush Units inside a Segment

On disk, a segment consists of a sequence of flush units. A segment has no header or footer. Each flush unit header records the SegId of the next segment in the log. This requires the next segment to be preallocated, so a flush unit can be written already knowing which segment will follow.

Alignment for unbuffered I/O

Windows unbuffered file I/O requires the buffer address, file offset and transfer size to have the required sector alignment. FileRAS opens the file with FILE_FLAG_NO_BUFFERING when file buffering is disabled. These alignment requirements apply to individual LFU writes as well as whole-segment I/O.

Segment buffers are allocated with VirtualAlloc(), which returns page-aligned memory. Segment file offsets and segment sizes are multiples of the reported disk sector size. Partial segment writes contain complete sectors: flush units begin and end on sector boundaries, so the buffer offset, file offset and transfer size remain sector-aligned.

The implementation currently reports a 512-byte disk sector size and applies these alignment rules using that value.

Linux

Linux O_DIRECT may require alignment of the user buffer address, file offset and transfer length. The requirements vary by filesystem and kernel. Since Linux 6.1 they can be queried with statx() using STATX_DIOALIGN:

• stx_dio_mem_align gives the required buffer alignment.
• stx_dio_offset_align gives the required file-offset and transfer-length alignment.

A Linux implementation using O_DIRECT must allocate segment buffers with the reported alignment, for example using posix_memalign(). LFU offsets and sizes must be multiples of stx_dio_offset_align. The current Linux RAS uses buffered I/O and does not use O_DIRECT.

SegmentBase

using SegId = int32;

class SegmentBase
{
public:
	SegmentBase(LSS& lss, SegId segid);
	~SegmentBase();

    void ClearBuffer();
    void ReadFromRAS();
    void WriteToRAS();
    void WriteSectionToRAS(int i1, int i2);
    SegId GetSegId() const { return segid_; }
    void SetSegId(SegId segid) { segid_ = segid; }
    int GetDiskSectorSize() const { return diskSectorSize_; }
    int GetSegmentSize() const { return segmentSize_; }
    octet_t* GetRawBuffer(int offset = 0) { return buffer_ + offset; }

protected:
    LSS& lss_;
    int diskSectorSize_;
    SegId segid_;
    octet_t* buffer_;
    const int segmentSize_;
};

Segment

class Segment : public SegmentBase
{
public:
	Segment(LSS& lss, SegId segid);
    ~Segment();
    void Release();
    void SetSize(int size) { size_ = size; }
    int GetSize() const { return size_; }

private:
    int size_ = 0;
    int accessCount_ = 1;
    Segment* prevInSEQ_ = nullptr;
    Segment* nextInSEQ_ = nullptr;
    ManualResetEvent loadedEvent_;
    std::atomic<bool> isLoading_ = false;
};