41.3 LazyWriterQueue

We simplify the discussion initially by only considering segments that are full.

As segments are filled with data in memory they are passed onto the LazyWriterQueue (LWQ). More specifically, when a segment becomes full the SegmentWriter calls

LazyWriterQueue::OnFilledSegment(Segment* s)

This pushes the given segment onto the back of the queue. The lazy writer pops (full) segments from the front of the queue by calling

Segment* LazyWriterQueue::GetNextFullSegment()

and writes them to disk.

The LWQ employs a private mutex in order to be threadsafe - i.e. to allow one thread (associated with the SegmentWriter) to push segments while another thread (the lazy writer) pops segments.

The LWQ can't be involved in a dead-lock scenario because all methods return without needing to block on external resources or events.

Segments in the LWQ are assumed to have a positive access count to protect them from eviction by the SEQ. The lazy writer will only release a segment after it is finished with it.

Segments that are full should be written to disk as fast as the lazy writer can go - i.e. there is no concept of timeout before directing the lazy writer to write segments that are full.

Allowing for partially full segments and flushing

The last segment in the LWQ may be partially full. In that case the lazy writer will only write it to disk when the log is flushed.

After a Log Record Set (LRS) is written by the SegmentWriter, it calls

void LazyWriterQueue::SetFlushPosition(Segment* s, int size)

This allows the LWQ to remember the partially full segment at the end of the log, and its "size". The size relates to the amount of data in the segment that should be flushed to disk. This may differ from the current size according to the SegmentWriter, because that is updated continually as new log records are written to the end of the log. There is no point flushing data between snapshot records, so the flush position only needs to advance each time a snap shot record is written to the log.

For a partially full segment the lazy writer will only access the data up to the last flush position set by the SegmentWriter. This range of bytes in the segment is immutable - because new log records written by the SegmentWriter never overwrite existing log records in a segment. Therefore a flush of the log can proceed in parallel with the SegmentWriter.

A single Log Record Set can involve many segments being written to disk. In that case there will be repeated calls to OnFilledSegment(), for each segment that is full, followed by a call to SetFlushPosition() for the partially full segment at the end of the log. This segment is not added to the queue (yet). However the partially full segment is available in the LWQ for the purpose of flushing the log.

Code

class LazyWriterQueue
{
public:
    bool IsEmpty() const;
    void ReleaseSegmentsAndClearLWQ();
    void SetFlushPosition(/*refs*/ Segment* s, int size);
    void OnFilledSegment(/*takes*/ Segment* s);
    Segment* GetNextFullSegment();
    void GetSegmentsToFlush(SegmentQueue& sq);

private:
    mutable std::mutex mutex_;
    SegmentQueue segmentQueue_;
};