41 SegmentWriter

Each log flush unit (LFU) stores a check point id and a 32 bit Flush Sequence Number (FSN). The check point is a guid that is generated each time a check point is performed. Consider the following depiction of the log broken up into LFUs.

Note the following

• An LFU stores the identity of the last completed check point. The LFUs written while check point cpid2 is being made therefore still contain cpid1.
• After the last check-point LFU is closed, the LSS records the boundary shown by the dotted line as positionOfLastCheckPoint_. This LogRecordPosition is written into the new root-block division.
• The open, empty LFU immediately after that position is then changed to cpid2, and its FSN is reset to 1. Later LFUs increment the FSN.
• If the preceding LFU exactly fills a segment, the stored position is the end of that segment; logically it is still the dotted boundary before the first cpid2 LFU.

When to start a new segment

The Segment Writer is responsible for breaking up serial elements into packets.

Serial elements that overflow without good reason will reduce performance. A small serial element that spans two segments reduces localisation which degrades read performance.

Criteria...

Let nFree be the number of bytes remaining in the segment.

Let nSize be the remaining size of the serial element. 

If nSize ≤ nFree then we should write the remainder of the serial element as a single packet. If nSize > nFree then the object will overflow. We choose to write part of the serial element to the current segment if nFree > T1 and nSize > T2 where T1 and T2 are suitable fixed thresholds

SegmentWriter

Takes on two responsibilities

• manages the writing of Log Record Sets to segments being prepared in memory
• manages a lazy writer that uses posted tasks to write segments to disk.

Writing a Log Record Set

Only one thread can write a Log Record Set (LRS) at a time. This is enforced by using a mutex, and providing the following two methods

    void BeginLRS(bool doingCheckPoint);
    void EndLRS();

It is crucial that any thread that calls BeginLRS() will (soon) call EndLRS(), otherwise other threads will be prevented from writing an LRS.

After calling BeginLRS(), a thread will typically call GetSegmentBeingWrittenInMemory() to gain access to the current segment being prepared in memory. It can be assumed that a flush unit has already been opened, and the client simply writes log records at the current write position.

A client will need to check whether the next log record will fit by calling SegmentBeingWrittenInMemory::GetNumFreeBytes(). If there is insufficient space in the current segment, then the client should call

    void StartNewSegment()

in order to start a fresh segment in memory. Again it can be assumed that a flush unit has already been opened, and the client simply writes log records at the current write position. The previous segment will be pushed onto the back of the LazyWriterQueue (LWQ) allowing it to be written to disk by the LazyWriter.

There is no advantage in flushing only part of an LRS. Therefore a client writing an LRS will not be interested in breaking up a segment into smaller flush units. Therefore the client should not call the following methods on SegmentBeingWrittenInMemory

    void StartFlushUnitOnNewSegment(Segment* s, SegId nextSegid);
    bool StartAnotherFlushUnit(LogRecordPosition& flushPosition, bool& needToStartANewSegment);

Managing the lazy writer

All that is required is to start and stop it. Also, a method is provided to allow the log to be flushed.

Support for check pointing

The thread performing a check point must open an LRS in order to write the dirty RPM nodes to the log. BeginLRS() takes a boolean flag to indicate whether the thread is performing a check point.

After writing all the dirty RPM nodes during a check point, the following function is called.

    LogRecordPosition MakeLRSFlushableForCheckPoint(const Guid& checkPointId);

This makes the LRS flushable by closing the current flush unit and opening a new fresh one. The given check point id uniquely identifies the check point. This is written to the freshly opened flush unit and all the subsequent flush unit headers to validate flush units read by a recovery scan. Also the freshly opened flush unit has its FSN reset back to 1.

The returned LogRecordPosition points at the end of the log (for the check point). The position will be aligned on a disk sector boundary and may equal the LSS segment size. It is the appropriate start position for a recovery scan, and therefore will be stored in the root block as part of the check point information.

After the LRS has been closed by calling EndLRS(), the log must be flushed. This is guaranteed to flush the last LRS because the call to MakeLRSFlushableForCheckPoint() made it flushable.

Determining when to perform a check point

The SegmentWriter is responsible for determining when check points should be performed. It simply counts the number of segments that have been prepared in memory since the last check point. When this exceeds a threshold (currently 128, corresponding to 64MB for 512k segments), the following method is called on the LSS.

    LSS::SignalNeedToCheckPoint()

This tells the LSS that another check point needs to be performed.

Code

class SegmentWriter
{
public:
	SegmentWriter(LSS& lss);
	~SegmentWriter();
    void Start(RecoveryScanInfo* rsi);
    void Stop();
    void Flush(bool flushForCheckPoint);
    void FlushLastFlushableLRSForCheckPoint();
    void BeginLRS(bool doingCheckPoint);
    void EndLRS();
    LogRecordPosition UpdateFlushPos();
    SegmentBeingWrittenInMemory& GetSegmentBeingWrittenInMemory() { return segmentBeingWrittenInMemory_; }
    void StartNewSegment();

private:
    LSS& lss_;
    bool isStarted_;
    LazyWriter lazyWriter_;
    TxnMutex txnMutex_;
    bool doingCheckPoint_;
    SegmentBeingWrittenInMemory segmentBeingWrittenInMemory_;
    int numSegmentsSinceLastCheckPoint_;
};