ILogStructuredStore

Any of the methods in the interface can throw either of the following types of exception 


struct ILogStructuredStore
{
    virtual void Close() = 0;
    virtual void SetMaxNumSegmentsInCache(int n) = 0;
    virtual MSSN GetMissingShutdownSeqNum() const = 0;

    // Serial Element Ids (Seids)
    virtual SeidHigh CreateSeidSpace() = 0;
    virtual Seid AllocateSeid(SeidHigh seidHigh) = 0;
    virtual bool ReserveSeid(Seid seid) = 0;
    virtual SeidLow PeekNextSeidLow(SeidHigh seidHigh) = 0;
    virtual bool AllocateAffiliateSeid(Seid& seid) = 0;
    virtual void GetSeidsInSeidSpace(xvector<SeidLow>& seidLows, SeidHigh seidHigh) const = 0;

    // Read serial elements
    virtual bool SerialElementExists(Seid seid) const = 0;
    virtual ICloseableInputStream* ReadSerialElement(Seid seid) const = 0;
    virtual IContiguousSerialElement* ReadContiguousSerialElement(Seid seid) const = 0;

    // Transactions
    virtual ILssTransaction* OpenTransaction() = 0;

    // Throttle control on a "producer"
    virtual void BlockUntilLowWaterMark() const = 0;
    virtual bool ReachedHighWaterMark() const = 0;

    // Diagnostics support
    virtual void GetStats(LssStats& stats) const = 0;
    virtual void DumpLSS(LssStats& stats, xostream& os, const LssDumpSettings& ds) const = 0;
    virtual bool RecurseSeidMap(
        xvector<Seid>& children,
        Seid seid = BEGIN_RECURSE_SEID_MAP,
        bool enableOverflowPackets = false) const = 0;
    virtual bool WriteInfoOnNodeForGivenSeid(Seid seid, xostream& os) const = 0;
};

void Close()

The LSS must be explicitly closed, even if exceptions have been thrown by the LSS It is an error to close the LSS while reading a serial element, or a transaction is open.

void SetMaxNumSegmentsInCache(int n)

Set the maximum number of segments in the segment cache. Note that the one segment cache serves as both a read and write cache. n must be at least 4.

MSSN GetMissingShutdownSeqNum() const

Returns the number of times the store has not been gracefully shut down over its entire life


Serial Element Ids (Seids)

Serial elements in the store are uniquely identified by a Serial Element Identifier (Seid).

SeidHigh CreateSeidSpace()

Create a new Seid space. This is the upper 32 bits of the Seids shared by a group of related serial elements that should be clustered together on disk. This function is threadsafe - i.e. multiple threads can safely call this function Never returns 0.

Seid AllocateSeid(SeidHigh seidHigh)

Allocate a new, unused Seid within the Seid space associated with the given SeidHigh. This function is threadsafe - i.e. multiple threads can safely call this function Never returns a null Seid.

bool ReserveSeid(Seid seid)

Ensures the seids in { Seid(low,high) | high == seid.high_ && low <= seid.low_ } are reserved. Returns false if that range of seids was already reserved.

SeidLow PeekNextSeidLow(SeidHigh seidHigh)

Returns the next available SeidLow for the given SeidHigh, without actually performing an allocation. Not compatible with using AllocateAffiliateSeid().

bool AllocateAffiliateSeid(Seid& seid)

The Seid passed by reference serves as both an in and out parameter to the function. This function allocates a fresh Seid (the out-parameter) that is "affiliated" with an existing Seid (passed as the in-parameter).

Consider that a new Seid needs to be allocated, and the new serial element should be clustered with some other existing serial element, called the "affiliate". Eg the affiliate may be a parent node in a tree of nodes. AllocateAffiliateSeid() does a good job of allocating Seids no matter the order in which nodes are added to the tree.

This function is threadsafe - i.e. multiple threads can safely call this function

void GetSeidsInSeidSpace(xvector<SeidLow>& seidLows, SeidHigh seidHigh) const

Retrieve all the Seids (actually only the low 32 bit part of each Seid) in the Seid space associated with the given SeidHigh. seidHigh must not be zero.


Read serial elements

bool SerialElementExists(Seid seid) const

Does a serial element with the given Seid exist? The given Seid must not be null This function is threadsafe - i.e. multiple threads can safely call this function

ICloseableInputStream* ReadSerialElement(Seid seid) const

Provides a stream for reading the serial element with the given Seid. The given Seid must not be null

The returned stream must be closed after it is used (including when exceptions are thrown by the LSS).

Returns nullptr if no serial element exists with the given Seid

It is an error to call this function on a serial element that is currently opened for writing (within a transaction), or being deleted using a call to DeleteSerialElement().

Shared reading of serial elements is supported. I.e. any number of threads can independently (and concurrently) read the same serial element, assuming each such thread has made an independent call to ReadSerialElement() - i.e. they don't try to share a returned ICloseableInputStream.

IContiguousSerialElement* ReadContiguousSerialElement(Seid seid) const

Provides an alternative to ReadSerialElement() for reading a serial element as a contiguous block of memory. Obviously this function shouldn't be called for very large serial elements that don't fit in physical memory and therefore would result in page faulting.

The given Seid must not be null

The returned IContiguousSerialElement must be closed after it is used (including when exceptions are thrown by the LSS).

Returns nullptr if no serial element exists with the given Seid

It is an error to call this function on a serial element that is currently opened for writing (within a transaction), or being deleted using a call to DeleteSerialElement().

Shared reading of serial elements is supported. I.e. any number of threads can independently (and concurrently) read the same serial element


Transactions

ILssTransaction* OpenTransaction()

All changes (i.e. mutative work) done on an LSS (apart from Seid allocations) must be done by a thread that has opened a transaction.

A transaction is intended for a single thread. Only the thread that called OpenTransaction() on the LSS is permitted to call the methods on the returned ILssTransaction

It is an error to close the LSS while there is an open transaction. The LSS internally uses a mutex to ensure that only one thread opens a transaction at a time.

There is no concept of aborting or rolling back a transaction.


Throttle control on a "producer"

void BlockUntilLowWaterMark() const

Blocks until it is appropriate for the producer to begin writing changes to the LSS again (because the LSS has written enough segments out to disk).

bool ReachedHighWaterMark() const

Returns immediately, and indicates whether the producer has written enough changes to the LSS such that it should call BlockUntilLowWaterMark() in order to wait for the LSS lazy writer to "catch up"


Diagnostics support

void GetStats(LssStats& stats) const

Retrieve stats about the LSS in an LssStats variable.

void DumpLSS(LssStats& stats, xostream& os, const LssDumpSettings& ds) const

Write information about all the segments, flush units and packets in the entire store to the given output stream according to the flags in the given LssDumpSettings. This function should only be called on relatively small stores because it will write many mega bytes of text for a large store.

bool RecurseSeidMap( xvector<Seid>& children, Seid seid = BEGIN_RECURSE_SEID_MAP, bool enableOverflowPackets = false) const

Can be used to recurse through all the Seids in the store. This is based on the fact that Seids form an 8 level hierarchical map.

RecurseSeidMap() should first be called with a seid equal to BEGIN_RECURSE_SEID_MAP. This will return the children at the top level. Up to 256 children may be returned. A child seid may then be passed in again to a call to RecurseSeidMap(), to recurse down through the Seid map. After doing this 4 times, a SeidHigh will be obtained. After doing this an extra 4 times the Seids of serial elements will be obtained. ReadSerialElement() may then call called to access their serialised state.

enableOverflowPackets determines whether seids of overflow packets will be returned in the iteration of the seids. It is usually appropriate to call with enableOverflowPackets = false

Returns false if the given Seid is invalid

bool WriteInfoOnNodeForGivenSeid(Seid seid, xostream& os) const

For the given Seid, writes diagnostic information to the given stream. This includes the total size of the packet in bytes.