56 Asynchronous loading of serial elements

Status: partially superseded proposal. The worker-task design discussed below has since evolved, but asynchronous reading of contiguous serial elements is not part of the established LSS interface.

At the time of this proposal, each opened instance of the LSS created four worker threads for the following:

The proposal considered using a shared worker facility for these tasks and, in addition, supporting asynchronous loading of contiguous serial elements. This is needed to implement asynchronous loading of objects in a PSpace.

Perhaps better still, when the LSS is opened, it is provided with some kind of thread-pool it can use? That way, if an application needs to create many LSS instances, it could avoid creating so many threads. In fact a boost asio context pool might be the way to go, because it provides a boost::asio::deadline_timer , which is exactly what we need to run jobs periodically without having threads call a Sleep() function.

The LSS has a Lazy check pointer etc as follows: 


class LazyWriter
{
    LazyFlusher lazyFlusher_;
};

class SegmentWriter
{
    LazyWriter lazyWriter_;
};

class LSS
{
    SegmentWriter segmentWriter_;
    LazyCleaner lazyCleaner_;
    LazyCheckPointer lazyCheckPointer_;
};

Interface to synchronously read contiguous serial elements

The following structs represent contiguous serial elements: 


struct ReadOnlyBuffer
{
    const octet_t* buffer;
    ssize_t size;
};

// Provides access to a serial element recorded as a contiguous buffer in memory.
struct IContiguousSerialElement
{
    // A contiguous serial element must be explicitly closed, even if exceptions have been
    // thrown by the LSS.
    // It is an error to close the LSS before the closing all the contiguous serial elements
    // that have been opened for reading.
    virtual void Close() = 0;

    virtual ReadOnlyBuffer GetBuffer() const = 0;
};

Synchronous loading of contiguous serial elements from the LSS involves calling the following method of ILogStructuredStore: 


/*
Provides an alternative to ReadSerialElement() for reading a serial element as a contiguous
block of memory.  Obviously this function shouldn't be called for very large serial elements
that don't fit in physical memory and therefore would result in page faulting.

The given Seid must not be null

The returned IContiguousSerialElement must be closed after it is used (including when
exceptions are thrown by the LSS).

Returns nullptr if no serial element exists with the given Seid

It is an error to call this function on a serial element that is currently opened for
writing (within a transaction), or being deleted using a call to DeleteSerialElement().

Shared reading of serial elements is supported. I.e. any number of threads can independently
(and concurrently) read the same serial element
*/
virtual IContiguousSerialElement* ReadContiguousSerialElement(Seid seid) const = 0;

Asynchronous API

It is proposed that the following method be added to interface ILogStructuredStore: 


virtual void AsyncReadContiguousSerialElement(
    Seid seid,
    std::function<std::error_code ec, const octet_t* buffer, ssze_t size> handler) const = 0;

'handler' is a callback function which is called when the serial element has been loaded, or else an error occurred. If an error occurred then the handler is called with buffer==nullptr and size==0.

The std::function is copied by value into what could be regarded as a job queue in the LSS.

Note that in C++11 passing by value is recommended if a copy is needed - it often produces more efficient code (see Want Speed? Pass by Value by Dave Abrahams). Therefore the std::function is passed by value since it will need to be copied to put it in a job queue.

A std::function value may contain a std::shared_ptr, allowing it to hold objects alive while the async I/O is pending. For example the caller may use this technique: 


class MyClass : std::enable_shared_from_this<MyClass>
{
    void do_read(Seid seid)
    {
        auto self(shared_from_this());
        lss_.AsyncReadContiguousSerialElement(seid,
            [this, self](std::error_code ec, const octet_t* buffer, ssze_t size)
            {
                if (!ec)
                {
                    // handle async load of the object
                }
            });
    }
    ILogStructuredStore& lss_;
};

If the LSS is closed then all the pending handlers are called with an error code that indicates the operation was aborted.

Review of existing support for asynchronous loads from the LSS

See AsyncObjectLoader. Also AsyncJobQueueOnThread.

Implementation

It might be appropriate to tailor the API to support the requirements of PSpaces:

  • It might make sense for each PSpace to use a different io_context for loading serial elements, to provide better concurrency. Note that the LSS supports concurrent loading of serial elements from its segment cache.
  • When a PSpace is closed there is a need to ensure all the async loading for that PSpace is stopped synchronously.
  • The PSpace ROT only needs a single call-back function to handle all async load serial element requests, we don't need a separate std::function instance for each OID.
  • All AsyncBinds in a PSpace are protected by the CSpace mutex. Therefore the "producer" of the sequence of OIDs to be loaded is effectively single threaded. For performance it would be good if most AsyncBinds simply push an OID onto a std::deque which is protected by the CSpace mutex.
  • All the loading of serial elements in a PSpace uses a single io_context which is run by a single thread. Therefore we have a single threaded "consumer".
  • For fairness to other clients that happen to use the same io_context, it shouldn't process too many OIDs in one batch.

Let a PSpace record a std::deque<OID> for the OIDs that need to be loaded asynchronously. This std::deque<OID> is protected by the CSpace mutex. PSpace::AsyncBind(OID oid) may cause an oid to be pushed onto this deque. If the deque transitions from empty to non-empty then a task to process the deque is posted.

For a given PSpace there is a single AsyncObjectLoader which uses a single io_context to load serial elements from the LSS. AsyncObjectLoader has a std::deque<OID> member which is only accessed by the single threaded io_context. When the AsyncObjectLoader deque is empty the io_context thread can lock the PSpace and swap the two deques.