13 CSpace async tasks for DGS nodes

The DGS allows for defining async dependents, causing async tasks to be automatically posted to the CSpace - both for CPU and I/O.

By the way, the distinction between CPU and I/O jobs is a bit hard to justify. A std::thread is supposed to only be used to perform CPU work, not be stuck waiting on I/O. Ideally all I/O would be asynchronous back to the OS, and threads never need to block on I/O.


$adt+ implement CSpace
{
    void PostTask(ThreadPoolTask task) const;
    {
        if (enablePostTasks_)
        {
            if (!taskExecuter_)
            {
                taskExecuter_ = GetTheThreadPool().CreateTaskExecuter();
            }
            taskExecuter_->PostTask(task);
        }
    }

    void PostTaskForIO(ThreadPoolTask task) const;
    {
        if (enablePostTasks_)
        {
            if (!taskExecuterForIO_)
            {
                taskExecuterForIO_ = GetTheThreadPoolForIO().CreateTaskExecuter();
            }
            taskExecuterForIO_->PostTask(task);
        }
    }

    void StopGcAndEvictAllDgsNodes()
    {
        // Don't allow any more tasks to be posted
        // This is an issue for async DGS nodes, which tend to be invalidated as we force eviction to
        // occur, and invalidation callbacks may try to post tasks
        // It might be safer to ensure OnInvalidate() is never called on DGS nodes when the CSpace is
        // being destroyed?
        enablePostTasks_ = false;

        // Closing the task executer without a CSpace lock is important to avoid dead-lock scenarios
        // (given that it is very common for tasks to try to get a CSpace lock).
        // Closing the task executer can mean DGS nodes which were non-evictable become evictable
        // so this is done before calling EvictAllDgsNodes().
        if (taskExecuterForIO_)
        {
            taskExecuterForIO_->Close();
            taskExecuterForIO_ = nullptr;
        }
        if (taskExecuter_)
        {
            taskExecuter_->Close();
            taskExecuter_ = nullptr;
        }

        ...
    }

    // Normally true, but set to false when the CSpace is being destroyed to prevent new tasks from
    // being posted to taskExecuter_ and taskExecuterForIO_.
    bool enablePostTasks_ = true;

    mutable ITaskExecuter* taskExecuter_ = nullptr;
    mutable ITaskExecuter* taskExecuterForIO_ = nullptr;
};

todo: If we use an IoContextPool instead of an ITaskExecuter, then we need some way of ensuring the CSpace doesn't destroy IObjects on which async tasks are running.

It is assumed that IObject's are allowed to be deleted as late as we like, it is purely for memory reclamation purposes. They are not supposed to do things like close files, close sockets, or stop threads.

Therefore it is reasonable to reference count a CSpace using shared_ptrs (see Use of shared_ptr), and it only deletes the objects in the GC extent when the CSpace destructs. This might only occur after async tasks have completed.

We currently don't have a concept of aborting the async tasks. The syntax for defining async $dep and $cache functions doesn't support aborting a calculation. Let's keep it simple for now and continue with that assumption.

See DGKeyedAsyncNodeOnInput. Async nodes which cache an input can push a task which gets a CSpace lock and calculates the input. That means read barriers can be called from async tasks. That means new DGS edges can be formed, and DGS nodes can be added to the eviction queue. This makes CSpace shutdown a bit tricky because it may try to evict all nodes and then afterwards find there are unevicted nodes still remaining in the eviction queue.

StopGcAndEvictAllDgsNodes() is part of a mess for how PSpaces and CSpaces are currently destroyed:


$adt+ CSpace
{
    // Destroy the CSpace.  The CSpace must not be locked (except perhaps internally by the GC
    // thread)
    void Destroy();

    // Must be called without a CSpace lock. Intended to be called before Destroy() to manually
    // close the garbage collector and evict all the DGS nodes.
    // This calls ForceEvict() on every node in the eviction queue.  It ignores whether the node
    // is evictable according to its implementation of IsEvictable()
    void StopGcAndEvictAllDgsNodes();

    ...
};

void PSpace::Close()
{
    {
        DeclareThreadPSpace dtp(this);

        // Block until all the async loads have completed.  This must be done before the call
        // to StopGcAndEvictAllDgsNodes() to evict all the DGS nodes
        rot_.Close();

        // Stop the GC thread and close the CSpace task executers then evict all the DGS nodes.
        // Must be called without a CSpace lock.
        StopGcAndEvictAllDgsNodes(m_cspace);

        // If this PSpace is dirty then make sure it is written to the LSS.
        m_ps.m_dosWriter.OnClosePSpace(this);

        /*
        Destroy(CSpace*) calls Destroy() (but not OnGarbageCollect()) on all the remaining objects in the
        CSpace.
        So Destroy(CSpace*) causes any remaining objects in the CSpace to be deleted from memory.
        This is done without first calling OnGarbageCollect() on each object.  Therefore they aren't
        removed from the ROT, as done during the normal operation of the store.
        */
        ceda::Destroy(m_cspace);
        m_cspace = nullptr;

        m_ps.m_spaceMap.RemovePSpace(this);
    }

    delete this;
}

Our task is to instead base the design on something which is so simple it is obviously correct.

Making sure we don't delete IObjects that are running async tasks

Typically a heap allocated IObject that is deleted by the CSpace GC is used when it has async DGS nodes, for example:


$struct X isa ceda::IObject
{
    $cache <<async>> int y(int a, int b) const
    {
        return a+b;
    }
};

See DGKeyedAsyncIndepNode, where it is proven that the IObject and the relevant cache map elements cannot be deleted while the async tasks are running.

Proposal

We will adhere to the following principles where possible:

DGS nodes that post async tasks are contained in IObjects in the GC extent
DGS nodes that have a running async task are in the eviction queue, cannot be evicted, and visit the containing IObject
IObjects in the GC extent don't do things like close sockets, close files or stop threads when they destruct
OnGarbageCollect() is not called on the remaining IObjects in the GC extent when the CSpace destructs
The remaining IObjects in the GC extent are deleted when the CSpace destructs
DGS async tasks cannot be aborted
The CSpace is reference counted using shared_ptrs
DGS async tasks hold a shared_ptr to the CSpace

Consider that we close a PersistStore. This must synchronously close the file, so there's no question that we have to ensure there are no async tasks still running which might be trying to load objects from the LSS.

Previous12 Xc++ code generation examples NextPart III: Cache-function nodes