37 DirtyPSpaceList

The dirty PSpaces form a double linked list, made threadsafe with a std::mutex. Each PSpace directly stores pointers to the previous and next PSpaces in the list. This allows a PSpace to very quickly remove itself from the double linked list.

When a PSpace first transitions to dirty it is added to the list with a call to PushBack(). When a PSpace is cleaned by writing the changes to the LSS it is removed from the list with a call to Remove().

DirtyPSpaceList is a member of the DosWriter. ProcessAllGroups() is used to process all the PSpace groups. This is done carefully to avoid getting stuck processing the same PSpaces over and over again. Note that a naive approach which pops dirty PSpaces until the queue is empty could take a long time if PSpaces are being pushed back onto the queue as fast as they are being removed. This implementation retrieves a snapshot of the list of dirty PSpaces then processes all PSpaces in the list. Note that this implementation locks the mutex for a long time, during which no PSpaces can be closed [a thread trying to close a PSpace will block in the call to DosWriter::OnClosePSpace()].

class DosWriter
{
    void ProcessAllGroups(bool flushToDisk)
    {
        std::lock_guard<std::mutex> lock(mutex_);

        xdeque<PSpace*> L;
        m_dirtyPSpaces.CopyIntoDeque(L);

        // Note that a PSpace cannot be closed while iterating through this deque.
        for (auto i : L)
        {
            PrivateProcessPSpaceTxnGroup(*i,flushToDisk);
        }
    }

    // Double linked list of PSpaces that have dirty objects
    DirtyPSpaceList m_dirtyPSpaces;
};

class PSpaceDos
{
    void MarkAsDirty() const
    {
        if (!m_dirty)
        {
            m_dirty = true;
            m_pspace.GetPersistStore()->m_dosWriter.m_dirtyPSpaces.PushBack(&m_pspace);
        }
    }

    void PSpaceDos::WriteChangesToLSS(ILssTransaction& txn)
    {
        for (auto& i : m_dirtyObjectSet)
        {
            ptr<const IPersistable> po = i;
            OID oid = GetOid(po);
            if (GetIObjectFlag(po,DBP_PO_SYNC_DELETED))
            {
                txn.DeleteSerialElement(oid);
            }
            else
            {
                AutoCloser<ICloseableOutputStream> stream(txn.WriteSerialElement(oid));
                {
                    Archive ar(stream.Get());
                    m_pspace.GetPersistStore()->DynCreateSerialise(ar,po);
                }
            }
            ClearPersistableDirtyFlag(po);
        }
        m_dirtyObjectSet.clear();
        m_dirty = false;
        m_pspace.GetPersistStore()->m_dosWriter.m_dirtyPSpaces.Remove(&m_pspace);
    }

    mutable std::atomic<bool> m_dirty;
};

Implementation

The implementation uses DoubleLinkedList. Note that mDefineDoubleLinkedList has a flag for whether the double linked list is protected by a std::mutex.

mDefineDoubleLinkedList(Name,Element,traceFlag,bool threadSafe,bool copyDeque) 

This requires that PSpace use mDoubleLinkedListElementPtrs() to declare the prev/next pointers.

$adt+ implement PSpace
{
    // The dirty PSpaces form a double linked list.
    mDoubleLinkedListElementPtrs(DirtyPSpaceList,PSpace)
};

DirtyPSpaceList.h

Source: Ceda/cxPersistStore/src/DirtyPSpaceList.h

DirtyPSpaceList.cpp

Source: Ceda/cxPersistStore/src/DirtyPSpaceList.cpp

Proposal 1 to simplify

Rather than have a double linked list of dirty PSpaces, consider instead that we simply iterate through all the open PSpaces using the PSpaceMap and check whether they are dirty using the PSpaceDos dirty flag. We don't expect to have large numbers of PSpaces in a given PersistStore, so in practise this solution is efficient.

Proposal 2 to simplify

Rather than have a double linked list of dirty PSpaces the DosWriter has a std::vector<PSpace*> protected with a std::mutex. As PSpaces become dirty they are pushed onto the vector. The PSpaceDos dirty flag is cleared when ProcessAllGroups() swaps the vector with an empty one.