25 Memory reclaimer proposal

Object lifetime issues

We have been having issues with using a ThreadBlockerWhileUsed to manage the lifetime of objects. Perhaps the issue relates to this: A surprise with mutexes and reference counts by Jonathan Corbet December 4, 2013.

Even if C++11 spec guarantees a ThreadBlockerWhileUsed can be used, maybe it's a bad idea because an implementation of std::mutex might be broken.

Consider that we go with essentially the current design which uses a ThreadBlockerWhileUsed, except the actual deletion of the object is delayed, by calling ScheduleWithDelay

struct X
{
    X() : count_(2)
    { 
        Async(DoWork()); 
    }

    void Dec()
    {
        std::lock_guard<std::mutex> lock(mutex_);
        if (--count_ == 0)
        {
            cv_.notify_all();
        }
    }
    
    void Close()
    {
        Dec();
        {
            std::unique_lock<std::mutex> lock(mutex_);
            cv_.wait(lock,[this]{return count_ == 0;});
        }
        delete this;
    }

    void DoWork()
    {
        ...
        Dec();
    }

    std::mutex mutex_;
    std::condition_variable cv_;
    int count_;
};

void main()
{
    X* x = new X;
    x->Close();
}

//////////// X.h
struct X;

X* Open();
void Close();

//////////// X.cpp

struct X
{
    X() : blocker(2) 
    { 
    }

    void Open()
    {
        Async(DoWork()); 
    }
    
    void Close()
    {
        --blocker;
        blocker.Wait();
    }

    void DoWork()
    {
        ...
        --blocker;
    }

    ThreadBlockerWhileUsed blocker;
};

X* Open()
{
    X* x = new X();
    x->Open();
    return x;
}

void Close(X* x)
{
    x->Close();

    // x will be deleted, but will take at least 100 msec!
    ScheduleWithDelay( [] { delete x; }, 100 );
};

class Scheduler
{
public:
    // Not copyable
    Scheduler& operator=(const Scheduler& rhs) = delete;
    Scheduler(const Scheduler& rhs) = delete;

    // Not movable
    Scheduler& operator=(Schedular&& rhs) = delete;
    Scheduler(Schedular&& rhs) = delete;

    void Close()
    {
        stop_ = true;
        thread_.join();

        for (auto& t : newTasks_)
        {
            t();    
        }
        newTasks_.clear();
    }
    
    void Add(Task t)
    {
        std::lock_guard lock(mutex_);
        newTasks_.push_back(t);
    }

    void Run()
    {
        xvector<Task> pendingTasks;

        while(1)
        {
            Sleep(100);

            // Execute the pending tasks
            for (auto& t : pendingTasks)
            {
                t();    
            }
            pendingTasks.clear();

            if (stop_) break;

            {
                std::lock_guard lock(mutex_);
                std::swap(newTasks_, pendingTasks);
            }
        }
    }

private:
    std::atomic<bool> stop_;
    std::thread thread_;
    std::mutex mutex_;
    xvector<Task> newTasks_;
};

Proposal

We want a facility for providing

1. a system wide thread pool
2. supports boost asio for I/O services
3. supports posting of tasks to be executed in the future.

This facility must be explicitly started and stopped.

It solves all the following:

1. cxMessage requirements for creating TCP servers, clients and connected sockets It elimninates the need to create Iocp?
2. LSS scheduled jobs:
   • lazy check pointer
   • lazy cleaner
   • lazy flusher
   • lazy writer
3. Postponed memory reclamation
4. Async DGS

This needs to be done in a way that's compatible with web assembly. Also it needs to use a small number of threads, so a ceda process doesn't waste too much memory for stack space.

IDEA for delayed tasks: We want to be able to post a task with a delay. Unfortunately this raises questions about how to abort it. So maybe it falls under the control of a TaskExecuter? That way we can close the task executer to quickly close all the delayed tasks it owns?

Reclaimer

The purpose is to reclaim memory with a delay. It's a thread-safe singleton. We don't care which thread does the deleting. It can be justified for two reasons:

• Memory deletion is actually expensive, best to offload it so GUI threads etc are more responsive
• Avoid assumptions about mutexes, such as when use ThreadBlockerWhileUsed to delete itself.

public API:

void StartTheMemoryReclaimer()
void StopTheMemoryReclaimer()
void AsyncReclaimMemory(const std::function<void()>& reclaimFn);