57 Proposed MVCC support in the LSS

Status: proposal. This chapter describes intended behaviour, not the current LSS implementation.

This chapter describes the intended changes required to make the LSS support multiversion concurrency control (MVCC). It is a proposal, not a description of the current implementation. The existing LSS has one mutable Recoverable Packet Map (RPM), and clients must prevent a serial element from being rewritten or deleted while it is open for reading. The proposed design replaces that restriction with stable read snapshots. A reader will continue to see the same committed rendition of every serial element for the lifetime of its read transaction, even while the writer publishes later renditions.

The proposal retains an important simplifying property of the LSS: there is one serial writer. MVCC is used to allow any number of readers to run concurrently with that writer; it is not intended to introduce concurrently mutating LSS transactions. Write transactions remain totally ordered, so the LSS does not require deadlock detection, general write-write conflict detection, or a multi-writer commit protocol.

At a high level, a snapshot is formalised as a map from Seids to serial elements. A consistent snapshot means that this map remains unchanged for the lifetime of the read transaction, regardless of writers creating new serial elements, deleting existing serial elements, or rewriting existing serial elements. A write transaction starts from the state left by preceding write transactions and updates its own map without changing the maps used by existing readers. Close() completes the transaction, and subsequent write transactions see its changes, but existing and new readers continue to use the most recently published LSS snapshot. The old map and every packet needed through it remain available until its last reader has finished.

LSS writer transactions are serialised. Each is assigned a unique, monotonically increasing 64-bit transaction sequence number from the history of the LSS, going back to when its empty file was first created. Only a subset of writer transactions are required to publish snapshots. In the following example, writers 100 and 102 decay directly into already-pinned readers at the instant they call CloseAndPublishSnapshot(), whereas writer 101 closes without publishing. The writer transactions need not be contiguous in time, and long-lived readers can overlap idle periods, later writers, and later readers.

LSS API changes

The API must express snapshot lifetime explicitly. It is not sufficient for each call to ReadSerialElement() to select the latest mapping independently: two such calls made by one logical operation could straddle publication of an LSS snapshot and observe an inconsistent mixture of states. A read transaction therefore owns a snapshot, and every read performed through that transaction resolves its Seid through the same RPM root.

Read transactions

The LSS will provide the following explicit read-transaction handle:

struct ILssReadTransaction
{
    virtual bool SerialElementExists(Seid seid) const = 0;
    virtual ICloseableInputStream* ReadSerialElement(Seid seid) const = 0;
    virtual IContiguousSerialElement* ReadContiguousSerialElement(Seid seid) const = 0;
    virtual void Close() = 0;
};

struct ILogStructuredStore
{
    virtual ILssReadTransaction* OpenReadTransaction() const = 0;
    // ...
};

Putting the read operations on the transaction handle ensures that every read in the transaction uses the same snapshot, preventing an operation from observing an inconsistent mixture of states. The handle is both the authority to read and the object that pins the selected root. This design rejects passing a separate snapshot handle to read methods on ILogStructuredStore, because putting the operations on ILssReadTransaction expresses their required snapshot directly. There is no implicit selection of a new root during a transaction.

Opening a read transaction selects and retains the most recently published snapshot. Closing the transaction releases that snapshot. The selected snapshot may already be stale when the read transaction begins: it does not include writer transactions completed since the most recent publication. A read transaction is read-only; it cannot be promoted into a write transaction. It may be used for an arbitrary number of serial elements and for serial elements discovered by following information read earlier in the same transaction.

The following rules form part of the API contract:

• Every operation on a read transaction observes exactly one committed snapshot.
• The snapshot's map from Seids to serial elements remains unchanged until the transaction is closed, regardless of writers creating, deleting, or rewriting serial elements.
• Every serial element in that map remains readable regardless of whether its RPM nodes or serial-element data are currently resident in memory.
• A deletion committed after the snapshot was opened does not remove the element from that snapshot.
• An element created after the snapshot was opened does not appear in that snapshot.
• Closing an LSS while read transactions or streams are open remains an error.
• A read transaction must be closed even when a read operation throws an exception.
• An ILssReadTransaction can serve concurrent immutable reads on several threads. Each returned stream is confined to the thread using that stream.

An input stream inherits the snapshot of its transaction. Closing the transaction while one of its streams remains open is an error, because the stream may still need RPM nodes or data packets retained by the snapshot. Closing a stream does not close the transaction; this permits several sequential or simultaneous streams to share one consistent view.

The owner of a read transaction closes it only after all readers and streams using that snapshot have released it. Concurrent read support does not require concurrent mutation of the RPM; it requires thread-safe immutable traversal, lazy loading, and segment cache access.

Rejected allowStale alternative

An alternative API could make OpenReadTransaction() take a Boolean allowStale argument. When true, it would immediately pin the most recently published root. When false, it would have to synchronise with the single writer, wait behind any write transaction that already owns the writer lock, publish the latest completed writer frontier, and only then return a read transaction. The resulting snapshot would include every write transaction completed before that publication point, but opening the read transaction could block for the duration of an active write transaction.

This design rejects that freshness flag as the mechanism for associating a read snapshot with a particular writer view. The LSS instead provides ILssWriteTransaction::CloseAndPublishSnapshot(), which completes the writer transaction, publishes its exact working frontier, and returns an ILssReadTransaction pinned to the resulting root. This directly transfers a mutable writer view into its immutable read view without a second operation racing to identify what "current" means.

Write transactions

The existing ILssTransaction is replaced by ILssWriteTransaction. There is still at most one open write transaction. It owns a committed base root and a private working root. Reads made through the write transaction observe the base snapshot plus that transaction's own writes and deletions.

struct ILssWriteTransaction
{
    virtual bool SerialElementExists(Seid seid) const = 0;
    virtual ICloseableInputStream* ReadSerialElement(Seid seid) const = 0;
    virtual IContiguousSerialElement* ReadContiguousSerialElement(Seid seid) const = 0;
    virtual ICloseableOutputStream* WriteSerialElement(Seid seid) = 0;
    virtual bool DeleteSerialElement(Seid seid) = 0;
    virtual void FlushWhenClose() = 0;
    virtual void Close() = 0;
    virtual ILssReadTransaction* CloseAndPublishSnapshot() = 0;
};

Read-your-writes behaviour is important. After replacing an element, a read through the same write transaction returns the replacement. After deleting it, existence tests and reads report that it is absent. Other read transactions continue to see the previously committed rendition until the write frontier is published at an LSS persistence boundary; read transactions already open continue to see their existing snapshot.

Both streaming and contiguous reads resolve through the write transaction's private working root. Any output stream for the element must be closed before it is read, and every returned input stream or contiguous serial element must be closed before the write transaction is closed or published.

There is no concept of aborting an LSS transaction. Close() completes the transaction but does not require creation of a reader-visible RPM snapshot. Its changes become part of the completed writer frontier from which the next write transaction proceeds. CloseAndPublishSnapshot() completes the transaction, atomically publishes that frontier as an immutable RPM root, consumes the write transaction, and returns an ILssReadTransaction pinned to exactly that root.

Snapshot identity and diagnostics

A transaction sequence number is useful as a diagnostic snapshot identity. It can be exposed for tracing and testing, but callers must not use it to locate packets or mutate retention state. The snapshot handle, rather than a bare sequence number supplied by a caller, proves that the corresponding root is still pinned.

Statistics should report the number of active read transactions, the oldest active snapshot, the age of that snapshot, the number of deferred root releases awaiting the writer, the oldest deferred release, the number of retained RPM nodes, and the bytes and segments retained solely because of old or deferred snapshots. These values are essential when diagnosing either a slow reader that prevents reclamation or an idle store whose writer has not yet drained completed releases.

Implementation changes

Separate committed, working and checkpoint roots

The current LSS owns one mutable RPMRoot. MVCC requires at least three distinct roles:

• Published root: the newest completed LSS root offered to readers.
• Working root: the writer-private frontier containing completed unpublished transactions and the active transaction's changes.
• Checkpoint root: a root pinned while its state is made durable and installed in the root block.

These roles may temporarily refer to the same root, but the implementation must not assume that they do. A reader can hold an older published root while many newer write transactions complete without publication. A checkpoint can serialise one completed root while the writer proceeds to construct and later publish another root at an LSS persistence boundary.

Copy-on-write RPM

The RPM is an eight-level radix tree selected by the eight bytes of a Seid. That structure is naturally suited to copy-on-write. To change one level-zero entry, the writer copies the nodes on the path from the root to that entry and shares every unchanged subtree. A transaction changing nearby Seids can reuse paths it has already copied.

old root R100 ── A ── B ── old leaf
                 └──────── many unchanged subtrees

new root R101 ── A'── B'── new leaf
                 └──────── the same unchanged subtrees

Published nodes are logically immutable. A writer must never alter a node reachable from a published, checkpoint or reader-pinned root. It first ensures that the path belongs exclusively to its working view, copying nodes where necessary, and then changes the private copy.

The present RPM representation has parent pointers, mutable child arrays, dirty flags, and eviction state. These concerns need to be separated. A node in a DAG can have more than one parent, so a single parent pointer is not meaningful. Dirty and construction state belongs to the writer or checkpoint machinery, not to a shared immutable logical node. Physical cache state may remain mutable only where it does not alter the logical contents of the snapshot.

Deferred single-writer reference counting

RPM nodes use intrusive reference counts, but those counts need not be atomic. The one LSS writer has exclusive authority to change every logical RPM-node reference count. Each retained logical root owns a reference to its top node, and each internal node owns references to its logical children. Copying a path, installing a child, publishing a root, releasing a writer base, checkpoint retention, and the recursive destruction of an unreachable subtree are all performed by the writer. Readers only traverse immutable nodes; they never increment or decrement references within the RPM DAG.

This deliberately differs from a graph built from shared_ptr. Atomic reference counts on every shared RPM edge would introduce read-modify-write operations and memory barriers throughout a large, frequently shared radix-tree DAG. The LSS already serialises mutation, so paying for concurrent ownership mutation at every node would discard an important benefit of its single-writer design. Centralising logical reference-count changes in the writer permits ordinary integer counts and makes the ordering of root retirement, packet retirement, SUT accounting and node destruction explicit.

Closing a read transaction

Closing an ILssReadTransaction does not immediately decrement the associated immutable RPM root. It makes the transaction unusable and submits a deferred root-release record to the LSS. The root remains physically retained until the LSS writer drains that record. Delayed release is safe: it can retain memory and packets for longer than necessary, but it cannot allow a reader-visible node or packet to be reclaimed too early.

last reference to an LSS read transaction is released
    └── closes the ILssReadTransaction
            └── enqueue deferred release of RPM root R100

next LSS writer obtains exclusive ownership
    └── drain deferred releases
            └── decref R100 and recursively release newly unreachable nodes

The deferred-release record must keep the root identity valid until it is consumed. Closing may occur on any reader thread, so transferring the record to the writer requires one coarse-grained synchronisation operation, such as a mutex-protected list, an MPSC queue, or a per-thread retirement list handed to the writer. This is one synchronisation event per retired LSS snapshot, not one atomic operation per reader, serial-element read, RPM node, or shared edge. Even the root-level event is expected to be relatively infrequent.

Draining deferred releases

After acquiring exclusive writer access, a write transaction drains all pending root releases before it begins modifying the working RPM. For each released root the writer decrements its ordinary reference count. A count reaching zero causes the writer to release the node's logical children, which may recursively make whole unshared paths unreachable. The same pass performs or records the corresponding packet-retirement and segment-utilisation changes. No reader can race with these count updates: a reader with a live transaction still retains its root, while a closed transaction is no longer permitted to traverse it.

void DrainDeferredRootReleases()
{
    for (RPMNode* root : TakeDeferredRootReleases())
        Decref(root);
}

void Decref(RPMNode* node)
{
    assert(HasExclusiveWriterAccess());
    assert(node->refCount > 0);

    if (--node->refCount == 0)
    {
        for (RPMNode* child : node->logicalChildren)
            if (child) Decref(child);
        RetireNodeMappingsAndDelete(node);
    }
}

The code is illustrative: a lazily unloaded child may be represented by a packet position rather than a resident pointer, and destruction must update the ownership representation actually used by the RPM. The invariant is that logical child ownership is changed only under exclusive writer access. A physical cache reference used to keep a lazily loaded node in memory is a separate concern and must not silently become another concurrently modified logical RPM reference count.

Creation and acquisition of snapshot handles

CloseAndPublishSnapshot() is the normal writer-to-reader acquisition path and requires no concurrent increment. While it already has exclusive writer access, the LSS installs the immutable root's ownership and constructs an ILssReadTransaction for it. Readers retain that transaction handle and never touch RPM reference counts.

The exceptional OpenReadTransaction() operation still needs a safe way to acquire the currently published root. The simplest initial implementation serialises this short acquisition with root publication and retirement, increments the root using the same exclusive ownership discipline, and then releases the lock before any serial-element I/O. If strict writer-thread-only count mutation is required, the acquisition can instead be routed through the writer; this additional machinery is introduced only if concurrent direct LSS clients require it.

Idle stores, maintenance and shutdown

If no later write transaction opens, deferred releases may otherwise remain pending indefinitely. That does not compromise correctness, but it delays memory and segment reclamation. The queue must also be drained at maintenance boundaries that already obtain exclusive LSS access, including checkpoint or cleaning preparation where appropriate, and while closing the LSS after all read transactions have closed. An implementation may additionally drain when the queue or retained-byte estimate crosses a threshold. A dedicated reclamation thread is unnecessary unless bounded release latency becomes an actual requirement.

Closing the LSS while read transactions remain open is still an error. Once they have all closed, shutdown drains the final deferred releases before destroying the RPM and verifies that every root reference is accounted for. Diagnostics must distinguish live reader-held roots from closed roots awaiting writer reclamation; otherwise a quiet store can appear to leak snapshots even though their release records are pending.

Lazy RPM loading and eviction

RPM nodes can currently be loaded lazily from their packet positions and evicted to limit memory use. MVCC must preserve that facility without modifying snapshot meaning. An immutable node may contain a thread-safe cache slot that changes from an unloaded packet position to a loaded immutable child. That is a physical caching transition, not a logical mapping change: every thread must obtain an equivalent child regardless of which thread performs the load.

Eviction similarly removes only a cached in-memory representation. It cannot remove or rewrite a logical child of an immutable snapshot. Loading, publication, reference release and eviction can run on different threads, so the ownership of a loaded child and the synchronisation of each cache slot must be specified carefully. It should be possible to prove that a node cannot be destroyed while a thread is loading or traversing one of its children.

Building a write transaction

Opening a write transaction continues from the latest completed writer frontier and creates a working-root reference. Writing or deleting a serial element appends the appropriate log records as it does now, but updates only the working RPM. It must not subtract the old packet from global live-byte accounting merely because the working view supersedes it; a published reader may still use the old mapping.

The working mapping must not expose a partially written serial element. The serial-element writer can accumulate the new packet chain and install its first mapping only after the output stream closes successfully. If writing fails, the incomplete records remain unreachable. A replacement should be represented as one logical mapping change from the old chain to the completed new chain.

Deletes are tombstones in the working view rather than destructive changes to older roots. Deleting and then rewriting the same Seid within one transaction must have well-defined read-your-writes behaviour and must produce only the final mapping when the root is published.

Atomic completion and root publication

Close() completes a log transaction and advances the unpublished writer frontier without changing which root future readers select. CloseAndPublishSnapshot() additionally converts that frontier into an immutable published root and returns a read handle pinned to it. Its required ordering is approximately:

1. Close every serial-element output stream and finish its packet chain.
2. Finish the working RPM and all transaction-local liveness changes.
3. Append the timestamped snapshot log record that marks the transaction complete for recovery.
4. Advance the committed transaction sequence number.
5. Make the root immutable and transfer one of the writer's references to a new ILssReadTransaction, thereby pinning exactly that root.
6. Publish the already-pinned root with release semantics.
7. Consume the write transaction and release its references to the old base and transaction-private state.
8. Perform a synchronous flush if requested by FlushWhenClose().

Ownership is continuous across this transition. The write transaction owns the working root from its creation, and CloseAndPublishSnapshot() establishes the returned read transaction's ownership before making the immutable root visible. There is no interval in which the root has been published but a reader must race to acquire a reference to it. This is both safer and cheaper than separate Close(), publish, and OpenReadTransaction() operations.

The returned read transaction owns the newly published LSS root. Publication is a non-failing in-memory step after all allocations and fallible preparation have completed. A mutex can initially protect selection and publication of the root and the exceptional acquisition performed by OpenReadTransaction(). It is never held while reading serial-element data. Readers with an existing transaction do not participate in this acquisition protocol.

Packet lifetime and segment utilisation

Packet retention is the most significant change outside the RPM. In the current implementation, a packet becomes dead when the one RPM no longer points to it. Under MVCC a packet is live while it is reachable from any reader-pinned root, the published root, the working root, or a root required by checkpoint and recovery processing.

reclaimable(packet) =
    not reachable from any retained logical root
    and not required by the recovery boundary
    and not reserved or actively accessed by the segment machinery

The SUT must therefore stop treating replacement in the newest mapping as immediate physical death. There are two promising accounting strategies:

• Reference accounting associates ownership with immutable level-zero RPM nodes or packet mappings. When the last node containing a mapping is destroyed, the corresponding packet chain loses its final logical reference and its bytes can be subtracted from segment utilisation.
• Epoch reclamation records the commit at which a packet was superseded. Because commits are totally ordered, it can be reclaimed when every active snapshot is newer than that retirement point and no checkpoint or recovery constraint retains it.

Deferred single-writer reference accounting follows the RPM DAG directly and permits node and packet ownership counts to remain non-atomic. Closing a read transaction merely queues its root release; the next writer can release batches of retired nodes and packet mappings while it has exclusive access. An epoch scheme remains a possible alternative if measurement shows that recursive reference accounting is too expensive, but it must preserve the same conservative lifetime guarantees. Whichever representation is chosen must account for every packet in a chained serial element, including chains spanning several segments.

Retirement and physical reuse are different events. A packet can cease to be visible to every logical root yet remain unavailable for overwrite until checkpoint, segment-cache access, reservations and the delta-FSS rules also permit reuse. MVCC adds a retention condition; it does not replace the existing crash-safety conditions.

Snapshot-aware cleaning

The cleaner currently determines liveness by comparing a packet with the position selected by the current RPM. It must instead respect all retained snapshots. A packet used only by an old reader is still live, and the containing segment cannot be returned to the FSS.

The simplest correct first design is for the cleaner to relocate packets selected by the current committed view while leaving old-snapshot-only packets in place. An old root contains an old physical position, so copying that packet elsewhere does not help the old reader unless the root can also be changed—which would violate its immutability. The source segment remains pinned until those readers finish.

A later design could introduce stable physical indirection so relocation updates a shared location cell without changing the logical rendition selected by a snapshot. That adds another concurrent data structure and recovery concern and should not be required for initial MVCC support. Long-running read transactions pinning sparsely used segments are an acceptable and diagnosable first trade-off.

Cleaner selection should take retained bytes into account. A segment with little current-state data but much snapshot-pinned data is not a useful cleaning candidate. Statistics should distinguish current live bytes, old-snapshot live bytes, reservations, and bytes that are logically retired but awaiting a checkpoint boundary.

Segment cache and open streams

Snapshot retention protects a packet's logical and on-disk lifetime. The segment cache continues to protect the memory containing a packet while an input stream accesses it. These are complementary: a snapshot may retain a packet for a long time without keeping its segment resident, and the segment may be loaded on demand when a later read through that snapshot reaches it.

Consequently, recycling checks must consider both MVCC reachability and the existing segment access and reservation counts. The transition to zero logical references must use the same synchronisation domain as the decision to enter the delta-FSS, so a reader cannot resolve a retained packet just as its segment becomes reusable.

Checkpoints

A checkpoint pins one committed RPM root and writes its dirty nodes bottom-up before installing its root in the distinguished root-block area. Writer transactions may continue to create later roots, provided all packets and RPM nodes needed by the checkpoint remain retained until installation is complete.

The checkpoint machinery must no longer use mutable dirty flags embedded in nodes shared with readers. It needs a way to identify which immutable nodes already have valid durable packet positions and which new nodes must be serialised. A newly constructed node can record immutable provenance or checkpoint metadata outside its logical contents. Once serialised, the durable position may be installed in a thread-safe physical metadata field or in checkpoint-owned tables.

The published root and checkpoint root advance independently. Publication provides in-process visibility; checkpointing shortens recovery and advances the boundary after which eligible segments can be reused. A graceful close must prevent new transactions, wait for readers or reject the close, finish required background work, and checkpoint an appropriate final committed root.

Recovery

Recovery still begins from the last valid checkpoint root and scans subsequent log records. A timestamped snapshot record identifies the end of a complete transaction. Changes after the last complete snapshot record are ignored. Replaying each complete transaction constructs the next committed RPM root, although recovery need retain only the newest root because no pre-crash read transactions survive process termination.

The on-disk format need not store all historical in-memory roots. MVCC history is retained for active runtime snapshots, not as a promise to reopen an arbitrary old snapshot after restart. If copy-on-write RPM packets change the checkpoint encoding or node metadata, the root-block schema must be versioned and old stores must either remain readable or receive an explicit migration tool.

Memory ordering and locking

The single writer serialises logical mapping changes, but several shorter-lived synchronisation problems remain. Readers concurrently retain the published root, traverse and lazily load RPM nodes, access the segment cache, and release old roots. Background checkpoint and cleaner tasks also retain roots and update physical bookkeeping.

The design should use a small publication lock or a proven atomic-reference scheme to prevent a root from being destroyed between loading its pointer and retaining it. After a reader has retained a root, ordinary logical RPM traversal should require no global writer lock. Locks for lazy loading, the segment cache, SUT and FSS should have a documented order, and root destruction must not call into those subsystems while holding a lock that creates an inverse ordering.

Resource limits and operational policy

MVCC replaces blocking between readers and the writer with retention pressure. A reader that remains open indefinitely can prevent old packets and poorly utilised segments from being reclaimed. The LSS must make this cost visible and may provide configurable warnings or limits for snapshot age and retained bytes.

Forcibly invalidating a snapshot would break the primary API guarantee and should not be the default. If an installation requires hard resource bounds, expiration must be an explicit policy under which later reads fail with a specific exception. Normal operation should instead identify the owner and age of long-lived transactions through tracing and statistics.

Validation and tests

Correct reclamation is more difficult than retaining versions, so implementation should proceed in stages. The first working version can conservatively retain every superseded packet. Once snapshot selection and atomic publication are thoroughly tested, packet retirement, SUT changes and cleaning can be enabled incrementally.

Required tests include:

• A reader continues reading an old multi-segment rendition while the writer replaces it.
• Several Seids read through one transaction always come from one committed root.
• A writer observes its own creations, replacements and deletions.
• Readers opened before and after CloseAndPublishSnapshot() observe the expected different renditions.
• Delete followed by recreation of the same Seid does not confuse an older snapshot.
• Releasing the last old snapshot makes its retired packets eligible for reclamation.
• Closing a read transaction queues a root release without changing any RPM-node reference count on the reader thread.
• The next writer drains queued releases and recursively decrements RPM nodes using only its exclusive access.
• Many reader threads can close snapshots concurrently without losing or processing a deferred release twice.
• A store with no subsequent writes drains deferred releases during maintenance or orderly shutdown.
• No snapshot-reachable segment enters the FSS, including during cleaner and checkpoint activity.
• A snapshot can reload an evicted RPM node and data segment after later commits.
• Failure at every log-write, snapshot-record, publication and flush boundary recovers a valid committed state.
• Many readers can share a root without copying the RPM or holding the writer lock during I/O.
• Diagnostics correctly attribute retained nodes, packets and segments to old snapshots.

Stress testing should combine long and short readers, large packet chains, repeated replacement of hot Seids, deletions, RPM eviction, segment-cache pressure, cleaning, checkpoints and injected I/O failures. The validator should be extended to calculate packet reachability from every retained root and compare it with SUT accounting. Assertions should favour retaining too much storage over reclaiming a packet whose reachability is uncertain.

Suggested implementation sequence

1. Specify API lifetime, visibility, thread-affinity and error semantics in executable tests.
2. Implement an in-memory copy-on-write RPM with retained immutable roots.
3. Add explicit read transactions and make write transactions use private working roots.
4. Implement CloseAndPublishSnapshot() as the atomic conversion from a write frontier to a pinned immutable read root.
5. Add deferred root-release submission and drain it under exclusive writer access.
6. Use non-atomic intrusive counts for logical RPM-node ownership and validate every increment and decrement as writer-owned.
7. Keep superseded packets conservatively live while validating snapshot behaviour.
8. Add snapshot-aware packet retirement and SUT accounting.
9. Make the cleaner and FSS rules aware of retained snapshots.
10. Adapt RPM eviction, checkpoints and recovery to immutable shared nodes.
11. Add operational statistics, leak detection and long-snapshot diagnostics.
12. Run correctness, crash-injection and performance tests before removing the old access-clash restrictions.

The log-structured layout already preserves old packet bytes until they are cleaned or their segments are reused. MVCC therefore does not require a second value store. The essential change is to retain the old mappings that identify those bytes and to make every reclamation path respect those mappings. Copy-on-write RPM roots provide stable logical snapshots; snapshot-aware SUT, cleaner, checkpoint and segment-reuse rules keep their physical packets alive for exactly as long as required.