17 Durability

Durability relates to the 'D' in the so called ACID properties typically expected of OO and relational database systems.

ACID properties are particularly relevant to the management of data that relates directly to real world processes such as airplane reservation systems, or financial systems. In these cases, a transaction on a computer is associated with events in the real world. For example, when a client withdraws cash from an ATM. Clearly it is necessary for the database to correctly record all such withdrawals. This leads to the durability requirement. In practice this means that every transaction must be flushed to disk as part of the commit.

Ordinary storage devices can retain write caching while supporting explicit durability barriers. The operating system translates a file flush into the appropriate device cache flush or FUA operation. The barrier is nevertheless expensive compared with an in-memory transaction: a thread executing serial durable transactions must wait for one before beginning an action that depends on the preceding commit. Write ordering on storage devices describes these mechanisms and the conditions under which they can be trusted.

The same limit applies when transactions are serialised by contention rather than by the application thread—for example, when they repeatedly update the same pages or shared state. A synchronously durable WAL system must complete a flush covering each commit before the next dependent commit can advance. Without a set of concurrent commits to batch, storage-barrier latency directly limits TPS.

On a rotating HDD, a forced flush can wait for cached writes, head movement, and media rotation. The durable transaction rate is bounded by the complete write-and-barrier latency, which varies with the drive and workload. A 10 ms barrier permits at most 100 serial durable transactions per second, before any transaction-processing cost is included. Barrier latencies in this range place a rotating HDD in the same general territory as the 50 to 70 flushes per second historically observed on some 5400 and 7200 RPM IDE drives.

An SSD removes mechanical seek and rotational delays, but does not make a durability barrier free. A synchronous request still crosses the operating system, filesystem, driver, controller, and device firmware, and may need to drain volatile state and complete NAND or mapping-metadata updates. Latency varies greatly between SATA SSDs, ordinary NVMe SSDs, devices with power-loss protection, and specialised low-latency media. Measurements in Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs show that even when device latency has fallen to tens of microseconds, operating-system overhead is a material part of a synchronous write()+fsync() path.

The arithmetic illustrates the limit independently of any particular device:

In practice, a rotating HDD supports only tens to low hundreds of serial forced-durability operations per second. Stock SATA and NVMe SSDs can raise this to hundreds, thousands, or in favourable cases low tens of thousands per second, with large differences between devices, filesystems, workloads, and power-loss-protection arrangements. These are barrier rates rather than complete transaction rates; transaction processing reduces the achievable result further. SSDs therefore raise the ceiling substantially, but a forced flush still destroys the ordinary LSS transaction-performance model.

By contrast the LSS focuses on the management of data that doesn't (in a transactional sense) relate to real world processes. Examples are editing of text documents, spreadsheets, statistical analysis, web browsing, GIS, multimedia databases, source code repositories and CAD. In these cases the durability constraints can be relaxed a little - by only flushing transactions to disk every few seconds. Atomicity is still required to protect the integrity of the data. However, a transaction only "commits" in the sense of defining an atomic unit of work, rather than demanding it go to non-volatile storage as part of the commit.

This of course means that a user may lose some edits on system failure, but losing at most a few seconds of work is fine for the type of data managed by the LSS.

The exceptional FlushWhenClose() mode allows an LSS transaction and all preceding transactions to request durability before close returns. It exists for external coordination protocols such as two-phase commit. Two-phase commit is blocking and imposes coordination and forced-durability costs that make it expensive and impractical. Those costs conflict with the normal LSS performance model; its mention here is not a recommendation. Normal transactions retain the high-throughput path described above. The current RAS interface still needs an explicit non-volatile-storage operation before the exceptional mode can make that guarantee reliably on every supported platform.