The LSS optionally supports hot standby and incremental backup.
Note however that at present, hot standby has fairly relaxed assumptions about how up to date the standby store must be.
The backup / hot-standby system is compatible with 24x7 operation of a store which continuously reads/writes large amounts of data. The LSS properly supports applications that are write bound for prolonged periods.
When the LSS is created or opened, a path to a directory for the delta files can optionally be provided. LSS delta-files will automatically be written to this directory. These files are named as follows
nnnnnn.lssdelta
where nnnnnn is a sequence number, called a Check Point Sequence Number (CPSN).
Note that the path to the main LSS file is independent of the path to the directory of delta-files. They could easily be on different hard-disks.
To avoid limiting the write performance of the system, it is recommended that a separate local hard-disk be used for storing the deltas. This will allow the deltas and the LSS file to be written concurrently. It also means either hard-disk can fail without losing data.
Note that with virtual file systems it is easy to have delta files written directly to a remote site. However that may expose the LSS to network outages. A better strategy may be to write deltas to a local hard-drive, and a separate process is responsible for copying these files to a remote site. During network outages the application is able to continue running.
A single delta-file is written for each check point on the LSS. The LSS stores a CPSN in the main LSS file. This helps ensure that deltas are applied in the right sequence. The CPSN directly corresponds to the sequence number used for naming the delta files.
With the default settings the LSS performs a check point after writing 128 segments (or 64 MB). A check point is also performed whenever the store is closed.
Delta files respect check point boundaries. Note in turn that check point boundaries respect both flush unit and transaction boundaries.
PROPOSAL [only partially implemented] Delta-files store time stamps, to allow the back-up system to generate a snapshot at a given epoch.
The LSS generates a 128 bit GUID called a Check Point Identifier (CPID) for each check point. The current CPID is stored in the root block of the LSS. Each delta-file stores an input CPID and output CPID. A delta file may only be applied if its input CPID matches the current CPID of the store. The store's CPID is then set to the output CPID defined by the delta-file.
Both the CPID and CPSN are used to validate a delta-file (i.e. to see whether it is allowed to be applied to the store). Note that two stores can share a common ancestry, then diverge. The use of CPIDs ensures that delta-files are never applied incorrectly.
Note that the first CPSN to be applied isn't specified on the command line to LssApplyDeltas.exe. Instead the LSS can work this out itself (because it stores the current CPSN in its root block). This, together with the CPID validation makes LssApplyDeltas.exe "idiot proof".
As long as the main application is not running, (and the main LSS file is not opened) it can be copied using the file system. This creates a "level 0 backup". The copy will of course have the same CPID and CPSN, recorded in the root block. However, there are two significant problems with making a complete copy of the store
Once a copy has been made the delta-files can be used to very efficiently and safely bring the copy into sync with the main store.
Consider that we have previously created a "standby" store (by making a file system copy). Let the 24x7 application be configured to automatically create the delta-files in the normal way. Let LssApplyDeltas.exe be run repeatedly so it applies deltas to the "standby" as soon as they become available. At quiescence the standby will match the main store.
PROPOSAL [not yet implemented] Let the LSS write each delta-file with exclusive write access but allow shared read access. This will allow LssApplyDeltas.exe to apply individual transactions to the standby, synchronising it with the main store as soon as possible.
Note that this process is compatible with 24x7 operation of the main store (because it never needs to be shut down).
It is easy to create any number of "standby" stores in various stages of how up to date they are, because deltas are not consumed when they are applied to a store. Furthermore the standby stores and the deltas can be backed up to tape etc. Therefore this approach provides a great deal of flexibility.
PROPOSAL [not yet implemented] :
A console application will be written that can merge a sequence of delta-files into a single "composite" delta-file, that now represents a range of CPSN. The application will confirm that the output CPID from each delta-file matches the input CPID of the next delta-file in the sequence. The composite delta inherits the input CPID from the first delta-file and the output CPID from the last delta-file in the sequence. When the composite is applied to a store, CPID validity testing is done in the normal way. Note that composite deltas themselves can take part in merging with other delta-files.
A composite delta file has a file name of the form
nnnnnn-mmmmmm.lssdelta
where nnnnnn is the first CPSN and mmmmmm is the last CPSN.
Note that a directory may contain individual delta-files as well as composite delta-files. LssApplyDeltas.exe is able to automatically find the files it needs to most efficiently bring a store up to date.
A merged delta may be somewhat smaller than the sum of the sizes of the individual deltas from which it was derived. For example, a serial element (with given OID) may have changed many times, or a serial element may have been deleted. Merging ignores the intermediate changes, and therefore is "lossy" because it is impossible to recover the fine grained shot-shots.
This approach is useful when the backup system should provide course granularity snapshot points in the longer past. For example consider the following requirements:-
The ability to merge deltas into composites, as many times as required allows such a backup system to be implemented easily and efficiently.
See: