Infscape UrBackup Appliance contains custom storage software specialized for backup and archival purposes. Several features such as a persistent write-back caching or background compression enable fast backups combined with slow storage devices (cloud storage or RAID), while maintaining data integrity on all levels. The ulimate storage location is either cloud storage (e.g. a Amazon S3 bucket) or multiple locally attached disks (RAID).
A persistent write-back cache means backups run at the speed of the caching device (e.g. a NVMe) as long as the data required for the backups fits into the cache device. All writes go first to the write-back cache and are then written after significant delay to the RAID disks/cloud storage. Data that is locally present in cache does not need to be loaded from the cloud storage or RAID reducing bandwith requirements or delays and charges for requests. Cache device failure does not corrupt the data and is recovered from without problems.
The RAID will automatically configure itself into groups with appropriate RAID levels. Disks with same sizes will automatically be arranged into the same groups. There are two parameters which influence how the disks are arranged. The maximum failure probability and the target overhead (influencing performance). The layouts are calculated by running simulations, taking into account for example unrecoverable read errors (URE) during recovery from a disk failure. The RAID can redistribute the data if e.g. a disk fails or the parameters are changed.
RAID metadata can be moved to a separate set of mirrored SSDs to further improve RAID performance. With this configuration, data will only be written to sequentially to the RAID disks.
All data is compressed and encrypted before writing it to the RAID/cloud storage. Currently zlib is used for compression. Authenticated encryption is used (AES-GCM).
If the RAID/cloud storage is idle it performs background compression. The much better, but also more CPU intensive, LZMA compression is used for this. If the background compression yields better results than the first zlib compression data will be rewritten to RAID/cloud storage with the better compression result.
The RAID/cloud storage dynamically sizes the stripe/object sizes. Stripes/objects which have partial writes automatically get transformed into smaller stripes/objects. This improves long-term performance, as the compressability and write performance of large stripes/objects is better, but decreases short-term performance because some large stripes/objects have to be rewritten as smaller stripes/objects in-order to decrease the large stripe/object write overhead.