Flash, HDD, Hybrid Optimized Deduplication
Flash, HDD, and hybrid environments each have their own unique benefits. Flash delivers the best performance, while HDD delivers the lowest capacity cost. Hybrid approaches attempt to balance the usage between these two types of storage and deliver some of each benefit. Because HDDs are less expensive than flash, it can be challenging to deliver data reduction that has a low enough footprint to be cost-effective in this environment. What is needed is a single, cost-effective data reduction technology that is optimized for flash, disk, and hybrid storage environments.
Permabit’s Albireo product line is unique in its ability to deliver inline data reduction that excels in Flash, HDD, and hybrid environments.
Permabit’s Flash, HDD, and hybrid optimizations deliver inline data reduction with industry-leading performance, scalability and efficiency across all three environments — while still providing a solution that is extremely cost-effective.
How It Works
Permabit utilizes a number of techniques in its Albireo products that specifically optimize for different types of storage media.
Support for fast hashes – Permabit products fully leverage the performance of the non-cryptographic MurmurHash3 hash function. Hash computations are combined with bit-compare operations to ensure data integrity. Support for MurmurHash3 also allows OEMs to leverage the inexpensive sequential read property of flash storage and HDD for increased performance.
Write minimization techniques – Permabit’s Deduplication Index technology takes advantage of low-latency reads while minimizing writes through the use of a log-structured index design. Write minimization techniques for the Albireo Index are also employed to help reduce the wear typically caused by maintaining a deduplication index on flash devices.
In-memory caching – Permabit’s Deduplication Index technology uses in-memory caching techniques to avoid unnecessary disk I/Os. The techniques take advantage of both temporal and spatial locality data properties determined through over 10 years of experience with real-world enterprise deduplication environments to ensure that fewer than 1000 disk I/Os are required per million new blocks written to a system utilizing the index.
Synchronous write support – A synchronous write mode is supported for use with Permabit’s Virtual Block Store technology to meet the unique reliability requirements of flash-based storage solutions. In synchronous mode, Permabit products will not acknowledge a write request until the data and the associated metadata are stored. This allows our software to be able to recover from unforeseen circumstance, such as unclean shutdowns, with no loss of data.
Write coalescence – Modern flash devices use a number of techniques that accelerate the performance of linear writes to solid state storage (relative to random writes). Permabit takes advantage of this property to provide more balanced write performance characteristics, accelerating the performance of random writes by translating them into sequential I/O operations.