The Albireo SDK provides a software library that is expressly designed to integrate with any block, file, or unified storage architecture for embedded deduplication without performance impact. Now in its fifth release, Albireo technology is field-proven in thousands of OEM deployments, and has passed rigorous internal testing and evaluation within the world’s largest storage OEM as well as leading industry analyst firms. While Permabit’s core design (as a deduplication advisory service) guarantees data safety, the outstanding results of these evaluations serve as proof points of performance, scalability and resource efficiency. They also serve to validate the overall effectiveness of the Permabit Deduplication Index and Grid Index technologies.
The table below is a compilation of results reported by Permabit OEMs, verified by internal resources, and validated by independent industry analyst firms when evaluating or implementing Albireo.
|Single-core throughput||720 MB/s (4 KB chunk), 11.25 GB/s (64 KB chunk)|
|Large grid (30 nodes)||13 GB/s (4 KB chunk), 200 GB/s (64 KB chunk)|
|Reads per million hash lookups||600|
|Writes per million hash lookups/insertions||780 (sequential)|
|Cache hit rate||>99%|
|Network burden (grid)||52 bytes per hash|
|Minimum configuration tested||256 MB, in-memory index (embedded)|
|Maximum grid tested||96 nodes (4 GB in-memory index per node)|
|Performance scaling||Scales linearly on a >73% slope|
|Optimum data capacity, minimum config||10 TB (4 KB chunk), 160 TB (64 KB chunk)|
|Optimum data capacity, maximum grid||15 PB (4 KB chunk), 240 PB (64 KB chunk)|
|Memory required||1 GB of memory per 40 TB of unique data (4 KB chunk)|
|Storage required||180 GB of storage per 40 TB of unique data (4 KB chunk)|
|Deduplication rate: User Directories||3.9:1 (variable chunk)|
|Deduplication rate: Exchange Backups||7.4:1 (fixed 4 KB chunk)|
|Deduplication rate: GNU Tar Backups||12.8:1 (13-week retention cycle)|
|Deduplication rate: VMware||21.33:1 (10 images, fixed 4 KB chunk)|
|Deduplication rate: Oracle 11g||10.39:1 (12 incremental copies, fixed 4 KB chunk)|
Sources of Data
OEM Validations and Implementations
The leading OEMs have measured Permabit Albireo performance, scalability, and efficiency in a variety of environments, and over a wide variety of enterprise workloads. These OEMs have validated Albireo capabilities for inline and post-process deduplication, single-node and scale-out architectures, and in block, NAS, and unified storage systems.
Permabit began testing the underlying indexing technology on which the SDK is based in 2000. The organization utilizes an automated testing framework consisting of 250 systems that run over 2,500 tests. A heavy emphasis is placed on regression testing, and no code changes are accepted without accompanying test coverage. All of these factors contribute to the rock-solid stability of the Albireo index in field deployments. Automated test tools are used to measure performance, scalability, and efficiency.
Independent Industry Analysis
Albireo performance has been independently evaluated by both the Enterprise Strategy Group (ESG) and Wikibon.
An August, 2011 report concluded, “ESG Lab has confirmed that Permabit Albireo deduplication advisory services work as advertised. Inline and post-process deduplication support was added to a user space file system with only six Albireo function calls and 52 lines of code. The capacity of real-world data sets were reduced between 33% and 97%. The patented deduplication lookup and indexing algorithm was fast and efficient. Permabit deduplication was observed running on more than one server for a scalable global pool of deduplication.“
Wikibon has developed the CORE (Capacity Optimization Ratio Effectiveness) metric for comparing data optimization technologies. In a recent report, Wikibon used this methodology to compare Albireo with data optimization solutions from NetApp and IBM. Based on strict criteria, Permabit Albireo outperformed all other solutions in this evaluation.