Permabit Albireo SDK: Real-World Results

The Albireo SDK provides a software library that is expressly designed to integrate with any block, file, or unified storage architecture for embedded deduplication without performance impact. Now in its fifth release, Albireo technology is field-proven in thousands of OEM deployments, and has passed rigorous internal testing and evaluation within the world’s largest storage OEM as well as leading industry analyst firms. While Permabit’s core design (as a deduplication advisory service) guarantees data safety, the outstanding results of these evaluations serve as proof points of performance, scalability and resource efficiency. They also serve to validate the overall effectiveness of the Permabit Deduplication Index and Grid Index technologies.

 

Albireo Results

The table below is a compilation of results reported by Permabit OEMs, verified by internal resources, and validated by independent industry analyst firms when evaluating or implementing Albireo.

Characteristic Results
Performance
Single-core throughput 720 MB/s (4 KB chunk), 11.25 GB/s (64 KB chunk)
Large grid (30 nodes) 13 GB/s (4 KB chunk), 200 GB/s (64 KB chunk)
Reads per million hash lookups 600
Writes per million hash lookups/insertions 780 (sequential)
Cache hit rate >99%
Network burden (grid) 52 bytes per hash
Scalability
Minimum configuration tested 256 MB, in-memory index (embedded)
Maximum grid tested 96 nodes (4 GB in-memory index per node)
Performance scaling Scales linearly on a >73% slope
Optimum data capacity, minimum config 10 TB (4 KB chunk), 160 TB (64 KB chunk)
Optimum data capacity, maximum grid 15 PB (4 KB chunk), 240 PB (64 KB chunk)
Efficiency
Memory required 1 GB of memory per 40 TB of unique data (4 KB chunk)
Storage required 180 GB of storage per 40 TB of unique data (4 KB chunk)
Deduplication rate: User Directories 3.9:1 (variable chunk)
Deduplication rate: Exchange Backups 7.4:1 (fixed 4 KB chunk)
Deduplication rate: GNU Tar Backups 12.8:1 (13-week retention cycle)
Deduplication rate: VMware 21.33:1 (10 images, fixed 4 KB chunk)
Deduplication rate: Oracle 11g 10.39:1 (12 incremental copies, fixed 4 KB chunk)

 

Sources of Data

OEM Validations and Implementations

The leading OEMs have measured Permabit Albireo performance, scalability, and efficiency in a variety of environments, and over a wide variety of enterprise workloads. These OEMs have validated Albireo capabilities for inline and post-process deduplication, single-node and scale-out architectures, and in block, NAS, and unified storage systems.

Internal Resources

Permabit began testing the underlying indexing technology on which the SDK is based in 2000. The organization utilizes an automated testing framework consisting of 250 systems that run over 2,500 tests. A heavy emphasis is placed on regression testing, and no code changes are accepted without accompanying test coverage. All of these factors contribute to the rock-solid stability of the Albireo index in field deployments. Automated test tools are used to measure performance, scalability, and efficiency.

Independent Industry Analysis

Albireo performance has been independently evaluated by both the Enterprise Strategy Group (ESG) and Wikibon.

ESG

An August, 2011 report concluded, “ESG Lab has confirmed that Permabit Albireo deduplication advisory services work as advertised. Inline and post-process deduplication support was added to a user space file system with only six Albireo function calls and 52 lines of code. The capacity of real-world data sets were reduced between 33% and 97%. The patented deduplication lookup and indexing algorithm was fast and efficient. Permabit deduplication was observed running on more than one server for a scalable global pool of deduplication.“

Wikibon

Wikibon has developed the CORE (Capacity Optimization Ratio Effectiveness) metric for comparing data optimization technologies. In a recent report, Wikibon used this methodology to compare Albireo with data optimization solutions from NetApp and IBM.  Based on strict criteria, Permabit Albireo outperformed all other solutions in this evaluation.