Latest VDO Performance Results – High-end Performance Meets Software Defined Storage
Today Permabit announced that our Virtual Data Optimizer (VDO), using HIOPS Compression, achieved 8 GB/s throughput when tested on the Samsung NVMe Reference Design platform from Samsung Electronics, Ltd. This is astounding performance from a single node Intel server platform, particularly one that has no custom hardware acceleration feature for compression. As a follow on, I wanted to provide some background on why we wanted to collect these numbers, how we got them, and what they mean to data center managers who are considering adopting Software Defined Storage (SDS).
Why we bothered
The last year has seen a real change in focus among data center managers who are beginning to turn away from the monolithic storage arrays of the past, and look to a more flexible SDS approach. SDS offers freedom from hardware lock-in, and allows data centers to purchase hardware that offers the greatest efficiency in terms of performance and capacity. Data reduction technologies such as deduplication and compression are key to controlling hardware costs, as well as increasing storage density to save on space power and cooling. At Permabit we’ve evaluated the effectiveness of our HIOPS Compression technique across a wide range of data types. We’ve seen HIOPS Compression extend storage density by between 1.5X and 5X across a wide range of datasets including everything from Hadoop data to virtual machine images.
As a leading provider of enterprise flash storage, Samsung is well positioned to deliver high performance solutions to the market at a very low cost. And with these latest results, we wanted to demonstrate that VDO could be utilized to substantially reduce the cost of capacity as well.
Where the numbers come from
To evaluate VDO’s HIOPS Compression performance, we chose to use 24 480GB Samsung PM953 U.2 NVMe PCIe SSDs (solid state drives) running on the Samsung NVMe Reference Design platform. Samsung Electronics is one of the first companies to offer U.2 Gen 3 X4 NVMe PCIe SSDs. The PM953 that was used in the testing also features nine watts TDP (Total Dissipated Power) and a Z-height of 7mm. We tested on a single server platform running Red Hat Ceph Storage. Here are the results we saw:
VDO Performance Summary
|Metric||Samsung NVMe Reference Design|
|Sequential Read Throughput (88% compressed data)||8,043 MB/s|
|Sequential Write Throughput (88% compressed data)||3,639 MB/s|
|Sequential Read Throughput (unique data)||3,933 MB/s|
|Sequential Write Throughput (unique data)||2,579 MB/s|
As you can see, the combined solution of VDO, Red Hat Ceph Storage and the Samsung NVMe Reference Design platform delivered up to 8 GB/s on reads and 3.6 GB/s on writes. To measure throughput, we used Ceph’s rados bench tool that is developed and maintained by the Ceph community. The results demonstrate the performance advantage that compression can bring when data is highly compressible. With inline compression, a single read of compressible data from media is able to fulfill many more logical read requests than for non-compressible data. The same is true for writes where a single write to physical media can be performed for many logical writes. It is important to understand performance advantage of compression techniques varies with the compressibility of the data, the characteristics of the storage media used and other characteristics such as CP and memory of the system being tested. The Samsung NVMe Reference Design platform demonstrated there was a clear advantage to compression for the data generated by rados bench.
What it means to the industry
A decade ago, implementing Software Defined Storage (SDS) required development from scratch, a feat that could only be performed by organizations with huge software engineering organizations such as Amazon. During that timeframe traditional IT organizations had been focused on supporting data center servers, switches and storage equipment. Today this is changing due to the availability of powerful server options (as demonstrated by the Samsung NVMe Reference Design platform) alongside mature, open source software from Red Hat, Canonical and other vendors offering services supporting the software defined data center.
With high performance data reduction for SDS, data center managers reap huge economic advantages from vendor neutrality, hardware independence, and increased utilization, while still customizing for their own unique business requirements. Mature open source technologies like Ceph and Gluster can be combined with VDO to enable organizations to leverage hyper-scale economics in a more bespoke fashion, without giving up control to the more homogeneous public cloud environments.