Effective use of data reduction in the Public Cloud

| By: (56)


Permabit CEO, Tom Cook recently wrote about how data reduction technology can simplify the problems associated with provisioning adequate storage resources in the public cloud, while balancing performance and efficiency.  The good news is, taking advantage of data reduction software in the public cloud is easier than ever.

For example,Permabit’s Virtual Disk Optimizer (VDO) is a pre-packaged software solution that installs and deploys in minutes on Red Hat Enterprise Linux and Ubuntu LTS Linux distributions.  To deploy VDO in Amazon AWS, you provision Elastic Block Storage (EBS) volumes, install the VDO package in your VMs and apply VDO to the block devices represented for your EBS volumes.  Since VDO is implemented in the Linux device mapper, it is transparent to your applications installed above it.

As data is written out to block storage volumes, VDO applies three reduction techniques:

  1. Zero-block elimination uses pattern matching techniques to eliminate 4 KB zero blocks
  2. Inline Deduplication eliminates 4 KB duplicate blocks
  3. HIOPS Compression™ compresses remaining blocks 

 VDO DRe process

This approach results in remarkable 6:1 data reduction rates across a wide range of data sets.  At Permabit we take advantage of these savings in our own AWS dev and test environment and, as Tom’s post notes, a 300 TB configuration, assuming 50% utilization time, can save $150,000 over a one year period.  But that’s just the start!

The way data reduction is being applied at the block level means that the process doesn’t just apply to the active data, but also to snapshots. Data reduction of snapshots minimizes the cost of long-term storage in the public cloud adding to the estimate mentioned above.  What’s more, the reduced data can be uploaded to other public cloud providers or downloaded to your data center (provided VDO is installed there) to allow access to the optimized format. In addition, this capability enables you to realize substantial bandwidth reduction when migrating data from one provider to another.  The massive savings that result from these use cases can substantially boost the agility of your public cloud deployment.

By: (56)

Louis Imershein As Vice President of Product at Permabit, Louis Imershein is responsible for product management and strategic planning. He has 25 years of technical leadership experience in product management, software development and technical support. Prior to Permabit, Imershein was a Senior Product Marketing Manager for the Sun Microsystems Data Management Group. Before joining Sun, he was at SCO where his titles included Product Manager, Software Architect and Senior Technical Lead. Imershein has a Bachelor’s degree in Biological Science from the University of California, Santa Cruz and has earned certificates in software design, device driver development, system administration, and software testing.

Comments are closed