Deduplication and the chasm
With the Data Domain battle over, you may think that dedupe has crossed the Chasm made famous by Geoffrey Moore. It has in terms of awareness, but dedupe is still in its infancy in terms of wide scale adoption. Estimates are dedupe has penetrated less than one-third of the backup market and a lower percentage of secondary and primary storage.
I think dedupe adoption will accelerate over the next 18 months (or the equivalent of one product release cycle) and become a standard feature in storage. Why am I so bullish on dedupe adoption? Well, because it makes sense and it saves MONEY.
Reason #1:
EMC just invested $2.1B to acquire Data Domain. They now have as many flavors of dedupe technology as they do RAID configurations. They will execute and do their best to exploit the technology base they have built. That means they will spread dedupe over their product lines to grow their overall business. And they will do it quickly to establish/maintain market leadership.
Reason #2:
If you want to be a successful storage vendor, you have to have dedupe efficiency in your product. Think about the following scenario:
Vendor A sells their storage for $10/GB at a 50% gross margin (GM) or $5 GM/GB.
Vendor B is the lower priced vendor and they sell their storage at $8/GB and achieve a 50% gross margin or $4 GM/GB.
Now vendor A introduces dedupe to their product and it lowers the effective cost of their storage to a customer to $5/GB (@ modest 50% dedupe rate). Vendor B doesn’t see this coming and now must drop the real price of their storage to $5/GB to match the effective price of Vendor A. Ruh roh! Vendor B has a problem! They now generate only $1 GM/GB or 20% gross margin while Vendor A (with dedupe) generates $5 GM/GB and a 50% gross margin.
Vendor B is road kill unless they move fast. For this reason, I believe Vendors B-Z will jump on accelerated dedupe initiatives as if their lives depend upon it, because they do. Market forces are at work.
Reason #3:
Customers will soon demand dedupe in every storage product they buy. I can even see them demanding it in products they have already deployed. Why? Because storage managers, CIOs, CTOs and CFOs all need to do more with less and they want to be successful. Dedupe lowers storage acquisition, management, power consumption and real estate costs. Just ask your neighbor. He/she has heard about it on CNBC, Bloomberg or read about it in the local paper. Dedupe is everywhere.
In a 12-month time frame, I believe most storage POs will require a tested dedupe rate for cost justification and approval. More and more storage administrators will face the following questions from their boss when they are buying storage, “what is the dedupe rate on this storage?” If the answer is “dunno” or “it doesn’t have dedupe”, the POs will get held up for review or put out for competitive bid. Again, market forces are at work.
Reason #4:
The technology is available today. Some dedupe will make the grade, some will not. As newer, more powerful and less expensive processors come to market, many solutions will benefit from that advancement. But, riding the Intel curve simply isn’t enough. Scalable dedupe requires much more and Permabit has that available, now. We call it Dedupe 2.0. Customers will demand dedupe cost efficiencies from their vendors. They will have to adopt if they want to thrive.
What do you think?