And the winner is…DDUP, NTAP or EMC?
Finally, there’s a winner in the battle for DDUP! It’s EMC, right? Well, maybe, but maybe not.
Of the three participants in the battle for DDUP, I would rank the winners in this order:
#1 Data Domain management and shareholders. Outstanding execution, superb market timing and an unmitigated homerun.
#2 NetApp CEO Dan Warmenhoven and shareholders. He made a rational decision and dodged a dilutive bullet.
#3 EMC salesforce. You have a category killer to sell for the next year. Make hay while you can, Dedupe 2.0 is coming.
Let’s drill down a little on my rationale.
Point solution or point-in-time solution?
In a recent discussion with an industry analyst we discussed what exactly was the problem that the Data Domain product was trying to solve. They went after a solution to a problem that nobody really cared about until the 24-hour backup window could no longer be met. If an enterprise could achieve their backups in 23:59, there was no sense of urgency to change anything in their backup process. Data Domain convinced the industry that backup to disk could help address the backup window problem and do it very efficiently with the advent of data deduplication technology. The DDUP team did an outstanding job creating a very specific form, fit and functionally complete point solution that solved this backup pain and they did it with a finely tuned dedupe scheme designed for backup. That’s right, DDUP’s backup point solution is the uncontested “Dedupe 1.0″ winner.
But Data Domain is really a point-in-time solution addressing one piece (backup) of a much bigger and more costly issue – data stored in all storage tiers – primary, secondary/value tier and archive. Ubiquitous dedupe, or as we call it, Dedupe 2.0, is coming like a freight train. That means all primary and secondary storage will soon be optimized. With ubiquitous dedupe the need for a specialized deduped backup solution will slow to a trickle and then to a mere drip, drip, drip.
The Devil is in the Dedupe Details
Dedupe for backup is a relatively simple technology compared to scalable Data dedupe for primary and secondary storage. Most vendors employ a technique called “temporal locality” meaning they only compare hashes (unique identifiers) of new data chunks to hashes of recently seen chunks for dedupe (rather than to the whole index). In many ways this is a short-cut to obtain the performance and efficiency that they wanted to achieve to entice customers. It makes all the sense in the world for backup, but it falls down when the same information isn’t being backed up daily and dedupe rates drop off precipitously. Strike one.
Data Domain avoids this problem by only scaling to the smallest challenge, a capacity maximum of around 40 TB or so. If you need more storage, you get separate deduplication domains, with no capability to scale beyond the limitations of a single box. The big problem is the challenge to convert DDUP to a comprehensive hashing scheme, as Permabit employs, is equivalent to winning the Tour de France on a mountain bike. Strike two.
The biggest problem is DDUP doesn’t scale with today’s data growth. That means it isn’t relevant to the enterprise considering the whole storage picture and does not qualify as Dedupe 2.0. Strike three. Note all the solutions that address backup such as Quantum, Falconstor, IBM/Diligent and Sepaton employ similar techniques and suffer the same exclusionary limitations for Dedupe 2.0 membership.
The Numbers are Numbing
We’ve all seen and heard the numbers. DDUP holds a dominant market share for backup with revenue of approximately $300M. I’ve heard analysts point to a Total Addressable Market for backup dedupe of $1B or so, but if Dedupe 2.0 sweeps through the market over a product cycle or 18 months, then the market for point solution backup or Dedupe 1.0 appliances will peak in the next year. Don’t forget we’ve seen declining customer acquisition rates and declining sales force productivity in recent quarters by Data Domain (I expect the current quarter will be OK due to DDUP sales spiffs). The outstanding EMC sales team can fix that in the near term, but $2.1B (enterprise value) is a numbing number to pay for the Dedupe 1.0 winner.
So, who is the winner?
Certainly Data Domain won Dedupe 1.0. But the bigger win will be the company or companies who emerge with Dedupe 2.0 capabilities.
In this case losing may be winning. Think about it, if Dan Warmenhoven and NetApp can get 20X dedupe on NearStore products, he’ll regain backup market share and will devalue the Data Domain product overnight! This is going to get very interesting for sure.
Stay tuned, in my next post, I’ll dig into dedupe economics that will propel the rapid adoption of Dedupe 2.0 in the market.