Unified Storage and Data Optimization The Next Step in Storage Efficiency
Last year I discussed how important storage efficiency has become in my post, ‘The New Normal in Storage‘. I pointed to advancements in unified storage solutions and primary dedupe as technologies with great promise. The market is moving fast and leading storage companies are driving product innovation by integrating unified storage with data optimization. The focus is on delivering efficiency to address accelerating data growth. We’ll see these products introduced into the market this year and by 2011-12 they will be the leading storage products.
New information sources continue to produce mind boggling data growth. IDC predicts there will be 35 ZB (that’s zettabytes!) of data stored by 2020. So we need to attack all cost components of storage.
Let’s look at two fundamental storage efficiency moves that will hit the market in force this year and become dominant trends by 2011: unified storage and data optimization.
Unified storage, standardized architectures or converged platforms – whatever you call them – are all about driving cost out of storage hardware. Products coming to market soon will combine SAN, NAS and Object based storage software using a common hardware platform. Think about it, this is the same cost efficiency move that server companies made by moving to standard components! And, more recently, similar to what Google, Amazon and other companies have achieved by deploying software innovation on top of highly redundant and inexpensive hardware.
This is a massive development undertaking to ‘unify’ their block and file storage offerings and the long-term benefits are huge. First, they will achieve cost efficiency and scale economies by supporting one common platform, rather than three or more. And, they will get out of the hardware platform development business! Second, and more importantly, they will achieve greater software development efficiency by deploying software across all of their platforms. In fact, R&D will be focused on delivering common software attributes that span the unified offering and leverage the block or file system capabilities while taking advantage of the multi-processor technology on their next generation platforms. Future software development efforts will deliver a software layer that delivers features over SAN, NAS and object storage. That is software development leveraged over a larger business base and that equals R&D efficiency.
One high-impact software solution that is becoming ‘table stakes’ by customers is data optimization – eliminating duplicate blocks or chunks of data and 50% or more of the storage footprint. In order to be a unified feature, the data optimization solution must globally address both block and file storage as a software layer that handles deduplication - remember this is a unified storage solution. One deduplication scheme for the block system and another for the file system data isn’t a unified software solution. I’ll get to other required attributes for unified storage data optimization in my next post, but for now let’s just say the unified software must leverage the huge development investment made in the unified platform.
I think the killer unified storage offerings will deliver massive storage efficiency with the one-two punch of unified platform and data optimization software in one comprehensive solution. That will be a challenge for many storage organizations. Both unified storage and data optimization promise to be table stakes for next generation storage and will be requirements in the battle among major storage vendors to protect margins and drive to expand market share.