Gigabytes, Terabytes and soon Zettabytes of data are building
Data Mountains.

Find out how Permabit will help you
Escape from Data Mountain!

You see, when the data gets ‘fixed,’ that really means it’s getting ‘altered.’ What this means is that for the user to retrieve their data, the altered data has to be brought back through the appliance to bring to its original state. Data is written and read through the appliance. This becomes a single point of failure. What if that appliance fails? All data that was written through it has now suddenly become inaccessible. Yes, you can argue that some have HA capabilities, but now you’re doubling your costs of implementation and adding more things to manage. Isn’t the goal to simplify storage management?

What happens if you don’t want to use that appliance in the future? What if that vendor ends up being acquired and the new company decides to EOL that product line? You can’t just simply pull it out of your infrastructure. All the data needs to be extracted back out, a very time consuming process, or you’ll never be able to access it again — an even more daunting prospect!

This was the fundamental challenge Permabit ran across when we met with our storage partners to discuss Albireo. They weren’t interested in adding any new piece of hardware between the servers and the storage. They wanted to own access to the data sitting on their boxes. This is where Albireo differs from any other data reduction solution on the planet. We do not alter data…ever! Albireo acts as an advisory service to the storage and the storage determines how the data gets written. The storage system does the extent management, not Albireo! This not only provides incredible write performance, but most importantly, it provides complete data safety because it does not change the data.  For read operations, Albireo is not in the read path so there is no rehydration performance penalty either.

Storage vendors already do extent mapping when they implement technologies such as snapshotting or thin provisioning. These mechanisms have been in place for years and are hardened vendor optimized technologies. The same way storage vendors write data for those features is the same way they write data when using Albireo. When it comes time to reading the data, they just simply read data off of disk like they would if our product wasn’t there. So, if you want to turn off Albireo or take it out of the storage system, there is absolutely NO risk to the data! Beautiful, isn’t it? We think so!

While virtual servers are approaching deployment numbers in the hundreds of servers in larger enterprises, deployments of virtual desktops are in the thousands and in some cases tens of thousands of images being deployed! So, same scenario here – redundant operating systems, applications, and our favorite friend – service packs! The savings can be even greater here, compared to virtual servers, when leveraging deduplication technology on primary storage that is hosting these images.

The biggest concern with virtual desktop deployments is what is called the ‘boot storm’ problem. Basically, everyone arrives at work at 9am and boots their virtual desktop at the same time. You also have the same problem at the end of the workday with everyone logging off (and flushing their cache) at the same time. Can a storage infrastructure that has built-in deduplication handle that challenge? The answer is…it depends.

First off, the storage will need to scale. When you’re looking to deploy these centralized virtual desktop (or server) environments, you want to have a storage system that can scale to the hundreds of terabytes, if not petabytes. This is especially true if you’re a cloud storage service provider (note the trend highlighted in the previous post)! Once you go down the path of virtual environments, the request for new servers/desktops will increase dramatically. After all, you’re saving the company a ton of money by not buying all those new physical servers and desktops, right? So, if you wanted to deploy storage with deduplication, you don’t want to be hindered by tiny volume sizes of 16TB or so. You need scale!

Next, you’ll need to determine if deduplication will impact performance.  Should it be an inline or post-process function (or some other implementation such as our parallel process implementation)? What really drives dedupe performance rates is how quickly a digital fingerprint can be looked up in an index to determine if it’s a duplicate or not.  If a system can do this in the scale of microseconds (like Albireo), you could easily deploy dedupe as an inline solution without users ever noticing a difference in performance. If that’s not good enough, the parallel or post-process implementations are still options. It all depends on what storage you implement and what type of throughput is needed.

The next question is how fast can data be read? Wouldn’t it be nice if the data can be read off of disk without having to go back through the dedupe engine?  Direct access to data that has already been deduped? Too good to be true? Not so! That is exactly what Albireo has been designed to do (based on unanimous demand from our storage partners). The storage vendor wants to (and should!) own access to data. If you ever want to disable/remove the dedupe engine, you always want access to your data! If the dedupe engine is needed to read data back, then if that engine were removed, you’d be effectively left with a bunch of encrypted/inaccessible data. That is not acceptable! Do you want to be the one to tell thousands of users (or customers if you’re a cloud provider) they can’t access their virtual desktops or servers because you can’t read the data from disk? Not me!

Whether you’re a customer looking to a cloud provider in the future to house your virtual environments or you’re an enterprise looking to deploy additional storage internally to support this, make sure you’re asking your vendor whether dedupe technology is included or not. If it’s not, see if they’ll give you a 100:1 discount on their pricing instead!

Why do I bring this up?

Deduplication technology within primary storage continues to be driven mostly by the need to deal with the storage bloat created by virtual environments. If mid-size enterprises begin to look to cloud offerings to deliver their virtual environments, these providers better make sure the storage they’re deploying has built-in deduplication technology or they’ll put themselves out of business!

Let’s look at why virtual environments are such storage hogs. First, I’ll look at virtual servers. Most organizations start out with a golden set of virtual server images. From here, anytime a new request comes in for a server, they deploy another copy of that image and then it gets customized from there.

Let’s look at these images. Every one has the same operating system and the same basic applications. That means 80-90% of that image is duplicated in every image! Think of the savings you can gain by implementing deduplication technology on that storage. Also, within these images there is typically a ton of empty space (e.g. within VMware VMDKs). These also are great for dedupe! So, before you even deploy the image and accumulate new data, you’re way ahead of the curve.

Going forward, you need to deploy service packs to all those different images. Again, another great opportunity for dedupe. This doesn’t even take into account the actual data, which can potentially be highly duplicated as well (depending on the use case). This is why we’ve seen dedupe rates as high as 100:1! Wouldn’t you love to be able to store 100TB of information in 1TB of physical storage?

In my next post, I’ll take this one step further and look at virtual desktops.

Let’s take a look at what happened here and how this type of scenario plays out in the world of storage.
* The Cleveland Cavaliers had ‘the King’, LeBron James, for 7 years and they failed to win a championship.

* The Toronto Raptors had Chris Bosh, same results.

* The Miami Heat won one championship with Dwayne Wade (but he had Shaq with him at the time).

Basically, all three of these guys by themselves were far from a ‘Dynasty.’ All great players, but on their own, their teams simply did not have the right formula to become champions!

Large storage companies have some of the same attributes. They can add, via internal development, the technology pieces to fill out their portfolios (the Cleveland approach), as long as they don’t lag behind the market. But, we have seen companies use a combination of internally, OEMed and acquired technologies to accelerate their time-to-market.

That takes a mentality like Dwayne Wade and the Heat have shown. He put his ego aside and said ‘let’s get better – fast!’ They need to be open to outside technology (i.e. welcome other stars to the team) that can quickly become part of the team, enable them to become relevant and win.  Just like this much anticipated summer of 2010 free agency, you never know when you’ll have the right technology, at the right time, to solve an immediate problem. The perfect trifecta.

So, although some vendors will go down like LeBron’s jersey in Cleveland last night – up in flames – many others will find the missing pieces that can help transform their company into the next dynasty.

Game on!

What I would like to highlight is the other star that was ‘born’ with this announcement — that is Jered himself!

Move over Pink, here comes Jered Floyd – our very own rock star. Ever since Jered conceived of the idea of Albireo, he has been racking up more frequent flyer miles than Bono. We’ve been meeting with many of the industry’s biggest names to discuss Albireo over the last 6 months and that was before we even officially announced the product to the public!

Now that the cat is out of the bag, Jered has become even more popular.  Our biggest challenge now is his schedule!  Between analyst briefings, press interviews and partner meetings, Jered has become one popular guy. I frequently have to battle with Brett Hawkes, our VP of Business Development, on when I can get some free cycles to have him to talk to another analyst or press person wanting to find out more details about Albireo. You see, many have thought that Primary Data Optimization could not be done without having a performance impact, manipulating the data and risking its integrity, or losing features already built in to the storage systems. You can’t have your cake and it eat it too.

Contrary to what the self-proclaimed ‘Dr. Dedupe’ believes, we have delivered this cake and we’d be happy to share a piece with everyone! It is time for data deduplication for primary storage to become main stream and for dedupe to truly become ubiquitous.

Thanks Jered and I’ll have the tour shirts created soon!

I quote from the article:

“Interest in primary data deduplication is even stronger as storage managers struggle to gain an upper hand on capacity growth. Data deduplication for backup has been the hottest storage technology for a couple of years, although our surveys indicate that implementation is still fairly modest with approximately 20% to 25% of enterprise data storage shops using it. However, that hasn’t dampened interest in applying the same techniques to primary or nearline storage. On the Storage Priorities survey, 17% are currently deduping at least some of their non-backup storage, and another 20% have set their sights on adding it in 2010. And like solid state, another 40% will give it a serious look and evaluate it in the coming year.”

Gartner also had similar findings at their Data Center Conference in December 2009.

So, it’s clear that users are starting to ask for this technology to be leveraged in other areas beyond backup.  The question will become whether vendors can deliver the solutions or not.  Where does the technology belong? SAN (block-based) storage? File-based storage? Or higher up in the Operating System or the Applications?  Or perhaps in all layers?  If you could dedupe data with zero impact to performance and no data risk in deploying, wouldn’t you want it to be everywhere?  It’s kind of like having your cake and eating it too!  Let’s see if the vendor community can truly deliver Dedupe 2.0 and give the customers what they are asking for in 2010!

The Ghosts of Storage Past, Present and Future

While the ghosts of Christmas continue to haunt Ebenezer Scrooge in the various versions of Dickens’ Christmas Carol this time of year, we thought it would be fun to visit with the ghosts of data storage as we prepare to put a wrap on the first decade of the new millennium. As in Dickens’ story, the ghosts of storage past are much more pleasant to think of for most data center managers than the harsh reality of what they face today and their rather chilly future prospects.

The ghosts of storage past resurrect times of nearly limitless budgets and resources while countless regulations and compliance initiatives had yet to be conceived in some back office of the capital building.  Yes, the ghosts of storage past remind IT managers of a time when they could continue to fill “boxes” with information without fear; if a box were full, simply buy another box, it was a blank check to your storage hardware vendor of choice.

The possibility of having too much data seemed inconceivable during this time. For data was a valuable resource and should be collected and saved with vigor. Too much data? Impossible.

But as time wore on, the ghosts of storage past grew weary and gave way to the gate keepers of storage present. A far scarier existence for IT managers. Suddenly, gone were the abundance of resources and un-scrutinized IT budgets of yester year. In their place now stand mountains of internal regulations and policies and titles such as “procurement.”

IT now finds itself haunted by government regulations and terms such as e-discovery, FRCP and Sarbanes-Oxley. While demands come down from up above to handle more requests for data than ever before with nary half the resources of IT’s glory days.  While data stores continue to multiply within most organizations, funding for crucial projects to support the data onslaught is issued with the frequency in which Ebenezer handed out Christmas bonuses.

“More with less” has become the phrase uttered over and over by the ghosts of storage present and these words continue to send chills down the spines of the IT managers they haunt. Is there time to change? Or has the fate of IT managers and data center administrators already been sealed? Perhaps a visit with the ghosts of storage future will help shed some light on the situation.

The ghosts of storage future don’t represent the kindness of storage past or the harshness of storage present, but rather confusion as to what things may look like come the day when data reaches the point where it can multiply faster than technology can process. No, what is haunting IT managers is the uncertainty as to how they will continue to manage data in the face of shrinking budgets and resources and an escalation in the creation of information.

With new and more stringent regulations all but assured of becoming reality as the next decade commences, IT managers will continue to fear the great unknown. Will technology keep pace? Will compliance rules strangle the life out of IT? Will funding for critical projects ever become available again? Many questions for a ghost with seemingly few answers.

Will there be a Scrooge like revelation for data storage industry in the year(s) to come? We believe there is a change on the horizon that will lead to a happier and more prosperous year to come!

Wishing all of you a Merry Christmas and a Happy New Year!

Your friends at Permabit Technology

I’m a techie type that knows enough to get most things working but, sometimes I change one too many configurations and everything gets screwed up.  In this case however, it’s not me…it’s them! I’m talking about my last two week battle trying to get my VoIP phone in working order. I changed nothing in my network (really, nothing!) and my voice quality started to fall apart. Unfortunately, my VoIP provider is not the same provider as my cable internet provider. One simple feature is keeping me from getting everything through my cable provider. But, until then, I have to live with two throats to choke.

My last two weeks went like this:

1. Call cable company – checked signal, looks great. Must be a VoIP problem.
2. Call VoIP provider – do some tweaks and test and things look good. Next day, back to the same issue.
3. Call VoIP provider again – make some more tweaks, firmware update, reboot. Try again. Good for a day. Then same old story. Must be a cable problem.
4. Call cable company – They send out “PC tech” to check it out. Modem’s acting a little flaky – enough for him to want to replace it.  He does. Good for a day. Same problem.
5. Back to VoIP provider – some more tests, tweaks. Sounds good. Until next day. Everything always sounds great after a fresh reboot. It’s the next day it sounds awful.
6. Back to cable company – They come back out. Check signal. Looks great. No problem.
7. Back to VoIP provider – agreed to get on phone with me and cable company. Says latency is too long, dropping some packets. Cable company says looks good on their side.  Suggested replacing modem again. Don’t know what else to do. So replaced and now I’m in a 24 hour wait period to see if it sounds like crap tomorrow.

This got me thinking about my little personal data center crisis. If I could just get everything from one provider, I’d have one throat to choke. But, the features just aren’t there. I started thinking about the folks that run the “real” data centers and now really understand the desire to buy from one supplier, but understand why they typically veer of course. One stop shopping doesn’t always give you the best of the best. It gives you one throat to choke, but it doesn’t give you every feature you want.

Why can’t everything just be “plug-and-play” vs. “plug-and-pray?” When we talk to customer prospects, it’s pretty standard to see that most aren’t a one vendor shop when it comes to storage. They buy the best for what they need for primary. Typically use something different for NAS vs. SAN and totally different for backup. Even with market consolidation, you still have best-of-breed solutions deployed to solve specific problems.

With Permabit, that’s exactly what we do. No matter what storage you already have, we’re simply a “plug-and-play” scale-out NAS solution that can have dramatic savings across your entire storage environment.  With our built-in Dedupe 2.0 technology, we can store more data in less space than primary storage solutions. By offloading static data to our storage, you also dramatically reduce your active backup sets saving both time and backup costs in the process.  So, while we can’t help you choke your other vendor’s throats, we can help put a choke on your storage budget dollars!

While everyone else is in meetings this week, I thought I’d get caught up with some blogging.  I ran across a story last month that I meant to comment on, but never got to it, so now’s my chance.  It’s about the email antics associated with Mayor Thomas M. Menino’s administration.  It turns out that even though they were told last year by a judge to stop deleting emails, the practice still continued.

Isn’t it amazing that in this day and age that people still don’t understand the ramifications of deleting emails and destroying evidence?  Especially when a judge explicitly tells you not to do so?  According to the story at boston.com, Michael J. Kineavy, one of Menino closest aides, was very efficient at keeping his inbox down to a manageable size.  Every day he was cleaning up his emails, deleting them, and then emptying the deleted folders.  He had been doing this for five years.  Turns out he assumed someone or something else was making copies to be saved elsewhere.  You know what they say about assuming….

So, who’s to blame?  An employee diligently trying to stay within their mailbox storage limits?  Or does that happen to be the convenient excuse for deleting tons of email?  Well, there’s no excuse for either today!  There are many email archiving applications from companies such as: Atempo, Commvault, Mimosa, Symantec, and ZL Technologies, that solve both the mailbox storage management problem as well as the data retention needed for corporate or regulatory compliance.  These applications capture the emails and store them so that even if somebody accidentally (or purposely) deleted and email a copy can be kept in the archive.

Permabit offers the most cost-effective (via Dedupe 2.0) and scalable archive for these applications to store the emails (among tons of other data – it’s just a NAS box after all). The added benefit we have in our storage is the ability to create WORM based volumes (this doesn’t cost extra, it’s built right in) to store some or all of the emails.  What that means is that nobody can alter or delete those emails until the retention policies expire — in the case of the city of Boston, for two years.  So, rather than face steep fines or even prison time, isn’t it about time to put the right tools in place to protect everyone from these types of news stories?