Over the last three years, the market has been abuzz with the news that
deduplication technology was going to change the economics of flash-based
storage systems forever. In fact just recently, in his article War Between
SSDs and HDDs Will Escalate Through 2016, industry analyst Ben Woo of
Neuralytix, Inc. noted:
"In the next two to five years, the only way flash-based storage vendors can
challenge HDD-based storage systems on price is by way of data efficiency.
The cost per unit of storage ($/GB) of HDDs is still 1/10the cost of NAND
flash. However, data efficiency technologies (such as deduplication and/or
compression) from a variety of vendors are showing data efficiency ratios
that are over 10:1. The cost of SSD storage media is now coming in line with
the cost of high-end HDDs."
But the fact is, the performance of these all flash storage systems that
The initial wave of data efficiency features for primary storage focus on
silos of information organized in terms of individual file systems.
Deduplication and compression features provided by some vendors are limited
by the scalability of those underlying file systems, essentially the file
systems have become silos of optimized data. For example, NetApp
deduplication can't scale beyond a 100 TB limit, because that's the limit in
size of its WAFL file system. But ask anyone who's ever used NetApp
deduplication if they've done it on a 100 TB file system, and you're likely
to hea... (more)