The initial wave of data efficiency features for primary storage focus on
silos of information organized in terms of individual file systems.
Deduplication and compression features provided by some vendors are limited
by the scalability of those underlying file systems, essentially the file
systems have become silos of optimized data. For example, NetApp
deduplication can't scale beyond a 100 TB limit, because that's the limit in
size of its WAFL file system. But ask anyone who's ever used NetApp
deduplication if they've done it on a 100 TB file system, and you're likely
to hear "are you crazy?" It's one thing to claim that data efficiency
features can scale, quite a different one to actually use them with
performance at scale.
Challenges around scalability generally center on two areas: scalability of
random IO and memory overhead. Older solutions, like the one f... (more)
Over the last three years, the market has been abuzz with the news that
deduplication technology was going to change the economics of flash-based
storage systems forever. In fact just recently, in his article War Between
SSDs and HDDs Will Escalate Through 2016, industry analyst Ben Woo of
Neuralytix, Inc. noted:
"In the next two to five years, the only way flash-based storage vendors can
challenge HDD-based storage systems on price is by way of data efficiency.
The cost per unit of storage ($/GB) of HDDs is still 1/10the cost of NAND
flash. However, data efficiency technologies... (more)