Unstoppable Data Growth: HDDs Remain Essential as New Applications, Use Cases, and Connected Devices Multiply
October 09, 2023
Blog
The cloud, artificial intelligence (AI), Internet of Things (IoT), autonomous vehicles, smart video, and other advanced applications are driving a level of data growth the world has never seen. According to analyst firm IDC, the amount of new data created, captured, replicated and consumed each year is expected to double by 2026 – eventually nearing the 200 zettabyte barrier. Yes, that’s zettabyte with a ‘z.’
Optimized for mass storage, hard disk drives (HDDs) will continue to be the foundational devices underpinning the majority of data center infrastructure. As global data creation continues to climb, organizations are going to have to rely on these cost-efficient storage solutions to keep total cost of ownership (TCO) in check, while ensuring they are meeting their unique and diverse data storage needs.
Emerging Applications and the Explosive Growth of Data
New data-driven applications and use cases are fueling this explosive growth. AI and machine learning (ML) are particularly data intensive, relying on massive datasets that need to be collected and preprocessed before fed into algorithms. While the long-term storage impact and growth implications are unknown. AI, ML, and other big data initiatives collect and process data in widely different ways — necessitating various data storage solutions to meet these diverse needs. SSD storage cannot meet these challenges on its own, so a tiered storage approach with HDDs is essential.
Hyperscale data center customers often use tiered storage architectures where frequently accessed or performance-critical “hot” data is stored on SSDs for fast retrieval while less frequently accessed “warm” or “cold” data resides on more cost-efficient HDD media. While described simplistically, this tiered approach helps optimize the cost of storing and accessing data and can be fine-tuned based on the capacity, performance, availability, and recovery needs of the system or application data.
The Right Storage for the Right Use Case
SSDs and HDDs each have advantages and disadvantages for certain applications.
First and foremost, there is a significant cost difference between the two. According to Western Digital’s analysis, enterprise-class HDDs can be 8 times less expensive per byte than enterprise-class SSD solutions. This pricing difference is expected to persist over time, as increasing NAND production is highly capital intensive and will only be pursued when profitable to do so.
Second, there is a significant performance difference between the two. SSDs can have several times higher sequential performance than HDDs, and orders of magnitude higher random performance. For the most performance-intensive workloads, SSDs are necessary.
Third, an enterprise must consider power consumption. There is a myth that SSDs consume less power than HDDs. This is true in many client and consumer applications but may not be true in enterprise applications. Enterprise-class SSDs during heavy use consume more power than enterprise-class HDDs, while enterprise-class SSDs consume power at idle similar to, or only slightly less than, enterprise-class HDDs. Given that few customers are paying a premium for SSDs to let them sit idle, an SSD deployment at scale may consume significantly more power than an HDD deployment at scale.
Deploying SSDs is more advantageous for use cases demanding high performance and for those customers willing to invest in it. On the other hand, HDDs are more beneficial for use cases with lower performance requirements where cost-effectiveness is favored due to scale.
The Rich Future For HDDs
Enterprises look beyond simply cost and performance when making buying decisions. They look at TCO, and here the case for HDD storage is strong. There are many factors that go into optimizing TCO: CapEx (equipment, floorspace, etc.) and OpEx (power, cooling, maintenance, etc.). HDDs play a key role in all of this and are the most economical media to store massive amounts of data online and at scale.
High-capacity HDDs occupy an ideal space relative to SSDs for large-scale storage. Assuming the fixed costs are similar between the two, such as power consumption, cooling, and data center infrastructure, the lower per-byte cost advantage of HDD storage is clear and too large to be overcome by savings elsewhere.
According to IDC’s Global StorageSphere 2023-2027 Forecast published in May 2023, approximately 80% of cloud data center storage today, on an exabyte basis, is based on HDDs and it is not predicted to decline significantly in the foreseeable future.
HDDs have a strong (and long) track record, and new innovations in HDD technology are driving higher capacities and performance while lowering TCO. Innovations such as energy-assisted magnetic recording (ePMR), shingled magnetic recording (SMR), flash-enabled HDDs and HelioSeal® technology are driving up storage density while driving down power.
Together, these innovations are making HDDs ideal for today’s cloud-based workloads and hyperscale build outs. The continuous growth of data and emerging applications are not slowing down. HDDs provide organizations with a cost-effective option in their data centers so they can meet this demand. HDDs compliment SSD deployments — helping data center architects optimize their workload infrastructure for speed, capacity and cost-effectiveness.
This is why HDD will remain the de facto standard in mass storage in the data center.
