Leading cloud companies are significantly upping their investments in AI infrastructure, fueling a rising demand for AI development and deployment. This move has exacerbated existing memory shortages. Market analyst TrendForce projects that global cloud heavyweights such as Google, Amazon, Meta, and Microsoft will channel over $710 billion into capital expenditures by 2026, marking a 61% increase over the previous year.

These funds will largely finance datacenters equipped with high-performance servers, many featuring Nvidia or AMD GPU accelerators, surpassing Ireland’s annual GDP. While these servers often employ GPUs, some companies, like Google, are increasingly turning to ASICs, which offer improved performance and energy efficiency for specific AI workloads.

Google leads the embrace of ASICs, with its Tensor Processing Units anticipated to feature in 78% of Google’s AI servers. In contrast, Amazon and Meta heavily rely on GPU-based systems, and Microsoft’s acquisitions remain focused on Nvidia’s rack-scale systems. The surge in demand is straining memory supply, prompting chipmakers like SK Hynix to explore new high-bandwidth memory solutions.

Innovations like High-bandwidth Flash (HBF) are emerging as potential complements to HBM, offering greater capacity at a similar cost, albeit at lower speeds. These developments are essential as businesses aim to support larger AI model workloads seamlessly, optimizing for both cost efficiency and scalability.