According to Omdia, the server market is going to be all about GPUs, GPUs and more GPUs for the foreseeable future. The market researcher estimates that Nvidia H100 GPU alone weighed more than 900 tons of shipped volume during calendar Q2.
The headline-grabbing figure comes from the company’s latest cloud and data center market update, which notes a shift in datacenter investment priorities that it previously highlighted is accelerating. The report states that demand for servers equipped with eight GPUs for AI processing work has driven up average prices, while also impacting investments in other sectors.
As a result, Omdia once again lowered its annual server shipment forecast for 2023 by another 1 million units to 11.6 million, down 17 percent from last year’s figure. At the same time, however, the average price of a server has risen more than 30 percent quarter-on-quarter and year-on-year as hyperscalers continue to invest in these high-spec boxes.
The report notes that only 300,000 of Nvidia’s H100 GPUs have made it onto server makers’ assembly lines, a 900 ton figure based on each GPU weighing around 3kg with its heatsink.
Each H100 costs an eye-watering $21,000 each, which paradoxically means that Omdia now expects total server market revenue to reach $114 billion by 2023, up 8 percent year-over-year, despite significantly lower unit shipments.
This huge demand for AI servers bristling with GPUs is largely driven by hyperscale companies and cloud providers, according to Omdia, the beneficiaries are original design manufacturers (ODM) server manufacturers – the so-called white box vendors.
The bulk of those eight-GPU servers shipped in the second quarter were built by ZT Systems, a US-based cloud server maker it claims is described as a fast-growing, privately held company. The evidence is based on a claim that as much as 17 percent of Nvidia’s $13.5 billion in second-quarter revenue was driven by one vendor, and Nvidia CEO Jensen Huang demonstrated DGX servers made by ZT Systems during Computex Taipei.
Omdia also believes Meta is likely to account for much of the growth in demand for eight-GPU servers during Q2, based on Nvidia previously saying 22 percent of its revenue in the quarter was driven by a single cloud service provider.
And there is more of this to come. The report predicts rapid adoption of servers configured for AI processing in the second half of this year and the first half of 2024.
However, Omdia forecasts a better-than-seasonal rebound in general-purpose server demand in the current third quarter and next quarter as enterprise demand picks up. This is expected to lead to revenue growth of 13 percent YoY in 3Q23, rising to 29 percent in 4Q23.
Specifically, Omdia expects the continued deployment of eight-GPU servers to drive a 51 percent increase in server market revenue in the first half of 2024, with one million H100 GPUs projected to find their way into systems.
Yet the analyst cautions that rapid investment in AI training capabilities among hyperscalers and cloud providers should not be confused with rapid adoption. AI usage is still low, citing a survey that found only 18 percent of US adults have used ChatGPT. Generative AI also currently makes up a small portion of cloud computing costs for enterprises, as well Register Recently reported.
This large investment in power-hungry GPU systems for AI training also has its drawbacks. A media industry exec recently warned that AI will “burn the world down” unless radical action is taken to implement more sustainable practices.
Speaking at the International Broadcasting Convention (IBC) in Amsterdam this month, AdSignal CEO and founder Tom Dunning said we can’t ask organizations to abandon the business benefits of AI, but it’s clear there is an “urgent need to minimize. Its carbon impact.”
Datacenters are now responsible for 2.5 to 3.7 percent of all carbon dioxide emissions, and AI adoption is projected to grow at a compound annual growth rate (CAGR) of 37 percent between now and 2030, Dunning claims.