Artificial intelligence (AI) is currently having a profound impact on the data center industry, and this impact can be attributed to OpenAI's launch of ChatGPT in late 2022, which quickly gained popularity due to its superior ability to provide complex and human-like responses to queries. As a result, generative AI, a subset of AI technologies, has become the focus of industry events, earnings reports, and discussions in the business ecosystem in the first half of 2023. The excitement is justified, as generative AI has generated dozens of discussions. Billions of dollars of investment are expected to continue to increase data center capital expenditures to more than $500 billion by 2027. However, due to the significant expansion of the computing power required to train and deploy large language models (LLMs) that support generative AI applications, changes to the architecture of the data center are required.
While the hardware required to support such AI applications is new to many, a portion of the data center industry has been deploying this type of infrastructure for years. This field is often referred to as the high-performance computing (HPC) or supercomputing industry. Historically, this segment has been primarily supported by ** and higher education institutions to deploy some of the world's most sophisticated and sophisticated computer systems.
What generative AI is doing is expanding AI applications and the infrastructure that supports them to a broader market of enterprises and service providers. Learning from the HPC industry gives us an idea of what infrastructure might look like.
To summarize the impact shown in Figure 1, AI workloads will require more computing power and higher network speeds. This will result in higher rack power density, which has a significant impact on the data center physical infrastructure (DCPI). For power infrastructure (also known as grey spaces), architectural changes are expected to be limited. AI workloads should increase the need for backup power (UPS) and IT rack (cabinet PDUs and busways) power distribution, but will not require any major technical changes. Where AI infrastructure will have a transformative impact on DCPI lies in the empty space of the data center.
First, higher power rackmount PDUs are required due to the high power consumption of AI IT hardware. At these power ratings, the costs associated with potential failures or inefficiencies can be high. This is expected to drive end-user adoption of smart rack PDUs with the ability to remotely monitor and manage power consumption and environmental factors. These rack PDUs cost many orders of magnitude more than basic rack PDUs, and end users are unable to monitor or manage their rack power distribution.
Even more transformative for data center architectures is the need for liquid cooling to manage the higher heat loads generated by next-generation CPUs and GPUs running AI workloads. The increasing adoption of liquid cooling, including direct liquid cooling and immersion cooling, in the broader data center industry is expected to accelerate with the deployment of AI infrastructure. However, given the long history of liquid-cooled runways, dell'oro expects the impact of generative AI on liquid cooling to be limited in the short term. It is still possible to deploy current-generation IT infrastructure with air-cooled technology, but at the expense of hardware utilization and efficiency.
To address this challenge, some end-users are retrofitting their existing facilities with closed-loop air-assisted liquid cooling systems. This infrastructure can be a form of backdoor heat exchanger (RDHX) or direct liquid cooling, which utilizes liquids to capture the heat generated inside the rack or server and drain it at the rear of the rack or server, directing it into the hot aisle. This design allows data center operators to take advantage of some of the benefits of liquid cooling without having to invest heavily in redesigning the facility. However, in order to achieve the desired efficiency of AI hardware at scale, purpose-built liquid cooling facilities are required. Current interest in liquid cooling is expected to begin to manifest itself in deployments in 2025, with liquid cooling revenues expected to approach $2 billion by 2027.
Plans to incorporate AI workloads into future data center construction have been realized. This is the main reason why dell'oro has raised its 5-year outlook for the data center physical infrastructure market, which is currently expected to grow revenue at a CAGR of 10% through 2027. However, while AI workloads are expected to bring significant market growth to the data center industry, there are some notable factors that may slow down this growth. The pandemic has accelerated the pace of digitalization and set off a wave of new data center construction. However, as demand materializes, the ** chain struggles to keep up, resulting in a delivery time of more than a year for the physical infrastructure of the data center at its peak. Now, with the easing of chain constraints, DCPI vendors are addressing the backlog and are starting to reduce lead times.
However, the demand for AI workloads is creating another wave of growth in the data center industry. This double growth has led to a discrepancy between the data center industry's growing energy demand and the speed at which utilities can deliver power to where it is needed. As a result, this has led data center service providers to explore the "bring your own power" model as a potential solution. While the viability of the model is still being determined, data center providers are eager for an innovative approach to support their long-term growth strategies, and the proliferation of AI workloads is a core driver.
As the need for more DCPI is balanced with the available power, one thing is clear: AI is ushering in a new era of DCPI. In this day and age, DCPI will not only play a key role in facilitating data center growth, but will also define performance, cost, and help achieve sustainable development. This is very different from the historical role that DCPI has played, especially when compared to the industry nearly a decade ago, when DCPI was an almost afterthought.
With the rapid arrival of the AI growth wave, it is critical to meet DCPI requirements in an AI strategy. Failure to do so can result in AI IT hardware going nowhere.
Reference: AI is ushering in a new era for data center physical infrastructure - Lucas Beran joined dell'oro group