An ASHRAE podcast recently delved into a critical evolution within data centres: the increasing necessity of liquid cooling. Host Justin Seter guided a panel of industry experts – David Quirk, Dustin Demetriou and Tom Davidson – through the intricacies of this technology, driven by the insatiable demands of artificial intelligence (AI) and high-performance Graphics Processing Unit (GPU) applications. This is Part 2 of a nine-part series.
This escalating power density naturally led to a discussion about the dramatic increase in heat loads within typical server racks. Image by Vecstock/Freepik.com
Demetriou elaborated on the two primary categories: direct-to-chip cooling, where a cold plate replaces the traditional air-cooled heat sink on the processor (CPU or GPU), utilising a fluid to extract heat which is then rejected via a technology fluid system loop; and immersion cooling, where electronic components, either partially or fully, are submerged in a dielectric coolant. He further explained the nuances within each category, including single-phase systems where the fluid remains liquid and two-phase systems where the fluid boils to enhance heat transfer before being recondensed. Demetriou also briefly touched upon close-coupled cooling solutions like rear-door heat exchangers and in-row cooling, clarifying that while bringing cooling closer to the IT equipment, these still rely on air as the final heat rejection medium and are distinct from true liquid cooling as defined by ASHRAE.
Quirk then addressed the crucial question of why direct-to-chip liquid cooling has become such a focal point in the industry today. His answer: artificial intelligence. He described software as the ‘occupant’ driving hardware design, which in turn dictates the necessary infrastructure. The intense computational and bandwidth demands of AI, particularly large language models and inference applications, necessitate a significant densification of hardware, requiring the deployment of thousands, even hundreds of thousands, of high-power GPUs within a relatively small data centre footprint. This unprecedented density, with individual chip power soaring from around 300 watts to well over 1000-1500 watts in GPU configurations, has simply outstripped the cooling capabilities of traditional air-based systems, making direct-to-chip liquid cooling the indispensable solution.
Building upon the foundational understanding of liquid cooling’s emergence driven by AI and GPU demands, the ASHRAE podcast segment further explored the escalating power densities within data centres and ASHRAE’s proactive response in providing industry guidance. Dustin Demetriou expanded on the increasing energy demands, noting that while GPUs initially spearheaded the shift towards liquid cooling, the trend of rising power consumption is now evident across all server components – CPUs, memory, and interconnects. He recalled a similar, albeit less intense, period in the early 2000s with multi-core processors, suggesting that the current surge is far more pervasive due to the stagnation of Moore’s Law, where performance gains no longer come without increased power draw. Even server configurations without discrete GPUs, packing numerous high-core CPUs and substantial memory, are increasingly challenging the limits of air cooling, particularly when considering preheated intake air and elevated exhaust temperatures.
This escalating power density naturally led to a discussion about the dramatic increase in heat loads within typical server racks. Demetriou provided a stark illustration: the average cloud data centre today operates at a rack density of 10 to 20 kilowatts – the equivalent heat output of a residential fireplace at full blast, concentrated within a confined space. However, with the adoption of liquid-to-chip cooling, these densities are skyrocketing to 40-60 kilowatts per rack in mass deployments, with conversations and even manufacturer listings pointing towards future densities of 120, 130, 300, 400, and even an astounding 500 kilowatts per rack and beyond.
This represents a tenfold to fiftyfold increase in heat generation within the same physical footprint, raising critical questions about design and operational paradigms. Seter highlighted the immense challenge this poses and the crucial role of ASHRAE in providing industry guidance to navigate this unprecedented thermal landscape.
