Direct-to-Chip Cooling: The Future Of The Data Center

Traditionally used to cool large supercomputers for decades, direct-to-chip cooling is becoming a more mainstream reality in the data center due to the introduction of AI. The introduction of thousands of GPUs into data centers to run parallelized AI workloads requires much more cooling and refrigeration, and air cooling is simply not enough.

AI workloads consume more power than traditional data center workloads, driving up server density. While traditional data center racks (no accelerated computing) used to have densities of 5-10kW, racks used in AI workloads can have densities of 60-100kW, up to 20 times more. With denser servers there is more heat per rack, and traditional air cooling equipment like raised floors or computer room air handlers (CRAHs) is not enough.

In an average data center air cooling systems are normally sufficient: they are technically simpler, easier to install and require less capital expenditures. But they are not enough for AI workloads, which involve massive computations from GPUs, also supported by CPUs and high-bandwidth-memory, among others.

In direct-to-chip cooling, also known as cold plate cooling, water flows through a conductive plate where the electronic components are placed, dissipating heat at a much faster pace than air cooling. Some data center IT equipment can be highly sensitive and cost thousands of dollars, so it is paramount to maintain it at a stable temperature to maximize its performance and avoid any issues. Industry leaders like Meta, Google or Equinix are already adopting this approach in their data centers. Direct-to-chip liquid cooling can also be complemented with air cooling for less sensitive components that may not come into direct contact with the cold plate.

John Niemann, Vertiv’s Senior Vice President, mentioned in a recent earnings call “direct-to-chip cooling is where we are seeing scale being deployed”, implying this will be the default solution for cooling AI data centers. Vertiv, which manufactures equipment and cooling systems for data centers, has seen its sales increase to $5 billion in the first nine months of 2023, a 24% year-over-year increase. This is, in part explained, by higher demand for direct to chip cooling systems.