Where next for data center cooling?

You can measure it in gallons, you can measure it in swimming pools, you can measure it in oceans but, however you decide to calculate the amount of water being used to cool today’s data centers, the number is staggering. According to some reports, certain areas of the US are using seven billion liters of water per year to cool these lifelines for tech innovation.

One of the key culprits of this water usage is evaporative cooling, in which warm air is drawn through pads that have been soaked in water, evaporating it and lowering the air temperature behind the fan.

While this method is generally more energy efficient than which uses drastically more water than traditional air cooling solutions. The demands on these systems are only deepening as data center real estate gets bought up, rack densities increase, and environmental concerns come further to the forefront. In some newer areas of the market, such as the Middle East, air cooling is a non-starter.

Sanjeev Khanna, the COO at Caddis Cloud Solutions and a former Intel, Microsoft, and Oracle employee sees three main alternatives to traditional evaporative cooling: closed loop systems, direct to chip cooling, and immersion cooling. He says that, especially when it comes to direct-to-chip cooling, there is a perception that this is entirely new technology. It’s something that he disputes.

“I joke about it like I've been schlepping direct-to-chip for 20 years. The good thing is, there are a lot more people. Innovation is going into it because I think there is demand now. It never took off [before.]”

He says that focus on new technology is being pushed by the increasing power demands of data centers worldwide and that the industry is taking notice.

“I was at Nvidia GTC and you couldn't walk ten feet without looking at another cooling solution…There's a physics problem now, and to solve the physics problem, people are going to need liquid. It's not a choice.”

The changing ways we're cooling data centers

Jenny Gerson, DataBank’s senior director of sustainability, says that – like for many companies – the way they’ve approached cooling has shifted as their customers’ needs and wants have evolved. Earlier on, that meant evaporative cooling solutions, but now they are edging towards direct to chip cooling with a default design that uses a closed loop. Microsoft recently announced that it is prioritizing this style of design as well.

“As data centers are continuing to evolve, as there's more high density and higher cooling needs, we're bringing the cooling closer to the chips,” Gerson explains. “So, either at the rack level, at the server level, or at the chips themselves. And so we'll do whatever customers need for their density.”

While adoption and education can prove to be bottlenecks in the short term, Gerson says that the main sticking point in any newer approach to cooling in the sector is power availability and the capability to distribute that power effectively.

For every person who spoke to ITPro for this story, power efficiency was a core consideration. Joe Capes, CEO at Liquid Stack, says that as we reach the outer limits of single phase cooling, liquid cooling is going to have to be the future.

“With these rising chip temperatures and rack densities, the only way to really deal with this is through liquid cooling,” he says.

“Whether it's two phase immersion, two phase direct to chip or hybridization of multiple liquid cooling technology, technologies like combining single phase and two phase systems together. And I think that what we're seeing from companies like Nvidia and others is looking forward and farther down the path of these roadmaps to see how two phase technologies can be adapted.

Pace of cooling innovation

Another question is the pace of innovation. Aaron Lewis, the chief commercial officer at Johnson Controls, says that the staggering speed at which data centers are expanding means an expedited timeline for those bringing products to market.

“When you think about new product introductions in the HVAC space that used to be a three to five year program that you would put in place,” he tells ITPro. “Today, if you were to wait that long, you'll miss three cycles of different iterations of data centers based on new chip requirements that are coming out, or new cooling technologies within the cold chain itself.”

Lewis says that another key shaper of innovation is where data centers are tending to be placed. In a warmer climate, like Arizona or the Middle East, mechanical air cooling solutions are limited in their effectiveness because of the high ambient temperatures. In a cooler climate, he says other forms of cooling can be relied upon.

“We developed data centers in Montreal, where it's very cold most of the year, and you can utilize some type of free cooling system, so you're able to run water through either the dry cooler or a free cooling coil on the chiller itself, so that you can maintain the space without having a mechanical cooling.”

For Capes, the pace of innovation means that the infrastructure and technology just can’t quite meet in the middle yet.

“The adoption and scale up of liquid cooling has really kind of transpired over the last 12 months. And so, as an industry, we're not quite there yet. It's going to take some time before, before data center facility designs and working fluid temperatures catch up with the way that these liquid cooling systems are designed to operate.”

Immersion cooling

Closer to the frontier of new innovation in cooling is immersion. This is where the rack is set into a pool of oils, often petroleum or vegetable. Khanna isn’t sure that the method is ready for primetime. For him, the logistical challenges—including difficulties regarding maintenance—mean that widespread adoption is a ways off yet.

Immersion cooling could be a gamechanger for data centers in extremely hot environments, as racks submerged directly in cooling liquids can disperse heat far more efficiently. Submerged racks are also protected from other environmental factors such as humidity.

But Khanna isn’t sure that the method is ready for primetime. For him, the logistical challenges – including difficulties regarding maintenance – mean that widespread adoption is a ways off yet.

“I think people will figure out a way to get creative there and how we can solve the operational challenges,” he says, adding “personally, I don't think it's ready yet. People who build these, they think that they're ready to go, but I don't think this is ready yet.”

That perspective won’t stop people like Seamus Egan, the general manager of immersion cooling at Modine Manufacturing. He says that immersion cooling allows for an interplay between already existing evaporative cooling techniques and this newer approach, with evaporative cooling being used to reduce the ambient temperature of the data center.

“When we take that entire server and immerse it in a liquid, and these are dielectric fluids, then we've got the ability to capture 100% of the heat…We can now take the heat that's captured inside of that liquid, do a liquid to liquid transfer, and then evacuate that heat on a full liquid path, end to end, out of the actual data center.”

While Eagan is bullish on the potential for this style of cooling, he isn’t naive when it comes to the barriers that will need to be addressed before widespread adoption can be on the horizon.

“Number one is that all customers want to use the least disruptive techniques that align with what they have today. So if 95% of today's centers in the market today are air cooled data centers, then people will tend to gravitate towards those or try to have a solution that's least disruptive, right? The second thing is just experiences and working with them.”

The pace of innovation is creating an environment where the technology to cool data centers is struggling to keep up. Water availability, power availability and distribution, and reducing cost and ecological impact, are all things the industry is grappling with. While the ideal solution isn’t apparent, what is clear is that the modus operandi of the last fifty years or so, air cooling, is no longer fit for purpose. Liquid, it appears, is the future.