René Raaijmakers
31 August

Gigabyte Technology has joined forces with Asperitas to offer immersion cooling solutions that deliver ultimate sustainable performance for data centers. The collaboration between the rising star in high-end servers and the Dutch cooling technology specialists promises to drastically reduce energy consumption and improve power usage effectiveness in data centers.

Computing power and data storage capacity are still its number one priority, but it’s remarkable how Taiwan’s Gigabyte Technology puts power conservation and energy use at the center of its product pitches these days. Not for nothing. Immersion cooling can radically reduce the operating costs of its servers and bring considerable environmental benefits to its customers.

Research shows that immersion liquid cooling is the most promising technology from a performance perspective. At the same time, it’s clean and environmentally friendly: the heat transfer from the compute to the cooling liquid uses natural convection without consuming any extra energy – a unique feature of the Asperitas tank.

“Market demand is what drives our products and performance still plays the most important role,” says Liam Quinn, a marketing specialist at Gigabyte. However, in recent years, his company has seen that developers of cooling systems attracted the attention of cloud providers with technology that highly increased the power effectiveness of server clusters.

Nowadays, customers often approach Gigabyte directly and they specifically ask them to add immersion to their proposal. “Customers are starting to approach us about immersion cooling at events and so forth,” describes Quinn. “That helps to get more traction.” That’s in short why Gigabyte chooses to partner with Asperitas and offer an integrated cooling solution to its customers.

Combined expertise

Asperitas’ strong presence in Europe was a solid argument for Gigabyte to partner with the Dutch data center cooling experts, with the Taiwanese hardware company identifying the EU as an important market. Quinn: “Our ambition is obviously to grow. We want our name out there in the immersion cooling landscape. When somebody sees the Asperitas product, we want them to also see Gigabyte.”

However, Quinn doesn’t see the EU as the only front-runner in more sustainable data centers. “We see the same interest in Asia and the US. In Europe, the Asperitas brand name is strong and that helps to build confidence in deploying immersion solutions globally.”

Asperitas started to work with Gigabyte in 2019 for a project in Europe. The partners worked for over a year on a server optimization project. “This is in fact a certification process where we look into the feasibility and simulate the server performance in immersion until we get to a final design,” says Maikel Bouricius, Asperitas’ chief commercial officer. During this process, an air-cooled server was prepared for immersion in liquid with minimal changes. Bouricius: “The liquid gives great benefits for cooling performance and for the life span of the product. In effect, the fluid is a more stable environment.”

To immerse a compute platform, simple development work needs to be done to check for chemical compatibility between the designed-for-air components and the dielectric fluid. “This work is done with our experience in immersing hundreds of servers in the liquid over the years,” says Bouricius. “We’re also working with partners like Shell who have a great depth of knowledge on this.”

Asperitas additionally looks into the heat sources and assesses what they need to focus on in an immersion-cooled environment. “We make calculations using computational fluid dynamics to assess whether the boards will be cooled sufficiently,” Bouricius explains. “If necessary, we propose changes to either the composition or the heat sinks to optimize for immersion.”

“Asperitas and Gigabyte’s combined expertise results in a winning formula,” asserts Andy Young, Asperitas’ chief technology officer. “We’ve been sharing our knowledge and have been working together on projects for specific customers. That translated to a general release: Gigabyte is now launching immersion-ready servers equipped to use with our solution.”

Asperitas Gigabyte 21 inch front closed lights dimmed
Credit: Asperitas

Larger deployments

Dell’Oro Group forecasts that immersion cooling will grow at the fastest rate relative to other data center thermal management technologies in 2022. Lucas Beran, a principal analyst at Dell’Oro Group, expects that will continue through 2025, driven by sustainability initiatives from cloud service providers and large enterprises. “With semiconductor component and server OEM validation for immersion cooling growing to directly respond to market demand for IT that suits various workloads, these efforts will further drive the scale up to larger deployments.”

The need for computing power places increasing demands on energy management and savings. “Single GPUs on our PCI cards are hitting a power consumption of 300 watts plus,” says Quinn. “CPUs are going the same direction. So we’re dealing with a lot more heat than usual.” Bouricius adds: “With that in mind, there are certain variables in a system to optimize: the immersion cooling tank, the fluid and the server. This is where our partnership ecosystem comes in. Asperitas and Shell are closely partnered on the cooling fluid products and the partnership with Gigabyte is focused on the server readiness. Partners working together in synergy turns into value for our users.”

As well as the power consumption per chip, server systems deal with an increasing component density. “Servers with two AMD Epyc processors and eight GPUs are now common. Those are demanding at least 3,000 watts. That’s a lot of heat that has to be removed,” says Quinn. “But in new server generations, we’re going to move to even greater heat dissipation. The current system configurations will simply not be sustainable in the future. If customers want this sort of peak performance in their large data centers, then they have to turn to more efficient means, like using immersion cooling technology.”

Core technology

Quinn compares the partnership with Asperitas to the way automotive companies work with their suppliers. Car manufacturers like Volkswagen and Mercedes concentrate on core technology like the engine and software. Parts like body, sunroof, doors and exhaust pipes are outsourced in close collaboration. Suppliers like Bosch and Volvo don’t simply produce to order, they have their own development roadmaps for car modules and subsystems. The only difference is that data centers often separately choose their compute platform and cooling systems.

For Gigabyte, a partnership with a provider that can concentrate on immersion cooling technology development and production makes a lot of sense. With a mind-boggling amount of computer platforms and other hardware like power supplies, it has a lot of product lines to develop and maintain. In the field of servers alone, it has seven series of compute platforms. In total, there are over 100 products to choose from with the current gen products.

For instance, Gigabyte’s high-performance G series of GPU-centric servers comprise more than ten families of configurations. Customers can choose from a processor menu of AMD’s Epyc and Ampere’s Altra to Intel’s Xeon. “Our G series can support heavy graphics and compute workloads,” says Quinn. “We have a lot of different models, segmented for what the market is asking for. At the same time, we do this because it makes us more price competitive. Coupling with immersion cooling technology gives us a reason to offer another variant.”

Demand creation

Motherboards bring in almost a third of Gigabyte’s revenue stream. “Motherboard manufacturing is the base of the company,” comments Quinn. “At the server division, we rely on the strength of our motherboards. This means that we can price things well by making small adjustments. That’s a major part of the demand creation.”

In March 2021, Gigabyte decided to spin off its server business as a wholly owned subsidiary under the name of Giga Computing Technology. Currently, servers account for 20 percent of Gigabyte’s total revenue stream – 4.4 billion dollars in 2021. Graphics cards bring in the biggest market value at 40 percent. Another 10 percent is income from its laptops and other PC components.

Gigabyte uses a system architecture in which processors, motherboards and cooling systems are integrated with ease. Quinn: “Modularity gives us plenty of room to differentiate. We don’t just use X86 chips from Intel and AMD; in our product lines, we also use Arm-based CPUs that are currently produced by Ampere Computing. Modification for immersion is one of the options. All these partnerships, with suppliers of microprocessors, graphics engines and immersion providers, give us the leverage to be fast to market.”

Optimal temperatures

Data centers utilize a number of methods to keep components at optimal temperatures. Traditionally, they opt to cool their IT by moving air in combination with heat exchangers and cooled water. However, air cooling is simply not sufficient to cool down 300 watts plus high-performance CPUs, FPGAs and GPUs.

With the introduction of immersion cooling, the air-cooled servers are optimized to make them suitable for contact with fluid. “We add brackets so we can install the systems vertically,” explains Quinn. “In this way, all I/O and power connections are plugged into the side facing up. Obviously, there’s no need for fans because there’s a passive circulation of the fluid in the tank. At the same time, the baseboard management controller can do without communication with the fan and its warning sensors. So it basically comes down to adjustments to make sure it’s compatible with the design of immersion cooling tanks.”

Specifically, high-end complex systems are moving to immersion. Quinn: “I expect that a respectable percentage of our server revenue will come from the immersion portfolio in five years’ time. You have to realize that when we’re talking immersion, we’re talking large scale. Customers aren’t buying immersion for two servers. It typically supports 24U tanks, but they can be easily scaled. It’s definitely targeted at the bigger data centers, like those in the cloud industry.”

The hunger for power in the industry is really pushing the limits of air cooling. “We’re now seeing servers emerging in the market where you need extra energy-consuming components like chillers to reduce the temperature,” says Andy Young. “That’s why you see chip OEMs offering SKUs, stock-keeping units, for their processors, which are specifically designed for liquid cooling. It’s a natural development. We’re excited to see equipment OEMs make design choices for heat sinks and materials to prepare their servers for immersion right from the start.”

This article was written in close collaboration with Asperitas.