Owing to the dramatic increase in IT power density and energy consumption, the data center (DC) sector has
started adopting thermally- and energy-efficient liquid cooling methods. This study examines a single-phase
direct-to-chip liquid cooling approach for three high-heat-density racks, utilizing two liquid-to-air (L2A)
cooled coolant distribution units (CDUs) and a combined total heat load of 128 kW. An experimental setup was
developed to test different types of CDUs, cooling loops, and thermal testing vehicles (TTVs) for different
operating conditions. IR images and the collected data were used to investigate the effect of air recirculation
between cold and hot aisle containments on the CDU’s performance and stability of supply air temperature
(SAT). Three different types of cooling loops (X, Y, and Z) were characterized thermally and hydraulically.
Results show that Type Y has the lowest cold plate thermal resistance and pressure drop, among others. In a later test that included a single rack at a heat load of 53 kW and a single CDU, the heat capture ratio for fluid was found to be 94%. Experiments show that using blanking panels on the back of the racks limits hot air recirculation and maintains a steady SAT in the cold aisle. Finally, the CDU performance was evaluated at a high heat load for the three racks at 128 kW, and the average cooling capacity of the units is 58.6 kW, and the effectiveness values for CDU 1 and CDU 2 are 0.83 and 0.82, respectively.
more »
« less
This content will become publicly available on October 24, 2024
L2A CDUs Performance and Considerations for Server Rooms Upgrade With Conventional Air Conditioning
As the online frameworks and services are growing rapidly with the evolution of web-based Artificial Intelligence (AI) applications, server rooms are upgrading in computational capacity and size to keep up with these demands. Enterprise companies with their limited capacity server rooms struggle to keep up with these increasing computational demands. Hence, some of them end up outsourcing their servers to co-located facilities (Co-Lo) and the others choose to upgrade their existing server rooms. Correspondingly, the thermal load associated with such upgrades is typically tremendous. Approximately around 40% of the power consumed by datacentres is dissipated as heat. Conventional HVAC systems fail to satisfy the requirements of such server capacities. Not only do they struggle to fulfil the cooling load, but their maldistribution of cool air into the server room forms a major cause for hotspots formation. To tackle this issue, Liquid-to-Air (L2A) Coolant Distribution Units (CDUs) are being used as a liquid-based cooling solution for rack-level cooling. This type of CDUs provide efficient cooling for servers through liquid coolant that is distributed into cooling loops mounted on top of each server board. The generated heat is curried away using this liquid coolant back to the CDU, which then dissipates it into the surrounding air using dedicated pumps, fans, and heat exchanger, hence the name Liquid-to-Air.
In the present work, one of the most popular liquid cooling strategies is explored based on various scenarios. the performance of a 24-kW liquid to Air (L2A) CDU is judged based on cooling effect, stability, and reliability. The study is curried out experimentally, in which a test rack with three thermal test vehicles (TTVs) are used to investigate various operation scenarios. Both liquid coolant and air sides of this experimental setup are equipped with the required instrumentations to monitor and analyse the tests. All test cases were taken in a room with limited air conditioning to resemble the environment of upgraded server rooms with conventional AC systems. Moreover, the impact of using such high-power density cooling unit on the server room environment with restricted HVAC system is also brought to light. Environmental and human comfort parameters such as noise, air velocity, and ambient temperature are measured under various operation conditions and benchmarked against their ranges for human comfort as listed in ASHREE standards. At the end of this research, recommendations for best practice are provided along with areas of enhancement for the selected system.
more » « less- Award ID(s):
- 2209776
- NSF-PAR ID:
- 10529807
- Publisher / Repository:
- American Society of Mechanical Engineers
- Date Published:
- ISBN:
- 978-0-7918-8751-6
- Subject(s) / Keyword(s):
- Electronic Thermal Management Data Center Liquid-to-Air CDU Rack Level Cooling
- Format(s):
- Medium: X
- Location:
- San Diego, California, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract Direct Liquid Cooling (DLC) has emerged as a promising technology for thermal management of high-performance computing servers, enabling efficient heat dissipation and reliable operation. Thermal performance is governed by several factors, including the coolant physical properties and flow parameters such as coolant inlet temperature and flow rate. The design and development of the coolant distribution manifold to the Information Technology Equipment (ITE) can significantly impact the overall performance of the computing system. This paper aims to investigate the hydraulic characterization and design validation of a rack-level coolant distribution manifold or rack manifold. To achieve this goal, a custom-built high power-density liquid-cooled ITE rack was assembled, and various cooling loops were plugged into the rack manifold to validate its thermal performance. The rack manifold is responsible for distributing the coolant to each of these cooling loops, which is pumped by a CDU (Coolant Distribution Unit). In this study, pressure drop characteristics of the rack manifold were obtained for flow rates that effectively dissipate the heat loads from the ITE. The pressure drop is a critical parameter in the design of the coolant distribution manifold since it influences the flow rate and ultimately the thermal performance of the system. By measuring the pressure drop at various flow rates, the researchers can accurately determine the optimum flow rate for efficient heat dissipation. Furthermore, 1D flow network and CFD models of the rack-level coolant loop, including the rack manifold, were developed, and validated against experimental test data. The validated models provide a useful tool for the design of facility-level modeling of a liquid-cooled data center. The CFD models enable the researchers to simulate the fluid flow and heat transfer within the cooling system accurately. These models can help to design the coolant distribution manifold at facility level. The results of this study demonstrate the importance of the design and development of the coolant distribution manifold in the thermal performance of a liquid-cooled data center. The study also highlights the usefulness of 1D flow network and CFD models for designing and validating liquid-cooled data center cooling systems. In conclusion, the hydraulic characterization and design validation of a rack-level coolant distribution manifold is critical in achieving efficient thermal management of high-performance computing servers. This study presents a comprehensive approach for hydraulic characterization of the coolant distribution manifold, which can significantly impact the overall thermal performance and reliability of the system. The validated models also provide a useful tool for the design of facility-level modeling of a liquid-cooled data center.
-
As the demand for faster and more reliable data processing is increasing in our daily lives, the power consumption of electronics and, correspondingly, Data Centers (DCs), also increases. It has been estimated that about 40% of this DCs power consumption is merely consumed by the cooling systems. A responsive and efficient cooling system would not only save energy and space but would also protect electronic devices and help enhance their performance. Although air cooling offers a simple and convenient solution for Electronic Thermal Management (ETM), it lacks the capacity to overcome higher heat flux rates. Liquid cooling techniques, on the other hand, have gained high attention due to their potential in overcoming higher thermal loads generated by small chip sizes. In the present work, one of the most commonly used liquid cooling techniques is investigated based on various conditions. The performance of liquid-to-liquid heat exchange is studied under multi-leveled thermal loads. Coolant Supply Temperature (CST) stability and case temperature uniformity on the Thermal Test Vehicles (TTVs) are the target indicators of the system performance in this study. This study was carried out experimentally using a rack-mount Coolant Distribution Unit (CDU) attached to primary and secondary cooling loops in a multi-server rack. The effect of various selected control settings on the aforementioned indicators is presented. Results show that the most impactful PID parameter when it comes to fluctuation reduction is the integral (reset) coefficient (IC). It is also concluded that fluctuation with amplitudes lower than 1 ᵒC is converged into higher amplitudesmore » « less
-
About 40% of the energy utilized in data centers is used for cooling systems, and this percentage has increased significantly in recent years. Data center server racks receive indirect cooling from computer room air conditioning units (CRACs) or computer room air handler units (CRAHs), and chilled air is sent to the racks through a raised floor plenum to cool the server room. This approach is inefficient because the server room has excessive cooling while the IT equipment has inadequate cooling. The data center industry has begun to use thermally and energy-efficient single-phase liquid cooling solutions as a result of the tremendous increase in IT power density and energy usage. One of the most popular liquid cooling systems will be examined in the current study, which is based on numerous circumstances. Under various secondary coolant conditions, the hydro-thermal performance of different liquid-to-air and liquid-to-liquid coolant distribution units (CDUs) will be assessed experimentally using multi-racks loaded with different numbers of thermal testing vehicles (TTVs), and the system response to any change in the flow will be examined by disconnecting and reconnecting the TTVs cooling loops in the multi-racks through different sequences of transient events. Moreover, a set of rules and guidelines will be established to commission these units.more » « less
-
more » « less
Abstract Demand is growing for the dense and high-performing IT computing capacity to support artificial intelligence, deep learning, machine learning, autonomous cars, the Internet of Things, etc. This led to an unprecedented growth in transistor density for high-end CPUs and GPUs, creating thermal design power (TDP) of even more than 700 watts for some of the NVIDIA existing GPUs. Cooling these high TDP chips with air cooling comes with a cost of the higher form factor of servers and noise produced by server fans close to the permissible limit. Direct-to-chip cold plate-based liquid cooling is highly efficient and becoming more reliable as the advancement in technology is taking place. Several components are used in the liquid-cooled data centers for the deployment of cold plate-based direct-to-chip liquid cooling like cooling loops, rack manifolds, CDUs, row manifolds, quick disconnects, flow control valves, etc. Row manifolds used in liquid cooling are used to distribute secondary coolant to the rack manifolds. Characterizing these row manifolds to understand the pressure drops and flow distribution for better data center design and energy efficiency is important. In this paper, the methodology is developed to characterize the row manifolds. Water-based coolant Propylene glycol 25% was used as the coolant for the experiments and experiments were conducted at 21 °C coolant supply temperature. Two, six-port row manifolds' P-Q curves were generated, and the value of supply pressure and the flowrate were measured at each port. The results obtained from the experiments were validated by a technique called flow network modeling (FNM). FNM technique uses the overall flow and thermal characteristics to represent the behavior of individual components.
