Abstract Direct Liquid Cooling (DLC) has emerged as a promising technology for thermal management of high-performance computing servers, enabling efficient heat dissipation and reliable operation. Thermal performance is governed by several factors, including the coolant physical properties and flow parameters such as coolant inlet temperature and flow rate. The design and development of the coolant distribution manifold to the Information Technology Equipment (ITE) can significantly impact the overall performance of the computing system. This paper aims to investigate the hydraulic characterization and design validation of a rack-level coolant distribution manifold or rack manifold. To achieve this goal, a custom-built high power-density liquid-cooled ITE rack was assembled, and various cooling loops were plugged into the rack manifold to validate its thermal performance. The rack manifold is responsible for distributing the coolant to each of these cooling loops, which is pumped by a CDU (Coolant Distribution Unit). In this study, pressure drop characteristics of the rack manifold were obtained for flow rates that effectively dissipate the heat loads from the ITE. The pressure drop is a critical parameter in the design of the coolant distribution manifold since it influences the flow rate and ultimately the thermal performance of the system. By measuring the pressure drop at various flow rates, the researchers can accurately determine the optimum flow rate for efficient heat dissipation. Furthermore, 1D flow network and CFD models of the rack-level coolant loop, including the rack manifold, were developed, and validated against experimental test data. The validated models provide a useful tool for the design of facility-level modeling of a liquid-cooled data center. The CFD models enable the researchers to simulate the fluid flow and heat transfer within the cooling system accurately. These models can help to design the coolant distribution manifold at facility level. The results of this study demonstrate the importance of the design and development of the coolant distribution manifold in the thermal performance of a liquid-cooled data center. The study also highlights the usefulness of 1D flow network and CFD models for designing and validating liquid-cooled data center cooling systems. In conclusion, the hydraulic characterization and design validation of a rack-level coolant distribution manifold is critical in achieving efficient thermal management of high-performance computing servers. This study presents a comprehensive approach for hydraulic characterization of the coolant distribution manifold, which can significantly impact the overall thermal performance and reliability of the system. The validated models also provide a useful tool for the design of facility-level modeling of a liquid-cooled data center.
more »
« less
A Novel Design of Rack Mount Server Thermal Simulator: Design, Assembly, and Experimental Verification
Abstract The practice of commissioning data centers (DCs) is necessary to confirm the compliance of the cooling system to the information technology equipment (ITE) load (design capacity). In a typical DC, there are different types of ITE, each having its physical characteristics. Considering these geometrical and internal differences among ITE, it is infeasible to use the actual ITE as a self-simulator. Hence, a separate device called load bank is employed for that purpose. Load banks create a dummy thermal load to analyze, test, and stress the cooling infrastructure. Available commercial load banks do not accurately replicate a server's airflow patterns and transient heat signatures which are governed by thermal inertia, energy dissipation, flow resistance, and fan system behavior. In this study, a novel prototype of the server called server simulator was designed and built with different components to be used as a server mockup. The server simulator accurately captured air resistance, heat dissipation, and the functionality of actual server behavior. Experimental data showed up to 93% improvement in ITE passive and active flow curves using the designed server simulator compared to the commercial load bank. Furthermore, the experimental results demonstrated a below 5% discrepancy on the critical back pressure and free delivery point between the actual ITE and the designed server simulator. In addition, experimental data indicated that the developed server simulator improved the actual ITE thermal mass by 27% compared to the commercial load bank.
more »
« less
- Award ID(s):
- 1738793
- PAR ID:
- 10338736
- Date Published:
- Journal Name:
- Journal of Electronic Packaging
- Volume:
- 144
- Issue:
- 4
- ISSN:
- 1043-7398
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The role of data centers in modern life has expanded rapidly over the past decades. In addition, this expansion has resulted in a significant increase in the share of data centers in total energy consumption of the world. Thus, reliability and energy efficiency have become a common concern in data centers. Information technology equipment (ITE) and cooling infrastructure are the largest power consumer in data centers. The cooling power in a data center depends on the amount of heat dissipated by ITE. Therefore, the thermal design of the ITE impacts not only the ITE power but also affects the infrastructure power and has a significant role in the overall efficiency of data centers. This paper studies the impact of fans location and airflow balancing on the thermal performance and power of a server. A detailed computational fluid dynamic (CFD) model of the server is built, calibrated and validated using experimental test results. Next, impacts of moving fans to the rear side of the chassis on the flow rate and temperature of components are investigated. Special attention is given to controlling airflow through power supplies.more » « less
-
Abstract This paper proposes a computational fluid dynamics (CFD) simulation methodology for the multi-design variable optimization of heat sinks for natural convection single-phase immersion cooling of high power-density Data Center server electronics. Immersion cooling provides the capability to cool higher power-densities than air cooling. Due to this, retrofitting Data Center servers initially designed for air-cooling for immersion cooling is of interest. A common area of improvement is in optimizing the air-cooled component heat sinks for the fluid and thermal properties of liquid cooling dielectric fluids. Current heat sink optimization methodologies for immersion cooling demonstrated within the literature rely on a server-level optimization approach. This paper proposes a server-agnostic approach to immersion cooling heat sink optimization by developing a heat sink-level CFD to generate a dataset of optimized heat sinks for a range of variable input parameters: inlet fluid temperature, power dissipation, fin thickness, and number of fins. The objective function of optimization is minimizing heat sink thermal resistance. This research demonstrates an effective modeling and optimization approach for heat sinks. The optimized heat sink designs exhibit improved cooling performance and reduced pressure drop compared to traditional heat sink designs. This study also shows the importance of considering multiple design variables in the heat sink optimization process and extends immersion heat sink optimization beyond server-dependent solutions. The proposed approach can also be extended to other cooling techniques and applications, where optimizing the design variables of heat sinks can improve cooling performance and reduce energy consumption.more » « less
-
Abstract This study aims to improve the combined energy efficiency of data center cooling systems and heating/cooling systems in surrounding premises by implementing a modular cooling approach on a 42 U IT rack. The cooling solution uses a close-coupled technique where the servers are air-cooled, and the air in turn is cooled within the rack enclosure using an air-to-refrigerant heat exchanger. The refrigerant passively circulates in a loop as a thermosyphon, making the system self-sustaining during startup and shutdown, self-regulating under varying heat loads, and virtually maintenance-free by eliminating mechanical parts (other than the cabinet fans). A heat load range of 2 kW–7.5 kW is tested on a prototype system. Experimental results reveal stable thermosyphon operation using R1233zd(E) as the working fluid, a maximum evaporator pressure drop of 21.5 kPa at the highest heat load and a minimum thermosyphon resistance of 6.8 mK/W at a heat load of 5.7 kW. The air temperature profile across the load banks (server simulators) and evaporator follow the same profiles with varying heat loads. Heat losses from the cabinet due to natural convection and radiation are of the order of several Watts for heat loads below 4 kW and rise sharply to 1 kW at the highest heat load tested. The system time constant is determined to be 25 min. The heat recovery process can be financially and environmentally beneficial depending on the downstream application.more » « less
-
null (Ed.)Abstract In today’s world, most data centers have multiple racks with numerous servers in each of them. The high amount of heat dissipation has become the largest server-level cooling problem for the data centers. The higher dissipation required, the higher is the total energy required to run the data center. Although still the most widely used cooling methodology, air cooling has reached its cooling capabilities especially for High-Performance Computing data centers. Liquid-cooled servers have several advantages over their air-cooled counterparts, primarily of which are high thermal mass, lower maintenance. Nano-fluids have been used in the past for improving the thermal efficiency of traditional dielectric coolants in the power electronics and automotive industry. Nanofluids have shown great promise in improving the convective heat transfer properties of the coolants due to a proven increase in thermal conductivity and specific heat capacity. The present research investigates the thermal enhancement of the performance of de-ionized water-based dielectric coolant with Copper nanoparticles for a higher heat transfer from the server cold plates. Detailed 3-D modeling of a commercial cold plate is completed and the CFD analysis is done in a commercially available CFD code ANSYS CFX. The obtained results compare the improvement in heat transfer due to improvement in coolant properties with data available in the literature.more » « less