An official website of the United States government
In typical data centers, the servers and IT equipment are cooled by air and almost half of total IT power is dedicated to cooling. Hybrid cooling is a combined cooling technology with both air and water, where the main heat generating components are cooled by water or water-based coolants and rest of the components are cooled by air supplied by CRAC or CRAH. Retrofitting the air-cooled servers with cold plates and pumps has the advantage over thermal management of CPUs and other high heat generating components. In a typical 1U server, the CPUs were retrofitted with cold plates and the server tested with raised coolant inlet conditions. The study showed the server can operate with maximum utilization for CPUs, DIMMs, and PCH for inlet coolant temperature from 25–45 °C following the ASHRAE guidelines. The server was also tested for failure scenarios of the pumps and fans with reducing numbers of fans and pumps. To reduce cooling power consumption at the facility level and increase air-side economizer hours, the hybrid cooled server can be operated at raised inlet air temperatures. The trade-off in energy savings at the facility level due to raising the inlet air temperatures versus the possible increase in server fan power and component temperatures is investigated. A detailed CFD analysis with a minimum number of server fans can provide a way to find an operating range of inlet air temperature for a hybrid cooled server. Changes in the model are carried out in 6SigmaET for an individual server and compared to the experimental data to validate the model. The results from this study can be helpful in determining the room level operating set points for data centers housing hybrid cooled server racks.
more »
« less
Energy Analysis of Rear Door Heat Exchangers in Data Centers With Spatial Workload Distribution
Abstract Data centers have complex environments that undergo constant changes due to fluctuations in IT load, commissioning and decommissioning of IT equipment, heterogeneous rack architectures and varying environmental conditions. These dynamic factors often pose challenges in effectively provisioning cooling systems, resulting in higher energy consumption. To address this issue, it is crucial to consider data center thermal heterogeneity when allocating workloads and controlling cooling, as it can impact operational efficiency. Computational Fluid Dynamics (CFD) models are used to simulate data center heterogeneity and analyze the impact of two different cooling mechanisms on operational efficiency. This research focuses on comparing the cooling based on facility water for Rear Door Heat Exchanger (RDHx) and conventional Computer Room Air Conditioning (CRAH) systems in two different data center configurations. Efficiency is measured in terms of ΔT across facility water. Higher ΔT will result in efficient operation of chillers. The actual chiller efficiency is not calculated as it would depend on local ambient conditions in which the chiller is operated. The first data center model represents a typical enterpriselevel configuration where all servers and racks have homogeneous IT power. The second model represents a colocation facility where server/rack power configurations are randomly distributed. These models predict temperature variations at different locations based on IT workload and cooling parameters. Traditionally, CRAH configurations are selected based on total IT power consumption, rack power density, and required cooling capacity for the entire data center space. On the other hand, RDHx can be scaled based on individual rack power density, offering localized cooling advantages. Multiple workload distribution scenarios were simulated for both CRAH and RDHx-based data center models. The results showed that RDHx provides a uniform thermal profile across the data center, irrespective of server/rack power density or workload distribution. This characteristic reduces the risk of over- or under-provisioning racks when using RDHx. Operational efficiency is compared in terms of difference in supply and return temperature of facility water for CRAH and RDHx units based on spatial heat dissipation and workload distribution. RDHx demonstrated excellent cooling capabilities while maintaining a higher ΔT, resulting in reduced cooling energy consumption, operational carbon footprint (?), and water usage.
more »
« less
- Award ID(s):
- 2209751
- PAR ID:
- 10536111
- Publisher / Repository:
- American Society of Mechanical Engineers
- Date Published:
- ISBN:
- 978-0-7918-8751-6
- Format(s):
- Medium: X
- Location:
- San Diego, California, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In response to the exponential growth of online platforms and the rise of web-based Artificial Intelligence (AI), the demand for computational power and the expansion of data centers have surged significantly. This trend necessitates advanced cooling strategies and heightened energy efficiency to address the increasing power densities of Information Technology (IT) equipment and the consequent rise in energy consumption. Consequently, there is a significant pivot towards efficient cooling mechanisms that emphasize thermal management and energy efficiency. Against this backdrop, our study thoroughly evaluates a two-phase direct-to-chip liquid cooling system's ability to effectively manage and dissipate heat in high-density rack environments. Central to our research is the deployment of a highly efficient Refrigerant-to-Liquid (R2L) Coolant Distribution Unit (CDU) across multi-racks, which face high thermal demands. This innovative system, featuring an in-row pumped two-phase CDU with a cooling capacity of 160 kW, is intricately integrated with row and rack manifolds and server cooling loops to ensure optimal cooling performance. To accurately simulate the thermal loads encountered in real-world data center operations, the study employs Thermal Testing Vehicles (TTVs). These 3U TTVs are equipped with 2.5 kW heaters, covering an extensive area of 2500 mm², thereby effectively replicating server thermal loads up to 10 kW. The investigation starts with a detailed description of the system's design and continues with the commissioning process. This process includes extensive hydraulic and thermal testing, along with a comprehensive assessment of the impact of pressure drops across the system, focusing on supply manifolds, cooling loops, dry breaks, and return manifolds, utilizing Cooling Loops (CLs) each containing four Cold Plates (CPs). The study culminates in the analysis of experimental data from heating the TTVs, focusing on the efficiency of two-phase cooling in transferring heat from the TTVs to chilled water using R134a refrigerant as the performance benchmark. Future directions include exploring eco-friendly cooling practices by investigating alternative green refrigerants with low Global Warming Potential (GWP) to replace R134a, aligning with global sustainability goals and the imperative to reduce greenhouse gas emissions. The observed maximum values were calculated at a specific volumetric flow rate of 0.48 LPM/kW and a Tcase as low as 56.4 °C was achieved. These results demonstrate the system's capability to significantly enhance thermal management in data centers, tackle the challenges presented by high-power density chips, and encourage broader adoption of two-phase cooling technologies as a sustainable strategy for thermal regulation in the face of increasing computational demands.more » « less
-
About 40% of the energy utilized in data centers is used for cooling systems, and this percentage has increased significantly in recent years. Data center server racks receive indirect cooling from computer room air conditioning units (CRACs) or computer room air handler units (CRAHs), and chilled air is sent to the racks through a raised floor plenum to cool the server room. This approach is inefficient because the server room has excessive cooling while the IT equipment has inadequate cooling. The data center industry has begun to use thermally and energy-efficient single-phase liquid cooling solutions as a result of the tremendous increase in IT power density and energy usage. One of the most popular liquid cooling systems will be examined in the current study, which is based on numerous circumstances. Under various secondary coolant conditions, the hydro-thermal performance of different liquid-to-air and liquid-to-liquid coolant distribution units (CDUs) will be assessed experimentally using multi-racks loaded with different numbers of thermal testing vehicles (TTVs), and the system response to any change in the flow will be examined by disconnecting and reconnecting the TTVs cooling loops in the multi-racks through different sequences of transient events. Moreover, a set of rules and guidelines will be established to commission these units.more » « less
-
Data centers house a variety of compute, storage, network IT hardware where equipment reliability is of utmost importance. Heat generated by the IT equipment can substantially reduce its service life if Tjmax, maximum temperature that the microelectronic device tolerates to guarantee reliable operation, is exceeded. Hence, data center rooms are bound to maintain continuous conditioning of the cooling medium becoming large energy consumers. The objective of this work is to introduce and evaluate a new end-of-aisle cooling design which consists of three cooling configurations. The key objectives of close-coupled cooling are to enable a controlled cooling of the IT equipment, flexible as well as modular design, and containment of hot air exhaust from the cold air. The thermal performance of the proposed solution is evaluated using CFD modeling. A computational model of a small size data center room has been developed. Larger axial fans are selected and placed at rack-level which constitute the rack-fan wall design. The model consists of 10 electronic racks each dissipating a heat load of 8kw. The room is modeled to be hot aisle containment i.e. the hot air exhaust exiting for each row is contained and directed within a specific volume. Each rack has passive IT with no server fans and the servers are cooled by means of rack fan wall. The cold aisle is separated with hot aisle by means of banks of heat exchangers placed on the either sides of the aisle containment. Based on the placement of rack fans, the design is divided to three sub designs — case 1: passive heat exchangers with rack fan walls; case 2: active heat exchangers (HXs coupled with fans) with rack fan walls; case 3: active heat exchangers (hxs coupled with fans) with no rack fans. The cooling performance is calculated based on the thermal and flow parameters obtained for all three configurations. The computational data obtained has shown that the case 1 is used only for lower system resistance IT. However, case 2 and Case 3 can handle denser IT systems. Case 3 is the design that can consume lower fan energy as well as handle denser IT systems. The paper also discusses the cooling behavior of each type of design.more » « less
-
The escalating information technology (IT) loads in modern data centers (DCs) present formidable challenges for traditional room-conditioning systems. The heat dissipated from IT equipment has witnessed a significant surge due to the rapid development of data processing, retrieval, and storage, driven by changing technology trends and the growing demand for online services. This evolving landscape poses a substantial burden on air-cooling systems, pushing them to their limits, especially with the prevailing trend of rising power densities in microprocessors and the emergence of hot spots. Amidst these challenges, singlephase cold plate cooling is gaining traction as IT power densities experience a dramatic climb. However, the widespread adoption of this cooling method faces impediments such as the limited availability of chilled water supplies, constrained air distribution pathways, and the absence of elevated floors in many older DCs. In response to these limitations, liquid-to-air (L2A) cooling distribution units (CDUs) have emerged as an alternative method. By incorporating hybrid air and liquid cooling technologies, the industry aims to achieve precise, ondemand cooling through the utilization of various techniques. In the realm of hybrid cooling systems that integrate both air and liquid cooling technologies, a partial failure of the Computer Room Air Handlers (CRAH) introduces unique challenges. Such a failure has the potential to disrupt the delicate balance between air and liquid cooling components, leading to uneven heat dissipation. Moreover, the interdependence of liquid and air cooling in hybrid systems means that even a partial failure can trigger a domino effect, reducing the overall cooling efficiency of the system. This comprehensive study delves into the implications of partial failure in the CRAH unit within the highpower density racks of a hybrid-cooled DC. The investigation explores how this partial failure impacts various critical parameters, including cooling capacity (CC), supply air temperature (SAT), air flow rate, supply fluid temperature (SFT), and thermal testing vehicle (TTV) heater case temperatures. For the purposes of this study, two L2A in-row CDUs were utilized, with a combined total heat load of 129 kW supplied to three racks. The experimental setup is meticulously equipped with the necessary instruments for monitoring and assessing tests on both the liquid coolant and air sides. By addressing these issues, the research contributes valuable insights to the ongoing efforts to optimize data center cooling solutions in the face of evolving IT demands and technological advancements.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1115/IPACK2023-112078
