0
Special Section Articles

Liquid Cooling of Compute System

[+] Author and Article Information
Jessica Gullbrand

Intel Corporation,
Data Center Engineering and Architecture Group,
2111 NE 25th Avenue,
Hillsboro, OR 97124
e-mail: jessica.gullbrand@intel.com

Mark J. Luckeroth

Intel Corporation,
Data Center Engineering and Architecture Group,
2111 NE 25th Avenue,
Hillsboro, OR 97124
e-mail: mark.j.luckeroth@intel.com

Mark E. Sprenger

Intel Corporation,
Data Center Engineering and Architecture Group,
2111 NE 25th Avenue,
Hillsboro, OR 97124
e-mail: mark.e.sprenger@intel.com

Casey Winkel

Intel Corporation,
Data Center Engineering and Architecture Group,
2111 NE 25th Avenue,
Hillsboro, OR 97124
e-mail: casey.winkel@intel.com

1Corresponding author.

Contributed by the Electronic and Photonic Packaging Division of ASME for publication in the JOURNAL OF ELECTRONIC PACKAGING. Manuscript received July 5, 2018; final manuscript received November 16, 2018; published online March 1, 2019. Assoc. Editor: Jin Yang.

J. Electron. Packag 141(1), 010802 (Mar 01, 2019) (10 pages) Paper No: EP-18-1054; doi: 10.1115/1.4042802 History: Received July 05, 2018; Revised November 16, 2018

The continued demand for increasing compute performance results in an increasing system power and power density of many computers. The increased power requires more efficient cooling solutions than traditionally used air cooling. Therefore, liquid cooling, which has traditionally been used for large data center deployments, is becoming more mainstream. Liquid cooling can be used selectively to cool the high power components or the whole compute system. In this paper, the example of a fully liquid cooled server is used to describe different ingredients needed, together with the design challenges associated with them. The liquid cooling ingredients are cooling distribution unit (CDU), fluid, manifold, quick disconnects (QDs), and cold plates. Intel is driving an initiative to accelerate liquid cooling implementation and deployment by enabling the ingredients above. The functionality of these ingredients is discussed in this paper, while cold plates are discussed in detail.

FIGURES IN THIS ARTICLE
<>
Copyright © 2019 by ASME
Your Session has timed out. Please sign back in to continue.

References

Gao, T. , Tang, H. , Cui, Y. , and Luo, Z. , 2018, “ A Test Study of Technology Cooling Loop in a Liquid Cooling System,” 17th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), San Diego, CA, May 29–June 1, pp. 740–747.
Patterson, M. K. , Krishnan, S. , and Walters, J. M. , 2016, “ On Energy Efficiency of Liquid Cooled HPC Datacenters,” 15th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm) Las Vegas, NV, May 31–June 3, pp. 685–693.
Chu, R. C. , Simons, R. E. , Ellsworth, M. J. , Schmidt, R. R. , and Cozzolino, V. , 2004, “ Review of Cooling Technologies for Computer Products,” IEEE Trans. Device Mater. Reliab., 4(4), pp. 568–585.
Ellsworth, M. J. , Campbell, L. A. , Simons, R. E. , Iyengar, M. K. , Schmidt, R. R. , and Chu, R. C. , 2008, “ The Evolution of Water Cooling for Large IBM Large Server Systems: Back to the Future,” 11th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Orlando, FL, May 28–31, pp. 266–274.
McFarlane, R. , 2012, “ Will Water-Cooled Servers Make Another Splash in the Data Center?,” Tech Target Network, Search Data Center, Newton, MA, accessed Feb. 26, 2012, https://searchdatacenter.techtarget.com/tip/Will-water-cooled-servers-make-another-splash-in-the-data-center
Schmidt, R. R. , 2005, “ Liquid Cooling Is Back,” Electronics Cooling, 11(3), (epub).
Patrizio, A. , 2018, “ Lenovo Introduces New Water-Cooled Server Technology,” Network World, Framingham, MA, accessed Feb. 26, 2018, https://www.networkworld.com/article/3258646/data-center/lenovo-introduces-new-water-cooled-server-technology.html
Koblentz, E. , 2018, “ How to Get Started With Liquid Cooling for Servers and Data Center Racks,” Data Centers Trends Newsletter, TechRepublic, US edition, accessed July 8, 2018, https://www.techrepublic.com/article/how-to-get-started-with-liquid-cooling-for-servers-and-data-center-racks/
Iyengar, M. , David, M. , Parida, P. , Kamath, V. , Kochuparambil, B. , Graybill, D. , Schultz, M. , Gaynes, M. , Simons, R. , Schmidt, R. , and Chainer, T. , 2012, “ Server Liquid Cooling With Chiller-Less Data Center Design to Enable Significant Energy Savings,” 28th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (SEMI-THERM), San Jose, CA, Mar. 18–22, pp. 212–223.
ASHRAE, 2014, Liquid Cooling Guidelines for Datacom Equipment Centers (ASHRAE Datacom Series 4), 2nd ed., Atlanta, GA.
Fan, Y. , Winkel, C. , Kulkarni, D. , and Tian, W. , 2018, “ Analytical Design Methodology for Liquid Based Cooling Solutions for High TDP CPUs,” 17th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), San Diego, CA, May 29–June 1, pp. 582–586.
Prasher, R. , and Chang, J.-Y. , 2008, “ Cooling of Electronic Chips Using Microchannel and Micro-Pin Fin Heat Exchangers,” ASME Paper No. ICNMM2008-62384.
Matsuda, M. , Mashiko, K. , Saito, Y. , Nguyen, T. , and Nguyen, T. , 2015, “ Micro-Channel Cold Plate Units for Cooling Super Computer,” Fujikura Tech. Rev., 2015, pp. 53–57.
Thome, J. R. , Olivier, J. A. , and Park, J. E. , 2009, “ Two-Phase Cooling of Targets and Electronics for Particle Physics Experiments,” Topical Workshop on Electronics for Particle Physics, Paris, France, Sept. 21–25, pp. 366–376.
Barth, T. A. , 2011, “ Brief Overview of Uncertainty Quantification and Error Estimation in Numerical Simulation,” NASA Ames Research Center, Mountain View, CA.
Perez, R. A. , 2008, Uncertainty Analysis of Computational Fluid Dynamics Via Polynomical Chaos, Virginia Polytechnic Institute and State University, Blacksburg, VA.
Xiu, D. , 2010, Numerical Methods for Stochastic Computations, Princeton University Press, Princeton, NJ.
Eldred, M. S. , 2009, “ Recent Advances in Non-Intrusive Polynomial Chaos and Stochastic Collocation Methods for Uncertainty Analysis and Design,” AIAA Paper No. 2009-2274.
Eldred, M. S. , and Burkardt, J. , 2009, “ Comparison of Non-Intrusive Polynomial Chaos and Stochastic Collocation Methods for Uncertainty Quantification,” AIAA Paper No. 2009-0976.
Bodla, K. K. , Murthy, J. Y. , and Garimella, S. V. , 2015, “ Optimization Under Uncertainty for Electronics Cooling Design,” Encyclopedia of Thermal Packaging, Thermal Packaging Tools, World Scientific Publishing Company, Singapore, pp. 233–265.
Feinberg, J. , and Langtangen, H. P. , 2015, “ ChaosPy, an Open Source Tool for Designing Methods of Uncertainty Quantification,” J. Comput. Sci., 11, pp. 45–57. [CrossRef]

Figures

Grahic Jump Location
Fig. 1

Example of a fluid loop from CDU to server

Grahic Jump Location
Fig. 2

Server universal QD

Grahic Jump Location
Fig. 3

Cross flow parallel plate fin cold plate design with fluid flow path

Grahic Jump Location
Fig. 4

Center flow parallel plate fin cold plate design with fluid flow path

Grahic Jump Location
Fig. 5

Removable DIMM cold plate design

Grahic Jump Location
Fig. 6

Thermal pathway for air cooled (top) and liquid cooled (bottom) components

Grahic Jump Location
Fig. 7

Thermal contour of board and component temperatures as a function of distance from a directly cooled component (right edge represents the centerline of the directly cooled component) for a fully liquid cooled system

Grahic Jump Location
Fig. 8

Indirectly cooled FET junction temperature as a function of component power and different heat transfer assumptions

Grahic Jump Location
Fig. 9

Indirectly cooled FET junction temperature as a function of board plane conductivity and distance to the directly cooled component

Grahic Jump Location
Fig. 10

Thermal contour of predicted temperatures for an area of an HPCS board

Grahic Jump Location
Fig. 11

Example of an HPCS fluid loop solution for cooling board components and pipe flattening used

Grahic Jump Location
Fig. 12

Pressure drop curve as a function of flow rate (top) and pressure drop distribution at the operating point (bottom)

Tables

Errata

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In