0
Research Papers

A Holistic Evaluation of Data Center Water Cooling Total Cost of Ownership

[+] Author and Article Information
Dustin W. Demetriou

Mem. ASME
Advanced Thermal Energy Efficiency Lab,
IBM Systems,
2455 South Road,
Poughkeepsie, NY 12601
e-mail: dwdemetr@us.ibm.com

Vinod Kamath

Cooling Architecture,
Lenovo US,
7001 Development Drive,
Morrisville, NC 27560
e-mail: vkamath1@lenovo.com

Howard Mahaney

Thermal Development,
IBM Systems,
11400 Burnet Road,
Austin, TX 78758
e-mail: mahaney@us.ibm.com

Contributed by the Electronic and Photonic Packaging Division of ASME for publication in the JOURNAL OF ELECTRONIC PACKAGING. Manuscript received September 25, 2015; final manuscript received January 4, 2016; published online March 10, 2016. Assoc. Editor: Mehdi Asheghi.

J. Electron. Packag 138(1), 010912 (Mar 10, 2016) (11 pages) Paper No: EP-15-1100; doi: 10.1115/1.4032494 History: Received September 25, 2015; Revised January 04, 2016

The generation-to-generation information technology (IT) performance and density demands continue to drive innovation in data center cooling technologies. For many applications, the ability to efficiently deliver cooling via traditional chilled air cooling approaches has become inadequate. Water cooling has been used in data centers for more than 50 years to improve heat dissipation, boost performance, and increase efficiency. While water cooling can undoubtedly have a higher initial capital cost, water cooling can be very cost effective when looking at the true life cycle cost of a water-cooled data center. This study aims at addressing how one should evaluate the true total cost of ownership (TCO) for water-cooled data centers by considering the combined capital and operational cost for both the IT systems and the data center facility. It compares several metrics, including return-on-investment for three cooling technologies: traditional air cooling, rack-level cooling using rear door heat exchangers, and direct water cooling (DWC) via cold plates. The results highlight several important variables, namely, IT power, data center location, site electric utility cost, and construction costs and how each of these influences the TCO of water cooling. The study further looks at implementing water cooling as part of a new data center construction project versus a retrofit or upgrade into an existing data center facility.

FIGURES IN THIS ARTICLE
<>
Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.

References

Koomey, J. , 2011, “ Growth in Data Center Electricity Use 2005 to 2010,” Analytical Press, Oakland, CA, Aug. 1, http://www.analyticspress.com/datacenters.html
IBM System and Technology White Paper, 2014, “ Why Understanding Data Center TCO is Critical to Your Business,” IBM White Paper, Last accessed Jan. 4, 2016, http://public.dhe.ibm.com/common/ssi/ecm/ql/en/qlw12347usen/QLW12347USEN.PDF
Goth, G. , Mullady, R. , Zoodsma, R. , VanDeventer, A. C. , Porter, D. , and Kelly, P. , 2014, “ An Overview of the IBM zEnterprise EC12 Processor Cooling System,” IEEE ITherm 2014, Orlando, FL, May 27–30, 588–597.
Demetriou, D. , 2015, “ Liquid Cooling Performance Capability,” ASHRAE Seminar Recordings: Seminar 23, ASHRAE Winter Conference, Chicago, IL
ASHRAE, 2014, Liquid Cooling Guidelines for Datacom Equipment Centers (ASHRAE Datacom Series Book), Vol. 4, 2nd ed., ASHRAE, Atlanta, GA.
Stansberry, M. , and Kudritzki, J. , 2012, “ Uptime Institute 2012 Data Center Industry Survey,” Last accessed Jan. 4, 2016, https://uptimeinstitute.com/research-publications
Ellsworth, M. J. , 2011, “ An Overview of the IBM Power 775 Supercomputer Water Cooling System,” ASME Paper No. IPACK2011-52130.
Ellsworth, M. J. , and Iyengar, M. , 2009, “ Energy Efficiency Analyses and Comparison of Air and Water Cooled High Performance Servers,” ASME Paper No. IPACK2009-89248.
David, M. , Iyengar, M. , Parida, P. , Simons, R. , Schultz, M. , Gaynes, M. , Schmidt, R. , and Chainer, T. , 2012, “ Experimental Characterization of an Energy Efficient Chiller-Less Data Center Test Facility With Warm Water Cooled Servers,” SEMI-THERM, San Jose, CA, Mar. 18–22, pp. 232–237.
Wei, J. , 2011, “ Hybrid Cooling Technology for Large Scale Computing Systems—From Back to the Future,” ASME Paper No. IPACK2011-52045.
“  Asetek Data Center Liquid Cooling,” Last accessed Jan. 4, 2016, http://www.asetek.com
“CoolIT Systems,” Last accessed Jan. 4, 2016, http://www.coolitsystems.com
Iyengar, M. , and Schmidt, R. , 2007, “ Analytical Modeling for Thermodynamic Characterization of Data Center Systems,” ASME J. Electron. Packag., 131(2), p. 021009. [CrossRef]
Demetriou, D. W. , Khalifa, H. E. , Iyengar, M. , and Schmidt, R. , 2011, “ Development and Experimental Validation of a Thermo-Hydraulic Model for Data Centers,” HVAC&R Res., 17(4), pp. 540–555.
Breen, T. J. , Walsh, E. , Punch, J. , Shah, A. , and Bash, C. , 2010, “ From Chip to Cooling Tower Data Center Modeling—Part I: Influence of Server Inlet Temperature and Temperature Rise across Cabinet,” IEEE ITherm, Las Vegas, NV, June 2–5, pp. 1–10.
Breen, T. J. , Walsh, E. , Punch, J. , Shah, A. , Bash, C. , Kumari, N. , and Cader, T. , 2011, “ From Chip to Cooling Tower Data Center Modeling: Chip Leakage Power and Its Impact on Cooling Infrastructure Energy Efficiency,” ASME InterPACK, Portland, OR, July 6–8, pp. 433–442.
Khalifa, H. E. , and Demetriou, D. W. , 2010, “ Energy Optimization of Air-Cooled Data Centers,” ASME J. Therm. Sci. Eng. Appl., 2(4), p. 041005. [CrossRef]
Demetriou, D. W. , and Khalifa, H. E. , 2012, “ Optimization of Enclosed Aisle Data Centers Using Bypass Recirculation,” ASME J. Electron. Packag., 134(2), p. 020904. [CrossRef]
Wemhoff, A. P. , del Valle, M. , Abbasi, K. , and Ortega, A. , 2013, “ Thermodynamic Modeling of Data Center Cooling Systems,” ASME Paper No. IPACK2013-73116.
United States Energy Information Administration, 2013, “ Electricity Statistics and Analysis,” http://www.eia.doe.gov/fuelelectric.html
United States Department of Energy, 2013, “ Energy Escalation Rate Calculator,” Federal Energy Management Program, Last accessed Jan. 4, 2016, http://energy.gov/eere/femp/energy-escalation-rate-calculator-download
Park, C. , 2012, Fundamentals of Engineering Economics, 3rd ed., Prentice Hall, New York.
RSMeans, 2014, Mechanical Cost Data Book, 37th ed., RSMeans, New York.
Coles, H. , Carter, B. , Zelghami, R. , and Sorell, V. , 2014, “ Total Cost of Ownership of Cold-Plate Liquid-Cooling Hybrid Solutions for Data Centers,” 24 × 7 Seminar Presentation, 24 × 7 Exchange Fall Conference, Phoenix, AZ, Oct. 26–29.
Marion, W. , and Urban, K. , 1995, User's Manual for TMY2s Typical Meteorological Years, National Renewable Energy Laboratory, Golden, CO.

Figures

Grahic Jump Location
Fig. 1

Schematic of (a) IT equipment chassis, (b) hybrid-cooled IT node, and (c) direct water cooled IT node

Grahic Jump Location
Fig. 3

Energy analysis results for Denver, CO assuming a 335 W air-cooled node with (a) Greenfield data center design and (b) Brownfield data center design, generated using methodology described in Ref. [14]

Grahic Jump Location
Fig. 4

Summary of TCO metrics for Denver, CO, assuming $0.12/kWh for electricity, 0.0% energy escalation, 335 W air-cooled node, and 5× DWC IT cooling hardware cost

Grahic Jump Location
Fig. 5

Impact of electricity cost on project payback assuming Denver, CO, 0.0% energy escalation, 335 W air-cooled node, and 5 × DWC IT cooling hardware cost

Grahic Jump Location
Fig. 6

Impact of water cooling hardware cost on project payback assuming Denver, CO, $0.12/kWh electricity cost, 0.0% energy escalation, and a 335 W air-cooled node. This includes the cost of cold plates, hoses, manifolds, and quick disconnects.

Grahic Jump Location
Fig. 7

Impact of node power consumption on project payback assuming Denver, CO, $0.12/kWh electricity cost, 0.0% energy escalation, and 5 × DWC IT cooling hardware cost

Grahic Jump Location
Fig. 8

Impact of location on the 5-year NPV assuming location-specific electricity cost and escalation rates, 335 W air-cooled node, and 5 × DWC hardware cost

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In