0
Research Papers

Steady State and Transient Experimentally Validated Analysis of Hybrid Data Centers

[+] Author and Article Information
Tianyi Gao

Binghamton University-SUNY,
Binghamton, NY 13902
e-mail: Tgao1@binghamton.edu

Bahgat Sammakia, Emad Samadiani

Binghamton University-SUNY,
Binghamton, NY 13902

Roger Schmidt

IBM Corporation,
Poughkeepsie, NY 12601

1Corresponding author.

Contributed by the Electronic and Photonic Packaging Division of ASME for publication in the JOURNAL OF ELECTRONIC PACKAGING. Manuscript received April 7, 2014; final manuscript received November 12, 2014; published online January 19, 2015. Assoc. Editor: Mehmet Arik.

J. Electron. Packag 137(2), 021007 (Jun 01, 2015) (12 pages) Paper No: EP-14-1041; doi: 10.1115/1.4029163 History: Received April 07, 2014; Revised November 12, 2014; Online January 19, 2015

Data centers consume a considerable amount of energy which is estimated to be about 2% of the total electrical energy consumed in the U.S. in the year 2010, and this number continues to increase every year. Thermal management is becoming increasingly important in the effort to improve the energy efficiency and reliability of data centers. The goal is to keep the information technologies (IT) equipment temperature within the allowable range in high power density data centers while reducing the energy used for cooling. In this regard, liquid and hybrid air/water cooling systems are alternatives to traditional air cooling. In particular, these options offer advantages for localized cooling higher power racks which may not be manageable using the room level air cooling system without requiring significantly more energy. In this paper, a hybrid cooling system in data centers is investigated. In addition to traditional raised floor, cold aisle-hot aisle configuration, a liquid–air heat exchanger attached to the back of racks is considered. First of all, the paper presents a review of literature of the study of this heat exchanger strategy in the thermal management of a data center. The discussion focus on rear door heat exchanger (RDHx) performance, both the steady state and transient impact are analyzed. The studies show that under some circumstances, this hybrid approach could be a viable alternative to meet the American Society of Heating, Refrigerating, and Air-Conditioning Engineers (ASHRAE) recommended inlet air temperatures, while at the same time reducing the overall energy consumption in high density data centers. The hybrid design approach can also significantly improve the dynamic performance during rack power increases or computer room air conditioner (CRAC) unit failure. And then, additional parametric steady state and dynamic analyses, are presented in detail for the different scenarios.

Copyright © 2015 by ASME
Your Session has timed out. Please sign back in to continue.

References

ASHRAE, 2005, Datacom Equipment Power Trends and Cooling Applications, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta, GA.
Scaramella, J., 2008, “Next-Generation Power and Cooling for Blade Environments,” IDC, Framingham, MA, Technical Report No. 215675.
Garimella, S. V., Fleischer, A. S., Murthy, J. Y., Keshavarzi, A., Prasher, R., Patel, C., and Raad, P. E., 2008, “Thermal Challenges in Next Generation Electronic Systems,” IEEE Trans. Compon. Packag. Technol., 31(4), pp. 801–815. [CrossRef]
Schmidt, R., and Iyengar, M., 2007, “Best Practices for Data Center Thermal and Energy Management: Review of Literature,” ASHRAE Trans., 113(1), pp. 206–218.
Pacific Gas and Electric Company Report, 2006, “High Performance Data Centers—A Design Guidelines Sourcebook,” Pacific Gas and Electric Co., San Francisco, CA.
Ramos, L., and Ricardo, B., 2008, “C-Oracle: Predictive Thermal Management for Data Centers,” IEEE 14th International Symposium on High Performance Computer Architecture (HPCA 2008), Salt Lake City, UT, Feb. 16–20, pp. 111–122. [CrossRef]
Rambo, J., and Joshi, Y., 2007, “Modeling of Data Center Airflow and Heat Transfer: State of the Art and Future Trends,” Distributed Parallel Databases, 21(2–3), pp. 193–225. [CrossRef]
Sharma, K. R., Bash, C. E., Patel, D. C., Friedrich, J. R., and Chase., S. J., 2005, “Balance of Power: Dynamic Thermal Management for Internet Data Centers,” IEEE Internet Comput., 9(1), pp. 42–49. [CrossRef]
Iyengar, M., Schmidt, R., Sharma, A., McVicker, G., Shrivastava, S., Sri-Jayantha, S., Amemiya, Y., Dang, H., Chainer, T., and Sammakia, B., 2005, “Thermal Characterization of Non-Raised Floor Air Cooled Data Centers Using Numerical Modeling,” ASME Paper No. IPACK2005-73387. [CrossRef]
Shrivastava, S., Sammakia, B., Schmidt, R., and Iyengar, M., 2005, “Comparative Analysis of Different Data Center Airflow Management Configurations,” ASME Paper No. IPACK2005-73234. [CrossRef]
Scofield, C. M., and Weaver, T. S., 2008, “Data Center Cooling—Using Wet-Bulb Economizers,” ASHRAE J., 50(8), pp. 52–59.
Koomey, J. G., “Estimating Total Power Consumption by Servers in the U.S. and the World, 2007,” Lawrence Berkeley National Laboratory, Berkeley, CA, available at: http://hightech.lbl.gov/documents/DATA_CENTERS/svrpwrusecompletefinal.pdf
Iyengar, M., and Schmidt, R., 2007, “Analytical Modeling of Energy Consumption and Thermal Performance of Data Center Cooling Systems—From the Chip to the Environment,” ASME Paper No. IPACK2007-33924. [CrossRef]
Schmidt, R., 2004, “Thermal Profile of a High-Density Data Center—Methodology to Thermally Characterize a Data Center,” ASHRAE Trans., 110(2), pp. 635–642.
Schmidt, R., Iyengar, M., Beaty, D., and Shrivastava, S., 2005, “Thermal Profile of a High-Density Data Center—Hot Spot Heat Fluxes of 512 W/ft2,” ASHRAE Trans., 111(2), pp. 765–777.
Schmidt, R., Iyengar, M., and Mayhugh, S., 2006, “Thermal Profile of World’s Third Fastest Supercomputer—IBM’s ASCI Purple Cluster,” ASHRAE Trans., 112(2), pp. 209–219.
Shrivastava, S., Iyengar, M., Sammakia, B., Schmidt, R., and VanGilder, J., 2009, “Experimental-Numerical Comprison for a High-Density Data Center: Hot Spot Heat Fluxes in Excess of 500 w/ft,” IEEE Trans. Compon. Packag. Technol., 32(1), pp. 166–172. [CrossRef]
Samadiani, E., Joshi, Y., and Mistree, F., 2008, “The Thermal Design of a Next Generation Data Center: A Conceptual Exposition,” ASME J. Electron. Packag., 130(4), p. 041104. [CrossRef]
Niemann, J., 2008, “Hot Aisle vs. Cold Aisle Containment,” American Power Conversion, West Kingston, RI, APC White Paper #35.
Schmidt, R., 2005, “Liquid Cooling is Back,” Electron. Cooling, 11(3), pp. 34–38.
Patterson, M. K., and Fenwick, D., 2008, “The State of Data Center Cooling—A Review of Current Air and Liquid Cooling Solutions,” Intel, Digital Enterprise Group, Santa Clara, CA.
Chu, R. E., Simons, R. E., and Moran, K. P., 1991, “System Cooling Design Considerations for Large Mainframe Computers,” Cooling Techniques for Computers, W.Aung, ed., Hemisphere Publishing, New York.
ASHRAE Technical Committee 9.9, 2011, “Thermal Guidelines for Liquid Cooled Data Processing Environments,” American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta, GA.
Schmidt, R., Chu, M., Ellsworth, M., Iyengar, M., Porter, D., Kamath, V., and Lehman, B., 2005, “Maintaining Datacomm Rack Inlet Temperatures With Water Cooled Heat Exchangers,” ASME Paper No. IPACK2005-73468. [CrossRef]
David, M., and Schmidt, R., 2014, “Impact of ASHRAE Environmental Classes on Data Center,” IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Orlando, FL, May 27–30, pp. 1092–1099. [CrossRef]
Bhopte, S., Iyengar, M., Sammakia, B., Schmidt, R., and Agonafer, D., 2006, “Numerical Modeling of Data Center Clusters—Impact of Model Complexity,” ASME Paper No. IMECE2006-13494. [CrossRef]
Bhopte, S., Sammakia, B., Iyengar, M., and Schmidt, R., 2006, “Guidelines on Managing Under Floor Blockages for Improved Data Center Performance,” ASME Paper No. IMECE2006-13711. [CrossRef]
Song, Z., Murray, B., and Sammakia, B., 2014, “Numerical Investigation of Inter-Zonal Boundary Conditions for Data Center Thermal Analysis,” Int. J. Heat Mass Transfer, 68(1), pp. 649–658. [CrossRef]
Gao, T., Sammakia, B., Geer, J., David, M., and Schmidt, R., “Experimentally Verified Transient Models of Data Center Crossflow Heat Exchangers,” ASME Paper No. IMECE2014-36022. [CrossRef]
Ibrahim, M., Sammakia, B., Afram, F., Ghose, K., Murray, B., Iyengar, M., and Schmidt, R., 2011, “Analytical Compact Model of a 2U Server,” ASME Paper No. IPACK2011-52165. [CrossRef]
Gao, T., Geer, J., and Sammakia, B., 2014, “Nonuniform Temperature Boundary Condition Effect on Data Center Cross Flow Heat Exchanger Dynamic Performance,” Int. J. Heat Mass Transfer, 79, pp. 1048–1058. [CrossRef]
Gao, T., Sammakia, B., Murray, B., Ortega, A., and Schmidt, R., 2014, “Cross Flow Heat Exchanger Modeling of Transient Temperature Input Conditions,” IEEE Trans. Compon. Packag. Manuf. Technol., 4(11), pp. 1796–1807. [CrossRef]
Gao, T., Sammakia, B., Geer, J., Ortega, A., and Schmidt, R., “Dynamic Analysis of Cross Flow Heat Exchangers in Data Centers Using Transient Effectiveness Method,” IEEE Trans. Compon., Packag. Manuf. Technol., (in press). [CrossRef]
Mentor Graphics, 2012, “FloTherm 9.3 Reference Manual,” Mentor Graphics, Wilsonville, OR.
Gao, T., Samadiani, E., Sammakia, B., and Schmidt, R., 2013, “Comparative Thermal and Energy Analysis of A Hybrid Cooling Data Center With Rear Door Heat Exchangers,” ASME Paper No. IPACK2013-73101. [CrossRef]
Coolcentric, 2011, “Rear Door Heat Exchanger Planning Guide,” Vette Corp., Pelham, NH.
Schmidt, R., and Iyengar, M., 2009, “Server Rack Rear Door Heat Exchanger and the New ASHRAE Recommended Environmental Guidelines,” ASME Paper No. IPACK2009-89212. [CrossRef]
ASHRAE, 2009, “Thermal Guidelines for Data Processing Equipment—Second Edition,” American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta, GA.
Mulay, V., Karajgikar, S., Agonafer, D., Schmidt, R., and Iyengar, M., 2007, “Computational Study of Hybrid Cooling Solution for Thermal Management of Data Centers,” ASME Paper No. IPACK2007-33000. [CrossRef]
Mulay, V., Karajgikar, S., Agonafer, D., Schmidt, R., and Iyengar, M., 2007, “Parametric Study of Hybrid Cooling Solution for Thermal Management of Data Centers,” ASME Paper No. IMECE2007-43761. [CrossRef]
Judge, J., Pouchet, J., Ekbote, A., and Dixit, S., 2008, “Reducing Data Center Energy Consumption,” ASHRAE J., Nov., pp. 14–26.
Gao, T., Schmidt, R., and Sammakia, B., 2013, “Computational Study of Air Cooled Data Centers Assisted With Locally Distributed Water to Air Heat Exchangers,” ASME Paper No. IMECE2013-65958. [CrossRef]
Mulay, V. P., 2010, “Analysis of Data Center Cooling Strategies and the Impact of the Dynamic Thermal Management on the Data Center Efficiency,” available at: http://hdl.handle.net/10106/2048
Sharma, R., Bash, C. E., Patel, C., Friedrich, R., and Chase, J., 2005, “Balance of Power—Dynamic Thermal Management of Internet Data Centers,” IEEE Comput. Soc., 9(1), pp. 42–49. [CrossRef]
Ibrahim, M., Gondipalli, S., Bhopte, S., Sammakia, B., Murray, B., Ghose, K., Iyengar, M., and Schmidt, R., 2010, “Numerical Modeling Approach to Dynamic Data Center Cooling,” 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV, June 2–5. [CrossRef]
Boucher, T. D., Auslander, D. M., Bash, C. E., Federspiel, C. C., and Patel, C. D., 2006, “Viability of Dynamic Cooling Control in a Data Center Environment,” ASME J. Electron. Packag., 128(2), pp. 137–144. [CrossRef]
Bash, C. E., Patel, C. D., and Sharma, R. K., 2006, “Dynamic Thermal Management of Air Cooled Data Centers,” 10th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronics Systems (ITHERM '06), San Diego, CA, May 30–June 2, pp. 444–452. [CrossRef]
Gao, T., Samadiani, E., Schmidt, R., and Sammakia, B., 2013, “Dynamic Analysis of Hybrid Cooling Data Centers Subject to the Failure of CRAC Units,” ASME Paper No. IPACK2013-73196. [CrossRef]

Figures

Grahic Jump Location
Fig. 1

Underfloor air supply configuration [1]

Grahic Jump Location
Fig. 2

Rack layout for data center used for study—plan view

Grahic Jump Location
Fig. 3

Representative CFD model of data center

Grahic Jump Location
Fig. 4

Average inlet and outlet temperatures of rack A [34]

Grahic Jump Location
Fig. 5

Effectiveness of heat exchanger [24]

Grahic Jump Location
Fig. 6

Typical performance of a rear door heat exchanger (numerical results compared with experimental data [36]; percentage heat removal as function of water temperature and flow rate) [35]

Grahic Jump Location
Fig. 7

Surface map showing effectiveness

Grahic Jump Location
Fig. 8

Rack inlet air temperature versus distance from raised floor for baseline performance comparison: (a) rack with rear door heat exchanger enhancement and (b) rack without cooling enhancement [24].

Grahic Jump Location
Fig. 9

Summary of data center impact of using rear cover heat exchanger [24]

Grahic Jump Location
Fig. 10

Impact of hybrid solution on data center with 2 and 3 ft underfloor plenum [40]

Grahic Jump Location
Fig. 11

Average inlet temperature of racks A and D [42]

Grahic Jump Location
Fig. 12

Average inlet temperature of racks A and D [42]

Grahic Jump Location
Fig. 13

Energy consumption of each operation conditions [42]

Grahic Jump Location
Fig. 14

Rack A average inlet temperature [35]

Grahic Jump Location
Fig. 15

Rack A average inlet temperature [35]

Grahic Jump Location
Fig. 16

Rack A average inlet temperature [35]

Grahic Jump Location
Fig. 17

Rack A average inlet temperature [35]

Grahic Jump Location
Fig. 18

Airflow profile due to CRAC failure with time [42]

Grahic Jump Location
Fig. 19

(a) Inlet temperature of rack B1 [42] and (b) inlet temperature of rack D1 [42]

Grahic Jump Location
Fig. 20

(a) Inlet temperature of rack B1 [42] and (b) inlet temperature of rack D1 [42]

Grahic Jump Location
Fig. 21

Power changing profile with time [42]

Grahic Jump Location
Fig. 22

Average Inlet temperature of rack A [42]

Grahic Jump Location
Fig. 23

Average Inlet temperature of rack A [42]

Grahic Jump Location
Fig. 24

Water and air increase pattern [48]

Grahic Jump Location
Fig. 25

(a) Average inlet temperature of row B racks [48] and (b) average inlet temperature of row D racks [48]

Grahic Jump Location
Fig. 26

(a) Average inlet temperature of row B racks [48] and (b) average inlet temperature of row D racks [48]

Grahic Jump Location
Fig. 27

Inlet temperature of rack D1 [48]

Grahic Jump Location
Fig. 28

Inlet temperature of rack D1 [48]

Grahic Jump Location
Fig. 29

Response time study results of rack B1 [48]

Grahic Jump Location
Fig. 30

Response time study results of rack B1 [48]

Grahic Jump Location
Fig. 31

Water and air increase pattern

Grahic Jump Location
Fig. 32

(a) Average inlet temperature of row A racks versus time for CRAC 1 and CRAC 4 failure combination and (b) average inlet temperature of row D racks versus time for CRAC 1 and CRAC 4 failure combination

Grahic Jump Location
Fig. 33

(a) Average inlet temperature of row A racks versus time for CRAC 2 and CRAC 3 failure combination and (b) average inlet temperature of row D racks versus time for CRAC 2 and CRAC 3 failure combination

Grahic Jump Location
Fig. 34

(a) Average inlet temperature of row A racks versus time for CRAC 3 and CRAC 4 failure combination and (b) average inlet temperature of row D racks versus time for CRAC 3 and CRAC 4 failure combination

Grahic Jump Location
Fig. 35

CRAC-less cooling solution design

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In