0
Research Papers

Thermally Aware, Energy-Based Load Placement in Open-Aisle, Air-Cooled Data Centers

[+] Author and Article Information
Dustin W. Demetriou

Research Assistant
Mem. ASME
Department of Mechanical and
Aerospace Engineering
e-mail: dustin.demetriou@gmail.com

H. Ezzat Khalifa

Professor
Mem. ASME
Mechanical and Aerospace Engineering
e-mail: hekhalif@syr.edu
Syracuse University,
Syracuse, NY 12344

ψT is the ratio of the perforated tile airflow to the required rack airflow. The perforated tile airflow is the cold air that is supplied through perforated tile in the cold aisle; however, this is not the cold air ingested by the IT equipment. Typically, the ingested cold air is less because of unintentional “spilling” of cold air out of the cold aisle and/or short-circuiting of cold air the CRAH units [39].

Contributed by the Electronic and Photonic Packaging Division of ASME for publication in the Journal of Electronic Packaging. Manuscript received April 26, 2012; final manuscript received July 6, 2013; published online July 24, 2013. Assoc. Editor: Saurabh Shrivastava.

J. Electron. Packag 135(3), 030906 (Jul 24, 2013) (14 pages) Paper No: EP-12-1048; doi: 10.1115/1.4024946 History: Received April 26, 2012; Revised July 06, 2013

This paper introduces a methodology for developing a reduced order model, using proper orthogonal decomposition (POD), to predict the IT rack's inlet temperature distribution within a raised floor air-cooled data center. The method used in this paper uses a limited set of computational fluid dynamics data at different useful IT levels and tile airflow fractions. The model was able to reconstruct these datasets to with 0.16 °C rms error and interpolate successfully for alternative configurations that were not included in the original dataset. The reduced order model can produce the temperature distribution in the data center in a fraction of a second on a standard personal computer. Several practical IT load placement options in open-aisle, air-cooled data centers, based on either geometrical traits of the data center, a prior physics-based knowledge of the airflow and temperature patterns or measurements that are easily obtainable during operation, are considered. The outcome of this work is the development of a robust set of guidelines that facilitate the energy efficient placement of the IT load amongst the operating servers in the data center. This work found that a robust approach was to use real-time temperature measurements at the inlet of the racks to remove the unnecessary IT load from the servers with the warmest inlet temperature. This strategy shows superior performance to the other strategies studied. The study considered the holistic optimization of the data center and cooling infrastructure for a range of data center IT utilization levels. The results showed that allowing for significant reductions in the supply air flow rate proved superior to providing a higher supply air temperature to meet the IT equipment's inlet air temperature constraint.

FIGURES IN THIS ARTICLE
<>
Copyright © 2013 by ASME
Your Session has timed out. Please sign back in to continue.

References

United States Environmental Protection Agency, 2007, “Report to Congress on Data Center Energy Efficiency,” Public Law 109-431.
Salim, M., and Tozer, R., 2010, “Data Centers' Energy Auditing and Benchmarking-Progress Update,” ASHRAE Trans., 116(1), pp. 109–117.
Koomey, J., 2011, “Growth in Data Center Electricity Use 2005 to 2010,” A report by Analytical Press, completed at the request of The New York Times.
ASHRAE, 2011, “2011 Thermal Guidelines for Data Processing Environments—Expanded Data Center Classes and Usage Guidance,” American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta, GA.
Lui, Y., 2010, “Waterside and Airside Economizers Design Considerations for Data Center Facilities,” ASHRAE Trans., 116(1), pp. 98–108.
Schmidt, R., 2004, “Thermal Profile of a High-Density Data Center—Methodology to Thermally Characterize a Data Center,” ASHRAE Trans., 110(2), pp. 635–642.
Schmidt, R., Cruz, E., and Iyengar, M., 2005, “Challenges of Data Center Thermal Management,” IBM J. Res. Dev., 49, pp. 709–723. [CrossRef]
ASHRAE, 2008, “High Density Data Centers: Case Studies and Best Practices,” American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta, GA.
Shrivastava, S., Iyengar, M., Sammakia, B., Schmidt, R., and Vangilder, J., 2006, “Experimental-Numerical Comparison for a High Density Data Center: Hot Spot Heat Fluxes in Excess of 500 W/ft2,” Tenth Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronics Systems (ITHERM ’06), San Diego, CA, May 30–June 2, pp. 402–411. [CrossRef]
Patel, C., Bash, C., Belady, L., Stahl, L., and Sullivan, D., 2001, “Computational Fluid Dynamics Modeling of High Compute Density Data Centers to Assure System Inlet Specifications,” Pacific Rim/ASME International Electronic Packaging Technical Conference and Exhibition (IPACK ’01), Kauai, HI, July 8–13, ASME Paper No. IPACK2001-15622.
Patel, C., Sharma, C., Bash, C., and Beitelmal, A., 2002, “Thermal Considerations in Cooling Large Scale High Compute Density Data Centers,” Eighth Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITHERM 2002), San Diego, CA, May 30–June 1, pp. 767–776. [CrossRef]
Abdelmaksoud, W., Dang, T. Q., Khalifa, H. E., Elhadidi, B., Schmidt, R., and Iyengar, M., 2010, “Experimental and Computational Study of Perforated Floor Tile in Data Centers,” 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV, June 2–5.
Abdelmaksoud, W., Dang, T. Q., Khalifa, H. E., Elhadidi, B., Schmidt, R., and Iyengar, M., 2010, “Improved CFD Modeling of a Small Data Center Test Cell,” 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV, June 2–5. [CrossRef]
Iyengar, M., and Schmidt, R., 2007, “Analytical Modeling of Thermodynamic Characterization of Data Center Cooling Systems,” ASME J. Electron. Packag., 131, p. 021009. [CrossRef]
Hellmer, B., 2010, “Consumption Analysis of Teleco and Data Center Cooling and Humidification Options,” ASHRAE Trans., 116(1), pp. 118–133.
Demetriou, D. W., Khalifa, H. E., Iyengar, M., and Schmidt, R., 2011, “Development and Experimental Validation of a Thermo-Hydraulic Model for Data Centers,” HVAC&R Res., 17(4), pp. 540–555. [CrossRef]
Walsh, E., Breen, T., Punch, J., Shah, A., and Bash, C., 2010, “From Chip to Cooling Tower Data Center Modeling: Part I Influence of Server Inlet Temperature and Temperature Rise Across Cabinet,” 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV, June 2–5. [CrossRef]
Walsh, E., Breen, T., Punch, J., Shah, A., and Bash, C., 2010, “From Chip to Cooling Tower Data Center Modeling: Part II Influence of Chip Temperature Control Philosophy,” 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV, June 2–5. [CrossRef]
VanGilder, J., Zhang, X., and Shrivastava, S., 2007, “Partially Decoupled Aisle Method for Estimating Rack Cooling Performance in Near-Real Time,” Proceedings of ASME InterPACK, Vancouver, BC, Canada, July 8–12, ASME Paper No. IPACK2007-33447, pp. 781–789. [CrossRef]
Shrivastava, S., 2008, Cooling Analysis of Data Centers: CFD Modeling and Real-Time Calculators, Ph.D. thesis, Binghamton University, Binghamton, NY.
Healey, C., VanGilder, J., Sheffer, Z., and Zhang, X., 2011, “Potential-Flow Modeling for Data Center Applications,” Proceedings of ASME InterPACK, Portland, OR, July 6–8, ASME Paper No. IPACK2011-52136, pp. 527–534. [CrossRef]
Toulouse, M., Doljac, G., Carey, V., and Bash, C., 2009, “Exploration of a Potential-Flow-Based Compact Model of Air-Flow Transport in Data Centers,” ASME 2009 International Mechanical Engineering Congress and Exposition (IMECE), Orlando, FL, November 13–19, ASME Paper No. IMECE2009-10806, pp. 41–50. [CrossRef]
Yarlanki, S., Das, R., Hamann, H., Lopez, V., and Stepanchuk, A., 2011, “Employing Thermal Zones for Energy Optimization in Data Centers,” Proceedings of ASME InterPACK, Portland, OR, July 6–8, ASME Paper No. IPACK2011-52234, pp. 645–652. [CrossRef]
Lopez, V., and Hamann, H., 2011, “A Numerical Technique for the Approximation of Thermal Zones,” Proceedings of ASME InterPACK, Portland, OR, July 6–8, ASME Paper No. IPACK2011-52213, pp. 637–644. [CrossRef]
Tang, Q., Mukherjee, T., Gupta, S. K. S., and Crayton, P., 2006, “Sensor-Based Fast Thermal Evaluation Model for Energy Efficient High-Performance Datacenters,” 4th International Conference on Intelligent Sensing and Information Processing (ICISIP 2006), Bangalore, India, December 15–18, pp. 203–208. [CrossRef]
Moore, J., Chase, J. S., and Ranganathan, P., 2006b, “Weatherman: Automated, Online and Predictive Thermal Mapping and Management for Data Centers,” IEEE International Conference on Autonomic Computing (ICAC ’06), Dublin, Ireland, June 12–16, pp. 155–164. [CrossRef]
Moore, J., Chase, J., and Ranganathan, P., 2006a, “ConSil: Low-Cost Thermal Mapping of Data Centers,” Proceedings of SysML, Saint Malo, France, June 27.
Song, Z., Murray, B., and Sammakia, B., 2011, “Multivariate Prediction of Airflow and Temperature Distributions Using Artificial Neural Networks,” Proceedings of ASME InterPACK, Portland, OR, July 6–8, ASME Paper No. IPACK2011-52167, pp. 595–604. [CrossRef]
Elhadidi, B., and Khalifa, H. E., 2005, “Application of Proper Orthogonal Decomposition to Indoor Airflows,” ASHRAE Trans., 111(1), pp. 625–634.
Khalifa, H. E., Elhadidi, B., and Dannenhoffer, J., 2007, “Efficient Coupling of Multizone and CFD Indoor Flow Models Through Proper Orthogonal Decomposition,” ASHRAE Trans., 113(1), pp. 282–289.
Samadiani, E., Joshi, Y., Hamann, H., Iyengar, M., Kamalsy, S., and Lacey, J., 2009, “Reduced Order Thermal Modeling of Data Centers Via Distributed Sensor Data,” Proceedings of ASME InterPACK, San Francisco, CA, July 19–23, ASME Paper No. InterPACK2009-89187, pp. 807–814. [CrossRef]
Sharma, R., Bash, C., Patel, C., Friedrich, R., and Chase, J., 2005, “Balance of Power: Dynamic Thermal Management for Internet Data Centers,” Internet Comput., 9(1), pp. 42–49. [CrossRef]
Moore, J., Chase, J., Ranganathan, P., and Sharma, R., 2005, “Making Scheduling ‘Cool’: Temperature-Aware Workload Placement in Data Centers,” Proceedings of USENIX Annual Technical Conference, Anaheim, CA, April 10–15, pp. 61–75.
Tang, Q., Gupta, S. K. S., and Varsamopoulou, G., 2007, “Thermal-Aware Task Scheduling for Data Centers Through Minimizing Heat Recirculation,” IEEE International Conference on Cluster Computing, Austin, TX, September 17–21, pp. 129–138. [CrossRef]
Tang, Q., 2009, “Thermally-Aware Scheduling in Environmentally Coupled Cyber-Physical Distributed Systems,” Ph.D. thesis, Arizona State University, Tempe, AZ.
Moore, J., 2005, “Automated Cost-Aware Data Center Management,” Ph.D. thesis, Duke University, Durham, NC.
Tang, Q., Gupta, S., and Varsamopoulos, G., 2008, “Energy-Efficient Thermal Aware Task Scheduling for Homogenous High-Performance Computing Data Centers: A Cyber-Physical Approach,” IEEE Trans. Parallel Distrib. Sys., 19(11), pp. 1458–1472. [CrossRef]
Mukherjee, T., Banerjee, A., Varsamopoulos, G., and Gupta, S., 2009, “Spatio-Temporal Thermal-Aware Job Scheduling to Minimize Energy Consumption in Virtualized Heterogeneous Data Centers,” Comput. Netw., 53(17), pp. 2888–2904. [CrossRef]
Demetriou, D. W., and Khalifa, H. E., 2011, “Evaluation of a Data Center Recirculation Non-Uniformity Metric Using Computational Fluid Dynamics,” Proceedings of ASME InterPACK, Portland, OR, July 6–8, ASME Paper No. IPACK2011–52005, pp. 405–414. [CrossRef]
Khalifa, H. E., and Demetriou, D. W., 2011, “Energy Optimization of Air-Cooled Data Centers,” ASME J. Therm. Sci. Eng. Appl., 2, p. 041005. [CrossRef]
Demetriou, D. W., and Khalifa, H. E., 2011, “Energy Modeling of Air-Cooled Data Centers: Part I—The Optimization of Enclosed Aisle Configurations,” Proceedings of ASME InterPACK, Portland, OR, July 6–8, ASME Paper No. IPACK2011-52003, pp. 385–394. [CrossRef]
Demetriou, D. W., and Khalifa, H. E., 2011, “Energy Modeling of Air-Cooled Data Centers: Part II—The Effect of Recirculation on the Energy Optimization of Open-Aisle, Air-Cooled Data Centers,” Proceedings of ASME InterPACK, Portland, OR, July 6–8, ASME Paper No. IPACK2011-52004, pp. 395–404. [CrossRef]
Holmes, P., Lumley, J., and Berkooz, G., 1996, “Turbulence, Coherent Structures, Dynamical Systems and Symmetry,” Cambridge Monographs on Mechanics, Cambridge University Press, Cambridge, UK.
Rambo, J., and Joshi, Y., 2005, “Reduced Order Modeling of Steady Turbulent Flows Using the POD,” Proceedings of ASME Summer Heat Transfer Conference, San Francisco, CA, July 17–22, ASME Paper No. HT2005-72143, pp. 837–846. [CrossRef]
Lumley, J. L., 1981, “Coherent Structures in Turbulence,” Transition and Turbulence: Proceedings of the Symposium on Transition and Turbulence in Fluids, Madison, WI, October 13–15, 1980, Academic Press, New York, pp. 215–242.
Sirovich, L., 1987, “Turbulence and the Dynamics of Coherent Structures, Part 1: Coherent Structures,” Quarterly of Applied Mathematics, 45(3), pp. 561–571.
Rasmussen, N., and Torell, W., 2007, “Data Center Projects: Establishing a Floor Plan,” American Power Conversion White Paper No.144.
Ansys Inc., 2010, “ANSYS FLUENT Theory Guide,” Release 13.
VanGilder, J., and Zhang, X., 2008, “Coarse-Grid CFD: The Effect of Grid Size on Data Center Modeling,” ASHRAE Trans., 114(2), pp. 166–181.
Sirovich, L., and Everson, R., 1992, “Management and Analysis of Large Scientific Datasets,” Int. J. High Performan. Supercomput. Appl., 6(1), pp. 50–68. [CrossRef]
Bui-Thanh, T., Damodaran, M., and Willcox, K., 2003, “Proper Orthogonal Decomposition Extensions for Parametric Applications in Transonic Aerodynamics,” Proceedings of 21st AIAA Applied Aerodynamics Dynamics Conference, Orlando, FL, June 23–26, AIAA Paper No. 2003–4213. [CrossRef]
Incropera, F., and DeWitt, D., 2002, Fundamentals of Heat and Mass Transfer, 5th ed., Wiley, New York.
Braun, J. E., Mitchell, J. W., and Klein, S. A., 1987, “Performance and Control Characteristics of a Large Cooling System,” ASHRAE Trans., 93(1), pp. 1830–1852.

Figures

Grahic Jump Location
Fig. 1

Computational geometry

Grahic Jump Location
Fig. 2

(a) Reconstruction error for 50% useful IT and (b) POD reconstruction comparison for different snapshots using 12 modes

Grahic Jump Location
Fig. 3

Mean of the rack's inlet temperature distribution from the 35 snapshots and the temperature reconstruction contours as the number of modes is increased

Grahic Jump Location
Fig. 4

(a) Example error plots (rms and Max) for interpolated datasets and (b) POD interpolation comparison

Grahic Jump Location
Fig. 5

Schematic of typical air-cooled data center cooling infrastructure

Grahic Jump Location
Fig. 6

Optimization flowchart

Grahic Jump Location
Fig. 7

Optimization results for 100% useful IT

Grahic Jump Location
Fig. 8

Proposed load placement scenarios for 75% useful IT

Grahic Jump Location
Fig. 9

Optimization results for different scenarios at 75% useful IT

Grahic Jump Location
Fig. 10

Summary of load placement analysis

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In