Research Papers

Single- and Two-Phase Particle Image Velocimetry Characterization of Fluid Flow Within a Liquid Immersion Cooled Electronics Module

[+] Author and Article Information
Joshua Gess

Mechanical, Industrial and Manufacturing
Engineering (MIME) Department,
Oregon State University,
113 Dearborn Hall,
Corvallis, OR 97331
e-mail: joshua.gess@oregonstate.edu

Sushil Bhavnani

Fellow ASME
Mechanical Engineering Department,
Auburn University,
1418 Wiggins Hall,
Auburn, AL 36849
e-mail: bhavnsh@auburn.edu

R. Wayne Johnson

Electrical and Computer Engineering Department,
Tennessee Tech University,
115 W. 10th Street,
Cookeville, TN 38505
e-mail: wjohnson@tntech.edu

Contributed by the Electronic and Photonic Packaging Division of ASME for publication in the JOURNAL OF ELECTRONIC PACKAGING. Manuscript received March 19, 2016; final manuscript received September 27, 2016; published online October 20, 2016. Assoc. Editor: Ashish Gupta.

J. Electron. Packag 138(4), 041007 (Oct 20, 2016) (11 pages) Paper No: EP-16-1051; doi: 10.1115/1.4034854 History: Received March 19, 2016; Revised September 27, 2016

With the growth and acceptance of liquid immersion cooling as a viable thermal management technique for high performance microelectronics, fundamental questions regarding the nature of the flow within the system will need to be addressed. Among these are how the coolant is directed toward components of primary interest as well as how other elements within the electronics package may affect the delivery of fluid to these more critical locations. The proposed study seeks to experimentally address these issues with particle image velocimetry (PIV) measurements of unheated and heated flow within an electronics enclosure. The effectiveness of three flow distribution designs at delivering coolant to elements of importance, in this case 6.45 cm2 (1 in.2) components meant to simulate processor chips, has been examined using the vectors yielded by the PIV experimentation in a control surface analysis around these critical components. While these previous scenarios are unheated, two-phase PIV has also been conducted with FC-72 as the working fluid while boiling is taking place. A control surface analysis around all four heated elements within the enclosure shows an expected roughly monotonic increase in the net liquid flow rate to the boiling elements as the power applied to them is increased. Additionally, discretized mapping of how the fluid is entering the area surrounding these boiling elements has been constructed to offer insight into how passive elements should be placed within an electronics enclosure so as to not obstruct or hinder the vital flow of coolant to the most critical components.

Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.


Markoff, J. , 2015, “ IBM Discloses Working Version of a Much Higher-Capacity Chip,” New York Times, New York, accessed on Oct. 15, 2015, http://www.nytimes.com/2015/07/09/technology/ibm-announces-computer-chips-more-powerful-than-any-in-existence.html?_r=0
Intel, 2015, “ Intel 14 nm Technology,” Intel, Santa Clara, CA, accessed on Oct. 15, 2015, http://www.intel.com/content/www/us/en/silicon-innovations/intel-14nm-technology.html
Bar-Cohen, A. , 2013, “ Needs and Future Trends for Enhanced Phase Change Heat Transfer-DARPA,” International Workshop on Micro- and Nanostructures for Phase Change Heat Transfer, Dedham, MA, Apr. 22–23.
Nakayama, W. , 2014, “ Heat in Computers: Applied Heat Transfer in Information Technology,” ASME J. Heat Transfer, 136(1), p. 013001. [CrossRef]
Suszko, A. , 2015, “ Enhancement of Nucleate Boiling on Rough and Dimpled Surfaces With Application to Composite Spreaders for Microprocessors Immersion Cooling,” Ph.D. dissertation, Department of Mechanical Engineering, University of New Mexico, Albuquerque, NM.
Khattar, M. K. , 2014, “ Free Cooling for Data Center,” ASHRAE J., 56(10), pp. 60–68.
Energy Star Program, 2007, “ Report to Congress on Server and Data Center Energy Efficiency Public Law 109-431,” Environmental Protection Agency, Washington, DC, Report No. LBNL 363E.
Gess, J. , Bhavnani, S. , and Johnson, R. W. , 2015, “ Experimental Investigation of a Direct Liquid Immersion Cooled Prototype for High Performance Electronic Systems,” IEEE Trans. Compon. Packag. Manuf. Technol., 5(10), pp. 1451–1464. [CrossRef]
Simons, R. E. , 1996, “ Direct Liquid Immersion Cooling for High Power Density Microelectronics,” Electron. Cool., epub.
Iceotope, 2014, “ New Cooling Technique Supercharges IT Performance, Reveals Iceotope Study,”BCW, Litchfield, UK, accessed on Oct. 20, 2015, http://www.businesscomputingworld.co.uk/news/pressrelease/new-cooling-technique-supercharges-it-performance-reveals-iceotope-study/
Chi, Y. Q. , Summers, J. , Hopton, P. , Deakin, K. , Real, A. , Kapur, N. , and Thompson, H. , 2014, “ Case Study of a Data Centre Using Enclosed, Immersed, Direct Liquid-Cooled Servers,” Semiconductor Thermal Measurement and Management Symposium (SEMI-THERM), San Jose, CA, Mar. 9–13, pp. 164–173.
Becker, A. , 2013, “ Bitcoin Mining Boosts Interest in Liquid Cooling,” Electron. Cool., epub.
Miller, R. , 2014, “ New From 3M: Boiling Liquid to Cool Your Servers,” Data Center Knowledge, West Chester, OH, http://www.datacenterknowledge.com/archives/2014/04/09/new-3m-boiling-liquid-cool-servers/
DatacenterDynamics, 2014, “ Is the Industry Getting Better at Using Power?,” Focus, San Francisco, CA, http://content.yudu.com/Library/A2nvau/FocusVolume3issue33/resources/index.htm?referrerUrl=
Garimella, S. V. , Persoons, T. , Weibel, J. , and Yeh, L.-T. , 2013, “ Technology Drivers in Data Centers and Telecom Systems: Multiscale Thermal, Electrical and Energy Management,” Appl. Energy, 107, pp. 66–80. [CrossRef]
Kheirabadi, A. C. , and Groulx, D. , 2016, “ Cooling of Server Electronics: A Design Review of Existing Technology,” Appl. Therm. Eng., 105, pp. 622–638. [CrossRef]
Alkharabsheh, S. , Fernandes, J. , Gebrehiwot, B. , Agonafer, D. , Ghose, K. , Ortega, A. , Joshi, Y. , and Sammakia, B. , 2015, “ A Brief Overview of Recent Developments in Thermal Management in Data Centers,” ASME J. Electron. Packag., 137(4), p. 040801. [CrossRef]
Bhopte, S. , Agonafer, D. , Schmidt, R. , and Sammakia, B. , 2006, “ Optimization of Data Center Room Layout to Minimize Rack Inlet Air Temperature,” ASME J. Electron. Packag., 128(4), pp. 380–387. [CrossRef]
Arghode, V. K. , Kumar, P. , Joshi, Y. , Weiss, T. , and Meyer, G. , 2013, “ Rack Level Modeling of Air Flow Through Perforated Tile in a Data Center,” ASME J. Electron. Packag., 135(3), p. 030902. [CrossRef]
Erden, H. S. , Khalifa, H. E. , and Schmidt, R. R. , 2014, “ Determination of the Lumped-Capacitance Parameters of Air-Cooled Servers Through Air Temperature Measurements,” ASME J. Electron. Packag., 136(3), p. 031005. [CrossRef]
Arghode, V. K. , and Joshi, Y. , 2015, “ Measurement of Air Flow Rate Sensitivity to the Differential Pressure Across a Server Rack in a Data Center,” ASME J. Electron. Packag., 137(4), p. 041002. [CrossRef]
Alkharabsheh, S. , Sammakia, B. , Murray, B. , Shrivastava, S. , and Schmidt, R. , 2014, “ Experimental Characterization of Pressure Drop in a Server Rack,” 14th Intersociety on Thermal and Thermomechanical Phenomena (ITHERM), Orlando, FL, May 27–30, pp. 547–556.
Green, C. , Kottke, P. , Han, X. , Woodrum, C. , Sarvey, T. , Asrar, P. , Zhang, X. , Joshi, Y. , Federov, A. , Sitaraman, S. , and Bakir, M. , 2015, “ A Review of Two-Phased Forced Cooling in Three-Dimensional Stacked Electronics: Technology Integration,” ASME J. Electron. Packag., 137(4), p. 040802. [CrossRef]
Sathe, M. J. , Thaker, I. J. , Strand, T. E. , and Joshi, J. B. , 2010, “ Advanced PIV/LIF and Shadowgraphy System to Visualize Flow Structure in Two-Phase Bubbly Flows,” Chem. Eng. Sci., 65(8), pp. 2431–2442. [CrossRef]
Lindken, R. , and Merzkirch, W. , 2002, “ A Novel PIV Technique for Measurements in Multiphase Flows and Its Application to Two-Phase Bubbly Flows,” Exp. Fluids, 33(6), pp. 814–825. [CrossRef]
Bröder, D. , and Sommerfeld, M. , 2003, “ Combined PIV/PTV-Measurements for the Analysis of Bubble Interactions and Coalescence in a Turbulent Flow,” Can. J. Chem. Eng., 81(3–4), pp. 756–763.
Tokuhiro, A. , Maekawa, M. , Iizuka, K. , Hishida, K. , and Maeda, M. , 1998, “ Turbulent Flow Past a Bubble and an Ellipsoid Using Shadow-Image and PIV Techniques,” Int. J. Multiphase Flow, 24(8), pp. 1383–1406. [CrossRef]
Teodori, E. , Moita, A. S. , Teodori, E. , and Moreira, A. L. N. , 2013, “ Evaluation of Pool Boiling Heat Transfer Over Micro-Structured Surfaces by Combining High-Speed Visualization and PIV Measurements,” 10th International Symposium on Particle Image Velocimetry (PIV13), Delft, The Netherlands, July 1–3.
Sridhar, A. , 2012, “ Experimental Evaluation of Immersion-Cooled Strategies for High Powered Server Modules,” M.S. thesis, Mechanical Engineering, Auburn University, Auburn, AL.
Gess, J. , Bhavnani, S. , Ramakrishnan, B. , Johnson, R. W. , Harris, D. , Knight, R. W. , Hamilton, M. , and Ellis, C. , 2014, “ Investigation and Characterization of a High Performance, Small Form Factor, Modular Liquid Immersion Cooled Server Model,” Thirtieth Symposium on Semiconductor Thermal Measurement and Management (SEMI-THERM), San Jose, CA, Mar. 9–13, pp. 8–16.
Gess, J. , Bhavnani, S. , Ramakrishnan, B. , Johnson, R. W. , Harris, D. , Knight, R. W. , Hamilton, M. , and Ellis, C. , 2014, “ Impact of Surface Enhancements Upon Boiling Heat Transfer in a Liquid Immersion Cooled High Performance Small Form Factor Server Model,” Fourteenth Intersociety on Thermal and Thermomechanical Phenomena (ITHERM), Orlando, FL, May 27–30, pp. 435–443.
Fincher, S. , 2014, “ Numerical Simulations of Boiling in Dielectric Fluid Immersion Cooling Scenarios of High Power Electronics,” M.S. thesis, Mechanical Engineering, Auburn University, Auburn, AL.
Intel, 2014, “ Intel® Xeon® Processor E7-8893 v2 (37.5 M Cache, 3.40 GHz),” Intel, Santa Clara, CA, accessed on Oct. 15, 2015, http://ark.intel.com/products/75260


Grahic Jump Location
Fig. 1

Process flow diagram (PFD) of experimental facility

Grahic Jump Location
Fig. 2

Flow distribution designs tested: (a) tube inlet, (b) flow distributor, and (c) flow guide

Grahic Jump Location
Fig. 3

Sample raw data PIV image of fluorescent particles seeded in FC-72 under the maximum power dissipation tested. Light reflected from the vapor, generated by the four square primary die outlined above, is not visible (filtered out) relative to the fluorescent light signature given by the particles distributed throughout the liquid phase.

Grahic Jump Location
Fig. 4

Sample PIV vector map from the single-phase experiments conducted. Vectors of interest for the effectiveness analysis conducted for all three flow distribution design conditions are highlighted, residing over the four primary elements within the module outlined in black. These vectors of interest are then resolved into a single vector, shown in the center of each square element, along with its x- and y-components, shown on the bottom left square element as an illustration.

Grahic Jump Location
Fig. 5

Illustration showing how the x- and y-components from the resolved average vector residing over each primary element, shown in Fig. 4, are used in the control surface analysis to determine the coolant mass flow rate into the region of interest

Grahic Jump Location
Fig. 6

Results of the coolant distribution effectiveness single-phase analysis conducted over all three flow distribution designs. The significant increase in effectiveness of the flow distributor design over the tube inlet design is shown by the diamond-shaped data. The modest increase in effectiveness of the flow guide design over the flow distributor design is shown by the triangle-shaped data. The solid lines are the average values yielded for each scenario over all of the flow rates tested.

Grahic Jump Location
Fig. 7

Sample PIV vector maps from all three flow distribution designs examined: (a) Tube inlet design vector map, (b) flow distributor design vector map, and (c) flow guide design vector map

Grahic Jump Location
Fig. 8

Illustration of the control surface proposed for the two-phase PIV analysis conducted. The surface area used in conjunction with the vectors around the periphery of the map yielded is shown. The filtered area, chosen as it was determined that this area consists primarily of vapor across the power dissipations tested, is shown by the small regions above each column of the heated component array.

Grahic Jump Location
Fig. 9

Illustration of the vectors, highlighted above, used in conjunction with the control surface illustrated by Fig. 8 to determine the net liquid mass flow rate entering the area surrounding the four primary elements within the electronics enclosure

Grahic Jump Location
Fig. 10

Results of the net incoming coolant mass flow rate analysis as a function of power applied uniformly to the four primary elements within the electronics enclosure. A roughly monotonic increase in liquid coolant mass flow rate to the boiling surfaces as power applied increases is on display. This is expected as an increase in power applied should increase the boiling activity, consequently increasing the demand for quenching fluid to these surfaces.

Grahic Jump Location
Fig. 11

Maps showing variation of vector set used for mass flow rate control surface analysis to show independence of area used. Periphery vector colors correspond to those shown in Fig. 10: (a) −79 × 52 array, (b) −74 × 52 array, (c) −69 × 52 array, and (d) −64 × 52 array.

Grahic Jump Location
Fig. 12

Discretized coolant flow map showing how fluid is entering the area surrounding the critical boiling elements within the electronics enclosure. Maps such as this can provide important insight to the packaging engineer for determination of where to place passive and slightly heated elements within the enclosure in support of the more computationally taxed components of the overall system architecture, which are typically power dense enough to sustain boiling heat transfer.



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In