HEAPY designed multiple Mechanical-Electrical system upgrades to the infrastructure within Indiana University’s Data Center. These projects have included multiple capacity expansions for various Enterprise and Research (HPC, High Performance Computing) Compute needs.
Initially, this work began with modifications to support the relocation of their Research Supercomputer, Big Red, from a separate existing building.
The original Big Red was one of the most powerful university-owned computers in the US, and one of the 50 fastest supercomputers in the world. Part of a comprehensive strategy to build an advanced cyberinfrastructure to support research at Indiana University, Big Red has a theoretical peak performance of more than 30 teraFLOPS, and has achieved more than 21 teraFLOPS on numerical computations.
In 2013, HEAPY completed a project to replace this supercomputer with the new, 25 times faster Big Red II, which was the first announced system capable of at least 1 petaFLOP that is owned by and operated solely for the benefit of a single university. As the Hoosier high performance computing community likes to explain, if you performed one calculation per second with a hand calculator, it would take you 31.7 million years to do what a petaflop supercomputer like the Cray Big Red II can do in a second!
In 2019 and 2020, HEAPY completed a project that has seen the University’s research supercomputer taken to the next level with Big Red 3 and Big Red 200. Big Red 3 is dedicated to reseachers, scholars, and artists with large-scale, compute-intensive applications that can take advantage of the system’s extreme processing capability and high-bandwidth network topology. It contains 22,464 compute cores and is capable of 934 teraFLOPS.
Big Red 200 is a supercomputer designed to support scientific and medical research, and advanced research in artificial intelligence, machine learning, and data analytics. Big Red 200 will have a theoretical peak performance greater than 6 petaFLOPS.
The mechanical infrastructure utilizes multiple concepts for cooling the various systems in the two data center pods. Upgrades have included a large, built-up cooling distribution system to circulate cooling water to the high-density rack mounted cooling units, including an energy-efficient direct to chip liquid cooling system. Additionally, some racks are equipped with rack direct, high-density cooling units and utilize hot-aisle/cold-aisle cooling strategy. HEAPY has completed full CFD (Computational Fluid Dynamics) modeling of the data center rack layouts to ensure proper thermal and airflow performance of the high-density racks in the modified rack configuration.
The electrical infrastructure includes multiple UPS systems based on both battery (lead-acid and Lithium-lon) back-up and flywheel technology supporting dual cord power distribution for the rack systems. Additionally, the power generation system has been significantly upgraded with additional capacity and resiliency.