Kao Data and Cadence

Kao Data uses digital twin simulation to validate complex indirect evaporative cooling design

Kao Data and Cadence Cover Image

The Highlights

Application

  • Colocation data center design and operations

Software

  • Cadence Reality DC Design

Benefits

  • Refine data center cooling design
  • Maximize operational efficiency
  • Validate proposed design concepts
  • Ensure informed decision making

Key Challenges

When Kao Data launched KDL1—the first of four planned data centers on a 36-acre campus—it was the UK’s first 100% free-cooling wholesale colocation facility. The data center campus is designed to support high performance computing (HPC) and intensive artificial intelligence (AI) requirements, but not at the expense of the environment. In fact, environmental performance is at the very heart of the data center’s design with a brief to ensure the campus is as energy efficient as possible.

With ambitions to cater to not only the increasingly high-density deployments characterized by machine learning and deep learning workloads, but also offer the ultimate in Open Compute Ready (OCP) infrastructure, the Kao Data campus will be split into four halls totaling around 150,000 square feet of technical space.

This space is engineered from the ground-up to specifically cater to advanced, GPU-powered supercomputing with Technology Suites incorporating slab-flooring, no-column data halls, and wide access corridors. Each data center will offer a total technical load of 8.8MW, accommodating rack densities of 20kW and beyond, in line with customer expectations.

Kao Data boasts an impressive 100% uptime track record ensuring power resiliency via an on-site substation. This uses 43.5MVA supplied by UK power networks (UKPN) national grid via a dual feed to the data center facilities. When needed, backup power generation is provided by generators configured with N+1 redundancy to satisfy the toughest client SLAs.

Under normal operating conditions, the entire campus is powered by 100% certified, renewable energy, and the architecture has received the Building Research Establishment’s (BRE) BREEAM excellence certification in construction for building energy efficiency, environmental, and sustainability competency.

IT operations boasts a PUE of 1.2, even at partial loads, and its non-mechanical cooling design eliminates the need for harmful refrigerant-based systems, ensuring it meets the ASHRAE TC 9.9 environmental guidelines.

Due to the demanding power and cooling requirements of GPU-powered systems, it is essential that Kao Data’s campus is as sustainable as possible. This requires a scientifically backed methodology that can help its team pinpoint operational efficiency and achieve such ambitious design specifications. To ensure an energy-efficient design, Kao Data considered an indirect evaporative cooling (IEC) system, which uses water evaporation in place of mechanical systems to cool the air.

The Solution

Kao Data investigated the IEC design to create a data center that was greener, as well as more efficient and cost-effective to run. Cadence worked alongside JCA Engineering and Kao Data throughout this project, providing computational fluid dynamics (CFD) simulations of both the external and internal environments of the data center campus using Cadence Reality DC Design (formerly part of Future Facilities 6SigmaDCX product line). The CFD simulations empowered Kao Data to make informed decisions around design, implementation, and operations. Ongoing simulations were used throughout the project to inform the order of the build-out sequence to keep operational risks at a minimum.

A data center digital twin model of the campus was created and CFD simulation was used to:

  • Evaluate the viability and performance of the proposed IEC design concept
  • Confirm satisfactory cooling performance of each of the proposed build-out stages
  • Demonstrate the effect of potential varying and extreme environmental conditions
  • Verify that predicted IEC inlet conditions for a variety of external conditions were acceptable and should not compromise IEC performance

External modeling was performed using physics-based simulation to evaluate cooling performance under different weather scenarios using a range of configurations. It is important to note that IECs are a type of evaporative cooling solution. The air that exhausts back out to the environment is much more humid due to the water addition. This means that a critical component of the IEC design is adequate segregation of the intake and exhaust of the external air path to avoid moist air being drawn back into the facility.

As such, it is important to assess the risk of recirculation between the exhausts and intakes. This is where the power of CFD is best utilized. A CFD simulation study of the external site enabled Kao Data to understand:

  • The thermal impact of the overall design
  • How this could be affected by different weather conditions
  • How the different stages of site build-out could affect performance

Engineers at JCA and Cadence assessed historical ASHRAE weather data for the area to establish conditions that were most likely to be risky. This allowed for a clear project definition involving the simulation of a reasonable number of scenarios including different wind speeds and directions. CFD simulations were used to map different external environmental scenarios, considering the effect of building obstructions on air intake and any potential difficulties, such as exhaust air mixing and raising the intake wet bulb (WB) temperature above the specified limits for the IEC units.

The simulations were used to identify risks, enabling the final design to take into consideration the placement of cooling chillers and how airflow might be restricted by placing them too close together or by issues such as louvres around the units.

CFD modeling also played an important part in whitespace configuration. Cadence Reality DC Design was used to verify the cooling system worked effectively at the design supply air temperature. It was crucial that airflow was distributed evenly throughout the data center and IT cooling availability was unaffected by location. As well as validating thermal performance under normal operating conditions, CFD simulation was used to verify that the design would cope in a failure scenario – especially in terms of the IT equipment located near the failed cooling unit.

Summary

Wholesale and high-performance colocation provider Kao Data worked with Cadence to validate its data center IEC design using Cadence Reality DC Design. Kao Data designed a high-performance data center which did not use any mechanical refrigeration. This was unprecedented when Kao Data’s first Technology Suite became operational.

The result was a high efficiency, industrial scale data center with a PUE of 1.2 even at low IT utilization, which gives long-term OPEX savings for both Kao Data and its customers. The design also allowed Kao Data to be flexible with how the space is filled. Unlike most conventional data centers, at 50% load the PUE was only 1.14, and for a wholesale colocation data hall this is very impressive.

Kao Data continues to work with Cadence to model complex designs to support enterprise, HPC, and AI customers.


 

Energy efficiency has always been one of the key pillars of our data center strategy, and with a PUE of 1.2 at both high and low IT utilization, it remains one of the primary differentiators of our campus. We chose to work with Cadence due to their industry-leading reputation for CFD analysis, which was pivotal in helping us to design and build one of the most sustainable and energy efficient facilities in the UK.
Paul Finch, Kao Data