A rack was once a unit of spatial design. You would count racks, measure the available square footage, ensure you had networking capability, and verify that your power budget was within normal parameters.
This paradigm does not hold for many computing environments today.
Machine learning training clusters, machine learning inference platforms, rendering farms, dense storage arrays, and high-performance computing systems can pack much more compute density per rack than traditional enterprise environments.
The aim is clear and straightforward: increase the density of every rack in the environment.
Start With the Workload, Not the Rack
The first mistake is designing around cabinet count before understanding what will run inside those cabinets.
A rack of general-purpose 1U servers has very different needs from a rack built around NVIDIA H100, H200, B200, or GB200 systems. GPU-heavy infrastructure changes the design’s shape because servers are not isolated pieces of hardware. They often depend on high-speed interconnects, predictable cooling paths, carefully planned power distribution, and enough physical space for larger cables, manifolds, and maintenance access.
NVIDIA’s GB200 NVL72 architecture, for example, is designed as a rack-scale system with 72 Blackwell GPUs and 36 Grace CPUs connected through NVLink, with cold plates mounted on CPUs and GPUs for direct liquid cooling. That is a useful example because it shows how modern computing is moving from “servers placed in a rack” toward “the rack as the system.”
Before choosing a facility, cabinet layout, or cooling method, define the workload profile:
What is the expected power draw per rack?
Will the environment support training, inference, HPC, storage, or mixed workloads?
Does the hardware require air cooling, rear-door heat exchangers, direct-to-chip liquid cooling, or immersion cooling?
How often will hardware be refreshed?
What network latency and bandwidth are required between nodes?
These answers prevent the design from being guesswork. They also help avoid a common problem: choosing a data center environment that looks suitable on paper but cannot support the thermal or electrical reality of the deployed equipment.
Plan Power as the Main Constraint
In the case of dense computing, floor space isn’t normally the most challenging constraint. It’s generally power.
Traditional enterprise racks were perhaps designed with 3-8 kW per rack in mind. AI and HPC racks can extend way past that. Some current racks are rated at 40 kW, 60 kW, 80 kW, or higher per rack, depending on GPU density. The conversation about the design of an AI data center now centers more on the power required to achieve greater rack density. This comes through clearly from the 2025 survey by Uptime Institute.
This is where high-density colocation becomes relevant. It is not just renting a cabinet in a facility. It uses a colocation environment built to support higher per-rack power draw, stronger cooling systems, resilient electrical distribution, and the operational practices needed for compact, power-intensive infrastructure.
A good power plan should account for more than just the server’s nameplate rating. It should include redundant feeds, usable capacity, breaker sizing, power distribution units, rack-level metering, and the likely impact of future hardware refreshes. If the design is already close to its limit on day one, it leaves little room for growth or unexpected workload changes.
For teams planning AI infrastructure, this often means working backward from the cluster design. If the intended deployment requires a specific number of GPUs, calculate the total power requirement, then translate that into rack count, cabinet density, redundancy model, and cooling requirement. This produces a more realistic footprint than starting with a preferred number of racks and trying to squeeze the workload into it later.
Match Cooling to the Heat Load
More computing in a smaller volume makes a simple physics problem – virtually all the electric energy used by the IT equipment gets transformed into heat.
Air cooling is well-suited to many situations, but it becomes increasingly difficult to rely solely on it as rack power increases. This is not only about the amount of heat produced. The question here is about where this heat appears, the rate at which it must be cooled, and whether the cooling system can do so consistently for each position in a rack.
The Uptime Institute 2025 survey on cooling shows that while adoption of liquid cooling continues to grow slowly, high rack density is a major motivator for evaluating this technology.
This is important because liquid cooling requires multiple approaches. They include direct chip cooling, rear-door heat exchangers, coolant distribution units, warm-water loops, or combinations of liquid and warm-water cooling technologies that also involve air. The Open Compute Project has even identified innovative cooling solutions, such as liquid and warm-water cooling, that efficiently dissipate heat at high power densities.
It depends on the type of equipment and operations.
If the servers are air-cooled, a hot-aisle/cold-aisle arrangement, blanking panels, proper air containment, and flow that does not involve recirculation are necessary. When servers are cooled with liquid, the facility will need to consider coolant distribution units, leak detection, maintenance, and personnel trained to handle liquid-cooled servers. Rear-door heat exchangers will involve racks with appropriate spacing, capacity, and water lines.
Cooling should be included in the design from the very beginning.
Design for Operations, Not Just Density
The densest layout is not always the best layout.
A rack that is difficult to cable, monitor, repair, or upgrade can become expensive even if it saves space. Dense compute environments need practical service access, clear labeling, structured cabling, accurate documentation, and real-time visibility into power and temperature conditions.
Examples of tools that can be used include Schneider Electric EcoStruxure IT, Sunbird DCIM, Nlyte, Device42, and Vertiv monitoring solutions. In fact, which solution is used is not critical – what is really important is the monitoring methodology. It should be clear which devices are used, how much capacity remains, which circuits feed which devices, and how the thermal environment changes under load.
Monitoring should also cover the rack level. It is possible that average parameters on the room level would fail to capture the hotspots. A data center could be OK while just one rack is on the verge of overheating. Power meters for racks, temperature sensors at the inlet, and alarms set to trigger at the operating threshold would be helpful in a crowded environment.
The cabling system should not be overlooked as well. High-Performance Computing and AI clusters use fast interconnection systems such as NVIDIA InfiniBand and 100G/200G/400G/800G Ethernet fabrics. Poor cable layout may interfere with airflow, servicing, and scaling. A properly developed cabling solution will ensure the safety of your systems.
Build in Room for the Next Hardware Cycle
The compute density is still on an upward curve. A data center that works for today’s installation may not work for tomorrow.
It does not imply that every company requires the latest cooling system right now. What it means is that the design should allow room for future upgrades. Is it possible to increase the power distribution? Are there provisions for the coolant distribution units? Can the floor support the extra weight of heavy racks? Are there proper pathways for larger cable runs or optical interconnects?
Good design not only optimizes the current footprint but also safeguards future choices.
It is, after all, an infrastructure coordination challenge to create more computing capabilities in less space. The rack, power delivery chain, cooling system, network structure, monitoring framework, and maintenance cycle must all fit within the intended workload.
The outcome is not just a compact solution but a stronger compute capability that can handle contemporary workloads while treating power and cooling as more than afterthoughts.
