HOT CHIPS 2025: NVIDIA GB200 / GB300 NVL72 — Why an AI Rack Is No Longer "a Collection of Servers," Seen Through Mechanics and Cooling

Why NVIDIA Started with MGX Instead of Jumping Straight to GB200

In this case study, NVIDIA didn't open with GB200 or GB300. It started with the MGX architecture — which actually gets to the heart of the problem.

• AI systems are no longer a single-server design problem

• Every customer differs slightly in CPU, GPU, DPU, NIC and management approach

• Changing even one module ripples into a redesign of the entire system

• Repeated customization makes time-to-market spiral out of control

MGX's core goal is to modularize system complexity, so that changes are absorbed locally instead of starting over every generation.

The Key to MGX Isn't Modularity — It's Openness

MGX is not an NVIDIA-proprietary internal architecture; it was contributed directly to OCP. That is a pivotal design decision.

• Module interfaces, dimensions and specifications are fully public

• 3D models, 2D drawings and complete specification documents are released externally

• Customers and the supply chain can customize on the same common foundation

This makes GB200 / GB300 more than NVIDIA products — they become a system architecture the ecosystem can absorb.

NVL72 Isn't Just About Compute — It's 120 kW-Class Systems Engineering

At the system level, GB200 / GB300 NVL72 operates at an entirely different scale.

• About 120 kW of power per rack

• About 7 kW per compute tray

• About 1.4 exaFLOPS of FP4 compute per rack

• NVLink built around copper interconnect at 200 Gb/s per lane

This is no longer the traditional "servers + switch" mindset — it's closer to a small supercomputer.

Why NVIDIA Insists on Copper for Scale-Up

The NVLink spine uses high-density copper cabling — not out of conservatism, but as a deliberate engineering choice.

• Copper has low latency and predictable behavior

• Compared with optics, it's more reliable and cheaper

• Rack-level distances are still within its acceptable reach

This also explains why the rack's entire mechanical and structural design has to make way for large bundles of thick copper cable.

From OCP Open Rack V3 to NVIDIA's Custom Rack

NVIDIA's rack didn't start from scratch — it evolved from OCP Open Rack V3, but with many critical modifications along the way.

• Keeps rear blind-mate power and front-side service access

• Switches to 19-inch EIA pitch (44.5 mm) instead of OCP's 48 mm

• Raises node density while shortening NVLink cables by about 120 mm

• Improves high-speed signal integrity

What looks like a mere mechanical dimension decision directly affects signal quality and system reliability.

Power Forced the Whole Rack to "Grow"

The power density of GB200 / GB300 far exceeds what OCP originally designed for.

• The original OCP bus bar is rated at roughly 35 kW

• NVIDIA needs more than 1,400 A

• That requires a thicker copper bus bar with a larger cross-section

So NVIDIA designed a rear extension frame for the rack.

• Rack depth grows from 1,068 mm to about 1,200 mm

• Making room for the larger bus bar, liquid-cooling manifolds and the NVLink spine

This isn't optional — it's the inevitable result of physical limits.

Liquid Cooling Isn't the Hard Part — Blind-Mating Is

In NVL72, liquid cooling is a given; the real difficulty is making it serviceable and manufacturable.

• More than 40 quick disconnects (QDs) per rack

• About 40 lb of insertion force per QD

• About 6,500 lb of cumulative insertion force across the rack

This means:

• The manifold will bend

• The rack structure bears enormous internal forces

• With poor design, mating and unmating can't be completed reliably

Why the Manifold Must Be Anchored at the Center

NVIDIA very deliberately chose a center-anchored manifold design.

• Stainless steel expands by millimeters between 25°C and 75°C

• Thermal displacement over a 2.1-meter manifold can't be ignored

• Anchoring at one end would cause QD misalignment or jamming

Center anchoring splits the thermal-expansion displacement in half, ensuring long-term reliability.

The Floating NVLink Spine Is Built for Volume Production, Not for Show

The NVLink spine's floating connectors aren't there for easy insertion — they're there to absorb manufacturing tolerances.

• ±3 mm of float in X/Y

• About 4 mm of float in the insertion direction

• Automatically absorbs tolerance stack-up

This is a classic production-oriented design rather than a lab-oriented one.

6,500 lb Isn't a Metaphor — It's a Real Structural Load

When all QDs and NVLink connectors are mated at once:

• The middle of the rack bears a sustained internal force of about 6,500 lb

• Equivalent to parking an SUV in the middle of the rack

• A standard commercial rack would simply bend and fail

So NVIDIA:

• Added about 80 lb of steel reinforcement

• Designed front-to-back structural supports

• Ensured deflection stays below 0.1 mm at full load

Why Moving to 19-Inch EIA Matters for Both Cooling and Cabling

19-inch EIA isn't just about compatibility.

• Narrower trays free up more front cabling space

• Room for thicker high-speed copper cables

• Keeps cables from blocking tray extraction

This is crucial for both high-speed NVLink copper and service access.

Modular Design of Compute Trays and Switch Trays

In tray design, the MGX modular philosophy is fully realized.

• Compute tray: 2 CPUs + 4 GPUs

• Switch tray: reuses the same mechanics and interfaces

• Front IO area can be customized for management, NICs and SSDs

The rear is highly standardized:

• Same UQDs

• Same power interfaces

• Same structure and handles

Why Air Cooling Is Still Needed

Even in a heavily liquid-cooled system, air cooling remains indispensable.

• Cable cartridges present very high airflow impedance

• Some components still rely on air cooling

• NVIDIA perforated the trays, rails and brackets throughout

This creates lateral airflow channels, which:

• Lower fan power

• Reduce noise

• Improve overall cooling efficiency

A Major Shift in GB300: Fully Liquid-Cooled Switches

In GB300, NVIDIA goes further and fully liquid-cools the NVLink switches.

• Removes switch-tray fans

• Moves more of the power into the liquid-cooling loop

• Lowers the heat load inside the rack

This is an important step from GB200 toward higher power density.

NVLink Fusion Extends This Architecture Rather Than Starting Over

Finally, NVIDIA discussed NVLink Fusion.

• Lets non-NVIDIA accelerators plug into MGX too

• Turns NVL72 from a "product" into a "platform"

• Makes MGX an open scale-up infrastructure

Simple Tech and Trend's Take

This case study makes it clear that an AI rack is no longer IT equipment — it's heavy-industry-grade systems engineering.

• The challenge isn't compute; it's structure, heat, mating and volume production

• Success or failure hides in millimeters, newtons and watts

• NVL72 is a system that was "forced to take this shape"

The real innovation of GB200 / GB300 isn't the GPU — it's how it gets safely and reliably installed in data centers around the world.