Hot Chips 2025: Why NVIDIA Had to Take Optical Interconnect to a Co-Packaged Silicon Photonics Switch

Oct 5

3 min read

NVIDIA's problem is no longer single-chip performance

In its Hot Chips 2025 talk, NVIDIA delivered a clear message: the problem it is solving now is not how fast a GPU is, but whether the entire AI system can run efficiently.

Single-GPU compute keeps rising, but system-level performance gains are slowing
Large models require thousands or even tens of thousands of GPUs working together
The system bottleneck is shifting to the interconnect between GPUs
Latency, power, and scalability have become the deciding factors

Against this backdrop, the interconnect is no longer a supporting player but the core of system design.

Why traditional pluggable optics are no longer enough

Pluggable optics successfully supported data center growth for more than a decade, but in hyperscale AI systems their limits are becoming obvious.

Optical modules sit at the front panel, so electrical traces are long
High-speed SerDes and retimer circuits add extra power
Front-panel density caps total bandwidth scaling
System latency and jitter are hard to control precisely

For NVIDIA, these problems are magnified enormously in AI Factory-scale systems.

Why Co-Packaged Optics became the inevitable choice

NVIDIA's decision to co-package optics directly with the switch ASIC is not about showing off technology; it is about removing a system-level bottleneck.

The distance between optical TX/RX and the switch ASIC shrinks dramatically
Shorter electrical traces cut both power and signal loss
The front panel no longer limits bandwidth scaling
System latency and behavior become more predictable

The value of Co-Packaged Optics lies not in the speed of any single link, but in overall system efficiency.

Why NVIDIA puts special emphasis on Silicon Photonics

Within its co-packaged architecture, NVIDIA explicitly chose silicon photonics as the core technology rather than other optical integration approaches.

Silicon photonics offers high integration and scalability
It can be tightly integrated with advanced packaging flows
It suits high-volume manufacturing and consistency control
It is highly compatible with the existing semiconductor supply chain

For NVIDIA, this is not merely a technology choice but a choice about supply chain and manufacturability at scale.

Why the laser is not co-packaged on the switch

One notable design choice: NVIDIA did not package the laser right next to the switch ASIC.

Laser reliability and lifetime management differ from those of an ASIC
Repair and replacement needs are closely tied to system availability
Laser thermal management conflicts with a high-power ASIC

NVIDIA therefore adopted an architecture with external, replaceable lasers.

Preserves system serviceability
Reduces thermal risk inside the package
Strikes a balance between reliability and integration

This is a highly system-driven decision, not a purely technology-driven one.

The role of the co-packaged switch in AI scale-up

For NVIDIA, the Co-Packaged Silicon Photonics Switch does not replace the entire network architecture; it serves a specific tier.

Used mainly for rack-level and pod-level scale-up
Supports high-bandwidth, low-latency GPU-to-GPU interconnect
Reduces reliance on multi-tier electrical switching

This lets many GPUs operate closer to a "single giant accelerator."

A clear division of labor: scale-up, scale-out, and scale-across

In the talk, NVIDIA clearly distinguished three ways to scale a system.

Scale-up: maximize bandwidth and synchronization within the shortest physical distance
Scale-out: flexible expansion within the data center
Scale-across: spanning data centers and wider geographic areas

Co-packaged silicon photonics mainly serves scale-up and some scale-out scenarios, rather than replacing the whole network.

Why only NVIDIA can push this path forward

Adopting Co-Packaged Optics is extremely risky for most system vendors, but NVIDIA has unique advantages.

It controls the GPU, the switch ASIC, and the system design
It can optimize holistically from chip to package to system
It has enough shipment volume to absorb the risk of adopting new technology
It can steer the design direction of the entire ecosystem

This makes NVIDIA one of the few companies able to bring CPO into real deployment.

The Simple Tech Trend view

NVIDIA's talk made one thing clear: optical interconnect is no longer a future research direction but a requirement of today's system design.

When systems scale up to AI Factory level
When interconnect becomes the core factor in performance, power, and cost
Co-packaged silicon photonics stops being an option and becomes a tool

This Hot Chips 2025 talk marks optical interconnect formally taking center stage in AI systems.