OCP Global Summit 2025 | Broadcom | Interconnect Options for Ethernet Scale-Up Networks
Introduction
At this year's OCP Global Summit, Broadcom once again showed its dominance in AI networking architecture. The talk, given by Pete Delbecq, product lead for the Tomahawk family, focused on one theme: how to make Ethernet the core backbone of scale-up networks (GPU communication within a rack or cluster).
This isn't just a question of technology evolution; it's a turning point in industry strategy. Broadcom is pushing a new idea:
"From scale-out to scale-up, Ethernet will unify the entire data center communication layer."
Content
1. Tomahawk Ultra: Built for HPC and AI Scale-Up
This year Broadcom officially launched Tomahawk Ultra, a chip built specifically for high-performance computing (HPC) and AI scale-up networks.
The chip's biggest highlights:
Sub-250 ns latency, enabling GPU-to-GPU end-to-end latency below 400 ns.
Supports single-hop scale-up clusters of up to 512 XPU nodes.
Provides full support for base Ethernet protocols, while stripping out unnecessary packet tables and virtualization logic in exchange for lower latency.
This means Tomahawk Ultra is no longer just a "data center switch chip," but a compute-oriented interconnect core designed for the AI fabric.
2. Ethernet for Scale-Up Networking (ESUN): Redefining Ethernet for the AI Era
At the event Broadcom announced the new ESUN (Ethernet for Scale-Up Networking) initiative, jointly driven by OpenAI, NVIDIA, AMD, Microsoft, Oracle and others.
ESUN's goals:
Establish an Ethernet specification targeted at scale-up networks.
While maintaining compatibility with the Ethernet ecosystem,
add key features such as lossless mechanisms and low-latency link control (LLR, CBFC).
Let GPUs, NICs, switches and CXL fabrics all operate under a unified protocol.
Broadcom calls this "Ethernet SU Transport" — a simplified Ethernet stack optimized for in-rack AI communication.
3. The Evolution from SU (Scale-Up Transport) to ESUN
Broadcom internally used to call this architecture SU (Scale-Up Transport), originally used to help customers integrate Ethernet when designing their own XPU ASICs.
Today the concept has evolved into ESUN and taken concrete shape as an open ecosystem:
ESUN will focus on the layers below the Ethernet header (PHY, Link, MAC).
Upper-layer protocols (such as UALink, UEC, UB Link) can be built freely on top of this foundation.
Supports Link-Layer Retry (LLR) and Credit-Based Flow Control (CBFC), ensuring lossless communication that behaves like memory transactions.
Broadcom sees it as a key bridge to AI fabric standardization.
4. Optics, Copper, and CPO: Strategy for Choosing Interconnect Technology
Delbecq said in his talk: "Copper is king—until it isn't."
In the 200G and 400G era, copper (especially Active Electrical Cables, AEC) is still the most economical, lowest-latency option.
But as bandwidth demand keeps climbing to 800G and even 1.6T, optical interconnect is becoming a necessity.
Broadcom's strategy is to "extend the life of copper and transition smoothly to optics."
This involves three tiers of options:
Copper (AEC/DAC): still advantageous at 200G–400G.
Near-Packaged Optics (NPO): a transitional solution.
Co-Packaged Optics (CPO): the ultimate integration goal.
Meta's test data was cited as key evidence:
CPO systems have accumulated more than 15M device-hours with no unrecoverable failures, zero link flaps, and up to 90% better training performance.
This evidence gives Broadcom more confidence to push its True CPO architecture —
optical engines soldered directly onto the substrate, with the shortest traces and highest reliability.
5. Packaging Options and System Design
Broadcom also showed three packaging options to support scale-up architectures at different stages:
Packaging option | Characteristics | Use case |
BGA (Baseline) | Conventional architecture, low cost | Mainstream volume servers |
CPC (On-Substrate Socket) | Pluggable design, easy to service | Chips with low-to-mid I/O density |
Interposer-based CPO | Direct optical engine connection, lowest loss | High-density switch chips and AI switches |
The third option in particular gets signals "off-substrate" quickly, sharply reducing insertion loss and heat spread; Broadcom internally regards it as the ultimate direction.
6. Why Ethernet?
Broadcom's core view is clear:
Ethernet is the only communication foundation that spans scale-up, scale-out and scale-across.
It has a unified toolchain (telemetry, management, supply chain) and a mature ecosystem.
Users can flexibly mix scale-up and scale-out deployments within a GPU fabric.
The ultimate goal is for data centers to no longer need three networks (Ethernet, InfiniBand, proprietary fabrics),
but to have Ethernet unify the communication layer, achieving a true "AI Superfabric."
Conclusion
Broadcom's message at OCP 2025 was crystal clear:
In the AI era, Ethernet will replace closed architectures as the common language of scale-up networks.
Tomahawk Ultra is only the beginning. The launch of EON and True CPO shows that Broadcom wants to own not just the chips, but the authority over the entire AI fabric standard.
They are no longer content to "supply Ethernet switch chips"; they want to redefine Ethernet's role, extending it from the data center backbone to the edge of the GPU package.
Further Perspectives
Technology impact
Through ESUN, Broadcom is making Ethernet the new common layer of the AI fabric, which will reshape the product design logic of every switch and NIC vendor.
If linear optics and CPO are adopted at scale, Ethernet will completely shed its old "high-latency image" in both latency and reliability.
Supply chain observations
Broadcom's strategy is a classic case of "packaging drives the standard": define the packaging architecture first, then let modules and protocols follow.
The spread of CPO packaging will drive a reshuffle across package substrates, optical engines, liquid cooling and the test chain.
Market trends
EON will complement, rather than compete with, the Ultra Ethernet Consortium (UEC) and UALink.
In the long run, Ethernet-based scale-up networks may replace InfiniBand as the mainstream architecture for AI supercomputing.
Comments