OCP Global Summit 2026 | Meta | Omar Baldonado | Betting on the Network: Meta's Case for Full-Stack, Open AI Networking

1. Why Meta Says It Is "Betting on the Network"

Baldonado opened by answering the question many partners ask privately: can this many gigawatts of capex keep going? His math: Meta has 3.5 billion users and a vision of a personalized superintelligence (the Muse agent) for each of them. However efficient GPUs get, at that scale it is not just a GPU problem.

He showed Muse's security architecture, a diagram full of containers, storage, memory, firewalls and networking flows. The conclusion fits in one line: AI infrastructure has become infrastructure itself. And what ties it all together is the network.

Meta splits the network into three domains: scale-up inside the rack, scale-out inside the building, and scale-across beyond the building, with one band of "shared hardware and software" running underneath. The real protagonist of this talk is that bottom band.

▲ Hardware, topology and software across Meta's three network domains, all sharing FBOSS + SAI, OSPA and MetaRoCE underneath (Image source: Simple Tech Trend original, data from Meta's 2026 OCP Global Summit presentation)

2. Scale-Out Breakdown: a 102.4T Rack and a Fabric with One Less Tier

IcePack is Meta's first liquid-cooled network rack, based on ORv3-HPRv1, built for high power density and thermal efficiency. Inside it sit IceCube / N-IceCube 102.4T switches with two silicon options in parallel: Broadcom TH6 or NVIDIA Spectrum-6. The systems come from Celestica and Accton, and the software is always FBOSS + SAI. Baldonado said this is the fabric foundation being deployed right now.

▲ Meta's IcePack liquid-cooled network rack and the 102.4T IceCube / N-IceCube switches, using Broadcom TH6 / NVIDIA Spectrum-6 silicon, with systems built by Celestica and Accton (Image source: Meta | 2026 OCP Global Summit presentation (05:49))

The topology change matters even more. After its scheduled (DSF) and non-scheduled fabrics, Meta's next step is the Flattened Planar Fabric (FPF):

• Each accelerator connects directly to multiple fabric planes (eight, per the transcript), removing top-of-rack

• Three-tier Clos becomes two tiers; the slide says "fewer switches/optics and lower latency"

• Load balancing moves to the NIC level, with a combined switch and host control plane, developed closely with NVIDIA

Put simply: per-switch bandwidth doubles, but each GPU passes through fewer switches and fewer optical modules. For optics vendors this is "higher ASP, compressed volume", not volume growth on autopilot.

Meta also holds an unusual position: it is both a hyperscaler with first-party infrastructure and a cloud customer renting capacity. It is bringing the FPF design and operational learnings it developed with NVIDIA into multi-plane networks at its cloud partners. In other words, Meta's topology choices spill over into the clouds it rents from.

3. Scale-Across Breakdown: 1 GW Does Not Fit in One Building

The latest Muse model was trained in Prometheus, Meta's 1 GW-class cluster, which is spread across multiple buildings. They are tied together by the BAG (Backend Aggregation) network; the slide shows dug-up roads and bundles of buried conduit.

Baldonado broke cross-building training down into three priorities:

• Keep availability high: hundreds of thousands of GPUs mean as many NICs, optics and memory parts, millions of components in all, so failures compound

• Design for failure: PAFT (Parallelism-Aware Fault Tolerance) isolates restarts to smaller clusters instead of restarting the whole job

• Co-design with model teams: measure which communication is exposed on the network, then optimize it

▲ GB200/GB300 interruptions (per 8K GPUs per day) fell from about 1,000 in July 2025 to near the target line of 2 by September 2026, while the cloud curve remains clearly higher (Image source: Meta | 2026 OCP Global Summit presentation (11:21))

This chart is the most persuasive data in the talk. Reading the log-scale axis by eye, STT estimates that on-prem GB200 interruptions fell from about 1,000 per 8K GPUs per day to near the 2 goal line in about 14 months, while the cloud curve that appears only in September 2026 still sits around 10-30. The same GPUs, on a network you designed versus in someone else's cloud, differ in reliability by an order of magnitude, and that is why Meta insists on owning the full stack.

One level up, Meta is adding a Regional Backbone (RBB) beneath its existing backbone: SRv6 implemented in FBOSS through SAI, starting with Cisco Silicon One shallow-buffer switches for long-distance interconnect. It went from design to production in 5 months, followed by 3 months to add long-haul and traffic engineering / SDN control.

4. Scale-Up Breakdown: Ethernet Is Already in Production Inside Meta's Own Silicon

Meta disclosed for the first time that its scale-up has been Ethernet all along. Tahan, the scale-up switch inside MTIA 300, uses Broadcom TH5 51.2T, runs FBOSS, and keeps only the feature subset scale-up needs; next comes a TH6 design and preparation for ESUN-compliant ASICs.

▲ Tahan, the Ethernet scale-up switch for MTIA 300, built on Broadcom TH5 51.2T and FBOSS with a liquid-cooled design (Image source: Meta | 2026 OCP Global Summit presentation (13:53))

Progress on ESUN (Ethernet for Scale-Up Networking):

• The 1.0 spec was published in March 2026; its core is a 4-byte ESUN Header replacing 28-48 bytes of IP/UDP headers to improve efficiency on small messages

• The 1.1 spec is in progress and targets latency: optimized headers (UFH) and IPG reduction, with Marvell and Cisco leading switch profiles

• Future candidates: low-latency FEC modes, Reduced Lane Mode and In-Network Collectives (INC), timed by the UEC working group

▲ ESUN replaces the 28–48 byte IP/UDP header with a 4-byte header, cutting header overhead for 64-byte small messages from 44–75% to about 6% (Image source: Simple Tech Trend original, data from Meta's 2026 OCP Global Summit presentation)

This matches the direction we covered in Ethernet starts eating scale-up, with one difference: Broadcom is the seller making the pitch, Meta is the buyer already in production.

5. The Real Moat Is the Shared Layer: OSPA, FBOSS, MetaRoCE

Baldonado said he saved the best for last, and these three items are what "open" actually means here:

• OSPA (Open Switch Platform Architecture): tackles slow onboarding of new platforms, duplicated engineering and lifecycle cost. It is a vendor-neutral and NOS-neutral platform-layer spec covering power, sensors, thermals, fans, inventory, out-of-band interfaces, firmware and troubleshooting. Started by Meta, Microsoft and Alibaba, it now has a dozen OCP members contributing, and the 1.0 spec is in review.

• FBOSS: with Nexthop AI, more of it (including routing protocols) is going open source, it is becoming config-driven, and AI skills for coding agents are open-sourced. The result: multiple platforms now come up without Meta's help.

• MetaRoCE: an RDMA transport announced in August that fixes RoCEv2 gaps: it runs on best-effort Ethernet, handles multipath inside a queue-pair, and tracks out-of-order delivery. AMD Pensando NICs already support it and work with NVIDIA is underway. Two implementations were open-sourced this week, user-space libsoftmetaroce and DPDK-based dpdkmetaroce, with the spec contributed to OCP.

Taken together: SAI standardized the chip interface, OSPA aims to standardize the whole box, and FBOSS plus AI skills compress onboarding time. For Meta, the easier it is to swap ODMs, the stronger its bargaining power.

6. Risks and Counterarguments

• Open does not mean dispersed value: OSPA and FBOSS lower the onboarding bar, which squeezes white-box ODM differentiation into manufacturing, liquid-cooling integration and lead time; margins will not necessarily double with bandwidth.

• FPF is a negative for optics volume: a two-tier fabric uses fewer switches and optics, so optical growth must come from scale-across, and Meta gave no fiber or module counts for BAG or RBB.

• ESUN 1.1 still depends on UEC timing: key items such as low-latency FEC and INC are marked "depends on UEC WG timeline" and may land after the TH6 generation.

• The interruption data is Meta's own metric: the definition of interruptions per 8K GPUs per day and the cloud sample size are not disclosed, and STT's numbers are eyeballed from a log scale.

• NVIDIA support for MetaRoCE is not done: only AMD Pensando supports it today; NVIDIA NICs are still "in progress".

7. What It Means for Taiwan's Supply Chain

• Accton is the only Taiwanese company named on the slides, alongside Celestica as a system builder for the 102.4T IceCube. This is the workhorse switch of Meta's next scale-out fabric, and it lives in a liquid-cooled rack; the thing to track is when 102.4T liquid-cooled platforms become a meaningful share of revenue. We broke down Celestica's AI networking momentum in Earnings Highlights: Celestica (CLS) Q2 FY2026.

• Dual silicon is engineering work for ODMs: IceCube comes in both Broadcom TH6 and NVIDIA Spectrum-6 versions; if OSPA is finalized, one platform spec must serve both ASICs, making platform engineering worth more than assembly scale.

• Optics should reset expectations on "volume per tier": with one tier and top-of-rack gone, scale-out optics per GPU trend down; the real upside is in 102.4T port speeds and in cross-building, cross-region scale-across. Meta is also helping define scale-up optical specs; see Meta, Broadcom and AMD define the OCI 200G line-side spec.

• No other Taiwanese companies were mentioned. Neither the slides nor the transcript named suppliers for liquid cooling, power or connectors, so we should not guess.

8. Conclusion

There was no new optical component in this talk, yet it was one of the most important network-architecture statements at this year's OCP. Meta has collapsed three domains into one open stack: ESUN Ethernet for scale-up, 102.4T plus a two-tier FPF for scale-out, SRv6 on FBOSS for scale-across, and an open-source MetaRoCE at the transport layer. Interruptions falling from about 1,000 to near 2 show this full-stack control is already paying off in real reliability.

Keep the timing separate: 102.4T IcePack is deploying now, ESUN 1.0 is published, and MTIA's Ethernet scale-up is in production; ESUN 1.1, OSPA 1.0 and TH6-generation scale-up are the next 12 months. Three things to watch: when OSPA 1.0 passes, who ships the first ESUN-compliant ASIC, and when MetaRoCE gets NVIDIA NIC support.

When the biggest buyer opens up its software, specs and platforms, hardware vendors no longer compete on who can build it, but on who builds it fastest, most reliably and cheapest.

This article is for technology and industry trend analysis only and does not constitute investment advice.