OCP Global Summit 2026 | Meta | Santosh Janardhan | Concrete to Code: AI Infrastructure at Gigawatt Scale

21 hours ago

7 min read

1. Why an Infrastructure Chief Talks About "Assumptions"

Janardhan set the timeline up front: in 2024 Meta finished its first truly large training cluster, more than 100,000 H100s at roughly 150 MW. The models came out well and the scaling laws held, which meant the next cluster had to be an order of magnitude bigger.

Meta is used to scale, but Janardhan said plainly that AI infrastructure runs on entirely different constraints. The result is Prometheus in New Albany, Ohio: more than 30 structures, 1 GW+, trained on while it was still being built. The new frontier model Meta released that day was trained right here.

The talk was built around four assumptions, and Meta's fix each time one broke:

▲ The four assumptions Meta broke while building Prometheus, and its fixes: multi-building networking, tent structures, an interruption budget, and fault-tolerant software (Image source: Simple Tech Trend original; data from Meta 2026 OCP Global Summit presentation)

When a big project's assumptions hold, life is simple; when they don't, that is where the real engineering starts.

2. Breaking Down Assumption 1: A Cluster Doesn't Need One Roof

Prometheus is not one building but 30-plus structures of different shapes and sizes. The slide shows building-to-building runs from 321 m to 3,800 m, and the wider schematic draws rings at 5 km, 10 km and 100 km.

The problem is physics. AI training moves at the speed of the slowest conversation between two GPUs, and light in fiber takes about 5 µs per km. At the longest 3.8 km run that is about 19 µs one way and 38 µs round trip; with millions of round trips per training job, latency compounds into lost compute. On top of that, the training network must be lossless, the opposite of how the internet works.

Meta's answer is BAG (Backend Aggregation): the spines of every building in a region are strung into an Ethernet super-spine, with deep buffers at that layer to absorb the lossless requirement, a distributed design with no single point of failure, and tens of petabits per second of capacity. At the transport layer, Meta swapped in MetaRoCE, built in-house and open-sourced to OCP.

▲ Prometheus is an AI training cluster spanning multiple buildings, with inter-building fiber distances from 321 m to 3,800 m (Image source: Meta | 2026 OCP Global Summit presentation (slide 3))

The network is physical too. Each conduit between buildings carries thousands of fiber strands, and Prometheus holds roughly 3 to 4 million km of fiber in total, enough, Janardhan said, to reach the Moon and back four times. It is the same bottleneck we covered in It Isn't Physics That Killed Slow-and-Wide, It's 60,000 Fibers (in Chinese), only scaled from the rack to the campus.

The same limit shows up inside the rack: you cannot fit enough copper for more bandwidth. So Meta built an optical backplane prototype, shown at the Meta booth, with the same rack, same serviceability and same cooling, aiming for an order-of-magnitude step up in bandwidth.

3. Breaking Down Assumption 2: Data Centers Don't Need Concrete and Steel

On conventional timelines, Janardhan's line is: "You should have talked to me two years ago." Time to market cannot wait, so Meta went back to a basic question: how much building do servers actually need?

The answer was tents. Working with Sprung Structures, each tent is at least two football fields long, sits on a large concrete pad, is built to data center standards and is rated for a Category 1 hurricane. The key is not the tent itself but the "minimum viable data center": just enough cooling and power, no more, no less. The result is 3 to 5 months to come online at a clearly lower cost.

▲ Tent data centers on the Prometheus campus, each at least two American football fields long, online in 3 to 5 months (Image source: Meta | 2026 OCP Global Summit presentation (slide 11))

Two years versus 3 to 5 months is roughly a 5x to 8x shorter build cycle. That means the pace of a gigawatt campus is no longer set by construction, but by lead times for power, racks and network gear.

Meta even reworked the concrete: an AI tool redesigned the mix, and the new concrete has 19% less carbon, cures 43% faster and is cost-neutral; both the tool and the mix are open-sourced.

4. Breaking Down Assumption 3: Built Doesn't Mean It Works

Janardhan calls himself a software person and frames an AI cluster as a distributed systems problem. His example: a component rated to fail once in five years becomes a frequent problem once you stack a thousand of them. By STT's math, 1,000 such parts produce a failure about every 1.8 days, 100,000 about every 26 minutes, and 1 million about every 2.6 minutes. His spoken "every few weeks" is in fact conservative.

The slide shows an NVIDIA GB200 compute system (described as 15,000 lb with three racks of cooling), and with the campus layout, cooling, racks and even the concrete all changing at once, every variable compounds complexity.

Meta's north star is 2 interruptions per 8,000 GPUs per day. The interruption trend chart on the slide shows:

Metric	Value
Early interruption rate	200+ per 8K GPUs/day
Latest actual	About 4 to 5
Trend improvement	GPU-1 5.5%/week, GPU-2 5.9%/week
Target	2.0

▲ Meta's interruption rates scaled to 100,000 GPUs: one interruption every 35 seconds early on, about every 29 minutes now, and still roughly once an hour even after hitting target (Image source: Simple Tech Trend original; data from Meta 2026 OCP Global Summit presentation)

On the trend lines, going from about 4 to 2 takes roughly another 11 to 12 weeks. Janardhan said Meta has made it and thanked vendors for staying in it for the long haul; but the last actual points on the chart still sit above the target line, and it is the projection that reaches it. The lesson is clear: at this scale you plan for failure, model it, log it, detect it and route around it.

5. Breaking Down Assumption 4: Software Is Not the Easy Part

Tents, networks and reliability problems all have to be wrapped by software into what looks like one single machine. Meta does it in three layers:

A fault-tolerance layer that is model-, parallelism- and partition-aware, routes around failures and reruns only the affected part
Continuous checkpointing, so a rollback goes back only the last 5 to 8 seconds
Growing while training: new racks and clusters join at the next checkpoint, the cluster simply gets bigger, and researchers never notice

Hardware interruptions never reach zero, but software pushes the cost of each one down to seconds; the product of the two is the effective compute of a gigawatt cluster.

6. Next Stop Hyperion: 5 GW on a Single Campus

Janardhan admitted Prometheus changed too many variables at once, and the lessons go straight into the next one.

Hyperion is in Richland Parish, Louisiana; the slide says 5 GW over the next few years, at a single location of about 4,000 acres. Janardhan called it, to the best of his knowledge, the largest data center on the planet. From 150 MW in 2024 to Prometheus at 1 GW+ and Hyperion at 5 GW, scale grows more than 30x.

▲ Meta's two Titan-class training clusters: Prometheus at 1 GW+ and Hyperion at 5 GW within the next few years (Image source: Meta | 2026 OCP Global Summit presentation (slide 20))

Meta's ask to the industry is on the "No one builds this alone" slide: data centers, network, chips and racks, co-designed at every layer and every stage. Meta uses a portfolio of partners across design, construction and operations, and asks that the results be given back to OCP. Meta's work with Broadcom and AMD on the OCI 200G spec (see Meta, Broadcom and AMD Define the OCI 200G Line-Side Spec) is one example of this model.

7. Risks and Counterarguments

The projection hits the target, not the actuals: the latest measured rate is about 4 to 5, still roughly double the target of 2, and Hyperion will bring a new set of variables.
No numbers on tent cost or lifetime: Meta only said "way cheaper," with no cost per MW and no expected service life for the structures.
Minimum viable power and cooling means less headroom: if next-generation rack power rises again, the talk did not say whether the tents can be retrofitted.
Some scale figures are conversational: the Hyperion slide says "over the next few years," not 5 GW at once; Janardhan likened 4,000 acres to half of Manhattan, but it is about 16 km², which overstates the ratio.
Power supply was never mentioned: sourcing and grid connection for 5 GW is the biggest schedule variable, and the talk did not touch it.

8. What It Means for Taiwan's Supply Chain

The talk named no Taiwanese companies; the following is inferred only from what was presented.

The scorecard for rack-scale systems is changing. Meta uses interruptions per 8K GPUs per day as its north star and thanked vendors for chasing it together. Taiwanese ODMs are the main force in GB200 rack assembly and test, so competition is moving beyond lead time and yield to field interruption rates and how fast issues converge.
Shorter build cycles move the pressure to equipment lead times. If tents come online in 3 to 5 months, server racks, power and cooling equipment that cannot keep up become the campus bottleneck. That favors Taiwanese suppliers that can quickly deliver full racks and modular power and cooling, provided capacity keeps pace.
The optical backplane is a new item worth tracking. Meta requires the same rack, serviceability and cooling, so an optical backplane must be designed around existing rack specs, which ties directly to Taiwan's rack, cable and connector assembly capabilities. It is only a booth prototype for now, with no production timeline.

This matches the call in ECOC2026 | No Tech Revolution in the Next Decade, Only a Volume Production War: at gigawatt scale, delivery and reliability decide the winners, not any single component's spec.

9. Conclusion

This talk had no new chip and no new spec, yet it was the clearest view at this OCP of a hyperscaler's priorities. Meta cares about two clocks: how many months from empty lot to online, and how long between interruptions once it is running. Tents cut the first from two years to 3 to 5 months, fault-tolerant software cuts the cost of the second to seconds, and the remaining gap depends on rack-scale systems continuing to drive down interruption rates.

Three things to watch over the next 12 months: whether measured interruptions actually reach 2 per 8K GPUs per day, when the rack optical backplane moves from prototype to deployment, and how Hyperion's 5 GW is phased and powered.

When a data center can go up in a tent in a few months, the bottleneck for gigawatt AI is no longer concrete, but whether every rack runs reliably.

This article is for technology and industry trend analysis only and does not constitute investment advice.