OCP Global Summit 2026 | AMD | Andrew Dieckmann, Steve Scott, Alan Smith | Inside AMD Helios: Engineering AI at Rack Scale

1. Why this session matters: with customers signed, AMD has to explain "why the rack"

This was a fireside chat with only five slides; the real information was in the conversation. It was hosted by Andrew Dieckmann, GM of AMD's Data Center GPU business, with Steve Scott, chief architect of AMD's rack-scale system, and Alan Smith, AMD's chief GPU architect.

The opening laid out the wins: AMD has announced multi-gigawatt Helios deals with Anthropic, Meta and OpenAI, plus cloud providers including Microsoft, Oracle, Vultr and TensorWave, with "several more yet to be announced." We covered what these orders mean for revenue in Earnings Highlights: AMD | Q2 FY2026 (Helios); this OCP session answered a different question: how can AMD take on NVIDIA head-to-head at the rack level, after NVIDIA has already shipped several generations of NVL72.

Steve Scott's answer was direct: the goal of rack-scale design is to make the whole rack "work like a single GPU," but with more memory, network bandwidth and compute than any single package can offer. NVIDIA has said the same thing; the difference is how AMD implements each step.

Same goal, but AMD picked a path that is more open at every step.

2. Choice one: why Ethernet for scale-up

Helios uses Ethernet for scale-up, and Steve Scott gave three reasons:

• Ecosystem: the most vendors for switches, SerDes and connectors, mature software for management, telemetry and error handling, and years of customer experience.

• Highest switch radix: Ethernet switches are always at the leading edge of switch capacity. High radix means a packet crosses a single switch to reach any GPU, with no multi-hop.

• Future scaling: he said explicitly that scale-up domains will grow in future generations, and high radix is key to keeping hop counts low as they do.

But traditional Ethernet has one trait that does not fit scale-up: external NICs are built to address hundreds of thousands of endpoints and carry complex data-movement engines. With the bandwidth Helios packs into a scale-up pod, that design is too heavy.

AMD's answer is to connect the Infinity Fabric / UALink load-store protocol directly to an extremely lightweight NIC, and to integrate that NIC into the GPU fabric itself. The GPU packs loads and stores straight into Ethernet packets and adapts dynamically to traffic:

• With light, latency-sensitive traffic, it emits small packets as soon as data is ready.

• With heavy, throughput-bound traffic, it packs more data into larger packets to raise effective bandwidth.

▲ Conventional Ethernet needs an external NIC and multiple switch hops; Helios integrates a lightweight NIC into the GPU, packetizes load/store directly and crosses a high-radix switch in one hop (Image source: Simple Tech Trend original, data from AMD's 2026 OCP Global Summit presentation)

This matches the direction we discussed in OCP APAC Summit 2026 | Ethernet Starts Eating Scale-Up: the scale-up battle has shifted from "NVLink or something else" to "how much must Ethernet change to be light enough." AMD's version puts the changes on the GPU side and keeps the switch standard Ethernet.

3. Choice two: the ORW double-wide rack, and "the last all-electrical generations"

Steve Scott described the rack as a chain of cause and effect:

• Tight coupling needs massive bandwidth, and this is probably one of the last generations done entirely electrically, so boards and cable cartridges are optimized for electrical signaling.

• Electrical reach is short, so everything is packed close together, and that density forces liquid cooling.

• Deployment scale is enormous, so compute and switch trays are blind-mate: slide one in and power, cooling and signals all connect at once.

▲ The AMD Helios rack with its compute and switch trays, integrating EPYC, Instinct, Pensando and ROCm (Image source: AMD | 2026 OCP Global Summit presentation (20:57))

Helios's most visible feature is that it is wide. It uses the OCP ORW (Open Rack Wide) double-wide standard, for reasons that came from customers:

• More front-panel space, allowing more network bandwidth.

• More internal space, room for what AMD and its customers want to add.

• Lower floor loading, since cramming too much into a rack makes weight per area unmanageable.

The timing signal is what matters most here. Copper backplanes plus liquid cooling is the path NVL72 opened, and AMD follows it this generation; but when its chief architect says "one of the last generations" in public, he is telling the supply chain: moving scale-up to optics is already on AMD's roadmap, just without a date yet. Steve Scott's own words were "aggressively introducing optics."

4. Choice three: one reference design, many data centers

AMD has reference designs, but Steve Scott stressed that "one size doesn't fit all," and Helios leaves room to adjust at every interface:

Item	Customer options
Power feed	Different feed voltages
Coolant temperature	Chiller-free at 40-45°C, or chilled at 25-30°C
Power redundancy	N+N, or 4-to-make-3
Ride-through	Optional supercaps to ride through brief failures
Scale-out topology	Rail-optimized, multiplane and more
Manufacturing	Customer-selected ODM

▲ Helios leaves room for customer choice in six areas: power feed, coolant temperature, redundancy, power-loss ride-through, network topology and ODM, with the spec released to OCP a year ago (Image source: Simple Tech Trend original, data from AMD's 2026 OCP Global Summit presentation)

More importantly, AMD publicly released all of the Helios specs to OCP more than a year ago, covering connectors, power supplies, cooling and even rack movers. The aim is to let customers build robust supply chains and work with the ODM of their choice.

This is the most fundamental difference between AMD and NVIDIA. Here are the two rack philosophies side by side:

Aspect	AMD Helios	NVIDIA NVL72 family
Scale-up protocol	UALink load-store carried over Ethernet	Proprietary NVLink
Scale-up switch	Standard Ethernet switches, multiple vendors	Proprietary NVLink Switch
Rack spec	OCP ORW double-wide, full rack spec public	NVIDIA MGX, parts of design contributed to OCP
Supply-chain control	Customers and ODMs	NVIDIA defines, partners execute

NVIDIA's advantage is integration and shipping experience. AMD's bet is that hyperscalers will accept some extra integration work to regain control of their supply chains and data-center design. Meta and OpenAI signing multi-gigawatt deals shows at least some customers accept that trade.

5. Memory and MX formats: what AMD really wants to protect is portability

Alan Smith offered a practical framework for memory: bandwidth directly sets token-generation speed, while capacity is harder to quantify but is a parameter you can "dial in." More capacity means fewer GPUs for the same model, so each GPU's compute is better utilized; the remaining space holds KV cache, letting each GPU serve more concurrent users under the same interactivity SLA.

The metrics he uses for Helios are equally practical: for inference, time to first token, tokens per second and tokens per second per user, with the goal of pushing the whole Pareto frontier up and to the right; for training, time to a trained model of a given capability. Compared with the MI355 generation a year ago, matrix compute, vector compute, memory, scale-up bandwidth, GPUs per scale-up domain and scale-out capability all scaled together, because if one falls behind, GPU utilization drops.

On software, Alan Smith highlighted AMD's work in OCP to standardize MX (microscaling) floating-point formats: model builders can use one set of quantization and training flows across multiple hardware vendors, without re-quantizing or retraining. For AMD this is both defense and offense: if models move painlessly, the cost for customers to switch suppliers falls.

6. Risks and counterarguments

• Openness costs integration effort: public specs, customer-chosen ODMs and many power and cooling combinations mean each customer's validation work differs. NVIDIA's integrated-rack delivery still has a time-to-market edge.

• No measured data on the lightweight NIC: the dynamic latency-versus-bandwidth mechanism sounds elegant, but the session gave no latency, effective bandwidth or NVLink comparison figures.

• No hard specs: the session never mentioned GPU count, HBM capacity or compute figures; check these against AMD's earlier official announcements.

• Vague optics timeline: neither "last all-electrical generations" nor "aggressively introducing optics" came with a year, so it is unclear whether this means the next generation or the one after.

• Customer concentration: three multi-gigawatt deals carry most of the visibility, and any change in one customer's deployment pace will show up directly in shipments.

7. What it means for Taiwan's supply chain

• Open specs directly widen the addressable market. AMD said openly that the public specs let customers build supply chains for connectors, power supplies, cooling and rack handling, and choose their own ODM. Taiwan has major players in each of these categories (AI server ODMs, power, liquid-cooling modules, mechanical parts), but the session did not say which ones have Helios orders; that will have to come from company earnings calls or customer announcements.

• Cooling must handle two water temperatures. With chiller-free 40-45°C and chilled 25-30°C designs coexisting, cold-plate and CDU vendors need two thermal designs and two validation tracks, a wider product line and higher qualification cost.

• ORW double-wide is a new spec for mechanics and cabling. A wider front panel means more network ports, and blind-mate trays mean power, cooling and signal connectors must all align in one motion, a spec-transition opportunity for Taiwan's mechanical-part and connector makers.

• The optics signal is here, but there is no timeline yet. Steve Scott said AMD will be "aggressively introducing optics." If scale-up moves to optics in later generations, the beneficiaries are optical engines, laser sources and fiber connectivity; we mapped the possible routes in After Copper Runs Out for AI: Seven Paths for Scale-Up Optics. For Taiwanese suppliers, now is the time to secure design-in positions, not to expect revenue.

8. Conclusion

The session brought no new specs, but it made AMD's rack strategy clear: NVIDIA sells an integrated machine; AMD sells an open design plus a GPU that speaks Ethernet on its own. Ethernet scale-up, ORW double-wide, public specs and MX formats each lower customers' switching costs and supply-chain lock-in.

Keep the timing separate. This generation of Helios is still copper backplane plus liquid cooling, and its competitiveness depends on landing on time and at volume in Meta, OpenAI and Anthropic data centers; the next generation's keywords are larger scale-up domains and optics. Over the next 12 months, watch three things: the schedule of Helios's first large-scale deployments, measured latency of the integrated Ethernet NIC, and when AMD announces the generation and timeline for scale-up optics.

When the chief architect says "the last all-electrical generations," the countdown for copper backplanes is no longer a supplier's guess but a customer's roadmap.

This article is for technology and industry trend analysis only and does not constitute investment advice.