OCP Global Summit 2025 | Arm | What AI Wants: New Silicon, New Systems, and a New Era for the Data Center

Introduction

At OCP Global Summit 2025, Arm shared its view of how data centers are evolving in the AI era. With AI workloads growing explosively, performance per watt has become the industry's single core metric. By reviewing the history of Neoverse, emphasizing the importance of custom silicon and proposing the Chiplet System Architecture (CSA), Arm laid out its role and strategy in driving next-generation data center infrastructure.

Content

Arm noted that from OCP's founding in 2011 to today, cloud and infrastructure have entered an entirely new phase. Arm entered the infrastructure market that same year, initially hoping that an open architecture and software investment would let more companies build their own CPUs. Reality showed, however, that designing a CPU is far harder and more expensive than expected — more than most companies can afford.

Then in 2018, Arm launched the Neoverse platform and became an OCP Platinum member. Today every major hyperscaler (AWS, Google, Microsoft, Meta and others) is deploying Arm Neoverse to meet AI's twin challenges of performance and efficiency. Arm highlighted that there are now more than 4 billion AI queries per day, that 16 zettaflops of AI compute were added in 2024, and that data center power consumption is projected to grow by 160 GW by 2030 — equivalent to total U.S. residential electricity use.

Faced with demand on this scale, Arm believes the traditional data center model of "assembling general-purpose parts" no longer suffices. The trend now is custom silicon design: deep co-design of CPUs, accelerators, memory and networking for AI training and inference to optimize performance per watt.

Arm also pointed out that building a new chip costs billions of dollars, but the real challenge isn't money — it's time, talent and verification complexity. To lower the barrier to entry, it introduced the Arm Total Design program, bringing together foundries, EDA, design services and firmware partners into an ecosystem that shortens time to market.

Building on that, Arm worked with more than 70 partners to launch the Chiplet System Architecture (CSA) in early 2025. CSA's goal is not to build a "true chiplet marketplace" but to define a consistent system architecture and interface standards so different companies can co-design on the same platform. Arm stressed that "the package is the new motherboard" — the center of gravity of heterogeneous integration has moved from the circuit board to the package, the core shift of the chiplet era.

Conclusion

The core message of Arm's talk: data centers in the AI era must move toward deep customization and maximum energy efficiency. Whether through the continued evolution of Neoverse or the introduction of CSA, Arm's strategy is to offer a "collaborative, accelerated, scalable" path amid the industry's rapid growth. For the industry, this is an era that no longer relies on off-the-shelf parts, but is driven by purpose-built design and ecosystem collaboration.

Further Perspectives

1. Technical impact

   • Arm's CSA proposal underscores the importance of chiplet heterogeneous integration in AI data centers, which closely parallels the concept of CPO (co-packaged optics) in optical communications today. Both pull "system optimization" down to the package level to break through energy-efficiency and bandwidth bottlenecks.

2. Supply chain observations

   • Through Total Design, Arm brings foundries, EDA and design services into one collaboration framework, meaning future semiconductor design will no longer be done by a single company alone but will look more like "modular collaboration." This is a key opportunity for TSMC, Samsung, EDA companies (Synopsys, Cadence) and OSATs.

3. Market trends

   • Data center demand in the AI era will drive deep integration of custom CPUs + AI accelerators + high-bandwidth memory, with Arm positioning itself as "the core IP supplier of the design ecosystem." This model may spread to automotive, industrial and even defense AI applications, so the market potential is not limited to the cloud.