Metaroce: an open network protocol for ai-scale ethernet

Training and serving frontier artificial intelligence models depends on fast, reliable networks that move data between graphics processing units (GPUs) — specialized chips designed to handle heavy parallel calculations — without wasting computing cycles. To meet this challenge at scale, Meta designed MetaRoCE, a clean-sheet remote direct memory access (RDMA) transport protocol purpose-built for artificial intelligence workloads on standard commodity Ethernet. The organization is releasing the specification, a reference software implementation, and a compliance test suite through the Open Compute Project to help the broader industry adopt and build on it.

The network bottleneck

As AI clusters grow to hundreds of thousands of accelerators spread across multiple data centers, network performance directly impacts training speeds and user response times. Collective operations synchronize thousands of processors during training, meaning the slowest data transfer sets the pace for the entire job. Standard network protocols expect the network to deliver every frame in perfect order, which creates bottlenecks and can strand significant computing capacity when tiny delays occur.

How the protocol works

MetaRoCE changes how data moves by shifting intelligence from the network switches to the endpoints. Traditional architectures rely on switches to maintain order and prevent data loss. In contrast, this new approach treats out-of-order packet arrival as normal. Every packet carries its own destination, allowing data to be written straight to its final memory location instantly without waiting for a reorder buffer.