Meta introduces MTIA 300 training chip with integrated network interfaces

Training recommendation models presents a distinct infrastructure challenge compared to large language models. While LLMs require enormous floating-point throughput, recommendation workloads are heavily bottlenecked by fast and efficient communication between accelerators. Because embedding tables can contain over 99 percent of a model's parameters, hybrid parallelism generates frequent AllReduce, AllToAll, and AllGather collectives across hundreds of accelerators. On traditional general-purpose graphics processing units, these communication operations compete directly with training computation for the same hardware resources, often leaving expensive silicon underutilized.

Integrating the network directly on the chip

To address this limitation, the MTIA 300 features network interfaces embedded directly inside the chip package. Two network chiplets, each containing six custom 800 Gbps RDMA NICs, provide 1.2 terabytes per second of total input and output bandwidth without crossing a peripheral component interconnect express bus. This layout eliminates the host-device-network interface card bottleneck found in conventional architectures where a central processing unit must mediate between the accelerator and the network.

The same 12 Ethernet-based NICs handle both scale-up communication within a rack of 16 nodes at up to one terabyte per second and scale-out communication across racks at 200 gigabytes per second. This flexible partitioning allows engineers to adjust to changing requirements by reconfiguring the network rather than altering hardware. To minimize transaction latency, the design introduces express doorbells where work request writes serve directly as the doorbell, saving approximately 800 nanoseconds per operation.

Offloading communication from the compute grid

On standard GPUs, collective communication libraries execute as kernels that consume streaming multiprocessors, which are the same resources required for training math. The MTIA 300 takes a different path by pairing its 12-by-6 grid of processing elements with 16 dedicated message engines that handle communication independently. Each message engine includes a RISC-V core for orchestration, a network interface controller routing requests, and a near-memory compute block that performs reductions at 128 bytes per cycle.

Positioned adjacent to high-bandwidth memory and cache, these near-memory compute blocks deliver more than 2.8 terabytes per second of reduction throughput. This capability enables line-rate execution of collective operations without touching the main compute grid, resulting in less than 0.5 percent compute degradation when running large matrix multiplications concurrently with communication.

A compiled communication model

Accompanying the hardware is the HCCL communication library, which was co-designed alongside the processor from the beginning. Instead of driving communication from the host during execution, the library compiles each collective into a complete set of subgraphs with explicit dependencies, which are then dispatched to the message engines for autonomous execution. Once work reaches the device, host intervention is no longer required.

In production testing with a 150-billion-parameter recommendation model distributed across 40 accelerators, the MTIA 300 achieved communication bandwidths up to 940 gigabytes per second within a single rack, completing total communication tasks 3.9 times faster than an equivalent GPU cluster. These architectural choices establish a foundation for future AI silicon as workload demands continue to evolve.