Optimizing ads retrieval through model and system co-design at Meta
Meta has updated its personalized ads retrieval system by jointly designing the machine learning model, system architecture, and GPU hardware.
2 min read·Thursday, 29 October
Conceptual overview of Meta Andromeda ads retrieval system architecture.
Personalized ads retrieval is the first step in the recommendation pipeline. It narrows a vast pool of eligible ads down to a small set of strong candidates in milliseconds, allowing a more expensive ranking stage to score those survivors in fine detail.
The updated system uses a mixture-of-experts approach, an architecture where the model routes inputs to specialized sub-networks. This design brings large language model techniques into ads retrieval. The user modeling side transitions from a single user tower to an ensemble of specialized user experts, capturing diverse interests and real-time signals.
Architecture diagram detailing the mixture-of-experts approach and gating network in Meta's ads retrieval system. · Source ↗
The interaction model uses a gating network that dynamically routes candidates across the most relevant user experts. This delivers near-ranking-level performance at a fraction of the computing cost.
Single GPU pipeline
At a scale of billions of ads per second, memory bandwidth becomes a major bottleneck. Passing intermediate tensors between graphics processing unit kernels can create terabytes of memory traffic per second.
To solve this, the engineering team built a fully fused GPU pipeline. By combining model execution and system logics into a single kernel, intermediate results stay in GPU registers and thread block shared memory rather than moving to high bandwidth memory.
Co-location strategy
Per-user request operations such as index matching traditionally run on separate central processing unit hosts. At multi-billion ads per second throughput, network traffic between hosts saturates the connection.
Meta solved this by co-locating index matching, scoring, and top-selection on the same GPU. Instead of shipping large candidate lists across hosts, the system sends a compact request payload to a single server, reducing network traffic significantly and keeping operations running smoothly.
The Ultimate 2026 Free AI API Engine: Top 10 Provider Blueprints & Setup GuideFree guide
Comments