Recall is the primary bottleneck for factual accuracy in large language models

When large language models get facts wrong, it is easy to assume they simply never learned them. However, a new behavioral framework called knowledge profiling suggests otherwise. Researchers at Google Research have introduced an evaluation approach that separates factual encoding—the storage of facts within model parameters—from factual recall, revealing that retrieval is the true bottleneck in modern systems.

To study this, researchers built WikiProfile, a benchmark comprising 2,150 Wikipedia-derived facts paired with tasks testing encoding, recall, and recognition. The evaluation covered 13 models, generating roughly 4.5 million responses to analyze how frontier systems handle factual retrieval.

The state of encoding versus recall

Across frontier systems such as Gemini 3 Pro and GPT-5, factual encoding is near saturation, with models successfully storing 95 to 98 percent of facts. Despite this high encoding rate, these models fail to directly recall 26 to 34 percent of those same facts. Even when using thinking mechanisms to aid retrieval, the models still miss 11 to 12 percent of facts.

This gap indicates that scaling model size or data coverage improves what a model stores far more effectively than it improves what a model can access. Factual errors in advanced models stem primarily from knowledge that is stored but remains inaccessible.

Why recall fails

Recall is closely tied to how a model originally encountered a fact during training. When a query diverges from the training-time context or phrasing, retrieval breaks down. This dynamic explains two persistent challenges in language models: