Factuality is crucial for making Massive Language Fashions (LLMs) dependable. When a mannequin solutions a factual query incorrectly, is it as a result of the actual fact was by no means encoded, or as a result of the actual fact is encoded however not accessible? Customary accuracy metrics collapse these instances collectively, regardless that they counsel very totally different limitations and really totally different interventions. Encoding failures name for scaling mannequin measurement or increasing information protection, whereas recall failures may also level to post-training and inference-time strategies that assist LLMs higher make the most of what they already encode.
In “Empty Cabinets or Misplaced Keys? Recall Is the Bottleneck for Parametric Factuality”, we introduce information profiling, a behavioral framework that measures each encoding and recall, and use it to look at the underlying bottlenecks of factuality in frontier LLMs (similar to Gemini3 and GPT-5). We then present that many factual errors in frontier LLMs are higher understood as misplaced keys (recall failures), not empty cabinets (encoding failures).
By analogy, we use encoding to indicate parametric illustration of details, recall to indicate retrieving encoded details with out exterior cues, and recognition to indicate figuring out the right reality when it’s introduced amongst options. To help this evaluation, we introduce WikiProfile, a benchmark of two,150 Wikipedia-derived details, every paired with ten questions that probe encoding, recall, and recognition.
