Mechanism 2: Factual priming
Once we analyze the pure reasoning traces generated for easy factual questions, we discover a typical sample. The fashions aren’t writing out logical proofs; they’re surfacing associated details.
In human cognition, there’s a idea referred to as spreading activation, the place processing a selected idea primes associated ideas in semantic reminiscence, making them simpler to retrieve. We hypothesize that language fashions exhibit the same generative self-retrieval mechanism, which we name factual priming. By producing details topically associated to the query, the mannequin builds a contextual bridge that facilitates the retrieval of the proper reply.
To check hypotheses, we extract simply the concrete details from the mannequin’s reasoning traces, making use of strict filtering to strip away any filler textual content, search plans, or specific mentions of the ultimate goal reply. We then isolate the impact of the recalled details, and present that conditioning on a brief listing of recalled details recovers most of reasoning’s positive factors and helps even when reasoning is OFF.
