Lengthy-context dealing with stays a core problem for language fashions: even with prolonged context home windows, fashions usually fail to reliably extract, cause over, and use the data throughout lengthy contexts. Latest works like Recursive Language Fashions (RLMs) have approached this problem by agentic manner of decomposing lengthy contexts into recursive sub-queries by way of programmatic interplay at inference. Whereas promising, the success of RLMs critically is dependent upon how these trajectories of context-interaction applications are chosen, which has remained unexplored. On this paper, we research this downside and introduce Self-Reflective Program Seek for Lengthy Context (SRLM), a framework that augments programming-based context interplay with uncertainty-aware self-reflection. SRLM leverages three intrinsic indicators: self-consistency, reasoning hint size, and verbalized confidence. These function complementary indicators of a mannequin’s inside uncertainty, and the mannequin makes use of them to judge and evaluate candidate context-interaction applications. Intensive experiments throughout various benchmark datasets, context lengths, and spine fashions, present that SRLM constantly outperforms state-of-the-art baselines, yielding as much as 22% enchancment over RLMs underneath the identical time funds. Our findings present that recursion itself isn’t the first driver of efficiency in RLMs, and a easy self-reflective program search can match or surpass RLM with out requiring self-query or specific recursion mechanisms. We discover that for context lengths throughout the mannequin’s context window, RLMs with recursion usually degrade efficiency relative to the bottom mannequin, whereas SRLM yields constant and sturdy features throughout each quick and lengthy contexts. We additionally discover that RLM is much less efficient in duties with semantically intensive nature, the place heuristic program search is inadequate and broader contextual understanding is required, whereas self-reflection in SRLM supplies a semantic sign that higher steers reasoning in these difficult long-context situations.
