[2606.06133] TLA-Prover: Verifiable TLA+ Specification Synthesis through Choice-Optimized Low-Rank Adaptation

View a PDF of the paper titled TLA-Prover: Verifiable TLA+ Specification Synthesis through Choice-Optimized Low-Rank Adaptation, by Eric Spencer and eight different authors

View PDF
HTML (experimental)

Summary:TLA+ is a proper specification language for verifying distributed methods and safety-critical protocols. Giant language fashions (LLMs) regularly produce TLA+ specs that fail the TLC mannequin checker for semantic causes. Throughout 25 LLMs, the very best public baseline is 26.6% syntactic parse and eight.6% semantic model-check. We current TLA-Prover, a 20-billion-parameter mannequin for TLA+ specification synthesis. Coaching combines supervised fine-tuning (SFT) on verified examples with repair-based group-relative coverage optimization (GRPO). Within the GRPO stage, the mannequin learns to repair its personal rejected specs. We additionally practice a direct choice optimization (DPO) variant from the identical SFT checkpoint as an ablation. TLC supplies the reward sign straight, with no realized reward mannequin. 4 tiers grade every output: Bronze (parses), Silver (no warnings), Gold (passes TLC), and Diamond. To achieve Diamond, the mannequin’s correctness property is robotically altered in a small method; TLC should then detect a violation. If TLC nonetheless passes, the property was always-true and contributes nothing; the output fails Diamond. TLA-Prover reaches 9/30 (i.e. go@1 = 30%) at each Gold and Diamond on a held-out 30-problem benchmark. That is roughly 3.5x the 8.6% untuned baseline. The DPO variant reaches 20% at Diamond. Gold and Diamond coincide at each checkpoint; this prevents the trivial-property failure mode.