Offering builders the most effective mannequin for the duty at hand has at all times been our purpose. Earlier this 12 months, we made that simpler by launching Auto mannequin choice, which evaluations your process and matches it to the best-suited mannequin for that process.
At the moment, we’re introducing Challenge HydraFusion, a analysis preview that delivers frontier intelligence by runtime orchestration. It creates a full execution plan, selecting from fashions throughout a number of suppliers to draft, critique and revise, or cascade to extra highly effective fashions to finish your process.
HydraFusion fills a key function in our general technique to ship automated semantic routing between native, cloud, and compound fashions. For builders, that complexity stays behind the scenes: you choose HydraFusion like another mannequin, and it chooses a workflow that balances efficiency, price, and latency for every process.
Now accessible as a analysis preview
HydraFusion is offered to customers on all GitHub Copilot plans by /experimental in GitHub Copilot CLI. Utilization relies on the tokens consumed by the fashions HydraFusion makes use of, priced at every mannequin’s commonplace fee.
To strive HydraFusion in Copilot CLI:
Run /replace to put in the most recent model
Run /experimental on
Run /mannequin, then choose HydraFusion (Analysis Preview)
Please submit suggestions within the GitHub Group.
HydraFusion treats workflow choice as an optimization downside. It makes use of functionality alerts for reasoning, code era, debugging, and gear use to pick essentially the most environment friendly execution sample to fulfill the standard bar.
For every request, HydraFusion at the moment chooses considered one of three execution patterns:
Single. One chosen mannequin solves the duty immediately.
Cascade. An environment friendly mannequin drafts an answer and a top quality gate decides whether or not to simply accept it or escalate to a stronger mannequin.
Critique. One mannequin drafts a end result, an unbiased read-only critic from a unique mannequin household evaluations it (following the identical evaluation sample as Rubber Duck), and the drafting mannequin revises as soon as.
Every sample addresses a unique quality-to-cost trade-off. Single preserves velocity and effectivity when one mannequin can clear up the duty immediately. Cascade provides an environment friendly mannequin the primary try whereas retaining a path to stronger inference when the candidate doesn’t clear the acceptance gate. Critique provides an unbiased perspective for duties the place evaluation is extra helpful than one other unaided try.
In offline evaluations throughout three agentic coding benchmarks, HydraFusion persistently demonstrated frontier-level high quality with substantial estimated price financial savings. On TerminalBench 2.1, it improved verified process high quality by 4.9 proportion factors at 67% decrease estimated price in contrast with Claude Opus 5.
Let’s dive into the strategy, the outcomes, and the benchmarks.
Adaptive multi-model orchestration
Builders already coordinate fashions manually: selecting one for a process, asking one other to evaluation the work, or escalating a troublesome downside to a extra succesful mannequin. HydraFusion brings that acquainted course of into the runtime. You select HydraFusion as soon as and keep centered in your process whereas it manages the fashions and workflow behind the scenes.
The secret is selectivity. Some coding duties could be solved immediately, whereas others profit from evaluation, revision, or escalation. HydraFusion evaluates every request and chooses the least advanced workflow anticipated to fulfill its wants, utilizing further mannequin calls solely when they’re possible to enhance the end result. This adaptive strategy balances high quality, price, and latency throughout fashions.
Because the mannequin frontier advances, so does HydraFusion. When new fashions grow to be accessible in GitHub Copilot, we will consider and incorporate them into its mannequin pool, bringing their strengths to the duties finest suited to them.
Constructing HydraFusion
Turning adaptive multi-model orchestration into one reliable coding expertise requires cautious management of execution, evaluation, price, and repository state. HydraFusion is constructed round 5 working ideas:
Full accounting. Combination price and utilization throughout each workflow leg, together with drafting, critique, revision, escalation, retry, and fallback.
Bounded execution. Give every leg express timeout and cancellation conduct to maintain execution and value inside outlined limits.
Remoted evaluation. Run evaluation steps in remoted, tool-less contexts, whereas solver steps use the shared workspace and regular permission-aware agent loop. This permits fashions to evaluate the work independently with out modifying the repository.
Fail-safe utility. Apply no patch when the workflow is cancelled or fails validation, stopping incomplete adjustments from reaching the repository.
Validated routing. Confirm workflow definitions, mannequin bindings, fallback conduct, and mannequin availability earlier than execution begins.
Collectively, these ideas make multi-model orchestration sensible for repository-level work. Internally, the runtime data the function, consequence, price, latency, and diagnostics of every leg so the workflow could be understood after execution. Externally, the developer receives one coherent response and one permission-aware change set.
Exhibiting progress with out exhibiting unfinished work
At the moment: HydraFusion reveals workflow levels however holds intermediate drafts till it returns one coherent end result.
Why: These drafts could also be reviewed, revised, or discarded, so exhibiting them reside may make unfinished work seem closing.
What we’re studying: Ready with out sufficient visibility is an actual trade-off for builders.
Subsequent: We’re actively exploring higher progress updates, guided by suggestions from the analysis preview.
Benchmarking outcomes
Mounted HydraFusion insurance policies have been evaluated throughout three agentic coding benchmarks — TerminalBench 2.1, DeepSWE, and CheckpointBench, our inner benchmark primarily based on actual GitHub Copilot periods — utilizing Claude Opus 5 and GPT-5.6 Sol as comparability baselines. Every coverage used the identical process inputs, instruments, execution limits, pricing assumptions, grading circumstances, and therapy of lacking outcomes. The analysis measured verified process high quality, which is the share of duties confirmed as appropriately answered, and the whole estimated workflow price. Value accounting included each invoked leg, akin to drafting, critique, revision, escalation, retry, and fallback. The outcomes beneath present the most effective tuned HydraFusion configuration.
These managed offline outcomes are particular to the evaluated benchmark revisions, workflow configurations, mannequin pool, and pricing assumptions, with all fashions evaluated on the similar medium reasoning degree. By this analysis preview, we’ll validate how these outcomes translate to actual developer workloads and use the findings to additional optimize HydraFusion for manufacturing high quality, latency, reliability, caching effectivity, price, and security.
TerminalBench 2.1
TerminalBench 2.1 evaluates coding brokers on advanced, multi-step duties in terminal environments.
Determine 2 compares HydraFusion and Opus 5 throughout verified process high quality and estimated workflow price.
DeepSWE
DeepSWE evaluates difficult repository-level software program engineering duties that require navigating giant codebases, understanding cross-file dependencies, and producing end-to-end fixes. On this benchmark, HydraFusion comes inside 1.5 proportion factors of Opus 5 whereas decreasing price by 36%, demonstrating a compelling quality-cost tradeoff for advanced real-world engineering duties.
CheckpointBench
CheckpointBench is an inner multi-turn benchmark curated from actual GitHub Copilot agentic coding periods. Every dialog is anchored to a particular public repository and immutable commit, guaranteeing each session is replayable. The benchmark is balanced throughout language, process sort, issue, scrubbed for high quality, leading to a sensible analysis set that carefully mirrors manufacturing agentic periods. On this benchmark, HydraFusion comes inside 0.1 proportion factors of Opus 5 at 65% decrease price.
Early inner testing has echoed that end result.
To date, the reasoning and process fixing functionality [of HydraFusion] is at or higher than Opus.
Principal Software program Engineer at Microsoft
Hill-climbing HydraFusion
HydraFusion’s routing insurance policies have been formed by how builders use GitHub Copilot on actual coding duties. To make these workflows reproducible, we curated CheckpointBench from actual Copilot coding-session trajectories. We refined HydraFusion repeatedly throughout CheckpointBench, DeepSWE, and TerminalBench 2.1, optimizing throughout the analysis units fairly than for any single benchmark.
HydraFusion’s per-capability scores supplied a constant foundation for evaluating candidate routing insurance policies. As an alternative of manually tuning thresholds, we used beam search to construct the optimum choice coverage. Every candidate was measured in opposition to a frozen baseline on high quality, price, and failure modes, so enhancements have been evaluated on steady floor.
TerminalBench 2.1 gives essentially the most full sequence of runs, making it the clearest view of this iterative enchancment. The development was not linear. Between August 11 and August 25, two operational failures within the analysis harness produced invalid runs. These failures have been excluded from the efficiency development, corrected, and adopted by continued positive aspects within the HydraFusion configurations. By August 25, HydraFusion had reached its strongest working factors within the recorded sequence.
This growth file reveals how the insurance policies improved from repeated experiments. TerminalBench 2.1 was considered one of a number of benchmarks used throughout growth. Its relative saturation makes broader validation essential, so the three-benchmark analysis additionally contains DeepSWE’s extra demanding repository-level duties. The analysis preview extends that studying loop to actual developer workloads.
Attempt the analysis preview
For this preview, first-turn, single-prompt coding duties are the most effective place to begin. We’ll be specializing in sturdy multi-turn efficiency with longer, iterative periods subsequent.
This preview is designed to be taught which duties profit from compound workflows and the way orchestration impacts latency and value in apply. For the most effective expertise at the moment, begin with substantial, well-scoped coding duties that you may hand to Copilot in autopilot mode in a single immediate. Share what you discover, together with the place it excels, the place it falls quick, and what you’d need to see subsequent, by /suggestions in Copilot CLI or within the GitHub Group dialogue.
HydraFusion stays an lively analysis effort. Outcomes, fashions, workflows, availability, names, and product conduct could change as we be taught from the preview. We imagine the following actual achieve in coding brokers will come from combining frontier intelligence with runtime orchestration. HydraFusion is our first wager on that concept: shifting from selecting the most effective mannequin to dynamically developing one of the best ways to unravel every process.
Acknowledgments
An enormous thank-you to the researchers, engineers, product managers, and designers throughout GitHub and Microsoft who curated the coaching knowledge and constructed the coaching pipeline, analysis suites, consumer expertise, and serving stack. We’re particularly grateful to the GitHub Copilot CLI, Copilot API and VS Code staff for overcoming quite a few challenges to carry this analysis preview to our clients.
Meet the Staff
Aashna Garg, Principal Utilized Scientist, Code AI
Shengyu Fu, Accomplice Utilized Science Supervisor, Code AI
Carlos Castro, Accomplice Architect, GitHub Copilot
Siddharth Singha Roy, Analysis Scientist II, Code AI
Andy Salerno, Principal Software program Engineer, GitHub Copilot
Written by
