The toughest a part of constructing AI techniques as we speak is now not gaining access to a succesful mannequin. It’s realizing how to decide on, validate, optimize, and function the suitable mannequin throughout the complete lifecycle of an actual utility.
Take a retrieval-augmented technology (RAG)-based buyer assist copilot or a tool-calling agent that helps workers full enterprise workflows. In a prototype, it might be sufficient to select a robust mannequin, join a number of knowledge sources, and get a helpful response. In manufacturing, the system must retrieve the suitable context, name the suitable instruments, meet high quality and security thresholds, keep inside latency targets, and run at a value the enterprise can maintain.
Fashions evolve, prices shift, and manufacturing necessities usually arrive after the primary model is already working. Success relies upon much less on selecting probably the most highly effective mannequin and extra on constructing a disciplined working method across the utility.
That’s the place Microsoft Foundry is available in: a unified platform to pick out, consider, optimize, function, and constantly enhance AI purposes at manufacturing scale.
What’s new
Microsoft Foundry continues to develop the mannequin ecosystem and working floor for builders constructing manufacturing AI techniques.
Fireworks AI on Microsoft Foundry is now typically obtainable, giving builders entry to production-grade open mannequin inference by way of a single Azure endpoint, with enterprise service-level agreements (SLAs) and zero-setup onboarding.
Foundry can be including new mannequin households and capabilities throughout modalities, together with Microsoft AI fashions, accomplice fashions, open-source fashions, customized fashions, and post-trained variants. Collectively, these updates give builders extra alternative whereas maintaining choice, analysis, deployment, and operations in a single constant workflow.
The problem is now not entry. It’s operations.
In a prototype, the questions are easy: Can the mannequin reply the immediate? Can it hook up with my knowledge? Can it full the joyful path?
In manufacturing, the questions change. Which mannequin suits every job? How do I validate it alone knowledge? What latency price range does this expertise require? How a lot throughput do I want at peak? What occurs when quota is constrained, prices spike, or a more recent mannequin turns into obtainable? How do I monitor high quality, detect eval drift, roll again safely, and show the system is ruled?
Agentic techniques usually fail when the mannequin is mismatched, analysis is incomplete, prices run unchecked, or governance arrives too late. Groups that depend on a single supplier face one other threat: lock-in, with no escape hatch when a mannequin degrades, pricing adjustments, or capability turns into constrained.
Foundry is constructed on the alternative philosophy. It’s a model-agnostic platform spanning Microsoft, open-source, and impartial software program vendor (ISV) accomplice fashions, all on the identical working floor.
The reply is to deal with mannequin choice and optimization as a steady working self-discipline:
1. Choose the suitable mannequin for the duty
Mannequin choice is about workload match, not leaderboard rank. Earlier than selecting a mannequin, outline the duty contract: what the mannequin must do, what attractiveness like, what constraints it should function inside, and which failure modes are unacceptable.
A routing step might have low latency. A coverage query might have grounded reasoning with citations. A coding agent might have deeper reasoning and gear use. A customer-facing copilot might have sturdy security boundaries, predictable latency, and price effectivity at scale.
A easy mannequin choice framework:
Workload want
Favor this method
Why
Classification, routing, extraction, or high-volume chat
Smaller, lower-latency mannequin
Retains value and latency low
Complicated reasoning, coding, or planning
Stronger reasoning mannequin
Improves high quality for more durable duties
Picture, speech, voice, or bodily AI
Modality-specific mannequin
Matches the mannequin to the enter and output kind
Combined workloads with completely different complexity
Mannequin Router
Routes every request primarily based on high quality, value, and latency
Area-specific conduct, tone, or format
Fantastic-tuned or customized mannequin
Improves consistency on your state of affairs
Efficient mannequin alternative is dependent upon 4 dimensions: functionality, security, latency, and price.
Foundry helps builders make these tradeoffs by way of a broad mannequin ecosystem and a constant working floor. Builders can entry Microsoft fashions, main base fashions, accomplice fashions like Fireworks AI, open-source fashions, customized fashions, and post-trained variants by way of one choice, analysis, and deployment workflow.
Developer tip: For builders who wish to bypass guide choice, Foundry offers Mannequin Router in Foundry Fashions. Mannequin Router mechanically routes every request to probably the most applicable mannequin primarily based on workload traits, value targets, and latency necessities.
2. Validate with your individual evals and knowledge
Benchmarks usually are not sufficient. A mannequin that leads a public leaderboard should underperform in your prompts, your knowledge, your customers, and your online business guidelines. Manufacturing confidence comes from evaluating towards the workloads your utility will truly run.
With Foundry, builders can deliver their very own analysis inputs, together with CSV or JSONL datasets with prompts, anticipated outputs, labels, or ground-truth solutions. They’ll run side-by-side comparisons throughout fashions and prompts, consider brokers and multi-step workflows, and examine outcomes throughout datasets, traces, and production-like situations.
Constructed-in high quality and security evaluators assist measure alerts similar to relevance, groundedness, coherence, fluency, security, and coverage adherence. Customized evaluators can seize application-specific guidelines, codecs, and enterprise logic.
A powerful analysis covers:
High quality: Did the mannequin full the duty accurately?
Accuracy and groundedness: Did it produce dependable solutions primarily based on the suitable context?
Security: Did it comply with insurance policies and keep away from unacceptable responses?
Efficiency: Did it meet latency, throughput, and reliability necessities?
Value: Did it ship the suitable consequence on the proper value?
Analysis ought to run constantly as new mannequin variations, fine-tuned variants, agent adjustments, or new mannequin households grow to be obtainable.
Developer tip: Outline success standards earlier than opening the mannequin catalog. Standards-first analysis prevents anchoring on mannequin repute as a substitute of workload match.
3. Optimize value and efficiency
Value is a first-class architectural concern, not an afterthought. In prototypes, it might be acceptable to ship each job to probably the most succesful mannequin. In manufacturing, that method breaks down shortly.
A easy classification job, a RAG response, a long-context reasoning workflow, and a multi-step agentic course of shouldn’t at all times use the identical mannequin or deployment technique.
Foundry provides builders levers to optimize throughout high quality, value, and latency on the system stage:
Clever routing: Ship every job to the suitable mannequin primarily based on complexity and price range.
Batching: Use asynchronous processing for workloads that don’t require real-time responses.
Caching: Keep away from paying repeatedly for an identical or near-identical requests.
Provisioned throughput: Use devoted capability for predictable efficiency at scale.
Quota administration: Scale extra predictably with quota tiering, international buyer quota, and knowledge zone buyer quota.
Mannequin optimization: Use mannequin compression, fine-tuning, or distillation the place applicable.
Fireworks AI on Foundry is now typically obtainable, giving builders entry to a high-performance open mannequin catalog by way of a single Azure endpoint, with enterprise SLAs, no separate infrastructure, and no separate contracts.
Developer tip: Profile value by job kind earlier than optimizing globally. Routing choices are workload-specific, not one-size-fits-all.
4. Function at scale with enterprise confidence
Deploying an endpoint shouldn’t be the identical as working a manufacturing AI system. Groups want to grasp how the system behaves, implement insurance policies, monitor utilization and price, check mannequin adjustments safely, and roll again when high quality or efficiency regresses.
Foundry brings these working capabilities into one floor: versioning, SLA-backed reliability, safety, governance, entry controls, audit logging, utilization monitoring, and managed upgrades.
Groups can monitor token utilization and throughput, examine logs and traces, consider mannequin and agent conduct, implement insurance policies, and examine adjustments earlier than rolling them out broadly. As new mannequin variations grow to be obtainable, they will check towards analysis datasets and traces, validate high quality, latency, and price impression, and cut back threat with versioning and rollback methods.
The Fireworks AI on Foundry typically obtainable (GA) launch is a concrete instance of this working mannequin, with enterprise SLAs, provisioned throughput unit (PTU) Knowledge Zone assist, SOC2 readiness, and the identical entry controls and audit logging that govern Foundry.
Manufacturing adopters span AI-native and conventional enterprise workloads, together with Perplexity, Motif, UiPath, and StackBlitz. Throughout preview, the platform processed greater than 176 billion tokens throughout 17 S&P 500 enterprises.
Developer tip: Deal with mannequin upgrades like dependency upgrades: check towards baselines, stage rollouts, monitor regressions, and preserve a rollback plan.
5. Repeatedly enhance as fashions and workloads evolve
AI techniques are dynamic. Fashions enhance, workloads shift, consumer conduct adjustments, pricing evolves, and new mannequin households arrive. The very best system as we speak will not be the perfect system six months from now.
That’s the reason the lifecycle loop issues:
Choose the suitable mannequin for the duty.
Consider it towards your individual knowledge and manufacturing baselines.
Optimize for high quality, value, latency, and throughput.
Function with governance, observability, and reliability.
Enhance as new fashions, instruments, and customization choices emerge.
For engineering groups, each mannequin, immediate, software, agent, or workflow change must be handled like a manufacturing change. New mannequin variations must be examined mechanically towards regression datasets, manufacturing traces, and recognized edge instances earlier than rollout.
A mannequin could enhance high quality however enhance latency, cut back value however weaken groundedness, or carry out higher on frequent instances whereas regressing on high-risk situations. Automated evaluations assist groups detect these tradeoffs early.
Developer tip: Automate your analysis pipeline so each new mannequin model is in contrast towards manufacturing baselines for high quality, security, latency, throughput, and price earlier than deployment.
What this implies for builders
The following section of AI growth is not going to be gained by groups that merely have entry to the largest fashions. Will probably be gained by groups that know the best way to function fashions effectively.
Meaning selecting by workload match, validating with actual knowledge, optimizing value and efficiency, deploying with governance, and bettering because the panorama shifts.
Microsoft Foundry is designed for precisely this actuality: a model-agnostic platform spanning Microsoft, open-source, and ISV fashions, all on one working floor. No lock-in. No re-architecture. No guesswork.
The way forward for AI growth shouldn’t be about guessing which mannequin would possibly work. It’s about constructing an working self-discipline that permits you to know.
