Hugging Face Fashions on Foundry Managed Compute

At Microsoft Construct 2026, we introduced Foundry Managed Compute and Hugging Face fashions on Foundry — a curated catalog of open-weight fashions from the Hugging Face ecosystem, refreshed weekly, deployable in a single click on onto Foundry Managed Compute. Weights are pre-staged in Azure, runtimes are constructed and scanned by Microsoft, and each mannequin within the Assortment ships with the identical enterprise safety, governance, observability, and billing that applies to each different mannequin on Foundry.

The Platform: Microsoft Foundry and Managed Compute

Microsoft Foundry is a platform for constructing and working agentic AI purposes. Foundry begins with the widest mannequin choice on any cloud — fashions from Microsoft, OpenAI, Anthropic, Meta, Mistral, DeepSeek, Hugging Face, and others, spanning frontier, open-source, and customized weights — all accessible by way of a single endpoint and a single set of SDKs in Python, C#, JavaScript, and Java.

On prime of these fashions sits the Foundry Agent Service: multi-agent orchestration with built-in reminiscence, information grounding by way of Foundry IQ, and a catalog of connectable instruments by way of agentic protocols, so brokers can work with enterprise knowledge. As soon as brokers are operating, Foundry supplies end-to-end tracing, real-time monitoring, steady evaluations, and a immediate optimizer that improves agent habits primarily based on eval outcomes — observability and high quality loops which might be a part of the platform.

Alongside that, builders get entry to:

Content material security filters
Activity-adherence guardrails
An AI Pink Teaming Agent for adversarial testing
Unified RBAC
Non-public networking
Azure Coverage integration straight inside the platform

Alongside pay-per-token (lowest-friction path to get began) and provisioned throughput (predictable, high-performance manufacturing workloads on frontier fashions), Foundry Managed Compute is the third deployment possibility in Foundry: a managed GPU platform-as-a-service for open-source and customized fashions.

You deploy a mannequin occasion described by the issues that matter to your workload — parameter rely, context size, and whether or not you need to optimize for latency or throughput — and Foundry handles the GPU topology beneath, whether or not the occasion lands on one accelerator or a number of, so that you assume and plan in mannequin phrases.

Microsoft takes care of the machine: container updates, runtime upgrades, and safety patches occur routinely on the supported runtimes — vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama.cpp — with out redeploying your mannequin, whereas mannequin configuration, deployment habits, and routing stick with you.

That consistency carries by way of the developer floor — pay-per-token, provisioned throughput, and Managed Compute share:

A single endpoint
The identical SDKs
The identical authentication
The identical observability
A single invoice

Open-source fashions combine with Foundry Brokers the identical manner frontier fashions do, so you’ll be able to combine mannequin sorts in a single agent and not using a separate integration path.

Managed Compute gives:

International deployments — broadest capability and greatest pricing
Knowledge Zone deployments — residency and sovereignty

Similar code, similar workflow. Quota is aligned to accelerator households, so a plan constructed on the H100 household immediately carries ahead as new {hardware} generations come on-line.

Why Hugging Face

Hugging Face is the general public sq. of open AI: 15 million builders, 400,000 organizations, and over 3 million open fashions revealed, with new frontier capabilities — agentic coding, video segmentation, speech, embeddings — touchdown weekly. It is the GitHub of open fashions, the place the neighborhood publishes weights, writes mannequin playing cards, compares evaluations, and pulls fashions for experimentation.

Open fashions have closed the hole with proprietary fashions on benchmark after benchmark, and so they unlock issues proprietary endpoints cannot:

State-of-the-art is now open. Main open-weight fashions are aggressive with the highest closed frontier fashions on essentially the most broadly used benchmarks.
Deep customization. Full weights make it potential to fine-tune, distill, quantize, and adapt with LoRA — tailoring fashions to your area, your knowledge, and your latency and value targets.
Your mannequin, your internet hosting. Weights run in your tenant on infrastructure you management, behind your inference endpoint, together with your identification and community boundaries.
Value shaping. Pay for accelerators by the hour, scale to zero when idle, and right-size GPUs to the particular mannequin — helpful for regular, high-volume, or latency-sensitive workloads the place per-token pricing is tougher to foretell.
Model management. Pin a particular mannequin model, consider it, deploy it, and transfer ahead or roll again by yourself launch cadence.

The catch has all the time been the operational layer: discovery, license evaluate, safety screening, runtime choice, GPU sizing, picture constructing, CVE patching, and standing the mannequin up behind an enterprise-grade endpoint. Hugging Face, by itself, will not be an enterprise serving platform. Hugging Face fashions on Foundry is that operational layer, run by Microsoft.

Hugging Face Fashions on Foundry

The Hugging Face Assortment brings a curated subset of fashions straight into the Foundry Mannequin Catalog:

Refreshed weekly — trending fashions from the Hugging Face ecosystem are added constantly because the neighborhood publishes them.
Each modality — textual content, imaginative and prescient, audio, and multimodal: LLMs and VLMs for chat and brokers, ASR and speech translation, embeddings, segmentation, picture technology.
Safetensors solely, no untrusted code — each mannequin within the Assortment is security-screened and ships within the SafeTensors weight format, with no trust_remote_code execution paths except rigorously reviewed.
The appropriate runtime for the mannequin — vLLM and SGLang for LLMs, TensorRT-LLM and NIM the place relevant, TEI for embeddings, llama.cpp for CPU — Foundry picks the engine that matches the mannequin.

Out of your facet, an open-weight mannequin within the Hugging Face Assortment appears and behaves like another mannequin within the Foundry Mannequin Catalog, and each mannequin within the Assortment has been put by way of a multi-stage publishing pipeline earlier than it ever exhibits up there.

The Curation Pipeline

Hugging Face and Microsoft work collectively to carry the preferred open-weight fashions from the Hugging Face ecosystem to Microsoft Foundry — production-ready for enterprise environments — by way of a scientific curation course of:

Determine trending fashions within the Hugging Face ecosystem — primarily based on neighborhood indicators, accomplice requests, and buyer demand — and choose candidates for enterprise readiness.
Display for compliance and safety — mannequin licenses are reviewed towards Microsoft’s enterprise distribution coverage (with license metadata captured and preserved on the catalog mannequin card), and repositories are inspected for trust_remote_code patterns and customized executable code; any mannequin that will require executing third-party Python at load time is both remediated or excluded.
Construct, scan, and publish runtimes — Microsoft builds inference container photos on supported runtimes (vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama.cpp), scans them for CVEs, and indicators and publishes them to a Microsoft-managed container registry.
Add weights to safe Azure storage — mannequin weights are pulled from Hugging Face as soon as, validated towards the revealed mannequin card, and saved in Microsoft-managed Azure storage within the areas the place the mannequin is served.
Validate and publish to the catalog — each mannequin + runtime + accelerator mixture is examined for API conformance (chat completions, embeddings, rerank, and so forth.) and efficiency (latency, throughput, time-to-first-token, inter-token decode time), then the validated mannequin — with its templates, runtime photos, and weights — is revealed to the Foundry Mannequin Catalog with a one-click deploy path onto Managed Compute.

As a result of weights are pre-staged in Azure storage and runtime photos stay in a Microsoft-managed registry, your deployments will not want outbound community entry to Hugging Face Hub — you’ll be able to deploy to manufacturing inside a non-public community.

Mannequin Runtimes

Hugging Face fashions on Foundry are powered by a flexible assortment of community-built, open-source inference runtimes — every chosen and tuned for Foundry Managed Compute, and matched to the mannequin architectures it serves greatest. Throughout all runtimes, the systematic curation course of means new variations and patches land on Foundry rapidly, and current mannequin deployments are upgraded routinely — with out requiring you to redeploy.

vLLM — the default high-throughput serving engine for open giant language fashions, tuned for manufacturing GPU workloads. As a result of Hugging Face is a direct contributor to vLLM, any mannequin within the Transformers library can run on vLLM out of the field — so when a brand new mannequin lands on Hugging Face, it may be served on Foundry the identical day, with no ready on a customized integration.

SGLang — a serving engine for language and multi-modal fashions, with robust assist for structured outputs (JSON, regex, grammar-constrained technology) that agentic and tool-using workloads rely upon. Hugging Face and the SGLang group have constructed a Transformers backend integration for SGLang, so any mannequin within the Transformers library runs on SGLang out of the field — and reaches Foundry the identical day it lands on Hugging Face.

Textual content Embeddings Inference (TEI) — the runtime for embedding, reranker, and sequence-classification fashions. Accelerator-specific photos ship with kernels compiled for every GPU and CPU household Foundry helps, preserving the embedding sizzling path lean for RAG and semantic-search workloads.

llama.cpp — the CPU and small-GPU path for GGUF-quantized fashions. Helpful for cost-optimized deployments, smaller fashions, and CPU-only areas, with the identical OpenAI-compatible API as vLLM and SGLang.

TensorRT-LLM and NIM — used on NVIDIA {hardware} the place NVIDIA’s optimized kernels and Triton-based serving ship meaningfully higher latency or throughput for particular mannequin households.

hf-serve — Hugging Face’s personal multi-model inference server, used for mannequin architectures exterior the LLM and embedding quick paths (imaginative and prescient, audio, segmentation, and different Transformers-native pipelines) so the Assortment can cowl each modality with a constant serving layer.

Deploying and Scoring an Open-Weight Mannequin

The Hugging Face Assortment within the Foundry Mannequin Catalog is the place you begin, and deployment is 5 steps:

Browse the catalog and decide a mannequin — the deploy wizard additionally surfaces the mannequin id, deployment template id, and acceleratorType you will want in the event you’re scripting the deploy by way of SDK or REST.
Select a deployment template — latency- vs throughput-optimized, accelerator household, context size, quantization.
Configure occasion rely — scale throughput by including mannequin cases.
Deploy — from the portal, CLI, SDK, or REST.
Rating by way of the unified Foundry endpoint with the SDK you already use.

Deployment Templates

A deployment template is the unit of alternative in step 2: a named, versioned asset that pins the runtime, the accelerator household and rely, the context size, and the runtime-specific tuning wanted to serve the mannequin properly — so choosing a template is the one knob you flip for “how do I need this mannequin to run.”

qwen3-32b, for instance, ships with 4 templates the deploy wizard exposes facet by facet:

Every template arrives pre-tuned for the mannequin — runtime settings, tool-call and reasoning parsers, scoring path, well being probes, request concurrency, and any model-specific context-extension settings are all set by Microsoft, with any trade-offs referred to as out inline within the template description. Once you script the deploy, you reference the template and Foundry handles the remaining.

Deploy — Python SDK

from azure.identification import DefaultAzureCredential
from azure.mgmt.cognitiveservices import CognitiveServicesManagementClient

shopper = CognitiveServicesManagementClient(DefaultAzureCredential(), SUBSCRIPTION_ID)

deployment = shopper.managed_compute_deployments.begin_create_or_update(
resource_group_name=RESOURCE_GROUP,
account_name=ACCOUNT_NAME,
deployment_name=“qwen3-32b”,
useful resource={
“sku”: {“title”: “GlobalManagedCompute”, “capability”: 1},
“properties”: {
“mannequin”: “azureml://registries/azure-huggingface/fashions/qwen–qwen3-32b/variations/1”,
“deploymentTemplate”: “azureml://registries/azure-huggingface/deploymenttemplates/qwen–qwen3-32b–40k-nvidia-h100/labels/newest”,
“acceleratorType”: “H100_80GB”,
},
},
).consequence()

Rating — OpenAI SDK

The deployment is reachable by way of the unified Foundry endpoint with the OpenAI SDK — the `mannequin` discipline takes the deployment title you simply created:

from openai import OpenAI

api_key = shopper.accounts.list_keys(RESOURCE_GROUP, ACCOUNT_NAME).key1
endpoint = f”https://{ACCOUNT_NAME}.companies.ai.azure.com/openai/v1″

openai_client = OpenAI(base_url=endpoint, api_key=api_key)

completion = openai_client.chat.completions.create(
mannequin=deployment.title,
messages=[{“role”: “user”, “content”: “What is the capital of France?”}],
)

print(completion.decisions[0].message)

Use It in an Agent

A chat-completions mannequin from the Assortment slots into Foundry Brokers as an admin-connected mannequin and is callable by way of the Foundry Responses API with the identical OpenAI SDK — similar auth, similar endpoint, similar observability.

What’s Accessible At present

Accessible now in preview: the Hugging Face Assortment within the Microsoft Foundry Mannequin Catalog — 1000’s of fashions throughout each modality, refreshed weekly, deployable onto Foundry Managed Compute with NVIDIA A100, NVIDIA H100, or AMD MI300X accelerators in International and Knowledge Zone scopes, behind a unified Foundry endpoint with Playground assist, first-class Azure Monitor metrics, per-deployment billing tags, and curated runtime upgrades and CVE patching utilized routinely to your deployments.

Join the preview: kinds.cloud.microsoft/r/8Jnx1LALLA

On the roadmap: broader protection of the Hugging Face ecosystem, further accelerator households, and Convey Your Personal Weights for fine-tuned and proprietary variants deployed by way of the identical templates and governance as Assortment fashions.

Hugging Face is the place open fashions are revealed and found. Microsoft Foundry is the place enterprises operationalize them — on curated, license-screened, security-screened weights hosted in Azure; on community-built and CVE-scanned runtimes; behind a single endpoint with enterprise identification, networking, observability, and agent integration on prime. **The breadth of the open-source ecosystem, with the operational layer Microsoft runs beneath.**For a deep dive on Foundry Managed Compute — pricing, accelerator SKUs, knowledge residency, enterprise readiness, observability, and the total Responses API + reminiscence sample — see the Managed Compute launch weblog.