Hugging Face Models on Foundry Managed Compute

At Microsoft Build 2026, the company unveiled a significant expansion to its AI infrastructure: Foundry Managed Compute and the integration of Hugging Face models on Foundry. This initiative introduces a curated, weekly-refreshed catalog of open-weight models from the Hugging Face ecosystem, designed for one-click deployment onto Microsoft’s managed infrastructure. By pre-staging model weights within Azure and utilizing runtimes that are built, scanned, and maintained by Microsoft, the platform ensures that every model in the collection inherits the enterprise-grade security, governance, observability, and billing standards synonymous with the broader Foundry ecosystem.

The Platform: Microsoft Foundry and Managed Compute

Microsoft Foundry serves as a comprehensive environment for the development and operation of agentic AI applications. The platform distinguishes itself by offering the most extensive model selection available on any cloud provider, encompassing offerings from Microsoft, OpenAI, Anthropic, Meta, Mistral, DeepSeek, and Hugging Face. Whether developers are working with frontier models, open-source weights, or custom-tuned variants, they interact with a unified endpoint and a consistent set of SDKs across Python, C#, JavaScript, and Java.

At the core of this ecosystem is the Foundry Agent Service. This service enables multi-agent orchestration, featuring built-in memory, knowledge grounding via Foundry IQ, and a robust catalog of tools accessible through agentic protocols. Once agents are deployed, Foundry provides a full suite of operational tools, including end-to-end tracing, real-time monitoring, continuous evaluations, and a prompt optimizer that refines agent behavior based on performance metrics.

Developers also benefit from a sophisticated security and management layer:

• Content safety filters and task-adherence guardrails.
• An AI Red Teaming Agent for proactive adversarial testing.
• Unified Role-Based Access Control (RBAC) and private networking.
• Direct integration with Azure Policy.

Foundry Managed Compute represents the third primary deployment option within the platform, joining pay-per-token and provisioned throughput models. It functions as a managed GPU platform-as-a-service, allowing developers to define deployments based on workload requirements—such as parameter count, context length, and latency versus throughput optimization—while Microsoft handles the underlying GPU topology.

"Microsoft takes care of the machine: container updates, runtime upgrades, and security patches happen automatically on the supported runtimes—vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama.cpp—without redeploying your model, while model configuration, deployment behavior, and routing stay with you."

Why Hugging Face?

Hugging Face has effectively become the "public square" of the open AI movement, boasting a community of 15 million builders and 400,000 organizations. With over 3 million open models available, it serves as the central hub for innovation in agentic coding, video segmentation, speech synthesis, and embeddings.

The integration of these models into Foundry addresses the "operational gap" that often prevents enterprises from leveraging open-source assets. While Hugging Face provides the community and the weights, it is not inherently an enterprise-serving platform. Microsoft Foundry bridges this gap by providing the necessary discovery, license review, security screening, and infrastructure management.

The Strategic Advantages of Open Models

• Competitive Performance: Leading open-weight models now rival the top closed-source frontier models across major benchmarks.
• Deep Customization: Full access to weights allows for fine-tuning, distillation, quantization, and LoRA adaptation to meet specific domain and cost requirements.
• Data Sovereignty: Weights run within the user’s tenant, behind private inference endpoints, and within defined network boundaries.
• Cost Predictability: By paying for accelerators by the hour and scaling to zero when idle, organizations can better manage costs for high-volume or latency-sensitive workloads.
• Version Control: Teams can pin specific model versions, evaluate them, and manage their own deployment and rollback cadences.

The Hugging Face Collection on Foundry

The Hugging Face Collection brings a curated subset of these models directly into the Foundry Model Catalog. Every model undergoes a rigorous multi-stage publishing pipeline before it becomes available for deployment.

• Weekly Refresh: Trending models are added continuously to reflect the rapid pace of community innovation.
• Broad Modality Support: The collection includes LLMs, VLMs, ASR, speech translation, embeddings, and image generation models.
• Security First: Only SafeTensors weights are utilized. No untrusted code is executed, and any model requiring trust_remote_code is either remediated or excluded.
• Optimized Runtimes: Foundry automatically selects the best-fit engine—such as vLLM or SGLang for LLMs, or TEI for embeddings—to ensure peak performance.

The Curation Pipeline

To ensure enterprise readiness, Microsoft and Hugging Face have implemented a systematic curation process:

1. Identification: Trending models are selected based on community signals, partner requests, and customer demand. 2. Compliance Screening: Licenses are reviewed against Microsoft’s enterprise distribution policy, and repositories are audited for security vulnerabilities. 3. Runtime Hardening: Microsoft builds and signs inference container images, scanning them for CVEs before publishing them to a managed registry. 4. Secure Storage: Model weights are validated and stored in Microsoft-managed Azure storage, eliminating the need for outbound network access to the Hugging Face Hub during production deployments. 5. Performance Validation: Every model, runtime, and accelerator combination is tested for API conformance and performance metrics (latency, throughput, and time-to-first-token).

Powering Performance with Advanced Runtimes

Foundry Managed Compute utilizes a versatile array of community-built, open-source inference runtimes, each tuned for specific architectures:

• vLLM: The default high-throughput engine for large language models, offering immediate support for new models appearing on the Transformers library.
• SGLang: Optimized for agentic and tool-using workloads that require structured outputs like JSON or grammar-constrained generation.
• Text Embeddings Inference (TEI): A lean, high-performance runtime for embedding and reranking models, essential for RAG (Retrieval-Augmented Generation) pipelines.
• llama.cpp: Provides a cost-optimized path for CPU-based or smaller GPU deployments using GGUF-quantized models.
• TensorRT-LLM and NIM: Leveraged on NVIDIA hardware to deliver superior latency and throughput through optimized kernels.
• hf-serve: Used for specialized vision, audio, and segmentation pipelines that fall outside the standard LLM fast paths.

Deploying and Scoring: A Five-Step Workflow

Deployment within the Foundry environment is designed to be intuitive, whether through the portal, CLI, or SDK.

1. Browse: Select a model from the Foundry Model Catalog. 2. Template Selection: Choose a deployment template that defines the runtime, accelerator family, context length, and quantization. 3. Configuration: Set the instance count to scale throughput. 4. Deploy: Execute the deployment via the portal or code. 5. Score: Interact with the model using the unified Foundry endpoint and standard OpenAI-compatible SDKs.

Example: Deployment via Python SDK

from azure.identity import DefaultAzureCredential
from azure.mgmt.cognitiveservices import CognitiveServicesManagementClient

client = CognitiveServicesManagementClient(DefaultAzureCredential(), SUBSCRIPTION_ID)
deployment = client.managed_compute_deployments.begin_create_or_update(
    resource_group_name=RESOURCE_GROUP,
    account_name=ACCOUNT_NAME,
    deployment_name="qwen3-32b",
    resource={
        "sku": {"name": "GlobalManagedCompute", "capacity": 1},
        "properties": {
            "model": "azureml://registries/azure-huggingface/models/qwen--qwen3-32b/versions/1",
            "deploymentTemplate": "azureml://registries/azure-huggingface/deploymenttemplates/qwen--qwen3-32b--40k-nvidia-h100/labels/latest",
            "acceleratorType": "H100_80GB",
        },
    },
).result()

Future Outlook

The Hugging Face Collection is currently available in preview, supporting NVIDIA A100, H100, and AMD MI300X accelerators. Microsoft has committed to an aggressive roadmap, which includes broader ecosystem coverage, support for additional accelerator families, and a "Bring Your Own Weights" feature for fine-tuned and proprietary model variants.

By combining the vast, rapid innovation of the open-source community with the operational rigor of Microsoft’s enterprise infrastructure, this integration marks a turning point for organizations looking to operationalize open-weight models at scale. Whether for RAG, agentic workflows, or specialized domain tasks, Foundry Managed Compute provides the security, governance, and performance required to turn open-source potential into enterprise reality.