Agentic AI adjustments the infrastructure sample for AI factories. One request can set off many mannequin calls, software calls, reminiscence lookups, coverage checks, storage accesses, and community transfers earlier than a last reply is produced. As extra brokers run directly and carry context throughout steps, customers, instruments, providers, and classes, infrastructure should transfer, defend, retrieve, and reuse knowledge quick sufficient to maintain GPUs and CPUs productive.
The NVIDIA BlueField platform brings devoted, programmable infrastructure processing within the AI manufacturing unit knowledge path. BlueField offloads infrastructure work from host CPUs, accelerates knowledge motion, enforces coverage inline, and allows context reuse. These capabilities assist ship manufacturing outcomes akin to increased GPU utilization, extra predictable latency, stronger isolation, decrease value per token, and extra tokens per watt.
It combines purpose-built infrastructure and storage processors with NVIDIA DOCA software program for AI factories. NVIDIA BlueField-4 DPUs offload, speed up, and isolate networking, storage, safety, telemetry, and control-plane providers from host CPUs whereas accelerating knowledge motion throughout GPU compute and CPU compute programs. NVIDIA Vera BlueField-4 STX storage processors energy a brand new class of knowledge platforms for context reminiscence, high-performance storage infrastructure, and safe knowledge providers throughout AI factories.
Throughout these processors, NVIDIA DOCA gives the software program basis for constructing and working these providers throughout networking, storage, safety, telemetry, and lifecycle administration. Within the NVIDIA Vera Rubin platform and the broader NVIDIA DSX structure for AI factories, BlueField gives accelerated infrastructure whereas DOCA gives the programmable software program mannequin for deploying these providers throughout the information heart.
This put up explains how agentic AI and long-context inference drive new infrastructure calls for, and the way BlueField-4, Vera BlueField-4 STX, and DOCA deal with them by offloading, accelerating, and isolating infrastructure providers throughout the AI manufacturing unit knowledge path.
Agentic AI makes infrastructure a part of inference
Agentic AI extends inference past mannequin execution right into a distributed workflow spanning GPUs, CPUs, reminiscence, networking, storage, and safety. Every step relies on shifting knowledge, preserving context, imposing coverage, and coordinating providers throughout the AI manufacturing unit, making the infrastructure knowledge path a part of the inference pipeline.
GPUs execute mannequin inference and generate tokens, whereas CPUs orchestrate the agent runtime by executing instruments, processing retrieval outcomes, making ready prompts, validating outputs, and coordinating subsequent reasoning steps. The infrastructure should additionally protect and retrieve the context that permits reasoning to proceed throughout turns.
Preserving and retrieving context is particularly essential for KV cache. Throughout prefill, an LLM creates KV cache knowledge that shops intermediate consideration state. As prompts, conversations, and agent workflows develop, cache state should more and more persist throughout reasoning steps and be reused throughout requests. When GPU reminiscence turns into constrained, programs evict and recompute KV cache, restrict context size, or transfer state into one other reminiscence tier, introducing trade-offs in latency, throughput, or value. This makes KV cache a part of the infrastructure knowledge path, the place it should be moved, positioned, protected, and retrieved with out slowing inference.
Because of this, infrastructure is not adjoining to inference. Infrastructure is now a part of the inference pipeline. Networking, storage, safety, telemetry, control-plane providers, and context-memory administration should course of agent site visitors with out delaying GPU inference or consuming host CPU assets required for agent execution.
BlueField powers the working system of the AI manufacturing unit
Agentic inference relies on a quick, safe, and programmable infrastructure knowledge path. BlueField gives the devoted infrastructure processor for that path, connecting, securing, isolating, and accelerating providers throughout the AI manufacturing unit.
BlueField-4 operates as the information processing unit (DPU) throughout Rubin GPUs and NVIDIA Vera CPUs, whereas the Vera BlueField-4 STX Storage Processor serves NVIDIA CMX for context reminiscence and AI-native storage. Throughout these roles, BlueField combines high-speed networking, embedded infrastructure compute, native reminiscence, PCIe connectivity, inline acceleration, isolation, and DOCA programmability right into a coordinated AI manufacturing unit knowledge path.
The BlueField-4 DPU integrates as much as 800 Gb/s Ethernet or InfiniBand connectivity, a 64-core NVIDIA Grace CPU, high-bandwidth LPDDR5X reminiscence, PCIe Gen6, inline acceleration for networking, storage, safety, and knowledge motion, and the DOCA software program platform.
In contrast with BlueField-3, it doubles networking bandwidth, delivers as much as 6x extra compute efficiency, 4x reminiscence capability, and greater than 3x reminiscence bandwidth.
The BlueField-4 STX Storage Processor combines the NVIDIA Vera CPU, NVIDIA ConnectX-9 SuperNIC, as much as 1.6 Tb/s of Spectrum-X Ethernet connectivity, high-performance NVMe storage entry, accelerated knowledge motion, in-silicon safety, and DOCA programmability.
DOCA makes the BlueField infrastructure-processing area programmable and gives the software program basis for constructing and deploying accelerated infrastructure providers via libraries and microservices. It offers builders, operators, and ISVs a constant technique to create and function BlueField and ConnectX-accelerated providers as necessities shift throughout KV-cache reuse, tenant isolation, storage metadata, congestion management, safe provisioning, and new agentic runtime patterns.
System-level capabilities of BlueField
Networking improves interactivity provided that embedded compute, reminiscence bandwidth, PCIe, acceleration, and software program can course of site visitors on the identical tempo. Extra compute improves throughput provided that it has enough reminiscence capability, I/O bandwidth, and programmable providers. BlueField was co-designed to stability these assets so infrastructure site visitors will be processed the place it arrives.
Excessive-speed connectivity brings AI workload, storage, safety, and management site visitors into the AI manufacturing unit knowledge path. Bluefield’s embedded compute and power-efficient LPDDR5X reminiscence maintain service logic and state near the information being processed, together with queues, insurance policies, metadata, telemetry, and KV-cache placement. PCIe Gen6, VirtIO, and DOCA SNAP storage virtualization allow hosts to make use of normal host-visible community and storage system fashions backed by BlueField acceleration to cut back host CPU overhead.
As agent requests transfer via the AI manufacturing unit, they’re carried by packets and set off RDMA transfers, storage instructions, coverage checks, metadata lookups, and telemetry occasions. BlueField accelerates the related circulate steering, knowledge motion, storage entry, encryption, integrity, and coverage enforcement so infrastructure providers can maintain tempo with agent site visitors with out consuming host CPU assets.
DOCA turns this {hardware} basis into programmable providers for the AI manufacturing unit, together with:
DOCA Host-Based mostly Networking (HBN): Helps server-side Layer 3 routing, with BlueField appearing as a BGP router for scalable multi-tenant designs.
BlueField ASTRA: Allows Spectrum-X zero-trust, multi-tenant bare-metal deployments, with BlueField appearing because the managed management level throughout a number of infrastructure planes with ConnectX-9.
DOCA Memos: Helps handle and share KV cache throughout compute and storage nodes so context will be reused throughout long-context and agentic inference.
DOCA safety providers: Assist zero-trust entry, coverage enforcement, runtime visibility, and isolation to guard knowledge, inference, and brokers throughout the AI manufacturing unit.
Collectively, these capabilities maintain networking, storage, safety, context administration, and management providers near the information path as a substitute of competing for host CPU assets. This helps BlueField enhance GPU utilization, cut back inference latency, strengthen multi-tenant isolation, decrease value per token, and enhance tokens per watt.
BlueField-4 throughout the Vera Rubin AI manufacturing unit
Excessive co-design means the AI manufacturing unit is engineered as an interdependent system. Every element has an outlined position that enhances the others, enabling the platform to maintain throughput, interactivity, isolation, and token effectivity underneath manufacturing workloads.
The Vera Rubin platform is constructed for agentic AI workloads that require high-throughput inference, dense CPU execution, large-scale context reminiscence, and safe knowledge motion. Inside this platform, Rubin GPUs present accelerated compute, and Vera CPUs help software calls, orchestration, and knowledge motion. NVLink gives scale-up communication, whereas the ConnectX-9 and Spectrum-X help scale-out networking. BlueField operates throughout this complete system, spanning compute, networking, storage, and safety (Determine 1).
GPU compute: Securing and accelerating Rubin GPUs
GPU efficiency relies on the total system’s capability to maintain accelerators provided with work, knowledge, and safe connectivity. BlueField-4 operates as an infrastructure processor within the GPU compute tray. It helps frontend (north-south) networking, host CPU offload, safe knowledge entry, administration, observability, and infrastructure isolation. By offloading and isolating these providers, BlueField helps maintain GPU compute from being gated by host CPU overhead, variable entry controls, or administration site visitors.
The north-south path is crucial for GPUs, as a result of it brings person requests, retrieved knowledge, context, storage site visitors, and administration providers into the Rubin compute layer. BlueField-4 expands and secures that frontend path, whereas inline infrastructure processing handles the related networking, storage, safety, and telemetry work earlier than it consumes host CPU cycles. Increased frontend bandwidth and decrease host CPU competition assist enhance knowledge and context availability to GPUs, supporting extra predictable inference latency and better tokens per second per person throughout the AI manufacturing unit.
CPU execution: Isolating infrastructure from Vera agentic workloads
Agentic AI and reinforcement studying enhance demand for CPU efficiency as a result of brokers execute software calls, run code, browse or question knowledge, remodel inputs, parse outputs, consider outcomes, and orchestrate workflows. The Vera CPU combines excessive per-core efficiency, concurrency, and power-efficient reminiscence bandwidth to run this CPU execution layer at AI manufacturing unit scale. BlueField-4 gives the infrastructure-processing layer round Vera CPU workloads, dealing with front-end networking, storage entry, safety and isolation, provisioning, coverage, telemetry, and administration.
This separation issues as a result of agentic CPU workloads sit contained in the reasoning loop. If infrastructure providers eat host CPU cycles or add scheduling jitter, software execution, retrieval, and validation can decelerate even when GPU compute is accessible. BlueField processes networking, storage, safety, and control-plane providers within the DPU area, enabling Vera CPUs to spend extra time on agent execution and fewer time on infrastructure work.
AI-native storage and context reminiscence
Agentic AI turns context into lively infrastructure knowledge. Multi-turn brokers, long-context reasoning, retrieval-augmented workflows, and multi-agent programs generate reusable inference, together with KV cache, that should be saved, shared, protected, and retrieved with out stalling inference.
The NVIDIA CMX context reminiscence storage platform creates a shareable, scalable, and power-efficient AI-native storage tier for inference context between GPU reminiscence and scalable shared storage. It delivers an Ethernet-attached flash tier optimized for KV cache utilizing the Vera BlueField-4 STX storage processor, which mixes Vera CPU, ConnectX-9 networking, and DOCA Memos.
The storage processor runs KV I/O, metadata administration, knowledge placement, safety, and management operations near the storage and community path. As a substitute of presenting flash as block storage, it tracks KV metadata, manages queues, helps cache recall and pre-staging, enforces tenant coverage, and handles knowledge safety. Putting these control-heavy, latency-sensitive operations within the storage processor helps protect interactivity as context grows.
With DOCA Memos, CMX can handle and share KV cache throughout AI compute and CMX knowledge nodes, making it an lively context-memory knowledge path quite than a passive storage tier. KV-cache reuse reduces repeated prefill, context recomputation, GPU idle time, and pointless knowledge motion, enhancing tokens per second and energy effectivity for long-context and agentic workloads.
In-silicon safety for knowledge, inference, and brokers
Agentic AI adjustments the safety mannequin as a result of brokers repeatedly entry knowledge, fashions, instruments, context reminiscence, and inference providers throughout AI factories. They might course of proprietary or regulated knowledge, mannequin state, embeddings, and KV cache earlier than producing a last reply. This makes knowledge, inference, and agent conduct a part of the safety floor.
Safety for AI factories can’t rely solely on host software program. Host-resident safety controls share assets and belief boundaries with the workloads they defend, which might expose them to tampering or evasion if the host is compromised. BlueField strikes safety processing in-silicon and outdoors the host workload area, strengthening the management boundary whereas preserving CPU and GPU assets for AI work.
For multi-tenant AI factories, this permits inline safety with out slowing the information path. BlueField gives a trusted infrastructure management level for tenant isolation, community coverage enforcement, safe entry, runtime detection, encryption, and telemetry throughout compute, storage, and inference infrastructure. DOCA extends this mannequin via programmable providers for zero-trust knowledge entry, inference safety, agent conduct visibility, and network-level isolation, serving to defend fashions, datasets, context reminiscence, and runtime interactions as agentic workloads scale.
Study extra
Agentic AI makes infrastructure a part of the inference pipeline, requiring AI factories to maneuver, defend, retrieve, and reuse knowledge with out slowing GPUs, CPUs, or context reminiscence. BlueField gives accelerated infrastructure processing for networking, storage, safety, telemetry, and context providers, whereas DOCA makes these providers programmable and deployable throughout the information heart. Collectively, BlueField-4, Vera BlueField-4 STX, and DOCA assist AI factories enhance interactivity, isolation, GPU utilization, value per token, and tokens per watt.
Get began
Obtain NVIDIA DOCA, set up the most recent launch, and construct the samples to start out programming BlueField for accelerated networking, storage, safety, telemetry, and lifecycle administration.
