Builders constructing real-time AI—reminiscent of chat assistants, copilots, and agentic workflows—are sometimes constrained by token-by-token era velocity. This limits responsiveness, will increase serving prices, and makes fluid, interactive experiences troublesome to attain.
DiffusionGemma, created by Google DeepMind and optimized to run effectively throughout NVIDIA platforms, introduces a brand new method to textual content era, producing tokens in parallel moderately than one after the other, enabling sooner, higher-throughput AI functions. The mannequin makes use of diffusion-based denoising to generate 256 tokens in parallel per step, delivering as much as 1,000 tokens/sec on a single NVIDIA H100 Tensor Core GPU, as much as 150 tokens/sec on NVIDIA DGX Spark, and as much as 2,000 tokens/sec on NVIDIA DGX Station.
For enterprise builders, this velocity interprets into decrease serving prices, larger concurrency, and extra responsive consumer experiences with out sacrificing mannequin high quality. DiffusionGemma is constructed on the Gemma 4 26B A4B MoE structure and optimized for low-latency, memory-bound inference.
Along with NVIDIA information heart GPUs, builders can take pleasure in optimum efficiency on a wide range of consumer GPUs and programs.
Construct and prototype on NVIDIA
Entry DiffusionGemma by way of Hugging Face Transformers for preliminary testing and prototyping on NVIDIA GeForce RTX 5090 or DGX Spark. For larger throughput or concurrent multi-user serving on DGX Spark, DGX Station, and RTX PRO, use vLLM by following our playbooks in Desk 2.
With Day 0 help throughout NVIDIA {hardware} and software program—from native prototyping to manufacturing deployment—builders can rapidly transfer from experimentation to real-world functions. NVIDIA GPU-accelerated endpoints
Begin constructing with DiffusionGemma with free entry for prototyping to GPU-accelerated endpoints on construct.nvidia.com as a part of the NVIDIA Developer Program. The browser expertise will also be related to customized information sources.
BF16 and NVFP4
The mannequin is offered as we speak on Hugging Face with BF16 checkpoints, and an NVFP4 quantized checkpoint for DiffusionGemma can be out there utilizing NVIDIA Mannequin Optimizer.
Enterprise deployments with NVIDIA NIM
NVIDIA NIM makes it easy to deploy DiffusionGemma from growth into manufacturing. NIM packages the mannequin as an optimized, containerized inference microservice — with efficiency tuning, standardized APIs, and the pliability to run on-premises, within the cloud, or throughout hybrid environments. NIM exposes a regular OpenAI-compatible API for sending inference requests to the server.
Obtain the container.
Begin the NIM server.
$ docker run –gpus=all
-e NGC_API_KEY=$NGC_API_KEY
-v “$LOCAL_NIM_CACHE:/decide/nim/.cache”
-p 8000:8000
${NIM_IMAGE_PATH}
Make a take a look at request and skim the total NIM documentation.
consumer = OpenAI(
base_url=”http://localhost:8000/v1″,
api_key=”not-required”
)
response = consumer.chat.completions.create(
mannequin=”google/diffusiongemma-26b-a4b-it”,
messages=[
{“role”: “user”, “content”: “Write a poem about text diffusion”}
],
max_tokens=256
)
print(response.decisions[0].message.content material)
Day 0 finetune with NVIDIA NeMo AutoModel
Positive-tuning guides and recipes can be found by way of the NVIDIA NeMo AutoModel library, a part of the NVIDIA NeMo Framework, for builders trying to adapt the mannequin to particular duties or domains. NeMo AutoModel permits customers to fine-tune fashions (LLMs, VLMs and DiffusionLMs) immediately on high of HuggingFace checkpoints with out conversion, so customers can begin fast experimentation on the newest frontier fashions.
NVIDIA is an lively contributor to the open-source ecosystem and has launched a number of hundred initiatives below open-source licenses. NVIDIA is dedicated to open fashions reminiscent of DiffusionGemma that promote AI transparency and allow customers to share their work in AI security and resilience.
Take a look at DiffusionGemma on Hugging Face or take a look at without cost utilizing NVIDIA APIs at construct.nvidia.com.
