Lengthy-running AI brokers spend most of their time on high-volume execution: instrument calls, end result validation, and subagent delegation. Utilizing a frontier reasoning mannequin for each execution step provides price and latency.
NVIDIA Nemotron 3.5 Lightning is an open 30B mixture-of-experts (MoE) mannequin with 3B energetic parameters constructed for that execution layer of always-on brokers. It’s designed for harnesses like OpenClaw and Hermes Agent—all supported by the NVIDIA NemoClaw open supply safety and administration stack for working always-on AI brokers.
The NVIDIA Nemotron open mannequin household is sort of a software program library, with every launch constantly enhancing accuracy and pace. As these fashions evolve, the speedy maturation of mannequin routing and orchestration can be underway.
That is vital as a result of builders more and more construct purposes with a system of fashions. Frontier reasoning fashions similar to Nemotron 3 Extremely deal with orchestration and sophisticated planning whereas smaller, extra environment friendly fashions deal with the high-volume execution layer.
This put up introduces NVIDIA Nemotron 3.5 Lightning and explains how its smaller MoE design is optimized for high-volume, low-latency execution in autonomous brokers. It additionally particulars the inference and coaching improvements that energy it. Lastly, the put up additionally introduces NVIDIA NeMo Switchyard, a library that intelligently routes every process to the very best mannequin for the job.
Why is Nemotron 3.5 Lightning excellent for long-running AI brokers?
Nemotron 3.5 Lightning is a customizable open 30B MoE mannequin with 3B energetic parameters, offering optimum high-volume execution for autonomous brokers. MoEs are quick and environment friendly as a result of a router sends every token to just some of its many specialists, so solely a fraction of the mannequin’s parameters run per token. This offers the capability of a bigger dense mannequin on the compute price of a small one.
Nemotron 3.5 Lightning is the smallest member of the Nemotron 3 mannequin household and ships with most of the identical methods confirmed throughout the household, together with:
Speculative decoding: Multi-token prediction was included throughout Nemotron 3.5 Lightning coaching (as for Nemotron 3 Tremendous and Nemotron 3 Extremely). Nemotron 3.5 Lightning additionally ships with DFlash and DSpark, enabling extra complete inference optimization throughout a spread of serving eventualities.
Harness-optimized coaching: The mannequin is skilled for well-liked agent harnesses, enabling brokers to make extra correct calls whereas decreasing latency for high-volume duties.
The result’s a mannequin constructed for execution-focused, excessive name volumes, and low latency—all at a measurement that deploys wherever from an NVIDIA DGX Spark to a knowledge middle.
Customise Nemotron 3.5 Lightning out of the field
Fashions earn their place in specialised AI agent techniques once they’re tailored to the job. And Lightning-class fashions are extremely customizable: small fashions fine-tune quicker, cheaper, and on much more modest {hardware} than their bigger counterparts.
You’ll be able to customise Nemotron 3.5 Lightning out of the field to suit your workload. As with each Nemotron open mannequin launch, the weights, coaching information, and recipes are launched as permissively as doable underneath OpenMDW-1.1, so you may:
This launch contains Nemotron-RL Agentic Terminal Pivot, an open agentic reinforcement studying dataset used to coach among the coding agent capabilities.
Route work to the fitting mannequin utilizing NeMo Switchyard
Whereas frontier fashions might win the headlines, fashions like Nemotron 3.5 Lightning earn their medals within the trenches. They deal with requests like git pull, validate instrument outputs, format outcomes, and run the routine calls that dominate any long-running agent’s token funds.
Mannequin routing and orchestration assist make this division of labor extra accessible. They’re now accessible by means of NVIDIA NeMo Switchyard. Switchyard can expose Nemotron 3.5 Lightning as a routing goal alongside your open and closed fashions, so each request lands on essentially the most succesful and environment friendly mannequin that may deal with it. Plans route as much as the frontier, execution routes right down to Lightning, making certain that your tokens are spent effectively and successfully.
How does Nemotron 3.5 Lightning carry out on the accuracy-speed Pareto frontier?
Nemotron 3.5 Lightning delivers main accuracy on the highest output pace in its class, profitable the accuracy-versus-speed Pareto frontier on the Synthetic Evaluation Intelligence Index. This index combines 9 evaluations to measure mannequin efficiency throughout agentic duties, coding, scientific reasoning, and normal intelligence.
Nemotron 3.5 Lightning combines sturdy intelligence with as much as 4x output pace of similar-sized fashions, inserting it on the accuracy-speed Pareto frontier for high-volume agent workloads.
Agent effectivity in the end comes right down to how rapidly a mannequin completes helpful work and never merely how briskly it generates tokens. On PinchBench, Nemotron 3.5 Lightning reaches 86% accuracy whereas finishing 10,000 duties 30% quicker than Qwen3.6 35B at related accuracy.
Increased inference throughput and token effectivity locations Nemotron 3.5 Lightning on the effectivity frontier, serving to always-on brokers end high-volume work quicker.
How does Nemotron 3.5 Lightning ship pace with out compromising accuracy?
Nemotron 3.5 Lightning delivers pace and customization with out compromising accuracy by means of speculative decoding, and quantization.
Speculative decoding
Nemotron 3.5 Lightning is constructed to rapidly generate tokens and has the power to generate a number of tokens by means of speculative decoding. It is a course of whereby the mannequin, or draft mannequin, will draft some variety of tokens that are effectively reviewed. Nemotron 3.5 Lightning underwent a devoted pretraining stage to bake multi-token prediction (MTP) into the mannequin, as with Nemotron 3 Tremendous and Extremely. After coaching, a devoted MTP-boosting part additional improved MTP accuracy.
Past MTP, two draft fashions are supplied with Nemotron 3.5 Lightning: DSpark, which is advisable for DGX Spark inference workloads and low concurrency information middle workloads. MTP is finest fitted to medium to excessive concurrency, with the optimum draft size lowering as concurrency will increase. NVIDIA can be releasing a DFlash draft mannequin, which will be measured towards the others and will carry out finest on your workloads.
Quantization
Nemotron 3.5 Lightning ships with an NVFP4 checkpoint alongside BF16, utilizing the identical specialised NVFP4 kernels that energy Nemotron 3 Extremely throughout NVIDIA Blackwell, NVIDIA Hopper, and NVIDIA Ampere GPUs. The identical file serves simply as effectively in information facilities because it does in your desktop DGX Spark.
How is Nemotron 3.5 Lightning excellent for native AI?
Nemotron 3.5 Lightning makes succesful agentic AI accessible on native techniques together with NVIDIA Jetson, GeForce RTX 5090, and DGX Spark.
NVIDIA has labored with various groups together with EXO Labs to grasp how this mannequin performs on DGX Spark.
As well as, you may run Nemotron 3.5 Lightning throughout the business customary set of instruments, together with LM Studio, llama.cpp, Ollama, and Unsloth.
Companion ecosystem
Nemotron 3.5 Lightning is supported by a rising ecosystem of companions throughout harnesses, customization, deployment, and inference, together with:
Submit-training: AgileRL, Utilized Compute, Deep Cogito, distil labs, Fastino Labs, Locai Labs, Prime Mind, Cheap, Considering Machines Lab, Thoughtworks, Trajectory, Uniphore
Inference software program: Ollama, Exo, Canonical, LM Studio, Unsloth
Harnesses and agent frameworks: Aible, Cline, Manufacturing unit AI, Hermes Agent, Kilo Code, LangChain, LM Studio Bionic, OpenClaw, OpenCode, OpenHands, Pi
Cloud service supplier platforms: Amazon SageMaker JumpStart, Google Cloud Gemini Enterprise Agent Platform, MSFT Foundry, OCI Enterprise AI
GSI: Accenture, Tata Consultancy Providers, Tech Mahindra, Wipro
AI natives: Arcos Labs, CodeRabbit, Dream, Harvey
Hosted inference service suppliers: Baseten, Bitdeer AI, BlackBox AI, CoreWeave, Crusoe, DeepInfra, Fireworks AI, FriendliAI, GMI Cloud, Modal, Nebius, Collectively AI
Begin constructing with Nemotron 3.5 Lightning
Nemotron 3.5 Lightning is absolutely open—weights, information, and recipes—so you may adapt it to your workflows and deploy it wherever. To get began, attempt it on construct.nvidia.com or by means of OpenRouter. Obtain the weights from Hugging Face, and ModelScope. Wish to dive deeper?
Keep updated on NVIDIA Nemotron by subscribing to NVIDIA information and following NVIDIA AI on LinkedIn, X, Discord, and YouTube.
Go to the Nemotron developer web page for sources to get began. Discover open Nemotron fashions and datasets on Hugging Face, ModelScope, and Blueprints on construct.nvidia.com.
Have interaction with Nemotron livestreams, tutorials, and the developer neighborhood on the NVIDIA discussion board and Discord.
