Alibaba launched the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), its largest open-weight mannequin, bringing near-frontier capabilities to the open ecosystem. It has 2.4T whole parameters with 95B activated per token. It has 2.4T whole parameters with 95B activated per token. It’s a fine-grained combination of specialists (MoE) structure with a hybrid of full and linear consideration, a context window of as much as a million tokens, and an output size of as much as 128K, designed for demanding reasoning and agentic workloads.
Deploying a 2.4T parameter open-weight mannequin requires data-center-scale accelerated compute. Inference at this scale relies on excessive co-design throughout chips, system structure, and software program. NVIDIA is working with the open-source ecosystem to convey the mannequin to multinode deployments via optimized kernels, inference runtimes, and distributed serving recipes.
With out extra mannequin tuning, the mannequin achieves a throughput of over 4K tokens per second per GPU and over 350 tokens per second per person on NVIDIA GB300 NVL72 in FP8 precision on Day 0. Additional optimizations, together with NVFP4 precision, are anticipated to ship enhanced efficiency positive factors over time.
Architectural improvements for long-context inference
Qwen3.8-2.4T-A95B is constructed for the toughest agentic workloads like coding, large-scale doc evaluation, and long-running multi-step workflows. In contrast to chat-first fashions that ship a single immediate and obtain a single reply, agentic functions accumulate system directions, device outputs, retrieved paperwork, code, logs, and multi-step reasoning traces throughout a workflow. As context grows, consideration, compute, and KV cache reminiscence develop into the binding constraints.
The complete-attention and linear-attention hybrid structure addresses this, and the mannequin alternates between the 2. Within the full-attention layers, each token attends to each different token, and within the linear-attention layers, the rising KV cache is changed with a bounded recurrent state. Qwen3.8-2.4T-A95B retains each compute and reminiscence bounded as context scales to as much as a million tokens.
Effective-grained MoE makes the two.4T parameter rely sensible to serve. As an alternative of a small variety of giant specialists, capability is distributed throughout a bigger inhabitants of smaller specialists, bettering specialization and routing effectivity per unit of activated compute. A discovered router prompts solely the specialists wanted per token, so serving prices monitor energetic parameters, not the total 2.4T parameters, delivering frontier-scale capability at a fraction of the price of a comparable dense mannequin.
Constructed-in reasoning controls (low/excessive/xhigh) allow builders to configure inference depth per request, buying and selling compute for reasoning high quality relying on the duty: dial up for advanced multi-step reasoning or dial down for high-throughput doc processing.
Qwen3.8-2.4T-A95B optimized efficiency on GB300 NVL72
The GB300 NVL72 includes a rack-scale structure that integrates 72 NVIDIA Blackwell Extremely GPUs right into a single platform. Its giant, 72-GPU NVIDIA NVLink area allows environment friendly all-to-all communication at 130 TB/s, eliminating bottlenecks that seem when knowledgeable visitors should cross conventional off-the-shelf networks.
Out of the field, Qwen3.8 2.4T-A95B working on NVIDIA Blackwell GB300 NVL72 delivers over 4K tokens per second per GPU and over 350 tokens per second per person, enabling AI factories to run large-parameter fashions in manufacturing at excessive throughput and low latency.
Put up-train Qwen3.8-2.4T-A95B and select a serving path
NVIDIA helps a number of inference stacks to fulfill quite a lot of developer wants. SGLang, vLLM, and NVIDIA Dynamo present open-source inference recipes for builders who require larger management over efficiency on the NVIDIA-accelerated platform.
It’s additionally out there to deploy through a model-free NVIDIA NIM, a single inference container that serves any supported mannequin. Obtain the mannequin weights and deploy on Day-0 to serve fine-tuned checkpoints, and scale to manufacturing.
Builders can post-train the mannequin for domain-specific use circumstances utilizing NVIDIA NeMo AutoModel, a PyTorch-native fine-tuning library with Day-0 Hugging Face checkpoint help. Practice immediately on present checkpoints with out mannequin conversion, with help for full SFT or memory-efficient LoRA fine-tuning.
Get began with Qwen3.8-2.4T-A95B
Obtain Qwen3.8-2.4T-A95B mannequin weights from Hugging Face or ModelScope and deploy with a model-free NVIDIA NIM from NVIDIA NGC.
