Alibaba launched the mannequin weights for Qwen3.8-Flash-Subsequent as a preview of the upcoming Qwen4 structure for builders to experiment with and consider. It’s a multimodal mixture-of-experts (MoE) mannequin with a 125B-parameter important mannequin supplemented by an extra 51B N-gram embeddings, with 6B parameters activated per token. It has a local 262,144-token context window, extensible to 1M tokens with YaRN.
NVIDIA supplies best-effort Day 0 practical help by SGLang, vLLM, and NVIDIA TensorRT LLM, validation throughout NVIDIA GB300 NVL72 for inference, and post-training recipes from NVIDIA NeMo AutoModel and NVIDIA NeMo RL.
Architectural improvements for long-context inference
Qwen3.8-Flash-Subsequent is designed for high-volume, context-intensive purposes similar to agentic coding, doc processing, and tool-driven workflows. As context grows, consideration compute and KV cache reminiscence grow to be bottlenecks. The mannequin addresses each with a hybrid structure combining Gated DeltaNet (GDN) and Qwen Sparse Consideration (QSA). Three out of each 4 layers use GDN to repeatedly compress historic context right into a fixed-size recurrent state, eliminating KV cache progress as sequences lengthen. The remaining layer makes use of QSA for exact retrieval throughout the total context.
Earlier sparse-attention approaches depend on token-level indexers that grow to be more and more computationally costly as context size grows. QSA aggregates the sequence into micro-blocks, estimates their significance on the block stage, and selects solely essentially the most related areas. This cuts consideration, compute, and indexing overhead inside every layer, making the design well-suited to architectures alternating between GDN and QSA layers.
Alibaba’s revealed benchmarks recommend that QSA can enhance the effectivity of 1M-token workloads. In contrast with full consideration, its consideration kernel delivered speedups of as much as 7.6x throughout prefill and 4.9x throughout decoding. In a cache-heavy on-line serving take a look at at a 1M-token context size and with a 90% prefix-cache hit charge, Qwen3.8-Flash-Subsequent achieved 8.6x the prefill throughput of Qwen3.7-Plus.
Operating Qwen3.8-Flash-Subsequent on NVIDIA GB300 NVL72
The GB300 NVL72 includes a rack-scale structure that integrates 72 NVIDIA Blackwell Extremely GPUs right into a single platform. Its massive, 72-GPU NVIDIA NVLink area permits environment friendly all-to-all communication at 130 TB/s, eliminating bottlenecks that seem when professional visitors should cross conventional off-the-shelf networks. Operating on NVIDIA GB300 NVL72 delivers over 16K tokens per second per GPU and over 200 tokens per second per consumer, enabling builders to experiment with agentic coding purposes at excessive throughput and low latency.
Past rack-scale deployment, Qwen3.8-Flash-Subsequent additionally runs on native NVIDIA {hardware}, together with NVIDIA DGX Station, NVIDIA DGX Spark clusters, and workstations with 4 NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Version GPUs. Builders can prototype and consider agentic coding workflows on native {hardware} and scale the identical mannequin to GB300 NVL72 for manufacturing serving.
Put up-train Qwen3.8-Flash-Subsequent and serve it together with your most popular inference engine
Builders can fine-tune the mannequin for domain-specific use circumstances utilizing NVIDIA NeMo AutoModel, a PyTorch-native fine-tuning library with Day-0 Hugging Face checkpoint help. Practice straight on present checkpoints with out mannequin conversion, with help for full SFT or memory-efficient LoRA fine-tuning. Customers can go a step to carry out reinforcement studying utilizing NVIDIA NeMo RL recipes.
NVIDIA helps a number of inference stacks to fulfill a wide range of developer wants. SGLang, vLLM, and TokenSpeed present open-source inference recipes for builders requiring higher management over efficiency on the NVIDIA-accelerated platform.
Get began with Qwen3.8-Flash-Subsequent
Check out the mannequin from QwenCloud.
Obtain the mannequin weights from Hugging Face or ModelScope.
