As much as 3.2x Sooner Inference with LFM2.5-DSpark

Right this moment, we launch DSpark draft mannequin checkpoints for 3 fashions from our LFM2.5 household: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal reminiscence enhance for a big decoding speedup with out altering output high quality:

Sooner inference: as much as 3.18 throughput enchancment on a GPU and as much as 2.87x on-device.
Towards on-device agentic inference: cuts function-calling latency by 57% on common for LFM2.5-2.6B
Day-one assist for llama.cpp and SGLang: LFM-compatible DSpark integration is open-sourced upstream

How does DSpark work

The decode section in LLM inference is historically memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation. Speculative decoding addresses this by utilizing a light-weight draft mannequin to provide candidate tokens, then having the goal mannequin confirm all of them in a single ahead go, sharing the price of loading the weights throughout all tokens we confirm.

Through the years, a number of approaches of hypothesis have been proposed, with probably the most distinguished being EAGLE-3, DFlash, and, most lately, DSpark, which mixes three parts:

DFlash-style parallel spine conditioned on the goal mannequin’s context options, producing hidden states for all draft tokens in a single ahead go.
A light-weight sequential head, modeled as a Markov chain between neighboring tokens, that provides inter-token dependency, elevating the acceptance charge at later positions.
A confidence-scheduled verifier that predicts every token’s survival chance and prunes low-confidence suffixes when verification would value greater than it saves.

Coaching and Structure

We observe the DSpark recipe with a bigger and extra various information combine masking SFT, chat, code, and function-calling information. Based mostly on our ablations, the primary variations of the draft fashions are simplified attention-only draft fashions, with 5 layers and a block of 9. For every draft mannequin, we ran 15 epochs on the whole dataset and chosen the epoch with the best acceptance charge somewhat than the bottom loss.

The ensuing draft fashions are comparatively small, with every round ~300M parameters.

High quality parity

Beneath grasping decoding, a draft token is just accepted if it matches the goal mannequin’s distribution. On rejection, the goal mannequin’s personal token takes its place. The emitted sequence is due to this fact similar to baseline grasping by development, so benchmark accuracy (go@1 or actual match) is unchanged.

Inference Velocity Up on CPU and GPU

Our DSpark draft fashions for LFM2.5 ship with day-one assist for llama.cpp (implementation builds on prime of the official codebase, which we run with experimental steel kernels) and **SGLang (**implementation builds on the official SGLang implementation of DSpark).

We measure on-device throughput with llama.cpp and Metallic on an M4 Max MacBook Professional utilizing FP16 GGUF weights and as much as 256 output tokens. We measure GPU throughput with SGLang on a single H100 80 GB in BF16. Each configurations use a DSpark block measurement of 9, a batch measurement of 1, and a temperature of 0. We consider them on 5 benchmark datasets.

All three drafter fashions ship noticeable throughput enhancements on each the large-scale accelerator (H100) and the sting deployment (M4 Max MacBook).

For LFM2.5-2.6B, speedup on the MacBook is particularly noticeable, because it pushes the interactivity stage a person can take pleasure in far past the throughput supplied by most proprietary cloud fashions (round ~140 tok/s, relying on the dataset).

Throughout numerous multi-tool situations, DSpark reduces the latency by 57% on common for LFM2.5-2.6B.

For LFM2.5-1.2B-Instruct, we see far more variance in dataset acceptance charges, so speedup varies by as a lot as 52% relying on the underlying textual content distribution.

For LFM2.5-8B-A1B, the acceptance charge will increase in comparison with two dense fashions, but on-device we get solely an 18% enchancment on common. This hole is because of the present MoE implementation in llama.cpp’s Metallic backend, and to the truth that verifying ok tokens prompts extra specialists and thus extra weight site visitors than a single decode step.

Learn how to use LFM2.5-DSpark

Working the DSpark draft fashions with SGLang requires an SGLang construct with DSpark assist for LFM2 targets (PR #31041). Launch the goal with the draft hooked up:

python -m sglang.launch_server
–model-path LiquidAI/LFM2.5-2.6B
–speculative-algorithm DSPARK
–speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark
–speculative-draft-attention-backend flashinfer
–disable-radix-cache –mem-fraction-static 0.75 –port 30000

Then question the OpenAI-compatible endpoint at http://localhost:30000/v1. The block measurement is learn from the draft’s config.json; the baseline is identical command with out the three –speculative-* flags.

Working them with llama.cpp requires the respective llama.cpp construct (PR#27383).

llama-server -m LFM2.5-2.6B-F16.gguf
-md LFM2.5-2.6B-DSpark-F16.gguf
–spec-type draft-dspark –spec-draft-n-max 10 –spec-draft-n-min 0
-fa on -ngl 99

The block measurement is learn from the sidecar metadata (n-max is clamped to it). Speculative decoding is actual: the goal verifies each proposed token, so grasping output equals the goal alone; per-response timings report draft_n / draft_n_accepted.

Get Began

The DSpark draft mannequin checkpoints can be found on Hugging Face as Safetensors and in GGUF format:

We will’t wait to see what you construct.

Quotation

For citations, please use the next reference or BibTeX:

Liquid AI, “LFM2.5-DSpark: As much as 3.2x Sooner Inference from H100 to MacBook”, Liquid AI Weblog, Aug 2026.

@article{liquidAI2026dspark,
creator = {Liquid AI},
title = {LFM2.5-DSpark: As much as 3.2x Sooner Inference from H100 to MacBook},
journal = {Liquid AI Weblog},
12 months = {2026},
word = {www.liquid.ai/weblog/lfm2.5-dspark},
}