Overview

Qwen3.8-27B-DFlash2 is a speculative decoding draft model maintained by z-lab that accelerates inference for the base Qwen/Qwen3.8-27B model. This is not a standalone language model—it functions exclusively as a drafter component within speculative decoding servers like SGLang or vLLM. The model implements block-diffusion drafting, predicting entire blocks of tokens in a single forward pass rather than one token at a time. It maintains top candidate tokens at every position and uses a lightweight selector to trace a coherent decoding path, with two-tap dynamic convolutions preventing quality degradation toward the end of the block. Decoding is mathematically lossless: greedy sampling produces identical output to the target model, and probability distributions are preserved under sampling. The architecture requires SGLang or vLLM with speculative decoding support to function. The model trains with a context window of 4096 tokens and uses an Apache 2.0 license.

Best use cases

High-throughput inference on reasoning tasks. DFlash 2 achieves 3.43× speedup over autoregressive decoding on GSM8K at single-request concurrency and maintains 2.84× speedup with eight concurrent requests. The acceptance length—tokens accepted per verification step—reaches 5.46 on GSM8K and 5.28 on MATH-500, substantially higher than Qwen3.8's native multi-token prediction (5.02 and 4.72 respectively). Use this when serving quantitative reasoning, mathematics, or code generation at scale where latency per request matters more than model quality (the target model handles quality).

Batch serving at scale with multiple concurrent users. At concurrency 32, this drafter maintains 1.45× throughput improvement on GSM8K, reaching 1,922.5 tokens/second compared to 1,329.8 for pure autoregressive. The gains persist across all tasks, making it ideal for production systems handling dozens of simultaneous requests where wall-clock time is the primary metric. This solves the latency cliff that occurs when switching from single-request to highly concurrent workloads.

Structured generation and code completion within speculative servers. HumanEval and MBPP show consistent wins (4.39 and 4.79 acceptance length respectively), indicating the drafter effectively predicts programming patterns and logic flow. The lightweight selector preserves the target model's output distribution, so you do not trade quality for speed in structured tasks.

Low-resource deployments using existing inference infrastructure. Because DFlash 2 runs inside your existing SGLang or vLLM server, no separate hardware or model serving infrastructure is needed. The drafter requires only one additional forward pass per verification block (typically 8 tokens), amortizing to negligible overhead compared to the target model's computation.

Limitations

Not a standalone model. This draft model produces no usable output alone. You must run it inside a speculative decoding server paired with Qwen/Qwen3.8-27B as the target model. Attempting to use this model directly will fail.

Limited inference engine support. Only SGLang and vLLM (via specific pull request branches) support DFlash 2. The requirements specify vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head rather than a stable release, indicating the integration is recent and not yet in official releases. This creates a maintenance and stability risk if you depend on unreleased infrastructure.

Throughput gains diminish at high concurrency. At 32 concurrent requests, speedups drop to 1.01× on MT-Bench compared to 2.67× at single-request concurrency. This occurs because the GPU becomes compute-bound on large batches, and the drafter's contribution becomes proportionally smaller. The model does not solve the high-concurrency saturation problem.

Hardware specificity. Evaluations ran on a single NVIDIA H200 with FlashAttention 3. Throughput numbers are not transferable to consumer GPUs, older professional hardware, or non-NVIDIA platforms. Actual performance on A100, H100, or CPU inference is unknown.

Quality dependence on target model. All output quality derives from Qwen/Qwen3.8-27B's capabilities. If the base model fails a reasoning task, the drafter cannot repair it. You inherit all limitations of the 27B Qwen model, including potential biases, knowledge cutoff, and instruction-following gaps.

Speculative decoding assumption. The model assumes draft tokens will be verified by the target model. If verification fails frequently (acceptance length near 1.0), the extra computation becomes wasted cycles. This happens with tasks where token prediction is genuinely unpredictable.

How it compares

Qwen3.8-27B-DFlash2 by incoai is the original incoai checkpoint; the z-lab version is an identical mirror. No meaningful difference exists between them.

Qwen3.5-27B-DFlash targets the older Qwen3.5 base model instead of Qwen3.8. Choose DFlash 2 if you run Qwen3.8 for better acceptance lengths and throughput; use DFlash if you specifically use Qwen3.5 or require older model behavior. DFlash 2 is strictly superior on Qwen3.8 hardware-for-hardware.

Qwen3.6-27B-DFlash drafts for Qwen3.6-27B, a different base model with different quality and speed characteristics. Pick this if you standardize on Qwen3.6; otherwise DFlash 2 and Qwen3.8 are the current stable choice. The z-lab page notes Qwen3.6-27B-DFlash is still under training and lacks full engine support.

Qwen3.6-35B-A3B-DFlash pairs with the larger Qwen3.6-35B-A3B model (35 billion parameters versus 27 billion). Choose this if you need the larger model's quality and can afford the compute cost; DFlash 2 is faster and uses less VRAM while maintaining Qwen3.8-27B quality.

Qwen3.8-27B-DFlash2-GGUF provides GGUF quantized versions of the same DFlash 2 drafter for CPU and local inference backends. Select GGUF if you run llama.cpp or similar CPU-focused servers; use the native version for GPU servers like SGLang and vLLM.

Technical specifications

Architecture and training: DFlash 2 implements block-diffusion drafting using dynamic convolutions with two-tap kernels. The model predicts entire token blocks in parallel rather than autoregressively, maintaining top candidates at each position. A lightweight selector traces coherent paths through the candidate space. Context window is 4096 tokens. The model was trained to achieve lossless greedy decoding and preserve sampling distributions.

Framework and deployment: Requires SGLang or vLLM with speculative decoding support. SGLang integration uses --speculative-algorithm DFLASH and --speculative-num-draft-tokens 8 flags. vLLM uses a JSON config specifying "method": "dflash", "num_speculative_tokens": 7. Both frameworks must have FlashAttention 3 for optimal performance.

Performance characteristics:

  • Acceptance length (tokens per verification step): 5.46 on GSM8K, 5.28 on MATH-500, 4.39 on HumanEval, 4.79 on MBPP, 4.10 on MT-Bench

  • Throughput at concurrency 1: 236.1 tok/s on GSM8K (3.43× speedup), 230.7 on MATH-500 (3.34×), 214.6 on HumanEval (3.11×)

  • Throughput at concurrency 8: 1,328.7 tok/s on GSM8K (2.84× speedup), 1,368.3 on MATH-500 (2.85×)

  • Throughput at concurrency 32: 1,922.5 tok/s on GSM8K (1.45× speedup), 1,951.8 on MATH-500 (1.30×)

  • Baseline autoregressive throughput: 68.9 tok/s at concurrency 1, 467.2 at concurrency 8, 1,329.8 at concurrency 32

Evaluation setup: Tests run on NVIDIA H200 with FlashAttention 3 for both target and draft attention. Maximum new tokens set to 4096. Sampling parameters: temperature 1.0, top-p 0.95, top-k 20, xhigh reasoning effort. Compared against autoregressive baseline, Qwen3.8's native seven-token multi-token prediction (MTP), and the community DSpark drafter. All speculative methods propose seven draft tokens per verification step (block size 8 including the verification token).

License: Apache 2.0 allows commercial use, modification, and redistribution with attribution and patent claims reversion.

Model inputs and outputs

Inputs

  • Prompts in natural language or structured formats (text strings passed to target model)

  • Speculative configuration specifying draft block size (typically 7-8 tokens)

  • Batch of prompts for concurrent requests

Outputs

  • Predicted token candidates at each position in the draft block

  • Logits or probabilities for the lightweight selector to trace the best coherent path

  • Final draft tokens passed to target model for verification

  • No direct user-facing output; results come from the target model's verification step

Getting started

SGLang setup:

pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python" python -m sglang.launch_server \ --model-path Qwen/Qwen3.8-27B \ --speculative-algorithm DFLASH \ --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \ --speculative-num-draft-tokens 8
vLLM setup:

pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head" vllm serve Qwen/Qwen3.8-27B \ --speculative-config '{ "method": "dflash", "model": "incoai/Qwen3.8-27B-DFlash2", "num_speculative_tokens": 7 }'
After server launch, send requests to the OpenAI-compatible endpoint. The drafter operates transparently; you query the target model as normal, and speculative decoding accelerates inference behind the scenes.

Frequently asked questions

Q: Can I use this model commercially?

A: Yes. DFlash 2 is licensed under Apache 2.0, which permits commercial use, modification, and redistribution as long as you include attribution and acknowledge patent rights.

Q: What GPU do I need to run this?

A: The drafter itself is lightweight, but you need the target model Qwen/Qwen3.8-27B running on GPU. The 27B model requires approximately 55GB VRAM in full precision (fp16 is standard). Evaluation used an NVIDIA H200. Older hardware like A100 or H100 works; consumer GPUs are not practical.

Q: How much faster is this than plain autoregressive decoding?

A: Speedups range from 1.45× to 3.43× depending on concurrency and task. Single-request inference shows 3.43× speedup on reasoning tasks like GSM8K; at 32 concurrent requests, speedups drop to 1.45×. MT-Bench shows lower gains (2.67× at concurrency 1, 1.01× at concurrency 32).

Q: Does this change the target model's output quality?

A: No. Decoding is mathematically lossless for greedy generation, and sampling preserves the target model's probability distribution. Output quality is identical to running Qwen/Qwen3.8-27B alone.

Q: Can I fine-tune this drafter model?

A: The README does not discuss fine-tuning. The drafter is trained specifically for the Qwen3.8-27B base model. Fine-tuning it for different base models or domains is not documented and likely requires access to the training pipeline and datasets used by Inco AI.

Q: Is this model still maintained?

A: The GitHub repository is at z-lab/dflash. The model was published recently (August 2026 according to citations), and vLLM support still requires unreleased pull request branches, indicating active development. Check the GitHub for the latest status.

Q: What happens if draft tokens are rejected frequently?

A: If the drafter makes poor predictions, acceptance length drops toward 1.0, and you waste computation verifying bad tokens. This typically occurs on tasks where token sequences are genuinely unpredictable. The benchmark shows this on MT-Bench (acceptance length 4.10), where speedups are modest.

Q: Does this work with quantized versions of the target model?

A: The README does not address quantization of either the drafter or target model. In principle, speculative decoding can work with quantized targets, but compatibility is not explicitly stated.