Train speculative decoding drafts for your model, your workload, your data.
DSpark · DFlash · DFlash2 · EAGLE3 · PEagle · Domino
Quick start · Draft methods · Backends · Example models · Documentation
Draftfit helps individual developers and agent builders train a smaller draft for the tasks their LLM actually handles. Start with your conversations, choose a target and draft method, then train, export and benchmark through one CLI. Inference teams and researchers can use the same workflow to compare methods.
Your target stays frozen. The draft proposes tokens; the target verifies them. Your workload. Your draft.
Source release 0.2.0 · Changelog · Upcoming work
draftfit-system-demo-en.mp4
Watch the 29-second workflow overview — an illustrated workflow, not a measured speedup demo.
- Use your own data. Prepare reviewed conversation exports with a separate holdout set.
- Make the draft yours. Configure the target, draft method, depth, feature layers and block size; fine-tune a compatible existing draft or train a new one.
- Choose how to train. Reuse offline target features or stream them to separate training workers. Recover interrupted runs and export the result.
- Measure what matters. Compare end-to-end throughput on your workload—not just training loss or token acceptance. Speedup is measured, not promised.
Your conversations → Prepare → Capture → Train → Export → Benchmark
teacher draft vs target only
Install the matching uv environment first. In a provisioned runtime, prepare your reviewed conversations and a DSpark training project:
draftfit data prepare --input ./my-conversations.jsonl \
--output ./data/train.jsonl --split-eval --eval-output ./data/holdout.jsonl
draftfit target prepare /path/to/target --local-only \
--strategy dspark --train-data ./data/train.jsonl --output-dir ./my-draft
draftfit train -c ./my-draft/train.json --plan
draftfit train -c ./my-draft/train.jsonThis online example defaults to GPU 0 for the teacher and GPU 1 for training. For saved-feature training, existing-draft fine-tuning, recovery and export, follow the complete workflow.
| Method | Offline features | Online capture | Key requirement |
|---|---|---|---|
| DSpark | Yes | Yes | Auxiliary taps and final target hidden states |
| DFlash | Yes | Yes | Compatible hidden features and mask token |
| DFlash2 | Yes | Yes | Full vocabulary; convolution and selector training |
| EAGLE3 | Yes | Yes | One draft layer; vocabulary mapping for online training |
| PEagle | No | Yes | Flex attention and method-specific feature layout |
| Domino | Yes | Yes | Explicit compatible draft configuration |
All teachers feed the same PyTorch draft trainer. These levels describe implementation and retained evidence—not universal production certification.
| Component | Role | Support level | Validation scope |
|---|---|---|---|
| PyTorch | Draft trainer | Core | Training, same-world resume and export; FSDP1 default, FSDP2 opt-in |
| SGLang | Offline / online teacher | Maintained, validated configurations | Real Ling capture, training and bounded pinned serving/state checks |
| Hugging Face Transformers | Offline / online teacher | Native, bounded GPU validation | Qwen3-0.6B BF16 feature parity; DSpark/DFlash2 train/resume/export |
| vLLM | Offline / online teacher | Native, experimental | Qwen3-0.6B FP32 parity and train/resume/export; BF16 parity failed |
| TensorRT-LLM / TokenSpeed | Teacher | Not implemented | No integration in this release |
Native HF/vLLM capture currently targets single-device Llama/Qwen2/Qwen3 text decoders. Teacher support does not imply exported-draft serving support; DFlash2 live serving remains unverified. Backend requirements and method limits →
Examples are starting configurations—not a promise that every model × method combination works. “Real-weight” below means bounded functional checks, not production or speedup certification.
| Target type | Current example | Available evidence |
|---|---|---|
| Dense + KV | Qwen3-8B, Llama3-8B | Config recipes; separate Qwen3-0.6B real-weight DSpark/DFlash2 train/resume/export checks |
| MoE + KV | Qwen3-30B-A3B | Config recipe; prior TP2 MoE capture used dummy weights |
| MoE / conv hybrid | Ling-3.0-tiny | Real-weight DSpark capture/train/export and bounded pinned serving/replay |
| Linear + KV hybrid | Qwen3.5-4B | Config recipe; full target workflow not certified |
| Conv + KV | LFM2.5-1.2B-Instruct | Real-weight load/chat smoke only; draft workflow pending |
| Recurrent | RWKV7-1.5B | Metadata/state fixture only; requires a compatible backend |
See all recipes, exact validation scope, and planned recent-model experiments. Add models through target configuration and adapters; configuration alone cannot add an unsupported capture or serving backend.
| I want to… | Guide |
|---|---|
| Install or select a teacher backend | Environment setup |
| Prepare data, train, fine-tune, resume and export | Complete workflow |
| Configure a new target or runtime | Target extension · Runtime profiles |
| Compare target-only and draft-enabled inference | Workload evaluation |
| Check supported and unverified combinations | Support matrix |
| Develop or reproduce validation | Documentation index · Source tools |
Automatic trace ingestion and continual deployment are future work. A longer accepted draft is not necessarily faster: measure end-to-end throughput on held-out requests and verify correctness separately.
Draftfit ships its own maintained training runtime derived from SpecForge, with attributed adaptations and references from TorchSpec, AngelSpec and DSpark-related projects. See code ownership, third-party notices and license.