Skip to content
JayYun98Public

About

Train and adapt speculative decoding drafts for LLM inference: DSpark, DFlash, DFlash2, EAGLE3, and official EAGLE4 checkpoints.

Topics

Resources

Stars

5 stars

Watchers

0 watching

Forks

Latest commit

 

History

8 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Draftfit

English | 한국어

Train speculative decoding drafts for your model, your workload, your data.

DSpark · DFlash · DFlash2 · EAGLE3 · PEagle · Domino

Quick start · Draft methods · Backends · Example models · Documentation

Draftfit helps individual developers and agent builders train a smaller draft for the tasks their LLM actually handles. Start with your conversations, choose a target and draft method, then train, export and benchmark through one CLI. Inference teams and researchers can use the same workflow to compare methods.

Your target stays frozen. The draft proposes tokens; the target verifies them. Your workload. Your draft.

Source release 0.2.0 · Changelog · Upcoming work

draftfit-system-demo-en.mp4

Watch the 29-second workflow overview — an illustrated workflow, not a measured speedup demo.

Why Draftfit?

  • Use your own data. Prepare reviewed conversation exports with a separate holdout set.
  • Make the draft yours. Configure the target, draft method, depth, feature layers and block size; fine-tune a compatible existing draft or train a new one.
  • Choose how to train. Reuse offline target features or stream them to separate training workers. Recover interrupted runs and export the result.
  • Measure what matters. Compare end-to-end throughput on your workload—not just training loss or token acceptance. Speedup is measured, not promised.
Your conversations → Prepare → Capture → Train → Export → Benchmark
                              teacher    draft            vs target only

Quick start

Install the matching uv environment first. In a provisioned runtime, prepare your reviewed conversations and a DSpark training project:

draftfit data prepare --input ./my-conversations.jsonl \
  --output ./data/train.jsonl --split-eval --eval-output ./data/holdout.jsonl

draftfit target prepare /path/to/target --local-only \
  --strategy dspark --train-data ./data/train.jsonl --output-dir ./my-draft

draftfit train -c ./my-draft/train.json --plan
draftfit train -c ./my-draft/train.json

This online example defaults to GPU 0 for the teacher and GPU 1 for training. For saved-feature training, existing-draft fine-tuning, recovery and export, follow the complete workflow.

Draft methods

Method Offline features Online capture Key requirement
DSpark Yes Yes Auxiliary taps and final target hidden states
DFlash Yes Yes Compatible hidden features and mask token
DFlash2 Yes Yes Full vocabulary; convolution and selector training
EAGLE3 Yes Yes One draft layer; vocabulary mapping for online training
PEagle No Yes Flex attention and method-specific feature layout
Domino Yes Yes Explicit compatible draft configuration

Backends

All teachers feed the same PyTorch draft trainer. These levels describe implementation and retained evidence—not universal production certification.

Component Role Support level Validation scope
PyTorch Draft trainer Core Training, same-world resume and export; FSDP1 default, FSDP2 opt-in
SGLang Offline / online teacher Maintained, validated configurations Real Ling capture, training and bounded pinned serving/state checks
Hugging Face Transformers Offline / online teacher Native, bounded GPU validation Qwen3-0.6B BF16 feature parity; DSpark/DFlash2 train/resume/export
vLLM Offline / online teacher Native, experimental Qwen3-0.6B FP32 parity and train/resume/export; BF16 parity failed
TensorRT-LLM / TokenSpeed Teacher Not implemented No integration in this release

Native HF/vLLM capture currently targets single-device Llama/Qwen2/Qwen3 text decoders. Teacher support does not imply exported-draft serving support; DFlash2 live serving remains unverified. Backend requirements and method limits →

Example models

Examples are starting configurations—not a promise that every model × method combination works. “Real-weight” below means bounded functional checks, not production or speedup certification.

Target type Current example Available evidence
Dense + KV Qwen3-8B, Llama3-8B Config recipes; separate Qwen3-0.6B real-weight DSpark/DFlash2 train/resume/export checks
MoE + KV Qwen3-30B-A3B Config recipe; prior TP2 MoE capture used dummy weights
MoE / conv hybrid Ling-3.0-tiny Real-weight DSpark capture/train/export and bounded pinned serving/replay
Linear + KV hybrid Qwen3.5-4B Config recipe; full target workflow not certified
Conv + KV LFM2.5-1.2B-Instruct Real-weight load/chat smoke only; draft workflow pending
Recurrent RWKV7-1.5B Metadata/state fixture only; requires a compatible backend

See all recipes, exact validation scope, and planned recent-model experiments. Add models through target configuration and adapters; configuration alone cannot add an unsupported capture or serving backend.

Documentation

I want to… Guide
Install or select a teacher backend Environment setup
Prepare data, train, fine-tune, resume and export Complete workflow
Configure a new target or runtime Target extension · Runtime profiles
Compare target-only and draft-enabled inference Workload evaluation
Check supported and unverified combinations Support matrix
Develop or reproduce validation Documentation index · Source tools

Automatic trace ingestion and continual deployment are future work. A longer accepted draft is not necessarily faster: measure end-to-end throughput on held-out requests and verify correctness separately.

Acknowledgments

Draftfit ships its own maintained training runtime derived from SpecForge, with attributed adaptations and references from TorchSpec, AngelSpec and DSpark-related projects. See code ownership, third-party notices and license.

About

Train and adapt speculative decoding drafts for LLM inference: DSpark, DFlash, DFlash2, EAGLE3, and official EAGLE4 checkpoints.

Topics

Resources

Stars

5 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages