DFlash split 1: Qwen3Next tap/tape foundation

[dusterbloom]

Why DFlash — and why alongside MTP

DFlash is a block-diffusion drafter for speculative decoding: a second draft path next to the existing MTP head — not a replacement. It is fully opt-in and per-request selectable (speculation: auto | dflash | mtp | none; auto uses DFlash when a drafter is loaded, otherwise falls back to MTP/AR), so it adds no risk to the current path.

Why carry it: DFlash out-throughputs MTP at matched settings. The upstream drafter (modal-labs) reports up to 5.01× @ concurrency 1 and 2.58× @ 32 — higher throughput than MTP at every matched setting where both completed. Locally on Qwen3.5-9B (4-bit MLX) we measure ~87% accept length (14/16 tokens), ~58 tok/s, with streamed output byte-identical to non-streaming (asserted by dflash_streaming_matches_nonstreaming). MoE targets are its sweet spot — cheaper verify amortization — directly relevant to the Qwen3.x MoE models this repo runs.

It reuses the existing speculative-decoding infrastructure (realized-speedup gate, accept_prefix), so MTP and DFlash share one machine and the engine picks the better drafter per request.

---

What

Split from #204. This PR contains only the Qwen3Next foundation needed by DFlash: tap-returning forward paths, GDN tape recording/replay, rollback support, and AnyModel/AnyCache dispatch helpers.

Why

This is the largest mechanical/model-internal part of the DFlash feature, separated so reviewers can focus on correctness of the target-model tape/rollback machinery before looking at the drafter or engine integration.

Validation

• cargo check -p higgs-models

Stack

1. This PR: Qwen3Next tap/tape foundation
2. DFlash drafter module
3. Engine draft-verify loop
4. Config, doctor validation, and docs

Summary by CodeRabbit

Release Notes

• New Features

  • Added conversion methods for accessing hybrid cache with appropriate error handling
  • Introduced specialized inference methods for DFlash and Qwen3Next model operations
  • Added standardized type alias for tape output shapes

• Chores

  • Updated GitHub Actions CI runner environment

[coderabbitai]

📝 Walkthrough

Walkthrough

Pins all five CI job runners from macos-latest to macos-15. Adds two hybrid-cache borrow accessors (as_hybrid, as_hybrid_mut) to AnyCache, a TapsTapeOutput public type alias, and five new AnyModel dispatcher methods for Qwen3Next/DFlash: forward_with_taps, forward_with_taps_tape, replay_tape_rollback, embed_token_ids, and forward_all_logits_from_hidden.

Changes

Qwen3Next/DFlash public API

Layer / File(s)	Summary
AnyCache hybrid accessors and TapsTapeOutput type alias crates/higgs-models/src/lib.rs	Adds as_hybrid and as_hybrid_mut to AnyCache returning Exception for KV-only cache, and introduces the TapsTapeOutput public type alias for the taps-tape output tuple.
AnyModel Qwen3Next/DFlash dispatcher methods crates/higgs-models/src/lib.rs	Adds forward_with_taps, forward_with_taps_tape, replay_tape_rollback, embed_token_ids, and forward_all_logits_from_hidden as thin dispatchers; each succeeds only for Qwen3Next with a Hybrid cache and returns a descriptive Exception for all other model/cache combinations.

CI Runner Pin

Layer / File(s)	Summary
Pin all CI jobs to macos-15 .github/workflows/ci.yml	Changes runs-on from macos-latest to macos-15 for the test, build, lint, msrv, and coverage jobs.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 Hop, hop, the cache knows its kind,
Hybrid or KV — no guessing behind!
Five new dispatchers line up in a row,
Qwen3Next leads them wherever they go.
The runner is pinned, no drifting allowed,
macos-15 stands steady and proud!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly relates to the main changeset, which introduces foundational DFlash feature components for Qwen3Next with tape/tape recording capabilities.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/higgs-models/src/lib.rs`:
- Around line 861-895: The `forward_with_taps` and `forward_with_taps_tape`
methods accept an optional mask parameter but the underlying Qwen3Next
implementation ignores it and builds causal masks internally, causing external
masks to be silently discarded. Add a guard condition at the beginning of each
method that checks if `mask.is_some()` and returns an error with a clear message
indicating that external masks are not supported by this operation, before
proceeding to the match statement that dispatches to the underlying
implementation.
- Around line 903-912: In the replay_tape_rollback method, add validation checks
before the match statement to ensure n_accepted and kv_rollback are non-negative
integers and that the layer_tapes array is properly aligned with the cache
structure. These validations should occur before any cache mutation happens when
matching the Qwen3Next and AnyCache::Hybrid variants, so that invalid counters
or mismatched tapes do not leave the hybrid cache in a partially rolled back
state. Return an appropriate error if any validation fails.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0c81dbd9-376d-452d-9e7a-a5b7b332f881

📥 Commits

Reviewing files that changed from the base of the PR and between 9a7437e and 428b09a.

📒 Files selected for processing (3)

• .github/workflows/ci.yml
• crates/higgs-models/src/lib.rs
• crates/higgs-models/src/qwen3_next.rs