Rust Rewrite with Deck Federation and Yaml Deck Source

.editorconfig

@@ -0,0 +1,15 @@
+root = true
+
+[*]
+charset = utf-8
+end_of_line = lf
+insert_final_newline = true
+trim_trailing_whitespace = true
+indent_style = space
+indent_size = 4
+
+[*.md]
+trim_trailing_whitespace = false
+
+[*.{yml,yaml,json,toml}]
+indent_size = 2

.envrc

@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+
+eval "$(devenv direnvrc)"
+use devenv

.frontloop/default/clarify/7500-explore-crowdanki-round-trip-import-workflow.md

@@ -0,0 +1,37 @@
+---
+title: Explore a CrowdAnki round-trip import workflow (pull in-Anki edits back to source)
+priority: low
+---
+
+## Goal
+
+A maintainer who edits notes directly in Anki can bring those edits home: export via CrowdAnki, run one supported Brain Brew workflow (ideally a single command), and get a reviewable change against the canonical source — restoring, in overlay-native form, what the old Python world's `anki_to_source.yaml` recipe provided.
+
+## Status: clarify — deliberately deferred
+
+The loss of the old Anki→source recipe was a **purposeful simplification** during the UG migration, not an oversight. This is a later thing. Do not promote to ready until the core review backlog (media integrity, include preservation, stale resolution, resolver unification) has landed and there is actual maintainer demand for the round trip.
+
+## Problem
+
+The primitives exist but are dead-ended:
+
+- `brainbrew import crowdanki <folder> --accept-suggested-ids --out deck.yaml` (`crates/brain-brew-cli/src/commands/import.rs` ~:9) does a full re-import into a fresh deck file — right for initial migration, destructive for round-tripping (would flatten includes, structured messages, media declarations, overlay federation).
+- `brainbrew diff <left> <right> --as-overlay` (`crates/brain-brew-cli/src/commands/diff.rs` ~:11) drafts an overlay from a semantic diff — the correct primitive for expressing "what changed in Anki" against a composed target.
+- Nothing connects them; UG's CONTRIBUTING documents CrowdAnki in one direction only (smoke-test import into Anki, ~:494).
+
+The plausible workflow: import the Anki export to a temp canonical deck → diff against the composed English target → emit a reviewable overlay/patch draft → human reviews and applies to source.
+
+## Questions to settle before ready
+
+1. **ID alignment.** Import derives suggested StableIds from names/first field. When do they line up with the existing deck's IDs, and what does the workflow do when they don't? (Interacts with the derived-ID-collision fail-closed fix.)
+2. **Reverse-mapping lossiness.** The diff runs against the composed target, where includes are materialized and structured messages are rendered. Edits landing in those fields can be detected but not automatically written back to source form. Is "reviewable draft + human step" the contract, or does any subset (plain scalar fields) get an automatic write-back path? (`!image` adoption would shrink this gap for image fields.)
+3. **Command shape.** A documented three-step recipe using existing commands vs a convenience command (`brainbrew pull`-style) wrapping import→compose→diff. What does the output artifact look like — overlay draft on stdout, a file, a workbench surface?
+4. **Scope of targets.** Old world was English-only. Is the round trip defined only against the base/EN composed target, or per-target (which would imply reverse-mapping through translation dictionaries — much harder, likely out of scope)?
+5. **Demand check.** Confirm a real UG maintainer scenario before building; if nobody edits in Anki anymore, close this instead.
+
+## Acceptance Criteria (draft, to firm up on promotion)
+
+- A supported, documented, tested workflow (command or recipe) producing a reviewable source-level change from a CrowdAnki export of an existing deck.
+- Non-destructive by construction: never overwrites `deck.yaml`/overlays directly; output is a draft a human applies.
+- Honest about lossiness: structured-message and include-backed fields are flagged as needing manual handling, not silently mangled.
+- UG-side follow-up task (CONTRIBUTING maintainer section) filed once the tool side ships.

.frontloop/default/done/0010-report-swallowed-message-errors-in-render-deck-variables.md

@@ -0,0 +1,37 @@
+---
+title: Report swallowed structured-message errors in render_deck_variables
+priority: high
+---
+
+## Goal
+
+`render_deck_variables` must fail when structured-message resolution fails after variable substitution, instead of silently returning a deck with broken/empty messages.
+
+## Problem
+
+In `crates/brain-brew-core/src/lib.rs` (~line 3623), the success branch does:
+
+```rust
+if errors.is_empty() {
+    let mut message_errors = Vec::new();
+    resolve_structured_messages_with_validation_errors(&mut rendered, &mut message_errors);
+    Ok(rendered)
+}
+```
+
+`message_errors` is populated by the resolver but never inspected, so any message that fails to resolve post-substitution (e.g. a `ref` to a nonexistent field produced via a variable) is silently dropped. Everywhere else in the pipeline this resolver's errors are surfaced (compose errors, validation errors); this is the only call site that discards them.
+
+## Acceptance Criteria
+
+- `render_deck_variables` returns an error when `resolve_structured_messages_with_validation_errors` reports any error.
+- The error output identifies the offending path/message, consistent with how compose/validate report the same failures.
+- Regression test: a deck whose variable expansion yields an unresolvable structured message causes `render_deck_variables` to fail; the same deck with the variable corrected succeeds.
+- `cargo test --workspace` passes.
+
+## Design Decisions
+
+- Whether to extend `VariableRenderReport` with the existing validation-error type or add a distinct variant is left to the implementor; prefer whichever keeps error display consistent.
+
+## Implementation Notes
+
+- Callers of `render_deck_variables` (e.g. CrowdAnki export path in `crates/brain-brew-formats/src/crowdanki.rs`) already handle the `Err` case, so no caller changes expected beyond error display.

.frontloop/default/done/0020-fail-closed-on-crowdanki-import-id-collisions.md

@@ -0,0 +1,29 @@
+---
+title: Fail closed on CrowdAnki import stable-ID collisions
+priority: high
+---
+
+## Goal
+
+CrowdAnki import must return an error when two source entities derive the same stable ID, instead of silently keeping only the last one.
+
+## Problem
+
+In `crates/brain-brew-formats/src/crowdanki.rs` (`into_deck`, ~lines 790-802), stable IDs are slug-derived (note types from model name, notes from first field value; derivation ~lines 1085-1111). Note types are inserted with `BTreeMap::insert` and notes collected into a `BTreeMap` — both last-wins on duplicate keys. Distinct entities whose names/first fields slugify identically (e.g. "São Tomé" vs "Sao Tome", two notes both starting "Congo") silently vanish from the imported deck, which then validates cleanly. Violates ADR-0010 (fail closed on unsupported adapter data) and ADR-0004 (reviewable stable IDs). Field-level duplicates are already caught by validation (`DuplicateFieldDefinition`); note and note-type level are not.
+
+## Acceptance Criteria
+
+- Importing a CrowdAnki export where two notes derive the same stable ID returns a `CrowdAnkiError` naming the colliding ID and identifying both source notes (e.g. their GUIDs and/or first-field values).
+- Same for two note models deriving the same stable ID (naming both model names).
+- Error message points the user at the suggested-IDs override path (`import_deck_accept_suggested_ids`) as the resolution.
+- The `note_type_by_uuid` map is also collision-checked (two models with the same UUID should already be impossible upstream, but the derived-ID map must not mask a collision).
+- Happy-path round-trip tests still pass; `cargo test --workspace` passes.
+
+## Process
+
+Use TDD / red-green-refactor: write the two failing collision tests in `crates/brain-brew-formats/tests/crowdanki.rs` first (duplicate note slug, duplicate note-model slug), confirm they fail with the current silent last-wins behavior, then implement the collision checks, then refactor.
+
+## Implementation Notes
+
+- Collision check is a few lines at each insert site (`BTreeMap::insert` returning `Some` / entry API); most of the work is the tests.
+- Keep error style consistent with existing `CrowdAnkiError::Unsupported` messages in the same file.

.frontloop/default/done/0030-cache-workspace-composition-in-workbench-server.md

@@ -0,0 +1,38 @@
+---
+title: Cache workspace composition in the Workbench server
+priority: high
+---
+
+## Goal
+
+Workbench request handlers must stop re-planning and re-composing the workspace from disk on every request; per-request work should be bounded (ADR-0015), keyed on the existing freshness generation.
+
+## Problem
+
+In `crates/brain-brew-cli/src/commands/workbench.rs`:
+
+- `media_path_declared` (~lines 978-1009) runs on every `GET /api/media/<path>` and, per request, re-reads the manifest and fully re-plans + re-composes **every overlay of every target** just to check whether the path is declared media. For UG (~80 targets across two manifests, 319 notes, 546 media files) a single note view (flag + map) triggers hundreds of full-deck compositions.
+- `current_manifest` (~line 471) re-reads the manifest file per request; `selected_translation_context` (~lines 1011-1072) re-plans and re-composes the selected target per request.
+- All of this blocking file IO and CPU-heavy composition runs inline in `async fn` handlers on Tokio worker threads — zero uses of `spawn_blocking` in the crate.
+- Minor: `media_file_candidates` (~lines 945-975) probes `external/<deckname>/...` under every ancestor directory of the manifest root — many stat calls per request; looks like a leftover dev-layout hack.
+
+The server already tracks a freshness generation that bumps when workspace files change (~lines 3525-3579) — the invalidation hook exists but nothing is cached on it.
+
+## Acceptance Criteria
+
+- `media_path_declared` is a lookup into a cached set of declared media paths (union across all targets), built at most once per freshness generation.
+- Composed deck / translation-context state for the selected target is cached keyed on the freshness generation (and invalidated by apply/new-language writes, which already bump the generation).
+- Manifest is not re-read from disk on requests that hit a valid cache.
+- Blocking plan/compose/file work in handlers runs via `tokio::task::spawn_blocking` (or an equivalent dedicated worker), not inline on async workers.
+- Behavior is unchanged when files change on disk: after an edit, the next request reflects the new state (existing freshness E2E semantics still pass).
+- Existing workbench integration tests in `crates/brain-brew-cli/tests/cli.rs` and the E2E suite pass. Add an integration test asserting media requests succeed after a workspace edit (cache invalidation) and that an undeclared path is still rejected.
+
+## Design Decisions
+
+- Single task on purpose: the media-path set and composed-state cache share the same generation-keyed invalidation; implement one caching layer, two consumers.
+- Consider tightening `media_file_candidates` to manifest root + media root only, but do not break the documented `--media-root` behavior; if the ancestor probing is load-bearing for some workflow, keep it and note why.
+
+## Implementation Notes
+
+- Cache lives on the shared server state struct behind a `tokio::sync::RwLock` (or `arc-swap`); key = generation number.
+- `compose_lenient_translation_overlay` (duplicated at `workbench.rs:~4414` and `translations.rs:~1314`) is in the hot path; hoisting it to a shared location can be done here or left to the separate dedup task.

.frontloop/default/done/0040-refresh-ug-fixture-from-upstream-via-sync-script.md

@@ -0,0 +1,34 @@
+---
+title: Refresh the UG fixture from upstream via a repeatable sync script
+priority: medium
+---
+
+## Goal
+
+`fixtures/ultimate-geography/` matches the real ultimate-geography repo (current layout, all 16 languages including Hebrew, both manifests), plus one deliberate, documented delta: the ADR-0012 `languages:`/`translation_profile` catalog. A checked-in script makes the refresh repeatable whenever UG moves again.
+
+## Problem
+
+The fixture is a mid-migration snapshot that has drifted from the real repo in both directions:
+
+- Behind: pre-refactor `source/` include layout (real repo uses `templates/<name>/{question,answer}.html`); missing Hebrew (`he.yaml` — the only RTL language, so script direction is currently untested here); missing the hardcore second manifest (the real repo builds two manifests; the hardcore shape earned two high-priority tasks in UG's pr736-review epic and is unrepresented in this repo's tests).
+- Ahead: carries the ADR-0012 `languages:`/`translation_profile` catalog (~240 lines of `brainbrew.yaml`) that the real UG doesn't use yet.
+
+Every "composes the UG fixture byte-identically" acceptance criterion in the queued refactor tasks is only as strong as the fixture's resemblance to production; right now it certifies a deck shape that no longer exists.
+
+## Acceptance Criteria
+
+- A script (e.g. `scripts/sync-ug-fixture.sh` or a small Rust/xtask — implementer's choice) that:
+  - Copies the fixture-relevant files from a UG checkout (path passed as an argument; default may assume `../external/ultimate-geography` but must not hardcode-require it) into `fixtures/ultimate-geography/`: both manifests, all overlays, templates, deck sources, and whatever media/metadata files the fixture tests need.
+  - Reapplies the translation_profile delta on top. Keep the delta maintainable: either a patch file or a separate YAML fragment the script splices in — NOT hand-editing after each sync. The delta and its rationale (ADR-0012 coverage until UG adopts it upstream — see the UG-side frontloop task) are documented in a README next to the script or fixture.
+  - Is idempotent: running it twice produces no diff.
+- The fixture after refresh: current UG layout, 16 languages including `he.yaml`, both `brainbrew.yaml` and `brainbrew-hardcore.yaml`, plus the translation_profile delta.
+- All fixture-dependent tests updated and passing: `cargo test --workspace`. Expect churn in expected outputs (`ultimate_geography_fixture.rs`, overlay/canonical-yaml tests, docs examples that cite fixture facts) — update expectations to the new fixture, but investigate any change that looks semantic rather than layout-driven before accepting it.
+- Hardcore manifest is actually exercised: at least one test composes/verifies a hardcore target (new coverage — the shape was previously untested here).
+- The sync script's usage is documented (one paragraph: when to run it, what the delta is).
+
+## Design Decisions
+
+- The fixture intentionally stays "real UG + translation_profile" rather than splitting into two fixtures; the delta is temporary — a UG-side task exists to adopt translation_profile upstream, after which the delta (and splice step) can be deleted.
+- Sync direction is one-way (UG → fixture). The script must never write into the UG checkout.
+- If some UG files are irrelevant to tests (e.g. full media binaries), the script may exclude them, but the exclusion list lives in the script, not in tribal knowledge.

.frontloop/default/done/0050-unify-translation-resolution-into-single-resolver.md

@@ -0,0 +1,37 @@
+---
+title: Unify translation resolution into a single resolver
+priority: high
+---
+
+## Goal
+
+One implementation of the translation decision procedure (direct / contextual / no_change / stale / missing), consumed by both the apply path and the coverage path, replacing the current hand-synced parallel copies.
+
+## Problem
+
+In `crates/brain-brew-core/src/lib.rs` the same ~100-line resolution decision tree is written out longhand at least four times:
+
+- `TranslationApplyContext::translate_string` (~:1909-2037) — mutating apply path used by compose.
+- `TranslationCoverageBuilder::record_string` (~:924-1050) — read-only classification used by verify/reports.
+- `translate_optional_string` (~:2039) and `record_optional_string` (~:1052) — near-identical nullable-field twins.
+- `has_explicit_string_entry` is duplicated verbatim in both structs (~:894-922 and ~:1879-1907).
+
+Nothing enforces agreement between the copies. Divergence means `verify` reports coverage that differs from what compose actually produces — breaking the tool's core promise that what verify says is what ships.
+
+## Acceptance Criteria
+
+- A single resolver function/type takes (source string, path, dictionary, options) and returns a `TranslationOutcome` enum (e.g. `Direct(value)` / `Contextual { value, context }` / `NoChange` / `Stale(record)` / `Missing`).
+- `TranslationApplyContext` and `TranslationCoverageBuilder` are thin consumers mapping outcomes to mutations/errors and report entries respectively; the `_optional` variants collapse into `Option` handling at call sites; `has_explicit_string_entry` exists exactly once.
+- Behavior-preserving: all existing tests pass unchanged (33 `overlay_compose` tests, translation CLI tests, UG fixture tests). If the parallel copies turn out to have ALREADY drifted (tests or UG fixture composition reveal a semantic difference between apply and coverage), STOP and report the divergence for a human decision rather than silently picking one side.
+- Composing the full ultimate-geography fixture (`fixtures/ultimate-geography`) produces byte-identical output before and after the refactor (capture a before snapshot as evidence).
+- `cargo test --workspace` passes.
+
+## Design Decisions
+
+- Pure refactor — no semantic changes, no new features. Any tempting behavior fix found along the way becomes a separate task.
+- Resolver should be a free function or small struct in core, not a trait hierarchy; keep it as boring as possible.
+
+## Implementation Notes
+
+- The translation subsystem spans roughly lib.rs:722-2331; this refactor is a natural precursor to splitting lib.rs into submodules (separate task) — do this one first so the module split moves one resolver, not four copies.
+- Guardrails are strong: work red-green against the existing suites; diff the resolution behavior across the 16 UG language overlays as an end-to-end check.