Backfill bibliography entries for citations a sync introduces

[mmcky]

Closes #117.

sync and forward translate {cite} roles, but the bibliography those keys resolve against is a shared file that never crossed the boundary. Under a strict build (-n -W) Sphinx promotes "could not find bibtex key" into a hard failure, so the target repo stops building on a file nobody edited.

The gap

src/sync-orchestrator.ts filters changed files with .endsWith('.md'), so a non-markdown file never enters the sync surface at all. More fundamentally, the engine had zero occurrences of .bib, bibtex or quant-econ anywhere in src/ — it had no bibliography concept. init only ever obtained one as a side effect of copying every non-markdown file wholesale ("Copied 108 non-markdown file(s)"), which is why seeded editions look fine and synced ones drift.

Field record: four lectures in one resync wave, then a 70-key backfill (see QuantEcon/lecture-python.zh-cn#203), then 21 more keys on 2026-07-27. Three manual repairs of a mechanism that did not exist.

The trigger is demand-driven, not diff-driven

The obvious design — "when the source PR touches the .bib, carry it" — misses the larger class. lecture-python-intro/lectures/msy_fishery.md cites five keys that lecture-intro.zh-cn's bibliography lacks, and no source PR touches the bib: the target simply has not translated that lecture yet. The moment it does, five citations dangle. So the trigger here is the set of keys a run introduces into the target, whatever the source diff happened to contain.

Safety model

A key that already resolves in the target is never a candidate, and appends never rewrite existing bytes. An edition that has localised a bibliography entry therefore cannot have it clobbered — that is structural, not a check that could be bypassed. The observation that the estate's bibliographies are currently strict subsets of their sources (480 vs 501 keys, 0 target-only) is not relied on anywhere in the design.

Ambiguity is reported, never guessed. A key declared twice in the source, or one differing from an existing target key only in case, fails the run rather than appending something that would produce an ambiguous reference. Keys that were already dangling before the run are surfaced as warnings and left alone — the run did not introduce them, and silently repairing pre-existing drift would hide it.

Outcomes

Situation	Result
Introduced key resolves in target	nothing happens
Introduced key resolves only in source	entry appended, listed in the run log
Introduced key resolves nowhere	run fails with file, line and nearest-key hint
Key declared twice in source, or case-collides in target	run fails, nothing appended
Key already dangling before this run	warning, left alone
Target configures no bibtex_bibfiles	check does not apply
Target configures a bibliography that cannot be read	run fails (fails closed)

Two measured details that drive the implementation

Indented entries. The entry parser tolerates leading whitespace because eight of the 501 entries in quant-econ.bib are indented. An ^@-anchored parser finds 493, would report those eight keys as missing, and would re-append them — producing exactly the duplicate-key -W failure this PR prevents. I hit this myself while measuring the estate before writing the fix, which is why there is a test named after it.

Citations inside admonitions. The citation walker is stack-based and selectively transparent: opaque inside {code-cell}, {raw}, {math} and bare fences, transparent inside {note}, {exercise}, {warning}, {prf:*}. Measured on lecture-python.myst: 71 of 922 citations (7.7%) live inside those containers — 65 in {note} alone. A flat skip-all-fences walker, like the one in structural-parity.ts, misses every one of them. The module header explains why all three fence walkers in this codebase differ deliberately.

Behaviour changes that turn previously-green runs red

Both are intentional, and both mean the target could not have built:

• A sync introducing a citation key that resolves in neither bibliography now fails.
• A target whose _config.yml names a bibtex_bibfiles entry that cannot be read now fails, rather than skipping the check silently.

The bibliography input selects backfill (default), lint (report without copying — the opt-out for editions that hand-author entries) or off. An unrecognised value fails the run, because a disabled guard looks exactly like a passing one.

Testing

src/__tests__/bibliography.test.ts — 33 cases covering mode parsing, the fence walker (including a code cell nested inside an admonition), indented entries, unterminated entries, duplicate keys, brace-nested field values, bibtex_bibfiles block and flow sequences, and every row of the outcomes table above.

Parser validated against the live estate before wiring: 501 source keys, 480 target keys, 21 source-only, 0 target-only, 0 duplicates, 0 unterminated — matching an independent measurement exactly. Citation extraction over all 139 source lectures finds 922 citations resolving to 356 distinct keys, with 0 unresolved against the source bibliography and 19 against the target.

Full suite: 1477 passed, 64 suites. npm run lint and npm run format:check clean. dist-action/ rebuilt on Node 24.

Deliberately not in scope

• The CLI forward path. The --github mode cuts an independent branch per file, so a shared key ledger would tell file B that a key is "already scheduled" while B's PR ships without it — forward/sync don't carry shared assets referenced by lectures — quant-econ.bib and _admonition/* dangle and fail strict builds #117 reintroduced by its own mitigation. The ledger parameter exists and is tested; wiring forward needs its own PR with that branch semantics handled properly.
• init's unconditional non-markdown copy, which overwrites a localised target bibliography during a gap-fill wave. That is the one path that currently destroys a localised shared asset — real, but a different fix (init: gap-fill into an existing edition needs manual clobber-recovery and TOC insertion #142, init -f is repo-scoped: single-file mode rewrites .translate/config.yml and clobbers localized non-markdown assets #134).
• Generalised shared-asset reachability (images, _static/… paths, data files). That is Review verdict says PASS with diff 10/10 on a sync PR that cannot build — no cross-file reachability check, and the reviewer never learns the run errored #157's cross-file check, which should absorb this shape rather than duplicate it.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds a bibliography-aware post-translation step to prevent sync/rebase runs from introducing {cite} keys that don’t resolve in the target repo’s configured BibTeX files, which would otherwise break strict Sphinx/JupyterBook builds.

Changes:

• Introduces src/bibliography.ts to extract {cite} keys, parse BibTeX entries, and plan/perform append-only backfills from source → target bibliographies.
• Threads a new bibliography action input through inputs/types into SyncOrchestrator, and fetches bib sources/targets based on the target repo’s _config.yml bibtex_bibfiles.
• Adds comprehensive Jest coverage plus documentation/architecture/changelog updates, and rebuilds dist-action/.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/types.ts	Adds bibliographyMode to action/rebase input types.
src/sync-orchestrator.ts	Runs citation backfill planning after translation and records the plan in results.
src/inputs.ts	Parses new bibliography input and normalizes it into bibliographyMode.
src/index.ts	Fetches bibliography sources/targets during sync/rebase and adds fetchBibliographies helper.
src/bibliography.ts	New module: citation extraction, BibTeX parsing, backfill planning, and reporting helpers.
src/tests/bibliography.test.ts	Adds tests covering mode parsing, citation walker, BibTeX parsing edge cases, and planner outcomes.
docs/user/action-reference.md	Documents the new bibliography input behavior and modes.
docs/developer/architecture.md	Adds bibliography.ts to the module map.
dist-action/index.js	Updates bundled action output for the new bibliography functionality.
CHANGELOG.md	Documents the user-visible fix for missing bib entries across sync.
action.yml	Adds the bibliography input with default backfill.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.