forward resync wrapped an entire document in a code fence and mis-derived its title (ak_aiyagari, v0.17.0)

[mmcky]

Silent whole-document corruption on one file in the lecture-python.zh-cn resync wave. Filing separately from #107/#116/#117 — this is a different failure mode from any of those, and the most severe defect the wave surfaced.

What happened

On lectures/ak_aiyagari.md, the resync emitted its new translation: frontmatter, then an opening code fence, then the entire original document verbatim, then a closing fence at EOF. It wrapped the document body in a code block.

The head of the file after the resync:

line	content
36	--- — the new translation frontmatter ends
37	(blank)
38	a bare triple-backtick — stray fence opener
39–50	a duplicated copy of the original jupytext/kernelspec frontmatter
53	# 长寿、异质性个体、世代交叠模型 — the H1
55–56	the {include} _admonition/gpu.md block

CommonMark closes a fence at the first bare triple-backtick, so the block opened at line 38 ran to line 56 — swallowing both the H1 title and the gpu.md include. A second stray triple-backtick sat at EOF.

How it presented

Not as anything resembling a fenced document. It surfaced as 13 × Document headings start at H2, not H1 [myst.header] under a strict build, which reads like a trivial "add a title" problem. It was filed downstream that way (QuantEcon/lecture-python.zh-cn#182) and the recommended fix was to restore an H1 — which would have papered over the real bug and left the include broken.

Nothing caught it at merge time. The edition's CI lacked -n -W on the notebook build (since fixed in QuantEcon/lecture-python.zh-cn#185), so it took a cold strict build weeks later to surface it.

Provenance

Introduced by the resync in QuantEcon/lecture-python.zh-cn#103, commit 17b5b98, 2026-07-18, tool v0.17.0.

	fence count	balanced
before resync (a664481)	120	yes
after resync (17b5b98)	124	no
English source	122	yes

The correct output was 122 — the pre-resync file was at 120 because it predated upstream's gpu.md include, which legitimately adds two. The engine added two more on top.

Same root cause, second symptom: the title

The same commit wrote translation.title as V, σ, μ.

That is not a title. It is taken from line 686 of the English source — a Python tuple-unpacking statement inside a code cell (V, σ, μ, K, L, r, w, τ, D, G, δ, r_old, w_old = state).

So the title-derivation path did not find the H1 either. That points at a single underlying cause rather than two coincidences: the engine failed to parse this file's document structure and fell through to a degraded path that both mis-derived the title and treated the body as opaque text to be fenced.

Worth investigating what is distinctive about ak_aiyagari.md that triggered the fallback — it is one of the largest lectures in the edition (1465 lines, 59 code cells) and is a JAX-migration rewrite, so size or a parse timeout are plausible starting points.

Scope

A sweep across all 8 edition clones in the program found no other file with this corruption — lecture-intro.zh-cn, the three lecture-python-programming editions, and the source repos are all clean, and lecture-python.zh-cn is clean apart from this one file. So it is rare, not systematic. That makes it more concerning rather than less: a rare silent corruption is one that will not be caught by pattern-matching on known-bad output.

Fixed downstream

Repaired in QuantEcon/lecture-python.zh-cn#183. The file now matches the English source at 122 balanced fences, with the H1 and include restored and the title corrected.

Suggested guard

Whatever the root cause turns out to be, a cheap post-write assertion would have caught this at generation time rather than weeks later: fence parity (openers must balance closers), and a sanity check that the derived title corresponds to an actual H1 rather than arbitrary body text. Both are O(lines) and would have failed this write immediately.

Related: #90 (silent data loss, but in the merge path rather than the resync writer); #65 (translator mangling MyST structural syntax). See also the companion issue on {raw} argument stripping filed alongside this one.

🤖 Generated with Claude Code

[mmcky]

Companion issue for the second defect from the same wave: #119 (MyST directive argument stripped, {raw} jupyter → {raw} in sir_model). Different construct and failure mode, but both are the resync mutating structural MyST that should have passed through verbatim.

[mmcky]

Still reproducing at v0.23.0, deterministically, via init -f — two attempts, same defect both times.

Context: the sync run for QuantEcon/lecture-python.myst#992 failed the structural parity check on lectures/phillips_lost_conquest.md (#156), so I retranslated that one file with the v0.23.0 CLI against the same source commit 370a58f:

node dist/cli/index.js init -s ../lecture-python.myst -t <scratch> --target-language zh-cn -f phillips_lost_conquest.md

Both runs reported ✓ Translated: 1/1 with no warning, and both produced the same two symptoms this issue describes — which is the notable part, because the symptoms are paired, and the second follows from the first.

Symptom 1 — the document is wrapped in a bare fence

The output opens with an injected translation: frontmatter block, then a stray bare , then the document's own jupytext frontmatter, and the file ends with a matching stray bare :

---
translation:
  title: 固定 IS 曲线，通过对全样本进行一次性 OLS 估计得到
  headings: ...
---

```                                    <- stray opening fence
---
jupytext: ...
---

(phillips_lost_conquest)=

So the real frontmatter ends up embedded in the body, and the whole document sits inside a code fence.

Symptom 2 — the title is derived from a figure caption

The two runs produced 固定 IS 曲线，通过对全样本进行一次性 OLS 估计得到 and 固定的IS曲线，在整个样本期内通过OLS估计一次 — both renderings of a figure caption, not of the H1 # The Lost Conquest: Fed Policy in the 2020s. The correct title is present in the body as # 失落的征服：2020 年代的美联储政策; only the derived metadata is wrong. Title derivation evidently runs over the fence-wrapped document, so it never sees the H1 as a heading.

The part worth acting on

The structural parity check that caught the sync-path failure does not catch this. On the sync path the same document was rejected — correctly — for a scrambled directive sequence. On the init path, a document wrapped entirely in a code fence with a title taken from a caption passes and is reported as a clean success.

Fence-wrapping is invisible to a directive-sequence comparison: I ran that check myself on the broken output and got 28/28 in order, because the wrapper adds no {directive} lines. Two checks would have caught it, both cheap and deterministic:

Check	Would catch
exactly one frontmatter block, at position 0	symptom 1
translation.title matches the translated H1	symptom 2

Body quality was fine

Worth recording, since it narrows the fault to the wrapper: after rebuilding the frontmatter by hand, the output validated clean against the English source on directive sequence (28/28, in order), anchors, heading count and levels, code-cell count, display-math count, fence balance, and every {doc}/{cite} link target. The only prose-level difference was one {doc} link's display text being translated, which is correct behaviour. The model is translating this document well; the pipeline is mis-assembling the result.

[mmcky]

Status update and cross-link, post-v0.24.0.

The damage path is now closed; the root cause is not. #159 (Wave 1 PR B, v0.24.0) put the structural-parity guard on init, so this corruption class now fails loudly instead of writing. The body predates that and reads more severe than the current state — silent corruption has become a refused file.

Sibling case: #203. init produced transposed {code-cell} ↔ {math} at v0.24.0, in one of three languages, and the guard caught it. Same guard, different model-side structural failure — together they are the evidence that the guard works and that the underlying parse/model behaviour is still wrong.

What remains here is Phase 2 material. A guard refusal is not a fix: the affected file still cannot be translated at all, and a refused file leaves exactly the #156-shaped hole (the run may still commit TOC and state entries referencing it). So the live ask is the root cause — whatever makes the model emit a whole document inside a fence, with a mis-derived title — not the write-path protection, which now exists.

[mmcky]

Field data 2026-08-20: reproduces on the init path too, and it is flaky rather than deterministic

Gathered while recovering learning_approximation into lecture-python.zh-cn, which this issue has been blocking since 2026-07-26. Two findings that narrow the target.

It is not sync-specific. The failure reproduces on translate init -f, which uses translateFullDocument rather than sync's section-splice path. So all three write paths hit it, not just sync.

It is not deterministic. Across six init attempts on the same lecture at engine v0.26.0, the parity check refused four times and passed twice. Every refusal carried the identical signature:

directive count differs: source has 36, output has 35 — a wholesale mismatch usually
means the document was re-fenced or truncated
directive #1 name changed: source line 15 has {raw}, output line 42 has {index}
directive #2 name changed: source line 25 has {index}, output line 45 has {contents}
…

The output drops the leading ```{raw} header block (the qe-notebook-header div), so directive #1 shifts from {raw} to {index}, every subsequent directive shifts by one, and the count lands exactly one short. That is a much narrower and more diagnosable target than "wholesale mismatch usually means the document was re-fenced or truncated".

The guard is fail-closed throughout — nothing was written on any of the four refusals, which is the v0.24.0 behaviour working as intended.

hoist_failure.md on the same edition showed the same flakiness on the forward path: one refusal, then a pass.

What this changes. The lecture landed on 2026-08-20 via lecture-python.zh-cn#261 on a run that happened to pass, verified after the fact rather than trusted (36/36 directives, 17/17 code cells, 15/15 headings, 0 python syntax errors, 4/4 captions localised). So the block is lifted in practice by retrying — but that is a workaround, not a fix, and it only works because the guard refuses loudly.

Worth distinguishing from the sync path, where the evidence runs the other way: lecture-python-programming#609 (original run, v0.25.0) and #613 (its \translate-resync retry, v0.26.0) have byte-identical bodies. A retry there reproduced the failure exactly. So "retry until it passes" is viable on init/forward and is not viable on sync.

Related: the fifteen translation-sync-failure issues that recorded this class estate-wide were all closed on 2026-08-20 — lecture-python.myst#1004 and #1007 are the two that carried this lecture's failing runs verbatim, and were the primary evidence for this block.

[mmcky]

Cross-link: the flakiness measurement above belongs with #203, which asked for exactly this count and now has it — 4 refusals in 6 attempts on a real lecture, against the 1-in-3 it saw on a 175-line fixture. Note the two issues describe different failure modes of the same guard: #203's body records a {code-cell} ↔ {math} transposition, whereas the refusals measured on 2026-08-20 drop the leading ````{raw}``` header block. Both are directive-sequence corruption, so a retry path would cover both.