Silent data loss in the sync merge path — five ways translations vanish while the run reports success

[mmcky]

Five defects found in the 2026-07-15 deep review (REVIEW-FABLE5-2026-07-15.md §6.1) that are not covered by PLAN Phase 2. They share one shape: sync loses or corrupts target content and reports success. Nothing fails, nothing warns, and the loss is only visible by diffing the output against what should have happened.

That shape is why they matter more than their individual severity suggests — and it is the same class as the truncation bug fixed in #83 (a cut-off translation committed as complete) and the harness bugs fixed in #86. Line references verified against main @ 9f317e5.

1. Heading-map lookups fail for every role-bearing heading — [M]

Map values are written role-stripped (src/heading-map.ts:94, via stripMystRoles), but Strategy-1 lookup compares them against the raw target heading (src/file-processor.ts:738):

const cleanTargetHeading = targetSection.heading.replace(/^#+\s+/, '').trim();

So the stored value 熊猫 is compared against {index}`熊猫 <single: 熊猫>` and never matches. QuantEcon lectures use {index} roles in headings routinely (v0.12.5 exists because of it), and translations preserve them.

Failure: with equal section counts the positional fallback masks it. With unequal counts the section is either skipped (silently dropped from the PR) or treated as new — re-translated from scratch, discarding accumulated human refinements, which is the one thing the pipeline is supposed to never do.

The rebase added-section path at file-processor.ts:226-231 strips both sides correctly. The asymmetry is the bug. Fix: one canonicalizeHeading() used at every write and lookup site (REVIEW §7.6 R12).

2. Target-only sections are silently deleted — [M]

Reconstruction iterates only newSource.sections (src/file-processor.ts:176-217, :436-441). Any section existing only in the target — human-added content — is omitted from the output with no warning. skippedSections/onSkippedSection covers only the source-side analog.

The CLI's forward path at least warns (TARGET_HAS_ADDITIONS); the action destroys silently.

3. A per-file fetch failure drops the file from the sync — [M]

src/index.ts:945 (also :1005, :436):

catch (error) { core.error(`Error fetching content for ${file.filename}: ${error}`); }

core.error is a log annotation — it neither records an error nor fails the run. A transient 5xx on one file yields a PR missing that file, a success comment listing the rest, no failure issue, and a green check.

Fix: have fetchAllFileContents return {files, errors} and treat fetch errors like processing errors (failure issue + setFailed). No happy-path change.

4. Heading level promotion/demotion loses translations or leaks English — [M]

src/diff-detector.ts:94-97 + src/file-processor.ts:697-752, :377-397.

• Promote ### Setup → ## Setup: findSectionById matches the nested old section recursively, producing a level-crossing "modified"; target lookup searches only top-level sections and the map key is path-based (Overview::Setup), so the leaf lookup of Setup misses → "treating as new" → existing translation discarded and re-translated.
• Demote ## Setup → ### Setup: the merge helper looks up Overview::Setup, which doesn't exist (the old key was top-level Setup), falls through to "keep source as-is" → an untranslated English subsection embedded in the zh-cn/fa document, run reports success.

Not covered by the tracked rewording (ARCHITECTURE R5) or duplicate-slug items.

5. injectHeadingMap re-types frontmatter scalars — [M→L]

src/heading-map.ts:193 round-trips the whole target frontmatter through yaml.load/yaml.dump. Verified with the repo's own js-yaml: date: 2026-07-15 → date: 2026-07-15T00:00:00.000Z; 1.10 → 1.1. Silent frontmatter rewriting on every processed file that carries an implicitly-typed scalar.

Why together

1 and 4 are the same root cause (heading identity is compared inconsistently across sites) and both discard human translations. 2 and 3 are both "output is quietly missing content". They want one fix each but a single verification: the Phase 2 round-trip invariant test would catch 1, 2 and 4 as a class, which is the argument for doing Phase 2 first rather than patching these individually.

Refs: REVIEW-FABLE5-2026-07-15.md §6.1 and §7.6 (R12), PLAN Phase 2, #65

[mmcky]

Progress and one design interaction worth recording (2026-07-21).

Defect 1 (role-bearing heading lookups) appears at least substantially addressed by v0.17.0 (#100): heading comparisons now canonicalize both sides (roles stripped, typography folded), and role-stripped keys are used at the lookup sites this defect names (section-matcher, mergeSubsectionsWithTargetTranslations). What #120 still lists as pending is a single canonicalizeHeading() used at every write and lookup site — a verification pass over the remaining sites is the outstanding work, not a fix from scratch.

Defects 3–5 (fetch-failure file drops, heading promotion/demotion, and the fifth) are untouched. Still the P0 item-4 remainder per #120.

Defect 2 (target-only sections silently deleted) — a design interaction with v0.20.0's structural parity guard (#128) that must be settled before fixing it. The guard asserts the output's structural tokens match the source's. Today's (defective) behaviour passes: dropped target-only sections mean the output mirrors source structure. But the fix — retaining human-added target sections — would make outputs carry directives/anchors the source lacks, which the guard flags as violations. Evidence already exists: the guard's corpus dry-run flagged ifp_egm among its 11 violating pairs, and that file is one of the known TARGET_HAS_ADDITIONS cases (translator-added exercise sections). So the defect-2 fix and the guard need co-design — most likely a declared-additions mechanism (e.g. sections marked target-only in the heading map are excluded from parity), not a loosening of the guard. Fixing defect 2 naively would trade silent deletion for loud false alarms.

[mmcky]

Defect 2 resolved 2026-07-22 (decision record): reclassified on evidence, fixed as removal-with-visibility (#137), declared-additions design shelved below. Decisions taken with Matt in session; this comment is the durable record. Defects 1 (canonicalization verification pass), 3, 4 and 5 remain this issue's live remainder.

The evidence that changed the plan

A fence-aware scan of all 211 source/target lecture pairs across the five production editions (origin/main of each repo, mirroring the parity guard's walk, at both ## and ### level) found zero files with target-extra sections. The only target-only file in the estate is intro.zh-cn's lectures/README.md — housekeeping. And the one case this issue's history cites as a live human addition dissolves on inspection: ifp_egm's extra 练习 section was the translation of the old source's ## Exercises, which upstream deleted in the optimal-savings reorganize (lecture-python.myst b27f1eb0a) — the wave resync (lecture-python.zh-cn#170) removed it correctly. It was upstream drift, not human authorship.

Two honest caveats. First, the drift-recovery waves already whole-file-resynced the estate with only a log warning, so a pre-wave human addition would have been destroyed rather than counted — but no confirmed human-authored in-file addition appears anywhere in the record either. Second, the TARGET_HAS_ADDITIONS triage verdict conflates two provenances needing opposite treatments: content the source once had and later deleted (removal correct) versus content a human added (removal destructive). The verdict cannot distinguish them; a reviewer can, which is what the fix below leans on.

Why declared additions were not built: the lifecycle bill

Working the translation.additions design through its lifecycle showed metadata with no natural expiry and a standing tax at every stage:

Stage	Behaviour under the design	Cost
Birth	Next sync fails loud; operator declares the section	Intended (one failed run)
Steady state	Retain + exclude from parity; forward extracts pre-model, splices back	Splice logic in two paths, forever
Heading rename	Declaration mismatch → loud failure → re-declare	A failed run per rename
Upstream adopts equivalent section	Sync translates the source's version and retains the human one → duplicate content, green run	Undetectable automatically; manual merge + declaration removal
Human deletes the section	Declaration goes stale; harmless cruft	New doctor check to flag it

Paying that for a class with zero live members is complexity ahead of need. The design is shelved, not rejected — agreed shape for when the first real instance appears: translation.additions list in the frontmatter block; the parity guard unchanged (exclusion happens at call sites — sync subtracts the retained sections it already holds; forward splices deterministically in finalizeResyncContent and checks parity pre-splice); matching via the existing cleanHeadingText/normalizeHeadingForMatch machinery; undeclared additions retained and failed loud (decided: a loud failure correctly signals a content issue); remediation documented step-by-step (decided: the docs are load-bearing for this design). Trigger to build: a genuine human-authored in-file addition, or a language team requesting the workflow.

What shipped instead (#137)

Removal stays the behaviour — it is correct for every occurrence that actually exists — but it is now visible at every layer: processSectionBased reports each dropped target-only section (mirroring the onSkippedSection plumbing), the run log warns, and the sync PR body carries a "Target-Only Sections Removed" notice naming file and sections, stating both provenances, and linking the new FAQ entry. forward's TARGET_HAS_ADDITIONS warning now states both provenances too. The supported pattern for human-added content is documented: target-only files — sync never touches a file without a source counterpart, status tracks them as TARGET_ONLY, no metadata needed.

Consequences for the guard and the P0 sequence

The v0.20.0 structural parity guard keeps its strict equality unchanged — the scan shows source/output structural parity is a true invariant of the corpus, not an over-tight assumption. And the co-design question that gated the P0 band is dissolved: current reconstruction semantics are correct for the world as it exists, so #94 Phase 2 (round-trip invariant + real validateMyST) is unblocked and next, with this issue's defects 3–5 and the defect-1 verification pass behind it.

[mmcky]

The 2026-07-23 technical-debt audit found the same failure shape from twelve independent angles and made it the spine of the resulting plan — a run completes, reports success, and the thing it was supposed to guarantee did not happen. This issue is the earliest statement of it.

Directly adjacent scheduled work:

• tech-debt B: translate init writes model output to disk with no structural-parity guard #159 — translate init writes model output with no structural-parity guard. checkStructuralParity has four non-test call sites and init is not one of them, so the bulk-seed path that creates a whole edition is unguarded.
• tech-debt H: The write path cannot detect its own corruption: validateMyST is structurally incapable of failing #165 — validateMyST is structurally incapable of returning valid: false (it try/catches parseSections, which never throws), so the two sync-path gates that call it are unreachable code that logs "validation passed" over corrupt output.
• tech-debt C: Failure is optional: rebase mode cannot fail and bulk runs exit 0 however many files failed #160 — rebase mode counts per-PR errors and never calls setFailed; bulk forward/backward exit 0 however many files failed.
• tech-debt T2·3: Create the write-path choke point: finalizeTranslatedDocument #172 — the write-path choke point, which is the structural fix behind tech-debt B: translate init writes model output to disk with no structural-parity guard #159.

One item needs reconciling rather than assuming: this issue's §1 (map values written role-stripped at heading-map.ts:94, compared against a raw target heading at file-processor.ts:738) is adjacent to audit findings #4/#118, which cite un-normalised parentPath derivations at file-processor.ts:466 and :488. Same class — heading identity has no owner and is re-derived at 14 sites — but not obviously the same call sites. #171 brands HeadingKey with cleanHeadingText as sole constructor, which should surface all of them as compile errors; worth checking this issue's §1 against that work when it lands.

Keeping open as the origin issue for the class.

[mmcky]

A field instance of mode 3 ("a per-file fetch failure drops the file from the sync"), on a production sync running v0.25.0. The metadata-versus-delivery mismatch is unambiguous here, so it is a clean reproduction rather than an inference.

The evidence

QuantEcon/lecture-python-programming.fr#29, syncing source PR QuantEcon/lecture-python-programming#604 at aacff38.

Its translation-sync-metadata block lists four files:

lectures/functions.md, lectures/numpy.md, lectures/pandas.md, lectures/python_by_example.md

The "Files Updated" section of the same PR body lists three, and the diff contains those same three. lectures/numpy.md is in the metadata array, absent from the body list, and absent from the diff.

This is the eight-against-nine signature from #222's Defect B, reproduced with a file that exists in both source and target — so it is not the missing-file escalation path, just a plain drop.

What it costs in this case

lectures/numpy.md exists in the target and upstream changed it in four places. Three of the four are coincidentally no-ops in French — --- was already spaced, **Partie 1** / **Partie 2** already carried the digit and space, NumPy was already capitalised correctly. The fourth is real: prose discreteRV should have become DiscreteRV, and lectures/numpy.md:1292 still reads discreteRV today.

Separately and independent of the prose fix, .translate/state/numpy.md.yml on main is still pinned at source-sha: a2b929f, synced-at: "2026-07-14", tool-version: 0.15.0. So the drop also leaves the state file stale, and the next sync will compute its diff from the wrong base.

Why it is worth one more data point

The run reported success, the review verdict scored diff quality 10/10, and both are defensible on what was actually delivered — the three files that landed are correct. Nothing in the PR body, the check output, or the verdict says a fourth file was attempted. The only trace is the metadata array being longer than the delivery, which is exactly what this issue says.

Worth noting for prioritisation that the sibling sync of the same source PR into .zh-cn (#86) carried all four files correctly. Same source PR, same engine version, same day — so whatever dropped numpy.md was per-run rather than something about the file or the source diff.