fix(matching): normalize heading comparisons; close remaining typography gaps

CHANGELOG.md

@@ -8,14 +8,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Fixed
+- **Heading matching tolerates typography drift instead of relying on byte equality** (#97 follow-up): matching a frontmatter heading-map value to its body heading was an exact string compare, and typography makes the two sides legitimately diverge — `applyTypography` masks role spans while `stripMystRoles` exposes their display text (so a map value can carry an NBSP its body heading structurally cannot), `apply.mjs` typesets map values independently, and human edits touch one side only. On a mismatch the sync drops unchanged sections from the output or retranslates modified ones from English, discarding human edits. Comparisons now canonicalize both sides: strip `#` markers and MyST roles, fold NBSP/narrow-NBSP, collapse whitespace runs, and drop spaces before `; : ! ?` — every shape the French transform produces, including the zero-gap case where `Quoi?!` gains an NBSP from nothing. Exact matches are tried first and a normalized match is accepted only when unique, so raw-distinct headings that canonicalize identically keep their old positional pairing instead of first-match landing on the wrong section. Written map values stay exactly as typeset — only comparisons normalize. Verified over the full French edition: all 401 real headings now match under every typography variant (one diverged before). Also closes a pre-existing hole the audit surfaced: the body side of the compare never stripped MyST roles while map values always did, so the corpus's role-wrapped headings (`## {index}` + backticked text) could never match by heading map and survived on the positional fallback alone.
+- **Role-stripped keys are used everywhere heading-map keys are looked up**: the CLI validator (`section-matcher.ts`) and subsection merging (`mergeSubsectionsWithTargetTranslations`) built lookup keys from raw source headings while keys are stored role-stripped, so role-wrapped headings silently skipped validation or missed their subsection map entries.
+- **`translate forward` resync output is typeset before it is written**: the whole-file resync path wrote raw model output, the one remaining write path without deterministic typography — on a French repo every resync stripped the NBSP the seed inserted, exactly the sync-path bug fixed for `mode: sync` below.
+- **`translate headingmap` no longer ping-pongs with `apply.mjs`**: its change detection compared the existing (typeset) map against freshly body-derived (plain, where roles masked typography) values byte-for-byte, so it reported `updated` forever and each tool rewrote the other's output. Equality is now typography-insensitive and the existing typeset values are kept.
 - **Sync now applies deterministic typography, closing the gap v0.16.0 left open** (#97): `applyTypography()` ran only in the `translate init` seed path, so every sync stripped the non-breaking spaces the seed inserted, leaving the sync path relying on the very prompt instruction #79 was opened to replace, since the model does not follow it. One merged sync of `lecture-python-programming.fr#6` took `numba.md` from 27 × U+00A0 to 14 — invisible in review, since the diff renders as unchanged text. Both sync entry points are now typeset: `processFull` before the heading map is derived (mirroring init), and `processSectionBased` on the reconstructed document before the map is injected. Document-scoped rather than section-scoped, so drift in sections the source PR did not touch is repaired on the next sync of that file; sync PRs may therefore carry whitespace-only changes on lines the source change did not touch. Existing editions still want a one-off `scripts/typography/apply.mjs --lang fr` to reset files that will not be synced soon.
-- **The heading map is typeset with the body, never after it**: `applyTypography` skips frontmatter by design, so applying it to already-injected content — the obvious fix, and the one the issue's "shape of the fix" implies — would have left map values plain-spaced while body headings gained non-breaking spaces. Heading lookup is an exact string compare with no whitespace normalisation, and the positional fallback is unavailable whenever the section count changes, so that mismatch silently drops unchanged sections and retranslates modified ones from English, discarding human edits. Body, map, and title are now typeset together, as `scripts/typography/apply.mjs` already did. Tests pin the invariant against both the missing transform and the body-only variant.
+- **The heading map is typeset with the body, never after it**: `applyTypography` skips frontmatter by design, so applying it to already-injected content — the obvious fix, and the one the issue's "shape of the fix" implies — would have left map values plain-spaced while body headings gained non-breaking spaces. Heading lookup was, at the time, an exact string compare with no whitespace normalisation (made typography-insensitive in the follow-up entry above), and the positional fallback is unavailable whenever the section count changes, so that mismatch silently drops unchanged sections and retranslates modified ones from English, discarding human edits. Body, map, and title are now typeset together, as `scripts/typography/apply.mjs` already did. Tests pin the invariant against both the missing transform and the body-only variant.
 - **Concurrent review runs no longer post duplicate review comments**: `postReviewComment` listed comments, looked for its own, then created one if absent — a check-then-act with nothing making it atomic, so every concurrent run saw "no comment yet" and created one. Observed on `lecture-python-programming.fr#6`, which carried two "Translation Quality Review" comments a second apart, each subsequently overwritten by a *different* run, leaving scores that matched no single review. Review comments now carry a hidden `<!-- action-translation-review -->` marker, and each run deletes any older marked comment after writing its own. Because ids increase with creation time and every run lists after it writes, the run holding the highest id always sees and removes the rest: exactly one comment survives any interleaving. Pre-existing duplicates (from v0.16.1 and earlier, which have no marker) are cleaned up on the next review of that PR.
 
 ### Changed
 - **Review comments are identified by marker, not prose**: the old predicate matched any comment containing both "Translation Quality Review" and "action-translation" anywhere in its body — it could match, and overwrite, a human comment quoting a review. Matching is now anchored at the start of the body, so quoted or reposted reviews are never touched.
 - **Review workflow templates gained a `concurrency` group** (docs): the action-side fix converges the *comment*, but each racing run still pays for a full review. A per-PR `concurrency` group with `cancel-in-progress` collapses the `opened` and `labeled` events a single sync produces into one review. The `connect-existing` template additionally ignores `labeled` events for labels other than `action-translation` — it triggers on `labeled` (necessary, since labels are applied after the PR is opened), and a sync applying two labels was starting a full review per label. Templates now also declare `permissions` explicitly (`pull-requests: write`, required to remove duplicate comments). Existing target repos should copy these guards; without them a sync bills several reviews of the same diff.
 
+## [0.16.1] - 2026-07-15
+
 ### Fixed
 - **French typography no longer corrupts footnote/link-reference definitions**: the NBSP pass rewrote `[^id]: text` as `[^id] : text`, which stops the line parsing as a definition — it rendered as literal text and broke every reference (shipped in the fr seed, e.g. `pandas.md`). Definition labels are now masked, and the exact corruption is repaired on contact, so running `scripts/typography/apply.mjs` over an affected repo heals it. The definition text after the colon is still typeset.
 - **GitHub API pagination** at all five unpaginated call sites: sync `pulls.listFiles` (PRs touching >30 files were silently truncated), review mode's two `listFiles` calls, rebase's `pulls.list` (sibling PRs beyond 100), and `postReviewComment`'s `listComments` (the existing review comment was missed past 30 comments, accumulating duplicates).

dist-action/index.js

@@ -34002,6 +34002,12 @@ var {
 
 // dist/heading-map.js
 var PATH_SEPARATOR = "::";
+function normalizeHeadingForMatch(heading) {
+  return MystParser.stripMystRoles(heading.replace(/^#+\s+/, "")).replace(/[\u00A0\u202F]/g, " ").replace(/\s+/g, " ").replace(/ ([;:!?])/g, "$1").trim();
+}
+function cleanHeadingText(heading) {
+  return MystParser.stripMystRoles(heading.replace(/^#+\s+/, "").trim());
+}
 function extractHeadingMap(content) {
   const map2 = /* @__PURE__ */ new Map();
   const frontmatterMatch = content.match(/^---\n(.*?)\n---/s);
@@ -34357,7 +34363,14 @@ var FileProcessor = class {
           const sourceHeading = MystParser.stripMystRoles(newSection.heading.replace(/^#+\s+/, "").trim());
           const translatedHeading = cachedHeadingMap.get(sourceHeading);
           if (translatedHeading) {
-            const cachedSection = cachedParsed.sections.find((s) => MystParser.stripMystRoles(s.heading.replace(/^#+\s+/, "").trim()) === translatedHeading);
+            const wantedCached = normalizeHeadingForMatch(translatedHeading);
+            let cachedSection = cachedParsed.sections.find((s) => cleanHeadingText(s.heading) === translatedHeading);
+            if (!cachedSection) {
+              const normalizedCached = cachedParsed.sections.filter((s) => normalizeHeadingForMatch(s.heading) === wantedCached);
+              if (normalizedCached.length === 1) {
+                cachedSection = normalizedCached[0];
+              }
+            }
             if (cachedSection) {
               resultSections.push(cachedSection);
               includedSourceSections.push(newSection);
@@ -34433,7 +34446,7 @@ ${bodyLines.join("\n")}`;
         };
         const mergeSubsectionsWithTargetTranslations = (sourceSubs, targetSubs, parentPath = "") => {
           return sourceSubs.map((sourceSub, i2) => {
-            const sourceHeading = sourceSub.heading.replace(/^#+\s+/, "");
+            const sourceHeading = cleanHeadingText(sourceSub.heading);
             const targetSub = targetSubs[i2];
             if (targetSub) {
               const currentPath = parentPath ? `${parentPath}::${sourceHeading}` : sourceHeading;
@@ -34453,10 +34466,20 @@ ${bodyLines.join("\n")}`;
             const sourcePath = parentPath ? `${parentPath}::${sourceHeading}` : sourceHeading;
             const chineseHeading = headingMap.get(sourcePath);
             if (chineseHeading) {
-              this.log(`Found Chinese heading in map for: ${sourcePath} \u2192 ${chineseHeading}`);
+              const bareHeading = sourceSub.heading.replace(/^#+\s+/, "").trim();
+              if (bareHeading !== cleanHeadingText(sourceSub.heading)) {
+                return sourceSub;
+              }
+              this.log(`Found translated heading in map for: ${sourcePath} \u2192 ${chineseHeading}`);
+              const translatedHeadingLine = sourceSub.heading.replace(/^(#+\s+).*/, `$1${chineseHeading}`);
               return {
                 ...sourceSub,
-                heading: sourceSub.heading.replace(/^(#+\s+).*/, `$1${chineseHeading}`),
+                heading: translatedHeadingLine,
+                // serializeSection writes .content, whose first line is the
+                // heading — the substitution must land there too, or the body
+                // keeps the English heading while updateHeadingMap records the
+                // translated one and the next sync can't match the section.
+                content: sourceSub.content.replace(/^#+[^\n]*/, translatedHeadingLine),
                 subsections: mergeSubsectionsWithTargetTranslations(sourceSub.subsections, [], sourcePath)
               };
             }
@@ -34655,11 +34678,21 @@ ${translatedContent}`;
       for (const targetSection of targetSections) {
         const cleanTargetHeading = targetSection.heading.replace(/^#+\s+/, "").trim();
         if (cleanTargetHeading === translatedHeading) {
-          this.log(`  \u2713 Found by heading map: "${translatedHeading}"`);
+          this.log(`  \u2713 Found by heading map (exact): "${translatedHeading}"`);
           return targetSection;
         }
       }
-      this.log(`  \u2717 Not found by heading map`);
+      const wantedHeading = normalizeHeadingForMatch(translatedHeading);
+      const normalizedMatches = targetSections.filter((t) => normalizeHeadingForMatch(t.heading) === wantedHeading);
+      if (normalizedMatches.length === 1) {
+        this.log(`  \u2713 Found by heading map (normalized): "${translatedHeading}"`);
+        return normalizedMatches[0];
+      }
+      if (normalizedMatches.length > 1) {
+        this.log(`  \u2717 ${normalizedMatches.length} sections normalize to "${wantedHeading}" \u2014 ambiguous, falling through`);
+      } else {
+        this.log(`  \u2717 Not found by heading map`);
+      }
     } else {
       this.log(`  Strategy 1: No heading map entry for "${sourceSection.heading.replace(/^#+\s+/, "").trim()}"`);
     }