Skip to content

Feedback: add a first-class consistency check for recurring document-local phrases (same source phrase → one target rendering) #133

Description

@mmcky

Field report from the QuantEcon/lecture-python.zh-cn#188 review (career resync, part of the QuantEcon/project-translation#14 mini-wave), filed as feedback on review mode.

What happened

The source lecture uses "stay put" as a coined, document-local term — 10 occurrences across prose, a NamedTuple docstring, code comments, and two ax.text figure annotations. The zh-cn translation rendered it three different ways: 原地不动 (7×, including the definitional line that introduces the term), 维持现状 (one figure label), 保持现状 (one prose line and the other figure label — i.e. the same annotation in two figures differed from each other). The pre-resync target on main was worse (a 10-way mix trending 保持现状); the resync narrowed it to 3 stray variants but did not unify. Fixed by hand in QuantEcon/lecture-python.zh-cn@8ae3b2d.

The reviewer did catch this — but only as free-prose in the Suggestions section, incidentally, because the whole document happened to be in context. Nothing in the structured output (criteria table, verdict) represents it, nothing enforces it at translation time, and a diff-scoped review of a smaller change would likely have missed it since the variants sat in unchanged sections.

Why the glossary doesn't cover it

The glossary is program-wide terminology (贴现因子, 贝尔曼算子, …). Coinages like "stay put"/"new job"/"new life" are lecture-specific labels that don't belong in a 357-term global glossary — but they function as terminology within their document and must be internally consistent, including across surfaces (prose ↔ code comments ↔ figure labels ↔ docstrings).

Suggested shape

  1. Deterministic pre-pass (cheap, no LLM): extract recurring quoted/emphasised source phrases (e.g. strings appearing ≥N times, quoted terms, repeated ax.text/label literals) and check the target uses exactly one rendering per phrase. The career case is fully catchable this way — "stay put" appears quoted in its definitional line and verbatim in figure labels.
  2. Reviewer criterion: promote intra-document phrase consistency from ad hoc prose into the structured criteria (it naturally belongs under Terminology, which scored 8/10 here without machine-readable detail), so it survives into the metadata contract (feat: document translation-sync-metadata as a stable, versioned contract for downstream tooling #66) and can gate/route per Design: route sync PRs by review verdict (auto-merge vs editor label) and make human review measurable #103.
  3. Translator-side prompt rule: instruct translation/resync to identify recurring source phrases and hold one rendering per document — cheapest place to prevent the drift, and the only place that helps section-scoped syncs, which never see the other occurrences.

Context sensitivity matters in the other direction too: the check should key on the source phrase, not the target string, so legitimately different translations of incidentally-identical source words in different senses don't false-positive.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementImprovement to existing content or functionality

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions