diff --git a/CHANGELOG.md b/CHANGELOG.md index c3d86c644..90c0ea9f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,15 @@ # Changelog +## 1.0.2-beta.0 — 2026-08-19 + +### Fixes + +- Stop the localized navigation referencing pages that were never translated, which is what still discarded a partial run. `--allow-partial` published what succeeded — and then `--update-nav` regenerated the nav from the ENGLISH tree, emitting an entry for the failed page in the language that failed it, so `mintlify validate` rejected the missing file and the job died before its push anyway. The 784 pages that HAD translated went with it, which is precisely the loss `--allow-partial` exists to prevent. Nav generation now omits any localized page whose file is not on disk, prunes a group left with no pages and a tab left with no groups, and keeps an `openapi` group that never had pages to begin with. The check is injected rather than hardcoded, so the pure transform stays testable and the two paths that actually write `docs.json` get the real one. This also closes the same hazard from every other direction it can arrive from — a pruned page, or a translation that only exists on an unmerged branch — because the nav is now derived from what is present rather than from what English says should be. (#725) + +- Stop the localized nav crashing on a group that has no pages, and stop it dropping the properties it does have. `buildLanguageNav` rebuilt every group as `{group, pages}` and called `group.pages.map(...)` unconditionally. The docs rebuild added `{group, expanded, openapi}` — a group whose content is an OpenAPI spec and has no pages at all — so `--update-nav` died with `TypeError: undefined is not an object`, **after 784 pages had already been translated**, taking the whole nightly run with it for the second night running. Groups are now rebuilt by spreading the English group, so `expanded`, `icon` and `openapi` survive instead of being silently discarded from every non-English nav; a group with no pages is carried through untouched (the spec is not translated, and dropping it would remove the API reference from thirteen languages); and `pages` entries that are themselves nested groups recurse rather than being prefixed as if they were paths. `pages` is optional on the type now, which is what it always was in the data. (#725) + +- Stop one bad page discarding a whole night's translation, and resample the draw that caused it. On 2026-08-18 the nightly job translated 782 pages, hit ONE failure, exited 1, and pushed nothing — 2.7M tokens in the bin over a single page. Two things were wrong. **`--allow-partial`** (used by the box job, off everywhere else) publishes what succeeded and carries the failures into the PR body, the Slack note and the run stamp, because the real hazard of a partial publish is a PR that looks complete. And the failure itself was misdiagnosed: `reference/cloud-cli.mdx [vi]` is a 16 KB source that emitted 64 000 output tokens, which is a repetition loop, not a page too large to translate. `stop_reason: "max_tokens"` was treated as a size problem no resample could fix, so it propagated uncaught and consumed no attempt. It is now split by the one signal that separates the two — output against the source's token estimate — and a runaway is retried through the existing validity loop with feedback telling the model to translate once and stop, while a source that genuinely approaches the ceiling keeps failing loudly rather than burning three attempts to arrive at the same place. (#725) + ## 1.0.1-beta.2 — 2026-08-17 ### Features @@ -483,7 +493,6 @@ never "blocked". ## 1.0.0-beta.8 — 2026-08-06 - ### Features - Add `failproofai uninstall` — the sanctioned way off a machine. npm runs no uninstall script, so `npm rm -g failproofai` deletes the package and leaves behind everything durable it installed: hook entries in up to twelve agent CLIs' settings files and a root-owned systemd unit. Those leftovers are not inert — the hook entries invoke `npx -y failproofai`, which re-downloads the package, so a "removed" failproofai keeps running on every tool call; and on a `daemonConfigured` machine the surviving unit points at a worker script npm just deleted, which under fail-closed semantics denies EVERY tool call with nothing on screen naming the cause. The command clears `daemonConfigured` **first**, before hooks and before the service, so a partial uninstall can only ever fail open — the intuitive order leaves a window where the flag demands a daemon that is already gone, and that window is a total agent lockout. `--purge` also deletes `~/.failproofai`; `--dry-run` shows the plan; `--yes` skips the prompt, which is required rather than assumed when there is no TTY. Incomplete cleanup exits non-zero and prints the exact `sudo` commands to finish, and `--purge` suppresses the command's own telemetry — resolving an instance id lazily WRITES `state/telemetry-id`, which re-created the whole directory seconds after deleting it and left a just-wiped machine holding a brand-new tracking identifier. (#694) @@ -502,7 +511,6 @@ never "blocked". - Say which fault the probe actually hit. `DaemonFailure` reports `unreachable` for BOTH a refused connection and a request that was accepted and never answered, so setup told people their worker would not start when nothing was listening at all — sending them to inspect a healthy process. `probeDaemon` now distinguishes "never accepted a connection" from "accepted, but could not answer a hook", and the wizard prints the matching remedy. (#694) - ### Chores - Add `scripts/repro-npm-install.sh`, which reproduces a real user install in the shape that actually breaks: `npm i -g` into a ROOT-owned prefix, then the CLI run by an unprivileged user, with real systemd in the container. A single-user laptop cannot exercise that split — its npm prefix is owned by the person running the hooks — which is how the root-owned policy-shim fail-open shipped. It also guards two traps found while writing it: cgroup v2 needs `--cgroupns=host` plus tmpfs mounts or the container exits 255 with an empty `docker logs`, and `npm pack --ignore-scripts` skips the `prepare` rebuild, so the script asserts the version inside the tarball rather than the one in `package.json`. diff --git a/__tests__/integration-suite/local-runner.test.ts b/__tests__/integration-suite/local-runner.test.ts index 9ee58ace8..0b8214004 100644 --- a/__tests__/integration-suite/local-runner.test.ts +++ b/__tests__/integration-suite/local-runner.test.ts @@ -546,6 +546,26 @@ describe("translate job", () => { expect(translateSh).toMatch(/\[ "\$drift" = 0 \] \|\| die/); }); + it("publishes what succeeded instead of discarding it on one bad page", () => { + // 2026-08-18: ONE page overran the output limit, cli.ts exited 1, this job + // died before its push, and 782 completed translations (2.7M tokens) went + // in the bin. --allow-partial makes the run publish and REPORT instead. + expect(translateSh).toMatch(/--allow-partial/); + // PIPESTATUS, not $? — `bun … | tee` would otherwise report tee's status + // and a real failure would sail through as success. + expect(translateSh).toMatch(/PIPESTATUS\[0\]/); + }); + + it("carries a partial run into the PR body and Slack, never silently", () => { + // The danger of publishing partial output is that the PR LOOKS complete. + // The failed pages have to travel with it. + expect(translateSh).toMatch(/grep -q "\^PARTIAL RUN"/); + expect(translateSh).toMatch(/This run was PARTIAL/); + expect(translateSh).toMatch(/published a PARTIAL run/); + // and the stamp says partial, so the weekly audit does not read it as ok + expect(translateSh).toMatch(/stamp partial/); + }); + it("stamps every exit so 'never ran' is detectable from outside", () => { // The one failure no error handler can report is the job not starting, so // the signal has to be a file whose AGE another job can read. diff --git a/__tests__/scripts/translate-docs/mintlify-nav.test.ts b/__tests__/scripts/translate-docs/mintlify-nav.test.ts index 1f8550706..e366b29f8 100644 --- a/__tests__/scripts/translate-docs/mintlify-nav.test.ts +++ b/__tests__/scripts/translate-docs/mintlify-nav.test.ts @@ -232,6 +232,7 @@ describe("localizeProductsNavigation", () => { }, ], ["es", "ja"], + () => true, ); expect(product.tabs).toBeUndefined(); @@ -244,15 +245,129 @@ describe("localizeProductsNavigation", () => { "es", "ja", ]); - expect(languages[1].tabs[0].groups[0].pages).toEqual([ + expect(languages[1].tabs[0].groups[0].pages!).toEqual([ "es/agenteye/overview", ]); }); + it("carries a group that has no pages, instead of crashing on it", () => { + // The docs rebuild introduced `{group, expanded, openapi}` — a group whose + // content is an OpenAPI spec, with no `pages` at all. buildLanguageNav did + // `group.pages.map(...)` unconditionally and took the whole nightly + // translation down with a TypeError, AFTER 784 pages had been translated. + const tabs = [ + { + tab: "Integrations and reference", + groups: [ + { group: "Guides", pages: ["intro"] }, + { group: "HTTP API", expanded: false, openapi: "reference/openapi.json" }, + ], + }, + ]; + const zh = buildLanguageNav(tabs as never, "zh"); + const groups = zh.tabs[0].groups; + expect(groups[0].pages![0]).toBe("zh/intro"); + // Passed through untouched — the spec is not translated, and dropping the + // group would remove the API reference from every non-English nav. + expect(groups[1].openapi).toBe("reference/openapi.json"); + expect(groups[1].pages).toBeUndefined(); + }); + + it("preserves group properties the old builder silently dropped", () => { + // It rebuilt each group as {group, pages}, so `expanded`, `icon` and + // anything else vanished from every localized nav. + const tabs = [{ tab: "Docs", groups: [{ group: "G", expanded: true, icon: "book", pages: ["a"] }] }]; + const de = buildLanguageNav(tabs as never, "de"); + expect(de.tabs[0].groups[0].expanded).toBe(true); + expect(de.tabs[0].groups[0].icon).toBe("book"); + }); + + it("recurses into nested groups rather than prefixing them as paths", () => { + const tabs = [ + { tab: "Docs", groups: [{ group: "Outer", pages: ["top", { group: "Inner", pages: ["deep"] }] }] }, + ]; + const ja = buildLanguageNav(tabs as never, "ja"); + const outer = ja.tabs[0].groups[0]; + expect(outer.pages![0]).toBe("ja/top"); + expect((outer.pages![1] as { pages?: string[] }).pages![0]).toBe("ja/deep"); + }); + + it("recurses through a group's `groups`, not only through its `pages`", () => { + // A group may nest via `groups` as well as inside `pages`, and those are two + // separate branches in localizeGroup. Deleting the `groups` branch entirely + // left the whole suite green, so this covers it independently. + const tabs = [ + { + tab: "Docs", + groups: [ + { + group: "Outer", + pages: ["top"], + groups: [{ group: "Nested", pages: ["deep"] }], + }, + ], + }, + ]; + const ko = buildLanguageNav(tabs as never, "ko"); + const outer = ko.tabs[0].groups[0]; + expect(outer.pages![0]).toBe("ko/top"); + expect(outer.groups![0].pages![0]).toBe("ko/deep"); + }); + + it("omits a page whose localized file is missing, and keeps the rest", () => { + // THE PARTIAL-RUN CASE. One page fails to translate for one language; the + // English tree still lists it, so the nav used to emit `vi/reference/cloud-cli` + // regardless, `mintlify validate` rejected the missing file, and the job died + // before its push — discarding 784 pages that HAD translated. The entry is + // dropped instead: the page simply does not exist in that language yet. + const tabs = [ + { + tab: "Docs", + groups: [{ group: "Reference", pages: ["index", "reference/cloud-cli"] }], + }, + ]; + const missing = "vi/reference/cloud-cli.mdx"; + const vi = buildLanguageNav(tabs as never, "vi", (rel) => rel !== missing); + + expect(vi.tabs[0].groups[0].pages).toEqual(["vi/index"]); + // and the language that DID translate it keeps it + const zh = buildLanguageNav(tabs as never, "zh", () => true); + expect(zh.tabs[0].groups[0].pages).toEqual(["zh/index", "zh/reference/cloud-cli"]); + }); + + it("drops a group left with no pages, and a tab left with no groups", () => { + // Filtering can empty a group, and an empty group is its own validation + // error — so the pruning has to go all the way up. + const tabs = [ + { tab: "Solo", groups: [{ group: "Only", pages: ["gone"] }] }, + { tab: "Mixed", groups: [{ group: "Kept", pages: ["here"] }, { group: "Empty", pages: ["gone2"] }] }, + ]; + const nav = buildLanguageNav(tabs as never, "ja", (rel) => !rel.includes("gone")); + + expect(nav.tabs.map((t) => t.tab)).toEqual(["Mixed"]); + expect(nav.tabs[0].groups.map((g) => g.group)).toEqual(["Kept"]); + }); + + it("keeps an openapi group even though it has no pages to check", () => { + // hasContent must not confuse "emptied by filtering" with "never had pages". + const tabs = [ + { + tab: "Reference", + groups: [ + { group: "HTTP API", expanded: false, openapi: "reference/openapi.json" }, + { group: "Guides", pages: ["gone"] }, + ], + }, + ]; + const de = buildLanguageNav(tabs as never, "de", () => false); + expect(de.tabs[0].groups.map((g) => g.group)).toEqual(["HTTP API"]); + expect(de.tabs[0].groups[0].openapi).toBe("reference/openapi.json"); + }); + it("uses Mintlify's canonical Portuguese locale with existing paths", () => { const portuguese = buildLanguageNav(sampleEnglishTabs, "pt-br"); expect(portuguese.language).toBe("pt-BR"); - expect(portuguese.tabs[0].groups[0].pages[0]).toBe("pt-br/introduction"); + expect(portuguese.tabs[0].groups[0].pages![0]).toBe("pt-br/introduction"); }); }); diff --git a/__tests__/scripts/translate-docs/translator.test.ts b/__tests__/scripts/translate-docs/translator.test.ts index 4ce1dd1ec..c96584d60 100644 --- a/__tests__/scripts/translate-docs/translator.test.ts +++ b/__tests__/scripts/translate-docs/translator.test.ts @@ -44,6 +44,76 @@ describe("translateContent", () => { streamMock.mockReset(); }); + it("ignores a non-positive TRANSLATE_RUNAWAY_RATIO", async () => { + // `parseInt(...) || 6` accepted a negative, and a negative ratio makes + // `output > source * ratio` true for EVERY response — so a genuinely + // oversized page would be misread as a runaway and burn all three attempts + // arriving exactly where it started. Re-imported under the hostile value: + // the guard must fall back to the default and still call this NOT retryable. + const prev = process.env.TRANSLATE_RUNAWAY_RATIO; + process.env.TRANSLATE_RUNAWAY_RATIO = "-1"; + vi.resetModules(); + try { + const fresh = await import("@/scripts/translate-docs/translator"); + mockFinalMessage({ + stop_reason: "max_tokens", + content: [{ type: "text", text: "big…" }], + usage: { input_tokens: 60000, output_tokens: 64000 }, + }); + let err!: Error & { retryable?: boolean }; + try { + await fresh.translateContent("y".repeat(200000), "vi", "Vietnamese"); + } catch (e) { + err = e as Error & { retryable?: boolean }; + } + expect(err.retryable).toBe(false); + expect(err.message).toMatch(/source too large/); + } finally { + if (prev === undefined) delete process.env.TRANSLATE_RUNAWAY_RATIO; + else process.env.TRANSLATE_RUNAWAY_RATIO = prev; + vi.resetModules(); + } + }); + + it("marks a truncation whose output dwarfs the source as retryable", async () => { + // reference/cloud-cli.mdx [vi], on the box, 2026-08-18: a 16 KB source + // emitted 64 000 output tokens. That is a repetition loop, not a page too + // big to translate — and it failed the whole nightly run because the throw + // was treated as a size problem no resample could fix. + mockFinalMessage({ + stop_reason: "max_tokens", + content: [{ type: "text", text: "loop…" }], + usage: { input_tokens: 5000, output_tokens: 64000 }, + }); + let err!: Error & { retryable?: boolean }; + try { + await translateContent("x".repeat(16000), "vi", "Vietnamese"); + } catch (e) { + err = e as Error & { retryable?: boolean }; + } + expect(err.message).toMatch(/runaway sample/); + expect(err.retryable).toBe(true); + }); + + it("leaves a genuinely oversized source NOT retryable", async () => { + // Output within a small multiple of the source is a page that really does + // not fit. Retrying it would burn the whole attempt budget to land in the + // same place, so the old fail-loud behaviour stands. + mockFinalMessage({ + stop_reason: "max_tokens", + content: [{ type: "text", text: "big…" }], + usage: { input_tokens: 60000, output_tokens: 64000 }, + }); + let err!: Error & { retryable?: boolean }; + try { + await translateContent("y".repeat(200000), "vi", "Vietnamese"); + } catch (e) { + err = e as Error & { retryable?: boolean }; + } + expect(err.message).toMatch(/source too large/); + expect(err.retryable).toBe(false); + }); + it("throws when the model truncates the output at max_tokens", async () => { // A truncated response leaves malformed MDX (unbalanced braces) that would // otherwise be written to disk and cached, then fail `mintlify validate`. @@ -210,26 +280,53 @@ describe("translateValidated", () => { expect(result.attempts).toBe(2); }); - it("does not retry a response truncated at max_tokens", async () => { - // translateContent throws on max_tokens before returning; that is not a - // validity failure, so translateValidated must let it propagate uncaught. + it("does not retry a truncation caused by a genuinely oversized source", async () => { + // Output within a small multiple of the source is a page that really does + // not fit; retrying spends the whole budget to land in the same place. Note + // `base.source` is what sets the ratio — a large source with a large output + // is the not-retryable shape. mockFinalMessage({ stop_reason: "max_tokens", content: [{ type: "text", text: "partial…" }], - usage: { input_tokens: 1, output_tokens: 64000 }, + usage: { input_tokens: 60000, output_tokens: 64000 }, }); await expect( translateValidated({ ...base, + source: "z".repeat(200000), lang: "he", langName: "Hebrew", validate: async () => null, }), - ).rejects.toThrow(/truncated at max_tokens/); + ).rejects.toThrow(/source too large/); expect(streamMock).toHaveBeenCalledTimes(1); }); + it("DOES retry a truncation whose output dwarfs the source", async () => { + // The narrowing of the rule above. A small source that emitted 64k tokens + // is a repetition loop, and a resample is exactly what fixes it — the + // failure that killed the 2026-08-18 nightly run and discarded 782 good + // pages with it. First draw loops, second draw succeeds. + streamMock.mockReturnValueOnce({ + finalMessage: async () => ({ + stop_reason: "max_tokens", + content: [{ type: "text", text: "loop…" }], + usage: { input_tokens: 10, output_tokens: 64000 }, + }), + }); + queueFinalMessage("# translated", { input_tokens: 10, output_tokens: 40 }); + + const result = await translateValidated({ + ...base, + lang: "vi", + langName: "Vietnamese", + validate: async () => null, + }); + expect(result.rendered).toContain("# translated"); + expect(streamMock).toHaveBeenCalledTimes(2); + }); + it("does not retry when the request itself throws", async () => { streamMock.mockReturnValue({ finalMessage: async () => { diff --git a/integration-suite/local/jobs/translate.sh b/integration-suite/local/jobs/translate.sh index 12f7dac97..89d0c484a 100644 --- a/integration-suite/local/jobs/translate.sh +++ b/integration-suite/local/jobs/translate.sh @@ -172,8 +172,22 @@ bun install --frozen-lockfile --ignore-scripts || die "bun install failed" step "translate" FORCE_FLAG="" [ "${TRANSLATE_FORCE:-0}" = "1" ] && FORCE_FLAG="--force" +# --allow-partial: publish what succeeded. Without it ONE failing page exits 1 +# and this job dies before its push — which on 2026-08-18 discarded 782 good +# translations (2.7M tokens) over a single page that overran the output limit. +# The failures are not swallowed: cli.ts prints FAILED PAGES + a PARTIAL RUN +# marker, and both are carried into Slack and the PR body below. +TR_LOG="$(mktemp)" # shellcheck disable=SC2086 -bun run translate --languages "$LANGS" $FORCE_FLAG || die "translation failed" +bun run translate --languages "$LANGS" $FORCE_FLAG --allow-partial 2>&1 | tee "$TR_LOG" +# PIPESTATUS, not $? — that would be tee's. +[ "${PIPESTATUS[0]}" = 0 ] || die "translation failed" +PARTIAL="" +if grep -q "^PARTIAL RUN" "$TR_LOG"; then + PARTIAL="$(sed -n '/^FAILED PAGES/,/^PARTIAL RUN/p' "$TR_LOG")" + echo "⚠ partial run — carrying the failures into the report" +fi +rm -f "$TR_LOG" # Parses every page AND checks image references resolve on disk — a broken # image path is valid MDX, so `mintlify validate` passes it to a reader's @@ -312,7 +326,12 @@ if [ -z "$PR_NUMBER" ]; then - Only changed pages were re-translated (content-hash cache) - All 14 languages across 3 tiers -- Box run \`$TS\` against \`$REF_DESC\` @ \`$FP_SHA\`" +- Box run \`$TS\` against \`$REF_DESC\` @ \`$FP_SHA\`${PARTIAL:+ + +**This run was PARTIAL — the pages below are missing from it and still need a translation:** +\`\`\` +$PARTIAL +\`\`\`}" CREATED="$(api POST /pulls "$(node -e 'process.stdout.write(JSON.stringify({title:process.argv[1],body:process.argv[2],base:process.argv[3],head:process.argv[4]}))' \ "$PR_TITLE" "$BODY" "$BASE_BRANCH" "$BRANCH")" \ | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{try{const p=JSON.parse(s);process.stdout.write(p.number?String(p.number):"")}catch{process.stdout.write("")}})')" @@ -323,5 +342,16 @@ else echo "pushed $CHANGED files to https://github.com/$REPO/pull/$PR_NUMBER" fi -stamp ok "PR #$PR_NUMBER on $BRANCH" +if [ -n "$PARTIAL" ]; then + stamp partial "PR #$PR_NUMBER on $BRANCH, with failures" + # Quiet-on-success does not extend to a run that silently dropped pages: the + # PR looks complete, and only this line says it is not. + slack_note "⚠️ *nightly translation published a PARTIAL run* — \`$REF_DESC\` @ \`$FP_SHA\` +PR #$PR_NUMBER on \`$BRANCH\` is missing the pages below. +\`\`\` +$PARTIAL +\`\`\`" +else + stamp ok "PR #$PR_NUMBER on $BRANCH" +fi echo "── done: PR #$PR_NUMBER on $BRANCH ──" diff --git a/scripts/translate-docs/cli.ts b/scripts/translate-docs/cli.ts index 5096d0d77..03a7a66bd 100644 --- a/scripts/translate-docs/cli.ts +++ b/scripts/translate-docs/cli.ts @@ -39,6 +39,11 @@ const { values: args } = parseArgs({ validate: { type: "boolean", default: false }, prune: { type: "boolean", default: false }, "no-prune": { type: "boolean", default: false }, + // Publish what succeeded instead of discarding it. One page failing used to + // exit 1, which made the box job die before its push — on 2026-08-18 that + // threw away 782 good translations (2.7M tokens) because ONE page overran + // the output limit. Opt-in, so CI and hand runs keep failing loudly. + "allow-partial": { type: "boolean", default: false }, model: { type: "string", short: "m" }, help: { type: "boolean", short: "h", default: false }, }, @@ -56,6 +61,8 @@ Options: --readme-only Only translate the README --docs-only Only translate Mintlify docs --dry-run Show what would be translated without calling the API + --allow-partial Exit 0 when SOME pages translated, listing the failures + (default: any failure exits 1 and publishes nothing) -f, --force Ignore cache, re-translate everything --update-nav Regenerate docs.json navigation after translation --validate Check all nav references resolve to files @@ -441,6 +448,20 @@ async function main() { } if (errors.length > 0) { + // Printed either way — a partial run must never read as a clean one. + console.log(`\nFAILED PAGES (${errors.length}):`); + for (const e of errors) { + console.log(` ${e.source} [${e.lang}]: ${e.error}`); + } + if (args["allow-partial"] && translated.length > 0) { + // A stable marker the box job greps for, so the failure reaches Slack and + // the PR body instead of being buried in a 700-line log. + console.log( + `\nPARTIAL RUN — ${translated.length} page(s) translated, ${errors.length} failed. ` + + `Publishing what succeeded.`, + ); + return; + } process.exit(1); } } diff --git a/scripts/translate-docs/mintlify-nav.ts b/scripts/translate-docs/mintlify-nav.ts index 24b6f5f32..4e7b058bb 100644 --- a/scripts/translate-docs/mintlify-nav.ts +++ b/scripts/translate-docs/mintlify-nav.ts @@ -1,19 +1,29 @@ -import { readFileSync, writeFileSync } from "node:fs"; +import { existsSync, readFileSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { getLanguageByCode, NAV_TRANSLATIONS } from "./config"; const __dirname = dirname(fileURLToPath(import.meta.url)); -const DOCS_JSON_PATH = join(__dirname, "..", "..", "docs", "docs.json"); +const DOCS_DIR = join(__dirname, "..", "..", "docs"); +const DOCS_JSON_PATH = join(DOCS_DIR, "docs.json"); interface NavGroup { group: string; - pages: string[]; + // OPTIONAL, and both of these are why: a Mintlify group may carry `pages`, or + // nested `groups`, or neither — the docs rebuild introduced a group whose + // content is an `openapi` spec and has no pages at all. `pages` entries are + // likewise strings OR nested groups. Typing these as required is what let + // `group.pages.map` crash the nightly run. + pages?: (string | NavGroup)[]; + groups?: NavGroup[]; + // Everything else (expanded, icon, openapi, …) rides along untouched. + [key: string]: unknown; } interface NavTab { tab: string; groups: NavGroup[]; + [key: string]: unknown; } interface LanguageNav { @@ -76,9 +86,33 @@ export function getNavigationPageReferences( * Build a navigation entry for a specific language by transforming the * English navigation structure. */ +/** + * Does the localized file for a nav entry exist on disk? + * + * Defaults to "yes" so the pure transform stays testable with fixtures, and the + * REAL check is injected by updateDocsJson / localizeProductsNavigation, which + * are the two paths that write docs.json. + */ +export type PageExists = (relativePath: string) => boolean; + +const fileOnDisk: PageExists = (rel) => existsSync(join(DOCS_DIR, rel)); + +/** + * Keep a group only if it still carries something to render. Dropping every + * missing page can empty a group, and an empty group is its own validation + * error — but a group whose content is an `openapi` spec has no pages by + * design and must survive. + */ +function hasContent(group: NavGroup): boolean { + if (Array.isArray(group.pages) && group.pages.length > 0) return true; + if (Array.isArray(group.groups) && group.groups.length > 0) return true; + return !Array.isArray(group.pages) && !Array.isArray(group.groups); +} + export function buildLanguageNav( englishTabs: NavTab[], lang: string, + exists: PageExists = () => true, ): LanguageNav { const t = NAV_TRANSLATIONS[lang]; if (!t) throw new Error(`No nav translations for language: ${lang}`); @@ -97,13 +131,47 @@ export function buildLanguageNav( Examples: t.examples, }; - const tabs: NavTab[] = englishTabs.map((tab) => ({ - tab: tabNameMap[tab.tab] || tab.tab, - groups: tab.groups.map((group) => ({ - group: groupNameMap[group.group] || group.group, - pages: group.pages.map((page) => `${lang}/${page}`), - })), - })); + // Rebuilt by SPREADING the English group, not by picking two fields off it. + // The old form silently dropped every other key — `expanded`, `icon`, + // `openapi` — so a localized nav quietly lost them, and it crashed outright + // on the first group that had no `pages` at all. + const localizeGroup = (group: NavGroup): NavGroup => { + const out: NavGroup = { ...group, group: groupNameMap[group.group] || group.group }; + if (Array.isArray(group.pages)) { + // A page entry is a path to prefix, or a nested group to recurse into. + // + // A path is DROPPED when its localized file is not on disk. The English + // tree is the source of the nav, so without this every English page gets + // an entry in all fourteen languages whether or not it was translated — + // and `mintlify validate` rejects the run. That took down a nightly job + // that had already translated 784 pages: one page failed, the nav still + // referenced it in that language, and every successful page was discarded + // with it. Omitting the entry is the honest state: the page does not + // exist in that language yet. + out.pages = group.pages + .map((page) => + typeof page === "string" ? `${lang}/${page}` : localizeGroup(page), + ) + .filter((page) => + typeof page === "string" ? exists(`${page}.mdx`) : hasContent(page), + ); + } + if (Array.isArray(group.groups)) { + out.groups = group.groups.map(localizeGroup).filter(hasContent); + } + // No pages and no groups (an `openapi` group): carried through as-is. The + // spec is not translated, and dropping the group would remove the API + // reference from every non-English nav. + return out; + }; + + const tabs: NavTab[] = englishTabs + .map((tab) => ({ + ...tab, + tab: tabNameMap[tab.tab] || tab.tab, + groups: (tab.groups ?? []).map(localizeGroup).filter(hasContent), + })) + .filter((tab) => tab.groups.length > 0); return { language: getLanguageByCode(lang)?.mintlifyCode ?? lang, @@ -125,10 +193,11 @@ export function readDocsConfig(): Record { export function generateLanguagesArray( englishTabs: NavTab[], langCodes: string[], + exists: PageExists = () => true, ): LanguageNav[] { // English first (default) const english: LanguageNav = { language: "en", tabs: englishTabs }; - const others = langCodes.map((code) => buildLanguageNav(englishTabs, code)); + const others = langCodes.map((code) => buildLanguageNav(englishTabs, code, exists)); return [english, ...others]; } @@ -136,6 +205,9 @@ export function generateLanguagesArray( export function localizeProductsNavigation( products: unknown[], langCodes: string[], + // Defaults to the real disk check, like updateDocsJson — injectable so the + // pure transform can be exercised against fixtures that are not on disk. + exists: PageExists = fileOnDisk, ): Record[] { return products.map((value) => { const product = value as Record; @@ -149,7 +221,7 @@ export function localizeProductsNavigation( const { tabs: _tabs, ...rest } = product; return { ...rest, - languages: generateLanguagesArray(englishTabs, langCodes), + languages: generateLanguagesArray(englishTabs, langCodes, exists), }; }); } @@ -177,7 +249,7 @@ export function updateDocsJson(langCodes: string[]): void { } const newNav: Record = { - languages: generateLanguagesArray(englishTabs, langCodes), + languages: generateLanguagesArray(englishTabs, langCodes, fileOnDisk), }; if (nav.global) { newNav.global = nav.global; diff --git a/scripts/translate-docs/translator.ts b/scripts/translate-docs/translator.ts index fc93f4d45..5c8166e17 100644 --- a/scripts/translate-docs/translator.ts +++ b/scripts/translate-docs/translator.ts @@ -44,6 +44,24 @@ const MAX_ATTEMPTS = ? parsedMaxAttempts : 3; +// How far past the source an output may run before a max_tokens stop reads as a +// repetition loop rather than a large page. Translations of the verbose targets +// (hi, vi, ja) sit near 2x the source's token estimate; 6x is far outside that +// band and was ~16x in the observed failure. +// Validated the same way MAX_TOKENS and MAX_ATTEMPTS above are, and for a +// sharper reason: `parseInt(...) || 6` accepts a NEGATIVE, and a negative ratio +// makes `output > source * ratio` true for every response — so a genuinely +// oversized page would be classified as a runaway and burn all three attempts +// to arrive exactly where it started. +const parsedRunawayRatio = Number.parseInt( + process.env.TRANSLATE_RUNAWAY_RATIO ?? "", + 10, +); +const RUNAWAY_RATIO = + Number.isInteger(parsedRunawayRatio) && parsedRunawayRatio > 0 + ? parsedRunawayRatio + : 6; + function getClient(): Anthropic { if (!client) { // Default 5 retries (up from SDK default of 2) so transient @@ -151,10 +169,32 @@ export async function translateContent( // consolidate publish step. If a page ever legitimately needs more than // MAX_TOKENS, raise TRANSLATE_MAX_TOKENS or split the source. if (response.stop_reason === "max_tokens") { - throw new Error( + // TWO different failures wear this stop_reason, and only one of them is + // "the page is too big". A 16 KB source whose Vietnamese translation + // emitted 64 000 output tokens — observed on the box, 2026-08-18, + // reference/cloud-cli.mdx [vi] — is a DEGENERATE SAMPLE: the model fell + // into a repetition loop. Resampling fixes that; splitting the source does + // not, and treating it as a size problem sent one page's hiccup on to fail + // the whole nightly run. + // + // The tell is the ratio. Source bytes / 4 is a rough token count, and a + // faithful translation lands within a small multiple of it — even for the + // verbose target languages. Far beyond that is the model talking to + // itself, so the error is marked RETRYABLE and translateValidated spends + // an attempt on it. A source that genuinely approaches the ceiling keeps + // the old fail-loud behaviour, because retrying it would burn the budget + // three times to reach the same place. + const sourceTokensApprox = Math.max(1, Math.ceil(content.length / 4)); + const runaway = response.usage.output_tokens > sourceTokensApprox * RUNAWAY_RATIO; + const err = new Error( `translation truncated at max_tokens=${MAX_TOKENS} ` + - `(output ${response.usage.output_tokens} tokens) — source too large to translate in one request`, + `(output ${response.usage.output_tokens} tokens, source ~${sourceTokensApprox}) — ` + + (runaway + ? "output dwarfs the source, which is a runaway sample rather than an oversized page" + : "source too large to translate in one request"), ); + (err as Error & { retryable?: boolean }).retryable = runaway; + throw err; } const translated = @@ -211,15 +251,30 @@ export async function translateValidated(opts: { ? { attempt, maxAttempts: MAX_ATTEMPTS, error: lastError } : undefined; - // No try/catch: a transport/auth/max_tokens throw is not a validity - // failure — let it propagate so it is never silently retried as one. - const result = await translateContent( - opts.source, - opts.lang, - opts.langName, - opts.model, - feedback, - ); + // Transport/auth throws still propagate untouched — they are not validity + // failures and must never be silently retried as one. The ONE exception is + // a truncation flagged `retryable` (a runaway sample, see translateContent): + // that is a bad draw, and a bad draw is exactly what this loop is for. + let result: Awaited>; + try { + result = await translateContent( + opts.source, + opts.lang, + opts.langName, + opts.model, + feedback, + ); + } catch (e) { + const err = e as Error & { retryable?: boolean }; + if (!err.retryable || attempt === MAX_ATTEMPTS) throw err; + lastError = + "The previous attempt ran past the output limit by repeating itself. " + + "Translate the document once, completely, and stop."; + console.warn( + ` ${opts.label} -> attempt ${attempt}/${MAX_ATTEMPTS} overran the output limit; retrying`, + ); + continue; + } inputTokens += result.inputTokens; outputTokens += result.outputTokens;