Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed
- **`forward` and `init` find the built-in glossary regardless of working directory, and say which one they loaded** (#149): both commands resolved glossary candidates only against `process.cwd()`. The built-in glossaries ship inside *this* package and **no edition repository carries one**, so a resync launched from the target repo — or a bench root, or anywhere a globally installed CLI is naturally invoked — translated with no glossary at all. Nothing was logged either way and parse errors were swallowed by a bare `catch {}`, so a run that dropped terminology enforcement was indistinguishable from one that applied it; that unobservability was the worst of it, because it made the difference unauditable after the fact. Production signature matching the defect exactly: in `lecture-python.zh-cn`, the `init`-seeded lectures (QuantEcon/lecture-python.zh-cn#196) all use the glossary's 边缘分布 for *Marginal distribution*, while the 2026-07-19 `forward` wave took `prob_matrix.md` from 12 wrong / 28 correct to **25 wrong / 35 correct** — newly generated text ignoring a glossary entry that exists. Resolution now lives in one testable module (`src/cli/glossary-loader.ts`) shared by both commands, with the packaged directory resolved relative to the installed CLI and threaded in as an option (`import.meta.url` cannot be loaded by the Jest CJS registry, so the entry point resolves it and the logic stays unit-testable). Precedence is `--glossary` → repo-local `glossary/<lang>.json` → built-in, and every outcome is reported: `✓ Loaded built-in glossary for zh-cn — 357 terms (…)` on success, a warning naming every path tried when a language has no glossary anywhere. **Loud on failure**: an explicit `--glossary` that is missing or malformed is a hard error rather than a silent fallback to different terminology, and any candidate that exists but does not parse is an error rather than a fall-through. Bulk resolves once before the first file, so a bad glossary stops the wave instead of surfacing after N files have been resynced against nothing.

### Added
- **`forward --glossary <path>`** (#149), matching `init` — previously the resync path had no way to override the glossary at all.

## [0.22.0] - 2026-07-22

### Added
Expand Down
5 changes: 3 additions & 2 deletions docs/user/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,7 @@ npx translate forward -s <source-path> -t <target-path> [options]
| `-m, --model <model>` | `claude-sonnet-5` | Claude model |
| `--test` | `false` | Use deterministic mock responses (no LLM) |
| `--github <owner/repo>` | *(none)* | Create one PR per file in the target repo |
| `--glossary <path>` | *(auto)* | Path to glossary JSON file (default: built-in glossary for the language) |
| `--exclude <pattern>` | *(none)* | Exclude files matching pattern |

**Single-file example:**
Expand Down Expand Up @@ -304,7 +305,7 @@ npx translate init -s <source-path> -t <target-path> --target-language <code> [o
| `--resume-from <file>` | *(none)* | Resume from a specific lecture file |
| `--skip-existing` | `false` | Skip lectures already translated (reads `.translate/state/`) |
| `-j, --parallel <n>` | `1` | Number of parallel translations |
| `--glossary <path>` | *(auto)* | Path to glossary JSON file (default: `glossary/<lang>.json`) |
| `--glossary <path>` | *(auto)* | Path to glossary JSON file (default: built-in glossary for the language) |
| `--localize <rules>` | `code-comments,figure-labels,i18n-font-config` | Localization rules for code cells (use `none` to disable) |
| `--dry-run` | `false` | Preview lectures without translating |

Expand Down Expand Up @@ -343,7 +344,7 @@ Download: [Source Han Serif SC](https://github.com/adobe-fonts/source-han-serif/

**7-phase pipeline:**

1. **Load glossary** — looks for `glossary/<lang>.json` in the current working directory
1. **Load glossary** — `--glossary` if given, else a repo-local `glossary/<lang>.json`, else the built-in glossary shipped with the package (see [Glossary](glossary.md#using-a-custom-glossary))
2. **Parse `_toc.yml`** — discovers lectures from the source repo's table of contents
3. **Setup target folder** — creates the target directory structure
4. **Copy non-markdown files** — images, config, data files, CSS (preserves directory structure)
Expand Down
10 changes: 9 additions & 1 deletion docs/user/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,15 @@ To use your own glossary instead of (or in addition to) the built-in one, specif
# ... other inputs
```

For the CLI, glossaries are loaded automatically based on the language code. The CLI looks for `glossary/{language}.json` in the action-translation repository.
For the CLI, glossaries are loaded automatically based on the language code. The `forward` and `init` commands resolve candidates in this order, and report which one they used:

| Order | Candidate | Notes |
|-------|-----------|-------|
| 1 | `--glossary <path>` | When given, this is the **only** candidate — a missing or malformed file is a hard error, never a silent fallback |
| 2 | `<cwd>/glossary/{language}.json`, `<cwd>/glossary-{language}.json` | Lets a project override the estate defaults by carrying its own glossary |
| 3 | `{action-translation}/glossary/{language}.json` | The built-in glossary, resolved relative to the installed package — **not** to the working directory |

Every run prints the glossary it loaded (`✓ Loaded built-in glossary for zh-cn — 357 terms (…)`) or warns that it found none and lists every path it tried. A malformed glossary is an error rather than a silent skip.

## Adding terms to a glossary

Expand Down
1 change: 1 addition & 0 deletions src/cli/__tests__/cli-smoke.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ chapters:
expect(stdout).toContain('--target');
expect(stdout).toContain('--github');
expect(stdout).toContain('--parallel');
expect(stdout).toContain('--glossary');
});
});

Expand Down
176 changes: 176 additions & 0 deletions src/cli/__tests__/forward-glossary.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
/**
* Glossary delivery on the resync path (#149).
*
* `forward` resolved its glossary only against `process.cwd()`. No edition
* repository carries `glossary/<lang>.json`, so a resync run from the target
* repo — the natural place to run it — translated with no glossary at all and
* logged nothing either way. These tests pin the end of the chain: what the
* translator actually receives, from a working directory that has no glossary.
*
* Separate file from forward.test.ts because it mocks triage and the translator,
* which the main suite exercises for real (in test mode).
*/

import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { resyncSingleFile } from '../commands/forward.js';
import { ForwardOptions } from '../types.js';
import { Glossary } from '../../types.js';

jest.mock('../forward-triage.js', () => ({
triageForward: jest.fn().mockResolvedValue({
verdict: 'CONTENT_CHANGES',
reason: 'mocked',
}),
}));

const mockResync = jest.fn();
jest.mock('../../translator.js', () => ({
TranslationService: jest.fn().mockImplementation(() => ({
translateDocumentResync: mockResync,
})),
}));

const BUILT_IN_DIR = path.join(__dirname, '..', '..', '..', 'glossary');

function createTestLogger() {
const messages: Array<{ level: 'info' | 'warn' | 'error'; text: string }> = [];
return {
messages,
info: (text: string) => messages.push({ level: 'info' as const, text }),
warn: (text: string) => messages.push({ level: 'warn' as const, text }),
error: (text: string) => messages.push({ level: 'error' as const, text }),
};
}

function makeOptions(overrides: Partial<ForwardOptions> = {}): ForwardOptions {
return {
source: '/tmp/source',
target: '/tmp/target',
docsFolder: 'lectures',
language: 'zh-cn',
sourceLanguage: 'en',
model: 'claude-sonnet-5',
test: false,
apiKey: 'test-key',
...overrides,
};
}

/** The glossary handed to the translator on the single (mocked) resync call. */
function glossaryPassedToTranslator(): Glossary | undefined {
expect(mockResync).toHaveBeenCalledTimes(1);
return mockResync.mock.calls[0][0].glossary;
}

describe('forward glossary resolution', () => {
let tmpDir: string;
let cwdSpy: jest.SpyInstance;

beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'forward-glossary-'));
fs.mkdirSync(path.join(tmpDir, 'source', 'lectures'), { recursive: true });
fs.mkdirSync(path.join(tmpDir, 'target', 'lectures'), { recursive: true });
fs.writeFileSync(
path.join(tmpDir, 'source', 'lectures', 'test.md'),
'# Title\n\nMarginal distribution.\n',
'utf-8'
);
fs.writeFileSync(
path.join(tmpDir, 'target', 'lectures', 'test.md'),
'# 标题\n\n旧内容。\n',
'utf-8'
);

// Run "from the target repo" — the working directory the wave is launched
// from in practice, and one that carries no glossary of its own.
cwdSpy = jest.spyOn(process, 'cwd').mockReturnValue(path.join(tmpDir, 'target'));

mockResync.mockReset();
mockResync.mockResolvedValue({
success: true,
translatedSection: '# 标题\n\n新内容。\n',
tokensUsed: 10,
});
});

afterEach(() => {
cwdSpy.mockRestore();
fs.rmSync(tmpDir, { recursive: true, force: true });
});

async function resync(options: ForwardOptions, logger = createTestLogger()) {
const result = await resyncSingleFile(
'test.md',
path.join(tmpDir, 'source'),
path.join(tmpDir, 'target'),
'lectures',
options,
logger
);
return { result, logger };
}

it('sends the built-in glossary even when the working directory has none', async () => {
const { result } = await resync(makeOptions({ builtInGlossaryDir: BUILT_IN_DIR }));

expect(result.summary.errors).toBe(0);
const glossary = glossaryPassedToTranslator();
expect(glossary).toBeDefined();
expect(glossary!.terms.length).toBeGreaterThan(0);
// The term whose absence surfaced this defect on QuantEcon/lecture-python.zh-cn#198.
expect(glossary!.terms.some((t) => t.en === 'Marginal distribution')).toBe(true);
});

it('reports the glossary it loaded', async () => {
const { logger } = await resync(makeOptions({ builtInGlossaryDir: BUILT_IN_DIR }));

const loaded = logger.messages.filter((m) => m.text.includes('glossary for zh-cn'));
expect(loaded).toHaveLength(1);
expect(loaded[0].level).toBe('info');
});

it('honours an explicit --glossary path', async () => {
const custom = path.join(tmpDir, 'custom.json');
fs.writeFileSync(
custom,
JSON.stringify({ version: '1.0', terms: [{ en: 'only', 'zh-cn': '唯一' }] }),
'utf-8'
);

await resync(makeOptions({ builtInGlossaryDir: BUILT_IN_DIR, glossaryPath: custom }));

expect(glossaryPassedToTranslator()!.terms).toHaveLength(1);
});

it('fails loudly on a --glossary path that does not exist', async () => {
await expect(
resync(makeOptions({ builtInGlossaryDir: BUILT_IN_DIR, glossaryPath: '/no/such.json' }))
).rejects.toThrow(/Glossary not found/);

expect(mockResync).not.toHaveBeenCalled();
});

it('uses a pre-resolved glossary without re-resolving it', async () => {
// How bulk threads one load through every file in the wave.
const preResolved: Glossary = { version: '1.0', terms: [{ en: 'threaded', 'zh-cn': '穿线' }] };

const { logger } = await resync(
makeOptions({ builtInGlossaryDir: BUILT_IN_DIR, glossary: preResolved })
);

expect(glossaryPassedToTranslator()).toBe(preResolved);
expect(logger.messages.filter((m) => m.text.includes('glossary for zh-cn'))).toHaveLength(0);
});

it('says so when it ends up with no glossary', async () => {
// No built-in directory threaded through and none in the working directory:
// the pre-fix situation, which must no longer be silent.
const { logger } = await resync(makeOptions());

expect(glossaryPassedToTranslator()).toBeUndefined();
const warnings = logger.messages.filter((m) => m.level === 'warn');
expect(warnings.some((w) => w.text.includes('WITHOUT terminology enforcement'))).toBe(true);
});
});
Loading
Loading