Repository navigation
Add adaptive translation reliability and diagnostics - #3
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 78bf233efc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - Analyzes each uploaded book and adaptively chooses batch size, concurrency, timeout, and retries; advanced overrides remain available in Settings. | ||
| - Splits timeout/context-limited batches and simple oversized paragraphs into smaller translation requests, then resumes from valid output. |
There was a problem hiding this comment.
This adds and documents user-visible adaptive processing and fallback behavior, but CHANGELOG.md is not included in the commit. Add a release-note entry so users upgrading Babel can discover the changed Web/API processing semantics.
AGENTS.md reference: AGENTS.md:L34-L34
Useful? React with 👍 / 👎.
| split_worthy = ( | ||
| is_provider_safety_rejection(exc) | ||
| or is_retryable_translation_output_error(exc) |
There was a problem hiding this comment.
When a provider returns parseable rows that are nevertheless incomplete or structurally invalid (for example, a context-truncated multi-row response), the error is raised later by validate_translation_rows in _translate_batch_with_retries. It therefore only retries the same full batch and never reaches this split_worthy path, so it ultimately fails instead of applying the advertised smaller-batch fallback. Validate provider results inside _translate_rows_with_safety_fallback, or route that later validation error back through the fallback.
Useful? React with 👍 / 👎.
| adaptive_enabled=bool(data.get("adaptive_enabled", True)), | ||
| adaptive_plan=dict(data.get("adaptive_plan") or {}), |
There was a problem hiding this comment.
Jobs persisted before this change have neither adaptive_enabled nor adaptive_plan; this default turns both into an enabled adaptive job with an empty plan. On the next resume, resolve_execution_plan replaces the settings supplied by the user with the generic 3-worker/480-second/2-retry defaults, so previously prepared jobs can unexpectedly ignore deliberately conservative concurrency, timeout, and retry settings. Treat missing adaptive fields as legacy custom mode, or derive and persist a plan during migration.
Useful? React with 👍 / 👎.
| translated_text = " ".join( | ||
| element_text(parse_snippet(str(part.get("translated_html", "")))) | ||
| for part in translated_parts | ||
| ).strip() |
There was a problem hiding this comment.
For an oversized leaf element whose formatting depends on whitespace, such as xml:space="preserve" or a white-space: pre/pre-wrap CSS rule, element_text normalizes every whitespace run and the join inserts ordinary spaces. The merged translated_html therefore loses newlines, tabs, and repeated spaces even when every split response is valid, changing the rendered EPUB layout. Retain the translated text nodes' whitespace when reassembling the original row.
AGENTS.md reference: AGENTS.md:L16-L16
Useful? React with 👍 / 👎.
Address unresolved review feedback from #3 and add regression coverage.
Summary
Validation
PYTHONPATH=src:tests python -m unittest discover -s tests(82 passed)npm test(8 passed)npm run buildpython -m compileall -q src testsgit diff --check