LAIT
_{^{Literary AI Translation}}

LAIT is a public research release for a reader-centered evaluation study of literary machine translation. The repository centers the three MT pipelines discussed in the study and the aggregate analyses used to compare machine translations with human translations.

This public GitHub release does not include source texts or human translations. Those materials are available for research access through a gated Hugging Face dataset, <HUGGING_FACE_DATASET_URL>, that requires users to agree to access terms. After downloading the gated files, run python3 scripts/restore_controlled_access_data.py --apply from the default hf_dataset/ layout, or pass --dir to point to a different download folder. The script restores the source, human-translation, and redacted segment-level text fields locally.

Start Here

• START_HERE.md: fastest path through the repository.
• docs/NAVIGATION.md: routes for readers with the preprint open, the abstract skimmed, or no prior context.
• docs/REPO_MAP.md: what each top-level directory is for.
• docs/DATA_ACCESS.md: what is public, what is withheld, and how to request controlled access.
• docs/REPRODUCIBILITY.md: setup and rerun notes.
• docs/paper_supplement/: human-evaluation interface screenshots and participant-facing guidelines referenced by the preprint.
• docs/release/WITHHELD_FILES.md: exact manifest of files withheld from this public branch.
• docs/release/PREPRINT_SCOPE_AUDIT.md: summary of the paper-scope review decisions.

Main Public Outputs

If you are trying to match the repository to the preprint, use docs/NAVIGATION.md first. It maps the dataset, MT pipeline, automatic-metric, and human-evaluation materials to public paths.

The primary public MT outputs are:

Pipeline	Path	Description
P1	books/MT/pipeline1/	First MT pipeline outputs, grouped by model.
P2	books/MT/pipeline2/	Second MT pipeline outputs, grouped by model.
P3	books/MT/pipeline3/	Agentic MT pipeline outputs, including the appendix multilingual target-language examples under extern/.

Secondary baseline outputs are kept where they do not expose source or human translation text, but the release navigation prioritizes P1, P2, and P3.

Analysis Outputs

• analysis/manuscript_tables/: derived LaTeX/CSV tables retained for the public release. Text-bearing source/HT fields are redacted where needed.
• human_eval/: public human-evaluation figures, aggregate model outputs, and count summaries. Row-level comments and annotation exports are withheld.
• results_all_metrics/, results_chunk_review_eval/, and results_mapped_metrics/: derived metric outputs with source/HT text fields redacted where needed.

Code Entry Points

Task	Entry point
Run the direct MT pipeline	python mt_pipeline.py --help
Run the agentic MT pipeline	python agents_pipeline/runner.py --help
Score MT outputs with automatic metrics	python mt_eval.py --help
Run the older evaluation wrapper	python eval_pipeline.py --help
Inspect analysis scripts	analysis/scripts/
Inspect utility scripts	scripts/

Some commands require controlled-access source or HT files. The public branch keeps the code and aggregate outputs, but not those restricted inputs.

Setup

Python 3.10+ is recommended.

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

On Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Copy .env.example to .env and fill only the provider credentials needed for the command you plan to run. Never commit .env.

Controlled-Access Restoration

Some public-release files contain the marker [withheld from public GitHub release] where source text, human translation text, or text-bearing segment fields were removed. Researchers with approved access can download the gated Hugging Face dataset contents from <HUGGING_FACE_DATASET_URL> into the default hf_dataset/ directory:

hf_dataset/
  lait_books_data.jsonl
  lait_restoration_data.jsonl

Then restore the controlled-access files into the checkout:

python3 scripts/restore_controlled_access_data.py --apply

If the Hugging Face files are somewhere else, point the script to that directory:

python3 scripts/restore_controlled_access_data.py \
  --dir /path/to/downloaded/lait-controlled-access \
  --apply

Run the same command without --apply for a dry run. The script restores books/dev/, books/eval/, books/HT/, and exact unredacted versions of the sanitized metric/alignment files covered by the gated dataset.

Public-Release Data Policy

Included on GitHub:

• P1/P2/P3 MT outputs under books/MT/.
• Aggregate metrics, tables, and derived analysis outputs.
• Code, prompts, configs, tests, and documentation.

Withheld from GitHub:

• Source-language texts.
• Human translations.
• Raw source/HT chunk-review inputs.
• Run workspaces that contain source chunks.
• Raw human-evaluation exports and internal study dumps.
• Row-level participant comments, annotation exports, and disagreement viewers.

See docs/release/withheld-files.tsv and docs/release/sanitized-files.tsv for the exact release audit manifests.

License

Repository code and documentation are released under the MIT License unless a file or subdirectory states a different license. Third-party components retain their own licenses, including par3/, metricx/, and LiTransProQA/.