EpiLang

EpiLang is a reproducible research codebase for studying whether language models can acquire and execute the currently valid semantics of synthetic programming languages. The completed E0b-W pilot isolates a causal data-order failure mode: homogeneous epoch-ending residue blocks determine a collapsed output class, while interleaving removes the collapse and enables train-set memorization without held-out modulo-11 generalization.

Main result

Across three preregistered seeds:

• reversing the homogeneous blocks moved the collapsed output from the original endpoint class to residue 0;
• 23/24 reverse-block epochs generated residue 0 on every train and tune example;
• interleaving increased final train exact execution by a paired mean of +106.7/176;
• final held-out tune performance changed by -3.3/176 on average;
• no run exceeded 27/176 tune accuracy against the preregistered 159/176 threshold.

The supported conclusion is deliberately narrow: training order causally controls the collapsed output class under the tested adaptation contract, and removing that collapse does not establish systematic arithmetic transfer.

Repository layout

• src/epilang/: generators, syntax, semantics, interpreters, evaluators, training utilities, and CLIs.
• tests/: unit, property, parity, regression, model, and training tests.
• configs/: immutable experiment, data, model, runtime, and analysis configurations.
• scripts/analysis/: deterministic table export utilities.
• scripts/truba/: historical Slurm recipes used for the reported TRUBA runs.
• reports/tables/: canonical seed-level and paired-result tables.
• reports/artifacts/: validated aggregate analysis artifacts.
• reports/audits/: final scientific audit for the completed pilot.
• reports/paper/: manuscript source draft.

Installation

The core package uses only the Python standard library. Create an isolated environment and install it in editable mode:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e .

The Qwen/H200 experiments require the pinned external model/runtime assets described by the immutable configs and audit records; model weights, Apptainer images, wheelhouses, checkpoints, and run directories are intentionally not included.

Run the test suite

The complete test suite additionally requires PyTorch:

python -m pip install -e ".[test]"
PYTHONPATH=src python scripts/run_public_tests.py

This public-CI profile excludes four historical lineage tests whose immutable datasets live outside Git in TRUBA scratch. With those assets mounted, run the full profile with:

PYTHONPATH=src python -m unittest discover -s tests -v

The complete package passed 265/265 tests with the external assets mounted at closure. Some provenance tests inspect immutable historical files, so do not rewrite their hashes casually.

Re-export the publication tables

python scripts/analysis/export_e0bw_order_confirmation_tables.py \
  --analysis reports/artifacts/e0bw_order_confirmation_analysis.json \
  --output-dir /tmp/epilang-tables

The exported CSV files should match the canonical files under reports/tables/.

Reproducibility boundary

Historical configs and Slurm scripts contain absolute TRUBA paths and scheduler account names because these values are part of the experiment lineage. They are not credentials. Replace them with paths and account settings for your own environment rather than running them unchanged.

The repository does not include:

• model weights or tokenizer snapshots;
• GPU container images and runtime wheelhouses;
• optimizer checkpoints or LoRA adapters;
• raw scratch run directories;
• secrets, API keys, passwords, or private keys.

See PROJECT_STATUS.md for the precise scientific boundary and reports/audits/e0bw-order-confirmation-final-20260717.md for the final audit.

Citation

Use CITATION.cff. After the Zenodo record is published, add the repository DOI to CITATION.cff, .zenodo.json, and this README.

License

Code is released under the Apache License 2.0. Third-party models, datasets, and runtime components retain their own licenses and are not redistributed here.

Acknowledgment

The numerical calculations reported in the associated paper were fully performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).