diff --git a/README.md b/README.md index 2e70f10..b1b8132 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Open Siddur Project (AI aided version) -[![Tests](https://github.com/efeins/opensiddur-ai/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/efeins/opensiddur-ai/actions/workflows/tests.yml) +[![Tests](https://github.com/opensiddur/opensiddur-ai/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/opensiddur/opensiddur-ai/actions/workflows/tests.yml) [![codecov](https://codecov.io/github/opensiddur/opensiddur-ai/branch/main/graph/badge.svg?token=S4DAU7F6VY)](https://codecov.io/github/opensiddur/opensiddur-ai) This is a work in progress to convert the Open Siddur Project to use AI to aid in the conversion of the liturgical texts. @@ -25,21 +25,42 @@ The output will be in the `schema` directory as RelaxNG XML (and, eventually, IS ## Sources Available sources in their original (or close to original) form are in the `sources` directory. +The canonical source texts repository is at [opensiddur/sourcetexts](https://github.com/opensiddur/sourcetexts). +Clone or symlink it as the `sources` root before running any importers. Input converters for each specific source are in the `importer` directory. +Example: run the WLC importer against an external clone: + +```bash +uv run python -m opensiddur.importer.wlc.wlc \ + --sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \ + --project-dir ~/src/opensiddur-repos/opensiddur-projects/project/wlc +``` + +Example: run the JPS 1917 importer against an external clone: + +```bash +uv run python -m opensiddur.importer.jps1917.convert_wikisource \ + --sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \ + --project-dir ~/src/opensiddur-repos/opensiddur-projects/project/jps1917 +``` + ## JLPTEI sources JLPTEI sources are compiled into the `project` directory. +The canonical projects repository is at [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects). +Clone or symlink it as the `project` root before running the compiler or exporter. ## Reference database The exporter resolves `urn:x-opensiddur:` URIs to project files via a SQLite database at `database/reference.db`. Whenever you add, remove, or rename files -in the `project/` directory, re-sync the database so the compiler can find them: +in the `project/` directory (e.g. a clone of [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects)), +re-sync the database so the compiler can find them, as in the example here: ```bash -uv run python -m opensiddur.exporter.refdb +uv run python -m opensiddur.exporter.refdb --project-directory ~/src/opensiddur-projects/project ``` The command scans every `project//` subdirectory, updates URN and @@ -51,7 +72,8 @@ You must re-sync before running the compiler on any newly-added project. ## Compilation (JLPTEI → compiled linear XML) -The compiler takes a `project//` file, resolves transclusions, annotations, and parallel texts, +The compiler takes a `project//` file (from [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects)), +resolves transclusions, annotations, and parallel texts, and outputs a single “compiled” XML file that can be converted into a final printable format (eg, PDF). @@ -59,6 +81,7 @@ Example (compile `project/wlc/ruth.xml` to `compiled.xml`): ```bash uv run python -m opensiddur.exporter.compiler \ + --project-directory ~/src/opensiddur-repos/opensiddur-projects/project \ --project wlc \ --file_name ruth.xml \ --output_file compiled.xml @@ -68,6 +91,7 @@ Example with a settings YAML (controls project priorities, annotations, and opti ```bash uv run python -m opensiddur.exporter.compiler \ + --project-directory ~/src/opensiddur-repos/opensiddur-projects/project \ --project wlc \ --file_name ruth.xml \ --settings doc/exporter-settings.example.yaml \ @@ -80,6 +104,7 @@ Convert the compiled XML file to LuaLaTeX using the `reledmac`/`reledpar` pipeli ```bash uv run python -m opensiddur.exporter.tex.latex \ + --project-directory ~/src/opensiddur-repos/opensiddur-projects/project \ compiled.xml \ --settings doc/exporter-settings.example.yaml \ --output compiled.tex @@ -91,6 +116,7 @@ Export directly to PDF (generates TeX internally, then runs LuaLaTeX/latexmk): ```bash uv run python -m opensiddur.exporter.pdf.pdf \ + --project-directory ~/src/opensiddur-repos/opensiddur-projects/project \ --settings doc/exporter-settings.example.yaml \ compiled.xml \ output.pdf @@ -100,6 +126,7 @@ Keep the intermediate TeX (helpful for debugging LaTeX issues): ```bash uv run python -m opensiddur.exporter.pdf.pdf \ + --project-directory ~/src/opensiddur-repos/opensiddur-projects/project \ --settings doc/exporter-settings.example.yaml \ --keep-tex \ compiled.xml \ diff --git a/opensiddur/importer/jps1917/README.md b/opensiddur/importer/jps1917/README.md index 8884fc9..c4382ab 100644 --- a/opensiddur/importer/jps1917/README.md +++ b/opensiddur/importer/jps1917/README.md @@ -74,6 +74,16 @@ The conversion process follows a three-stage pipeline: uv run python -m opensiddur.importer.jps1917.convert_wikisource ``` +If you are using external clones of [opensiddur/sourcetexts](https://github.com/opensiddur/sourcetexts) +and [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects), pass the +paths explicitly: + +```bash +uv run python -m opensiddur.importer.jps1917.convert_wikisource \ + --sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \ + --project-dir ~/src/opensiddur-repos/opensiddur-projects/project/jps1917 +``` + ### Complete Workflow (Programmatic) ```python diff --git a/opensiddur/importer/wlc/README.md b/opensiddur/importer/wlc/README.md index f442205..219e1ea 100644 --- a/opensiddur/importer/wlc/README.md +++ b/opensiddur/importer/wlc/README.md @@ -19,3 +19,13 @@ uv run python -m opensiddur.importer.wlc.wlc ``` The output will be in the `project/wlc` directory. + +If you are using external clones of [opensiddur/sourcetexts](https://github.com/opensiddur/sourcetexts) +and [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects), pass the +paths explicitly: + +```bash +uv run python -m opensiddur.importer.wlc.wlc \ + --sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \ + --project-dir ~/src/opensiddur-repos/opensiddur-projects/project/wlc +```