Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 31 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Open Siddur Project (AI aided version)

[![Tests](https://github.com/efeins/opensiddur-ai/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/efeins/opensiddur-ai/actions/workflows/tests.yml)
[![Tests](https://github.com/opensiddur/opensiddur-ai/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/opensiddur/opensiddur-ai/actions/workflows/tests.yml)
[![codecov](https://codecov.io/github/opensiddur/opensiddur-ai/branch/main/graph/badge.svg?token=S4DAU7F6VY)](https://codecov.io/github/opensiddur/opensiddur-ai)

This is a work in progress to convert the Open Siddur Project to use AI to aid in the conversion of the liturgical texts.
Expand All @@ -25,21 +25,42 @@ The output will be in the `schema` directory as RelaxNG XML (and, eventually, IS
## Sources

Available sources in their original (or close to original) form are in the `sources` directory.
The canonical source texts repository is at [opensiddur/sourcetexts](https://github.com/opensiddur/sourcetexts).
Clone or symlink it as the `sources` root before running any importers.

Input converters for each specific source are in the `importer` directory.

Example: run the WLC importer against an external clone:

```bash
uv run python -m opensiddur.importer.wlc.wlc \
--sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \
--project-dir ~/src/opensiddur-repos/opensiddur-projects/project/wlc
```

Example: run the JPS 1917 importer against an external clone:

```bash
uv run python -m opensiddur.importer.jps1917.convert_wikisource \
--sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \
--project-dir ~/src/opensiddur-repos/opensiddur-projects/project/jps1917
```

## JLPTEI sources

JLPTEI sources are compiled into the `project` directory.
The canonical projects repository is at [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects).
Clone or symlink it as the `project` root before running the compiler or exporter.

## Reference database

The exporter resolves `urn:x-opensiddur:` URIs to project files via a SQLite
database at `database/reference.db`. Whenever you add, remove, or rename files
in the `project/` directory, re-sync the database so the compiler can find them:
in the `project/` directory (e.g. a clone of [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects)),
re-sync the database so the compiler can find them, as in the example here:

```bash
uv run python -m opensiddur.exporter.refdb
uv run python -m opensiddur.exporter.refdb --project-directory ~/src/opensiddur-projects/project
```

The command scans every `project/<name>/` subdirectory, updates URN and
Expand All @@ -51,14 +72,16 @@ You must re-sync before running the compiler on any newly-added project.

## Compilation (JLPTEI → compiled linear XML)

The compiler takes a `project/<name>/` file, resolves transclusions, annotations, and parallel texts,
The compiler takes a `project/<name>/` file (from [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects)),
resolves transclusions, annotations, and parallel texts,
and outputs a single “compiled” XML file that can be
converted into a final printable format (eg, PDF).

Example (compile `project/wlc/ruth.xml` to `compiled.xml`):

```bash
uv run python -m opensiddur.exporter.compiler \
--project-directory ~/src/opensiddur-repos/opensiddur-projects/project \
--project wlc \
--file_name ruth.xml \
--output_file compiled.xml
Expand All @@ -68,6 +91,7 @@ Example with a settings YAML (controls project priorities, annotations, and opti

```bash
uv run python -m opensiddur.exporter.compiler \
--project-directory ~/src/opensiddur-repos/opensiddur-projects/project \
--project wlc \
--file_name ruth.xml \
--settings doc/exporter-settings.example.yaml \
Expand All @@ -80,6 +104,7 @@ Convert the compiled XML file to LuaLaTeX using the `reledmac`/`reledpar` pipeli

```bash
uv run python -m opensiddur.exporter.tex.latex \
--project-directory ~/src/opensiddur-repos/opensiddur-projects/project \
compiled.xml \
--settings doc/exporter-settings.example.yaml \
--output compiled.tex
Expand All @@ -91,6 +116,7 @@ Export directly to PDF (generates TeX internally, then runs LuaLaTeX/latexmk):

```bash
uv run python -m opensiddur.exporter.pdf.pdf \
--project-directory ~/src/opensiddur-repos/opensiddur-projects/project \
--settings doc/exporter-settings.example.yaml \
compiled.xml \
output.pdf
Expand All @@ -100,6 +126,7 @@ Keep the intermediate TeX (helpful for debugging LaTeX issues):

```bash
uv run python -m opensiddur.exporter.pdf.pdf \
--project-directory ~/src/opensiddur-repos/opensiddur-projects/project \
--settings doc/exporter-settings.example.yaml \
--keep-tex \
compiled.xml \
Expand Down
10 changes: 10 additions & 0 deletions opensiddur/importer/jps1917/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,16 @@ The conversion process follows a three-stage pipeline:
uv run python -m opensiddur.importer.jps1917.convert_wikisource
```

If you are using external clones of [opensiddur/sourcetexts](https://github.com/opensiddur/sourcetexts)
and [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects), pass the
paths explicitly:

```bash
uv run python -m opensiddur.importer.jps1917.convert_wikisource \
--sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \
--project-dir ~/src/opensiddur-repos/opensiddur-projects/project/jps1917
```

### Complete Workflow (Programmatic)

```python
Expand Down
10 changes: 10 additions & 0 deletions opensiddur/importer/wlc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,13 @@ uv run python -m opensiddur.importer.wlc.wlc
```

The output will be in the `project/wlc` directory.

If you are using external clones of [opensiddur/sourcetexts](https://github.com/opensiddur/sourcetexts)
and [opensiddur/opensiddur-projects](https://github.com/opensiddur/opensiddur-projects), pass the
paths explicitly:

```bash
uv run python -m opensiddur.importer.wlc.wlc \
--sourcetexts-root ~/src/opensiddur-repos/sourcetexts/sources \
--project-dir ~/src/opensiddur-repos/opensiddur-projects/project/wlc
```