[DR] Codebase Restructuring: Modularity, Documentation, and CI

[Maxwell-Rosen]

Problem

The codebase has grown organically and has several structural issues that make it hard to maintain, test, and document:

• CLI commands mix Click wiring with business logic, making library functions untestable in isolation
• 31k lines of code with only ~400 lines of tests (~1% coverage); no coverage gate in CI
• No in-repo documentation or examples; public API is undiscovered
• CI only runs pytest — no lint, no type checking, no docs build

Proposed Changes

1. Thin out CLI commands

Commands in commands/ should be ~15-line Click shells that delegate to library functions. Move business logic out of Click handlers so it's callable and testable without the CLI.

This refactor will make contributing new functions dramatically more straightforward than the current organization. It clearly marks a separation between how the CLI is used versus how script mode utilizes the library.

An example is the select function.

# commands/select.py today — ~100 lines of indexing logic inside the click function
@click.command()
@click.pass_context
def select(ctx, z0, z1, ...):
    for dataset in ctx.obj["data"]:
        arr = dataset.get_values()
        # 60 lines of slicing logic here
        dataset.set_values(arr[...])

Proposed pattern — Click handler is 10–15 lines, delegates to library:

# commands/select.py after refactor
@click.command()
@click.pass_context
def select(ctx, z0, z1, ...):
    for dataset in ctx.obj["data"]:
        pg.data.select(dataset, z0=z0, z1=z1, ...)   # library call

2. Extract interp/ from data/

Move data/dg.py → interp/modal.py + interp/nodal.py. Replace computeInterpolationMatrices.py and computeDerivativeMatrices.py with pre-computed .npy files loaded at runtime. Keep a interp/generate_matrices.py script for regeneration. Preserves the public API via re-exports.

This refactor clearly separates the logic into a separate interpolation directory, as this is a specific operation for interpreting some data.

3. Documentation and examples live in the source files

I recently interacted with the PyVista codebase and I was amazed with their CI. They have the documentation built directly from the codebase using the docstrings, with testable examples. Using something like that will dramatically improve the readability and usability of the codebase.

Every public function gets a NumPy-style docstring with a working Examples section. Examples use >>> doctest syntax so they are executable and validated in CI via pytest --doctest-modules — no separate example files, no drift between docs and code.

Function-level docstring (short, runnable): Example generated by LLMs. This is not intended to be an accurate docstring for anything, but an example of what each function should have.

def euler(values: np.ndarray) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
    """
    Compute primitive variables from Euler conserved moments.

    Parameters
    ----------
    values : ndarray
        Array of shape (..., 5) containing [rho, rho*ux, rho*uy, rho*uz, E].

    Returns
    -------
    rho : ndarray
        Mass density.
    u : ndarray
        Velocity vector, shape (..., 3).
    p : ndarray
        Scalar pressure.

    Examples
    --------
    >>> import numpy as np
    >>> import postgkyl as pg
    >>> vals = np.array([[1.0, 0.1, 0.0, 0.0, 1.5]])
    >>> rho, u, p = pg.tools.prim_vars.euler(vals)
    >>> float(rho[0])
    1.0
    """

Packages required:

I'm really not sure what packages to use. The goal is to have the documentation website built directly from the repository so that we don't have to update both independently. The examples should be treated as unit tests and CI should make sure all examples are working.

Package	Role
pytest --doctest-modules	Runs all >>> examples in docstrings as tests
numpydoc	Sphinx extension that renders NumPy docstrings into formatted API pages
sphinx.ext.autosummary	Auto-generates API index from docstrings
sphinx.ext.doctest	Tests .. doctest:: blocks in .rst narrative pages

The Sphinx conf.py enables doctest checking during the docs build:

extensions = ["numpydoc", "sphinx.ext.autosummary", "sphinx.ext.doctest"]
doctest_global_setup = "import postgkyl as pg; import numpy as np"

CI runs two complementary checks:

- run: pytest --doctest-modules src/postgkyl   # fast, runs with unit tests
- run: sphinx-build -W -b doctest docs _build  # catches .rst narrative examples

This may require a seperate documentation website for the postgkyl documentation. We can put it under the general readthedocs, but it will have its own subdomain, built directly from the python.