Skip to content

development.md

Latest commit

 

History

History
249 lines (187 loc) · 9.53 KB

development.md

File metadata and controls

249 lines (187 loc) · 9.53 KB
Raw

Development guide

ptfkit is specification-driven. YAML files describe source publications and function contracts; the code generator validates those files and writes the language targets, tests, and PTF catalog. Generated files are committed so that package builds and documentation builds do not depend on running codegen first.

Repository layout

  • specs/functions/ contains one YAML specification per source publication.
  • specs/schema/ defines the specification format.
  • codegen/ contains validation and generation code.
  • targets/ptfkit-native/ contains the C11 headers and C++23 modules.
  • targets/ptfkit-py/ contains the Python package and its native extension.
  • targets/ptfkit-rs/ contains the Rust crate.
  • docs/mkdocs.yml configures the documentation site.
  • docs/src/ contains published Markdown, navigation, and static assets including generated files.
  • .agents/skills/ contains the assisted extraction, generation, and review workflows.

For the specification format and its cross-target contracts, see the PTF source specification guide.

Toolchains and dependencies

Install Mise and activate it in your shell. mise.toml is the source of truth for the pinned Rust, uv, CMake, Ninja, Clang, Ruff, ty, and prek toolchains; mise.lock records their resolved downloads and checksums. The native target requires the C11/C++23 compiler provided by Clang.

From the repository root, trust the repository configuration and install the pinned toolchains:

mise trust
mise install

Install the Git hook shim once the prek toolchain is available:

prek install

Use cargo to manage dependencies for the codegen and Rust target, and use uv to manage the Python environment. Add, update, or remove dependencies through these tools instead of manually editing dependency lists in pyproject.toml or Cargo.toml.

Prepare the Python environment with:

mise run python:sync

List the available project tasks with mise tasks ls, inspect one with mise tasks info <task>, and run one with mise run <task>. For example:

mise tasks ls
mise run python:test

To update a toolchain, change its exact version in mise.toml, regenerate the corresponding lock entry with mise lock <tool>, and install it locally with mise install <tool>. Commit both mise.toml and mise.lock; do not edit the lockfile by hand. Use mise lock without a tool name to refresh every existing lock entry.

Specifications and generated targets

The PTF source specification guide explains the scientific information represented by each YAML file. The JSON Schema and mise run validate define the complete structural and semantic contract. Verification-case provenance and the shared cross-target comparison policy are documented in the scientific verification policy. Calculated expected values are fixed in YAML and are not regenerated from semantic IR.

Do not edit generated target sources, tests, the PTF catalog, or generated API reference pages directly. Regenerate every target and the PTF catalog with:

mise run generate

When changing the generator, specification schema, output formatting, or generation infrastructure, verify deterministic regeneration across every codegen-owned target family with:

mise run generate-check

The command regenerates the targets through the normal pipeline and reports added, removed, or modified generated files.

Generation conventions

The specification filename stem is the APA-style source slug. Codegen uses it for Rust, Python, and C++ modules, C headers, tests, and documentation. Generated function names follow calc_ptf_<first-author><year>[_<extra>] where applicable.

Each source has one public Python module, ptfkit.<apa_article_key>. Codegen creates it by default. Set generation.public_python: manual only when the public wrapper must be maintained manually; the wrapper must still delegate to the generated native ufuncs rather than duplicate formulas.

Codegen compiles each implementation into a shared semantic model and renders the Rust, C, C++, and native NumPy implementations independently. Documentation is generated from the same validated specification. Put target-independent summaries, parameter descriptions, return descriptions, notes, and warnings in the YAML rather than adding them to generated files.

Test infrastructure

Codegen generates PTF verification tests from specification cases. Shared test helpers and their comparator tests are maintained by hand in each target:

  • Rust: targets/ptfkit-rs/src/test_support.rs, included only under cfg(test).
  • Python: targets/ptfkit-py/tests/_helpers.py and tests/test_comparator.py.
  • C and C++: targets/ptfkit-native/tests/support/close_enough.h and tests/support/comparator.c (compiled as C11 and C++23).

Native comparator rejection cases use names such as reject relative and reject nan. tests/support/check_rejection.cmake requires exit code 1 and an assertion diagnostic naming the requested case; unrelated failures do not pass.

These files have no generated-file marker and are not codegen outputs. Edit them directly when changing test infrastructure, then run the target verification suites. Keep the shared comparison policy consistent across targets.

The handwritten native tests tests/support/umbrella.c and tests/support/umbrella.cpp check function and result-type access through <ptfkit/ptfkit.h> and import ptfkit, respectively. The C test is compiled as both C11 and C++23 and checks that all public headers can be included together. Generated verification tests use individual source headers or modules without repeating the suite through the umbrella.

Adding a PTF

The assisted workflow uses the skills in .agents/skills/:

  1. Give ptf-extract a readable local source paper. It writes a blocked or review-ready draft under specs/functions/ using only information supported by that source.
  2. Review the YAML and resolve every blocker, including missing metadata.
  3. Run ptf-generate <apa_article_key> to validate, generate, run the complete verification suite, and mark the reviewed source implemented.
  4. Optionally run ptf-review <apa_article_key> for an independent, read-only pre-merge review.

The source paper is transient input. The reviewed specification is the persisted record and the source of truth for generated implementations.

Corpus reporting

The code generator can summarize the current validated specification corpus in the terminal:

mise run corpus-report

Generate deterministic JSON for CI or publication-oriented tables with:

mise run corpus-report --format json

All counts are derived from specs/functions/ through the normal loader, validation, and compilation pipeline. The report intentionally includes schema-valid blocked functions. Its verification coverage describes declared verification cases and edge cases in the specifications; it is neither predictive benchmarking nor external validation against soil datasets, and descriptive edge cases are not claimed to be executable tests.

Publication years are derived only from a four-digit suffix on the APA-style source slug because the current schema has no explicit year field. Unresolved slugs are reported rather than guessed. Prediction targets and hydraulic model descriptions are reported exactly as structured in the schema; the command does not infer scientific-property groups from free text. The JSON document uses stable sources, functions, verification, inputs, outputs, scope, and blocked_functions sections with explicit counts.

Checks

Use the smallest relevant checks while iterating. Before submitting changes that affect multiple targets, run the complete target verification suite:

mise run verify

For a narrower change, run the component verification suite:

mise run codegen:verify
mise run rust:verify
mise run native:verify
mise run python:verify

Each component verify runs all of its format, lint, type-checking, and test checks. mise run test runs only the target test suites together. Python tests build the local native extension before exercising the public API.

Documentation

Build or serve the site through the locked MkDocs environment:

mise run docs:build
mise run docs:serve

Run mise run generate before building documentation if specifications or codegen changed.

The MkDocs configuration is docs/mkdocs.yml; it renders docs/src/ into docs/dist/. Handwritten pages are maintained under docs/src/. The PTF catalog and API reference pages are generated from reviewed specifications and the compiled semantic model. C reference Markdown is generated under docs/src/reference/c/; C++ reference Markdown is generated under docs/src/reference/cpp/; Python module pages are generated under docs/src/reference/python/ and render public docstrings through mkdocstrings. Rust API documentation is published by docs.rs.

Change scope and commits

Follow Adding a PTF for new functions. Keep the extraction, review, generation, and other changes for one source publication in a single commit. Keep formula changes limited to one function or a closely related group, and do not combine them with an unrelated refactor.

Use English Conventional Commits:

<type>(<scope>): <short summary>

[optional body]

[optional footer(s)]

Use imperative mood, keep the subject to 50 characters or fewer, and avoid emojis. If a body is needed, wrap it at 72 characters.