Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 2 additions & 3 deletions .github/actions/toolchains/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ runs:
with:
python-version: '3.12'
# The python fixture check type-checks the generated package with mypy
# against Python 3.9, the oldest version it supports; mypy 2 can't
# target 3.9.
# against Python 3.10, the oldest version it supports.
- if: inputs.lang == 'python'
shell: bash
run: python -m pip install 'mypy==1.20.2'
Expand All @@ -48,7 +47,7 @@ runs:
- if: inputs.lang == 'go'
uses: actions/setup-go@v5
with:
go-version: '1.23'
go-version: '1.24'
- if: inputs.lang == 'dotnet'
uses: actions/setup-dotnet@v4
with:
Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,20 +70,20 @@ jobs:
targets: wasm32-unknown-unknown
- uses: Swatinem/rust-cache@v2
- run: cargo build -p weaveffi-cli
# `weaveffi diff --check` is the CI gate we recommend to users, so it
# must hold for our own samples: a fresh tree diffs clean, and a tree
# with a stale file is reported instead of silently passing.
- name: weaveffi diff --check on every sample
# `weaveffi generate --check` is the CI gate we recommend to users, so
# it must hold for our own samples: a fresh tree checks clean, and a
# tree with a stale file is reported instead of silently passing.
- name: weaveffi generate --check on every sample
run: |
set -euo pipefail
BIN=target/debug/weaveffi
for sample in calculator codec kvstore; do
"$BIN" generate "samples/$sample" -o "diffcheck/$sample" --quiet
"$BIN" diff "samples/$sample" -o "diffcheck/$sample" --check --quiet
"$BIN" generate "samples/$sample" -o "diffcheck/$sample" --check --quiet
done
echo "stale" >> diffcheck/calculator/c/calculator.h
if "$BIN" diff samples/calculator -o diffcheck/calculator --check --quiet; then
echo "diff --check did not detect a modified file" >&2
if "$BIN" generate samples/calculator -o diffcheck/calculator --check --quiet; then
echo "generate --check did not detect a modified file" >&2
exit 1
fi
- name: weaveffi.schema.json is up to date
Expand Down Expand Up @@ -197,7 +197,7 @@ jobs:
ANDROID_NDK_HOME: ${{ steps.ndk.outputs.ndk-path }}
run: |
set -euxo pipefail
cargo run -q -p weaveffi-cli -- build samples/kvstore --platforms android-arm64 --target kotlin
cargo run -q -p weaveffi-cli -- build samples/kvstore --platforms android-arm64 --target kotlin --strict
readelf="$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-readelf"
"$readelf" -d target/weaveffi/android-arm64/libkvstore.so | grep 'SONAME.*libkvstore.so'
"$readelf" -d target/weaveffi/android-arm64/libkvstore_jni.so | grep 'NEEDED.*libkvstore.so'
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/fuzz.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ jobs:
target:
- fuzz_parse_yaml
- fuzz_parse_json
- fuzz_parse_toml
- fuzz_parse_type_ref
- fuzz_validate
- fuzz_value_buffer
Expand All @@ -50,7 +49,7 @@ jobs:

- name: Seed the corpus
run: |
python -m pip install pyyaml tomli-w
python -m pip install pyyaml
python crates/weaveffi-fuzz/seed_corpus.py ${{ matrix.target }} \
crates/weaveffi-fuzz/corpus/${{ matrix.target }}

Expand Down
40 changes: 23 additions & 17 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,33 +2,39 @@

The contributor guide to the architecture lives in
[`docs/src/architecture.md`](docs/src/architecture.md), and the normative C
ABI is [`docs/src/reference/abi.md`](docs/src/reference/abi.md). Read them
before changing a generator, the IDL or IR, validation, the producer macro,
the runtime, or the orchestrator.
ABI (revision 5) is [`docs/src/reference/abi.md`](docs/src/reference/abi.md).
Read them before changing a generator, the IDL or IR, validation, the
producer macro, the runtime, or the orchestrator.

The pipeline:

```text
a Rust producer crate (the API read from its built library's metadata)
or an IDL (YAML/JSON/TOML)
or an IDL (YAML/JSON)
→ IR (Api)
→ validate (every rule, including the C symbol table, under the
Identity from weaveffi.toml / cargo metadata)
→ Model (C symbols and ABI signatures, computed once; the plan)
→ Target::render (pure, per language)
→ Orchestrator (changed-file writes, generation records, stale-file
removal)
→ validate (every rule, including the C symbol and slot tables, under
the Identity from weaveffi.toml / cargo metadata)
→ Model (v2: types split by position as Ty / ParamTy / RetTy; every
binding stores its AbiFn and its passing contracts, ArgPass, RetPass,
ResultPass, ItemPass, CallbackRetPass, and ErrorStrategy, resolved once)
→ Target::render (pure, per language, through the shared emitters in
codegen/; never re-derives a family, slot, or error domain)
→ Orchestrator (a Changeset: changed-file writes, generation records,
stale-file removal; `generate --check` and `--diff` report it)
```

The workspace crates:

- `weaveffi-model`: the IR, IDL parsing, the macro's Rust extraction,
library metadata frames, validation, package identity, the model, ABI
lowering, the marshalling plan, and contract tables.
- `weaveffi-cli`: the `weaveffi` command and the library behind it: the
backend framework, the orchestrator and generation records, packaging, and
the eleven language targets under `src/targets/`.
- `weaveffi-macros`: `#[weaveffi::module]` and `export_runtime!`.
- `weaveffi-model`: the IR, YAML and JSON parsing, library metadata frames,
validation, package identity, the model and its symbol table, C ABI
lowering, the passing contracts (`plan`), error-type naming, and contract
tables. It doesn't depend on `syn`.
- `weaveffi-cli`: the `weaveffi` command and the library behind it: the one
`Target` trait and its `REGISTRY`, the shared emitters (`codegen::codecs`,
`contract`, `errors`, `docs`), the orchestrator and generation records,
packaging, and the eleven language targets under `src/targets/`.
- `weaveffi-macros`: `#[weaveffi::module]`, the marker attributes, and
`export_runtime!`; its `src/extract.rs` lowers a module tree to the IR.
- `weaveffi`: the producer facade crate and the C ABI runtime
(`weaveffi::abi`) linked into every producer.
- `weaveffi-fuzz`: unpublished fuzz harnesses.
48 changes: 29 additions & 19 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,17 @@ should also pass that target's fixture check and conformance lanes.

### Snapshot tests

Snapshot tests (`crates/weaveffi-cli/tests/snapshots.rs`) pin the exact
output of every generator for the fixtures in
`crates/weaveffi-cli/tests/fixtures/`, using
Snapshot tests (`crates/weaveffi-cli/tests/snapshots.rs`) render every
registered target for every fixture in `crates/weaveffi-cli/tests/fixtures/`
and pin the exact output for the `kitchen_sink` fixture, file by file, using
[`cargo-insta`](https://insta.rs/). Fixed files (runtimes, package
manifests, and READMEs, listed per target in `FIXED_FILES`) are snapshotted
once, from `kitchen_sink`; a target's copy of the C header is asserted
byte-equal to the C target's own header instead of snapshotted; every other
file is snapshotted for every fixture. When output changes on purpose:
manifests, and READMEs, listed per target by `Target::fixed_files`) aren't
snapshotted, and a target's copy of the C header is asserted byte-equal to
the C target's own header instead. The other fixtures are covered by the
fixture compile checks, the determinism test, and targeted assertions in
each target's `tests.rs` (which should test only target-specific naming,
escaping, and configuration, not what the snapshot already pins). When
output changes on purpose:

```bash
cargo install cargo-insta --locked
Expand All @@ -80,7 +83,11 @@ type-checks it with the target language's toolchain (through
`scripts/fixtures/<target>.sh`). Run it for any target whose output you
change; CI runs one job per target. A missing tool (including `mypy` 1.x for
Python and the node-gyp headers from `npx node-gyp install` for Node.js) is
reported as a skip locally and fails the check when `CI=true`.
reported as a skip locally and fails the check when `CI=true`. The Python
and Ruby checks (and conformance lanes) need Python 3.10 and Ruby 3.2 or
newer: they use the `python3` and `ruby` on PATH when those are new enough,
and otherwise look for a `python3.N` on PATH or Homebrew's Ruby (see
`scripts/toolchains.sh`).

### Conformance

Expand Down Expand Up @@ -121,11 +128,15 @@ Preview the book with `mdbook serve docs -p 3000 -n 127.0.0.1`.

Read the [architecture guide](docs/src/architecture.md) first; its "Adding a
generator" section is the checklist. In short: add
`crates/weaveffi-cli/src/targets/<lang>/` implementing `LanguageBackend`,
register it with one line in the `cli_targets!` registry in
`crates/weaveffi-cli/src/config.rs`, add it to the snapshot tests, add
`scripts/fixtures/<lang>.sh` and `conformance/<lang>/`, and document it under
`docs/src/generators/`.
`crates/weaveffi-cli/src/targets/<lang>/` implementing `Target` (rendering
from the passing contracts stored on the model and through the shared
emitters in `codegen/`, never matching on a type's family), register
it with one entry in `REGISTRY` in `crates/weaveffi-cli/src/targets/mod.rs`
(which also adds it to `--target`, the config tables, and the snapshot
tests), add `scripts/fixtures/<lang>.sh` and `conformance/<lang>/` and list
the target in the CI matrices, `scripts/check-fixtures.sh`, and
`conformance/run.sh` (a test checks that every list matches the registry),
and document it, with its tier, under `docs/src/generators/`.

## Fuzzing

Expand All @@ -143,10 +154,9 @@ cargo install cargo-fuzz --locked
```

Seed a target's corpus from its committed seeds and the snapshot fixtures
(needs `pip install pyyaml tomli-w`), then run it for 60 seconds (swap the
target name for any of `fuzz_parse_yaml`, `fuzz_parse_json`,
`fuzz_parse_toml`, `fuzz_parse_type_ref`, `fuzz_validate`,
`fuzz_value_buffer`):
(needs `pip install pyyaml`), then run it for 60 seconds (swap the target
name for any of `fuzz_parse_yaml`, `fuzz_parse_json`, `fuzz_parse_type_ref`,
`fuzz_validate`, `fuzz_value_buffer`):

```bash
python3 crates/weaveffi-fuzz/seed_corpus.py fuzz_parse_yaml \
Expand Down Expand Up @@ -302,8 +312,8 @@ fix/android-jni-crash

- **CI** (`ci.yml`): formatting, clippy, rustdoc, and a build of
`weaveffi-model` without its IDL features; the test suite with snapshot
checks on Linux, macOS, and Windows; `weaveffi diff --check` on every
sample, the JSON Schema drift check, and a `wasm32` build of every sample
checks on Linux, macOS, and Windows; `weaveffi generate --check` on
every sample, the JSON Schema drift check, and a `wasm32` build of every sample
whose embedded metadata must match the native build's; the fixture compile
check per target; the conformance harness per language on Linux and macOS;
an Android job that links the Kotlin JNI shim with the NDK and runs
Expand Down
Loading
Loading