Skip to content

feat!: ship C ABI 5, model v2, and a producer API without boilerplate - #57

Open
owenthcarey wants to merge 2 commits into
mainfrom
feat/foundations-abi5
Open

owenthcarey wants to merge 2 commits into
mainfrom
feat/foundations-abi5

Conversation

@owenthcarey

Copy link
Copy Markdown
Contributor

Summary

This is the foundations overhaul: a pre-1.0 rewrite with no compatibility shims. The goal is that later work only adds things. That means an ABI designed to be extended additively, a model that generators can't second-guess, and a producer macro that needs no hand-written glue.

  • C ABI revision 5
    • Enums: C-style enums, rich-enum tags and error-code types are int32_t typedefs, never typedef enum. The size of a C enum is implementation-defined, for example under -fshort-enums.
    • Error struct: the message is (message_ptr, message_len) instead of a NUL-terminated string, and {p}_error_set takes a pointer and a length.
    • OptDirect family: scalar optionals (i32?, bool?, Color?) cross as a flag plus a value in every position, with no value buffer.
    • Slice family: numeric lists ([i32], [f64], …) cross as typed arrays in every position.
    • Alignment: every byte run is 8-aligned.
    • Contract tables:
      • Canonical signatures exclude parameter and field names.
      • Every error code and every callback method gets its own entry, so adding either never breaks a deployed binding.
      • Error domains are open: a code the bindings don't know maps to the domain's base error.
    • THREAD_AFFINE vtable flag: a value-returning callback called off its thread fails with -4 instead of aborting the process. Dart sets it.
  • Schema 0.12.0
    • throws: <Domain> or throws: any replaces throws: true.
    • A module may declare several error domains.
    • The IDL is YAML or JSON only; TOML IDL support is removed.
    • serde_yaml_ng replaces the archived serde_yaml.
  • Model v2
    • Ty, ParamTy and RetTy are split by position.
    • Every binding stores its passing contracts with release symbols and error domains already resolved: ArgPass, RetPass, ResultPass, ItemPass, CallbackRetPass and ErrorStrategy. No generator matches on Family:: anymore.
    • The symbol table is typed, there's a new SlotCollision rule, and the unknown-name fallback is gone.
    • Error type names are no longer doubled: KitchenErrors becomes KitchenError.
    • The macro's extractor moved into weaveffi-macros, so weaveffi-model no longer depends on syn.
  • Producer API
    • #[weaveffi::error] generates Display and Error from doc comments or from #[weaveffi(message = "... {field} ...")].
    • Error types are per function: an error type that isn't a declared domain (String, io::Error, anyhow::Error) becomes throws any.
    • Callback methods return Result<T, E> with E: From<ForeignError>.
    • New supported shapes: usize, isize, char, custom types via #[weaveffi::custom], and #[weaveffi::skip].
    • Thunk identifiers are hygienic, and a stray marker or a missing export_runtime! is now a compile error.
    • Tokio is the default executor; the hand-rolled worker pool is deleted.
  • Generators
    • One Target trait and one registry, which drives config, --target, snapshots and a CI-matrix check.
    • Shared emitters for composite codecs, contract tables, error tables and doc rewriting.
    • All 11 targets are ported, and every target reports a failed load check as a catchable error.
    • Per-target fixes:
      • Node.js: fixed undefined behavior in 64-bit argument conversion; integer arguments are range-checked.
      • Wasm: the instance is poisoned after a trap; Emscripten mode is removed.
      • Dart: per-call out slots and Uint8List.
      • Kotlin: content equality for array fields, plus @JvmStatic and @Throws.
      • Go: Check() error and a configurable package name.
      • Swift: check() and identity Hashable.
      • C++: an Error root for every exception and generic templates.
      • Python: 3.10 floor and frozen dataclasses.
      • Ruby: 3.2 floor and Data.define records.
      • .NET: sealed record types and re-enumerable iterators.
  • CLI
    • diff folds into generate --check and generate --diff; schema --version replaces schema-version.
    • One --profile flag replaces --release and --debug.
    • Producers build with cargo rustc --crate-type cdylib, so crates no longer need crate-type.
    • build and package gain --strict.
    • The manylinux tag is read from ELF symbol versions.
    • Fixed a race in reading the built library: Cargo re-creates the uplifted cdylib even on a no-op build, so the CLI now reads rustc's original in deps/.
  • Tests and docs
    • Full snapshots for kitchen_sink only; the other fixtures are covered by compile checks.
    • The conformance consumers in all 11 languages exercise every new shape.
    • The docs cover ABI 5, schema 0.12, target tiers (Tier 1: C, C++, Swift, Kotlin, Python, Node.js, .NET; Tier 2: Go, Ruby, Dart, WebAssembly), and the migration notes in docs/src/stability.md.

Test plan

  • Added or updated tests where appropriate: runtime ABI 5 tests, new and updated macro trybuild cases, model lowering, contract and validation tests, CLI tests (including a registry/CI-matrix check), and conformance consumers for every new sample API
  • Ran cargo test --workspace (cargo insta test --workspace --check passes), cargo clippy --workspace --all-targets -- -D warnings (also -p weaveffi --no-default-features), the strict rustdoc build, mdbook build docs, and cargo check -p weaveffi-fuzz
  • Ran fixture compile checks with each language's toolchain: 55/55 (5 fixtures × 11 targets)
  • Ran the full conformance harness locally on macOS: 34/34 lanes, leak counters at zero
  • Reviewed and accepted intentional snapshot changes
  • Updated documentation for user-facing changes
  • PR title follows Conventional Commits

Notes for reviewers

  • Breaking everywhere: C ABI 4 → 5 and schema 0.11 → 0.12, so every binding must be regenerated and every producer rebuilt. CHANGELOG.md, release-plz.toml and crate versions are untouched, for release automation to handle.
  • Behavior changes to know:
    • Rust callback methods always return Result, so callback methods extracted from Rust are throws any.
    • A typed callback error reaches the caller with the domain's own message.
    • Kotlin field names are now lowerCamelCase, and Python exception classes are {Code}Error.
    • Ruby flattens IDL modules into one Ruby module (names are global).
    • weaveffi dev prints the library path under target/<profile>/deps/.
  • Design choices worth a look:
    • Type names stay strings rather than typed IDs: they're globally unique, and every lookup is checked at validation.
    • Swift's check() throws, but a call made despite a mismatch still traps; the alternative would make every Swift call throws.
    • The module (bridge) macro stays, because item-level macros would need a single-slot ABI and lose the idiomatic C header.
  • Out of scope / follow-ups:
    • Replacing the Node.js addon transport.
    • Kotlin on the JVM's FFM API or Kotlin Multiplatform.
    • Dart native assets.
    • Duration and timestamp types.
    • Windows conformance lanes.
    • Async callback methods. ABI 5's per-method contract entries and vtable size check leave room to add them without a new revision.
  • Local environment caveats: conformance ran on macOS x64 only; GCC wasn't available, so C++ ran only under clang. CI covers Linux.

Settle the foundations so later work only adds things: an ABI designed to
be extended additively, a model generators can't second-guess, and a
producer macro that needs no hand-written glue.

C ABI revision 5:
- C-style enums, rich-enum tags, and error-code types are `int32_t`
  typedefs, never `typedef enum`.
- The error struct carries the message as `(message_ptr, message_len)`;
  `{p}_error_set` takes a pointer and length.
- New OptDirect family: scalar optionals (`i32?`, `bool?`, `Color?`) cross
  as a flag plus a value in every position instead of a value buffer.
- New Slice family: numeric lists (`[i32]`, `[f64]`, ...) cross as typed
  arrays in every position.
- Every byte run is 8-aligned.
- Contract hashes exclude parameter and field names; every error code and
  every callback method gets its own entry, so adding either never breaks
  a deployed binding. Error domains are open: an unknown code maps to the
  domain's base error.
- Vtable flag `THREAD_AFFINE`: a value-returning callback called off its
  thread fails with -4 instead of aborting (Dart sets it).

Schema 0.12.0: `throws: <Domain>` or `throws: any` replaces `throws:
true`; a module may declare several error domains (`errors:` is a list);
YAML and JSON only (TOML IDL removed); serde_yaml_ng replaces serde_yaml.

Model v2: `Ty`, `ParamTy`, and `RetTy` split by position; owned
`ArgPass`, `RetPass`, `ResultPass`, `ItemPass`, `CallbackRetPass`, and
`ErrorStrategy` stored on every binding with release symbols and domains
resolved; callback methods carry an `AbiFn`; a typed symbol table; a
`SlotCollision` rule; no unknown-name fallback or public escape hatches.
Error type names are no longer doubled (`KitchenErrors` -> `KitchenError`).
The macro's extractor moved into weaveffi-macros, so the model no longer
depends on syn.

Producer API: `#[weaveffi::error]` generates `Display` and `Error` from doc
comments or `#[weaveffi(message = "...")]` (opt out with `no_display`);
per-function error types, with any `Display` error becoming `throws any`;
callback methods return `Result<T, E>` with `E: From<ForeignError>`;
`usize`, `isize`, and `char` cross; custom types via
`#[weaveffi::custom]`; `#[weaveffi::skip]`; hygienic thunks; stray
markers and a missing `export_runtime!` fail to compile. Tokio is the
default executor and the hand-rolled worker pool is gone.

Generators: one `Target` trait and one registry; shared emitters for
composite codecs, contract tables, error tables, and doc rewriting; every
target ported, with catchable load checks everywhere and per-target fixes
(Node.js 64-bit UB and integer range checks, wasm instance poisoning,
Emscripten mode removed, Dart per-call out slots and `Uint8List`, Kotlin
content equality and `@JvmStatic`/`@Throws`, Go `Check() error` and a
configurable package, Swift `check()` and identity equality, C++ `Error`
root and generic templates, Python 3.10 frozen dataclasses, Ruby 3.2
`Data.define` records, .NET records and re-enumerable iterators).

CLI: `diff` folds into `generate --check`/`--diff`, `schema --version`
replaces `schema-version`, one `--profile` flag, producers build with
`cargo rustc --crate-type cdylib` (no `crate-type` needed), `--strict`
for build and package, manylinux tags read from ELF symbol versions.

Tests and docs: full snapshots for kitchen_sink only, conformance
consumers in all 11 languages cover every new shape, and the docs
describe ABI 5, schema 0.12, target tiers, and migration notes.

BREAKING CHANGE: C ABI revision 5 and IDL schema 0.12.0. Regenerate every
binding and rebuild every producer. See docs/src/stability.md for the
migration notes.
- Dart: build `part` paths with `/` instead of platform joins, so Windows
  emits `src/runtime/loader.dart`, not `src/runtime\loader.dart`.
- CLI test: compare the TOML-rejection message without spaces, since
  miette wraps it at the terminal width.
- Conformance: the missing-library checks (Python, Kotlin, .NET) use a
  leaf name the real library doesn't have. dyld searches
  DYLD_LIBRARY_PATH by leaf name even for an absolute path, which found
  the real library on CI runners, where DYLD_* reaches the process.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant