Skip to content

feat!: ship C ABI 3 and schema 0.10 on a consolidated engine - #51

Merged
owenthcarey merged 5 commits into
mainfrom
overhaul/abi-3
Oct 3, 2026
Merged

owenthcarey merged 5 commits into
mainfrom
overhaul/abi-3

Conversation

@owenthcarey

Copy link
Copy Markdown
Contributor

This is the overhaul from the repo analysis: C ABI revision 3 and IDL schema 0.10.0, built on a consolidated engine. It's one squashed commit, breaking by design (pre-1.0), with no compatibility shims.

Excluding snapshots: 557 files changed, +54K / −79K lines. Snapshots, mostly regenerated: 395 files, +77K / −55K.

Why

The analysis found problems the README didn't mention:

  • Undefined behavior: validation accepted IDLs whose C header declared conflicting symbols. The macro also emitted the wrong thunk signature for C-style enums and interfaces declared in a sibling module.
  • Generated code that was unsafe or crashed:
    • Python and Ruby passed bool as 32 bits.
    • .NET and Kotlin had a use-after-free when the finalizer ran mid-call.
    • Dart aborted on off-thread callbacks.
    • Wasm only loaded with an experimental Node flag.
  • Packages that only built inside the test harness: Swift, Node, Kotlin and Go.
  • Fixed weaveffi_* runtime symbols, so two WeaveFFI libraries couldn't share one process.

What changed

Identity

  • Package name, C prefix and native library name come from Cargo.toml (Rust producers) or [package] (IDLs).
  • Every symbol is {prefix}_…, including the runtime.
  • Nothing generated is named after WeaveFFI.

ABI 3 (see docs/src/reference/abi.md)

  • Strings are (ptr, len) and freed with {prefix}_free_bytes.
  • New -5 cancelled code.
  • Cancel tokens are reference counted, and cancelling one drops the future.
  • Async completion fires exactly once.
  • Per-module contract checksums are checked at load.
  • Leak counters for tests.

Validation

  • A global C symbol table catches every collision.
  • Qualified names must match their module.
  • Float map keys and synchronous cancellable functions are rejected.
  • Resolved types use absolute names.

Macro

  • Thunks are unsafe and doc-hidden.
  • Error messages come from Display.
  • Callback methods can return Result<T, ForeignError>.
  • &str and &[u8] parameters are zero-copy.
  • A type from a sibling module tree must be a record or rich enum. The old silent ABI mismatch is now a compile error with guidance.

Crates

  • 18 become 6: weaveffi, -abi, -macros, -model, -gen, -cli.
  • The proc macro no longer pulls YAML, schemars, miette or rayon into producer builds.

Generators

  • All 11 targets are ported, with standalone packages, idiomatic cancellation, lifetime safety, and their fixed runtime code moved into real source files.
  • Node and Wasm share one JS layer.
  • C gets a generated value-buffer helper header.
  • package builds an XCFramework when iOS slices are included.

CLI

  • weaveffi init.
  • A [project] table (input, out, targets), so bare weaveffi generate works.
  • Generation writes only changed files and removes files it no longer produces.
  • diff --target without running hooks.
  • package defaults to the host, or to the platforms already in --binaries.

Tests and CI

  • Integration tests are one binary, and the brittle prose-grep tests are gone.
  • Every snapshot fixture is compile-checked with each language's own toolchain.
  • Conformance runs per language with lane timeouts and leak assertions, on Linux and macOS.
  • release-plz with prebuilt binaries replaces semantic-release and the Node tooling.

Docs

  • Rewritten for the new design, and about 60% smaller.

Verification (local, macOS)

  • cargo fmt --check, cargo clippy --workspace --all-targets -D warnings, cargo doc (with -D warnings -D rustdoc::all), and cargo machete: clean.
  • cargo insta test --workspace --check: all pass.
  • bash scripts/check-fixtures.sh: 55 of 55 pass (5 fixtures × 11 targets).
  • bash conformance/run.sh: 61 of 61 lanes pass across all 11 languages, each ending with every leak counter at zero. The C and C++ lanes run under ASan/UBSan.
  • weaveffi package --target swift --platforms ios-arm64,ios-sim-arm64,darwin-x64 assembled an XCFramework that swift build links against.

Not done, or not verified

  • Not run locally: cargo deny (not installed) and the new CI workflows themselves (the matrix, the composite toolchain action, the Android NDK job, release-plz, and binary releases).
  • No Windows conformance lanes yet. On the roadmap.
  • Codec golden vectors: not done. Each language's codec consumer still asserts its values by hand. On the roadmap.
  • Rich callback returns, async callback methods and vtable versioning: deferred, as planned.
  • Known limitations, documented on each language's page:
    • In Dart, a callback method that returns a value can't be called from another thread.
    • {PREFIX}_LIBRARY is a build-time setting for Go, Swift and C++, because they link the library rather than load it.
    • Kotlin's Gradle builds weren't run (no Gradle locally). The sources and the JNI shim are verified with kotlinc and CMake.

BREAKING CHANGE: the C ABI moves to revision 3, the IDL schema to 0.10.0,
and every generated package changes shape.

- Identity drives every name. The library's package name, C symbol prefix,
  and native library name come from its Cargo.toml (Rust producers) or
  `[package]` in weaveffi.toml (IDLs). Every C symbol and type is
  `{prefix}_...`, including the runtime, so any number of WeaveFFI-built
  libraries can share a process; no generated package is named after
  WeaveFFI.
- ABI 3: strings cross as (ptr, len) UTF-8 runs freed with
  `{prefix}_free_bytes` (no `free_string`, no NUL stripping); a new `-5`
  cancelled code; reference-counted cancel tokens whose cancellation drops the
  future; exactly-once async completion even when a spawner drops a future;
  per-module contract checksums every consumer verifies at load; leak
  counters (`{prefix}_debug_live`) behind the `leak-check` feature.
- Validation: a global C symbol table rejects every collision (flattened
  module paths, async completion types, iterator symbols, constants, buffer
  helpers); qualified names must match their module; float map keys and
  synchronous `cancellable` functions are rejected. Resolved types carry
  absolute names.
- Producer macro: thunks are `unsafe extern "C"` and hidden; the prefix is the
  crate name; `export_runtime!` is a proc macro; error messages come from
  `Display`; callback methods may return `Result<T, ForeignError>`; borrowed
  `&str`/`&[u8]` parameters are zero-copy; a type referenced from a sibling
  module tree must be a record or rich enum (a compile error with guidance
  replaces a silent ABI mismatch); iterators are synchronized; buffers use
  memcpy fast paths.
- Crates: 18 crates become 6 (`weaveffi`, `weaveffi-abi`, `weaveffi-macros`,
  `weaveffi-model`, `weaveffi-gen`, `weaveffi-cli`); the proc macro no longer
  pulls YAML, TOML, JSON Schema, miette, or rayon into producer builds.
- Generators: every target is ported to ABI 3 with identity-driven names,
  standalone packages, cancellation wired to the language's idiom,
  use-after-free-safe object lifetimes, and fixed runtime code moved into real
  source files. Node and Wasm share one JS layer; Wasm loads without
  experimental flags; C gains a generated value-buffer helper header; C++
  includes the C header; Swift is Sendable and packages an XCFramework for
  iOS; Kotlin uses per-module objects, reachability-safe calls, and supports
  Android API 21 and the JVM; .NET uses SafeHandle and LibraryImport; Python
  binds prototypes once and passes bytes without per-byte loops; Go takes a
  context.Context; Dart callbacks are safe from other threads where the VM
  allows; Ruby releases the GVL around blocking calls.
- CLI: a `[project]` table (input, out, targets) so bare `weaveffi generate`
  works; `weaveffi init`; the orchestrator renders in memory, writes only
  changed files, and removes files it no longer generates; `diff --target`
  without running hooks; `package` defaults to the host (or the platforms in
  `--binaries`) and assembles a Swift XCFramework for iOS slices.
- Tests and CI: one CLI test binary; brittle prose tests removed; snapshot
  fixtures compile-checked with every language's toolchain; conformance split
  per language with per-lane timeouts and leak assertions, run on Linux and
  macOS; release-plz and prebuilt binaries replace semantic-release.
- Docs rewritten for the new design (about 60 percent smaller).
- Replace expression-position bail! (an error on Rust 1.99) with Err(miette!).
- Force LF checkouts so include_str! runtime templates render identically on
  Windows, and normalize generated output paths to forward slashes.
- Install the Dart SDK directly; setup-dart can't run inside a composite action.
- Update the yanked yoke-derive 0.8.3 to 0.8.4.
…aunch

The test let another thread destroy the consumer's token before the launcher
adopted it, a use-after-free the contract forbids; it now races cancel and
destroy against the running future instead. Also normalize path separators in
the Swift identity test for Windows.
@owenthcarey
owenthcarey merged commit 0948ebb into main Oct 3, 2026
44 checks passed
@owenthcarey
owenthcarey deleted the overhaul/abi-3 branch October 3, 2026 04:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant