feat(rust): armonik-transport-ffi, the skeleton of the C ABI - #744
Draft
wkirschenmann wants to merge 7 commits into
Draft
wkirschenmann wants to merge 7 commits into
wkirschenmann wants to merge 7 commits into
Conversation
☂️ Python Coverage
Overall Coverage
New FilesNo new covered files... Modified FilesNo covered modified files...
|
wkirschenmann
force-pushed
the
wk/feat/rust-ffi-skeleton
branch
from
August 9, 2026 08:37
231cac6 to
6e978b4
Compare
wkirschenmann
force-pushed
the
wk/feat/rust-ffi-skeleton
branch
from
August 9, 2026 10:40
6e978b4 to
8c4f061
Compare
The crate that carries the C ABI over `armonik-transport`, with the vocabulary every entry point
answers in and the one invariant that holds before anything crosses at all: a panic stays on this
side. Unwinding out of an `extern "C"` function into whatever called it is undefined behaviour, so
an entry point runs its body through a guard rather than running it directly.
Result codes are a `#[repr(i32)]` enum rather than a list of constants, and that is a decision the
generators make rather than a matter of taste: a constant's value is read as a literal by the C#
generator, so `-1` parses as a unary expression and reaches the bindings as nothing at all, while an
enum's discriminants come across sign and all. Entry points still return a plain `i32`: a C enum's
underlying type is implementation-defined, so a signature naming one would promise a width this ABI
would then have to keep. -7 is reserved, and the gap is the point - a caller comparing against it
matches nothing rather than matching something new.
The names carry `AK_` in the source rather than acquiring it in a generator's rename table, so one
name reads the same in every artefact and in Rust, and a code added later cannot reach one of them
unprefixed.
`ak_status::code` and the guards land together because they are one unit to the compiler: the guards
are the only thing that turns a code into the `i32` an entry point returns, and a commit with the
method and no caller would be a commit of dead code.
`dead_code` measures reachability from a public Rust API, which is not this crate's contract - what
it offers is the `extern "C"` surface - so the modules behind that surface carry an allow, and their
own tests are what covers them.
Verification, from `packages/rust`:
cargo test -p armonik-transport-ffi 7 pass
cargo clippy -p armonik-transport-ffi --all-targets --all-features -- -D warnings clean
cargo fmt --check clean
An opaque handle reaches the caller as a pointer, and every entry point has to turn one back into
something it can read. A set of live addresses cannot make that safe: checking that a pointer is
live and using what it points at are two separate moments, so a release on another thread lands
between them and deallocates the object a call is halfway through reading. A host application whose
UI thread abandons a request while a pool thread is still writing to it does exactly that.
So the registry owns the handles. Each is an `Arc` it holds, and a call takes a counted reference for
its own duration; a release drops the registry's reference and nothing else. Which is why the name is
`_release` rather than `_free`: the call gives back a reference, it does not destroy an object.
The race is what the tests are for, and the interesting one needs contention rather than a story
about interleaving: 64 threads read the same handle while every one of them also tries to release it,
and exactly one release owns it. Scaled down under Miri, which interprets every access. What remains
open is stated in the module: a pointer used after its release is refused only until the allocator
hands that address to a new handle of the same type, and closing that needs generation-tagged slots
that nothing here warrants yet.
Verification, from `packages/rust`:
cargo test -p armonik-transport-ffi 13 pass
cargo +nightly miri test -p armonik-transport-ffi --lib 13 pass
cargo clippy -p armonik-transport-ffi --all-targets --all-features -- -D warnings clean
cargo fmt --check clean
`ak_bytes` is what leaves this crate owned, and `ak_bytes_in` is what it borrows. The first is given
up by exactly one `ak_bytes_release`; the second is a view into memory the caller owns and is never
released here.
`ak_bytes` keeps its `ptr`/`len` view apart from its `owner`, and that split is what lets a buffer
this crate already holds cross without being copied. A `Bytes` may be a refcounted view into the
middle of a larger allocation, so `ptr` is not on its own something that can be deallocated;
`owner` is a boxed handle to whatever really owns those bytes, and releasing means dropping that.
An implementation that treated the view as the allocation would corrupt the heap, which is what the
sub-slice test asks Miri to check.
The guards gain the variant that reports through one of these: on a panic, `catch_unwind_status`
renders the payload into the caller's `out_err` exactly as any other failure would, and answers
`AK_INTERNAL_PANIC`. `out_err` stays optional, so a caller that wants only the code is not a special
case on the error path.
Verification, from `packages/rust`:
cargo test -p armonik-transport-ffi 19 pass
cargo +nightly miri test -p armonik-transport-ffi --lib 19 pass
cargo clippy -p armonik-transport-ffi --all-targets --all-features -- -D warnings clean
cargo fmt --check clean
A caller of this ABI gets one string and nothing else, which decides both halves of this.
The chain is flattened. `armonik-transport` reports "Could not establish TLS connection to the remote
..." and leaves why - a key that does not match its certificate, a CA file that could not be read -
in the source beneath it. Nothing survives a C ABI but the bytes handed across it, and there is no
error chain left to walk on the other side, so a message that stopped at the outermost error would
drop the only part that says what to fix.
And the locations go. `error-locations`, off by default, is a disclosure rule rather than a matter of
taste: a released build says what went wrong, never whereabouts in the source it happened, and being
open source changes nothing - what someone can find by reading a repository is not what a library
volunteers in a log. Two coordinates fall under it: the ` [file.rs:12:34]` a Rust error carries, and
the ` at line 1 column 56` `serde_json` names inside a configuration document, which the options
layer generates and whose positions are therefore as internal as a file path. A panic payload does
not fall under it - it is prose this crate wrote, names no file and no type, and is rendered in both
builds.
Build-time rather than a configuration option, so a released library carries neither the branch nor
the choice, and the ABI is identical either way: the two builds are interchangeable, and a consumer
picks one by which library it ships rather than by how it calls. The removal tests call
`remove_locations` directly so they assert the same thing in either build, and the policy tests
assert per feature.
`FfiError` is kept apart from `armonik_transport::ConfigError` rather than reusing its variants:
that type is `#[non_exhaustive]` and this crate only ever constructs errors, so there is nothing to
gain by fighting the boundary.
Verification, from `packages/rust`:
cargo test -p armonik-transport-ffi 28 pass
cargo test -p armonik-transport-ffi --all-features 28 pass
cargo +nightly miri test -p armonik-transport-ffi --lib 28 pass
cargo +nightly miri test -p armonik-transport-ffi --lib --all-features 28 pass
cargo clippy -p armonik-transport-ffi --all-targets --all-features -- -D warnings clean
cargo fmt --check clean
Request headers, response headers and trailers all cross the boundary the same way: a `u32` count,
then that many length-prefixed key and value pairs. One encoder and decoder rather than a format per
use means one implementation to get right, one set of bounds checks, and one test suite covering the
malformed cases.
Integers are in native byte order, and what makes that correct is that both ends are one process
built for one target: a reader decodes exactly what the writer encoded, whatever that target's byte
order happens to be. Nothing here is gated on an operating system or an architecture, and nothing
needs to be - the only thing that would make byte order anyone else's business is this encoding being
mistaken for a wire format.
Keys and values are opaque bytes, duplicates are allowed and order is kept, because that is what a
header list is: a `-bin` value is raw binary, and validating it would put policy in a decoder. What
the decoder does refuse is a malformed blob - a truncated chunk, a length running past the end, and a
count so large it could not possibly fit, which is rejected on the arithmetic before anything is
allocated for it.
Verification, from `packages/rust`:
cargo test -p armonik-transport-ffi 35 pass
cargo +nightly miri test -p armonik-transport-ffi --lib 35 pass
cargo clippy -p armonik-transport-ffi --all-targets --all-features -- -D warnings clean
cargo fmt --check clean
… rest
A process-wide multi-threaded tokio runtime, created lazily on first use and living for the life of
the process. There is no entry point that shuts it down, which is how a native library loaded into a
host process is expected to behave.
`alive_tasks` is exported hidden, and is the only thing here that is not for the library's own use.
Leak assertions have nothing else to look at: work that left a task parked forever is invisible from
the outside, and "the tests passed" is not evidence that the runtime came back to rest. Nothing in
the module is `extern "C"`, so none of it is part of the ABI.
Its own tests are skipped under Miri, which cannot drive tokio's I/O reactor: it reaches the
platform's completion-port calls and stops with "unsupported operation". They are the only three that
touch the OS, and marking them is what keeps the rest of the crate runnable there.
Verification, from `packages/rust`:
cargo test -p armonik-transport-ffi 38 pass
cargo +nightly miri test -p armonik-transport-ffi --lib 35 pass, 3 ignored
cargo clippy -p armonik-transport-ffi --all-targets --all-features -- -D warnings clean
cargo fmt --check clean
`build.rs` generates both from the same sources and writes them into the tree, under `include/`,
where they are committed. They are what a reviewer reads to see the whole contract at once and what a
caller's own declarations are checked against; generating them into `OUT_DIR` would put them
somewhere nobody looks, and not committing them would leave a change to the ABI visible only in a
compiled library. Neither is an input to compiling this crate, so a generator that fails warns rather
than breaking the build.
`ak_abi_version` comes with them, because a version nobody can ask for is not much use. A host process
loads one native module and every add-in in it shares whichever was loaded first, so an add-in that
did not bring its own has to be able to find out what it got; asking turns a mismatch into a
diagnosis, where reaching for an entry point that is not there surfaces as an
`EntryPointNotFoundException` from somewhere unrelated. The header's preamble states what may be
added within a revision and what may never change, so a caller can reason about the answer.
A cleaning pass runs over both artefacts, in the spirit of `strip_rust_details` in
`armonik-transport`: documentation written for one language reads as a leak in another. Links whose
paths resolve to nothing, headings from a documentation format the reader does not use, and the word
for a source tree where a contract has a library all go, while the prose under them stays - what a
caller must guarantee is the part a C reader most needs. Whoever opens either file has a compiler for
their own language and no way to look any of that up, and a test pins that: a reader must not be able
to tell what these were generated from.
The C# settings are what netstandard2.0 and a 32-bit host allow. `IntPtr`/`UIntPtr` rather than
`nint`/`nuint`, and delegates rather than function pointers, because neither those keywords nor
`UnmanagedCallersOnly` exist on .NET Framework. `CallingConvention.Cdecl` is spelled out on every
entry point, because the default is `StdCall` and the difference is a corrupted stack on x86, where
32-bit Office still lives. A test asserts the Cdecl count equals the `DllImport` count, and the same
for delegates, so an entry point added later cannot arrive unmarked.
Two types reach the artefacts only because they are asked for by name: a generator emits a type when
a signature mentions one, and `ak_status` and `ak_bytes_in` are part of the contract without being an
argument to anything yet. Without that the preamble would describe a shape the header does not
define.
`.gitattributes` pins `eol=lf` on `include/` and on `cbindgen.toml`. It belongs with the artefacts
rather than beside them: a generated file that cannot be reproduced byte for byte on the machine
someone is sitting at is not a pinned contract. `cbindgen.toml` is there because its `header` string
is embedded in the generated header verbatim, so a CR in the checkout is a CR in the artefact - which
is exactly what a checkout with `core.autocrlf=true` produced, leaving a rebuild permanently dirty by
one CR per line of the banner.
Verification, from `packages/rust`:
cargo test -p armonik-transport-ffi 39 unit, 8 header, 6 bindings; all pass
cargo test -p armonik-transport-ffi --all-features same 53, all pass
cargo clippy -p armonik-transport-ffi --all-targets --all-features -- -D warnings clean
cargo clippy -p armonik-transport-ffi --all-targets -- -D warnings clean
cargo fmt --check clean
cargo build --workspace --all-features builds
cargo +nightly miri test -p armonik-transport-ffi --lib 36 pass, 3 ignored
cargo +nightly miri test -p armonik-transport-ffi --lib --all-features 36 pass, 3 ignored
pnpm run verify-versions 3.29.2 for all projects, script unmodified
`cargo build -p armonik-transport-ffi` followed by `git status --short` is empty on a Windows
checkout, which is what the line-ending pin is for.
wkirschenmann
force-pushed
the
wk/feat/rust-ffi-skeleton
branch
from
August 9, 2026 10:52
8c4f061 to
7d2e485
Compare
|
❌ The last analysis has failed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sits on
wk/refactor/rust-transport-drops-tonic, to merge first.Motivation
Host applications that cannot link Rust need a C ABI over
armonik-transport, and the boundary'srules are worth settling before anything crosses it.
Description
A new crate,
armonik-transport-ffi, holding the primitives every entry point will be built from:panic guards, the reference-counted handle registry, the lazy process-wide runtime, the key/value
blob encoder, the owned and borrowed buffers, and the error renderer. Two entry points so far,
ak_abi_versionandak_bytes_release.Result codes are a
#[repr(i32)]enum rather than constants, which the generators decided:csbindgenreads a constant's value as a literal, so-1reaches the C# bindings as nothing atall. Entry points still return a plain
i32, a C enum's width being implementation-defined. -7 isreserved and carried by no code, so a caller comparing against it matches nothing rather than
whatever gets added later.
include/armonik_transport_ffi.handinclude/NativeMethods.g.csare generated bybuild.rsonevery build and committed, so a contract change shows up in a diff; tests pin both, including that a
reader cannot tell what either was generated from. The registry lends a reference for a call's
duration, because checking that a pointer is live and using what it points at are two moments.
error-locationsis off by default, so a released build never says where a failure happened in thesource; the ABI is identical either way.
Testing
Adds 53: 39 unit, 8 on the header, 6 on the bindings.
cargo test -p armonik-transport-ffiandcargo +nightly miri test -p armonik-transport-ffi --lib, both green, the second ignoring the threeruntime tests that reach the platform's completion ports.
The two runs catch different things. Natively the handle tests put 64 threads x 200 iterations on one
handle: real contention, which is what can surface a use-after-free between a release on one thread
and a read on another. Under Miri they scale to 4 x 20, an interpreter checking aliasing and memory
validity on every access needing enough interleaving to reach the paths, not volume.
Impact
No network function, no client, no reactor - those are later PRs.
publish = false, and nothingoutside this workspace consumes the crate.
Additional Information
None.
Checklist