From 2ff474bf33dd46a902df7cf2c5ce1d165862d4c9 Mon Sep 17 00:00:00 2001 From: Owen Carey <37121709+owenthcarey@users.noreply.github.com> Date: Thu, 8 Oct 2026 13:03:23 -0700 Subject: [PATCH 1/2] feat!: rebuild WeaveFFI on C ABI 4, one validated model, and library mode Engine v2 overhaul. No compatibility shims: old code paths are removed. Crates - Four published crates: `weaveffi` (facade plus the runtime as `weaveffi::abi`), `weaveffi-macros`, `weaveffi-model`, and `weaveffi-cli` (library plus binary, absorbing the generators). `weaveffi-abi` and `weaveffi-gen` are gone. Model and schema 0.11.0 - One `Model` built once by `validate(&Api, &Identity)`; `ResolvedApi` is deleted and generators never see the document IR. - Global names: types, free functions, and error codes are unique across the API; qualified type references are removed; `Ty::Prim` wraps one primitive enum. Better diagnostics (scoped spans, unsupported primitives). C ABI revision 4 - Per-declaration contract tables replace per-module checksums, so adding a declaration no longer breaks older bindings. - Callback vtables carry a `{size, flags, free}` header; callback methods return every family (consumer runs allocated with `{p}_alloc`), may `throw` typed domain errors with payloads, and callback parameters may be optional. - Soundness: token-adopting decoders are `unsafe`, thunks adopt every input before failing, duplicate map keys are rejected, iterators never hold a lock across `next()`, and the foreign-error unwinding machinery is gone. - A pooled default executor and a `tokio` feature replace thread-per-call; wasm32 no longer spins on a stalled future. Producer macro and library mode - The macro runs the shared validator with real spans, honors item `#[cfg]`, resolves type aliases, and emits much smaller thunks through runtime helpers. - The macro embeds the API as metadata in the built library; `generate`, `diff`, `validate`, and `extract` build the crate and read it, replacing the Rust source parser. New `weaveffi dev` and a `Project` library API. Generators - A slim backend trait; the capability gate and hash cache are removed. - All eleven targets implement ABI 4 with one codec per composite type, documented trap policy, and reserved-member-name escaping. Python drops its `.pyi`; Kotlin uses unsigned types; Node.js guards against callback deadlocks; Dart loads natives relative to the package. Build and packaging - New `weaveffi build` cross-compiles per platform with `@rpath` install names, iOS static libraries, NDK detection, deployment targets, and prebuilt Node.js and JNI glue. `weaveffi package` writes installable artifacts: wheels, npm tarballs, gems, NuGet packages, an XCFramework zip with its checksum, and C/C++ tarballs. - Config cleanup: one `name` override key, `[build]`, `[package] dist`; hooks, `[global]`, and `--force` are removed. Samples, tests, CI, docs - Three samples (calculator, codec with shared vectors, feature-complete kvstore) with consumers in all eleven languages; 34 conformance lanes. - Snapshot corpus cut by about a third; missing tools fail CI; fuzz, bench, and release workflows fixed; a package-install smoke job. - Every documentation page updated, including a normative ABI 4 contract, migration notes, roadmap, and comparison. --- .github/actions/toolchains/action.yml | 16 +- .github/workflows/bench.yml | 37 +- .github/workflows/ci.yml | 75 +- .github/workflows/docs.yml | 8 +- .github/workflows/fuzz.yml | 29 +- .github/workflows/pr-lint.yml | 6 + .github/workflows/quality.yml | 7 + .github/workflows/release-binaries.yml | 26 +- .github/workflows/release.yml | 2 + .gitignore | 11 +- ARCHITECTURE.md | 30 +- CONTRIBUTING.md | 67 +- Cargo.lock | 158 +- Cargo.toml | 9 +- README.md | 101 +- SECURITY.md | 8 +- conformance/c/async_demo.c | 245 --- conformance/c/calculator.c | 58 + conformance/c/codec.c | 1078 +++++------ conformance/c/contacts.c | 247 --- conformance/c/events.c | 359 ---- conformance/c/kvstore.c | 1407 ++++++++++---- conformance/c/producer.c | 74 +- conformance/c/producer_check.c | 31 +- conformance/c/run.sh | 26 +- conformance/cpp/async_demo.cpp | 143 -- conformance/cpp/calculator.cpp | 62 +- conformance/cpp/check.hpp | 53 +- conformance/cpp/codec.cpp | 662 +++---- conformance/cpp/contacts.cpp | 123 -- conformance/cpp/events.cpp | 367 ---- conformance/cpp/kvstore.cpp | 887 +++++---- conformance/cpp/run.sh | 11 +- conformance/dart/async_demo.dart | 77 - conformance/dart/calculator.dart | 46 +- conformance/dart/codec.dart | 1028 +++++----- conformance/dart/contacts.dart | 88 - conformance/dart/events.dart | 200 -- conformance/dart/kvstore.dart | 1119 +++++++---- conformance/dart/run.sh | 16 +- conformance/dart/support.dart | 43 +- conformance/dotnet/AsyncDemo.cs | 129 -- conformance/dotnet/Calculator.cs | 59 + conformance/dotnet/Codec.cs | 553 ++---- conformance/dotnet/Contacts.cs | 143 -- conformance/dotnet/Events.cs | 310 --- conformance/dotnet/Kvstore.cs | 840 +++++--- conformance/dotnet/LeakCheck.cs | 41 +- conformance/dotnet/run.sh | 19 +- conformance/go/async_demo.go | 105 - conformance/go/calculator.go | 58 +- conformance/go/codec.go | 686 +++---- conformance/go/common.go | 21 +- conformance/go/contacts.go | 115 -- conformance/go/events.go | 264 --- conformance/go/kvstore.go | 968 ++++++---- conformance/go/run.sh | 18 +- conformance/kotlin/async_demo.kt | 108 -- conformance/kotlin/calculator.kt | 39 + conformance/kotlin/codec.kt | 609 +++--- conformance/kotlin/contacts.kt | 144 -- conformance/kotlin/events.kt | 254 --- conformance/kotlin/kvstore.kt | 805 ++++---- conformance/kotlin/run.sh | 19 +- conformance/lib.sh | 23 +- conformance/node/async_demo.mjs | 62 - conformance/node/calculator.mjs | 43 + conformance/node/codec.mjs | 417 ++-- conformance/node/contacts.mjs | 119 -- conformance/node/events.mjs | 139 -- conformance/node/kvstore.mjs | 604 ++++-- conformance/node/run.sh | 20 +- conformance/python/async_demo_consumer.py | 110 -- conformance/python/calculator_consumer.py | 56 +- conformance/python/codec_consumer.py | 611 +++--- conformance/python/contacts_consumer.py | 132 -- conformance/python/events_consumer.py | 292 --- conformance/python/harness.py | 46 +- conformance/python/kvstore_consumer.py | 824 ++++---- conformance/python/run.sh | 18 +- conformance/ruby/async_demo.rb | 120 -- conformance/ruby/calculator.rb | 61 +- conformance/ruby/codec.rb | 551 ++---- conformance/ruby/contacts.rb | 145 -- conformance/ruby/events.rb | 291 --- conformance/ruby/kvstore.rb | 800 ++++---- conformance/ruby/run.sh | 18 +- conformance/ruby/support.rb | 31 +- conformance/run.sh | 42 +- conformance/swift/async_demo.swift | 126 -- conformance/swift/calculator.swift | 79 + conformance/swift/codec.swift | 545 +++--- conformance/swift/contacts.swift | 135 -- conformance/swift/events.swift | 245 --- conformance/swift/kvstore.swift | 947 ++++++--- conformance/swift/run.sh | 21 +- conformance/wasm/run.sh | 20 +- crates/weaveffi-abi/Cargo.toml | 23 - crates/weaveffi-abi/src/callback.rs | 516 ----- crates/weaveffi-abi/src/lib.rs | 117 -- crates/weaveffi-abi/src/spawn.rs | 389 ---- crates/weaveffi-cli/Cargo.toml | 19 +- .../benches/codegen_bench.rs | 88 +- crates/weaveffi-cli/src/backend/mod.rs | 186 ++ crates/weaveffi-cli/src/build/glue.rs | 303 +++ crates/weaveffi-cli/src/build/mod.rs | 280 +++ crates/weaveffi-cli/src/build/ndk.rs | 180 ++ .../src/cabi.rs | 216 ++- crates/weaveffi-cli/src/cache.rs | 73 + crates/weaveffi-cli/src/cargo.rs | 316 +++ .../src/codegen/common.rs | 129 +- crates/weaveffi-cli/src/codegen/mod.rs | 360 ++++ .../src/codegen/writer.rs | 4 +- crates/weaveffi-cli/src/commands/build.rs | 337 ++++ crates/weaveffi-cli/src/commands/dev.rs | 132 ++ crates/weaveffi-cli/src/commands/diff.rs | 19 +- crates/weaveffi-cli/src/commands/extract.rs | 58 + crates/weaveffi-cli/src/commands/generate.rs | 76 +- crates/weaveffi-cli/src/commands/init.rs | 45 +- crates/weaveffi-cli/src/commands/mod.rs | 324 +--- crates/weaveffi-cli/src/commands/package.rs | 691 +++---- crates/weaveffi-cli/src/commands/validate.rs | 250 +-- crates/weaveffi-cli/src/config.rs | 422 ++-- crates/weaveffi-cli/src/extract.rs | 84 - .../src/lang.rs | 25 + crates/weaveffi-cli/src/lib.rs | 33 + crates/weaveffi-cli/src/library.rs | 161 ++ crates/weaveffi-cli/src/main.rs | 214 ++- .../src/manifest.rs | 6 - crates/weaveffi-cli/src/package/archive.rs | 769 ++++++++ crates/weaveffi-cli/src/package/mod.rs | 347 ++++ .../src/platform.rs | 414 +++- crates/weaveffi-cli/src/project.rs | 553 ++++++ .../src/targets/c/buffer.rs | 102 +- .../src/targets/c/header.rs | 26 +- crates/weaveffi-cli/src/targets/c/mod.rs | 116 ++ .../src/targets/c/package.rs | 17 +- .../src/targets/c/runtime/buffer_runtime.h | 0 .../weaveffi-cli/src/targets/cpp/callbacks.rs | 329 ++++ crates/weaveffi-cli/src/targets/cpp/calls.rs | 730 +++++++ crates/weaveffi-cli/src/targets/cpp/codec.rs | 366 ++++ .../weaveffi-cli/src/targets/cpp/entities.rs | 633 ++++++ crates/weaveffi-cli/src/targets/cpp/mod.rs | 361 ++++ .../src/targets/cpp/package.rs | 14 +- .../weaveffi-cli/src/targets/cpp/runtime.rs | 97 + .../src/targets/cpp/runtime/buffer.hpp | 53 +- .../src/targets/cpp/runtime/prelude.hpp | 131 +- crates/weaveffi-cli/src/targets/cpp/tests.rs | 452 +++++ crates/weaveffi-cli/src/targets/cpp/types.rs | 186 ++ .../src/targets/dart/callbacks.rs | 303 ++- .../src/targets/dart/calls.rs | 364 ++-- crates/weaveffi-cli/src/targets/dart/codec.rs | 180 ++ .../src/targets/dart/docs.rs | 3 +- .../weaveffi-cli/src/targets/dart/entities.rs | 544 ++++++ crates/weaveffi-cli/src/targets/dart/mod.rs | 244 +++ .../src/targets/dart/package.rs | 38 +- .../weaveffi-cli/src/targets/dart/runtime.rs | 174 ++ .../src/targets/dart/runtime/async.dart | 4 +- .../src/targets/dart/runtime/bundled.dart | 32 + .../src/targets/dart/runtime/callbacks.dart | 67 +- .../src/targets/dart/runtime/cancel.dart | 0 .../src/targets/dart/runtime/codec.dart | 69 + .../src/targets/dart/runtime/core.dart | 55 +- .../src/targets/dart/runtime/iterator.dart | 0 .../src/targets/dart/runtime/loader.dart | 121 ++ .../src/targets/dart/runtime/object.dart | 0 crates/weaveffi-cli/src/targets/dart/tests.rs | 559 ++++++ .../src/targets/dart/types.rs | 95 +- .../src/targets/dotnet/callbacks.rs | 284 +++ .../src/targets/dotnet/calls.rs | 136 +- .../weaveffi-cli/src/targets/dotnet/codec.rs | 252 +++ .../src/targets/dotnet/docs.rs | 0 .../src/targets/dotnet/entities.rs | 37 +- .../src/targets/dotnet/mod.rs | 168 +- .../src/targets/dotnet/package.rs | 12 +- .../src/targets/dotnet/pinvoke.rs | 45 +- .../src/targets/dotnet/runtime.rs | 235 +++ .../src/targets/dotnet/runtime/Runtime.cs | 229 ++- .../weaveffi-cli/src/targets/dotnet/tests.rs | 693 +++++++ .../src/targets/dotnet/types.rs | 125 +- .../weaveffi-cli/src/targets/go/callbacks.rs | 343 ++++ crates/weaveffi-cli/src/targets/go/calls.rs | 544 ++++++ crates/weaveffi-cli/src/targets/go/codec.rs | 228 +++ crates/weaveffi-cli/src/targets/go/docs.rs | 361 ++++ .../weaveffi-cli/src/targets/go/entities.rs | 528 +++++ crates/weaveffi-cli/src/targets/go/mod.rs | 327 ++++ crates/weaveffi-cli/src/targets/go/names.rs | 324 ++++ .../src/targets/go/package.rs | 75 +- .../src/targets/go/runtime.rs | 23 +- .../src/targets/go/runtime/codec.go | 115 +- .../src/targets/go/runtime/runtime.go | 417 ++++ crates/weaveffi-cli/src/targets/go/tests.rs | 374 ++++ crates/weaveffi-cli/src/targets/go/types.rs | 139 ++ .../src/targets/js/api.rs | 343 ++-- crates/weaveffi-cli/src/targets/js/codec.rs | 396 ++++ .../src/targets/js/dts.rs | 135 +- .../src/targets/js/mod.rs | 245 ++- .../src/targets/js/names.rs | 169 +- .../src/targets/js/runtime/runtime.js | 106 +- .../weaveffi-cli/src/targets/kotlin/bridge.rs | 302 +++ .../src/targets/kotlin/calls.rs | 149 +- .../weaveffi-cli/src/targets/kotlin/codec.rs | 198 ++ .../weaveffi-cli/src/targets/kotlin/docs.rs | 49 + .../src/targets/kotlin/entities.rs | 145 +- .../src/targets/kotlin/jni.rs | 372 ++-- .../src/targets/kotlin/mod.rs | 222 ++- .../src/targets/kotlin/names.rs | 240 ++- .../src/targets/kotlin/package.rs | 99 +- .../src/targets/kotlin/runtime.rs | 23 +- .../src/targets/kotlin/runtime/Async.kt | 0 .../src/targets/kotlin/runtime/Buffers.kt | 20 +- .../src/targets/kotlin/runtime/Runtime.kt | 25 +- .../src/targets/kotlin/runtime/jni_async.c | 9 +- .../kotlin/runtime/jni_callback_runs.c | 20 + .../targets/kotlin/runtime/jni_callbacks.c | 74 + .../src/targets/kotlin/runtime/jni_core.c | 52 +- .../src/targets/kotlin/runtime/jni_threads.c | 86 + .../weaveffi-cli/src/targets/kotlin/tests.rs | 599 ++++++ .../src/targets/mod.rs | 8 +- .../src/targets/node/addon.rs | 299 +-- crates/weaveffi-cli/src/targets/node/mod.rs | 206 ++ .../src/targets/node/package.rs | 84 +- .../src/targets/node/runtime/addon.c | 349 +++- .../src/targets/node/runtime/loader.js | 34 + crates/weaveffi-cli/src/targets/node/tests.rs | 263 +++ .../src/targets/python/callbacks.rs | 284 +++ .../src/targets/python/calls.rs | 544 ++---- .../weaveffi-cli/src/targets/python/codec.rs | 354 ++++ .../weaveffi-cli/src/targets/python/docs.rs | 121 ++ .../src/targets/python/entities.rs | 433 +++++ crates/weaveffi-cli/src/targets/python/mod.rs | 326 ++++ .../src/targets/python/package.rs | 104 +- .../src/targets/python/runtime.rs | 92 + .../src/targets/python/runtime/aio.py | 6 +- .../src/targets/python/runtime/callbacks.py | 99 + .../src/targets/python/runtime/cancel.py | 2 +- .../src/targets/python/runtime/core.py | 173 +- .../weaveffi-cli/src/targets/python/tests.rs | 582 ++++++ .../src/targets/python/types.rs | 138 +- .../src/targets/ruby/callbacks.rs | 372 ++++ .../src/targets/ruby/calls.rs | 594 +++--- crates/weaveffi-cli/src/targets/ruby/codec.rs | 475 +++++ crates/weaveffi-cli/src/targets/ruby/docs.rs | 41 + .../weaveffi-cli/src/targets/ruby/entities.rs | 431 +++++ crates/weaveffi-cli/src/targets/ruby/mod.rs | 305 +++ .../src/targets/ruby/package.rs | 68 +- .../src/targets/ruby/runtime.rs | 9 +- .../src/targets/ruby/runtime/runtime.rb | 375 +++- crates/weaveffi-cli/src/targets/ruby/tests.rs | 595 ++++++ crates/weaveffi-cli/src/targets/ruby/types.rs | 146 ++ .../src/targets/swift/callbacks.rs | 299 +++ .../src/targets/swift/calls.rs | 223 +-- .../weaveffi-cli/src/targets/swift/codec.rs | 149 ++ crates/weaveffi-cli/src/targets/swift/docs.rs | 47 + .../src/targets/swift/entities.rs | 476 +++++ crates/weaveffi-cli/src/targets/swift/mod.rs | 279 +++ .../weaveffi-cli/src/targets/swift/package.rs | 104 + .../weaveffi-cli/src/targets/swift/runtime.rs | 140 ++ .../src/targets/swift/runtime/Package.swift | 10 +- .../src/targets/swift/runtime/README.md | 28 + .../src/targets/swift/runtime/Runtime.swift | 296 ++- .../targets/swift/runtime/module.modulemap | 0 .../weaveffi-cli/src/targets/swift/tests.rs | 429 +++++ .../src/targets/swift/types.rs | 134 +- .../src/targets/wasm/glue.rs | 190 +- .../src/targets/wasm/mod.rs | 206 +- .../src/targets/wasm/package.rs | 12 +- .../targets/wasm/runtime/linear.js} | 179 +- .../src/targets/wasm/tests.rs | 111 +- crates/weaveffi-cli/src/utils.rs | 103 + crates/weaveffi-cli/tests/abi_version.rs | 9 + .../tests/c_buffer.rs | 50 +- crates/weaveffi-cli/tests/cli/check.rs | 56 +- crates/weaveffi-cli/tests/cli/determinism.rs | 17 +- crates/weaveffi-cli/tests/cli/diagnostics.rs | 6 +- crates/weaveffi-cli/tests/cli/diff.rs | 6 +- crates/weaveffi-cli/tests/cli/errors.rs | 10 +- crates/weaveffi-cli/tests/cli/extract.rs | 800 ++------ .../tests/cli/extract_roundtrip.rs | 322 ---- crates/weaveffi-cli/tests/cli/formats.rs | 27 +- crates/weaveffi-cli/tests/cli/generate.rs | 52 +- crates/weaveffi-cli/tests/cli/main.rs | 1 - .../weaveffi-cli/tests/cli/no_silent_stubs.rs | 47 +- crates/weaveffi-cli/tests/cli/package.rs | 319 ++- .../weaveffi-cli/tests/cli/project_config.rs | 51 +- .../tests/fixtures/docs_everywhere.yml | 4 +- .../tests/fixtures/edge_cases.yml | 28 +- .../tests/fixtures/kitchen_sink.yml | 20 +- .../tests/fixtures/kitchen_sink_annotated.rs | 248 --- .../tests/fixtures/nested_modules.yml | 8 +- .../tests/fixtures/producer/Cargo.toml | 20 + .../tests/fixtures/producer/expected.yml | 102 + .../tests/fixtures/producer/src/lib.rs | 177 ++ crates/weaveffi-cli/tests/fixtures/shapes.yml | 2 +- crates/weaveffi-cli/tests/snapshots.rs | 164 +- ..._everywhere__docs_everywhere_buffer_h.snap | 4 +- .../c_docs_everywhere__docs_everywhere_h.snap | 87 +- .../c_edge_cases__edge_cases_buffer_h.snap | 20 +- .../snapshots/c_edge_cases__edge_cases_h.snap | 134 +- ...c_kitchen_sink__kitchen_sink_buffer_h.snap | 8 +- .../c_kitchen_sink__kitchen_sink_h.snap | 147 +- ...sted_modules__nested_modules_buffer_h.snap | 4 +- .../c_nested_modules__nested_modules_h.snap | 76 +- .../snapshots/c_shapes__shapes_buffer_h.snap | 4 +- .../tests/snapshots/c_shapes__shapes_h.snap | 74 +- .../cpp_docs_everywhere__CMakeLists_txt.snap | 26 - .../cpp_docs_everywhere__README_md.snap | 43 - ...pp_docs_everywhere__docs_everywhere_h.snap | 196 -- ..._docs_everywhere__docs_everywhere_hpp.snap | 438 +++-- .../cpp_edge_cases__CMakeLists_txt.snap | 26 - .../snapshots/cpp_edge_cases__README_md.snap | 43 - .../cpp_edge_cases__edge_cases_h.snap | 245 --- .../cpp_edge_cases__edge_cases_hpp.snap | 1637 ++++++++++------ .../cpp_kitchen_sink__CMakeLists_txt.snap | 4 +- .../cpp_kitchen_sink__README_md.snap | 4 +- .../cpp_kitchen_sink__kitchen_sink_h.snap | 219 --- .../cpp_kitchen_sink__kitchen_sink_hpp.snap | 812 +++++--- .../cpp_nested_modules__CMakeLists_txt.snap | 26 - .../cpp_nested_modules__README_md.snap | 43 - .../cpp_nested_modules__nested_modules_h.snap | 144 -- ...pp_nested_modules__nested_modules_hpp.snap | 317 ++- .../snapshots/cpp_shapes__CMakeLists_txt.snap | 26 - .../snapshots/cpp_shapes__README_md.snap | 43 - .../tests/snapshots/cpp_shapes__shapes_h.snap | 151 -- .../snapshots/cpp_shapes__shapes_hpp.snap | 376 ++-- .../dart_docs_everywhere__README_md.snap | 39 - ..._everywhere__lib_docs_everywhere_dart.snap | 460 +++-- .../dart_docs_everywhere__pubspec_yaml.snap | 18 - .../snapshots/dart_edge_cases__README_md.snap | 39 - .../dart_edge_cases__lib_edge_cases_dart.snap | 1261 ++++++++---- .../dart_edge_cases__pubspec_yaml.snap | 18 - .../dart_kitchen_sink__README_md.snap | 13 +- ...t_kitchen_sink__lib_kitchen_sink_dart.snap | 661 +++++-- .../dart_kitchen_sink__pubspec_yaml.snap | 6 +- .../dart_nested_modules__README_md.snap | 39 - ...sted_modules__lib_nested_modules_dart.snap | 355 +++- .../dart_nested_modules__pubspec_yaml.snap | 18 - .../snapshots/dart_shapes__README_md.snap | 39 - .../dart_shapes__lib_shapes_dart.snap | 412 +++- .../snapshots/dart_shapes__pubspec_yaml.snap | 18 - ...et_docs_everywhere__DocsEverywhere_cs.snap | 221 ++- ...ocs_everywhere__DocsEverywhere_csproj.snap | 26 - .../dotnet_docs_everywhere__README_md.snap | 32 - .../dotnet_docs_everywhere__Runtime_cs.snap | 698 ------- .../dotnet_edge_cases__EdgeCases_cs.snap | 1255 +++++++----- .../dotnet_edge_cases__EdgeCases_csproj.snap | 26 - .../dotnet_edge_cases__README_md.snap | 32 - .../dotnet_edge_cases__Runtime_cs.snap | 698 ------- .../dotnet_kitchen_sink__KitchenSink_cs.snap | 577 ++++-- ...tnet_kitchen_sink__KitchenSink_csproj.snap | 4 +- .../dotnet_kitchen_sink__README_md.snap | 4 +- .../dotnet_kitchen_sink__Runtime_cs.snap | 235 ++- ...tnet_nested_modules__NestedModules_cs.snap | 114 +- ..._nested_modules__NestedModules_csproj.snap | 26 - .../dotnet_nested_modules__README_md.snap | 32 - .../dotnet_nested_modules__Runtime_cs.snap | 698 ------- .../snapshots/dotnet_shapes__README_md.snap | 32 - .../snapshots/dotnet_shapes__Runtime_cs.snap | 698 ------- .../snapshots/dotnet_shapes__Shapes_cs.snap | 100 +- .../dotnet_shapes__Shapes_csproj.snap | 26 - .../go_docs_everywhere__README_md.snap | 49 - .../go_docs_everywhere__bindings_go.snap | 308 +-- .../go_docs_everywhere__codec_go.snap | 205 -- ...go_docs_everywhere__docs_everywhere_h.snap | 196 -- .../snapshots/go_docs_everywhere__go_mod.snap | 12 - .../go_docs_everywhere__runtime_go.snap | 305 --- .../snapshots/go_edge_cases__README_md.snap | 49 - .../snapshots/go_edge_cases__bindings_go.snap | 1207 ++++++------ .../snapshots/go_edge_cases__codec_go.snap | 205 -- .../go_edge_cases__edge_cases_h.snap | 245 --- .../snapshots/go_edge_cases__go_mod.snap | 12 - .../snapshots/go_edge_cases__runtime_go.snap | 305 --- .../snapshots/go_kitchen_sink__README_md.snap | 22 +- .../go_kitchen_sink__bindings_go.snap | 591 +++--- .../snapshots/go_kitchen_sink__codec_go.snap | 119 +- .../snapshots/go_kitchen_sink__go_mod.snap | 4 +- .../go_kitchen_sink__kitchen_sink_h.snap | 219 --- .../go_kitchen_sink__runtime_go.snap | 227 ++- .../go_nested_modules__README_md.snap | 49 - .../go_nested_modules__bindings_go.snap | 157 +- .../go_nested_modules__codec_go.snap | 205 -- .../snapshots/go_nested_modules__go_mod.snap | 12 - .../go_nested_modules__nested_modules_h.snap | 144 -- .../go_nested_modules__runtime_go.snap | 305 --- .../tests/snapshots/go_shapes__README_md.snap | 49 - .../snapshots/go_shapes__bindings_go.snap | 65 +- .../tests/snapshots/go_shapes__codec_go.snap | 205 -- .../tests/snapshots/go_shapes__go_mod.snap | 12 - .../snapshots/go_shapes__runtime_go.snap | 305 --- .../tests/snapshots/go_shapes__shapes_h.snap | 151 -- ...lin_docs_everywhere__build_gradle_kts.snap | 63 - ...n_docs_everywhere__consumer_rules_pro.snap | 11 - ..._docs_everywhere__settings_gradle_kts.snap | 25 - ...erywhere__src_main_cpp_CMakeLists_txt.snap | 34 - ...where__src_main_cpp_docs_everywhere_h.snap | 196 -- ...e__src_main_cpp_docs_everywhere_jni_c.snap | 257 ++- ..._main_kotlin_docs_everywhere_Async_kt.snap | 124 -- ...ain_kotlin_docs_everywhere_Buffers_kt.snap | 255 --- ...main_kotlin_docs_everywhere_Codecs_kt.snap | 18 + ...c_main_kotlin_docs_everywhere_Docs_kt.snap | 17 +- ...n_kotlin_docs_everywhere_JniBridge_kt.snap | 41 +- ...ain_kotlin_docs_everywhere_Runtime_kt.snap | 212 -- .../kotlin_edge_cases__build_gradle_kts.snap | 63 - ...kotlin_edge_cases__consumer_rules_pro.snap | 11 - ...otlin_edge_cases__settings_gradle_kts.snap | 25 - ...ge_cases__src_main_cpp_CMakeLists_txt.snap | 34 - ...edge_cases__src_main_cpp_edge_cases_h.snap | 245 --- ..._cases__src_main_cpp_edge_cases_jni_c.snap | 436 +++-- ...__src_main_kotlin_edge_cases_Async_kt.snap | 124 -- ...src_main_kotlin_edge_cases_Buffers_kt.snap | 255 --- ..._src_main_kotlin_edge_cases_Codecs_kt.snap | 104 + ...c_main_kotlin_edge_cases_EdgeCases_kt.snap | 106 +- ...c_main_kotlin_edge_cases_JniBridge_kt.snap | 94 +- ...src_main_kotlin_edge_cases_Runtime_kt.snap | 212 -- ...kotlin_kitchen_sink__build_gradle_kts.snap | 16 +- ...tlin_kitchen_sink__consumer_rules_pro.snap | 4 +- ...lin_kitchen_sink__settings_gradle_kts.snap | 4 +- ...hen_sink__src_main_cpp_CMakeLists_txt.snap | 4 +- ...hen_sink__src_main_cpp_kitchen_sink_h.snap | 219 --- ...sink__src_main_cpp_kitchen_sink_jni_c.snap | 379 +++- ...src_main_kotlin_kitchen_sink_Async_kt.snap | 4 +- ...c_main_kotlin_kitchen_sink_Buffers_kt.snap | 24 +- ...rc_main_kotlin_kitchen_sink_Codecs_kt.snap | 42 + ...main_kotlin_kitchen_sink_JniBridge_kt.snap | 78 +- ...c_main_kotlin_kitchen_sink_Kitchen_kt.snap | 54 +- ...c_main_kotlin_kitchen_sink_Runtime_kt.snap | 29 +- ...rc_main_kotlin_kitchen_sink_Shared_kt.snap | 4 +- ...tlin_nested_modules__build_gradle_kts.snap | 59 - ...in_nested_modules__consumer_rules_pro.snap | 11 - ...n_nested_modules__settings_gradle_kts.snap | 25 - ..._modules__src_main_cpp_CMakeLists_txt.snap | 34 - ...odules__src_main_cpp_nested_modules_h.snap | 144 -- ...es__src_main_cpp_nested_modules_jni_c.snap | 76 +- ...main_kotlin_nested_modules_Buffers_kt.snap | 255 --- ...ain_kotlin_nested_modules_Graphics_kt.snap | 6 +- ...in_kotlin_nested_modules_JniBridge_kt.snap | 14 +- ...main_kotlin_nested_modules_Runtime_kt.snap | 212 -- .../kotlin_shapes__build_gradle_kts.snap | 59 - .../kotlin_shapes__consumer_rules_pro.snap | 11 - .../kotlin_shapes__settings_gradle_kts.snap | 25 - ...n_shapes__src_main_cpp_CMakeLists_txt.snap | 34 - .../kotlin_shapes__src_main_cpp_shapes_h.snap | 151 -- ...lin_shapes__src_main_cpp_shapes_jni_c.snap | 76 +- ...es__src_main_kotlin_shapes_Buffers_kt.snap | 255 --- ...__src_main_kotlin_shapes_JniBridge_kt.snap | 16 +- ...es__src_main_kotlin_shapes_Runtime_kt.snap | 212 -- ...pes__src_main_kotlin_shapes_Shapes_kt.snap | 18 +- .../node_docs_everywhere__README_md.snap | 24 - .../node_docs_everywhere__binding_gyp.snap | 35 - ...de_docs_everywhere__docs_everywhere_h.snap | 196 -- ...cs_everywhere__docs_everywhere_node_c.snap | 409 +++- .../node_docs_everywhere__index_d_ts.snap | 16 +- .../node_docs_everywhere__index_js.snap | 91 +- .../node_docs_everywhere__package_json.snap | 35 - .../node_docs_everywhere__runtime_js.snap | 578 ------ .../snapshots/node_edge_cases__README_md.snap | 24 - .../node_edge_cases__binding_gyp.snap | 35 - .../node_edge_cases__edge_cases_h.snap | 245 --- .../node_edge_cases__edge_cases_node_c.snap | 654 +++++-- .../node_edge_cases__index_d_ts.snap | 18 +- .../snapshots/node_edge_cases__index_js.snap | 375 +++- .../node_edge_cases__package_json.snap | 35 - .../node_edge_cases__runtime_js.snap | 578 ------ .../node_kitchen_sink__README_md.snap | 6 +- .../node_kitchen_sink__binding_gyp.snap | 4 +- .../node_kitchen_sink__index_d_ts.snap | 23 +- .../node_kitchen_sink__index_js.snap | 202 +- .../node_kitchen_sink__kitchen_sink_h.snap | 219 --- ...ode_kitchen_sink__kitchen_sink_node_c.snap | 607 +++++- .../node_kitchen_sink__package_json.snap | 6 +- .../node_kitchen_sink__runtime_js.snap | 110 +- .../node_nested_modules__README_md.snap | 24 - .../node_nested_modules__binding_gyp.snap | 35 - .../node_nested_modules__index_d_ts.snap | 12 +- .../node_nested_modules__index_js.snap | 63 +- ...node_nested_modules__nested_modules_h.snap | 144 -- ...nested_modules__nested_modules_node_c.snap | 372 +++- .../node_nested_modules__package_json.snap | 35 - .../node_nested_modules__runtime_js.snap | 578 ------ .../snapshots/node_shapes__README_md.snap | 24 - .../snapshots/node_shapes__binding_gyp.snap | 35 - .../snapshots/node_shapes__index_d_ts.snap | 9 +- .../snapshots/node_shapes__index_js.snap | 54 +- .../snapshots/node_shapes__package_json.snap | 35 - .../snapshots/node_shapes__runtime_js.snap | 578 ------ .../snapshots/node_shapes__shapes_h.snap | 151 -- .../snapshots/node_shapes__shapes_node_c.snap | 372 +++- .../python_docs_everywhere__README_md.snap | 35 - ...erywhere__docs_everywhere___init___py.snap | 10 - ...e__docs_everywhere_docs_everywhere_py.snap | 470 +++-- ...__docs_everywhere_docs_everywhere_pyi.snap | 100 - ..._everywhere__docs_everywhere_py_typed.snap | 6 - ...ython_docs_everywhere__pyproject_toml.snap | 24 - .../python_edge_cases__README_md.snap | 35 - ...on_edge_cases__edge_cases___init___py.snap | 10 - ..._edge_cases__edge_cases_edge_cases_py.snap | 1169 +++++++---- ...edge_cases__edge_cases_edge_cases_pyi.snap | 246 --- ...ython_edge_cases__edge_cases_py_typed.snap | 6 - .../python_edge_cases__pyproject_toml.snap | 24 - .../python_kitchen_sink__README_md.snap | 4 +- ...itchen_sink__kitchen_sink___init___py.snap | 4 +- ...en_sink__kitchen_sink_kitchen_sink_py.snap | 747 +++++-- ...n_sink__kitchen_sink_kitchen_sink_pyi.snap | 155 -- ...n_kitchen_sink__kitchen_sink_py_typed.snap | 4 +- .../python_kitchen_sink__pyproject_toml.snap | 6 +- .../python_nested_modules__README_md.snap | 35 - ...d_modules__nested_modules___init___py.snap | 10 - ...les__nested_modules_nested_modules_py.snap | 271 ++- ...es__nested_modules_nested_modules_pyi.snap | 69 - ...sted_modules__nested_modules_py_typed.snap | 6 - ...python_nested_modules__pyproject_toml.snap | 24 - .../snapshots/python_shapes__README_md.snap | 35 - .../python_shapes__pyproject_toml.snap | 24 - .../python_shapes__shapes___init___py.snap | 10 - .../python_shapes__shapes_py_typed.snap | 6 - .../python_shapes__shapes_shapes_py.snap | 262 ++- .../python_shapes__shapes_shapes_pyi.snap | 110 -- .../ruby_docs_everywhere__README_md.snap | 33 - ...s_everywhere__docs_everywhere_gemspec.snap | 20 - ...cs_everywhere__lib_docs_everywhere_rb.snap | 263 +-- ...where__lib_docs_everywhere_runtime_rb.snap | 705 ------- .../snapshots/ruby_edge_cases__README_md.snap | 33 - .../ruby_edge_cases__edge_cases_gemspec.snap | 20 - .../ruby_edge_cases__lib_edge_cases_rb.snap | 1661 +++++++++------- ...edge_cases__lib_edge_cases_runtime_rb.snap | 705 ------- .../ruby_kitchen_sink__README_md.snap | 4 +- ...by_kitchen_sink__kitchen_sink_gemspec.snap | 4 +- ...uby_kitchen_sink__lib_kitchen_sink_rb.snap | 648 ++++--- ...hen_sink__lib_kitchen_sink_runtime_rb.snap | 381 +++- .../ruby_nested_modules__README_md.snap | 33 - ...nested_modules__lib_nested_modules_rb.snap | 136 +- ...odules__lib_nested_modules_runtime_rb.snap | 705 ------- ...ested_modules__nested_modules_gemspec.snap | 20 - .../snapshots/ruby_shapes__README_md.snap | 33 - .../snapshots/ruby_shapes__lib_shapes_rb.snap | 164 +- .../ruby_shapes__lib_shapes_runtime_rb.snap | 705 ------- .../ruby_shapes__shapes_gemspec.snap | 20 - .../swift_docs_everywhere__Package_swift.snap | 34 - ...ces_CDocsEverywhere_docs_everywhere_h.snap | 196 -- ...rces_CDocsEverywhere_module_modulemap.snap | 14 - ...s_DocsEverywhere_DocsEverywhere_swift.snap | 449 ++++- .../swift_edge_cases__Package_swift.snap | 34 - ...ases__Sources_CEdgeCases_edge_cases_h.snap | 245 --- ...__Sources_CEdgeCases_module_modulemap.snap | 14 - ...es__Sources_EdgeCases_EdgeCases_swift.snap | 1239 ++++++------ .../swift_kitchen_sink__Package_swift.snap | 14 +- ...__Sources_CKitchenSink_kitchen_sink_h.snap | 219 --- ...Sources_CKitchenSink_module_modulemap.snap | 4 +- ...Sources_KitchenSink_KitchenSink_swift.snap | 638 ++++-- .../swift_nested_modules__Package_swift.snap | 34 - ...urces_CNestedModules_module_modulemap.snap | 14 - ...urces_CNestedModules_nested_modules_h.snap | 144 -- ...ces_NestedModules_NestedModules_swift.snap | 394 +++- .../swift_shapes__Package_swift.snap | 34 - ...pes__Sources_CShapes_module_modulemap.snap | 14 - ...wift_shapes__Sources_CShapes_shapes_h.snap | 151 -- ...t_shapes__Sources_Shapes_Shapes_swift.snap | 455 +++-- .../wasm_docs_everywhere__README_md.snap | 20 - .../wasm_docs_everywhere__index_d_ts.snap | 16 +- .../wasm_docs_everywhere__index_js.snap | 72 +- .../wasm_docs_everywhere__linear_js.snap | 403 ---- .../wasm_docs_everywhere__package_json.snap | 26 - .../wasm_docs_everywhere__runtime_js.snap | 578 ------ .../snapshots/wasm_edge_cases__README_md.snap | 20 - .../wasm_edge_cases__index_d_ts.snap | 18 +- .../snapshots/wasm_edge_cases__index_js.snap | 375 +++- .../wasm_edge_cases__package_json.snap | 26 - .../wasm_edge_cases__runtime_js.snap | 578 ------ .../wasm_kitchen_sink__README_md.snap | 4 +- .../wasm_kitchen_sink__index_d_ts.snap | 23 +- .../wasm_kitchen_sink__index_js.snap | 212 +- .../wasm_kitchen_sink__linear_js.snap | 174 +- .../wasm_kitchen_sink__package_json.snap | 4 +- .../wasm_kitchen_sink__runtime_js.snap | 110 +- .../wasm_nested_modules__README_md.snap | 20 - .../wasm_nested_modules__index_d_ts.snap | 12 +- .../wasm_nested_modules__index_js.snap | 44 +- .../wasm_nested_modules__linear_js.snap | 403 ---- .../wasm_nested_modules__package_json.snap | 26 - .../wasm_nested_modules__runtime_js.snap | 578 ------ .../snapshots/wasm_shapes__README_md.snap | 20 - .../snapshots/wasm_shapes__index_d_ts.snap | 9 +- .../snapshots/wasm_shapes__index_js.snap | 35 +- .../snapshots/wasm_shapes__linear_js.snap | 403 ---- .../snapshots/wasm_shapes__package_json.snap | 26 - .../snapshots/wasm_shapes__runtime_js.snap | 578 ------ crates/weaveffi-fuzz/Cargo.toml | 8 + crates/weaveffi-fuzz/README.md | 36 +- .../fuzz/seeds/fuzz_parse_json/minimal.json | 2 +- .../fuzz/seeds/fuzz_parse_toml/minimal.toml | 2 +- .../fuzz/seeds/fuzz_parse_yaml/minimal.yml | 2 +- .../fuzz/seeds/fuzz_validate/minimal.yml | 2 +- .../fuzz/seeds/fuzz_value_buffer/f64_list.bin | Bin 0 -> 21 bytes .../fuzz_value_buffer/nested_bytes_list.bin | Bin 0 -> 17 bytes .../optional_string_list.bin | Bin 0 -> 12 bytes .../fuzz/seeds/fuzz_value_buffer/string.bin | Bin 0 -> 10 bytes .../seeds/fuzz_value_buffer/string_map.bin | Bin 0 -> 20 bytes .../fuzz_targets/fuzz_validate.rs | 6 +- .../fuzz_targets/value_buffer.rs | 67 + crates/weaveffi-fuzz/seed_corpus.py | 90 + crates/weaveffi-fuzz/tests/seeds.rs | 99 +- crates/weaveffi-gen/Cargo.toml | 33 - crates/weaveffi-gen/src/backend/mod.rs | 379 ---- crates/weaveffi-gen/src/cache.rs | 213 -- crates/weaveffi-gen/src/capabilities.rs | 333 ---- crates/weaveffi-gen/src/codegen/mod.rs | 562 ------ crates/weaveffi-gen/src/lib.rs | 23 - crates/weaveffi-gen/src/package.rs | 219 --- crates/weaveffi-gen/src/targets/c/mod.rs | 155 -- .../weaveffi-gen/src/targets/cpp/callbacks.rs | 282 --- crates/weaveffi-gen/src/targets/cpp/calls.rs | 882 --------- crates/weaveffi-gen/src/targets/cpp/codec.rs | 302 --- .../weaveffi-gen/src/targets/cpp/entities.rs | 545 ------ crates/weaveffi-gen/src/targets/cpp/mod.rs | 465 ----- .../weaveffi-gen/src/targets/cpp/runtime.rs | 46 - crates/weaveffi-gen/src/targets/cpp/tests.rs | 755 -------- crates/weaveffi-gen/src/targets/cpp/types.rs | 170 -- crates/weaveffi-gen/src/targets/dart/codec.rs | 113 -- .../weaveffi-gen/src/targets/dart/entities.rs | 451 ----- crates/weaveffi-gen/src/targets/dart/mod.rs | 231 --- .../weaveffi-gen/src/targets/dart/runtime.rs | 133 -- .../src/targets/dart/runtime/loader.dart | 74 - .../src/targets/dotnet/callbacks.rs | 193 -- .../weaveffi-gen/src/targets/dotnet/codec.rs | 146 -- .../src/targets/dotnet/runtime.rs | 156 -- .../weaveffi-gen/src/targets/dotnet/tests.rs | 506 ----- crates/weaveffi-gen/src/targets/go/calls.rs | 967 ---------- crates/weaveffi-gen/src/targets/go/codec.rs | 153 -- crates/weaveffi-gen/src/targets/go/docs.rs | 104 - .../weaveffi-gen/src/targets/go/entities.rs | 578 ------ crates/weaveffi-gen/src/targets/go/mod.rs | 884 --------- .../src/targets/go/runtime/runtime.go | 296 --- crates/weaveffi-gen/src/targets/go/types.rs | 213 -- crates/weaveffi-gen/src/targets/js/codec.rs | 200 -- .../weaveffi-gen/src/targets/kotlin/bridge.rs | 206 -- .../weaveffi-gen/src/targets/kotlin/codec.rs | 88 - .../weaveffi-gen/src/targets/kotlin/docs.rs | 98 - .../targets/kotlin/runtime/jni_callbacks.c | 63 - .../src/targets/kotlin/runtime/jni_threads.c | 49 - .../weaveffi-gen/src/targets/kotlin/tests.rs | 384 ---- crates/weaveffi-gen/src/targets/node/mod.rs | 243 --- .../src/targets/node/runtime/loader.js | 26 - crates/weaveffi-gen/src/targets/node/tests.rs | 180 -- .../weaveffi-gen/src/targets/python/codec.rs | 272 --- .../weaveffi-gen/src/targets/python/docs.rs | 140 -- .../src/targets/python/entities.rs | 460 ----- crates/weaveffi-gen/src/targets/python/mod.rs | 313 --- .../src/targets/python/runtime.rs | 77 - .../src/targets/python/runtime/callbacks.py | 42 - .../weaveffi-gen/src/targets/python/stubs.rs | 298 --- .../weaveffi-gen/src/targets/python/tests.rs | 461 ----- .../src/targets/ruby/callbacks.rs | 326 ---- crates/weaveffi-gen/src/targets/ruby/codec.rs | 287 --- crates/weaveffi-gen/src/targets/ruby/docs.rs | 41 - .../weaveffi-gen/src/targets/ruby/entities.rs | 443 ----- crates/weaveffi-gen/src/targets/ruby/mod.rs | 748 ------- crates/weaveffi-gen/src/targets/ruby/types.rs | 90 - .../src/targets/swift/callbacks.rs | 276 --- .../weaveffi-gen/src/targets/swift/codec.rs | 190 -- crates/weaveffi-gen/src/targets/swift/docs.rs | 51 - .../src/targets/swift/entities.rs | 551 ------ crates/weaveffi-gen/src/targets/swift/mod.rs | 564 ------ .../weaveffi-gen/src/targets/swift/package.rs | 59 - .../weaveffi-gen/src/targets/swift/runtime.rs | 108 -- .../src/targets/swift/runtime/README.md | 25 - .../src/targets/wasm/runtime/linear.js | 394 ---- crates/weaveffi-gen/src/utils.rs | 213 -- crates/weaveffi-gen/tests/dart_render.rs | 457 ----- .../weaveffi-macros/src/codegen/async_fns.rs | 374 +--- .../weaveffi-macros/src/codegen/callbacks.rs | 269 +-- .../weaveffi-macros/src/codegen/contract.rs | 63 + .../src/codegen/diagnostics.rs | 144 ++ crates/weaveffi-macros/src/codegen/enums.rs | 153 +- crates/weaveffi-macros/src/codegen/helpers.rs | 235 +-- .../weaveffi-macros/src/codegen/interfaces.rs | 2 +- .../weaveffi-macros/src/codegen/iterators.rs | 115 +- crates/weaveffi-macros/src/codegen/lift.rs | 227 +++ crates/weaveffi-macros/src/codegen/marshal.rs | 305 --- crates/weaveffi-macros/src/codegen/meta.rs | 67 + crates/weaveffi-macros/src/codegen/mod.rs | 478 +++-- crates/weaveffi-macros/src/codegen/records.rs | 27 +- crates/weaveffi-macros/src/codegen/sync.rs | 74 +- crates/weaveffi-macros/src/lib.rs | 47 +- crates/weaveffi-macros/src/runtime.rs | 33 +- .../tests/ui/fail_callback_plain_return.rs | 13 + .../ui/fail_callback_plain_return.stderr | 5 + .../tests/ui/fail_callback_string_return.rs | 9 - .../ui/fail_callback_string_return.stderr | 5 - .../tests/ui/fail_iter_param.stderr | 2 +- .../tests/ui/fail_member_cfg.rs | 14 + .../tests/ui/fail_member_cfg.stderr | 5 + .../tests/ui/fail_nested_iter_return.stderr | 2 +- .../tests/ui/fail_out_of_line_module.rs | 10 + .../tests/ui/fail_out_of_line_module.stderr | 5 + ...g_c_enum.rs => fail_out_of_tree_c_enum.rs} | 0 ....stderr => fail_out_of_tree_c_enum.stderr} | 8 +- .../fail_result_without_error_domain.stderr | 6 +- .../tests/ui/fail_unsupported_primitive.rs | 11 + .../ui/fail_unsupported_primitive.stderr | 5 + .../tests/ui/pass_aliases_and_cfg.rs | 53 + .../tests/ui/pass_async_factory.rs | 58 + .../tests/ui/pass_full_surface.rs | 15 +- .../tests/ui/pass_rich_callbacks.rs | 68 + crates/weaveffi-model/Cargo.toml | 8 +- crates/weaveffi-model/src/abi/ctype.rs | 2 +- crates/weaveffi-model/src/abi/lower.rs | 262 ++- crates/weaveffi-model/src/abi/mod.rs | 191 +- crates/weaveffi-model/src/checksum.rs | 215 --- crates/weaveffi-model/src/contract.rs | 347 ++++ crates/weaveffi-model/src/errors.rs | 2 +- crates/weaveffi-model/src/ir.rs | 208 +- crates/weaveffi-model/src/lib.rs | 34 +- crates/weaveffi-model/src/meta.rs | 744 +++++++ crates/weaveffi-model/src/model/build.rs | 506 +++++ crates/weaveffi-model/src/model/mod.rs | 972 ++++------ crates/weaveffi-model/src/parse.rs | 17 +- crates/weaveffi-model/src/pkg.rs | 17 +- crates/weaveffi-model/src/plan.rs | 345 ++-- crates/weaveffi-model/src/resolved.rs | 431 ----- crates/weaveffi-model/src/rust.rs | 1396 +++++++++----- crates/weaveffi-model/src/{model => }/ty.rs | 450 +++-- crates/weaveffi-model/src/validate.rs | 672 ++++--- .../weaveffi-model/src/validate/diagnostic.rs | 256 ++- crates/weaveffi-model/src/validate/rules.rs | 1405 +++++++------- crates/weaveffi-model/src/validate/tests.rs | 265 ++- crates/weaveffi/Cargo.toml | 27 +- .../src => weaveffi/src/abi}/buffer.rs | 189 +- crates/weaveffi/src/abi/callback.rs | 585 ++++++ .../src => weaveffi/src/abi}/cancel.rs | 200 +- crates/weaveffi/src/abi/contract.rs | 118 ++ .../src => weaveffi/src/abi}/convert.rs | 69 +- .../src => weaveffi/src/abi}/error.rs | 121 +- .../src => weaveffi/src/abi}/iter.rs | 116 +- .../src => weaveffi/src/abi}/leak.rs | 16 +- crates/weaveffi/src/abi/marshal.rs | 520 +++++ crates/weaveffi/src/abi/mod.rs | 84 + .../src => weaveffi/src/abi}/object.rs | 10 +- crates/weaveffi/src/abi/spawn.rs | 709 +++++++ crates/weaveffi/src/lib.rs | 67 +- crates/weaveffi/tests/abi_version.rs | 11 +- .../tests/buffer_proptest.rs | 29 +- crates/weaveffi/tests/dropped_future.rs | 6 +- crates/weaveffi/tests/export_runtime.rs | 33 +- crates/weaveffi/tests/leak.rs | 72 +- crates/weaveffi/tests/runtime.rs | 715 +++++-- crates/weaveffi/tests/spawn_failure.rs | 43 + crates/weaveffi/tests/tokio.rs | 64 + docs/src/SUMMARY.md | 2 +- docs/src/api/README.md | 12 +- docs/src/api/doc-style.md | 35 +- docs/src/api/rust.md | 54 +- docs/src/architecture.md | 239 +-- docs/src/comparison.md | 100 +- docs/src/faq.md | 50 +- docs/src/generators/README.md | 70 +- docs/src/generators/c.md | 138 +- docs/src/generators/cpp.md | 196 +- docs/src/generators/dart.md | 217 ++- docs/src/generators/dotnet.md | 228 ++- docs/src/generators/go.md | 295 ++- docs/src/generators/kotlin.md | 204 +- docs/src/generators/node.md | 193 +- docs/src/generators/python.md | 265 ++- docs/src/generators/ruby.md | 273 ++- docs/src/generators/swift.md | 242 ++- docs/src/generators/wasm.md | 79 +- docs/src/getting-started.md | 141 +- docs/src/guides/async.md | 107 +- docs/src/guides/config.md | 155 +- docs/src/guides/errors-and-memory.md | 155 +- docs/src/guides/extract.md | 214 ++- docs/src/guides/packaging.md | 256 ++- docs/src/guides/producer-macro.md | 355 +++- docs/src/intro.md | 22 +- docs/src/reference/abi.md | 321 +++- docs/src/reference/idl.md | 242 ++- docs/src/reference/naming.md | 119 +- docs/src/reference/value-buffers.md | 21 +- docs/src/roadmap.md | 111 +- docs/src/samples.md | 139 +- docs/src/stability.md | 275 ++- release-plz.toml | 10 +- samples/async-demo/Cargo.toml | 14 - samples/async-demo/src/lib.rs | 449 ----- samples/async-demo/weaveffi.toml | 8 - samples/calculator/src/lib.rs | 83 +- samples/calculator/weaveffi.toml | 2 +- samples/codec/src/lib.rs | 861 +++++++-- samples/codec/weaveffi.toml | 10 +- samples/contacts/Cargo.toml | 14 - samples/contacts/src/lib.rs | 542 ------ samples/contacts/weaveffi.toml | 8 - samples/events/Cargo.toml | 14 - samples/events/src/lib.rs | 434 ----- samples/events/weaveffi.toml | 8 - samples/kvstore/src/lib.rs | 1711 ++++++++--------- samples/kvstore/src/tests.rs | 615 ++++++ samples/kvstore/weaveffi.toml | 6 +- scripts/check-fixtures.sh | 20 +- scripts/fixtures/c.sh | 2 + scripts/fixtures/cpp.sh | 2 + scripts/fixtures/dart.sh | 2 + scripts/fixtures/dotnet.sh | 2 + scripts/fixtures/go.sh | 2 + scripts/fixtures/js.sh | 4 +- scripts/fixtures/kotlin.sh | 6 +- scripts/fixtures/lib.sh | 27 + scripts/fixtures/node.sh | 21 +- scripts/fixtures/python.sh | 19 +- scripts/fixtures/ruby.sh | 2 + scripts/fixtures/swift.sh | 2 + scripts/fixtures/wasm.sh | 2 + scripts/package-smoke.sh | 159 ++ weaveffi.schema.json | 4 +- 814 files changed, 76988 insertions(+), 82157 deletions(-) delete mode 100644 conformance/c/async_demo.c create mode 100644 conformance/c/calculator.c delete mode 100644 conformance/c/contacts.c delete mode 100644 conformance/c/events.c delete mode 100644 conformance/cpp/async_demo.cpp delete mode 100644 conformance/cpp/contacts.cpp delete mode 100644 conformance/cpp/events.cpp delete mode 100644 conformance/dart/async_demo.dart delete mode 100644 conformance/dart/contacts.dart delete mode 100644 conformance/dart/events.dart delete mode 100644 conformance/dotnet/AsyncDemo.cs create mode 100644 conformance/dotnet/Calculator.cs delete mode 100644 conformance/dotnet/Contacts.cs delete mode 100644 conformance/dotnet/Events.cs delete mode 100644 conformance/go/async_demo.go delete mode 100644 conformance/go/contacts.go delete mode 100644 conformance/go/events.go delete mode 100644 conformance/kotlin/async_demo.kt create mode 100644 conformance/kotlin/calculator.kt delete mode 100644 conformance/kotlin/contacts.kt delete mode 100644 conformance/kotlin/events.kt delete mode 100644 conformance/node/async_demo.mjs create mode 100644 conformance/node/calculator.mjs delete mode 100644 conformance/node/contacts.mjs delete mode 100644 conformance/node/events.mjs delete mode 100644 conformance/python/async_demo_consumer.py delete mode 100644 conformance/python/contacts_consumer.py delete mode 100644 conformance/python/events_consumer.py delete mode 100644 conformance/ruby/async_demo.rb delete mode 100644 conformance/ruby/contacts.rb delete mode 100644 conformance/ruby/events.rb delete mode 100644 conformance/swift/async_demo.swift create mode 100644 conformance/swift/calculator.swift delete mode 100644 conformance/swift/contacts.swift delete mode 100644 conformance/swift/events.swift delete mode 100644 crates/weaveffi-abi/Cargo.toml delete mode 100644 crates/weaveffi-abi/src/callback.rs delete mode 100644 crates/weaveffi-abi/src/lib.rs delete mode 100644 crates/weaveffi-abi/src/spawn.rs rename crates/{weaveffi-gen => weaveffi-cli}/benches/codegen_bench.rs (77%) create mode 100644 crates/weaveffi-cli/src/backend/mod.rs create mode 100644 crates/weaveffi-cli/src/build/glue.rs create mode 100644 crates/weaveffi-cli/src/build/mod.rs create mode 100644 crates/weaveffi-cli/src/build/ndk.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/cabi.rs (70%) create mode 100644 crates/weaveffi-cli/src/cache.rs create mode 100644 crates/weaveffi-cli/src/cargo.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/codegen/common.rs (66%) create mode 100644 crates/weaveffi-cli/src/codegen/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/codegen/writer.rs (99%) create mode 100644 crates/weaveffi-cli/src/commands/build.rs create mode 100644 crates/weaveffi-cli/src/commands/dev.rs create mode 100644 crates/weaveffi-cli/src/commands/extract.rs delete mode 100644 crates/weaveffi-cli/src/extract.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/lang.rs (90%) create mode 100644 crates/weaveffi-cli/src/lib.rs create mode 100644 crates/weaveffi-cli/src/library.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/manifest.rs (98%) create mode 100644 crates/weaveffi-cli/src/package/archive.rs create mode 100644 crates/weaveffi-cli/src/package/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/platform.rs (53%) create mode 100644 crates/weaveffi-cli/src/project.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/c/buffer.rs (92%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/c/header.rs (73%) create mode 100644 crates/weaveffi-cli/src/targets/c/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/c/package.rs (85%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/c/runtime/buffer_runtime.h (100%) create mode 100644 crates/weaveffi-cli/src/targets/cpp/callbacks.rs create mode 100644 crates/weaveffi-cli/src/targets/cpp/calls.rs create mode 100644 crates/weaveffi-cli/src/targets/cpp/codec.rs create mode 100644 crates/weaveffi-cli/src/targets/cpp/entities.rs create mode 100644 crates/weaveffi-cli/src/targets/cpp/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/cpp/package.rs (94%) create mode 100644 crates/weaveffi-cli/src/targets/cpp/runtime.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/cpp/runtime/buffer.hpp (76%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/cpp/runtime/prelude.hpp (54%) create mode 100644 crates/weaveffi-cli/src/targets/cpp/tests.rs create mode 100644 crates/weaveffi-cli/src/targets/cpp/types.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/callbacks.rs (52%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/calls.rs (65%) create mode 100644 crates/weaveffi-cli/src/targets/dart/codec.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/docs.rs (97%) create mode 100644 crates/weaveffi-cli/src/targets/dart/entities.rs create mode 100644 crates/weaveffi-cli/src/targets/dart/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/package.rs (74%) create mode 100644 crates/weaveffi-cli/src/targets/dart/runtime.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/runtime/async.dart (77%) create mode 100644 crates/weaveffi-cli/src/targets/dart/runtime/bundled.dart rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/runtime/callbacks.dart (76%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/runtime/cancel.dart (100%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/runtime/codec.dart (68%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/runtime/core.dart (65%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/runtime/iterator.dart (100%) create mode 100644 crates/weaveffi-cli/src/targets/dart/runtime/loader.dart rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/runtime/object.dart (100%) create mode 100644 crates/weaveffi-cli/src/targets/dart/tests.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dart/types.rs (72%) create mode 100644 crates/weaveffi-cli/src/targets/dotnet/callbacks.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/calls.rs (86%) create mode 100644 crates/weaveffi-cli/src/targets/dotnet/codec.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/docs.rs (100%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/entities.rs (90%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/mod.rs (61%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/package.rs (93%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/pinvoke.rs (78%) create mode 100644 crates/weaveffi-cli/src/targets/dotnet/runtime.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/runtime/Runtime.cs (70%) create mode 100644 crates/weaveffi-cli/src/targets/dotnet/tests.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/dotnet/types.rs (62%) create mode 100644 crates/weaveffi-cli/src/targets/go/callbacks.rs create mode 100644 crates/weaveffi-cli/src/targets/go/calls.rs create mode 100644 crates/weaveffi-cli/src/targets/go/codec.rs create mode 100644 crates/weaveffi-cli/src/targets/go/docs.rs create mode 100644 crates/weaveffi-cli/src/targets/go/entities.rs create mode 100644 crates/weaveffi-cli/src/targets/go/mod.rs create mode 100644 crates/weaveffi-cli/src/targets/go/names.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/go/package.rs (71%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/go/runtime.rs (58%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/go/runtime/codec.go (59%) create mode 100644 crates/weaveffi-cli/src/targets/go/runtime/runtime.go create mode 100644 crates/weaveffi-cli/src/targets/go/tests.rs create mode 100644 crates/weaveffi-cli/src/targets/go/types.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/js/api.rs (60%) create mode 100644 crates/weaveffi-cli/src/targets/js/codec.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/js/dts.rs (72%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/js/mod.rs (58%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/js/names.rs (56%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/js/runtime/runtime.js (82%) create mode 100644 crates/weaveffi-cli/src/targets/kotlin/bridge.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/calls.rs (67%) create mode 100644 crates/weaveffi-cli/src/targets/kotlin/codec.rs create mode 100644 crates/weaveffi-cli/src/targets/kotlin/docs.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/entities.rs (76%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/jni.rs (62%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/mod.rs (54%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/names.rs (59%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/package.rs (76%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/runtime.rs (76%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/runtime/Async.kt (100%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/runtime/Buffers.kt (92%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/runtime/Runtime.kt (87%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/runtime/jni_async.c (94%) create mode 100644 crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callback_runs.c create mode 100644 crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callbacks.c rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/kotlin/runtime/jni_core.c (69%) create mode 100644 crates/weaveffi-cli/src/targets/kotlin/runtime/jni_threads.c create mode 100644 crates/weaveffi-cli/src/targets/kotlin/tests.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/mod.rs (87%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/node/addon.rs (72%) create mode 100644 crates/weaveffi-cli/src/targets/node/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/node/package.rs (68%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/node/runtime/addon.c (63%) create mode 100644 crates/weaveffi-cli/src/targets/node/runtime/loader.js create mode 100644 crates/weaveffi-cli/src/targets/node/tests.rs create mode 100644 crates/weaveffi-cli/src/targets/python/callbacks.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/python/calls.rs (50%) create mode 100644 crates/weaveffi-cli/src/targets/python/codec.rs create mode 100644 crates/weaveffi-cli/src/targets/python/docs.rs create mode 100644 crates/weaveffi-cli/src/targets/python/entities.rs create mode 100644 crates/weaveffi-cli/src/targets/python/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/python/package.rs (52%) create mode 100644 crates/weaveffi-cli/src/targets/python/runtime.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/python/runtime/aio.py (90%) create mode 100644 crates/weaveffi-cli/src/targets/python/runtime/callbacks.py rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/python/runtime/cancel.py (91%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/python/runtime/core.py (74%) create mode 100644 crates/weaveffi-cli/src/targets/python/tests.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/python/types.rs (63%) create mode 100644 crates/weaveffi-cli/src/targets/ruby/callbacks.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/ruby/calls.rs (53%) create mode 100644 crates/weaveffi-cli/src/targets/ruby/codec.rs create mode 100644 crates/weaveffi-cli/src/targets/ruby/docs.rs create mode 100644 crates/weaveffi-cli/src/targets/ruby/entities.rs create mode 100644 crates/weaveffi-cli/src/targets/ruby/mod.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/ruby/package.rs (70%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/ruby/runtime.rs (87%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/ruby/runtime/runtime.rb (59%) create mode 100644 crates/weaveffi-cli/src/targets/ruby/tests.rs create mode 100644 crates/weaveffi-cli/src/targets/ruby/types.rs create mode 100644 crates/weaveffi-cli/src/targets/swift/callbacks.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/swift/calls.rs (76%) create mode 100644 crates/weaveffi-cli/src/targets/swift/codec.rs create mode 100644 crates/weaveffi-cli/src/targets/swift/docs.rs create mode 100644 crates/weaveffi-cli/src/targets/swift/entities.rs create mode 100644 crates/weaveffi-cli/src/targets/swift/mod.rs create mode 100644 crates/weaveffi-cli/src/targets/swift/package.rs create mode 100644 crates/weaveffi-cli/src/targets/swift/runtime.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/swift/runtime/Package.swift (50%) create mode 100644 crates/weaveffi-cli/src/targets/swift/runtime/README.md rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/swift/runtime/Runtime.swift (56%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/swift/runtime/module.modulemap (100%) create mode 100644 crates/weaveffi-cli/src/targets/swift/tests.rs rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/swift/types.rs (60%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/wasm/glue.rs (75%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/wasm/mod.rs (54%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/wasm/package.rs (87%) rename crates/weaveffi-cli/{tests/snapshots/wasm_edge_cases__linear_js.snap => src/targets/wasm/runtime/linear.js} (67%) rename crates/{weaveffi-gen => weaveffi-cli}/src/targets/wasm/tests.rs (61%) create mode 100644 crates/weaveffi-cli/src/utils.rs create mode 100644 crates/weaveffi-cli/tests/abi_version.rs rename crates/{weaveffi-gen => weaveffi-cli}/tests/c_buffer.rs (90%) delete mode 100644 crates/weaveffi-cli/tests/cli/extract_roundtrip.rs delete mode 100644 crates/weaveffi-cli/tests/fixtures/kitchen_sink_annotated.rs create mode 100644 crates/weaveffi-cli/tests/fixtures/producer/Cargo.toml create mode 100644 crates/weaveffi-cli/tests/fixtures/producer/expected.yml create mode 100644 crates/weaveffi-cli/tests/fixtures/producer/src/lib.rs delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_shapes__CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__pubspec_yaml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_edge_cases__pubspec_yaml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_nested_modules__pubspec_yaml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dart_shapes__pubspec_yaml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_csproj.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__Runtime_cs.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_csproj.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__Runtime_cs.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_csproj.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__Runtime_cs.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Runtime_cs.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_csproj.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__codec_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__docs_everywhere_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__go_mod.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__runtime_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_edge_cases__codec_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_edge_cases__edge_cases_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_edge_cases__go_mod.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_edge_cases__runtime_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__kitchen_sink_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_nested_modules__codec_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_nested_modules__go_mod.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_nested_modules__nested_modules_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_nested_modules__runtime_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_shapes__codec_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_shapes__go_mod.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_shapes__runtime_go.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/go_shapes__shapes_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__build_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__consumer_rules_pro.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__settings_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Async_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Buffers_kt.snap create mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Codecs_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Runtime_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__build_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__consumer_rules_pro.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__settings_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Async_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Buffers_kt.snap create mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Codecs_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Runtime_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_h.snap create mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Codecs_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__build_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__consumer_rules_pro.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__settings_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Buffers_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Runtime_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_shapes__build_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_shapes__consumer_rules_pro.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_shapes__settings_gradle_kts.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_CMakeLists_txt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Buffers_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Runtime_kt.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__binding_gyp.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__runtime_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_edge_cases__binding_gyp.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_edge_cases__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_edge_cases__runtime_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_nested_modules__binding_gyp.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_nested_modules__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_nested_modules__runtime_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_shapes__binding_gyp.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_shapes__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_shapes__runtime_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere___init___py.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_pyi.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_py_typed.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__pyproject_toml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases___init___py.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_pyi.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_py_typed.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_edge_cases__pyproject_toml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_pyi.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules___init___py.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_pyi.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_py_typed.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_nested_modules__pyproject_toml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_shapes__pyproject_toml.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_shapes__shapes___init___py.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_py_typed.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_pyi.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__docs_everywhere_gemspec.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_runtime_rb.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__edge_cases_gemspec.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_runtime_rb.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_runtime_rb.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__nested_modules_gemspec.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_runtime_rb.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/ruby_shapes__shapes_gemspec.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Package_swift.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_docs_everywhere_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_module_modulemap.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Package_swift.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_edge_cases_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_module_modulemap.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_kitchen_sink_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Package_swift.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_module_modulemap.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_nested_modules_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_shapes__Package_swift.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_module_modulemap.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_shapes_h.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__linear_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__runtime_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__runtime_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__linear_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__runtime_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_shapes__README_md.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_shapes__linear_js.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_shapes__package_json.snap delete mode 100644 crates/weaveffi-cli/tests/snapshots/wasm_shapes__runtime_js.snap create mode 100644 crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/f64_list.bin create mode 100644 crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/nested_bytes_list.bin create mode 100644 crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/optional_string_list.bin create mode 100644 crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/string.bin create mode 100644 crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/string_map.bin create mode 100644 crates/weaveffi-fuzz/fuzz_targets/value_buffer.rs create mode 100644 crates/weaveffi-fuzz/seed_corpus.py delete mode 100644 crates/weaveffi-gen/Cargo.toml delete mode 100644 crates/weaveffi-gen/src/backend/mod.rs delete mode 100644 crates/weaveffi-gen/src/cache.rs delete mode 100644 crates/weaveffi-gen/src/capabilities.rs delete mode 100644 crates/weaveffi-gen/src/codegen/mod.rs delete mode 100644 crates/weaveffi-gen/src/lib.rs delete mode 100644 crates/weaveffi-gen/src/package.rs delete mode 100644 crates/weaveffi-gen/src/targets/c/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/callbacks.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/calls.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/entities.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/runtime.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/tests.rs delete mode 100644 crates/weaveffi-gen/src/targets/cpp/types.rs delete mode 100644 crates/weaveffi-gen/src/targets/dart/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/dart/entities.rs delete mode 100644 crates/weaveffi-gen/src/targets/dart/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/dart/runtime.rs delete mode 100644 crates/weaveffi-gen/src/targets/dart/runtime/loader.dart delete mode 100644 crates/weaveffi-gen/src/targets/dotnet/callbacks.rs delete mode 100644 crates/weaveffi-gen/src/targets/dotnet/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/dotnet/runtime.rs delete mode 100644 crates/weaveffi-gen/src/targets/dotnet/tests.rs delete mode 100644 crates/weaveffi-gen/src/targets/go/calls.rs delete mode 100644 crates/weaveffi-gen/src/targets/go/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/go/docs.rs delete mode 100644 crates/weaveffi-gen/src/targets/go/entities.rs delete mode 100644 crates/weaveffi-gen/src/targets/go/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/go/runtime/runtime.go delete mode 100644 crates/weaveffi-gen/src/targets/go/types.rs delete mode 100644 crates/weaveffi-gen/src/targets/js/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/kotlin/bridge.rs delete mode 100644 crates/weaveffi-gen/src/targets/kotlin/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/kotlin/docs.rs delete mode 100644 crates/weaveffi-gen/src/targets/kotlin/runtime/jni_callbacks.c delete mode 100644 crates/weaveffi-gen/src/targets/kotlin/runtime/jni_threads.c delete mode 100644 crates/weaveffi-gen/src/targets/kotlin/tests.rs delete mode 100644 crates/weaveffi-gen/src/targets/node/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/node/runtime/loader.js delete mode 100644 crates/weaveffi-gen/src/targets/node/tests.rs delete mode 100644 crates/weaveffi-gen/src/targets/python/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/python/docs.rs delete mode 100644 crates/weaveffi-gen/src/targets/python/entities.rs delete mode 100644 crates/weaveffi-gen/src/targets/python/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/python/runtime.rs delete mode 100644 crates/weaveffi-gen/src/targets/python/runtime/callbacks.py delete mode 100644 crates/weaveffi-gen/src/targets/python/stubs.rs delete mode 100644 crates/weaveffi-gen/src/targets/python/tests.rs delete mode 100644 crates/weaveffi-gen/src/targets/ruby/callbacks.rs delete mode 100644 crates/weaveffi-gen/src/targets/ruby/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/ruby/docs.rs delete mode 100644 crates/weaveffi-gen/src/targets/ruby/entities.rs delete mode 100644 crates/weaveffi-gen/src/targets/ruby/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/ruby/types.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/callbacks.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/codec.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/docs.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/entities.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/mod.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/package.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/runtime.rs delete mode 100644 crates/weaveffi-gen/src/targets/swift/runtime/README.md delete mode 100644 crates/weaveffi-gen/src/targets/wasm/runtime/linear.js delete mode 100644 crates/weaveffi-gen/src/utils.rs delete mode 100644 crates/weaveffi-gen/tests/dart_render.rs create mode 100644 crates/weaveffi-macros/src/codegen/contract.rs create mode 100644 crates/weaveffi-macros/src/codegen/diagnostics.rs create mode 100644 crates/weaveffi-macros/src/codegen/lift.rs delete mode 100644 crates/weaveffi-macros/src/codegen/marshal.rs create mode 100644 crates/weaveffi-macros/src/codegen/meta.rs create mode 100644 crates/weaveffi-macros/tests/ui/fail_callback_plain_return.rs create mode 100644 crates/weaveffi-macros/tests/ui/fail_callback_plain_return.stderr delete mode 100644 crates/weaveffi-macros/tests/ui/fail_callback_string_return.rs delete mode 100644 crates/weaveffi-macros/tests/ui/fail_callback_string_return.stderr create mode 100644 crates/weaveffi-macros/tests/ui/fail_member_cfg.rs create mode 100644 crates/weaveffi-macros/tests/ui/fail_member_cfg.stderr create mode 100644 crates/weaveffi-macros/tests/ui/fail_out_of_line_module.rs create mode 100644 crates/weaveffi-macros/tests/ui/fail_out_of_line_module.stderr rename crates/weaveffi-macros/tests/ui/{fail_sibling_c_enum.rs => fail_out_of_tree_c_enum.rs} (100%) rename crates/weaveffi-macros/tests/ui/{fail_sibling_c_enum.stderr => fail_out_of_tree_c_enum.stderr} (72%) create mode 100644 crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.rs create mode 100644 crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.stderr create mode 100644 crates/weaveffi-macros/tests/ui/pass_aliases_and_cfg.rs create mode 100644 crates/weaveffi-macros/tests/ui/pass_async_factory.rs create mode 100644 crates/weaveffi-macros/tests/ui/pass_rich_callbacks.rs delete mode 100644 crates/weaveffi-model/src/checksum.rs create mode 100644 crates/weaveffi-model/src/contract.rs create mode 100644 crates/weaveffi-model/src/meta.rs create mode 100644 crates/weaveffi-model/src/model/build.rs delete mode 100644 crates/weaveffi-model/src/resolved.rs rename crates/weaveffi-model/src/{model => }/ty.rs (61%) rename crates/{weaveffi-abi/src => weaveffi/src/abi}/buffer.rs (81%) create mode 100644 crates/weaveffi/src/abi/callback.rs rename crates/{weaveffi-abi/src => weaveffi/src/abi}/cancel.rs (58%) create mode 100644 crates/weaveffi/src/abi/contract.rs rename crates/{weaveffi-abi/src => weaveffi/src/abi}/convert.rs (72%) rename crates/{weaveffi-abi/src => weaveffi/src/abi}/error.rs (79%) rename crates/{weaveffi-abi/src => weaveffi/src/abi}/iter.rs (51%) rename crates/{weaveffi-abi/src => weaveffi/src/abi}/leak.rs (84%) create mode 100644 crates/weaveffi/src/abi/marshal.rs create mode 100644 crates/weaveffi/src/abi/mod.rs rename crates/{weaveffi-abi/src => weaveffi/src/abi}/object.rs (96%) create mode 100644 crates/weaveffi/src/abi/spawn.rs rename crates/{weaveffi-abi => weaveffi}/tests/buffer_proptest.rs (90%) create mode 100644 crates/weaveffi/tests/spawn_failure.rs create mode 100644 crates/weaveffi/tests/tokio.rs delete mode 100644 samples/async-demo/Cargo.toml delete mode 100644 samples/async-demo/src/lib.rs delete mode 100644 samples/async-demo/weaveffi.toml delete mode 100644 samples/contacts/Cargo.toml delete mode 100644 samples/contacts/src/lib.rs delete mode 100644 samples/contacts/weaveffi.toml delete mode 100644 samples/events/Cargo.toml delete mode 100644 samples/events/src/lib.rs delete mode 100644 samples/events/weaveffi.toml create mode 100644 samples/kvstore/src/tests.rs create mode 100644 scripts/fixtures/lib.sh create mode 100755 scripts/package-smoke.sh diff --git a/.github/actions/toolchains/action.yml b/.github/actions/toolchains/action.yml index efae43f1..6f35a7ac 100644 --- a/.github/actions/toolchains/action.yml +++ b/.github/actions/toolchains/action.yml @@ -11,10 +11,20 @@ runs: uses: actions/setup-node@v4 with: node-version: '22' + # The node fixture check compiles the N-API addon against these headers. + - if: inputs.lang == 'node' + shell: bash + run: npx --yes node-gyp install - if: inputs.lang == 'python' uses: actions/setup-python@v5 with: python-version: '3.12' + # The python fixture check type-checks the generated package with mypy + # against Python 3.9, the oldest version it supports; mypy 2 can't + # target 3.9. + - if: inputs.lang == 'python' + shell: bash + run: python -m pip install 'mypy==1.20.2' - if: inputs.lang == 'ruby' uses: ruby/setup-ruby@v1 with: @@ -26,11 +36,13 @@ runs: # a problem matcher relative to the calling action), so fetch the SDK. - if: inputs.lang == 'dart' shell: bash + env: + DART_VERSION: 3.13.5 run: | os=linux; [ "$RUNNER_OS" = macOS ] && os=macos arch=x64; case "$(uname -m)" in arm64|aarch64) arch=arm64 ;; esac curl -fsSL -o "$RUNNER_TEMP/dart.zip" \ - "https://storage.googleapis.com/dart-archive/channels/stable/release/latest/sdk/dartsdk-$os-$arch-release.zip" + "https://storage.googleapis.com/dart-archive/channels/stable/release/$DART_VERSION/sdk/dartsdk-$os-$arch-release.zip" unzip -q -o "$RUNNER_TEMP/dart.zip" -d "$RUNNER_TEMP" echo "$RUNNER_TEMP/dart-sdk/bin" >> "$GITHUB_PATH" - if: inputs.lang == 'go' @@ -51,7 +63,7 @@ runs: distribution: temurin java-version: '17' - if: inputs.lang == 'kotlin' - uses: fwilhe2/setup-kotlin@main + uses: fwilhe2/setup-kotlin@v2.1 with: version: '2.0.21' - if: (inputs.lang == 'c' || inputs.lang == 'cpp') && runner.os == 'Linux' diff --git a/.github/workflows/bench.yml b/.github/workflows/bench.yml index 1a96aea3..c411d4e5 100644 --- a/.github/workflows/bench.yml +++ b/.github/workflows/bench.yml @@ -1,14 +1,22 @@ name: Bench +# Weekly (and on demand) sampled benchmark run. The uploaded artifact is the +# historical record of the numbers. Pull requests get a cheaper smoke run in +# ci.yml, which executes every benchmark body once. on: workflow_dispatch: schedule: - cron: '0 9 * * 1' +concurrency: + group: bench-${{ github.ref }} + cancel-in-progress: true + jobs: bench: name: cargo bench runs-on: ubuntu-latest + timeout-minutes: 60 steps: - name: Checkout uses: actions/checkout@v4 @@ -21,39 +29,18 @@ jobs: with: cache-targets: true - # PR smoke run: criterion's `--test` mode executes every benchmark - # body exactly once, with no warmup or sampling. This catches - # panics and stale fixtures (clippy already compile-checks the - # harnesses on PRs, but only execution catches runtime failures) - # without spending minutes collecting numbers that are too noisy - # to compare on shared runners. - - name: Run cargo bench (smoke) - if: github.event_name == 'pull_request' - run: | - set -euo pipefail - cargo bench -p weaveffi-gen --bench codegen_bench -- --test - cargo bench -p weaveffi-cli --bench generate_bench -- --test - - # Full sampled run on main; the uploaded artifact is the historical - # record of the numbers. - name: Run cargo bench - if: github.event_name == 'push' - # The two criterion harnesses (`weaveffi-gen` and `weaveffi-cli`) - # accept `--noplot`; the workspace `cargo bench --workspace` flag + # The criterion harness accepts `--noplot`; `cargo bench --workspace` # also fans out to the default `libtest` harness in every other - # crate, which rejects `--noplot`, so we invoke each criterion - # bench explicitly. `--noplot` skips the gnuplot/plotters HTML + # crate, which rejects `--noplot`, so we invoke the criterion bench + # explicitly. `--noplot` skips the gnuplot/plotters HTML # reports so the artifact stays small (a few KB of text instead # of MBs of SVG plots). run: | set -euo pipefail - { - cargo bench -p weaveffi-gen --bench codegen_bench -- --noplot - cargo bench -p weaveffi-cli --bench generate_bench -- --noplot - } 2>&1 | tee bench.txt + cargo bench -p weaveffi-cli --bench codegen_bench -- --noplot 2>&1 | tee bench.txt - name: Upload bench results - if: github.event_name == 'push' uses: actions/upload-artifact@v4 with: name: bench-results diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7e7fa8bc..b9c08aab 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,6 +17,7 @@ jobs: lint: name: Format, clippy, and rustdoc runs-on: ubuntu-latest + timeout-minutes: 30 steps: - uses: actions/checkout@v4 - uses: dtolnay/rust-toolchain@stable @@ -35,6 +36,7 @@ jobs: test: name: Test (${{ matrix.os }}) runs-on: ${{ matrix.os }} + timeout-minutes: 45 strategy: fail-fast: false matrix: @@ -48,13 +50,19 @@ jobs: tool: cargo-insta # One pass runs every test and fails on any snapshot drift. - run: cargo insta test --workspace --check - - name: Benchmarks compile + # Bench smoke run: every benchmark compiles, and criterion's `--test` + # mode executes each body once, with no warmup or sampling, which + # catches panics and stale inputs. bench.yml collects the numbers. + - name: Benchmarks build and run once if: runner.os == 'Linux' - run: cargo bench --workspace --no-run + run: | + cargo bench --workspace --no-run + cargo bench -p weaveffi-cli --bench codegen_bench -- --test generated-drift: name: Samples, schema, and wasm build runs-on: ubuntu-latest + timeout-minutes: 30 steps: - uses: actions/checkout@v4 - uses: dtolnay/rust-toolchain@stable @@ -69,12 +77,12 @@ jobs: run: | set -euo pipefail BIN=target/debug/weaveffi - for sample in calculator contacts events kvstore async-demo codec; do - "$BIN" generate "samples/$sample/src/lib.rs" -o "diffcheck/$sample" --quiet - "$BIN" diff "samples/$sample/src/lib.rs" -o "diffcheck/$sample" --check --quiet + for sample in calculator codec kvstore; do + "$BIN" generate "samples/$sample" -o "diffcheck/$sample" --quiet + "$BIN" diff "samples/$sample" -o "diffcheck/$sample" --check --quiet done echo "stale" >> diffcheck/calculator/c/calculator.h - if "$BIN" diff samples/calculator/src/lib.rs -o diffcheck/calculator --check --quiet; then + if "$BIN" diff samples/calculator -o diffcheck/calculator --check --quiet; then echo "diff --check did not detect a modified file" >&2 exit 1 fi @@ -83,14 +91,27 @@ jobs: target/debug/weaveffi schema > weaveffi.schema.json.regenerated diff -u weaveffi.schema.json weaveffi.schema.json.regenerated - name: Every sample builds for wasm32 - run: cargo build --target wasm32-unknown-unknown -p calculator -p contacts -p events -p kvstore -p async-demo -p codec + run: cargo build --target wasm32-unknown-unknown -p calculator -p codec -p kvstore + # A `.wasm` module carries its metadata in a custom section rather + # than in symbols; it must describe the same API as the native build. + - name: The wasm32 metadata matches the native metadata + run: | + set -euo pipefail + for sample in calculator codec kvstore; do + target/debug/weaveffi extract -q "samples/$sample" > "native-$sample.yml" + target/debug/weaveffi extract -q "samples/$sample" \ + --library "target/wasm32-unknown-unknown/debug/$sample.wasm" > "wasm-$sample.yml" + diff -u "native-$sample.yml" "wasm-$sample.yml" + done fixtures: # Snapshot tests pin the generated text; this compiles or type-checks it # with each language's own toolchain, for every fixture, so invalid code - # in shapes no sample exercises still fails CI. + # in shapes no sample exercises still fails CI. GitHub sets CI=true, so a + # missing tool fails the job instead of skipping a check. name: Fixture compile check (${{ matrix.target }}) runs-on: ${{ matrix.os }} + timeout-minutes: 30 strategy: fail-fast: false matrix: @@ -144,6 +165,7 @@ jobs: # producer built for Android. name: Android (NDK link) runs-on: ubuntu-latest + timeout-minutes: 30 steps: - uses: actions/checkout@v4 - uses: dtolnay/rust-toolchain@stable @@ -163,9 +185,44 @@ jobs: NDK: ${{ steps.ndk.outputs.ndk-path }} run: | set -euxo pipefail - cargo run -q -p weaveffi-cli -- generate samples/kvstore/src/lib.rs -o generated --target c,kotlin + cargo run -q -p weaveffi-cli -- generate samples/kvstore -o generated --target c,kotlin \ + --library target/aarch64-linux-android/release/libkvstore.so CLANG="$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang" shim=$(ls generated/kotlin/src/main/cpp/*.c | head -1) "$CLANG" -shared -fPIC -Werror -I generated/kotlin/src/main/cpp -I generated/c "$shim" \ -L target/aarch64-linux-android/release -lkvstore -o libkvstore_jni.so "$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-readelf" -d libkvstore_jni.so | grep 'libkvstore.so' + - name: weaveffi build cross-compiles the producer and prebuilds the shim + env: + ANDROID_NDK_HOME: ${{ steps.ndk.outputs.ndk-path }} + run: | + set -euxo pipefail + cargo run -q -p weaveffi-cli -- build samples/kvstore --platforms android-arm64 --target kotlin + readelf="$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-readelf" + "$readelf" -d target/weaveffi/android-arm64/libkvstore.so | grep 'SONAME.*libkvstore.so' + "$readelf" -d target/weaveffi/android-arm64/libkvstore_jni.so | grep 'NEEDED.*libkvstore.so' + + package-smoke: + # `weaveffi package` must produce artifacts each ecosystem's own tool + # installs with no compiler: the wheel into a venv, the npm tarballs into + # a throwaway project (using the prebuilt addon), and on macOS the + # SwiftPM package over its XCFramework. GitHub sets CI=true, so a missing + # tool fails the job instead of skipping a target. + name: Package install smoke (${{ matrix.os }}) + runs-on: ${{ matrix.os }} + timeout-minutes: 30 + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest] + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + - uses: Swatinem/rust-cache@v2 + - uses: ./.github/actions/toolchains + with: + lang: python + - uses: ./.github/actions/toolchains + with: + lang: node + - run: bash scripts/package-smoke.sh diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 9d3017e1..19e5487f 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -17,6 +17,7 @@ concurrency: jobs: build: runs-on: ubuntu-latest + timeout-minutes: 30 steps: - uses: actions/checkout@v4 - uses: actions/configure-pages@v5 @@ -29,9 +30,9 @@ jobs: - name: Install mdbook + mdbook-linkcheck2 # Keep this in sync with the docs linkcheck job in quality.yml. - run: | - cargo install mdbook --locked - cargo install mdbook-linkcheck2 --locked + uses: taiki-e/install-action@v2 + with: + tool: mdbook,mdbook-linkcheck2 - name: Build mdBook run: mdbook build docs @@ -60,6 +61,7 @@ jobs: url: ${{ steps.deployment.outputs.page_url }} needs: build runs-on: ubuntu-latest + timeout-minutes: 10 steps: - id: deployment uses: actions/deploy-pages@v4 diff --git a/.github/workflows/fuzz.yml b/.github/workflows/fuzz.yml index 65711754..cf7d7b6a 100644 --- a/.github/workflows/fuzz.yml +++ b/.github/workflows/fuzz.yml @@ -1,15 +1,23 @@ name: Fuzz +# Daily (and on demand), each target runs for 60 seconds from a corpus seeded +# with its committed seeds plus inputs derived from the snapshot fixtures. +# Extend a run by bumping `-max_total_time` below. on: workflow_dispatch: schedule: - # 08:00 UTC daily. Extend a run by bumping `-max_total_time` below. + # 08:00 UTC daily. - cron: '0 8 * * *' +concurrency: + group: fuzz-${{ github.ref }} + cancel-in-progress: true + jobs: fuzz: name: cargo-fuzz (${{ matrix.target }}) runs-on: ubuntu-latest + timeout-minutes: 30 strategy: fail-fast: false matrix: @@ -19,6 +27,7 @@ jobs: - fuzz_parse_toml - fuzz_parse_type_ref - fuzz_validate + - fuzz_value_buffer steps: - name: Checkout uses: actions/checkout@v4 @@ -35,16 +44,24 @@ jobs: with: tool: cargo-fuzz + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + + - name: Seed the corpus + run: | + python -m pip install pyyaml tomli-w + python crates/weaveffi-fuzz/seed_corpus.py ${{ matrix.target }} \ + crates/weaveffi-fuzz/corpus/${{ matrix.target }} + - name: Run ${{ matrix.target }} for 60s - # Each target is fuzzed for 60s in PRs (≈5 min total). The cron run - # uses the same budget today; bump `-max_total_time` to extend it. - # Any crash is written to fuzz/artifacts/ and fails the job. + # Any crash is written to artifacts/ and fails the job. run: | cargo +nightly fuzz run \ --fuzz-dir crates/weaveffi-fuzz \ --features fuzzing \ ${{ matrix.target }} \ - crates/weaveffi-fuzz/fuzz/seeds/${{ matrix.target }} \ + crates/weaveffi-fuzz/corpus/${{ matrix.target }} \ -- -max_total_time=60 -seed=1 - name: Upload crash artifacts @@ -52,6 +69,6 @@ jobs: uses: actions/upload-artifact@v4 with: name: fuzz-artifacts-${{ matrix.target }} - path: crates/weaveffi-fuzz/fuzz/artifacts/ + path: crates/weaveffi-fuzz/artifacts/ if-no-files-found: ignore retention-days: 14 diff --git a/.github/workflows/pr-lint.yml b/.github/workflows/pr-lint.yml index 3cb10c6b..db1ed839 100644 --- a/.github/workflows/pr-lint.yml +++ b/.github/workflows/pr-lint.yml @@ -7,10 +7,15 @@ on: permissions: pull-requests: read +concurrency: + group: pr-lint-${{ github.event.pull_request.number }} + cancel-in-progress: true + jobs: pr-title: name: PR title (Conventional Commits) runs-on: ubuntu-latest + timeout-minutes: 5 steps: - name: Validate PR title uses: amannn/action-semantic-pull-request@v5 @@ -39,6 +44,7 @@ jobs: commits: name: Commit messages runs-on: ubuntu-latest + timeout-minutes: 10 steps: - uses: actions/checkout@v4 with: diff --git a/.github/workflows/quality.yml b/.github/workflows/quality.yml index 5f53505a..52d89c54 100644 --- a/.github/workflows/quality.yml +++ b/.github/workflows/quality.yml @@ -5,10 +5,15 @@ on: branches: [main] pull_request: +concurrency: + group: quality-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + jobs: supply-chain: name: cargo-deny, cargo-audit, cargo-machete runs-on: ubuntu-latest + timeout-minutes: 15 steps: - uses: actions/checkout@v4 - uses: dtolnay/rust-toolchain@stable @@ -22,6 +27,7 @@ jobs: coverage: name: Coverage runs-on: ubuntu-latest + timeout-minutes: 45 continue-on-error: true steps: - uses: actions/checkout@v4 @@ -41,6 +47,7 @@ jobs: docs-linkcheck: name: Docs link check runs-on: ubuntu-latest + timeout-minutes: 15 steps: - uses: actions/checkout@v4 - uses: taiki-e/install-action@v2 diff --git a/.github/workflows/release-binaries.yml b/.github/workflows/release-binaries.yml index 00ae79b1..3dd3332c 100644 --- a/.github/workflows/release-binaries.yml +++ b/.github/workflows/release-binaries.yml @@ -19,15 +19,20 @@ jobs: build: name: ${{ matrix.target }} runs-on: ${{ matrix.os }} + timeout-minutes: 45 strategy: fail-fast: false matrix: + # `native` marks a binary the runner can execute, so the job runs its + # `--version` before uploading. x86_64-apple-darwin is cross-compiled + # on the arm64 macOS runner (GitHub retired the Intel images) and + # isn't run there. include: - - { target: x86_64-unknown-linux-gnu, os: ubuntu-latest } - - { target: aarch64-unknown-linux-gnu, os: ubuntu-24.04-arm } - - { target: x86_64-apple-darwin, os: macos-13 } - - { target: aarch64-apple-darwin, os: macos-latest } - - { target: x86_64-pc-windows-msvc, os: windows-latest } + - { target: x86_64-unknown-linux-gnu, os: ubuntu-latest, native: true } + - { target: aarch64-unknown-linux-gnu, os: ubuntu-24.04-arm, native: true } + - { target: x86_64-apple-darwin, os: macos-latest, native: false } + - { target: aarch64-apple-darwin, os: macos-latest, native: true } + - { target: x86_64-pc-windows-msvc, os: windows-latest, native: true } env: TAG: ${{ github.event.release.tag_name || inputs.tag }} steps: @@ -38,6 +43,17 @@ jobs: with: targets: ${{ matrix.target }} - run: cargo build -p weaveffi-cli --release --locked --target ${{ matrix.target }} + - name: Smoke-test the binary + if: matrix.native + shell: bash + run: | + set -euo pipefail + version=$("target/${{ matrix.target }}/release/weaveffi" --version) + echo "$version" + case "$version" in + *" ${TAG#v}"*) ;; + *) echo "expected version ${TAG#v} from $TAG" >&2; exit 1 ;; + esac - name: Package and upload shell: bash env: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 3392cdcd..5e5361a2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -14,6 +14,7 @@ jobs: release-pr: name: Release PR runs-on: ubuntu-latest + timeout-minutes: 20 if: ${{ github.repository_owner == 'weavefoundry' }} concurrency: group: release-pr @@ -38,6 +39,7 @@ jobs: release: name: Publish runs-on: ubuntu-latest + timeout-minutes: 45 if: ${{ github.repository_owner == 'weavefoundry' }} concurrency: group: release diff --git a/.gitignore b/.gitignore index 80a4628c..866f7ad4 100644 --- a/.gitignore +++ b/.gitignore @@ -20,8 +20,9 @@ target # option (not recommended) you can uncomment the following to ignore the entire idea folder. .idea/ -# Generated by WeaveFFI CLI (outputs from `weaveffi generate`) -/generated/ +# Generated by WeaveFFI CLI (`weaveffi generate` and `weaveffi package`) +/bindings/ +/dist/ # Compiled C example binary /c_example @@ -61,9 +62,9 @@ examples/dotnet/Contacts/bin/ examples/dotnet/Contacts/obj/ # cargo-fuzz runtime artifacts (seeds/ is committed; the rest is generated) -crates/weaveffi-fuzz/fuzz/artifacts/ -crates/weaveffi-fuzz/fuzz/corpus/ -crates/weaveffi-fuzz/fuzz/coverage/ +crates/weaveffi-fuzz/artifacts/ +crates/weaveffi-fuzz/corpus/ +crates/weaveffi-fuzz/coverage/ # Bindings generated inside a sample by `weaveffi generate` samples/*/bindings/ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 924c7e59..aa7e25ba 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -9,26 +9,26 @@ the runtime, or the orchestrator. The pipeline: ```text -annotated Rust (.rs) or IDL (YAML/JSON/TOML) +a Rust producer crate (the API read from its built library's metadata) +or an IDL (YAML/JSON/TOML) → IR (Api) - → validate (every rule, including the C symbol table) - → ResolvedApi + Identity (names from weaveffi.toml / Cargo.toml) - → BindingModel (C symbols and ABI signatures, computed once) - → marshalling plan + → validate (every rule, including the C symbol table, under the + Identity from weaveffi.toml / cargo metadata) + → Model (C symbols and ABI signatures, computed once; the plan) → Target::render (pure, per language) - → Orchestrator (capability gate, cache records, changed-file writes, - stale-file removal) + → Orchestrator (changed-file writes, generation records, stale-file + removal) ``` The workspace crates: -- `weaveffi-model`: the IR, IDL parsing, Rust extraction, validation, the - resolved view, package identity, the binding model, ABI lowering, the - marshalling plan, and contract checksums. -- `weaveffi-gen`: the backend framework, the orchestrator and cache, and the - eleven language targets under `targets/`. -- `weaveffi-cli`: the `weaveffi` command. -- `weaveffi-abi`: the C ABI runtime linked into every producer. +- `weaveffi-model`: the IR, IDL parsing, the macro's Rust extraction, + library metadata frames, validation, package identity, the model, ABI + lowering, the marshalling plan, and contract tables. +- `weaveffi-cli`: the `weaveffi` command and the library behind it: the + backend framework, the orchestrator and generation records, packaging, and + the eleven language targets under `src/targets/`. - `weaveffi-macros`: `#[weaveffi::module]` and `export_runtime!`. -- `weaveffi`: the producer facade crate. +- `weaveffi`: the producer facade crate and the C ABI runtime + (`weaveffi::abi`) linked into every producer. - `weaveffi-fuzz`: unpublished fuzz harnesses. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c2677c84..71b2696b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -54,9 +54,13 @@ should also pass that target's fixture check and conformance lanes. ### Snapshot tests Snapshot tests (`crates/weaveffi-cli/tests/snapshots.rs`) pin the exact -output of every generator for every fixture in +output of every generator for the fixtures in `crates/weaveffi-cli/tests/fixtures/`, using -[`cargo-insta`](https://insta.rs/). When output changes on purpose: +[`cargo-insta`](https://insta.rs/). Fixed files (runtimes, package +manifests, and READMEs, listed per target in `FIXED_FILES`) are snapshotted +once, from `kitchen_sink`; a target's copy of the C header is asserted +byte-equal to the C target's own header instead of snapshotted; every other +file is snapshotted for every fixture. When output changes on purpose: ```bash cargo install cargo-insta --locked @@ -74,13 +78,20 @@ Snapshots prove the text didn't change, not that it compiles. `scripts/check-fixtures.sh ` generates every fixture and compiles or type-checks it with the target language's toolchain (through `scripts/fixtures/.sh`). Run it for any target whose output you -change; CI runs one job per target. +change; CI runs one job per target. A missing tool (including `mypy` 1.x for +Python and the node-gyp headers from `npx node-gyp install` for Node.js) is +reported as a skip locally and fails the check when `CI=true`. ### Conformance -`conformance/run.sh` builds every sample producer with leak counters on, -generates bindings, and runs real consumers in every language against them. -Each consumer must exit 0 with every leak counter at zero. +`conformance/run.sh` builds the three sample producers (`calculator`, +`codec`, and `kvstore`; see [Samples](docs/src/samples.md)) with leak counters +on, generates bindings, and runs real consumers in every language against +them: one lane per sample per language, named `-`, with its +consumer in `conformance//`. Each consumer must exit 0 with every leak +counter at zero. A new feature belongs in `samples/kvstore` (or in +`samples/codec`'s vector table, for a new wire shape), with assertions in +every language's consumer. ```bash ONLY=python bash conformance/run.sh # one language @@ -89,7 +100,8 @@ SKIP=wasm SKIP_GEN=1 bash conformance/run.sh # skip lanes; reuse generated bi ``` `LANE_TIMEOUT` sets the per-lane limit in seconds (default 300). A missing -toolchain fails its lanes; install it or skip them. +toolchain skips its language with a note locally and fails it when +`CI=true`. ## Documentation @@ -109,7 +121,7 @@ Preview the book with `mdbook serve docs -p 3000 -n 127.0.0.1`. Read the [architecture guide](docs/src/architecture.md) first; its "Adding a generator" section is the checklist. In short: add -`crates/weaveffi-gen/src/targets//` implementing `LanguageBackend`, +`crates/weaveffi-cli/src/targets//` implementing `LanguageBackend`, register it with one line in the `cli_targets!` registry in `crates/weaveffi-cli/src/config.rs`, add it to the snapshot tests, add `scripts/fixtures/.sh` and `conformance//`, and document it under @@ -117,7 +129,8 @@ register it with one line in the `cli_targets!` registry in ## Fuzzing -Parser and validator fuzz harnesses live in `crates/weaveffi-fuzz` and are +Fuzz harnesses for the parsers, the validator, and the runtime's +value-buffer decoders live in `crates/weaveffi-fuzz` and are driven by [`cargo-fuzz`](https://github.com/rust-fuzz/cargo-fuzz) + `libFuzzer`. They require nightly Rust because the libFuzzer sanitizer flags are unstable. @@ -129,15 +142,19 @@ rustup toolchain install nightly cargo install cargo-fuzz --locked ``` -Run a target for 60 seconds (swap the target name for any of `fuzz_parse_yaml`, -`fuzz_parse_json`, `fuzz_parse_toml`, `fuzz_parse_type_ref`, `fuzz_validate`): +Seed a target's corpus from its committed seeds and the snapshot fixtures +(needs `pip install pyyaml tomli-w`), then run it for 60 seconds (swap the +target name for any of `fuzz_parse_yaml`, `fuzz_parse_json`, +`fuzz_parse_toml`, `fuzz_parse_type_ref`, `fuzz_validate`, +`fuzz_value_buffer`): ```bash +python3 crates/weaveffi-fuzz/seed_corpus.py fuzz_parse_yaml \ + crates/weaveffi-fuzz/corpus/fuzz_parse_yaml cargo +nightly fuzz run \ --fuzz-dir crates/weaveffi-fuzz \ --features fuzzing \ fuzz_parse_yaml \ - crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_yaml \ -- -max_total_time=60 ``` @@ -146,7 +163,7 @@ Drop `-max_total_time=60` to fuzz indefinitely. ### Triaging a crash When libFuzzer finds an input that panics or aborts it writes the bytes to -`crates/weaveffi-fuzz/fuzz/artifacts//crash-`. To triage: +`crates/weaveffi-fuzz/artifacts//crash-`. To triage: 1. Pretty-print the input as the target sees it: @@ -155,7 +172,7 @@ When libFuzzer finds an input that panics or aborts it writes the bytes to --fuzz-dir crates/weaveffi-fuzz \ --features fuzzing \ \ - crates/weaveffi-fuzz/fuzz/artifacts//crash- + crates/weaveffi-fuzz/artifacts//crash- ``` 2. Minimize the reproducer: @@ -165,12 +182,13 @@ When libFuzzer finds an input that panics or aborts it writes the bytes to --fuzz-dir crates/weaveffi-fuzz \ --features fuzzing \ \ - crates/weaveffi-fuzz/fuzz/artifacts//crash- + crates/weaveffi-fuzz/artifacts//crash- ``` 3. Convert the minimized input into a regression test in `weaveffi-model` - (which owns the parsers and the validator) **before** fixing the bug, so - the failure is locked in and can't regress. + (which owns the parsers and the validator) or `weaveffi` (which owns the + value-buffer decoders) **before** fixing the bug, so the failure is + locked in and can't regress. ## Commit conventions @@ -285,14 +303,19 @@ fix/android-jni-crash - **CI** (`ci.yml`): formatting, clippy, rustdoc, and a build of `weaveffi-model` without its IDL features; the test suite with snapshot checks on Linux, macOS, and Windows; `weaveffi diff --check` on every - sample, the JSON Schema drift check, and a `wasm32` build of every sample; - the fixture compile check per target; the conformance harness per language - on Linux and macOS; and an Android NDK link of the Kotlin JNI shim. + sample, the JSON Schema drift check, and a `wasm32` build of every sample + whose embedded metadata must match the native build's; the fixture compile + check per target; the conformance harness per language on Linux and macOS; + an Android job that links the Kotlin JNI shim with the NDK and runs + `weaveffi build` for Android; and a package install smoke test + (`scripts/package-smoke.sh`) on Linux and macOS. - **Quality** (`quality.yml`): `cargo deny`, `cargo audit`, `cargo machete`, coverage with `cargo llvm-cov`, and the docs link check. - **Docs** (`docs.yml`): builds and deploys the mdBook and rustdoc. -- **Bench** (`bench.yml`), **Fuzz** (`fuzz.yml`): benchmark and fuzzing - runs. +- **Bench** (`bench.yml`): a weekly sampled benchmark run, uploaded as an + artifact. Pull requests run each benchmark once in `ci.yml`. +- **Fuzz** (`fuzz.yml`): a daily 60-second run of every fuzz target from a + corpus seeded with the committed seeds and the snapshot fixtures. - **PR Lint** (`pr-lint.yml`): checks the PR title and commit messages against Conventional Commits. - **Release** (`release.yml`) and **Release binaries** diff --git a/Cargo.lock b/Cargo.lock index e01fdd96..03aaa2e4 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -109,13 +109,6 @@ dependencies = [ "wait-timeout", ] -[[package]] -name = "async-demo" -version = "0.1.0" -dependencies = [ - "weaveffi", -] - [[package]] name = "autocfg" version = "1.5.1" @@ -131,7 +124,7 @@ dependencies = [ "addr2line", "cfg-if", "libc", - "miniz_oxide", + "miniz_oxide 0.8.9", "object", "rustc-demangle", "windows-link", @@ -333,13 +326,6 @@ dependencies = [ "windows-sys", ] -[[package]] -name = "contacts" -version = "0.1.0" -dependencies = [ - "weaveffi", -] - [[package]] name = "cpufeatures" version = "0.2.17" @@ -349,6 +335,15 @@ dependencies = [ "libc", ] +[[package]] +name = "crc32fast" +version = "1.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78" +dependencies = [ + "cfg-if", +] + [[package]] name = "criterion" version = "0.5.1" @@ -487,25 +482,39 @@ dependencies = [ "windows-sys", ] -[[package]] -name = "events" -version = "0.1.0" -dependencies = [ - "weaveffi", -] - [[package]] name = "fastrand" version = "2.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" +[[package]] +name = "filetime" +version = "0.2.29" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c287a33c7f0a620c38e641e7f60827713987b3c0f26e8ddc9462cc69cf75759" +dependencies = [ + "cfg-if", + "libc", +] + [[package]] name = "find-msvc-tools" version = "0.1.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d" +[[package]] +name = "flate2" +version = "1.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb" +dependencies = [ + "crc32fast", + "miniz_oxide 0.9.1", + "zlib-rs", +] + [[package]] name = "float-cmp" version = "0.10.0" @@ -885,6 +894,16 @@ dependencies = [ "adler2", ] +[[package]] +name = "miniz_oxide" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c" +dependencies = [ + "adler2", + "simd-adler32", +] + [[package]] name = "normalize-line-endings" version = "0.3.0" @@ -907,6 +926,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ff76201f031d8863c38aa7f905eca4f53abbfa15f609db4277d44cd8938f33fe" dependencies = [ "memchr", + "wasmparser", ] [[package]] @@ -1026,6 +1046,13 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "producer-fixture" +version = "0.3.0" +dependencies = [ + "weaveffi", +] + [[package]] name = "proptest" version = "1.11.0" @@ -1338,6 +1365,12 @@ version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" +[[package]] +name = "simd-adler32" +version = "0.3.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea" + [[package]] name = "similar" version = "2.7.0" @@ -1422,6 +1455,16 @@ dependencies = [ "syn 3.0.6", ] +[[package]] +name = "tar" +version = "0.4.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f6221d9a6003c78398e3b239969f352578258df48c8eb051caadae0015bc840" +dependencies = [ + "filetime", + "libc", +] + [[package]] name = "target-tuple" version = "1.0.2" @@ -1517,6 +1560,27 @@ dependencies = [ "serde_json", ] +[[package]] +name = "tokio" +version = "1.53.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e95f91fcc7a621e8b030f6aa23c71fe9838ae2fb4d8118b75602a328f5144044" +dependencies = [ + "pin-project-lite", + "tokio-macros", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.6", +] + [[package]] name = "toml" version = "0.8.23" @@ -1740,19 +1804,22 @@ dependencies = [ ] [[package]] -name = "weaveffi" -version = "0.24.0" +name = "wasmparser" +version = "0.236.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9b1e81f3eb254cf7404a82cee6926a4a3ccc5aad80cc3d43608a070c67aa1d7" dependencies = [ - "weaveffi-abi", - "weaveffi-gen", - "weaveffi-macros", + "bitflags", ] [[package]] -name = "weaveffi-abi" +name = "weaveffi" version = "0.24.0" dependencies = [ "proptest", + "tokio", + "weaveffi", + "weaveffi-macros", ] [[package]] @@ -1764,17 +1831,25 @@ dependencies = [ "camino", "clap", "clap_complete", + "crc32fast", + "criterion", + "flate2", + "heck", "insta", "miette", + "object", "predicates", + "rayon", "schemars", "serde", "serde_json", "serde_yaml", + "sha2", "similar", + "tar", "tempfile", "toml 0.8.23", - "weaveffi-gen", + "weaveffi", "weaveffi-model", ] @@ -1783,23 +1858,7 @@ name = "weaveffi-fuzz" version = "0.24.0" dependencies = [ "libfuzzer-sys", - "weaveffi-model", -] - -[[package]] -name = "weaveffi-gen" -version = "0.24.0" -dependencies = [ - "anyhow", - "camino", - "criterion", - "heck", - "rayon", - "serde", - "serde_json", - "sha2", - "tempfile", - "thiserror", + "weaveffi", "weaveffi-model", ] @@ -1821,6 +1880,7 @@ version = "0.24.0" dependencies = [ "heck", "miette", + "proc-macro2", "schemars", "serde", "serde_json", @@ -1988,6 +2048,12 @@ dependencies = [ "syn 3.0.6", ] +[[package]] +name = "zlib-rs" +version = "0.6.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112" + [[package]] name = "zmij" version = "1.0.23" diff --git a/Cargo.toml b/Cargo.toml index 9f1efcf9..b63cfa6f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,17 +1,13 @@ [workspace] members = [ "crates/weaveffi-model", - "crates/weaveffi-abi", "crates/weaveffi-macros", "crates/weaveffi", - "crates/weaveffi-gen", "crates/weaveffi-cli", "samples/calculator", - "samples/contacts", - "samples/async-demo", - "samples/events", "samples/kvstore", "samples/codec", + "crates/weaveffi-cli/tests/fixtures/producer", "crates/weaveffi-fuzz", ] resolver = "2" @@ -31,10 +27,8 @@ description = "WeaveFFI: Rust core with stable C ABI and multi-language bindings [workspace.dependencies] weaveffi = { path = "crates/weaveffi", version = "0.24.0" } -weaveffi-abi = { path = "crates/weaveffi-abi", version = "0.24.0" } weaveffi-macros = { path = "crates/weaveffi-macros", version = "0.24.0" } weaveffi-model = { path = "crates/weaveffi-model", version = "0.24.0", default-features = false } -weaveffi-gen = { path = "crates/weaveffi-gen", version = "0.24.0" } serde = { version = "1.0", features = ["derive"] } serde_yaml = "0.9" serde_json = "1.0" @@ -44,7 +38,6 @@ anyhow = "1.0" clap = { version = "4.5", features = ["derive"] } clap_complete = "4.5" camino = { version = "1.1", features = ["serde1"] } -sha2 = "0.10" heck = "0.5" syn = { version = "2.0", features = ["full", "parsing", "visit"] } quote = "1.0" diff --git a/README.md b/README.md index c4de2e91..03b5209f 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # WeaveFFI -[![CI](https://github.com/weavefoundry/weaveffi/actions/workflows/ci.yml/badge.svg)](https://github.com/weavefoundry/weaveffi/actions/workflows/ci.yml) [![License: MIT OR Apache-2.0](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue)](LICENSE-MIT) [![crates.io](https://img.shields.io/crates/v/weaveffi-cli.svg)](https://crates.io/crates/weaveffi-cli) [![Schema](https://img.shields.io/badge/schema-0.10.0-orange)](./weaveffi.schema.json) [![C ABI](https://img.shields.io/badge/C%20ABI-3-orange)](docs/src/reference/abi.md) +[![CI](https://github.com/weavefoundry/weaveffi/actions/workflows/ci.yml/badge.svg)](https://github.com/weavefoundry/weaveffi/actions/workflows/ci.yml) [![License: MIT OR Apache-2.0](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue)](LICENSE-MIT) [![crates.io](https://img.shields.io/crates/v/weaveffi-cli.svg)](https://crates.io/crates/weaveffi-cli) [![Schema](https://img.shields.io/badge/schema-0.11.0-orange)](./weaveffi.schema.json) [![C ABI](https://img.shields.io/badge/C%20ABI-4-orange)](docs/src/reference/abi.md) WeaveFFI generates idiomatic, type-safe bindings for 11 languages (C, C++, Swift, Kotlin, Node.js, WebAssembly, Python, .NET, Dart, Go, and Ruby) for any @@ -26,8 +26,9 @@ What you get, in every language: abstract class the consumer implements and the library calls from any thread. - **Load-time safety.** Each generated package checks the library's ABI - revision and a per-module contract checksum before the first call, so a - stale binding fails with a clear error instead of corrupting memory. + revision and its contract table before the first call, so a stale binding + fails with an error naming the declaration that changed instead of + corrupting memory. ## Quickstart @@ -37,8 +38,15 @@ What you get, in every language: cargo install weaveffi-cli ``` -**2. Write the library.** In a Rust crate with `crate-type = ["cdylib"]` and -`cargo add weaveffi`: +**2. Write the library.** + +```bash +cargo new --lib kvstore && cd kvstore +cargo add weaveffi +``` + +Add `crate-type = ["cdylib"]` under `[lib]` in `Cargo.toml`, then put this in +`src/lib.rs`: ```rust #[weaveffi::module] @@ -99,39 +107,51 @@ weaveffi::export_runtime!(); The macro emits the `extern "C"` functions; you write no `unsafe` code. An interface type must be `Send + Sync`, because the object is shared across the -boundary as an `Arc`. +boundary as an `Arc`. This is a trimmed-down store; the full +[`samples/kvstore`](samples/kvstore/src/lib.rs) adds records, enums, +callbacks, iterators, async, and the rest (see [Samples](docs/src/samples.md)). -**3. Generate bindings.** +**3. Generate bindings.** The package name, the C prefix (`kvstore_`), and +the library name all come from the crate. ```bash -weaveffi init # writes weaveffi.toml: [project] input = "src/lib.rs" -weaveffi generate # every target, into ./bindings -cargo build # builds libkvstore +weaveffi init # writes weaveffi.toml: [project] input = "." (this crate) +weaveffi generate # builds libkvstore, reads its API, writes every target to ./bindings ``` +`weaveffi generate` builds the crate and reads the API from the library it +compiled (the macro embeds it), so the bindings always describe exactly what +the library exports. While iterating, use `weaveffi dev` instead: it does the +same and also copies the fresh debug library into the generated Python, +Node.js, and Ruby packages (and prints how to point the other targets at it). + **4. Use them.** From Python, for example: +```bash +weaveffi dev # regenerate and copy libkvstore into bindings/python/kvstore +pip install ./bindings/python +``` + ```python import kvstore with kvstore.Store.open("data.kv") as store: store.put("greeting", b"hello") print(store.count()) # 1 - -try: - ... -except kvstore.StoreFull as e: - print(e.code, e.message) # 1003 store has reached capacity + try: + store.put("another", b"value") + except kvstore.StoreFull as e: + print(e.code, e.message) # 1003 store has reached capacity ```
The C header every target binds to (bindings/c/kvstore.h) ```c -#define KVSTORE_ABI_VERSION 3u +#define KVSTORE_ABI_VERSION 4u KVSTORE_API uint32_t kvstore_abi_version(void); -#define KVSTORE_KV_CHECKSUM 0xf6b8170a59ae07a7ull -KVSTORE_API uint64_t kvstore_kv_checksum(void); +KVSTORE_API const kvstore_contract_entry* kvstore_kv_contract(size_t* out_len); +#define KVSTORE_KV_CONTRACT_LEN 5 typedef enum { kvstore_kv_KvError_KeyNotFound = 1001, @@ -172,19 +192,28 @@ public final class Store: @unchecked Sendable {
-Python type stubs (bindings/python/kvstore/kvstore.pyi) +Python, fully annotated (bindings/python/kvstore/kvstore.py) ```python -class KvError(Error): ... -class KeyNotFound(KvError): ... -class StoreFull(KvError): ... - -class Store: - def close(self) -> None: ... - def __enter__(self) -> "Store": ... - def __exit__(self, *exc: object) -> None: ... +class KvError(Error): + """Base exception for the `kv` module's error domain.""" + +class KeyNotFound(KvError): + CODE = 1001 + +class StoreFull(KvError): + CODE = 1003 + +class Store(_Object): # close() it, or use `with` @classmethod - def open(cls, path: str) -> "Store": ... + def open(cls, path: str) -> Store: + _path_b = path.encode("utf-8") + _err = _ErrorStruct() + _ret = _c_kv_Store_open(_path_b, len(_path_b), ctypes.byref(_err)) + if _err.code: + raise _kv_error_from(*_read_error(_err)) + return cls._adopt(_required(_ret)) + def put(self, key: str, value: bytes) -> bool: ... def count(self) -> int: ... ``` @@ -206,7 +235,7 @@ Started](docs/src/getting-started.md). | Kotlin | Gradle (Android or JVM) | `AutoCloseable` + cleaner | `suspend` | coroutine cancellation | `interface` | | Node.js | npm package (N-API addon) | `close()`, `Symbol.dispose` | `Promise` | `AbortSignal` | object | | WebAssembly | npm package (ESM) | `close()`, `Symbol.dispose` | `Promise` | `AbortSignal` | object | -| Python | `pyproject.toml` package + `.pyi` | `close()`, `with` | awaitable | task cancellation | ABC | +| Python | `pyproject.toml` package (typed, `py.typed`) | `close()`, `with` | awaitable | task cancellation | ABC | | .NET | `.csproj` (NuGet) | `IDisposable` over `SafeHandle` | `Task` | `CancellationToken` | interface | | Dart | pub package | `dispose()` + `NativeFinalizer` | `Future` | `CancelToken` | abstract class | | Go | Go module (cgo) | `Close() error` | blocking call | `context.Context` | interface | @@ -231,11 +260,13 @@ release](https://github.com/weavefoundry/weaveffi/releases). | Command | What it does | |---|---| | `weaveffi init [dir]` | Write `weaveffi.toml` (and a starter IDL outside a Rust crate) | -| `weaveffi generate [input]` | Generate bindings; `--target c,swift` to subset, `--dry-run` to list files. Only changed files are rewritten, and files a previous run produced that are no longer generated are removed | +| `weaveffi generate [input]` | Generate bindings from a Rust producer crate (built first; `--library ` reads an existing build) or an IDL; `--target c,swift` to subset, `--dry-run` to list files. Only changed files are rewritten, and files a previous run produced that are no longer generated are removed | | `weaveffi validate [input]` | Validate without generating; `--warn` for lints, `--format json` | | `weaveffi diff [input]` | Show what regenerating would change; `--check` exits non-zero for CI | -| `weaveffi package [input]` | Assemble publishable packages that bundle prebuilt native libraries | -| `weaveffi extract ` | Print the IDL derived from annotated Rust | +| `weaveffi build [input]` | Cross-compile the Rust producer per platform (`--platforms`) into `target/weaveffi//`, with prebuilt Node.js and JNI glue | +| `weaveffi package [input]` | Build, then write installable artifacts per target to `dist/`: wheels, npm tarballs, gems, an `XCFramework` SwiftPM package, a `.nupkg`, and more | +| `weaveffi dev [input]` | Build the producer's debug library, generate, and point the bindings at the library | +| `weaveffi extract [input]` | Print the IDL a Rust producer's built library embeds (`--library ` to read a given build) | | `weaveffi schema` | Print the JSON Schema of the IDL | With a `[project]` table in `weaveffi.toml`, every command works without @@ -243,7 +274,7 @@ arguments from anywhere in the project: ```toml [project] -input = "src/lib.rs" # or an IDL such as "kvstore.yml" +input = "." # this crate, or an IDL such as "kvstore.yml" out = "bindings" targets = ["c", "swift", "kotlin", "python"] @@ -252,7 +283,7 @@ version = "1.0.0" license = "MIT" [generators.swift] -module_name = "KVStore" +name = "KVStore" ``` A Rust producer's package name, C prefix, and library name come from its @@ -273,7 +304,7 @@ others. ## Status -WeaveFFI is pre-1.0: the IDL schema is `0.10.0` and the C ABI is revision 3, +WeaveFFI is pre-1.0: the IDL schema is `0.11.0` and the C ABI is revision 4, and either may change in a minor release. See [Stability and Versioning](docs/src/stability.md) for the policy and the migration notes, and the [Roadmap](docs/src/roadmap.md) for what's next. diff --git a/SECURITY.md b/SECURITY.md index ca58776f..d6df903a 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -60,10 +60,12 @@ Expected flow: Security-sensitive areas include: -- The `weaveffi-abi` runtime and generated C ABI ownership rules. +- The `weaveffi::abi` runtime and generated C ABI ownership rules. - Generated bindings that manage pointers, callbacks, async contexts, or language-runtime handles. -- The parser, validator, and `weaveffi extract` input handling. -- CLI behavior that reads project files, runs hooks, or writes generated output. +- The parser and validator, and the library-metadata reader behind + `weaveffi generate` and `weaveffi extract` (which parses built libraries). +- CLI behavior that reads project files, builds a producer crate with Cargo, + or writes generated output and packages. Bug reports outside this scope are still welcome through normal GitHub issues. diff --git a/conformance/c/async_demo.c b/conformance/c/async_demo.c deleted file mode 100644 index 66883e34..00000000 --- a/conformance/c/async_demo.c +++ /dev/null @@ -1,245 +0,0 @@ -// Conformance consumer: async-demo sample, C target (ABI revision 3). -// -// Exercises the raw async launcher convention: each async function's symbol -// takes its inputs, a completion callback, and a context pointer, returns at -// once, and fires the callback exactly once from a producer thread. Buffered -// results (the TaskResult record, the list-of-records batch) arrive as -// consumer-owned (ptr, len) runs released with async_demo_free_bytes; a -// non-null error is heap-boxed and released with async_demo_error_free. -// The cancellable `wait` takes a cancel token: cancelling it completes the -// call with code -5 long before its timeout, whether the token fires before -// or after launch, and the consumer may destroy its token reference at any -// time. Also covers many concurrent launches, the plain sync function beside -// the async ones, and the producer's leak counters settling to zero. - -#include "harness.h" - -#include - -#include "async_demo_buffer.h" - -#define CANCELLED (-5) - -// One in-flight call's completion slot; the callback runs on a producer -// thread, so the main thread polls `done`. -typedef struct { - atomic_int done; - atomic_int fired; // completions observed (must end at exactly 1) - int32_t code; - char message[96]; - int64_t i64; - int32_t i32; - uint8_t* buf; // owned result run - size_t buf_len; -} call; - -static void call_reset(call* c) { - memset(c, 0, sizeof *c); - atomic_init(&c->done, 0); - atomic_init(&c->fired, 0); -} - -static void record_err(call* c, async_demo_error* err) { - c->code = err ? err->code : 0; - if (err && err->message) snprintf(c->message, sizeof c->message, "%s", err->message); - async_demo_error_free(err); -} - -static void finish(call* c) { - atomic_fetch_add(&c->fired, 1); - atomic_store(&c->done, 1); -} - -static void on_buffer(void* context, async_demo_error* err, const uint8_t* result_ptr, - size_t result_len) { - call* c = (call*)context; - record_err(c, err); - // The result run is ours: keep it and release it after decoding. - c->buf = (uint8_t*)result_ptr; - c->buf_len = result_len; - finish(c); -} - -static void on_i64(void* context, async_demo_error* err, int64_t result) { - call* c = (call*)context; - record_err(c, err); - c->i64 = result; - finish(c); -} - -static void on_i32(void* context, async_demo_error* err, int32_t result) { - call* c = (call*)context; - record_err(c, err); - c->i32 = result; - finish(c); -} - -// Wait for a completion, failing after `limit_ms`. Returns the elapsed time. -static long await_call(call* c, long limit_ms) { - long waited = 0; - while (!atomic_load(&c->done)) { - assert(waited < limit_ms && "async call did not complete in time"); - sleep_ms(1); - waited++; - } - assert(atomic_load(&c->fired) == 1); - return waited; -} - -static void run_task(void) { - call c; - call_reset(&c); - async_demo_tasks_run_task(STR("alpha"), on_buffer, &c); - await_call(&c, 5000); - assert(c.code == 0 && c.buf != NULL); - async_demo_tasks_TaskResult r; - assert(async_demo_tasks_TaskResult_decode(c.buf, c.buf_len, &r)); - async_demo_free_bytes(c.buf, c.buf_len); - assert(r.id > 0); - assert(strcmp(r.value.ptr, "completed: alpha") == 0); - assert(r.success); - async_demo_tasks_TaskResult_free(&r); - - // Typed async error: the empty name reports InvalidName with its message - // and no result run. - call_reset(&c); - async_demo_tasks_run_task(NULL, 0, on_buffer, &c); - await_call(&c, 5000); - assert(c.code == async_demo_tasks_TaskError_InvalidName); - assert(strcmp(c.message, "task name must not be empty") == 0); - assert(c.buf == NULL && c.buf_len == 0); -} - -static void run_batch(void) { - async_demo_str names[3] = {async_demo_str_of("a"), async_demo_str_of("b"), - async_demo_str_of("c")}; - async_demo_list_string in; - in.items = names; - in.len = 3; - async_demo_writer w; - memset(&w, 0, sizeof w); - async_demo_list_string_write(&w, &in); - call c; - call_reset(&c); - async_demo_tasks_run_batch(w.ptr, w.len, on_buffer, &c); - // Inputs are lifted before the launcher returns, so the buffer can go now. - async_demo_writer_free(&w); - await_call(&c, 5000); - assert(c.code == 0); - async_demo_list_tasks_TaskResult out; - assert(async_demo_list_tasks_TaskResult_decode(c.buf, c.buf_len, &out)); - async_demo_free_bytes(c.buf, c.buf_len); - assert(out.len == 3); - const char* expected[3] = {"completed: a", "completed: b", "completed: c"}; - for (size_t i = 0; i < 3; i++) { - assert(strcmp(out.items[i].value.ptr, expected[i]) == 0); - assert(out.items[i].success); - } - assert(out.items[0].id < out.items[1].id && out.items[1].id < out.items[2].id); - async_demo_list_tasks_TaskResult_free(&out); -} - -static void run_n_tasks(void) { - // Many concurrent launches, each with its own context: every completion - // fires exactly once with its own value. - enum { N = 64 }; - static call calls[N]; - for (int i = 0; i < N; i++) { - call_reset(&calls[i]); - async_demo_tasks_run_n_tasks(i, on_i32, &calls[i]); - } - for (int i = 0; i < N; i++) { - await_call(&calls[i], 5000); - assert(calls[i].code == 0 && calls[i].i32 == i); - } -} - -static void wait_and_cancel(void) { - call c; - - // No token: wait runs to its (short) timeout. - call_reset(&c); - async_demo_tasks_wait(5, NULL, on_i64, &c); - await_call(&c, 5000); - assert(c.code == 0 && c.i64 == 5); - - // A token that is never cancelled changes nothing. - async_demo_cancel_token* token = async_demo_cancel_token_create(); - assert(token != NULL && !async_demo_cancel_token_is_cancelled(token)); - call_reset(&c); - async_demo_tasks_wait(10, token, on_i64, &c); - await_call(&c, 5000); - assert(c.code == 0 && c.i64 == 10); - async_demo_cancel_token_destroy(token); - - // Cancel mid-flight: a one-minute wait completes with -5 almost at once. - // The consumer drops its token reference right after cancelling; the - // producer holds its own until the call completes. - token = async_demo_cancel_token_create(); - call_reset(&c); - async_demo_tasks_wait(60000, token, on_i64, &c); - sleep_ms(20); - assert(!atomic_load(&c.done) && "the wait is still pending"); - async_demo_cancel_token_cancel(token); - async_demo_cancel_token_cancel(token); // idempotent - assert(async_demo_cancel_token_is_cancelled(token)); - async_demo_cancel_token_destroy(token); - long waited = await_call(&c, 5000); - assert(waited < 5000); - assert(c.code == CANCELLED); - assert(c.message[0] != '\0'); - - // Cancelled before launch: completes with -5 without waiting. - token = async_demo_cancel_token_create(); - async_demo_cancel_token_cancel(token); - call_reset(&c); - async_demo_tasks_wait(60000, token, on_i64, &c); - await_call(&c, 5000); - assert(c.code == CANCELLED); - async_demo_cancel_token_destroy(token); - - // One token cancels several in-flight calls at once. - token = async_demo_cancel_token_create(); - call many[3]; - for (int i = 0; i < 3; i++) { - call_reset(&many[i]); - async_demo_tasks_wait(60000, token, on_i64, &many[i]); - } - async_demo_cancel_token_cancel(token); - for (int i = 0; i < 3; i++) { - await_call(&many[i], 5000); - assert(many[i].code == CANCELLED); - } - async_demo_cancel_token_destroy(token); - - // Destroying the consumer's reference is not cancellation: the call - // keeps its own reference and runs to its timeout. - token = async_demo_cancel_token_create(); - call_reset(&c); - async_demo_tasks_wait(100, token, on_i64, &c); - async_demo_cancel_token_destroy(token); - await_call(&c, 5000); - assert(c.code == 0 && c.i64 == 100); -} - -int main(void) { - async_demo_error err = {0}; - - assert(ASYNC_DEMO_ABI_VERSION == 3u); - assert(async_demo_abi_version() == ASYNC_DEMO_ABI_VERSION); - assert(async_demo_tasks_checksum() == ASYNC_DEMO_TASKS_CHECKSUM); - - run_task(); - run_batch(); - run_n_tasks(); - wait_and_cancel(); - - // Every task body, cancelled ones included, has been dropped. - for (int i = 0; i < 2000 && async_demo_tasks_active_callbacks(&err) != 0; i++) sleep_ms(1); - assert(async_demo_tasks_active_callbacks(&err) == 0); - assert(err.code == 0); - - ASSERT_NO_LEAKS(async_demo_debug_live); - printf("c/async-demo: OK\n"); - return 0; -} diff --git a/conformance/c/calculator.c b/conformance/c/calculator.c new file mode 100644 index 00000000..deb1ed01 --- /dev/null +++ b/conformance/c/calculator.c @@ -0,0 +1,58 @@ +// Conformance consumer: calculator sample, C target (ABI revision 4). +// +// The getting-started surface: a direct-value call, a throwing call that +// reports the typed `CalcError` domain, and a string in and out (non-ASCII +// text and an interior NUL byte survive). Ends by asserting the producer's +// leak counters are zero. + +#include "harness.h" + +#include "calculator.h" + +static void greet(const void* name, size_t name_len, const void* expected, size_t expected_len) { + calculator_error err = {0}; + size_t len = 0; + const uint8_t* out = + calculator_calculator_greet((const uint8_t*)name, name_len, &len, &err); + assert(err.code == 0); + assert(len == expected_len && memcmp(out, expected, len) == 0); + calculator_free_bytes((uint8_t*)out, len); +} + +int main(void) { + calculator_error err = {0}; + + assert(calculator_abi_version() == CALCULATOR_ABI_VERSION); + assert(calculator_calculator_contract_check() == 0); + + assert(calculator_calculator_add(2, 3, &err) == 5 && err.code == 0); + assert(calculator_calculator_add(-7, 7, &err) == 0); + assert(calculator_calculator_add(INT32_MAX, 1, &err) == INT32_MIN && "wraps"); + + assert(calculator_calculator_divide(10, 2, &err) == 5 && err.code == 0); + assert(calculator_calculator_divide(-7, 2, &err) == -3 && "rounds toward zero"); + assert(calculator_calculator_divide(INT32_MIN, -1, &err) == INT32_MIN); + + assert(calculator_calculator_divide(1, 0, &err) == 0); + assert(err.code == calculator_calculator_CalcError_DivisionByZero); + assert(strcmp(err.message, "division by zero") == 0); + assert(err.payload_ptr == NULL && err.payload_len == 0); + calculator_error_clear(&err); + assert(err.code == 0 && err.message == NULL); + + greet(STR("World"), STR("Hello, World!")); + greet(NULL, 0, STR("Hello, !")); + greet(STR("W\xc3\xb6rld \xf0\x9f\xa6\x80"), STR("Hello, W\xc3\xb6rld \xf0\x9f\xa6\x80!")); + greet("a\0b", 3, "Hello, a\0b!", 11); + + // Invalid input is a marshalling failure (-3), not a crash. + size_t len = 0; + const uint8_t bad_utf8[2] = {0xC3, 0x28}; + assert(calculator_calculator_greet(BYTES(bad_utf8), &len, &err) == NULL); + assert(err.code == -3); + calculator_error_clear(&err); + + ASSERT_NO_LEAKS(calculator_debug_live); + printf("c/calculator: OK\n"); + return 0; +} diff --git a/conformance/c/codec.c b/conformance/c/codec.c index 7b18b6ae..32019734 100644 --- a/conformance/c/codec.c +++ b/conformance/c/codec.c @@ -1,18 +1,15 @@ -// Conformance consumer: codec sample, C target (ABI revision 3). +// Conformance consumer: codec sample, C target (ABI revision 4). // -// A round-trip check of the value-buffer protocol between the producer's own -// codec and the generated `codec_buffer.h`: every fixed-width scalar, -// strings (non-ASCII and with interior NUL bytes), bytes, present and absent -// optionals, lists (nested and empty), string- and integer-keyed maps with -// record values, nested records, every rich-enum variant, lists of optionals -// and enums, and objects in a record field, an optional, and a list. -// Fixtures fetched from the producer are checked field by field (producer -// encodes, consumer decodes), handed back to `verify_*` (consumer encodes, -// producer decodes), and re-encoded through `roundtrip_*`, which must return -// the very same bytes; hand-built edge values (empty containers, 64-bit -// extremes, NaN, infinities, negative zero) take the same path. Malformed -// input is rejected on both sides. Ends by asserting the producer's leak -// counters are zero. +// The shared-vector loop: for every vector the producer serves, decode it +// with the generated `codec_buffer.h`, re-encode it (which must reproduce the +// producer's bytes exactly, objects aside), pass it back to `check_vector`, +// and push each primitive vector's value through the matching direct-family +// `echo_*`. Then vectors built from literals (so a symmetric encode/decode +// bug can't hide), spot checks of decoded fields, the typed out-of-range +// error and its payload, malformed buffers (truncated, unknown tag, trailing +// bytes, a bad bool, a duplicate map key) rejected as marshalling failures, +// and object identity and reference counting through buffers. Ends by +// asserting the producer's leak counters are zero. #include "harness.h" @@ -20,720 +17,515 @@ #include "codec_buffer.h" -// ── comparisons ──────────────────────────────────────────────────────────── - -// Bitwise float equality, so NaN payloads and the sign of zero count. static int f32_bits_eq(float a, float b) { return memcmp(&a, &b, 4) == 0; } static int f64_bits_eq(double a, double b) { return memcmp(&a, &b, 8) == 0; } -static int str_eq(codec_str a, codec_str b) { - return a.len == b.len && (a.len == 0 || memcmp(a.ptr, b.ptr, a.len) == 0); -} - -static int bytes_view_eq(codec_bytes a, codec_bytes b) { - return a.len == b.len && (a.len == 0 || memcmp(a.ptr, b.ptr, a.len) == 0); -} - -static int opt_str_eq(const codec_str* a, const codec_str* b) { - return (a == NULL) == (b == NULL) && (a == NULL || str_eq(*a, *b)); -} - -static int scalars_eq(const codec_codec_Scalars* a, const codec_codec_Scalars* b) { - return a->i8_value == b->i8_value && a->u8_value == b->u8_value && - a->i16_value == b->i16_value && a->u16_value == b->u16_value && - a->i32_value == b->i32_value && a->u32_value == b->u32_value && - a->i64_value == b->i64_value && a->u64_value == b->u64_value && - f32_bits_eq(a->f32_value, b->f32_value) && f64_bits_eq(a->f64_value, b->f64_value) && - a->flag == b->flag && a->color == b->color; +static int str_is(codec_str s, const char* expected, size_t n) { + return s.len == n && (n == 0 || memcmp(s.ptr, expected, n) == 0); } -static int shape_eq(const codec_codec_Shape* a, const codec_codec_Shape* b) { - if (a->tag != b->tag) return 0; - switch (a->tag) { - case codec_codec_Shape_Empty: - return 1; - case codec_codec_Shape_Circle: - return f64_bits_eq(a->as.Circle.radius, b->as.Circle.radius); - case codec_codec_Shape_Rect: - return f32_bits_eq(a->as.Rect.width, b->as.Rect.width) && - f32_bits_eq(a->as.Rect.height, b->as.Rect.height); - case codec_codec_Shape_Labeled: - return str_eq(a->as.Labeled.label, b->as.Labeled.label) && - a->as.Labeled.count == b->as.Labeled.count; - case codec_codec_Shape_Nested: - return scalars_eq(&a->as.Nested.inner, &b->as.Nested.inner) && - opt_str_eq(a->as.Nested.note, b->as.Nested.note); - default: - return 0; - } -} - -static int composite_eq(const codec_codec_Composite* a, const codec_codec_Composite* b) { - if (!str_eq(a->name, b->name) || !bytes_view_eq(a->blob, b->blob)) return 0; - if ((a->some_i64 == NULL) != (b->some_i64 == NULL)) return 0; - if (a->some_i64 && *a->some_i64 != *b->some_i64) return 0; - if ((a->none_i64 == NULL) != (b->none_i64 == NULL)) return 0; - if (a->none_i64 && *a->none_i64 != *b->none_i64) return 0; - if (!opt_str_eq(a->some_text, b->some_text)) return 0; - if (a->names.len != b->names.len) return 0; - for (size_t i = 0; i < a->names.len; i++) { - if (!str_eq(a->names.items[i], b->names.items[i])) return 0; - } - if (a->matrix.len != b->matrix.len) return 0; - for (size_t i = 0; i < a->matrix.len; i++) { - const codec_list_i32* x = &a->matrix.items[i]; - const codec_list_i32* y = &b->matrix.items[i]; - if (x->len != y->len) return 0; - for (size_t j = 0; j < x->len; j++) { - if (x->items[j] != y->items[j]) return 0; - } - } - if (a->empty.len != b->empty.len) return 0; - for (size_t i = 0; i < a->empty.len; i++) { - if (!f64_bits_eq(a->empty.items[i], b->empty.items[i])) return 0; - } - if (a->by_name.len != b->by_name.len) return 0; - for (size_t i = 0; i < a->by_name.len; i++) { - if (!str_eq(a->by_name.keys[i], b->by_name.keys[i]) || - a->by_name.values[i] != b->by_name.values[i]) { - return 0; - } - } - if (a->by_id.len != b->by_id.len) return 0; - for (size_t i = 0; i < a->by_id.len; i++) { - if (a->by_id.keys[i] != b->by_id.keys[i] || - !scalars_eq(&a->by_id.values[i], &b->by_id.values[i])) { - return 0; - } - } - if (!scalars_eq(&a->scalars, &b->scalars) || !shape_eq(&a->shape, &b->shape)) return 0; - if (a->shapes.len != b->shapes.len) return 0; - for (size_t i = 0; i < a->shapes.len; i++) { - if (!shape_eq(&a->shapes.items[i], &b->shapes.items[i])) return 0; - } - if ((a->maybe_shape == NULL) != (b->maybe_shape == NULL)) return 0; - if (a->maybe_shape && !shape_eq(a->maybe_shape, b->maybe_shape)) return 0; - if ((a->maybe_list == NULL) != (b->maybe_list == NULL)) return 0; - if (a->maybe_list && !bytes_view_eq(*a->maybe_list, *b->maybe_list)) return 0; - if (a->sparse.len != b->sparse.len) return 0; - for (size_t i = 0; i < a->sparse.len; i++) { - const bool* x = a->sparse.items[i]; - const bool* y = b->sparse.items[i]; - if ((x == NULL) != (y == NULL) || (x && *x != *y)) return 0; - } - if (a->colors.len != b->colors.len) return 0; - for (size_t i = 0; i < a->colors.len; i++) { - if (a->colors.items[i] != b->colors.items[i]) return 0; +// Fetch vector `i`, keeping the producer's encoding in `*raw` (owned). +static codec_codec_Vector fetch(uint32_t i, uint8_t** raw, size_t* raw_len) { + codec_error err = {0}; + size_t len = 0; + const uint8_t* buf = codec_codec_vector(i, &len, &err); + assert(err.code == 0 && buf != NULL); + codec_codec_Vector v; + assert(codec_codec_Vector_decode(buf, len, &v)); + if (raw != NULL) { + *raw = (uint8_t*)malloc(len); + memcpy(*raw, buf, len); + *raw_len = len; } - return 1; + codec_free_bytes((uint8_t*)buf, len); + return v; } -// The canonical fixture the producer hands out and expects back. -static const codec_codec_Scalars CANONICAL_SCALARS = { - -8, 200, -16000, 60000, -2000000000, 4000000000u, -9007199254740993LL, UINT64_MAX, - 1.5f, -2.25e100, true, codec_codec_Color_Blue, -}; - -// ── ABI helpers ──────────────────────────────────────────────────────────── - -// A function taking one buffered value and returning one buffered value. -typedef const uint8_t* (*buffer_fn)(const uint8_t*, size_t, size_t*, codec_error*); - -// Call `fn` with the bytes in `w` and return its owned result, asserting it -// succeeded and (when `exact`) echoed the input byte for byte. The caller -// decodes `*out_len` bytes and releases them with codec_free_bytes. -static const uint8_t* call_buffer(buffer_fn fn, codec_writer* w, size_t* out_len, int exact) { - assert(!w->failed); +static bool check(uint32_t i, const codec_codec_Vector* v) { codec_error err = {0}; - const uint8_t* p = fn(w->ptr, w->len, out_len, &err); - assert(err.code == 0 && p != NULL); - if (exact) assert(*out_len == w->len && memcmp(p, w->ptr, w->len) == 0); - codec_writer_free(w); - return p; + codec_writer w; + memset(&w, 0, sizeof w); + codec_codec_Vector_write(&w, v); + assert(!w.failed); + bool ok = codec_codec_check_vector(i, w.ptr, w.len, &err); + assert(err.code == 0); + codec_writer_free(&w); + return ok; } -#define ENCODE(write, value, w) \ - codec_writer w; \ - memset(&w, 0, sizeof w); \ - write(&w, value) - -// Decode an owned buffer with `decode`, then release it. -#define TAKE(decode, p, len, out) \ - do { \ - assert(decode(p, len, out)); \ - codec_free_bytes((uint8_t*)(p), len); \ - } while (0) +// An owned, NUL-terminated copy of a returned string. +static char* take_string(const uint8_t* ptr, size_t len) { + char* s = (char*)calloc(len + 1, 1); + if (len > 0) memcpy(s, ptr, len); + codec_free_bytes((uint8_t*)ptr, len); + return s; +} -static void fetch_scalars(codec_codec_Scalars* out) { +static char* vector_name(uint32_t i) { codec_error err = {0}; size_t len = 0; - const uint8_t* p = codec_codec_sample_scalars(&len, &err); + const uint8_t* p = codec_codec_vector_name(i, &len, &err); assert(err.code == 0); - TAKE(codec_codec_Scalars_decode, p, len, out); + return take_string(p, len); } -static void fetch_composite(codec_codec_Composite* out) { +static char* describe(const codec_codec_Vector* v) { codec_error err = {0}; + codec_writer w; + memset(&w, 0, sizeof w); + codec_codec_Vector_write(&w, v); size_t len = 0; - const uint8_t* p = codec_codec_sample_composite(&len, &err); + const uint8_t* p = codec_codec_describe_vector(w.ptr, w.len, &len, &err); + codec_writer_free(&w); assert(err.code == 0); - TAKE(codec_codec_Composite_decode, p, len, out); + return take_string(p, len); } -static void roundtrip_scalars(const codec_codec_Scalars* in, codec_codec_Scalars* out) { - ENCODE(codec_codec_Scalars_write, in, w); - size_t len = 0; - const uint8_t* p = call_buffer(codec_codec_roundtrip_scalars, &w, &len, 1); - TAKE(codec_codec_Scalars_decode, p, len, out); +static uint32_t find(uint32_t n, const char* name) { + for (uint32_t i = 0; i < n; i++) { + char* s = vector_name(i); + int hit = strcmp(s, name) == 0; + free(s); + if (hit) return i; + } + fprintf(stderr, "no vector named %s\n", name); + abort(); } -static void roundtrip_composite(const codec_codec_Composite* in, codec_codec_Composite* out) { - ENCODE(codec_codec_Composite_write, in, w); +// Push a primitive vector's value through its direct-family echo. +static void echo(const codec_codec_Vector* v) { + codec_error err = {0}; size_t len = 0; - const uint8_t* p = call_buffer(codec_codec_roundtrip_composite, &w, &len, 1); - TAKE(codec_codec_Composite_decode, p, len, out); + switch (v->tag) { + case codec_codec_Vector_I8: + assert(codec_codec_echo_i8(v->as.I8.value, &err) == v->as.I8.value); + break; + case codec_codec_Vector_U8: + assert(codec_codec_echo_u8(v->as.U8.value, &err) == v->as.U8.value); + break; + case codec_codec_Vector_I16: + assert(codec_codec_echo_i16(v->as.I16.value, &err) == v->as.I16.value); + break; + case codec_codec_Vector_U16: + assert(codec_codec_echo_u16(v->as.U16.value, &err) == v->as.U16.value); + break; + case codec_codec_Vector_I32: + assert(codec_codec_echo_i32(v->as.I32.value, &err) == v->as.I32.value); + break; + case codec_codec_Vector_U32: + assert(codec_codec_echo_u32(v->as.U32.value, &err) == v->as.U32.value); + break; + case codec_codec_Vector_I64: + assert(codec_codec_echo_i64(v->as.I64.value, &err) == v->as.I64.value); + break; + case codec_codec_Vector_U64: + assert(codec_codec_echo_u64(v->as.U64.value, &err) == v->as.U64.value); + break; + case codec_codec_Vector_F32: + assert(f32_bits_eq(codec_codec_echo_f32(v->as.F32.value, &err), v->as.F32.value)); + break; + case codec_codec_Vector_F64: + assert(f64_bits_eq(codec_codec_echo_f64(v->as.F64.value, &err), v->as.F64.value)); + break; + case codec_codec_Vector_Flag: + assert(codec_codec_echo_bool(v->as.Flag.value, &err) == v->as.Flag.value); + break; + case codec_codec_Vector_Hue: + assert(codec_codec_echo_color(v->as.Hue.value, &err) == v->as.Hue.value); + break; + case codec_codec_Vector_Text: { + codec_str s = v->as.Text.value; + const uint8_t* out = codec_codec_echo_text((const uint8_t*)s.ptr, s.len, &len, &err); + assert(len == s.len && (len == 0 || memcmp(out, s.ptr, len) == 0)); + codec_free_bytes((uint8_t*)out, len); + break; + } + case codec_codec_Vector_Blob: { + codec_bytes b = v->as.Blob.value; + const uint8_t* out = codec_codec_echo_blob(b.ptr, b.len, &len, &err); + assert(len == b.len && (len == 0 || memcmp(out, b.ptr, len) == 0)); + codec_free_bytes((uint8_t*)out, len); + break; + } + default: + break; + } + assert(err.code == 0); } -static void roundtrip_shape(const codec_codec_Shape* in, codec_codec_Shape* out) { - ENCODE(codec_codec_Shape_write, in, w); - size_t len = 0; - const uint8_t* p = call_buffer(codec_codec_roundtrip_shape, &w, &len, 1); - TAKE(codec_codec_Shape_decode, p, len, out); +static void every_vector(uint32_t n) { + for (uint32_t i = 0; i < n; i++) { + uint8_t* raw = NULL; + size_t raw_len = 0; + codec_codec_Vector v = fetch(i, &raw, &raw_len); + if (!check(i, &v)) { + char* name = vector_name(i); + char* seen = describe(&v); + fprintf(stderr, "vector %u (%s) did not round-trip; producer saw %s\n", i, name, + seen); + abort(); + } + assert(!check((i + 1) % n, &v) && "a vector never matches its neighbor"); + // The generated encoder is deterministic, so re-encoding reproduces + // the producer's bytes, except for object tokens (fresh references). + if (v.tag != codec_codec_Vector_Objects) { + codec_writer w; + memset(&w, 0, sizeof w); + codec_codec_Vector_write(&w, &v); + assert(w.len == raw_len && memcmp(w.ptr, raw, raw_len) == 0); + codec_writer_free(&w); + } + echo(&v); + free(raw); + codec_codec_Vector_free(&v); + } } -static int verify_scalars(const codec_codec_Scalars* s, codec_error* err) { - ENCODE(codec_codec_Scalars_write, s, w); - int ok = codec_codec_verify_scalars(w.ptr, w.len, err); - codec_writer_free(&w); - return ok; +static codec_codec_Scalars canonical_scalars(void) { + codec_codec_Scalars s; + s.i8_value = -8; + s.u8_value = 200; + s.i16_value = -16000; + s.u16_value = 60000; + s.i32_value = -2000000000; + s.u32_value = 4000000000u; + s.i64_value = -9007199254740993LL; + s.u64_value = UINT64_MAX; + s.f32_value = 1.5f; + s.f64_value = -2.25e100; + s.flag = true; + s.color = codec_codec_Color_Blue; + return s; } -static int verify_composite(const codec_codec_Composite* c, codec_error* err) { - ENCODE(codec_codec_Composite_write, c, w); - int ok = codec_codec_verify_composite(w.ptr, w.len, err); - codec_writer_free(&w); - return ok; +static void literal_vectors(uint32_t n) { + codec_codec_Vector v; + + v.tag = codec_codec_Vector_AllScalars; + v.as.AllScalars.value = canonical_scalars(); + assert(check(find(n, "scalars canonical"), &v)); + v.as.AllScalars.value.u16_value = 60001; + assert(!check(find(n, "scalars canonical"), &v)); + + v.tag = codec_codec_Vector_Figure; + v.as.Figure.value.tag = codec_codec_Shape_Labeled; + v.as.Figure.value.as.Labeled.label = codec_str_of("tag"); + v.as.Figure.value.as.Labeled.count = 3; + assert(check(find(n, "shape labeled"), &v)); + + v.tag = codec_codec_Vector_Text; + v.as.Text.value.ptr = "nul\0inside\0"; + v.as.Text.value.len = 11; + assert(check(find(n, "string interior nul"), &v)); + + // Any NaN matches the NaN vector; zero keeps its sign. + uint64_t bits = 0x7ff8000000000001ull; + v.tag = codec_codec_Vector_F64; + memcpy(&v.as.F64.value, &bits, 8); + assert(check(find(n, "f64 nan"), &v)); + v.as.F64.value = -0.0; + assert(check(find(n, "f64 -0"), &v)); + v.as.F64.value = 0.0; + assert(!check(find(n, "f64 -0"), &v)); + + v.tag = codec_codec_Vector_U64; + v.as.U64.value = UINT64_MAX; + assert(check(find(n, "u64 max"), &v)); + + v.tag = codec_codec_Vector_Hue; + v.as.Hue.value = codec_codec_Color_Infrared; + assert(check(find(n, "enum infrared"), &v)); + + int64_t zero = 0; + v.tag = codec_codec_Vector_MaybeI64; + v.as.MaybeI64.value = &zero; + assert(check(find(n, "optional zero"), &v)); + v.as.MaybeI64.value = NULL; + assert(check(find(n, "optional absent"), &v)); + assert(!check(find(n, "optional zero"), &v)); + + // A map's entry order doesn't matter on the wire. + codec_str keys[3] = {codec_str_of("x"), codec_str_of("h\xc3\xa9llo"), codec_str_of("")}; + int64_t values[3] = {0, -1, INT64_MAX}; + v.tag = codec_codec_Vector_Counts; + v.as.Counts.value.keys = keys; + v.as.Counts.value.values = values; + v.as.Counts.value.len = 3; + assert(check(find(n, "map of strings"), &v)); + + v.tag = codec_codec_Vector_Blank; + assert(check(find(n, "blank"), &v)); } -// describe_* return an owned (ptr, len) string; copy it out NUL-terminated. -static char* take_string(const uint8_t* p, size_t len) { - char* s = (char*)calloc(len + 1, 1); - assert(s != NULL); - if (len) memcpy(s, p, len); - codec_free_bytes((uint8_t*)p, len); - return s; +static void spot_checks(uint32_t n) { + codec_codec_Vector v = fetch(find(n, "i64 past 2^53"), NULL, NULL); + assert(v.tag == codec_codec_Vector_I64 && v.as.I64.value == -9007199254740993LL); + codec_codec_Vector_free(&v); + + v = fetch(find(n, "f32 min subnormal"), NULL, NULL); + uint32_t f32_bits; + memcpy(&f32_bits, &v.as.F32.value, 4); + assert(v.tag == codec_codec_Vector_F32 && f32_bits == 1u); + codec_codec_Vector_free(&v); + + v = fetch(find(n, "string astral"), NULL, NULL); + assert(str_is(v.as.Text.value, "\xf0\x9f\xa6\x80 crab \xf0\x9f\x98\x80", 14)); + codec_codec_Vector_free(&v); + + v = fetch(find(n, "scalars minimum"), NULL, NULL); + assert(v.tag == codec_codec_Vector_AllScalars); + codec_codec_Scalars* m = &v.as.AllScalars.value; + assert(m->i8_value == INT8_MIN && m->i16_value == INT16_MIN && m->i32_value == INT32_MIN); + assert(m->i64_value == INT64_MIN && m->u64_value == 0); + assert(isinf(m->f32_value) && m->f32_value < 0 && isnan(m->f64_value)); + assert(m->color == codec_codec_Color_Infrared && !m->flag); + codec_codec_Vector_free(&v); + + v = fetch(find(n, "composite canonical"), NULL, NULL); + assert(v.tag == codec_codec_Vector_Deep); + codec_codec_Composite* c = &v.as.Deep.value; + assert(str_is(c->name, "h\xc3\xa9llo w\xc3\xb6rld \xe2\x9c\x93", 17)); + assert(c->blob.len == 6 && c->blob.ptr[5] == 255); + assert(c->some_i64 != NULL && *c->some_i64 == INT64_MIN && c->none_i64 == NULL); + assert(c->some_text != NULL && c->some_text->len == 0); + assert(c->names.len == 3 && c->names.items[1].len == 0); + assert(c->matrix.len == 3 && c->matrix.items[1].len == 0 && c->matrix.items[2].items[0] == -4); + assert(c->floats.len == 6 && isnan(c->floats.items[0]) && signbit(c->floats.items[3])); + assert(c->by_name.len == 4 && c->by_id.len == 3 && c->by_color.len == 2 && c->flags.len == 2); + assert(c->scalars.u32_value == 4000000000u); + assert(c->shape.tag == codec_codec_Shape_Labeled && c->shape.as.Labeled.count == 3); + assert(c->shapes.len == 6 && c->shapes.items[5].as.Nested.note == NULL); + assert(c->maybe_shape != NULL && c->maybe_shape->tag == codec_codec_Shape_Nested); + assert(c->maybe_list != NULL && c->maybe_list->len == 2); + assert(c->sparse.len == 3 && c->sparse.items[1] == NULL && *c->sparse.items[0]); + assert(c->colors.len == 4 && c->colors.items[3] == codec_codec_Color_Infrared); + codec_codec_Vector_free(&v); } -static char* describe_shape(const codec_codec_Shape* s) { - ENCODE(codec_codec_Shape_write, s, w); +static void out_of_range(uint32_t n) { codec_error err = {0}; size_t len = 0; - const uint8_t* p = codec_codec_describe_shape(w.ptr, w.len, &len, &err); - codec_writer_free(&w); - assert(err.code == 0); - return take_string(p, len); + assert(codec_codec_vector(n, &len, &err) == NULL); + assert(err.code == codec_codec_CodecError_OutOfRange); + char expected[64]; + snprintf(expected, sizeof expected, "vector %u is out of range (count %u)", n, n); + assert(strcmp(err.message, expected) == 0); + codec_codec_CodecError_OutOfRange_payload p; + assert(codec_codec_CodecError_OutOfRange_payload_decode(err.payload_ptr, err.payload_len, + &p)); + assert(p.index == n && p.count == n); + codec_error_clear(&err); + + assert(codec_codec_vector_name(n + 5, &len, &err) == NULL); + assert(err.code == codec_codec_CodecError_OutOfRange); + assert(codec_codec_CodecError_OutOfRange_payload_decode(err.payload_ptr, err.payload_len, + &p)); + assert(p.index == n + 5 && p.count == n); + codec_error_clear(&err); + + codec_codec_Vector blank; + blank.tag = codec_codec_Vector_Blank; + assert(!check(n, &blank)); } -static codec_codec_Shape shape_of(codec_codec_Shape_Tag tag) { - codec_codec_Shape s; - memset(&s, 0, sizeof s); - s.tag = tag; - return s; +// Hand a malformed buffer to check_vector, which must reject it with -3. +static void reject(const codec_writer* w) { + codec_error err = {0}; + assert(!codec_codec_check_vector(0, w->ptr, w->len, &err)); + assert(err.code == -3); + codec_error_clear(&err); } -// ── sections ─────────────────────────────────────────────────────────────── +static void malformed(void) { + codec_writer w; -static void scalars(void) { - codec_error err = {0}; + memset(&w, 0, sizeof w); // truncated mid-value + codec_writer_put_i32(&w, codec_codec_Vector_I64); + codec_writer_put_u32(&w, 7); + reject(&w); + codec_writer_free(&w); - // Producer encodes, consumer decodes. - codec_codec_Scalars s; - fetch_scalars(&s); - assert(s.i8_value == -8); - assert(s.u8_value == 200); - assert(s.i16_value == -16000); - assert(s.u16_value == 60000); - assert(s.i32_value == -2000000000); - assert(s.u32_value == 4000000000u); - assert(s.i64_value == -9007199254740993LL); - assert(s.u64_value == UINT64_MAX); - assert(s.f32_value == 1.5f); - assert(s.f64_value == -2.25e100); - assert(s.flag); - assert(s.color == codec_codec_Color_Blue && s.color == 7); - assert(scalars_eq(&s, &CANONICAL_SCALARS)); - - // Consumer encodes, producer decodes and compares to its canonical value. - assert(verify_scalars(&s, &err) && err.code == 0); - - // Consumer encodes, producer re-encodes the same bytes. - codec_codec_Scalars s2; - roundtrip_scalars(&s, &s2); - assert(scalars_eq(&s, &s2)); - - // A one-field change is a Mismatch (code 1) with the Display message. - s2.flag = false; - assert(!verify_scalars(&s2, &err)); - assert(err.code == codec_codec_CodecError_Mismatch && err.code == 1); - assert(err.message != NULL && - strcmp(err.message, "value does not match the canonical fixture") == 0); - assert(err.payload_ptr == NULL && err.payload_len == 0); - codec_error_clear(&err); + memset(&w, 0, sizeof w); // unknown tag + codec_writer_put_i32(&w, 999); + reject(&w); + codec_writer_free(&w); - // Hand-built edge scalars: extremes, NaN, infinities, negative zero. - codec_codec_Scalars edge = { - INT8_MIN, UINT8_MAX, INT16_MIN, UINT16_MAX, INT32_MIN, UINT32_MAX, - INT64_MIN, UINT64_MAX, NAN, -0.0, false, codec_codec_Color_Red, - }; - roundtrip_scalars(&edge, &s2); - assert(scalars_eq(&edge, &s2)); - assert(isnan(s2.f32_value) && signbit(s2.f64_value) && s2.f64_value == 0.0); - edge.i8_value = INT8_MAX; - edge.i16_value = INT16_MAX; - edge.i32_value = INT32_MAX; - edge.i64_value = INT64_MAX; - edge.u8_value = 0; - edge.u16_value = 0; - edge.u32_value = 0; - edge.u64_value = 0; - edge.f32_value = -INFINITY; - edge.f64_value = INFINITY; - edge.color = codec_codec_Color_Green; - roundtrip_scalars(&edge, &s2); - assert(scalars_eq(&edge, &s2)); - assert(isinf(s2.f32_value) && s2.f32_value < 0 && isinf(s2.f64_value) && s2.f64_value > 0); -} + memset(&w, 0, sizeof w); // trailing bytes + codec_writer_put_i32(&w, codec_codec_Vector_Blank); + codec_writer_put_u8(&w, 0); + reject(&w); + codec_writer_free(&w); -static void composite(void) { - codec_error err = {0}; + memset(&w, 0, sizeof w); // a bool that is neither 0 nor 1 + codec_writer_put_i32(&w, codec_codec_Vector_Flag); + codec_writer_put_u8(&w, 2); + reject(&w); + codec_writer_free(&w); - // Producer encodes, consumer decodes. - codec_codec_Composite c; - fetch_composite(&c); - assert(strcmp(c.name.ptr, "h\xC3\xA9llo w\xC3\xB6rld \xE2\x9C\x93") == 0); - const uint8_t blob[6] = {0, 1, 2, 253, 254, 255}; - assert(c.blob.len == 6 && memcmp(c.blob.ptr, blob, 6) == 0); - assert(c.some_i64 != NULL && *c.some_i64 == INT64_MIN); - assert(c.none_i64 == NULL); - assert(c.some_text != NULL && c.some_text->len == 0 && c.some_text->ptr[0] == '\0'); - assert(c.names.len == 3 && strcmp(c.names.items[0].ptr, "a") == 0 && - c.names.items[1].len == 0 && strcmp(c.names.items[2].ptr, "ccc") == 0); - assert(c.matrix.len == 3); - assert(c.matrix.items[0].len == 3 && c.matrix.items[0].items[0] == 1 && - c.matrix.items[0].items[1] == 2 && c.matrix.items[0].items[2] == 3); - assert(c.matrix.items[1].len == 0 && c.matrix.items[1].items == NULL); - assert(c.matrix.items[2].len == 1 && c.matrix.items[2].items[0] == -4); - assert(c.empty.len == 0); - // BTreeMap keys arrive sorted. - assert(c.by_name.len == 3); - assert(strcmp(c.by_name.keys[0].ptr, "neg") == 0 && c.by_name.values[0] == -3); - assert(strcmp(c.by_name.keys[1].ptr, "one") == 0 && c.by_name.values[1] == 1); - assert(strcmp(c.by_name.keys[2].ptr, "two") == 0 && c.by_name.values[2] == 2); - assert(c.by_id.len == 2); - assert(c.by_id.keys[0] == -1 && scalars_eq(&c.by_id.values[0], &CANONICAL_SCALARS)); - assert(c.by_id.keys[1] == 42 && !c.by_id.values[1].flag && - c.by_id.values[1].u64_value == UINT64_MAX); - assert(scalars_eq(&c.scalars, &CANONICAL_SCALARS)); - assert(c.shape.tag == codec_codec_Shape_Labeled && - strcmp(c.shape.as.Labeled.label.ptr, "tag") == 0 && c.shape.as.Labeled.count == 3); - assert(c.shapes.len == 5); - const codec_codec_Shape* sh = c.shapes.items; - assert(sh[0].tag == codec_codec_Shape_Empty); - assert(sh[1].tag == codec_codec_Shape_Circle && sh[1].as.Circle.radius == 2.5); - assert(sh[2].tag == codec_codec_Shape_Rect && sh[2].as.Rect.width == 1.0f && - sh[2].as.Rect.height == 0.5f); - assert(sh[3].tag == codec_codec_Shape_Labeled && sh[3].as.Labeled.label.len == 0 && - sh[3].as.Labeled.count == -1); - assert(sh[4].tag == codec_codec_Shape_Nested && - scalars_eq(&sh[4].as.Nested.inner, &CANONICAL_SCALARS) && - sh[4].as.Nested.note != NULL && strcmp(sh[4].as.Nested.note->ptr, "n") == 0); - assert(c.maybe_shape != NULL && c.maybe_shape->tag == codec_codec_Shape_Nested && - c.maybe_shape->as.Nested.note == NULL && - scalars_eq(&c.maybe_shape->as.Nested.inner, &CANONICAL_SCALARS)); - assert(c.maybe_list != NULL && c.maybe_list->len == 2 && c.maybe_list->ptr[0] == 9 && - c.maybe_list->ptr[1] == 8); - assert(c.sparse.len == 3); - assert(c.sparse.items[0] != NULL && *c.sparse.items[0] == true); - assert(c.sparse.items[1] == NULL); - assert(c.sparse.items[2] != NULL && *c.sparse.items[2] == false); - assert(c.colors.len == 3 && c.colors.items[0] == codec_codec_Color_Red && - c.colors.items[1] == codec_codec_Color_Green && - c.colors.items[2] == codec_codec_Color_Blue); - - // Consumer encodes, producer decodes. - assert(verify_composite(&c, &err) && err.code == 0); - - // Consumer encodes, producer re-encodes the same bytes, consumer decodes. - codec_codec_Composite c2; - roundtrip_composite(&c, &c2); - assert(composite_eq(&c, &c2)); - - // Perturb one deeply nested field: Mismatch. - *c2.sparse.items[2] = true; - assert(!verify_composite(&c2, &err)); - assert(err.code == codec_codec_CodecError_Mismatch); - codec_error_clear(&err); - codec_codec_Composite_free(&c2); - - // describe_composite renders what the producer saw. - { - ENCODE(codec_codec_Composite_write, &c, w); - size_t len = 0; - const uint8_t* p = codec_codec_describe_composite(w.ptr, w.len, &len, &err); - codec_writer_free(&w); - assert(err.code == 0); - char* text = take_string(p, len); - assert(strstr(text, "name: \"h\xC3\xA9llo w\xC3\xB6rld \xE2\x9C\x93\"") != NULL); - assert(strstr(text, "some_i64: Some(-9223372036854775808)") != NULL); - assert(strstr(text, "shape: Labeled { label: \"tag\", count: 3 }") != NULL); - free(text); - } - codec_codec_Composite_free(&c); - assert(c.names.items == NULL && c.some_i64 == NULL && "free zeroes the value"); -} + memset(&w, 0, sizeof w); // an undeclared C-style enum value + codec_writer_put_i32(&w, codec_codec_Vector_Hue); + codec_writer_put_i32(&w, 3); + reject(&w); + codec_writer_free(&w); -static void edge_composite(void) { - codec_error err = {0}; + memset(&w, 0, sizeof w); // a map that repeats a key + codec_writer_put_i32(&w, codec_codec_Vector_Counts); + codec_writer_put_u32(&w, 2); + codec_writer_put_string(&w, codec_str_of("a")); + codec_writer_put_i64(&w, 1); + codec_writer_put_string(&w, codec_str_of("a")); + codec_writer_put_i64(&w, 2); + reject(&w); + codec_writer_free(&w); - // Hand-built edge composite borrowing caller memory: everything empty or - // absent, a name with interior NUL bytes, maps with extreme keys, a list - // of absent optionals, NaN and negative zero. - static const char name[] = "\xE6\x97\xA5\xE6\x9C\xAC\0nul\0 \xF0\x9F\x8E\x89"; - int64_t some = INT64_MAX; - int64_t none = -1; - int32_t row[2] = {INT32_MIN, INT32_MAX}; - codec_list_i32 rows[1] = {{row, 2}}; - double floats[2] = {-0.0, NAN}; - codec_str keys[1] = {{"", 0}}; - int64_t values[1] = {INT64_MIN}; - codec_bytes empty_bytes = {NULL, 0}; - bool* sparse[2] = {NULL, NULL}; - - codec_codec_Composite e; - memset(&e, 0, sizeof e); - e.name.ptr = name; - e.name.len = sizeof name - 1; - e.some_i64 = &some; - e.none_i64 = &none; - e.matrix.items = rows; - e.matrix.len = 1; - e.empty.items = floats; - e.empty.len = 2; - e.by_name.keys = keys; - e.by_name.values = values; - e.by_name.len = 1; - e.scalars = CANONICAL_SCALARS; - e.shape = shape_of(codec_codec_Shape_Empty); - e.maybe_list = &empty_bytes; - e.sparse.items = sparse; - e.sparse.len = 2; - - codec_codec_Composite e2; - roundtrip_composite(&e, &e2); - assert(composite_eq(&e, &e2)); - assert(e2.name.len == sizeof name - 1 && memcmp(e2.name.ptr, name, sizeof name - 1) == 0); - assert(e2.blob.len == 0 && e2.maybe_list != NULL && e2.maybe_list->len == 0); - assert(isnan(e2.empty.items[1]) && signbit(e2.empty.items[0])); - assert(e2.sparse.len == 2 && e2.sparse.items[0] == NULL && e2.sparse.items[1] == NULL); - assert(!verify_composite(&e, &err) && err.code == 1); - codec_error_clear(&err); - codec_codec_Composite_free(&e2); -} + memset(&w, 0, sizeof w); // a string that isn't UTF-8 + codec_writer_put_i32(&w, codec_codec_Vector_Text); + const uint8_t bad[2] = {0xC3, 0x28}; + codec_writer_put_bytes(&w, codec_bytes_of(bad, 2)); + reject(&w); + codec_writer_free(&w); -static void shapes(void) { - codec_error err = {0}; - codec_str n = codec_str_of("n"); - codec_codec_Shape v[5]; - v[0] = shape_of(codec_codec_Shape_Empty); - v[1] = shape_of(codec_codec_Shape_Circle); - v[1].as.Circle.radius = 2.5; - v[2] = shape_of(codec_codec_Shape_Rect); - v[2].as.Rect.width = 1.0f; - v[2].as.Rect.height = 0.5f; - v[3] = shape_of(codec_codec_Shape_Labeled); - v[3].as.Labeled.label = codec_str_of("tag"); - v[3].as.Labeled.count = 3; - v[4] = shape_of(codec_codec_Shape_Nested); - v[4].as.Nested.inner = CANONICAL_SCALARS; - v[4].as.Nested.note = &n; - const char* descriptions[5] = { - "Empty", - "Circle { radius: 2.5 }", - "Rect { width: 1.0, height: 0.5 }", - "Labeled { label: \"tag\", count: 3 }", - NULL, - }; - for (int i = 0; i < 5; i++) { - codec_codec_Shape back; - roundtrip_shape(&v[i], &back); - assert(shape_eq(&v[i], &back)); - char* text = describe_shape(&back); - if (descriptions[i]) { - assert(strcmp(text, descriptions[i]) == 0); - } else { - assert(strstr(text, "Nested { inner: Scalars { i8_value: -8,") != NULL && - strstr(text, "note: Some(\"n\") }") != NULL); - } - free(text); - codec_codec_Shape_free(&back); - } + // The generated decoder rejects the same inputs. + codec_codec_Vector v; + const uint8_t unknown[4] = {0xE7, 0x03, 0, 0}; + assert(!codec_codec_Vector_decode(BYTES(unknown), &v)); - codec_codec_Shape no_note = shape_of(codec_codec_Shape_Nested); - no_note.as.Nested.inner = CANONICAL_SCALARS; - codec_codec_Shape back; - roundtrip_shape(&no_note, &back); - assert(shape_eq(&no_note, &back) && back.as.Nested.note == NULL); - codec_codec_Shape_free(&back); - - codec_codec_Shape low = shape_of(codec_codec_Shape_Labeled); - low.as.Labeled.count = INT32_MIN; - roundtrip_shape(&low, &back); - assert(shape_eq(&low, &back)); - codec_codec_Shape_free(&back); - - // roundtrip_shapes: a list of every variant, then an empty list. - codec_list_codec_Shape list; - list.items = v; - list.len = 5; - ENCODE(codec_list_codec_Shape_write, &list, w); + // Direct families: an undeclared enum value, a null string with a + // length, and invalid UTF-8. + codec_error err = {0}; size_t len = 0; - const uint8_t* p = call_buffer(codec_codec_roundtrip_shapes, &w, &len, 1); - codec_list_codec_Shape out; - TAKE(codec_list_codec_Shape_decode, p, len, &out); - assert(out.len == 5); - for (int i = 0; i < 5; i++) assert(shape_eq(&v[i], &out.items[i])); - codec_list_codec_Shape_free(&out); - list.len = 0; - ENCODE(codec_list_codec_Shape_write, &list, w2); - p = call_buffer(codec_codec_roundtrip_shapes, &w2, &len, 1); - TAKE(codec_list_codec_Shape_decode, p, len, &out); - assert(out.len == 0 && out.items == NULL); - - // The producer rejects malformed buffers as marshalling failures (-3): - // truncated mid-value, an unknown tag, and trailing bytes. - const uint8_t truncated[3] = {1, 0, 0}; - assert(codec_codec_roundtrip_shape(BYTES(truncated), &len, &err) == NULL); + codec_codec_echo_color((codec_codec_Color)3, &err); assert(err.code == -3); codec_error_clear(&err); - const uint8_t bad_tag[4] = {99, 0, 0, 0}; - assert(codec_codec_roundtrip_shape(BYTES(bad_tag), &len, &err) == NULL); - assert(err.code == -3); + assert(codec_codec_echo_text(NULL, 2, &len, &err) == NULL && err.code == -3); codec_error_clear(&err); - const uint8_t trailing[5] = {0, 0, 0, 0, 0}; - assert(codec_codec_roundtrip_shape(BYTES(trailing), &len, &err) == NULL); - assert(err.code == -3); + assert(codec_codec_echo_text(BYTES(bad), &len, &err) == NULL && err.code == -3); codec_error_clear(&err); - - // The consumer's decoder rejects them too, leaving nothing allocated. - assert(!codec_codec_Shape_decode(BYTES(truncated), &back)); - assert(!codec_codec_Shape_decode(BYTES(bad_tag), &back)); - assert(!codec_codec_Shape_decode(BYTES(trailing), &back)); - const uint8_t bad_bool[2] = {1, 2}; - bool* flag = NULL; - assert(!codec_opt_bool_decode(BYTES(bad_bool), &flag) && flag == NULL); + assert(codec_codec_echo_text(NULL, 0, &len, &err) == NULL && len == 0 && err.code == 0); } -static void standalone(void) { +static codec_codec_Token* token(int64_t value) { codec_error err = {0}; - size_t len = 0; - - // Standalone optional: present, then absent. - int64_t minus_one = -1; - ENCODE(codec_opt_i64_write, &minus_one, w); - const uint8_t* p = call_buffer(codec_codec_roundtrip_opt_i64, &w, &len, 1); - int64_t* opt = NULL; - TAKE(codec_opt_i64_decode, p, len, &opt); - assert(opt != NULL && *opt == -1); - codec_opt_i64_free(&opt); - ENCODE(codec_opt_i64_write, NULL, w2); - p = call_buffer(codec_codec_roundtrip_opt_i64, &w2, &len, 1); - TAKE(codec_opt_i64_decode, p, len, &opt); - assert(opt == NULL); - - // Map: encoded out of key order, comes back sorted by the BTreeMap. - codec_str keys[3] = {codec_str_of("b"), codec_str_of("a"), codec_str_of("")}; - int64_t values[3] = {-2, 1, INT64_MAX}; - codec_map_string_i64 map; - map.keys = keys; - map.values = values; - map.len = 3; - ENCODE(codec_map_string_i64_write, &map, w3); - p = call_buffer(codec_codec_roundtrip_map, &w3, &len, 0); - codec_map_string_i64 sorted; - TAKE(codec_map_string_i64_decode, p, len, &sorted); - assert(sorted.len == 3); - assert(sorted.keys[0].len == 0 && sorted.values[0] == INT64_MAX); - assert(strcmp(sorted.keys[1].ptr, "a") == 0 && sorted.values[1] == 1); - assert(strcmp(sorted.keys[2].ptr, "b") == 0 && sorted.values[2] == -2); - codec_map_string_i64_free(&sorted); - map.len = 0; - ENCODE(codec_map_string_i64_write, &map, w4); - p = call_buffer(codec_codec_roundtrip_map, &w4, &len, 1); - TAKE(codec_map_string_i64_decode, p, len, &sorted); - assert(sorted.len == 0); - - // Strings cross as (ptr, len) in both directions, interior NULs included. - static const char text[] = "h\xC3\xA9llo \0w\xC3\xB6rld\0"; - p = codec_codec_roundtrip_string((const uint8_t*)text, sizeof text - 1, &len, &err); - assert(err.code == 0 && len == sizeof text - 1 && memcmp(p, text, len) == 0); - codec_free_bytes((uint8_t*)p, len); - p = codec_codec_roundtrip_string(NULL, 0, &len, &err); - assert(err.code == 0 && len == 0); - codec_free_bytes((uint8_t*)p, len); - // A NULL pointer with a length, and invalid UTF-8, are marshalling - // failures. - assert(codec_codec_roundtrip_string(NULL, 2, &len, &err) == NULL && err.code == -3); - codec_error_clear(&err); - const uint8_t bad_utf8[3] = {'a', 0xFF, 'b'}; - assert(codec_codec_roundtrip_string(BYTES(bad_utf8), &len, &err) == NULL && err.code == -3); - codec_error_clear(&err); - - // Bytes, including the empty run. - const uint8_t raw[4] = {0, 255, 0, 128}; - p = codec_codec_roundtrip_bytes(BYTES(raw), &len, &err); - assert(err.code == 0 && len == 4 && memcmp(p, raw, 4) == 0); - codec_free_bytes((uint8_t*)p, len); - p = codec_codec_roundtrip_bytes(raw, 0, &len, &err); - assert(err.code == 0 && len == 0); - codec_free_bytes((uint8_t*)p, len); - - // Direct family. - assert(codec_codec_roundtrip_i64(INT64_MIN, &err) == INT64_MIN); - assert(codec_codec_roundtrip_i64(INT64_MAX, &err) == INT64_MAX); - assert(codec_codec_roundtrip_u64(UINT64_MAX, &err) == UINT64_MAX); - assert(codec_codec_roundtrip_u64(1ULL << 63, &err) == (1ULL << 63)); - assert(isnan(codec_codec_roundtrip_f64(NAN, &err))); - assert(signbit(codec_codec_roundtrip_f64(-0.0, &err))); - assert(codec_codec_roundtrip_f64(INFINITY, &err) == INFINITY); - assert(codec_codec_roundtrip_f64(-2.25e100, &err) == -2.25e100); - assert(codec_codec_roundtrip_bool(true, &err) == true); - assert(codec_codec_roundtrip_bool(false, &err) == false); - assert(codec_codec_roundtrip_color(codec_codec_Color_Blue, &err) == codec_codec_Color_Blue); - assert(codec_codec_roundtrip_color(codec_codec_Color_Red, &err) == 0); - assert(err.code == 0); - codec_codec_roundtrip_color((codec_codec_Color)3, &err); - assert(err.code == -3 && "an undeclared discriminant is a marshalling failure"); - codec_error_clear(&err); + codec_codec_Token* t = codec_codec_Token_new(value, &err); + assert(err.code == 0 && t != NULL); + return t; } -static int64_t token_value(const codec_codec_Token* t) { +static int64_t value_of(const codec_codec_Token* t) { codec_error err = {0}; int64_t v = codec_codec_Token_value(t, &err); assert(err.code == 0); return v; } -static codec_codec_Holder make_holder(int64_t base, bool with_spare) { +static int64_t sum(const codec_codec_Holder* h) { codec_error err = {0}; - size_t len = 0; - const uint8_t* p = codec_codec_make_holder(base, with_spare, &len, &err); + codec_writer w; + memset(&w, 0, sizeof w); + codec_codec_Holder_write(&w, h); + int64_t total = codec_codec_sum_holder(w.ptr, w.len, &err); assert(err.code == 0); - codec_codec_Holder h; - TAKE(codec_codec_Holder_decode, p, len, &h); // adopts every token - return h; + codec_writer_free(&w); + return total; } -static void objects(void) { +static void objects(uint32_t n) { codec_error err = {0}; - codec_codec_Token* t = codec_codec_Token_new(5, &err); - assert(err.code == 0 && t != NULL); - assert(token_value(t) == 5); - codec_codec_Token* t2 = codec_codec_Token_clone(t); - assert(t2 == t); - codec_codec_Token_destroy(t); - assert(token_value(t2) == 5 && "clone survives destroying the original"); - codec_codec_Token_destroy(t2); - codec_codec_Token_destroy(NULL); - assert(codec_codec_Token_clone(NULL) == NULL); - - codec_codec_Holder h = make_holder(10, true); - assert(token_value(h.primary) == 10); - assert(h.spare != NULL && token_value(h.spare) == 11); - assert(h.many.len == 3); - assert(token_value(h.many.items[0]) == 12 && token_value(h.many.items[1]) == 13 && - token_value(h.many.items[2]) == 14); - assert(h.primary != h.spare && h.primary != h.many.items[0]); - - // sum_holder consumes one encoding; the writer stored a fresh reference - // per token, so ours stay intact. - ENCODE(codec_codec_Holder_write, &h, w); - assert(codec_codec_sum_holder(w.ptr, w.len, &err) == 10 + 11 + 12 + 13 + 14); - assert(err.code == 0); + // The full object vector decodes to live tokens with the table's values. + codec_codec_Vector v = fetch(find(n, "objects full"), NULL, NULL); + assert(v.tag == codec_codec_Vector_Objects); + codec_codec_Holder* h = &v.as.Objects.value; + assert(value_of(h->primary) == 10 && h->spare != NULL && value_of(h->spare) == 11); + assert(h->many.len == 3 && value_of(h->many.items[2]) == INT64_MIN); + assert(h->by_name.len == 2 && value_of(h->by_name.values[1]) == 21); + // Each encoding mints fresh references, so the holder can be sent twice. + int64_t expected = 10 + 11 + 12 + 13 + 20 + 21 + INT64_MIN; + assert(sum(h) == expected && sum(h) == expected); + + // primary_of returns the very same object (a new reference to it). + codec_writer w; + memset(&w, 0, sizeof w); + codec_codec_Holder_write(&w, h); + codec_codec_Token* p = codec_codec_primary_of(w.ptr, w.len, &err); codec_writer_free(&w); - assert(token_value(h.primary) == 10 && "our references are intact"); - - // primary_of returns the very same object as holder.primary, as an owned - // reference we release. - ENCODE(codec_codec_Holder_write, &h, w2); - codec_codec_Token* primary = codec_codec_primary_of(w2.ptr, w2.len, &err); - codec_writer_free(&w2); - assert(err.code == 0 && primary == h.primary); - codec_codec_Token_destroy(primary); - assert(token_value(h.primary) == 10); - - // same_primary: two encodings of one holder share a primary; a different - // holder does not. - codec_codec_Holder h2 = make_holder(100, false); - assert(h2.spare == NULL && h2.many.len == 3 && token_value(h2.primary) == 100); - ENCODE(codec_codec_Holder_write, &h, a); - ENCODE(codec_codec_Holder_write, &h, b); + assert(err.code == 0 && p == h->primary); + + // same_primary compares identity, not value. + codec_codec_Holder other; + memset(&other, 0, sizeof other); + other.primary = p; + codec_writer a, b; + memset(&a, 0, sizeof a); + memset(&b, 0, sizeof b); + codec_codec_Holder_write(&a, h); + codec_codec_Holder_write(&b, &other); assert(codec_codec_same_primary(a.ptr, a.len, b.ptr, b.len, &err)); - assert(err.code == 0); codec_writer_free(&a); codec_writer_free(&b); - ENCODE(codec_codec_Holder_write, &h, a2); - ENCODE(codec_codec_Holder_write, &h2, b2); - assert(!codec_codec_same_primary(a2.ptr, a2.len, b2.ptr, b2.len, &err)); - assert(err.code == 0); - codec_writer_free(&a2); - codec_writer_free(&b2); - - // A holder mixing our own tokens with the producer's, built from - // borrowed pointers. - codec_codec_Token* mine = codec_codec_Token_new(1000, &err); - codec_codec_Token* neg = codec_codec_Token_new(-1, &err); - codec_codec_Token* many[2] = {h2.primary, neg}; - codec_codec_Holder mixed; - mixed.primary = mine; - mixed.spare = h.primary; - mixed.many.items = many; - mixed.many.len = 2; - ENCODE(codec_codec_Holder_write, &mixed, m1); - assert(codec_codec_sum_holder(m1.ptr, m1.len, &err) == 1000 + 10 + 100 - 1); - codec_writer_free(&m1); - ENCODE(codec_codec_Holder_write, &mixed, m2); - codec_codec_Token* p = codec_codec_primary_of(m2.ptr, m2.len, &err); - codec_writer_free(&m2); - assert(p == mine && token_value(p) == 1000); - codec_codec_Token_destroy(p); - codec_codec_Token_destroy(mine); - codec_codec_Token_destroy(neg); - assert(token_value(h.primary) == 10 && token_value(h2.primary) == 100 && - "the references written into `mixed` never touched ours"); + codec_codec_Token* twin = token(10); + other.primary = twin; + memset(&a, 0, sizeof a); + memset(&b, 0, sizeof b); + codec_codec_Holder_write(&a, h); + codec_codec_Holder_write(&b, &other); + assert(!codec_codec_same_primary(a.ptr, a.len, b.ptr, b.len, &err)); + codec_writer_free(&a); + codec_writer_free(&b); - // A zero token in a required position is a marshalling failure. - const uint8_t zero_token[13] = {0}; + // A holder built from consumer-made tokens, one of them in every slot. + codec_codec_Token* many[3] = {twin, twin, token(-4)}; + codec_str names[1] = {codec_str_of("k")}; + codec_codec_Token* values[1] = {twin}; + codec_codec_Holder mine; + mine.primary = twin; + mine.spare = twin; + mine.many.items = many; + mine.many.len = 3; + mine.by_name.keys = names; + mine.by_name.values = values; + mine.by_name.len = 1; + assert(sum(&mine) == 10 * 5 - 4); + + // The consumer-built sparse holder checks against the table. + codec_codec_Vector sparse; + sparse.tag = codec_codec_Vector_Objects; + memset(&sparse.as.Objects.value, 0, sizeof sparse.as.Objects.value); + codec_codec_Token* lone = token(-1); + sparse.as.Objects.value.primary = lone; + assert(check(find(n, "objects sparse"), &sparse)); + codec_codec_Token_destroy(lone); + + // A zero token is a marshalling failure, not a crash. + const uint8_t zero_token[17] = {0}; assert(codec_codec_sum_holder(BYTES(zero_token), &err) == 0 && err.code == -3); codec_error_clear(&err); - codec_codec_Holder_free(&h2); - codec_codec_Holder_free(&h); - assert(h.primary == NULL && h.many.items == NULL); + codec_codec_Token_destroy(many[2]); + codec_codec_Token_destroy(twin); + codec_codec_Token_destroy(p); + codec_codec_Vector_free(&v); // releases every decoded token + assert(codec_codec_Token_clone(NULL) == NULL); + codec_codec_Token_destroy(NULL); } int main(void) { - assert(CODEC_ABI_VERSION == 3u && codec_abi_version() == CODEC_ABI_VERSION); - assert(codec_codec_checksum() == CODEC_CODEC_CHECKSUM); + codec_error err = {0}; + assert(CODEC_ABI_VERSION == 4u && codec_abi_version() == CODEC_ABI_VERSION); + assert(codec_codec_contract_check() == 0); + assert(codec_debug_live(-1) == 1); + + uint32_t n = codec_codec_vector_count(&err); + assert(err.code == 0 && n >= 60); - scalars(); - composite(); - edge_composite(); - shapes(); - standalone(); - objects(); + every_vector(n); + literal_vectors(n); + spot_checks(n); + out_of_range(n); + malformed(); + objects(n); ASSERT_NO_LEAKS(codec_debug_live); - printf("c/codec: OK\n"); + printf("c/codec: OK (%u vectors)\n", n); return 0; } diff --git a/conformance/c/contacts.c b/conformance/c/contacts.c deleted file mode 100644 index 69d620d4..00000000 --- a/conformance/c/contacts.c +++ /dev/null @@ -1,247 +0,0 @@ -// Conformance consumer: contacts sample, C target (ABI revision 3). -// -// Includes the generated `contacts.h` and `contacts_buffer.h` and links the -// contacts cdylib. Covers the load-time contract check (ABI revision and -// both roots' checksums), the ContactBook interface (constructor, methods, -// destroy), (ptr, len) string parameters, the Contact record and its -// optional email crossing as value buffers, typed error codes and messages, -// the nested `contacts.groups` module (which uses the parent's C-style enum, -// interface, and error domain), and the sibling `directory` root sharing the -// Contact record. Ends by asserting the producer's leak counters are zero. - -#include "harness.h" - -#include "contacts_buffer.h" - -static void check_contract(void) { - assert(CONTACTS_ABI_VERSION == 3u); - assert(contacts_abi_version() == CONTACTS_ABI_VERSION); - assert(contacts_contacts_checksum() == CONTACTS_CONTACTS_CHECKSUM); - assert(contacts_directory_checksum() == CONTACTS_DIRECTORY_CHECKSUM); - assert(CONTACTS_CONTACTS_CHECKSUM != CONTACTS_DIRECTORY_CHECKSUM); -} - -// Decode an owned Contact return, release the buffer, and return the value -// (which the caller frees with contacts_contacts_Contact_free). -static contacts_contacts_Contact take_contact(const uint8_t* ptr, size_t len) { - assert(ptr != NULL); - contacts_contacts_Contact c; - assert(contacts_contacts_Contact_decode(ptr, len, &c)); - contacts_free_bytes((uint8_t*)ptr, len); - return c; -} - -// Add a contact (with an optional email), returning the stored record. -static contacts_contacts_Contact add(contacts_contacts_ContactBook* book, const char* first, - const char* last, const char* email, - contacts_contacts_ContactType type) { - contacts_error err = {0}; - contacts_str email_view = contacts_str_of(email); - contacts_writer w; - memset(&w, 0, sizeof w); - contacts_opt_string_write(&w, email ? &email_view : NULL); - size_t len = 0; - const uint8_t* buf = contacts_contacts_ContactBook_add(book, STR(first), STR(last), w.ptr, - w.len, type, &len, &err); - assert(err.code == 0); - contacts_writer_free(&w); - return take_contact(buf, len); -} - -static void book_surface(void) { - contacts_error err = {0}; - contacts_contacts_ContactBook* book = contacts_contacts_ContactBook_new(&err); - assert(err.code == 0 && book != NULL); - - contacts_contacts_Contact alice = - add(book, "Alice", "Smith", "alice@example.com", contacts_contacts_ContactType_Work); - assert(alice.id > 0); - assert(strcmp(alice.first_name.ptr, "Alice") == 0); - assert(strcmp(alice.last_name.ptr, "Smith") == 0); - assert(alice.email != NULL && strcmp(alice.email->ptr, "alice@example.com") == 0); - assert(alice.contact_type == contacts_contacts_ContactType_Work); - - // get -> a fresh snapshot of the same record. - size_t len = 0; - const uint8_t* buf = contacts_contacts_ContactBook_get(book, alice.id, &len, &err); - assert(err.code == 0); - contacts_contacts_Contact snap = take_contact(buf, len); - assert(snap.id == alice.id && strcmp(snap.last_name.ptr, "Smith") == 0); - contacts_contacts_Contact_free(&snap); - - // An absent email round-trips as an absent optional. - contacts_contacts_Contact bob = - add(book, "Bob", "Jones", NULL, contacts_contacts_ContactType_Personal); - assert(bob.email == NULL); - - // A non-ASCII name crosses intact. - contacts_contacts_Contact zoe = - add(book, "Zo\xC3\xAB", "Quinn", NULL, contacts_contacts_ContactType_Other); - assert(zoe.first_name.len == 4 && strcmp(zoe.first_name.ptr, "Zo\xC3\xAB") == 0); - - // count + list: the list return is one buffer holding every record. - assert(contacts_contacts_ContactBook_count(book, &err) == 3); - buf = contacts_contacts_ContactBook_list(book, &len, &err); - assert(err.code == 0 && buf != NULL); - contacts_list_contacts_Contact all; - assert(contacts_list_contacts_Contact_decode(buf, len, &all)); - contacts_free_bytes((uint8_t*)buf, len); - assert(all.len == 3); - assert(strcmp(all.items[0].first_name.ptr, "Alice") == 0); - assert(strcmp(all.items[1].first_name.ptr, "Bob") == 0); - assert(all.items[2].id == zoe.id); - contacts_list_contacts_Contact_free(&all); - - // remove, then the typed NotFound error for the missing id. - assert(contacts_contacts_ContactBook_remove(book, alice.id, &err)); - assert(!contacts_contacts_ContactBook_remove(book, alice.id, &err)); - assert(contacts_contacts_ContactBook_count(book, &err) == 2); - assert(contacts_contacts_ContactBook_get(book, alice.id, &len, &err) == NULL); - assert(err.code == contacts_contacts_ContactsError_NotFound); - assert(strcmp(err.message, "contact not found") == 0); - contacts_error_clear(&err); - assert(err.code == 0 && err.message == NULL); - - // An empty first name reports InvalidName. A NULL pointer with length 0 - // is the empty string. - const uint8_t absent[1] = {0}; - buf = contacts_contacts_ContactBook_add(book, NULL, 0, STR("Nameless"), BYTES(absent), - contacts_contacts_ContactType_Other, &len, &err); - assert(buf == NULL); - assert(err.code == contacts_contacts_ContactsError_InvalidName); - assert(strcmp(err.message, "name must not be empty") == 0); - contacts_error_clear(&err); - - // A NULL pointer with a nonzero length is a marshalling failure (-3). - buf = contacts_contacts_ContactBook_add(book, NULL, 3, STR("X"), BYTES(absent), - contacts_contacts_ContactType_Other, &len, &err); - assert(buf == NULL && err.code == -3); - contacts_error_clear(&err); - - contacts_contacts_Contact_free(&alice); - contacts_contacts_Contact_free(&bob); - contacts_contacts_Contact_free(&zoe); - contacts_contacts_ContactBook_destroy(book); -} - -// contacts.groups: nested under the contacts root, so its functions take -// the parent's ContactBook and ContactType and report the parent's errors. -static void nested_module(void) { - contacts_error err = {0}; - contacts_contacts_ContactBook* book = contacts_contacts_ContactBook_new(&err); - contacts_contacts_Contact c[3] = { - add(book, "Ann", "Lee", NULL, contacts_contacts_ContactType_Work), - add(book, "Bo", "Kim", "bo@example.com", contacts_contacts_ContactType_Work), - add(book, "Cy", "Ng", NULL, contacts_contacts_ContactType_Personal), - }; - - assert(contacts_contacts_groups_count_of_type(book, contacts_contacts_ContactType_Work, - &err) == 2); - assert(contacts_contacts_groups_count_of_type(book, contacts_contacts_ContactType_Other, - &err) == 0); - assert(err.code == 0); - assert(contacts_contacts_groups_dominant_type(book, &err) == - contacts_contacts_ContactType_Work); - - contacts_contacts_ContactBook* work = - contacts_contacts_groups_split_by_type(book, contacts_contacts_ContactType_Work, &err); - assert(err.code == 0 && work != NULL && work != book); - assert(contacts_contacts_ContactBook_count(work, &err) == 2); - assert(contacts_contacts_groups_dominant_type(work, &err) == - contacts_contacts_ContactType_Work); - contacts_contacts_ContactBook_destroy(work); - - size_t len = 0; - const uint8_t* buf = contacts_contacts_groups_first_of_type( - book, contacts_contacts_ContactType_Work, &len, &err); - assert(err.code == 0); - contacts_contacts_Contact first = take_contact(buf, len); - assert(strcmp(first.first_name.ptr, "Ann") == 0); - contacts_contacts_Contact_free(&first); - - // The parent domain's NotFound code, reported from the nested module. - buf = contacts_contacts_groups_first_of_type(book, contacts_contacts_ContactType_Other, - &len, &err); - assert(buf == NULL); - assert(err.code == contacts_contacts_ContactsError_NotFound); - contacts_error_clear(&err); - - // An out-of-range enum discriminant is a marshalling failure (-3). - assert(contacts_contacts_groups_count_of_type(book, (contacts_contacts_ContactType)9, - &err) == 0); - assert(err.code == -3); - contacts_error_clear(&err); - - for (int i = 0; i < 3; i++) contacts_contacts_Contact_free(&c[i]); - contacts_contacts_ContactBook_destroy(book); -} - -// directory: a sibling root that shares only the Contact record. -static void sibling_root(void) { - contacts_error err = {0}; - contacts_contacts_ContactBook* book = contacts_contacts_ContactBook_new(&err); - contacts_contacts_Contact zoe = - add(book, "Zoe", "Quinn", NULL, contacts_contacts_ContactType_Personal); - contacts_contacts_Contact ada = - add(book, "Ada", "Byron", "ada@example.com", contacts_contacts_ContactType_Work); - contacts_contacts_ContactBook_destroy(book); - - // card(Contact) -> Card: a record built in one root, read by the other. - contacts_writer w; - memset(&w, 0, sizeof w); - contacts_contacts_Contact_write(&w, &ada); - size_t len = 0; - const uint8_t* buf = contacts_directory_card(w.ptr, w.len, &len, &err); - contacts_writer_free(&w); - assert(err.code == 0 && buf != NULL); - contacts_directory_Card card; - assert(contacts_directory_Card_decode(buf, len, &card)); - contacts_free_bytes((uint8_t*)buf, len); - assert(strcmp(card.display_name.ptr, "Byron, Ada") == 0); - assert(strcmp(card.initials.ptr, "AB") == 0); - assert(card.has_email); - contacts_directory_Card_free(&card); - - // sorted([Contact]) -> [Contact], ordered by last name. - contacts_contacts_Contact pair[2] = {zoe, ada}; - contacts_list_contacts_Contact in; - in.items = pair; - in.len = 2; - memset(&w, 0, sizeof w); - contacts_list_contacts_Contact_write(&w, &in); - buf = contacts_directory_sorted(w.ptr, w.len, &len, &err); - contacts_writer_free(&w); - assert(err.code == 0); - contacts_list_contacts_Contact sorted; - assert(contacts_list_contacts_Contact_decode(buf, len, &sorted)); - contacts_free_bytes((uint8_t*)buf, len); - assert(sorted.len == 2); - assert(strcmp(sorted.items[0].last_name.ptr, "Byron") == 0); - assert(sorted.items[0].email != NULL); - assert(strcmp(sorted.items[1].last_name.ptr, "Quinn") == 0); - assert(sorted.items[1].email == NULL); - contacts_list_contacts_Contact_free(&sorted); - - // An empty list reports the directory's own Empty code. - in.len = 0; - memset(&w, 0, sizeof w); - contacts_list_contacts_Contact_write(&w, &in); - assert(contacts_directory_sorted(w.ptr, w.len, &len, &err) == NULL); - contacts_writer_free(&w); - assert(err.code == contacts_directory_DirectoryError_Empty); - assert(strcmp(err.message, "no contacts to list") == 0); - contacts_error_clear(&err); - - contacts_contacts_Contact_free(&zoe); - contacts_contacts_Contact_free(&ada); -} - -int main(void) { - check_contract(); - book_surface(); - nested_module(); - sibling_root(); - ASSERT_NO_LEAKS(contacts_debug_live); - printf("c/contacts: OK\n"); - return 0; -} diff --git a/conformance/c/events.c b/conformance/c/events.c deleted file mode 100644 index 03b9bc31..00000000 --- a/conformance/c/events.c +++ /dev/null @@ -1,359 +0,0 @@ -// Conformance consumer: events sample, C target (ABI revision 3). -// -// Exercises the raw callback-interface ABI: a hand-written `Subscriber` -// vtable (one function per method taking `void* ctx` first and a trailing -// `events_error* out_err`, plus the `free` entry the producer calls when it -// drops its last reference), the reference-counted `EventBus` object -// (`_clone`/`_destroy`, an object handed *to* the consumer through -// `on_attached`), `Delivery` return values steering `publish`'s accepted -// count, a consumer failure reported through `out_err` from `route` (the -// method whose Rust signature returns `Result<_, ForeignError>`) surfacing -// to the caller as code -4, the `Message` record decoded from a borrowed -// value buffer, the `messages()` iterator yielding owned strings, the -// `last_message()` optional, and the async `publish_later` launcher, whose -// subscriber callbacks run on a producer thread. Ends by asserting the -// producer's leak counters are zero. - -#include "harness.h" - -#include - -#include "events_buffer.h" - -// ── consumer-side subscriber ─────────────────────────────────────────────── - -// Per-subscriber state the producer sees only as an opaque `void* ctx`. -typedef struct { - const char* skip_topic; // route() answers Skip for this topic - const char* stop_topic; // route() answers AcceptAndStop for this topic - const char* fail_topic; // route() reports a consumer failure - int keep_bus; // on_attached keeps the bus reference in `bus` - int64_t received; // running count returned from on_message - int attached; // on_attached invocations - int routed; // route invocations - events_events_EventBus* bus; // kept reference (when keep_bus) - int64_t attached_count; // subscriber_count observed inside on_attached - // Fields of the last decoded Message. - int64_t last_seq; - char last_topic[64]; - char last_text[64]; - size_t last_tag_count; - char last_tag0[64]; -} sub_ctx; - -static atomic_int g_freed = 0; - -static int topic_is(const uint8_t* ptr, size_t len, const char* topic) { - return topic != NULL && bytes_eq(ptr, len, topic); -} - -static events_events_Delivery sub_route(void* ctx, const uint8_t* topic_ptr, size_t topic_len, - events_error* out_err) { - sub_ctx* s = (sub_ctx*)ctx; - s->routed++; - if (topic_is(topic_ptr, topic_len, s->fail_topic)) { - events_error_set(out_err, -4, "subscriber rejected topic"); - return events_events_Delivery_Skip; - } - if (topic_is(topic_ptr, topic_len, s->skip_topic)) { - return events_events_Delivery_Skip; - } - if (topic_is(topic_ptr, topic_len, s->stop_topic)) { - return events_events_Delivery_AcceptAndStop; - } - return events_events_Delivery_Accept; -} - -// The Message buffer is borrowed for the duration of the call. -static int64_t sub_on_message(void* ctx, const uint8_t* message_ptr, size_t message_len, - events_error* out_err) { - (void)out_err; - sub_ctx* s = (sub_ctx*)ctx; - events_events_Message m; - assert(events_events_Message_decode(message_ptr, message_len, &m)); - s->last_seq = m.seq; - snprintf(s->last_topic, sizeof s->last_topic, "%s", m.topic.ptr); - snprintf(s->last_text, sizeof s->last_text, "%s", m.text.ptr); - s->last_tag_count = m.tags.len; - s->last_tag0[0] = '\0'; - if (m.tags.len > 0) snprintf(s->last_tag0, sizeof s->last_tag0, "%s", m.tags.items[0].ptr); - events_events_Message_free(&m); - s->received++; - return s->received; -} - -// The bus arrives as one strong reference the consumer adopts: it is usable -// right here, and it is ours to keep or release. -static void sub_on_attached(void* ctx, events_events_EventBus* bus, events_error* out_err) { - (void)out_err; - sub_ctx* s = (sub_ctx*)ctx; - s->attached++; - events_error err = {0}; - s->attached_count = events_events_EventBus_subscriber_count(bus, &err); - assert(err.code == 0); - if (s->keep_bus) { - s->bus = bus; - } else { - events_events_EventBus_destroy(bus); - } -} - -static void sub_free(void* ctx) { - sub_ctx* s = (sub_ctx*)ctx; - events_events_EventBus_destroy(s->bus); // null is a no-op - free(s); - atomic_fetch_add(&g_freed, 1); -} - -static const events_events_Subscriber_vtable SUB_VTABLE = { - sub_route, - sub_on_message, - sub_on_attached, - sub_free, -}; - -static sub_ctx* new_sub(const char* skip, const char* stop, const char* fail, int keep_bus) { - sub_ctx* s = (sub_ctx*)calloc(1, sizeof *s); - assert(s != NULL); - s->skip_topic = skip; - s->stop_topic = stop; - s->fail_topic = fail; - s->keep_bus = keep_bus; - return s; -} - -// ── helpers ──────────────────────────────────────────────────────────────── - -// publish(topic, text, tags): `tags` is a buffered `[string]`. -static int64_t publish(events_events_EventBus* bus, const char* topic, const char* text, - const char** tags, size_t ntags, events_error* err) { - events_str views[4]; - assert(ntags <= 4); - for (size_t i = 0; i < ntags; i++) views[i] = events_str_of(tags[i]); - events_list_string list; - list.items = views; - list.len = ntags; - events_writer w; - memset(&w, 0, sizeof w); - events_list_string_write(&w, &list); - int64_t n = events_events_EventBus_publish(bus, STR(topic), STR(text), w.ptr, w.len, err); - events_writer_free(&w); - return n; -} - -// Collect messages() into `out`, returning how many were yielded. Each item -// is an owned (ptr, len) string released with events_free_bytes. -static int collect_messages(events_events_EventBus* bus, char** out, int cap) { - events_error err = {0}; - events_events_EventBus_MessagesIterator* it = events_events_EventBus_messages(bus, &err); - assert(err.code == 0 && it != NULL); - int n = 0; - const uint8_t* item = NULL; - size_t len = 0; - while (events_events_EventBus_MessagesIterator_next(it, &item, &len, &err) == 1) { - assert(err.code == 0 && n < cap); - out[n] = (char*)calloc(len + 1, 1); - memcpy(out[n++], item, len); - events_free_bytes((uint8_t*)item, len); - } - assert(err.code == 0); - events_events_EventBus_MessagesIterator_destroy(it); - return n; -} - -static void free_texts(char** texts, int n) { - for (int i = 0; i < n; i++) free(texts[i]); -} - -// Decode last_message()'s buffered `Message?`. -static events_events_Message* last_message(events_events_EventBus* bus) { - events_error err = {0}; - size_t len = 0; - const uint8_t* buf = events_events_EventBus_last_message(bus, &len, &err); - assert(err.code == 0 && buf != NULL); - events_events_Message* m = NULL; - assert(events_opt_events_Message_decode(buf, len, &m)); - events_free_bytes((uint8_t*)buf, len); - return m; -} - -// ── async completion state ───────────────────────────────────────────────── -static atomic_int g_later_done = 0; -static int32_t g_later_err = -1; -static int64_t g_later_result = -1; - -static void on_publish_later(void* context, events_error* err, int64_t result) { - assert(context == (void*)0x1234); - g_later_err = err ? err->code : 0; - events_error_free(err); - g_later_result = result; - atomic_store(&g_later_done, 1); -} - -int main(void) { - events_error err = {0}; - - assert(EVENTS_ABI_VERSION == 3u && events_abi_version() == EVENTS_ABI_VERSION); - assert(events_events_checksum() == EVENTS_EVENTS_CHECKSUM); - - events_events_EventBus* bus = events_events_EventBus_new(&err); - assert(err.code == 0 && bus != NULL); - assert(events_events_EventBus_subscriber_count(bus, &err) == 0); - - // last_message() on an empty bus: an absent optional. - assert(last_message(bus) == NULL); - - // messages() on an empty bus yields nothing. - char* texts[8]; - assert(collect_messages(bus, texts, 8) == 0); - - // Three subscribers: `a` skips "quiet", stops on "stop", and keeps the - // bus reference handed to on_attached; `b` accepts everything and drops - // its bus reference; `c` fails on "boom". - sub_ctx* a = new_sub("quiet", "stop", NULL, 1); - sub_ctx* b = new_sub(NULL, NULL, NULL, 0); - sub_ctx* c = new_sub(NULL, NULL, "boom", 0); - - assert(events_events_EventBus_subscribe(bus, a, &SUB_VTABLE, &err) == 1); - assert(err.code == 0); - assert(a->attached == 1); - assert(a->attached_count == 0 && "on_attached runs before the bus retains us"); - assert(a->bus == bus && "the object handed to the callback is the same bus"); - assert(events_events_EventBus_subscribe(bus, b, &SUB_VTABLE, &err) == 2); - assert(b->attached == 1 && b->attached_count == 1); - assert(events_events_EventBus_subscribe(bus, c, &SUB_VTABLE, &err) == 3); - assert(c->attached == 1 && c->attached_count == 2); - assert(events_events_EventBus_subscriber_count(bus, &err) == 3); - - // The kept reference is usable independently of the original pointer. - assert(events_events_EventBus_subscriber_count(a->bus, &err) == 3); - - // Everyone accepts "news": 3 deliveries, and each saw the same Message. - const char* tags[] = {"x", "y"}; - assert(publish(bus, "news", "hello", tags, 2, &err) == 3); - assert(err.code == 0); - assert(a->received == 1 && b->received == 1 && c->received == 1); - assert(a->last_seq == 1); - assert(strcmp(a->last_topic, "news") == 0); - assert(strcmp(a->last_text, "hello") == 0); - assert(a->last_tag_count == 2 && strcmp(a->last_tag0, "x") == 0); - assert(b->last_seq == 1 && strcmp(c->last_text, "hello") == 0); - - // `a` skips "quiet": 2 deliveries, a's count unchanged. - assert(publish(bus, "quiet", "psst", NULL, 0, &err) == 2); - assert(a->received == 1 && b->received == 2 && c->received == 2); - assert(b->last_seq == 2 && b->last_tag_count == 0); - - // `a` answers AcceptAndStop for "stop": exactly 1 delivery, later - // subscribers are never asked. - int b_routed = b->routed; - assert(publish(bus, "stop", "last", NULL, 0, &err) == 1); - assert(a->received == 2 && b->received == 2 && c->received == 2); - assert(a->last_seq == 3 && strcmp(a->last_text, "last") == 0); - assert(b->routed == b_routed); - - // `route` is the Result-returning callback method: `c` reports a failure - // through out_err on "boom", the producer receives it as an Err and - // chooses to fail the whole publish with code -4 and the message. - publish(bus, "boom", "x", NULL, 0, &err); - assert(err.code == -4); - assert(err.message != NULL && strstr(err.message, "rejected topic") != NULL); - events_error_clear(&err); - assert(err.code == 0 && err.message == NULL); - assert(a->received == 3 && b->received == 3 && c->received == 2 && - "subscribers before the failure were delivered to"); - - // The bus (and its subscribers) stay usable afterward. - assert(publish(bus, "ok", "y", NULL, 0, &err) == 3); - assert(err.code == 0); - - // Async publish: the launcher returns at once; the subscribers' callbacks - // and then the completion run on a producer thread. - events_events_EventBus_publish_later(bus, STR("later"), STR("z"), on_publish_later, - (void*)0x1234); - for (int i = 0; i < 5000 && !atomic_load(&g_later_done); i++) sleep_ms(1); - assert(atomic_load(&g_later_done)); - assert(g_later_err == 0); - assert(g_later_result == 3); - assert(strcmp(b->last_text, "z") == 0); - - // messages(): every published text in order, including the aborted one - // (the bus logs before it dispatches). - int n = collect_messages(bus, texts, 8); - const char* expected[] = {"hello", "psst", "last", "x", "y", "z"}; - assert(n == 6); - for (int i = 0; i < n; i++) assert(strcmp(texts[i], expected[i]) == 0); - free_texts(texts, n); - - // last_message(): present, with the async publish's fields. - events_events_Message* last = last_message(bus); - assert(last != NULL); - assert(last->seq == 6); - assert(strcmp(last->topic.ptr, "later") == 0); - assert(strcmp(last->text.ptr, "z") == 0); - assert(last->tags.len == 0 && "publish_later attaches no tags"); - events_opt_events_Message_free(&last); - assert(last == NULL); - - // route_once: a free function taking the callback interface. The - // producer does not retain it, so `free` runs before the call returns. - assert(atomic_load(&g_freed) == 0); - sub_ctx* d = new_sub("quiet", NULL, NULL, 0); - assert(events_events_route_once(d, &SUB_VTABLE, STR("quiet"), &err) == - events_events_Delivery_Skip); - assert(err.code == 0); - assert(atomic_load(&g_freed) == 1 && "route_once released its subscriber"); - sub_ctx* e = new_sub(NULL, "stop", NULL, 0); - assert(events_events_route_once(e, &SUB_VTABLE, STR("stop"), &err) == - events_events_Delivery_AcceptAndStop); - assert(events_events_route_once(new_sub(NULL, NULL, NULL, 0), &SUB_VTABLE, - STR("anything"), &err) == events_events_Delivery_Accept); - assert(atomic_load(&g_freed) == 3); - - // A failure from the Result-returning route surfaces the same way, and - // the subscriber is still released exactly once. - sub_ctx* f = new_sub(NULL, NULL, "boom", 0); - events_events_route_once(f, &SUB_VTABLE, STR("boom"), &err); - assert(err.code == -4); - assert(strstr(err.message, "rejected topic") != NULL); - events_error_clear(&err); - assert(atomic_load(&g_freed) == 4); - - // Release the bus reference `a` kept, then drop every subscriber: each - // `free` entry runs exactly once. - events_events_EventBus_destroy(a->bus); - a->bus = NULL; - assert(events_events_EventBus_subscriber_count(bus, &err) == 3); - events_events_EventBus_clear_subscribers(bus, &err); - assert(err.code == 0); - assert(atomic_load(&g_freed) == 7 && "clear_subscribers freed a, b, and c"); - assert(events_events_EventBus_subscriber_count(bus, &err) == 0); - assert(publish(bus, "empty", "nobody", NULL, 0, &err) == 0); - - // Reference counting: clone yields the same pointer; destroying the - // original leaves the clone usable. - events_events_EventBus* again = events_events_EventBus_clone(bus); - assert(again == bus); - events_events_EventBus_destroy(bus); - assert(events_events_EventBus_subscriber_count(again, &err) == 0); - assert(err.code == 0); - n = collect_messages(again, texts, 8); - assert(n == 7); - free_texts(texts, n); - events_events_EventBus_destroy(again); - events_events_EventBus_destroy(NULL); - assert(events_events_EventBus_clone(NULL) == NULL); - - // Destroying a bus releases its subscribers too. - events_events_EventBus* bus2 = events_events_EventBus_new(&err); - sub_ctx* g = new_sub(NULL, NULL, NULL, 0); - assert(events_events_EventBus_subscribe(bus2, g, &SUB_VTABLE, &err) == 1); - assert(publish(bus2, "t", "u", NULL, 0, &err) == 1); - assert(g->received == 1); - events_events_EventBus_destroy(bus2); - assert(atomic_load(&g_freed) == 8 && "destroying the bus frees its subscriber"); - - ASSERT_NO_LEAKS(events_debug_live); - printf("c/events: OK\n"); - return 0; -} diff --git a/conformance/c/kvstore.c b/conformance/c/kvstore.c index b5941f71..3045ed33 100644 --- a/conformance/c/kvstore.c +++ b/conformance/c/kvstore.c @@ -1,523 +1,1168 @@ -// Conformance consumer: kvstore sample, C target (ABI revision 3). +// Conformance consumer: kvstore sample, C target (ABI revision 4). +// +// Drives the feature-complete producer through its generated header and +// `kvstore_buffer.h`: +// +// * the load-time checks (ABI revision, both modules' contract tables); +// * the `Store` interface: fallible and infallible constructors, methods, +// statics, the deprecated `size`, records (`Entry`, `StoreInfo`), the +// C-style `EntryKind`, maps and optionals, and the logical clock; +// * `KvError` codes with their payload fields decoded (`KeyNotFound`, +// `Expired`, `StoreFull`, `Rejected`), plus marshalling failures (-3); +// * lazy iterators of strings (throwing), records, and objects; +// * three callback interfaces implemented in C: a `Listener` (retained, +// filtered by `accepts`, told about every `Change`, detached when it +// fails, and notified from a producer thread during compaction), a +// `Policy` (a record return, a throwing method whose typed error and +// payload reach the `put` caller, an object parameter and object +// return, a null object and a malformed return rejected), and a `Loader` +// passed as an optional callback (string, bytes, and optional-object +// returns; typed errors decoded by the producer or passed through); +// * `Store` objects in every position: parameter, return, optional, list, +// map value, record field, iterator element, async result, and callback +// parameter and return; +// * async calls: an async free function returning an object, a cancellable +// method cancelled mid-pause (completing with -5 while its background +// work stops cooperatively, shown by `active_jobs`), an async list, an +// async free function in the nested `kv.stats` module, and concurrent +// launches; +// * the nested `kv.stats` module (the parent's `Store` and error domain) +// and the sibling `report` root (the shared `Entry` record and its own +// error domain). // -// Exercises the Store interface ABI end to end: the fallible constructor -// (`Store_open`), instance methods taking the receiver as the leading -// argument, the static method (`Store_default_capacity`), (ptr, len) string -// and bytes parameters, the typed error-domain codes surfaced through the -// error-out slot, the iterator's owned (ptr, len) items, the `Entry` record -// (list and map fields included) decoded through `kvstore_buffer.h`, the -// `kv.stats` submodule, the hand-written `EvictionListener` vtable (set, -// fire synchronously on delete and on expiry-on-read, replace, clear, -// detach, `free`), a consumer failure raised from the listener surfacing as -// code -4, the reference-counted object graph (`share`, `fork`, -// `clone`/`destroy`, `larger` with `Store?` both ways, `describe` returning -// a record that carries objects, `open_many` returning a list of objects, -// `total_count` taking objects inside a list and an optional record), and -// the cancellable async `compact`, whose cancelled call completes with -5. // Ends by asserting the producer's leak counters are zero. #include "harness.h" +#include #include #include "kvstore_buffer.h" -// ── eviction listener (consumer-implemented callback interface) ──────────── +static pthread_t g_main_thread; + +// ── small helpers ────────────────────────────────────────────────────────── + +static int str_is(kvstore_str s, const char* expected) { + return bytes_eq((const uint8_t*)s.ptr, s.len, expected); +} + +static int starts_with(kvstore_str s, const char* prefix) { + size_t n = strlen(prefix); + return s.len >= n && memcmp(s.ptr, prefix, n) == 0; +} + +static char* take_string(const uint8_t* ptr, size_t len) { + char* s = (char*)calloc(len + 1, 1); + if (len > 0) memcpy(s, ptr, len); + kvstore_free_bytes((uint8_t*)ptr, len); + return s; +} + +// Copy a writer's bytes into a run the producer adopts. +static void hand_over(kvstore_writer* w, uint8_t** out_ptr, size_t* out_len) { + assert(!w->failed); + uint8_t* run = kvstore_alloc(w->len); + if (w->len > 0) memcpy(run, w->ptr, w->len); + *out_ptr = run; + *out_len = w->len; + kvstore_writer_free(w); +} + +static void hand_over_text(const char* s, uint8_t** out_ptr, size_t* out_len) { + size_t n = strlen(s); + uint8_t* run = kvstore_alloc(n); + if (n > 0) memcpy(run, s, n); + *out_ptr = run; + *out_len = n; +} + +static kvstore_kv_Store* open_store(const char* path) { + kvstore_error err = {0}; + kvstore_kv_Store* s = kvstore_kv_Store_open(STR(path), &err); + assert(err.code == 0 && s != NULL); + return s; +} + +static uint32_t count(const kvstore_kv_Store* s) { + kvstore_error err = {0}; + uint32_t n = kvstore_kv_Store_count(s, &err); + assert(err.code == 0); + return n; +} + +static char* path_of(const kvstore_kv_Store* s) { + kvstore_error err = {0}; + size_t len = 0; + const uint8_t* p = kvstore_kv_Store_path(s, &len, &err); + assert(err.code == 0); + return take_string(p, len); +} + +static int path_is(const kvstore_kv_Store* s, const char* expected) { + char* p = path_of(s); + int ok = strcmp(p, expected) == 0; + free(p); + return ok; +} + +// put(key, value, kind, ttl?): decodes the returned entry into `out` on +// success; on failure leaves the error in `err`. +static bool try_put(const kvstore_kv_Store* s, const char* key, const char* value, + kvstore_kv_EntryKind kind, const int64_t* ttl, kvstore_kv_Entry* out, + kvstore_error* err) { + kvstore_writer t; + memset(&t, 0, sizeof t); + kvstore_opt_i64_write(&t, ttl); + size_t len = 0; + const uint8_t* buf = kvstore_kv_Store_put(s, STR(key), (const uint8_t*)value, + strlen(value), kind, t.ptr, t.len, &len, err); + kvstore_writer_free(&t); + if (buf == NULL) { + assert(err->code != 0); + return false; + } + assert(err->code == 0); + kvstore_kv_Entry e; + assert(kvstore_kv_Entry_decode(buf, len, &e)); + kvstore_free_bytes((uint8_t*)buf, len); + if (out != NULL) { + *out = e; + } else { + kvstore_kv_Entry_free(&e); + } + return true; +} + +static void put(const kvstore_kv_Store* s, const char* key, const char* value) { + kvstore_error err = {0}; + assert(try_put(s, key, value, kvstore_kv_EntryKind_Persistent, NULL, NULL, &err)); +} + +// The optional-string parameter buffer for a prefix. +static kvstore_writer prefix_buf(const char* prefix) { + kvstore_writer w; + memset(&w, 0, sizeof w); + kvstore_str view = kvstore_str_of(prefix); + kvstore_opt_string_write(&w, prefix ? &view : NULL); + return w; +} + +static void expect_key_not_found(kvstore_error* err, const char* key) { + assert(err->code == kvstore_kv_KvError_KeyNotFound); + kvstore_kv_KvError_KeyNotFound_payload p; + assert(kvstore_kv_KvError_KeyNotFound_payload_decode(err->payload_ptr, err->payload_len, &p)); + assert(str_is(p.key, key)); + kvstore_kv_KvError_KeyNotFound_payload_free(&p); + kvstore_error_clear(err); +} + +// ── listener (consumer-implemented, retained) ────────────────────────────── typedef struct { - int evictions; + const char* skip; // accepts() answers false for this key + const char* fail_on; // accepts() reports a failure for this key + atomic_int puts, removed, expired, cleared; + atomic_int off_main_thread; // on_change calls on a producer thread + // The last Put's entry fields and the last Removed key. + uint32_t last_version; + bool last_replaced; char last_key[64]; - int32_t last_reason; - size_t last_value_len; - int detach_after; // return false (detach) once this many evictions ran - int fail; // report a consumer failure instead of observing + uint32_t last_cleared; } listener_ctx; -static int g_listener_freed = 0; +static atomic_int g_listeners_freed; -// The Entry buffer is borrowed for the duration of the dispatch. -static bool on_evict(void* ctx, const uint8_t* entry_ptr, size_t entry_len, - kvstore_kv_EvictionReason reason, kvstore_error* out_err) { +static bool listener_accepts(void* ctx, const uint8_t* key_ptr, size_t key_len, + kvstore_error* out_err) { listener_ctx* l = (listener_ctx*)ctx; - if (l->fail) { - kvstore_error_set(out_err, -4, "listener exploded"); - return true; + if (l->fail_on != NULL && bytes_eq(key_ptr, key_len, l->fail_on)) { + kvstore_error_set(out_err, -4, "listener refused"); + return false; } - kvstore_kv_Entry e; - assert(kvstore_kv_Entry_decode(entry_ptr, entry_len, &e)); - assert(e.id > 0 && e.created_at > 0); - snprintf(l->last_key, sizeof l->last_key, "%s", e.key.ptr); - l->last_value_len = e.value.len; - kvstore_kv_Entry_free(&e); - l->last_reason = (int32_t)reason; - l->evictions++; - return l->detach_after == 0 || l->evictions < l->detach_after; + return l->skip == NULL || !bytes_eq(key_ptr, key_len, l->skip); +} + +// The Change buffer is borrowed for the call. +static void listener_on_change(void* ctx, const uint8_t* change_ptr, size_t change_len, + kvstore_error* out_err) { + (void)out_err; + listener_ctx* l = (listener_ctx*)ctx; + if (!pthread_equal(pthread_self(), g_main_thread)) atomic_fetch_add(&l->off_main_thread, 1); + kvstore_kv_Change c; + assert(kvstore_kv_Change_decode(change_ptr, change_len, &c)); + switch (c.tag) { + case kvstore_kv_Change_Put: + l->last_version = c.as.Put.entry.version; + l->last_replaced = c.as.Put.replaced; + snprintf(l->last_key, sizeof l->last_key, "%s", c.as.Put.entry.key.ptr); + atomic_fetch_add(&l->puts, 1); + break; + case kvstore_kv_Change_Removed: + snprintf(l->last_key, sizeof l->last_key, "%s", c.as.Removed.key.ptr); + atomic_fetch_add(c.as.Removed.expired ? &l->expired : &l->removed, 1); + break; + case kvstore_kv_Change_Cleared: + l->last_cleared = c.as.Cleared.count; + atomic_fetch_add(&l->cleared, 1); + break; + } + kvstore_kv_Change_free(&c); } static void listener_free(void* ctx) { free(ctx); - g_listener_freed++; + atomic_fetch_add(&g_listeners_freed, 1); } -static const kvstore_kv_EvictionListener_vtable LISTENER_VTABLE = { - on_evict, - listener_free, +static const kvstore_kv_Listener_vtable LISTENER_VTABLE = { + sizeof(kvstore_kv_Listener_vtable), 0, listener_free, listener_accepts, listener_on_change, }; -static listener_ctx* new_listener(int detach_after, int fail) { +static listener_ctx* new_listener(const char* skip, const char* fail_on) { listener_ctx* l = (listener_ctx*)calloc(1, sizeof *l); assert(l != NULL); - l->detach_after = detach_after; - l->fail = fail; + l->skip = skip; + l->fail_on = fail_on; return l; } -// ── async completion state ───────────────────────────────────────────────── -static atomic_int g_compact_done = 0; -static int64_t g_compact_result = -1; -static int32_t g_compact_err = -1; +// ── policy (consumer-implemented, rich returns, throws) ──────────────────── -static void on_compact_done(void* context, kvstore_error* err, int64_t result) { - assert(context == (void*)0x5150); - g_compact_err = err ? err->code : 0; - kvstore_error_free(err); - g_compact_result = result; - atomic_store(&g_compact_done, 1); +typedef struct { + kvstore_kv_Store* other; // owned reference: where "b/" keys go + atomic_int admitted; +} policy_ctx; + +static atomic_int g_policies_freed; + +static void policy_admit(void* ctx, const uint8_t* entry_ptr, size_t entry_len, + uint8_t** out_ptr, size_t* out_len, kvstore_error* out_err) { + policy_ctx* p = (policy_ctx*)ctx; + atomic_fetch_add(&p->admitted, 1); + kvstore_kv_Entry e; + assert(kvstore_kv_Entry_decode(entry_ptr, entry_len, &e)); + assert(e.version == 0 && "the store assigns the version after admission"); + kvstore_writer w; + memset(&w, 0, sizeof w); + if (starts_with(e.key, "secret")) { + // A typed domain error with its fields as the payload. + kvstore_error_set(out_err, kvstore_kv_KvError_Rejected, "secrets are not stored"); + kvstore_kv_KvError_Rejected_payload r; + r.key = e.key; + r.reason = kvstore_str_of("no secrets"); + kvstore_kv_KvError_Rejected_payload_write(&w, &r); + kvstore_error_set_payload(out_err, w.ptr, w.len); + kvstore_writer_free(&w); + } else if (starts_with(e.key, "boom")) { + kvstore_error_set(out_err, 4242, "policy exploded"); // not a domain code: -4 + } else if (starts_with(e.key, "garbage")) { + hand_over_text("?", out_ptr, out_len); // not an Entry: -3 + } else { + // Rewrite: tag it and store it encrypted (and try to rename it, + // which the store ignores). + kvstore_list_string original_tags = e.tags; + kvstore_str original_key = e.key; + kvstore_str tags[1] = {kvstore_str_of("admitted")}; + e.tags.items = tags; + e.tags.len = 1; + e.kind = kvstore_kv_EntryKind_Encrypted; + e.key = kvstore_str_of("renamed"); + kvstore_kv_Entry_write(&w, &e); + e.tags = original_tags; + e.key = original_key; + hand_over(&w, out_ptr, out_len); + } + kvstore_kv_Entry_free(&e); } -static void wait_compact(void) { - for (int i = 0; i < 5000 && !atomic_load(&g_compact_done); i++) sleep_ms(1); - assert(atomic_load(&g_compact_done)); - atomic_store(&g_compact_done, 0); +// `home` arrives as one strong reference this policy owns; the return is one +// strong reference the producer adopts. +static kvstore_kv_Store* policy_route(void* ctx, const uint8_t* key_ptr, size_t key_len, + kvstore_kv_Store* home, kvstore_error* out_err) { + (void)out_err; + policy_ctx* p = (policy_ctx*)ctx; + kvstore_str key = {(const char*)key_ptr, key_len}; + if (starts_with(key, "b/")) { + kvstore_kv_Store_destroy(home); + return kvstore_kv_Store_clone(p->other); + } + if (starts_with(key, "null/")) { + kvstore_kv_Store_destroy(home); + return NULL; // a required object may not be null: -3 + } + return home; } -// ── helpers ──────────────────────────────────────────────────────────────── +static void policy_free(void* ctx) { + policy_ctx* p = (policy_ctx*)ctx; + kvstore_kv_Store_destroy(p->other); + free(p); + atomic_fetch_add(&g_policies_freed, 1); +} + +static const kvstore_kv_Policy_vtable POLICY_VTABLE = { + sizeof(kvstore_kv_Policy_vtable), 0, policy_free, policy_admit, policy_route, +}; + +// ── loader (consumer-implemented, passed as an optional parameter) ───────── -// The `ttl_seconds: i64?` parameter, encoded present or absent. typedef struct { + kvstore_kv_Store* fallback; // owned reference or NULL +} loader_ctx; + +static atomic_int g_loaders_freed; + +static void loader_name(void* ctx, uint8_t** out_ptr, size_t* out_len, kvstore_error* out_err) { + (void)ctx; + (void)out_err; + hand_over_text("c-loader", out_ptr, out_len); +} + +static kvstore_kv_Store* loader_fallback(void* ctx, const uint8_t* key_ptr, size_t key_len, + kvstore_error* out_err) { + (void)out_err; + loader_ctx* l = (loader_ctx*)ctx; + if (l->fallback != NULL && bytes_eq(key_ptr, key_len, "fb")) { + return kvstore_kv_Store_clone(l->fallback); + } + return NULL; // `Store?`: none +} + +static void fail_not_found(kvstore_error* out_err, const char* key) { + kvstore_error_set(out_err, kvstore_kv_KvError_KeyNotFound, "not in the loader"); + kvstore_kv_KvError_KeyNotFound_payload p; + p.key = kvstore_str_of(key); kvstore_writer w; -} ttl_buf; + memset(&w, 0, sizeof w); + kvstore_kv_KvError_KeyNotFound_payload_write(&w, &p); + kvstore_error_set_payload(out_err, w.ptr, w.len); + kvstore_writer_free(&w); +} -static ttl_buf ttl(const int64_t* seconds) { - ttl_buf t; - memset(&t, 0, sizeof t); - kvstore_opt_i64_write(&t.w, seconds); - return t; +static void loader_load(void* ctx, const uint8_t* key_ptr, size_t key_len, uint8_t** out_ptr, + size_t* out_len, kvstore_error* out_err) { + (void)ctx; + if (bytes_eq(key_ptr, key_len, "missing")) { + fail_not_found(out_err, "missing"); // this key: get_or_load returns none + return; + } + if (bytes_eq(key_ptr, key_len, "elsewhere")) { + fail_not_found(out_err, "other"); // another key: passed through + return; + } + if (bytes_eq(key_ptr, key_len, "broken")) { + kvstore_error_set(out_err, -1, "loader is broken"); + return; + } + char value[80]; + snprintf(value, sizeof value, "loaded:%.*s", (int)key_len, (const char*)key_ptr); + hand_over_text(value, out_ptr, out_len); } -static kvstore_kv_Store* open_store(const char* path) { - kvstore_error err = {0}; - kvstore_kv_Store* s = kvstore_kv_Store_open(STR(path), &err); - assert(err.code == 0 && s != NULL); - return s; +static void loader_free(void* ctx) { + loader_ctx* l = (loader_ctx*)ctx; + kvstore_kv_Store_destroy(l->fallback); + free(l); + atomic_fetch_add(&g_loaders_freed, 1); } -static void put_ttl(kvstore_kv_Store* s, const char* key, const uint8_t* v, size_t n, - const int64_t* seconds) { - kvstore_error err = {0}; - ttl_buf t = ttl(seconds); - assert(kvstore_kv_Store_put(s, STR(key), v, n, kvstore_kv_EntryKind_Persistent, t.w.ptr, - t.w.len, &err)); - assert(err.code == 0); - kvstore_writer_free(&t.w); +static const kvstore_kv_Loader_vtable LOADER_VTABLE = { + sizeof(kvstore_kv_Loader_vtable), 0, loader_free, loader_name, loader_fallback, loader_load, +}; + +static loader_ctx* new_loader(kvstore_kv_Store* fallback) { + loader_ctx* l = (loader_ctx*)calloc(1, sizeof *l); + assert(l != NULL); + l->fallback = fallback; + return l; +} + +// get_or_load through a fresh loader (or none); decodes `Entry?` into `out` +// (NULL when absent) and returns false with `err` set on failure. +static bool get_or_load(const kvstore_kv_Store* s, const char* key, loader_ctx* loader, + kvstore_kv_Entry** out, kvstore_error* err) { + size_t len = 0; + const uint8_t* buf = kvstore_kv_Store_get_or_load( + s, STR(key), loader, loader ? &LOADER_VTABLE : NULL, &len, err); + if (err->code != 0) { + assert(buf == NULL); + return false; + } + assert(kvstore_opt_kv_Entry_decode(buf, len, out)); + kvstore_free_bytes((uint8_t*)buf, len); + return true; +} + +// ── async completion state ───────────────────────────────────────────────── + +typedef struct { + atomic_int done; + int32_t code; + char message[128]; + uint32_t u32; + kvstore_kv_Store* store; + uint8_t* buf; + size_t len; +} call_state; + +static void settle(call_state* c, kvstore_error* err) { + c->code = err ? err->code : 0; + if (err != NULL && err->message != NULL) { + snprintf(c->message, sizeof c->message, "%s", err->message); + } + kvstore_error_free(err); } -static void put(kvstore_kv_Store* s, const char* key, const uint8_t* v, size_t n) { - put_ttl(s, key, v, n, NULL); +static void on_u32(void* context, kvstore_error* err, uint32_t result) { + call_state* c = (call_state*)context; + settle(c, err); + c->u32 = result; + atomic_store(&c->done, 1); } -static int64_t count(const kvstore_kv_Store* s) { +static void on_store(void* context, kvstore_error* err, kvstore_kv_Store* result) { + call_state* c = (call_state*)context; + settle(c, err); + c->store = result; + atomic_store(&c->done, 1); +} + +// A buffered async result is an owned run: copy it, then release it. +static void on_buffer(void* context, kvstore_error* err, const uint8_t* ptr, size_t len) { + call_state* c = (call_state*)context; + settle(c, err); + if (ptr != NULL) { + c->buf = (uint8_t*)malloc(len); + memcpy(c->buf, ptr, len); + c->len = len; + kvstore_free_bytes((uint8_t*)ptr, len); + } + atomic_store(&c->done, 1); +} + +static void wait_for(call_state* c) { + for (int i = 0; i < 5000 && !atomic_load(&c->done); i++) sleep_ms(1); + assert(atomic_load(&c->done) && "async call completed"); +} + +// ── sections ─────────────────────────────────────────────────────────────── + +static void constructors(void) { kvstore_error err = {0}; - int64_t n = kvstore_kv_Store_count(s, &err); - assert(err.code == 0); - return n; + + // Fallible constructor: an empty path is InvalidPath (no payload). + assert(kvstore_kv_Store_open(STR(""), &err) == NULL); + assert(err.code == kvstore_kv_KvError_InvalidPath); + assert(strcmp(err.message, "invalid path") == 0); + assert(err.payload_ptr == NULL && err.payload_len == 0); + kvstore_error_clear(&err); + // A NULL string with a nonzero length is a marshalling failure. + assert(kvstore_kv_Store_open(NULL, 4, &err) == NULL && err.code == -3); + kvstore_error_clear(&err); + + kvstore_kv_Store* s = kvstore_kv_Store_new(&err); + assert(err.code == 0 && s != NULL && path_is(s, "memory")); + assert(kvstore_kv_Store_capacity(s, &err) == kvstore_kv_Store_default_capacity(&err)); + assert(kvstore_kv_Store_default_capacity(&err) == 1000000); + kvstore_kv_Store_destroy(s); + + // The async free function completes with a new object... + call_state c; + memset(&c, 0, sizeof c); + kvstore_kv_open_store(STR("/async"), on_store, &c); + wait_for(&c); + assert(c.code == 0 && c.store != NULL && path_is(c.store, "/async")); + kvstore_kv_Store_destroy(c.store); + // ...or with the typed error. + memset(&c, 0, sizeof c); + kvstore_kv_open_store(NULL, 0, on_store, &c); + wait_for(&c); + assert(c.code == kvstore_kv_KvError_InvalidPath && c.store == NULL); + assert(strcmp(c.message, "invalid path") == 0); } -// Collect list_keys(prefix?) into `out`; each key is an owned string. -static int list_keys(kvstore_kv_Store* s, const char* prefix, char** out, int cap) { +static void basics(void) { kvstore_error err = {0}; - kvstore_str view = kvstore_str_of(prefix); - kvstore_writer w; - memset(&w, 0, sizeof w); - kvstore_opt_string_write(&w, prefix ? &view : NULL); - kvstore_kv_Store_ListKeysIterator* it = kvstore_kv_Store_list_keys(s, w.ptr, w.len, &err); + kvstore_kv_Store* s = open_store("/basics"); + size_t len = 0; + + // put returns the stored entry; the version counts puts of the key. + kvstore_kv_Entry e; + assert(try_put(s, "alpha", "one", kvstore_kv_EntryKind_Persistent, NULL, &e, &err)); + assert(str_is(e.key, "alpha") && e.value.len == 3 && memcmp(e.value.ptr, "one", 3) == 0); + assert(e.kind == kvstore_kv_EntryKind_Persistent && e.version == 1); + assert(e.expires_at == NULL && e.tags.len == 0 && e.metadata.len == 0); + kvstore_kv_Entry_free(&e); + assert(try_put(s, "alpha", "two", kvstore_kv_EntryKind_Volatile, NULL, &e, &err)); + assert(e.version == 2 && e.kind == kvstore_kv_EntryKind_Volatile); + kvstore_kv_Entry_free(&e); + + // get (throwing record) and find (optional record). + const uint8_t* buf = kvstore_kv_Store_get(s, STR("alpha"), &len, &err); + assert(err.code == 0 && buf != NULL); + assert(kvstore_kv_Entry_decode(buf, len, &e)); + kvstore_free_bytes((uint8_t*)buf, len); + assert(e.value.len == 3 && memcmp(e.value.ptr, "two", 3) == 0); + kvstore_kv_Entry_free(&e); + assert(kvstore_kv_Store_get(s, STR("nope"), &len, &err) == NULL); + assert(strcmp(err.message, "key not found: nope") == 0); + expect_key_not_found(&err, "nope"); + + buf = kvstore_kv_Store_find(s, STR("alpha"), &len, &err); + kvstore_kv_Entry* found = NULL; + assert(err.code == 0 && kvstore_opt_kv_Entry_decode(buf, len, &found)); + kvstore_free_bytes((uint8_t*)buf, len); + assert(found != NULL && found->version == 2); + kvstore_opt_kv_Entry_free(&found); + buf = kvstore_kv_Store_find(s, STR("nope"), &len, &err); + assert(err.code == 0 && kvstore_opt_kv_Entry_decode(buf, len, &found) && found == NULL); + kvstore_free_bytes((uint8_t*)buf, len); + + // TTLs follow the logical clock; an expired get reports when. + const int64_t ttl = 10; + assert(try_put(s, "ttl", "x", kvstore_kv_EntryKind_Volatile, &ttl, &e, &err)); + assert(e.expires_at != NULL && *e.expires_at == 10); + kvstore_kv_Entry_free(&e); + assert(kvstore_kv_Store_now(s, &err) == 0); + assert(kvstore_kv_Store_tick(s, 9, &err) == 9 && count(s) == 2); + assert(kvstore_kv_Store_tick(s, 1, &err) == 10 && count(s) == 1); + assert(kvstore_kv_Store_get(s, STR("ttl"), &len, &err) == NULL); + assert(err.code == kvstore_kv_KvError_Expired); + kvstore_kv_KvError_Expired_payload expired; + assert(kvstore_kv_KvError_Expired_payload_decode(err.payload_ptr, err.payload_len, &expired)); + assert(str_is(expired.key, "ttl") && expired.expired_at == 10); + kvstore_kv_KvError_Expired_payload_free(&expired); + kvstore_error_clear(&err); + assert(kvstore_kv_Store_get(s, STR("ttl"), &len, &err) == NULL); + expect_key_not_found(&err, "ttl"); // the expired read removed it + + // Capacity: a new key past it is StoreFull { capacity }. + kvstore_kv_Store_set_capacity(s, 1, &err); + assert(kvstore_kv_Store_capacity(s, &err) == 1); + put(s, "alpha", "three"); // replacing is fine + assert(!try_put(s, "beta", "b", kvstore_kv_EntryKind_Volatile, NULL, NULL, &err)); + assert(err.code == kvstore_kv_KvError_StoreFull); + kvstore_kv_KvError_StoreFull_payload full; + assert(kvstore_kv_KvError_StoreFull_payload_decode(err.payload_ptr, err.payload_len, &full)); + assert(full.capacity == 1); + kvstore_error_clear(&err); + kvstore_kv_Store_set_capacity(s, 100, &err); + + // An undeclared enum value and invalid UTF-8 are marshalling failures. + assert(!try_put(s, "k", "v", (kvstore_kv_EntryKind)9, NULL, NULL, &err)); + assert(err.code == -3); + kvstore_error_clear(&err); + kvstore_writer none = prefix_buf(NULL); + const uint8_t bad_utf8[2] = {0xC3, 0x28}; + assert(kvstore_kv_Store_put(s, BYTES(bad_utf8), BYTES(bad_utf8), + kvstore_kv_EntryKind_Volatile, none.ptr, none.len, &len, + &err) == NULL); + assert(err.code == -3); + kvstore_error_clear(&err); + kvstore_writer_free(&none); + // A method on a null receiver too. + kvstore_kv_Store_count(NULL, &err); + assert(err.code == -3); + kvstore_error_clear(&err); + + // delete, clear, and the deprecated size(). + put(s, "beta", "b"); + assert(kvstore_kv_Store_delete(s, STR("beta"), &err) && err.code == 0); + assert(!kvstore_kv_Store_delete(s, STR("beta"), &err)); +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wdeprecated-declarations" + assert(kvstore_kv_Store_size(s, &err) == 1); +#pragma clang diagnostic pop + assert(kvstore_kv_Store_clear(s, &err) == 1 && count(s) == 0); + kvstore_kv_Store_destroy(s); +} + +static void iterators(void) { + kvstore_error err = {0}; + kvstore_kv_Store* s = open_store("/iter"); + put(s, "user.bob", "b"); + put(s, "user.alice", "a"); + put(s, "sys.x", "xx"); + + // keys(prefix?): owned (ptr, len) strings in key order. + const char* want_all[3] = {"sys.x", "user.alice", "user.bob"}; + kvstore_writer w = prefix_buf(NULL); + kvstore_kv_Store_KeysIterator* it = kvstore_kv_Store_keys(s, w.ptr, w.len, &err); kvstore_writer_free(&w); assert(err.code == 0 && it != NULL); - int n = 0; const uint8_t* item = NULL; size_t len = 0; - while (kvstore_kv_Store_ListKeysIterator_next(it, &item, &len, &err) == 1) { - assert(n < cap); - out[n] = (char*)calloc(len + 1, 1); - memcpy(out[n++], item, len); + int n = 0; + while (kvstore_kv_Store_KeysIterator_next(it, &item, &len, &err) == 1) { + assert(n < 3 && bytes_eq(item, len, want_all[n])); kvstore_free_bytes((uint8_t*)item, len); + n++; } - assert(err.code == 0); - kvstore_kv_Store_ListKeysIterator_destroy(it); - return n; + assert(n == 3 && err.code == 0); + // Pulling past the end stays at the end. + assert(kvstore_kv_Store_KeysIterator_next(it, &item, &len, &err) == 0); + kvstore_kv_Store_KeysIterator_destroy(it); + + // A prefix that matches nothing is KeyNotFound { key: prefix }. + w = prefix_buf("zzz"); + assert(kvstore_kv_Store_keys(s, w.ptr, w.len, &err) == NULL); + kvstore_writer_free(&w); + expect_key_not_found(&err, "zzz"); + + // Abandoning an iterator part-way releases it (and its items so far). + w = prefix_buf("user."); + it = kvstore_kv_Store_keys(s, w.ptr, w.len, &err); + kvstore_writer_free(&w); + assert(kvstore_kv_Store_KeysIterator_next(it, &item, &len, &err) == 1); + assert(bytes_eq(item, len, "user.alice")); + kvstore_free_bytes((uint8_t*)item, len); + kvstore_kv_Store_KeysIterator_destroy(it); + + // entries(prefix?): records. + w = prefix_buf("sys."); + kvstore_kv_Store_EntriesIterator* et = kvstore_kv_Store_entries(s, w.ptr, w.len, &err); + kvstore_writer_free(&w); + assert(kvstore_kv_Store_EntriesIterator_next(et, &item, &len, &err) == 1); + kvstore_kv_Entry e; + assert(kvstore_kv_Entry_decode(item, len, &e)); + kvstore_free_bytes((uint8_t*)item, len); + assert(str_is(e.key, "sys.x") && e.value.len == 2); + kvstore_kv_Entry_free(&e); + assert(kvstore_kv_Store_EntriesIterator_next(et, &item, &len, &err) == 0); + kvstore_kv_Store_EntriesIterator_destroy(et); + + // partition(prefixes): objects, created as they're pulled. + kvstore_str prefixes[3] = {kvstore_str_of("user."), kvstore_str_of("sys."), + kvstore_str_of("none.")}; + kvstore_list_string pl; + pl.items = prefixes; + pl.len = 3; + memset(&w, 0, sizeof w); + kvstore_list_string_write(&w, &pl); + kvstore_kv_Store_PartitionIterator* pt = kvstore_kv_Store_partition(s, w.ptr, w.len, &err); + kvstore_writer_free(&w); + const uint32_t want_counts[3] = {2, 1, 0}; + kvstore_kv_Store* part = NULL; + n = 0; + while (kvstore_kv_Store_PartitionIterator_next(pt, &part, &err) == 1) { + assert(part != NULL && part != s && count(part) == want_counts[n]); + char* p = path_of(part); + assert(bytes_eq((const uint8_t*)p, strlen(p), prefixes[n].ptr)); + free(p); + kvstore_kv_Store_destroy(part); // each element is an owned reference + n++; + } + assert(n == 3 && err.code == 0); + kvstore_kv_Store_PartitionIterator_destroy(pt); + kvstore_kv_Store_destroy(s); } -static kvstore_kv_StoreInfo describe(kvstore_kv_Store* s, const char* label, - const kvstore_kv_Store* mirror) { +static void listeners(void) { kvstore_error err = {0}; + kvstore_kv_Store* s = open_store("/listen"); size_t len = 0; - const uint8_t* buf = kvstore_kv_Store_describe(s, STR(label), mirror, &len, &err); - assert(err.code == 0 && buf != NULL); - kvstore_kv_StoreInfo info; - assert(kvstore_kv_StoreInfo_decode(buf, len, &info)); - kvstore_free_bytes((uint8_t*)buf, len); - return info; + + listener_ctx* l = new_listener("quiet", NULL); + uint32_t id = kvstore_kv_Store_subscribe(s, l, &LISTENER_VTABLE, &err); + assert(err.code == 0 && id > 0); + assert(kvstore_kv_Store_listener_count(s, &err) == 1); + + put(s, "a", "1"); + assert(atomic_load(&l->puts) == 1 && l->last_version == 1 && !l->last_replaced); + put(s, "a", "2"); + assert(atomic_load(&l->puts) == 2 && l->last_version == 2 && l->last_replaced); + assert(strcmp(l->last_key, "a") == 0); + put(s, "quiet", "x"); // accepts() said no + assert(atomic_load(&l->puts) == 2); + assert(kvstore_kv_Store_delete(s, STR("a"), &err)); + assert(atomic_load(&l->removed) == 1 && strcmp(l->last_key, "a") == 0); + + // An expired read removes the entry and says so. + const int64_t ttl = 1; + assert(try_put(s, "short", "x", kvstore_kv_EntryKind_Volatile, &ttl, NULL, &err)); + kvstore_kv_Store_tick(s, 1, &err); + assert(kvstore_kv_Store_get(s, STR("short"), &len, &err) == NULL); + assert(err.code == kvstore_kv_KvError_Expired); + kvstore_error_clear(&err); + assert(atomic_load(&l->expired) == 1 && strcmp(l->last_key, "short") == 0); + + assert(kvstore_kv_Store_clear(s, &err) == 1); // "quiet" was left + assert(atomic_load(&l->cleared) == 1 && l->last_cleared == 1); + assert(atomic_load(&l->off_main_thread) == 0 && "synchronous calls notify inline"); + + // Unsubscribing releases the listener once. + assert(kvstore_kv_Store_unsubscribe(s, id, &err)); + assert(atomic_load(&g_listeners_freed) == 1); + assert(!kvstore_kv_Store_unsubscribe(s, id, &err)); + assert(kvstore_kv_Store_listener_count(s, &err) == 0); + + // A listener that fails is detached (and released); the put succeeds. + listener_ctx* failing = new_listener(NULL, "boom"); + kvstore_kv_Store_subscribe(s, failing, &LISTENER_VTABLE, &err); + put(s, "fine", "1"); + assert(atomic_load(&failing->puts) == 1); + put(s, "boom", "1"); + assert(err.code == 0 && count(s) == 2); + assert(kvstore_kv_Store_listener_count(s, &err) == 0); + assert(atomic_load(&g_listeners_freed) == 2); + + // A vtable smaller than the producer's is rejected (-3) and released. + kvstore_kv_Listener_vtable short_vtable = LISTENER_VTABLE; + short_vtable.size = (uint32_t)(sizeof short_vtable - sizeof short_vtable.on_change); + kvstore_kv_Store_subscribe(s, new_listener(NULL, NULL), &short_vtable, &err); + assert(err.code == -3); + kvstore_error_clear(&err); + assert(atomic_load(&g_listeners_freed) == 3); + + // Destroying the store releases the listeners it still holds. + kvstore_kv_Store_subscribe(s, new_listener(NULL, NULL), &LISTENER_VTABLE, &err); + kvstore_kv_Store_subscribe(s, new_listener(NULL, NULL), &LISTENER_VTABLE, &err); + assert(kvstore_kv_Store_listener_count(s, &err) == 2); + kvstore_kv_Store_destroy(s); + assert(atomic_load(&g_listeners_freed) == 5); } -static void basics(kvstore_kv_Store* store) { +static void policies(void) { kvstore_error err = {0}; - const uint8_t payload[3] = {1, 2, 3}; + kvstore_kv_Store* s = open_store("/policy"); + kvstore_kv_Store* other = open_store("/other"); - // Populate two keys so count/iterator/stats have something to report. - put(store, "alpha", BYTES(payload)); - ttl_buf none = ttl(NULL); - assert(kvstore_kv_Store_put(store, STR("beta"), BYTES(payload), - kvstore_kv_EntryKind_Volatile, none.w.ptr, none.w.len, &err)); - assert(err.code == 0); - assert(count(store) == 2); + policy_ctx* p = (policy_ctx*)calloc(1, sizeof *p); + p->other = kvstore_kv_Store_clone(other); + kvstore_kv_Store_set_policy(s, p, &POLICY_VTABLE, &err); + assert(err.code == 0 && kvstore_kv_Store_has_policy(s, &err)); + + // admit's record return is what's stored (its key and version aside). + kvstore_kv_Entry e; + assert(try_put(s, "a", "1", kvstore_kv_EntryKind_Volatile, NULL, &e, &err)); + assert(str_is(e.key, "a") && e.version == 1 && e.kind == kvstore_kv_EntryKind_Encrypted); + assert(e.tags.len == 1 && str_is(e.tags.items[0], "admitted")); + kvstore_kv_Entry_free(&e); - // An out-of-range enum discriminant is a marshalling failure (-3). - assert(!kvstore_kv_Store_put(store, STR("bad"), BYTES(payload), (kvstore_kv_EntryKind)999, - none.w.ptr, none.w.len, &err)); + // route: the object parameter and object return redirect a write. + assert(try_put(s, "b/x", "2", kvstore_kv_EntryKind_Volatile, NULL, NULL, &err)); + assert(count(s) == 1 && count(other) == 1); + + // A typed error from the throwing callback reaches the caller with its + // code, message, and payload. + assert(!try_put(s, "secret", "3", kvstore_kv_EntryKind_Volatile, NULL, NULL, &err)); + assert(err.code == kvstore_kv_KvError_Rejected); + assert(strcmp(err.message, "secrets are not stored") == 0); + kvstore_kv_KvError_Rejected_payload r; + assert(kvstore_kv_KvError_Rejected_payload_decode(err.payload_ptr, err.payload_len, &r)); + assert(str_is(r.key, "secret") && str_is(r.reason, "no secrets")); + kvstore_kv_KvError_Rejected_payload_free(&r); + kvstore_error_clear(&err); + + // A code outside the domain arrives as -4 with the consumer's message. + assert(!try_put(s, "boom", "4", kvstore_kv_EntryKind_Volatile, NULL, NULL, &err)); + assert(err.code == -4 && strcmp(err.message, "policy exploded") == 0); + kvstore_error_clear(&err); + // A return the runtime can't accept is -3: a malformed record... + assert(!try_put(s, "garbage", "5", kvstore_kv_EntryKind_Volatile, NULL, NULL, &err)); assert(err.code == -3); kvstore_error_clear(&err); - // So is invalid UTF-8 in a string parameter. - const uint8_t bad_utf8[2] = {0xC3, 0x28}; - assert(!kvstore_kv_Store_put(store, BYTES(bad_utf8), BYTES(payload), - kvstore_kv_EntryKind_Volatile, none.w.ptr, none.w.len, &err)); + // ...or a null required object. + assert(!try_put(s, "null/x", "6", kvstore_kv_EntryKind_Volatile, NULL, NULL, &err)); assert(err.code == -3); kvstore_error_clear(&err); - kvstore_writer_free(&none.w); - assert(count(store) == 2); + assert(count(s) == 1 && count(other) == 1); + assert(atomic_load(&p->admitted) == 6); - // Typed error path on a method: a missing key reports KeyNotFound. - size_t len = 0; - assert(kvstore_kv_Store_get(store, STR("missing"), &len, &err) == NULL); - assert(err.code == kvstore_kv_KvError_KeyNotFound); - assert(strcmp(err.message, "key not found") == 0); - kvstore_error_clear(&err); + // Replacing the policy releases the old one; NULL removes it. + policy_ctx* p2 = (policy_ctx*)calloc(1, sizeof *p2); + p2->other = kvstore_kv_Store_clone(other); + kvstore_kv_Store_set_policy(s, p2, &POLICY_VTABLE, &err); + assert(atomic_load(&g_policies_freed) == 1); + kvstore_kv_Store_set_policy(s, NULL, NULL, &err); + assert(err.code == 0 && atomic_load(&g_policies_freed) == 2); + assert(!kvstore_kv_Store_has_policy(s, &err)); + put(s, "secret", "now allowed"); + assert(count(s) == 2); - // The deprecated method still works; the header marks it deprecated, so - // silence the warning for this one deliberate call. -#pragma clang diagnostic push -#pragma clang diagnostic ignored "-Wdeprecated-declarations" - assert(kvstore_kv_Store_legacy_put(store, STR("old"), BYTES(payload), &err)); -#pragma clang diagnostic pop - assert(err.code == 0); - assert(kvstore_kv_Store_delete(store, STR("old"), &err) && err.code == 0); - assert(!kvstore_kv_Store_delete(store, STR("old"), &err) && err.code == 0); - - // Iterator: keys come back sorted, filtered by the optional prefix. - char* keys[4]; - int n = list_keys(store, NULL, keys, 4); - assert(n == 2 && strcmp(keys[0], "alpha") == 0 && strcmp(keys[1], "beta") == 0); - for (int i = 0; i < n; i++) free(keys[i]); - n = list_keys(store, "be", keys, 4); - assert(n == 1 && strcmp(keys[0], "beta") == 0); - free(keys[0]); - assert(list_keys(store, "zzz", keys, 4) == 0); - - // get -> buffered `Entry?` with nested list and map fields. - const uint8_t* buf = kvstore_kv_Store_get(store, STR("alpha"), &len, &err); - assert(err.code == 0 && buf != NULL); + kvstore_kv_Store_destroy(other); + kvstore_kv_Store_destroy(s); +} + +static void loaders(void) { + kvstore_error err = {0}; + kvstore_kv_Store* s = open_store("/load"); kvstore_kv_Entry* e = NULL; - assert(kvstore_opt_kv_Entry_decode(buf, len, &e)); - kvstore_free_bytes((uint8_t*)buf, len); - assert(e != NULL); - assert(e->id == 1 && "first id handed out by this store"); - assert(strcmp(e->key.ptr, "alpha") == 0); - assert(e->value.len == 3 && memcmp(e->value.ptr, payload, 3) == 0); - assert(e->created_at > 0); - assert(e->expires_at == NULL && "no TTL"); - assert(e->tags.len == 0 && e->metadata.len == 0); + + // No loader (a null optional callback): a miss is none. + assert(get_or_load(s, "k", NULL, &e, &err) && e == NULL); + + // load's bytes are stored, tagged with the loader's name. + assert(get_or_load(s, "k", new_loader(NULL), &e, &err) && e != NULL); + assert(e->value.len == 8 && memcmp(e->value.ptr, "loaded:k", 8) == 0); + assert(e->kind == kvstore_kv_EntryKind_Volatile && e->metadata.len == 1); + assert(str_is(e->metadata.keys[0], "source") && str_is(e->metadata.values[0], "c-loader")); + kvstore_opt_kv_Entry_free(&e); + assert(atomic_load(&g_loaders_freed) == 1 && "a loader is released after the call"); + assert(count(s) == 1); + // A hit doesn't consult the loader. + assert(get_or_load(s, "k", new_loader(NULL), &e, &err) && e != NULL && e->version == 1); kvstore_opt_kv_Entry_free(&e); - // kv.stats submodule: takes the parent module's Store. - buf = kvstore_kv_stats_get_stats(store, &len, &err); - assert(err.code == 0 && buf != NULL); - kvstore_kv_stats_Stats stats; - assert(kvstore_kv_stats_Stats_decode(buf, len, &stats)); - kvstore_free_bytes((uint8_t*)buf, len); - assert(stats.total_entries == 2); - assert(stats.total_bytes == 6 && "two 3-byte values"); - assert(stats.expired_entries == 0); + // The fallback store (an optional object return) is consulted first. + kvstore_kv_Store* backup = open_store("/backup"); + put(backup, "fb", "from backup"); + assert(get_or_load(s, "fb", new_loader(kvstore_kv_Store_clone(backup)), &e, &err)); + assert(e != NULL && e->value.len == 11 && e->kind == kvstore_kv_EntryKind_Persistent); + kvstore_opt_kv_Entry_free(&e); + kvstore_kv_Store_destroy(backup); + + // KeyNotFound for this key: the producer decoded the payload and + // answers none. + assert(get_or_load(s, "missing", new_loader(NULL), &e, &err) && e == NULL); + // KeyNotFound for another key: passed through, payload intact. + assert(!get_or_load(s, "elsewhere", new_loader(NULL), &e, &err)); + assert(strcmp(err.message, "not in the loader") == 0); + expect_key_not_found(&err, "other"); + // Any other failure is -4. + assert(!get_or_load(s, "broken", new_loader(NULL), &e, &err)); + assert(err.code == -4 && strcmp(err.message, "loader is broken") == 0); + kvstore_error_clear(&err); + + assert(atomic_load(&g_loaders_freed) == 6); + kvstore_kv_Store_destroy(s); } -static void listeners(kvstore_kv_Store* store) { +static void async_calls(void) { kvstore_error err = {0}; - const uint8_t payload[3] = {1, 2, 3}; - size_t len = 0; + kvstore_kv_Store* s = open_store("/async-calls"); + listener_ctx* l = new_listener(NULL, NULL); + kvstore_kv_Store_subscribe(s, l, &LISTENER_VTABLE, &err); - // delete fires on_evict synchronously with the Entry and reason Deleted. - listener_ctx* l1 = new_listener(0, 0); - kvstore_kv_Store_set_eviction_listener(store, l1, &LISTENER_VTABLE, &err); - assert(err.code == 0); - assert(kvstore_kv_Store_delete(store, STR("beta"), &err) && err.code == 0); - assert(l1->evictions == 1); - assert(strcmp(l1->last_key, "beta") == 0); - assert(l1->last_reason == kvstore_kv_EvictionReason_Deleted); - assert(l1->last_value_len == 3); - - // An expired entry is evicted on read: get reports Expired and the - // listener sees reason Expired. - const int64_t past = -1; - put_ttl(store, "stale", payload, 2, &past); - assert(kvstore_kv_Store_get(store, STR("stale"), &len, &err) == NULL); - assert(err.code == kvstore_kv_KvError_Expired); - kvstore_error_clear(&err); - assert(l1->evictions == 2); - assert(strcmp(l1->last_key, "stale") == 0); - assert(l1->last_reason == kvstore_kv_EvictionReason_Expired); - assert(l1->last_value_len == 2); - - // Replacing the listener frees the previous one; clearing frees the - // current one. - assert(g_listener_freed == 0); - kvstore_kv_Store_set_eviction_listener(store, new_listener(0, 0), &LISTENER_VTABLE, &err); - assert(g_listener_freed == 1 && "replaced listener freed"); - kvstore_kv_Store_clear_eviction_listener(store, &err); - assert(err.code == 0 && g_listener_freed == 2); - kvstore_kv_Store_clear_eviction_listener(store, &err); - assert(err.code == 0 && g_listener_freed == 2); - - // Nothing is attached now: a delete is not observed anywhere. - put(store, "gamma", payload, 1); - assert(kvstore_kv_Store_delete(store, STR("gamma"), &err)); - - // A listener returning false detaches itself (and is freed). - kvstore_kv_Store_set_eviction_listener(store, new_listener(1, 0), &LISTENER_VTABLE, &err); - put(store, "d1", payload, 1); - put(store, "d2", payload, 1); - assert(kvstore_kv_Store_delete(store, STR("d1"), &err)); - assert(g_listener_freed == 3 && "listener that answered false is freed"); - assert(kvstore_kv_Store_delete(store, STR("d2"), &err)); - assert(g_listener_freed == 3); - - // A listener that reports a failure aborts the delete with -4; the store - // stays usable and still holds the listener until it is cleared. - kvstore_kv_Store_set_eviction_listener(store, new_listener(0, 1), &LISTENER_VTABLE, &err); - put(store, "boom", payload, 1); - kvstore_kv_Store_delete(store, STR("boom"), &err); - assert(err.code == -4); - assert(err.message != NULL && strstr(err.message, "listener exploded") != NULL); - kvstore_error_clear(&err); - assert(count(store) == 1 && "the entry was removed before the listener ran"); - assert(g_listener_freed == 3); - kvstore_kv_Store_clear_eviction_listener(store, &err); - assert(g_listener_freed == 4); -} - -static void compaction(kvstore_kv_Store* store) { - const uint8_t payload[3] = {1, 2, 3}; - - // An immediately-expired entry gives compact 3 bytes to reclaim. The - // launcher returns at once; completion arrives on a producer thread. - const int64_t now = 0; - put_ttl(store, "doomed", BYTES(payload), &now); - kvstore_kv_Store_compact(store, NULL, on_compact_done, (void*)0x5150); - wait_compact(); - assert(g_compact_err == 0); - assert(g_compact_result == 3); - assert(count(store) == 1); - - // With a live token, compaction runs normally. + const int64_t ttl = 1; + assert(try_put(s, "old1", "x", kvstore_kv_EntryKind_Volatile, &ttl, NULL, &err)); + assert(try_put(s, "old2", "x", kvstore_kv_EntryKind_Volatile, &ttl, NULL, &err)); + put(s, "keep", "x"); + kvstore_kv_Store_tick(s, 5, &err); + + // compact runs on a producer thread and notifies listeners there. + call_state c; + memset(&c, 0, sizeof c); kvstore_cancel_token* token = kvstore_cancel_token_create(); - kvstore_kv_Store_compact(store, token, on_compact_done, (void*)0x5150); - wait_compact(); - assert(g_compact_err == 0 && g_compact_result == 0); + kvstore_kv_Store_compact(s, 0, token, on_u32, &c); kvstore_cancel_token_destroy(token); + wait_for(&c); + assert(c.code == 0 && c.u32 == 2); + assert(atomic_load(&l->expired) == 2); + assert(atomic_load(&l->off_main_thread) == 2 && "notified from a producer thread"); + assert(count(s) == 1); + + // A null token never cancels. + memset(&c, 0, sizeof c); + kvstore_kv_Store_compact(s, 5, NULL, on_u32, &c); + wait_for(&c); + assert(c.code == 0 && c.u32 == 0); - // A cancelled token completes the call with the cancelled code (-5) - // through the heap-boxed error. + // Cancel mid-pause: the call completes with -5 at once, and the + // background pause notices the token and stops. + memset(&c, 0, sizeof c); token = kvstore_cancel_token_create(); - assert(!kvstore_cancel_token_is_cancelled(token)); + kvstore_kv_Store_compact(s, 60000, token, on_u32, &c); + sleep_ms(20); + assert(!atomic_load(&c.done)); + assert(kvstore_kv_Store_active_jobs(&err) >= 1); kvstore_cancel_token_cancel(token); - assert(kvstore_cancel_token_is_cancelled(token)); - kvstore_kv_Store_compact(store, token, on_compact_done, (void*)0x5150); kvstore_cancel_token_destroy(token); - wait_compact(); - assert(g_compact_err == -5); - assert(count(store) == 1); + wait_for(&c); + assert(c.code == -5); + int stopped = 0; + for (int i = 0; i < 2000 && !stopped; i++) { + stopped = kvstore_kv_Store_active_jobs(&err) == 0; + if (!stopped) sleep_ms(1); + } + assert(stopped && "the cancelled pause stopped cooperatively"); + + // get_many: an async list of optional records, launched concurrently. + kvstore_str keys[3] = {kvstore_str_of("keep"), kvstore_str_of("gone"), + kvstore_str_of("keep")}; + kvstore_list_string kl; + kl.items = keys; + kl.len = 3; + kvstore_writer w; + memset(&w, 0, sizeof w); + kvstore_list_string_write(&w, &kl); + enum { CONCURRENT = 32 }; + static call_state many[CONCURRENT]; + memset(many, 0, sizeof many); + for (int i = 0; i < CONCURRENT; i++) { + kvstore_kv_Store_get_many(s, w.ptr, w.len, on_buffer, &many[i]); + } + kvstore_writer_free(&w); + for (int i = 0; i < CONCURRENT; i++) { + wait_for(&many[i]); + assert(many[i].code == 0); + kvstore_list_opt_kv_Entry got; + assert(kvstore_list_opt_kv_Entry_decode(many[i].buf, many[i].len, &got)); + assert(got.len == 3 && got.items[0] != NULL && got.items[1] == NULL); + assert(got.items[2] != NULL && str_is(got.items[2]->key, "keep")); + kvstore_list_opt_kv_Entry_free(&got); + free(many[i].buf); + } + + // The nested module's async free function: objects in a list in, + // a record out. + kvstore_kv_Store* other = open_store("/other"); + put(other, "a", "123"); + kvstore_kv_Store* both[2] = {s, other}; + kvstore_list_kv_Store sl; + sl.items = both; + sl.len = 2; + memset(&w, 0, sizeof w); + kvstore_list_kv_Store_write(&w, &sl); + memset(&c, 0, sizeof c); + kvstore_kv_stats_summarize_all(w.ptr, w.len, on_buffer, &c); + kvstore_writer_free(&w); + wait_for(&c); + assert(c.code == 0); + kvstore_kv_stats_Stats st; + assert(kvstore_kv_stats_Stats_decode(c.buf, c.len, &st)); + free(c.buf); + assert(st.entries == 2 && st.bytes == 4); + assert(st.by_kind.len == 1 && st.by_kind.keys[0] == kvstore_kv_EntryKind_Persistent); + assert(st.by_kind.values[0] == 2); + kvstore_kv_stats_Stats_free(&st); + + kvstore_kv_Store_destroy(other); + kvstore_kv_Store_destroy(s); } -static void object_graph(kvstore_kv_Store* store) { +static kvstore_kv_StoreInfo describe(kvstore_kv_Store* s, const char* label, + const kvstore_kv_Store* mirror) { kvstore_error err = {0}; - const uint8_t payload[3] = {1, 2, 3}; size_t len = 0; + const uint8_t* buf = kvstore_kv_Store_describe(s, STR(label), mirror, &len, &err); + assert(err.code == 0 && buf != NULL); + kvstore_kv_StoreInfo info; + assert(kvstore_kv_StoreInfo_decode(buf, len, &info)); + kvstore_free_bytes((uint8_t*)buf, len); + return info; +} - // share() returns the very same object; after the original reference is - // released the shared one still sees the data. - kvstore_kv_Store* shared = kvstore_kv_Store_share(store, &err); - assert(err.code == 0 && shared == store); - kvstore_kv_Store_destroy(store); - assert(count(shared) == 1 && "still alive through the shared reference"); - put(shared, "via-shared", payload, 1); - assert(count(shared) == 2); - store = shared; - - // clone: same pointer; destroy the original, the clone still works. - kvstore_kv_Store* cloned = kvstore_kv_Store_clone(store); - assert(cloned == store); - kvstore_kv_Store_destroy(store); - assert(count(cloned) == 2 && "clone outlives the destroyed original"); - store = cloned; +static void object_graph(void) { + kvstore_error err = {0}; + size_t len = 0; + kvstore_kv_Store* s = open_store("/graph"); + put(s, "k", "v"); + + // share(): the same object; clone/destroy count references. + kvstore_kv_Store* shared = kvstore_kv_Store_share(s, &err); + assert(err.code == 0 && shared == s); + kvstore_kv_Store_destroy(s); + assert(count(shared) == 1 && "alive through the shared reference"); + s = shared; + kvstore_kv_Store* cloned = kvstore_kv_Store_clone(s); + assert(cloned == s); + kvstore_kv_Store_destroy(cloned); assert(kvstore_kv_Store_clone(NULL) == NULL); kvstore_kv_Store_destroy(NULL); - // fork() is a distinct object with a copy of the entries. - kvstore_kv_Store* forked = kvstore_kv_Store_fork(store, &err); - assert(err.code == 0 && forked != NULL && forked != store); - assert(count(forked) == 2); - put(forked, "only-in-fork", payload, 1); - assert(count(forked) == 3 && count(store) == 2); - - // larger(): `Store?` in and out; NULL is "none" both ways, and a returned - // object is an owned reference. - kvstore_kv_Store* empty = open_store("/tmp/conformance-kvstore-c-empty"); - assert(kvstore_kv_Store_larger(empty, NULL, &err) == NULL); - assert(err.code == 0); - kvstore_kv_Store* bigger = kvstore_kv_Store_larger(empty, forked, &err); - assert(bigger == forked); - kvstore_kv_Store_destroy(bigger); - bigger = kvstore_kv_Store_larger(forked, store, &err); - assert(bigger == forked && "self wins when it holds more"); + // fork(): a distinct object with a copy of the entries. + kvstore_kv_Store* fork = kvstore_kv_Store_fork(s, &err); + assert(fork != NULL && fork != s && count(fork) == 1 && path_is(fork, "/graph")); + put(fork, "k2", "v"); + assert(count(fork) == 2 && count(s) == 1); + + // larger(): `Store?` in and out. + kvstore_kv_Store* empty = open_store("/empty"); + assert(kvstore_kv_Store_larger(empty, NULL, &err) == NULL && err.code == 0); + kvstore_kv_Store* bigger = kvstore_kv_Store_larger(empty, fork, &err); + assert(bigger == fork); kvstore_kv_Store_destroy(bigger); - bigger = kvstore_kv_Store_larger(store, NULL, &err); - assert(bigger == store && "a non-empty self is returned when other is absent"); + bigger = kvstore_kv_Store_larger(s, NULL, &err); + assert(bigger == s); kvstore_kv_Store_destroy(bigger); - // describe(): a record carrying the object itself and an optional object. - kvstore_kv_StoreInfo info = describe(store, "primary", NULL); - assert(strcmp(info.label.ptr, "primary") == 0); - assert(info.store == store && "describe().store is the receiver"); - assert(info.mirror == NULL && info.count == 2); - kvstore_kv_StoreInfo_free(&info); // releases the adopted reference - - info = describe(forked, "with-mirror", store); - assert(info.store == forked && info.mirror == store && info.count == 3); - assert(count(info.mirror) == 2 && "the mirror token is a live reference"); - kvstore_kv_StoreInfo_free(&info); + // describe(): a record whose fields carry objects. + kvstore_kv_StoreInfo info = describe(s, "main", fork); + assert(str_is(info.label, "main") && info.store == s && info.mirror == fork); + assert(info.count == 1 && count(info.mirror) == 2); - // open_many(): a list of objects as a return. + // open_many(): a list of objects; one bad path fails the whole call. kvstore_str paths[2] = {kvstore_str_of("/a"), kvstore_str_of("/b")}; - kvstore_list_string path_list; - path_list.items = paths; - path_list.len = 2; + kvstore_list_string pl; + pl.items = paths; + pl.len = 2; kvstore_writer w; memset(&w, 0, sizeof w); - kvstore_list_string_write(&w, &path_list); + kvstore_list_string_write(&w, &pl); const uint8_t* buf = kvstore_kv_Store_open_many(w.ptr, w.len, &len, &err); kvstore_writer_free(&w); - assert(err.code == 0 && buf != NULL); kvstore_list_kv_Store many; - assert(kvstore_list_kv_Store_decode(buf, len, &many)); + assert(err.code == 0 && kvstore_list_kv_Store_decode(buf, len, &many)); kvstore_free_bytes((uint8_t*)buf, len); - assert(many.len == 2 && many.items[0] != many.items[1]); - put(many.items[0], "m", payload, 1); - assert(count(many.items[0]) == 1 && count(many.items[1]) == 0); - - // A failing path fails the whole call with the typed code and no list. + assert(many.len == 2 && path_is(many.items[0], "/a") && path_is(many.items[1], "/b")); paths[1] = kvstore_str_of(""); memset(&w, 0, sizeof w); - kvstore_list_string_write(&w, &path_list); + kvstore_list_string_write(&w, &pl); assert(kvstore_kv_Store_open_many(w.ptr, w.len, &len, &err) == NULL); kvstore_writer_free(&w); - assert(err.code == kvstore_kv_KvError_IoError); - assert(strcmp(err.message, "I/O failure") == 0); + assert(err.code == kvstore_kv_KvError_InvalidPath); kvstore_error_clear(&err); - // total_count(): objects inside a list and inside an optional record. - // The writer stores a fresh `_clone` per object, which the producer - // adopts and drops; our own references stay valid. - kvstore_kv_Store* three[3] = {many.items[0], many.items[1], forked}; + // by_label(): records with objects in, a map with object values out. + kvstore_kv_StoreInfo infos[2]; + infos[0] = info; + infos[1].label = kvstore_str_of("first"); + infos[1].store = many.items[0]; + infos[1].mirror = NULL; + infos[1].count = 0; + kvstore_list_kv_StoreInfo il; + il.items = infos; + il.len = 2; + memset(&w, 0, sizeof w); + kvstore_list_kv_StoreInfo_write(&w, &il); + buf = kvstore_kv_Store_by_label(w.ptr, w.len, &len, &err); + kvstore_writer_free(&w); + kvstore_map_string_kv_Store named; + assert(err.code == 0 && kvstore_map_string_kv_Store_decode(buf, len, &named)); + kvstore_free_bytes((uint8_t*)buf, len); + assert(named.len == 2); + for (size_t i = 0; i < named.len; i++) { + if (str_is(named.keys[i], "main")) assert(named.values[i] == s); + else assert(str_is(named.keys[i], "first") && named.values[i] == many.items[0]); + } + + // total_count(): a list, a map, and an optional record, all carrying + // objects (each written as a fresh reference the producer adopts). + put(many.items[0], "m", "1"); + kvstore_kv_Store* three[3] = {many.items[0], many.items[1], fork}; kvstore_list_kv_Store stores; stores.items = three; stores.len = 3; - kvstore_kv_StoreInfo extra; - extra.label = kvstore_str_of("extra"); - extra.store = store; - extra.mirror = forked; - extra.count = count(store); - kvstore_writer ws, we; + kvstore_writer ws, wn, we; memset(&ws, 0, sizeof ws); + memset(&wn, 0, sizeof wn); memset(&we, 0, sizeof we); kvstore_list_kv_Store_write(&ws, &stores); - kvstore_opt_kv_StoreInfo_write(&we, &extra); - assert(kvstore_kv_Store_total_count(ws.ptr, ws.len, we.ptr, we.len, &err) == - 1 + 0 + 3 + 2); + kvstore_map_string_kv_Store_write(&wn, &named); + kvstore_opt_kv_StoreInfo_write(&we, &info); + // stores: 1 + 0 + 2; named: main 1 + first 1; extra: main 1. + assert(kvstore_kv_Store_total_count(ws.ptr, ws.len, wn.ptr, wn.len, we.ptr, we.len, &err) == + 6); assert(err.code == 0); kvstore_writer_free(&ws); + kvstore_writer_free(&wn); kvstore_writer_free(&we); - - // ... and with the optional record absent, plus an empty list. - stores.len = 1; + // A buffer carrying objects is consumed by the call that receives it + // (its references are adopted), so each call gets fresh buffers. memset(&ws, 0, sizeof ws); + memset(&wn, 0, sizeof wn); memset(&we, 0, sizeof we); kvstore_list_kv_Store_write(&ws, &stores); + kvstore_map_string_kv_Store_write(&wn, &named); kvstore_opt_kv_StoreInfo_write(&we, NULL); - assert(kvstore_kv_Store_total_count(ws.ptr, ws.len, we.ptr, we.len, &err) == 1); - kvstore_writer_free(&ws); - stores.len = 0; - memset(&ws, 0, sizeof ws); - kvstore_list_kv_Store_write(&ws, &stores); - assert(kvstore_kv_Store_total_count(ws.ptr, ws.len, we.ptr, we.len, &err) == 0); + assert(kvstore_kv_Store_total_count(ws.ptr, ws.len, wn.ptr, wn.len, we.ptr, we.len, &err) == + 5); kvstore_writer_free(&ws); + kvstore_writer_free(&wn); kvstore_writer_free(&we); - // Everything we still hold is intact. - assert(count(many.items[0]) == 1 && count(forked) == 3 && count(store) == 2); - - // clear() and release every reference exactly once. - kvstore_kv_Store_clear(forked, &err); - assert(err.code == 0 && count(forked) == 0); - kvstore_list_kv_Store_free(&many); // destroys both opened stores + // Everything we hold is still intact; release each reference once. + assert(count(s) == 1 && count(fork) == 2 && count(many.items[0]) == 1); + kvstore_map_string_kv_Store_free(&named); + kvstore_kv_StoreInfo_free(&info); + kvstore_list_kv_Store_free(&many); kvstore_kv_Store_destroy(empty); - kvstore_kv_Store_destroy(forked); - kvstore_kv_Store_destroy(store); + kvstore_kv_Store_destroy(fork); + kvstore_kv_Store_destroy(s); } -int main(void) { +static void stats_and_report(void) { kvstore_error err = {0}; + size_t len = 0; + kvstore_kv_Store* s = open_store("/stats"); + put(s, "b", "12"); + put(s, "a", "1"); + put(s, "a", "123"); + assert(try_put(s, "c", "x", kvstore_kv_EntryKind_Encrypted, NULL, NULL, &err)); - assert(KVSTORE_ABI_VERSION == 3u && kvstore_abi_version() == KVSTORE_ABI_VERSION); - assert(kvstore_kv_checksum() == KVSTORE_KV_CHECKSUM); + // kv.stats.summarize: the parent's Store as a parameter, the parent's + // error domain for a prefix that matches nothing. + kvstore_writer w = prefix_buf(NULL); + const uint8_t* buf = kvstore_kv_stats_summarize(s, w.ptr, w.len, &len, &err); + kvstore_writer_free(&w); + kvstore_kv_stats_Stats st; + assert(err.code == 0 && kvstore_kv_stats_Stats_decode(buf, len, &st)); + kvstore_free_bytes((uint8_t*)buf, len); + assert(st.entries == 3 && st.bytes == 6 && st.by_kind.len == 2); + for (size_t i = 0; i < st.by_kind.len; i++) { + if (st.by_kind.keys[i] == kvstore_kv_EntryKind_Persistent) assert(st.by_kind.values[i] == 2); + else assert(st.by_kind.keys[i] == kvstore_kv_EntryKind_Encrypted && st.by_kind.values[i] == 1); + } + kvstore_kv_stats_Stats_free(&st); + w = prefix_buf("q"); + assert(kvstore_kv_stats_summarize(s, w.ptr, w.len, &len, &err) == NULL); + kvstore_writer_free(&w); + expect_key_not_found(&err, "q"); - // Static method: no receiver, plain error-out slot. - assert(kvstore_kv_Store_default_capacity(&err) == 1000000); - assert(err.code == 0); + // report.render_report: the sibling root shares the Entry record. + w = prefix_buf(NULL); + kvstore_kv_Store_EntriesIterator* it = kvstore_kv_Store_entries(s, w.ptr, w.len, &err); + kvstore_writer_free(&w); + kvstore_kv_Entry entries[3]; + const uint8_t* item = NULL; + size_t n = 0; + while (kvstore_kv_Store_EntriesIterator_next(it, &item, &len, &err) == 1) { + assert(kvstore_kv_Entry_decode(item, len, &entries[n++])); + kvstore_free_bytes((uint8_t*)item, len); + } + kvstore_kv_Store_EntriesIterator_destroy(it); + assert(n == 3); + kvstore_list_kv_Entry el; + el.items = entries; + el.len = n; + memset(&w, 0, sizeof w); + kvstore_list_kv_Entry_write(&w, &el); + buf = kvstore_report_render_report(w.ptr, w.len, &len, &err); + kvstore_writer_free(&w); + kvstore_list_string lines; + assert(err.code == 0 && kvstore_list_string_decode(buf, len, &lines)); + kvstore_free_bytes((uint8_t*)buf, len); + assert(lines.len == 3); + assert(str_is(lines.items[0], "a: 3 bytes, Persistent, v2")); + assert(str_is(lines.items[1], "b: 2 bytes, Persistent")); + assert(str_is(lines.items[2], "c: 1 bytes, Encrypted")); + kvstore_list_string_free(&lines); + for (size_t i = 0; i < n; i++) kvstore_kv_Entry_free(&entries[i]); - // Fallible constructor, typed error path: an empty path reports IoError - // with the error's Display message. - assert(kvstore_kv_Store_open(STR(""), &err) == NULL); - assert(err.code == kvstore_kv_KvError_IoError); - assert(err.message != NULL && strcmp(err.message, "I/O failure") == 0); + el.len = 0; + memset(&w, 0, sizeof w); + kvstore_list_kv_Entry_write(&w, &el); + assert(kvstore_report_render_report(w.ptr, w.len, &len, &err) == NULL); + kvstore_writer_free(&w); + assert(err.code == kvstore_report_ReportError_NothingToReport); + assert(strcmp(err.message, "nothing to report") == 0); kvstore_error_clear(&err); + kvstore_kv_Store_destroy(s); +} - // A NULL string pointer with a nonzero length is a marshalling failure. - assert(kvstore_kv_Store_open(NULL, 4, &err) == NULL); - assert(err.code == -3); - kvstore_error_clear(&err); +int main(void) { + g_main_thread = pthread_self(); - kvstore_kv_Store* store = open_store("/tmp/conformance-kvstore-c"); - basics(store); - listeners(store); - compaction(store); - object_graph(store); // consumes `store` + assert(KVSTORE_ABI_VERSION == 4u && kvstore_abi_version() == KVSTORE_ABI_VERSION); + assert(kvstore_kv_contract_check() == 0); + assert(kvstore_report_contract_check() == 0); + assert(kvstore_debug_live(-1) == 1 && "the sample counts live allocations"); - // A store dropped with a listener attached frees the listener. - kvstore_kv_Store* with_listener = open_store("/tmp/conformance-kvstore-c-l"); - kvstore_kv_Store_set_eviction_listener(with_listener, new_listener(0, 0), &LISTENER_VTABLE, - &err); - assert(g_listener_freed == 4); - kvstore_kv_Store_destroy(with_listener); - assert(g_listener_freed == 5 && "dropping the store frees its listener"); + constructors(); + basics(); + iterators(); + listeners(); + policies(); + loaders(); + async_calls(); + object_graph(); + stats_and_report(); ASSERT_NO_LEAKS(kvstore_debug_live); + assert(atomic_load(&g_listeners_freed) == 6); + assert(atomic_load(&g_policies_freed) == 2); printf("c/kvstore: OK\n"); return 0; } diff --git a/conformance/c/producer.c b/conformance/c/producer.c index 6721b7f7..ff7e238a 100644 --- a/conformance/c/producer.c +++ b/conformance/c/producer.c @@ -6,11 +6,11 @@ // // * the API's own functions (`calculator_calculator_add`, ...); // * the runtime surface every consumer relies on: the ABI revision, the -// error helpers, `free_bytes` for returned strings and buffers, cancel -// tokens, and the debug leak counter; -// * one checksum function per top-level module, returning the constant the +// error helpers, `alloc` and `free_bytes` for byte runs, cancel tokens, +// and the debug leak counter; +// * one contract table per top-level module, returning the entries the // header was generated with, so a consumer generated from a different -// contract refuses to load. +// contract refuses to load and names what changed. // // The harness builds this file with -fvisibility=hidden and checks with `nm` // that every symbol is still exported (the header's CALCULATOR_API macro @@ -29,7 +29,13 @@ static atomic_uint_fast64_t live_tokens; uint32_t calculator_abi_version(void) { return CALCULATOR_ABI_VERSION; } -uint64_t calculator_calculator_checksum(void) { return CALCULATOR_CALCULATOR_CHECKSUM; } +// The table the header was generated with is exactly what this producer +// implements, so it returns the header's own entries (sorted by id). +const calculator_contract_entry* calculator_calculator_contract(size_t* out_len) { + static const calculator_contract_entry table[] = CALCULATOR_CALCULATOR_CONTRACT; + if (out_len != NULL) *out_len = CALCULATOR_CALCULATOR_CONTRACT_LEN; + return table; +} void calculator_error_set(calculator_error* err, int32_t code, const char* message) { if (err == NULL) return; @@ -43,6 +49,19 @@ void calculator_error_set(calculator_error* err, int32_t code, const char* messa } } +void calculator_error_set_payload(calculator_error* err, const uint8_t* ptr, size_t len) { + if (err == NULL) return; + free((void*)err->payload_ptr); + err->payload_ptr = NULL; + err->payload_len = 0; + if (ptr == NULL || len == 0) return; + uint8_t* copy = (uint8_t*)malloc(len); + if (copy == NULL) return; + memcpy(copy, ptr, len); + err->payload_ptr = copy; + err->payload_len = len; +} + void calculator_error_clear(calculator_error* err) { if (err == NULL) return; free((void*)err->message); @@ -58,9 +77,17 @@ void calculator_error_free(calculator_error* err) { free(err); } +// Every byte run, whether this library returned it or a consumer +// allocated it, is one malloc block, so one release function covers both. +uint8_t* calculator_alloc(size_t len) { + if (len == 0) return NULL; + uint8_t* run = (uint8_t*)calloc(len, 1); + if (run != NULL) atomic_fetch_add(&live_allocations, 1); + return run; +} + void calculator_free_bytes(uint8_t* ptr, size_t len) { - (void)len; - if (ptr != NULL) { + if (ptr != NULL && len != 0) { atomic_fetch_sub(&live_allocations, 1); free(ptr); } @@ -97,6 +124,8 @@ void calculator_cancel_token_destroy(calculator_cancel_token* token) { uint64_t calculator_debug_live(int32_t kind) { switch (kind) { + case -1: + return 1; // this producer counts tokens and allocations case 3: return atomic_load(&live_tokens); case 4: @@ -110,40 +139,39 @@ uint64_t calculator_debug_live(int32_t kind) { int32_t calculator_calculator_add(int32_t a, int32_t b, calculator_error* out_err) { (void)out_err; - return a + b; -} - -int32_t calculator_calculator_mul(int32_t a, int32_t b, calculator_error* out_err) { - (void)out_err; - return a * b; + return (int32_t)((uint32_t)a + (uint32_t)b); } -int32_t calculator_calculator_div(int32_t a, int32_t b, calculator_error* out_err) { +int32_t calculator_calculator_divide(int32_t a, int32_t b, calculator_error* out_err) { if (b == 0) { calculator_error_set(out_err, calculator_calculator_CalcError_DivisionByZero, "division by zero"); return 0; } + if (a == INT32_MIN && b == -1) return INT32_MIN; // wraps, like the Rust sample return a / b; } // Strings arrive as borrowed (ptr, len) and return as an owned run the // consumer releases with calculator_free_bytes; NULL + 0 is the empty string. -const uint8_t* calculator_calculator_echo(const uint8_t* s_ptr, size_t s_len, size_t* out_len, - calculator_error* out_err) { +const uint8_t* calculator_calculator_greet(const uint8_t* name_ptr, size_t name_len, + size_t* out_len, calculator_error* out_err) { + static const char prefix[] = "Hello, "; *out_len = 0; - if (s_ptr == NULL && s_len != 0) { + if (name_ptr == NULL && name_len != 0) { calculator_error_set(out_err, -3, "null string with a nonzero length"); return NULL; } - if (s_len == 0) return NULL; - uint8_t* copy = (uint8_t*)malloc(s_len); - if (copy == NULL) { + size_t n = (sizeof prefix - 1) + name_len + 1; + uint8_t* run = (uint8_t*)malloc(n); + if (run == NULL) { calculator_error_set(out_err, -1, "out of memory"); return NULL; } - memcpy(copy, s_ptr, s_len); + memcpy(run, prefix, sizeof prefix - 1); + if (name_len != 0) memcpy(run + sizeof prefix - 1, name_ptr, name_len); + run[n - 1] = '!'; atomic_fetch_add(&live_allocations, 1); - *out_len = s_len; - return copy; + *out_len = n; + return run; } diff --git a/conformance/c/producer_check.c b/conformance/c/producer_check.c index e82238c5..b040262a 100644 --- a/conformance/c/producer_check.c +++ b/conformance/c/producer_check.c @@ -1,7 +1,8 @@ // Drives the hand-written C producer (producer.c) through the generated // calculator header exactly as a consumer of the Rust producer would: the -// load-time contract check, a call, a typed error, a (ptr, len) string round -// trip with an interior NUL, a cancel token, and the leak counters. +// load-time contract check, calls, a typed error (and an error payload), a +// (ptr, len) string in and out with an interior NUL, a consumer-allocated +// run, a cancel token, and the leak counters. #include "harness.h" @@ -11,23 +12,33 @@ int main(void) { calculator_error err = {0}; assert(calculator_abi_version() == CALCULATOR_ABI_VERSION); - assert(calculator_calculator_checksum() == CALCULATOR_CALCULATOR_CHECKSUM); + assert(calculator_calculator_contract_check() == 0); + assert(calculator_debug_live(-1) == 1); assert(calculator_calculator_add(2, 3, &err) == 5 && err.code == 0); - assert(calculator_calculator_mul(4, -5, &err) == -20); - assert(calculator_calculator_div(9, 0, &err) == 0); + assert(calculator_calculator_add(INT32_MAX, 1, &err) == INT32_MIN); + assert(calculator_calculator_divide(-7, 2, &err) == -3 && err.code == 0); + assert(calculator_calculator_divide(9, 0, &err) == 0); assert(err.code == calculator_calculator_CalcError_DivisionByZero); assert(strcmp(err.message, "division by zero") == 0); calculator_error_clear(&err); assert(err.code == 0 && err.message == NULL); - static const char text[] = "a\0b"; + static const char name[] = "a\0b"; size_t len = 0; - const uint8_t* echoed = - calculator_calculator_echo((const uint8_t*)text, sizeof text - 1, &len, &err); - assert(err.code == 0 && len == 3 && memcmp(echoed, text, 3) == 0); + const uint8_t* greeting = + calculator_calculator_greet((const uint8_t*)name, sizeof name - 1, &len, &err); + assert(err.code == 0 && len == 11 && memcmp(greeting, "Hello, a\0b!", 11) == 0); assert(calculator_debug_live(4) == 1); - calculator_free_bytes((uint8_t*)echoed, len); + calculator_free_bytes((uint8_t*)greeting, len); + + uint8_t* staged = calculator_alloc(8); + assert(staged != NULL && calculator_alloc(0) == NULL); + calculator_error_set(&err, calculator_calculator_CalcError_DivisionByZero, "with payload"); + calculator_error_set_payload(&err, staged, 8); + assert(err.payload_len == 8); + calculator_error_clear(&err); + calculator_free_bytes(staged, 8); calculator_cancel_token* token = calculator_cancel_token_create(); assert(!calculator_cancel_token_is_cancelled(token)); diff --git a/conformance/c/run.sh b/conformance/c/run.sh index 4792aae1..bd36c7eb 100755 --- a/conformance/c/run.sh +++ b/conformance/c/run.sh @@ -6,6 +6,7 @@ # plus the value-buffer helper header) and must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools c clang # c_consumer : compile conformance/c/ against the # sample's generated headers with warnings as errors and the address and @@ -20,15 +21,13 @@ c_consumer() { && "$exe" } -c_contacts() { c_consumer contacts contacts.c; } -c_events() { c_consumer events events.c; } -c_kvstore() { c_consumer kvstore kvstore.c; } -c_async_demo() { c_consumer async-demo async_demo.c; } +c_calculator() { c_consumer calculator calculator.c; } c_codec() { c_consumer codec codec.c; } +c_kvstore() { c_consumer kvstore kvstore.c; } # Producer lane: unlike every other lane (which *consumes* a prebuilt cdylib), # this compiles a C library that *implements* the generated calculator header, -# including every runtime and checksum symbol a hand-written producer must +# including every runtime and contract symbol a hand-written producer must # export, under hidden default visibility (-fvisibility=hidden, the release # norm and the MSVC default). The header tags each prototype with # CALCULATOR_API, so the definitions stay exported; the lane checks each one @@ -43,11 +42,12 @@ c_producer_exports() { local syms syms=$(nm -g --defined-only "$lib" 2>/dev/null) || syms=$(nm -gU "$lib" 2>/dev/null) local sym - for sym in calculator_calculator_add calculator_calculator_mul \ - calculator_calculator_div calculator_calculator_echo \ - calculator_abi_version calculator_calculator_checksum \ - calculator_error_set calculator_error_clear calculator_error_free \ - calculator_free_bytes calculator_cancel_token_create \ + for sym in calculator_calculator_add calculator_calculator_divide \ + calculator_calculator_greet \ + calculator_abi_version calculator_calculator_contract \ + calculator_error_set calculator_error_set_payload calculator_error_clear \ + calculator_error_free calculator_alloc calculator_free_bytes \ + calculator_cancel_token_create \ calculator_cancel_token_cancel calculator_cancel_token_is_cancelled \ calculator_cancel_token_destroy calculator_debug_live; do if ! printf '%s\n' "$syms" | grep -Eq "(^| )_?${sym}\$"; then @@ -68,11 +68,9 @@ c_producer_exports() { LD_LIBRARY_PATH="$OUT:${LD_LIBRARY_PATH:-}" "$OUT/c_producer_check" } -lane c-contacts c_contacts -lane c-events c_events -lane c-kvstore c_kvstore -lane c-async-demo c_async_demo +lane c-calculator c_calculator lane c-codec c_codec +lane c-kvstore c_kvstore lane c-producer-exports c_producer_exports finish_lanes diff --git a/conformance/cpp/async_demo.cpp b/conformance/cpp/async_demo.cpp deleted file mode 100644 index 6298b19c..00000000 --- a/conformance/cpp/async_demo.cpp +++ /dev/null @@ -1,143 +0,0 @@ -// Conformance consumer: async-demo sample, C++ target. -// -// Drives the std::future-backed async surface end to end: `run_task` -// settled from a producer thread with a decoded TaskResult record, the typed -// InvalidNameError (extending TaskError extending Error) rethrown by -// future::get, the buffered list-of-records round trip through `run_batch`, -// the direct-scalar `run_n_tasks`, and cancellation of the cancellable -// `wait` through an RAII CancelToken: cancelling an in-flight call settles -// its future with Cancelled (code -5) long before its timeout, a token -// destroyed mid-flight leaves the call running, and omitting the token runs -// to completion. Ends by asserting no task body is still active and the -// producer's leak counters are all zero. - -#include -#include -#include -#include -#include -#include - -#include "async_demo.hpp" -#include "check.hpp" - -namespace ad = ::async_demo; - -static void run() { - ad::check_library(); - - ad::TaskResult result = ad::tasks::run_task("alpha").get(); - CHECK(result.id > 0); - CHECK(result.value == "completed: alpha"); - CHECK(result.success); - - bool threw = false; - try { - ad::tasks::run_task("").get(); - } catch (const ad::InvalidNameError& e) { - threw = e.code() == 1; - CHECK(dynamic_cast(&e) != nullptr); - CHECK(dynamic_cast(&e) != nullptr); - } - CHECK(threw); - - std::vector batch = ad::tasks::run_batch({"a", "b", "c"}).get(); - CHECK(batch.size() == 3); - const char* expected[3] = {"completed: a", "completed: b", "completed: c"}; - for (size_t i = 0; i < 3; i++) { - CHECK(batch[i].value == expected[i]); - CHECK(batch[i].success); - } - CHECK(ad::tasks::run_batch({}).get().empty()); - - CHECK(ad::tasks::run_n_tasks(7).get() == 7); - - // Many concurrent calls, each settled exactly once. - { - std::vector> pending; - for (int32_t i = 0; i < 64; i++) pending.push_back(ad::tasks::run_n_tasks(i)); - for (int32_t i = 0; i < 64; i++) CHECK(pending[i].get() == i); - } - - // No token: the call runs to its timeout. - CHECK(ad::tasks::wait(20).get() == 20); - - // Cancelling an in-flight call settles it with Cancelled right away. - { - ad::CancelToken token; - auto start = std::chrono::steady_clock::now(); - std::future pending = ad::tasks::wait(30000, token); - std::this_thread::sleep_for(std::chrono::milliseconds(20)); - CHECK(pending.wait_for(std::chrono::seconds(0)) == std::future_status::timeout); - token.cancel(); - CHECK(token.is_cancelled()); - bool cancelled = false; - try { - pending.get(); - } catch (const ad::Cancelled& e) { - cancelled = e.code() == -5; - } - CHECK(cancelled); - CHECK(std::chrono::steady_clock::now() - start < std::chrono::seconds(10)); - } - - // A token cancelled before the call starts cancels it too, and one token - // can cancel several calls. - { - ad::CancelToken token; - std::future first = ad::tasks::wait(30000, token); - std::future second = ad::tasks::wait(30000, token); - token.cancel(); - int cancelled = 0; - for (auto* f : {&first, &second}) { - try { - f->get(); - } catch (const ad::Cancelled&) { - cancelled++; - } - } - CHECK(cancelled == 2); - std::future late = ad::tasks::wait(30000, token); - bool caught = false; - try { - late.get(); - } catch (const ad::Cancelled&) { - caught = true; - } - CHECK(caught); - } - - // Destroying the consumer's token mid-flight releases only its own - // reference: the call keeps running and completes normally. - { - std::future pending; - { - ad::CancelToken token; - pending = ad::tasks::wait(50, token); - } - CHECK(pending.get() == 50); - } - - // A moved-from token is empty and cancelling it is a no-op. - { - ad::CancelToken token; - ad::CancelToken moved = std::move(token); - token.cancel(); - CHECK(!moved.is_cancelled()); - CHECK(ad::tasks::wait(1, moved).get() == 1); - } - - // Every spawned task body has finished, including the cancelled ones, - // whose futures the runtime dropped. - for (int i = 0; i < 200 && ad::tasks::active_callbacks() != 0; i++) { - std::this_thread::sleep_for(std::chrono::milliseconds(10)); - } - CHECK(ad::tasks::active_callbacks() == 0); -} - -int main() { - run(); - check_no_leaks(async_demo_debug_live, "async-demo"); - std::printf("cpp/async-demo: OK\n"); - return 0; -} diff --git a/conformance/cpp/calculator.cpp b/conformance/cpp/calculator.cpp index b1aeb4bb..d283638d 100644 --- a/conformance/cpp/calculator.cpp +++ b/conformance/cpp/calculator.cpp @@ -1,44 +1,52 @@ -// Conformance consumer: calculator sample, C++ target. +// Conformance consumer: calculator sample, C++ target (ABI revision 4). // -// Covers the smallest surface: direct scalars, a throwing function whose -// typed DivisionByZeroError derives from CalcError and Error, a string round -// trip through `std::string_view` (including an interior NUL, which the -// pointer-and-length ABI preserves), and the explicit library check. +// The getting-started surface: the load-time check, a direct-value call +// (wrapping), a throwing call whose typed DivisionByZeroError derives from +// CalcError and Error, and a string in and out through `std::string_view` +// (non-ASCII text, an astral character, and an interior NUL survive the +// pointer-and-length ABI). Invalid UTF-8 is a producer-side marshalling +// failure on a call that declares no errors, so it throws InternalError. // Ends by asserting the producer's leak counters are all zero. +#include #include #include #include "calculator.hpp" #include "check.hpp" -using namespace calculator; namespace calc = ::calculator::calculator; +using calculator::CalcError; +using calculator::DivisionByZeroError; +using calculator::Error; +using calculator::InternalError; static void run() { - check_library(); + calculator::check_library(); CHECK(calc::add(2, 3) == 5); - CHECK(calc::mul(-4, 6) == -24); - CHECK(calc::div(17, 5) == 3); - - bool caught = false; - try { - calc::div(1, 0); - } catch (const DivisionByZeroError& e) { - caught = e.code() == 1 && std::string(e.what()) == "division by zero"; - CHECK(dynamic_cast(&e) != nullptr); - CHECK(dynamic_cast(&e) != nullptr); - } - CHECK(caught); - - CHECK(calc::echo("hello") == "hello"); - CHECK(calc::echo("").empty()); - CHECK(calc::echo("héllo ✓") == "héllo ✓"); - const std::string with_nul("a\0b\0", 4); - const std::string echoed = calc::echo(with_nul); - CHECK(echoed.size() == 4); - CHECK(echoed == with_nul); + CHECK(calc::add(-7, 7) == 0); + CHECK(calc::add(INT32_MAX, 1) == INT32_MIN); + + CHECK(calc::divide(10, 2) == 5); + CHECK(calc::divide(-7, 2) == -3); + CHECK(calc::divide(INT32_MIN, -1) == INT32_MIN); + + DivisionByZeroError e = expect_throw([] { calc::divide(1, 0); }, "divide(1, 0)"); + CHECK(e.code() == 1); + CHECK(std::string(e.what()) == "division by zero"); + CHECK(dynamic_cast(&e) != nullptr); + CHECK(dynamic_cast(&e) != nullptr); + + CHECK(calc::greet("World") == "Hello, World!"); + CHECK(calc::greet("") == "Hello, !"); + CHECK(calc::greet("W\xc3\xb6rld \xf0\x9f\xa6\x80") == "Hello, W\xc3\xb6rld \xf0\x9f\xa6\x80!"); + const std::string with_nul("a\0b", 3); + const std::string greeted = calc::greet(with_nul); + CHECK(greeted == std::string("Hello, a\0b!", 11)); + + InternalError bad = expect_throw([] { calc::greet("\xc3\x28"); }, "greet(non-UTF-8)"); + CHECK(bad.code() == -3); } int main() { diff --git a/conformance/cpp/check.hpp b/conformance/cpp/check.hpp index 61c8e926..fe7c1ce4 100644 --- a/conformance/cpp/check.hpp +++ b/conformance/cpp/check.hpp @@ -1,11 +1,13 @@ // Shared helpers for the C++ conformance consumers. // // CHECK aborts the consumer with a message on a failed assertion (unlike -// assert, it isn't compiled out under NDEBUG). check_no_leaks polls a -// producer's `{prefix}_debug_live` counters until every kind reads zero: -// async completions release their last producer-side references (the task, -// the cancel token reference) just after the consumer's future is ready, so -// the counters settle shortly after the last call returns. +// assert, it isn't compiled out under NDEBUG). expect_throw runs a call that +// must throw exactly `E` (or a subclass) and returns the caught exception. +// check_no_leaks confirms the producer counts (`debug_live(-1) == 1`) and +// polls every `{prefix}_debug_live` counter until it reads zero: async +// completions release their last producer-side references (the task, the +// cancel token reference) just after the consumer's future is ready, so the +// counters settle shortly after the last call returns. #pragma once @@ -13,21 +15,46 @@ #include #include #include +#include +#include #include +#include -#define CHECK(cond) \ - do { \ - if (!(cond)) { \ +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ std::fprintf(stderr, "%s:%d: check failed: %s\n", __FILE__, __LINE__, #cond); \ - std::exit(1); \ - } \ + std::exit(1); \ + } \ } while (0) -// Assert that `debug_live(kind)` settles to zero for every kind: 0 objects, -// 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. +// Run `fn`, which must throw an `E`; return a copy of what it threw. Any +// other outcome fails with `what` naming the call. +template +E expect_throw(F&& fn, const char* what) { + try { + fn(); + } catch (const E& e) { + return e; + } catch (const std::exception& e) { + std::fprintf(stderr, "%s: threw %s (%s), not the expected exception\n", what, + typeid(e).name(), e.what()); + std::exit(1); + } + std::fprintf(stderr, "%s: returned instead of throwing\n", what); + std::exit(1); +} + +// Assert that the producer counts resources and that `debug_live(kind)` +// settles to zero for every kind: 0 objects, 1 callbacks, 2 iterators, +// 3 cancel tokens, 4 byte runs. inline void check_no_leaks(uint64_t (*debug_live)(int32_t), const char* sample) { static const char* const kinds[] = {"objects", "callbacks", "iterators", "cancel tokens", - "allocations"}; + "byte runs"}; + if (debug_live(-1) != 1) { + std::fprintf(stderr, "cpp/%s: the producer doesn't count live resources\n", sample); + std::exit(1); + } for (int32_t kind = 0; kind <= 4; ++kind) { uint64_t live = debug_live(kind); for (int i = 0; live != 0 && i < 200; ++i) { diff --git a/conformance/cpp/codec.cpp b/conformance/cpp/codec.cpp index 8d98c94c..187b007b 100644 --- a/conformance/cpp/codec.cpp +++ b/conformance/cpp/codec.cpp @@ -1,491 +1,263 @@ -// Conformance consumer: codec sample, C++ target. +// Conformance consumer: codec sample, C++ target (ABI revision 4). // -// Round-trips every value-buffer wire shape through the producer oracle with -// the generated header-only wrapper: -// - `sample_*` (producer encodes, consumer decodes) is checked field by -// field against the canonical fixture, then handed back to `verify_*` -// (consumer encodes, producer decodes) which throws `MismatchError` -// unless the bytes decode to exactly the canonical value; -// - `roundtrip_*` covers the direct, string, and bytes families plus -// consumer-built values with edge cases (empty strings, lists, and maps, -// non-ASCII text, interior NULs, i64/u64 extremes, NaN, infinities, -// negative zero); -// - `Shape` exercises every rich-enum variant as a std::variant; -// - `Holder` puts `Token` objects inside a record field, an optional, and a -// list: decoding adopts one reference per token, encoding clones one per -// token, `primary_of` returns the same object as `holder.primary`, and -// `same_primary` compares identity through two encodings. -// Ends by asserting the producer's leak counters are all zero. -// Exits non-zero on the first failed check. +// The shared-vector loop: fetch every vector the producer serves (decoded by +// the generated codecs), hand it back to `check_vector` (re-encoded by them), +// confirm it never matches its neighbor, and push each primitive vector's +// value through the matching direct-family `echo_*`. Then vectors built from +// literals (so a symmetric encode/decode bug can't hide), spot checks of +// decoded fields, the typed out-of-range error and its payload, malformed +// input rejected on both sides, and object identity and reference counting +// through value buffers. Ends by asserting the producer's leak counters are +// zero. #include #include #include -#include +#include #include -#include #include #include -#include #include #include #include "check.hpp" #include "codec.hpp" -// The root namespace is `codec` and the module namespace is `codec::codec`, -// so types live at `codec::Scalars` and functions at `codec::codec::f`. -namespace oracle = codec::codec; +namespace api = ::codec::codec; using codec::Color; using codec::Composite; using codec::Holder; +using codec::InternalError; +using codec::OutOfRangeError; using codec::Scalars; using codec::Shape; using codec::Token; +using codec::Vector; -static void check(bool ok, const char* what) { - if (!ok) { - std::fprintf(stderr, "cpp/codec: FAIL: %s\n", what); - std::exit(1); - } +template +static bool bits_eq(T a, T b) { + return std::memcmp(&a, &b, sizeof(T)) == 0; } -static bool same_f32(float a, float b) { - if (std::isnan(a) && std::isnan(b)) return true; - return a == b && std::signbit(a) == std::signbit(b); +static uint32_t find(uint32_t n, const std::string& name) { + for (uint32_t i = 0; i < n; ++i) { + if (api::vector_name(i) == name) return i; + } + std::fprintf(stderr, "no vector named %s\n", name.c_str()); + std::exit(1); } -static bool same_f64(double a, double b) { - if (std::isnan(a) && std::isnan(b)) return true; - return a == b && std::signbit(a) == std::signbit(b); +static Vector fetch(uint32_t n, const std::string& name) { return api::vector(find(n, name)); } + +// Push a primitive vector's value through its direct-family echo. +static void echo(const Vector& v) { + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_i8(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_u8(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_i16(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_u16(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_i32(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_u32(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_i64(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_u64(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(bits_eq(api::echo_f32(x->value), x->value)); + if (auto* x = std::get_if(&v.value)) CHECK(bits_eq(api::echo_f64(x->value), x->value)); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_bool(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_text(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_blob(x->value) == x->value); + if (auto* x = std::get_if(&v.value)) CHECK(api::echo_color(x->value) == x->value); } -static bool scalars_equal(const Scalars& a, const Scalars& b) { - return a.i8_value == b.i8_value && a.u8_value == b.u8_value && a.i16_value == b.i16_value && - a.u16_value == b.u16_value && a.i32_value == b.i32_value && - a.u32_value == b.u32_value && a.i64_value == b.i64_value && - a.u64_value == b.u64_value && same_f32(a.f32_value, b.f32_value) && - same_f64(a.f64_value, b.f64_value) && a.flag == b.flag && a.color == b.color; +static void every_vector(uint32_t n) { + for (uint32_t i = 0; i < n; ++i) { + Vector v = api::vector(i); + if (!api::check_vector(i, v)) { + std::fprintf(stderr, "vector %u (%s) did not round-trip; producer saw %s\n", i, + api::vector_name(i).c_str(), api::describe_vector(v).c_str()); + std::exit(1); + } + CHECK(!api::check_vector((i + 1) % n, v)); + echo(v); + } // `v` releases every token it holds. } -static bool shapes_equal(const Shape& a, const Shape& b) { - if (a.tag() != b.tag()) return false; - switch (a.tag()) { - case Shape::Tag::Empty: - return true; - case Shape::Tag::Circle: - return same_f64(std::get(a.value).radius, - std::get(b.value).radius); - case Shape::Tag::Rect: { - const auto& x = std::get(a.value); - const auto& y = std::get(b.value); - return same_f32(x.width, y.width) && same_f32(x.height, y.height); - } - case Shape::Tag::Labeled: { - const auto& x = std::get(a.value); - const auto& y = std::get(b.value); - return x.label == y.label && x.count == y.count; - } - case Shape::Tag::Nested: { - const auto& x = std::get(a.value); - const auto& y = std::get(b.value); - return scalars_equal(x.inner, y.inner) && x.note == y.note; - } - } - return false; +static Scalars canonical_scalars() { + return Scalars{-8, + 200, + -16000, + 60000, + -2000000000, + 4000000000u, + -9007199254740993LL, + UINT64_MAX, + 1.5f, + -2.25e100, + true, + Color::Blue}; } -static bool composites_equal(const Composite& a, const Composite& b) { - if (a.name != b.name || a.blob != b.blob || a.some_i64 != b.some_i64 || - a.none_i64 != b.none_i64 || a.some_text != b.some_text || a.names != b.names || - a.matrix != b.matrix || a.by_name != b.by_name || a.maybe_list != b.maybe_list || - a.sparse != b.sparse || a.colors != b.colors) { - return false; - } - if (a.empty.size() != b.empty.size()) return false; - for (size_t i = 0; i < a.empty.size(); i++) { - if (!same_f64(a.empty[i], b.empty[i])) return false; - } - if (a.by_id.size() != b.by_id.size()) return false; - for (const auto& kv : a.by_id) { - auto it = b.by_id.find(kv.first); - if (it == b.by_id.end() || !scalars_equal(kv.second, it->second)) return false; - } - if (!scalars_equal(a.scalars, b.scalars) || !shapes_equal(a.shape, b.shape)) return false; - if (a.shapes.size() != b.shapes.size()) return false; - for (size_t i = 0; i < a.shapes.size(); i++) { - if (!shapes_equal(a.shapes[i], b.shapes[i])) return false; - } - if (a.maybe_shape.has_value() != b.maybe_shape.has_value()) return false; - if (a.maybe_shape.has_value() && !shapes_equal(*a.maybe_shape, *b.maybe_shape)) return false; - return true; +static void literal_vectors(uint32_t n) { + Vector scalars{Vector::AllScalars{canonical_scalars()}}; + CHECK(api::check_vector(find(n, "scalars canonical"), scalars)); + std::get(scalars.value).value.u16_value = 60001; + CHECK(!api::check_vector(find(n, "scalars canonical"), scalars)); + + CHECK(api::check_vector(find(n, "shape labeled"), + Vector{Vector::Figure{Shape{Shape::Labeled{"tag", 3}}}})); + CHECK(api::check_vector(find(n, "string interior nul"), + Vector{Vector::Text{std::string("nul\0inside\0", 11)}})); + + // Any NaN matches the NaN vector; zero keeps its sign. + uint64_t nan_bits = 0x7ff8000000000001ull; + double nan = 0; + std::memcpy(&nan, &nan_bits, sizeof nan); + CHECK(api::check_vector(find(n, "f64 nan"), Vector{Vector::F64{nan}})); + CHECK(api::check_vector(find(n, "f64 -0"), Vector{Vector::F64{-0.0}})); + CHECK(!api::check_vector(find(n, "f64 -0"), Vector{Vector::F64{0.0}})); + + CHECK(api::check_vector(find(n, "u64 max"), Vector{Vector::U64{UINT64_MAX}})); + CHECK(api::check_vector(find(n, "enum infrared"), Vector{Vector::Hue{Color::Infrared}})); + + CHECK(api::check_vector(find(n, "optional zero"), Vector{Vector::MaybeI64{0}})); + CHECK(api::check_vector(find(n, "optional absent"), Vector{Vector::MaybeI64{std::nullopt}})); + CHECK(!api::check_vector(find(n, "optional zero"), Vector{Vector::MaybeI64{std::nullopt}})); + + // A map's entry order doesn't matter on the wire. + std::unordered_map counts; + counts.emplace("x", 0); + counts.emplace("h\xc3\xa9llo", -1); + counts.emplace("", INT64_MAX); + CHECK(api::check_vector(find(n, "map of strings"), Vector{Vector::Counts{counts}})); + + CHECK(api::check_vector(find(n, "blank"), Vector{Vector::Blank{}})); } -static void check_canonical_scalars(const Scalars& s, const char* who) { - std::string prefix = std::string(who) + ": canonical scalars "; - check(s.i8_value == -8, (prefix + "i8").c_str()); - check(s.u8_value == 200, (prefix + "u8").c_str()); - check(s.i16_value == -16000, (prefix + "i16").c_str()); - check(s.u16_value == 60000, (prefix + "u16").c_str()); - check(s.i32_value == -2000000000, (prefix + "i32").c_str()); - check(s.u32_value == 4000000000u, (prefix + "u32").c_str()); - check(s.i64_value == -9007199254740993LL, (prefix + "i64").c_str()); - check(s.u64_value == std::numeric_limits::max(), (prefix + "u64").c_str()); - check(s.f32_value == 1.5f, (prefix + "f32").c_str()); - check(s.f64_value == -2.25e100, (prefix + "f64").c_str()); - check(s.flag, (prefix + "flag").c_str()); - check(s.color == Color::Blue, (prefix + "color").c_str()); +static void spot_checks(uint32_t n) { + Vector v = fetch(n, "i64 past 2^53"); + CHECK(std::get(v.value).value == -9007199254740993LL); + + v = fetch(n, "f32 min subnormal"); + uint32_t f32_bits = 0; + float f = std::get(v.value).value; + std::memcpy(&f32_bits, &f, sizeof f32_bits); + CHECK(f32_bits == 1u); + + v = fetch(n, "string astral"); + CHECK(std::get(v.value).value == "\xf0\x9f\xa6\x80 crab \xf0\x9f\x98\x80"); + + v = fetch(n, "scalars minimum"); + const Scalars& m = std::get(v.value).value; + CHECK(m.i8_value == INT8_MIN && m.i16_value == INT16_MIN && m.i32_value == INT32_MIN); + CHECK(m.i64_value == INT64_MIN && m.u64_value == 0); + CHECK(std::isinf(m.f32_value) && m.f32_value < 0 && std::isnan(m.f64_value)); + CHECK(m.color == Color::Infrared && !m.flag); + + v = fetch(n, "composite canonical"); + CHECK(v.tag() == Vector::Tag::Deep); + const Composite& c = std::get(v.value).value; + CHECK(c.name == "h\xc3\xa9llo w\xc3\xb6rld \xe2\x9c\x93"); + CHECK(c.blob.size() == 6 && c.blob[5] == 255); + CHECK(c.some_i64 == INT64_MIN && !c.none_i64.has_value()); + CHECK(c.some_text.has_value() && c.some_text->empty()); + CHECK(c.names.size() == 3 && c.names[1].empty()); + CHECK(c.matrix.size() == 3 && c.matrix[1].empty() && c.matrix[2][0] == -4); + CHECK(c.floats.size() == 6 && std::isnan(c.floats[0]) && std::signbit(c.floats[3])); + CHECK(c.by_name.size() == 4 && c.by_id.size() == 3 && c.by_color.size() == 2 && + c.flags.size() == 2); + CHECK(c.scalars.u32_value == 4000000000u); + CHECK(c.shape.tag() == Shape::Tag::Labeled && std::get(c.shape.value).count == 3); + CHECK(c.shapes.size() == 6 && !std::get(c.shapes[5].value).note.has_value()); + CHECK(c.maybe_shape.has_value() && c.maybe_shape->tag() == Shape::Tag::Nested); + CHECK(c.maybe_list.has_value() && c.maybe_list->size() == 2); + CHECK(c.sparse.size() == 3 && !c.sparse[1].has_value() && c.sparse[0] == true); + CHECK(c.colors.size() == 4 && c.colors[3] == Color::Infrared); } -static void run() { - codec::check_library(); - - // Scalars: producer encodes, consumer decodes, and back again. - Scalars sample = oracle::sample_scalars(); - check_canonical_scalars(sample, "sample_scalars"); - check(oracle::verify_scalars(sample), "verify_scalars accepts the re-encoded sample"); - Scalars echoed = oracle::roundtrip_scalars(sample); - check_canonical_scalars(echoed, "roundtrip_scalars"); - check(scalars_equal(sample, echoed), "roundtrip_scalars is identical"); - - // A mismatch is the typed domain exception with its declared code. - { - Scalars changed = sample; - changed.u8_value = 201; - bool caught = false; - try { - oracle::verify_scalars(changed); - } catch (const codec::MismatchError& e) { - caught = (e.code() == 1); - check(dynamic_cast(&e) != nullptr, - "MismatchError is a CodecError"); - check(std::string(e.what()) == "value does not match the canonical fixture", - "MismatchError carries the doc-comment message"); - } - check(caught, "verify_scalars throws MismatchError for a changed value"); - changed = sample; - changed.flag = false; - caught = false; - try { - oracle::verify_scalars(changed); - } catch (const codec::CodecError& e) { - caught = (e.code() == 1); - } - check(caught, "a flipped bool is detected"); - } - - // Consumer-built scalars at the edges of every width. - { - Scalars edge{std::numeric_limits::min(), std::numeric_limits::max(), - std::numeric_limits::min(), std::numeric_limits::max(), - std::numeric_limits::min(), std::numeric_limits::max(), - std::numeric_limits::min(), std::numeric_limits::max(), - -0.0f, std::numeric_limits::quiet_NaN(), - false, Color::Red}; - Scalars back = oracle::roundtrip_scalars(edge); - check(scalars_equal(edge, back), "edge scalars round-trip"); - check(std::signbit(back.f32_value) && back.f32_value == 0.0f, "negative zero f32 keeps its sign"); - check(std::isnan(back.f64_value), "NaN f64 round-trips as NaN"); - - Scalars inf = edge; - inf.f32_value = std::numeric_limits::infinity(); - inf.f64_value = -std::numeric_limits::infinity(); - inf.i64_value = std::numeric_limits::max(); - inf.color = Color::Green; - back = oracle::roundtrip_scalars(inf); - check(scalars_equal(inf, back), "infinity scalars round-trip"); - check(std::isinf(back.f32_value) && back.f32_value > 0, "+inf f32"); - check(std::isinf(back.f64_value) && back.f64_value < 0, "-inf f64"); - } - - // Composite: every nested shape, checked against the canonical fixture. - Composite comp = oracle::sample_composite(); - check(comp.name == "h\xC3\xA9llo w\xC3\xB6rld \xE2\x9C\x93", "composite name (UTF-8)"); - check(comp.blob == std::vector({0, 1, 2, 253, 254, 255}), "composite blob"); - check(comp.some_i64.has_value() && *comp.some_i64 == std::numeric_limits::min(), - "composite some_i64 is i64::MIN"); - check(!comp.none_i64.has_value(), "composite none_i64 absent"); - check(comp.some_text.has_value() && comp.some_text->empty(), "composite some_text is Some(\"\")"); - check(comp.names.size() == 3 && comp.names[0] == "a" && comp.names[1].empty() && - comp.names[2] == "ccc", - "composite names"); - check(comp.matrix.size() == 3 && comp.matrix[0] == std::vector({1, 2, 3}) && - comp.matrix[1].empty() && comp.matrix[2] == std::vector({-4}), - "composite matrix"); - check(comp.empty.empty(), "composite empty list"); - check(comp.by_name.size() == 3 && comp.by_name.at("one") == 1 && comp.by_name.at("two") == 2 && - comp.by_name.at("neg") == -3, - "composite by_name"); - check(comp.by_id.size() == 2, "composite by_id has two entries"); - check_canonical_scalars(comp.by_id.at(-1), "by_id[-1]"); - check(!comp.by_id.at(42).flag && comp.by_id.at(42).u64_value == std::numeric_limits::max(), - "composite by_id[42] differs only in flag"); - check_canonical_scalars(comp.scalars, "composite.scalars"); - check(comp.shape.tag() == Shape::Tag::Labeled && - std::get(comp.shape.value).label == "tag" && - std::get(comp.shape.value).count == 3, - "composite shape is Labeled{tag, 3}"); - check(comp.shapes.size() == 5, "composite shapes has one of each variant"); - check(comp.shapes[0].tag() == Shape::Tag::Empty, "shapes[0] Empty"); - check(comp.shapes[1].tag() == Shape::Tag::Circle && - std::get(comp.shapes[1].value).radius == 2.5, - "shapes[1] Circle{2.5}"); - check(comp.shapes[2].tag() == Shape::Tag::Rect && - std::get(comp.shapes[2].value).width == 1.0f && - std::get(comp.shapes[2].value).height == 0.5f, - "shapes[2] Rect{1, 0.5}"); - check(comp.shapes[3].tag() == Shape::Tag::Labeled && - std::get(comp.shapes[3].value).label.empty() && - std::get(comp.shapes[3].value).count == -1, - "shapes[3] Labeled{\"\", -1}"); - check(comp.shapes[4].tag() == Shape::Tag::Nested, "shapes[4] Nested"); - { - const auto& nested = std::get(comp.shapes[4].value); - check_canonical_scalars(nested.inner, "shapes[4].inner"); - check(nested.note.has_value() && *nested.note == "n", "shapes[4].note is Some(n)"); - } - check(comp.maybe_shape.has_value() && comp.maybe_shape->tag() == Shape::Tag::Nested && - !std::get(comp.maybe_shape->value).note.has_value(), - "composite maybe_shape is Nested with no note"); - check(comp.maybe_list.has_value() && *comp.maybe_list == std::vector({9, 8}), - "composite maybe_list"); - check(comp.sparse.size() == 3 && comp.sparse[0] == std::optional(true) && - !comp.sparse[1].has_value() && comp.sparse[2] == std::optional(false), - "composite sparse"); - check(comp.colors == std::vector({Color::Red, Color::Green, Color::Blue}), - "composite colors"); - - check(oracle::verify_composite(comp), "verify_composite accepts the re-encoded sample"); - Composite comp_back = oracle::roundtrip_composite(comp); - check(composites_equal(comp, comp_back), "roundtrip_composite is identical"); - check(oracle::verify_composite(comp_back), "the round-tripped composite still verifies"); - { - std::string text = oracle::describe_composite(comp); - check(text.find("h\xC3\xA9llo w\xC3\xB6rld \xE2\x9C\x93") != std::string::npos, - "describe_composite renders the name"); - check(text.find("Labeled") != std::string::npos, "describe_composite renders the shape"); - - Composite changed = comp; - changed.sparse[1] = true; - bool caught = false; - try { - oracle::verify_composite(changed); - } catch (const codec::MismatchError&) { - caught = true; - } - check(caught, "a changed nested optional is detected"); - changed = comp; - changed.by_id.erase(42); - caught = false; - try { - oracle::verify_composite(changed); - } catch (const codec::MismatchError&) { - caught = true; - } - check(caught, "a removed map entry is detected"); - } +static void out_of_range(uint32_t n) { + OutOfRangeError e = expect_throw([n] { api::vector(n); }, "vector(n)"); + CHECK(e.code() == 1 && e.index == n && e.count == n); + CHECK(std::string(e.what()) == + "vector " + std::to_string(n) + " is out of range (count " + std::to_string(n) + ")"); + CHECK(dynamic_cast(&e) != nullptr); - // A consumer-built composite with the sparse/empty corners. - { - Scalars zero{0, 0, 0, 0, 0, 0, 0, 0, 0.0f, 0.0, false, Color::Red}; - Composite mine; - mine.name = ""; - mine.blob = {}; - mine.some_i64 = std::numeric_limits::max(); - mine.none_i64 = -1; - mine.some_text = std::nullopt; - mine.names = {"\xF0\x9F\x8E\x89", "", "\xE6\x97\xA5\xE6\x9C\xAC\xE8\xAA\x9E"}; - mine.matrix = {}; - mine.empty = {-0.0, std::numeric_limits::infinity(), 1e-300}; - mine.by_name = {}; - mine.by_id = {{std::numeric_limits::min(), zero}, - {0, sample}, - {std::numeric_limits::max(), zero}}; - mine.scalars = zero; - mine.shape = Shape{Shape::Empty{}}; - mine.shapes = {}; - mine.maybe_shape = std::nullopt; - mine.maybe_list = std::vector{}; - mine.sparse = {std::nullopt, std::nullopt}; - mine.colors = {}; - Composite back = oracle::roundtrip_composite(mine); - check(composites_equal(mine, back), "consumer-built composite round-trips"); - check(back.name.empty() && back.blob.empty() && back.matrix.empty() && - back.by_name.empty() && back.shapes.empty() && back.colors.empty(), - "empty containers stay empty"); - check(back.maybe_list.has_value() && back.maybe_list->empty(), - "Some(empty list) stays Some"); - check(!back.some_text.has_value() && back.none_i64 == std::optional(-1), - "optional fields keep their presence"); - check(back.by_id.size() == 3 && back.by_id.count(std::numeric_limits::min()) == 1, - "int-keyed map keeps extreme keys"); - check(std::signbit(back.empty[0]) && std::isinf(back.empty[1]) && back.empty[2] == 1e-300, - "f64 list keeps -0.0, inf, and denormal-range values"); - bool caught = false; - try { - oracle::verify_composite(mine); - } catch (const codec::MismatchError&) { - caught = true; - } - check(caught, "a non-canonical composite is rejected"); - } - - // Rich enum variants one at a time, plus a list of them. - { - std::vector all{ - Shape{Shape::Empty{}}, - Shape{Shape::Circle{-0.0}}, - Shape{Shape::Rect{std::numeric_limits::infinity(), -1.25f}}, - Shape{Shape::Labeled{"h\xC3\xA9", std::numeric_limits::min()}}, - Shape{Shape::Nested{sample, std::string("")}}, - Shape{Shape::Nested{sample, std::nullopt}}, - }; - for (const Shape& s : all) { - Shape back = oracle::roundtrip_shape(s); - check(shapes_equal(s, back), "roundtrip_shape preserves the variant"); - } - check(oracle::describe_shape(all[0]) == "Empty", "describe_shape Empty"); - check(oracle::describe_shape(Shape{Shape::Circle{2.5}}) == "Circle { radius: 2.5 }", - "describe_shape Circle"); - check(oracle::describe_shape(Shape{Shape::Labeled{"x", 3}}) == - "Labeled { label: \"x\", count: 3 }", - "describe_shape Labeled"); - std::vector many_back = oracle::roundtrip_shapes(all); - check(many_back.size() == all.size(), "roundtrip_shapes keeps the count"); - for (size_t i = 0; i < all.size(); i++) { - check(shapes_equal(all[i], many_back[i]), "roundtrip_shapes preserves each element"); - } - check(oracle::roundtrip_shapes({}).empty(), "roundtrip_shapes of an empty list"); - } + e = expect_throw([n] { api::vector_name(n + 5); }, "vector_name(n + 5)"); + CHECK(e.index == n + 5 && e.count == n); - // Top-level optionals, maps, strings, bytes, and direct scalars. - { - check(oracle::roundtrip_opt_i64(std::nullopt) == std::nullopt, "roundtrip_opt_i64 none"); - check(oracle::roundtrip_opt_i64(std::numeric_limits::min()) == - std::optional(std::numeric_limits::min()), - "roundtrip_opt_i64 some(i64::MIN)"); - check(oracle::roundtrip_opt_i64(0) == std::optional(0), "roundtrip_opt_i64 some(0)"); - - std::unordered_map m{{"", 0}, {"k", -1}, {"\xE2\x9C\x93", 7}}; - check(oracle::roundtrip_map(m) == m, "roundtrip_map"); - check(oracle::roundtrip_map({}).empty(), "roundtrip_map empty"); - - check(oracle::roundtrip_string("") == "", "roundtrip_string empty"); - check(oracle::roundtrip_string("h\xC3\xA9llo \xE2\x9C\x93 \xF0\x9F\x8E\x89") == - "h\xC3\xA9llo \xE2\x9C\x93 \xF0\x9F\x8E\x89", - "roundtrip_string non-ASCII"); - // Strings cross as pointer and length, so interior NULs survive both - // as a direct argument and inside a value buffer. - const std::string with_nul("a\0b\0\0c", 6); - check(oracle::roundtrip_string(with_nul) == with_nul, "roundtrip_string interior NUL"); - check(oracle::roundtrip_string(std::string(1, '\0')).size() == 1, - "roundtrip_string lone NUL"); - std::unordered_map nul_keys{{with_nul, 1}, {std::string("\0", 1), 2}}; - check(oracle::roundtrip_map(nul_keys) == nul_keys, "roundtrip_map with NUL-bearing keys"); - - check(oracle::roundtrip_bytes({}).empty(), "roundtrip_bytes empty"); - std::vector all_bytes; - for (int i = 0; i < 256; i++) all_bytes.push_back(static_cast(i)); - check(oracle::roundtrip_bytes(all_bytes) == all_bytes, "roundtrip_bytes 0..255"); - - check(oracle::roundtrip_i64(std::numeric_limits::min()) == - std::numeric_limits::min(), - "roundtrip_i64 min"); - check(oracle::roundtrip_i64(std::numeric_limits::max()) == - std::numeric_limits::max(), - "roundtrip_i64 max"); - check(oracle::roundtrip_u64(std::numeric_limits::max()) == - std::numeric_limits::max(), - "roundtrip_u64 max"); - check(oracle::roundtrip_u64(1ull << 63) == (1ull << 63), "roundtrip_u64 2^63"); - check(std::isnan(oracle::roundtrip_f64(std::numeric_limits::quiet_NaN())), - "roundtrip_f64 NaN"); - check(oracle::roundtrip_f64(-std::numeric_limits::infinity()) == - -std::numeric_limits::infinity(), - "roundtrip_f64 -inf"); - double neg_zero = oracle::roundtrip_f64(-0.0); - check(neg_zero == 0.0 && std::signbit(neg_zero), "roundtrip_f64 -0.0"); - check(oracle::roundtrip_bool(true) && !oracle::roundtrip_bool(false), "roundtrip_bool"); - check(oracle::roundtrip_color(Color::Blue) == Color::Blue, "roundtrip_color Blue (7)"); - check(oracle::roundtrip_color(Color::Red) == Color::Red, "roundtrip_color Red"); - } + CHECK(!api::check_vector(n, Vector{Vector::Blank{}})); +} - // Objects inside buffers. - { - Token t(5); - check(t.value() == 5, "Token constructor and method"); - Token t_copy = t; - check(t_copy.handle() == t.handle() && t_copy.value() == 5, "Token copy shares the object"); - - Holder holder = oracle::make_holder(10, true); - check(holder.primary.value() == 10, "holder.primary"); - check(holder.spare.has_value() && holder.spare->value() == 11, "holder.spare"); - check(holder.many.size() == 3 && holder.many[0].value() == 12 && - holder.many[1].value() == 13 && holder.many[2].value() == 14, - "holder.many"); - check(holder.primary.handle() != holder.spare->handle(), "distinct tokens are distinct objects"); - - // Encoding clones one reference per token; the wrappers stay valid. - check(oracle::sum_holder(holder) == 10 + 11 + 12 + 13 + 14, "sum_holder"); - check(oracle::sum_holder(holder) == 60, "sum_holder is repeatable (buffers are single use)"); - check(holder.primary.value() == 10 && holder.many[2].value() == 14, - "wrappers survive being encoded"); - - Token primary = oracle::primary_of(holder); - check(primary.handle() == holder.primary.handle(), "primary_of returns the same object"); - check(primary.value() == 10, "primary_of is usable"); - - check(oracle::same_primary(holder, holder), "same_primary(h, h)"); - Holder holder_copy = holder; - check(holder_copy.primary.handle() == holder.primary.handle(), - "a copied record shares its objects"); - check(oracle::same_primary(holder, holder_copy), "same_primary with a copied holder"); - Holder other = oracle::make_holder(10, true); - check(!oracle::same_primary(holder, other), "same_primary with a fresh holder is false"); - check(oracle::sum_holder(other) == 60, "fresh holder sums the same"); - - Holder without = oracle::make_holder(0, false); - check(!without.spare.has_value(), "make_holder(_, false) has no spare"); - check(oracle::sum_holder(without) == 0 + 2 + 3 + 4, "sum_holder without spare"); - - // A consumer-built holder: objects the consumer created cross inside - // a buffer and come back as the same objects. - Holder mine{Token(1), Token(2), {Token(3), Token(4)}}; - check(oracle::sum_holder(mine) == 10, "consumer-built holder sums"); - Token mine_primary = oracle::primary_of(mine); - check(mine_primary.handle() == mine.primary.handle(), "consumer-built primary identity"); - Holder bare{Token(100), std::nullopt, {}}; - check(oracle::sum_holder(bare) == 100, "holder with no spare and an empty list"); - Holder mixed{t, holder.spare, {t, primary, without.primary}}; - check(oracle::sum_holder(mixed) == 5 + 11 + 5 + 10 + 0, "holder mixing shared tokens"); - check(oracle::same_primary(mixed, Holder{t_copy, std::nullopt, {}}), - "same_primary through a consumer-built pair"); - - // Moving out leaves an empty wrapper whose destructor is a no-op. - Token moved = std::move(t_copy); - check(t_copy.handle() == nullptr && moved.value() == 5, "Token move"); - Holder moved_holder = std::move(holder_copy); - check(holder_copy.primary.handle() == nullptr && moved_holder.primary.value() == 10, - "Holder move"); - - // Many encode/decode cycles must neither leak nor double free: every - // wrapper here is still valid at scope exit and releases exactly once. - for (int i = 0; i < 1000; i++) { - Holder h = oracle::make_holder(i, (i % 2) == 0); - int64_t expected = i + (i % 2 == 0 ? i + 1 : 0) + (i + 2) + (i + 3) + (i + 4); - check(oracle::sum_holder(h) == expected, "sum_holder in a loop"); - Token p = oracle::primary_of(h); - check(p.handle() == h.primary.handle(), "primary_of in a loop"); - } - check(t.value() == 5 && primary.value() == 10 && mine.many[1].value() == 4, - "long-lived tokens are still alive after the loop"); +static void malformed() { + // An undeclared enum value is a marshalling failure (-3) on a call that + // declares no errors. + InternalError e = expect_throw( + [] { api::echo_color(static_cast(3)); }, "echo_color(3)"); + CHECK(e.code() == -3); + + // The generated decoder rejects what the producer would: an unknown tag, + // a truncated value, trailing bytes, a bad bool, and a repeated map key. + const std::vector> bad = { + {0xE7, 0x03, 0, 0}, + {7, 0, 0, 0, 1, 2, 3}, + {0, 0, 0, 0, 0}, + {11, 0, 0, 0, 2}, + {19, 0, 0, 0, 2, 0, 0, 0, 1, 0, 0, 0, 'a', 1, 0, 0, 0, 0, 0, 0, 0, + 1, 0, 0, 0, 'a', 2, 0, 0, 0, 0, 0, 0, 0}, + }; + for (const auto& buf : bad) { + InternalError d = expect_throw( + [&buf] { codec::detail::decode(buf.data(), buf.size(), &codec::detail::read_Vector); }, + "decoding a malformed Vector"); + CHECK(d.code() == -3); } +} +static void objects(uint32_t n) { + // The full object vector decodes to live tokens with the table's values. + Vector v = fetch(n, "objects full"); + const Holder& h = std::get(v.value).value; + CHECK(h.primary.value() == 10 && h.spare.has_value() && h.spare->value() == 11); + CHECK(h.many.size() == 3 && h.many[2].value() == INT64_MIN); + CHECK(h.by_name.size() == 2 && h.by_name.at("b").value() == 21); + + // Each encoding mints fresh references, so the holder can be sent twice. + const int64_t expected = 10 + 11 + 12 + 13 + 20 + 21 + INT64_MIN; + CHECK(api::sum_holder(h) == expected && api::sum_holder(h) == expected); + + // primary_of returns the very same object (a new reference to it). + Token p = api::primary_of(h); + CHECK(p.handle() == h.primary.handle()); + CHECK(api::same_primary(h, Holder{p, std::nullopt, {}, {}})); + Token twin(10); + CHECK(twin.value() == 10 && twin.handle() != h.primary.handle()); + CHECK(!api::same_primary(h, Holder{twin, std::nullopt, {}, {}})); + + // A holder built from consumer tokens, with one token in every slot. + Holder mine{twin, twin, {twin, twin, Token(-4)}, {{"k", twin}}}; + CHECK(api::sum_holder(mine) == 10 * 5 - 4); + + // The consumer-built sparse holder checks against the table. + CHECK(api::check_vector(find(n, "objects sparse"), + Vector{Vector::Objects{Holder{Token(-1), std::nullopt, {}, {}}}})); + + // Copies share the object; a moved-from wrapper is empty. + Token copy = twin; + CHECK(copy.handle() == twin.handle()); + Token moved = std::move(copy); + CHECK(moved.handle() == twin.handle() && copy.handle() == nullptr); } int main() { - run(); + codec::check_library(); + uint32_t n = api::vector_count(); + CHECK(n >= 60); + + every_vector(n); + literal_vectors(n); + spot_checks(n); + out_of_range(n); + malformed(); + objects(n); + check_no_leaks(codec_debug_live, "codec"); - std::printf("cpp/codec: OK\n"); + std::printf("cpp/codec: OK (%u vectors)\n", n); return 0; } diff --git a/conformance/cpp/contacts.cpp b/conformance/cpp/contacts.cpp deleted file mode 100644 index ee0dd270..00000000 --- a/conformance/cpp/contacts.cpp +++ /dev/null @@ -1,123 +0,0 @@ -// Conformance consumer: contacts sample, C++ target. -// -// Drives the generated header-only wrappers over the struct/enum/optional -// surface and the module graph: -// - `ContactBook` is an RAII interface class whose canonical `new` -// constructor maps to the default constructor; `Contact` is a value struct -// decoded from a value buffer; enums cross as `enum class` values; and -// throwing wrappers surface the `ContactsError` hierarchy. -// - `contacts::groups` is nested under the `contacts` root: its functions -// take and return the parent's `ContactBook` interface and `ContactType` -// enum, and report the parent's `ContactsError` domain. -// - `directory` is a sibling root sharing the `Contact` record and declaring -// its own `DirectoryError` domain. -// Ends by asserting the producer's leak counters are all zero. - -#include -#include -#include -#include - -#include "check.hpp" -#include "contacts.hpp" - -using namespace contacts; -namespace grp = ::contacts::contacts::groups; -namespace dir = ::contacts::directory; - -static void run() { - // Canonical `new` constructor maps to the default constructor. - ContactBook book; - CHECK(book.count() == 0); - - Contact alice = book.add("Alice", "Smith", std::string("alice@example.com"), ContactType::Work); - CHECK(alice.id > 0); - CHECK(alice.first_name == "Alice"); - CHECK(alice.last_name == "Smith"); - CHECK(alice.email.has_value() && *alice.email == "alice@example.com"); - CHECK(alice.contact_type == ContactType::Work); - - Contact snap = book.get(alice.id); - CHECK(snap.first_name == "Alice" && snap.last_name == "Smith"); - CHECK(snap.email.has_value() && *snap.email == "alice@example.com"); - - Contact bob = book.add("Bob", "Jones", std::nullopt, ContactType::Personal); - CHECK(!bob.email.has_value()); - CHECK(bob.contact_type == ContactType::Personal); - - Contact carol = book.add("Carol", "Adams", std::nullopt, ContactType::Work); - CHECK(book.count() == 3); - std::vector everyone = book.list(); - CHECK(everyone.size() == 3); - - // Typed errors: the per-code subclass derives from the domain and Error. - bool caught = false; - try { - book.get(9999); - } catch (const NotFoundError& e) { - caught = e.code() == 2 && std::string(e.what()) == "contact not found"; - CHECK(dynamic_cast(&e) != nullptr); - CHECK(dynamic_cast(&e) != nullptr); - } - CHECK(caught); - caught = false; - try { - book.add("", "Nameless", std::nullopt, ContactType::Other); - } catch (const InvalidNameError& e) { - caught = e.code() == 1; - } - CHECK(caught); - CHECK(book.count() == 3); - - // The nested module uses the parent's interface and enum. - CHECK(grp::count_of_type(book, ContactType::Work) == 2); - CHECK(grp::count_of_type(book, ContactType::Other) == 0); - CHECK(grp::dominant_type(book) == ContactType::Work); - ContactBook work = grp::split_by_type(book, ContactType::Work); - CHECK(work.count() == 2); - CHECK(book.count() == 3); - Contact first_personal = grp::first_of_type(book, ContactType::Personal); - CHECK(first_personal.first_name == "Bob"); - // ...and reports the parent's error domain. - caught = false; - try { - grp::first_of_type(work, ContactType::Personal); - } catch (const NotFoundError& e) { - caught = e.code() == 2; - } - CHECK(caught); - - // The sibling root shares the Contact record. - Card card = dir::card(alice); - CHECK(card.display_name == "Smith, Alice"); - CHECK(card.initials == "AS"); - CHECK(card.has_email); - CHECK(!dir::card(bob).has_email); - std::vector sorted = dir::sorted({alice, bob, carol}); - CHECK(sorted.size() == 3); - CHECK(sorted[0].last_name == "Adams" && sorted[1].last_name == "Jones" && - sorted[2].last_name == "Smith"); - caught = false; - try { - dir::sorted({}); - } catch (const EmptyError& e) { - caught = e.code() == 1; - CHECK(dynamic_cast(&e) != nullptr); - } - CHECK(caught); - - CHECK(book.remove(alice.id)); - CHECK(!book.remove(alice.id)); - CHECK(book.count() == 2); - - // Each ContactBook owns its own state. - ContactBook other; - CHECK(other.count() == 0); -} - -int main() { - run(); - check_no_leaks(contacts_debug_live, "contacts"); - std::printf("cpp/contacts: OK\n"); - return 0; -} diff --git a/conformance/cpp/events.cpp b/conformance/cpp/events.cpp deleted file mode 100644 index e36e8919..00000000 --- a/conformance/cpp/events.cpp +++ /dev/null @@ -1,367 +0,0 @@ -// Conformance consumer: events sample, C++ target. -// -// Drives the generated header-only wrapper over the callback-interface and -// reference-counted-object surface: -// - `Subscriber` is an abstract class the consumer subclasses; a -// `std::shared_ptr` crosses into the producer, which calls -// `route`, `on_message`, and `on_attached` through a static vtable and -// releases the box (running the consumer's destructor) when it drops its -// last reference. -// - `EventBus` is a copyable RAII wrapper: copies `_clone` the producer -// object and share it; moves transfer the handle; destructors release. -// - the `bus` handed to `on_attached` is an owned wrapper the consumer may -// keep and use. -// - `Delivery` return values steer `publish`'s accepted count. -// - a subscriber that throws surfaces to the caller as `Error` with code -// -4 and leaves the bus usable. `route` is a `Result`-returning method on -// the producer side, so its failure arrives there as an `Err` value the -// bus propagates; `on_message` returns a plain value, so its failure -// aborts the producer's call directly. Both reach C++ the same way. -// - `messages()` is a lazy single-pass range; `last_message()` is an -// optional record; `publish_later` is a std::future settled from a -// producer thread. -// Ends by asserting the producer's leak counters are all zero. -// Exits non-zero on the first failed check. - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "check.hpp" -#include "events.hpp" - -using namespace events; -namespace ev = ::events::events; - -static void check(bool ok, const char* what) { - if (!ok) { - std::fprintf(stderr, "cpp/events: FAIL: %s\n", what); - std::exit(1); - } -} - -// State outlives the subscriber so the test can inspect it after the -// producer has freed the implementation. -struct SubState { - std::vector routed; - std::vector received; - int attached = 0; - int64_t attached_count_seen = -1; - const events_events_EventBus* attached_handle = nullptr; - std::optional kept_bus; - int freed = 0; -}; - -class RecordingSubscriber : public Subscriber { - std::shared_ptr state_; - std::string skip_topic_; - std::string stop_topic_; - bool keep_bus_; - -public: - RecordingSubscriber(std::shared_ptr state, std::string skip_topic, - std::string stop_topic, bool keep_bus) - : state_(std::move(state)), - skip_topic_(std::move(skip_topic)), - stop_topic_(std::move(stop_topic)), - keep_bus_(keep_bus) {} - - ~RecordingSubscriber() override { state_->freed++; } - - Delivery route(std::string_view topic) override { - state_->routed.emplace_back(topic); - if (topic == skip_topic_) return Delivery::Skip; - if (topic == stop_topic_) return Delivery::AcceptAndStop; - return Delivery::Accept; - } - - int64_t on_message(const Message& message) override { - state_->received.push_back(message); - return static_cast(state_->received.size()); - } - - void on_attached(EventBus bus) override { - state_->attached++; - state_->attached_handle = bus.handle(); - // The wrapper is fully usable inside the callback; `subscribe` calls - // `on_attached` before it pushes the subscriber, so the count is the - // pre-subscribe value. - state_->attached_count_seen = bus.subscriber_count(); - if (keep_bus_) state_->kept_bus = std::move(bus); - // Otherwise `bus` goes out of scope here and releases its reference. - } -}; - -// Throws from `route` for one topic and from `on_message` for one text. -class ThrowingSubscriber : public Subscriber { - std::string route_bomb_; - std::string message_bomb_; - -public: - ThrowingSubscriber(std::string route_bomb, std::string message_bomb) - : route_bomb_(std::move(route_bomb)), message_bomb_(std::move(message_bomb)) {} - - Delivery route(std::string_view topic) override { - if (topic == route_bomb_) throw std::runtime_error("route rejected " + std::string(topic)); - return Delivery::Accept; - } - - int64_t on_message(const Message& message) override { - if (message.text == message_bomb_) throw std::logic_error("on_message exploded"); - return 1; - } - - void on_attached(EventBus) override {} -}; - -static void run() { - check_library(); - - EventBus bus; - check(bus.handle() != nullptr, "constructor yields a live handle"); - check(bus.subscriber_count() == 0, "fresh bus has no subscribers"); - check(!bus.last_message().has_value(), "fresh bus has no last message"); - { - int n = 0; - for (auto&& m : bus.messages()) { - (void)m; - n++; - } - check(n == 0, "fresh bus streams no messages"); - } - - // Copy semantics: a copy shares the object (same pointer), both usable. - { - EventBus copy = bus; - check(copy.handle() == bus.handle(), "copy constructor clones the same object"); - check(copy.subscriber_count() == 0, "copy is usable"); - EventBus assigned; - check(assigned.handle() != bus.handle(), "a second constructor makes a distinct object"); - assigned = bus; - check(assigned.handle() == bus.handle(), "copy assignment clones the same object"); - EventBus& same = assigned; - assigned = same; - check(assigned.handle() == bus.handle(), "self-assignment is a no-op"); - EventBus moved = std::move(copy); - check(moved.handle() == bus.handle(), "move constructor transfers the handle"); - check(copy.handle() == nullptr, "moved-from wrapper is empty"); - // All three destruct here: two releases plus a harmless empty one. - } - check(bus.subscriber_count() == 0, "bus survives its copies' destructors"); - - // Subscribers: A skips "quiet", B stops on "stop", C accepts everything. - auto a_state = std::make_shared(); - auto b_state = std::make_shared(); - auto c_state = std::make_shared(); - check(bus.subscribe(std::make_shared(a_state, "quiet", "", true)) == 1, - "subscribe returns 1"); - check(a_state->attached == 1, "on_attached fired once for A"); - check(a_state->attached_handle == bus.handle(), "on_attached received the same bus object"); - check(a_state->attached_count_seen == 0, "on_attached ran before the subscriber was retained"); - check(a_state->kept_bus.has_value(), "A kept the bus handed to it"); - check(a_state->kept_bus->subscriber_count() == 1, "the kept bus is usable after the callback"); - - check(bus.subscribe(std::make_shared(b_state, "", "stop", false)) == 2, - "subscribe returns 2"); - check(b_state->attached == 1 && b_state->attached_count_seen == 1, "B attached second"); - check(bus.subscribe(std::make_shared(c_state, "", "", false)) == 3, - "subscribe returns 3"); - check(bus.subscriber_count() == 3, "subscriber_count is 3"); - check(a_state->freed == 0 && b_state->freed == 0 && c_state->freed == 0, - "retained subscribers are not freed"); - - // Everyone accepts. - check(bus.publish("news", "hello", {"a", "b"}) == 3, "publish delivers to all three"); - check(a_state->received.size() == 1 && b_state->received.size() == 1 && - c_state->received.size() == 1, - "each subscriber received one message"); - { - const Message& m = a_state->received[0]; - check(m.seq == 1, "first message has seq 1"); - check(m.topic == "news", "message topic"); - check(m.text == "hello", "message text"); - check(m.tags.size() == 2 && m.tags[0] == "a" && m.tags[1] == "b", "message tags"); - } - check(a_state->routed.size() == 1 && a_state->routed[0] == "news", "route saw the topic"); - - // A skips "quiet". - check(bus.publish("quiet", "psst", {}) == 2, "Skip lowers the accepted count"); - check(a_state->received.size() == 1, "A was skipped"); - check(a_state->routed.size() == 2 && a_state->routed[1] == "quiet", "A was still asked"); - check(b_state->received.size() == 2 && b_state->received[1].seq == 2, "B got seq 2"); - check(c_state->received[1].tags.empty(), "empty tags list round-trips"); - - // B stops the chain, so C is neither asked nor delivered. - check(bus.publish("stop", "last", {"x"}) == 2, "AcceptAndStop halts later subscribers"); - check(a_state->received.size() == 2, "A accepted before the stop"); - check(b_state->received.size() == 3, "B accepted with stop"); - check(c_state->received.size() == 2, "C did not receive after the stop"); - check(c_state->routed.size() == 2, "C was not asked after the stop"); - - // Iterator: one producer `next` per step, in publish order. - { - std::vector texts; - for (auto&& t : bus.messages()) texts.push_back(t); - check(texts.size() == 3, "messages streams three texts"); - check(texts[0] == "hello" && texts[1] == "psst" && texts[2] == "last", - "messages are in publish order"); - } - // Abandoning the range early destroys the producer iterator via RAII. - { - auto range = bus.messages(); - auto it = range.begin(); - check(*it == "hello", "early-abandoned range yields the first element"); - } - // Manual `next()` drains and then reports exhaustion. - { - auto range = bus.messages(); - int n = 0; - while (range.next().has_value()) n++; - check(n == 3, "manual next() yields three elements"); - check(!range.next().has_value(), "exhausted range stays exhausted"); - } - - // Optional record return. - { - std::optional last = bus.last_message(); - check(last.has_value(), "last_message is present"); - check(last->seq == 3 && last->topic == "stop" && last->text == "last", - "last_message is the third message"); - check(last->tags.size() == 1 && last->tags[0] == "x", "last_message tags"); - } - - // Async: settled from a producer thread; subscribers run there too. - { - std::future pending = bus.publish_later("later", "async"); - check(pending.get() == 3, "publish_later resolves with the accepted count"); - check(a_state->received.size() == 3 && a_state->received[2].seq == 4, - "A received the async message"); - check(b_state->received.size() == 4 && b_state->received[3].text == "async", - "B received the async message"); - check(c_state->received.size() == 3 && c_state->received[2].topic == "later", - "C received the async message"); - std::optional last = bus.last_message(); - check(last.has_value() && last->seq == 4 && last->tags.empty(), - "async publish is logged with empty tags"); - } - - // Free function taking a callback interface: routes without retaining. - { - auto tmp_state = std::make_shared(); - auto tmp = std::make_shared(tmp_state, "quiet", "stop", false); - std::weak_ptr weak = tmp; - check(ev::route_once(std::move(tmp), "quiet") == Delivery::Skip, "route_once Skip"); - check(weak.expired() && tmp_state->freed == 1, - "route_once released the subscriber when the call returned"); - tmp_state = std::make_shared(); - check(ev::route_once(std::make_shared(tmp_state, "", "stop", false), - "stop") == Delivery::AcceptAndStop, - "route_once AcceptAndStop"); - check(ev::route_once(std::make_shared(tmp_state, "", "", false), - "other") == Delivery::Accept, - "route_once Accept"); - check(tmp_state->attached == 0, "route_once never calls on_attached"); - } - - // A null callback interface is rejected before crossing the ABI. - { - bool threw = false; - try { - bus.subscribe(nullptr); - } catch (const std::invalid_argument&) { - threw = true; - } - check(threw, "null subscriber throws std::invalid_argument"); - check(bus.subscriber_count() == 3, "null subscribe left the bus unchanged"); - } - - // Foreign errors: a throwing implementation aborts the call with -4. - { - EventBus bus2; - auto ok_state = std::make_shared(); - check(bus2.subscribe(std::make_shared(ok_state, "", "", false)) == 1, - "bus2 first subscriber"); - check(bus2.subscribe(std::make_shared("boom", "explode")) == 2, - "bus2 throwing subscriber"); - - bool caught = false; - try { - bus2.publish("boom", "x", {}); - } catch (const Error& e) { - caught = true; - check(e.code() == -4, "route exception maps to FOREIGN_ERROR_CODE (-4)"); - check(std::string(e.what()).find("route rejected boom") != std::string::npos, - "foreign error carries the C++ exception message"); - } - check(caught, "publish threw for a throwing route"); - check(ok_state->received.size() == 1, "earlier subscriber was delivered before the abort"); - - caught = false; - try { - bus2.publish("fine", "explode", {}); - } catch (const Error& e) { - caught = true; - check(e.code() == -4, "on_message exception maps to -4"); - check(std::string(e.what()).find("on_message exploded") != std::string::npos, - "on_message foreign error message"); - } - check(caught, "publish threw for a throwing on_message"); - - // The bus is still usable afterward, and the log kept both messages. - check(bus2.publish("ok", "y", {}) == 2, "bus is usable after a foreign error"); - std::optional last = bus2.last_message(); - check(last.has_value() && last->seq == 3, "aborted publishes were still logged"); - - // The async path surfaces the same code through the future. - caught = false; - try { - bus2.publish_later("boom", "z").get(); - } catch (const Error& e) { - caught = true; - check(e.code() == -4, "async foreign error maps to -4"); - } - check(caught, "publish_later future rethrows the foreign error"); - - // route_once with a throwing subscriber. - caught = false; - try { - ev::route_once(std::make_shared("boom", ""), "boom"); - } catch (const Error& e) { - caught = (e.code() == -4); - } - check(caught, "route_once surfaces the foreign error"); - - bus2.clear_subscribers(); - check(ok_state->freed == 1, "clearing bus2 freed its subscribers"); - } - - // Releasing: clear_subscribers drops the producer's references, which - // runs every implementation's destructor (A's releases its kept bus). - bus.clear_subscribers(); - check(bus.subscriber_count() == 0, "clear_subscribers empties the bus"); - check(a_state->freed == 1 && b_state->freed == 1 && c_state->freed == 1, - "free ran exactly once per subscriber"); - // The wrapper A stashed in its state still owns a reference and is - // usable; dropping it releases that reference. - check(a_state->kept_bus.has_value() && a_state->kept_bus->handle() == bus.handle(), - "kept bus still points at the bus"); - check(a_state->kept_bus->subscriber_count() == 0, "kept bus is usable after free"); - a_state->kept_bus.reset(); - check(bus.publish("after", "clear", {}) == 0, "publishing with no subscribers accepts none"); - check(bus.last_message()->seq == 5, "log kept counting"); - - // `bus` releases the last reference when run returns. -} - -int main() { - run(); - check_no_leaks(events_debug_live, "events"); - std::printf("cpp/events: OK\n"); - return 0; -} diff --git a/conformance/cpp/kvstore.cpp b/conformance/cpp/kvstore.cpp index 129a8c16..8838dea2 100644 --- a/conformance/cpp/kvstore.cpp +++ b/conformance/cpp/kvstore.cpp @@ -1,417 +1,602 @@ -// Conformance consumer: kvstore sample, C++ target. +// Conformance consumer: kvstore sample, C++ target (ABI revision 4). // -// Drives the generated header-only wrapper over the object-graph surface: -// - `Store` is a copyable RAII interface wrapper (copies `_clone`, moves -// transfer, destructors release) constructed via the `Store::open` -// static factory, which throws the typed `IoError` on failure. -// - `share()` returns a wrapper to the SAME producer object; `fork()` a new -// one; `larger(std::optional)` exercises `Store?` both ways; -// `describe()` returns a record carrying a `Store` field and an optional -// one; `open_many` returns a vector of objects; `total_count` takes a -// vector of objects plus an optional record holding an object. -// - `EvictionListener` is an abstract class the consumer subclasses; a -// returned `false` detaches (and frees) it, replacing or clearing frees -// the previous one, and an implementation that throws surfaces to the -// caller as `Error` code -4. -// - the pre-existing surface still works: value records with optional, -// list, map, and bytes fields, the `KvError` hierarchy, the lazy -// `list_keys` range, the `kv::stats` nested module, and the -// std::future-backed cancellable `compact`, cancelled through an RAII -// `CancelToken` into `Cancelled`. -// Ends by asserting the producer's leak counters are all zero. -// Exits non-zero on the first failed check. +// Drives the feature-complete producer through the generated header: +// +// * the load-time check (ABI revision and both modules' contract tables; +// a doctored table is refused by name); +// * the `Store` RAII class: fallible and infallible constructors, methods, +// statics, the deprecated `size`, records (`Entry`, `StoreInfo`), the +// `EntryKind` enum, maps and optionals, and the logical clock; +// * the `KvError` hierarchy with payload fields (`KeyNotFoundError`, +// `ExpiredError`, `StoreFullError`, `RejectedError`) and runtime codes on +// throwing calls (the root `Error`); +// * lazy ranges of strings (throwing), records, and objects; +// * three callback interfaces implemented as subclasses: a `Listener` +// (retained, filtered by `accepts`, told about every `Change`, detached +// when it throws, and notified from a producer thread during +// compaction), a `Policy` (a record return, a typed `RejectedError` +// thrown back through `put` with its fields, an object parameter and +// object return, and returns the producer rejects), and a `Loader` passed +// as an optional callback (string, bytes, and optional-object returns; +// typed errors decoded by the producer or passed through); +// * `Store` objects in every position, compared by native identity; +// * futures: an async free function returning an object, a cancellable +// method cancelled mid-pause, an async list, an async function in the +// nested `kv.stats` module, and concurrent calls; +// * the nested `kv.stats` module and the sibling `report` root. +// +// Ends by asserting every callback implementation was released and the +// producer's leak counters are all zero. +#include +#include +#include #include -#include #include #include +#include #include #include #include +#include +#include +#include #include +#include #include #include "check.hpp" #include "kvstore.hpp" using namespace kvstore; +namespace stats = ::kvstore::kv::stats; -static void check(bool ok, const char* what) { - if (!ok) { - std::fprintf(stderr, "cpp/kvstore: FAIL: %s\n", what); - std::exit(1); - } +static const std::thread::id g_main_thread = std::this_thread::get_id(); +static std::atomic g_listeners_freed{0}; +static std::atomic g_policies_freed{0}; +static std::atomic g_loaders_freed{0}; + +static std::vector bytes(std::string_view s) { return std::vector(s.begin(), s.end()); } + +static bool starts_with(std::string_view s, std::string_view prefix) { + return s.substr(0, prefix.size()) == prefix; } -// State outlives the listener so the test can inspect it after the producer -// has freed the implementation. -struct ListenerState { - std::vector> evictions; - std::vector ids; - int freed = 0; -}; +static Entry put(const Store& s, std::string_view key, std::string_view value, + EntryKind kind = EntryKind::Persistent, std::optional ttl = std::nullopt) { + return s.put(key, bytes(value), kind, ttl); +} -class RecordingListener : public EvictionListener { - std::shared_ptr state_; - size_t keep_while_fewer_than_; - std::string bomb_key_; +template +static KeyNotFoundError expect_key_not_found(F&& fn, const std::string& key, const char* what) { + KeyNotFoundError e = expect_throw(std::forward(fn), what); + CHECK(e.code() == 1001 && e.key == key); + return e; +} -public: - RecordingListener(std::shared_ptr state, size_t keep_while_fewer_than, - std::string bomb_key = "") - : state_(std::move(state)), - keep_while_fewer_than_(keep_while_fewer_than), - bomb_key_(std::move(bomb_key)) {} - - ~RecordingListener() override { state_->freed++; } - - bool on_evict(const Entry& entry, EvictionReason reason) override { - if (entry.key == bomb_key_) throw std::runtime_error("listener refused " + entry.key); - state_->evictions.emplace_back(entry.key, reason); - state_->ids.push_back(entry.id); - return state_->evictions.size() < keep_while_fewer_than_; - } -}; +// ── load ──────────────────────────────────────────────────────────────────── -static void run() { +static void load_checks() { check_library(); - check(Store::default_capacity() == 1000000, "static default_capacity"); + // A table entry the library lacks, or one whose signature changed, is + // refused by its path. + const detail::ContractEntry missing[] = {{1, 2, "kv.Store.vanished"}}; + CHECK(detail::contract_mismatch(kvstore_kv_contract, missing) == + "kv.Store.vanished is missing from the library"); + const detail::ContractEntry& first = detail::kv_contract[0]; + const detail::ContractEntry changed[] = {{first.id, first.hash ^ 1, first.path}}; + CHECK(detail::contract_mismatch(kvstore_kv_contract, changed) == + std::string(first.path) + " changed since these bindings were generated"); + CHECK(detail::contract_mismatch(kvstore_kv_contract, detail::kv_contract).empty()); + CHECK(detail::contract_mismatch(kvstore_report_contract, detail::report_contract).empty()); +} - Store store = Store::open("/tmp/conformance-kvstore-cpp"); - check(store.handle() != nullptr, "open yields a live handle"); +// ── constructors ──────────────────────────────────────────────────────────── - // A failing constructor throws the most-derived typed exception. - { - bool caught = false; - try { - Store bad = Store::open(""); - } catch (const IoError& e) { - caught = (e.code() == 1004); - check(dynamic_cast(&e) != nullptr, "IoError is a KvError"); - check(dynamic_cast(&e) != nullptr, "IoError is an Error"); - } - check(caught, "open(\"\") throws IoError 1004"); - } +static void constructors() { + InvalidPathError e = expect_throw([] { Store::open(""); }, "open('')"); + CHECK(e.code() == 1004 && std::string(e.what()) == "invalid path"); + CHECK(dynamic_cast(&e) != nullptr && dynamic_cast(&e) != nullptr); - const std::vector payload{1, 2, 3}; - check(store.put("alpha", payload, EntryKind::Persistent, std::nullopt), "put alpha"); - check(store.put("beta", payload, EntryKind::Volatile, std::nullopt), "put beta"); - check(store.count() == 2, "count is 2"); + Store s; + CHECK(s.path() == "memory"); + CHECK(s.capacity() == Store::default_capacity() && s.capacity() == 1000000); - // Copy semantics: copies share the object; mutations through one are - // visible through the other; destructors release without killing it. - { - Store copy = store; - check(copy.handle() == store.handle(), "copy constructor clones the same object"); - check(copy.count() == 2, "copy is usable"); - check(copy.put("gamma", payload, EntryKind::Volatile, std::nullopt), "put through copy"); - check(store.count() == 3, "mutation through the copy is visible through the original"); - - Store other = Store::open("/tmp/other"); - check(other.handle() != store.handle(), "a second open is a distinct object"); - other = store; - check(other.handle() == store.handle(), "copy assignment clones the same object"); - Store& same = other; - other = same; - check(other.handle() == store.handle(), "self-assignment keeps the handle"); - - Store moved = std::move(copy); - check(moved.handle() == store.handle(), "move constructor transfers the handle"); - check(copy.handle() == nullptr, "moved-from wrapper is empty"); - Store moved_into = Store::open("/tmp/x"); - moved_into = std::move(other); - check(moved_into.handle() == store.handle() && other.handle() == nullptr, - "move assignment transfers the handle"); - check(store.delete_("gamma"), "delete gamma through the original"); - // copy (empty), other (empty), moved, moved_into all destruct here. - } - check(store.count() == 2, "store survives its copies' destructors"); + Store opened = kv::open_store("/async").get(); + CHECK(opened.path() == "/async"); + std::future failing = kv::open_store(""); + e = expect_throw([&] { failing.get(); }, "open_store('')"); + CHECK(e.code() == 1004 && std::string(e.what()) == "invalid path"); +} + +// ── basics ────────────────────────────────────────────────────────────────── + +static void basics() { + Store s = Store::open("/basics"); + Entry e = put(s, "alpha", "one"); + CHECK(e.key == "alpha" && e.value == bytes("one") && e.kind == EntryKind::Persistent); + CHECK(e.version == 1 && !e.expires_at.has_value() && e.tags.empty() && e.metadata.empty()); + e = put(s, "alpha", "two", EntryKind::Volatile); + CHECK(e.version == 2 && e.kind == EntryKind::Volatile); + CHECK(s.get("alpha").value == bytes("two")); + KeyNotFoundError nf = expect_key_not_found([&] { s.get("nope"); }, "nope", "get('nope')"); + CHECK(std::string(nf.what()) == "key not found: nope"); + std::optional found = s.find("alpha"); + CHECK(found.has_value() && found->version == 2); + CHECK(!s.find("nope").has_value()); + + // TTLs follow the logical clock; an expired get reports when. + CHECK(put(s, "ttl", "x", EntryKind::Volatile, 10).expires_at == 10); + CHECK(s.now() == 0); + CHECK(s.tick(9) == 9 && s.count() == 2); + CHECK(s.tick(1) == 10 && s.count() == 1); + ExpiredError ex = expect_throw([&] { s.get("ttl"); }, "get expired"); + CHECK(ex.code() == 1002 && ex.key == "ttl" && ex.expired_at == 10); + expect_key_not_found([&] { s.get("ttl"); }, "ttl", "the expired read removed it"); + + // Capacity: a new key past it is StoreFull { capacity }. + s.set_capacity(1); + CHECK(s.capacity() == 1); + put(s, "alpha", "three"); + StoreFullError full = expect_throw([&] { put(s, "beta", "b"); }, "StoreFull"); + CHECK(full.code() == 1003 && full.capacity == 1); + s.set_capacity(100); + + // An undeclared enum value is a marshalling failure: on a call that + // declares errors it's the root Error, not a KvError. + Error bad = expect_throw( + [&] { s.put("k", bytes("v"), static_cast(9), std::nullopt); }, "bad EntryKind"); + CHECK(bad.code() == -3 && dynamic_cast(&bad) == nullptr); + + put(s, "beta", "b"); + CHECK(s.delete_("beta") && !s.delete_("beta")); +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" + CHECK(s.size() == s.count() && s.count() == 1); +#pragma GCC diagnostic pop + CHECK(s.clear() == 1 && s.count() == 0); +} + +// ── iterators ─────────────────────────────────────────────────────────────── + +static void iterators() { + Store s = Store::open("/iter"); + put(s, "user.bob", "b"); + put(s, "user.alice", "a"); + put(s, "sys.x", "xx"); - // share(): the same object again (refcount bump), not a snapshot. + std::vector keys; + for (const std::string& k : s.keys(std::nullopt)) keys.push_back(k); + CHECK((keys == std::vector{"sys.x", "user.alice", "user.bob"})); + expect_key_not_found([&] { s.keys(std::string("zzz")); }, "zzz", "keys('zzz')"); + + // Abandoning a range part-way releases the producer iterator. { - Store shared = store.share(); - check(shared.handle() == store.handle(), "share returns the same object"); - check(shared.put("via-share", payload, EntryKind::Volatile, std::nullopt), - "put through share"); - check(store.count() == 3, "mutation through share is visible through the original"); - check(store.delete_("via-share"), "delete via original"); - check(shared.count() == 2, "deletion visible through share"); + auto range = s.keys(std::string("user.")); + std::optional next = range.next(); + CHECK(next == "user.alice"); + CHECK(kvstore_debug_live(2) == 1); + } + CHECK(kvstore_debug_live(2) == 0); + + std::vector entries; + for (Entry& e : s.entries(std::string("sys."))) entries.push_back(std::move(e)); + CHECK(entries.size() == 1 && entries[0].key == "sys.x" && entries[0].value == bytes("xx")); + + const std::vector prefixes{"user.", "sys.", "none."}; + std::vector parts; + for (Store& p : s.partition(prefixes)) parts.push_back(std::move(p)); + CHECK(parts.size() == 3); + const uint32_t counts[] = {2, 1, 0}; + for (size_t i = 0; i < parts.size(); ++i) { + CHECK(parts[i].handle() != s.handle()); + for (size_t j = 0; j < i; ++j) CHECK(parts[i].handle() != parts[j].handle()); + CHECK(parts[i].count() == counts[i] && parts[i].path() == prefixes[i]); } +} - // fork(): a distinct object with a copy of the entries. - { - Store forked = store.fork(); - check(forked.handle() != store.handle(), "fork is a distinct object"); - check(forked.count() == 2, "fork copied the live entries"); - check(forked.put("only-in-fork", payload, EntryKind::Volatile, std::nullopt), - "put into fork"); - check(forked.count() == 3 && store.count() == 2, "fork is independent"); +// ── listener ──────────────────────────────────────────────────────────────── + +// Records every change; `accepts` says no to `skip` and throws on `fail_on`. +class RecordingListener : public Listener { + std::string skip_; + std::string fail_on_; + mutable std::mutex mu_; + std::vector changes_; + std::vector threads_; + +public: + explicit RecordingListener(std::string skip = "", std::string fail_on = "") + : skip_(std::move(skip)), fail_on_(std::move(fail_on)) {} + + ~RecordingListener() override { ++g_listeners_freed; } + + bool accepts(std::string_view key) override { + if (!fail_on_.empty() && key == fail_on_) throw std::runtime_error("listener refused"); + return key != skip_; } - // larger(): `Store?` as a parameter and as a return. - { - Store empty = Store::open("/tmp/empty"); - check(!empty.larger(std::nullopt).has_value(), "larger(null) on an empty store is none"); - std::optional own = store.larger(std::nullopt); - check(own.has_value() && own->handle() == store.handle(), - "larger(null) on a non-empty store returns itself"); - std::optional bigger = empty.larger(store); - check(bigger.has_value() && bigger->handle() == store.handle(), - "larger(other) picks the bigger other"); - std::optional self_wins = store.larger(empty); - check(self_wins.has_value() && self_wins->handle() == store.handle(), - "larger(smaller) returns self"); - std::optional none; - check(!empty.larger(none).has_value(), "disengaged optional passes as null"); + void on_change(const Change& change) override { + std::lock_guard lock(mu_); + threads_.push_back(std::this_thread::get_id()); + changes_.push_back(change); } - // describe(): a record carrying the object itself plus an optional one. - { - StoreInfo info = store.describe("primary", std::nullopt); - check(info.label == "primary", "describe label"); - check(info.count == 2, "describe count"); - check(info.store.handle() == store.handle(), "describe().store is the same object"); - check(!info.mirror.has_value(), "describe mirror absent"); - check(info.store.count() == 2, "the record's store is usable"); - - Store mirror = Store::open("/tmp/mirror"); - StoreInfo with_mirror = store.describe("mirrored", mirror); - check(with_mirror.mirror.has_value() && with_mirror.mirror->handle() == mirror.handle(), - "describe mirror present and identical"); - - // A record copy clones its object fields. - StoreInfo info_copy = info; - check(info_copy.store.handle() == store.handle(), "copied record shares the object"); + std::vector changes() const { + std::lock_guard lock(mu_); + return changes_; } - // open_many(): a list of objects as a return; total_count(): a list of - // objects and an optional record holding an object as parameters. - { - std::vector many = Store::open_many({"/a", "/b", "/c"}); - check(many.size() == 3, "open_many returns three stores"); - check(many[0].handle() != many[1].handle() && many[1].handle() != many[2].handle(), - "open_many stores are distinct"); - check(many[0].put("m0", payload, EntryKind::Volatile, std::nullopt), "put into many[0]"); - check(many[0].put("m1", payload, EntryKind::Volatile, std::nullopt), "put into many[0]"); - check(many[2].put("m2", payload, EntryKind::Volatile, std::nullopt), "put into many[2]"); - check(many[0].count() == 2 && many[1].count() == 0 && many[2].count() == 1, - "open_many stores are independent"); - - check(Store::total_count(many, std::nullopt) == 3, "total_count over the list"); - StoreInfo info = store.describe("extra", std::nullopt); - check(Store::total_count(many, info) == 5, "total_count adds the record's store"); - check(Store::total_count({}, info) == 2, "total_count with an empty list"); - check(Store::total_count({}, std::nullopt) == 0, "total_count with nothing"); - // Encoding the parameters cloned each object; every wrapper is still - // valid and holds its own reference. - check(many[0].count() == 2 && info.store.count() == 2, - "wrappers survive being encoded into parameter buffers"); - - bool caught = false; - try { - Store::open_many({"/ok", ""}); - } catch (const IoError& e) { - caught = (e.code() == 1004); - } - check(caught, "open_many with an empty path throws IoError"); + std::vector threads() const { + std::lock_guard lock(mu_); + return threads_; } - // Lazy iterator: sorted keys, one producer `next` per step. - { - std::vector keys; - for (auto&& k : store.list_keys(std::nullopt)) keys.push_back(k); - check(keys.size() == 2 && keys[0] == "alpha" && keys[1] == "beta", - "list_keys yields sorted keys"); - keys.clear(); - for (auto&& k : store.list_keys(std::string("al"))) keys.push_back(k); - check(keys.size() == 1 && keys[0] == "alpha", "list_keys honors the prefix"); - auto range = store.list_keys(std::nullopt); - auto it = range.begin(); - check(*it == "alpha", "early-abandoned range yields the first key"); + Change last() const { + std::lock_guard lock(mu_); + CHECK(!changes_.empty()); + return changes_.back(); } - // Value record with optional, list, map, and bytes fields. - { - std::optional found = store.get("alpha"); - check(found.has_value(), "get alpha present"); - check(found->id > 0 && found->key == "alpha" && found->value == payload, - "entry fields"); - check(found->created_at > 0 && !found->expires_at.has_value(), "entry timestamps"); - check(found->tags.empty() && found->metadata.empty(), "entry collections empty"); - - check(store.put("ttl", payload, EntryKind::Volatile, 3600), "put with ttl"); - std::optional ttl_entry = store.get("ttl"); - check(ttl_entry.has_value() && ttl_entry->expires_at.has_value() && - *ttl_entry->expires_at > ttl_entry->created_at, - "ttl surfaces as a present optional"); - check(store.delete_("ttl"), "delete ttl"); - check(!store.delete_("ttl"), "second delete reports false"); - - // Drive the generated pack/unpack pair directly for the list and map - // fields the sample never accepts as a parameter. - Entry entry{7, "k", payload, 1000, 55, {"hot", "fast"}, {{"source", "test"}, {"env", "prod"}}}; - detail::BufferWriter w; - detail::write_Entry(w, entry); - detail::BufferReader r(w.data(), w.size()); - Entry back = detail::read_Entry(r); - r.expect_end(); - check(back.id == 7 && back.key == "k" && back.value == payload && back.created_at == 1000, - "entry scalar fields round-trip"); - check(back.expires_at.has_value() && *back.expires_at == 55, "entry optional round-trips"); - check(back.tags.size() == 2 && back.tags[0] == "hot" && back.tags[1] == "fast", - "entry list round-trips in order"); - check(back.metadata.size() == 2 && back.metadata.at("source") == "test" && - back.metadata.at("env") == "prod", - "entry map round-trips"); + template + size_t count_of() const { + std::lock_guard lock(mu_); + size_t n = 0; + for (const Change& c : changes_) n += std::holds_alternative(c.value) ? 1 : 0; + return n; } +}; - // Typed errors: the per-code subclass is the most-derived type. - { - bool caught_base = false; - try { - store.get("missing"); - } catch (const KvError& e) { - caught_base = (e.code() == 1001); +static void listeners() { + Store s = Store::open("/listen"); + auto listener = std::make_shared("quiet"); + uint32_t sub = s.subscribe(listener); + CHECK(sub > 0 && s.listener_count() == 1 && listener.use_count() == 2); + + put(s, "a", "1"); + Change last = listener->last(); + const auto* p = std::get_if(&last.value); + CHECK(p != nullptr && p->entry.version == 1 && !p->replaced); + put(s, "a", "2"); + last = listener->last(); + p = std::get_if(&last.value); + CHECK(p != nullptr && p->entry.version == 2 && p->replaced); + put(s, "quiet", "x"); + CHECK(listener->count_of() == 2); + CHECK(s.delete_("a")); + last = listener->last(); + const auto* r = std::get_if(&last.value); + CHECK(r != nullptr && r->key == "a" && !r->expired); + + // An expired read removes the entry and says so. + put(s, "short", "x", EntryKind::Volatile, 1); + s.tick(1); + expect_throw([&] { s.get("short"); }, "get short"); + last = listener->last(); + r = std::get_if(&last.value); + CHECK(r != nullptr && r->key == "short" && r->expired); + CHECK(s.clear() == 1); + last = listener->last(); + CHECK(last.tag() == Change::Tag::Cleared && std::get(last.value).count == 1); + for (std::thread::id t : listener->threads()) CHECK(t == g_main_thread); + + // Unsubscribing releases the producer's reference exactly once. + CHECK(s.unsubscribe(sub) && listener.use_count() == 1); + CHECK(!s.unsubscribe(sub) && s.listener_count() == 0); + int freed = g_listeners_freed; + listener.reset(); + CHECK(g_listeners_freed == freed + 1); + + // A listener that throws is detached (and released); the put succeeds. + auto failing = std::make_shared("", "boom"); + s.subscribe(failing); + put(s, "fine", "1"); + CHECK(failing->count_of() == 1); + put(s, "boom", "1"); + CHECK(s.count() == 2 && s.listener_count() == 0 && failing.use_count() == 1); + + // A null listener is refused before the call. + expect_throw([&] { s.subscribe(nullptr); }, "subscribe(nullptr)"); + + // Destroying the store releases the listeners it still holds. + freed = g_listeners_freed; + s.subscribe(std::make_shared()); + s.subscribe(std::make_shared()); + CHECK(s.listener_count() == 2 && g_listeners_freed == freed); + { Store gone = std::move(s); } + CHECK(g_listeners_freed == freed + 2); +} + +// ── policy ────────────────────────────────────────────────────────────────── + +// Tags and encrypts admitted entries, routes `b/` keys to `other`, and fails +// in various ways for some keys. +class TestPolicy : public Policy { + Store other_; + +public: + std::atomic admitted{0}; + std::atomic saw_version_zero{true}; + + explicit TestPolicy(Store other) : other_(std::move(other)) {} + ~TestPolicy() override { ++g_policies_freed; } + + Entry admit(const Entry& entry) override { + ++admitted; + if (entry.version != 0) saw_version_zero = false; + if (starts_with(entry.key, "secret")) { + throw RejectedError("secrets are not stored", entry.key, "no secrets"); } - check(caught_base, "missing key throws KvError 1001"); - bool caught_typed = false; - try { - store.get("missing"); - } catch (const KeyNotFoundError& e) { - caught_typed = (std::string(e.what()) == "key not found"); + if (starts_with(entry.key, "boom")) throw std::runtime_error("policy exploded"); + Entry out = entry; + out.tags = {"admitted"}; + out.kind = EntryKind::Encrypted; + out.key = "renamed"; // ignored by the store + // An undeclared enum value makes the record malformed (-3). + if (starts_with(entry.key, "garbage")) out.kind = static_cast(9); + return out; + } + + Store route(std::string_view key, Store home) override { + if (starts_with(key, "b/")) return other_; + if (starts_with(key, "null/")) { + // A moved-from wrapper is a null object, which `route` can't + // return (-3). + Store taken = std::move(home); + return home; } - check(caught_typed, "missing key throws KeyNotFoundError with the doc message"); + return home; } +}; - // kv.stats nested module takes the interface by const reference. - { - Stats st = kv::stats::get_stats(store); - check(st.total_entries == 2 && st.total_bytes == 6 && st.expired_entries == 0, - "get_stats snapshot"); +static void policies() { + Store s = Store::open("/policy"); + Store other = Store::open("/other"); + auto policy = std::make_shared(other); + s.set_policy(policy); + CHECK(s.has_policy() && policy.use_count() == 2); + + // admit's record return is what's stored (its key and version aside). + Entry e = put(s, "a", "1", EntryKind::Volatile); + CHECK(e.key == "a" && e.version == 1 && e.kind == EntryKind::Encrypted); + CHECK((e.tags == std::vector{"admitted"})); + + // route: the object parameter and object return redirect a write. + put(s, "b/x", "2", EntryKind::Volatile); + CHECK(s.count() == 1 && other.count() == 1); + + // A typed error thrown by the throwing callback reaches the caller with + // its code, message, and fields. + RejectedError rej = expect_throw([&] { put(s, "secret", "3"); }, "Rejected"); + CHECK(rej.code() == 1005 && rej.key == "secret" && rej.reason == "no secrets"); + CHECK(std::string(rej.what()) == "secrets are not stored"); + + // Any other exception arrives as -4 with its message. + Error boom = expect_throw([&] { put(s, "boom", "4"); }, "non-domain failure"); + CHECK(boom.code() == -4 && std::string(boom.what()) == "policy exploded"); + CHECK(dynamic_cast(&boom) == nullptr); + + // A return the producer can't accept is -3: a malformed record, or a + // null required object. + Error garbage = expect_throw([&] { put(s, "garbage", "5"); }, "malformed admit"); + CHECK(garbage.code() == -3); + Error null_route = expect_throw([&] { put(s, "null/x", "6"); }, "null route"); + CHECK(null_route.code() == -3); + CHECK(s.count() == 1 && other.count() == 1); + CHECK(policy->admitted == 6 && policy->saw_version_zero); + + // Replacing the policy releases the old one; an empty pointer removes it. + s.set_policy(std::make_shared(other)); + CHECK(policy.use_count() == 1); + int freed = g_policies_freed; + s.set_policy(nullptr); + CHECK(g_policies_freed == freed + 1 && !s.has_policy()); + put(s, "secret", "now allowed"); + CHECK(s.count() == 2); +} + +// ── loader ────────────────────────────────────────────────────────────────── + +class TestLoader : public Loader { + std::optional backup_; + +public: + explicit TestLoader(std::optional backup = std::nullopt) : backup_(std::move(backup)) {} + ~TestLoader() override { ++g_loaders_freed; } + + std::string name() override { return "cpp-loader"; } + + std::optional fallback(std::string_view key) override { + return key == "fb" ? backup_ : std::nullopt; } - // Eviction listener: delete and expiry-on-read notify it; returning - // false detaches it (the producer frees it); replacement and clear free. - { - auto state = std::make_shared(); - store.set_eviction_listener(std::make_shared(state, 2)); - check(state->freed == 0, "attached listener is retained"); - - check(store.put("evict-me", payload, EntryKind::Volatile, std::nullopt), "put evict-me"); - std::optional to_evict = store.get("evict-me"); - check(store.delete_("evict-me"), "delete evict-me"); - check(state->evictions.size() == 1 && state->evictions[0].first == "evict-me" && - state->evictions[0].second == EvictionReason::Deleted, - "delete notified the listener with Deleted"); - check(to_evict.has_value() && state->ids[0] == to_evict->id, - "the evicted entry carries its id"); - - check(store.put("expiring", payload, EntryKind::Volatile, -1), "put already expired"); - bool caught = false; - try { - store.get("expiring"); - } catch (const ExpiredError& e) { - caught = (e.code() == 1002); - } - check(caught, "reading an expired entry throws ExpiredError 1002"); - check(state->evictions.size() == 2 && state->evictions[1].first == "expiring" && - state->evictions[1].second == EvictionReason::Expired, - "expiry on read notified the listener with Expired"); - // The second eviction returned false, so the store detached and - // freed the listener; a third eviction is not observed. - check(state->freed == 1, "detached listener was freed"); - check(store.put("again", payload, EntryKind::Volatile, std::nullopt), "put again"); - check(store.delete_("again"), "delete again"); - check(state->evictions.size() == 2, "detached listener is not notified"); - - auto first = std::make_shared(); - auto second = std::make_shared(); - store.set_eviction_listener(std::make_shared(first, 1000)); - check(first->freed == 0, "first listener retained"); - store.set_eviction_listener(std::make_shared(second, 1000)); - check(first->freed == 1, "replaced listener is freed"); - check(second->freed == 0, "replacement is retained"); - store.clear_eviction_listener(); - check(second->freed == 1, "cleared listener is freed"); - store.clear_eviction_listener(); - - bool threw = false; - try { - store.set_eviction_listener(nullptr); - } catch (const std::invalid_argument&) { - threw = true; - } - check(threw, "null listener throws std::invalid_argument"); + std::vector load(std::string_view key) override { + if (key == "missing") throw KeyNotFoundError("not in the loader", "missing"); + if (key == "elsewhere") throw KeyNotFoundError("not in the loader", "other"); + if (key == "broken") throw std::runtime_error("loader is broken"); + return bytes("loaded:" + std::string(key)); } +}; + +static void loaders() { + Store s = Store::open("/load"); + CHECK(!s.get_or_load("k", nullptr).has_value()); + + auto loader = std::make_shared(); + std::optional e = s.get_or_load("k", loader); + CHECK(e.has_value() && e->value == bytes("loaded:k") && e->kind == EntryKind::Volatile); + CHECK(e->metadata.size() == 1 && e->metadata.at("source") == "cpp-loader"); + CHECK(loader.use_count() == 1); // released before the call returned + CHECK(s.count() == 1); + e = s.get_or_load("k", loader); + CHECK(e.has_value() && e->version == 1); - // A listener that throws surfaces to the caller as a foreign error (-4) - // on a `throws` method: the generic Error, not a KvError. { - auto state = std::make_shared(); - store.set_eviction_listener(std::make_shared(state, 1000, "bomb")); - check(store.put("bomb", payload, EntryKind::Volatile, std::nullopt), "put bomb"); - bool caught = false; - try { - store.delete_("bomb"); - } catch (const KvError&) { - check(false, "a foreign error must not be mapped to a domain exception"); - } catch (const Error& e) { - caught = true; - check(e.code() == -4, "listener exception maps to FOREIGN_ERROR_CODE (-4)"); - check(std::string(e.what()).find("listener refused bomb") != std::string::npos, - "foreign error carries the C++ exception message"); - } - check(caught, "delete threw for a throwing listener"); - check(!store.delete_("bomb"), "the entry was removed before the listener ran"); - check(state->freed == 0, "a throwing listener stays attached"); - // Still usable, and a non-bomb key is observed normally. - check(store.put("fine", payload, EntryKind::Volatile, std::nullopt), "put fine"); - check(store.delete_("fine"), "delete fine"); - check(state->evictions.size() == 1 && state->evictions[0].first == "fine", - "listener still attached after the foreign error"); - store.clear_eviction_listener(); - check(state->freed == 1, "listener freed on clear"); + Store backup = Store::open("/backup"); + put(backup, "fb", "from backup"); + e = s.get_or_load("fb", std::make_shared(backup)); + CHECK(e.has_value() && e->value == bytes("from backup") && e->kind == EntryKind::Persistent); } - // Async: an immediately-expired entry gives compact 3 bytes to reclaim. - { - check(store.put("doomed", payload, EntryKind::Volatile, 0), "put doomed"); - std::future pending = store.compact(); - check(pending.get() == 3, "compact reclaimed the expired bytes"); - check(store.count() == 2, "compact left the live entries"); + CHECK(!s.get_or_load("missing", loader).has_value()); + KeyNotFoundError nf = + expect_key_not_found([&] { s.get_or_load("elsewhere", loader); }, "other", "elsewhere"); + CHECK(std::string(nf.what()) == "not in the loader"); + Error broken = expect_throw([&] { s.get_or_load("broken", loader); }, "broken loader"); + CHECK(broken.code() == -4 && std::string(broken.what()) == "loader is broken"); + CHECK(loader.use_count() == 1); + int freed = g_loaders_freed; + loader.reset(); + CHECK(g_loaders_freed == freed + 1); +} - CancelToken token; - check(!token.is_cancelled(), "a new token is not cancelled"); - token.cancel(); - check(token.is_cancelled(), "cancel() marks the token"); - std::future cancelled = store.compact(token); - bool caught = false; - try { - cancelled.get(); - } catch (const Cancelled& e) { - caught = (e.code() == -5); - } - check(caught, "a pre-cancelled compact settles the future with Cancelled"); - } +// ── async ─────────────────────────────────────────────────────────────────── - // Non-throwing method still works, then release everything via RAII. - store.clear(); - check(store.count() == 0, "clear empties the store"); +static void async_calls() { + Store s = Store::open("/async-calls"); + auto listener = std::make_shared(); + s.subscribe(listener); + put(s, "old1", "x", EntryKind::Volatile, 1); + put(s, "old2", "x", EntryKind::Volatile, 1); + put(s, "keep", "x"); + s.tick(5); - // Destroying the last wrapper releases the object; a move-emptied wrapper - // destructs as a no-op. + // compact runs on a producer thread and notifies listeners there. { - Store last = std::move(store); - check(store.handle() == nullptr, "moved-from store is empty"); - check(last.count() == 0, "moved-to store is usable"); + CancelToken token; + CHECK(s.compact(0, token).get() == 2); + } + CHECK(listener->count_of() == 2); + std::vector threads = listener->threads(); + CHECK(threads.size() == 5); // three puts, then the two removals + CHECK(threads[3] != g_main_thread && threads[4] != g_main_thread); + CHECK(s.count() == 1); + + // No token never cancels. + CHECK(s.compact(5).get() == 0); + + // Cancel mid-pause: the call stops at once, and the background pause + // notices the token and stops. + CancelToken token; + std::future pending = s.compact(60000, token); + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + CHECK(pending.wait_for(std::chrono::seconds(0)) != std::future_status::ready); + CHECK(Store::active_jobs() >= 1); + auto started = std::chrono::steady_clock::now(); + token.cancel(); + expect_throw([&] { pending.get(); }, "cancelled compact"); + CHECK(std::chrono::steady_clock::now() - started < std::chrono::seconds(1)); + for (int i = 0; i < 2000 && Store::active_jobs() != 0; ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + CHECK(Store::active_jobs() == 0); + + // get_many: an async list of optional records, launched concurrently. + std::vector>>> many; + for (int i = 0; i < 32; ++i) many.push_back(s.get_many({"keep", "gone", "keep"})); + for (auto& f : many) { + std::vector> got = f.get(); + CHECK(got.size() == 3 && got[0].has_value() && !got[1].has_value()); + CHECK(got[2].has_value() && got[2]->key == "keep"); } + // The nested module's async free function: objects in a list in, a + // record out. + Store other = Store::open("/other"); + put(other, "a", "123"); + Stats st = stats::summarize_all({s, other}).get(); + CHECK(st.entries == 2 && st.bytes == 4); + CHECK(st.by_kind.size() == 1 && st.by_kind.at(EntryKind::Persistent) == 2); +} + +// ── object graph ──────────────────────────────────────────────────────────── + +static void object_graph() { + std::optional original = Store::open("/graph"); + put(*original, "k", "v"); + + // share(): the same object behind a new wrapper; copies share it too. + Store s = original->share(); + CHECK(s.handle() == original->handle()); + Store copy = s; + CHECK(copy.handle() == s.handle()); + original.reset(); + CHECK(s.count() == 1 && copy.count() == 1); + + Store fork = s.fork(); + CHECK(fork.handle() != s.handle() && fork.count() == 1 && fork.path() == "/graph"); + put(fork, "k2", "v"); + CHECK(fork.count() == 2 && s.count() == 1); + + Store empty = Store::open("/empty"); + CHECK(!empty.larger(std::nullopt).has_value()); + std::optional larger = empty.larger(fork); + CHECK(larger.has_value() && larger->handle() == fork.handle()); + larger = s.larger(std::nullopt); + CHECK(larger.has_value() && larger->handle() == s.handle()); + + StoreInfo info = s.describe("main", fork); + CHECK(info.label == "main" && info.store.handle() == s.handle() && info.count == 1); + CHECK(info.mirror.has_value() && info.mirror->handle() == fork.handle()); + CHECK(info.mirror->count() == 2); + + std::vector many = Store::open_many({"/a", "/b"}); + CHECK(many.size() == 2 && many[0].path() == "/a" && many[1].path() == "/b"); + expect_throw([] { Store::open_many({"/a", ""}); }, "open_many with ''"); + + std::unordered_map named = + Store::by_label({info, StoreInfo{"first", many[0], std::nullopt, 0}}); + CHECK(named.size() == 2 && named.at("main").handle() == s.handle()); + CHECK(named.at("first").handle() == many[0].handle()); + + put(many[0], "m", "1"); + CHECK(Store::total_count({many[0], many[1], fork}, named, info) == 6); + CHECK(Store::total_count({many[0], many[1], fork}, named, std::nullopt) == 5); + CHECK(s.count() == 1 && fork.count() == 2 && many[0].count() == 1); +} + +// ── stats and report ──────────────────────────────────────────────────────── + +static void stats_and_report() { + Store s = Store::open("/stats"); + put(s, "b", "12"); + put(s, "a", "1"); + put(s, "a", "123"); + put(s, "c", "x", EntryKind::Encrypted); + + Stats st = stats::summarize(s, std::nullopt); + CHECK(st.entries == 3 && st.bytes == 6 && st.by_kind.size() == 2); + CHECK(st.by_kind.at(EntryKind::Persistent) == 2 && st.by_kind.at(EntryKind::Encrypted) == 1); + expect_key_not_found([&] { stats::summarize(s, std::string("q")); }, "q", "summarize('q')"); + + std::vector entries; + for (Entry& e : s.entries(std::nullopt)) entries.push_back(std::move(e)); + std::vector lines = report::render_report(entries); + CHECK((lines == std::vector{"a: 3 bytes, Persistent, v2", "b: 2 bytes, Persistent", + "c: 1 bytes, Encrypted"})); + NothingToReportError none = + expect_throw([] { report::render_report({}); }, "empty report"); + CHECK(none.code() == 2001 && std::string(none.what()) == "nothing to report"); + CHECK(dynamic_cast(&none) != nullptr); + CHECK(dynamic_cast(&none) == nullptr); } int main() { - run(); + load_checks(); + constructors(); + basics(); + iterators(); + listeners(); + policies(); + loaders(); + async_calls(); + object_graph(); + stats_and_report(); + + CHECK(kvstore_debug_live(1) == 0); check_no_leaks(kvstore_debug_live, "kvstore"); - std::printf("cpp/kvstore: OK\n"); + std::printf("cpp/kvstore: OK (%d listeners, %d policies, %d loaders released)\n", + g_listeners_freed.load(), g_policies_freed.load(), g_loaders_freed.load()); return 0; } diff --git a/conformance/cpp/run.sh b/conformance/cpp/run.sh index 4117cd06..91f7fd33 100755 --- a/conformance/cpp/run.sh +++ b/conformance/cpp/run.sh @@ -12,6 +12,7 @@ # UndefinedBehaviorSanitizer. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools cpp cmake "${CXX:-clang++}" # cpp_lane cpp_lane() { @@ -38,17 +39,11 @@ EOF } cpp_calculator() { cpp_lane calculator calculator; } -cpp_contacts() { cpp_lane contacts contacts; } -cpp_events() { cpp_lane events events; } -cpp_kvstore() { cpp_lane kvstore kvstore; } -cpp_async_demo() { cpp_lane async-demo async_demo; } cpp_codec() { cpp_lane codec codec; } +cpp_kvstore() { cpp_lane kvstore kvstore; } lane cpp-calculator cpp_calculator -lane cpp-contacts cpp_contacts -lane cpp-events cpp_events -lane cpp-kvstore cpp_kvstore -lane cpp-async-demo cpp_async_demo lane cpp-codec cpp_codec +lane cpp-kvstore cpp_kvstore finish_lanes diff --git a/conformance/dart/async_demo.dart b/conformance/dart/async_demo.dart deleted file mode 100644 index ab6a7b19..00000000 --- a/conformance/dart/async_demo.dart +++ /dev/null @@ -1,77 +0,0 @@ -// Conformance consumer: async-demo sample, Dart target. -// -// Futures completed from the producer's worker threads: a record result, the -// typed InvalidNameException, a buffered list both ways, a direct scalar, -// many concurrent calls, and cancellation through CancelToken (cancelling a -// pending call, a token cancelled before launch, one token shared by two -// calls, and a cancel after completion), each cancelled call failing with -// CancelledException (code -5) long before its timeout. - -import 'package:async_demo/async_demo.dart' as tasks; - -import 'support.dart'; - -Future run() async { - final result = await tasks.runTask('alpha'); - expect(result.id > 0, 'runTask id'); - expect(result.value == 'completed: alpha', 'runTask value (${result.value})'); - expect(result.success, 'runTask success'); - - final invalid = await expectThrowsAsync( - () => tasks.runTask(''), 'empty name'); - expect(invalid.code == 1 && invalid is tasks.TaskException, 'InvalidName'); - - final batch = await tasks.runBatch(['a', 'b', 'c']); - expect(batch.map((r) => r.value).join('|') == - 'completed: a|completed: b|completed: c', - 'runBatch values'); - expect((await tasks.runBatch([])).isEmpty, 'runBatch empty'); - - expect(await tasks.runNTasks(7) == 7, 'runNTasks'); - final many = await Future.wait( - >[for (var i = 0; i < 64; i++) tasks.runNTasks(i)]); - expect(many.length == 64 && many[63] == 63, 'concurrent runNTasks'); - - // A timeout that elapses completes normally, with or without a token. - expect(await tasks.wait(20) == 20, 'wait elapses'); - final idle = tasks.CancelToken(); - expect(await tasks.wait(10, cancelToken: idle) == 10, 'uncancelled token'); - idle.cancel(); - expect(idle.isCancelled, 'cancel after completion is harmless'); - - // Cancelling a pending call completes it with CancelledException. - final watch = Stopwatch()..start(); - final token = tasks.CancelToken(); - final pending = outcome(tasks.wait(60000, cancelToken: token)); - await Future.delayed(const Duration(milliseconds: 20)); - token.cancel(); - final cancelled = await pending; - expect(cancelled is tasks.CancelledException, 'cancelled wait (got $cancelled)'); - final code = (cancelled! as tasks.NativeException).code; - expect(code == -5 && code == tasks.NativeException.cancelledCode, - 'cancelled code (got $code)'); - expect(watch.elapsed < const Duration(seconds: 5), 'cancel is prompt'); - - // A token cancelled before launch, and one token shared by two calls. - await expectThrowsAsync( - () => tasks.wait(60000, cancelToken: token), 'pre-cancelled token'); - final shared = tasks.CancelToken(); - final both = Future.wait(>[ - outcome(tasks.wait(60000, cancelToken: shared)), - outcome(tasks.wait(60000, cancelToken: shared)), - ]); - shared.cancel(); - shared.cancel(); - final outcomes = await both; - expect(outcomes.every((o) => o is tasks.CancelledException), - 'one token cancels both calls (got $outcomes)'); - expect(watch.elapsed < const Duration(seconds: 10), 'all cancels prompt'); - - await settle(() => tasks.activeCallbacks() == 0, 'task bodies finish'); -} - -Future main() async { - await run(); - await expectNoLeaks('async_demo'); - print('dart/async-demo: OK'); -} diff --git a/conformance/dart/calculator.dart b/conformance/dart/calculator.dart index b30bd874..3eb9bb4e 100644 --- a/conformance/dart/calculator.dart +++ b/conformance/dart/calculator.dart @@ -1,30 +1,46 @@ // Conformance consumer: calculator sample, Dart target. // -// Direct scalars, the typed CalcException domain, and strings crossing as -// (ptr, len) UTF-8 runs, including interior NUL and non-ASCII text. +// The getting-started surface: a direct-value call (wrapping at the i32 +// edges), a throwing call that raises the typed DivisionByZeroException, and +// a string in and out (non-ASCII and astral text and an interior NUL +// survive). Loading the bindings checks the ABI revision and the +// `calculator` contract. Ends by asserting the producer's leak counters are +// zero. import 'package:calculator/calculator.dart' as calc; import 'support.dart'; +const int i32Max = 0x7fffffff; +const int i32Min = -0x80000000; + void run() { - expect(calc.add(2, 3) == 5, 'add'); - expect(calc.mul(-4, 6) == -24, 'mul'); - expect(calc.div(17, 5) == 3, 'div'); + expect(calc.add(2, 3) == 5, 'add(2, 3)'); + expect(calc.add(-7, 7) == 0, 'add(-7, 7)'); + expect(calc.add(i32Max, 1) == i32Min, 'add wraps'); + + expect(calc.divide(10, 2) == 5, 'divide(10, 2)'); + expect(calc.divide(-7, 2) == -3, 'divide rounds toward zero'); + expect(calc.divide(i32Min, -1) == i32Min, 'divide(MIN, -1)'); final e = expectThrows( - () => calc.div(1, 0), 'div by zero'); + () => calc.divide(1, 0), + 'divide(1, 0) raises DivisionByZero', + ); expect(e.code == 1, 'DivisionByZero code (got ${e.code})'); expect(e.message == 'division by zero', 'message (got ${e.message})'); - expect(e is calc.CalcException && e is calc.NativeException, - 'domain hierarchy'); - - for (final s in ['', 'hello', 'héllo wörld ✓ \u{1F600}', 'a\u0000b']) { - final back = calc.echo(s); - expect(back == s, 'echo round trip (got ${back.codeUnits})'); - } - final nul = calc.echo('nul\u0000inside'); - expect(nul.length == 10 && nul.codeUnitAt(3) == 0, 'interior NUL survives'); + + expect(calc.greet('World') == 'Hello, World!', 'greet(World)'); + expect(calc.greet('') == 'Hello, !', 'greet(empty)'); + expect( + calc.greet('Wörld 🦀') == 'Hello, Wörld 🦀!', + 'greet keeps non-ASCII and astral text', + ); + final nul = calc.greet('a\u0000b'); + expect( + nul == 'Hello, a\u0000b!', + 'greet keeps an interior NUL (got ${nul.codeUnits})', + ); } Future main() async { diff --git a/conformance/dart/codec.dart b/conformance/dart/codec.dart index eb5a3c99..adf94e7e 100644 --- a/conformance/dart/codec.dart +++ b/conformance/dart/codec.dart @@ -1,599 +1,517 @@ // Conformance consumer: codec sample, Dart target. // -// Round-trips every value-buffer wire shape through the producer oracle in -// both directions. `sample*` proves the Dart decoder reads exactly what Rust -// encoded (checked field by field against the canonical fixture and by -// handing the value back to `verify*`); `roundtrip*` proves the Dart encoder -// matches (the producer decodes, re-encodes, and the result is compared -// field by field); Dart-built values with edge cases (empty strings, lists, -// and maps, non-ASCII and NUL-bearing text, i64/u64 extremes, NaN, the -// infinities, negative zero, f32 rounding) cover what the fixture does not; -// `Shape` exercises every rich-enum variant; `Holder` exercises object -// tokens inside records, optionals, and lists (each encoding mints a cloned -// reference, `primaryOf` returns the same object, wrappers stay usable and -// dispose cleanly, double dispose is safe). Dart ints are signed 64-bit, so -// the generator carries `u64` as its two's-complement bit pattern: u64::MAX -// is -1. Throws (non-zero exit) on any mismatch. - +// The shared-vector loop: for every vector the producer serves, decode it +// with the generated codec, pass it back to `checkVector` (which re-encodes +// it), and push each primitive vector's value through the matching +// direct-family `echo*` (u64 as its two's-complement bit pattern, floats +// compared bitwise). Then vectors built from literals (so a symmetric +// encode/decode bug can't hide), spot checks of decoded fields, the typed +// OutOfRangeException and its payload, malformed input rejected as +// marshalling failures (sent through raw `dart:ffi`, since the bindings only +// encode well-formed values), and object identity and reference counting +// through buffers. Ends by asserting the producer's leak counters are zero. + +import 'dart:convert'; +import 'dart:ffi'; +import 'dart:io'; import 'dart:typed_data'; -import 'package:codec/codec.dart' as wv; +import 'package:codec/codec.dart' as c; +import 'package:ffi/ffi.dart'; import 'support.dart'; -const int i64Min = -9223372036854775808; -const int i64Max = 9223372036854775807; -// u64::MAX and 2^63 as Dart's signed 64-bit bit patterns. -const int u64Max = -1; -const int u64High = i64Min; - -/// The value an f64 takes after passing through an f32 slot. -double f32(double v) => (Float32List(1)..[0] = v)[0]; +const int i8Min = -0x80; +const int i16Min = -0x8000; +const int i32Min = -0x80000000; +const int i64Min = -0x8000000000000000; +const int i64Max = 0x7fffffffffffffff; + +/// Release the tokens a decoded vector carries (only `Objects` has any). +void release(c.Vector v) { + if (v is c.VectorObjects) { + final h = v.value; + h.primary.dispose(); + h.spare?.dispose(); + for (final t in h.many) { + t.dispose(); + } + for (final t in h.byName.values) { + t.dispose(); + } + } +} -bool sameDouble(double a, double b) { - if (a.isNaN || b.isNaN) return a.isNaN && b.isNaN; - return a == b && a.isNegative == b.isNegative; +/// The index of the vector named [name]. +int find(int n, String name) { + for (var i = 0; i < n; i++) { + if (c.vectorName(i) == name) return i; + } + throw StateError('no vector named $name'); } -bool listEq(List a, List b, [bool Function(T, T)? eq]) { +int f32Bits(double v) => + (ByteData(4)..setFloat32(0, v, Endian.little)).getUint32(0, Endian.little); + +int f64Bits(double v) => + (ByteData(8)..setFloat64(0, v, Endian.little)).getUint64(0, Endian.little); + +bool bytesEqual(List a, List b) { if (a.length != b.length) return false; for (var i = 0; i < a.length; i++) { - if (!(eq?.call(a[i], b[i]) ?? a[i] == b[i])) return false; + if (a[i] != b[i]) return false; } return true; } -bool scalarsEq(wv.Scalars a, wv.Scalars b) => - a.i8Value == b.i8Value && - a.u8Value == b.u8Value && - a.i16Value == b.i16Value && - a.u16Value == b.u16Value && - a.i32Value == b.i32Value && - a.u32Value == b.u32Value && - a.i64Value == b.i64Value && - a.u64Value == b.u64Value && - sameDouble(a.f32Value, b.f32Value) && - sameDouble(a.f64Value, b.f64Value) && - a.flag == b.flag && - a.color == b.color; - -bool shapeEq(wv.Shape a, wv.Shape b) { - switch ((a, b)) { - case (wv.ShapeEmpty(), wv.ShapeEmpty()): - return true; - case (wv.ShapeCircle x, wv.ShapeCircle y): - return sameDouble(x.radius, y.radius); - case (wv.ShapeRect x, wv.ShapeRect y): - return sameDouble(x.width, y.width) && sameDouble(x.height, y.height); - case (wv.ShapeLabeled x, wv.ShapeLabeled y): - return x.label == y.label && x.count == y.count; - case (wv.ShapeNested x, wv.ShapeNested y): - return scalarsEq(x.inner, y.inner) && x.note == y.note; +/// Push a primitive vector's value through its direct-family echo. +void echo(c.Vector v) { + switch (v) { + case c.VectorI8(:final value): + expect(c.echoI8(value) == value, 'echoI8 $value'); + case c.VectorU8(:final value): + expect(c.echoU8(value) == value, 'echoU8 $value'); + case c.VectorI16(:final value): + expect(c.echoI16(value) == value, 'echoI16 $value'); + case c.VectorU16(:final value): + expect(c.echoU16(value) == value, 'echoU16 $value'); + case c.VectorI32(:final value): + expect(c.echoI32(value) == value, 'echoI32 $value'); + case c.VectorU32(:final value): + expect(c.echoU32(value) == value, 'echoU32 $value'); + case c.VectorI64(:final value): + expect(c.echoI64(value) == value, 'echoI64 $value'); + case c.VectorU64(:final value): + expect(c.echoU64(value) == value, 'echoU64 $value'); + case c.VectorF32(:final value): + expect(f32Bits(c.echoF32(value)) == f32Bits(value), 'echoF32 $value'); + case c.VectorF64(:final value): + expect(f64Bits(c.echoF64(value)) == f64Bits(value), 'echoF64 $value'); + case c.VectorFlag(:final value): + expect(c.echoBool(value) == value, 'echoBool $value'); + case c.VectorHue(:final value): + expect(c.echoColor(value) == value, 'echoColor $value'); + case c.VectorText(:final value): + expect(c.echoText(value) == value, 'echoText'); + case c.VectorBlob(:final value): + expect(bytesEqual(c.echoBlob(value), value), 'echoBlob'); default: - return false; + break; } } -bool mapEq(Map a, Map b, [bool Function(V, V)? eq]) { - if (a.length != b.length) return false; - for (final e in a.entries) { - if (!b.containsKey(e.key)) return false; - final other = b[e.key] as V; - if (!(eq?.call(e.value, other) ?? e.value == other)) return false; +void everyVector(int n) { + for (var i = 0; i < n; i++) { + final v = c.vector(i); + if (!c.checkVector(i, v)) { + throw StateError( + 'vector $i (${c.vectorName(i)}) did not round-trip; ' + 'producer saw ${c.describeVector(v)}', + ); + } + expect( + !c.checkVector((i + 1) % n, v), + 'vector $i never matches its neighbor', + ); + echo(v); + release(v); } - return true; } -bool compositeEq(wv.Composite a, wv.Composite b) => - a.name == b.name && - listEq(a.blob, b.blob) && - a.someI64 == b.someI64 && - a.noneI64 == b.noneI64 && - a.someText == b.someText && - listEq(a.names, b.names) && - listEq(a.matrix, b.matrix, (x, y) => listEq(x, y)) && - listEq(a.empty, b.empty, sameDouble) && - mapEq(a.byName, b.byName) && - mapEq(a.byId, b.byId, scalarsEq) && - scalarsEq(a.scalars, b.scalars) && - shapeEq(a.shape, b.shape) && - listEq(a.shapes, b.shapes, shapeEq) && - ((a.maybeShape == null && b.maybeShape == null) || - (a.maybeShape != null && - b.maybeShape != null && - shapeEq(a.maybeShape!, b.maybeShape!))) && - ((a.maybeList == null && b.maybeList == null) || - (a.maybeList != null && - b.maybeList != null && - listEq(a.maybeList!, b.maybeList!))) && - listEq(a.sparse, b.sparse) && - listEq(a.colors, b.colors); - -wv.Scalars canonicalScalars({bool flag = true}) => wv.Scalars( - i8Value: -8, - u8Value: 200, - i16Value: -16000, - u16Value: 60000, - i32Value: -2000000000, - u32Value: 4000000000, - i64Value: -9007199254740993, - u64Value: u64Max, - f32Value: 1.5, - f64Value: -2.25e100, - flag: flag, - color: wv.Color.blue, - ); +c.Scalars canonicalScalars({int u16 = 60000}) => c.Scalars( + i8Value: -8, + u8Value: 200, + i16Value: -16000, + u16Value: u16, + i32Value: -2000000000, + u32Value: 4000000000, + i64Value: -9007199254740993, + u64Value: -1, // u64::MAX + f32Value: 1.5, + f64Value: -2.25e100, + flag: true, + color: c.Color.blue, +); + +void literalVectors(int n) { + final canonical = canonicalScalars(); + expect( + c.checkVector(find(n, 'scalars canonical'), c.VectorAllScalars(canonical)), + 'scalars canonical', + ); + expect(canonical == canonicalScalars(), 'records compare by value'); + expect( + !c.checkVector( + find(n, 'scalars canonical'), + c.VectorAllScalars(canonicalScalars(u16: 60001)), + ), + 'a changed field no longer matches', + ); + expect( + c.checkVector( + find(n, 'shape labeled'), + c.VectorFigure(c.ShapeLabeled('tag', 3)), + ), + 'shape labeled', + ); + expect( + c.checkVector( + find(n, 'string interior nul'), + c.VectorText('nul\u0000inside\u0000'), + ), + 'string interior nul', + ); + // Any NaN matches the NaN vector; zero keeps its sign. + final nan = (ByteData(8)..setUint64(0, 0x7ff8000000000001)).getFloat64(0); + expect( + c.checkVector(find(n, 'f64 nan'), c.VectorF64(nan)), + 'a NaN payload matches f64 nan', + ); + expect( + c.checkVector(find(n, 'f64 -0'), c.VectorF64(-0.0)), + '-0.0 matches f64 -0', + ); + expect( + !c.checkVector(find(n, 'f64 -0'), c.VectorF64(0.0)), + "+0.0 doesn't match f64 -0", + ); + expect(c.checkVector(find(n, 'u64 max'), c.VectorU64(-1)), 'u64 max'); + expect( + c.checkVector(find(n, 'enum infrared'), c.VectorHue(c.Color.infrared)), + 'enum infrared', + ); + expect( + c.checkVector(find(n, 'optional zero'), c.VectorMaybeI64(0)), + 'optional zero', + ); + expect( + c.checkVector(find(n, 'optional absent'), c.VectorMaybeI64(null)), + 'optional absent', + ); + expect( + !c.checkVector(find(n, 'optional zero'), c.VectorMaybeI64(null)), + "absent isn't zero", + ); + // A map's entry order doesn't matter on the wire. + final counts = {'x': 0, 'héllo': -1, '': i64Max}; + expect( + c.checkVector(find(n, 'map of strings'), c.VectorCounts(counts)), + 'map of strings', + ); + expect(c.checkVector(find(n, 'blank'), const c.VectorBlank()), 'blank'); + // A holder of a consumer-made token. + final lone = c.Token(-1); + final sparse = c.VectorObjects( + c.Holder(primary: lone, many: const [], byName: const {}), + ); + expect(c.checkVector(find(n, 'objects sparse'), sparse), 'objects sparse'); + lone.dispose(); +} -wv.Composite canonicalComposite() => wv.Composite( - name: 'héllo wörld ✓', - blob: [0, 1, 2, 253, 254, 255], - someI64: i64Min, - noneI64: null, - someText: '', - names: ['a', '', 'ccc'], - matrix: >[ - [1, 2, 3], - [], - [-4] - ], - empty: [], - byName: {'one': 1, 'two': 2, 'neg': -3}, - byId: { - -1: canonicalScalars(), - 42: canonicalScalars(flag: false), - }, - scalars: canonicalScalars(), - shape: wv.ShapeLabeled('tag', 3), - shapes: [ - wv.ShapeEmpty(), - wv.ShapeCircle(2.5), - wv.ShapeRect(1.0, 0.5), - wv.ShapeLabeled('', -1), - wv.ShapeNested(canonicalScalars(), 'n'), - ], - maybeShape: wv.ShapeNested(canonicalScalars(), null), - maybeList: [9, 8], - sparse: [true, null, false], - colors: [wv.Color.red, wv.Color.green, wv.Color.blue], - ); +void spotChecks(int n) { + final past53 = c.vector(find(n, 'i64 past 2^53')); + expect( + past53 == c.VectorI64(-9007199254740993), + 'i64 past 2^53 is exact (got $past53)', + ); -void checkScalars() { - // Producer encodes, Dart decodes: every field of the canonical fixture. - final s = wv.sampleScalars(); - expect(s.i8Value == -8, 'i8 (got ${s.i8Value})'); - expect(s.u8Value == 200, 'u8 (got ${s.u8Value})'); - expect(s.i16Value == -16000, 'i16 (got ${s.i16Value})'); - expect(s.u16Value == 60000, 'u16 (got ${s.u16Value})'); - expect(s.i32Value == -2000000000, 'i32 (got ${s.i32Value})'); - expect(s.u32Value == 4000000000, 'u32 (got ${s.u32Value})'); - expect(s.i64Value == -9007199254740993, 'i64 (got ${s.i64Value})'); - expect(s.u64Value == u64Max, 'u64::MAX arrives as -1 (got ${s.u64Value})'); - expect(s.f32Value == 1.5, 'f32 (got ${s.f32Value})'); - expect(s.f64Value == -2.25e100, 'f64 (got ${s.f64Value})'); - expect(s.flag, 'flag'); - expect(s.color == wv.Color.blue, 'color blue'); - expect(scalarsEq(s, canonicalScalars()), 'sample equals the Dart fixture'); - - // Dart encodes, producer decodes. - expect(wv.verifyScalars(s), 'verifyScalars(sample)'); - expect(wv.verifyScalars(canonicalScalars()), 'verifyScalars(Dart-built)'); - expect(scalarsEq(wv.roundtripScalars(s), s), 'roundtripScalars(sample)'); - - // A mismatch is the typed domain error. - final tweaked = wv.Scalars( - i8Value: s.i8Value, - u8Value: s.u8Value, - i16Value: s.i16Value, - u16Value: s.u16Value, - i32Value: s.i32Value, - u32Value: s.u32Value, - i64Value: s.i64Value, - u64Value: s.u64Value, - f32Value: s.f32Value, - f64Value: s.f64Value, - flag: false, - color: s.color, - ); - try { - wv.verifyScalars(tweaked); - throw StateError('expected MismatchException'); - } on wv.MismatchException catch (e) { - expect(e.code == 1, 'Mismatch code == 1 (got ${e.code})'); - expect(e.message == 'value does not match the canonical fixture', - 'Mismatch message (got ${e.message})'); - expect(e is wv.CodecException && e is wv.NativeException, - 'Mismatch extends the domain and brand exceptions'); - } + final subnormal = c.vector(find(n, 'f32 min subnormal')); + expect( + subnormal is c.VectorF32 && f32Bits(subnormal.value) == 1, + 'f32 min subnormal has bits 1', + ); + + expect( + c.vector(find(n, 'string astral')) == c.VectorText('🦀 crab 😀'), + 'string astral', + ); - // Edge values in every slot: signed and unsigned extremes, u64 above 2^63, - // negative zero in f32, NaN in f64. - final edges = wv.Scalars( - i8Value: -128, - u8Value: 255, - i16Value: -32768, - u16Value: 65535, - i32Value: -2147483648, - u32Value: 4294967295, - i64Value: i64Min, - u64Value: u64High, - f32Value: -0.0, - f64Value: double.nan, - flag: false, - color: wv.Color.red, - ); - final back = wv.roundtripScalars(edges); - expect(scalarsEq(back, edges), 'roundtripScalars(edges)'); - expect(back.f32Value == 0 && back.f32Value.isNegative, 'f32 -0.0 kept'); - expect(back.f64Value.isNaN, 'f64 NaN kept'); - expect(back.u64Value == u64High, 'u64 2^63 bit pattern kept'); - final highs = wv.Scalars( - i8Value: 127, - u8Value: 0, - i16Value: 32767, - u16Value: 0, - i32Value: 2147483647, - u32Value: 0, - i64Value: i64Max, - u64Value: 0, - f32Value: f32(0.1), - f64Value: double.negativeInfinity, - flag: true, - color: wv.Color.green, - ); - final back2 = wv.roundtripScalars(highs); - expect(scalarsEq(back2, highs), 'roundtripScalars(highs)'); - expect(back2.f32Value == f32(0.1), 'f32 0.1 rounds once'); - expect(back2.f64Value == double.negativeInfinity, 'f64 -inf kept'); + final minimum = c.vector(find(n, 'scalars minimum')); + expect(minimum is c.VectorAllScalars, 'scalars minimum is AllScalars'); + final m = (minimum as c.VectorAllScalars).value; + expect(m.i8Value == i8Min && m.i16Value == i16Min, 'scalars minimum i8/i16'); + expect( + m.i32Value == i32Min && m.i64Value == i64Min, + 'scalars minimum i32/i64', + ); + expect(m.u8Value == 0 && m.u64Value == 0, 'scalars minimum unsigned'); + expect( + m.f32Value == double.negativeInfinity && m.f64Value.isNaN, + 'scalars minimum floats', + ); + expect( + m.color == c.Color.infrared && !m.flag, + 'scalars minimum color and flag', + ); + + final deep = c.vector(find(n, 'composite canonical')); + expect(deep is c.VectorDeep, 'composite canonical is Deep'); + final d = (deep as c.VectorDeep).value; + expect(d.name == 'héllo wörld ✓', 'composite name (got ${d.name})'); + expect(d.blob.length == 6 && d.blob[5] == 255, 'composite blob'); + expect(d.someI64 == i64Min && d.noneI64 == null, 'composite optionals'); + expect(d.someText == '', 'composite someText is present and empty'); + expect(d.names.length == 3 && d.names[1] == '', 'composite names'); + expect( + d.matrix.length == 3 && d.matrix[1].isEmpty && d.matrix[2][0] == -4, + 'composite matrix', + ); + expect( + d.floats.length == 6 && d.floats[0].isNaN && d.floats[3].isNegative, + 'composite floats', + ); + expect( + d.byName.length == 4 && + d.byId.length == 3 && + d.byColor.length == 2 && + d.flags.length == 2, + 'composite maps', + ); + expect(d.scalars.u32Value == 4000000000, 'composite scalars.u32Value'); + final shape = d.shape; + expect(shape is c.ShapeLabeled && shape.count == 3, 'composite shape'); + final last = d.shapes.last; + expect( + d.shapes.length == 6 && last is c.ShapeNested && last.note == null, + 'composite shapes', + ); + expect(d.maybeShape is c.ShapeNested, 'composite maybeShape'); + expect(d.maybeList?.length == 2, 'composite maybeList'); + expect( + d.sparse.length == 3 && d.sparse[0] == true && d.sparse[1] == null, + 'composite sparse', + ); + expect( + d.colors.length == 4 && d.colors[3] == c.Color.infrared, + 'composite colors', + ); + // Decoding the same vector twice gives equal (but distinct) values. + final again = c.vector(find(n, 'composite canonical')); + expect( + !identical(again, deep) && again.hashCode == deep.hashCode, + 'composites hash by value', + ); } -void checkComposite() { - final c = wv.sampleComposite(); - expect(c.name == 'héllo wörld ✓', 'name (got ${c.name})'); - expect(listEq(c.blob, [0, 1, 2, 253, 254, 255]), 'blob (got ${c.blob})'); - expect(c.someI64 == i64Min, 'someI64 == i64::MIN (got ${c.someI64})'); - expect(c.noneI64 == null, 'noneI64 absent'); - expect(c.someText == '', 'someText present and empty (got ${c.someText})'); - expect(listEq(c.names, ['a', '', 'ccc']), 'names (got ${c.names})'); - expect(c.matrix.length == 3 && - listEq(c.matrix[0], [1, 2, 3]) && - c.matrix[1].isEmpty && - listEq(c.matrix[2], [-4]), - 'matrix (got ${c.matrix})'); - expect(c.empty.isEmpty, 'empty list'); - expect(c.byName.length == 3 && - c.byName['one'] == 1 && - c.byName['two'] == 2 && - c.byName['neg'] == -3, - 'byName (got ${c.byName})'); - expect(c.byId.length == 2, 'byId size'); - expect(scalarsEq(c.byId[-1]!, canonicalScalars()), 'byId[-1]'); - expect(scalarsEq(c.byId[42]!, canonicalScalars(flag: false)), 'byId[42]'); - expect(scalarsEq(c.scalars, canonicalScalars()), 'nested scalars'); - expect(c.shape is wv.ShapeLabeled, 'shape variant'); - final labeled = c.shape as wv.ShapeLabeled; - expect(labeled.label == 'tag' && labeled.count == 3, 'shape fields'); - expect(c.shapes.length == 5, 'shapes length'); - expect(c.shapes[0] is wv.ShapeEmpty, 'shapes[0] Empty'); - expect((c.shapes[1] as wv.ShapeCircle).radius == 2.5, 'shapes[1] Circle'); - final rect = c.shapes[2] as wv.ShapeRect; - expect(rect.width == 1.0 && rect.height == 0.5, 'shapes[2] Rect'); - final lab = c.shapes[3] as wv.ShapeLabeled; - expect(lab.label == '' && lab.count == -1, 'shapes[3] Labeled'); - final nested = c.shapes[4] as wv.ShapeNested; - expect(scalarsEq(nested.inner, canonicalScalars()) && nested.note == 'n', - 'shapes[4] Nested'); - final maybe = c.maybeShape; - expect(maybe is wv.ShapeNested && maybe.note == null, - 'maybeShape Nested with absent note'); - expect(listEq(c.maybeList!, [9, 8]), 'maybeList (got ${c.maybeList})'); - expect(c.sparse.length == 3 && - c.sparse[0] == true && - c.sparse[1] == null && - c.sparse[2] == false, - 'sparse (got ${c.sparse})'); - expect(listEq(c.colors, [wv.Color.red, wv.Color.green, wv.Color.blue]), - 'colors (got ${c.colors})'); - expect(compositeEq(c, canonicalComposite()), 'sample equals the Dart fixture'); - - // Dart encodes, producer decodes and compares. - expect(wv.verifyComposite(c), 'verifyComposite(sample)'); - expect(wv.verifyComposite(canonicalComposite()), - 'verifyComposite(Dart-built)'); - expect(compositeEq(wv.roundtripComposite(c), c), 'roundtripComposite'); - final text = wv.describeComposite(c); - expect(text.contains('héllo wörld ✓') && text.contains('Labeled'), - 'describeComposite renders the value (got $text)'); - - // A one-element change deep inside is a Mismatch. - final changed = wv.Composite( - name: c.name, - blob: c.blob, - someI64: c.someI64, - noneI64: c.noneI64, - someText: c.someText, - names: c.names, - matrix: c.matrix, - empty: c.empty, - byName: c.byName, - byId: c.byId, - scalars: c.scalars, - shape: c.shape, - shapes: c.shapes, - maybeShape: c.maybeShape, - maybeList: c.maybeList, - sparse: [true, true, false], - colors: c.colors, - ); - try { - wv.verifyComposite(changed); - throw StateError('expected MismatchException for changed composite'); - } on wv.MismatchException catch (e) { - expect(e.code == 1, 'composite Mismatch code'); - } +void outOfRange(int n) { + final e = expectThrows( + () => c.vector(n), + 'vector(n) raises OutOfRange', + ); + expect( + e.code == 1 && e.index == n && e.count == n, + 'OutOfRange payload (index ${e.index}, count ${e.count})', + ); + expect( + e.message == 'vector $n is out of range (count $n)', + 'OutOfRange message (got ${e.message})', + ); - // A Dart-built composite full of edge cases: empty everything, NUL and - // astral-plane text, special floats, 32/64-bit extremes, absent optionals. - final edge = wv.Composite( - name: 'nul\u0000inside \u{1F600} end', - blob: [], - someI64: null, - noneI64: i64Max, - someText: null, - names: [], - matrix: >[[], [-2147483648, 2147483647]], - empty: [ - double.nan, - double.infinity, - double.negativeInfinity, - -0.0, - 5e-324, - 1.7976931348623157e308, - ], - byName: {'': i64Min, 'max': i64Max, 'z': 0}, - byId: { - 2147483647: canonicalScalars(flag: false), - -2147483648: canonicalScalars(), - 0: canonicalScalars(), - }, - scalars: canonicalScalars(), - shape: wv.ShapeEmpty(), - shapes: [], - maybeShape: null, - maybeList: null, - sparse: [null, null], - colors: [], - ); - final edgeBack = wv.roundtripComposite(edge); - expect(compositeEq(edgeBack, edge), 'roundtripComposite(edge)'); - expect(edgeBack.name.length == edge.name.length && - edgeBack.name.codeUnitAt(3) == 0, - 'NUL survives inside a length-prefixed string'); - expect(edgeBack.empty[0].isNaN, 'NaN in a list'); - expect(edgeBack.empty[3] == 0 && edgeBack.empty[3].isNegative, - 'negative zero in a list'); - expect(edgeBack.byName[''] == i64Min, 'empty-string key with i64::MIN'); - expect( - listEq(edgeBack.byId.keys.toList()..sort(), - [-2147483648, 0, 2147483647]), - 'i32 map keys (got ${edgeBack.byId.keys})'); - expect(edgeBack.maybeShape == null && edgeBack.maybeList == null, - 'absent optionals stay absent'); - expect(edgeBack.someI64 == null && edgeBack.noneI64 == i64Max, - 'optional i64 both ways'); - - // Fully empty composite. - final bare = wv.Composite( - name: '', - blob: [], - names: [], - matrix: >[], - empty: [], - byName: {}, - byId: {}, - scalars: canonicalScalars(), - shape: wv.ShapeEmpty(), - shapes: [], - sparse: [], - colors: [], - ); - expect(compositeEq(wv.roundtripComposite(bare), bare), - 'roundtripComposite(bare)'); + final e2 = expectThrows( + () => c.vectorName(n + 5), + 'vectorName(n + 5) raises OutOfRange', + ); + expect(e2.index == n + 5 && e2.count == n, 'vectorName OutOfRange payload'); + + expect( + !c.checkVector(n, const c.VectorBlank()), + 'checkVector past the end is false', + ); +} + +/// The C `codec_error` struct, for raw calls. +final class RawError extends Struct { + @Int32() + external int code; + external Pointer message; + external Pointer payloadPtr; + @Size() + external int payloadLen; } -void checkShapes() { - final variants = [ - wv.ShapeEmpty(), - wv.ShapeCircle(2.5), - wv.ShapeCircle(double.nan), - wv.ShapeCircle(-0.0), - wv.ShapeRect(1.0, 0.5), - wv.ShapeRect(f32(0.1), double.infinity), - wv.ShapeLabeled('tag', 3), - wv.ShapeLabeled('', -2147483648), - wv.ShapeLabeled('ünïcödé ✓', 2147483647), - wv.ShapeNested(canonicalScalars(), 'note'), - wv.ShapeNested(canonicalScalars(flag: false), null), - ]; - for (final v in variants) { - final back = wv.roundtripShape(v); - expect(back.runtimeType == v.runtimeType, - 'roundtripShape variant ${v.runtimeType} (got ${back.runtimeType})'); - expect(shapeEq(back, v), 'roundtripShape fields for ${v.runtimeType}'); +/// The producer's symbols, bound directly so malformed input can be sent. +final class Raw { + Raw() : _lib = DynamicLibrary.open(Platform.environment['CODEC_LIBRARY']!); + + final DynamicLibrary _lib; + + late final checkVector = _lib + .lookupFunction< + Bool Function(Uint32, Pointer, Size, Pointer), + bool Function(int, Pointer, int, Pointer) + >('codec_codec_check_vector'); + late final echoColor = _lib + .lookupFunction< + Int32 Function(Int32, Pointer), + int Function(int, Pointer) + >('codec_codec_echo_color'); + late final echoText = _lib + .lookupFunction< + Pointer Function( + Pointer, + Size, + Pointer, + Pointer, + ), + Pointer Function( + Pointer, + int, + Pointer, + Pointer, + ) + >('codec_codec_echo_text'); + late final sumHolder = _lib + .lookupFunction< + Int64 Function(Pointer, Size, Pointer), + int Function(Pointer, int, Pointer) + >('codec_codec_sum_holder'); + late final errorClear = _lib + .lookupFunction< + Void Function(Pointer), + void Function(Pointer) + >('codec_error_clear'); + + /// Runs [call] with a staged copy of [bytes] and a fresh error slot, and + /// returns the error code it reported. + int code( + List bytes, + void Function(Pointer, int, Pointer) call, + ) { + final ptr = calloc(bytes.length + 1); + final err = calloc(); + try { + ptr.asTypedList(bytes.length).setAll(0, bytes); + call(ptr, bytes.length, err); + final code = err.ref.code; + errorClear(err); + return code; + } finally { + calloc.free(ptr); + calloc.free(err); + } } - final all = wv.roundtripShapes(variants); - expect(listEq(all, variants, shapeEq), 'roundtripShapes list'); - expect(wv.roundtripShapes([]).isEmpty, 'roundtripShapes empty'); - - // describeShape renders the Rust Debug form, so the tag and payload the - // producer actually decoded are visible. - expect(wv.describeShape(wv.ShapeEmpty()) == 'Empty', - 'describe Empty (got ${wv.describeShape(wv.ShapeEmpty())})'); - expect(wv.describeShape(wv.ShapeCircle(2.5)) == 'Circle { radius: 2.5 }', - 'describe Circle (got ${wv.describeShape(wv.ShapeCircle(2.5))})'); - expect( - wv.describeShape(wv.ShapeRect(1.0, 0.5)) == - 'Rect { width: 1.0, height: 0.5 }', - 'describe Rect (got ${wv.describeShape(wv.ShapeRect(1.0, 0.5))})'); - expect( - wv.describeShape(wv.ShapeLabeled('tag', 3)) == - 'Labeled { label: "tag", count: 3 }', - 'describe Labeled (got ${wv.describeShape(wv.ShapeLabeled('tag', 3))})'); - final nestedText = wv.describeShape(wv.ShapeNested(canonicalScalars(), null)); - expect( - nestedText.startsWith('Nested { inner: Scalars {') && - nestedText.contains('u64_value: 18446744073709551615') && - nestedText.endsWith('note: None }'), - 'describe Nested shows u64::MAX from the -1 bit pattern (got $nestedText)'); } -void checkDirect() { - // Optionals, maps, strings, and bytes as top-level buffered or pointer - // values. - expect(wv.roundtripOptI64(null) == null, 'opt i64 null'); - expect(wv.roundtripOptI64(0) == 0, 'opt i64 zero'); - expect(wv.roundtripOptI64(i64Min) == i64Min, 'opt i64 min'); - expect(wv.roundtripOptI64(i64Max) == i64Max, 'opt i64 max'); - expect(wv.roundtripMap({}).isEmpty, 'empty map'); - final m = wv.roundtripMap({'': i64Min, 'k': -1, 'ü': i64Max}); - expect(m.length == 3 && m[''] == i64Min && m['k'] == -1 && m['ü'] == i64Max, - 'map values (got $m)'); - expect(wv.roundtripString('') == '', 'empty string'); - final nul = wv.roundtripString('a\u0000b\u0000'); - expect(nul.length == 4 && nul.codeUnitAt(1) == 0 && nul.codeUnitAt(3) == 0, - 'interior and trailing NUL survive a (ptr, len) string'); - expect(wv.roundtripString('héllo wörld ✓ \u{1F600}') == - 'héllo wörld ✓ \u{1F600}', - 'unicode string'); - expect(wv.roundtripBytes([]).isEmpty, 'empty bytes'); - expect(listEq(wv.roundtripBytes([0, 127, 128, 255]), - [0, 127, 128, 255]), - 'bytes'); - - // 64-bit extremes by value; u64 above 2^63 travels as its bit pattern. - expect(wv.roundtripI64(i64Min) == i64Min, 'i64 min'); - expect(wv.roundtripI64(i64Max) == i64Max, 'i64 max'); - expect(wv.roundtripI64(0) == 0, 'i64 zero'); - expect(wv.roundtripU64(0) == 0, 'u64 zero'); - expect(wv.roundtripU64(i64Max) == i64Max, 'u64 2^63-1'); - expect(wv.roundtripU64(u64High) == u64High, 'u64 2^63'); - expect(wv.roundtripU64(u64Max) == u64Max, 'u64 max'); - - // Floats: NaN, infinities, negative zero, subnormal, and the largest finite. - expect(wv.roundtripF64(double.nan).isNaN, 'f64 NaN'); - expect(wv.roundtripF64(double.infinity) == double.infinity, 'f64 +inf'); - expect(wv.roundtripF64(double.negativeInfinity) == double.negativeInfinity, - 'f64 -inf'); - final negZero = wv.roundtripF64(-0.0); - expect(negZero == 0 && negZero.isNegative, 'f64 -0.0'); - expect(wv.roundtripF64(5e-324) == 5e-324, 'f64 subnormal'); - expect(wv.roundtripF64(1.7976931348623157e308) == 1.7976931348623157e308, - 'f64 max'); - expect(wv.roundtripF64(0.1) == 0.1, 'f64 0.1 exact'); - - expect(wv.roundtripBool(true) && !wv.roundtripBool(false), 'bool'); - for (final c in wv.Color.values) { - expect(wv.roundtripColor(c) == c, 'color $c'); +/// Little-endian buffer bytes for the malformed vectors. +List i32(int v) => + (ByteData(4)..setInt32(0, v, Endian.little)).buffer.asUint8List(); +List u32(int v) => + (ByteData(4)..setUint32(0, v, Endian.little)).buffer.asUint8List(); +List i64(int v) => + (ByteData(8)..setInt64(0, v, Endian.little)).buffer.asUint8List(); +List str(String s) => [...u32(utf8.encode(s).length), ...utf8.encode(s)]; + +void malformed(Raw raw) { + // Tags are the declaration order of the Vector variants. + const tagI64 = 7, tagFlag = 11, tagText = 12, tagHue = 14, tagCounts = 19; + final cases = >{ + 'a truncated buffer': [...i32(tagI64), ...u32(7)], + 'an unknown tag': i32(999), + 'trailing bytes': [...i32(0), 0], + 'a bool that is neither 0 nor 1': [...i32(tagFlag), 2], + 'an undeclared enum value': [...i32(tagHue), ...i32(3)], + 'a repeated map key': [ + ...i32(tagCounts), + ...u32(2), + ...str('a'), + ...i64(1), + ...str('a'), + ...i64(2), + ], + "a string that isn't UTF-8": [...i32(tagText), ...u32(2), 0xC3, 0x28], + }; + for (final MapEntry(key: what, value: bytes) in cases.entries) { + final code = raw.code(bytes, (p, n, e) => raw.checkVector(0, p, n, e)); + expect(code == -3, '$what is rejected with -3 (got $code)'); } - expect(wv.Color.blue.value == 7 && wv.Color.fromValue(7) == wv.Color.blue, - 'Color non-contiguous discriminant'); + // Color can't spell an undeclared value, but a raw call can. + final colorCode = raw.code(const [], (_, __, e) => raw.echoColor(3, e)); + expect(colorCode == -3, 'an undeclared Color is rejected (got $colorCode)'); + final len = calloc(); + final textCode = raw.code([ + 0xC3, + 0x28, + ], (p, n, e) => raw.echoText(p, n, len, e)); + calloc.free(len); + expect(textCode == -3, 'invalid UTF-8 is rejected (got $textCode)'); + // A zero token is a marshalling failure, not a crash. + final zeroCode = raw.code( + List.filled(17, 0), + (p, n, e) => raw.sumHolder(p, n, e), + ); + expect(zeroCode == -3, 'a zero token is rejected (got $zeroCode)'); } -void checkHolder() { - // Producer-built holder: object tokens in a field, an optional, and a list - // each decode into their own wrapper holding one reference. - final h = wv.makeHolder(10, true); - expect(h.primary.value() == 10, 'primary value'); - expect(h.spare != null && h.spare!.value() == 11, 'spare value'); - expect(h.many.length == 3, 'many length'); - expect(listEq(h.many.map((t) => t.value()).toList(), [12, 13, 14]), - 'many values'); - - // Each encoding clones a fresh reference per token, so the same holder can - // be encoded repeatedly and every wrapper stays valid. - expect(wv.sumHolder(h) == 60, 'sumHolder (got ${wv.sumHolder(h)})'); - expect(wv.sumHolder(h) == 60, 'sumHolder again'); - expect(h.primary.value() == 10 && h.many[2].value() == 14, - 'wrappers alive after encoding'); - - // primaryOf returns the SAME object as h.primary: the producer compares - // Arc identity, so a holder built around the returned wrapper matches, - // while a fresh Token with the same value does not. - final p = wv.primaryOf(h); +void objects(int n) { + final full = c.vector(find(n, 'objects full')); + expect(full is c.VectorObjects, 'objects full is Objects'); + final h = (full as c.VectorObjects).value; + expect( + h.primary.value() == 10 && h.spare?.value() == 11, + 'holder primary and spare', + ); + expect( + bytesEqual([for (final t in h.many) t.value()], [12, 13, i64Min]), + 'holder many', + ); + expect( + h.byName.length == 2 && + h.byName['a']!.value() == 20 && + h.byName['b']!.value() == 21, + 'holder byName', + ); + // Each encoding mints fresh references, so the holder can be sent twice. + const expected = 10 + 11 + 12 + 13 + 20 + 21 + i64Min; + expect( + c.sumHolder(h) == expected && c.sumHolder(h) == expected, + 'sumHolder twice', + ); + + // primaryOf returns the very same object (a new reference to it). + final p = c.primaryOf(h); expect(p.value() == 10, 'primaryOf value'); - final viaPrimaryOf = wv.Holder(primary: p, spare: null, many: []); - expect(wv.samePrimary(h, viaPrimaryOf), 'primaryOf is the same object'); - expect(wv.samePrimary(h, h), 'samePrimary reflexive'); - final lookalike = wv.Token(10); - final other = - wv.Holder(primary: lookalike, spare: null, many: []); - expect(!wv.samePrimary(h, other), 'equal value, different object'); - expect(wv.sumHolder(viaPrimaryOf) == 10, 'sum of a Dart-built holder'); - - // Absent optional object. - final bare = wv.makeHolder(0, false); - expect(bare.spare == null, 'spare absent'); - expect(wv.sumHolder(bare) == 0 + 2 + 3 + 4, 'sum without spare'); - expect(bare.primary.value() == 0, 'zero-valued token'); - - // Dart-built holder with Dart-created tokens, one of them repeated in every - // position: every occurrence is a separate cloned reference. - final t = wv.Token(5); - final repeated = wv.Holder(primary: t, spare: t, many: [t, t, t]); - expect(wv.sumHolder(repeated) == 25, 'repeated token summed 5 times'); - expect(wv.samePrimary(repeated, repeated), 'repeated reflexive'); - final mixed = wv.Holder( - primary: wv.Token(-1), spare: wv.Token(i64Min + 1), many: [t]); - expect(wv.sumHolder(mixed) == i64Min + 5, 'i64 extremes in tokens'); - final empty = wv.Holder(primary: wv.Token(7), many: []); - expect(wv.sumHolder(empty) == 7, 'holder with empty list'); - - // Release: dispose every wrapper, twice; disposed wrappers are refused at - // encode time (a Dart StateError, never a native double free), and - // wrappers that share an object with a disposed one stay valid. - p.dispose(); + expect( + c.samePrimary(h, c.Holder(primary: p, many: const [], byName: const {})), + 'primaryOf is the same object', + ); + final twin = c.Token(10); + expect( + !c.samePrimary( + h, + c.Holder(primary: twin, many: const [], byName: const {}), + ), + "an equal value isn't the same object", + ); + + // A holder built from consumer tokens, the same wrapper in several slots. + final minus4 = c.Token(-4); + final mine = c.Holder( + primary: twin, + spare: twin, + many: [twin, twin, minus4], + byName: {'k': twin}, + ); + expect(c.sumHolder(mine) == 10 * 5 - 4, 'consumer holder sums to 46'); + + minus4.dispose(); + twin.dispose(); + twin.dispose(); // disposing twice is safe p.dispose(); - expect(h.primary.value() == 10, 'h.primary alive after primaryOf disposed'); - try { - wv.sumHolder(viaPrimaryOf); - throw StateError('expected StateError encoding a disposed token'); - } on StateError catch (e) { - expect(e.message.contains('dispose'), 'disposed token in a buffer'); - } - t.dispose(); - t.dispose(); - try { - t.value(); - throw StateError('expected StateError after dispose'); - } on StateError catch (_) {} - for (final holder in [h, bare]) { - holder.primary.dispose(); - holder.spare?.dispose(); - for (final m in holder.many) { - m.dispose(); - m.dispose(); - } - } - lookalike.dispose(); - mixed.primary.dispose(); - mixed.spare!.dispose(); - empty.primary.dispose(); - - // Fresh objects still work after all of the above were released. - final again = wv.makeHolder(100, true); - expect(wv.sumHolder(again) == 100 + 101 + 102 + 103 + 104, 'fresh holder'); - again.primary.dispose(); - again.spare!.dispose(); - for (final m in again.many) { - m.dispose(); - } + release(full); + expectThrows(() => p.value(), "a disposed wrapper can't be used"); } Future main() async { - checkScalars(); - checkComposite(); - checkShapes(); - checkDirect(); - checkHolder(); + final n = c.vectorCount(); + expect(n >= 60, 'at least 60 vectors (got $n)'); + + everyVector(n); + literalVectors(n); + spotChecks(n); + outOfRange(n); + malformed(Raw()); + objects(n); + await expectNoLeaks('codec'); - print('dart/codec: OK'); + print('dart/codec: OK ($n vectors)'); } diff --git a/conformance/dart/contacts.dart b/conformance/dart/contacts.dart deleted file mode 100644 index 89397eb8..00000000 --- a/conformance/dart/contacts.dart +++ /dev/null @@ -1,88 +0,0 @@ -// Conformance consumer: contacts sample, Dart target. -// -// The ContactBook interface (unnamed factory, instance methods, dispose and -// use after dispose), the Contact record with an optional string and a -// C-style enum field, the typed ContactsException domain, the nested -// `contacts.groups` module taking and returning the parent's ContactType -// enum and ContactBook interface and reporting the parent's domain, and the -// sibling `directory` root sharing the Contact record and declaring its own -// DirectoryException domain. - -import 'package:contacts/contacts.dart' as ct; - -import 'support.dart'; - -void run() { - final book = ct.ContactBook(); - - final alice = - book.add('Alice', 'Smith', 'alice@example.com', ct.ContactType.work); - expect(alice.id > 0, 'alice id positive'); - expect(alice.firstName == 'Alice' && alice.lastName == 'Smith', 'names'); - expect(alice.email == 'alice@example.com', 'email'); - expect(alice.contactType == ct.ContactType.work, 'contactType'); - expect(book.get(alice.id).firstName == 'Alice', 'get'); - - final bob = book.add('Bob', 'Jones', null, ct.ContactType.personal); - expect(bob.email == null, 'absent email'); - book.add('Carol', 'Adams', 'carol@example.com', ct.ContactType.work); - book.add('Dan', 'Brown', null, ct.ContactType.other); - expect(book.count() == 4, 'count'); - final names = book.list().map((c) => c.firstName).toList()..sort(); - expect(names.join(',') == 'Alice,Bob,Carol,Dan', 'list (got $names)'); - - final invalid = expectThrows( - () => book.add('', 'Smith', null, ct.ContactType.personal), - 'empty name'); - expect(invalid is ct.InvalidNameException && invalid.code == 1, - 'InvalidName (got $invalid)'); - final missing = - expectThrows(() => book.get(9999), 'missing id'); - expect(missing.code == 2 && missing is ct.NativeException, 'NotFound'); - - // Nested module: the parent's enum and interface cross the module - // boundary, and the parent's error domain applies. - expect(ct.countOfType(book, ct.ContactType.work) == 2, 'countOfType work'); - expect(ct.countOfType(book, ct.ContactType.other) == 1, 'countOfType other'); - expect(ct.dominantType(book) == ct.ContactType.work, 'dominantType'); - final work = ct.splitByType(book, ct.ContactType.work); - expect(work.count() == 2, 'splitByType returns a new book'); - expect(work.list().every((c) => c.contactType == ct.ContactType.work), - 'split book holds only work contacts'); - expect(ct.firstOfType(book, ct.ContactType.personal).firstName == 'Bob', - 'firstOfType'); - work.dispose(); - final empty = ct.ContactBook(); - expectThrows( - () => ct.firstOfType(empty, ct.ContactType.work), - 'nested module raises the parent domain'); - expect(ct.dominantType(empty) == ct.ContactType.personal, 'empty book'); - empty.dispose(); - - // Sibling root: the shared record crosses into another module tree. - final card = ct.card(alice); - expect(card.displayName == 'Smith, Alice', 'card name'); - expect(card.initials == 'AS' && card.hasEmail, 'card initials and email'); - expect(!ct.card(bob).hasEmail, 'card without email'); - final sorted = ct.sorted(book.list()); - expect(sorted.map((c) => c.lastName).join(',') == 'Adams,Brown,Jones,Smith', - 'sorted (got ${sorted.map((c) => c.lastName)})'); - final none = expectThrows( - () => ct.sorted([]), 'sorted empty'); - expect(none is ct.EmptyException && none.code == 1, 'EmptyException'); - - expect(book.remove(alice.id), 'remove'); - expect(!book.remove(alice.id), 'second remove'); - expect(book.count() == 3, 'count after remove'); - - book.dispose(); - book.dispose(); - final gone = expectThrows(() => book.count(), 'use after dispose'); - expect(gone.message.contains('dispose'), 'use after dispose message'); -} - -Future main() async { - run(); - await expectNoLeaks('contacts'); - print('dart/contacts: OK'); -} diff --git a/conformance/dart/events.dart b/conformance/dart/events.dart deleted file mode 100644 index 7fa78ff6..00000000 --- a/conformance/dart/events.dart +++ /dev/null @@ -1,200 +0,0 @@ -// Conformance consumer: events sample, Dart target. -// -// A consumer-implemented `Subscriber`: `route` (a `Result`-returning method -// on the producer side) and `onMessage` return values synchronously through -// isolate-local trampolines, while the void `onAttached` is forwarded to the -// event loop and receives an `EventBus` it owns. Also covers the -// reference-counted `EventBus` (dispose, double dispose, use after dispose, -// an adopted second reference outliving the original), the `Delivery` enum, -// the buffered `Message` record decoded inside a callback, the lazy -// `messages()` iterator (including an abandoned iteration), the optional -// `lastMessage()`, the Future-backed `publishLater`, Dart exceptions in -// value methods surfacing to the caller as the foreign code (-4), and an -// exception in a void method reaching the zone that passed the subscriber. - -import 'dart:async'; - -import 'package:events/events.dart' as ev; - -import 'support.dart'; - -/// Records every callback. `skipTopic` routes as skip, `stop` as -/// acceptAndStop, `failTopic` throws from `route`, and `failOnMessage` -/// throws from `onMessage`. -class RecordingSubscriber extends ev.Subscriber { - RecordingSubscriber(this.name, - {this.skipTopic = '', this.failTopic = '', this.failOnMessage = false}); - - final String name; - final String skipTopic; - final String failTopic; - final bool failOnMessage; - - final List routed = []; - final List received = []; - int attachedCalls = 0; - int? subscribersSeenOnAttach; - ev.EventBus? adoptedBus; - - @override - ev.Delivery route(String topic) { - routed.add(topic); - if (topic == failTopic) throw StateError('$name rejected topic $topic'); - if (topic == skipTopic) return ev.Delivery.skip; - if (topic == 'stop') return ev.Delivery.acceptAndStop; - return ev.Delivery.accept; - } - - @override - int onMessage(ev.Message message) { - if (failOnMessage) throw ArgumentError('$name cannot take ${message.text}'); - received.add(message); - return received.length; - } - - @override - void onAttached(ev.EventBus bus) { - attachedCalls++; - subscribersSeenOnAttach = bus.subscriberCount(); - // The reference is ours: the keeper holds on to it, the others drop it. - if (name == 'keeper') { - adoptedBus = bus; - } else { - bus.dispose(); - } - } -} - -/// A subscriber whose void `onAttached` throws. -class FailingAttachSubscriber extends ev.Subscriber { - @override - ev.Delivery route(String topic) => ev.Delivery.skip; - - @override - int onMessage(ev.Message message) => 0; - - @override - void onAttached(ev.EventBus bus) { - bus.dispose(); - throw StateError('refuses to attach'); - } -} - -Future run() async { - // A free function taking the callback interface; the Delivery return - // crosses back as its discriminant. - final probe = RecordingSubscriber('probe', skipTopic: 'quiet', failTopic: 'boom'); - expect(ev.routeOnce(probe, 'quiet') == ev.Delivery.skip, 'routeOnce skip'); - expect(ev.routeOnce(probe, 'news') == ev.Delivery.accept, 'routeOnce accept'); - expect(ev.routeOnce(probe, 'stop') == ev.Delivery.acceptAndStop, - 'routeOnce acceptAndStop'); - final routeFailure = expectThrows( - () => ev.routeOnce(probe, 'boom'), 'routeOnce failure'); - expect(routeFailure.code == ev.NativeException.foreignCode, - 'Result-returning callback failure is -4 (got ${routeFailure.code})'); - expect(routeFailure.message.contains('probe rejected topic boom'), - 'foreign message (got ${routeFailure.message})'); - expect(probe.routed.join(',') == 'quiet,news,stop,boom', 'routeOnce log'); - expect(ev.Delivery.fromValue(2) == ev.Delivery.acceptAndStop, 'fromValue'); - - final bus = ev.EventBus(); - expect(bus.subscriberCount() == 0, 'fresh bus'); - expect(bus.messages().isEmpty, 'no messages'); - expect(bus.lastMessage() == null, 'no last message'); - - // subscribe returns the new count; the void onAttached arrives on the - // event loop with an EventBus reference the subscriber owns. - final keeper = RecordingSubscriber('keeper', skipTopic: 'quiet'); - final second = RecordingSubscriber('second'); - expect(bus.subscribe(keeper) == 1, 'first subscribe'); - expect(bus.subscribe(second) == 2, 'second subscribe'); - await settle(() => keeper.attachedCalls == 1 && second.attachedCalls == 1, - 'onAttached delivered'); - expect(keeper.subscribersSeenOnAttach == 2 && keeper.adoptedBus != null, - 'keeper adopted a live bus (saw ${keeper.subscribersSeenOnAttach})'); - expect(keeper.adoptedBus!.subscriberCount() == 2, 'adopted bus is the same'); - - // publish: route and onMessage run synchronously during the call. - expect(bus.publish('news', 'hello', ['x', 'y']) == 2, 'publish news'); - final m = keeper.received.single; - expect(m.seq == 1 && m.topic == 'news' && m.text == 'hello', 'message'); - expect(m.tags.join(',') == 'x,y', 'message tags'); - expect(bus.publish('quiet', 'psst', []) == 1, 'publish quiet'); - expect(second.received.last.seq == 2 && second.received.last.tags.isEmpty, - 'second received quiet'); - expect(bus.publish('stop', 'last', []) == 1, 'publish stop'); - expect(keeper.received.last.text == 'last' && second.received.length == 2, - 'acceptAndStop'); - - // A lazy iterator; each iteration is a fresh native iterator, and an - // abandoned one is released by its finalizer. - expect(bus.messages().join('|') == 'hello|psst|last', 'messages'); - expect(bus.messages().first == 'hello', 'abandoned iteration'); - expect(bus.messages().skip(1).first == 'psst', 'skip then first'); - final last = bus.lastMessage()!; - expect(last.seq == 3 && last.text == 'last', 'lastMessage'); - - // A Dart exception in route (a Result-returning producer method) aborts - // publish with the foreign code; earlier subscribers already accepted. - final rejecting = RecordingSubscriber('rejecting', failTopic: 'boom'); - expect(bus.subscribe(rejecting) == 3, 'third subscribe'); - final rejected = expectThrows( - () => bus.publish('boom', 'x', []), 'route throws'); - expect(rejected.code == -4 && rejected.message.contains('rejected topic boom'), - 'route failure (got $rejected)'); - expect(keeper.received.length == 3 && second.received.length == 3, - 'earlier subscribers accepted boom'); - expect(bus.publish('ok', 'y', ['t']) == 3, 'bus usable afterwards'); - - // A Dart exception in onMessage (a plain-return producer method). - final fragile = RecordingSubscriber('fragile', failOnMessage: true); - expect(bus.subscribe(fragile) == 4, 'fourth subscribe'); - final failed = expectThrows( - () => bus.publish('any', 'payload', []), 'onMessage throws'); - expect(failed.code == ev.NativeException.foreignCode && - failed.message.contains('fragile cannot take payload'), - 'onMessage failure (got $failed)'); - - // An exception in a void method is an uncaught error of the zone that - // passed the subscriber; the call itself succeeded. - final zoneErrors = []; - runZonedGuarded(() { - expect(bus.subscribe(FailingAttachSubscriber()) == 5, 'fifth subscribe'); - }, (e, _) => zoneErrors.add(e)); - await settle(() => zoneErrors.isNotEmpty, 'void method error reaches zone'); - expect(zoneErrors.single.toString().contains('refuses to attach'), - 'zone error (got $zoneErrors)'); - - bus.clearSubscribers(); - expect(bus.subscriberCount() == 0, 'cleared'); - expect(bus.publish('news', 'nobody', []) == 0, 'no subscribers'); - - // publishLater runs on a producer thread. Value-returning callback - // methods only run on the isolate's thread, so it is driven with no - // subscribers attached. - expect(await bus.publishLater('later', 'zzz') == 0, 'publishLater'); - expect(bus.lastMessage()!.text == 'zzz', 'publishLater logged'); - - // Lifetime: the adopted reference outlives the original wrapper. - bus.dispose(); - bus.dispose(); - final gone = expectThrows(() => bus.subscriberCount(), 'disposed'); - expect(gone.message.contains('dispose'), 'use after dispose message'); - final adopted = keeper.adoptedBus!; - expect(adopted.lastMessage()!.text == 'zzz', 'adopted bus alive'); - expect(adopted.publish('final', 'bye', []) == 0, 'adopted publish'); - adopted.dispose(); - - // A bus dropped while holding subscribers releases them. - final short = ev.EventBus(); - final tenant = RecordingSubscriber('tenant'); - short.subscribe(tenant); - await settle(() => tenant.attachedCalls == 1, 'tenant attached'); - short.dispose(); -} - -Future main() async { - await run(); - await expectNoLeaks('events'); - print('dart/events: OK'); -} diff --git a/conformance/dart/kvstore.dart b/conformance/dart/kvstore.dart index 51224a17..77b61e4b 100644 --- a/conformance/dart/kvstore.dart +++ b/conformance/dart/kvstore.dart @@ -1,407 +1,778 @@ // Conformance consumer: kvstore sample, Dart target. // -// Full-surface drive of the generated dart:ffi wrapper: the Store interface -// (fallible `Store.open` named factory, instance methods passing the object -// pointer, the `defaultCapacity` static, the deprecated `legacyPut`), the -// typed KvException hierarchy (KeyNotFoundException = 1001, -// ExpiredException = 1002, IoException = 1004) thrown by throwing members, -// optional record returns (`Entry?`) decoded from value buffers with bytes / -// optional-scalar / list / map fields, the plain-value Entry class, the -// iterator-backed `listKeys` method, the cross-module `getStats`, the -// consumer-implemented `EvictionListener` callback interface (set, replaced, -// detached by returning false, cleared, and throwing so the caller sees the -// foreign error code -4), the object graph (`share()` as a second wrapper to -// the same native object, `fork()`, `Store?` in and out of `larger`, the -// `StoreInfo` record carrying a `Store` field plus an optional `Store`, -// `openMany` returning a list of objects, `totalCount` taking a list of -// objects and a Dart-built record holding objects), the Future-returning, -// cancellable `compact`, dispose / double dispose / use after dispose on -// every wrapper, and the producer's leak counters at exit. Throws (non-zero -// exit) on any mismatch. - -import 'package:kvstore/kvstore.dart' as wv; +// Drives the feature-complete producer through the generated bindings: +// +// * the load-time checks (ABI revision, both modules' contract tables), +// which loading the bindings performs; +// * `Store`: fallible and infallible constructors (`Store.open`, `Store()`), +// methods, statics, the deprecated `size()`, the `Entry` and `StoreInfo` +// value classes, the `EntryKind` enum, maps and optionals, and the +// logical clock; +// * the sealed `KvException` hierarchy with its payload fields +// (`KeyNotFound`, `Expired`, `StoreFull`, `Rejected`), and runtime +// failures (-4) as the root `NativeException`; +// * lazy `Iterable`s of strings (throwing at launch), records, and objects, +// including one abandoned part-way; +// * three callback interfaces implemented in Dart: a `Listener` (retained, +// filtered by `accepts`, told about every `Change` on the event loop, +// and detached when it throws), a `Policy` (a record return, a throwing +// method whose `RejectedException` reaches the `put` caller with its +// payload, an object parameter and object return), and a `Loader` passed +// as an optional callback (string, bytes, and optional-object returns; +// typed errors decoded by the producer or passed through); +// * `Store` objects in every position: parameter, return, optional, list, +// map value, record field, iterator element, async result, and callback +// parameter and return; +// * futures: an async free function returning an object, a cancellable +// method cancelled mid-pause (completing at once with +// `CancelledException` while its background work stops cooperatively, +// shown by `activeJobs`), an async list launched concurrently, and an +// async function in the nested `kv.stats` module; +// * the sibling `report` root (the shared `Entry` record and its own error +// domain). +// +// One assertion other languages make is skipped: compaction notifies +// listeners from a producer thread, and a listener's `accepts` returns a +// value, which Dart can only run on the isolate's thread (a call from +// another thread aborts the process; see the Dart generator's threading +// rules). So no listener is subscribed while `compact` runs. +// +// Releases of consumer callbacks are observed through the producer's +// callback counter (`debug_live(1)`). Ends by asserting the producer's leak +// counters are zero. + +import 'dart:convert'; + +import 'package:kvstore/kvstore.dart'; import 'support.dart'; -/// Records every eviction; returns false (detaching itself) once it has seen -/// `keepFor` evictions, and throws when `fail` is set. -class RecordingListener extends wv.EvictionListener { - RecordingListener(this.name, {this.keepFor = 1 << 30, this.fail = false}); +final int Function(int) live = debugLive('kvstore'); - final String name; - final int keepFor; - final bool fail; - final List<(wv.Entry, wv.EvictionReason)> evictions = - <(wv.Entry, wv.EvictionReason)>[]; +/// Live consumer callbacks the producer holds. +int liveCallbacks() => live(1); - @override - bool onEvict(wv.Entry entry, wv.EvictionReason reason) { - if (fail) throw StateError('$name refuses eviction of ${entry.key}'); - evictions.add((entry, reason)); - return evictions.length < keepFor; - } +String text(List bytes) => utf8.decode(bytes); - List get keys => evictions.map((e) => e.$1.key).toList(); +extension on Store { + Entry putText( + String key, + String value, [ + EntryKind kind = EntryKind.persistent, + int? ttl, + ]) => put(key, utf8.encode(value), kind, ttl); } -Future run() async { - // Fallible constructor: an empty path reports the IoError domain code - // through the typed exception hierarchy. - try { - wv.Store.open(''); - throw StateError('expected IoException for empty path'); - } on wv.IoException catch (e) { - expect(e.code == 1004, 'IoError code == 1004 (got ${e.code})'); - expect(e.message == 'I/O failure', 'IoError message (got ${e.message})'); - expect(e is wv.KvException, 'IoException extends KvException'); - expect(e is wv.NativeException, 'IoException extends the generic brand'); - } +/// Whether [a] and [b] wrap the same native store: a write through one is +/// visible through the other (distinct stores never share entries). +bool sameStore(Store a, Store b) { + const probe = '__identity_probe__'; + a.putText(probe, 'x'); + final same = b.find(probe) != null; + a.delete(probe); + return same; +} - final store = wv.Store.open('/tmp/conformance-kvstore-dart'); - - // Static method on the interface. - expect(wv.Store.defaultCapacity() == 1000000, 'default capacity'); - - final payload = [1, 2, 3]; - expect(store.put('alpha', payload, wv.EntryKind.persistent, null), - 'put alpha'); - expect(store.put('beta', payload, wv.EntryKind.volatile, 3600), 'put beta'); - expect(store.count() == 2, 'count == 2'); - - // Iterator-backed list-of-string method, with and without the prefix; the - // producer yields sorted keys, and each iteration is a fresh native - // iterator. - final keys = store.listKeys(null).toList(); - expect(keys.length == 2 && keys[0] == 'alpha' && keys[1] == 'beta', - 'listKeys values (got $keys)'); - final filtered = store.listKeys('al').toList(); - expect(filtered.length == 1 && filtered[0] == 'alpha', - 'listKeys prefix filter'); - expect(store.listKeys('zzz').isEmpty, 'listKeys no match'); - expect(store.listKeys(null).first == 'alpha', 'abandoned partial iteration'); - - // Optional record return: the `Entry?` buffer decodes into a plain value - // class covering every complex field type. - final alpha = store.get('alpha')!; - expect(alpha.id > 0, 'entry id positive'); - expect(alpha.key == 'alpha', 'entry key'); - final value = alpha.value; - expect(value.length == 3 && value[0] == 1 && value[2] == 3, 'entry value'); - expect(alpha.createdAt > 0, 'entry createdAt positive'); - expect(alpha.expiresAt == null, 'entry expiresAt null'); - expect(alpha.tags.isEmpty, 'entry tags empty'); - expect(alpha.metadata.isEmpty, 'entry metadata empty'); - - final beta = store.get('beta')!; - expect(beta.expiresAt != null && beta.expiresAt! > beta.createdAt, - 'beta expiresAt set'); - expect(beta.id == alpha.id + 1, 'ids are monotonic'); - - // Typed error: a missing key throws the KeyNotFoundException class of the - // KvException domain, carrying its stable code and default message. - try { - store.get('missing'); - throw StateError('expected KeyNotFoundException for missing key'); - } on wv.KeyNotFoundException catch (e) { - expect(e.code == 1001, 'KeyNotFound code == 1001 (got ${e.code})'); - expect(e.message == 'key not found', 'KeyNotFound message'); - expect(e is wv.KvException, 'KeyNotFound extends KvException'); +void expectKeyNotFound(String key, void Function() body) { + final e = expectThrows(body, 'key $key'); + expect( + e.code == 1001 && e.key == key, + 'KeyNotFound payload key $key (got ${e.key})', + ); +} + +bool listEquals(List a, List b) { + if (a.length != b.length) return false; + for (var i = 0; i < a.length; i++) { + if (a[i] != b[i]) return false; } + return true; +} - // Deprecated method still works. - expect(store.legacyPut('legacy', payload), 'legacy put'); - expect(store.count() == 3, 'count == 3 after legacy put'); - expect(store.delete('legacy'), 'delete legacy'); - expect(!store.delete('legacy'), 'second delete returns false'); - - // Record construction: scalars, bytes, optional, list, and map fields on - // the plain-value Entry class. - final entry = wv.Entry( - id: 7, - key: 'built', - value: payload, - createdAt: 1000, - expiresAt: null, - tags: ['hot', 'fast'], - metadata: {'source': 'test', 'env': 'prod'}, - ); - expect(entry.id == 7, 'entry id == 7'); - expect(entry.expiresAt == null, 'entry expiresAt null'); - final tags = entry.tags..sort(); - expect(tags.length == 2 && tags[0] == 'fast' && tags[1] == 'hot', - 'entry tags'); - final md = entry.metadata; - expect(md.length == 2 && md['source'] == 'test' && md['env'] == 'prod', - 'entry metadata'); - - // Cross-module call: getStats lives in kv.stats and takes the parent - // module's Store interface as a parameter. - final st = wv.getStats(store); - expect(st.totalEntries == 2, 'stats total entries == 2'); - expect(st.totalBytes == 6, 'stats total bytes == 6 (got ${st.totalBytes})'); - expect(st.expiredEntries == 0, 'stats expired entries == 0'); - - // ── Eviction listener (callback interface) ── - // delete fires onEvict synchronously on the calling thread with the full - // Entry record and the Deleted reason. - final listener = RecordingListener('first'); - store.setEvictionListener(listener); - expect(store.delete('beta'), 'delete beta'); - expect(listener.evictions.length == 1, 'one eviction (got ${listener.keys})'); - final (evictedBeta, betaReason) = listener.evictions.single; - expect(betaReason == wv.EvictionReason.deleted, 'reason deleted'); - expect(evictedBeta.key == 'beta', 'evicted key'); - expect(evictedBeta.id == beta.id, 'evicted id matches the stored entry'); - expect(evictedBeta.value.length == 3 && evictedBeta.value[1] == 2, - 'evicted value bytes'); - expect(evictedBeta.expiresAt == beta.expiresAt, 'evicted expiresAt'); - expect(evictedBeta.tags.isEmpty && evictedBeta.metadata.isEmpty, - 'evicted empty list and map'); - - // An expired entry is evicted on read with the Expired reason, and the - // read itself reports ExpiredException (1002). - expect(store.put('expiring', [9], wv.EntryKind.volatile, -1), - 'put expiring'); - try { - store.get('expiring'); - throw StateError('expected ExpiredException'); - } on wv.ExpiredException catch (e) { - expect(e.code == 1002, 'Expired code == 1002 (got ${e.code})'); +/// Churns garbage until [done] holds, so finalizers run. +Future collectUntil(bool Function() done, String msg) async { + final watch = Stopwatch()..start(); + var junk = []; + while (!done()) { + if (watch.elapsed > const Duration(seconds: 10)) + throw StateError('timed out: $msg'); + for (var i = 0; i < 20000; i++) { + junk.add(List.filled(64, i)); + } + junk = []; + await Future.delayed(const Duration(milliseconds: 5)); } - expect(listener.evictions.length == 2, 'expiry evicted'); - expect(listener.evictions.last.$1.key == 'expiring' && - listener.evictions.last.$2 == wv.EvictionReason.expired, - 'expiry eviction details'); - expect(store.count() == 1, 'only alpha remains'); - - // Replacing the listener: the new one receives, the old one does not. - final replacement = RecordingListener('replacement'); - store.setEvictionListener(replacement); - expect(store.put('r1', payload, wv.EntryKind.persistent, null), 'put r1'); - expect(store.delete('r1'), 'delete r1'); - expect(listener.evictions.length == 2, 'replaced listener is silent'); - expect(replacement.keys.join(',') == 'r1', 'replacement received r1'); - - // Clearing the listener stops notifications. - store.clearEvictionListener(); - expect(store.put('r2', payload, wv.EntryKind.persistent, null), 'put r2'); - expect(store.delete('r2'), 'delete r2'); - expect(replacement.evictions.length == 1, 'cleared listener is silent'); - store.clearEvictionListener(); - - // Returning false detaches the listener after that eviction. - final brief = RecordingListener('brief', keepFor: 2); - store.setEvictionListener(brief); - for (final k in ['d1', 'd2', 'd3']) { - expect(store.put(k, payload, wv.EntryKind.persistent, null), 'put $k'); - expect(store.delete(k), 'delete $k'); +} + +// ── listener (consumer-implemented, retained) ───────────────────────────── + +class RecordingListener implements Listener { + RecordingListener({this.skip, this.failOn}); + + final String? skip; + final String? failOn; + final List changes = []; + + @override + bool accepts(String key) { + if (key == failOn) throw StateError('listener refused'); + return key != skip; } - expect(brief.keys.join(',') == 'd1,d2', - 'listener detached after returning false (got ${brief.keys})'); - - // A throwing listener: the Dart exception surfaces to the caller of - // `delete` as the generic exception with the foreign code (-4), not as a - // KvException; the entry was already removed, the VM keeps running, and - // the listener stays attached. - final hostile = RecordingListener('hostile', fail: true); - store.setEvictionListener(hostile); - expect(store.put('doomed', payload, wv.EntryKind.persistent, null), - 'put doomed'); - try { - store.delete('doomed'); - throw StateError('expected NativeException from throwing listener'); - } on wv.NativeException catch (e) { - expect(e is! wv.KvException, 'foreign error is not a domain error'); - expect(e.code == wv.NativeException.foreignCode && e.code == -4, - 'foreign code -4 (got ${e.code})'); - expect(e.message.contains('hostile refuses eviction of doomed'), - 'foreign message carries the exception text (got ${e.message})'); + + @override + void onChange(Change change) => changes.add(change); + + List get puts => changes.whereType().toList(); +} + +// ── policy (consumer-implemented, rich returns, throws) ─────────────────── + +/// An exception whose text is exactly its message. +class Failure implements Exception { + Failure(this.message); + + final String message; + + @override + String toString() => message; +} + +/// Routes `b/` keys to [other], a reference this policy owns. +class TestPolicy implements Policy { + TestPolicy(this.other); + + final Store other; + int admitted = 0; + + @override + Entry admit(Entry entry) { + admitted++; + expect(entry.version == 0, 'the store assigns the version after admission'); + if (entry.key.startsWith('secret')) { + throw RejectedException( + entry.key, + 'no secrets', + 'secrets are not stored', + ); + } + if (entry.key.startsWith('boom')) throw Failure('policy exploded'); + // Tag it and store it encrypted (and try to rename it, which the store + // ignores). + return Entry( + key: 'renamed', + value: entry.value, + kind: EntryKind.encrypted, + version: entry.version, + expiresAt: entry.expiresAt, + tags: const ['admitted'], + metadata: entry.metadata, + ); } - expect(store.count() == 1, 'entry removed despite the listener failing'); - expect(store.put('doomed2', payload, wv.EntryKind.persistent, null), - 'put doomed2'); - try { - store.delete('doomed2'); - throw StateError('expected a second foreign error'); - } on wv.NativeException catch (e) { - expect(e.code == -4, 'listener still attached after failing'); + + @override + Store route(String key, Store home) { + final target = key.startsWith('b/') ? other : home; + // The producer gets its own reference; `home` was ours to release. + if (!identical(target, home)) home.dispose(); + return target; } - store.clearEvictionListener(); - expect(store.put('calm', payload, wv.EntryKind.persistent, null), 'put calm'); - expect(store.delete('calm'), 'delete after clearing the hostile listener'); - - // ── Object graph ── - // share() returns a second wrapper to the SAME native object: a mutation - // through one is visible through the other, and disposing one (twice) leaves - // the other usable. - final shared = store.share(); - expect(shared.count() == store.count(), 'share sees the same count'); - expect(shared.put('via-share', [4, 4], wv.EntryKind.persistent, null), - 'put through share'); - expect(store.get('via-share')!.value.length == 2, - 'mutation through share visible in the original'); - expect(store.put('via-orig', [5], wv.EntryKind.persistent, null), - 'put through original'); - expect(shared.get('via-orig')!.value.single == 5, - 'mutation through original visible in share'); - expect(store.count() == 3 && shared.count() == 3, 'counts agree'); - shared.dispose(); - shared.dispose(); - try { - shared.count(); - throw StateError('expected StateError after dispose'); - } on StateError catch (e) { - expect(e.message.contains('dispose'), 'use after dispose message'); +} + +// ── loader (consumer-implemented, passed as an optional parameter) ──────── + +class TestLoader implements Loader { + TestLoader([this.backup]); + + final Store? backup; + + @override + String name() => 'dart-loader'; + + @override + Store? fallback(String key) => key == 'fb' ? backup : null; + + @override + List load(String key) => switch (key) { + 'missing' => throw KeyNotFoundException('missing', 'not in the loader'), + 'elsewhere' => throw KeyNotFoundException('other', 'not in the loader'), + 'broken' => throw Failure('loader is broken'), + _ => utf8.encode('loaded:$key'), + }; +} + +// ── sections ────────────────────────────────────────────────────────────── + +Future constructors() async { + final e = expectThrows( + () => Store.open(''), + 'Store.open("")', + ); + expect( + e.code == 1004 && e.message == 'invalid path', + 'InvalidPath code and message', + ); + + final s = Store(); + expect(s.path() == 'memory', 'Store() path'); + expect( + s.capacity() == Store.defaultCapacity(), + 'capacity == defaultCapacity', + ); + expect(Store.defaultCapacity() == 1000000, 'defaultCapacity == 1000000'); + s.dispose(); + + final opened = await openStore('/async'); + expect(opened.path() == '/async', 'openStore path'); + opened.dispose(); + await expectThrowsAsync( + () => openStore(''), + 'openStore("") rejects with InvalidPath', + ); +} + +void basics() { + final s = Store.open('/basics'); + // put returns the stored entry; the version counts puts of the key. + final e1 = s.putText('alpha', 'one', EntryKind.persistent); + expect( + e1.key == 'alpha' && + text(e1.value) == 'one' && + e1.kind == EntryKind.persistent, + 'put alpha', + ); + expect( + e1 == + Entry( + key: 'alpha', + value: utf8.encode('one'), + kind: EntryKind.persistent, + version: 1, + tags: const [], + metadata: const {}, + ), + 'put alpha fields (got $e1)', + ); + final e2 = s.putText('alpha', 'two', EntryKind.volatile); + expect(e2.version == 2 && e2.kind == EntryKind.volatile, 'put alpha again'); + + expect(text(s.get('alpha').value) == 'two', 'get alpha'); + final missing = expectThrows( + () => s.get('nope'), + 'get(nope)', + ); + expect( + missing.key == 'nope' && missing.message == 'key not found: nope', + 'KeyNotFound(nope)', + ); + expect(s.find('alpha')?.version == 2, 'find alpha'); + expect(s.find('nope') == null, 'find nope'); + + // TTLs follow the logical clock; an expired get reports when. + expect( + s.putText('ttl', 'x', EntryKind.volatile, 10).expiresAt == 10, + 'ttl expiresAt', + ); + expect(s.now() == 0, 'clock starts at 0'); + expect(s.tick(9) == 9 && s.count() == 2, 'tick 9'); + expect(s.tick(1) == 10 && s.count() == 1, 'tick 10'); + final expired = expectThrows( + () => s.get('ttl'), + 'get(ttl)', + ); + expect(expired.key == 'ttl' && expired.expiredAt == 10, 'Expired payload'); + expect( + expired.message == 'entry ttl expired at 10', + 'Expired message (got ${expired.message})', + ); + expectKeyNotFound('ttl', () => s.get('ttl')); // the expired read removed it + + // Capacity: a new key past it is StoreFull { capacity }. + s.setCapacity(1); + expect(s.capacity() == 1, 'setCapacity'); + s.putText('alpha', 'three'); // replacing is fine + final full = expectThrows( + () => s.putText('beta', 'b'), + 'put beta at capacity', + ); + expect(full.capacity == 1, 'StoreFull capacity'); + s.setCapacity(100); + + // delete, clear, and the deprecated size(). + s.putText('beta', 'b'); + expect(s.delete('beta') && !s.delete('beta'), 'delete twice'); + // ignore: deprecated_member_use + final size = s.size(); + expect(size == s.count() && size == 1, 'deprecated size() == count()'); + expect(s.clear() == 1 && s.count() == 0, 'clear'); + s.dispose(); +} + +Future iterators() async { + final s = Store.open('/iter'); + s.putText('user.bob', 'b'); + s.putText('user.alice', 'a'); + s.putText('sys.x', 'xx'); + + expect( + listEquals(s.keys(null).toList(), ['sys.x', 'user.alice', 'user.bob']), + 'keys in order', + ); + expect(live(2) == 0, 'an exhausted iterator is released'); + expectKeyNotFound('zzz', () => s.keys('zzz').toList()); + + // Abandoning an iterator part-way releases it (through its finalizer). + void abandon() { + final keys = s.keys('user.').iterator; + expect(keys.moveNext() && keys.current == 'user.alice', 'first user key'); + expect(live(2) == 1, 'the iterator is live'); } - expect(store.count() == 3, 'original alive after disposing its share'); - - // fork() is a distinct object: a copy that diverges. - final forked = store.fork(); - expect(forked.count() == 3, 'fork copies live entries'); - expect(forked.put('only-fork', payload, wv.EntryKind.persistent, null), - 'put into fork'); - expect(forked.count() == 4 && store.count() == 3, 'fork diverged'); - try { - store.get('only-fork'); - throw StateError('fork mutation leaked into the original'); - } on wv.KeyNotFoundException catch (_) {} - - // larger(): `Store?` both ways. - final empty = wv.Store.open('/tmp/conformance-kvstore-dart-empty'); - expect(empty.larger(null) == null, 'empty.larger(null) is null'); - final self = store.larger(null); - expect(self != null, 'store.larger(null) present'); - expect(self!.count() == 3, 'larger(null) is the receiver'); - expect(self.put('via-larger', payload, wv.EntryKind.persistent, null), - 'put through larger(null)'); - expect(store.count() == 4, 'larger(null) aliases the receiver'); - self.dispose(); - final bigger = empty.larger(forked); - expect(bigger != null && bigger.count() == 4, 'empty.larger(fork) is fork'); - expect(bigger!.put('via-bigger', payload, wv.EntryKind.persistent, null), - 'put through larger(other)'); - expect(forked.count() == 5, 'larger(other) aliases other'); - bigger.dispose(); - final own = forked.larger(empty); - expect(own != null && own.count() == 5, 'fork.larger(empty) is fork'); - own!.dispose(); - own.dispose(); - - // describe(): a record carrying the object itself plus an optional object. - final info = store.describe('primary', null); - expect(info.label == 'primary', 'describe label'); - expect(info.count == 4, 'describe count (got ${info.count})'); - expect(info.mirror == null, 'describe mirror absent'); - expect(info.store.count() == 4, 'describe store usable'); - expect(info.store.put('via-info', payload, wv.EntryKind.persistent, null), - 'put through the record object'); - expect(store.count() == 5, 'record object aliases the store'); - final mirrored = store.describe('with-mirror', empty); - expect(mirrored.mirror != null, 'describe mirror present'); - expect(mirrored.mirror!.count() == 0, 'mirror is the empty store'); - expect(mirrored.mirror!.put('m', payload, wv.EntryKind.persistent, null), - 'put through mirror'); - expect(empty.count() == 1, 'mirror aliases the empty store'); - expect(mirrored.store.count() == 5 && mirrored.count == 5, - 'mirrored describe snapshot'); - - // openMany(): a list of objects as a return; the error path is typed. - final many = wv.Store.openMany(['/tmp/many-a', '/tmp/many-b']); - expect(many.length == 2, 'openMany returns 2 stores'); - expect(many[0].count() == 0 && many[1].count() == 0, 'fresh stores'); - expect(many[0].put('m0', payload, wv.EntryKind.persistent, null), 'put m0'); - expect(many[0].count() == 1 && many[1].count() == 0, - 'openMany stores are distinct'); - expect(wv.Store.openMany([]).isEmpty, 'openMany empty list'); - try { - wv.Store.openMany(['/tmp/ok', '']); - throw StateError('expected IoException from openMany'); - } on wv.IoException catch (e) { - expect(e.code == 1004, 'openMany IoError code'); + + abandon(); + await collectUntil(() => live(2) == 0, 'an abandoned iterator is released'); + + final sys = s.entries('sys.').toList(); + expect( + sys.length == 1 && sys[0].key == 'sys.x' && text(sys[0].value) == 'xx', + 'entries(sys.)', + ); + + // partition: objects, created as they're pulled. + const prefixes = ['user.', 'sys.', 'none.']; + const counts = [2, 1, 0]; + var i = 0; + for (final part in s.partition(prefixes)) { + expect(!sameStore(part, s), 'partition yields new stores'); + expect( + part.count() == counts[i] && part.path() == prefixes[i], + 'partition $i', + ); + part.dispose(); + i++; } + expect(i == 3, 'three partitions'); + s.dispose(); +} + +Future listeners() async { + final base = liveCallbacks(); + final s = Store.open('/listen'); + final l = RecordingListener(skip: 'quiet'); + final id = s.subscribe(l); + expect( + id > 0 && s.listenerCount() == 1 && liveCallbacks() == base + 1, + 'subscribe', + ); + + // Void methods are delivered on the event loop, after the call returns. + s.putText('a', '1'); + await settle(() => l.puts.length == 1, 'Put v1 delivered'); + var put = l.puts.last; + expect(put.entry.version == 1 && !put.replaced, 'Put v1'); + s.putText('a', '2'); + await settle(() => l.puts.length == 2, 'Put v2 delivered'); + put = l.puts.last; + expect( + put.entry.version == 2 && put.replaced && put.entry.key == 'a', + 'Put v2', + ); + s.putText('quiet', 'x'); // accepts() said no + expect(s.delete('a'), 'delete a'); + await settle(() => l.changes.length == 3, 'Removed delivered'); + expect(l.puts.length == 2, 'the filter skipped quiet'); + expect( + l.changes.last == ChangeRemoved('a', false), + 'Removed(a) (got ${l.changes.last})', + ); + + // An expired read removes the entry and says so. + s.putText('short', 'x', EntryKind.volatile, 1); + s.tick(1); + expectThrows(() => s.get('short'), 'get(short)'); + await settle(() => l.changes.length == 5, 'Removed(short) delivered'); + expect( + l.changes.last == ChangeRemoved('short', true), + 'Removed(short, expired)', + ); + + expect(s.clear() == 1, 'clear leaves quiet'); // "quiet" was left + await settle(() => l.changes.length == 6, 'Cleared delivered'); + expect(l.changes.last == ChangeCleared(1), 'Cleared(1)'); + + // Unsubscribing releases the listener once. + expect( + s.unsubscribe(id) && liveCallbacks() == base, + 'unsubscribe releases the listener', + ); + expect(!s.unsubscribe(id) && s.listenerCount() == 0, 'unsubscribe twice'); + + // A listener that fails is detached (and released); the put succeeds. + final failing = RecordingListener(failOn: 'boom'); + s.subscribe(failing); + s.putText('fine', '1'); + s.putText('boom', '1'); + expect( + s.count() == 2 && s.listenerCount() == 0, + 'a failing listener is detached', + ); + expect(liveCallbacks() == base, 'the failing listener was released'); + await settle(() => failing.puts.isNotEmpty, 'the failing listener saw fine'); + expect( + failing.puts.single.entry.key == 'fine', + 'the failing listener saw only fine', + ); + + // Disposing the store releases the listeners it still holds. + s.subscribe(RecordingListener()); + s.subscribe(RecordingListener()); + expect( + s.listenerCount() == 2 && liveCallbacks() == base + 2, + 'two listeners', + ); + s.dispose(); + expect(liveCallbacks() == base, 'disposing the store released its listeners'); +} + +void policies() { + final base = liveCallbacks(); + final s = Store.open('/policy'); + final other = Store.open('/other'); + final p = TestPolicy(other.share()); + s.setPolicy(p); + expect(s.hasPolicy() && liveCallbacks() == base + 1, 'setPolicy'); + + // admit's record return is what's stored (its key and version aside). + final a = s.putText('a', '1', EntryKind.volatile); + expect( + a.key == 'a' && a.version == 1 && a.kind == EntryKind.encrypted, + 'admitted entry', + ); + expect(listEquals(a.tags, ['admitted']), "admit's rewrite"); + + // route: the object parameter and object return redirect a write. + s.putText('b/x', '2', EntryKind.volatile); + expect(s.count() == 1 && other.count() == 1, 'route redirected b/x'); + + // A typed error from the throwing callback reaches the caller with its + // code, message, and payload. + final rejected = expectThrows( + () => s.putText('secret', '3'), + 'put(secret)', + ); + expect( + rejected.code == 1005 && rejected.message == 'secrets are not stored', + 'Rejected code and message (got ${rejected.code}: ${rejected.message})', + ); + expect( + rejected.key == 'secret' && rejected.reason == 'no secrets', + 'Rejected payload', + ); + + // Anything else arrives as -4 with the consumer's message. + final boom = expectThrows( + () => s.putText('boom', '4'), + 'put(boom)', + ); + expect( + boom is! KvException && + boom.code == -4 && + boom.message == 'policy exploded', + '-4 (got ${boom.code}: ${boom.message})', + ); + expect( + s.count() == 1 && other.count() == 1 && p.admitted == 4, + 'failed puts changed nothing', + ); + + // Replacing the policy releases the old one; null removes it. + final replacement = TestPolicy(other.share()); + s.setPolicy(replacement); + expect( + liveCallbacks() == base + 1, + 'replacing the policy released the old one', + ); + s.setPolicy(null); + expect( + !s.hasPolicy() && liveCallbacks() == base, + 'setPolicy(null) released it', + ); + s.putText('secret', 'now allowed'); + expect(s.count() == 2, 'no policy, no veto'); + + p.other.dispose(); + replacement.other.dispose(); + other.dispose(); + s.dispose(); +} + +void loaders() { + final s = Store.open('/load'); + final base = liveCallbacks(); + + // No loader (a null optional callback): a miss is none. + expect(s.getOrLoad('k', null) == null, 'no loader'); - // totalCount(): objects inside a list parameter and inside an optional - // record parameter (each written as a cloned token, so the wrappers stay - // valid afterward). The record is built in Dart from live wrappers. - expect(wv.Store.totalCount([], null) == 0, 'totalCount of nothing'); - expect(wv.Store.totalCount(many, null) == 1, 'totalCount of many'); - expect(wv.Store.totalCount([store, forked, empty], null) == 11, - 'totalCount of three stores'); - expect(wv.Store.totalCount([], info) == 5, - 'totalCount of a record only'); - expect(wv.Store.totalCount(many, mirrored) == 6, 'totalCount list + record'); - final built = wv.StoreInfo( - label: 'built', store: forked, mirror: empty, count: -1); - expect(wv.Store.totalCount(many, built) == 6, - 'totalCount with a Dart-built record holding objects'); - final builtNoMirror = wv.StoreInfo(label: 'b2', store: many[0], count: 0); - expect(wv.Store.totalCount([many[0]], builtNoMirror) == 2, - 'totalCount counts the same object twice'); - expect(store.count() == 5 && forked.count() == 5 && empty.count() == 1, - 'wrappers untouched after being encoded into buffers'); - - // Encoding a disposed wrapper into a buffer is a Dart-side StateError, not - // a native fault. - final gone = store.share(); - gone.dispose(); - try { - wv.Store.totalCount([gone], null); - throw StateError('expected StateError encoding a disposed wrapper'); - } on StateError catch (e) { - expect(e.message.contains('dispose'), 'disposed wrapper in a buffer'); + // load's bytes are stored, tagged with the loader's name. + final loaded = s.getOrLoad('k', TestLoader()); + expect(loaded != null && text(loaded.value) == 'loaded:k', 'loaded value'); + expect( + loaded!.kind == EntryKind.volatile && + loaded.metadata['source'] == 'dart-loader', + 'loaded entry (got $loaded)', + ); + expect(liveCallbacks() == base, 'a loader is released after the call'); + expect(s.count() == 1, 'the loaded entry is stored'); + // A hit doesn't consult the loader. + expect(s.getOrLoad('k', TestLoader())?.version == 1, 'a hit'); + + // The fallback store (an optional object return) is consulted first. + final backup = Store.open('/backup'); + backup.putText('fb', 'from backup'); + final fb = s.getOrLoad('fb', TestLoader(backup)); + expect( + fb != null && + text(fb.value) == 'from backup' && + fb.kind == EntryKind.persistent, + 'fallback entry', + ); + backup.dispose(); + + // KeyNotFound for this key: the producer decoded the payload and answers + // none. + expect( + s.getOrLoad('missing', TestLoader()) == null, + 'KeyNotFound for the same key is none', + ); + // KeyNotFound for another key: passed through, payload intact. + final other = expectThrows( + () => s.getOrLoad('elsewhere', TestLoader()), + 'getOrLoad(elsewhere)', + ); + expect( + other.key == 'other' && other.message == 'not in the loader', + 'KeyNotFound(other) passed through', + ); + // Any other failure is -4. + final broken = expectThrows( + () => s.getOrLoad('broken', TestLoader()), + 'getOrLoad(broken)', + ); + expect( + broken.code == -4 && broken.message == 'loader is broken', + '-4 (got ${broken.code}: ${broken.message})', + ); + expect(liveCallbacks() == base, 'every loader was released'); + s.dispose(); +} + +Future asyncCalls() async { + final s = Store.open('/async-calls'); + s.putText('old1', 'x', EntryKind.volatile, 1); + s.putText('old2', 'x', EntryKind.volatile, 1); + s.putText('keep', 'x'); + s.tick(5); + + // compact runs on a producer thread (no listener: see the header). + final token = CancelToken(); + expect(await s.compact(0, cancelToken: token) == 2, 'compact(0) removed 2'); + expect(s.count() == 1, 'one entry left'); + expect(await s.compact(5) == 0, 'compact(5) removed nothing'); + + // Cancel mid-pause: the call completes at once with a cancellation, and + // the background pause notices the token and stops. + final cancel = CancelToken(); + var done = false; + final pending = s + .compact(60000, cancelToken: cancel) + .whenComplete(() => done = true); + await Future.delayed(const Duration(milliseconds: 20)); + expect(!done, 'compact(60000) is pausing'); + expect(Store.activeJobs() >= 1, 'the pause is running'); + final started = Stopwatch()..start(); + cancel.cancel(); + await expectThrowsAsync( + () => pending, + 'a cancelled compact', + ); + expect( + started.elapsed < const Duration(seconds: 2), + 'cancellation is prompt', + ); + await settle( + () => Store.activeJobs() == 0, + 'the cancelled pause stopped cooperatively', + timeout: const Duration(seconds: 2), + ); + + // getMany: an async list of optional records, launched concurrently. + final results = await Future.wait([ + for (var i = 0; i < 32; i++) s.getMany(['keep', 'gone', 'keep']), + ]); + for (final got in results) { + expect( + got.length == 3 && + got[0]?.key == 'keep' && + got[1] == null && + got[2]?.key == 'keep', + 'getMany', + ); } - // Release every object from the graph; the store itself must survive all - // of them because it still holds its own reference. + // The nested module's async function: objects in a list in, a record out. + final other = Store.open('/other'); + other.putText('a', '123'); + final st = await summarizeAll([s, other]); + expect( + st == Stats(entries: 2, bytes: 4, byKind: {EntryKind.persistent: 2}), + 'summarizeAll (got $st)', + ); + other.dispose(); + s.dispose(); +} + +void objectGraph() { + final s0 = Store.open('/graph'); + s0.putText('k', 'v'); + + // share(): the same object; the original can go. + final s = s0.share(); + expect(sameStore(s, s0), 'share() is the same object'); + s0.dispose(); + expect(s.count() == 1, 'alive through the shared reference'); + + // fork(): a distinct object with a copy of the entries. + final fork = s.fork(); + expect( + !sameStore(fork, s) && fork.count() == 1 && fork.path() == '/graph', + 'fork', + ); + fork.putText('k2', 'v'); + expect(fork.count() == 2 && s.count() == 1, 'fork is independent'); + + // larger(): `Store?` in and out. + final empty = Store.open('/empty'); + expect(empty.larger(null) == null, 'larger(null) on an empty store'); + final larger = empty.larger(fork)!; + expect(sameStore(larger, fork), 'larger(fork) is fork'); + larger.dispose(); + final self = s.larger(null)!; + expect(sameStore(self, s), 'larger(null) is self'); + self.dispose(); + + // describe(): a record whose fields carry objects. + final info = s.describe('main', fork); + expect(info.label == 'main' && info.count == 1, 'describe label and count'); + expect( + sameStore(info.store, s) && sameStore(info.mirror!, fork), + 'describe objects', + ); + expect(info.mirror?.count() == 2, 'the mirror is live'); + + // openMany(): a list of objects; one bad path fails the whole call. + final many = Store.openMany(['/a', '/b']); + expect( + listEquals([for (final m in many) m.path()], ['/a', '/b']), + 'openMany paths', + ); + expectThrows( + () => Store.openMany(['/a', '']), + 'openMany with an empty path', + ); + + // byLabel(): records with objects in, a map with object values out. + final named = Store.byLabel([ + info, + StoreInfo(label: 'first', store: many[0], count: 0), + ]); + expect( + named.length == 2 && + named.containsKey('main') && + named.containsKey('first'), + 'byLabel keys', + ); + expect(sameStore(named['main']!, s), 'byLabel main'); + expect(sameStore(named['first']!, many[0]), 'byLabel first'); + + // totalCount(): a list, a map, and an optional record, all carrying + // objects (each written as a fresh reference the producer adopts). + many[0].putText('m', '1'); + final stores = [many[0], many[1], fork]; + expect(Store.totalCount(stores, named, info) == 6, 'totalCount with extra'); + expect( + Store.totalCount(stores, named, null) == 5, + 'totalCount without extra', + ); + + // Everything is still intact; release each reference once. + expect( + s.count() == 1 && fork.count() == 2 && many[0].count() == 1, + 'still usable', + ); + for (final m in named.values) { + m.dispose(); + } info.store.dispose(); - mirrored.store.dispose(); - mirrored.mirror!.dispose(); - mirrored.mirror!.dispose(); - for (final s in many) { - s.dispose(); + info.mirror?.dispose(); + for (final m in many) { + m.dispose(); } - forked.dispose(); empty.dispose(); - expect(store.count() == 5, 'store alive after releasing the graph'); - - // Async: an immediately-expired entry gives compact 3 bytes to reclaim; the - // Future settles via a NativeCallable.listener message from the producer's - // worker thread. - expect(store.put('doomed', payload, wv.EntryKind.volatile, 0), 'put doomed'); - final reclaimed = await store.compact(); - expect(reclaimed == 3, 'compact reclaimed 3 bytes (got $reclaimed)'); - expect(store.count() == 5, 'live entries untouched by compact'); - expect((await store.compact()) == 0, 'second compact reclaims nothing'); - - store.clear(); - expect(store.count() == 0, 'clear empties the store'); - - store.dispose(); - store.dispose(); - try { - store.count(); - throw StateError('expected StateError after dispose'); - } on StateError catch (_) {} - - await expectCompactCancellation(); + fork.dispose(); + s.dispose(); + expectThrows(() => s.count(), "a disposed wrapper can't be used"); } -/// `compact` is cancellable: a token cancelled before launch completes the -/// call with CancelledException, and an unused token changes nothing. -Future expectCompactCancellation() async { - final store = wv.Store.open('/tmp/conformance-kvstore-dart-cancel'); - final idle = wv.CancelToken(); - expect(await store.compact(cancelToken: idle) == 0, 'compact with a token'); - final token = wv.CancelToken()..cancel(); - final cancelled = await expectThrowsAsync( - () => store.compact(cancelToken: token), 'cancelled compact'); - expect(cancelled.code == -5 && cancelled is! wv.KvException, - 'cancellation is not a domain error'); - store.dispose(); +void statsAndReport() { + final s = Store.open('/stats'); + s.putText('b', '12'); + s.putText('a', '1'); + s.putText('a', '123'); + s.putText('c', 'x', EntryKind.encrypted); + + // kv.stats: the parent's Store as a parameter, the parent's error domain. + final st = summarize(s, null); + expect( + st == + Stats( + entries: 3, + bytes: 6, + byKind: {EntryKind.persistent: 2, EntryKind.encrypted: 1}, + ), + 'summarize (got $st)', + ); + expectKeyNotFound('q', () => summarize(s, 'q')); + + // report: the sibling root shares the Entry record. + final lines = renderReport(s.entries(null).toList()); + expect( + listEquals(lines, [ + 'a: 3 bytes, Persistent, v2', + 'b: 2 bytes, Persistent', + 'c: 1 bytes, Encrypted', + ]), + 'renderReport (got $lines)', + ); + final nothing = expectThrows( + () => renderReport(const []), + 'renderReport([])', + ); + expect( + nothing.code == 2001 && nothing.message == 'nothing to report', + 'NothingToReport', + ); + s.dispose(); } Future main() async { - await run(); + await constructors(); + basics(); + await iterators(); + await listeners(); + policies(); + loaders(); + await asyncCalls(); + objectGraph(); + statsAndReport(); + await expectNoLeaks('kvstore'); print('dart/kvstore: OK'); } diff --git a/conformance/dart/run.sh b/conformance/dart/run.sh index e89553dd..9c7fa5da 100755 --- a/conformance/dart/run.sh +++ b/conformance/dart/run.sh @@ -5,6 +5,7 @@ # consumer against one sample and must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools dart dart # Run one consumer as its own small app that depends on the generated # package through a `path:` dependency, exactly as a user would. The library @@ -21,8 +22,9 @@ dart_consumer() { name: ${pkg}_conformance publish_to: none environment: - sdk: '>=3.12.0 <4.0.0' + sdk: '>=3.10.0 <4.0.0' dependencies: + ffi: ^2.1.0 $pkg: path: $pkgdir EOF @@ -34,17 +36,11 @@ EOF } dart_calculator() { dart_consumer calculator calculator.dart; } -dart_contacts() { dart_consumer contacts contacts.dart; } -dart_events() { dart_consumer events events.dart; } -dart_kvstore() { dart_consumer kvstore kvstore.dart; } -dart_async_demo() { dart_consumer async-demo async_demo.dart; } -dart_codec() { dart_consumer codec codec.dart; } +dart_codec() { dart_consumer codec codec.dart; } +dart_kvstore() { dart_consumer kvstore kvstore.dart; } lane dart-calculator dart_calculator -lane dart-contacts dart_contacts -lane dart-events dart_events -lane dart-kvstore dart_kvstore -lane dart-async-demo dart_async_demo lane dart-codec dart_codec +lane dart-kvstore dart_kvstore finish_lanes diff --git a/conformance/dart/support.dart b/conformance/dart/support.dart index f18cabe6..319ebe60 100644 --- a/conformance/dart/support.dart +++ b/conformance/dart/support.dart @@ -21,7 +21,9 @@ T expectThrows(void Function() body, String msg) { /// Expects the future from [body] to fail with a [T] and returns it. Future expectThrowsAsync( - Future Function() body, String msg) async { + Future Function() body, + String msg, +) async { try { await body(); } on T catch (e) { @@ -30,14 +32,12 @@ Future expectThrowsAsync( throw StateError('assertion failed: $msg (nothing thrown)'); } -/// The value or error [future] completes with. Attaching it right away -/// keeps an early error from counting as unhandled. -Future outcome(Future future) => - future.then((v) => v, onError: (Object e) => e); - /// Turns the event loop until [done] holds, failing after [timeout]. -Future settle(bool Function() done, String msg, - {Duration timeout = const Duration(seconds: 10)}) async { +Future settle( + bool Function() done, + String msg, { + Duration timeout = const Duration(seconds: 10), +}) async { final watch = Stopwatch()..start(); while (!done()) { if (watch.elapsed > timeout) throw StateError('timed out: $msg'); @@ -45,22 +45,31 @@ Future settle(bool Function() done, String msg, } } -/// Checks the producer's leak counters through raw `dart:ffi`: live objects, -/// callbacks, iterators, cancel tokens, and returned allocations must all -/// drop to zero. Garbage is churned between checks so the GC runs and the -/// bindings' finalizers release whatever was left to them. -Future expectNoLeaks(String prefix) async { +/// The producer's `{prefix}_debug_live`, bound through raw `dart:ffi`. +int Function(int) debugLive(String prefix) { final env = '${prefix.toUpperCase()}_LIBRARY'; final lib = DynamicLibrary.open(Platform.environment[env]!); - final live = lib.lookupFunction( - '${prefix}_debug_live'); + return lib.lookupFunction( + '${prefix}_debug_live', + ); +} + +/// Checks the producer's leak counters: the build counts at all, and live +/// objects, callbacks, iterators, cancel tokens, and byte runs all drop to +/// zero. Garbage is churned between checks so the GC runs and the bindings' +/// finalizers release whatever was left to them. +Future expectNoLeaks(String prefix) async { + final live = debugLive(prefix); + expect(live(-1) == 1, 'the sample counts live resources'); List counts() => [for (var k = 0; k <= 4; k++) live(k)]; final watch = Stopwatch()..start(); var junk = []; while (counts().any((c) => c != 0)) { if (watch.elapsed > const Duration(seconds: 20)) { - throw StateError('leaked native resources ' - '[objects, callbacks, iterators, tokens, allocations]: ${counts()}'); + throw StateError( + 'leaked native resources ' + '[objects, callbacks, iterators, tokens, byte runs]: ${counts()}', + ); } for (var i = 0; i < 20000; i++) { junk.add(List.filled(64, i)); diff --git a/conformance/dotnet/AsyncDemo.cs b/conformance/dotnet/AsyncDemo.cs deleted file mode 100644 index ae2c2cc3..00000000 --- a/conformance/dotnet/AsyncDemo.cs +++ /dev/null @@ -1,129 +0,0 @@ -// Conformance consumer: async-demo sample, .NET target. -// -// Drives the Task-based async surface of the generated AsyncDemo project: -// RunTask completing from a producer thread with a decoded TaskResult, the -// typed TaskException for an empty name, the buffered list round trip -// through RunBatch, the direct scalar RunNTasks, and real cancellation of -// Wait through a CancellationToken (cancelled mid-flight, cancelled before -// launch, and left to time out). Ends by asserting the producer's leak -// counters are zero. - -using System; -using System.Diagnostics; -using System.Linq; -using System.Threading; -using System.Threading.Tasks; -using AsyncDemo; - -internal static class Program -{ - static void Expect(bool cond, string msg) - { - if (!cond) - { - Console.Error.WriteLine($"assertion failed: {msg}"); - Environment.Exit(1); - } - } - - static async Task Run() - { - // Async record return. - var result = await Tasks.RunTask("alpha"); - Expect(result.Id > 0, "RunTask assigns an id"); - Expect(result.Value == "completed: alpha", $"RunTask value (got {result.Value})"); - Expect(result.Success, "RunTask success flag"); - - // Typed async error. - try - { - await Tasks.RunTask(""); - Expect(false, "expected TaskException for empty name"); - } - catch (TaskException e) - { - Expect(e.Code == TaskException.InvalidName, $"InvalidName code == 1 (got {e.Code})"); - Expect(e is NativeException, "typed exception extends NativeException"); - } - - // Buffered list-of-records both ways. - var batch = await Tasks.RunBatch(new[] { "a", "b", "c" }); - Expect( - batch.Select(r => r.Value).SequenceEqual( - new[] { "completed: a", "completed: b", "completed: c" }), - $"RunBatch values (got [{string.Join(", ", batch.Select(r => r.Value))}])"); - Expect((await Tasks.RunBatch(new string[0])).Length == 0, "RunBatch of nothing"); - - // Direct scalar, many in flight at once. - var many = await Task.WhenAll(Enumerable.Range(0, 64).Select(i => Tasks.RunNTasks(i))); - Expect(many.SequenceEqual(Enumerable.Range(0, 64)), "RunNTasks echoes n under load"); - - // A cancellable call left alone completes after its timeout. - Expect(await Tasks.Wait(20) == 20, "Wait(20) completes with 20"); - using (var unused = new CancellationTokenSource()) - { - Expect(await Tasks.Wait(10, unused.Token) == 10, "Wait with an untouched token"); - } - - // Cancelling mid-flight cancels the native call: the task completes - // as canceled long before the 60 s timeout. - var clock = Stopwatch.StartNew(); - using (var cts = new CancellationTokenSource()) - { - var pending = Tasks.Wait(60_000, cts.Token); - await Task.Delay(50); - Expect(!pending.IsCompleted, "Wait(60 s) is still pending"); - cts.Cancel(); - try - { - await pending; - Expect(false, "expected OperationCanceledException"); - } - catch (OperationCanceledException) - { - } - Expect(pending.IsCanceled, "the task is canceled"); - } - Expect(clock.Elapsed < TimeSpan.FromSeconds(10), $"cancel didn't wait for the timeout ({clock.Elapsed})"); - - // A token cancelled before the call never launches it. - using (var cts = new CancellationTokenSource()) - { - cts.Cancel(); - var task = Tasks.Wait(60_000, cts.Token); - Expect(task.IsCanceled, "pre-cancelled Wait is canceled immediately"); - } - - // Cancelling after a timer through CancelAfter, several at once. - using (var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(30))) - { - var waits = Enumerable.Range(0, 8).Select(_ => Tasks.Wait(60_000, cts.Token)).ToArray(); - foreach (var w in waits) - { - try - { - await w; - Expect(false, "expected cancellation"); - } - catch (OperationCanceledException) - { - } - } - } - - // Every spawned task body has finished. - for (var i = 0; i < 100 && Tasks.ActiveCallbacks() != 0; i++) - { - await Task.Delay(20); - } - Expect(Tasks.ActiveCallbacks() == 0, "ActiveCallbacks settles to zero"); - } - - static int Main() - { - Run().GetAwaiter().GetResult(); - LeakCheck.AssertNoLeaks("async_demo"); - Console.WriteLine("dotnet/async-demo: OK"); - return 0; - } -} diff --git a/conformance/dotnet/Calculator.cs b/conformance/dotnet/Calculator.cs new file mode 100644 index 00000000..ac5e0cce --- /dev/null +++ b/conformance/dotnet/Calculator.cs @@ -0,0 +1,59 @@ +// Conformance consumer: calculator sample, .NET target. +// +// Loading the generated project checks the ABI revision and the +// `calculator` contract table (the first call runs NativeMethods' static +// constructor). Covers wrapping addition, division rounding toward zero +// (including i32::MIN / -1), the typed CalcException.DivisionByZero (code 1, +// message "division by zero", no fields), and strings crossing as UTF-8 +// (pointer, length) pairs: empty, non-ASCII, astral, and an interior NUL. +// Ends by asserting the producer's leak counters are zero. +// +// The namespace is `Calculator` and the module is also `calculator`, so the +// free-function class is `Calculator.Calculator`; `using static` imports +// its statics. + +using System; +using Calculator; +using static Calculator.Calculator; + +internal static class Program +{ + static void Expect(bool cond, string msg) + { + if (!cond) + { + Console.Error.WriteLine($"assertion failed: {msg}"); + Environment.Exit(1); + } + } + + static int Main() + { + Expect(Add(2, 3) == 5, "add(2, 3)"); + Expect(Add(int.MaxValue, 1) == int.MinValue, "add wraps"); + + Expect(Divide(10, 2) == 5, "divide(10, 2)"); + Expect(Divide(-7, 2) == -3, "divide rounds toward zero"); + Expect(Divide(int.MinValue, -1) == int.MinValue, "divide(MIN, -1) wraps"); + try + { + Divide(1, 0); + Expect(false, "divide(1, 0) throws"); + } + catch (CalcException.DivisionByZero e) + { + Expect(e.Code == 1 && e.Code == CalcException.DivisionByZero.ErrorCode, $"code (got {e.Code})"); + Expect(e.Message == "division by zero", $"message (got '{e.Message}')"); + Expect(e is CalcException && e is NativeException, "hierarchy"); + } + + Expect(Greet("World") == "Hello, World!", "greet"); + Expect(Greet("") == "Hello, !", "greet empty"); + Expect(Greet("Wörld 🦀") == "Hello, Wörld 🦀!", "greet non-ASCII and astral"); + Expect(Greet("a\0b") == "Hello, a\0b!", "greet interior NUL"); + + LeakCheck.AssertNoLeaks("calculator"); + Console.WriteLine("dotnet/calculator: OK"); + return 0; + } +} diff --git a/conformance/dotnet/Codec.cs b/conformance/dotnet/Codec.cs index 05002bbf..e83c945d 100644 --- a/conformance/dotnet/Codec.cs +++ b/conformance/dotnet/Codec.cs @@ -1,31 +1,30 @@ -// Conformance consumer: codec sample, .NET target. +// Conformance consumer: codec sample, .NET target (the wire oracle). // -// Round-trips every value-buffer wire shape through the generated Codec -// project against the producer's oracle: the -// canonical Scalars and Composite fixtures are decoded and checked field by -// field (producer encodes, consumer decodes), handed back through Verify* -// (consumer encodes, producer decodes) and Roundtrip* (both), then rebuilt -// from scratch with edge values (empty strings, lists, and maps; unicode; -// long/ulong extremes; NaN, the infinities, and negative zero) and -// round-tripped again. Also covers every Shape variant of the rich enum, the -// typed CodecException (Mismatch=1), and the Holder record carrying Token -// objects in a required field, an optional, and a list: each encoded token is -// a fresh clone, PrimaryOf returns a wrapper to the same native object as the -// holder's Primary, and every wrapper is disposed (double Dispose safe, -// ObjectDisposedException after). Strings with interior NULs cross the -// function boundary intact, since strings are (pointer, length) pairs. -// Wrapper equality is native object identity. Ends by asserting the -// producer's leak counters are zero. +// Loading the generated project checks the ABI revision and the `codec` +// contract table. Then, against the producer's shared vectors: // -// The namespace is `Codec` and the module is also `codec`, so the -// free-function class is `Codec.Codec`; `using static` imports its statics. +// 1. every vector decodes (producer encodes, consumer decodes), checks +// against its own index and not its neighbor's (consumer encodes, +// producer decodes), and primitive vectors echo back identically; +// 2. vectors built from C# literals check against their named entries +// (catching symmetric codec bugs), including NaN, signed zero, maps in +// another insertion order, and a holder of consumer-made tokens; +// 3. spot checks on decoded values (i64 past 2^53, the f32 subnormal's +// bits, astral text, every scalar minimum, the canonical composite); +// 4. the typed CodecException.OutOfRange with its index and count fields; +// 5. an undeclared Color value is rejected by the producer (-3, which a +// non-throwing call raises as NativeBugException); +// 6. objects: token values, sum_holder twice over the same holder, +// primary_of returning the same native object, identity (not value) +// in same_primary, one wrapper in several slots; +// +// and ends by asserting the producer's leak counters are zero. using System; using System.Collections.Generic; using System.Linq; -using System.Runtime.CompilerServices; using Codec; -using static Codec.Codec; +using C = Codec.Codec; internal static class Program { @@ -48,376 +47,250 @@ static bool SameBits(float a, float b) return BitConverter.SingleToInt32Bits(a) == BitConverter.SingleToInt32Bits(b); } - static Scalars CanonicalScalars() + // Look a vector up by name, so the table can grow without breaking us. + static uint Find(uint n, string name) { - return new Scalars(-8, 200, -16_000, 60_000, -2_000_000_000, 4_000_000_000U, - -9_007_199_254_740_993L, ulong.MaxValue, 1.5f, -2.25e100, true, Color.Blue); + for (uint i = 0; i < n; i++) + { + if (C.VectorName(i) == name) + { + return i; + } + } + Expect(false, $"no vector named '{name}'"); + return 0; } - static void CheckScalars(Scalars s, string what) + static void Release(Vector v) { - Expect(s.I8Value == -8, $"{what}: i8 (got {s.I8Value})"); - Expect(s.U8Value == 200, $"{what}: u8 (got {s.U8Value})"); - Expect(s.I16Value == -16_000, $"{what}: i16 (got {s.I16Value})"); - Expect(s.U16Value == 60_000, $"{what}: u16 (got {s.U16Value})"); - Expect(s.I32Value == -2_000_000_000, $"{what}: i32 (got {s.I32Value})"); - Expect(s.U32Value == 4_000_000_000U, $"{what}: u32 (got {s.U32Value})"); - Expect(s.I64Value == -9_007_199_254_740_993L, $"{what}: i64 (got {s.I64Value})"); - Expect(s.U64Value == ulong.MaxValue, $"{what}: u64 (got {s.U64Value})"); - Expect(s.F32Value == 1.5f, $"{what}: f32 (got {s.F32Value})"); - Expect(s.F64Value == -2.25e100, $"{what}: f64 (got {s.F64Value})"); - Expect(s.Flag, $"{what}: flag"); - Expect(s.Color == Color.Blue, $"{what}: color (got {s.Color})"); + if (v is Vector.Objects o) + { + Release(o.Value); + } } - static void ExpectScalarsEqual(Scalars a, Scalars b, string what) + static void Release(Holder h) { - Expect(a.I8Value == b.I8Value && a.U8Value == b.U8Value - && a.I16Value == b.I16Value && a.U16Value == b.U16Value - && a.I32Value == b.I32Value && a.U32Value == b.U32Value - && a.I64Value == b.I64Value && a.U64Value == b.U64Value - && SameBits(a.F32Value, b.F32Value) && SameBits(a.F64Value, b.F64Value) - && a.Flag == b.Flag && a.Color == b.Color, - $"{what}: scalars equal"); + h.Primary.Dispose(); + h.Spare?.Dispose(); + foreach (var t in h.Many) + { + t.Dispose(); + } + foreach (var t in h.ByName.Values) + { + t.Dispose(); + } } - static void ExpectShapesEqual(Shape a, Shape b, string what) + // A primitive vector's value comes back unchanged from its echo_*. + static void Echo(Vector v, string name) { - Expect(a.GetType() == b.GetType(), $"{what}: same variant ({a.GetType().Name} vs {b.GetType().Name})"); - switch (a) + switch (v) { - case Shape.Empty _: - break; - case Shape.Circle ca: - Expect(SameBits(ca.Radius, ((Shape.Circle)b).Radius), $"{what}: circle radius"); - break; - case Shape.Rect ra: - var rb = (Shape.Rect)b; - Expect(SameBits(ra.Width, rb.Width) && SameBits(ra.Height, rb.Height), $"{what}: rect dims"); - break; - case Shape.Labeled la: - var lb = (Shape.Labeled)b; - Expect(la.Label == lb.Label && la.Count == lb.Count, $"{what}: labeled payload"); - break; - case Shape.Nested na: - var nb = (Shape.Nested)b; - ExpectScalarsEqual(na.Inner, nb.Inner, what + ": nested inner"); - Expect(na.Note == nb.Note, $"{what}: nested note"); - break; - default: - Expect(false, $"{what}: unknown variant"); - break; + case Vector.I8 x: Expect(C.EchoI8(x.Value) == x.Value, name); break; + case Vector.U8 x: Expect(C.EchoU8(x.Value) == x.Value, name); break; + case Vector.I16 x: Expect(C.EchoI16(x.Value) == x.Value, name); break; + case Vector.U16 x: Expect(C.EchoU16(x.Value) == x.Value, name); break; + case Vector.I32 x: Expect(C.EchoI32(x.Value) == x.Value, name); break; + case Vector.U32 x: Expect(C.EchoU32(x.Value) == x.Value, name); break; + case Vector.I64 x: Expect(C.EchoI64(x.Value) == x.Value, name); break; + case Vector.U64 x: Expect(C.EchoU64(x.Value) == x.Value, name); break; + case Vector.F32 x: Expect(SameBits(C.EchoF32(x.Value), x.Value), name); break; + case Vector.F64 x: Expect(SameBits(C.EchoF64(x.Value), x.Value), name); break; + case Vector.Flag x: Expect(C.EchoBool(x.Value) == x.Value, name); break; + case Vector.Text x: Expect(C.EchoText(x.Value) == x.Value, name); break; + case Vector.Blob x: Expect(C.EchoBlob(x.Value).SequenceEqual(x.Value), name); break; + case Vector.Hue x: Expect(C.EchoColor(x.Value) == x.Value, name); break; } } - static void ExpectCompositesEqual(Composite a, Composite b, string what) + static void EveryVector(uint n) { - Expect(a.Name == b.Name, $"{what}: name"); - Expect(a.Blob.SequenceEqual(b.Blob), $"{what}: blob"); - Expect(a.SomeI64 == b.SomeI64, $"{what}: some_i64"); - Expect(a.NoneI64 == b.NoneI64, $"{what}: none_i64"); - Expect(a.SomeText == b.SomeText, $"{what}: some_text"); - Expect(a.Names.SequenceEqual(b.Names), $"{what}: names"); - Expect(a.Matrix.Length == b.Matrix.Length - && a.Matrix.Zip(b.Matrix, (x, y) => x.SequenceEqual(y)).All(eq => eq), - $"{what}: matrix"); - Expect(a.Empty.Length == b.Empty.Length - && a.Empty.Zip(b.Empty, (x, y) => SameBits(x, y)).All(eq => eq), - $"{what}: empty list"); - Expect(a.ByName.Count == b.ByName.Count - && a.ByName.All(kv => b.ByName.TryGetValue(kv.Key, out var v) && v == kv.Value), - $"{what}: by_name"); - Expect(a.ById.Count == b.ById.Count && a.ById.Keys.All(k => b.ById.ContainsKey(k)), - $"{what}: by_id keys"); - foreach (var kv in a.ById) - { - ExpectScalarsEqual(kv.Value, b.ById[kv.Key], $"{what}: by_id[{kv.Key}]"); - } - ExpectScalarsEqual(a.Scalars, b.Scalars, what + ": scalars"); - ExpectShapesEqual(a.Shape, b.Shape, what + ": shape"); - Expect(a.Shapes.Length == b.Shapes.Length, $"{what}: shapes length"); - for (int i = 0; i < a.Shapes.Length; i++) + for (uint i = 0; i < n; i++) { - ExpectShapesEqual(a.Shapes[i], b.Shapes[i], $"{what}: shapes[{i}]"); + var name = C.VectorName(i); + var v = C.Vector(i); + if (!C.CheckVector(i, v)) + { + Expect(false, $"vector {i} ({name}) did not round-trip; producer saw {C.DescribeVector(v)}"); + } + Expect(!C.CheckVector((i + 1) % n, v), $"vector {i} ({name}) matches its neighbor"); + Echo(v, $"echo {name}"); + Release(v); } - Expect((a.MaybeShape == null) == (b.MaybeShape == null), $"{what}: maybe_shape presence"); - if (a.MaybeShape != null) + } + + static Scalars CanonicalScalars(ushort u16 = 60_000) + { + return new Scalars(-8, 200, -16_000, u16, -2_000_000_000, 4_000_000_000U, + -9_007_199_254_740_993L, ulong.MaxValue, 1.5f, -2.25e100, true, Color.Blue); + } + + static void LiteralVectors(uint n) + { + Expect(C.CheckVector(Find(n, "scalars canonical"), new Vector.AllScalars(CanonicalScalars())), + "literal scalars canonical"); + Expect(!C.CheckVector(Find(n, "scalars canonical"), new Vector.AllScalars(CanonicalScalars(60_001))), + "changed scalars don't match"); + Expect(C.CheckVector(Find(n, "shape labeled"), new Vector.Figure(new Shape.Labeled("tag", 3))), + "literal shape labeled"); + Expect(C.CheckVector(Find(n, "string interior nul"), new Vector.Text("nul\0inside\0")), + "literal interior nul"); + // Any NaN matches the NaN vector; zero keeps its sign. + var nan = BitConverter.Int64BitsToDouble(0x7ff8000000000001L); + Expect(C.CheckVector(Find(n, "f64 nan"), new Vector.F64(nan)), "any NaN matches"); + Expect(C.CheckVector(Find(n, "f64 -0"), new Vector.F64(-0.0)), "-0.0 matches -0"); + Expect(!C.CheckVector(Find(n, "f64 -0"), new Vector.F64(0.0)), "+0.0 doesn't match -0"); + Expect(C.CheckVector(Find(n, "u64 max"), new Vector.U64(ulong.MaxValue)), "literal u64 max"); + Expect(C.CheckVector(Find(n, "enum infrared"), new Vector.Hue(Color.Infrared)), "literal infrared"); + Expect(C.CheckVector(Find(n, "optional zero"), new Vector.MaybeI64(0)), "Some(0)"); + Expect(C.CheckVector(Find(n, "optional absent"), new Vector.MaybeI64(null)), "None"); + Expect(!C.CheckVector(Find(n, "optional zero"), new Vector.MaybeI64(null)), "None isn't Some(0)"); + // A map's entry order doesn't matter on the wire. + var counts = new Dictionary { ["x"] = 0, ["héllo"] = -1, [""] = long.MaxValue }; + Expect(C.CheckVector(Find(n, "map of strings"), new Vector.Counts(counts)), "map in another order"); + Expect(C.CheckVector(Find(n, "blank"), new Vector.Blank()), "literal blank"); + using (var lone = new Token(-1)) { - ExpectShapesEqual(a.MaybeShape, b.MaybeShape, what + ": maybe_shape"); + var sparse = new Holder(lone, null, Array.Empty(), new Dictionary()); + Expect(C.CheckVector(Find(n, "objects sparse"), new Vector.Objects(sparse)), "consumer-made sparse holder"); } - Expect((a.MaybeList == null) == (b.MaybeList == null) - && (a.MaybeList == null || a.MaybeList.SequenceEqual(b.MaybeList)), - $"{what}: maybe_list"); - Expect(a.Sparse.SequenceEqual(b.Sparse), $"{what}: sparse"); - Expect(a.Colors.SequenceEqual(b.Colors), $"{what}: colors"); } - static int Main() + static T Fetch(uint n, string name) where T : Vector { - Run(); - LeakCheck.AssertNoLeaks("codec"); - Console.WriteLine("dotnet/codec: OK"); - return 0; + var v = C.Vector(Find(n, name)); + Expect(v is T, $"{name} is a {typeof(T).Name} (got {v.GetType().Name})"); + return (T)v; } - [MethodImpl(MethodImplOptions.NoInlining)] - static void Run() + static void SpotChecks(uint n) { - // --- Scalars: producer encodes, consumer decodes; then back. --- - var scalars = SampleScalars(); - CheckScalars(scalars, "sample_scalars"); - Expect(VerifyScalars(scalars), "verify_scalars accepts the decoded sample"); - CheckScalars(RoundtripScalars(scalars), "roundtrip_scalars"); - var mine = CanonicalScalars(); - Expect(VerifyScalars(mine), "verify_scalars accepts a consumer-built canonical value"); + Expect(Fetch(n, "i64 past 2^53").Value == -9_007_199_254_740_993L, "i64 past 2^53"); + Expect(BitConverter.SingleToInt32Bits(Fetch(n, "f32 min subnormal").Value) == 1, + "f32 min subnormal bits"); + Expect(Fetch(n, "string astral").Value == "🦀 crab 😀", "string astral"); + + var m = Fetch(n, "scalars minimum").Value; + Expect(m.I8Value == sbyte.MinValue && m.I16Value == short.MinValue && m.I32Value == int.MinValue, + "signed minimums"); + Expect(m.I64Value == long.MinValue && m.U64Value == 0 && m.U8Value == 0 && m.U16Value == 0 && m.U32Value == 0, + "i64 and unsigned minimums"); + Expect(float.IsNegativeInfinity(m.F32Value) && double.IsNaN(m.F64Value), "float minimums"); + Expect(m.Color == Color.Infrared && !m.Flag, "color and flag minimums"); + + var c = Fetch(n, "composite canonical").Value; + Expect(c.Name == "héllo wörld ✓", $"composite name (got {c.Name})"); + Expect(c.Blob.Length == 6 && c.Blob[5] == 255, "composite blob"); + Expect(c.SomeI64 == long.MinValue && c.NoneI64 == null, "composite optionals"); + Expect(c.SomeText == "", "composite some_text is present and empty"); + Expect(c.Names.Length == 3 && c.Names[1] == "", "composite names"); + Expect(c.Matrix.Length == 3 && c.Matrix[1].Length == 0 && c.Matrix[2][0] == -4, "composite matrix"); + Expect(c.Floats.Length == 6 && double.IsNaN(c.Floats[0]) && double.IsNegative(c.Floats[3]), + "composite floats"); + Expect(c.ByName.Count == 4 && c.ById.Count == 3 && c.ByColor.Count == 2 && c.Flags.Count == 2, + "composite map sizes"); + Expect(c.Scalars.U32Value == 4_000_000_000U, "composite scalars"); + Expect(c.Shape is Shape.Labeled { Count: 3 }, "composite shape"); + Expect(c.Shapes.Length == 6 && c.Shapes[5] is Shape.Nested { Note: null }, "composite shapes"); + Expect(c.MaybeShape is Shape.Nested, "composite maybe_shape"); + Expect(c.MaybeList != null && c.MaybeList.Length == 2, "composite maybe_list"); + Expect(c.Sparse.Length == 3 && c.Sparse[1] == null && c.Sparse[0] == true, "composite sparse"); + Expect(c.Colors.Length == 4 && c.Colors[3] == Color.Infrared, "composite colors"); + } - // A one-field mismatch reports the typed domain error. - var wrong = new Scalars(-8, 200, -16_000, 60_000, -2_000_000_000, 4_000_000_000U, - -9_007_199_254_740_993L, ulong.MaxValue, 1.5f, -2.25e100, false, Color.Blue); + static void OutOfRange(uint n) + { try { - VerifyScalars(wrong); - Expect(false, "expected CodecException for a mismatched Scalars"); + C.Vector(n); + Expect(false, "vector(n) throws"); } - catch (CodecException e) + catch (CodecException.OutOfRange e) { - Expect(e.Code == CodecException.Mismatch, $"Mismatch code == 1 (got {e.Code})"); - Expect(e.Code == 1, "Mismatch constant is 1"); - Expect(e is NativeException, "typed exception extends the brand exception"); + Expect(e.Code == CodecException.OutOfRange.ErrorCode && e.Code == 1, $"code (got {e.Code})"); + Expect(e.Index == n && e.Count == n, $"payload (got {e.Index}, {e.Count})"); + Expect(e.Message == $"vector {n} is out of range (count {n})", $"message (got '{e.Message}')"); } - - // Scalars at the extremes, including float edge values whose bit - // patterns must survive both directions. - var extremes = new Scalars(sbyte.MinValue, byte.MaxValue, short.MinValue, ushort.MaxValue, - int.MinValue, uint.MaxValue, long.MinValue, ulong.MaxValue, - float.NegativeInfinity, -0.0, false, Color.Red); - var extremesBack = RoundtripScalars(extremes); - ExpectScalarsEqual(extremes, extremesBack, "extremes"); - Expect(BitConverter.DoubleToInt64Bits(extremesBack.F64Value) == long.MinValue, "negative zero keeps its sign bit"); - var nans = new Scalars(sbyte.MaxValue, 0, short.MaxValue, 0, int.MaxValue, 0, long.MaxValue, 0, - float.NaN, double.NaN, true, Color.Green); - var nansBack = RoundtripScalars(nans); - Expect(float.IsNaN(nansBack.F32Value) && double.IsNaN(nansBack.F64Value), "NaN survives both floats"); - Expect(nansBack.I64Value == long.MaxValue && nansBack.I8Value == sbyte.MaxValue, "max extremes"); - var infs = new Scalars(0, 0, 0, 0, 0, 0, 0, 0, float.PositiveInfinity, double.PositiveInfinity, false, Color.Red); - var infsBack = RoundtripScalars(infs); - Expect(float.IsPositiveInfinity(infsBack.F32Value) && double.IsPositiveInfinity(infsBack.F64Value), - "+inf survives both floats"); - Expect(double.IsNegativeInfinity(RoundtripScalars(new Scalars(0, 0, 0, 0, 0, 0, 0, 0, 0f, - double.NegativeInfinity, false, Color.Red)).F64Value), "-inf survives f64"); - - // --- Composite: every nested wire shape. --- - var composite = SampleComposite(); - Expect(composite.Name == "héllo wörld ✓", $"name (got '{composite.Name}')"); - Expect(composite.Blob.SequenceEqual(new byte[] { 0, 1, 2, 253, 254, 255 }), "blob"); - Expect(composite.SomeI64 == long.MinValue, $"some_i64 == i64::MIN (got {composite.SomeI64})"); - Expect(composite.NoneI64 == null, "none_i64 absent"); - Expect(composite.SomeText != null && composite.SomeText == "", "some_text is a present empty string"); - Expect(composite.Names.SequenceEqual(new[] { "a", "", "ccc" }), "names"); - Expect(composite.Matrix.Length == 3 - && composite.Matrix[0].SequenceEqual(new[] { 1, 2, 3 }) - && composite.Matrix[1].Length == 0 - && composite.Matrix[2].SequenceEqual(new[] { -4 }), - "matrix"); - Expect(composite.Empty.Length == 0, "empty list"); - Expect(composite.ByName.Count == 3 - && composite.ByName["one"] == 1 && composite.ByName["two"] == 2 && composite.ByName["neg"] == -3, - "by_name"); - Expect(composite.ById.Count == 2 && composite.ById.ContainsKey(-1) && composite.ById.ContainsKey(42), "by_id keys"); - CheckScalars(composite.ById[-1], "by_id[-1]"); - Expect(!composite.ById[42].Flag && composite.ById[42].U64Value == ulong.MaxValue, "by_id[42] differs only in flag"); - CheckScalars(composite.Scalars, "composite.scalars"); - Expect(composite.Shape is Shape.Labeled sl && sl.Label == "tag" && sl.Count == 3, "shape is Labeled(tag, 3)"); - Expect(composite.Shapes.Length == 5, "five shapes"); - Expect(composite.Shapes[0] is Shape.Empty, "shapes[0] Empty"); - Expect(composite.Shapes[1] is Shape.Circle c1 && c1.Radius == 2.5, "shapes[1] Circle(2.5)"); - Expect(composite.Shapes[2] is Shape.Rect r2 && r2.Width == 1.0f && r2.Height == 0.5f, "shapes[2] Rect(1, 0.5)"); - Expect(composite.Shapes[3] is Shape.Labeled l3 && l3.Label == "" && l3.Count == -1, "shapes[3] Labeled('', -1)"); - Expect(composite.Shapes[4] is Shape.Nested n4 && n4.Note == "n", "shapes[4] Nested(note n)"); - CheckScalars(((Shape.Nested)composite.Shapes[4]).Inner, "shapes[4].inner"); - Expect(composite.MaybeShape is Shape.Nested mn && mn.Note == null, "maybe_shape Nested(note absent)"); - CheckScalars(((Shape.Nested)composite.MaybeShape).Inner, "maybe_shape.inner"); - Expect(composite.MaybeList != null && composite.MaybeList.SequenceEqual(new byte[] { 9, 8 }), "maybe_list"); - Expect(composite.Sparse.SequenceEqual(new bool?[] { true, null, false }), "sparse"); - Expect(composite.Colors.SequenceEqual(new[] { Color.Red, Color.Green, Color.Blue }), "colors"); - - Expect(VerifyComposite(composite), "verify_composite accepts the decoded sample"); - ExpectCompositesEqual(composite, RoundtripComposite(composite), "roundtrip_composite"); - var described = DescribeComposite(composite); - Expect(described.Contains("héllo wörld ✓") && described.Contains("Labeled"), - $"describe_composite renders the value (got '{described}')"); - - // The same composite with one nested change fails verification. - var tweaked = new Composite(composite.Name, composite.Blob, composite.SomeI64, composite.NoneI64, - composite.SomeText, composite.Names, composite.Matrix, composite.Empty, composite.ByName, - composite.ById, composite.Scalars, composite.Shape, composite.Shapes, composite.MaybeShape, - composite.MaybeList, new bool?[] { true, true, false }, composite.Colors); try { - VerifyComposite(tweaked); - Expect(false, "expected CodecException for a tweaked Composite"); + C.VectorName(n + 5); + Expect(false, "vector_name(n + 5) throws"); } - catch (CodecException e) + catch (CodecException.OutOfRange e) { - Expect(e.Code == CodecException.Mismatch, "tweaked composite reports Mismatch"); + Expect(e.Index == n + 5 && e.Count == n, $"vector_name payload (got {e.Index}, {e.Count})"); } + Expect(!C.CheckVector(n, new Vector.Blank()), "check_vector past the end is false"); + } - // A consumer-built composite with edge values: empty everything, - // unicode, extremes, NaN, the infinities, and negative zero. - var edge = new Composite( - "", - new byte[0], - long.MaxValue, - null, - "日本語 🎉 \u0000 tail", - new string[0], - new int[][] { new int[0], new[] { int.MinValue, int.MaxValue } }, - new[] { double.NaN, double.PositiveInfinity, double.NegativeInfinity, -0.0, double.Epsilon, double.MaxValue }, - new Dictionary(), - new Dictionary { [int.MinValue] = extremes, [0] = nans, [int.MaxValue] = infs }, - extremes, - new Shape.Empty(), - new Shape[0], - null, - null, - new bool?[0], - new Color[0]); - var edgeBack = RoundtripComposite(edge); - ExpectCompositesEqual(edge, edgeBack, "edge composite"); - Expect(edgeBack.SomeText == "日本語 🎉 \u0000 tail", "embedded NUL and astral characters survive inside a buffer"); - Expect(double.IsNaN(edgeBack.Empty[0]) && double.IsPositiveInfinity(edgeBack.Empty[1]) - && double.IsNegativeInfinity(edgeBack.Empty[2]) - && BitConverter.DoubleToInt64Bits(edgeBack.Empty[3]) == long.MinValue, - "float edge values in a list"); - Expect(edgeBack.MaybeShape == null && edgeBack.MaybeList == null && edgeBack.NoneI64 == null, - "absent optionals stay absent"); - - // Rich enum variants on their own and in a list. - Shape[] variants = - { - new Shape.Empty(), - new Shape.Circle(double.NaN), - new Shape.Circle(-0.0), - new Shape.Rect(float.MaxValue, float.Epsilon), - new Shape.Labeled("", int.MinValue), - new Shape.Labeled("ünïcödé", int.MaxValue), - new Shape.Nested(extremes, null), - new Shape.Nested(nans, ""), - }; - foreach (var v in variants) + static void Malformed() + { + // C# can name an undeclared enum value; the producer rejects it, and + // a non-throwing call traps with the marshalling code. + try { - ExpectShapesEqual(v, RoundtripShape(v), "roundtrip_shape " + v.GetType().Name); + C.EchoColor((Color)3); + Expect(false, "echo_color(3) is rejected"); } - var listBack = RoundtripShapes(variants); - Expect(listBack.Length == variants.Length, "roundtrip_shapes length"); - for (int i = 0; i < variants.Length; i++) + catch (NativeBugException e) { - ExpectShapesEqual(variants[i], listBack[i], $"roundtrip_shapes[{i}]"); + Expect(e.Code == NativeException.MarshalErrorCode, $"code -3 (got {e.Code})"); + Expect(e.Message.Contains("-3"), $"the message names the code (got '{e.Message}')"); } - Expect(RoundtripShapes(new Shape[0]).Length == 0, "roundtrip_shapes empty"); - Expect(DescribeShape(new Shape.Empty()) == "Empty", "describe_shape Empty"); - Expect(DescribeShape(new Shape.Circle(2.5)) == "Circle { radius: 2.5 }", - $"describe_shape Circle (got '{DescribeShape(new Shape.Circle(2.5))}')"); - Expect(DescribeShape(new Shape.Labeled("x", 7)) == "Labeled { label: \"x\", count: 7 }", - "describe_shape Labeled"); - - // Top-level optionals, maps, strings, bytes, and direct scalars. - Expect(RoundtripOptI64(null) == null, "opt_i64 absent"); - Expect(RoundtripOptI64(long.MinValue) == long.MinValue, "opt_i64 i64::MIN"); - Expect(RoundtripOptI64(0) == 0, "opt_i64 zero is present"); - Expect(RoundtripMap(new Dictionary()).Count == 0, "empty map"); - var map = RoundtripMap(new Dictionary { [""] = long.MinValue, ["k"] = 0, ["ключ"] = long.MaxValue }); - Expect(map.Count == 3 && map[""] == long.MinValue && map["k"] == 0 && map["ключ"] == long.MaxValue, "map contents"); - Expect(RoundtripString("") == "", "empty string"); - Expect(RoundtripString("héllo wörld ✓ 🎉") == "héllo wörld ✓ 🎉", "unicode string"); - var nul = RoundtripString("a\0b\0"); - Expect(nul == "a\0b\0" && nul.Length == 4, "interior NULs survive the function boundary"); - Expect(RoundtripString("\0") == "\0", "a lone NUL survives"); - Expect(RoundtripBytes(new byte[0]).Length == 0, "empty bytes"); - ReadOnlySpan span = stackalloc byte[] { 7, 0, 9 }; - Expect(RoundtripBytes(span).SequenceEqual(new byte[] { 7, 0, 9 }), "bytes from a span"); - Expect(RoundtripBytes(new byte[] { 0, 127, 128, 255 }).SequenceEqual(new byte[] { 0, 127, 128, 255 }), "bytes"); - Expect(RoundtripI64(long.MinValue) == long.MinValue, "i64::MIN direct"); - Expect(RoundtripI64(long.MaxValue) == long.MaxValue, "i64::MAX direct"); - Expect(RoundtripU64(ulong.MaxValue) == ulong.MaxValue, "u64::MAX direct"); - Expect(RoundtripU64(1UL << 63) == (1UL << 63), "2^63 direct"); - Expect(double.IsNaN(RoundtripF64(double.NaN)), "f64 NaN direct"); - Expect(double.IsPositiveInfinity(RoundtripF64(double.PositiveInfinity)), "f64 +inf direct"); - Expect(double.IsNegativeInfinity(RoundtripF64(double.NegativeInfinity)), "f64 -inf direct"); - Expect(BitConverter.DoubleToInt64Bits(RoundtripF64(-0.0)) == long.MinValue, "f64 -0.0 direct"); - Expect(RoundtripF64(double.MaxValue) == double.MaxValue, "f64 max direct"); - Expect(RoundtripBool(true) && !RoundtripBool(false), "bool direct"); - Expect(RoundtripColor(Color.Blue) == Color.Blue && (int)RoundtripColor(Color.Blue) == 7, "enum direct"); - Expect(RoundtripColor(Color.Red) == Color.Red, "enum zero direct"); - - // --- Objects inside buffers. --- - var holder = MakeHolder(10, true); - Expect(holder.Primary.Value() == 10, "holder.primary"); - Expect(holder.Spare != null && holder.Spare.Value() == 11, "holder.spare"); - Expect(holder.Many.Select(t => t.Value()).SequenceEqual(new long[] { 12, 13, 14 }), "holder.many"); - Expect(holder.Many.Select(t => t).Distinct().Count() == 3, "holder.many are distinct objects"); - Expect(SumHolder(holder) == 10 + 11 + 12 + 13 + 14, "sum_holder"); - // Encoding cloned each token, so the holder is still fully usable. - Expect(SumHolder(holder) == 60, "sum_holder again after re-encoding"); - Expect(holder.Primary.Value() == 10, "primary alive after encoding twice"); + } - var primary = PrimaryOf(holder); - Expect(!ReferenceEquals(primary, holder.Primary), "primary_of is a new wrapper"); - Expect(primary.Equals(holder.Primary), "primary_of wraps the same native object"); - Expect(primary.Value() == 10, "primary_of value"); - Expect(SamePrimary(holder, holder), "same_primary with itself"); - var other = MakeHolder(10, true); - Expect(!SamePrimary(holder, other), "same_primary across holders"); - var rebuilt = new Holder(primary, null, new Token[0]); - Expect(SamePrimary(holder, rebuilt), "same_primary through a consumer-built holder"); - Expect(SumHolder(rebuilt) == 10, "sum_holder of a consumer-built holder"); + static void Objects(uint n) + { + var full = Fetch(n, "objects full").Value; + Expect(full.Primary.Value() == 10 && full.Spare != null && full.Spare.Value() == 11, "primary and spare"); + Expect(full.Many.Length == 3 && full.Many[2].Value() == long.MinValue, "many"); + Expect(full.ByName.Count == 2 && full.ByName["b"].Value() == 21, "by_name"); + // Each encoding mints fresh references, so the holder can be sent twice. + var expected = unchecked(10L + 11 + 12 + 13 + 20 + 21 + long.MinValue); + Expect(C.SumHolder(full) == expected && C.SumHolder(full) == expected, "sum_holder twice"); - var noSpare = MakeHolder(0, false); - Expect(noSpare.Spare == null, "make_holder without spare"); - Expect(SumHolder(noSpare) == 0 + 2 + 3 + 4, "sum_holder without spare"); + // primary_of returns the very same native object. + using var p = C.PrimaryOf(full); + Expect(p.Equals(full.Primary), "primary_of is the same object"); + var none = new Dictionary(); + Expect(C.SamePrimary(full, new Holder(p, null, Array.Empty(), none)), "same_primary by identity"); + using var twin = new Token(10); + Expect(!C.SamePrimary(full, new Holder(twin, null, Array.Empty(), none)), + "an equal value isn't the same object"); - // Consumer-created tokens travel into buffers too. - var t1 = new Token(1); - var t2 = new Token(2); - var t3 = new Token(long.MinValue); - var own = new Holder(t1, t2, new[] { t3, t1 }); - Expect(SumHolder(own) == 1 + 2 + long.MinValue + 1, "sum_holder over consumer tokens"); - var ownPrimary = PrimaryOf(own); - Expect(ownPrimary.Equals(t1) && ownPrimary.Value() == 1, "primary_of consumer token"); + // One consumer-made wrapper in every slot. + using var minus = new Token(-4); + var mine = new Holder(twin, twin, new[] { twin, twin, minus }, new Dictionary { ["k"] = twin }); + Expect(C.SumHolder(mine) == 10 * 5 - 4, "holder of consumer tokens"); + Expect(twin.Value() == 10, "the wrapper is still usable"); - // Reference counting: dropping one wrapper leaves the others valid; - // double Dispose is a no-op; a disposed wrapper throws. - holder.Primary.Dispose(); - holder.Primary.Dispose(); - Expect(primary.Value() == 10, "primary_of wrapper outlives holder.Primary"); - try - { - holder.Primary.Value(); - Expect(false, "expected ObjectDisposedException"); - } - catch (ObjectDisposedException) - { - } + Release(full); try { - SumHolder(holder); - Expect(false, "encoding a disposed token must throw before calling the producer"); + full.Primary.Value(); + Expect(false, "a disposed wrapper throws"); } catch (ObjectDisposedException) { } - ownPrimary.Dispose(); - Expect(t1.Value() == 1, "t1 alive after disposing its twin"); + } - foreach (var t in new[] { primary, holder.Spare, other.Primary, other.Spare, noSpare.Primary, t1, t2, t3 } - .Concat(holder.Many).Concat(other.Many).Concat(noSpare.Many)) - { - t.Dispose(); - t.Dispose(); - } - using (var scoped = new Token(99)) - { - Expect(scoped.Value() == 99, "scoped token"); - } + static int Main() + { + var n = C.VectorCount(); + Expect(n >= 60, $"vector_count >= 60 (got {n})"); + + EveryVector(n); + LiteralVectors(n); + SpotChecks(n); + OutOfRange(n); + Malformed(); + Objects(n); + + LeakCheck.AssertNoLeaks("codec"); + Console.WriteLine($"dotnet/codec: OK ({n} vectors)"); + return 0; } } diff --git a/conformance/dotnet/Contacts.cs b/conformance/dotnet/Contacts.cs deleted file mode 100644 index f91b50d2..00000000 --- a/conformance/dotnet/Contacts.cs +++ /dev/null @@ -1,143 +0,0 @@ -// Conformance consumer: contacts sample, .NET target. -// -// Drives the generated Contacts project: the ContactBook interface (real -// constructor, instance methods, Dispose), the ContactType enum, the plain -// Contact record decoded from value buffers, optional strings, list returns, -// and the typed ContactsException (InvalidName=1, NotFound=2). The nested -// contacts.groups module (ContactsGroups) takes and returns the parent's -// enum and interface and reports the parent's error domain; the sibling -// directory root (Directory) shares the Contact record and has its own -// DirectoryException. Ends by asserting the producer's leak counters are -// zero. - -using System; -using System.Linq; -using System.Runtime.CompilerServices; -using Contacts; - -internal static class Program -{ - static void Expect(bool cond, string msg) - { - if (!cond) - { - Console.Error.WriteLine($"assertion failed: {msg}"); - Environment.Exit(1); - } - } - - [MethodImpl(MethodImplOptions.NoInlining)] - static void Run() - { - using (var book = new ContactBook()) - { - var alice = book.Add("Alice", "Smith", "alice@example.com", ContactType.Work); - Expect(alice.Id > 0, "alice id positive"); - Expect(alice.FirstName == "Alice" && alice.LastName == "Smith", "names"); - Expect(alice.Email == "alice@example.com", "email"); - Expect(alice.ContactType == ContactType.Work, "contact type"); - - var bob = book.Add("Bob", "Jones", null, ContactType.Personal); - Expect(bob.Email == null, "bob email null"); - var carol = book.Add("Carol", "Adams", null, ContactType.Work); - - Expect(book.Get(alice.Id).FirstName == "Alice", "get returns alice"); - Expect(book.Count() == 3, "count == 3"); - var names = book.List().Select(p => p.FirstName).OrderBy(s => s).ToArray(); - Expect(names.SequenceEqual(new[] { "Alice", "Bob", "Carol" }), "list names"); - - // The nested module uses the parent's enum and interface. - Expect(ContactsGroups.CountOfType(book, ContactType.Work) == 2, "two work contacts"); - Expect(ContactsGroups.CountOfType(book, ContactType.Other) == 0, "no other contacts"); - Expect(ContactsGroups.DominantType(book) == ContactType.Work, "work dominates"); - using (var work = ContactsGroups.SplitByType(book, ContactType.Work)) - { - Expect(work.Count() == 2, "split copied the work contacts"); - Expect(!work.Equals(book), "split is a new book"); - Expect(work.List().All(c => c.ContactType == ContactType.Work), "split holds only work"); - } - Expect(ContactsGroups.FirstOfType(book, ContactType.Personal).FirstName == "Bob", - "first personal contact"); - try - { - ContactsGroups.FirstOfType(book, ContactType.Other); - Expect(false, "expected ContactsException from the nested module"); - } - catch (ContactsException e) - { - Expect(e.Code == ContactsException.NotFound, "nested module reports the parent's NotFound"); - } - - // The sibling root shares the Contact record. - var card = Directory.Card(alice); - Expect(card.DisplayName == "Smith, Alice", $"card display name (got {card.DisplayName})"); - Expect(card.Initials == "AS" && card.HasEmail, "card initials and email"); - Expect(!Directory.Card(bob).HasEmail, "bob has no email"); - var sorted = Directory.Sorted(book.List()); - Expect(sorted.Select(c => c.LastName).SequenceEqual(new[] { "Adams", "Jones", "Smith" }), - "sorted by last name"); - try - { - Directory.Sorted(new Contact[0]); - Expect(false, "expected DirectoryException"); - } - catch (DirectoryException e) - { - Expect(e.Code == DirectoryException.Empty, "Empty code == 1"); - } - - Expect(book.Remove(alice.Id), "remove returns true"); - Expect(!book.Remove(alice.Id), "second remove returns false"); - Expect(book.Count() == 2, "count == 2 after remove"); - - try - { - book.Add("", "Smith", null, ContactType.Personal); - Expect(false, "expected ContactsException for empty name"); - } - catch (ContactsException e) - { - Expect(e.Code == ContactsException.InvalidName, "InvalidName code == 1"); - Expect(e.Message == "name must not be empty", $"InvalidName message (got '{e.Message}')"); - } - try - { - book.Get(9999); - Expect(false, "expected ContactsException for missing contact"); - } - catch (ContactsException e) - { - Expect(e.Code == ContactsException.NotFound, "NotFound code == 2"); - Expect(e is NativeException, "typed exception extends NativeException"); - } - Expect(carol.Id > 0, "carol kept"); - } - - // A disposed book refuses further calls. - var gone = new ContactBook(); - gone.Dispose(); - gone.Dispose(); - try - { - gone.Count(); - Expect(false, "expected ObjectDisposedException"); - } - catch (ObjectDisposedException) - { - } - - // Books left to the garbage collector are released by finalizers. - for (var i = 0; i < 10; i++) - { - new ContactBook().Add("Temp", "Book", null, ContactType.Other); - } - } - - static int Main() - { - Run(); - LeakCheck.AssertNoLeaks("contacts"); - Console.WriteLine("dotnet/contacts: OK"); - return 0; - } -} diff --git a/conformance/dotnet/Events.cs b/conformance/dotnet/Events.cs deleted file mode 100644 index cb81a128..00000000 --- a/conformance/dotnet/Events.cs +++ /dev/null @@ -1,310 +0,0 @@ -// Conformance consumer: events sample, .NET target. -// -// Drives the generated Events project: the `ISubscriber` callback interface -// implemented in C# and adapted to the producer's vtable by the generated -// trampolines (Route, whose producer side returns `Result<_, ForeignError>`, -// steering Publish through the `Delivery` enum; OnMessage decoding the -// buffered `Message` record; OnAttached adopting an `EventBus` handed through -// the callback), the `EventBus` class (constructor, Task-based PublishLater, -// single-use Messages iterator, optional LastMessage, Dispose), the free -// function RouteOnce, subscribers that throw (reported to the caller as -// NativeException with ForeignErrorCode -4 for both the Result-returning -// and the plain callback methods), callbacks fired from producer threads, -// and wrapper equality as native identity. Ends by asserting the producer's -// leak counters are zero. - -using System; -using System.Collections.Generic; -using System.Linq; -using System.Runtime.CompilerServices; -using System.Threading; -using System.Threading.Tasks; -using Events; -using EventsApi = Events.Events; - -internal sealed class TestSubscriber : ISubscriber -{ - private readonly string _skipTopic; - private readonly string _failTopic; - private readonly bool _keepBus; - - public readonly List Routed = new List(); - public readonly List Received = new List(); - public int Attached; - public long AttachedCount = -1; - public EventBus KeptBus; - - public TestSubscriber(string skipTopic, string failTopic, bool keepBus = false) - { - _skipTopic = skipTopic; - _failTopic = failTopic; - _keepBus = keepBus; - } - - public Delivery Route(string topic) - { - Routed.Add(topic); - if (topic == _failTopic) - { - throw new InvalidOperationException("subscriber rejected topic " + topic); - } - if (topic == _skipTopic) - { - return Delivery.Skip; - } - return topic == "stop" ? Delivery.AcceptAndStop : Delivery.Accept; - } - - public long OnMessage(Message message) - { - if (message.Text == "explode") - { - throw new ApplicationException("subscriber exploded on " + message.Text); - } - Received.Add(message); - return Received.Count; - } - - public void OnAttached(EventBus bus) - { - Attached++; - // The reference is ours: the bus is usable here, and releasing it - // must not tear down the caller's live bus. - AttachedCount = bus.SubscriberCount(); - if (_keepBus) - { - KeptBus = bus; - } - else - { - bus.Dispose(); - } - } -} - -internal static class Program -{ - static void Expect(bool cond, string msg) - { - if (!cond) - { - Console.Error.WriteLine($"assertion failed: {msg}"); - Environment.Exit(1); - } - } - - // Subscribes a subscriber nothing else references, so once the producer's - // `free(ctx)` releases the GCHandle the object becomes collectible. - [MethodImpl(MethodImplOptions.NoInlining)] - static WeakReference SubscribeTransient(EventBus bus) - { - var sub = new TestSubscriber("", ""); - bus.Subscribe(sub); - return new WeakReference(sub); - } - - static int Main() - { - Run().GetAwaiter().GetResult(); - LeakCheck.AssertNoLeaks("events"); - Console.WriteLine("dotnet/events: OK"); - return 0; - } - - static async Task Run() - { - var bus = new EventBus(); - Expect(bus.SubscriberCount() == 0, "fresh bus has no subscribers"); - Expect(bus.LastMessage() == null, "fresh bus has no last message"); - Expect(!bus.Messages().Any(), "fresh bus has no messages"); - - // Subscribe: the producer calls OnAttached with a reference-counted - // bus object before retaining the subscriber. - var a = new TestSubscriber("quiet", "", keepBus: true); - var b = new TestSubscriber("", ""); - Expect(bus.Subscribe(a) == 1, "first subscribe returns 1"); - Expect(bus.Subscribe(b) == 2, "second subscribe returns 2"); - Expect(a.Attached == 1 && b.Attached == 1, "OnAttached fired once per subscriber"); - Expect(a.AttachedCount == 0, $"a saw 0 subscribers at attach (got {a.AttachedCount})"); - Expect(b.AttachedCount == 1, $"b saw 1 subscriber at attach (got {b.AttachedCount})"); - Expect(a.KeptBus != null && a.KeptBus.Equals(bus), - "bus handed to OnAttached is the same native object"); - Expect(bus.SubscriberCount() == 2, "subscriber count == 2"); - - // Delivery steers the accepted count: Accept delivers, Skip does not, - // AcceptAndStop delivers and halts later subscribers. - Expect(bus.Publish("news", "hello", new[] { "a", "b" }) == 2, "news accepted by both"); - Expect(bus.Publish("quiet", "psst", new string[0]) == 1, "quiet skipped by a"); - Expect(bus.Publish("stop", "last", new[] { "z" }) == 1, "stop halts after a"); - - Expect(a.Routed.SequenceEqual(new[] { "news", "quiet", "stop" }), - $"a routed every topic (got [{string.Join(", ", a.Routed)}])"); - Expect(b.Routed.SequenceEqual(new[] { "news", "quiet" }), - $"b never routed the stopped topic (got [{string.Join(", ", b.Routed)}])"); - Expect(a.Received.Select(m => m.Text).SequenceEqual(new[] { "hello", "last" }), - "a received hello and last"); - Expect(b.Received.Select(m => m.Text).SequenceEqual(new[] { "hello", "psst" }), - "b received hello and psst"); - - // The buffered Message record decodes intact inside the callback. - var first = a.Received[0]; - Expect(first.Seq == 1, $"first seq == 1 (got {first.Seq})"); - Expect(first.Topic == "news", "first topic"); - Expect(first.Text == "hello", "first text"); - Expect(first.Tags.SequenceEqual(new[] { "a", "b" }), - $"first tags (got [{string.Join(", ", first.Tags)}])"); - Expect(a.Received[1].Seq == 3 && a.Received[1].Tags.SequenceEqual(new[] { "z" }), - "third message decoded for a"); - Expect(b.Received[1].Seq == 2 && b.Received[1].Tags.Length == 0, - "second message decoded for b with empty tags"); - - // Async publish settles the Task from the producer's worker thread. - long later = await bus.PublishLater("async", "later"); - Expect(later == 2, $"PublishLater accepted by both (got {later})"); - Expect(a.Received.Last().Text == "later" && b.Received.Last().Text == "later", - "async publish reached both subscribers"); - - // Iterator and optional record return. - var texts = bus.Messages().ToList(); - Expect(texts.SequenceEqual(new[] { "hello", "psst", "last", "later" }), - $"messages in order (got [{string.Join(", ", texts)}])"); - var last = bus.LastMessage(); - Expect(last != null && last.Seq == 4 && last.Topic == "async" && last.Text == "later", - "LastMessage is the async publish"); - - // Early exit disposes the enumerator (and the native iterator). - foreach (var t in bus.Messages()) - { - Expect(t == "hello", "first streamed message"); - break; - } - var once = bus.Messages(); - once.ToList(); - try - { - once.ToList(); - Expect(false, "second enumeration should throw"); - } - catch (InvalidOperationException) - { - } - - // Free function taking the callback interface without a bus. - var probe = new TestSubscriber("quiet", ""); - Expect(EventsApi.RouteOnce(probe, "quiet") == Delivery.Skip, "RouteOnce skip"); - Expect(EventsApi.RouteOnce(probe, "stop") == Delivery.AcceptAndStop, "RouteOnce accept-and-stop"); - Expect(EventsApi.RouteOnce(probe, "x") == Delivery.Accept, "RouteOnce accept"); - Expect(probe.Routed.SequenceEqual(new[] { "quiet", "stop", "x" }), "RouteOnce routed topics"); - Expect(probe.Attached == 0, "RouteOnce never attaches"); - - // A throwing subscriber surfaces to the caller as the foreign error - // (code -4) with the exception's message, and the bus keeps working. - var c = new TestSubscriber("", "boom"); - Expect(bus.Subscribe(c) == 3, "third subscribe returns 3"); - try - { - bus.Publish("boom", "x", new string[0]); - Expect(false, "expected NativeException from throwing Route"); - } - catch (NativeException e) - { - Expect(e.Code == NativeException.ForeignErrorCode, - $"foreign error code == -4 (got {e.Code})"); - Expect(e.Code == -4, "ForeignErrorCode constant is -4"); - Expect(e.Message.Contains("subscriber rejected topic boom"), - $"foreign error carries the exception message (got '{e.Message}')"); - } - try - { - bus.Publish("ok", "explode", new string[0]); - Expect(false, "expected NativeException from throwing OnMessage"); - } - catch (NativeException e) - { - Expect(e.Code == NativeException.ForeignErrorCode, - $"OnMessage foreign error code == -4 (got {e.Code})"); - Expect(e.Message.Contains("subscriber exploded"), - $"OnMessage foreign error message (got '{e.Message}')"); - } - Expect(bus.Publish("ok", "y", new string[0]) == 3, "bus still delivers after a foreign error"); - try - { - EventsApi.RouteOnce(c, "boom"); - Expect(false, "expected NativeException from RouteOnce"); - } - catch (NativeException e) - { - Expect(e.Code == NativeException.ForeignErrorCode, "RouteOnce foreign error code"); - } - - // ClearSubscribers drops the producer's references; the consumer's - // `free(ctx)` releases the GCHandle so the subscriber is collectible. - var weak = SubscribeTransient(bus); - Expect(bus.SubscriberCount() == 4, "subscriber count == 4"); - bus.ClearSubscribers(); - Expect(bus.SubscriberCount() == 0, "subscriber count == 0 after clear"); - GC.Collect(); - GC.WaitForPendingFinalizers(); - GC.Collect(); - Expect(!weak.IsAlive, "freed subscriber was collected"); - Expect(bus.Publish("nobody", "home", new string[0]) == 0, "no subscribers accept"); - - // Reference counting: the wrapper adopted in OnAttached outlives the - // original wrapper; disposing either releases only its own reference. - var kept = a.KeptBus; - bus.Dispose(); - bus.Dispose(); - try - { - bus.SubscriberCount(); - Expect(false, "expected ObjectDisposedException"); - } - catch (ObjectDisposedException) - { - } - // Every publish is logged, including the two aborted by foreign errors. - var keptCount = kept.Messages().Count(); - Expect(keptCount == 8, $"kept bus still alive with 8 messages (got {keptCount})"); - var keptLast = kept.LastMessage(); - Expect(keptLast != null && keptLast.Text == "home", "kept bus sees the last publish"); - kept.Dispose(); - kept.Dispose(); - - using (var scoped = new EventBus()) - { - Expect(scoped.Publish("t", "u", new string[0]) == 0, "scoped bus publishes"); - } - - // Callbacks arrive on whichever thread calls in, several at once, and - // from the producer's own threads for async publishes. - var counter = new CountingSubscriber(); - using (var busy = new EventBus()) - { - busy.Subscribe(counter); - await Task.WhenAll(Enumerable.Range(0, 8).Select(_ => Task.Run(() => - { - for (var j = 0; j < 50; j++) - { - busy.Publish("t", "m", new string[0]); - } - }))); - Expect(counter.Count == 400, $"400 concurrent deliveries (got {counter.Count})"); - var laters = await Task.WhenAll(Enumerable.Range(0, 16).Select(_ => busy.PublishLater("t", "later"))); - Expect(laters.All(n => n == 1), "every async publish delivered once"); - Expect(counter.Count == 416, $"416 deliveries in all (got {counter.Count})"); - } - } -} - -internal sealed class CountingSubscriber : ISubscriber -{ - private long _count; - - public long Count => Interlocked.Read(ref _count); - - public Delivery Route(string topic) => Delivery.Accept; - - public long OnMessage(Message message) => Interlocked.Increment(ref _count); - - public void OnAttached(EventBus bus) => bus.Dispose(); -} diff --git a/conformance/dotnet/Kvstore.cs b/conformance/dotnet/Kvstore.cs index 00693cb6..c0276c87 100644 --- a/conformance/dotnet/Kvstore.cs +++ b/conformance/dotnet/Kvstore.cs @@ -1,63 +1,174 @@ -// Conformance consumer: kvstore sample, .NET target. +// Conformance consumer: kvstore sample, .NET target (feature-complete). // -// Full-surface drive of the generated Kvstore project: the Store class (static Open -// factory throwing the typed KvException IoError=1004 on an empty path, -// instance methods put/get/delete/list_keys/count/clear, the deprecated -// legacy_put, the cancellable Task-returning Compact, the DefaultCapacity static), the -// optional buffered `Entry?` return decoded into a plain value class, direct -// value-class construction of Entry, the IEnumerable-backed ListKeys iterator, -// the cross-module KvStats.GetStats, the `IEvictionListener` callback -// interface implemented in C# (fired synchronously on delete and on an -// expired read, detached when it returns false, replaced and cleared, and a -// throwing listener surfacing to the caller as NativeException with -// ForeignErrorCode -4), the object graph (Share returning a second wrapper to -// the same native object, Fork, `Store?` both ways through Larger, the -// StoreInfo record carrying Store objects, OpenMany returning a list of -// objects, TotalCount taking a list of objects plus an optional record with -// objects), the typed KvException codes (KeyNotFound=1001, Expired=1002), and -// IDisposable release (safe double Dispose, ObjectDisposedException after). -// Wrapper equality is native object identity. Ends by asserting the -// producer's leak counters are zero. +// Loading the generated project checks the ABI revision and the `kv` and +// `report` contract tables. Then it drives the generated Kvstore project: +// +// * the Store class: the throwing Open factory and the infallible +// constructor, methods, statics, the deprecated Size, records (Entry, +// StoreInfo), the EntryKind enum, maps and optionals, the logical clock; +// * the KvException hierarchy with typed fields (KeyNotFound, Expired, +// StoreFull, InvalidPath, Rejected) and ReportException.NothingToReport; +// * lazy IEnumerable iterators of strings (throwing), records, and +// objects, including one abandoned part-way; +// * three callback interfaces implemented in C#: an IListener (filtered +// by Accepts, told about every Change, detached when it throws, and +// notified from a producer thread during compaction), an IPolicy (a +// record return, a throwing method whose KvException.Rejected reaches +// the Put caller with its fields, any other exception reaching it as +// -4, an object parameter and object return, a null object rejected as +// -3), and an ILoader passed as an optional parameter (string, bytes, +// and optional-object returns; typed errors decoded by the producer or +// passed through); +// * Store objects in every position: parameter, return, optional, list, +// map value, record field, iterator element, async result, and callback +// parameter and return; +// * Task-returning async calls: an async free function returning an +// object (and failing with a typed error), a cancellable method +// cancelled through a CancellationToken mid-pause (an +// OperationCanceledException at once, its background work stopping +// cooperatively), 32 concurrent async lists, and the nested module's +// async function; +// * the nested KvStats class and the sibling Report root. +// +// Callback releases are observed through the producer's live-callback +// counter. Ends by asserting every leak counter is zero. using System; using System.Collections.Generic; using System.Linq; +using System.Text; using System.Threading; using System.Threading.Tasks; using Kvstore; -internal sealed class RecordingListener : IEvictionListener +internal sealed class TestListener : IListener +{ + private readonly string _skip; + private readonly string _failOn; + public int Puts, Removed, Expired, Cleared, OffMainThread; + public uint LastVersion, LastCleared; + public bool LastReplaced; + public string LastKey; + + public TestListener(string skip = null, string failOn = null) + { + _skip = skip; + _failOn = failOn; + } + + public bool Accepts(string key) + { + if (key == _failOn) + { + throw new InvalidOperationException("listener refused"); + } + return key != _skip; + } + + public void OnChange(Change change) + { + if (Environment.CurrentManagedThreadId != Program.MainThread) + { + Interlocked.Increment(ref OffMainThread); + } + switch (change) + { + case Change.Put put: + LastVersion = put.Entry.Version; + LastReplaced = put.Replaced; + LastKey = put.Entry.Key; + Interlocked.Increment(ref Puts); + break; + case Change.Removed removed: + LastKey = removed.Key; + Interlocked.Increment(ref removed.Expired ? ref Expired : ref Removed); + break; + case Change.Cleared cleared: + LastCleared = cleared.Count; + Interlocked.Increment(ref Cleared); + break; + } + } +} + +internal sealed class TestPolicy : IPolicy { - private readonly int _keepAfter; - public readonly List> Evictions = - new List>(); + private readonly Store _other; + public int Admitted; - public RecordingListener(int keepAfter) + public TestPolicy(Store other) { - _keepAfter = keepAfter; + _other = other; } - public bool OnEvict(Entry entry, EvictionReason reason) + public Entry Admit(Entry entry) { - Evictions.Add(new KeyValuePair(entry.Key, reason)); - return Evictions.Count < _keepAfter; + Interlocked.Increment(ref Admitted); + Program.Expect(entry.Version == 0, "the store assigns the version after admission"); + if (entry.Key.StartsWith("secret")) + { + throw new KvException.Rejected(entry.Key, "no secrets", "secrets are not stored"); + } + if (entry.Key.StartsWith("boom")) + { + throw new InvalidOperationException("policy exploded"); + } + // Tag it and store it encrypted (and try to rename it, which the + // store ignores). + return new Entry("renamed", entry.Value, EntryKind.Encrypted, entry.Version, entry.ExpiresAt, + new[] { "admitted" }, entry.Metadata); + } + + public Store Route(string key, Store home) + { + if (key.StartsWith("b/")) + { + return _other; + } + if (key.StartsWith("null/")) + { + return null; // a required object may not be null: -3 + } + return home; } } -internal sealed class ThrowingListener : IEvictionListener +internal sealed class TestLoader : ILoader { - public int Calls; + private readonly Store _fallback; - public bool OnEvict(Entry entry, EvictionReason reason) + public TestLoader(Store fallback = null) { - Calls++; - throw new InvalidOperationException("listener refused " + entry.Key); + _fallback = fallback; + } + + public string Name() => "dotnet-loader"; + + public Store Fallback(string key) => key == "fb" ? _fallback : null; + + public byte[] Load(string key) + { + switch (key) + { + case "missing": + throw new KvException.KeyNotFound("missing", "not in the loader"); // this key: none + case "elsewhere": + throw new KvException.KeyNotFound("other", "not in the loader"); // passed through + case "broken": + throw new InvalidOperationException("loader is broken"); // -4 + default: + return Encoding.UTF8.GetBytes("loaded:" + key); + } } } internal static class Program { - static void Expect(bool cond, string msg) + // The thread synchronous calls notify on (async continuations hop threads, + // so each section records its own). + public static int MainThread; + + public static void Expect(bool cond, string msg) { if (!cond) { @@ -66,310 +177,461 @@ static void Expect(bool cond, string msg) } } - static int Main() + static byte[] Bytes(string s) => Encoding.UTF8.GetBytes(s); + + static string Text(byte[] b) => Encoding.UTF8.GetString(b); + + static ulong LiveCallbacks() => LeakCheck.Live("kvstore", LeakCheck.Callbacks); + + static Entry Put(Store s, string key, string value, EntryKind kind = EntryKind.Persistent, long? ttl = null) { - Run().GetAwaiter().GetResult(); - LeakCheck.AssertNoLeaks("kvstore"); - Console.WriteLine("dotnet/kvstore: OK"); - return 0; + return s.Put(key, Bytes(value), kind, ttl); } - static async Task Run() + // Runs `action`, which must throw `T`, and returns the exception. + static T Throws(Action action, string what) where T : Exception { - // Typed constructor error: an empty path reports KvError::IoError - // through the domain exception. try { - Store.Open(""); - Expect(false, "expected KvException for empty path"); - } - catch (KvException e) - { - Expect(e.Code == KvException.IoError, $"IoError code == 1004 (got {e.Code})"); - Expect(e.Message == "I/O failure", $"IoError default message (got '{e.Message}')"); + action(); } - - Expect(Store.DefaultCapacity() == 1_000_000, "static default capacity"); - - var store = Store.Open("/tmp/conformance-kvstore-dotnet"); - var payload = new byte[] { 1, 2, 3 }; - - // The optional TTL crosses as a buffered `i64?` parameter. - Expect(store.Put("alpha", payload, EntryKind.Persistent, null), "put alpha"); - Expect(store.Put("beta", payload, EntryKind.Volatile, 3600), "put beta with ttl"); - Expect(store.Count() == 2, "count == 2"); - - // Iterator-backed list-of-string return drained through IEnumerable; - // the optional prefix is a buffered `string?`. - var keys = store.ListKeys(null).ToList(); - Expect(keys.SequenceEqual(new[] { "alpha", "beta" }), - $"list_keys sorted values (got [{string.Join(", ", keys)}])"); - Expect(store.ListKeys("al").SequenceEqual(new[] { "alpha" }), "list_keys with prefix"); - Expect(!store.ListKeys("zzz").Any(), "list_keys with unmatched prefix is empty"); - - // Optional buffered `Entry?` return decoded into a plain value class - // covering every complex field type. - var alpha = store.Get("alpha"); - Expect(alpha != null, "get alpha present"); - Expect(alpha.Id > 0, "entry id positive"); - Expect(alpha.Key == "alpha", "entry key"); - Expect(alpha.Value.SequenceEqual(payload), "entry value bytes"); - Expect(alpha.CreatedAt > 0, "entry created_at positive"); - Expect(alpha.ExpiresAt == null, "alpha ExpiresAt null"); - Expect(alpha.Tags.Length == 0, "alpha tags empty"); - Expect(alpha.Metadata.Count == 0, "alpha metadata empty"); - - var beta = store.Get("beta"); - Expect(beta != null && beta.ExpiresAt != null && beta.ExpiresAt > beta.CreatedAt, - "beta ExpiresAt present and after CreatedAt"); - Expect(beta.Id == alpha.Id + 1, "ids are monotonic"); - - // Typed method error: a missing key reports KvError::KeyNotFound. - try + catch (T e) { - store.Get("missing"); - Expect(false, "expected KvException for missing key"); + return e; } - catch (KvException e) + catch (Exception e) { - Expect(e.Code == KvException.KeyNotFound, $"KeyNotFound code == 1001 (got {e.Code})"); - Expect(e.Message == "key not found", $"KeyNotFound message (got '{e.Message}')"); - Expect(e is NativeException, "typed exception extends the brand exception"); + Expect(false, $"{what}: expected {typeof(T).Name}, got {e.GetType().Name}: {e.Message}"); } + Expect(false, $"{what}: expected {typeof(T).Name}, nothing thrown"); + return null; + } - // Entry is a plain value class: non-empty list and map fields live - // directly on the instance. - var built = new Entry( - 7, - "built", - payload, - 1000, - null, - new[] { "hot", "fast" }, - new Dictionary { ["source"] = "test", ["env"] = "prod" }); - Expect(built.Tags.SequenceEqual(new[] { "hot", "fast" }), "built tags"); - Expect(built.Metadata["source"] == "test" && built.Metadata["env"] == "prod", "built metadata"); - Expect(built.ExpiresAt == null, "built ExpiresAt null"); - - // Cross-module call: Stats lives in kv.stats, store is a kv.Store. - var stats = KvStats.GetStats(store); - Expect(stats.TotalEntries == 2, "stats total entries == 2"); - Expect(stats.TotalBytes == 6, "stats total bytes == 6"); - Expect(stats.ExpiredEntries == 0, "stats expired entries == 0"); - - // Eviction listener: a C# implementation of the callback interface. - // Delete fires it synchronously with the decoded Entry and the reason. - var listener = new RecordingListener(keepAfter: 2); - store.SetEvictionListener(listener); - Expect(store.Delete("beta"), "delete beta"); - Expect(listener.Evictions.Count == 1 - && listener.Evictions[0].Key == "beta" - && listener.Evictions[0].Value == EvictionReason.Deleted, - "eviction fired for beta with Deleted"); - Expect(!store.Delete("beta"), "second delete reports missing"); - Expect(listener.Evictions.Count == 1, "no eviction for a missing key"); - - // An already-expired entry is evicted on read with Expired, and the - // typed error still reaches the caller. The listener returned false - // on this second call, so the store detaches it. - Expect(store.Put("expiring", new byte[] { 9 }, EntryKind.Volatile, -1), "put expired entry"); + static async Task ThrowsAsync(Func action, string what) where T : Exception + { try { - store.Get("expiring"); - Expect(false, "expected KvException for expired key"); + await action(); } - catch (KvException e) + catch (T e) { - Expect(e.Code == KvException.Expired, $"Expired code == 1002 (got {e.Code})"); + return e; } - Expect(listener.Evictions.Count == 2 - && listener.Evictions[1].Key == "expiring" - && listener.Evictions[1].Value == EvictionReason.Expired, - "eviction fired for expiring with Expired"); - Expect(store.Put("again", payload, EntryKind.Volatile, null), "put again"); - Expect(store.Delete("again"), "delete again"); - Expect(listener.Evictions.Count == 2, "detached listener is not notified"); - - // A throwing listener surfaces to the caller as the foreign error - // (code -4). Delete has a domain, so the wrapper's typed check falls - // back to the brand exception for the reserved code. - var thrower = new ThrowingListener(); - store.SetEvictionListener(thrower); - Expect(store.Put("doomed", payload, EntryKind.Volatile, null), "put doomed"); - try + catch (Exception e) { - store.Delete("doomed"); - Expect(false, "expected NativeException from throwing listener"); + Expect(false, $"{what}: expected {typeof(T).Name}, got {e.GetType().Name}: {e.Message}"); } - catch (KvException) + Expect(false, $"{what}: expected {typeof(T).Name}, nothing thrown"); + return null; + } + + static void ExpectKeyNotFound(Action action, string key, string what) + { + var e = Throws(action, what); + Expect(e.Code == 1001 && e.Key == key, $"{what}: KeyNotFound {{key: {key}}} (got {e.Key})"); + } + + static async Task Constructors() + { + var invalid = Throws(() => Store.Open(""), "Open(\"\")"); + Expect(invalid.Code == 1004 && invalid.Message == "invalid path", $"InvalidPath (got {invalid.Code} '{invalid.Message}')"); + using (var memory = new Store()) { - Expect(false, "foreign error must not wear the domain exception type"); + Expect(memory.Path() == "memory", "new Store() path"); + Expect(memory.Capacity() == Store.DefaultCapacity() && Store.DefaultCapacity() == 1_000_000, "default capacity"); } - catch (NativeException e) + using (var opened = await Kv.OpenStore("/async")) { - Expect(e.Code == NativeException.ForeignErrorCode, $"foreign error code == -4 (got {e.Code})"); - Expect(e.Message.Contains("listener refused doomed"), - $"foreign error carries the exception message (got '{e.Message}')"); + Expect(opened.Path() == "/async", "OpenStore path"); } - Expect(thrower.Calls == 1, "throwing listener was called once"); - Expect(store.Count() == 1, "doomed was removed before the listener ran"); - - // Replacing and clearing the listener stops delivery. - var replacement = new RecordingListener(keepAfter: int.MaxValue); - store.SetEvictionListener(replacement); - Expect(store.Put("r1", payload, EntryKind.Volatile, null) && store.Delete("r1"), "delete r1"); - Expect(replacement.Evictions.Count == 1 && thrower.Calls == 1, "replacement receives, old does not"); - store.ClearEvictionListener(); - Expect(store.Put("r2", payload, EntryKind.Volatile, null) && store.Delete("r2"), "delete r2"); - Expect(replacement.Evictions.Count == 1, "cleared listener is not notified"); - store.ClearEvictionListener(); - - // Deprecated method still round-trips (volatile put, no TTL). + var rejected = await ThrowsAsync(() => Kv.OpenStore(""), "OpenStore(\"\")"); + Expect(rejected.Message == "invalid path", "async InvalidPath message"); + } + + static void Basics() + { + using var s = Store.Open("/basics"); + + // put returns the stored entry; the version counts puts of the key. + var e = Put(s, "alpha", "one"); + Expect(e.Key == "alpha" && Text(e.Value) == "one" && e.Kind == EntryKind.Persistent && e.Version == 1, + "first put"); + Expect(e.ExpiresAt == null && e.Tags.Length == 0 && e.Metadata.Count == 0, "first put defaults"); + e = Put(s, "alpha", "two", EntryKind.Volatile); + Expect(e.Version == 2 && e.Kind == EntryKind.Volatile, "second put"); + + // get (throwing record) and find (optional record). + Expect(Text(s.Get("alpha").Value) == "two", "get"); + var missing = Throws(() => s.Get("nope"), "get(nope)"); + Expect(missing.Key == "nope" && missing.Message == "key not found: nope", $"KeyNotFound (got '{missing.Message}')"); + Expect(s.Find("alpha")?.Version == 2, "find present"); + Expect(s.Find("nope") == null, "find absent"); + + // TTLs follow the logical clock; an expired get reports when. + Expect(Put(s, "ttl", "x", EntryKind.Volatile, 10).ExpiresAt == 10, "expires_at"); + Expect(s.Now() == 0, "now"); + Expect(s.Tick(9) == 9 && s.Count() == 2, "tick 9"); + Expect(s.Tick(1) == 10 && s.Count() == 1, "tick 1"); + var expired = Throws(() => s.Get("ttl"), "get(ttl)"); + Expect(expired.Key == "ttl" && expired.ExpiredAt == 10, "Expired fields"); + ExpectKeyNotFound(() => s.Get("ttl"), "ttl", "the expired read removed it"); + + // Capacity: a new key past it is StoreFull { capacity }. + s.SetCapacity(1); + Expect(s.Capacity() == 1, "capacity"); + Put(s, "alpha", "three"); // replacing is fine + var full = Throws(() => Put(s, "beta", "b"), "put past capacity"); + Expect(full.Capacity == 1, "StoreFull capacity"); + s.SetCapacity(100); + + // An undeclared enum value is a marshalling failure, which a + // throwing call reports as the root exception. + var marshal = Throws(() => s.Put("k", Bytes("v"), (EntryKind)9, null), "undeclared kind"); + Expect(marshal.Code == NativeException.MarshalErrorCode && !(marshal is KvException), "undeclared kind is -3"); + + // delete, clear, and the deprecated size(). + Put(s, "beta", "b"); + Expect(s.Delete("beta") && !s.Delete("beta"), "delete"); #pragma warning disable CS0618 - Expect(store.LegacyPut("legacy", payload), "legacy_put inserts"); + Expect(s.Size() == 1, "size"); #pragma warning restore CS0618 - Expect(store.Count() == 2, "count == 2 after legacy_put"); - - // Share: a second wrapper to the SAME native object. Mutations through - // one are visible through the other; disposing one leaves the other - // usable. - var twin = store.Share(); - Expect(!ReferenceEquals(twin, store), "Share returns a distinct wrapper"); - Expect(twin.Equals(store), "Share wraps the same native pointer"); - Expect(twin.Count() == 2, "twin sees the same entries"); - Expect(twin.Put("via-twin", payload, EntryKind.Persistent, null), "put via twin"); - Expect(store.Count() == 3 && store.Get("via-twin") != null, "store sees the twin's put"); - twin.Dispose(); - twin.Dispose(); - try - { - twin.Count(); - Expect(false, "expected ObjectDisposedException on disposed twin"); - } - catch (ObjectDisposedException) + Expect(s.Clear() == 1 && s.Count() == 0, "clear"); + } + + static void Iterators() + { + using var s = Store.Open("/iter"); + Put(s, "user.bob", "b"); + Put(s, "user.alice", "a"); + Put(s, "sys.x", "xx"); + + Expect(s.Keys(null).SequenceEqual(new[] { "sys.x", "user.alice", "user.bob" }), "keys in order"); + ExpectKeyNotFound(() => s.Keys("zzz"), "zzz", "keys(zzz)"); + + // Abandoning an iterator part-way releases it. + using (var keys = s.Keys("user.").GetEnumerator()) { + Expect(keys.MoveNext() && keys.Current == "user.alice", "first key"); + Expect(LeakCheck.Live("kvstore", LeakCheck.Iterators) == 1, "the iterator is live"); } - Expect(store.Count() == 3, "store survives disposing its twin"); - - // Fork: an independent copy. - var forked = store.Fork(); - Expect(!forked.Equals(store), "Fork is a different native object"); - Expect(forked.Count() == 3, "fork copied live entries"); - Expect(forked.Put("only-in-fork", payload, EntryKind.Volatile, null), "put into fork"); - Expect(forked.Count() == 4 && store.Count() == 3, "fork and original diverge"); - - // `Store?` as parameter and return. - var empty = Store.Open("/tmp/conformance-kvstore-dotnet-empty"); - Expect(empty.Larger(null) == null, "empty.Larger(null) is null"); - var bigger = empty.Larger(store); - Expect(bigger != null && bigger.Equals(store), "empty.Larger(store) is store"); - Expect(bigger.Count() == 3, "returned larger store is usable"); - bigger.Dispose(); - var own = store.Larger(null); - Expect(own != null && own.Equals(store), "store.Larger(null) is store itself"); - own.Dispose(); - var forkWins = store.Larger(forked); - Expect(forkWins != null && forkWins.Equals(forked), "store.Larger(fork) is the fork"); - forkWins.Dispose(); - Expect(store.Count() == 3 && forked.Count() == 4, "both stores alive after Larger releases"); - - // A record carrying objects: `store` is required, `mirror` optional. - var info = store.Describe("primary", null); - Expect(info.Label == "primary", "describe label"); - Expect(info.Count == 3, $"describe count == 3 (got {info.Count})"); - Expect(info.Store.Equals(store), "describe().Store is the same native object"); - Expect(info.Mirror == null, "describe().Mirror absent"); - Expect(info.Store.Count() == 3, "describe().Store is usable"); - var mirrored = store.Describe("mirrored", forked); - Expect(mirrored.Mirror != null && mirrored.Mirror.Equals(forked), - "describe().Mirror is the fork"); - Expect(mirrored.Mirror.Count() == 4, "describe().Mirror is usable"); - - // A list of objects as a return, and objects inside a list and an - // optional record as parameters (each encoded token is a fresh clone). - var many = Store.OpenMany(new[] { "/tmp/many-a", "/tmp/many-b" }); - Expect(many.Length == 2, "open_many returns two stores"); - Expect(!many[0].Equals(many[1]), "open_many stores are distinct"); - Expect(many[0].Count() == 0 && many[1].Count() == 0, "open_many stores start empty"); - Expect(many[0].Put("m", payload, EntryKind.Volatile, null), "put into many[0]"); - Expect(Store.TotalCount(many, null) == 1, "total_count over the list"); - Expect(Store.TotalCount(many, info) == 1 + 3, "total_count adds the record's store"); - Expect(Store.TotalCount(many, mirrored) == 1 + 3, "total_count ignores the mirror"); - Expect(Store.TotalCount(new Store[0], null) == 0, "total_count of nothing"); - var handBuilt = new StoreInfo("hand", forked, store, 0); - Expect(Store.TotalCount(new[] { store, forked }, handBuilt) == 3 + 4 + 4, - "total_count with a consumer-built record"); - Expect(store.Count() == 3 && forked.Count() == 4 && many[0].Count() == 1, - "every store survives being encoded into buffers"); - try + Expect(LeakCheck.Live("kvstore", LeakCheck.Iterators) == 0, "the abandoned iterator is released"); + + var entries = s.Entries("sys.").ToList(); + Expect(entries.Count == 1 && entries[0].Key == "sys.x" && Text(entries[0].Value) == "xx", "entries(sys.)"); + + // partition: objects, created as they're pulled. + var prefixes = new[] { "user.", "sys.", "none." }; + var counts = new uint[] { 2, 1, 0 }; + var n = 0; + foreach (var part in s.Partition(prefixes)) { - Store.OpenMany(new[] { "/tmp/ok", "" }); - Expect(false, "expected KvException from open_many"); + using (part) + { + Expect(!part.Equals(s) && part.Count() == counts[n] && part.Path() == prefixes[n], $"partition {n}"); + } + n++; } - catch (KvException e) + Expect(n == 3, "three partitions"); + } + + static void Listeners() + { + MainThread = Environment.CurrentManagedThreadId; + using var s = Store.Open("/listen"); + var baseline = LiveCallbacks(); + + var l = new TestListener(skip: "quiet"); + var id = s.Subscribe(l); + Expect(id > 0 && s.ListenerCount() == 1, "subscribe"); + + Put(s, "a", "1"); + Expect(l.Puts == 1 && l.LastVersion == 1 && !l.LastReplaced, "Put v1"); + Put(s, "a", "2"); + Expect(l.Puts == 2 && l.LastVersion == 2 && l.LastReplaced && l.LastKey == "a", "Put v2"); + Put(s, "quiet", "x"); // accepts() said no + Expect(l.Puts == 2, "filtered"); + Expect(s.Delete("a") && l.Removed == 1 && l.LastKey == "a", "Removed"); + + // An expired read removes the entry and says so. + Put(s, "short", "x", EntryKind.Volatile, 1); + s.Tick(1); + Throws(() => s.Get("short"), "get(short)"); + Expect(l.Expired == 1 && l.LastKey == "short", "Removed expired"); + + Expect(s.Clear() == 1, "clear"); // "quiet" was left + Expect(l.Cleared == 1 && l.LastCleared == 1, "Cleared"); + Expect(l.OffMainThread == 0, "synchronous calls notify on the calling thread"); + + // Unsubscribing releases the listener once. + Expect(LiveCallbacks() == baseline + 1, "one listener held"); + Expect(s.Unsubscribe(id), "unsubscribe"); + Expect(LiveCallbacks() == baseline, "the listener is released"); + Expect(!s.Unsubscribe(id) && s.ListenerCount() == 0, "unsubscribe again"); + + // A listener that throws is detached (and released); the put succeeds. + var failing = new TestListener(failOn: "boom"); + s.Subscribe(failing); + Put(s, "fine", "1"); + Expect(failing.Puts == 1, "fine delivered"); + Put(s, "boom", "1"); + Expect(s.Count() == 2 && s.ListenerCount() == 0, "the failing listener is detached"); + Expect(LiveCallbacks() == baseline, "the failing listener is released"); + + // Disposing the store releases the listeners it still holds. + s.Subscribe(new TestListener()); + s.Subscribe(new TestListener()); + Expect(s.ListenerCount() == 2 && LiveCallbacks() == baseline + 2, "two listeners held"); + s.Dispose(); + Expect(LiveCallbacks() == baseline, "disposing the store releases its listeners"); + } + + static void Policies() + { + using var s = Store.Open("/policy"); + using var other = Store.Open("/other"); + var baseline = LiveCallbacks(); + + var p = new TestPolicy(other); + s.SetPolicy(p); + Expect(s.HasPolicy(), "has policy"); + + // admit's record return is what's stored (its key and version aside). + var e = Put(s, "a", "1", EntryKind.Volatile); + Expect(e.Key == "a" && e.Version == 1 && e.Kind == EntryKind.Encrypted, "admitted entry"); + Expect(e.Tags.SequenceEqual(new[] { "admitted" }), "admitted tags"); + + // route: the object parameter and object return redirect a write. + Put(s, "b/x", "2", EntryKind.Volatile); + Expect(s.Count() == 1 && other.Count() == 1, "routed"); + + // A typed error from the throwing callback reaches the caller with + // its code, message, and fields. + var rejected = Throws(() => Put(s, "secret", "3", EntryKind.Volatile), "secret"); + Expect(rejected.Code == 1005 && rejected.Message == "secrets are not stored", $"Rejected (got '{rejected.Message}')"); + Expect(rejected.Key == "secret" && rejected.Reason == "no secrets", "Rejected fields"); + + // Any other exception arrives as -4 with the implementation's message. + var foreign = Throws(() => Put(s, "boom", "4", EntryKind.Volatile), "boom"); + Expect(foreign.Code == NativeException.ForeignErrorCode && foreign.Message == "policy exploded", + $"foreign failure (got {foreign.Code} '{foreign.Message}')"); + // A null required object is a return the runtime can't accept: -3. + var nullRoute = Throws(() => Put(s, "null/x", "6", EntryKind.Volatile), "null route"); + Expect(nullRoute.Code == NativeException.MarshalErrorCode, $"null route is -3 (got {nullRoute.Code})"); + Expect(s.Count() == 1 && other.Count() == 1, "failed puts store nothing"); + Expect(p.Admitted == 5, $"admitted 5 times (got {p.Admitted})"); + + // Replacing the policy releases the old one; null removes it. + Expect(LiveCallbacks() == baseline + 1, "one policy held"); + s.SetPolicy(new TestPolicy(other)); + Expect(LiveCallbacks() == baseline + 1, "replacing releases the old policy"); + s.SetPolicy(null); + Expect(LiveCallbacks() == baseline && !s.HasPolicy(), "SetPolicy(null) releases it"); + Put(s, "secret", "now allowed"); + Expect(s.Count() == 2, "no policy"); + } + + static void Loaders() + { + using var s = Store.Open("/load"); + var baseline = LiveCallbacks(); + + // No loader (a null optional callback): a miss is none. + Expect(s.GetOrLoad("k", null) == null, "no loader"); + + // load's bytes are stored, tagged with the loader's name. + var e = s.GetOrLoad("k", new TestLoader()); + Expect(e != null && Text(e.Value) == "loaded:k" && e.Kind == EntryKind.Volatile, "loaded"); + Expect(e.Metadata.Count == 1 && e.Metadata["source"] == "dotnet-loader", "loaded metadata"); + Expect(LiveCallbacks() == baseline, "the loader is released after the call"); + Expect(s.Count() == 1, "stored"); + // A hit doesn't consult the loader. + Expect(s.GetOrLoad("k", new TestLoader())?.Version == 1, "hit"); + + // The fallback store (an optional object return) is consulted first. + using (var backup = Store.Open("/backup")) { - Expect(e.Code == KvException.IoError, "open_many reports IoError"); + Put(backup, "fb", "from backup"); + var copied = s.GetOrLoad("fb", new TestLoader(backup)); + Expect(copied != null && Text(copied.Value) == "from backup" && copied.Kind == EntryKind.Persistent, + "copied from the fallback"); } - // Async: an immediately-expired entry gives compact 3 bytes to - // reclaim; the Task completes from the producer's worker thread. - Expect(store.Put("doomed", payload, EntryKind.Volatile, 0), "put doomed"); - long reclaimed = await store.Compact(); - Expect(reclaimed == 3, $"compact reclaimed 3 bytes (got {reclaimed})"); - Expect(store.Count() == 3, "live entries survive compact"); - Expect((await forked.Compact()) == 0, "nothing to compact in the fork"); + // KeyNotFound for this key: the producer decoded the fields and + // answers none. + Expect(s.GetOrLoad("missing", new TestLoader()) == null, "missing is none"); + // KeyNotFound for another key: passed through, fields intact. + var elsewhere = Throws(() => s.GetOrLoad("elsewhere", new TestLoader()), "elsewhere"); + Expect(elsewhere.Key == "other" && elsewhere.Message == "not in the loader", "passed through"); + // Any other failure is -4. + var broken = Throws(() => s.GetOrLoad("broken", new TestLoader()), "broken"); + Expect(broken.Code == NativeException.ForeignErrorCode && broken.Message == "loader is broken", + $"broken loader (got {broken.Code} '{broken.Message}')"); + Expect(LiveCallbacks() == baseline, "every loader is released"); + } + + static async Task AsyncCalls() + { + MainThread = Environment.CurrentManagedThreadId; + using var s = Store.Open("/async-calls"); + var l = new TestListener(); + s.Subscribe(l); + Put(s, "old1", "x", EntryKind.Volatile, 1); + Put(s, "old2", "x", EntryKind.Volatile, 1); + Put(s, "keep", "x"); + s.Tick(5); + + // compact runs on a producer thread and notifies listeners there. using (var cts = new CancellationTokenSource()) { - Expect((await forked.Compact(cts.Token)) == 0, "compact with a live token"); - cts.Cancel(); - try - { - await forked.Compact(cts.Token); - Expect(false, "expected a canceled compact"); - } - catch (OperationCanceledException) - { - } + Expect(await s.Compact(0, cts.Token) == 2, "compact(0) removes two"); } + Expect(l.Expired == 2 && l.OffMainThread == 2, $"notified from a producer thread ({l.OffMainThread})"); + Expect(s.Count() == 1, "one left"); - store.Clear(); - Expect(store.Count() == 0, "store empty after clear"); - Expect(info.Store.Count() == 0, "describe().Store observes the clear"); + // Without a token the call never cancels. + Expect(await s.Compact(5) == 0, "compact(5)"); - // Release every wrapper. Records holding objects expose IDisposable - // wrappers the consumer disposes; double Dispose is a no-op. - foreach (var s in many) + // Cancel mid-pause: the task cancels at once, and the background + // pause notices the native token and stops. + using (var cts = new CancellationTokenSource()) { - s.Dispose(); - s.Dispose(); + var pending = s.Compact(60_000, cts.Token); + await Task.Delay(20); + Expect(!pending.IsCompleted && Store.ActiveJobs() >= 1, "the pause is running"); + cts.Cancel(); + await ThrowsAsync(() => pending, "cancelled compact"); + Expect(pending.IsCanceled, "the task is canceled"); } - info.Store.Dispose(); - mirrored.Store.Dispose(); - mirrored.Mirror.Dispose(); - empty.Dispose(); - forked.Dispose(); - store.Dispose(); - store.Dispose(); - try + var stopped = false; + for (var i = 0; i < 200 && !stopped; i++) { - KvStats.GetStats(store); - Expect(false, "expected ObjectDisposedException for a disposed store argument"); + stopped = Store.ActiveJobs() == 0; + if (!stopped) + { + await Task.Delay(10); + } } - catch (ObjectDisposedException) + Expect(stopped, "the cancelled pause stopped cooperatively"); + // An already-cancelled token doesn't launch. + await ThrowsAsync(() => s.Compact(5, new CancellationToken(true)), "pre-cancelled"); + + // get_many: an async list of optional records, launched concurrently. + var many = Enumerable.Range(0, 32).Select(_ => s.GetMany(new[] { "keep", "gone", "keep" })).ToArray(); + foreach (var got in await Task.WhenAll(many)) { + Expect(got.Length == 3 && got[0] != null && got[1] == null && got[2]?.Key == "keep", "get_many"); } - try + + // The nested module's async free function: objects in a list in, a + // record out. + using var other = Store.Open("/other"); + Put(other, "a", "123"); + var st = await KvStats.SummarizeAll(new[] { s, other }); + Expect(st.Entries == 2 && st.Bytes == 4, "summarize_all totals"); + Expect(st.ByKind.Count == 1 && st.ByKind[EntryKind.Persistent] == 2, "summarize_all by_kind"); + } + + static void ObjectGraph() + { + var s = Store.Open("/graph"); + Put(s, "k", "v"); + + // share(): the same object, alive through either wrapper. + var shared = s.Share(); + Expect(shared.Equals(s), "share is the same object"); + s.Dispose(); + s.Dispose(); // safe twice + Throws(() => s.Count(), "a disposed wrapper"); + Expect(shared.Count() == 1, "alive through the shared reference"); + s = shared; + + // fork(): a distinct object with a copy of the entries. + using var fork = s.Fork(); + Expect(!fork.Equals(s) && fork.Count() == 1 && fork.Path() == "/graph", "fork"); + Put(fork, "k2", "v"); + Expect(fork.Count() == 2 && s.Count() == 1, "fork is a copy"); + + // larger(): `Store?` in and out. + using var empty = Store.Open("/empty"); + Expect(empty.Larger(null) == null, "larger(none) on empty"); + using (var bigger = empty.Larger(fork)) { - await store.Compact(); - Expect(false, "expected ObjectDisposedException from async on a disposed store"); + Expect(bigger.Equals(fork), "larger(fork)"); } - catch (ObjectDisposedException) + using (var self = s.Larger(null)) { + Expect(self.Equals(s), "larger(none) on non-empty is self"); } - using (var scoped = Store.Open("/tmp/conformance-kvstore-dotnet-scoped")) + // describe(): a record whose fields carry objects. + var info = s.Describe("main", fork); + Expect(info.Label == "main" && info.Store.Equals(s) && info.Mirror.Equals(fork) && info.Count == 1, "describe"); + Expect(info.Mirror.Count() == 2, "the mirror is usable"); + + // open_many(): a list of objects; one bad path fails the whole call. + var many = Store.OpenMany(new[] { "/a", "/b" }); + Expect(many.Length == 2 && many[0].Path() == "/a" && many[1].Path() == "/b", "open_many"); + Throws(() => Store.OpenMany(new[] { "/a", "" }), "open_many with a bad path"); + + // by_label(): records with objects in, a map with object values out. + var named = Store.ByLabel(new[] { info, new StoreInfo("first", many[0], null, 0) }); + Expect(named.Count == 2 && named["main"].Equals(s) && named["first"].Equals(many[0]), "by_label"); + + // total_count(): a list, a map, and an optional record, all carrying + // objects (each encoding mints fresh references). + Put(many[0], "m", "1"); + var three = new[] { many[0], many[1], fork }; + Expect(Store.TotalCount(three, named, info) == 6, "total_count with extra"); + Expect(Store.TotalCount(three, named, null) == 5, "total_count without extra"); + + // Everything is still usable after crossing in buffers. + Expect(s.Count() == 1 && fork.Count() == 2 && many[0].Count() == 1, "still usable"); + foreach (var store in many.Concat(named.Values).Concat(new[] { info.Store, info.Mirror, s })) { - Expect(scoped.Count() == 0, "scoped store opens empty"); + store.Dispose(); } } + + static void StatsAndReport() + { + using var s = Store.Open("/stats"); + Put(s, "b", "12"); + Put(s, "a", "1"); + Put(s, "a", "123"); + Put(s, "c", "x", EntryKind.Encrypted); + + // kv.stats.summarize: the parent's Store and error domain. + var st = KvStats.Summarize(s, null); + Expect(st.Entries == 3 && st.Bytes == 6 && st.ByKind.Count == 2, "summarize totals"); + Expect(st.ByKind[EntryKind.Persistent] == 2 && st.ByKind[EntryKind.Encrypted] == 1, "summarize by_kind"); + ExpectKeyNotFound(() => KvStats.Summarize(s, "q"), "q", "summarize(q)"); + + // report.render_report: the sibling root shares the Entry record. + var lines = Report.RenderReport(s.Entries(null).ToArray()); + Expect(lines.SequenceEqual(new[] { "a: 3 bytes, Persistent, v2", "b: 2 bytes, Persistent", "c: 1 bytes, Encrypted" }), + $"render_report (got [{string.Join("; ", lines)}])"); + var nothing = Throws(() => Report.RenderReport(Array.Empty()), "empty report"); + Expect(nothing.Code == 2001 && nothing.Message == "nothing to report", "NothingToReport"); + } + + static async Task Run() + { + await Constructors(); + Basics(); + Iterators(); + Listeners(); + Policies(); + Loaders(); + await AsyncCalls(); + ObjectGraph(); + StatsAndReport(); + } + + static int Main() + { + Run().GetAwaiter().GetResult(); + LeakCheck.AssertNoLeaks("kvstore"); + Console.WriteLine("dotnet/kvstore: OK"); + return 0; + } } diff --git a/conformance/dotnet/LeakCheck.cs b/conformance/dotnet/LeakCheck.cs index 2cccd6bf..32a662cb 100644 --- a/conformance/dotnet/LeakCheck.cs +++ b/conformance/dotnet/LeakCheck.cs @@ -1,7 +1,7 @@ -// Shared by every .NET conformance consumer: after the consumer has dropped -// its wrappers, force the garbage collector and finalizers, then assert that -// the producer's leak counters (live objects, foreign callbacks, iterators, -// cancel tokens, and returned allocations) are all back to zero. The +// Shared by every .NET conformance consumer: reads the producer's leak +// counters and, at the end, forces the garbage collector and finalizers, +// then asserts that the counters (live objects, foreign callbacks, +// iterators, cancel tokens, and byte runs) are all back to zero. The // counter export is reached through raw FFI on the same library the // generated bindings load ({PREFIX}_LIBRARY). @@ -11,18 +11,41 @@ internal static unsafe class LeakCheck { + /// The counter kinds `debug_live` reports. + public const int Objects = 0, Callbacks = 1, Iterators = 2, CancelTokens = 3, Allocations = 4; + private static readonly string[] Kinds = { "objects", "callbacks", "iterators", "cancel tokens", "allocations" }; - public static void AssertNoLeaks(string prefix) + private static delegate* unmanaged[Cdecl] Counter(string prefix) { var path = Environment.GetEnvironmentVariable(prefix.ToUpperInvariant() + "_LIBRARY"); var lib = NativeLibrary.Load(path!); - var live = (delegate* unmanaged[Cdecl])NativeLibrary.GetExport(lib, prefix + "_debug_live"); + return (delegate* unmanaged[Cdecl])NativeLibrary.GetExport(lib, prefix + "_debug_live"); + } + + /// The producer's live count of one kind, after letting the + /// garbage collector release whatever wrappers are unreachable. + public static ulong Live(string prefix, int kind) + { + GC.Collect(); + GC.WaitForPendingFinalizers(); + GC.Collect(); + return Counter(prefix)(kind); + } + + public static void AssertNoLeaks(string prefix) + { + var live = Counter(prefix); + if (live(-1) != 1) + { + Console.Error.WriteLine("leak: the producer doesn't count live resources (build it with leak-check)"); + Environment.Exit(1); + } var counts = new ulong[Kinds.Length]; - // Releasing one object can release others (a bus drops its - // subscribers), and async completions finish on producer threads, so - // settle for a moment before failing. + // Releasing one object can release others (a store drops its + // listeners and policy), and async workers may still be dropping a + // finished future, so settle for up to ~2 s before failing. for (var attempt = 0; attempt < 100; attempt++) { GC.Collect(); diff --git a/conformance/dotnet/run.sh b/conformance/dotnet/run.sh index a9fc5052..fcbd08bb 100755 --- a/conformance/dotnet/run.sh +++ b/conformance/dotnet/run.sh @@ -5,6 +5,7 @@ # consumer against one sample and must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools dotnet dotnet # .NET: a console app with a ProjectReference to the generated project, used # as generated. The generated loader finds the producer through @@ -39,20 +40,12 @@ EOF dotnet run -c Release --nologo -v quiet 2>&1 ) } -dotnet_contacts() { dotnet_consumer contacts Contacts.cs; } +dotnet_calculator() { dotnet_consumer calculator Calculator.cs; } +dotnet_codec() { dotnet_consumer codec Codec.cs; } +dotnet_kvstore() { dotnet_consumer kvstore Kvstore.cs; } -dotnet_events() { dotnet_consumer events Events.cs; } - -dotnet_kvstore() { dotnet_consumer kvstore Kvstore.cs; } - -dotnet_async_demo() { dotnet_consumer async-demo AsyncDemo.cs; } - -dotnet_codec() { dotnet_consumer codec Codec.cs; } - -lane dotnet-contacts dotnet_contacts -lane dotnet-events dotnet_events -lane dotnet-kvstore dotnet_kvstore -lane dotnet-async-demo dotnet_async_demo +lane dotnet-calculator dotnet_calculator lane dotnet-codec dotnet_codec +lane dotnet-kvstore dotnet_kvstore finish_lanes diff --git a/conformance/go/async_demo.go b/conformance/go/async_demo.go deleted file mode 100644 index b7f4468e..00000000 --- a/conformance/go/async_demo.go +++ /dev/null @@ -1,105 +0,0 @@ -// Conformance consumer: async-demo sample, Go target. -// -// Drives the async bridge end to end: every async wrapper takes a -// context.Context and blocks until the producer's completion callback fires. -// RunTask decodes the TaskResult record from a value buffer, an empty name -// reports the typed *TaskError (matched via errors.As), RunBatch round-trips -// a list of records both ways, and RunNTasks returns a direct scalar. -// -// Cancellation: Wait is cancellable. Left alone it completes after its -// timeout; cancelling its context (or letting a deadline expire) cancels the -// native token, and the call returns promptly with the context's error. An -// already-cancelled context never launches. Exits 0 on success; aborts -// (non-zero) on any mismatch. - -package main - -import ( - "context" - "errors" - "fmt" - "time" - - wv "__MODPATH__" -) - -func main() { - ctx := context.Background() - - // Async record return. - result, err := wv.RunTask(ctx, "alpha") - expect(err == nil, fmt.Sprintf("run_task succeeds (got %v)", err)) - expect(result.Id > 0, "run_task assigns an id") - expect(result.Value == "completed: alpha", fmt.Sprintf("run_task value (got %q)", result.Value)) - expect(result.Success, "run_task success flag") - - // Typed async error: the empty name reports TaskError InvalidName. - _, err = wv.RunTask(ctx, "") - var taskErr *wv.TaskError - expect(errors.As(err, &taskErr), fmt.Sprintf("typed *TaskError (got %T)", err)) - expect(taskErr.Code == wv.TaskErrorInvalidName, fmt.Sprintf("InvalidName carries code 1 (got %d)", taskErr.Code)) - - // Buffered list-of-records both ways. - batch, err := wv.RunBatch(ctx, []string{"a", "b", "c"}) - expect(err == nil && len(batch) == 3, fmt.Sprintf("run_batch returns 3 results (got %d, %v)", len(batch), err)) - for i, want := range []string{"completed: a", "completed: b", "completed: c"} { - expect(batch[i].Value == want, fmt.Sprintf("run_batch[%d] value (got %q)", i, batch[i].Value)) - expect(batch[i].Success, "run_batch success flag") - } - - // Direct scalar through the async callback. - n, err := wv.RunNTasks(ctx, 7) - expect(err == nil && n == 7, "run_n_tasks echoes n") - - // A cancellable call left alone completes normally. - waited, err := wv.Wait(ctx, 20) - expect(err == nil && waited == 20, fmt.Sprintf("wait(20) completes (got %d, %v)", waited, err)) - - // Cancelling the context cancels the native call promptly. - cctx, cancel := context.WithCancel(ctx) - time.AfterFunc(50*time.Millisecond, cancel) - start := time.Now() - _, err = wv.Wait(cctx, 30_000) - elapsed := time.Since(start) - expect(errors.Is(err, context.Canceled), fmt.Sprintf("cancelled wait returns context.Canceled (got %v)", err)) - expect(elapsed < 5*time.Second, fmt.Sprintf("cancellation is prompt (took %v)", elapsed)) - - // A deadline cancels the same way and reports DeadlineExceeded. - dctx, stop := context.WithTimeout(ctx, 50*time.Millisecond) - _, err = wv.Wait(dctx, 30_000) - stop() - expect(errors.Is(err, context.DeadlineExceeded), fmt.Sprintf("deadline yields DeadlineExceeded (got %v)", err)) - - // An already-cancelled context never launches the call. - done, cancelNow := context.WithCancel(ctx) - cancelNow() - _, err = wv.Wait(done, 30_000) - expect(errors.Is(err, context.Canceled), "pre-cancelled wait returns context.Canceled") - _, err = wv.RunTask(done, "never") - expect(errors.Is(err, context.Canceled), "pre-cancelled run_task returns context.Canceled") - - // Many concurrent cancellations all settle. - errs := make(chan error, 16) - for range 16 { - go func() { - c, cancel := context.WithTimeout(ctx, 10*time.Millisecond) - defer cancel() - _, err := wv.Wait(c, 30_000) - errs <- err - }() - } - for range 16 { - expect(errors.Is(<-errs, context.DeadlineExceeded), "concurrent cancellation") - } - - // Every task body has finished: cancelled ones were dropped by the - // runtime before their completion fired. - deadline := time.Now().Add(5 * time.Second) - for wv.ActiveCallbacks() != 0 && time.Now().Before(deadline) { - time.Sleep(5 * time.Millisecond) - } - expect(wv.ActiveCallbacks() == 0, "active_callbacks settles to zero") - - expectNoLeaks(wv.DebugLive) - fmt.Println("go/async-demo: OK") -} diff --git a/conformance/go/calculator.go b/conformance/go/calculator.go index 4c5ea887..ada165f3 100644 --- a/conformance/go/calculator.go +++ b/conformance/go/calculator.go @@ -1,35 +1,51 @@ -// Conformance consumer: calculator sample, Go target. +// Conformance consumer: calculator sample, Go target (ABI revision 4). // -// The minimal surface: plain scalar functions, the throwing Div with its -// typed *CalcError, and a string round trip that keeps interior NUL bytes and -// non-ASCII text intact through the (pointer, length) string ABI. Exits 0 on -// success; aborts (non-zero) on any mismatch. +// The getting-started surface: a direct-value call (wrapping on overflow), +// a throwing call that returns the typed *DivisionByZeroError, and a string +// in and out (non-ASCII and astral text and an interior NUL byte survive). +// Importing the bindings checks the ABI revision and the contract table. +// Ends by asserting the library's leak counters are zero. package main import ( - "errors" "fmt" + "math" - wv "__MODPATH__" + calc "__MODPATH__" ) func main() { - expect(wv.Add(2, 3) == 5, "add") - expect(wv.Mul(-4, 6) == -24, "mul") - - q, err := wv.Div(17, 5) - expect(err == nil && q == 3, fmt.Sprintf("div (got %d, %v)", q, err)) - _, err = wv.Div(1, 0) - var cerr *wv.CalcError - expect(errors.As(err, &cerr), fmt.Sprintf("div by zero yields *CalcError (got %T %v)", err, err)) - expect(cerr.Code == wv.CalcErrorDivisionByZero, "division-by-zero code") - expect(cerr.Message == "division by zero", fmt.Sprintf("message (got %q)", cerr.Message)) - - for _, s := range []string{"", "hello", "a\x00b\x00", "héllo ✓ 😀"} { - expect(wv.Echo(s) == s, fmt.Sprintf("echo %q", s)) + expect(calc.Add(2, 3) == 5, "add(2, 3)") + expect(calc.Add(-7, 7) == 0, "add(-7, 7)") + expect(calc.Add(math.MaxInt32, 1) == math.MinInt32, "add wraps") + + for _, c := range []struct{ a, b, q int32 }{ + {10, 2, 5}, + {-7, 2, -3}, // rounds toward zero + {math.MinInt32, -1, math.MinInt32}, + } { + q, err := calc.Divide(c.a, c.b) + expect(err == nil && q == c.q, fmt.Sprintf("divide(%d, %d) = %d, %v", c.a, c.b, q, err)) + } + + q, err := calc.Divide(1, 0) + expect(q == 0, "a failed divide returns zero") + e := expectAs[*calc.DivisionByZeroError](err, "divide(1, 0)") + expect(e.Code() == 1 && e.Error() == "division by zero", fmt.Sprintf("DivisionByZero code and message (got %d %q)", e.Code(), e.Error())) + expectAs[calc.CalcError](err, "divide(1, 0) is a CalcError") + + for name, want := range map[string]string{ + "World": "Hello, World!", + "": "Hello, !", + "Wörld 🦀": "Hello, Wörld 🦀!", + "a\x00b": "Hello, a\x00b!", + "😀 astral": "Hello, 😀 astral!", + } { + got := calc.Greet(name) + expect(got == want, fmt.Sprintf("greet(%q) = %q", name, got)) } - expectNoLeaks(wv.DebugLive) + expectNoLeaks(calc.DebugLive) fmt.Println("go/calculator: OK") } diff --git a/conformance/go/codec.go b/conformance/go/codec.go index 6ce25545..d891f23a 100644 --- a/conformance/go/codec.go +++ b/conformance/go/codec.go @@ -1,475 +1,299 @@ -// Conformance consumer: codec sample, Go target. +// Conformance consumer: codec sample, Go target (ABI revision 4). // -// Drives the value-buffer round-trip oracle through the generated cgo -// bindings. For Scalars and Composite it fetches the producer's canonical -// fixture (producer encodes, Go decodes), checks every field against the -// values in samples/codec/src/lib.rs, hands the fixture back to verify_* -// (Go encodes, producer decodes and compares), and re-encodes through -// roundtrip_* comparing field by field. It then builds its own values with -// edge cases (empty strings, lists and maps, embedded NUL and non-BMP text, -// int64/uint64 extremes, NaN, the infinities, negative zero, present-but- -// empty optionals, lists of absent optionals) and round-trips them, exercises -// every Shape variant, the top-level optional/list/map/string/bytes/scalar -// echoes, the typed CodecError on a mismatch, and Holder: object tokens in a -// record field, an optional, and a list, with primary_of returning a wrapper -// to the same object, same_primary proving identity, and every wrapper -// released (double Close harmless, use after Close trapped). Exits 0 on -// success; aborts (non-zero) on any mismatch. +// The shared-vector loop: for every vector the library serves, decode it +// through the generated bindings, hand it back to CheckVector (which must +// accept it and reject it at the neighboring index), and push each primitive +// vector's value through the matching direct-family Echo function. Then +// vectors built from literals (so a symmetric encode/decode bug can't hide), +// spot checks of decoded fields, the typed *OutOfRangeError and its payload, +// malformed input rejected (an undeclared enum value, text that isn't UTF-8, +// a nil sum-type value the encoder refuses), and object identity and +// reference counting through buffers. Ends by asserting the library's leak +// counters are zero. package main import ( - "errors" + "bytes" "fmt" "math" - "strings" - wv "__MODPATH__" + codec "__MODPATH__" ) -// Bit-exact float comparison so NaN payloads and the sign of zero count. -func sameF64(a, b float64) bool { return math.Float64bits(a) == math.Float64bits(b) } func sameF32(a, b float32) bool { return math.Float32bits(a) == math.Float32bits(b) } +func sameF64(a, b float64) bool { return math.Float64bits(a) == math.Float64bits(b) } -func sameStrPtr(a, b *string) bool { - if a == nil || b == nil { - return a == nil && b == nil - } - return *a == *b +func fetch(i uint32) codec.Vector { + v, err := codec.CodecVector(i) + expect(err == nil && v != nil, fmt.Sprintf("vector(%d): %v", i, err)) + return v } -func sameI64Ptr(a, b *int64) bool { - if a == nil || b == nil { - return a == nil && b == nil - } - return *a == *b +func vectorName(i uint32) string { + name, err := codec.VectorName(i) + expect(err == nil, fmt.Sprintf("vector_name(%d): %v", i, err)) + return name } -func sameBytes(a, b []byte) bool { - if (a == nil) != (b == nil) || len(a) != len(b) { - return false - } - for i := range a { - if a[i] != b[i] { - return false +// find returns the index of the vector called name. +func find(n uint32, name string) uint32 { + for i := range n { + if vectorName(i) == name { + return i } } - return true + expect(false, "no vector named "+name) + return 0 } -func sameScalars(a, b wv.Scalars) bool { - return a.I8Value == b.I8Value && a.U8Value == b.U8Value && - a.I16Value == b.I16Value && a.U16Value == b.U16Value && - a.I32Value == b.I32Value && a.U32Value == b.U32Value && - a.I64Value == b.I64Value && a.U64Value == b.U64Value && - sameF32(a.F32Value, b.F32Value) && sameF64(a.F64Value, b.F64Value) && - a.Flag == b.Flag && a.Color == b.Color -} - -func sameShape(a, b wv.Shape) bool { - switch x := a.(type) { - case nil: - return b == nil - case wv.ShapeEmpty: - _, ok := b.(wv.ShapeEmpty) - return ok - case wv.ShapeCircle: - y, ok := b.(wv.ShapeCircle) - return ok && sameF64(x.Radius, y.Radius) - case wv.ShapeRect: - y, ok := b.(wv.ShapeRect) - return ok && sameF32(x.Width, y.Width) && sameF32(x.Height, y.Height) - case wv.ShapeLabeled: - y, ok := b.(wv.ShapeLabeled) - return ok && x.Label == y.Label && x.Count == y.Count - case wv.ShapeNested: - y, ok := b.(wv.ShapeNested) - return ok && sameScalars(x.Inner, y.Inner) && sameStrPtr(x.Note, y.Note) +// release closes every token a decoded vector holds. +func release(v codec.Vector) { + if o, ok := v.(codec.VectorObjects); ok { + closeHolder(o.Value) } - return false } -func sameComposite(a, b wv.Composite) (bool, string) { - switch { - case a.Name != b.Name: - return false, "name" - case !sameBytes(a.Blob, b.Blob): - return false, "blob" - case !sameI64Ptr(a.SomeI64, b.SomeI64): - return false, "some_i64" - case !sameI64Ptr(a.NoneI64, b.NoneI64): - return false, "none_i64" - case !sameStrPtr(a.SomeText, b.SomeText): - return false, "some_text" - case len(a.Names) != len(b.Names): - return false, "names" - case len(a.Matrix) != len(b.Matrix): - return false, "matrix" - case len(a.Empty) != len(b.Empty): - return false, "empty" - case len(a.ByName) != len(b.ByName): - return false, "by_name" - case len(a.ById) != len(b.ById): - return false, "by_id" - case !sameScalars(a.Scalars, b.Scalars): - return false, "scalars" - case !sameShape(a.Shape, b.Shape): - return false, "shape" - case len(a.Shapes) != len(b.Shapes): - return false, "shapes" - case !sameShape(a.MaybeShape, b.MaybeShape): - return false, "maybe_shape" - case !sameBytes(a.MaybeList, b.MaybeList): - return false, "maybe_list" - case len(a.Sparse) != len(b.Sparse): - return false, "sparse" - case len(a.Colors) != len(b.Colors): - return false, "colors" - } - for i := range a.Names { - if a.Names[i] != b.Names[i] { - return false, fmt.Sprintf("names[%d]", i) - } - } - for i := range a.Matrix { - if len(a.Matrix[i]) != len(b.Matrix[i]) { - return false, fmt.Sprintf("matrix[%d]", i) - } - for j := range a.Matrix[i] { - if a.Matrix[i][j] != b.Matrix[i][j] { - return false, fmt.Sprintf("matrix[%d][%d]", i, j) - } - } - } - for i := range a.Empty { - if !sameF64(a.Empty[i], b.Empty[i]) { - return false, fmt.Sprintf("empty[%d]", i) - } - } - for k, v := range a.ByName { - if w, ok := b.ByName[k]; !ok || v != w { - return false, "by_name[" + k + "]" - } - } - for k, v := range a.ById { - if w, ok := b.ById[k]; !ok || !sameScalars(v, w) { - return false, fmt.Sprintf("by_id[%d]", k) - } - } - for i := range a.Shapes { - if !sameShape(a.Shapes[i], b.Shapes[i]) { - return false, fmt.Sprintf("shapes[%d]", i) - } +func closeHolder(h codec.Holder) { + h.Primary.Close() + if h.Spare != nil { + h.Spare.Close() } - for i := range a.Sparse { - x, y := a.Sparse[i], b.Sparse[i] - if (x == nil) != (y == nil) || (x != nil && *x != *y) { - return false, fmt.Sprintf("sparse[%d]", i) - } + for _, t := range h.Many { + t.Close() } - for i := range a.Colors { - if a.Colors[i] != b.Colors[i] { - return false, fmt.Sprintf("colors[%d]", i) - } + for _, t := range h.ByName { + t.Close() } - return true, "" } -func expectSameComposite(a, b wv.Composite, what string) { - same, field := sameComposite(a, b) - expect(same, fmt.Sprintf("%s: field %s differs\n a: %+v\n b: %+v", what, field, a, b)) +// echo pushes a primitive vector's value through its direct-family echo. +func echo(v codec.Vector) { + switch x := v.(type) { + case codec.VectorI8: + expect(codec.EchoI8(x.Value) == x.Value, "echo i8") + case codec.VectorU8: + expect(codec.EchoU8(x.Value) == x.Value, "echo u8") + case codec.VectorI16: + expect(codec.EchoI16(x.Value) == x.Value, "echo i16") + case codec.VectorU16: + expect(codec.EchoU16(x.Value) == x.Value, "echo u16") + case codec.VectorI32: + expect(codec.EchoI32(x.Value) == x.Value, "echo i32") + case codec.VectorU32: + expect(codec.EchoU32(x.Value) == x.Value, "echo u32") + case codec.VectorI64: + expect(codec.EchoI64(x.Value) == x.Value, "echo i64") + case codec.VectorU64: + expect(codec.EchoU64(x.Value) == x.Value, "echo u64") + case codec.VectorF32: + expect(sameF32(codec.EchoF32(x.Value), x.Value), "echo f32 (bitwise)") + case codec.VectorF64: + expect(sameF64(codec.EchoF64(x.Value), x.Value), "echo f64 (bitwise)") + case codec.VectorFlag: + expect(codec.EchoBool(x.Value) == x.Value, "echo bool") + case codec.VectorText: + expect(codec.EchoText(x.Value) == x.Value, fmt.Sprintf("echo text %q", x.Value)) + case codec.VectorBlob: + expect(bytes.Equal(codec.EchoBlob(x.Value), x.Value), "echo blob") + case codec.VectorHue: + expect(codec.EchoColor(x.Value) == x.Value, "echo color") + } } -func ptrI64(v int64) *int64 { return &v } -func ptrStr(v string) *string { return &v } -func ptrBool(v bool) *bool { return &v } +func everyVector(n uint32) { + for i := range n { + v := fetch(i) + if !codec.CheckVector(i, v) { + expect(false, fmt.Sprintf("vector %d (%s) did not round-trip; the library saw %s", + i, vectorName(i), codec.DescribeVector(v))) + } + expect(!codec.CheckVector((i+1)%n, v), fmt.Sprintf("vector %d matches its neighbor", i)) + echo(v) + release(v) + } +} -// canonicalScalars mirrors `canonical_scalars()` in the producer. -func canonicalScalars() wv.Scalars { - return wv.Scalars{ +func canonicalScalars() codec.Scalars { + return codec.Scalars{ I8Value: -8, U8Value: 200, - I16Value: -16_000, - U16Value: 60_000, - I32Value: -2_000_000_000, - U32Value: 4_000_000_000, - I64Value: -9_007_199_254_740_993, - U64Value: 18_446_744_073_709_551_615, + I16Value: -16000, + U16Value: 60000, + I32Value: -2000000000, + U32Value: 4000000000, + I64Value: -9007199254740993, + U64Value: math.MaxUint64, F32Value: 1.5, F64Value: -2.25e100, Flag: true, - Color: wv.ColorBlue, + Color: codec.ColorBlue, } } -func main() { - // ── Scalars: producer encodes, Go decodes ── - s := wv.SampleScalars() - expect(sameScalars(s, canonicalScalars()), fmt.Sprintf("sample_scalars matches the fixture (got %+v)", s)) - expect(s.Color == 7, "Blue keeps its explicit discriminant 7") - - // Go encodes, producer decodes and compares. - ok, err := wv.VerifyScalars(s) - expect(err == nil && ok, fmt.Sprintf("verify_scalars(sample) (err %v)", err)) - back := wv.RoundtripScalars(s) - expect(sameScalars(back, s), fmt.Sprintf("roundtrip_scalars echoes the sample (got %+v)", back)) - - // A mismatch is the typed CodecError. - changed := s - changed.U64Value-- - ok, err = wv.VerifyScalars(changed) - var cerr *wv.CodecError - expect(!ok && errors.As(err, &cerr), fmt.Sprintf("verify_scalars(changed) yields *CodecError (got %T %v)", err, err)) - expect(cerr.Code == wv.CodecErrorMismatch, fmt.Sprintf("mismatch code (got %d)", cerr.Code)) - expect(cerr.Code == 1, fmt.Sprintf("mismatch code is 1 (got %d)", cerr.Code)) - expect(cerr.Message == "value does not match the canonical fixture", fmt.Sprintf("mismatch default message (got %q)", cerr.Message)) - - // Extremes and IEEE specials through the buffered record. - extreme := wv.Scalars{ - I8Value: math.MinInt8, - U8Value: math.MaxUint8, - I16Value: math.MinInt16, - U16Value: math.MaxUint16, - I32Value: math.MinInt32, - U32Value: math.MaxUint32, - I64Value: math.MaxInt64, - U64Value: math.MaxUint64, - F32Value: float32(math.NaN()), - F64Value: math.Copysign(0, -1), - Flag: false, - Color: wv.ColorRed, - } - back = wv.RoundtripScalars(extreme) - expect(sameScalars(back, extreme), fmt.Sprintf("roundtrip_scalars keeps extremes, NaN, and -0 (got %+v)", back)) - expect(math.IsNaN(float64(back.F32Value)) && math.Signbit(back.F64Value) && back.F64Value == 0, - "NaN and negative zero decode as such") - extreme.I64Value, extreme.F64Value, extreme.F32Value = math.MinInt64, math.Inf(-1), float32(math.Inf(1)) - back = wv.RoundtripScalars(extreme) - expect(sameScalars(back, extreme), "roundtrip_scalars keeps MinInt64 and the infinities") - - // ── Composite: producer encodes, Go decodes ── - c := wv.SampleComposite() - expect(c.Name == "héllo wörld ✓", fmt.Sprintf("name (got %q)", c.Name)) - expect(sameBytes(c.Blob, []byte{0, 1, 2, 253, 254, 255}), fmt.Sprintf("blob (got %v)", c.Blob)) - expect(c.SomeI64 != nil && *c.SomeI64 == math.MinInt64, "some_i64 == i64::MIN") - expect(c.NoneI64 == nil, "none_i64 absent") - expect(c.SomeText != nil && *c.SomeText == "", "some_text is a present empty string") - expect(len(c.Names) == 3 && c.Names[0] == "a" && c.Names[1] == "" && c.Names[2] == "ccc", - fmt.Sprintf("names (got %q)", c.Names)) - expect(len(c.Matrix) == 3 && len(c.Matrix[0]) == 3 && c.Matrix[0][2] == 3 && - len(c.Matrix[1]) == 0 && len(c.Matrix[2]) == 1 && c.Matrix[2][0] == -4, - fmt.Sprintf("matrix (got %v)", c.Matrix)) - expect(len(c.Empty) == 0, "empty list") - expect(len(c.ByName) == 3 && c.ByName["one"] == 1 && c.ByName["two"] == 2 && c.ByName["neg"] == -3, - fmt.Sprintf("by_name (got %v)", c.ByName)) - expect(len(c.ById) == 2, "by_id has two entries") - expect(sameScalars(c.ById[-1], canonicalScalars()), "by_id[-1] is the canonical scalars") - unflagged := canonicalScalars() - unflagged.Flag = false - expect(sameScalars(c.ById[42], unflagged), "by_id[42] is the canonical scalars with flag false") - expect(sameScalars(c.Scalars, canonicalScalars()), "nested scalars") - expect(sameShape(c.Shape, wv.ShapeLabeled{Label: "tag", Count: 3}), fmt.Sprintf("shape (got %+v)", c.Shape)) - expect(len(c.Shapes) == 5, fmt.Sprintf("shapes has one of each variant (got %d)", len(c.Shapes))) - expect(sameShape(c.Shapes[0], wv.ShapeEmpty{}), "shapes[0] Empty") - expect(sameShape(c.Shapes[1], wv.ShapeCircle{Radius: 2.5}), "shapes[1] Circle") - expect(sameShape(c.Shapes[2], wv.ShapeRect{Width: 1.0, Height: 0.5}), "shapes[2] Rect") - expect(sameShape(c.Shapes[3], wv.ShapeLabeled{Label: "", Count: -1}), "shapes[3] Labeled") - expect(sameShape(c.Shapes[4], wv.ShapeNested{Inner: canonicalScalars(), Note: ptrStr("n")}), "shapes[4] Nested") - expect(sameShape(c.MaybeShape, wv.ShapeNested{Inner: canonicalScalars(), Note: nil}), "maybe_shape Nested without note") - expect(sameBytes(c.MaybeList, []byte{9, 8}), fmt.Sprintf("maybe_list (got %v)", c.MaybeList)) - expect(len(c.Sparse) == 3 && c.Sparse[0] != nil && *c.Sparse[0] && c.Sparse[1] == nil && - c.Sparse[2] != nil && !*c.Sparse[2], "sparse [true, nil, false]") - expect(len(c.Colors) == 3 && c.Colors[0] == wv.ColorRed && c.Colors[1] == wv.ColorGreen && c.Colors[2] == wv.ColorBlue, - fmt.Sprintf("colors (got %v)", c.Colors)) - - // Go encodes, producer decodes and compares; then echo and compare. - ok, err = wv.VerifyComposite(c) - expect(err == nil && ok, fmt.Sprintf("verify_composite(sample) (err %v)", err)) - expectSameComposite(wv.RoundtripComposite(c), c, "roundtrip_composite(sample)") - desc := wv.DescribeComposite(c) - expect(strings.Contains(desc, "name: \"héllo wörld ✓\"") && strings.Contains(desc, "some_i64: Some(-9223372036854775808)"), - fmt.Sprintf("describe_composite renders the sample (got %s)", desc)) - - // A one-field change is a mismatch. - c2 := wv.RoundtripComposite(c) - c2.Sparse[1] = ptrBool(true) - ok, err = wv.VerifyComposite(c2) - cerr = nil - expect(!ok && errors.As(err, &cerr) && cerr.Code == wv.CodecErrorMismatch, "verify_composite(changed) is Mismatch") - - // A Composite built from scratch with edge values. - custom := wv.Composite{ - Name: "a\x00b \U0001F600 ünï", - Blob: []byte{}, - SomeI64: ptrI64(math.MaxInt64), - NoneI64: ptrI64(math.MinInt64), - SomeText: nil, - Names: []string{}, - Matrix: [][]int32{{}, {math.MinInt32, math.MaxInt32}}, - Empty: []float64{math.NaN(), math.Inf(1), math.Inf(-1), math.Copysign(0, -1), 5e-324}, - ByName: map[string]int64{"": 0, "k": math.MinInt64, "z": math.MaxInt64}, - ById: map[int32]wv.Scalars{math.MinInt32: extreme, math.MaxInt32: canonicalScalars(), 0: {}}, - Scalars: wv.Scalars{}, - Shape: wv.ShapeEmpty{}, - Shapes: []wv.Shape{}, - MaybeShape: wv.ShapeNested{ - Inner: extreme, - Note: ptrStr(""), - }, - MaybeList: []byte{}, - Sparse: []*bool{nil, nil}, - Colors: []wv.Color{}, - } - echoed := wv.RoundtripComposite(custom) - expectSameComposite(echoed, custom, "roundtrip_composite(custom)") - expect(echoed.MaybeList != nil && len(echoed.MaybeList) == 0, "present-but-empty optional list stays present") - expect(echoed.SomeText == nil, "absent optional string stays absent") - expect(math.IsNaN(echoed.Empty[0]) && math.IsInf(echoed.Empty[1], 1) && math.IsInf(echoed.Empty[2], -1) && - math.Signbit(echoed.Empty[3]) && echoed.Empty[4] == 5e-324, "float specials survive inside a list") - ok, err = wv.VerifyComposite(custom) - cerr = nil - expect(!ok && errors.As(err, &cerr), "the custom composite is not the fixture") - desc = wv.DescribeComposite(custom) - expect(strings.Contains(desc, "name: \"a\\0b 😀 ünï\"") && strings.Contains(desc, "empty: [NaN, inf, -inf, -0.0, 5e-324]"), - fmt.Sprintf("describe_composite renders the custom value (got %s)", desc)) - - // An absent optional list and absent optional shape. - custom.MaybeList = nil - custom.MaybeShape = nil - echoed = wv.RoundtripComposite(custom) - expect(echoed.MaybeList == nil && echoed.MaybeShape == nil, "absent optionals stay absent") - - // ── Shape: every variant, alone and in a list ── - shapes := []wv.Shape{ - wv.ShapeEmpty{}, - wv.ShapeCircle{Radius: math.Inf(1)}, - wv.ShapeRect{Width: -0.0, Height: float32(math.Inf(-1))}, - wv.ShapeLabeled{Label: "✓ label", Count: math.MinInt32}, - wv.ShapeNested{Inner: extreme, Note: nil}, - wv.ShapeNested{Inner: canonicalScalars(), Note: ptrStr("note")}, - } - for i, sh := range shapes { - expect(sameShape(wv.RoundtripShape(sh), sh), fmt.Sprintf("roundtrip_shape[%d] (%+v)", i, sh)) - } - echoedShapes := wv.RoundtripShapes(shapes) - expect(len(echoedShapes) == len(shapes), "roundtrip_shapes length") - for i := range shapes { - expect(sameShape(echoedShapes[i], shapes[i]), fmt.Sprintf("roundtrip_shapes[%d]", i)) - } - expect(len(wv.RoundtripShapes(nil)) == 0, "roundtrip_shapes of nothing") - expect(wv.DescribeShape(wv.ShapeEmpty{}) == "Empty", "describe_shape(Empty)") - expect(wv.DescribeShape(wv.ShapeCircle{Radius: 2.5}) == "Circle { radius: 2.5 }", - fmt.Sprintf("describe_shape(Circle) (got %q)", wv.DescribeShape(wv.ShapeCircle{Radius: 2.5}))) - expect(wv.DescribeShape(wv.ShapeLabeled{Label: "tag", Count: 3}) == "Labeled { label: \"tag\", count: 3 }", - "describe_shape(Labeled)") - switch v := wv.RoundtripShape(wv.ShapeRect{Width: 1, Height: 2}).(type) { - case wv.ShapeRect: - expect(v.Width == 1 && v.Height == 2, "Rect payload") - default: - expect(false, fmt.Sprintf("roundtrip_shape(Rect) returned %T", v)) - } +func literalVectors(n uint32) { + check := func(name string, v codec.Vector) bool { return codec.CheckVector(find(n, name), v) } + + s := canonicalScalars() + expect(check("scalars canonical", codec.VectorAllScalars{Value: s}), "scalars canonical") + s.U16Value = 60001 + expect(!check("scalars canonical", codec.VectorAllScalars{Value: s}), "one changed field") + + labeled := codec.VectorFigure{Value: codec.ShapeLabeled{Label: "tag", Count: 3}} + expect(check("shape labeled", labeled), "shape labeled") + expect(check("string interior nul", codec.VectorText{Value: "nul\x00inside\x00"}), "interior nul") + + // Any NaN matches the NaN vector; zero keeps its sign. + nan := math.Float64frombits(0x7ff8000000000001) + expect(check("f64 nan", codec.VectorF64{Value: nan}), "a NaN payload") + expect(check("f64 -0", codec.VectorF64{Value: math.Copysign(0, -1)}), "-0") + expect(!check("f64 -0", codec.VectorF64{Value: 0}), "+0 is not -0") - // ── Top-level buffered and direct echoes ── - expect(wv.RoundtripOptI64(nil) == nil, "roundtrip_opt_i64(nil)") - o := wv.RoundtripOptI64(ptrI64(math.MinInt64)) - expect(o != nil && *o == math.MinInt64, "roundtrip_opt_i64(MinInt64)") - o = wv.RoundtripOptI64(ptrI64(0)) - expect(o != nil && *o == 0, "roundtrip_opt_i64(0) is present") - m := wv.RoundtripMap(map[string]int64{"": math.MaxInt64, "héllo": -1, "x": 0}) - expect(len(m) == 3 && m[""] == math.MaxInt64 && m["héllo"] == -1 && m["x"] == 0, fmt.Sprintf("roundtrip_map (got %v)", m)) - expect(len(wv.RoundtripMap(nil)) == 0, "roundtrip_map(empty)") - expect(wv.RoundtripString("héllo wörld ✓ 😀") == "héllo wörld ✓ 😀", "roundtrip_string unicode") - expect(wv.RoundtripString("") == "", "roundtrip_string empty") - expect(wv.RoundtripString("a\x00b\x00") == "a\x00b\x00", "roundtrip_string keeps interior NUL bytes") - expect(wv.RoundtripString("\x00") == "\x00", "roundtrip_string of a lone NUL") - expect(sameBytes(wv.RoundtripBytes([]byte{0, 255, 128}), []byte{0, 255, 128}), "roundtrip_bytes") - expect(len(wv.RoundtripBytes([]byte{})) == 0, "roundtrip_bytes empty") - expect(wv.RoundtripI64(math.MinInt64) == math.MinInt64, "roundtrip_i64 MinInt64") - expect(wv.RoundtripI64(math.MaxInt64) == math.MaxInt64, "roundtrip_i64 MaxInt64") - expect(wv.RoundtripU64(math.MaxUint64) == math.MaxUint64, "roundtrip_u64 MaxUint64") - expect(wv.RoundtripU64(1<<63) == 1<<63, "roundtrip_u64 2^63") - expect(math.IsNaN(wv.RoundtripF64(math.NaN())), "roundtrip_f64 NaN") - expect(math.IsInf(wv.RoundtripF64(math.Inf(1)), 1) && math.IsInf(wv.RoundtripF64(math.Inf(-1)), -1), "roundtrip_f64 infinities") - negZero := wv.RoundtripF64(math.Copysign(0, -1)) - expect(negZero == 0 && math.Signbit(negZero), "roundtrip_f64 -0") - expect(wv.RoundtripF64(5e-324) == 5e-324 && wv.RoundtripF64(math.MaxFloat64) == math.MaxFloat64, "roundtrip_f64 subnormal and max") - expect(wv.RoundtripBool(true) && !wv.RoundtripBool(false), "roundtrip_bool") - expect(wv.RoundtripColor(wv.ColorBlue) == wv.ColorBlue && wv.RoundtripColor(wv.ColorGreen) == 1, "roundtrip_color") - - // ── Holder: objects inside buffers ── - h := wv.MakeHolder(10, true) - expect(h.Primary != nil && h.Primary.Value() == 10, "holder.primary value 10") - expect(h.Spare != nil && h.Spare.Value() == 11, "holder.spare value 11") - expect(len(h.Many) == 3 && h.Many[0].Value() == 12 && h.Many[1].Value() == 13 && h.Many[2].Value() == 14, - "holder.many values 12..14") - // Each encoding mints fresh references; the producer adopts and drops - // them, so repeated calls keep the wrappers valid. - expect(wv.SumHolder(h) == 10+11+12+13+14, "sum_holder(holder)") - expect(wv.SumHolder(h) == 60, "sum_holder is repeatable") - expect(h.Primary.Value() == 10, "wrapper alive after being encoded twice") - - // primary_of returns a wrapper to the SAME object as holder.primary. - p := wv.PrimaryOf(h) - expect(p != nil && p != h.Primary, "primary_of yields a distinct wrapper") - expect(p.Value() == 10, "primary_of value 10") - expect(wv.SamePrimary(h, wv.Holder{Primary: p}), "same_primary(holder, {primary_of}) is true") - expect(wv.SamePrimary(h, h), "same_primary(holder, holder) is true") - expect(!wv.SamePrimary(h, wv.Holder{Primary: h.Spare}), "same_primary against the spare is false") - other := wv.MakeHolder(10, true) - expect(!wv.SamePrimary(h, other), "equal values, distinct objects") - - // Without a spare. - bare := wv.MakeHolder(0, false) - expect(bare.Spare == nil, "make_holder(0, false) has no spare") - expect(wv.SumHolder(bare) == 0+2+3+4, "sum_holder without spare") - - // A holder built from Go-constructed tokens. - mine := wv.Holder{ - Primary: wv.NewToken(-5), - Spare: nil, - Many: []*wv.Token{wv.NewToken(1), wv.NewToken(2)}, + expect(check("u64 max", codec.VectorU64{Value: math.MaxUint64}), "u64 max") + expect(check("enum infrared", codec.VectorHue{Value: codec.ColorInfrared}), "enum infrared") + + zero := int64(0) + expect(check("optional zero", codec.VectorMaybeI64{Value: &zero}), "Some(0)") + expect(check("optional absent", codec.VectorMaybeI64{Value: nil}), "None") + expect(!check("optional zero", codec.VectorMaybeI64{Value: nil}), "None is not Some(0)") + + // A map's entry order doesn't matter on the wire. + counts := map[string]int64{"x": 0, "héllo": -1, "": math.MaxInt64} + expect(check("map of strings", codec.VectorCounts{Value: counts}), "map of strings") + expect(check("blank", codec.VectorBlank{}), "blank") + + // The sparse holder, from a consumer-made token. + lone := codec.NewToken(-1) + sparse := codec.VectorObjects{Value: codec.Holder{Primary: lone}} + expect(check("objects sparse", sparse), "objects sparse") + lone.Close() +} + +func spotChecks(n uint32) { + v := fetch(find(n, "i64 past 2^53")).(codec.VectorI64) + expect(v.Value == -9007199254740993, "i64 past 2^53 is exact") + + f := fetch(find(n, "f32 min subnormal")).(codec.VectorF32) + expect(math.Float32bits(f.Value) == 1, "f32 min subnormal bits") + + t := fetch(find(n, "string astral")).(codec.VectorText) + expect(t.Value == "🦀 crab 😀", "string astral") + + m := fetch(find(n, "scalars minimum")).(codec.VectorAllScalars).Value + expect(m.I8Value == math.MinInt8 && m.I16Value == math.MinInt16 && m.I32Value == math.MinInt32, "minimum signed") + expect(m.I64Value == math.MinInt64 && m.U8Value == 0 && m.U16Value == 0 && m.U32Value == 0 && m.U64Value == 0, "minimum") + expect(math.IsInf(float64(m.F32Value), -1) && math.IsNaN(m.F64Value), "minimum floats") + expect(m.Color == codec.ColorInfrared && !m.Flag, "minimum enum and flag") + + c := fetch(find(n, "composite canonical")).(codec.VectorDeep).Value + expect(c.Name == "héllo wörld ✓", "name") + expect(len(c.Blob) == 6 && c.Blob[5] == 255, "blob") + expect(c.SomeI64 != nil && *c.SomeI64 == math.MinInt64 && c.NoneI64 == nil, "optional i64s") + expect(c.SomeText != nil && *c.SomeText == "", "present empty text") + expect(len(c.Names) == 3 && c.Names[1] == "", "names") + expect(len(c.Matrix) == 3 && len(c.Matrix[1]) == 0 && c.Matrix[2][0] == -4, "matrix") + expect(len(c.Floats) == 6 && math.IsNaN(c.Floats[0]) && math.Signbit(c.Floats[3]), "floats") + expect(len(c.ByName) == 4 && len(c.ByID) == 3 && len(c.ByColor) == 2 && len(c.Flags) == 2, "maps") + expect(c.Scalars.U32Value == 4000000000, "nested scalars") + shape, ok := c.Shape.(codec.ShapeLabeled) + expect(ok && shape.Count == 3, "shape") + nested, ok := c.Shapes[5].(codec.ShapeNested) + expect(len(c.Shapes) == 6 && ok && nested.Note == nil, "shapes") + _, ok = c.MaybeShape.(codec.ShapeNested) + expect(ok, "maybe shape") + expect(c.MaybeList != nil && len(c.MaybeList) == 2, "maybe list") + expect(len(c.Sparse) == 3 && c.Sparse[1] == nil && *c.Sparse[0], "sparse") + expect(len(c.Colors) == 4 && c.Colors[3] == codec.ColorInfrared, "colors") +} + +func outOfRange(n uint32) { + v, err := codec.CodecVector(n) + expect(v == nil, "a failed vector call returns nil") + e := expectAs[*codec.OutOfRangeError](err, "vector(n)") + expect(e.Index == n && e.Count == n, fmt.Sprintf("payload index %d count %d", e.Index, e.Count)) + expect(e.Error() == fmt.Sprintf("vector %d is out of range (count %d)", n, n), "message: "+e.Error()) + expect(e.Code() == 1, "code") + expectAs[codec.CodecError](err, "an OutOfRangeError is a CodecError") + + _, err = codec.VectorName(n + 5) + e = expectAs[*codec.OutOfRangeError](err, "vector_name(n + 5)") + expect(e.Index == n+5 && e.Count == n, "vector_name payload") + + expect(!codec.CheckVector(n, codec.VectorBlank{}), "check_vector past the end") +} + +// expectMarshalFailure asserts that f panics with the runtime error for a +// value the library can't take (-3). +func expectMarshalFailure(what string, f func()) { + r := catchPanic(f) + e, ok := r.(*codec.Error) + expect(ok && e.Code == -3, fmt.Sprintf("%s: want a -3 *Error panic, got %v", what, r)) +} + +func malformed() { + // Go can spell values the library rejects: an undeclared enum value and + // a string that isn't UTF-8, directly and inside a buffer. + expectMarshalFailure("echo_color(3)", func() { codec.EchoColor(codec.Color(3)) }) + expectMarshalFailure("echo_text(bad UTF-8)", func() { codec.EchoText("\xc3\x28") }) + expectMarshalFailure("check_vector(Hue 3)", func() { codec.CheckVector(0, codec.VectorHue{Value: 3}) }) + expectMarshalFailure("check_vector(bad UTF-8)", func() { + codec.CheckVector(0, codec.VectorText{Value: "\xc3\x28"}) + }) + expect(codec.EchoText("") == "", "empty text") + + // The encoder refuses a nil sum-type value instead of writing a bad tag. + expect(catchPanic(func() { codec.CheckVector(0, nil) }) != nil, "nil Vector") + expect(catchPanic(func() { codec.CheckVector(0, codec.VectorFigure{}) }) != nil, "nil Shape") +} + +func objects(n uint32) { + // The full object vector decodes to live tokens with the table's values. + full := fetch(find(n, "objects full")).(codec.VectorObjects).Value + expect(full.Primary.Value() == 10 && full.Spare != nil && full.Spare.Value() == 11, "primary and spare") + expect(len(full.Many) == 3 && full.Many[2].Value() == math.MinInt64, "many") + expect(len(full.ByName) == 2 && full.ByName["b"].Value() == 21, "by name") + // Each encoding mints fresh references, so the holder can be sent twice. + want := int64(10+11+12+13+20+21) + math.MinInt64 + expect(codec.SumHolder(full) == want && codec.SumHolder(full) == want, "sum_holder twice") + + // primary_of returns the very same object: identity, not value. + p := codec.PrimaryOf(full) + expect(p.Value() == 10, "primary_of value") + expect(codec.SamePrimary(full, codec.Holder{Primary: p}), "primary_of is the same object") + twin := codec.NewToken(10) + expect(!codec.SamePrimary(full, codec.Holder{Primary: twin}), "an equal value is another object") + + // A holder built from consumer tokens, one wrapper in several slots. + minus := codec.NewToken(-4) + mine := codec.Holder{ + Primary: twin, + Spare: twin, + Many: []*codec.Token{twin, twin, minus}, + ByName: map[string]*codec.Token{"k": twin}, } - expect(mine.Primary.Value() == -5, "new token value") - expect(wv.SumHolder(mine) == -2, "sum_holder(mine)") - mp := wv.PrimaryOf(mine) - expect(mp.Value() == -5 && wv.SamePrimary(mine, wv.Holder{Primary: mp}), "primary_of(mine) aliases the Go-made token") - expect(wv.SumHolder(wv.Holder{Primary: mp, Spare: mp, Many: []*wv.Token{mp, mp}}) == -20, - "the same wrapper may be encoded in several positions at once") - big := wv.NewToken(math.MaxInt64) - expect(big.Value() == math.MaxInt64 && wv.SumHolder(wv.Holder{Primary: big}) == math.MaxInt64, "token carries MaxInt64") - big.Close() - - // Release: closing the holder's wrapper leaves primary_of's reference - // alive; every Close is idempotent; a closed wrapper traps on use and - // can't be encoded. - h.Primary.Close() - h.Primary.Close() - expect(p.Value() == 10, "primary_of's reference survives closing holder.primary") - expect(catchPanic(func() { h.Primary.Value() }) != nil, "use after Close traps") - expect(catchPanic(func() { wv.SumHolder(h) }) != nil, "encoding a closed token traps") - p.Close() + expect(codec.SumHolder(mine) == 10*5-4, "consumer holder sum") + + // Close is idempotent; a closed wrapper can't be used. + minus.Close() + minus.Close() + expect(catchPanic(func() { minus.Value() }) != nil, "use after Close panics") + + twin.Close() p.Close() - h.Spare.Close() - for _, t := range h.Many { - t.Close() - t.Close() - } - other.Primary.Close() - other.Spare.Close() - for _, t := range other.Many { - t.Close() - } - bare.Primary.Close() - for _, t := range bare.Many { - t.Close() - } - mine.Primary.Close() - for _, t := range mine.Many { - t.Close() - } - mp.Close() - mp.Close() + closeHolder(full) +} + +func main() { + n := codec.VectorCount() + expect(n >= 60, fmt.Sprintf("vector_count() = %d", n)) + + everyVector(n) + literalVectors(n) + spotChecks(n) + outOfRange(n) + malformed() + objects(n) - expectNoLeaks(wv.DebugLive) - fmt.Println("go/codec: OK") + expectNoLeaks(codec.DebugLive) + fmt.Printf("go/codec: OK (%d vectors)\n", n) } diff --git a/conformance/go/common.go b/conformance/go/common.go index 5b1dd0f6..87ab6a36 100644 --- a/conformance/go/common.go +++ b/conformance/go/common.go @@ -4,6 +4,7 @@ package main import ( + "errors" "fmt" "os" "runtime" @@ -24,12 +25,22 @@ func catchPanic(f func()) (v any) { return nil } -// expectNoLeaks drives the garbage collector until every finalizer has run -// and then asserts that the producer holds no live objects, callbacks, -// iterators, cancel tokens, or returned allocations. +// expectAs asserts that err matches the error type T and returns the match. +func expectAs[T error](err error, what string) T { + var target T + expect(errors.As(err, &target), fmt.Sprintf("%s: want %T, got %T (%v)", what, target, err, err)) + return target +} + +// expectNoLeaks checks that the library counts live resources, then drives +// the garbage collector until every finalizer has run and asserts that the +// library holds no live objects, callbacks, iterators, cancel tokens, or byte +// runs. Async workers may still be dropping a finished call, so the counters +// get a few seconds to settle. func expectNoLeaks(debugLive func(int32) uint64) { - kinds := []string{"objects", "callbacks", "iterators", "cancel tokens", "allocations"} - deadline := time.Now().Add(10 * time.Second) + expect(debugLive(-1) == 1, "the library counts live resources") + kinds := []string{"objects", "callbacks", "iterators", "cancel tokens", "byte runs"} + deadline := time.Now().Add(5 * time.Second) for { runtime.GC() leaked := "" diff --git a/conformance/go/contacts.go b/conformance/go/contacts.go deleted file mode 100644 index 3ccbabde..00000000 --- a/conformance/go/contacts.go +++ /dev/null @@ -1,115 +0,0 @@ -// Conformance consumer: contacts sample, Go target. -// -// Imports the generated cgo package and asserts the contacts surface: the -// ContactBook interface (factory constructor, methods on the wrapper, explicit -// Close), enum constants, plain value-struct records returned by value and -// decoded from value buffers, optional strings (pointer email) present and -// absent, list-of-record returns, the throws split (plain returns for -// non-throwing methods), and the typed ContactsError domain via errors.As. -// -// Cross-module types: the nested contacts.groups module takes and returns -// the parent's ContactType enum and ContactBook interface and reports the -// parent's ContactsError; the sibling directory root shares the Contact -// record and has its own DirectoryError. Its `card` function keeps the -// module prefix (DirectoryCard) because the Card record owns that name in the -// flat Go package. - -package main - -import ( - "errors" - "fmt" - "sort" - - wv "__MODPATH__" -) - -func main() { - book := wv.NewContactBook() - expect(wv.DebugLive(0) >= 1, "the producer counts live objects (leak checks are on)") - - email := "alice@example.com" - alice, err := book.Add("Alice", "Smith", &email, wv.ContactTypeWork) - expect(err == nil, "add alice") - expect(alice.Id > 0, "alice id positive") - expect(alice.FirstName == "Alice", "first name") - expect(alice.LastName == "Smith", "last name") - expect(alice.Email != nil && *alice.Email == "alice@example.com", "email") - expect(alice.ContactType == wv.ContactTypeWork, "contact type") - - // Typed error: an empty name reports ContactsError InvalidName. - _, err = book.Add("", "Smith", nil, wv.ContactTypePersonal) - var cerr *wv.ContactsError - expect(errors.As(err, &cerr), "empty name yields a *ContactsError") - expect(cerr.Code == wv.ContactsErrorInvalidName, - fmt.Sprintf("invalid name code == 1 (got %d)", cerr.Code)) - expect(cerr.Message == "name must not be empty", "invalid name message") - - // Optional string: a missing email round-trips as a nil pointer. - bob, err := book.Add("Bob", "Jones", nil, wv.ContactTypePersonal) - expect(err == nil, "add bob") - cb, err := book.Get(bob.Id) - expect(err == nil, "get bob") - expect(cb.Email == nil, "bob email nil") - - // Non-throwing methods have plain returns (no error result). - expect(book.Count() == 2, "count == 2") - - all := book.List() - expect(len(all) == 2, "list length == 2") - names := []string{all[0].FirstName, all[1].FirstName} - sort.Strings(names) - expect(names[0] == "Alice" && names[1] == "Bob", "list names") - - expect(book.Remove(alice.Id), "remove returns true") - expect(book.Count() == 1, "count == 1 after remove") - - // Typed error: a missing id reports ContactsError NotFound. - _, err = book.Get(9999) - cerr = nil - expect(errors.As(err, &cerr), "missing contact yields a *ContactsError") - expect(cerr.Code == wv.ContactsErrorNotFound, - fmt.Sprintf("not found code == 2 (got %d)", cerr.Code)) - - // ── contacts.groups: the parent's enum, interface, and error domain ── - _, err = book.Add("Ann", "Lee", nil, wv.ContactTypeWork) - expect(err == nil, "add ann") - _, err = book.Add("Bo", "Kim", nil, wv.ContactTypeWork) - expect(err == nil, "add bo") - expect(wv.CountOfType(book, wv.ContactTypeWork) == 2, "count_of_type(Work) == 2") - expect(wv.CountOfType(book, wv.ContactTypePersonal) == 1, "count_of_type(Personal) == 1") - expect(wv.DominantType(book) == wv.ContactTypeWork, "dominant_type is Work") - - work := wv.SplitByType(book, wv.ContactTypeWork) - expect(work != nil && work.Count() == 2, "split_by_type returns a new book with two contacts") - expect(book.Count() == 3, "the source book is unchanged") - first, err := wv.FirstOfType(work, wv.ContactTypeWork) - expect(err == nil && first.FirstName == "Ann", fmt.Sprintf("first_of_type(Work) (got %+v, %v)", first, err)) - _, err = wv.FirstOfType(work, wv.ContactTypeOther) - cerr = nil - expect(errors.As(err, &cerr) && cerr.Code == wv.ContactsErrorNotFound, - fmt.Sprintf("the nested module reports the parent's NotFound (got %v)", err)) - expect(work.Close() == nil, "close the split book") - - // ── directory: a sibling root sharing the Contact record ── - zoe := wv.Contact{Id: 7, FirstName: "Zoe", LastName: "Quinn", ContactType: wv.ContactTypePersonal} - card := wv.DirectoryCard(zoe) - expect(card == wv.Card{DisplayName: "Quinn, Zoe", Initials: "ZQ", HasEmail: false}, - fmt.Sprintf("directory card (got %+v)", card)) - sorted, err := wv.Sorted(book.List()) - expect(err == nil && len(sorted) == 3, fmt.Sprintf("sorted (got %v)", err)) - expect(sorted[0].LastName == "Jones" && sorted[1].LastName == "Kim" && sorted[2].LastName == "Lee", - fmt.Sprintf("sorted by last name (got %+v)", sorted)) - _, err = wv.Sorted(nil) - var derr *wv.DirectoryError - expect(errors.As(err, &derr) && derr.Code == wv.DirectoryErrorEmpty, - fmt.Sprintf("empty list yields *DirectoryError (got %T %v)", err, err)) - expect(derr.Message == "no contacts to list", "directory error message") - - // Close is idempotent, and a closed wrapper traps on use. - expect(book.Close() == nil && book.Close() == nil, "double close") - expect(catchPanic(func() { book.Count() }) != nil, "use after Close panics") - - expectNoLeaks(wv.DebugLive) - fmt.Println("go/contacts: OK") -} diff --git a/conformance/go/events.go b/conformance/go/events.go deleted file mode 100644 index 6662342a..00000000 --- a/conformance/go/events.go +++ /dev/null @@ -1,264 +0,0 @@ -// Conformance consumer: events sample, Go target. -// -// Exercises the callback surface end to end: a Go type implementing the -// generated Subscriber callback interface (crossing as a cgo.Handle plus a -// static vtable of //export trampolines), the reference-counted EventBus -// object (including a second wrapper handed back through OnAttached, its -// explicit Close, and a harmless double Close), Delivery return values -// steering Publish's accepted count, the Message record decoded from a -// borrowed value buffer inside a callback, the async PublishLater bridge, -// the lazy iter.Seq behind Messages, the optional LastMessage, the free -// function RouteOnce, and the foreign-error path: a subscriber that panics -// surfaces to the caller as a recoverable *Error with code -4 and -// leaves the bus usable. Route returns Result on the -// producer side, so its failure comes back to the bus as a value that the -// bus propagates; OnMessage returns a plain value, so its failure aborts the -// call directly. Both look the same from Go. Finally ClearSubscribers must let the producer's -// `free` entry run, which is observed through a Go finalizer on the -// implementation once its handle is deleted, and nothing leaks. Exits 0 on -// success; aborts (non-zero) on any mismatch. - -package main - -import ( - "context" - "errors" - "fmt" - "runtime" - "strings" - "sync/atomic" - "time" - - wv "__MODPATH__" -) - -// subState is the observable side of a subscriber. It's held separately -// from the implementing value so the implementation itself can become -// unreachable (and be finalized) once the producer frees its handle. -type subState struct { - name string - skip string - fails bool - routed []string - received []wv.Message - attached int - bus *wv.EventBus - freed *atomic.Int32 -} - -// recorder implements wv.Subscriber. The subscriber's configured skip topic -// is skipped and "stop" accepts and stops later subscribers. A subscriber -// marked `fails` panics inside Route for "boom-route" and inside OnMessage -// for "boom-message"; the others accept both, so the earlier subscribers -// still receive them. -type recorder struct { - st *subState -} - -func (r *recorder) Route(topic string) wv.Delivery { - r.st.routed = append(r.st.routed, topic) - switch topic { - case r.st.skip: - return wv.DeliverySkip - case "stop": - return wv.DeliveryAcceptAndStop - case "boom-route": - if r.st.fails { - panic("subscriber " + r.st.name + " rejected topic") - } - } - return wv.DeliveryAccept -} - -func (r *recorder) OnMessage(message wv.Message) int64 { - if message.Topic == "boom-message" && r.st.fails { - panic(fmt.Errorf("subscriber %s failed on message %d", r.st.name, message.Seq)) - } - r.st.received = append(r.st.received, message) - return int64(len(r.st.received)) -} - -func (r *recorder) OnAttached(bus *wv.EventBus) { - r.st.attached++ - // The reference is ours; keep the wrapper so main can prove it aliases - // the bus that called subscribe. - r.st.bus = bus -} - -// closingSub closes a bus wrapper from inside a callback, while the call -// that triggered the callback is still using that wrapper. -type closingSub struct { - target *wv.EventBus -} - -func (c *closingSub) Route(string) wv.Delivery { - expect(c.target.Close() == nil, "close inside a callback") - return wv.DeliveryAccept -} - -func (c *closingSub) OnMessage(wv.Message) int64 { return 1 } - -func (c *closingSub) OnAttached(bus *wv.EventBus) { bus.Close() } - -// attach subscribes a fresh recorder and returns only its state. The -// recorder value stays reachable solely through the cgo.Handle the -// producer holds, so its finalizer fires once the producer calls `free`. -func attach(bus *wv.EventBus, name, skip string, fails bool, freed *atomic.Int32) (*subState, int64) { - st := &subState{name: name, skip: skip, fails: fails, freed: freed} - r := &recorder{st: st} - runtime.SetFinalizer(r, func(*recorder) { freed.Add(1) }) - return st, bus.Subscribe(r) -} - -// expectForeignError asserts that v is the generated brand error carrying -// FOREIGN_ERROR_CODE (-4) and the Go panic text. -func expectForeignError(v any, needle string, what string) { - expect(v != nil, what+": expected a panic") - err, isErr := v.(error) - expect(isErr, fmt.Sprintf("%s: panic value is an error (got %T)", what, v)) - var ferr *wv.Error - expect(errors.As(err, &ferr), fmt.Sprintf("%s: *Error (got %T: %v)", what, v, v)) - expect(ferr.Code == -4, fmt.Sprintf("%s: code -4 (got %d)", what, ferr.Code)) - expect(strings.Contains(ferr.Message, needle), - fmt.Sprintf("%s: message carries the Go panic text (got %q)", what, ferr.Message)) -} - -// waitFreed drives the GC until n implementations have been finalized. -func waitFreed(freed *atomic.Int32, n int32) bool { - for i := 0; i < 200; i++ { - runtime.GC() - if freed.Load() >= n { - return true - } - time.Sleep(5 * time.Millisecond) - } - return freed.Load() >= n -} - -func main() { - var freed atomic.Int32 - bus := wv.NewEventBus() - expect(bus.SubscriberCount() == 0, "fresh bus has no subscribers") - expect(bus.LastMessage() == nil, "fresh bus has no last message") - - // Subscribe: the producer calls OnAttached with a strong reference to - // the bus before retaining the subscriber, then returns the new count. - a, n := attach(bus, "a", "quiet", false, &freed) - expect(n == 1, fmt.Sprintf("first subscribe returns 1 (got %d)", n)) - expect(a.attached == 1, "OnAttached called exactly once") - expect(a.bus != nil, "OnAttached received a bus wrapper") - expect(a.bus.SubscriberCount() == 1, "the attached wrapper aliases the live bus") - - b, n := attach(bus, "b", "", false, &freed) - expect(n == 2, fmt.Sprintf("second subscribe returns 2 (got %d)", n)) - expect(bus.SubscriberCount() == 2, "subscriber_count == 2") - - // Delivery steers the accepted count: both accept "news". - delivered := bus.Publish("news", "hello", []string{"x", "yy"}) - expect(delivered == 2, fmt.Sprintf("news accepted by both (got %d)", delivered)) - expect(len(a.received) == 1 && len(b.received) == 1, "both subscribers received news") - m := a.received[0] - expect(m.Seq == 1, fmt.Sprintf("seq starts at 1 (got %d)", m.Seq)) - expect(m.Topic == "news" && m.Text == "hello", "message topic/text") - expect(len(m.Tags) == 2 && m.Tags[0] == "x" && m.Tags[1] == "yy", - fmt.Sprintf("tags round-trip through the borrowed buffer (got %v)", m.Tags)) - expect(a.routed[0] == "news" && b.routed[0] == "news", "route called with the topic") - - // "quiet": a skips, b accepts. - delivered = bus.Publish("quiet", "psst", nil) - expect(delivered == 1, fmt.Sprintf("quiet accepted by one (got %d)", delivered)) - expect(len(a.received) == 1, "a skipped quiet") - expect(len(b.received) == 2 && b.received[1].Seq == 2 && len(b.received[1].Tags) == 0, - "b received quiet with seq 2 and empty tags") - - // "stop": a accepts and stops, so b is never routed. - delivered = bus.Publish("stop", "last", []string{}) - expect(delivered == 1, fmt.Sprintf("stop accepted by one (got %d)", delivered)) - expect(len(a.received) == 2 && a.received[1].Text == "last", "a received stop") - expect(len(b.routed) == 2, fmt.Sprintf("b not routed after AcceptAndStop (got %v)", b.routed)) - - // The wrapper handed through OnAttached sees the same log. - last := a.bus.LastMessage() - expect(last != nil && last.Seq == 3 && last.Topic == "stop" && last.Text == "last", - fmt.Sprintf("last_message via the attached wrapper (got %+v)", last)) - - // Async: the producer publishes from its own thread, and the callback - // trampolines run there; the wrapper parks the goroutine until done. - delivered, err := bus.PublishLater(context.Background(), "async", "later") - expect(err == nil && delivered == 2, fmt.Sprintf("publish_later accepted by both (got %d)", delivered)) - expect(len(a.received) == 3 && a.received[2].Seq == 4 && a.received[2].Topic == "async", - "a received the async message") - expect(len(b.received) == 3 && b.received[2].Text == "later", "b received the async message") - - // Iterator: lazy iter.Seq over every message text, in order. - var texts []string - for t := range bus.Messages() { - texts = append(texts, t) - } - expect(len(texts) == 4 && texts[0] == "hello" && texts[1] == "psst" && - texts[2] == "last" && texts[3] == "later", - fmt.Sprintf("messages iterator in order (got %v)", texts)) - first := "" - for t := range bus.Messages() { - first = t - break - } - expect(first == "hello", "early break yields only the first message") - - last = bus.LastMessage() - expect(last != nil && last.Seq == 4 && last.Text == "later" && len(last.Tags) == 0, - fmt.Sprintf("last_message optional present (got %+v)", last)) - - // Free function taking the callback interface: the producer frees the - // handle right after the call. - probe := &recorder{st: &subState{name: "probe", skip: "quiet", fails: true}} - expect(wv.RouteOnce(probe, "quiet") == wv.DeliverySkip, "route_once quiet -> Skip") - expect(wv.RouteOnce(probe, "stop") == wv.DeliveryAcceptAndStop, "route_once stop -> AcceptAndStop") - expect(wv.RouteOnce(probe, "other") == wv.DeliveryAccept, "route_once other -> Accept") - expect(len(probe.st.routed) == 3, "route_once called Route each time") - expectForeignError(catchPanic(func() { wv.RouteOnce(probe, "boom-route") }), - "subscriber probe rejected topic", "route_once foreign error") - - // Foreign errors through the bus: a Go panic inside Route or OnMessage - // aborts publish with code -4, and the bus stays usable afterwards. - f, n := attach(bus, "f", "", true, &freed) - expect(n == 3 && f.attached == 1, "third subscriber attached") - expectForeignError(catchPanic(func() { bus.Publish("boom-route", "x", nil) }), - "subscriber f rejected topic", "publish with panicking Route") - expectForeignError(catchPanic(func() { bus.Publish("boom-message", "y", nil) }), - "subscriber f failed on message", "publish with panicking OnMessage") - // The bus logged both aborted publishes and still delivers. - last = bus.LastMessage() - expect(last != nil && last.Seq == 6 && last.Topic == "boom-message", "aborted publishes were logged") - delivered = bus.Publish("ok", "z", nil) - expect(delivered == 3, fmt.Sprintf("bus usable after foreign error (got %d)", delivered)) - expect(len(f.received) == 1 && f.received[0].Seq == 7, "f received the follow-up message") - expect(bus.SubscriberCount() == 3, "subscriber_count == 3") - - // Releasing subscribers runs each `free`, which deletes the cgo.Handle - // and lets the Go implementation be collected. - bus.ClearSubscribers() - expect(bus.SubscriberCount() == 0, "subscriber_count == 0 after clear") - expect(waitFreed(&freed, 3), fmt.Sprintf("all three subscribers freed (got %d)", freed.Load())) - delivered = bus.Publish("after", "nobody", nil) - expect(delivered == 0, "no subscribers accept after clear") - - // Close racing an in-flight call: the subscriber closes the wrapper - // Publish is running on; the reference is released once Publish returns. - racy := wv.NewEventBus() - racy.Subscribe(&closingSub{target: racy}) - expect(racy.Publish("x", "y", nil) == 1, "publish completes although its wrapper closed mid-call") - expect(catchPanic(func() { racy.SubscriberCount() }) != nil, "the closed wrapper traps afterwards") - - // Every wrapper releases its own reference; Close is idempotent. - a.bus.Close() - a.bus.Close() - expect(bus.SubscriberCount() == 0, "bus outlives the attached wrapper's release") - b.bus.Close() - f.bus.Close() - bus.Close() - bus.Close() - - expectNoLeaks(wv.DebugLive) - fmt.Println("go/events: OK") -} diff --git a/conformance/go/kvstore.go b/conformance/go/kvstore.go index 6eac63f9..89cfb965 100644 --- a/conformance/go/kvstore.go +++ b/conformance/go/kvstore.go @@ -1,388 +1,694 @@ -// Conformance consumer: kvstore sample, Go target. +// Conformance consumer: kvstore sample, Go target (ABI revision 4). // -// Exercises the Store interface end to end: the throwing factory constructor -// (OpenStore), methods on the wrapper across every shape (sync put/get/delete, -// the iterator-backed ListKeys, plain count/clear, the async cancellable -// Compact, the deprecated LegacyPut), the package-level statics -// (StoreDefaultCapacity, StoreOpenMany, StoreTotalCount), and the explicit, -// idempotent Close. Asserts the typed KvError domain via errors.As (IoError -// on an empty open path, KeyNotFound on a missing get, Expired on a stale -// get). Covers the Entry record decoded from a value buffer (bytes, optional -// present and absent, empty list, and empty map fields), the buffered -// optional TTL and prefix parameters, and the nested kv.stats submodule -// borrowing the Store across the module boundary. +// Drives the feature-complete producer through the generated bindings: // -// Callbacks and objects: a Go type implementing the EvictionListener callback -// interface (called with the decoded Entry and the EvictionReason, its bool -// return detaching it, replacement and clear releasing the old handle, and a -// panicking implementation surfacing to the throwing caller as a -// *Error with code -4), plus reference-counted objects everywhere: -// Share() aliasing the same store, Fork() copying it, Store? both ways in -// Larger, a Store inside the StoreInfo record (Describe), a list of stores -// returned by StoreOpenMany, and stores encoded into a parameter buffer by -// StoreTotalCount. Exits 0 on success; aborts (non-zero) on any mismatch. +// - the load-time checks (ABI revision, both modules' contract tables), +// which importing the package performs; +// - Store: the throwing OpenStore and plain NewStore factories, methods, +// statics (StoreOpenMany, StoreDefaultCapacity, ...), the deprecated +// Size, the Entry and StoreInfo structs, the EntryKind constants, maps, +// optionals as pointers, and the logical clock; +// - KvError code types with their payload fields (*KeyNotFoundError, +// *ExpiredError, *StoreFullError, *InvalidPathError, *RejectedError), +// matched with errors.As, and runtime failures (-3, -4) as *Error; +// - lazy iter.Seq2 and iter.Seq sequences of strings (throwing at launch), +// records, and objects, including one abandoned part-way; +// - three callback interfaces implemented in Go: a Listener (retained, +// filtered by Accepts, told about every Change, detached when it +// panics, and called on a producer thread during compaction), a Policy +// (a record return, a throwing method whose *RejectedError reaches the +// Put caller with its payload, a plain error that arrives as -4, a +// malformed record return and a nil required object rejected as -3, an +// object parameter and object return), and a Loader passed as an +// optional callback (string, bytes, and optional-object returns; typed +// errors decoded by the producer or passed through); +// - Store objects in every position: parameter, return, optional, list, +// map value, record field, sequence element, async result, and callback +// parameter and return; +// - async calls that block on a context.Context: an async free function +// returning an object, a cancellable method cancelled mid-pause +// (returning context.Canceled at once while its background work stops +// cooperatively, shown by StoreActiveJobs), an async list launched from +// 32 goroutines, and an async function in the nested kv.stats module; +// - the sibling report root (the shared Entry record and its own error +// domain). +// +// Releases of consumer callbacks are observed through the producer's +// callback counter (DebugLive(1)). Ends by asserting the producer's leak +// counters are zero. package main +/* +#include + +static pthread_t main_thread; +static void remember_main_thread(void) { main_thread = pthread_self(); } +static int on_main_thread(void) { return pthread_equal(pthread_self(), main_thread); } +*/ +import "C" + import ( "context" "errors" "fmt" "runtime" - "sort" + "slices" "strings" "sync" "sync/atomic" "time" - wv "__MODPATH__" + kv "__MODPATH__" ) -type eviction struct { - key string - value []byte - reason wv.EvictionReason +func init() { + // Keep main on the process's main thread so a callback can tell the + // calling thread from a producer thread. + runtime.LockOSThread() + C.remember_main_thread() +} + +func ptr[T any](v T) *T { return &v } + +// liveCallbacks is the number of consumer callbacks the producer holds. +func liveCallbacks() uint64 { return kv.DebugLive(1) } + +func put(s *kv.Store, key, value string) kv.Entry { + e, err := s.Put(key, []byte(value), kv.EntryKindPersistent, nil) + expect(err == nil, fmt.Sprintf("put(%s): %v", key, err)) + return e } -// listenerState is the observable side of a listener, held apart from the -// implementing value so that value can be finalized once the producer -// releases its handle. -type listenerState struct { - seen []eviction - limit int +func putKind(s *kv.Store, key, value string, kind kv.EntryKind, ttl *int64) (kv.Entry, error) { + return s.Put(key, []byte(value), kind, ttl) } -// watcher implements wv.EvictionListener. It keeps receiving until it has -// seen `limit` evictions, then returns false to detach itself. A key -// starting with "boom" makes it panic instead. -type watcher struct { - st *listenerState +// sameStore reports whether a and b wrap the same native object: a write +// through one is visible through the other. +func sameStore(a, b *kv.Store) bool { + const probe = "\x00identity-probe" + put(a, probe, "") + same := b.Find(probe) != nil + a.Delete(probe) + return same } -func (l *watcher) OnEvict(entry wv.Entry, reason wv.EvictionReason) bool { - if strings.HasPrefix(entry.Key, "boom") { - panic("listener refused " + entry.Key) +func expectKeyNotFound(err error, key string) { + e := expectAs[*kv.KeyNotFoundError](err, "KeyNotFound "+key) + expect(e.Code() == 1001 && e.Key == key, fmt.Sprintf("KeyNotFound key %q (got %q)", key, e.Key)) +} + +// expectRuntime asserts that err is the runtime *Error with code. +func expectRuntime(err error, code int32, message string) { + e := expectAs[*kv.Error](err, fmt.Sprintf("code %d", code)) + expect(e.Code == code && e.Message == message, fmt.Sprintf("want %d %q, got %d %q", code, message, e.Code, e.Message)) + var domain kv.KvError + expect(!errors.As(err, &domain), "a runtime failure isn't a KvError") +} + +// ── listener (consumer-implemented, retained) ───────────────────────────── + +type recordingListener struct { + skip, failOn string + mu sync.Mutex + changes []kv.Change + offMain atomic.Int32 +} + +func (l *recordingListener) Accepts(key string) bool { + if l.failOn != "" && key == l.failOn { + panic("listener refused") } - l.st.seen = append(l.st.seen, eviction{key: entry.Key, value: entry.Value, reason: reason}) - return len(l.st.seen) < l.st.limit + return key != l.skip +} + +func (l *recordingListener) OnChange(change kv.Change) { + if C.on_main_thread() == 0 { + l.offMain.Add(1) + } + l.mu.Lock() + defer l.mu.Unlock() + l.changes = append(l.changes, change) +} + +func (l *recordingListener) last() kv.Change { + l.mu.Lock() + defer l.mu.Unlock() + return l.changes[len(l.changes)-1] } -// listen attaches a fresh watcher and returns only its state; the watcher -// itself stays reachable solely through the producer's cgo.Handle. -func listen(store *wv.Store, limit int, freed *atomic.Int32) *listenerState { - st := &listenerState{limit: limit} - l := &watcher{st: st} - runtime.SetFinalizer(l, func(*watcher) { freed.Add(1) }) - store.SetEvictionListener(l) - return st +func (l *recordingListener) puts() []kv.ChangePut { + l.mu.Lock() + defer l.mu.Unlock() + var out []kv.ChangePut + for _, c := range l.changes { + if p, ok := c.(kv.ChangePut); ok { + out = append(out, p) + } + } + return out } -func waitFreed(freed *atomic.Int32, n int32) bool { - for i := 0; i < 200; i++ { - runtime.GC() - if freed.Load() >= n { - return true +func (l *recordingListener) removed(expired bool) int { + l.mu.Lock() + defer l.mu.Unlock() + n := 0 + for _, c := range l.changes { + if r, ok := c.(kv.ChangeRemoved); ok && r.Expired == expired { + n++ } - time.Sleep(5 * time.Millisecond) } - return freed.Load() >= n + return n } -func put(store *wv.Store, key string, value []byte, ttl *int64) { - ok, err := store.Put(key, value, wv.EntryKindPersistent, ttl) - expect(err == nil && ok, fmt.Sprintf("put %s (err %v)", key, err)) +// ── policy (consumer-implemented, rich returns, throws) ─────────────────── + +// testPolicy routes "b/" keys to other, a reference it owns. +type testPolicy struct { + other *kv.Store + admitted atomic.Int32 } -func main() { - store, err := wv.OpenStore("/tmp/conformance-kvstore-go") - expect(err == nil, "open store") - - // Typed error: an empty path reports KvError IoError. - _, err = wv.OpenStore("") - var kerr *wv.KvError - expect(errors.As(err, &kerr), "empty path yields a *KvError") - expect(kerr.Code == wv.KvErrorIoError, - fmt.Sprintf("empty path code == 1004 (got %d)", kerr.Code)) - expect(kerr.Message == "I/O failure", "io error default message") - - // Static: package-level func namespaced by the type, plain return. - expect(wv.StoreDefaultCapacity() == 1_000_000, "default capacity") - - payload := []byte{1, 2, 3} - ok, err := store.Put("alpha", payload, wv.EntryKindPersistent, nil) - expect(err == nil && ok, "put alpha") - ok, err = store.Put("beta", payload, wv.EntryKindVolatile, nil) - expect(err == nil && ok, "put beta") - - // Non-throwing method: plain return. - expect(store.Count() == 2, "count == 2") - - // Optional record return through a throwing method: the value buffer - // decodes into a plain Entry struct (scalars, bytes, the absent optional, - // and the empty list and map fields). - e, err := store.Get("alpha") - expect(err == nil && e != nil, "get alpha") - expect(e.Id > 0, "entry id positive") - expect(e.Key == "alpha", "entry key") - expect(len(e.Value) == 3 && e.Value[0] == 1 && e.Value[2] == 3, "entry value bytes") - expect(e.CreatedAt > 0, "entry created_at set") - expect(e.ExpiresAt == nil, "no ttl decodes as nil expires_at") - expect(len(e.Tags) == 0, "empty tags len 0") - expect(len(e.Metadata) == 0, "empty metadata len 0") - - // Typed error: a missing key reports KvError KeyNotFound. - _, err = store.Get("missing") - kerr = nil - expect(errors.As(err, &kerr), "missing key yields a *KvError") - expect(kerr.Code == wv.KvErrorKeyNotFound, - fmt.Sprintf("missing key code == 1001 (got %d)", kerr.Code)) - expect(kerr.Message == "key not found", "key not found default message") - - // Iterator-backed method: a lazy iter.Seq2[string, error], with and - // without the buffered optional prefix. Errors surface per step through - // the second value. +func (p *testPolicy) Admit(entry kv.Entry) (kv.Entry, error) { + p.admitted.Add(1) + expect(entry.Version == 0, "the store assigns the version after admission") + switch { + case strings.HasPrefix(entry.Key, "secret"): + return kv.Entry{}, &kv.RejectedError{Key: entry.Key, Reason: "no secrets", Message: "secrets are not stored"} + case strings.HasPrefix(entry.Key, "boom"): + return kv.Entry{}, errors.New("policy exploded") + case strings.HasPrefix(entry.Key, "garbage"): + // Go can spell a record the producer can't read: a tag that isn't + // UTF-8. + entry.Tags = []string{"\xff"} + return entry, nil + } + // Tag it and store it encrypted (and try to rename it, which the store + // ignores). + entry.Key = "renamed" + entry.Kind = kv.EntryKindEncrypted + entry.Tags = []string{"admitted"} + return entry, nil +} + +// Route receives home as a wrapper of its own; the bindings hand the +// producer a fresh reference to whatever it returns. +func (p *testPolicy) Route(key string, home *kv.Store) *kv.Store { + switch { + case strings.HasPrefix(key, "b/"): + home.Close() + return p.other + case strings.HasPrefix(key, "null/"): + home.Close() + return nil // a required object may not be nil: -3 + } + return home +} + +// ── loader (consumer-implemented, passed as an optional parameter) ──────── + +type testLoader struct{ backup *kv.Store } + +func (testLoader) Name() string { return "go-loader" } + +func (l testLoader) Fallback(key string) *kv.Store { + if key == "fb" { + return l.backup + } + return nil +} + +func (testLoader) Load(key string) ([]byte, error) { + switch key { + case "missing": + return nil, &kv.KeyNotFoundError{Key: "missing", Message: "not in the loader"} + case "elsewhere": + return nil, &kv.KeyNotFoundError{Key: "other", Message: "not in the loader"} + case "broken": + return nil, errors.New("loader is broken") + } + return []byte("loaded:" + key), nil +} + +// ── sections ────────────────────────────────────────────────────────────── + +func constructors() { + s, err := kv.OpenStore("") + expect(s == nil, "a failed OpenStore returns nil") + e := expectAs[*kv.InvalidPathError](err, `OpenStore("")`) + expect(e.Code() == 1004 && e.Error() == "invalid path", "InvalidPath code and message") + expectAs[kv.KvError](err, "an InvalidPathError is a KvError") + + s = kv.NewStore() + expect(s.Path() == "memory", "NewStore path") + expect(s.Capacity() == kv.StoreDefaultCapacity() && kv.StoreDefaultCapacity() == 1000000, "default capacity") + s.Close() + + ctx := context.Background() + a, err := kv.KvOpenStore(ctx, "/async") + expect(err == nil && a.Path() == "/async", fmt.Sprintf("KvOpenStore: %v", err)) + a.Close() + _, err = kv.KvOpenStore(ctx, "") + expectAs[*kv.InvalidPathError](err, `KvOpenStore("")`) +} + +func basics() { + s, err := kv.OpenStore("/basics") + expect(err == nil, "OpenStore") + defer s.Close() + + // Put returns the stored entry; the version counts puts of the key. + e := put(s, "alpha", "one") + expect(e.Key == "alpha" && string(e.Value) == "one" && e.Kind == kv.EntryKindPersistent, "put alpha") + expect(e.Version == 1 && e.ExpiresAt == nil && len(e.Tags) == 0 && len(e.Metadata) == 0, "put alpha fields") + e, err = putKind(s, "alpha", "two", kv.EntryKindVolatile, nil) + expect(err == nil && e.Version == 2 && e.Kind == kv.EntryKindVolatile, "put alpha again") + + got, err := s.Get("alpha") + expect(err == nil && string(got.Value) == "two", "get alpha") + _, err = s.Get("nope") + expectKeyNotFound(err, "nope") + expect(err.Error() == "key not found: nope", "KeyNotFound message: "+err.Error()) + found := s.Find("alpha") + expect(found != nil && found.Version == 2, "find alpha") + expect(s.Find("nope") == nil, "find nope") + + // TTLs follow the logical clock; an expired get reports when. + e, err = putKind(s, "ttl", "x", kv.EntryKindVolatile, ptr(int64(10))) + expect(err == nil && e.ExpiresAt != nil && *e.ExpiresAt == 10, "ttl expires_at") + expect(s.Now() == 0, "the clock starts at 0") + expect(s.Tick(9) == 9 && s.Count() == 2, "tick 9") + expect(s.Tick(1) == 10 && s.Count() == 1, "tick 10") + _, err = s.Get("ttl") + expired := expectAs[*kv.ExpiredError](err, "get(ttl)") + expect(expired.Key == "ttl" && expired.ExpiredAt == 10, "Expired payload") + _, err = s.Get("ttl") + expectKeyNotFound(err, "ttl") // the expired read removed it + + // Capacity: a new key past it is StoreFull { capacity }. + s.SetCapacity(1) + expect(s.Capacity() == 1, "SetCapacity") + put(s, "alpha", "three") // replacing is fine + _, err = putKind(s, "beta", "b", kv.EntryKindVolatile, nil) + full := expectAs[*kv.StoreFullError](err, "put beta at capacity") + expect(full.Capacity == 1, "StoreFull capacity") + s.SetCapacity(100) + + // An undeclared enum value and invalid UTF-8 are marshalling failures. + _, err = putKind(s, "k", "v", kv.EntryKind(9), nil) + expect(expectAs[*kv.Error](err, "an undeclared EntryKind").Code == -3, "an undeclared enum value is -3") + _, err = putKind(s, "\xc3\x28", "v", kv.EntryKindVolatile, nil) + expect(expectAs[*kv.Error](err, "bad UTF-8 key").Code == -3, "bad UTF-8 key is -3") + + // Delete, Clear, and the deprecated Size. + put(s, "beta", "b") + expect(s.Delete("beta") && !s.Delete("beta"), "delete twice") + size := s.Size() //nolint:staticcheck // the deprecated method still works + expect(size == s.Count() && size == 1, "deprecated Size == Count") + expect(s.Clear() == 1 && s.Count() == 0, "clear") +} + +func iterators() { + s, _ := kv.OpenStore("/iter") + defer s.Close() + put(s, "user.bob", "b") + put(s, "user.alice", "a") + put(s, "sys.x", "xx") + var keys []string - for k, serr := range store.ListKeys(nil) { - expect(serr == nil, "list_keys step error") + for k, err := range s.Keys(nil) { + expect(err == nil, fmt.Sprintf("keys: %v", err)) keys = append(keys, k) } - expect(len(keys) == 2, "list_keys len == 2") - sort.Strings(keys) - expect(keys[0] == "alpha" && keys[1] == "beta", "list_keys values") - - prefix := "al" - keys = keys[:0] - for k, serr := range store.ListKeys(&prefix) { - expect(serr == nil, "list_keys prefix step error") - keys = append(keys, k) + expect(slices.Equal(keys, []string{"sys.x", "user.alice", "user.bob"}), fmt.Sprintf("keys in order (got %v)", keys)) + expect(kv.DebugLive(2) == 0, "an exhausted iterator is released") + + n := 0 + for k, err := range s.Keys(ptr("zzz")) { + expect(k == "", "a failed sequence yields the zero value") + expectKeyNotFound(err, "zzz") + n++ } - expect(len(keys) == 1 && keys[0] == "alpha", "list_keys prefix filter") + expect(n == 1, "a failed launch yields one error") - // Early break destroys the producer iterator without draining it. - first := "" - for k, serr := range store.ListKeys(nil) { - expect(serr == nil, "list_keys early-break step error") - first = k + // Abandoning a sequence part-way releases its iterator. + for k, err := range s.Keys(ptr("user.")) { + expect(err == nil && k == "user.alice", "first user key") + expect(kv.DebugLive(2) == 1, "the iterator is live") break } - expect(first == "alpha", "early break yields the first sorted key") - - // Deprecated member keeps working. - ok, err = store.LegacyPut("legacy", payload) - expect(err == nil && ok, "legacy put") - ok, err = store.Delete("legacy") - expect(err == nil && ok, "delete legacy") - ok, err = store.Delete("legacy") - expect(err == nil && !ok, "second delete reports false") - - // Present optional: a TTL'd put round-trips as a non-nil ExpiresAt - // pointing past CreatedAt. - put(store, "gamma", payload, ptrInt64(3600)) - g, err := store.Get("gamma") - expect(err == nil && g != nil, "get gamma") - expect(g.ExpiresAt != nil, "ttl decodes as non-nil expires_at") - expect(*g.ExpiresAt == g.CreatedAt+3600, "expires_at == created_at + ttl") - ok, err = store.Delete("gamma") - expect(err == nil && ok, "delete gamma") - - // kv.stats submodule borrows the Store across the module boundary and - // returns the Stats record by value. - st, err := wv.GetStats(store) - expect(err == nil, "get stats") - expect(st.TotalEntries == 2, "stats total entries == 2") - expect(st.TotalBytes == 6, fmt.Sprintf("stats total bytes == 6 (got %d)", st.TotalBytes)) - expect(st.ExpiredEntries == 0, "stats expired entries == 0") - - // ── Eviction listener (callback interface) ── - var freed atomic.Int32 - l1 := listen(store, 2, &freed) - - // Delete fires the trampoline synchronously with the decoded Entry. - ok, err = store.Delete("beta") - expect(err == nil && ok, "delete beta") - expect(len(l1.seen) == 1, fmt.Sprintf("eviction fired once (got %v)", l1.seen)) - expect(l1.seen[0].key == "beta" && l1.seen[0].reason == wv.EvictionReasonDeleted, - fmt.Sprintf("delete evicts with reason Deleted (got %+v)", l1.seen[0])) - expect(len(l1.seen[0].value) == 3 && l1.seen[0].value[1] == 2, "evicted entry carries its bytes") - - // A stale read evicts with reason Expired and reports KvError Expired. - put(store, "stale", []byte{9}, ptrInt64(-1)) - _, err = store.Get("stale") - kerr = nil - expect(errors.As(err, &kerr) && kerr.Code == wv.KvErrorExpired, - fmt.Sprintf("stale get reports Expired (got %v)", err)) - expect(len(l1.seen) == 2 && l1.seen[1].key == "stale" && l1.seen[1].reason == wv.EvictionReasonExpired, - fmt.Sprintf("expiry evicts with reason Expired (got %v)", l1.seen)) - - // The second eviction returned false, so the store detached and freed - // the listener; a third eviction is not observed. - expect(waitFreed(&freed, 1), "detached listener is freed") - put(store, "unseen", payload, nil) - ok, err = store.Delete("unseen") - expect(err == nil && ok, "delete unseen") - expect(len(l1.seen) == 2, "no eviction after the listener detached") - - // Replacing a listener frees the previous one; clearing frees the last. - l2 := listen(store, 1000, &freed) - l3 := listen(store, 1000, &freed) - expect(waitFreed(&freed, 2), "replaced listener is freed") - put(store, "delta", payload, nil) - ok, err = store.Delete("delta") - expect(err == nil && ok, "delete delta") - expect(len(l2.seen) == 0 && len(l3.seen) == 1 && l3.seen[0].key == "delta", - "only the current listener observes evictions") - store.ClearEvictionListener() - expect(waitFreed(&freed, 3), "cleared listener is freed") - put(store, "epsilon", payload, nil) - ok, err = store.Delete("epsilon") - expect(err == nil && ok, "delete epsilon") - expect(len(l3.seen) == 1, "no eviction after clear") - store.ClearEvictionListener() - - // A panicking listener surfaces to the throwing caller as the generic - // error with FOREIGN_ERROR_CODE, and the store keeps working. - l4 := listen(store, 1000, &freed) - put(store, "boom", payload, nil) - _, err = store.Delete("boom") - var ferr *wv.Error - expect(errors.As(err, &ferr), fmt.Sprintf("panicking listener yields *Error (got %T %v)", err, err)) - expect(ferr.Code == -4, fmt.Sprintf("foreign error code -4 (got %d)", ferr.Code)) - expect(strings.Contains(ferr.Message, "listener refused boom"), - fmt.Sprintf("foreign error carries the panic text (got %q)", ferr.Message)) - expect(store.Count() == 1, "entry was removed before the listener ran") - ok, err = store.Delete("boom") - expect(err == nil && !ok, "store usable after the foreign error") - put(store, "zeta", payload, nil) - ok, err = store.Delete("zeta") - expect(err == nil && ok && len(l4.seen) == 1 && l4.seen[0].key == "zeta", - "listener still attached after its panic") - store.ClearEvictionListener() - expect(waitFreed(&freed, 4), "all four listeners freed") - - // ── Objects: share, fork, optional, records, lists ── - // Share returns a second wrapper for the SAME object: a write through one - // is visible through the other, and releasing one leaves it alive. - shared := store.Share() - expect(shared != store, "share yields a distinct wrapper") - expect(shared.Count() == 1, "shared sees alpha") - put(shared, "via-shared", []byte{7, 7}, nil) - expect(store.Count() == 2, "put through share visible through the original") - got, err := store.Get("via-shared") - expect(err == nil && got != nil && len(got.Value) == 2, "value written through share readable") - shared.Close() - shared.Close() - expect(store.Count() == 2, "original alive after the shared wrapper closed") - - // Fork copies the live entries into an independent object. - forked := store.Fork() - expect(forked.Count() == 2, "fork copies entries") - put(forked, "fork-only", payload, nil) - put(forked, "fork-only-2", payload, nil) - expect(forked.Count() == 4 && store.Count() == 2, "fork is independent") - - // Store? as parameter and return. - empty, err := wv.OpenStore("/tmp/empty") - expect(err == nil, "open empty") - expect(empty.Larger(nil) == nil, "larger(nil) on an empty store is nil") - self := store.Larger(nil) - expect(self != nil, "larger(nil) on a non-empty store returns itself") - put(self, "via-larger", payload, nil) - expect(store.Count() == 3, "larger(nil) aliases the receiver") - self.Close() - bigger := store.Larger(forked) - expect(bigger != nil && bigger.Count() == 4, "larger(other) picks the bigger store") - put(bigger, "via-bigger", payload, nil) - expect(forked.Count() == 5 && store.Count() == 3, "larger(other) aliases the other store") - bigger.Close() - own := store.Larger(empty) - expect(own != nil && own.Count() == 3, "larger(smaller) returns the receiver") - own.Close() - - // A record carrying an object (and an absent, then present, optional). - info := store.Describe("primary", nil) - expect(info.Label == "primary", "describe label") - expect(info.Count == 3, fmt.Sprintf("describe count == 3 (got %d)", info.Count)) - expect(info.Mirror == nil, "describe mirror absent") - expect(info.Store != nil && info.Store.Count() == 3, "describe.store is usable") - put(info.Store, "via-info", payload, nil) - expect(store.Count() == 4, "describe.store aliases the receiver") - withMirror := forked.Describe("mirrored", store) - expect(withMirror.Count == 5, "mirrored describe count") - expect(withMirror.Mirror != nil && withMirror.Mirror.Count() == 4, "describe.mirror adopted") - expect(withMirror.Store.Count() == 5, "describe.store on the fork") - - // A list of objects as a throwing static return. - many, err := wv.StoreOpenMany([]string{"/tmp/a", "/tmp/b", "/tmp/c"}) - expect(err == nil && len(many) == 3, fmt.Sprintf("open_many opens 3 (got %d, %v)", len(many), err)) - put(many[0], "m0", payload, nil) - put(many[2], "m2a", payload, nil) - put(many[2], "m2b", payload, nil) - expect(many[0].Count() == 1 && many[1].Count() == 0 && many[2].Count() == 2, "open_many stores are independent") - _, err = wv.StoreOpenMany([]string{"/tmp/ok", ""}) - kerr = nil - expect(errors.As(err, &kerr) && kerr.Code == wv.KvErrorIoError, "open_many propagates IoError") - none, err := wv.StoreOpenMany(nil) - expect(err == nil && len(none) == 0, "open_many of nothing is empty") - - // Objects encoded into a parameter buffer (list and record fields). - expect(wv.StoreTotalCount(many, nil) == 3, "total_count sums the list") - expect(wv.StoreTotalCount(nil, nil) == 0, "total_count of nothing is 0") - expect(wv.StoreTotalCount(many, &info) == 3+4, "total_count adds extra.store") - expect(wv.StoreTotalCount([]*wv.Store{store, forked}, &withMirror) == 4+5+5, - "total_count with a mirrored extra") - // The wrappers keep their own references after being encoded. - expect(store.Count() == 4 && forked.Count() == 5 && info.Store.Count() == 4, "wrappers alive after encoding") - - // Async: an immediately-expired entry gives compact 3 bytes to reclaim; - // the cgo trampoline bridges the producer's worker thread to a channel. - put(store, "doomed", payload, ptrInt64(0)) - reclaimed, err := store.Compact(context.Background()) - expect(err == nil, "compact async") - expect(reclaimed == 3, fmt.Sprintf("compact reclaimed 3 bytes (got %d)", reclaimed)) - expect(store.Count() == 4, "store count after compact") - cancelled, cancel := context.WithCancel(context.Background()) + expect(kv.DebugLive(2) == 0, "an abandoned iterator is released") + + var sys []kv.Entry + for e := range s.Entries(ptr("sys.")) { + sys = append(sys, e) + } + expect(len(sys) == 1 && sys[0].Key == "sys.x" && string(sys[0].Value) == "xx", "entries(sys.)") + + // Partition: objects, created as they're pulled. + prefixes := []string{"user.", "sys.", "none."} + counts := []uint32{2, 1, 0} + i := 0 + for part := range s.Partition(prefixes) { + expect(part.Count() == counts[i] && part.Path() == prefixes[i], fmt.Sprintf("partition %d", i)) + expect(!sameStore(part, s), "partition yields new stores") + part.Close() + i++ + } + expect(i == 3, "three partitions") +} + +func listeners() { + base := liveCallbacks() + s, _ := kv.OpenStore("/listen") + l := &recordingListener{skip: "quiet"} + id := s.Subscribe(l) + expect(id > 0 && s.ListenerCount() == 1 && liveCallbacks() == base+1, "subscribe") + + put(s, "a", "1") + p := l.puts() + expect(len(p) == 1 && p[0].Entry.Version == 1 && !p[0].Replaced, "Put v1") + put(s, "a", "2") + p = l.puts() + expect(len(p) == 2 && p[1].Entry.Version == 2 && p[1].Replaced && p[1].Entry.Key == "a", "Put v2") + put(s, "quiet", "x") // Accepts said no + expect(len(l.puts()) == 2, "the filter skipped quiet") + expect(s.Delete("a"), "delete a") + expect(l.last() == kv.ChangeRemoved{Key: "a", Expired: false}, "Removed(a)") + + // An expired read removes the entry and says so. + _, err := putKind(s, "short", "x", kv.EntryKindVolatile, ptr(int64(1))) + expect(err == nil, "put short") + s.Tick(1) + _, err = s.Get("short") + expectAs[*kv.ExpiredError](err, "get(short)") + expect(l.last() == kv.ChangeRemoved{Key: "short", Expired: true}, "Removed(short, expired)") + + expect(s.Clear() == 1, "clear") // "quiet" was left + expect(l.last() == kv.ChangeCleared{Count: 1}, "Cleared(1)") + expect(l.offMain.Load() == 0, "synchronous calls notify on the calling thread") + + // Unsubscribing releases the listener once. + expect(s.Unsubscribe(id) && liveCallbacks() == base, "unsubscribe releases the listener") + expect(!s.Unsubscribe(id) && s.ListenerCount() == 0, "unsubscribe twice") + + // A listener that panics is detached (and released); the put succeeds. + failing := &recordingListener{failOn: "boom"} + s.Subscribe(failing) + put(s, "fine", "1") + expect(len(failing.puts()) == 1, "the failing listener saw fine") + put(s, "boom", "1") + expect(s.Count() == 2 && s.ListenerCount() == 0, "a failing listener is detached") + expect(liveCallbacks() == base, "the failing listener was released") + + // A required callback can't be nil. + expect(catchPanic(func() { s.Subscribe(nil) }) != nil, "Subscribe(nil) panics") + + // Closing the store releases the listeners it still holds. + s.Subscribe(&recordingListener{}) + s.Subscribe(&recordingListener{}) + expect(s.ListenerCount() == 2 && liveCallbacks() == base+2, "two listeners") + s.Close() + expect(liveCallbacks() == base, "closing the store released its listeners") +} + +func policies() { + base := liveCallbacks() + s, _ := kv.OpenStore("/policy") + other, _ := kv.OpenStore("/other") + p := &testPolicy{other: other.Share()} + s.SetPolicy(p) + expect(s.HasPolicy() && liveCallbacks() == base+1, "SetPolicy") + + // Admit's record return is what's stored (its key and version aside). + a, err := putKind(s, "a", "1", kv.EntryKindVolatile, nil) + expect(err == nil && a.Key == "a" && a.Version == 1 && a.Kind == kv.EntryKindEncrypted, fmt.Sprintf("admitted entry (%v)", err)) + expect(slices.Equal(a.Tags, []string{"admitted"}), "admit's rewrite") + + // Route: the object parameter and object return redirect a write. + _, err = putKind(s, "b/x", "2", kv.EntryKindVolatile, nil) + expect(err == nil && s.Count() == 1 && other.Count() == 1, "route redirected b/x") + + // A typed error from the throwing callback reaches the caller with its + // code, message, and payload. + _, err = putKind(s, "secret", "3", kv.EntryKindVolatile, nil) + r := expectAs[*kv.RejectedError](err, "put(secret)") + expect(r.Code() == 1005 && r.Error() == "secrets are not stored", "Rejected code and message") + expect(r.Key == "secret" && r.Reason == "no secrets", "Rejected payload") + + // Any other error arrives as -4 with the consumer's message. + _, err = putKind(s, "boom", "4", kv.EntryKindVolatile, nil) + expectRuntime(err, -4, "policy exploded") + // A return the producer can't accept is -3: a malformed record or a nil + // required object. + _, err = putKind(s, "garbage", "5", kv.EntryKindVolatile, nil) + expect(expectAs[*kv.Error](err, "put(garbage)").Code == -3, "a malformed admit return is -3") + _, err = putKind(s, "null/x", "6", kv.EntryKindVolatile, nil) + expect(expectAs[*kv.Error](err, "put(null/x)").Code == -3, "a nil route return is -3") + expect(s.Count() == 1 && other.Count() == 1 && p.admitted.Load() == 6, "failed puts changed nothing") + + // Replacing the policy releases the old one; nil removes it. + s.SetPolicy(&testPolicy{other: other.Share()}) + expect(liveCallbacks() == base+1, "replacing the policy released the old one") + s.SetPolicy(nil) + expect(!s.HasPolicy() && liveCallbacks() == base, "SetPolicy(nil) released it") + put(s, "secret", "now allowed") + expect(s.Count() == 2, "no policy, no veto") + + other.Close() + s.Close() +} + +func loaders() { + s, _ := kv.OpenStore("/load") + defer s.Close() + base := liveCallbacks() + + // No loader (a nil optional callback): a miss is none. + e, err := s.GetOrLoad("k", nil) + expect(err == nil && e == nil, "no loader") + + // Load's bytes are stored, tagged with the loader's name. + e, err = s.GetOrLoad("k", testLoader{}) + expect(err == nil && e != nil && string(e.Value) == "loaded:k", fmt.Sprintf("loaded value (%v)", err)) + expect(e.Kind == kv.EntryKindVolatile && len(e.Metadata) == 1 && e.Metadata["source"] == "go-loader", "loaded entry") + expect(liveCallbacks() == base, "a loader is released after the call") + expect(s.Count() == 1, "the loaded entry is stored") + // A hit doesn't consult the loader. + e, err = s.GetOrLoad("k", testLoader{}) + expect(err == nil && e != nil && e.Version == 1, "a hit") + + // The fallback store (an optional object return) is consulted first. + backup, _ := kv.OpenStore("/backup") + put(backup, "fb", "from backup") + e, err = s.GetOrLoad("fb", testLoader{backup: backup}) + expect(err == nil && e != nil && string(e.Value) == "from backup" && e.Kind == kv.EntryKindPersistent, "fallback entry") + backup.Close() + + // KeyNotFound for this key: the producer decoded the payload and + // answers none. + e, err = s.GetOrLoad("missing", testLoader{}) + expect(err == nil && e == nil, "KeyNotFound for the same key is none") + // KeyNotFound for another key: passed through, payload intact. + _, err = s.GetOrLoad("elsewhere", testLoader{}) + expectKeyNotFound(err, "other") + expect(err.Error() == "not in the loader", "the loader's message passed through") + // Any other error is -4. + _, err = s.GetOrLoad("broken", testLoader{}) + expectRuntime(err, -4, "loader is broken") + expect(liveCallbacks() == base, "every loader was released") +} + +func asyncCalls() { + ctx := context.Background() + s, _ := kv.OpenStore("/async-calls") + l := &recordingListener{} + s.Subscribe(l) + putKind(s, "old1", "x", kv.EntryKindVolatile, ptr(int64(1))) + putKind(s, "old2", "x", kv.EntryKindVolatile, ptr(int64(1))) + put(s, "keep", "x") + s.Tick(5) + + // Compact runs on a producer thread and notifies listeners there. + n, err := s.Compact(ctx, 0) + expect(err == nil && n == 2, fmt.Sprintf("compact(0) = %d, %v", n, err)) + expect(l.removed(true) == 2, "the listener saw both expirations") + expect(l.offMain.Load() == 2, "notified from a producer thread") + expect(s.Count() == 1, "one entry left") + n, err = s.Compact(ctx, 5) + expect(err == nil && n == 0, "compact(5) removed nothing") + + // Cancel mid-pause: the call returns context.Canceled at once, and the + // background pause notices the token and stops. + cctx, cancel := context.WithCancel(ctx) + done := make(chan error, 1) + go func() { + _, err := s.Compact(cctx, 60000) + done <- err + }() + time.Sleep(20 * time.Millisecond) + select { + case <-done: + expect(false, "compact(60000) finished early") + default: + } + expect(kv.StoreActiveJobs() >= 1, "the pause is running") + started := time.Now() cancel() - _, err = store.Compact(cancelled) - expect(errors.Is(err, context.Canceled), fmt.Sprintf("compact with a cancelled context (got %v)", err)) + err = <-done + expect(errors.Is(err, context.Canceled), fmt.Sprintf("a cancelled compact returns context.Canceled (got %v)", err)) + expect(time.Since(started) < 2*time.Second, "cancellation is prompt") + stopped := false + for range 2000 { + if kv.StoreActiveJobs() == 0 { + stopped = true + break + } + time.Sleep(time.Millisecond) + } + expect(stopped, "the cancelled pause stopped cooperatively") - // Close racing calls on other goroutines: each call either completes or - // panics with "used after Close"; none sees a freed store. - racer, err := wv.OpenStore("/tmp/race") - expect(err == nil, "open racer") + // GetMany: an async list of optional records, from 32 goroutines. var wg sync.WaitGroup - for range 8 { + results := make([][]*kv.Entry, 32) + errs := make([]error, 32) + for i := range 32 { wg.Add(1) go func() { defer wg.Done() - for range 500 { - if catchPanic(func() { racer.Count() }) != nil { - return - } - } + results[i], errs[i] = s.GetMany(ctx, []string{"keep", "gone", "keep"}) }() } - racer.Close() wg.Wait() + for i, got := range results { + expect(errs[i] == nil && len(got) == 3, "GetMany") + expect(got[0] != nil && got[0].Key == "keep" && got[1] == nil && got[2] != nil && got[2].Key == "keep", "GetMany entries") + } + + // The nested module's async function: objects in a list in, a record out. + other, _ := kv.OpenStore("/other") + put(other, "a", "123") + st, err := kv.SummarizeAll(ctx, []*kv.Store{s, other}) + expect(err == nil && st.Entries == 2 && st.Bytes == 4, fmt.Sprintf("SummarizeAll (got %+v, %v)", st, err)) + expect(len(st.ByKind) == 1 && st.ByKind[kv.EntryKindPersistent] == 2, "SummarizeAll by kind") + + // A context that's already done never launches. + _, err = s.Compact(cctx, 0) + expect(errors.Is(err, context.Canceled), "a done context returns at once") + + other.Close() + s.Close() +} + +func objectGraph() { + s0, _ := kv.OpenStore("/graph") + put(s0, "k", "v") + + // Share: the same object; the original wrapper can go. + s := s0.Share() + expect(sameStore(s, s0), "Share is the same object") + s0.Close() + expect(s.Count() == 1, "alive through the shared reference") + + // Fork: a distinct object with a copy of the entries. + fork := s.Fork() + expect(fork.Count() == 1 && fork.Path() == "/graph" && !sameStore(fork, s), "fork") + put(fork, "k2", "v") + expect(fork.Count() == 2 && s.Count() == 1, "fork is independent") + + // Larger: Store? in and out. + empty, _ := kv.OpenStore("/empty") + expect(empty.Larger(nil) == nil, "Larger(nil) on an empty store") + bigger := empty.Larger(fork) + expect(bigger != nil && sameStore(bigger, fork), "Larger(fork) is fork") + bigger.Close() + bigger = s.Larger(nil) + expect(bigger != nil && sameStore(bigger, s), "Larger(nil) is self") + bigger.Close() - // Plain void method, then release every object; Close is idempotent. - store.Clear() - expect(store.Count() == 0 && info.Store.Count() == 0, "clear visible through every alias") - for _, s := range many { - s.Close() - s.Close() + // Describe: a record whose fields carry objects. + info := s.Describe("main", fork) + expect(info.Label == "main" && info.Count == 1, "describe label and count") + expect(sameStore(info.Store, s) && info.Mirror != nil && sameStore(info.Mirror, fork), "describe objects") + expect(info.Mirror.Count() == 2, "the mirror is live") + + // StoreOpenMany: a list of objects; one bad path fails the whole call. + many, err := kv.StoreOpenMany([]string{"/a", "/b"}) + expect(err == nil && len(many) == 2 && many[0].Path() == "/a" && many[1].Path() == "/b", "StoreOpenMany") + _, err = kv.StoreOpenMany([]string{"/a", ""}) + expectAs[*kv.InvalidPathError](err, "StoreOpenMany with an empty path") + + // StoreByLabel: records with objects in, a map with object values out. + named := kv.StoreByLabel([]kv.StoreInfo{info, {Label: "first", Store: many[0]}}) + expect(len(named) == 2 && sameStore(named["main"], s) && sameStore(named["first"], many[0]), "StoreByLabel") + + // StoreTotalCount: a list, a map, and an optional record, all carrying + // objects (each written as a fresh reference the producer adopts). + put(many[0], "m", "1") + stores := []*kv.Store{many[0], many[1], fork} + expect(kv.StoreTotalCount(stores, named, &info) == 6, "StoreTotalCount with extra") + expect(kv.StoreTotalCount(stores, named, nil) == 5, "StoreTotalCount without extra") + + // Everything is still intact; release each wrapper once. + expect(s.Count() == 1 && fork.Count() == 2 && many[0].Count() == 1, "still usable") + for _, n := range named { + n.Close() } - withMirror.Store.Close() - withMirror.Mirror.Close() - info.Store.Close() info.Store.Close() + info.Mirror.Close() + for _, m := range many { + m.Close() + } empty.Close() - forked.Close() - store.Close() - store.Close() - expect(catchPanic(func() { store.Count() }) != nil, "use after Close panics") + fork.Close() + s.Close() + expect(catchPanic(func() { s.Count() }) != nil, "a closed wrapper panics") + s.Close() // idempotent +} - expectNoLeaks(wv.DebugLive) - fmt.Println("go/kvstore: OK") +func statsAndReport() { + s, _ := kv.OpenStore("/stats") + defer s.Close() + put(s, "b", "12") + put(s, "a", "1") + put(s, "a", "123") + putKind(s, "c", "x", kv.EntryKindEncrypted, nil) + + // kv.stats: the parent's Store as a parameter, the parent's error domain. + st, err := kv.Summarize(s, nil) + expect(err == nil && st.Entries == 3 && st.Bytes == 6, fmt.Sprintf("Summarize (got %+v)", st)) + expect(len(st.ByKind) == 2 && st.ByKind[kv.EntryKindPersistent] == 2 && st.ByKind[kv.EntryKindEncrypted] == 1, "Summarize by kind") + _, err = kv.Summarize(s, ptr("q")) + expectKeyNotFound(err, "q") + + // report: the sibling root shares the Entry record. + var entries []kv.Entry + for e := range s.Entries(nil) { + entries = append(entries, e) + } + lines, err := kv.RenderReport(entries) + want := []string{"a: 3 bytes, Persistent, v2", "b: 2 bytes, Persistent", "c: 1 bytes, Encrypted"} + expect(err == nil && slices.Equal(lines, want), fmt.Sprintf("RenderReport (got %q)", lines)) + _, err = kv.RenderReport(nil) + nothing := expectAs[*kv.NothingToReportError](err, "RenderReport(nil)") + expect(nothing.Code() == 2001 && nothing.Error() == "nothing to report", "NothingToReport") + expectAs[kv.ReportError](err, "a NothingToReportError is a ReportError") + var domain kv.KvError + expect(!errors.As(err, &domain), "a ReportError isn't a KvError") } -func ptrInt64(v int64) *int64 { return &v } +func main() { + constructors() + basics() + iterators() + listeners() + policies() + loaders() + asyncCalls() + objectGraph() + statsAndReport() + + expectNoLeaks(kv.DebugLive) + fmt.Println("go/kvstore: OK") +} diff --git a/conformance/go/run.sh b/conformance/go/run.sh index 8ff03114..63048e77 100755 --- a/conformance/go/run.sh +++ b/conformance/go/run.sh @@ -5,6 +5,7 @@ # consumer against one sample and must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools go go # Run a Go consumer in a throwaway module that requires the generated module # as-is (a `replace` points at its directory). The module path follows the @@ -38,22 +39,11 @@ EOF } go_calculator() { go_consumer calculator calculator.go; } - -go_contacts() { go_consumer contacts contacts.go; } - -go_events() { go_consumer events events.go; } - -go_kvstore() { go_consumer kvstore kvstore.go; } - -go_async_demo() { go_consumer async-demo async_demo.go; } - -go_codec() { go_consumer codec codec.go; } +go_codec() { go_consumer codec codec.go; } +go_kvstore() { go_consumer kvstore kvstore.go; } lane go-calculator go_calculator -lane go-contacts go_contacts -lane go-events go_events -lane go-kvstore go_kvstore -lane go-async-demo go_async_demo lane go-codec go_codec +lane go-kvstore go_kvstore finish_lanes diff --git a/conformance/kotlin/async_demo.kt b/conformance/kotlin/async_demo.kt deleted file mode 100644 index 3c14c8b1..00000000 --- a/conformance/kotlin/async_demo.kt +++ /dev/null @@ -1,108 +0,0 @@ -// Conformance consumer: async-demo sample, Kotlin (JVM via JNI) target. -// -// Drives the suspend-function async surface end to end: `runTask` resumed -// through a `NativeCompletion` from the producer's worker thread and decoded -// from a value buffer into the `TaskResult` data class, the typed -// `TaskException.InvalidName` for an empty name, the buffered list-of-records -// round trip through `runBatch`, the direct-scalar `runNTasks`, and the -// cancellable `wait`: it completes normally when left alone, and cancelling -// the awaiting coroutine (directly or through `withTimeout`) cancels the -// native token, so the producer drops the pending work long before its -// timeout. At exit it checks that every native resource was released. -@file:JvmName("Main") - -import async_demo.FfiException -import async_demo.JniBridge -import async_demo.TaskException -import async_demo.Tasks -import kotlinx.coroutines.CancellationException -import kotlinx.coroutines.TimeoutCancellationException -import kotlinx.coroutines.async -import kotlinx.coroutines.delay -import kotlinx.coroutines.runBlocking -import kotlinx.coroutines.withTimeout - -/** Poll `activeCallbacks` until it reads zero or `ms` milliseconds pass. */ -fun settled(ms: Long): Boolean { - val deadline = System.currentTimeMillis() + ms - while (System.currentTimeMillis() < deadline) { - if (Tasks.activeCallbacks() == 0L) return true - Thread.sleep(5) - } - return Tasks.activeCallbacks() == 0L -} - -fun run() = runBlocking { - // Async record return: the suspend fun resumes with a TaskResult decoded - // from the completion's value buffer. - val result = Tasks.runTask("alpha") - expect(result.id > 0, "runTask assigns an id") - expect(result.value == "completed: alpha", "runTask value (got ${result.value})") - expect(result.success, "runTask success flag") - - // Typed async error: the empty name throws the typed subclass. - val invalid = try { - Tasks.runTask("") - null - } catch (e: TaskException.InvalidName) { - e - } - expect(invalid != null, "expected TaskException.InvalidName for an empty name") - expect(invalid!!.code == 1, "InvalidName carries code 1 (got ${invalid.code})") - expect(invalid is FfiException, "InvalidName is an FfiException") - - // Buffered list-of-records both ways. - val batch = Tasks.runBatch(listOf("a", "b", "c")) - expect( - batch.map { it.value } == listOf("completed: a", "completed: b", "completed: c"), - "runBatch values (got ${batch.map { it.value }})", - ) - expect(batch.all { it.success }, "runBatch success flags") - - // Direct scalar through the async completion, many at once. - expect(Tasks.runNTasks(7) == 7, "runNTasks echoes n") - val many = (1..50).map { n -> async { Tasks.runNTasks(n) } }.map { it.await() } - expect(many == (1..50).toList(), "concurrent runNTasks") - - // A cancellable call left alone completes normally. - expect(Tasks.wait(20L) == 20L, "wait(20) completes with 20") - - // Cancelling the awaiting coroutine cancels the native token: the call - // ends with CancellationException at once, and the producer drops the - // pending future (and its in-flight guard) instead of waiting 10 s. - val started = System.currentTimeMillis() - val job = async { Tasks.wait(10_000L) } - delay(50) - expect(Tasks.activeCallbacks() == 1L, "wait is in flight (got ${Tasks.activeCallbacks()})") - job.cancel() - val cancelled = try { - job.await() - null - } catch (e: CancellationException) { - e - } - expect(cancelled != null, "a cancelled wait throws CancellationException") - expect(settled(2_000), "the producer dropped the cancelled wait (active ${Tasks.activeCallbacks()})") - expect(System.currentTimeMillis() - started < 5_000, "cancellation didn't wait for the timeout") - - // withTimeout cancels the same way. - val timedOut = try { - withTimeout(50) { Tasks.wait(10_000L) } - null - } catch (e: TimeoutCancellationException) { - e - } - expect(timedOut != null, "withTimeout cancels wait") - expect(settled(2_000), "the producer dropped the timed-out wait") - expect(System.currentTimeMillis() - started < 5_000, "timeouts didn't wait for the producer's timer") - - // Every spawned task body has completed by the time its completion fires. - expect(Tasks.activeCallbacks() == 0L, "activeCallbacks settles to zero") -} - -fun main() { - run() - println("kotlin/async-demo: OK") - expectNoLeaks { JniBridge.debug_live(it) } - println("kotlin/async-demo: no leaks") -} diff --git a/conformance/kotlin/calculator.kt b/conformance/kotlin/calculator.kt new file mode 100644 index 00000000..6237f53a --- /dev/null +++ b/conformance/kotlin/calculator.kt @@ -0,0 +1,39 @@ +// Conformance consumer: calculator sample, Kotlin (JVM via JNI) target. +// +// The getting-started surface through the generated `Calculator` object: a +// direct-value call (wrapping at the `Int` edges), a throwing call that +// raises the typed `CalcException.DivisionByZero`, and a string in and out +// (non-ASCII and astral text and an interior NUL survive). Loading the +// bindings checks the ABI revision and the `calculator` contract. Ends by +// asserting the producer's leak counters are zero. +@file:JvmName("Main") + +import calculator.CalcException +import calculator.Calculator +import calculator.FfiException +import calculator.JniBridge + +fun main() { + expect(JniBridge.debug_live(-1) == 1L, "the sample counts live allocations") + + expect(Calculator.add(2, 3) == 5, "add(2, 3)") + expect(Calculator.add(-7, 7) == 0, "add(-7, 7)") + expect(Calculator.add(Int.MAX_VALUE, 1) == Int.MIN_VALUE, "add wraps") + + expect(Calculator.divide(10, 2) == 5, "divide(10, 2)") + expect(Calculator.divide(-7, 2) == -3, "divide rounds toward zero") + expect(Calculator.divide(Int.MIN_VALUE, -1) == Int.MIN_VALUE, "divide(MIN, -1)") + + val err = thrownBy { Calculator.divide(1, 0) } + expect(err is CalcException.DivisionByZero, "divide(1, 0) raises DivisionByZero (got $err)") + expect(err is FfiException && err.code == 1, "DivisionByZero has code 1") + expect(err?.message == "division by zero", "DivisionByZero message (got ${err?.message})") + + expect(Calculator.greet("World") == "Hello, World!", "greet(World)") + expect(Calculator.greet("") == "Hello, !", "greet(empty)") + expect(Calculator.greet("Wörld 🦀") == "Hello, Wörld 🦀!", "greet keeps non-ASCII and astral text") + expect(Calculator.greet("a\u0000b") == "Hello, a\u0000b!", "greet keeps an interior NUL") + + expectNoLeaks(JniBridge::debug_live) + println("kotlin/calculator: OK") +} diff --git a/conformance/kotlin/codec.kt b/conformance/kotlin/codec.kt index 970c2d78..4adcbb80 100644 --- a/conformance/kotlin/codec.kt +++ b/conformance/kotlin/codec.kt @@ -1,401 +1,304 @@ // Conformance consumer: codec sample, Kotlin (JVM via JNI) target. // -// Drives the value-buffer round-trip oracle through the generated Kotlin -// encoder and decoder (`BufferWriter`/`BufferReader` plus the per-type -// `pack*`/`unpack*` routines). For `Scalars` and `Composite` the consumer -// decodes the producer's canonical fixture and checks every field, hands the -// same value back through `verify*` (proving the Kotlin encoder produced -// exactly the bytes Rust encodes), and compares `roundtrip*` output field by -// field. It then round-trips its own edge cases (empty strings, lists, maps, -// and byte arrays; present-but-empty optionals; BMP and supplementary -// unicode; strings with interior NULs, both inside buffers and as direct -// arguments; the i8..u64 extremes; NaN, infinities, negative zero, and a -// subnormal), every `Shape` variant, the typed `CodecException.Mismatch`, -// and `Holder` (objects inside records, optionals, and lists, with clean -// release of every adopted reference). At exit it checks that every native -// resource was released. +// The shared-vector loop: for every vector the producer serves, decode it +// with the generated codec, re-encode it (which must reproduce the +// producer's bytes exactly, objects aside), pass it back to `checkVector`, +// and push each primitive vector's value through the matching direct-family +// `echo*` (unsigned values as `UByte`/`UShort`/`UInt`/`ULong`). Then vectors +// built from literals (so a symmetric encode/decode bug can't hide), spot +// checks of decoded fields, the typed `CodecException.OutOfRange` and its +// payload, malformed input rejected as marshalling failures (through the +// internal JNI bridge, which takes raw bytes), and object identity and +// reference counting through buffers. Ends by asserting the producer's leak +// counters are zero. Compiled with `-Xfriend-paths`, so the bindings' +// `internal` codec and bridge are reachable. @file:JvmName("Main") +import codec.BufferWriter import codec.CodecException +import codec.Codec import codec.Color import codec.Composite import codec.Holder +import codec.JniBridge +import codec.NativeBugException import codec.Scalars import codec.Shape import codec.Token -import codec.Codec -import codec.JniBridge -import codec.FfiException -import codec.packComposite -import codec.unpackComposite +import codec.Vector import codec.decodeBuffer import codec.encodeBuffer +import codec.packVector +import codec.unpackVector + +/** Release the tokens a decoded vector carries (only `Objects` has any). */ +fun release(v: Vector) { + if (v is Vector.Objects) { + val h = v.value + h.primary.close() + h.spare?.close() + h.many.forEach { it.close() } + h.by_name.values.forEach { it.close() } + } +} + +fun encode(v: Vector): ByteArray = encodeBuffer { packVector(it, v) } + +/** The index of the vector named [name]. */ +fun find(n: UInt, name: String): UInt { + for (i in 0u until n) { + if (Codec.vectorName(i) == name) return i + } + expect(false, "no vector named $name") + return 0u +} -/** The canonical `Scalars` fixture, spelled with Kotlin's signed carriers for the unsigned fields. */ -fun canonicalScalars(): Scalars = Scalars( - i8_value = (-8).toByte(), - u8_value = 200.toByte(), - i16_value = (-16_000).toShort(), - u16_value = 60_000.toShort(), - i32_value = -2_000_000_000, - u32_value = 4_000_000_000L, - i64_value = -9_007_199_254_740_993L, - u64_value = ULong.MAX_VALUE.toLong(), +/** Push a primitive vector's value through its direct-family echo. */ +fun echo(v: Vector) { + when (v) { + is Vector.I8 -> expect(Codec.echoI8(v.value) == v.value, "echoI8 ${v.value}") + is Vector.U8 -> expect(Codec.echoU8(v.value) == v.value, "echoU8 ${v.value}") + is Vector.I16 -> expect(Codec.echoI16(v.value) == v.value, "echoI16 ${v.value}") + is Vector.U16 -> expect(Codec.echoU16(v.value) == v.value, "echoU16 ${v.value}") + is Vector.I32 -> expect(Codec.echoI32(v.value) == v.value, "echoI32 ${v.value}") + is Vector.U32 -> expect(Codec.echoU32(v.value) == v.value, "echoU32 ${v.value}") + is Vector.I64 -> expect(Codec.echoI64(v.value) == v.value, "echoI64 ${v.value}") + is Vector.U64 -> expect(Codec.echoU64(v.value) == v.value, "echoU64 ${v.value}") + is Vector.F32 -> expect( + Codec.echoF32(v.value).toRawBits() == v.value.toRawBits(), + "echoF32 ${v.value}", + ) + is Vector.F64 -> expect( + Codec.echoF64(v.value).toRawBits() == v.value.toRawBits(), + "echoF64 ${v.value}", + ) + is Vector.Flag -> expect(Codec.echoBool(v.value) == v.value, "echoBool ${v.value}") + is Vector.Hue -> expect(Codec.echoColor(v.value) == v.value, "echoColor ${v.value}") + is Vector.Text -> expect(Codec.echoText(v.value) == v.value, "echoText") + is Vector.Blob -> expect(Codec.echoBlob(v.value).contentEquals(v.value), "echoBlob") + else -> {} + } +} + +fun everyVector(n: UInt) { + for (i in 0u until n) { + val raw = JniBridge.codec_vector(i.toInt()) + val v = decodeBuffer(raw) { unpackVector(it) } + if (!Codec.checkVector(i, v)) { + expect( + false, + "vector $i (${Codec.vectorName(i)}) did not round-trip; producer saw ${Codec.describeVector(v)}", + ) + } + expect(!Codec.checkVector((i + 1u) % n, v), "vector $i never matches its neighbor") + // The generated encoder is deterministic, so re-encoding reproduces + // the producer's bytes, except for object tokens (fresh references). + if (v !is Vector.Objects) { + expect(encode(v).contentEquals(raw), "vector $i re-encodes to the producer's bytes") + } + echo(v) + release(v) + } +} + +fun canonicalScalars() = Scalars( + i8_value = -8, + u8_value = 200u, + i16_value = -16000, + u16_value = 60000u, + i32_value = -2000000000, + u32_value = 4000000000u, + i64_value = -9007199254740993L, + u64_value = ULong.MAX_VALUE, f32_value = 1.5f, f64_value = -2.25e100, flag = true, color = Color.Blue, ) -/** - * Field-by-field equality for `Composite`: the data class holds `ByteArray` - * fields, whose generated `equals` compares identity, so the comparison has - * to spell them out with `contentEquals`. - */ -fun compositeEquals(a: Composite, b: Composite): Boolean = - a.name == b.name && - a.blob.contentEquals(b.blob) && - a.some_i64 == b.some_i64 && - a.none_i64 == b.none_i64 && - a.some_text == b.some_text && - a.names == b.names && - a.matrix == b.matrix && - a.empty == b.empty && - a.by_name == b.by_name && - a.by_id == b.by_id && - a.scalars == b.scalars && - a.shape == b.shape && - a.shapes == b.shapes && - a.maybe_shape == b.maybe_shape && - ((a.maybe_list == null && b.maybe_list == null) || - (a.maybe_list != null && b.maybe_list != null && a.maybe_list.contentEquals(b.maybe_list))) && - a.sparse == b.sparse && - a.colors == b.colors - -fun checkScalars() { +fun literalVectors(n: UInt) { val canonical = canonicalScalars() - val sample = Codec.sampleScalars() - // Producer encodes, consumer decodes: every field, with the unsigned ones - // read back through their signed carriers. - expect(sample.i8_value.toInt() == -8, "i8 (got ${sample.i8_value})") - expect(sample.u8_value.toUByte().toInt() == 200, "u8 (got ${sample.u8_value.toUByte()})") - expect(sample.i16_value.toInt() == -16_000, "i16 (got ${sample.i16_value})") - expect(sample.u16_value.toUShort().toInt() == 60_000, "u16 (got ${sample.u16_value.toUShort()})") - expect(sample.i32_value == -2_000_000_000, "i32 (got ${sample.i32_value})") - expect(sample.u32_value == 4_000_000_000L, "u32 (got ${sample.u32_value})") - expect(sample.i64_value == -9_007_199_254_740_993L, "i64 (got ${sample.i64_value})") - expect(sample.u64_value.toULong() == ULong.MAX_VALUE, "u64 max (got ${sample.u64_value.toULong()})") - expect(sample.f32_value == 1.5f, "f32 (got ${sample.f32_value})") - expect(sample.f64_value == -2.25e100, "f64 (got ${sample.f64_value})") - expect(sample.flag, "flag") - expect(sample.color == Color.Blue, "color Blue (got ${sample.color})") - expect(sample == canonical, "sample equals the locally built canonical Scalars") - - // Consumer encodes, producer decodes and compares with its canonical. - expect(Codec.verifyScalars(sample), "verifyScalars(sample)") - expect(Codec.verifyScalars(canonical), "verifyScalars(locally built canonical)") - expect(Codec.roundtripScalars(sample) == sample, "roundtripScalars(sample) equals sample") - - // A rejected fixture is the typed domain error. - val mismatch = thrownBy { Codec.verifyScalars(canonical.copy(flag = false)) } - expect(mismatch is CodecException.Mismatch, "verifyScalars(modified) throws Mismatch (got $mismatch)") - expect((mismatch as FfiException).code == 1, "Mismatch code 1 (got ${mismatch.code})") - expect(mismatch.message == "value does not match the canonical fixture", "Mismatch message (got ${mismatch.message})") - expect(thrownBy { Codec.verifyScalars(canonical.copy(u64_value = 0L)) } is CodecException.Mismatch, "u64 difference detected") - expect(thrownBy { Codec.verifyScalars(canonical.copy(color = Color.Red)) } is CodecException.Mismatch, "enum difference detected") - - // From scratch, at the extremes of every width, plus the special floats. - val edge = Scalars( - i8_value = Byte.MIN_VALUE, - u8_value = (-1).toByte(), - i16_value = Short.MIN_VALUE, - u16_value = (-1).toShort(), - i32_value = Int.MIN_VALUE, - u32_value = 0xFFFF_FFFFL, - i64_value = Long.MIN_VALUE, - u64_value = Long.MIN_VALUE, - f32_value = Float.NaN, - f64_value = -0.0, - flag = false, - color = Color.Red, + expect(Codec.checkVector(find(n, "scalars canonical"), Vector.AllScalars(canonical)), "scalars canonical") + expect( + !Codec.checkVector(find(n, "scalars canonical"), Vector.AllScalars(canonical.copy(u16_value = 60001u))), + "a changed field no longer matches", + ) + expect(Codec.checkVector(find(n, "shape labeled"), Vector.Figure(Shape.Labeled("tag", 3))), "shape labeled") + expect( + Codec.checkVector(find(n, "string interior nul"), Vector.Text("nul\u0000inside\u0000")), + "string interior nul", ) - val edgeBack = Codec.roundtripScalars(edge) - expect(edgeBack.i8_value == Byte.MIN_VALUE, "i8 min round-trips") - expect(edgeBack.u8_value.toUByte() == UByte.MAX_VALUE, "u8 255 round-trips") - expect(edgeBack.i16_value == Short.MIN_VALUE, "i16 min round-trips") - expect(edgeBack.u16_value.toUShort() == UShort.MAX_VALUE, "u16 65535 round-trips") - expect(edgeBack.i32_value == Int.MIN_VALUE, "i32 min round-trips") - expect(edgeBack.u32_value == 0xFFFF_FFFFL, "u32 max round-trips (got ${edgeBack.u32_value})") - expect(edgeBack.i64_value == Long.MIN_VALUE, "i64 min round-trips") - expect(edgeBack.u64_value.toULong() == 9_223_372_036_854_775_808UL, "u64 2^63 round-trips") - expect(edgeBack.f32_value.isNaN(), "f32 NaN round-trips") - expect(edgeBack.f32_value.toRawBits() == Float.NaN.toRawBits(), "f32 NaN bits preserved") - expect(edgeBack.f64_value == 0.0 && 1.0 / edgeBack.f64_value < 0.0, "f64 -0.0 round-trips as negative zero") - expect(edgeBack.f64_value.toRawBits() == (-0.0).toRawBits(), "f64 -0.0 bits preserved") - expect(edgeBack.color == Color.Red && !edgeBack.flag, "enum and flag round-trip") - expect(edgeBack == edge, "edge Scalars equals (data class compares floats by bits)") - val maxes = edge.copy( - i8_value = Byte.MAX_VALUE, - u8_value = 0, - i16_value = Short.MAX_VALUE, - u16_value = 0, - i32_value = Int.MAX_VALUE, - u32_value = 0L, - i64_value = Long.MAX_VALUE, - u64_value = 0L, - f32_value = Float.NEGATIVE_INFINITY, - f64_value = Double.POSITIVE_INFINITY, - color = Color.Green, + // Any NaN matches the NaN vector; zero keeps its sign. + expect( + Codec.checkVector(find(n, "f64 nan"), Vector.F64(Double.fromBits(0x7ff8000000000001L))), + "a NaN payload matches f64 nan", ) - expect(Codec.roundtripScalars(maxes) == maxes, "max Scalars round-trip") - val subnormal = edge.copy(f32_value = Float.MIN_VALUE, f64_value = Double.MIN_VALUE) - expect(Codec.roundtripScalars(subnormal) == subnormal, "subnormal floats round-trip") + expect(Codec.checkVector(find(n, "f64 -0"), Vector.F64(-0.0)), "-0.0 matches f64 -0") + expect(!Codec.checkVector(find(n, "f64 -0"), Vector.F64(0.0)), "+0.0 doesn't match f64 -0") + expect(Codec.checkVector(find(n, "u64 max"), Vector.U64(ULong.MAX_VALUE)), "u64 max") + expect(Codec.checkVector(find(n, "enum infrared"), Vector.Hue(Color.Infrared)), "enum infrared") + expect(Codec.checkVector(find(n, "optional zero"), Vector.MaybeI64(0L)), "optional zero") + expect(Codec.checkVector(find(n, "optional absent"), Vector.MaybeI64(null)), "optional absent") + expect(!Codec.checkVector(find(n, "optional zero"), Vector.MaybeI64(null)), "absent isn't zero") + // A map's entry order doesn't matter on the wire. + val counts = linkedMapOf("x" to 0L, "héllo" to -1L, "" to Long.MAX_VALUE) + expect(Codec.checkVector(find(n, "map of strings"), Vector.Counts(counts)), "map of strings") + expect(Codec.checkVector(find(n, "blank"), Vector.Blank), "blank") + // A holder of a consumer-made token. + Token(-1).use { lone -> + val sparse = Vector.Objects(Holder(lone, null, emptyList(), emptyMap())) + expect(Codec.checkVector(find(n, "objects sparse"), sparse), "objects sparse") + } } -fun checkComposite() { - val canonical = canonicalScalars() - val sample = Codec.sampleComposite() - expect(sample.name == "héllo wörld ✓", "name (got ${sample.name})") - expect(sample.blob.contentEquals(byteArrayOf(0, 1, 2, 253.toByte(), 254.toByte(), 255.toByte())), "blob (got ${sample.blob.toList()})") - expect(sample.some_i64 == Long.MIN_VALUE, "some_i64 is i64::MIN (got ${sample.some_i64})") - expect(sample.none_i64 == null, "none_i64 absent") - expect(sample.some_text != null && sample.some_text.isEmpty(), "some_text present and empty (got ${sample.some_text})") - expect(sample.names == listOf("a", "", "ccc"), "names (got ${sample.names})") - expect(sample.matrix == listOf(listOf(1, 2, 3), listOf(), listOf(-4)), "matrix (got ${sample.matrix})") - expect(sample.empty.isEmpty(), "empty list") - expect(sample.by_name == mapOf("one" to 1L, "two" to 2L, "neg" to -3L), "by_name (got ${sample.by_name})") - expect(sample.by_id.keys == setOf(-1, 42), "by_id keys (got ${sample.by_id.keys})") - expect(sample.by_id[-1] == canonical, "by_id[-1] is the canonical Scalars") - expect(sample.by_id[42] == canonical.copy(flag = false), "by_id[42] is canonical with flag=false") - expect(sample.scalars == canonical, "scalars nested record") - expect(sample.shape == Shape.Labeled("tag", 3), "shape (got ${sample.shape})") - expect( - sample.shapes == listOf( - Shape.Empty, - Shape.Circle(2.5), - Shape.Rect(1.0f, 0.5f), - Shape.Labeled("", -1), - Shape.Nested(canonical, "n"), - ), - "shapes (got ${sample.shapes})" - ) - expect(sample.maybe_shape == Shape.Nested(canonical, null), "maybe_shape (got ${sample.maybe_shape})") - expect(sample.maybe_list != null && sample.maybe_list.contentEquals(byteArrayOf(9, 8)), "maybe_list (got ${sample.maybe_list?.toList()})") - expect(sample.sparse == listOf(true, null, false), "sparse (got ${sample.sparse})") - expect(sample.colors == listOf(Color.Red, Color.Green, Color.Blue), "colors (got ${sample.colors})") +fun fetch(i: UInt): Vector = Codec.vector(i) - // Consumer encodes, producer decodes and compares. - expect(Codec.verifyComposite(sample), "verifyComposite(sample)") - val back = Codec.roundtripComposite(sample) - expect(compositeEquals(back, sample), "roundtripComposite(sample) equals sample") - // The local encoder and decoder agree with each other too. - val local = decodeBuffer(encodeBuffer { w -> packComposite(w, sample) }) { r -> unpackComposite(r) } - expect(compositeEquals(local, sample), "local encode/decode round trip") +fun spotChecks(n: UInt) { + val past53 = fetch(find(n, "i64 past 2^53")) + expect(past53 == Vector.I64(-9007199254740993L), "i64 past 2^53 is exact (got $past53)") - // The producer's rendering is a debugging aid: check it saw our unicode. - val described = Codec.describeComposite(sample) - expect(described.contains("héllo wörld ✓"), "describeComposite carries the name (got $described)") - expect(described.contains("Labeled { label: \"tag\", count: 3 }"), "describeComposite carries the shape") + val subnormal = fetch(find(n, "f32 min subnormal")) + expect(subnormal is Vector.F32 && subnormal.value.toRawBits() == 1, "f32 min subnormal has bits 1") - // Any single change is detected by the producer. - val changed = sample.copy(sparse = listOf(true, true, false)) - val mismatch = thrownBy { Codec.verifyComposite(changed) } - expect(mismatch is CodecException.Mismatch, "verifyComposite(changed) throws Mismatch (got $mismatch)") - expect(thrownBy { Codec.verifyComposite(sample.copy(none_i64 = 0L)) } is CodecException.Mismatch, "present vs absent optional detected") - expect(thrownBy { Codec.verifyComposite(sample.copy(maybe_list = null)) } is CodecException.Mismatch, "absent list detected") - expect(thrownBy { Codec.verifyComposite(sample.copy(some_text = null)) } is CodecException.Mismatch, "absent empty string detected") + expect(fetch(find(n, "string astral")) == Vector.Text("🦀 crab 😀"), "string astral") - // From scratch: everything empty or absent, with supplementary unicode. - val bare = Composite( - name = "", - blob = byteArrayOf(), - some_i64 = null, - none_i64 = null, - some_text = null, - names = listOf(), - matrix = listOf(), - empty = listOf(), - by_name = mapOf(), - by_id = mapOf(), - scalars = canonical, - shape = Shape.Empty, - shapes = listOf(), - maybe_shape = null, - maybe_list = null, - sparse = listOf(), - colors = listOf(), - ) - val bareBack = Codec.roundtripComposite(bare) - expect(compositeEquals(bareBack, bare), "bare Composite round-trips (got ${Codec.describeComposite(bareBack)})") - expect(bareBack.maybe_list == null && bareBack.some_text == null, "absent optionals stay absent") + val minimum = fetch(find(n, "scalars minimum")) + expect(minimum is Vector.AllScalars, "scalars minimum is AllScalars") + val m = (minimum as Vector.AllScalars).value + expect(m.i8_value == Byte.MIN_VALUE && m.i16_value == Short.MIN_VALUE, "scalars minimum i8/i16") + expect(m.i32_value == Int.MIN_VALUE && m.i64_value == Long.MIN_VALUE, "scalars minimum i32/i64") + expect(m.u8_value == UByte.MIN_VALUE && m.u64_value == 0uL, "scalars minimum unsigned") + expect(m.f32_value == Float.NEGATIVE_INFINITY && m.f64_value.isNaN(), "scalars minimum floats") + expect(m.color == Color.Infrared && !m.flag, "scalars minimum color and flag") - val full = Composite( - name = "日本語 🚀 \u0000 emoji and NUL", - blob = ByteArray(256) { it.toByte() }, - some_i64 = Long.MAX_VALUE, - none_i64 = Long.MIN_VALUE, - some_text = "🚀🚀", - names = listOf("", "x".repeat(1000), "ünïcödé"), - matrix = listOf(listOf(), listOf(Int.MIN_VALUE, Int.MAX_VALUE), listOf(0)), - empty = listOf(Double.NaN, Double.NEGATIVE_INFINITY, -0.0, Double.MIN_VALUE), - by_name = mapOf("" to 0L, "k" to Long.MIN_VALUE, "🚀" to Long.MAX_VALUE), - by_id = mapOf(Int.MIN_VALUE to canonical, Int.MAX_VALUE to canonical.copy(color = Color.Red), 0 to canonical), - scalars = canonical.copy(f32_value = Float.POSITIVE_INFINITY), - shape = Shape.Nested(canonical, ""), - shapes = listOf(Shape.Circle(Double.NaN), Shape.Rect(-0.0f, Float.MIN_VALUE), Shape.Labeled("🚀", Int.MIN_VALUE), Shape.Empty, Shape.Empty), - maybe_shape = Shape.Circle(-0.0), - maybe_list = byteArrayOf(), - sparse = listOf(null, null, true), - colors = listOf(Color.Blue, Color.Blue, Color.Red), + val deep = fetch(find(n, "composite canonical")) + expect(deep is Vector.Deep, "composite canonical is Deep") + val c: Composite = (deep as Vector.Deep).value + expect(c.name == "héllo wörld ✓", "composite name (got ${c.name})") + expect(c.blob.size == 6 && c.blob[5] == 255.toByte(), "composite blob") + expect(c.some_i64 == Long.MIN_VALUE && c.none_i64 == null, "composite optionals") + expect(c.some_text == "", "composite some_text is present and empty") + expect(c.names.size == 3 && c.names[1] == "", "composite names") + expect(c.matrix.size == 3 && c.matrix[1].isEmpty() && c.matrix[2][0] == -4, "composite matrix") + expect( + c.floats.size == 6 && c.floats[0].isNaN() && c.floats[3].toRawBits() < 0, + "composite floats", ) - val fullBack = Codec.roundtripComposite(full) - expect(compositeEquals(fullBack, full), "full Composite round-trips (got ${Codec.describeComposite(fullBack)})") - expect(fullBack.maybe_list != null && fullBack.maybe_list.isEmpty(), "present-but-empty bytes stay present") - expect(fullBack.empty[0].isNaN() && fullBack.empty[2].toRawBits() == (-0.0).toRawBits(), "special doubles inside a list") - expect((fullBack.shapes[0] as Shape.Circle).radius.isNaN(), "NaN inside a rich enum") - expect((fullBack.shapes[1] as Shape.Rect).width.toRawBits() == (-0.0f).toRawBits(), "-0.0f inside a rich enum") - expect(fullBack.name == full.name, "supplementary characters and NUL survive inside a buffer") - expect(Codec.describeComposite(full).contains("日本語 🚀 \\0 emoji and NUL"), "producer decoded the unicode name") -} - -fun checkShapes() { - val canonical = canonicalScalars() - val all = listOf( - Shape.Empty, - Shape.Circle(2.5), - Shape.Rect(1.0f, 0.5f), - Shape.Labeled("tag", 3), - Shape.Nested(canonical, "n"), - Shape.Nested(canonical.copy(flag = false), null), + expect( + c.by_name.size == 4 && c.by_id.size == 3 && c.by_color.size == 2 && c.flags.size == 2, + "composite maps", ) - for (s in all) { - expect(Codec.roundtripShape(s) == s, "roundtripShape($s)") - } - expect(Codec.roundtripShapes(all) == all, "roundtripShapes(all)") - expect(Codec.roundtripShapes(listOf()).isEmpty(), "roundtripShapes(empty)") - expect(Codec.roundtripShape(Shape.Empty) === Shape.Empty, "Empty decodes to the singleton") - expect(Codec.describeShape(Shape.Empty) == "Empty", "describe Empty") - expect(Codec.describeShape(Shape.Circle(2.5)) == "Circle { radius: 2.5 }", "describe Circle (got ${Codec.describeShape(Shape.Circle(2.5))})") - expect(Codec.describeShape(Shape.Rect(1.0f, 0.5f)) == "Rect { width: 1.0, height: 0.5 }", "describe Rect") - expect(Codec.describeShape(Shape.Labeled("tag", 3)) == "Labeled { label: \"tag\", count: 3 }", "describe Labeled") - val nested = Codec.describeShape(Shape.Nested(canonical, null)) - expect(nested.startsWith("Nested { inner: Scalars { i8_value: -8, u8_value: 200,") && nested.endsWith("note: None }"), "describe Nested (got $nested)") - expect(Codec.describeShape(Shape.Labeled("🚀", -1)) == "Labeled { label: \"🚀\", count: -1 }", "describe Labeled with emoji") + expect(c.scalars.u32_value == 4000000000u, "composite scalars.u32_value") + expect(c.shape is Shape.Labeled && (c.shape as Shape.Labeled).count == 3, "composite shape") + val lastShape = c.shapes.last() + expect(c.shapes.size == 6 && lastShape is Shape.Nested && lastShape.note == null, "composite shapes") + expect(c.maybe_shape is Shape.Nested, "composite maybe_shape") + expect(c.maybe_list?.size == 2, "composite maybe_list") + expect(c.sparse.size == 3 && c.sparse[0] == true && c.sparse[1] == null, "composite sparse") + expect(c.colors.size == 4 && c.colors[3] == Color.Infrared, "composite colors") } -fun checkPrimitives() { - expect(Codec.roundtripOptI64(null) == null, "roundtripOptI64(null)") - expect(Codec.roundtripOptI64(Long.MIN_VALUE) == Long.MIN_VALUE, "roundtripOptI64(min)") - expect(Codec.roundtripOptI64(0L) == 0L, "roundtripOptI64(0)") - expect(Codec.roundtripMap(mapOf()) == mapOf(), "roundtripMap(empty)") - val m = mapOf("" to 0L, "a" to -1L, "héllo 🚀" to Long.MAX_VALUE) - expect(Codec.roundtripMap(m) == m, "roundtripMap (got ${Codec.roundtripMap(m)})") +fun outOfRange(n: UInt) { + val e = thrownBy { Codec.vector(n) } + expect(e is CodecException.OutOfRange, "vector(n) raises OutOfRange (got $e)") + val oor = e as CodecException.OutOfRange + expect(oor.code == 1 && oor.index == n && oor.count == n, "OutOfRange payload (index ${oor.index}, count ${oor.count})") + expect(oor.message == "vector $n is out of range (count $n)", "OutOfRange message (got ${oor.message})") - // Direct strings: BMP and supplementary characters both survive the JNI - // crossing (the bridge converts to standard UTF-8, not modified UTF-8). - for (s in listOf("", "plain", "héllo wörld ✓", "rocket 🚀 end", "🚀", "\uFFFF")) { - expect(Codec.roundtripString(s) == s, "roundtripString(${s.toByteArray().toList()})") - } - // Strings cross as UTF-8 bytes plus a length, so interior NULs survive. - for (s in listOf("a\u0000b", "\u0000", "\u0000\u0000end", "\uD83D\uDE80\u0000\uD83D\uDE80")) { - expect(Codec.roundtripString(s) == s, "roundtripString with NUL (${s.toByteArray().toList()})") - } - expect(Codec.roundtripBytes(byteArrayOf()).isEmpty(), "roundtripBytes(empty)") - val allBytes = ByteArray(256) { it.toByte() } - expect(Codec.roundtripBytes(allBytes).contentEquals(allBytes), "roundtripBytes(0..255)") + val e2 = thrownBy { Codec.vectorName(n + 5u) } + expect(e2 is CodecException.OutOfRange, "vectorName(n + 5) raises OutOfRange (got $e2)") + e2 as CodecException.OutOfRange + expect(e2.index == n + 5u && e2.count == n, "vectorName OutOfRange payload") - expect(Codec.roundtripI64(Long.MIN_VALUE) == Long.MIN_VALUE, "roundtripI64(min)") - expect(Codec.roundtripI64(Long.MAX_VALUE) == Long.MAX_VALUE, "roundtripI64(max)") - expect(Codec.roundtripI64(-1L) == -1L, "roundtripI64(-1)") - expect(Codec.roundtripU64(ULong.MAX_VALUE.toLong()).toULong() == ULong.MAX_VALUE, "roundtripU64(max)") - expect(Codec.roundtripU64(Long.MIN_VALUE).toULong() == 9_223_372_036_854_775_808UL, "roundtripU64(2^63)") - expect(Codec.roundtripU64(0L) == 0L, "roundtripU64(0)") - expect(Codec.roundtripF64(Double.NaN).isNaN(), "roundtripF64(NaN)") - expect(Codec.roundtripF64(Double.POSITIVE_INFINITY) == Double.POSITIVE_INFINITY, "roundtripF64(+inf)") - expect(Codec.roundtripF64(Double.NEGATIVE_INFINITY) == Double.NEGATIVE_INFINITY, "roundtripF64(-inf)") - expect(Codec.roundtripF64(-0.0).toRawBits() == (-0.0).toRawBits(), "roundtripF64(-0.0) keeps the sign") - expect(Codec.roundtripF64(Double.MIN_VALUE) == Double.MIN_VALUE, "roundtripF64(subnormal)") - expect(Codec.roundtripF64(Double.MAX_VALUE) == Double.MAX_VALUE, "roundtripF64(max)") - expect(Codec.roundtripBool(true) && !Codec.roundtripBool(false), "roundtripBool") - expect(Codec.roundtripColor(Color.Blue) == Color.Blue, "roundtripColor(Blue)") - expect(Codec.roundtripColor(Color.Red) == Color.Red, "roundtripColor(Red)") - expect(Color.Blue.value == 7 && Color.fromValue(7) == Color.Blue, "Color discriminants") + expect(!Codec.checkVector(n, Vector.Blank), "checkVector past the end is false") } -fun checkHolders() { - // Objects inside a record decoded from a buffer: each token is one adopted - // strong reference wrapped in a Token. - val holder = Codec.makeHolder(10L, true) - expect(holder.primary.value() == 10L, "primary value (got ${holder.primary.value()})") - expect(holder.spare != null && holder.spare.value() == 11L, "spare value (got ${holder.spare?.value()})") - expect(holder.many.map { it.value() } == listOf(12L, 13L, 14L), "many values (got ${holder.many.map { it.value() }})") - expect(holder.many.map { it.handle.address }.toSet().size == 3, "many are distinct objects") +/** A raw buffer that `check_vector` must reject as a marshalling failure. */ +internal fun reject(what: String, write: (BufferWriter) -> Unit) { + val bytes = encodeBuffer(write) + val e = thrownBy { JniBridge.codec_check_vector(0, bytes) } + expect(e is NativeBugException && e.code == -3, "$what is rejected with -3 (got $e)") +} - // Encoding a holder mints one fresh reference per token, so the wrappers - // stay valid after the producer consumed the buffer. - expect(Codec.sumHolder(holder) == 10L + 11L + 12L + 13L + 14L, "sumHolder (got ${Codec.sumHolder(holder)})") - expect(Codec.sumHolder(holder) == 60L, "sumHolder again (wrappers still alive)") - expect(holder.primary.value() == 10L, "primary still usable after encoding") +fun malformed() { + val tagI64 = 7 + reject("a truncated buffer") { it.writeI32(tagI64); it.writeU32(7u) } + reject("an unknown tag") { it.writeI32(999) } + reject("trailing bytes") { it.writeI32(0); it.writeU8(0u) } + reject("a bool that is neither 0 nor 1") { it.writeI32(11); it.writeU8(2u) } + reject("an undeclared enum value") { it.writeI32(14); it.writeI32(3) } + reject("a repeated map key") { + it.writeI32(19) + it.writeU32(2u) + it.writeString("a") + it.writeI64(1) + it.writeString("a") + it.writeI64(2) + } + reject("a string that isn't UTF-8") { it.writeI32(12); it.writeBytes(byteArrayOf(0xC3.toByte(), 0x28)) } + // The tags above are the declaration order of the Vector variants. + expect(encode(Vector.I64(0)).copyOfRange(0, 4).contentEquals(byteArrayOf(7, 0, 0, 0)), "I64 tag") + expect(encode(Vector.Counts(emptyMap()))[0] == 19.toByte(), "Counts tag") - // An object return is the same native object as the record's field. - val primary = Codec.primaryOf(holder) - expect(primary !== holder.primary, "primaryOf returns a new wrapper") - expect(primary.handle.address == holder.primary.handle.address, "primaryOf wraps the same native object") - expect(primary.value() == 10L, "primaryOf value") - expect(Codec.samePrimary(holder, holder), "samePrimary(holder, holder)") - val other = Codec.makeHolder(10L, false) - expect(other.spare == null, "spare absent when not requested") - expect(other.primary.value() == 10L, "other primary value equal but ...") - expect(!Codec.samePrimary(holder, other), "... samePrimary distinguishes distinct objects") - expect(Codec.sumHolder(other) == 10L + 12L + 13L + 14L, "sumHolder without spare") + // The generated decoder rejects the same inputs. + val unknown = thrownBy { decodeBuffer(byteArrayOf(0xE7.toByte(), 0x03, 0, 0)) { unpackVector(it) } } + expect(unknown is NativeBugException && unknown.code == -3, "an unknown tag fails to decode") - // A holder assembled in Kotlin from consumer-created tokens, sharing one - // token across positions. - val shared = Token(100L) - val mine = Holder(primary = shared, spare = shared, many = listOf(shared, Token(1L), primary)) - expect(Codec.sumHolder(mine) == 100L + 100L + 100L + 1L + 10L, "sumHolder over a Kotlin-built holder") - expect(Codec.samePrimary(mine, Holder(shared, null, listOf())), "samePrimary across Kotlin-built holders") - expect(Codec.samePrimary(Holder(primary, null, listOf()), holder), "samePrimary through primaryOf's wrapper") - val mineBack = Codec.primaryOf(mine) - expect(mineBack.handle.address == shared.handle.address && mineBack.value() == 100L, "primaryOf a Kotlin-built holder") + // Color can't spell an undeclared value, but the raw bridge can: the + // producer rejects it with -3, a trap for a call that can't fail. + val badColor = thrownBy { JniBridge.codec_echo_color(3) } + expect(badColor is NativeBugException && badColor.code == -3, "an undeclared Color is rejected (got $badColor)") + val badText = thrownBy { JniBridge.codec_echo_text(byteArrayOf(0xC3.toByte(), 0x28)) } + expect(badText is NativeBugException && badText.code == -3, "invalid UTF-8 is rejected (got $badText)") +} - // Release everything. Closing one wrapper over a shared object leaves the - // others valid; double close is safe; use after close throws; closing a - // token that another wrapper still references never frees the object. - primary.close() - primary.close() - expect(thrownBy { primary.value() } is IllegalStateException, "closed token rejects use") - expect(holder.primary.value() == 10L, "the record's wrapper survives closing primaryOf's wrapper") - expect(thrownBy { Codec.sumHolder(mine) } is IllegalStateException, "encoding a holder with a closed token throws") - mineBack.close() - expect(shared.value() == 100L, "shared token alive after closing its second wrapper") - shared.close() - mine.many[1].close() - holder.primary.close() - holder.spare!!.close() - holder.many.forEach { it.close() } - holder.many.forEach { it.close() } - other.primary.close() - other.many.forEach { it.close() } - expect(thrownBy { Codec.sumHolder(holder) } is IllegalStateException, "fully closed holder rejects encoding") +fun objects(n: UInt) { + val full = fetch(find(n, "objects full")) + expect(full is Vector.Objects, "objects full is Objects") + val h = (full as Vector.Objects).value + expect(h.primary.value() == 10L && h.spare?.value() == 11L, "holder primary and spare") + expect(h.many.map { it.value() } == listOf(12L, 13L, Long.MIN_VALUE), "holder many") + expect(h.by_name.mapValues { it.value.value() } == mapOf("a" to 20L, "b" to 21L), "holder by_name") + // Each encoding mints fresh references, so the holder can be sent twice. + val expected = 10L + 11 + 12 + 13 + 20 + 21 + Long.MIN_VALUE + expect(Codec.sumHolder(h) == expected && Codec.sumHolder(h) == expected, "sumHolder twice") - // Wrappers that are never closed are released by the Cleaner; create a - // batch and let them go. - repeat(50) { Codec.makeHolder(it.toLong(), true) } - System.gc() -} + // primaryOf returns the very same object (a new reference to it). + val p = Codec.primaryOf(h) + expect(p.value() == 10L, "primaryOf value") + expect(Codec.samePrimary(h, Holder(p, null, emptyList(), emptyMap())), "primaryOf is the same object") + val twin = Token(10) + expect(!Codec.samePrimary(h, Holder(twin, null, emptyList(), emptyMap())), "an equal value isn't the same object") + + // A holder built from consumer tokens, the same wrapper in several slots. + val minus4 = Token(-4) + val mine = Holder(twin, twin, listOf(twin, twin, minus4), mapOf("k" to twin)) + expect(Codec.sumHolder(mine) == 10L * 5 - 4, "consumer holder sums to 46") + + // A zero token is a marshalling failure, not a crash. + val zero = thrownBy { JniBridge.codec_sum_holder(ByteArray(17)) } + expect(zero is NativeBugException && zero.code == -3, "a zero token is rejected (got $zero)") -fun run() { - checkScalars() - checkComposite() - checkShapes() - checkPrimitives() - checkHolders() - println("kotlin/codec: OK") + minus4.close() + twin.close() + twin.close() // closing twice is safe + p.close() + release(full) + expect(thrownBy { p.value() } is IllegalStateException, "a closed wrapper can't be used") } fun main() { - run() - expectNoLeaks { JniBridge.debug_live(it) } - println("kotlin/codec: no leaks") + expect(JniBridge.debug_live(-1) == 1L, "the sample counts live allocations") + val n = Codec.vectorCount() + expect(n >= 60u, "at least 60 vectors (got $n)") + + everyVector(n) + literalVectors(n) + spotChecks(n) + outOfRange(n) + malformed() + objects(n) + + expectNoLeaks(JniBridge::debug_live) + println("kotlin/codec: OK ($n vectors)") } diff --git a/conformance/kotlin/contacts.kt b/conformance/kotlin/contacts.kt deleted file mode 100644 index 6b4366b4..00000000 --- a/conformance/kotlin/contacts.kt +++ /dev/null @@ -1,144 +0,0 @@ -// Conformance consumer: contacts sample, Kotlin (JVM via JNI) target. -// -// Exercises the struct/enum/optional surface across three modules: -// `ContactBook` is a generated `AutoCloseable` class (companion `invoke` for -// the `new` constructor, instance methods, release through `close()`), -// `Contact` is a data class decoded from a value buffer (the nullable email -// rides inside it), enums cross as `ContactType` values, and `ContactsError` -// is a typed exception domain (`ContactsException` sealed subclasses -// extending `FfiException`). The nested `contacts.groups` module -// (`Contacts.Groups`) takes the parent's interface and C-style enum, returns -// a new parent-module object and record, and reports the parent's domain; the -// sibling top-level `directory` module (`Directory`) takes the `contacts` -// record by value, alone and in a list, and reports its own domain. At exit -// it checks that every native resource was released. -@file:JvmName("Main") - -import contacts.Card -import contacts.Contact -import contacts.ContactBook -import contacts.ContactType -import contacts.Contacts -import contacts.ContactsException -import contacts.Directory -import contacts.DirectoryException -import contacts.FfiException -import contacts.JniBridge - -fun checkBook() { - ContactBook().use { book -> - expect(book.count() == 0, "fresh book empty") - - // add() returns the stored record with its assigned id. - val alice = book.add("Alice", "Smith", "alice@example.com", ContactType.Work) - expect(alice.id > 0L, "alice id positive") - expect(alice.first_name == "Alice", "alice first_name (got ${alice.first_name})") - expect(alice.last_name == "Smith", "alice last_name") - expect(alice.email == "alice@example.com", "alice email") - expect(alice.contact_type == ContactType.Work, "alice contact_type") - - // Nullable email: a missing value round-trips as null. Records are - // plain data classes, so a fetched contact compares equal to a - // locally constructed one field by field. - val bob = book.add("Bob", "Jones", null, ContactType.Personal) - val fetched = book.get(bob.id) - expect(fetched.email == null, "bob email null (got ${fetched.email})") - expect(fetched.contact_type == ContactType.Personal, "bob contact_type") - expect( - fetched == Contact(bob.id, "Bob", "Jones", null, ContactType.Personal), - "fetched bob equals constructed Contact (got $fetched)" - ) - - expect(book.count() == 2, "count == 2") - val everyone = book.list() - expect(everyone.size == 2, "list length == 2 (got ${everyone.size})") - expect( - everyone.map { it.first_name }.toSet() == setOf("Alice", "Bob"), - "list names (got ${everyone.map { it.first_name }})" - ) - - // remove returns whether the id existed; a second remove reports false. - expect(book.remove(alice.id), "remove returns true") - expect(!book.remove(alice.id), "second remove returns false") - expect(book.count() == 1, "count == 1 after remove") - - // Typed error from a method: a missing id reports NotFound (2). - val missingErr = thrownBy { book.get(9999L) } - expect( - missingErr is ContactsException.NotFound, - "get(9999) throws ContactsException.NotFound (got $missingErr)" - ) - expect(missingErr is FfiException, "NotFound is an FfiException") - val missingCode = (missingErr as? FfiException)?.code - expect(missingCode == 2, "NotFound code 2 (got $missingCode)") - - // Typed error: an empty first name is rejected with InvalidName (1) - // and leaves the book unchanged. - val invalidErr = thrownBy { book.add("", "Nameless", null, ContactType.Other) } - expect( - invalidErr is ContactsException.InvalidName, - "add(\"\") throws ContactsException.InvalidName (got $invalidErr)" - ) - val invalidCode = (invalidErr as? FfiException)?.code - expect(invalidCode == 1, "InvalidName code 1 (got $invalidCode)") - expect(book.count() == 1, "failed add leaves count == 1") - - // Each ContactBook object owns its own state. - ContactBook().use { other -> - expect(other.count() == 0, "fresh second book empty") - expect(book.count() == 1, "first book unaffected") - } - } -} - -fun checkGroups() { - ContactBook().use { book -> - book.add("Ann", "Lee", null, ContactType.Work) - book.add("Ben", "Ito", "ben@example.com", ContactType.Work) - book.add("Cy", "Ng", null, ContactType.Personal) - - // The nested module takes the parent's interface and enum. - expect(Contacts.Groups.countOfType(book, ContactType.Work) == 2, "two work contacts") - expect(Contacts.Groups.countOfType(book, ContactType.Other) == 0, "no other contacts") - expect(Contacts.Groups.dominantType(book) == ContactType.Work, "work dominates") - - // It returns a new object of the parent module's interface type. - Contacts.Groups.splitByType(book, ContactType.Work).use { work -> - expect(work.count() == 2, "split book holds the work contacts (got ${work.count()})") - expect(work.list().all { it.contact_type == ContactType.Work }, "split book is all work") - expect(book.count() == 3, "source book unchanged") - } - - // And the parent's record and error domain. - val first = Contacts.Groups.firstOfType(book, ContactType.Personal) - expect(first.first_name == "Cy", "first personal contact (got $first)") - val none = thrownBy { Contacts.Groups.firstOfType(book, ContactType.Other) } - expect(none is ContactsException.NotFound, "firstOfType(Other) throws NotFound (got $none)") - } - ContactBook().use { empty -> - expect(Contacts.Groups.dominantType(empty) == ContactType.Personal, "empty book defaults to Personal") - } -} - -fun checkDirectory() { - // A sibling top-level module taking the `contacts` record by value. - val ann = Contact(1L, "Ann", "Lee", "ann@example.com", ContactType.Work) - val cy = Contact(2L, "Cy", "Ng", null, ContactType.Personal) - expect(Directory.card(ann) == Card("Lee, Ann", "AL", true), "card(ann) (got ${Directory.card(ann)})") - expect(Directory.card(cy) == Card("Ng, Cy", "CN", false), "card(cy)") - val zed = Contact(3L, "Zed", "Abe", null, ContactType.Other) - expect(Directory.sorted(listOf(cy, ann, zed)) == listOf(zed, ann, cy), "sorted by last name") - val empty = thrownBy { Directory.sorted(listOf()) } - expect(empty is DirectoryException.Empty, "sorted(empty) throws DirectoryException.Empty (got $empty)") - expect((empty as FfiException).code == 1, "Empty code 1") - expect(empty !is ContactsException, "the directory domain is its own") -} - -fun main() { - checkBook() - checkGroups() - checkDirectory() - println("kotlin/contacts: OK") - expectNoLeaks { JniBridge.debug_live(it) } - println("kotlin/contacts: no leaks") -} diff --git a/conformance/kotlin/events.kt b/conformance/kotlin/events.kt deleted file mode 100644 index 871720ab..00000000 --- a/conformance/kotlin/events.kt +++ /dev/null @@ -1,254 +0,0 @@ -// Conformance consumer: events sample, Kotlin (JVM via JNI) target. -// -// Exercises the callback-interface and object surface: `Subscriber` is a -// generated Kotlin interface the consumer implements (the JNI vtable -// trampolines pin the implementing object with a global reference and call -// the `JniBridge` dispatch shims from any producer thread), `EventBus` is a -// reference-counted `AutoCloseable` wrapper, and `Message` is a data class -// decoded from a value buffer. Asserts that every callback method is invoked -// with the right arguments (including the bus object handed to -// `onAttached`, which is usable and independently closeable), that -// `Delivery` return values steer `publish`'s accepted count, that a Kotlin -// exception thrown inside a callback (including `route`, which the producer -// declares as returning `Result<_, ForeignError>`) surfaces to the caller as -// `FfiException` code -4, the `suspend` `publishLater`, the -// `NativeIterator`-backed `messages()`, the nullable `lastMessage()`, -// `Events.routeOnce` without a bus, close semantics, and, at exit, that -// every native resource was released. -@file:JvmName("Main") - -import events.Delivery -import events.EventBus -import events.Message -import events.Subscriber -import events.Events -import events.JniBridge -import events.FfiException -import java.lang.ref.WeakReference -import kotlinx.coroutines.runBlocking - -/** - * A subscriber that records every callback. `skipTopic` is routed as Skip, - * `stopTopic` as AcceptAndStop, `failTopic` throws from `route`, and - * `failOnMessage` throws from `onMessage` instead. - */ -class RecordingSubscriber( - val name: String, - private val skipTopic: String? = null, - private val stopTopic: String? = null, - private val failTopic: String? = null, - private val failOnMessage: Boolean = false, - private val failOnAttached: Boolean = false, -) : Subscriber { - val routed = mutableListOf() - val messages = mutableListOf() - var attachedCount = 0 - var subscriberCountAtAttach = -1L - var keptBus: EventBus? = null - - override fun route(topic: String): Delivery { - routed.add(topic) - if (topic == failTopic) throw IllegalStateException("subscriber $name rejected $topic") - return when (topic) { - skipTopic -> Delivery.Skip - stopTopic -> Delivery.AcceptAndStop - else -> Delivery.Accept - } - } - - override fun onMessage(message: Message): Long { - if (failOnMessage) throw RuntimeException("subscriber $name failed on message ${message.seq}") - messages.add(message) - return messages.size.toLong() - } - - override fun onAttached(bus: EventBus) { - attachedCount++ - if (failOnAttached) throw IllegalArgumentException("subscriber $name refused to attach") - // The bus arrives as an adopted strong reference: it's usable right - // here (subscribe hands it over before it takes its lock) and we keep - // the wrapper so a later close proves it's independent of the caller's. - subscriberCountAtAttach = bus.subscriberCount() - keptBus = bus - } -} - -/** Route once through a subscriber nothing else references, returning a weak handle to it. */ -fun routeOnceThrowaway(): WeakReference { - val sub = RecordingSubscriber("throwaway") - expect(Events.routeOnce(sub, "t") == Delivery.Accept, "routeOnce throwaway") - return WeakReference(sub) -} - -/** Subscribe a subscriber nothing else references, returning a weak handle to it. */ -fun subscribeThrowaway(bus: EventBus): WeakReference { - val sub = RecordingSubscriber("gone") - expect(bus.subscribe(sub) == 4L, "subscribe gone") - sub.keptBus!!.close() - return WeakReference(sub) -} - -fun run() { - val bus = EventBus() - expect(bus.subscriberCount() == 0L, "fresh bus has no subscribers") - expect(bus.lastMessage() == null, "fresh bus has no last message") - expect(bus.messages().asSequence().toList().isEmpty(), "fresh bus has no messages") - expect(bus.publish("void", "nobody home", listOf()) == 0L, "publish with no subscribers accepts 0") - - // Subscribe three subscribers; each gets exactly one onAttached with the - // bus, observing the count before it was added. - val quiet = RecordingSubscriber("quiet", skipTopic = "quiet") - val stopper = RecordingSubscriber("stopper", stopTopic = "stop") - val tail = RecordingSubscriber("tail") - expect(bus.subscribe(quiet) == 1L, "first subscribe returns 1") - expect(bus.subscribe(stopper) == 2L, "second subscribe returns 2") - expect(bus.subscribe(tail) == 3L, "third subscribe returns 3") - expect(bus.subscriberCount() == 3L, "subscriberCount == 3") - for ((i, s) in listOf(quiet, stopper, tail).withIndex()) { - expect(s.attachedCount == 1, "${s.name} attached once (got ${s.attachedCount})") - expect( - s.subscriberCountAtAttach == i.toLong(), - "${s.name} saw $i subscribers at attach (got ${s.subscriberCountAtAttach})" - ) - expect(s.keptBus != null, "${s.name} kept the bus") - } - - // Delivery steering: Accept everywhere -> 3; Skip for `quiet` -> 2; - // AcceptAndStop in the middle -> 2 and `tail` never sees it. - expect(bus.publish("news", "hello", listOf("a", "b")) == 2L + 1L, "publish news accepted by 3") - expect(bus.publish("quiet", "psst", listOf()) == 2L, "publish quiet accepted by 2") - expect(bus.publish("stop", "last", listOf("z")) == 2L, "publish stop accepted by 2") - expect(quiet.routed == listOf("news", "quiet", "stop"), "quiet routed every topic (got ${quiet.routed})") - expect(stopper.routed == listOf("news", "quiet", "stop"), "stopper routed every topic") - expect(tail.routed == listOf("news", "quiet"), "tail never routed stop (got ${tail.routed})") - expect(quiet.messages.map { it.text } == listOf("hello", "last"), "quiet got hello,last (got ${quiet.messages.map { it.text }})") - expect(stopper.messages.map { it.text } == listOf("hello", "psst", "last"), "stopper got all three") - expect(tail.messages.map { it.text } == listOf("hello", "psst"), "tail got hello,psst (got ${tail.messages.map { it.text }})") - - // Message record fields decoded from the callback's buffer. seq counts - // from 1 and includes the subscriber-less publish above. - val first = stopper.messages[0] - expect(first == Message(2L, "news", "hello", listOf("a", "b")), "first message record (got $first)") - expect(stopper.messages[1].tags.isEmpty(), "psst carries no tags") - expect(stopper.messages[2] == Message(4L, "stop", "last", listOf("z")), "last message record (got ${stopper.messages[2]})") - - // Async publish via the suspend wrapper, resumed from the producer thread. - val later = runBlocking { bus.publishLater("later", "async hello") } - expect(later == 3L, "publishLater accepted by 3 (got $later)") - expect(stopper.messages.last().text == "async hello", "async message delivered") - expect(stopper.messages.last().tags.isEmpty(), "async message has no tags") - expect(stopper.messages.last().seq == 5L, "async message seq 5 (got ${stopper.messages.last().seq})") - - // Iterator-backed messages(): drained as a Sequence, and again by hand. - val texts = bus.messages().asSequence().toList() - expect( - texts == listOf("nobody home", "hello", "psst", "last", "async hello"), - "messages in publish order (got $texts)" - ) - val byHand = mutableListOf() - val it = bus.messages() - while (it.hasNext()) byHand.add(it.next()) - expect(byHand == texts, "manual iteration matches") - expect(thrownBy { it.next() } is NoSuchElementException, "exhausted iterator throws NoSuchElementException") - // An abandoned iterator can be closed early, and closed twice. - val partial = bus.messages() - expect(partial.next() == "nobody home", "partial iterator first element") - partial.close() - partial.close() - expect(!partial.hasNext(), "closed iterator reports no more elements") - - // Nullable record return. - val last = bus.lastMessage() - expect(last == Message(5L, "later", "async hello", listOf()), "lastMessage (got $last)") - - // routeOnce: a free function taking the callback interface without a bus. - expect(Events.routeOnce(quiet, "quiet") == Delivery.Skip, "routeOnce quiet -> Skip") - expect(Events.routeOnce(stopper, "stop") == Delivery.AcceptAndStop, "routeOnce stop -> AcceptAndStop") - expect(Events.routeOnce(tail, "anything") == Delivery.Accept, "routeOnce -> Accept") - expect(quiet.routed.last() == "quiet", "routeOnce reached route()") - // The producer drops its only reference when routeOnce returns, so the - // GlobalRef pinning a throwaway subscriber is released and it becomes - // collectable once the frame that created it is gone. - expect(collected(routeOnceThrowaway()), "routeOnce released its subscriber (free ran)") - - // The bus handed to onAttached is a distinct wrapper over the same - // object: usable, and closing it leaves the caller's wrapper alive. - val kept = tail.keptBus!! - expect(kept.handle.address == bus.handle.address, "onAttached bus is the same native object") - expect(kept.subscriberCount() == 3L, "kept bus reads the live subscriber count") - kept.close() - kept.close() - expect(thrownBy { kept.subscriberCount() } is IllegalStateException, "kept bus rejects use after close") - expect(bus.subscriberCount() == 3L, "original bus still alive after closing the kept wrapper") - quiet.keptBus!!.close() - stopper.keptBus!!.close() - - // Foreign errors: a Kotlin exception thrown from any callback method - // surfaces to the caller as FfiException(-4) carrying the throwable's - // text, and the JVM keeps running. Use a fresh bus for exact counts. - EventBus().use { bus2 -> - val ok = RecordingSubscriber("ok") - val rejecter = RecordingSubscriber("rejecter", failTopic = "boom") - bus2.subscribe(ok) - bus2.subscribe(rejecter) - ok.keptBus!!.close() - rejecter.keptBus!!.close() - val routeErr = thrownBy { bus2.publish("boom", "x", listOf()) } - expect(routeErr is FfiException, "throwing route() surfaces as FfiException (got $routeErr)") - expect((routeErr as FfiException).code == -4, "foreign error code -4 (got ${routeErr.code})") - expect( - routeErr.message?.contains("rejected boom") == true, - "foreign error carries the Kotlin message (got ${routeErr.message})" - ) - expect(routeErr.message?.contains("IllegalStateException") == true, "foreign error names the exception class") - // The earlier subscriber was still delivered to, the producer logged - // the message, and the bus remains usable. - expect(ok.messages.map { it.text } == listOf("x"), "subscriber before the failure still received x") - expect(bus2.lastMessage()?.text == "x", "producer logged the message despite the failure") - expect(bus2.publish("fine", "y", listOf()) == 2L, "bus usable after a foreign error") - - val onMsg = RecordingSubscriber("onMsg", failOnMessage = true) - bus2.subscribe(onMsg) - onMsg.keptBus!!.close() - val msgErr = thrownBy { bus2.publish("any", "z", listOf()) } - expect(msgErr is FfiException && msgErr.code == -4, "throwing onMessage() surfaces as -4 (got $msgErr)") - expect(msgErr?.message?.contains("failed on message") == true, "onMessage error text (got ${msgErr?.message})") - - // Throwing from onAttached aborts subscribe itself and the subscriber - // isn't retained. - val refuser = RecordingSubscriber("refuser", failOnAttached = true) - val attachErr = thrownBy { bus2.subscribe(refuser) } - expect(attachErr is FfiException && attachErr.code == -4, "throwing onAttached() surfaces as -4 (got $attachErr)") - expect(bus2.subscriberCount() == 3L, "refused subscriber not retained (got ${bus2.subscriberCount()})") - - // routeOnce also propagates the foreign error. - val onceErr = thrownBy { Events.routeOnce(rejecter, "boom") } - expect(onceErr is FfiException && onceErr.code == -4, "routeOnce propagates -4 (got $onceErr)") - expect(Events.routeOnce(rejecter, "calm") == Delivery.Accept, "rejecter still works for other topics") - } - - // clearSubscribers drops every retained subscriber (each free runs), and - // the bus keeps its log. - val goneRef = subscribeThrowaway(bus) - expect(bus.subscriberCount() == 4L, "subscribe gone") - bus.clearSubscribers() - expect(bus.subscriberCount() == 0L, "clearSubscribers empties the bus") - expect(collected(goneRef), "cleared subscriber was released by the producer") - expect(bus.publish("after", "nobody", listOf()) == 0L, "publish after clear accepts 0") - expect(bus.lastMessage()?.text == "nobody", "log continues after clear") - expect(quiet.messages.size == 3, "cleared subscribers receive nothing more") - - // Close semantics: double close is safe, use after close throws. - bus.close() - bus.close() - expect(thrownBy { bus.subscriberCount() } is IllegalStateException, "bus rejects use after close") - expect(thrownBy { bus.publish("x", "y", listOf()) } is IllegalStateException, "publish rejects use after close") - - println("kotlin/events: OK") -} - -fun main() { - run() - expectNoLeaks { JniBridge.debug_live(it) } - println("kotlin/events: no leaks") -} diff --git a/conformance/kotlin/kvstore.kt b/conformance/kotlin/kvstore.kt index 0fb527d5..fd06c69c 100644 --- a/conformance/kotlin/kvstore.kt +++ b/conformance/kotlin/kvstore.kt @@ -1,347 +1,510 @@ // Conformance consumer: kvstore sample, Kotlin (JVM via JNI) target. // -// Exercises the reference-counted interface surface: `Store` is a generated -// `AutoCloseable` class (companion factory `open`, instance methods, statics -// `defaultCapacity`/`openMany`/`totalCount`; `close()` releases one strong -// reference and the phantom-reference cleaner is the backstop), `Entry`, -// `Stats`, and `StoreInfo` are data classes decoded from value buffers -// (`StoreInfo` carries a `Store` object and a nullable one as fields), and -// `KvError` is a typed exception domain (`KvException` sealed subclasses -// extending `FfiException`). Asserts the typed-error paths (IoError from -// `open("")` and `openMany`, KeyNotFound, Expired), the object graph -// (`share()` wraps the same object, `fork()` is independent, `larger(null)`, -// `describe().store`, `openMany`, `totalCount` with objects inside lists and -// records), the consumer-implemented `EvictionListener` callback interface -// (decoded `Entry` and `EvictionReason`; returning false detaches; replacing -// or clearing releases the previous listener; a Kotlin exception thrown -// inside `onEvict` surfaces as `FfiException` code -4), record -// materialization, buffered optional parameters, the `NativeIterator`-backed -// `listKeys`, the `Entry` pack/unpack round trip, the nested `kv.stats` -// module (`Kv.StatsModule`, renamed beside the `Stats` record), the -// deprecated `legacyPut`, the cancellable suspend `compact`, close semantics, -// and, at exit, that every native resource was released. Compiled against -// the generated bindings jar with `-Xfriend-paths`, so `internal` helpers are -// reachable. +// Drives the feature-complete producer through the generated bindings: +// +// * the load-time checks (ABI revision, both modules' contract tables), +// which loading the bindings performs; +// * `Store`: fallible and infallible constructors (`Store.open`, `Store()`), +// methods, statics, the deprecated `size()`, the `Entry` and `StoreInfo` +// data classes, the `EntryKind` enum class, maps and optionals, unsigned +// counts as `UInt`/`ULong`, and the logical clock; +// * `KvException` subclasses with their payload fields (`KeyNotFound`, +// `Expired`, `StoreFull`, `Rejected`), and runtime failures (-4) as the +// root `FfiException`; +// * lazy `NativeIterator`s of strings (throwing at launch), records, and +// objects, including one abandoned part-way; +// * three callback interfaces implemented in Kotlin: a `Listener` +// (retained, filtered by `accepts`, told about every `Change`, detached +// when it throws, and notified from a producer thread during +// compaction), a `Policy` (a record return, a throwing method whose +// `KvException.Rejected` reaches the `put` caller with its payload, an +// object parameter and object return), and a `Loader` passed as an +// optional callback (string, bytes, and optional-object returns; typed +// errors decoded by the producer or passed through); +// * `Store` objects in every position: parameter, return, optional, list, +// map value, record field, iterator element, async result, and callback +// parameter and return; +// * suspend functions: an async free function returning an object, a +// cancellable method cancelled mid-pause (completing at once while its +// background work stops cooperatively, shown by `activeJobs`), an async +// list launched concurrently, and an async function in the nested +// `kv.stats` module (`Kv.StatsModule`); +// * the sibling `report` root (the shared `Entry` record and its own error +// domain). +// +// Releases of consumer callbacks are observed through the producer's +// callback counter (`debug_live(1)`). Ends by asserting the producer's leak +// counters are zero. Compiled with `-Xfriend-paths`, so the bindings' +// `internal` members (the bridge and each wrapper's handle) are reachable. @file:JvmName("Main") +import java.util.Collections +import java.util.concurrent.atomic.AtomicInteger +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.async +import kotlinx.coroutines.awaitAll +import kotlinx.coroutines.delay +import kotlinx.coroutines.runBlocking +import kvstore.Change import kvstore.Entry import kvstore.EntryKind -import kvstore.EvictionListener -import kvstore.EvictionReason +import kvstore.FfiException +import kvstore.JniBridge +import kvstore.Kv import kvstore.KvException +import kvstore.Listener +import kvstore.Loader +import kvstore.Policy +import kvstore.Report +import kvstore.ReportException +import kvstore.Stats import kvstore.Store import kvstore.StoreInfo -import kvstore.JniBridge -import kvstore.Kv -import kvstore.FfiException -import kvstore.packEntry -import kvstore.unpackEntry -import kvstore.decodeBuffer -import kvstore.encodeBuffer -import java.lang.ref.WeakReference -import kotlinx.coroutines.runBlocking -/** - * A listener that records every eviction. It detaches itself (returns false) - * once it has seen `keepAfter` evictions, and throws when the evicted key is - * `failOnKey`. - */ -class RecordingListener( - val name: String, - private val keepAfter: Int = Int.MAX_VALUE, - private val failOnKey: String? = null, -) : EvictionListener { - val evictions = mutableListOf>() - val entries = mutableListOf() - - override fun onEvict(entry: Entry, reason: EvictionReason): Boolean { - if (entry.key == failOnKey) throw IllegalStateException("listener $name refused ${entry.key}") - evictions.add(entry.key to reason) - entries.add(entry) - return evictions.size < keepAfter +val mainThread: Thread = Thread.currentThread() + +/** Live consumer callbacks the producer holds. */ +fun liveCallbacks(): Long = JniBridge.debug_live(1) + +/** The native object a wrapper refers to (identity, not value). */ +fun Store.address(): Long = handle.address + +fun bytes(s: String) = s.toByteArray(Charsets.UTF_8) + +fun text(b: ByteArray) = String(b, Charsets.UTF_8) + +fun Store.putText(key: String, value: String, kind: EntryKind = EntryKind.Persistent, ttl: Long? = null): Entry = + put(key, bytes(value), kind, ttl) + +inline fun expectThrows(what: String, block: () -> Unit): E { + val e = thrownBy(block) + expect(e is E, "$what raises ${E::class.simpleName} (got $e)") + return e as E +} + +fun expectKeyNotFound(key: String, block: () -> Unit) { + val e = expectThrows("key $key", block) + expect(e.code == 1001 && e.key == key, "KeyNotFound payload key $key (got ${e.key})") +} + +// ── listener (consumer-implemented, retained) ───────────────────────────── + +class RecordingListener(private val skip: String? = null, private val failOn: String? = null) : Listener { + val changes: MutableList = Collections.synchronizedList(mutableListOf()) + val offMainThread = AtomicInteger() + + override fun accepts(key: String): Boolean { + if (key == failOn) throw IllegalStateException("listener refused") + return key != skip + } + + override fun onChange(change: Change) { + if (Thread.currentThread() !== mainThread) offMainThread.incrementAndGet() + changes.add(change) } + + fun puts() = changes.filterIsInstance() + + fun removed(expired: Boolean) = changes.filterIsInstance().filter { it.expired == expired } } -/** Attach a listener nothing else references, returning a weak handle to it. */ -fun attachThrowaway(store: Store, name: String): WeakReference { - val listener = RecordingListener(name) - store.setEvictionListener(listener) - return WeakReference(listener) +// ── policy (consumer-implemented, rich returns, throws) ─────────────────── + +/** Routes `b/` keys to [other], a reference this policy owns. */ +class TestPolicy(private val other: Store) : Policy { + val admitted = AtomicInteger() + + override fun admit(entry: Entry): Entry { + admitted.incrementAndGet() + expect(entry.version == 0u, "the store assigns the version after admission") + return when { + entry.key.startsWith("secret") -> + throw KvException.Rejected(entry.key, "no secrets", "secrets are not stored") + entry.key.startsWith("boom") -> throw IllegalStateException("policy exploded") + // Tag it and store it encrypted (and try to rename it, which the + // store ignores). + else -> entry.copy(key = "renamed", kind = EntryKind.Encrypted, tags = listOf("admitted")) + } + } + + override fun route(key: String, home: Store): Store = if (key.startsWith("b/")) other else home } -/** Attach a self-detaching listener and drive one eviction, returning a weak handle to it. */ -fun attachSelfDetaching(store: Store, payload: ByteArray): WeakReference { - val listener = RecordingListener("once", keepAfter = 1) - store.setEvictionListener(listener) - expect(store.put("first", payload, EntryKind.Volatile, null), "put first") - expect(store.put("second", payload, EntryKind.Volatile, null), "put second") - expect(store.delete("first"), "delete first") - expect(listener.evictions == listOf("first" to EvictionReason.Deleted), "self-detaching listener saw first") - // It returned false, so the store dropped it; the next eviction isn't seen. - expect(store.delete("second"), "delete second") - expect(listener.evictions.size == 1, "detached listener saw nothing more (got ${listener.evictions})") - return WeakReference(listener) +// ── loader (consumer-implemented, passed as an optional parameter) ──────── + +class TestLoader(private val backup: Store? = null) : Loader { + override fun name() = "kotlin-loader" + + override fun fallback(key: String): Store? = if (key == "fb") backup else null + + override fun load(key: String): ByteArray = when (key) { + "missing" -> throw KvException.KeyNotFound("missing", "not in the loader") + "elsewhere" -> throw KvException.KeyNotFound("other", "not in the loader") + "broken" -> throw IllegalStateException("loader is broken") + else -> bytes("loaded:$key") + } +} + +// ── sections ────────────────────────────────────────────────────────────── + +fun constructors() { + val e = expectThrows("Store.open(\"\")") { Store.open("") } + expect(e.code == 1004 && e.message == "invalid path", "InvalidPath code and message") + + Store().use { s -> + expect(s.path() == "memory", "Store() path") + expect(s.capacity() == Store.defaultCapacity(), "capacity == defaultCapacity") + expect(Store.defaultCapacity() == 1_000_000u, "defaultCapacity == 1000000") + } + + runBlocking { + Kv.openStore("/async").use { s -> expect(s.path() == "/async", "openStore path") } + val err = thrownBy { Kv.openStore("") } + expect(err is KvException.InvalidPath, "openStore(\"\") rejects with InvalidPath (got $err)") + } } -fun run() { - // Typed error from a constructor: an empty path is rejected with the - // domain's IoError (1004), which is both the sealed domain type and the - // generic brand exception. - val openErr = thrownBy { Store.open("") } - expect(openErr is KvException.IoError, "open(\"\") throws KvException.IoError (got $openErr)") - expect(openErr is KvException, "IoError is a KvException") - expect(openErr is FfiException, "IoError is a FfiException") - val openCode = (openErr as? FfiException)?.code - expect(openCode == 1004, "IoError code 1004 (got $openCode)") - expect(openErr?.message == "I/O failure", "IoError message (got ${openErr?.message})") - - // Static on the interface's companion. - expect(Store.defaultCapacity() == 1_000_000L, "defaultCapacity == 1000000") - - Store.open("/tmp/conformance-kvstore-kotlin").use { store -> - val payload = byteArrayOf(1, 2, 3) - expect(store.put("alpha", payload, EntryKind.Persistent, null), "put alpha") - expect(store.put("beta", payload, EntryKind.Volatile, 3600L), "put beta with ttl") - expect(store.count() == 2L, "count == 2") - - // Deprecated method still round-trips (volatile, no TTL). - @Suppress("DEPRECATION") - val legacyOk = store.legacyPut("legacy", payload) - expect(legacyOk, "legacyPut inserts") - expect(store.count() == 3L, "count == 3 after legacyPut") - expect(store.delete("legacy"), "delete legacy") - expect(!store.delete("legacy"), "second delete reports false") - - // Iterator-backed list-of-string return, drained through Kotlin's - // Iterator and as a Sequence; the backing BTreeMap yields sorted - // order. The absent prefix crosses as a buffered `string?`. - val keys = mutableListOf() - val it = store.listKeys(null) - while (it.hasNext()) keys.add(it.next()) - expect(keys == listOf("alpha", "beta"), "listKeys sorted (got $keys)") - expect(thrownBy { it.next() } is NoSuchElementException, "exhausted iterator throws") - expect(store.listKeys("al").asSequence().toList() == listOf("alpha"), "listKeys prefix filter") - expect(store.listKeys("zz").asSequence().toList().isEmpty(), "listKeys unmatched prefix") - - // Buffered `Entry?` return, decoded into the data class. - val alpha = store.get("alpha") - expect(alpha != null, "get alpha present") - expect(alpha!!.id > 0, "entry id positive") - expect(alpha.key == "alpha", "entry key") - expect(alpha.value.contentEquals(payload), "entry value bytes") - expect(alpha.expires_at == null, "alpha expires_at null") - expect(alpha.tags.isEmpty(), "alpha tags empty") - expect(alpha.metadata.isEmpty(), "alpha metadata empty") - - val beta = store.get("beta") - expect(beta != null && beta.expires_at != null && beta.expires_at > beta.created_at, "beta expires_at present") - expect(beta!!.id == alpha.id + 1, "ids are monotonic (got ${alpha.id}, ${beta.id})") - - // Typed error from a method: a missing key reports KeyNotFound (1001). - val missingErr = thrownBy { store.get("missing") } - expect(missingErr is KvException.KeyNotFound, "get(missing) throws KvException.KeyNotFound (got $missingErr)") - expect((missingErr as FfiException).code == 1001, "KeyNotFound code 1001 (got ${missingErr.code})") - expect(missingErr.message == "key not found", "KeyNotFound message (got ${missingErr.message})") - - // TTL expiry: a zero-TTL entry is already expired, so `get` reports - // Expired (1002) and evicts the entry on read. - expect(store.put("ephemeral", payload, EntryKind.Volatile, 0L), "put ephemeral") - val expiredErr = thrownBy { store.get("ephemeral") } - expect(expiredErr is KvException.Expired, "get(expired) throws KvException.Expired (got $expiredErr)") - expect((expiredErr as FfiException).code == 1002, "Expired code 1002") - expect(store.count() == 2L, "expired entry evicted on read") - - // An Entry with a non-empty list + map round-trips through the - // generated pack/unpack routines. - val built = Entry( - id = 7L, - key = "built", - value = payload, - created_at = 1000L, - expires_at = null, - tags = listOf("hot", "fast"), - metadata = mapOf("source" to "test", "env" to "prod"), - ) - val builtBack = decodeBuffer(encodeBuffer { w -> packEntry(w, built) }) { r -> unpackEntry(r) } - expect(builtBack.tags == listOf("hot", "fast"), "built tags") - expect(builtBack.metadata == mapOf("source" to "test", "env" to "prod"), "built metadata") - expect(builtBack.value.contentEquals(payload), "built value bytes") - expect(builtBack.expires_at == null, "built expires_at null") - - val empty = Entry(8L, "empty", payload, 1L, 99L, listOf(), emptyMap()) - val emptyBack = decodeBuffer(encodeBuffer { w -> packEntry(w, empty) }) { r -> unpackEntry(r) } - expect(emptyBack.tags.isEmpty() && emptyBack.metadata.isEmpty(), "empty collections") - expect(emptyBack.expires_at == 99L, "empty expires_at present") - - // kv.stats submodule: free function taking the interface (borrowed - // handle) and returning a buffered record. - val stats = Kv.StatsModule.getStats(store) - expect(stats.total_entries == 2L, "stats total entries == 2") - expect(stats.total_bytes == 6L, "stats total bytes == 6 (got ${stats.total_bytes})") - expect(stats.expired_entries == 0L, "stats expired == 0") - - // --- Callback interface: EvictionListener ------------------------- - - // Deleting an existing key fires onEvict synchronously with the - // decoded Entry and reason Deleted; a TTL-expired read fires Expired. - val listener = RecordingListener("main") - store.setEvictionListener(listener) - expect(store.delete("beta"), "delete beta") - expect(listener.evictions == listOf("beta" to EvictionReason.Deleted), "eviction for beta (got ${listener.evictions})") - expect(listener.entries[0].id == beta.id, "evicted entry carries beta's id") - expect(listener.entries[0].value.contentEquals(payload), "evicted entry carries beta's bytes") - expect(listener.entries[0].expires_at == beta.expires_at, "evicted entry carries beta's expiry") - expect(!store.delete("beta"), "delete of a missing key") - expect(listener.evictions.size == 1, "missing key evicts nothing") - expect(store.put("gone", payload, EntryKind.Volatile, 0L), "put gone with zero TTL") - expect(thrownBy { store.get("gone") } is KvException.Expired, "gone is expired") - expect( - listener.evictions == listOf("beta" to EvictionReason.Deleted, "gone" to EvictionReason.Expired), - "expiry eviction (got ${listener.evictions})" - ) - // compact does not notify (it reclaims without going through the - // listener), so listener state is unchanged afterwards. - expect(store.put("doomed", payload, EntryKind.Volatile, 0L), "put doomed") - val reclaimed = runBlocking { store.compact() } - expect(reclaimed == 3L, "compact reclaimed 3 bytes (got $reclaimed)") - expect(listener.evictions.size == 2, "compact bypasses the listener") - - // Replacing the listener releases the previous one (the producer's - // `free` drops the GlobalRef, so it becomes collectable), as does - // clearing it. - val first = attachThrowaway(store, "first") - System.gc() - expect(first.get() != null, "attached listener stays pinned by the producer") - val second = attachThrowaway(store, "second") - expect(collected(first), "replaced listener released") - expect(second.get() != null, "current listener still pinned") - store.clearEvictionListener() - expect(collected(second), "cleared listener released") - expect(store.delete("alpha"), "delete alpha with no listener") - expect(listener.evictions.size == 2, "detached listener sees nothing") - - // A listener returning false is detached by the store and released. - expect(collected(attachSelfDetaching(store, payload)), "self-detached listener released") - expect(store.count() == 0L, "store empty after self-detach test") - - // A Kotlin exception thrown from onEvict surfaces to the caller as - // FfiException(-4), not as a KvException, even though `delete` - // has the KvError domain; the JVM keeps running, the entry is gone - // (the producer removes before notifying), and the listener stays - // attached. - val thrower = RecordingListener("thrower", failOnKey = "boom") - store.setEvictionListener(thrower) - expect(store.put("boom", payload, EntryKind.Persistent, null), "put boom") - val foreign = thrownBy { store.delete("boom") } - expect(foreign is FfiException, "throwing onEvict surfaces as FfiException (got $foreign)") - expect(foreign !is KvException, "foreign error is not a domain error") - expect((foreign as FfiException).code == -4, "foreign error code -4 (got ${foreign.code})") - expect(foreign.message?.contains("refused boom") == true, "foreign message text (got ${foreign.message})") - expect(store.count() == 0L, "boom was removed before the listener ran") - expect(store.put("calm", payload, EntryKind.Persistent, null), "store usable after foreign error") - expect(store.delete("calm"), "delete calm") - expect(thrower.evictions == listOf("calm" to EvictionReason.Deleted), "listener still attached after throwing") - store.clearEvictionListener() - - // --- Object graph: share / fork / larger / describe / openMany ----- - - expect(store.put("one", payload, EntryKind.Persistent, null), "put one") - val shared = store.share() - expect(shared !== store, "share() returns a distinct wrapper") - expect(shared.handle.address == store.handle.address, "share() wraps the same native object") - expect(shared.count() == 1L, "shared sees existing entries") - expect(shared.put("two", payload, EntryKind.Persistent, null), "put through shared") - expect(store.count() == 2L, "write through shared is visible through the original") - expect(store.get("two")!!.key == "two", "entry written through shared readable through original") - shared.close() - shared.close() - expect(thrownBy { shared.count() } is IllegalStateException, "closed shared wrapper rejects use") - expect(store.count() == 2L, "original still alive after closing the shared wrapper") - - val forked = store.fork() - expect(forked.handle.address != store.handle.address, "fork() is a new object") - expect(forked.count() == 2L, "fork copies live entries") - expect(forked.put("three", payload, EntryKind.Persistent, null), "put into fork") - expect(forked.count() == 3L && store.count() == 2L, "fork is independent") - expect(forked.listKeys(null).asSequence().toList() == listOf("one", "three", "two"), "fork keys") - - // `Store?` both ways. - Store.open("/tmp/empty").use { emptyStore -> - expect(emptyStore.larger(null) == null, "empty.larger(null) is null") - val own = store.larger(null) - expect(own != null && own.handle.address == store.handle.address, "store.larger(null) is the store itself") - own!!.close() - val bigger = store.larger(forked) - expect(bigger != null && bigger.handle.address == forked.handle.address, "store.larger(fork) is the fork") - bigger!!.close() - val self = forked.larger(emptyStore) - expect(self != null && self.handle.address == forked.handle.address, "fork.larger(empty) is the fork") - self!!.close() - expect(store.larger(emptyStore)!!.use { it.handle.address } == store.handle.address, "store.larger(empty) is the store") +fun basics() = Store.open("/basics").use { s -> + // put returns the stored entry; the version counts puts of the key. + val e1 = s.putText("alpha", "one", EntryKind.Persistent) + expect(e1.key == "alpha" && text(e1.value) == "one" && e1.kind == EntryKind.Persistent, "put alpha") + expect(e1.version == 1u && e1.expires_at == null && e1.tags.isEmpty() && e1.metadata.isEmpty(), "put alpha fields") + val e2 = s.putText("alpha", "two", EntryKind.Volatile) + expect(e2.version == 2u && e2.kind == EntryKind.Volatile, "put alpha again") + + expect(text(s.get("alpha").value) == "two", "get alpha") + val missing = expectThrows("get(nope)") { s.get("nope") } + expect(missing.key == "nope" && missing.message == "key not found: nope", "KeyNotFound(nope)") + expect(s.find("alpha")?.version == 2u, "find alpha") + expect(s.find("nope") == null, "find nope") + + // TTLs follow the logical clock; an expired get reports when. + expect(s.putText("ttl", "x", EntryKind.Volatile, 10).expires_at == 10L, "ttl expires_at") + expect(s.now() == 0L, "clock starts at 0") + expect(s.tick(9) == 9L && s.count() == 2u, "tick 9") + expect(s.tick(1) == 10L && s.count() == 1u, "tick 10") + val expired = expectThrows("get(ttl)") { s.get("ttl") } + expect(expired.key == "ttl" && expired.expired_at == 10L, "Expired payload") + expectKeyNotFound("ttl") { s.get("ttl") } // the expired read removed it + + // Capacity: a new key past it is StoreFull { capacity }. + s.setCapacity(1u) + expect(s.capacity() == 1u, "setCapacity") + s.putText("alpha", "three") // replacing is fine + val full = expectThrows("put beta at capacity") { s.putText("beta", "b") } + expect(full.capacity == 1u, "StoreFull capacity") + s.setCapacity(100u) + + // delete, clear, and the deprecated size(). + s.putText("beta", "b") + expect(s.delete("beta") && !s.delete("beta"), "delete twice") + @Suppress("DEPRECATION") + val size = s.size() + expect(size == s.count() && size == 1u, "deprecated size() == count()") + expect(s.clear() == 1u && s.count() == 0u, "clear") +} + +fun iterators() = Store.open("/iter").use { s -> + s.putText("user.bob", "b") + s.putText("user.alice", "a") + s.putText("sys.x", "xx") + + expect(s.keys(null).asSequence().toList() == listOf("sys.x", "user.alice", "user.bob"), "keys in order") + expect(JniBridge.debug_live(2) == 0L, "an exhausted iterator is released") + expectKeyNotFound("zzz") { s.keys("zzz") } + + // Abandoning an iterator part-way releases it. + s.keys("user.").use { keys -> + expect(keys.next() == "user.alice", "first user key") + expect(JniBridge.debug_live(2) == 1L, "the iterator is live") + } + expect(JniBridge.debug_live(2) == 0L, "an abandoned iterator is released") + + val sys = s.entries("sys.").asSequence().toList() + expect(sys.size == 1 && sys[0].key == "sys.x" && text(sys[0].value) == "xx", "entries(sys.)") + + // partition: objects, created as they're pulled. + val prefixes = listOf("user.", "sys.", "none.") + var i = 0 + for (part in s.partition(prefixes)) { + part.use { + expect(it.address() != s.address(), "partition yields new stores") + expect(it.count() == listOf(2u, 1u, 0u)[i] && it.path() == prefixes[i], "partition $i") } + i++ + } + expect(i == 3, "three partitions") +} - // Objects inside a record, with the optional absent and present. - val info = store.describe("primary", null) - expect(info.label == "primary", "describe label") - expect(info.count == 2L, "describe count (got ${info.count})") - expect(info.mirror == null, "describe mirror absent") - expect(info.store.handle.address == store.handle.address, "describe().store is the described object") - expect(info.store.count() == 2L, "describe().store is usable") - val mirrored = store.describe("mirrored", forked) - val mirror = mirrored.mirror - expect(mirror != null && mirror.handle.address == forked.handle.address, "describe mirror present") - expect(mirror!!.count() == 3L, "mirror usable") - - // A list of objects as a return, and the typed error from the static. - val many = Store.openMany(listOf("/a", "/b", "/c")) - expect(many.size == 3, "openMany returns 3 stores (got ${many.size})") - expect(many.map { it.handle.address }.toSet().size == 3, "openMany stores are distinct") - expect(many[0].put("m", payload, EntryKind.Persistent, null), "put into openMany[0]") - expect(many[1].put("n", payload, EntryKind.Persistent, null), "put into openMany[1]") - expect(many.map { it.count() } == listOf(1L, 1L, 0L), "openMany counts") - val manyErr = thrownBy { Store.openMany(listOf("/ok", "")) } - expect(manyErr is KvException.IoError, "openMany with an empty path throws IoError (got $manyErr)") - - // A list of objects and an object inside a record as parameters. The - // encoder mints one reference per object, so the wrappers stay valid. - expect(Store.totalCount(listOf(), null) == 0L, "totalCount of nothing") - expect(Store.totalCount(many, null) == 2L, "totalCount over openMany (got ${Store.totalCount(many, null)})") - expect(Store.totalCount(listOf(store, forked), null) == 5L, "totalCount store + fork") - expect(Store.totalCount(listOf(store, forked), info) == 7L, "totalCount with extra record") - expect(Store.totalCount(listOf(forked, forked), mirrored) == 8L, "totalCount repeats and mirror record") - expect(store.count() == 2L && forked.count() == 3L, "stores still alive after being encoded") - expect(many.all { !it.handle.isClosed }, "openMany wrappers still alive after being encoded") - - // Release every wrapper we minted; the originals keep working. - info.store.close() - mirrored.store.close() - mirror.close() - many.forEach { it.close() } - expect(thrownBy { Store.totalCount(many, null) } is IllegalStateException, "encoding a closed store throws") - expect(store.count() == 2L, "store alive after releasing record and list references") - expect(forked.count() == 3L, "fork alive after releasing record references") - forked.close() - - // clear() drops everything that remains. - expect(store.put("last", payload, EntryKind.Persistent, null), "put last") - store.clear() - expect(store.count() == 0L, "store empty after clear") +fun listeners() { + val base = liveCallbacks() + val s = Store.open("/listen") + val l = RecordingListener(skip = "quiet") + val id = s.subscribe(l) + expect(id > 0u && s.listenerCount() == 1u && liveCallbacks() == base + 1, "subscribe") + + s.putText("a", "1") + var put = l.puts().last() + expect(l.puts().size == 1 && put.entry.version == 1u && !put.replaced, "Put v1") + s.putText("a", "2") + put = l.puts().last() + expect(l.puts().size == 2 && put.entry.version == 2u && put.replaced && put.entry.key == "a", "Put v2") + s.putText("quiet", "x") // accepts() said no + expect(l.puts().size == 2, "the filter skipped quiet") + expect(s.delete("a"), "delete a") + expect(l.changes.last() == Change.Removed("a", false), "Removed(a)") + + // An expired read removes the entry and says so. + s.putText("short", "x", EntryKind.Volatile, 1) + s.tick(1) + expectThrows("get(short)") { s.get("short") } + expect(l.changes.last() == Change.Removed("short", true), "Removed(short, expired)") + + expect(s.clear() == 1u, "clear leaves quiet's count") // "quiet" was left + expect(l.changes.last() == Change.Cleared(1u), "Cleared(1)") + expect(l.offMainThread.get() == 0, "synchronous calls notify on the calling thread") + + // Unsubscribing releases the listener once. + expect(s.unsubscribe(id) && liveCallbacks() == base, "unsubscribe releases the listener") + expect(!s.unsubscribe(id) && s.listenerCount() == 0u, "unsubscribe twice") + + // A listener that fails is detached (and released); the put succeeds. + val failing = RecordingListener(failOn = "boom") + s.subscribe(failing) + s.putText("fine", "1") + expect(failing.puts().size == 1, "the failing listener saw fine") + s.putText("boom", "1") + expect(s.count() == 2u && s.listenerCount() == 0u, "a failing listener is detached") + expect(liveCallbacks() == base, "the failing listener was released") + + // Closing the store releases the listeners it still holds. + s.subscribe(RecordingListener()) + s.subscribe(RecordingListener()) + expect(s.listenerCount() == 2u && liveCallbacks() == base + 2, "two listeners") + s.close() + expect(liveCallbacks() == base, "closing the store released its listeners") +} + +fun policies() { + val base = liveCallbacks() + val s = Store.open("/policy") + val other = Store.open("/other") + val p = TestPolicy(other.share()) + s.setPolicy(p) + expect(s.hasPolicy() && liveCallbacks() == base + 1, "setPolicy") + + // admit's record return is what's stored (its key and version aside). + val a = s.putText("a", "1", EntryKind.Volatile) + expect(a.key == "a" && a.version == 1u && a.kind == EntryKind.Encrypted, "admitted entry") + expect(a.tags == listOf("admitted"), "admit's rewrite") + + // route: the object parameter and object return redirect a write. + s.putText("b/x", "2", EntryKind.Volatile) + expect(s.count() == 1u && other.count() == 1u, "route redirected b/x") + + // A typed error from the throwing callback reaches the caller with its + // code, message, and payload. + val rejected = expectThrows("put(secret)") { s.putText("secret", "3") } + expect(rejected.code == 1005 && rejected.message == "secrets are not stored", "Rejected code and message") + expect(rejected.key == "secret" && rejected.reason == "no secrets", "Rejected payload") + + // Anything else arrives as -4 with the consumer's message. + val boom = expectThrows("put(boom)") { s.putText("boom", "4") } + expect(boom !is KvException && boom.code == -4 && boom.message == "policy exploded", "-4 (got ${boom.code}: ${boom.message})") + expect(s.count() == 1u && other.count() == 1u && p.admitted.get() == 4, "failed puts changed nothing") + + // Replacing the policy releases the old one; null removes it. + s.setPolicy(TestPolicy(other.share())) + expect(liveCallbacks() == base + 1, "replacing the policy released the old one") + s.setPolicy(null) + expect(!s.hasPolicy() && liveCallbacks() == base, "setPolicy(null) released it") + s.putText("secret", "now allowed") + expect(s.count() == 2u, "no policy, no veto") + + other.close() + s.close() +} + +fun loaders() = Store.open("/load").use { s -> + val base = liveCallbacks() + + // No loader (a null optional callback): a miss is none. + expect(s.getOrLoad("k", null) == null, "no loader") + + // load's bytes are stored, tagged with the loader's name. + val loaded = s.getOrLoad("k", TestLoader()) + expect(loaded != null && text(loaded.value) == "loaded:k", "loaded value") + expect(loaded!!.kind == EntryKind.Volatile && loaded.metadata == mapOf("source" to "kotlin-loader"), "loaded entry") + expect(liveCallbacks() == base, "a loader is released after the call") + expect(s.count() == 1u, "the loaded entry is stored") + // A hit doesn't consult the loader. + expect(s.getOrLoad("k", TestLoader())?.version == 1u, "a hit") + + // The fallback store (an optional object return) is consulted first. + Store.open("/backup").use { backup -> + backup.putText("fb", "from backup") + val fb = s.getOrLoad("fb", TestLoader(backup.share())) + expect(fb != null && text(fb.value) == "from backup" && fb.kind == EntryKind.Persistent, "fallback entry") } - // Close semantics: `use` closed it; a second close is safe and any use - // afterwards throws. - val closed = Store.open("/tmp/closed") - closed.close() - closed.close() - expect(thrownBy { closed.count() } is IllegalStateException, "use after close throws") - expect(thrownBy { closed.share() } is IllegalStateException, "share after close throws") - expect(thrownBy { Kv.StatsModule.getStats(closed) } is IllegalStateException, "borrowing a closed store throws") + // KeyNotFound for this key: the producer decoded the payload and + // answers none. + expect(s.getOrLoad("missing", TestLoader()) == null, "KeyNotFound for the same key is none") + // KeyNotFound for another key: passed through, payload intact. + val other = expectThrows("getOrLoad(elsewhere)") { s.getOrLoad("elsewhere", TestLoader()) } + expect(other.key == "other" && other.message == "not in the loader", "KeyNotFound(other) passed through") + // Any other failure is -4. + val broken = expectThrows("getOrLoad(broken)") { s.getOrLoad("broken", TestLoader()) } + expect(broken.code == -4 && broken.message == "loader is broken", "-4 (got ${broken.code}: ${broken.message})") + expect(liveCallbacks() == base, "every loader was released") +} - println("kotlin/kvstore: OK") +fun asyncCalls() = runBlocking { + val s = Store.open("/async-calls") + val l = RecordingListener() + s.subscribe(l) + s.putText("old1", "x", EntryKind.Volatile, 1) + s.putText("old2", "x", EntryKind.Volatile, 1) + s.putText("keep", "x") + s.tick(5) + + // compact runs on a producer thread and notifies listeners there. + expect(s.compact(0u) == 2u, "compact(0) removed 2") + expect(l.removed(expired = true).size == 2, "the listener saw both expirations") + expect(l.offMainThread.get() == 2, "notified from a producer thread") + expect(s.count() == 1u, "one entry left") + expect(s.compact(5u) == 0u, "compact(5) removed nothing") + + // Cancel mid-pause: the call completes at once with a cancellation, and + // the background pause notices the token and stops. + val job = async { s.compact(60_000u) } + delay(20) + expect(!job.isCompleted, "compact(60000) is pausing") + expect(Store.activeJobs() >= 1u, "the pause is running") + val started = System.nanoTime() + job.cancel() + val cancelled = thrownBy { job.await() } + expect(cancelled is CancellationException, "a cancelled compact raises CancellationException (got $cancelled)") + expect(System.nanoTime() - started < 2_000_000_000L, "cancellation is prompt") + var stopped = false + for (i in 0 until 2000) { + if (Store.activeJobs() == 0u) { + stopped = true + break + } + delay(1) + } + expect(stopped, "the cancelled pause stopped cooperatively") + + // getMany: an async list of optional records, launched concurrently. + val results = (0 until 32).map { + async(Dispatchers.Default) { s.getMany(listOf("keep", "gone", "keep")) } + }.awaitAll() + for (got in results) { + expect(got.size == 3 && got[0]?.key == "keep" && got[1] == null && got[2]?.key == "keep", "getMany") + } + + // The nested module's async function: objects in a list in, a record out. + Store.open("/other").use { other -> + other.putText("a", "123") + val st = Kv.StatsModule.summarizeAll(listOf(s, other)) + expect(st == Stats(2u, 4uL, mapOf(EntryKind.Persistent to 2u)), "summarizeAll (got $st)") + } + s.close() +} + +fun objectGraph() { + val s0 = Store.open("/graph") + s0.putText("k", "v") + + // share(): the same object; the original can go. + val s = s0.share() + expect(s.address() == s0.address(), "share() is the same object") + s0.close() + expect(s.count() == 1u, "alive through the shared reference") + + // fork(): a distinct object with a copy of the entries. + val fork = s.fork() + expect(fork.address() != s.address() && fork.count() == 1u && fork.path() == "/graph", "fork") + fork.putText("k2", "v") + expect(fork.count() == 2u && s.count() == 1u, "fork is independent") + + // larger(): `Store?` in and out. + val empty = Store.open("/empty") + expect(empty.larger(null) == null, "larger(null) on an empty store") + empty.larger(fork)!!.use { expect(it.address() == fork.address(), "larger(fork) is fork") } + s.larger(null)!!.use { expect(it.address() == s.address(), "larger(null) is self") } + + // describe(): a record whose fields carry objects. + val info = s.describe("main", fork) + expect(info.label == "main" && info.count == 1u, "describe label and count") + expect(info.store.address() == s.address() && info.mirror?.address() == fork.address(), "describe objects") + expect(info.mirror?.count() == 2u, "the mirror is live") + + // openMany(): a list of objects; one bad path fails the whole call. + val many = Store.openMany(listOf("/a", "/b")) + expect(many.map { it.path() } == listOf("/a", "/b"), "openMany paths") + expectThrows("openMany with an empty path") { Store.openMany(listOf("/a", "")) } + + // byLabel(): records with objects in, a map with object values out. + val named = Store.byLabel(listOf(info, StoreInfo("first", many[0], null, 0u))) + expect(named.keys == setOf("main", "first"), "byLabel keys") + expect(named.getValue("main").address() == s.address(), "byLabel main") + expect(named.getValue("first").address() == many[0].address(), "byLabel first") + + // totalCount(): a list, a map, and an optional record, all carrying + // objects (each written as a fresh reference the producer adopts). + many[0].putText("m", "1") + val stores = listOf(many[0], many[1], fork) + expect(Store.totalCount(stores, named, info) == 6u, "totalCount with extra") + expect(Store.totalCount(stores, named, null) == 5u, "totalCount without extra") + + // Everything is still intact; release each reference once. + expect(s.count() == 1u && fork.count() == 2u && many[0].count() == 1u, "still usable") + named.values.forEach { it.close() } + info.store.close() + info.mirror?.close() + many.forEach { it.close() } + empty.close() + fork.close() + s.close() +} + +fun statsAndReport() = Store.open("/stats").use { s -> + s.putText("b", "12") + s.putText("a", "1") + s.putText("a", "123") + s.putText("c", "x", EntryKind.Encrypted) + + // kv.stats: the parent's Store as a parameter, the parent's error domain. + val st = Kv.StatsModule.summarize(s, null) + expect(st == Stats(3u, 6uL, mapOf(EntryKind.Persistent to 2u, EntryKind.Encrypted to 1u)), "summarize (got $st)") + expectKeyNotFound("q") { Kv.StatsModule.summarize(s, "q") } + + // report: the sibling root shares the Entry record. + val lines = Report.renderReport(s.entries(null).asSequence().toList()) + expect( + lines == listOf("a: 3 bytes, Persistent, v2", "b: 2 bytes, Persistent", "c: 1 bytes, Encrypted"), + "renderReport (got $lines)", + ) + val nothing = expectThrows("renderReport([])") { Report.renderReport(emptyList()) } + expect(nothing.code == 2001 && nothing.message == "nothing to report", "NothingToReport") } fun main() { - run() - expectNoLeaks { JniBridge.debug_live(it) } - println("kotlin/kvstore: no leaks") + expect(JniBridge.debug_live(-1) == 1L, "the sample counts live allocations") + + constructors() + basics() + iterators() + listeners() + policies() + loaders() + asyncCalls() + objectGraph() + statsAndReport() + + expectNoLeaks(JniBridge::debug_live) + println("kotlin/kvstore: OK") } diff --git a/conformance/kotlin/run.sh b/conformance/kotlin/run.sh index 666aa74d..a3a4dfa7 100755 --- a/conformance/kotlin/run.sh +++ b/conformance/kotlin/run.sh @@ -5,6 +5,7 @@ # consumer against one sample and must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools kotlin kotlinc java cmake # Kotlin (JVM): build the generated JNI shim with its own CMakeLists.txt # against the producer in $LIBDIR, compile the generated Kotlin sources on @@ -44,20 +45,12 @@ kotlin_consumer() { java -Djava.library.path="$b/shim" -cp "$b/app.jar:$b/bindings.jar:$coro:$kstdlib" Main } -kotlin_contacts() { kotlin_consumer contacts contacts.kt; } +kotlin_calculator() { kotlin_consumer calculator calculator.kt; } +kotlin_codec() { kotlin_consumer codec codec.kt; } +kotlin_kvstore() { kotlin_consumer kvstore kvstore.kt; } -kotlin_events() { kotlin_consumer events events.kt; } - -kotlin_kvstore() { kotlin_consumer kvstore kvstore.kt; } - -kotlin_async_demo() { kotlin_consumer async-demo async_demo.kt; } - -kotlin_codec() { kotlin_consumer codec codec.kt; } - -lane kotlin-contacts kotlin_contacts -lane kotlin-events kotlin_events -lane kotlin-kvstore kotlin_kvstore -lane kotlin-async-demo kotlin_async_demo +lane kotlin-calculator kotlin_calculator lane kotlin-codec kotlin_codec +lane kotlin-kvstore kotlin_kvstore finish_lanes diff --git a/conformance/lib.sh b/conformance/lib.sh index d02712e0..8d1a1f27 100644 --- a/conformance/lib.sh +++ b/conformance/lib.sh @@ -34,6 +34,27 @@ export ROOT TARGET_DIR LIBDIR GENROOT OUT RESULTS LANE_TIMEOUT EXT LANES_FAILED=0 +# require_tools ...: check a language's toolchain before its +# lanes run. Under CI (CI=true), where the toolchains action installs +# everything, a missing tool fails the language; elsewhere its lanes are +# skipped with a note. Either way the outcome lands in $RESULTS. +require_tools() { + local lang=$1 tool missing="" + shift + for tool in "$@"; do + command -v "$tool" >/dev/null 2>&1 || missing="$missing $tool" + done + [ -z "$missing" ] && return 0 + if [ "${CI:-}" = true ]; then + echo "[FAIL] $lang: missing$missing" >&2 + echo "FAIL $lang-toolchain" >> "$RESULTS" + exit 1 + fi + echo "[SKIP] $lang: missing$missing" + echo "SKIP $lang" >> "$RESULTS" + exit 0 +} + # Is lane `$1` selected by ONLY / SKIP (comma-separated lane names or # language names)? selected() { @@ -79,7 +100,7 @@ finish_lanes() { } # The platform file name of a sample's cdylib. Cargo maps `-` in crate names -# to `_` in artifact names (async-demo -> libasync_demo). +# to `_` in artifact names (my-sample -> libmy_sample). sample_lib() { local n=${1//-/_} case "$EXT" in diff --git a/conformance/node/async_demo.mjs b/conformance/node/async_demo.mjs deleted file mode 100644 index 718b9235..00000000 --- a/conformance/node/async_demo.mjs +++ /dev/null @@ -1,62 +0,0 @@ -// Conformance consumer: async-demo sample (node and wasm lanes). -// -// Promise-returning async functions: a record result, a typed rejection, a -// list of records, a direct result, and the cancellable `wait`, which only -// completes early when cancelled: aborting its `AbortSignal` cancels the -// native token and the promise rejects with `CancelledError` (code -5). On -// WebAssembly every async call completes before its launcher returns, so -// only an already-aborted signal can cancel one there. - -import { expect, finish, isWasm, load, rejects, same } from './harness.mjs'; - -const api = await load('async-demo'); -const { tasks, AsyncDemoError, CancelledError } = api; - -async function main() { - same(await tasks.runTask('alpha'), { id: 1n, value: 'completed: alpha', success: true }, 'runTask'); - await rejects( - tasks.runTask(''), - (e) => e instanceof tasks.InvalidNameError && e instanceof tasks.TaskError && e instanceof AsyncDemoError && e.code === 1, - 'a typed async rejection', - ); - same( - (await tasks.runBatch(['a', 'b', 'c'])).map((r) => r.value), - ['completed: a', 'completed: b', 'completed: c'], - 'runBatch', - ); - expect((await tasks.runNTasks(7)) === 7, 'runNTasks'); - const many = await Promise.all(Array.from({ length: 50 }, (_, i) => tasks.runNTasks(i))); - same(many, Array.from({ length: 50 }, (_, i) => i), 'concurrent calls settle independently'); - await rejects(tasks.runNTasks('7'), (e) => e instanceof TypeError, 'async arguments are type-checked'); - - // Cancellation. The sample's timer reads the clock, which traps on - // wasm32-unknown-unknown, so there only a call cancelled before it starts - // can run. - if (!isWasm(api)) { - expect((await tasks.wait(5n)) === 5n, 'wait completes after its timeout'); - expect((await tasks.wait(5n, {})) === 5n, 'wait without a signal'); - } - const aborted = new AbortController(); - aborted.abort(); - await rejects( - tasks.wait(60_000n, { signal: aborted.signal }), - (e) => e instanceof CancelledError && e instanceof AsyncDemoError && e.code === -5, - 'an already-aborted signal cancels the call', - ); - if (!isWasm(api)) { - const controller = new AbortController(); - const started = Date.now(); - const pending = tasks.wait(60_000n, { signal: controller.signal }); - setTimeout(() => controller.abort(), 20); - await rejects(pending, (e) => e instanceof CancelledError && e.code === -5, 'aborting mid-call cancels it'); - expect(Date.now() - started < 10_000, 'cancellation completes promptly'); - // A signal aborted after the call completed is harmless. - const late = new AbortController(); - expect((await tasks.wait(1n, { signal: late.signal })) === 1n, 'completes before the abort'); - late.abort(); - } - expect(tasks.activeCallbacks() === 0n, 'every task body has finished'); -} - -await main(); -await finish(api, 'async-demo'); diff --git a/conformance/node/calculator.mjs b/conformance/node/calculator.mjs new file mode 100644 index 00000000..2b6d9197 --- /dev/null +++ b/conformance/node/calculator.mjs @@ -0,0 +1,43 @@ +// Conformance consumer: calculator sample (node and wasm lanes). +// +// The minimal sample: the load-time checks, i32 arithmetic that wraps, a +// throwing function whose typed error has no payload, and strings that keep +// non-ASCII text, astral characters, and interior NULs. Ends with every leak +// counter at zero. + +import { expect, finish, load, throws } from './harness.mjs'; + +const api = await load('calculator'); +const { calculator, CalculatorError } = api; + +const I32_MAX = 2147483647; +const I32_MIN = -2147483648; + +expect(api.__debugLive(-1) === 1n, 'the sample counts live resources'); + +expect(calculator.add(2, 3) === 5, 'add'); +expect(calculator.add(I32_MAX, 1) === I32_MIN, 'add wraps'); + +expect(calculator.divide(10, 2) === 5, 'divide'); +expect(calculator.divide(-7, 2) === -3, 'divide rounds toward zero'); +expect(calculator.divide(I32_MIN, -1) === I32_MIN, 'divide wraps'); +throws( + () => calculator.divide(1, 0), + (e) => + e instanceof calculator.DivisionByZeroError && + e instanceof calculator.CalcError && + e instanceof CalculatorError && + e.code === 1 && + calculator.DivisionByZeroError.CODE === 1 && + e.message === 'division by zero' && + Object.keys(e).every((k) => k === 'name' || k === 'code'), + 'divide by zero is the typed error, with no payload fields', +); + +expect(calculator.greet('World') === 'Hello, World!', 'greet'); +expect(calculator.greet('') === 'Hello, !', 'greet an empty name'); +expect(calculator.greet('Wörld 🦀') === 'Hello, Wörld 🦀!', 'non-ASCII and astral text'); +expect(calculator.greet('a\0b') === 'Hello, a\0b!', 'an interior NUL'); +throws(() => calculator.greet(42), (e) => e instanceof TypeError, 'a non-string is a TypeError'); + +await finish(api, 'calculator'); diff --git a/conformance/node/codec.mjs b/conformance/node/codec.mjs index b5f13e86..d06c65a4 100644 --- a/conformance/node/codec.mjs +++ b/conformance/node/codec.mjs @@ -1,29 +1,90 @@ -// Conformance consumer: codec sample (node and wasm lanes). +// Conformance consumer: codec sample, the wire oracle (node and wasm lanes). // -// The value-buffer codec against the producer's oracle in both directions: -// `sample*` fixtures decoded field by field, `verify*` accepting them back, -// and `roundtrip*` returning consumer-built edge cases (empty and unicode -// strings, strings with an interior NUL, 64-bit extremes, NaN, the -// infinities and -0, every Shape variant), the typed MismatchError, range -// and type checks, and objects inside buffers through `Holder`. +// The shared-vector loop: for every vector the producer serves, decode it, +// hand it back to `checkVector` (which re-encodes it), and push each +// primitive vector's value through the matching direct-family `echo*`. Then +// vectors built from literals (so a symmetric encode/decode bug can't hide), +// spot checks of decoded fields (64-bit integers exact as bigints, float +// bits, astral text), the typed out-of-range error and its payload, the +// marshalling failures the API can express, and object identity and +// reference counting through buffers. Ends with every leak counter at zero. import { expect, finish, load, same, throws } from './harness.mjs'; const api = await load('codec'); const { codec, CodecError } = api; -const { Color } = codec; +const { Color, Token } = codec; -const bytes = (...xs) => new Uint8Array(xs); +const I64_MIN = -(2n ** 63n); +const I64_MAX = 2n ** 63n - 1n; +const U64_MAX = 2n ** 64n - 1n; -function mismatch(fn, msg) { - throws( - fn, - (e) => e instanceof codec.MismatchError && e instanceof codec.CodecError && e instanceof CodecError && e.code === 1, - msg, - ); +expect(api.__debugLive(-1) === 1n, 'the sample counts live resources'); + +const n = codec.vectorCount(); +expect(n >= 60, `at least 60 vectors (got ${n})`); +const names = Array.from({ length: n }, (_, i) => codec.vectorName(i)); + +/** The index of the vector named `name`. */ +function find(name) { + const i = names.indexOf(name); + if (i < 0) throw new Error(`no vector named ${name}`); + return i; +} + +/** Close every object wrapper inside a decoded value. */ +function release(v) { + if (v instanceof Token) { + v.close(); + } else if (Array.isArray(v)) { + v.forEach(release); + } else if (v !== null && typeof v === 'object' && !(v instanceof Uint8Array)) { + Object.values(v).forEach(release); + } +} + +const f32Bits = (x) => { + const view = new DataView(new ArrayBuffer(4)); + view.setFloat32(0, x, true); + return view.getUint32(0, true); +}; + +// The direct-family echo of each primitive vector, and how to compare. +const echoes = { + I8: codec.echoI8, + U8: codec.echoU8, + I16: codec.echoI16, + U16: codec.echoU16, + I32: codec.echoI32, + U32: codec.echoU32, + I64: codec.echoI64, + U64: codec.echoU64, + F32: codec.echoF32, + F64: codec.echoF64, + Flag: codec.echoBool, + Text: codec.echoText, + Blob: codec.echoBlob, + Hue: codec.echoColor, +}; + +// 1. The loop. +for (let i = 0; i < n; i++) { + const v = codec.vector(i); + if (!codec.checkVector(i, v)) { + expect(false, `vector ${i} (${names[i]}) did not round-trip; producer saw ${codec.describeVector(v)}`); + } + expect(!codec.checkVector((i + 1) % n, v), `vector ${i} (${names[i]}) never matches its neighbor`); + const echo = echoes[v.tag]; + if (echo !== undefined) { + // Object.is: NaN is NaN, and the sign of zero counts. + same(echo(v.value), v.value, `echo of vector ${i} (${names[i]})`); + if (v.tag === 'F32') expect(f32Bits(echo(v.value)) === f32Bits(v.value), `f32 bits of ${names[i]}`); + } + release(v); } -const canonicalScalars = { +// 2. Literal vectors. +const canonical = { i8_value: -8, u8_value: 200, i16_value: -16000, @@ -31,197 +92,153 @@ const canonicalScalars = { i32_value: -2000000000, u32_value: 4000000000, i64_value: -9007199254740993n, - u64_value: 18446744073709551615n, + u64_value: U64_MAX, f32_value: 1.5, f64_value: -2.25e100, flag: true, color: Color.Blue, }; -const extremes = { - i8_value: -128, - u8_value: 255, - i16_value: -32768, - u16_value: 65535, - i32_value: -2147483648, - u32_value: 4294967295, - i64_value: -9223372036854775808n, - u64_value: 0n, - f32_value: -0, - f64_value: NaN, - flag: false, - color: Color.Red, -}; -const maxes = { - ...extremes, - i8_value: 127, - i16_value: 32767, - i32_value: 2147483647, - i64_value: 9223372036854775807n, - u64_value: 18446744073709551615n, - f32_value: Infinity, - f64_value: -Infinity, - color: Color.Green, -}; +expect(codec.checkVector(find('scalars canonical'), { tag: 'AllScalars', value: canonical }), 'scalars canonical'); +expect( + !codec.checkVector(find('scalars canonical'), { tag: 'AllScalars', value: { ...canonical, u16_value: 60001 } }), + 'scalars canonical with one field changed', +); +expect( + codec.checkVector(find('shape labeled'), { tag: 'Figure', value: { tag: 'Labeled', label: 'tag', count: 3 } }), + 'shape labeled', +); +expect(codec.checkVector(find('string interior nul'), { tag: 'Text', value: 'nul\0inside\0' }), 'string interior nul'); +const nanView = new DataView(new ArrayBuffer(8)); +nanView.setBigUint64(0, 0x7ff8000000000001n, true); +expect(codec.checkVector(find('f64 nan'), { tag: 'F64', value: nanView.getFloat64(0, true) }), 'any NaN is the NaN vector'); +expect(codec.checkVector(find('f64 -0'), { tag: 'F64', value: -0 }), 'negative zero'); +expect(!codec.checkVector(find('f64 -0'), { tag: 'F64', value: 0 }), 'positive zero is not negative zero'); +expect(codec.checkVector(find('u64 max'), { tag: 'U64', value: U64_MAX }), 'u64 max'); +expect(codec.checkVector(find('enum infrared'), { tag: 'Hue', value: Color.Infrared }), 'enum infrared'); +expect(Color.Infrared === -1 && Color[-1] === 'Infrared', 'a negative enum value maps both ways'); +expect(codec.checkVector(find('optional zero'), { tag: 'MaybeI64', value: 0n }), 'optional zero'); +expect(codec.checkVector(find('optional absent'), { tag: 'MaybeI64', value: null }), 'optional absent'); +expect(!codec.checkVector(find('optional zero'), { tag: 'MaybeI64', value: null }), 'absent is not zero'); +expect( + codec.checkVector(find('map of strings'), { tag: 'Counts', value: { x: 0n, 'héllo': -1n, '': I64_MAX } }), + 'a map built in another order', +); +expect( + codec.checkVector( + find('map of strings'), + { tag: 'Counts', value: new Map([['héllo', -1n], ['', I64_MAX], ['x', 0n]]) }, + ), + 'a Map is a map too', +); +expect(codec.checkVector(find('blank'), { tag: 'Blank' }), 'blank'); +const lone = new Token(-1n); +expect( + codec.checkVector(find('objects sparse'), { tag: 'Objects', value: { primary: lone, spare: null, many: [], by_name: {} } }), + 'objects sparse from a consumer-made token', +); +lone.close(); -function scalars() { - expect(Color.Red === 0 && Color.Blue === 7 && Color[7] === 'Blue', 'Color values'); - const s = codec.sampleScalars(); - same(s, canonicalScalars, 'sampleScalars'); - expect(codec.verifyScalars(s) && codec.verifyScalars(canonicalScalars), 'verifyScalars'); - same(codec.roundtripScalars(canonicalScalars), canonicalScalars, 'roundtripScalars'); - mismatch(() => codec.verifyScalars({ ...canonicalScalars, u64_value: 18446744073709551614n }), 'u64 off by one'); - mismatch(() => codec.verifyScalars({ ...canonicalScalars, flag: false }), 'flag flipped'); - const back = codec.roundtripScalars(extremes); - same(back, extremes, 'extreme scalars'); - expect(Object.is(back.f32_value, -0) && Number.isNaN(back.f64_value), '-0 and NaN survive'); - same(codec.roundtripScalars(maxes), maxes, 'max scalars'); - same( - codec.roundtripScalars({ ...extremes, i64_value: 42, u64_value: 7 }), - { ...extremes, i64_value: 42n, u64_value: 7n }, - 'integral numbers widen to bigint', - ); - throws(() => codec.roundtripScalars({ ...extremes, u64_value: -1n }), (e) => e instanceof RangeError, 'an out-of-range bigint field'); - throws(() => codec.roundtripScalars({ ...extremes, i32_value: '1' }), (e) => e instanceof TypeError, 'a string for a number field'); - throws(() => codec.roundtripScalars({ ...extremes, flag: 1 }), (e) => e instanceof TypeError, 'a number for a bool field'); - - for (const v of [0n, -1n, 9007199254740993n, 9223372036854775807n, -9223372036854775808n]) { - expect(codec.roundtripI64(v) === v, `roundtripI64(${v})`); - } - expect(codec.roundtripI64(5) === 5n, 'roundtripI64 accepts an integral number'); - for (const v of [0n, 9223372036854775808n, 18446744073709551615n]) { - expect(codec.roundtripU64(v) === v, `roundtripU64(${v})`); - } - for (const bad of [9223372036854775808n, -9223372036854775809n]) { - throws(() => codec.roundtripI64(bad), (e) => e instanceof RangeError, `roundtripI64(${bad})`); - } - throws(() => codec.roundtripU64(-1n), (e) => e instanceof RangeError, 'roundtripU64(-1n)'); - throws(() => codec.roundtripI64(1.5), (e) => e instanceof TypeError, 'a fraction for an i64'); - for (const v of [0, -0, 1.5, 5e-324, Number.MAX_VALUE, Infinity, -Infinity]) { - expect(Object.is(codec.roundtripF64(v), v), `roundtripF64(${v})`); - } - expect(Number.isNaN(codec.roundtripF64(NaN)), 'roundtripF64(NaN)'); - expect(codec.roundtripBool(true) === true && codec.roundtripBool(false) === false, 'roundtripBool'); - throws(() => codec.roundtripBool(1), (e) => e instanceof TypeError, 'a number for a bool'); - expect(codec.roundtripColor(Color.Blue) === 7, 'roundtripColor'); -} +// 3. Spot checks on decoded values. +same(codec.vector(find('i64 past 2^53')), { tag: 'I64', value: -9007199254740993n }, 'i64 past 2^53 is exact'); +const subnormal = codec.vector(find('f32 min subnormal')); +expect(subnormal.tag === 'F32' && f32Bits(subnormal.value) === 1, 'f32 min subnormal bits'); +same(codec.vector(find('string astral')), { tag: 'Text', value: '🦀 crab 😀' }, 'string astral'); +const minimum = codec.vector(find('scalars minimum')).value; +expect( + minimum.i8_value === -128 && + minimum.i16_value === -32768 && + minimum.i32_value === -2147483648 && + minimum.i64_value === I64_MIN && + minimum.u64_value === 0n, + 'scalars minimum integers', +); +expect(minimum.f32_value === -Infinity && Number.isNaN(minimum.f64_value), 'scalars minimum floats'); +expect(minimum.color === Color.Infrared && minimum.flag === false, 'scalars minimum enum and flag'); +const deep = codec.vector(find('composite canonical')); +expect(deep.tag === 'Deep', 'composite canonical is Deep'); +const c = deep.value; +expect(c.name === 'héllo wörld ✓', 'composite name'); +expect(c.blob instanceof Uint8Array && c.blob.length === 6 && c.blob[5] === 255, 'composite blob'); +expect(c.some_i64 === I64_MIN && c.none_i64 === null && c.some_text === '', 'composite optionals'); +expect(c.names.length === 3 && c.names[1] === '', 'composite names'); +expect(c.matrix.length === 3 && c.matrix[1].length === 0 && c.matrix[2][0] === -4, 'composite matrix'); +expect( + c.floats.length === 6 && Number.isNaN(c.floats[0]) && (Object.is(c.floats[3], -0) || c.floats[3] < 0), + 'composite floats', +); +expect( + Object.keys(c.by_name).length === 4 && + Object.keys(c.by_id).length === 3 && + Object.keys(c.by_color).length === 2 && + Object.keys(c.flags).length === 2, + 'composite maps', +); +expect(c.scalars.u32_value === 4000000000, 'composite scalars'); +expect(c.shape.tag === 'Labeled' && c.shape.count === 3, 'composite shape'); +expect(c.shapes.length === 6 && c.shapes[5].tag === 'Nested' && c.shapes[5].note === null, 'composite shapes'); +expect(c.maybe_shape !== null && c.maybe_shape.tag === 'Nested', 'composite maybe_shape'); +expect(c.maybe_list instanceof Uint8Array && c.maybe_list.length === 2, 'composite maybe_list'); +expect(c.sparse.length === 3 && c.sparse[0] === true && c.sparse[1] === null, 'composite sparse'); +expect(c.colors.length === 4 && c.colors[3] === Color.Infrared, 'composite colors'); -function strings() { - for (const s of ['', 'ascii', 'héllo wörld ✓', '日本語', 'emoji 🎉 pair', 'nul\0inside\0', '\0', 'a'.repeat(70000)]) { - expect(codec.roundtripString(s) === s, `roundtripString(${JSON.stringify(s.slice(0, 20))})`); - } - throws(() => codec.roundtripString(5), (e) => e instanceof TypeError, 'a number for a string'); - const all = new Uint8Array(256).map((_, i) => i); - same(codec.roundtripBytes(all), all, 'roundtripBytes(0..255)'); - const empty = codec.roundtripBytes(new Uint8Array(0)); - expect(empty instanceof Uint8Array && empty.length === 0, 'empty bytes'); - same(codec.roundtripBytes(all.subarray(10, 20)), all.slice(10, 20), 'a Uint8Array view'); - throws(() => codec.roundtripBytes([1, 2]), (e) => e instanceof TypeError, 'an array for bytes'); - expect(codec.roundtripOptI64(null) === null && codec.roundtripOptI64(undefined) === null, 'absent optional'); - expect(codec.roundtripOptI64(0n) === 0n, 'present zero'); - same(codec.roundtripMap({}), {}, 'empty map'); - same(codec.roundtripMap({ a: 1n, '': -2n, 'ключ': 9223372036854775807n }), { a: 1n, '': -2n, 'ключ': 9223372036854775807n }, 'odd keys'); - same(codec.roundtripMap(new Map([['k', 3n]])), { k: 3n }, 'a Map argument'); -} - -function shapes() { - const cases = [ - { tag: 'Empty' }, - { tag: 'Circle', radius: 2.5 }, - { tag: 'Circle', radius: -0 }, - { tag: 'Rect', width: 1, height: 0.5 }, - { tag: 'Labeled', label: 'tag', count: 3 }, - { tag: 'Labeled', label: '', count: -2147483648 }, - { tag: 'Nested', inner: canonicalScalars, note: 'n' }, - { tag: 'Nested', inner: extremes, note: null }, - ]; - for (const s of cases) same(codec.roundtripShape(s), s, `roundtripShape(${s.tag})`); - same(codec.roundtripShapes(cases), cases, 'roundtripShapes'); - expect(codec.describeShape({ tag: 'Circle', radius: 2.5 }) === 'Circle { radius: 2.5 }', 'describeShape'); - throws(() => codec.roundtripShape({ tag: 'Hexagon' }), (e) => e instanceof TypeError, 'an unknown tag'); -} +// 4. The typed error with its payload. +throws( + () => codec.vector(n), + (e) => + e instanceof codec.OutOfRangeError && + e instanceof codec.CodecError && + e instanceof CodecError && + e.code === 1 && + e.index === n && + e.count === n && + e.message === `vector ${n} is out of range (count ${n})`, + 'vector past the end', +); +throws( + () => codec.vectorName(n + 5), + (e) => e instanceof codec.OutOfRangeError && e.index === n + 5 && e.count === n, + 'vectorName past the end', +); +expect(!codec.checkVector(n, { tag: 'Blank' }), 'checkVector past the end is false'); -function composites() { - const canonical = { - name: 'héllo wörld ✓', - blob: bytes(0, 1, 2, 253, 254, 255), - some_i64: -9223372036854775808n, - none_i64: null, - some_text: '', - names: ['a', '', 'ccc'], - matrix: [[1, 2, 3], [], [-4]], - empty: [], - by_name: { one: 1n, two: 2n, neg: -3n }, - by_id: { '-1': canonicalScalars, 42: { ...canonicalScalars, flag: false } }, - scalars: canonicalScalars, - shape: { tag: 'Labeled', label: 'tag', count: 3 }, - shapes: [ - { tag: 'Empty' }, - { tag: 'Circle', radius: 2.5 }, - { tag: 'Rect', width: 1, height: 0.5 }, - { tag: 'Labeled', label: '', count: -1 }, - { tag: 'Nested', inner: canonicalScalars, note: 'n' }, - ], - maybe_shape: { tag: 'Nested', inner: canonicalScalars, note: null }, - maybe_list: bytes(9, 8), - sparse: [true, null, false], - colors: [Color.Red, Color.Green, Color.Blue], - }; - const c = codec.sampleComposite(); - same(c, canonical, 'sampleComposite'); - expect(codec.verifyComposite(c) && codec.verifyComposite(canonical), 'verifyComposite'); - same(codec.roundtripComposite(c), canonical, 'roundtripComposite'); - expect(codec.describeComposite(c).startsWith('Composite {'), 'describeComposite'); - mismatch(() => codec.verifyComposite({ ...canonical, sparse: [true, true, false] }), 'sparse changed'); - mismatch(() => codec.verifyComposite({ ...canonical, by_name: { one: 1n, two: 2n } }), 'map entry missing'); - const edge = { - name: '', - blob: new Uint8Array(0), - some_i64: 9223372036854775807n, - none_i64: -1n, - some_text: null, - names: [], - matrix: [[], [-2147483648, 2147483647]], - empty: [NaN, -0, Infinity, -Infinity, 5e-324], - by_name: {}, - by_id: { '-2147483648': extremes, 0: maxes }, - scalars: extremes, - shape: { tag: 'Empty' }, - shapes: [], - maybe_shape: null, - maybe_list: null, - sparse: [null, null], - colors: [], - }; - same(codec.roundtripComposite(edge), edge, 'edge composite'); - const big = { ...edge, names: Array.from({ length: 1000 }, (_, i) => 'name-' + i), blob: new Uint8Array(70000).fill(7) }; - same(codec.roundtripComposite(big), big, 'a large composite'); -} +// 5. Malformed input: an undeclared enum value reaches the producer (and +// fails the non-throwing call as a marshalling trap); the encoder refuses +// what the wire format can't carry. +throws( + () => codec.echoColor(3), + (e) => e instanceof CodecError && !(e instanceof codec.CodecError) && e.code === -3, + 'an undeclared enum value', +); +throws(() => codec.checkVector(0, { tag: 'Flag', value: 2 }), (e) => e instanceof TypeError, 'a non-boolean bool'); +throws(() => codec.checkVector(0, { tag: 'Nope' }), (e) => e instanceof TypeError, 'an unknown tag'); +throws(() => codec.checkVector(0, { tag: 'I64', value: 2n ** 64n }), (e) => e instanceof RangeError, 'an i64 out of range'); +throws(() => codec.echoText(null), (e) => e instanceof TypeError, 'a null string'); -function holders() { - const holder = codec.makeHolder(10n, true); - expect(holder.primary instanceof codec.Token && holder.primary.value() === 10n, 'an object field'); - expect(holder.spare.value() === 11n && holder.many.length === 3, 'optional and list object fields'); - expect(codec.sumHolder(holder) === 60n && codec.sumHolder(holder) === 60n, 'encoding clones each object'); - const primary = codec.primaryOf(holder); - expect(primary !== holder.primary && primary.value() === 10n, 'a new wrapper over the same object'); - expect(codec.samePrimary(holder, { primary, spare: null, many: [] }), 'samePrimary'); - const other = codec.makeHolder(10n, false); - expect(!codec.samePrimary(holder, other) && other.spare === null, 'distinct objects'); - const t1 = new codec.Token(100n); - const t2 = new codec.Token(-9223372036854775808n); - expect(codec.sumHolder({ primary: t1, spare: t2, many: [t1, t1] }) === 300n - 9223372036854775808n, 'consumer-built objects'); - for (const t of [holder.primary, holder.spare, ...holder.many, primary, other.primary, ...other.many, t1, t2]) { - t.close(); - } - throws(() => holder.primary.value(), (e) => e instanceof CodecError && e.code === -3, 'use after close'); - throws(() => codec.sumHolder(holder), (e) => e instanceof CodecError && e.code === -3, 'a closed object in a buffer'); - // A holder dropped without closing its objects is released by the GC. - codec.makeHolder(1n, true); -} +// 6. Objects. +const full = codec.vector(find('objects full')); +expect(full.tag === 'Objects', 'objects full is Objects'); +const h = full.value; +expect(h.primary instanceof Token && h.primary.value() === 10n, 'primary token'); +expect(h.spare !== null && h.spare.value() === 11n, 'spare token'); +expect(h.many.length === 3 && h.many[2].value() === I64_MIN, 'many tokens'); +expect(Object.keys(h.by_name).length === 2 && h.by_name.b.value() === 21n, 'tokens by name'); +// Each encoding mints fresh references, so the holder can be sent twice. +const expected = BigInt.asIntN(64, 10n + 11n + 12n + 13n + 20n + 21n + I64_MIN); +expect(codec.sumHolder(h) === expected && codec.sumHolder(h) === expected, 'sumHolder twice'); +// primaryOf returns the very same object (a new wrapper of it). +const p = codec.primaryOf(h); +const holder = (primary) => ({ primary, spare: null, many: [], by_name: {} }); +expect(codec.samePrimary(h, holder(p)), 'primaryOf is the same object'); +const twin = new Token(10n); +expect(!codec.samePrimary(h, holder(twin)), 'an equal value is not the same object'); +// A holder of consumer-made tokens, one wrapper in every slot. +const minus4 = new Token(-4n); +expect( + codec.sumHolder({ primary: twin, spare: twin, many: [twin, twin, minus4], by_name: { k: twin } }) === 46n, + 'a holder of consumer tokens', +); +for (const t of [p, twin, minus4]) t.close(); +release(full); +throws(() => p.value(), (e) => e instanceof CodecError && e.code === -3, 'a closed token'); -scalars(); -strings(); -shapes(); -composites(); -holders(); -await finish(api, 'codec'); +await finish(api, `codec (${n} vectors)`); diff --git a/conformance/node/contacts.mjs b/conformance/node/contacts.mjs deleted file mode 100644 index a2fa12fb..00000000 --- a/conformance/node/contacts.mjs +++ /dev/null @@ -1,119 +0,0 @@ -// Conformance consumer: contacts sample (node and wasm lanes). -// -// The ContactBook class (canonical constructor, methods, per-object state), -// records as plain objects, an optional string, a C-style enum, a list of -// records, the typed error classes (NotFoundError and InvalidNameError -// extending the `contacts.ContactsError` domain and the package root), the -// nested `contacts.groups` module taking and returning the parent's enum and -// interface and reporting the parent's error domain, the sibling -// `directory` root taking a `contacts` record, and `close()`. - -import { expect, finish, isWasm, load, same, throws } from './harness.mjs'; - -const api = await load('contacts'); -const { contacts, directory, ContactsError } = api; -const { ContactType } = contacts; - -function main() { - expect(ContactType.Personal === 0 && ContactType.Work === 1 && ContactType.Other === 2, 'ContactType values'); - expect(ContactType[1] === 'Work', 'ContactType reverse mapping'); - expect(Object.isFrozen(contacts) && Object.isFrozen(contacts.groups), 'namespaces are frozen'); - - const book = new contacts.ContactBook(); - expect(book instanceof contacts.ContactBook, 'book is a ContactBook'); - expect(api.__debugLive(0) === 1n, 'the leak counters see the live object'); - - const alice = book.add('Alice', 'Smith', 'alice@example.com', ContactType.Work); - expect(typeof alice.id === 'bigint' && alice.id > 0n, 'add returns the stored record with a bigint id'); - same( - book.get(alice.id), - { id: alice.id, first_name: 'Alice', last_name: 'Smith', email: 'alice@example.com', contact_type: ContactType.Work }, - 'get returns every field', - ); - const bob = book.add('Bob', 'Jones', null, ContactType.Personal); - expect(book.get(bob.id).email === null, 'a missing email round-trips as null'); - const carol = book.add('Carol', 'Adams', null, ContactType.Work); - expect(book.count() === 3, 'count is a number'); - same(book.list().map((c) => c.first_name).sort(), ['Alice', 'Bob', 'Carol'], 'list returns every record'); - - // The nested module uses the parent's enum and interface. - expect(contacts.groups.countOfType(book, ContactType.Work) === 2, 'countOfType(Work)'); - expect(contacts.groups.dominantType(book) === ContactType.Work, 'dominantType returns the parent enum'); - const work = contacts.groups.splitByType(book, ContactType.Work); - expect(work instanceof contacts.ContactBook, 'splitByType returns the parent interface'); - expect(work.count() === 2 && book.count() === 3, 'the split book is a new object'); - expect(contacts.groups.firstOfType(book, ContactType.Personal).first_name === 'Bob', 'firstOfType'); - throws( - () => contacts.groups.firstOfType(work, ContactType.Other), - (e) => e instanceof contacts.NotFoundError && e instanceof contacts.ContactsError && e.code === 2, - 'the nested module reports the parent domain', - ); - work.close(); - - // The sibling root takes a record declared in `contacts`. - same( - directory.card(book.get(alice.id)), - { display_name: 'Smith, Alice', initials: 'AS', has_email: true }, - 'directory.card takes a contacts record', - ); - same( - directory.sorted(book.list()).map((c) => c.last_name), - ['Adams', 'Jones', 'Smith'], - 'directory.sorted round-trips a list of contacts records', - ); - throws( - () => directory.sorted([]), - (e) => e instanceof directory.EmptyError && e instanceof directory.DirectoryError && e instanceof ContactsError && e.code === 1, - 'directory reports its own domain', - ); - expect(!(new directory.EmptyError() instanceof contacts.ContactsError), 'the two domains are unrelated'); - expect(directory.EmptyError.CODE === 1 && new directory.EmptyError().message === 'no contacts to list', 'default code message'); - - expect(book.remove(carol.id) === true && book.remove(carol.id) === false, 'remove'); - - throws( - () => book.get(9999n), - (e) => e instanceof contacts.NotFoundError && e instanceof ContactsError && e.code === 2 && e.message === 'contact not found', - 'missing contact', - ); - throws( - () => book.add('', 'Nameless', null, ContactType.Other), - (e) => e instanceof contacts.InvalidNameError && e.code === 1, - 'empty name', - ); - throws(() => book.add('X', 'Y', null, 'Work'), (e) => e instanceof TypeError, 'an enum must be a number'); - - const other = new contacts.ContactBook(); - expect(other.count() === 0 && book.count() === 2, 'each book owns its state'); - other.close(); - other.close(); - book.close(); - throws(() => book.count(), (e) => e instanceof ContactsError && e.code === -3, 'use after close'); -} - -// On Node.js the addon keeps its state per environment, so a worker thread -// can load and use the bindings alongside the main thread. -async function worker() { - const { Worker } = await import('node:worker_threads'); - const url = new URL('./node_modules/contacts/index.js', import.meta.url).href; - const thread = new Worker( - `const { parentPort } = require('node:worker_threads'); - import(${JSON.stringify(url)}).then(({ contacts }) => { - const book = new contacts.ContactBook(); - book.add('Wanda', 'Worker', null, contacts.ContactType.Work); - parentPort.postMessage(contacts.groups.countOfType(book, contacts.ContactType.Work)); - book.close(); - });`, - { eval: true }, - ); - const count = await new Promise((resolve, reject) => { - thread.once('message', resolve); - thread.once('error', reject); - }); - expect(count === 1, `a worker thread uses the bindings (got ${count})`); - await thread.terminate(); -} - -main(); -if (!isWasm(api)) await worker(); -await finish(api, 'contacts'); diff --git a/conformance/node/events.mjs b/conformance/node/events.mjs deleted file mode 100644 index 2430feae..00000000 --- a/conformance/node/events.mjs +++ /dev/null @@ -1,139 +0,0 @@ -// Conformance consumer: events sample (node and wasm lanes). -// -// A `Subscriber` callback interface implemented by a class and by object -// literals: `route` (whose producer-side trait method returns -// `Result`) steering delivery, `onMessage` returning -// a bigint, and `onAttached` receiving the EventBus object. Also the -// EventBus class, `publishLater` (whose callbacks run on a producer thread -// on Node.js and hop to the JavaScript thread), the lazy `messages()` -// iterator, a nullable record, `routeOnce`, failures thrown by an -// implementation surfacing as code -4, and leak-free teardown. - -import { expect, finish, isWasm, load, rejects, same, throws } from './harness.mjs'; - -const api = await load('events'); -const { events, EventsError } = api; -const { Delivery } = events; - -class Recorder { - constructor(name, opts = {}) { - this.name = name; - this.opts = opts; - this.routed = []; - this.received = []; - this.attached = []; - } - - route(topic) { - this.routed.push(topic); - if (topic === this.opts.throwOn) throw new Error(`subscriber ${this.name} rejects ${topic}`); - if (topic === this.opts.skip) return Delivery.Skip; - if (topic === this.opts.stop) return Delivery.AcceptAndStop; - return Delivery.Accept; - } - - onMessage(message) { - this.received.push(message); - return BigInt(this.received.length); - } - - onAttached(bus) { - expect(bus instanceof events.EventBus, `${this.name}: onAttached receives an EventBus`); - this.attached.push(bus.subscriberCount()); - // The adopted wrapper is this subscriber's to release. - bus.close(); - } -} - -async function main() { - expect(Delivery.Accept === 0 && Delivery.Skip === 1 && Delivery.AcceptAndStop === 2, 'Delivery values'); - - const bus = new events.EventBus(); - expect(bus.subscriberCount() === 0n && bus.lastMessage() === null, 'fresh bus'); - same([...bus.messages()], [], 'no messages yet'); - - const a = new Recorder('a', { skip: 'quiet', stop: 'stop' }); - const b = new Recorder('b'); - expect(bus.subscribe(a) === 1n && bus.subscribe(b) === 2n, 'subscribe returns the count'); - same(a.attached, [0n], 'a attached before it was added'); - same(b.attached, [1n], 'b attached after a'); - - expect(bus.publish('news', 'hello', ['x', 'y']) === 2n, 'both accept'); - same(a.received[0], { seq: 1n, topic: 'news', text: 'hello', tags: ['x', 'y'] }, 'the message record'); - expect(bus.publish('quiet', 'psst', []) === 1n && a.received.length === 1, 'Skip'); - expect(bus.publish('stop', 'last', ['z']) === 1n && b.routed.length === 2, 'AcceptAndStop'); - - const later = bus.publishLater('news', 'later'); - expect(later instanceof Promise, 'publishLater returns a Promise'); - expect((await later) === 2n, 'publishLater resolves with the accepted count'); - expect(a.received[2].text === 'later' && b.received[2].seq === 4n, 'async delivery'); - - same([...bus.messages()], ['hello', 'psst', 'last', 'later'], 'messages() in order'); - const it = bus.messages(); - expect(it.next().value === 'hello', 'lazy first element'); - it.return(); - expect(it.next().done === true, 'done after return()'); - for (const text of bus.messages()) { - if (text === 'psst') break; - } - same(bus.lastMessage(), { seq: 4n, topic: 'news', text: 'later', tags: [] }, 'lastMessage'); - - // `route` returns a Result on the producer side: a thrown exception comes - // back to the bus as an Err, which it propagates as code -4. - const bad = new Recorder('bad', { throwOn: 'boom' }); - expect(bus.subscribe(bad) === 3n, 'subscribe bad'); - throws( - () => bus.publish('boom', 'x', []), - (e) => e instanceof EventsError && e.code === -4 && e.message.includes('rejects boom'), - 'a Result-returning callback failure', - ); - expect(a.received.length === 4 && b.received.length === 4, 'earlier subscribers still delivered'); - expect(bus.publish('ok', 'y', []) === 3n, 'the bus works after a callback failure'); - await rejects( - bus.publishLater('boom', 'z'), - (e) => e instanceof EventsError && e.code === -4 && e.message.includes('rejects boom'), - 'an async callback failure', - ); - - // `onMessage` returns a plain value: a failure there aborts the call. - const failing = { route: () => Delivery.Accept, onMessage() { throw new TypeError('no thanks'); }, onAttached: (bus) => bus.close() }; - const solo = new events.EventBus(); - solo.subscribe(failing); - throws( - () => solo.publish('t', 'x', []), - (e) => e instanceof EventsError && e.code === -4 && e.message.includes('no thanks'), - 'a plain-return callback failure', - ); - // A wrong return type is a failure of the implementation too. - const weird = { route: () => 'not a number', onMessage: () => 0n, onAttached: () => {} }; - throws(() => events.routeOnce(weird, 'x'), (e) => e instanceof EventsError && e.code === -4, 'a bad return type'); - throws(() => events.routeOnce(null, 'x'), (e) => e instanceof TypeError, 'a missing implementation'); - expect(events.routeOnce(new Recorder('r', { skip: 'quiet' }), 'quiet') === Delivery.Skip, 'routeOnce'); - expect(events.routeOnce({ route: () => Delivery.AcceptAndStop }, 'x') === Delivery.AcceptAndStop, 'an object literal'); - - if (!isWasm(api)) { - // A callback made on a producer thread runs on this thread while the - // caller awaits, so it may use the bindings freely. - const nested = { route: () => Delivery.Accept, onMessage: (m) => BigInt(bus.subscriberCount()) + m.seq, onAttached: (b) => b.close() }; - const busy = new events.EventBus(); - busy.subscribe(nested); - expect((await busy.publishLater('t', 'x')) === 1n, 'a producer-thread callback can call back in'); - busy.close(); - } - - // Release everything. The subscribers are freed when their bus is. - solo.clearSubscribers(); - solo.close(); - bus.clearSubscribers(); - expect(bus.subscriberCount() === 0n && bus.publish('news', 'nobody', []) === 0n, 'clearSubscribers'); - bus.close(); - bus.close(); - throws(() => bus.subscriberCount(), (e) => e instanceof EventsError && e.code === -3, 'use after close'); - const disposed = new events.EventBus(); - disposed[Symbol.dispose](); - disposed[Symbol.dispose](); - throws(() => disposed.subscriberCount(), (e) => e.code === -3, 'Symbol.dispose releases'); -} - -await main(); -await finish(api, 'events'); diff --git a/conformance/node/kvstore.mjs b/conformance/node/kvstore.mjs index 14549f2b..9750c318 100644 --- a/conformance/node/kvstore.mjs +++ b/conformance/node/kvstore.mjs @@ -1,176 +1,476 @@ -// Conformance consumer: kvstore sample (node and wasm lanes). +// Conformance consumer: kvstore sample, the feature-complete producer (node +// and wasm lanes). // -// The Store class (static factories, methods, statics, close()), the typed -// errors of the `kv` domain, records with bytes, optional, list, and map -// fields, the lazy `listKeys` iterator, the `kv.stats` submodule, the -// `EvictionListener` callback interface, objects in every buffered position -// (`share`, `fork`, `larger`, `describe`, `openMany`, `totalCount`), the -// cancellable async `compact`, and leak-free teardown. +// The `Store` class (a throwing factory, the `new` constructor, methods, +// statics, the deprecated `size`, close()), typed `KvError`s with their +// payload fields, records with bytes, optional, list, and map fields, lazy +// iterators of strings, records, and objects, three callback interfaces +// implemented in JavaScript (a `Listener` that is retained, filtered, and +// detached when it throws; a `Policy` with a record return, a typed error +// raised back through `put`, and object parameters and returns; a `Loader` +// passed as an optional callback with string, bytes, and optional-object +// returns), objects in every buffered position, async calls with `AbortSignal` +// cancellation, the nested `kv.stats` module, the sibling `report` root, and +// leak-free teardown. `__debugLive(1)` (live callbacks) shows when the +// producer released an implementation. +// +// On wasm32 the producer has no threads: compaction pauses for no time and +// notifies inline, so the mid-pause cancellation and the off-thread delivery +// checks run on Node.js only. import { expect, finish, isWasm, load, rejects, same, throws } from './harness.mjs'; const api = await load('kvstore'); -const { kv, KvstoreError, CancelledError } = api; -const { EntryKind, EvictionReason } = kv; +const { kv, report, KvstoreError, CancelledError } = api; +const { EntryKind, Store } = kv; +const wasm = isWasm(api); + +const enc = (s) => new TextEncoder().encode(s); +const dec = (b) => new TextDecoder().decode(b); +const callbacks = () => api.__debugLive(1); + +function put(store, key, value, kind = EntryKind.Persistent, ttl = null) { + return store.put(key, enc(value), kind, ttl); +} + +const keyNotFound = (key) => (e) => + e instanceof kv.KeyNotFoundError && e instanceof kv.KvError && e.code === 1001 && e.key === key; + +// 1. Load: the import checked the ABI revision and both contract tables. +expect(api.__debugLive(-1) === 1n, 'the sample counts live resources'); + +async function constructors() { + throws( + () => Store.open(''), + (e) => + e instanceof kv.InvalidPathError && + e instanceof KvstoreError && + e.code === 1004 && + kv.InvalidPathError.CODE === 1004 && + e.message === 'invalid path' && + Object.keys(e).every((k) => k === 'name' || k === 'code'), + 'open("") is InvalidPath, with no payload', + ); + const s = new Store(); + expect(s.path() === 'memory', 'new Store() is in memory'); + expect(s.capacity() === Store.defaultCapacity() && Store.defaultCapacity() === 1000000, 'default capacity'); + s.close(); + const opened = await kv.openStore('/async'); + expect(opened instanceof Store && opened.path() === '/async', 'openStore resolves to a Store'); + opened.close(); + await rejects(kv.openStore(''), (e) => e instanceof kv.InvalidPathError, 'openStore("") rejects'); +} + +function basics() { + const s = Store.open('/basics'); + same( + put(s, 'alpha', 'one'), + { key: 'alpha', value: enc('one'), kind: EntryKind.Persistent, version: 1, expires_at: null, tags: [], metadata: {} }, + 'put returns the stored entry', + ); + const second = put(s, 'alpha', 'two', EntryKind.Volatile); + expect(second.version === 2 && second.kind === EntryKind.Volatile, 'a second put bumps the version'); + expect(dec(s.get('alpha').value) === 'two', 'get'); + throws( + () => s.get('nope'), + (e) => keyNotFound('nope')(e) && e.message === 'key not found: nope', + 'get a missing key', + ); + expect(s.find('alpha')?.version === 2 && s.find('nope') === null, 'find'); + + expect(put(s, 'ttl', 'x', EntryKind.Volatile, 10n).expires_at === 10n, 'a ttl sets expires_at'); + expect(s.now() === 0n, 'the clock starts at zero'); + expect(s.tick(9n) === 9n && s.count() === 2, 'tick(9)'); + expect(s.tick(1) === 10n && s.count() === 1, 'tick(1) expires the entry'); + throws( + () => s.get('ttl'), + (e) => + e instanceof kv.ExpiredError && e.key === 'ttl' && e.expired_at === 10n && e.message === 'entry ttl expired at 10', + 'get an expired key', + ); + throws(() => s.get('ttl'), keyNotFound('ttl'), 'the expired read removed it'); + + s.setCapacity(1); + expect(s.capacity() === 1, 'setCapacity'); + put(s, 'alpha', 'three'); + throws( + () => put(s, 'beta', 'b'), + (e) => e instanceof kv.StoreFullError && e.capacity === 1, + 'a new key past the capacity', + ); + s.setCapacity(100); + throws( + () => put(s, 'k', 'v', 9), + (e) => e instanceof KvstoreError && !(e instanceof kv.KvError) && e.code === -3, + 'an undeclared enum value is a marshalling failure', + ); + + put(s, 'beta', 'b'); + expect(s.delete('beta') === true && s.delete('beta') === false, 'delete'); + expect(s.size() === s.count() && s.count() === 1, 'the deprecated size'); + expect(s.clear() === 1 && s.count() === 0, 'clear'); + s.close(); + throws(() => s.count(), (e) => e instanceof KvstoreError && e.code === -3, 'a closed store'); +} + +function iterators() { + const s = Store.open('/iter'); + put(s, 'user.bob', 'b'); + put(s, 'user.alice', 'a'); + put(s, 'sys.x', 'xx'); + + same([...s.keys(null)], ['sys.x', 'user.alice', 'user.bob'], 'keys in order'); + throws(() => s.keys('zzz'), keyNotFound('zzz'), 'a prefix that matches nothing'); + const partial = s.keys('user.'); + expect(partial.next().value === 'user.alice', 'a lazy first key'); + partial.return(); + expect(partial.next().done && api.__debugLive(2) === 0n, 'abandoning an iterator releases it'); + + const entries = [...s.entries('sys.')]; + expect(entries.length === 1 && entries[0].key === 'sys.x' && dec(entries[0].value) === 'xx', 'entries'); + + const prefixes = ['user.', 'sys.', 'none.']; + const parts = [...s.partition(prefixes)]; + expect(parts.length === 3, 'one store per prefix'); + parts.forEach((p, i) => { + expect(p instanceof Store && p.path() === prefixes[i], `partition ${i} path`); + expect(p.count() === [2, 1, 0][i], `partition ${i} count`); + p.close(); + }); + s.close(); +} class Listener { - constructor(stopAfter = Infinity) { - this.stopAfter = stopAfter; - this.seen = []; + constructor({ skip = null, failOn = null } = {}) { + this.skip = skip; + this.failOn = failOn; + this.changes = []; + } + + accepts(key) { + if (key === this.failOn) throw new Error('listener refused'); + return key !== this.skip; + } + + onChange(change) { + this.changes.push(change); + } + + count(tag, pred = () => true) { + return this.changes.filter((c) => c.tag === tag && pred(c)).length; + } +} + +function listeners() { + const base = callbacks(); + const s = Store.open('/listen'); + const l = new Listener({ skip: 'quiet' }); + const id = s.subscribe(l); + expect(id > 0 && s.listenerCount() === 1 && callbacks() === base + 1n, 'subscribe'); + + // Synchronous calls notify before they return. + const last = () => l.changes.at(-1); + put(s, 'a', '1'); + expect(last().tag === 'Put' && last().entry.version === 1 && last().replaced === false, 'a new key'); + put(s, 'a', '2'); + expect(last().tag === 'Put' && last().entry.version === 2 && last().replaced === true, 'a replaced key'); + put(s, 'quiet', 'x'); + expect(l.count('Put') === 2, 'accepts filters a key'); + s.delete('a'); + same(l.changes.at(-1), { tag: 'Removed', key: 'a', expired: false }, 'a delete'); + put(s, 'short', 'x', EntryKind.Volatile, 1n); + s.tick(1n); + throws(() => s.get('short'), (e) => e instanceof kv.ExpiredError, 'an expired read'); + same(l.changes.at(-1), { tag: 'Removed', key: 'short', expired: true }, 'an expired read notifies'); + expect(s.clear() === 1, 'clear leaves quiet'); + same(l.changes.at(-1), { tag: 'Cleared', count: 1 }, 'a clear'); + + expect(s.unsubscribe(id) === true && callbacks() === base, 'unsubscribe releases the listener'); + expect(s.unsubscribe(id) === false && s.listenerCount() === 0, 'a second unsubscribe'); + + // A listener that throws is detached (and released); the put succeeds. + const failing = new Listener({ failOn: 'boom' }); + s.subscribe(failing); + put(s, 'fine', '1'); + expect(failing.count('Put') === 1, 'the failing listener sees fine'); + put(s, 'boom', '1'); + expect(s.count() === 2 && s.listenerCount() === 0 && callbacks() === base, 'a throwing listener is detached'); + + // Releasing the store releases the listeners it still holds. + s.subscribe(new Listener()); + s.subscribe(new Listener()); + expect(s.listenerCount() === 2 && callbacks() === base + 2n, 'two more listeners'); + s.close(); + expect(callbacks() === base, 'closing the store releases its listeners'); +} + +class Policy { + constructor(other) { + this.other = other; + this.admitted = 0; } - onEvict(entry, reason) { - this.seen.push({ key: entry.key, reason, entry }); - return this.seen.length < this.stopAfter; + admit(entry) { + this.admitted++; + expect(entry.version === 0, 'admit sees version 0'); + if (entry.key.startsWith('secret')) { + throw new kv.RejectedError({ key: entry.key, reason: 'no secrets' }, 'secrets are not stored'); + } + if (entry.key.startsWith('boom')) throw new Error('policy exploded'); + if (entry.key.startsWith('garbage')) return { key: entry.key }; + return { ...entry, key: 'renamed', kind: EntryKind.Encrypted, tags: ['admitted'] }; + } + + route(key, home) { + if (key.startsWith('b/')) { + home.close(); + return this.other; + } + if (key.startsWith('null/')) { + home.close(); + return null; + } + return home; } } -const isClosed = (e) => e instanceof KvstoreError && e.code === -3; +function policies() { + const base = callbacks(); + const s = Store.open('/policy'); + const other = Store.open('/other'); + const p = new Policy(other); + s.setPolicy(p); + expect(s.hasPolicy() && callbacks() === base + 1n, 'setPolicy'); + + const e = put(s, 'a', '1', EntryKind.Volatile); + expect(e.key === 'a' && e.version === 1 && e.kind === EntryKind.Encrypted, "admit's rewrite is stored"); + same(e.tags, ['admitted'], "admit's tags"); + put(s, 'b/x', '2', EntryKind.Volatile); + expect(s.count() === 1 && other.count() === 1, 'route redirects a write'); -async function main() { - expect(EntryKind.Persistent === 1 && EvictionReason[1] === 'Expired', 'enums'); throws( - () => kv.Store.open(''), - (e) => e instanceof kv.IoError && e instanceof kv.KvError && e instanceof KvstoreError && e.code === 1004 && kv.IoError.CODE === 1004, - 'a typed error from a factory', + () => put(s, 'secret', '3'), + (err) => + err instanceof kv.RejectedError && + err.code === 1005 && + err.key === 'secret' && + err.reason === 'no secrets' && + err.message === 'secrets are not stored', + 'a typed error from admit reaches the caller with its payload', ); - throws(() => new kv.Store(), (e) => e instanceof TypeError, 'Store has no public constructor'); - - const store = kv.Store.open('/tmp/conformance-kvstore'); - expect(kv.Store.defaultCapacity() === 1000000n, 'a static method'); - const payload = new Uint8Array([1, 2, 3]); - expect(store.put('alpha', payload, EntryKind.Persistent, null), 'put without a ttl'); - expect(store.put('beta', payload, EntryKind.Volatile, 3600n), 'put with a bigint ttl'); - expect(store.put('gamma', payload, EntryKind.Encrypted, 7200), 'put with a number ttl'); - expect(store.count() === 3n, 'count'); - - same([...store.listKeys(null)], ['alpha', 'beta', 'gamma'], 'listKeys'); - same([...store.listKeys('al')], ['alpha'], 'listKeys with a prefix'); - const keys = store.listKeys(null); - expect(keys.next().value === 'alpha', 'lazy first key'); - keys.close(); - expect(keys.next().done, 'a closed iterator is done'); - store.listKeys(null).next(); - - const alpha = store.get('alpha'); - expect(alpha.id === 1n && alpha.key === 'alpha' && alpha.created_at > 0n, 'record fields'); - expect(alpha.value instanceof Uint8Array, 'bytes are a Uint8Array'); - same(alpha.value, payload, 'bytes round-trip'); - same([alpha.expires_at, alpha.tags, alpha.metadata], [null, [], {}], 'empty optional, list, and map'); - const beta = store.get('beta'); - expect(beta.expires_at === beta.created_at + 3600n, 'an optional bigint'); + const foreign = (message) => (err) => + err instanceof KvstoreError && !(err instanceof kv.KvError) && err.code === -4 && err.message.includes(message); + throws(() => put(s, 'boom', '4'), foreign('policy exploded'), 'any other exception is -4 with its message'); + // The binding checks returns, so a malformed record or a null object + // never reaches the producer: each fails the callback (-4) instead. + throws(() => put(s, 'garbage', '5'), foreign('expected'), 'a malformed admit return'); + throws(() => put(s, 'null/x', '6'), foreign('expected a Store'), 'a null route return'); + expect(s.count() === 1 && other.count() === 1 && p.admitted === 6, 'failed puts change nothing'); + + s.setPolicy(new Policy(other)); + expect(callbacks() === base + 1n, 'replacing the policy releases the old one'); + s.setPolicy(null); + expect(!s.hasPolicy() && callbacks() === base, 'setPolicy(null) releases it'); + put(s, 'secret', 'now allowed'); + expect(s.count() === 2, 'no policy, no veto'); + other.close(); + s.close(); +} + +class Loader { + constructor(fallback = null) { + this.store = fallback; + } + + name() { + return 'js-loader'; + } + + fallback(key) { + return key === 'fb' ? this.store : null; + } + + load(key) { + if (key === 'missing') throw new kv.KeyNotFoundError({ key: 'missing' }, 'not in the loader'); + if (key === 'elsewhere') throw new kv.KeyNotFoundError({ key: 'other' }, 'not in the loader'); + if (key === 'broken') throw new Error('loader is broken'); + return enc(`loaded:${key}`); + } +} + +function loaders() { + const base = callbacks(); + const s = Store.open('/load'); + expect(s.getOrLoad('k', null) === null, 'no loader, no entry'); + const loaded = s.getOrLoad('k', new Loader()); + expect(dec(loaded.value) === 'loaded:k' && loaded.kind === EntryKind.Volatile, 'a loaded entry'); + same(loaded.metadata, { source: 'js-loader' }, 'the loader name is recorded'); + expect(callbacks() === base, 'the loader is released after the call'); + expect(s.getOrLoad('k', new Loader())?.version === 1, 'a hit skips the loader'); + + const backup = Store.open('/backup'); + put(backup, 'fb', 'from backup'); + const copied = s.getOrLoad('fb', new Loader(backup)); + expect(dec(copied.value) === 'from backup' && copied.kind === EntryKind.Persistent, 'the fallback store'); + backup.close(); + + expect(s.getOrLoad('missing', new Loader()) === null, 'KeyNotFound for this key is none'); throws( - () => store.get('missing'), - (e) => e instanceof kv.KeyNotFoundError && e.code === 1001 && e.message === 'key not found', - 'a typed error from a method', + () => s.getOrLoad('elsewhere', new Loader()), + (e) => keyNotFound('other')(e) && e.message === 'not in the loader', + 'KeyNotFound for another key passes through', ); - expect(store.legacyPut('legacy', payload) && store.delete('legacy') && !store.delete('legacy'), 'legacyPut and delete'); - same(kv.stats.getStats(store), { total_entries: 3n, total_bytes: 9n, expired_entries: 0n }, 'the stats submodule'); - - // The eviction listener. - const l1 = new Listener(); - store.setEvictionListener(l1); - expect(store.delete('gamma') && l1.seen.length === 1, 'the listener fires synchronously'); - expect(l1.seen[0].key === 'gamma' && l1.seen[0].reason === EvictionReason.Deleted, 'listener arguments'); - same(l1.seen[0].entry.value, payload, 'the listener decodes its record'); - expect(store.put('doomed', new Uint8Array([9]), EntryKind.Volatile, -1n), 'put an expired entry'); - throws(() => store.get('doomed'), (e) => e instanceof kv.ExpiredError && e.code === 1002, 'expired'); - expect(l1.seen[1].reason === EvictionReason.Expired, 'eviction on read'); - const l2 = new Listener(1); - store.setEvictionListener(l2); - store.put('x1', payload, EntryKind.Volatile, null); - store.put('x2', payload, EntryKind.Volatile, null); - store.delete('x1'); - store.delete('x2'); - expect(l2.seen.length === 1 && l1.seen.length === 2, 'a listener returning false detaches'); - store.setEvictionListener({ - onEvict(entry) { - throw new TypeError('listener refuses ' + entry.key); - }, - }); - store.put('x3', payload, EntryKind.Volatile, null); throws( - () => store.delete('x3'), - (e) => e instanceof KvstoreError && !(e instanceof kv.KvError) && e.code === -4 && e.message.includes('listener refuses x3'), - 'a throwing listener fails the call with code -4', + () => s.getOrLoad('broken', new Loader()), + (e) => e instanceof KvstoreError && e.code === -4 && e.message === 'loader is broken', + 'any other loader failure is -4', ); - store.setEvictionListener({ onEvict: () => 'yes' }); - store.put('x4', payload, EntryKind.Volatile, null); - throws(() => store.delete('x4'), (e) => e.code === -4, 'a wrong return type is code -4'); - store.clearEvictionListener(); - expect(store.count() === 2n, 'alpha and beta remain'); - - // Objects in every position. - const shared = store.share(); - expect(shared instanceof kv.Store && shared !== store, 'share returns a new wrapper'); - shared.put('via-shared', payload, EntryKind.Volatile, null); - expect(store.count() === 3n, 'the wrappers share the object'); - shared.close(); - store.delete('via-shared'); - const forked = store.fork(); - forked.put('only-in-fork', payload, EntryKind.Volatile, null); - expect(forked.count() === 3n && store.count() === 2n, 'fork is independent'); - const empty = kv.Store.open('/tmp/empty'); - expect(empty.larger(null) === null && empty.larger(undefined) === null, 'a null object result'); - const bigger = store.larger(forked); - expect(bigger.count() === 3n, 'an object argument and result'); + expect(callbacks() === base, 'every loader is released'); + s.close(); +} + +async function asyncCalls() { + const s = Store.open('/async-calls'); + const l = new Listener(); + s.subscribe(l); + put(s, 'old1', 'x', EntryKind.Volatile, 1n); + put(s, 'old2', 'x', EntryKind.Volatile, 1n); + put(s, 'keep', 'x'); + s.tick(5n); + + // compact notifies from a producer thread; Node.js runs the listener on + // the JavaScript thread, so nothing arrives before control returns to the + // event loop. On wasm32 it notifies inline. + const expired = () => l.count('Removed', (c) => c.expired); + const pending = s.compact(0, { signal: new AbortController().signal }); + if (!wasm) expect(expired() === 0, 'off-thread notifications wait for the event loop'); + expect((await pending) === 2, 'compact removes the expired entries'); + expect(expired() === 2 && s.count() === 1, 'the listener saw both removals'); + expect((await s.compact(5)) === 0, 'compact without a signal'); + await rejects(s.compact(0, { signal: AbortSignal.abort() }), (e) => e instanceof CancelledError, 'an aborted signal'); + + if (!wasm) { + const controller = new AbortController(); + const started = Date.now(); + const long = s.compact(60000, { signal: controller.signal }); + await new Promise((resolve) => setTimeout(resolve, 20)); + expect(Store.activeJobs() >= 1, 'the pause is running'); + controller.abort(); + await rejects(long, (e) => e instanceof CancelledError && e.code === -5, 'cancelled mid-pause'); + expect(Date.now() - started < 5000, 'cancellation is prompt'); + let stopped = false; + for (let i = 0; i < 200 && !stopped; i++) { + stopped = Store.activeJobs() === 0; + if (!stopped) await new Promise((resolve) => setTimeout(resolve, 10)); + } + expect(stopped, 'the cancelled pause stopped cooperatively'); + } + + const many = await Promise.all(Array.from({ length: 32 }, () => s.getMany(['keep', 'gone', 'keep']))); + expect( + many.every((r) => r.length === 3 && r[0]?.key === 'keep' && r[1] === null && r[2]?.key === 'keep'), + '32 concurrent getMany calls', + ); + + const other = Store.open('/other'); + put(other, 'a', '123'); + same( + await kv.stats.summarizeAll([s, other]), + { entries: 2, bytes: 4n, by_kind: { [EntryKind.Persistent]: 2 } }, + 'summarizeAll', + ); + other.close(); + s.close(); +} + +function objectGraph() { + const s = Store.open('/graph'); + put(s, 'k', 'v'); + + const shared = s.share(); + put(s, 'via-original', 'v'); + expect(shared.find('via-original') !== null, 'share() is the same object'); + s.delete('via-original'); + s.close(); + expect(shared.count() === 1, 'still alive through the shared reference'); + + const fork = shared.fork(); + expect(fork.count() === 1 && fork.path() === '/graph', 'fork copies the entries'); + put(fork, 'k2', 'v'); + expect(fork.count() === 2 && shared.count() === 1, 'fork is distinct'); + + const empty = Store.open('/empty'); + expect(empty.larger(null) === null, 'larger(null) on an empty store'); + const bigger = empty.larger(fork); + expect(bigger.count() === 2, 'larger picks the fork'); bigger.close(); - const info = store.describe('primary', forked); - expect(info.label === 'primary' && info.count === 2n, 'a record with objects'); - expect(info.store instanceof kv.Store && info.mirror instanceof kv.Store, 'object fields'); - expect(info.mirror.get('only-in-fork').key === 'only-in-fork', 'an object field aliases its object'); - const many = kv.Store.openMany(['/tmp/m1', '/tmp/m2']); - expect(many.length === 2 && many.every((s) => s.count() === 0n), 'a list of objects'); - throws(() => kv.Store.openMany(['/tmp/ok', '']), (e) => e instanceof kv.IoError, 'openMany fails as a whole'); - expect(kv.Store.totalCount([store, forked], null) === 5n, 'a list of objects as an argument'); - expect(kv.Store.totalCount(many, info) === 2n, 'a record of objects as an argument'); - expect(kv.Store.totalCount([], { label: 'x', store: forked, mirror: null, count: 0n }) === 3n, 'a hand-built record'); - info.store.close(); - info.mirror.close(); - for (const s of many) s.close(); - forked.close(); - throws(() => forked.count(), isClosed, 'use after close'); - throws(() => kv.Store.totalCount([forked], null), isClosed, 'a closed object in a buffer'); - throws(() => kv.stats.getStats(forked), isClosed, 'a closed object argument'); - throws(() => kv.stats.getStats({}), (e) => e instanceof TypeError, 'a non-Store argument'); - - // A close() racing a call: the listener closes the store mid-call, and - // the release waits for the call to return. - const racing = kv.Store.open('/tmp/racing'); - racing.setEvictionListener({ - onEvict() { - racing.close(); - return true; - }, - }); - racing.put('k', payload, EntryKind.Volatile, null); - expect(racing.delete('k') === true, 'the call completes after a close() from its callback'); - throws(() => racing.count(), isClosed, 'the deferred close took effect'); - - // Async compaction, and its cancellation. - store.put('dead', payload, EntryKind.Volatile, 0n); - const pending = store.compact(); - expect(pending instanceof Promise, 'compact returns a Promise'); - expect((await pending) === 3n, 'compact reclaims the expired entry'); - const aborted = new AbortController(); - aborted.abort(); - await rejects( - store.compact({ signal: aborted.signal }), - (e) => e instanceof CancelledError && e.code === -5, - 'a cancelled compact', + const self = shared.larger(null); + put(self, 'via-larger', 'v'); + expect(shared.count() === 2, 'larger(null) on a non-empty store is itself'); + self.delete('via-larger'); + self.close(); + + const info = shared.describe('main', fork); + expect(info.label === 'main' && info.count === 1, 'describe'); + expect(info.store.count() === 1 && info.mirror.count() === 2, 'describe carries both stores'); + put(info.store, 'via-info', 'v'); + expect(shared.count() === 2, 'describe carries this store itself'); + shared.delete('via-info'); + + const opened = Store.openMany(['/a', '/b']); + same(opened.map((o) => o.path()), ['/a', '/b'], 'openMany'); + throws(() => Store.openMany(['/a', '']), (e) => e instanceof kv.InvalidPathError, 'openMany with a bad path'); + + const named = Store.byLabel([info, { label: 'first', store: opened[0], mirror: null, count: 0 }]); + same(Object.keys(named).sort(), ['first', 'main'], 'byLabel keys'); + expect(named.main.count() === 1 && named.first.path() === '/a', 'byLabel values'); + + put(opened[0], 'm', '1'); + expect(Store.totalCount([opened[0], opened[1], fork], named, info) === 6, 'totalCount with extra'); + expect(Store.totalCount([opened[0], opened[1], fork], named, null) === 5, 'totalCount without extra'); + expect(shared.count() === 1 && fork.count() === 2 && opened[0].count() === 1, 'still usable after buffers'); + + for (const o of [info.store, info.mirror, named.main, named.first, ...opened, empty, fork, shared]) o.close(); +} + +function statsAndReport() { + const s = Store.open('/stats'); + put(s, 'b', '12'); + put(s, 'a', '1'); + put(s, 'a', '123'); + put(s, 'c', 'x', EntryKind.Encrypted); + same( + kv.stats.summarize(s, null), + { entries: 3, bytes: 6n, by_kind: { [EntryKind.Persistent]: 2, [EntryKind.Encrypted]: 1 } }, + 'summarize', ); - expect((await store.compact({ signal: new AbortController().signal })) === 0n, 'compact with a live signal'); - if (!isWasm(api)) { - // The pending call keeps the store alive past close(). - const p = store.compact(); - store.close(); - expect((await p) === 0n, 'a pending call survives close()'); - } else { - store.close(); - } - await rejects(store.compact(), isClosed, 'compact after close'); - empty.close(); + throws(() => kv.stats.summarize(s, 'q'), keyNotFound('q'), 'summarize a prefix that matches nothing'); + same( + report.renderReport([...s.entries(null)]), + ['a: 3 bytes, Persistent, v2', 'b: 2 bytes, Persistent', 'c: 1 bytes, Encrypted'], + 'renderReport', + ); + throws( + () => report.renderReport([]), + (e) => + e instanceof report.NothingToReportError && + e instanceof report.ReportError && + e.code === 2001 && + e.message === 'nothing to report', + 'renderReport([])', + ); + s.close(); } -await main(); +await constructors(); +basics(); +iterators(); +listeners(); +policies(); +loaders(); +await asyncCalls(); +objectGraph(); +statsAndReport(); +expect(callbacks() === 0n, 'every callback implementation was released'); + await finish(api, 'kvstore'); diff --git a/conformance/node/run.sh b/conformance/node/run.sh index 153e8731..383abfba 100755 --- a/conformance/node/run.sh +++ b/conformance/node/run.sh @@ -12,11 +12,11 @@ # targets generate the same JavaScript API. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools node node npm node_consumer() { local sample="$1" src="$2" local dir="$OUT/node-$sample" tarball - command -v npm >/dev/null 2>&1 || { echo "npm not found" >&2; return 1; } rm -rf "$dir" mkdir -p "$dir" tarball=$(cd "$dir" && npm pack --silent "$GENROOT/$sample/node") \ @@ -29,20 +29,12 @@ node_consumer() { ( cd "$dir" && node --expose-gc "$src" ) } -node_contacts() { node_consumer contacts contacts.mjs; } +node_calculator() { node_consumer calculator calculator.mjs; } +node_codec() { node_consumer codec codec.mjs; } +node_kvstore() { node_consumer kvstore kvstore.mjs; } -node_events() { node_consumer events events.mjs; } - -node_kvstore() { node_consumer kvstore kvstore.mjs; } - -node_async_demo() { node_consumer async-demo async_demo.mjs; } - -node_codec() { node_consumer codec codec.mjs; } - -lane node-contacts node_contacts -lane node-events node_events -lane node-kvstore node_kvstore -lane node-async-demo node_async_demo +lane node-calculator node_calculator lane node-codec node_codec +lane node-kvstore node_kvstore finish_lanes diff --git a/conformance/python/async_demo_consumer.py b/conformance/python/async_demo_consumer.py deleted file mode 100644 index 9a42c1db..00000000 --- a/conformance/python/async_demo_consumer.py +++ /dev/null @@ -1,110 +0,0 @@ -"""Conformance consumer: async-demo sample, Python target. - -Drives the asyncio-bridged async surface end to end: `run_task` as a -coroutine settled from the producer's worker thread and decoded from a value -buffer into the `TaskResult` dataclass, the typed `TaskError.InvalidName` -raised by a throwing coroutine, the buffered list-of-records round trip -through `run_batch`, the direct-scalar `run_n_tasks` (many at once), and -real cancellation: cancelling the task awaiting the cancellable `wait` -cancels the native token, so the call ends promptly with CancelledError -instead of waiting out its timeout, while an uncancelled `wait` completes -with its value. Ends with `active_callbacks` back at zero and the leak check -(see harness.py). -""" -import asyncio -import time - -import async_demo as wv -from harness import Consumer - -consumer = Consumer("async-demo") -check = consumer.check - - -async def basics() -> None: - result = await wv.run_task("alpha") - check(isinstance(result, wv.TaskResult), f"run_task type {type(result)}") - check(result.id > 0 and result.value == "completed: alpha" and result.success is True, - f"run_task {result}") - - try: - await wv.run_task("") - check(False, "expected TaskError.InvalidName for an empty name") - except wv.TaskError.InvalidName as exc: - check(exc.code == 1 and isinstance(exc, wv.TaskError) and isinstance(exc, wv.Error), - f"InvalidName {exc!r}") - check(exc.message == "task name must not be empty", f"message {exc.message!r}") - - batch = await wv.run_batch(["a", "b", "c"]) - check([r.value for r in batch] == ["completed: a", "completed: b", "completed: c"], - f"run_batch {batch}") - check(await wv.run_batch([]) == [], "empty run_batch") - - results = await asyncio.gather(*(wv.run_n_tasks(i) for i in range(32))) - check(results == list(range(32)), f"run_n_tasks gather {results}") - - -async def cancellation() -> None: - # An uncancelled wait completes with the milliseconds waited. - check(await wv.wait(10) == 10, "wait(10)") - - # Cancelling the awaiting task cancels the native call: it ends right - # away with CancelledError instead of after its one-minute timeout. - start = time.monotonic() - task = asyncio.ensure_future(wv.wait(60_000)) - await asyncio.sleep(0.05) - task.cancel() - try: - await task - check(False, "expected CancelledError") - except asyncio.CancelledError: - pass - check(task.cancelled(), "task reports cancelled") - check(time.monotonic() - start < 5, "cancellation did not wait for the timeout") - - # A timeout is a cancellation too. - start = time.monotonic() - try: - await asyncio.wait_for(wv.wait(60_000), 0.05) - check(False, "expected TimeoutError") - except asyncio.TimeoutError: - pass - check(time.monotonic() - start < 5, "wait_for did not wait for the native timeout") - - # Cancelling before the call ever runs is fine as well. - task = asyncio.ensure_future(wv.wait(60_000)) - task.cancel() - try: - await task - except asyncio.CancelledError: - pass - check(task.cancelled(), "immediately cancelled task") - - # Many concurrent cancellable calls, half of them cancelled. - tasks = [asyncio.ensure_future(wv.wait(20 if i % 2 else 60_000)) for i in range(16)] - await asyncio.sleep(0.05) - for i, t in enumerate(tasks): - if not i % 2: - t.cancel() - outcomes = await asyncio.gather(*tasks, return_exceptions=True) - for i, outcome in enumerate(outcomes): - if i % 2: - check(outcome == 20, f"wait {i} completed with {outcome!r}") - else: - check(isinstance(outcome, asyncio.CancelledError), f"wait {i} cancelled: {outcome!r}") - - -def main() -> None: - asyncio.run(basics()) - asyncio.run(cancellation()) - # Every task body has finished (or been dropped) by the time its - # completion fires. - for _ in range(50): - if wv.active_callbacks() == 0: - break - time.sleep(0.02) - check(wv.active_callbacks() == 0, "active_callbacks back at zero") - consumer.finish(wv) - - -main() diff --git a/conformance/python/calculator_consumer.py b/conformance/python/calculator_consumer.py index 22844a4e..e77d3f5d 100644 --- a/conformance/python/calculator_consumer.py +++ b/conformance/python/calculator_consumer.py @@ -1,31 +1,47 @@ -"""Conformance consumer: calculator sample, Python target. +"""Conformance consumer: calculator sample, Python target (ABI revision 4). -The smallest surface: direct `i32` arguments and returns, a throwing call -raising the typed `CalcError.DivisionByZero`, and a string round trip -(including an interior NUL and non-ASCII text, since strings cross as -pointer + length). Ends with the leak check (see harness.py). +The getting-started surface: direct `i32` arguments and returns (with +wrapping), a throwing call raising the typed `CalcError.DivisionByZero`, and +a string round trip (non-ASCII and astral text and an interior NUL survive, +since strings cross as pointer + length). Importing the package already +checked the ABI revision and the `calculator` contract table. Ends with the +leak check (see harness.py). """ -import calculator as wv +import calculator as calc from harness import Consumer -consumer = Consumer("calculator") +INT32_MIN = -(2**31) +INT32_MAX = 2**31 - 1 + +consumer = Consumer("calculator", calc) check = consumer.check def main() -> None: - check(wv.add(2, 3) == 5, "add") - check(wv.mul(-4, 6) == -24, "mul") - check(wv.div(17, 5) == 3, "div") - try: - wv.div(1, 0) - check(False, "expected DivisionByZero") - except wv.CalcError.DivisionByZero as exc: - check(exc.code == 1 and isinstance(exc, wv.CalcError) and isinstance(exc, wv.Error), - f"DivisionByZero {exc!r}") - check(exc.message == "division by zero", f"message {exc.message!r}") - for s in ["", "hello", "nul\0inside", "\0", "héllo wörld ✓"]: - check(wv.echo(s) == s, f"echo {s!r}") - consumer.finish(wv) + check(calc.add(2, 3) == 5, "add") + check(calc.add(-7, 7) == 0, "add to zero") + check(calc.add(INT32_MAX, 1) == INT32_MIN, "add wraps") + + check(calc.divide(10, 2) == 5, "divide") + check(calc.divide(-7, 2) == -3, "divide rounds toward zero") + check(calc.divide(INT32_MIN, -1) == INT32_MIN, "divide MIN by -1") + + exc = consumer.raises(calc.CalcError.DivisionByZero, lambda: calc.divide(1, 0), "divide(1, 0)") + check(exc.code == 1 and exc.CODE == 1, f"DivisionByZero code {exc.code}") + check(exc.message == "division by zero", f"DivisionByZero message {exc.message!r}") + check(isinstance(exc, calc.CalcError) and isinstance(exc, calc.Error), "error hierarchy") + check(calc.DivisionByZero is calc.CalcError.DivisionByZero, "scoped alias") + + for name, expected in [ + ("World", "Hello, World!"), + ("", "Hello, !"), + ("Wörld 🦀", "Hello, Wörld 🦀!"), + ("a\0b", "Hello, a\0b!"), + ]: + got = calc.greet(name) + check(got == expected, f"greet({name!r}) == {got!r}") + + consumer.finish() main() diff --git a/conformance/python/codec_consumer.py b/conformance/python/codec_consumer.py index fd82496e..c4a3e3d4 100644 --- a/conformance/python/codec_consumer.py +++ b/conformance/python/codec_consumer.py @@ -1,370 +1,283 @@ -"""Conformance consumer: codec sample, Python target. - -Drives the generated ctypes wrapper's value-buffer encoder and decoder against -the producer's round-trip oracle: the canonical `Scalars` and `Composite` -fixtures are decoded and checked field by field (producer encodes, consumer -decodes), handed back through `verify_*` (consumer encodes, producer decodes), -and re-encoded through `roundtrip_*` and compared; consumer-built values with -edge cases (every integer extreme, NaN, the infinities, negative zero, empty -strings, lists, and maps, non-BMP unicode) round-trip; every `Shape` variant -crosses alone, in a list, and inside `Composite`; the direct, string, bytes, -and optional families round-trip; and `Holder` carries `Token` objects in a -field, an optional, and a list (`sum_holder`, `primary_of` returning the same -underlying object, `same_primary`, and every wrapper released with an -idempotent `close()`). Ends with the leak check (see harness.py). +"""Conformance consumer: codec sample, Python target (ABI revision 4). + +The shared-vector loop: for every vector the producer serves, decode it with +the generated codecs, pass it back to `check_vector` (which re-decodes the +generated encoding and compares), confirm it never matches its neighbor, and +push each primitive vector's value through the matching direct-family +`echo_*`. Then vectors built from literals (so a symmetric encode/decode bug +can't hide), spot checks of decoded values, the typed out-of-range error +with its payload, malformed input rejected on both sides, and object +identity and reference counting through buffers. Ends with the leak check +(see harness.py). """ -import dataclasses -import gc +import ctypes import math +import struct +from typing import Any, Callable, Dict -import codec as wv +import codec from harness import Consumer -I64_MIN = -(1 << 63) -I64_MAX = (1 << 63) - 1 -U64_MAX = (1 << 64) - 1 +consumer = Consumer("codec", codec) +check = consumer.check +impl = consumer.impl +INT64_MIN = -(2**63) +INT64_MAX = 2**63 - 1 +UINT64_MAX = 2**64 - 1 -consumer = Consumer("codec") -check = consumer.check +def f32_bits(x: float) -> bytes: + return struct.pack(" bool: - """Bitwise-aware float equality: NaN matches NaN and -0.0 differs from 0.0.""" - if math.isnan(a) or math.isnan(b): - return math.isnan(a) and math.isnan(b) - return a == b and math.copysign(1.0, a) == math.copysign(1.0, b) - - -def canonical_scalars() -> wv.Scalars: - return wv.Scalars( - i8_value=-8, - u8_value=200, - i16_value=-16_000, - u16_value=60_000, - i32_value=-2_000_000_000, - u32_value=4_000_000_000, - i64_value=-9_007_199_254_740_993, - u64_value=U64_MAX, - f32_value=1.5, - f64_value=-2.25e100, - flag=True, - color=wv.Color.Blue, - ) +def f64_bits(x: float) -> bytes: + return struct.pack(" wv.Composite: - scalars = canonical_scalars() - return wv.Composite( - name="héllo wörld ✓", - blob=bytes([0, 1, 2, 253, 254, 255]), - some_i64=I64_MIN, - none_i64=None, - some_text="", - names=["a", "", "ccc"], - matrix=[[1, 2, 3], [], [-4]], - empty=[], - by_name={"neg": -3, "one": 1, "two": 2}, - by_id={-1: scalars, 42: dataclasses.replace(scalars, flag=False)}, - scalars=scalars, - shape=wv.Shape.Labeled(label="tag", count=3), - shapes=[ - wv.Shape.Empty(), - wv.Shape.Circle(radius=2.5), - wv.Shape.Rect(width=1.0, height=0.5), - wv.Shape.Labeled(label="", count=-1), - wv.Shape.Nested(inner=scalars, note="n"), - ], - maybe_shape=wv.Shape.Nested(inner=scalars, note=None), - maybe_list=bytes([9, 8]), - sparse=[True, None, False], - colors=[wv.Color.Red, wv.Color.Green, wv.Color.Blue], - ) +def same_f32(a: float, b: float) -> bool: + return f32_bits(a) == f32_bits(b) or (math.isnan(a) and math.isnan(b)) -def scalars() -> None: - check(wv.Color.Red == 0 and wv.Color.Green == 1 and wv.Color.Blue == 7, "Color discriminants") - - # Producer encodes, consumer decodes: every field of the fixture. - sample = wv.sample_scalars() - check(isinstance(sample, wv.Scalars), "sample_scalars type") - check(sample.i8_value == -8 and sample.u8_value == 200, "i8/u8") - check(sample.i16_value == -16_000 and sample.u16_value == 60_000, "i16/u16") - check(sample.i32_value == -2_000_000_000 and sample.u32_value == 4_000_000_000, "i32/u32") - check(sample.i64_value == -9_007_199_254_740_993, f"i64 {sample.i64_value}") - check(sample.u64_value == U64_MAX, f"u64 {sample.u64_value}") - check(sample.f32_value == 1.5 and sample.f64_value == -2.25e100, "f32/f64") - check(sample.flag is True and sample.color is wv.Color.Blue, "flag/color") - check(sample == canonical_scalars(), "sample_scalars equals the local canonical value") - - # Consumer encodes, producer decodes. - check(wv.verify_scalars(sample) is True, "verify_scalars(sample)") - check(wv.verify_scalars(canonical_scalars()) is True, "verify_scalars(local canonical)") - check(wv.roundtrip_scalars(sample) == sample, "roundtrip_scalars(sample)") - - # A one-field change is a typed Mismatch. - changed = dataclasses.replace(sample, u8_value=201) - try: - wv.verify_scalars(changed) - check(False, "expected CodecError.Mismatch") - except wv.CodecError.Mismatch as exc: - check(exc.code == 1 and exc.CODE == 1, f"Mismatch code {exc.code}") - check(isinstance(exc, wv.CodecError) and isinstance(exc, wv.Error), "hierarchy") - check(exc.message == "value does not match the canonical fixture", - f"Mismatch message {exc.message!r}") - check(wv.Mismatch is wv.CodecError.Mismatch, "bare Mismatch is the scoped alias") - - # Consumer-built extremes: every integer bound, negative zero, NaN, inf. - edges = wv.Scalars( - i8_value=-128, u8_value=255, i16_value=-32768, u16_value=65535, - i32_value=-(1 << 31), u32_value=(1 << 32) - 1, i64_value=I64_MIN, u64_value=U64_MAX, - f32_value=-0.0, f64_value=float("nan"), flag=False, color=wv.Color.Red, - ) - back = wv.roundtrip_scalars(edges) - check(back.i8_value == -128 and back.u8_value == 255, "edge i8/u8") - check(back.i16_value == -32768 and back.u16_value == 65535, "edge i16/u16") - check(back.i32_value == -(1 << 31) and back.u32_value == (1 << 32) - 1, "edge i32/u32") - check(back.i64_value == I64_MIN and back.u64_value == U64_MAX, "edge i64/u64") - check(same_float(back.f32_value, -0.0), f"f32 negative zero {back.f32_value!r}") - check(same_float(back.f64_value, float("nan")), f"f64 NaN {back.f64_value!r}") - check(back.flag is False and back.color is wv.Color.Red, "edge flag/color") - highs = wv.Scalars( - i8_value=127, u8_value=0, i16_value=32767, u16_value=0, - i32_value=(1 << 31) - 1, u32_value=0, i64_value=I64_MAX, u64_value=0, - f32_value=float("inf"), f64_value=float("-inf"), flag=True, color=wv.Color.Green, - ) - back = wv.roundtrip_scalars(highs) - check(back.i8_value == 127 and back.i16_value == 32767 and back.i32_value == (1 << 31) - 1, - "high signed bounds") - check(back.u8_value == 0 and back.u16_value == 0 and back.u32_value == 0 - and back.u64_value == 0, "unsigned zeros") - check(back.i64_value == I64_MAX, "i64 max") - check(back.f32_value == float("inf") and back.f64_value == float("-inf"), "infinities") - check(back.color is wv.Color.Green, "Green through a record") - # f32 rounds to the nearest single: 0.1 does not survive, 0.25 does. - check(wv.roundtrip_scalars(dataclasses.replace(highs, f32_value=0.25)).f32_value - == 0.25, "f32 exact value") - check(wv.roundtrip_scalars(dataclasses.replace(highs, f32_value=0.1)).f32_value - != 0.1, "f32 narrows to single precision") - - -def composite() -> None: - sample = wv.sample_composite() - check(isinstance(sample, wv.Composite), "sample_composite type") - check(sample.name == "héllo wörld ✓", f"name {sample.name!r}") - check(sample.blob == bytes([0, 1, 2, 253, 254, 255]), f"blob {sample.blob!r}") - check(sample.some_i64 == I64_MIN and sample.none_i64 is None, "optionals") - check(sample.some_text == "", f"some_text {sample.some_text!r}") - check(sample.names == ["a", "", "ccc"], f"names {sample.names}") - check(sample.matrix == [[1, 2, 3], [], [-4]], f"matrix {sample.matrix}") - check(sample.empty == [], "empty list") - check(sample.by_name == {"one": 1, "two": 2, "neg": -3}, f"by_name {sample.by_name}") - check(sorted(sample.by_id) == [-1, 42], f"by_id keys {sorted(sample.by_id)}") - check(sample.by_id[-1] == canonical_scalars(), "by_id[-1]") - check(sample.by_id[42].flag is False and sample.by_id[42].u64_value == U64_MAX, "by_id[42]") - check(sample.scalars == canonical_scalars(), "nested scalars") - check(sample.shape == wv.Shape.Labeled("tag", 3), f"shape {sample.shape}") - check(sample.shape.tag == wv.Shape.Tag.Labeled and isinstance(sample.shape, wv.Shape), - "shape tag") - check([s.tag for s in sample.shapes] == list(wv.Shape.Tag), "shapes one of each variant") - check(sample.shapes[1] == wv.Shape.Circle(2.5) and sample.shapes[2] == wv.Shape.Rect(1.0, 0.5), - "circle / rect") - check(sample.shapes[3] == wv.Shape.Labeled("", -1), "labeled with empty label") - check(sample.shapes[4] == wv.Shape.Nested(canonical_scalars(), "n"), "nested with note") - check(sample.maybe_shape == wv.Shape.Nested(canonical_scalars(), None), "maybe_shape") - check(sample.maybe_list == bytes([9, 8]), f"maybe_list {sample.maybe_list!r}") - check(sample.sparse == [True, None, False], f"sparse {sample.sparse}") - check(sample.colors == [wv.Color.Red, wv.Color.Green, wv.Color.Blue], f"colors {sample.colors}") - check(all(isinstance(c, wv.Color) for c in sample.colors), "colors are enum members") - check(sample == canonical_composite(), "sample_composite equals the local canonical value") - - check(wv.verify_composite(sample) is True, "verify_composite(sample)") - check(wv.verify_composite(canonical_composite()) is True, "verify_composite(local canonical)") - check(wv.roundtrip_composite(sample) == sample, "roundtrip_composite(sample)") - described = wv.describe_composite(sample) - check(described.startswith("Composite { name: \"héllo wörld ✓\"") and "sparse: [Some(true), None, Some(false)]" in described, - f"describe_composite {described[:80]!r}") - - changed = dataclasses.replace(sample, sparse=[True, True, False]) - try: - wv.verify_composite(changed) - check(False, "expected CodecError.Mismatch for a changed composite") - except wv.CodecError.Mismatch: - pass - - # A consumer-built composite full of empties and extremes. - nan_scalars = wv.Scalars(0, 0, 0, 0, 0, 0, 0, 0, float("nan"), -0.0, False, wv.Color.Green) - mine = wv.Composite( - name="", - blob=b"", - some_i64=I64_MAX, - none_i64=-1, - some_text=None, - names=[], - matrix=[[], [(1 << 31) - 1, -(1 << 31)]], - empty=[float("inf"), float("-inf"), -0.0, float("nan"), 1e-308, 1.7976931348623157e308], - by_name={"": I64_MIN, "𝔘𝔫𝔦𝔠𝔬𝔡𝔢 日本語 🧵": I64_MAX}, - by_id={}, - scalars=nan_scalars, - shape=wv.Shape.Empty(), - shapes=[], - maybe_shape=None, - maybe_list=None, - sparse=[], - colors=[], + +def same_f64(a: float, b: float) -> bool: + return f64_bits(a) == f64_bits(b) or (math.isnan(a) and math.isnan(b)) + + +# The direct-family echo for each primitive vector variant, and how to +# compare its result with the vector's value. +ECHOES: Dict[type, Callable[[Any], bool]] = { + codec.VectorI8: lambda v: codec.echo_i8(v) == v, + codec.VectorU8: lambda v: codec.echo_u8(v) == v, + codec.VectorI16: lambda v: codec.echo_i16(v) == v, + codec.VectorU16: lambda v: codec.echo_u16(v) == v, + codec.VectorI32: lambda v: codec.echo_i32(v) == v, + codec.VectorU32: lambda v: codec.echo_u32(v) == v, + codec.VectorI64: lambda v: codec.echo_i64(v) == v, + codec.VectorU64: lambda v: codec.echo_u64(v) == v, + codec.VectorF32: lambda v: same_f32(codec.echo_f32(v), v), + codec.VectorF64: lambda v: same_f64(codec.echo_f64(v), v), + codec.VectorFlag: lambda v: codec.echo_bool(v) is v, + codec.VectorText: lambda v: codec.echo_text(v) == v, + codec.VectorBlob: lambda v: codec.echo_blob(v) == v, + codec.VectorHue: lambda v: codec.echo_color(v) is v, +} + + +def find(n: int, name: str) -> int: + for i in range(n): + if codec.vector_name(i) == name: + return i + check(False, f"no vector named {name}") + raise AssertionError # unreachable + + +def every_vector(n: int) -> None: + for i in range(n): + v = codec.vector(i) + if not codec.check_vector(i, v): + check(False, f"vector {i} ({codec.vector_name(i)}) did not round-trip; " + f"producer saw {codec.describe_vector(v)}") + check(not codec.check_vector((i + 1) % n, v), f"vector {i} matches its neighbor") + echo = ECHOES.get(type(v)) + if echo is not None: + check(echo(v.value), f"echo of vector {i} ({codec.vector_name(i)})") + del v # releases any tokens inside + + +def canonical_scalars() -> codec.Scalars: + return codec.Scalars( + i8_value=-8, u8_value=200, i16_value=-16000, u16_value=60000, + i32_value=-2000000000, u32_value=4000000000, i64_value=-9007199254740993, + u64_value=UINT64_MAX, f32_value=1.5, f64_value=-2.25e100, flag=True, + color=codec.Color.Blue, ) - back = wv.roundtrip_composite(mine) - check(back.name == "" and back.blob == b"" and back.names == [], "empties") - check(back.some_i64 == I64_MAX and back.none_i64 == -1 and back.some_text is None, - "swapped optionals") - check(back.matrix == [[], [(1 << 31) - 1, -(1 << 31)]], f"matrix {back.matrix}") - check(len(back.empty) == 6 and all(same_float(a, b) for a, b in zip(back.empty, mine.empty)), - f"float list {back.empty}") - check(back.by_name == mine.by_name, f"unicode-keyed map {back.by_name}") - check(back.by_id == {} and back.shapes == [] and back.sparse == [] and back.colors == [], - "empty containers") - check(back.maybe_shape is None and back.maybe_list is None, "absent optionals") - check(back.shape == wv.Shape.Empty(), "unit variant") - check(same_float(back.scalars.f32_value, float("nan")) - and same_float(back.scalars.f64_value, -0.0), "NaN / -0.0 inside a nested record") - check(back.scalars.color is wv.Color.Green, "nested enum") - check(wv.describe_composite(mine).startswith("Composite { name: \"\", blob: []"), - "describe_composite of the local value") - # Presence flags matter: an absent optional list differs from an empty one. - with_empty_list = dataclasses.replace(mine, maybe_list=b"") - check(wv.roundtrip_composite(with_empty_list).maybe_list == b"", "present empty optional list") - - -def shapes() -> None: - variants = [ - wv.Shape.Empty(), - wv.Shape.Circle(radius=-0.0), - wv.Shape.Rect(width=0.5, height=float("inf")), - wv.Shape.Labeled(label="ünïcødé ✓", count=-(1 << 31)), - wv.Shape.Nested(inner=canonical_scalars(), note=""), - wv.Shape.Nested(inner=canonical_scalars(), note=None), + + +def literal_vectors(n: int) -> None: + s = canonical_scalars() + check(codec.check_vector(find(n, "scalars canonical"), codec.VectorAllScalars(s)), + "scalars canonical") + s.u16_value = 60001 + check(not codec.check_vector(find(n, "scalars canonical"), codec.VectorAllScalars(s)), + "a changed field no longer matches") + + check(codec.check_vector(find(n, "shape labeled"), + codec.VectorFigure(codec.ShapeLabeled(label="tag", count=3))), + "shape labeled") + check(codec.check_vector(find(n, "string interior nul"), codec.VectorText("nul\0inside\0")), + "string interior nul") + + # Any NaN matches the NaN vector; zero keeps its sign. + nan = struct.unpack(" None: + v: Any = codec.vector(find(n, "i64 past 2^53")) + check(isinstance(v, codec.VectorI64) and v.value == -9007199254740993, "i64 past 2^53") + + v = codec.vector(find(n, "f32 min subnormal")) + check(isinstance(v, codec.VectorF32) and f32_bits(v.value) == b"\x01\0\0\0", + "f32 min subnormal bits") + + v = codec.vector(find(n, "string astral")) + check(v == codec.VectorText("🦀 crab 😀"), f"string astral {v!r}") + + v = codec.vector(find(n, "scalars minimum")) + check(isinstance(v, codec.VectorAllScalars), "scalars minimum variant") + m = v.value + check((m.i8_value, m.u8_value, m.i16_value, m.u16_value, m.i32_value, m.u32_value, + m.i64_value, m.u64_value) == (-128, 0, -32768, 0, -(2**31), 0, INT64_MIN, 0), + f"scalars minimum integers {m!r}") + check(m.f32_value == -math.inf and math.isnan(m.f64_value), "scalars minimum floats") + check(m.color is codec.Color.Infrared and m.flag is False, "scalars minimum enum and flag") + + v = codec.vector(find(n, "composite canonical")) + check(isinstance(v, codec.VectorDeep), "composite canonical variant") + c = v.value + check(c.name == "héllo wörld ✓", "name") + check(c.blob == bytes([0, 1, 2, 253, 254, 255]), "blob") + check(c.some_i64 == INT64_MIN and c.none_i64 is None and c.some_text == "", "optionals") + check(c.names == ["a", "", "ccc"], "names") + check(c.matrix == [[1, 2, 3], [], [-4]], "matrix") + check(len(c.floats) == 6 and math.isnan(c.floats[0]) and c.floats[1:3] == [math.inf, -math.inf] + and f64_bits(c.floats[3]) == f64_bits(-0.0) and c.floats[4] == 5e-324 + and c.floats[5] == 0.1, f"floats {c.floats!r}") + check(c.by_name == {"one": 1, "two": 2, "neg": -3, "": INT64_MAX}, "by_name") + check(sorted(c.by_id) == [-1, 42, 2**31 - 1] and c.by_id[-1] == canonical_scalars(), "by_id") + check(c.by_color == {codec.Color.Infrared: "below", codec.Color.Blue: "sky"}, "by_color") + check(c.flags == {0: False, UINT64_MAX: True}, "flags") + check(c.scalars == canonical_scalars(), "scalars") + check(c.shape == codec.ShapeLabeled(label="tag", count=3), "shape") + check(len(c.shapes) == 6 and isinstance(c.shapes[5], codec.ShapeNested) + and c.shapes[5].note is None, f"shapes {c.shapes!r}") + check(isinstance(c.maybe_shape, codec.ShapeNested) and c.maybe_shape.note is None, + "maybe_shape") + check(c.maybe_list == b"\x09\x08", "maybe_list") + check(c.sparse == [True, None, False], "sparse") + check(c.colors == [codec.Color.Red, codec.Color.Green, codec.Color.Blue, codec.Color.Infrared], + "colors") + check(c.shape.tag is codec.Shape.Tag.Labeled and codec.Shape.Labeled is codec.ShapeLabeled, + "rich enum tag and scoped alias") + + +def out_of_range(n: int) -> None: + exc = consumer.raises(codec.CodecError.OutOfRange, lambda: codec.vector(n), "vector(n)") + check(exc.code == 1 and exc.index == n and exc.count == n, f"OutOfRange payload {exc!r}") + check(exc.message == f"vector {n} is out of range (count {n})", f"message {exc.message!r}") + check(isinstance(exc, codec.CodecError) and isinstance(exc, codec.Error), "hierarchy") + + exc = consumer.raises(codec.OutOfRange, lambda: codec.vector_name(n + 5), "vector_name") + check(exc.index == n + 5 and exc.count == n, "vector_name payload") + + check(not codec.check_vector(n, codec.VectorBlank()), "check_vector past the end") + + +def check_raw(data: bytes) -> int: + """Send `data` to check_vector through its bound C function, returning the + error code.""" + err = impl._ErrorStruct() + impl._c_codec_check_vector(0, data, len(data), ctypes.byref(err)) + code = err.code + impl._error_clear(ctypes.byref(err)) + return code + + +def malformed() -> None: + i32 = struct.Struct(" {back}") - check(same_float(wv.roundtrip_shape(variants[1]).radius, -0.0), "Circle -0.0 radius") - check(wv.roundtrip_shape(variants[2]).height == float("inf"), "Rect inf height") - check(wv.roundtrip_shapes(variants) == variants, "roundtrip_shapes") - check(wv.roundtrip_shapes([]) == [], "roundtrip_shapes([])") - check(wv.describe_shape(wv.Shape.Empty()) == "Empty", "describe Empty") - check(wv.describe_shape(wv.Shape.Circle(2.5)) == "Circle { radius: 2.5 }", "describe Circle") - check(wv.describe_shape(wv.Shape.Rect(1.0, 0.5)) == "Rect { width: 1.0, height: 0.5 }", - "describe Rect") - check(wv.describe_shape(wv.Shape.Labeled("x", 7)) == "Labeled { label: \"x\", count: 7 }", - "describe Labeled") - check(wv.describe_shape(wv.Shape.Nested(canonical_scalars(), None)).startswith( - "Nested { inner: Scalars { i8_value: -8,"), "describe Nested") - check(wv.Shape.Tag.Nested == 4 and wv.Shape.Nested.TAG == wv.Shape.Tag.Nested, "Tag values") - - -def direct_families() -> None: - check(wv.roundtrip_opt_i64(None) is None, "opt_i64 None") - check(wv.roundtrip_opt_i64(0) == 0, "opt_i64 0") - check(wv.roundtrip_opt_i64(I64_MIN) == I64_MIN, "opt_i64 min") - check(wv.roundtrip_opt_i64(I64_MAX) == I64_MAX, "opt_i64 max") - check(wv.roundtrip_map({}) == {}, "map {}") - m = {"": 0, "a": I64_MIN, "ü": I64_MAX, "z" * 300: -1} - check(wv.roundtrip_map(m) == m, "map contents") - check(wv.roundtrip_string("") == "", "string ''") - for s in ("a\0b", "\0", "trailing\0", "\0\0leading"): - check(wv.roundtrip_string(s) == s, f"interior NUL string {s!r}") - check(wv.roundtrip_string("héllo 日本語 🧵") == "héllo 日本語 🧵", "unicode string") - check(wv.roundtrip_bytes(b"") == b"", "bytes b''") - check(wv.roundtrip_bytes(bytes(range(256))) == bytes(range(256)), "all byte values") - for v in (0, 1, -1, I64_MIN, I64_MAX): - check(wv.roundtrip_i64(v) == v, f"i64 {v}") - for v in (0, 1, 1 << 63, U64_MAX): - check(wv.roundtrip_u64(v) == v, f"u64 {v}") - for v in (0.0, -0.0, 1.5, -2.25e100, float("inf"), float("-inf"), float("nan"), - 5e-324, 1.7976931348623157e308): - check(same_float(wv.roundtrip_f64(v), v), f"f64 {v!r} -> {wv.roundtrip_f64(v)!r}") - check(wv.roundtrip_bool(True) is True and wv.roundtrip_bool(False) is False, "bool") - for c in wv.Color: - check(wv.roundtrip_color(c) is c, f"color {c}") - - -def holders() -> None: - holder = wv.make_holder(10, True) - check(isinstance(holder, wv.Holder), "make_holder type") - check(isinstance(holder.primary, wv.Token) and holder.primary.value() == 10, "primary") - check(holder.spare is not None and holder.spare.value() == 11, "spare present") - check([t.value() for t in holder.many] == [12, 13, 14], "many") - check(wv.sum_holder(holder) == 10 + 11 + 12 + 13 + 14, "sum_holder") - # Encoding clones each token, so the wrappers stay usable afterwards and - # the holder can be sent again and again. - check(wv.sum_holder(holder) == 60 and holder.primary.value() == 10, "sum_holder twice") - - # primary_of hands back a wrapper over the very same object as - # holder.primary, which the producer confirms by pointer identity. - primary = wv.primary_of(holder) - check(isinstance(primary, wv.Token) and primary is not holder.primary, "primary_of wrapper") - check(primary.value() == 10, "primary_of value") - check(wv.same_primary(holder, wv.Holder(primary=primary, spare=None, many=[])) is True, - "primary_of is the same object as holder.primary") - check(wv.same_primary(holder, holder) is True, "same_primary(holder, holder)") - check(wv.same_primary(holder, wv.make_holder(10, True)) is False, - "distinct tokens with equal values are not the same object") - primary.close() - primary.close() - check(holder.primary.value() == 10, "holder.primary alive after closing the other wrapper") - - without = wv.make_holder(0, False) - check(without.spare is None, "spare absent") - check(wv.sum_holder(without) == 0 + 2 + 3 + 4, "sum without spare") - - # Consumer-built tokens in every buffered position. - with wv.Token(1) as a, wv.Token(2) as b: - c, d = wv.Token(3), wv.Token(4) - mine = wv.Holder(primary=a, spare=b, many=[c, d, a]) - check(wv.sum_holder(mine) == 1 + 2 + 3 + 4 + 1, "sum of a local holder") - check(wv.sum_holder(wv.Holder(primary=a, spare=None, many=[])) == 1, "minimal holder") - p = wv.primary_of(mine) - check(p.value() == 1 and wv.same_primary(mine, wv.Holder(p, None, [])) is True, - "primary_of a local holder") - check(wv.same_primary(mine, wv.Holder(b, None, [])) is False, "different primary") - p.close() - c.close() - d.close() - c.close() - # A closed token anywhere in the holder rejects the whole encoding - # before any reference is minted, so the live tokens leak nothing. - for broken in (wv.Holder(primary=c, spare=None, many=[]), - wv.Holder(primary=a, spare=b, many=[a, d])): - try: - wv.sum_holder(broken) - check(False, "expected error encoding a closed token") - except wv.Error as exc: - check("Token used after close" in exc.message, - f"closed token message {exc.message!r}") - check(a.value() == 1, "a still alive inside the with block") - try: - a.value() - check(False, "expected use-after-close error") - except wv.Error: - pass - - # Release every producer-built wrapper explicitly (twice), then let the - # rest go to the garbage collector. - for t in [holder.primary, holder.spare, without.primary] + holder.many + without.many: - t.close() - t.close() - del holder, without, primary - gc.collect() + for what, data in bad: + check(check_raw(data) == -3, f"producer rejects {what}") + # The generated decoder rejects the same input as a trap. + exc = consumer.raises(codec.InternalError, lambda: impl._decode(data, impl._read_Vector), + f"decoder rejects {what}") + check(exc.code == -3, f"decoder code for {what}") + + # An undeclared C-style enum value in a direct slot is a marshalling + # failure; echo_color declares no errors, so it traps. + exc = consumer.raises(codec.InternalError, lambda: codec.echo_color(3), "echo_color(3)") + check(exc.code == -3 and isinstance(exc, RuntimeError), f"echo_color trap {exc!r}") + + +def objects(n: int) -> None: + v: Any = codec.vector(find(n, "objects full")) + check(isinstance(v, codec.VectorObjects), "objects full variant") + h = v.value + check(h.primary.value() == 10 and h.spare is not None and h.spare.value() == 11, "tokens") + check([t.value() for t in h.many] == [12, 13, INT64_MIN], "many") + check({k: t.value() for k, t in h.by_name.items()} == {"a": 20, "b": 21}, "by_name") + + # Each encoding mints fresh references, so the holder can be sent twice. + expected = (10 + 11 + 12 + 13 + 20 + 21 + INT64_MIN + 2**63) % 2**64 - 2**63 + check(codec.sum_holder(h) == expected and codec.sum_holder(h) == expected, "sum_holder twice") + + # primary_of returns the same native object (a new wrapper over it). + p = codec.primary_of(h) + check(p == h.primary and p is not h.primary, "primary_of is the same object") + check(codec.same_primary(h, codec.Holder(primary=p, spare=None, many=[], by_name={})), + "same_primary by identity") + twin = codec.Token(10) + check(twin != h.primary, "an equal value is a different object") + check(not codec.same_primary(h, codec.Holder(primary=twin, spare=None, many=[], by_name={})), + "same_primary is identity, not value") + + # A holder built from consumer-made tokens, one wrapper in every slot. + minus = codec.Token(-4) + mine = codec.Holder(primary=twin, spare=twin, many=[twin, twin, minus], by_name={"k": twin}) + check(codec.sum_holder(mine) == 10 * 5 - 4, "consumer-built holder") + + # A closed wrapper can't be encoded. + minus.close() + consumer.raises(ValueError, lambda: codec.sum_holder(mine), "closed token") + twin.close() + twin.close() + p.close() def main() -> None: - scalars() - composite() - shapes() - direct_families() - holders() - consumer.finish(wv) + n = codec.vector_count() + check(n >= 60, f"vector_count {n}") + every_vector(n) + literal_vectors(n) + spot_checks(n) + out_of_range(n) + malformed() + objects(n) + consumer.finish() main() diff --git a/conformance/python/contacts_consumer.py b/conformance/python/contacts_consumer.py deleted file mode 100644 index dac73433..00000000 --- a/conformance/python/contacts_consumer.py +++ /dev/null @@ -1,132 +0,0 @@ -"""Conformance consumer: contacts sample, Python target. - -Exercises the generated ctypes wrapper end to end: the reference-counted -`ContactBook` interface (its `__init__` calls the C constructor; `close()`, -the `with` statement, or `__del__` release the reference), enum marshalling, -the `Contact` record decoded from a value buffer into a plain dataclass with -value equality, optional strings (a buffered `string?` parameter and field), -list-of-record returns, boolean returns, the typed `ContactsError` -subclasses raised by throwing methods, and cross-module types: the nested -`contacts.groups` module taking and returning the parent's `ContactBook`, -`ContactType`, and `Contact`, and the sibling `directory` module taking a -`Contact` record and raising its own `DirectoryError` domain. Ends with the -leak check (see harness.py). -""" -import contacts as wv -from harness import Consumer - -consumer = Consumer("contacts") -check = consumer.check - - -def book_basics() -> None: - book = wv.ContactBook() - check(book.count() == 0, "fresh book is empty") - - alice = book.add("Alice", "Smith", "alice@example.com", wv.ContactType.Work) - check(alice.id > 0 and alice.first_name == "Alice" and alice.last_name == "Smith", - f"alice {alice}") - check(alice.email == "alice@example.com" and alice.contact_type == wv.ContactType.Work, - f"alice email and type {alice}") - - # Optional string: a missing email round-trips as None. Records are - # plain dataclass values, so a fetched contact compares equal to a - # locally constructed one field by field. - bob = book.add("Bob", "Jones", None, wv.ContactType.Personal) - fetched = book.get(bob.id) - check(fetched == wv.Contact(id=bob.id, first_name="Bob", last_name="Jones", email=None, - contact_type=wv.ContactType.Personal), f"fetched {fetched}") - # Strings cross as pointer + length, so an interior NUL survives. - nul = book.add("Nul\0Name", "Tail\0", "a\0b", wv.ContactType.Other) - check(book.get(nul.id).first_name == "Nul\0Name" and nul.email == "a\0b", - f"interior NUL round trip {nul}") - check(book.remove(nul.id) is True, "remove nul") - - check(book.count() == 2, "two contacts") - everyone = book.list() - check({p.first_name for p in everyone} == {"Alice", "Bob"}, f"list {everyone}") - - # Typed errors carry stable codes and the class hierarchy. - try: - book.add("", "Smith", None, wv.ContactType.Personal) - check(False, "expected InvalidName for an empty first name") - except wv.ContactsError.InvalidName as exc: - check(exc.code == 1 and isinstance(exc, wv.ContactsError) and isinstance(exc, wv.Error), - f"InvalidName {exc!r}") - check(exc.message == "name must not be empty", f"InvalidName message {exc.message!r}") - check(book.count() == 2, "failed add left the book unchanged") - - check(book.remove(alice.id) is True, "remove alice") - check(book.remove(alice.id) is False, "remove alice twice") - check(book.count() == 1, "one contact left") - - try: - book.get(9999) - check(False, "expected NotFound for a missing contact") - except wv.NotFound as exc: - check(exc.code == 2 and isinstance(exc, wv.ContactsError), f"NotFound {exc!r}") - check(wv.NotFound is wv.ContactsError.NotFound, "bare NotFound is the scoped alias") - - # Objects are context managers with an idempotent close(); a closed - # wrapper rejects further use. - with wv.ContactBook() as other: - check(other.count() == 0, "second book is independent") - other.close() - other.close() - try: - other.count() - check(False, "expected a use-after-close error") - except wv.Error as exc: - check("after close" in exc.message, f"use-after-close message {exc.message!r}") - book.close() - - -def nested_and_sibling_modules() -> None: - book = wv.ContactBook() - work = book.add("Wanda", "Work", None, wv.ContactType.Work) - book.add("Walt", "Work", "w@example.com", wv.ContactType.Work) - book.add("Pat", "Home", None, wv.ContactType.Personal) - - # contacts.groups takes the parent module's interface and enum. - check(wv.count_of_type(book, wv.ContactType.Work) == 2, "count_of_type Work") - check(wv.count_of_type(book, wv.ContactType.Other) == 0, "count_of_type Other") - dominant = wv.dominant_type(book) - check(dominant is wv.ContactType.Work, f"dominant_type {dominant!r}") - - # ... and returns the parent's interface and record. - split = wv.split_by_type(book, wv.ContactType.Work) - check(isinstance(split, wv.ContactBook) and split.count() == 2, "split_by_type") - check(book.count() == 3, "split_by_type copies instead of moving") - first = wv.first_of_type(book, wv.ContactType.Work) - check(first == work, f"first_of_type {first}") - try: - wv.first_of_type(split, wv.ContactType.Personal) - check(False, "expected NotFound from the nested module") - except wv.ContactsError.NotFound as exc: - check(exc.code == 2, f"nested NotFound {exc!r}") - split.close() - - # The sibling directory module takes a contacts record by value. - card = wv.card(first) - check(card == wv.Card(display_name="Work, Wanda", initials="WW", has_email=False), - f"card {card}") - ordered = wv.sorted(book.list()) - check([c.first_name for c in ordered] == ["Pat", "Walt", "Wanda"], - f"sorted {[c.first_name for c in ordered]}") - try: - wv.sorted([]) - check(False, "expected DirectoryError.Empty") - except wv.DirectoryError.Empty as exc: - check(exc.code == 1 and not isinstance(exc, wv.ContactsError), - f"Empty is its own domain {exc!r}") - check(exc.message == "no contacts to list", f"Empty message {exc.message!r}") - book.close() - - -def main() -> None: - book_basics() - nested_and_sibling_modules() - consumer.finish(wv) - - -main() diff --git a/conformance/python/events_consumer.py b/conformance/python/events_consumer.py deleted file mode 100644 index 047231fc..00000000 --- a/conformance/python/events_consumer.py +++ /dev/null @@ -1,292 +0,0 @@ -"""Conformance consumer: events sample, Python target. - -Exercises the ABI 3 events surface through the generated ctypes wrapper: the -`Subscriber` callback interface implemented by subclassing the generated ABC -(every method is called with the right arguments, the `EventBus` handed to -`on_attached` is a usable wrapper that owns its own strong reference), the -`Delivery` return of `route` (a `Result`-returning method on the producer -side, so a raising implementation reaches the producer as a `ForeignError` -value) steering `publish`'s accepted count, the reference-counted -`EventBus` object (`with` statement, idempotent `close()`, use after close -rejected), the asyncio-bridged `publish_later` method, the `messages()` -iterator, the `last_message()` optional record, `route_once` releasing a -subscriber it doesn't retain, and a subscriber that raises a Python exception -surfacing to the caller as `Error` with `FOREIGN_ERROR_CODE` (-4) without -taking the interpreter down, and subscribers called from a producer thread -other than the one Python called in on. Ends with the leak check (see -harness.py). -""" -import asyncio -import gc -import threading -import weakref - -import events as wv -from harness import Consumer - -consumer = Consumer("events") -check = consumer.check - - -class Recorder(wv.Subscriber): - """A subscriber that records every callback it receives. - - `skip_topic` is routed Skip, `stop_topic` AcceptAndStop, `fail_topic` - raises from `route`, `fail_text` raises from `on_message`, and - everything else is Accept. The bus received in `on_attached` is kept so - the consumer can prove it stays alive independently of the caller's - wrapper. - """ - - def __init__(self, name: str, skip_topic: str = "", stop_topic: str = "", - fail_topic: str = "", fail_text: str = "") -> None: - self.name = name - self.skip_topic = skip_topic - self.stop_topic = stop_topic - self.fail_topic = fail_topic - self.fail_text = fail_text - self.routed: list[str] = [] - self.messages: list[wv.Message] = [] - self.attached: list[wv.EventBus] = [] - self.attach_counts: list[int] = [] - - def route(self, topic: str) -> wv.Delivery: - self.routed.append(topic) - if topic == self.fail_topic: - raise ValueError(f"{self.name} rejected topic {topic}") - if topic == self.skip_topic: - return wv.Delivery.Skip - if topic == self.stop_topic: - return wv.Delivery.AcceptAndStop - return wv.Delivery.Accept - - def on_message(self, message: wv.Message) -> int: - if message.text == self.fail_text: - raise RuntimeError(f"{self.name} choked on {message.text}") - self.messages.append(message) - return len(self.messages) - - def on_attached(self, bus: wv.EventBus) -> None: - # The wrapper is usable right away: subscribe has not pushed us yet. - self.attach_counts.append(bus.subscriber_count()) - self.attached.append(bus) - - -class Exploding(wv.Subscriber): - """A subscriber whose `on_attached` raises, so `subscribe` itself fails.""" - - def route(self, topic: str) -> wv.Delivery: - return wv.Delivery.Accept - - def on_message(self, message: wv.Message) -> int: - return 0 - - def on_attached(self, bus: wv.EventBus) -> None: - bus.close() - raise KeyError("refusing to attach") - - -def main() -> None: - check(wv.Delivery.Accept == 0 and wv.Delivery.Skip == 1 - and wv.Delivery.AcceptAndStop == 2, "Delivery discriminants") - - with wv.EventBus() as bus: - check(bus.subscriber_count() == 0, "fresh bus has no subscribers") - check(bus.last_message() is None, "fresh bus has no last message") - check(list(bus.messages()) == [], "fresh bus has no messages") - - quiet = Recorder("quiet", skip_topic="quiet") - loud = Recorder("loud", stop_topic="stop") - tail = Recorder("tail") - - check(bus.subscribe(quiet) == 1, "first subscribe returns 1") - check(bus.subscribe(loud) == 2, "second subscribe returns 2") - check(bus.subscribe(tail) == 3, "third subscribe returns 3") - check(bus.subscriber_count() == 3, "subscriber_count after three subscribes") - - # on_attached fired once per subscriber, before the push, with a - # live bus wrapper. - check(quiet.attach_counts == [0], f"quiet attach count {quiet.attach_counts}") - check(loud.attach_counts == [1], f"loud attach count {loud.attach_counts}") - check(tail.attach_counts == [2], f"tail attach count {tail.attach_counts}") - check(len(quiet.attached) == 1, "quiet kept its attached bus") - check(isinstance(quiet.attached[0], wv.EventBus), - "on_attached receives an EventBus wrapper") - # The adopted reference is independent of the caller's wrapper: it is - # a different Python object on the same underlying bus. - check(quiet.attached[0] is not bus, "attached wrapper is a distinct object") - check(quiet.attached[0].subscriber_count() == 3, - "attached bus wrapper observes the shared state") - - # Accept everywhere: every subscriber gets it. - n = bus.publish("news", "hello", ["a", "b"]) - check(n == 3, f"publish news accepted {n}") - check(quiet.routed == ["news"] and loud.routed == ["news"] - and tail.routed == ["news"], "route called on each subscriber") - for sub in (quiet, loud, tail): - check(len(sub.messages) == 1, f"{sub.name} received one message") - m = sub.messages[0] - check(m == wv.Message(seq=1, topic="news", text="hello", tags=["a", "b"]), - f"{sub.name} message {m}") - - # Skip steers the count: quiet drops its own topic. - n = bus.publish("quiet", "psst", []) - check(n == 2, f"publish quiet accepted {n}") - check(len(quiet.messages) == 1, "quiet skipped its topic") - check(loud.messages[-1].seq == 2 and loud.messages[-1].tags == [], - "seq advances and empty tags round-trip") - - # AcceptAndStop: loud accepts and stops delivery to tail. - n = bus.publish("stop", "last", ["z"]) - check(n == 2, f"publish stop accepted {n}") - check(len(quiet.messages) == 2, "quiet accepted stop") - check(len(loud.messages) == 3, "loud accepted stop") - check(len(tail.messages) == 2, "tail was not reached after AcceptAndStop") - check(tail.routed == ["news", "quiet"], f"tail routed {tail.routed}") - - # Async publish: resolved from the producer's thread through asyncio. - n = asyncio.run(bus.publish_later("later", "async")) - check(n == 3, f"publish_later accepted {n}") - check(tail.messages[-1] == wv.Message(4, "later", "async", []), - f"async message {tail.messages[-1]}") - - # Iterator and optional record. - check(list(bus.messages()) == ["hello", "psst", "last", "async"], - "messages() streams every text in order") - it = bus.messages() - check(next(it) == "hello", "manual next on iterator") - it.close() - it.close() - last = bus.last_message() - check(last == wv.Message(seq=4, topic="later", text="async", tags=[]), - f"last_message {last}") - - # A subscriber that raises from route (the Result-returning method) - # aborts publish with the foreign error code, and the bus is still - # usable afterwards. - grumpy = Recorder("grumpy", fail_topic="boom", fail_text="gag") - check(bus.subscribe(grumpy) == 4, "grumpy subscribed") - try: - bus.publish("boom", "x", []) - check(False, "expected Error from a raising route") - except wv.Error as exc: - check(exc.code == wv.Error.FOREIGN_ERROR_CODE == -4, - f"foreign error code {exc.code}") - check("grumpy rejected topic boom" in exc.message, - f"foreign error message {exc.message!r}") - # Earlier subscribers were already delivered before grumpy raised. - check(len(quiet.messages) == 4, "quiet got the aborted message first") - # A raise from on_message is reported the same way. - try: - bus.publish("ok", "gag", []) - check(False, "expected Error from a raising on_message") - except wv.Error as exc: - check(exc.code == -4, f"on_message foreign code {exc.code}") - check("choked on gag" in exc.message, f"on_message message {exc.message!r}") - # The log still recorded both aborted publishes. - check(bus.last_message().text == "gag" and bus.last_message().seq == 6, - "aborted publishes are still logged") - n = bus.publish("ok", "recovered", []) - check(n == 4, f"bus usable after foreign error, accepted {n}") - check(grumpy.messages[-1].text == "recovered", "grumpy delivered after recovering") - - # A raise from on_attached fails subscribe itself and leaves the - # subscriber unretained (its free runs during the abort). - exploding = Exploding() - exploding_ref = weakref.ref(exploding) - try: - bus.subscribe(exploding) - check(False, "expected Error from a raising on_attached") - except wv.Error as exc: - check(exc.code == -4, f"on_attached foreign code {exc.code}") - check("refusing to attach" in exc.message, f"on_attached message {exc.message!r}") - check(bus.subscriber_count() == 4, "failed subscribe did not retain") - del exploding - gc.collect() - check(exploding_ref() is None, "unretained subscriber released after failed subscribe") - - # route_once does not retain the subscriber: its free runs before the - # call returns, so nothing but our local keeps it alive. - probe = Recorder("probe", skip_topic="quiet") - probe_ref = weakref.ref(probe) - check(wv.route_once(probe, "quiet") == wv.Delivery.Skip, "route_once Skip") - check(wv.route_once(probe, "other") == wv.Delivery.Accept, "route_once Accept") - check(probe.routed == ["quiet", "other"], f"route_once routed {probe.routed}") - check(probe.attached == [], "route_once never attaches") - del probe - gc.collect() - check(probe_ref() is None, "route_once released its subscriber") - try: - wv.route_once(Recorder("x", fail_topic="bad"), "bad") - check(False, "expected Error from route_once") - except wv.Error as exc: - check(exc.code == -4, f"route_once foreign code {exc.code}") - - # Clearing the subscribers runs each free: once the consumer drops its - # own references, nothing keeps the implementations alive. - refs = [weakref.ref(s) for s in (quiet, loud, tail, grumpy)] - kept_bus = quiet.attached[0] - del quiet, loud, tail, grumpy, sub - gc.collect() - check(all(r() is not None for r in refs), "retained subscribers stay alive") - bus.clear_subscribers() - check(bus.subscriber_count() == 0, "clear_subscribers empties the bus") - gc.collect() - check(all(r() is None for r in refs), "cleared subscribers were freed") - - # The bus reference adopted through on_attached outlives the caller's - # wrapper: closing ours leaves the object alive through the other. - bus.close() - bus.close() - try: - bus.subscriber_count() - check(False, "expected use-after-close error") - except wv.Error as exc: - check("after close" in exc.message, f"use-after-close message {exc.message!r}") - check(list(kept_bus.messages()) == ["hello", "psst", "last", "async", "x", "gag", "recovered"], - "bus alive through the callback-adopted reference") - kept_bus.close() - kept_bus.close() - # Leaving the with block closes an already-closed bus: a no-op. - - # A fresh bus is independent state. - other = wv.EventBus() - check(other.subscriber_count() == 0 and other.last_message() is None, - "second bus is independent") - other.close() - - foreign_threads() - consumer.finish(wv) - - -def foreign_threads() -> None: - # Python threads publishing concurrently: each publish calls back into - # the subscribers from that thread while the others hold or wait for - # the GIL, and the async completion arrives on a producer thread. - bus = wv.EventBus() - sub = Recorder("threads") - bus.subscribe(sub) - errors = [] - - def publish(n: int) -> None: - try: - for i in range(25): - bus.publish(f"t{n}", f"{n}-{i}", []) - except BaseException as exc: - errors.append(exc) - - threads = [threading.Thread(target=publish, args=(n,)) for n in range(4)] - for t in threads: - t.start() - for t in threads: - t.join(30) - check(not errors, f"publishing threads failed: {errors}") - check(len(sub.messages) == 100, f"threads delivered {len(sub.messages)}") - check(asyncio.run(bus.publish_later("async", "x")) == 1, "publish_later after threads") - bus.clear_subscribers() - for b in sub.attached: - b.close() - bus.close() - - -main() diff --git a/conformance/python/harness.py b/conformance/python/harness.py index 953ab13a..dca55da3 100644 --- a/conformance/python/harness.py +++ b/conformance/python/harness.py @@ -2,18 +2,21 @@ Every consumer imports the generated package as-is (conformance/python/run.sh puts it on PYTHONPATH and selects the producer library through the loader's -`{PREFIX}_LIBRARY` override). This module adds the assertion helper, a hook +`{PREFIX}_LIBRARY` override). This module adds the assertion helpers, a hook that turns any exception Python could only report as "unraisable" (raised in a ctypes callback or a finalizer) into a failure, and the end-of-run leak -check against the producer's live-allocation counters. +check against the producer's live-resource counters. """ import gc import sys +import time import types -from typing import Any, List +from typing import Any, Callable, List, Type, TypeVar _unraisable: List[str] = [] +E = TypeVar("E", bound=BaseException) + def _record_unraisable(info: Any) -> None: _unraisable.append(f"{info.exc_type.__name__}: {info.exc_value} ({info.object!r})") @@ -25,25 +28,42 @@ def _record_unraisable(info: Any) -> None: class Consumer: """Assertions for one consumer, labelled `python/`.""" - def __init__(self, name: str) -> None: + def __init__(self, name: str, package: types.ModuleType) -> None: self.name = name + self.impl = sys.modules[f"{package.__name__}.{package.__name__}"] + self.check(self.impl._debug_live(-1) == 1, "the producer counts live resources") def check(self, cond: bool, what: str) -> None: if not cond: print(f"python/{self.name}: FAIL: {what}", file=sys.stderr) sys.exit(1) - def finish(self, package: types.ModuleType) -> None: + def raises(self, cls: Type[E], fn: Callable[[], Any], what: str) -> E: + """Call `fn`, require it to raise `cls`, and return the exception.""" + try: + fn() + except cls as exc: + return exc + except BaseException as exc: # noqa: BLE001 + self.check(False, f"{what}: expected {cls.__name__}, got {exc!r}") + self.check(False, f"{what}: expected {cls.__name__}, nothing raised") + raise AssertionError # unreachable + + def finish(self) -> None: """Collect garbage (running every finalizer), then require that the producer holds no live object, callback, iterator, cancel token, or - returned allocation, and that nothing was raised where Python could - not propagate it.""" - for _ in range(3): + byte run, and that nothing was raised where Python could not + propagate it. A producer worker may still be dropping a finished + async call, so the counters get up to two seconds to settle.""" + kinds = ["objects", "callbacks", "iterators", "cancel tokens", "byte runs"] + deadline = time.monotonic() + 2.0 + while True: gc.collect() - impl = sys.modules[f"{package.__name__}.{package.__name__}"] - kinds = ["objects", "callbacks", "iterators", "cancel tokens", "allocations"] - for kind, label in enumerate(kinds): - live = impl._debug_live(kind) - self.check(live == 0, f"{live} live {label} at exit") + live = [self.impl._debug_live(kind) for kind in range(len(kinds))] + if not any(live) or time.monotonic() > deadline: + break + time.sleep(0.001) + for label, count in zip(kinds, live): + self.check(count == 0, f"{count} live {label} at exit") self.check(not _unraisable, f"unraisable exceptions: {_unraisable}") print(f"python/{self.name}: OK") diff --git a/conformance/python/kvstore_consumer.py b/conformance/python/kvstore_consumer.py index d2c49baf..1ff6322c 100644 --- a/conformance/python/kvstore_consumer.py +++ b/conformance/python/kvstore_consumer.py @@ -1,385 +1,515 @@ -"""Conformance consumer: kvstore sample, Python target. - -Full-surface drive of the generated ctypes wrapper against ABI 3: the -reference-counted `Store` interface (fallible `open` factory, `with` -statement, idempotent `close()`, use after close rejected, the -`default_capacity` static, the deprecated `legacy_put`), typed `KvError` -subclasses raised by throwing callables, the optional-record return -(`Entry | None`) decoded from a value buffer, buffered `i64?` and `string?` -parameters, the iterator-backed `list_keys`, the cross-module `get_stats`, -the `EvictionListener` callback interface implemented by subclassing the -generated ABC (fired on delete and on expired reads, detached by returning -False, replaced and cleared, a raising listener surfacing as -`FOREIGN_ERROR_CODE`), objects in every buffered position (`share()` as a -second wrapper over the same object, `fork()`, `larger(None)` and -`larger(other)`, `describe().store`, `open_many`, `total_count` with a -locally built `StoreInfo`), the asyncio-bridged cancellable `compact` -coroutine, and wrappers that stay usable while another thread closes them -mid-call. Ends with the leak check (see harness.py). +"""Conformance consumer: kvstore sample, Python target (ABI revision 4). + +Drives the feature-complete producer through the generated package: + + * the load-time checks (importing checked the ABI revision and both + modules' contract tables; a doctored table is refused by name); + * the `Store` interface: fallible and infallible constructors, methods, + statics, the deprecated `size`, records (`Entry`, `StoreInfo`), the + `EntryKind` enum, maps and optionals, and the logical clock; + * `KvError` codes with their payload fields (`KeyNotFound`, `Expired`, + `StoreFull`, `Rejected`) and runtime codes on throwing calls; + * lazy iterators of strings (throwing), records, and objects; + * three callback interfaces implemented by subclassing the generated + ABCs: a `Listener` (retained, filtered by `accepts`, told about every + `Change`, detached when it raises, and notified from a producer thread + during compaction), a `Policy` (a record return, a typed `KvError` raised + back through `put` with its fields, an object parameter and object + return), and a `Loader` passed as an optional callback (string, bytes, + and optional-object returns; typed errors decoded by the producer or + passed through); + * `Store` objects in every position, compared by native identity; + * coroutines: an async free function returning an object, a cancellable + method cancelled mid-pause, an async list, an async function in the + nested `kv.stats` module, and concurrent calls; + * the nested `kv.stats` module and the sibling `report` root. + +Ends with every callback released and the leak check (see harness.py). """ import asyncio -import gc import threading +import time import warnings -import weakref +from typing import List, Optional -import kvstore as wv +import kvstore as kv from harness import Consumer -PATH = "/tmp/conformance-kvstore-py" -consumer = Consumer("kvstore") +consumer = Consumer("kvstore", kv) check = consumer.check +raises = consumer.raises +impl = consumer.impl +MAIN_THREAD = threading.get_ident() +P = kv.EntryKind.Persistent +V = kv.EntryKind.Volatile +E = kv.EntryKind.Encrypted -class Listener(wv.EvictionListener): - """Records `(key, reason)` pairs; detaches itself after `keep` events and - raises when it sees `poison` as the evicted key.""" - def __init__(self, keep: int = 1 << 30, poison: str = "") -> None: - self.keep = keep - self.poison = poison - self.seen: list[tuple[str, wv.EvictionReason]] = [] - self.entries: list[wv.Entry] = [] +def held_callbacks() -> int: + """How many callback implementations the producer currently holds.""" + return len(impl._callbacks) - def on_evict(self, entry: wv.Entry, reason: wv.EvictionReason) -> bool: - if entry.key == self.poison: - raise RuntimeError(f"listener refused {entry.key}") - self.seen.append((entry.key, reason)) - self.entries.append(entry) - return len(self.seen) < self.keep +def put(store: kv.Store, key: str, value: str, kind: kv.EntryKind = P, + ttl: Optional[int] = None) -> kv.Entry: + return store.put(key, value.encode(), kind, ttl) -def expect_kv_error(fn, cls, code: int, what: str) -> wv.KvError: - try: - fn() - except cls as exc: - check(exc.code == code and exc.CODE == code, f"{what}: code {exc.code}") - check(isinstance(exc, wv.KvError) and isinstance(exc, wv.Error), - f"{what}: hierarchy") - return exc - check(False, f"{what}: expected {cls.__name__}") - raise AssertionError # unreachable +def expect_key_not_found(fn, key: str, what: str) -> kv.KeyNotFound: + exc = raises(kv.KvError.KeyNotFound, fn, what) + check(exc.code == 1001 and exc.key == key, f"{what}: KeyNotFound {exc.key!r}") + return exc -def basics() -> None: - # Fallible constructor: an empty path reports the IoError domain code. - expect_kv_error(lambda: wv.Store.open(""), wv.KvError.IoError, 1004, "open('')") - check(wv.IoError is wv.KvError.IoError, "bare IoError is the scoped alias") - try: - wv.Store() - check(False, "Store() must not construct") - except TypeError: - pass - check(wv.Store.default_capacity() == 1_000_000, "default_capacity static") - - with wv.Store.open(PATH) as store: - payload = b"\x01\x02\x03" - check(store.put("alpha", payload, wv.EntryKind.Persistent, None) is True, "put alpha") - check(store.put("beta", payload, wv.EntryKind.Volatile, 3600) is True, "put beta") - check(store.count() == 2, "count after two puts") - - # Iterator-backed list-of-string return with an absent and a present - # buffered `string?` prefix. - check(list(store.list_keys(None)) == ["alpha", "beta"], "list_keys(None)") - check(list(store.list_keys("al")) == ["alpha"], "list_keys('al')") - it = store.list_keys(None) - check(next(it) == "alpha", "manual next on key iterator") - it.close() - it.close() - - # Optional-record return decoded into an Entry dataclass. - alpha = store.get("alpha") - check(alpha is not None and alpha.id > 0 and alpha.key == "alpha", f"alpha {alpha}") - check(alpha.value == payload and alpha.expires_at is None, "alpha payload / no expiry") - check(alpha.tags == [] and alpha.metadata == {}, "alpha empty tags / metadata") - beta = store.get("beta") - check(beta is not None and beta.expires_at is not None - and beta.expires_at > beta.created_at, "beta expiry set") - - # Typed errors carry stable codes and the class hierarchy. - exc = expect_kv_error(lambda: store.get("missing"), wv.KvError.KeyNotFound, 1001, - "get missing") - check(exc.message == "key not found", f"KeyNotFound message {exc.message!r}") - check(wv.KeyNotFound is wv.KvError.KeyNotFound, "bare KeyNotFound is the scoped alias") - check(store.put("gone", b"x", wv.EntryKind.Volatile, -1) is True, "put expired") - expect_kv_error(lambda: store.get("gone"), wv.KvError.Expired, 1002, "get expired") - check(store.count() == 2, "expired entry evicted on read") - - # Deprecated method still works but warns. - with warnings.catch_warnings(record=True) as caught: - warnings.simplefilter("always") - check(store.legacy_put("legacy", b"zz") is True, "legacy_put") - check(any(issubclass(w.category, DeprecationWarning) for w in caught), - "legacy_put warns DeprecationWarning") - check(store.delete("legacy") is True, "delete legacy") - check(store.delete("legacy") is False, "delete missing is False") - - # Records are plain value types with native list/map/optional fields. - built = wv.Entry(id=7, key="built", value=payload, created_at=1000, expires_at=None, - tags=["hot", "fast"], metadata={"source": "test", "env": "prod"}) - check(built == wv.Entry(7, "built", payload, 1000, None, ["hot", "fast"], - {"source": "test", "env": "prod"}), "Entry value equality") - - # Cross-module call: kv.stats.get_stats takes the parent's Store. - stats = wv.get_stats(store) - check(stats == wv.Stats(total_entries=2, total_bytes=6, expired_entries=0), - f"stats {stats}") - - # Async: an immediately expired entry gives compact 3 bytes to reclaim. - check(store.put("doomed", payload, wv.EntryKind.Volatile, 0) is True, "put doomed") - check(store.count() == 2, "doomed is already expired") - reclaimed = asyncio.run(store.compact()) - check(reclaimed == 3, f"compact reclaimed {reclaimed}") - check(asyncio.run(store.compact()) == 0, "second compact reclaims nothing") - store.clear() - check(store.count() == 0, "clear empties the store") - closed = store - # __exit__ closed the wrapper; every further use is rejected, and both - # a second close and the eventual __del__ are no-ops. - closed.close() +def load_checks() -> None: + # A table entry the library lacks, or one whose signature changed, is + # refused by its path. + saved = dict(impl._CONTRACTS) + symbol, entries = next(iter(saved.items())) try: - closed.count() - check(False, "expected use-after-close error") - except wv.Error as exc: - check("after close" in exc.message, f"use-after-close message {exc.message!r}") - try: - wv.get_stats(closed) - check(False, "expected use-after-close error as a parameter") - except wv.Error: - pass + impl._CONTRACTS[symbol] = entries + [(1, 2, "kv.Store.vanished")] + exc = raises(ImportError, impl._check_contract, "missing entry") + check(str(exc).startswith("kv.Store.vanished is missing from the library"), str(exc)) + entry_id, entry_hash, path = entries[0] + impl._CONTRACTS[symbol] = [(entry_id, entry_hash ^ 1, path)] + exc = raises(ImportError, impl._check_contract, "changed entry") + check(str(exc).startswith(f"{path} changed since these bindings were generated"), + str(exc)) + finally: + impl._CONTRACTS.clear() + impl._CONTRACTS.update(saved) + impl._check_contract() + + +def constructors() -> None: + exc = raises(kv.KvError.InvalidPath, lambda: kv.Store.open(""), "open('')") + check(exc.code == 1004 and exc.message == "invalid path", f"InvalidPath {exc!r}") + check(isinstance(exc, kv.KvError) and isinstance(exc, kv.Error), "error hierarchy") + + with kv.Store() as s: + check(s.path() == "memory", "Store() path") + check(s.capacity() == kv.Store.default_capacity() == 1_000_000, "capacity") + raises(ValueError, s.path, "use after close") + + async def run() -> None: + opened = await kv.open_store("/async") + check(opened.path() == "/async", "open_store path") + opened.close() + try: + await kv.open_store("") + check(False, "open_store('') returned") + except kv.InvalidPath as exc: + check(exc.code == 1004 and exc.message == "invalid path", f"open_store {exc!r}") + asyncio.run(run()) -def eviction_listener() -> None: - store = wv.Store.open(PATH) - store.put("a", b"1", wv.EntryKind.Persistent, None) - store.put("b", b"22", wv.EntryKind.Persistent, None) - store.put("old", b"333", wv.EntryKind.Volatile, -5) - - # Delete fires the trampoline synchronously with the removed Entry. - first = Listener(keep=2) - first_ref = weakref.ref(first) - store.set_eviction_listener(first) - check(store.delete("a") is True, "delete a") - check(first.seen == [("a", wv.EvictionReason.Deleted)], f"first saw {first.seen}") - check(first.entries[0].value == b"1" and first.entries[0].id > 0, "evicted Entry decoded") - - # An expired read evicts with reason Expired; that was the second event, - # so the listener asked to detach and the store freed it. - expect_kv_error(lambda: store.get("old"), wv.KvError.Expired, 1002, "expired read") - check(first.seen == [("a", wv.EvictionReason.Deleted), ("old", wv.EvictionReason.Expired)], - f"first saw {first.seen}") - del first - gc.collect() - check(first_ref() is None, "detached listener was freed by the producer") - check(store.delete("b") is True, "delete b with no listener") - - # Replacing a listener frees the previous one; clearing frees the current. - second = Listener() - second_ref = weakref.ref(second) - store.set_eviction_listener(second) - third = Listener() - third_ref = weakref.ref(third) - store.set_eviction_listener(third) - del second - gc.collect() - check(second_ref() is None, "replaced listener was freed") - store.put("c", b"3", wv.EntryKind.Persistent, None) - check(store.delete("c") is True, "delete c") - check(third.seen == [("c", wv.EvictionReason.Deleted)], f"third saw {third.seen}") - store.clear_eviction_listener() - del third - gc.collect() - check(third_ref() is None, "cleared listener was freed") - - # A listener that raises aborts the delete with the foreign error code; - # the entry is gone regardless and the store stays usable. - angry = Listener(poison="bad") - store.set_eviction_listener(angry) - store.put("bad", b"!", wv.EntryKind.Persistent, None) - store.put("fine", b"?", wv.EntryKind.Persistent, None) - try: - store.delete("bad") - check(False, "expected Error from a raising listener") - except wv.KvError: - check(False, "a foreign error is not a KvError") - except wv.Error as exc: - check(exc.code == wv.Error.FOREIGN_ERROR_CODE == -4, - f"foreign error code {exc.code}") - check("listener refused bad" in exc.message, f"foreign message {exc.message!r}") - check(store.count() == 1, "entry removed despite the listener raising") - check(store.delete("fine") is True, "delete after foreign error") - check(angry.seen == [("fine", wv.EvictionReason.Deleted)], f"angry saw {angry.seen}") - - # Destroying the store frees the listener it still holds. - angry_ref = weakref.ref(angry) - del angry - gc.collect() - check(angry_ref() is not None, "attached listener kept alive by the store") - store.close() - gc.collect() - check(angry_ref() is None, "closing the store freed its listener") +def basics() -> None: + s = kv.Store.open("/basics") + e = put(s, "alpha", "one") + check(e == kv.Entry(key="alpha", value=b"one", kind=P, version=1, expires_at=None, tags=[], + metadata={}), f"first put {e!r}") + e = put(s, "alpha", "two", V) + check(e.version == 2 and e.kind is V, "second put") + check(s.get("alpha").value == b"two", "get") + exc = expect_key_not_found(lambda: s.get("nope"), "nope", "get('nope')") + check(exc.message == "key not found: nope", f"message {exc.message!r}") + found = s.find("alpha") + check(found is not None and found.version == 2, "find present") + check(s.find("nope") is None, "find absent") + + # TTLs follow the logical clock; an expired get reports when. + check(put(s, "ttl", "x", V, 10).expires_at == 10, "expires_at") + check(s.now() == 0, "now") + check(s.tick(9) == 9 and s.count() == 2, "tick 9") + check(s.tick(1) == 10 and s.count() == 1, "tick 1") + exc = raises(kv.Expired, lambda: s.get("ttl"), "get expired") + check(exc.code == 1002 and exc.key == "ttl" and exc.expired_at == 10, f"Expired {exc!r}") + expect_key_not_found(lambda: s.get("ttl"), "ttl", "the expired read removed it") + + # Capacity: a new key past it is StoreFull { capacity }. + s.set_capacity(1) + check(s.capacity() == 1, "set_capacity") + put(s, "alpha", "three") + exc = raises(kv.StoreFull, lambda: put(s, "beta", "b"), "StoreFull") + check(exc.capacity == 1, "StoreFull payload") + s.set_capacity(100) + + # An undeclared enum value is a marshalling failure, raised as the root + # error on a call that declares errors. + exc = raises(kv.Error, lambda: s.put("k", b"v", 9, None), "undeclared EntryKind") + check(exc.code == kv.Error.MARSHAL_ERROR_CODE and not isinstance(exc, kv.KvError), + f"marshal error {exc!r}") + + put(s, "beta", "b") + check(s.delete("beta") is True and s.delete("beta") is False, "delete") + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + check(s.size() == s.count() == 1, "size") + check([w.category for w in caught] == [DeprecationWarning], "size warns") + check(s.clear() == 1 and s.count() == 0, "clear") + s.close() + s.close() -def object_graph() -> None: - store = wv.Store.open(PATH) - store.put("k", b"v", wv.EntryKind.Persistent, None) - - # share(): a second wrapper over the same object. - shared = store.share() - check(isinstance(shared, wv.Store) and shared is not store, "share returns a new wrapper") - check(shared.put("k2", b"vv", wv.EntryKind.Persistent, None) is True, "put via shared") - check(store.count() == 2, "mutation through share visible through the original") - store.close() - check(shared.count() == 2, "object alive through the shared wrapper") - - # fork(): an independent copy. - forked = shared.fork() - check(forked.count() == 2, "fork copies live entries") - forked.put("only-forked", b"x", wv.EntryKind.Persistent, None) - check(forked.count() == 3 and shared.count() == 2, "fork is independent") - - # larger(): Store? in both directions. - empty = wv.Store.open(PATH) - check(empty.larger(None) is None, "larger(None) on an empty store is None") - own = shared.larger(None) - check(own is not None and own.count() == 2, "larger(None) on a populated store is itself") - own.put("k3", b"3", wv.EntryKind.Persistent, None) - check(shared.count() == 3, "larger(None) returned the same object") - own.close() - bigger = empty.larger(forked) - check(bigger is not None and bigger.count() == 3, "larger(other) picks the bigger") - bigger.put("k4", b"4", wv.EntryKind.Persistent, None) - check(forked.count() == 4, "larger(other) returned the other object") - bigger.close() - check(shared.larger(empty).count() == 3, "larger prefers self over a smaller other") - - # describe(): a record carrying the object (and an optional second one). - info = shared.describe("primary", None) - check(isinstance(info, wv.StoreInfo), "describe returns StoreInfo") - check(info.label == "primary" and info.count == 3 and info.mirror is None, f"info {info}") - check(isinstance(info.store, wv.Store) and info.store is not shared, - "record object field is a wrapper") - info.store.put("k5", b"5", wv.EntryKind.Persistent, None) - check(shared.count() == 4, "record object field refers to the same store") - mirrored = shared.describe("mirrored", forked) - check(mirrored.mirror is not None and mirrored.mirror.count() == 4, "mirror present") - mirrored.mirror.put("k6", b"6", wv.EntryKind.Persistent, None) - check(forked.count() == 5, "mirror refers to the other store") - - # open_many(): a list of objects as a return, and its throwing path. - many = wv.Store.open_many(["/a", "/b", "/c"]) - check(len(many) == 3 and all(isinstance(s, wv.Store) for s in many), "open_many list") - check([s.count() for s in many] == [0, 0, 0], "open_many stores are empty") - many[0].put("m", b"1", wv.EntryKind.Persistent, None) - many[2].put("n", b"2", wv.EntryKind.Persistent, None) - check([s.count() for s in many] == [1, 0, 1], "open_many stores are distinct") - expect_kv_error(lambda: wv.Store.open_many(["/ok", ""]), wv.KvError.IoError, 1004, - "open_many with an empty path") - check(wv.Store.open_many([]) == [], "open_many([])") - - # total_count(): list of objects and an object inside an optional record - # as parameters. Encoding clones each object; the wrappers stay usable. - check(wv.Store.total_count(many, None) == 2, "total_count without extra") - check(wv.Store.total_count([], None) == 0, "total_count of nothing") - extra = wv.StoreInfo(label="extra", store=shared, mirror=forked, count=-1) - check(wv.Store.total_count(many, extra) == 2 + 4, "total_count with a local StoreInfo") - check(wv.Store.total_count(many + [shared, forked], info) == 2 + 4 + 5 + 4, - "total_count with duplicates and a producer-built StoreInfo") - check(shared.count() == 4 and forked.count() == 5 and many[0].count() == 1, - "wrappers usable after being encoded") - # A closed store in a buffered position is rejected before any reference - # is minted, so the live store beside it leaks nothing. - try: - wv.Store.total_count([empty, closed_store()], None) - check(False, "expected error encoding a closed store") - except wv.Error as exc: - check("Store used after close" in exc.message, f"closed store in list {exc.message!r}") - try: - wv.Store.total_count([], wv.StoreInfo("x", empty, closed_store(), 0)) - check(False, "expected error encoding a closed mirror") - except wv.Error as exc: - check("Store used after close" in exc.message, f"closed mirror {exc.message!r}") - check(empty.count() == 0, "live store usable after a rejected encoding") - - # Release everything, twice where it is cheap to prove idempotence. - for s in [shared, forked, empty, info.store, mirrored.store, mirrored.mirror] + many: - s.close() - s.close() - for s in [shared, forked, empty] + many: - try: - s.count() - check(False, "closed store still usable") - except wv.Error: - pass +def iterators() -> None: + s = kv.Store.open("/iter") + put(s, "user.bob", "b") + put(s, "user.alice", "a") + put(s, "sys.x", "xx") + + check(list(s.keys(None)) == ["sys.x", "user.alice", "user.bob"], "keys(None)") + expect_key_not_found(lambda: s.keys("zzz"), "zzz", "keys('zzz')") + + # Abandoning an iterator part-way releases it. + it = s.keys("user.") + check(next(it) == "user.alice", "first key") + check(impl._debug_live(2) == 1, "one live iterator") + it.close() # type: ignore[attr-defined] + check(impl._debug_live(2) == 0, "abandoned iterator released") + check(list(it) == [], "a closed iterator is exhausted") + + entries = list(s.entries("sys.")) + check(len(entries) == 1 and entries[0].key == "sys.x" and entries[0].value == b"xx", + f"entries('sys.') {entries!r}") + + prefixes = ["user.", "sys.", "none."] + parts = list(s.partition(prefixes)) + check(len(parts) == 3 and len(set(parts)) == 3 and s not in parts, "distinct stores") + check([p.count() for p in parts] == [2, 1, 0], "partition counts") + check([p.path() for p in parts] == prefixes, "partition paths") + for p in parts: + p.close() + s.close() -def closed_store() -> wv.Store: - s = wv.Store.open(PATH) + +class Listener(kv.Listener): + """Records every change; `accepts` says no to `skip` and raises on + `fail_on`.""" + + def __init__(self, skip: str = "", fail_on: str = "") -> None: + self.skip = skip + self.fail_on = fail_on + self.changes: List[kv.Change] = [] + self.threads: List[int] = [] + + def accepts(self, key: str) -> bool: + if key == self.fail_on: + raise RuntimeError("listener refused") + return key != self.skip + + def on_change(self, change: kv.Change) -> None: + self.threads.append(threading.get_ident()) + self.changes.append(change) + + def of(self, cls: type) -> List[kv.Change]: + return [c for c in self.changes if isinstance(c, cls)] + + +def listeners() -> None: + s = kv.Store.open("/listen") + base = held_callbacks() + listener = Listener(skip="quiet") + sub = s.subscribe(listener) + check(sub > 0 and s.listener_count() == 1 and held_callbacks() == base + 1, "subscribe") + + put(s, "a", "1") + last = listener.changes[-1] + check(isinstance(last, kv.ChangePut) and last.entry.version == 1 and not last.replaced, + f"first Put {last!r}") + put(s, "a", "2") + last = listener.changes[-1] + check(isinstance(last, kv.ChangePut) and last.entry.version == 2 and last.replaced, + "second Put") + put(s, "quiet", "x") + check(len(listener.of(kv.ChangePut)) == 2, "accepts() filtered quiet") + check(s.delete("a"), "delete a") + check(listener.changes[-1] == kv.ChangeRemoved(key="a", expired=False), "Removed") + + # An expired read removes the entry and says so. + put(s, "short", "x", V, 1) + s.tick(1) + raises(kv.Expired, lambda: s.get("short"), "get short") + check(listener.changes[-1] == kv.ChangeRemoved(key="short", expired=True), "expired Removed") + check(s.clear() == 1, "clear") + check(listener.changes[-1] == kv.ChangeCleared(count=1), "Cleared") + check(listener.changes[-1].tag is kv.Change.Tag.Cleared, "Change tag") + check(set(listener.threads) == {MAIN_THREAD}, "synchronous calls notify inline") + + # Unsubscribing releases the listener once. + check(s.unsubscribe(sub) is True and held_callbacks() == base, "unsubscribe releases") + check(s.unsubscribe(sub) is False and s.listener_count() == 0, "unsubscribe again") + + # A listener that raises is detached (and released); the put succeeds. + failing = Listener(fail_on="boom") + s.subscribe(failing) + put(s, "fine", "1") + check(len(failing.of(kv.ChangePut)) == 1, "failing listener saw fine") + put(s, "boom", "1") + check(s.count() == 2 and s.listener_count() == 0, "failing listener detached") + check(held_callbacks() == base, "failing listener released") + + # A listener of the wrong type is rejected before the call. + raises(TypeError, lambda: s.subscribe(object()), "subscribe(object())") + check(held_callbacks() == base, "nothing registered") + + # Destroying the store releases the listeners it still holds. + s.subscribe(Listener()) + s.subscribe(Listener()) + check(s.listener_count() == 2 and held_callbacks() == base + 2, "two held") s.close() - return s - - -class Blocking(wv.EvictionListener): - """A listener that blocks inside the producer's `delete` until released, - so the test can act while that call is in flight.""" - - def __init__(self) -> None: - self.entered = threading.Event() - self.release = threading.Event() - - def on_evict(self, entry: wv.Entry, reason: wv.EvictionReason) -> bool: - self.entered.set() - self.release.wait(10) - return True - - -def close_during_call() -> None: - # close() from another thread while a call is in flight defers the - # native release until the call returns, instead of freeing the store - # out from under it. - store = wv.Store.open(PATH) - listener = Blocking() - store.set_eviction_listener(listener) - store.put("k", b"v", wv.EntryKind.Persistent, None) - result = [] - worker = threading.Thread(target=lambda: result.append(store.delete("k"))) - worker.start() - check(listener.entered.wait(10), "listener entered") - store.close() - try: - store.count() - check(False, "a closing wrapper rejects new calls") - except wv.Error as exc: - check("after close" in exc.message, f"closing message {exc.message!r}") - listener.release.set() - worker.join(10) - check(result == [True], f"in-flight delete completed: {result}") - - -async def cancel_compact() -> None: - # compact is cancellable: cancelling the awaiting task cancels the native - # token, and the coroutine raises CancelledError (or, when compaction - # already finished, the task simply completes). - store = wv.Store.open(PATH) - task = asyncio.ensure_future(store.compact()) + check(held_callbacks() == base, "closing the store released its listeners") + + +class Policy(kv.Policy): + """Tags and encrypts admitted entries, routes `b/` keys to `other`, and + fails in various ways for some keys.""" + + def __init__(self, other: kv.Store) -> None: + self.other = other + self.admitted = 0 + + def admit(self, entry: kv.Entry) -> kv.Entry: + self.admitted += 1 + check(entry.version == 0, "the store assigns the version after admission") + if entry.key.startswith("secret"): + raise kv.KvError.Rejected(key=entry.key, reason="no secrets", + message="secrets are not stored") + if entry.key.startswith("boom"): + raise RuntimeError("policy exploded") + entry.tags = ["admitted"] + entry.kind = E + entry.key = "renamed" # ignored by the store + return entry + + def route(self, key: str, home: kv.Store) -> kv.Store: + if key.startswith("b/"): + return self.other + if key.startswith("null/"): + return None # type: ignore[return-value] + return home + + +def policies() -> None: + s = kv.Store.open("/policy") + other = kv.Store.open("/other") + base = held_callbacks() + policy = Policy(other) + s.set_policy(policy) + check(s.has_policy() and held_callbacks() == base + 1, "set_policy") + + e = put(s, "a", "1", V) + check(e.key == "a" and e.version == 1 and e.kind is E and e.tags == ["admitted"], + f"admitted entry {e!r}") + + put(s, "b/x", "2", V) + check(s.count() == 1 and other.count() == 1, "route redirected the write") + + exc = raises(kv.Rejected, lambda: put(s, "secret", "3", V), "Rejected") + check(exc.code == 1005 and exc.key == "secret" and exc.reason == "no secrets", + f"Rejected payload {exc!r}") + check(exc.message == "secrets are not stored", f"Rejected message {exc.message!r}") + + exc = raises(kv.Error, lambda: put(s, "boom", "4", V), "non-domain failure") + check(exc.code == kv.Error.FOREIGN_ERROR_CODE and exc.message == "policy exploded", + f"foreign error {exc!r}") + check(not isinstance(exc, kv.KvError), "a foreign error isn't a domain error") + + # A None where an object is required is the implementation's failure. + exc = raises(kv.Error, lambda: put(s, "null/x", "6", V), "None route") + check(exc.code == kv.Error.FOREIGN_ERROR_CODE, f"None route {exc!r}") + check(s.count() == 1 and other.count() == 1, "failed puts stored nothing") + check(policy.admitted == 5, f"admitted {policy.admitted}") + + # Replacing the policy releases the old one; None removes it. + s.set_policy(Policy(other)) + check(held_callbacks() == base + 1, "replacing released the old policy") + s.set_policy(None) + check(not s.has_policy() and held_callbacks() == base, "set_policy(None) released it") + put(s, "secret", "now allowed") + check(s.count() == 2, "no policy") + other.close() + s.close() + + +class Loader(kv.Loader): + def __init__(self, fallback: Optional[kv.Store] = None) -> None: + self.backup = fallback + + def name(self) -> str: + return "py-loader" + + def fallback(self, key: str) -> Optional[kv.Store]: + return self.backup if key == "fb" else None + + def load(self, key: str) -> bytes: + if key == "missing": + raise kv.KeyNotFound(key="missing", message="not in the loader") + if key == "elsewhere": + raise kv.KeyNotFound(key="other", message="not in the loader") + if key == "broken": + raise RuntimeError("loader is broken") + return f"loaded:{key}".encode() + + +def loaders() -> None: + s = kv.Store.open("/load") + base = held_callbacks() + check(s.get_or_load("k", None) is None, "no loader") + + e = s.get_or_load("k", Loader()) + check(e is not None and e.value == b"loaded:k" and e.kind is V + and e.metadata == {"source": "py-loader"}, f"loaded {e!r}") + check(held_callbacks() == base, "a loader is released after the call") + check(s.count() == 1, "the loaded entry is stored") + e = s.get_or_load("k", Loader()) + check(e is not None and e.version == 1, "a hit doesn't consult the loader") + + with kv.Store.open("/backup") as backup: + put(backup, "fb", "from backup") + e = s.get_or_load("fb", Loader(backup)) + check(e is not None and e.value == b"from backup" and e.kind is P, f"fallback {e!r}") + + check(s.get_or_load("missing", Loader()) is None, "KeyNotFound for this key") + exc = expect_key_not_found(lambda: s.get_or_load("elsewhere", Loader()), "other", + "KeyNotFound for another key") + check(exc.message == "not in the loader", f"passed-through message {exc.message!r}") + exc = raises(kv.Error, lambda: s.get_or_load("broken", Loader()), "broken loader") + check(exc.code == kv.Error.FOREIGN_ERROR_CODE and exc.message == "loader is broken", + f"broken loader {exc!r}") + check(held_callbacks() == base, "every loader released") + s.close() + + +async def async_calls() -> None: + s = kv.Store.open("/async-calls") + listener = Listener() + s.subscribe(listener) + put(s, "old1", "x", V, 1) + put(s, "old2", "x", V, 1) + put(s, "keep", "x") + s.tick(5) + + # compact runs on a producer thread and notifies listeners there. + check(await s.compact(0) == 2, "compact(0)") + removed = listener.of(kv.ChangeRemoved) + check(len(removed) == 2 and all(c.expired for c in removed), f"compact notified {removed!r}") + threads = listener.threads[-2:] + check(MAIN_THREAD not in threads, "notified from a producer thread") + check(s.count() == 1, "compacted") + + check(await s.compact(5) == 0, "compact(5)") + + # Cancel mid-pause: the call stops at once, and the background pause + # notices the token and stops. + task = asyncio.ensure_future(s.compact(60000)) + await asyncio.sleep(0.02) + check(not task.done() and kv.Store.active_jobs() >= 1, "compact(60000) running") + started = time.monotonic() task.cancel() try: await task + check(False, "a cancelled compact returned") except asyncio.CancelledError: pass - check(task.cancelled() or task.result() == 0, "cancelled compact") - check(await store.compact() == 0, "compact still works after a cancellation") - store.close() + check(time.monotonic() - started < 1.0, "cancelled promptly") + deadline = time.monotonic() + 2.0 + while kv.Store.active_jobs() and time.monotonic() < deadline: + await asyncio.sleep(0.001) + check(kv.Store.active_jobs() == 0, "the cancelled pause stopped cooperatively") + + # get_many: an async list of optional records, launched concurrently. + results = await asyncio.gather(*(s.get_many(["keep", "gone", "keep"]) for _ in range(32))) + for got in results: + check(len(got) == 3 and got[1] is None and got[0] is not None and got[2] is not None + and got[2].key == "keep", f"get_many {got!r}") + + other = kv.Store.open("/other") + put(other, "a", "123") + stats = await kv.summarize_all([s, other]) + check(stats == kv.Stats(entries=2, bytes=4, by_kind={P: 2}), f"summarize_all {stats!r}") + other.close() + s.close() + + +def object_graph() -> None: + s = kv.Store.open("/graph") + put(s, "k", "v") + + # share(): the same object behind a new wrapper. + shared = s.share() + check(shared == s and shared is not s and hash(shared) == hash(s), "share is the same object") + s.close() + check(shared.count() == 1, "alive through the shared reference") + s = shared + + fork = s.fork() + check(fork != s and fork.count() == 1 and fork.path() == "/graph", "fork") + put(fork, "k2", "v") + check(fork.count() == 2 and s.count() == 1, "fork is a copy") + + empty = kv.Store.open("/empty") + check(empty.larger(None) is None, "larger(None) on an empty store") + check(empty.larger(fork) == fork, "larger(fork)") + check(s.larger(None) == s, "larger(None) on a non-empty store is self") + + info = s.describe("main", fork) + check(info == kv.StoreInfo(label="main", store=s, mirror=fork, count=1), f"describe {info!r}") + check(info.mirror is not None and info.mirror.count() == 2, "mirror usable") + + many = kv.Store.open_many(["/a", "/b"]) + check([m.path() for m in many] == ["/a", "/b"], "open_many") + raises(kv.InvalidPath, lambda: kv.Store.open_many(["/a", ""]), "open_many with ''") + + named = kv.Store.by_label([info, kv.StoreInfo(label="first", store=many[0], mirror=None, + count=0)]) + check(named == {"main": s, "first": many[0]}, f"by_label {named!r}") + + put(many[0], "m", "1") + check(kv.Store.total_count([many[0], many[1], fork], named, info) == 6, "total_count") + check(kv.Store.total_count([many[0], many[1], fork], named, None) == 5, "total_count None") + check(s.count() == 1 and fork.count() == 2 and many[0].count() == 1, "all still usable") + + for obj in [*many, *named.values(), info.store, empty, fork, s]: + obj.close() + check(info.mirror is not None, "a record keeps its wrappers") + info.mirror.close() + + +def stats_and_report() -> None: + s = kv.Store.open("/stats") + put(s, "b", "12") + put(s, "a", "1") + put(s, "a", "123") + put(s, "c", "x", E) + + stats = kv.summarize(s, None) + check(stats == kv.Stats(entries=3, bytes=6, by_kind={P: 2, E: 1}), f"summarize {stats!r}") + expect_key_not_found(lambda: kv.summarize(s, "q"), "q", "summarize('q')") + + lines = kv.render_report(list(s.entries(None))) + check(lines == ["a: 3 bytes, Persistent, v2", "b: 2 bytes, Persistent", + "c: 1 bytes, Encrypted"], f"render_report {lines!r}") + exc = raises(kv.ReportError.NothingToReport, lambda: kv.render_report([]), "empty report") + check(exc.code == 2001 and exc.message == "nothing to report", f"NothingToReport {exc!r}") + check(isinstance(exc, kv.ReportError) and not isinstance(exc, kv.KvError), "report domain") + s.close() def main() -> None: + load_checks() + constructors() basics() - eviction_listener() + iterators() + listeners() + policies() + loaders() + asyncio.run(async_calls()) object_graph() - close_during_call() - asyncio.run(cancel_compact()) - consumer.finish(wv) + stats_and_report() + check(held_callbacks() == 0, "every callback implementation released") + consumer.finish() main() diff --git a/conformance/python/run.sh b/conformance/python/run.sh index ea62a728..6aa25e05 100755 --- a/conformance/python/run.sh +++ b/conformance/python/run.sh @@ -5,6 +5,7 @@ # against one sample and must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools python python3 # Run a Python consumer against the generated package as-is: its directory # goes on PYTHONPATH, and the producer cdylib is selected through the @@ -18,22 +19,11 @@ py_consumer() { } python_calculator() { py_consumer calculator calculator_consumer.py; } - -python_contacts() { py_consumer contacts contacts_consumer.py; } - -python_events() { py_consumer events events_consumer.py; } - -python_kvstore() { py_consumer kvstore kvstore_consumer.py; } - -python_async_demo() { py_consumer async-demo async_demo_consumer.py; } - -python_codec() { py_consumer codec codec_consumer.py; } +python_codec() { py_consumer codec codec_consumer.py; } +python_kvstore() { py_consumer kvstore kvstore_consumer.py; } lane python-calculator python_calculator -lane python-contacts python_contacts -lane python-events python_events -lane python-kvstore python_kvstore -lane python-async-demo python_async_demo lane python-codec python_codec +lane python-kvstore python_kvstore finish_lanes diff --git a/conformance/ruby/async_demo.rb b/conformance/ruby/async_demo.rb deleted file mode 100644 index fada720b..00000000 --- a/conformance/ruby/async_demo.rb +++ /dev/null @@ -1,120 +0,0 @@ -# frozen_string_literal: true -# Conformance consumer: async-demo sample, Ruby target. -# -# Drives the blocking async bridge end to end: `run_task` blocks the calling -# thread until the producer's completion callback fires and decodes the -# TaskResult value class from a value buffer, an empty name raises the typed -# TaskError::InvalidName subclass, `run_batch` round-trips a list of records -# through value buffers both ways, `run_n_tasks` returns a direct scalar -# through the callback (also from many threads at once), and the cancellable -# `wait` honors a CancelToken passed as `cancel:` (cancelled from another -# thread, or before the call) and an interrupted wait (Thread#raise), each -# raising AsyncDemo::Cancelled or the interruption long before the timeout. -# `active_callbacks` is back to zero once every task body has completed or -# been dropped. The gem is installed from the generated tree; the cdylib is -# selected through ASYNC_DEMO_LIBRARY. - -require_relative 'support' -require 'async_demo' - -# Polls the producer until no task body is in flight. -def settle_tasks - 100.times do - return true if AsyncDemo.active_callbacks.zero? - - sleep 0.02 - end - false -end - -run_and_check_leaks(AsyncDemo) do - expect(AsyncDemo::ABI_VERSION == 3, 'bindings target ABI revision 3') - - # Async record return: blocks until the worker-thread callback delivers the - # encoded TaskResult. - result = AsyncDemo.run_task('alpha') - expect(result.id.positive?, 'run_task assigns an id') - expect(result.value == 'completed: alpha', "run_task value (got #{result.value})") - expect(result.success == true, 'run_task success flag') - - # Typed async error: the empty name settles with the InvalidName subclass. - begin - AsyncDemo.run_task('') - expect(false, 'expected TaskError::InvalidName for empty name') - rescue AsyncDemo::TaskError::InvalidName => e - expect(e.code == 1, "InvalidName carries code 1 (got #{e.code})") - expect(e.is_a?(AsyncDemo::TaskError), 'subclass of TaskError') - expect(e.is_a?(AsyncDemo::Error), 'subclass of AsyncDemo::Error') - end - - # Buffered list-of-records both ways. - batch = AsyncDemo.run_batch(%w[a b c]) - expect(batch.map(&:value) == ['completed: a', 'completed: b', 'completed: c'], - "run_batch values (got #{batch.map(&:value)})") - expect(batch.all?(&:success), 'run_batch success flags') - expect(AsyncDemo.run_batch([]) == [], 'run_batch of nothing') - - # Direct scalar through the async callback, from many threads at once. - expect(AsyncDemo.run_n_tasks(7) == 7, 'run_n_tasks echoes n') - results = Array.new(8) { |i| Thread.new { Array.new(25) { |j| AsyncDemo.run_n_tasks(i * 100 + j) } } }.map(&:value) - expect(results.each_with_index.all? { |rs, i| rs == Array.new(25) { |j| i * 100 + j } }, - 'concurrent async calls each receive their own result') - - # --- Cancellation -------------------------------------------------------- - # Without cancellation, wait completes with the milliseconds waited. - expect(AsyncDemo.wait(20) == 20, 'wait completes after its timeout') - idle = AsyncDemo::CancelToken.new - expect(AsyncDemo.wait(5, cancel: idle) == 5, 'an untouched token changes nothing') - idle.close - - # Cancelling the token from another thread completes the call at once with - # AsyncDemo::Cancelled (code -5) instead of waiting out the timeout. - token = AsyncDemo::CancelToken.new - canceller = Thread.new do - sleep 0.05 - token.cancel - end - started = Time.now - begin - AsyncDemo.wait(60_000, cancel: token) - raise 'expected AsyncDemo::Cancelled' - rescue AsyncDemo::Cancelled => e - expect(e.code == -5 && e.code == AsyncDemo::CANCELLED_ERROR_CODE, "Cancelled code -5 (got #{e.code})") - expect(e.is_a?(AsyncDemo::Error), 'Cancelled is an AsyncDemo::Error') - end - expect(Time.now - started < 5, "cancellation didn't wait for the timeout") - canceller.join - expect(token.cancelled?, 'token reports cancelled') - - # A token cancelled before the call never runs the body. - begin - AsyncDemo.wait(60_000, cancel: token) - raise 'expected AsyncDemo::Cancelled for a pre-cancelled token' - rescue AsyncDemo::Cancelled => e - expect(e.code == -5, "pre-cancelled code (got #{e.code})") - end - token.close - - # Interrupting the waiting thread cancels the call's private token, so the - # producer drops its work (active_callbacks settles) instead of running out - # the timeout. - interrupted = Class.new(StandardError) - waiter = Thread.new do - AsyncDemo.wait(60_000) - :completed - rescue interrupted - :interrupted - end - sleep 0.1 - started = Time.now - waiter.raise(interrupted) - expect(waiter.value == :interrupted, 'the interrupted wait raised the interruption') - expect(settle_tasks, 'interrupting the wait cancelled the producer work') - expect(Time.now - started < 5, "the interrupted work didn't run out its timeout") - - # Sync functions beside the async ones; every task body has completed by - # the time its callback fires. - expect(settle_tasks, 'active_callbacks settles to zero') -end - -puts 'ruby/async-demo: OK' diff --git a/conformance/ruby/calculator.rb b/conformance/ruby/calculator.rb index 3b8b9c74..383d3505 100644 --- a/conformance/ruby/calculator.rb +++ b/conformance/ruby/calculator.rb @@ -1,35 +1,52 @@ # frozen_string_literal: true + # Conformance consumer: calculator sample, Ruby target. # -# The minimal surface: direct scalars, the typed CalcError domain, and a -# string round trip (including non-ASCII text and an interior NUL, which -# cross the ABI as a pointer plus a length). The gem is installed from the -# generated tree; the cdylib is selected through CALCULATOR_LIBRARY. +# The minimal surface: the load-time checks (requiring the gem checked the +# ABI revision and the contract table; here a stale entry is shown to fail +# with the declaration's path), i32 arithmetic that wraps, a throwing +# function whose typed error has no payload, and strings that keep +# non-ASCII text, astral characters, and interior NULs. The gem is installed +# from the generated tree; the cdylib is selected through CALCULATOR_LIBRARY. require_relative 'support' require 'calculator' +I32_MAX = (2**31) - 1 +I32_MIN = -(2**31) + run_and_check_leaks(Calculator) do - expect(Calculator::ABI_VERSION == 3, 'bindings target ABI revision 3') - expect(Calculator.add(2, 40) == 42, 'add') - expect(Calculator.mul(6, 7) == 42, 'mul') - expect(Calculator.div(10, 2) == 5, 'div') - - begin - Calculator.div(1, 0) - raise 'expected CalcError::DivisionByZero' - rescue Calculator::CalcError::DivisionByZero => e - expect(e.code == 1, "DivisionByZero code (got #{e.code})") - expect(e.message == 'division by zero', "message (got #{e.message.inspect})") - expect(e.is_a?(Calculator::Error), 'domain errors subclass Calculator::Error') + expect(Calculator::ABI_VERSION == 4, 'bindings target ABI revision 4') + missing = expect_raise(LoadError, 'an entry the library lacks') do + Calculator._wv_check_contract!(calculator_calculator_contract: [[1, 2, 'calculator.gone']]) + end + expect(missing.message.end_with?('calculator.gone is missing from the library'), missing.message) + stale = expect_raise(LoadError, 'an entry whose signature changed') do + Calculator._wv_check_contract!(calculator_calculator_contract: [[fnv1a64('calculator.add'), 0, 'calculator.add']]) end + expect(stale.message.end_with?('calculator.add changed since these bindings were generated'), stale.message) + + expect(Calculator.add(2, 3) == 5, 'add') + expect(Calculator.add(I32_MAX, 1) == I32_MIN, 'add wraps') + expect_raise(RangeError, 'an i32 out of range') { Calculator.add(I32_MAX + 1, 0) } + + expect(Calculator.divide(10, 2) == 5, 'divide') + expect(Calculator.divide(-7, 2) == -3, 'divide rounds toward zero') + expect(Calculator.divide(I32_MIN, -1) == I32_MIN, 'divide wraps') + e = expect_raise(Calculator::CalcError::DivisionByZero, 'divide by zero') { Calculator.divide(1, 0) } + expect(e.is_a?(Calculator::CalcError) && e.is_a?(Calculator::Error), 'domain errors subclass the root error') + expect(e.code == 1 && Calculator::CalcError::DivisionByZero::CODE == 1, "DivisionByZero code (got #{e.code})") + expect(e.message == 'division by zero', "message (got #{e.message.inspect})") + expect(e.instance_variables == [:@code], 'DivisionByZero carries no payload fields') - expect(Calculator.echo('hello') == 'hello', 'echo') - expect(Calculator.echo('') == '', 'echo empty') - unicode = Calculator.echo('héllo ✓ 日本') - expect(unicode == 'héllo ✓ 日本', "echo unicode (got #{unicode.inspect})") - expect(unicode.encoding == Encoding::UTF_8, 'echo returns UTF-8') - expect(Calculator.echo("a\0b") == "a\0b", 'echo keeps an interior NUL') + expect(Calculator.greet('World') == 'Hello, World!', 'greet') + expect(Calculator.greet('') == 'Hello, !', 'greet an empty name') + crab = Calculator.greet('Wörld 🦀') + expect(crab == 'Hello, Wörld 🦀!', "non-ASCII and astral text (got #{crab.inspect})") + expect(crab.encoding == Encoding::UTF_8, 'greet returns UTF-8') + expect(Calculator.greet("a\0b") == "Hello, a\0b!", 'an interior NUL') + expect_raise(TypeError, 'a non-string') { Calculator.greet(42) } + expect_raise(ArgumentError, 'invalid UTF-8') { Calculator.greet((+"\xC3\x28").force_encoding(Encoding::UTF_8)) } end puts 'ruby/calculator: OK' diff --git a/conformance/ruby/codec.rb b/conformance/ruby/codec.rb index ec4eaedc..e12e94ec 100644 --- a/conformance/ruby/codec.rb +++ b/conformance/ruby/codec.rb @@ -1,366 +1,225 @@ # frozen_string_literal: true -# Conformance consumer: codec sample, Ruby target. + +# Conformance consumer: codec sample, the wire oracle, Ruby target. # -# Round-trip oracle for the generated value-buffer codec. For each fixture the -# producer encodes and Ruby decodes (`sample_*`, checked field by field), Ruby -# encodes and the producer decodes (`verify_*` must return true), and both -# directions compose (`roundtrip_*` must equal the input). Ruby-built values -# cover the edges: empty strings, lists, and maps; non-ASCII text; strings -# with interior NULs; i64/u64 extremes; Float::NAN, +/-Float::INFINITY, and -# -0.0; every Shape variant; and objects inside buffers (Holder: a Token -# field, an optional Token, and a list of Tokens; `primary_of` returns a -# wrapper to the SAME object as `holder.primary`; every wrapper closes -# exactly once). Exit is non-zero on any mismatch. The gem is installed from -# the generated tree; the cdylib is selected through CODEC_LIBRARY. +# The shared-vector loop: for every vector the producer serves, decode it, +# hand it back to `check_vector` (which re-encodes it), and push each +# primitive vector's value through the matching direct-family `echo_*`. +# Then vectors built from literals (so a symmetric encode/decode bug can't +# hide), spot checks of decoded fields (64-bit integers exact, float bits, +# astral text), the typed out-of-range error and its payload, malformed +# buffers sent through the raw attached function (each rejected with -3) +# and the encoder's own range checks, and object identity and reference +# counting through buffers. Ends with every leak counter at zero. require_relative 'support' require 'codec' -def negative_zero?(x) - x.zero? && (1.0 / x) == -Float::INFINITY -end - I64_MIN = -(2**63) -I64_MAX = 2**63 - 1 -U64_MAX = 2**64 - 1 - -run_and_check_leaks(Codec) do - expect(Codec::ABI_VERSION == 3, "bindings target ABI revision 3") - expect(Codec::Color::RED.zero? && Codec::Color::GREEN == 1 && Codec::Color::BLUE == 7, "Color constants") - - # --- Scalars ------------------------------------------------------------- - scalars = Codec.sample_scalars - expect(scalars.is_a?(Codec::Scalars), "sample_scalars returns a Scalars") - expect(scalars.i8_value == -8, "i8 (got #{scalars.i8_value})") - expect(scalars.u8_value == 200, "u8 (got #{scalars.u8_value})") - expect(scalars.i16_value == -16_000, "i16 (got #{scalars.i16_value})") - expect(scalars.u16_value == 60_000, "u16 (got #{scalars.u16_value})") - expect(scalars.i32_value == -2_000_000_000, "i32 (got #{scalars.i32_value})") - expect(scalars.u32_value == 4_000_000_000, "u32 (got #{scalars.u32_value})") - expect(scalars.i64_value == -9_007_199_254_740_993, "i64 beyond 2^53 (got #{scalars.i64_value})") - expect(scalars.u64_value == U64_MAX, "u64::MAX (got #{scalars.u64_value})") - expect(scalars.f32_value == 1.5, "f32 (got #{scalars.f32_value})") - expect(scalars.f64_value == -2.25e100, "f64 (got #{scalars.f64_value})") - expect(scalars.flag == true, "flag") - expect(scalars.color == Codec::Color::BLUE, "color BLUE (got #{scalars.color})") - - expect(Codec.verify_scalars(scalars) == true, "producer accepts the re-encoded Scalars") - expect(Codec.roundtrip_scalars(scalars) == scalars, "roundtrip_scalars equals the sample") - - # The same fixture built from scratch in Ruby verifies; one changed field is a - # typed Mismatch (code 1). - rebuilt = Codec::Scalars.new( - i8_value: -8, u8_value: 200, i16_value: -16_000, u16_value: 60_000, - i32_value: -2_000_000_000, u32_value: 4_000_000_000, - i64_value: -9_007_199_254_740_993, u64_value: U64_MAX, - f32_value: 1.5, f64_value: -2.25e100, flag: true, color: Codec::Color::BLUE - ) - expect(rebuilt == scalars, "Ruby-built Scalars equals the decoded sample") - expect(Codec.verify_scalars(rebuilt) == true, "Ruby-built Scalars verifies") - changed = Codec::Scalars.new( - i8_value: -8, u8_value: 200, i16_value: -16_000, u16_value: 60_000, - i32_value: -2_000_000_000, u32_value: 4_000_000_000, - i64_value: -9_007_199_254_740_993, u64_value: U64_MAX, - f32_value: 1.5, f64_value: -2.25e100, flag: false, color: Codec::Color::BLUE - ) - begin - Codec.verify_scalars(changed) - raise "expected CodecError::Mismatch" - rescue Codec::CodecError::Mismatch => e - expect(e.code == 1, "Mismatch code == 1 (got #{e.code})") - expect(e.is_a?(Codec::CodecError) && e.is_a?(Codec::Error), "Mismatch subclasses the domain and Codec::Error") - end - - # Every integer extreme plus float specials round-trip bit for bit. - extremes = Codec::Scalars.new( - i8_value: -128, u8_value: 255, i16_value: -32_768, u16_value: 65_535, - i32_value: -(2**31), u32_value: 2**32 - 1, - i64_value: I64_MIN, u64_value: U64_MAX, - f32_value: -Float::INFINITY, f64_value: -0.0, flag: false, color: Codec::Color::RED - ) - back = Codec.roundtrip_scalars(extremes) - expect(back == extremes, "extreme Scalars round-trip") - expect(back.i64_value == I64_MIN && back.u64_value == U64_MAX, "64-bit extremes inside a record") - expect(back.f32_value == -Float::INFINITY, "f32 -inf inside a record") - expect(negative_zero?(back.f64_value), "f64 -0.0 keeps its sign inside a record") - highs = Codec::Scalars.new( - i8_value: 127, u8_value: 0, i16_value: 32_767, u16_value: 0, - i32_value: 2**31 - 1, u32_value: 0, i64_value: I64_MAX, u64_value: 0, - f32_value: Float::NAN, f64_value: Float::INFINITY, flag: true, color: Codec::Color::GREEN - ) - back = Codec.roundtrip_scalars(highs) - expect(back.i8_value == 127 && back.i16_value == 32_767 && back.i32_value == 2**31 - 1, "signed maxima") - expect(back.i64_value == I64_MAX, "i64::MAX inside a record") - expect(back.u8_value.zero? && back.u16_value.zero? && back.u32_value.zero? && back.u64_value.zero?, "unsigned zeros") - expect(back.f32_value.nan?, "f32 NaN survives inside a record") - expect(back.f64_value == Float::INFINITY, "f64 +inf inside a record") - expect(back.color == Codec::Color::GREEN, "enum inside a record") +I64_MAX = (2**63) - 1 +U64_MAX = (2**64) - 1 - # --- Composite ----------------------------------------------------------- - composite = Codec.sample_composite - expect(composite.is_a?(Codec::Composite), "sample_composite returns a Composite") - expect(composite.name == "héllo wörld ✓", "unicode name (got #{composite.name.inspect})") - expect(composite.name.encoding == Encoding::UTF_8, "strings decode as UTF-8") - expect(composite.blob == "\x00\x01\x02\xFD\xFE\xFF".b, "blob bytes (got #{composite.blob.inspect})") - expect(composite.blob.encoding == Encoding::BINARY, "bytes decode as BINARY") - expect(composite.some_i64 == I64_MIN, "present optional i64::MIN (got #{composite.some_i64})") - expect(composite.none_i64.nil?, "absent optional is nil") - expect(composite.some_text == "", "present optional empty string (got #{composite.some_text.inspect})") - expect(composite.names == ["a", "", "ccc"], "list of strings (got #{composite.names})") - expect(composite.matrix == [[1, 2, 3], [], [-4]], "list of lists (got #{composite.matrix})") - expect(composite.empty == [], "empty list") - expect(composite.by_name == { "one" => 1, "two" => 2, "neg" => -3 }, "string-keyed map (got #{composite.by_name})") - expect(composite.by_id.keys.sort == [-1, 42], "int-keyed map keys (got #{composite.by_id.keys})") - expect(composite.by_id[-1] == scalars, "by_id[-1] is the canonical Scalars") - expect(composite.by_id[42].flag == false && composite.by_id[42].u64_value == U64_MAX, "by_id[42] differs only in flag") - expect(composite.scalars == scalars, "nested record equals sample_scalars") - expect(composite.shape == Codec::Shape::Labeled.new(label: "tag", count: 3), "rich enum field") - expect(composite.shape.tag == 3 && composite.shape.tag == Codec::Shape::Labeled::TAG, "variant tag") - expect(composite.shapes.length == 5, "one shape per variant") - expect(composite.shapes[0] == Codec::Shape::Empty.new, "shapes[0] Empty") - expect(composite.shapes[1] == Codec::Shape::Circle.new(radius: 2.5), "shapes[1] Circle") - expect(composite.shapes[2] == Codec::Shape::Rect.new(width: 1.0, height: 0.5), "shapes[2] Rect") - expect(composite.shapes[3] == Codec::Shape::Labeled.new(label: "", count: -1), "shapes[3] Labeled") - expect(composite.shapes[4] == Codec::Shape::Nested.new(inner: scalars, note: "n"), "shapes[4] Nested") - expect(composite.shapes.map(&:tag) == [0, 1, 2, 3, 4], "variant tags in order") - expect(composite.maybe_shape == Codec::Shape::Nested.new(inner: scalars, note: nil), "optional rich enum") - expect(composite.maybe_list == "\x09\x08".b, "optional bytes list (got #{composite.maybe_list.inspect})") - expect(composite.sparse == [true, nil, false], "list of optionals (got #{composite.sparse})") - expect(composite.colors == [Codec::Color::RED, Codec::Color::GREEN, Codec::Color::BLUE], "list of enums") +V = Codec::Vector - expect(Codec.verify_composite(composite) == true, "producer accepts the re-encoded Composite") - expect(Codec.roundtrip_composite(composite) == composite, "roundtrip_composite equals the sample") - described = Codec.describe_composite(composite) - expect(described.include?("héllo wörld ✓") && described.include?("Labeled"), "describe_composite renders (got #{described[0, 60].inspect})") +def f32_bits(x) + [x].pack('e').unpack1('L<') +end - # Rebuilt in Ruby with hashes in a different insertion order: maps are - # unordered on the wire, so the producer still sees the canonical value. - rebuilt_composite = Codec::Composite.new( - name: "héllo wörld ✓", - blob: [0, 1, 2, 253, 254, 255].pack("C*"), - some_i64: I64_MIN, - none_i64: nil, - some_text: "", - names: ["a", "", "ccc"], - matrix: [[1, 2, 3], [], [-4]], - empty: [], - by_name: { "two" => 2, "neg" => -3, "one" => 1 }, - by_id: { 42 => changed, -1 => rebuilt }, - scalars: rebuilt, - shape: Codec::Shape::Labeled.new(label: "tag", count: 3), - shapes: [ - Codec::Shape::Empty.new, - Codec::Shape::Circle.new(radius: 2.5), - Codec::Shape::Rect.new(width: 1.0, height: 0.5), - Codec::Shape::Labeled.new(label: "", count: -1), - Codec::Shape::Nested.new(inner: rebuilt, note: "n") - ], - maybe_shape: Codec::Shape::Nested.new(inner: rebuilt, note: nil), - maybe_list: "\x09\x08".b, - sparse: [true, nil, false], - colors: [Codec::Color::RED, Codec::Color::GREEN, Codec::Color::BLUE] - ) - expect(rebuilt_composite == composite, "Ruby-built Composite equals the decoded sample") - expect(Codec.verify_composite(rebuilt_composite) == true, "Ruby-built Composite verifies") - tweaked = Codec::Composite.new( - name: composite.name, blob: composite.blob, some_i64: composite.some_i64, none_i64: nil, - some_text: composite.some_text, names: composite.names, matrix: composite.matrix, empty: [], - by_name: composite.by_name, by_id: composite.by_id, scalars: composite.scalars, - shape: composite.shape, shapes: composite.shapes, maybe_shape: composite.maybe_shape, - maybe_list: composite.maybe_list, sparse: [true, true, false], colors: composite.colors - ) - begin - Codec.verify_composite(tweaked) - raise "expected CodecError::Mismatch for a tweaked Composite" - rescue Codec::CodecError::Mismatch => e - expect(e.code == 1, "tweaked Composite -> Mismatch (got #{e.code})") - end +def f64_bits(x) + [x].pack('E').unpack1('Q<') +end - # An edge-value Composite: everything empty or absent, plus float specials in - # a list and nested extremes. - edge = Codec::Composite.new( - name: "", - blob: "".b, - some_i64: I64_MAX, - none_i64: -1, - some_text: nil, - names: [], - matrix: [[], [-(2**31), 0, 2**31 - 1]], - empty: [1.5, Float::INFINITY, -Float::INFINITY, -0.0, 0.0], - by_name: {}, - by_id: { 0 => extremes }, - scalars: extremes, - shape: Codec::Shape::Empty.new, - shapes: [], - maybe_shape: nil, - maybe_list: "".b, - sparse: [nil, nil], - colors: [] - ) - edge_back = Codec.roundtrip_composite(edge) - expect(edge_back == edge, "edge Composite round-trips") - expect(edge_back.name == "" && edge_back.blob == "".b && edge_back.names == [] && edge_back.by_name == {}, - "empties survive") - expect(edge_back.some_text.nil? && edge_back.maybe_shape.nil? && edge_back.some_i64 == I64_MAX && edge_back.none_i64 == -1, - "optionals in both states") - expect(edge_back.matrix == [[], [-(2**31), 0, 2**31 - 1]], "i32 extremes in a nested list (got #{edge_back.matrix})") - expect(edge_back.empty[1] == Float::INFINITY && edge_back.empty[2] == -Float::INFINITY, "infinities in a list") - expect(negative_zero?(edge_back.empty[3]) && !negative_zero?(edge_back.empty[4]), "-0.0 and 0.0 keep their signs") - expect(edge_back.maybe_list == "".b && !edge_back.maybe_list.nil?, "present empty optional bytes") - expect(edge_back.sparse == [nil, nil], "list of absent optionals") - expect(edge_back.by_id[0] == extremes, "record inside a map with an integer key") - expect(Codec.describe_composite(edge).start_with?("Composite {"), "describe_composite on the edge value") +# Floats compare bitwise, except that any NaN matches any NaN. +def same_float?(a, b) + (a.nan? && b.nan?) || f64_bits(a) == f64_bits(b) +end - # --- Shape (rich enum) --------------------------------------------------- - [ - Codec::Shape::Empty.new, - Codec::Shape::Circle.new(radius: -0.0), - Codec::Shape::Rect.new(width: 0.25, height: -Float::INFINITY), - Codec::Shape::Labeled.new(label: "ünïcödé ✓", count: -(2**31)), - Codec::Shape::Nested.new(inner: extremes, note: ""), - Codec::Shape::Nested.new(inner: rebuilt, note: nil) - ].each do |shape| - back = Codec.roundtrip_shape(shape) - expect(back == shape, "roundtrip_shape #{shape.class} (got #{Codec.describe_shape(back)})") - expect(back.class == shape.class && back.tag == shape.tag, "roundtrip_shape keeps the variant") +# Closes every object wrapper inside a decoded value. +def release(value) + case value + when Codec::Token then value.close + when Array then value.each { |v| release(v) } + when Hash then value.each_value { |v| release(v) } + when Codec::Holder then [value.primary, value.spare, value.many, value.by_name].each { |v| release(v) } + when V::Objects then release(value.value) end - expect(negative_zero?(Codec.roundtrip_shape(Codec::Shape::Circle.new(radius: -0.0)).radius), "-0.0 in a variant") - expect(Codec.describe_shape(Codec::Shape::Empty.new) == "Empty", "describe_shape Empty") - expect(Codec.describe_shape(Codec::Shape::Circle.new(radius: 2.5)) == "Circle { radius: 2.5 }", "describe_shape Circle") - expect(Codec.describe_shape(Codec::Shape::Labeled.new(label: "x", count: 1)) == 'Labeled { label: "x", count: 1 }', - "describe_shape Labeled") - expect(Codec.roundtrip_shapes([]) == [], "empty list of shapes") - expect(Codec.roundtrip_shapes(composite.shapes) == composite.shapes, "list of every variant") - expect(Codec.roundtrip_shapes(composite.shapes).map(&:tag) == [0, 1, 2, 3, 4], "tags preserved in a list") - - # --- Standalone shapes: optionals, maps, strings, bytes, scalars --------- - expect(Codec.roundtrip_opt_i64(nil).nil?, "opt i64 absent") - expect(Codec.roundtrip_opt_i64(0).zero?, "opt i64 zero is present") - expect(Codec.roundtrip_opt_i64(I64_MIN) == I64_MIN, "opt i64::MIN") - expect(Codec.roundtrip_opt_i64(I64_MAX) == I64_MAX, "opt i64::MAX") - expect(Codec.roundtrip_map({}) == {}, "empty map") - unicode_map = { "" => 0, "ключ" => I64_MIN, "✓" => I64_MAX, "a" => -1 } - expect(Codec.roundtrip_map(unicode_map) == unicode_map, "unicode-keyed map with extremes") - expect(Codec.roundtrip_string("") == "", "empty string") - expect(Codec.roundtrip_string("héllo ✓ 日本") == "héllo ✓ 日本", "unicode string") - expect(Codec.roundtrip_string("héllo").encoding == Encoding::UTF_8, "returned string is UTF-8") - # Strings cross the ABI as a pointer plus a length, so an interior NUL - # survives in both directions (and inside a value buffer). - expect(Codec.roundtrip_string("a\0b") == "a\0b", "interior NUL string (got #{Codec.roundtrip_string("a\0b").inspect})") - expect(Codec.roundtrip_string("\0") == "\0", "lone NUL string") - expect(Codec.roundtrip_string("x\0✓\0") == "x\0✓\0", "NULs around non-ASCII text") - nul_map = { "k\0ey" => 1, "\0" => 2 } - expect(Codec.roundtrip_map(nul_map) == nul_map, "interior NULs in buffered map keys") - expect(Codec.roundtrip_bytes("".b) == "".b, "empty bytes") - all_bytes = (0..255).to_a.pack("C*") - expect(Codec.roundtrip_bytes(all_bytes) == all_bytes, "every byte value, including NUL") - expect(Codec.roundtrip_bytes(all_bytes).encoding == Encoding::BINARY, "returned bytes are BINARY") - expect(Codec.roundtrip_i64(I64_MIN) == I64_MIN, "direct i64::MIN") - expect(Codec.roundtrip_i64(I64_MAX) == I64_MAX, "direct i64::MAX") - expect(Codec.roundtrip_i64(-1) == -1, "direct -1") - expect(Codec.roundtrip_u64(U64_MAX) == U64_MAX, "direct u64::MAX") - expect(Codec.roundtrip_u64(2**63) == 2**63, "direct 2^63 stays unsigned") - expect(Codec.roundtrip_u64(0).zero?, "direct u64 zero") - expect(Codec.roundtrip_f64(Float::NAN).nan?, "direct NaN") - expect(Codec.roundtrip_f64(Float::INFINITY) == Float::INFINITY, "direct +inf") - expect(Codec.roundtrip_f64(-Float::INFINITY) == -Float::INFINITY, "direct -inf") - expect(negative_zero?(Codec.roundtrip_f64(-0.0)), "direct -0.0 keeps its sign") - expect(Codec.roundtrip_f64(Float::MAX) == Float::MAX, "direct f64::MAX") - expect(Codec.roundtrip_f64(Float::MIN) == Float::MIN, "direct smallest normal") - expect(Codec.roundtrip_f64(5e-324) == 5e-324, "direct subnormal") - expect(Codec.roundtrip_bool(true) == true && Codec.roundtrip_bool(false) == false, "direct bool") - expect(Codec.roundtrip_color(Codec::Color::BLUE) == Codec::Color::BLUE, "direct enum BLUE") - expect(Codec.roundtrip_color(Codec::Color::RED) == Codec::Color::RED, "direct enum RED") - - # --- Holder (objects inside buffers) --------------------------------------- - holder = Codec.make_holder(10, true) - expect(holder.is_a?(Codec::Holder), "make_holder returns a Holder") - expect(holder.primary.is_a?(Codec::Token) && !holder.primary.closed?, "primary is an open Token") - expect(holder.primary.value == 10, "primary value (got #{holder.primary.value})") - expect(holder.spare.is_a?(Codec::Token) && holder.spare.value == 11, "spare value") - expect(holder.many.length == 3 && holder.many.map(&:value) == [12, 13, 14], "many values (got #{holder.many.map(&:value)})") - addresses = ([holder.primary, holder.spare] + holder.many).map { |t| t.handle.address } - expect(addresses.uniq.length == 5, "five distinct token objects") - - # Encoding a Holder mints one reference per token, which the producer consumes. - expect(Codec.sum_holder(holder) == 10 + 11 + 12 + 13 + 14, "sum_holder (got #{Codec.sum_holder(holder)})") - expect(Codec.sum_holder(holder) == 60, "sum_holder is repeatable (tokens not consumed)") - expect(holder.primary.value == 10 && holder.spare.value == 11, "tokens usable after being encoded") - - # primary_of returns a wrapper to the SAME object as holder.primary. - primary = Codec.primary_of(holder) - expect(primary.is_a?(Codec::Token), "primary_of returns a Token") - expect(!primary.equal?(holder.primary), "primary_of returns a distinct wrapper") - expect(primary.handle.address == holder.primary.handle.address, "primary_of wraps the same object") - expect(primary.value == 10, "primary_of value") - primary.close - primary.close - expect(primary.closed?, "primary_of wrapper closed idempotently") - expect(holder.primary.value == 10, "original primary alive after closing the returned wrapper") - - # same_primary compares object identity through two encoded buffers. - expect(Codec.same_primary(holder, holder) == true, "same_primary(h, h)") - other = Codec.make_holder(10, true) - expect(other.primary.value == 10, "other holder has equal values") - expect(Codec.same_primary(holder, other) == false, "same_primary is identity, not value") - aliased = Codec::Holder.new(primary: holder.primary, spare: nil, many: []) - expect(Codec.same_primary(holder, aliased) == true, "a Ruby-built Holder sharing the wrapper is the same primary") - twin = holder.primary.dup - expect(twin.handle.address == holder.primary.handle.address, "dup of a Token wraps the same object") - expect(Codec.same_primary(holder, Codec::Holder.new(primary: twin, spare: nil, many: [])) == true, - "a dup'd wrapper is still the same object") - twin.close +end - # A Holder built entirely in Ruby from constructor-made tokens. - t1 = Codec::Token.new(1) - t2 = Codec::Token.new(2) - t3 = Codec::Token.new(-3) - expect(t1.value == 1 && t3.value == -3, "Token.new values") - built = Codec::Holder.new(primary: t1, spare: t2, many: [t3, t1, t1]) - expect(Codec.sum_holder(built) == 1 + 2 + -3 + 1 + 1, "sum_holder over a Ruby-built Holder (repeated object)") - expect(Codec.primary_of(built).handle.address == t1.handle.address, "primary_of a Ruby-built Holder") - expect(Codec.sum_holder(Codec::Holder.new(primary: t1, spare: nil, many: [])) == 1, "Holder with nothing optional") - expect(Codec.sum_holder(Codec::Holder.new(primary: Codec::Token.new(I64_MAX), spare: nil, many: [])) == I64_MAX, - "i64::MAX through a token") +# The direct-family echo of each primitive vector. +ECHOES = { + V::I8 => :echo_i8, + V::U8 => :echo_u8, + V::I16 => :echo_i16, + V::U16 => :echo_u16, + V::I32 => :echo_i32, + V::U32 => :echo_u32, + V::I64 => :echo_i64, + V::U64 => :echo_u64, + V::F32 => :echo_f32, + V::F64 => :echo_f64, + V::Flag => :echo_bool, + V::Text => :echo_text, + V::Blob => :echo_blob, + V::Hue => :echo_color +}.freeze + +def holder(primary, spare: nil, many: [], by_name: {}) + Codec::Holder.new(primary: primary, spare: spare, many: many, by_name: by_name) +end - # Absent optional object. - without = Codec.make_holder(0, false) - expect(without.spare.nil?, "make_holder(_, false) has no spare") - expect(without.primary.value.zero? && without.many.map(&:value) == [2, 3, 4], "values without spare") - expect(Codec.sum_holder(without) == 9, "sum without spare") +# Sends a raw (malformed) Vector encoding to `check_vector` through the +# attached C function, which must reject it as a marshalling failure. +def reject(bytes, what) + err = Codec::ErrorStruct.new + ok = Codec.codec_codec_check_vector(0, bytes, bytes.bytesize, err) + code = err[:code] + Codec.codec_error_clear(err) + expect(!ok && code == -3, "#{what} is rejected with -3 (got #{code})") +end - # A closed token inside a Holder is refused before anything is encoded. - t2.close - begin - Codec.sum_holder(built) - raise "expected Error encoding a closed Token" - rescue Codec::Error => e - expect(e.message.include?("after close"), "closed token in a buffer (got #{e.message.inspect})") - end - begin - t2.value - raise "expected Error when using a closed Token" - rescue Codec::Error => e - expect(e.message.include?("after close"), "use-after-close message") +def run_codec + n = Codec.vector_count + expect(n >= 60, "at least 60 vectors (got #{n})") + names = Array.new(n) { |i| Codec.vector_name(i) } + find = ->(name) { names.index(name) || raise("no vector named #{name}") } + + # 1. The loop. + n.times do |i| + v = Codec.vector(i) + unless Codec.check_vector(i, v) + raise "vector #{i} (#{names[i]}) did not round-trip; the producer saw #{Codec.describe_vector(v)}" + end + expect(!Codec.check_vector((i + 1) % n, v), "vector #{i} (#{names[i]}) never matches its neighbor") + echo = ECHOES[v.class] + if echo + back = Codec.public_send(echo, v.value) + same = v.value.is_a?(Float) ? same_float?(back, v.value) : back == v.value + expect(same, "echo of vector #{i} (#{names[i]}): #{back.inspect} vs #{v.value.inspect}") + expect(f32_bits(back) == f32_bits(v.value), "f32 bits of #{names[i]}") if v.is_a?(V::F32) + end + release(v) end - # Release: every wrapper closes exactly once, double close is a no-op, and - # wrappers left to GC release without double frees. - [holder, other, without].each do |h| - h.primary.close - h.spare&.close - h.many.each(&:close) - h.primary.close - h.many.each(&:close) - expect(h.primary.closed? && h.many.all?(&:closed?), "holder tokens closed") - end - t1.close - t3.close - t1.close - 200.times { |i| Codec.sum_holder(Codec.make_holder(i, i.odd?)) } - GC.start - GC.start - expect(Codec.roundtrip_i64(42) == 42, "library still healthy after GC of unclosed wrappers") + # 2. Literal vectors. + canonical = { + i8_value: -8, u8_value: 200, i16_value: -16_000, u16_value: 60_000, + i32_value: -2_000_000_000, u32_value: 4_000_000_000, i64_value: -9_007_199_254_740_993, + u64_value: U64_MAX, f32_value: 1.5, f64_value: -2.25e100, flag: true, color: Codec::Color::BLUE + } + scalars = ->(fields) { V::AllScalars.new(value: Codec::Scalars.new(**fields)) } + expect(Codec.check_vector(find['scalars canonical'], scalars[canonical]), 'scalars canonical') + expect(!Codec.check_vector(find['scalars canonical'], scalars[canonical.merge(u16_value: 60_001)]), + 'scalars canonical with one field changed') + labeled = V::Figure.new(value: Codec::Shape::Labeled.new(label: 'tag', count: 3)) + expect(Codec.check_vector(find['shape labeled'], labeled), 'shape labeled') + expect(Codec.check_vector(find['string interior nul'], V::Text.new(value: "nul\0inside\0")), 'interior nul') + other_nan = [0x7ff8000000000001].pack('Q<').unpack1('E') + expect(Codec.check_vector(find['f64 nan'], V::F64.new(value: other_nan)), 'any NaN is the NaN vector') + expect(Codec.check_vector(find['f64 -0'], V::F64.new(value: -0.0)), 'negative zero') + expect(!Codec.check_vector(find['f64 -0'], V::F64.new(value: 0.0)), 'positive zero is not negative zero') + expect(Codec.check_vector(find['u64 max'], V::U64.new(value: U64_MAX)), 'u64 max') + expect(Codec.check_vector(find['enum infrared'], V::Hue.new(value: Codec::Color::INFRARED)), 'enum infrared') + expect(Codec.check_vector(find['optional zero'], V::MaybeI64.new(value: 0)), 'optional zero') + expect(Codec.check_vector(find['optional absent'], V::MaybeI64.new(value: nil)), 'optional absent') + expect(!Codec.check_vector(find['optional zero'], V::MaybeI64.new(value: nil)), 'absent is not zero') + reordered = V::Counts.new(value: { 'x' => 0, 'héllo' => -1, '' => I64_MAX }) + expect(Codec.check_vector(find['map of strings'], reordered), 'a map built in another order') + expect(Codec.check_vector(find['blank'], V::Blank.new), 'blank') + lone = Codec::Token.new(-1) + expect(Codec.check_vector(find['objects sparse'], V::Objects.new(value: holder(lone))), + 'objects sparse from a consumer-made token') + lone.close + + # 3. Spot checks on decoded values. + expect(Codec.vector(find['i64 past 2^53']) == V::I64.new(value: -9_007_199_254_740_993), 'i64 past 2^53 is exact') + subnormal = Codec.vector(find['f32 min subnormal']) + expect(subnormal.is_a?(V::F32) && f32_bits(subnormal.value) == 1, 'f32 min subnormal bits') + astral = Codec.vector(find['string astral']) + expect(astral == V::Text.new(value: '🦀 crab 😀') && astral.value.encoding == Encoding::UTF_8, 'string astral') + m = Codec.vector(find['scalars minimum']).value + expect([m.i8_value, m.i16_value, m.i32_value, m.i64_value, m.u64_value] == + [-128, -32_768, -(2**31), I64_MIN, 0], 'scalars minimum integers') + expect(m.f32_value == -Float::INFINITY && m.f64_value.nan?, 'scalars minimum floats') + expect(m.color == Codec::Color::INFRARED && m.flag == false, 'scalars minimum enum and flag') + deep = Codec.vector(find['composite canonical']) + expect(deep.is_a?(V::Deep), 'composite canonical is Deep') + c = deep.value + expect(c.name == 'héllo wörld ✓', 'composite name') + expect(c.blob == [0, 1, 2, 253, 254, 255].pack('C*') && c.blob.encoding == Encoding::BINARY, 'composite blob') + expect(c.some_i64 == I64_MIN && c.none_i64.nil? && c.some_text == '', 'composite optionals') + expect(c.names == ['a', '', 'ccc'], 'composite names') + expect(c.matrix == [[1, 2, 3], [], [-4]], 'composite matrix') + expect(c.floats.length == 6 && c.floats[0].nan? && f64_bits(c.floats[3]) == f64_bits(-0.0) && + c.floats[4] == 5e-324, 'composite floats') + expect(c.by_name == { 'one' => 1, 'two' => 2, 'neg' => -3, '' => I64_MAX }, 'composite by_name') + expect(c.by_id.keys.sort == [-1, 42, (2**31) - 1] && c.by_id[42].u64_value.zero?, 'composite by_id') + expect(c.by_color == { Codec::Color::INFRARED => 'below', Codec::Color::BLUE => 'sky' }, 'composite by_color') + expect(c.flags == { 0 => false, U64_MAX => true }, 'composite flags') + expect(c.scalars.u32_value == 4_000_000_000, 'composite scalars') + expect(c.shape == Codec::Shape::Labeled.new(label: 'tag', count: 3), 'composite shape') + expect(c.shapes.length == 6 && c.shapes[5].is_a?(Codec::Shape::Nested) && c.shapes[5].note.nil?, 'composite shapes') + expect(c.maybe_shape.is_a?(Codec::Shape::Nested), 'composite maybe_shape') + expect(c.maybe_list == "\x09\x08".b, 'composite maybe_list') + expect(c.sparse == [true, nil, false], 'composite sparse') + expect(c.colors == [0, 1, 7, -1], 'composite colors') + + # 4. The typed error with its payload. + e = expect_raise(Codec::CodecError::OutOfRange, 'vector past the end') { Codec.vector(n) } + expect(e.is_a?(Codec::CodecError) && e.is_a?(Codec::Error) && e.code == 1, 'OutOfRange is the domain error') + expect(e.index == n && e.count == n, "OutOfRange payload (got #{e.index}, #{e.count})") + expect(e.message == "vector #{n} is out of range (count #{n})", "OutOfRange message (got #{e.message})") + e = expect_raise(Codec::CodecError::OutOfRange, 'vector_name past the end') { Codec.vector_name(n + 5) } + expect(e.index == n + 5 && e.count == n, 'vector_name payload') + expect(!Codec.check_vector(n, V::Blank.new), 'check_vector past the end is false') + + # 5. Malformed input: raw buffers the encoder would never produce reach + # the producer through the attached C function; an undeclared enum value + # through the direct family traps; the encoder refuses what the wire + # can't carry. + str = ->(s) { [s.bytesize].pack('L<') + s.b } + reject([V::I8::TAG].pack('l<'), 'a truncated buffer') + reject([99].pack('l<'), 'an unknown tag') + reject([V::Blank::TAG].pack('l<') + "\x00".b, 'trailing bytes') + reject([V::Flag::TAG].pack('l<') + "\x02".b, 'a bool byte of 2') + reject([V::Hue::TAG, 3].pack('l 20, 'b' => 21 }, 'tokens by name') + # Each encoding mints fresh references, so the holder can be sent twice. + total = 10 + 11 + 12 + 13 + 20 + 21 + I64_MIN + expect(Codec.sum_holder(h) == total && Codec.sum_holder(h) == total, 'sum_holder twice') + # primary_of returns the very same object (a new wrapper of it). + p = Codec.primary_of(h) + expect(Codec.same_primary(h, holder(p)), 'primary_of is the same object') + twin = Codec::Token.new(10) + expect(!Codec.same_primary(h, holder(twin)), 'an equal value is not the same object') + minus4 = Codec::Token.new(-4) + shared = holder(twin, spare: twin, many: [twin, twin, minus4], by_name: { 'k' => twin }) + expect(Codec.sum_holder(shared) == 46, 'a holder of consumer tokens, one wrapper in several slots') + expect_raise(TypeError, 'a wrapper of the wrong type') { Codec.sum_holder(holder('not a token')) } + [p, twin, minus4].each(&:close) + release(full) + closed = expect_raise(Codec::Error, 'a closed token') { p.value } + expect(closed.message.include?('used after close'), closed.message) + n end -puts "ruby/codec: OK" +count = nil +run_and_check_leaks(Codec) { count = run_codec } + +puts "ruby/codec: OK (#{count} vectors)" diff --git a/conformance/ruby/contacts.rb b/conformance/ruby/contacts.rb deleted file mode 100644 index b11041fc..00000000 --- a/conformance/ruby/contacts.rb +++ /dev/null @@ -1,145 +0,0 @@ -# frozen_string_literal: true -# Conformance consumer: contacts sample, Ruby target. -# -# Drives the value-type surface: Contact is a plain Ruby value class decoded -# from the value buffer the producer returns (the optional email is nil-able -# and the list return is a native Array), while ContactBook is a -# reference-counted interface wrapper whose `close` releases its strong -# reference early (idempotent; the AutoPointer otherwise releases it at GC -# time), and whose `dup` mints an independent wrapper. Throwing methods -# raise the typed ContactsError subclasses (InvalidName=1, NotFound=2); -# non-throwing methods keep the generic Contacts::Error for panics only. The -# nested contacts.groups module uses the parent's enum, interface, and error -# domain; the sibling directory root shares the Contact record. The gem is -# installed from the generated tree; the cdylib is selected through -# CONTACTS_LIBRARY. - -require_relative 'support' -require 'contacts' - -run_and_check_leaks(Contacts) do - expect(Contacts::ABI_VERSION == 3, "bindings target ABI revision 3") - book = Contacts::ContactBook.new - - alice = book.add("Alice", "Smith", "alice@example.com", Contacts::ContactType::WORK) - expect(alice.is_a?(Contacts::Contact), "add returns a Contact value") - expect(alice.id.positive?, "alice id positive") - expect(alice.first_name == "Alice", "first_name") - expect(alice.last_name == "Smith", "last_name") - expect(alice.email == "alice@example.com", "email") - expect(alice.contact_type == Contacts::ContactType::WORK, "contact_type") - - # get decodes a fresh snapshot; value classes compare structurally. - expect(book.get(alice.id) == alice, "get snapshot equals added contact") - - # Optional string: a missing email round-trips as nil. - bob = book.add("Bob", "Jones", nil, Contacts::ContactType::PERSONAL) - expect(book.get(bob.id).email.nil?, "bob email nil") - - expect(book.count == 2, "count == 2") - everyone = book.list - expect(everyone.is_a?(Array), "list returns a native Array") - expect(everyone.length == 2, "list length == 2") - expect(everyone.map(&:first_name).sort == %w[Alice Bob], "list names") - - expect(book.remove(alice.id) == true, "remove returns true") - expect(book.count == 1, "count == 1 after remove") - expect(book.remove(alice.id) == false, "second remove returns false") - - # Typed errors: each domain code raises its own subclass of the domain - # class, which itself subclasses the generic error. - begin - book.add("", "Smith", nil, Contacts::ContactType::PERSONAL) - raise "expected ContactsError::InvalidName for empty name" - rescue Contacts::ContactsError::InvalidName => e - expect(e.code == 1, "InvalidName code == 1 (got #{e.code})") - end - - begin - book.get(9999) - raise "expected ContactsError::NotFound for missing contact" - rescue Contacts::ContactsError::NotFound => e - expect(e.code == 2, "NotFound code == 2 (got #{e.code})") - expect(e.is_a?(Contacts::ContactsError), "NotFound is a ContactsError") - expect(e.is_a?(Contacts::Error), "domain errors subclass Contacts::Error") - end - - # Rescuing the domain base class catches any code in the domain. - begin - book.get(9999) - raise "expected ContactsError" - rescue Contacts::ContactsError => e - expect(e.code == 2, "domain rescue sees NotFound code (got #{e.code})") - end - - # Each book owns independent state. `dup` produces a second wrapper holding - # its own strong reference to the same object, so state is shared and closing - # one wrapper leaves the other usable. - other = Contacts::ContactBook.new - expect(other.count.zero?, "fresh book empty") - twin = other.dup - expect(twin.handle.address == other.handle.address, "dup wraps the same object") - other.add("Carol", "White", nil, Contacts::ContactType::PERSONAL) - expect(twin.count == 1, "state visible through the dup (got #{twin.count})") - twin.close - expect(twin.closed?, "dup reports closed") - expect(!other.closed?, "original still open after closing the dup") - expect(other.count == 1, "original usable after closing the dup") - - # Explicit close releases the wrapper's reference early; a second close is a - # no-op (the AutoPointer's GC release is also a no-op afterward, so no - # double-free), and using a closed wrapper raises the brand error. - other.close - other.close - expect(other.closed?, "closed? after close") - begin - other.count - raise "expected Error when using a closed ContactBook" - rescue Contacts::Error => e - expect(e.message.include?("after close"), "use-after-close message (got #{e.message.inspect})") - end - book.close - - # contacts.groups is nested under the contacts root: its functions take and - # return the parent's ContactType enum and ContactBook interface and report - # the parent's ContactsError. - team = Contacts::ContactBook.new - team.add("Ada", "Lovelace", "ada@example.com", Contacts::ContactType::WORK) - team.add("Grace", "Hopper", nil, Contacts::ContactType::WORK) - team.add("Linus", "Pauling", nil, Contacts::ContactType::PERSONAL) - expect(Contacts.count_of_type(team, Contacts::ContactType::WORK) == 2, "count_of_type WORK") - expect(Contacts.count_of_type(team, Contacts::ContactType::OTHER).zero?, "count_of_type OTHER") - expect(Contacts.dominant_type(team) == Contacts::ContactType::WORK, "dominant_type is WORK") - work = Contacts.split_by_type(team, Contacts::ContactType::WORK) - expect(work.is_a?(Contacts::ContactBook), "split_by_type returns a ContactBook") - expect(work.handle.address != team.handle.address, "split_by_type returns a new book") - expect(work.count == 2 && team.count == 3, "split copies only WORK contacts") - first = Contacts.first_of_type(team, Contacts::ContactType::PERSONAL) - expect(first.is_a?(Contacts::Contact) && first.first_name == "Linus", "first_of_type PERSONAL") - begin - Contacts.first_of_type(work, Contacts::ContactType::PERSONAL) - raise "expected ContactsError::NotFound from the nested module" - rescue Contacts::ContactsError::NotFound => e - expect(e.code == 2, "nested module reports the parent's domain (got #{e.code})") - end - - # directory is a sibling root sharing the Contact record (a value type) and - # declaring its own DirectoryError domain. - card = Contacts.card(first) - expect(card == Contacts::Card.new(display_name: "Pauling, Linus", initials: "LP", has_email: false), - "card (got #{card.display_name.inspect} #{card.initials.inspect} #{card.has_email})") - expect(Contacts.card(team.list[0]).has_email, "card sees the email") - sorted = Contacts.sorted(team.list) - expect(sorted.map(&:last_name) == %w[Hopper Lovelace Pauling], "sorted by last name (got #{sorted.map(&:last_name)})") - begin - Contacts.sorted([]) - raise "expected DirectoryError::Empty" - rescue Contacts::DirectoryError::Empty => e - expect(e.code == 1, "DirectoryError::Empty code (got #{e.code})") - expect(!e.is_a?(Contacts::ContactsError), "the sibling root's domain is distinct") - end - work.close - team.close -end - -puts "ruby/contacts: OK" diff --git a/conformance/ruby/events.rb b/conformance/ruby/events.rb deleted file mode 100644 index ae4a4021..00000000 --- a/conformance/ruby/events.rb +++ /dev/null @@ -1,291 +0,0 @@ -# frozen_string_literal: true -# Conformance consumer: events sample, Ruby target. -# -# Drives the ABI 3 callback-interface surface: a Ruby class that includes -# the generated `Events::Subscriber` mixin and overrides its three methods, -# a reference-counted `EventBus` object (constructor, `dup`, `close`, use -# after close), the bus handing a strong `EventBus` reference to the -# consumer through `on_attached`, `Delivery` return values steering -# `publish`'s accepted count, a Ruby exception inside a callback (both the -# Result-returning `route` and the plain `on_message`) surfacing to the -# caller as `Events::Error` with FOREIGN_ERROR_CODE (-4) without crashing -# the VM, the blocking async `publish_later` bridge (whose callbacks run on -# a producer thread), the iterator-backed `messages` Enumerator, the -# optional `last_message` record, and the release of every consumer -# implementation (the producer's `free` entry drops the handle-table slot). -# The gem is installed from the generated tree; the cdylib is selected -# through EVENTS_LIBRARY. - -require_relative 'support' -require 'events' - -# The handle table is private generated state, but with no async call -# pending its size is the number of live callback implementations: the -# only observable proof that the producer's `free` entry ran (and that a -# consumer implementation is not leaked or freed twice). -def live_callbacks - Events.instance_variable_get(:@wv_handles).size -end - -# A subscriber that records every call so the arguments can be asserted. -class Recorder - include Events::Subscriber - - attr_reader :routed, :messages, :attached - - def initialize(skip: nil, stop: nil, fail_on: nil, fail_in_message: nil) - @skip = skip - @stop = stop - @fail_on = fail_on - @fail_in_message = fail_in_message - @routed = [] - @messages = [] - @attached = [] - end - - def route(topic) - @routed << topic - raise "boom: #{topic}" if topic == @fail_on - return Events::Delivery::SKIP if topic == @skip - return Events::Delivery::ACCEPT_AND_STOP if topic == @stop - - Events::Delivery::ACCEPT - end - - def on_message(message) - raise ArgumentError, "on_message rejected #{message.text}" if message.text == @fail_in_message - - @messages << message - @messages.length - end - - def on_attached(bus) - @attached << bus - end -end - -# The mixin supplies NotImplementedError defaults for anything not overridden. -class Bare - include Events::Subscriber -end - -# A subscriber that closes a bus wrapper from inside route, while the -# publish call on that same wrapper is still in flight. -class Closer < Recorder - attr_accessor :target - - def route(topic) - target&.close - super - end -end - -run_and_check_leaks(Events) do - expect(Events::FOREIGN_ERROR_CODE == -4, "FOREIGN_ERROR_CODE is -4") - expect(Events::ABI_VERSION == 3, "bindings target ABI revision 3") - expect(Events::Delivery::ACCEPT.zero?, "Delivery::ACCEPT == 0") - expect(Events::Delivery::SKIP == 1, "Delivery::SKIP == 1") - expect(Events::Delivery::ACCEPT_AND_STOP == 2, "Delivery::ACCEPT_AND_STOP == 2") - - baseline = live_callbacks - - bus = Events::EventBus.new - expect(bus.is_a?(Events::EventBus), "constructor returns an EventBus") - expect(!bus.closed?, "fresh bus is open") - expect(bus.subscriber_count.zero?, "fresh bus has no subscribers") - expect(bus.last_message.nil?, "last_message is nil before any publish") - expect(bus.messages.is_a?(Enumerator), "messages returns an Enumerator") - expect(bus.messages.to_a == [], "no messages before any publish") - - # subscribe: the bus calls on_attached synchronously with a strong reference - # to itself, which the consumer adopts as a fresh wrapper to the same object. - first = Recorder.new(skip: "quiet", stop: "stop") - expect(bus.subscribe(first) == 1, "first subscribe returns count 1") - expect(first.attached.length == 1, "on_attached called once (got #{first.attached.length})") - handed = first.attached[0] - expect(handed.is_a?(Events::EventBus), "on_attached receives an EventBus wrapper") - expect(handed.handle.address == bus.handle.address, "handed bus is the same underlying object") - expect(handed.subscriber_count == 1, "handed bus is usable (got #{handed.subscriber_count})") - handed.close - handed.close - expect(handed.closed?, "handed bus wrapper closed (idempotently)") - expect(bus.subscriber_count == 1, "original bus survives closing the handed reference") - - second = Recorder.new - expect(bus.subscribe(second) == 2, "second subscribe returns count 2") - expect(second.attached.length == 1, "second on_attached called once") - second.attached[0].close - expect(live_callbacks == baseline + 2, "two live implementations registered") - - # publish: Accept from both subscribers -> 2 accepted; the Message record - # arrives fully decoded (seq, topic, text, tags). - expect(bus.publish("news", "hello", %w[x y]) == 2, "both subscribers accept news") - expect(first.routed == %w[news], "first.route saw news (got #{first.routed})") - expect(second.routed == %w[news], "second.route saw news (got #{second.routed})") - expect(first.messages.length == 1 && second.messages.length == 1, "each got one message") - msg = first.messages[0] - expect(msg.is_a?(Events::Message), "on_message receives a Message") - expect(msg.seq == 1, "seq starts at 1 (got #{msg.seq})") - expect(msg.topic == "news", "message topic (got #{msg.topic.inspect})") - expect(msg.text == "hello", "message text (got #{msg.text.inspect})") - expect(msg.tags == %w[x y], "message tags (got #{msg.tags})") - expect(second.messages[0] == msg, "both subscribers saw the same message (structural equality)") - - # Skip from the first subscriber -> only the second accepts. - expect(bus.publish("quiet", "psst", []) == 1, "skip lowers the accepted count") - expect(first.routed == %w[news quiet], "first.route saw quiet") - expect(first.messages.length == 1, "skipped subscriber got no message") - expect(second.messages.length == 2, "second subscriber got psst") - expect(second.messages[1].seq == 2 && second.messages[1].tags == [], "second message seq 2, no tags") - - # AcceptAndStop from the first subscriber -> delivered to it, second not asked. - expect(bus.publish("stop", "last", ["t"]) == 1, "accept-and-stop halts delivery") - expect(first.messages.length == 2, "stopping subscriber still receives") - expect(first.messages[1].text == "last", "stopping subscriber got last") - expect(second.routed == %w[news quiet], "later subscriber not routed after stop (got #{second.routed})") - expect(second.messages.length == 2, "later subscriber not delivered after stop") - - # Async publish: blocks the caller until the producer-thread completion fires; - # the subscriber callbacks run from that thread through the FFI dispatcher. - expect(bus.publish_later("later", "async") == 2, "publish_later resolves with the accepted count") - expect(first.messages.length == 3 && second.messages.length == 3, "async publish delivered to both") - expect(first.messages[2].topic == "later" && first.messages[2].tags == [], "async message decoded") - - # Iterator: every text in order, lazily, with early termination releasing the - # producer iterator; and the optional last_message record. - expect(bus.messages.to_a == %w[hello psst last async], "messages iterator (got #{bus.messages.to_a})") - expect(bus.messages.first(2) == %w[hello psst], "early break through first(2)") - collected = [] - bus.messages.each { |t| collected << t.upcase } - expect(collected == %w[HELLO PSST LAST ASYNC], "block iteration") - expect(bus.messages.each_with_index.to_a.last == ["async", 3], "Enumerator composes with each_with_index") - last = bus.last_message - expect(last == Events::Message.new(seq: 4, topic: "later", text: "async", tags: []), - "last_message (got seq=#{last.seq} topic=#{last.topic} text=#{last.text} tags=#{last.tags})") - - # A Ruby exception in route surfaces to the publishing caller as - # Events::Error with FOREIGN_ERROR_CODE and the exception text; the VM and - # the bus both keep working, and the subscriber stays attached. route is the - # producer's Result-returning callback method: the failure reaches the - # producer as an Err value it chooses to propagate, so the error is not a - # panic (-2) and the bus's state stays consistent. - failing = Recorder.new(fail_on: "boom", fail_in_message: "bad-body") - expect(bus.subscribe(failing) == 3, "third subscribe returns count 3") - failing.attached[0].close - begin - bus.publish("boom", "x", []) - raise "expected Events::Error from a raising route" - rescue Events::Error => e - expect(e.code == Events::FOREIGN_ERROR_CODE, "raising route -> code -4 (got #{e.code})") - expect(e.code != Events::PANIC_ERROR_CODE, "a Result-returning callback failure is not a panic") - expect(e.message.include?("boom: boom"), "foreign error carries the exception text (got #{e.message.inspect})") - end - expect(failing.routed == %w[boom], "failing subscriber's route ran") - expect(first.messages.length == 4, "earlier subscribers were delivered before the failure") - expect(bus.messages.to_a.last == "x", "the failed publish was still logged") - - # The same for an exception raised in on_message. - begin - bus.publish("body", "bad-body", []) - raise "expected Events::Error from a raising on_message" - rescue Events::Error => e - expect(e.code == -4, "raising on_message -> code -4 (got #{e.code})") - expect(e.message.include?("on_message rejected bad-body"), "on_message failure text (got #{e.message.inspect})") - end - expect(failing.messages.empty?, "raising on_message recorded nothing") - - expect(bus.publish("ok", "y", []) == 3, "bus still usable after foreign errors") - expect(failing.messages.length == 1 && failing.messages[0].text == "y", "failing subscriber accepts ok") - expect(bus.subscriber_count == 3, "subscriber_count == 3") - - # Non-ASCII text survives every string path: the C-string topic handed to - # route, the buffered Message, the iterator item, and the last_message record. - expect(bus.publish("тема ✓", "日本語", ["ü"]) == 3, "unicode publish accepted by all") - expect(first.routed.last == "тема ✓", "route received the unicode topic (got #{first.routed.last.inspect})") - expect(first.routed.last.encoding == Encoding::UTF_8, "route topic is UTF-8 (got #{first.routed.last.encoding})") - expect(first.messages.last.text == "日本語" && first.messages.last.tags == ["ü"], "unicode message fields") - expect(bus.messages.to_a.last == "日本語", "iterator item is unicode (got #{bus.messages.to_a.last.inspect})") - expect(bus.messages.to_a.last.encoding == Encoding::UTF_8, "iterator item is UTF-8") - expect(bus.last_message.topic == "тема ✓", "last_message topic is unicode") - - # The producer logs a message before routing, so failed publishes are logged. - expect(bus.messages.to_a == ["hello", "psst", "last", "async", "x", "bad-body", "y", "日本語"], - "log includes failed publishes (got #{bus.messages.to_a})") - - # route_once: a free function taking the callback interface; the producer - # drops its only reference on return, so `free` runs and the slot is released. - before = live_callbacks - probe = Recorder.new(skip: "quiet") - expect(Events.route_once(probe, "quiet") == Events::Delivery::SKIP, "route_once -> SKIP") - expect(Events.route_once(probe, "news") == Events::Delivery::ACCEPT, "route_once -> ACCEPT") - expect(probe.routed == %w[quiet news], "route_once called route with the topic") - expect(probe.attached.empty?, "route_once never attaches") - expect(live_callbacks == before, "route_once released its implementation (#{live_callbacks} vs #{before})") - - # A mixin default (NotImplementedError) is a foreign error too. - begin - Events.route_once(Bare.new, "x") - raise "expected Events::Error from the NotImplementedError default" - rescue Events::Error => e - expect(e.code == -4, "NotImplementedError default -> code -4 (got #{e.code})") - expect(e.message.include?("route is not implemented"), "default message (got #{e.message.inspect})") - end - expect(live_callbacks == before, "failing route_once released its implementation") - - # dup mints an independent wrapper (own strong reference) to the same bus. - twin = bus.dup - expect(twin.handle.address == bus.handle.address, "dup shares the object") - expect(twin.subscriber_count == 3, "dup sees the subscribers") - twin.close - expect(bus.subscriber_count == 3, "bus survives closing the dup") - - # clear_subscribers drops every retained implementation; each `free` runs - # synchronously, and publishing to an empty bus accepts nothing. - bus.clear_subscribers - expect(bus.subscriber_count.zero?, "no subscribers after clear") - expect(live_callbacks == baseline, "every implementation released after clear (#{live_callbacks} vs #{baseline})") - expect(bus.publish("after", "nobody", []) == 0, "publish to an empty bus accepts 0") - expect(first.messages.length == 7, "cleared subscriber no longer delivered (got #{first.messages.length})") - - # A subscriber held only by the bus is freed when the bus's last reference - # goes away. The reference handed through on_attached counts, so the bus - # outlives `short.close` until that wrapper is closed too. - short = Events::EventBus.new - tail = Recorder.new - short.subscribe(tail) - expect(live_callbacks == baseline + 1, "bus retains its subscriber") - short.close - expect(live_callbacks == baseline + 1, "the on_attached reference keeps the bus (and subscriber) alive") - expect(tail.attached[0].subscriber_count == 1, "bus usable through the on_attached reference alone") - tail.attached[0].close - expect(live_callbacks == baseline, "dropping the last bus reference frees its subscriber") - - # Closing a wrapper while a call on it is in flight (here from a callback - # the call made) defers the release until the call returns, so the producer - # never sees its object freed mid-call. - inflight = Events::EventBus.new - closer = Closer.new - inflight.subscribe(closer) - closer.attached[0].close - closer.target = inflight - expect(inflight.publish("t", "mid-call close", []) == 1, "publish completes despite the close") - expect(inflight.closed?, "the deferred close took effect") - expect(closer.messages.length == 1, "the subscriber still received the message") - expect(live_callbacks == baseline, "releasing the bus released its subscriber") - - # close is idempotent and a closed wrapper refuses further calls. - bus.close - bus.close - expect(bus.closed?, "closed? after close") - begin - bus.subscriber_count - raise "expected Error when using a closed EventBus" - rescue Events::Error => e - expect(e.message.include?("after close"), "use-after-close message (got #{e.message.inspect})") - end - - # Ruby-side wrappers that were never closed release at GC without double - # frees; force a couple of collections to shake that out. -end - -puts "ruby/events: OK" diff --git a/conformance/ruby/kvstore.rb b/conformance/ruby/kvstore.rb index 8379fc75..f991d8f9 100644 --- a/conformance/ruby/kvstore.rb +++ b/conformance/ruby/kvstore.rb @@ -1,389 +1,473 @@ # frozen_string_literal: true -# Conformance consumer: kvstore sample, Ruby target. + +# Conformance consumer: kvstore sample, the feature-complete producer, Ruby +# target. # -# Full-surface drive of the ABI 3 wrapper: the reference-counted Store -# interface (the open(path) factory constructor, sync methods, the -# Enumerator-backed list_keys, the blocking compact bridge over the async -# ABI with its cancel: token, the deprecated legacy_put, the -# default_capacity static, `close` and `dup` semantics), the typed KvError -# domain (codes 1001-1004) with per-code subclasses raised by throwing -# members, the Entry record as a plain value class decoded from a value -# buffer, a Ruby class including the generated `EvictionListener` mixin -# (attach, fire on delete and on expiry-on-read, detach by returning false, -# replace, clear, and a raising listener surfacing as Kvstore::Error with -# code -4), the object graph (share() wraps the SAME object, fork(), Store? -# both ways in larger(), a Store inside the StoreInfo record from -# describe(), [Store] from open_many(), and [Store] plus StoreInfo? as -# parameters to total_count()), and the nested module's -# `Kvstore.get_stats(store)`. The gem is installed from the generated tree; -# the cdylib is selected through KVSTORE_LIBRARY. +# The `Store` class (a throwing factory, the `new` constructor, methods, +# statics, the deprecated `size`, close), typed `KvError`s with their +# payload fields, records with bytes, optional, list, and map fields, lazy +# iterators of strings, records, and objects, three callback interfaces +# implemented in Ruby (a `Listener` that is retained, filtered, detached +# when it raises, and called from a producer thread during compaction; a +# `Policy` with a record return, a typed error raised back through `put`, +# object parameters and returns, and malformed returns; a `Loader` passed as +# an optional callback with string, bytes, and optional-object returns), +# objects in every buffered position, blocking async calls with `cancel:` +# tokens, the nested `kv.stats` module, the sibling `report` root, and +# leak-free teardown. `_wv_debug_live(1)` (live callbacks) shows when the +# producer released an implementation. require_relative 'support' require 'kvstore' -# Size of the generated handle table: with no async call pending, it holds -# exactly the live callback implementations, so it proves the producer's -# `free` entry ran exactly once. -def live_callbacks - Kvstore.instance_variable_get(:@wv_handles).size +Store = Kvstore::Store +EntryKind = Kvstore::EntryKind +KvError = Kvstore::KvError + +def callbacks + Kvstore._wv_debug_live(1) +end + +def put(store, key, value, kind = EntryKind::PERSISTENT, ttl = nil) + store.put(key, value.b, kind, ttl) end -# An eviction listener recording (key, reason) pairs; returns false (detach) -# once `keep_after` evictions were seen, and raises for `raise_on`. -class Listener - include Kvstore::EvictionListener +def entry(key, value, kind: EntryKind::PERSISTENT, version: 1) + Kvstore::Entry.new(key: key, value: value.b, kind: kind, version: version, expires_at: nil, tags: [], + metadata: {}) +end + +def key_not_found(key, msg) + e = expect_raise(KvError::KeyNotFound, msg) { yield } + expect(e.is_a?(KvError) && e.code == 1001 && e.key == key, "#{msg}: KeyNotFound for #{key} (got #{e.key})") + e +end - attr_reader :seen, :entries +# Runs the block with Ruby warnings off (the deprecated `size` warns). +def quietly + verbose = $VERBOSE + $VERBOSE = nil + yield +ensure + $VERBOSE = verbose +end - def initialize(keep_after: nil, raise_on: nil) - @keep_after = keep_after - @raise_on = raise_on - @seen = [] - @entries = [] +# A listener recording every change it's told about, and on which thread. +class Recorder + include Kvstore::Listener + + def initialize(skip: nil, fail_on: nil) + @skip = skip + @fail_on = fail_on + @lock = Mutex.new + @changes = [] + @threads = [] + end + + def accepts(key) + raise 'listener refused' if key == @fail_on + + key != @skip end - def on_evict(entry, reason) - @seen << [entry.key, reason] - @entries << entry - raise "listener exploded on #{entry.key}" if entry.key == @raise_on + def on_change(change) + @lock.synchronize do + @changes << change + @threads << Thread.current + end + end + + def changes + @lock.synchronize { @changes.dup } + end - @keep_after.nil? || @seen.length < @keep_after + def threads + @lock.synchronize { @threads.dup } + end + + def count(cls) + changes.count { |c| c.is_a?(cls) && (!block_given? || yield(c)) } end end -run_and_check_leaks(Kvstore) do - expect(Kvstore::ABI_VERSION == 3, "bindings target ABI revision 3") - expect(Kvstore::EvictionReason::DELETED.zero?, "EvictionReason::DELETED == 0") - expect(Kvstore::EvictionReason::EXPIRED == 1, "EvictionReason::EXPIRED == 1") - - # Interface static: a class method with no self slot. - expect(Kvstore::Store.default_capacity == 1_000_000, "default_capacity") - - # The open(path) factory throws the typed IoError (1004) on an empty path. - begin - Kvstore::Store.open("") - raise "expected KvError::IoError for empty path" - rescue Kvstore::KvError::IoError => e - expect(e.code == 1004, "IoError code == 1004 (got #{e.code})") - expect(e.is_a?(Kvstore::KvError), "IoError is a KvError") - expect(e.is_a?(Kvstore::Error), "domain errors subclass Kvstore::Error") +# A policy that rewrites admitted entries, vetoes secrets with a typed +# error, and routes `b/` keys to another store. +class Rewriter + include Kvstore::Policy + + attr_reader :admitted + + def initialize(other) + @other = other + @admitted = 0 end - store = Kvstore::Store.open("/tmp/conformance-kvstore-rb") - expect(store.is_a?(Kvstore::Store) && !store.closed?, "open returns an open Store") - payload = "\x01\x02\x03".b - - # put takes a buffered `i64?` TTL: nil and a present value both encode. - expect(store.put("alpha", payload, Kvstore::EntryKind::PERSISTENT, nil) == true, "put alpha") - expect(store.put("beta", payload, Kvstore::EntryKind::VOLATILE, 3600) == true, "put beta") - expect(store.count == 2, "count == 2") - - # Iterator-backed list-of-string return, optionally prefix-filtered (the - # `string?` prefix is a buffered optional parameter). The Enumerator is lazy - # and releases the producer iterator on early termination. - expect(store.list_keys(nil).is_a?(Enumerator), "list_keys returns an Enumerator") - keys = store.list_keys(nil).to_a - expect(keys == %w[alpha beta], "list_keys values in sorted order (got #{keys})") - expect(store.list_keys("al").to_a == %w[alpha], "list_keys prefix filter") - expect(store.list_keys("zz").to_a == [], "list_keys unmatched prefix is empty") - expect(store.list_keys(nil).first(1) == %w[alpha], "early break through first(1)") - seen = [] - store.list_keys(nil).each { |k| seen << k.upcase } - expect(seen == %w[ALPHA BETA], "block iteration") - - # Optional record return: get decodes a buffered `Entry?` into a plain value - # object covering every complex field type. - alpha = store.get("alpha") - expect(!alpha.nil?, "get alpha present") - expect(alpha.is_a?(Kvstore::Entry), "get returns an Entry value") - expect(alpha.id.positive?, "entry id positive") - expect(alpha.key == "alpha", "entry key") - expect(alpha.value == payload, "entry value bytes") - expect(alpha.expires_at.nil?, "alpha expires_at nil") - expect(alpha.tags == [], "alpha tags empty") - expect(alpha.metadata == {}, "alpha metadata empty") - - # Repeated gets decode fresh snapshots that compare structurally equal. - expect(store.get("alpha") == alpha, "second get equals first snapshot") - - beta = store.get("beta") - expect(!beta.expires_at.nil? && beta.expires_at.positive?, "beta expires_at present") - expect(beta.expires_at - beta.created_at == 3600, "beta TTL applied (got #{beta.expires_at - beta.created_at})") - - # A missing key raises the typed KeyNotFound (1001). - begin - store.get("missing") - raise "expected KvError::KeyNotFound" - rescue Kvstore::KvError::KeyNotFound => e - expect(e.code == 1001, "KeyNotFound code == 1001 (got #{e.code})") - expect(e.message == "key not found", "KeyNotFound default message (got #{e.message.inspect})") + def admit(entry) + @admitted += 1 + raise "admit saw version #{entry.version}" unless entry.version.zero? + + case entry.key + when /\Asecret/ + raise KvError::Rejected.new('secrets are not stored', key: entry.key, reason: 'no secrets') + when /\Aboom/ + raise 'policy exploded' + when /\Agarbage/ + # An undeclared EntryKind: the producer rejects the record with -3. + Kvstore::Entry.new(key: entry.key, value: entry.value, kind: 9, version: 0, expires_at: nil, tags: [], + metadata: {}) + else + Kvstore::Entry.new(key: 'renamed', value: entry.value, kind: EntryKind::ENCRYPTED, version: entry.version, + expires_at: entry.expires_at, tags: ['admitted'], metadata: entry.metadata) + end end - # Rescuing the domain base class catches any code in the domain. - begin - store.get("missing") - raise "expected KvError" - rescue Kvstore::KvError => e - expect(e.code == 1001, "domain rescue sees KeyNotFound (got #{e.code})") + def route(key, home) + case key + when %r{\Ab/} then @other + when %r{\Anull/} then nil # a required Store: the producer rejects it with -3 + else home + end end +end + +# A loader with an optional fallback store. +class Source + include Kvstore::Loader - # The deprecated method still works (and warns on stderr). - expect(store.legacy_put("legacy", payload) == true, "legacy_put") - expect(store.delete("legacy") == true, "delete legacy") - expect(store.delete("legacy") == false, "second delete returns false") - - # Entry is a plain value class: keyword construction, non-empty native - # list/map fields, a nil-able optional, and structural equality. - built = Kvstore::Entry.new( - id: 7, - key: "built", - value: payload, - created_at: 1000, - expires_at: nil, - tags: %w[hot fast], - metadata: { "source" => "test", "env" => "prod" } - ) - expect(built.tags.sort == %w[fast hot], "built tags (got #{built.tags})") - expect(built.metadata == { "source" => "test", "env" => "prod" }, "built metadata") - expect(built.expires_at.nil?, "built expires_at nil") - same = Kvstore::Entry.new( - id: 7, key: "built", value: payload, created_at: 1000, - expires_at: nil, tags: %w[hot fast], - metadata: { "source" => "test", "env" => "prod" } - ) - expect(built == same, "entries compare structurally") - expect(built != alpha, "different entries are unequal") - - # The cross-module call under test: kv.stats.get_stats borrows the kv.Store - # interface pointer and returns the Stats record as a value. - stats = Kvstore.get_stats(store) - expect(stats.is_a?(Kvstore::Stats), "get_stats returns a Stats value") - expect(stats.total_entries == 2, "total_entries == 2 (got #{stats.total_entries})") - expect(stats.total_bytes == 6, "total_bytes == 6 (got #{stats.total_bytes})") - expect(stats.expired_entries == 0, "expired_entries == 0 (got #{stats.expired_entries})") - - # --- Eviction listener (callback interface) ------------------------------ - baseline = live_callbacks - - listener = Listener.new - store.set_eviction_listener(listener) - expect(live_callbacks == baseline + 1, "listener registered") - - # delete fires on_evict synchronously with the removed Entry and DELETED. - expect(store.delete("beta") == true, "delete beta") - expect(listener.seen == [["beta", Kvstore::EvictionReason::DELETED]], "eviction fired for beta (got #{listener.seen})") - evicted = listener.entries[0] - expect(evicted.is_a?(Kvstore::Entry), "on_evict receives an Entry value") - expect(evicted.value == payload && evicted.expires_at == beta.expires_at, "evicted entry matches the stored one") - expect(evicted == beta, "evicted entry equals the earlier snapshot") - - # An entry whose TTL already elapsed is evicted on read: get raises Expired - # (1002) and the listener sees EXPIRED. - expect(store.put("stale", "zz".b, Kvstore::EntryKind::VOLATILE, -1) == true, "put stale") - begin - store.get("stale") - raise "expected KvError::Expired" - rescue Kvstore::KvError::Expired => e - expect(e.code == 1002, "Expired code == 1002 (got #{e.code})") + def initialize(fallback = nil) + @store = fallback end - expect(listener.seen.last == ["stale", Kvstore::EvictionReason::EXPIRED], "expiry eviction (got #{listener.seen.last})") - expect(listener.entries.last.value == "zz".b, "expired entry payload") - expect(store.count == 1, "stale entry gone after eviction") - - # Replacing the listener frees the previous one; clearing frees the current. - second = Listener.new - store.set_eviction_listener(second) - expect(live_callbacks == baseline + 1, "replaced listener freed, replacement registered") - store.put("gamma", payload, Kvstore::EntryKind::VOLATILE, nil) - store.delete("gamma") - expect(listener.seen.length == 2, "replaced listener no longer notified") - expect(second.seen == [["gamma", Kvstore::EvictionReason::DELETED]], "replacement notified (got #{second.seen})") - store.clear_eviction_listener - expect(live_callbacks == baseline, "clear_eviction_listener freed the listener") - store.put("delta", payload, Kvstore::EntryKind::VOLATILE, nil) - store.delete("delta") - expect(second.seen.length == 1, "no notification after clear") - store.clear_eviction_listener - - # Returning false detaches (and frees) the listener after that eviction. - brief = Listener.new(keep_after: 2) - store.set_eviction_listener(brief) - store.put("one", payload, Kvstore::EntryKind::VOLATILE, nil) - store.put("two", payload, Kvstore::EntryKind::VOLATILE, nil) - store.put("three", payload, Kvstore::EntryKind::VOLATILE, nil) - store.delete("one") - expect(live_callbacks == baseline + 1, "listener retained after returning true") - store.delete("two") - expect(live_callbacks == baseline, "listener freed after returning false") - store.delete("three") - expect(brief.seen.map(&:first) == %w[one two], "detached listener misses the third eviction (got #{brief.seen})") - - # A Ruby exception inside on_evict surfaces to the caller of delete as the - # brand error with FOREIGN_ERROR_CODE (-4), not as a KvError; the entry was - # already removed, the VM is fine, and the store keeps working. - angry = Listener.new(raise_on: "kaboom") - store.set_eviction_listener(angry) - store.put("kaboom", payload, Kvstore::EntryKind::VOLATILE, nil) - begin - store.delete("kaboom") - raise "expected Kvstore::Error from a raising listener" - rescue Kvstore::KvError - raise "a foreign error must not be mapped to a KvError" - rescue Kvstore::Error => e - expect(e.code == Kvstore::FOREIGN_ERROR_CODE, "raising listener -> code -4 (got #{e.code})") - expect(e.code == -4, "FOREIGN_ERROR_CODE is -4") - expect(e.message.include?("listener exploded on kaboom"), "foreign error text (got #{e.message.inspect})") + + def name + 'ruby-loader' end - expect(store.count == 1, "entry removed despite the listener failure (got #{store.count})") - expect(store.delete("kaboom") == false, "kaboom is gone") - expect(store.put("calm", payload, Kvstore::EntryKind::VOLATILE, nil) && store.delete("calm"), - "store usable after a foreign error") - expect(angry.seen.map(&:first) == %w[kaboom calm], "raising listener stays attached (got #{angry.seen})") - store.clear_eviction_listener - expect(live_callbacks == baseline, "all listeners released") - - # --- Object graph -------------------------------------------------------- - # share() returns a wrapper to the SAME object: writes through one are visible - # through the other, and closing one wrapper leaves the object alive. - shared = store.share - expect(shared.is_a?(Kvstore::Store), "share returns a Store") - expect(!shared.equal?(store), "share returns a distinct wrapper") - expect(shared.handle.address == store.handle.address, "share wraps the same object") - expect(shared.put("via-shared", payload, Kvstore::EntryKind::VOLATILE, nil), "put through the shared wrapper") - expect(store.count == 2, "write through share visible in the original (got #{store.count})") - expect(store.get("via-shared").value == payload, "entry readable through the original") - shared.close - shared.close - expect(shared.closed?, "shared wrapper closed idempotently") - expect(store.count == 2, "object survives closing the shared wrapper") - - # dup and clone also mint independent wrappers with their own reference. - twin = store.dup - expect(twin.handle.address == store.handle.address, "dup wraps the same object") - expect(twin.count == 2, "dup sees the same entries") - twin.close - expect(store.count == 2, "object survives closing the dup") - - # fork() is a new, independent object. - forked = store.fork - expect(forked.handle.address != store.handle.address, "fork is a different object") - expect(forked.count == 2, "fork copied the live entries (got #{forked.count})") - forked.put("only-in-fork", payload, Kvstore::EntryKind::VOLATILE, nil) - expect(forked.count == 3 && store.count == 2, "fork is independent of the original") - - # larger(): Store? in and out. - empty = Kvstore::Store.open("/tmp/conformance-kvstore-rb-empty") - expect(empty.larger(nil).nil?, "larger(nil) on an empty store is nil") - own = store.larger(nil) - expect(!own.nil? && own.handle.address == store.handle.address, "larger(nil) on a non-empty store is itself") - own.close - bigger = store.larger(forked) - expect(bigger.handle.address == forked.handle.address, "larger picks the fuller store") - bigger.close - still = forked.larger(empty) - expect(still.handle.address == forked.handle.address, "larger keeps self when other is smaller") - still.close - expect(forked.count == 3 && empty.count.zero?, "larger left both stores intact") - - # describe(): a record carrying the object itself plus an optional object. - info = store.describe("primary", nil) - expect(info.is_a?(Kvstore::StoreInfo), "describe returns a StoreInfo") - expect(info.label == "primary", "info.label (got #{info.label.inspect})") - expect(info.count == 2, "info.count (got #{info.count})") - expect(info.mirror.nil?, "info.mirror nil when absent") - expect(info.store.is_a?(Kvstore::Store), "info.store is a Store wrapper") - expect(info.store.handle.address == store.handle.address, "info.store is the described object") - expect(info.store.count == 2, "info.store is usable") - mirrored = store.describe("mirrored", forked) - expect(!mirrored.mirror.nil? && mirrored.mirror.handle.address == forked.handle.address, "info.mirror is the passed store") - expect(mirrored.mirror.count == 3, "info.mirror is usable") - expect(forked.count == 3, "borrowed mirror parameter still alive") - - # open_many(): a list of objects as a return; a bad path raises the typed error - # for the whole call. - many = Kvstore::Store.open_many(["/tmp/a", "/tmp/b", "/tmp/c"]) - expect(many.is_a?(Array) && many.length == 3, "open_many returns 3 stores") - expect(many.all? { |s| s.is_a?(Kvstore::Store) && !s.closed? }, "every element is an open Store") - expect(many.map { |s| s.handle.address }.uniq.length == 3, "open_many stores are distinct objects") - many[0].put("m0", payload, Kvstore::EntryKind::VOLATILE, nil) - many[2].put("m2a", payload, Kvstore::EntryKind::VOLATILE, nil) - many[2].put("m2b", payload, Kvstore::EntryKind::VOLATILE, nil) - expect(many.map(&:count) == [1, 0, 2], "open_many stores hold independent state") - expect(Kvstore::Store.open_many([]) == [], "open_many of nothing is empty") - begin - Kvstore::Store.open_many(["/tmp/ok", ""]) - raise "expected KvError::IoError from open_many" - rescue Kvstore::KvError::IoError => e - expect(e.code == 1004, "open_many IoError code (got #{e.code})") + + def fallback(key) + key == 'fb' ? @store : nil end - # total_count(): a list of objects and an optional record holding an object as - # parameters (the encoder mints one reference per object token). - expect(Kvstore::Store.total_count([], nil).zero?, "total_count of nothing") - expect(Kvstore::Store.total_count(many, nil) == 3, "total_count over open_many (got #{Kvstore::Store.total_count(many, nil)})") - expect(Kvstore::Store.total_count(many, info) == 5, "total_count with the described store (got #{Kvstore::Store.total_count(many, info)})") - expect(Kvstore::Store.total_count([store, store, forked], nil) == 7, "the same object may appear repeatedly") - ruby_info = Kvstore::StoreInfo.new(label: "ruby-built", store: forked, mirror: store, count: 0) - expect(Kvstore::Store.total_count([empty], ruby_info) == 3, "Ruby-built StoreInfo encodes its object fields") - expect(Kvstore::Store.total_count([], mirrored) == 2, "StoreInfo with a mirror encodes both objects") - expect(store.count == 2 && forked.count == 3 && many.map(&:count) == [1, 0, 2], - "borrowed parameter objects survive total_count") - - # --- Async --------------------------------------------------------------- - # An immediately-expired entry gives compact 3 bytes to reclaim; the wrapper - # blocks on a Queue fed from the producer's worker-thread callback. - expect(store.put("doomed", payload, Kvstore::EntryKind::VOLATILE, 0) == true, "put doomed") - reclaimed = store.compact - expect(reclaimed == 3, "compact reclaimed 3 bytes (got #{reclaimed})") - expect(store.compact.zero?, "second compact reclaims nothing") - expect(store.count == 2, "live entries untouched by compact") - - # compact is cancellable: a token cancelled before the call completes it - # with Kvstore::Cancelled (code -5); an untouched token changes nothing. - token = Kvstore::CancelToken.new - expect(!token.cancelled?, "fresh token is not cancelled") - expect(store.compact(cancel: token).zero?, "compact with an idle token") - token.cancel - token.cancel - expect(token.cancelled?, "token reports cancelled") - begin - store.compact(cancel: token) - raise "expected Kvstore::Cancelled" - rescue Kvstore::Cancelled => e - expect(e.code == Kvstore::CANCELLED_ERROR_CODE && e.code == -5, "Cancelled code -5 (got #{e.code})") - expect(e.is_a?(Kvstore::Error), "Cancelled is a Kvstore::Error") + def load(key) + case key + when 'missing' then raise KvError::KeyNotFound.new('not in the loader', key: 'missing') + when 'elsewhere' then raise KvError::KeyNotFound.new('not in the loader', key: 'other') + when 'broken' then raise 'loader is broken' + else "loaded:#{key}".b + end end +end + +def constructors + e = expect_raise(KvError::InvalidPath, 'open("")') { Store.open('') } + expect(e.code == 1004 && KvError::InvalidPath::CODE == 1004 && e.message == 'invalid path', 'InvalidPath') + expect(e.instance_variables == [:@code], 'InvalidPath carries no payload fields') + s = Store.new + expect(s.path == 'memory', 'Store.new is in memory') + expect(s.capacity == Store.default_capacity && Store.default_capacity == 1_000_000, 'default capacity') + s.close + opened = Kvstore.open_store('/async') + expect(opened.is_a?(Store) && opened.path == '/async', 'open_store returns a Store') + opened.close + expect_raise(KvError::InvalidPath, 'open_store("")') { Kvstore.open_store('') } +end + +def basics + s = Store.open('/basics') + expect(put(s, 'alpha', 'one') == entry('alpha', 'one'), 'put returns the stored entry') + second = put(s, 'alpha', 'two', EntryKind::VOLATILE) + expect(second.version == 2 && second.kind == EntryKind::VOLATILE, 'a second put bumps the version') + expect(s.get('alpha').value == 'two'.b, 'get') + e = key_not_found('nope', 'get a missing key') { s.get('nope') } + expect(e.message == 'key not found: nope', "the producer's message (got #{e.message})") + expect(s.find('alpha')&.version == 2 && s.find('nope').nil?, 'find') + + expect(put(s, 'ttl', 'x', EntryKind::VOLATILE, 10).expires_at == 10, 'a ttl sets expires_at') + expect(s.now.zero?, 'the clock starts at zero') + expect(s.tick(9) == 9 && s.count == 2, 'tick(9)') + expect(s.tick(1) == 10 && s.count == 1, 'tick(1) expires the entry') + e = expect_raise(KvError::Expired, 'get an expired key') { s.get('ttl') } + expect(e.key == 'ttl' && e.expired_at == 10 && e.message == 'entry ttl expired at 10', 'Expired payload') + key_not_found('ttl', 'the expired read removed it') { s.get('ttl') } + + s.set_capacity(1) + expect(s.capacity == 1, 'set_capacity') + put(s, 'alpha', 'three') + e = expect_raise(KvError::StoreFull, 'a new key past the capacity') { put(s, 'beta', 'b') } + expect(e.capacity == 1 && e.code == 1003, 'StoreFull payload') + s.set_capacity(100) + e = expect_raise(Kvstore::Error, 'an undeclared enum value') { put(s, 'k', 'v', 9) } + expect(!e.is_a?(KvError) && e.code == -3, "an undeclared enum value is a marshalling failure (got #{e.code})") + + put(s, 'beta', 'b') + expect(s.delete('beta') == true && s.delete('beta') == false, 'delete') + expect(quietly { s.size } == s.count && s.count == 1, 'the deprecated size') + expect(s.clear == 1 && s.count.zero?, 'clear') + s.close + closed = expect_raise(Kvstore::Error, 'a closed store') { s.count } + expect(closed.message.include?('used after close'), closed.message) + expect_raise(TypeError, 'a store of the wrong type') { Kvstore.summarize('not a store') } +end + +def iterators + s = Store.open('/iter') + put(s, 'user.bob', 'b') + put(s, 'user.alice', 'a') + put(s, 'sys.x', 'xx') + + expect(s.keys.to_a == ['sys.x', 'user.alice', 'user.bob'], 'keys in order') + key_not_found('zzz', 'a prefix that matches nothing') { s.keys('zzz').to_a } + expect(s.keys('user.').first == 'user.alice', 'a lazy first key') + expect(Kvstore._wv_debug_live(2).zero?, 'abandoning an iterator releases it') + + entries = s.entries('sys.').to_a + expect(entries.length == 1 && entries[0].key == 'sys.x' && entries[0].value == 'xx'.b, 'entries') + + prefixes = ['user.', 'sys.', 'none.'] + parts = s.partition(prefixes).to_a + expect(parts.length == 3, 'one store per prefix') + parts.each_with_index do |p, i| + expect(p.is_a?(Store) && p.path == prefixes[i], "partition #{i} path") + expect(p.count == [2, 1, 0][i], "partition #{i} count") + p.close + end + s.close +end + +def listeners(main) + base = callbacks + s = Store.open('/listen') + l = Recorder.new(skip: 'quiet') + id = s.subscribe(l) + expect(id.positive? && s.listener_count == 1 && callbacks == base + 1, 'subscribe') + + put(s, 'a', '1') + last = l.changes.last + expect(last.is_a?(Kvstore::Change::Put) && last.entry.version == 1 && last.replaced == false, 'a new key') + put(s, 'a', '2') + last = l.changes.last + expect(last.is_a?(Kvstore::Change::Put) && last.entry.version == 2 && last.replaced == true, 'a replaced key') + put(s, 'quiet', 'x') + expect(l.count(Kvstore::Change::Put) == 2, 'accepts filters a key') + s.delete('a') + expect(l.changes.last == Kvstore::Change::Removed.new(key: 'a', expired: false), 'a delete') + put(s, 'short', 'x', EntryKind::VOLATILE, 1) + s.tick(1) + expect_raise(KvError::Expired, 'an expired read') { s.get('short') } + expect(l.changes.last == Kvstore::Change::Removed.new(key: 'short', expired: true), 'an expired read notifies') + expect(s.clear == 1, 'clear leaves quiet') + expect(l.changes.last == Kvstore::Change::Cleared.new(count: 1), 'a clear') + expect(l.threads.all? { |t| t == main }, 'synchronous calls notify on the calling thread') + + expect(s.unsubscribe(id) == true && callbacks == base, 'unsubscribe releases the listener') + expect(s.unsubscribe(id) == false && s.listener_count.zero?, 'a second unsubscribe') + + # A listener that raises is detached (and released); the put succeeds. + failing = Recorder.new(fail_on: 'boom') + s.subscribe(failing) + put(s, 'fine', '1') + expect(failing.count(Kvstore::Change::Put) == 1, 'the failing listener sees fine') + put(s, 'boom', '1') + expect(s.count == 2 && s.listener_count.zero? && callbacks == base, 'a raising listener is detached') + + # Releasing the store releases the listeners it still holds. + s.subscribe(Recorder.new) + s.subscribe(Recorder.new) + expect(s.listener_count == 2 && callbacks == base + 2, 'two more listeners') + s.close + expect(callbacks == base, 'closing the store releases its listeners') + expect_raise(TypeError, 'a nil listener') { Store.new.subscribe(nil) } +end + +def policies + base = callbacks + s = Store.open('/policy') + other = Store.open('/other') + p = Rewriter.new(other) + s.set_policy(p) + expect(s.has_policy && callbacks == base + 1, 'set_policy') + + e = put(s, 'a', '1', EntryKind::VOLATILE) + expect(e.key == 'a' && e.version == 1 && e.kind == EntryKind::ENCRYPTED, "admit's rewrite is stored") + expect(e.tags == ['admitted'], "admit's tags") + put(s, 'b/x', '2', EntryKind::VOLATILE) + expect(s.count == 1 && other.count == 1, 'route redirects a write') + + e = expect_raise(KvError::Rejected, 'a typed error from admit') { put(s, 'secret', '3') } + expect(e.code == 1005 && e.key == 'secret' && e.reason == 'no secrets', 'Rejected payload') + expect(e.message == 'secrets are not stored', "the policy's message (got #{e.message})") + e = expect_raise(Kvstore::Error, 'any other exception') { put(s, 'boom', '4') } + expect(!e.is_a?(KvError) && e.code == -4 && e.message == 'policy exploded', "-4 (got #{e.code}: #{e.message})") + e = expect_raise(Kvstore::Error, 'a malformed admit return') { put(s, 'garbage', '5') } + expect(!e.is_a?(KvError) && e.code == -3, "a malformed record is -3 (got #{e.code}: #{e.message})") + e = expect_raise(Kvstore::Error, 'a nil route return') { put(s, 'null/x', '6') } + expect(!e.is_a?(KvError) && e.code == -3, "a null Store is -3 (got #{e.code}: #{e.message})") + expect(s.count == 1 && other.count == 1 && p.admitted == 6, 'failed puts change nothing') + + s.set_policy(Rewriter.new(other)) + expect(callbacks == base + 1, 'replacing the policy releases the old one') + s.set_policy(nil) + expect(!s.has_policy && callbacks == base, 'set_policy(nil) releases it') + put(s, 'secret', 'now allowed') + expect(s.count == 2, 'no policy, no veto') + other.close + s.close +end + +def loaders + base = callbacks + s = Store.open('/load') + expect(s.get_or_load('k').nil?, 'no loader, no entry') + loaded = s.get_or_load('k', Source.new) + expect(loaded.value == 'loaded:k'.b && loaded.kind == EntryKind::VOLATILE, 'a loaded entry') + expect(loaded.metadata == { 'source' => 'ruby-loader' }, 'the loader name is recorded') + expect(callbacks == base, 'the loader is released after the call') + expect(s.get_or_load('k', Source.new)&.version == 1, 'a hit skips the loader') + + backup = Store.open('/backup') + put(backup, 'fb', 'from backup') + copied = s.get_or_load('fb', Source.new(backup)) + expect(copied.value == 'from backup'.b && copied.kind == EntryKind::PERSISTENT, 'the fallback store') + backup.close + + expect(s.get_or_load('missing', Source.new).nil?, 'KeyNotFound for this key is none') + e = key_not_found('other', 'KeyNotFound for another key passes through') { s.get_or_load('elsewhere', Source.new) } + expect(e.message == 'not in the loader', "the loader's message (got #{e.message})") + e = expect_raise(Kvstore::Error, 'any other loader failure') { s.get_or_load('broken', Source.new) } + expect(!e.is_a?(KvError) && e.code == -4 && e.message == 'loader is broken', "-4 (got #{e.code}: #{e.message})") + expect(callbacks == base, 'every loader is released') + s.close +end + +def async_calls(main) + s = Store.open('/async-calls') + l = Recorder.new + s.subscribe(l) + put(s, 'old1', 'x', EntryKind::VOLATILE, 1) + put(s, 'old2', 'x', EntryKind::VOLATILE, 1) + put(s, 'keep', 'x') + s.tick(5) + + token = Kvstore::CancelToken.new + expect(s.compact(0, cancel: token) == 2, 'compact removes the expired entries') token.close - token.close - expect(store.count == 2, "a cancelled compact leaves the store intact") - - # clear drops everything left in one call. - store.clear - expect(store.count.zero?, "count == 0 after clear") - expect(store.list_keys(nil).to_a == [], "no keys after clear") - - # --- Release ------------------------------------------------------------- - # Every wrapper closes exactly once; a second close is a no-op; the object - # itself lives until its last wrapper (info.store, mirrored.store, ...) goes. - store.close - store.close - expect(store.closed?, "store closed") - begin - store.count - raise "expected Error when using a closed Store" - rescue Kvstore::Error => e - expect(e.message.include?("after close"), "use-after-close message (got #{e.message.inspect})") + expired = l.changes.each_index.select { |i| l.changes[i].is_a?(Kvstore::Change::Removed) && l.changes[i].expired } + expect(expired.length == 2 && s.count == 1, 'the listener saw both removals') + expect(expired.all? { |i| l.threads[i] != main }, 'compaction notifies on a producer thread') + expect(s.compact(5).zero?, 'compact without a token') + cancelled = Kvstore::CancelToken.new + cancelled.cancel + expect(cancelled.cancelled?, 'a cancelled token stays cancelled') + expect_raise(Kvstore::Cancelled, 'an already-cancelled token') { s.compact(0, cancel: cancelled) } + cancelled.close + + token = Kvstore::CancelToken.new + canceller = Thread.new do + sleep 0.02 + jobs = Store.active_jobs + token.cancel + jobs end - expect(info.store.count.zero?, "object still alive through the record's reference") - info.store.close - mirrored.store.close - mirrored.mirror.close - expect(forked.count == 3, "fork still alive after the mirror wrapper closed") - forked.close - empty.close - many.each(&:close) - many.each(&:close) - expect(many.all?(&:closed?), "every open_many store closed") + started = Time.now + e = expect_raise(Kvstore::Cancelled, 'cancelled mid-pause') { s.compact(60_000, cancel: token) } + expect(e.code == -5 && e.is_a?(Kvstore::Error), 'Cancelled carries -5') + expect(Time.now - started < 5, 'cancellation is prompt') + expect(canceller.value >= 1, 'the pause was running') + expect(eventually { Store.active_jobs.zero? }, 'the cancelled pause stopped cooperatively') + token.close + + results = Array.new(32) { Thread.new { s.get_many(%w[keep gone keep]) } }.map(&:value) + expect(results.all? { |r| r.length == 3 && r[0]&.key == 'keep' && r[1].nil? && r[2]&.key == 'keep' }, + '32 concurrent get_many calls') + + other = Store.open('/other') + put(other, 'a', '123') + stats = Kvstore.summarize_all([s, other]) + expect(stats == Kvstore::Stats.new(entries: 2, bytes: 4, by_kind: { EntryKind::PERSISTENT => 2 }), 'summarize_all') + other.close + s.close +end + +def object_graph + s = Store.open('/graph') + put(s, 'k', 'v') + + shared = s.share + put(s, 'via-original', 'v') + expect(!shared.find('via-original').nil?, 'share is the same object') + s.delete('via-original') + s.close + expect(shared.count == 1, 'still alive through the shared reference') + + fork = shared.fork + expect(fork.count == 1 && fork.path == '/graph', 'fork copies the entries') + put(fork, 'k2', 'v') + expect(fork.count == 2 && shared.count == 1, 'fork is distinct') + + empty = Store.open('/empty') + expect(empty.larger.nil?, 'larger(nil) on an empty store') + bigger = empty.larger(fork) + expect(bigger.count == 2, 'larger picks the fork') + bigger.close + itself = shared.larger + put(itself, 'via-larger', 'v') + expect(shared.count == 2, 'larger(nil) on a non-empty store is itself') + itself.delete('via-larger') + itself.close + + info = shared.describe('main', fork) + expect(info.label == 'main' && info.count == 1, 'describe') + expect(info.store.count == 1 && info.mirror.count == 2, 'describe carries both stores') + put(info.store, 'via-info', 'v') + expect(shared.count == 2, 'describe carries this store itself') + shared.delete('via-info') + + opened = Store.open_many(['/a', '/b']) + expect(opened.map(&:path) == ['/a', '/b'], 'open_many') + expect_raise(KvError::InvalidPath, 'open_many with a bad path') { Store.open_many(['/a', '']) } + + first = Kvstore::StoreInfo.new(label: 'first', store: opened[0], mirror: nil, count: 0) + named = Store.by_label([info, first]) + expect(named.keys.sort == %w[first main], 'by_label keys') + expect(named['main'].count == 1 && named['first'].path == '/a', 'by_label values') + + put(opened[0], 'm', '1') + expect(Store.total_count([opened[0], opened[1], fork], named, info) == 6, 'total_count with extra') + expect(Store.total_count([opened[0], opened[1], fork], named) == 5, 'total_count without extra') + expect(shared.count == 1 && fork.count == 2 && opened[0].count == 1, 'still usable after buffers') + + [info.store, info.mirror, *named.values, *opened, empty, fork, shared].each(&:close) +end + +def stats_and_report + s = Store.open('/stats') + put(s, 'b', '12') + put(s, 'a', '1') + put(s, 'a', '123') + put(s, 'c', 'x', EntryKind::ENCRYPTED) + expected = Kvstore::Stats.new(entries: 3, bytes: 6, + by_kind: { EntryKind::PERSISTENT => 2, EntryKind::ENCRYPTED => 1 }) + expect(Kvstore.summarize(s) == expected, 'summarize') + key_not_found('q', 'summarize a prefix that matches nothing') { Kvstore.summarize(s, 'q') } + lines = Kvstore.render_report(s.entries.to_a) + expect(lines == ['a: 3 bytes, Persistent, v2', 'b: 2 bytes, Persistent', 'c: 1 bytes, Encrypted'], + "render_report (got #{lines})") + e = expect_raise(Kvstore::ReportError::NothingToReport, 'render_report([])') { Kvstore.render_report([]) } + expect(e.is_a?(Kvstore::ReportError) && e.code == 2001 && e.message == 'nothing to report', 'NothingToReport') + s.close +end + +run_and_check_leaks(Kvstore) do + main = Thread.current + expect(Kvstore::ABI_VERSION == 4, 'bindings target ABI revision 4') + constructors + basics + iterators + listeners(main) + policies + loaders + async_calls(main) + object_graph + stats_and_report + expect(callbacks.zero?, 'every callback implementation was released') end -puts "ruby/kvstore: OK" +puts 'ruby/kvstore: OK' diff --git a/conformance/ruby/run.sh b/conformance/ruby/run.sh index 4e226e40..70cc195b 100755 --- a/conformance/ruby/run.sh +++ b/conformance/ruby/run.sh @@ -5,6 +5,7 @@ # one generated gem and runs one consumer against it, which must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools ruby ruby gem # Build the sample's generated gem with `gem build`, install it into a # private gem home, and run the consumer against the installed gem. The @@ -25,22 +26,11 @@ rb_consumer() { } ruby_calculator() { rb_consumer calculator calculator.rb; } - -ruby_contacts() { rb_consumer contacts contacts.rb; } - -ruby_events() { rb_consumer events events.rb; } - -ruby_kvstore() { rb_consumer kvstore kvstore.rb; } - -ruby_async_demo() { rb_consumer async-demo async_demo.rb; } - -ruby_codec() { rb_consumer codec codec.rb; } +ruby_codec() { rb_consumer codec codec.rb; } +ruby_kvstore() { rb_consumer kvstore kvstore.rb; } lane ruby-calculator ruby_calculator -lane ruby-contacts ruby_contacts -lane ruby-events ruby_events -lane ruby-kvstore ruby_kvstore -lane ruby-async-demo ruby_async_demo lane ruby-codec ruby_codec +lane ruby-kvstore ruby_kvstore finish_lanes diff --git a/conformance/ruby/support.rb b/conformance/ruby/support.rb index 96f448af..c7e66653 100644 --- a/conformance/ruby/support.rb +++ b/conformance/ruby/support.rb @@ -6,16 +6,41 @@ def expect(cond, msg) raise "assertion failed: #{msg}" unless cond end +# Runs the block, which must raise a `cls`, and returns the exception. +def expect_raise(cls, msg) + yield +rescue cls => e + e +else + raise "assertion failed: #{msg}: expected #{cls}, nothing was raised" +end + +# Polls the block (up to about two seconds) until it's truthy. +def eventually + 200.times do + return true if yield + + sleep 0.01 + end + false +end + +# The 64-bit FNV-1a hash the contract tables use. +def fnv1a64(text) + text.bytes.reduce(0xcbf29ce484222325) { |h, b| ((h ^ b) * 0x100000001b3) & 0xffff_ffff_ffff_ffff } +end + # Runs the consumer body on its own thread, so none of its locals survive on # a live stack once it returns, then forces collection (and the # AutoPointer finalizers that release unclosed wrappers) and asserts the # producer reports no live native resources: objects, callbacks, iterators, -# cancel tokens, or returned byte allocations. +# cancel tokens, or byte runs. def run_and_check_leaks(mod) + expect(mod._wv_debug_live(-1) == 1, 'the library counts live resources') Thread.new { yield }.join - kinds = %w[objects callbacks iterators tokens allocations] + kinds = %w[objects callbacks iterators tokens runs] live = nil - 50.times do + 100.times do GC.start(full_mark: true, immediate_sweep: true) live = (0..4).map { |k| mod._wv_debug_live(k) } break if live.all?(&:zero?) diff --git a/conformance/run.sh b/conformance/run.sh index 8fd7a3cc..e73fbd60 100755 --- a/conformance/run.sh +++ b/conformance/run.sh @@ -5,24 +5,28 @@ # results, exiting non-zero on any mismatch. This driver: # # 1. builds each sample producer cdylib (with leak counters enabled), -# 2. runs `weaveffi generate` on each sample's annotated `src/lib.rs` into -# the cargo target dir, and +# 2. runs `weaveffi generate` on each sample crate into the cargo target +# dir, reading the API from that same leak-check build (`--library`), +# and # 3. runs conformance//run.sh for every language, each of which -# compiles and runs one consumer per sample under a per-lane timeout. +# declares one lane per sample (`-calculator`, `-codec`, +# `-kvstore`) and compiles and runs that sample's consumer under a +# per-lane timeout. # # Selection (comma-separated lane names like `python-kvstore`, or languages # like `python`): -# ONLY=c,python-kvstore run only these +# ONLY=c,python-kvstore run only these (and generate only their languages) # SKIP=wasm skip these # SKIP_GEN=1 reuse previously generated bindings # LANE_TIMEOUT=600 per-lane limit in seconds (default 300) # -# A missing toolchain fails the affected lanes; skip them explicitly with -# SKIP=. +# A missing toolchain fails its language under CI (CI=true) and skips it, +# with a note, otherwise (see require_tools in conformance/lib.sh). Skip a +# language explicitly with SKIP=. set -uo pipefail . "$(dirname "$0")/lib.sh" -SAMPLES="calculator contacts events kvstore async-demo codec" +SAMPLES="calculator codec kvstore" LANGS="c cpp python ruby dart go swift dotnet node kotlin wasm" export ROOT TARGET_DIR LIBDIR GENROOT OUT RESULTS LANE_TIMEOUT EXT @@ -30,16 +34,34 @@ export ONLY="${ONLY:-}" SKIP="${SKIP:-}" rm -f "$RESULTS" echo "--- building producers" +# The samples depend on `weaveffi` with `leak-check`; the feature is named +# here too so the build the consumers load is unmistakably a counting one. # shellcheck disable=SC2086 -cargo build -q $(for s in $SAMPLES; do printf -- '-p %s ' "$s"; done) || exit 1 +cargo build -q --features weaveffi/leak-check \ + $(for s in $SAMPLES; do printf -- '-p %s ' "$s"; done) || exit 1 cargo build -q -p weaveffi-cli || exit 1 WEAVEFFI="$TARGET_DIR/debug/weaveffi" +# With ONLY set, generate just the selected languages (plus the C header +# every other target builds on). +TARGETS="" +if [ -n "$ONLY" ]; then + TARGETS="c" + for sel in ${ONLY//,/ }; do + lang=${sel%%-*} + case ",$TARGETS," in + *",$lang,"*) ;; + *) TARGETS="$TARGETS,$lang" ;; + esac + done +fi + if [ -z "${SKIP_GEN:-}" ]; then for s in $SAMPLES; do echo "--- generating bindings: $s" rm -rf "${GENROOT:?}/$s" - "$WEAVEFFI" generate "samples/$s/src/lib.rs" -o "$GENROOT/$s" --force || exit 1 + "$WEAVEFFI" generate "samples/$s" --library "$(sample_lib "$s")" \ + -o "$GENROOT/$s" ${TARGETS:+--target "$TARGETS"} || exit 1 done fi @@ -56,8 +78,10 @@ done PASS=$(grep -c '^OK ' "$RESULTS" 2>/dev/null || true) FAIL=$(grep -c '^FAIL ' "$RESULTS" 2>/dev/null || true) +SKIPPED=$(grep '^SKIP ' "$RESULTS" 2>/dev/null | cut -d' ' -f2 | tr '\n' ' ' || true) echo echo "conformance: ${PASS:-0} passed, ${FAIL:-0} failed" +[ -z "$SKIPPED" ] || echo "skipped (missing tools): $SKIPPED" if [ "${FAIL:-0}" -ne 0 ]; then echo "failed: $(grep '^FAIL ' "$RESULTS" | cut -d' ' -f2 | tr '\n' ' ')" >&2 exit 1 diff --git a/conformance/swift/async_demo.swift b/conformance/swift/async_demo.swift deleted file mode 100644 index 50dc3519..00000000 --- a/conformance/swift/async_demo.swift +++ /dev/null @@ -1,126 +0,0 @@ -// Conformance consumer: async-demo sample, Swift target. -// -// Drives the checked-continuation async surface end to end: `Tasks.runTask` -// as a throwing async static resumed from the producer's worker thread and -// decoded from a value buffer into the `TaskResult` struct, the typed -// `TaskError.invalidName` case thrown for an empty name, the buffered -// list-of-records round trip through the non-throwing async `runBatch`, and -// the direct-scalar `runNTasks`. The cancellable `wait` completes after its -// timeout when left alone; cancelling the awaiting task cancels the native -// token, so the call ends at once with `CancellationError`, including when -// the task was cancelled before the call started. `activeCallbacks` settles -// to zero, and at exit every cancel token and allocation has been released. - -import CAsyncDemo -import AsyncDemo -import Foundation - -func fail(_ msg: String) -> Never { - FileHandle.standardError.write(Data("assertion failed: \(msg)\n".utf8)) - exit(1) -} - -func expect(_ cond: Bool, _ msg: String) { - if !cond { fail(msg) } -} - -/// Every live-object counter the producer keeps must settle at zero. -func assertNoLeaks() { - let kinds = ["objects", "callbacks", "iterators", "cancel tokens", "allocations"] - var live: [UInt64] = [] - for _ in 0..<200 { - live = (0..<5).map { async_demo_debug_live(Int32($0)) } - if live.allSatisfy({ $0 == 0 }) { return } - usleep(10_000) - } - for (kind, n) in live.enumerated() where n != 0 { - fail("\(n) live \(kinds[kind]) at exit") - } -} - -/// Await `task`, which must end with `CancellationError` within five seconds. -func expectCancelled(_ task: Task, _ what: String) async { - let start = Date() - do { - let value = try await task.value - fail("\(what): expected CancellationError, got \(value)") - } catch is CancellationError { - } catch { - fail("\(what): expected CancellationError, got \(error)") - } - expect(Date().timeIntervalSince(start) < 5, "\(what): didn't wait for the timeout") -} - -func run() async { - // Async record return: resumed with a TaskResult decoded from the - // completion callback's value buffer. - do { - let result = try await Tasks.runTask(name: "alpha") - expect(result.id > 0, "runTask assigns an id") - expect(result.value == "completed: alpha", "runTask value (got \(result.value))") - expect(result.success, "runTask success flag") - } catch { - fail("runTask threw unexpectedly: \(error)") - } - - // Typed async error: the empty name resumes by throwing the typed case. - do { - _ = try await Tasks.runTask(name: "") - fail("expected TaskError.invalidName for empty name") - } catch let e as TaskError { - guard case let .invalidName(message) = e else { fail("expected .invalidName (got \(e))") } - expect(e.errorCode == 1, "invalidName carries code 1 (got \(e.errorCode))") - expect(message == "task name must not be empty", "invalidName message (got \(message))") - } catch { - fail("expected TaskError (got \(error))") - } - - // Buffered list-of-records both ways through the non-throwing async. - let batch = await Tasks.runBatch(names: ["a", "b", "c"]) - expect(batch.map { $0.value } == ["completed: a", "completed: b", "completed: c"], "runBatch values") - expect(batch.allSatisfy { $0.success }, "runBatch success flags") - expect(await Tasks.runBatch(names: []).isEmpty, "runBatch of nothing") - - // Direct scalar through the async callback, many in flight at once. - let sums = await withTaskGroup(of: Int32.self) { group in - for i in 0..<32 { - group.addTask { await Tasks.runNTasks(n: Int32(i)) } - } - var total: Int32 = 0 - for await n in group { total += n } - return total - } - expect(sums == (0..<32).reduce(0, +), "runNTasks echoes n (sum \(sums))") - - // Cancellable: left alone, `wait` completes after its timeout. - do { - let waited = try await Tasks.wait(timeoutMs: 20) - expect(waited == 20, "wait completes with its timeout (got \(waited))") - } catch { - fail("uncancelled wait threw: \(error)") - } - - // Cancelling the awaiting task cancels the native token mid-call. - let running = Task { try await Tasks.wait(timeoutMs: 60_000) } - try? await Task.sleep(nanoseconds: 50_000_000) - running.cancel() - await expectCancelled(running, "cancel mid-call") - - // A task cancelled before the call starts never runs the body. - let early = Task { () async throws -> Int64 in - while !Task.isCancelled { await Task.yield() } - return try await Tasks.wait(timeoutMs: 60_000) - } - early.cancel() - await expectCancelled(early, "cancel before the call") - - // Every spawned task body finishes, cancelled ones included. - for _ in 0..<100 where Tasks.activeCallbacks() != 0 { - try? await Task.sleep(nanoseconds: 10_000_000) - } - expect(Tasks.activeCallbacks() == 0, "activeCallbacks settles to zero") -} - -await run() -assertNoLeaks() -print("swift/async-demo: OK") diff --git a/conformance/swift/calculator.swift b/conformance/swift/calculator.swift new file mode 100644 index 00000000..c1fb68b3 --- /dev/null +++ b/conformance/swift/calculator.swift @@ -0,0 +1,79 @@ +// Conformance consumer: calculator sample, Swift target (ABI revision 4). +// +// The getting-started surface through the generated `Calculator` module: a +// direct-value call (wrapping on overflow), a throwing call whose failure is +// the typed `CalcError.divisionByZero`, and a string in and out (non-ASCII +// text, an astral character, and an interior NUL survive). The bindings check +// the ABI revision and the `calculator` contract table before the first call. +// Ends by asserting the producer's leak counters are zero. Exits non-zero on +// any mismatch. + +import CCalculator +import Calculator +import Foundation + +func fail(_ msg: String) -> Never { + FileHandle.standardError.write(Data("assertion failed: \(msg)\n".utf8)) + exit(1) +} + +func expect(_ cond: Bool, _ msg: @autoclosure () -> String) { + if !cond { fail(msg()) } +} + +/// Every live-resource counter must settle at zero: 0 objects, 1 callbacks, +/// 2 iterators, 3 cancel tokens, 4 allocations. +func assertNoLeaks() { + let kinds = ["objects", "callbacks", "iterators", "cancel tokens", "allocations"] + var live: [UInt64] = [] + for _ in 0..<2000 { + live = (0..<5).map { calculator_debug_live(Int32($0)) } + if live.allSatisfy({ $0 == 0 }) { return } + usleep(1000) + } + for (kind, n) in live.enumerated() where n != 0 { + fail("\(n) live \(kinds[kind]) at exit") + } +} + +func greet(_ name: String, _ expected: String) { + let got = Calculator.greet(name: name) + expect(Array(got.utf8) == Array(expected.utf8), "greet(\(name.debugDescription)) == \(got.debugDescription)") +} + +func run() { + expect(Calculator.add(a: 2, b: 3) == 5, "add(2, 3)") + expect(Calculator.add(a: -7, b: 7) == 0, "add(-7, 7)") + expect(Calculator.add(a: .max, b: 1) == .min, "add wraps") + + do { + expect(try Calculator.divide(a: 10, b: 2) == 5, "divide(10, 2)") + expect(try Calculator.divide(a: -7, b: 2) == -3, "divide rounds toward zero") + expect(try Calculator.divide(a: .min, b: -1) == .min, "divide(MIN, -1)") + } catch { + fail("divide threw \(error)") + } + + do { + _ = try Calculator.divide(a: 1, b: 0) + fail("divide(1, 0) returned") + } catch let error as CalcError { + guard case let .divisionByZero(message) = error else { fail("unexpected \(error)") } + expect(error.errorCode == 1, "DivisionByZero code (got \(error.errorCode))") + expect(message == "division by zero", "DivisionByZero message (got \(message))") + expect(error.localizedDescription == "division by zero", "localizedDescription") + } catch { + fail("divide(1, 0) threw \(error), not CalcError") + } + + greet("World", "Hello, World!") + greet("", "Hello, !") + greet("Wörld 🦀", "Hello, Wörld 🦀!") + greet("a\0b", "Hello, a\0b!") +} + +run() +expect(calculator_abi_version() == 4, "ABI revision 4") +expect(calculator_debug_live(-1) == 1, "the sample counts live resources") +assertNoLeaks() +print("swift/calculator: OK") diff --git a/conformance/swift/codec.swift b/conformance/swift/codec.swift index 8949186e..f682cbc5 100644 --- a/conformance/swift/codec.swift +++ b/conformance/swift/codec.swift @@ -1,20 +1,15 @@ -// Conformance consumer: codec sample, Swift target. +// Conformance consumer: codec sample, Swift target (ABI revision 4). // -// Binds through the generated `Codec` module and checks the value-buffer -// codec the Swift backend ships in both directions. `sample*` fixtures the -// producer encodes are decoded and checked field by field against the -// canonical values in `samples/codec/src/lib.rs`; `verify*` sends them back -// (the producer decodes what Swift encoded and compares against its own -// canonical value); `roundtrip*` returns the argument unchanged so consumer -// encoder and decoder are checked against each other, including a `Scalars` -// and a `Composite` built from scratch with edge values (empty strings, lists, -// and maps; unicode; Int64/UInt64 extremes; NaN, infinities, and negative -// zero). `Shape` covers every rich-enum variant, and `Holder` covers object -// tokens inside buffers: a field, an optional, and a list, each carrying one -// strong reference that the wrapper's deinit releases. Strings with interior -// NULs survive, since strings cross as pointer and length. At exit, every -// native object and allocation has been released. Exits non-zero on any -// mismatch. +// The shared-vector loop through the generated `Codec` module: every vector +// the producer serves is decoded by the Swift codec, re-encoded on the way +// back into `checkVector` (which must accept it, and reject it for the +// neighboring index), and, for primitive vectors, pushed through the +// matching direct-family `echo*`. Then vectors built from Swift literals (so +// a symmetric encode/decode bug can't hide), spot checks of decoded values, +// the typed `CodecError.outOfRange` with its payload, malformed buffers sent +// through the raw C entry points (the Swift encoder can't produce them), and +// object identity and reference counting through buffers. Ends by asserting +// the producer's leak counters are zero. Exits non-zero on any mismatch. import CCodec import Codec @@ -25,358 +20,266 @@ func fail(_ msg: String) -> Never { exit(1) } -func expect(_ cond: Bool, _ msg: String) { - if !cond { fail(msg) } +func expect(_ cond: Bool, _ msg: @autoclosure () -> String) { + if !cond { fail(msg()) } } -/// Every live-object counter the producer keeps must settle at zero. +/// Every live-resource counter must settle at zero: 0 objects, 1 callbacks, +/// 2 iterators, 3 cancel tokens, 4 allocations. func assertNoLeaks() { let kinds = ["objects", "callbacks", "iterators", "cancel tokens", "allocations"] var live: [UInt64] = [] - for _ in 0..<200 { + for _ in 0..<2000 { live = (0..<5).map { codec_debug_live(Int32($0)) } if live.allSatisfy({ $0 == 0 }) { return } - usleep(10_000) + usleep(1000) } for (kind, n) in live.enumerated() where n != 0 { fail("\(n) live \(kinds[kind]) at exit") } } -// --- Equality helpers: NaN and negative zero need bit-pattern comparisons. - -func scalarsEqual(_ a: Scalars, _ b: Scalars) -> Bool { - a.i8Value == b.i8Value && a.u8Value == b.u8Value - && a.i16Value == b.i16Value && a.u16Value == b.u16Value - && a.i32Value == b.i32Value && a.u32Value == b.u32Value - && a.i64Value == b.i64Value && a.u64Value == b.u64Value - && a.f32Value.bitPattern == b.f32Value.bitPattern - && a.f64Value.bitPattern == b.f64Value.bitPattern - && a.flag == b.flag && a.color == b.color +/// Byte-exact string equality (Swift's `==` compares canonical equivalence). +func same(_ a: String, _ b: String) -> Bool { + Array(a.utf8) == Array(b.utf8) } -func shapeEqual(_ a: Shape, _ b: Shape) -> Bool { - switch (a, b) { - case (.empty, .empty): - return true - case let (.circle(r1), .circle(r2)): - return r1.bitPattern == r2.bitPattern - case let (.rect(w1, h1), .rect(w2, h2)): - return w1.bitPattern == w2.bitPattern && h1.bitPattern == h2.bitPattern - case let (.labeled(l1, c1), .labeled(l2, c2)): - return l1 == l2 && c1 == c2 - case let (.nested(i1, n1), .nested(i2, n2)): - return scalarsEqual(i1, i2) && n1 == n2 - default: - return false +func vector(_ i: UInt32) -> Vector { + do { + return try Codec.vector(index: i) + } catch { + fail("vector(\(i)) threw \(error)") } } -func shapesEqual(_ a: [Shape], _ b: [Shape]) -> Bool { - a.count == b.count && zip(a, b).allSatisfy { shapeEqual($0, $1) } +/// The index of every vector, by name. +func vectorIndex(_ n: UInt32) -> [String: UInt32] { + var names: [String: UInt32] = [:] + for i in 0.. Bool { - switch (a, b) { - case (nil, nil): return true - case let (x?, y?): return shapeEqual(x, y) - default: return false +/// Push a primitive vector's value through its direct-family echo. +func echo(_ v: Vector) { + switch v { + case let .i8(x): expect(Codec.echoI8(value: x) == x, "echoI8(\(x))") + case let .u8(x): expect(Codec.echoU8(value: x) == x, "echoU8(\(x))") + case let .i16(x): expect(Codec.echoI16(value: x) == x, "echoI16(\(x))") + case let .u16(x): expect(Codec.echoU16(value: x) == x, "echoU16(\(x))") + case let .i32(x): expect(Codec.echoI32(value: x) == x, "echoI32(\(x))") + case let .u32(x): expect(Codec.echoU32(value: x) == x, "echoU32(\(x))") + case let .i64(x): expect(Codec.echoI64(value: x) == x, "echoI64(\(x))") + case let .u64(x): expect(Codec.echoU64(value: x) == x, "echoU64(\(x))") + case let .f32(x): + expect(Codec.echoF32(value: x).bitPattern == x.bitPattern, "echoF32(\(x)) bit for bit") + case let .f64(x): + expect(Codec.echoF64(value: x).bitPattern == x.bitPattern, "echoF64(\(x)) bit for bit") + case let .flag(x): expect(Codec.echoBool(value: x) == x, "echoBool(\(x))") + case let .text(x): expect(same(Codec.echoText(value: x), x), "echoText(\(x.debugDescription))") + case let .blob(x): expect(Codec.echoBlob(value: x) == x, "echoBlob(\(x as NSData))") + case let .hue(x): expect(Codec.echoColor(value: x) == x, "echoColor(\(x))") + default: break } } -func compositeEqual(_ a: Composite, _ b: Composite) -> Bool { - guard a.name == b.name, a.blob == b.blob, a.someI64 == b.someI64, a.noneI64 == b.noneI64, - a.someText == b.someText, a.names == b.names, a.matrix == b.matrix, - a.empty.map({ $0.bitPattern }) == b.empty.map({ $0.bitPattern }), - a.byName == b.byName, a.byId.count == b.byId.count, - scalarsEqual(a.scalars, b.scalars), shapeEqual(a.shape, b.shape), - shapesEqual(a.shapes, b.shapes), optionalShapeEqual(a.maybeShape, b.maybeShape), - a.maybeList == b.maybeList, a.sparse == b.sparse, a.colors == b.colors - else { return false } - for (k, v) in a.byId { - guard let w = b.byId[k], scalarsEqual(v, w) else { return false } +func everyVector(_ n: UInt32) { + for i in 0.. Vector { + guard let i = index[name] else { fail("no vector named \(name)") } + return vector(i) } - expect(scalarsEqual(Codec.roundtripScalars(value: scalars), scalars), "roundtripScalars is the identity") - // A single changed field is a typed CodecError.mismatch (code 1). - var tweaked = canonical - tweaked.u64Value -= 1 - do { - _ = try Codec.verifyScalars(value: tweaked) - fail("expected CodecError.mismatch for a tweaked Scalars") - } catch let e as CodecError { - guard case let .mismatch(message) = e else { fail("expected .mismatch, got \(e)") } - expect(e.errorCode == 1, "mismatch code == 1 (got \(e.errorCode))") - expect(message == "value does not match the canonical fixture", "mismatch message (got \(message))") - } catch { - fail("expected CodecError, got \(error)") + guard case let .i64(big) = named("i64 past 2^53"), big == -9_007_199_254_740_993 else { + fail("i64 past 2^53") } - - // Scalars built from scratch at the extremes, with non-finite floats and a - // negative zero, survive a round trip bit for bit. - let extremes = Scalars( - i8Value: Int8.min, u8Value: UInt8.max, i16Value: Int16.max, u16Value: UInt16.max, - i32Value: Int32.min, u32Value: UInt32.max, i64Value: Int64.min, u64Value: 1 << 63, - f32Value: -0.0, f64Value: Double.nan, flag: false, color: .red) - let extremesBack = Codec.roundtripScalars(value: extremes) - expect(extremesBack.i64Value == Int64.min && extremesBack.u64Value == 1 << 63, "64-bit extremes") - expect(extremesBack.f32Value == 0 && extremesBack.f32Value.sign == .minus, "negative zero f32 keeps its sign") - expect(extremesBack.f64Value.isNaN, "NaN f64 round-trips") - expect(scalarsEqual(extremesBack, extremes), "extreme Scalars round-trip bit for bit") - let infinities = Scalars( - i8Value: Int8.max, u8Value: 0, i16Value: Int16.min, u16Value: 0, i32Value: Int32.max, u32Value: 0, - i64Value: Int64.max, u64Value: 0, f32Value: Float.infinity, f64Value: -Double.infinity, - flag: true, color: .green) - expect(scalarsEqual(Codec.roundtripScalars(value: infinities), infinities), "infinities round-trip") - - // --- Composite ---------------------------------------------------------------- - - let composite = Codec.sampleComposite() - expect(composite.name == "héllo wörld ✓", "name (got \(composite.name))") - expect(composite.blob == Data([0, 1, 2, 253, 254, 255]), "blob (got \(Array(composite.blob)))") - expect(composite.someI64 == Int64.min, "someI64 (got \(String(describing: composite.someI64)))") - expect(composite.noneI64 == nil, "noneI64 nil") - expect(composite.someText == "", "someText is a present empty string (got \(String(describing: composite.someText)))") - expect(composite.names == ["a", "", "ccc"], "names (got \(composite.names))") - expect(composite.matrix == [[1, 2, 3], [], [-4]], "matrix (got \(composite.matrix))") - expect(composite.empty.isEmpty, "empty list") - expect(composite.byName == ["one": 1, "two": 2, "neg": -3], "byName (got \(composite.byName))") - expect(composite.byId.count == 2, "byId has two entries (got \(composite.byId.count))") - expect(composite.byId[-1].map { scalarsEqual($0, canonical) } == true, "byId[-1] is the canonical Scalars") - expect(composite.byId[42]?.flag == false, "byId[42] has flag false") - var flagless = canonical - flagless.flag = false - expect(composite.byId[42].map { scalarsEqual($0, flagless) } == true, "byId[42] otherwise canonical") - expect(scalarsEqual(composite.scalars, canonical), "nested scalars") - expect(shapeEqual(composite.shape, .labeled(label: "tag", count: 3)), "shape (got \(composite.shape))") - expect(composite.shapes.count == 5, "five shapes (got \(composite.shapes.count))") - expect(shapesEqual(composite.shapes, [ - .empty, - .circle(radius: 2.5), - .rect(width: 1.0, height: 0.5), - .labeled(label: "", count: -1), - .nested(inner: canonical, note: "n"), - ]), "shapes one of each variant (got \(composite.shapes))") - expect(optionalShapeEqual(composite.maybeShape, .nested(inner: canonical, note: nil)), - "maybeShape is a nested variant with an absent note (got \(String(describing: composite.maybeShape)))") - expect(composite.maybeList == Data([9, 8]), "maybeList (got \(String(describing: composite.maybeList)))") - expect(composite.sparse == [true, nil, false], "sparse (got \(composite.sparse))") - expect(composite.colors == [.red, .green, .blue], "colors (got \(composite.colors))") - - do { - expect(try Codec.verifyComposite(value: composite) == true, "producer accepts the re-encoded composite") - } catch { - fail("verifyComposite threw: \(error); producer saw: \(Codec.describeComposite(value: composite))") + _ = big + guard case let .f32(tiny) = named("f32 min subnormal"), tiny.bitPattern == 1 else { + fail("f32 min subnormal") } - expect(compositeEqual(Codec.roundtripComposite(value: composite), composite), "roundtripComposite is the identity") - let described = Codec.describeComposite(value: composite) - expect(described.hasPrefix("Composite {") && described.contains("héllo wörld ✓"), "describeComposite (got \(described))") + _ = tiny + guard case let .text(astral) = named("string astral"), same(astral, "🦀 crab 😀") else { + fail("string astral") + } + _ = astral + + guard case let .allScalars(m) = named("scalars minimum") else { fail("scalars minimum") } + expect(m.i8Value == .min && m.u8Value == 0 && m.i16Value == .min && m.u16Value == 0, "scalars minimum ints") + expect(m.i32Value == .min && m.u32Value == 0 && m.i64Value == .min && m.u64Value == 0, "scalars minimum wide ints") + expect(m.f32Value == -.infinity && m.f64Value.isNaN, "scalars minimum floats") + expect(m.color == .infrared && !m.flag, "scalars minimum color and flag") + + guard case let .deep(c) = named("composite canonical") else { fail("composite canonical") } + expect(same(c.name, "héllo wörld ✓"), "composite name") + expect(c.blob == Data([0, 1, 2, 253, 254, 255]), "composite blob") + expect(c.someI64 == .min && c.noneI64 == nil && c.someText == "", "composite optionals") + expect(c.names == ["a", "", "ccc"] && c.matrix == [[1, 2, 3], [], [-4]], "composite lists") + expect(c.floats.count == 6 && c.floats[0].isNaN && c.floats[1] == .infinity, "composite floats") + expect(c.floats[3].bitPattern == (-0.0 as Double).bitPattern && c.floats[4] == .leastNonzeroMagnitude, "composite float edges") + expect(c.byName == ["one": 1, "two": 2, "neg": -3, "": .max], "composite byName") + expect(c.byId.count == 3 && c.byId[-1]?.u32Value == 4_000_000_000 && c.byId[42]?.color == .red, "composite byId") + expect(c.byColor == [.infrared: "below", .blue: "sky"], "composite byColor") + expect(c.flags == [0: false, .max: true], "composite flags") + expect(c.scalars == canonicalScalars, "composite scalars") + expect(c.shape == .labeled(label: "tag", count: 3), "composite shape") + expect(c.shapes.count == 6, "composite shapes") + guard case .rect(let width, let height) = c.shapes[2] else { fail("composite shapes[2]") } + expect(width.bitPattern == (-0.0 as Float).bitPattern && height == .infinity, "composite rect") + guard case .nested(_, nil) = c.shapes[5] else { fail("composite shapes[5]") } + guard case .nested(_, nil)? = c.maybeShape else { fail("composite maybeShape") } + expect(c.maybeList == Data([9, 8]), "composite maybeList") + expect(c.sparse == [true, nil, false], "composite sparse") + expect(c.colors == [.red, .green, .blue, .infrared], "composite colors") +} - // A changed nested value is rejected. - var changed = composite - changed.sparse[1] = true +func outOfRange(_ n: UInt32) { do { - _ = try Codec.verifyComposite(value: changed) - fail("expected CodecError.mismatch for a changed composite") - } catch let e as CodecError { - expect(e.errorCode == 1, "composite mismatch code == 1") + _ = try Codec.vector(index: n) + fail("vector(\(n)) returned") + } catch let CodecError.outOfRange(message, index, count) { + expect(index == n && count == n, "outOfRange payload (got \(index), \(count))") + expect(message == "vector \(n) is out of range (count \(n))", "outOfRange message (got \(message))") } catch { - fail("expected CodecError, got \(error)") + fail("vector(\(n)) threw \(error)") } - - // A Composite built from scratch with edge values: empty text, bytes, lists, - // and maps; unicode beyond the BMP; present-but-empty optionals; and - // non-finite, negative-zero, and subnormal floats inside nested positions. - let homemade = Composite( - name: "", - blob: Data(), - someI64: Int64.max, - noneI64: -1, - someText: nil, - names: ["日本語", "🚀 rocket", "", "\u{0}nul"], - matrix: [[], [Int32.min, Int32.max], []], - empty: [Double.nan, -0.0, Double.infinity, -Double.infinity, Double.leastNonzeroMagnitude], - byName: [:], - byId: [Int32.min: extremes, 0: canonical, Int32.max: infinities], - scalars: extremes, - shape: .empty, - shapes: [], - maybeShape: nil, - maybeList: Data(), - sparse: [nil, nil, true], - colors: [.blue, .blue, .red, .green]) - let homemadeBack = Codec.roundtripComposite(value: homemade) - expect(homemadeBack.name.isEmpty && homemadeBack.blob.isEmpty, "empty string and bytes") - expect(homemadeBack.someText == nil, "absent optional string") - expect(homemadeBack.maybeList != nil && homemadeBack.maybeList!.isEmpty, "present empty optional bytes") - expect(homemadeBack.names == homemade.names, "unicode names (got \(homemadeBack.names))") - expect(homemadeBack.matrix == [[], [Int32.min, Int32.max], []], "matrix with empty rows") - expect(homemadeBack.empty[0].isNaN, "NaN inside a list") - expect(homemadeBack.empty[1] == 0 && homemadeBack.empty[1].sign == .minus, "negative zero inside a list") - expect(homemadeBack.empty[2] == Double.infinity && homemadeBack.empty[3] == -Double.infinity, "infinities inside a list") - expect(homemadeBack.empty[4] == Double.leastNonzeroMagnitude, "subnormal double inside a list") - expect(homemadeBack.byName.isEmpty && homemadeBack.byId.count == 3, "empty map, three-entry map") - expect(homemadeBack.byId[Int32.min].map { scalarsEqual($0, extremes) } == true, "Int32.min key") - expect(homemadeBack.shapes.isEmpty && homemadeBack.maybeShape == nil, "empty shapes, absent maybeShape") - expect(compositeEqual(homemadeBack, homemade), "homemade Composite round-trips") - expect(Codec.describeComposite(value: homemade).contains("日本語"), "describe renders unicode") do { - _ = try Codec.verifyComposite(value: homemade) - fail("homemade composite is not the canonical one") - } catch let e as CodecError { - expect(e.errorCode == 1, "homemade composite mismatch code == 1") + _ = try Codec.vectorName(index: n + 5) + fail("vectorName(\(n + 5)) returned") + } catch let error as CodecError { + guard case let .outOfRange(_, index, count) = error else { fail("unexpected \(error)") } + expect(error.errorCode == 1 && index == n + 5 && count == n, "vectorName outOfRange payload") } catch { - fail("expected CodecError, got \(error)") + fail("vectorName(\(n + 5)) threw \(error)") } + expect(!Codec.checkVector(index: n, value: .blank), "checkVector past the end is false") +} - // --- Shape (rich enum) --------------------------------------------------------- - - let allShapes: [Shape] = [ - .empty, - .circle(radius: -0.0), - .rect(width: Float.nan, height: -Float.infinity), - .labeled(label: "ünïcödé ✓", count: Int32.min), - .nested(inner: extremes, note: ""), - .nested(inner: canonical, note: nil), - ] - for s in allShapes { - expect(shapeEqual(Codec.roundtripShape(value: s), s), "roundtripShape identity for \(s)") - } - expect(shapesEqual(Codec.roundtripShapes(value: allShapes), allShapes), "roundtripShapes identity") - expect(Codec.roundtripShapes(value: []).isEmpty, "roundtripShapes of an empty list") - expect(Codec.describeShape(value: .empty) == "Empty", "describe Empty") - expect(Codec.describeShape(value: .circle(radius: 2.5)) == "Circle { radius: 2.5 }", "describe Circle") - expect(Codec.describeShape(value: .rect(width: 1.0, height: 0.5)) == "Rect { width: 1.0, height: 0.5 }", - "describe Rect") - expect(Codec.describeShape(value: .labeled(label: "tag", count: 3)) == "Labeled { label: \"tag\", count: 3 }", - "describe Labeled") - expect(Codec.describeShape(value: .nested(inner: canonical, note: nil)).hasPrefix("Nested { inner: Scalars {"), - "describe Nested") - - // --- Direct, string, bytes, and standalone buffered families -------------------- - - expect(Codec.roundtripOptI64(value: nil) == nil, "optional i64 absent") - expect(Codec.roundtripOptI64(value: Int64.min) == Int64.min, "optional i64 present") - expect(Codec.roundtripOptI64(value: 0) == 0, "optional i64 zero is present, not absent") - expect(Codec.roundtripMap(value: [:]).isEmpty, "empty map") - let bigMap: [String: Int64] = ["": 0, "a": Int64.max, "ключ": Int64.min, "z": -1] - expect(Codec.roundtripMap(value: bigMap) == bigMap, "map with empty and unicode keys") - expect(Codec.roundtripString(value: "") == "", "empty string") - expect(Codec.roundtripString(value: "héllo wörld ✓ 🚀") == "héllo wörld ✓ 🚀", "unicode string") - // Strings cross as pointer and length, so an interior NUL survives both - // directions, alone and inside a buffered record. - let withNul = "before\u{0}after\u{0}" - expect(Codec.roundtripString(value: withNul) == withNul, "interior NUL string round-trips") - expect(Codec.roundtripString(value: "\u{0}") == "\u{0}", "lone NUL string round-trips") - expect(Codec.roundtripComposite(value: homemade).names[3] == "\u{0}nul", "NUL inside a record field") - expect(Codec.roundtripBytes(value: Data()).isEmpty, "empty bytes") - let allBytes = Data((0...255).map { UInt8($0) }) - expect(Codec.roundtripBytes(value: allBytes) == allBytes, "every byte value") - expect(Codec.roundtripI64(value: Int64.min) == Int64.min, "i64 min") - expect(Codec.roundtripI64(value: Int64.max) == Int64.max, "i64 max") - expect(Codec.roundtripI64(value: -1) == -1, "i64 -1") - expect(Codec.roundtripU64(value: UInt64.max) == UInt64.max, "u64 max") - expect(Codec.roundtripU64(value: 1 << 63) == 1 << 63, "u64 above 2^63") - expect(Codec.roundtripU64(value: 0) == 0, "u64 zero") - expect(Codec.roundtripF64(value: Double.nan).isNaN, "f64 NaN") - expect(Codec.roundtripF64(value: Double.infinity) == Double.infinity, "f64 +inf") - expect(Codec.roundtripF64(value: -Double.infinity) == -Double.infinity, "f64 -inf") - let negZero = Codec.roundtripF64(value: -0.0) - expect(negZero == 0 && negZero.sign == .minus, "f64 negative zero keeps its sign") - expect(Codec.roundtripF64(value: Double.leastNonzeroMagnitude) == Double.leastNonzeroMagnitude, "f64 subnormal") - expect(Codec.roundtripF64(value: -2.25e100) == -2.25e100, "f64 large negative") - expect(Codec.roundtripBool(value: true) == true && Codec.roundtripBool(value: false) == false, "bool") - expect(Codec.roundtripColor(value: .blue) == .blue, "color blue (discriminant 7)") - expect(Codec.roundtripColor(value: .red) == .red, "color red") - - // --- Holder: objects inside buffers ------------------------------------------- - - let holder = Codec.makeHolder(base: 10, withSpare: true) - expect(holder.primary.value() == 10, "primary token (got \(holder.primary.value()))") - expect(holder.spare?.value() == 11, "spare token (got \(String(describing: holder.spare?.value())))") - expect(holder.many.map { $0.value() } == [12, 13, 14], "many tokens (got \(holder.many.map { $0.value() }))") - // Encoding clones every token; the producer adopts and drops those clones, - // and the wrappers remain valid for the next call. - expect(Codec.sumHolder(holder: holder) == 60, "sumHolder (got \(Codec.sumHolder(holder: holder)))") - expect(Codec.sumHolder(holder: holder) == 60, "sumHolder again: no reference was consumed") - expect(holder.primary.value() == 10 && holder.many[2].value() == 14, "tokens still alive after encoding") +/// Hand `bytes` to `check_vector` through the raw C entry point: the Swift +/// encoder never produces a malformed buffer, but the producer must reject +/// one with a marshalling failure (-3). +func reject(_ bytes: [UInt8], _ what: String) { + var err = codec_error() + let ok = bytes.withUnsafeBufferPointer { codec_codec_check_vector(0, $0.baseAddress, $0.count, &err) } + expect(!ok && err.code == -3, "\(what) is rejected with -3 (got \(err.code))") + codec_error_clear(&err) +} - // primaryOf returns the very object stored in holder.primary. - let primary = Codec.primaryOf(holder: holder) - expect(primary.value() == 10, "primaryOf value") - let rebuilt = Holder(primary: primary, spare: nil, many: []) - expect(Codec.samePrimary(a: holder, b: rebuilt), "primaryOf returned the same object as holder.primary") - expect(Codec.samePrimary(a: holder, b: holder), "a holder shares its own primary") - expect(!Codec.samePrimary(a: holder, b: Codec.makeHolder(base: 10, withSpare: true)), - "a fresh holder with equal values is a different object") - expect(Codec.sumHolder(holder: rebuilt) == 10, "sumHolder over a consumer-built holder") +func le(_ v: T) -> [UInt8] { + withUnsafeBytes(of: v.littleEndian) { Array($0) } +} - let bare = Codec.makeHolder(base: 0, withSpare: false) - expect(bare.spare == nil, "withSpare false yields no spare") - expect(bare.primary.value() == 0 && bare.many.map { $0.value() } == [2, 3, 4], "bare tokens") - expect(Codec.sumHolder(holder: bare) == 9, "sumHolder without spare (got \(Codec.sumHolder(holder: bare)))") +func malformed() { + let i64Tag: Int32 = 7, flagTag: Int32 = 11, textTag: Int32 = 12, hueTag: Int32 = 14, countsTag: Int32 = 19 + reject(le(i64Tag) + le(UInt32(7)), "a truncated value") + reject(le(Int32(999)), "an unknown tag") + reject(le(Int32(0)) + [0], "trailing bytes") + reject(le(flagTag) + [2], "a bool of 2") + reject(le(hueTag) + le(Int32(3)), "an undeclared enum value") + let a = le(UInt32(1)) + Array("a".utf8) + reject(le(countsTag) + le(UInt32(2)) + a + le(Int64(1)) + a + le(Int64(2)), "a repeated map key") + reject(le(textTag) + le(UInt32(2)) + [0xC3, 0x28], "a string that isn't UTF-8") + + // Direct families: an undeclared enum value (which Swift's `Color` can't + // even spell) and invalid UTF-8. + var err = codec_error() + _ = codec_codec_echo_color(codec_codec_Color(rawValue: 3), &err) + expect(err.code == -3, "echo_color(3) is rejected (got \(err.code))") + codec_error_clear(&err) + var len = 0 + let bad: [UInt8] = [0xC3, 0x28] + let out = bad.withUnsafeBufferPointer { codec_codec_echo_text($0.baseAddress, $0.count, &len, &err) } + expect(out == nil && err.code == -3, "echo_text(invalid UTF-8) is rejected") + codec_error_clear(&err) +} - // Consumer-created tokens, including one object referenced twice from the - // same buffer (two clones, two adoptions). - let shared = Token(value: 100) - let mine = Holder(primary: shared, spare: Token(value: 5), many: [shared, Token(value: 1), shared]) - expect(Codec.sumHolder(holder: mine) == 100 + 5 + 100 + 1 + 100, "sumHolder over shared tokens") - expect(Codec.samePrimary(a: mine, b: Holder(primary: shared, spare: nil, many: [])), "shared primary identity") - expect(shared.value() == 100, "shared token alive after three encodings") - expect(Codec.sumHolder(holder: Holder(primary: Token(value: -7), spare: nil, many: [])) == -7, "minimal holder") +func objects(_ index: [String: UInt32]) { + guard let full = index["objects full"], case let .objects(h) = vector(full) else { fail("objects full") } + expect(h.primary.value() == 10 && h.spare?.value() == 11, "objects full primary and spare") + expect(h.many.map { $0.value() } == [12, 13, .min], "objects full many") + expect(h.byName.mapValues { $0.value() } == ["a": 20, "b": 21], "objects full byName") + // Each encoding mints fresh references, so the holder can be sent twice. + let expected = [10, 11, 12, 13, 20, 21, Int64.min].reduce(0, &+) + expect(Codec.sumHolder(holder: h) == expected && Codec.sumHolder(holder: h) == expected, "sumHolder twice") + + // primaryOf returns the very same native object (a new wrapper). + let p = Codec.primaryOf(holder: h) + expect(p.value() == 10, "primaryOf value") + expect(Codec.samePrimary(a: h, b: Holder(primary: p, spare: nil, many: [], byName: [:])), "samePrimary identity") + let twin = Token(value: 10) + expect(!Codec.samePrimary(a: h, b: Holder(primary: twin, spare: nil, many: [], byName: [:])), "an equal value is not the same object") + + // A holder of Swift-made tokens, the same wrapper in several slots. + let mine = Holder(primary: twin, spare: twin, many: [twin, twin, Token(value: -4)], byName: ["k": twin]) + expect(Codec.sumHolder(holder: mine) == 10 * 5 - 4, "sumHolder of consumer tokens") + expect(twin.value() == 10, "the twin is still usable") +} - // Release: every wrapper's deinit releases its strong reference, including - // the ones adopted from a decoded buffer and the one returned by primaryOf. - weak var weakPrimary: Token? - weak var weakSpare: Token? - weak var weakMany: Token? - weak var weakReturned: Token? - do { - let h = Codec.makeHolder(base: 20, withSpare: true) - weakPrimary = h.primary - weakSpare = h.spare - weakMany = h.many[0] - let p = Codec.primaryOf(holder: h) - weakReturned = p - expect(p.value() == 20 && Codec.sumHolder(holder: h) == 20 + 21 + 22 + 23 + 24, "scoped holder usable") - } - expect(weakPrimary == nil && weakSpare == nil && weakMany == nil && weakReturned == nil, - "every token wrapper was deinitialized when the holder left scope") +func run() { + let n = Codec.vectorCount() + expect(n >= 60, "vectorCount (got \(n))") + let index = vectorIndex(n) + expect(index.count == Int(n), "vector names are unique") + everyVector(n) + literalVectors(index) + spotChecks(index) + outOfRange(n) + malformed() + objects(index) + print("swift/codec: \(n) vectors") } run() +expect(codec_abi_version() == 4, "ABI revision 4") +expect(codec_debug_live(-1) == 1, "the sample counts live resources") assertNoLeaks() print("swift/codec: OK") diff --git a/conformance/swift/contacts.swift b/conformance/swift/contacts.swift deleted file mode 100644 index 3b104d5b..00000000 --- a/conformance/swift/contacts.swift +++ /dev/null @@ -1,135 +0,0 @@ -// Conformance consumer: contacts sample, Swift target. -// -// Binds through the generated `Contacts` package and asserts the interface -// surface: `ContactBook` as a final class whose `new` constructor is a plain -// `init()`, throwing methods (`add`, `get`) that raise the typed -// `ContactsError` domain enum, non-throwing methods (`list`, `remove`, -// `count`) called without `try`, enum marshalling, and `Contact` as a -// `Hashable` struct decoded from a value buffer. The nested `Contacts.Groups` -// namespace takes and returns the parent module's `ContactType` enum and -// `ContactBook` interface and throws the parent's domain; the sibling -// `Directory` root shares the `Contact` record and has its own domain. At -// exit, every native object and allocation has been released. - -import CContacts -import Contacts -import Foundation - -func fail(_ msg: String) -> Never { - FileHandle.standardError.write(Data("assertion failed: \(msg)\n".utf8)) - exit(1) -} - -func expect(_ cond: Bool, _ msg: String) { - if !cond { fail(msg) } -} - -/// Every live-object counter the producer keeps must settle at zero. -func assertNoLeaks() { - let kinds = ["objects", "callbacks", "iterators", "cancel tokens", "allocations"] - var live: [UInt64] = [] - for _ in 0..<200 { - live = (0..<5).map { contacts_debug_live(Int32($0)) } - if live.allSatisfy({ $0 == 0 }) { return } - usleep(10_000) - } - for (kind, n) in live.enumerated() where n != 0 { - fail("\(n) live \(kinds[kind]) at exit") - } -} - -func run() throws { - let book = ContactBook() - expect(contacts_debug_live(0) == 1, "the leak counters see the live book") - - let alice = try book.add( - firstName: "Alice", lastName: "Smith", - email: "alice@example.com", contactType: .work) - expect(alice.id > 0, "alice id positive") - - // The record is a value type: `get` decodes a fresh Contact struct equal - // to the one `add` returned. - let c = try book.get(id: alice.id) - expect(c == alice, "get returns the added contact") - expect(c.firstName == "Alice" && c.lastName == "Smith", "names") - expect(c.email == "alice@example.com", "email") - expect(c.contactType == .work, "contactType") - - // Optional string: a missing email round-trips as nil. - let bob = try book.add(firstName: "Bob", lastName: "Jones", email: nil, contactType: .personal) - let cb = try book.get(id: bob.id) - expect(cb.email == nil, "bob email nil") - expect(Set([alice, bob, cb]).count == 2, "Contact is Hashable") - - // The memberwise init builds a local value without touching the ABI. - let local = Contact(id: 42, firstName: "Carol", lastName: "Doe", email: nil, contactType: .other) - expect(local.id == 42 && local.email == nil, "memberwise init") - - // Non-throwing methods need no `try`. - expect(book.count() == 2, "count == 2") - let everyone = book.list() - expect(everyone.map { $0.firstName }.sorted() == ["Alice", "Bob"], "list names") - - // A missing id raises the typed domain error's notFound case (code 2). - do { - _ = try book.get(id: 999) - fail("expected ContactsError.notFound for missing contact") - } catch let e as ContactsError { - guard case let .notFound(message) = e else { fail("expected .notFound, got \(e)") } - expect(e.errorCode == 2, "notFound code == 2 (got \(e.errorCode))") - expect(message == "contact not found", "notFound message (got \(message))") - } - - // An empty name raises the invalidName case (code 1). - do { - _ = try book.add(firstName: "", lastName: "Smith", email: nil, contactType: .personal) - fail("expected ContactsError.invalidName for empty first name") - } catch let e as ContactsError { - guard case .invalidName = e else { fail("expected .invalidName, got \(e)") } - expect(e.errorCode == 1, "invalidName code == 1 (got \(e.errorCode))") - } - expect(book.count() == 2, "rejected add stores nothing") - - // --- Nested module: the parent's enum, interface, and error domain ------ - _ = try book.add(firstName: "Dana", lastName: "Wu", email: nil, contactType: .work) - expect(Contacts.Groups.countOfType(book: book, contactType: .work) == 2, "two work contacts") - expect(Contacts.Groups.countOfType(book: book, contactType: .other) == 0, "no other contacts") - expect(Contacts.Groups.dominantType(book: book) == .work, "work dominates") - let work = Contacts.Groups.splitByType(book: book, contactType: .work) - expect(work.count() == 2, "split book holds the work contacts") - expect(work.list().allSatisfy { $0.contactType == .work }, "split book is all work") - expect(book.count() == 3, "splitting leaves the source book intact") - let firstWork = try Contacts.Groups.firstOfType(book: book, contactType: .work) - expect(firstWork.firstName == "Alice", "first work contact (got \(firstWork.firstName))") - do { - _ = try Contacts.Groups.firstOfType(book: work, contactType: .personal) - fail("expected ContactsError.notFound from the nested module") - } catch ContactsError.notFound { - } - - // --- Sibling root: shares the Contact record, has its own domain -------- - let card = Directory.card(contact: alice) - expect(card == Card(displayName: "Smith, Alice", initials: "AS", hasEmail: true), - "card (got \(card))") - expect(!Directory.card(contact: bob).hasEmail, "bob's card has no email") - let sorted = try Directory.sorted(contacts: [alice, bob, local]) - expect(sorted.map { $0.lastName } == ["Doe", "Jones", "Smith"], "sorted by last name") - do { - _ = try Directory.sorted(contacts: []) - fail("expected DirectoryError.empty") - } catch let e as DirectoryError { - guard case .empty = e else { fail("expected .empty, got \(e)") } - expect(e.errorCode == 1, "empty code == 1") - } - - expect(book.remove(id: alice.id) == true, "remove returns true") - expect(book.count() == 2, "count == 2 after remove") -} - -do { - try run() -} catch { - fail("unexpected error: \(error)") -} -assertNoLeaks() -print("swift/contacts: OK") diff --git a/conformance/swift/events.swift b/conformance/swift/events.swift deleted file mode 100644 index ef48b4a9..00000000 --- a/conformance/swift/events.swift +++ /dev/null @@ -1,245 +0,0 @@ -// Conformance consumer: events sample, Swift target. -// -// Binds through the generated `Events` package and drives the `Subscriber` -// callback interface implemented as a Swift class (every method observed -// with its arguments, including the `EventBus` object the producer hands to -// `onAttached`), `Delivery` return values steering `publish`'s accepted -// count, the reference-counted `EventBus` object, the `publishLater` async -// wrapper, `messages()` as a lazy Sequence, the `lastMessage()` optional -// record, and the free function `routeOnce`. Release is observed through -// weak references. -// -// `route` is the `Result`-returning callback method: a subscriber that -// throws from it makes the producer fail the call with the foreign code -4. -// The events module declares no error domain, so `publish` doesn't throw and -// the failure stops the process with a message naming the code; a child -// process (this executable run with `trap-route`) proves that. At exit, -// every native object, callback, iterator, and allocation has been released. - -import CEvents -import Events -import Foundation - -func fail(_ msg: String) -> Never { - FileHandle.standardError.write(Data("assertion failed: \(msg)\n".utf8)) - exit(1) -} - -func expect(_ cond: Bool, _ msg: String) { - if !cond { fail(msg) } -} - -/// Every live-object counter the producer keeps must settle at zero. -func assertNoLeaks() { - let kinds = ["objects", "callbacks", "iterators", "cancel tokens", "allocations"] - var live: [UInt64] = [] - for _ in 0..<200 { - live = (0..<5).map { events_debug_live(Int32($0)) } - if live.allSatisfy({ $0 == 0 }) { return } - usleep(10_000) - } - for (kind, n) in live.enumerated() where n != 0 { - fail("\(n) live \(kinds[kind]) at exit") - } -} - -struct RouteRejected: Error, LocalizedError { - let topic: String - var errorDescription: String? { "route rejected \(topic)" } -} - -/// A consumer-side subscriber. Records every callback so the test can assert -/// the arguments the producer passed, and counts live instances so the -/// producer's `free` (which releases the generated box, and with it the last -/// strong reference to this object) is observable. Callbacks arrive on the -/// calling thread here, so plain mutable state is fine. -final class TestSubscriber: Subscriber, @unchecked Sendable { - static var live = 0 - - let skipTopic: String - let failTopic: String - let keepBus: Bool - var routed: [String] = [] - var received: [Message] = [] - var attachedCount = 0 - /// Subscriber count observed through the bus object inside `onAttached`. - var countSeenOnAttach: Int64 = -1 - /// The bus adopted from `onAttached` when `keepBus` is set. - var attachedBus: EventBus? - /// A weak view of the bus wrapper handed to `onAttached`. - weak var weakAttachedBus: EventBus? - - init(skipTopic: String = "", failTopic: String = "", keepBus: Bool = false) { - self.skipTopic = skipTopic - self.failTopic = failTopic - self.keepBus = keepBus - TestSubscriber.live += 1 - } - - deinit { TestSubscriber.live -= 1 } - - func route(topic: String) throws -> Delivery { - routed.append(topic) - if topic == failTopic { throw RouteRejected(topic: topic) } - if topic == skipTopic { return .skip } - if topic == "stop" { return .acceptAndStop } - return .accept - } - - func onMessage(message: Message) throws -> Int64 { - received.append(message) - return Int64(received.count) - } - - func onAttached(bus: EventBus) throws { - attachedCount += 1 - // The object is live and usable inside the callback: `subscribe` - // attaches before it pushes, so the count excludes this subscriber. - countSeenOnAttach = bus.subscriberCount() - weakAttachedBus = bus - if keepBus { attachedBus = bus } - } -} - -/// Child-process mode: a subscriber whose `route` throws makes the -/// non-throwing `publish` stop the process. -if CommandLine.arguments.dropFirst().first == "trap-route" { - let bus = EventBus() - _ = bus.subscribe(subscriber: TestSubscriber(failTopic: "boom")) - _ = bus.publish(topic: "boom", text: "never delivered", tags: []) - print("publish returned after a failing route") - exit(0) -} - -func runTrapRouteChild() { - let child = Process() - child.executableURL = URL(fileURLWithPath: CommandLine.arguments[0]) - child.arguments = ["trap-route"] - let stderr = Pipe() - child.standardError = stderr - child.standardOutput = Pipe() - do { - try child.run() - } catch { - fail("could not start the trap-route child: \(error)") - } - let output = String(decoding: stderr.fileHandleForReading.readDataToEndOfFile(), as: UTF8.self) - child.waitUntilExit() - expect(child.terminationReason == .uncaughtSignal, - "a throwing route stops the process (status \(child.terminationStatus))") - expect(output.contains("failed with code -4: route rejected boom"), - "the fatal error names the foreign code and message (got \(output))") -} - -func run() async { - // --- Empty bus ----------------------------------------------------------- - var bus: EventBus? = EventBus() - expect(bus!.subscriberCount() == 0, "new bus has no subscribers") - expect(bus!.lastMessage() == nil, "new bus has no last message") - expect(Array(bus!.messages()).isEmpty, "new bus has no messages") - - // --- subscribe: onAttached receives a usable bus object ------------------ - var a: TestSubscriber? = TestSubscriber(skipTopic: "quiet") - var b: TestSubscriber? = TestSubscriber(keepBus: true) - weak var weakA = a - weak var weakB = b - - expect(bus!.subscribe(subscriber: a!) == 1, "first subscribe returns 1") - expect(a!.attachedCount == 1, "a attached once") - expect(a!.countSeenOnAttach == 0, "a saw an empty bus on attach (got \(a!.countSeenOnAttach))") - expect(a!.weakAttachedBus == nil, "bus wrapper handed to a was released after onAttached") - - expect(bus!.subscribe(subscriber: b!) == 2, "second subscribe returns 2") - expect(b!.countSeenOnAttach == 1, "b saw one subscriber on attach (got \(b!.countSeenOnAttach))") - expect(b!.attachedBus != nil && b!.weakAttachedBus != nil, "b kept the adopted bus") - expect(bus!.subscriberCount() == 2, "subscriberCount == 2") - - // --- publish: Delivery steers the accepted count ------------------------- - expect(bus!.publish(topic: "news", text: "hello", tags: ["x", "y"]) == 2, - "both subscribers accept 'news'") - expect(a!.routed == ["news"] && b!.routed == ["news"], "route asked once per subscriber") - let first = a!.received[0] - expect(first == Message(seq: 1, topic: "news", text: "hello", tags: ["x", "y"]), - "first message (got \(first))") - - // a skips 'quiet'; b accepts it. - expect(bus!.publish(topic: "quiet", text: "psst", tags: []) == 1, "skip lowers the accepted count") - expect(a!.received.count == 1, "a did not receive the skipped message") - expect(b!.received.count == 2 && b!.received[1].seq == 2 && b!.received[1].tags.isEmpty, - "b received the second message with empty tags") - - // AcceptAndStop from the first subscriber stops delivery to the rest. - let c = TestSubscriber() - expect(bus!.subscribe(subscriber: c) == 3, "third subscribe returns 3") - expect(bus!.publish(topic: "stop", text: "last", tags: ["z"]) == 1, "acceptAndStop delivers once") - expect(a!.received.count == 2 && a!.received[1].text == "last", "a took the stop message") - expect(b!.routed == ["news", "quiet"], "b was not asked after the stop (got \(b!.routed))") - expect(c.routed.isEmpty && c.received.isEmpty, "c was never reached") - - // --- messages(): iterator as a lazy Sequence ----------------------------- - expect(Array(bus!.messages()) == ["hello", "psst", "last"], "messages in order") - var joined: [String] = [] - for m in bus!.messages() { joined.append(m.uppercased()) } - expect(joined == ["HELLO", "PSST", "LAST"], "for-in over the sequence") - // Abandoning early releases the producer iterator through deinit. - expect(bus!.messages().first(where: { $0.hasPrefix("p") }) == "psst", "early exit") - - // --- lastMessage(): optional record -------------------------------------- - expect(bus!.lastMessage() == Message(seq: 3, topic: "stop", text: "last", tags: ["z"]), - "lastMessage fields") - - // --- publishLater: async wrapper resumed from a producer thread ---------- - let later = await bus!.publishLater(topic: "news", text: "later") - expect(later == 3, "publishLater accepted by all three (got \(later))") - expect(bus!.lastMessage()?.seq == 4, "async publish took seq 4") - expect(c.received.count == 1, "c received the async message") - - // --- The bus adopted in onAttached is the same object -------------------- - let kept = b!.attachedBus! - expect(kept.publish(topic: "via-kept", text: "same object", tags: []) == 3, - "publish through the kept wrapper reaches every subscriber") - expect(bus!.lastMessage()?.topic == "via-kept", "publish via kept wrapper visible through the original") - - // --- routeOnce: a callback passed to a free function is freed on return -- - weak var weakD: TestSubscriber? - do { - let d = TestSubscriber(skipTopic: "quiet") - weakD = d - expect(Events.routeOnce(subscriber: d, topic: "quiet") == .skip, "routeOnce skip") - expect(Events.routeOnce(subscriber: d, topic: "stop") == .acceptAndStop, "routeOnce acceptAndStop") - expect(Events.routeOnce(subscriber: d, topic: "other") == .accept, "routeOnce accept") - expect(d.attachedCount == 0, "routeOnce never attaches") - } - expect(weakD == nil, "routeOnce released its subscriber (free ran)") - - // --- clearSubscribers releases every subscriber -------------------------- - a = nil - b = nil - expect(weakA != nil && weakB != nil, "the bus retains a and b after the consumer drops them") - bus!.clearSubscribers() - expect(weakA == nil, "a freed by clearSubscribers") - expect(weakB == nil, "b (and the bus reference it kept) freed by clearSubscribers") - expect(bus!.publish(topic: "news", text: "nobody", tags: []) == 0, "publish with no subscribers") - - // --- Destroying the bus frees a retained subscriber ---------------------- - weak var weakE: TestSubscriber? - weak var weakBus2: EventBus? - do { - let bus2 = EventBus() - weakBus2 = bus2 - let e = TestSubscriber() - weakE = e - expect(bus2.subscribe(subscriber: e) == 1, "bus2 subscribe") - } - expect(weakBus2 == nil, "bus2 wrapper deinitialized") - expect(weakE == nil, "dropping the last bus reference freed its subscriber") - - bus = nil - expect(TestSubscriber.live == 1, "only c is still alive (got \(TestSubscriber.live))") -} - -await run() -runTrapRouteChild() -expect(TestSubscriber.live == 0, "every subscriber was released (got \(TestSubscriber.live))") -assertNoLeaks() -print("swift/events: OK") diff --git a/conformance/swift/kvstore.swift b/conformance/swift/kvstore.swift index e2ea5353..6e0659a6 100644 --- a/conformance/swift/kvstore.swift +++ b/conformance/swift/kvstore.swift @@ -1,358 +1,683 @@ -// Conformance consumer: kvstore sample, Swift target. +// Conformance consumer: kvstore sample, Swift target (ABI revision 4). // -// Binds through the generated `Kvstore` package and exercises the full -// surface: `Store` as a reference-counted final class opened via the throwing -// static factory `Store.open(path:)`, throwing methods raising the typed -// `KvError` domain enum (put/get/delete/listKeys), non-throwing methods -// without `try` (count/clear), the buffered optional parameter (`ttlSeconds`), -// `Entry` decoded from a value buffer, the `listKeys` iterator as a lazy -// Sequence, the nested `Kv.Stats` submodule, the CheckedContinuation-backed -// `compact()` async method, the `EvictionListener` callback interface -// implemented as a Swift class (arguments observed, detach by returning -// false, replace/clear releasing the previous listener, a thrown error -// surfacing to the caller as `KvstoreRuntimeError` with code -4), and the -// object graph: `share()` returning a wrapper to the same object, `fork()`, -// `larger(other:)` with `Store?` both ways, `describe(label:mirror:)` -// returning a record that carries objects, `openMany` returning a list of -// objects, and `totalCount` taking objects inside a list and a record. -// Release is observed through weak references to the wrappers, whose deinit -// calls the generated destroy. Typed-error asserts pin the case and the -// numeric code (keyNotFound 1001, expired 1002, ioError 1004). At exit, every -// native object, callback, iterator, token, and allocation has been released. +// Drives the feature-complete producer through the generated `Kvstore` +// module: +// +// * the load-time checks (ABI revision, the `kv` and `report` contract +// tables), run by the bindings before the first call; +// * the `Store` class: a throwing static factory and an `init`, methods, +// statics, the deprecated `size()`, records (`Entry`, `StoreInfo`), the +// C-style `EntryKind`, maps and optionals, and the logical clock; +// * `KvError` cases with their payload fields (`keyNotFound`, `expired`, +// `storeFull`, `rejected`, `invalidPath`); +// * lazy sequences of strings (throwing), records, and objects; +// * three callback protocols implemented in Swift: a `Listener` (retained, +// filtered by `accepts`, told about every `Change`, detached when it +// throws, and notified from a producer thread during compaction), a +// `Policy` (a record return, a throwing method whose `KvError` and fields +// reach the `put` caller, an object parameter and object return), and a +// `Loader` passed as an optional callback (string, bytes, and +// optional-object returns; `KvError`s decoded by the producer or passed +// through); +// * `Store` objects in every position: parameter, return, optional, list, +// map value, record field, iterator element, async result, and callback +// parameter and return; +// * async calls: an async free function returning an object, a cancellable +// method cancelled mid-pause by cancelling its task (throwing +// `CancellationError` while its background work stops, shown by +// `activeJobs()`), an async list, an async function in the nested +// `kv.stats` module, and concurrent calls; +// * the nested `kv.stats` module and the sibling `report` root. +// +// Ends by asserting the producer's leak counters are zero and that the +// library released every callback implementation. Exits non-zero on any +// mismatch. import CKvstore import Foundation import Kvstore -func fail(_ msg: String) -> Never { - FileHandle.standardError.write(Data("assertion failed: \(msg)\n".utf8)) +func fail(_ msg: String, line: UInt = #line) -> Never { + FileHandle.standardError.write(Data("assertion failed (line \(line)): \(msg)\n".utf8)) exit(1) } -func expect(_ cond: Bool, _ msg: String) { - if !cond { fail(msg) } +func expect(_ cond: Bool, _ msg: @autoclosure () -> String, line: UInt = #line) { + if !cond { fail(msg(), line: line) } } -/// Every live-object counter the producer keeps must settle at zero. +/// Every live-resource counter must settle at zero: 0 objects, 1 callbacks, +/// 2 iterators, 3 cancel tokens, 4 allocations. A producer worker may still +/// be dropping a finished async call, so the counters get two seconds. func assertNoLeaks() { let kinds = ["objects", "callbacks", "iterators", "cancel tokens", "allocations"] var live: [UInt64] = [] - for _ in 0..<200 { + for _ in 0..<2000 { live = (0..<5).map { kvstore_debug_live(Int32($0)) } if live.allSatisfy({ $0 == 0 }) { return } - usleep(10_000) + usleep(1000) } for (kind, n) in live.enumerated() where n != 0 { fail("\(n) live \(kinds[kind]) at exit") } } -struct ListenerFailure: Error, LocalizedError { - let key: String - var errorDescription: String? { "listener rejected \(key)" } +/// Run `body`, which must throw the `KvError` `check` accepts. +func expectKvError(_ what: String, line: UInt = #line, _ body: () throws -> T, _ check: (KvError) -> Bool) { + do { + _ = try body() + fail("\(what) returned", line: line) + } catch let error as KvError { + expect(check(error), "\(what) threw \(error)", line: line) + } catch { + fail("\(what) threw \(error), not KvError", line: line) + } } -/// A consumer-side eviction listener. Records every (key, reason) pair, -/// detaches itself by returning false once `keepAfter` evictions were seen, -/// and throws for `failKey` so the foreign-error path can be observed. -final class Listener: EvictionListener, @unchecked Sendable { - var evicted: [(key: String, reason: EvictionReason)] = [] - var lastEntry: Entry? - let keepAfter: Int - let failKey: String? +/// Run `body`, which must throw the runtime error with `code` and `message`. +func expectRuntimeError(_ what: String, code: Int32, message: String, line: UInt = #line, _ body: () throws -> T) { + do { + _ = try body() + fail("\(what) returned", line: line) + } catch let error as KvstoreRuntimeError { + expect(error.errorCode == code && error.message == message, "\(what) threw \(error)", line: line) + } catch { + fail("\(what) threw \(error), not KvstoreRuntimeError", line: line) + } +} - init(keepAfter: Int = Int.max, failKey: String? = nil) { - self.keepAfter = keepAfter - self.failKey = failKey +func isKeyNotFound(_ key: String) -> (KvError) -> Bool { + { error in + guard case let .keyNotFound(_, got) = error else { return false } + return got == key && error.errorCode == 1001 } +} - func onEvict(entry: Entry, reason: EvictionReason) throws -> Bool { - if entry.key == failKey { throw ListenerFailure(key: entry.key) } - evicted.append((entry.key, reason)) - lastEntry = entry - return evicted.count < keepAfter +/// `true` when `a` and `b` wrap the same native store: a write through one +/// is visible through the other. +func isSame(_ a: Store, _ b: Store) -> Bool { + let probe = "identity-probe" + _ = try! a.put(key: probe, value: Data(), kind: .volatile, ttlSeconds: nil) + defer { _ = a.delete(key: probe) } + return b.find(key: probe) != nil +} + +@discardableResult +func put(_ s: Store, _ key: String, _ value: String, _ kind: EntryKind = .persistent, ttl: Int64? = nil) -> Entry { + do { + return try s.put(key: key, value: Data(value.utf8), kind: kind, ttlSeconds: ttl) + } catch { + fail("put(\(key)) threw \(error)") } } -do { - let store = try Store.open(path: "/tmp/conformance-kvstore-swift") - - let payload = Data([1, 2, 3]) - _ = try store.put(key: "alpha", value: payload, kind: .persistent, ttlSeconds: nil) - _ = try store.put(key: "beta", value: payload, kind: .volatile, ttlSeconds: nil) - expect(store.count() == 2, "count == 2") - - // A present key decodes the buffered `Entry?` into a struct value. The - // producer stores no tags or metadata, so the nested list and map fields - // decode as empty collections, and the absent TTL as nil. - let fetched = try store.get(key: "alpha") - expect(fetched?.key == "alpha", "get alpha key") - expect(fetched?.value == payload, "get alpha value") - expect(fetched?.expiresAt == nil, "get alpha expiresAt nil") - expect(fetched?.tags == [], "get alpha tags empty") - expect(fetched?.metadata == [:], "get alpha metadata empty") - expect(fetched?.id == 1, "first entry id is 1 (got \(fetched?.id ?? -1))") - - // A TTL round-trips through the buffered optional in both directions. - _ = try store.put(key: "timed", value: payload, kind: .volatile, ttlSeconds: 3600) - let timed = try store.get(key: "timed") - expect(timed?.expiresAt != nil && timed!.expiresAt! - timed!.createdAt == 3600, - "ttl produces expiresAt = createdAt + 3600") - expect(try store.delete(key: "timed"), "delete timed") - - // Iterator lowering: a lazy single-pass Sequence pulled one element per - // step, drained here into [String] (the BTreeMap's sorted order). A - // per-next producer error would end iteration and set `.error`. - let keysIter = try store.listKeys(prefix: nil) - let keys = Array(keysIter) - expect(keysIter.error == nil, "listKeys iteration error-free") - expect(keys == ["alpha", "beta"], "listKeys sorted (got \(keys))") - expect(Array(try store.listKeys(prefix: "al")) == ["alpha"], "listKeys prefix filter") - var seen: [String] = [] - for k in try store.listKeys(prefix: nil) { seen.append(k) } - expect(seen == ["alpha", "beta"], "for-in over listKeys") - expect(try store.listKeys(prefix: "zzz").next() == nil, "empty prefix match yields nothing") - - // A missing key raises the typed domain error's keyNotFound case (1001). +func open(_ path: String) -> Store { do { - _ = try store.get(key: "missing") - fail("expected KvError.keyNotFound for missing key") - } catch let e as KvError { - guard case let .keyNotFound(message) = e else { fail("expected .keyNotFound, got \(e)") } - expect(e.errorCode == 1001, "keyNotFound code == 1001 (got \(e.errorCode))") - expect(message == "key not found", "keyNotFound message (got \(message))") - } - - // The memberwise init builds a local Entry value with its list and map - // fields populated; the struct's stored properties read straight back. - let entry = Entry( - id: 7, key: "alpha", value: payload, createdAt: 1000, expiresAt: nil, - tags: ["hot", "fast"], metadata: ["source": "test", "env": "prod"]) - expect(entry.id == 7, "entry id") - expect(entry.tags == ["hot", "fast"], "tags") - expect(entry.metadata.count == 2 && entry.metadata["source"] == "test" && entry.metadata["env"] == "prod", - "metadata") - - // Nested Kv.Stats submodule (name collides with the module-level Stats - // struct), passing the Store interface across the module boundary and - // decoding the Stats record from a value buffer. - let stats = try Kv.Stats.getStats(store: store) - expect(stats.totalEntries == 2, "stats totalEntries") - expect(stats.totalBytes == 6, "stats totalBytes") - expect(stats.expiredEntries == 0, "stats expiredEntries") - - // --- EvictionListener callback interface --------------------------------- - var listener: Listener? = Listener() - weak var weakListener = listener - store.setEvictionListener(listener: listener!) - expect(try store.delete(key: "beta"), "delete beta") - expect(listener!.evicted.count == 1, "one eviction observed") - expect(listener!.evicted[0].key == "beta" && listener!.evicted[0].reason == .deleted, - "delete evicts beta with reason .deleted (got \(listener!.evicted))") - expect(listener!.lastEntry?.value == payload && listener!.lastEntry?.id == 2, - "the evicted Entry carried its payload and id") - expect(!(try store.delete(key: "beta")), "second delete reports absent") - expect(listener!.evicted.count == 1, "absent delete does not notify") - - // TTL expiry: reading an already-expired entry raises the expired case - // (1002) and evicts it with reason .expired. - _ = try store.put(key: "ttl", value: payload, kind: .volatile, ttlSeconds: -1) + return try Store.open(path: path) + } catch { + fail("open(\(path)) threw \(error)") + } +} + +/// A thread-safe tally shared by the test and the callbacks it creates. +final class Tally: @unchecked Sendable { + private let lock = NSLock() + private var counts: [String: Int] = [:] + private var notes: [String: String] = [:] + + func add(_ name: String) { + lock.lock() + counts[name, default: 0] += 1 + lock.unlock() + } + + subscript(name: String) -> Int { + lock.lock() + defer { lock.unlock() } + return counts[name, default: 0] + } + + func note(_ name: String, _ value: String) { + lock.lock() + notes[name] = value + lock.unlock() + } + + func noted(_ name: String) -> String? { + lock.lock() + defer { lock.unlock() } + return notes[name] + } +} + +/// Every callback implementation the library released, by kind. +enum Released { + static let tally = Tally() +} + +struct Refused: LocalizedError { + let errorDescription: String? + init(_ message: String) { errorDescription = message } +} + +// MARK: Callback implementations + +/// Records every change into `tally`; refuses to look at `failOn`. +final class RecordingListener: Listener, @unchecked Sendable { + let tally: Tally + let skip: String? + let failOn: String? + + init(_ tally: Tally, skip: String? = nil, failOn: String? = nil) { + self.tally = tally + self.skip = skip + self.failOn = failOn + } + + deinit { + Released.tally.add("listener") + } + + func accepts(key: String) throws -> Bool { + if key == failOn { throw Refused("listener refused") } + return key != skip + } + + func onChange(change: Change) throws { + if !Thread.isMainThread { tally.add("offMain") } + switch change { + case let .put(entry, replaced): + tally.note("version", String(entry.version)) + tally.note("replaced", String(replaced)) + tally.note("key", entry.key) + tally.add("puts") + case let .removed(key, expired): + tally.note("key", key) + tally.add(expired ? "expired" : "removed") + case let .cleared(count): + tally.note("cleared", String(count)) + tally.add("cleared") + } + } +} + +/// Tags admitted entries, vetoes secrets with a typed error, fails on +/// "boom", and routes "b/" keys to `other`. +final class TestPolicy: Policy, @unchecked Sendable { + let tally: Tally + let other: Store + + init(_ tally: Tally, other: Store) { + self.tally = tally + self.other = other + } + + deinit { + Released.tally.add("policy") + } + + func admit(entry: Entry) throws -> Entry { + tally.add("admitted") + expect(entry.version == 0, "admit sees version 0") + if entry.key.hasPrefix("secret") { + throw KvError.rejected(message: "secrets are not stored", key: entry.key, reason: "no secrets") + } + if entry.key.hasPrefix("boom") { + throw Refused("policy exploded") + } + var admitted = entry + admitted.tags = ["admitted"] + admitted.kind = .encrypted + admitted.key = "renamed" // the store keeps the original key + return admitted + } + + func route(key: String, home: Store) throws -> Store { + key.hasPrefix("b/") ? other : home + } +} + +/// Loads "loaded:{key}", falls back to `backup` for "fb", and fails for +/// "missing" (this key), "elsewhere" (another key), and "broken". +final class TestLoader: Loader, @unchecked Sendable { + let backup: Store? + + init(backup: Store? = nil) { + self.backup = backup + } + + deinit { + Released.tally.add("loader") + } + + func name() throws -> String { + "swift-loader" + } + + func fallback(key: String) throws -> Store? { + key == "fb" ? backup : nil + } + + func load(key: String) throws -> Data { + switch key { + case "missing": throw KvError.keyNotFound(message: "not in the loader", key: "missing") + case "elsewhere": throw KvError.keyNotFound(message: "not in the loader", key: "other") + case "broken": throw Refused("loader is broken") + default: return Data("loaded:\(key)".utf8) + } + } +} + +/// Calls the deprecated `size()` through a protocol witness, which doesn't +/// warn at the call site. +protocol Sized { + func size() -> UInt32 +} + +extension Store: Sized {} + +// MARK: Sections + +func constructors() async { + expectKvError("open(\"\")", { try Store.open(path: "") }) { error in + guard case let .invalidPath(message) = error else { return false } + return message == "invalid path" && error.errorCode == 1004 + } + let s = Store() + expect(s.path() == "memory", "Store() path") + expect(s.capacity() == Store.defaultCapacity() && Store.defaultCapacity() == 1_000_000, "capacity") + do { - _ = try store.get(key: "ttl") - fail("expected KvError.expired for expired entry") - } catch let e as KvError { - guard case .expired = e else { fail("expected .expired, got \(e)") } - expect(e.errorCode == 1002, "expired code == 1002 (got \(e.errorCode))") - } - expect(listener!.evicted.count == 2 && listener!.evicted[1].key == "ttl" - && listener!.evicted[1].reason == .expired, - "expiry evicts ttl with reason .expired (got \(listener!.evicted))") - expect(store.count() == 1, "expired entry left the store") - - // The store retains the listener after the consumer drops it, and - // releases it (running the generated `free`) when it is replaced. - listener = nil - expect(weakListener != nil, "store retains the listener") - var second: Listener? = Listener(keepAfter: 2) - weak var weakSecond = second - store.setEvictionListener(listener: second!) - expect(weakListener == nil, "replaced listener is freed") - - // Returning false detaches (and frees) the listener; later evictions are - // not observed. - _ = try store.put(key: "one", value: payload, kind: .volatile, ttlSeconds: nil) - _ = try store.put(key: "two", value: payload, kind: .volatile, ttlSeconds: nil) - _ = try store.put(key: "three", value: payload, kind: .volatile, ttlSeconds: nil) - expect(try store.delete(key: "one"), "delete one") - expect(try store.delete(key: "two"), "delete two") - expect(second!.evicted.map { $0.key } == ["one", "two"], "second saw two evictions") - second = nil - expect(weakSecond == nil, "listener that returned false was detached and freed") - expect(try store.delete(key: "three"), "delete three after detach") - - // A listener that throws surfaces to the caller as the foreign error - // (code -4) carrying the thrown description; the eviction itself already - // happened, and the store stays usable. - let thrower = Listener(failKey: "boom") - store.setEvictionListener(listener: thrower) - _ = try store.put(key: "boom", value: payload, kind: .volatile, ttlSeconds: nil) - _ = try store.put(key: "fine", value: payload, kind: .volatile, ttlSeconds: nil) + let opened = try await Kv.openStore(path: "/async") + expect(opened.path() == "/async", "openStore path") + } catch { + fail("openStore threw \(error)") + } do { - _ = try store.delete(key: "boom") - fail("expected a foreign error from the throwing listener") - } catch let e as KvstoreRuntimeError { - expect(e.errorCode == -4, "foreign error code == -4 (got \(e.errorCode))") - expect(e.localizedDescription.contains("listener rejected boom"), - "foreign error carries the thrown message (got \(e.localizedDescription))") - } - expect(store.count() == 2, "boom was removed before the listener ran") - expect(try store.delete(key: "fine"), "store usable after a foreign error") - expect(thrower.evicted.map { $0.key } == ["fine"], "non-failing eviction still observed") - // An expired read that trips the listener also surfaces as -4 rather - // than the domain code. - _ = try store.put(key: "boom", value: payload, kind: .volatile, ttlSeconds: -1) + _ = try await Kv.openStore(path: "") + fail("openStore(\"\") returned") + } catch let error as KvError { + guard case let .invalidPath(message) = error else { fail("openStore(\"\") threw \(error)") } + expect(message == "invalid path", "openStore(\"\") message") + } catch { + fail("openStore(\"\") threw \(error)") + } +} + +func basics() { + let s = open("/basics") + let first = put(s, "alpha", "one") + expect(first == Entry(key: "alpha", value: Data("one".utf8), kind: .persistent, version: 1, expiresAt: nil, tags: [], metadata: [:]), "first put \(first)") + let second = put(s, "alpha", "two", .volatile) + expect(second.version == 2 && second.kind == .volatile, "second put") + expect((try? s.get(key: "alpha"))?.value == Data("two".utf8), "get alpha") + expectKvError("get(nope)", { try s.get(key: "nope") }) { error in + isKeyNotFound("nope")(error) && error.localizedDescription == "key not found: nope" + } + expect(s.find(key: "alpha")?.version == 2 && s.find(key: "nope") == nil, "find") + + // TTLs follow the logical clock; an expired get reports when. + expect(put(s, "ttl", "x", .volatile, ttl: 10).expiresAt == 10, "expiresAt") + expect(s.now() == 0, "now starts at 0") + expect(s.tick(seconds: 9) == 9 && s.count() == 2, "tick 9") + expect(s.tick(seconds: 1) == 10 && s.count() == 1, "tick 10") + expectKvError("get(ttl)", { try s.get(key: "ttl") }) { error in + guard case let .expired(_, key, expiredAt) = error else { return false } + return key == "ttl" && expiredAt == 10 + } + expectKvError("get(ttl) again", { try s.get(key: "ttl") }, isKeyNotFound("ttl")) + + // Capacity: a new key past it is StoreFull { capacity }. + s.setCapacity(capacity: 1) + expect(s.capacity() == 1, "setCapacity") + put(s, "alpha", "three") + expectKvError("put(beta)", { try s.put(key: "beta", value: Data(), kind: .volatile, ttlSeconds: nil) }) { error in + guard case let .storeFull(_, capacity) = error else { return false } + return capacity == 1 + } + s.setCapacity(capacity: 100) + + put(s, "beta", "b") + expect(s.delete(key: "beta") && !s.delete(key: "beta"), "delete") + expect((s as Sized).size() == s.count() && s.count() == 1, "deprecated size") + expect(s.clear() == 1 && s.count() == 0, "clear") +} + +func iterators() { + let s = open("/iter") + put(s, "user.bob", "b") + put(s, "user.alice", "a") + put(s, "sys.x", "xx") + do { - _ = try store.get(key: "boom") - fail("expected a foreign error from the throwing listener on expiry") - } catch let e as KvstoreRuntimeError { - expect(e.errorCode == -4, "expiry foreign error code == -4 (got \(e.errorCode))") - } - store.clearEvictionListener() - _ = try store.put(key: "boom", value: payload, kind: .volatile, ttlSeconds: nil) - expect(try store.delete(key: "boom"), "delete boom with no listener attached") - expect(thrower.evicted.count == 1, "cleared listener no longer notified") - store.clearEvictionListener() - - // --- Object graph: share, fork, larger, describe, openMany, totalCount --- - expect(store.count() == 1, "only alpha remains (got \(store.count()))") - - // share(): a second wrapper to the SAME object; mutation through one is - // visible through the other, and dropping one leaves the other usable. - var shared: Store? = store.share() - weak var weakShared = shared - _ = try shared!.put(key: "via-share", value: payload, kind: .volatile, ttlSeconds: nil) - expect(store.count() == 2, "put through share() visible through the original") - expect(Array(try shared!.listKeys(prefix: nil)) == ["alpha", "via-share"], "share() reads the same entries") - shared = nil - expect(weakShared == nil, "shared wrapper deinitialized") - expect(store.count() == 2, "original still alive after releasing the shared wrapper") - - // fork(): an independent copy. - let forked = store.fork() - expect(forked.count() == 2, "fork copies live entries") - _ = try forked.put(key: "fork-only", value: payload, kind: .volatile, ttlSeconds: nil) - expect(forked.count() == 3 && store.count() == 2, "fork is independent of the original") + let keys = try s.keys(prefix: nil) + expect(Array(keys) == ["sys.x", "user.alice", "user.bob"], "keys in order") + expect(keys.error == nil && keys.next() == nil, "keys stays at the end") + } catch { + fail("keys threw \(error)") + } + expectKvError("keys(zzz)", { try s.keys(prefix: "zzz") }, isKeyNotFound("zzz")) + + // Abandoning a sequence part-way releases the native iterator. do { - _ = try store.get(key: "fork-only") - fail("expected KvError.keyNotFound: a fork's insert must not reach the original") - } catch let e as KvError { - guard case .keyNotFound = e else { fail("expected .keyNotFound from the fork probe, got \(e)") } + let keys = try s.keys(prefix: "user.") + withExtendedLifetime(keys) { + expect(keys.next() == "user.alice", "first user key") + expect(kvstore_debug_live(2) == 1, "one live iterator") + } + } catch { + fail("keys(user.) threw \(error)") + } + expect(kvstore_debug_live(2) == 0, "the abandoned iterator was destroyed") + + let entries = Array(s.entries(prefix: "sys.")) + expect(entries.count == 1 && entries[0].key == "sys.x" && entries[0].value == Data("xx".utf8), "entries(sys.)") + + let prefixes = ["user.", "sys.", "none."] + let parts = Array(s.partition(prefixes: prefixes)) + expect(parts.map { $0.count() } == [2, 1, 0], "partition counts") + expect(parts.map { $0.path() } == prefixes, "partition paths") + expect(!parts.contains { isSame($0, s) }, "partitions are new stores") +} + +func listeners() { + let tally = Tally() + let store = open("/listen") + + let id = store.subscribe(listener: RecordingListener(tally, skip: "quiet")) + expect(id > 0 && store.listenerCount() == 1, "subscribe") + + put(store, "a", "1") + expect(tally["puts"] == 1 && tally.noted("version") == "1" && tally.noted("replaced") == "false", "first Put") + put(store, "a", "2") + expect(tally["puts"] == 2 && tally.noted("version") == "2" && tally.noted("replaced") == "true", "second Put") + expect(tally.noted("key") == "a", "Put key") + put(store, "quiet", "x") + expect(tally["puts"] == 2, "accepts filtered quiet") + expect(store.delete(key: "a") && tally["removed"] == 1 && tally.noted("key") == "a", "Removed") + + // An expired read removes the entry and says so. + put(store, "short", "x", .volatile, ttl: 1) + _ = store.tick(seconds: 1) + expectKvError("get(short)", { try store.get(key: "short") }) { error in + if case .expired = error { return true } + return false + } + expect(tally["expired"] == 1 && tally.noted("key") == "short", "Removed(expired)") + expect(store.clear() == 1 && tally["cleared"] == 1 && tally.noted("cleared") == "1", "Cleared") + expect(tally["offMain"] == 0, "synchronous calls notify on the calling thread") + + // Unsubscribing releases the listener once. + expect(store.unsubscribe(id: id) && Released.tally["listener"] == 1, "unsubscribe releases") + expect(!store.unsubscribe(id: id) && store.listenerCount() == 0, "unsubscribe again") + + // A listener that throws is detached (and released); the put succeeds. + let failing = Tally() + _ = store.subscribe(listener: RecordingListener(failing, failOn: "boom")) + put(store, "fine", "1") + expect(failing["puts"] == 1, "failing listener saw fine") + put(store, "boom", "1") + expect(store.count() == 2 && store.listenerCount() == 0, "failing listener detached") + expect(Released.tally["listener"] == 2, "failing listener released") + + // Releasing the store releases the listeners it still holds. + _ = store.subscribe(listener: RecordingListener(Tally())) + _ = store.subscribe(listener: RecordingListener(Tally())) + expect(store.listenerCount() == 2, "two listeners") +} + +func listenersReleasedWithStore() { + listeners() + expect(Released.tally["listener"] == 4, "releasing the store released its listeners (got \(Released.tally["listener"]))") +} + +func policies() { + let tally = Tally() + let s = open("/policy") + let other = open("/other") + + s.setPolicy(policy: TestPolicy(tally, other: other)) + expect(s.hasPolicy(), "hasPolicy") + + // admit's record return is what's stored (its key and version aside). + let a = put(s, "a", "1", .volatile) + expect(a.key == "a" && a.version == 1 && a.kind == .encrypted && a.tags == ["admitted"], "admitted entry \(a)") + + // route: the object parameter and object return redirect a write. + put(s, "b/x", "2", .volatile) + expect(s.count() == 1 && other.count() == 1, "route redirected b/x") + + // A typed error from the throwing callback reaches the caller with its + // code, message, and fields. + expectKvError("put(secret)", { try s.put(key: "secret", value: Data(), kind: .volatile, ttlSeconds: nil) }) { error in + guard case let .rejected(message, key, reason) = error else { return false } + return message == "secrets are not stored" && key == "secret" && reason == "no secrets" && error.errorCode == 1005 + } + // Any other error arrives as -4 with the consumer's message. + expectRuntimeError("put(boom)", code: -4, message: "policy exploded") { + try s.put(key: "boom", value: Data(), kind: .volatile, ttlSeconds: nil) } -} catch { - fail("threw: \(error)") + expect(s.count() == 1 && other.count() == 1, "failed puts stored nothing") + expect(tally["admitted"] == 4, "admit calls") + + // Replacing the policy releases the old one; nil removes it. + s.setPolicy(policy: TestPolicy(tally, other: other)) + expect(Released.tally["policy"] == 1, "replaced policy released") + s.setPolicy(policy: nil) + expect(Released.tally["policy"] == 2 && !s.hasPolicy(), "removed policy released") + put(s, "secret", "now allowed") + expect(s.count() == 2, "no policy, no veto") } -do { - let payload = Data([1, 2, 3]) - let small = try Store.open(path: "/tmp/small") - let big = try Store.open(path: "/tmp/big") - _ = try big.put(key: "k", value: payload, kind: .volatile, ttlSeconds: nil) - - // larger(): Store? as parameter and return. - expect(small.larger(other: nil) == nil, "larger(nil) on an empty store is nil") - let bigger = small.larger(other: big) - expect(bigger != nil && bigger!.count() == 1, "larger picks the fuller store") - _ = try bigger!.put(key: "k2", value: payload, kind: .volatile, ttlSeconds: nil) - expect(big.count() == 2, "the returned wrapper is the same object as `big`") - let own = big.larger(other: nil) - expect(own != nil && own!.count() == 2, "larger(nil) on a non-empty store returns itself") - _ = try own!.put(key: "k3", value: payload, kind: .volatile, ttlSeconds: nil) - expect(big.count() == 3, "self returned from larger(nil) is the same object") - expect(small.larger(other: small)?.count() == 0, "larger(self) returns a store") - - // describe(): a record carrying an object field and an optional object. - let info = big.describe(label: "primary", mirror: nil) - expect(info.label == "primary", "describe label") - expect(info.count == 3, "describe count (got \(info.count))") - expect(info.mirror == nil, "describe mirror absent") - expect(info.store.count() == 3, "describe().store is usable") - _ = try info.store.put(key: "k4", value: payload, kind: .volatile, ttlSeconds: nil) - expect(big.count() == 4, "describe().store is the same object as the receiver") - let mirrored = big.describe(label: "with-mirror", mirror: small) - expect(mirrored.mirror != nil && mirrored.mirror!.count() == 0, "describe mirror present") - _ = try mirrored.mirror!.put(key: "s", value: payload, kind: .volatile, ttlSeconds: nil) - expect(small.count() == 1, "describe().mirror is the same object as the argument") - - // openMany(): a list of objects as a return; each is a distinct store. - let many = try Store.openMany(paths: ["/a", "/b", "/c"]) - expect(many.count == 3, "openMany returns three stores (got \(many.count))") - _ = try many[0].put(key: "m", value: payload, kind: .volatile, ttlSeconds: nil) - expect(many[0].count() == 1 && many[1].count() == 0 && many[2].count() == 0, - "openMany stores are independent") +func loaders() { + let s = open("/load") + func load(_ key: String, _ loader: TestLoader?) throws -> Entry? { + try s.getOrLoad(key: key, loader: loader) + } + func loadOK(_ key: String, _ loader: TestLoader?) -> Entry? { + do { + return try load(key, loader) + } catch { + fail("getOrLoad(\(key)) threw \(error)") + } + } + + // No loader (a nil optional callback): a miss is nil. + expect(loadOK("k", nil) == nil, "no loader") + + // load's bytes are stored, tagged with the loader's name. + guard let loaded = loadOK("k", TestLoader()) else { fail("getOrLoad(k)") } + expect(loaded.value == Data("loaded:k".utf8) && loaded.kind == .volatile, "loaded entry") + expect(loaded.metadata == ["source": "swift-loader"], "loaded metadata") + expect(Released.tally["loader"] == 1, "a loader is released after the call") + expect(s.count() == 1, "loaded entry stored") + // A hit doesn't consult the loader. + expect(loadOK("k", TestLoader())?.version == 1, "hit") + + // The fallback store (an optional object return) is consulted first. + let backup = open("/backup") + put(backup, "fb", "from backup") + guard let copied = loadOK("fb", TestLoader(backup: backup)) else { fail("getOrLoad(fb)") } + expect(copied.value == Data("from backup".utf8) && copied.kind == .persistent, "fallback entry") + + // KeyNotFound for this key: the producer decoded the payload and + // answers nil. + expect(loadOK("missing", TestLoader()) == nil, "missing") + // KeyNotFound for another key: passed through, payload intact. + expectKvError("getOrLoad(elsewhere)", { try load("elsewhere", TestLoader()) }) { error in + error.localizedDescription == "not in the loader" && isKeyNotFound("other")(error) + } + // Any other failure is -4. + expectRuntimeError("getOrLoad(broken)", code: -4, message: "loader is broken") { + try load("broken", TestLoader()) + } + expect(Released.tally["loader"] == 6, "every loader released (got \(Released.tally["loader"]))") +} + +@MainActor +func asyncCalls() async { + let tally = Tally() + let s = open("/async-calls") + _ = s.subscribe(listener: RecordingListener(tally)) + put(s, "old1", "x", .volatile, ttl: 1) + put(s, "old2", "x", .volatile, ttl: 1) + put(s, "keep", "x") + _ = s.tick(seconds: 5) + do { - _ = try Store.openMany(paths: ["/ok", ""]) - fail("expected KvError.ioError from openMany with an empty path") - } catch let e as KvError { - guard case .ioError = e else { fail("expected .ioError, got \(e)") } - expect(e.errorCode == 1004, "openMany ioError code == 1004") - } - - // totalCount(): objects inside a list parameter and inside an optional - // record parameter. Every object is cloned into the buffer, so the - // wrappers stay valid afterwards. - expect(Store.totalCount(stores: many, extra: nil) == 1, "totalCount over the list only") - expect(Store.totalCount(stores: [big, small, many[0]], extra: nil) == 6, "totalCount mixed (4 + 1 + 1)") - let extra = StoreInfo(label: "extra", store: big, mirror: small, count: -1) - expect(Store.totalCount(stores: many, extra: extra) == 5, "totalCount adds extra.store (1 + 4)") - expect(Store.totalCount(stores: [], extra: info) == 4, "totalCount with an empty list and a record") - expect(Store.totalCount(stores: [], extra: nil) == 0, "totalCount of nothing") - expect(big.count() == 4 && small.count() == 1 && many[0].count() == 1, - "every store is still usable after being encoded into buffers") - - // Wrappers release through deinit; a weak reference proves it ran. - weak var weakTmp: Store? + // compact runs on a producer thread and notifies listeners there. + expect(try await s.compact(pauseMs: 0) == 2, "compact(0)") + expect(tally["expired"] == 2 && tally["offMain"] == 2, "compaction notified from a producer thread") + expect(s.count() == 1, "compacted") + expect(try await s.compact(pauseMs: 5) == 0, "compact(5)") + } catch { + fail("compact threw \(error)") + } + + // Cancelling the task cancels the call mid-pause: it throws + // CancellationError at once, and the background pause stops. + let pause = Task { try await s.compact(pauseMs: 60_000) } + try? await Task.sleep(nanoseconds: 20_000_000) + expect(Store.activeJobs() >= 1, "the pause is running") + let cancelledAt = Date() + pause.cancel() do { - let tmp = try Store.open(path: "/tmp/tmp") - weakTmp = tmp - expect(tmp.count() == 0, "tmp usable") - } - expect(weakTmp == nil, "temporary store wrapper deinitialized") - // A wrapper adopted from a record keeps the object alive after the - // wrapper it came from is gone. - var owner: Store? = try Store.open(path: "/tmp/owner") - _ = try owner!.put(key: "x", value: payload, kind: .volatile, ttlSeconds: nil) - let ownerInfo = owner!.describe(label: "o", mirror: nil) - owner = nil - expect(ownerInfo.store.count() == 1, "object outlives the wrapper that described it") - - // Async: an immediately-expired entry gives compact 3 bytes to reclaim; - // the continuation resumes from the producer's worker thread. - _ = try big.put(key: "doomed", value: payload, kind: .volatile, ttlSeconds: 0) - expect(try Kv.Stats.getStats(store: big).expiredEntries == 1, "stats counts the doomed entry as expired") - let reclaimed = try await big.compact() - expect(reclaimed == 3, "compact reclaimed 3 bytes (got \(reclaimed))") - expect(big.count() == 4, "compact dropped only the expired entry") - expect(try await small.compact() == 0, "compact with nothing expired reclaims 0") - - // Non-throwing void method plus the static, neither needing `try`. - big.clear() - expect(big.count() == 0, "clear drops everything") - expect(Store.defaultCapacity() == 1_000_000, "defaultCapacity") - - // The throwing constructor rejects an empty path with ioError (1004). + _ = try await pause.value + fail("the cancelled compact returned") + } catch is CancellationError { + expect(Date().timeIntervalSince(cancelledAt) < 2, "cancelled promptly") + } catch { + fail("the cancelled compact threw \(error)") + } + var stopped = false + for _ in 0..<2000 where !stopped { + stopped = Store.activeJobs() == 0 + if !stopped { usleep(1000) } + } + expect(stopped, "the cancelled pause stopped cooperatively") + + // getMany: an async list of optional records, called concurrently. + let results = await withTaskGroup(of: [Entry?].self) { group in + for _ in 0..<32 { + group.addTask { await s.getMany(keys: ["keep", "gone", "keep"]) } + } + var all: [[Entry?]] = [] + for await r in group { all.append(r) } + return all + } + expect(results.count == 32, "32 getMany calls") + for r in results { + expect(r.count == 3 && r[0]?.key == "keep" && r[1] == nil && r[2]?.key == "keep", "getMany result \(r)") + } + + // The nested module's async function: objects in a list in, a record out. + let other = open("/other") + put(other, "a", "123") + let stats = await Kv.Stats.summarizeAll(stores: [s, other]) + expect(stats == Stats(entries: 2, bytes: 4, byKind: [.persistent: 2]), "summarizeAll \(stats)") +} + +func objectGraph() { + var original: Store? = open("/graph") + put(original!, "k", "v") + + // share(): the same object; the original reference can go. + let s = original!.share() + expect(isSame(s, original!), "share is the same store") + original = nil + _ = original + expect(s.count() == 1, "alive through the shared reference") + + // fork(): a distinct object with a copy of the entries. + let fork = s.fork() + expect(!isSame(fork, s) && fork.count() == 1 && fork.path() == "/graph", "fork") + put(fork, "k2", "v") + expect(fork.count() == 2 && s.count() == 1, "fork is independent") + + // larger(): `Store?` in and out. + let empty = open("/empty") + expect(empty.larger(other: nil) == nil, "larger(nil) on an empty store") + expect(empty.larger(other: fork).map { isSame($0, fork) } == true, "larger(fork)") + expect(s.larger(other: nil).map { isSame($0, s) } == true, "larger(nil) is self") + + // describe(): a record whose fields carry objects. + let info = s.describe(label: "main", mirror: fork) + expect(info.label == "main" && isSame(info.store, s) && info.count == 1, "describe") + expect(info.mirror.map { isSame($0, fork) } == true && info.mirror?.count() == 2, "describe mirror") + + // openMany(): a list of objects; one bad path fails the whole call. + guard let many = try? Store.openMany(paths: ["/a", "/b"]) else { fail("openMany") } + expect(many.map { $0.path() } == ["/a", "/b"], "openMany paths") + expectKvError("openMany([/a, \"\"])", { try Store.openMany(paths: ["/a", ""]) }) { error in + if case .invalidPath = error { return true } + return false + } + + // byLabel(): records with objects in, a map with object values out. + let named = Store.byLabel(infos: [info, StoreInfo(label: "first", store: many[0], mirror: nil, count: 0)]) + expect(named.count == 2, "byLabel count") + expect(named["main"].map { isSame($0, s) } == true, "byLabel main") + expect(named["first"].map { isSame($0, many[0]) } == true, "byLabel first") + + // totalCount(): a list, a map, and an optional record, all carrying + // objects (each encoded as a fresh reference the producer adopts). + put(many[0], "m", "1") + // stores: 1 + 0 + 2; named: main 1 + first 1; extra: main 1. + expect(Store.totalCount(stores: [many[0], many[1], fork], named: named, extra: info) == 6, "totalCount") + expect(Store.totalCount(stores: [many[0], many[1], fork], named: named, extra: nil) == 5, "totalCount without extra") + + // Everything is still intact after crossing in buffers. + expect(s.count() == 1 && fork.count() == 2 && many[0].count() == 1, "objects intact") +} + +func statsAndReport() { + let s = open("/stats") + put(s, "b", "12") + put(s, "a", "1") + put(s, "a", "123") + put(s, "c", "x", .encrypted) + + // kv.stats.summarize: the parent's Store as a parameter, the parent's + // error domain for a prefix that matches nothing. do { - _ = try Store.open(path: "") - fail("expected KvError.ioError for empty path") - } catch let e as KvError { - guard case let .ioError(message) = e else { fail("expected .ioError, got \(e)") } - expect(e.errorCode == 1004, "ioError code == 1004 (got \(e.errorCode))") - expect(message == "I/O failure", "ioError message (got \(message))") + let stats = try Kv.Stats.summarize(store: s, prefix: nil) + expect(stats == Stats(entries: 3, bytes: 6, byKind: [.persistent: 2, .encrypted: 1]), "summarize \(stats)") + } catch { + fail("summarize threw \(error)") } + expectKvError("summarize(q)", { try Kv.Stats.summarize(store: s, prefix: "q") }, isKeyNotFound("q")) - // No explicit close: every Store deinit calls the generated destroy. -} catch { - fail("threw: \(error)") + // report.render_report: the sibling root shares the Entry record. + do { + let lines = try Report.renderReport(entries: Array(s.entries(prefix: nil))) + expect(lines == ["a: 3 bytes, Persistent, v2", "b: 2 bytes, Persistent", "c: 1 bytes, Encrypted"], "renderReport \(lines)") + } catch { + fail("renderReport threw \(error)") + } + do { + _ = try Report.renderReport(entries: []) + fail("renderReport([]) returned") + } catch let error as ReportError { + guard case let .nothingToReport(message) = error else { fail("renderReport([]) threw \(error)") } + expect(message == "nothing to report" && error.errorCode == 2001, "NothingToReport") + } catch { + fail("renderReport([]) threw \(error)") + } } +await constructors() +basics() +iterators() +listenersReleasedWithStore() +policies() +loaders() +await asyncCalls() +objectGraph() +statsAndReport() + +expect(kvstore_abi_version() == 4, "ABI revision 4") +expect(kvstore_debug_live(-1) == 1, "the sample counts live resources") assertNoLeaks() +expect(Released.tally["listener"] == 5, "every listener released (got \(Released.tally["listener"]))") +expect(Released.tally["policy"] == 2, "every policy released (got \(Released.tally["policy"]))") print("swift/kvstore: OK") diff --git a/conformance/swift/run.sh b/conformance/swift/run.sh index 43417783..7a278fe4 100755 --- a/conformance/swift/run.sh +++ b/conformance/swift/run.sh @@ -5,6 +5,7 @@ # consumer against one sample and must exit 0. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools swift swift # Swift: a throwaway executable package depends on the generated SwiftPM # package as-is through a local path dependency. The generated C module links @@ -26,7 +27,7 @@ swift_consumer() { import PackageDescription let package = Package( name: "conformance", - platforms: [.macOS(.v10_15)], + platforms: [.macOS("11.0")], dependencies: [.package(path: "$gen")], targets: [ .executableTarget( @@ -39,20 +40,12 @@ EOF ( cd "$pkg" && swift build -Xlinker -L"$LIBDIR" 2>&1 && .build/debug/conformance ) } -swift_contacts() { swift_consumer contacts contacts.swift; } +swift_calculator() { swift_consumer calculator calculator.swift; } +swift_codec() { swift_consumer codec codec.swift; } +swift_kvstore() { swift_consumer kvstore kvstore.swift; } -swift_events() { swift_consumer events events.swift; } - -swift_kvstore() { swift_consumer kvstore kvstore.swift; } - -swift_async_demo() { swift_consumer async-demo async_demo.swift; } - -swift_codec() { swift_consumer codec codec.swift; } - -lane swift-contacts swift_contacts -lane swift-events swift_events -lane swift-kvstore swift_kvstore -lane swift-async-demo swift_async_demo +lane swift-calculator swift_calculator lane swift-codec swift_codec +lane swift-kvstore swift_kvstore finish_lanes diff --git a/conformance/wasm/run.sh b/conformance/wasm/run.sh index f7bb954c..21a15264 100755 --- a/conformance/wasm/run.sh +++ b/conformance/wasm/run.sh @@ -12,11 +12,11 @@ # (conformance/node/*.mjs): both targets generate the same JavaScript API. set -uo pipefail . "$(dirname "$0")/../lib.sh" +require_tools wasm node npm rustup wasm_consumer() { local sample="$1" src="$2" local dir="$OUT/wasm-$sample" tarball - command -v npm >/dev/null 2>&1 || { echo "npm not found" >&2; return 1; } rustup target list --installed 2>/dev/null | grep -qx 'wasm32-unknown-unknown' \ || { echo "wasm32-unknown-unknown target missing (rustup target add wasm32-unknown-unknown)" >&2; return 1; } cargo build -q -p "$sample" --release --target wasm32-unknown-unknown \ @@ -34,20 +34,12 @@ wasm_consumer() { ( cd "$dir" && env "$(library_env "$sample")=$wasm" node --expose-gc "$src" ) } -wasm_contacts() { wasm_consumer contacts contacts.mjs; } +wasm_calculator() { wasm_consumer calculator calculator.mjs; } +wasm_codec() { wasm_consumer codec codec.mjs; } +wasm_kvstore() { wasm_consumer kvstore kvstore.mjs; } -wasm_events() { wasm_consumer events events.mjs; } - -wasm_kvstore() { wasm_consumer kvstore kvstore.mjs; } - -wasm_async_demo() { wasm_consumer async-demo async_demo.mjs; } - -wasm_codec() { wasm_consumer codec codec.mjs; } - -lane wasm-contacts wasm_contacts -lane wasm-events wasm_events -lane wasm-kvstore wasm_kvstore -lane wasm-async-demo wasm_async_demo +lane wasm-calculator wasm_calculator lane wasm-codec wasm_codec +lane wasm-kvstore wasm_kvstore finish_lanes diff --git a/crates/weaveffi-abi/Cargo.toml b/crates/weaveffi-abi/Cargo.toml deleted file mode 100644 index e7ce5dd5..00000000 --- a/crates/weaveffi-abi/Cargo.toml +++ /dev/null @@ -1,23 +0,0 @@ -[package] -name = "weaveffi-abi" -version.workspace = true -edition.workspace = true -rust-version.workspace = true -license.workspace = true -authors.workspace = true -repository.workspace = true -description = "Stable C ABI runtime types and helpers for WeaveFFI" -keywords = ["ffi", "abi", "c-abi", "bindings", "codegen"] -categories = ["development-tools::ffi"] - -[features] -# Count live objects, callbacks, iterators, cancel tokens, and returned byte -# allocations so `{prefix}_debug_live` can report leaks. Off by default -# because every counted operation pays for an atomic update. -leak-check = [] - -[lints] -workspace = true - -[dev-dependencies] -proptest = "1.11.0" diff --git a/crates/weaveffi-abi/src/callback.rs b/crates/weaveffi-abi/src/callback.rs deleted file mode 100644 index f04a5d39..00000000 --- a/crates/weaveffi-abi/src/callback.rs +++ /dev/null @@ -1,516 +0,0 @@ -//! Consumer-implemented callback interfaces: the producer half of the ABI's -//! vtable contract. -//! -//! A callback-interface parameter arrives as two slots, a `void* ctx` the -//! consumer owns and a pointer to a process-wide static vtable with one -//! `extern "C"` function pointer per method plus a trailing `free(ctx)`. The -//! `#[weaveffi::module]` expansion wraps the pair in a [`ForeignCallback`], -//! implements the producer's trait on top of it (each method lowers its -//! arguments, calls the vtable entry, then checks `out_err`), and hands the -//! producer an `Arc`. When the last `Arc` drops, [`ForeignCallback`] -//! calls `free(ctx)` exactly once. -//! -//! A consumer implementation that fails reports through the method's -//! `out_err` slot (normally with [`FOREIGN_ERROR_CODE`](crate::FOREIGN_ERROR_CODE)), -//! and [`foreign_status`] turns that slot into a `Result`. What happens next -//! depends on how the producer declared the trait method: -//! -//! * A method returning `Result` gets the failure as an -//! `Err` and nothing unwinds. This is the recommended spelling. -//! * A method returning a plain `T` can't return the failure, so -//! [`raise_foreign_error`] delivers it to the enclosing thunk. On a -//! `panic = "unwind"` build (every native target by default) it unwinds -//! with a [`ForeignError`] payload, and the thunk's `catch_unwind` reports -//! it with the consumer's message. On a `panic = "abort"` build (notably -//! `wasm32-unknown-unknown`) it records the failure for the innermost -//! active thunk on this thread (see [`ThunkScope`]) and returns the vtable -//! entry's default value; the thunk reports the recorded failure in place -//! of its result. With no thunk active (the producer called back from a -//! thread of its own) the failure is written to stderr and dropped, so it -//! can never leak into a later, unrelated call. - -use std::cell::{Cell, RefCell}; -use std::ffi::c_void; -use std::marker::PhantomData; -use std::sync::Arc; - -use crate::FfiError; - -/// Implemented by every generated vtable struct so [`ForeignCallback`] can -/// find the trailing `free` entry without knowing the method layout. -pub trait Vtable: 'static { - /// The consumer's release hook, called exactly once when the producer - /// drops its last reference to the callback. - fn free(&self) -> unsafe extern "C" fn(*mut c_void); -} - -/// Ties a callback interface's `dyn Trait` to its generated vtable and -/// foreign wrapper. -/// -/// The `#[weaveffi::module]` expansion implements this for `dyn Trait` of -/// every `#[weaveffi::callback_interface]`, which lets a thunk that only knows -/// the producer's written type (`Arc`) name the vtable slot type -/// (`::Vtable`) and lift the pair into a -/// shared trait object with [`lift_callback`]. -pub trait CallbackInterface { - /// The generated `#[repr(C)]` vtable struct for this interface. - type Vtable: Vtable; - - /// Wrap a lifted foreign callback as a shared trait object. - fn from_foreign(cb: ForeignCallback) -> Arc; -} - -/// Lift a callback-interface parameter's `(ctx, vtable)` slots into the -/// `Arc` the producer's function takes. A null vtable yields -/// `None`, which the thunk reports as a marshalling failure. -/// -/// # Safety -/// -/// Same contract as [`ForeignCallback::from_raw`]: `vtable` must be null or -/// point to a fully initialized, immutable vtable that outlives every clone of -/// the returned `Arc`, and `ctx` must stay valid until the vtable's `free` -/// entry is called with it. -#[must_use] -pub unsafe fn lift_callback( - ctx: *mut c_void, - vtable: *const C::Vtable, -) -> Option> { - // SAFETY: forwarded from the caller. - unsafe { ForeignCallback::from_raw(ctx, vtable) }.map(C::from_foreign) -} - -/// A consumer-implemented callback interface held by the producer. -/// -/// Holds the consumer's `ctx` and a pointer to its static vtable `V`. The -/// generated `impl Trait for ForeignX` calls -/// [`vtable`](Self::vtable)`().method(ctx, ...)` per method; dropping the -/// last owner calls `free(ctx)`. -pub struct ForeignCallback { - ctx: *mut c_void, - vtable: *const V, -} - -// SAFETY: the ABI contract obliges the consumer to make every vtable entry -// callable from any thread, and `ctx` is only ever handed back to those -// entries. The producer never dereferences `ctx` itself. -unsafe impl Send for ForeignCallback {} -// SAFETY: see the `Send` impl; the vtable is immutable static consumer data. -unsafe impl Sync for ForeignCallback {} - -impl ForeignCallback { - /// Adopt a `(ctx, vtable)` pair lifted from a callback-interface - /// parameter's slots. Returns `None` when the vtable pointer is null, - /// which the thunk reports as a marshalling failure. - /// - /// # Safety - /// - /// `vtable` must be null or point to a fully initialized `V` that stays - /// valid and unmodified for the life of the returned value (consumers - /// use a process-wide static). `ctx` must remain valid until the - /// vtable's `free` entry is called with it. - #[must_use] - pub unsafe fn from_raw(ctx: *mut c_void, vtable: *const V) -> Option { - if vtable.is_null() { - return None; - } - crate::leak::track(crate::leak::CALLBACKS, 1); - Some(Self { ctx, vtable }) - } - - /// The consumer's context pointer, passed as the first argument of every - /// vtable entry. - #[must_use] - pub fn ctx(&self) -> *mut c_void { - self.ctx - } - - /// The consumer's vtable. - #[must_use] - pub fn vtable(&self) -> &V { - // SAFETY: `from_raw` rejected null and its contract keeps the vtable - // alive and immutable for the life of `self`. - unsafe { &*self.vtable } - } -} - -impl Drop for ForeignCallback { - fn drop(&mut self) { - let free = self.vtable().free(); - crate::leak::track(crate::leak::CALLBACKS, -1); - // SAFETY: `ctx` is handed back to the consumer's own release hook - // exactly once, as the ABI contract requires. - unsafe { free(self.ctx) }; - } -} - -/// A failure a consumer's callback-interface implementation reported. -/// -/// A trait method declared to return `Result` receives it as -/// an `Err`. For a method returning a plain `T` it's the unwind payload -/// [`raise_foreign_error`] raises, which the `#[weaveffi::module]` thunks -/// catch and report with `code` and `message`, so the consumer's own error -/// text round-trips through the producer. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct ForeignError { - /// The code the consumer wrote to `out_err` (normally - /// [`FOREIGN_ERROR_CODE`](crate::FOREIGN_ERROR_CODE)). - pub code: i32, - /// The consumer's message, copied out of `out_err` before it was freed. - pub message: String, -} - -impl std::fmt::Display for ForeignError { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str(&self.message) - } -} - -impl std::error::Error for ForeignError {} - -/// A `ForeignError` is reported with its own code and message, so a producer -/// function can propagate a callback failure with `?` when its error type is -/// `ForeignError`. -impl crate::ErrorReport for ForeignError { - fn code(&self) -> i32 { - self.code - } - fn message(&self) -> String { - self.message.clone() - } -} - -thread_local! { - /// How many thunks are running on this thread (nested calls stack). - static THUNK_DEPTH: Cell = const { Cell::new(0) }; - /// The failure recorded for the innermost running thunk on a - /// `panic = "abort"` build, waiting for that thunk to pick it up. - static PENDING_FOREIGN: RefCell> = const { RefCell::new(None) }; -} - -/// Marks a WeaveFFI thunk (or one poll of an async call's future) as running -/// on the current thread, for the lifetime of the value. -/// -/// A deferred foreign failure ([`defer_foreign_error`]) is only recorded -/// while a scope is active, and belongs to the innermost one: entering a -/// scope sets aside whatever an outer scope had pending and dropping it puts -/// that back, so a nested call (a consumer callback that calls into the -/// producer again) can neither steal nor leak its caller's failure. Read the -/// scope's own failure with [`take_foreign_error`] before it drops. -#[derive(Debug)] -pub struct ThunkScope { - outer: Option, - // Thread-local bookkeeping: the scope must end on the thread it began. - _not_send: PhantomData<*const ()>, -} - -impl ThunkScope { - /// Enter a scope on the current thread. - #[must_use] - pub fn enter() -> Self { - THUNK_DEPTH.with(|d| d.set(d.get() + 1)); - let outer = PENDING_FOREIGN.with(|slot| slot.borrow_mut().take()); - Self { - outer, - _not_send: PhantomData, - } - } - - /// Whether any scope is active on the current thread. - #[must_use] - pub fn is_active() -> bool { - THUNK_DEPTH.with(Cell::get) > 0 - } -} - -impl Drop for ThunkScope { - fn drop(&mut self) { - THUNK_DEPTH.with(|d| d.set(d.get().saturating_sub(1))); - let outer = self.outer.take(); - PENDING_FOREIGN.with(|slot| *slot.borrow_mut() = outer); - } -} - -/// Read the `out_err` slot a vtable entry wrote: `Ok(())` when the consumer -/// succeeded, otherwise the consumer's failure (whose allocations `err`'s -/// drop then releases). -/// -/// A positive code is reported as -/// [`FOREIGN_ERROR_CODE`](crate::FOREIGN_ERROR_CODE), since it must not -/// masquerade as one of the producer's domain errors on the outer call; the -/// reserved negative codes are kept. -/// -/// # Errors -/// -/// Returns the consumer's failure when `err.code` isn't `0`. -pub fn foreign_status(err: &FfiError) -> Result<(), ForeignError> { - if err.code == 0 { - return Ok(()); - } - let code = if err.code < 0 { - err.code - } else { - crate::FOREIGN_ERROR_CODE - }; - // SAFETY: the consumer fills `out_err` only through `{prefix}_error_set`, - // so a non-null message is a NUL-terminated string this runtime owns. - let message = unsafe { err.message_str() } - .unwrap_or("callback interface implementation failed") - .to_string(); - Err(ForeignError { code, message }) -} - -/// Deliver a consumer-side failure from a callback method that can't return -/// it (one declared to return a plain `T`). -/// -/// On a `panic = "unwind"` build this never returns: it unwinds with `err` as -/// the payload via [`std::panic::resume_unwind`], so the panic hook doesn't -/// fire (this is control flow, not a bug report). On a `panic = "abort"` -/// build it calls [`defer_foreign_error`] and returns. Generated code always -/// follows a call to this function with a fallback value so that both builds -/// type-check. -/// -/// # Panics -/// -/// Unwinds with a [`ForeignError`] payload on a `panic = "unwind"` build. -pub fn raise_foreign_error(err: ForeignError) { - #[cfg(panic = "unwind")] - { - std::panic::resume_unwind(Box::new(err)); - } - #[cfg(not(panic = "unwind"))] - { - defer_foreign_error(err); - } -} - -/// Record `err` for the innermost [`ThunkScope`] on this thread without -/// unwinding, keeping an already-recorded failure if there is one. With no -/// scope active the failure is written to stderr and dropped. -/// -/// This is the `panic = "abort"` half of [`raise_foreign_error`]; it's public -/// so tests and unusual producers can exercise the deferred route on any -/// build. -pub fn defer_foreign_error(err: ForeignError) { - if !ThunkScope::is_active() { - eprintln!( - "weaveffi: a callback interface implementation failed outside any exported call \ - (code {}): {}", - err.code, err.message - ); - return; - } - PENDING_FOREIGN.with(|slot| { - let mut slot = slot.borrow_mut(); - if slot.is_none() { - *slot = Some(err); - } - }); -} - -/// Take the failure recorded for the innermost [`ThunkScope`] on this -/// thread, if any. -/// -/// Generated thunks call this right after the producer's code returns and, -/// when it yields `Some`, report the failure through `out_err` instead of -/// the result. On a `panic = "unwind"` build it's normally `None`, because a -/// failure unwinds past the producer's code instead of being recorded. -#[must_use] -pub fn take_foreign_error() -> Option { - PENDING_FOREIGN.with(|slot| slot.borrow_mut().take()) -} - -#[cfg(test)] -mod tests { - use super::*; - use std::sync::atomic::{AtomicUsize, Ordering}; - - #[repr(C)] - struct TestVtable { - ping: unsafe extern "C" fn(*mut c_void, i32, *mut FfiError) -> i32, - free: unsafe extern "C" fn(*mut c_void), - } - - impl Vtable for TestVtable { - fn free(&self) -> unsafe extern "C" fn(*mut c_void) { - self.free - } - } - - unsafe extern "C" fn ping(_ctx: *mut c_void, x: i32, out_err: *mut FfiError) -> i32 { - if x < 0 { - unsafe { crate::error_set(out_err, crate::FOREIGN_ERROR_CODE, "negative") }; - return 0; - } - x * 2 - } - - /// Each test owns its counter: the context points at it, so tests running - /// in parallel never observe each other's releases. - unsafe extern "C" fn free(ctx: *mut c_void) { - unsafe { &*ctx.cast::() }.fetch_add(1, Ordering::SeqCst); - } - - static VTABLE: TestVtable = TestVtable { ping, free }; - - fn ctx(freed: &AtomicUsize) -> *mut c_void { - std::ptr::from_ref(freed).cast_mut().cast() - } - - fn call(cb: &ForeignCallback, x: i32) -> Result { - let mut err = FfiError::default(); - let out = unsafe { (cb.vtable().ping)(cb.ctx(), x, &mut err) }; - foreign_status(&err).map(|()| out) - } - - #[test] - fn calls_through_and_frees_once() { - let freed = AtomicUsize::new(0); - let cb = Arc::new(unsafe { ForeignCallback::from_raw(ctx(&freed), &VTABLE) }.unwrap()); - assert_eq!(call(&cb, 21), Ok(42)); - let second = Arc::clone(&cb); - drop(cb); - assert_eq!(freed.load(Ordering::SeqCst), 0); - drop(second); - assert_eq!(freed.load(Ordering::SeqCst), 1); - } - - #[test] - fn foreign_failure_is_returned() { - let freed = AtomicUsize::new(0); - let cb = unsafe { ForeignCallback::from_raw(ctx(&freed), &VTABLE) }.unwrap(); - let err = call(&cb, -1).unwrap_err(); - assert_eq!(err.code, crate::FOREIGN_ERROR_CODE); - assert_eq!(err.message, "negative"); - } - - #[test] - fn raised_failure_unwinds_with_the_payload() { - let payload = std::panic::catch_unwind(|| { - raise_foreign_error(ForeignError { - code: crate::FOREIGN_ERROR_CODE, - message: "raised".into(), - }); - }) - .unwrap_err(); - let fe = payload.downcast_ref::().unwrap(); - assert_eq!(fe.message, "raised"); - } - - #[test] - fn positive_codes_are_reported_as_foreign() { - let err = crate::FfiError::new(7, "domain-looking"); - assert_eq!( - foreign_status(&err).unwrap_err().code, - crate::FOREIGN_ERROR_CODE - ); - let err = crate::FfiError::new(crate::MARSHAL_ERROR_CODE, "kept"); - assert_eq!( - foreign_status(&err).unwrap_err().code, - crate::MARSHAL_ERROR_CODE - ); - } - - trait Pinger: Send + Sync { - fn ping(&self, x: i32) -> Result; - } - - struct ForeignPinger(ForeignCallback); - - impl Pinger for ForeignPinger { - fn ping(&self, x: i32) -> Result { - call(&self.0, x) - } - } - - impl CallbackInterface for dyn Pinger { - type Vtable = TestVtable; - fn from_foreign(cb: ForeignCallback) -> Arc { - Arc::new(ForeignPinger(cb)) - } - } - - #[test] - fn lift_callback_builds_the_trait_object() { - let freed = AtomicUsize::new(0); - let pinger: Arc = - unsafe { lift_callback(ctx(&freed), &VTABLE) }.expect("non-null vtable"); - assert_eq!(pinger.ping(4), Ok(8)); - drop(pinger); - assert_eq!(freed.load(Ordering::SeqCst), 1); - assert!( - unsafe { lift_callback::(std::ptr::null_mut(), std::ptr::null()) } - .is_none() - ); - } - - #[test] - fn deferred_failures_keep_the_first_and_are_taken_once() { - let _scope = ThunkScope::enter(); - assert!(take_foreign_error().is_none()); - defer_foreign_error(ForeignError { - code: crate::FOREIGN_ERROR_CODE, - message: "first".into(), - }); - defer_foreign_error(ForeignError { - code: crate::MARSHAL_ERROR_CODE, - message: "second".into(), - }); - let taken = take_foreign_error().expect("a pending failure"); - assert_eq!(taken.message, "first"); - assert!(take_foreign_error().is_none()); - } - - #[test] - fn deferral_without_a_scope_is_dropped() { - assert!(!ThunkScope::is_active()); - defer_foreign_error(ForeignError { - code: crate::FOREIGN_ERROR_CODE, - message: "nobody is listening".into(), - }); - let _scope = ThunkScope::enter(); - assert!(take_foreign_error().is_none()); - } - - #[test] - fn nested_scopes_keep_their_own_failures() { - let outer = ThunkScope::enter(); - defer_foreign_error(ForeignError { - code: crate::FOREIGN_ERROR_CODE, - message: "outer".into(), - }); - { - let _inner = ThunkScope::enter(); - assert!(take_foreign_error().is_none()); - defer_foreign_error(ForeignError { - code: crate::FOREIGN_ERROR_CODE, - message: "inner".into(), - }); - } - assert_eq!( - take_foreign_error().map(|e| e.message), - Some("outer".into()) - ); - drop(outer); - assert!(!ThunkScope::is_active()); - } - - #[test] - fn deferred_failures_are_thread_local() { - let _scope = ThunkScope::enter(); - defer_foreign_error(ForeignError { - code: crate::FOREIGN_ERROR_CODE, - message: "mine".into(), - }); - let other = std::thread::spawn(take_foreign_error).join().unwrap(); - assert!(other.is_none()); - assert_eq!(take_foreign_error().map(|e| e.message), Some("mine".into())); - } - - #[test] - fn null_vtable_is_rejected() { - assert!(unsafe { - ForeignCallback::::from_raw(std::ptr::null_mut(), std::ptr::null()) - } - .is_none()); - } -} diff --git a/crates/weaveffi-abi/src/lib.rs b/crates/weaveffi-abi/src/lib.rs deleted file mode 100644 index 53719bae..00000000 --- a/crates/weaveffi-abi/src/lib.rs +++ /dev/null @@ -1,117 +0,0 @@ -//! C ABI runtime: the error struct, memory helpers, reference-counted -//! objects, cancel tokens, callback-interface vtables, iterators, the -//! value-buffer codec, and the async executor hook. -//! -//! Producers don't use this crate directly. The `#[weaveffi::module]` -//! expansion and `weaveffi::export_runtime!()` call into it (re-exported as -//! `weaveffi::abi`), so every `unsafe` pointer operation a producer performs -//! has one audited home. The normative description of the contract it -//! implements is `docs/src/reference/abi.md` in the WeaveFFI repository. -#![deny(missing_docs)] -#![warn(clippy::missing_errors_doc)] -#![warn(clippy::missing_panics_doc)] -#![warn(clippy::missing_safety_doc)] -#![warn(clippy::doc_markdown)] -#![allow(unsafe_code)] - -pub mod buffer; -pub mod callback; -pub mod cancel; -pub mod convert; -pub mod error; -pub mod iter; -pub mod leak; -pub mod object; -pub mod spawn; - -pub use buffer::{ - decode_value, encode_value, BufferDecodeError, BufferReader, BufferValue, BufferWriter, - ByValue, FixedWidth, -}; -pub use callback::{ - defer_foreign_error, foreign_status, lift_callback, raise_foreign_error, take_foreign_error, - CallbackInterface, ForeignCallback, ForeignError, ThunkScope, Vtable, -}; -pub use cancel::{ - cancel_token_cancel, cancel_token_create, cancel_token_destroy, cancel_token_is_cancelled, - CancelToken, Cancellable, FfiCancelToken, -}; -pub use convert::{ - bytes_into_raw, free_bytes, lift_byte_slice, lift_bytes, lift_str, lift_string, lower_bytes, - lower_string, -}; -pub use error::{ - boxed_error, error_clear, error_free, error_set, error_set_c, error_store, panic_message, - ErrorReport, FfiError, CANCELLED_ERROR_CODE, FOREIGN_ERROR_CODE, GENERIC_ERROR_CODE, - MARSHAL_ERROR_CODE, PANIC_ERROR_CODE, -}; -pub use iter::{iter_destroy, iter_into_raw, iter_next, Iter, IterHandle}; -pub use leak::debug_live; -pub use object::{ - lower_object, lower_object_opt, object_arc, object_clone, object_destroy, object_from_token, - object_ref, object_to_token, -}; -pub use spawn::{ - block_on, run_async, set_spawner, spawn, BoxFuture, CatchUnwind, Spawner, SpawnerAlreadySet, -}; - -/// The revision of the WeaveFFI C ABI this runtime implements. -/// -/// Every producer exports it as `{prefix}_abi_version()` (via -/// `weaveffi::export_runtime!()`), and every generated consumer compares it -/// with the revision it was generated against when it loads the library, -/// turning a silent memory-layout mismatch into a clear error. -/// -/// The number only changes when the runtime surface (the error layout, the -/// value-buffer encoding, the object or callback-interface conventions, or -/// the set and signatures of the runtime symbols) changes incompatibly. It's -/// independent of the crate version and of the IDL schema version. -/// -/// Revision 3 prefixed every runtime symbol with the library's own C prefix, -/// passed strings as `(ptr, len)` UTF-8 runs freed with `{prefix}_free_bytes` -/// (removing `free_string`), made cancel tokens reference counted, and added -/// [`CANCELLED_ERROR_CODE`] and the per-module contract checksums. -pub const ABI_VERSION: u32 = 3; - -/// Fixed alignment used for every Wasm linear-memory allocation handed to JS. -/// -/// 8 bytes over-aligns scalar and byte buffers but is required for the -/// `{i32 ptr, i32 len}` and wider return slots that JS reads back through -/// `DataView`. -#[cfg(target_arch = "wasm32")] -const WASM_ALLOC_ALIGN: usize = 8; - -/// The body of `{prefix}_alloc` (`wasm32` only): allocate `size` bytes in this -/// module's linear memory. -/// -/// The Wasm backend has no host-provided allocator, so the generated JS glue -/// stages input strings and buffers, and reserves return slots, through this. -/// The caller releases the block with [`wasm_dealloc`] using the same `size`. -#[cfg(target_arch = "wasm32")] -#[must_use] -pub fn wasm_alloc(size: usize) -> *mut u8 { - let size = size.max(1); - match std::alloc::Layout::from_size_align(size, WASM_ALLOC_ALIGN) { - // SAFETY: the layout has a non-zero size. - Ok(layout) => unsafe { std::alloc::alloc(layout) }, - Err(_) => std::ptr::null_mut(), - } -} - -/// The body of `{prefix}_dealloc` (`wasm32` only): release a block from -/// [`wasm_alloc`]. -/// -/// # Safety -/// -/// `ptr` must be null or a block [`wasm_alloc`] returned for this same -/// `size`, released exactly once. -#[cfg(target_arch = "wasm32")] -pub unsafe fn wasm_dealloc(ptr: *mut u8, size: usize) { - if ptr.is_null() { - return; - } - if let Ok(layout) = std::alloc::Layout::from_size_align(size.max(1), WASM_ALLOC_ALIGN) { - // SAFETY: `ptr` came from `wasm_alloc` with this exact layout. - unsafe { std::alloc::dealloc(ptr, layout) }; - } -} diff --git a/crates/weaveffi-abi/src/spawn.rs b/crates/weaveffi-abi/src/spawn.rs deleted file mode 100644 index 64a86642..00000000 --- a/crates/weaveffi-abi/src/spawn.rs +++ /dev/null @@ -1,389 +0,0 @@ -//! The pluggable executor async exports run on, and the launcher driver that -//! keeps the completion promise. -//! -//! An exported `async fn` lowers to a launcher that returns immediately and a -//! completion callback that fires when the future resolves. Something has to -//! drive the future in between. By default WeaveFFI drives each one on a -//! dedicated thread with [`block_on`], which needs no runtime -//! and is enough for CPU-bound work and futures woken from other threads. A -//! producer whose futures depend on a reactor (Tokio's I/O and timers, for -//! example) installs its own [`Spawner`] once at startup with [`set_spawner`]: -//! -//! ``` -//! # fn spawn_on_my_runtime(_f: weaveffi_abi::BoxFuture) {} -//! // e.g. `|fut| { tokio_handle.spawn(fut); }` -//! let _ = weaveffi_abi::set_spawner(|fut| spawn_on_my_runtime(fut)); -//! ``` -//! -//! Generated launchers call [`run_async`], which routes to the installed -//! spawner or the default through [`spawn`]. Every future handed to a spawner -//! is `Send + 'static` and already wrapped so a panic inside it is caught and -//! reported through the completion callback; a spawner never observes an -//! unwinding future. A spawner may also drop a future without finishing it -//! (a runtime shutting down, say): the completion still fires, with -//! [`CANCELLED_ERROR_CODE`](crate::CANCELLED_ERROR_CODE). - -use std::future::Future; -use std::marker::PhantomData; -use std::pin::Pin; -use std::sync::{Arc, OnceLock}; -use std::task::{Context, Poll, Wake, Waker}; - -use crate::{CancelToken, Cancellable, FfiError, ForeignError, ThunkScope}; - -/// The type-erased future a [`Spawner`] receives. -pub type BoxFuture = Pin + Send + 'static>>; - -/// An executor hook that drives the futures produced by async exports. -/// -/// Implemented for any `Fn(BoxFuture) + Send + Sync + 'static`, so a closure -/// that forwards to a runtime handle is enough. -pub trait Spawner: Send + Sync + 'static { - /// Schedule `fut` to run to completion. Must not block the caller: the - /// launcher that invokes it is on the consumer's thread. - fn spawn(&self, fut: BoxFuture); -} - -impl Spawner for F -where - F: Fn(BoxFuture) + Send + Sync + 'static, -{ - fn spawn(&self, fut: BoxFuture) { - self(fut); - } -} - -static SPAWNER: OnceLock> = OnceLock::new(); - -/// The error returned when [`set_spawner`] is called a second time. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct SpawnerAlreadySet; - -impl std::fmt::Display for SpawnerAlreadySet { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str("a WeaveFFI spawner has already been installed") - } -} - -impl std::error::Error for SpawnerAlreadySet {} - -/// Install the process-wide spawner async exports run on. -/// -/// Call once, before the first async export is launched (an initialization -/// export or a library constructor is the natural place). Until it's called, -/// and forever after if it never is, the default thread-per-future spawner -/// is used. -/// -/// # Errors -/// -/// Returns [`SpawnerAlreadySet`] if a spawner was installed already; the -/// first one wins. -pub fn set_spawner(spawner: impl Spawner) -> Result<(), SpawnerAlreadySet> { - SPAWNER - .set(Box::new(spawner)) - .map_err(|_| SpawnerAlreadySet) -} - -/// Run `fut` on the installed spawner, or on the default one. -/// -/// The default drives the future on a freshly spawned thread with -/// [`block_on`]. On `wasm32`, which has no threads, the -/// future is driven inline before this returns. -pub fn spawn(fut: impl Future + Send + 'static) { - let fut: BoxFuture = Box::pin(fut); - match SPAWNER.get() { - Some(spawner) => spawner.spawn(fut), - None => default_spawn(fut), - } -} - -#[cfg(not(target_arch = "wasm32"))] -fn default_spawn(fut: BoxFuture) { - std::thread::spawn(move || block_on(fut)); -} - -#[cfg(target_arch = "wasm32")] -fn default_spawn(fut: BoxFuture) { - block_on(fut); -} - -/// Drive a future to completion on the current thread, blocking until it -/// resolves. -/// -/// This is the minimal, dependency-free executor behind the default -/// [`Spawner`]. It parks the thread between polls and wakes on -/// `Waker::wake`, so a future that yields (for example, one awaiting a -/// channel woken from another thread) makes progress without busy-spinning. -/// There is no reactor, so a future that depends on an external runtime's -/// I/O driver (such as Tokio's) needs that runtime installed with -/// [`set_spawner`] instead. -/// -/// # Examples -/// -/// ``` -/// let n = weaveffi_abi::block_on(async { 1 + 2 }); -/// assert_eq!(n, 3); -/// ``` -pub fn block_on(fut: F) -> F::Output { - struct ThreadWaker(std::thread::Thread); - impl Wake for ThreadWaker { - fn wake(self: Arc) { - self.0.unpark(); - } - fn wake_by_ref(self: &Arc) { - self.0.unpark(); - } - } - - let mut fut = std::pin::pin!(fut); - let waker = Waker::from(Arc::new(ThreadWaker(std::thread::current()))); - let mut cx = Context::from_waker(&waker); - loop { - match fut.as_mut().poll(&mut cx) { - Poll::Ready(out) => return out, - Poll::Pending => std::thread::park(), - } - } -} - -/// Fires an async call's completion exactly once: through -/// [`complete`](Self::complete) on the normal path, or with a cancellation -/// error from `Drop` if the spawner drops the task first. -struct CompletionGuard)> { - complete: Option, - _result: PhantomData, -} - -impl)> CompletionGuard { - fn complete(mut self, result: Result) { - if let Some(c) = self.complete.take() { - c(result); - } - } -} - -impl)> Drop for CompletionGuard { - fn drop(&mut self) { - if let Some(c) = self.complete.take() { - c(Err(FfiError::cancelled())); - } - } -} - -/// Run an async export to completion and deliver its result exactly once. -/// -/// Every generated async launcher calls this. `fut` is the producer's work, -/// already mapped to `Result` (a domain error becomes its -/// [`FfiError`]); `complete` lowers the outcome into the C completion -/// callback's slots and invokes it. The runtime calls `complete` exactly -/// once: -/// -/// * with `fut`'s result when it resolves; -/// * with a [`PANIC_ERROR_CODE`](crate::PANIC_ERROR_CODE) error (or the -/// consumer's [`ForeignError`]) when polling it panics; -/// * with [`CANCELLED_ERROR_CODE`](crate::CANCELLED_ERROR_CODE) as soon as -/// `cancel` fires, after dropping `fut`; -/// * with [`CANCELLED_ERROR_CODE`](crate::CANCELLED_ERROR_CODE) when the -/// spawner drops the task before it resolves (or panics instead of -/// accepting it). -pub fn run_async(fut: Fut, cancel: Option, complete: C) -where - T: 'static, - Fut: Future> + Send + 'static, - C: FnOnce(Result) + Send + 'static, -{ - let guard = CompletionGuard { - complete: Some(complete), - _result: PhantomData, - }; - let task = async move { - let guard = guard; - let caught = CatchUnwind::new(fut); - let outcome = match cancel { - Some(token) => Cancellable::new(caught, token).await, - None => Some(caught.await), - }; - guard.complete(match outcome { - None => Err(FfiError::cancelled()), - Some(Ok(result)) => result, - Some(Err(payload)) => Err(FfiError::from_panic(&*payload)), - }); - }; - // A spawner that panics drops the task on the way out, which fires the - // completion with the cancelled code; the panic itself must not unwind - // into the launcher's C caller. - let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| spawn(task))); -} - -/// A future adapter that catches a panic from the inner future's `poll` and -/// resolves to `Err(payload)` instead of unwinding into the executor. -/// -/// This is how generated async launchers keep the "callback fires exactly -/// once" promise even when the producer's future panics: the launcher awaits -/// `CatchUnwind::new(user_future)` and reports an `Err` through the -/// completion callback with the reserved panic code. -/// -/// It also closes the deferred foreign-error route for async producers: each -/// poll runs inside a [`ThunkScope`], and a failure recorded by -/// [`defer_foreign_error`](crate::defer_foreign_error) during any poll (a -/// `panic = "abort"` build) is kept with this future, so the adapter resolves -/// to `Err` carrying that [`ForeignError`] instead of the producer's value. -pub struct CatchUnwind { - inner: Option>>, - foreign: Option, -} - -impl CatchUnwind { - /// Wrap `fut`. - pub fn new(fut: F) -> Self { - Self { - inner: Some(Box::pin(fut)), - foreign: None, - } - } -} - -impl Future for CatchUnwind { - type Output = Result>; - - fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll { - let this = &mut *self; - let Some(inner) = this.inner.as_mut() else { - // Polled after completion; a well-behaved executor never does - // this, and there is nothing left to drive. - return Poll::Pending; - }; - let polled = { - let _scope = ThunkScope::enter(); - let polled = - std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| inner.as_mut().poll(cx))); - if let Some(foreign) = crate::take_foreign_error() { - this.foreign.get_or_insert(foreign); - } - polled - }; - match polled { - Ok(Poll::Pending) => Poll::Pending, - Ok(Poll::Ready(out)) => { - this.inner = None; - match this.foreign.take() { - Some(foreign) => Poll::Ready(Err(Box::new(foreign))), - None => Poll::Ready(Ok(out)), - } - } - Err(payload) => { - this.inner = None; - Poll::Ready(Err(payload)) - } - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - use std::sync::mpsc; - - #[test] - fn default_spawner_runs_the_future() { - let (tx, rx) = mpsc::channel(); - spawn(async move { - tx.send(7).unwrap(); - }); - assert_eq!(rx.recv_timeout(std::time::Duration::from_secs(5)), Ok(7)); - } - - #[test] - fn catch_unwind_reports_panics_and_values() { - let ok = block_on(CatchUnwind::new(async { 5 })); - assert_eq!(ok.unwrap(), 5); - let err = block_on(CatchUnwind::new(async { - if true { - panic!("boom"); - } - 1 - })); - let payload = err.unwrap_err(); - assert_eq!(crate::panic_message(&*payload), "boom"); - } - - #[test] - fn catch_unwind_surfaces_a_deferred_foreign_error() { - let err = block_on(CatchUnwind::new(async { - crate::defer_foreign_error(crate::ForeignError { - code: crate::FOREIGN_ERROR_CODE, - message: "consumer failed".into(), - }); - 9 - })); - let payload = err.unwrap_err(); - let foreign = payload - .downcast_ref::() - .expect("ForeignError payload"); - assert_eq!(foreign.code, crate::FOREIGN_ERROR_CODE); - assert_eq!(foreign.message, "consumer failed"); - } - - fn run( - fut: impl Future> + Send + 'static, - cancel: Option, - ) -> mpsc::Receiver> { - let (tx, rx) = mpsc::channel(); - run_async(fut, cancel, move |r: Result| { - tx.send(r.map_err(|e| e.code)).unwrap(); - }); - rx - } - - const WAIT: std::time::Duration = std::time::Duration::from_secs(5); - - #[test] - fn run_async_delivers_values_errors_and_panics() { - assert_eq!(run(async { Ok(4) }, None).recv_timeout(WAIT), Ok(Ok(4))); - let rx = run(async { Err(FfiError::new(9, "domain")) }, None); - assert_eq!(rx.recv_timeout(WAIT), Ok(Err(9))); - let rx = run( - async { - if true { - panic!("bug"); - } - Ok(0) - }, - None, - ); - assert_eq!(rx.recv_timeout(WAIT), Ok(Err(crate::PANIC_ERROR_CODE))); - } - - #[test] - fn run_async_completes_cancelled_when_the_token_fires() { - let raw = crate::cancel_token_create(); - let token = unsafe { CancelToken::from_raw(raw) }; - let rx = run( - async { - std::future::pending::<()>().await; - Ok(1) - }, - Some(token), - ); - unsafe { crate::cancel_token_cancel(raw) }; - unsafe { crate::cancel_token_destroy(raw) }; - assert_eq!(rx.recv_timeout(WAIT), Ok(Err(crate::CANCELLED_ERROR_CODE))); - assert!(rx - .recv_timeout(std::time::Duration::from_millis(50)) - .is_err()); - } - - #[test] - fn a_dropped_guard_completes_cancelled_once() { - let (tx, rx) = mpsc::channel(); - let guard = CompletionGuard { - complete: Some(move |r: Result<(), FfiError>| { - tx.send(r.map_err(|e| e.code)).unwrap(); - }), - _result: PhantomData, - }; - drop(guard); - assert_eq!(rx.recv(), Ok(Err(crate::CANCELLED_ERROR_CODE))); - assert!(rx.recv().is_err()); - } -} diff --git a/crates/weaveffi-cli/Cargo.toml b/crates/weaveffi-cli/Cargo.toml index e3365836..609510f0 100644 --- a/crates/weaveffi-cli/Cargo.toml +++ b/crates/weaveffi-cli/Cargo.toml @@ -6,7 +6,7 @@ rust-version.workspace = true license.workspace = true authors.workspace = true repository.workspace = true -description = "Command-line tool for generating multi-language FFI bindings from API definitions" +description = "The WeaveFFI command-line tool and code-generation library: the orchestrator and the eleven language generators" keywords = ["ffi", "codegen", "bindings", "cli", "code-generation"] categories = ["development-tools::ffi", "command-line-utilities"] exclude = [ @@ -21,6 +21,10 @@ path = "src/main.rs" # both would collide in `target/doc`. doc = false +[[bench]] +name = "codegen_bench" +harness = false + [package.metadata.binstall] pkg-url = "{ repo }/releases/download/v{ version }/weaveffi-{ target }{ archive-suffix }" bin-dir = "weaveffi-{ target }/{ bin }{ binary-ext }" @@ -38,12 +42,18 @@ camino = { workspace = true } serde = { workspace = true } toml = { workspace = true } weaveffi-model = { workspace = true, features = ["idl"] } -weaveffi-gen = { workspace = true } +rayon = { workspace = true } +heck = { workspace = true } serde_yaml = { workspace = true } serde_json = { workspace = true } similar = { workspace = true } tempfile = { workspace = true } schemars = { workspace = true } +flate2 = "1" +crc32fast = "1" +sha2 = "0.10" +tar = { version = "0.4", default-features = false } +object = { version = "0.37", default-features = false, features = ["read", "wasm", "std"] } [dev-dependencies] assert_cmd = "2" @@ -52,7 +62,10 @@ serde_yaml = { workspace = true } serde_json = { workspace = true } toml = { workspace = true } insta = { workspace = true } - +criterion = { workspace = true } +# Only for the ABI-revision lockstep test. Path-only, so `cargo publish` +# strips it and this crate never has to wait for `weaveffi` on crates.io. +weaveffi = { path = "../weaveffi" } [lints] workspace = true diff --git a/crates/weaveffi-gen/benches/codegen_bench.rs b/crates/weaveffi-cli/benches/codegen_bench.rs similarity index 77% rename from crates/weaveffi-gen/benches/codegen_bench.rs rename to crates/weaveffi-cli/benches/codegen_bench.rs index 54184c58..0be03a27 100644 --- a/crates/weaveffi-gen/benches/codegen_bench.rs +++ b/crates/weaveffi-cli/benches/codegen_bench.rs @@ -1,14 +1,24 @@ use std::path::Path; +use camino::Utf8Path; use criterion::{black_box, criterion_group, criterion_main, Criterion}; use weaveffi_model::ir::{ Api, EnumDef, EnumVariant, Function, Module, Param, StructDef, StructField, TypeRef, }; +use weaveffi_model::model::Model; use weaveffi_model::parse::parse_api_str; +use weaveffi_model::pkg::Identity; +use weaveffi_model::ty::Prim; +use weaveffi_model::validate::validate; + +/// Validate `api` into its model under a fixed identity. +fn model(api: &Api) -> Model { + validate(api, &Identity::named("bench"), None).unwrap() +} fn calculator_api() -> Api { Api { - version: "0.10.0".to_string(), + version: "0.11.0".to_string(), modules: vec![Module { name: "calculator".to_string(), doc: None, @@ -19,16 +29,16 @@ fn calculator_api() -> Api { params: vec![ Param { name: "a".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, Param { name: "b".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, ], - returns: Some(TypeRef::I32), + returns: Some(TypeRef::Prim(Prim::I32)), r#async: false, cancellable: false, throws: false, @@ -40,16 +50,16 @@ fn calculator_api() -> Api { params: vec![ Param { name: "a".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, Param { name: "b".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, ], - returns: Some(TypeRef::I32), + returns: Some(TypeRef::Prim(Prim::I32)), r#async: false, cancellable: false, throws: false, @@ -61,16 +71,16 @@ fn calculator_api() -> Api { params: vec![ Param { name: "a".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, Param { name: "b".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, ], - returns: Some(TypeRef::I32), + returns: Some(TypeRef::Prim(Prim::I32)), r#async: false, cancellable: false, throws: false, @@ -81,10 +91,10 @@ fn calculator_api() -> Api { doc: Some("Echo a string back".to_string()), params: vec![Param { name: "s".to_string(), - ty: TypeRef::StringUtf8, + ty: TypeRef::Prim(Prim::String), doc: None, }], - returns: Some(TypeRef::StringUtf8), + returns: Some(TypeRef::Prim(Prim::String)), r#async: false, cancellable: false, throws: false, @@ -115,17 +125,17 @@ fn large_api() -> Api { fields: vec![ StructField { name: "id".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, StructField { name: "name".to_string(), - ty: TypeRef::StringUtf8, + ty: TypeRef::Prim(Prim::String), doc: None, }, StructField { name: "active".to_string(), - ty: TypeRef::Bool, + ty: TypeRef::Prim(Prim::Bool), doc: None, }, ], @@ -162,17 +172,17 @@ fn large_api() -> Api { let functions: Vec = (0..50) .map(|f| Function { - name: format!("func{f}"), + name: format!("m{m}_func{f}"), doc: Some(format!("Function {f} in module {m}")), params: vec![ Param { name: "a".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, Param { name: "b".to_string(), - ty: TypeRef::StringUtf8, + ty: TypeRef::Prim(Prim::String), doc: None, }, Param { @@ -206,7 +216,7 @@ fn large_api() -> Api { .collect(); Api { - version: "0.10.0".to_string(), + version: "0.11.0".to_string(), modules, } } @@ -215,7 +225,7 @@ fn bench_validate_small_api(c: &mut Criterion) { let api = calculator_api(); c.bench_function("validate_small_api", |b| { b.iter(|| { - weaveffi_model::validate::validate_api(black_box(api.clone()), None).unwrap(); + model(black_box(&api)); }); }); } @@ -224,48 +234,41 @@ fn bench_validate_large_api(c: &mut Criterion) { let api = large_api(); c.bench_function("validate_large_api", |b| { b.iter(|| { - weaveffi_model::validate::validate_api(black_box(api.clone()), None).unwrap(); - }); - }); -} - -fn bench_hash_api(c: &mut Criterion) { - let api = weaveffi_model::validate::validate_api(large_api(), None).unwrap(); - c.bench_function("hash_large_api", |b| { - b.iter(|| { - black_box(weaveffi_gen::cache::hash_api(black_box(api.api()))); + model(black_box(&api)); }); }); } /// Read the kitchen-sink fixture without validating it, so the validate bench -/// measures a complete pre-resolved → validated pass on every iteration. +/// measures a complete parsed-to-model pass on every iteration. fn load_kitchen_sink_unvalidated() -> Api { - let path = Path::new(env!("CARGO_MANIFEST_DIR")) - .join("../weaveffi-cli/tests/fixtures/kitchen_sink.yml"); + let path = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/kitchen_sink.yml"); let contents = std::fs::read_to_string(&path) .unwrap_or_else(|e| panic!("read fixture {}: {e}", path.display())); parse_api_str(&contents, "yaml") .unwrap_or_else(|e| panic!("parse fixture {}: {e}", path.display())) } -/// Target: validate_api < 5ms for the kitchen-sink fixture. +/// Target: `validate` < 5ms for the kitchen-sink fixture. fn bench_validate_kitchen_sink(c: &mut Criterion) { let api = load_kitchen_sink_unvalidated(); c.bench_function("validate_kitchen_sink", |b| { b.iter(|| { - weaveffi_model::validate::validate_api(black_box(api.clone()), None).unwrap(); + model(black_box(&api)); }); }); } -/// Target: hash_api < 1ms for the kitchen-sink fixture. -fn bench_hash_kitchen_sink(c: &mut Criterion) { - let api = - weaveffi_model::validate::validate_api(load_kitchen_sink_unvalidated(), None).unwrap(); - c.bench_function("hash_kitchen_sink", |b| { +/// Every target rendering the kitchen-sink fixture in memory (no I/O). +fn bench_render_kitchen_sink(c: &mut Criterion) { + let model = model(&load_kitchen_sink_unvalidated()); + let targets = weaveffi_cli::targets::all_default(); + let out_dir = Utf8Path::new("out"); + c.bench_function("render_kitchen_sink", |b| { b.iter(|| { - black_box(weaveffi_gen::cache::hash_api(black_box(api.api()))); + for t in &targets { + black_box(t.render(black_box(&model), out_dir)); + } }); }); } @@ -274,8 +277,7 @@ criterion_group!( benches, bench_validate_small_api, bench_validate_large_api, - bench_hash_api, bench_validate_kitchen_sink, - bench_hash_kitchen_sink, + bench_render_kitchen_sink, ); criterion_main!(benches); diff --git a/crates/weaveffi-cli/src/backend/mod.rs b/crates/weaveffi-cli/src/backend/mod.rs new file mode 100644 index 00000000..3f364a0b --- /dev/null +++ b/crates/weaveffi-cli/src/backend/mod.rs @@ -0,0 +1,186 @@ +//! The language-backend framework. +//! +//! [`LanguageBackend`] is the one trait a target implements: a stable name, +//! a pure [`files`](LanguageBackend::files) that renders the complete output +//! set from the [`Model`], and an optional +//! [`package`](LanguageBackend::package) for `weaveffi package`. +//! [`ConfiguredBackend`](crate::codegen::ConfiguredBackend) exposes the +//! backend to the orchestrator through the object-safe +//! [`Target`](crate::codegen::Target) trait. A backend owns *only* +//! language-specific rendering: type mapping, marshalling, file layout, and +//! the exact text of each declaration. Model construction, erasure, and every +//! write live elsewhere, once. +//! +//! [`Model`]: weaveffi_model::model::Model + +use camino::{Utf8Path, Utf8PathBuf}; + +use crate::package::{Artifact, PackageContext}; +use weaveffi_model::model::Model; + +/// A single generated file: its full path (under the output directory) and the +/// rendered contents. Backends return these from [`LanguageBackend::files`]; +/// the driver creates parent directories and writes them. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct OutputFile { + /// Full path to write, under (or anchored at) the output directory. + pub path: Utf8PathBuf, + /// The rendered file contents. + pub contents: String, +} + +impl OutputFile { + /// Pair a destination path with its rendered contents. + /// + /// The path is normalized to `/` separators, which every platform's file + /// APIs accept, so listings, cache records, and tests see the same path + /// on Windows as elsewhere. + pub fn new(path: impl Into, contents: impl Into) -> Self { + let path: Utf8PathBuf = path.into(); + let path = if path.as_str().contains('\\') { + Utf8PathBuf::from(path.as_str().replace('\\', "/")) + } else { + path + }; + Self { + path, + contents: contents.into(), + } + } +} + +/// An idiomatic language backend over the shared [`Model`]. +/// +/// The required methods are [`name`](Self::name) and [`files`](Self::files), +/// which assembles the complete output set; wrap the type in +/// [`ConfiguredBackend`](crate::codegen::ConfiguredBackend) to hand it to the +/// orchestrator. Rendering is pure: a backend returns [`OutputFile`]s and the +/// orchestrator does the I/O. Each backend walks the model in whatever order +/// its language needs (C and C++ order declarations by dependency, Swift +/// splits types from the namespaced module body, Kotlin, Node.js, and Wasm +/// render parallel files) and emits its own doc comments, sharing +/// [`emit_doc`](crate::codegen::common::emit_doc) for the common line and +/// block flavors. +pub trait LanguageBackend: Send + Sync { + /// Per-target, fully typed configuration (the `[generators.]` + /// table). + type Config: Default + Clone + Send + Sync; + + /// Stable short name (`"swift"`, `"python"`, ...): the `--target` token + /// and the generation-record file basename. + fn name(&self) -> &'static str; + + /// Assemble the complete output set from the validated `model`, which + /// carries the library's identity alongside every symbol and signature. + /// Most backends render a primary source file from `model.modules`, then + /// append package manifests (`package.json`, `pyproject.toml`, `go.mod`, + /// ...) as additional [`OutputFile`]s. + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec; + + /// Assemble the installable artifacts that bundle the per-platform builds + /// in `ctx.binaries`, returning `None` when this target has no packaging. + /// + /// This is the `weaveffi package` analogue of [`files`](Self::files): it + /// returns [`Artifact`]s (wheels, npm tarballs, gems, directory trees) + /// with paths relative to the dist directory, and the + /// [`write_artifact`](crate::package::write_artifact) driver does the + /// I/O. A target skips platforms its ecosystem has no slot for and + /// returns an empty list when none of `ctx.binaries` fits. The default + /// returns `None`. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let _ = (model, ctx, config); + None + } +} + +#[cfg(test)] +mod tests { + use super::*; + use weaveffi_model::ir::{Api, Function, Module, Param, TypeRef}; + use weaveffi_model::pkg::Identity; + use weaveffi_model::ty::Prim; + + #[derive(Default, Clone)] + struct FakeConfig; + + /// A trivial backend that lists each function's C symbol, so we can + /// assert the model carries the identity's prefix. + struct FakeBackend; + + impl LanguageBackend for FakeBackend { + type Config = FakeConfig; + + fn name(&self) -> &'static str { + "fake" + } + + fn files( + &self, + model: &Model, + out_dir: &Utf8Path, + _config: &Self::Config, + ) -> Vec { + let mut out = String::new(); + for m in &model.modules { + out.push_str(&format!("module {}\n", m.path)); + for f in &m.functions { + out.push_str(&format!("fn {} {}\n", f.name, f.c_base)); + } + } + vec![OutputFile::new(out_dir.join("fake/out.txt"), out)] + } + } + + fn model(name: &str) -> Model { + let api = Api { + version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), + modules: vec![Module { + name: "math".into(), + doc: None, + functions: vec![Function { + name: "add".into(), + params: vec![Param { + name: "x".into(), + ty: TypeRef::Prim(Prim::I32), + doc: None, + }], + returns: Some(TypeRef::Prim(Prim::I32)), + doc: None, + throws: false, + r#async: false, + cancellable: false, + deprecated: None, + }], + interfaces: vec![], + structs: vec![], + enums: vec![], + callback_interfaces: vec![], + errors: None, + modules: vec![], + }], + }; + weaveffi_model::validate::validate(&api, &Identity::named(name), None).unwrap() + } + + #[test] + fn backends_render_with_the_identity_prefix() { + use crate::codegen::{ConfiguredBackend, Target}; + let out_dir = Utf8Path::new("out"); + let render = |n: &str| { + let files = ConfiguredBackend::new(FakeBackend, FakeConfig).render(&model(n), out_dir); + assert_eq!(files.len(), 1); + assert_eq!(files[0].path, out_dir.join("fake/out.txt")); + files[0].contents.clone() + }; + assert_eq!( + render("weaveffi"), + "module math\nfn add weaveffi_math_add\n" + ); + assert_eq!(render("acme"), "module math\nfn add acme_math_add\n"); + } +} diff --git a/crates/weaveffi-cli/src/build/glue.rs b/crates/weaveffi-cli/src/build/glue.rs new file mode 100644 index 00000000..c5cb4ef1 --- /dev/null +++ b/crates/weaveffi-cli/src/build/glue.rs @@ -0,0 +1,303 @@ +//! Prebuilding the per-language C glue, so consumers never compile. +//! +//! The Node.js and Kotlin targets reach the C ABI through a small C library +//! of their own: an N-API addon and a JNI shim. Both are generated as C +//! source (so a producer written in any language works), and `weaveffi +//! build` compiles them per platform next to the producer library, linked +//! against it with a relative run path, so packages ship them prebuilt. + +use std::process::Command; + +use anyhow::{bail, Context, Result}; +use camino::{Utf8Path, Utf8PathBuf}; + +use crate::build::ndk::Ndk; +use crate::build::BuildSettings; +use crate::platform::{jni_shim_name, node_addon_name, Os, Platform}; + +/// One glue library to compile: its C source and the directory holding the +/// header it includes. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct GlueSource { + /// The C source file. + pub source: Utf8PathBuf, + /// The directory holding the producer's C header. + pub include_dir: Utf8PathBuf, +} + +/// Check that this host can compile desktop glue for `platform`: macOS +/// compiles both macOS architectures, Linux compiles its own architecture. +/// +/// # Errors +/// +/// Returns an error explaining which host can compile it instead. +pub fn check_desktop_host(platform: Platform) -> Result<()> { + let ok = match platform.os() { + Os::MacOs => cfg!(target_os = "macos"), + Os::Linux => cfg!(target_os = "linux") && Platform::host() == Some(platform), + _ => false, + }; + if !ok { + bail!( + "this host can't compile glue for {}; build it on a {} machine or CI runner", + platform.id(), + platform.display_name() + ); + } + Ok(()) +} + +/// The C compiler for desktop glue: `$CC`, else `cc`. +fn desktop_cc() -> String { + std::env::var("CC") + .ok() + .filter(|cc| !cc.is_empty()) + .unwrap_or_else(|| "cc".to_string()) +} + +/// Architecture and deployment-target flags for a macOS compile. +fn apple_flags(platform: Platform, settings: &BuildSettings) -> Vec { + let arch = match platform { + Platform::MacosArm64 => "arm64", + _ => "x86_64", + }; + vec![ + "-arch".into(), + arch.into(), + format!("-mmacosx-version-min={}", settings.macos_deployment_target), + ] +} + +/// The directory holding Node.js's C headers (`node_api.h`, `uv.h`): +/// `$npm_config_nodedir/include/node`, the running Node.js installation's +/// `include/node`, or node-gyp's header cache for its version. +/// +/// # Errors +/// +/// Returns an error explaining how to make the headers available when none +/// are found. +pub fn node_headers() -> Result { + let has_headers = + |dir: &Utf8Path| dir.join("node_api.h").is_file() && dir.join("uv.h").is_file(); + if let Ok(nodedir) = std::env::var("npm_config_nodedir") { + let dir = Utf8PathBuf::from(nodedir).join("include/node"); + if has_headers(&dir) { + return Ok(dir); + } + } + let node = |expr: &str| -> Option { + let out = Command::new("node").args(["-p", expr]).output().ok()?; + out.status + .success() + .then(|| String::from_utf8_lossy(&out.stdout).trim().to_string()) + }; + let Some(exec) = node("process.execPath") else { + bail!("Node.js isn't installed (no `node` on PATH), so its addon headers can't be found"); + }; + let mut candidates = Vec::new(); + if let Some(prefix) = Utf8Path::new(&exec).parent().and_then(Utf8Path::parent) { + candidates.push(prefix.join("include/node")); + } + if let Some(version) = node("process.versions.node") { + let home = std::env::var("HOME").unwrap_or_default(); + for cache in ["Library/Caches/node-gyp", ".cache/node-gyp"] { + candidates.push( + Utf8PathBuf::from(&home) + .join(cache) + .join(&version) + .join("include/node"), + ); + } + if let Ok(local) = std::env::var("LOCALAPPDATA") { + candidates.push( + Utf8PathBuf::from(local) + .join("node-gyp/Cache") + .join(&version) + .join("include/node"), + ); + } + } + candidates + .into_iter() + .find(|d| has_headers(d)) + .ok_or_else(|| { + anyhow::anyhow!( + "Node.js's C headers (node_api.h) weren't found next to {exec}; install them with \ + `npx node-gyp install` or point npm_config_nodedir at a Node.js source or headers \ + directory" + ) + }) +} + +/// Compile the Node.js addon `glue` for desktop `platform`, linked against +/// the producer library in `dir`, into `dir/{library}_node.node`. +/// +/// # Errors +/// +/// Returns an error when this host can't compile for `platform`, the +/// headers are missing, or the compiler fails. +pub fn compile_node_addon( + glue: &GlueSource, + platform: Platform, + dir: &Utf8Path, + library: &str, + node_include: &Utf8Path, + settings: &BuildSettings, +) -> Result { + if platform.os() == Os::Windows { + bail!( + "prebuilding the Node.js addon for windows-x64 isn't supported (it links node.lib); \ + the package falls back to node-gyp at install time" + ); + } + check_desktop_host(platform)?; + let output = dir.join(format!("{}.node", node_addon_name(library))); + let mut cmd = Command::new(desktop_cc()); + cmd.args(["-shared", "-fPIC", "-O2"]) + .arg(format!("-I{node_include}")) + .arg(format!("-I{}", glue.include_dir)) + .arg(glue.source.as_str()) + .arg(format!("-L{dir}")) + .arg(format!("-l{library}")) + .arg("-o") + .arg(output.as_str()); + if platform.os() == Os::MacOs { + cmd.args(apple_flags(platform, settings)) + .args(["-undefined", "dynamic_lookup", "-Wl,-rpath,@loader_path"]) + .arg(format!( + "-Wl,-install_name,@rpath/{}", + output.file_name().unwrap_or_default() + )); + } else { + cmd.arg("-Wl,-rpath,$ORIGIN"); + } + run(cmd, "the Node.js addon", platform)?; + Ok(output) +} + +/// The JDK include directories JNI code compiles against: +/// `$JAVA_HOME/include` and its OS subdirectory, finding the JDK through +/// `/usr/libexec/java_home` or `javac` on PATH when `JAVA_HOME` is unset. +/// +/// # Errors +/// +/// Returns an error when no JDK with `jni.h` is found. +pub fn jdk_includes() -> Result> { + let mut homes = Vec::new(); + if let Some(home) = std::env::var("JAVA_HOME").ok().filter(|h| !h.is_empty()) { + homes.push(Utf8PathBuf::from(home)); + } + if cfg!(target_os = "macos") { + if let Ok(out) = Command::new("/usr/libexec/java_home").output() { + if out.status.success() { + homes.push(Utf8PathBuf::from( + String::from_utf8_lossy(&out.stdout).trim(), + )); + } + } + } + if let Ok(out) = Command::new("which").arg("javac").output() { + let javac = Utf8PathBuf::from(String::from_utf8_lossy(&out.stdout).trim()); + if let Ok(real) = javac.canonicalize_utf8() { + if let Some(home) = real.parent().and_then(Utf8Path::parent) { + homes.push(home.to_path_buf()); + } + } + } + let os_dir = if cfg!(target_os = "macos") { + "darwin" + } else if cfg!(windows) { + "win32" + } else { + "linux" + }; + for home in homes { + let include = home.join("include"); + if include.join("jni.h").is_file() { + return Ok(vec![include.clone(), include.join(os_dir)]); + } + } + bail!("no JDK with jni.h was found; install a JDK and set JAVA_HOME") +} + +/// Compile the JNI shim `glue` for `platform` (a desktop JVM platform, or +/// an Android ABI through `ndk`), linked against the producer library in +/// `dir`, into `dir/lib{library}_jni.{so,dylib}`. +/// +/// # Errors +/// +/// Returns an error when this host can't compile for `platform`, the JDK or +/// NDK is missing, or the compiler fails. +pub fn compile_jni_shim( + glue: &GlueSource, + platform: Platform, + dir: &Utf8Path, + library: &str, + ndk: Option<&Ndk>, + settings: &BuildSettings, +) -> Result { + let shim = jni_shim_name(library); + let file = platform.lib_filename(&shim); + let output = dir.join(&file); + let mut cmd = match platform.os() { + Os::Android => { + let Some(ndk) = ndk else { + bail!( + "compiling the JNI shim for {} needs the Android NDK", + platform.id() + ); + }; + let mut cmd = Command::new(ndk.clang(platform, settings.android_api)?.as_str()); + cmd.arg(format!("-Wl,-soname,{file}")) + .arg("-Wl,-z,max-page-size=16384"); + cmd + } + Os::Windows => bail!( + "prebuilding the JNI shim for windows-x64 isn't supported; build it with the \ + generated CMake project on Windows" + ), + Os::MacOs | Os::Linux => { + check_desktop_host(platform)?; + let mut cmd = Command::new(desktop_cc()); + for include in jdk_includes()? { + cmd.arg(format!("-I{include}")); + } + if platform.os() == Os::MacOs { + cmd.args(apple_flags(platform, settings)) + .arg("-Wl,-rpath,@loader_path") + .arg(format!("-Wl,-install_name,@rpath/{file}")); + } else { + cmd.arg("-Wl,-rpath,$ORIGIN") + .arg(format!("-Wl,-soname,{file}")) + .arg("-lpthread"); + } + cmd + } + Os::Ios | Os::Wasm => bail!("{} has no JVM", platform.id()), + }; + cmd.args(["-shared", "-fPIC", "-O2"]) + .arg(format!("-I{}", glue.include_dir)) + .arg(glue.source.as_str()) + .arg(format!("-L{dir}")) + .arg(format!("-l{library}")) + .arg("-o") + .arg(output.as_str()); + run(cmd, "the JNI shim", platform)?; + Ok(output) +} + +/// Run a compiler command, capturing its output for the error message. +fn run(mut cmd: Command, what: &str, platform: Platform) -> Result<()> { + let program = cmd.get_program().to_string_lossy().into_owned(); + let out = cmd + .output() + .with_context(|| format!("failed to run `{program}` to compile {what}"))?; + if !out.status.success() { + bail!( + "compiling {what} for {} failed:\n{}", + platform.id(), + String::from_utf8_lossy(&out.stderr).trim_end() + ); + } + Ok(()) +} diff --git a/crates/weaveffi-cli/src/build/mod.rs b/crates/weaveffi-cli/src/build/mod.rs new file mode 100644 index 00000000..9a09554d --- /dev/null +++ b/crates/weaveffi-cli/src/build/mod.rs @@ -0,0 +1,280 @@ +//! Cross-compiling a Rust producer for each platform: `weaveffi build`. +//! +//! [`Builder`] runs `cargo rustc --lib` once per [`Platform`] with the +//! artifact kinds that platform's packages need (a `cdylib` for desktop and +//! Android, plus a `staticlib` for an `XCFramework` on Apple platforms, a +//! `staticlib` alone for iOS, and a `.wasm` module for `wasm32`), so a +//! producer's `Cargo.toml` needs no particular `crate-type`. It fixes what a +//! plain `cargo build` leaves machine-specific (a macOS library's install +//! name becomes `@rpath/lib{library}.dylib`, a Linux or Android library's +//! soname `lib{library}.so`), sets Apple deployment targets and the Android +//! NDK toolchain, and lays the results out in +//! `{target_dir}/weaveffi//`, where [`glue`] adds the prebuilt +//! Node.js addon and JNI shim and `weaveffi package` reads them back as a +//! [`BinarySet`](crate::platform::BinarySet). + +pub mod glue; +pub mod ndk; + +use std::process::{Command, Stdio}; +use std::sync::OnceLock; + +use anyhow::{bail, Context, Result}; +use camino::Utf8PathBuf; + +use crate::cargo::{cargo_bin, CargoCrate}; +use crate::platform::{NativeBinary, Os, Platform}; + +use self::ndk::Ndk; + +/// Build settings shared by every platform (the `[build]` table). +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct BuildSettings { + /// Build with the release profile (`--release`). + pub release: bool, + /// The minimum macOS version (`MACOSX_DEPLOYMENT_TARGET`). + pub macos_deployment_target: String, + /// The minimum iOS version (`IPHONEOS_DEPLOYMENT_TARGET`). + pub ios_deployment_target: String, + /// The minimum Android API level, which selects the NDK compiler. + pub android_api: u32, +} + +impl Default for BuildSettings { + /// Release builds for macOS 11.0, iOS 13.0, and Android API 21. + fn default() -> Self { + Self { + release: true, + macos_deployment_target: "11.0".into(), + ios_deployment_target: "13.0".into(), + android_api: 21, + } + } +} + +/// Cross-compiles one producer crate into `{target_dir}/weaveffi//`. +pub struct Builder<'a> { + krate: &'a CargoCrate, + library: &'a str, + settings: &'a BuildSettings, + quiet: bool, + installed_targets: OnceLock>>, + ndk: OnceLock>, +} + +impl<'a> Builder<'a> { + /// A builder for `krate` whose outputs are named after `library` (the + /// identity's library, which the generated bindings load). + pub fn new(krate: &'a CargoCrate, library: &'a str, settings: &'a BuildSettings) -> Self { + Self { + krate, + library, + settings, + quiet: false, + installed_targets: OnceLock::new(), + ndk: OnceLock::new(), + } + } + + /// Pass `--quiet` to Cargo. + #[must_use] + pub fn quiet(mut self, quiet: bool) -> Self { + self.quiet = quiet; + self + } + + /// The directory a platform's outputs are laid out in: + /// `{target_dir}/weaveffi/`. + #[must_use] + pub fn output_dir(&self, platform: Platform) -> Utf8PathBuf { + self.krate.weaveffi_dir().join(platform.id()) + } + + /// The Android NDK, located once. + /// + /// # Errors + /// + /// Returns an error when no NDK is found. + pub fn ndk(&self) -> Result<&Ndk> { + self.ndk + .get_or_init(|| Ndk::locate().map_err(|e| e.to_string())) + .as_ref() + .map_err(|e| anyhow::anyhow!("{e}")) + } + + /// Check, before compiling anything, that `platform` can be built here: + /// a suitable host, an installed rustup target, and (for Android) an + /// NDK. + /// + /// # Errors + /// + /// Returns an error naming the missing piece and how to add it. + pub fn check(&self, platform: Platform) -> Result<()> { + platform.check_host()?; + let triple = platform.rust_target(); + if let Some(installed) = self.installed_targets() { + if !installed.iter().any(|t| t == triple) { + bail!( + "the Rust target `{triple}` ({}) isn't installed; run `rustup target add \ + {triple}`", + platform.id() + ); + } + } + if platform.os() == Os::Android { + self.ndk()?.clang(platform, self.settings.android_api)?; + } + Ok(()) + } + + /// The rustup targets installed for the crate's toolchain, or `None` + /// when rustup isn't available (a distribution's Rust, for example). + fn installed_targets(&self) -> Option<&Vec> { + self.installed_targets + .get_or_init(|| { + let dir = self.krate.manifest_path.parent()?; + let out = Command::new("rustup") + .args(["target", "list", "--installed"]) + .current_dir(dir) + .output() + .ok()?; + out.status.success().then(|| { + String::from_utf8_lossy(&out.stdout) + .lines() + .map(|l| l.trim().to_string()) + .collect() + }) + }) + .as_ref() + } + + /// Cross-compile the crate for `platform` and lay the results out in + /// [`output_dir`](Self::output_dir), replacing anything there. + /// + /// # Errors + /// + /// Returns an error when [`check`](Self::check) fails, Cargo fails, or + /// Cargo produces none of the expected libraries. + pub fn build(&self, platform: Platform) -> Result { + self.check(platform)?; + let triple = platform.rust_target(); + let library = self.library; + let crate_types = match platform.os() { + Os::Ios => "staticlib", + Os::MacOs => "cdylib,staticlib", + _ => "cdylib", + }; + let mut link_args: Vec = Vec::new(); + match platform.os() { + Os::MacOs => { + link_args.push(format!("-Wl,-install_name,@rpath/lib{library}.dylib")); + } + Os::Linux => link_args.push(format!("-Wl,-soname,lib{library}.so")), + Os::Android => { + link_args.push(format!("-Wl,-soname,lib{library}.so")); + link_args.push("-Wl,-z,max-page-size=16384".into()); + } + Os::Wasm => { + link_args.push("--export-table".into()); + link_args.push("--growable-table".into()); + } + Os::Windows | Os::Ios => {} + } + + let mut cmd = Command::new(cargo_bin()); + cmd.args(["rustc", "--lib", "--message-format=json-render-diagnostics"]) + .arg("--manifest-path") + .arg(self.krate.manifest_path.as_str()) + .args(["--target", triple, "--crate-type", crate_types]); + if self.settings.release { + cmd.arg("--release"); + } + if self.quiet { + cmd.arg("--quiet"); + } + if !link_args.is_empty() { + cmd.arg("--"); + for arg in &link_args { + cmd.arg("-C").arg(format!("link-arg={arg}")); + } + } + match platform.os() { + Os::MacOs => { + cmd.env( + "MACOSX_DEPLOYMENT_TARGET", + &self.settings.macos_deployment_target, + ); + } + Os::Ios => { + cmd.env( + "IPHONEOS_DEPLOYMENT_TARGET", + &self.settings.ios_deployment_target, + ); + } + Os::Android => { + let ndk = self.ndk()?; + let clang = ndk.clang(platform, self.settings.android_api)?; + let env_triple = triple.replace('-', "_"); + cmd.env( + format!("CARGO_TARGET_{}_LINKER", env_triple.to_uppercase()), + clang.as_str(), + ) + .env(format!("CC_{env_triple}"), clang.as_str()) + .env(format!("AR_{env_triple}"), ndk.ar()?.as_str()); + } + Os::Linux | Os::Windows | Os::Wasm => {} + } + let output = cmd + .stdout(Stdio::piped()) + .stderr(Stdio::inherit()) + .output() + .context("failed to run cargo")?; + if !output.status.success() { + bail!( + "cargo failed to build `{}` for {} ({triple}); see its errors above", + self.krate.name, + platform.id() + ); + } + let produced = self.krate.lib_artifacts(&output.stdout); + + let dir = self.output_dir(platform); + if dir.exists() { + std::fs::remove_dir_all(dir.as_std_path()) + .with_context(|| format!("failed to clear {dir}"))?; + } + std::fs::create_dir_all(dir.as_std_path()) + .with_context(|| format!("failed to create {dir}"))?; + let place = |suffix: &str, name: String| -> Result> { + let Some(source) = produced.iter().find(|p| p.as_str().ends_with(suffix)) else { + return Ok(None); + }; + let dest = dir.join(name); + std::fs::copy(source.as_std_path(), dest.as_std_path()) + .with_context(|| format!("failed to copy {source} to {dest}"))?; + Ok(Some(dest)) + }; + let primary = platform.lib_filename(library); + let (suffix, name) = match platform.os() { + Os::MacOs => (".dylib", primary), + Os::Linux | Os::Android => (".so", primary), + Os::Windows => (".dll", primary), + Os::Wasm => (".wasm", primary), + Os::Ios => (".a", primary), + }; + let Some(lib) = place(suffix, name)? else { + bail!( + "cargo built `{}` for {triple} but produced no {suffix} library", + self.krate.name + ); + }; + let mut binary = NativeBinary::new(platform, lib); + match platform.os() { + Os::Ios => binary.staticlib = Some(binary.library.clone()), + Os::MacOs => binary.staticlib = place(".a", format!("lib{library}.a"))?, + Os::Windows => binary.import_library = place(".dll.lib", format!("{library}.dll.lib"))?, + _ => {} + } + Ok(binary) + } +} diff --git a/crates/weaveffi-cli/src/build/ndk.rs b/crates/weaveffi-cli/src/build/ndk.rs new file mode 100644 index 00000000..87e4e856 --- /dev/null +++ b/crates/weaveffi-cli/src/build/ndk.rs @@ -0,0 +1,180 @@ +//! Locating the Android NDK and its per-API-level compilers. + +use anyhow::{bail, Context, Result}; +use camino::{Utf8Path, Utf8PathBuf}; + +use crate::platform::Platform; + +/// An Android NDK installation. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Ndk { + /// The NDK's root directory (holding `toolchains/` and + /// `source.properties`). + pub root: Utf8PathBuf, +} + +impl Ndk { + /// Find the NDK: `ANDROID_NDK_HOME`, then `ANDROID_NDK_ROOT`, then the + /// newest `ndk/` (or `ndk-bundle`) of the Android SDK named by + /// `ANDROID_HOME` or `ANDROID_SDK_ROOT`, or installed at the Android + /// Studio default location. + /// + /// # Errors + /// + /// Returns an error explaining how to install or point at an NDK when + /// none is found. + pub fn locate() -> Result { + let var = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty()); + for name in ["ANDROID_NDK_HOME", "ANDROID_NDK_ROOT"] { + if let Some(dir) = var(name) { + let root = Utf8PathBuf::from(dir); + if !root.join("toolchains").is_dir() { + bail!("{name} is set to {root}, which isn't an Android NDK (no toolchains/)"); + } + return Ok(Self { root }); + } + } + let home = var("HOME") + .or_else(|| var("USERPROFILE")) + .map(Utf8PathBuf::from); + let mut sdks: Vec = ["ANDROID_HOME", "ANDROID_SDK_ROOT"] + .into_iter() + .filter_map(|n| var(n).map(Utf8PathBuf::from)) + .collect(); + if let Some(home) = &home { + sdks.push(home.join("Library/Android/sdk")); + sdks.push(home.join("Android/Sdk")); + } + if let Some(local) = var("LOCALAPPDATA") { + sdks.push(Utf8PathBuf::from(local).join("Android/Sdk")); + } + for sdk in &sdks { + if let Some(root) = newest_version(&sdk.join("ndk")) { + return Ok(Self { root }); + } + let bundle = sdk.join("ndk-bundle"); + if bundle.join("toolchains").is_dir() { + return Ok(Self { root: bundle }); + } + } + bail!( + "the Android NDK wasn't found: set ANDROID_NDK_HOME to an NDK directory, or install \ + one with the Android SDK manager (`sdkmanager --install \"ndk;27.2.12479018\"`) so \ + it lands in $ANDROID_HOME/ndk/" + ) + } + + /// The directory of the NDK's LLVM toolchain binaries for this host. + /// + /// # Errors + /// + /// Returns an error when the NDK has no prebuilt LLVM toolchain. + pub fn bin_dir(&self) -> Result { + let prebuilt = self.root.join("toolchains/llvm/prebuilt"); + let host = prebuilt + .read_dir_utf8() + .with_context(|| format!("the NDK at {} has no {prebuilt}", self.root))? + .filter_map(Result::ok) + .map(|e| e.path().to_path_buf()) + .find(|p| p.join("bin").is_dir()); + match host { + Some(host) => Ok(host.join("bin")), + None => bail!( + "the NDK at {} has no LLVM toolchain under {prebuilt}", + self.root + ), + } + } + + /// The C compiler driver targeting `platform` at Android `api_level` + /// (`aarch64-linux-android21-clang`). + /// + /// # Errors + /// + /// Returns an error when the NDK has no compiler for that platform and + /// API level. + pub fn clang(&self, platform: Platform, api_level: u32) -> Result { + let suffix = if cfg!(windows) { ".cmd" } else { "" }; + let clang = self.bin_dir()?.join(format!( + "{}{api_level}-clang{suffix}", + platform.rust_target() + )); + if !clang.is_file() { + bail!( + "the NDK at {} has no compiler for {} at API level {api_level} ({clang}); \ + check `[build] android_api`", + self.root, + platform.id() + ); + } + Ok(clang) + } + + /// The NDK's `llvm-ar`. + /// + /// # Errors + /// + /// Returns an error when the NDK has no prebuilt LLVM toolchain. + pub fn ar(&self) -> Result { + let suffix = if cfg!(windows) { ".exe" } else { "" }; + Ok(self.bin_dir()?.join(format!("llvm-ar{suffix}"))) + } +} + +/// The newest NDK under an SDK's `ndk/` directory, comparing the numeric +/// components of each version directory's name. +fn newest_version(dir: &Utf8Path) -> Option { + let key = |name: &str| -> Vec { + name.split('.') + .map(|part| part.parse().unwrap_or(0)) + .collect() + }; + dir.read_dir_utf8() + .ok()? + .filter_map(Result::ok) + .map(|e| e.path().to_path_buf()) + .filter(|p| p.join("toolchains").is_dir()) + .max_by_key(|p| key(p.file_name().unwrap_or(""))) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_newest_ndk_version_wins() { + let dir = tempfile::tempdir().unwrap(); + let ndk = Utf8Path::from_path(dir.path()).unwrap().join("ndk"); + for v in ["25.2.9519653", "27.1.12297006", "9.0.1", "notes"] { + let toolchains = ndk.join(v).join("toolchains"); + if v != "notes" { + std::fs::create_dir_all(toolchains).unwrap(); + } else { + std::fs::create_dir_all(ndk.join(v)).unwrap(); + } + } + assert_eq!(newest_version(&ndk), Some(ndk.join("27.1.12297006"))); + assert_eq!(newest_version(&ndk.join("missing")), None); + } + + #[test] + fn compilers_are_named_by_triple_and_api_level() { + let dir = tempfile::tempdir().unwrap(); + let root = Utf8Path::from_path(dir.path()).unwrap().to_path_buf(); + let bin = root.join("toolchains/llvm/prebuilt/test-host/bin"); + std::fs::create_dir_all(&bin).unwrap(); + let suffix = if cfg!(windows) { ".cmd" } else { "" }; + std::fs::write( + bin.join(format!("aarch64-linux-android24-clang{suffix}")), + "", + ) + .unwrap(); + let ndk = Ndk { root }; + assert_eq!( + ndk.clang(Platform::AndroidArm64, 24).unwrap(), + bin.join(format!("aarch64-linux-android24-clang{suffix}")) + ); + let err = ndk.clang(Platform::AndroidArm64, 19).unwrap_err(); + assert!(err.to_string().contains("android_api"), "{err}"); + } +} diff --git a/crates/weaveffi-gen/src/cabi.rs b/crates/weaveffi-cli/src/cabi.rs similarity index 70% rename from crates/weaveffi-gen/src/cabi.rs rename to crates/weaveffi-cli/src/cabi.rs index 7fc88834..e7273912 100644 --- a/crates/weaveffi-gen/src/cabi.rs +++ b/crates/weaveffi-cli/src/cabi.rs @@ -1,5 +1,5 @@ //! Shared rendering of the **C ABI declarations** from a -//! [`BindingModel`]. +//! [`Model`]. //! //! Both the C generator (which emits the canonical `{library}.h`) and the C++ //! generator (whose idiomatic wrapper opens an `extern "C"` block re-declaring @@ -11,27 +11,22 @@ use std::fmt::Write; -use crate::codegen::common::{emit_doc as common_emit_doc, DocCommentStyle}; +use crate::codegen::common::{emit_doc, DocCommentStyle}; use crate::codegen::CodeWriter; use crate::lang::{is_reserved, CPP_KEYWORDS, C_KEYWORDS}; use weaveffi_model::abi::AbiParam; use weaveffi_model::model::{ - checksum_symbol, AbiFn, BindingModel, CallShape, CallbackInterfaceBinding, EnumBinding, - ErrorBinding, FnBinding, InterfaceBinding, ModuleBinding, + contract_check_symbol, contract_symbol, AbiFn, CallShape, CallbackInterfaceBinding, + EnumBinding, ErrorBinding, FnBinding, InterfaceBinding, Model, ModuleBinding, }; /// The revision of the WeaveFFI C ABI the generators emit bindings for. /// /// The C ABI revision the generated headers declare. Mirrors -/// `weaveffi_abi::ABI_VERSION`; the two are kept equal by a test in the -/// `weaveffi` crate. +/// `weaveffi::abi::ABI_VERSION`; the two are kept equal by a test in this +/// crate. pub use weaveffi_model::model::ABI_VERSION; -/// Emit a `/** ... */` doc comment at `indent`. -pub fn emit_doc(out: &mut String, doc: &Option, indent: &str) { - common_emit_doc(out, doc, indent, DocCommentStyle::Javadoc); -} - /// Join lowered ABI slots into a `" , ..."` declaration string. /// /// Parameter names are the one position in a header where an IDL-chosen @@ -152,11 +147,10 @@ pub fn fn_decl(out: &mut String, f: &AbiFn, prefix: &str) { /// Render the runtime surface every producer exports (see the C ABI /// contract): the ABI revision check, the error struct and its helpers, the -/// allocation release function, cancel tokens, the Wasm staging allocator, -/// and the debug leak counter, followed by one checksum declaration per -/// top-level module. -pub fn render_runtime_decls(out: &mut String, model: &BindingModel) { - let prefix = model.prefix.as_str(); +/// allocation functions, cancel tokens, and the debug leak counter, followed +/// by each top-level module's contract table (see [`render_contract`]). +pub fn render_runtime_decls(out: &mut String, model: &Model) { + let prefix = model.prefix(); let api = export_macro(prefix); let upper = prefix.to_uppercase(); let _ = write!( @@ -180,15 +174,24 @@ pub fn render_runtime_decls(out: &mut String, model: &BindingModel) { size_t payload_len;\n\ }} {prefix}_error;\n\n\ /* Fill `err` with `code` and a producer-owned copy of `message`. Callback\n \ - interface implementations call this to report a failure (code -4)\n \ - without allocating with a foreign allocator. */\n\ + interface implementations call this to report a failure without\n \ + allocating with a foreign allocator. */\n\ {api} void {prefix}_error_set({prefix}_error* err, int32_t code, const char* message);\n\ + /* Replace `err`'s payload with a producer-owned copy of `len` bytes: the\n \ + fields of a declared error code, which a callback method that throws\n \ + attaches after {prefix}_error_set. */\n\ + {api} void {prefix}_error_set_payload({prefix}_error* err, const uint8_t* ptr, size_t len);\n\ {api} void {prefix}_error_clear({prefix}_error* err);\n\n\ /* Async completions receive a heap-boxed error the consumer owns;\n \ {prefix}_error_free releases the message, the payload, and the box.\n \ NULL is a no-op. */\n\ {api} void {prefix}_error_free({prefix}_error* err);\n\n\ - /* Release a string, bytes, or value buffer the producer returned. */\n\ + /* Allocate a zero-filled run of `len` bytes (NULL for 0). A callback\n \ + method returns a string, bytes, or buffer as such a run, which the\n \ + producer adopts; any other run is released with {prefix}_free_bytes. */\n\ + {api} uint8_t* {prefix}_alloc(size_t len);\n\ + /* Release a string, bytes, or value buffer the producer returned, or a\n \ + run from {prefix}_alloc, with its exact length. NULL or 0 is a no-op. */\n\ {api} void {prefix}_free_bytes(uint8_t* ptr, size_t len);\n\n\ /* Cancellation for `cancellable` async functions. A token starts with\n \ one reference owned by the consumer; the producer takes its own for\n \ @@ -199,33 +202,89 @@ pub fn render_runtime_decls(out: &mut String, model: &BindingModel) { {api} void {prefix}_cancel_token_cancel({prefix}_cancel_token* token);\n\ {api} bool {prefix}_cancel_token_is_cancelled(const {prefix}_cancel_token* token);\n\ {api} void {prefix}_cancel_token_destroy({prefix}_cancel_token* token);\n\n\ - /* Linear-memory allocator the Wasm glue stages arguments through.\n \ - Exported by wasm32 producers only. */\n\ - {api} uint8_t* {prefix}_alloc(uint32_t size);\n\ - {api} void {prefix}_dealloc(uint8_t* ptr, uint32_t size);\n\n\ /* Live-allocation counters for leak checks in tests: 0 objects,\n \ 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations.\n \ - Always 0 unless the producer enables the `leak-check` feature. */\n\ - {api} uint64_t {prefix}_debug_live(int32_t kind);\n\n", + Kind -1 is 1 when the producer counts at all (its `leak-check`\n \ + feature) and 0 when every counter reads 0 regardless. */\n\ + {api} uint64_t {prefix}_debug_live(int32_t kind);\n\n\ + /* One declaration's fingerprint in a module's contract table: FNV-1a 64\n \ + of its dotted path and of its canonical signature. */\n\ + typedef struct {prefix}_contract_entry {{\n \ + uint64_t id;\n \ + uint64_t hash;\n\ + }} {prefix}_contract_entry;\n\n", ); - let roots: Vec<&ModuleBinding> = model.roots().collect(); - if !roots.is_empty() { - out.push_str( - "/* Contract checksums, one per top-level module. A consumer compares\n \ - each function's result with the constant it was generated against\n \ - before making any other call. */\n", - ); + for root in model.roots() { + render_contract(out, model, root); + } +} + +/// Render one top-level module's contract: the producer's table function, +/// the entries these bindings were generated with as +/// `{PREFIX}_{MODULE}_CONTRACT` and `{PREFIX}_{MODULE}_CONTRACT_LEN`, and a +/// `static inline` checker returning the id of the first expected entry the +/// producer lacks or hashes differently (`0` when compatible). +pub fn render_contract(out: &mut String, model: &Model, root: &ModuleBinding) { + let prefix = model.prefix(); + let api = export_macro(prefix); + let symbol = contract_symbol(prefix, &root.name); + let check = contract_check_symbol(prefix, &root.name); + let upper = symbol.to_uppercase(); + let entries = weaveffi_model::contract::entries(model, root); + let _ = writeln!( + out, + "/* The contract of module `{}`: one entry per declaration, sorted by\n \ + id. The producer's table comes from the function below; a consumer\n \ + checks each entry it was generated with against it once, before any\n \ + other call (producer entries it doesn't know are fine). */\n\ + {api} const {prefix}_contract_entry* {symbol}(size_t* out_len);\n\ + #define {upper}_LEN {}\n\ + #define {upper} {{ \\", + root.name, + entries.len() + ); + if entries.is_empty() { + out.push_str(" {0ull, 0ull} \\\n"); } - for m in roots { - let symbol = checksum_symbol(prefix, &m.name); - let value = m.checksum.unwrap_or_default(); + for e in &entries { let _ = writeln!( out, - "#define {} 0x{value:016x}ull\n{api} uint64_t {symbol}(void);", - symbol.to_uppercase() + " {{0x{:016x}ull, 0x{:016x}ull}}, /* {} */ \\", + e.id, e.hash, e.path ); } - out.push('\n'); + out.push_str("}\n"); + if entries.is_empty() { + let _ = write!( + out, + "/* Nothing to check: the module declares nothing. */\n\ + static inline uint64_t {check}(void) {{\n \ + return 0;\n\ + }}\n\n" + ); + return; + } + let _ = write!( + out, + "/* The id of the first expected entry the loaded library lacks or\n \ + declares differently, or 0 when these bindings match it. */\n\ + static inline uint64_t {check}(void) {{\n \ + static const {prefix}_contract_entry expected[] = {upper};\n \ + size_t len = 0;\n \ + const {prefix}_contract_entry* table = {symbol}(&len);\n \ + for (size_t i = 0; i < {upper}_LEN; i++) {{\n \ + size_t lo = 0, hi = len;\n \ + while (lo < hi) {{\n \ + size_t mid = lo + (hi - lo) / 2;\n \ + if (table[mid].id < expected[i].id) lo = mid + 1; else hi = mid;\n \ + }}\n \ + if (lo == len || table[lo].id != expected[i].id || table[lo].hash != expected[i].hash) {{\n \ + return expected[i].id;\n \ + }}\n \ + }}\n \ + return 0;\n\ + }}\n\n" + ); } fn render_enum_constants(out: &mut String, e: &EnumBinding, type_name: &str) { @@ -312,10 +371,8 @@ pub fn render_module_enum_defs(out: &mut String, module: &ModuleBinding) { render_enum_decl(out, e); } } - if let Some(err) = &module.error { - if err.declared_here { - render_error_domain_decl(out, err, &module.dot_path); - } + if let Some(err) = &module.errors { + render_error_domain_decl(out, err, &module.dot_path); } } @@ -337,23 +394,28 @@ pub fn render_module_type_tags(out: &mut String, module: &ModuleBinding) { } } -/// Render one callback interface's vtable struct: one function pointer per -/// method in declaration order, then the trailing `free`. -fn render_vtable_decl(out: &mut String, cb: &CallbackInterfaceBinding, prefix: &str) { +/// Render one callback interface's vtable struct: the fixed header (`size`, +/// `flags`, `free`), then one function pointer per method in declaration +/// order. +fn render_vtable_decl( + out: &mut String, + cb: &CallbackInterfaceBinding, + domain: Option<&ErrorBinding>, + prefix: &str, +) { let mut w = CodeWriter::four_space(); + let contract = "Consumer-implemented callback interface. The consumer passes a context \ + pointer plus a pointer to a static instance of this vtable, with `size` \ + set to `sizeof` the vtable and `flags` to 0 (the producer rejects a smaller \ + vtable). The producer may call any entry from any thread until it calls \ + `free(ctx)` exactly once, also from any thread. A method reports failure \ + with `error_set(out_err, ...)`; a string, bytes, or buffer return is a \ + run the consumer allocates with `alloc` and writes to `out_ptr` and \ + `out_len`, which the producer adopts."; w.doc( &Some(match &cb.doc { - Some(doc) => format!( - "{doc}\n\nConsumer-implemented callback interface. The consumer passes a \ - context pointer plus a pointer to a static instance of this vtable; the \ - producer may call any entry from any thread until it calls `free(ctx)` \ - exactly once." - ), - None => "Consumer-implemented callback interface. The consumer passes a \ - context pointer plus a pointer to a static instance of this vtable; the \ - producer may call any entry from any thread until it calls `free(ctx)` \ - exactly once." - .to_string(), + Some(doc) => format!("{doc}\n\n{contract}"), + None => contract.to_string(), }), DocCommentStyle::Javadoc, ); @@ -361,8 +423,23 @@ fn render_vtable_decl(out: &mut String, cb: &CallbackInterfaceBinding, prefix: & format!("typedef struct {} {{", cb.vtable_tag), format!("}} {};", cb.vtable_tag), |w| { + w.line("uint32_t size;"); + w.line("uint32_t flags;"); + w.line("void (*free)(void* ctx);"); for m in &cb.methods { - w.doc(&m.doc, DocCommentStyle::Javadoc); + let throws = match (m.throws, domain) { + (true, Some(d)) => Some(format!( + "May fail with a `{}` code (its fields attached with \ + `{prefix}_error_set_payload`).", + d.c_tag + )), + _ => None, + }; + let doc = match (&m.doc, throws) { + (Some(doc), Some(t)) => Some(format!("{doc}\n\n{t}")), + (doc, t) => doc.clone().or(t), + }; + w.doc(&doc, DocCommentStyle::Javadoc); w.line(format!( "{} (*{})({});", m.abi_ret.render_c(prefix), @@ -370,7 +447,6 @@ fn render_vtable_decl(out: &mut String, cb: &CallbackInterfaceBinding, prefix: & params_str(&m.abi_params, prefix) )); } - w.line("void (*free)(void* ctx);"); }, ); out.push_str(&w.finish()); @@ -380,9 +456,21 @@ fn render_vtable_decl(out: &mut String, cb: &CallbackInterfaceBinding, prefix: & /// function-pointer typedefs for one module. These may reference enums (by /// value) and interfaces (by pointer), so they are emitted after every /// module's enums and type tags. -pub fn render_module_callback_types(out: &mut String, module: &ModuleBinding, prefix: &str) { +pub fn render_module_callback_types( + out: &mut String, + module: &ModuleBinding, + modules: &[ModuleBinding], + prefix: &str, +) { + // The domain in scope: the module's own, else the nearest ancestor's. + let domain = (1..=module.segments.len()).rev().find_map(|n| { + modules + .iter() + .find(|m| m.segments[..] == module.segments[..n]) + .and_then(|m| m.errors.as_ref()) + }); for cb in &module.callback_interfaces { - render_vtable_decl(out, cb, prefix); + render_vtable_decl(out, cb, domain, prefix); } for f in module.callables() { if let CallShape::Async(a) = &f.shape { @@ -402,7 +490,7 @@ pub fn render_module_callback_types(out: &mut String, module: &ModuleBinding, pr fn render_callable_decl(out: &mut String, f: &FnBinding, prefix: &str) { let api = export_macro(prefix); let deprecated = deprecated_macro(prefix); - emit_doc(out, &f.doc, ""); + emit_doc(out, &f.doc, "", DocCommentStyle::Javadoc); if let Some(msg) = &f.deprecated { let _ = writeln!(out, "{deprecated}(\"{}\")", msg.replace('"', "\\\"")); } @@ -428,7 +516,7 @@ fn render_callable_decl(out: &mut String, f: &FnBinding, prefix: &str) { fn render_interface_fn_decls(out: &mut String, i: &InterfaceBinding, prefix: &str) { let api = export_macro(prefix); let tag = &i.c_tag; - emit_doc(out, &i.doc, ""); + emit_doc(out, &i.doc, "", DocCommentStyle::Javadoc); for c in &i.constructors { render_callable_decl(out, c, prefix); } @@ -446,6 +534,7 @@ fn render_interface_fn_decls(out: &mut String, i: &InterfaceBinding, prefix: &st .to_string(), ), "", + DocCommentStyle::Javadoc, ); let _ = writeln!(out, "{api} {tag}* {}(const {tag}* self);", i.clone_symbol); emit_doc( @@ -456,6 +545,7 @@ fn render_interface_fn_decls(out: &mut String, i: &InterfaceBinding, prefix: &st .to_string(), ), "", + DocCommentStyle::Javadoc, ); let _ = writeln!(out, "{api} void {}({tag}* self);", i.destroy_symbol); out.push('\n'); @@ -497,7 +587,7 @@ pub fn render_decls( render_module_type_tags(out, m); } for m in modules { - render_module_callback_types(out, m, prefix); + render_module_callback_types(out, m, modules, prefix); } out.push('\n'); for m in modules { diff --git a/crates/weaveffi-cli/src/cache.rs b/crates/weaveffi-cli/src/cache.rs new file mode 100644 index 00000000..481905ac --- /dev/null +++ b/crates/weaveffi-cli/src/cache.rs @@ -0,0 +1,73 @@ +//! Per-target generation records for stale-file cleanup. +//! +//! After generating a target, the orchestrator writes +//! `{out_dir}/.weaveffi-cache/{target}.json`: the path of every file the +//! target produced. Files recorded by the previous run that the current run +//! no longer produces are removed, so a renamed type never leaves a stale +//! file behind, while files the user added (a `node_modules/`, a build +//! directory) are never touched. + +use std::collections::BTreeSet; + +use anyhow::{Context, Result}; +use camino::{Utf8Path, Utf8PathBuf}; +use serde::{Deserialize, Serialize}; + +const CACHE_DIR: &str = ".weaveffi-cache"; + +/// What one generation of one target produced. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct Record { + /// Every file the target produced, relative to the output directory + /// (with `/` separators). + pub files: BTreeSet, +} + +fn record_path(out_dir: &Utf8Path, target: &str) -> Utf8PathBuf { + out_dir.join(CACHE_DIR).join(format!("{target}.json")) +} + +/// Read the record of the last generation of `target` under `out_dir`, if +/// any (a missing or unreadable record means "never generated"). +#[must_use] +pub fn read_record(out_dir: &Utf8Path, target: &str) -> Option { + let text = std::fs::read_to_string(record_path(out_dir, target)).ok()?; + serde_json::from_str(&text).ok() +} + +/// Persist the record of a generation of `target`. +/// +/// # Errors +/// +/// Returns an error when the cache directory cannot be created or the record +/// cannot be written. +/// +/// # Panics +/// +/// Panics if the record cannot be serialized, which does not happen. +pub fn write_record(out_dir: &Utf8Path, target: &str, record: &Record) -> Result<()> { + let path = record_path(out_dir, target); + let dir = out_dir.join(CACHE_DIR); + std::fs::create_dir_all(dir.as_std_path()) + .with_context(|| format!("failed to create cache directory: {dir}"))?; + let json = serde_json::to_string_pretty(record).expect("record serializes"); + std::fs::write(path.as_std_path(), json) + .with_context(|| format!("failed to write cache file: {path}")) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn records_round_trip() { + let dir = tempfile::tempdir().unwrap(); + let out = Utf8Path::from_path(dir.path()).unwrap(); + assert_eq!(read_record(out, "swift"), None); + let record = Record { + files: ["swift/a.swift".to_string()].into(), + }; + write_record(out, "swift", &record).unwrap(); + assert_eq!(read_record(out, "swift"), Some(record)); + } +} diff --git a/crates/weaveffi-cli/src/cargo.rs b/crates/weaveffi-cli/src/cargo.rs new file mode 100644 index 00000000..fa73a36e --- /dev/null +++ b/crates/weaveffi-cli/src/cargo.rs @@ -0,0 +1,316 @@ +//! Resolving a Rust producer crate with `cargo metadata`. +//! +//! Cargo is the authority on a crate's name, version (including +//! `version.workspace = true`), library target name, and target directory, so +//! WeaveFFI asks it instead of parsing `Cargo.toml` itself. Every command +//! that reads a Rust producer's API uses [`CargoCrate`] to locate the crate +//! and [`CargoCrate::build_library`] to build the library the API is read +//! from, and every build output lands under its +//! [`target_dir`](CargoCrate::target_dir). + +use std::process::{Command, Stdio}; + +use anyhow::{bail, Context, Result}; +use camino::{Utf8Path, Utf8PathBuf}; +use serde::Deserialize; + +/// The facts about a Rust producer crate that building and packaging need, +/// as `cargo metadata` reports them. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CargoCrate { + /// The Cargo package name (`my-kv`). + pub name: String, + /// The resolved package version, with workspace inheritance applied. + pub version: String, + /// The library target's name with `-` mapped to `_` (`my_kv`): the base + /// name of the `lib{lib_name}.so` Cargo produces. + pub lib_name: String, + /// The crate types declared for the library target (`cdylib`, `rlib`, …). + pub crate_types: Vec, + /// The absolute path of the crate's `Cargo.toml`. + pub manifest_path: Utf8PathBuf, + /// The Cargo target directory (respecting `CARGO_TARGET_DIR`, + /// `build.target-dir`, and workspaces). + pub target_dir: Utf8PathBuf, + /// The package description, if declared. + pub description: Option, + /// The package license expression, if declared. + pub license: Option, + /// The package authors. + pub authors: Vec, + /// The package homepage, if declared. + pub homepage: Option, + /// The package repository, if declared. + pub repository: Option, +} + +impl CargoCrate { + /// Resolve the crate whose manifest is `manifest` by running + /// `cargo metadata --no-deps`. + /// + /// # Errors + /// + /// Returns an error when `manifest` doesn't exist, when Cargo can't be + /// run or rejects the manifest, when the manifest is a virtual workspace + /// manifest with no package of its own, or when the package has no + /// library target. + pub fn resolve(manifest: &Utf8Path) -> Result { + if !manifest.is_file() { + bail!("no Cargo.toml at {manifest}"); + } + let output = Command::new(cargo_bin()) + .args(["metadata", "--format-version", "1", "--no-deps"]) + .arg("--manifest-path") + .arg(manifest.as_str()) + .output() + .context("failed to run `cargo metadata`; is Cargo installed and on PATH?")?; + if !output.status.success() { + bail!( + "`cargo metadata` failed for {manifest}:\n{}", + String::from_utf8_lossy(&output.stderr).trim_end() + ); + } + let metadata: Metadata = serde_json::from_slice(&output.stdout) + .context("failed to parse `cargo metadata` output")?; + let wanted = canonical(manifest); + let Some(package) = metadata + .packages + .into_iter() + .find(|p| canonical(&p.manifest_path) == wanted) + else { + bail!( + "{manifest} is a virtual workspace manifest; point at the producer crate's own \ + Cargo.toml instead" + ); + }; + let Some(lib) = package.targets.iter().find(|t| t.is_lib()) else { + bail!( + "the crate `{}` has no library target; a WeaveFFI producer is a library \ + (src/lib.rs)", + package.name + ); + }; + Ok(Self { + lib_name: lib.name.replace('-', "_"), + crate_types: lib.crate_types.clone(), + name: package.name, + version: package.version, + manifest_path: package.manifest_path, + target_dir: metadata.target_directory, + description: package.description, + license: package.license, + authors: package.authors, + homepage: package.homepage, + repository: package.repository, + }) + } + + /// The directory `weaveffi build` lays its per-platform outputs out in: + /// `{target_dir}/weaveffi`. + #[must_use] + pub fn weaveffi_dir(&self) -> Utf8PathBuf { + self.target_dir.join("weaveffi") + } + + /// The directory holding the crate's `Cargo.toml`. + #[must_use] + pub fn dir(&self) -> &Utf8Path { + self.manifest_path.parent().unwrap_or(Utf8Path::new(".")) + } + + /// Build the crate's library for the host with + /// `cargo build --lib --message-format=json-render-diagnostics` (in + /// release mode with `release`), forwarding Cargo's diagnostics to + /// stderr, and return the library it produced: the `cdylib` if the crate + /// declares one, else the `staticlib`. + /// + /// # Errors + /// + /// Returns an error when Cargo can't be run or fails, or when the crate + /// produces neither a `cdylib` nor a `staticlib`. + pub fn build_library(&self, release: bool, quiet: bool) -> Result { + let mut cmd = Command::new(cargo_bin()); + cmd.args(["build", "--lib", "--message-format=json-render-diagnostics"]) + .arg("--manifest-path") + .arg(self.manifest_path.as_str()); + if release { + cmd.arg("--release"); + } + if quiet { + cmd.arg("--quiet"); + } + let output = cmd + .stdout(Stdio::piped()) + .stderr(Stdio::inherit()) + .output() + .context("failed to run `cargo build`; is Cargo installed and on PATH?")?; + if !output.status.success() { + bail!( + "cargo failed to build `{}`; see its errors above", + self.name + ); + } + let files = self.lib_artifacts(&output.stdout); + let is = |suffixes: &[&str]| { + files + .iter() + .find(|f| suffixes.iter().any(|s| f.as_str().ends_with(s))) + .cloned() + }; + let library = is(&[".dylib", ".so", ".dll", ".wasm"]).or_else(|| { + files + .iter() + .find(|f| { + f.as_str().ends_with(".a") + || (f.as_str().ends_with(".lib") && !f.as_str().ends_with(".dll.lib")) + }) + .cloned() + }); + library.ok_or_else(|| { + anyhow::anyhow!( + "`{}` builds no cdylib or staticlib to read its API from; add `crate-type = \ + [\"cdylib\"]` under [lib] in {}", + self.name, + self.manifest_path + ) + }) + } + + /// The files Cargo reported for this crate's library target, from the + /// JSON messages (`--message-format=json`) it printed to stdout. + #[must_use] + pub fn lib_artifacts(&self, stdout: &[u8]) -> Vec { + let manifest = canonical(&self.manifest_path); + let mut out = Vec::new(); + for line in String::from_utf8_lossy(stdout).lines() { + let Ok(msg) = serde_json::from_str::(line) else { + continue; + }; + if msg["reason"] != "compiler-artifact" { + continue; + } + let ours = msg["manifest_path"] + .as_str() + .is_some_and(|p| canonical(Utf8Path::new(p)) == manifest); + let lib = msg["target"]["kind"] + .as_array() + .is_some_and(|kinds| kinds.iter().any(|k| k != "custom-build")); + if !ours || !lib { + continue; + } + if let Some(files) = msg["filenames"].as_array() { + out.extend( + files + .iter() + .filter_map(|f| f.as_str()) + .map(Utf8PathBuf::from), + ); + } + } + out + } +} + +/// The nearest `Cargo.toml` at or above `start` (a file or a directory), +/// or `None` when there is none up to the filesystem root. +#[must_use] +pub fn find_manifest(start: &Utf8Path) -> Option { + let start = std::path::absolute(start.as_std_path()).ok()?; + let start = Utf8PathBuf::from_path_buf(start).ok()?; + start + .ancestors() + .map(|dir| dir.join("Cargo.toml")) + .find(|candidate| candidate.is_file()) +} + +/// The `cargo` executable: `$CARGO` when running under Cargo, else `cargo`. +#[must_use] +pub fn cargo_bin() -> String { + std::env::var("CARGO").unwrap_or_else(|_| "cargo".to_string()) +} + +/// `path` with symlinks and `..` resolved where possible, for comparing +/// manifest paths. +fn canonical(path: &Utf8Path) -> Utf8PathBuf { + path.canonicalize_utf8() + .unwrap_or_else(|_| path.to_path_buf()) +} + +#[derive(Deserialize)] +struct Metadata { + packages: Vec, + target_directory: Utf8PathBuf, +} + +#[derive(Deserialize)] +struct Package { + name: String, + version: String, + manifest_path: Utf8PathBuf, + targets: Vec, + description: Option, + license: Option, + #[serde(default)] + authors: Vec, + homepage: Option, + repository: Option, +} + +#[derive(Deserialize)] +struct Target { + name: String, + kind: Vec, + #[serde(default)] + crate_types: Vec, +} + +impl Target { + fn is_lib(&self) -> bool { + self.kind.iter().any(|k| { + matches!( + k.as_str(), + "lib" | "rlib" | "dylib" | "cdylib" | "staticlib" | "proc-macro" + ) + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn repo_root() -> Utf8PathBuf { + Utf8Path::new(env!("CARGO_MANIFEST_DIR")).join("../..") + } + + #[test] + fn resolves_a_workspace_member() { + let manifest = repo_root().join("samples/calculator/Cargo.toml"); + let krate = CargoCrate::resolve(&manifest).unwrap(); + assert_eq!(krate.name, "calculator"); + assert_eq!(krate.lib_name, "calculator"); + assert!(krate.crate_types.iter().any(|t| t == "cdylib")); + assert!(krate.target_dir.ends_with("target"), "{}", krate.target_dir); + assert_eq!(krate.weaveffi_dir(), krate.target_dir.join("weaveffi")); + } + + #[test] + fn inherited_versions_are_resolved() { + let krate = CargoCrate::resolve(&repo_root().join("crates/weaveffi/Cargo.toml")).unwrap(); + assert_eq!(krate.version, env!("CARGO_PKG_VERSION")); + } + + #[test] + fn virtual_and_missing_manifests_are_errors() { + let err = CargoCrate::resolve(&repo_root().join("Cargo.toml")).unwrap_err(); + assert!(err.to_string().contains("virtual workspace"), "{err}"); + let err = CargoCrate::resolve(Utf8Path::new("/nonexistent/Cargo.toml")).unwrap_err(); + assert!(err.to_string().contains("no Cargo.toml"), "{err}"); + } + + #[test] + fn manifests_are_found_from_nested_paths() { + let found = find_manifest(&repo_root().join("samples/calculator/src/lib.rs")).unwrap(); + assert!(found.ends_with("samples/calculator/Cargo.toml"), "{found}"); + } +} diff --git a/crates/weaveffi-gen/src/codegen/common.rs b/crates/weaveffi-cli/src/codegen/common.rs similarity index 66% rename from crates/weaveffi-gen/src/codegen/common.rs rename to crates/weaveffi-cli/src/codegen/common.rs index 42e038ea..dc6ab93f 100644 --- a/crates/weaveffi-gen/src/codegen/common.rs +++ b/crates/weaveffi-cli/src/codegen/common.rs @@ -1,9 +1,5 @@ -//! Shared codegen primitives that every language generator can reuse. -//! -//! Until 0.4.0, every generator hand-rolled its own copy of the module -//! tree walker and the doc-comment emitter. Pulling them in here gives -//! the generators one source of truth and shrinks each crate by a few -//! dozen lines of near-identical helper code. +//! Shared codegen primitives that every language generator can reuse: the +//! doc-comment emitter and `PascalCase` conversion. //! //! Specialised flavours that exist in only one generator (Go's //! godoc-style first-line symbol prefix, .NET's `` XML tags, @@ -11,8 +7,6 @@ //! their behaviour is non-uniform; this module deliberately covers //! only the common 80%. -use weaveffi_model::ir::Module; - /// Doc-comment flavour used by [`emit_doc`]. /// /// Specialised flavours like Go's godoc-symbol prefix or .NET's @@ -92,41 +86,6 @@ fn emit_javadoc(out: &mut String, doc: &str, indent: &str) { } } -/// Iterate over every module in `roots` and its descendants in -/// depth-first pre-order: each module is yielded before its children, -/// and children are yielded left-to-right. -/// -/// Equivalent to the recursive `collect_all_modules` helper that -/// every generator used to define locally. -pub fn walk_modules<'a>(roots: &'a [Module]) -> impl Iterator { - let mut stack: Vec<&'a Module> = roots.iter().rev().collect(); - std::iter::from_fn(move || { - let m = stack.pop()?; - for child in m.modules.iter().rev() { - stack.push(child); - } - Some(m) - }) -} - -/// Like [`walk_modules`], but each module is paired with its -/// underscore-joined path (e.g. `parent_child_grandchild`). The path -/// matches the canonical C symbol prefix segment that the C generator -/// builds when emitting `{c_prefix}_{module_path}_{name}`. -pub fn walk_modules_with_path<'a>( - roots: &'a [Module], -) -> impl Iterator { - let mut stack: Vec<(&'a Module, String)> = - roots.iter().rev().map(|m| (m, m.name.clone())).collect(); - std::iter::from_fn(move || { - let (m, path) = stack.pop()?; - for child in m.modules.iter().rev() { - stack.push((child, format!("{path}_{}", child.name))); - } - Some((m, path)) - }) -} - /// Convert a `snake_case` identifier to `PascalCase` by uppercasing the /// first character of each `_`-separated segment and preserving the rest. /// @@ -150,90 +109,6 @@ pub fn pascal_case(s: &str) -> String { #[cfg(test)] mod tests { use super::*; - use weaveffi_model::ir::Module; - - fn leaf(name: &str) -> Module { - Module { - name: name.to_string(), - doc: None, - functions: vec![], - interfaces: vec![], - structs: vec![], - enums: vec![], - callback_interfaces: vec![], - errors: None, - modules: vec![], - } - } - - fn with_children(name: &str, children: Vec) -> Module { - Module { - modules: children, - ..leaf(name) - } - } - - // --- walk_modules --- - - #[test] - fn walk_modules_visits_pre_order() { - let roots = vec![ - with_children("a", vec![leaf("a1"), leaf("a2")]), - with_children("b", vec![leaf("b1")]), - ]; - let names: Vec<&str> = walk_modules(&roots).map(|m| m.name.as_str()).collect(); - assert_eq!(names, vec!["a", "a1", "a2", "b", "b1"]); - } - - #[test] - fn walk_modules_descends_to_arbitrary_depth() { - let roots = vec![with_children( - "a", - vec![with_children( - "b", - vec![with_children("c", vec![leaf("d")])], - )], - )]; - let names: Vec<&str> = walk_modules(&roots).map(|m| m.name.as_str()).collect(); - assert_eq!(names, vec!["a", "b", "c", "d"]); - } - - #[test] - fn walk_modules_empty_input_yields_nothing() { - let roots: Vec = vec![]; - assert_eq!(walk_modules(&roots).count(), 0); - } - - // --- walk_modules_with_path --- - - #[test] - fn walk_modules_with_path_joins_with_underscore() { - let roots = vec![with_children( - "outer", - vec![with_children("inner", vec![leaf("leaf")])], - )]; - let pairs: Vec<(String, String)> = walk_modules_with_path(&roots) - .map(|(m, p)| (m.name.clone(), p)) - .collect(); - assert_eq!( - pairs, - vec![ - ("outer".into(), "outer".into()), - ("inner".into(), "outer_inner".into()), - ("leaf".into(), "outer_inner_leaf".into()), - ] - ); - } - - #[test] - fn walk_modules_with_path_independent_roots() { - let roots = vec![ - with_children("a", vec![leaf("a1")]), - with_children("b", vec![leaf("b1")]), - ]; - let paths: Vec = walk_modules_with_path(&roots).map(|(_, p)| p).collect(); - assert_eq!(paths, vec!["a", "a_a1", "b", "b_b1"]); - } // --- emit_doc --- diff --git a/crates/weaveffi-cli/src/codegen/mod.rs b/crates/weaveffi-cli/src/codegen/mod.rs new file mode 100644 index 00000000..a7ca6de0 --- /dev/null +++ b/crates/weaveffi-cli/src/codegen/mod.rs @@ -0,0 +1,360 @@ +//! Target erasure and orchestration. +//! +//! Each language target implements [`LanguageBackend`] with its own typed +//! `Config`. The orchestrator works on the object-safe [`Target`] trait, which +//! erases the concrete config; [`ConfiguredBackend`] is the adapter that pairs +//! a backend with a concrete config value and is what the CLI and tests pass +//! into [`Orchestrator::with_target`]. +//! +//! Rendering is pure: a target returns its files in memory, and the +//! [`Orchestrator`] does every write. That is what lets it rewrite only files +//! whose contents changed, remove files a previous run wrote that the +//! current run no longer produces (see [`crate::cache`]), and lets +//! `weaveffi diff` compare without touching disk. + +use anyhow::{Context, Result}; +use camino::Utf8Path; +use rayon::prelude::*; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::cache::{self, Record}; +use crate::package::{Artifact, PackageContext}; +use weaveffi_model::model::Model; + +pub mod common; +pub mod writer; + +pub use writer::CodeWriter; + +/// Object-safe view of a [`LanguageBackend`] paired with a concrete config. +/// +/// The orchestrator stores targets as `&dyn Target` so it can hold a +/// heterogeneous set whose `Config` types differ. [`ConfiguredBackend`] is +/// the canonical adapter. +pub trait Target: Send + Sync { + /// The target's stable short name. Mirrors [`LanguageBackend::name`]. + fn name(&self) -> &'static str; + /// Render every file the target produces for `model`, with paths under + /// `out_dir`. Pure: nothing is written. + fn render(&self, model: &Model, out_dir: &Utf8Path) -> Vec; + /// Assemble the installable artifacts for this target, using the bound + /// config. Returns `None` when the target has no packaging. + fn package(&self, model: &Model, ctx: &PackageContext) -> Option>; +} + +/// Binds a [`LanguageBackend`] to a concrete config value so it can be erased +/// to `&dyn Target`. +/// +/// ```ignore +/// let swift = ConfiguredBackend::new(SwiftGenerator, SwiftConfig::default()); +/// orchestrator.with_target(&swift); +/// ``` +pub struct ConfiguredBackend { + inner: B, + config: B::Config, +} + +impl ConfiguredBackend { + /// Pair a backend with the concrete config it should run under. + pub fn new(inner: B, config: B::Config) -> Self { + Self { inner, config } + } +} + +impl Target for ConfiguredBackend { + fn name(&self) -> &'static str { + self.inner.name() + } + + fn render(&self, model: &Model, out_dir: &Utf8Path) -> Vec { + self.inner.files(model, out_dir, &self.config) + } + + fn package(&self, model: &Model, ctx: &PackageContext) -> Option> { + self.inner.package(model, ctx, &self.config) + } +} + +/// The path of `file` relative to `out_dir`, with `/` separators on every +/// platform, so listings and cache records are OS-independent. +/// +/// # Panics +/// +/// Panics if a backend emitted a path outside `out_dir`, which is a backend +/// bug. +#[must_use] +pub fn relative_path(out_dir: &Utf8Path, file: &Utf8Path) -> String { + let rel = file + .strip_prefix(out_dir) + .unwrap_or_else(|_| panic!("generated file {file} is outside {out_dir}")); + rel.components() + .map(|c| c.as_str()) + .collect::>() + .join("/") +} + +/// What a generation run did. +#[derive(Debug, Default, Clone, PartialEq, Eq)] +pub struct GenerateReport { + /// Targets that wrote or removed a file. + pub generated: Vec<&'static str>, + /// Targets whose output already matched the output directory. + pub up_to_date: Vec<&'static str>, + /// Files written because they were new or their contents changed. + pub written: usize, + /// Files rendered with the contents already on disk. + pub unchanged: usize, + /// Files removed because the previous run wrote them and this run did + /// not (paths relative to the output directory). + pub removed: Vec, +} + +/// One target's rendered output, paired with its previous record. +struct Rendered<'a> { + target: &'a dyn Target, + files: Vec, + previous: Option, + record: Record, +} + +impl Rendered<'_> { + /// Files the previous run wrote that this run no longer produces and + /// that are still on disk. + fn stale<'r>(&'r self, out_dir: &'r Utf8Path) -> impl Iterator { + self.previous + .iter() + .flat_map(|p| p.files.iter()) + .filter(|rel| !self.record.files.contains(*rel) && out_dir.join(rel).exists()) + } + + /// Whether writing this target would change anything on disk. + fn is_current(&self, out_dir: &Utf8Path) -> bool { + self.previous.as_ref() == Some(&self.record) + && self.stale(out_dir).next().is_none() + && self.files.iter().all(on_disk) + } +} + +/// Whether `file` is already on disk with its rendered contents. +fn on_disk(file: &OutputFile) -> bool { + std::fs::read(file.path.as_std_path()).is_ok_and(|bytes| bytes == file.contents.as_bytes()) +} + +/// Runs a set of targets against one API and output directory. +#[derive(Default)] +pub struct Orchestrator<'a> { + targets: Vec<&'a dyn Target>, +} + +impl<'a> Orchestrator<'a> { + /// An orchestrator with no targets. + pub fn new() -> Self { + Self::default() + } + + /// Add a target. + #[must_use] + pub fn with_target(mut self, target: &'a dyn Target) -> Self { + self.targets.push(target); + self + } + + /// Generate every target under `out_dir`. + /// + /// Every target renders, in parallel. Only files whose contents differ + /// from the output directory are written, and files the previous run of + /// a target wrote that this run no longer produces are removed. + /// + /// # Errors + /// + /// Returns an error when a file cannot be written or removed. + pub fn run(&self, model: &Model, out_dir: &Utf8Path) -> Result { + let rendered: Vec> = self + .targets + .par_iter() + .map(|&target| { + let files = target.render(model, out_dir); + let record = Record { + files: files + .iter() + .map(|f| relative_path(out_dir, &f.path)) + .collect(), + }; + Rendered { + target, + files, + previous: cache::read_record(out_dir, target.name()), + record, + } + }) + .collect(); + + let mut report = GenerateReport::default(); + if rendered.iter().all(|r| r.is_current(out_dir)) { + report.up_to_date = rendered.iter().map(|r| r.target.name()).collect(); + return Ok(report); + } + for r in &rendered { + let mut changed = r.previous.as_ref() != Some(&r.record); + for file in &r.files { + if on_disk(file) { + report.unchanged += 1; + continue; + } + if let Some(parent) = file.path.parent() { + std::fs::create_dir_all(parent.as_std_path()) + .with_context(|| format!("failed to create {parent}"))?; + } + std::fs::write(file.path.as_std_path(), &file.contents) + .with_context(|| format!("failed to write {}", file.path))?; + report.written += 1; + changed = true; + } + let stale: Vec = r.stale(out_dir).cloned().collect(); + for rel in stale { + let path = out_dir.join(&rel); + std::fs::remove_file(path.as_std_path()) + .with_context(|| format!("failed to remove stale file {path}"))?; + report.removed.push(rel); + changed = true; + } + if r.previous.as_ref() != Some(&r.record) { + cache::write_record(out_dir, r.target.name(), &r.record)?; + } + if changed { + report.generated.push(r.target.name()); + } else { + report.up_to_date.push(r.target.name()); + } + } + Ok(report) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::sync::{Arc, Mutex}; + use weaveffi_model::ir::{Api, Module}; + + #[derive(Default, Clone)] + struct TestConfig; + + struct Counting { + name: &'static str, + calls: Arc, + files: Arc>>, + } + + impl LanguageBackend for Counting { + type Config = TestConfig; + + fn name(&self) -> &'static str { + self.name + } + + fn files( + &self, + model: &Model, + out_dir: &Utf8Path, + _config: &Self::Config, + ) -> Vec { + self.calls.fetch_add(1, Ordering::SeqCst); + self.files + .lock() + .unwrap() + .iter() + .map(|f| { + OutputFile::new( + out_dir.join(self.name).join(f), + format!("{} {}", model.prefix(), model.modules[0].name), + ) + }) + .collect() + } + } + + fn model_named(name: &str) -> Model { + let api = Api { + version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), + modules: vec![Module { + name: name.into(), + doc: None, + functions: vec![], + interfaces: vec![], + structs: vec![], + enums: vec![], + callback_interfaces: vec![], + errors: None, + modules: vec![], + }], + }; + let identity = weaveffi_model::pkg::Identity::named("api"); + weaveffi_model::validate::validate(&api, &identity, None).unwrap() + } + + fn backend(name: &'static str, calls: &Arc) -> ConfiguredBackend { + ConfiguredBackend::new( + Counting { + name, + calls: Arc::clone(calls), + files: Arc::new(Mutex::new(vec!["out.txt"])), + }, + TestConfig, + ) + } + + #[test] + fn every_run_renders_but_only_changed_files_are_written() { + let dir = tempfile::tempdir().unwrap(); + let out = Utf8Path::from_path(dir.path()).unwrap(); + let calls = Arc::new(AtomicUsize::new(0)); + let c = backend("c", &calls); + let orch = Orchestrator::new().with_target(&c); + let model = model_named("math"); + + let first = orch.run(&model, out).unwrap(); + assert_eq!((first.generated.len(), first.written), (1, 1)); + let rendered = std::fs::read_to_string(out.join("c/out.txt")).unwrap(); + let second = orch.run(&model, out).unwrap(); + assert_eq!(second.up_to_date, ["c"]); + assert_eq!(second.written, 0); + assert_eq!(calls.load(Ordering::SeqCst), 2); + + std::fs::write(out.join("c/out.txt"), "edited").unwrap(); + let third = orch.run(&model, out).unwrap(); + assert_eq!((third.generated.len(), third.written), (1, 1)); + assert_eq!( + std::fs::read_to_string(out.join("c/out.txt")).unwrap(), + rendered + ); + } + + #[test] + fn stale_files_are_removed_but_user_files_are_kept() { + let dir = tempfile::tempdir().unwrap(); + let out = Utf8Path::from_path(dir.path()).unwrap(); + let calls = Arc::new(AtomicUsize::new(0)); + let c = backend("c", &calls); + let files = Arc::clone(&c.inner.files); + *files.lock().unwrap() = vec!["a.txt", "b.txt"]; + let orch = Orchestrator::new().with_target(&c); + orch.run(&model_named("math"), out).unwrap(); + std::fs::write(out.join("c/user.txt"), "mine").unwrap(); + + *files.lock().unwrap() = vec!["a.txt"]; + let report = orch.run(&model_named("math"), out).unwrap(); + assert_eq!(report.removed, ["c/b.txt"]); + assert_eq!(report.generated, ["c"]); + assert!(out.join("c/a.txt").exists()); + assert!(!out.join("c/b.txt").exists()); + assert!(out.join("c/user.txt").exists()); + } + + #[test] + fn relative_paths_use_forward_slashes() { + let out = Utf8Path::new("gen"); + assert_eq!(relative_path(out, &out.join("c").join("x.h")), "c/x.h"); + } +} diff --git a/crates/weaveffi-gen/src/codegen/writer.rs b/crates/weaveffi-cli/src/codegen/writer.rs similarity index 99% rename from crates/weaveffi-gen/src/codegen/writer.rs rename to crates/weaveffi-cli/src/codegen/writer.rs index 6a4d8745..77652192 100644 --- a/crates/weaveffi-gen/src/codegen/writer.rs +++ b/crates/weaveffi-cli/src/codegen/writer.rs @@ -14,7 +14,7 @@ //! carry trailing whitespace, and the indent unit is fixed per writer. //! //! ``` -//! use weaveffi_gen::codegen::writer::CodeWriter; +//! use weaveffi_cli::codegen::writer::CodeWriter; //! //! let mut w = CodeWriter::new(" "); //! w.line("class Greeter:"); @@ -188,7 +188,7 @@ impl CodeWriter { /// block. /// /// ``` - /// use weaveffi_gen::codegen::writer::CodeWriter; + /// use weaveffi_cli::codegen::writer::CodeWriter; /// let mut w = CodeWriter::four_space(); /// w.block("fn main() {", "}", |w| { /// w.line("println!(\"hi\");"); diff --git a/crates/weaveffi-cli/src/commands/build.rs b/crates/weaveffi-cli/src/commands/build.rs new file mode 100644 index 00000000..0a7359cd --- /dev/null +++ b/crates/weaveffi-cli/src/commands/build.rs @@ -0,0 +1,337 @@ +//! `weaveffi build`: cross-compile the Rust producer for each platform and +//! prebuild the per-language C glue, laying everything out in +//! `{target_dir}/weaveffi//` for `weaveffi package`. + +use camino::Utf8PathBuf; +use miette::{bail, miette, IntoDiagnostic, Result, WrapErr}; +use weaveffi_cli::build::glue::{self, GlueSource}; +use weaveffi_cli::build::{BuildSettings, Builder}; +use weaveffi_cli::codegen::Target; +use weaveffi_cli::platform::{jni_shim_name, node_addon_name, BinarySet, Os, Platform}; +use weaveffi_cli::project::{Project, Source}; +use weaveffi_model::model::Model; + +/// Options for [`cmd_build`]. +pub(crate) struct BuildArgs<'a> { + pub(crate) input: Option<&'a str>, + pub(crate) config: Option<&'a str>, + pub(crate) platforms: Option<&'a str>, + pub(crate) targets: Option<&'a str>, + pub(crate) debug: bool, + pub(crate) manifest_path: Option<&'a str>, + pub(crate) warn: bool, + pub(crate) quiet: bool, +} + +pub(crate) fn cmd_build(args: &BuildArgs<'_>) -> Result<()> { + let project = Project::locate(args.config, args.input, None)?.quiet(args.quiet); + let targets = project.config.select_targets(args.targets)?; + let platforms = resolve_platforms(args.platforms, &project)?; + let settings = project.config.build.settings(args.debug); + let built = build_project( + &project, + &targets, + &platforms, + &settings, + args.manifest_path, + args.warn, + )?; + if !args.quiet { + println!("Built `{}` into {}", built.binaries.lib_name, built.dir); + for nb in &built.binaries.binaries { + let mut files = vec![nb.library.file_name().unwrap_or_default().to_string()]; + for extra in [&nb.staticlib, &nb.node_addon, &nb.jni_shim] + .into_iter() + .flatten() + .filter(|p| **p != nb.library) + { + files.push(extra.file_name().unwrap_or_default().to_string()); + } + println!(" {}: {}", nb.platform.id(), files.join(", ")); + } + } + Ok(()) +} + +/// The platforms to build: `--platforms`, else `[build] platforms`, else +/// the host. +pub(crate) fn resolve_platforms(flag: Option<&str>, project: &Project) -> Result> { + let ids: Vec = match (flag, &project.config.build.platforms) { + (Some(list), _) => list.split(',').map(str::to_string).collect(), + (None, Some(list)) => list.clone(), + (None, None) => { + let host = Platform::host().ok_or_else(|| { + miette!( + "this host ({} {}) isn't one of the packaging platforms; pass --platforms", + std::env::consts::OS, + std::env::consts::ARCH + ) + })?; + return Ok(vec![host]); + } + }; + Platform::parse_list(&ids).map_err(|e| miette!("{e}")) +} + +/// What [`build_project`] produced. +pub(crate) struct Built { + /// The per-platform outputs. + pub(crate) binaries: BinarySet, + /// The directory they were laid out in (`{target_dir}/weaveffi`). + pub(crate) dir: Utf8PathBuf, + /// The validated API. + pub(crate) model: Model, +} + +/// The name of the library a project's bindings load, and its model when +/// that can be read before anything is built. A Rust producer crate's +/// library is named after the crate, and its API is read from the library +/// once it's built; anything else (an IDL) has its model up front. +pub(crate) fn library_name(project: &Project, warn: bool) -> Result<(String, Option)> { + match &project.source { + Source::Crate(krate) => Ok((krate.lib_name.clone(), None)), + Source::Idl(_) | Source::Library(_) => { + let model = super::load_model(project, warn)?; + Ok((model.identity.library.clone(), Some(model))) + } + } +} + +/// The model of a Rust producer whose libraries are `binaries`, read from +/// the first one's metadata. +pub(crate) fn model_from_binaries( + project: &Project, + binaries: &BinarySet, + warn: bool, +) -> Result { + let Some(first) = binaries.binaries.first() else { + bail!("no libraries were built to read the API from"); + }; + super::load_model(&project.clone().library(first.library.clone())?, warn) +} + +/// Build the project's producer for every platform, then prebuild the glue +/// of the selected targets that need it (the Node.js addon for `node`, the +/// JNI shim for `kotlin`). Every platform is checked before anything +/// compiles, so a missing rustup target or NDK fails fast with every +/// problem listed. Glue a host can't compile is skipped with a warning; the +/// packages then fall back to compiling it at install or build time. A Rust +/// producer's API is read from the first library built. +pub(crate) fn build_project( + project: &Project, + targets: &[Box], + platforms: &[Platform], + settings: &BuildSettings, + manifest: Option<&str>, + warn: bool, +) -> Result { + let quiet = project.is_quiet(); + let krate = super::producer_crate(project, manifest)?; + let (library, model) = library_name(project, warn)?; + let library = &library; + if krate.lib_name != *library && !quiet { + eprintln!( + "note: the crate's library `{}` is laid out as `{library}`, the library the \ + bindings load", + krate.lib_name + ); + } + let builder = Builder::new(&krate, library, settings).quiet(quiet); + let problems: Vec = platforms + .iter() + .filter_map(|p| builder.check(*p).err().map(|e| format!(" - {e:#}"))) + .collect(); + if !problems.is_empty() { + bail!( + "can't build every requested platform on this machine:\n{}", + problems.join("\n") + ); + } + + let mut binaries = BinarySet::new(library.as_str()); + for &platform in platforms { + if !quiet { + println!( + "Building `{}` for {} ({})...", + krate.name, + platform.display_name(), + platform.rust_target() + ); + } + let binary = builder.build(platform).map_err(|e| miette!("{e:#}"))?; + binaries.insert(binary); + } + let model = match model { + Some(model) => model, + None => model_from_binaries(project, &binaries, warn)?, + }; + + let dir = krate.weaveffi_dir(); + let wants = |name: &str| targets.iter().any(|t| t.name() == name); + if wants("node") { + prebuild_node(&model, targets, &dir, &mut binaries, settings, quiet)?; + } + if wants("kotlin") { + prebuild_jni( + &model, + targets, + &dir, + &mut binaries, + &builder, + settings, + quiet, + )?; + } + Ok(Built { + binaries, + dir, + model, + }) +} + +/// Render `target_name`'s files in memory and stage its glue source and the +/// C header under `{dir}/.glue/{target_name}/`. +fn stage_glue( + model: &Model, + targets: &[Box], + target_name: &str, + source_name: &str, + dir: &Utf8PathBuf, +) -> Result { + let target = targets + .iter() + .find(|t| t.name() == target_name) + .expect("the target is selected"); + let header_name = format!("{}.h", model.identity.library); + let staging = dir.join(".glue").join(target_name); + std::fs::create_dir_all(staging.as_std_path()) + .into_diagnostic() + .wrap_err_with(|| format!("failed to create {staging}"))?; + let files = target.render(model, &staging); + for wanted in [source_name, header_name.as_str()] { + let file = files + .iter() + .find(|f| f.path.file_name() == Some(wanted)) + .ok_or_else(|| miette!("the {target_name} target rendered no {wanted}"))?; + std::fs::write(staging.join(wanted).as_std_path(), &file.contents) + .into_diagnostic() + .wrap_err_with(|| format!("failed to stage {wanted}"))?; + } + Ok(GlueSource { + source: staging.join(source_name), + include_dir: staging, + }) +} + +fn warn_skip(quiet: bool, what: &str, platform: Platform, reason: &str) { + if !quiet { + eprintln!("warning: skipped {what} for {}: {reason}", platform.id()); + } +} + +/// Prebuild the Node.js addon for every desktop platform this host can +/// compile for. +fn prebuild_node( + model: &Model, + targets: &[Box], + dir: &Utf8PathBuf, + binaries: &mut BinarySet, + settings: &BuildSettings, + quiet: bool, +) -> Result<()> { + let library = binaries.lib_name.clone(); + let desktop: Vec = binaries.platforms().filter(|p| p.is_desktop()).collect(); + if desktop.is_empty() { + return Ok(()); + } + let glue_source = stage_glue( + model, + targets, + "node", + &format!("{}.c", node_addon_name(&library)), + dir, + )?; + let headers = match glue::node_headers() { + Ok(headers) => headers, + Err(e) => { + for p in desktop { + warn_skip(quiet, "the Node.js addon", p, &format!("{e:#}")); + } + return Ok(()); + } + }; + for platform in desktop { + let platform_dir = dir.join(platform.id()); + match glue::compile_node_addon( + &glue_source, + platform, + &platform_dir, + &library, + &headers, + settings, + ) { + Ok(addon) => { + if let Some(nb) = binaries + .binaries + .iter_mut() + .find(|b| b.platform == platform) + { + nb.node_addon = Some(addon); + } + } + Err(e) => warn_skip(quiet, "the Node.js addon", platform, &format!("{e:#}")), + } + } + Ok(()) +} + +/// Prebuild the JNI shim for every Android ABI (with the NDK) and every +/// desktop platform this host can compile for (with the JDK's headers). +fn prebuild_jni( + model: &Model, + targets: &[Box], + dir: &Utf8PathBuf, + binaries: &mut BinarySet, + builder: &Builder<'_>, + settings: &BuildSettings, + quiet: bool, +) -> Result<()> { + let library = binaries.lib_name.clone(); + let platforms: Vec = binaries + .platforms() + .filter(|p| p.is_desktop() || p.os() == Os::Android) + .collect(); + if platforms.is_empty() { + return Ok(()); + } + let shim = jni_shim_name(&library); + let glue_source = stage_glue(model, targets, "kotlin", &format!("{shim}.c"), dir)?; + for platform in platforms { + let ndk = if platform.os() == Os::Android { + Some(builder.ndk().map_err(|e| miette!("{e:#}"))?) + } else { + None + }; + let platform_dir = dir.join(platform.id()); + match glue::compile_jni_shim( + &glue_source, + platform, + &platform_dir, + &library, + ndk, + settings, + ) { + Ok(path) => { + if let Some(nb) = binaries + .binaries + .iter_mut() + .find(|b| b.platform == platform) + { + nb.jni_shim = Some(path); + } + } + Err(e) => warn_skip(quiet, "the JNI shim", platform, &format!("{e:#}")), + } + } + Ok(()) +} diff --git a/crates/weaveffi-cli/src/commands/dev.rs b/crates/weaveffi-cli/src/commands/dev.rs new file mode 100644 index 00000000..6a508ba9 --- /dev/null +++ b/crates/weaveffi-cli/src/commands/dev.rs @@ -0,0 +1,132 @@ +//! `weaveffi dev`: the inner loop for a Rust producer. +//! +//! It builds the crate's debug library, generates the bindings from it, and +//! then makes the generated packages find that library without any setup +//! where a package looks for a bundled copy: the library is copied into the +//! Python package, next to the Node.js package's `binding.gyp` (which links +//! and loads it from there), and into the Ruby gem's `lib/native/`. For the +//! other targets it prints the one environment variable to set (or, for +//! targets that link at build time, the directory to link from). + +use camino::{Utf8Path, Utf8PathBuf}; +use miette::{bail, IntoDiagnostic, Result, WrapErr}; + +use super::generate::{generate, report_summary}; +use super::Locate; + +/// Options for [`cmd_dev`]. +pub(crate) struct DevArgs<'a> { + pub(crate) locate: Locate<'a>, + pub(crate) out: Option<&'a str>, + pub(crate) targets: Option<&'a str>, +} + +pub(crate) fn cmd_dev(args: &DevArgs<'_>) -> Result<()> { + let project = args.locate.project()?; + let Some(krate) = project.krate() else { + bail!( + "`weaveffi dev` builds a Rust producer crate, but the project's input is an IDL; \ + use `weaveffi generate`" + ); + }; + let name = krate.name.clone(); + let library = project.library_path()?; + let library = std::path::absolute(library.as_std_path()) + .ok() + .and_then(|p| Utf8PathBuf::from_path_buf(p).ok()) + .unwrap_or(library); + let project = project.library(library.clone())?; + let model = super::load_model(&project, false)?; + let out_dir = project.config.out_dir(args.out); + let quiet = args.locate.quiet; + if let Some(report) = generate(&project, &model, &out_dir, args.targets, false)? { + if !quiet { + println!("{}", report_summary(&report, &out_dir)); + } + } + + let shared = matches!(library.extension(), Some("so" | "dylib" | "dll")); + let mut placed = Vec::new(); + let mut by_env = Vec::new(); + let mut by_link = Vec::new(); + for target in project.config.select_targets(args.targets)? { + let files = target.render(&model, &out_dir); + let file_dir = |suffix: &str| { + files + .iter() + .find(|f| f.path.as_str().ends_with(suffix)) + .and_then(|f| f.path.parent().map(Utf8Path::to_path_buf)) + }; + let dest = match target.name() { + "python" => file_dir("/__init__.py"), + "node" => file_dir("/binding.gyp"), + "ruby" => file_dir("runtime.rb").and_then(|d| d.parent().map(|p| p.join("native"))), + "c" | "cpp" | "swift" | "go" => { + by_link.push(target.name()); + None + } + "wasm" => None, + other => { + by_env.push(other); + None + } + }; + match dest { + Some(dir) if shared => { + copy_into(&library, &dir)?; + placed.push(dir); + } + Some(_) | None => { + if matches!(target.name(), "python" | "node" | "ruby") { + by_env.push(target.name()); + } + } + } + } + + if quiet { + return Ok(()); + } + println!("Built `{name}`: {library}"); + for dir in &placed { + println!(" copied into {dir}"); + } + let env = model.identity.library_env_var(); + if !by_env.is_empty() { + println!(" {}: export {env}={library}", by_env.join(", ")); + } + if !by_link.is_empty() { + let dir = library.parent().unwrap_or(Utf8Path::new(".")); + let path_var = match std::env::consts::OS { + "macos" => "DYLD_LIBRARY_PATH", + "windows" => "PATH", + _ => "LD_LIBRARY_PATH", + }; + println!( + " {}: link with -L{dir} -l{} and run with {path_var}={dir}", + by_link.join(", "), + model.identity.library + ); + } + if project + .config + .select_targets(args.targets)? + .iter() + .any(|t| t.name() == "wasm") + { + println!(" wasm: needs a wasm32 build: `weaveffi build --platforms wasm32`"); + } + Ok(()) +} + +/// Copy `library` into `dir`, replacing an older copy. +fn copy_into(library: &Utf8Path, dir: &Utf8Path) -> Result<()> { + std::fs::create_dir_all(dir.as_std_path()) + .into_diagnostic() + .wrap_err_with(|| format!("failed to create {dir}"))?; + let dest = dir.join(library.file_name().unwrap_or_default()); + std::fs::copy(library.as_std_path(), dest.as_std_path()) + .into_diagnostic() + .wrap_err_with(|| format!("failed to copy {library} to {dest}"))?; + Ok(()) +} diff --git a/crates/weaveffi-cli/src/commands/diff.rs b/crates/weaveffi-cli/src/commands/diff.rs index e06bae8f..c6a8dad7 100644 --- a/crates/weaveffi-cli/src/commands/diff.rs +++ b/crates/weaveffi-cli/src/commands/diff.rs @@ -1,5 +1,5 @@ //! `weaveffi diff`: show how regenerating would change an existing output -//! directory, without writing anything or running hooks. `--check` turns it +//! directory, without writing anything. `--check` turns it //! into a CI gate. use std::collections::{BTreeMap, BTreeSet}; @@ -7,17 +7,15 @@ use std::collections::{BTreeMap, BTreeSet}; use camino::Utf8Path; use miette::{IntoDiagnostic, Result}; use similar::TextDiff; -use weaveffi_gen::cache; -use weaveffi_gen::codegen::relative_path; +use weaveffi_cli::cache; +use weaveffi_cli::codegen::relative_path; /// Options for [`cmd_diff`]. pub(crate) struct DiffArgs<'a> { - pub(crate) input: Option<&'a str>, + pub(crate) locate: super::Locate<'a>, pub(crate) out: Option<&'a str>, pub(crate) targets: Option<&'a str>, - pub(crate) config: Option<&'a str>, pub(crate) check: bool, - pub(crate) quiet: bool, } /// Exit status of `diff --check` when files differ. @@ -26,14 +24,15 @@ const EXIT_MODIFIED: i32 = 2; const EXIT_ADDED_OR_REMOVED: i32 = 3; pub(crate) fn cmd_diff(args: &DiffArgs<'_>) -> Result<()> { - let project = super::load_project(args.input, args.config, false)?; + let project = args.locate.project()?; + let model = super::load_model(&project, false)?; let out_dir = project.config.out_dir(args.out); let targets = project.config.select_targets(args.targets)?; let mut generated: BTreeMap = BTreeMap::new(); let mut existing: BTreeSet = BTreeSet::new(); for target in &targets { - for file in target.render(&project.api, &out_dir) { + for file in target.render(&model, &out_dir) { generated.insert(relative_path(&out_dir, &file.path), file.contents); } // Files a previous generation recorded are the generator's; without a @@ -42,7 +41,7 @@ pub(crate) fn cmd_diff(args: &DiffArgs<'_>) -> Result<()> { Some(record) => existing.extend( record .files - .into_keys() + .into_iter() .filter(|rel| out_dir.join(rel).exists()), ), None => collect_files(&out_dir, &out_dir.join(target.name()), &mut existing)?, @@ -87,7 +86,7 @@ pub(crate) fn cmd_diff(args: &DiffArgs<'_>) -> Result<()> { if modified > 0 { std::process::exit(EXIT_MODIFIED); } - } else if added == 0 && removed == 0 && modified == 0 && !args.quiet { + } else if added == 0 && removed == 0 && modified == 0 && !args.locate.quiet { println!("No differences found."); } Ok(()) diff --git a/crates/weaveffi-cli/src/commands/extract.rs b/crates/weaveffi-cli/src/commands/extract.rs new file mode 100644 index 00000000..c8832ca7 --- /dev/null +++ b/crates/weaveffi-cli/src/commands/extract.rs @@ -0,0 +1,58 @@ +//! `weaveffi extract`: print the IDL a Rust producer's built library +//! embeds. +//! +//! The library is the project's crate, built here (`--release` for a +//! release build), or the one `--library` names. The document is the API +//! exactly as `weaveffi generate` would read it, validated against the +//! crate's identity first, so what it prints always generates. + +use miette::{miette, IntoDiagnostic, Report, Result, WrapErr}; +use weaveffi_cli::project::Source; + +use crate::IdlFormat; + +/// Extract the project's API and write it as `format` to `output` (else +/// stdout). +pub(crate) fn cmd_extract( + locate: &super::Locate<'_>, + output: Option<&str>, + format: IdlFormat, +) -> Result<()> { + let project = locate.project()?; + if let Source::Idl(idl) = &project.source { + return Err(miette!( + "{idl} is already an IDL; `weaveffi extract` reads a Rust producer's API from its \ + built library (pass the crate, or `--library`)" + )); + } + let definition = project.definition()?; + definition.validate().map_err(Report::new)?; + let api = &definition.api; + let serialized = match format { + IdlFormat::Yaml => serde_yaml::to_string(api) + .into_diagnostic() + .wrap_err("failed to serialize API as YAML")?, + IdlFormat::Json => { + let mut json = serde_json::to_string_pretty(api) + .into_diagnostic() + .wrap_err("failed to serialize API as JSON")?; + json.push('\n'); + json + } + IdlFormat::Toml => toml::to_string_pretty(api) + .into_diagnostic() + .wrap_err("failed to serialize API as TOML")?, + }; + match output { + Some(path) => { + std::fs::write(path, &serialized) + .into_diagnostic() + .wrap_err_with(|| format!("failed to write output file: {path}"))?; + if !locate.quiet { + println!("Extracted API written to {path}"); + } + } + None => print!("{serialized}"), + } + Ok(()) +} diff --git a/crates/weaveffi-cli/src/commands/generate.rs b/crates/weaveffi-cli/src/commands/generate.rs index 834f40ef..1bf97c86 100644 --- a/crates/weaveffi-cli/src/commands/generate.rs +++ b/crates/weaveffi-cli/src/commands/generate.rs @@ -3,32 +3,51 @@ use camino::Utf8Path; use miette::{miette, IntoDiagnostic, Result, WrapErr}; -use weaveffi_gen::codegen::{relative_path, GenerateReport, Hook, Orchestrator}; +use weaveffi_cli::codegen::{relative_path, GenerateReport, Orchestrator}; +use weaveffi_cli::project::Project; +use weaveffi_model::model::Model; + +use super::Locate; /// Options for [`cmd_generate`]. pub(crate) struct GenerateArgs<'a> { - pub(crate) input: Option<&'a str>, + pub(crate) locate: Locate<'a>, pub(crate) out: Option<&'a str>, pub(crate) targets: Option<&'a str>, - pub(crate) config: Option<&'a str>, pub(crate) warn: bool, - pub(crate) force: bool, pub(crate) dry_run: bool, - pub(crate) quiet: bool, } pub(crate) fn cmd_generate(args: &GenerateArgs<'_>) -> Result<()> { - let project = super::load_project(args.input, args.config, args.warn)?; + let project = args.locate.project()?; + let model = super::load_model(&project, args.warn)?; let out_dir = project.config.out_dir(args.out); - let selected = project.config.select_targets(args.targets)?; + let report = generate(&project, &model, &out_dir, args.targets, args.dry_run)?; + if let Some(report) = report { + if !args.locate.quiet { + println!("{}", report_summary(&report, &out_dir)); + } + } + Ok(()) +} - if args.dry_run { +/// Generate the selected targets into `out_dir` (or, with `dry_run`, list +/// the files that would be written), returning the run's report. +pub(crate) fn generate( + project: &Project, + model: &Model, + out_dir: &Utf8Path, + targets: Option<&str>, + dry_run: bool, +) -> Result> { + let selected = project.config.select_targets(targets)?; + if dry_run { for target in &selected { - for file in target.render(&project.api, &out_dir) { - println!("{}", relative_path(&out_dir, &file.path)); + for file in target.render(model, out_dir) { + println!("{}", relative_path(out_dir, &file.path)); } } - return Ok(()); + return Ok(None); } std::fs::create_dir_all(out_dir.as_std_path()) @@ -39,33 +58,14 @@ pub(crate) fn cmd_generate(args: &GenerateArgs<'_>) -> Result<()> { for target in &selected { orchestrator = orchestrator.with_target(target.as_ref()); } - let global = &project.config.global; let report = orchestrator - .run(&project.api, &out_dir, args.force, &mut |hook| { - let cmd = match hook { - Hook::BeforeWrite => global.pre_generate.as_deref(), - Hook::AfterWrite => global.post_generate.as_deref(), - }; - match cmd { - Some(cmd) => { - super::run_hook(&format!("{hook:?}"), cmd).map_err(|e| anyhow::anyhow!("{e}")) - } - None => Ok(()), - } - }) + .run(model, out_dir) .map_err(|e| miette!("{:#}", e))?; - - for w in &report.warnings { - eprintln!("warning: {w}"); - } - if !args.quiet { - println!("{}", report_summary(&report, &out_dir)); - } - Ok(()) + Ok(Some(report)) } /// One line describing what a generation run did. -fn report_summary(report: &GenerateReport, out_dir: &Utf8Path) -> String { +pub(crate) fn report_summary(report: &GenerateReport, out_dir: &Utf8Path) -> String { if report.generated.is_empty() { return format!( "{out_dir} is up to date ({} targets)", @@ -97,19 +97,19 @@ mod tests { let yml = dir.path().join("api.yml"); std::fs::write( &yml, - "version: \"0.10.0\"\nmodules:\n - name: math\n functions:\n - { name: add, params: [{ name: a, type: i32 }], return: i32 }\n", + "version: \"0.11.0\"\nmodules:\n - name: math\n functions:\n - { name: add, params: [{ name: a, type: i32 }], return: i32 }\n", ) .unwrap(); let out = dir.path().join("out"); cmd_generate(&GenerateArgs { - input: yml.to_str(), + locate: Locate { + input: yml.to_str(), + ..Locate::default() + }, out: out.to_str(), targets: Some("c"), - config: None, warn: false, - force: false, dry_run: true, - quiet: false, }) .unwrap(); assert!(!out.exists(), "dry-run should not create output directory"); diff --git a/crates/weaveffi-cli/src/commands/init.rs b/crates/weaveffi-cli/src/commands/init.rs index cc113388..70757789 100644 --- a/crates/weaveffi-cli/src/commands/init.rs +++ b/crates/weaveffi-cli/src/commands/init.rs @@ -1,9 +1,10 @@ //! `weaveffi init`: set up a project so a bare `weaveffi generate` works. //! //! In a Rust crate (a directory with a `Cargo.toml` that has a `[package]`), -//! it writes a `weaveffi.toml` whose `[project] input` is the crate's -//! `src/lib.rs`, then checks the three things a producer needs (a `cdylib` -//! crate type, a `weaveffi` dependency, and one `weaveffi::export_runtime!()` +//! it writes a `weaveffi.toml` whose `[project] input` is the crate itself +//! (`"."`), whose API WeaveFFI reads from the library the crate builds, then +//! checks the things a producer needs (a `cdylib` crate type, a `weaveffi` +//! dependency, a `#[weaveffi::module]`, and one `weaveffi::export_runtime!()` //! call) and prints any that are missing. It never edits `Cargo.toml` or //! source files. Anywhere else, it writes a starter IDL plus a //! `weaveffi.toml` that points at it. @@ -12,7 +13,7 @@ use camino::{Utf8Path, Utf8PathBuf}; use miette::{bail, IntoDiagnostic, Result, WrapErr}; use weaveffi_model::pkg::c_ident; -use crate::config::CONFIG_FILE_NAME; +use weaveffi_cli::config::CONFIG_FILE_NAME; /// Options for [`cmd_init`]. pub(crate) struct InitArgs<'a> { @@ -58,7 +59,7 @@ fn init_rust(dir: &Utf8Path, config_path: &Utf8Path, crate_name: &str, quiet: bo config_path, "# WeaveFFI project configuration. See https://weaveffi.com/guides/config.html\n\ [project]\n\ - input = \"src/lib.rs\"\n\ + input = \".\"\n\ out = \"bindings\"\n\ # targets = [\"c\", \"swift\", \"kotlin\", \"python\"]\n", )?; @@ -72,7 +73,8 @@ fn init_rust(dir: &Utf8Path, config_path: &Utf8Path, crate_name: &str, quiet: bo let mut todo = Vec::new(); if !manifest.contains("cdylib") { todo.push( - "add `crate-type = [\"cdylib\"]` (and \"staticlib\" for iOS) under [lib] in Cargo.toml", + "add `crate-type = [\"cdylib\"]` under [lib] in Cargo.toml so `cargo build` \ + produces the shared library (`weaveffi build` picks the right kinds itself)", ); } if !manifest.contains("weaveffi") { @@ -167,12 +169,21 @@ mod tests { quiet: true, }) .unwrap(); - let (cfg, input) = - crate::config::ProjectConfig::locate(Some(root.join(CONFIG_FILE_NAME).as_str()), None) - .unwrap(); - assert_eq!(input, root.join("greeter.yml")); - assert_eq!(cfg.package.name.as_deref(), Some("greeter")); - crate::commands::load_validated_api(input.as_str()).unwrap(); + let project = weaveffi_cli::project::Project::locate( + Some(root.join(CONFIG_FILE_NAME).as_str()), + None, + None, + ) + .unwrap(); + assert!(matches!( + &project.source, + weaveffi_cli::project::Source::Idl(p) if *p == root.join("greeter.yml") + )); + assert_eq!( + project.config.package.identity.name.as_deref(), + Some("greeter") + ); + project.model().unwrap(); let again = cmd_init(&InitArgs { dir: root.as_str(), name: None, @@ -183,7 +194,7 @@ mod tests { } #[test] - fn rust_project_points_at_lib_rs() { + fn rust_project_points_at_the_crate() { let dir = tempfile::tempdir().unwrap(); let root = Utf8Path::from_path(dir.path()).unwrap(); std::fs::write(root.join("Cargo.toml"), "[package]\nname = \"kv\"\n").unwrap(); @@ -194,10 +205,8 @@ mod tests { quiet: true, }) .unwrap(); - let cfg = crate::config::ProjectConfig::from_file(&root.join(CONFIG_FILE_NAME)).unwrap(); - assert_eq!( - cfg.project.input.as_deref(), - Some(Utf8Path::new("src/lib.rs")) - ); + let cfg = + weaveffi_cli::config::ProjectConfig::from_file(&root.join(CONFIG_FILE_NAME)).unwrap(); + assert_eq!(cfg.project.input.as_deref(), Some(Utf8Path::new("."))); } } diff --git a/crates/weaveffi-cli/src/commands/mod.rs b/crates/weaveffi-cli/src/commands/mod.rs index c2ae44b0..e260bf01 100644 --- a/crates/weaveffi-cli/src/commands/mod.rs +++ b/crates/weaveffi-cli/src/commands/mod.rs @@ -1,293 +1,79 @@ //! One module per subcommand. `main.rs` holds only argument parsing and -//! dispatch; everything a command does lives here. +//! dispatch; everything a command does lives here, on top of the library's +//! [`Project`]. +pub(crate) mod build; +pub(crate) mod dev; pub(crate) mod diff; +pub(crate) mod extract; pub(crate) mod generate; pub(crate) mod init; pub(crate) mod package; pub(crate) mod validate; -use crate::config::ProjectConfig; -use crate::report::with_named_source; use camino::{Utf8Path, Utf8PathBuf}; -use miette::{bail, IntoDiagnostic, Report, Result, WrapErr}; -use weaveffi_model::ir::Api; -use weaveffi_model::parse::parse_api_str; -use weaveffi_model::pkg::{name_from_basename, Identity, Package}; -use weaveffi_model::validate::{collect_warnings, validate_api}; -use weaveffi_model::ResolvedApi; - -/// Map the input file extension onto the parser's format token. -pub(crate) fn input_format(in_path: &Utf8Path) -> Result<&'static str> { - let ext = in_path.extension().unwrap_or(""); - if ext.is_empty() { - bail!("input file has no extension (expected rs|yml|yaml|json|toml)"); - } - match ext { - "yml" | "yaml" => Ok("yaml"), - "json" => Ok("json"), - "toml" => Ok("toml"), - other => Err(miette::miette!( - "unsupported input format: {} (expected rs|yml|yaml|json|toml)", - other - )), - } +use miette::{bail, Result}; +use weaveffi_cli::cargo::CargoCrate; +use weaveffi_cli::project::{Project, Source}; +use weaveffi_model::model::Model; + +/// How a command finds its project: the positional input, `--config`, and +/// `--library`. +#[derive(Clone, Copy, Default)] +pub(crate) struct Locate<'a> { + pub(crate) input: Option<&'a str>, + pub(crate) config: Option<&'a str>, + pub(crate) library: Option<&'a str>, + pub(crate) release: bool, + pub(crate) quiet: bool, } -/// Read and parse the API at `input` without validating it. Returns the parsed -/// [`Api`] and the raw file contents (for snippet-rendered diagnostics). -/// -/// A `.rs` input is treated as annotated Rust source and lowered to the IR -/// through [`weaveffi_model::rust`] (the same extraction the `#[weaveffi::module]` -/// macro uses), so generating from a producer's source and building that -/// producer cannot drift. Any other extension is parsed as an IDL document -/// (yaml/json/toml). -pub(crate) fn load_api(input: &str) -> Result<(Api, String)> { - let in_path = Utf8Path::new(input); - let contents = std::fs::read_to_string(in_path.as_std_path()) - .into_diagnostic() - .wrap_err_with(|| format!("failed to read input file: {}", input))?; - if in_path.extension() == Some("rs") { - let api = weaveffi_model::rust::api_from_src_stringly(&contents) - .map_err(|e| miette::miette!("failed to extract API from Rust source {input}:\n{e}"))?; - return Ok((api, contents)); +impl Locate<'_> { + /// Locate the project. + pub(crate) fn project(&self) -> Result { + Ok(Project::locate(self.config, self.input, self.library)? + .release(self.release) + .quiet(self.quiet)) } - let format = input_format(in_path)?; - let api = - parse_api_str(&contents, format).map_err(|e| with_named_source(e, input, &contents))?; - Ok((api, contents)) -} - -/// [`load_api`] plus validation. Returns the [`ResolvedApi`] that every -/// downstream consumer (orchestrator, packagers) requires. -pub(crate) fn load_validated_api(input: &str) -> Result<(ResolvedApi, String)> { - let (api, contents) = load_api(input)?; - let api = validate_api(api, Some((input, &contents))).map_err(Report::new)?; - Ok((api, contents)) } -/// A loaded project: its configuration, the input it was loaded from, and -/// the validated API with the library identity attached. -pub(crate) struct Project { - pub(crate) config: ProjectConfig, - pub(crate) input: Utf8PathBuf, - pub(crate) api: ResolvedApi, -} - -/// The shared front half of `generate`, `package`, and `diff`: locate the -/// project config and input (an explicit input, or `[project] input` from the -/// nearest `weaveffi.toml`), load and validate the API, attach the library's -/// [`Identity`], and optionally print advisory warnings. -pub(crate) fn load_project( - input: Option<&str>, - config_path: Option<&str>, - warn: bool, -) -> Result { - let (config, input) = ProjectConfig::locate(config_path, input)?; - let identity = resolve_identity(&input, &config.package)?; - let (api, _contents) = load_validated_api(input.as_str())?; - let api = api.with_identity(identity); +/// Load the project's validated [`Model`], printing advisory warnings with +/// `warn`. +pub(crate) fn load_model(project: &Project, warn: bool) -> Result { + let definition = project.definition()?; + let model = definition.validate().map_err(miette::Report::new)?; if warn { - for w in collect_warnings(&api) { + for w in definition.warnings() { eprintln!("warning: {w}"); } } - Ok(Project { config, input, api }) -} - -/// Run one of the project's `[global]` shell hooks (`sh -c` or `cmd /C`). -pub(crate) fn run_hook(label: &str, cmd: &str) -> Result<()> { - let status = if cfg!(target_os = "windows") { - std::process::Command::new("cmd").args(["/C", cmd]).status() - } else { - std::process::Command::new("sh").arg("-c").arg(cmd).status() - } - .into_diagnostic() - .wrap_err_with(|| format!("failed to run {label} hook"))?; - if !status.success() { - bail!("{label} hook `{cmd}` failed with {status}"); - } - Ok(()) + Ok(model) } -/// Resolve the library's identity. -/// -/// A Rust producer (`.rs` input) is named after its crate: the nearest -/// `Cargo.toml` at or above the input supplies the package name and metadata, -/// and the crate's library name (`[lib] name`, else the package name with `-` -/// mapped to `_`) is both the C symbol prefix the `#[weaveffi::module]` macro -/// uses and the file name Cargo gives the cdylib. `weaveffi.toml` may rename -/// the published package and override metadata, but not the prefix or the -/// library. An IDL input is named by `[package] name`, else its file stem. -pub(crate) fn resolve_identity(input: &Utf8Path, package: &Package) -> Result { - if input.extension() != Some("rs") { - let stem = name_from_basename(input.as_str()); - return Ok(Identity::new(&stem, package)); - } - if package.c_prefix.is_some() || package.library.is_some() { - bail!( - "`[package] c_prefix` and `[package] library` apply only to IDL inputs: a Rust \ - producer's C prefix and library name are its crate's library name" - ); - } - let Some(cargo) = CargoPackage::find(input)? else { - // No manifest: name a `src/lib.rs`-style input after its crate - // directory, as Cargo would, rather than after the file stem. - let stem = crate_dir_name(input).unwrap_or_else(|| name_from_basename(input.as_str())); - return Ok(Identity::new(&stem, package)); +/// The project's Rust producer crate, resolved with `cargo metadata`: the +/// manifest named by `manifest` (`--manifest-path`), else `[build] +/// manifest`, else the project's own crate, else the `Cargo.toml` next to +/// `weaveffi.toml` (or, without one, next to the IDL). +pub(crate) fn producer_crate(project: &Project, manifest: Option<&str>) -> Result { + let config = &project.config; + let path = match (manifest, &config.build.manifest, &project.source) { + (Some(m), _, _) => Utf8PathBuf::from(m), + (None, Some(m), _) => config.relative_to_config(m), + (None, None, Source::Crate(krate)) => return Ok(krate.as_ref().clone()), + (None, None, Source::Idl(idl)) => match config.source.as_deref().and_then(Utf8Path::parent) + { + Some(dir) => dir.join("Cargo.toml"), + None => idl.parent().unwrap_or(Utf8Path::new("")).join("Cargo.toml"), + }, + (None, None, Source::Library(lib)) => lib.with_file_name("Cargo.toml"), }; - let mut merged = package.clone(); - merged.version = merged.version.or(cargo.version); - merged.description = merged.description.or(cargo.description); - merged.license = merged.license.or(cargo.license); - merged.homepage = merged.homepage.or(cargo.homepage); - merged.repository = merged.repository.or(cargo.repository); - if merged.authors.is_empty() { - merged.authors = cargo.authors; - } - merged.c_prefix = Some(cargo.lib_name.clone()); - merged.library = Some(cargo.lib_name); - Ok(Identity::new(&cargo.name, &merged)) -} - -/// For a Rust input named `lib.rs`, `main.rs`, or `mod.rs`, the enclosing -/// crate directory's name (skipping a `src/` component). -fn crate_dir_name(input: &Utf8Path) -> Option { - if !matches!(input.file_stem(), Some("lib" | "main" | "mod")) { - return None; - } - let abs = std::path::absolute(input.as_std_path()).ok()?; - let mut dir = abs.parent()?; - if dir.file_name().is_some_and(|n| n == "src") { - dir = dir.parent()?; - } - dir.file_name().map(|n| n.to_string_lossy().into_owned()) -} - -/// The parts of a crate's `Cargo.toml` that name a Rust producer. -#[derive(Debug, Default, PartialEq)] -struct CargoPackage { - name: String, - lib_name: String, - version: Option, - description: Option, - license: Option, - homepage: Option, - repository: Option, - authors: Vec, -} - -impl CargoPackage { - /// Find and read the nearest `Cargo.toml` with a `[package]` table at or - /// above `input`. - fn find(input: &Utf8Path) -> Result> { - let start = std::path::absolute(input.as_std_path()).into_diagnostic()?; - for dir in start.ancestors().skip(1) { - let manifest = dir.join("Cargo.toml"); - if !manifest.is_file() { - continue; - } - let text = std::fs::read_to_string(&manifest) - .into_diagnostic() - .wrap_err_with(|| format!("failed to read {}", manifest.display()))?; - if let Some(pkg) = Self::parse(&text) - .wrap_err_with(|| format!("failed to parse {}", manifest.display()))? - { - return Ok(Some(pkg)); - } - } - Ok(None) - } - - /// Parse a manifest; `None` for a virtual (workspace-only) manifest. - fn parse(text: &str) -> Result> { - let doc: toml::Table = toml::from_str(text).into_diagnostic()?; - let Some(pkg) = doc.get("package").and_then(toml::Value::as_table) else { - return Ok(None); - }; - let field = |key: &str| { - pkg.get(key) - .and_then(toml::Value::as_str) - .map(str::to_string) - }; - let Some(name) = field("name") else { - bail!("[package] has no name"); - }; - let lib_name = doc - .get("lib") - .and_then(|l| l.get("name")) - .and_then(toml::Value::as_str) - .map_or_else(|| name.replace('-', "_"), str::to_string); - let authors = pkg - .get("authors") - .and_then(toml::Value::as_array) - .map(|a| { - a.iter() - .filter_map(|v| v.as_str().map(str::to_string)) - .collect() - }) - .unwrap_or_default(); - Ok(Some(Self { - lib_name, - version: field("version"), - description: field("description"), - license: field("license"), - homepage: field("homepage"), - repository: field("repository"), - authors, - name, - })) - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn cargo_manifest_names_the_producer() { - let pkg = CargoPackage::parse( - "[package]\nname = \"my-kv\"\nversion = \"1.2.3\"\nlicense = \"MIT\"\n", - ) - .unwrap() - .unwrap(); - assert_eq!(pkg.name, "my-kv"); - assert_eq!(pkg.lib_name, "my_kv"); - assert_eq!(pkg.version.as_deref(), Some("1.2.3")); - let renamed = CargoPackage::parse("[package]\nname = \"kv\"\n[lib]\nname = \"kvcore\"\n") - .unwrap() - .unwrap(); - assert_eq!(renamed.lib_name, "kvcore"); - assert!(CargoPackage::parse("[workspace]\nmembers = []\n") - .unwrap() - .is_none()); - } - - #[test] - fn sample_producer_identity_comes_from_its_crate() { - let root = - Utf8Path::new(env!("CARGO_MANIFEST_DIR")).join("../../samples/kvstore/src/lib.rs"); - let id = resolve_identity(&root, &Package::default()).unwrap(); - assert_eq!(id.name, "kvstore"); - assert_eq!(id.prefix, "kvstore"); - assert_eq!(id.library, "kvstore"); - } - - #[test] - fn idl_identity_comes_from_the_file_stem_or_package() { - let id = - resolve_identity(Utf8Path::new("api/kitchen_sink.yml"), &Package::default()).unwrap(); - assert_eq!(id.prefix, "kitchen_sink"); - let pkg = Package { - name: Some("acme-kv".into()), - c_prefix: Some("akv".into()), - ..Package::default() - }; - let id = resolve_identity(Utf8Path::new("api/kv.yml"), &pkg).unwrap(); - assert_eq!( - (id.name.as_str(), id.prefix.as_str(), id.library.as_str()), - ("acme-kv", "akv", "acme_kv") + if !path.is_file() { + bail!( + "no Cargo.toml at {path}: `weaveffi build` compiles a Rust producer crate. Point at \ + it with `--manifest-path` or `[build] manifest`, or, for a producer built another \ + way, lay its libraries out as // and pass `weaveffi package \ + --binaries `" ); - assert!(resolve_identity(Utf8Path::new("src/lib.rs"), &pkg).is_err()); } + CargoCrate::resolve(&path).map_err(|e| miette::miette!("{e:#}")) } diff --git a/crates/weaveffi-cli/src/commands/package.rs b/crates/weaveffi-cli/src/commands/package.rs index 918c0a7a..4be5990a 100644 --- a/crates/weaveffi-cli/src/commands/package.rs +++ b/crates/weaveffi-cli/src/commands/package.rs @@ -1,209 +1,259 @@ -//! `weaveffi package`: assemble publishable, per-platform packages that bundle -//! a prebuilt native library for each target platform. +//! `weaveffi package`: turn the per-platform builds into installable +//! artifacts, one set per target, in the dist directory. //! -//! `generate` emits binding *source*; `package` goes one step further and -//! produces ready-to-publish ecosystem packages (an npm tarball tree with -//! `optionalDependencies`, a NuGet `runtimes/` project, platform-tagged Python -//! wheels, …) with the native library bundled so consumers need no local -//! toolchain. The native libraries come from one of two sources: -//! -//! * `--binaries `: prebuilt libraries laid out as `//` -//! (the platform tokens are [`Platform::id`] values, e.g. `darwin-arm64`). -//! This is the path CI uses, building each platform on its own runner. -//! * `--build `: cross-compile the given Cargo package as a `cdylib` -//! for each platform's Rust target triple. Convenient locally, but each -//! target needs its rustup target and a working cross-linker installed. +//! The builds come from `weaveffi build` (run here first, for the requested +//! platforms) or, with `--binaries `, from a directory laid out the same +//! way, `//`, which is how CI hands libraries built on +//! separate runners to one packaging job. Each target then writes its +//! ecosystem's artifacts: wheels, npm tarballs, gems, a SwiftPM package with +//! its `XCFramework` archive, a NuGet package, and so on. + +use std::process::{Command, Stdio}; use camino::{Utf8Path, Utf8PathBuf}; use miette::{bail, miette, IntoDiagnostic, Result, WrapErr}; -use std::process::{Command, Stdio}; -use weaveffi_gen::package::{summarize, write_package, PackageContext}; -use weaveffi_gen::platform::{BinarySet, Os, Platform}; +use weaveffi_cli::package::{ + archive, write_artifact, Artifact, PackageContext, PackagedFile, XcframeworkArchive, +}; +use weaveffi_cli::platform::{BinarySet, Os, Platform}; -/// How `weaveffi package` should obtain the native libraries it bundles. -pub(crate) enum BinarySource<'a> { - /// A directory laid out as `//`. - Prebuilt(&'a str), - /// A Cargo package to cross-compile as a `cdylib` for each platform. - Build(&'a str), -} +use weaveffi_cli::project::Project; +use weaveffi_model::model::Model; + +use super::build::{build_project, library_name, model_from_binaries, resolve_platforms}; /// Options for [`cmd_package`]. pub(crate) struct PackageArgs<'a> { pub(crate) input: Option<&'a str>, - pub(crate) out: &'a str, + pub(crate) out: Option<&'a str>, pub(crate) targets: Option<&'a str>, pub(crate) config: Option<&'a str>, pub(crate) binaries: Option<&'a str>, - pub(crate) build: Option<&'a str>, pub(crate) platforms: Option<&'a str>, + pub(crate) debug: bool, + pub(crate) manifest_path: Option<&'a str>, pub(crate) warn: bool, pub(crate) quiet: bool, } pub(crate) fn cmd_package(args: &PackageArgs<'_>) -> Result<()> { - let PackageArgs { - out, - targets, - binaries, - build, - platforms, - quiet, - .. - } = *args; - let source = match (binaries, build) { - (Some(_), Some(_)) => { - return Err(miette::miette!("--binaries and --build are mutually exclusive; choose one source for the native libraries")) - } - (Some(dir), None) => BinarySource::Prebuilt(dir), - (None, Some(crate_name)) => BinarySource::Build(crate_name), - (None, None) => return Err(miette::miette!( - "provide native libraries with --binaries (laid out as //) \ - or --build to cross-compile a Rust producer" - )), - }; - - let project = super::load_project(args.input, args.config, args.warn)?; - let (config, api) = (&project.config, &project.api); - - let explicit_platforms = platforms.is_some(); - let input_basename = project.input.file_name(); - let lib_name = api.identity().library.clone(); + let quiet = args.quiet; + let project = Project::locate(args.config, args.input, None)?.quiet(quiet); + let config = &project.config; + let selected = config.select_targets(args.targets)?; + if selected.is_empty() { + bail!("no targets selected to package"); + } + let settings = config.build.settings(args.debug); - let binary_set = match source { - BinarySource::Prebuilt(dir) => { - // Without --platforms, package every platform the directory has. - let selected = match platforms { - Some(_) => parse_platforms(platforms)?, - None => Platform::ALL.to_vec(), + let (binaries, model) = match args.binaries { + Some(dir) => { + let platforms = match args.platforms { + Some(_) => Some(resolve_platforms(args.platforms, &project)?), + None => None, }; - discover_prebuilt( - Utf8Path::new(dir), - &lib_name, - &selected, - quiet || !explicit_platforms, - )? + let (library, model) = library_name(&project, args.warn)?; + let binaries = BinarySet::read_dir(Utf8Path::new(dir), &library, platforms.as_deref()) + .map_err(|e| miette!("{e:#}"))?; + let model = match model { + Some(model) => model, + None => model_from_binaries(&project, &binaries, args.warn)?, + }; + (binaries, model) } - BinarySource::Build(crate_name) => { - cross_build(crate_name, &lib_name, &parse_platforms(platforms)?, quiet)? + None => { + let platforms = resolve_platforms(args.platforms, &project)?; + let built = build_project( + &project, + &selected, + &platforms, + &settings, + args.manifest_path, + args.warn, + )?; + (built.binaries, built.model) } }; + let library = model.identity.library.clone(); - if binary_set.is_empty() { - bail!("no native libraries were found for any requested platform; nothing to package"); - } - - let out_dir = Utf8Path::new(out); - std::fs::create_dir_all(out_dir.as_std_path()) + let dist = config.dist_dir(args.out); + std::fs::create_dir_all(dist.as_std_path()) .into_diagnostic() - .wrap_err_with(|| format!("failed to create output directory: {out}"))?; - - let selected = config.select_targets(targets)?; - - if selected.is_empty() { - bail!("no targets selected to package"); + .wrap_err_with(|| format!("failed to create the dist directory {dist}"))?; + if !quiet { + let plats: Vec<&str> = binaries.platforms().map(Platform::id).collect(); + println!("Packaging `{library}` for {} into {dist}", plats.join(", ")); } + let xcframework = if selected.iter().any(|t| t.name() == "swift") { + assemble_xcframework(&project, &model, &binaries, &dist, quiet)? + } else { + None + }; let ctx = PackageContext { - binaries: &binary_set, - input_basename, + binaries: &binaries, + macos_deployment_target: &settings.macos_deployment_target, + ios_deployment_target: &settings.ios_deployment_target, + xcframework: xcframework.as_ref(), }; - if !quiet { - let plats: Vec<&str> = binary_set.platforms().map(Platform::id).collect(); - println!("Packaging '{lib_name}' for platforms: {}", plats.join(", ")); - } - - let mut packaged = 0usize; - let mut skipped: Vec<&str> = Vec::new(); - for gen in &selected { - match gen.package(api, &ctx, out_dir) { - Some(files) => { - write_package(&files).map_err(|e| miette!("{:#}", e))?; - let (text, bins) = summarize(&files); - packaged += 1; + let mut written = 0usize; + let mut skipped = Vec::new(); + let mut failures = Vec::new(); + for target in &selected { + let artifacts = target.package(&model, &ctx).unwrap_or_default(); + if artifacts.is_empty() { + skipped.push(target.name()); + continue; + } + if target.name() == "swift" { + if let Some(archive) = &xcframework { if !quiet { - println!( - " {}: {text} file(s), {bins} bundled binary(ies)", - gen.name() - ); + println!(" swift: swift/{}", archive.file_name); + } + } + } + for artifact in &artifacts { + let path = write_artifact(&dist, artifact).map_err(|e| miette!("{e:#}"))?; + written += 1; + if !quiet { + println!(" {}: {}", target.name(), relative(&dist, &path)); + } + } + if target.name() == "dotnet" { + match dotnet_pack(&dist, &artifacts) { + Ok(nupkgs) => { + for nupkg in nupkgs { + if !quiet { + println!(" dotnet: dotnet/{}", nupkg.file_name().unwrap_or_default()); + } + } } + Err(e) => failures.push(format!("{e}")), } - None => skipped.push(gen.name()), } } if !skipped.is_empty() && !quiet { eprintln!( - "note: these targets produced no package and were skipped: {}. A target is \ - skipped when it has no binary packaging or when no binary was found for the \ - platforms it ships (for example `wasm32` for wasm). Run `weaveffi generate` \ - for their source bindings.", - skipped.join(", ") + "note: no artifacts for {}: none of the built platforms ({}) is one they ship. \ + Run `weaveffi generate` for their source bindings.", + skipped.join(", "), + binaries + .platforms() + .map(Platform::id) + .collect::>() + .join(", ") ); } - - if packaged == 0 { + if !failures.is_empty() { + bail!("{}", failures.join("\n")); + } + if written == 0 { bail!( - "none of the selected targets produced a package: check that --binaries (or \ - --build) provides a library for a platform each target ships" + "none of the selected targets produced an artifact: build a platform each one ships \ + (with --platforms)" ); } + Ok(()) +} - if selected.iter().any(|t| t.name() == "swift") { - assemble_xcframework(&project, &binary_set, out_dir, quiet)?; - } +/// `path` relative to `dist`, for progress lines. +fn relative(dist: &Utf8Path, path: &Utf8Path) -> String { + path.strip_prefix(dist) + .map(|p| p.to_string()) + .unwrap_or_else(|_| path.to_string()) +} - if !quiet { - println!("Packaged {packaged} target(s) into {out}"); +/// Run `dotnet pack` on the project directory the .NET target wrote, +/// returning the `.nupkg` files it produced under `dist/dotnet/`. +fn dotnet_pack(dist: &Utf8Path, artifacts: &[Artifact]) -> Result> { + let mut produced = Vec::new(); + for artifact in artifacts { + let Some(csproj) = artifact + .files + .iter() + .find(|f| f.path.extension() == Some("csproj")) + else { + continue; + }; + let project = dist.join(&artifact.path).join(&csproj.path); + let out = std::path::absolute(dist.join("dotnet").as_std_path()) + .ok() + .and_then(|p| Utf8PathBuf::from_path_buf(p).ok()) + .unwrap_or_else(|| dist.join("dotnet")); + // `PackageOutputPath` rather than `--output`, which some SDKs + // mis-forward when a project path is given. + let Ok(output) = Command::new("dotnet") + .args(["pack", project.as_str(), "--configuration", "Release"]) + .arg(format!("-p:PackageOutputPath={out}")) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .output() + else { + bail!( + "the .NET SDK isn't installed (no `dotnet` on PATH), so the NuGet package wasn't \ + built; install the SDK and rerun, or run `dotnet pack {project} -c Release`" + ); + }; + if !output.status.success() { + bail!( + "`dotnet pack {project}` failed:\n{}{}", + String::from_utf8_lossy(&output.stdout).trim_end(), + String::from_utf8_lossy(&output.stderr).trim_end() + ); + } + let stem = csproj.path.file_stem().unwrap_or_default(); + for entry in out.read_dir_utf8().into_diagnostic()? { + let path = entry.into_diagnostic()?.path().to_path_buf(); + if path.extension() == Some("nupkg") + && path + .file_name() + .is_some_and(|n| n.starts_with(&format!("{stem}."))) + { + produced.push(path); + } + } } - Ok(()) + Ok(produced) } -/// Parse the comma-separated `--platforms` list into [`Platform`] values, -/// defaulting to the full v1 matrix when omitted. -/// The platforms to build with `--build`; without `--platforms`, just the -/// host (cross builds need targets and linkers most machines don't have, so -/// they are opt-in). -/// Fuse the Apple slices into `swift/C{Module}.xcframework` beside the -/// generated `Package.swift`, which switches to it as a binary target. +/// Fuse the Apple static libraries into `C{Module}.xcframework`, zip it into +/// `dist/swift/C{Module}.xcframework.zip`, and write its SHA-256 checksum +/// next to it, returning the archive's name and checksum for the Swift +/// package's binary target. /// -/// Runs only when the set includes an iOS slice (iOS needs the XCFramework; -/// macOS alone links the bundled dylib through the system-library target). -/// Each slice is a static library: iOS slices are the crate's `staticlib` -/// output, and a macOS slice is the `lib{library}.a` next to its dylib (so a -/// producer that ships to both declares `crate-type = ["cdylib", "staticlib"]`). -/// Simulator and macOS architectures are fused with `lipo`. +/// Slices are grouped the way `xcodebuild -create-xcframework` needs them +/// (iOS device, iOS simulator, macOS), with the architectures of a group +/// fused by `lipo`. Returns `None` (with a note) when there are no Apple +/// static libraries or the host isn't macOS. fn assemble_xcframework( - project: &super::Project, + project: &Project, + model: &Model, binaries: &BinarySet, - out_dir: &Utf8Path, + dist: &Utf8Path, quiet: bool, -) -> Result<()> { - if !binaries.platforms().any(|p| p.os() == Os::Ios) { - return Ok(()); +) -> Result> { + let static_lib = |p: Platform| binaries.get(p).and_then(|nb| nb.staticlib.clone()); + let any_apple = binaries + .platforms() + .any(|p| matches!(p.os(), Os::MacOs | Os::Ios) && static_lib(p).is_some()); + if !any_apple { + if !quiet { + eprintln!( + "note: the Swift package needs an Apple platform (darwin-*, ios-*) built with \ + `weaveffi build`, which adds the static library its XCFramework is made from" + ); + } + return Ok(None); } if !cfg!(target_os = "macos") { eprintln!("warning: skipping the Swift XCFramework: xcodebuild needs macOS"); - return Ok(()); + return Ok(None); } - let identity = project.api.identity(); - let library = &identity.library; - let module = project - .config - .generators - .swift - .module_name - .clone() - .unwrap_or_else(|| identity.pascal_name()); - let static_lib = |p: Platform| -> Option { - let nb = binaries.get(p)?; - if p.os() == Os::Ios { - return Some(nb.source.clone()); - } - let sibling = nb.source.parent()?.join(format!("lib{library}.a")); - sibling.exists().then_some(sibling) - }; + let library = &model.identity.library; + let c_module = project.config.generators.swift.c_module_name(model); + let scratch = tempfile::tempdir().into_diagnostic()?; let scratch = Utf8Path::from_path(scratch.path()) .ok_or_else(|| miette!("temp directory path is not valid UTF-8"))? @@ -211,16 +261,14 @@ fn assemble_xcframework( let headers = scratch.join("Headers"); std::fs::create_dir_all(headers.as_std_path()).into_diagnostic()?; let header = format!("{library}.h"); - let model = weaveffi_model::model::BindingModel::build(&project.api); - let input_basename = project.input.file_name().unwrap_or("api"); std::fs::write( headers.join(&header).as_std_path(), - weaveffi_gen::targets::c::render_c_header_from_model(&model, input_basename, &header), + weaveffi_cli::targets::c::render_c_header_from_model(model, &header), ) .into_diagnostic()?; std::fs::write( headers.join("module.modulemap").as_std_path(), - format!("module C{module} {{\n header \"{header}\"\n export *\n}}\n"), + format!("module {c_module} {{\n header \"{header}\"\n export *\n}}\n"), ) .into_diagnostic()?; @@ -256,305 +304,60 @@ fn assemble_xcframework( args.extend(["-library".into(), lib.to_string()]); args.extend(["-headers".into(), headers.to_string()]); } - let output = out_dir.join("swift").join(format!("C{module}.xcframework")); - if output.exists() { - std::fs::remove_dir_all(output.as_std_path()).into_diagnostic()?; - } - args.extend(["-output".into(), output.to_string()]); - let status = Command::new("xcodebuild") + let framework_name = format!("{c_module}.xcframework"); + let framework = scratch.join(&framework_name); + args.extend(["-output".into(), framework.to_string()]); + let output = Command::new("xcodebuild") .args(&args) - .stdout(Stdio::null()) - .status() - .into_diagnostic() - .wrap_err("failed to run xcodebuild")?; - if !status.success() { - bail!("xcodebuild -create-xcframework failed"); - } - if !quiet { - println!(" swift: assembled {output}"); - } - Ok(()) -} - -fn parse_platforms(platforms: Option<&str>) -> Result> { - let Some(list) = platforms else { - return Platform::host().map(|p| vec![p]).ok_or_else(|| { - miette!("the host is not a packaging platform; pass --platforms explicitly") - }); - }; - let mut out = Vec::new(); - for token in list.split(',').map(str::trim).filter(|s| !s.is_empty()) { - let p = Platform::from_id(token).ok_or_else(|| { - let known: Vec<&str> = Platform::ALL.iter().map(|p| p.id()).collect(); - miette!( - "unknown platform '{token}'; expected one of: {}", - known.join(", ") - ) - })?; - if !out.contains(&p) { - out.push(p); - } - } - if out.is_empty() { - bail!("--platforms was empty; expected a comma-separated list of platform ids"); - } - Ok(out) -} - -/// Locate prebuilt libraries under `dir//`, one per requested -/// platform. Missing platforms are warned about and skipped rather than fatal, -/// so a partial matrix still produces artifacts for what is available. -fn discover_prebuilt( - dir: &Utf8Path, - lib_name: &str, - platforms: &[Platform], - quiet: bool, -) -> Result { - if !dir.as_std_path().is_dir() { - bail!("--binaries path is not a directory: {dir}"); - } - let mut set = BinarySet::new(lib_name); - for &platform in platforms { - let platform_dir = dir.join(platform.id()); - if !platform_dir.as_std_path().is_dir() { - if !quiet { - eprintln!( - "warning: no directory for platform {} at {platform_dir}; skipping", - platform.id() - ); - } - continue; - } - match find_library(&platform_dir, platform, lib_name)? { - Some(path) => set.insert(platform, path), - None => { - if !quiet { - eprintln!( - "warning: no .{} library found in {platform_dir}; skipping {}", - platform.lib_extension(), - platform.id() - ); - } - } - } - } - Ok(set) -} - -/// Find the single shared library with `platform`'s extension inside -/// `platform_dir`. When several are present, prefer the one whose name matches -/// the canonical `lib_name`; otherwise the choice is ambiguous and is an error. -fn find_library( - platform_dir: &Utf8Path, - platform: Platform, - lib_name: &str, -) -> Result> { - let ext = platform.lib_extension(); - let mut matches: Vec = Vec::new(); - let entries = std::fs::read_dir(platform_dir.as_std_path()) - .into_diagnostic() - .wrap_err_with(|| format!("failed to read {platform_dir}"))?; - for entry in entries { - let entry = entry.into_diagnostic()?; - let path = Utf8PathBuf::from_path_buf(entry.path()) - .map_err(|p| miette!("non-UTF-8 path in binaries directory: {}", p.display()))?; - if path.extension() == Some(ext) { - matches.push(path); - } - } - matches.sort(); - match matches.len() { - 0 => Ok(None), - 1 => Ok(Some(matches.into_iter().next().unwrap())), - _ => { - let canonical = platform.lib_filename(lib_name); - if let Some(hit) = matches.iter().find(|p| p.file_name() == Some(&canonical)) { - Ok(Some(hit.clone())) - } else { - Err(miette::miette!( - "multiple .{ext} libraries in {platform_dir}; \ - name one '{canonical}' to disambiguate" - )) - } - } - } -} - -/// Cross-compile `crate_name` as a `cdylib` for each requested platform and -/// collect the produced libraries. -fn cross_build( - crate_name: &str, - lib_name: &str, - platforms: &[Platform], - quiet: bool, -) -> Result { - let mut set = BinarySet::new(lib_name); - let mut failed = Vec::new(); - for &platform in platforms { - if !quiet { - println!( - "Building {crate_name} for {} ({})...", - platform.display_name(), - platform.rust_target() - ); - } - match build_one(crate_name, platform) { - Ok(lib) => set.insert(platform, lib), - Err(e) => { - eprintln!("error: {e:?}"); - failed.push(platform.id()); - } - } - } - if !failed.is_empty() { - bail!( - "cargo build failed for {}; the other platforms built (fix or drop the failing \ - ones with --platforms)", - failed.join(", ") - ); - } - Ok(set) -} - -/// Run `cargo build --release` for one platform and return the path to the -/// produced `cdylib`, parsed from cargo's JSON artifact messages. -fn build_one(crate_name: &str, platform: Platform) -> Result { - let triple = platform.rust_target(); - let child = Command::new("cargo") - .args([ - "build", - "--release", - "--message-format=json-render-diagnostics", - "--target", - triple, - "-p", - crate_name, - ]) - .stdout(Stdio::piped()) - .stderr(Stdio::inherit()) - .spawn() + .output() .into_diagnostic() - .wrap_err("failed to launch cargo")?; - let output = child - .wait_with_output() - .into_diagnostic() - .wrap_err("failed to wait for cargo")?; + .wrap_err("failed to run xcodebuild (is Xcode installed?)")?; if !output.status.success() { bail!( - "cargo build for {} failed (is the `{triple}` target installed and a cross-linker available? \ - `rustup target add {triple}`)", - platform.id() + "xcodebuild -create-xcframework failed:\n{}", + String::from_utf8_lossy(&output.stderr).trim_end() ); } - let stdout = String::from_utf8_lossy(&output.stdout); - let ext = platform.lib_extension(); - let mut produced: Vec = Vec::new(); - for line in stdout.lines() { - let Ok(msg) = serde_json::from_str::(line) else { - continue; - }; - if msg.get("reason").and_then(|r| r.as_str()) != Some("compiler-artifact") { - continue; - } - let want = if platform.os() == Os::Ios { - "staticlib" - } else { - "cdylib" - }; - let is_cdylib = msg - .get("target") - .and_then(|t| t.get("kind")) - .and_then(|k| k.as_array()) - .map(|kinds| kinds.iter().any(|k| k.as_str() == Some(want))) - .unwrap_or(false); - if !is_cdylib { - continue; - } - if let Some(files) = msg.get("filenames").and_then(|f| f.as_array()) { - for f in files.iter().filter_map(|f| f.as_str()) { - if f.ends_with(&format!(".{ext}")) { - produced.push(Utf8PathBuf::from(f)); - } - } - } - } - - match produced.len() { - 0 => Err(miette::miette!( - "cargo built {crate_name} for {triple} but produced no .{ext} cdylib; \ - ensure the crate declares `crate-type = [\"cdylib\", \"staticlib\"]` \ - (iOS slices are static libraries)" - )), - 1 => Ok(produced.into_iter().next().unwrap()), - _ => { - // Prefer the artifact whose stem matches the crate's normalized lib name. - let normalized = crate_name.replace('-', "_"); - let preferred = produced.iter().find(|p| { - p.file_stem() - .map(|s| s == normalized || s == format!("lib{normalized}")) - .unwrap_or(false) - }); - Ok(preferred.cloned().unwrap_or_else(|| produced[0].clone())) - } - } + let mut files = Vec::new(); + collect_files(&framework, &framework_name, &mut files)?; + let entries = archive::entries(&files, None).map_err(|e| miette!("{e:#}"))?; + let zip = archive::zip(&entries).map_err(|e| miette!("{e:#}"))?; + let checksum = archive::sha256_hex(&zip); + let file_name = format!("{framework_name}.zip"); + let swift_dist = dist.join("swift"); + std::fs::create_dir_all(swift_dist.as_std_path()).into_diagnostic()?; + std::fs::write(swift_dist.join(&file_name).as_std_path(), &zip) + .into_diagnostic() + .wrap_err("failed to write the XCFramework archive")?; + std::fs::write( + swift_dist.join(format!("{file_name}.sha256")).as_std_path(), + format!("{checksum} {file_name}\n"), + ) + .into_diagnostic()?; + Ok(Some(XcframeworkArchive { + file_name, + checksum, + })) } -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn parse_platforms_defaults_to_the_host() { - if let Some(host) = Platform::host() { - assert_eq!(parse_platforms(None).unwrap(), vec![host]); +/// Every file under `dir`, in sorted order, as an archive entry under +/// `prefix/`. +fn collect_files(dir: &Utf8Path, prefix: &str, out: &mut Vec) -> Result<()> { + let mut entries: Vec = dir + .read_dir_utf8() + .into_diagnostic()? + .map(|e| e.map(|e| e.path().to_path_buf())) + .collect::>() + .into_diagnostic()?; + entries.sort(); + for path in entries { + let name = format!("{prefix}/{}", path.file_name().unwrap_or_default()); + if path.is_dir() { + collect_files(&path, &name, out)?; + } else { + out.push(PackagedFile::copy(name, path)); } } - - #[test] - fn parse_platforms_selects_and_dedups() { - let got = parse_platforms(Some("darwin-arm64, linux-x64 , darwin-arm64")).unwrap(); - assert_eq!(got, vec![Platform::MacosArm64, Platform::LinuxX64]); - } - - #[test] - fn parse_platforms_rejects_unknown() { - let err = parse_platforms(Some("solaris-sparc")).unwrap_err(); - assert!(err.to_string().contains("unknown platform")); - } - - #[test] - fn discover_prebuilt_finds_per_platform_libs() { - let dir = tempfile::tempdir().unwrap(); - let root = Utf8Path::from_path(dir.path()).unwrap(); - // darwin-arm64/libcontacts.dylib and linux-x64/libcontacts.so - let mac = root.join("darwin-arm64"); - let lin = root.join("linux-x64"); - std::fs::create_dir_all(mac.as_std_path()).unwrap(); - std::fs::create_dir_all(lin.as_std_path()).unwrap(); - std::fs::write(mac.join("libcontacts.dylib").as_std_path(), b"m").unwrap(); - std::fs::write(lin.join("libcontacts.so").as_std_path(), b"l").unwrap(); - - let set = discover_prebuilt(root, "contacts", &Platform::ALL, true).unwrap(); - assert_eq!(set.binaries.len(), 2); - assert!(set.get(Platform::MacosArm64).is_some()); - assert!(set.get(Platform::LinuxX64).is_some()); - assert!(set.get(Platform::WindowsX64).is_none()); - } - - #[test] - fn discover_prebuilt_disambiguates_by_canonical_name() { - let dir = tempfile::tempdir().unwrap(); - let root = Utf8Path::from_path(dir.path()).unwrap(); - let mac = root.join("darwin-arm64"); - std::fs::create_dir_all(mac.as_std_path()).unwrap(); - std::fs::write(mac.join("libcontacts.dylib").as_std_path(), b"m").unwrap(); - std::fs::write(mac.join("libother.dylib").as_std_path(), b"o").unwrap(); - - let set = discover_prebuilt(root, "contacts", &[Platform::MacosArm64], true).unwrap(); - assert_eq!( - set.get(Platform::MacosArm64).unwrap().source.file_name(), - Some("libcontacts.dylib") - ); - } + Ok(()) } diff --git a/crates/weaveffi-cli/src/commands/validate.rs b/crates/weaveffi-cli/src/commands/validate.rs index e79831b8..83f590d7 100644 --- a/crates/weaveffi-cli/src/commands/validate.rs +++ b/crates/weaveffi-cli/src/commands/validate.rs @@ -2,29 +2,19 @@ //! json` output, plus advisory warnings under `--warn`. use miette::{Report, Result}; -use weaveffi_model::validate::{ - collect_warnings, validate_api, ValidationError, ValidationWarning, -}; +use weaveffi_model::validate::{ValidationError, ValidationWarning}; -pub(crate) fn cmd_validate( - input: Option<&str>, - config: Option<&str>, - warn: bool, - json_mode: bool, - quiet: bool, -) -> Result<()> { - let (_, input) = crate::config::ProjectConfig::locate(config, input)?; - let input = input.as_str(); - let (api, contents) = super::load_api(input)?; - - match validate_api(api, Some((input, &contents))) { - Ok(api) => { +pub(crate) fn cmd_validate(locate: &super::Locate<'_>, warn: bool, json_mode: bool) -> Result<()> { + let quiet = locate.quiet; + let definition = locate.project()?.definition()?; + match definition.validate() { + Ok(_) => { let warnings = if warn { - collect_warnings(&api) + definition.warnings() } else { Vec::new() }; - let counts = Counts::of(&api.modules); + let counts = Counts::of(&definition.api.modules); if json_mode { let json = serde_json::json!({ "ok": true, @@ -120,224 +110,20 @@ impl Counts { } } -/// Stable string code for a [`ValidationError`] variant, used as the `code` -/// field in `weaveffi validate --format json` failure output. Keeping the -/// codes in lock-step with the variant identifiers makes them ergonomic to -/// match against in CI scripts. -fn validation_error_code(err: &ValidationError) -> &'static str { - match err { - ValidationError::NoModuleName => "NoModuleName", - ValidationError::DuplicateModuleName(_) => "DuplicateModuleName", - ValidationError::InvalidModuleName(_, _) => "InvalidModuleName", - ValidationError::DuplicateFunctionName { .. } => "DuplicateFunctionName", - ValidationError::DuplicateParamName { .. } => "DuplicateParamName", - ValidationError::ReservedKeyword(_) => "ReservedKeyword", - ValidationError::InvalidIdentifier(_, _) => "InvalidIdentifier", - ValidationError::ErrorDomainMissingName(_) => "ErrorDomainMissingName", - ValidationError::DuplicateErrorName { .. } => "DuplicateErrorName", - ValidationError::DuplicateErrorCode { .. } => "DuplicateErrorCode", - ValidationError::InvalidErrorCode { .. } => "InvalidErrorCode", - ValidationError::NameCollisionWithErrorDomain { .. } => "NameCollisionWithErrorDomain", - ValidationError::DuplicateStructName { .. } => "DuplicateStructName", - ValidationError::DuplicateStructField { .. } => "DuplicateStructField", - ValidationError::EmptyStruct { .. } => "EmptyStruct", - ValidationError::DuplicateEnumName { .. } => "DuplicateEnumName", - ValidationError::EmptyEnum { .. } => "EmptyEnum", - ValidationError::DuplicateEnumVariant { .. } => "DuplicateEnumVariant", - ValidationError::DuplicateEnumVariantField { .. } => "DuplicateEnumVariantField", - ValidationError::DuplicateEnumValue { .. } => "DuplicateEnumValue", - ValidationError::UnknownTypeRef { .. } => "UnknownTypeRef", - ValidationError::InvalidMapKey { .. } => "InvalidMapKey", - ValidationError::DuplicateCallbackInterfaceName { .. } => "DuplicateCallbackInterfaceName", - ValidationError::EmptyCallbackInterface { .. } => "EmptyCallbackInterface", - ValidationError::DuplicateCallbackMethod { .. } => "DuplicateCallbackMethod", - ValidationError::InvalidCallbackMethod { .. } => "InvalidCallbackMethod", - ValidationError::CallbackInterfaceInInvalidPosition { .. } => { - "CallbackInterfaceInInvalidPosition" - } - ValidationError::IteratorInInvalidPosition { .. } => "IteratorInInvalidPosition", - ValidationError::AsyncIteratorReturn { .. } => "AsyncIteratorReturn", - ValidationError::UnsupportedSchemaVersion { .. } => "UnsupportedSchemaVersion", - ValidationError::DuplicateInterfaceName { .. } => "DuplicateInterfaceName", - ValidationError::DuplicateInterfaceMember { .. } => "DuplicateInterfaceMember", - ValidationError::EmptyInterface { .. } => "EmptyInterface", - ValidationError::ConstructorHasReturn { .. } => "ConstructorHasReturn", - ValidationError::AsyncConstructor { .. } => "AsyncConstructor", - ValidationError::InterfaceInInvalidPosition { .. } => "InterfaceInInvalidPosition", - ValidationError::DuplicateTypeName { .. } => "DuplicateTypeName", - ValidationError::DuplicateErrorCodeName { .. } => "DuplicateErrorCodeName", - ValidationError::ThrowsWithoutErrorDomain { .. } => "ThrowsWithoutErrorDomain", - ValidationError::SymbolCollision { .. } => "SymbolCollision", - ValidationError::CancellableNotAsync { .. } => "CancellableNotAsync", - } -} - -/// Convert a [`ValidationError`] into a JSON object with `code`, the -/// variant-specific identifying fields (`module`, `function`, `name`, …), -/// `message`, and `suggestion` derived from the [`miette::Diagnostic`] help. +/// Convert a [`ValidationError`] into a JSON object: its `code` (the variant +/// name) and identifying fields from its serialized form, plus `message` and +/// a `suggestion` derived from the [`miette::Diagnostic`] help. fn validation_error_to_json(err: &ValidationError) -> serde_json::Value { use miette::Diagnostic; - use serde_json::{Map, Value}; - let mut obj = Map::new(); - obj.insert( - "code".into(), - Value::String(validation_error_code(err).into()), - ); - match err { - ValidationError::NoModuleName => {} - ValidationError::DuplicateModuleName(name) | ValidationError::ReservedKeyword(name) => { - obj.insert("name".into(), Value::String(name.clone())); - } - ValidationError::InvalidModuleName(name, reason) - | ValidationError::InvalidIdentifier(name, reason) => { - obj.insert("name".into(), Value::String(name.clone())); - obj.insert("reason".into(), Value::String((*reason).into())); - } - ValidationError::DuplicateFunctionName { module, function } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("function".into(), Value::String(function.clone())); - } - ValidationError::DuplicateParamName { - module, - function, - param, - } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("function".into(), Value::String(function.clone())); - obj.insert("param".into(), Value::String(param.clone())); - } - ValidationError::ErrorDomainMissingName(module) => { - obj.insert("module".into(), Value::String(module.clone())); - } - ValidationError::DuplicateErrorName { module, name } - | ValidationError::NameCollisionWithErrorDomain { module, name } - | ValidationError::InvalidErrorCode { module, name } - | ValidationError::DuplicateStructName { module, name } - | ValidationError::EmptyStruct { module, name } - | ValidationError::DuplicateEnumName { module, name } - | ValidationError::EmptyEnum { module, name } - | ValidationError::DuplicateCallbackInterfaceName { module, name } - | ValidationError::EmptyCallbackInterface { module, name } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("name".into(), Value::String(name.clone())); - } - ValidationError::DuplicateErrorCode { module, code } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("error_code".into(), Value::Number((*code).into())); - } - ValidationError::DuplicateStructField { struct_name, field } => { - obj.insert("struct".into(), Value::String(struct_name.clone())); - obj.insert("field".into(), Value::String(field.clone())); - } - ValidationError::DuplicateEnumVariant { enum_name, variant } => { - obj.insert("enum".into(), Value::String(enum_name.clone())); - obj.insert("variant".into(), Value::String(variant.clone())); - } - ValidationError::DuplicateEnumVariantField { - enum_name, - variant, - field, - } => { - obj.insert("enum".into(), Value::String(enum_name.clone())); - obj.insert("variant".into(), Value::String(variant.clone())); - obj.insert("field".into(), Value::String(field.clone())); - } - ValidationError::DuplicateEnumValue { enum_name, value } => { - obj.insert("enum".into(), Value::String(enum_name.clone())); - obj.insert("value".into(), Value::Number((*value).into())); - } - ValidationError::UnknownTypeRef { name } => { - obj.insert("name".into(), Value::String(name.clone())); - } - ValidationError::InvalidMapKey { key_type } => { - obj.insert("key_type".into(), Value::String(key_type.clone())); - } - ValidationError::DuplicateCallbackMethod { interface, name } => { - obj.insert("interface".into(), Value::String(interface.clone())); - obj.insert("name".into(), Value::String(name.clone())); - } - ValidationError::InvalidCallbackMethod { - interface, - method, - reason, - } => { - obj.insert("interface".into(), Value::String(interface.clone())); - obj.insert("method".into(), Value::String(method.clone())); - obj.insert("reason".into(), Value::String((*reason).into())); - } - ValidationError::IteratorInInvalidPosition { location } => { - obj.insert("location".into(), Value::String(location.clone())); - } - ValidationError::AsyncIteratorReturn { module, function } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("function".into(), Value::String(function.clone())); - } - ValidationError::UnsupportedSchemaVersion { version, supported } => { - obj.insert("version".into(), Value::String(version.clone())); - obj.insert("supported".into(), Value::String(supported.clone())); - } - ValidationError::DuplicateInterfaceName { module, name } - | ValidationError::EmptyInterface { module, name } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("name".into(), Value::String(name.clone())); - } - ValidationError::DuplicateInterfaceMember { interface, name } => { - obj.insert("interface".into(), Value::String(interface.clone())); - obj.insert("name".into(), Value::String(name.clone())); - } - ValidationError::ConstructorHasReturn { - interface, - constructor, - } - | ValidationError::AsyncConstructor { - interface, - constructor, - } => { - obj.insert("interface".into(), Value::String(interface.clone())); - obj.insert("constructor".into(), Value::String(constructor.clone())); - } - ValidationError::InterfaceInInvalidPosition { name, location } - | ValidationError::CallbackInterfaceInInvalidPosition { name, location } => { - obj.insert("name".into(), Value::String(name.clone())); - obj.insert("location".into(), Value::String(location.clone())); - } - ValidationError::DuplicateTypeName { - name, - first, - second, - } - | ValidationError::DuplicateErrorCodeName { - name, - first, - second, - } => { - obj.insert("name".into(), Value::String(name.clone())); - obj.insert("first".into(), Value::String(first.clone())); - obj.insert("second".into(), Value::String(second.clone())); - } - ValidationError::ThrowsWithoutErrorDomain { module, function } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("function".into(), Value::String(function.clone())); - } - ValidationError::SymbolCollision { - symbol, - first, - second, - } => { - obj.insert("symbol".into(), Value::String(symbol.clone())); - obj.insert("first".into(), Value::String(first.clone())); - obj.insert("second".into(), Value::String(second.clone())); - } - ValidationError::CancellableNotAsync { module, function } => { - obj.insert("module".into(), Value::String(module.clone())); - obj.insert("function".into(), Value::String(function.clone())); - } - } - obj.insert("message".into(), Value::String(err.to_string())); + let mut obj = match serde_json::to_value(err) { + Ok(serde_json::Value::Object(obj)) => obj, + _ => serde_json::Map::new(), + }; + obj.insert("message".into(), err.to_string().into()); if let Some(help) = err.help() { - obj.insert("suggestion".into(), Value::String(help.to_string())); + obj.insert("suggestion".into(), help.to_string().into()); } - Value::Object(obj) + serde_json::Value::Object(obj) } /// Convert a [`ValidationWarning`] into a JSON object of `{ code, location, diff --git a/crates/weaveffi-cli/src/config.rs b/crates/weaveffi-cli/src/config.rs index 2fe1dc8f..65b4c7ac 100644 --- a/crates/weaveffi-cli/src/config.rs +++ b/crates/weaveffi-cli/src/config.rs @@ -2,12 +2,12 @@ //! `cli_targets!` registry, and the per-target [`Targets`] struct it //! generates. //! -//! A project's identity, inputs, and generator options live in one TOML file -//! at the project root: +//! A project's identity, inputs, build settings, and generator options live +//! in one TOML file at the project root: //! //! ```toml //! [project] -//! input = "src/lib.rs" # or an IDL such as "kvstore.yml" +//! input = "." # the producer crate, or an IDL such as "kvstore.yml" //! out = "bindings" //! targets = ["c", "swift", "python"] //! @@ -15,9 +15,13 @@ //! name = "kvstore" //! version = "1.2.0" //! license = "MIT" +//! dist = "dist" +//! +//! [build] +//! platforms = ["darwin-arm64", "ios-arm64", "ios-sim-arm64"] //! //! [generators.swift] -//! module_name = "KVStore" +//! name = "KVStore" //! ``` //! //! The API definition describes only the API surface. With a `[project]` @@ -26,25 +30,26 @@ //! nearest `weaveffi.toml` at or above the input's directory; `--config //! ` names one explicitly. +use crate::build::BuildSettings; +use crate::codegen::{ConfiguredBackend, Target}; +use crate::targets::c::{CConfig, CGenerator}; +use crate::targets::cpp::{CppConfig, CppGenerator}; +use crate::targets::dart::{DartConfig, DartGenerator}; +use crate::targets::dotnet::{DotnetConfig, DotnetGenerator}; +use crate::targets::go::{GoConfig, GoGenerator}; +use crate::targets::kotlin::{KotlinConfig, KotlinGenerator}; +use crate::targets::node::{NodeConfig, NodeGenerator}; +use crate::targets::python::{PythonConfig, PythonGenerator}; +use crate::targets::ruby::{RubyConfig, RubyGenerator}; +use crate::targets::swift::{SwiftConfig, SwiftGenerator}; +use crate::targets::wasm::{WasmConfig, WasmGenerator}; use camino::{Utf8Path, Utf8PathBuf}; use miette::{IntoDiagnostic, Result, WrapErr}; -use serde::Deserialize; -use weaveffi_gen::codegen::{ConfiguredBackend, Target}; -use weaveffi_gen::targets::c::{CConfig, CGenerator}; -use weaveffi_gen::targets::cpp::{CppConfig, CppGenerator}; -use weaveffi_gen::targets::dart::{DartConfig, DartGenerator}; -use weaveffi_gen::targets::dotnet::{DotnetConfig, DotnetGenerator}; -use weaveffi_gen::targets::go::{GoConfig, GoGenerator}; -use weaveffi_gen::targets::kotlin::{KotlinConfig, KotlinGenerator}; -use weaveffi_gen::targets::node::{NodeConfig, NodeGenerator}; -use weaveffi_gen::targets::python::{PythonConfig, PythonGenerator}; -use weaveffi_gen::targets::ruby::{RubyConfig, RubyGenerator}; -use weaveffi_gen::targets::swift::{SwiftConfig, SwiftGenerator}; -use weaveffi_gen::targets::wasm::{WasmConfig, WasmGenerator}; +use serde::{Deserialize, Deserializer}; use weaveffi_model::pkg::Package; /// The file name `weaveffi generate` looks for when `--config` is absent. -pub(crate) const CONFIG_FILE_NAME: &str = "weaveffi.toml"; +pub const CONFIG_FILE_NAME: &str = "weaveffi.toml"; /// Declarative registry of every language target the CLI can drive. /// @@ -52,20 +57,16 @@ pub(crate) const CONFIG_FILE_NAME: &str = "weaveffi.toml"; /// invocation (below) expands to: /// * the [`Targets`] struct: one typed field per target, deserialized from /// the `[generators.]` tables; -/// * `build`: the ordered, object-safe target list; -/// * `stamp_input_basename` / `fan_strip_module_prefix`: the `finalize` -/// fan-outs. +/// * `build`: the ordered, object-safe target list. /// /// Adding a language is therefore a one-line edit here (plus the generator /// crate and its workspace dependency), keeping WeaveFFI's promise that a new /// platform is a code-gen change, not a rewrite. /// -/// Per-entry syntax: `"" => : via [, strip]`. -/// The trailing `strip` tags targets whose config honors -/// `global.strip_module_prefix`. +/// Per-entry syntax: `"" => : via `. macro_rules! cli_targets { ( - $( $key:literal => $field:ident : $cfg:ident via $gen:ident $(, $strip:ident)? ),* $(,)? + $( $key:literal => $field:ident : $cfg:ident via $gen:ident ),* $(,)? ) => { /// Per-target generator options, one field per registered language. /// Each field is the generator crate's own `Config` type, so adding a @@ -73,69 +74,116 @@ macro_rules! cli_targets { /// Generated by `cli_targets!`. #[derive(Debug, Default, Clone, Deserialize)] #[serde(default, deny_unknown_fields)] - pub(crate) struct Targets { - $( pub(crate) $field: $cfg, )* + pub struct Targets { + $( + #[doc = concat!("The `[generators.", $key, "]` table.")] + pub $field: $cfg, + )* } impl Targets { /// Every registered target name, in registry order. - pub(crate) const NAMES: &'static [&'static str] = &[ $( $key, )* ]; + pub const NAMES: &'static [&'static str] = &[ $( $key, )* ]; /// Build the heterogeneous list of object-safe targets that the /// orchestrator and dry-run paths share. Order matches the entry /// order in the `cli_targets!` invocation for stable output. - pub(crate) fn build(&self) -> Vec> { + pub fn build(&self) -> Vec> { vec![ $( Box::new(ConfiguredBackend::new($gen, self.$field.clone())), )* ] } - - /// Stamp the IDL basename onto every per-target config. - fn stamp_input_basename(&mut self, input_basename: Option) { - $( self.$field.input_basename = input_basename.clone(); )* - } - - /// Fan `global.strip_module_prefix` out to every target whose - /// config honors it (those tagged `strip` in `cli_targets!`). - fn fan_strip_module_prefix(&mut self, value: bool) { - $( $( cli_targets!(@strip self, $field, value, $strip); )? )* - } } }; - (@strip $self:ident, $field:ident, $value:ident, strip) => { - $self.$field.strip_module_prefix = $value; - }; } cli_targets! { "c" => c: CConfig via CGenerator, "cpp" => cpp: CppConfig via CppGenerator, - "swift" => swift: SwiftConfig via SwiftGenerator, strip, - "kotlin" => kotlin: KotlinConfig via KotlinGenerator, strip, + "swift" => swift: SwiftConfig via SwiftGenerator, + "kotlin" => kotlin: KotlinConfig via KotlinGenerator, "node" => node: NodeConfig via NodeGenerator, "wasm" => wasm: WasmConfig via WasmGenerator, - "python" => python: PythonConfig via PythonGenerator, strip, - "dotnet" => dotnet: DotnetConfig via DotnetGenerator, strip, - "dart" => dart: DartConfig via DartGenerator, strip, - "go" => go: GoConfig via GoGenerator, strip, - "ruby" => ruby: RubyConfig via RubyGenerator, strip, + "python" => python: PythonConfig via PythonGenerator, + "dotnet" => dotnet: DotnetConfig via DotnetGenerator, + "dart" => dart: DartConfig via DartGenerator, + "go" => go: GoConfig via GoGenerator, + "ruby" => ruby: RubyConfig via RubyGenerator, } -/// Knobs that affect multiple generators or the orchestrator itself -/// (the `[global]` table). -#[derive(Debug, Default, Clone, Deserialize)] +/// The `[package]` table: the distribution identity shared by every +/// generated manifest, plus where `weaveffi package` writes its artifacts. +#[derive(Debug, Default, Clone)] +pub struct PackageTable { + /// The identity keys (`name`, `version`, `license`, ...). + pub identity: Package, + /// The dist directory for `weaveffi package` (default `dist`). + pub dist: Option, +} + +impl<'de> Deserialize<'de> for PackageTable { + /// Split `dist` off and parse the remaining keys as the identity, which + /// keeps rejecting unknown keys. + fn deserialize>(deserializer: D) -> Result { + use serde::de::Error; + let mut table = toml::Table::deserialize(deserializer)?; + let dist = table + .remove("dist") + .map(|v| v.try_into::()) + .transpose() + .map_err(D::Error::custom)?; + let identity = toml::Value::Table(table) + .try_into::() + .map_err(D::Error::custom)?; + Ok(Self { identity, dist }) + } +} + +/// The `[build]` table: how `weaveffi build` (and `weaveffi package`) +/// compile a Rust producer. +#[derive(Debug, Clone, Deserialize)] #[serde(default, deny_unknown_fields)] -pub(crate) struct GlobalConfig { - /// Shorthand: sets `strip_module_prefix` on every per-target config that - /// supports it. Stripping is the default, so `strip_module_prefix = - /// false` here restores module-prefixed wrapper names everywhere at - /// once. When set, this overrides per-target sections; omit it to - /// control targets individually. - pub(crate) strip_module_prefix: Option, - /// Shell command executed once before any generator runs. - pub(crate) pre_generate: Option, - /// Shell command executed once after every generator succeeds. - pub(crate) post_generate: Option, +pub struct BuildTable { + /// The producer crate's `Cargo.toml` (default: the project's crate, + /// else `Cargo.toml` next to `weaveffi.toml`). + pub manifest: Option, + /// The platform ids to build when `--platforms` isn't given (default: + /// the host). + pub platforms: Option>, + /// Build with the release profile. + pub release: bool, + /// `MACOSX_DEPLOYMENT_TARGET` for macOS builds. + pub macos_deployment_target: String, + /// `IPHONEOS_DEPLOYMENT_TARGET` for iOS builds. + pub ios_deployment_target: String, + /// The minimum Android API level, which selects the NDK compiler. + pub android_api: u32, +} + +impl Default for BuildTable { + fn default() -> Self { + let settings = BuildSettings::default(); + Self { + manifest: None, + platforms: None, + release: settings.release, + macos_deployment_target: settings.macos_deployment_target, + ios_deployment_target: settings.ios_deployment_target, + android_api: settings.android_api, + } + } +} + +impl BuildTable { + /// The build settings, in debug when `debug` overrides `release`. + pub fn settings(&self, debug: bool) -> BuildSettings { + BuildSettings { + release: self.release && !debug, + macos_deployment_target: self.macos_deployment_target.clone(), + ios_deployment_target: self.ios_deployment_target.clone(), + android_api: self.android_api, + } + } } /// The `[project]` table: what to generate from, where to, and for which @@ -143,32 +191,32 @@ pub(crate) struct GlobalConfig { /// are relative to the directory containing `weaveffi.toml`. #[derive(Debug, Default, Clone, Deserialize)] #[serde(default, deny_unknown_fields)] -pub(crate) struct ProjectTable { - /// The API definition: annotated Rust source or an IDL document. - pub(crate) input: Option, - /// The output directory for `generate` and `diff` (default `generated`). - pub(crate) out: Option, +pub struct ProjectTable { + /// The API definition: a Rust producer crate (its directory, usually + /// `"."`, or its `Cargo.toml`) or an IDL document. + pub input: Option, + /// The output directory for `generate` and `diff` (default `bindings`). + pub out: Option, /// The targets to generate when `--target` is not given (default: all). - pub(crate) targets: Option>, + pub targets: Option>, } -/// The whole `weaveffi.toml`: project paths, package identity, global knobs, -/// and per-target generator options. +/// The whole `weaveffi.toml`: project paths, package identity, build +/// settings, and per-target generator options. #[derive(Debug, Default, Clone, Deserialize)] #[serde(default, deny_unknown_fields)] -pub(crate) struct ProjectConfig { +pub struct ProjectConfig { /// The `[project]` table. - pub(crate) project: ProjectTable, - /// The `[package]` table: distribution identity shared by every - /// generated manifest. - pub(crate) package: Package, - /// The `[global]` table. - pub(crate) global: GlobalConfig, + pub project: ProjectTable, + /// The `[package]` table. + pub package: PackageTable, + /// The `[build]` table. + pub build: BuildTable, /// The `[generators.]` tables. - pub(crate) generators: Targets, + pub generators: Targets, /// Where the config was loaded from, when it came from a file. #[serde(skip)] - pub(crate) source: Option, + pub source: Option, } impl ProjectConfig { @@ -180,7 +228,7 @@ impl ProjectConfig { /// /// Returns an error when an explicitly named file cannot be read, or when /// any located file fails to parse. - pub(crate) fn load(explicit: Option<&str>, input: &Utf8Path) -> Result { + pub fn load(explicit: Option<&str>, input: &Utf8Path) -> Result { let path = match explicit { Some(p) => Some(Utf8PathBuf::from(p)), None => discover(input), @@ -197,7 +245,7 @@ impl ProjectConfig { /// /// Returns an error when the file cannot be read or is not a valid /// `weaveffi.toml`. - pub(crate) fn from_file(path: &Utf8Path) -> Result { + pub fn from_file(path: &Utf8Path) -> Result { let contents = std::fs::read_to_string(path.as_std_path()) .into_diagnostic() .wrap_err_with(|| format!("failed to read config file: {path}"))?; @@ -208,30 +256,16 @@ impl ProjectConfig { Ok(cfg) } - /// Fan `global.strip_module_prefix` out to every per-target config that - /// honors it, then stamp the IDL `input_basename` on every per-target - /// config. - pub(crate) fn finalize(&mut self, input_basename: Option) { - if let Some(v) = self.global.strip_module_prefix { - self.generators.fan_strip_module_prefix(v); - } - self.generators.stamp_input_basename(input_basename); - } - /// Resolve the project for a command: the config (explicit, discovered /// from `input`, or discovered from the current directory) and the input - /// file (the argument, else `[project] input`), finalized against the - /// input. + /// file (the argument, else `[project] input`). /// /// # Errors /// /// Returns an error when no input is given and no discovered config /// names one, or when a config file cannot be read or parsed. - pub(crate) fn locate( - explicit: Option<&str>, - input: Option<&str>, - ) -> Result<(Self, Utf8PathBuf)> { - let (mut cfg, input) = match input { + pub fn locate(explicit: Option<&str>, input: Option<&str>) -> Result<(Self, Utf8PathBuf)> { + let (cfg, input) = match input { Some(i) => ( Self::load(explicit, Utf8Path::new(i))?, Utf8PathBuf::from(i), @@ -260,13 +294,12 @@ impl ProjectConfig { (cfg, input) } }; - cfg.finalize(input.file_name().map(str::to_string)); Ok((cfg, input)) } /// Resolve `path` against the directory holding the config file (or the /// current directory when there is no file). - pub(crate) fn relative_to_config(&self, path: &Utf8Path) -> Utf8PathBuf { + pub fn relative_to_config(&self, path: &Utf8Path) -> Utf8PathBuf { match self.source.as_deref().and_then(Utf8Path::parent) { Some(dir) if path.is_relative() && !dir.as_str().is_empty() => dir.join(path), _ => path.to_path_buf(), @@ -274,12 +307,23 @@ impl ProjectConfig { } /// The output directory: `out` if given on the command line, else - /// `[project] out`, else `./generated`. - pub(crate) fn out_dir(&self, out: Option<&str>) -> Utf8PathBuf { + /// `[project] out`, else `bindings` next to `weaveffi.toml`. + pub fn out_dir(&self, out: Option<&str>) -> Utf8PathBuf { match (out, &self.project.out) { (Some(o), _) => Utf8PathBuf::from(o), (None, Some(o)) => self.relative_to_config(o), - (None, None) => Utf8PathBuf::from("./generated"), + (None, None) => self.relative_to_config(Utf8Path::new("bindings")), + } + } + + /// The dist directory for `weaveffi package`: `out` if given on the + /// command line, else `[package] dist`, else `dist` next to + /// `weaveffi.toml`. + pub fn dist_dir(&self, out: Option<&str>) -> Utf8PathBuf { + match (out, &self.package.dist) { + (Some(o), _) => Utf8PathBuf::from(o), + (None, Some(d)) => self.relative_to_config(d), + (None, None) => self.relative_to_config(Utf8Path::new("dist")), } } @@ -292,7 +336,7 @@ impl ProjectConfig { /// Returns an error naming any filter entry that is not a registered /// target, so a typo in `--target` fails instead of silently generating /// nothing. - pub(crate) fn select_targets(&self, filter: Option<&str>) -> Result>> { + pub fn select_targets(&self, filter: Option<&str>) -> Result>> { let wanted: Option> = match filter { Some(t) => Some( t.split(',') @@ -337,7 +381,7 @@ fn current_dir() -> std::io::Result { /// Walk from the input file's directory up to the filesystem root and return /// the first `weaveffi.toml` found. -fn discover(input: &Utf8Path) -> Option { +pub(crate) fn discover(input: &Utf8Path) -> Option { let start = input .parent() .filter(|p| !p.as_str().is_empty()) @@ -365,7 +409,7 @@ mod tests { } #[test] - fn parses_package_global_and_generator_tables() { + fn parses_package_build_and_generator_tables() { let dir = tempfile::tempdir().unwrap(); let cfg_path = write( dir.path(), @@ -374,28 +418,108 @@ mod tests { "[package]\n", "name = \"kvstore\"\n", "version = \"1.2.0\"\n", - "[global]\n", - "strip_module_prefix = true\n", + "dist = \"out/dist\"\n", + "[build]\n", + "platforms = [\"darwin-arm64\", \"ios-arm64\"]\n", + "release = false\n", + "macos_deployment_target = \"12.0\"\n", + "android_api = 24\n", "[generators.swift]\n", - "module_name = \"MyApp\"\n", + "name = \"MyApp\"\n", + "min_ios = \"15.0\"\n", "[generators.kotlin]\n", - "package = \"com.example.myapp\"\n", + "name = \"com.example.myapp\"\n", + "min_sdk = 24\n", + "[generators.python]\n", + "requires_python = \">=3.10\"\n", + "[generators.node]\n", + "node_engine = \">=20\"\n", + "[generators.dart]\n", + "sdk = \">=3.5.0 <4.0.0\"\n", ), ); - let mut cfg = ProjectConfig::from_file(&cfg_path).unwrap(); - assert_eq!(cfg.package.name.as_deref(), Some("kvstore")); - assert_eq!(cfg.package.version.as_deref(), Some("1.2.0")); - assert_eq!(cfg.generators.swift.module_name.as_deref(), Some("MyApp")); + let cfg = ProjectConfig::from_file(&cfg_path).unwrap(); + assert_eq!(cfg.package.identity.name.as_deref(), Some("kvstore")); + assert_eq!(cfg.package.identity.version.as_deref(), Some("1.2.0")); + let root = cfg_path.parent().unwrap(); + assert_eq!(cfg.dist_dir(None), root.join("out/dist")); + assert_eq!( + cfg.dist_dir(Some("elsewhere")), + Utf8PathBuf::from("elsewhere") + ); + assert_eq!( + cfg.build.platforms.as_deref(), + Some(&["darwin-arm64".to_string(), "ios-arm64".to_string()][..]) + ); + let settings = cfg.build.settings(false); + assert!(!settings.release); + assert_eq!(settings.macos_deployment_target, "12.0"); + assert_eq!(settings.ios_deployment_target, "13.0"); + assert_eq!(settings.android_api, 24); + assert_eq!(cfg.generators.swift.name.as_deref(), Some("MyApp")); + assert_eq!(cfg.generators.swift.min_ios, "15.0"); + assert_eq!(cfg.generators.swift.min_macos, "11.0"); assert_eq!(cfg.generators.kotlin.package(), Some("com.example.myapp")); - assert_eq!(cfg.global.strip_module_prefix, Some(true)); - assert_eq!(cfg.source.as_deref(), Some(cfg_path.as_path())); - - cfg.finalize(Some("api.yml".into())); - assert!(cfg.generators.python.strip_module_prefix); assert_eq!( - cfg.generators.swift.input_basename.as_deref(), - Some("api.yml") + ( + cfg.generators.kotlin.min_sdk, + cfg.generators.kotlin.compile_sdk + ), + (24, 35) ); + assert_eq!(cfg.generators.python.requires_python, ">=3.10"); + assert_eq!(cfg.generators.node.node_engine, ">=20"); + assert_eq!(cfg.generators.dart.sdk, ">=3.5.0 <4.0.0"); + assert_eq!(cfg.source.as_deref(), Some(cfg_path.as_path())); + } + + #[test] + fn build_and_package_defaults() { + let cfg = ProjectConfig::default(); + let settings = cfg.build.settings(false); + assert!(settings.release); + assert_eq!(settings.macos_deployment_target, "11.0"); + assert_eq!(settings.ios_deployment_target, "13.0"); + assert_eq!(settings.android_api, 21); + assert!(!cfg.build.settings(true).release); + assert_eq!(cfg.dist_dir(None), Utf8PathBuf::from("dist")); + assert_eq!(cfg.out_dir(None), Utf8PathBuf::from("bindings")); + } + + #[test] + fn removed_and_unknown_keys_are_errors() { + let dir = tempfile::tempdir().unwrap(); + for (contents, needle) in [ + ("[global]\npre_generate = \"true\"\n", "global"), + ("[package]\nname = \"x\"\ndistdir = \"d\"\n", "distdir"), + ("[build]\nplatform = [\"linux-x64\"]\n", "platform"), + ("[generators.swift]\nmodule_name = \"X\"\n", "module_name"), + ("[generators.kotlin]\npackage = \"x\"\n", "package"), + ("[generators.node]\npackage_name = \"x\"\n", "package_name"), + ("[generators.go]\nmodule_path = \"x\"\n", "module_path"), + ("[generators.dotnet]\nnamespace = \"X\"\n", "namespace"), + ] { + let cfg_path = write(dir.path(), CONFIG_FILE_NAME, contents); + let err = ProjectConfig::from_file(&cfg_path).unwrap_err(); + assert!(format!("{err:?}").contains(needle), "{contents}: {err:?}"); + } + } + + #[test] + fn removed_strip_module_prefix_keys_are_errors() { + let dir = tempfile::tempdir().unwrap(); + for table in ["[package]", "[generators.python]"] { + let cfg_path = write( + dir.path(), + CONFIG_FILE_NAME, + &format!("{table}\nstrip_module_prefix = false\n"), + ); + let err = ProjectConfig::from_file(&cfg_path).unwrap_err(); + assert!( + format!("{err:?}").contains("strip_module_prefix"), + "{err:?}" + ); + } } #[test] @@ -420,20 +544,19 @@ mod tests { CONFIG_FILE_NAME, "[package]\nname = \"found\"\n", ); - let input = write(&nested, "api.yml", "version: \"0.10.0\"\nmodules: []\n"); + let input = write(&nested, "api.yml", "version: \"0.11.0\"\nmodules: []\n"); let cfg = ProjectConfig::load(None, &input).unwrap(); - assert_eq!(cfg.package.name.as_deref(), Some("found")); + assert_eq!(cfg.package.identity.name.as_deref(), Some("found")); } #[test] fn defaults_when_no_config_exists() { let dir = tempfile::tempdir().unwrap(); - let input = write(dir.path(), "api.yml", "version: \"0.10.0\"\nmodules: []\n"); + let input = write(dir.path(), "api.yml", "version: \"0.11.0\"\nmodules: []\n"); let cfg = ProjectConfig::load(None, &input).unwrap(); assert!(cfg.source.is_none(), "unexpected config: {:?}", cfg.source); - assert_eq!(cfg.global.strip_module_prefix, None); - assert!(cfg.package.name.is_none()); + assert!(cfg.package.identity.name.is_none()); } #[test] @@ -468,7 +591,7 @@ mod tests { CONFIG_FILE_NAME, "[project]\ninput = \"api.yml\"\nout = \"bindings\"\ntargets = [\"c\", \"python\"]\n", ); - write(dir.path(), "api.yml", "version: \"0.10.0\"\nmodules: []\n"); + write(dir.path(), "api.yml", "version: \"0.11.0\"\nmodules: []\n"); let (cfg, input) = ProjectConfig::locate(Some(cfg_path.as_str()), None).unwrap(); let root = cfg_path.parent().unwrap(); assert_eq!(input, root.join("api.yml")); @@ -487,45 +610,4 @@ mod tests { let err = ProjectConfig::locate(Some(bare.as_str()), None).unwrap_err(); assert!(format!("{err}").contains("[project] input"), "{err}"); } - - /// Every registered target's *declared* [`TargetCapabilities`] must match - /// the documented feature matrix. This pins the declarations so a backend - /// cannot silently claim (or drop) a gated feature: the matrix here, the - /// generator's `capabilities()`, and `docs/src/generators/README.md` are - /// changed together or this test fails. - /// - /// Declaration accuracy is only half the contract. The companion guard in - /// `tests/no_silent_stubs.rs` proves a *declared* feature is actually - /// emitted (no `unimplemented!`/`todo!` source, no stub markers in output), - /// and the conformance harness proves it runs. - #[test] - fn declared_capabilities_match_documented_matrix() { - use std::collections::BTreeMap; - use weaveffi_gen::capabilities::TargetCapabilities; - - // Every target is full. Wasm delivers callbacks/listeners - // synchronously through function-table trampolines in its standard - // loader (Emscripten mode stubs them, mirroring its async stubs). - let expected: Vec<(&str, TargetCapabilities)> = Targets::NAMES - .iter() - .map(|n| (*n, TargetCapabilities::full())) - .collect(); - - let targets = Targets::default().build(); - let actual: BTreeMap<&str, TargetCapabilities> = targets - .iter() - .map(|g| (g.name(), g.capabilities())) - .collect(); - - assert_eq!(actual.len(), expected.len()); - for (name, caps) in expected { - let got = actual - .get(name) - .unwrap_or_else(|| panic!("target '{name}' is missing from the registry")); - assert_eq!( - *got, caps, - "declared capabilities for '{name}' drifted from the documented matrix", - ); - } - } } diff --git a/crates/weaveffi-cli/src/extract.rs b/crates/weaveffi-cli/src/extract.rs deleted file mode 100644 index 5fc15d33..00000000 --- a/crates/weaveffi-cli/src/extract.rs +++ /dev/null @@ -1,84 +0,0 @@ -//! `weaveffi extract`: read annotated Rust source and emit the IDL. -//! -//! This is thin CLI glue around [`weaveffi_model::rust`], the single Rust-to-IR -//! bridge that the `#[weaveffi::module]` proc-macro also uses. Because both the -//! macro (which builds the C ABI scaffolding) and this command read the *same* -//! extraction, the emitted IDL and the producer's compiled symbols cannot -//! drift. -//! -//! Validation is **fail-loud by default**: an extracted API that would not -//! generate (an undeclared type reference, a duplicate name, a listener -//! pointing at a missing callback, ...) aborts with the diagnostic rather than -//! emitting a silently-broken IDL. Passing `warn` downgrades those errors to a -//! `warning:` line and emits the IDL anyway, which is useful when bootstrapping -//! from source that references types it does not yet declare. - -use miette::{miette, IntoDiagnostic, WrapErr}; - -/// Read the annotated Rust file at `input`, extract its [`Api`](weaveffi_model::ir::Api), -/// validate, and serialize to the requested `format` (`yaml`, `json`, or `toml`). -/// -/// # Errors -/// -/// Returns an error when the source cannot be read or parsed, when extraction -/// fails, when validation fails and `warn` is not set, or when serialization or -/// writing the output fails. -pub(crate) fn cmd_extract( - input: &str, - output: Option<&str>, - format: crate::IdlFormat, - lenient: bool, - quiet: bool, -) -> miette::Result<()> { - let source = std::fs::read_to_string(input) - .into_diagnostic() - .wrap_err_with(|| format!("failed to read source file: {input}"))?; - - let api = weaveffi_model::rust::api_from_src_stringly(&source) - .map_err(|e| miette!("failed to extract API from Rust source {input}:\n{e}"))?; - - // Serialize the resolved form when the API validates (so cross-module - // references are emitted qualified and stable across runs); fall back - // to the raw extracted document under `--warn`. - let api = match weaveffi_model::validate::validate_api(api.clone(), None) { - Ok(resolved) => resolved.api().clone(), - Err(e) => { - if lenient { - eprintln!("warning: {e}"); - api - } else { - return Err(miette!( - "{e:?}\n\nThe extracted API does not validate, so it would not \ - generate. Fix the source (e.g. declare the referenced types), \ - or pass `--lenient` to emit the IDL anyway." - )); - } - } - }; - - let serialized = match format { - crate::IdlFormat::Yaml => serde_yaml::to_string(&api) - .into_diagnostic() - .wrap_err("failed to serialize API as YAML")?, - crate::IdlFormat::Json => serde_json::to_string_pretty(&api) - .into_diagnostic() - .wrap_err("failed to serialize API as JSON")?, - crate::IdlFormat::Toml => toml::to_string_pretty(&api) - .into_diagnostic() - .wrap_err("failed to serialize API as TOML")?, - }; - - match output { - Some(path) => { - std::fs::write(path, &serialized) - .into_diagnostic() - .wrap_err_with(|| format!("failed to write output file: {path}"))?; - if !quiet { - println!("Extracted API written to {path}"); - } - } - None => print!("{serialized}"), - } - - Ok(()) -} diff --git a/crates/weaveffi-gen/src/lang.rs b/crates/weaveffi-cli/src/lang.rs similarity index 90% rename from crates/weaveffi-gen/src/lang.rs rename to crates/weaveffi-cli/src/lang.rs index f7616be8..2dbed97f 100644 --- a/crates/weaveffi-gen/src/lang.rs +++ b/crates/weaveffi-cli/src/lang.rs @@ -412,10 +412,35 @@ pub fn escape_ident(name: &str, keywords: &[&str]) -> String { } } +/// Escape the target spelling of an interface member (`name`, already +/// case-converted and keyword-escaped) away from `members`, the names the +/// target's object wrapper declares or inherits (`close`, `Dispose`, +/// `hashCode`, ...): a name in `members` gains a trailing `_`, the same rule +/// as [`escape_ident`]. +/// +/// Without it a member spelled like one of those would replace the +/// wrapper's own (a Ruby method `close` overriding the release), or fail to +/// compile (a second Go `Close`). The rule is idempotent as long as no +/// table lists a name ending in `_`. +pub fn escape_member(name: &str, members: &[&str]) -> String { + if members.contains(&name) { + format!("{name}_") + } else { + name.to_string() + } +} + #[cfg(test)] mod tests { use super::*; + #[test] + fn reserved_members_gain_a_trailing_underscore() { + assert_eq!(escape_member("close", &["close", "handle"]), "close_"); + assert_eq!(escape_member("close_", &["close", "handle"]), "close_"); + assert_eq!(escape_member("closed", &["close", "handle"]), "closed"); + } + #[test] fn tables_are_sorted_for_binary_search() { for (label, table) in [ diff --git a/crates/weaveffi-cli/src/lib.rs b/crates/weaveffi-cli/src/lib.rs new file mode 100644 index 00000000..dfd7570f --- /dev/null +++ b/crates/weaveffi-cli/src/lib.rs @@ -0,0 +1,33 @@ +//! The library behind the `weaveffi` command-line tool: the +//! [`Project`](project::Project) that locates a project and loads its API +//! (from an IDL, or from a Rust producer's built [`library`]), the +//! `weaveffi.toml` [`config`], the [`LanguageBackend`](backend::LanguageBackend) +//! trait, the code-generation [`Orchestrator`](codegen::Orchestrator), the +//! shared C ABI declaration renderer, the eleven language generators under +//! [`targets`], and the [`build`] and [`package`] layers that turn a producer +//! crate into installable per-ecosystem artifacts. +//! +//! Every generator renders from the validated +//! [`Model`](weaveffi_model::model::Model) alone, so symbol names and +//! parameter lowering are computed once and shared. +#![deny(missing_docs)] +#![warn(clippy::missing_errors_doc)] +#![warn(clippy::missing_panics_doc)] +#![warn(clippy::doc_markdown)] + +pub mod backend; +pub mod build; +pub mod cabi; +pub mod cache; +pub mod cargo; +pub mod codegen; +pub mod config; +pub mod lang; +pub mod library; +pub mod manifest; +pub mod package; +pub mod platform; +pub mod project; +mod report; +pub mod targets; +pub mod utils; diff --git a/crates/weaveffi-cli/src/library.rs b/crates/weaveffi-cli/src/library.rs new file mode 100644 index 00000000..d1e0e252 --- /dev/null +++ b/crates/weaveffi-cli/src/library.rs @@ -0,0 +1,161 @@ +//! Reading a Rust producer's API out of its built library. +//! +//! The `#[weaveffi::module]` macro embeds one metadata +//! [`Frame`] per declaration in the producer's +//! library. [`read_frames`] finds them with the `object` crate: in a native +//! library (ELF, Mach-O, or PE, or every member of a static archive) by +//! walking the symbol tables for the `{PREFIX}_META_{HASH16}` statics and +//! reading each one's bytes from the section that holds it; in a `.wasm` +//! module from the `weaveffi_meta` custom section, which the linker fills +//! with every frame back to back. + +use std::collections::BTreeSet; + +use anyhow::{bail, Context, Result}; +use camino::Utf8Path; +use object::read::archive::ArchiveFile; +use object::{BinaryFormat, Object, ObjectSection, ObjectSymbol, SymbolSection}; +use weaveffi_model::meta::{self, Frame}; + +/// Every metadata frame in the library at `path`: a shared library +/// (`.so`, `.dylib`, `.dll`), a static archive (`.a`, `.lib`), or a `.wasm` +/// module. The frames of every crate linked into the library are returned; +/// [`meta::assemble`] keeps one crate's. +/// +/// # Errors +/// +/// Returns an error when the file can't be read or isn't an object file +/// `object` understands, or when a frame is malformed. +pub fn read_frames(path: &Utf8Path) -> Result> { + let data = std::fs::read(path).with_context(|| format!("failed to read {path}"))?; + let mut frames = Vec::new(); + if let Ok(archive) = ArchiveFile::parse(&*data) { + for member in archive.members() { + let member = member.with_context(|| format!("{path} is a malformed archive"))?; + let bytes = member + .data(&*data) + .with_context(|| format!("{path} is a malformed archive"))?; + // Archives also carry symbol tables and other non-object members. + if let Ok(file) = object::File::parse(bytes) { + frames.extend(object_frames(&file).with_context(|| { + format!( + "{path}({}) has malformed WeaveFFI metadata", + String::from_utf8_lossy(member.name()) + ) + })?); + } + } + return Ok(frames); + } + let file = object::File::parse(&*data) + .with_context(|| format!("{path} isn't a library WeaveFFI can read"))?; + object_frames(&file).with_context(|| format!("{path} has malformed WeaveFFI metadata")) +} + +/// The frames in one object file. +fn object_frames(file: &object::File<'_>) -> Result> { + if file.format() == BinaryFormat::Wasm { + return match file.section_by_name(meta::SECTION) { + Some(section) => Ok(meta::decode(section.data()?)?), + None => Ok(Vec::new()), + }; + } + // Statically linked and stripped libraries keep the statics in different + // tables (the symbol table, the dynamic symbol table, a PE export + // table), so look in all of them and read each symbol once. + let mut candidates: Vec<(String, u64, Option)> = Vec::new(); + for symbol in file.symbols().chain(file.dynamic_symbols()) { + let Ok(name) = symbol.name() else { continue }; + if !meta::is_symbol(name) { + continue; + } + let section = match symbol.section() { + SymbolSection::Section(index) => Some(index), + _ => continue, + }; + candidates.push((name.to_string(), symbol.address(), section)); + } + for export in file.exports().unwrap_or_default() { + let name = String::from_utf8_lossy(export.name()).into_owned(); + if meta::is_symbol(&name) { + candidates.push((name, export.address(), None)); + } + } + let mut seen = BTreeSet::new(); + let mut frames = Vec::new(); + for (name, address, section) in candidates { + if !seen.insert(name.trim_start_matches('_').to_string()) { + continue; + } + let Some(bytes) = data_at(file, address, section) else { + bail!("the data of `{name}` isn't in the file"); + }; + let len = meta::frame_len(bytes)?; + let Some(frame) = bytes.get(..len) else { + bail!("`{name}` is truncated"); + }; + frames.extend(meta::decode(frame)?); + } + Ok(frames) +} + +/// The file's bytes from `address` to the end of the section holding it +/// (`section`, when the symbol names it). +fn data_at<'d>( + file: &object::File<'d>, + address: u64, + section: Option, +) -> Option<&'d [u8]> { + let section = match section { + Some(index) => file.section_by_index(index).ok()?, + None => file + .sections() + .find(|s| s.address() <= address && address < s.address().saturating_add(s.size()))?, + }; + let data = section.data().ok()?; + let offset = usize::try_from(address.checked_sub(section.address())?).ok()?; + data.get(offset..) +} + +/// The base name a library file is loaded by: `libkv.so`, `libkv.dylib`, +/// `libkv.a`, `kv.dll`, `kv.lib`, and `kv.wasm` are all `kv`. +#[must_use] +pub fn library_name(path: &Utf8Path) -> String { + let stem = path.file_stem().unwrap_or_default(); + match path.extension() { + Some("so" | "dylib" | "a") => stem.strip_prefix("lib").unwrap_or(stem).to_string(), + _ => stem.to_string(), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn library_names_drop_platform_decoration() { + for (file, name) in [ + ("target/debug/libkv_store.so", "kv_store"), + ("libkv.dylib", "kv"), + ("libkv.a", "kv"), + ("kv.dll", "kv"), + ("kv.lib", "kv"), + ("out/kv.wasm", "kv"), + ] { + assert_eq!(library_name(Utf8Path::new(file)), name, "{file}"); + } + } + + #[test] + fn files_without_metadata_have_no_frames_and_junk_is_an_error() { + let dir = tempfile::tempdir().unwrap(); + let junk = Utf8Path::from_path(dir.path()).unwrap().join("libjunk.so"); + std::fs::write(&junk, b"\x00fake-native\x01").unwrap(); + let err = read_frames(&junk).unwrap_err(); + assert!(format!("{err:#}").contains("isn't a library"), "{err:#}"); + // A real object file with no frames: this test binary. + let me = std::env::current_exe().unwrap(); + let me = Utf8Path::from_path(&me).unwrap(); + assert!(read_frames(me).unwrap().is_empty()); + } +} diff --git a/crates/weaveffi-cli/src/main.rs b/crates/weaveffi-cli/src/main.rs index 29c35496..7d186932 100644 --- a/crates/weaveffi-cli/src/main.rs +++ b/crates/weaveffi-cli/src/main.rs @@ -1,21 +1,28 @@ //! `weaveffi` command-line entry point: `clap` definitions and dispatch. //! -//! Each subcommand's implementation lives in `commands` (or `extract` for the -//! Rust-source extractor); project configuration and the generator registry -//! live in `config`. +//! Each subcommand's implementation lives in `commands`, on top of the +//! library's `project` (locating a project and loading its API) and +//! `config` (`weaveffi.toml` and the generator registry). mod commands; -mod config; -mod extract; -mod report; use clap::{CommandFactory, Parser, Subcommand, ValueEnum}; +use commands::Locate; use miette::{IntoDiagnostic, Result, WrapErr}; use weaveffi_model::ir::CURRENT_SCHEMA_VERSION; -const INPUT_HELP: &str = "Annotated Rust source (.rs) or an IDL document (yaml|yml|json|toml); \ +const PLATFORMS_HELP: &str = "Comma-separated platform ids: darwin-arm64, darwin-x64, linux-x64, \ + linux-arm64, windows-x64, ios-arm64, ios-sim-arm64, ios-sim-x64, \ + android-arm64, android-x64, wasm32 (default: `[build] platforms`, \ + else the host)"; + +const INPUT_HELP: &str = "A Rust producer crate (its directory or Cargo.toml), whose API is read \ + from its built library, or an IDL document (yaml|yml|json|toml); \ defaults to `[project] input` from the nearest weaveffi.toml"; +const LIBRARY_HELP: &str = "Read a Rust producer's API from this built library (cdylib, \ + staticlib, or .wasm) instead of building the crate"; + #[derive(Parser, Debug)] #[command( name = "weaveffi", @@ -75,7 +82,7 @@ enum Commands { Generate { #[arg(help = INPUT_HELP)] input: Option, - /// Output directory (default: `[project] out`, else ./generated) + /// Output directory (default: `[project] out`, else ./bindings) #[arg(short, long)] out: Option, /// Comma-separated targets: c, cpp, swift, kotlin, node, wasm, python, dotnet, dart, go, ruby (default: `[project] targets`, else all) @@ -87,12 +94,28 @@ enum Commands { /// Print advisory lints after validation #[arg(long)] warn: bool, - /// Regenerate every target even if its inputs and files are unchanged - #[arg(long)] - force: bool, /// Validate and list the files that would be written, without writing #[arg(long)] dry_run: bool, + #[arg(long, help = LIBRARY_HELP)] + library: Option, + /// Build a Rust producer's library in release mode + #[arg(long)] + release: bool, + }, + /// Build a Rust producer's debug library, generate, and point the bindings at it + Dev { + #[arg(help = INPUT_HELP)] + input: Option, + /// Output directory (default: `[project] out`, else ./bindings) + #[arg(short, long)] + out: Option, + /// Comma-separated targets (default: `[project] targets`, else all) + #[arg(short, long)] + target: Option, + /// Path to weaveffi.toml (default: the nearest one at or above the input) + #[arg(long)] + config: Option, }, /// Validate an API definition without generating anything Validate { @@ -107,12 +130,17 @@ enum Commands { /// Output format #[arg(long, value_enum, default_value_t = ReportFormat::Human)] format: ReportFormat, + #[arg(long, help = LIBRARY_HELP)] + library: Option, + /// Build a Rust producer's library in release mode + #[arg(long)] + release: bool, }, - /// Show how regenerating would change the output directory (writes nothing, runs no hooks) + /// Show how regenerating would change the output directory (writes nothing) Diff { #[arg(help = INPUT_HELP)] input: Option, - /// Output directory to compare against (default: `[project] out`, else ./generated) + /// Output directory to compare against (default: `[project] out`, else ./bindings) #[arg(short, long)] out: Option, /// Comma-separated targets to compare (default: `[project] targets`, else all) @@ -124,47 +152,83 @@ enum Commands { /// Print only a summary and exit 2 if files differ, 3 if files would be added or removed #[arg(long)] check: bool, + #[arg(long, help = LIBRARY_HELP)] + library: Option, + /// Build a Rust producer's library in release mode + #[arg(long)] + release: bool, + }, + /// Cross-compile the Rust producer per platform into target/weaveffi// + Build { + #[arg(help = INPUT_HELP)] + input: Option, + /// Comma-separated platform ids (default: `[build] platforms`, else the host) + #[arg(long, help = PLATFORMS_HELP)] + platforms: Option, + /// Comma-separated targets whose C glue to prebuild (default: `[project] targets`, else all) + #[arg(short, long)] + target: Option, + /// Path to weaveffi.toml (default: the nearest one at or above the input) + #[arg(long)] + config: Option, + /// The producer crate's Cargo.toml (default: `[build] manifest`, else the input's crate) + #[arg(long)] + manifest_path: Option, + /// Build with the dev profile instead of `--release` + #[arg(long)] + debug: bool, + /// Print advisory lints after validation + #[arg(long)] + warn: bool, }, - /// Assemble publishable packages that bundle prebuilt native libraries + /// Build (unless --binaries) and write installable artifacts for each target Package { #[arg(help = INPUT_HELP)] input: Option, - /// Output directory for the packaged artifacts - #[arg(short, long, default_value = "./dist")] - out: String, + /// Dist directory for the artifacts (default: `[package] dist`, else ./dist) + #[arg(short, long)] + out: Option, /// Comma-separated targets to package (default: `[project] targets`, else all) #[arg(short, long)] target: Option, /// Path to weaveffi.toml (default: the nearest one at or above the input) #[arg(long)] config: Option, - /// Directory of prebuilt native libraries laid out as `//` + /// Package existing builds laid out as `//` instead of building #[arg(long)] binaries: Option, - /// Cargo package to build as the native producer, once per platform + /// Comma-separated platform ids (default: `[build] platforms`, else the host; with + /// --binaries, every platform directory present) + #[arg(long, help = PLATFORMS_HELP)] + platforms: Option, + /// The producer crate's Cargo.toml (default: `[build] manifest`, else the input's crate) #[arg(long)] - build: Option, - /// Comma-separated platform ids (default: the host): darwin-arm64, darwin-x64, linux-x64, linux-arm64, windows-x64, android-arm64, android-x64, wasm32 + manifest_path: Option, + /// Build with the dev profile instead of `--release` #[arg(long)] - platforms: Option, + debug: bool, /// Print advisory lints after validation #[arg(long)] warn: bool, }, - /// Extract an IDL document from annotated Rust source + /// Print the IDL a Rust producer's built library embeds Extract { - /// Rust source file to extract the API from - input: String, + /// The producer crate (its directory or Cargo.toml); defaults to `[project] input` + input: Option, /// Output file (default: stdout) #[arg(short, long)] output: Option, /// Output format #[arg(short, long, value_enum, default_value_t = IdlFormat::Yaml)] format: IdlFormat, - /// Emit the IDL even if it does not validate (for example, it - /// references types declared elsewhere) + /// Path to weaveffi.toml (default: the nearest one at or above the input) + #[arg(long)] + config: Option, + #[arg(long, help = LIBRARY_HELP)] + library: Option, + /// Build the library in release mode #[arg(long)] - lenient: bool, + release: bool, }, /// Print shell completions Completions { @@ -208,29 +272,55 @@ fn main() -> Result<()> { target, config, warn, - force, dry_run, + library, + release, } => commands::generate::cmd_generate(&commands::generate::GenerateArgs { - input: input.as_deref(), + locate: Locate { + input: input.as_deref(), + config: config.as_deref(), + library: library.as_deref(), + release, + quiet, + }, out: out.as_deref(), targets: target.as_deref(), - config: config.as_deref(), warn, - force, dry_run, - quiet, + })?, + Commands::Dev { + input, + out, + target, + config, + } => commands::dev::cmd_dev(&commands::dev::DevArgs { + locate: Locate { + input: input.as_deref(), + config: config.as_deref(), + library: None, + release: false, + quiet, + }, + out: out.as_deref(), + targets: target.as_deref(), })?, Commands::Validate { input, config, warn, format, + library, + release, } => commands::validate::cmd_validate( - input.as_deref(), - config.as_deref(), + &Locate { + input: input.as_deref(), + config: config.as_deref(), + library: library.as_deref(), + release, + quiet, + }, warn, format == ReportFormat::Json, - quiet, )?, Commands::Diff { input, @@ -238,12 +328,36 @@ fn main() -> Result<()> { target, config, check, + library, + release, } => commands::diff::cmd_diff(&commands::diff::DiffArgs { - input: input.as_deref(), + locate: Locate { + input: input.as_deref(), + config: config.as_deref(), + library: library.as_deref(), + release, + quiet, + }, out: out.as_deref(), targets: target.as_deref(), - config: config.as_deref(), check, + })?, + Commands::Build { + input, + platforms, + target, + config, + manifest_path, + debug, + warn, + } => commands::build::cmd_build(&commands::build::BuildArgs { + input: input.as_deref(), + config: config.as_deref(), + platforms: platforms.as_deref(), + targets: target.as_deref(), + debug, + manifest_path: manifest_path.as_deref(), + warn, quiet, })?, Commands::Package { @@ -252,17 +366,19 @@ fn main() -> Result<()> { target, config, binaries, - build, platforms, + manifest_path, + debug, warn, } => commands::package::cmd_package(&commands::package::PackageArgs { input: input.as_deref(), - out: &out, + out: out.as_deref(), targets: target.as_deref(), config: config.as_deref(), binaries: binaries.as_deref(), - build: build.as_deref(), platforms: platforms.as_deref(), + debug, + manifest_path: manifest_path.as_deref(), warn, quiet, })?, @@ -270,8 +386,20 @@ fn main() -> Result<()> { input, output, format, - lenient, - } => extract::cmd_extract(&input, output.as_deref(), format, lenient, quiet)?, + config, + library, + release, + } => commands::extract::cmd_extract( + &Locate { + input: input.as_deref(), + config: config.as_deref(), + library: library.as_deref(), + release, + quiet, + }, + output.as_deref(), + format, + )?, Commands::Completions { shell } => cmd_completions(shell), Commands::SchemaVersion => println!("{CURRENT_SCHEMA_VERSION}"), Commands::Schema { format } => cmd_schema(format)?, diff --git a/crates/weaveffi-gen/src/manifest.rs b/crates/weaveffi-cli/src/manifest.rs similarity index 98% rename from crates/weaveffi-gen/src/manifest.rs rename to crates/weaveffi-cli/src/manifest.rs index 5dc786a3..06563148 100644 --- a/crates/weaveffi-gen/src/manifest.rs +++ b/crates/weaveffi-cli/src/manifest.rs @@ -156,12 +156,6 @@ impl JsonObject { } } - /// `true` when no entries have been added. - #[must_use] - pub fn is_empty(&self) -> bool { - self.entries.is_empty() - } - /// Render the object as a pretty-printed JSON document with two-space /// indentation and a trailing newline, the conventional shape of /// `package.json`-style manifests. diff --git a/crates/weaveffi-cli/src/package/archive.rs b/crates/weaveffi-cli/src/package/archive.rs new file mode 100644 index 00000000..05d048cd --- /dev/null +++ b/crates/weaveffi-cli/src/package/archive.rs @@ -0,0 +1,769 @@ +//! Archive writers for the installable artifacts: gzip-compressed tarballs +//! (npm, C and C++), zip archives (`XCFramework`), Python wheels, and RubyGems +//! packages. +//! +//! Every writer builds the archive in memory and is deterministic: entries +//! keep their given order, timestamps are fixed (npm's 1985-10-26), and +//! owners are root. The same inputs therefore always produce the same bytes, +//! so checksums (a SwiftPM binary target's, a wheel's `RECORD`) are stable +//! across runs. + +use std::borrow::Cow; +use std::io::Write as _; + +use anyhow::{bail, Context, Result}; +use sha2::{Digest, Sha256, Sha512}; + +use crate::package::PackagedFile; + +/// The modification time of every archive entry: 1985-10-26T08:15:00Z, the +/// fixed timestamp `npm pack` uses. +const MTIME: u64 = 499_162_500; + +/// [`MTIME`] as an MS-DOS date and time, for zip entries. +const DOS_DATE: u16 = ((1985 - 1980) << 9) | (10 << 5) | 26; +const DOS_TIME: u16 = (8 << 11) | (15 << 5); + +/// One file inside an archive. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Entry<'a> { + /// The path inside the archive, `/`-separated. + pub path: String, + /// The file's bytes. + pub data: Cow<'a, [u8]>, + /// Whether the file is executable (native libraries and addons are). + pub executable: bool, +} + +impl<'a> Entry<'a> { + /// An entry at `path` holding `data`, executable when its name looks like + /// a native library or addon. + pub fn new(path: impl Into, data: impl Into>) -> Self { + let path = path.into(); + let executable = is_native(&path); + Self { + path, + data: data.into(), + executable, + } + } + + fn mode(&self) -> u32 { + if self.executable { + 0o755 + } else { + 0o644 + } + } +} + +/// Whether `path` names a native library or addon, which archives mark +/// executable. +fn is_native(path: &str) -> bool { + let name = path.rsplit('/').next().unwrap_or(path); + [".so", ".dylib", ".dll", ".node", ".a"] + .iter() + .any(|ext| name.ends_with(ext)) +} + +/// The archive entries for `files`, read from memory or disk, each placed +/// under `prefix/` when given. +/// +/// # Errors +/// +/// Returns an error when a copied file can't be read. +pub fn entries<'a>(files: &'a [PackagedFile], prefix: Option<&str>) -> Result>> { + files + .iter() + .map(|f| { + let path = match prefix { + Some(p) => format!("{p}/{}", f.path), + None => f.path.to_string(), + }; + Ok(Entry::new(path, f.content.bytes()?)) + }) + .collect() +} + +/// An uncompressed tar archive of `entries`. +/// +/// # Errors +/// +/// Returns an error when an entry's path can't be represented in a tar +/// header. +pub fn tar(entries: &[Entry<'_>]) -> Result> { + let mut builder = tar::Builder::new(Vec::new()); + for entry in entries { + let mut header = tar::Header::new_ustar(); + header.set_size(entry.data.len() as u64); + header.set_mode(entry.mode()); + header.set_mtime(MTIME); + header.set_uid(0); + header.set_gid(0); + header.set_entry_type(tar::EntryType::Regular); + builder + .append_data(&mut header, &entry.path, entry.data.as_ref()) + .with_context(|| format!("failed to add {} to a tar archive", entry.path))?; + } + builder + .into_inner() + .context("failed to finish a tar archive") +} + +/// `bytes` compressed with gzip (with a zeroed header timestamp). +/// +/// # Errors +/// +/// Returns an error when compression fails. +pub fn gzip(bytes: &[u8]) -> Result> { + let mut encoder = flate2::GzBuilder::new() + .mtime(0) + .write(Vec::new(), flate2::Compression::default()); + encoder.write_all(bytes).context("failed to gzip")?; + encoder.finish().context("failed to gzip") +} + +/// A zip archive of `entries`, each deflated, with Unix permissions. +/// +/// # Errors +/// +/// Returns an error when an entry is too large for a non-Zip64 archive or +/// compression fails. +pub fn zip(entries: &[Entry<'_>]) -> Result> { + let mut out = Vec::new(); + let mut central = Vec::new(); + for entry in entries { + let mut encoder = + flate2::write::DeflateEncoder::new(Vec::new(), flate2::Compression::default()); + encoder + .write_all(&entry.data) + .context("failed to deflate")?; + let compressed = encoder.finish().context("failed to deflate")?; + let crc = crc32fast::hash(&entry.data); + let (Ok(size), Ok(packed), Ok(offset), Ok(name_len)) = ( + u32::try_from(entry.data.len()), + u32::try_from(compressed.len()), + u32::try_from(out.len()), + u16::try_from(entry.path.len()), + ) else { + bail!("{} is too large for a zip archive", entry.path); + }; + // Local file header: version 2.0, UTF-8 names, deflate. + out.extend_from_slice(&0x0403_4b50u32.to_le_bytes()); + out.extend_from_slice(&20u16.to_le_bytes()); + out.extend_from_slice(&0x0800u16.to_le_bytes()); + out.extend_from_slice(&8u16.to_le_bytes()); + out.extend_from_slice(&DOS_TIME.to_le_bytes()); + out.extend_from_slice(&DOS_DATE.to_le_bytes()); + out.extend_from_slice(&crc.to_le_bytes()); + out.extend_from_slice(&packed.to_le_bytes()); + out.extend_from_slice(&size.to_le_bytes()); + out.extend_from_slice(&name_len.to_le_bytes()); + out.extend_from_slice(&0u16.to_le_bytes()); + out.extend_from_slice(entry.path.as_bytes()); + out.extend_from_slice(&compressed); + // Central directory record: made by Unix (3) so the external + // attributes carry the file mode. + central.extend_from_slice(&0x0201_4b50u32.to_le_bytes()); + central.extend_from_slice(&((3u16 << 8) | 20).to_le_bytes()); + central.extend_from_slice(&20u16.to_le_bytes()); + central.extend_from_slice(&0x0800u16.to_le_bytes()); + central.extend_from_slice(&8u16.to_le_bytes()); + central.extend_from_slice(&DOS_TIME.to_le_bytes()); + central.extend_from_slice(&DOS_DATE.to_le_bytes()); + central.extend_from_slice(&crc.to_le_bytes()); + central.extend_from_slice(&packed.to_le_bytes()); + central.extend_from_slice(&size.to_le_bytes()); + central.extend_from_slice(&name_len.to_le_bytes()); + central.extend_from_slice(&[0; 8]); + central.extend_from_slice(&((0o100_000 | entry.mode()) << 16).to_le_bytes()); + central.extend_from_slice(&offset.to_le_bytes()); + central.extend_from_slice(entry.path.as_bytes()); + } + let (Ok(count), Ok(central_len), Ok(central_offset)) = ( + u16::try_from(entries.len()), + u32::try_from(central.len()), + u32::try_from(out.len()), + ) else { + bail!("too many entries for a zip archive"); + }; + out.extend_from_slice(¢ral); + out.extend_from_slice(&0x0605_4b50u32.to_le_bytes()); + out.extend_from_slice(&[0; 4]); + out.extend_from_slice(&count.to_le_bytes()); + out.extend_from_slice(&count.to_le_bytes()); + out.extend_from_slice(¢ral_len.to_le_bytes()); + out.extend_from_slice(¢ral_offset.to_le_bytes()); + out.extend_from_slice(&0u16.to_le_bytes()); + Ok(out) +} + +/// The lowercase hexadecimal SHA-256 of `bytes`. +#[must_use] +pub fn sha256_hex(bytes: &[u8]) -> String { + hex(&Sha256::digest(bytes)) +} + +fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() +} + +/// URL-safe base64 without padding, the digest encoding of a wheel's +/// `RECORD`. +fn base64_url(bytes: &[u8]) -> String { + const ALPHABET: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"; + let mut out = String::with_capacity(bytes.len().div_ceil(3) * 4); + for chunk in bytes.chunks(3) { + let n = chunk + .iter() + .enumerate() + .fold(0u32, |n, (i, b)| n | (u32::from(*b) << (16 - 8 * i))); + for i in 0..=chunk.len() { + out.push(char::from(ALPHABET[((n >> (18 - 6 * i)) & 63) as usize])); + } + } + out +} + +/// The metadata of a Python wheel. The writer renders the `.dist-info` +/// directory (`METADATA`, `WHEEL`, `RECORD`) from it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct WheelMeta { + /// The distribution name (`kvstore`). + pub name: String, + /// The version (`1.2.0`). + pub version: String, + /// The platform tag (`macosx_11_0_arm64`); the wheel is tagged + /// `py3-none-{platform}`. + pub platform: String, + /// The one-line summary. + pub summary: String, + /// The `Requires-Python` specifier, if any. + pub requires_python: Option, + /// The license expression, if any. + pub license: Option, + /// The authors, joined with `, `, if any. + pub author: Option, + /// `Project-URL` entries as `(label, url)` pairs. + pub urls: Vec<(String, String)>, + /// The long description (Markdown), if any. + pub readme: Option, +} + +impl WheelMeta { + /// The distribution name escaped for file names: runs of `-`, `_`, and + /// `.` become `_`, lowercased. + #[must_use] + pub fn escaped_name(&self) -> String { + let mut out = String::with_capacity(self.name.len()); + for c in self.name.chars() { + if matches!(c, '-' | '_' | '.') { + if !out.ends_with('_') { + out.push('_'); + } + } else { + out.extend(c.to_lowercase()); + } + } + out + } + + /// The full compatibility tag, `py3-none-{platform}`. + #[must_use] + pub fn tag(&self) -> String { + format!("py3-none-{}", self.platform) + } + + /// The wheel's file name: `{name}-{version}-py3-none-{platform}.whl`. + #[must_use] + pub fn file_name(&self) -> String { + format!( + "{}-{}-{}.whl", + self.escaped_name(), + self.version, + self.tag() + ) + } + + /// The `.dist-info` directory name. + #[must_use] + pub fn dist_info(&self) -> String { + format!("{}-{}.dist-info", self.escaped_name(), self.version) + } + + fn metadata(&self) -> String { + let mut out = format!( + "Metadata-Version: 2.1\nName: {}\nVersion: {}\nSummary: {}\n", + self.name, + self.version, + one_line(&self.summary) + ); + if let Some(r) = &self.requires_python { + out.push_str(&format!("Requires-Python: {r}\n")); + } + if let Some(l) = &self.license { + out.push_str(&format!("License: {}\n", one_line(l))); + } + if let Some(a) = &self.author { + out.push_str(&format!("Author: {}\n", one_line(a))); + } + for (label, url) in &self.urls { + out.push_str(&format!("Project-URL: {label}, {url}\n")); + } + if let Some(readme) = &self.readme { + out.push_str("Description-Content-Type: text/markdown\n\n"); + out.push_str(readme); + } + out + } + + fn wheel(&self) -> String { + format!( + "Wheel-Version: 1.0\nGenerator: weaveffi ({})\nRoot-Is-Purelib: false\nTag: {}\n", + env!("CARGO_PKG_VERSION"), + self.tag() + ) + } +} + +fn one_line(s: &str) -> String { + s.split_whitespace().collect::>().join(" ") +} + +/// A Python wheel holding `files` plus the `.dist-info` files rendered from +/// `meta`, with a `RECORD` of every file's SHA-256 and size. +/// +/// # Errors +/// +/// Returns an error when a file can't be read or the archive can't be built. +pub fn wheel(meta: &WheelMeta, files: &[PackagedFile]) -> Result> { + let dist_info = meta.dist_info(); + let mut all = entries(files, None)?; + all.push(Entry::new( + format!("{dist_info}/METADATA"), + meta.metadata().into_bytes(), + )); + all.push(Entry::new( + format!("{dist_info}/WHEEL"), + meta.wheel().into_bytes(), + )); + let mut record = String::new(); + for entry in &all { + record.push_str(&format!( + "{},sha256={},{}\n", + csv_field(&entry.path), + base64_url(&Sha256::digest(&entry.data)), + entry.data.len() + )); + } + let record_path = format!("{dist_info}/RECORD"); + record.push_str(&format!("{},,\n", csv_field(&record_path))); + all.push(Entry::new(record_path, record.into_bytes())); + zip(&all) +} + +fn csv_field(s: &str) -> Cow<'_, str> { + if s.contains([',', '"', '\n']) { + Cow::Owned(format!("\"{}\"", s.replace('"', "\"\""))) + } else { + Cow::Borrowed(s) + } +} + +/// The specification of a RubyGems package. The writer renders it as the +/// gem's YAML `metadata.gz`. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct GemSpec { + /// The gem name. + pub name: String, + /// The version. + pub version: String, + /// The RubyGems platform (`x86_64-linux`, `arm64-darwin`), or `ruby` for + /// a pure gem. + pub platform: String, + /// The one-line summary. + pub summary: String, + /// The authors (RubyGems requires at least one). + pub authors: Vec, + /// The license, if any. + pub license: Option, + /// The homepage, if any. + pub homepage: Option, + /// The `required_ruby_version` requirement (`>= 2.7`). + pub required_ruby_version: String, + /// Runtime dependencies as `(name, requirement)` pairs + /// (`("ffi", "~> 1.15")`). + pub dependencies: Vec<(String, String)>, +} + +impl GemSpec { + /// The gem's file name: `{name}-{version}-{platform}.gem`, or + /// `{name}-{version}.gem` for a pure gem. + #[must_use] + pub fn file_name(&self) -> String { + if self.platform == "ruby" { + format!("{}-{}.gem", self.name, self.version) + } else { + format!("{}-{}-{}.gem", self.name, self.version, self.platform) + } + } + + /// The specification as the YAML document RubyGems stores in + /// `metadata.gz`, listing `files`. + #[must_use] + pub fn to_yaml(&self, files: &[String]) -> String { + let requirement = |req: &str, indent: &str| { + let (op, version) = req.split_once(' ').unwrap_or((">=", req)); + format!( + "{indent}requirements:\n{indent}- - {}\n{indent} - !ruby/object:Gem::Version\n\ + {indent} version: {}\n", + yaml_str(op), + yaml_str(version) + ) + }; + let list = |items: &[String]| { + if items.is_empty() { + " []\n".to_string() + } else { + let mut out = "\n".to_string(); + for item in items { + out.push_str(&format!("- {}\n", yaml_str(item))); + } + out + } + }; + let mut out = format!( + "--- !ruby/object:Gem::Specification\nname: {}\nversion: !ruby/object:Gem::Version\n \ + version: {}\nplatform: {}\nauthors:{}autorequire:\nbindir: bin\ncert_chain: []\n\ + date: 1985-10-26 00:00:00.000000000 Z\ndependencies:", + yaml_str(&self.name), + yaml_str(&self.version), + yaml_str(&self.platform), + list(&self.authors), + ); + if self.dependencies.is_empty() { + out.push_str(" []\n"); + } else { + out.push('\n'); + for (name, req) in &self.dependencies { + out.push_str(&format!( + "- !ruby/object:Gem::Dependency\n name: {}\n requirement: \ + !ruby/object:Gem::Requirement\n{} type: :runtime\n prerelease: false\n \ + version_requirements: !ruby/object:Gem::Requirement\n{}", + yaml_str(name), + requirement(req, " "), + requirement(req, " "), + )); + } + } + out.push_str(&format!( + "description:\nemail:\nexecutables: []\nextensions: []\nextra_rdoc_files: []\nfiles:{}", + list(files) + )); + out.push_str(&format!( + "homepage:{}\nlicenses:{}metadata: {{}}\npost_install_message:\nrdoc_options: []\n\ + require_paths:\n- lib\nrequired_ruby_version: !ruby/object:Gem::Requirement\n{}\ + required_rubygems_version: !ruby/object:Gem::Requirement\n{}requirements: []\n\ + rubygems_version: 3.4.10\nsigning_key:\nspecification_version: 4\nsummary: {}\n\ + test_files: []\n", + self.homepage + .as_deref() + .map(|h| format!(" {}", yaml_str(h))) + .unwrap_or_default(), + list(&self.license.iter().cloned().collect::>()), + requirement(&self.required_ruby_version, " "), + requirement(">= 0", " "), + yaml_str(&one_line(&self.summary)), + )); + out + } +} + +/// A single-quoted YAML scalar. +fn yaml_str(s: &str) -> String { + format!("'{}'", s.replace('\'', "''")) +} + +/// A RubyGems package holding `files`: a tar of the gzipped YAML +/// specification (`metadata.gz`), the gzipped data tarball +/// (`data.tar.gz`), and their SHA-256 and SHA-512 checksums +/// (`checksums.yaml.gz`). +/// +/// # Errors +/// +/// Returns an error when a file can't be read or an archive can't be built. +pub fn gem(spec: &GemSpec, files: &[PackagedFile]) -> Result> { + let names: Vec = files.iter().map(|f| f.path.to_string()).collect(); + let metadata = gzip(spec.to_yaml(&names).as_bytes())?; + let data = gzip(&tar(&entries(files, None)?)?)?; + let checksums = format!( + "---\nSHA256:\n metadata.gz: '{}'\n data.tar.gz: '{}'\nSHA512:\n metadata.gz: '{}'\n \ + data.tar.gz: '{}'\n", + hex(&Sha256::digest(&metadata)), + hex(&Sha256::digest(&data)), + hex(&Sha512::digest(&metadata)), + hex(&Sha512::digest(&data)), + ); + let checksums = gzip(checksums.as_bytes())?; + let mut outer = Vec::new(); + for (name, bytes) in [ + ("metadata.gz", metadata), + ("data.tar.gz", data), + ("checksums.yaml.gz", checksums), + ] { + let mut entry = Entry::new(name, bytes); + entry.executable = false; + outer.push(entry); + } + tar(&outer) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::io::Read as _; + + /// Parse a zip written by [`zip`] back into `(name, mode, bytes)` + /// triples by walking its central directory. + fn unzip(bytes: &[u8]) -> Vec<(String, u32, Vec)> { + let u16_at = |at: usize| u16::from_le_bytes([bytes[at], bytes[at + 1]]) as usize; + let u32_at = |at: usize| { + u32::from_le_bytes([bytes[at], bytes[at + 1], bytes[at + 2], bytes[at + 3]]) + }; + let eocd = bytes.len() - 22; + assert_eq!(u32_at(eocd), 0x0605_4b50); + let count = u16_at(eocd + 10); + let mut at = u32_at(eocd + 16) as usize; + let mut out = Vec::new(); + for _ in 0..count { + assert_eq!(u32_at(at), 0x0201_4b50); + let packed = u32_at(at + 20) as usize; + let name_len = u16_at(at + 28); + let mode = u32_at(at + 38) >> 16; + let offset = u32_at(at + 42) as usize; + let name = String::from_utf8(bytes[at + 46..at + 46 + name_len].to_vec()).unwrap(); + let data_at = offset + 30 + u16_at(offset + 26) + u16_at(offset + 28); + let mut data = Vec::new(); + flate2::read::DeflateDecoder::new(&bytes[data_at..data_at + packed]) + .read_to_end(&mut data) + .unwrap(); + assert_eq!(crc32fast::hash(&data), u32_at(at + 16)); + out.push((name, mode, data)); + at += 46 + name_len; + } + out + } + + fn untar(bytes: &[u8]) -> Vec<(String, u32, Vec)> { + let mut archive = tar::Archive::new(bytes); + archive + .entries() + .unwrap() + .map(|e| { + let mut e = e.unwrap(); + let path = e.path().unwrap().to_string_lossy().into_owned(); + let mode = e.header().mode().unwrap(); + let mut data = Vec::new(); + e.read_to_end(&mut data).unwrap(); + (path, mode, data) + }) + .collect() + } + + fn gunzip(bytes: &[u8]) -> Vec { + let mut out = Vec::new(); + flate2::read::GzDecoder::new(bytes) + .read_to_end(&mut out) + .unwrap(); + out + } + + fn meta() -> WheelMeta { + WheelMeta { + name: "My.Kv-Store".into(), + version: "1.2.0".into(), + platform: "macosx_11_0_arm64".into(), + summary: "An embedded\nkey-value store".into(), + requires_python: Some(">=3.9".into()), + license: Some("MIT".into()), + author: Some("Ada".into()), + urls: vec![("Repository".into(), "https://example.com/kv".into())], + readme: Some("# kv\n".into()), + } + } + + #[test] + fn base64_matches_the_reference_encoding() { + assert_eq!(base64_url(b""), ""); + assert_eq!(base64_url(b"f"), "Zg"); + assert_eq!(base64_url(b"fo"), "Zm8"); + assert_eq!(base64_url(b"foo"), "Zm9v"); + assert_eq!(base64_url(&[0xfb, 0xff]), "-_8"); + assert_eq!( + sha256_hex(b"abc"), + "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" + ); + } + + #[test] + fn zips_round_trip_with_unix_modes() { + let bytes = zip(&[ + Entry::new("a/b.txt", &b"hello"[..]), + Entry::new("lib/libx.so", &b"\x7fELF"[..]), + ]) + .unwrap(); + let files = unzip(&bytes); + assert_eq!(files[0], ("a/b.txt".into(), 0o100_644, b"hello".to_vec())); + assert_eq!(files[1].1, 0o100_755); + assert_eq!( + bytes, + zip(&[ + Entry::new("a/b.txt", &b"hello"[..]), + Entry::new("lib/libx.so", &b"\x7fELF"[..]) + ]) + .unwrap() + ); + } + + #[test] + fn wheels_carry_dist_info_tags_and_a_verifiable_record() { + let m = meta(); + assert_eq!( + m.file_name(), + "my_kv_store-1.2.0-py3-none-macosx_11_0_arm64.whl" + ); + let files = vec![ + PackagedFile::text("kv/__init__.py", "from .kv import *\n"), + PackagedFile::bytes("kv/libkv.dylib", b"\xcf\xfa\xed\xfe".to_vec()), + ]; + let entries = unzip(&wheel(&m, &files).unwrap()); + let names: Vec<&str> = entries.iter().map(|e| e.0.as_str()).collect(); + assert_eq!( + names, + [ + "kv/__init__.py", + "kv/libkv.dylib", + "my_kv_store-1.2.0.dist-info/METADATA", + "my_kv_store-1.2.0.dist-info/WHEEL", + "my_kv_store-1.2.0.dist-info/RECORD", + ] + ); + let text = |name: &str| { + String::from_utf8(entries.iter().find(|e| e.0 == name).unwrap().2.clone()).unwrap() + }; + let wheel_file = text("my_kv_store-1.2.0.dist-info/WHEEL"); + assert!( + wheel_file.contains("Root-Is-Purelib: false\n"), + "{wheel_file}" + ); + assert!( + wheel_file.contains("Tag: py3-none-macosx_11_0_arm64\n"), + "{wheel_file}" + ); + let metadata = text("my_kv_store-1.2.0.dist-info/METADATA"); + assert!(metadata.starts_with("Metadata-Version: 2.1\nName: My.Kv-Store\nVersion: 1.2.0\n")); + assert!(metadata.contains("Summary: An embedded key-value store\n")); + assert!(metadata.contains("Requires-Python: >=3.9\n")); + assert!(metadata.contains("Project-URL: Repository, https://example.com/kv\n")); + assert!(metadata.ends_with("\n\n# kv\n")); + + // Every RECORD line but its own names a file with its real digest + // and size. + let record = text("my_kv_store-1.2.0.dist-info/RECORD"); + let lines: Vec<&str> = record.lines().collect(); + assert_eq!(lines.len(), entries.len()); + assert_eq!( + *lines.last().unwrap(), + "my_kv_store-1.2.0.dist-info/RECORD,," + ); + for line in &lines[..lines.len() - 1] { + let mut parts = line.split(','); + let (path, hash, size) = ( + parts.next().unwrap(), + parts.next().unwrap(), + parts.next().unwrap(), + ); + let data = &entries.iter().find(|e| e.0 == path).unwrap().2; + assert_eq!( + hash, + format!("sha256={}", base64_url(&Sha256::digest(data))) + ); + assert_eq!(size, data.len().to_string()); + } + } + + #[test] + fn npm_tarballs_prefix_every_entry_with_package() { + let files = vec![ + PackagedFile::text("package.json", "{}"), + PackagedFile::bytes("kv_node.node", b"addon".to_vec()), + ]; + let tgz = gzip(&tar(&entries(&files, Some("package")).unwrap()).unwrap()).unwrap(); + let unpacked = untar(&gunzip(&tgz)); + assert_eq!( + unpacked, + [ + ("package/package.json".into(), 0o644, b"{}".to_vec()), + ("package/kv_node.node".into(), 0o755, b"addon".to_vec()), + ] + ); + assert_eq!( + tgz, + gzip(&tar(&entries(&files, Some("package")).unwrap()).unwrap()).unwrap() + ); + } + + #[test] + fn gems_hold_metadata_data_and_checksums() { + let spec = GemSpec { + name: "kvstore".into(), + version: "1.2.0".into(), + platform: "x86_64-linux".into(), + summary: "It's a store".into(), + authors: vec!["Ada".into()], + license: Some("MIT".into()), + homepage: Some("https://example.com".into()), + required_ruby_version: ">= 2.7".into(), + dependencies: vec![("ffi".into(), "~> 1.15".into())], + }; + assert_eq!(spec.file_name(), "kvstore-1.2.0-x86_64-linux.gem"); + let files = vec![ + PackagedFile::text("lib/kvstore.rb", "require 'ffi'\n"), + PackagedFile::bytes("lib/native/libkvstore.so", b"elf".to_vec()), + ]; + let outer = untar(&gem(&spec, &files).unwrap()); + let names: Vec<&str> = outer.iter().map(|e| e.0.as_str()).collect(); + assert_eq!(names, ["metadata.gz", "data.tar.gz", "checksums.yaml.gz"]); + + let yaml = String::from_utf8(gunzip(&outer[0].2)).unwrap(); + for needle in [ + "--- !ruby/object:Gem::Specification\nname: 'kvstore'\n", + "version: !ruby/object:Gem::Version\n version: '1.2.0'\n", + "platform: 'x86_64-linux'\n", + "authors:\n- 'Ada'\n", + "- !ruby/object:Gem::Dependency\n name: 'ffi'\n", + " - - '~>'\n - !ruby/object:Gem::Version\n version: '1.15'\n", + "files:\n- 'lib/kvstore.rb'\n- 'lib/native/libkvstore.so'\n", + "licenses:\n- 'MIT'\n", + "required_ruby_version: !ruby/object:Gem::Requirement\n requirements:\n - - '>='\n", + "summary: 'It''s a store'\n", + ] { + assert!(yaml.contains(needle), "missing {needle:?} in\n{yaml}"); + } + + let data = untar(&gunzip(&outer[1].2)); + assert_eq!(data[0].0, "lib/kvstore.rb"); + assert_eq!( + (data[1].0.as_str(), data[1].1), + ("lib/native/libkvstore.so", 0o755) + ); + + let checksums = String::from_utf8(gunzip(&outer[2].2)).unwrap(); + assert!(checksums.contains(&format!( + "SHA256:\n metadata.gz: '{}'\n data.tar.gz: '{}'\n", + sha256_hex(&outer[0].2), + sha256_hex(&outer[1].2) + ))); + assert!(checksums.contains("SHA512:\n")); + } +} diff --git a/crates/weaveffi-cli/src/package/mod.rs b/crates/weaveffi-cli/src/package/mod.rs new file mode 100644 index 00000000..5475d905 --- /dev/null +++ b/crates/weaveffi-cli/src/package/mod.rs @@ -0,0 +1,347 @@ +//! The packaging layer: the artifacts a backend assembles from a set of +//! per-platform builds, and the driver that writes them to the dist +//! directory. +//! +//! `weaveffi generate` emits binding *source* that a consumer must compile or +//! point at a native library themselves. `weaveffi package` produces the +//! next artifact up: an installable package for each ecosystem (a wheel, an +//! npm tarball, a gem, a SwiftPM `XCFramework` archive, …) with the prebuilt +//! native library for each [`Platform`] inside, so +//! `pip install`, `npm install`, and `gem install` work with no local +//! toolchain. +//! +//! A backend opts in by overriding +//! [`LanguageBackend::package`](crate::backend::LanguageBackend::package), +//! returning [`Artifact`]s: a directory tree or an archive, each a list of +//! [`PackagedFile`]s. Rendering stays pure (it returns values and does no +//! I/O), so package layouts are testable exactly like generated source, and +//! [`write_artifact`] does the I/O with the writers in [`archive`]. + +pub mod archive; + +use std::borrow::Cow; + +use anyhow::{Context, Result}; +use camino::{Utf8Path, Utf8PathBuf}; + +use crate::platform::{read_file, BinarySet, Platform}; + +pub use archive::{GemSpec, WheelMeta}; + +/// The contents of one [`PackagedFile`]. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum FileContent { + /// Rendered text (a manifest, loader, README, or binding source file). + Text(String), + /// Raw bytes rendered in memory. + Bytes(Vec), + /// A file copied byte-for-byte from this path, such as a prebuilt native + /// library. Keeping binaries out of memory until the artifact is written + /// means rendering never holds a multi-megabyte library as a value. + Copy(Utf8PathBuf), +} + +impl FileContent { + /// The file's bytes, reading a [`Copy`](Self::Copy) source from disk. + /// + /// # Errors + /// + /// Returns an error when a copied file can't be read. + pub fn bytes(&self) -> Result> { + Ok(match self { + Self::Text(text) => Cow::Borrowed(text.as_bytes()), + Self::Bytes(bytes) => Cow::Borrowed(bytes), + Self::Copy(source) => Cow::Owned(read_file(source)?), + }) + } +} + +/// One file in an [`Artifact`]: its path inside the artifact and its +/// contents. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PackagedFile { + /// Path relative to the artifact's root, with `/` separators. + pub path: Utf8PathBuf, + /// What the file contains. + pub content: FileContent, +} + +impl PackagedFile { + /// A file whose contents are rendered text. + pub fn text(path: impl Into, contents: impl Into) -> Self { + Self { + path: normalize_separators(path.into()), + content: FileContent::Text(contents.into()), + } + } + + /// A file whose contents are bytes rendered in memory. + pub fn bytes(path: impl Into, contents: Vec) -> Self { + Self { + path: normalize_separators(path.into()), + content: FileContent::Bytes(contents), + } + } + + /// A file copied from `source` on disk (a prebuilt native library). + pub fn copy(path: impl Into, source: impl Into) -> Self { + Self { + path: normalize_separators(path.into()), + content: FileContent::Copy(source.into()), + } + } + + /// True when this entry copies in a file from disk rather than holding + /// rendered contents. + pub fn is_binary(&self) -> bool { + matches!(self.content, FileContent::Copy(_)) + } +} + +/// Normalize a package path to use `/` separators on every host, so package +/// layouts are identical (and testable) across platforms. +fn normalize_separators(path: Utf8PathBuf) -> Utf8PathBuf { + if path.as_str().contains('\\') { + Utf8PathBuf::from(path.as_str().replace('\\', "/")) + } else { + path + } +} + +/// How an [`Artifact`]'s files are materialized. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ArtifactKind { + /// A directory tree (a Gradle project, a Go module, a pub package). + Directory, + /// A gzip-compressed tarball whose entries all sit under `prefix/` + /// (`package/` for npm). + TarGz { + /// The directory every entry is placed under. + prefix: String, + }, + /// A zip archive. + Zip, + /// A Python wheel: a zip whose `.dist-info` directory the writer adds. + Wheel(WheelMeta), + /// A RubyGems package: a tar of the gzipped specification and data. + Gem(GemSpec), +} + +/// One installable artifact a target produces. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Artifact { + /// Where the artifact goes, relative to the dist directory: the + /// directory for [`ArtifactKind::Directory`], else the archive's file. + pub path: Utf8PathBuf, + /// How the files are written. + pub kind: ArtifactKind, + /// The artifact's files, with paths relative to its root. + pub files: Vec, +} + +impl Artifact { + /// A directory artifact at `path`. + pub fn directory(path: impl Into, files: Vec) -> Self { + Self { + path: normalize_separators(path.into()), + kind: ArtifactKind::Directory, + files, + } + } + + /// A `.tar.gz` artifact at `path` with every entry under `prefix/`. + pub fn tar_gz( + path: impl Into, + prefix: impl Into, + files: Vec, + ) -> Self { + Self { + path: normalize_separators(path.into()), + kind: ArtifactKind::TarGz { + prefix: prefix.into(), + }, + files, + } + } + + /// The file at `path` inside this artifact, if any. + pub fn file(&self, path: &str) -> Option<&PackagedFile> { + self.files.iter().find(|f| f.path == path) + } +} + +/// The prebuilt `XCFramework` archive a Swift package's binary target points +/// at. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct XcframeworkArchive { + /// The archive's file name (`CKvstore.xcframework.zip`). + pub file_name: String, + /// The SHA-256 checksum of the archive, as `swift package + /// compute-checksum` prints it. + pub checksum: String, +} + +/// Everything a backend's [`package`](crate::backend::LanguageBackend::package) +/// needs beyond the [`Model`](weaveffi_model::model::Model) and its own +/// configuration. +#[derive(Debug, Clone, Copy)] +pub struct PackageContext<'a> { + /// The per-platform builds to bundle. + pub binaries: &'a BinarySet, + /// The macOS deployment target the libraries were built for (`"11.0"`), + /// which wheel tags and Swift platforms carry. + pub macos_deployment_target: &'a str, + /// The iOS deployment target the libraries were built for (`"13.0"`). + pub ios_deployment_target: &'a str, + /// The assembled `XCFramework` archive, when the Swift target packages + /// Apple static libraries. + pub xcframework: Option<&'a XcframeworkArchive>, +} + +impl<'a> PackageContext<'a> { + /// A context for `binaries` built for the default deployment targets + /// (macOS 11.0, iOS 13.0), without an `XCFramework` archive. + #[must_use] + pub fn new(binaries: &'a BinarySet) -> Self { + Self { + binaries, + macos_deployment_target: "11.0", + ios_deployment_target: "13.0", + xcframework: None, + } + } +} + +/// The producer library of every build `keep` accepts, laid out as +/// `{dir}//`, plus the import library MSVC links +/// a Windows DLL through. +pub fn per_platform_libraries( + binaries: &BinarySet, + dir: &str, + keep: impl Fn(Platform) -> bool, +) -> Vec { + let mut files = Vec::new(); + for nb in binaries.binaries.iter().filter(|nb| keep(nb.platform)) { + let platform_dir = format!("{dir}/{}", nb.platform.id()); + files.push(PackagedFile::copy( + format!("{platform_dir}/{}", binaries.bundled_filename(nb.platform)), + nb.library.clone(), + )); + if let Some(import) = &nb.import_library { + files.push(PackagedFile::copy( + format!("{platform_dir}/{}.dll.lib", binaries.lib_name), + import.clone(), + )); + } + } + files +} + +/// Write `artifact` under `dist`, returning the path written. +/// +/// A directory artifact replaces any previous directory at its path, so +/// files from an earlier run never linger; an archive is built in memory and +/// written in one go. +/// +/// # Errors +/// +/// Returns an error when a directory can't be created or replaced, a file +/// can't be read or written, or an archive can't be assembled. +pub fn write_artifact(dist: &Utf8Path, artifact: &Artifact) -> Result { + let path = dist.join(&artifact.path); + if let ArtifactKind::Directory = artifact.kind { + if path.exists() { + std::fs::remove_dir_all(path.as_std_path()) + .with_context(|| format!("failed to replace {path}"))?; + } + for file in &artifact.files { + let dest = path.join(&file.path); + create_parent(&dest)?; + match &file.content { + FileContent::Copy(source) => { + std::fs::copy(source.as_std_path(), dest.as_std_path()) + .with_context(|| format!("failed to copy {source} to {dest}"))?; + } + content => std::fs::write(dest.as_std_path(), content.bytes()?) + .with_context(|| format!("failed to write {dest}"))?, + } + } + return Ok(path); + } + let bytes = match &artifact.kind { + ArtifactKind::Directory => unreachable!("handled above"), + ArtifactKind::TarGz { prefix } => { + let entries = archive::entries(&artifact.files, Some(prefix))?; + archive::gzip(&archive::tar(&entries)?)? + } + ArtifactKind::Zip => archive::zip(&archive::entries(&artifact.files, None)?)?, + ArtifactKind::Wheel(meta) => archive::wheel(meta, &artifact.files)?, + ArtifactKind::Gem(spec) => archive::gem(spec, &artifact.files)?, + }; + create_parent(&path)?; + std::fs::write(path.as_std_path(), bytes).with_context(|| format!("failed to write {path}"))?; + Ok(path) +} + +fn create_parent(path: &Utf8Path) -> Result<()> { + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent.as_std_path()) + .with_context(|| format!("failed to create {parent}"))?; + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn directories_are_replaced_and_binaries_copied() { + let dir = tempfile::tempdir().unwrap(); + let root = Utf8Path::from_path(dir.path()).unwrap(); + let src = root.join("src-lib.bin"); + std::fs::write(&src, b"\x00native\x01").unwrap(); + std::fs::create_dir_all(root.join("dist/pkg")).unwrap(); + std::fs::write(root.join("dist/pkg/stale.txt"), "old").unwrap(); + + let artifact = Artifact::directory( + "pkg", + vec![ + PackagedFile::text("manifest.json", "{\"name\":\"x\"}"), + PackagedFile::copy("native/lib.bin", src.clone()), + ], + ); + let written = write_artifact(&root.join("dist"), &artifact).unwrap(); + assert_eq!(written, root.join("dist/pkg")); + assert_eq!( + std::fs::read_to_string(written.join("manifest.json")).unwrap(), + "{\"name\":\"x\"}" + ); + assert_eq!( + std::fs::read(written.join("native/lib.bin")).unwrap(), + b"\x00native\x01" + ); + assert!(!written.join("stale.txt").exists()); + } + + #[test] + fn missing_copy_sources_are_errors() { + let dir = tempfile::tempdir().unwrap(); + let root = Utf8Path::from_path(dir.path()).unwrap(); + let artifact = Artifact::tar_gz( + "x.tar.gz", + "x", + vec![PackagedFile::copy("lib.bin", root.join("missing.bin"))], + ); + let err = write_artifact(root, &artifact).unwrap_err(); + assert!(format!("{err:#}").contains("missing.bin"), "{err:#}"); + } + + #[test] + fn paths_are_forward_slashed() { + let text = PackagedFile::text(Utf8Path::new("dotnet").join("x.cs"), "x"); + assert_eq!(text.path.as_str(), "dotnet/x.cs"); + let artifact = Artifact::directory("a\\b", vec![]); + assert_eq!(artifact.path.as_str(), "a/b"); + } +} diff --git a/crates/weaveffi-gen/src/platform.rs b/crates/weaveffi-cli/src/platform.rs similarity index 53% rename from crates/weaveffi-gen/src/platform.rs rename to crates/weaveffi-cli/src/platform.rs index a9e37052..cb50dece 100644 --- a/crates/weaveffi-gen/src/platform.rs +++ b/crates/weaveffi-cli/src/platform.rs @@ -1,23 +1,26 @@ //! The target-platform model that the packaging pipeline shares. //! -//! `weaveffi generate` emits binding *source*; `weaveffi package` goes further -//! and assembles publishable packages that bundle a prebuilt native library for -//! each platform. To do that without every backend re-deriving the same facts, -//! this module is the single source of truth for the supported platforms and -//! the per-ecosystem identifiers each one maps to: +//! `weaveffi build` cross-compiles a producer once per platform and +//! `weaveffi package` bundles the results into each ecosystem's artifacts. To +//! do that without every backend re-deriving the same facts, this module is +//! the single source of truth for the supported platforms and the +//! per-ecosystem identifiers each one maps to: //! -//! * the Rust target triple (`aarch64-apple-darwin`, …) used by `--build`; +//! * the Rust target triple (`aarch64-apple-darwin`, …); //! * the shared-library file name (`libfoo.dylib`, `foo.dll`, …); //! * the NuGet runtime identifier (`osx-arm64`, …); //! * the Node.js `process.platform`/`process.arch` tokens; //! * the Python wheel platform tag (`macosx_11_0_arm64`, …); and //! * the RubyGems platform string (`arm64-darwin`, …). //! -//! A [`BinarySet`] pairs each [`Platform`] with the on-disk path to its -//! prebuilt library; the [`crate::package`] driver and every packaging backend -//! consume it. +//! A [`BinarySet`] records, per [`Platform`], the files `weaveffi build` laid +//! out in `target/weaveffi//`: the producer library plus any +//! prebuilt glue (the Node.js addon, the JNI shim, the static library an +//! `XCFramework` needs). The [`crate::package`] driver and every packaging +//! backend consume it. -use camino::Utf8PathBuf; +use anyhow::{bail, Context, Result}; +use camino::{Utf8Path, Utf8PathBuf}; /// The operating-system family of a [`Platform`]. #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] @@ -293,15 +296,31 @@ impl Platform { } /// The Python wheel platform tag (the final segment of a wheel filename), - /// for example `macosx_11_0_arm64` or `manylinux2014_x86_64`, or `None` - /// for a platform wheels do not target. - pub fn python_platform_tag(self) -> Option<&'static str> { + /// or `None` for a platform wheels don't target. + /// + /// A macOS tag carries the deployment target the library was built for + /// (`"11.0"` gives `macosx_11_0_arm64`); a Linux tag carries the newest + /// glibc the library links against (`(2, 17)` gives + /// `manylinux_2_17_x86_64`, see [`glibc_requirement`]). + #[must_use] + pub fn python_platform_tag( + self, + macos_deployment_target: &str, + glibc: (u32, u32), + ) -> Option { + let macos = |arch: &str| { + let mut parts = macos_deployment_target.split('.'); + let major = parts.next().filter(|p| !p.is_empty()).unwrap_or("11"); + let minor = parts.next().unwrap_or("0"); + format!("macosx_{major}_{minor}_{arch}") + }; + let linux = |arch: &str| format!("manylinux_{}_{}_{arch}", glibc.0, glibc.1); match self { - Platform::MacosArm64 => Some("macosx_11_0_arm64"), - Platform::MacosX64 => Some("macosx_10_12_x86_64"), - Platform::LinuxX64 => Some("manylinux2014_x86_64"), - Platform::LinuxArm64 => Some("manylinux2014_aarch64"), - Platform::WindowsX64 => Some("win_amd64"), + Platform::MacosArm64 => Some(macos("arm64")), + Platform::MacosX64 => Some(macos("x86_64")), + Platform::LinuxX64 => Some(linux("x86_64")), + Platform::LinuxArm64 => Some(linux("aarch64")), + Platform::WindowsX64 => Some("win_amd64".to_string()), Platform::AndroidArm64 | Platform::AndroidX64 | Platform::Wasm32 @@ -311,6 +330,65 @@ impl Platform { } } + /// Check that this host can build this platform. + /// + /// Apple platforms need a macOS host (the Apple SDKs), Windows needs a + /// Windows host (MSVC), and Linux needs a Linux host (another + /// architecture also needs a cross linker). Android builds anywhere the + /// NDK runs and `wasm32` builds anywhere. + /// + /// # Errors + /// + /// Returns an error naming the host the platform needs. + pub fn check_host(self) -> Result<()> { + let (needed, host_name) = match self.os() { + Os::MacOs | Os::Ios => ("macos", "macOS"), + Os::Windows => ("windows", "Windows"), + Os::Linux => ("linux", "Linux"), + Os::Android | Os::Wasm => return Ok(()), + }; + if std::env::consts::OS != needed { + bail!( + "{} ({}) can only be built on a {host_name} host; build it on a {host_name} \ + machine or CI runner and hand the results to `weaveffi package --binaries`", + self.id(), + self.display_name(), + ); + } + Ok(()) + } + + /// Parse a comma-separated list of platform ids, dropping duplicates and + /// keeping the first-seen order. + /// + /// # Errors + /// + /// Returns an error naming an unknown id (with the valid ones), or when + /// the list is empty. + pub fn parse_list>(ids: &[S]) -> Result> { + let mut out = Vec::new(); + for token in ids + .iter() + .map(|s| s.as_ref().trim()) + .filter(|s| !s.is_empty()) + { + let Some(p) = Platform::from_id(token) else { + let known: Vec<&str> = Platform::ALL.iter().map(|p| p.id()).collect(); + bail!( + "unknown platform `{token}`; expected one of: {}", + known.join(", ") + ); + }; + if !out.contains(&p) { + out.push(p); + } + } + if out.is_empty() { + bail!("the platform list is empty; expected one or more platform ids"); + } + Ok(out) + } + /// The RubyGems platform string used for a precompiled platform gem, for /// example `arm64-darwin` or `x86_64-linux`, or `None` for a platform gems /// do not target. @@ -349,28 +427,123 @@ impl Platform { } } -/// One prebuilt native library: the [`Platform`] it targets and the on-disk -/// path to the shared library file to bundle. +/// The Node.js addon's base name for a library: `{library}_node`, built as +/// `{library}_node.node`. +#[must_use] +pub fn node_addon_name(library: &str) -> String { + format!("{library}_node") +} + +/// The JNI shim's base name for a library: `{library}_jni`, built as +/// `lib{library}_jni.so` (or `.dylib`, or `.dll`). +#[must_use] +pub fn jni_shim_name(library: &str) -> String { + format!("{library}_jni") +} + +/// The newest glibc version a Linux shared library links against, read from +/// the `GLIBC_2.x` symbol-version names in its bytes, and never older than +/// 2.17 (the oldest glibc Rust supports). +/// +/// This picks the `manylinux_2_N` tag of a wheel: a library built on a +/// recent distribution needs that distribution's glibc, whatever the target +/// triple's minimum. +#[must_use] +pub fn glibc_requirement(library: &[u8]) -> (u32, u32) { + const NEEDLE: &[u8] = b"GLIBC_2."; + let mut best = (2, 17); + let mut rest = library; + while let Some(at) = rest.windows(NEEDLE.len()).position(|w| w == NEEDLE) { + rest = &rest[at + NEEDLE.len()..]; + let digits = rest.iter().take_while(|b| b.is_ascii_digit()).count(); + if let Some(minor) = std::str::from_utf8(&rest[..digits]) + .ok() + .and_then(|d| d.parse::().ok()) + { + best = best.max((2, minor)); + } + } + best +} + +/// The files `weaveffi build` laid out for one platform in +/// `target/weaveffi//`. +/// +/// Only [`library`](Self::library) is always present: the producer's shared +/// library (the `.wasm` module on `wasm32`, the static library on iOS). The +/// rest are prebuilt extras that packaging bundles when it finds them. #[derive(Debug, Clone, PartialEq, Eq)] pub struct NativeBinary { - /// The platform this library was built for. + /// The platform these files were built for. pub platform: Platform, - /// Absolute or relative path to the shared library file on disk. - pub source: Utf8PathBuf, + /// The producer library: `lib{library}.dylib`, `lib{library}.so`, + /// `{library}.dll`, `{library}.wasm`, or (iOS) `lib{library}.a`. + pub library: Utf8PathBuf, + /// The static library (`lib{library}.a`) an `XCFramework` slice is made + /// from, on Apple platforms. + pub staticlib: Option, + /// The import library (`{library}.dll.lib`) MSVC links a DLL through, on + /// Windows. + pub import_library: Option, + /// The prebuilt Node.js addon (`{library}_node.node`). + pub node_addon: Option, + /// The prebuilt JNI shim (`lib{library}_jni.so`, `.dylib`, or `.dll`). + pub jni_shim: Option, } -/// The set of prebuilt native libraries to bundle into a package, keyed by -/// platform. +impl NativeBinary { + /// A platform with only its producer library. + pub fn new(platform: Platform, library: impl Into) -> Self { + Self { + platform, + library: library.into(), + staticlib: None, + import_library: None, + node_addon: None, + jni_shim: None, + } + } + + /// Read the files of one `target/weaveffi//` directory, or + /// `None` when it has no producer library named for `lib_name`. + #[must_use] + pub fn read_dir(dir: &Utf8Path, platform: Platform, lib_name: &str) -> Option { + let library = dir.join(platform.lib_filename(lib_name)); + if !library.is_file() { + return None; + } + let existing = |name: String| { + let path = dir.join(name); + path.is_file().then_some(path) + }; + let mut binary = Self::new(platform, library); + binary.staticlib = match platform.os() { + Os::Ios => Some(binary.library.clone()), + Os::MacOs => existing(format!("lib{lib_name}.a")), + _ => None, + }; + if platform.os() == Os::Windows { + binary.import_library = existing(format!("{lib_name}.dll.lib")); + } + binary.node_addon = existing(format!("{}.node", node_addon_name(lib_name))); + if !matches!(platform.os(), Os::Ios | Os::Wasm) { + binary.jni_shim = existing(platform.lib_filename(&jni_shim_name(lib_name))); + } + Some(binary) + } +} + +/// The per-platform build outputs to bundle into packages, keyed by platform. /// -/// `lib_name` is the logical base name every generated loader, import name, and -/// bundled filename is derived from (for example `contacts`, yielding -/// `libcontacts.dylib` / `contacts.dll`). It is the resolved package identity, -/// not the WeaveFFI brand, so the bundled file matches what the producer emits. +/// `lib_name` is the logical base name every generated loader, import name, +/// and bundled filename derives from (for example `contacts`, yielding +/// `libcontacts.dylib` and `contacts.dll`). It is the resolved package +/// identity's library, so the bundled file matches what the bindings load. #[derive(Debug, Clone, PartialEq, Eq)] pub struct BinarySet { - /// Logical shared-library base name (resolved package identity). + /// Logical shared-library base name (the identity's `library`). pub lib_name: String, - /// One entry per platform that has a prebuilt library available. + /// One entry per platform with a build. pub binaries: Vec, } @@ -383,28 +556,72 @@ impl BinarySet { } } - /// Record the prebuilt library `source` for `platform`, replacing any - /// previous entry for the same platform. - pub fn insert(&mut self, platform: Platform, source: impl Into) { - let source = source.into(); - if let Some(existing) = self.binaries.iter_mut().find(|b| b.platform == platform) { - existing.source = source; + /// Read the build outputs under `dir` (laid out as `dir//`) + /// for each of `platforms`, or for every platform with a directory when + /// `platforms` is `None`. + /// + /// # Errors + /// + /// Returns an error when `dir` isn't a directory, or when a requested + /// platform has no producer library under `dir`. + pub fn read_dir( + dir: &Utf8Path, + lib_name: &str, + platforms: Option<&[Platform]>, + ) -> Result { + if !dir.is_dir() { + bail!("{dir} isn't a directory of per-platform builds (`//`)"); + } + let mut set = Self::new(lib_name); + for platform in platforms.unwrap_or(&Platform::ALL) { + let platform_dir = dir.join(platform.id()); + match NativeBinary::read_dir(&platform_dir, *platform, lib_name) { + Some(binary) => set.insert(binary), + None if platforms.is_some() => bail!( + "no {} in {platform_dir}; build {} first (`weaveffi build --platforms {}`)", + platform.lib_filename(lib_name), + platform.id(), + platform.id() + ), + None => {} + } + } + if set.is_empty() { + bail!( + "{dir} has no per-platform builds of `{lib_name}`; expected \ + `{dir}//{}` or similar", + Platform::host() + .unwrap_or(Platform::LinuxX64) + .lib_filename(lib_name) + ); + } + Ok(set) + } + + /// Record `binary`, replacing any previous entry for the same platform. + pub fn insert(&mut self, binary: NativeBinary) { + if let Some(existing) = self + .binaries + .iter_mut() + .find(|b| b.platform == binary.platform) + { + *existing = binary; } else { - self.binaries.push(NativeBinary { platform, source }); + self.binaries.push(binary); } } - /// The library built for `platform`, if present. + /// The build for `platform`, if present. pub fn get(&self, platform: Platform) -> Option<&NativeBinary> { self.binaries.iter().find(|b| b.platform == platform) } - /// Every platform with a bundled library, in insertion order. + /// Every platform with a build, in insertion order. pub fn platforms(&self) -> impl Iterator + '_ { self.binaries.iter().map(|b| b.platform) } - /// True when no binaries have been recorded. + /// True when no builds have been recorded. pub fn is_empty(&self) -> bool { self.binaries.is_empty() } @@ -416,6 +633,15 @@ impl BinarySet { } } +/// Read a whole file, naming it in the error. +/// +/// # Errors +/// +/// Returns an error when the file can't be read. +pub fn read_file(path: &Utf8Path) -> Result> { + std::fs::read(path.as_std_path()).with_context(|| format!("failed to read {path}")) +} + #[cfg(test)] mod tests { use super::*; @@ -465,8 +691,22 @@ mod tests { assert_eq!(Platform::MacosX64.node_cpu(), Some("x64")); assert_eq!(Platform::WindowsX64.node_os(), Some("win32")); assert_eq!( - Platform::LinuxArm64.python_platform_tag(), - Some("manylinux2014_aarch64") + Platform::LinuxArm64 + .python_platform_tag("11.0", (2, 17)) + .as_deref(), + Some("manylinux_2_17_aarch64") + ); + assert_eq!( + Platform::MacosArm64 + .python_platform_tag("12.3", (2, 17)) + .as_deref(), + Some("macosx_12_3_arm64") + ); + assert_eq!( + Platform::WindowsX64 + .python_platform_tag("11.0", (2, 17)) + .as_deref(), + Some("win_amd64") ); assert_eq!(Platform::MacosArm64.ruby_platform(), Some("arm64-darwin")); assert_eq!(Platform::LinuxX64.ruby_platform(), Some("x86_64-linux")); @@ -480,7 +720,7 @@ mod tests { ] { assert!(!p.is_desktop()); assert_eq!(p.nuget_rid(), None); - assert_eq!(p.python_platform_tag(), None); + assert_eq!(p.python_platform_tag("11.0", (2, 17)), None); assert_eq!(p.ruby_platform(), None); } assert!(Platform::DESKTOP.iter().all(|p| p.is_desktop())); @@ -490,20 +730,96 @@ mod tests { fn binary_set_insert_get_and_replace() { let mut set = BinarySet::new("contacts"); assert!(set.is_empty()); - set.insert(Platform::MacosArm64, "/tmp/a/libcontacts.dylib"); - set.insert(Platform::LinuxX64, "/tmp/b/libcontacts.so"); + set.insert(NativeBinary::new( + Platform::MacosArm64, + "/a/libcontacts.dylib", + )); + set.insert(NativeBinary::new(Platform::LinuxX64, "/b/libcontacts.so")); assert_eq!(set.binaries.len(), 2); // Re-inserting the same platform replaces rather than duplicates. - set.insert(Platform::MacosArm64, "/tmp/c/libcontacts.dylib"); + set.insert(NativeBinary::new( + Platform::MacosArm64, + "/c/libcontacts.dylib", + )); assert_eq!(set.binaries.len(), 2); assert_eq!( - set.get(Platform::MacosArm64).unwrap().source.as_str(), - "/tmp/c/libcontacts.dylib" + set.get(Platform::MacosArm64).unwrap().library.as_str(), + "/c/libcontacts.dylib" ); let platforms: Vec = set.platforms().collect(); assert_eq!(platforms, vec![Platform::MacosArm64, Platform::LinuxX64]); assert_eq!(set.bundled_filename(Platform::WindowsX64), "contacts.dll"); } + + #[test] + fn build_directories_are_read_with_their_extras() { + let dir = tempfile::tempdir().unwrap(); + let root = Utf8Path::from_path(dir.path()).unwrap(); + let mac = root.join("darwin-arm64"); + let linux = root.join("linux-x64"); + std::fs::create_dir_all(&mac).unwrap(); + std::fs::create_dir_all(&linux).unwrap(); + for f in [ + "libcontacts.dylib", + "libcontacts.a", + "contacts_node.node", + "libcontacts_jni.dylib", + ] { + std::fs::write(mac.join(f), b"x").unwrap(); + } + std::fs::write(linux.join("libcontacts.so"), b"x").unwrap(); + + let set = BinarySet::read_dir(root, "contacts", None).unwrap(); + assert_eq!(set.binaries.len(), 2); + let m = set.get(Platform::MacosArm64).unwrap(); + assert_eq!( + m.staticlib.as_deref(), + Some(mac.join("libcontacts.a").as_path()) + ); + assert!(m.node_addon.is_some() && m.jni_shim.is_some()); + let l = set.get(Platform::LinuxX64).unwrap(); + assert!(l.staticlib.is_none() && l.node_addon.is_none() && l.jni_shim.is_none()); + + let err = BinarySet::read_dir(root, "contacts", Some(&[Platform::WindowsX64])).unwrap_err(); + assert!(err.to_string().contains("contacts.dll"), "{err}"); + assert!(BinarySet::read_dir(&root.join("missing"), "contacts", None).is_err()); + } + + #[test] + fn platform_lists_parse_and_reject_unknown_ids() { + assert_eq!( + Platform::parse_list(&["darwin-arm64", " linux-x64", "darwin-arm64"]).unwrap(), + [Platform::MacosArm64, Platform::LinuxX64] + ); + let err = Platform::parse_list(&["solaris-sparc"]).unwrap_err(); + assert!( + err.to_string().contains("unknown platform `solaris-sparc`"), + "{err}" + ); + assert!(Platform::parse_list::<&str>(&[]).is_err()); + } + + #[test] + fn host_checks_name_the_needed_host() { + assert!(Platform::Wasm32.check_host().is_ok()); + assert!(Platform::AndroidArm64.check_host().is_ok()); + let foreign = if cfg!(target_os = "windows") { + Platform::MacosArm64 + } else { + Platform::WindowsX64 + }; + let err = foreign.check_host().unwrap_err(); + assert!(err.to_string().contains("can only be built on"), "{err}"); + } + + #[test] + fn glibc_requirement_takes_the_newest_version_named() { + assert_eq!(glibc_requirement(b"nothing here"), (2, 17)); + assert_eq!( + glibc_requirement(b"\0GLIBC_2.2.5\0GLIBC_2.28\0GLIBC_2.3.4\0"), + (2, 28) + ); + } } diff --git a/crates/weaveffi-cli/src/project.rs b/crates/weaveffi-cli/src/project.rs new file mode 100644 index 00000000..0dd8ea87 --- /dev/null +++ b/crates/weaveffi-cli/src/project.rs @@ -0,0 +1,553 @@ +//! A WeaveFFI project: where its API comes from, what its library is +//! called, and how to turn it into bindings. +//! +//! A project's input is either an IDL document or a Rust producer crate. +//! A crate's API is read from its built library (see [`crate::library`]): +//! [`Project`] builds the crate with Cargo, or reads a library built +//! earlier ([`Project::library`]). Every `weaveffi` subcommand starts here, +//! and so can a `build.rs`: +//! +//! ```no_run +//! // build.rs of a crate that ships bindings for an IDL-defined library. +//! fn main() -> miette::Result<()> { +//! println!("cargo::rerun-if-changed=weaveffi.toml"); +//! println!("cargo::rerun-if-changed=api.yml"); +//! weaveffi_cli::project::Project::discover(env!("CARGO_MANIFEST_DIR"))?.generate()?; +//! Ok(()) +//! } +//! ``` +//! +//! A build script can't read the library of the crate it's building, which +//! doesn't exist yet, so from a `build.rs` the project's input must be an +//! IDL, or a library built beforehand that [`Project::library`] names. + +use camino::{Utf8Path, Utf8PathBuf}; +use miette::{bail, miette, IntoDiagnostic, Report, Result, WrapErr}; +use weaveffi_model::ir::Api; +use weaveffi_model::meta; +use weaveffi_model::model::Model; +use weaveffi_model::parse::parse_api_str; +use weaveffi_model::pkg::{name_from_basename, Identity, Package}; +use weaveffi_model::validate::{ + collect_warnings, validate, ValidationDiagnostics, ValidationWarning, +}; + +use crate::cargo::CargoCrate; +use crate::codegen::{GenerateReport, Orchestrator}; +use crate::config::{ProjectConfig, CONFIG_FILE_NAME}; +use crate::library; +use crate::report::with_named_source; + +/// Where a project's API comes from. +#[derive(Debug, Clone)] +pub enum Source { + /// An IDL document (`.yml`, `.yaml`, `.json`, or `.toml`). + Idl(Utf8PathBuf), + /// A Rust producer crate, whose API is read from its built library. + Crate(Box), + /// A built library with no crate or IDL beside it. + Library(Utf8PathBuf), +} + +/// A located project: its configuration and the source of its API. +#[derive(Debug, Clone)] +pub struct Project { + /// The `weaveffi.toml` settings (the defaults when there is none). + pub config: ProjectConfig, + /// Where the API comes from. + pub source: Source, + /// A built library to read a crate's API from instead of building it. + library: Option, + release: bool, + quiet: bool, +} + +/// A project's API before validation, with the identity it's validated +/// against. +#[derive(Debug, Clone)] +pub struct Definition { + /// The API as written in the IDL or embedded in the library. + pub api: Api, + /// The library's identity. + pub identity: Identity, + /// The IDL's file name and text, for diagnostics that quote it. + text: Option<(String, String)>, +} + +impl Definition { + /// Validate the API into the [`Model`] every generator renders from. + /// + /// # Errors + /// + /// Returns every validation error at once. + pub fn validate(&self) -> Result { + let source = self.text.as_ref().map(|(n, t)| (n.as_str(), t.as_str())); + validate(&self.api, &self.identity, source) + } + + /// The advisory warnings for the API. + #[must_use] + pub fn warnings(&self) -> Vec { + collect_warnings(&self.api) + } +} + +impl Project { + /// The project of the nearest `weaveffi.toml` at or above `dir`, whose + /// `[project] input` names the API; without one, the crate at `dir`. + /// + /// # Errors + /// + /// Returns an error when there is neither, when the config doesn't + /// parse, or when its input can't be resolved. + pub fn discover(dir: impl AsRef) -> Result { + let dir = dir.as_ref(); + if let Some(config) = discover_config(&dir.join(CONFIG_FILE_NAME)) { + return Self::locate(Some(config.as_str()), None, None); + } + if dir.join("Cargo.toml").is_file() { + return Self::locate(None, Some(dir.as_str()), None); + } + Err(miette!( + "no {CONFIG_FILE_NAME} at or above {dir} and no Cargo.toml in it; run `weaveffi init`" + )) + } + + /// Locate a project the way the CLI does: from `input` (a crate + /// directory, a `Cargo.toml`, or an IDL file) and the nearest + /// `weaveffi.toml` at or above it, or from the config named by `config` + /// (else found from the current directory) and its `[project] input`. + /// `library` is a built library to read the API from: the crate's own, + /// or, with no input at all, a library standing alone. + /// + /// # Errors + /// + /// Returns an error when no input can be found, a config file doesn't + /// parse, the input isn't a crate or an IDL, or `library` is given for + /// an IDL input. + pub fn locate( + config: Option<&str>, + input: Option<&str>, + library: Option<&str>, + ) -> Result { + let (config, input) = match (input, library) { + (Some(input), _) => { + let path = Utf8PathBuf::from(input); + let start = if path.is_dir() { + path.join(CONFIG_FILE_NAME) + } else { + path.clone() + }; + (ProjectConfig::load(config, &start)?, Some(path)) + } + (None, Some(_)) => { + let config = match config { + Some(p) => ProjectConfig::from_file(Utf8Path::new(p))?, + None => current_dir() + .ok() + .and_then(|cwd| discover_config(&cwd.join(CONFIG_FILE_NAME))) + .map(|p| ProjectConfig::from_file(&p)) + .transpose()? + .unwrap_or_default(), + }; + let input = config + .project + .input + .as_deref() + .map(|i| config.relative_to_config(i)); + (config, input) + } + (None, None) => { + let (config, input) = ProjectConfig::locate(config, None)?; + (config, Some(input)) + } + }; + let source = match input { + Some(input) => source_of(&input)?, + None => Source::Library(Utf8PathBuf::from(library.unwrap_or_default())), + }; + let mut project = Self { + config, + source, + library: None, + release: false, + quiet: false, + }; + if let Some(library) = library { + project = project.library(library)?; + } + Ok(project) + } + + /// Read a Rust producer's API from the library at `path` instead of + /// building the crate. + /// + /// # Errors + /// + /// Returns an error when the project's input is an IDL. + pub fn library(mut self, path: impl Into) -> Result { + let path = path.into(); + match &mut self.source { + Source::Idl(idl) => { + return Err(miette!( + "--library reads a Rust producer's API from its built library, but {idl} is \ + an IDL; pass the producer crate instead" + )) + } + Source::Crate(_) => self.library = Some(path), + Source::Library(lib) => *lib = path, + } + Ok(self) + } + + /// Build a crate's library in release mode instead of debug. + #[must_use] + pub fn release(mut self, release: bool) -> Self { + self.release = release; + self + } + + /// Pass `--quiet` to Cargo. + #[must_use] + pub fn quiet(mut self, quiet: bool) -> Self { + self.quiet = quiet; + self + } + + /// Whether Cargo runs with `--quiet`. + #[must_use] + pub fn is_quiet(&self) -> bool { + self.quiet + } + + /// The producer crate, when the input is one. + #[must_use] + pub fn krate(&self) -> Option<&CargoCrate> { + match &self.source { + Source::Crate(k) => Some(k), + _ => None, + } + } + + /// The library a Rust producer's API is read from: the one named with + /// [`library`](Self::library), else the crate's, built now. + /// + /// # Errors + /// + /// Returns an error for an IDL input, from inside the crate's own build + /// script, or when the build fails. + pub fn library_path(&self) -> Result { + match &self.source { + Source::Idl(idl) => Err(miette!("{idl} is an IDL; it has no library to read")), + Source::Library(lib) => Ok(lib.clone()), + Source::Crate(krate) => { + if let Some(lib) = &self.library { + return Ok(lib.clone()); + } + if in_own_build_script(krate) { + bail!( + "`{}`'s build script can't read the API of the library it's building: \ + point `[project] input` at an IDL, or read a library built earlier \ + with `Project::library`", + krate.name + ); + } + krate + .build_library(self.release, self.quiet) + .map_err(|e| miette!("{e:#}")) + } + } + } + + /// Read the API and resolve the library's identity. + /// + /// # Errors + /// + /// Returns an error when the IDL can't be read or parsed, the library + /// can't be built or read or holds no metadata for the crate, or the + /// `[package]` table sets a key that doesn't apply to the input. + pub fn definition(&self) -> Result { + let package = &self.config.package.identity; + match &self.source { + Source::Idl(path) => { + let contents = std::fs::read_to_string(path.as_std_path()) + .into_diagnostic() + .wrap_err_with(|| format!("failed to read input file: {path}"))?; + let api = parse_api_str(&contents, idl_format(path)?) + .map_err(|e| with_named_source(e, path.as_str(), &contents))?; + Ok(Definition { + api, + identity: Identity::new(&name_from_basename(path.as_str()), package), + text: Some((path.to_string(), contents)), + }) + } + Source::Crate(krate) => { + rust_only_keys(package)?; + let path = self.library_path()?; + let api = read_api(&path, &krate.lib_name)?; + Ok(Definition { + api, + identity: crate_identity(krate, package), + text: None, + }) + } + Source::Library(path) => { + rust_only_keys(package)?; + let frames = library::read_frames(path).map_err(|e| miette!("{e:#}"))?; + let prefixes: std::collections::BTreeSet<&str> = + frames.iter().map(|f| f.prefix.as_str()).collect(); + let prefix = match prefixes.len() { + 0 => return Err(no_metadata(path, None)), + 1 => prefixes.into_iter().next().unwrap_or_default().to_string(), + _ => { + return Err(miette!( + "{path} holds the APIs of several crates ({}); pass the producer \ + crate so WeaveFFI knows which one to bind", + prefixes.into_iter().collect::>().join(", ") + )) + } + }; + let api = meta::assemble(&frames, &prefix).map_err(|e| miette!("{path}: {e}"))?; + let mut merged = package.clone(); + merged.c_prefix = Some(prefix.clone()); + merged.library = Some(library::library_name(path)); + Ok(Definition { + api, + identity: Identity::new(&prefix, &merged), + text: None, + }) + } + } + } + + /// Read and validate the API into the [`Model`]. + /// + /// # Errors + /// + /// Returns the errors of [`definition`](Self::definition) and every + /// validation error. + pub fn model(&self) -> Result { + self.definition()?.validate().map_err(Report::new) + } + + /// Generate the `[project] targets` (else every target) into the + /// `[project] out` directory, writing only changed files and removing + /// stale ones. + /// + /// # Errors + /// + /// Returns the errors of [`model`](Self::model), an unknown target name, + /// or a failure to write the output. + pub fn generate(&self) -> Result { + let model = self.model()?; + let out = self.config.out_dir(None); + let targets = self.config.select_targets(None)?; + std::fs::create_dir_all(out.as_std_path()) + .into_diagnostic() + .wrap_err_with(|| format!("failed to create output directory: {out}"))?; + let mut orchestrator = Orchestrator::new(); + for target in &targets { + orchestrator = orchestrator.with_target(target.as_ref()); + } + orchestrator.run(&model, &out).map_err(|e| miette!("{e:#}")) + } +} + +/// What `input` is: a crate (a directory with a `Cargo.toml`, or the +/// manifest itself) or an IDL file. +fn source_of(input: &Utf8Path) -> Result { + let manifest = if input.is_dir() { + let manifest = input.join("Cargo.toml"); + if !manifest.is_file() { + bail!( + "{input} is a directory without a Cargo.toml; the input is a Rust producer crate \ + or an IDL file (yml|yaml|json|toml)" + ); + } + Some(manifest) + } else if input.file_name() == Some("Cargo.toml") { + Some(input.to_path_buf()) + } else { + None + }; + match manifest { + Some(m) => Ok(Source::Crate(Box::new( + CargoCrate::resolve(&m).map_err(|e| miette!("{e:#}"))?, + ))), + None => { + idl_format(input)?; + Ok(Source::Idl(input.to_path_buf())) + } + } +} + +/// The parser's format token for an IDL file's extension. +fn idl_format(path: &Utf8Path) -> Result<&'static str> { + match path.extension().unwrap_or("") { + "yml" | "yaml" => Ok("yaml"), + "json" => Ok("json"), + "toml" => Ok("toml"), + "rs" => Err(miette!( + "{path} is Rust source; WeaveFFI reads a Rust producer's API from its built \ + library, so pass the crate (its directory or Cargo.toml) instead" + )), + other => Err(miette!( + "unsupported input {path}{}: expected a Rust producer crate (its directory or \ + Cargo.toml) or an IDL file (yml|yaml|json|toml)", + if other.is_empty() { + String::new() + } else { + format!(" (.{other})") + } + )), + } +} + +/// Reject `[package]` keys a Rust producer can't override. +fn rust_only_keys(package: &Package) -> Result<()> { + if package.c_prefix.is_some() || package.library.is_some() { + bail!( + "`[package] c_prefix` and `[package] library` apply only to IDL inputs: a Rust \ + producer's C prefix and library name are its crate's library name" + ); + } + Ok(()) +} + +/// A Rust producer's identity: named after its package, with its library +/// name as both the C prefix (what `#[weaveffi::module]` uses) and the +/// library, and `[package]` metadata over the crate's. +fn crate_identity(krate: &CargoCrate, package: &Package) -> Identity { + let mut merged = package.clone(); + merged.version = merged.version.or_else(|| Some(krate.version.clone())); + merged.description = merged.description.or_else(|| krate.description.clone()); + merged.license = merged.license.or_else(|| krate.license.clone()); + merged.homepage = merged.homepage.or_else(|| krate.homepage.clone()); + merged.repository = merged.repository.or_else(|| krate.repository.clone()); + if merged.authors.is_empty() { + merged.authors = krate.authors.clone(); + } + merged.c_prefix = Some(krate.lib_name.clone()); + merged.library = Some(krate.lib_name.clone()); + Identity::new(&krate.name, &merged) +} + +/// The API the library at `path` embeds for the crate with `prefix`. +fn read_api(path: &Utf8Path, prefix: &str) -> Result { + let frames = library::read_frames(path).map_err(|e| miette!("{e:#}"))?; + let api = meta::assemble(&frames, prefix).map_err(|e| miette!("{path}: {e}"))?; + if api.modules.is_empty() { + let others: std::collections::BTreeSet<&str> = + frames.iter().map(|f| f.prefix.as_str()).collect(); + return Err(no_metadata( + path, + Some((prefix, others.into_iter().collect())), + )); + } + Ok(api) +} + +/// The error for a library without (the crate's) metadata. +fn no_metadata(path: &Utf8Path, crate_prefix: Option<(&str, Vec<&str>)>) -> Report { + let whose = match &crate_prefix { + Some((prefix, _)) => format!(" for `{prefix}`"), + None => String::new(), + }; + let others = match &crate_prefix { + Some((_, others)) if !others.is_empty() => { + format!(" (it has metadata for {})", others.join(", ")) + } + _ => String::new(), + }; + miette!( + "{path} has no WeaveFFI metadata{whose}{others}. The API is read from the library the \ + producer builds: annotate it with `#[weaveffi::module]`, call \ + `weaveffi::export_runtime!();` once in the crate, and build it again" + ) +} + +/// Whether this process is `krate`'s own build script. +fn in_own_build_script(krate: &CargoCrate) -> bool { + std::env::var_os("OUT_DIR").is_some() + && std::env::var("CARGO_MANIFEST_DIR").is_ok_and(|dir| { + Utf8Path::new(&dir).canonicalize_utf8().ok() == krate.dir().canonicalize_utf8().ok() + }) +} + +/// The current directory as a UTF-8 path. +fn current_dir() -> std::io::Result { + Utf8PathBuf::from_path_buf(std::env::current_dir()?) + .map_err(|_| std::io::Error::other("current directory is not valid UTF-8")) +} + +/// The nearest `weaveffi.toml` at or above the directory of `start`. +fn discover_config(start: &Utf8Path) -> Option { + crate::config::discover(start) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn repo_root() -> Utf8PathBuf { + Utf8Path::new(env!("CARGO_MANIFEST_DIR")).join("../..") + } + + #[test] + fn crates_and_idls_are_told_apart() { + let root = repo_root(); + for input in [ + root.join("samples/kvstore"), + root.join("samples/kvstore/Cargo.toml"), + ] { + let project = Project::locate(None, Some(input.as_str()), None).unwrap(); + let krate = project.krate().expect("a crate"); + assert_eq!(krate.lib_name, "kvstore"); + // The sample's weaveffi.toml sits beside its Cargo.toml. + assert_eq!( + project.config.package.identity.version.as_deref(), + Some("1.0.0") + ); + } + let idl = root.join("crates/weaveffi-cli/tests/fixtures/kitchen_sink.yml"); + let project = Project::locate(None, Some(idl.as_str()), None).unwrap(); + assert!(matches!(project.source, Source::Idl(_))); + let def = project.definition().unwrap(); + assert_eq!(def.identity.prefix, "kitchen_sink"); + let err = project.library("libx.so").unwrap_err(); + assert!(format!("{err}").contains("is an IDL"), "{err}"); + + let err = Project::locate(None, Some("src/lib.rs"), None).unwrap_err(); + assert!(format!("{err}").contains("pass the crate"), "{err}"); + let err = Project::locate(None, Some(root.join("docs").as_str()), None).unwrap_err(); + assert!(format!("{err}").contains("without a Cargo.toml"), "{err}"); + } + + #[test] + fn crate_identity_comes_from_cargo_and_the_package_table() { + let krate = CargoCrate::resolve(&repo_root().join("samples/kvstore/Cargo.toml")).unwrap(); + let id = crate_identity(&krate, &Package::default()); + assert_eq!( + (id.name.as_str(), id.prefix.as_str(), id.library.as_str()), + ("kvstore", "kvstore", "kvstore") + ); + assert_eq!(id.version, "0.1.0"); + let id = crate_identity( + &krate, + &Package { + name: Some("acme-kv".into()), + version: Some("2.0.0".into()), + ..Package::default() + }, + ); + assert_eq!( + (id.name.as_str(), id.prefix.as_str(), id.version.as_str()), + ("acme-kv", "kvstore", "2.0.0") + ); + assert!(rust_only_keys(&Package { + c_prefix: Some("x".into()), + ..Package::default() + }) + .is_err()); + } +} diff --git a/crates/weaveffi-gen/src/targets/c/buffer.rs b/crates/weaveffi-cli/src/targets/c/buffer.rs similarity index 92% rename from crates/weaveffi-gen/src/targets/c/buffer.rs rename to crates/weaveffi-cli/src/targets/c/buffer.rs index 116ddeaa..92eb8632 100644 --- a/crates/weaveffi-gen/src/targets/c/buffer.rs +++ b/crates/weaveffi-cli/src/targets/c/buffer.rs @@ -10,7 +10,7 @@ //! //! The fixed reader and writer live in `runtime/buffer_runtime.h` and are //! spliced in with the prefix substituted; everything after them is rendered -//! from the [`BindingModel`]. +//! from the [`Model`]. use std::collections::{HashMap, HashSet}; @@ -18,9 +18,8 @@ use crate::cabi::c_param_name; use crate::codegen::common::DocCommentStyle; use crate::codegen::CodeWriter; use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use weaveffi_model::model::{ - BindingModel, EnumBinding, FieldBinding, InterfaceBinding, Prim, Ty, WireType, -}; +use weaveffi_model::model::{EnumBinding, FieldBinding, InterfaceBinding, Model}; +use weaveffi_model::ty::{Prim, Ty, WireType}; /// The fixed reader, writer, and string view types. const RUNTIME: &str = include_str!("runtime/buffer_runtime.h"); @@ -47,15 +46,12 @@ struct UserType<'a> { /// Every type the header renders, indexed for lookup by resolved name. struct Codecs<'a> { + model: &'a Model, prefix: &'a str, /// Records and rich enums in declaration order, then error payloads. users: Vec>, - /// Absolute dot name (`kv.Entry`) to an index into `users`. - user_index: HashMap, - /// C-style enums by absolute dot name. - enums: HashMap, - /// Interfaces by absolute dot name. - interfaces: HashMap, + /// Record or rich enum name to an index into `users`. + user_index: HashMap<&'a str, usize>, /// Optional, list, and map shapes, innermost first, without duplicates. composites: Vec, } @@ -72,20 +68,15 @@ pub fn buffer_header_name(library: &str) -> String { /// `header_name` is the main header the helper includes; `file_name` is the /// helper's own name, used in its trailer. #[must_use] -pub fn render_buffer_header( - model: &BindingModel, - input_basename: &str, - header_name: &str, - file_name: &str, -) -> Option { +pub fn render_buffer_header(model: &Model, header_name: &str, file_name: &str) -> Option { if !model.has_buffers() { return None; } let codecs = Codecs::new(model); - let prefix = model.prefix.as_str(); + let prefix = model.prefix(); let guard = format!("{}_BUFFER_H", prefix.to_uppercase()); - let mut out = render_prelude(CommentStyle::DoubleSlash, input_basename); + let mut out = render_prelude(CommentStyle::DoubleSlash); out.push_str(&format!("#ifndef {guard}\n#define {guard}\n\n")); out.push_str(&format!("#include \"{header_name}\"\n\n")); out.push_str("#include \n#include \n\n"); @@ -112,45 +103,35 @@ struct Unit { } impl<'a> Codecs<'a> { - fn new(model: &'a BindingModel) -> Self { - let prefix = model.prefix.as_str(); + fn new(model: &'a Model) -> Self { + let prefix = model.prefix(); let mut users = Vec::new(); let mut user_index = HashMap::new(); - let mut enums = HashMap::new(); - let mut interfaces = HashMap::new(); for m in &model.modules { - let qualify = |name: &str| format!("{}.{name}", m.dot_path); for s in &m.structs { - user_index.insert(qualify(&s.name), users.len()); + user_index.insert(s.name.as_str(), users.len()); users.push(UserType { name: format!("{prefix}_{}_{}", m.path, s.name), doc: with_deprecation(&s.doc, &s.deprecated), body: Body::Fields(&s.fields), }); } - for e in &m.enums { - if e.rich { - user_index.insert(qualify(&e.name), users.len()); - users.push(UserType { - name: e.c_tag.clone(), - doc: with_deprecation(&e.doc, &e.deprecated), - body: Body::Variants(e), - }); - } else { - enums.insert(qualify(&e.name), e); - } - } - for i in &m.interfaces { - interfaces.insert(qualify(&i.name), i); + for e in m.enums.iter().filter(|e| e.rich) { + user_index.insert(e.name.as_str(), users.len()); + users.push(UserType { + name: e.c_tag.clone(), + doc: with_deprecation(&e.doc, &e.deprecated), + body: Body::Variants(e), + }); } } for m in &model.modules { - let Some(err) = m.error.as_ref().filter(|e| e.declared_here) else { + let Some(err) = m.errors.as_ref() else { continue; }; for code in err.codes.iter().filter(|c| !c.fields.is_empty()) { users.push(UserType { - name: format!("{}_payload", code.c_const), + name: code.payload_tag(), doc: Some(format!( "The fields a `{}` error carries in `payload_ptr`/`payload_len`.", code.c_const @@ -196,11 +177,10 @@ impl<'a> Codecs<'a> { } Codecs { + model, prefix, users, user_index, - enums, - interfaces, composites, } } @@ -214,17 +194,11 @@ impl<'a> Codecs<'a> { } fn interface(&self, name: &str) -> &InterfaceBinding { - self.interfaces - .get(name) - .unwrap_or_else(|| panic!("unresolved interface '{name}'")) + self.model.interface(name) } fn enum_tag(&self, name: &str) -> &str { - &self - .enums - .get(name) - .unwrap_or_else(|| panic!("unresolved enum '{name}'")) - .c_tag + &self.model.enumeration(name).c_tag } /// The codec stem of a record, rich enum, optional, list, or map. @@ -232,7 +206,7 @@ impl<'a> Codecs<'a> { match ty { Ty::Record(n) | Ty::RichEnum(n) => Some(self.user(n).name.clone()), Ty::Optional(_) | Ty::List(_) | Ty::Map(_, _) => { - Some(format!("{}_{}", self.prefix, mangle(ty))) + Some(format!("{}_{}", self.prefix, mangle(self.model, ty))) } _ => None, } @@ -249,8 +223,8 @@ impl<'a> Codecs<'a> { Ty::Interface(n) => format!("{}*", self.interface(n).c_tag), Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => self.vtype(inner), Ty::Optional(inner) => format!("{}*", self.vtype(inner)), - Ty::StringUtf8 => format!("{p}_str"), - Ty::Bytes => format!("{p}_bytes"), + Ty::Prim(Prim::String) => format!("{p}_str"), + Ty::Prim(Prim::Bytes) => format!("{p}_bytes"), _ => prim_ctype(prim(ty)).to_string(), } } @@ -310,8 +284,8 @@ impl<'a> Codecs<'a> { } let p = self.prefix; Some(match ty { - Ty::StringUtf8 => format!("{p}_str_free(&{lv});"), - Ty::Bytes => format!("{p}_bytes_free(&{lv});"), + Ty::Prim(Prim::String) => format!("{p}_str_free(&{lv});"), + Ty::Prim(Prim::Bytes) => format!("{p}_bytes_free(&{lv});"), Ty::Interface(n) => format!("{}({lv});", self.interface(n).destroy_symbol), _ => format!("{}_free(&{lv});", self.named(ty).unwrap_or_default()), }) @@ -320,8 +294,7 @@ impl<'a> Codecs<'a> { /// Whether a decoded value of `ty` owns memory or object references. fn needs_free(&self, ty: &Ty, depth: usize) -> bool { match ty { - Ty::StringUtf8 - | Ty::Bytes + Ty::Prim(Prim::String | Prim::Bytes) | Ty::Interface(_) | Ty::Optional(_) | Ty::List(_) @@ -506,7 +479,7 @@ impl<'a> Codecs<'a> { }; for f in fields { if let Ty::Record(n) | Ty::RichEnum(n) = &f.ty { - if let Some(dep) = c.user_index.get(n) { + if let Some(dep) = c.user_index.get(n.as_str()) { visit(c, *dep, done, order); } } @@ -825,14 +798,15 @@ fn collect(ty: &Ty, out: &mut Vec, seen: &mut HashSet) { } /// The identifier fragment naming `ty` inside a composite's codec stem: -/// `string`, `kv_Entry`, `opt_i64`, `list_string`, `map_string_i64`. -fn mangle(ty: &Ty) -> String { +/// `string`, `kv_Entry` (a user type with its module path), `opt_i64`, +/// `list_string`, `map_string_i64`. +fn mangle(model: &Model, ty: &Ty) -> String { match ty { - Ty::Optional(inner) => format!("opt_{}", mangle(inner)), - Ty::List(inner) => format!("list_{}", mangle(inner)), - Ty::Map(k, v) => format!("map_{}_{}", mangle(k), mangle(v)), + Ty::Optional(inner) => format!("opt_{}", mangle(model, inner)), + Ty::List(inner) => format!("list_{}", mangle(model, inner)), + Ty::Map(k, v) => format!("map_{}_{}", mangle(model, k), mangle(model, v)), _ => match ty.user_name() { - Some(name) => name.replace('.', "_"), + Some(name) => format!("{}_{name}", model.owner(name).path), None => prim(ty).snake().to_string(), }, } diff --git a/crates/weaveffi-gen/src/targets/c/header.rs b/crates/weaveffi-cli/src/targets/c/header.rs similarity index 73% rename from crates/weaveffi-gen/src/targets/c/header.rs rename to crates/weaveffi-cli/src/targets/c/header.rs index 0214cd45..0fe0ee32 100644 --- a/crates/weaveffi-gen/src/targets/c/header.rs +++ b/crates/weaveffi-cli/src/targets/c/header.rs @@ -8,36 +8,22 @@ use std::fmt::Write; use crate::cabi; use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; +use weaveffi_model::model::Model; -/// Render the complete header for `api`. -/// -/// Thin wrapper over [`render_c_header_from_model`] for tests and callers -/// that only hold a [`ResolvedApi`]; the production path renders directly -/// from the driver-built [`BindingModel`] without re-deriving it. -pub fn render_c_header(api: &ResolvedApi, input_basename: &str, filename: &str) -> String { - render_c_header_from_model(&BindingModel::build(api), input_basename, filename) -} - -/// Render the complete header from the shared [`BindingModel`]. +/// Render the complete header from the shared [`Model`]. /// /// The per-declaration rendering is shared with the C++ backend via /// [`crate::cabi`]; this function only adds the header framing /// (include guard, includes, the value-buffer convention comment). The -/// C symbol prefix is read from [`BindingModel::prefix`], so every name already +/// C symbol prefix is read from [`Model::prefix`], so every name already /// agrees with the symbols baked into the model. Parameter names that collide /// with a C or C++ keyword are escaped inside `cabi` itself (see /// [`crate::cabi::c_param_name`]). -pub fn render_c_header_from_model( - model: &BindingModel, - input_basename: &str, - filename: &str, -) -> String { - let prefix = model.prefix.as_str(); +pub fn render_c_header_from_model(model: &Model, filename: &str) -> String { + let prefix = model.prefix(); let guard = format!("{}_H", prefix.to_uppercase()); let mut out = String::with_capacity(2048 + model.modules.len() * 4096); - out.push_str(&render_prelude(CommentStyle::DoubleSlash, input_basename)); + out.push_str(&render_prelude(CommentStyle::DoubleSlash)); let _ = write!(out, "#ifndef {guard}\n#define {guard}\n\n"); out.push_str("#include \n"); out.push_str("#include \n"); diff --git a/crates/weaveffi-cli/src/targets/c/mod.rs b/crates/weaveffi-cli/src/targets/c/mod.rs new file mode 100644 index 00000000..d86d30e1 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/c/mod.rs @@ -0,0 +1,116 @@ +//! C header generator for WeaveFFI. +//! +//! Emits `{library}.h`, describing the stable C ABI surface of a [`Model`]. +//! This is the canonical backend: the header it emits *is* the C ABI every +//! other language binds to, and a C or C++ producer implementing +//! an IDL by hand implements exactly the functions it declares. +//! +//! When the API passes any value buffers and [`CConfig::buffer_helpers`] is +//! on (the default), it also emits `{library}_buffer.h`: C structs for the +//! API's records, rich enums, lists, and maps, with `static inline` codecs +//! that encode and decode them (see [`render_buffer_header`]). +//! +//! Like every WeaveFFI backend it renders from the shared [`Model`], so +//! symbol names and parameter lowering are computed once and shared, never +//! re-derived here. + +mod buffer; +mod header; +mod package; +use crate::backend::{LanguageBackend, OutputFile}; +use crate::package::{per_platform_libraries, Artifact, PackageContext, PackagedFile}; +use camino::Utf8Path; +use serde::{Deserialize, Serialize}; +use weaveffi_model::model::Model; + +pub use buffer::{buffer_header_name, render_buffer_header}; +pub use header::render_c_header_from_model; + +use package::{render_packaged_cmake, render_packaged_readme}; + +/// Per-target configuration for [`CGenerator`]. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct CConfig { + /// Emit `{library}_buffer.h`, the value-buffer helper header, next to + /// `{library}.h` when the API passes any value buffers. On by default. + pub buffer_helpers: bool, +} + +impl Default for CConfig { + fn default() -> Self { + Self { + buffer_helpers: true, + } + } +} + +/// The header file name for an API: `{library}.h`. +#[must_use] +pub fn header_name(model: &Model) -> String { + format!("{}.h", model.identity.library) +} + +/// The helper header's name and contents, when the configuration asks for it +/// and the API has value buffers to describe. +fn buffer_file(model: &Model, config: &CConfig) -> Option<(String, String)> { + if !config.buffer_helpers { + return None; + } + let name = buffer_header_name(&model.identity.library); + render_buffer_header(model, &header_name(model), &name).map(|contents| (name, contents)) +} + +/// The C header backend. +pub struct CGenerator; + +impl LanguageBackend for CGenerator { + type Config = CConfig; + + fn name(&self) -> &'static str { + "c" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let header = header_name(model); + let mut files = vec![OutputFile::new( + out_dir.join("c").join(&header), + render_c_header_from_model(model, &header), + )]; + if let Some((name, contents)) = buffer_file(model, config) { + files.push(OutputFile::new(out_dir.join("c").join(name), contents)); + } + files + } + + /// One `{library}-{version}-c.tar.gz` holding the headers under + /// `include/`, every platform's library under `lib//`, and a + /// `CMakeLists.txt` exposing the host's library as an imported target. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let header_name = header_name(model); + let lib = &ctx.binaries.lib_name; + let mut files = vec![ + PackagedFile::text( + format!("include/{header_name}"), + render_c_header_from_model(model, &header_name), + ), + PackagedFile::text("CMakeLists.txt", render_packaged_cmake(lib)), + PackagedFile::text("README.md", render_packaged_readme(lib, &header_name, ctx)), + ]; + if let Some((name, contents)) = buffer_file(model, config) { + files.push(PackagedFile::text(format!("include/{name}"), contents)); + } + files.extend(per_platform_libraries(ctx.binaries, "lib", |_| true)); + let stem = format!("{lib}-{}", model.identity.version); + Some(vec![Artifact::tar_gz( + format!("c/{stem}-c.tar.gz"), + stem, + files, + )]) + } +} diff --git a/crates/weaveffi-gen/src/targets/c/package.rs b/crates/weaveffi-cli/src/targets/c/package.rs similarity index 85% rename from crates/weaveffi-gen/src/targets/c/package.rs rename to crates/weaveffi-cli/src/targets/c/package.rs index 569b3a07..c423109a 100644 --- a/crates/weaveffi-gen/src/targets/c/package.rs +++ b/crates/weaveffi-cli/src/targets/c/package.rs @@ -8,8 +8,8 @@ use crate::utils::{render_prelude, render_trailer, CommentStyle}; /// Render a `CMakeLists.txt` that exposes the bundled per-platform library as /// an `IMPORTED` target (`::`) with the header's include directory, /// selecting the right library for the host platform and architecture. -pub(crate) fn render_packaged_cmake(lib: &str, input_basename: &str) -> String { - let prelude = render_prelude(CommentStyle::Hash, input_basename); +pub(crate) fn render_packaged_cmake(lib: &str) -> String { + let prelude = render_prelude(CommentStyle::Hash); let trailer = render_trailer(CommentStyle::Hash, "CMakeLists.txt"); let body = r#"cmake_minimum_required(VERSION 3.15) project(@LIB@ C) @@ -38,6 +38,10 @@ add_library(@LIB@ SHARED IMPORTED GLOBAL) set_target_properties(@LIB@ PROPERTIES IMPORTED_LOCATION "${CMAKE_CURRENT_LIST_DIR}/lib/${_@LIB@_platform}/${_@LIB@_libfile}" INTERFACE_INCLUDE_DIRECTORIES "${CMAKE_CURRENT_LIST_DIR}/include") +if(WIN32) + set_target_properties(@LIB@ PROPERTIES + IMPORTED_IMPLIB "${CMAKE_CURRENT_LIST_DIR}/lib/${_@LIB@_platform}/@LIB@.dll.lib") +endif() add_library(@LIB@::@LIB@ ALIAS @LIB@) "# .replace("@LIB@", lib); @@ -45,13 +49,8 @@ add_library(@LIB@::@LIB@ ALIAS @LIB@) } /// README for a packaged C artifact bundling the header and per-platform libs. -pub(crate) fn render_packaged_readme( - lib: &str, - header_name: &str, - ctx: &PackageContext, - input_basename: &str, -) -> String { - let prelude = render_prelude(CommentStyle::Xml, input_basename); +pub(crate) fn render_packaged_readme(lib: &str, header_name: &str, ctx: &PackageContext) -> String { + let prelude = render_prelude(CommentStyle::Xml); let trailer = render_trailer(CommentStyle::Xml, "README.md"); let platforms: Vec = ctx .binaries diff --git a/crates/weaveffi-gen/src/targets/c/runtime/buffer_runtime.h b/crates/weaveffi-cli/src/targets/c/runtime/buffer_runtime.h similarity index 100% rename from crates/weaveffi-gen/src/targets/c/runtime/buffer_runtime.h rename to crates/weaveffi-cli/src/targets/c/runtime/buffer_runtime.h diff --git a/crates/weaveffi-cli/src/targets/cpp/callbacks.rs b/crates/weaveffi-cli/src/targets/cpp/callbacks.rs new file mode 100644 index 00000000..90546ffe --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/callbacks.rs @@ -0,0 +1,329 @@ +//! Callback interfaces: the abstract class the consumer implements, the +//! trampolines that adapt an implementation to the C vtable, and the +//! process-wide static vtable, satisfying +//! [`weaveffi_model::plan::CallbackProtocol`]. + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ErrorBinding}; +use weaveffi_model::plan::{ArgPass, RetPass}; +use weaveffi_model::ty::Ty; + +use crate::targets::cpp::codec::{read_fn, write_stmt}; +use crate::targets::cpp::entities::{doc_text, report_fn}; +use crate::targets::cpp::types::{ + cpp_cb_param_decl, cpp_fn_name, cpp_ident, cpp_type, render_param_decls, slot_name, + trampoline_struct, vtable_accessor, +}; + +/// The code a trampoline reports for any failure other than a declared +/// domain code. +const FOREIGN_ERROR_CODE: i32 = -4; + +/// Append a callback interface: the abstract class, then (in `detail`) its +/// trampolines and static vtable. `domain` is the error domain in scope for +/// the interface's module, which its `throws` methods may report. +pub(crate) fn render_callback_interface( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + domain: Option<&ErrorBinding>, + prefix: &str, +) { + render_callback_class(w, cb, domain); + render_trampolines(w, cb, domain, prefix); +} + +/// Append the abstract class a consumer subclasses: a virtual destructor and +/// one pure virtual method per IDL method. Strings arrive as views valid for +/// the call, buffered values by const reference, and objects by value as +/// wrappers the implementation owns; every family can be returned. +fn render_callback_class( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + domain: Option<&ErrorBinding>, +) { + let name = &cb.name; + let throws = match domain.filter(|_| cb.methods.iter().any(|m| m.throws)) { + Some(eb) => format!( + " A method that declares errors may throw a `{}` code's exception, \ + which reaches the producer with its fields; any other exception \ + reaches it as a callback failure (code -4).", + eb.type_name + ), + None => " An exception a method throws reaches the producer as a callback \ + failure (code -4)." + .to_string(), + }; + let usage = format!( + "Subclass it and pass a `std::shared_ptr<{name}>` where the API takes a \ + `{name}`. The producer may call the methods from any thread, so they must \ + be thread-safe, until it releases the implementation (also from any \ + thread).{throws}" + ); + let doc = match doc_text(cb.doc.as_deref(), cb.deprecated.as_deref()) { + Some(d) => format!("{d}\n\n{usage}"), + None => usage, + }; + w.doc(&Some(doc), DocCommentStyle::Javadoc); + w.line(format!("class {name} {{")); + w.line("public:"); + w.scope(|w| { + w.line(format!("virtual ~{name}() = default;")); + for m in &cb.methods { + w.blank(); + let mut doc = doc_text(m.doc.as_deref(), m.deprecated.as_deref()); + if let Some(eb) = domain.filter(|_| m.throws) { + let tag = format!( + "@throws {} to report one of the domain's codes to the producer.", + eb.type_name + ); + doc = Some(match doc { + Some(d) => format!("{d}\n\n{tag}"), + None => tag, + }); + } + w.doc(&doc, DocCommentStyle::Javadoc); + let ret = m.ret.as_ref().map_or("void".to_string(), cpp_type); + let params: Vec = m + .params + .iter() + .map(|p| cpp_cb_param_decl(&p.ty, &cpp_ident(&p.name))) + .collect(); + w.line(format!( + "virtual {ret} {}({}) = 0;", + cpp_fn_name(&m.name), + params.join(", ") + )); + } + }); + w.line("};"); + w.blank(); +} + +/// Append, in `detail`, the trampolines and the static vtable of a callback +/// interface. +/// +/// `ctx` is a heap-allocated `std::shared_ptr` box made when the +/// implementation is passed to the producer; the vtable's `free` deletes +/// it. Each trampoline receives its arguments per the callback protocol, +/// calls the method, and hands the return back through the C return or the +/// `out_ptr`/`out_len` slots (a run from `{prefix}_alloc`). Any exception is +/// reported through `out_err` and nothing unwinds through the C frame. +fn render_trampolines( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + domain: Option<&ErrorBinding>, + prefix: &str, +) { + let name = &cb.name; + let strukt = trampoline_struct(name); + let vtable = &cb.vtable_tag; + w.line("namespace detail {"); + w.blank(); + w.line(format!( + "/** Trampolines adapting a `{name}` implementation to `{vtable}`. */" + )); + w.block(format!("struct {strukt} {{"), "};", |w| { + for m in &cb.methods { + render_trampoline(w, cb, m, domain.filter(|_| m.throws), prefix); + } + w.line("/** Deletes the implementation box once the producer releases it. */"); + w.block("static void free_ctx(void* ctx) {", "}", |w| { + w.line(format!( + "delete static_cast*>(ctx);" + )); + }); + }); + w.blank(); + w.line(format!( + "/** The vtable every `{name}` implementation is passed with. */" + )); + w.block( + format!("inline const {vtable}& {}() {{", vtable_accessor(name)), + "}", + |w| { + w.block(format!("static const {vtable} vtable = {{"), "};", |w| { + w.line(format!("static_cast(sizeof({vtable})),")); + w.line("0,"); + w.line(format!("&{strukt}::free_ctx,")); + for m in &cb.methods { + w.line(format!("&{strukt}::{},", cpp_ident(&m.name))); + } + }); + w.line("return vtable;"); + }, + ); + w.blank(); + w.line("} // namespace detail"); + w.blank(); +} + +/// Append one trampoline: a static function with the vtable entry's exact C +/// signature. `domain` is set when the method declares errors. +fn render_trampoline( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, + domain: Option<&ErrorBinding>, + prefix: &str, +) { + let iface = &cb.name; + let method = cpp_fn_name(&m.name); + let c_ret = m.abi_ret.render_c(prefix); + let slots = &m.abi_params; + let ctx = slot_name(&slots[0]); + let out_err = slot_name(&slots[slots.len() - 1]); + let ret_pass = RetPass::of(m.ret.as_ref()); + let returns_run = matches!(ret_pass, RetPass::String | RetPass::Bytes | RetPass::Buffer); + let (out_ptr, out_len) = if returns_run { + ( + slot_name(&slots[slots.len() - 3]), + slot_name(&slots[slots.len() - 2]), + ) + } else { + (String::new(), String::new()) + }; + let params = render_param_decls(slots, prefix).join(", "); + + w.block( + format!("static {c_ret} {}({params}) {{", cpp_ident(&m.name)), + "}", + |w| { + // Object arguments each transfer one strong reference. Adopting + // can't throw, so it comes first, which releases every reference + // even when another argument fails to decode. + for p in &m.params { + if let ArgPass::Object { slot, nullable } = p.arg_pass() { + let class = p.ty.interface_name().expect("objects name an interface"); + let slot = slot_name(slot); + let var = format!("{}_arg", p.name); + if nullable { + w.line(format!("std::optional<{class}> {var};")); + w.line(format!("if ({slot} != nullptr) {var}.emplace(adopt, {slot});")); + } else { + w.line(format!("{class} {var}(adopt, {slot});")); + } + } + } + w.line("try {"); + w.scope(|w| { + w.line(format!( + "{iface}& impl = **static_cast*>({ctx});" + )); + // Buffers carrying object tokens decode first, so a failure + // decoding another argument still adopts their references. + let mut order: Vec<_> = m.params.iter().collect(); + order.sort_by_key(|p| !p.ty.contains_object()); + for p in order { + if let ArgPass::Buffer { ptr, len } = p.arg_pass() { + w.line(format!( + "{} {}_arg = detail::decode({}, {}, &{});", + cpp_type(&p.ty), + p.name, + slot_name(ptr), + slot_name(len), + read_fn(&p.ty) + )); + } + } + let args: Vec = m.params.iter().map(trampoline_arg).collect(); + let call = format!("impl.{method}({})", args.join(", ")); + match (&m.ret, ret_pass.clone()) { + (None, _) => { + w.line(format!("{call};")); + } + (Some(Ty::Enum(_)), _) => { + w.line(format!( + "return static_cast<{c_ret}>(static_cast({call}));" + )); + } + (Some(_), RetPass::Direct) => { + w.line(format!("return {call};")); + } + (Some(_), RetPass::Object { nullable: false, .. }) => { + // A moved-from wrapper returns null, which the + // producer rejects (-3). + w.line(format!("return {call}.clone_handle();")); + } + (Some(ty), RetPass::Object { nullable: true, .. }) => { + w.line(format!("{} ret = {call};", cpp_type(ty))); + w.line("return ret.has_value() ? ret->clone_handle() : nullptr;"); + } + (Some(ty), RetPass::String | RetPass::Bytes) => { + w.line(format!("{} ret = {call};", cpp_type(ty))); + w.line(format!( + "detail::hand_over(ret.data(), ret.size(), {out_ptr}, {out_len});" + )); + } + (Some(ty), RetPass::Buffer) => { + w.line(format!("{} ret = {call};", cpp_type(ty))); + w.line("detail::BufferWriter ret_buf;"); + w.line(write_stmt(ty, "ret", "ret_buf")); + w.line(format!( + "detail::hand_over(ret_buf.data(), ret_buf.size(), {out_ptr}, {out_len});" + )); + } + (Some(_), RetPass::Void) => unreachable!("a return type is never void"), + } + }); + if let Some(eb) = domain { + w.line(format!("}} catch (const {}& e) {{", eb.type_name)); + w.scope(|w| { + w.line(format!("{}(e, {out_err});", report_fn(eb))); + }); + } + w.line("} catch (const std::exception& e) {"); + w.scope(|w| { + w.line(format!( + "{prefix}_error_set({out_err}, {FOREIGN_ERROR_CODE}, e.what());" + )); + }); + w.line("} catch (...) {"); + w.scope(|w| { + w.line(format!( + "{prefix}_error_set({out_err}, {FOREIGN_ERROR_CODE}, \"{iface}::{method} threw a non-standard exception\");" + )); + }); + w.line("}"); + match ret_pass { + RetPass::Direct => { + w.line(format!("return {c_ret}{{}};")); + } + RetPass::Object { .. } => { + w.line("return nullptr;"); + } + _ => {} + } + }, + ); + w.blank(); +} + +/// The expression handing one argument to the implementation: a string +/// viewed in place, bytes copied, a decoded buffer or adopted object moved +/// in, and a direct value converted. +fn trampoline_arg(p: &weaveffi_model::model::ParamBinding) -> String { + match p.arg_pass() { + ArgPass::Buffer { .. } | ArgPass::Object { .. } => format!("std::move({}_arg)", p.name), + ArgPass::String { ptr, len } => format!( + "detail::borrow_string({}, {})", + slot_name(ptr), + slot_name(len) + ), + ArgPass::Bytes { ptr, len } => format!( + "detail::borrow_bytes({}, {})", + slot_name(ptr), + slot_name(len) + ), + ArgPass::Direct { slot } => match &p.ty { + Ty::Enum(e) => format!( + "static_cast<{e}>(static_cast({}))", + slot_name(slot) + ), + _ => slot_name(slot), + }, + ArgPass::Callback { .. } => { + unreachable!("callback interfaces are never callback-method parameters") + } + } +} diff --git a/crates/weaveffi-cli/src/targets/cpp/calls.rs b/crates/weaveffi-cli/src/targets/cpp/calls.rs new file mode 100644 index 00000000..6a8110aa --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/calls.rs @@ -0,0 +1,730 @@ +//! Callable rendering: free functions and interface members in the sync, +//! iterator, and async call shapes, each marshalled per the shared passing +//! plans ([`ArgPass`], [`RetPass`], `IteratorProtocol`, `AsyncProtocol`). +//! +//! Interface members are rendered twice: a declaration inside the class body +//! (so the class is complete before any record that holds it by value) and an +//! out-of-line `inline` definition after every value type and codec exists. +//! Free functions are defined inline inside their module namespace, which is +//! rendered last. + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use heck::ToUpperCamelCase; +use weaveffi_model::model::{ + AbiFn, AsyncBinding, CallShape, ErrorBinding, FnBinding, IteratorBinding, ModuleBinding, + ParamBinding, +}; +use weaveffi_model::plan::{ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::ty::Ty; + +use crate::targets::cpp::codec::{read_fn, write_stmt}; +use crate::targets::cpp::entities::{check_fn, make_error_fn}; +use crate::targets::cpp::types::{ + cpp_fn_name, cpp_ident, cpp_namespace_path, cpp_param_decl, cpp_type, render_param_decls, + slot_name, vtable_accessor, +}; + +// ── Error routing ── + +/// The domain a callable reports typed errors from: the one in scope when +/// it's declared `throws`, else `None` (its failures are bugs). +fn throwing_domain<'a>(f: &FnBinding, error: Option<&'a ErrorBinding>) -> Option<&'a ErrorBinding> { + match f.error_strategy() { + ErrorStrategy::Throws => error, + ErrorStrategy::Trap => None, + } +} + +/// The `detail::check*` helper a wrapper calls after the C call returns: the +/// domain's (throwing its typed exceptions) for a throwing callable, the +/// `InternalError` check otherwise. +fn check_helper(f: &FnBinding, error: Option<&ErrorBinding>) -> String { + throwing_domain(f, error).map_or_else(|| "detail::check".to_string(), check_fn) +} + +/// The expression an async completion uses to turn its boxed `err` into the +/// `std::exception_ptr` it settles the promise with. +fn make_error_call(f: &FnBinding, error: Option<&ErrorBinding>) -> String { + match throwing_domain(f, error) { + Some(eb) => format!( + "{}(err->code, message, err->payload_ptr, err->payload_len)", + make_error_fn(eb) + ), + None => "detail::make_internal_error(err->code, message)".to_string(), + } +} + +// ── Parameter and return marshalling ── + +/// The wrapper class behind an object-passed type: the interface itself, or +/// the interface inside `Interface?`. +fn object_class(ty: &Ty) -> &str { + ty.interface_name() + .expect("object passing only applies to interfaces") +} + +/// Append the setup statements for one parameter and return the C argument +/// expressions its ABI slots receive, dispatching on its [`ArgPass`]. +/// +/// * A string passes its UTF-8 pointer and length (no terminator, so +/// interior NULs survive); bytes likewise. +/// * A buffered value is encoded into a local `detail::BufferWriter` that +/// outlives the call. +/// * An object is borrowed: the wrapper's pointer is passed and the wrapper +/// keeps its reference. `Interface?` passes null for none. +/// * A callback interface moves the caller's `std::shared_ptr` into a heap +/// box that becomes `ctx`; the producer deletes it through the vtable's +/// `free`. A `std::unique_ptr` owns the box until the argument list +/// releases it, so a throw while marshalling a later parameter frees it. +/// An optional callback interface passes a null vtable for an empty +/// pointer. +fn emit_param_setup(w: &mut CodeWriter, p: &ParamBinding, prefix: &str) -> Vec { + let name = cpp_ident(&p.name); + match p.arg_pass() { + ArgPass::Buffer { .. } => { + let buf = format!("{name}_buf"); + w.line(format!("detail::BufferWriter {buf};")); + w.line(write_stmt(&p.ty, &name, &buf)); + vec![format!("{buf}.data()"), format!("{buf}.size()")] + } + ArgPass::String { .. } => vec![format!("detail::utf8({name})"), format!("{name}.size()")], + ArgPass::Bytes { .. } => vec![format!("{name}.data()"), format!("{name}.size()")], + ArgPass::Object { nullable: true, .. } => { + vec![format!("{name}.has_value() ? {name}->handle() : nullptr")] + } + ArgPass::Object { + nullable: false, .. + } => vec![format!("{name}.handle()")], + ArgPass::Callback { nullable, .. } => { + let iface = + p.ty.callback_interface_name() + .expect("callback passing only applies to callback interfaces"); + let vtable = format!("&detail::{}()", vtable_accessor(iface)); + let ctx = format!("{name}_ctx"); + if nullable { + w.line(format!( + "const auto* {name}_vtable = {name} ? {vtable} : nullptr;" + )); + w.line(format!( + "std::unique_ptr> {ctx}({name} ? new std::shared_ptr<{iface}>(std::move({name})) : nullptr);" + )); + vec![ + format!("static_cast({ctx}.release())"), + format!("{name}_vtable"), + ] + } else { + w.line(format!( + "if (!{name}) throw std::invalid_argument(\"{name}: null callback interface\");" + )); + w.line(format!( + "auto {ctx} = std::make_unique>(std::move({name}));" + )); + vec![format!("static_cast({ctx}.release())"), vtable] + } + } + ArgPass::Direct { slot } => match &p.ty { + Ty::Enum(_) => vec![format!( + "static_cast<{}>(static_cast({name}))", + slot.ty.render_c(prefix) + )], + _ => vec![name], + }, + } +} + +/// The C++ value of a returned element held in C slots `value` (and `len` +/// for strings, bytes, and buffers), per its [`RetPass`]: strings, bytes, and +/// buffers are copied or decoded and then released, an object is adopted, +/// and a direct value converts. `Void` has no value. +fn received_value(ty: &Ty, value: &str, len: &str) -> String { + match RetPass::of(Some(ty)) { + RetPass::Buffer => format!("detail::take({value}, {len}, &{})", read_fn(ty)), + RetPass::String => format!("detail::take_string({value}, {len})"), + RetPass::Bytes => format!("detail::take_bytes({value}, {len})"), + RetPass::Object { + nullable: false, .. + } => { + format!("{}(adopt, {value})", object_class(ty)) + } + RetPass::Object { nullable: true, .. } => { + let class = object_class(ty); + format!( + "{value} != nullptr ? std::optional<{class}>(std::in_place, adopt, {value}) : std::nullopt" + ) + } + RetPass::Direct => match ty { + Ty::Enum(n) => format!("static_cast<{n}>({value})"), + _ => value.to_string(), + }, + RetPass::Void => unreachable!("void returns carry no value"), + } +} + +// ── Callable kinds ── + +/// How a rendered callable is declared in the C++ surface. +#[derive(Clone, Copy)] +pub(crate) enum FnKind<'a> { + /// A namespace-scope free function. + Free, + /// An instance method: passes the wrapped pointer as the leading C + /// argument and is `const` (the ABI receiver is a const pointer). + Method { + /// The wrapper class name. + class: &'a str, + }, + /// A static member: interface statics and the factory form of + /// constructors not named `new`. + Static { + /// The wrapper class name. + class: &'a str, + }, + /// The interface constructor named `new`, rendered as a C++ constructor. + Ctor { + /// The wrapper class name. + class: &'a str, + }, +} + +impl<'a> FnKind<'a> { + /// The owning class, for member kinds. + fn class(self) -> Option<&'a str> { + match self { + FnKind::Free => None, + FnKind::Method { class } | FnKind::Static { class } | FnKind::Ctor { class } => { + Some(class) + } + } + } +} + +/// The name of the range class an iterator-returning callable yields: +/// `{PascalName}Iterator` for a free function and +/// `{Class}{PascalName}Iterator` for an interface member. +pub(crate) fn iterator_class_name(f: &FnBinding, kind: FnKind<'_>) -> String { + let pascal = f.name.to_upper_camel_case(); + match kind.class() { + Some(class) => format!("{class}{pascal}Iterator"), + None => format!("{pascal}Iterator"), + } +} + +/// The C++ return type of a callable: the mapped type (or `void`) for sync, +/// `std::future` for async, and the range class for an iterator. `None` +/// for a constructor. +fn return_type(f: &FnBinding, kind: FnKind<'_>) -> Option { + if matches!(kind, FnKind::Ctor { .. }) { + return None; + } + let value = || f.ret.as_ref().map_or("void".to_string(), cpp_type); + Some(match &f.shape { + CallShape::Sync(_) => value(), + CallShape::Async(_) => format!("std::future<{}>", value()), + CallShape::Iterator(_) => iterator_class_name(f, kind), + }) +} + +/// The C++ parameter list of a callable. A cancellable async call takes a +/// trailing `const CancelToken&`; `with_defaults` adds its +/// `= CancelToken::none()` default, which belongs on the first declaration +/// only. +fn param_decls(f: &FnBinding, with_defaults: bool) -> String { + let mut decls: Vec = f + .params + .iter() + .map(|p| cpp_param_decl(&p.ty, &cpp_ident(&p.name))) + .collect(); + if f.is_async() && f.cancellable { + let default = if with_defaults { + " = CancelToken::none()" + } else { + "" + }; + decls.push(format!("const CancelToken& cancel_token{default}")); + } + decls.join(", ") +} + +/// Append the doc comment and any `[[deprecated]]` attribute of a callable: +/// its IDL doc, `@param` for each documented parameter, what an iterator or +/// async call returns, and the exceptions it throws. +fn emit_callable_attrs( + w: &mut CodeWriter, + f: &FnBinding, + kind: FnKind<'_>, + error: Option<&ErrorBinding>, +) { + let mut sections: Vec = f.doc.iter().cloned().collect(); + let mut tags: Vec = f + .params + .iter() + .filter_map(|p| { + p.doc + .as_ref() + .map(|d| format!("@param {} {d}", cpp_ident(&p.name))) + }) + .collect(); + if f.is_async() && f.cancellable { + tags.push("@param cancel_token Cancels the call; the future then throws Cancelled.".into()); + } + match &f.shape { + CallShape::Iterator(_) => tags.push(format!( + "@return A lazy `{}` range that pulls one element per step and releases the \ + producer iterator when exhausted or destroyed.", + iterator_class_name(f, kind) + )), + CallShape::Async(_) => tags.push( + "@return A future settled from a producer thread; get() rethrows the call's error." + .into(), + ), + CallShape::Sync(_) => {} + } + if let Some(eb) = throwing_domain(f, error) { + tags.push(format!( + "@throws {} when the call fails with one of the domain's codes.", + eb.type_name + )); + } + if !tags.is_empty() { + sections.push(tags.join("\n")); + } + let doc = (!sections.is_empty()).then(|| sections.join("\n\n")); + w.doc(&doc, DocCommentStyle::Javadoc); + if let Some(msg) = &f.deprecated { + w.line(format!("[[deprecated(\"{}\")]]", msg.replace('"', "\\\""))); + } +} + +/// Append the in-class declaration of an interface member, with its doc +/// comment and deprecation marker. +pub(crate) fn render_member_decl( + w: &mut CodeWriter, + f: &FnBinding, + cpp_name: &str, + kind: FnKind<'_>, + error: Option<&ErrorBinding>, +) { + emit_callable_attrs(w, f, kind, error); + let params = param_decls(f, true); + match (kind, return_type(f, kind)) { + // Explicit, so no argument list converts into a new producer object + // behind the caller's back. + (FnKind::Ctor { class }, _) if f.params.is_empty() => w.line(format!("{class}();")), + (FnKind::Ctor { class }, _) => w.line(format!("explicit {class}({params});")), + (FnKind::Method { .. }, Some(ret)) => w.line(format!("{ret} {cpp_name}({params}) const;")), + (FnKind::Static { .. }, Some(ret)) => w.line(format!("static {ret} {cpp_name}({params});")), + _ => unreachable!("free functions are defined inline, never declared"), + }; + w.blank(); +} + +/// Append the definition of a callable: an `inline` free function inside its +/// module namespace (with its doc comment), or the out-of-line `inline` +/// definition of an interface member declared by [`render_member_decl`]. +/// +/// Wrappers are never `noexcept`: a call that declares no errors still +/// surfaces a producer bug as `InternalError`. +/// +/// Every free function, constructor, and static member begins with +/// `check_library()`, which verifies the linked library once. Instance +/// methods skip it: their receiver could only come from a checked call. +pub(crate) fn render_definition( + w: &mut CodeWriter, + f: &FnBinding, + cpp_name: &str, + kind: FnKind<'_>, + error: Option<&ErrorBinding>, + prefix: &str, +) { + if matches!(kind, FnKind::Free) { + emit_callable_attrs(w, f, kind, error); + } + let params = param_decls(f, matches!(kind, FnKind::Free)); + let header = match (kind, return_type(f, kind)) { + // `raw_` starts null so a throw from the error check leaves + // nothing for the destructor to release. + (FnKind::Ctor { class }, _) => { + format!("inline {class}::{class}({params}) : raw_(nullptr) {{") + } + (FnKind::Method { class }, Some(ret)) => { + format!("inline {ret} {class}::{cpp_name}({params}) const {{") + } + (FnKind::Static { class }, Some(ret)) => { + format!("inline {ret} {class}::{cpp_name}({params}) {{") + } + (FnKind::Free, Some(ret)) => format!("inline {ret} {cpp_name}({params}) {{"), + _ => unreachable!("every non-constructor callable has a return type"), + }; + w.block(header, "}", |w| { + if !matches!(kind, FnKind::Method { .. }) { + w.line("check_library();"); + } + let mut c_args = Vec::new(); + if matches!(kind, FnKind::Method { .. }) { + c_args.push("raw_".to_string()); + } + for p in &f.params { + c_args.extend(emit_param_setup(w, p, prefix)); + } + match &f.shape { + CallShape::Sync(abi) => emit_sync_body(w, f, abi, kind, c_args, error, prefix), + CallShape::Iterator(it) => { + emit_iterator_launch_body(w, f, it, kind, c_args, error, prefix); + } + CallShape::Async(a) => emit_async_body(w, f, a, c_args, error, prefix), + } + }); + w.blank(); +} + +/// Append a synchronous callable's call, error check, and return. A +/// constructor stores the returned reference in `raw_`. +fn emit_sync_body( + w: &mut CodeWriter, + f: &FnBinding, + abi: &AbiFn, + kind: FnKind<'_>, + mut c_args: Vec, + error: Option<&ErrorBinding>, + prefix: &str, +) { + // A string, bytes, or buffered return carries a trailing `size_t* out_len`. + let has_out_len = matches!( + RetPass::of(f.ret.as_ref()), + RetPass::Buffer | RetPass::String | RetPass::Bytes + ); + if has_out_len { + w.line("size_t out_len = 0;"); + c_args.push("&out_len".into()); + } + c_args.push("&err".into()); + w.line(format!("{prefix}_error err{{}};")); + let call = format!("{}({})", abi.symbol, c_args.join(", ")); + if f.ret.is_none() { + w.line(format!("{call};")); + } else { + w.line(format!("auto result = {call};")); + } + w.line(format!("{}(err);", check_helper(f, error))); + match (kind, &f.ret) { + (FnKind::Ctor { .. }, _) => { + w.line("raw_ = result;"); + } + (_, Some(ret)) => { + w.line(format!( + "return {};", + received_value(ret, "result", "out_len") + )); + } + (_, None) => {} + } +} + +// ── Iterators ── + +/// Append the lazy range class an iterator-returning callable yields. +/// +/// The range is move-only, owns the producer iterator, and pulls exactly one +/// element per step (`weaveffi_model::plan::IteratorProtocol`): +/// +/// * `begin()`/`end()` expose a single-pass input iterator with a sentinel +/// end, so `for (auto&& item : fn())` streams in constant memory. +/// * Each element is received per its family (strings, bytes, and buffers +/// copied or decoded then released; objects adopted). +/// * `_destroy` runs exactly once: eagerly on exhaustion or a `next` error, +/// from the destructor otherwise. +/// * Errors follow the callable's [`ErrorStrategy`]. +pub(crate) fn render_iterator_range( + w: &mut CodeWriter, + f: &FnBinding, + it: &IteratorBinding, + cpp_name: &str, + kind: FnKind<'_>, + error: Option<&ErrorBinding>, + prefix: &str, +) { + let elem = cpp_type(&it.elem); + let class = iterator_class_name(f, kind); + let tag = &it.iter_tag; + let destroy = &it.destroy_symbol; + let owner = match kind.class() { + Some(c) => format!("{c}::{cpp_name}()"), + None => format!("{cpp_name}()"), + }; + + w.doc( + &Some(format!( + "A lazy, move-only range over the `{elem}` elements `{owner}` produces.\n\n\ + Each step pulls one element from the producer, so results stream in \ + constant memory. The range releases the producer iterator exactly once: \ + when it's exhausted, or from the destructor when iteration stops early." + )), + DocCommentStyle::Javadoc, + ); + w.line(format!("class {class} {{")); + w.scope(|w| { + w.line(format!("{tag}* handle_;")); + w.blank(); + }); + w.line("public:"); + w.scope(|w| { + w.line("/** Adopts a producer iterator. */"); + w.line(format!( + "explicit {class}(adopt_t, {tag}* h) noexcept : handle_(h) {{}}" + )); + w.blank(); + w.line("/** Releases the producer iterator if it's still held. */"); + w.block(format!("~{class}() {{"), "}", |w| { + w.line(format!("if (handle_ != nullptr) {destroy}(handle_);")); + }); + w.blank(); + w.line(format!("{class}(const {class}&) = delete;")); + w.line(format!("{class}& operator=(const {class}&) = delete;")); + w.blank(); + w.line("/** Transfers `other`'s iterator; `other` becomes empty. */"); + w.line(format!( + "{class}({class}&& other) noexcept : handle_(other.handle_) {{ other.handle_ = nullptr; }}" + )); + w.blank(); + w.line("/** Releases the current iterator and takes over `other`'s. */"); + w.block( + format!("{class}& operator=({class}&& other) noexcept {{"), + "}", + |w| { + w.block("if (this != &other) {", "}", |w| { + w.line(format!("if (handle_ != nullptr) {destroy}(handle_);")); + w.line("handle_ = other.handle_;"); + w.line("other.handle_ = nullptr;"); + }); + w.line("return *this;"); + }, + ); + w.blank(); + render_iterator_next(w, f, it, error, prefix); + w.line("/** Sentinel type marking the end of the range. */"); + w.line("struct sentinel {};"); + w.blank(); + w.line("/** Single-pass input iterator; each increment pulls one element. */"); + w.line("class iterator {"); + w.scope(|w| { + w.line(format!("{class}* range_;")); + w.line(format!("std::optional<{elem}> current_;")); + w.blank(); + }); + w.line("public:"); + w.scope(|w| { + w.line("using iterator_category = std::input_iterator_tag;"); + w.line(format!("using value_type = {elem};")); + w.line("using difference_type = std::ptrdiff_t;"); + w.line(format!("using pointer = {elem}*;")); + w.line(format!("using reference = {elem}&;")); + w.blank(); + w.line("/** Binds to `range` and pulls the first element. */"); + w.line(format!( + "explicit iterator({class}* range) : range_(range), current_(range->next()) {{}}" + )); + w.line("reference operator*() { return *current_; }"); + w.line("pointer operator->() { return &*current_; }"); + w.line("iterator& operator++() { current_ = range_->next(); return *this; }"); + w.line("void operator++(int) { current_ = range_->next(); }"); + w.line("bool operator==(sentinel) const { return !current_.has_value(); }"); + w.line("bool operator!=(sentinel) const { return current_.has_value(); }"); + }); + w.line("};"); + w.blank(); + w.line("/** Begins iteration by pulling the first element. */"); + w.line("iterator begin() { return iterator(this); }"); + w.blank(); + w.line("/** The past-the-end sentinel. */"); + w.line("sentinel end() const { return sentinel{}; }"); + }); + w.line("};"); + w.blank(); +} + +/// Append an iterator-returning callable's launch: call the launcher, check +/// `out_err`, and wrap the returned pointer in the range class. +fn emit_iterator_launch_body( + w: &mut CodeWriter, + f: &FnBinding, + it: &IteratorBinding, + kind: FnKind<'_>, + mut c_args: Vec, + error: Option<&ErrorBinding>, + prefix: &str, +) { + c_args.push("&err".into()); + w.line(format!("{prefix}_error err{{}};")); + w.line(format!( + "{}* iter = {}({});", + it.iter_tag, + it.launch.symbol, + c_args.join(", ") + )); + w.line(format!("{}(err);", check_helper(f, error))); + w.line(format!( + "return {}(adopt, iter);", + iterator_class_name(f, kind) + )); +} + +/// Append the range class's `next()`: one producer `_next` call yielding the +/// received element, or `std::nullopt` on exhaustion, destroying the +/// iterator exactly once on exhaustion or error. +fn render_iterator_next( + w: &mut CodeWriter, + f: &FnBinding, + it: &IteratorBinding, + error: Option<&ErrorBinding>, + prefix: &str, +) { + let elem = cpp_type(&it.elem); + let destroy = &it.destroy_symbol; + let item_ty = it.item_ctype().render_c(prefix); + let raises = match throwing_domain(f, error) { + Some(eb) => eb.type_name.clone(), + None => "InternalError".to_string(), + }; + w.doc( + &Some(format!( + "Pulls the next element, or `std::nullopt` once exhausted (which releases \ + the producer iterator). A producer error releases the iterator and throws \ + {raises}." + )), + DocCommentStyle::Javadoc, + ); + w.block(format!("std::optional<{elem}> next() {{"), "}", |w| { + w.line("if (handle_ == nullptr) return std::nullopt;"); + w.line(format!("{prefix}_error err{{}};")); + w.line(format!("{item_ty} item{{}};")); + let mut args = vec!["handle_".to_string(), "&item".to_string()]; + if matches!( + RetPass::of(Some(&it.elem)), + RetPass::Buffer | RetPass::String | RetPass::Bytes + ) { + w.line("size_t item_len = 0;"); + args.push("&item_len".to_string()); + } + args.push("&err".to_string()); + w.line(format!( + "int32_t has_item = {}({});", + it.next.symbol, + args.join(", ") + )); + w.block("if (err.code != 0 || has_item == 0) {", "}", |w| { + w.line(format!("{destroy}(handle_);")); + w.line("handle_ = nullptr;"); + w.line(format!("{}(err);", check_helper(f, error))); + w.line("return std::nullopt;"); + }); + // A nullable object element yields an engaged outer optional holding + // an empty inner one for null, which is distinct from exhaustion. + w.line(format!( + "return std::optional<{elem}>(std::in_place, {});", + received_value(&it.elem, "item", "item_len") + )); + }); + w.blank(); +} + +// ── Async ── + +/// Append an asynchronous callable's body: a `std::future` wrapper. +/// +/// The parameters are marshalled first, then a heap-allocated promise +/// travels through the C `context` and the completion adopts it back into a +/// `std::unique_ptr`, settling it exactly once. A failure settles the +/// promise with the typed domain exception (a throwing call), `Cancelled` +/// for -5, or `InternalError`. The completion owns everything it receives: +/// it releases the boxed error and any string or buffer result after copying +/// and adopts an object result. A cancellable call passes its token's native +/// handle (null for `CancelToken::none()`); the producer takes its own +/// reference. +fn emit_async_body( + w: &mut CodeWriter, + f: &FnBinding, + a: &AsyncBinding, + mut c_args: Vec, + error: Option<&ErrorBinding>, + prefix: &str, +) { + let value = f.ret.as_ref().map_or("void".to_string(), cpp_type); + let promise = format!("std::promise<{value}>"); + let cb_params = render_param_decls(&a.callback_params, prefix).join(", "); + if f.cancellable { + c_args.push("cancel_token.handle()".to_string()); + } + w.line(format!("auto promise = std::make_unique<{promise}>();")); + w.line("auto future = promise->get_future();"); + c_args.push(format!("[]({cb_params}) {{")); + w.line(format!("{}({}", a.launch.symbol, c_args.join(", "))); + w.scope(|w| { + w.line(format!( + "std::unique_ptr<{promise}> p(static_cast<{promise}*>(context));" + )); + // The completion runs on a producer thread inside a C frame, so a + // decode failure (of the result or an error payload) settles the + // promise instead of unwinding. + w.line("try {"); + w.scope(|w| { + w.line("if (err != nullptr && err->code != 0) {"); + w.scope(|w| { + w.line("std::string message = detail::error_message(*err);"); + w.line(format!("p->set_exception({});", make_error_call(f, error))); + }); + w.line("} else {"); + w.scope(|w| match &f.ret { + Some(ret) => { + let slots = &a.callback_params[2..]; + let slot = |i: usize| slots.get(i).map(slot_name).unwrap_or_default(); + w.line(format!( + "p->set_value({});", + received_value(ret, &slot(0), &slot(1)) + )); + } + None => { + w.line("p->set_value();"); + } + }); + w.line("}"); + }); + w.line("} catch (...) {"); + w.scope(|w| { + w.line("p->set_exception(std::current_exception());"); + }); + w.line("}"); + w.line(format!("{prefix}_error_free(err);")); + }); + w.line("}, static_cast(promise.release()));"); + w.line("return future;"); +} + +// ── Module namespaces ── + +/// Append one module's nested namespace holding its free functions +/// (`namespace kv::stats { ... }`). An iterator-returning function is +/// preceded by its range class. A module without functions emits nothing; +/// its types live at the namespace root. +pub(crate) fn render_cpp_module_ns( + w: &mut CodeWriter, + module: &ModuleBinding, + error: Option<&ErrorBinding>, + prefix: &str, +) { + if module.functions.is_empty() { + return; + } + let ns = cpp_namespace_path(module); + w.line(format!("namespace {ns} {{")); + w.blank(); + for f in &module.functions { + let name = cpp_fn_name(&f.name); + if let CallShape::Iterator(it) = &f.shape { + render_iterator_range(w, f, it, &name, FnKind::Free, error, prefix); + } + render_definition(w, f, &name, FnKind::Free, error, prefix); + } + w.line(format!("}} // namespace {ns}")); + w.blank(); +} diff --git a/crates/weaveffi-cli/src/targets/cpp/codec.rs b/crates/weaveffi-cli/src/targets/cpp/codec.rs new file mode 100644 index 00000000..3dab0268 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/codec.rs @@ -0,0 +1,366 @@ +//! Value-buffer codecs: one writer and one reader function per record, rich +//! enum, and distinct composite type (`[Entry]`, `{string:i64}`, `Store?`), +//! plus the read expressions and write statements that call them. +//! +//! A primitive, C-style enum, or object token is read or written inline (one +//! `BufferReader`/`BufferWriter` call); everything else delegates to its +//! named function in `detail`, so no loop is ever inlined at a call site. +//! Every dispatch goes through [`Ty::wire`], and primitives dispatch on +//! [`Prim::snake`](weaveffi_model::ty::Prim::snake) (`write_i32`, +//! `read_string`). + +use std::collections::HashSet; + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{EnumBinding, Model, StructBinding}; +use weaveffi_model::ty::{Ty, WireType}; + +use crate::targets::cpp::types::{cpp_ident, cpp_type}; + +/// The identifier fragment naming `ty` in its codec functions: a +/// primitive's spelling (`i64`), a user type's name (`Entry`), or the +/// composite's shape (`opt_i64`, `list_string`, `map_string_Store`). +fn mangle(ty: &Ty) -> String { + match ty { + Ty::Optional(inner) => format!("opt_{}", mangle(inner)), + Ty::List(inner) => format!("list_{}", mangle(inner)), + Ty::Map(k, v) => format!("map_{}_{}", mangle(k), mangle(v)), + Ty::Prim(p) => p.snake().to_string(), + _ => ty + .user_name() + .expect("only optionals, lists, maps, primitives, and user types are mangled") + .to_string(), + } +} + +/// `detail::write_{stem}`: the writer of a record, rich enum, or composite. +pub(crate) fn write_fn(ty: &Ty) -> String { + format!("detail::write_{}", mangle(ty)) +} + +/// `detail::read_{stem}`: the reader of a record, rich enum, or composite. +pub(crate) fn read_fn(ty: &Ty) -> String { + format!("detail::read_{}", mangle(ty)) +} + +/// The expression reading one `ty` value from the reader `r`. An object +/// token is adopted into a new wrapper, whose destructor releases it if a +/// later read fails. +pub(crate) fn read_expr(ty: &Ty, r: &str) -> String { + match ty.wire() { + WireType::Prim(p) => format!("{r}.read_{}()", p.snake()), + WireType::Enum(n) => format!("static_cast<{n}>({r}.read_i32())"), + WireType::Object(n) => format!("{r}.read_object<{n}>()"), + WireType::User(_) | WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{}({r})", read_fn(ty)) + } + } +} + +/// The statement writing `expr` (one `ty` value) into the writer `w`. An +/// object is written as a fresh reference from its `_clone`. +pub(crate) fn write_stmt(ty: &Ty, expr: &str, w: &str) -> String { + match ty.wire() { + WireType::Prim(p) => format!("{w}.write_{}({expr});", p.snake()), + WireType::Enum(_) => format!("{w}.write_i32(static_cast({expr}));"), + WireType::Object(_) => format!("{w}.write_object({expr});"), + WireType::User(_) | WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{}({w}, {expr});", write_fn(ty)) + } + } +} + +/// Every value type and distinct composite type the API's value buffers +/// carry, records and rich enums first, then composites in first-use order: +/// inside records, rich-enum variants, and error payloads, and at every +/// buffered call boundary (parameters, returns, iterator elements, and +/// callback parameters and returns). +fn codec_types(model: &Model) -> Vec { + let mut out = Vec::new(); + for m in &model.modules { + out.extend(m.structs.iter().map(|s| Ty::Record(s.name.clone()))); + out.extend( + m.enums + .iter() + .filter(|e| e.is_rich()) + .map(|e| Ty::RichEnum(e.name.clone())), + ); + } + let mut seen = HashSet::new(); + for m in &model.modules { + let fields = m + .structs + .iter() + .flat_map(|s| &s.fields) + .chain( + m.enums + .iter() + .flat_map(|e| &e.variants) + .flat_map(|v| &v.fields), + ) + .chain( + m.errors + .iter() + .flat_map(|e| &e.codes) + .flat_map(|c| &c.fields), + ); + for f in fields { + collect(&f.ty, &mut out, &mut seen); + } + } + let mut boundary = |ty: &Ty| { + let ty = ty.iterator_elem().unwrap_or(ty); + if ty.is_buffered() { + collect(ty, &mut out, &mut seen); + } + }; + for m in &model.modules { + for f in m.callables() { + f.params.iter().for_each(|p| boundary(&p.ty)); + f.ret.iter().for_each(&mut boundary); + } + for cb in &m.callback_interfaces { + for meth in &cb.methods { + meth.params.iter().for_each(|p| boundary(&p.ty)); + meth.ret.iter().for_each(&mut boundary); + } + } + } + out +} + +/// Record every optional, list, and map shape inside `ty`, innermost first. +fn collect(ty: &Ty, out: &mut Vec, seen: &mut HashSet) { + match ty { + Ty::Optional(inner) | Ty::List(inner) => collect(inner, out, seen), + Ty::Map(k, v) => { + collect(k, out, seen); + collect(v, out, seen); + } + _ => return, + } + if seen.insert(ty.clone()) { + out.push(ty.clone()); + } +} + +/// Append, inside `detail`, every codec the API needs: a declaration of each +/// first (so a codec may call any other regardless of nesting), then the +/// definitions. Requires every value type to be complete. +pub(crate) fn render_codecs(w: &mut CodeWriter, model: &Model) { + let types = codec_types(model); + if types.is_empty() { + return; + } + w.line("namespace detail {"); + w.blank(); + for ty in &types { + let cpp = cpp_type(ty); + let stem = mangle(ty); + w.line(format!( + "inline void write_{stem}(BufferWriter& w, const {cpp}& v);" + )); + w.line(format!("inline {cpp} read_{stem}(BufferReader& r);")); + } + w.blank(); + for ty in &types { + match ty { + Ty::Record(name) => { + let s = model + .modules + .iter() + .flat_map(|m| &m.structs) + .find(|s| &s.name == name) + .expect("a record type names a declared record"); + render_record_codec(w, s); + } + Ty::RichEnum(name) => render_rich_enum_codec(w, model.enumeration(name)), + _ => render_composite_codec(w, ty), + } + } + w.line("} // namespace detail"); + w.blank(); +} + +/// Append a record's writer and reader. The reader aggregate-initializes +/// the record from a braced list, whose elements C++ evaluates in order, so +/// the fields are read in wire order. +fn render_record_codec(w: &mut CodeWriter, s: &StructBinding) { + let name = &s.name; + w.line(format!("/** Writes one `{name}` to a value buffer. */")); + w.block( + format!("inline void write_{name}(BufferWriter& w, const {name}& v) {{"), + "}", + |w| { + if s.fields.is_empty() { + w.line("(void)w;"); + w.line("(void)v;"); + } + for f in &s.fields { + w.line(write_stmt(&f.ty, &format!("v.{}", cpp_ident(&f.name)), "w")); + } + }, + ); + w.blank(); + w.line(format!("/** Reads one `{name}` from a value buffer. */")); + w.block( + format!("inline {name} read_{name}(BufferReader& r) {{"), + "}", + |w| { + if s.fields.is_empty() { + w.line("(void)r;"); + w.line(format!("return {name}{{}};")); + return; + } + w.line(format!("return {name}{{")); + w.scope(|w| { + for f in &s.fields { + w.line(format!("{},", read_expr(&f.ty, "r"))); + } + }); + w.line("};"); + }, + ); + w.blank(); +} + +/// Append a rich enum's writer and reader: an `i32` tag followed by the +/// active variant's fields in wire order. +fn render_rich_enum_codec(w: &mut CodeWriter, e: &EnumBinding) { + let name = &e.name; + w.line(format!("/** Writes one `{name}` to a value buffer. */")); + w.block( + format!("inline void write_{name}(BufferWriter& w, const {name}& v) {{"), + "}", + |w| { + w.block("switch (v.value.index()) {", "}", |w| { + for (i, variant) in e.variants.iter().enumerate() { + w.block(format!("case {i}: {{"), "}", |w| { + w.line(format!("w.write_i32({});", variant.value)); + if !variant.fields.is_empty() { + w.line(format!( + "const {name}::{}& p = std::get<{i}>(v.value);", + cpp_ident(&variant.name) + )); + for f in &variant.fields { + w.line(write_stmt( + &f.ty, + &format!("p.{}", cpp_ident(&f.name)), + "w", + )); + } + } + w.line("break;"); + }); + } + }); + }, + ); + w.blank(); + w.line(format!("/** Reads one `{name}` from a value buffer. */")); + w.block( + format!("inline {name} read_{name}(BufferReader& r) {{"), + "}", + |w| { + w.line("int32_t tag = r.read_i32();"); + w.block("switch (tag) {", "}", |w| { + for variant in &e.variants { + let vn = cpp_ident(&variant.name); + if variant.fields.is_empty() { + w.line(format!( + "case {}: return {name}{{{name}::{vn}{{}}}};", + variant.value + )); + continue; + } + w.line(format!("case {}:", variant.value)); + w.scope(|w| { + w.line(format!("return {name}{{{name}::{vn}{{")); + w.scope(|w| { + for f in &variant.fields { + w.line(format!("{},", read_expr(&f.ty, "r"))); + } + }); + w.line("}};"); + }); + } + w.line("default: break;"); + }); + w.line(format!("BufferReader::fail(\"unknown {name} tag\");")); + }, + ); + w.blank(); +} + +/// Append one composite type's writer and reader. A map rejects a repeated +/// key, which would otherwise drop an entry silently. +fn render_composite_codec(w: &mut CodeWriter, ty: &Ty) { + let cpp = cpp_type(ty); + let stem = mangle(ty); + w.line(format!("/** Writes one `{ty}` to a value buffer. */")); + w.block( + format!("inline void write_{stem}(BufferWriter& w, const {cpp}& v) {{"), + "}", + |w| match ty { + Ty::Optional(inner) => { + w.line("w.write_option_flag(v.has_value());"); + w.line(format!( + "if (v.has_value()) {}", + write_stmt(inner, "*v", "w") + )); + } + Ty::List(inner) => { + w.line("w.write_len(v.size());"); + w.line(format!( + "for (const auto& item : v) {}", + write_stmt(inner, "item", "w") + )); + } + Ty::Map(k, val) => { + w.line("w.write_len(v.size());"); + w.block("for (const auto& entry : v) {", "}", |w| { + w.line(write_stmt(k, "entry.first", "w")); + w.line(write_stmt(val, "entry.second", "w")); + }); + } + _ => unreachable!("only optionals, lists, and maps are composites"), + }, + ); + w.blank(); + w.line(format!("/** Reads one `{ty}` from a value buffer. */")); + w.block( + format!("inline {cpp} read_{stem}(BufferReader& r) {{"), + "}", + |w| match ty { + Ty::Optional(inner) => { + w.line("if (!r.read_option_flag()) return std::nullopt;"); + w.line(format!("return {cpp}({});", read_expr(inner, "r"))); + } + Ty::List(inner) => { + w.line("size_t n = r.read_count();"); + w.line(format!("{cpp} v;")); + w.line("v.reserve(r.reserve_hint(n));"); + w.line(format!( + "for (size_t i = 0; i < n; ++i) v.push_back({});", + read_expr(inner, "r") + )); + w.line("return v;"); + } + Ty::Map(k, val) => { + w.line("size_t n = r.read_count();"); + w.line(format!("{cpp} v;")); + w.line("v.reserve(r.reserve_hint(n));"); + w.block("for (size_t i = 0; i < n; ++i) {", "}", |w| { + w.line(format!("{} key = {};", cpp_type(k), read_expr(k, "r"))); + w.line(format!("{} value = {};", cpp_type(val), read_expr(val, "r"))); + w.line( + "if (!v.emplace(std::move(key), std::move(value)).second) BufferReader::fail(\"repeated map key\");", + ); + }); + w.line("return v;"); + } + _ => unreachable!("only optionals, lists, and maps are composites"), + }, + ); + w.blank(); +} diff --git a/crates/weaveffi-cli/src/targets/cpp/entities.rs b/crates/weaveffi-cli/src/targets/cpp/entities.rs new file mode 100644 index 00000000..50452e4b --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/entities.rs @@ -0,0 +1,633 @@ +//! Declared-entity rendering: plain enums, value types (records and rich +//! enums), typed error domains, and interface RAII classes, plus the +//! dependency ordering that keeps by-value members complete before use. + +use std::collections::HashMap; + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use weaveffi_model::model::{ + CallShape, EnumBinding, ErrorBinding, FnBinding, InterfaceBinding, ModuleBinding, StructBinding, +}; +use weaveffi_model::ty::{Family, Ty}; + +use crate::targets::cpp::calls::{ + iterator_class_name, render_definition, render_iterator_range, render_member_decl, FnKind, +}; +use crate::targets::cpp::codec::{read_expr, write_stmt}; +use crate::targets::cpp::types::{cpp_error_class, cpp_ident, cpp_member_name, cpp_type}; + +/// A doc comment with a trailing `@deprecated` line when the declaration is +/// deprecated. Types carry the deprecation in their docs rather than as an +/// attribute, so the generated code that marshals them compiles warning-free. +pub(crate) fn doc_text(doc: Option<&str>, deprecated: Option<&str>) -> Option { + match (doc, deprecated) { + (Some(d), Some(msg)) => Some(format!("{d}\n\n@deprecated {msg}")), + (None, Some(msg)) => Some(format!("@deprecated {msg}")), + (d, None) => d.map(str::to_string), + } +} + +/// `expr` handed on to a constructor: moved, unless it's a direct value +/// (a scalar or enum), which copies. +fn pass_on(ty: &Ty, expr: &str) -> String { + if ty.family() == Family::Direct { + expr.to_string() + } else { + format!("std::move({expr})") + } +} + +// ── Enums ── + +/// Append one module's C-style enums as `enum class {Name} : int32_t`. Rich +/// enums are value types, rendered with the records. +pub(crate) fn render_cpp_enums(w: &mut CodeWriter, module: &ModuleBinding) { + for e in module.enums.iter().filter(|e| !e.is_rich()) { + w.doc( + &doc_text(e.doc.as_deref(), e.deprecated.as_deref()), + DocCommentStyle::Javadoc, + ); + w.block(format!("enum class {} : int32_t {{", e.name), "};", |w| { + for v in &e.variants { + w.doc(&v.doc, DocCommentStyle::Javadoc); + w.line(format!("{} = {},", cpp_ident(&v.name), v.value)); + } + }); + w.blank(); + } +} + +// ── Value types: records and rich enums ── + +/// A value type emitted as a plain C++ struct: a record or a rich enum. Both +/// cross the ABI in value buffers, may nest one another, and are ordered +/// together so a by-value member's type is complete before its holder. +pub(crate) enum ValueDef<'a> { + /// A record: a plain struct with typed members. + Record(&'a StructBinding), + /// A rich enum: a `std::variant`-backed sum type. + Rich(&'a EnumBinding), +} + +impl ValueDef<'_> { + /// The value type's C++ name. + pub(crate) fn name(&self) -> &str { + match self { + ValueDef::Record(s) => &s.name, + ValueDef::Rich(e) => &e.name, + } + } + + /// Names of other value types this one holds by value. + fn deps(&self) -> Vec { + let mut deps = Vec::new(); + let fields: Vec<&Ty> = match self { + ValueDef::Record(s) => s.fields.iter().map(|f| &f.ty).collect(), + ValueDef::Rich(e) => e + .variants + .iter() + .flat_map(|v| &v.fields) + .map(|f| &f.ty) + .collect(), + }; + for ty in fields { + collect_value_deps(ty, &mut deps); + } + deps + } + + /// Append the type's definition. + pub(crate) fn render(&self, w: &mut CodeWriter) { + match self { + ValueDef::Record(s) => render_cpp_record(w, s), + ValueDef::Rich(e) => render_cpp_rich_enum(w, e), + } + } +} + +/// Collect the names of value types reachable from `ty` through optionals, +/// lists, and maps. Interfaces aren't collected: every interface class is +/// complete before any value type. +fn collect_value_deps(ty: &Ty, deps: &mut Vec) { + match ty { + Ty::Record(n) | Ty::RichEnum(n) => deps.push(n.clone()), + Ty::Optional(inner) | Ty::List(inner) => collect_value_deps(inner, deps), + Ty::Map(k, v) => { + collect_value_deps(k, deps); + collect_value_deps(v, deps); + } + _ => {} + } +} + +/// Every record and rich enum, ordered so anything a type holds is defined +/// before it (depth-first post-order, declaration order breaking ties). +pub(crate) fn value_types_in_order(modules: &[ModuleBinding]) -> Vec> { + let mut defs: Vec>> = modules + .iter() + .flat_map(|m| { + let records = m.structs.iter().map(ValueDef::Record); + let rich = m.enums.iter().filter(|e| e.is_rich()).map(ValueDef::Rich); + records.chain(rich) + }) + .map(Some) + .collect(); + let deps: Vec> = defs.iter().flatten().map(ValueDef::deps).collect(); + let index: HashMap = defs + .iter() + .flatten() + .enumerate() + .map(|(i, d)| (d.name().to_string(), i)) + .collect(); + + fn visit( + i: usize, + deps: &[Vec], + index: &HashMap, + state: &mut [u8], + order: &mut Vec, + ) { + // 0 unvisited, 1 on the stack (a cycle stops here), 2 emitted. + if state[i] != 0 { + return; + } + state[i] = 1; + for d in &deps[i] { + if let Some(&j) = index.get(d) { + visit(j, deps, index, state, order); + } + } + state[i] = 2; + order.push(i); + } + + let mut state = vec![0u8; defs.len()]; + let mut order = Vec::with_capacity(defs.len()); + for i in 0..defs.len() { + visit(i, &deps, &index, &mut state, &mut order); + } + order + .into_iter() + .map(|i| defs[i].take().expect("each type is emitted once")) + .collect() +} + +/// Render a record as a plain C++ value struct: typed members in declaration +/// (and wire) order. An interface-typed member is the RAII wrapper held by +/// value, so copying the record clones the reference and destroying it +/// releases one. +fn render_cpp_record(w: &mut CodeWriter, s: &StructBinding) { + w.doc( + &doc_text(s.doc.as_deref(), s.deprecated.as_deref()), + DocCommentStyle::Javadoc, + ); + w.block(format!("struct {} {{", s.name), "};", |w| { + for f in &s.fields { + w.doc(&f.doc, DocCommentStyle::Javadoc); + w.line(format!("{} {};", cpp_type(&f.ty), cpp_ident(&f.name))); + } + }); + w.blank(); +} + +/// Render a rich enum as a `std::variant`-backed sum type: one payload struct +/// per variant, a `value` member holding the active payload, a nested `Tag` +/// enum mirroring the wire discriminants, and a `tag()` reader. Construct one +/// as `Shape{Shape::Circle{2.0}}`. +fn render_cpp_rich_enum(w: &mut CodeWriter, e: &EnumBinding) { + let name = &e.name; + w.doc( + &doc_text(e.doc.as_deref(), e.deprecated.as_deref()), + DocCommentStyle::Javadoc, + ); + w.block(format!("struct {name} {{"), "};", |w| { + w.line(format!( + "/** Discriminant identifying the active variant of `{name}`. */" + )); + w.block("enum class Tag : int32_t {", "};", |w| { + for v in &e.variants { + w.line(format!("{} = {},", cpp_ident(&v.name), v.value)); + } + }); + w.blank(); + for v in &e.variants { + w.doc(&v.doc, DocCommentStyle::Javadoc); + w.block(format!("struct {} {{", cpp_ident(&v.name)), "};", |w| { + for f in &v.fields { + w.doc(&f.doc, DocCommentStyle::Javadoc); + w.line(format!("{} {};", cpp_type(&f.ty), cpp_ident(&f.name))); + } + }); + w.blank(); + } + let alts: Vec = e.variants.iter().map(|v| cpp_ident(&v.name)).collect(); + w.line("/** The active variant's payload. */"); + w.line(format!("std::variant<{}> value;", alts.join(", "))); + w.blank(); + w.line("/** The tag of the active variant. */"); + w.block("Tag tag() const noexcept {", "}", |w| { + let tags: Vec = alts.iter().map(|v| format!("Tag::{v}")).collect(); + w.line(format!( + "static constexpr Tag tags[] = {{{}}};", + tags.join(", ") + )); + w.line("return tags[value.index()];"); + }); + }); + w.blank(); +} + +// ── Typed error domains ── + +/// `detail::make_{path}_error`: builds the exception for a code, message, +/// and payload of the domain. +pub(crate) fn make_error_fn(eb: &ErrorBinding) -> String { + format!("detail::make_{}_error", eb.owner_path) +} + +/// `detail::check_{path}`: throws the domain's exception for a failed call. +pub(crate) fn check_fn(eb: &ErrorBinding) -> String { + format!("detail::check_{}", eb.owner_path) +} + +/// `detail::report_{path}_error`: reports a domain exception a callback +/// implementation threw through the vtable entry's `out_err`. +pub(crate) fn report_fn(eb: &ErrorBinding) -> String { + format!("detail::report_{}_error", eb.owner_path) +} + +/// Append one module's typed error domain: a domain exception derived from +/// `Error`, one subclass per declared code (with typed members for the +/// code's payload fields), and the `detail` helpers that map a failed call's +/// `out_err` to the typed exception. With `reports`, also the helper that +/// reports a thrown domain exception from a callback trampoline. +/// +/// Domain codes are positive and the runtime owns every negative code, so +/// the mapping sends -5 to `Cancelled` and any other negative code to the +/// root `Error` before consulting the domain's codes; an undeclared positive +/// code falls back to the domain class itself. +pub(crate) fn render_domain_error( + w: &mut CodeWriter, + module: &ModuleBinding, + eb: &ErrorBinding, + prefix: &str, + reports: bool, +) { + let domain = &eb.type_name; + w.line(format!( + "/** The errors the `{}` module's throwing calls report; catch a code's subclass or this. */", + module.dot_path + )); + w.line(format!("class {domain} : public Error {{")); + w.line("public:"); + w.scope(|w| { + w.line("/** Builds an error carrying a domain `code` and `message`. */"); + w.line(format!( + "{domain}(int32_t code, const std::string& message) : Error(code, message) {{}}" + )); + }); + w.line("};"); + w.blank(); + + for code in &eb.codes { + let class = cpp_error_class(&code.name); + let doc = code.doc.clone().unwrap_or_else(|| code.message.clone()); + w.doc( + &Some(format!("{doc}\n\nCode {}.", code.value)), + DocCommentStyle::Javadoc, + ); + w.line(format!("class {class} : public {domain} {{")); + w.line("public:"); + w.scope(|w| { + for f in &code.fields { + w.doc(&f.doc, DocCommentStyle::Javadoc); + w.line(format!("{} {};", cpp_type(&f.ty), cpp_ident(&f.name))); + } + if !code.fields.is_empty() { + w.blank(); + } + let mut params = vec!["const std::string& message".to_string()]; + let mut inits = vec![format!("{domain}({}, message)", code.value)]; + for f in &code.fields { + let name = cpp_ident(&f.name); + params.push(format!("{} {name}", cpp_type(&f.ty))); + inits.push(format!("{name}({})", pass_on(&f.ty, &name))); + } + w.line("/** Builds the error with the producer's `message` and the code's fields. */"); + let explicit = if code.fields.is_empty() { + "explicit " + } else { + "" + }; + w.line(format!( + "{explicit}{class}({}) : {} {{}}", + params.join(", "), + inits.join(", ") + )); + }); + w.line("};"); + w.blank(); + } + + let path = &eb.owner_path; + w.line("namespace detail {"); + w.blank(); + w.line(format!( + "/** The exception for a failed `{domain}` call: its code's subclass, Cancelled, or the root Error. */" + )); + w.block( + format!( + "inline std::exception_ptr make_{path}_error(int32_t code, const std::string& message, const uint8_t* payload_ptr, size_t payload_len) {{" + ), + "}", + |w| { + w.line("if (code == -5) return std::make_exception_ptr(Cancelled(message));"); + w.line("if (code < 0) return std::make_exception_ptr(Error(code, message));"); + if eb.codes.iter().all(|c| c.fields.is_empty()) { + w.line("(void)payload_ptr;"); + w.line("(void)payload_len;"); + } + w.block("switch (code) {", "}", |w| { + for code in &eb.codes { + let class = cpp_error_class(&code.name); + if code.fields.is_empty() { + w.line(format!( + "case {}: return std::make_exception_ptr({class}(message));", + code.value + )); + continue; + } + w.block(format!("case {}: {{", code.value), "}", |w| { + // Locals, because constructor arguments evaluate in + // an unspecified order and the fields are read in + // wire order. + w.line("BufferReader r(payload_ptr, payload_len);"); + let mut args = vec!["message".to_string()]; + for f in &code.fields { + let var = format!("f_{}", f.name); + w.line(format!("{} {var} = {};", cpp_type(&f.ty), read_expr(&f.ty, "r"))); + args.push(pass_on(&f.ty, &var)); + } + w.line("r.expect_end();"); + w.line(format!( + "return std::make_exception_ptr({class}({}));", + args.join(", ") + )); + }); + } + w.line(format!( + "default: return std::make_exception_ptr({domain}(code, message));" + )); + }); + }, + ); + w.blank(); + w.line(format!( + "/** Throws the typed `{domain}` exception if `err` carries a nonzero code. */" + )); + w.block( + format!("inline void check_{path}({prefix}_error& err) {{"), + "}", + |w| { + w.line("if (err.code == 0) return;"); + // The error owns the payload, so the exception (which decodes + // it) is built before error_clear releases it. + w.line(format!( + "std::exception_ptr ex = make_{path}_error(err.code, error_message(err), err.payload_ptr, err.payload_len);" + )); + w.line(format!("{prefix}_error_clear(&err);")); + w.line("std::rethrow_exception(ex);"); + }, + ); + w.blank(); + if reports { + render_report_fn(w, eb, prefix); + } + w.line("} // namespace detail"); + w.blank(); +} + +/// Append the helper a throwing callback method's trampoline uses to report a +/// `{domain}` exception: a declared code goes back with its fields encoded as +/// the payload; an exception whose code the domain doesn't declare (or a +/// code with fields thrown as the bare domain class, which has no fields to +/// send) is a callback failure, -4. +fn render_report_fn(w: &mut CodeWriter, eb: &ErrorBinding, prefix: &str) { + let domain = &eb.type_name; + w.line(format!( + "/** Reports a `{domain}` a callback implementation threw through the vtable entry's `out_err`. */" + )); + w.block( + format!( + "inline void report_{}_error(const {domain}& e, {prefix}_error* out_err) {{", + eb.owner_path + ), + "}", + |w| { + w.block("switch (e.code()) {", "}", |w| { + for code in &eb.codes { + let class = cpp_error_class(&code.name); + if code.fields.is_empty() { + w.line(format!( + "case {}: {prefix}_error_set(out_err, {}, e.what()); return;", + code.value, code.value + )); + continue; + } + w.block(format!("case {}: {{", code.value), "}", |w| { + w.line(format!( + "const auto* typed = dynamic_cast(&e);" + )); + w.line("if (typed == nullptr) break;"); + w.line("BufferWriter payload;"); + for f in &code.fields { + w.line(write_stmt( + &f.ty, + &format!("typed->{}", cpp_ident(&f.name)), + "payload", + )); + } + w.line(format!( + "{prefix}_error_set(out_err, {}, e.what());", + code.value + )); + w.line(format!( + "{prefix}_error_set_payload(out_err, payload.data(), payload.size());" + )); + w.line("return;"); + }); + } + w.line("default: break;"); + }); + w.line(format!("{prefix}_error_set(out_err, -4, e.what());")); + }, + ); + w.blank(); +} + +// ── Interfaces ── + +/// Append the reference-counting RAII skeleton of an interface class: the +/// `raw_type` alias of the C tag, the tagged constructor adopting one strong +/// reference, a destructor releasing it, copies that take a new reference +/// through `_clone`, moves that transfer the pointer, and the +/// `handle()`/`clone_handle()` readers. +fn render_raii_skeleton(w: &mut CodeWriter, i: &InterfaceBinding) { + let name = &i.name; + let tag = &i.c_tag; + let clone = &i.clone_symbol; + let destroy = &i.destroy_symbol; + + w.line("/** The C type this class wraps. */"); + w.line(format!("using raw_type = {tag};")); + w.blank(); + w.line(format!( + "/** Adopts one strong reference to a producer object: `{name}(adopt, raw)`. */" + )); + w.line(format!( + "explicit {name}(adopt_t, {tag}* h) noexcept : raw_(h) {{}}" + )); + w.blank(); + w.line("/** Releases this wrapper's reference; the object is dropped with its last one. */"); + w.block(format!("~{name}() {{"), "}", |w| { + w.line(format!("if (raw_ != nullptr) {destroy}(raw_);")); + }); + w.blank(); + w.line("/** Copies share the object: the copy takes a new strong reference. */"); + w.line(format!( + "{name}(const {name}& other) : raw_({clone}(other.raw_)) {{}}" + )); + w.blank(); + w.line("/** Takes a new reference to `other`'s object and releases the current one. */"); + w.block( + format!("{name}& operator=(const {name}& other) {{"), + "}", + |w| { + w.block("if (this != &other) {", "}", |w| { + w.line(format!("{tag}* h = {clone}(other.raw_);")); + w.line(format!("if (raw_ != nullptr) {destroy}(raw_);")); + w.line("raw_ = h;"); + }); + w.line("return *this;"); + }, + ); + w.blank(); + w.line("/** Transfers `other`'s reference; `other` becomes empty. */"); + w.line(format!( + "{name}({name}&& other) noexcept : raw_(other.raw_) {{ other.raw_ = nullptr; }}" + )); + w.blank(); + w.line("/** Releases the current reference and takes over `other`'s. */"); + w.block( + format!("{name}& operator=({name}&& other) noexcept {{"), + "}", + |w| { + w.block("if (this != &other) {", "}", |w| { + w.line(format!("if (raw_ != nullptr) {destroy}(raw_);")); + w.line("raw_ = other.raw_;"); + w.line("other.raw_ = nullptr;"); + }); + w.line("return *this;"); + }, + ); + w.blank(); + w.line("/** The wrapped pointer, borrowed (null after a move): this wrapper keeps its reference. */"); + w.line(format!( + "const {tag}* handle() const noexcept {{ return raw_; }}" + )); + w.blank(); + w.line("/** A new strong reference to the object, which the caller owns. */"); + w.line(format!( + "{tag}* clone_handle() const {{ return {clone}(raw_); }}" + )); + w.blank(); +} + +/// The C++ name and declaration kind of each interface member. The +/// synchronous constructor named `new` becomes the C++ constructor; every +/// other constructor becomes a static factory named after it. +fn member_kinds(i: &InterfaceBinding) -> Vec<(&FnBinding, String, FnKind<'_>)> { + let class = i.name.as_str(); + let mut members = Vec::new(); + for c in &i.constructors { + if c.name == "new" && matches!(c.shape, CallShape::Sync(_)) { + members.push((c, class.to_string(), FnKind::Ctor { class })); + } else { + members.push((c, cpp_member_name(&c.name), FnKind::Static { class })); + } + } + for m in &i.methods { + members.push((m, cpp_member_name(&m.name), FnKind::Method { class })); + } + for s in &i.statics { + members.push((s, cpp_member_name(&s.name), FnKind::Static { class })); + } + members +} + +/// Append the forward declarations an interface needs before any class body: +/// the wrapper class and the range class of every iterator-returning member. +pub(crate) fn render_cpp_interface_forward_decls(w: &mut CodeWriter, i: &InterfaceBinding) { + w.line(format!("class {};", i.name)); + for (f, _, kind) in member_kinds(i) { + if matches!(f.shape, CallShape::Iterator(_)) { + w.line(format!("class {};", iterator_class_name(f, kind))); + } + } +} + +/// Append an interface's class definition: the RAII skeleton plus the +/// *declarations* of its members. The member bodies follow every value type +/// and codec ([`render_cpp_interface_members`]); the class itself comes +/// first so records can hold it by value. +pub(crate) fn render_cpp_interface_class( + w: &mut CodeWriter, + i: &InterfaceBinding, + error: Option<&ErrorBinding>, +) { + w.doc( + &doc_text(i.doc.as_deref(), i.deprecated.as_deref()), + DocCommentStyle::Javadoc, + ); + w.line(format!("class {} {{", i.name)); + w.scope(|w| { + w.line(format!("{}* raw_;", i.c_tag)); + w.blank(); + }); + w.line("public:"); + w.scope(|w| { + render_raii_skeleton(w, i); + for (f, cpp_name, kind) in member_kinds(i) { + render_member_decl(w, f, &cpp_name, kind, error); + } + }); + w.line("};"); + w.blank(); +} + +/// Append the range classes of an interface's iterator-returning members. +pub(crate) fn render_cpp_interface_iterators( + w: &mut CodeWriter, + i: &InterfaceBinding, + error: Option<&ErrorBinding>, + prefix: &str, +) { + for (f, cpp_name, kind) in member_kinds(i) { + if let CallShape::Iterator(it) = &f.shape { + render_iterator_range(w, f, it, &cpp_name, kind, error, prefix); + } + } +} + +/// Append the out-of-line `inline` definitions of an interface's members. +pub(crate) fn render_cpp_interface_members( + w: &mut CodeWriter, + i: &InterfaceBinding, + error: Option<&ErrorBinding>, + prefix: &str, +) { + for (f, cpp_name, kind) in member_kinds(i) { + render_definition(w, f, &cpp_name, kind, error, prefix); + } +} diff --git a/crates/weaveffi-cli/src/targets/cpp/mod.rs b/crates/weaveffi-cli/src/targets/cpp/mod.rs new file mode 100644 index 00000000..e92990be --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/mod.rs @@ -0,0 +1,361 @@ +//! C++ wrapper generator. +//! +//! Produces an idiomatic header-only C++17 library over the C ABI: the C +//! header `{library}.h` (the same file the [`c`](super::c) target emits), a +//! wrapper header `{library}.hpp` that includes it, a `CMakeLists.txt` +//! exporting an INTERFACE target, and a README. Everything lives in +//! `namespace {prefix}` by default. Implements [`LanguageBackend`]; the +//! shared driver bridges it into the generator pipeline. +//! +//! The generated surface follows C ABI revision 4: +//! +//! * Strings cross as a UTF-8 pointer and length: parameters are +//! `std::string_view`, returns are `std::string` copied out of the +//! producer's run and released with `{prefix}_free_bytes`. +//! * Records are plain value structs; rich enums are `std::variant`-backed +//! sum types with one payload struct per variant. Both cross as value +//! buffers through one generated writer and reader per record, rich enum, +//! and distinct composite type (`[Entry]`, `{string:i64}`, `Store?`) in +//! `detail`. +//! * Interfaces are RAII classes holding one strong reference: the destructor +//! calls `_destroy`, copying calls `_clone`, and moving transfers the +//! pointer. A parameter is borrowed; a returned object, an async result, +//! an iterator element, and an object argument handed to a callback are +//! adopted. Inside a value buffer an object is a token minted with +//! `_clone`. +//! * Callback interfaces are abstract classes passed as `std::shared_ptr` +//! (empty for an optional `Cb?`). The wrapper boxes the pointer as `ctx` +//! and hands the producer a static vtable with the `{size, flags, free}` +//! header. Methods return any family (strings, bytes, and buffers through +//! a `{prefix}_alloc` run); a `throws` method reports the module's domain +//! exception with its fields as the payload, and any other exception as +//! -4. +//! * Free functions live in a nested namespace per IDL module +//! (`kvstore::kv::stats::summarize`). +//! * An `iter` callable returns a move-only lazy range; async callables +//! return `std::future`, and a cancellable one takes a trailing +//! `const CancelToken&`. +//! * A throwing call throws its module's domain exception (`KvError` and a +//! subclass per code), the root `Error` for a runtime code, or `Cancelled`. +//! A call that declares no errors throws `InternalError` (the trap policy). +//! * `check_library()` verifies the producer's ABI revision and every +//! top-level module's contract table once, and throws `LoadError` naming +//! the first declaration the library lacks or changed. Every free +//! function, constructor, and static member calls it. + +mod callbacks; +mod calls; +mod codec; +mod entities; +mod package; +mod runtime; +mod types; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::codegen::CodeWriter; +use crate::package::{per_platform_libraries, Artifact, PackageContext, PackagedFile}; +use crate::platform::Platform; +use crate::targets::c::{header_name as c_header_name, render_c_header_from_model}; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; +use camino::Utf8Path; +use serde::{Deserialize, Serialize}; +use weaveffi_model::model::{EnumBinding, Model}; +use weaveffi_model::pkg::Identity; + +use crate::targets::cpp::callbacks::render_callback_interface; +use crate::targets::cpp::calls::render_cpp_module_ns; +use crate::targets::cpp::codec::render_codecs; +use crate::targets::cpp::entities::{ + render_cpp_enums, render_cpp_interface_class, render_cpp_interface_forward_decls, + render_cpp_interface_iterators, render_cpp_interface_members, render_domain_error, + value_types_in_order, +}; +use crate::targets::cpp::package::{ + render_cmake, render_packaged_cmake, render_packaged_readme, render_readme, CmakeNames, +}; +use crate::targets::cpp::runtime::{render_buffer_runtime, render_prelude_runtime}; + +/// Per-target configuration for [`CppGenerator`]. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct CppConfig { + /// The C++ namespace holding every generated declaration (default: the + /// package's C prefix). + pub name: Option, + /// Filename of the wrapper header (default `{library}.hpp`). + pub header_name: Option, + /// C++ standard advertised in the generated `CMakeLists.txt` (default + /// `"17"`). + pub standard: Option, +} + +impl CppConfig { + /// Returns the configured C++ namespace, falling back to the identity's + /// C prefix. + pub fn namespace(&self, identity: &Identity) -> String { + self.name.clone().unwrap_or_else(|| identity.prefix.clone()) + } + + /// Returns the wrapper header's filename, falling back to + /// `{library}.hpp`. + pub fn header_name(&self, identity: &Identity) -> String { + self.header_name + .clone() + .unwrap_or_else(|| format!("{}.hpp", identity.library)) + } + + /// Returns the C++ standard advertised in the generated `CMakeLists.txt`, + /// falling back to `"17"`. + pub fn standard(&self) -> &str { + self.standard.as_deref().unwrap_or("17") + } +} + +/// C++ backend: emits a wrapper header (`{library}.hpp` by default), a copy +/// of the C header it includes, a `CMakeLists.txt`, and a README. +pub struct CppGenerator; + +/// The names every rendered file agrees on, resolved once per run. +struct Names { + namespace: String, + header: String, + c_header: String, +} + +impl Names { + fn new(model: &Model, config: &CppConfig) -> Self { + Self { + namespace: config.namespace(&model.identity), + header: config.header_name(&model.identity), + c_header: c_header_name(model), + } + } + + fn cmake<'a>(&'a self, identity: &'a Identity, config: &'a CppConfig) -> CmakeNames<'a> { + CmakeNames { + identity, + namespace: &self.namespace, + header: &self.header, + standard: config.standard(), + } + } + + fn render_header(&self, model: &Model) -> String { + render_cpp_header(model, &self.namespace, &self.c_header, &self.header) + } +} + +impl LanguageBackend for CppGenerator { + type Config = CppConfig; + + fn name(&self) -> &'static str { + "cpp" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let dir = out_dir.join("cpp"); + let names = Names::new(model, config); + let cmake = names.cmake(&model.identity, config); + vec![ + OutputFile::new( + dir.join(&names.c_header), + render_c_header_from_model(model, &names.c_header), + ), + OutputFile::new(dir.join(&names.header), names.render_header(model)), + OutputFile::new(dir.join("CMakeLists.txt"), render_cmake(&cmake)), + OutputFile::new(dir.join("README.md"), render_readme(&cmake)), + ] + } + + /// One `{library}-{version}-cpp.tar.gz` holding both headers under + /// `include/`, the desktop libraries under `lib//`, and a + /// `CMakeLists.txt` that links the host's library into the wrapper's + /// interface target. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let identity = &model.identity; + let names = Names::new(model, config); + let cmake = names.cmake(identity, config); + let lib = &ctx.binaries.lib_name; + let mut files = vec![ + PackagedFile::text( + format!("include/{}", names.c_header), + render_c_header_from_model(model, &names.c_header), + ), + PackagedFile::text( + format!("include/{}", names.header), + names.render_header(model), + ), + PackagedFile::text("CMakeLists.txt", render_packaged_cmake(&cmake, lib)), + PackagedFile::text("README.md", render_packaged_readme(&cmake, ctx)), + ]; + // The packaged CMake selects among the desktop platforms only, so + // Android and Wasm binaries (which belong to other ecosystems' + // packages) are not bundled here. + let desktop = per_platform_libraries(ctx.binaries, "lib", Platform::is_desktop); + if desktop.is_empty() { + return Some(Vec::new()); + } + files.extend(desktop); + let stem = format!("{lib}-{}", identity.version); + Some(vec![Artifact::tar_gz( + format!("cpp/{stem}-cpp.tar.gz"), + stem, + files, + )]) + } +} + +/// Render the complete C++ wrapper header. `c_header` is the C header it +/// includes and `filename` its own file name. +/// +/// Layout inside `namespace {namespace}`, in an order that keeps every type +/// complete before it's held by value or marshalled: +/// +/// 1. the runtime: error types, the adopt tag, `CancelToken`, helpers, +/// `check_library()` with the expected contract tables, and the +/// value-buffer reader and writer (when any value crosses as a buffer); +/// 2. C-style enums; +/// 3. forward declarations of every value type, interface class, +/// callback-interface class, and member range class; +/// 4. interface classes (the RAII skeleton plus member *declarations*), so +/// records can hold objects by value; +/// 5. value types (records and rich enums) in dependency order; +/// 6. the codecs of every value and composite type; +/// 7. typed exception domains; +/// 8. callback-interface abstract classes, trampolines, and vtables; +/// 9. the range classes of iterator-returning members; +/// 10. the out-of-line definitions of every interface member; and +/// 11. one nested namespace per module holding its free functions. +pub(crate) fn render_cpp_header( + model: &Model, + namespace: &str, + c_header: &str, + filename: &str, +) -> String { + let prefix = model.prefix(); + let mut includes = vec![ + "cstddef", + "cstdint", + "cstring", + "exception", + "memory", + "new", + "optional", + "stdexcept", + "string", + "string_view", + "unordered_map", + "utility", + "vector", + ]; + if model + .modules + .iter() + .any(|m| m.enums.iter().any(EnumBinding::is_rich)) + { + includes.push("variant"); + } + if model.has_async() { + includes.push("future"); + } + if model.has_iterators() { + includes.push("iterator"); + } + includes.sort_unstable(); + + let mut w = CodeWriter::four_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + w.line("#pragma once"); + w.blank(); + w.line(format!("#include \"{c_header}\"")); + w.blank(); + for inc in includes { + w.line(format!("#include <{inc}>")); + } + w.blank(); + w.line(format!("namespace {namespace} {{")); + w.blank(); + render_prelude_runtime(&mut w, model); + if model.has_buffers() { + render_buffer_runtime(&mut w, prefix); + } + + for module in &model.modules { + render_cpp_enums(&mut w, module); + } + + // Forward declarations let member declarations name any type before its + // definition. + let mut forward = false; + for module in &model.modules { + for s in &module.structs { + w.line(format!("struct {};", s.name)); + forward = true; + } + for e in module.enums.iter().filter(|e| e.is_rich()) { + w.line(format!("struct {};", e.name)); + forward = true; + } + for i in &module.interfaces { + render_cpp_interface_forward_decls(&mut w, i); + forward = true; + } + for cb in &module.callback_interfaces { + w.line(format!("class {};", cb.name)); + forward = true; + } + } + if forward { + w.blank(); + } + + for module in &model.modules { + for i in &module.interfaces { + render_cpp_interface_class(&mut w, i, model.error_domain(module)); + } + } + for def in value_types_in_order(&model.modules) { + def.render(&mut w); + } + render_codecs(&mut w, model); + + // A domain reports exceptions back to the producer only when a throwing + // callback method has it in scope. + for module in &model.modules { + if let Some(eb) = &module.errors { + let reports = model.callback_interfaces().any(|(m, cb)| { + model.error_domain(m).is_some_and(|d| d.c_tag == eb.c_tag) + && cb.methods.iter().any(|meth| meth.throws) + }); + render_domain_error(&mut w, module, eb, prefix, reports); + } + } + for (module, cb) in model.callback_interfaces() { + render_callback_interface(&mut w, cb, model.error_domain(module), prefix); + } + for module in &model.modules { + for i in &module.interfaces { + render_cpp_interface_iterators(&mut w, i, model.error_domain(module), prefix); + } + } + for module in &model.modules { + for i in &module.interfaces { + render_cpp_interface_members(&mut w, i, model.error_domain(module), prefix); + } + } + for module in &model.modules { + render_cpp_module_ns(&mut w, module, model.error_domain(module), prefix); + } + w.line(format!("}} // namespace {namespace}")); + w.blank(); + w.raw(render_trailer(CommentStyle::DoubleSlash, filename)); + w.finish() +} + +#[cfg(test)] +mod tests; diff --git a/crates/weaveffi-gen/src/targets/cpp/package.rs b/crates/weaveffi-cli/src/targets/cpp/package.rs similarity index 94% rename from crates/weaveffi-gen/src/targets/cpp/package.rs rename to crates/weaveffi-cli/src/targets/cpp/package.rs index 3f6fb573..f4088b9b 100644 --- a/crates/weaveffi-gen/src/targets/cpp/package.rs +++ b/crates/weaveffi-cli/src/targets/cpp/package.rs @@ -20,8 +20,6 @@ pub(crate) struct CmakeNames<'a> { pub(crate) header: &'a str, /// The C++ standard (`17`). pub(crate) standard: &'a str, - /// The IDL basename for the generated-file prelude. - pub(crate) input_basename: &'a str, } impl CmakeNames<'_> { @@ -64,7 +62,7 @@ endif() "#; format!( "{}{}\n{}", - render_prelude(CommentStyle::Hash, names.input_basename), + render_prelude(CommentStyle::Hash), names.fill(body), render_trailer(CommentStyle::Hash, "CMakeLists.txt") ) @@ -102,6 +100,10 @@ if(NOT TARGET @LIBRARY@) add_library(@LIBRARY@ SHARED IMPORTED GLOBAL) set_target_properties(@LIBRARY@ PROPERTIES IMPORTED_LOCATION "${CMAKE_CURRENT_LIST_DIR}/lib/${_plat}/${_libfile}") + if(WIN32) + set_target_properties(@LIBRARY@ PROPERTIES + IMPORTED_IMPLIB "${CMAKE_CURRENT_LIST_DIR}/lib/${_plat}/@LIB@.dll.lib") + endif() endif() add_library(@LIBRARY@_cpp INTERFACE) @@ -112,7 +114,7 @@ target_compile_features(@LIBRARY@_cpp INTERFACE cxx_std_@STD@) "#; format!( "{}{}\n{}", - render_prelude(CommentStyle::Hash, names.input_basename), + render_prelude(CommentStyle::Hash), names.fill(body).replace("@LIB@", lib), render_trailer(CommentStyle::Hash, "CMakeLists.txt") ) @@ -148,7 +150,7 @@ it into the `@LIBRARY@::cpp` interface target. "#; format!( "{}{}{}\n\n{}", - render_prelude(CommentStyle::Xml, names.input_basename), + render_prelude(CommentStyle::Xml), names.fill(body), platforms.join("\n"), render_trailer(CommentStyle::Xml, "README.md") @@ -196,7 +198,7 @@ at startup to check eagerly. "#; format!( "{}{}{}", - render_prelude(CommentStyle::Xml, names.input_basename), + render_prelude(CommentStyle::Xml), names.fill(body), render_trailer(CommentStyle::Xml, "README.md") ) diff --git a/crates/weaveffi-cli/src/targets/cpp/runtime.rs b/crates/weaveffi-cli/src/targets/cpp/runtime.rs new file mode 100644 index 00000000..81fb6acc --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/runtime.rs @@ -0,0 +1,97 @@ +//! The fixed runtime of the generated header: the error types, the adopt +//! tag, the cancel token, the string and callback-return helpers, the +//! load-time library check, and the value-buffer reader and writer. +//! +//! The code lives in real C++ sources under `runtime/`, written into the +//! header with `{{PLACEHOLDER}}` substitution. The library check embeds the +//! contract table of every top-level module (from +//! [`weaveffi_model::contract::entries`]) as `(id, hash, path)` triples, so +//! a mismatch names the declaration. + +use crate::codegen::CodeWriter; +use weaveffi_model::contract; +use weaveffi_model::model::{contract_symbol, Model, ABI_VERSION}; + +/// The error types, adopt tag, cancel token, helpers, and `check_library()`. +/// `{{CONTRACTS}}` marks where the expected tables go and `{{CHECKS}}` where +/// `check_library()` compares them. +const PRELUDE: &str = include_str!("runtime/prelude.hpp"); + +/// The value-buffer reader, writer, and release guard. +const BUFFER: &str = include_str!("runtime/buffer.hpp"); + +/// The `detail` array holding a top-level module's expected contract. +fn contract_array(module: &str) -> String { + format!("{module}_contract") +} + +/// Append the runtime prelude. `check_library()` compares the producer's +/// ABI revision and every top-level module's contract table against the +/// declarations this header was generated with. +pub(crate) fn render_prelude_runtime(w: &mut CodeWriter, model: &Model) { + let prefix = model.prefix(); + let library = &model.identity.library; + let text = PRELUDE + .replace("{{PREFIX}}", prefix) + .replace("{{MACRO}}", &model.identity.macro_prefix()) + .replace("{{LIBRARY}}", library) + .replace("{{ABI}}", &ABI_VERSION.to_string()); + let (head, rest) = text + .split_once("{{CONTRACTS}}") + .expect("the prelude marks the contract tables"); + let (middle, tail) = rest + .split_once("{{CHECKS}}") + .expect("the prelude marks the contract checks"); + let roots: Vec<_> = model + .roots() + .map(|root| (root, contract::entries(model, root))) + .filter(|(_, entries)| !entries.is_empty()) + .collect(); + + w.raw(head); + for (root, entries) in &roots { + w.line(format!( + "/** The declarations of module `{}` this header was generated with. */", + root.dot_path + )); + w.block( + format!( + "inline constexpr ContractEntry {}[] = {{", + contract_array(&root.name) + ), + "};", + |w| { + for e in entries { + w.line(format!( + "{{{:#018x}ull, {:#018x}ull, \"{}\"}},", + e.id, e.hash, e.path + )); + } + }, + ); + w.blank(); + } + w.raw(middle); + // The checks sit inside `check_library()`'s initializing lambda. + w.indent().indent(); + for (root, _) in &roots { + w.block( + format!( + "if (std::string why = detail::contract_mismatch({}, detail::{}); !why.empty()) {{", + contract_symbol(prefix, &root.name), + contract_array(&root.name) + ), + "}", + |w| { + w.line(format!("return \"{library}: \" + why;")); + }, + ); + } + w.dedent().dedent(); + w.raw(tail); +} + +/// Append the value-buffer runtime. +pub(crate) fn render_buffer_runtime(w: &mut CodeWriter, prefix: &str) { + w.raw(BUFFER.replace("{{PREFIX}}", prefix)); +} diff --git a/crates/weaveffi-gen/src/targets/cpp/runtime/buffer.hpp b/crates/weaveffi-cli/src/targets/cpp/runtime/buffer.hpp similarity index 76% rename from crates/weaveffi-gen/src/targets/cpp/runtime/buffer.hpp rename to crates/weaveffi-cli/src/targets/cpp/runtime/buffer.hpp index f72d7698..a04f2e4e 100644 --- a/crates/weaveffi-gen/src/targets/cpp/runtime/buffer.hpp +++ b/crates/weaveffi-cli/src/targets/cpp/runtime/buffer.hpp @@ -45,7 +45,7 @@ class BufferWriter { /** Writes a byte length or an element count as a u32. */ void write_len(size_t n) { - if (static_cast(n) > UINT32_MAX) throw Error(-3, "value buffer length exceeds u32"); + if (static_cast(n) > UINT32_MAX) throw std::length_error("value buffer length exceeds u32"); append_le(static_cast(n)); } @@ -61,24 +61,29 @@ class BufferWriter { /** Writes an optional's presence flag: 0 absent, 1 present. */ void write_option_flag(bool present) { buf_.push_back(present ? 1 : 0); } + + /** + * Writes an object token: a fresh strong reference from the wrapper's + * `_clone`, which the reader adopts while `v` keeps its own. + */ + template + void write_object(const T& v) { + write_u64(static_cast(reinterpret_cast(v.clone_handle()))); + } }; /** * Decodes values from the value-buffer wire format. A malformed buffer is a - * producer/consumer contract violation (both sides are generated from one - * API), so every decode failure throws the generic Error with the - * marshalling code -3. Object tokens adopted before a failure live in RAII - * wrappers that release them on unwind. + * producer bug (both sides are generated from one API), so every decode + * failure throws InternalError with the marshalling code -3. Object tokens + * adopted before a failure live in RAII wrappers that release them on + * unwind. */ class BufferReader { const uint8_t* data_; size_t len_; size_t pos_; - [[noreturn]] static void fail(const char* what) { - throw Error(-3, std::string("malformed value buffer: ") + what); - } - void require(size_t n, const char* what) const { if (len_ - pos_ < n) fail(what); } @@ -95,6 +100,12 @@ class BufferReader { } public: + /** Throws the marshalling failure (-3) for a malformed buffer. */ + [[noreturn]] static void fail(const char* what) { + throw InternalError(-3, std::string("malformed value buffer: ") + what); + } + + /** Reads from the `len` bytes at `data` (null when empty). */ BufferReader(const uint8_t* data, size_t len) noexcept : data_(data), len_(data != nullptr ? len : 0), pos_(0) {} /** Bytes not yet consumed. */ @@ -166,6 +177,14 @@ class BufferReader { return b != 0; } + /** Reads an object token, adopting the strong reference it carries into a `T` wrapper. */ + template + T read_object() { + auto* raw = reinterpret_cast(static_cast(read_u64())); + if (raw == nullptr) fail("null object token"); + return T(adopt, raw); + } + /** Rejects unconsumed bytes after decoding a complete value. */ void expect_end() const { if (pos_ != len_) fail("trailing bytes after value"); @@ -188,5 +207,21 @@ struct BufferGuard { } }; +/** Decodes one complete `T` from a borrowed buffer with `read`, rejecting trailing bytes. */ +template +T decode(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferReader r(ptr, len); + T value = read(r); + r.expect_end(); + return value; +} + +/** Decodes one complete `T` from a producer-returned buffer with `read`, then releases the buffer. */ +template +T take(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferGuard guard(ptr, len); + return decode(ptr, len, read); +} + } // namespace detail diff --git a/crates/weaveffi-gen/src/targets/cpp/runtime/prelude.hpp b/crates/weaveffi-cli/src/targets/cpp/runtime/prelude.hpp similarity index 54% rename from crates/weaveffi-gen/src/targets/cpp/runtime/prelude.hpp rename to crates/weaveffi-cli/src/targets/cpp/runtime/prelude.hpp index 49b707e7..04c37774 100644 --- a/crates/weaveffi-gen/src/targets/cpp/runtime/prelude.hpp +++ b/crates/weaveffi-cli/src/targets/cpp/runtime/prelude.hpp @@ -1,11 +1,11 @@ /** - * Base exception for every error reported through the C ABI. + * An error a call that declares errors reports. * - * Positive codes are the declared error codes (see the typed domain - * exceptions); negative codes are runtime traps: -1 a generic producer - * error, -2 a producer panic, -3 a marshalling failure, -4 a - * callback-interface implementation that threw, and -5 a cancelled call - * (thrown as Cancelled). + * A positive code is a declared code of the module's error domain, thrown as + * that domain's typed subclass. A negative code is a runtime failure: -1 a + * generic producer error, -2 a producer panic, -3 a marshalling failure, and + * -4 a callback-interface implementation that failed. A cancelled async call + * (-5) throws Cancelled. */ class Error : public std::runtime_error { int32_t code_; @@ -18,7 +18,7 @@ class Error : public std::runtime_error { int32_t code() const noexcept { return code_; } }; -/** Thrown by an async call whose CancelToken was cancelled (code -5). */ +/** Thrown by an async call that was cancelled through its CancelToken (code -5). */ class Cancelled : public Error { public: /** Builds a cancellation error carrying the producer's message. */ @@ -26,13 +26,39 @@ class Cancelled : public Error { }; /** - * Thrown on first use when the linked {{LIBRARY}} library was built for a - * different C ABI revision or a different API contract than this header. + * Thrown by a call that declares no errors when it fails anyway: a producer + * panic (-2), a marshalling failure (-3), a callback-interface failure the + * producer let through (-4), or another runtime failure (-1). These are bugs, + * so no signature declares them; `what()` names the code and the producer's + * message. */ -class LoadError : public Error { +class InternalError : public std::runtime_error { + int32_t code_; + std::string message_; + +public: + /** Builds an internal error carrying `code` and the producer's `message`. */ + InternalError(int32_t code, const std::string& message) + : std::runtime_error("{{LIBRARY}}: runtime error " + std::to_string(code) + ": " + message), + code_(code), + message_(message) {} + + /** The runtime error code (negative). */ + int32_t code() const noexcept { return code_; } + + /** The producer's message. */ + const std::string& message() const noexcept { return message_; } +}; + +/** + * Thrown on first use when the linked {{LIBRARY}} library implements a + * different C ABI revision than this header, or lacks or changed a + * declaration this header was generated with. + */ +class LoadError : public std::runtime_error { public: /** Builds a load error describing the mismatch. */ - explicit LoadError(const std::string& message) : Error(-1, message) {} + explicit LoadError(const std::string& message) : std::runtime_error(message) {} }; /** @@ -43,7 +69,7 @@ struct adopt_t { explicit adopt_t() = default; }; -/** The adopt_t tag value: `Gadget g(adopt, raw)`. */ +/** The adopt_t tag value: `Store s(adopt, raw)`. */ inline constexpr adopt_t adopt{}; /** @@ -109,10 +135,10 @@ class CancelToken { namespace detail { -/** The exception for a runtime (non-domain) error code. */ -inline std::exception_ptr make_error(int32_t code, const std::string& message) { +/** The exception for a failed call that declares no errors: Cancelled for -5, else InternalError. */ +inline std::exception_ptr make_internal_error(int32_t code, const std::string& message) { if (code == -5) return std::make_exception_ptr(Cancelled(message)); - return std::make_exception_ptr(Error(code, message)); + return std::make_exception_ptr(InternalError(code, message)); } /** The message of a nonzero error slot. */ @@ -120,10 +146,10 @@ inline std::string error_message(const {{PREFIX}}_error& err) { return std::string(err.message != nullptr ? err.message : "unknown error"); } -/** Throws the generic Error (or Cancelled) if `err` carries a nonzero code. */ +/** Throws InternalError if `err` carries a nonzero code (a call that declares no errors). */ inline void check({{PREFIX}}_error& err) { if (err.code == 0) return; - std::exception_ptr ex = make_error(err.code, error_message(err)); + std::exception_ptr ex = make_internal_error(err.code, error_message(err)); {{PREFIX}}_error_clear(&err); std::rethrow_exception(ex); } @@ -156,7 +182,7 @@ inline std::string_view borrow_string(const uint8_t* ptr, size_t len) noexcept { return std::string_view(reinterpret_cast(ptr), len); } -/** Views a borrowed byte run (null when empty) as an owned vector. */ +/** Copies a borrowed byte run (null when empty) into a vector. */ inline std::vector borrow_bytes(const uint8_t* ptr, size_t len) { if (ptr == nullptr || len == 0) return std::vector(); return std::vector(ptr, ptr + len); @@ -167,23 +193,82 @@ inline const uint8_t* utf8(std::string_view s) noexcept { return reinterpret_cast(s.data()); } -} // namespace detail +/** + * Hands `len` bytes from `data` to the producer through a callback method's + * out slots: copies them into a run from {{PREFIX}}_alloc, which the + * producer adopts and frees. + */ +inline void hand_over(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = nullptr; + if (len > 0) { + run = {{PREFIX}}_alloc(len); + if (run == nullptr) throw std::bad_alloc(); + std::memcpy(run, data, len); + } + *out_ptr = run; + *out_len = len; +} + +/** One declaration this header was generated with: its contract id and hash, and its dotted path. */ +struct ContractEntry { + /** FNV-1a 64 of the declaration's dotted path. */ + uint64_t id; + /** FNV-1a 64 of the declaration's canonical signature. */ + uint64_t hash; + /** The declaration's dotted path (`kv.Store.put`). */ + const char* path; +}; + +/** + * Why the contract table `contract` returns doesn't satisfy `expected`, + * naming the first declaration it lacks or declares differently, or an empty + * string when it has them all. Entries the producer has and `expected` + * doesn't are fine. + */ +template +std::string contract_mismatch(const {{PREFIX}}_contract_entry* (*contract)(size_t*), + const ContractEntry (&expected)[N]) { + size_t len = 0; + const {{PREFIX}}_contract_entry* table = contract(&len); + if (table == nullptr) len = 0; + for (const ContractEntry& want : expected) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < want.id) { + lo = mid + 1; + } else { + hi = mid; + } + } + if (lo == len || table[lo].id != want.id) { + return std::string(want.path) + " is missing from the library"; + } + if (table[lo].hash != want.hash) { + return std::string(want.path) + " changed since these bindings were generated"; + } + } + return std::string(); +} + +{{CONTRACTS}}} // namespace detail /** - * Verifies that the linked {{LIBRARY}} library implements the C ABI revision - * and the module contracts this header was generated against, throwing + * Verifies that the linked {{LIBRARY}} library implements C ABI revision + * {{ABI}} and every declaration this header was generated with, throwing * LoadError naming the first mismatch. The check runs once; every free * function, constructor, and static member calls this before its first * native call, so calling it at startup is optional. */ inline void check_library() { - static const std::string failure = [] { + static const std::string failure = []() -> std::string { uint32_t abi = {{PREFIX}}_abi_version(); if (abi != {{MACRO}}_ABI_VERSION) { return "{{LIBRARY}}: C ABI revision mismatch (header " + std::to_string({{MACRO}}_ABI_VERSION) + ", library " + std::to_string(abi) + ")"; } -{{CHECKSUMS}} return std::string(); +{{CHECKS}} return std::string(); }(); if (!failure.empty()) throw LoadError(failure); } diff --git a/crates/weaveffi-cli/src/targets/cpp/tests.rs b/crates/weaveffi-cli/src/targets/cpp/tests.rs new file mode 100644 index 00000000..ed779231 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/tests.rs @@ -0,0 +1,452 @@ +//! Unit tests for the behavior the snapshots don't make obvious: the +//! contract table the library check embeds, the vtable layout, callback +//! returns and errors, optional callbacks, the trap policy, one codec per +//! composite type, and the ordering that keeps adopted objects from leaking. + +use camino::Utf8Path; +use weaveffi_model::contract; +use weaveffi_model::ir::{ + Api, CallbackInterfaceDef, EnumDef, EnumVariant, ErrorCode, ErrorDomain, Function, + InterfaceDef, Module, Param, StructDef, StructField, TypeRef, +}; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; +use weaveffi_model::ty::Prim; +use weaveffi_model::validate::validate; + +use crate::backend::LanguageBackend; +use crate::targets::cpp::{render_cpp_header, CppConfig, CppGenerator}; + +fn param(name: &str, ty: TypeRef) -> Param { + Param { + name: name.into(), + ty, + doc: None, + } +} + +fn func(name: &str, params: Vec, returns: Option) -> Function { + Function { + name: name.into(), + params, + returns, + doc: None, + throws: false, + r#async: false, + cancellable: false, + deprecated: None, + } +} + +fn throwing(f: Function) -> Function { + Function { throws: true, ..f } +} + +fn field(name: &str, ty: TypeRef) -> StructField { + StructField { + name: name.into(), + ty, + doc: None, + } +} + +fn named(name: &str) -> TypeRef { + TypeRef::Named(name.into()) +} + +fn optional(ty: TypeRef) -> TypeRef { + TypeRef::Optional(Box::new(ty)) +} + +fn list(ty: TypeRef) -> TypeRef { + TypeRef::List(Box::new(ty)) +} + +fn string() -> TypeRef { + TypeRef::Prim(Prim::String) +} + +/// One module with an interface, a record holding objects, a C-style enum, +/// an error domain with a payload-carrying code, and a callback interface +/// whose methods take objects and return every family (one of them +/// `throws`), passed both required and optional. +fn fixture() -> Api { + let store = InterfaceDef { + name: "Store".into(), + doc: None, + deprecated: None, + constructors: vec![func("new", vec![param("path", string())], None)], + methods: vec![ + throwing(func("save", vec![param("contact", named("Contact"))], None)), + func("count", vec![], Some(TypeRef::Prim(Prim::U32))), + ], + statics: vec![], + }; + let listener = CallbackInterfaceDef { + name: "Listener".into(), + doc: None, + deprecated: None, + methods: vec![ + func( + "on_message", + vec![ + param("text", string()), + param("contact", named("Contact")), + param("store", named("Store")), + ], + Some(TypeRef::Prim(Prim::Bool)), + ), + func( + "on_reset", + vec![param("alt", optional(named("Store")))], + None, + ), + func( + "pick", + vec![param("color", named("Color"))], + Some(named("Color")), + ), + throwing(func("label", vec![], Some(string()))), + func("blob", vec![], Some(TypeRef::Prim(Prim::Bytes))), + func("latest", vec![], Some(optional(named("Contact")))), + func("home", vec![], Some(named("Store"))), + func("spare", vec![], Some(optional(named("Store")))), + ], + }; + let variant = |name: &str, value: i32| EnumVariant { + name: name.into(), + value, + doc: None, + fields: vec![], + }; + Api { + version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), + modules: vec![Module { + name: "kv".into(), + doc: None, + functions: vec![ + func("count", vec![], Some(TypeRef::Prim(Prim::U32))), + throwing(func("fetch", vec![], Some(named("Contact")))), + func( + "subscribe", + vec![param("listener", named("Listener"))], + None, + ), + func( + "maybe_subscribe", + vec![param("listener", optional(named("Listener")))], + None, + ), + func( + "first", + vec![param("stores", list(named("Store")))], + Some(named("Store")), + ), + ], + interfaces: vec![store], + callback_interfaces: vec![listener], + structs: vec![StructDef { + name: "Contact".into(), + doc: None, + deprecated: None, + fields: vec![ + field("name", string()), + field("store", named("Store")), + field("mirrors", list(named("Store"))), + field("primary", optional(named("Store"))), + ], + }], + enums: vec![EnumDef { + name: "Color".into(), + doc: None, + deprecated: None, + variants: vec![variant("Red", 0), variant("Blue", 1)], + }], + errors: Some(ErrorDomain { + name: "KvError".into(), + codes: vec![ + ErrorCode { + name: "NOT_FOUND".into(), + code: 1, + message: "not found".into(), + doc: None, + fields: vec![], + }, + ErrorCode { + name: "TOO_BIG".into(), + code: 2, + message: "too big".into(), + doc: None, + fields: vec![field("limit", TypeRef::Prim(Prim::I64))], + }, + ], + }), + modules: vec![], + }], + } +} + +fn render() -> (Model, String) { + let model = validate(&fixture(), &Identity::named("api"), None).expect("fixture validates"); + let header = render_cpp_header(&model, "api", "api.h", "api.hpp"); + (model, header) +} + +fn assert_contains(header: &str, needle: &str) { + assert!( + header.contains(needle), + "expected the header to contain {needle:?}\n---\n{header}" + ); +} + +/// The text of the function or block that starts with `start`, up to the +/// first line that closes it at the same indentation. +fn block<'a>(header: &'a str, start: &str) -> &'a str { + let at = header + .find(start) + .unwrap_or_else(|| panic!("missing {start:?}")); + let line_start = header[..at].rfind('\n').map_or(0, |i| i + 1); + let indent = &header[line_start..at]; + let close = format!("\n{indent}}}"); + let end = header[at..] + .find(&close) + .unwrap_or_else(|| panic!("unterminated {start:?}")); + &header[at..at + end + close.len()] +} + +#[test] +fn library_check_embeds_every_contract_entry() { + let (model, h) = render(); + let root = model.roots().next().expect("one root"); + let entries = contract::entries(&model, root); + assert!(!entries.is_empty()); + for e in &entries { + assert_contains( + &h, + &format!( + "{{{:#018x}ull, {:#018x}ull, \"{}\"}},", + e.id, e.hash, e.path + ), + ); + } + assert_contains( + &h, + "if (std::string why = detail::contract_mismatch(api_kv_contract, detail::kv_contract); !why.empty()) {", + ); + assert_contains(&h, "\" is missing from the library\""); + assert_contains(&h, "\" changed since these bindings were generated\""); + assert!(!h.contains("checksum"), "the revision 3 checksum is gone"); +} + +#[test] +fn vtable_starts_with_the_header_then_methods_in_order() { + let (_, h) = render(); + let vtable = block(&h, "static const api_kv_Listener_vtable vtable = {"); + let order: Vec = [ + "static_cast(sizeof(api_kv_Listener_vtable)),", + "0,", + "&Listener_trampolines::free_ctx,", + "&Listener_trampolines::on_message,", + "&Listener_trampolines::on_reset,", + "&Listener_trampolines::pick,", + "&Listener_trampolines::label,", + "&Listener_trampolines::blob,", + "&Listener_trampolines::latest,", + "&Listener_trampolines::home,", + "&Listener_trampolines::spare,", + ] + .iter() + .map(|entry| { + vtable + .find(entry) + .unwrap_or_else(|| panic!("missing {entry}")) + }) + .collect(); + assert!(order.windows(2).all(|w| w[0] < w[1]), "{vtable}"); +} + +#[test] +fn callback_returns_cross_in_every_family() { + let (_, h) = render(); + // Strings, bytes, and buffers go back through the out slots as an + // `alloc` run the producer adopts. + let label = block(&h, "static void label("); + assert_contains( + label, + "uint8_t** out_ptr, size_t* out_len, api_error* out_err) {", + ); + assert_contains( + label, + "detail::hand_over(ret.data(), ret.size(), out_ptr, out_len);", + ); + let blob = block(&h, "static void blob("); + assert_contains(blob, "std::vector ret = impl.blob();"); + let latest = block(&h, "static void latest("); + assert_contains(latest, "detail::write_opt_Contact(ret_buf, ret);"); + assert_contains( + latest, + "detail::hand_over(ret_buf.data(), ret_buf.size(), out_ptr, out_len);", + ); + assert_contains(&h, "inline void hand_over("); + assert_contains(&h, "run = api_alloc(len);"); + // An object return hands over a fresh reference; `I?` may be null. + let home = block(&h, "static api_kv_Store* home("); + assert_contains(home, "return impl.home().clone_handle();"); + assert_contains(home, "return nullptr;"); + let spare = block(&h, "static api_kv_Store* spare("); + assert_contains( + spare, + "return ret.has_value() ? ret->clone_handle() : nullptr;", + ); + // A C-style enum converts both ways. + assert_contains( + &h, + "return static_cast(static_cast(impl.pick(static_cast(static_cast(color)))));", + ); + assert_contains(&h, "virtual std::optional latest() = 0;"); + assert_contains(&h, "virtual Store home() = 0;"); +} + +#[test] +fn only_throwing_callback_methods_report_the_domain() { + let (_, h) = render(); + let label = block(&h, "static void label("); + assert_contains(label, "} catch (const KvError& e) {"); + assert_contains(label, "detail::report_kv_error(e, out_err);"); + let blob = block(&h, "static void blob("); + assert!(!blob.contains("KvError"), "{blob}"); + assert_contains(blob, "api_error_set(out_err, -4, e.what());"); + + // A declared code goes back with its fields as the payload; anything + // else is a callback failure. + let report = block(&h, "inline void report_kv_error("); + assert_contains( + report, + "case 1: api_error_set(out_err, 1, e.what()); return;", + ); + assert_contains( + report, + "const auto* typed = dynamic_cast(&e);", + ); + assert_contains(report, "payload.write_i64(typed->limit);"); + assert_contains( + report, + "api_error_set_payload(out_err, payload.data(), payload.size());", + ); + assert!(report + .trim_end() + .ends_with("api_error_set(out_err, -4, e.what());\n}")); +} + +#[test] +fn optional_callbacks_pass_a_null_vtable() { + let (_, h) = render(); + let maybe = block(&h, "inline void maybe_subscribe("); + assert_contains(maybe, "(std::shared_ptr listener) {"); + assert_contains( + maybe, + "const auto* listener_vtable = listener ? &detail::Listener_vtable() : nullptr;", + ); + assert_contains( + maybe, + "api_kv_maybe_subscribe(static_cast(listener_ctx.release()), listener_vtable, &err);", + ); + // A required one refuses an empty pointer before the call. + let required = block(&h, "inline void subscribe("); + assert_contains( + required, + "if (!listener) throw std::invalid_argument(\"listener: null callback interface\");", + ); +} + +#[test] +fn calls_without_errors_trap_and_throwing_calls_use_the_domain() { + let (_, h) = render(); + assert_contains(block(&h, "inline uint32_t count("), "detail::check(err);"); + assert_contains(block(&h, "inline Contact fetch("), "detail::check_kv(err);"); + assert_contains( + block(&h, "inline void Store::save("), + "detail::check_kv(err);", + ); + assert_contains(&h, "class InternalError : public std::runtime_error {"); + assert_contains( + block(&h, "inline void check(api_error& err) {"), + "make_internal_error(err.code, error_message(err));", + ); + // A throwing call's runtime codes are the root Error, and -5 cancels. + let make = block(&h, "inline std::exception_ptr make_kv_error("); + assert_contains( + make, + "if (code == -5) return std::make_exception_ptr(Cancelled(message));", + ); + assert_contains( + make, + "if (code < 0) return std::make_exception_ptr(Error(code, message));", + ); +} + +#[test] +fn each_composite_has_one_codec() { + let (_, h) = render(); + for def in [ + "inline std::vector read_list_Store(BufferReader& r) {", + "inline void write_list_Store(BufferWriter& w, const std::vector& v) {", + "inline std::optional read_opt_Store(BufferReader& r) {", + "inline std::optional read_opt_Contact(BufferReader& r) {", + ] { + assert_eq!(h.matches(def).count(), 1, "{def}"); + } + // Call sites and record codecs delegate instead of looping inline. + assert_contains( + block(&h, "inline Store first("), + "detail::write_list_Store(stores_buf, stores);", + ); + let record = block(&h, "inline Contact read_Contact(BufferReader& r) {"); + assert_contains(record, "detail::read_list_Store(r),"); + assert!(!record.contains("for ("), "{record}"); +} + +#[test] +fn trampolines_adopt_objects_before_anything_can_throw() { + let (_, h) = render(); + let on_message = block(&h, "static bool on_message("); + let adopt = on_message + .find("Store store_arg(adopt, store);") + .expect("adopts the object argument"); + let attempt = on_message.find("try {").expect("try"); + assert!(adopt < attempt, "{on_message}"); + let on_reset = block(&h, "static void on_reset("); + assert_contains(on_reset, "if (alt != nullptr) alt_arg.emplace(adopt, alt);"); +} + +#[test] +fn identity_names_every_file_and_config_overrides_win() { + let model = + validate(&fixture(), &Identity::named("kv-store"), None).expect("fixture validates"); + let out = Utf8Path::new("/out"); + let files = CppGenerator.files(&model, out, &CppConfig::default()); + let paths: Vec = files.iter().map(|f| f.path.to_string()).collect(); + assert_eq!( + paths, + [ + "/out/cpp/kv_store.h", + "/out/cpp/kv_store.hpp", + "/out/cpp/CMakeLists.txt", + "/out/cpp/README.md", + ] + ); + let hpp = &files[1].contents; + assert_contains(hpp, "#include \"kv_store.h\""); + assert_contains(hpp, "namespace kv_store {"); + assert_contains(hpp, "return \"kv_store: \" + why;"); + + let config = CppConfig { + name: Some("acme::kv".into()), + header_name: Some("acme_kv.hpp".into()), + ..CppConfig::default() + }; + let files = CppGenerator.files(&model, out, &config); + assert_eq!(files[1].path, "/out/cpp/acme_kv.hpp"); + assert_contains(&files[1].contents, "namespace acme::kv {"); + assert_contains(&files[1].contents, "#include \"kv_store.h\""); +} diff --git a/crates/weaveffi-cli/src/targets/cpp/types.rs b/crates/weaveffi-cli/src/targets/cpp/types.rs new file mode 100644 index 00000000..e6908148 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/cpp/types.rs @@ -0,0 +1,186 @@ +//! C++ spellings: identifier escaping, name casing, namespace paths, and the +//! IR-to-C++ type mapping. + +use crate::cabi::c_param_name; +use crate::lang::{self, CPP_KEYWORDS}; +use heck::ToSnakeCase; +use weaveffi_model::abi::AbiParam; +use weaveffi_model::errors; +use weaveffi_model::model::ModuleBinding; +use weaveffi_model::ty::{Prim, Ty}; + +/// Idiomatic C++ exception class name for an error code: PascalCase with a +/// single `Error` suffix (`KEY_NOT_FOUND` becomes `KeyNotFoundError`). +pub(crate) fn cpp_error_class(name: &str) -> String { + errors::type_name(name, "Error") +} + +/// C++ reserved words the shared [`CPP_KEYWORDS`] table doesn't carry: the +/// alternative operator tokens spelled with `_eq`, the extended character +/// types, the two remaining cast keywords, and `thread_local`. Sorted for +/// binary search; kept disjoint from the shared table so each name is listed +/// exactly once. +pub(crate) const CPP_EXTRA_KEYWORDS: &[&str] = &[ + "and_eq", + "char16_t", + "char32_t", + "char8_t", + "const_cast", + "not_eq", + "or_eq", + "reinterpret_cast", + "thread_local", + "wchar_t", + "xor_eq", +]; + +/// Escape an identifier that collides with a C++ keyword by appending an +/// underscore (`delete` becomes `delete_`); other names pass through. +pub(crate) fn cpp_ident(name: &str) -> String { + if lang::is_reserved(name, CPP_EXTRA_KEYWORDS) { + return format!("{name}_"); + } + lang::escape_ident(name, CPP_KEYWORDS) +} + +/// The C++ spelling of a callable name: snake_case with C++ keyword +/// collisions escaped. +pub(crate) fn cpp_fn_name(name: &str) -> String { + cpp_ident(&name.to_snake_case()) +} + +/// Members every interface wrapper class declares besides its special +/// members: the `raw_type` alias and the `handle()` and `clone_handle()` +/// readers. (The private `raw_` field can't collide: a snake-cased name +/// only ends in `_` once escaped, and `raw` is reserved nowhere.) +const OBJECT_MEMBERS: &[&str] = &["clone_handle", "handle", "raw_type"]; + +/// The C++ spelling of an interface member (a factory constructor, method, +/// or static): [`cpp_fn_name`], with a trailing underscore when it would +/// collide with a member the wrapper class declares (`handle` is +/// `handle_`). +pub(crate) fn cpp_member_name(name: &str) -> String { + lang::escape_member(&cpp_fn_name(name), OBJECT_MEMBERS) +} + +/// The nested C++ namespace path for a module: each IDL segment converted to +/// snake case and keyword-escaped, joined with `::` (`kv.stats` becomes +/// `kv::stats`). +pub(crate) fn cpp_namespace_path(module: &ModuleBinding) -> String { + module + .segments + .iter() + .map(|s| cpp_ident(&s.to_snake_case())) + .collect::>() + .join("::") +} + +/// The C++ spelling of one ABI slot name: the IDL-chosen identifier with the +/// same keyword escape the C declarations apply, so trampoline and lambda +/// parameter lists match the `extern "C"` prototypes. +pub(crate) fn slot_name(p: &AbiParam) -> String { + c_param_name(&p.name) +} + +/// Renders ABI parameter slots to C declarations (` `), the form +/// used inside async completion lambdas and callback-interface trampolines. +pub(crate) fn render_param_decls(params: &[AbiParam], prefix: &str) -> Vec { + params + .iter() + .map(|p| format!("{} {}", p.ty.render_c(prefix), slot_name(p))) + .collect() +} + +/// The `detail` accessor returning the process-wide static vtable for a +/// callback interface. +pub(crate) fn vtable_accessor(name: &str) -> String { + format!("{name}_vtable") +} + +/// The `detail` struct holding a callback interface's trampolines. +pub(crate) fn trampoline_struct(name: &str) -> String { + format!("{name}_trampolines") +} + +/// The idiomatic C++ spelling of an IR type. Interfaces map to their RAII +/// wrapper class and `Interface?` to `std::optional` of it; a callback +/// interface (bare or optional) is a `std::shared_ptr` of the abstract class +/// the consumer implements, empty for none. +pub(crate) fn cpp_type(ty: &Ty) -> String { + match ty { + Ty::Prim(p) => match p { + Prim::I8 => "int8_t", + Prim::I16 => "int16_t", + Prim::I32 => "int32_t", + Prim::I64 => "int64_t", + Prim::U8 => "uint8_t", + Prim::U16 => "uint16_t", + Prim::U32 => "uint32_t", + Prim::U64 => "uint64_t", + Prim::F32 => "float", + Prim::F64 => "double", + Prim::Bool => "bool", + Prim::String => "std::string", + Prim::Bytes => "std::vector", + } + .to_string(), + Ty::Record(n) | Ty::RichEnum(n) | Ty::Enum(n) | Ty::Interface(n) => n.clone(), + Ty::CallbackInterface(n) => format!("std::shared_ptr<{n}>"), + Ty::Optional(inner) if matches!(inner.as_ref(), Ty::CallbackInterface(_)) => { + cpp_type(inner) + } + Ty::Optional(inner) => format!("std::optional<{}>", cpp_type(inner)), + Ty::List(inner) => format!("std::vector<{}>", cpp_type(inner)), + Ty::Map(k, v) => format!("std::unordered_map<{}, {}>", cpp_type(k), cpp_type(v)), + Ty::Iterator(_) => unreachable!("iterator returns render as range classes"), + } +} + +/// One C++ parameter declaration (` `) for a wrapper signature. +/// Strings borrow as `std::string_view` (the ABI passes a pointer and a +/// length, so interior NULs survive); heavier types borrow by const +/// reference; scalars and enums pass by value. A callback interface is a +/// `std::shared_ptr` by value, which the wrapper moves into the box it hands +/// the producer as `ctx`. +pub(crate) fn cpp_param_decl(ty: &Ty, name: &str) -> String { + match ty { + Ty::Prim(Prim::String) => format!("std::string_view {name}"), + Ty::Prim(Prim::Bytes) + | Ty::Record(_) + | Ty::RichEnum(_) + | Ty::Interface(_) + | Ty::List(_) + | Ty::Map(_, _) => format!("const {}& {name}", cpp_type(ty)), + Ty::Optional(inner) if !matches!(inner.as_ref(), Ty::CallbackInterface(_)) => { + format!("const {}& {name}", cpp_type(ty)) + } + _ => format!("{} {name}", cpp_type(ty)), + } +} + +/// One C++ parameter declaration for a callback-interface method the consumer +/// implements. Strings arrive as a `std::string_view` valid for the call; +/// bytes and buffered values by const reference to the trampoline's decoded +/// copy; objects transfer one strong reference, so they arrive by value as +/// the wrapper (or `std::optional` of it) the implementation now owns. +pub(crate) fn cpp_cb_param_decl(ty: &Ty, name: &str) -> String { + if ty.interface_name().is_some() { + format!("{} {name}", cpp_type(ty)) + } else { + cpp_param_decl(ty, name) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn interface_members_avoid_the_wrapper_members() { + assert_eq!(cpp_member_name("handle"), "handle_"); + assert_eq!(cpp_member_name("clone_handle"), "clone_handle_"); + assert_eq!(cpp_member_name("raw_type"), "raw_type_"); + assert_eq!(cpp_member_name("delete"), "delete_"); + assert_eq!(cpp_member_name("close"), "close"); + } +} diff --git a/crates/weaveffi-gen/src/targets/dart/callbacks.rs b/crates/weaveffi-cli/src/targets/dart/callbacks.rs similarity index 52% rename from crates/weaveffi-gen/src/targets/dart/callbacks.rs rename to crates/weaveffi-cli/src/targets/dart/callbacks.rs index b9f0be7c..b86a6caf 100644 --- a/crates/weaveffi-gen/src/targets/dart/callbacks.rs +++ b/crates/weaveffi-cli/src/targets/dart/callbacks.rs @@ -1,25 +1,31 @@ //! Callback-interface rendering: the abstract class a consumer implements, -//! the `Struct` mirroring the C vtable, one entry per method, the dispatcher -//! that delivers forwarded void methods, and the one process-wide vtable. +//! the `Struct` mirroring the C vtable (the `size`/`flags`/`free` header, +//! then one entry per method), the vtable entries, the dispatcher that +//! delivers forwarded void methods, and the vtable itself. //! //! A method that returns a value is a `NativeCallable.isolateLocal` //! trampoline: it runs synchronously on the isolate's thread, receives its -//! borrowed arguments in place, adopts object arguments, and reports a thrown -//! exception through `{prefix}_error_set` with the foreign code instead of -//! unwinding. A void method is a `NativeCallable.isolateGroupBound` -//! forwarder: it may run on any thread, copies its arguments into a message -//! for the isolate, and returns at once; the dispatcher then runs the -//! implementation on the event loop. +//! borrowed arguments in place, adopts object arguments, and hands its return +//! to the producer (by value, as a fresh object reference, or as a +//! `{prefix}_alloc` run in the out slots). A thrown exception never unwinds: +//! a `throws` method reports its domain error's code and payload, and +//! anything else is reported as `-4`. A void method is a +//! `NativeCallable.isolateGroupBound` forwarder: it may run on any thread, +//! copies its arguments into a message for the isolate, and returns at once; +//! the dispatcher then runs the implementation on the event loop. use crate::codegen::CodeWriter; use heck::ToUpperCamelCase; -use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding, Ty}; -use weaveffi_model::plan::ArgPass; +use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding}; +use weaveffi_model::plan::{ArgPass, RetPass}; +use weaveffi_model::ty::{Prim, Ty}; -use crate::targets::dart::calls::{adopt_expr, decoder}; +use crate::targets::dart::calls::adopt_expr; +use crate::targets::dart::codec::{pack_fn, unpack_fn}; use crate::targets::dart::docs::{write_deprecated, write_doc}; +use crate::targets::dart::entities::payload_fn; use crate::targets::dart::types::{ - dart_class, dart_ident, dart_type, default_literal, ffi_type, ffi_typedef, object_class, + dart_class, dart_ident, dart_type, ffi_type, ffi_typedef, object_class, zero_literal, }; /// The private global holding a callback interface's vtable. @@ -49,26 +55,34 @@ fn lower_first(s: &str) -> String { }) } -/// Render one callback interface. -pub(crate) fn render_callback_interface(out: &mut String, cb: &CallbackInterfaceBinding) { +/// Render one callback interface; `exception` is the domain exception +/// class in scope for its module, which its `throws` methods report. +pub(crate) fn render_callback_interface( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + exception: Option<&str>, +) { let class = dart_class(&cb.name); - let mut w = CodeWriter::two_space(); - render_abstract_class(&mut w, cb, &class); - render_vtable_struct(&mut w, cb, &class); + render_abstract_class(w, cb, &class, exception); + render_vtable_struct(w, cb, &class); for m in &cb.methods { if m.ret.is_some() { - render_trampoline(&mut w, cb, m, &class); + render_trampoline(w, cb, m, &class, exception.filter(|_| m.throws)); } else { - render_forwarder(&mut w, cb, m, &class); + render_forwarder(w, cb, m, &class); } } - render_dispatch(&mut w, cb, &class); - render_vtable(&mut w, cb, &class); - out.push_str(&w.finish()); + render_dispatch(w, cb, &class); + render_vtable(w, cb, &class); } /// The class a consumer extends or implements. -fn render_abstract_class(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class: &str) { +fn render_abstract_class( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + class: &str, + exception: Option<&str>, +) { w.blank(); write_doc(w, &cb.doc); if cb.doc.is_some() { @@ -82,11 +96,13 @@ fn render_abstract_class(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, clas w.line("///"); w.line("/// Methods that return a value run synchronously and must be called by"); w.line("/// the producer on this isolate's thread during a call from Dart; a"); - w.line("/// thrown exception fails that call with [NativeException.foreignCode]."); - w.line("/// Void methods may be called from any thread: they're delivered"); - w.line("/// asynchronously on this isolate's event loop, in the zone that passed"); - w.line("/// the instance, where a thrown exception is an uncaught error of that"); - w.line("/// zone. Object arguments are owned by the implementation."); + w.line("/// thrown exception fails that call with [NativeException.foreignCode]"); + w.line("/// (or, from a method that throws, the domain exception's code and"); + w.line("/// fields). Void methods may be called from any thread: they're"); + w.line("/// delivered asynchronously on this isolate's event loop, in the zone"); + w.line("/// that passed the instance, where a thrown exception is an uncaught"); + w.line("/// error of that zone. Object arguments are owned by the"); + w.line("/// implementation."); write_deprecated(w, &cb.deprecated); w.block(format!("abstract class {class} {{"), "}", |w| { for (i, m) in cb.methods.iter().enumerate() { @@ -94,6 +110,14 @@ fn render_abstract_class(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, clas w.blank(); } write_doc(w, &m.doc); + if let (true, Some(exc), Some(_)) = (m.throws, exception, &m.ret) { + if m.doc.is_some() { + w.line("///"); + } + w.line(format!( + "/// Throw a [{exc}] to fail the producer's call with it." + )); + } write_deprecated(w, &m.deprecated); let params: Vec = m .params @@ -115,8 +139,8 @@ fn entry_typedef(cb: &CallbackInterfaceBinding, m: &CallbackMethodBinding) -> St ffi_typedef(&format!("{}_{}", cb.c_tag, m.name)) } -/// The `Struct` whose layout is the C vtable: one function pointer per -/// method in declaration order, then `free`. +/// The `Struct` whose layout is the C vtable: the `size`, `flags`, and +/// `free` header, then one function pointer per method in declaration order. fn render_vtable_struct(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class: &str) { w.blank(); for m in &cb.methods { @@ -134,6 +158,11 @@ fn render_vtable_struct(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class format!("final class {} extends Struct {{", vtable_struct(class)), "}", |w| { + w.line("@Uint32()"); + w.line("external int size;"); + w.line("@Uint32()"); + w.line("external int flags;"); + w.line("external Pointer)>> free;"); for m in &cb.methods { w.line(format!( "external Pointer> {};", @@ -141,96 +170,160 @@ fn render_vtable_struct(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class dart_ident(&m.name) )); } - w.line("external Pointer)>> free;"); }, ); } -/// The `(dart type, dart name)` pairs of one vtable entry's parameters. -fn entry_params(m: &CallbackMethodBinding) -> Vec { +/// The Dart parameter list of one vtable entry. +fn entry_params(m: &CallbackMethodBinding) -> String { m.abi_params .iter() .map(|p| format!("{} {}", ffi_type(&p.ty).1, dart_ident(&p.name))) - .collect() + .collect::>() + .join(", ") +} + +/// The Dart name of one of an entry's trailing slots (`out_ptr`, `out_len`, +/// `out_err`), counted from the end. +fn trailing_slot(m: &CallbackMethodBinding, from_end: usize) -> String { + dart_ident(&m.abi_params[m.abi_params.len() - from_end].name) } /// A value-returning method's `isolateLocal` trampoline: adopt object /// arguments first (so a later failure still leaves each reference owned by /// a finalizable wrapper), receive the borrowed ones in place, call the -/// implementation, and return its value. A thrown exception is reported -/// through `_foreignError` and the zero value is returned. +/// implementation, and hand its value to the producer. A thrown exception is +/// reported through `out_err` and the zero value is returned; `exception` is +/// the domain the method reports when it's declared `throws`. fn render_trampoline( w: &mut CodeWriter, cb: &CallbackInterfaceBinding, m: &CallbackMethodBinding, class: &str, + exception: Option<&str>, ) { let ret = m.ret.as_ref().expect("value-returning method"); + let pass = RetPass::of(Some(ret)); let (_, dart_ret) = ffi_type(&m.abi_ret); + let out_err = trailing_slot(m, 1); + let zero = match &pass { + RetPass::Direct => Some(zero_literal(ret)), + RetPass::Object { .. } => Some("nullptr"), + _ => None, + }; w.blank(); w.line(format!("// Vtable entry `{}.{}`.", cb.vtable_tag, m.name)); w.block( format!( "{dart_ret} {}({}) {{", entry_fn(class, &m.name), - entry_params(m).join(", ") + entry_params(m) ), "}", |w| { w.line("try {"); w.scope(|w| { - let mut args = Vec::new(); - for p in &m.params { - let base = dart_ident(&p.name); - let slot = dart_ident(&p.abi[0].name); - args.push(match p.arg_pass() { - ArgPass::Direct { .. } => match &p.ty { - Ty::Enum(n) => format!("{}.fromValue({slot})", dart_class(n)), - _ => slot, - }, - ArgPass::String { len, .. } => { - format!("_readString({slot}, {})", dart_ident(&len.name)) - } - ArgPass::Bytes { len, .. } => { - format!("_copyBytes({slot}, {})", dart_ident(&len.name)) - } - ArgPass::Buffer { len, .. } => format!( - "_decode(_copyBytes({slot}, {}), {})", - dart_ident(&len.name), - decoder(&p.ty) - ), - ArgPass::Object { nullable, .. } => { - w.line(format!( - "final _{base} = {};", - adopt_expr(&slot, &p.ty, nullable) - )); - format!("_{base}") - } - ArgPass::Callback { .. } => { - unreachable!("callback methods never take callback interfaces") - } - }); + // Objects first: nothing before them can throw. + let is_object = |p: &ParamBinding| matches!(p.arg_pass(), ArgPass::Object { .. }); + let mut args = vec![String::new(); m.params.len()]; + for (i, p) in m.params.iter().enumerate().filter(|(_, p)| is_object(p)) { + args[i] = receive_arg(w, p); + } + for (i, p) in m.params.iter().enumerate().filter(|(_, p)| !is_object(p)) { + args[i] = receive_arg(w, p); } let call = format!( "(_callbackTarget(ctx) as {class}).{}({})", dart_ident(&m.name), args.join(", ") ); - match ret { - Ty::Enum(_) => w.line(format!("return {call}.value;")), - _ => w.line(format!("return {call};")), - }; + if matches!(pass, RetPass::Direct) { + let value = if matches!(ret, Ty::Enum(_)) { + ".value" + } else { + "" + }; + w.line(format!("return {call}{value};")); + return; + } + w.line(format!("final result = {call};")); + // The out slots of a string, bytes, or buffer return. + let out = || format!("{}, {}", trailing_slot(m, 3), trailing_slot(m, 2)); + w.line(match &pass { + // A fresh strong reference the producer adopts. + RetPass::Object { nullable: false } => "return result._cloneRef();".into(), + RetPass::Object { nullable: true } => { + "return result?._cloneRef() ?? nullptr;".into() + } + RetPass::String => { + format!("_handOver(utf8.encode(result), {});", out()) + } + RetPass::Bytes => format!("_handOver(result, {});", out()), + RetPass::Buffer => { + format!("_handOver(_encode(result, {}), {});", pack_fn(ret), out()) + } + RetPass::Void | RetPass::Direct => unreachable!("handled above"), + }); }); + if let Some(exc) = exception { + w.line(format!("}} on {exc} catch (e) {{")); + w.scope(|w| { + w.line(format!( + "_failCallback({out_err}, e.code, e.message, {}(e));", + payload_fn(exc) + )); + }); + } w.line("} catch (e) {"); w.scope(|w| { - w.line("_foreignError(outErr, e);"); - w.line(format!("return {};", default_literal(ret))); + w.line(format!("_foreignError({out_err}, e);")); }); w.line("}"); + if let Some(zero) = zero { + w.line(format!("return {zero};")); + } }, ); } +/// The expression receiving one parameter inside a trampoline: borrowed runs +/// are copied, buffers decoded, and objects adopted. Objects and buffers +/// (which may carry object tokens) are received into locals declared first, +/// so each reference is owned even if a later step throws. +fn receive_arg(w: &mut CodeWriter, p: &ParamBinding) -> String { + let slot = dart_ident(&p.abi[0].name); + match p.arg_pass() { + ArgPass::Direct { .. } => match &p.ty { + Ty::Enum(n) => format!("{}.fromValue({slot})", dart_class(n)), + _ => slot, + }, + ArgPass::String { len, .. } => format!("_readString({slot}, {})", dart_ident(&len.name)), + ArgPass::Bytes { len, .. } => format!("_copyBytes({slot}, {})", dart_ident(&len.name)), + ArgPass::Buffer { len, .. } => { + let decode = format!( + "_decode(_copyBytes({slot}, {}), {})", + dart_ident(&len.name), + unpack_fn(&p.ty) + ); + // Any object tokens inside are adopted references. + let local = format!("_{}", dart_ident(&p.name)); + w.line(format!("final {local} = {decode};")); + local + } + ArgPass::Object { nullable, .. } => { + let local = format!("_{}", dart_ident(&p.name)); + w.line(format!( + "final {local} = {};", + adopt_expr(&slot, &p.ty, nullable) + )); + local + } + ArgPass::Callback { .. } => { + unreachable!("callback methods never take callback interfaces") + } + } +} + /// A void method's `isolateGroupBound` forwarder: it runs outside the /// isolate, so it only copies its slots into a message (borrowed bytes are /// copied by the post) and touches no Dart globals. @@ -251,11 +344,7 @@ fn render_forwarder( cb.vtable_tag, m.name )); w.block( - format!( - "void {}({}) {{", - entry_fn(class, &m.name), - entry_params(m).join(", ") - ), + format!("void {}({}) {{", entry_fn(class, &m.name), entry_params(m)), "}", |w| { w.line(format!( @@ -268,8 +357,10 @@ fn render_forwarder( let slot = dart_ident(&p.abi[0].name); w.line(match p.arg_pass() { ArgPass::Direct { .. } => match &p.ty { - Ty::Bool => format!("..boolean({slot})"), - Ty::F32 | Ty::F64 => format!("..float64({slot})"), + Ty::Prim(Prim::Bool) => format!("..boolean({slot})"), + Ty::Prim(Prim::F32 | Prim::F64) => { + format!("..float64({slot})") + } _ => format!("..int64({slot})"), }, ArgPass::String { len, .. } @@ -302,6 +393,9 @@ fn render_dispatch(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class: &st "}", |w| { if cb.methods.iter().all(|m| m.ret.is_some()) { + w.line(format!( + "// Every {class} method returns a value: nothing is forwarded." + )); return; } w.line(format!("final target = impl as {class};")); @@ -324,9 +418,8 @@ fn render_dispatch(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class: &st for (i, p) in params.iter().filter(|(_, p)| !is_object(p)) { message_arg(w, p, i + 2); } - let args: Vec = (0..m.params.len()) - .map(|i| format!("_a{}", i + 2)) - .collect(); + let args: Vec = + (0..m.params.len()).map(|i| format!("a{}", i + 2)).collect(); w.line(format!( "target.{}({});", dart_ident(&m.name), @@ -339,10 +432,10 @@ fn render_dispatch(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class: &st ); } -/// Decode forwarded argument `i` of `message` into the local `_a{i}`. +/// Decode forwarded argument `i` of `message` into the local `a{i}`. fn message_arg(w: &mut CodeWriter, p: &ParamBinding, i: usize) { let field = format!("message[{i}]!"); - let local = format!("_a{i}"); + let local = format!("a{i}"); let value = match p.arg_pass() { ArgPass::Direct { .. } => match &p.ty { Ty::Enum(n) => format!("{}.fromValue({field} as int)", dart_class(n)), @@ -351,21 +444,19 @@ fn message_arg(w: &mut CodeWriter, p: &ParamBinding, i: usize) { ArgPass::String { .. } => format!("utf8.decode({field} as Uint8List)"), ArgPass::Bytes { .. } => format!("{field} as Uint8List"), ArgPass::Buffer { .. } => { - format!("_decode({field} as Uint8List, {})", decoder(&p.ty)) + format!("_decode({field} as Uint8List, {})", unpack_fn(&p.ty)) } ArgPass::Object { nullable, .. } => { let address = format!("{local}Address"); w.line(format!("final {address} = {field} as int;")); + let adopt = format!( + "{}._(Pointer.fromAddress({address}))", + object_class(&p.ty) + ); if nullable { - format!( - "{address} == 0 ? null : {}._(Pointer.fromAddress({address}))", - object_class(&p.ty) - ) + format!("{address} == 0 ? null : {adopt}") } else { - format!( - "{}._(Pointer.fromAddress({address}))", - object_class(&p.ty) - ) + adopt } } ArgPass::Callback { .. } => { @@ -375,7 +466,7 @@ fn message_arg(w: &mut CodeWriter, p: &ParamBinding, i: usize) { w.line(format!("final {local} = {value};")); } -/// The one process-wide vtable every instance passed to the producer +/// The vtable every instance passed to the producer from this isolate /// shares; the per-instance state travels in `ctx`. fn render_vtable(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class: &str) { let strukt = vtable_struct(class); @@ -388,19 +479,25 @@ fn render_vtable(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, class: &str) w.line("vtable.ref"); w.scope(|w| { w.scope(|w| { - for m in &cb.methods { + w.line(format!("..size = sizeOf<{strukt}>()")); + w.line("..flags = 0"); + w.line("..free = _callbackFree"); + for (i, m) in cb.methods.iter().enumerate() { let td = entry_typedef(cb, m); let entry = entry_fn(class, &m.name); - let callable = match &m.ret { - Some(ret) => format!( + let callable = match RetPass::of(m.ret.as_ref()) { + RetPass::Void => { + format!("NativeCallable<{td}>.isolateGroupBound({entry})") + } + RetPass::Direct => format!( "NativeCallable<{td}>.isolateLocal({entry}, exceptionalReturn: {})", - default_literal(ret) + zero_literal(m.ret.as_ref().expect("direct return")) ), - None => format!("NativeCallable<{td}>.isolateGroupBound({entry})"), + _ => format!("NativeCallable<{td}>.isolateLocal({entry})"), }; - w.line(format!("..{} = _pin({callable})", dart_ident(&m.name))); + let end = if i + 1 == cb.methods.len() { ";" } else { "" }; + w.line(format!("..{} = _pin({callable}){end}", dart_ident(&m.name))); } - w.line("..free = _callbackFree;"); }); }); w.line("return vtable;"); diff --git a/crates/weaveffi-gen/src/targets/dart/calls.rs b/crates/weaveffi-cli/src/targets/dart/calls.rs similarity index 65% rename from crates/weaveffi-gen/src/targets/dart/calls.rs rename to crates/weaveffi-cli/src/targets/dart/calls.rs index 5f620851..c898f980 100644 --- a/crates/weaveffi-gen/src/targets/dart/calls.rs +++ b/crates/weaveffi-cli/src/targets/dart/calls.rs @@ -3,54 +3,58 @@ //! //! FFI signatures come straight from the model's lowered [`AbiFn`]s, so they //! match the C header slot for slot. Parameter marshalling dispatches on the -//! shared [`ArgPass`] contract and result handling on [`RetPass`]. +//! shared [`ArgPass`] contract, result handling on [`RetPass`], and error +//! handling on [`ErrorStrategy`]. use crate::codegen::CodeWriter; use weaveffi_model::abi::{AbiParam, CType}; -use weaveffi_model::model::{ - AbiFn, AsyncBinding, CallShape, FnBinding, IteratorBinding, ModuleBinding, ParamBinding, Ty, -}; -use weaveffi_model::plan::{self, ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::model::{AbiFn, AsyncBinding, CallShape, FnBinding, IteratorBinding}; +use weaveffi_model::plan::{ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::ty::Ty; use crate::targets::dart::callbacks::{dispatch_fn, vtable_var}; -use crate::targets::dart::codec::{read_expr, unpack_fn, write_stmts}; +use crate::targets::dart::codec::{pack_fn, unpack_fn}; use crate::targets::dart::docs::emit_wrapper_doc; -use crate::targets::dart::entities::dart_exception_name; use crate::targets::dart::types::{ - dart_class, dart_fn_name, dart_ident, dart_type, ffi_type, ffi_typedef, ffi_var, object_class, - pointee_ffi, + dart_class, dart_ident, dart_type, ffi_type, ffi_typedef, ffi_var, object_class, }; -/// Error-reporting context for one wrapper: the domain mapper its error -/// checks route through. +/// Error-reporting context for one wrapper: the mapper its error checks +/// route through. /// /// The split follows [`ErrorStrategy`]: a throwing callable maps codes onto -/// the module's typed domain exceptions, while a non-throwing callable maps -/// every failure onto the runtime exceptions (a reported error there is only -/// ever a producer bug or a runtime trap, never a domain error). +/// the module's typed domain exceptions, while a non-throwing callable traps +/// (a failure there is a producer bug, raised as a `NativeError`). #[derive(Clone, Copy)] pub(crate) struct ErrCtx<'a> { - /// `true` when the wrapper surfaces typed domain errors. - throws: bool, - /// The domain exception class in effect, if any. + /// The domain exception class a throwing callable maps codes onto, or + /// `None` for a callable that traps. exception: Option<&'a str>, } impl<'a> ErrCtx<'a> { + /// The context of `f`, whose module has the domain exception class + /// `exception` in scope (its own or an ancestor's). + pub(crate) fn of(f: &FnBinding, exception: Option<&'a str>) -> Self { + Self { + exception: exception.filter(|_| f.error_strategy() == ErrorStrategy::Throws), + } + } + /// The domain exception this wrapper throws, if it throws one. pub(crate) fn thrown_exception(&self) -> Option<&'a str> { - self.exception.filter(|_| self.throws) + self.exception } /// The `_ErrorMapper` the wrapper's checks use. fn mapper(&self) -> String { - self.thrown_exception() - .map_or_else(|| "_runtimeError".to_string(), mapper_fn) + self.exception + .map_or_else(|| "_trap".to_string(), mapper_fn) } /// The statement checking the shared error slot after a call. fn check_stmt(&self) -> String { - match self.thrown_exception() { + match self.exception { Some(exc) => format!("_check(_err, {});", mapper_fn(exc)), None => "_check(_err);".to_string(), } @@ -62,15 +66,6 @@ pub(crate) fn mapper_fn(exception: &str) -> String { format!("_map{exception}") } -/// The [`ErrCtx`] of one callable, paired with the exception class of the -/// domain in effect (own or inherited). -pub(crate) fn err_ctx<'a>(f: &FnBinding, exception: Option<&'a str>) -> ErrCtx<'a> { - ErrCtx { - throws: matches!(f.error_strategy(), ErrorStrategy::Throws), - exception, - } -} - /// How one wrapper is declared: a top-level function, or a member of an /// interface class. pub(crate) enum DartDecl<'a> { @@ -113,66 +108,45 @@ impl DartDecl<'_> { } } -/// Render one free function of `module` at top level. -pub(crate) fn render_function( - out: &mut String, - module: &ModuleBinding, - f: &FnBinding, - strip: bool, - leaf: bool, -) { - let exc = module - .error - .as_ref() - .map(|e| dart_exception_name(&e.type_name)); - let mut decl = String::new(); - render_callable( - out, - &mut decl, - f, - &DartDecl::TopLevel, - &dart_fn_name(&module.path, &f.name, strip), - err_ctx(f, exc.as_deref()), - leaf, - ); - out.push_str(&decl); +/// Bind every C symbol `f` calls, at top level. `leaf` marks synchronous +/// calls as leaf calls (only sound when the API declares no callback +/// interfaces, so no call can re-enter Dart). +pub(crate) fn emit_bindings(w: &mut CodeWriter, f: &FnBinding, leaf: bool) { + match &f.shape { + CallShape::Sync(abi) => emit_lookup(w, abi, leaf, None), + CallShape::Async(a) => { + let cb = ffi_typedef(&a.callback_type); + let (natives, _) = slot_types(&a.callback_params, None); + w.blank(); + w.line(format!( + "typedef {cb} = Void Function({});", + natives.join(", ") + )); + emit_lookup(w, &a.launch, false, Some(&cb)); + } + CallShape::Iterator(ib) => { + emit_lookup(w, &ib.launch, false, None); + emit_lookup(w, &ib.next, false, None); + emit_destroy(w, &ib.destroy_symbol, false); + } + } } -/// Render one callable: its FFI bindings into `lookups` (always top level) -/// and its wrapper declaration into `decl`. `leaf` marks synchronous calls -/// as leaf calls (only sound when the API declares no callback interfaces, -/// so no call can re-enter Dart). -pub(crate) fn render_callable( - lookups: &mut String, - decl: &mut String, +/// Render `f`'s wrapper, declared as `kind` and named `name`. +pub(crate) fn emit_wrapper( + w: &mut CodeWriter, f: &FnBinding, kind: &DartDecl, name: &str, err: ErrCtx, - leaf: bool, ) { - let mut w = CodeWriter::two_space(); w.blank(); - emit_wrapper_doc(&mut w, f, err); + emit_wrapper_doc(w, f, err); match &f.shape { - CallShape::Sync(abi) => { - emit_lookup(lookups, abi, leaf, None); - render_sync(&mut w, f, abi, kind, name, err); - } - CallShape::Async(a) => { - let cb = ffi_typedef(&a.callback_type); - emit_callback_typedef(lookups, &cb, &a.callback_params); - emit_lookup(lookups, &a.launch, false, Some(&cb)); - render_async(&mut w, f, a, &cb, kind, name, err); - } - CallShape::Iterator(ib) => { - emit_lookup(lookups, &ib.launch, false, None); - emit_lookup(lookups, &ib.next, false, None); - emit_iter_destroy(lookups, ib); - render_iterator(&mut w, f, ib, kind, name, err); - } + CallShape::Sync(abi) => render_sync(w, f, abi, kind, name, err), + CallShape::Async(a) => render_async(w, f, a, kind, name, err), + CallShape::Iterator(ib) => render_iterator(w, f, ib, kind, name, err), } - decl.push_str(&w.finish()); } /// The (native, Dart) FFI types of `params`, substituting `callback` for a @@ -191,56 +165,45 @@ fn slot_types(params: &[AbiParam], callback: Option<&str>) -> (Vec, Vec< } /// Bind one C symbol with `lookupFunction`. -pub(crate) fn emit_lookup(out: &mut String, abi: &AbiFn, leaf: bool, callback: Option<&str>) { +fn emit_lookup(w: &mut CodeWriter, abi: &AbiFn, leaf: bool, callback: Option<&str>) { let (natives, darts) = slot_types(&abi.params, callback); let (ret_n, ret_d) = ffi_type(&abi.ret); - out.push_str(&lookup( + lookup( + w, &abi.symbol, &format!("{ret_n} Function({})", natives.join(", ")), &format!("{ret_d} Function({})", darts.join(", ")), leaf, - )); + ); } -/// The `final _sym = _lib.lookupFunction<...>('sym');` binding of C symbol +/// Emit `final _sym = _lib.lookupFunction<...>('sym');`, binding C symbol /// `sym` with the given native and Dart function types. -pub(crate) fn lookup(sym: &str, native: &str, dart: &str, leaf: bool) -> String { +pub(crate) fn lookup(w: &mut CodeWriter, sym: &str, native: &str, dart: &str, leaf: bool) { let leaf = if leaf { ", isLeaf: true" } else { "" }; - format!( - "\nfinal {} = _lib.lookupFunction<\n {native},\n {dart}>('{sym}'{leaf});\n", - ffi_var(sym) - ) -} - -/// The `NativeFinalizer` over a destroy symbol `sym`, named after its -/// binding. -pub(crate) fn finalizer(sym: &str) -> String { - format!( - "final {}Finalizer = NativeFinalizer(\n _lib.lookup)>>('{sym}'));\n", - ffi_var(sym) - ) -} - -/// The native typedef of an async completion callback. -fn emit_callback_typedef(out: &mut String, name: &str, params: &[AbiParam]) { - let (natives, _) = slot_types(params, None); - out.push_str(&format!( - "\ntypedef {name} = Void Function({});\n", - natives.join(", ") - )); + w.blank(); + w.line(format!("final {} = _lib.lookupFunction<", ffi_var(sym))); + w.line(format!(" {native},")); + w.line(format!(" {dart}>('{sym}'{leaf});")); } -/// Bind an iterator's destroy symbol and the finalizer that backs abandoned -/// iterations. -fn emit_iter_destroy(out: &mut String, ib: &IteratorBinding) { - let sym = &ib.destroy_symbol; - out.push_str(&lookup( +/// Bind a destroy symbol `sym` (`void (*)(void*)`) and the `NativeFinalizer` +/// over it, named after the binding. +pub(crate) fn emit_destroy(w: &mut CodeWriter, sym: &str, leaf: bool) { + lookup( + w, sym, "Void Function(Pointer)", "void Function(Pointer)", - false, + leaf, + ); + w.line(format!( + "final {}Finalizer = NativeFinalizer(", + ffi_var(sym) + )); + w.line(format!( + " _lib.lookup)>>('{sym}'));" )); - out.push_str(&finalizer(sym)); } /// The wrapper's parameter list: the IDL parameters, plus a named @@ -285,93 +248,76 @@ struct Inputs { /// Callback registrations run last, after everything that can throw, so a /// failed staging step never strands a registered implementation. fn marshal_inputs(f: &FnBinding) -> Inputs { - let mut w = CodeWriter::two_space(); - let mut registrations: Vec = Vec::new(); - let mut args: Vec = Vec::new(); + let mut stage = Vec::new(); + let mut registrations = Vec::new(); + let mut args = Vec::new(); let mut arena = false; - let mut tmp = 0usize; if f.has_self { args.push("_self".into()); } for p in &f.params { - args.extend(marshal_param( - &mut w, - &mut registrations, - p, - &mut arena, - &mut tmp, - )); - } - let mut stage: Vec = w.finish().lines().map(str::to_string).collect(); - stage.extend(registrations); - Inputs { stage, args, arena } -} - -/// Marshal one parameter per its [`ArgPass`], returning its argument -/// expressions. -fn marshal_param( - w: &mut CodeWriter, - registrations: &mut Vec, - p: &ParamBinding, - arena: &mut bool, - tmp: &mut usize, -) -> Vec { - let name = dart_ident(&p.name); - match p.arg_pass() { - ArgPass::Direct { .. } => match &p.ty { - Ty::Enum(_) => vec![format!("{name}.value")], - _ => vec![name], - }, - ArgPass::String { .. } => { - *arena = true; - w.line(format!("final _{name}Bytes = utf8.encode({name});")); - vec![ - format!("_stage(_arena, _{name}Bytes)"), - format!("_{name}Bytes.length"), - ] - } - ArgPass::Bytes { .. } => { - *arena = true; - vec![format!("_stage(_arena, {name})"), format!("{name}.length")] - } - ArgPass::Buffer { .. } => { - *arena = true; - let writer = format!("_{name}Writer"); - w.line(format!("final {writer} = _BufferWriter();")); - write_stmts(w, &writer, &name, &p.ty, tmp); - w.line(format!("final _{name}Bytes = {writer}.takeBytes();")); - vec![ - format!("_stage(_arena, _{name}Bytes)"), - format!("_{name}Bytes.length"), - ] - } - // The wrapper keeps its reference and is borrowed for the call. - ArgPass::Object { nullable, .. } => { - *arena = true; - let ptr = format!("_{name}Ptr"); - if nullable { - w.line(format!( - "final {ptr} = {name} == null ? nullptr : _borrow(_arena, {name});" + let name = dart_ident(&p.name); + match p.arg_pass() { + ArgPass::Direct { .. } => args.push(match &p.ty { + Ty::Enum(_) => format!("{name}.value"), + _ => name, + }), + ArgPass::String { .. } => { + arena = true; + stage.push(format!("final _{name}Bytes = utf8.encode({name});")); + args.push(format!("_stage(_arena, _{name}Bytes)")); + args.push(format!("_{name}Bytes.length")); + } + ArgPass::Bytes { .. } => { + arena = true; + args.push(format!("_stage(_arena, {name})")); + args.push(format!("{name}.length")); + } + ArgPass::Buffer { .. } => { + arena = true; + stage.push(format!( + "final _{name}Bytes = _encode({name}, {});", + pack_fn(&p.ty) )); - } else { - w.line(format!("final {ptr} = _borrow(_arena, {name});")); + args.push(format!("_stage(_arena, _{name}Bytes)")); + args.push(format!("_{name}Bytes.length")); + } + // The wrapper keeps its reference and is borrowed for the call. + ArgPass::Object { nullable, .. } => { + arena = true; + let ptr = format!("_{name}Ptr"); + stage.push(if nullable { + format!("final {ptr} = {name} == null ? nullptr : _borrow(_arena, {name});") + } else { + format!("final {ptr} = _borrow(_arena, {name});") + }); + args.push(ptr); + } + // The producer owns the registration and releases it through the + // vtable's `free`; a null vtable means no implementation. + ArgPass::Callback { nullable, .. } => { + let cb = + p.ty.callback_interface_name() + .expect("callback family names a callback interface"); + let ctx = format!("_{name}Ctx"); + let register = format!("_registerCallback({name}, {})", dispatch_fn(cb)); + let vtable = format!("{}.cast()", vtable_var(cb)); + if nullable { + registrations.push(format!( + "final {ctx} = {name} == null ? nullptr : {register};" + )); + args.push(ctx); + args.push(format!("{name} == null ? nullptr : {vtable}")); + } else { + registrations.push(format!("final {ctx} = {register};")); + args.push(ctx); + args.push(vtable); + } } - vec![ptr] - } - // The producer owns the registration and releases it through the - // vtable's `free`. - ArgPass::Callback { .. } => { - let cb = - p.ty.callback_interface_name() - .expect("callback family names a callback interface"); - let ctx = format!("_{name}Ctx"); - registrations.push(format!( - "final {ctx} = _registerCallback({name}, {});", - dispatch_fn(cb) - )); - vec![ctx, format!("{}.cast()", vtable_var(cb))] } } + stage.extend(registrations); + Inputs { stage, args, arena } } /// The expression adopting the owned object pointer `expr` into its wrapper; @@ -385,15 +331,6 @@ pub(crate) fn adopt_expr(expr: &str, ty: &Ty, nullable: bool) -> String { } } -/// The decoder argument `_decode` takes for a buffered `ty`: the record's -/// own unpack function, or a closure over the inline read expression. -pub(crate) fn decoder(ty: &Ty) -> String { - match ty { - Ty::Record(n) | Ty::RichEnum(n) => unpack_fn(n), - _ => format!("(r) => {}", read_expr("r", ty)), - } -} - /// The expression turning a produced value into its Dart value per its /// [`RetPass`]: `value` is the pointer or scalar and `len` its length slot. /// Owned strings, bytes, and buffers are copied and released; objects are @@ -407,7 +344,7 @@ pub(crate) fn receive_expr(ty: &Ty, pass: &RetPass, value: &str, len: &str) -> S }, RetPass::String => format!("_takeString({value}, {len})"), RetPass::Bytes => format!("_takeBytes({value}, {len})"), - RetPass::Buffer => format!("_decode(_takeBytes({value}, {len}), {})", decoder(ty)), + RetPass::Buffer => format!("_decode(_takeBytes({value}, {len}), {})", unpack_fn(ty)), RetPass::Object { nullable, .. } => adopt_expr(value, ty, *nullable), } } @@ -449,7 +386,7 @@ fn render_sync( err: ErrCtx, ) { let ret = f.ret.as_ref().map_or("void".to_string(), dart_type); - let pass = plan::ret_pass(f.ret.as_ref(), ""); + let pass = RetPass::of(f.ret.as_ref()); let mut inputs = marshal_inputs(f); if matches!(pass, RetPass::String | RetPass::Bytes | RetPass::Buffer) { inputs.args.push("_outLen".into()); @@ -464,15 +401,14 @@ fn render_sync( for s in &inputs.stage { w.line(s); } - match &pass { - RetPass::Void => { + match (&pass, f.ret.as_ref()) { + (RetPass::Void, _) | (_, None) => { w.line(format!("{call};")); w.line(err.check_stmt()); } - _ => { + (_, Some(ty)) => { w.line(format!("final _result = {call};")); w.line(err.check_stmt()); - let ty = f.ret.as_ref().expect("non-void return"); w.line(format!( "return {};", receive_expr(ty, &pass, "_result", "_outLen.value") @@ -493,13 +429,13 @@ fn render_async( w: &mut CodeWriter, f: &FnBinding, a: &AsyncBinding, - cb: &str, kind: &DartDecl, name: &str, err: ErrCtx, ) { + let cb = ffi_typedef(&a.callback_type); let ret = f.ret.as_ref().map_or("void".to_string(), dart_type); - let pass = plan::ret_pass(f.ret.as_ref(), ""); + let pass = RetPass::of(f.ret.as_ref()); let mut inputs = marshal_inputs(f); if f.cancellable { inputs.args.push("_cancel?.pointer ?? nullptr".into()); @@ -510,7 +446,7 @@ fn render_async( // The listener's parameters, named after the callback's result slots. let (_, darts) = slot_types(&a.callback_params, None); - let names = ["_", "_error", "_value", "_valueLen"]; + let names = ["_", "error", "value", "valueLen"]; let listener_params: Vec = darts .iter() .zip(names) @@ -518,7 +454,7 @@ fn render_async( .collect(); let result = match (&pass, f.ret.as_ref()) { (RetPass::Void, _) | (_, None) => String::new(), - (_, Some(ty)) => receive_expr(ty, &pass, "_value", "_valueLen"), + (_, Some(ty)) => receive_expr(ty, &pass, "value", "valueLen"), }; let open = kind.open_line(&format!("Future<{ret}>"), name, &wrapper_params(f), ""); @@ -548,7 +484,7 @@ fn render_async( w.line("try {"); w.scope(|w| { w.line(format!( - "if (_error != nullptr) throw _takeAsyncError(_error, {});", + "if (error != nullptr) throw _takeAsyncError(error, {});", err.mapper() )); w.line(format!("_completer.complete({result});")); @@ -604,13 +540,13 @@ fn render_iterator( err: ErrCtx, ) { let ret = f.ret.as_ref().map_or("void".to_string(), dart_type); - let elem = plan::ret_pass(Some(&ib.elem), ""); + let elem = RetPass::of(Some(&ib.elem)); let mut inputs = marshal_inputs(f); inputs.args.push("_err".into()); let launch = format!("{}({})", ffi_var(&ib.launch.symbol), inputs.args.join(", ")); // `next(iter, out_item, [out_len,] out_err)`: `_outItem` is read as the // item slot's pointee. - let item_type = pointee_ffi(&ib.next.params[1].ty); + let item_type = ffi_type(ib.item_ctype()).0; let has_len = ib.next.params.len() == 4; let next_args = if has_len { "_iter, _outItem.cast(), _outLen, _err" diff --git a/crates/weaveffi-cli/src/targets/dart/codec.rs b/crates/weaveffi-cli/src/targets/dart/codec.rs new file mode 100644 index 00000000..b224c9b3 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dart/codec.rs @@ -0,0 +1,180 @@ +//! Value-buffer codec emitters: the expression writing and the expression +//! reading every wire shape, and the composite codecs, one +//! `_pack{Name}`/`_unpack{Name}` pair per distinct optional, list, or map +//! type (`[Entry]`, `{string:i64}`, `Shape?`), so no call site inlines a +//! loop. +//! +//! Every dispatch goes through [`Ty::wire`], and primitives map onto the +//! runtime's `read{Prim}`/`write{Prim}` methods by +//! [`Prim::pascal`](weaveffi_model::ty::Prim::pascal), so this module never +//! re-derives the wire format. +//! +//! An object token carries one strong reference. Writing one calls the +//! wrapper's `_cloneRef()` so the encoding owns a fresh reference and the +//! wrapper keeps its own; reading one adopts the pointer into a new wrapper. + +use std::collections::BTreeMap; + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{FieldBinding, Model, ParamBinding}; +use weaveffi_model::ty::{Ty, WireType}; + +use crate::targets::dart::types::{dart_class, dart_type}; + +/// The name of a buffered type's codec functions after `_pack`/`_unpack`: +/// a record or rich enum's class, or a composite spelled in prefix order so +/// it's unambiguous (`[Entry]` is `ListOfEntry`, `{string:[i32]}` is +/// `MapOfStringToListOfI32`, `Shape?` is `OptionalOfShape`). +fn codec_name(t: &Ty) -> String { + match t { + Ty::Prim(p) => p.pascal().to_string(), + Ty::Optional(inner) => format!("OptionalOf{}", codec_name(inner)), + Ty::List(inner) => format!("ListOf{}", codec_name(inner)), + Ty::Map(k, v) => format!("MapOf{}To{}", codec_name(k), codec_name(v)), + other => dart_class(other.user_name().expect("only user types remain")), + } +} + +/// The `_pack{Name}` function writing a value of the buffered type `t`. +pub(crate) fn pack_fn(t: &Ty) -> String { + format!("_pack{}", codec_name(t)) +} + +/// The `_unpack{Name}` function reading a value of the buffered type `t`. +pub(crate) fn unpack_fn(t: &Ty) -> String { + format!("_unpack{}", codec_name(t)) +} + +/// The Dart expression writing `expr` (a value of `t`) into the writer `w`. +pub(crate) fn write_expr(t: &Ty, w: &str, expr: &str) -> String { + match t.wire() { + WireType::Prim(p) => format!("{w}.write{}({expr})", p.pascal()), + // A fresh reference the reader adopts. + WireType::Object(_) => format!("{w}.writeU64({expr}._cloneRef().address)"), + WireType::Enum(_) => format!("{w}.writeI32({expr}.value)"), + WireType::User(_) | WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{}({w}, {expr})", pack_fn(t)) + } + } +} + +/// The Dart expression reading a value of `t` from the reader `r`; the +/// inverse of [`write_expr`]. An object token is adopted into a new wrapper. +pub(crate) fn read_expr(t: &Ty, r: &str) -> String { + match t.wire() { + WireType::Prim(p) => format!("{r}.read{}()", p.pascal()), + WireType::Object(n) => format!( + "{}._(Pointer.fromAddress({r}.readU64()))", + dart_class(n) + ), + WireType::Enum(n) => format!("{}.fromValue({r}.readI32())", dart_class(n)), + WireType::User(_) | WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{}({r})", unpack_fn(t)) + } + } +} + +/// Every distinct composite type (optional, list, or map) that crosses +/// inside a value buffer, by codec name. +fn composites(model: &Model) -> BTreeMap { + /// A type in a top-level position: a parameter, return, or iterator + /// element. `Interface?` and `Cb?` aren't buffers there. + fn top(t: &Ty, out: &mut BTreeMap) { + match t { + Ty::Iterator(elem) => top(elem, out), + _ if t.is_buffered() => nested(t, out), + _ => {} + } + } + /// A type inside a value buffer, where every optional is encoded. + fn nested(t: &Ty, out: &mut BTreeMap) { + match t { + Ty::Optional(inner) | Ty::List(inner) => { + out.insert(codec_name(t), t.clone()); + nested(inner, out); + } + Ty::Map(k, v) => { + out.insert(codec_name(t), t.clone()); + nested(k, out); + nested(v, out); + } + _ => {} + } + } + fn fields(fs: &[FieldBinding], out: &mut BTreeMap) { + for f in fs { + nested(&f.ty, out); + } + } + fn signature(ps: &[ParamBinding], ret: &Option, out: &mut BTreeMap) { + for p in ps { + top(&p.ty, out); + } + if let Some(t) = ret { + top(t, out); + } + } + let mut out = BTreeMap::new(); + for m in &model.modules { + for s in &m.structs { + fields(&s.fields, &mut out); + } + for e in &m.enums { + for v in &e.variants { + fields(&v.fields, &mut out); + } + } + if let Some(eb) = &m.errors { + for c in &eb.codes { + fields(&c.fields, &mut out); + } + } + for f in m.callables() { + signature(&f.params, &f.ret, &mut out); + } + for cb in &m.callback_interfaces { + for f in &cb.methods { + signature(&f.params, &f.ret, &mut out); + } + } + } + out +} + +/// Render the composite codecs `model` uses, one `_pack`/`_unpack` pair per +/// distinct type, built on the runtime's generic `writeOptional`, +/// `writeList`, and `writeMap` (and their readers). +pub(crate) fn render_codecs(w: &mut CodeWriter, model: &Model) { + for t in composites(model).values() { + let ty = dart_type(t); + let (write, read) = match t { + Ty::Optional(inner) => ( + format!("w.writeOptional(v, (e) => {})", write_expr(inner, "w", "e")), + format!("r.readOptional(() => {})", read_expr(inner, "r")), + ), + Ty::List(inner) => ( + format!("w.writeList(v, (e) => {})", write_expr(inner, "w", "e")), + format!("r.readList(() => {})", read_expr(inner, "r")), + ), + Ty::Map(k, v) => ( + format!( + "w.writeMap(v, (k) => {}, (e) => {})", + write_expr(k, "w", "k"), + write_expr(v, "w", "e") + ), + format!( + "r.readMap(() => {}, () => {})", + read_expr(k, "r"), + read_expr(v, "r") + ), + ), + _ => unreachable!("only composites are collected"), + }; + w.blank(); + w.line(format!("void {}(_BufferWriter w, {ty} v) =>", pack_fn(t))); + w.line(format!(" {write};")); + w.blank(); + w.line(format!("{ty} {}(_BufferReader r) =>", unpack_fn(t))); + w.line(format!(" {read};")); + } +} diff --git a/crates/weaveffi-gen/src/targets/dart/docs.rs b/crates/weaveffi-cli/src/targets/dart/docs.rs similarity index 97% rename from crates/weaveffi-gen/src/targets/dart/docs.rs rename to crates/weaveffi-cli/src/targets/dart/docs.rs index 848345df..81173cbb 100644 --- a/crates/weaveffi-gen/src/targets/dart/docs.rs +++ b/crates/weaveffi-cli/src/targets/dart/docs.rs @@ -4,7 +4,8 @@ use crate::codegen::common::DocCommentStyle; use crate::codegen::CodeWriter; -use weaveffi_model::model::{CallShape, FnBinding, Ty}; +use weaveffi_model::model::{CallShape, FnBinding}; +use weaveffi_model::ty::Ty; use crate::targets::dart::calls::ErrCtx; use crate::targets::dart::types::dart_str_literal; diff --git a/crates/weaveffi-cli/src/targets/dart/entities.rs b/crates/weaveffi-cli/src/targets/dart/entities.rs new file mode 100644 index 00000000..d489bdbf --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dart/entities.rs @@ -0,0 +1,544 @@ +//! Entity rendering: error domains, C-style enums, rich enums, records, and +//! interface wrapper classes, plus their value-buffer codec helpers. + +use crate::codegen::CodeWriter; +use heck::ToUpperCamelCase; +use weaveffi_model::model::{ + EnumBinding, EnumVariantBinding, ErrorBinding, FieldBinding, InterfaceBinding, ModuleBinding, + StructBinding, +}; +use weaveffi_model::ty::Ty; + +use crate::targets::dart::calls::{ + emit_bindings, emit_destroy, emit_wrapper, lookup, mapper_fn, DartDecl, ErrCtx, +}; +use crate::targets::dart::codec::{read_expr, write_expr}; +use crate::targets::dart::docs::{write_deprecated, write_doc}; +use crate::targets::dart::types::{ + dart_class, dart_ident, dart_member, dart_str_literal, dart_type, ffi_var, +}; + +/// The Dart exception class of an error domain or code: the PascalCase name +/// with a trailing `Error` swapped for `Exception` (`KvError` becomes +/// `KvException`, a code `IoError` becomes `IoException`, and a bare +/// `Error` becomes `ErrorException`). +pub(crate) fn dart_exception_name(raw: &str) -> String { + let pascal = raw.to_upper_camel_case(); + let stem = pascal + .strip_suffix("Error") + .filter(|s| !s.is_empty()) + .unwrap_or(&pascal); + if stem.ends_with("Exception") { + dart_class(stem) + } else { + dart_class(&format!("{stem}Exception")) + } +} + +/// The `_{exception}Payload` function encoding a domain exception's fields +/// for a callback that reports it. +pub(crate) fn payload_fn(exception: &str) -> String { + format!("_payloadOf{exception}") +} + +/// The indefinite article for a class name (`an Entry`, `a Contact`). +fn article(class: &str) -> &'static str { + if class.starts_with(['A', 'E', 'I', 'O', 'U']) { + "an" + } else { + "a" + } +} + +/// Render the final fields of a value class, with their docs. +fn render_fields(w: &mut CodeWriter, fields: &[FieldBinding]) { + for f in fields { + write_doc(w, &f.doc); + w.line(format!( + "final {} {};", + dart_type(&f.ty), + dart_ident(&f.name) + )); + } +} + +/// Whether a field of `ty` compares element by element (a list, map, or +/// byte array, possibly optional) rather than with `==`. +fn is_collection(ty: &Ty) -> bool { + match ty { + Ty::Optional(inner) => is_collection(inner), + Ty::List(_) | Ty::Map(..) | Ty::Prim(weaveffi_model::ty::Prim::Bytes) => true, + _ => false, + } +} + +/// Render `==`, `hashCode`, and `toString` for a value class: fields compare +/// by value, collections element by element. +fn render_value_members(w: &mut CodeWriter, class: &str, fields: &[FieldBinding]) { + w.blank(); + w.line("@override"); + if fields.is_empty() { + w.line(format!( + "bool operator ==(Object other) => other is {class};" + )); + w.blank(); + w.line("@override"); + w.line(format!("int get hashCode => ({class}).hashCode;")); + w.blank(); + w.line("@override"); + w.line(format!("String toString() => '{class}()';")); + return; + } + w.line("bool operator ==(Object other) =>"); + w.scope(|w| { + w.scope(|w| { + w.line("identical(this, other) ||"); + w.line(format!("other is {class} &&")); + for (i, f) in fields.iter().enumerate() { + let n = dart_ident(&f.name); + let end = if i + 1 == fields.len() { ";" } else { " &&" }; + if is_collection(&f.ty) { + w.line(format!("_deepEquals({n}, other.{n}){end}")); + } else { + w.line(format!("{n} == other.{n}{end}")); + } + } + }); + }); + w.blank(); + w.line("@override"); + let hashes: Vec = fields + .iter() + .map(|f| { + let n = dart_ident(&f.name); + if is_collection(&f.ty) { + format!("_deepHash({n})") + } else { + n + } + }) + .collect(); + w.line(format!( + "int get hashCode => Object.hashAll([{class}, {}]);", + hashes.join(", ") + )); + w.blank(); + w.line("@override"); + let shown: Vec = fields + .iter() + .map(|f| { + let n = dart_ident(&f.name); + format!("{n}: ${n}") + }) + .collect(); + w.line(format!( + "String toString() => '{class}({})';", + shown.join(", ") + )); +} + +/// Render one module's declared error domain: a sealed domain exception (a +/// `NativeException` subclass) with one final subclass per code carrying +/// its code, default message, and decoded payload fields; the +/// `_map{Domain}` mapper that throwing wrappers route codes through (only +/// declared codes gain a `case`; every other code, including the negative +/// runtime range, maps onto the runtime exceptions); and, when `reported` +/// (a callback method of the API reports this domain), the encoder of each +/// code's fields. +pub(crate) fn render_error( + w: &mut CodeWriter, + module: &ModuleBinding, + eb: &ErrorBinding, + reported: bool, +) { + let exc = dart_exception_name(&eb.type_name); + w.blank(); + w.line(format!( + "/// The `{}` error domain of module `{}`: catch it to handle any of", + eb.name, module.dot_path + )); + w.line("/// its codes, or a subclass for one code."); + w.block( + format!("sealed class {exc} extends NativeException {{"), + "}", + |w| { + w.line("/// Creates a domain error carrying [code] and [message]."); + w.line(format!("{exc}(super.code, super.message);")); + }, + ); + + for c in &eb.codes { + let class = dart_exception_name(&c.name); + let message = dart_str_literal(&c.message); + w.blank(); + write_doc(w, &c.doc.clone().or_else(|| Some(c.message.clone()))); + w.block(format!("final class {class} extends {exc} {{"), "}", |w| { + w.line(format!( + "/// Creates {} [{class}] (code {}).", + article(&class), + c.value + )); + if c.fields.is_empty() { + w.line(format!( + "{class}([String message = '{message}']) : super({}, message);", + c.value + )); + } else { + let params: Vec = c + .fields + .iter() + .map(|f| format!("this.{}", dart_ident(&f.name))) + .collect(); + w.line(format!( + "{class}({}, [String message = '{message}'])", + params.join(", ") + )); + w.line(format!(" : super({}, message);", c.value)); + w.blank(); + render_fields(w, &c.fields); + } + }); + } + + w.blank(); + w.block( + format!( + "NativeException {}(int code, String message, Uint8List payload) {{", + mapper_fn(&exc) + ), + "}", + |w| { + w.block("switch (code) {", "}", |w| { + for c in &eb.codes { + let class = dart_exception_name(&c.name); + w.line(format!("case {}:", c.value)); + w.scope(|w| { + if c.fields.is_empty() { + w.line(format!("return {class}(message);")); + } else { + let args: Vec = + c.fields.iter().map(|f| read_expr(&f.ty, "r")).collect(); + w.line(format!( + "return _decode(payload, (r) => {class}({}, message));", + args.join(", ") + )); + } + }); + } + w.line("default:"); + w.scope(|w| { + w.line("return _runtimeException(code, message, payload);"); + }); + }); + }, + ); + + if !reported { + return; + } + w.blank(); + w.line("/// The value buffer of [e]'s fields, or null for a code without any."); + w.block( + format!("Uint8List? {}({exc} e) {{", payload_fn(&exc)), + "}", + |w| { + w.block("switch (e) {", "}", |w| { + for c in &eb.codes { + let class = dart_exception_name(&c.name); + w.line(format!("case {class}():")); + w.scope(|w| { + if c.fields.is_empty() { + w.line("return null;"); + } else { + w.line("final w = _BufferWriter();"); + for f in &c.fields { + let expr = format!("e.{}", dart_ident(&f.name)); + w.line(format!("{};", write_expr(&f.ty, "w", &expr))); + } + w.line("return w.takeBytes();"); + } + }); + } + }); + }, + ); +} + +/// Render one interface: its lifecycle and member bindings at top level, +/// then a wrapper class over the runtime's `_NativeObject` where the +/// canonical `new` constructor is an unnamed factory, other constructors are +/// named factories, methods borrow the wrapper's pointer, and statics are +/// `static`. `exception` is the domain exception class in scope for the +/// interface's module. +pub(crate) fn render_interface( + w: &mut CodeWriter, + exception: Option<&str>, + i: &InterfaceBinding, + leaf: bool, +) { + let class = dart_class(&i.name); + let destroy = ffi_var(&i.destroy_symbol); + let clone = ffi_var(&i.clone_symbol); + lookup( + w, + &i.clone_symbol, + "Pointer Function(Pointer)", + "Pointer Function(Pointer)", + leaf, + ); + emit_destroy(w, &i.destroy_symbol, leaf); + let members = || { + let ctors = i.constructors.iter().map(|c| { + let kind = DartDecl::Factory { + class_name: &class, + named: c.name != "new", + }; + (c, kind) + }); + let methods = i.methods.iter().map(|m| (m, DartDecl::Method)); + let statics = i.statics.iter().map(|s| (s, DartDecl::Static)); + ctors.chain(methods).chain(statics) + }; + for (f, _) in members() { + emit_bindings(w, f, leaf); + } + + w.blank(); + write_doc(w, &i.doc); + if i.doc.is_some() { + w.line("///"); + } + w.line("/// A reference-counted native object. Each instance holds one strong"); + w.line("/// reference, released by [dispose] or, when an undisposed instance is"); + w.line("/// collected, by a finalizer."); + write_deprecated(w, &i.deprecated); + w.block( + format!("final class {class} extends _NativeObject {{"), + "}", + |w| { + w.line(format!("{class}._(super._ptr);")); + w.blank(); + w.line("@override"); + w.line(format!( + "NativeFinalizer get _finalizer => {destroy}Finalizer;" + )); + w.blank(); + w.line("@override"); + w.line(format!( + "void _destroy(Pointer ptr) => {destroy}(ptr);" + )); + w.blank(); + w.line("@override"); + w.line(format!( + "Pointer _clone(Pointer ptr) => {clone}(ptr);" + )); + for (f, kind) in members() { + emit_wrapper(w, f, &kind, &dart_member(&f.name), ErrCtx::of(f, exception)); + } + }, + ); +} + +/// Enum members `enum` declarations already define; a variant spelled like +/// one gains a trailing `_`. +const ENUM_MEMBERS: &[&str] = &["fromValue", "index", "name", "value", "values"]; + +/// Render one enum: a C-style enum becomes an enhanced Dart `enum` carrying +/// its discriminant; a rich enum becomes a sealed class hierarchy. +pub(crate) fn render_enum(w: &mut CodeWriter, e: &EnumBinding) { + if e.is_rich() { + render_rich_enum(w, e); + return; + } + let name = dart_class(&e.name); + w.blank(); + write_doc(w, &e.doc); + write_deprecated(w, &e.deprecated); + w.block(format!("enum {name} {{"), "}", |w| { + for (i, v) in e.variants.iter().enumerate() { + let mut variant = dart_ident(&v.name); + if ENUM_MEMBERS.contains(&variant.as_str()) { + variant.push('_'); + } + write_doc(w, &v.doc); + let end = if i + 1 == e.variants.len() { ';' } else { ',' }; + w.line(format!("{variant}({}){end}", v.value)); + } + w.blank(); + w.line(format!("const {name}(this.value);")); + w.blank(); + w.line("/// The C discriminant."); + w.line("final int value;"); + w.blank(); + w.line("/// The variant whose discriminant is [value]."); + w.line(format!("static {name} fromValue(int value) => values.firstWhere(")); + w.line(" (e) => e.value == value,"); + w.line(format!( + " orElse: () => throw ArgumentError.value(value, 'value', 'not {} {name} discriminant'));", + article(&name) + )); + }); +} + +/// Render one record as a Dart value class (final fields, one named +/// constructor argument per field, optional fields not required, value +/// equality) plus its `_pack{Name}`/`_unpack{Name}` helpers. +pub(crate) fn render_struct(w: &mut CodeWriter, s: &StructBinding) { + let class = dart_class(&s.name); + w.blank(); + write_doc(w, &s.doc); + write_deprecated(w, &s.deprecated); + w.block(format!("final class {class} {{"), "}", |w| { + w.line(format!("/// Creates {} [{class}].", article(&class))); + if s.fields.is_empty() { + w.line(format!("const {class}();")); + } else { + let params: Vec = s + .fields + .iter() + .map(|f| { + let n = dart_ident(&f.name); + if matches!(f.ty, Ty::Optional(_)) { + format!("this.{n}") + } else { + format!("required this.{n}") + } + }) + .collect(); + w.line(format!("{class}({{{}}});", params.join(", "))); + w.blank(); + render_fields(w, &s.fields); + } + render_value_members(w, &class, &s.fields); + }); + + w.blank(); + if s.fields.is_empty() { + w.line(format!( + "void _pack{class}(_BufferWriter w, {class} v) {{}}" + )); + } else { + w.block( + format!("void _pack{class}(_BufferWriter w, {class} v) {{"), + "}", + |w| { + for f in &s.fields { + let expr = format!("v.{}", dart_ident(&f.name)); + w.line(format!("{};", write_expr(&f.ty, "w", &expr))); + } + }, + ); + } + + // Named arguments evaluate in source order, which is the wire order. + w.blank(); + if s.fields.is_empty() { + w.line(format!( + "{class} _unpack{class}(_BufferReader r) => const {class}();" + )); + } else { + w.line(format!( + "{class} _unpack{class}(_BufferReader r) => {class}(" + )); + w.indent().indent().indent(); + for f in &s.fields { + w.line(format!( + "{}: {},", + dart_ident(&f.name), + read_expr(&f.ty, "r") + )); + } + w.dedent().dedent().dedent(); + w.line(" );"); + } +} + +/// The class of one rich-enum variant. +fn variant_class(e: &EnumBinding, v: &EnumVariantBinding) -> String { + dart_class(&format!("{}{}", e.name, v.name)) +} + +/// Render one rich enum as a sealed class hierarchy: a sealed base plus one +/// final subclass per variant carrying its fields (positional constructor, +/// value equality), and `_pack{Name}`/`_unpack{Name}` helpers encoding the +/// `i32` tag followed by the active variant's fields. +fn render_rich_enum(w: &mut CodeWriter, e: &EnumBinding) { + let base = dart_class(&e.name); + w.blank(); + write_doc(w, &e.doc); + write_deprecated(w, &e.deprecated); + w.block(format!("sealed class {base} {{"), "}", |w| { + w.line(format!("const {base}();")); + }); + + for v in &e.variants { + let cls = variant_class(e, v); + w.blank(); + write_doc(w, &v.doc); + w.block(format!("final class {cls} extends {base} {{"), "}", |w| { + w.line(format!("/// Creates {} [{cls}].", article(&cls))); + if v.fields.is_empty() { + w.line(format!("const {cls}();")); + } else { + let params: Vec = v + .fields + .iter() + .map(|f| format!("this.{}", dart_ident(&f.name))) + .collect(); + w.line(format!("{cls}({});", params.join(", "))); + w.blank(); + render_fields(w, &v.fields); + } + render_value_members(w, &cls, &v.fields); + }); + } + + // Pack: the tag, then the active variant's fields. The sealed base makes + // the switch exhaustive. + w.blank(); + w.block( + format!("void _pack{base}(_BufferWriter w, {base} v) {{"), + "}", + |w| { + w.block("switch (v) {", "}", |w| { + for var in &e.variants { + let cls = variant_class(e, var); + w.line(format!("case {cls}():")); + w.scope(|w| { + w.line(format!("w.writeI32({});", var.value)); + for f in &var.fields { + let expr = format!("v.{}", dart_ident(&f.name)); + w.line(format!("{};", write_expr(&f.ty, "w", &expr))); + } + }); + } + }); + }, + ); + + // Unpack: positional arguments evaluate left to right, in wire order. + w.blank(); + w.block( + format!("{base} _unpack{base}(_BufferReader r) {{"), + "}", + |w| { + w.line("final tag = r.readI32();"); + w.block("switch (tag) {", "}", |w| { + for v in &e.variants { + let cls = variant_class(e, v); + w.line(format!("case {}:", v.value)); + w.scope(|w| { + let args: Vec = + v.fields.iter().map(|f| read_expr(&f.ty, "r")).collect(); + let ctor = if v.fields.is_empty() { "const " } else { "" }; + w.line(format!("return {ctor}{cls}({});", args.join(", "))); + }); + } + w.line("default:"); + w.scope(|w| { + w.line(format!("_bufferError('unknown {base} tag $tag');")); + }); + }); + }, + ); +} diff --git a/crates/weaveffi-cli/src/targets/dart/mod.rs b/crates/weaveffi-cli/src/targets/dart/mod.rs new file mode 100644 index 00000000..604a9019 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dart/mod.rs @@ -0,0 +1,244 @@ +//! Dart (`dart:ffi`) binding generator. +//! +//! Emits a standalone Dart package (`pubspec.yaml`, `README.md`, and +//! `lib/{package}.dart`) whose bindings call the C ABI through `dart:ffi`. +//! The package is named after the identity's C prefix and loads the +//! identity's library, honoring the `{PREFIX}_LIBRARY` override. On first use +//! the library's ABI revision and every top-level module's contract table +//! are verified. +//! +//! Records and rich enums are value types: plain Dart classes (a sealed +//! hierarchy for a rich enum) with value equality that cross the ABI +//! serialized in the value buffer format. Interfaces are wrapper classes +//! owning one strong reference, released by `dispose()` or a +//! `NativeFinalizer`, and guarded so a `dispose()` during an in-flight call +//! defers the release until the call returns. Async functions return a +//! `Future` completed by a `NativeCallable.listener`; cancellable ones take a +//! `CancelToken`. Callback interfaces are abstract classes: value-returning +//! methods are isolate-local trampolines (callable only on the isolate's +//! thread), and void methods are isolate-group-bound forwarders that may run +//! on any thread and deliver the call on the isolate's event loop. + +mod callbacks; +mod calls; +mod codec; +mod docs; +mod entities; +mod package; +mod runtime; +mod types; + +#[cfg(test)] +mod tests; + +use std::collections::BTreeSet; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::codegen::CodeWriter; +use crate::package::{per_platform_libraries, Artifact, PackageContext, PackagedFile}; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; +use camino::Utf8Path; +use serde::{Deserialize, Serialize}; +use weaveffi_model::model::Model; + +use crate::targets::dart::callbacks::render_callback_interface; +use crate::targets::dart::calls::{emit_bindings, emit_wrapper, DartDecl, ErrCtx}; +use crate::targets::dart::codec::render_codecs; +use crate::targets::dart::entities::{ + dart_exception_name, render_enum, render_error, render_interface, render_struct, +}; +use crate::targets::dart::package::{render_packaged_readme, render_pubspec, render_readme}; +use crate::targets::dart::runtime::{bundles_platform, render_runtime, Bundle}; +use crate::targets::dart::types::dart_ident; + +/// The lowest Dart SDK the generated bindings support: +/// `NativeCallable.isolateGroupBound`, which delivers void callback methods +/// from any thread, first shipped in Dart 3.10. +pub const MIN_DART_SDK: &str = "3.10.0"; + +/// Per-target configuration for [`DartGenerator`]. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct DartConfig { + /// Dart package name: the pubspec `name` and the library file + /// `lib/{name}.dart`. Defaults to the identity's C prefix. + pub name: Option, + /// The pubspec's Dart SDK constraint (default `>=3.10.0 <4.0.0`; see + /// [`MIN_DART_SDK`]). + pub sdk: String, +} + +impl Default for DartConfig { + fn default() -> Self { + Self { + name: None, + sdk: format!(">={MIN_DART_SDK} <4.0.0"), + } + } +} + +impl DartConfig { + /// The Dart package name for `model`: the configured name, else the + /// identity's C prefix. + pub fn package_name(&self, model: &Model) -> String { + self.name + .clone() + .unwrap_or_else(|| model.identity.prefix.clone()) + } +} + +/// The Dart backend. +pub struct DartGenerator; + +/// Render `lib/{package}.dart`: imports, the runtime sections the API uses, +/// the composite codecs, then every module's declarations. +fn render_library(model: &Model, package: &str, bundle: &Bundle, file_name: &str) -> String { + let mut w = CodeWriter::two_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + w.line("// ignore_for_file: camel_case_types, non_constant_identifier_names, unused_element"); + w.blank(); + let callbacks = model.has_callback_interfaces(); + let bundled = matches!(bundle, Bundle::Packaged { .. }); + if model.has_async() || callbacks { + w.line("import 'dart:async';"); + } + w.line("import 'dart:convert';"); + w.line("import 'dart:ffi';"); + if bundled { + w.line("import 'dart:io' show Directory, File, Platform;"); + } else { + w.line("import 'dart:io' show Platform;"); + } + if callbacks || bundled { + w.line("import 'dart:isolate';"); + } + w.line("import 'dart:typed_data';"); + w.blank(); + w.line("import 'package:ffi/ffi.dart';"); + + render_runtime(&mut w, model, package, bundle); + render_codecs(&mut w, model); + + // Synchronous calls are leaf calls only when no callback interface + // exists, so no call can re-enter Dart. + let leaf = !callbacks; + let reported = reported_domains(model); + for module in &model.modules { + let exception = model + .error_domain(module) + .map(|e| dart_exception_name(&e.type_name)); + let exception = exception.as_deref(); + if let Some(e) = module.errors.as_ref() { + render_error(&mut w, module, e, reported.contains(&e.type_name)); + } + for e in &module.enums { + render_enum(&mut w, e); + } + for s in &module.structs { + render_struct(&mut w, s); + } + for cb in &module.callback_interfaces { + render_callback_interface(&mut w, cb, exception); + } + for i in &module.interfaces { + render_interface(&mut w, exception, i, leaf); + } + for f in &module.functions { + emit_bindings(&mut w, f, leaf); + emit_wrapper( + &mut w, + f, + &DartDecl::TopLevel, + &dart_ident(&f.name), + ErrCtx::of(f, exception), + ); + } + } + w.blank(); + w.raw(render_trailer(CommentStyle::DoubleSlash, file_name)); + w.finish() +} + +/// The error domains (by type name) that a value-returning callback method +/// declared `throws` reports, which need an encoder for their fields. +fn reported_domains(model: &Model) -> BTreeSet { + model + .modules + .iter() + .filter(|m| { + m.callback_interfaces + .iter() + .flat_map(|cb| &cb.methods) + .any(|f| f.throws && f.ret.is_some()) + }) + .filter_map(|m| model.error_domain(m)) + .map(|e| e.type_name.clone()) + .collect() +} + +impl LanguageBackend for DartGenerator { + type Config = DartConfig; + + fn name(&self) -> &'static str { + "dart" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let package = config.package_name(model); + let dir = out_dir.join("dart"); + let file = format!("{package}.dart"); + vec![ + OutputFile::new( + dir.join("lib").join(&file), + render_library(model, &package, &Bundle::None, &file), + ), + OutputFile::new( + dir.join("pubspec.yaml"), + render_pubspec(&model.identity, &package, &config.sdk), + ), + OutputFile::new( + dir.join("README.md"), + render_readme(&model.identity, &package, &config.sdk), + ), + ] + } + + /// A pub package directory at `dart/{package}/` with the desktop + /// libraries under `native//`, which the loader tries (relative + /// to the package, then to the working directory) before the system + /// search path. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let natives = per_platform_libraries(ctx.binaries, "native", bundles_platform); + if natives.is_empty() { + return Some(Vec::new()); + } + let package = config.package_name(model); + let file = format!("{package}.dart"); + let bundle = Bundle::Packaged { + lib: &ctx.binaries.lib_name, + platforms: ctx + .binaries + .platforms() + .filter(|p| bundles_platform(*p)) + .collect(), + }; + let mut files = vec![ + PackagedFile::text( + format!("lib/{file}"), + render_library(model, &package, &bundle, &file), + ), + PackagedFile::text( + "pubspec.yaml", + render_pubspec(&model.identity, &package, &config.sdk), + ), + PackagedFile::text("README.md", render_packaged_readme(&model.identity, ctx)), + ]; + files.extend(natives); + Some(vec![Artifact::directory(format!("dart/{package}"), files)]) + } +} diff --git a/crates/weaveffi-gen/src/targets/dart/package.rs b/crates/weaveffi-cli/src/targets/dart/package.rs similarity index 74% rename from crates/weaveffi-gen/src/targets/dart/package.rs rename to crates/weaveffi-cli/src/targets/dart/package.rs index c1592558..335f0756 100644 --- a/crates/weaveffi-gen/src/targets/dart/package.rs +++ b/crates/weaveffi-cli/src/targets/dart/package.rs @@ -7,19 +7,14 @@ use weaveffi_model::pkg::Identity; use crate::targets::dart::runtime::bundles_platform; -/// The Dart SDK range the generated code needs: -/// `NativeCallable.isolateGroupBound` (callback interfaces) is verified -/// against 3.12. -pub(crate) const SDK_CONSTRAINT: &str = ">=3.12.0 <4.0.0"; - /// Quote a YAML scalar. fn yaml_str(s: &str) -> String { format!("'{}'", s.replace('\'', "''")) } /// Render `pubspec.yaml` for package `package`. -pub(crate) fn render_pubspec(identity: &Identity, package: &str, input_basename: &str) -> String { - let mut out = render_prelude(CommentStyle::Hash, input_basename); +pub(crate) fn render_pubspec(identity: &Identity, package: &str, sdk: &str) -> String { + let mut out = render_prelude(CommentStyle::Hash); out.push_str(&format!("name: {package}\n")); out.push_str(&format!("version: {}\n", identity.version)); out.push_str(&format!( @@ -33,14 +28,15 @@ pub(crate) fn render_pubspec(identity: &Identity, package: &str, input_basename: out.push_str(&format!("repository: {}\n", yaml_str(repository))); } out.push_str(&format!( - "\nenvironment:\n sdk: '{SDK_CONSTRAINT}'\n\ndependencies:\n ffi: ^2.1.0\n\n" + "\nenvironment:\n sdk: {}\n\ndependencies:\n ffi: ^2.1.0\n\n", + yaml_str(sdk) )); out.push_str(&render_trailer(CommentStyle::Hash, "pubspec.yaml")); out } /// Render the README of the generated package. -pub(crate) fn render_readme(identity: &Identity, package: &str, input_basename: &str) -> String { +pub(crate) fn render_readme(identity: &Identity, package: &str, sdk: &str) -> String { let (macos, linux, windows) = identity.library_files(); let env = identity.library_env_var(); let name = &identity.name; @@ -67,27 +63,24 @@ fallback, the bindings look the symbols up in the running executable. Each interface wrapper holds one native reference: call `dispose()` when you're done, or let the garbage collector's finalizer release it. -Callback-interface methods that return a value run synchronously on the -isolate's thread; void methods may be called from any thread and are -delivered on the event loop. +Callback-interface methods that return a value run synchronously and must +be called by the native library on the isolate's thread during a call from +Dart; void methods may be called from any thread and are delivered on the +event loop. ## Requirements -- Dart SDK `{SDK_CONSTRAINT}` +- Dart SDK `{sdk}` - The `ffi` package {trailer}", - prelude = render_prelude(CommentStyle::Xml, input_basename), + prelude = render_prelude(CommentStyle::Xml), trailer = render_trailer(CommentStyle::Xml, "README.md"), ) } /// Render the README of a packaged library that bundles native binaries. -pub(crate) fn render_packaged_readme( - identity: &Identity, - ctx: &PackageContext, - input_basename: &str, -) -> String { +pub(crate) fn render_packaged_readme(identity: &Identity, ctx: &PackageContext) -> String { let platforms: Vec = ctx .binaries .platforms() @@ -98,15 +91,16 @@ pub(crate) fn render_packaged_readme( "{prelude}# {name} (Dart) `dart:ffi` bindings with prebuilt native libraries bundled under -`native//`. The loader tries the bundled library (relative to the -working directory) before the system search path; `{env}` overrides both. +`native//`. The loader tries the running platform's bundled +library (in this package, then under the working directory) before the +system search path; `{env}` overrides all of them. ## Bundled platforms {platforms} {trailer}", - prelude = render_prelude(CommentStyle::Xml, input_basename), + prelude = render_prelude(CommentStyle::Xml), name = identity.name, env = identity.library_env_var(), platforms = platforms.join("\n"), diff --git a/crates/weaveffi-cli/src/targets/dart/runtime.rs b/crates/weaveffi-cli/src/targets/dart/runtime.rs new file mode 100644 index 00000000..3276a463 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dart/runtime.rs @@ -0,0 +1,174 @@ +//! The fixed Dart runtime every generated library ships, kept as real Dart +//! source under `runtime/` and spliced in with `{{PLACEHOLDER}}` +//! substitution: the loader and load-time checks, the error and memory +//! helpers, the object base class, the value-buffer codec, async and +//! cancellation glue, the callback-interface machinery, and the iterator +//! anchor. Sections an API doesn't use are left out. The contract tables the +//! loader checks are rendered from the model. + +use crate::codegen::CodeWriter; +use crate::platform::{Arch, Os, Platform}; +use weaveffi_model::contract::entries; +use weaveffi_model::model::{contract_symbol, Model, ABI_VERSION}; + +use crate::targets::dart::types::dart_str_literal; + +const LOADER: &str = include_str!("runtime/loader.dart"); +const BUNDLED: &str = include_str!("runtime/bundled.dart"); +const CORE: &str = include_str!("runtime/core.dart"); +const OBJECT: &str = include_str!("runtime/object.dart"); +const CODEC: &str = include_str!("runtime/codec.dart"); +const ASYNC: &str = include_str!("runtime/async.dart"); +const CANCEL: &str = include_str!("runtime/cancel.dart"); +const CALLBACKS: &str = include_str!("runtime/callbacks.dart"); +const ITERATOR: &str = include_str!("runtime/iterator.dart"); + +/// Where the loader looks for the library before the system search path. +pub(crate) enum Bundle<'a> { + /// Nowhere: the generated (unpackaged) bindings. + None, + /// `native//` under the package root (then the working + /// directory), for the packaged library `lib` of the given platforms. + Packaged { + /// The library's base name (`kvstore` for `libkvstore.dylib`). + lib: &'a str, + /// The platforms the package bundles. + platforms: Vec, + }, +} + +/// Whether the packaged library bundles a prebuilt binary for `platform`: +/// the loader resolves `native//` for the desktop matrix only +/// (Android libraries ship through the app's own packaging, and a `.wasm` +/// module can't be opened with `DynamicLibrary`). +pub(crate) fn bundles_platform(platform: Platform) -> bool { + platform.is_desktop() +} + +/// The `dart:ffi` `Abi` constant of a desktop platform. +fn dart_abi(platform: Platform) -> Option<&'static str> { + Some(match (platform.os(), platform.arch()) { + (Os::MacOs, Arch::Arm64) => "Abi.macosArm64", + (Os::MacOs, Arch::X64) => "Abi.macosX64", + (Os::Linux, Arch::Arm64) => "Abi.linuxArm64", + (Os::Linux, Arch::X64) => "Abi.linuxX64", + (Os::Windows, Arch::Arm64) => "Abi.windowsArm64", + (Os::Windows, Arch::X64) => "Abi.windowsX64", + _ => return None, + }) +} + +/// Replace every `{{KEY}}` in `template` with its value. +fn fill(template: &str, vars: &[(&str, &str)]) -> String { + let mut out = template.to_string(); + for (key, value) in vars { + out = out.replace(&format!("{{{{{key}}}}}"), value); + } + debug_assert!(!out.contains("{{"), "unfilled placeholder in:\n{out}"); + out +} + +/// Render the runtime sections `model` needs into `w`; `package` is the +/// Dart package name and `bundle` says where the loader looks first. +pub(crate) fn render_runtime(w: &mut CodeWriter, model: &Model, package: &str, bundle: &Bundle) { + let identity = &model.identity; + let (macos, linux, windows) = identity.library_files(); + let prefix = identity.prefix.as_str(); + let env = identity.library_env_var(); + let library = dart_str_literal(&identity.library); + let name = dart_str_literal(&identity.name); + let abi = ABI_VERSION.to_string(); + let common = [ + ("PREFIX", prefix), + ("LIBRARY_ENV", env.as_str()), + ("LIBRARY", library.as_str()), + ("NAME", name.as_str()), + ]; + + let bundled_lookup = match bundle { + Bundle::None => "", + Bundle::Packaged { .. } => { + " final bundled = _openBundled();\n if (bundled != null) return bundled;\n" + } + }; + let (macos, linux, windows) = ( + dart_str_literal(&macos), + dart_str_literal(&linux), + dart_str_literal(&windows), + ); + let mut vars = common.to_vec(); + vars.extend([ + ("MACOS_FILE", macos.as_str()), + ("LINUX_FILE", linux.as_str()), + ("WINDOWS_FILE", windows.as_str()), + ("BUNDLED_LOOKUP", bundled_lookup), + ("ABI_VERSION", abi.as_str()), + ]); + w.blank(); + w.raw(fill(LOADER, &vars)); + render_contracts(w, model); + if let Bundle::Packaged { lib, platforms } = bundle { + let cases: String = platforms + .iter() + .filter_map(|p| { + let abi = dart_abi(*p)?; + Some(format!( + " {abi} => 'native/{}/{}',\n", + p.id(), + dart_str_literal(&p.lib_filename(lib)) + )) + }) + .collect(); + let package = dart_str_literal(package); + let mut vars = common.to_vec(); + vars.extend([ + ("BUNDLED_CASES", cases.as_str()), + ("PACKAGE", package.as_str()), + ]); + w.raw(fill(BUNDLED, &vars)); + } + + let cancellable = model.callables().any(|(_, f)| f.cancellable); + let sections = [ + (true, CORE), + (model.has_interfaces(), OBJECT), + (model.has_buffers(), CODEC), + (model.has_async(), ASYNC), + (cancellable, CANCEL), + (model.has_callback_interfaces(), CALLBACKS), + (model.has_iterators(), ITERATOR), + ]; + for (_, source) in sections.iter().filter(|(used, _)| *used) { + w.blank(); + w.raw(fill(source, &common)); + } +} + +/// The contract tables the loader checks: for every top-level module, its +/// contract symbol and each declaration these bindings were generated with. +fn render_contracts(w: &mut CodeWriter, model: &Model) { + w.blank(); + w.line("/// Each top-level module's contract-table symbol and the declarations"); + w.line("/// these bindings were generated with: (id, signature hash, path)."); + w.line("const List<(String, List<(int, int, String)>)> _contracts = ["); + w.scope(|w| { + for root in model.roots() { + w.line(format!( + "('{}', [", + contract_symbol(model.prefix(), &root.name) + )); + w.scope(|w| { + for e in entries(model, root) { + w.line(format!( + "(0x{:016x}, 0x{:016x}, '{}'),", + e.id, + e.hash, + dart_str_literal(&e.path) + )); + } + }); + w.line("]),"); + } + }); + w.line("];"); +} diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/async.dart b/crates/weaveffi-cli/src/targets/dart/runtime/async.dart similarity index 77% rename from crates/weaveffi-gen/src/targets/dart/runtime/async.dart rename to crates/weaveffi-cli/src/targets/dart/runtime/async.dart index 72917a15..f0c63bf6 100644 --- a/crates/weaveffi-gen/src/targets/dart/runtime/async.dart +++ b/crates/weaveffi-cli/src/targets/dart/runtime/async.dart @@ -8,8 +8,8 @@ final _errorFree = _lib.lookupFunction), void Function(Pointer<_Error>)>( '{{PREFIX}}_error_free'); -/// Builds the exception a completion reports and frees its boxed error. -NativeException _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error a completion reports and frees its boxed error. +Object _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { final error = _readError(err, map); _errorFree(err); return error; diff --git a/crates/weaveffi-cli/src/targets/dart/runtime/bundled.dart b/crates/weaveffi-cli/src/targets/dart/runtime/bundled.dart new file mode 100644 index 00000000..6d3058b4 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dart/runtime/bundled.dart @@ -0,0 +1,32 @@ + +/// The library bundled for the running platform under `native//`, +/// relative to the package root, or null when none is bundled for it. +String? get _bundledLibrary => switch (Abi.current()) { +{{BUNDLED_CASES}} _ => null, + }; + +/// The packaged library's root directory (the parent of its `lib/`), or null +/// when the package can't be resolved (a compiled executable, say). +String? _packageRoot() { + try { + final uri = Isolate.resolvePackageUriSync( + Uri.parse('package:{{PACKAGE}}/{{PACKAGE}}.dart')); + if (uri == null || !uri.isScheme('file')) return null; + return File.fromUri(uri).parent.parent.path; + } on UnsupportedError { + return null; + } +} + +/// Opens the bundled library from the package, then from the working +/// directory (where an app may have copied `native/`), or returns null. +DynamicLibrary? _openBundled() { + final relative = _bundledLibrary; + if (relative == null) return null; + for (final root in [_packageRoot(), Directory.current.path]) { + if (root == null) continue; + final path = '$root${Platform.pathSeparator}$relative'; + if (File(path).existsSync()) return _open(path); + } + return null; +} diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/callbacks.dart b/crates/weaveffi-cli/src/targets/dart/runtime/callbacks.dart similarity index 76% rename from crates/weaveffi-gen/src/targets/dart/runtime/callbacks.dart rename to crates/weaveffi-cli/src/targets/dart/runtime/callbacks.dart index 88bfca02..40b517a4 100644 --- a/crates/weaveffi-gen/src/targets/dart/runtime/callbacks.dart +++ b/crates/weaveffi-cli/src/targets/dart/runtime/callbacks.dart @@ -7,11 +7,14 @@ // // Methods that return a value are `NativeCallable.isolateLocal` trampolines: // they run synchronously, so the producer must call them on this isolate's -// thread while a call from Dart is in progress. Void methods and `free` are -// `NativeCallable.isolateGroupBound` forwarders, which may run on any thread: -// they copy their arguments into a message for this isolate (borrowed bytes -// are copied before the forwarder returns) and the implementation runs later -// on the event loop, in the zone that passed the callback. +// thread while a call from Dart is in progress (the VM aborts the process +// otherwise, and an entry point that may run on any thread can't re-enter +// the isolate to run the method). Void methods and `free` are +// `NativeCallable.isolateGroupBound` forwarders, which may run on any +// thread: they copy their arguments into a message for this isolate +// (borrowed bytes are copied before the forwarder returns) and the +// implementation runs later on the event loop, in the zone that passed the +// callback. /// Where forwarders send messages: the allocator they build a message with /// and the isolate port that receives it. Forwarders run outside the @@ -219,18 +222,54 @@ final _errorSet = _lib.lookupFunction< Void Function(Pointer<_Error>, Int32, Pointer), void Function(Pointer<_Error>, int, Pointer)>('{{PREFIX}}_error_set'); -/// Reports an implementation's exception to the producer through -/// `{{PREFIX}}_error_set` with the foreign code (-4). Never throws: it runs -/// in the catch path of a trampoline the exception must not unwind through. -void _foreignError(Pointer<_Error> outErr, Object error) { +final _errorSetPayload = _lib.lookupFunction< + Void Function(Pointer<_Error>, Pointer, Size), + void Function( + Pointer<_Error>, Pointer, int)>('{{PREFIX}}_error_set_payload'); + +final _alloc = _lib.lookupFunction Function(Size), + Pointer Function(int)>('{{PREFIX}}_alloc', isLeaf: true); + +/// Reports a failed callback method to the producer through [outErr]: +/// `{{PREFIX}}_error_set` copies [message], and `{{PREFIX}}_error_set_payload` +/// copies a domain code's encoded fields. Never throws: it runs in the catch +/// path of a trampoline an exception must not unwind through. +void _failCallback( + Pointer<_Error> outErr, int code, String message, Uint8List? payload) { if (outErr == nullptr) return; - Pointer message = nullptr; + final text = message.toNativeUtf8(allocator: calloc); + try { + _errorSet(outErr, code, text); + } finally { + calloc.free(text); + } + if (payload == null || payload.isEmpty) return; + final bytes = calloc(payload.length); try { - message = error.toString().toNativeUtf8(allocator: calloc); - _errorSet(outErr, NativeException.foreignCode, message); + bytes.asTypedList(payload.length).setAll(0, payload); + _errorSetPayload(outErr, bytes, payload.length); + } finally { + calloc.free(bytes); + } +} + +/// Reports an implementation's exception that isn't a domain error of the +/// method with the foreign code (-4) and the exception's text. +void _foreignError(Pointer<_Error> outErr, Object error) { + try { + _failCallback(outErr, NativeException.foreignCode, '$error', null); } catch (_) { _errorSet(outErr, NativeException.foreignCode, nullptr); - } finally { - if (message != nullptr) calloc.free(message); } } + +/// Hands [bytes] to the producer through a method's `out_ptr`/`out_len` +/// slots as a run allocated with `{{PREFIX}}_alloc`, which the producer +/// adopts and frees. An empty run is null with length 0. +void _handOver( + List bytes, Pointer> outPtr, Pointer outLen) { + final run = bytes.isEmpty ? nullptr : _alloc(bytes.length); + if (run != nullptr) run.asTypedList(bytes.length).setAll(0, bytes); + outPtr.value = run; + outLen.value = bytes.length; +} diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/cancel.dart b/crates/weaveffi-cli/src/targets/dart/runtime/cancel.dart similarity index 100% rename from crates/weaveffi-gen/src/targets/dart/runtime/cancel.dart rename to crates/weaveffi-cli/src/targets/dart/runtime/cancel.dart diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/codec.dart b/crates/weaveffi-cli/src/targets/dart/runtime/codec.dart similarity index 68% rename from crates/weaveffi-gen/src/targets/dart/runtime/codec.dart rename to crates/weaveffi-cli/src/targets/dart/runtime/codec.dart index f58f6e9f..b3d95da5 100644 --- a/crates/weaveffi-gen/src/targets/dart/runtime/codec.dart +++ b/crates/weaveffi-cli/src/targets/dart/runtime/codec.dart @@ -95,6 +95,27 @@ final class _BufferWriter { _buf.setRange(_len, _len + v.length, v); _len += v.length; } + + void writeOptional(T? v, void Function(T) write) { + writeFlag(v != null); + if (v != null) write(v); + } + + void writeList(List v, void Function(T) write) { + writeLength(v.length); + for (final e in v) { + write(e); + } + } + + void writeMap( + Map v, void Function(K) writeKey, void Function(V) writeValue) { + writeLength(v.length); + for (final e in v.entries) { + writeKey(e.key); + writeValue(e.value); + } + } } final class _BufferReader { @@ -158,6 +179,24 @@ final class _BufferReader { return Uint8List.fromList(Uint8List.sublistView(_buf, at, at + n)); } + T? readOptional(T Function() read) => + readFlag() ? read() : null; + + // Collections grow as they decode, so a corrupt count can't force a huge + // allocation up front. + List readList(T Function() read) => + [for (var i = readLength(); i > 0; i--) read()]; + + Map readMap(K Function() readKey, V Function() readValue) { + final map = {}; + for (var i = readLength(); i > 0; i--) { + final key = readKey(); + if (map.containsKey(key)) _bufferError('duplicate map key $key'); + map[key] = readValue(); + } + return map; + } + void expectEnd() { if (_pos != _buf.length) _bufferError('trailing bytes'); } @@ -177,3 +216,33 @@ T _decode(Uint8List bytes, T Function(_BufferReader) read) { reader.expectEnd(); return value; } + +// Records and rich enums compare by value: lists, maps, and byte arrays +// element by element, and everything else with `==` (interface wrappers by +// identity). + +bool _deepEquals(Object? a, Object? b) { + if (identical(a, b)) return true; + if (a is List && b is List) { + if (a.length != b.length) return false; + for (var i = 0; i < a.length; i++) { + if (!_deepEquals(a[i], b[i])) return false; + } + return true; + } + if (a is Map && b is Map) { + if (a.length != b.length) return false; + for (final key in a.keys) { + if (!b.containsKey(key) || !_deepEquals(a[key], b[key])) return false; + } + return true; + } + return a == b; +} + +int _deepHash(Object? v) => switch (v) { + List() => Object.hashAll(v.map(_deepHash)), + Map() => Object.hashAllUnordered( + v.entries.map((e) => Object.hash(_deepHash(e.key), _deepHash(e.value)))), + _ => v.hashCode, + }; diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/core.dart b/crates/weaveffi-cli/src/targets/dart/runtime/core.dart similarity index 65% rename from crates/weaveffi-gen/src/targets/dart/runtime/core.dart rename to crates/weaveffi-cli/src/targets/dart/runtime/core.dart index 5bb37d74..faf6d464 100644 --- a/crates/weaveffi-gen/src/targets/dart/runtime/core.dart +++ b/crates/weaveffi-cli/src/targets/dart/runtime/core.dart @@ -12,10 +12,8 @@ final class _Error extends Struct { external int payloadLen; } -final _errorClear = - _lib.lookupFunction), void Function(Pointer<_Error>)>( - '{{PREFIX}}_error_clear', - isLeaf: true); +final _errorClear = _lib.lookupFunction), + void Function(Pointer<_Error>)>('{{PREFIX}}_error_clear', isLeaf: true); final _freeBytes = _lib.lookupFunction, Size), void Function(Pointer, int)>('{{PREFIX}}_free_bytes', isLeaf: true); @@ -26,9 +24,9 @@ final _freeBytes = _lib.lookupFunction, Size), final Pointer<_Error> _err = calloc<_Error>(); final Pointer _outLen = calloc(); -/// A failure the native library reported. Domain errors extend this class -/// with a positive [code]; the negative runtime codes are the constants -/// below. +/// A failure the native library reported to a function declared to throw. +/// Domain errors extend this class with a positive [code]; a negative code is +/// one of the runtime constants below. class NativeException implements Exception { /// Creates an exception carrying [code] and [message]. NativeException(this.code, this.message); @@ -39,10 +37,10 @@ class NativeException implements Exception { /// The producer panicked; [message] carries the panic text. static const int panicCode = -2; - /// An argument couldn't be lifted by the producer. + /// An argument or a callback's return couldn't be lifted by the producer. static const int marshalCode = -3; - /// A callback implementation threw; [message] carries its text. + /// A callback implementation failed; [message] carries its text. static const int foreignCode = -4; /// The call was cancelled (see [CancelledException]). @@ -66,24 +64,51 @@ class CancelledException extends NativeException { : super(NativeException.cancelledCode, message); } -typedef _ErrorMapper = NativeException Function( +/// A function that isn't declared to throw failed anyway. That's a bug in +/// the native library (a panic, an argument it couldn't lift, a callback +/// failure it let through), so it's an [Error], not an [Exception]. +final class NativeError extends Error { + /// Creates the error a failed call reported. + NativeError(this.code, this.message); + + /// The runtime code the producer reported (see [NativeException]). + final int code; + + /// The producer's message. + final String message; + + @override + String toString() => 'NativeError($code): $message'; +} + +/// Maps a reported code, message, and payload onto the object to throw. +typedef _ErrorMapper = Object Function( int code, String message, Uint8List payload); -/// Maps a runtime failure (or an undeclared code) onto its exception. -NativeException _runtimeError(int code, String message, Uint8List payload) => +/// The mapper of a function that can't fail: every code but cancellation +/// is a [NativeError]. +Object _trap(int code, String message, Uint8List payload) => + code == NativeException.cancelledCode + ? CancelledException(message) + : NativeError(code, message); + +/// Maps a runtime code (or one the domain doesn't declare) onto its +/// exception, for a function declared to throw. +NativeException _runtimeException(int code, String message, Uint8List payload) => code == NativeException.cancelledCode ? CancelledException(message) : NativeException(code, message); -/// Builds the exception [err] reports, using [map] for domain codes. -NativeException _readError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error [err] reports, using [map]. +Object _readError(Pointer<_Error> err, _ErrorMapper map) { final e = err.ref; final message = e.message == nullptr ? '' : e.message.toDartString(); return map(e.code, message, _copyBytes(e.payloadPtr, e.payloadLen)); } /// Throws the error a synchronous call left in [err], clearing the slot. -void _check(Pointer<_Error> err, [_ErrorMapper map = _runtimeError]) { +/// [map] is the domain's mapper for a function declared to throw. +void _check(Pointer<_Error> err, [_ErrorMapper map = _trap]) { if (err.ref.code == 0) return; final error = _readError(err, map); _errorClear(err); diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/iterator.dart b/crates/weaveffi-cli/src/targets/dart/runtime/iterator.dart similarity index 100% rename from crates/weaveffi-gen/src/targets/dart/runtime/iterator.dart rename to crates/weaveffi-cli/src/targets/dart/runtime/iterator.dart diff --git a/crates/weaveffi-cli/src/targets/dart/runtime/loader.dart b/crates/weaveffi-cli/src/targets/dart/runtime/loader.dart new file mode 100644 index 00000000..81e4921b --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dart/runtime/loader.dart @@ -0,0 +1,121 @@ +// ── Library loading ── +// The native library opens on first use and is verified before any other +// lookup: it must implement the C ABI revision these bindings were generated +// against and carry every declaration they were generated with, unchanged. + +/// The native library every binding in this file calls. +final DynamicLibrary _lib = _verifyLibrary(_openLibrary()); + +/// The native library couldn't be loaded, or it doesn't match these +/// bindings: a different C ABI revision, or a declaration that's missing or +/// changed. Regenerate the bindings from the library you ship. +final class NativeLibraryError extends Error { + /// Creates a load failure described by [message]. + NativeLibraryError(this.message); + + /// What failed, naming the library or declaration. + final String message; + + @override + String toString() => 'NativeLibraryError: $message'; +} + +/// The platform file name of the library on the system search path. +String get _libraryFile => Platform.isMacOS + ? '{{MACOS_FILE}}' + : Platform.isWindows + ? '{{WINDOWS_FILE}}' + : '{{LINUX_FILE}}'; + +DynamicLibrary _openLibrary() { + // An explicit path wins, so a caller can point at any build artifact. + final override = Platform.environment['{{LIBRARY_ENV}}']; + if (override != null && override.isNotEmpty) { + return _open(override); + } + // iOS links native code into the app binary. + if (Platform.isIOS) return DynamicLibrary.process(); +{{BUNDLED_LOOKUP}} try { + return DynamicLibrary.open(_libraryFile); + } on ArgumentError catch (e) { + // A macOS app may link the library into its executable instead. + if (Platform.isMacOS) return DynamicLibrary.process(); + throw NativeLibraryError("can't open '$_libraryFile' ($e); set " + '{{LIBRARY_ENV}} to its path'); + } +} + +DynamicLibrary _open(String path) { + try { + return DynamicLibrary.open(path); + } on ArgumentError catch (e) { + throw NativeLibraryError("can't open '$path': $e"); + } +} + +/// The C ABI revision these bindings were generated against. +const int _abiVersion = {{ABI_VERSION}}; + +/// One `{{PREFIX}}_contract_entry` of a module's contract table. +final class _ContractEntry extends Struct { + @Uint64() + external int id; + @Uint64() + external int hash; +} + +DynamicLibrary _verifyLibrary(DynamicLibrary lib) { + final int Function() abiVersion; + try { + abiVersion = lib.lookupFunction( + '{{PREFIX}}_abi_version'); + } on ArgumentError { + throw NativeLibraryError( + "the native library '{{LIBRARY}}' isn't loaded; set {{LIBRARY_ENV}} " + 'to its path'); + } + final found = abiVersion(); + if (found != _abiVersion) { + throw NativeLibraryError('{{NAME}}: these bindings expect C ABI revision ' + '$_abiVersion, but the loaded library implements revision $found'); + } + for (final (symbol, expected) in _contracts) { + _checkContract(lib, symbol, expected); + } + return lib; +} + +/// Checks that every `(id, hash, path)` entry in [expected] is in the table +/// [symbol] returns with an equal hash. Entries only the library has are +/// fine: they're declarations added after these bindings were generated. +void _checkContract( + DynamicLibrary lib, String symbol, List<(int, int, String)> expected) { + final Pointer<_ContractEntry> Function(Pointer) table; + try { + table = lib.lookupFunction Function(Pointer), + Pointer<_ContractEntry> Function(Pointer)>(symbol); + } on ArgumentError { + throw NativeLibraryError( + "{{NAME}}: the loaded library doesn't export '$symbol'"); + } + final hashes = {}; + final len = calloc(); + try { + final entries = table(len); + for (var i = 0; i < len.value; i++) { + hashes[entries[i].id] = entries[i].hash; + } + } finally { + calloc.free(len); + } + for (final (id, hash, path) in expected) { + final found = hashes[id]; + if (found == null) { + throw NativeLibraryError('{{NAME}}: $path is missing from the library'); + } + if (found != hash) { + throw NativeLibraryError( + '{{NAME}}: $path changed since these bindings were generated'); + } + } +} diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/object.dart b/crates/weaveffi-cli/src/targets/dart/runtime/object.dart similarity index 100% rename from crates/weaveffi-gen/src/targets/dart/runtime/object.dart rename to crates/weaveffi-cli/src/targets/dart/runtime/object.dart diff --git a/crates/weaveffi-cli/src/targets/dart/tests.rs b/crates/weaveffi-cli/src/targets/dart/tests.rs new file mode 100644 index 00000000..bf607172 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dart/tests.rs @@ -0,0 +1,559 @@ +//! Unit tests: render a small API that exercises every C ABI revision 4 +//! shape the Dart bindings distinguish and assert the key pieces of each +//! contract. + +use camino::Utf8Path; +use weaveffi_model::contract::entries; +use weaveffi_model::ir::Api; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; + +use crate::backend::LanguageBackend; +use crate::package::{FileContent, PackageContext}; +use crate::platform::{BinarySet, NativeBinary, Platform}; + +use super::{DartConfig, DartGenerator, MIN_DART_SDK}; + +/// One module with an error domain carrying a field; a C-style enum; a +/// record carrying objects; an interface with a `new` constructor, a +/// method, and a throwing static; nullable objects; an iterator of objects; +/// a cancellable async function; composites; and a callback interface whose +/// methods take every argument family and return a direct value, an enum, a +/// string, a record, an object, and an optional object, one of them +/// `throws`, plus a void method, passed both required and optional. +const FIXTURE: &str = r#" +version: "0.11.0" +modules: + - name: shop + errors: + name: ShopError + codes: + - { name: OutOfStock, code: 1, message: "out of stock", fields: [{ name: sku, type: string }] } + - { name: Closed, code: 2, message: "closed" } + enums: + - name: Mood + variants: [{ name: Happy, value: 0 }, { name: Grumpy, value: 1 }] + structs: + - name: Order + fields: + - { name: cart, type: Cart } + - { name: history, type: "[Cart]" } + - { name: note, type: "string?" } + interfaces: + - name: Cart + constructors: + - { name: new, params: [{ name: owner, type: string }] } + methods: + - { name: add, params: [{ name: sku, type: string }], return: bool } + statics: + - { name: restore, params: [{ name: id, type: i64 }], return: Cart, throws: true } + callback_interfaces: + - name: Watcher + methods: + - name: on_event + params: + - { name: name, type: string } + - { name: count, type: i32 } + - { name: order, type: Order } + - { name: cart, type: Cart } + return: bool + - { name: mood, return: Mood } + - { name: label, return: string, throws: true } + - { name: latest, return: "Order?" } + - { name: favorite, return: Cart } + - { name: maybe, return: "Cart?" } + - { name: on_done, params: [{ name: summary, type: string }, { name: cart, type: "Cart?" }] } + functions: + - { name: find_cart, params: [{ name: current, type: "Cart?" }], return: "Cart?" } + - { name: all_carts, return: "iter" } + - { name: watch, params: [{ name: watcher, type: Watcher }] } + - { name: maybe_watch, params: [{ name: watcher, type: "Watcher?" }] } + - { name: echo, params: [{ name: text, type: string }], return: string } + - { name: tally, params: [{ name: counts, type: "{string:[i32?]}" }], return: "[Order]" } + - { name: wait, params: [{ name: ms, type: i64 }], return: i64, async: true, cancellable: true } + - { name: checkout, params: [{ name: order, type: Order }], return: Order, throws: true } +"#; + +fn model() -> Model { + let api: Api = serde_yaml::from_str(FIXTURE).unwrap(); + validate(&api, &Identity::named("shop_kit"), None).unwrap() +} + +fn render() -> String { + let files = DartGenerator.files(&model(), Utf8Path::new("out"), &DartConfig::default()); + let paths: Vec<&str> = files.iter().map(|f| f.path.as_str()).collect(); + assert_eq!( + paths, + [ + "out/dart/lib/shop_kit.dart", + "out/dart/pubspec.yaml", + "out/dart/README.md" + ] + ); + files[0].contents.clone() +} + +fn assert_has(out: &str, needle: &str) { + assert!(out.contains(needle), "missing `{needle}` in:\n{out}"); +} + +/// The packaged library of the fixture, bundling every platform. +fn packaged() -> String { + let mut binaries = BinarySet::new("shop_kit"); + for p in Platform::ALL { + binaries.insert(NativeBinary::new(p, format!("/prebuilt/{}/lib", p.id()))); + } + let ctx = PackageContext::new(&binaries); + let artifacts = DartGenerator + .package(&model(), &ctx, &DartConfig::default()) + .expect("dart packages"); + assert_eq!(artifacts.len(), 1); + assert_eq!(artifacts[0].path, "dart/shop_kit"); + let files = &artifacts[0].files; + let bundled: Vec<&str> = files + .iter() + .filter(|f| f.is_binary()) + .map(|f| f.path.as_str()) + .collect(); + assert_eq!(bundled.len(), Platform::DESKTOP.len()); + for p in [ + Platform::AndroidArm64, + Platform::AndroidX64, + Platform::Wasm32, + ] { + assert!( + !bundled.iter().any(|b| b.contains(p.id())), + "{} bundled", + p.id() + ); + } + files + .iter() + .find(|f| f.path.as_str() == "lib/shop_kit.dart") + .and_then(|f| match &f.content { + FileContent::Text(s) => Some(s.clone()), + _ => None, + }) + .expect("packaged source") +} + +#[test] +fn names_come_from_the_identity() { + let model = model(); + let files = DartGenerator.files(&model, Utf8Path::new("out"), &DartConfig::default()); + assert_has(&files[1].contents, "name: shop_kit\n"); + assert_has( + &files[2].contents, + "import 'package:shop_kit/shop_kit.dart';", + ); + + let config = DartConfig { + name: Some("custom".into()), + ..DartConfig::default() + }; + let files = DartGenerator.files(&model, Utf8Path::new("out"), &config); + assert_eq!(files[0].path.as_str(), "out/dart/lib/custom.dart"); + assert_has(&files[1].contents, "name: custom\n"); + + let out = render(); + assert_has(&out, "Platform.environment['SHOP_KIT_LIBRARY']"); + assert_has(&out, "? 'libshop_kit.dylib'"); + assert_has(&out, "? 'shop_kit.dll'"); + assert_has(&out, ": 'libshop_kit.so';"); + assert_has(&out, "if (Platform.isIOS) return DynamicLibrary.process();"); + assert!( + !out.contains("_openBundled"), + "unpackaged bindings look for bundles" + ); + assert!(!out.to_lowercase().contains("weaveffi_"), "branded symbol"); +} + +#[test] +fn pubspec_states_the_minimum_sdk() { + let files = DartGenerator.files(&model(), Utf8Path::new("out"), &DartConfig::default()); + assert_has( + &files[1].contents, + &format!("sdk: '>={MIN_DART_SDK} <4.0.0'"), + ); + assert_has( + &files[2].contents, + &format!("Dart SDK `>={MIN_DART_SDK} <4.0.0`"), + ); +} + +#[test] +fn load_checks_embed_every_contract_entry() { + let model = model(); + let out = render(); + assert_has(&out, "const int _abiVersion = 4;"); + assert_has(&out, "'shop_kit_abi_version'"); + assert_has(&out, " ('shop_kit_shop_contract', [\n"); + let root = model.roots().next().expect("one root"); + let expected = entries(&model, root); + assert!(expected.len() > 10, "every declaration has an entry"); + for e in expected { + assert_has( + &out, + &format!(" (0x{:016x}, 0x{:016x}, '{}'),\n", e.id, e.hash, e.path), + ); + } + assert_has(&out, "$path is missing from the library"); + assert_has(&out, "$path changed since these bindings were generated"); + assert_has(&out, "final class NativeLibraryError extends Error {"); + assert!( + !out.contains("checksum"), + "the revision 3 checksum survives" + ); +} + +#[test] +fn strings_cross_as_pointer_and_length() { + let out = render(); + assert_has( + &out, + "Pointer Function(Pointer, Size, Pointer, Pointer<_Error>)", + ); + assert_has(&out, "final _textBytes = utf8.encode(text);"); + assert_has( + &out, + "_shopKitShopEcho(_stage(_arena, _textBytes), _textBytes.length, _outLen, _err)", + ); + assert_has(&out, "return _takeString(_result, _outLen.value);"); + assert_has(&out, "'shop_kit_free_bytes'"); + assert!(!out.contains("dealloc"), "the revision 3 dealloc survives"); +} + +#[test] +fn composites_share_one_codec_per_type() { + let out = render(); + let name = "_packMapOfStringToListOfOptionalOfI32"; + assert_eq!(out.matches(&format!("void {name}(")).count(), 1); + assert_has( + &out, + " w.writeMap(v, (k) => w.writeString(k), (e) => _packListOfOptionalOfI32(w, e));", + ); + assert_has( + &out, + &format!("final _countsBytes = _encode(counts, {name});"), + ); + assert_has( + &out, + "return _decode(_takeBytes(_result, _outLen.value), _unpackListOfOrder);", + ); + assert_has(&out, " _packListOfCart(w, v.history);"); + assert_has(&out, " history: _unpackListOfCart(r),"); + assert!( + !out.contains("for (var i = r."), + "a call site inlines a loop" + ); +} + +#[test] +fn records_compare_by_value() { + let out = render(); + assert_has(&out, "final class Order {"); + assert_has(&out, " _deepEquals(history, other.history) &&"); + assert_has(&out, " note == other.note;"); + assert_has( + &out, + "int get hashCode => Object.hashAll([Order, cart, _deepHash(history), note]);", + ); + assert_has( + &out, + "String toString() => 'Order(cart: $cart, history: $history, note: $note)';", + ); +} + +#[test] +fn interface_class_owns_one_reference() { + let out = render(); + assert_has(&out, "final class Cart extends _NativeObject {"); + assert_has(&out, "('shop_kit_shop_Cart_clone')"); + assert_has(&out, "('shop_kit_shop_Cart_destroy')"); + assert_has( + &out, + "NativeFinalizer get _finalizer => _shopKitShopCartDestroyFinalizer;", + ); + // A dispose during an in-flight call defers the release. + assert_has(&out, "if (_calls == 0) _release();"); + assert_has(&out, "if (--_calls == 0 && _disposed) _release();"); + assert_has(&out, "factory Cart(String owner) {"); + assert_has(&out, "return Cart._(_result);"); + assert_has(&out, "final _self = _enter();"); + assert_has(&out, "static Cart restore(int id) {"); +} + +#[test] +fn throwing_calls_map_the_domain_and_others_trap() { + let out = render(); + assert_has(&out, "sealed class ShopException extends NativeException {"); + assert_has( + &out, + "final class OutOfStockException extends ShopException {", + ); + assert_has( + &out, + "return _decode(payload, (r) => OutOfStockException(r.readString(), message));", + ); + assert_has(&out, "return _runtimeException(code, message, payload);"); + // `restore` throws; `echo` can't, so a failure there is a trap. + let restore = &out[out.find("static Cart restore").unwrap()..]; + assert_has(&restore[..300], "_check(_err, _mapShopException);"); + let echo = &out[out.find("String echo(String text)").unwrap()..]; + assert_has(&echo[..300], "_check(_err);"); + assert_has(&out, "final class NativeError extends Error {"); + assert_has(&out, ": NativeError(code, message);"); +} + +#[test] +fn nullable_objects_are_borrowed_and_adopted() { + let out = render(); + assert_has(&out, "Cart? findCart(Cart? current) {"); + assert_has( + &out, + "final _currentPtr = current == null ? nullptr : _borrow(_arena, current);", + ); + assert_has(&out, "return _result == nullptr ? null : Cart._(_result);"); +} + +#[test] +fn objects_inside_records_are_cloned_tokens() { + let out = render(); + assert_has(&out, "w.writeU64(v.cart._cloneRef().address);"); + assert_has( + &out, + "cart: Cart._(Pointer.fromAddress(r.readU64())),", + ); +} + +#[test] +fn object_iterator_adopts_each_element() { + let out = render(); + assert_has(&out, "Iterable allCarts() sync* {"); + assert_has( + &out, + "_shopKitShopAllCartsIteratorNext(_iter, _outItem.cast(), _err)", + ); + assert_has(&out, "yield Cart._(_outItem.cast>().value);"); + assert_has( + &out, + "_shopKitShopAllCartsIteratorDestroyFinalizer.attach(_anchor, _iter, detach: _anchor);", + ); + assert_has(&out, "_shopKitShopAllCartsIteratorDestroy(_iter);"); +} + +#[test] +fn cancellable_async_binds_a_native_token() { + let out = render(); + assert_has( + &out, + "Future wait(int ms, {CancelToken? cancelToken}) {", + ); + assert_has(&out, "final _cancel = _NativeCancel.bind(cancelToken);"); + assert_has(&out, "NativeCallable<_ShopKitShopWaitCallback>.listener(("); + assert_has( + &out, + "_shopKitShopWait(ms, _cancel?.pointer ?? nullptr, _callback.nativeFunction, nullptr);", + ); + assert_has( + &out, + "if (error != nullptr) throw _takeAsyncError(error, _trap);", + ); + assert_has(&out, "'shop_kit_cancel_token_create'"); + assert_has(&out, "class CancelledException extends NativeException {"); +} + +#[test] +fn callback_vtable_starts_with_the_header() { + let out = render(); + assert_has( + &out, + "final class _WatcherVtable extends Struct {\n @Uint32()\n external int size;\n \ + @Uint32()\n external int flags;\n \ + external Pointer)>> free;\n \ + external Pointer> onEvent;", + ); + assert_has( + &out, + " ..size = sizeOf<_WatcherVtable>()\n ..flags = 0\n ..free = _callbackFree\n", + ); +} + +#[test] +fn callback_arguments_are_copied_or_adopted() { + let out = render(); + // The object first, then the buffer (which may carry tokens). + assert_has( + &out, + " final _cart = Cart._(cart);\n \ + final _order = _decode(_copyBytes(orderPtr, orderLen), _unpackOrder);\n \ + return (_callbackTarget(ctx) as Watcher).onEvent(_readString(namePtr, nameLen), count, _order, _cart);", + ); + assert_has( + &out, + " } catch (e) {\n _foreignError(outErr, e);\n }\n return false;", + ); +} + +#[test] +fn callback_returns_cover_every_family() { + let out = render(); + assert_has( + &out, + "return (_callbackTarget(ctx) as Watcher).mood().value;", + ); + assert_has(&out, "_handOver(utf8.encode(result), outPtr, outLen);"); + assert_has( + &out, + "_handOver(_encode(result, _packOptionalOfOrder), outPtr, outLen);", + ); + assert_has(&out, " return result._cloneRef();\n"); + assert_has(&out, " return result?._cloneRef() ?? nullptr;\n"); + assert_has( + &out, + "..mood = _pin(NativeCallable<_ShopKitShopWatcherMood>.isolateLocal(_watcherMood, exceptionalReturn: 0))", + ); + assert_has( + &out, + "..favorite = _pin(NativeCallable<_ShopKitShopWatcherFavorite>.isolateLocal(_watcherFavorite))", + ); + assert_has( + &out, + "typedef _ShopKitShopWatcherLabel = Void Function(Pointer, Pointer>, Pointer, Pointer<_Error>);", + ); + assert_has(&out, "'shop_kit_alloc'"); + assert_has(&out, "outPtr.value = run;"); +} + +#[test] +fn throwing_callback_methods_report_the_domain_error() { + let out = render(); + assert_has( + &out, + " } on ShopException catch (e) {\n \ + _failCallback(outErr, e.code, e.message, _payloadOfShopException(e));\n \ + } catch (e) {\n _foreignError(outErr, e);\n }", + ); + assert_eq!( + out.matches("} on ShopException catch (e) {").count(), + 1, + "only `label` throws" + ); + assert_has( + &out, + "Uint8List? _payloadOfShopException(ShopException e) {", + ); + assert_has( + &out, + " case OutOfStockException():\n final w = _BufferWriter();\n \ + w.writeString(e.sku);\n return w.takeBytes();", + ); + assert_has(&out, " case ClosedException():\n return null;"); + assert_has(&out, "'shop_kit_error_set_payload'"); +} + +#[test] +fn void_methods_are_forwarded_from_any_thread() { + let out = render(); + assert_has( + &out, + "..onDone = _pin(NativeCallable<_ShopKitShopWatcherOnDone>.isolateGroupBound(_watcherOnDone));", + ); + assert_has( + &out, + " _CallbackMessage(ctx, 6, 2)\n ..bytes(summaryPtr, summaryLen)\n \ + ..pointer(cart)\n ..send();", + ); + assert_has( + &out, + " final a3 = a3Address == 0 ? null : Cart._(Pointer.fromAddress(a3Address));\n \ + final a2 = utf8.decode(message[2]! as Uint8List);\n target.onDone(a2, a3);", + ); +} + +#[test] +fn callback_parameters_register_the_implementation() { + let out = render(); + assert_has( + &out, + "final _watcherCtx = _registerCallback(watcher, _watcherDispatch);\n \ + _shopKitShopWatch(_watcherCtx, _watcherVtable.cast(), _err);", + ); + // An optional callback passes a null vtable for none. + assert_has( + &out, + "final _watcherCtx = watcher == null ? nullptr : _registerCallback(watcher, _watcherDispatch);", + ); + assert_has( + &out, + "_shopKitShopMaybeWatch(_watcherCtx, watcher == null ? nullptr : _watcherVtable.cast(), _err);", + ); + // No call may be a leaf call when a callback can re-enter Dart. + assert!(!out.contains("isLeaf: true);\n\nfinal _shopKitShop")); +} + +#[test] +fn calls_are_leaf_calls_without_callbacks() { + let api: Api = serde_yaml::from_str( + r#" +version: "0.11.0" +modules: + - name: math + functions: + - { name: add, params: [{ name: a, type: i32 }, { name: b, type: i32 }], return: i32 } +"#, + ) + .unwrap(); + let model = validate(&api, &Identity::named("calc"), None).unwrap(); + let files = DartGenerator.files(&model, Utf8Path::new("out"), &DartConfig::default()); + let out = &files[0].contents; + assert_has(out, "Int32 Function(Int32, Int32, Pointer<_Error>),"); + assert_has(out, "('calc_math_add', isLeaf: true);"); + assert!( + !out.contains("dart:isolate"), + "isolate glue without callbacks" + ); +} + +#[test] +fn packaged_loader_resolves_the_bundle_from_the_package() { + let out = packaged(); + assert_has(&out, "import 'dart:io' show Directory, File, Platform;"); + assert_has( + &out, + " Abi.macosArm64 => 'native/darwin-arm64/libshop_kit.dylib',", + ); + assert_has( + &out, + " Abi.windowsX64 => 'native/windows-x64/shop_kit.dll',", + ); + assert_has( + &out, + " Abi.linuxArm64 => 'native/linux-arm64/libshop_kit.so',", + ); + assert_has(&out, "Uri.parse('package:shop_kit/shop_kit.dart')"); + assert_has( + &out, + "for (final root in [_packageRoot(), Directory.current.path]) {", + ); + assert_has( + &out, + " final bundled = _openBundled();\n if (bundled != null) return bundled;\n", + ); + assert_has(&out, "Platform.environment['SHOP_KIT_LIBRARY']"); +} + +#[test] +fn every_line_is_free_of_trailing_whitespace() { + for out in [render(), packaged()] { + for (i, line) in out.lines().enumerate() { + assert_eq!( + line.trim_end(), + line, + "trailing whitespace on line {}", + i + 1 + ); + } + } +} diff --git a/crates/weaveffi-gen/src/targets/dart/types.rs b/crates/weaveffi-cli/src/targets/dart/types.rs similarity index 72% rename from crates/weaveffi-gen/src/targets/dart/types.rs rename to crates/weaveffi-cli/src/targets/dart/types.rs index 2eb595ed..af6f0bce 100644 --- a/crates/weaveffi-gen/src/targets/dart/types.rs +++ b/crates/weaveffi-cli/src/targets/dart/types.rs @@ -3,15 +3,15 @@ //! to user-chosen IDL names before they land in generated Dart. use crate::lang; -use crate::utils::{local_type_name, wrapper_name}; use heck::{ToLowerCamelCase, ToUpperCamelCase}; use weaveffi_model::abi::CType; -use weaveffi_model::model::Ty; +use weaveffi_model::ty::{Prim, Ty}; /// Type names the generated library declares or uses unqualified. A user /// type with one of these names would shadow or collide with it, so it gains /// a trailing `_` like a keyword. const RESERVED_TYPES: &[&str] = &[ + "Abi", "Arena", "ArgumentError", "Bool", @@ -19,10 +19,13 @@ const RESERVED_TYPES: &[&str] = &[ "CancelToken", "CancelledException", "Completer", + "Directory", "Double", "DynamicLibrary", "Endian", + "Error", "Exception", + "File", "Finalizable", "Float", "Function", @@ -32,14 +35,17 @@ const RESERVED_TYPES: &[&str] = &[ "Int64", "Int8", "IntPtr", + "Isolate", "Iterable", "List", "Map", "NativeApi", "NativeCallable", + "NativeError", "NativeException", "NativeFinalizer", "NativeFunction", + "NativeLibraryError", "Never", "Object", "Platform", @@ -55,6 +61,7 @@ const RESERVED_TYPES: &[&str] = &[ "Uint8", "Uint8List", "Union", + "Uri", "Utf8", "Void", "Zone", @@ -68,16 +75,31 @@ pub(crate) fn dart_ident(name: &str) -> String { lang::escape_ident(&name.to_lower_camel_case(), lang::DART_KEYWORDS) } -/// The Dart spelling of a module-level function: the module path applied per -/// `strip_module_prefix`, then lowerCamelCase and keyword escaping. -pub(crate) fn dart_fn_name(module_path: &str, name: &str, strip_module_prefix: bool) -> String { - dart_ident(&wrapper_name(module_path, name, strip_module_prefix)) +/// Members every interface wrapper class declares or inherits: +/// `_NativeObject`'s `dispose` and `Object`'s members. A named constructor +/// or static can't share a name with an instance member either, so every +/// interface member is escaped against the same set. The wrapper's private +/// members start with `_`, which [`dart_ident`] never does. +const OBJECT_MEMBERS: &[&str] = &[ + "dispose", + "hashCode", + "noSuchMethod", + "runtimeType", + "toString", +]; + +/// The Dart spelling of an interface member (a named constructor, method, +/// or static): [`dart_ident`], with a trailing `_` when it would collide +/// with a member the wrapper class declares or inherits (`dispose` is +/// `dispose_`). +pub(crate) fn dart_member(name: &str) -> String { + lang::escape_member(&dart_ident(name), OBJECT_MEMBERS) } -/// The Dart class of a (possibly dot-qualified) user type: its local name in -/// UpperCamelCase, escaped when it would collide with a reserved type. +/// The Dart class of a user type: its name in UpperCamelCase, escaped when +/// it would collide with a reserved type. pub(crate) fn dart_class(name: &str) -> String { - let class = local_type_name(name).to_upper_camel_case(); + let class = name.to_upper_camel_case(); if RESERVED_TYPES.binary_search(&class.as_str()).is_ok() { format!("{class}_") } else { @@ -89,11 +111,20 @@ pub(crate) fn dart_class(name: &str) -> String { /// their two's-complement bit pattern in a Dart `int`. pub(crate) fn dart_type(ty: &Ty) -> String { match ty { - Ty::I8 | Ty::I16 | Ty::I32 | Ty::I64 | Ty::U8 | Ty::U16 | Ty::U32 | Ty::U64 => "int".into(), - Ty::F32 | Ty::F64 => "double".into(), - Ty::Bool => "bool".into(), - Ty::StringUtf8 => "String".into(), - Ty::Bytes => "List".into(), + Ty::Prim( + Prim::I8 + | Prim::I16 + | Prim::I32 + | Prim::I64 + | Prim::U8 + | Prim::U16 + | Prim::U32 + | Prim::U64, + ) => "int".into(), + Ty::Prim(Prim::F32 | Prim::F64) => "double".into(), + Ty::Prim(Prim::Bool) => "bool".into(), + Ty::Prim(Prim::String) => "String".into(), + Ty::Prim(Prim::Bytes) => "List".into(), Ty::Enum(n) | Ty::Record(n) | Ty::RichEnum(n) @@ -168,19 +199,6 @@ fn pointee_type(ct: &CType) -> String { } } -/// The `dart:ffi` type a pointer slot `ct` points at (`Int32` for an -/// `int32_t*` slot, `Pointer` for a `const uint8_t**` slot). -/// -/// # Panics -/// -/// Panics when `ct` is not a pointer. -pub(crate) fn pointee_ffi(ct: &CType) -> String { - let CType::Ptr { pointee, .. } = ct else { - unreachable!("{ct:?} is not a pointer slot") - }; - pointee_type(pointee) -} - /// The private Dart variable holding the bound function of C symbol `sym`. pub(crate) fn ffi_var(sym: &str) -> String { format!("_{}", sym.to_lower_camel_case()) @@ -191,12 +209,12 @@ pub(crate) fn ffi_typedef(c_name: &str) -> String { format!("_{}", c_name.to_upper_camel_case()) } -/// The Dart literal a callback trampoline returns when the implementation -/// threw: the zero value of the method's direct return type. -pub(crate) fn default_literal(ty: &Ty) -> &'static str { +/// The zero value of a direct type: what a callback trampoline returns when +/// its implementation threw. +pub(crate) fn zero_literal(ty: &Ty) -> &'static str { match ty { - Ty::Bool => "false", - Ty::F32 | Ty::F64 => "0.0", + Ty::Prim(Prim::Bool) => "false", + Ty::Prim(Prim::F32 | Prim::F64) => "0.0", _ => "0", } } @@ -216,9 +234,18 @@ mod tests { #[test] fn reserved_types_are_sorted_and_escaped() { assert!(RESERVED_TYPES.windows(2).all(|w| w[0] < w[1])); - assert_eq!(dart_class("kv.Store"), "Store"); + assert_eq!(dart_class("Store"), "Store"); assert_eq!(dart_class("CancelToken"), "CancelToken_"); - assert_eq!(dart_class("app.String"), "String_"); + assert_eq!(dart_class("String"), "String_"); + } + + #[test] + fn interface_members_avoid_the_wrapper_members() { + assert_eq!(dart_member("dispose"), "dispose_"); + assert_eq!(dart_member("to_string"), "toString_"); + assert_eq!(dart_member("hash_code"), "hashCode_"); + assert_eq!(dart_member("close"), "close"); + assert_eq!(dart_member("class"), "class_"); } #[test] diff --git a/crates/weaveffi-cli/src/targets/dotnet/callbacks.rs b/crates/weaveffi-cli/src/targets/dotnet/callbacks.rs new file mode 100644 index 00000000..0b5c8701 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dotnet/callbacks.rs @@ -0,0 +1,284 @@ +//! Callback-interface renderers: the consumer-facing C# `interface` plus the +//! process-wide vtable and its `[UnmanagedCallersOnly]` trampolines, +//! rendering every clause of [`CallbackProtocol`]. +//! +//! * One static vtable per callback interface, allocated once in native +//! memory and never freed, so its address is stable for the process. It +//! starts with the `{size, flags, free}` header, `size` being the struct's +//! real size as C# lays it out. +//! * `ctx` is a `GCHandle` to the implementation, released by the vtable's +//! `free` entry, so the implementation lives exactly as long as the +//! producer holds it. +//! * Trampoline arguments are received like returns, except that strings, +//! bytes, and buffers are borrowed (copied, never freed); objects are +//! adopted into wrappers the implementation owns. +//! * Returns cross per [`RetPass`]: a direct value by value, an object as a +//! fresh strong reference (`CloneHandle()`), and a string, bytes, or +//! buffer as a `{prefix}_alloc` run written to the out slots. +//! * A thrown exception never unwinds into native code: a method declared +//! `throws` reports its module's domain exception with its code and +//! payload, and any other exception is reported with the foreign error +//! code (`-4`). Trampolines run on whatever thread the producer calls +//! from. + +use crate::codegen::CodeWriter; +use heck::{ToLowerCamelCase, ToUpperCamelCase}; +use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ErrorBinding}; +use weaveffi_model::plan::{CallbackProtocol, ErrorStrategy, RetPass}; + +use crate::targets::dotnet::calls::{ + direct_to_slot, receive, write_obsolete, Own, UNMANAGED_CALLERS_ONLY, +}; +use crate::targets::dotnet::codec::write_stmt; +use crate::targets::dotnet::docs::{write_doc, write_fn_doc}; +use crate::targets::dotnet::runtime::dotnet_exception_name; +use crate::targets::dotnet::types::{ + callback_interface_cs, cs_ctype, cs_type, fn_pointer_type, safe_cs_name, vtable_class_cs, Cx, +}; + +/// Render one callback interface declared by the module at `module_path`: +/// the `public interface I{Name}` the consumer implements, then the +/// internal class hosting its vtable. `domain` is the error domain in scope +/// for the module, which methods declared `throws` report. +pub(crate) fn render_callback_interface( + w: &mut CodeWriter, + module_path: &str, + cb: &CallbackInterfaceBinding, + domain: Option<&ErrorBinding>, + cx: Cx<'_>, +) { + let domain = domain.map(dotnet_exception_name); + render_consumer_interface(w, cb, domain.as_deref(), cx); + render_vtable_class(w, cx, module_path, cb, domain.as_deref()); +} + +fn method_cs(m: &CallbackMethodBinding) -> String { + m.name.to_upper_camel_case() +} + +/// The consumer-facing interface: one method per callback method. +fn render_consumer_interface( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + domain: Option<&str>, + cx: Cx<'_>, +) { + let iface = callback_interface_cs(&cb.name); + write_doc(w, &cb.doc); + w.line("/// The native library may call any method from any thread until"); + w.line("/// it releases its last reference to the implementation. Object arguments"); + w.line("/// belong to the implementation. An exception thrown by a method fails the"); + w.line("/// native library's call in progress: a method documented to throw a domain"); + w.line("/// exception reports it with its code and fields, and any other exception"); + w.line(format!( + "/// reaches the library as .", + cx.base + )); + write_obsolete(w, &cb.deprecated); + w.line(format!("public interface {iface}")); + w.block("{", "}", |w| { + for (i, m) in cb.methods.iter().enumerate() { + if i > 0 { + w.blank(); + } + let mut params = m.params.clone(); + for p in &mut params { + p.name = p.name.to_lower_camel_case(); + } + write_fn_doc(w, &m.doc, ¶ms); + if let (true, Some(exc)) = (m.throws, domain) { + w.line(format!( + "/// Reported to the library with its code and fields." + )); + } + write_obsolete(w, &m.deprecated); + let ret = m.ret.as_ref().map(cs_type).unwrap_or_else(|| "void".into()); + let sig: Vec = params + .iter() + .map(|p| format!("{} {}", cs_type(&p.ty), safe_cs_name(&p.name))) + .collect(); + w.line(format!("{ret} {}({});", method_cs(m), sig.join(", "))); + } + }); + w.blank(); +} + +/// The internal class owning the vtable (the header, then function pointers +/// in declaration order) and the trampolines it points at. +fn render_vtable_class( + w: &mut CodeWriter, + cx: Cx<'_>, + module_path: &str, + cb: &CallbackInterfaceBinding, + domain: Option<&str>, +) { + let protocol = cb.protocol(); + let iface = callback_interface_cs(&cb.name); + let class = vtable_class_cs(module_path, &cb.name); + w.line(format!( + "/// The process-wide {} and the trampolines behind it,", + cb.vtable_tag + )); + w.line(format!( + "/// adapting an to the C ABI." + )); + w.line(format!("internal static unsafe class {class}")); + w.block("{", "}", |w| { + w.line("[StructLayout(LayoutKind.Sequential)]"); + w.line("private struct Layout"); + w.block("{", "}", |w| { + w.line("public uint Size;"); + w.line("public uint Flags;"); + w.line("public delegate* unmanaged[Cdecl] Free;"); + for m in &cb.methods { + w.line(format!( + "public {} {};", + fn_pointer_type(&m.abi_params, &m.abi_ret), + safe_cs_name(&m.name) + )); + } + }); + w.blank(); + w.line("/// The vtable's address, valid for the process lifetime."); + w.line("internal static readonly IntPtr Pointer = Allocate();"); + w.blank(); + w.line("private static IntPtr Allocate()"); + w.block("{", "}", |w| { + w.line("var vtable = (Layout*)NativeMemory.AllocZeroed((nuint)sizeof(Layout));"); + w.line("vtable->Size = (uint)sizeof(Layout);"); + w.line("vtable->Flags = 0;"); + w.line("vtable->Free = &FreeTrampoline;"); + for m in &cb.methods { + w.line(format!( + "vtable->{} = &{}Trampoline;", + safe_cs_name(&m.name), + method_cs(m) + )); + } + w.line("return (IntPtr)vtable;"); + }); + w.blank(); + w.line(UNMANAGED_CALLERS_ONLY); + w.line("private static void FreeTrampoline(IntPtr ctx)"); + w.block("{", "}", |w| { + w.line("Ffi.Unregister(ctx);"); + }); + for (i, m) in cb.methods.iter().enumerate() { + w.blank(); + let reports = match protocol.method_errors[i] { + ErrorStrategy::Throws => domain, + ErrorStrategy::Trap => None, + }; + render_trampoline(w, cx, &iface, m, &protocol, i, reports); + } + }); + w.blank(); +} + +/// The name of the slot at `from_end` places before the end of `m`'s slot +/// list (`1` is `out_err`). +fn slot_from_end(m: &CallbackMethodBinding, from_end: usize) -> String { + safe_cs_name(&m.abi_params[m.abi_params.len() - from_end].name) +} + +/// One trampoline: receive the slots, call the implementation, and hand +/// its result back; any exception is reported through `out_err`. `domain` +/// is the exception class a `throws` method reports with its code and +/// payload. +fn render_trampoline( + w: &mut CodeWriter, + cx: Cx<'_>, + iface: &str, + m: &CallbackMethodBinding, + protocol: &CallbackProtocol<'_>, + index: usize, + domain: Option<&str>, +) { + let sig: Vec = m + .abi_params + .iter() + .map(|s| format!("{} {}", cs_ctype(&s.ty), safe_cs_name(&s.name))) + .collect(); + let ret = cs_ctype(&m.abi_ret); + let returns_value = ret != "void"; + let ctx = safe_cs_name(&m.abi_params[0].name); + let out_err = slot_from_end(m, 1); + w.line(UNMANAGED_CALLERS_ONLY); + w.line(format!( + "private static {ret} {}Trampoline({})", + method_cs(m), + sig.join(", ") + )); + w.block("{", "}", |w| { + w.line("try"); + w.block("{", "}", |w| { + let args: Vec = m + .params + .iter() + .map(|p| { + let ptr = safe_cs_name(&p.abi[0].name); + let len = p + .abi + .get(1) + .map(|s| safe_cs_name(&s.name)) + .unwrap_or_default(); + receive(cx, &p.ty, &ptr, &len, Own::Borrow) + }) + .collect(); + let call = format!( + "Ffi.Target<{iface}>({ctx}).{}({})", + method_cs(m), + args.join(", ") + ); + match (&protocol.method_returns[index], &m.ret) { + (RetPass::Void, _) | (_, None) => w.line(format!("{call};")), + (RetPass::Direct, Some(ty)) => { + w.line(format!("return {};", direct_to_slot(ty, &call))) + } + // One strong reference the producer adopts. A null for a + // required object passes through as null, which the + // producer rejects as a marshalling failure (-3). + (RetPass::Object { .. }, _) => { + w.line(format!("return {call}?.CloneHandle() ?? IntPtr.Zero;")) + } + (rp, Some(ty)) => { + let (out_ptr, out_len) = (slot_from_end(m, 3), slot_from_end(m, 2)); + match rp { + RetPass::String => { + w.line(format!("Ffi.ReturnString({call}, {out_ptr}, {out_len});")) + } + RetPass::Bytes => { + w.line(format!("Ffi.ReturnBytes({call}, {out_ptr}, {out_len});")) + } + _ => { + w.line(format!("var ffiResult = {call};")); + w.line("var ffiWriter = new FfiBufferWriter();"); + w.line(write_stmt(ty, "ffiWriter", "ffiResult")); + w.line(format!( + "Ffi.ReturnBuffer(ffiWriter, {out_ptr}, {out_len});" + )) + } + } + } + }; + }); + if let Some(exc) = domain { + w.line(format!("catch ({} e)", cx.ty(exc))); + w.block("{", "}", |w| { + w.line(format!( + "Ffi.SetDomainError({out_err}, e.Code, e, e.WritePayload);" + )); + if returns_value { + w.line("return default;"); + } + }); + } + w.line("catch (Exception e)"); + w.block("{", "}", |w| { + w.line(format!("Ffi.SetForeignError({out_err}, e);")); + if returns_value { + w.line("return default;"); + } + }); + }); +} diff --git a/crates/weaveffi-gen/src/targets/dotnet/calls.rs b/crates/weaveffi-cli/src/targets/dotnet/calls.rs similarity index 86% rename from crates/weaveffi-gen/src/targets/dotnet/calls.rs rename to crates/weaveffi-cli/src/targets/dotnet/calls.rs index b5a668b3..95e294ec 100644 --- a/crates/weaveffi-gen/src/targets/dotnet/calls.rs +++ b/crates/weaveffi-cli/src/targets/dotnet/calls.rs @@ -7,16 +7,15 @@ //! call the import with a stack `FfiError`, check it, and receive the result. use crate::codegen::CodeWriter; -use crate::utils::{local_type_name, wrapper_name}; use heck::ToUpperCamelCase; use weaveffi_model::abi::{AbiParam, CType}; use weaveffi_model::model::{ - AsyncBinding, CallShape, ErrorBinding, Family, FnBinding, IteratorBinding, ModuleBinding, - ParamBinding, Ty, + AsyncBinding, CallShape, ErrorBinding, FnBinding, IteratorBinding, ModuleBinding, ParamBinding, }; use weaveffi_model::plan::{ArgPass, ErrorStrategy}; +use weaveffi_model::ty::{Family, Prim, Ty}; -use crate::targets::dotnet::codec::{emit_decode, emit_write}; +use crate::targets::dotnet::codec::{decode_expr, write_stmt}; use crate::targets::dotnet::docs::{write_doc, write_fn_doc}; use crate::targets::dotnet::runtime::dotnet_exception_name; use crate::targets::dotnet::types::{ @@ -28,9 +27,9 @@ pub(crate) const UNMANAGED_CALLERS_ONLY: &str = "[UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })]"; /// How a wrapper maps a non-zero error slot to an exception, rendering -/// [`ErrorStrategy`]: a throwing function with a domain in scope uses the -/// domain's typed `FromError`; everything else (producer traps) uses the -/// base exception's. +/// [`ErrorStrategy`]: a throwing function uses its domain's typed +/// `FromError` (or the root exception's, without a domain in scope); a +/// non-throwing one traps through the bug exception's. #[derive(Clone)] pub(crate) struct ErrCtx<'a> { cx: Cx<'a>, @@ -47,14 +46,19 @@ impl<'a> ErrCtx<'a> { ErrCtx { cx, map: format!("{}.FromError", cx.ty(&exc)), - domain: Some((exc, eb.type_name.clone())), + domain: Some((exc, eb.name.clone())), } } - _ => ErrCtx { + (ErrorStrategy::Throws, None) => ErrCtx { cx, map: format!("{}.FromError", cx.ty(cx.base)), domain: None, }, + (ErrorStrategy::Trap, _) => ErrCtx { + cx, + map: format!("{}.FromError", cx.ty(cx.bug)), + domain: None, + }, } } @@ -100,7 +104,7 @@ pub(crate) fn write_obsolete(w: &mut CodeWriter, deprecated: &Option) { /// the call, the buffers pinned for it, one expression per ABI slot, and the /// callback registrations to release if the call never reaches the producer. struct Marshal { - setup: String, + setup: Vec, pins: Vec, args: Vec, callbacks: Vec, @@ -112,14 +116,15 @@ impl Marshal { let mut pins = Vec::new(); let mut args = Vec::new(); let mut callbacks = Vec::new(); - let mut setup = CodeWriter::four_space(); + let mut setup = Vec::new(); + let mut registrations = Vec::new(); for p in params { let n = safe_cs_name(&p.name); let local = p.name.as_str(); match p.arg_pass() { ArgPass::Direct { .. } => args.push(direct_to_slot(&p.ty, &n)), ArgPass::String { .. } => { - setup.line(format!("var {local}Bytes = Ffi.Utf8({n});")); + setup.push(format!("var {local}Bytes = Ffi.Utf8({n});")); pins.push(format!("{local}Ptr = {local}Bytes")); args.push(format!("{local}Ptr")); args.push(format!("(nuint){local}Bytes.Length")); @@ -131,8 +136,8 @@ impl Marshal { } ArgPass::Buffer { .. } => { let writer = format!("{local}Writer"); - setup.line(format!("var {writer} = new FfiBufferWriter();")); - emit_write(&mut setup, &p.ty, &n, &writer, 0); + setup.push(format!("var {writer} = new FfiBufferWriter();")); + setup.push(write_stmt(&p.ty, &writer, &n)); pins.push(format!("{local}Ptr = {writer}.Written")); args.push(format!("{local}Ptr")); args.push(format!("(nuint){writer}.Length")); @@ -148,22 +153,29 @@ impl Marshal { } } // The producer owns the registration once the call reaches - // it: its vtable `free(ctx)` releases it. - ArgPass::Callback { vtable, .. } => { - args.push(format!("{local}Ctx")); - args.push(format!("{}.Pointer", vtable_class_for_slot(&vtable.ty))); - callbacks.push((format!("{local}Ctx"), n)); + // it: its vtable `free(ctx)` releases it. An absent optional + // implementation passes a null vtable. + ArgPass::Callback { + vtable, nullable, .. + } => { + let ctx = format!("{local}Ctx"); + let pointer = format!("{}.Pointer", vtable_class_for_slot(&vtable.ty)); + args.push(ctx.clone()); + if nullable { + args.push(format!("{n} == null ? IntPtr.Zero : {pointer}")); + } else { + args.push(pointer); + } + registrations.push(format!("var {ctx} = Ffi.Register({n});")); + callbacks.push(ctx); } } } // Register last, so a failing encode above can't strand a // registration. - for (ctx, n) in &callbacks { - setup.line(format!("var {ctx} = Ffi.Register({n});")); - } - let callbacks = callbacks.into_iter().map(|(ctx, _)| ctx).collect(); + setup.extend(registrations); Marshal { - setup: setup.finish(), + setup, pins, args, callbacks, @@ -172,7 +184,9 @@ impl Marshal { /// Emit the setup statements. fn write_setup(&self, w: &mut CodeWriter) { - w.block_raw(&self.setup); + for line in &self.setup { + w.line(line); + } } /// Emit `body` inside one `fixed` statement pinning every buffer, or bare @@ -212,17 +226,17 @@ fn vtable_class_for_slot(vtable: &CType) -> String { /// as a byte, a C-style enum as its `int`, every other scalar as is. pub(crate) fn direct_to_slot(ty: &Ty, expr: &str) -> String { match ty { - Ty::Bool => format!("(byte)({expr} ? 1 : 0)"), + Ty::Prim(Prim::Bool) => format!("(byte)({expr} ? 1 : 0)"), Ty::Enum(_) => format!("(int){expr}"), _ => expr.to_string(), } } /// The expression converting a by-value slot into its surface type. -pub(crate) fn direct_from_slot(ty: &Ty, slot: &str) -> String { +pub(crate) fn direct_from_slot(cx: Cx<'_>, ty: &Ty, slot: &str) -> String { match ty { - Ty::Bool => format!("{slot} != 0"), - Ty::Enum(name) => format!("({}){slot}", local_type_name(name)), + Ty::Prim(Prim::Bool) => format!("{slot} != 0"), + Ty::Enum(name) => format!("({}){slot}", cx.ty(name)), _ => slot.to_string(), } } @@ -237,35 +251,19 @@ pub(crate) enum Own { Borrow, } -/// Emit any statements receiving a value of `ty` from its slots (`ptr`, plus -/// `len` for the `(ptr, len)` families) and return the expression holding -/// the received value. `var` names the decoded local for buffers. Objects -/// always transfer one strong reference, which the new wrapper adopts. -pub(crate) fn receive( - w: &mut CodeWriter, - cx: Cx<'_>, - ty: &Ty, - ptr: &str, - len: &str, - own: Own, - var: &str, -) -> String { +/// The expression receiving a value of `ty` from its slots (`ptr`, plus +/// `len` for the `(ptr, len)` families). Objects always transfer one strong +/// reference, which the new wrapper adopts. +pub(crate) fn receive(cx: Cx<'_>, ty: &Ty, ptr: &str, len: &str, own: Own) -> String { let verb = match own { Own::Take => "Take", Own::Borrow => "Read", }; match ty.family() { - Family::Direct => direct_from_slot(ty, ptr), + Family::Direct => direct_from_slot(cx, ty, ptr), Family::String => format!("Ffi.{verb}String({ptr}, {len})"), Family::Bytes => format!("Ffi.{verb}Bytes({ptr}, {len})"), - Family::Buffer => { - let reader = match own { - Own::Take => format!("Ffi.TakeBuffer({ptr}, {len})"), - Own::Borrow => format!("new FfiBufferReader(Ffi.ReadBytes({ptr}, {len}))"), - }; - emit_decode(w, cx, ty, var, &reader); - var.to_string() - } + Family::Buffer => decode_expr(cx, ty, &format!("Ffi.{verb}Buffer({ptr}, {len})")), Family::Object { nullable } => { let class = cx.ty(ty.interface_name().expect("object types name an interface")); if nullable { @@ -274,7 +272,7 @@ pub(crate) fn receive( format!("{class}.Adopt({ptr})") } } - Family::Callback | Family::Iterator => unreachable!("{ty} is never received"), + Family::Callback { .. } | Family::Iterator => unreachable!("{ty} is never received"), } } @@ -284,7 +282,7 @@ fn params_sig(params: &[ParamBinding], span: bool) -> Vec { params .iter() .map(|p| { - let ty = if span && p.ty == Ty::Bytes { + let ty = if span && p.ty == Ty::Prim(Prim::Bytes) { "ReadOnlySpan".to_string() } else { cs_type(&p.ty) @@ -300,7 +298,7 @@ fn forward_args(params: &[ParamBinding]) -> String { .iter() .map(|p| { let n = safe_cs_name(&p.name); - if p.ty == Ty::Bytes { + if p.ty == Ty::Prim(Prim::Bytes) { format!("(ReadOnlySpan){n}") } else { n @@ -343,7 +341,7 @@ pub(crate) fn render_callable( let f = camel_fn(f); match &f.shape { CallShape::Sync(abi) => { - let span = f.params.iter().any(|p| p.ty == Ty::Bytes); + let span = f.params.iter().any(|p| p.ty == Ty::Prim(Prim::Bytes)); render_sync(w, &f, &abi.params, &abi.ret, name, receiver, err, span); if span { render_array_overload(w, &f, name, receiver, err); @@ -437,8 +435,7 @@ fn render_sync( w.line("Handle = new NativeHandle(ffiResult);"); } (Some(ty), _) => { - let expr = - receive(w, cx, ty, "ffiResult", "ffiOutLen", Own::Take, "ffiDecoded"); + let expr = receive(cx, ty, "ffiResult", "ffiOutLen", Own::Take); w.line(format!("return {expr};")); } (None, _) => {} @@ -592,7 +589,7 @@ fn render_async( } else { ("result", "") }; - let expr = receive(w, cx, ty, ptr, len, Own::Take, "ffiDecoded"); + let expr = receive(cx, ty, ptr, len, Own::Take); w.line(format!("ffiCall.SetResult({expr});")); } } @@ -703,15 +700,7 @@ fn render_iterator( w.line("item = default!;"); w.line("return false;"); }); - let expr = receive( - w, - cx, - &it.elem, - "ffiItem", - "ffiLen", - Own::Take, - "ffiDecoded", - ); + let expr = receive(cx, &it.elem, "ffiItem", "ffiLen", Own::Take); w.line(format!("item = {expr};")); w.line("return true;"); }); @@ -723,24 +712,19 @@ fn render_iterator( pub(crate) fn render_module_class( w: &mut CodeWriter, m: &ModuleBinding, - strip_module_prefix: bool, + error: Option<&ErrorBinding>, cx: Cx<'_>, ) { if m.functions.is_empty() { return; } let class: String = m.segments.iter().map(|s| s.to_upper_camel_case()).collect(); - // The model falls back to the first function's doc when the module has - // none; that would mislabel the class, so only a module's own doc is - // used. - if !m.functions.iter().any(|f| f.doc == m.doc) { - write_doc(w, &m.doc); - } + write_doc(w, &m.doc); w.line(format!("public static unsafe class {class}")); w.block("{", "}", |w| { for f in &m.functions { - let name = wrapper_name(&m.path, &f.name, strip_module_prefix).to_upper_camel_case(); - let err = ErrCtx::for_fn(f, m.error.as_ref(), cx); + let name = f.name.to_upper_camel_case(); + let err = ErrCtx::for_fn(f, error, cx); render_callable(w, f, &name, Receiver::Static, &err); } }); diff --git a/crates/weaveffi-cli/src/targets/dotnet/codec.rs b/crates/weaveffi-cli/src/targets/dotnet/codec.rs new file mode 100644 index 00000000..b8cf7452 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dotnet/codec.rs @@ -0,0 +1,252 @@ +//! Value-buffer codec: the C# statement writing and the expression reading +//! every wire shape, dispatched on [`Ty::wire`], and the internal +//! `FfiCodecs` class, which holds one `Write`/`Read` pair per distinct +//! composite type (`[Entry]`, `{string:i64}`, `Shape?`) so no call site +//! inlines a loop. +//! +//! Records and rich enums carry their own `WriteTo`/`ReadFrom` pair. The +//! runtime's `FfiBufferWriter`/`FfiBufferReader` name one method per +//! [`Prim`](weaveffi_model::ty::Prim) (`WriteI32`, `ReadString`), so +//! primitives need a single arm. An object token carries one strong +//! reference: writing one clones the wrapper's handle through the +//! interface's `_clone` symbol, and reading one adopts the pointer into a +//! new wrapper. + +use std::collections::BTreeMap; + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{FieldBinding, Model, ParamBinding}; +use weaveffi_model::ty::{Ty, WireType}; + +use crate::targets::dotnet::types::{cs_type, Cx}; + +/// The name of a composite's codec methods after `Write`/`Read`, spelled in +/// prefix order so it's unambiguous: `[Entry]` is `ListOfEntry`, +/// `{string:[i32]}` is `MapOfStringToListOfI32`, `Shape?` is +/// `OptionalOfShape`. +pub(crate) fn codec_name(t: &Ty) -> String { + match t { + Ty::Prim(p) => p.pascal().to_string(), + Ty::Optional(inner) => format!("OptionalOf{}", codec_name(inner)), + Ty::List(inner) => format!("ListOf{}", codec_name(inner)), + Ty::Map(k, v) => format!("MapOf{}To{}", codec_name(k), codec_name(v)), + other => other + .user_name() + .or_else(|| other.interface_name()) + .expect("only named types remain") + .to_string(), + } +} + +/// The C# statement writing `expr` (the surface value of `ty`) into the +/// writer `writer`. +/// +/// An object is written as a token carrying a new strong reference +/// (`CloneHandle()`), so the encoding never hands over the reference the +/// wrapper still owns. +pub(crate) fn write_stmt(ty: &Ty, writer: &str, expr: &str) -> String { + match ty.wire() { + WireType::Prim(p) => format!("{writer}.Write{}({expr});", p.pascal()), + WireType::Enum(_) => format!("{writer}.WriteI32((int){expr});"), + WireType::Object(_) => format!("{writer}.WriteObject({expr}.CloneHandle());"), + WireType::User(_) => format!("{expr}.WriteTo({writer});"), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("FfiCodecs.Write{}({writer}, {expr});", codec_name(ty)) + } + } +} + +/// The C# expression reading a value of `ty` from the reader `reader`; the +/// inverse of [`write_stmt`]. An object token is adopted into a new +/// wrapper, which owes the reference's release. +pub(crate) fn read_expr(cx: Cx<'_>, ty: &Ty, reader: &str) -> String { + match ty.wire() { + WireType::Prim(p) => format!("{reader}.Read{}()", p.pascal()), + WireType::Enum(name) => format!("({}){reader}.ReadI32()", cx.ty(name)), + WireType::Object(name) => format!("{}.Adopt({reader}.ReadObject())", cx.ty(name)), + WireType::User(name) => format!("{}.ReadFrom({reader})", cx.ty(name)), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("FfiCodecs.Read{}({reader})", codec_name(ty)) + } + } +} + +/// The C# expression decoding a whole value buffer of `ty` from the reader +/// expression `reader` and checking that nothing trails it. +pub(crate) fn decode_expr(cx: Cx<'_>, ty: &Ty, reader: &str) -> String { + format!( + "FfiCodecs.Decode({reader}, static r => {})", + read_expr(cx, ty, "r") + ) +} + +/// Every distinct composite type (optional, list, or map) that crosses +/// inside a value buffer, by codec name. +fn composites(model: &Model) -> BTreeMap { + /// A type in a top-level position: a parameter, return, or iterator + /// element. `Interface?` and `Cb?` aren't buffers there. + fn top(t: &Ty, out: &mut BTreeMap) { + match t { + Ty::Iterator(elem) => top(elem, out), + _ if t.is_buffered() => nested(t, out), + _ => {} + } + } + /// A type inside a value buffer, where every optional is encoded. + fn nested(t: &Ty, out: &mut BTreeMap) { + match t { + Ty::Optional(inner) | Ty::List(inner) => { + out.insert(codec_name(t), t.clone()); + nested(inner, out); + } + Ty::Map(k, v) => { + out.insert(codec_name(t), t.clone()); + nested(k, out); + nested(v, out); + } + _ => {} + } + } + fn fields(fs: &[FieldBinding], out: &mut BTreeMap) { + for f in fs { + nested(&f.ty, out); + } + } + fn signature(ps: &[ParamBinding], ret: &Option, out: &mut BTreeMap) { + for p in ps { + top(&p.ty, out); + } + if let Some(t) = ret { + top(t, out); + } + } + let mut out = BTreeMap::new(); + for m in &model.modules { + for s in &m.structs { + fields(&s.fields, &mut out); + } + for e in &m.enums { + for v in &e.variants { + fields(&v.fields, &mut out); + } + } + if let Some(eb) = &m.errors { + for c in &eb.codes { + fields(&c.fields, &mut out); + } + } + for f in m.callables() { + signature(&f.params, &f.ret, &mut out); + } + for cb in &m.callback_interfaces { + for f in &cb.methods { + signature(&f.params, &f.ret, &mut out); + } + } + } + out +} + +/// Render the internal `FfiCodecs` class: `Decode`, which reads a whole +/// buffer, plus one `Write{Name}`/`Read{Name}` pair per composite type. +/// +/// A list or map count isn't bounded by the remaining input (elements can +/// encode to zero bytes), so a read grows its collection from a capped +/// capacity instead of preallocating whatever the count claims. A map that +/// repeats a key is malformed. +pub(crate) fn render_codecs(w: &mut CodeWriter, cx: Cx<'_>, model: &Model) { + w.line("/// The value-buffer codec of every composite type the API"); + w.line("/// carries."); + w.line("internal static class FfiCodecs"); + w.block("{", "}", |w| { + w.line("/// Reads one value that must fill the whole buffer."); + w.line( + "internal static T Decode(FfiBufferReader reader, Func read)", + ); + w.block("{", "}", |w| { + w.line("var value = read(reader);"); + w.line("reader.ExpectEnd();"); + w.line("return value;"); + }); + for t in composites(model).values() { + w.blank(); + render_pair(w, cx, t); + } + }); + w.blank(); +} + +/// One composite's `Write{Name}`/`Read{Name}` pair. +fn render_pair(w: &mut CodeWriter, cx: Cx<'_>, t: &Ty) { + let name = codec_name(t); + let cs = cs_type(t); + w.line(format!( + "internal static void Write{name}(FfiBufferWriter writer, {cs} value)" + )); + w.block("{", "}", |w| match t { + Ty::Optional(inner) => { + w.line("writer.WriteBool(value != null);"); + w.line("if (value is { } present)"); + w.block("{", "}", |w| { + w.line(write_stmt(inner, "writer", "present")); + }); + } + Ty::List(inner) => { + w.line("writer.WriteLen(value.Length);"); + w.line("foreach (var item in value)"); + w.block("{", "}", |w| { + w.line(write_stmt(inner, "writer", "item")); + }); + } + Ty::Map(k, v) => { + w.line("writer.WriteLen(value.Count);"); + w.line("foreach (var entry in value)"); + w.block("{", "}", |w| { + w.line(write_stmt(k, "writer", "entry.Key")); + w.line(write_stmt(v, "writer", "entry.Value")); + }); + } + _ => unreachable!("only composites are collected"), + }); + w.blank(); + w.line(format!( + "internal static {cs} Read{name}(FfiBufferReader reader)" + )); + w.block("{", "}", |w| match t { + Ty::Optional(inner) => { + w.line(format!( + "return reader.ReadBool() ? {} : null;", + read_expr(cx, inner, "reader") + )); + } + Ty::List(inner) => { + w.line("var count = reader.ReadLen();"); + w.line(format!( + "var list = new List<{}>(reader.Capacity(count));", + cs_type(inner) + )); + w.line("for (var i = 0; i < count; i++)"); + w.block("{", "}", |w| { + w.line(format!("list.Add({});", read_expr(cx, inner, "reader"))); + }); + w.line("return list.ToArray();"); + } + Ty::Map(k, v) => { + w.line("var count = reader.ReadLen();"); + w.line(format!("var map = new {cs}(reader.Capacity(count));")); + w.line("for (var i = 0; i < count; i++)"); + w.block("{", "}", |w| { + w.line(format!("var key = {};", read_expr(cx, k, "reader"))); + w.line(format!( + "if (!map.TryAdd(key, {}))", + read_expr(cx, v, "reader") + )); + w.block("{", "}", |w| { + w.line("throw FfiBufferReader.Malformed(\"repeated map key\");"); + }); + }); + w.line("return map;"); + } + _ => unreachable!("only composites are collected"), + }); +} diff --git a/crates/weaveffi-gen/src/targets/dotnet/docs.rs b/crates/weaveffi-cli/src/targets/dotnet/docs.rs similarity index 100% rename from crates/weaveffi-gen/src/targets/dotnet/docs.rs rename to crates/weaveffi-cli/src/targets/dotnet/docs.rs diff --git a/crates/weaveffi-gen/src/targets/dotnet/entities.rs b/crates/weaveffi-cli/src/targets/dotnet/entities.rs similarity index 90% rename from crates/weaveffi-gen/src/targets/dotnet/entities.rs rename to crates/weaveffi-cli/src/targets/dotnet/entities.rs index f5d270de..8b9653e3 100644 --- a/crates/weaveffi-gen/src/targets/dotnet/entities.rs +++ b/crates/weaveffi-cli/src/targets/dotnet/entities.rs @@ -9,9 +9,9 @@ use weaveffi_model::model::{ }; use crate::targets::dotnet::calls::{render_callable, write_obsolete, ErrCtx, Receiver}; -use crate::targets::dotnet::codec::{emit_read, emit_write}; +use crate::targets::dotnet::codec::{read_expr, write_stmt}; use crate::targets::dotnet::docs::write_doc; -use crate::targets::dotnet::types::{cs_type, safe_cs_name, Cx}; +use crate::targets::dotnet::types::{cs_member, cs_type, safe_cs_name, Cx}; /// Render a C-style enum as a C# `enum` with its ABI discriminants. pub(crate) fn render_enum(w: &mut CodeWriter, e: &EnumBinding) { @@ -64,16 +64,21 @@ fn render_value_members(w: &mut CodeWriter, class: &str, fields: &[FieldBinding] }); } -/// Emit the reads of every field into `f{Field}` locals, then return the -/// constructed `class`. +/// Emit `return new {class}(...)` reading every field in declaration (and +/// wire) order. C# evaluates arguments left to right, so the reads happen in +/// order. fn render_field_reads(w: &mut CodeWriter, cx: Cx<'_>, class: &str, fields: &[FieldBinding]) { - let mut args = Vec::new(); - for f in fields { - let var = format!("f{}", f.name.to_upper_camel_case()); - emit_read(w, cx, &f.ty, &var, "reader", 0); - args.push(var); + if fields.is_empty() { + w.line(format!("return new {class}();")); + return; + } + w.line(format!("return new {class}(")); + w.indent(); + for (i, f) in fields.iter().enumerate() { + let sep = if i + 1 == fields.len() { ");" } else { "," }; + w.line(format!("{}{sep}", read_expr(cx, &f.ty, "reader"))); } - w.line(format!("return new {class}({});", args.join(", "))); + w.dedent(); } /// Render a record as a sealed data class: get-only properties, a @@ -89,7 +94,7 @@ pub(crate) fn render_record(w: &mut CodeWriter, cx: Cx<'_>, s: &StructBinding) { w.line("internal void WriteTo(FfiBufferWriter writer)"); w.block("{", "}", |w| { for f in &s.fields { - emit_write(w, &f.ty, &f.name.to_upper_camel_case(), "writer", 0); + w.line(write_stmt(&f.ty, "writer", &f.name.to_upper_camel_case())); } }); w.blank(); @@ -132,7 +137,7 @@ pub(crate) fn render_rich_enum(w: &mut CodeWriter, cx: Cx<'_>, e: &EnumBinding) w.line(format!("writer.WriteI32({});", v.value)); for f in &v.fields { let expr = format!("v.{}", f.name.to_upper_camel_case()); - emit_write(w, &f.ty, &expr, "writer", 0); + w.line(write_stmt(&f.ty, "writer", &expr)); } w.line("break;"); }); @@ -162,7 +167,7 @@ pub(crate) fn render_rich_enum(w: &mut CodeWriter, cx: Cx<'_>, e: &EnumBinding) w.line("default:"); w.scope(|w| { w.line(format!( - "throw new InvalidOperationException(\"malformed value buffer: unknown {name} tag \" + tag);" + "throw FfiBufferReader.Malformed(\"unknown {name} tag \" + tag);" )); }); }); @@ -226,7 +231,7 @@ pub(crate) fn render_interface( for c in &i.constructors { let err = ErrCtx::for_fn(c, error, cx); - let method = c.name.to_upper_camel_case(); + let method = cs_member(&c.name, name); let receiver = if c.name == "new" && matches!(c.shape, CallShape::Sync(_)) { Receiver::Constructor(name) } else { @@ -236,11 +241,11 @@ pub(crate) fn render_interface( } for m in &i.methods { let err = ErrCtx::for_fn(m, error, cx); - render_callable(w, m, &m.name.to_upper_camel_case(), Receiver::Instance, &err); + render_callable(w, m, &cs_member(&m.name, name), Receiver::Instance, &err); } for s in &i.statics { let err = ErrCtx::for_fn(s, error, cx); - render_callable(w, s, &s.name.to_upper_camel_case(), Receiver::Static, &err); + render_callable(w, s, &cs_member(&s.name, name), Receiver::Static, &err); } w.line("/// Releases this wrapper's reference. The native object is"); diff --git a/crates/weaveffi-gen/src/targets/dotnet/mod.rs b/crates/weaveffi-cli/src/targets/dotnet/mod.rs similarity index 61% rename from crates/weaveffi-gen/src/targets/dotnet/mod.rs rename to crates/weaveffi-cli/src/targets/dotnet/mod.rs index b6aee571..2c97747a 100644 --- a/crates/weaveffi-gen/src/targets/dotnet/mod.rs +++ b/crates/weaveffi-cli/src/targets/dotnet/mod.rs @@ -1,6 +1,6 @@ //! .NET binding generator. //! -//! Emits a standalone C# project over the C ABI, revision 3: a `.csproj`, +//! Emits a standalone C# project over the C ABI, revision 4: a `.csproj`, //! the generated API (`{Namespace}.cs`), the fixed runtime (`Runtime.cs`, //! from `runtime/Runtime.cs`), and a README. Implements [`LanguageBackend`]. //! @@ -9,12 +9,19 @@ //! trim- and AOT-friendly. Strings cross as UTF-8 `(ptr, len)` pairs. //! * The first native call runs `NativeMethods`' static constructor, which //! installs a resolver honoring `{PREFIX}_LIBRARY`, then checks the ABI -//! revision and every top-level module's contract checksum. -//! * Records, rich enums, optionals, lists, and maps are value types encoded -//! in the value-buffer format by per-type `WriteTo`/`ReadFrom` pairs. +//! revision and every top-level module's contract table against the +//! entries the bindings were generated with. +//! * Records and rich enums are classes encoded in the value-buffer format +//! by per-type `WriteTo`/`ReadFrom` pairs; optionals, lists, and maps by +//! one `FfiCodecs` pair per distinct composite type. //! * Interfaces are sealed `IDisposable` wrappers over a `SafeHandle` //! subclass whose `ReleaseHandle` calls `_destroy`. Every import takes the //! handle itself, so the interop stub keeps the object alive for the call. +//! * A throwing call raises its domain's exception hierarchy (one nested +//! class per code, its fields as properties) or the root +//! `NativeException` for a runtime code; a failed non-throwing call is a +//! producer bug and raises `NativeBugException`; cancellation raises +//! `OperationCanceledException`. //! * Callback interfaces are C# `interface`s; a passed implementation is //! pinned in a `GCHandle` and paired with one static vtable of //! `[UnmanagedCallersOnly]` trampolines. @@ -34,63 +41,41 @@ mod runtime; mod types; use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; use crate::codegen::CodeWriter; -use crate::package::{PackageContext, PackagedFile}; +use crate::package::{Artifact, PackageContext, PackagedFile}; use crate::utils::{render_prelude, render_trailer, CommentStyle}; use camino::Utf8Path; use serde::{Deserialize, Serialize}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; +use weaveffi_model::model::Model; use crate::targets::dotnet::callbacks::render_callback_interface; use crate::targets::dotnet::calls::render_module_class; +use crate::targets::dotnet::codec::render_codecs; use crate::targets::dotnet::entities::{ render_enum, render_interface, render_record, render_rich_enum, }; use crate::targets::dotnet::package::{render_csproj, render_readme, Project, NATIVE_ASSETS}; use crate::targets::dotnet::pinvoke::render_native_methods; use crate::targets::dotnet::runtime::{ - base_exception_name, render_domain_exception, render_runtime, RuntimeNames, + exception_names, render_domain_exception, render_runtime, RuntimeNames, }; use crate::targets::dotnet::types::Cx; /// Per-target configuration for [`DotnetGenerator`]. -#[derive(Debug, Clone, Serialize, Deserialize)] +#[derive(Debug, Clone, Default, Serialize, Deserialize)] #[serde(default, deny_unknown_fields)] pub struct DotnetConfig { /// C# namespace, assembly name, and NuGet id. Defaults to the package /// name in PascalCase (`my-kv` becomes `MyKv`). - pub namespace: Option, - /// When `true` (the default), strip the module name from wrapper method - /// names; the per-module static class already namespaces them. - pub strip_module_prefix: bool, - /// Basename of the IDL the CLI was invoked with. Set by the CLI. - #[serde(skip)] - pub input_basename: Option, -} - -impl Default for DotnetConfig { - fn default() -> Self { - Self { - namespace: None, - strip_module_prefix: true, - input_basename: None, - } - } + pub name: Option, } impl DotnetConfig { /// The C# namespace: the configured one, else `PascalCase(name)`. - pub fn namespace(&self, api: &ResolvedApi) -> String { - self.namespace + pub fn namespace(&self, model: &Model) -> String { + self.name .clone() - .unwrap_or_else(|| api.identity().pascal_name()) - } - - /// The IDL basename for generated-file preludes. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") + .unwrap_or_else(|| model.identity.pascal_name()) } } @@ -99,27 +84,25 @@ pub struct DotnetGenerator; /// The text files of a generated project, at `dir`, loading `library`. fn project_files( - api: &ResolvedApi, - model: &BindingModel, + model: &Model, dir: &Utf8Path, config: &DotnetConfig, library: &str, ctx: Option<&PackageContext>, ) -> Vec<(camino::Utf8PathBuf, String)> { - let namespace = config.namespace(api); - let identity = api.identity(); - let input = config.input_basename(); + let namespace = config.namespace(model); + let identity = &model.identity; let project = Project { identity, namespace: &namespace, library, - input_basename: input, }; - let base = base_exception_name(model, &namespace); + let (base, bug) = exception_names(model, &namespace); let names = RuntimeNames { namespace: &namespace, exception: &base, - prefix: &model.prefix, + bug_exception: &bug, + prefix: model.prefix(), library, library_env: &identity.library_env_var(), }; @@ -129,19 +112,9 @@ fn project_files( vec![ ( dir.join(&cs), - render_csharp( - model, - &namespace, - &base, - config.strip_module_prefix, - input, - &cs, - ), - ), - ( - dir.join("Runtime.cs"), - render_runtime(&names, input, "Runtime.cs"), + render_csharp(model, &namespace, (&base, &bug), &cs), ), + (dir.join("Runtime.cs"), render_runtime(&names, "Runtime.cs")), (dir.join(&csproj), render_csproj(&project, &csproj, assets)), (dir.join("README.md"), render_readme(&project, ctx)), ] @@ -154,70 +127,66 @@ impl LanguageBackend for DotnetGenerator { "dotnet" } - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let library = &api.identity().library; - project_files(api, model, &out_dir.join("dotnet"), config, library, None) + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let library = &model.identity.library; + project_files(model, &out_dir.join("dotnet"), config, library, None) .into_iter() .map(|(path, text)| OutputFile::new(path, text)) .collect() } + /// The NuGet-ready project at `dotnet/{Namespace}/`, with every desktop + /// library under the `runtimes//native/` layout NuGet resolves at + /// restore time; `weaveffi package` then runs `dotnet pack` on it. + /// Platforms without a RID (Android, iOS, `wasm32`) have no slot. fn package( &self, - api: &ResolvedApi, - model: &BindingModel, + model: &Model, ctx: &PackageContext, - out_dir: &Utf8Path, config: &Self::Config, - ) -> Option> { - let dir = out_dir.join("dotnet"); + ) -> Option> { let library = &ctx.binaries.lib_name; - let mut files: Vec = - project_files(api, model, &dir, config, library, Some(ctx)) - .into_iter() - .map(|(path, text)| PackagedFile::text(path, text)) - .collect(); - // Bundle each prebuilt library under the `runtimes//native/` - // layout NuGet resolves at restore time. Platforms without a RID - // (Android, wasm32) have no slot in the package. + let mut natives = Vec::new(); for nb in &ctx.binaries.binaries { let Some(rid) = nb.platform.nuget_rid() else { continue; }; - let dest = dir - .join("runtimes") - .join(rid) - .join("native") - .join(ctx.binaries.bundled_filename(nb.platform)); - files.push(PackagedFile::copy(dest, nb.source.clone())); + natives.push(PackagedFile::copy( + format!( + "runtimes/{rid}/native/{}", + ctx.binaries.bundled_filename(nb.platform) + ), + nb.library.clone(), + )); + } + if natives.is_empty() { + return Some(Vec::new()); } - Some(files) + let mut files: Vec = + project_files(model, Utf8Path::new(""), config, library, Some(ctx)) + .into_iter() + .map(|(path, text)| PackagedFile::text(path, text)) + .collect(); + files.extend(natives); + Some(vec![Artifact::directory( + format!("dotnet/{}", config.namespace(model)), + files, + )]) } } /// Render the generated API file: typed exceptions, every module's entities, /// the generated half of `NativeMethods`, and the per-module static classes. pub(crate) fn render_csharp( - model: &BindingModel, + model: &Model, namespace: &str, - base: &str, - strip_module_prefix: bool, - input_basename: &str, + (base, bug): (&str, &str), filename: &str, ) -> String { let cx = Cx { ns: namespace, base, + bug, }; let mut w = CodeWriter::four_space(); w.line("#nullable enable"); @@ -236,9 +205,9 @@ pub(crate) fn render_csharp( w.line(format!("namespace {namespace};")); w.blank(); - for m in model.modules.iter() { - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { - render_domain_exception(&mut w, eb, cx); + for m in &model.modules { + if let Some(eb) = &m.errors { + render_domain_exception(&mut w, m, eb, cx); } } for m in &model.modules { @@ -253,18 +222,19 @@ pub(crate) fn render_csharp( render_record(&mut w, cx, s); } for cb in &m.callback_interfaces { - render_callback_interface(&mut w, m, cb, cx); + render_callback_interface(&mut w, &m.path, cb, model.error_domain(m), cx); } for i in &m.interfaces { - render_interface(&mut w, i, m.error.as_ref(), cx); + render_interface(&mut w, i, model.error_domain(m), cx); } } for m in &model.modules { - render_module_class(&mut w, m, strip_module_prefix, cx); + render_module_class(&mut w, m, model.error_domain(m), cx); } + render_codecs(&mut w, cx, model); render_native_methods(&mut w, model); - let mut out = render_prelude(CommentStyle::DoubleSlash, input_basename); + let mut out = render_prelude(CommentStyle::DoubleSlash); out.push_str(&tidy(&w.finish())); out.push_str(&render_trailer(CommentStyle::DoubleSlash, filename)); out diff --git a/crates/weaveffi-gen/src/targets/dotnet/package.rs b/crates/weaveffi-cli/src/targets/dotnet/package.rs similarity index 93% rename from crates/weaveffi-gen/src/targets/dotnet/package.rs rename to crates/weaveffi-cli/src/targets/dotnet/package.rs index e34dcbef..f0c7ae03 100644 --- a/crates/weaveffi-gen/src/targets/dotnet/package.rs +++ b/crates/weaveffi-cli/src/targets/dotnet/package.rs @@ -18,8 +18,6 @@ pub(crate) struct Project<'a> { pub namespace: &'a str, /// The native library base name the bindings load. pub library: &'a str, - /// The IDL basename for generated-file preludes. - pub input_basename: &'a str, } /// Render the `.csproj`. `extra` is spliced in after the main property group @@ -74,14 +72,18 @@ pub(crate) fn render_csproj(p: &Project<'_>, filename: &str, extra: &str) -> Str {trailer}"#, - prelude = render_prelude(CommentStyle::Xml, p.input_basename), + prelude = render_prelude(CommentStyle::Xml), trailer = render_trailer(CommentStyle::Xml, filename), ) } /// The `` shipping every bundled native library under /// `runtimes//native/`, where NuGet resolves it at restore time. -pub(crate) const NATIVE_ASSETS: &str = " +pub(crate) const NATIVE_ASSETS: &str = " + README.md + + + PreserveNewest @@ -94,7 +96,7 @@ pub(crate) fn render_readme(p: &Project<'_>, ctx: Option<&PackageContext>) -> St let ns = p.namespace; let env = p.identity.library_env_var(); let (mac, linux, win) = p.identity.library_files(); - let mut out = render_prelude(CommentStyle::Xml, p.input_basename); + let mut out = render_prelude(CommentStyle::Xml); out.push_str(&format!( "# {ns} (.NET)\n\n\ {desc}\n\n\ diff --git a/crates/weaveffi-gen/src/targets/dotnet/pinvoke.rs b/crates/weaveffi-cli/src/targets/dotnet/pinvoke.rs similarity index 78% rename from crates/weaveffi-gen/src/targets/dotnet/pinvoke.rs rename to crates/weaveffi-cli/src/targets/dotnet/pinvoke.rs index 4a346e50..2ee31d74 100644 --- a/crates/weaveffi-gen/src/targets/dotnet/pinvoke.rs +++ b/crates/weaveffi-cli/src/targets/dotnet/pinvoke.rs @@ -1,5 +1,6 @@ -//! The generated half of `NativeMethods`: the contract checksum check and one -//! `[LibraryImport]` per lowered C symbol, matching each callable's shape. +//! The generated half of `NativeMethods`: the load-time contract check and +//! one `[LibraryImport]` per lowered C symbol, matching each callable's +//! shape. //! //! Every slot is blittable (`byte*`, `nuint`, `FfiError*`, function //! pointers) except object slots, which take the interface's `SafeHandle` @@ -10,10 +11,10 @@ use std::collections::HashMap; use crate::codegen::CodeWriter; -use crate::utils::local_type_name; use weaveffi_model::abi::AbiParam; +use weaveffi_model::contract; use weaveffi_model::model::{ - checksum_symbol, AbiFn, BindingModel, CallShape, FnBinding, InterfaceBinding, + contract_symbol, AbiFn, CallShape, FnBinding, InterfaceBinding, Model, }; use weaveffi_model::plan::ArgPass; @@ -21,7 +22,7 @@ use crate::targets::dotnet::types::{cs_ctype, fn_pointer_type, safe_cs_name}; /// The C# type of an interface's `SafeHandle` subclass. pub(crate) fn handle_type(interface: &str) -> String { - format!("{}.NativeHandle", local_type_name(interface)) + format!("{interface}.NativeHandle") } /// Slot types that differ from the plain [`cs_ctype`] mapping for one @@ -120,27 +121,41 @@ fn render_interface(w: &mut CodeWriter, i: &InterfaceBinding) { } } -/// Render the generated half of `NativeMethods`: the checksum check run at -/// load, each root's checksum import, and every interface and function -/// import in declaration order. -pub(crate) fn render_native_methods(w: &mut CodeWriter, model: &BindingModel) { +/// Render the generated half of `NativeMethods`: the contract check run at +/// load (every top-level module's table must carry each declaration these +/// bindings were generated with, unchanged), each root's contract import, +/// and every interface and function import in declaration order. +pub(crate) fn render_native_methods(w: &mut CodeWriter, model: &Model) { w.line("internal static unsafe partial class NativeMethods"); w.line("{"); w.indent(); - w.line("static partial void VerifyChecksums()"); + w.line("static partial void VerifyContracts()"); w.block("{", "}", |w| { for root in model.roots() { - let checksum = root.checksum.expect("top-level modules carry a checksum"); - let symbol = checksum_symbol(&model.prefix, &root.name); + let symbol = contract_symbol(model.prefix(), &root.name); w.line(format!( - "VerifyChecksum(\"{}\", &{symbol}, 0x{checksum:016x}UL);", - root.name + "VerifyContract(\"{symbol}\", &{symbol}, new (ulong, ulong, string)[]" )); + w.line("{"); + w.indent(); + for e in contract::entries(model, root) { + w.line(format!( + "(0x{:016x}UL, 0x{:016x}UL, \"{}\"),", + e.id, e.hash, e.path + )); + } + w.dedent(); + w.line("});"); } }); w.blank(); for root in model.roots() { - import(w, &checksum_symbol(&model.prefix, &root.name), "ulong", &[]); + import( + w, + &contract_symbol(model.prefix(), &root.name), + "FfiContractEntry*", + &["nuint* out_len".to_string()], + ); } for m in &model.modules { for i in &m.interfaces { diff --git a/crates/weaveffi-cli/src/targets/dotnet/runtime.rs b/crates/weaveffi-cli/src/targets/dotnet/runtime.rs new file mode 100644 index 00000000..149bc13e --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dotnet/runtime.rs @@ -0,0 +1,235 @@ +//! The fixed runtime (`runtime/Runtime.cs`, spliced with the library's +//! names) and the per-domain exception hierarchies. + +use crate::codegen::CodeWriter; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; +use heck::{ToLowerCamelCase, ToUpperCamelCase}; +use weaveffi_model::errors; +use weaveffi_model::model::{ErrorBinding, ErrorCodeBinding, Model, ModuleBinding, ABI_VERSION}; + +use crate::targets::dotnet::codec::{read_expr, write_stmt}; +use crate::targets::dotnet::docs::write_doc; +use crate::targets::dotnet::types::{cs_str, cs_type, safe_cs_name, xml_text, Cx}; + +/// The runtime source shared by every generated library. +const RUNTIME_CS: &str = include_str!("runtime/Runtime.cs"); + +/// The names a generated library splices into the fixed runtime. +pub(crate) struct RuntimeNames<'a> { + /// The C# namespace. + pub namespace: &'a str, + /// The root exception class (see [`exception_names`]). + pub exception: &'a str, + /// The trap exception class (see [`exception_names`]). + pub bug_exception: &'a str, + /// The C symbol prefix. + pub prefix: &'a str, + /// The native library base name P/Invoke loads. + pub library: &'a str, + /// The `{PREFIX}_LIBRARY` override variable. + pub library_env: &'a str, +} + +/// Render `Runtime.cs`: the fixed runtime with every placeholder replaced. +pub(crate) fn render_runtime(names: &RuntimeNames<'_>, filename: &str) -> String { + let body = RUNTIME_CS + .replace("{{NAMESPACE}}", names.namespace) + .replace("{{BUG_EXCEPTION}}", names.bug_exception) + .replace("{{EXCEPTION}}", names.exception) + .replace("{{PREFIX}}", names.prefix) + .replace("{{LIBRARY}}", names.library) + .replace("{{LIBRARY_ENV}}", names.library_env) + .replace("{{ABI_VERSION}}", &ABI_VERSION.to_string()); + debug_assert!(!body.contains("{{"), "unfilled placeholder in Runtime.cs"); + format!( + "{}{body}\n{}", + render_prelude(CommentStyle::DoubleSlash), + render_trailer(CommentStyle::DoubleSlash, filename) + ) +} + +/// The runtime's two exception classes: the root of every error a throwing +/// call reports (`NativeException`) and the trap a failed non-throwing call +/// raises (`NativeBugException`). A name the API already declares is +/// qualified by the namespace (`KvstoreNativeException`). +pub(crate) fn exception_names(model: &Model, namespace: &str) -> (String, String) { + let taken = |name: &str| { + model.modules.iter().any(|m| { + m.structs.iter().any(|s| s.name == name) + || m.enums.iter().any(|e| e.name == name) + || m.interfaces.iter().any(|i| i.name == name) + || m.callback_interfaces.iter().any(|c| c.name == name) + || m.errors + .as_ref() + .is_some_and(|e| dotnet_exception_name(e) == name) + }) + }; + let pick = |name: &str| { + if taken(name) { + format!("{namespace}{name}") + } else { + name.to_string() + } + }; + (pick("NativeException"), pick("NativeBugException")) +} + +/// The C# exception class for one error domain: the domain stem with one +/// `Exception` suffix, so `KvError` becomes `KvException`. +pub(crate) fn dotnet_exception_name(eb: &ErrorBinding) -> String { + errors::exception_type_name(&eb.type_name) +} + +/// The nested exception class for one code (`KvException.KeyNotFound`). +fn code_class(c: &ErrorCodeBinding) -> String { + errors::pascal(&c.name) +} + +/// The exception hierarchy of the error domain `eb`, declared by `module`: +/// an abstract class deriving from the root exception with one nested +/// sealed class per code. Each code class exposes its fields as typed +/// properties and can be constructed by a callback-interface implementation, +/// which reports it (code, message, and fields) to the native caller. +/// `FromError` maps a raw error to the typed exception, decoding the +/// payload; an undeclared code falls back to the root exception. +pub(crate) fn render_domain_exception( + w: &mut CodeWriter, + module: &ModuleBinding, + eb: &ErrorBinding, + cx: Cx<'_>, +) { + let exc = dotnet_exception_name(eb); + let base = cx.base; + w.line(format!( + "/// The errors of the {} domain, declared by module {}.", + eb.name, module.dot_path + )); + w.line("/// Catch a nested class for one code, or this class for any of them."); + w.line(format!("public abstract class {exc} : {base}")); + w.block("{", "}", |w| { + w.line(format!( + "private protected {exc}(int code, string message) : base(code, message)" + )); + w.line("{"); + w.line("}"); + w.blank(); + w.line("/// Writes this code's fields, the error payload a callback"); + w.line("/// reports with it."); + w.line("internal virtual void WritePayload(FfiBufferWriter writer)"); + w.line("{"); + w.line("}"); + w.blank(); + for c in &eb.codes { + render_code_class(w, &exc, c); + } + w.line("/// The exception for a raw code and its serialized payload."); + w.line( + "internal static new Exception FromError(int code, string message, byte[]? payload)", + ); + w.block("{", "}", |w| { + w.line("var given = message.Length == 0 ? null : message;"); + w.line("switch (code)"); + w.block("{", "}", |w| { + for c in &eb.codes { + let class = code_class(c); + w.line(format!("case {class}.ErrorCode:")); + if c.fields.is_empty() { + w.indent(); + w.line(format!("return new {class}(given);")); + w.dedent(); + continue; + } + w.block("{", "}", |w| { + w.line("var reader = new FfiBufferReader(payload ?? Array.Empty());"); + w.line(format!("var e = new {class}(")); + w.indent(); + for f in &c.fields { + w.line(format!("{},", read_expr(cx, &f.ty, "reader"))); + } + w.line("given);"); + w.dedent(); + w.line("reader.ExpectEnd();"); + w.line("return e;"); + }); + } + w.line("default:"); + w.indent(); + w.line(format!( + "return {}.FromError(code, message, payload);", + cx.ty(base) + )); + w.dedent(); + }); + }); + }); + w.blank(); +} + +/// One code's nested exception class. +fn render_code_class(w: &mut CodeWriter, exc: &str, c: &ErrorCodeBinding) { + let class = code_class(c); + if c.doc.is_some() { + write_doc(w, &c.doc); + } else { + w.line(format!("/// {}", xml_text(&c.message))); + } + w.line(format!("public sealed class {class} : {exc}")); + w.block("{", "}", |w| { + w.line(format!( + "/// The domain code, {}.", + c.value + )); + w.line(format!("public const int ErrorCode = {};", c.value)); + w.blank(); + for f in &c.fields { + write_doc(w, &f.doc); + w.line(format!( + "public {} {} {{ get; }}", + cs_type(&f.ty), + f.name.to_upper_camel_case() + )); + w.blank(); + } + let mut params: Vec = c + .fields + .iter() + .map(|f| { + format!( + "{} {}", + cs_type(&f.ty), + safe_cs_name(&f.name.to_lower_camel_case()) + ) + }) + .collect(); + params.push("string? message = null".into()); + w.line("/// Creates the error with , or the"); + w.line(format!( + "/// default message \"{}\" when it's null.", + xml_text(&c.message) + )); + w.line(format!( + "public {class}({}) : base(ErrorCode, message ?? \"{}\")", + params.join(", "), + cs_str(&c.message) + )); + w.block("{", "}", |w| { + for f in &c.fields { + w.line(format!( + "{} = {};", + f.name.to_upper_camel_case(), + safe_cs_name(&f.name.to_lower_camel_case()) + )); + } + }); + if !c.fields.is_empty() { + w.blank(); + w.line("internal override void WritePayload(FfiBufferWriter writer)"); + w.block("{", "}", |w| { + for f in &c.fields { + w.line(write_stmt(&f.ty, "writer", &f.name.to_upper_camel_case())); + } + }); + } + }); + w.blank(); +} diff --git a/crates/weaveffi-gen/src/targets/dotnet/runtime/Runtime.cs b/crates/weaveffi-cli/src/targets/dotnet/runtime/Runtime.cs similarity index 70% rename from crates/weaveffi-gen/src/targets/dotnet/runtime/Runtime.cs rename to crates/weaveffi-cli/src/targets/dotnet/runtime/Runtime.cs index faf30ec4..5369686a 100644 --- a/crates/weaveffi-gen/src/targets/dotnet/runtime/Runtime.cs +++ b/crates/weaveffi-cli/src/targets/dotnet/runtime/Runtime.cs @@ -12,10 +12,11 @@ namespace {{NAMESPACE}}; -/// Raised for a failure the native library reports that isn't a typed -/// domain error: producer bugs, panics, marshalling failures, and exceptions -/// thrown by a callback-interface implementation. Every typed domain -/// exception derives from it. +/// The root of every error a throwing call reports. Each typed domain +/// exception derives from it, and a runtime failure of a throwing call (an +/// untyped producer error, a panic, a marshalling failure, or a failed +/// callback-interface implementation) surfaces as this type itself, with its +/// negative . public class {{EXCEPTION}} : Exception { /// An untyped producer error. @@ -24,14 +25,15 @@ public class {{EXCEPTION}} : Exception public const int PanicErrorCode = -2; /// An argument couldn't be lifted by the producer. public const int MarshalErrorCode = -3; - /// A callback-interface implementation threw; the message carries - /// the original exception's message. + /// A callback-interface implementation failed; the message carries + /// the implementation's message. public const int ForeignErrorCode = -4; /// The call was cancelled. Surfaces as /// , never as this type. public const int CancelledErrorCode = -5; - /// The error code the native library reported. + /// The error code the native library reported: positive for a + /// domain error, negative for a runtime failure. public int Code { get; } /// Creates an exception carrying a native error code. @@ -40,9 +42,9 @@ public class {{EXCEPTION}} : Exception Code = code; } - /// Maps a runtime error code to its exception: cancellation to - /// , anything else to this - /// type. + /// Maps a code outside every declared domain to its exception: + /// cancellation to , anything else + /// to this type. internal static Exception FromError(int code, string message, byte[]? payload) { if (code == CancelledErrorCode) @@ -53,6 +55,37 @@ internal static Exception FromError(int code, string message, byte[]? payload) } } +/// A call that can't fail failed anyway: a bug in the native library, +/// or a callback failure the call couldn't report, rather than an error to +/// handle. is the runtime code (-2 a producer panic, -3 a +/// marshalling failure, -4 a failed callback, and so on), and the message +/// names it along with the producer's message. +public sealed class {{BUG_EXCEPTION}} : InvalidOperationException +{ + /// The error code the native library reported. + public int Code { get; } + + /// Creates the exception for a failed call that can't + /// fail. + public {{BUG_EXCEPTION}}(int code, string message) + : base($"native call failed with code {code}: {message}") + { + Code = code; + } + + /// Maps a failure of a call that isn't declared to throw: + /// cancellation to , anything else + /// to this type. + internal static Exception FromError(int code, string message, byte[]? payload) + { + if (code == {{EXCEPTION}}.CancelledErrorCode) + { + return new OperationCanceledException(message); + } + return new {{BUG_EXCEPTION}}(code, message); + } +} + /// Maps a native error code, message, and serialized payload to the /// exception a call throws. internal delegate Exception FfiErrorMap(int code, string message, byte[]? payload); @@ -67,6 +100,15 @@ internal unsafe struct FfiError public nuint PayloadLen; } +/// The native {{PREFIX}}_contract_entry: one declaration's +/// fingerprint in a module's contract table. +[StructLayout(LayoutKind.Sequential)] +internal struct FfiContractEntry +{ + public ulong Id; + public ulong Hash; +} + /// Marshalling helpers shared by every generated call. internal static unsafe class Ffi { @@ -91,6 +133,13 @@ internal static byte[] ReadBytes(byte* ptr, nuint len) return ptr == null || len == 0 ? Array.Empty() : new ReadOnlySpan(ptr, checked((int)len)).ToArray(); } + /// A reader over a copy of a borrowed value buffer; the native + /// side keeps ownership. + internal static FfiBufferReader ReadBuffer(byte* ptr, nuint len) + { + return new FfiBufferReader(ReadBytes(ptr, len)); + } + /// Decodes a returned UTF-8 string and releases it. internal static string TakeString(byte* ptr, nuint len) { @@ -151,30 +200,103 @@ private static string ErrorMessage(FfiError* err) return Marshal.PtrToStringUTF8((IntPtr)err->Message) ?? ""; } - /// Reports a callback-interface failure to the producer through - /// out_err. The producer copies the message. - internal static void SetForeignError(FfiError* err, Exception e) + /// Reports a failed callback-interface method to the producer + /// through out_err: with the exception's + /// message and, for a domain error with fields, the encoded fields as + /// its payload. The producer copies both. + internal static void SetError(FfiError* err, int code, Exception e, FfiBufferWriter? payload = null) { var bytes = Encoding.UTF8.GetBytes(e.Message + "\0"); fixed (byte* message = bytes) { - NativeMethods.ErrorSet(err, {{EXCEPTION}}.ForeignErrorCode, message); + NativeMethods.ErrorSet(err, code, message); + } + if (payload != null && payload.Length > 0) + { + fixed (byte* ptr = payload.Written) + { + NativeMethods.ErrorSetPayload(err, ptr, (nuint)payload.Length); + } + } + } + + /// Reports a domain exception a throwing callback-interface + /// method raised: its positive , its message, and + /// the fields encodes. A failure to + /// encode them is reported as a foreign error instead. + internal static void SetDomainError(FfiError* err, int code, Exception e, Action writePayload) + { + var payload = new FfiBufferWriter(); + try + { + writePayload(payload); + } + catch (Exception inner) + { + SetForeignError(err, inner); + return; + } + SetError(err, code, e, payload); + } + + /// Reports a failed callback-interface method that isn't a + /// declared domain error: the producer receives the foreign error code + /// with the exception's message. + internal static void SetForeignError(FfiError* err, Exception e) + { + SetError(err, {{EXCEPTION}}.ForeignErrorCode, e); + } + + /// Hands to the producer through a + /// callback method's out slots, as a run allocated with + /// {{PREFIX}}_alloc that the producer adopts. + internal static void Return(ReadOnlySpan bytes, byte** outPtr, nuint* outLen) + { + var run = NativeMethods.Alloc((nuint)bytes.Length); + if (run == null && bytes.Length > 0) + { + throw new OutOfMemoryException("the native allocator returned null"); } + bytes.CopyTo(new Span(run, bytes.Length)); + *outPtr = run; + *outLen = (nuint)bytes.Length; + } + + /// Returns a string from a callback method. + internal static void ReturnString(string value, byte** outPtr, nuint* outLen) + { + Return(Encoding.UTF8.GetBytes(value), outPtr, outLen); + } + + /// Returns bytes from a callback method. + internal static void ReturnBytes(byte[] value, byte** outPtr, nuint* outLen) + { + Return(value, outPtr, outLen); + } + + /// Returns an encoded value buffer from a callback method. + internal static void ReturnBuffer(FfiBufferWriter writer, byte** outPtr, nuint* outLen) + { + Return(writer.Written, outPtr, outLen); } /// Pins a callback-interface implementation for the producer and /// returns the ctx it hands back to every trampoline. The - /// producer releases it through the vtable's free. - internal static IntPtr Register(object implementation) + /// producer releases it through the vtable's free. An absent + /// optional implementation is a null ctx. + internal static IntPtr Register(object? implementation) { - return GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); + return implementation == null ? IntPtr.Zero : GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); } /// Releases a ctx the producer never received (the call /// failed before reaching it) or no longer holds. internal static void Unregister(IntPtr ctx) { - GCHandle.FromIntPtr(ctx).Free(); + if (ctx != IntPtr.Zero) + { + GCHandle.FromIntPtr(ctx).Free(); + } } /// The implementation behind a trampoline's ctx. @@ -184,9 +306,9 @@ internal static T Target(IntPtr ctx) where T : class } } -/// Imports of the native runtime surface plus the load-time contract -/// check. The generated API declares its own imports in the other half of -/// this partial class. +/// Imports of the native runtime surface plus the load-time checks. +/// The generated API declares its own imports, and the contract it was +/// generated with, in the other half of this partial class. internal static unsafe partial class NativeMethods { /// The native library's base name. @@ -201,8 +323,8 @@ internal static unsafe partial class NativeMethods internal const uint AbiVersion = {{ABI_VERSION}}; // Runs before the first native call through this class, so a library - // built for another ABI revision or another contract fails here instead - // of misreading memory later. + // built for another ABI revision or missing a declaration these bindings + // use fails here instead of misreading memory later. static NativeMethods() { try @@ -228,10 +350,10 @@ static NativeMethods() throw new InvalidOperationException( $"ABI mismatch: these bindings expect revision {AbiVersion} but the loaded {LibName} library reports revision {found}"); } - VerifyChecksums(); + VerifyContracts(); } - static partial void VerifyChecksums(); + static partial void VerifyContracts(); private static IntPtr Resolve(string name, Assembly assembly, DllImportSearchPath? searchPath) { @@ -243,25 +365,44 @@ private static IntPtr Resolve(string name, Assembly assembly, DllImportSearchPat return string.IsNullOrEmpty(path) ? IntPtr.Zero : NativeLibrary.Load(path); } - /// Fails the load when module was - /// generated from a different contract than the loaded library - /// implements. - private static void VerifyChecksum(string module, delegate* checksum, ulong expected) + /// Fails the load unless the loaded library's contract table, + /// read through (the import of + /// ), carries every + /// declaration with an unchanged signature hash. Declarations the + /// library has and these bindings don't are fine. + private static void VerifyContract( + string symbol, + delegate* table, + (ulong Id, ulong Hash, string Path)[] expected) { - ulong found; + nuint len = 0; + FfiContractEntry* entries; try { - found = checksum(); + entries = table(&len); } catch (EntryPointNotFoundException e) { throw new InvalidOperationException( - $"the loaded {LibName} library doesn't implement module '{module}'", e); + $"the loaded {LibName} library doesn't export {symbol}; regenerate the bindings or rebuild the library", e); } - if (found != expected) + var hashes = new Dictionary(checked((int)len)); + for (nuint i = 0; i < len; i++) { - throw new InvalidOperationException( - $"contract mismatch for module '{module}': these bindings expect checksum 0x{expected:x16} but the loaded {LibName} library reports 0x{found:x16}; regenerate the bindings for this library"); + hashes[entries[i].Id] = entries[i].Hash; + } + foreach (var (id, hash, path) in expected) + { + if (!hashes.TryGetValue(id, out var actual)) + { + throw new InvalidOperationException( + $"{path} is missing from the library {LibName}; regenerate the bindings or rebuild the library"); + } + if (actual != hash) + { + throw new InvalidOperationException( + $"{path} changed since these bindings were generated; regenerate the bindings or rebuild the library {LibName}"); + } } } @@ -273,6 +414,10 @@ private static void VerifyChecksum(string module, delegate* checksum, ulo [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void ErrorSet(FfiError* err, int code, byte* message); + [LibraryImport(LibName, EntryPoint = "{{PREFIX}}_error_set_payload")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial void ErrorSetPayload(FfiError* err, byte* ptr, nuint len); + [LibraryImport(LibName, EntryPoint = "{{PREFIX}}_error_clear")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void ErrorClear(FfiError* err); @@ -281,6 +426,10 @@ private static void VerifyChecksum(string module, delegate* checksum, ulo [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void ErrorFree(FfiError* err); + [LibraryImport(LibName, EntryPoint = "{{PREFIX}}_alloc")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial byte* Alloc(nuint len); + [LibraryImport(LibName, EntryPoint = "{{PREFIX}}_free_bytes")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void FreeBytes(byte* ptr, nuint len); @@ -576,8 +725,8 @@ internal void WriteBytes(byte[] v) } /// Decodes values from the value-buffer wire format. A malformed -/// buffer is a producer or consumer bug and throws -/// . +/// buffer is a bug on the side that encoded it and throws +/// with the marshalling code. internal sealed class FfiBufferReader { private static readonly Encoding Utf8Strict = new UTF8Encoding(false, true); @@ -590,9 +739,11 @@ internal FfiBufferReader(byte[] data) _data = data; } - private static InvalidOperationException Malformed(string what) + /// The exception for a buffer that breaks the wire + /// format. + internal static {{BUG_EXCEPTION}} Malformed(string what) { - return new InvalidOperationException("malformed value buffer: " + what); + return new {{BUG_EXCEPTION}}({{EXCEPTION}}.MarshalErrorCode, "malformed value buffer: " + what); } private ReadOnlySpan Take(int n) diff --git a/crates/weaveffi-cli/src/targets/dotnet/tests.rs b/crates/weaveffi-cli/src/targets/dotnet/tests.rs new file mode 100644 index 00000000..faa27d0b --- /dev/null +++ b/crates/weaveffi-cli/src/targets/dotnet/tests.rs @@ -0,0 +1,693 @@ +//! Rendering tests over a small API exercising every C ABI revision 4 shape +//! the .NET wrapper distinguishes: the contract check, objects, nullable +//! objects, objects in records and lists, composite codecs, iterators of +//! objects, typed errors with fields, a callback interface returning every +//! family (one method `throws`) passed both required and optional, async +//! and cancellable functions, and bytes parameters. + +use crate::backend::LanguageBackend; +use crate::package::PackageContext; +use crate::platform::{BinarySet, NativeBinary, Platform}; +use camino::Utf8Path; +use weaveffi_model::contract::entries; +use weaveffi_model::ir::Api; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; + +use crate::targets::dotnet::{DotnetConfig, DotnetGenerator}; + +/// A `bus` module with an error domain (one code carrying a field), a +/// `Ticker` interface, a `Bundle` record holding objects, a `Subscriber` +/// callback interface, and free functions covering nullable objects, an +/// iterator of objects, async and cancellable calls, required and optional +/// callback parameters, bytes, and a nested composite; plus a nested +/// `bus.deep_inner` module inheriting the domain. +const FIXTURE: &str = r#" +version: "0.11.0" +modules: + - name: bus + errors: + name: BusError + codes: + - { name: Closed, code: 1, message: "bus closed" } + - { name: Full, code: 2, message: "bus full", fields: [{ name: capacity, type: u32 }, { name: topic, type: string }] } + enums: + - name: Level + variants: [{ name: Low, value: 0 }, { name: High, value: 1 }] + structs: + - name: Bundle + fields: + - { name: primary, type: Ticker } + - { name: others, type: "[Ticker]" } + - { name: label, type: string } + interfaces: + - name: Ticker + doc: A counter the producer owns. + constructors: + - { name: new, params: [{ name: start, type: i32 }] } + methods: + - { name: value, return: i32 } + callback_interfaces: + - name: Subscriber + doc: Receives bus events. + methods: + - name: on_message + params: + - { name: text, type: string } + - { name: weight, type: i32 } + - { name: bundle, type: Bundle } + - { name: ticker, type: Ticker } + return: bool + - { name: on_close, params: [{ name: code, type: i64 }] } + - { name: level, return: Level } + - { name: label, return: string, throws: true } + - { name: payload, return: bytes } + - { name: latest, return: "Bundle?" } + - { name: favorite, return: Ticker } + - { name: spare, return: "Ticker?" } + functions: + - { name: maybe_ticker, params: [{ name: input, type: "Ticker?" }], return: "Ticker?" } + - { name: tickers, return: "iter" } + - { name: subscribe, params: [{ name: listener, type: Subscriber }] } + - { name: maybe_subscribe, params: [{ name: listener, type: "Subscriber?" }], throws: true } + - { name: spawn, return: Ticker, async: true } + - { name: wait, params: [{ name: ms, type: i64 }], return: i64, async: true, cancellable: true } + - { name: roundtrip_bundle, params: [{ name: b, type: Bundle }], return: Bundle } + - { name: bundle, return: Bundle } + - { name: digest, params: [{ name: data, type: bytes }, { name: label, type: string }], return: string } + - { name: tally, params: [{ name: counts, type: "{string:[i32?]}" }], return: "[Bundle]", throws: true } + modules: + - name: deep_inner + functions: + - { name: drain, return: u32, throws: true } +"#; + +fn fixture() -> Model { + let api: Api = serde_yaml::from_str(FIXTURE).expect("fixture parses"); + validate(&api, &Identity::named("bus-lib"), None).expect("fixture validates") +} + +/// Every generated file's text, keyed by file name. +fn render_files(config: &DotnetConfig) -> Vec<(String, String)> { + let model = fixture(); + DotnetGenerator + .files(&model, Utf8Path::new("out"), config) + .into_iter() + .map(|f| (f.path.file_name().unwrap().to_string(), f.contents)) + .collect() +} + +fn file(files: &[(String, String)], name: &str) -> String { + files + .iter() + .find(|(n, _)| n == name) + .unwrap_or_else(|| { + panic!( + "no {name} in {:?}", + files.iter().map(|f| &f.0).collect::>() + ) + }) + .1 + .clone() +} + +fn render() -> String { + file(&render_files(&DotnetConfig::default()), "BusLib.cs") +} + +fn runtime() -> String { + file(&render_files(&DotnetConfig::default()), "Runtime.cs") +} + +#[track_caller] +fn assert_has(out: &str, needle: &str) { + assert!(out.contains(needle), "missing `{needle}` in:\n{out}"); +} + +/// The text of the C# member starting at `start`, up to the next blank +/// line at its indentation. +fn member<'a>(out: &'a str, start: &str) -> &'a str { + let at = out + .find(start) + .unwrap_or_else(|| panic!("no `{start}` in:\n{out}")); + let rest = &out[at..]; + let end = rest.find("\n\n").unwrap_or(rest.len()); + &rest[..end] +} + +#[test] +fn names_come_from_the_identity() { + let files = render_files(&DotnetConfig::default()); + let names: Vec<&str> = files.iter().map(|f| f.0.as_str()).collect(); + assert_eq!( + names, + ["BusLib.cs", "Runtime.cs", "BusLib.csproj", "README.md"] + ); + let runtime = file(&files, "Runtime.cs"); + assert_has(&runtime, "namespace BusLib;"); + assert_has(&runtime, "internal const string LibName = \"bus_lib\";"); + assert_has( + &runtime, + "internal const string LibraryEnvVar = \"BUS_LIB_LIBRARY\";", + ); + assert_has(&runtime, "EntryPoint = \"bus_lib_abi_version\""); + assert_has(&runtime, "internal const uint AbiVersion = 4;"); + assert_has(&runtime, "public class NativeException : Exception"); + assert_has( + &runtime, + "public sealed class NativeBugException : InvalidOperationException", + ); + assert_has(&runtime, "public const int CancelledErrorCode = -5;"); + assert!( + !runtime.contains("{{"), + "unreplaced placeholder:\n{runtime}" + ); + let csproj = file(&files, "BusLib.csproj"); + assert_has(&csproj, "BusLib"); + assert_has(&csproj, "BusLib"); + for (name, text) in &files { + let body: String = text + .lines() + .filter(|l| !l.contains("Generated by WeaveFFI") && !l.contains("weaveffi generate")) + .collect(); + assert!( + !body.to_lowercase().contains("weaveffi"), + "{name} is branded:\n{text}" + ); + } +} + +#[test] +fn configured_namespace_wins() { + let config = DotnetConfig { + name: Some("Acme.Bus".into()), + }; + let files = render_files(&config); + assert_has(&file(&files, "Acme.Bus.cs"), "namespace Acme.Bus;"); + assert_has( + &file(&files, "Acme.Bus.csproj"), + "Acme.Bus", + ); +} + +#[test] +fn the_runtime_surface_is_revision_4() { + let runtime = runtime(); + assert_has(&runtime, "EntryPoint = \"bus_lib_alloc\""); + assert_has(&runtime, "internal static partial byte* Alloc(nuint len);"); + assert_has(&runtime, "EntryPoint = \"bus_lib_error_set_payload\""); + assert!(!runtime.contains("_dealloc"), "{runtime}"); + assert!(!runtime.to_lowercase().contains("checksum"), "{runtime}"); +} + +#[test] +fn load_checks_the_abi_revision_and_every_root_contract() { + let model = fixture(); + let out = render(); + assert_has(&out, "static partial void VerifyContracts()"); + assert_has( + &out, + "VerifyContract(\"bus_lib_bus_contract\", &bus_lib_bus_contract, new (ulong, ulong, string)[]", + ); + assert_has( + &out, + "internal static partial FfiContractEntry* bus_lib_bus_contract(nuint* out_len);", + ); + let root = model.roots().next().unwrap(); + let expected = entries(&model, root); + assert!(expected.iter().any(|e| e.path == "bus.deep_inner.drain")); + for e in &expected { + assert_has( + &out, + &format!("(0x{:016x}UL, 0x{:016x}UL, \"{}\"),", e.id, e.hash, e.path), + ); + } + let runtime = runtime(); + assert_has(&runtime, "VerifyContracts();"); + assert_has(&runtime, "is missing from the library"); + assert_has(&runtime, "changed since these bindings were generated"); +} + +#[test] +fn interfaces_wrap_a_safe_handle() { + let out = render(); + assert_has( + &out, + "public sealed unsafe class Ticker : IDisposable, IEquatable", + ); + assert_has(&out, "internal sealed class NativeHandle : SafeHandle"); + assert_has(&out, "NativeMethods.bus_lib_bus_Ticker_destroy(handle);"); + assert!( + !out.contains("~Ticker()"), + "the SafeHandle is the finalizer" + ); + // Every import borrows through the handle, never a raw pointer. + assert_has( + &out, + "internal static partial int bus_lib_bus_Ticker_value(Ticker.NativeHandle self, FfiError* out_err);", + ); + assert_has( + &out, + "internal static partial IntPtr bus_lib_bus_Ticker_clone(Ticker.NativeHandle self);", + ); + assert_has( + &out, + "var ffiResult = NativeMethods.bus_lib_bus_Ticker_value(Handle, &ffiErr);", + ); + assert_has(&out, "public Ticker(int start)"); + assert_has(&out, "Handle = new NativeHandle(ffiResult);"); + assert_has( + &out, + "return NativeMethods.bus_lib_bus_Ticker_clone(Handle);", + ); +} + +#[test] +fn nullable_objects_pass_the_null_handle() { + let out = render(); + assert_has(&out, "public static Ticker? MaybeTicker(Ticker? input)"); + assert_has( + &out, + "NativeMethods.bus_lib_bus_maybe_ticker(input?.Handle ?? global::BusLib.Ticker.NativeHandle.Null, &ffiErr);", + ); + assert_has( + &out, + "return ffiResult == IntPtr.Zero ? null : global::BusLib.Ticker.Adopt(ffiResult);", + ); +} + +#[test] +fn objects_in_records_use_cloned_tokens_and_adopt_on_read() { + let out = render(); + assert_has(&out, "public sealed class Bundle"); + assert_has(&out, "writer.WriteObject(Primary.CloneHandle());"); + assert_has(&out, "FfiCodecs.WriteListOfTicker(writer, Others);"); + assert_has(&out, "writer.WriteObject(item.CloneHandle());"); + assert_has(&out, "global::BusLib.Ticker.Adopt(reader.ReadObject()),"); + assert_has( + &out, + "list.Add(global::BusLib.Ticker.Adopt(reader.ReadObject()));", + ); +} + +#[test] +fn composites_get_one_codec_pair_each() { + let out = render(); + for name in [ + "ListOfTicker", + "ListOfBundle", + "ListOfOptionalOfI32", + "MapOfStringToListOfOptionalOfI32", + "OptionalOfBundle", + ] { + assert_eq!( + out.matches(&format!("internal static void Write{name}(")) + .count(), + 1, + "{name}" + ); + assert_eq!( + out.matches(&format!(" Read{name}(FfiBufferReader reader)")) + .count(), + 1, + "{name}" + ); + } + // The call site writes through the pair instead of inlining a loop. + let tally = member(&out, "public static Bundle[] Tally("); + assert_has( + tally, + "FfiCodecs.WriteMapOfStringToListOfOptionalOfI32(countsWriter, counts);", + ); + assert!(!tally.contains("foreach"), "{tally}"); + // A repeated map key is malformed. + assert_has( + &out, + "throw FfiBufferReader.Malformed(\"repeated map key\");", + ); +} + +#[test] +fn type_references_survive_a_member_of_the_same_name() { + let out = render(); + // Inside `Bus`, the method `Bundle()` would shadow the type `Bundle` in + // an expression, so static accesses go through the global namespace. + assert_has(&out, "public static Bundle Bundle()"); + assert_has( + &out, + "return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::BusLib.Bundle.ReadFrom(r));", + ); +} + +#[test] +fn strings_and_bytes_cross_as_pointer_and_length() { + let out = render(); + assert_has( + &out, + "internal static partial byte* bus_lib_bus_digest(byte* data_ptr, nuint data_len, byte* label_ptr, nuint label_len, nuint* out_len, FfiError* out_err);", + ); + assert_has( + &out, + "public static string Digest(ReadOnlySpan data, string label)", + ); + assert_has(&out, "var labelBytes = Ffi.Utf8(label);"); + assert_has(&out, "fixed (byte* dataPtr = data, labelPtr = labelBytes)"); + assert_has(&out, "return Ffi.TakeString(ffiResult, ffiOutLen);"); + // The byte[] overload forwards to the span one. + assert_has( + &out, + "public static string Digest(byte[] data, string label) => Digest((ReadOnlySpan)data, label);", + ); +} + +#[test] +fn errors_follow_the_throws_flag() { + let out = render(); + // A domain is an abstract class with one nested class per code; fields + // are typed properties and the payload decodes into them. + assert_has(&out, "public abstract class BusException : NativeException"); + assert_has(&out, "public sealed class Full : BusException"); + assert_has(&out, "public const int ErrorCode = 2;"); + assert_has(&out, "public uint Capacity { get; }"); + assert_has(&out, "public string Topic { get; }"); + assert_has( + &out, + "public Full(uint capacity, string topic, string? message = null) : base(ErrorCode, message ?? \"bus full\")", + ); + assert_has(&out, "case Full.ErrorCode:"); + assert_has(&out, "return new Closed(given);"); + // A throwing call maps through its domain, a non-throwing one traps. + let tally = member(&out, "public static Bundle[] Tally("); + assert_has( + tally, + "throw Ffi.TakeError(&ffiErr, global::BusLib.BusException.FromError);", + ); + let digest = member(&out, "public static string Digest(ReadOnlySpan"); + assert_has( + digest, + "throw Ffi.TakeError(&ffiErr, global::BusLib.NativeBugException.FromError);", + ); + // A nested module inherits the domain, and the doc names the + // declaring module's dotted path. + let drain = member(&out, "public static uint Drain()"); + assert_has(drain, "global::BusLib.BusException.FromError"); + assert_has(&out, "declared by module bus."); + let runtime = runtime(); + assert_has(&runtime, "native call failed with code {code}: {message}"); + assert_has(&runtime, "return new OperationCanceledException(message);"); +} + +#[test] +fn the_domain_doc_names_the_dotted_module_path() { + let yaml = r#" +version: "0.11.0" +modules: + - name: edge_cases + errors: + name: EdgeError + codes: [{ name: Busy, code: 7, message: "busy" }] + functions: + - { name: poke, throws: true } +"#; + let api: Api = serde_yaml::from_str(yaml).unwrap(); + let model = validate(&api, &Identity::named("edge"), None).unwrap(); + let out = super::render_csharp( + &model, + "Edge", + ("NativeException", "NativeBugException"), + "Edge.cs", + ); + assert_has(&out, "declared by module edge_cases."); + assert!(!out.contains("edge.cases"), "{out}"); +} + +#[test] +fn iterators_stream_through_a_safe_handle() { + let out = render(); + assert_has(&out, "public static IEnumerable Tickers()"); + assert_has( + &out, + "return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.bus_lib_bus_TickersIterator_destroy), &NextTickers);", + ); + assert_has( + &out, + "private static bool NextTickers(FfiIteratorHandle iter, out Ticker item)", + ); + assert_has( + &out, + "internal static partial int bus_lib_bus_TickersIterator_next(FfiIteratorHandle iter, IntPtr* out_item, FfiError* out_err);", + ); + assert_has(&out, "item = global::BusLib.Ticker.Adopt(ffiItem);"); +} + +#[test] +fn async_calls_complete_through_an_unmanaged_callback() { + let out = render(); + assert_has(&out, "public static Task Spawn()"); + assert_has(&out, "var ffiCall = new FfiCall();"); + assert_has( + &out, + "NativeMethods.bus_lib_bus_spawn(&CompleteSpawn, ffiCall.Context);", + ); + assert_has( + &out, + "internal static partial void bus_lib_bus_spawn(delegate* unmanaged[Cdecl] callback, IntPtr context);", + ); + assert_has( + &out, + "private static void CompleteSpawn(IntPtr context, FfiError* err, IntPtr result)", + ); + assert_has( + &out, + "ffiCall.SetResult(global::BusLib.Ticker.Adopt(result));", + ); + assert_has(&out, "ffiCall.Abandon();"); +} + +#[test] +fn cancellable_calls_take_a_cancellation_token() { + let out = render(); + assert_has( + &out, + "public static Task Wait(long ms, CancellationToken cancellationToken = default)", + ); + assert_has(&out, "return Task.FromCanceled(cancellationToken);"); + assert_has(&out, "var ffiCall = new FfiCall(cancellationToken);"); + assert_has( + &out, + "NativeMethods.bus_lib_bus_wait(ms, ffiCall.CancelToken(), &CompleteWait, ffiCall.Context);", + ); + let runtime = runtime(); + assert_has(&runtime, "_tcs.TrySetCanceled(_cancellation);"); + assert_has(&runtime, "NativeMethods.CancelTokenCancel(_token);"); +} + +#[test] +fn callback_vtables_start_with_the_header() { + let out = render(); + assert_has( + &out, + "internal static unsafe class FfiVtable_bus_Subscriber", + ); + let layout = member(&out, "private struct Layout"); + let header = [ + "public uint Size;", + "public uint Flags;", + "public delegate* unmanaged[Cdecl] Free;", + "public delegate* unmanaged[Cdecl] on_message;", + ]; + let mut at = 0; + for field in header { + let found = layout[at..] + .find(field) + .unwrap_or_else(|| panic!("`{field}` out of order in:\n{layout}")); + at += found + field.len(); + } + assert_has(&out, "vtable->Size = (uint)sizeof(Layout);"); + assert_has(&out, "vtable->Flags = 0;"); + assert_has(&out, "vtable->Free = &FreeTrampoline;"); + assert_has(&out, "vtable->on_message = &OnMessageTrampoline;"); +} + +#[test] +fn callback_methods_receive_borrowed_arguments_and_adopt_objects() { + let out = render(); + assert_has(&out, "public interface ISubscriber"); + assert_has( + &out, + "bool OnMessage(string text, int weight, Bundle bundle, Ticker ticker);", + ); + let trampoline = member(&out, "private static byte OnMessageTrampoline("); + assert_has( + trampoline, + "private static byte OnMessageTrampoline(IntPtr ctx, byte* text_ptr, nuint text_len, int weight, byte* bundle_ptr, nuint bundle_len, IntPtr ticker, FfiError* out_err)", + ); + assert_has(trampoline, "Ffi.ReadString(text_ptr, text_len)"); + assert_has( + trampoline, + "FfiCodecs.Decode(Ffi.ReadBuffer(bundle_ptr, bundle_len), static r => global::BusLib.Bundle.ReadFrom(r))", + ); + assert_has(trampoline, "global::BusLib.Ticker.Adopt(ticker)"); + assert_has(trampoline, "Ffi.SetForeignError(out_err, e);"); + // Arguments are borrowed: nothing is freed inside a trampoline. + assert!(!trampoline.contains("Take"), "{trampoline}"); +} + +#[test] +fn callback_methods_return_every_family() { + let out = render(); + assert_has(&out, "Level Level();"); + assert_has(&out, "string Label();"); + assert_has(&out, "byte[] Payload();"); + assert_has(&out, "Bundle? Latest();"); + assert_has(&out, "Ticker Favorite();"); + assert_has(&out, "Ticker? Spare();"); + // Direct values by value. + assert_has( + member(&out, "private static int LevelTrampoline("), + "return (int)Ffi.Target(ctx).Level();", + ); + // Strings, bytes, and buffers as `{prefix}_alloc` runs in the out slots. + let label = member(&out, "private static void LabelTrampoline("); + assert_has( + label, + "LabelTrampoline(IntPtr ctx, byte** out_ptr, nuint* out_len, FfiError* out_err)", + ); + assert_has( + label, + "Ffi.ReturnString(Ffi.Target(ctx).Label(), out_ptr, out_len);", + ); + assert_has( + member(&out, "private static void PayloadTrampoline("), + "Ffi.ReturnBytes(Ffi.Target(ctx).Payload(), out_ptr, out_len);", + ); + let latest = member(&out, "private static void LatestTrampoline("); + assert_has( + latest, + "FfiCodecs.WriteOptionalOfBundle(ffiWriter, ffiResult);", + ); + assert_has(latest, "Ffi.ReturnBuffer(ffiWriter, out_ptr, out_len);"); + // Objects as a fresh reference; a null passes through for the producer + // to judge. + assert_has( + member(&out, "private static IntPtr FavoriteTrampoline("), + "return Ffi.Target(ctx).Favorite()?.CloneHandle() ?? IntPtr.Zero;", + ); + assert_has( + member(&out, "private static IntPtr SpareTrampoline("), + "return Ffi.Target(ctx).Spare()?.CloneHandle() ?? IntPtr.Zero;", + ); + let runtime = runtime(); + assert_has( + &runtime, + "var run = NativeMethods.Alloc((nuint)bytes.Length);", + ); +} + +#[test] +fn throwing_callback_methods_report_the_domain_error() { + let out = render(); + let label = member(&out, "private static void LabelTrampoline("); + assert_has(label, "catch (global::BusLib.BusException e)"); + assert_has( + label, + "Ffi.SetDomainError(out_err, e.Code, e, e.WritePayload);", + ); + assert_has(label, "catch (Exception e)"); + // Only `throws` methods report the domain; the rest are foreign errors. + let payload = member(&out, "private static void PayloadTrampoline("); + assert!(!payload.contains("BusException"), "{payload}"); + // Each code with fields encodes them as the payload. + assert_has( + &out, + "internal override void WritePayload(FfiBufferWriter writer)\n {\n writer.WriteU32(Capacity);\n writer.WriteString(Topic);", + ); + let runtime = runtime(); + assert_has( + &runtime, + "NativeMethods.ErrorSetPayload(err, ptr, (nuint)payload.Length);", + ); +} + +#[test] +fn passing_a_callback_interface_registers_it_after_encoding() { + let out = render(); + assert_has(&out, "public static void Subscribe(ISubscriber listener)"); + assert_has(&out, "var listenerCtx = Ffi.Register(listener);"); + assert_has( + &out, + "NativeMethods.bus_lib_bus_subscribe(listenerCtx, FfiVtable_bus_Subscriber.Pointer, &ffiErr);", + ); + // A call that never reaches the producer releases the registration. + assert_has(&out, "Ffi.Unregister(listenerCtx);"); +} + +#[test] +fn an_absent_optional_callback_passes_a_null_vtable() { + let out = render(); + assert_has( + &out, + "public static void MaybeSubscribe(ISubscriber? listener)", + ); + assert_has( + &out, + "NativeMethods.bus_lib_bus_maybe_subscribe(listenerCtx, listener == null ? IntPtr.Zero : FfiVtable_bus_Subscriber.Pointer, &ffiErr);", + ); + let runtime = runtime(); + assert_has( + &runtime, + "return implementation == null ? IntPtr.Zero : GCHandle.ToIntPtr(GCHandle.Alloc(implementation));", + ); +} + +#[test] +fn output_is_deterministic() { + assert_eq!( + render_files(&DotnetConfig::default()), + render_files(&DotnetConfig::default()) + ); +} + +#[test] +fn package_bundles_desktop_runtimes_only() { + let model = fixture(); + let mut binaries = BinarySet::new("bus_lib"); + for p in Platform::ALL { + binaries.insert(NativeBinary::new(p, format!("/tmp/{}/lib", p.id()))); + } + let ctx = PackageContext::new(&binaries); + let artifacts = DotnetGenerator + .package(&model, &ctx, &DotnetConfig::default()) + .expect("dotnet packages"); + assert_eq!(artifacts.len(), 1); + assert_eq!(artifacts[0].path, "dotnet/BusLib"); + assert!(artifacts[0].file("BusLib.csproj").is_some()); + let paths: Vec = artifacts[0] + .files + .iter() + .map(|f| f.path.to_string()) + .collect(); + let native: Vec<&String> = paths + .iter() + .filter(|p| p.starts_with("runtimes/")) + .collect(); + assert_eq!(native.len(), Platform::DESKTOP.len(), "{paths:?}"); + for rid in [ + "osx-arm64", + "osx-x64", + "linux-x64", + "linux-arm64", + "win-x64", + ] { + assert!( + paths + .iter() + .any(|p| p.starts_with(&format!("runtimes/{rid}/native/"))), + "{paths:?}" + ); + } + assert!(!paths + .iter() + .any(|p| p.contains("android") || p.contains("wasm"))); +} diff --git a/crates/weaveffi-gen/src/targets/dotnet/types.rs b/crates/weaveffi-cli/src/targets/dotnet/types.rs similarity index 62% rename from crates/weaveffi-gen/src/targets/dotnet/types.rs rename to crates/weaveffi-cli/src/targets/dotnet/types.rs index 5f9fc02b..8589fb8d 100644 --- a/crates/weaveffi-gen/src/targets/dotnet/types.rs +++ b/crates/weaveffi-cli/src/targets/dotnet/types.rs @@ -3,18 +3,20 @@ //! string-literal escaping. use crate::lang; -use crate::utils::local_type_name; -use heck::ToLowerCamelCase; +use heck::{ToLowerCamelCase, ToUpperCamelCase}; use weaveffi_model::abi::{AbiParam, CType}; -use weaveffi_model::model::{FnBinding, Ty}; +use weaveffi_model::model::FnBinding; +use weaveffi_model::ty::{Prim, Ty}; /// The names every renderer shares. #[derive(Clone, Copy)] pub(crate) struct Cx<'a> { /// The C# namespace. pub ns: &'a str, - /// The base exception class. + /// The root exception class of throwing calls. pub base: &'a str, + /// The exception a failed non-throwing call raises. + pub bug: &'a str, } impl Cx<'_> { @@ -22,47 +24,43 @@ impl Cx<'_> { /// contexts (`Card.ReadFrom(...)`) where a member of the enclosing class /// with the same name (a `Card()` method) would otherwise shadow it. pub(crate) fn ty(&self, name: &str) -> String { - format!("global::{}.{}", self.ns, local_type_name(name)) + format!("global::{}.{}", self.ns, name) } } -/// The C# interface a consumer implements for one callback interface: the -/// bare local name with the conventional `I` prefix (`ISubscriber` for -/// `Subscriber` or `bus.Subscriber`). +/// The C# interface a consumer implements for one callback interface: its +/// name with the conventional `I` prefix (`ISubscriber` for `Subscriber`). pub(crate) fn callback_interface_cs(name: &str) -> String { - format!("I{}", local_type_name(name)) + format!("I{name}") } /// The internal static class hosting one callback interface's vtable and -/// trampolines, named from the module path and the bare name so two modules -/// declaring the same callback interface name never collide. +/// trampolines, named from the declaring module's path and the name, like +/// its C vtable tag. pub(crate) fn vtable_class_cs(module_path: &str, name: &str) -> String { - format!("FfiVtable_{module_path}_{}", local_type_name(name)) + format!("FfiVtable_{module_path}_{}", name) } /// The idiomatic C# surface type for one IR type, as it appears in wrapper /// signatures, properties, and locals. pub(crate) fn cs_type(ty: &Ty) -> String { match ty { - Ty::I8 => "sbyte".into(), - Ty::I16 => "short".into(), - Ty::I32 => "int".into(), - Ty::U8 => "byte".into(), - Ty::U16 => "ushort".into(), - Ty::U32 => "uint".into(), - Ty::I64 => "long".into(), - Ty::U64 => "ulong".into(), - Ty::F32 => "float".into(), - Ty::F64 => "double".into(), - Ty::Bool => "bool".into(), - Ty::StringUtf8 => "string".into(), - Ty::Bytes => "byte[]".into(), + Ty::Prim(Prim::I8) => "sbyte".into(), + Ty::Prim(Prim::I16) => "short".into(), + Ty::Prim(Prim::I32) => "int".into(), + Ty::Prim(Prim::U8) => "byte".into(), + Ty::Prim(Prim::U16) => "ushort".into(), + Ty::Prim(Prim::U32) => "uint".into(), + Ty::Prim(Prim::I64) => "long".into(), + Ty::Prim(Prim::U64) => "ulong".into(), + Ty::Prim(Prim::F32) => "float".into(), + Ty::Prim(Prim::F64) => "double".into(), + Ty::Prim(Prim::Bool) => "bool".into(), + Ty::Prim(Prim::String) => "string".into(), + Ty::Prim(Prim::Bytes) => "byte[]".into(), // Records, rich enums, C-style enums, and interfaces surface under - // their bare local name; a cross-module reference (`kv.Store`) uses - // the same name because every module shares one namespace. - Ty::Record(name) | Ty::RichEnum(name) | Ty::Enum(name) | Ty::Interface(name) => { - local_type_name(name).into() - } + // their (global) name, because every module shares one namespace. + Ty::Record(name) | Ty::RichEnum(name) | Ty::Enum(name) | Ty::Interface(name) => name.into(), Ty::Optional(inner) => format!("{}?", cs_type(inner)), Ty::List(inner) => format!("{}[]", cs_type(inner)), Ty::Iterator(inner) => format!("IEnumerable<{}>", cs_type(inner)), @@ -71,26 +69,6 @@ pub(crate) fn cs_type(ty: &Ty) -> String { } } -/// True when `ty` surfaces as a C# value type, so its optional wrapper is -/// `Nullable` and a present value is read through `.Value`. -pub(crate) fn is_cs_value_type(ty: &Ty) -> bool { - matches!( - ty, - Ty::I8 - | Ty::I16 - | Ty::I32 - | Ty::U8 - | Ty::U16 - | Ty::U32 - | Ty::I64 - | Ty::U64 - | Ty::F32 - | Ty::F64 - | Ty::Bool - | Ty::Enum(_) - ) -} - /// True for C types that only ever appear behind a pointer and are opaque /// to C#: interface objects, iterators, vtables, cancel tokens, and `void`. fn is_opaque(ty: &CType) -> bool { @@ -161,6 +139,39 @@ pub(crate) fn safe_cs_name(name: &str) -> String { } } +/// Members every interface wrapper class declares or inherits: its handle +/// plumbing (`Handle`, `Adopt`, `CloneHandle`, the nested `NativeHandle`), +/// `IDisposable.Dispose`, and `object`'s members. Statics share the +/// namespace with instance members, so every interface member is escaped +/// against the same set. +const OBJECT_MEMBERS: &[&str] = &[ + "Adopt", + "CloneHandle", + "Dispose", + "Equals", + "Finalize", + "GetHashCode", + "GetType", + "Handle", + "MemberwiseClone", + "NativeHandle", + "ReferenceEquals", + "ToString", +]; + +/// The C# name of a member of the interface `class`: PascalCase, with a +/// trailing `_` when it would collide with a member the wrapper declares or +/// inherits (`dispose` is `Dispose_`) or with the class's own name, which +/// C# reserves for constructors. +pub(crate) fn cs_member(name: &str, class: &str) -> String { + let member = lang::escape_member(&name.to_upper_camel_case(), OBJECT_MEMBERS); + if member == class { + format!("{member}_") + } else { + member + } +} + /// A copy of `f` whose parameter names are lowerCamelCase, the C# parameter /// convention. Only wrapper signatures and their locals derive from these /// names; ABI slot names keep the IDL spelling. @@ -171,3 +182,17 @@ pub(crate) fn camel_fn(f: &FnBinding) -> FnBinding { } f } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn interface_members_avoid_the_wrapper_members() { + assert_eq!(cs_member("dispose", "Store"), "Dispose_"); + assert_eq!(cs_member("to_string", "Store"), "ToString_"); + assert_eq!(cs_member("handle", "Store"), "Handle_"); + assert_eq!(cs_member("store", "Store"), "Store_"); + assert_eq!(cs_member("close", "Store"), "Close"); + } +} diff --git a/crates/weaveffi-cli/src/targets/go/callbacks.rs b/crates/weaveffi-cli/src/targets/go/callbacks.rs new file mode 100644 index 00000000..7e584115 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/callbacks.rs @@ -0,0 +1,343 @@ +//! Callback interfaces: the Go `interface` a consumer implements, the +//! exported trampolines behind each vtable entry, and the cgo preamble +//! declaring them and the one static vtable per interface. +//! +//! An implementation crosses as a handle-table context (see +//! `wvNewCallback`) plus the address of its interface's static vtable, +//! whose header records the vtable's `size` and the `free` trampoline that +//! deletes the handle once the native library is done with it. + +use crate::cabi::c_param_name; +use crate::codegen::CodeWriter; +use weaveffi_model::abi::{AbiParam, CType}; +use weaveffi_model::model::{CallShape, CallbackInterfaceBinding, CallbackMethodBinding, Model}; +use weaveffi_model::plan::{ArgPass, ErrorStrategy, RetPass}; + +use crate::targets::go::calls::{completion_trampoline, vtable_accessor}; +use crate::targets::go::codec::{func, read_fn, write_fn}; +use crate::targets::go::docs::{GoDoc, Kind}; +use crate::targets::go::names::{self, domain_type, pascal}; +use crate::targets::go::types::{cgo_type, from_c_direct, go_type, strip_const, to_c_direct}; +use crate::targets::go::Ctx; + +/// The C name of the exported trampoline behind one vtable entry; +/// `free` names the header's release entry. +fn trampoline(c_tag: &str, method: &str) -> String { + format!("goWv_{c_tag}_{method}") +} + +/// The C identifier of the process-wide static vtable of `vtable_tag`. +fn vtable_var(vtable_tag: &str) -> String { + format!("wvVtable_{vtable_tag}") +} + +/// The preamble `extern` declaration of one exported trampoline. Pointer +/// types are const-free to match the prototypes cgo writes into +/// `_cgo_export.h` from the Go signature. +fn extern_decl(name: &str, ret: &CType, params: &[AbiParam], prefix: &str) -> String { + let args: Vec = params + .iter() + .map(|p| { + format!( + "{} {}", + strip_const(&p.ty).render_c(prefix), + c_param_name(&p.name) + ) + }) + .collect(); + format!( + "extern {} {name}({});", + ret.render_c(prefix), + args.join(", ") + ) +} + +/// Write every declaration the cgo preamble needs beyond the header: for +/// each callback interface, the `extern` prototypes of its trampolines, +/// its static vtable, and the accessor returning the vtable's address; then +/// one `extern` per async completion trampoline. +/// +/// A file that uses `//export` may only declare in its preamble (the +/// preamble is compiled into two C translation units), so the vtable is a +/// `static const`: each unit gets a private copy and no symbol is +/// duplicated. Go reaches the copy in its own unit through the accessor, +/// so the native library always sees one vtable that lives for the +/// process. +pub(crate) fn emit_preamble_decls(w: &mut CodeWriter, model: &Model) { + let prefix = model.prefix(); + for (_, cb) in model.callback_interfaces() { + for m in &cb.methods { + w.line(extern_decl( + &trampoline(&cb.c_tag, &m.name), + &m.abi_ret, + &m.abi_params, + prefix, + )); + } + let free = trampoline(&cb.c_tag, "free"); + w.line(extern_decl( + &free, + &CType::Void, + &[AbiParam::new("ctx", CType::ptr(CType::Void))], + prefix, + )); + let tag = &cb.vtable_tag; + w.block( + format!("static const {tag} {} = {{", vtable_var(tag)), + "};", + |w| { + w.line(format!(".size = sizeof({tag}),")); + w.line(".flags = 0,"); + w.line(format!(".free = {free},")); + for m in &cb.methods { + let tramp = trampoline(&cb.c_tag, &m.name); + // An entry whose C signature has `const` pointers is + // cast to the field's exact type, since the exported + // Go function is declared const-free. + if m.abi_params.iter().any(|p| strip_const(&p.ty) != p.ty) { + let types: Vec = + m.abi_params.iter().map(|p| p.ty.render_c(prefix)).collect(); + w.line(format!( + ".{} = ({} (*)({})){tramp},", + c_param_name(&m.name), + m.abi_ret.render_c(prefix), + types.join(", ") + )); + } else { + w.line(format!(".{} = {tramp},", c_param_name(&m.name))); + } + } + }, + ); + w.line(format!( + "__attribute__((unused)) static const {tag}* {}(void) {{ return &{}; }}", + vtable_accessor(tag), + vtable_var(tag) + )); + } + for (_, f) in model.callables() { + if let CallShape::Async(ab) = &f.shape { + w.line(extern_decl( + &completion_trampoline(&ab.callback_type), + &CType::Void, + &ab.callback_params, + prefix, + )); + } + } +} + +/// The Go method signature of one callback method: its parameters, and its +/// result, as `(T, error)` or `error` when it throws. +fn method_sig(m: &CallbackMethodBinding) -> String { + let params: Vec = m + .params + .iter() + .map(|p| format!("{} {}", names::method_param(&p.name), go_type(&p.ty))) + .collect(); + let ret = match (&m.ret, m.throws) { + (Some(ty), true) => format!(" ({}, error)", go_type(ty)), + (Some(ty), false) => format!(" {}", go_type(ty)), + (None, true) => " error".into(), + (None, false) => String::new(), + }; + format!("{}({}){ret}", pascal(&m.name), params.join(", ")) +} + +/// Render one callback interface: the Go `interface` the consumer +/// implements, the trampoline behind each vtable entry, and the `free` +/// trampoline. +pub(crate) fn render_callback_interface( + w: &mut CodeWriter, + ctx: &Ctx, + cb: &CallbackInterfaceBinding, +) { + let name = pascal(&cb.name); + let domain = ctx.domain.map(domain_type); + GoDoc::new(&name, cb.doc.as_deref(), Kind::Value, None) + .para(&format!( + "Implement {name} in Go and pass the value to the functions that take it. \ + The native library may call its methods from any thread until it \ + releases the implementation. A method that panics, or a throwing method \ + that returns an error outside its domain, fails the native call in \ + progress as a callback failure (code -4)." + )) + .deprecated(cb.deprecated.as_deref()) + .emit(w); + w.block(format!("type {name} interface {{"), "}", |w| { + for m in &cb.methods { + let kind = Kind::callable(m.ret.is_some()); + let mut doc = GoDoc::new(&pascal(&m.name), m.doc.as_deref(), kind, None) + .params(&m.params, names::method_param); + if m.throws { + let line = match &domain { + Some(d) => format!( + "Return a {d} to report that code, with its fields, to the native caller." + ), + None => "A returned error reaches the native caller as code -4.".into(), + }; + doc = doc.para(&line); + } + doc.deprecated(m.deprecated.as_deref()).emit(w); + w.line(method_sig(m)); + } + }); + w.blank(); + + for m in &cb.methods { + render_trampoline(w, ctx, cb, m, &name, domain.as_deref()); + } + + let free = trampoline(&cb.c_tag, "free"); + w.line(format!("//export {free}")); + func( + w, + &format!("func {free}(ctx unsafe.Pointer)"), + "wvFreeCallback(ctx)", + ); +} + +/// The Go expression converting one callback argument's C slots into the +/// Go value the implementation receives: strings, bytes, and buffers are +/// borrowed for the call, so they're copied or decoded; an object transfers +/// one strong reference, adopted into a new wrapper. +fn argument(p: &weaveffi_model::model::ParamBinding) -> String { + match p.arg_pass() { + ArgPass::Buffer { ptr, len } => format!( + "wvDecodeBorrowed({}, {}, {})", + names::slot(&ptr.name), + names::slot(&len.name), + read_fn(&p.ty) + ), + ArgPass::String { ptr, len } => format!( + "wvBorrowString({}, {})", + names::slot(&ptr.name), + names::slot(&len.name) + ), + ArgPass::Bytes { ptr, len } => format!( + "wvBorrowBytes({}, {})", + names::slot(&ptr.name), + names::slot(&len.name) + ), + ArgPass::Object { slot, .. } => { + let n = p.ty.interface_name().expect("an object names an interface"); + format!("wvAdopt{}({})", pascal(n), names::slot(&slot.name)) + } + ArgPass::Direct { slot } => from_c_direct(&names::slot(&slot.name), &p.ty), + ArgPass::Callback { .. } => { + unreachable!("validation rejects callback interfaces as callback-method parameters") + } + } +} + +/// Render the exported trampoline behind one vtable entry. It recovers the +/// implementation from the context, converts the arguments, calls the Go +/// method, and hands the result back: a direct value as the C return, an +/// object as a fresh strong reference the native library adopts (nil stays +/// null), and a string, bytes, or buffer as a run allocated with the +/// library's allocator in the out slots. A throwing method's error is +/// reported through `out_err` with its domain code and payload (code -4 +/// for any other error), and a panic is recovered into code -4; nothing +/// unwinds through the C frame. +fn render_trampoline( + w: &mut CodeWriter, + ctx: &Ctx, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, + iface: &str, + domain: Option<&str>, +) { + let tramp = trampoline(&cb.c_tag, &m.name); + let formals: Vec = m + .abi_params + .iter() + .map(|s| format!("{} {}", names::slot(&s.name), cgo_type(&s.ty, ctx.prefix))) + .collect(); + let slot_named = |n: &str| { + m.abi_params + .iter() + .find(|p| p.name == n) + .map(|p| names::slot(&p.name)) + }; + let first = names::slot(&m.abi_params[0].name); + let out_err = names::slot(&m.abi_params[m.abi_params.len() - 1].name); + let direct_ret = m.abi_ret != CType::Void; + let ret_sig = if direct_ret { + format!(" (cRet {})", cgo_type(&m.abi_ret, ctx.prefix)) + } else { + String::new() + }; + let pass = RetPass::of(m.ret.as_ref()); + let throws = m.error_strategy() == ErrorStrategy::Throws; + let args: Vec = m.params.iter().map(argument).collect(); + let call = format!( + "wvCallback[{iface}]({first}).{}({})", + pascal(&m.name), + args.join(", ") + ); + + w.line(format!("//export {tramp}")); + w.block( + format!("func {tramp}({}){ret_sig} {{", formals.join(", ")), + "}", + |w| { + w.line(format!("defer wvRecoverCallback({out_err})")); + let failed = format!( + "wvCallbackFailed[{}]({out_err}, err)", + domain.unwrap_or("error") + ); + match (&m.ret, throws) { + (None, false) => { + w.line(call); + return; + } + (None, true) => { + w.block(format!("if err := {call}; err != nil {{"), "}", |w| { + w.line(failed); + }); + return; + } + (Some(_), false) => { + w.line(format!("ret := {call}")); + } + (Some(_), true) => { + w.line(format!("ret, err := {call}")); + w.block("if err != nil {", "}", |w| { + w.line(failed); + w.line("return"); + }); + } + } + let ty = m.ret.as_ref().expect("a returning method"); + match pass { + RetPass::Direct => { + w.line(format!( + "return {}", + to_c_direct("ret", &m.abi_ret, ctx.prefix) + )); + } + RetPass::Object { .. } => { + w.line("return ret.share()"); + } + RetPass::String | RetPass::Bytes | RetPass::Buffer => { + let out_ptr = slot_named("out_ptr").expect("a run return has out_ptr"); + let out_len = slot_named("out_len").expect("a run return has out_len"); + let hand = match pass { + RetPass::String => format!("wvHandOverString(ret, {out_ptr}, {out_len})"), + RetPass::Bytes => format!("wvHandOverBytes(ret, {out_ptr}, {out_len})"), + _ => format!( + "wvHandOverBytes(wvEncode(ret, {}), {out_ptr}, {out_len})", + write_fn(ty) + ), + }; + w.line(hand); + if direct_ret { + w.line("return"); + } + } + RetPass::Void => unreachable!("a returning method"), + } + }, + ); + w.blank(); +} diff --git a/crates/weaveffi-cli/src/targets/go/calls.rs b/crates/weaveffi-cli/src/targets/go/calls.rs new file mode 100644 index 00000000..30d2b4a0 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/calls.rs @@ -0,0 +1,544 @@ +//! Call rendering: sync, async, and iterator wrappers and the argument and +//! result marshalling they share. +//! +//! Marshalling follows the shared plan: each parameter's passing contract +//! comes from [`ParamBinding::arg_pass`] and each result's receiving +//! contract from [`RetPass::of`], so this module only spells those +//! contracts in Go. + +use crate::codegen::CodeWriter; +use weaveffi_model::abi::CType; +use weaveffi_model::model::{AsyncBinding, CallShape, FnBinding, IteratorBinding, ParamBinding}; +use weaveffi_model::plan::{ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::ty::Ty; + +use crate::targets::go::codec::{read_fn, write_fn}; +use crate::targets::go::docs::{GoDoc, Kind}; +use crate::targets::go::entities::domain_mapper; +use crate::targets::go::names::{self, pascal}; +use crate::targets::go::types::{ + cgo_type, from_c_direct, go_type, go_zero, strip_const, to_c_direct, +}; +use crate::targets::go::Ctx; + +/// The wrapper type a method is declared on. +#[derive(Clone, Copy)] +pub(crate) struct Receiver<'a> { + /// The Go type name of the interface wrapper (`Store`). + pub(crate) ty: &'a str, +} + +// ── Errors ── + +/// How a wrapper reports a non-zero error slot: a throwing callable returns +/// `(T, error)` with the failure mapped onto its domain's code types; any +/// other callable panics with an `*Error`, since its failure is a bug. +struct ErrCtx { + throws: bool, + /// The domain's mapping helper (`wvKvError`), or `None` for the generic + /// `*Error` when no domain is in scope. + mapper: Option, +} + +impl ErrCtx { + fn of(ctx: &Ctx, f: &FnBinding) -> Self { + Self { + throws: f.error_strategy() == ErrorStrategy::Throws, + mapper: ctx.domain.map(domain_mapper), + } + } + + /// The Go expression converting the `wvFailure` value `failure` into an + /// `error`. + fn map(&self, failure: &str) -> String { + match &self.mapper { + Some(m) => format!("{m}({failure})"), + None => format!("{failure}.err()"), + } + } + + /// The return type (with its leading space) of a sync wrapper. + fn ret_sig(&self, ret: Option<&Ty>) -> String { + match (ret, self.throws) { + (Some(r), true) => format!(" ({}, error)", go_type(r)), + (Some(r), false) => format!(" {}", go_type(r)), + (None, true) => " error".into(), + (None, false) => String::new(), + } + } +} + +// ── Shared pieces ── + +/// The wrapper's signature line. +fn header(recv: Option, go_name: &str, params: &[String], ret_sig: &str) -> String { + let params = params.join(", "); + match recv { + Some(r) => format!("func (s *{}) {go_name}({params}){ret_sig} {{", r.ty), + None => format!("func {go_name}({params}){ret_sig} {{"), + } +} + +/// The wrapper's Go parameter list. +fn go_params(f: &FnBinding) -> Vec { + let is_async = f.is_async(); + let mut params = Vec::new(); + if is_async { + params.push("ctx context.Context".to_string()); + } + params.extend( + f.params + .iter() + .map(|p| format!("{} {}", names::param(&p.name, f), go_type(&p.ty))), + ); + params +} + +/// The receiver borrow every method opens with: `s.native()` panics when +/// the wrapper was already closed, and the deferred release keeps the +/// object alive (even across a concurrent `Close`) until the call returns. +fn emit_receiver(w: &mut CodeWriter, args: &mut Vec) { + w.line("cSelf := s.native()"); + w.line("defer s.ref.release()"); + args.push("cSelf".into()); +} + +/// Emit the staging statements for one parameter and push its C argument +/// expressions, per its [`ArgPass`]. +/// +/// Strings and bytes pass a borrowed view of the Go memory (cgo pins it +/// for the call); a buffered value is encoded first. An object parameter +/// borrows the wrapper's pointer for the call, like a receiver, and a nil +/// `Interface?` passes null. A callback implementation is kept in a handle +/// table, passed as the context, beside the interface's static vtable; a +/// nil `Cb?` passes a null vtable. +fn emit_param( + w: &mut CodeWriter, + ctx: &Ctx, + args: &mut Vec, + p: &ParamBinding, + f: &FnBinding, +) { + let name = names::param(&p.name, f); + let c = format!("c{}", pascal(&p.name)); + match p.arg_pass() { + ArgPass::Buffer { .. } => { + w.line(format!( + "{c}Ptr, {c}Len := wvBytes(wvEncode({name}, {}))", + write_fn(&p.ty) + )); + args.extend([format!("{c}Ptr"), format!("{c}Len")]); + } + ArgPass::String { .. } => { + w.line(format!("{c}Ptr, {c}Len := wvStr({name})")); + args.extend([format!("{c}Ptr"), format!("{c}Len")]); + } + ArgPass::Bytes { .. } => { + w.line(format!("{c}Ptr, {c}Len := wvBytes({name})")); + args.extend([format!("{c}Ptr"), format!("{c}Len")]); + } + ArgPass::Object { slot, nullable } => { + if nullable { + w.line(format!( + "var {c} {}", + cgo_type(&strip_const(&slot.ty), ctx.prefix) + )); + w.block(format!("if {name} != nil {{"), "}", |w| { + w.line(format!("{c} = {name}.native()")); + w.line(format!("defer {name}.ref.release()")); + }); + } else { + w.line(format!("{c} := {name}.native()")); + w.line(format!("defer {name}.ref.release()")); + } + args.push(c); + } + ArgPass::Callback { + vtable, nullable, .. + } => { + let CType::Ptr { pointee, .. } = &vtable.ty else { + unreachable!("a callback vtable slot is a pointer") + }; + let accessor = format!("C.{}()", vtable_accessor(&pointee.render_c(ctx.prefix))); + if nullable { + w.line(format!("var {c}Ctx unsafe.Pointer")); + w.line(format!( + "var {c}Vtable {}", + cgo_type(&strip_const(&vtable.ty), ctx.prefix) + )); + w.block(format!("if {name} != nil {{"), "}", |w| { + w.line(format!( + "{c}Ctx, {c}Vtable = wvNewCallback({name}), {accessor}" + )); + }); + args.extend([format!("{c}Ctx"), format!("{c}Vtable")]); + } else { + args.extend([format!("wvNewCallback({name})"), accessor]); + } + } + ArgPass::Direct { slot } => args.push(to_c_direct(&name, &slot.ty, ctx.prefix)), + } +} + +/// The Go expression receiving one value the native library hands over, +/// per its [`RetPass`]: a direct value converted, a string or bytes copied +/// and released, a buffer decoded and released, an object adopted (a null +/// `Interface?` adopts to nil). `len` names the length slot of a run. +fn receive(ty: &Ty, pass: &RetPass, ptr: &str, len: &str) -> String { + match pass { + RetPass::Void => unreachable!("a present value is never void"), + RetPass::Direct => from_c_direct(ptr, ty), + RetPass::String => format!("wvTakeString({ptr}, {len})"), + RetPass::Bytes => format!("wvTakeBytes({ptr}, {len})"), + RetPass::Buffer => format!("wvDecode({ptr}, {len}, {})", read_fn(ty)), + RetPass::Object { .. } => { + let n = ty + .interface_name() + .expect("an object result names an interface"); + format!("wvAdopt{}({ptr})", pascal(n)) + } + } +} + +/// `true` when a value received per `pass` arrives as a `(ptr, len)` run. +fn has_len(pass: &RetPass) -> bool { + matches!(pass, RetPass::String | RetPass::Bytes | RetPass::Buffer) +} + +/// The doc comment of a wrapper: its IDL doc and documented parameters, +/// then `extra` paragraphs, then `Deprecated:`. +fn wrapper_doc(f: &FnBinding, go_name: &str, extra: &[&str]) -> GoDoc { + let kind = Kind::callable(f.ret.is_some()); + let mut doc = + GoDoc::new(go_name, f.doc.as_deref(), kind, None).params(&f.params, |n| names::param(n, f)); + for p in extra { + doc = doc.para(p); + } + doc.deprecated(f.deprecated.as_deref()) +} + +// ── Sync ── + +/// Render a sync or iterator callable: the Go wrapper marshalling the +/// parameters, calling the C symbol, checking the error slot (a typed +/// `(T, error)` when throwing, a panic otherwise), and receiving the +/// result. With `recv` set, it's a method passing the wrapper's pointer +/// first. +pub(crate) fn render_sync( + w: &mut CodeWriter, + ctx: &Ctx, + f: &FnBinding, + go_name: &str, + recv: Option, +) { + if let CallShape::Iterator(ib) = &f.shape { + render_iterator(w, ctx, f, ib, go_name, recv); + return; + } + let err = ErrCtx::of(ctx, f); + let pass = RetPass::of(f.ret.as_ref()); + wrapper_doc(f, go_name, &[]).emit(w); + w.block( + header(recv, go_name, &go_params(f), &err.ret_sig(f.ret.as_ref())), + "}", + |w| { + let mut args = Vec::new(); + if recv.is_some() { + emit_receiver(w, &mut args); + } + for p in &f.params { + emit_param(w, ctx, &mut args, p, f); + } + if has_len(&pass) { + w.line("var cRetLen C.size_t"); + args.push("&cRetLen".into()); + } + w.line(format!("var cErr C.{}_error", ctx.prefix)); + args.push("&cErr".into()); + let call = format!("C.{}({})", f.c_base, args.join(", ")); + match &f.ret { + Some(_) => w.line(format!("cRet := {call}")), + None => w.line(call), + }; + let ok = if err.throws { ", nil" } else { "" }; + if err.throws { + let mapped = err.map("wvTakeError(&cErr)"); + w.block("if cErr.code != 0 {", "}", |w| { + match &f.ret { + Some(ty) => w.line(format!("return {}, {mapped}", go_zero(ty))), + None => w.line(format!("return {mapped}")), + }; + }); + } else { + w.line("wvTrap(&cErr)"); + } + match &f.ret { + Some(ty) => { + w.line(format!( + "return {}{ok}", + receive(ty, &pass, "cRet", "cRetLen") + )); + } + None if err.throws => { + w.line("return nil"); + } + None => {} + } + }, + ); + w.blank(); +} + +/// Render an `iter` callable as Go's lazy sequence idiom: `iter.Seq[T]`, +/// or `iter.Seq2[T, error]` when it throws, where a launch or `next` +/// failure is yielded as a final `(zero, err)` pair. The native iterator is +/// launched on each range, `next` runs once per element, and `destroy` runs +/// exactly once when the range ends, early or not. +fn render_iterator( + w: &mut CodeWriter, + ctx: &Ctx, + f: &FnBinding, + ib: &IteratorBinding, + go_name: &str, + recv: Option, +) { + let err = ErrCtx::of(ctx, f); + let proto = ib.protocol(f); + let elem = &ib.elem; + let elem_go = go_type(elem); + let CType::Ptr { pointee, .. } = &ib.next.params[1].ty else { + unreachable!("an iterator's item slot is a pointer") + }; + let item_ty = cgo_type(&strip_const(pointee), ctx.prefix); + let with_len = has_len(&proto.elem); + let (seq, yield_ty) = if err.throws { + ( + format!("iter.Seq2[{elem_go}, error]"), + format!("func({elem_go}, error) bool"), + ) + } else { + ( + format!("iter.Seq[{elem_go}]"), + format!("func({elem_go}) bool"), + ) + }; + + let notes = if err.throws { + "The sequence launches the native iterator each time it's ranged over and \ + pulls one element per step, yielding a failure as a final (zero value, \ + error) pair; the iterator is released when the range ends, early or not." + } else { + "The sequence launches the native iterator each time it's ranged over and \ + pulls one element per step; the iterator is released when the range ends, \ + early or not." + }; + wrapper_doc(f, go_name, &[notes]).emit(w); + + // Surfaces a non-zero error slot: a throwing sequence yields the mapped + // error and stops; any other traps. + let check = |w: &mut CodeWriter, slot: &str| { + if err.throws { + let mapped = err.map(&format!("wvTakeError(&{slot})")); + w.block(format!("if {slot}.code != 0 {{"), "}", |w| { + w.line(format!("yield({}, {mapped})", go_zero(elem))); + w.line("return"); + }); + } else { + w.line(format!("wvTrap(&{slot})")); + } + }; + + w.block( + header(recv, go_name, &go_params(f), &format!(" {seq}")), + "}", + |w| { + w.block(format!("return func(yield {yield_ty}) {{"), "}", |w| { + let mut args = Vec::new(); + if recv.is_some() { + emit_receiver(w, &mut args); + } + for p in &f.params { + emit_param(w, ctx, &mut args, p, f); + } + w.line(format!("var cErr C.{}_error", ctx.prefix)); + args.push("&cErr".into()); + w.line(format!( + "cIter := C.{}({})", + ib.launch.symbol, + args.join(", ") + )); + check(w, "cErr"); + w.line(format!("defer C.{}(cIter)", ib.destroy_symbol)); + w.block("for {", "}", |w| { + w.line(format!("var cItem {item_ty}")); + let next_args = if with_len { + w.line("var cItemLen C.size_t"); + "cIter, &cItem, &cItemLen, &cIterErr" + } else { + "cIter, &cItem, &cIterErr" + }; + w.line(format!("var cIterErr C.{}_error", ctx.prefix)); + w.line(format!("more := C.{}({next_args}) != 0", ib.next.symbol)); + check(w, "cIterErr"); + w.block("if !more {", "}", |w| { + w.line("return"); + }); + let item = receive(elem, &proto.elem, "cItem", "cItemLen"); + let call = if err.throws { + format!("if !yield({item}, nil) {{") + } else { + format!("if !yield({item}) {{") + }; + w.block(call, "}", |w| { + w.line("return"); + }); + }); + }); + }, + ); + w.blank(); +} + +// ── Async ── + +/// The C name of the exported Go trampoline behind an async completion +/// typedef. +pub(crate) fn completion_trampoline(callback_type: &str) -> String { + format!("goWv_{callback_type}") +} + +/// The C identifier of the static preamble function returning the address +/// of a callback interface's one vtable (Go can't take the address of a +/// `static` C variable through cgo, so wrappers call this instead). +pub(crate) fn vtable_accessor(vtable_tag: &str) -> String { + format!("wvVtablePtr_{vtable_tag}") +} + +/// Render an async callable: the exported completion trampoline, which +/// converts the result and delivers it on the call's channel, and a +/// blocking wrapper taking a leading `context.Context`. +/// +/// Every async wrapper returns an `error`. A cancellable one creates a +/// native cancel token, cancels it when `ctx` is done, and returns +/// `ctx.Err()` for the cancelled completion; any other returns `ctx.Err()` +/// as soon as `ctx` is done and abandons the result. A failure maps onto +/// the domain when the function throws and panics with an `*Error` +/// otherwise. +pub(crate) fn render_async( + w: &mut CodeWriter, + ctx: &Ctx, + f: &FnBinding, + ab: &AsyncBinding, + go_name: &str, + recv: Option, +) { + let err = ErrCtx::of(ctx, f); + let proto = ab.protocol(f); + let tramp = completion_trampoline(&ab.callback_type); + let val_ty = f + .ret + .as_ref() + .map_or_else(|| "struct{}".to_string(), go_type); + + let formals: Vec = ab + .callback_params + .iter() + .map(|s| format!("{} {}", names::slot(&s.name), cgo_type(&s.ty, ctx.prefix))) + .collect(); + let context = names::slot(&ab.callback_params[0].name); + let err_slot = names::slot(&ab.callback_params[1].name); + let results: Vec = ab.callback_params[2..] + .iter() + .map(|s| names::slot(&s.name)) + .collect(); + w.line(format!("//export {tramp}")); + w.block( + format!("func {tramp}({}) {{", formals.join(", ")), + "}", + |w| { + let value = match &f.ret { + None => "struct{}{}".to_string(), + Some(ty) => { + let len = results.get(1).map_or("", String::as_str); + receive(ty, &proto.result, &results[0], len) + } + }; + w.block( + format!("wvComplete({context}, {err_slot}, func() {val_ty} {{"), + "})", + |w| { + w.line(format!("return {value}")); + }, + ); + }, + ); + w.blank(); + + let notes = if f.cancellable { + "It blocks until the call completes. Cancelling ctx cancels the native \ + call, which then returns ctx.Err()." + } else { + "It blocks until the call completes or ctx is done; in the latter case it \ + returns ctx.Err() at once and the native result is discarded." + }; + let kind = Kind::callable(f.ret.is_some()); + let mut doc = + GoDoc::new(go_name, f.doc.as_deref(), kind, None).params(&f.params, |n| names::param(n, f)); + if doc.is_empty() { + doc = doc.para("Calls the native async function."); + } + doc = doc.para(notes); + doc.deprecated(f.deprecated.as_deref()).emit(w); + + let (ret_sig, zero) = match &f.ret { + Some(r) => ( + format!(" ({}, error)", go_type(r)), + format!("{}, ", go_zero(r)), + ), + None => (" error".to_string(), String::new()), + }; + w.block(header(recv, go_name, &go_params(f), &ret_sig), "}", |w| { + w.block("if err := ctx.Err(); err != nil {", "}", |w| { + w.line(format!("return {zero}err")); + }); + let mut args = Vec::new(); + if recv.is_some() { + emit_receiver(w, &mut args); + } + for p in &f.params { + emit_param(w, ctx, &mut args, p, f); + } + let token = if proto.cancellable { + w.line(format!("token := C.{}_cancel_token_create()", ctx.prefix)); + w.line(format!( + "defer C.{}_cancel_token_destroy(token)", + ctx.prefix + )); + args.push("token".into()); + "token" + } else { + "nil" + }; + w.line(format!("call := wvNewAsyncCall[{val_ty}]()")); + args.push(format!("C.{}(unsafe.Pointer(C.{tramp}))", ab.callback_type)); + args.push("call.context()".into()); + w.line(format!("C.{}({})", ab.launch.symbol, args.join(", "))); + let res = if f.ret.is_some() { "res" } else { "_" }; + w.line(format!("{res}, fail, err := call.wait(ctx, {token})")); + w.block("if err != nil {", "}", |w| { + w.line(format!("return {zero}err")); + }); + w.block("if fail != nil {", "}", |w| { + if err.throws { + w.line(format!("return {zero}{}", err.map("*fail"))); + } else { + w.line("panic(fail.err())"); + } + }); + match &f.ret { + Some(_) => w.line("return res, nil"), + None => w.line("return nil"), + }; + }); + w.blank(); +} diff --git a/crates/weaveffi-cli/src/targets/go/codec.rs b/crates/weaveffi-cli/src/targets/go/codec.rs new file mode 100644 index 00000000..715b2b1e --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/codec.rs @@ -0,0 +1,228 @@ +//! Value-buffer codecs: one `wvWrite{T}`/`wvRead{T}` pair per type that +//! crosses inside a buffer, against the `wvWriter`/`wvReader` pair and the +//! generic list, map, and optional helpers in `runtime/codec.go`. +//! +//! Every composite type the API uses (`[string]`, `{string:Store}`, +//! `Entry?`) gets exactly one named pair, built from the pairs of its parts, +//! so a call site, a record field, and a callback argument of the same type +//! share one encoder and one decoder. Primitives use the writer and reader +//! methods directly (`(*wvWriter).writeString`); records, rich enums, +//! C-style enums, and interfaces get their pairs from the entity renderers. + +use std::collections::{BTreeMap, BTreeSet}; + +use crate::codegen::CodeWriter; +use weaveffi_model::model::Model; +use weaveffi_model::ty::{Family, Ty, WireType}; + +use crate::targets::go::names::pascal; +use crate::targets::go::types::{go_type, optional_derefs}; + +/// Write a Go function whose body is the one statement `body`, followed by +/// a blank line. +pub(crate) fn func(w: &mut CodeWriter, signature: &str, body: &str) { + w.block(format!("{signature} {{"), "}", |w| { + w.line(body); + }); + w.blank(); +} + +/// The stem naming a type's codec pair: `String`, `Entry`, `ListString`, +/// `MapStringStore`, `OptEntry`. Prefix notation with fixed arities, so +/// distinct types never share a stem. +fn stem(ty: &Ty) -> String { + match ty { + Ty::Prim(p) => p.pascal().to_string(), + Ty::Optional(inner) => format!("Opt{}", stem(inner)), + Ty::List(inner) => format!("List{}", stem(inner)), + Ty::Map(k, v) => format!("Map{}{}", stem(k), stem(v)), + Ty::Iterator(inner) => format!("Iter{}", stem(inner)), + Ty::Record(n) + | Ty::RichEnum(n) + | Ty::Enum(n) + | Ty::Interface(n) + | Ty::CallbackInterface(n) => pascal(n), + } +} + +/// The encoder for `ty` as a Go function value of type +/// `func(*wvWriter, T)`. +pub(crate) fn write_fn(ty: &Ty) -> String { + match ty { + Ty::Prim(p) => format!("(*wvWriter).write{}", p.pascal()), + _ => format!("wvWrite{}", stem(ty)), + } +} + +/// The decoder for `ty` as a Go function value of type +/// `func(*wvReader) T`. +pub(crate) fn read_fn(ty: &Ty) -> String { + match ty { + Ty::Prim(p) => format!("(*wvReader).read{}", p.pascal()), + _ => format!("wvRead{}", stem(ty)), + } +} + +/// The statement writing `expr` (of type `ty`) to the writer `w`. +pub(crate) fn write_stmt(w: &str, expr: &str, ty: &Ty) -> String { + match ty { + Ty::Prim(p) => format!("{w}.write{}({expr})", p.pascal()), + _ => format!("wvWrite{}({w}, {expr})", stem(ty)), + } +} + +/// The expression reading one value of type `ty` from the reader `r`. +pub(crate) fn read_expr(r: &str, ty: &Ty) -> String { + match ty { + Ty::Prim(p) => format!("{r}.read{}()", p.pascal()), + _ => format!("wvRead{}({r})", stem(ty)), + } +} + +/// Every type the API carries inside value buffers, by kind of codec it +/// needs. +#[derive(Default)] +pub(crate) struct BufferTypes { + /// Optionals, lists, and maps, by stem. + composites: BTreeMap, + /// C-style enums that appear inside a buffer. + pub(crate) enums: BTreeSet, + /// Interfaces that appear inside a buffer (as object tokens). + pub(crate) interfaces: BTreeSet, +} + +impl BufferTypes { + /// Walk every position of `model` where a value crosses in a buffer. + pub(crate) fn of(model: &Model) -> Self { + let mut t = Self::default(); + for m in &model.modules { + for s in &m.structs { + for f in &s.fields { + t.visit(&f.ty); + } + } + for e in &m.enums { + for v in &e.variants { + for f in &v.fields { + t.visit(&f.ty); + } + } + } + if let Some(e) = &m.errors { + for c in &e.codes { + for f in &c.fields { + t.visit(&f.ty); + } + } + } + let callables = m.callables().map(|f| (&f.params, f.ret.as_ref())); + let methods = m + .callback_interfaces + .iter() + .flat_map(|cb| &cb.methods) + .map(|f| (&f.params, f.ret.as_ref())); + for (params, ret) in callables.chain(methods) { + for p in params { + t.visit_top(&p.ty); + } + if let Some(ret) = ret { + t.visit_top(ret.iterator_elem().unwrap_or(ret)); + } + } + } + t + } + + /// A parameter, return, or element: only a buffer-family value crosses + /// in a buffer. + fn visit_top(&mut self, ty: &Ty) { + if ty.family() == Family::Buffer { + self.visit(ty); + } + } + + fn visit(&mut self, ty: &Ty) { + match ty.wire() { + WireType::Prim(_) | WireType::User(_) => {} + WireType::Enum(n) => { + self.enums.insert(n.to_string()); + } + WireType::Object(n) => { + self.interfaces.insert(n.to_string()); + } + WireType::Optional(inner) | WireType::List(inner) => { + self.composites.insert(stem(ty), ty.clone()); + self.visit(inner); + } + WireType::Map(k, v) => { + self.composites.insert(stem(ty), ty.clone()); + self.visit(k); + self.visit(v); + } + } + } + + /// Render the composite pairs, sorted by stem. + pub(crate) fn render_composites(&self, w: &mut CodeWriter) { + for (stem, ty) in &self.composites { + let go = go_type(ty); + let (write, read) = match ty { + Ty::List(inner) => ( + format!("wvWriteList(w, v, {})", write_fn(inner)), + format!("wvReadList(r, {})", read_fn(inner)), + ), + Ty::Map(k, v) => ( + format!("wvWriteMap(w, v, {}, {})", write_fn(k), write_fn(v)), + format!("wvReadMap(r, {}, {})", read_fn(k), read_fn(v)), + ), + Ty::Optional(inner) if optional_derefs(inner) => ( + format!("wvWriteOptional(w, v, {})", write_fn(inner)), + format!("wvReadOptional(r, {})", read_fn(inner)), + ), + Ty::Optional(inner) => ( + format!("wvWriteNilable(w, v, v != nil, {})", write_fn(inner)), + format!("wvReadNilable(r, {})", read_fn(inner)), + ), + _ => unreachable!("only optionals, lists, and maps are composites"), + }; + func( + w, + &format!("func wvWrite{stem}(w *wvWriter, v {go})"), + &write, + ); + func( + w, + &format!("func wvRead{stem}(r *wvReader) {go}"), + &format!("return {read}"), + ); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use weaveffi_model::ty::Prim; + + #[test] + fn stems_are_prefix_notation() { + let s = Ty::Prim(Prim::String); + let store = Ty::Interface("Store".into()); + assert_eq!(stem(&Ty::List(Box::new(s.clone()))), "ListString"); + assert_eq!( + stem(&Ty::Map(Box::new(s.clone()), Box::new(store))), + "MapStringStore" + ); + assert_eq!( + stem(&Ty::List(Box::new(Ty::Optional(Box::new(Ty::Record( + "Entry".into() + )))))), + "ListOptEntry" + ); + assert_eq!(write_fn(&s), "(*wvWriter).writeString"); + assert_eq!( + read_expr("r", &Ty::Record("Entry".into())), + "wvReadEntry(r)" + ); + } +} diff --git a/crates/weaveffi-cli/src/targets/go/docs.rs b/crates/weaveffi-cli/src/targets/go/docs.rs new file mode 100644 index 00000000..4321b8d7 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/docs.rs @@ -0,0 +1,361 @@ +//! Godoc comments: a declaration's doc starts with its name ("Path returns +//! the path the store was opened with."), followed by any further +//! paragraphs and a final `Deprecated:` paragraph. + +use crate::codegen::CodeWriter; +use weaveffi_model::model::ParamBinding; + +/// The width generated comment text wraps at (IDL docs keep their own line +/// breaks). +const WRAP: usize = 76; + +/// What a doc comment describes, which decides how a summary that doesn't +/// start with the name is joined to it. +#[derive(Clone, Copy, PartialEq, Eq)] +pub(crate) enum Kind { + /// A function or method with a result: "The path ..." reads "Path + /// returns the path ...". + Func, + /// A function or method without one: only a third-person verb or + /// "Whether" joins the name directly. + Proc, + /// A type or field: "The store's clock" reads "Clock is the store's + /// clock". + Value, + /// A variant, constant, or error code, whose IDL doc is often a clause + /// ("An entry was stored."): only a third-person verb joins the name + /// directly. + Clause, +} + +impl Kind { + /// [`Kind::Func`] for a callable with a result, else [`Kind::Proc`]. + pub(crate) fn callable(returns: bool) -> Self { + if returns { + Kind::Func + } else { + Kind::Proc + } + } +} + +/// A godoc comment under construction: the summary paragraph naming the +/// declaration, then further paragraphs, then `Deprecated:`. +pub(crate) struct GoDoc { + name: String, + paragraphs: Vec>, + deprecated: Option, +} + +impl GoDoc { + /// A doc for `name` whose summary is the IDL doc `doc`, joined to the + /// name per `kind`, or `fallback` (which must start with the name) + /// when there's no IDL doc. + pub(crate) fn new(name: &str, doc: Option<&str>, kind: Kind, fallback: Option) -> Self { + let mut paragraphs = Vec::new(); + match doc.map(str::trim).filter(|d| !d.is_empty()) { + Some(d) => { + // Blank lines separate paragraphs. + let mut current = Vec::new(); + for (i, line) in d.lines().enumerate() { + let line = line.trim_end(); + if line.is_empty() { + if !current.is_empty() { + paragraphs.push(std::mem::take(&mut current)); + } + } else if i == 0 { + current.push(lead(name, line, kind)); + } else { + current.push(line.to_string()); + } + } + if !current.is_empty() { + paragraphs.push(current); + } + } + None => paragraphs.extend(fallback.map(|f| wrap(&f))), + } + Self { + name: name.to_string(), + paragraphs, + deprecated: None, + } + } + + /// Append a paragraph of generated text, rewrapped. When it's the + /// first, it's joined to the name ("Wait blocks until ..."). + pub(crate) fn para(mut self, text: &str) -> Self { + let text = if self.paragraphs.is_empty() { + format!("{} {}", self.name, lower_first(text)) + } else { + text.to_string() + }; + self.paragraphs.push(wrap(&text)); + self + } + + /// Append a "Parameters:" list for the documented parameters, labelled + /// with their Go spellings (`name`). + pub(crate) fn params(mut self, params: &[ParamBinding], name: impl Fn(&str) -> String) -> Self { + let mut list = Vec::new(); + for p in params { + let Some(doc) = p.doc.as_deref().map(str::trim).filter(|d| !d.is_empty()) else { + continue; + }; + let mut lines = doc.lines(); + let first = lines.next().unwrap_or_default(); + list.push(format!(" - {}: {first}", name(&p.name))); + list.extend(lines.map(|l| { + if l.is_empty() { + String::new() + } else { + format!(" {l}") + } + })); + } + if !list.is_empty() { + if self.paragraphs.is_empty() { + self.paragraphs + .push(vec![format!("{} takes these parameters:", self.name)]); + } else { + self.paragraphs.push(vec!["Parameters:".into()]); + } + self.paragraphs.push(list); + } + self + } + + /// Set the `Deprecated:` paragraph. + pub(crate) fn deprecated(mut self, msg: Option<&str>) -> Self { + self.deprecated = msg.map(str::to_string); + self + } + + /// `true` when the comment has no paragraph yet. + pub(crate) fn is_empty(&self) -> bool { + self.paragraphs.is_empty() + } + + /// Write the comment at the writer's indentation. + pub(crate) fn emit(self, w: &mut CodeWriter) { + let mut paragraphs = self.paragraphs; + if let Some(msg) = self.deprecated { + paragraphs.push(vec![format!("Deprecated: {msg}")]); + } + for (i, p) in paragraphs.iter().enumerate() { + if i > 0 { + w.line("//"); + } + for line in p { + if line.is_empty() { + w.line("//"); + } else { + w.line(format!("// {line}")); + } + } + } + } +} + +/// The first line of an IDL summary, made to start with `name`: kept when +/// it already does; a third-person verb ("Returns ...") follows the name in +/// lowercase; "Whether ..." becomes "reports whether ..." for a callable or +/// field; "The ...", "A ...", or a wh-word becomes "returns ..." for a +/// callable with a result and "is ..." for a type or field; anything else +/// follows the name after a colon. +fn lead(name: &str, first: &str, kind: Kind) -> String { + let word = first.split_whitespace().next().unwrap_or_default(); + if word.trim_end_matches(['.', ',', ':', '\'']) == name { + return first.to_string(); + } + let lowered = lower_first(first); + let noun = matches!( + word, + "The" | "A" | "An" | "How" | "What" | "When" | "Where" | "Which" | "Who" | "Why" + ); + match kind { + _ if third_person(word) => format!("{name} {lowered}"), + Kind::Func | Kind::Proc | Kind::Value if word == "Whether" => { + format!("{name} reports {lowered}") + } + Kind::Func if noun => format!("{name} returns {lowered}"), + Kind::Value if noun => format!("{name} is {lowered}"), + _ => format!("{name}: {first}"), + } +} + +/// `true` for a capitalized third-person verb that opens doc summaries +/// ("Returns", "Creates", "Reports"). A word that can also be a plural +/// noun ("Stores", "Lists", "Statistics") isn't one. +fn third_person(word: &str) -> bool { + const VERBS: &[&str] = &[ + "Accepts", + "Adds", + "Advances", + "Allocates", + "Applies", + "Builds", + "Checks", + "Clears", + "Closes", + "Computes", + "Converts", + "Copies", + "Creates", + "Decodes", + "Deletes", + "Describes", + "Determines", + "Divides", + "Echoes", + "Emits", + "Encodes", + "Fetches", + "Finds", + "Fires", + "Formats", + "Frees", + "Gets", + "Gives", + "Greets", + "Identifies", + "Indicates", + "Installs", + "Loads", + "Makes", + "Merges", + "Notifies", + "Opens", + "Parses", + "Produces", + "Raises", + "Reads", + "Receives", + "Registers", + "Releases", + "Removes", + "Renders", + "Replaces", + "Reports", + "Represents", + "Resets", + "Resolves", + "Returns", + "Runs", + "Sends", + "Sets", + "Starts", + "Stops", + "Subscribes", + "Takes", + "Tells", + "Unsubscribes", + "Updates", + "Validates", + "Waits", + "Wraps", + "Writes", + "Yields", + ]; + VERBS.contains(&word) +} + +/// `s` with its first character lowercased, unless the first word is an +/// acronym or initialism (all capitals). +fn lower_first(s: &str) -> String { + let word = s.split_whitespace().next().unwrap_or_default(); + if word.len() > 1 && word.chars().all(|c| !c.is_lowercase()) { + return s.to_string(); + } + let mut chars = s.chars(); + match chars.next() { + Some(first) => first.to_lowercase().chain(chars).collect(), + None => String::new(), + } +} + +/// Generated text wrapped at [`WRAP`] columns. +fn wrap(text: &str) -> Vec { + let mut lines = Vec::new(); + let mut line = String::new(); + for word in text.split_whitespace() { + if !line.is_empty() && line.len() + 1 + word.len() > WRAP { + lines.push(std::mem::take(&mut line)); + } + if !line.is_empty() { + line.push(' '); + } + line.push_str(word); + } + if !line.is_empty() { + lines.push(line); + } + lines +} + +#[cfg(test)] +mod tests { + use super::*; + + fn render(doc: GoDoc) -> String { + let mut w = CodeWriter::tabs(); + doc.emit(&mut w); + w.finish() + } + + #[test] + fn summaries_start_with_the_name() { + assert_eq!( + lead("Add", "Add two numbers.", Kind::Func), + "Add two numbers." + ); + assert_eq!( + lead("Sum", "Returns the sum.", Kind::Func), + "Sum returns the sum." + ); + assert_eq!( + lead("Path", "The path it was opened with.", Kind::Func), + "Path returns the path it was opened with." + ); + assert_eq!( + lead("Entry", "A stored value.", Kind::Value), + "Entry is a stored value." + ); + assert_eq!( + lead("Reason", "Why it failed.", Kind::Value), + "Reason is why it failed." + ); + assert_eq!( + lead("Accepts", "Whether to accept.", Kind::Func), + "Accepts reports whether to accept." + ); + assert_eq!( + lead("OnChange", "A change it accepted.", Kind::Proc), + "OnChange: A change it accepted." + ); + assert_eq!( + lead("ChangePut", "An entry was stored.", Kind::Clause), + "ChangePut: An entry was stored." + ); + assert_eq!(lead("Open", "Open a store.", Kind::Func), "Open a store."); + assert_eq!( + lead("Connect", "Open a store.", Kind::Func), + "Connect: Open a store." + ); + } + + #[test] + fn paragraphs_wrap_and_deprecation_comes_last() { + let doc = GoDoc::new("Old", Some("Returns it.\n\nMore."), Kind::Func, None) + .para("Extra.") + .deprecated(Some("use New")); + assert_eq!( + render(doc), + "// Old returns it.\n//\n// More.\n//\n// Extra.\n//\n// Deprecated: use New\n" + ); + let bare = GoDoc::new("Wait", None, Kind::Func, None).para("Blocks until done."); + assert_eq!(render(bare), "// Wait blocks until done.\n"); + let long = "word ".repeat(30); + assert!(wrap(&long).iter().all(|l| l.len() <= WRAP)); + } +} diff --git a/crates/weaveffi-cli/src/targets/go/entities.rs b/crates/weaveffi-cli/src/targets/go/entities.rs new file mode 100644 index 00000000..341a4656 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/entities.rs @@ -0,0 +1,528 @@ +//! Entity rendering: C-style and rich enums, records, error domains, and +//! interface (object) wrappers, each with its value-buffer codec pair. + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{ + CallShape, EnumBinding, ErrorBinding, FieldBinding, InterfaceBinding, ModuleBinding, + StructBinding, +}; + +use crate::targets::go::calls::{render_async, render_sync, Receiver}; +use crate::targets::go::codec::{func, read_expr, write_stmt, BufferTypes}; +use crate::targets::go::docs::{GoDoc, Kind}; +use crate::targets::go::names::{self, constructor, domain_type, pascal}; +use crate::targets::go::types::{go_str, go_type}; +use crate::targets::go::Ctx; + +/// One `name rest` line inside a struct or const block, with its optional +/// doc comment, for [`emit_aligned`]. +struct AlignedEntry { + doc: Option, + name: String, + rest: String, +} + +/// Emit `entries` as the body of a struct or const block the way gofmt lays +/// it out: gofmt aligns the second column of consecutive lines and starts a +/// fresh column after a comment line, so each run of entries following a +/// documented one (or the start) is padded to the widest name in that run. +fn emit_aligned(w: &mut CodeWriter, entries: Vec, kind: Kind) { + let mut i = 0; + while i < entries.len() { + let mut j = i + 1; + while j < entries.len() && entries[j].doc.is_none() { + j += 1; + } + let width = entries[i..j] + .iter() + .map(|e| e.name.len()) + .max() + .unwrap_or(0); + for e in &entries[i..j] { + GoDoc::new(&e.name, e.doc.as_deref(), kind, None).emit(w); + w.line(format!("{: String) -> Vec { + fields + .iter() + .map(|f| AlignedEntry { + doc: f.doc.clone().filter(|d| !d.trim().is_empty()), + name: name(&f.name), + rest: go_type(&f.ty), + }) + .collect() +} + +/// Render a C-style enum: an `int32` type with one constant per variant, +/// plus its codec pair when it appears inside a buffer. +pub(crate) fn render_enum(w: &mut CodeWriter, e: &EnumBinding, codecs: &BufferTypes) { + let name = pascal(&e.name); + GoDoc::new(&name, e.doc.as_deref(), Kind::Value, None) + .deprecated(e.deprecated.as_deref()) + .emit(w); + w.line(format!("type {name} int32")); + w.blank(); + w.block("const (", ")", |w| { + let entries = e + .variants + .iter() + .map(|v| AlignedEntry { + doc: v.doc.clone().filter(|d| !d.trim().is_empty()), + name: format!("{name}{}", pascal(&v.name)), + rest: format!("{name} = {}", v.value), + }) + .collect(); + emit_aligned(w, entries, Kind::Clause); + }); + w.blank(); + if codecs.enums.contains(&e.name) { + func( + w, + &format!("func wvWrite{name}(w *wvWriter, v {name})"), + "w.writeI32(int32(v))", + ); + func( + w, + &format!("func wvRead{name}(r *wvReader) {name}"), + &format!("return {name}(r.readI32())"), + ); + } +} + +/// Render a rich enum as a sealed interface (`type Shape interface { +/// isShape() }`) with one struct per variant (`ShapeCircle`), plus its codec +/// pair: the `i32` tag, then the variant's fields in order. +pub(crate) fn render_rich_enum(w: &mut CodeWriter, package: &str, e: &EnumBinding) { + let name = pascal(&e.name); + GoDoc::new(&name, e.doc.as_deref(), Kind::Value, None) + .para(&format!( + "{name} is a sealed sum type: exactly one of its variant structs is the \ + value at a time." + )) + .deprecated(e.deprecated.as_deref()) + .emit(w); + w.block(format!("type {name} interface {{"), "}", |w| { + w.line(format!("is{name}()")); + }); + w.blank(); + + for v in &e.variants { + let vn = format!("{name}{}", pascal(&v.name)); + GoDoc::new( + &vn, + v.doc.as_deref(), + Kind::Clause, + Some(format!("{vn} is the {} variant of {name}.", v.name)), + ) + .emit(w); + if v.fields.is_empty() { + w.line(format!("type {vn} struct{{}}")); + } else { + w.block(format!("type {vn} struct {{"), "}", |w| { + emit_aligned(w, field_entries(&v.fields, names::field), Kind::Value); + }); + } + w.blank(); + w.line(format!("func ({vn}) is{name}() {{}}")); + w.blank(); + } + + let any_fields = e.variants.iter().any(|v| !v.fields.is_empty()); + let switch = if any_fields { + "switch x := v.(type) {" + } else { + "switch v.(type) {" + }; + w.block( + format!("func wvWrite{name}(w *wvWriter, v {name}) {{"), + "}", + |w| { + w.line(switch); + for v in &e.variants { + w.line(format!("case {name}{}:", pascal(&v.name))); + w.scope(|w| { + w.line(format!("w.writeI32({})", v.value)); + for f in &v.fields { + let expr = format!("x.{}", names::field(&f.name)); + w.line(write_stmt("w", &expr, &f.ty)); + } + }); + } + w.line("default:"); + w.scope(|w| { + w.line(format!( + "panic(\"{package}: {name} value is not one of its variants\")" + )); + }); + w.line("}"); + }, + ); + w.blank(); + w.block( + format!("func wvRead{name}(r *wvReader) {name} {{"), + "}", + |w| { + w.line("switch r.readI32() {"); + for v in &e.variants { + let vn = format!("{name}{}", pascal(&v.name)); + w.line(format!("case {}:", v.value)); + w.scope(|w| { + if v.fields.is_empty() { + w.line(format!("return {vn}{{}}")); + } else { + w.line(format!("var x {vn}")); + for f in &v.fields { + w.line(format!( + "x.{} = {}", + names::field(&f.name), + read_expr("r", &f.ty) + )); + } + w.line("return x"); + } + }); + } + w.line("}"); + w.line(format!("wvMalformed(\"{name} tag out of range\")")); + w.line("return nil"); + }, + ); + w.blank(); +} + +/// Render a record as a value struct with exported fields, plus its codec +/// pair: the fields in declaration order. +pub(crate) fn render_struct(w: &mut CodeWriter, s: &StructBinding) { + let name = pascal(&s.name); + GoDoc::new(&name, s.doc.as_deref(), Kind::Value, None) + .deprecated(s.deprecated.as_deref()) + .emit(w); + w.block(format!("type {name} struct {{"), "}", |w| { + emit_aligned(w, field_entries(&s.fields, names::field), Kind::Value); + }); + w.blank(); + w.block( + format!("func wvWrite{name}(w *wvWriter, v {name}) {{"), + "}", + |w| { + for f in &s.fields { + let expr = format!("v.{}", names::field(&f.name)); + w.line(write_stmt("w", &expr, &f.ty)); + } + }, + ); + w.blank(); + w.block( + format!("func wvRead{name}(r *wvReader) {name} {{"), + "}", + |w| { + w.line(format!("var v {name}")); + for f in &s.fields { + w.line(format!( + "v.{} = {}", + names::field(&f.name), + read_expr("r", &f.ty) + )); + } + w.line("return v"); + }, + ); + w.blank(); +} + +/// The name of the helper converting a failure in the domain `e` into its +/// error-code type (`wvKvError`). +pub(crate) fn domain_mapper(e: &ErrorBinding) -> String { + format!("wv{}", domain_type(e)) +} + +/// Render an error domain: the sealed `{Domain}` interface every code type +/// implements, one `*{Code}Error` struct per code (its payload fields, a +/// `Message`, `Error`, and `Code`), and the helper mapping a failure to the +/// code's type, with an *Error for any code outside the domain. +pub(crate) fn render_error(w: &mut CodeWriter, ctx: &Ctx, m: &ModuleBinding, e: &ErrorBinding) { + let gn = ctx.names; + let domain = domain_type(e); + let marker = format!("is{domain}"); + let codes: Vec<&str> = e.codes.iter().map(|c| gn.code(c)).collect(); + let list = match codes.as_slice() { + [one] => format!("*{one}"), + [init @ .., last] => format!( + "{}, or *{last}", + init.iter() + .map(|c| format!("*{c}")) + .collect::>() + .join(", ") + ), + [] => unreachable!("an error domain declares at least one code"), + }; + GoDoc::new( + &domain, + None, + Kind::Value, + Some(format!( + "{domain} is an error the {} module reports: {list}.", + m.dot_path + )), + ) + .para(&if ctx.throwing_callbacks { + format!( + "Match one code with errors.As and its type, or any of them with a \ + {domain}. A throwing callback method returns one to report that code." + ) + } else { + format!("Match one code with errors.As and its type, or any of them with a {domain}.") + }) + .emit(w); + w.block(format!("type {domain} interface {{"), "}", |w| { + w.line("error"); + w.line("// Code returns the error's numeric code."); + w.line("Code() int32"); + w.line(format!("{marker}()")); + }); + w.blank(); + + for c in &e.codes { + let ty = gn.code(c); + let doc = c + .doc + .as_deref() + .filter(|d| d.trim() != c.message.trim()) + .map(str::to_string); + GoDoc::new( + ty, + doc.as_deref(), + Kind::Clause, + Some(format!( + "{ty} is the {domain} code {} ({}): {}.", + c.name, + c.value, + c.message.trim_end_matches('.') + )), + ) + .emit(w); + w.block(format!("type {ty} struct {{"), "}", |w| { + emit_aligned(w, field_entries(&c.fields, names::error_field), Kind::Value); + w.line("// Message describes the failure; when it's empty, Error returns the"); + w.line(format!( + "// code's default message, {}.", + go_str(&c.message) + )); + w.line("Message string"); + }); + w.blank(); + w.line("// Error returns the error's message."); + w.block(format!("func (e *{ty}) Error() string {{"), "}", |w| { + w.block("if e.Message == \"\" {", "}", |w| { + w.line(format!("return {}", go_str(&c.message))); + }); + w.line("return e.Message"); + }); + w.blank(); + w.line(format!("// Code returns {}.", c.value)); + func( + w, + &format!("func (*{ty}) Code() int32"), + &format!("return {}", c.value), + ); + w.line(format!("func (*{ty}) {marker}() {{}}")); + w.blank(); + if c.fields.is_empty() { + w.block(format!("func (*{ty}) wvPayload() []byte {{"), "}", |w| { + w.line("return nil"); + }); + } else { + w.block(format!("func (e *{ty}) wvPayload() []byte {{"), "}", |w| { + w.line("w := &wvWriter{}"); + for f in &c.fields { + let expr = format!("e.{}", names::error_field(&f.name)); + w.line(write_stmt("w", &expr, &f.ty)); + } + w.line("return w.buf"); + }); + } + w.blank(); + } + + w.block( + format!("func {}(f wvFailure) error {{", domain_mapper(e)), + "}", + |w| { + w.line("switch f.code {"); + for c in &e.codes { + let ty = gn.code(c); + w.line(format!("case {}:", c.value)); + w.scope(|w| { + if c.fields.is_empty() { + w.line(format!("return &{ty}{{Message: f.message}}")); + return; + } + w.line(format!("e := &{ty}{{Message: f.message}}")); + w.block("if f.payload != nil {", "}", |w| { + w.line("r := &wvReader{buf: f.payload}"); + for f in &c.fields { + w.line(format!( + "e.{} = {}", + names::error_field(&f.name), + read_expr("r", &f.ty) + )); + } + w.line("r.expectEnd()"); + }); + w.line("return e"); + }); + } + w.line("}"); + w.line("return f.err()"); + }, + ); + w.blank(); +} + +/// Render an interface as a reference-counted object wrapper whose `wvRef` +/// owns one strong reference, released by `Close` (idempotent, and safe to +/// race with in-flight calls) or, as a backstop, by a finalizer; its codec +/// pair when it appears inside a buffer (an object token carrying a fresh +/// reference); and its members: constructors as factories (`OpenStore`), +/// methods on the wrapper, and statics prefixed with the type +/// (`StoreDefaultCapacity`). +pub(crate) fn render_interface(w: &mut CodeWriter, ctx: &Ctx, iface: &InterfaceBinding) { + let package = ctx.package; + let name = pascal(&iface.name); + let c_tag = &iface.c_tag; + GoDoc::new( + &name, + iface.doc.as_deref(), + Kind::Value, + Some(format!( + "{name} is a reference-counted object owned by the native library." + )), + ) + .para( + "Each wrapper holds one strong reference; Close releases it, and a \ + finalizer releases it if the wrapper is garbage collected first.", + ) + .deprecated(iface.deprecated.as_deref()) + .emit(w); + w.block(format!("type {name} struct {{"), "}", |w| { + w.line("ref wvRef"); + }); + w.blank(); + + w.block( + format!("func wvDestroy{name}(ptr unsafe.Pointer) {{"), + "}", + |w| { + w.line(format!("C.{}((*C.{c_tag})(ptr))", iface.destroy_symbol)); + }, + ); + w.blank(); + + w.line(format!( + "// wvAdopt{name} wraps one owned strong reference. A null pointer adopts to nil." + )); + w.block( + format!("func wvAdopt{name}(ptr *C.{c_tag}) *{name} {{"), + "}", + |w| { + w.block("if ptr == nil {", "}", |w| { + w.line("return nil"); + }); + w.line(format!("s := &{name}{{}}")); + w.line(format!("s.ref.init(unsafe.Pointer(ptr), wvDestroy{name})")); + w.line(format!("runtime.SetFinalizer(s, (*{name}).Close)")); + w.line("return s"); + }, + ); + w.blank(); + + w.line("// native borrows the wrapper's pointer for one call; pair it with a"); + w.line("// deferred s.ref.release()."); + w.block( + format!("func (s *{name}) native() *C.{c_tag} {{"), + "}", + |w| { + w.block("if s == nil {", "}", |w| { + w.line(format!("panic(\"{package}: nil *{name}\")")); + }); + w.line(format!("return (*C.{c_tag})(s.ref.acquire(\"{name}\"))")); + }, + ); + w.blank(); + + w.line("// share returns a new strong reference to the wrapper's object for the"); + w.line("// native library to adopt (the wrapper keeps its own), or null for nil."); + w.block( + format!("func (s *{name}) share() *C.{c_tag} {{"), + "}", + |w| { + w.block("if s == nil {", "}", |w| { + w.line("return nil"); + }); + w.line("ptr := s.native()"); + w.line("defer s.ref.release()"); + w.line(format!("return C.{}(ptr)", iface.clone_symbol)); + }, + ); + w.blank(); + + if ctx.codecs.interfaces.contains(&iface.name) { + func( + w, + &format!("func wvWrite{name}(w *wvWriter, v *{name})"), + "w.writeU64(uint64(uintptr(unsafe.Pointer(v.share()))))", + ); + w.block( + format!("func wvRead{name}(r *wvReader) *{name} {{"), + "}", + |w| { + w.line("token := r.readU64()"); + w.block("if token == 0 {", "}", |w| { + w.line("wvMalformed(\"null object token\")"); + }); + w.line(format!( + "return wvAdopt{name}((*C.{c_tag})(C.wvHandlePtr(C.uintptr_t(token))))" + )); + }, + ); + w.blank(); + } + + w.line("// Close releases the wrapper's strong reference and always returns nil."); + w.line("// It's idempotent and safe to call from any goroutine, even while a call"); + w.line("// on the wrapper is in flight: the reference is then released when that"); + w.line("// call returns. The object itself is dropped once its last reference, from"); + w.line("// any wrapper, record, or the native library, is gone."); + w.block(format!("func (s *{name}) Close() error {{"), "}", |w| { + w.line("runtime.SetFinalizer(s, nil)"); + w.line("s.ref.close()"); + w.line("return nil"); + }); + w.blank(); + + for c in &iface.constructors { + render_sync(w, ctx, c, &constructor(&name, c), None); + } + for f in &iface.methods { + let go_name = names::method(&f.name); + let recv = Some(Receiver { ty: &name }); + if let CallShape::Async(ab) = &f.shape { + render_async(w, ctx, f, ab, &go_name, recv); + } else { + render_sync(w, ctx, f, &go_name, recv); + } + } + for f in &iface.statics { + let go_name = format!("{name}{}", pascal(&f.name)); + if let CallShape::Async(ab) = &f.shape { + render_async(w, ctx, f, ab, &go_name, None); + } else { + render_sync(w, ctx, f, &go_name, None); + } + } +} diff --git a/crates/weaveffi-cli/src/targets/go/mod.rs b/crates/weaveffi-cli/src/targets/go/mod.rs new file mode 100644 index 00000000..0ca29bff --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/mod.rs @@ -0,0 +1,327 @@ +//! Go (cgo) binding generator for WeaveFFI. +//! +//! Emits a self-contained Go module under `go/`: `go.mod`, a copy of the C +//! header, `bindings.go` (the API), `runtime.go` (load-time checks, errors, +//! strings, objects, callbacks, and async plumbing), and `codec.go` (the +//! value-buffer writer and reader). The module path defaults to the package +//! name and the Go package name is the C prefix; the bindings link +//! `-l{library}` at build time. Implements [`LanguageBackend`]. +//! +//! Records are value structs, rich enums sealed interfaces, and optionals, +//! lists, and maps Go pointers, slices, and maps; all of them cross the C +//! ABI as one value buffer, with one codec pair per type. Interfaces are +//! reference-counted objects: each wrapper holds one strong reference +//! released by `Close` or, as a backstop, a finalizer. Callback interfaces +//! are Go interfaces the consumer implements, crossing as a handle-table +//! context plus the address of one static vtable per interface, filled with +//! exported trampolines. A throwing call returns `error` values of its +//! domain's code types; any other failure panics with an `*Error`. Async +//! functions block on a `context.Context`, and a cancellable one cancels +//! its native token when the context is done. Loading checks the ABI +//! revision and every top-level module's contract table. + +mod callbacks; +mod calls; +mod codec; +mod docs; +mod entities; +mod names; +mod package; +mod runtime; +#[cfg(test)] +mod tests; +mod types; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::codegen::CodeWriter; +use crate::lang; +use crate::package::{Artifact, PackageContext}; +use crate::targets::c::render_c_header_from_model; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; +use camino::Utf8Path; +use serde::{Deserialize, Serialize}; +use weaveffi_model::contract::entries; +use weaveffi_model::model::{contract_symbol, CallShape, ErrorBinding, Model, ModuleBinding}; + +use crate::targets::go::callbacks::{emit_preamble_decls, render_callback_interface}; +use crate::targets::go::calls::{render_async, render_sync}; +use crate::targets::go::codec::BufferTypes; +use crate::targets::go::entities::{ + render_enum, render_error, render_interface, render_rich_enum, render_struct, +}; +use crate::targets::go::names::GoNames; +use crate::targets::go::package::{package_files, render_go_mod, render_readme}; +use crate::targets::go::runtime::{render_codec, render_runtime, RuntimeNames}; +use crate::targets::go::types::go_str; + +/// Per-target configuration for [`GoGenerator`]. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct GoConfig { + /// Go module path written to `go.mod`. Defaults to the package name. + pub name: Option, +} + +/// Every identity-derived name the generated module uses. +pub(crate) struct Names { + /// The Go module path (`go.mod`): the configured `name`, else the + /// package name. + pub(crate) module_path: String, + /// The Go package name: the C prefix, escaped if it's a Go keyword. + pub(crate) package: String, + /// The native library base name the bindings link (`-l{library}`). + pub(crate) library: String, + /// The bundled C header's file name, `{library}.h`. + pub(crate) header: String, +} + +impl Names { + pub(crate) fn new(model: &Model, config: &GoConfig) -> Self { + let id = &model.identity; + Self { + module_path: config + .name + .as_deref() + .map(str::trim) + .filter(|p| !p.is_empty()) + .unwrap_or(&id.name) + .to_string(), + package: lang::escape_ident(&id.prefix, lang::GO_KEYWORDS), + library: id.library.clone(), + header: crate::targets::c::header_name(model), + } + } +} + +/// What every renderer needs beyond the declaration it renders. +#[derive(Clone, Copy)] +pub(crate) struct Ctx<'a> { + /// The C symbol prefix. + pub(crate) prefix: &'a str, + /// The Go package name. + pub(crate) package: &'a str, + /// The Go name of every declaration. + pub(crate) names: &'a GoNames, + /// The types that cross inside value buffers. + pub(crate) codecs: &'a BufferTypes, + /// The error domain in scope for the module being rendered. + pub(crate) domain: Option<&'a ErrorBinding>, + /// Whether any callback method is declared `throws` (so consumers + /// construct domain errors themselves). + pub(crate) throwing_callbacks: bool, +} + +/// Go backend: emits a cgo module binding the C ABI exposed by the +/// underlying cdylib. +pub struct GoGenerator; + +impl LanguageBackend for GoGenerator { + type Config = GoConfig; + + fn name(&self) -> &'static str { + "go" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let dir = out_dir.join("go"); + render_files(model, config) + .into_iter() + .map(|(name, contents)| OutputFile::new(dir.join(name), contents)) + .collect() + } + + /// The module directory at `go/{package}/` with each desktop library + /// under `lib//` and a cgo preamble that links the right one. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + Some(package_files(model, ctx, config)) + } +} + +/// Every file of the generated module, as `(file name, contents)` pairs. +pub(crate) fn render_files(model: &Model, config: &GoConfig) -> Vec<(String, String)> { + let names = Names::new(model, config); + let runtime_names = RuntimeNames { + package: &names.package, + prefix: model.prefix(), + header: &names.header, + }; + vec![ + ("go.mod".to_string(), render_go_mod(&names.module_path)), + ("README.md".to_string(), render_readme(&names)), + ( + names.header.clone(), + render_c_header_from_model(model, &names.header), + ), + ("bindings.go".to_string(), render_bindings(model, &names)), + ("runtime.go".to_string(), render_runtime(&runtime_names)), + ("codec.go".to_string(), render_codec(&runtime_names)), + ] +} + +/// Render `bindings.go`: the cgo preamble, the imports, the load-time +/// checks, every module's declarations, and the composite codecs. +pub(crate) fn render_bindings(model: &Model, names: &Names) -> String { + let go_names = GoNames::new(model); + let codecs = BufferTypes::of(model); + let base = Ctx { + prefix: model.prefix(), + package: &names.package, + names: &go_names, + codecs: &codecs, + domain: None, + throwing_callbacks: model + .callback_interfaces() + .any(|(_, cb)| cb.methods.iter().any(|m| m.throws)), + }; + + let mut w = CodeWriter::tabs(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + w.line(format!( + "// Package {} binds the {} native library.", + names.package, names.library + )); + w.line(format!("package {}", names.package)); + w.blank(); + render_preamble(&mut w, model, names, &codecs); + render_imports(&mut w, model); + render_init(&mut w, model); + for m in &model.modules { + let ctx = Ctx { + domain: model.error_domain(m), + ..base + }; + render_module(&mut w, &ctx, m); + } + codecs.render_composites(&mut w); + + // Exactly one blank line before the trailer keeps the file gofmt-clean. + let mut out = w.finish(); + out.truncate(out.trim_end().len()); + out.push_str("\n\n"); + out.push_str(&render_trailer(CommentStyle::DoubleSlash, "bindings.go")); + out +} + +/// Write the cgo preamble: the link flag, the header, and the declarations +/// of the exported trampolines and static vtables. +fn render_preamble(w: &mut CodeWriter, model: &Model, names: &Names, codecs: &BufferTypes) { + w.line("/*"); + w.line(format!("#cgo LDFLAGS: -l{}", names.library)); + if model.callables().any(|(_, f)| f.deprecated.is_some()) { + // The bindings call deprecated functions on the user's behalf; only + // the user's own calls should warn, through Go's `Deprecated:` docs. + w.line("#cgo CFLAGS: -Wno-deprecated-declarations"); + } + w.line(format!("#include \"{}\"", names.header)); + if !codecs.interfaces.is_empty() { + // Widening an object token to a pointer in C keeps `go vet` from + // flagging a "possible misuse of unsafe.Pointer". Each preamble + // helper is compiled into every cgo translation unit, some of which + // never call it. + w.line( + "__attribute__((unused)) static void* wvHandlePtr(uintptr_t h) { return (void*)h; }", + ); + } + emit_preamble_decls(w, model); + w.line("*/"); + w.line("import \"C\""); + w.blank(); +} + +/// Write the standard-library imports `bindings.go` uses; everything else +/// lives in `runtime.go` and `codec.go`, whose imports are fixed. +fn render_imports(w: &mut CodeWriter, model: &Model) { + let objects = model.has_interfaces(); + let callbacks = model.has_callback_interfaces(); + let has_async = model.has_async(); + let packages = [ + (has_async, "context"), + (model.has_iterators(), "iter"), + (objects, "runtime"), + (objects || callbacks || has_async, "unsafe"), + ]; + let used: Vec<&str> = packages + .iter() + .filter(|(on, _)| *on) + .map(|(_, p)| *p) + .collect(); + if used.is_empty() { + return; + } + w.block("import (", ")", |w| { + for p in used { + w.line(format!("\"{p}\"")); + } + }); + w.blank(); +} + +/// Write the load-time checks: the ABI revision, then every top-level +/// module's contract table against the entries these bindings were +/// generated with. +fn render_init(w: &mut CodeWriter, model: &Model) { + w.block("func init() {", "}", |w| { + w.line("wvCheckABI()"); + let mut first = true; + for root in model.roots() { + if first { + w.line("var n C.size_t"); + } + let assign = if first { ":=" } else { "=" }; + first = false; + w.line(format!( + "table {assign} C.{}(&n)", + contract_symbol(model.prefix(), &root.name) + )); + w.block("wvCheckContract(table, n, []wvContractEntry{", "})", |w| { + for e in entries(model, root) { + w.line(format!( + "{{{:#018x}, {:#018x}, {}}},", + e.id, + e.hash, + go_str(&e.path) + )); + } + }); + } + }); + w.blank(); +} + +/// Render every declaration of `m`: its error domain (in the declaring +/// module; submodules use the ancestor's), enums, records, callback +/// interfaces, interfaces, and functions. +fn render_module(w: &mut CodeWriter, ctx: &Ctx, m: &ModuleBinding) { + if let Some(e) = &m.errors { + render_error(w, ctx, m, e); + } + for e in &m.enums { + if e.is_rich() { + render_rich_enum(w, ctx.package, e); + } else { + render_enum(w, e, ctx.codecs); + } + } + for s in &m.structs { + render_struct(w, s); + } + for cb in &m.callback_interfaces { + render_callback_interface(w, ctx, cb); + } + for i in &m.interfaces { + render_interface(w, ctx, i); + } + for f in &m.functions { + let go_name = ctx.names.function(f); + if let CallShape::Async(ab) = &f.shape { + render_async(w, ctx, f, ab, go_name, None); + } else { + render_sync(w, ctx, f, go_name, None); + } + } +} diff --git a/crates/weaveffi-cli/src/targets/go/names.rs b/crates/weaveffi-cli/src/targets/go/names.rs new file mode 100644 index 00000000..00431b45 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/names.rs @@ -0,0 +1,324 @@ +//! Go identifiers: case conversion with Go's initialisms, keyword escaping, +//! and the names the flat package gives every declaration. +//! +//! The whole module tree renders into one Go package, so every exported +//! name is computed here once ([`GoNames`]) and looked up by the renderers: +//! types keep their IDL name (`Store`), constructors become factories +//! (`OpenStore`), statics are prefixed with their type (`StoreOpenMany`), +//! error codes become `*{Code}Error` types, and a free function whose name +//! would clash with any of those keeps its module path (`KvOpenStore`). + +use heck::ToSnakeCase; +use std::collections::{BTreeMap, BTreeSet}; +use weaveffi_model::model::{ErrorBinding, ErrorCodeBinding, FnBinding, Model}; + +use crate::lang; + +/// The initialisms Go spells in one case (`UserID`, `ParseURL`, +/// `HTTPServer`), from the standard Go style guide's list. +const INITIALISMS: &[&str] = &[ + "ACL", "API", "ASCII", "CPU", "CSS", "DNS", "EOF", "GUID", "HTML", "HTTP", "HTTPS", "ID", "IP", + "JSON", "LHS", "QPS", "RAM", "RHS", "RPC", "SLA", "SMTP", "SQL", "SSH", "TCP", "TLS", "TTL", + "UDP", "UI", "UID", "URI", "URL", "UTF8", "UUID", "VM", "XML", "XMPP", "XSRF", "XSS", +]; + +/// Names a wrapper or trampoline body uses besides its parameters: the +/// packages it calls into, the receiver, and its locals. A parameter or +/// slot spelled like one of them gains a trailing underscore. +const RESERVED: &[&str] = &[ + "C", "cErr", "cIter", "cItem", "cItemLen", "cIterErr", "cRet", "cRetLen", "cSelf", "call", + "context", "err", "fail", "iter", "more", "res", "ret", "runtime", "token", "unsafe", "yield", +]; + +/// The words of an identifier in any IDL spelling (`snake_case`, +/// `PascalCase`, `SCREAMING_CASE`), lowercased. +fn words(name: &str) -> Vec { + name.to_snake_case() + .split('_') + .filter(|w| !w.is_empty()) + .map(str::to_string) + .collect() +} + +/// One word in exported position: an initialism in capitals, anything else +/// capitalized. +fn capitalize(word: &str) -> String { + let upper = word.to_ascii_uppercase(); + if INITIALISMS.contains(&upper.as_str()) { + return upper; + } + let mut chars = word.chars(); + match chars.next() { + Some(first) => first.to_uppercase().chain(chars).collect(), + None => String::new(), + } +} + +/// The exported (PascalCase) Go spelling of an IDL name, with Go's +/// initialisms: `user_id` is `UserID`, `ttl_seconds` is `TTLSeconds`. +pub(crate) fn pascal(name: &str) -> String { + words(name).iter().map(|w| capitalize(w)).collect() +} + +/// The unexported (lowerCamelCase) Go spelling of an IDL name, with Go's +/// initialisms after the first word: `user_id` is `userID`, `url` is `url`. +pub(crate) fn camel(name: &str) -> String { + let words = words(name); + let mut out = String::new(); + for (i, w) in words.iter().enumerate() { + if i == 0 { + out.push_str(w); + } else { + out.push_str(&capitalize(w)); + } + } + out +} + +/// Methods every object wrapper declares: `Close` releases the wrapper's +/// reference (its unexported helpers can't collide with an exported name). +const OBJECT_METHODS: &[&str] = &["Close"]; + +/// The Go name of an interface method: [`pascal`], with a trailing +/// underscore when it would redeclare a method the wrapper already has +/// (`close` is `Close_`). Constructors and statics are package functions +/// prefixed or suffixed with the type name, so they can't collide. +pub(crate) fn method(name: &str) -> String { + lang::escape_member(&pascal(name), OBJECT_METHODS) +} + +/// The Go spelling of a parameter of the wrapper for `f`: lowerCamelCase, +/// escaped against Go's keywords and the names wrapper bodies use. A +/// method's receiver is `s`, and an async wrapper's leading +/// `context.Context` is `ctx`, so a parameter spelled like either is +/// escaped there too. +pub(crate) fn param(name: &str, f: &FnBinding) -> String { + let ident = lang::escape_ident(&camel(name), lang::GO_KEYWORDS); + let taken = RESERVED.contains(&ident.as_str()) + || (f.has_self && ident == "s") + || (f.is_async() && ident == "ctx"); + if taken { + format!("{ident}_") + } else { + ident + } +} + +/// The Go spelling of a callback method's parameter in the interface the +/// consumer implements: lowerCamelCase, escaped against Go's keywords (a +/// method signature has no body to collide with). +pub(crate) fn method_param(name: &str) -> String { + lang::escape_ident(&camel(name), lang::GO_KEYWORDS) +} + +/// The Go spelling of one C ABI slot inside an exported trampoline: the C +/// name, escaped against Go's keywords and the names bodies use. The +/// preamble `extern` uses the same spelling so the two prototypes agree. +pub(crate) fn slot(name: &str) -> String { + let ident = lang::escape_ident(name, lang::GO_KEYWORDS); + if RESERVED.contains(&ident.as_str()) { + format!("{ident}_") + } else { + ident + } +} + +/// The exported Go name of a struct field or variant field. `Error`, +/// `Code`, and `Message` are taken on error-code types, so a field spelled +/// like one of those gains a trailing underscore there (see +/// [`error_field`]). +pub(crate) fn field(name: &str) -> String { + pascal(name) +} + +/// The Go name of an error code's payload field: [`field`], with a +/// trailing underscore when it would collide with the error type's +/// `Message` field or its `Code` and `Error` methods. +pub(crate) fn error_field(name: &str) -> String { + let f = field(name); + if matches!(f.as_str(), "Message" | "Code" | "Error") { + format!("{f}_") + } else { + f + } +} + +/// Every exported name the generated package declares for the API, keyed by +/// the declaration it names. +pub(crate) struct GoNames { + /// Free functions by C base symbol. + functions: BTreeMap, + /// Error-code types by C constant. + codes: BTreeMap, +} + +impl GoNames { + /// Name every declaration of `model`, resolving clashes in the flat + /// package (see the module docs). + pub(crate) fn new(model: &Model) -> Self { + // Names the runtime declares, then every type-level name. + let mut taken: BTreeSet = ["Error", "DebugLive"].map(String::from).into(); + for m in &model.modules { + if let Some(e) = &m.errors { + taken.insert(domain_type(e)); + } + for e in &m.enums { + let name = pascal(&e.name); + for v in &e.variants { + taken.insert(format!("{name}{}", pascal(&v.name))); + } + taken.insert(name); + } + for s in &m.structs { + taken.insert(pascal(&s.name)); + } + for cb in &m.callback_interfaces { + taken.insert(pascal(&cb.name)); + } + for i in &m.interfaces { + let name = pascal(&i.name); + for c in &i.constructors { + taken.insert(constructor(&name, c)); + } + for f in &i.statics { + taken.insert(format!("{name}{}", pascal(&f.name))); + } + taken.insert(name); + } + } + + // Error codes: `{Code}Error`, or `{Domain}{Code}Error` past a clash. + let mut codes = BTreeMap::new(); + for m in &model.modules { + let Some(e) = &m.errors else { continue }; + for c in &e.codes { + let plain = lang_error_name(&pascal(&c.name)); + let name = if taken.contains(&plain) { + lang_error_name(&format!( + "{}{}", + domain_type(e).trim_end_matches("Error"), + pascal(&c.name) + )) + } else { + plain + }; + taken.insert(name.clone()); + codes.insert(c.c_const.clone(), name); + } + } + + // Free functions: the bare name unless it clashes with a declared + // name or another function's. + let mut seen: BTreeMap = BTreeMap::new(); + for (_, f) in model.functions() { + *seen.entry(pascal(&f.name)).or_default() += 1; + } + let functions = model + .functions() + .map(|(m, f)| { + let bare = pascal(&f.name); + let name = if taken.contains(&bare) || seen[&bare] > 1 { + pascal(&format!("{}_{}", m.path, f.name)) + } else { + bare + }; + (f.c_base.clone(), name) + }) + .collect(); + Self { functions, codes } + } + + /// The Go name of the free function `f`. + pub(crate) fn function(&self, f: &FnBinding) -> &str { + &self.functions[&f.c_base] + } + + /// The Go type name of the error code `c`. + pub(crate) fn code(&self, c: &ErrorCodeBinding) -> &str { + &self.codes[&c.c_const] + } +} + +/// `name` with exactly one `Error` suffix. +fn lang_error_name(name: &str) -> String { + if name.ends_with("Error") { + name.to_string() + } else { + format!("{name}Error") + } +} + +/// The Go name of an error domain's interface type (`KvError`). +pub(crate) fn domain_type(e: &ErrorBinding) -> String { + pascal(&e.type_name) +} + +/// The factory name of the constructor `c` on the interface `iface` (Go +/// spelling): `new` on `Store` is `NewStore`, `open` is `OpenStore`. +pub(crate) fn constructor(iface: &str, c: &FnBinding) -> String { + format!("{}{iface}", pascal(&c.name)) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn initialisms_keep_one_case() { + assert_eq!(pascal("user_id"), "UserID"); + assert_eq!(pascal("ttl_seconds"), "TTLSeconds"); + assert_eq!(pascal("http_url"), "HTTPURL"); + assert_eq!(pascal("UserId"), "UserID"); + assert_eq!(pascal("KEY_NOT_FOUND"), "KeyNotFound"); + assert_eq!(pascal("MaybeI64"), "MaybeI64"); + assert_eq!(pascal("i8_value"), "I8Value"); + assert_eq!(camel("user_id"), "userID"); + assert_eq!(camel("id"), "id"); + assert_eq!(camel("url_path"), "urlPath"); + assert_eq!(camel("ttl_seconds"), "ttlSeconds"); + } + + #[test] + fn parameters_avoid_keywords_and_body_names() { + use weaveffi_model::ir::Api; + let yaml = r#" +version: "0.11.0" +modules: + - name: m + interfaces: + - name: Box + methods: + - { name: put, params: [{ name: s, type: i32 }, { name: type, type: i32 }] } + functions: + - { name: plain, params: [{ name: s, type: i32 }, { name: ctx, type: i32 }] } + - { name: later, params: [{ name: ctx, type: i32 }], return: i32, async: true } +"#; + let api: Api = serde_yaml::from_str(yaml).unwrap(); + let model = Model::assume_valid(&api, weaveffi_model::pkg::Identity::named("p")); + let m = &model.modules[0]; + let (put, plain, later) = ( + &m.interfaces[0].methods[0], + &m.functions[0], + &m.functions[1], + ); + assert_eq!(param("s", put), "s_"); + assert_eq!(param("type", put), "type_"); + assert_eq!(param("s", plain), "s"); + assert_eq!(param("ctx", plain), "ctx"); + assert_eq!(param("ctx", later), "ctx_"); + assert_eq!(param("context", plain), "context_"); + assert_eq!(method_param("type"), "type_"); + assert_eq!(method_param("item"), "item"); + assert_eq!(slot("err"), "err_"); + assert_eq!(error_field("message"), "Message_"); + assert_eq!(error_field("key"), "Key"); + } + + #[test] + fn methods_avoid_the_wrapper_close() { + assert_eq!(method("close"), "Close_"); + assert_eq!(method("Close"), "Close_"); + assert_eq!(method("close_all"), "CloseAll"); + assert_eq!(method("dispose"), "Dispose"); + } +} diff --git a/crates/weaveffi-gen/src/targets/go/package.rs b/crates/weaveffi-cli/src/targets/go/package.rs similarity index 71% rename from crates/weaveffi-gen/src/targets/go/package.rs rename to crates/weaveffi-cli/src/targets/go/package.rs index 07f056c1..4ee6bf82 100644 --- a/crates/weaveffi-gen/src/targets/go/package.rs +++ b/crates/weaveffi-cli/src/targets/go/package.rs @@ -1,12 +1,10 @@ //! Module scaffold: `go.mod`, the README flavors, and the `package` hook //! bundling per-platform shared libraries with a rewritten cgo preamble. -use crate::package::{PackageContext, PackagedFile}; +use crate::package::{per_platform_libraries, Artifact, PackageContext, PackagedFile}; use crate::platform::{NativeBinary, Platform}; use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use camino::Utf8Path; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; +use weaveffi_model::model::Model; use crate::targets::go::{render_files, GoConfig, Names}; @@ -41,8 +39,8 @@ fn bundled<'a>( } /// The generated `go.mod` for the emitted module. -pub(crate) fn render_go_mod(module_path: &str, input_basename: &str) -> String { - let prelude = render_prelude(CommentStyle::DoubleSlash, input_basename); +pub(crate) fn render_go_mod(module_path: &str) -> String { + let prelude = render_prelude(CommentStyle::DoubleSlash); let trailer = render_trailer(CommentStyle::DoubleSlash, "go.mod"); // Go 1.23 is required for the standard `iter` package the lazy // `iter` wrappers return. @@ -50,8 +48,8 @@ pub(crate) fn render_go_mod(module_path: &str, input_basename: &str) -> String { } /// README for the generate-mode output. -pub(crate) fn render_readme(names: &Names, input_basename: &str) -> String { - let prelude = render_prelude(CommentStyle::Xml, input_basename); +pub(crate) fn render_readme(names: &Names) -> String { + let prelude = render_prelude(CommentStyle::Xml); let trailer = render_trailer(CommentStyle::Xml, "README.md"); let Names { module_path, @@ -78,6 +76,12 @@ export LD_LIBRARY_PATH="/path/to/lib" # DYLD_LIBRARY_PATH on macOS go build ./... ``` +cgo links the library when the program is built and the platform loader +finds it when the program starts, so the library-path environment variable +the runtime-loading bindings (Python, .NET, and others) honor doesn't apply. +Add `-Wl,-rpath,/path/to/lib` to `CGO_LDFLAGS` to record the directory in the +binary instead of setting the loader path. + Depend on the module from your own project with a `require` (and, for a local checkout, a `replace`) of `{module_path}`, then import it: @@ -94,21 +98,25 @@ Go 1.23 or newer is required. - Interfaces are reference-counted objects. Call `Close` when you're done with one (a finalizer releases it otherwise); it's safe to call twice or while a call is in flight. +- A call that declares errors returns `error` values you match with + `errors.As`; any other call panics with an `*Error` when the library + reports a failure, since that's a bug. - Async functions take a `context.Context` first. Cancellable ones cancel the native call when the context is done. - Callback interfaces are Go interfaces you implement. The library may call - them from any thread, and a panic is reported to the native caller as a - foreign error instead of crashing. -- Loading checks the library's ABI revision and contract checksums and - panics with the stale module's name on a mismatch. + them from any thread; a panic is reported to the native caller as a + callback failure instead of crashing. +- Importing the package checks the library's ABI revision and every + module's contract table, and panics naming the first declaration that's + missing from the library or changed since the bindings were generated. {trailer}"# ) } /// README for a packaged Go module that bundles per-platform libraries. -fn render_packaged_readme(ctx: &PackageContext, names: &Names, input_basename: &str) -> String { - let prelude = render_prelude(CommentStyle::Xml, input_basename); +fn render_packaged_readme(ctx: &PackageContext, names: &Names) -> String { + let prelude = render_prelude(CommentStyle::Xml); let trailer = render_trailer(CommentStyle::Xml, "README.md"); let platforms: Vec = bundled(ctx) .map(|(nb, _, _)| format!("- `lib/{}/`", nb.platform.id())) @@ -131,21 +139,20 @@ library with no manual `CGO_LDFLAGS`. ) } -/// The full packaged file set: the generated module with a self-contained -/// cgo preamble plus one bundled shared library per desktop platform. -/// Binaries for platforms without a `GOOS,GOARCH` cgo slot (Android, wasm32) -/// are skipped. +/// The packaged module: the generated module with a self-contained cgo +/// preamble plus one bundled shared library per desktop platform, or +/// nothing when no desktop platform was built. Binaries for platforms +/// without a `GOOS,GOARCH` cgo slot (Android, iOS, `wasm32`) are skipped. pub(crate) fn package_files( - api: &ResolvedApi, - model: &BindingModel, + model: &Model, ctx: &PackageContext, - out_dir: &Utf8Path, config: &GoConfig, -) -> Vec { - let dir = out_dir.join("go"); - let input_basename = config.input_basename(); - let names = Names::new(api, config); +) -> Vec { + let names = Names::new(model, config); let library = &names.library; + if bundled(ctx).next().is_none() { + return Vec::new(); + } // Expand the single generate-mode `#cgo LDFLAGS` line into per // GOOS/GOARCH library search and rpath directives (all `${SRCDIR}` @@ -166,23 +173,19 @@ pub(crate) fn package_files( } cgo.push_str(&original); - let mut files: Vec = render_files(api, model, config) + let mut files: Vec = render_files(model, config) .into_iter() .map(|(name, contents)| { let contents = match name.as_str() { "bindings.go" => contents.replace(&original, &cgo), - "README.md" => render_packaged_readme(ctx, &names, input_basename), + "README.md" => render_packaged_readme(ctx, &names), _ => contents, }; - PackagedFile::text(dir.join(name), contents) + PackagedFile::text(name, contents) }) .collect(); - for (nb, _, _) in bundled(ctx) { - let dest = dir - .join("lib") - .join(nb.platform.id()) - .join(ctx.binaries.bundled_filename(nb.platform)); - files.push(PackagedFile::copy(dest, nb.source.clone())); - } - files + files.extend(per_platform_libraries(ctx.binaries, "lib", |p| { + go_build_tags(p).is_some() + })); + vec![Artifact::directory(format!("go/{}", names.package), files)] } diff --git a/crates/weaveffi-gen/src/targets/go/runtime.rs b/crates/weaveffi-cli/src/targets/go/runtime.rs similarity index 58% rename from crates/weaveffi-gen/src/targets/go/runtime.rs rename to crates/weaveffi-cli/src/targets/go/runtime.rs index c95037ca..ce69849c 100644 --- a/crates/weaveffi-gen/src/targets/go/runtime.rs +++ b/crates/weaveffi-cli/src/targets/go/runtime.rs @@ -3,11 +3,12 @@ //! The sources live in `runtime/` as ordinary Go files and are spliced into //! the output with `{{PLACEHOLDER}}` substitution: //! -//! * `runtime.go`: the load-time ABI and checksum checks, the generic `Error` -//! type, error-slot and string/bytes helpers, the object reference guard -//! (`wvRef`), the async completion bridge, and `DebugLive`. -//! * `codec.go`: the value-buffer writer and reader, emitted only when the -//! API has buffered values. +//! * `runtime.go`: the load-time ABI and contract checks, the generic +//! `Error` type, error-slot, callback, string, and byte-run helpers, the +//! object reference guard (`wvRef`), the async completion bridge, and +//! `DebugLive`. +//! * `codec.go`: the value-buffer writer and reader and the generic list, +//! map, and optional codecs the generated pairs are built from. use crate::utils::{render_prelude, render_trailer, CommentStyle}; use weaveffi_model::model::ABI_VERSION; @@ -25,7 +26,7 @@ pub(crate) struct RuntimeNames<'a> { pub(crate) header: &'a str, } -fn splice(template: &str, names: &RuntimeNames, input_basename: &str, file: &str) -> String { +fn splice(template: &str, names: &RuntimeNames, file: &str) -> String { let body = template .replace("{{PACKAGE}}", names.package) .replace("{{PREFIX}}", names.prefix) @@ -33,17 +34,17 @@ fn splice(template: &str, names: &RuntimeNames, input_basename: &str, file: &str .replace("{{ABI_VERSION}}", &ABI_VERSION.to_string()); format!( "{}{body}\n{}", - render_prelude(CommentStyle::DoubleSlash, input_basename), + render_prelude(CommentStyle::DoubleSlash), render_trailer(CommentStyle::DoubleSlash, file) ) } /// Render `runtime.go`. -pub(crate) fn render_runtime(names: &RuntimeNames, input_basename: &str) -> String { - splice(RUNTIME_GO, names, input_basename, "runtime.go") +pub(crate) fn render_runtime(names: &RuntimeNames) -> String { + splice(RUNTIME_GO, names, "runtime.go") } /// Render `codec.go`. -pub(crate) fn render_codec(names: &RuntimeNames, input_basename: &str) -> String { - splice(CODEC_GO, names, input_basename, "codec.go") +pub(crate) fn render_codec(names: &RuntimeNames) -> String { + splice(CODEC_GO, names, "codec.go") } diff --git a/crates/weaveffi-gen/src/targets/go/runtime/codec.go b/crates/weaveffi-cli/src/targets/go/runtime/codec.go similarity index 59% rename from crates/weaveffi-gen/src/targets/go/runtime/codec.go rename to crates/weaveffi-cli/src/targets/go/runtime/codec.go index d61154d0..9a9c70a1 100644 --- a/crates/weaveffi-gen/src/targets/go/runtime/codec.go +++ b/crates/weaveffi-cli/src/targets/go/runtime/codec.go @@ -1,11 +1,122 @@ package {{PACKAGE}} +/* +#include "{{HEADER}}" +*/ +import "C" + import ( "encoding/binary" "math" "unicode/utf8" ) +// wvEncode encodes v with write into a new value buffer. +func wvEncode[T any](v T, write func(*wvWriter, T)) []byte { + var w wvWriter + write(&w, v) + return w.buf +} + +// wvDecode decodes one value from a returned value buffer, checks that +// nothing trails it, and releases the native allocation. +func wvDecode[T any](ptr *C.uint8_t, n C.size_t, read func(*wvReader) T) T { + return wvDecodeBytes(wvTakeBytes(ptr, n), read) +} + +// wvDecodeBorrowed is wvDecode for a buffer the native library only lends +// (a callback argument). +func wvDecodeBorrowed[T any](ptr *C.uint8_t, n C.size_t, read func(*wvReader) T) T { + return wvDecodeBytes(wvBorrowBytes(ptr, n), read) +} + +func wvDecodeBytes[T any](b []byte, read func(*wvReader) T) T { + r := wvReader{buf: b} + v := read(&r) + r.expectEnd() + return v +} + +// wvWriteList writes a u32 count, then each element. +func wvWriteList[T any](w *wvWriter, v []T, elem func(*wvWriter, T)) { + w.writeLen(len(v)) + for _, e := range v { + elem(w, e) + } +} + +// wvReadList reads a u32 count, then that many elements. +func wvReadList[T any](r *wvReader, elem func(*wvReader) T) []T { + n := r.readLen() + v := make([]T, 0, r.capHint(n)) + for range n { + v = append(v, elem(r)) + } + return v +} + +// wvWriteMap writes a u32 count, then each key and value. Entry order +// carries no meaning. +func wvWriteMap[K comparable, V any](w *wvWriter, v map[K]V, key func(*wvWriter, K), val func(*wvWriter, V)) { + w.writeLen(len(v)) + for k, e := range v { + key(w, k) + val(w, e) + } +} + +// wvReadMap reads a u32 count, then that many keys and values. A repeated +// key is a malformed buffer. +func wvReadMap[K comparable, V any](r *wvReader, key func(*wvReader) K, val func(*wvReader) V) map[K]V { + n := r.readLen() + v := make(map[K]V, r.capHint(n)) + for range n { + k := key(r) + if _, dup := v[k]; dup { + wvMalformed("repeated map key") + } + v[k] = val(r) + } + return v +} + +// wvWriteOptional writes an optional held as a pointer: a presence flag, +// then the value when present. +func wvWriteOptional[T any](w *wvWriter, v *T, elem func(*wvWriter, T)) { + w.writeBool(v != nil) + if v != nil { + elem(w, *v) + } +} + +// wvReadOptional reads an optional held as a pointer. +func wvReadOptional[T any](r *wvReader, elem func(*wvReader) T) *T { + if !r.readOptionFlag() { + return nil + } + v := elem(r) + return &v +} + +// wvWriteNilable writes an optional whose Go type is already nil-able (a +// slice, map, interface, or wrapper pointer): present says it isn't nil. +func wvWriteNilable[T any](w *wvWriter, v T, present bool, elem func(*wvWriter, T)) { + w.writeBool(present) + if present { + elem(w, v) + } +} + +// wvReadNilable reads an optional whose Go type is already nil-able; absent +// is the zero value (nil). +func wvReadNilable[T any](r *wvReader, elem func(*wvReader) T) T { + if !r.readOptionFlag() { + var zero T + return zero + } + return elem(r) +} + // wvWriter serializes values into the value-buffer format: little-endian, // packed, u32 length prefixes. type wvWriter struct { @@ -77,10 +188,6 @@ func (w *wvWriter) writeBytes(v []byte) { w.buf = append(w.buf, v...) } -func (w *wvWriter) writeOptionFlag(present bool) { - w.writeBool(present) -} - // wvReader decodes values from the value-buffer format. A malformed buffer // is a producer/consumer contract violation, so every read panics instead of // returning a typed domain error. diff --git a/crates/weaveffi-cli/src/targets/go/runtime/runtime.go b/crates/weaveffi-cli/src/targets/go/runtime/runtime.go new file mode 100644 index 00000000..6a532ce5 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/runtime/runtime.go @@ -0,0 +1,417 @@ +package {{PACKAGE}} + +/* +#include +#include "{{HEADER}}" + +static void* wvRuntimeHandlePtr(uintptr_t h) { return (void*)h; } +*/ +import "C" + +import ( + "cmp" + "context" + "errors" + "fmt" + "runtime/cgo" + "slices" + "sync/atomic" + "unsafe" +) + +// wvABIVersion is the C ABI revision these bindings were generated against. +const wvABIVersion uint32 = {{ABI_VERSION}} + +// The runtime error codes this package produces or interprets itself. +const ( + wvCodeForeign int32 = -4 + wvCodeCancelled int32 = -5 +) + +// Error is a failure no error domain claims: a generic failure (code -1), a +// panic in the native library (-2), an argument or result that couldn't be +// marshalled (-3), a callback implementation that failed (-4), or a code +// outside the domain. A call that doesn't declare errors panics with an +// *Error, since its failure is a bug; a call that does returns one. +type Error struct { + // Code is the runtime code. + Code int32 + // Message is the native library's message. + Message string +} + +func (e *Error) Error() string { + return fmt.Sprintf("{{PACKAGE}}: %s (code %d)", e.Message, e.Code) +} + +// DebugLive reports how many resources of one kind the native library holds: +// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 byte runs. Kind -1 +// reports 1 when the library counts at all. Every count is 0 unless the +// library was built with leak checks enabled; tests use it to assert that +// nothing leaked. +func DebugLive(kind int32) uint64 { + return uint64(C.{{PREFIX}}_debug_live(C.int32_t(kind))) +} + +// ── Load-time checks ── + +// wvCheckABI panics when the linked library implements a different C ABI +// revision than these bindings were generated for. +func wvCheckABI() { + if found := uint32(C.{{PREFIX}}_abi_version()); found != wvABIVersion { + panic(fmt.Errorf("{{PACKAGE}}: the linked library implements C ABI revision %d, but these bindings were generated for revision %d", found, wvABIVersion)) + } +} + +// wvContractEntry is one declaration these bindings were generated with: +// the FNV-1a hashes of its dotted path and of its signature. +type wvContractEntry struct { + id, hash uint64 + path string +} + +// wvCheckContract panics unless every entry in want is in the library's +// contract table for one top-level module (n entries sorted by id) with an +// equal signature hash. Entries only the library has are fine. +func wvCheckContract(table *C.{{PREFIX}}_contract_entry, n C.size_t, want []wvContractEntry) { + have := unsafe.Slice(table, int(n)) + for _, w := range want { + i, found := slices.BinarySearchFunc(have, w.id, func(e C.{{PREFIX}}_contract_entry, id uint64) int { + return cmp.Compare(uint64(e.id), id) + }) + if !found { + panic(fmt.Errorf("{{PACKAGE}}: %s is missing from the library", w.path)) + } + if uint64(have[i].hash) != w.hash { + panic(fmt.Errorf("{{PACKAGE}}: %s changed since these bindings were generated", w.path)) + } + } +} + +// ── Errors ── + +// wvFailure is a non-zero error slot copied into Go memory. +type wvFailure struct { + code int32 + message string + payload []byte +} + +// err converts the failure into the generic *Error. +func (f wvFailure) err() error { + return &Error{Code: f.code, Message: f.message} +} + +func wvCopyFailure(cErr *C.{{PREFIX}}_error) wvFailure { + f := wvFailure{code: int32(cErr.code)} + if cErr.message != nil { + f.message = C.GoString(cErr.message) + } + if cErr.payload_ptr != nil { + f.payload = wvBorrowBytes(cErr.payload_ptr, cErr.payload_len) + } + return f +} + +// wvTakeError copies a non-zero error slot and clears it. +func wvTakeError(cErr *C.{{PREFIX}}_error) wvFailure { + f := wvCopyFailure(cErr) + C.{{PREFIX}}_error_clear(cErr) + return f +} + +// wvTakeBoxedError copies the heap-boxed error an async completion receives +// and frees the box. +func wvTakeBoxedError(cErr *C.{{PREFIX}}_error) wvFailure { + f := wvCopyFailure(cErr) + C.{{PREFIX}}_error_free(cErr) + return f +} + +// wvTrap panics with an *Error when the slot reports a failure. Wrappers of +// calls that declare no errors check their slot with it: a failure there is +// a bug in the native library, an argument it couldn't take, or a callback +// failure it let through. +func wvTrap(cErr *C.{{PREFIX}}_error) { + if cErr.code != 0 { + panic(wvTakeError(cErr).err()) + } +} + +// wvDomainError is implemented by every error-code type: its code and its +// fields encoded as a value buffer (nil when it has none). +type wvDomainError interface { + error + Code() int32 + wvPayload() []byte +} + +// wvSetError reports a failure through a callback's error slot. The native +// library copies the message and payload, so both are only borrowed. +func wvSetError(outErr *C.{{PREFIX}}_error, code int32, message string, payload []byte) { + msg := C.CString(message) + defer C.free(unsafe.Pointer(msg)) + C.{{PREFIX}}_error_set(outErr, C.int32_t(code), msg) + if len(payload) > 0 { + ptr, n := wvBytes(payload) + C.{{PREFIX}}_error_set_payload(outErr, ptr, n) + } +} + +// wvCallbackFailed reports the error a throwing callback method returned: +// an error of the domain D in scope for the method reports its code and +// payload, and anything else is a callback failure (code -4). +func wvCallbackFailed[D error](outErr *C.{{PREFIX}}_error, err error) { + var domain D + if errors.As(err, &domain) { + if d, ok := any(domain).(wvDomainError); ok { + wvSetError(outErr, d.Code(), d.Error(), d.wvPayload()) + return + } + } + wvSetError(outErr, wvCodeForeign, err.Error(), nil) +} + +// wvRecoverCallback, deferred by every callback trampoline, reports a panic +// in the implementation as a callback failure (code -4) instead of letting +// it unwind into the native caller. +func wvRecoverCallback(outErr *C.{{PREFIX}}_error) { + if r := recover(); r != nil { + msg := fmt.Sprint(r) + if err, ok := r.(error); ok { + msg = err.Error() + } + wvSetError(outErr, wvCodeForeign, msg, nil) + } +} + +// ── Callback contexts ── + +// wvNewCallback keeps impl alive in a handle table for as long as the +// native library holds it and returns the handle as the callback's context +// pointer; the vtable's free entry deletes it (see wvFreeCallback). A nil +// implementation of a required callback panics. +func wvNewCallback(impl any) unsafe.Pointer { + if impl == nil { + panic("{{PACKAGE}}: nil callback implementation") + } + return C.wvRuntimeHandlePtr(C.uintptr_t(cgo.NewHandle(impl))) +} + +// wvCallback recovers the implementation behind a callback context. +func wvCallback[T any](ctx unsafe.Pointer) T { + return cgo.Handle(uintptr(ctx)).Value().(T) +} + +// wvFreeCallback releases the implementation behind a callback context once +// the native library is done with it. +func wvFreeCallback(ctx unsafe.Pointer) { + cgo.Handle(uintptr(ctx)).Delete() +} + +// ── Strings and bytes ── + +// wvStr returns a borrowed (pointer, length) view of s's UTF-8 bytes for the +// duration of one call. Strings may contain NUL bytes. +func wvStr(s string) (*C.uint8_t, C.size_t) { + return (*C.uint8_t)(unsafe.Pointer(unsafe.StringData(s))), C.size_t(len(s)) +} + +// wvBytes returns a borrowed (pointer, length) view of b for the duration of +// one call. +func wvBytes(b []byte) (*C.uint8_t, C.size_t) { + return (*C.uint8_t)(unsafe.Pointer(unsafe.SliceData(b))), C.size_t(len(b)) +} + +// wvBorrowString copies a borrowed UTF-8 run into a Go string. +func wvBorrowString(ptr *C.uint8_t, n C.size_t) string { + if ptr == nil { + return "" + } + return string(unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) +} + +// wvTakeString copies a returned UTF-8 run into a Go string and releases the +// native allocation. +func wvTakeString(ptr *C.uint8_t, n C.size_t) string { + s := wvBorrowString(ptr, n) + C.{{PREFIX}}_free_bytes(ptr, n) + return s +} + +// wvBorrowBytes copies a borrowed byte run into Go memory. A null pointer +// copies to an empty slice. +func wvBorrowBytes(ptr *C.uint8_t, n C.size_t) []byte { + if ptr == nil { + return []byte{} + } + return append([]byte{}, unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)...) +} + +// wvTakeBytes copies a returned byte run into Go memory and releases the +// native allocation. +func wvTakeBytes(ptr *C.uint8_t, n C.size_t) []byte { + b := wvBorrowBytes(ptr, n) + C.{{PREFIX}}_free_bytes(ptr, n) + return b +} + +// wvHandOverString copies s into a run allocated with the native library's +// allocator and writes it to a callback's return slots; the library adopts +// the run. +func wvHandOverString(s string, outPtr **C.uint8_t, outLen *C.size_t) { + ptr, n := wvStr(s) + wvHandOver(ptr, n, outPtr, outLen) +} + +// wvHandOverBytes is wvHandOverString for a byte slice (raw bytes or an +// encoded value buffer). +func wvHandOverBytes(b []byte, outPtr **C.uint8_t, outLen *C.size_t) { + ptr, n := wvBytes(b) + wvHandOver(ptr, n, outPtr, outLen) +} + +func wvHandOver(ptr *C.uint8_t, n C.size_t, outPtr **C.uint8_t, outLen *C.size_t) { + run := C.{{PREFIX}}_alloc(n) + if n > 0 { + copy(unsafe.Slice((*byte)(unsafe.Pointer(run)), n), unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) + } + *outPtr, *outLen = run, n +} + +// ── Objects ── + +// wvRef owns one strong reference to a native object on behalf of a wrapper. +// Calls borrow the pointer between acquire and release. Close may race them: +// the reference is destroyed exactly once, by close when no call is in +// flight, or else by the last in-flight call to finish. +type wvRef struct { + ptr unsafe.Pointer + destroy func(unsafe.Pointer) + // state counts in-flight calls in its upper bits; the low bit is set + // once close has run. + state atomic.Uint64 +} + +func (r *wvRef) init(ptr unsafe.Pointer, destroy func(unsafe.Pointer)) { + r.ptr = ptr + r.destroy = destroy +} + +// acquire borrows the pointer for one call, panicking when the wrapper was +// already closed. +func (r *wvRef) acquire(typeName string) unsafe.Pointer { + for { + s := r.state.Load() + if s&1 != 0 { + panic("{{PACKAGE}}: " + typeName + " used after Close") + } + if r.state.CompareAndSwap(s, s+2) { + return r.ptr + } + } +} + +// release ends one call's borrow. +func (r *wvRef) release() { + if r.state.Add(^uint64(1)) == 1 { + r.destroy(r.ptr) + } +} + +// close releases the wrapper's reference once no call is in flight. Calling +// it again is a no-op. +func (r *wvRef) close() { + for { + s := r.state.Load() + if s&1 != 0 { + return + } + if r.state.CompareAndSwap(s, s|1) { + if s == 0 { + r.destroy(r.ptr) + } + return + } + } +} + +// ── Async ── + +// wvOutcome is one async completion: a converted result, or the failure the +// native library reported. +type wvOutcome[T any] struct { + val T + fail *wvFailure +} + +// wvAsyncCall is one pending async call: the channel its completion is +// delivered on, kept alive in a handle table while the call is in flight. +type wvAsyncCall[T any] struct { + done chan wvOutcome[T] + handle cgo.Handle +} + +func wvNewAsyncCall[T any]() *wvAsyncCall[T] { + c := &wvAsyncCall[T]{done: make(chan wvOutcome[T], 1)} + c.handle = cgo.NewHandle(c.done) + return c +} + +// context is the completion's context pointer. +func (c *wvAsyncCall[T]) context() unsafe.Pointer { + return C.wvRuntimeHandlePtr(C.uintptr_t(c.handle)) +} + +// wvComplete delivers one async completion to the waiting wrapper. It runs on +// a native thread inside an exported trampoline, so it never panics: a result +// that fails to decode is delivered as a marshalling failure. +func wvComplete[T any](context unsafe.Pointer, cErr *C.{{PREFIX}}_error, convert func() T) { + h := cgo.Handle(uintptr(context)) + done := h.Value().(chan wvOutcome[T]) + h.Delete() + done <- wvSettle(cErr, convert) +} + +func wvSettle[T any](cErr *C.{{PREFIX}}_error, convert func() T) (o wvOutcome[T]) { + if cErr != nil { + if f := wvTakeBoxedError(cErr); f.code != 0 { + o.fail = &f + return o + } + } + defer func() { + if r := recover(); r != nil { + o = wvOutcome[T]{fail: &wvFailure{code: -3, message: fmt.Sprint(r)}} + } + }() + o.val = convert() + return o +} + +// wait blocks until the call completes or ctx is done. With a cancel token, +// ctx's cancellation cancels the native call and waits for its completion, +// which the native library delivers promptly; without one, the call is +// abandoned and ctx.Err() returned at once. A cancelled completion (code -5) +// returns ctx.Err(), or context.Canceled when ctx itself isn't done. +func (c *wvAsyncCall[T]) wait(ctx context.Context, token *C.{{PREFIX}}_cancel_token) (T, *wvFailure, error) { + var o wvOutcome[T] + select { + case o = <-c.done: + case <-ctx.Done(): + if token == nil { + var zero T + return zero, nil, ctx.Err() + } + C.{{PREFIX}}_cancel_token_cancel(token) + o = <-c.done + } + if o.fail != nil && o.fail.code == wvCodeCancelled { + var zero T + if err := ctx.Err(); err != nil { + return zero, nil, err + } + return zero, nil, context.Canceled + } + return o.val, o.fail, nil +} diff --git a/crates/weaveffi-cli/src/targets/go/tests.rs b/crates/weaveffi-cli/src/targets/go/tests.rs new file mode 100644 index 00000000..004231e8 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/tests.rs @@ -0,0 +1,374 @@ +//! Unit tests: render a small API that exercises every C ABI revision 4 +//! shape the Go bindings distinguish and assert the key pieces of each +//! contract. + +use weaveffi_model::contract::entries; +use weaveffi_model::ir::Api; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; + +use super::{render_files, GoConfig}; + +/// One module with an error domain carrying a field; a C-style enum; a +/// record carrying objects; an interface with a `new` constructor, a +/// method, and a throwing static; nullable objects; an iterator of objects; +/// a cancellable async function; a composite map; and a callback interface +/// whose methods take every argument family and return a direct value, an +/// enum, a string, an optional record, an object, and an optional object, +/// one of them `throws`, passed both required and optional. +const FIXTURE: &str = r#" +version: "0.11.0" +modules: + - name: shop + errors: + name: ShopError + codes: + - { name: OutOfStock, code: 1, message: "out of stock", fields: [{ name: sku, type: string }] } + - { name: Closed, code: 2, message: "closed" } + enums: + - name: Mood + variants: [{ name: Happy, value: 0 }, { name: Grumpy, value: 1 }] + structs: + - name: Order + fields: + - { name: order_id, type: u64 } + - { name: cart, type: Cart } + - { name: history, type: "[Cart]" } + - { name: note, type: "string?" } + interfaces: + - name: Cart + constructors: + - { name: new, params: [{ name: owner, type: string }] } + methods: + - { name: add, params: [{ name: sku, type: string }], return: bool } + statics: + - { name: restore, params: [{ name: id, type: i64 }], return: Cart, throws: true } + callback_interfaces: + - name: Watcher + methods: + - name: on_event + params: + - { name: name, type: string } + - { name: count, type: i32 } + - { name: order, type: Order } + - { name: cart, type: Cart } + return: bool + - { name: mood, return: Mood } + - { name: label, return: string, throws: true } + - { name: latest, return: "Order?" } + - { name: favorite, return: Cart } + - { name: maybe, return: "Cart?" } + - { name: on_done } + functions: + - { name: find_cart, params: [{ name: current, type: "Cart?" }], return: "Cart?" } + - { name: all_carts, return: "iter" } + - { name: watch, params: [{ name: watcher, type: Watcher }] } + - { name: maybe_watch, params: [{ name: watcher, type: "Watcher?" }] } + - { name: echo, params: [{ name: text, type: string }], return: string } + - { name: tally, params: [{ name: counts, type: "{string:[i32?]}" }], return: "[Order]", throws: true } + - { name: wait, params: [{ name: ms, type: i64 }], return: i64, async: true, cancellable: true } +"#; + +fn model() -> Model { + let api: Api = serde_yaml::from_str(FIXTURE).unwrap(); + validate(&api, &Identity::named("shop_kit"), None).unwrap() +} + +fn files() -> Vec<(String, String)> { + render_files(&model(), &GoConfig::default()) +} + +fn file(name: &str) -> String { + files() + .into_iter() + .find(|(n, _)| n == name) + .map(|(_, c)| c) + .unwrap_or_else(|| panic!("no {name}")) +} + +fn bindings() -> String { + file("bindings.go") +} + +#[track_caller] +fn assert_has(out: &str, needle: &str) { + assert!(out.contains(needle), "missing {needle:?} in:\n{out}"); +} + +#[test] +fn names_come_from_the_identity() { + let files = files(); + let names: Vec<&str> = files.iter().map(|(n, _)| n.as_str()).collect(); + assert_eq!( + names, + [ + "go.mod", + "README.md", + "shop_kit.h", + "bindings.go", + "runtime.go", + "codec.go" + ] + ); + assert_has(&files[0].1, "module shop_kit\n"); + let src = &files[3].1; + assert_has( + src, + "// Package shop_kit binds the shop_kit native library.\npackage shop_kit\n", + ); + assert_has(src, "#cgo LDFLAGS: -lshop_kit\n"); + assert_has(src, "#include \"shop_kit.h\"\n"); + for (name, contents) in &files { + if name.ends_with(".go") || name == "go.mod" { + let body: String = contents.lines().skip(3).collect::>().join("\n"); + assert!( + !body.to_lowercase().contains("weaveffi"), + "{name} is branded:\n{body}" + ); + assert!(!body.contains("{{"), "{name} has a placeholder:\n{body}"); + } + } + + let custom = render_files( + &model(), + &GoConfig { + name: Some("example.com/shop".into()), + }, + ); + assert_has(&custom[0].1, "module example.com/shop\n"); +} + +#[test] +fn load_checks_embed_every_contract_entry() { + let model = model(); + let src = bindings(); + assert_has( + &src, + "func init() {\n\twvCheckABI()\n\tvar n C.size_t\n\ttable := C.shop_kit_shop_contract(&n)\n\twvCheckContract(table, n, []wvContractEntry{\n", + ); + let root = model.roots().next().unwrap(); + for e in entries(&model, root) { + assert_has( + &src, + &format!( + "\t\t{{{:#018x}, {:#018x}, \"{}\"}},\n", + e.id, e.hash, e.path + ), + ); + } + let rt = file("runtime.go"); + assert_has(&rt, "const wvABIVersion uint32 = 4\n"); + assert_has(&rt, "%s is missing from the library"); + assert_has(&rt, "%s changed since these bindings were generated"); + for f in files() { + assert!(!f.1.contains("checksum"), "{} mentions a checksum", f.0); + assert!(!f.1.contains("_dealloc"), "{} mentions dealloc", f.0); + } +} + +#[test] +fn vtables_carry_the_header_and_every_method() { + let src = bindings(); + assert_has( + &src, + "static const shop_kit_shop_Watcher_vtable wvVtable_shop_kit_shop_Watcher_vtable = {\n\t.size = sizeof(shop_kit_shop_Watcher_vtable),\n\t.flags = 0,\n\t.free = goWv_shop_kit_shop_Watcher_free,\n", + ); + assert_has( + &src, + "\t.on_event = (bool (*)(void*, const uint8_t*, size_t, int32_t, const uint8_t*, size_t, shop_kit_shop_Cart*, shop_kit_error*))goWv_shop_kit_shop_Watcher_on_event,\n", + ); + assert_has(&src, "\t.label = goWv_shop_kit_shop_Watcher_label,\n"); + assert_has( + &src, + "extern void goWv_shop_kit_shop_Watcher_label(void* ctx, uint8_t** out_ptr, size_t* out_len, shop_kit_error* out_err);", + ); + assert_has( + &src, + "extern shop_kit_shop_Cart* goWv_shop_kit_shop_Watcher_favorite(void* ctx, shop_kit_error* out_err);", + ); +} + +#[test] +fn callback_methods_return_every_family() { + let src = bindings(); + assert_has( + &src, + "\tOnEvent(name string, count int32, order Order, cart *Cart) bool\n", + ); + assert_has(&src, "\tMood() Mood\n"); + assert_has(&src, "\tLabel() (string, error)\n"); + assert_has(&src, "\tLatest() *Order\n"); + assert_has(&src, "\tFavorite() *Cart\n"); + assert_has(&src, "\tMaybe() *Cart\n"); + assert_has(&src, "\tOnDone()\n"); + + // Arguments: borrowed runs are copied or decoded; an object is adopted. + assert_has( + &src, + "ret := wvCallback[Watcher](ctx).OnEvent(wvBorrowString(name_ptr, name_len), int32(count), wvDecodeBorrowed(order_ptr, order_len, wvReadOrder), wvAdoptCart(cart))\n\treturn C._Bool(ret)\n", + ); + // Direct values return by value, objects as a fresh reference, and runs + // through the out slots as a library allocation. + assert_has(&src, "(cRet C.shop_kit_shop_Mood) {"); + assert_has(&src, "return C.shop_kit_shop_Mood(ret)"); + assert_has(&src, "wvHandOverString(ret, out_ptr, out_len)"); + assert_has( + &src, + "wvHandOverBytes(wvEncode(ret, wvWriteOptOrder), out_ptr, out_len)", + ); + assert_has(&src, "(cRet *C.shop_kit_shop_Cart) {"); + assert_has(&src, "return ret.share()"); + let rt = file("runtime.go"); + assert_has(&rt, "run := C.shop_kit_alloc(n)"); + assert_has(&src, "defer wvRecoverCallback(out_err)"); +} + +#[test] +fn throwing_callbacks_report_domain_codes_and_payloads() { + let src = bindings(); + assert_has( + &src, + "ret, err := wvCallback[Watcher](ctx).Label()\n\tif err != nil {\n\t\twvCallbackFailed[ShopError](out_err, err)\n\t\treturn\n\t}\n", + ); + assert_has( + &src, + "func (e *OutOfStockError) wvPayload() []byte {\n\tw := &wvWriter{}\n\tw.writeString(e.Sku)\n\treturn w.buf\n}", + ); + let rt = file("runtime.go"); + assert_has(&rt, "C.shop_kit_error_set_payload(outErr, ptr, n)"); + assert_has(&rt, "wvSetError(outErr, wvCodeForeign, err.Error(), nil)"); +} + +#[test] +fn callback_parameters_pass_a_handle_and_the_static_vtable() { + let src = bindings(); + assert_has( + &src, + "C.shop_kit_shop_watch(wvNewCallback(watcher), C.wvVtablePtr_shop_kit_shop_Watcher_vtable(), &cErr)", + ); + // An optional callback passes a null vtable for nil. + assert_has( + &src, + "\tvar cWatcherCtx unsafe.Pointer\n\tvar cWatcherVtable *C.shop_kit_shop_Watcher_vtable\n\tif watcher != nil {\n\t\tcWatcherCtx, cWatcherVtable = wvNewCallback(watcher), C.wvVtablePtr_shop_kit_shop_Watcher_vtable()\n\t}\n", + ); + assert_has( + &src, + "func goWv_shop_kit_shop_Watcher_free(ctx unsafe.Pointer) {\n\twvFreeCallback(ctx)\n}", + ); +} + +#[test] +fn errors_map_to_code_types() { + let src = bindings(); + assert_has( + &src, + "type ShopError interface {\n\terror\n\t// Code returns the error's numeric code.\n\tCode() int32\n\tisShopError()\n}", + ); + assert_has(&src, "type OutOfStockError struct {\n\tSku string\n"); + assert_has(&src, "func (*ClosedError) Code() int32 {\n\treturn 2\n}"); + assert_has(&src, "func (*ClosedError) isShopError() {}"); + assert_has(&src, "func wvShopError(f wvFailure) error {"); + assert_has(&src, "\t\te.Sku = r.readString()\n"); + // A throwing call returns the mapped error; any other call traps. + assert_has(&src, "func CartRestore(id int64) (*Cart, error) {"); + assert_has(&src, "return nil, wvShopError(wvTakeError(&cErr))"); + assert_has(&src, "func Echo(text string) string {"); + assert_has( + &src, + "\twvTrap(&cErr)\n\treturn wvTakeString(cRet, cRetLen)\n", + ); +} + +#[test] +fn composite_codecs_are_rendered_once_per_type() { + let src = bindings(); + for stem in [ + "MapStringListOptI32", + "ListOptI32", + "OptI32", + "ListOrder", + "ListCart", + "OptString", + "OptOrder", + ] { + let def = format!("func wvWrite{stem}(w *wvWriter"); + assert_eq!(src.matches(&def).count(), 1, "{def}"); + assert_eq!( + src.matches(&format!("func wvRead{stem}(r *wvReader)")) + .count(), + 1, + "{stem}" + ); + } + assert_has( + &src, + "func wvWriteMapStringListOptI32(w *wvWriter, v map[string][]*int32) {\n\twvWriteMap(w, v, (*wvWriter).writeString, wvWriteListOptI32)\n}", + ); + assert_has( + &src, + "func wvReadOptOrder(r *wvReader) *Order {\n\treturn wvReadOptional(r, wvReadOrder)\n}", + ); + assert_has( + &src, + "cCountsPtr, cCountsLen := wvBytes(wvEncode(counts, wvWriteMapStringListOptI32))", + ); + assert_has(&src, "return wvDecode(cRet, cRetLen, wvReadListOrder), nil"); + // Records read and write their fields through the shared pairs. + assert_has(&src, "\tOrderID uint64\n"); + assert_has( + &src, + "\twvWriteListCart(w, v.History)\n\twvWriteOptString(w, v.Note)\n", + ); +} + +#[test] +fn objects_cross_as_borrowed_pointers_and_fresh_tokens() { + let src = bindings(); + assert_has( + &src, + "\tvar cCurrent *C.shop_kit_shop_Cart\n\tif current != nil {\n\t\tcCurrent = current.native()\n\t\tdefer current.ref.release()\n\t}\n", + ); + assert_has(&src, "return wvAdoptCart(cRet)"); + assert_has( + &src, + "func wvWriteCart(w *wvWriter, v *Cart) {\n\tw.writeU64(uint64(uintptr(unsafe.Pointer(v.share()))))\n}", + ); + assert_has(&src, "return C.shop_kit_shop_Cart_clone(ptr)"); + assert_has(&src, "runtime.SetFinalizer(s, (*Cart).Close)"); +} + +#[test] +fn async_calls_block_on_a_context() { + let src = bindings(); + assert_has( + &src, + "func Wait(ctx context.Context, ms int64) (int64, error) {", + ); + assert_has(&src, "token := C.shop_kit_cancel_token_create()"); + assert_has(&src, "defer C.shop_kit_cancel_token_destroy(token)"); + assert_has(&src, "call := wvNewAsyncCall[int64]()"); + assert_has(&src, "res, fail, err := call.wait(ctx, token)"); + assert_has( + &src, + "wvComplete(context_, err_, func() int64 {\n\t\treturn int64(result)\n\t})", + ); + // A non-throwing async failure is a bug: it panics. + assert_has(&src, "\t\tpanic(fail.err())\n"); +} + +#[test] +fn iterators_are_lazy_sequences() { + let src = bindings(); + assert_has(&src, "func AllCarts() iter.Seq[*Cart] {"); + assert_has( + &src, + "defer C.shop_kit_shop_AllCartsIterator_destroy(cIter)", + ); + assert_has(&src, "if !yield(wvAdoptCart(cItem)) {"); +} + +#[test] +fn output_is_deterministic() { + assert_eq!(files(), files()); +} diff --git a/crates/weaveffi-cli/src/targets/go/types.rs b/crates/weaveffi-cli/src/targets/go/types.rs new file mode 100644 index 00000000..c12fa6c4 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/go/types.rs @@ -0,0 +1,139 @@ +//! Go type mapping: how resolved types, zero values, scalar conversions, +//! and C ABI slot types are spelled in the generated package. + +use weaveffi_model::abi::{CType, ConstPos}; +use weaveffi_model::ty::{Prim, Ty}; + +use crate::targets::go::names::pascal; + +/// The Go type spelling of a resolved type. +/// +/// Records are value structs, rich enums sealed interfaces, interfaces +/// wrapper pointers (one Go value owns each strong reference), and callback +/// interfaces Go interfaces. `T?` is `*T` unless `T` is already nil-able +/// (see [`optional_derefs`]). +pub(crate) fn go_type(ty: &Ty) -> String { + match ty { + Ty::Prim(p) => prim_type(*p).into(), + Ty::Record(n) | Ty::RichEnum(n) | Ty::Enum(n) | Ty::CallbackInterface(n) => pascal(n), + Ty::Interface(n) => format!("*{}", pascal(n)), + Ty::Optional(inner) if optional_derefs(inner) => format!("*{}", go_type(inner)), + Ty::Optional(inner) => go_type(inner), + Ty::List(inner) => format!("[]{}", go_type(inner)), + Ty::Map(k, v) => format!("map[{}]{}", go_type(k), go_type(v)), + // A throwing iterator wrapper spells `iter.Seq2[T, error]` at its + // signature site instead. + Ty::Iterator(inner) => format!("iter.Seq[{}]", go_type(inner)), + } +} + +/// The Go type of a primitive. +fn prim_type(p: Prim) -> &'static str { + match p { + Prim::I8 => "int8", + Prim::I16 => "int16", + Prim::I32 => "int32", + Prim::I64 => "int64", + Prim::U8 => "uint8", + Prim::U16 => "uint16", + Prim::U32 => "uint32", + Prim::U64 => "uint64", + Prim::F32 => "float32", + Prim::F64 => "float64", + Prim::Bool => "bool", + Prim::String => "string", + Prim::Bytes => "[]byte", + } +} + +/// `true` when `T?` surfaces as `*T` in Go. Types that are already +/// nil-able (rich enums, slices, maps, byte slices, wrapper pointers, and +/// callback interfaces) use nil as the none marker instead. +pub(crate) fn optional_derefs(inner: &Ty) -> bool { + !matches!( + inner, + Ty::RichEnum(_) + | Ty::List(_) + | Ty::Map(_, _) + | Ty::Prim(Prim::Bytes) + | Ty::Interface(_) + | Ty::CallbackInterface(_) + ) +} + +/// The Go zero-value expression of a type, returned on an error path. +pub(crate) fn go_zero(ty: &Ty) -> String { + match ty { + Ty::Prim(Prim::Bool) => "false".into(), + Ty::Prim(Prim::String) => "\"\"".into(), + Ty::Prim(Prim::Bytes) => "nil".into(), + Ty::Prim(_) | Ty::Enum(_) => "0".into(), + Ty::Record(n) => format!("{}{{}}", pascal(n)), + _ => "nil".into(), + } +} + +/// The Go expression converting the Go value `expr` into the by-value C +/// slot of type `slot` (a scalar, `bool`, or C-style enum). +pub(crate) fn to_c_direct(expr: &str, slot: &CType, prefix: &str) -> String { + format!("{}({expr})", cgo_type(slot, prefix)) +} + +/// The Go expression converting the by-value C slot `expr` back into the +/// Go value of type `ty`. +pub(crate) fn from_c_direct(expr: &str, ty: &Ty) -> String { + format!("{}({expr})", go_type(ty)) +} + +/// The cgo spelling of one C ABI type: `C.int32_t`, `*C.uint8_t`, +/// `**C.uint8_t`, `*C.kv_kv_Store`, and `unsafe.Pointer` for `void*`. +pub(crate) fn cgo_type(ct: &CType, prefix: &str) -> String { + match ct { + CType::Int8 => "C.int8_t".into(), + CType::Int16 => "C.int16_t".into(), + CType::Int32 => "C.int32_t".into(), + CType::Uint8 => "C.uint8_t".into(), + CType::Uint16 => "C.uint16_t".into(), + CType::Uint32 => "C.uint32_t".into(), + CType::Int64 => "C.int64_t".into(), + CType::Uint64 => "C.uint64_t".into(), + CType::Float => "C.float".into(), + CType::Double => "C.double".into(), + CType::Bool => "C._Bool".into(), + CType::Size => "C.size_t".into(), + CType::Char => "C.char".into(), + CType::Ptr { pointee, .. } if **pointee == CType::Void => "unsafe.Pointer".into(), + CType::Ptr { pointee, .. } => format!("*{}", cgo_type(pointee, prefix)), + CType::Void => unreachable!("a void slot is never spelled in Go"), + named => format!("C.{}", named.render_c(prefix)), + } +} + +/// `ct` with every `const` qualifier dropped, matching the const-free +/// prototypes cgo writes into `_cgo_export.h` for exported Go functions. +pub(crate) fn strip_const(ct: &CType) -> CType { + match ct { + CType::Ptr { pointee, .. } => CType::Ptr { + konst: ConstPos::None, + pointee: Box::new(strip_const(pointee)), + }, + other => other.clone(), + } +} + +/// Quote `s` as a Go string literal. +pub(crate) fn go_str(s: &str) -> String { + let mut out = String::with_capacity(s.len() + 2); + out.push('"'); + for ch in s.chars() { + match ch { + '\\' => out.push_str("\\\\"), + '"' => out.push_str("\\\""), + '\n' => out.push_str("\\n"), + '\t' => out.push_str("\\t"), + _ => out.push(ch), + } + } + out.push('"'); + out +} diff --git a/crates/weaveffi-gen/src/targets/js/api.rs b/crates/weaveffi-cli/src/targets/js/api.rs similarity index 60% rename from crates/weaveffi-gen/src/targets/js/api.rs rename to crates/weaveffi-cli/src/targets/js/api.rs index 1e05efbf..142e4c7c 100644 --- a/crates/weaveffi-gen/src/targets/js/api.rs +++ b/crates/weaveffi-cli/src/targets/js/api.rs @@ -9,37 +9,48 @@ //! direct value as a `number`, `bigint`, or `boolean`; a string as a //! `string`; bytes and value buffers as a `Uint8Array`; an object as its //! handle (`null` for an absent `Interface?`); a callback interface as an -//! adapter object whose methods, named as in the IDL, take and return raw -//! values; and, last, a cancellable call's cancel token handle (or -//! `null`). +//! adapter object (`null` for an absent `Cb?`); and, last, a cancellable +//! call's cancel token handle (or `null`). //! * **Results** come back the same way: a direct value, a `string`, a //! `Uint8Array`, or an object handle (one strong reference). An async //! launcher returns a `Promise` of its result; an iterator launcher //! returns a handle, and the iterator's `next` returns an element or -//! `undefined` when the iterator is exhausted. +//! `undefined` when the iterator is exhausted. A module's contract +//! function (`{prefix}_{module}_contract`) returns its table as a +//! `BigUint64Array` of `id, hash` pairs. //! * **Failures** throw (or reject with) a runtime `$Fault`. +//! * **Callback adapters** have one method per callback method, named as in +//! the IDL, taking raw arguments (objects as adopted handles) and +//! returning the raw value of the method's return: a direct value, a +//! `string`, a `Uint8Array` (bytes or an encoded buffer, which the +//! transport copies into a run from `{prefix}_alloc`), or an object handle +//! that is a fresh strong reference (`null` for none). An adapter method +//! that throws a `$Fault` reports its code, message, and payload; any +//! other exception is reported as code -4 with its message. //! //! The transport also defines `$token`, which turns an object token read //! from a value buffer into a handle. +use std::collections::BTreeSet; + use weaveffi_model::model::{ - BindingModel, CallShape, CallbackInterfaceBinding, ErrorBinding, FnBinding, InterfaceBinding, - ModuleBinding, ParamBinding, Ty, + CallShape, CallbackInterfaceBinding, CallbackMethodBinding, ErrorBinding, FnBinding, + InterfaceBinding, Model, ModuleBinding, ParamBinding, }; -use weaveffi_model::plan::{self, ArgPass, RetPass}; +use weaveffi_model::plan::{ArgPass, RetPass}; +use weaveffi_model::ty::Ty; use crate::codegen::CodeWriter; -use crate::targets::js::codec::{emit_codecs, read_expr, reader_fn, writer_fn}; +use crate::targets::js::codec::{emit_codecs, read_expr, reader_fn, write_expr, writer_fn}; use crate::targets::js::names::{ - callback_method_name, code_class, decl, direct_prim, error_mapper, fn_name, helper, js_string, - member_name, module_name, param_name, type_decl, + callback_method_name, code_class, decl, direct_prim, error_mapper, error_owner, fn_name, + helper, js_string, member_name, module_name, param_name, type_decl, }; /// Render the shared API section of `index.js`: everything after the /// transport has defined `$raw` and `$token`. -pub(crate) fn render_api(model: &BindingModel) -> String { - let mut w = CodeWriter::two_space(); - let prefix = &model.prefix; +pub(crate) fn render_api(w: &mut CodeWriter, model: &Model) { + let prefix = model.prefix(); if model.callables().any(|(_, f)| f.cancellable) { w.line("// The native cancel tokens behind `AbortSignal`s."); @@ -58,18 +69,20 @@ pub(crate) fn render_api(model: &BindingModel) -> String { } w.line("/**"); - w.line(" * The native library's live-allocation counters (0 objects, 1 callbacks,"); - w.line(" * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always"); - w.line(" * zero unless the library enables its `leak-check` feature."); + w.line(" * The native library's live-resource counters (0 objects, 1 callbacks,"); + w.line(" * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is"); + w.line(" * `1n` when the library counts at all (its `leak-check` feature), else"); + w.line(" * every kind is `0n`."); w.line(" */"); w.block("export function __debugLive(kind) {", "}", |w| { w.line(format!("return $raw.{prefix}_debug_live(kind);")); }); w.blank(); + let raised = raised_domains(model); for m in &model.modules { - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { - emit_error_domain(&mut w, m, eb); + if let Some(eb) = m.errors.as_ref() { + emit_error_domain(w, model, m, eb, &raised); } for e in m.enums.iter().filter(|e| !e.rich) { w.block( @@ -80,7 +93,13 @@ pub(crate) fn render_api(model: &BindingModel) -> String { w.line(format!("{}: {},", v.name, v.value)); } for v in &e.variants { - w.line(format!("{}: {},", v.value, js_string(&v.name))); + // A negative number isn't a property name literal. + let key = if v.value < 0 { + js_string(&v.value.to_string()) + } else { + v.value.to_string() + }; + w.line(format!("{key}: {},", js_string(&v.name))); } }, ); @@ -88,25 +107,22 @@ pub(crate) fn render_api(model: &BindingModel) -> String { } } - if model.has_buffers() { - let mut codecs = CodeWriter::two_space(); - emit_codecs(&mut codecs, model); - if !codecs.is_empty() { - w.raw(codecs.finish()); - w.blank(); - } + let before = w.as_str().len(); + emit_codecs(w, model); + if w.as_str().len() != before { + w.blank(); } for m in &model.modules { for cb in &m.callback_interfaces { - emit_adapter(&mut w, m, cb, prefix); + emit_adapter(w, model, m, cb); } for i in &m.interfaces { - emit_class(&mut w, model, m, i); + emit_class(w, model, m, i); } for f in &m.functions { - emit_iterator_spec(&mut w, model, m, f, &decl(&m.segments, &f.name)); - let is_async = if f.is_async { "async " } else { "" }; + emit_iterator_spec(w, model, m, f, &decl(&m.segments, &f.name)); + let is_async = if f.is_async() { "async " } else { "" }; let mut params = js_params(f); if f.cancellable { params.push("$options".into()); @@ -128,7 +144,7 @@ pub(crate) fn render_api(model: &BindingModel) -> String { None, &decl(&m.segments, &f.name), Finish::Return, - ) + ); }, ); w.blank(); @@ -136,15 +152,9 @@ pub(crate) fn render_api(model: &BindingModel) -> String { } for m in model.roots() { - emit_namespace( - &mut w, - model, - m, - &format!("export const {} = ", module_name(m)), - ); + emit_namespace(w, model, m, &format!("export const {} = ", module_name(m))); w.blank(); } - w.finish() } /// The JavaScript parameter names of a callable. @@ -152,9 +162,33 @@ fn js_params(f: &FnBinding) -> Vec { f.params.iter().map(|p| param_name(&p.name)).collect() } -/// Emit one error domain: the domain class, one class per code, and the +/// The declaration keys (`kv$KvError`) of the error domains some callback +/// method may raise, which need a `$raise$...` mapper. +fn raised_domains(model: &Model) -> BTreeSet { + model + .callback_interfaces() + .filter(|(_, cb)| cb.methods.iter().any(|m| m.throws)) + .filter_map(|(m, _)| model.error_domain(m)) + .map(|eb| { + let owner = error_owner(&model.modules, eb); + decl(&owner.segments, &eb.name) + }) + .collect() +} + +/// Emit one error domain: the domain class, one class per code (whose +/// constructor takes the code's fields first, when it has any), and the /// mapper (`$from$kv$KvError`) its throwing callables route faults through. -fn emit_error_domain(w: &mut CodeWriter, m: &ModuleBinding, eb: &ErrorBinding) { +/// When a callback method may raise the domain, also its fields' writers +/// and the mapper (`$raise$kv$KvError`) that turns a raised error into the +/// fault the transport reports. +fn emit_error_domain( + w: &mut CodeWriter, + model: &Model, + m: &ModuleBinding, + eb: &ErrorBinding, + raised: &BTreeSet, +) { let domain = decl(&m.segments, &eb.type_name); let key = decl(&m.segments, &eb.name); w.line(format!( @@ -171,13 +205,23 @@ fn emit_error_domain(w: &mut CodeWriter, m: &ModuleBinding, eb: &ErrorBinding) { "};", |w| { w.line(format!("static CODE = {};", c.value)); - w.block( - format!("constructor(message = {}) {{", js_string(&c.message)), - "}", - |w| { + let message = js_string(&c.message); + if c.fields.is_empty() { + w.block(format!("constructor(message = {message}) {{"), "}", |w| { w.line(format!("super({}, message);", c.value)); - }, - ); + }); + } else { + w.block( + format!("constructor(fields, message = {message}) {{"), + "}", + |w| { + w.line(format!("super({}, message);", c.value)); + for f in &c.fields { + w.line(format!("this.{0} = fields.{0};", f.name)); + } + }, + ); + } }, ); } @@ -186,60 +230,89 @@ fn emit_error_domain(w: &mut CodeWriter, m: &ModuleBinding, eb: &ErrorBinding) { .iter() .map(|c| format!("[{}, {}]", c.value, decl(&m.segments, &code_class(&c.name)))) .collect(); - w.line(format!( - "const $codes${key} = new Map([{}]);", - codes.join(", ") - )); - let payloads: Vec = eb - .codes - .iter() - .filter(|c| !c.fields.is_empty()) + emit_map(w, &format!("$codes${key}"), &codes); + let fielded = || eb.codes.iter().filter(|c| !c.fields.is_empty()); + let payloads: Vec = fielded() .map(|c| { let fields: Vec = c .fields .iter() - .map(|f| format!("{}: {}", f.name, read_expr(&f.ty))) + .map(|f| format!("{}: {}", f.name, read_expr(model, &f.ty))) .collect(); format!("[{}, (r) => ({{ {} }})]", c.value, fields.join(", ")) }) .collect(); - if payloads.is_empty() { - w.block(format!("function $from${key}(e) {{"), "}", |w| { - w.line(format!("return $domain(e, $codes${key});")); - }); - } else { - w.line(format!( - "const $payloads${key} = new Map([{}]);", - payloads.join(", ") - )); - w.block(format!("function $from${key}(e) {{"), "}", |w| { - w.line(format!("return $domain(e, $codes${key}, $payloads${key});")); + emit_map(w, &format!("$payloads${key}"), &payloads); + w.block(format!("function $from${key}(e) {{"), "}", |w| { + w.line(format!("return $domain(e, $codes${key}, $payloads${key});")); + }); + if raised.contains(&key) { + let writers: Vec = fielded() + .map(|c| { + let fields: Vec = c + .fields + .iter() + .map(|f| format!("{};", write_expr(model, &f.ty, &format!("e.{}", f.name)))) + .collect(); + format!("[{}, (w, e) => {{ {} }}]", c.value, fields.join(" ")) + }) + .collect(); + emit_map(w, &format!("$fields${key}"), &writers); + w.block(format!("function $raise${key}(e) {{"), "}", |w| { + w.line(format!( + "return $raise(e, {domain}, $codes${key}, $fields${key});" + )); }); } w.blank(); } +/// Emit `const {name} = new Map([...]);`, one entry (`[key, value]`) per +/// line. +fn emit_map(w: &mut CodeWriter, name: &str, entries: &[String]) { + if entries.is_empty() { + w.line(format!("const {name} = new Map();")); + return; + } + w.block(format!("const {name} = new Map(["), "]);", |w| { + for e in entries { + w.line(format!("{e},")); + } + }); +} + /// Emit the adapter for one callback interface: it checks the consumer's /// implementation and returns the raw object the transport calls, whose /// methods (named as in the IDL) convert raw arguments into idiomatic values -/// (decoding buffers, adopting objects), call the implementation, and check -/// the return value. A thrown exception reaches the transport, which -/// reports it to the producer as a code -4 failure. +/// (decoding buffers, adopting objects), call the implementation, and turn +/// the return value into its raw form (checking direct values, strings, and +/// bytes, encoding buffers, cloning objects). An exception reaches the +/// transport, which reports it to the producer as a code -4 failure, except +/// that a method declared `throws` first maps its module's domain errors +/// onto the `$Fault` that carries their code and fields. fn emit_adapter( w: &mut CodeWriter, + model: &Model, m: &ModuleBinding, cb: &CallbackInterfaceBinding, - prefix: &str, ) { - let protocol = cb.protocol(prefix); - let dotted = format!("{}.{}", m.dot_path, cb.name); + let protocol = cb.protocol(); + let raise = model.error_domain(m).map(|eb| { + let owner = error_owner(&model.modules, eb); + format!("$raise${}", decl(&owner.segments, &eb.name)) + }); w.block( - format!("function {}(impl) {{", helper("adapt", &dotted)), + format!("function {}(impl) {{", helper(model, "adapt", &cb.name)), "}", |w| { w.line(format!("$impl(impl, {});", js_string(&cb.name))); w.block("return {", "};", |w| { - for (method, passes) in cb.methods.iter().zip(&protocol.method_args) { + for ((method, passes), ret) in cb + .methods + .iter() + .zip(&protocol.method_args) + .zip(&protocol.method_returns) + { let raw: Vec = (0..method.params.len()).map(|i| format!("$a{i}")).collect(); let args: Vec = method @@ -247,30 +320,31 @@ fn emit_adapter( .iter() .zip(passes) .zip(&raw) - .map(|((p, pass), a)| receive(&p.ty, pass, a)) + .map(|((p, pass), a)| receive(model, &p.ty, pass, a)) .collect(); let call = format!( "impl.{}({})", callback_method_name(&method.name), args.join(", ") ); + let stmt = callback_return(model, cb, method, ret, &call); w.block( format!("{}({}) {{", method.name, raw.join(", ")), "},", - |w| match &method.ret { - None => { - w.line(format!("{call};")); + |w| match raise.as_deref().filter(|_| method.throws) { + Some(raise) => { + w.line("try {"); + w.scope(|w| { + w.line(&stmt); + }); + w.line("} catch ($e) {"); + w.scope(|w| { + w.line(format!("throw {raise}($e);")); + }); + w.line("}"); } - Some(ty) => { - let where_ = js_string(&format!( - "{}.{}", - cb.name, - callback_method_name(&method.name) - )); - w.line(format!( - "return $ret.{}({call}, {where_});", - direct_prim(ty).pascal() - )); + None => { + w.line(&stmt); } }, ); @@ -281,14 +355,55 @@ fn emit_adapter( w.blank(); } +/// The statement an adapter method ends with: the implementation's `call`, +/// with its return value turned into the raw form the transport hands the +/// producer. +fn callback_return( + model: &Model, + cb: &CallbackInterfaceBinding, + method: &CallbackMethodBinding, + pass: &RetPass, + call: &str, +) -> String { + let where_ = || { + js_string(&format!( + "{}.{}", + cb.name, + callback_method_name(&method.name) + )) + }; + let Some(ty) = &method.ret else { + return format!("{call};"); + }; + let value = match pass { + RetPass::Void => unreachable!("a method with a return type returns a value"), + RetPass::Direct => format!("$ret.{}({call}, {})", direct_prim(ty).pascal(), where_()), + RetPass::String => format!("$ret.String({call}, {})", where_()), + RetPass::Bytes => format!("$ret.Bytes({call}, {})", where_()), + RetPass::Buffer => format!("$encode({call}, {})", writer_fn(model, ty)), + RetPass::Object { nullable } => { + let cls = type_decl( + model, + ty.interface_name().expect("object type names an interface"), + ); + let clone = if *nullable { "$cloneOpt" } else { "$clone" }; + format!("{clone}({call}, {cls})") + } + }; + format!("return {value};") +} + /// The idiomatic value of a raw value `raw` of type `ty` received from the /// native side per `pass`: decoded when buffered, adopted when an object, /// unchanged otherwise. -fn receive(ty: &Ty, pass: &RetPass, raw: &str) -> String { +fn receive(model: &Model, ty: &Ty, pass: &RetPass, raw: &str) -> String { match pass { - RetPass::Buffer => format!("$decode({raw}, {})", reader_fn(ty)), - RetPass::Object { nullable, .. } => { - let cls = type_decl(ty.interface_name().expect("object type names an interface")); + RetPass::Buffer => format!("$decode({raw}, {})", reader_fn(model, ty)), + RetPass::Object { nullable } => { + let cls = type_decl( + model, + ty.interface_name().expect("object type names an interface"), + ); if *nullable { format!("$adoptOpt({cls}, {raw})") } else { @@ -304,7 +419,7 @@ fn receive(ty: &Ty, pass: &RetPass, raw: &str) -> String { /// conversion its `$Iterator`s use. fn emit_iterator_spec( w: &mut CodeWriter, - model: &BindingModel, + model: &Model, m: &ModuleBinding, f: &FnBinding, key: &str, @@ -312,8 +427,8 @@ fn emit_iterator_spec( let CallShape::Iterator(it) = &f.shape else { return; }; - let protocol = it.protocol(f, &model.prefix); - let convert = match receive(&it.elem, &protocol.elem, "v") { + let protocol = it.protocol(f); + let convert = match receive(model, &it.elem, &protocol.elem, "v") { v if v == "v" => "null".to_string(), other => format!("(v) => {other}"), }; @@ -322,7 +437,7 @@ fn emit_iterator_spec( w.line(format!("destroy: (h) => $raw.{}(h),", it.destroy_symbol)); w.line(format!( "map: {},", - error_mapper(&model.modules, f, m.error.as_ref()) + error_mapper(&model.modules, f, model.error_domain(m)) )); w.line(format!("convert: {convert},")); }); @@ -333,12 +448,12 @@ fn emit_iterator_spec( /// synchronous `new` constructor as the JavaScript constructor and every /// other constructor as a static factory, and gives the runtime its native /// `$destroy` and `$clone`. -fn emit_class(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, i: &InterfaceBinding) { +fn emit_class(w: &mut CodeWriter, model: &Model, m: &ModuleBinding, i: &InterfaceBinding) { let cls = decl(&m.segments, &i.name); let canonical = i .constructors .iter() - .find(|c| c.name == "new" && !c.is_async); + .find(|c| c.name == "new" && !c.is_async()); for f in i.constructors.iter().chain(&i.methods).chain(&i.statics) { emit_iterator_spec(w, model, m, f, &format!("{cls}${}", f.name)); } @@ -391,7 +506,7 @@ fn emit_class(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, i: &I let head = format!( "{}{}{}({}) {{", if is_static { "static " } else { "" }, - if f.is_async { "async " } else { "" }, + if f.is_async() { "async " } else { "" }, member_name(&f.name, is_static), params.join(", ") ); @@ -428,7 +543,7 @@ struct Loan { /// method's receiver; `key` names the callable's iterator spec. fn emit_body( w: &mut CodeWriter, - model: &BindingModel, + model: &Model, m: &ModuleBinding, f: &FnBinding, recv: Option<&str>, @@ -447,7 +562,7 @@ fn emit_body( args.push("$self".into()); } for p in &f.params { - args.push(arg_expr(p, &mut loans)); + args.push(arg_expr(model, p, &mut loans)); } let symbol = match &f.shape { CallShape::Sync(abi) => &abi.symbol, @@ -460,7 +575,7 @@ fn emit_body( "await $cancellable($options?.signal, $tokens, ($t) => $raw.{symbol}({}))", args.join(", ") ) - } else if f.is_async { + } else if f.is_async() { format!("await $raw.{symbol}({})", args.join(", ")) } else { format!("$raw.{symbol}({})", args.join(", ")) @@ -468,15 +583,15 @@ fn emit_body( let stmt = match (&f.shape, finish) { (_, Finish::Own) => format!("$own(this, {call});"), (CallShape::Iterator(_), _) => format!("return new $Iterator({call}, $it${key});"), - _ => match plan::ret_pass(f.ret.as_ref(), &model.prefix) { + _ => match RetPass::of(f.ret.as_ref()) { RetPass::Void => format!("{call};"), pass => format!( "return {};", - receive(f.ret.as_ref().expect("non-void"), &pass, &call) + receive(model, f.ret.as_ref().expect("non-void"), &pass, &call) ), }, }; - let map = error_mapper(&model.modules, f, m.error.as_ref()); + let map = error_mapper(&model.modules, f, model.error_domain(m)); emit_loans(w, &loans, &stmt, &map); } @@ -525,13 +640,14 @@ fn emit_catch(w: &mut CodeWriter, map: &str) { /// The raw argument expression of one parameter, recording any object it /// lends. -fn arg_expr(p: &ParamBinding, loans: &mut Vec) -> String { +fn arg_expr(model: &Model, p: &ParamBinding, loans: &mut Vec) -> String { let name = param_name(&p.name); match p.arg_pass() { ArgPass::Direct { .. } | ArgPass::String { .. } | ArgPass::Bytes { .. } => name, - ArgPass::Buffer { .. } => format!("$encode({name}, {})", writer_fn(&p.ty)), + ArgPass::Buffer { .. } => format!("$encode({name}, {})", writer_fn(model, &p.ty)), ArgPass::Object { nullable, .. } => { let cls = type_decl( + model, p.ty.interface_name() .expect("object parameter names an interface"), ); @@ -544,11 +660,16 @@ fn arg_expr(p: &ParamBinding, loans: &mut Vec) -> String { }); local } - ArgPass::Callback { .. } => { + ArgPass::Callback { nullable, .. } => { let cb = p.ty.callback_interface_name() .expect("callback parameter names a callback interface"); - format!("{}({name})", helper("adapt", cb)) + let adapt = format!("{}({name})", helper(model, "adapt", cb)); + if nullable { + format!("{name} == null ? null : {adapt}") + } else { + adapt + } } } } @@ -556,10 +677,10 @@ fn arg_expr(p: &ParamBinding, loans: &mut Vec) -> String { /// Emit the frozen namespace object of `m` and, nested, its submodules. /// `lead` precedes the opening line (`export const kv = ` for a root, /// `stats: ` for a submodule). -fn emit_namespace(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, lead: &str) { +fn emit_namespace(w: &mut CodeWriter, model: &Model, m: &ModuleBinding, lead: &str) { w.line(format!("{lead}Object.freeze({{")); w.scope(|w| { - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { + if let Some(eb) = m.errors.as_ref() { w.line(format!( "{}: {},", eb.type_name, diff --git a/crates/weaveffi-cli/src/targets/js/codec.rs b/crates/weaveffi-cli/src/targets/js/codec.rs new file mode 100644 index 00000000..48bac80b --- /dev/null +++ b/crates/weaveffi-cli/src/targets/js/codec.rs @@ -0,0 +1,396 @@ +//! Value-buffer codec emitters. +//! +//! Records, rich enums, optionals, lists, maps, and error payloads cross the +//! ABI as value buffers. The runtime's `$Writer` and `$Reader` implement the +//! wire primitives (`writeI32`, `readString`, ...) and the optional, list, +//! and map combinators. This module emits one writer and one reader +//! function per type a buffer can hold beyond the primitives: +//! +//! * each record and rich enum (`$w$kv$Entry`, `$r$kv$Entry`), named by its +//! declaring module's path; +//! * each interface carried as an object token (`$w$kv$Store`, +//! `$r$kv$Store`): writing one clones the wrapper's native object +//! (`$clone`), and reading one adopts the reference into a new wrapper +//! (the transport defines `$token`, which turns the `u64` token into its +//! handle representation); +//! * each distinct optional, list, and map type anywhere in the API +//! (`$w_list_opt_Entry` writes `[Entry?]`), named after the type's shape +//! with bare type names, which are unique across the API. +//! +//! Call sites then name these functions instead of building closures. Every +//! dispatch goes through [`Ty::wire`], and primitives collapse onto one arm +//! through [`Prim::pascal`]. + +use std::collections::{BTreeMap, BTreeSet}; + +use weaveffi_model::model::{CallShape, Model}; +use weaveffi_model::ty::{Family, Ty, WireType}; + +use crate::codegen::CodeWriter; +use crate::targets::js::names::{helper, js_string, type_decl}; + +/// The name stem of a composite (optional, list, or map) type's codec +/// functions: `opt_Entry`, `list_opt_Entry`, `map_string_Store`. +fn mangle(ty: &Ty) -> String { + match ty.wire() { + WireType::Prim(p) => p.snake().to_string(), + WireType::Enum(n) | WireType::User(n) | WireType::Object(n) => n.to_string(), + WireType::Optional(inner) => format!("opt_{}", mangle(inner)), + WireType::List(inner) => format!("list_{}", mangle(inner)), + WireType::Map(k, v) => format!("map_{}_{}", mangle(k), mangle(v)), + } +} + +/// A function expression `(w, v) => ...` writing one value of `ty`. +pub(crate) fn writer_fn(model: &Model, ty: &Ty) -> String { + match ty.wire() { + WireType::Prim(p) => format!("$W.{}", p.pascal()), + WireType::Enum(_) => "$W.I32".into(), + WireType::User(n) | WireType::Object(n) => helper(model, "w", n), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("$w_{}", mangle(ty)) + } + } +} + +/// A function expression `(r) => value` reading one value of `ty`. +pub(crate) fn reader_fn(model: &Model, ty: &Ty) -> String { + match ty.wire() { + WireType::Prim(p) => format!("$R.{}", p.pascal()), + WireType::Enum(_) => "$R.I32".into(), + WireType::User(n) | WireType::Object(n) => helper(model, "r", n), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("$r_{}", mangle(ty)) + } + } +} + +/// The statement expression writing `val` of type `ty` to the writer `w`. +pub(crate) fn write_expr(model: &Model, ty: &Ty, val: &str) -> String { + match ty.wire() { + WireType::Prim(p) => format!("w.write{}({val})", p.pascal()), + WireType::Enum(_) => format!("w.writeI32({val})"), + _ => format!("{}(w, {val})", writer_fn(model, ty)), + } +} + +/// The expression reading one value of type `ty` from the reader `r`. +pub(crate) fn read_expr(model: &Model, ty: &Ty) -> String { + match ty.wire() { + WireType::Prim(p) => format!("r.read{}()", p.pascal()), + WireType::Enum(_) => "r.readI32()".into(), + _ => format!("{}(r)", reader_fn(model, ty)), + } +} + +/// The writer of one map key (see the runtime's `$WK`). +fn key_writer_fn(ty: &Ty) -> String { + match ty.wire() { + WireType::Prim(p) => format!("$WK.{}", p.pascal()), + WireType::Enum(_) => "$WK.I32".into(), + other => unreachable!("validation admits no {other:?} map keys"), + } +} + +/// Every type a value buffer in `model` can hold that needs codec functions +/// of its own beyond records and rich enums: the composites by name stem, +/// and the interfaces carried as object tokens. +#[derive(Default)] +struct Shapes<'a> { + composites: BTreeMap, + objects: BTreeSet<&'a str>, +} + +impl<'a> Shapes<'a> { + fn of(model: &'a Model) -> Self { + let mut shapes = Shapes::default(); + for m in &model.modules { + let fields = m + .structs + .iter() + .flat_map(|s| &s.fields) + .chain( + m.enums + .iter() + .flat_map(|e| e.variants.iter()) + .flat_map(|v| &v.fields), + ) + .chain( + m.errors + .iter() + .flat_map(|e| &e.codes) + .flat_map(|c| &c.fields), + ); + for f in fields { + shapes.visit(&f.ty); + } + for f in m.callables() { + for p in &f.params { + shapes.visit_slot(&p.ty); + } + match &f.shape { + CallShape::Iterator(it) => shapes.visit_slot(&it.elem), + _ => { + if let Some(ty) = &f.ret { + shapes.visit_slot(ty); + } + } + } + } + for method in m.callback_interfaces.iter().flat_map(|cb| &cb.methods) { + for p in &method.params { + shapes.visit_slot(&p.ty); + } + if let Some(ty) = &method.ret { + shapes.visit_slot(ty); + } + } + } + shapes + } + + /// A parameter, return, or element type: only buffers hold codecs. + fn visit_slot(&mut self, ty: &'a Ty) { + if ty.family() == Family::Buffer { + self.visit(ty); + } + } + + /// A type inside a value buffer. + fn visit(&mut self, ty: &'a Ty) { + match ty.wire() { + WireType::Prim(_) | WireType::Enum(_) | WireType::User(_) => {} + WireType::Object(n) => { + self.objects.insert(n); + } + WireType::Optional(inner) | WireType::List(inner) => { + self.composites.insert(mangle(ty), ty); + self.visit(inner); + } + WireType::Map(k, v) => { + self.composites.insert(mangle(ty), ty); + self.visit(k); + self.visit(v); + } + } + } +} + +/// Emit the writer and reader of every record, rich enum, object token, and +/// composite type in the model. +pub(crate) fn emit_codecs(w: &mut CodeWriter, model: &Model) { + let shapes = Shapes::of(model); + for m in &model.modules { + for s in &m.structs { + let name = &s.name; + w.block( + format!("function {}(w, v) {{", helper(model, "w", name)), + "}", + |w| { + for f in &s.fields { + w.line(format!( + "{};", + write_expr(model, &f.ty, &format!("v.{}", f.name)) + )); + } + }, + ); + w.block( + format!("function {}(r) {{", helper(model, "r", name)), + "}", + |w| { + if s.fields.is_empty() { + w.line("return {};"); + } else { + w.block("return {", "};", |w| { + for f in &s.fields { + w.line(format!("{}: {},", f.name, read_expr(model, &f.ty))); + } + }); + } + }, + ); + } + for e in m.enums.iter().filter(|e| e.rich) { + let name = &e.name; + let what = js_string(&format!("unknown {} tag: ", e.name)); + w.block( + format!("function {}(w, v) {{", helper(model, "w", name)), + "}", + |w| { + w.block("switch (v.tag) {", "}", |w| { + for v in &e.variants { + w.line(format!("case {}:", js_string(&v.name))); + w.scope(|w| { + w.line(format!("w.writeI32({});", v.value)); + for f in &v.fields { + w.line(format!( + "{};", + write_expr(model, &f.ty, &format!("v.{}", f.name)) + )); + } + w.line("return;"); + }); + } + w.line("default:"); + w.line(format!(" throw new TypeError({what} + (v && v.tag));")); + }); + }, + ); + w.block( + format!("function {}(r) {{", helper(model, "r", name)), + "}", + |w| { + w.line("const tag = r.readI32();"); + w.block("switch (tag) {", "}", |w| { + for v in &e.variants { + let fields: String = v + .fields + .iter() + .map(|f| format!(", {}: {}", f.name, read_expr(model, &f.ty))) + .collect(); + w.line(format!( + "case {}: return {{ tag: {}{fields} }};", + v.value, + js_string(&v.name) + )); + } + w.line("default:"); + w.line(format!(" throw new $Error(-3, {what} + tag);")); + }); + }, + ); + } + } + for name in &shapes.objects { + let cls = type_decl(model, name); + w.block( + format!("function {}(w, v) {{", helper(model, "w", name)), + "}", + |w| { + w.line(format!("w.writeU64(BigInt($clone(v, {cls})));")); + }, + ); + w.block( + format!("function {}(r) {{", helper(model, "r", name)), + "}", + |w| { + w.line(format!("return $adopt({cls}, $token(r.readU64()));")); + }, + ); + } + for ty in shapes.composites.values() { + let (write, read) = match ty.wire() { + WireType::Optional(inner) => ( + format!("w.writeOpt(v, {});", writer_fn(model, inner)), + format!("return r.readOpt({});", reader_fn(model, inner)), + ), + WireType::List(inner) => ( + format!("w.writeList(v, {});", writer_fn(model, inner)), + format!("return r.readList({});", reader_fn(model, inner)), + ), + WireType::Map(k, v) => ( + format!( + "w.writeMap(v, {}, {});", + key_writer_fn(k), + writer_fn(model, v) + ), + format!( + "return r.readMap({}, {});", + reader_fn(model, k), + reader_fn(model, v) + ), + ), + other => unreachable!("{other:?} is not a composite"), + }; + w.line(format!("/** `{ty}` */")); + w.block( + format!("function {}(w, v) {{", writer_fn(model, ty)), + "}", + |w| { + w.line(write); + }, + ); + w.block( + format!("function {}(r) {{", reader_fn(model, ty)), + "}", + |w| { + w.line(read); + }, + ); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::targets::js::names::tests::model; + use weaveffi_model::ty::Prim; + + #[test] + fn primitives_collapse_onto_pascal_names() { + let m = model(); + assert_eq!( + write_expr(&m, &Ty::Prim(Prim::I64), "v.n"), + "w.writeI64(v.n)" + ); + assert_eq!(read_expr(&m, &Ty::Prim(Prim::String)), "r.readString()"); + assert_eq!(writer_fn(&m, &Ty::Prim(Prim::Bytes)), "$W.Bytes"); + assert_eq!(reader_fn(&m, &Ty::Enum("Kind".into())), "$R.I32"); + } + + #[test] + fn composites_name_one_function_per_shape() { + let m = model(); + let ty = Ty::Optional(Box::new(Ty::List(Box::new(Ty::Record("Entry".into()))))); + assert_eq!(write_expr(&m, &ty, "v.x"), "$w_opt_list_Entry(w, v.x)"); + assert_eq!(read_expr(&m, &ty), "$r_opt_list_Entry(r)"); + let map = Ty::Map( + Box::new(Ty::Prim(Prim::U64)), + Box::new(Ty::Interface("Store".into())), + ); + assert_eq!(writer_fn(&m, &map), "$w_map_u64_Store"); + assert_eq!(reader_fn(&m, &Ty::Interface("Store".into())), "$r$kv$Store"); + } + + #[test] + fn codecs_cover_every_buffered_shape_once() { + let yaml = r#" +version: "0.11.0" +modules: + - name: kv + structs: + - name: Entry + fields: + - { name: tags, type: "[string]" } + - { name: by_kind, type: "{Kind:u32}" } + - { name: owner, type: "Store?" } + enums: [{ name: Kind, variants: [{ name: A, value: 0 }] }] + interfaces: [{ name: Store, methods: [{ name: get }] }] + functions: + - { name: all, params: [{ name: tags, type: "[string]" }], return: "[Entry?]" } +"#; + let api = weaveffi_model::parse::parse_api_str(yaml, "yaml").unwrap(); + let m = weaveffi_model::validate::validate( + &api, + &weaveffi_model::pkg::Identity::named("t"), + None, + ) + .unwrap(); + let mut w = CodeWriter::two_space(); + emit_codecs(&mut w, &m); + let js = w.finish(); + for needle in [ + "function $w_list_string(w, v) {\n w.writeList(v, $W.String);\n}", + "function $r_map_Kind_u32(r) {\n return r.readMap($R.I32, $R.U32);\n}", + "function $w_map_Kind_u32(w, v) {\n w.writeMap(v, $WK.I32, $W.U32);\n}", + "function $w_list_opt_Entry(w, v) {\n w.writeList(v, $w_opt_Entry);\n}", + "function $r_opt_Store(r) {\n return r.readOpt($r$kv$Store);\n}", + "function $w$kv$Store(w, v) {\n w.writeU64(BigInt($clone(v, kv$Store)));\n}", + " $w_opt_Store(w, v.owner);\n", + ] { + assert!(js.contains(needle), "missing `{needle}` in:\n{js}"); + } + assert_eq!(js.matches("function $w_list_string(").count(), 1); + } +} diff --git a/crates/weaveffi-gen/src/targets/js/dts.rs b/crates/weaveffi-cli/src/targets/js/dts.rs similarity index 72% rename from crates/weaveffi-gen/src/targets/js/dts.rs rename to crates/weaveffi-cli/src/targets/js/dts.rs index bd2ada70..cb2cb8f6 100644 --- a/crates/weaveffi-gen/src/targets/js/dts.rs +++ b/crates/weaveffi-cli/src/targets/js/dts.rs @@ -2,26 +2,30 @@ //! //! Each top-level IDL module is an exported namespace (`kv`), nested //! modules are nested namespaces, and user types are referenced by their -//! full path from the root (`kv.Entry`), so equal names in different modules -//! never collide. +//! full path from the root (`kv.Entry`). use weaveffi_model::model::{ - BindingModel, CallShape, CallbackInterfaceBinding, EnumBinding, ErrorBinding, FnBinding, - InterfaceBinding, ModuleBinding, ParamBinding, StructBinding, Ty, + CallShape, CallbackInterfaceBinding, EnumBinding, ErrorBinding, FnBinding, InterfaceBinding, + Model, ModuleBinding, ParamBinding, StructBinding, }; +use weaveffi_model::ty::Ty; use crate::codegen::CodeWriter; use crate::targets::js::names::{ callback_method_name, code_class, error_owner, fn_name, js_string, member_name, module_name, - param_name, ts_path, ts_type, + param_name, ts_path_in, ts_type, }; /// Render the declarations body: the root error classes, `extra` (the /// transport's own exports, such as WebAssembly's `init`), the leak-counter /// hook, and one namespace per top-level module. `root_error` is the root /// error class name. -pub(crate) fn render_declarations(model: &BindingModel, root_error: &str, extra: &str) -> String { - let mut w = CodeWriter::two_space(); +pub(crate) fn render_declarations( + w: &mut CodeWriter, + model: &Model, + root_error: &str, + extra: &str, +) { w.line("/// "); w.blank(); w.line("/**"); @@ -50,35 +54,35 @@ pub(crate) fn render_declarations(model: &BindingModel, root_error: &str, extra: w.line(format!("declare const $Error: typeof {root_error};")); w.raw(extra); w.line("/**"); - w.line(" * The native library's live-allocation counters (0 objects, 1 callbacks,"); - w.line(" * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests."); + w.line(" * The native library's live-resource counters (0 objects, 1 callbacks,"); + w.line(" * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is"); + w.line(" * `1n` when the library counts at all."); w.line(" * @internal"); w.line(" */"); w.line("export declare function __debugLive(kind: number): bigint;"); for m in model.roots() { w.blank(); - doc(&mut w, m.doc.as_deref(), &[]); + doc(w, m.doc.as_deref(), &[]); w.block( format!("export declare namespace {} {{", module_name(m)), "}", |w| namespace_body(w, model, m), ); } - w.finish() } -fn namespace_body(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding) { - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { - error_domain(w, m, eb); +fn namespace_body(w: &mut CodeWriter, model: &Model, m: &ModuleBinding) { + if let Some(eb) = m.errors.as_ref() { + error_domain(w, model, m, eb); } for e in &m.enums { - enumeration(w, e); + enumeration(w, model, e); } for s in &m.structs { - record(w, s); + record(w, model, s); } for cb in &m.callback_interfaces { - callback_interface(w, cb); + callback_interface(w, model, m, cb); } for i in &m.interfaces { class(w, model, m, i); @@ -88,8 +92,8 @@ fn namespace_body(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding) { w.line(format!( "export function {}({}): {};", fn_name(&f.name), - params(f), - ret(f) + params(model, f), + ret(model, f) )); } for child in model.children(m) { @@ -136,13 +140,13 @@ fn deprecated(msg: Option<&String>) -> Vec { /// The JSDoc tags of a callable: documented parameters, the domain it /// throws, and any deprecation. -fn fn_tags(model: &BindingModel, m: &ModuleBinding, f: &FnBinding) -> Vec { +fn fn_tags(model: &Model, m: &ModuleBinding, f: &FnBinding) -> Vec { let mut tags = param_tags(&f.params); - if let (true, Some(eb)) = (f.throws, m.error.as_ref()) { + if let (true, Some(eb)) = (f.throws, model.error_domain(m)) { let owner = error_owner(&model.modules, eb); tags.push(format!( "@throws {{{}}}", - ts_path(&format!("{}.{}", owner.dot_path, eb.type_name)) + ts_path_in(&owner.segments, &eb.type_name) )); } tags.extend(deprecated(f.deprecated.as_ref())); @@ -160,11 +164,11 @@ fn param_tags(params: &[ParamBinding]) -> Vec { .collect() } -fn params(f: &FnBinding) -> String { +fn params(model: &Model, f: &FnBinding) -> String { let mut out: Vec = f .params .iter() - .map(|p| format!("{}: {}", param_name(&p.name), ts_type(&p.ty))) + .map(|p| format!("{}: {}", param_name(&p.name), ts_type(model, &p.ty))) .collect(); if f.cancellable { out.push("options?: { signal?: AbortSignal }".into()); @@ -172,21 +176,21 @@ fn params(f: &FnBinding) -> String { out.join(", ") } -fn ret(f: &FnBinding) -> String { +fn ret(model: &Model, f: &FnBinding) -> String { let base = match (&f.shape, &f.ret) { - (CallShape::Iterator(it), _) => format!("IterableIterator<{}>", ts_type(&it.elem)), - (_, Some(ty)) => ts_type(ty), + (CallShape::Iterator(it), _) => format!("IterableIterator<{}>", ts_type(model, &it.elem)), + (_, Some(ty)) => ts_type(model, ty), (_, None) => "void".into(), }; - if f.is_async { + if f.is_async() { format!("Promise<{base}>") } else { base } } -fn error_domain(w: &mut CodeWriter, m: &ModuleBinding, eb: &ErrorBinding) { - let domain_ts = ts_path(&format!("{}.{}", m.dot_path, eb.type_name)); +fn error_domain(w: &mut CodeWriter, model: &Model, m: &ModuleBinding, eb: &ErrorBinding) { + let domain_ts = ts_path_in(&m.segments, &eb.type_name); w.line(format!( "/** The errors of the `{}` error domain. */", m.dot_path @@ -210,15 +214,27 @@ fn error_domain(w: &mut CodeWriter, m: &ModuleBinding, eb: &ErrorBinding) { w.line(format!("static readonly CODE: {};", c.value)); for f in &c.fields { doc(w, f.doc.as_deref(), &[]); - w.line(format!("readonly {}: {};", f.name, ts_type(&f.ty))); + w.line(format!("readonly {}: {};", f.name, ts_type(model, &f.ty))); + } + if c.fields.is_empty() { + w.line("constructor(message?: string);"); + } else { + let fields: Vec = c + .fields + .iter() + .map(|f| format!("{}: {}", f.name, ts_type(model, &f.ty))) + .collect(); + w.line(format!( + "constructor(fields: {{ {} }}, message?: string);", + fields.join("; ") + )); } - w.line("constructor(message?: string);"); }, ); } } -fn enumeration(w: &mut CodeWriter, e: &EnumBinding) { +fn enumeration(w: &mut CodeWriter, model: &Model, e: &EnumBinding) { doc(w, e.doc.as_deref(), &deprecated(e.deprecated.as_ref())); if e.rich { w.line(format!("export type {} =", e.name)); @@ -229,7 +245,7 @@ fn enumeration(w: &mut CodeWriter, e: &EnumBinding) { let fields: String = v .fields .iter() - .map(|f| format!("; {}: {}", f.name, ts_type(&f.ty))) + .map(|f| format!("; {}: {}", f.name, ts_type(model, &f.ty))) .collect(); let end = if i == last { ";" } else { "" }; w.line(format!("| {{ tag: {}{fields} }}{end}", js_string(&v.name))); @@ -245,30 +261,49 @@ fn enumeration(w: &mut CodeWriter, e: &EnumBinding) { }); } -fn record(w: &mut CodeWriter, s: &StructBinding) { +fn record(w: &mut CodeWriter, model: &Model, s: &StructBinding) { doc(w, s.doc.as_deref(), &deprecated(s.deprecated.as_ref())); w.block(format!("export interface {} {{", s.name), "}", |w| { for f in &s.fields { doc(w, f.doc.as_deref(), &[]); - w.line(format!("{}: {};", f.name, ts_type(&f.ty))); + w.line(format!("{}: {};", f.name, ts_type(model, &f.ty))); } }); } -/// A callback interface: any object with these methods implements it. -pub(crate) fn callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding) { +/// A callback interface: any object with these methods implements it. A +/// method that throws may raise its module's domain errors, which reach the +/// library with their codes and fields. +fn callback_interface( + w: &mut CodeWriter, + model: &Model, + module: &ModuleBinding, + cb: &CallbackInterfaceBinding, +) { + let domain = model.error_domain(module).map(|eb| { + let owner = error_owner(&model.modules, eb); + ts_path_in(&owner.segments, &eb.type_name) + }); doc(w, cb.doc.as_deref(), &deprecated(cb.deprecated.as_ref())); w.block(format!("export interface {} {{", cb.name), "}", |w| { for m in &cb.methods { let mut tags = param_tags(&m.params); + if let (true, Some(domain)) = (m.throws, &domain) { + tags.push(format!( + "@throws {{{domain}}} to report that error, with its fields, to the library" + )); + } tags.extend(deprecated(m.deprecated.as_ref())); doc(w, m.doc.as_deref(), &tags); let ps: Vec = m .params .iter() - .map(|p| format!("{}: {}", param_name(&p.name), ts_type(&p.ty))) + .map(|p| format!("{}: {}", param_name(&p.name), ts_type(model, &p.ty))) .collect(); - let r = m.ret.as_ref().map_or_else(|| "void".to_string(), ts_type); + let r = m + .ret + .as_ref() + .map_or_else(|| "void".to_string(), |t| ts_type(model, t)); w.line(format!( "{}({}): {r};", callback_method_name(&m.name), @@ -278,18 +313,18 @@ pub(crate) fn callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBindi }); } -fn class(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, i: &InterfaceBinding) { - let self_ty = ts_type(&Ty::Interface(format!("{}.{}", m.dot_path, i.name))); +fn class(w: &mut CodeWriter, model: &Model, m: &ModuleBinding, i: &InterfaceBinding) { + let self_ty = ts_type(model, &Ty::Interface(i.name.clone())); doc(w, i.doc.as_deref(), &deprecated(i.deprecated.as_ref())); w.block(format!("export class {} {{", i.name), "}", |w| { let canonical = i .constructors .iter() - .find(|c| c.name == "new" && !c.is_async); + .find(|c| c.name == "new" && !c.is_async()); match canonical { Some(c) => { doc(w, c.doc.as_deref(), &fn_tags(model, m, c)); - w.line(format!("constructor({});", params(c))); + w.line(format!("constructor({});", params(model, c))); } None => { w.line("private constructor();"); @@ -301,7 +336,7 @@ fn class(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, i: &Interf .filter(|c| canonical.is_none_or(|k| !std::ptr::eq(k, *c))) { doc(w, c.doc.as_deref(), &fn_tags(model, m, c)); - let r = if c.is_async { + let r = if c.is_async() { format!("Promise<{self_ty}>") } else { self_ty.clone() @@ -309,7 +344,7 @@ fn class(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, i: &Interf w.line(format!( "static {}({}): {r};", member_name(&c.name, true), - params(c) + params(model, c) )); } for f in &i.methods { @@ -317,8 +352,8 @@ fn class(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, i: &Interf w.line(format!( "{}({}): {};", member_name(&f.name, false), - params(f), - ret(f) + params(model, f), + ret(model, f) )); } for f in &i.statics { @@ -326,8 +361,8 @@ fn class(w: &mut CodeWriter, model: &BindingModel, m: &ModuleBinding, i: &Interf w.line(format!( "static {}({}): {};", member_name(&f.name, true), - params(f), - ret(f) + params(model, f), + ret(model, f) )); } w.line("/**"); diff --git a/crates/weaveffi-gen/src/targets/js/mod.rs b/crates/weaveffi-cli/src/targets/js/mod.rs similarity index 58% rename from crates/weaveffi-gen/src/targets/js/mod.rs rename to crates/weaveffi-cli/src/targets/js/mod.rs index d65678ce..017e3b1c 100644 --- a/crates/weaveffi-gen/src/targets/js/mod.rs +++ b/crates/weaveffi-cli/src/targets/js/mod.rs @@ -14,7 +14,7 @@ //! documented in [`api`] as an object named `$raw`. //! //! The fixed runtime (codec, error classes, object and iterator wrappers, -//! cancellation, the load-time check) is a real JavaScript file, +//! cancellation, the load-time checks) is a real JavaScript file, //! `runtime/runtime.js`, emitted next to `index.js`. mod api; @@ -22,7 +22,8 @@ mod codec; mod dts; pub(crate) mod names; -use weaveffi_model::model::{BindingModel, ABI_VERSION}; +use weaveffi_model::contract; +use weaveffi_model::model::{Model, ABI_VERSION}; use weaveffi_model::pkg::Identity; use crate::codegen::CodeWriter; @@ -30,7 +31,6 @@ use crate::manifest::{JsonObject, JsonValue}; use crate::utils::{render_prelude, render_trailer, CommentStyle}; pub(crate) use api::render_api; -pub(crate) use dts::callback_interface as render_callback_interface_dts; /// The shared runtime, with `{{ERROR_CLASS}}` naming the root error class. const RUNTIME_JS: &str = include_str!("runtime/runtime.js"); @@ -40,7 +40,9 @@ const RUNTIME_IMPORTS: &[&str] = &[ "$Fault", "$fault", "$domain", + "$raise", "$W", + "$WK", "$R", "$encode", "$decode", @@ -52,6 +54,7 @@ const RUNTIME_IMPORTS: &[&str] = &[ "$lendOpt", "$unlend", "$clone", + "$cloneOpt", "$Iterator", "$cancellable", "$impl", @@ -65,18 +68,18 @@ pub(crate) fn root_error_class(identity: &Identity) -> String { } /// Render `runtime.js`. -pub(crate) fn render_runtime(identity: &Identity, input_basename: &str) -> String { - let mut out = render_prelude(CommentStyle::DoubleSlash, input_basename); - out.push_str(&RUNTIME_JS.replace("{{ERROR_CLASS}}", &root_error_class(identity))); - out.push('\n'); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, "runtime.js")); - out +pub(crate) fn render_runtime(identity: &Identity) -> String { + let mut w = CodeWriter::two_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + w.raw(RUNTIME_JS.replace("{{ERROR_CLASS}}", &root_error_class(identity))); + w.blank(); + w.raw(render_trailer(CommentStyle::DoubleSlash, "runtime.js")); + w.finish() } /// The import and re-export lines every `index.js` starts with. -pub(crate) fn render_imports(identity: &Identity) -> String { +pub(crate) fn render_imports(w: &mut CodeWriter, identity: &Identity) { let root = root_error_class(identity); - let mut w = CodeWriter::two_space(); w.line("import {"); w.scope(|w| { for name in RUNTIME_IMPORTS { @@ -88,65 +91,67 @@ pub(crate) fn render_imports(identity: &Identity) -> String { w.line(format!( "export {{ {root}, CancelledError }} from './runtime.js';" )); - w.finish() } -/// The load-time contract check: the ABI revision and every top-level -/// module's checksum. -pub(crate) fn render_verify(model: &BindingModel, identity: &Identity) -> String { - let sums: Vec = model - .roots() - .map(|m| { - format!( - "['{}', 0x{:016x}n]", - m.name, - m.checksum.expect("a top-level module has a checksum") - ) - }) - .collect(); +/// The contract these bindings were generated with (`const $contract`): +/// for every top-level module, each declaration's `[id, hash, path]` from +/// [`contract::entries`], which the load-time check looks up in the +/// library's own tables. +pub(crate) fn render_contract(w: &mut CodeWriter, model: &Model) { + w.line("// The declarations these bindings were generated with, per top-level"); + w.line("// module, as [id, hash, path]: the library's contract tables must hold"); + w.line("// each one (see $verify)."); + w.block("const $contract = [", "];", |w| { + for root in model.roots() { + w.block( + format!("[{}, [", names::js_string(&root.name)), + "]],", + |w| { + for e in contract::entries(model, root) { + w.line(format!( + "[0x{:016x}n, 0x{:016x}n, {}],", + e.id, + e.hash, + names::js_string(&e.path) + )); + } + }, + ); + } + }); +} + +/// The load-time check of the ABI revision and of `$contract` (see +/// [`render_contract`]) against the library behind `$raw`. +pub(crate) fn verify_call(model: &Model, identity: &Identity) -> String { format!( - "$verify($raw, {}, '{}', {ABI_VERSION}, [{}]);\n", + "$verify($raw, {}, '{}', {ABI_VERSION}, $contract);", names::js_string(&identity.name), - model.prefix, - sums.join(", ") + model.prefix(), ) } /// Render `index.d.ts`. `extra` carries the transport's own declarations. -pub(crate) fn render_dts( - model: &BindingModel, - identity: &Identity, - input_basename: &str, - extra: &str, -) -> String { - let mut out = render_prelude(CommentStyle::DoubleSlash, input_basename); - out.push_str(&dts::render_declarations( - model, - &root_error_class(identity), - extra, - )); - out.push('\n'); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, "index.d.ts")); - out +pub(crate) fn render_dts(model: &Model, identity: &Identity, extra: &str) -> String { + let mut w = CodeWriter::two_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + dts::render_declarations(&mut w, model, &root_error_class(identity), extra); + w.blank(); + w.raw(render_trailer(CommentStyle::DoubleSlash, "index.d.ts")); + w.finish() } /// The npm metadata both `package.json` files share: the generated-file /// notice, then name, version, description, and the optional fields. -pub(crate) fn npm_metadata(identity: &Identity, name: &str, input_basename: &str) -> JsonObject { +pub(crate) fn npm_metadata(identity: &Identity, name: &str) -> JsonObject { let version = env!("CARGO_PKG_VERSION"); let mut obj = JsonObject::new() - .str_entry( - "//", - format!("Generated by WeaveFFI {version} from {input_basename}"), - ) + .str_entry("//", format!("Generated by WeaveFFI {version}")) .str_entry( "//warning", "DO NOT EDIT. Your changes will be overwritten.", ) - .str_entry( - "//regenerate", - format!("To regenerate: weaveffi generate {input_basename} -o "), - ) + .str_entry("//regenerate", "To regenerate: weaveffi generate") .str_entry("name", name) .str_entry("version", &identity.version) .str_entry("description", identity.description_or_default()) @@ -185,14 +190,16 @@ pub(crate) fn npm_metadata(identity: &Identity, name: &str, input_basename: &str /// A small API exercising every shape both JavaScript targets render, for /// their unit tests: an error domain with a payload, a C-style and a rich -/// enum, a record holding objects, a callback interface, an interface with -/// a canonical constructor, a method named `close`, a throwing method, an +/// enum, a record holding objects, two callback interfaces (one with a +/// return of every family and a method that throws), an interface with a +/// canonical constructor, a method named `close`, a throwing method, an /// iterator of objects, a nullable async result, and a cancellable async -/// method, free functions (one named after a keyword), and a nested module. +/// method, free functions (one named after a keyword, one taking an +/// optional callback), and a nested module. #[cfg(test)] -pub(crate) fn test_api(name: &str) -> weaveffi_model::resolved::ResolvedApi { +pub(crate) fn test_api(name: &str) -> weaveffi_model::model::Model { const YAML: &str = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: kv doc: A key-value store. @@ -206,6 +213,7 @@ modules: variants: - { name: Fast, value: 0 } - { name: Safe, value: 7 } + - { name: Off, value: -1 } - name: Shape variants: - { name: Empty, value: 0 } @@ -225,6 +233,15 @@ modules: return: bool - name: on_bundle params: [{ name: bundle, type: Bundle }, { name: store, type: Store }, { name: alt, type: "Store?" }] + - name: Policy + methods: + - { name: admit, params: [{ name: bundle, type: Bundle }], return: Bundle, throws: true } + - { name: label, params: [], return: string } + - { name: blob, params: [], return: bytes } + - { name: pick, params: [{ name: key, type: string }], return: Store } + - { name: maybe, params: [], return: "Store?" } + - { name: mode, params: [], return: Mode } + - { name: weight, params: [], return: i64 } interfaces: - name: Store doc: A store. @@ -238,6 +255,7 @@ modules: - { name: wait, params: [{ name: ms, type: i64 }], return: i64, async: true, cancellable: true } functions: - { name: subscribe, params: [{ name: listener, type: Listener }, { name: mode, type: Mode }] } + - { name: install, params: [{ name: policy, type: "Policy?" }] } - { name: widen, params: [{ name: n, type: i64 }, { name: m, type: u64 }], return: u64 } - { name: delete, params: [{ name: class, type: "Store?" }], return: bool } - { name: shapes, params: [], return: "iter" } @@ -247,9 +265,8 @@ modules: - { name: count, params: [{ name: store, type: Store }], return: i64, throws: true } "#; let api = weaveffi_model::parse::parse_api_str(YAML, "yaml").expect("test fixture parses"); - weaveffi_model::validate::validate_api(api, None) + weaveffi_model::validate::validate(&api, &Identity::named(name), None) .expect("test fixture validates") - .with_identity(Identity::named(name)) } #[cfg(test)] @@ -258,24 +275,51 @@ mod tests { #[test] fn runtime_names_the_package_root_error() { - let rt = render_runtime(&Identity::named("my-kv"), "kv.yml"); + let rt = render_runtime(&Identity::named("my-kv")); assert!( rt.contains("export class MyKvError extends Error {"), "{rt}" ); assert!(rt.contains("export class CancelledError extends MyKvError {")); assert!(!rt.contains("{{")); + assert!(!rt.contains("checksum")); assert_eq!(root_error_class(&Identity::named("io_error")), "IoError"); } fn api_js() -> String { - render_api(&BindingModel::build(&test_api("acme-kv"))) + let mut w = CodeWriter::two_space(); + render_api(&mut w, &test_api("acme-kv")); + w.finish() } fn has(src: &str, needle: &str) { assert!(src.contains(needle), "missing `{needle}` in:\n{src}"); } + #[test] + fn the_contract_lists_every_declaration_with_its_path() { + let model = test_api("acme-kv"); + let mut w = CodeWriter::two_space(); + render_contract(&mut w, &model); + let js = w.finish(); + let root = model.roots().next().unwrap(); + let entries = contract::entries(&model, root); + assert_eq!(js.matches("n, 0x").count(), entries.len()); + let put = entries.iter().find(|e| e.path == "kv.Store.get").unwrap(); + has(&js, "const $contract = [\n ['kv', [\n"); + has( + &js, + &format!( + " [0x{:016x}n, 0x{:016x}n, 'kv.Store.get'],\n", + put.id, put.hash + ), + ); + assert_eq!( + verify_call(&model, &model.identity), + "$verify($raw, 'acme-kv', 'acme_kv', 4, $contract);" + ); + } + #[test] fn modules_are_frozen_namespaces_with_escaped_members() { let js = api_js(); @@ -288,6 +332,7 @@ mod tests { ); has(&js, "const kv$delete_ = function delete_(class_) {"); has(&js, " close_() {"); + has(&js, " 7: 'Safe',\n '-1': 'Off',\n"); } #[test] @@ -303,12 +348,35 @@ mod tests { &js, "return $domain(e, $codes$kv$KvError, $payloads$kv$KvError);", ); - has(&js, "[1, (r) => ({ key: r.readString() })]"); - has(&js, "constructor(message = 'it\\'s full') {"); + has( + &js, + "const $payloads$kv$KvError = new Map([\n [1, (r) => ({ key: r.readString() })],\n]);", + ); } #[test] - fn async_iterators_and_callbacks_follow_the_raw_convention() { + fn error_codes_with_fields_take_them_first() { + let js = api_js(); + has( + &js, + " constructor(fields, message = 'key not found') {\n super(1, message);\n this.key = fields.key;\n }", + ); + has( + &js, + " constructor(message = 'it\\'s full') {\n super(2, message);\n }", + ); + has( + &js, + "const $fields$kv$KvError = new Map([\n [1, (w, e) => { w.writeString(e.key); }],\n]);", + ); + has( + &js, + "return $raise(e, kv$KvError, $codes$kv$KvError, $fields$kv$KvError);", + ); + } + + #[test] + fn async_and_iterators_follow_the_raw_convention() { let js = api_js(); has( &js, @@ -324,38 +392,70 @@ mod tests { ); has(&js, "convert: (v) => $adopt(kv$Store, v),"); has(&js, "convert: (v) => $decode(v, $r$kv$Shape),"); + } + + #[test] + fn adapters_convert_arguments_and_every_return_family() { + let js = api_js(); has( &js, "return $ret.Bool(impl.onMessage($a0, $a1), 'Listener.onMessage');", ); has(&js, "impl.onBundle($decode($a0, $r$kv$Bundle), $adopt(kv$Store, $a1), $adoptOpt(kv$Store, $a2));"); + has( + &js, + " admit($a0) {\n try {\n return $encode(impl.admit($decode($a0, $r$kv$Bundle)), $w$kv$Bundle);\n } catch ($e) {\n throw $raise$kv$KvError($e);\n }\n },", + ); + has(&js, "return $ret.String(impl.label(), 'Policy.label');"); + has(&js, "return $ret.Bytes(impl.blob(), 'Policy.blob');"); + has(&js, "return $clone(impl.pick($a0), kv$Store);"); + has(&js, "return $cloneOpt(impl.maybe(), kv$Store);"); + has(&js, "return $ret.I32(impl.mode(), 'Policy.mode');"); + has(&js, "return $ret.I64(impl.weight(), 'Policy.weight');"); has( &js, "$raw.acme_kv_kv_subscribe($adapt$kv$Listener(listener), mode);", ); + has( + &js, + "$raw.acme_kv_kv_install(policy == null ? null : $adapt$kv$Policy(policy));", + ); } #[test] - fn codecs_clone_object_tokens_and_convert_map_keys() { + fn codecs_name_one_function_per_type() { let js = api_js(); - has(&js, "w.writeU64(BigInt($clone(v.primary, kv$Store)));"); - has(&js, "primary: $adopt(kv$Store, $token(r.readU64())),"); + has(&js, " $w$kv$Store(w, v.primary);\n"); + has(&js, " primary: $r$kv$Store(r),\n"); + has(&js, " $w_list_Store(w, v.extras);\n"); + has( + &js, + "function $w_map_Mode_bool(w, v) {\n w.writeMap(v, $WK.I32, $W.Bool);\n}", + ); + has( + &js, + "function $w$kv$Store(w, v) {\n w.writeU64(BigInt($clone(v, kv$Store)));\n}", + ); has( &js, - "w.writeMap(v.by_mode, (w, k) => w.writeI32(Number(k)), $W.Bool);", + "function $r$kv$Store(r) {\n return $adopt(kv$Store, $token(r.readU64()));\n}", ); } #[test] fn declarations_qualify_types_and_alias_the_root_error() { - let model = BindingModel::build(&test_api("kv")); - let dts = render_dts(&model, &Identity::named("kv"), "kv.yml", ""); + let model = test_api("kv"); + let dts = render_dts(&model, &Identity::named("kv"), ""); has(&dts, "export declare class KvError extends Error {"); has(&dts, "declare const $Error: typeof KvError;"); has(&dts, "export declare namespace kv {"); has(&dts, " export class KvError extends $Error {"); has(&dts, " export class KeyNotFoundError extends kv.KvError {"); has(&dts, " readonly key: string;"); + has( + &dts, + " constructor(fields: { key: string }, message?: string);", + ); has( &dts, " wait(ms: bigint, options?: { signal?: AbortSignal }): Promise;", @@ -367,6 +467,11 @@ mod tests { &dts, " export function delete_(class_: kv.Store | null): boolean;", ); + has( + &dts, + " export function install(policy: kv.Policy | null): void;", + ); + has(&dts, " maybe(): kv.Store | null;"); has(&dts, " export function count(store: kv.Store): bigint;"); } } diff --git a/crates/weaveffi-gen/src/targets/js/names.rs b/crates/weaveffi-cli/src/targets/js/names.rs similarity index 56% rename from crates/weaveffi-gen/src/targets/js/names.rs rename to crates/weaveffi-cli/src/targets/js/names.rs index 79beef4b..a3372038 100644 --- a/crates/weaveffi-gen/src/targets/js/names.rs +++ b/crates/weaveffi-cli/src/targets/js/names.rs @@ -5,18 +5,18 @@ //! * **Public names** are what consumers write: `kv.Store`, `kv.openStore`. //! Identifiers that are reserved words gain a trailing underscore. //! * **Declaration names** are the module-scope bindings of user -//! declarations, the dotted path joined with `$` (`kv$Store`, -//! `kv$stats$getStats`). IDL identifiers never contain `$`, so two modules +//! declarations, the declaring module's path and the name joined with `$` +//! (`kv$Store`, `kv$stats$getStats`). IDL identifiers never contain `$`, so two modules //! declaring the same name never clash, and no declaration can shadow a //! JavaScript global. //! * **Helper names** start with `$` (`$lend`, `$w$kv$Entry`): runtime //! imports and generated support code, which no IDL name can reach. use heck::ToLowerCamelCase; -use weaveffi_model::model::{ErrorBinding, FnBinding, ModuleBinding, Prim, Ty, WireType}; +use weaveffi_model::model::{ErrorBinding, FnBinding, Model, ModuleBinding}; +use weaveffi_model::ty::{Prim, Ty, WireType}; use crate::lang; -use crate::utils::local_type_name; /// Identifiers that are not JS keywords but can't name a binding or /// parameter in strict-mode (module) code. @@ -37,22 +37,26 @@ pub(crate) fn fn_name(name: &str) -> String { js_ident(&name.to_lower_camel_case()) } -/// The name of an interface member. Instance members can't be named -/// `constructor` or `close` (the wrapper's own release method), and statics -/// can't replace the class's `name`, `length`, or `prototype`; those gain a -/// trailing underscore. +/// Instance members every interface class declares: the JavaScript +/// `constructor` and the runtime `$Object`'s `close`. (Its other members +/// start with `$` or are symbols, which no IDL name can reach.) +const OBJECT_MEMBERS: &[&str] = &["close", "constructor"]; + +/// Static members every class has (`Function`'s own properties), which a +/// static can't replace. +const CLASS_MEMBERS: &[&str] = &["caller", "length", "name", "prototype"]; + +/// The name of an interface member: lowerCamelCase, with a trailing +/// underscore when an instance member would replace one of +/// [`OBJECT_MEMBERS`] or a static (or factory) one of [`CLASS_MEMBERS`]. +/// Property names may be reserved words, so no keyword escape applies. pub(crate) fn member_name(name: &str, is_static: bool) -> String { - let camel = name.to_lower_camel_case(); - let clash = if is_static { - matches!(camel.as_str(), "name" | "length" | "prototype" | "caller") + let members = if is_static { + CLASS_MEMBERS } else { - matches!(camel.as_str(), "constructor" | "close") + OBJECT_MEMBERS }; - if clash { - format!("{camel}_") - } else { - camel - } + lang::escape_member(&name.to_lower_camel_case(), members) } /// The method a callback-interface implementation provides (`on_evict` @@ -71,16 +75,16 @@ pub(crate) fn decl(segments: &[String], name: &str) -> String { format!("{}${name}", segments.join("$")) } -/// The declaration name of a user type from its absolute dotted path -/// (`kv.Store` becomes `kv$Store`). -pub(crate) fn type_decl(dotted: &str) -> String { - dotted.replace('.', "$") +/// The declaration name of the user type `name`, from its declaring +/// module's path (`Store` in `kv` becomes `kv$Store`). +pub(crate) fn type_decl(model: &Model, name: &str) -> String { + decl(&model.owner(name).segments, name) } -/// A helper name derived from a user type (`$w` and `kv.Entry` give +/// A helper name derived from a user type (`$w` and `Entry` in `kv` give /// `$w$kv$Entry`). -pub(crate) fn helper(kind: &str, dotted: &str) -> String { - format!("${kind}${}", type_decl(dotted)) +pub(crate) fn helper(model: &Model, kind: &str, name: &str) -> String { + format!("${kind}${}", type_decl(model, name)) } /// The public name of a module (its namespace object). @@ -88,18 +92,17 @@ pub(crate) fn module_name(m: &ModuleBinding) -> String { js_ident(&m.name) } -/// The TypeScript path of a user type from its absolute dotted path, each -/// module segment escaped like the namespace it names (`kv.Store`). -pub(crate) fn ts_path(dotted: &str) -> String { - let local = local_type_name(dotted); - match dotted.rsplit_once('.') { - Some((modules, _)) => { - let mut out: Vec = modules.split('.').map(js_ident).collect(); - out.push(local.to_string()); - out.join(".") - } - None => local.to_string(), - } +/// The TypeScript path of `name` declared in module `segments`, each module +/// segment escaped like the namespace it names (`kv.Store`). +pub(crate) fn ts_path_in(segments: &[String], name: &str) -> String { + let mut out: Vec = segments.iter().map(|s| js_ident(s)).collect(); + out.push(name.to_string()); + out.join(".") +} + +/// The TypeScript path of the user type `name` (`kv.Store`). +pub(crate) fn ts_path(model: &Model, name: &str) -> String { + ts_path_in(&model.owner(name).segments, name) } /// The error class name of an error code (`KEY_NOT_FOUND` becomes @@ -115,7 +118,7 @@ pub(crate) fn error_owner<'a>( ) -> &'a ModuleBinding { modules .iter() - .find(|m| m.path == eb.owner_path && m.declares_error()) + .find(|m| m.path == eb.owner_path && m.errors.is_some()) .expect("an error binding's owner module declares the domain") } @@ -137,23 +140,30 @@ pub(crate) fn error_mapper( } /// The TypeScript type of `ty`. -pub(crate) fn ts_type(ty: &Ty) -> String { +pub(crate) fn ts_type(model: &Model, ty: &Ty) -> String { match ty { - Ty::I8 | Ty::I16 | Ty::I32 | Ty::U8 | Ty::U16 | Ty::U32 | Ty::F32 | Ty::F64 => { - "number".into() - } - Ty::I64 | Ty::U64 => "bigint".into(), - Ty::Bool => "boolean".into(), - Ty::StringUtf8 => "string".into(), - Ty::Bytes => "Uint8Array".into(), + Ty::Prim( + Prim::I8 + | Prim::I16 + | Prim::I32 + | Prim::U8 + | Prim::U16 + | Prim::U32 + | Prim::F32 + | Prim::F64, + ) => "number".into(), + Ty::Prim(Prim::I64 | Prim::U64) => "bigint".into(), + Ty::Prim(Prim::Bool) => "boolean".into(), + Ty::Prim(Prim::String) => "string".into(), + Ty::Prim(Prim::Bytes) => "Uint8Array".into(), Ty::Record(n) | Ty::RichEnum(n) | Ty::Enum(n) | Ty::Interface(n) - | Ty::CallbackInterface(n) => ts_path(n), - Ty::Optional(inner) => format!("{} | null", ts_type(inner)), + | Ty::CallbackInterface(n) => ts_path(model, n), + Ty::Optional(inner) => format!("{} | null", ts_type(model, inner)), Ty::List(inner) => { - let t = ts_type(inner); + let t = ts_type(model, inner); if t.contains(' ') { format!("({t})[]") } else { @@ -161,16 +171,16 @@ pub(crate) fn ts_type(ty: &Ty) -> String { } } Ty::Map(k, v) => { - let value = ts_type(v); + let value = ts_type(model, v); match k.as_ref() { - Ty::Enum(e) => format!("Partial>", ts_path(e)), - Ty::I8 | Ty::I16 | Ty::I32 | Ty::U8 | Ty::U16 | Ty::U32 => { + Ty::Enum(e) => format!("Partial>", ts_path(model, e)), + Ty::Prim(Prim::I8 | Prim::I16 | Prim::I32 | Prim::U8 | Prim::U16 | Prim::U32) => { format!("Record") } _ => format!("Record"), } } - Ty::Iterator(inner) => format!("IterableIterator<{}>", ts_type(inner)), + Ty::Iterator(inner) => format!("IterableIterator<{}>", ts_type(model, inner)), } } @@ -210,7 +220,7 @@ pub(crate) fn js_string(s: &str) -> String { } #[cfg(test)] -mod tests { +pub(crate) mod tests { use super::*; #[test] @@ -226,31 +236,64 @@ mod tests { assert_eq!(member_name("name", false), "name"); } + /// `Entry` and `Kind` in `kv`, `C` in `a.b`, `Widget` in `class.inner`. + pub(crate) fn model() -> Model { + let yaml = r#" +version: "0.11.0" +modules: + - name: kv + enums: [{ name: Kind, variants: [{ name: A, value: 0 }] }] + structs: [{ name: Entry, fields: [{ name: k, type: i32 }] }] + interfaces: [{ name: Store, methods: [{ name: get }] }] + - name: a + modules: [{ name: b, structs: [{ name: C, fields: [{ name: x, type: i32 }] }] }] + - name: class + modules: [{ name: inner, structs: [{ name: Widget, fields: [{ name: x, type: i32 }] }] }] +"#; + let api = weaveffi_model::parse::parse_api_str(yaml, "yaml").unwrap(); + weaveffi_model::validate::validate(&api, &weaveffi_model::pkg::Identity::named("t"), None) + .unwrap() + } + #[test] fn declaration_names_join_the_path_with_dollars() { + let m = model(); let segs = vec!["a".to_string(), "b".to_string()]; assert_eq!(decl(&segs, "C"), "a$b$C"); - assert_eq!(type_decl("a.b.C"), "a$b$C"); - assert_eq!(helper("w", "kv.Entry"), "$w$kv$Entry"); - assert_eq!(ts_path("class.inner.Widget"), "class_.inner.Widget"); + assert_eq!(type_decl(&m, "C"), "a$b$C"); + assert_eq!(helper(&m, "w", "Entry"), "$w$kv$Entry"); + assert_eq!(ts_path(&m, "Widget"), "class_.inner.Widget"); } #[test] fn ts_types_qualify_user_types_from_the_root() { - assert_eq!(ts_type(&Ty::Record("kv.Entry".into())), "kv.Entry"); + let m = model(); + assert_eq!(ts_type(&m, &Ty::Record("Entry".into())), "kv.Entry"); assert_eq!( - ts_type(&Ty::List(Box::new(Ty::Optional(Box::new(Ty::I64))))), + ts_type( + &m, + &Ty::List(Box::new(Ty::Optional(Box::new(Ty::Prim(Prim::I64))))) + ), "(bigint | null)[]" ); assert_eq!( - ts_type(&Ty::Map( - Box::new(Ty::Enum("kv.Kind".into())), - Box::new(Ty::Bool) - )), + ts_type( + &m, + &Ty::Map( + Box::new(Ty::Enum("Kind".into())), + Box::new(Ty::Prim(Prim::Bool)) + ) + ), "Partial>" ); assert_eq!( - ts_type(&Ty::Map(Box::new(Ty::U64), Box::new(Ty::Bytes))), + ts_type( + &m, + &Ty::Map( + Box::new(Ty::Prim(Prim::U64)), + Box::new(Ty::Prim(Prim::Bytes)) + ) + ), "Record" ); } diff --git a/crates/weaveffi-gen/src/targets/js/runtime/runtime.js b/crates/weaveffi-cli/src/targets/js/runtime/runtime.js similarity index 82% rename from crates/weaveffi-gen/src/targets/js/runtime/runtime.js rename to crates/weaveffi-cli/src/targets/js/runtime/runtime.js index 48ac9191..e740a252 100644 --- a/crates/weaveffi-gen/src/targets/js/runtime/runtime.js +++ b/crates/weaveffi-cli/src/targets/js/runtime/runtime.js @@ -59,25 +59,38 @@ export function $fault(e) { /** * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. + * class, and `payloads` maps each code that carries fields to the reader + * that decodes them (the class's constructor then takes the fields first). + * Codes outside the domain fall back to `$fault`. */ export function $domain(e, codes, payloads) { if (e instanceof $Fault) { const cls = codes.get(e.code); if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; + const message = e.message || undefined; + const read = payloads.get(e.code); + if (read === undefined) return new cls(message); + return new cls(e.payload === null ? {} : $decode(e.payload, read), message); } } return $fault(e); } +/** + * The fault a callback implementation reports for exception `e` from a + * method that throws the error domain `domain`: a declared code of the + * domain travels with its fields (`fields` maps each code that has any to + * the writer that encodes them); anything else is rethrown unchanged and + * reported as a code -4 failure. + */ +export function $raise(e, domain, codes, fields) { + if (e instanceof domain && codes.has(e.code)) { + const write = fields.get(e.code); + return new $Fault(e.code, e.message, write === undefined ? null : $encode(e, write)); + } + return e; +} + function $malformed(what) { return new {{ERROR_CLASS}}(-3, 'malformed value buffer: ' + what); } @@ -306,6 +319,27 @@ export const $W = Object.freeze({ Bytes: (w, v) => w.writeBytes(v), }); +/** + * Map key writers. Plain-object keys are always strings, so numeric, + * `bigint`, and `bool` keys convert first (a `Map`'s typed keys pass + * through the same conversions unchanged). C-style enum keys use `I32`. + */ +export const $WK = Object.freeze({ + Bool: (w, k) => w.writeBool(k === true || k === 'true'), + I8: (w, k) => w.writeI8(Number(k)), + I16: (w, k) => w.writeI16(Number(k)), + I32: (w, k) => w.writeI32(Number(k)), + I64: (w, k) => w.writeI64(BigInt(k)), + U8: (w, k) => w.writeU8(Number(k)), + U16: (w, k) => w.writeU16(Number(k)), + U32: (w, k) => w.writeU32(Number(k)), + U64: (w, k) => w.writeU64(BigInt(k)), + F32: (w, k) => w.writeF32(Number(k)), + F64: (w, k) => w.writeF64(Number(k)), + String: (w, k) => w.writeString(k), + Bytes: (w, k) => w.writeBytes(k), +}); + /** Element readers for the fixed-width primitives, strings, and bytes. */ export const $R = Object.freeze({ Bool: (r) => r.readBool(), @@ -411,6 +445,11 @@ export function $clone(o, cls) { } } +/** `$clone`, with `null` for an absent object. */ +export function $cloneOpt(o, cls) { + return o === null || o === undefined ? null : $clone(o, cls); +} + const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); /** @@ -519,8 +558,9 @@ function $retBigint(v, where) { } /** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. + * Checks for a callback method's return value, by wire primitive (plus + * `String` and `Bytes`). A value of the wrong type throws, which reports + * the call as a code -4 failure. */ export const $ret = Object.freeze({ Bool(v, where) { @@ -537,14 +577,25 @@ export const $ret = Object.freeze({ F64: $num, I64: $retBigint, U64: $retBigint, + String(v, where) { + if (typeof v !== 'string') throw new TypeError(`${where} must return a string`); + return v; + }, + Bytes(v, where) { + if (!(v instanceof Uint8Array)) throw new TypeError(`${where} must return a Uint8Array`); + return v; + }, }); /** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. + * The load-time checks: the native library must implement ABI revision + * `abi`, and, for every top-level module, its contract table must hold each + * entry these bindings were generated with. `contract` lists + * `[module, [[id, hash, path], ...]]` per module; the native side returns + * a module's table as a `BigUint64Array` of `id, hash` pairs. Entries the + * library has and the bindings don't are fine. */ -export function $verify(raw, library, prefix, abi, checksums) { +export function $verify(raw, library, prefix, abi, contract) { const version = raw[`${prefix}_abi_version`]; if (typeof version !== 'function') { throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); @@ -555,15 +606,22 @@ export function $verify(raw, library, prefix, abi, checksums) { `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, ); } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); + for (const [module, expected] of contract) { + const table = raw[`${prefix}_${module}_contract`]; + if (typeof table !== 'function') { + throw new Error(`${library}: the native library does not export ${prefix}_${module}_contract`); + } + const pairs = table(); + const hashes = new Map(); + for (let i = 0; i + 1 < pairs.length; i += 2) hashes.set(pairs[i], pairs[i + 1]); + for (const [id, hash, path] of expected) { + const got = hashes.get(id); + if (got === undefined) { + throw new Error(`${library}: ${path} is missing from the library`); + } + if (got !== hash) { + throw new Error(`${library}: ${path} changed since these bindings were generated`); + } } } } diff --git a/crates/weaveffi-cli/src/targets/kotlin/bridge.rs b/crates/weaveffi-cli/src/targets/kotlin/bridge.rs new file mode 100644 index 00000000..7dad5d9f --- /dev/null +++ b/crates/weaveffi-cli/src/targets/kotlin/bridge.rs @@ -0,0 +1,302 @@ +//! `JniBridge.kt`: the internal object declaring one `external fun` per C +//! symbol (named after the symbol, so two modules' functions never +//! collide), the error factory the shim throws through, and the static +//! dispatch shims callback trampolines call. + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{ + CallShape, CallbackInterfaceBinding, CallbackMethodBinding, ErrorBinding, FnBinding, Model, +}; +use weaveffi_model::ty::{Family, Ty}; + +use crate::targets::kotlin::calls::{item_slot, lift, lower}; +use crate::targets::kotlin::names::{jni_kind, kt_member, kt_param, Names}; + +/// The `JniBridge` member name of a callback method's dispatch shim. +pub(crate) fn shim_name( + n: &Names, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, +) -> String { + n.native(&format!("{}_{}", cb.c_tag, m.name)) +} + +/// The JVM descriptor of a callback method's dispatch shim: the +/// implementation, the `out_err` address, the parameters, then the return. +pub(crate) fn shim_descriptor( + n: &Names, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, +) -> String { + let mut sig = format!("(L{}/{};J", n.package_path, n.ty(&cb.name)); + for p in &m.params { + sig.push_str(&n.jni_descriptor(Some(&p.ty))); + } + sig.push(')'); + sig.push_str(&n.jni_descriptor(m.ret.as_ref())); + sig +} + +/// The `external fun` declarations backing callable `f`. +fn callable_natives(w: &mut CodeWriter, n: &Names, f: &FnBinding) { + let mut params: Vec = Vec::new(); + if f.has_self { + params.push("_self: Long".to_string()); + } + params.extend( + f.params + .iter() + .map(|p| format!("{}: {}", kt_param(&p.name), n.jni_type(&p.ty))), + ); + match &f.shape { + CallShape::Sync(abi) => { + let ret = f + .ret + .as_ref() + .map(|t| format!(": {}", n.jni_type(t))) + .unwrap_or_default(); + w.line(format!( + "@JvmStatic external fun {}({}){ret}", + n.native(&abi.symbol), + params.join(", ") + )); + } + CallShape::Async(ab) => { + if f.cancellable { + params.push("_token: Long".to_string()); + } + params.push("_done: NativeCompletion<*>".to_string()); + w.line(format!( + "@JvmStatic external fun {}({})", + n.native(&ab.launch.symbol), + params.join(", ") + )); + } + CallShape::Iterator(it) => { + w.line(format!( + "@JvmStatic external fun {}({}): Long", + n.native(&it.launch.symbol), + params.join(", ") + )); + match item_slot(&it.elem) { + None => w.line(format!( + "@JvmStatic external fun {}(_iter: Long): ByteArray?", + n.native(&it.next.symbol) + )), + Some(slot) => w.line(format!( + "@JvmStatic external fun {}(_iter: Long, _out: {}): Boolean", + n.native(&it.next.symbol), + slot.kotlin_array + )), + }; + w.line(format!( + "@JvmStatic external fun {}(_iter: Long)", + n.native(&it.destroy_symbol) + )); + } + } +} + +/// The value a dispatch shim returns after reporting a failure (the +/// producer ignores it). +fn failure_value(t: &Ty) -> &'static str { + match t.family() { + Family::Direct => match jni_kind(t) { + "Boolean" => "false", + "Byte" => "0.toByte()", + "Short" => "0.toShort()", + "Int" => "0", + "Float" => "0f", + "Double" => "0.0", + _ => "0L", + }, + Family::Object { .. } => "0L", + _ => "null", + } +} + +/// The static shim a trampoline calls for one callback method: adopts its +/// object arguments, lifts every other argument from its JNI form, calls +/// the implementation, and lowers the result. A failure never unwinds into +/// the producer: the shim reports it through `out_err` (`_err`) and returns +/// a zero value. `domain` is the error domain of a method declared +/// `throws`, whose typed errors keep their code and payload. +fn callback_shim( + w: &mut CodeWriter, + n: &Names, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, + domain: Option<&ErrorBinding>, +) { + let mut params = vec![format!("_impl: {}", n.ty(&cb.name)), "_err: Long".into()]; + params.extend( + m.params + .iter() + .map(|p| format!("{}: {}", kt_param(&p.name), n.jni_type(&p.ty))), + ); + // Objects are adopted before anything that can fail, so a failure never + // strands a reference the producer handed over. + let mut adopted = Vec::new(); + let args: Vec = m + .params + .iter() + .map(|p| { + let name = kt_param(&p.name); + match p.ty.family() { + Family::Object { .. } => { + adopted.push(format!("val _{name} = {}", lift(n, &p.ty, &name))); + format!("_{name}") + } + _ => lift(n, &p.ty, &name), + } + }) + .collect(); + let call = format!("_impl.{}({})", kt_member(&m.name), args.join(", ")); + let typed = match domain { + Some(eb) if m.throws => format!("_e as? {}", n.exception(eb)), + _ => "null".to_string(), + }; + let ret = m + .ret + .as_ref() + .map(|t| match t.family() { + Family::String | Family::Bytes | Family::Buffer => ": ByteArray?".to_string(), + _ => format!(": {}", n.jni_type(t)), + }) + .unwrap_or_default(); + let name = shim_name(n, cb, m); + let sig = format!("fun {name}({}){ret}", params.join(", ")); + // `head` is the line opening the `try`. + let attempt = |w: &mut CodeWriter, head: String| { + w.line(head); + w.scope(|w| { + match &m.ret { + Some(t) => w.line(lower(n, t, &call)), + None => w.line(&call), + }; + }); + w.line("} catch (_e: Throwable) {"); + w.scope(|w| { + w.line(format!("fail(_err, _e, {typed})")); + if let Some(t) = &m.ret { + w.line(failure_value(t)); + } + }); + w.line("}"); + }; + w.line("@JvmStatic"); + if adopted.is_empty() && m.ret.is_some() { + attempt(w, format!("{sig} = try {{")); + } else { + w.line(format!("{sig} {{")); + w.scope(|w| { + for a in &adopted { + w.line(a); + } + let head = if m.ret.is_some() { + "return try {" + } else { + "try {" + }; + attempt(w, head.to_string()); + }); + w.line("}"); + } +} + +/// Render the body of `JniBridge.kt` (after the package line). +pub(crate) fn render_bridge(n: &Names, model: &Model) -> String { + let mut w = CodeWriter::four_space(); + w.blank(); + w.line("/**"); + w.line(format!( + " * The JNI entry points of `lib{}_jni`: one native per C symbol, named after", + n.library + )); + w.line(" * it, plus the hooks the shim calls back into. Internal plumbing for the"); + w.line(" * public wrappers."); + w.line(" */"); + w.block("internal object JniBridge {", "}", |w| { + w.block("init {", "}", |w| { + w.line("NativeLibrary.ensureLoaded()"); + }); + w.blank(); + w.line("/**"); + w.line(" * The exception for a native failure: through error domain [domain] for a"); + w.line(" * throwing call, or a [NativeBugException] (domain 0) for one that can't fail."); + w.line(" */"); + w.line("@JvmStatic"); + w.block( + "fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): Throwable {", + "}", + |w| { + w.line("val text = decodeUtf8(message)"); + w.block("return when (domain) {", "}", |w| { + for (i, eb) in n.domains(model) { + w.line(format!( + "{i} -> {}.fromCode(code, text, payload)", + n.exception(eb) + )); + } + w.line("else -> NativeBugException(code, text)"); + }); + }, + ); + if model.has_callback_interfaces() { + w.blank(); + w.line("/**"); + w.line(" * Reports a failed callback through its vtable entry's `out_err` ([err]):"); + w.line(" * [typed], a domain error the method is declared to throw, with its code and"); + w.line(" * payload fields, and anything else as code -4 with its message."); + w.line(" */"); + w.block( + "private fun fail(err: Long, error: Throwable, typed: FfiException?) {", + "}", + |w| { + w.line("if (typed != null) {"); + w.scope(|w| { + w.line("error_set(err, typed.code, encodeUtf8(typed.message ?: \"\"))"); + w.line("typed.encodePayload()?.let { error_set_payload(err, it) }"); + }); + w.line("} else {"); + w.scope(|w| { + w.line("error_set(err, -4, encodeUtf8(error.message ?: error.toString()))"); + }); + w.line("}"); + }, + ); + w.blank(); + w.line("@JvmStatic external fun error_set(err: Long, code: Int, message: ByteArray)"); + w.line("@JvmStatic external fun error_set_payload(err: Long, payload: ByteArray)"); + } + w.blank(); + w.line("@JvmStatic external fun debug_live(kind: Int): Long"); + if model.has_async() { + w.line("@JvmStatic external fun cancel_token_create(): Long"); + w.line("@JvmStatic external fun cancel_token_cancel(token: Long)"); + w.line("@JvmStatic external fun cancel_token_destroy(token: Long)"); + } + for m in &model.modules { + for i in &m.interfaces { + w.line(format!( + "@JvmStatic external fun {}(_self: Long): Long", + n.native(&i.clone_symbol) + )); + w.line(format!( + "@JvmStatic external fun {}(_self: Long)", + n.native(&i.destroy_symbol) + )); + } + for f in m.callables() { + callable_natives(w, n, f); + } + } + for (owner, cb) in model.callback_interfaces() { + for m in &cb.methods { + w.blank(); + callback_shim(w, n, cb, m, model.error_domain(owner)); + } + } + }); + w.finish() +} diff --git a/crates/weaveffi-gen/src/targets/kotlin/calls.rs b/crates/weaveffi-cli/src/targets/kotlin/calls.rs similarity index 67% rename from crates/weaveffi-gen/src/targets/kotlin/calls.rs rename to crates/weaveffi-cli/src/targets/kotlin/calls.rs index 450ebdf0..d543a06d 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/calls.rs +++ b/crates/weaveffi-cli/src/targets/kotlin/calls.rs @@ -4,22 +4,46 @@ //! native call. use crate::codegen::CodeWriter; -use weaveffi_model::model::{CallShape, ErrorBinding, FnBinding, ParamBinding, Ty}; +use weaveffi_model::model::{CallShape, ErrorBinding, FnBinding, ParamBinding}; +use weaveffi_model::ty::{Family, Prim, Ty}; use crate::targets::kotlin::codec::{decode_expr, encode_expr}; -use crate::targets::kotlin::names::{kt_param, Names}; +use crate::targets::kotlin::names::{jni_kind, kt_param, unsigned_conversions, Names}; -/// One line of a generated expression, with its depth relative to the -/// expression's first line. +/// One line of a generated body, with its depth relative to the body's +/// first line. type Line = (usize, String); -/// The Kotlin expression lowering a non-object argument `expr` of `t` to its -/// JNI form. +/// `expr` in receiver position: as is when it's a postfix chain (a name, +/// member, call, or index, with no space outside its brackets), else +/// parenthesized. +fn receiver(expr: &str) -> String { + let mut depth = 0usize; + for c in expr.chars() { + match c { + '(' | '[' | '{' => depth += 1, + ')' | ']' | '}' => depth = depth.saturating_sub(1), + c if c.is_whitespace() && depth == 0 => return format!("({expr})"), + _ => {} + } + } + expr.to_string() +} + +/// The Kotlin expression lowering the public value `expr` of `t` to its JNI +/// form. An object lowers to a new strong reference (a callback method's +/// return, which the producer adopts); call arguments borrow objects +/// instead (see [`emit_callable`]). pub(crate) fn lower(n: &Names, t: &Ty, expr: &str) -> String { - match t { - Ty::Enum(_) => format!("{expr}.value"), - Ty::StringUtf8 => format!("encodeUtf8({expr})"), - _ if t.is_buffered() => encode_expr(n, t, expr), + if let Some((_, to_signed)) = unsigned_conversions(t) { + return format!("{}.{to_signed}()", receiver(expr)); + } + match t.family() { + Family::Direct if matches!(t, Ty::Enum(_)) => format!("{}.value", receiver(expr)), + Family::String => format!("encodeUtf8({expr})"), + Family::Buffer => encode_expr(n, t, expr), + Family::Object { nullable: false } => format!("{}.cloneHandle()", receiver(expr)), + Family::Object { nullable: true } => format!("{}?.cloneHandle() ?: 0L", receiver(expr)), _ => expr.to_string(), } } @@ -28,14 +52,17 @@ pub(crate) fn lower(n: &Names, t: &Ty, expr: &str) -> String { /// public value. Objects are adopted: the wrapper owes the reference's /// release. pub(crate) fn lift(n: &Names, t: &Ty, expr: &str) -> String { + if let Some((to_unsigned, _)) = unsigned_conversions(t) { + return format!("{}.{to_unsigned}()", receiver(expr)); + } match t { Ty::Enum(name) => format!("{}.fromValue({expr})", n.ty(name)), - Ty::StringUtf8 => format!("decodeUtf8({expr})"), + Ty::Prim(Prim::String) => format!("decodeUtf8({expr})"), Ty::Interface(name) => format!("{}.fromHandle({expr})", n.ty(name)), Ty::Optional(inner) if !t.is_buffered() => { let name = inner .interface_name() - .expect("only Interface? is unbuffered"); + .expect("only Interface? is an unbuffered optional value"); format!("{}.fromHandleOrNull({expr})", n.ty(name)) } _ if t.is_buffered() => decode_expr(n, t, expr), @@ -45,6 +72,7 @@ pub(crate) fn lift(n: &Names, t: &Ty, expr: &str) -> String { /// The borrow scopes and JNI arguments for a call to `f`: the receiver /// (when `has_self`) and every object argument open one `borrow` scope each, +/// a callback interface passes the implementing object (the shim pins it), /// and every other argument lowers inline. fn lower_args(n: &Names, f: &FnBinding, has_self: bool) -> (Vec, Vec) { let mut opens = Vec::new(); @@ -55,16 +83,17 @@ fn lower_args(n: &Names, f: &FnBinding, has_self: bool) -> (Vec, Vec { + match p.ty.family() { + Family::Object { nullable: false } => { opens.push(format!("{name}.handle.borrow {{ _h{i} ->")); args.push(format!("_h{i}")); } - Ty::Optional(_) if !p.ty.is_buffered() => { + Family::Object { nullable: true } => { opens.push(format!("{name}?.handle.borrowOrNull {{ _h{i} ->")); args.push(format!("_h{i}")); } - t => args.push(lower(n, t, &name)), + Family::Callback { .. } => args.push(name), + _ => args.push(lower(n, &p.ty, &name)), } } (opens, args) @@ -84,23 +113,31 @@ fn nest(opens: &[String], body: Vec) -> Vec { out } +/// Write `lines` one level below the writer's depth plus each line's own. +fn write_lines(w: &mut CodeWriter, lines: impl IntoIterator) { + for (d, l) in lines { + for _ in 0..d { + w.indent(); + } + w.line(l); + for _ in 0..d { + w.dedent(); + } + } +} + /// Emit a function declaration `sig` (everything before `=` or `{`) whose /// body is the expression `lines`: an expression body when the function /// returns a value, a block body otherwise. fn emit_body(w: &mut CodeWriter, sig: &str, returns: bool, lines: Vec) { - let base = w.depth(); if returns { let mut iter = lines.into_iter(); let (_, first) = iter.next().expect("a body has at least one line"); w.line(format!("{sig} = {first}")); - for (d, l) in iter { - w.raw(format!("{}{l}\n", " ".repeat(base + d))); - } + write_lines(w, iter); } else { w.line(format!("{sig} {{")); - for (d, l) in lines { - w.raw(format!("{}{l}\n", " ".repeat(base + 1 + d))); - } + w.scope(|w| write_lines(w, lines)); w.line("}"); } } @@ -115,19 +152,21 @@ pub(crate) fn params_sig(n: &Names, params: &[ParamBinding]) -> String { } /// The `: Type` return annotation of a public wrapper, or nothing for void. -fn ret_sig(n: &Names, ret: Option<&Ty>) -> String { +pub(crate) fn ret_sig(n: &Names, ret: Option<&Ty>) -> String { ret.map(|t| format!(": {}", n.kt_type(t))) .unwrap_or_default() } /// The `@Deprecated` annotation for a deprecation message. pub(crate) fn deprecated_line(msg: &str) -> String { - format!( - "@Deprecated(\"{}\")", - msg.replace('\\', "\\\\") - .replace('"', "\\\"") - .replace('$', "\\$") - ) + format!("@Deprecated(\"{}\")", kt_string(msg)) +} + +/// Escape `s` for the inside of a Kotlin string literal. +pub(crate) fn kt_string(s: &str) -> String { + s.replace('\\', "\\\\") + .replace('"', "\\\"") + .replace('$', "\\$") } /// The step function of a `NativeIterator`: pulls one element through the @@ -162,15 +201,15 @@ fn iterator_step(n: &Names, f: &FnBinding) -> Vec { } /// How an iterator's `_next` native hands back a direct or object element: -/// through a one-element primitive array (strings, bytes, and buffers come -/// back as a nullable `ByteArray` instead). +/// through a one-element primitive array of its JNI carrier (strings, +/// bytes, and buffers come back as a nullable `ByteArray` instead). pub(crate) struct ItemSlot { /// The Kotlin array type (`IntArray`). - pub kotlin_array: &'static str, + pub kotlin_array: String, /// The JNI array type (`jintArray`). - pub jni_array: &'static str, + pub jni_array: String, /// The JNI element type (`jint`). - pub jni_elem: &'static str, + pub jni_elem: String, /// The JNI array-region setter stem (`Int` in `SetIntArrayRegion`). pub region: &'static str, } @@ -178,29 +217,24 @@ pub(crate) struct ItemSlot { /// The slot an iterator element of `t` comes back through, or `None` for /// the `ByteArray` families. pub(crate) fn item_slot(t: &Ty) -> Option { - if matches!(t, Ty::StringUtf8 | Ty::Bytes) || t.is_buffered() { - return None; + match t.family() { + Family::Direct | Family::Object { .. } => { + let kind = jni_kind(t); + Some(ItemSlot { + kotlin_array: format!("{kind}Array"), + jni_array: format!("j{}Array", kind.to_lowercase()), + jni_elem: format!("j{}", kind.to_lowercase()), + region: kind, + }) + } + _ => None, } - let (kotlin_array, jni_array, jni_elem, region) = match t { - Ty::Bool => ("BooleanArray", "jbooleanArray", "jboolean", "Boolean"), - Ty::I8 | Ty::U8 => ("ByteArray", "jbyteArray", "jbyte", "Byte"), - Ty::I16 | Ty::U16 => ("ShortArray", "jshortArray", "jshort", "Short"), - Ty::I32 | Ty::Enum(_) => ("IntArray", "jintArray", "jint", "Int"), - Ty::F32 => ("FloatArray", "jfloatArray", "jfloat", "Float"), - Ty::F64 => ("DoubleArray", "jdoubleArray", "jdouble", "Double"), - _ => ("LongArray", "jlongArray", "jlong", "Long"), - }; - Some(ItemSlot { - kotlin_array, - jni_array, - jni_elem, - region, - }) } /// Emit the public wrapper for callable `f`: `decl` is everything before the -/// parameter list (`fun count`, `operator fun invoke`, `suspend fun fetch` -/// is added for async callables), `has_self` borrows the receiver. +/// parameter list (`fun count`, `operator fun invoke`; `suspend` is added +/// for async callables), `has_self` borrows the receiver, and `error` is the +/// domain a throwing callable maps failures through. pub(crate) fn emit_callable( w: &mut CodeWriter, n: &Names, @@ -263,9 +297,10 @@ pub(crate) fn emit_callable( ); let convert = match &f.ret { None => "{ }".to_string(), - // `lift` only ever places its operand alone or in argument - // position, so the bare cast needs no parentheses. - Some(t) => format!("{{ {} }}", lift(n, t, &format!("it as {}", n.jni_type(t)))), + Some(t) => format!( + "{{ _raw -> {} }}", + lift(n, t, &format!("_raw as {}", n.jni_type(t))) + ), }; let token = if f.cancellable { "_token" } else { "_" }; let mut body = vec![( diff --git a/crates/weaveffi-cli/src/targets/kotlin/codec.rs b/crates/weaveffi-cli/src/targets/kotlin/codec.rs new file mode 100644 index 00000000..f2680212 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/kotlin/codec.rs @@ -0,0 +1,198 @@ +//! Value-buffer codec: the Kotlin statement writing and the expression +//! reading every wire shape, dispatched on [`Ty::wire`], and `Codecs.kt`, +//! which holds one `pack`/`unpack` pair per distinct composite type +//! (`[Entry]`, `{string:i64}`, `Shape?`) so no call site inlines a loop. +//! The runtime's `BufferWriter`/`BufferReader` name one method per [`Prim`] +//! (`writeI32`, `readString`), so primitives need a single arm. +//! +//! [`Prim`]: weaveffi_model::ty::Prim + +use std::collections::BTreeMap; + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{FieldBinding, Model, ParamBinding}; +use weaveffi_model::ty::{Ty, WireType}; + +use crate::targets::kotlin::names::Names; + +/// The name of a composite's codec functions after `pack`/`unpack`, spelled +/// in prefix order so it's unambiguous: `[Entry]` is `ListOfEntry`, +/// `{string:[i32]}` is `MapOfStringToListOfI32`, `Shape?` is +/// `OptionalOfShape`. +fn codec_name(n: &Names, t: &Ty) -> String { + match t { + Ty::Prim(p) => p.pascal().to_string(), + Ty::Optional(inner) => format!("OptionalOf{}", codec_name(n, inner)), + Ty::List(inner) => format!("ListOf{}", codec_name(n, inner)), + Ty::Map(k, v) => format!("MapOf{}To{}", codec_name(n, k), codec_name(n, v)), + other => n.ty(other.user_name().expect("only user types remain")), + } +} + +/// The Kotlin statement writing `expr` (the public Kotlin value of `t`) into +/// the writer `w`. +/// +/// An object is written as a token carrying a new strong reference +/// (`cloneHandle()`), so the encoding never hands over the reference the +/// wrapper still owns; the writer releases the token again if the encoding +/// fails before the buffer reaches the producer. +pub(crate) fn write_expr(n: &Names, t: &Ty, w: &str, expr: &str) -> String { + match t.wire() { + WireType::Prim(p) => format!("{w}.write{}({expr})", p.pascal()), + WireType::Enum(_) => format!("{w}.writeI32({expr}.value)"), + WireType::Object(name) => format!( + "{w}.writeObject({expr}.cloneHandle(), JniBridge::{})", + n.native(n.destroy_symbol(name)) + ), + WireType::User(name) => format!("pack{}({w}, {expr})", n.ty(name)), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("pack{}({w}, {expr})", codec_name(n, t)) + } + } +} + +/// The Kotlin expression reading a value of `t` from the reader `r`; the +/// inverse of [`write_expr`]. An object token is adopted into a new wrapper, +/// which owes the reference's release. +pub(crate) fn read_expr(n: &Names, t: &Ty, r: &str) -> String { + match t.wire() { + WireType::Prim(p) => format!("{r}.read{}()", p.pascal()), + WireType::Enum(name) => format!("{}.fromValue({r}.readI32())", n.ty(name)), + WireType::Object(name) => format!("{}.fromHandle({r}.readObject())", n.ty(name)), + WireType::User(name) => format!("unpack{}({r})", n.ty(name)), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("unpack{}({r})", codec_name(n, t)) + } + } +} + +/// The Kotlin expression encoding the public value `expr` of `t` into a +/// fresh value buffer. +pub(crate) fn encode_expr(n: &Names, t: &Ty, expr: &str) -> String { + format!("encodeBuffer {{ _w -> {} }}", write_expr(n, t, "_w", expr)) +} + +/// The Kotlin expression decoding the value buffer `expr` into the public +/// value of `t`. +pub(crate) fn decode_expr(n: &Names, t: &Ty, expr: &str) -> String { + format!("decodeBuffer({expr}) {{ _r -> {} }}", read_expr(n, t, "_r")) +} + +/// Every distinct composite type (optional, list, or map) that crosses +/// inside a value buffer, by codec name. +fn composites(n: &Names, model: &Model) -> BTreeMap { + /// A type in a top-level position: a parameter, return, or iterator + /// element. `Interface?` and `Cb?` aren't buffers there. + fn top(n: &Names, t: &Ty, out: &mut BTreeMap) { + match t { + Ty::Iterator(elem) => top(n, elem, out), + _ if t.is_buffered() => nested(n, t, out), + _ => {} + } + } + /// A type inside a value buffer, where every optional is encoded. + fn nested(n: &Names, t: &Ty, out: &mut BTreeMap) { + match t { + Ty::Optional(inner) | Ty::List(inner) => { + out.insert(codec_name(n, t), t.clone()); + nested(n, inner, out); + } + Ty::Map(k, v) => { + out.insert(codec_name(n, t), t.clone()); + nested(n, k, out); + nested(n, v, out); + } + _ => {} + } + } + let mut out = BTreeMap::new(); + let fields = |fs: &[FieldBinding], out: &mut BTreeMap| { + for f in fs { + nested(n, &f.ty, out); + } + }; + let signature = |ps: &[ParamBinding], ret: &Option, out: &mut BTreeMap| { + for p in ps { + top(n, &p.ty, out); + } + if let Some(t) = ret { + top(n, t, out); + } + }; + for m in &model.modules { + for s in &m.structs { + fields(&s.fields, &mut out); + } + for e in &m.enums { + for v in &e.variants { + fields(&v.fields, &mut out); + } + } + if let Some(eb) = &m.errors { + for c in &eb.codes { + fields(&c.fields, &mut out); + } + } + for f in m.callables() { + signature(&f.params, &f.ret, &mut out); + } + for cb in &m.callback_interfaces { + for f in &cb.methods { + signature(&f.params, &f.ret, &mut out); + } + } + } + out +} + +/// The body of `Codecs.kt` (after the package line), or `None` when no +/// composite type crosses in a value buffer. +pub(crate) fn render_codecs(n: &Names, model: &Model) -> Option { + let all = composites(n, model); + if all.is_empty() { + return None; + } + let mut w = CodeWriter::four_space(); + for t in all.values() { + let name = codec_name(n, t); + let kt = n.kt_type(t); + let (write, read) = match t { + Ty::Optional(inner) => ( + format!( + "_w.writeOptional(_v) {{ {} }}", + write_expr(n, inner, "_w", "it") + ), + format!("_r.readOptional {{ {} }}", read_expr(n, inner, "_r")), + ), + Ty::List(inner) => ( + format!( + "_w.writeList(_v) {{ {} }}", + write_expr(n, inner, "_w", "it") + ), + format!("_r.readList {{ {} }}", read_expr(n, inner, "_r")), + ), + Ty::Map(k, v) => ( + format!( + "_w.writeMap(_v, {{ {} }}, {{ {} }})", + write_expr(n, k, "_w", "it"), + write_expr(n, v, "_w", "it") + ), + format!( + "_r.readMap({{ {} }}, {{ {} }})", + read_expr(n, k, "_r"), + read_expr(n, v, "_r") + ), + ), + _ => unreachable!("only composites are collected"), + }; + w.blank(); + w.line(format!( + "internal fun pack{name}(_w: BufferWriter, _v: {kt}) = {write}" + )); + w.blank(); + w.line(format!( + "internal fun unpack{name}(_r: BufferReader): {kt} = {read}" + )); + } + Some(w.finish()) +} diff --git a/crates/weaveffi-cli/src/targets/kotlin/docs.rs b/crates/weaveffi-cli/src/targets/kotlin/docs.rs new file mode 100644 index 00000000..b9a1462c --- /dev/null +++ b/crates/weaveffi-cli/src/targets/kotlin/docs.rs @@ -0,0 +1,49 @@ +//! KDoc comment emission. + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use weaveffi_model::model::ParamBinding; + +use crate::targets::kotlin::names::kt_param; + +/// Emit one line of a KDoc block: ` * text`, or ` *` for an empty line. +fn kdoc_line(w: &mut CodeWriter, prefix: &str, text: &str) { + if text.is_empty() { + w.line(" *"); + } else { + w.line(format!(" *{prefix}{text}")); + } +} + +/// Emit the KDoc block of a function: its doc plus one `@param` tag per +/// documented parameter, named with its Kotlin spelling. Without parameter +/// docs this is the plain doc comment. +pub(crate) fn fn_doc(w: &mut CodeWriter, doc: &Option, params: &[ParamBinding]) { + let documented: Vec<(String, &str)> = params + .iter() + .filter_map(|p| { + let d = p.doc.as_deref()?.trim(); + (!d.is_empty()).then(|| (kt_param(&p.name), d)) + }) + .collect(); + if documented.is_empty() { + w.doc(doc, DocCommentStyle::Javadoc); + return; + } + w.line("/**"); + if let Some(d) = doc.as_deref().map(str::trim).filter(|d| !d.is_empty()) { + for line in d.lines() { + kdoc_line(w, " ", line); + } + } + for (name, d) in documented { + let mut lines = d.lines(); + if let Some(first) = lines.next() { + w.line(format!(" * @param {name} {first}")); + } + for line in lines { + kdoc_line(w, " ", line); + } + } + w.line(" */"); +} diff --git a/crates/weaveffi-gen/src/targets/kotlin/entities.rs b/crates/weaveffi-cli/src/targets/kotlin/entities.rs similarity index 76% rename from crates/weaveffi-gen/src/targets/kotlin/entities.rs rename to crates/weaveffi-cli/src/targets/kotlin/entities.rs index d0456cf4..8cf67047 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/entities.rs +++ b/crates/weaveffi-cli/src/targets/kotlin/entities.rs @@ -3,18 +3,20 @@ //! and the module object holding the free functions (nested modules become //! nested objects). -use crate::codegen::common::pascal_case; +use crate::codegen::common::{pascal_case, DocCommentStyle}; use crate::codegen::CodeWriter; use weaveffi_model::errors; use weaveffi_model::model::{ - BindingModel, CallbackInterfaceBinding, EnumBinding, ErrorBinding, FieldBinding, - InterfaceBinding, ModuleBinding, StructBinding, + CallbackInterfaceBinding, EnumBinding, ErrorBinding, FieldBinding, InterfaceBinding, Model, + ModuleBinding, StructBinding, }; -use crate::targets::kotlin::calls::{deprecated_line, emit_callable}; +use crate::targets::kotlin::calls::{ + deprecated_line, emit_callable, kt_string, params_sig, ret_sig, +}; use crate::targets::kotlin::codec::{read_expr, write_expr}; -use crate::targets::kotlin::docs::{camel_params, writer_doc, writer_fn_doc}; -use crate::targets::kotlin::names::{kt_escape, kt_member, kt_param, Names}; +use crate::targets::kotlin::docs::fn_doc; +use crate::targets::kotlin::names::{kt_escape, kt_member, Names}; /// Emit `@Deprecated` for a deprecated declaration. fn deprecated(w: &mut CodeWriter, msg: &Option) { @@ -39,7 +41,7 @@ fn emit_properties(w: &mut CodeWriter, n: &Names, head: &str, fields: &[FieldBin w.line(format!("{head}(")); w.scope(|w| { for (f, decl) in fields.iter().zip(field_decls(n, fields)) { - writer_doc(w, &f.doc); + w.doc(&f.doc, DocCommentStyle::Javadoc); w.line(format!("{decl},")); } }); @@ -56,12 +58,12 @@ fn emit_properties(w: &mut CodeWriter, n: &Names, head: &str, fields: &[FieldBin fn render_enum(w: &mut CodeWriter, n: &Names, e: &EnumBinding) { let name = n.ty(&e.name); w.blank(); - writer_doc(w, &e.doc); + w.doc(&e.doc, DocCommentStyle::Javadoc); deprecated(w, &e.deprecated); w.line(format!("enum class {name}(val value: Int) {{")); w.scope(|w| { for (i, v) in e.variants.iter().enumerate() { - writer_doc(w, &v.doc); + w.doc(&v.doc, DocCommentStyle::Javadoc); let end = if i + 1 == e.variants.len() { ";" } else { "," }; w.line(format!("{}({}){end}", kt_escape(&v.name), v.value)); } @@ -73,7 +75,7 @@ fn render_enum(w: &mut CodeWriter, n: &Names, e: &EnumBinding) { "fun fromValue(value: Int): {name} = entries.firstOrNull {{ it.value == value }}" )); w.line(format!( - " ?: throw FfiException(-3, \"unknown {name} value $value\")" + " ?: throw NativeBugException(-3, \"unknown {name} value $value\")" )); }); w.line("}"); @@ -86,12 +88,12 @@ fn render_enum(w: &mut CodeWriter, n: &Names, e: &EnumBinding) { fn render_rich_enum(w: &mut CodeWriter, n: &Names, e: &EnumBinding) { let name = n.ty(&e.name); w.blank(); - writer_doc(w, &e.doc); + w.doc(&e.doc, DocCommentStyle::Javadoc); deprecated(w, &e.deprecated); w.line(format!("sealed class {name} {{")); w.scope(|w| { for v in &e.variants { - writer_doc(w, &v.doc); + w.doc(&v.doc, DocCommentStyle::Javadoc); let vn = kt_escape(&pascal_case(&v.name)); if v.fields.is_empty() { w.line(format!("object {vn} : {name}()")); @@ -124,7 +126,7 @@ fn render_rich_enum(w: &mut CodeWriter, n: &Names, e: &EnumBinding) { w.line(format!("_w.writeI32({})", v.value)); for f in &v.fields { let expr = format!("_v.{}", kt_escape(&f.name)); - w.line(write_expr(n, &f.ty, "_w", &expr, 0)); + w.line(write_expr(n, &f.ty, "_w", &expr)); } }); w.line("}"); @@ -150,7 +152,7 @@ fn render_rich_enum(w: &mut CodeWriter, n: &Names, e: &EnumBinding) { } } w.line(format!( - "else -> throw FfiException(-3, \"malformed value buffer: unknown {name} tag $_tag\")" + "else -> throw NativeBugException(-3, \"malformed value buffer: unknown {name} tag $_tag\")" )); }); w.line("}"); @@ -161,7 +163,7 @@ fn render_rich_enum(w: &mut CodeWriter, n: &Names, e: &EnumBinding) { fn render_struct(w: &mut CodeWriter, n: &Names, s: &StructBinding) { let name = n.ty(&s.name); w.blank(); - writer_doc(w, &s.doc); + w.doc(&s.doc, DocCommentStyle::Javadoc); deprecated(w, &s.deprecated); if s.fields.is_empty() { w.line(format!("class {name}")); @@ -183,7 +185,7 @@ fn render_struct(w: &mut CodeWriter, n: &Names, s: &StructBinding) { w.scope(|w| { for f in &s.fields { let expr = format!("_v.{}", kt_escape(&f.name)); - w.line(write_expr(n, &f.ty, "_w", &expr, 0)); + w.line(write_expr(n, &f.ty, "_w", &expr)); } }); w.line("}"); @@ -200,10 +202,17 @@ fn render_struct(w: &mut CodeWriter, n: &Names, s: &StructBinding) { } /// A callback interface: the Kotlin `interface` the consumer implements. -/// Its JNI dispatch shims live on `JniBridge`. -fn render_callback_interface(w: &mut CodeWriter, n: &Names, cb: &CallbackInterfaceBinding) { +/// Its JNI dispatch shims live on `JniBridge`. A method declared `throws` +/// may throw the module's domain exception, which reaches the producer with +/// its code and fields; anything else it throws reaches it as code -4. +fn render_callback_interface( + w: &mut CodeWriter, + n: &Names, + cb: &CallbackInterfaceBinding, + error: Option<&ErrorBinding>, +) { w.blank(); - writer_doc(w, &cb.doc); + w.doc(&cb.doc, DocCommentStyle::Javadoc); deprecated(w, &cb.deprecated); w.line(format!("interface {} {{", n.ty(&cb.name))); w.scope(|w| { @@ -211,22 +220,26 @@ fn render_callback_interface(w: &mut CodeWriter, n: &Names, cb: &CallbackInterfa if i > 0 { w.blank(); } - writer_fn_doc(w, &m.doc, &camel_params(&m.params)); + let doc = match (m.throws, error) { + (true, Some(eb)) => { + let throws = format!( + "Throw a [{}] to report a typed error to the library.", + n.exception(eb) + ); + Some(match m.doc.as_deref().map(str::trim) { + Some(d) if !d.is_empty() => format!("{d}\n\n{throws}"), + _ => throws, + }) + } + _ => m.doc.clone(), + }; + fn_doc(w, &doc, &m.params); deprecated(w, &m.deprecated); - let params: Vec = m - .params - .iter() - .map(|p| format!("{}: {}", kt_param(&p.name), n.kt_type(&p.ty))) - .collect(); - let ret = m - .ret - .as_ref() - .map(|t| format!(": {}", n.kt_type(t))) - .unwrap_or_default(); w.line(format!( - "fun {}({}){ret}", + "fun {}({}){}", kt_member(&m.name), - params.join(", ") + params_sig(n, &m.params), + ret_sig(n, m.ret.as_ref()) )); } }); @@ -247,7 +260,7 @@ fn render_interface( let destroy = n.native(&i.destroy_symbol); let clone = n.native(&i.clone_symbol); w.blank(); - writer_doc(w, &i.doc); + w.doc(&i.doc, DocCommentStyle::Javadoc); deprecated(w, &i.deprecated); // The address constructor is private so a `new` constructor taking one // `Long` (exposed as `invoke`) is never shadowed by it. @@ -265,7 +278,7 @@ fn render_interface( )); for f in &i.methods { w.blank(); - writer_fn_doc(w, &f.doc, &camel_params(&f.params)); + fn_doc(w, &f.doc, &f.params); emit_callable(w, n, f, &format!("fun {}", kt_member(&f.name)), true, error); } w.blank(); @@ -283,7 +296,7 @@ fn render_interface( )); for c in &i.constructors { w.blank(); - writer_fn_doc(w, &c.doc, &camel_params(&c.params)); + fn_doc(w, &c.doc, &c.params); let decl = if c.name == "new" { "operator fun invoke".to_string() } else { @@ -293,7 +306,7 @@ fn render_interface( } for f in &i.statics { w.blank(); - writer_fn_doc(w, &f.doc, &camel_params(&f.params)); + fn_doc(w, &f.doc, &f.params); emit_callable(w, n, f, &format!("fun {}", kt_member(&f.name)), false, error); } }); @@ -303,9 +316,11 @@ fn render_interface( } /// An error domain: a sealed exception class with one subclass per code -/// (payload fields as constructor properties) and the `fromCode` factory -/// the bridge maps raw codes through. Unknown codes, including every -/// runtime (negative) code, map to the generic `FfiException`. +/// (payload fields as constructor properties, then the message with the +/// documented default) and the `fromCode` factory the bridge maps raw codes +/// through. Unknown codes, including every runtime (negative) code, map to +/// the generic `FfiException`. A subclass with fields encodes them for a +/// callback that throws it. fn render_error_domain(w: &mut CodeWriter, n: &Names, eb: &ErrorBinding) { let exc = n.exception(eb); w.blank(); @@ -315,16 +330,30 @@ fn render_error_domain(w: &mut CodeWriter, n: &Names, eb: &ErrorBinding) { )); w.scope(|w| { for ec in &eb.codes { - writer_doc(w, &ec.doc); + w.doc(&ec.doc, DocCommentStyle::Javadoc); let class = errors::pascal(&ec.name); - let default = ec.message.replace('\\', "\\\\").replace('"', "\\\"").replace('$', "\\$"); - let mut params = vec![format!("message: String = \"{default}\"")]; - params.extend(field_decls(n, &ec.fields)); - w.line(format!( + let mut params = field_decls(n, &ec.fields); + params.push(format!("message: String = \"{}\"", kt_string(&ec.message))); + let head = format!( "class {class}({}) : {exc}({}, message)", params.join(", "), ec.value - )); + ); + if ec.fields.is_empty() { + w.line(head); + continue; + } + w.block(format!("{head} {{"), "}", |w| { + w.block( + "override fun encodePayload(): ByteArray = encodeBuffer { _w ->", + "}", + |w| { + for f in &ec.fields { + w.line(write_expr(n, &f.ty, "_w", &kt_escape(&f.name))); + } + }, + ); + }); } w.blank(); w.line("internal companion object {"); @@ -340,7 +369,7 @@ fn render_error_domain(w: &mut CodeWriter, n: &Names, eb: &ErrorBinding) { let reads: Vec = ec.fields.iter().map(|f| read_expr(n, &f.ty, "_r")).collect(); w.line(format!( - "{} -> if (payload == null) FfiException(code, message) else decodeBuffer(payload) {{ _r -> {class}(message, {}) }}", + "{} -> if (payload == null) FfiException(code, message) else decodeBuffer(payload) {{ _r -> {class}({}, message) }}", ec.value, reads.join(", ") )); @@ -356,18 +385,14 @@ fn render_error_domain(w: &mut CodeWriter, n: &Names, eb: &ErrorBinding) { } /// Whether `m` or any module below it declares a free function. -fn has_functions(model: &BindingModel, m: &ModuleBinding) -> bool { +fn has_functions(model: &Model, m: &ModuleBinding) -> bool { !m.functions.is_empty() || model.children(m).any(|c| has_functions(model, c)) } /// The module object for `m`: its free functions, then one nested object /// per submodule that has functions of its own. -fn render_module_object(w: &mut CodeWriter, n: &Names, model: &BindingModel, m: &ModuleBinding) { - // The model falls back to a function's doc for an undocumented module; - // that reads wrong on the object, so only a module's own doc is kept. - if !m.functions.iter().any(|f| f.doc == m.doc) { - writer_doc(w, &m.doc); - } +fn render_module_object(w: &mut CodeWriter, n: &Names, model: &Model, m: &ModuleBinding) { + w.doc(&m.doc, DocCommentStyle::Javadoc); w.line(format!("object {} {{", n.object(m))); w.scope(|w| { let mut first = true; @@ -376,14 +401,14 @@ fn render_module_object(w: &mut CodeWriter, n: &Names, model: &BindingModel, m: w.blank(); } first = false; - writer_fn_doc(w, &f.doc, &camel_params(&f.params)); + fn_doc(w, &f.doc, &f.params); emit_callable( w, n, f, - &format!("fun {}", n.function(m, f)), + &format!("fun {}", n.function(f)), false, - m.error.as_ref(), + model.error_domain(m), ); } for child in model.children(m).filter(|c| has_functions(model, c)) { @@ -399,14 +424,14 @@ fn render_module_object(w: &mut CodeWriter, n: &Names, model: &BindingModel, m: /// Render every declaration of the top-level module `root` and its /// submodules (the body of `{Root}.kt`, after the package line). -pub(crate) fn render_root(n: &Names, model: &BindingModel, root: &ModuleBinding) -> String { +pub(crate) fn render_root(n: &Names, model: &Model, root: &ModuleBinding) -> String { let mut w = CodeWriter::four_space(); let tree = model .modules .iter() .filter(|m| m.segments.first() == root.segments.first()); for m in tree { - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { + if let Some(eb) = m.errors.as_ref() { render_error_domain(&mut w, n, eb); } for e in &m.enums { @@ -420,10 +445,10 @@ pub(crate) fn render_root(n: &Names, model: &BindingModel, root: &ModuleBinding) render_struct(&mut w, n, s); } for cb in &m.callback_interfaces { - render_callback_interface(&mut w, n, cb); + render_callback_interface(&mut w, n, cb, model.error_domain(m)); } for i in &m.interfaces { - render_interface(&mut w, n, i, m.error.as_ref()); + render_interface(&mut w, n, i, model.error_domain(m)); } } if has_functions(model, root) { diff --git a/crates/weaveffi-gen/src/targets/kotlin/jni.rs b/crates/weaveffi-cli/src/targets/kotlin/jni.rs similarity index 62% rename from crates/weaveffi-gen/src/targets/kotlin/jni.rs rename to crates/weaveffi-cli/src/targets/kotlin/jni.rs index 3342e9aa..f3391e3a 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/jni.rs +++ b/crates/weaveffi-cli/src/targets/kotlin/jni.rs @@ -9,13 +9,14 @@ //! callable's lowered [`AbiFn`] slots, so it matches the header exactly. use crate::codegen::CodeWriter; -use crate::utils::local_type_name; use weaveffi_model::abi::{AbiParam, CType}; +use weaveffi_model::contract; use weaveffi_model::model::{ - checksum_symbol, AbiFn, BindingModel, CallShape, CallbackInterfaceBinding, Family, FnBinding, - InterfaceBinding, IteratorBinding, Ty, + contract_symbol, AbiFn, CallShape, CallbackInterfaceBinding, CallbackMethodBinding, FnBinding, + InterfaceBinding, IteratorBinding, Model, }; use weaveffi_model::plan::{ArgPass, RetPass}; +use weaveffi_model::ty::{Family, Prim, Ty}; use crate::targets::kotlin::bridge::{shim_descriptor, shim_name}; use crate::targets::kotlin::calls::item_slot; @@ -42,6 +43,11 @@ fn trampoline(n: &Names, cb: &CallbackInterfaceBinding, method: &str) -> String format!("Jni_{}_{method}", n.native(&cb.c_tag)) } +/// The expected contract table of the top-level module `root`. +fn contract_var(root: &str) -> String { + format!("Jni_contract_{root}") +} + /// The pointee of a pointer slot (an iterator's `out_item`). fn pointee(t: &CType) -> &CType { match t { @@ -62,7 +68,7 @@ struct Export { /// Lower `f`'s receiver and parameters onto the slots of `abi`, then the /// trailing slots (`out_len`, `out_err`, `cancel_token`, `callback`, /// `context`) by name. -fn lower_export(n: &Names, model: &BindingModel, f: &FnBinding, abi: &AbiFn, done: &str) -> Export { +fn lower_export(n: &Names, model: &Model, f: &FnBinding, abi: &AbiFn, done: &str) -> Export { let p = &n.prefix; let mut e = Export { params: Vec::new(), @@ -84,7 +90,7 @@ fn lower_export(n: &Names, model: &BindingModel, f: &FnBinding, abi: &AbiFn, don let name = c_param(¶m.name); e.params.push(format!("{} {name}", jni_c_type(¶m.ty))); match param.arg_pass() { - ArgPass::Direct { slot } => e.args.push(if matches!(param.ty, Ty::Bool) { + ArgPass::Direct { slot } => e.args.push(if matches!(param.ty, Ty::Prim(Prim::Bool)) { format!("{name} == JNI_TRUE") } else { format!("({}){name}", slot.ty.render_c(p)) @@ -101,20 +107,24 @@ fn lower_export(n: &Names, model: &BindingModel, f: &FnBinding, abi: &AbiFn, don e.args .push(format!("({})(intptr_t){name}", slot.ty.render_c(p))); } - ArgPass::Callback { .. } => { - let cb_name = param - .ty - .callback_interface_name() - .expect("callback slots name a callback interface"); - let (_, cb) = model - .callback_interfaces() - .find(|(_, c)| c.name == local_type_name(cb_name)) - .expect("callback interfaces resolve"); + ArgPass::Callback { nullable, .. } => { + let cb = model.callback_interface( + param + .ty + .callback_interface_name() + .expect("callback slots name a callback interface"), + ); // The producer owns this global reference from here on and // drops it through the vtable's `free`. - e.args - .push(format!("(void*)(*env)->NewGlobalRef(env, {name})")); - e.args.push(format!("&{}", vtable_var(n, cb))); + let ctx = format!("(void*)(*env)->NewGlobalRef(env, {name})"); + let vtable = format!("&{}", vtable_var(n, cb)); + if nullable { + e.args.push(format!("{name} != NULL ? {ctx} : NULL")); + e.args.push(format!("{name} != NULL ? {vtable} : NULL")); + } else { + e.args.push(ctx); + e.args.push(vtable); + } } } } @@ -156,7 +166,7 @@ fn error_check(w: &mut CodeWriter, domain: u32, fail: &str) { /// The JNI value of a direct C value `v` of `t`. fn direct_to_jni(t: &Ty, v: &str) -> String { - if matches!(t, Ty::Bool) { + if matches!(t, Ty::Prim(Prim::Bool)) { format!("{v} ? JNI_TRUE : JNI_FALSE") } else { format!("({}){v}", jni_c_type(t)) @@ -167,7 +177,7 @@ fn direct_to_jni(t: &Ty, v: &str) -> String { fn render_sync( w: &mut CodeWriter, n: &Names, - model: &BindingModel, + model: &Model, f: &FnBinding, abi: &AbiFn, domain: u32, @@ -177,23 +187,20 @@ fn render_sync( let e = lower_export(n, model, f, abi, ""); let is_iter = matches!(f.shape, CallShape::Iterator(_)); let ret = if is_iter { - RetPass::Direct + RetPass::Object { nullable: false } } else { - weaveffi_model::plan::ret_pass(f.ret.as_ref(), p) + RetPass::of(f.ret.as_ref()) }; - let (jret, fail) = match (&ret, is_iter) { - (_, true) => ("jlong".to_string(), " 0"), - (RetPass::Void, _) => ("void".to_string(), ""), - (RetPass::String | RetPass::Bytes | RetPass::Buffer, _) => { - ("jbyteArray".to_string(), " NULL") - } - (RetPass::Object { .. }, _) => ("jlong".to_string(), " 0"), - (RetPass::Direct, _) => ( - jni_c_type(f.ret.as_ref().expect("direct returns have a type")).to_string(), + let (jret, fail) = match &ret { + RetPass::Void => ("void", ""), + RetPass::String | RetPass::Bytes | RetPass::Buffer => ("jbyteArray", " NULL"), + RetPass::Object { .. } => ("jlong", " 0"), + RetPass::Direct => ( + jni_c_type(f.ret.as_ref().expect("direct returns have a type")), " 0", ), }; - open_export(w, n, &jret, &native, &e.params); + open_export(w, n, jret, &native, &e.params); w.scope(|w| { w.line(format!("{p}_error err = {{0, NULL, NULL, 0}};")); w.line("(void)cls;"); @@ -201,9 +208,9 @@ fn render_sync( w.line(s); } let call = format!("{}({})", abi.symbol, e.args.join(", ")); - match (&ret, is_iter) { - (RetPass::Void, false) => w.line(format!("{call};")), - (RetPass::String | RetPass::Bytes | RetPass::Buffer, false) => { + match &ret { + RetPass::Void => w.line(format!("{call};")), + RetPass::String | RetPass::Bytes | RetPass::Buffer => { w.line("size_t out_len = 0;"); w.line(format!("{} rv = {call};", abi.ret.render_c(p))) } @@ -213,15 +220,15 @@ fn render_sync( w.line(s); } error_check(w, domain, fail); - match (&ret, is_iter) { - (_, true) | (RetPass::Object { .. }, _) => { + match &ret { + RetPass::Void => {} + RetPass::Object { .. } => { w.line("return (jlong)(intptr_t)rv;"); } - (RetPass::Void, _) => {} - (RetPass::String | RetPass::Bytes | RetPass::Buffer, _) => { + RetPass::String | RetPass::Bytes | RetPass::Buffer => { w.line("return Jni_take_bytes(env, rv, out_len);"); } - (RetPass::Direct, _) => { + RetPass::Direct => { let t = f.ret.as_ref().expect("direct returns have a type"); w.line(format!("return {};", direct_to_jni(t, "rv"))); } @@ -232,7 +239,7 @@ fn render_sync( } /// An async callable: its completion trampoline, then the launcher export. -fn render_async(w: &mut CodeWriter, n: &Names, model: &BindingModel, f: &FnBinding) { +fn render_async(w: &mut CodeWriter, n: &Names, model: &Model, f: &FnBinding) { let CallShape::Async(ab) = &f.shape else { unreachable!("render_async needs an async call shape"); }; @@ -248,11 +255,12 @@ fn render_async(w: &mut CodeWriter, n: &Names, model: &BindingModel, f: &FnBindi "static void {done}(void* context, {p}_error* err{decls}) {{" )); w.scope(|w| { - w.line("JNIEnv* env = Jni_complete_begin(context, err);"); + w.line("int detach = 0;"); + w.line("JNIEnv* env = Jni_complete_begin(context, err, &detach);"); w.line("if (env == NULL) {"); w.line(" return;"); w.line("}"); - let deliver = match weaveffi_model::plan::ret_pass(f.ret.as_ref(), p) { + let deliver = match RetPass::of(f.ret.as_ref()) { RetPass::Void => "Jni_on_unit".to_string(), RetPass::String | RetPass::Bytes | RetPass::Buffer => { "Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)".to_string() @@ -270,7 +278,7 @@ fn render_async(w: &mut CodeWriter, n: &Names, model: &BindingModel, f: &FnBindi w.line(format!( "(*env)->CallVoidMethod(env, (jobject)context, {deliver});" )); - w.line("Jni_complete_end(env, context);"); + w.line("Jni_complete_end(env, context, detach);"); }); w.line("}"); w.blank(); @@ -298,7 +306,6 @@ fn render_iterator_natives(w: &mut CodeWriter, n: &Names, it: &IteratorBinding, let p = &n.prefix; let tag = &it.iter_tag; let item_ty = pointee(&it.next.params[1].ty).render_c(p); - let has_len = it.next.params.iter().any(|s| s.name == "out_len"); let next = n.native(&it.next.symbol); match item_slot(&it.elem) { None => { @@ -331,18 +338,17 @@ fn render_iterator_natives(w: &mut CodeWriter, n: &Names, it: &IteratorBinding, w.line(format!("{p}_error err = {{0, NULL, NULL, 0}};")); w.line(format!("{item_ty} item = ({item_ty})0;")); w.line("(void)cls;"); - let len = if has_len { "&out_len, " } else { "" }; w.line(format!( - "int32_t has = {}(({tag}*)(intptr_t)iter, &item, {len}&err);", + "int32_t has = {}(({tag}*)(intptr_t)iter, &item, &err);", it.next.symbol )); error_check(w, domain, " JNI_FALSE"); w.line("if (has == 0) {"); w.line(" return JNI_FALSE;"); w.line("}"); - let value = match &it.elem { - Ty::Interface(_) | Ty::Optional(_) => "(jlong)(intptr_t)item".to_string(), - t => direct_to_jni(t, "item"), + let value = match it.elem.family() { + Family::Object { .. } => "(jlong)(intptr_t)item".to_string(), + _ => direct_to_jni(&it.elem, "item"), }; w.line(format!("{} value = {value};", slot.jni_elem)); w.line(format!( @@ -407,10 +413,135 @@ fn render_interface_natives(w: &mut CodeWriter, n: &Names, i: &InterfaceBinding) w.blank(); } -/// One callback interface: the cached shim method IDs, one trampoline per -/// method, and the static vtable. -fn render_callback_interface(w: &mut CodeWriter, n: &Names, cb: &CallbackInterfaceBinding) { +/// One trampoline: calls the method's dispatch shim with the implementation +/// (`ctx`), the `out_err` address, and every argument in its JNI form, then +/// hands the result back in the vtable entry's return or out slots. +fn render_trampoline( + w: &mut CodeWriter, + n: &Names, + model: &Model, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, +) { let p = &n.prefix; + // `ctx` first and `out_err` last; the parameters' slots, then the + // return's out slots, in between. + let inner = &m.abi_params[1..m.abi_params.len() - 1]; + let param_slots: usize = m.params.iter().map(|x| x.abi.len()).sum(); + let (params, outs) = inner.split_at(param_slots); + let mut decls = vec!["void* ctx".to_string()]; + decls.extend( + params + .iter() + .map(|s| format!("{} {}", s.ty.render_c(p), c_param(&s.name))), + ); + decls.extend( + outs.iter() + .map(|s| format!("{} {}", s.ty.render_c(p), s.name)), + ); + decls.push(format!("{p}_error* out_err")); + let ret_c = m.abi_ret.render_c(p); + let ret = RetPass::of(m.ret.as_ref()); + let fail = match ret { + RetPass::Direct | RetPass::Object { .. } => format!(" ({ret_c})0"), + _ => String::new(), + }; + w.line(format!( + "static {ret_c} {}({}) {{", + trampoline(n, cb, &m.name), + decls.join(", ") + )); + w.scope(|w| { + w.line("int detach = 0;"); + w.line("JNIEnv* env = Jni_callback_begin(out_err, &detach);"); + w.line("if (env == NULL) {"); + w.scope(|w| { + // The object arguments are ours to release when the call can't + // reach the implementation. + let mut slots = params.iter(); + for param in &m.params { + let own: Vec<&AbiParam> = param.abi.iter().filter_map(|_| slots.next()).collect(); + if let Some(name) = param.ty.interface_name() { + w.line(format!( + "{}({});", + model.interface(name).destroy_symbol, + c_param(&own[0].name) + )); + } + } + w.line(format!("return{fail};")); + }); + w.line("}"); + let mut args = vec![ + "(jobject)ctx".to_string(), + "(jlong)(intptr_t)out_err".to_string(), + ]; + let mut slots = params.iter(); + for param in &m.params { + let own: Vec<&AbiParam> = param.abi.iter().filter_map(|_| slots.next()).collect(); + args.push(match param.ty.family() { + Family::String | Family::Bytes | Family::Buffer => format!( + "Jni_new_bytes(env, {}, {})", + c_param(&own[0].name), + c_param(&own[1].name) + ), + Family::Object { .. } => format!("(jlong)(intptr_t){}", c_param(&own[0].name)), + _ => direct_to_jni(¶m.ty, &c_param(&own[0].name)), + }); + } + let kind = match (&ret, m.ret.as_ref()) { + (RetPass::Void, _) => "Void", + (RetPass::String | RetPass::Bytes | RetPass::Buffer, _) => "Object", + (RetPass::Object { .. }, _) => "Long", + (RetPass::Direct, Some(t)) => jni_kind(t), + (RetPass::Direct, None) => unreachable!("direct returns have a type"), + }; + let call = format!( + "(*env)->CallStatic{kind}Method(env, Jni_bridge, {}, {})", + shim_mid(n, cb, &m.name), + args.join(", ") + ); + match (&ret, m.ret.as_ref()) { + (RetPass::Void, _) => { + w.line(format!("{call};")); + w.line("Jni_callback_end(env, out_err, detach);"); + } + (RetPass::String | RetPass::Bytes | RetPass::Buffer, _) => { + w.line(format!("jbyteArray rv = (jbyteArray){call};")); + w.line("Jni_callback_return_bytes(env, rv, out_ptr, out_len, out_err);"); + w.line("Jni_callback_end(env, out_err, detach);"); + } + (RetPass::Object { .. }, _) => { + w.line(format!("jlong rv = {call};")); + w.line("Jni_callback_end(env, out_err, detach);"); + w.line(format!("return ({ret_c})(intptr_t)rv;")); + } + (RetPass::Direct, Some(t)) => { + // A failure leaves the zero value the shim (or JNI) returned. + w.line(format!("{} rv = {call};", jni_c_type(t))); + w.line("Jni_callback_end(env, out_err, detach);"); + if matches!(t, Ty::Prim(Prim::Bool)) { + w.line("return rv == JNI_TRUE;"); + } else { + w.line(format!("return ({ret_c})rv;")); + } + } + (RetPass::Direct, None) => unreachable!("direct returns have a type"), + } + }); + w.line("}"); + w.blank(); +} + +/// One callback interface: the cached shim method IDs, one trampoline per +/// method, and the static vtable (header first: its size, flags 0, and the +/// `free` hook that drops the implementation's global reference). +fn render_callback_interface( + w: &mut CodeWriter, + n: &Names, + model: &Model, + cb: &CallbackInterfaceBinding, +) { for m in &cb.methods { w.line(format!( "static jmethodID {} = NULL;", @@ -419,82 +550,14 @@ fn render_callback_interface(w: &mut CodeWriter, n: &Names, cb: &CallbackInterfa } w.blank(); for m in &cb.methods { - // The first slot is `ctx` and the last `out_err`; everything between - // belongs to the method's parameters, in order. - let inner = &m.abi_params[1..m.abi_params.len() - 1]; - let mut decls = vec!["void* ctx".to_string()]; - decls.extend( - inner - .iter() - .map(|s| format!("{} {}", s.ty.render_c(p), c_param(&s.name))), - ); - decls.push(format!("{p}_error* out_err")); - let ret_c = m.abi_ret.render_c(p); - let fail = if m.ret.is_some() { - format!(" ({ret_c})0") - } else { - String::new() - }; - w.line(format!( - "static {ret_c} {}({}) {{", - trampoline(n, cb, &m.name), - decls.join(", ") - )); - w.scope(|w| { - w.line("JNIEnv* env = Jni_callback_begin(out_err);"); - w.line("if (env == NULL) {"); - w.line(format!(" return{fail};")); - w.line("}"); - let mut args = vec!["(jobject)ctx".to_string()]; - let mut slots = inner.iter(); - for param in &m.params { - let own: Vec<&AbiParam> = param.abi.iter().filter_map(|_| slots.next()).collect(); - let arg = match param.ty.family() { - Family::String | Family::Bytes | Family::Buffer => format!( - "Jni_new_bytes(env, {}, {})", - c_param(&own[0].name), - c_param(&own[1].name) - ), - Family::Object { .. } => { - format!("(jlong)(intptr_t){}", c_param(&own[0].name)) - } - _ => direct_to_jni(¶m.ty, &c_param(&own[0].name)), - }; - args.push(arg); - } - let kind = m.ret.as_ref().map(jni_kind).unwrap_or("Void"); - let call = format!( - "(*env)->CallStatic{kind}Method(env, Jni_bridge, {}, {})", - shim_mid(n, cb, &m.name), - args.join(", ") - ); - match &m.ret { - None => { - w.line(format!("{call};")); - w.line("Jni_callback_end(env, out_err);"); - } - Some(t) => { - w.line(format!("{} rv = {call};", jni_c_type(t))); - w.line("if (Jni_callback_end(env, out_err)) {"); - w.line(format!(" return{fail};")); - w.line("}"); - if matches!(t, Ty::Bool) { - w.line("return rv == JNI_TRUE;"); - } else { - w.line(format!("return ({ret_c})rv;")); - } - } - } - }); - w.line("}"); - w.blank(); + render_trampoline(w, n, model, cb, m); } - let mut entries: Vec = cb - .methods - .iter() - .map(|m| trampoline(n, cb, &m.name)) - .collect(); - entries.push("Jni_release_callback".to_string()); + let mut entries = vec![ + format!("sizeof({})", cb.vtable_tag), + "0".to_string(), + "Jni_release_callback".to_string(), + ]; + entries.extend(cb.methods.iter().map(|m| trampoline(n, cb, &m.name))); w.line(format!( "static const {} {} = {{{}}};", cb.vtable_tag, @@ -504,22 +567,42 @@ fn render_callback_interface(w: &mut CodeWriter, n: &Names, cb: &CallbackInterfa w.blank(); } -/// `Jni_load`: the per-module contract checks and the method IDs the -/// generated code caches. -fn render_load(w: &mut CodeWriter, n: &Names, model: &BindingModel, name: &str) { +/// The contract tables these bindings were generated with (one per +/// top-level module that declares anything), then `Jni_load`: the contract +/// checks and the method IDs the generated code caches. +fn render_load(w: &mut CodeWriter, n: &Names, model: &Model) { + let mut checked = Vec::new(); + for root in model.roots() { + let entries = contract::entries(model, root); + if entries.is_empty() { + continue; + } + w.line(format!( + "static const Jni_contract_entry {}[] = {{", + contract_var(&root.name) + )); + w.scope(|w| { + for e in &entries { + w.line(format!( + "{{UINT64_C(0x{:016x}), UINT64_C(0x{:016x}), \"{}\"}},", + e.id, e.hash, e.path + )); + } + }); + w.line("};"); + w.blank(); + checked.push(root); + } w.line("static jint Jni_load(JNIEnv* env) {"); w.scope(|w| { w.line("(void)env;"); - for root in model.roots() { - let Some(sum) = root.checksum else { continue }; - w.line(format!( - "if ({}() != UINT64_C(0x{sum:016x})) {{", - checksum_symbol(&n.prefix, &root.name) - )); + for root in checked { + let var = contract_var(&root.name); w.line(format!( - " return Jni_load_error(env, \"{name}: module '{}' does not match these bindings (its contract checksum differs); regenerate the bindings or rebuild the library\");", - root.name + "if (Jni_check_contract(env, {}, {var}, sizeof {var} / sizeof {var}[0]) != JNI_OK) {{", + contract_symbol(&n.prefix, &root.name) )); + w.line(" return JNI_ERR;"); w.line("}"); } if model.has_async() { @@ -527,11 +610,6 @@ fn render_load(w: &mut CodeWriter, n: &Names, model: &BindingModel, name: &str) w.line(" return JNI_ERR;"); w.line("}"); } - if model.has_callback_interfaces() { - w.line("if (Jni_load_callbacks(env) != JNI_OK) {"); - w.line(" return JNI_ERR;"); - w.line("}"); - } for (_, cb) in model.callback_interfaces() { for m in &cb.methods { let mid = shim_mid(n, cb, &m.name); @@ -552,19 +630,19 @@ fn render_load(w: &mut CodeWriter, n: &Names, model: &BindingModel, name: &str) } /// Render the generated part of the shim (after the runtime templates). -pub(crate) fn render_exports(n: &Names, model: &BindingModel, name: &str) -> String { +pub(crate) fn render_exports(n: &Names, model: &Model) -> String { let mut w = CodeWriter::four_space(); w.blank(); for (_, cb) in model.callback_interfaces() { - render_callback_interface(&mut w, n, cb); + render_callback_interface(&mut w, n, model, cb); } - render_load(&mut w, n, model, name); + render_load(&mut w, n, model); for m in &model.modules { for i in &m.interfaces { render_interface_natives(&mut w, n, i); } for f in m.callables() { - let domain = n.domain(f, m.error.as_ref()); + let domain = n.domain(f, model.error_domain(m)); match &f.shape { CallShape::Sync(abi) => render_sync(&mut w, n, model, f, abi, domain), CallShape::Iterator(it) => { diff --git a/crates/weaveffi-gen/src/targets/kotlin/mod.rs b/crates/weaveffi-cli/src/targets/kotlin/mod.rs similarity index 54% rename from crates/weaveffi-gen/src/targets/kotlin/mod.rs rename to crates/weaveffi-cli/src/targets/kotlin/mod.rs index 3dabe792..6f20f568 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/mod.rs +++ b/crates/weaveffi-cli/src/targets/kotlin/mod.rs @@ -3,19 +3,25 @@ //! Emits a Gradle library module (`kotlin/`) in the identity's package //! (`{prefix}` unless configured): Kotlin sources over a JNI shim //! (`lib{library}_jni`) that calls the C ABI of the producer library -//! (`lib{library}`). The default flavor is an Android library (the shim builds -//! through the NDK's CMake); the `jvm` flavor is a plain Kotlin/JVM library -//! whose shim is built with CMake directly. +//! (`lib{library}`). The default flavor is an Android library; the `jvm` +//! flavor is a plain Kotlin/JVM library. `weaveffi build` prebuilds the shim +//! per platform and `weaveffi package` bundles it; without a prebuilt shim, +//! the Android flavor builds it through the NDK's CMake and the JVM flavor +//! with CMake directly. //! //! The Kotlin surface: records as `data class`es and rich enums as sealed -//! classes (decoded from value buffers), C-style enums as `enum class`es, -//! interfaces as `AutoCloseable` wrappers whose calls borrow the native -//! reference (so neither `close()` nor the phantom-reference cleaner frees it -//! mid-call), `iter` returns as `NativeIterator`, async callables as -//! `suspend fun`s (cancellation cancels the native token), callback -//! interfaces as Kotlin `interface`s, and one `object` per module holding its -//! free functions. `JNI_OnLoad` checks the ABI revision and every top-level -//! module's contract checksum before the first call. +//! classes (decoded from value buffers, with one codec function per +//! composite type in `Codecs.kt`), C-style enums as `enum class`es, unsigned +//! integers as `UByte`/`UShort`/`UInt`/`ULong`, interfaces as +//! `AutoCloseable` wrappers whose calls borrow the native reference (so +//! neither `close()` nor the phantom-reference cleaner frees it mid-call), +//! `iter` returns as `NativeIterator`, async callables as `suspend +//! fun`s (cancellation cancels the native token), callback interfaces as +//! Kotlin `interface`s (whose methods return any family and may throw the +//! module's domain exception), and one `object` per module holding its free +//! functions. A failure of a call that can't fail raises the unchecked +//! `NativeBugException`. `JNI_OnLoad` checks the ABI revision and every +//! top-level module's contract table before the first call. mod bridge; mod calls; @@ -29,12 +35,11 @@ mod runtime; use camino::{Utf8Path, Utf8PathBuf}; use serde::{Deserialize, Serialize}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; +use weaveffi_model::model::Model; use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; +use crate::package::{Artifact, PackageContext, PackagedFile}; +use crate::targets::kotlin::runtime::jni_library; use crate::utils::{render_prelude, render_trailer, CommentStyle}; use crate::targets::kotlin::names::{resolve_package, Names}; @@ -43,12 +48,13 @@ use crate::targets::kotlin::names::{resolve_package, Names}; #[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "lowercase")] pub enum KotlinFlavor { - /// An Android library module (`com.android.library`) whose JNI shim - /// builds through the NDK's CMake. + /// An Android library module (`com.android.library`) whose JNI shim is + /// prebuilt under `src/main/jniLibs/` or builds through the NDK's CMake. #[default] Android, - /// A plain Kotlin/JVM library; the JNI shim is built with CMake - /// separately and loaded from `java.library.path`. + /// A plain Kotlin/JVM library whose JNI shim is a prebuilt classpath + /// resource or is built with CMake separately and loaded from + /// `java.library.path`. Jvm, } @@ -58,26 +64,23 @@ pub enum KotlinFlavor { pub struct KotlinConfig { /// The Kotlin package of the generated sources (default: the identity's /// C prefix, such as `kvstore`). - pub package: Option, + pub name: Option, /// The Gradle module flavor (default `android`). pub flavor: KotlinFlavor, - /// When `true` (the default), free functions drop their module path from - /// their Kotlin name (`Contacts.count()` rather than - /// `Contacts.contactsCount()`). - pub strip_module_prefix: bool, - /// Basename of the IDL the CLI was invoked with. - #[serde(skip)] - pub input_basename: Option, + /// The Android flavor's `minSdk` (default 21). + pub min_sdk: u32, + /// The Android flavor's `compileSdk` (default 35). + pub compile_sdk: u32, } impl Default for KotlinConfig { - /// Android flavor, module prefixes stripped, package from the identity. + /// Android flavor, package from the identity. fn default() -> Self { Self { - package: None, + name: None, flavor: KotlinFlavor::Android, - strip_module_prefix: true, - input_basename: None, + min_sdk: 21, + compile_sdk: 35, } } } @@ -86,13 +89,7 @@ impl KotlinConfig { /// The configured Kotlin package, if any; the identity's prefix applies /// otherwise. pub fn package(&self) -> Option<&str> { - self.package.as_deref() - } - - /// Returns the input IDL basename embedded in generated file headers, - /// falling back to `"api.yml"`. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") + self.name.as_deref() } } @@ -101,25 +98,18 @@ pub struct KotlinGenerator; /// Everything one generation run renders from. struct Plan<'a> { - api: &'a ResolvedApi, - model: &'a BindingModel, + model: &'a Model, names: Names, config: &'a KotlinConfig, dir: Utf8PathBuf, } impl<'a> Plan<'a> { - fn new( - api: &'a ResolvedApi, - model: &'a BindingModel, - out_dir: &Utf8Path, - config: &'a KotlinConfig, - ) -> Self { - let id = api.identity(); + fn new(model: &'a Model, out_dir: &Utf8Path, config: &'a KotlinConfig) -> Self { + let id = &model.identity; let package = resolve_package(config.package(), &id.prefix); - let names = Names::new(model, &package, &id.library, config.strip_module_prefix); + let names = Names::new(model, &package, &id.library); Self { - api, model, names, config, @@ -130,7 +120,7 @@ impl<'a> Plan<'a> { /// Wrap a Kotlin source body: header, optional deprecation suppression, /// the package line, the body, and the trailer. fn kotlin_file(&self, file: &str, body: &str, suppress_deprecation: bool) -> String { - let mut out = render_prelude(CommentStyle::DoubleSlash, self.config.input_basename()); + let mut out = render_prelude(CommentStyle::DoubleSlash); if suppress_deprecation { out.push_str("@file:Suppress(\"DEPRECATION\")\n\n"); } @@ -145,16 +135,15 @@ impl<'a> Plan<'a> { fn runtime_file(&self, file: &str, source: &str) -> String { format!( "{}{source}\n{}", - render_prelude(CommentStyle::DoubleSlash, self.config.input_basename()), + render_prelude(CommentStyle::DoubleSlash), render_trailer(CommentStyle::DoubleSlash, file) ) } fn files(&self) -> Vec<(Utf8PathBuf, String)> { - let id = self.api.identity(); + let id = &self.model.identity; let n = &self.names; let model = self.model; - let input = self.config.input_basename(); let src = self .dir .join("src/main/kotlin") @@ -167,30 +156,30 @@ impl<'a> Plan<'a> { let mut files = vec![ ( self.dir.join("settings.gradle.kts"), - package::settings_gradle(id, self.config.flavor, input), + package::settings_gradle(id, self.config.flavor), ), ( self.dir.join("build.gradle.kts"), - package::build_gradle(id, &n.package, self.config.flavor, model.has_async(), input), + package::build_gradle(id, &n.package, self.config, model.has_async()), ), ]; if self.config.flavor == KotlinFlavor::Android { files.push(( self.dir.join("consumer-rules.pro"), - package::consumer_rules(&n.package, input), + package::consumer_rules(&n.package), )); } - files.push((cpp.join("CMakeLists.txt"), package::cmake_lists(id, input))); + files.push((cpp.join("CMakeLists.txt"), package::cmake_lists(id))); files.push(( cpp.join(&header), - crate::targets::c::render_c_header_from_model(model, input, &header), + crate::targets::c::render_c_header_from_model(model, &header), )); let c_name = id.name.replace('\\', "\\\\").replace('"', "\\\""); let jni_source = format!( "{}{}{}\n{}", - render_prelude(CommentStyle::DoubleSlash, input), + render_prelude(CommentStyle::DoubleSlash), runtime::jni_runtime(n, model, &header, &c_name), - jni::render_exports(n, model, &c_name), + jni::render_exports(n, model), render_trailer(CommentStyle::DoubleSlash, &jni_c) ); files.push((cpp.join(&jni_c), jni_source)); @@ -211,6 +200,12 @@ impl<'a> Plan<'a> { self.runtime_file("Async.kt", &runtime::async_kt(n)), )); } + if let Some(body) = codec::render_codecs(n, model) { + files.push(( + src.join("Codecs.kt"), + self.kotlin_file("Codecs.kt", &body, deprecations), + )); + } files.push(( src.join("JniBridge.kt"), self.kotlin_file( @@ -233,7 +228,7 @@ impl<'a> Plan<'a> { /// Whether generated code references a deprecated declaration (a type or a /// callback method), so its files suppress the warning. -fn uses_deprecated(model: &BindingModel) -> bool { +fn uses_deprecated(model: &Model) -> bool { model.modules.iter().any(|m| { m.enums.iter().any(|e| e.deprecated.is_some()) || m.structs.iter().any(|s| s.deprecated.is_some()) @@ -251,65 +246,84 @@ impl LanguageBackend for KotlinGenerator { "kotlin" } - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - Plan::new(api, model, out_dir, config) + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + Plan::new(model, out_dir, config) .files() .into_iter() .map(|(path, contents)| OutputFile::new(path, contents)) .collect() } + /// The Gradle project at `kotlin/{name}/` with the prebuilt natives: the + /// producer library and JNI shim per Android ABI under + /// `src/main/jniLibs//` (so Gradle skips the CMake build), and per + /// desktop platform under `src/main/resources/natives//`, + /// where the loader extracts them from the classpath. fn package( &self, - api: &ResolvedApi, - model: &BindingModel, + model: &Model, ctx: &PackageContext, - out_dir: &Utf8Path, config: &Self::Config, - ) -> Option> { - let plan = Plan::new(api, model, out_dir, config); + ) -> Option> { + let id = &model.identity; + let jni = jni_library(&id.library); + let android: Vec<_> = ctx + .binaries + .binaries + .iter() + .filter(|nb| nb.platform.android_abi().is_some()) + .collect(); + // Prebuilt shims ship only when every Android ABI has one; otherwise + // the CMake build makes all of them (a prebuilt and a built copy of + // the same ABI would collide in the AAR). + let prebuilt_android = + !android.is_empty() && android.iter().all(|nb| nb.jni_shim.is_some()); + let mut natives = Vec::new(); + for nb in &ctx.binaries.binaries { + let dir = if let Some(abi) = nb.platform.android_abi() { + format!("src/main/jniLibs/{abi}") + } else if nb.platform.is_desktop() { + format!("src/main/resources/natives/{}", nb.platform.id()) + } else { + continue; + }; + natives.push(PackagedFile::copy( + format!("{dir}/{}", ctx.binaries.bundled_filename(nb.platform)), + nb.library.clone(), + )); + let shim = match &nb.jni_shim { + Some(shim) if nb.platform.android_abi().is_none() || prebuilt_android => shim, + _ => continue, + }; + natives.push(PackagedFile::copy( + format!("{dir}/{}", nb.platform.lib_filename(&jni)), + shim.clone(), + )); + } + if natives.is_empty() { + return Some(Vec::new()); + } + let plan = Plan::new(model, Utf8Path::new(""), config); let mut files: Vec = plan .files() .into_iter() - .map(|(path, contents)| PackagedFile::text(path, contents)) + .map(|(path, contents)| { + let path = path + .strip_prefix("kotlin") + .map(Utf8Path::to_path_buf) + .unwrap_or(path); + PackagedFile::text(path, contents) + }) .collect(); files.push(PackagedFile::text( - plan.dir.join("README.md"), - package::packaged_readme( - api.identity(), - &plan.names.package, - ctx, - config.input_basename(), - ), + "README.md", + package::packaged_readme(id, &plan.names.package, ctx), )); - // Android binaries land in `jniLibs//`, where the shim's CMake - // build links them and the AAR packages them; desktop binaries are - // classpath resources the loader extracts at run time. - for nb in &ctx.binaries.binaries { - let filename = ctx.binaries.bundled_filename(nb.platform); - if let Some(abi) = nb.platform.android_abi() { - let dest = plan.dir.join("src/main/jniLibs").join(abi).join(filename); - files.push(PackagedFile::copy(dest, nb.source.clone())); - } else if nb.platform.is_desktop() { - let dest = plan - .dir - .join("src/main/resources/natives") - .join(nb.platform.id()) - .join(filename); - files.push(PackagedFile::copy(dest, nb.source.clone())); - } - } - Some(files) + files.extend(natives); + Some(vec![Artifact::directory( + format!("kotlin/{}", id.name), + files, + )]) } } diff --git a/crates/weaveffi-gen/src/targets/kotlin/names.rs b/crates/weaveffi-cli/src/targets/kotlin/names.rs similarity index 59% rename from crates/weaveffi-gen/src/targets/kotlin/names.rs rename to crates/weaveffi-cli/src/targets/kotlin/names.rs index d31fc220..94f58f95 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/names.rs +++ b/crates/weaveffi-cli/src/targets/kotlin/names.rs @@ -14,9 +14,9 @@ use std::fmt::Write as _; use crate::codegen::common::pascal_case; use crate::lang; -use crate::utils::{local_type_name, wrapper_name}; use weaveffi_model::errors; -use weaveffi_model::model::{BindingModel, ErrorBinding, FnBinding, ModuleBinding, Ty}; +use weaveffi_model::model::{ErrorBinding, FnBinding, Model, ModuleBinding}; +use weaveffi_model::ty::{Family, Prim, Ty}; /// Names user types must not take: Kotlin and Java types the compiler /// imports into every file, plus the declarations of the generated runtime. @@ -65,6 +65,7 @@ pub(crate) const RESERVED_TYPE_NAMES: &[&str] = &[ "NativeCompletion", "NativeHandle", "NativeIterator", + "NativeBugException", "NativeLibrary", "NoSuchElementException", "Nothing", @@ -84,13 +85,24 @@ pub(crate) const RESERVED_TYPE_NAMES: &[&str] = &[ "Thread", "Throwable", "Triple", + "UByte", + "UInt", + "ULong", + "UShort", "Unit", "Volatile", ]; +/// The generated files besides the module objects' (`{Object}.kt`): a module +/// object spelled like one of them gains a `Module` suffix, so the files +/// never collide. +const GENERATED_FILE_STEMS: &[&str] = &["Async", "Buffers", "Codecs", "JniBridge", "Runtime"]; + /// Member names the generated wrapper classes already use (or inherit from -/// `Any`); a method, static, or callback method spelled the same gains a -/// trailing underscore. +/// `Any`, or from `java.lang.Object`, whose final `wait` and `notify` a +/// same-named function would accidentally override), and the companion's +/// `invoke` that a `new` constructor becomes; a method, static, or callback +/// method spelled the same gains a trailing underscore. const RESERVED_MEMBER_NAMES: &[&str] = &[ "cloneHandle", "close", @@ -99,7 +111,11 @@ const RESERVED_MEMBER_NAMES: &[&str] = &[ "fromHandleOrNull", "handle", "hashCode", + "invoke", + "notify", + "notifyAll", "toString", + "wait", ]; /// Escape a user identifier for a Kotlin declaration or expression position: @@ -129,12 +145,7 @@ pub(crate) fn kt_param(name: &str) -> String { /// [`kt_param`], and also escaped away from the members every wrapper /// declares. pub(crate) fn kt_member(name: &str) -> String { - let n = kt_param(name); - if RESERVED_MEMBER_NAMES.contains(&n.as_str()) { - format!("{n}_") - } else { - n - } + lang::escape_member(&kt_param(name), RESERVED_MEMBER_NAMES) } /// Escape a user type name away from keywords and reserved type names. @@ -176,27 +187,26 @@ pub(crate) struct Names { /// The library name the producer is loaded as and the JNI shim derives /// from (`kvstore`). pub library: String, - /// Whether free-function names drop their module path. - pub strip_module_prefix: bool, types: HashMap, exceptions: HashMap, domains: HashMap, objects: HashMap, + /// Every interface's `_destroy` symbol, by interface name. + destroys: HashMap, } impl Names { /// Resolve every name for `model`, with `package` the Kotlin package /// (already validated) and `library` the producer library name. - pub(crate) fn new( - model: &BindingModel, - package: &str, - library: &str, - strip_module_prefix: bool, - ) -> Self { + pub(crate) fn new(model: &Model, package: &str, library: &str) -> Self { let mut types = HashMap::new(); let mut exceptions = HashMap::new(); let mut domains = HashMap::new(); + let mut destroys = HashMap::new(); for m in &model.modules { + for i in &m.interfaces { + destroys.insert(i.name.clone(), i.destroy_symbol.clone()); + } let declared = m .enums .iter() @@ -207,7 +217,7 @@ impl Names { for name in declared { types.insert(name.clone(), kt_type_ident(name)); } - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { + if let Some(eb) = m.errors.as_ref() { exceptions.insert( eb.c_tag.clone(), kt_type_ident(&errors::exception_type_name(&eb.name)), @@ -222,7 +232,7 @@ impl Names { .iter() .map(|m| { let mut name = kt_type_ident(&pascal_case(&m.name)); - if taken.contains(&name) { + if taken.contains(&name) || GENERATED_FILE_STEMS.contains(&name.as_str()) { name.push_str("Module"); } (m.path.clone(), name) @@ -231,23 +241,33 @@ impl Names { Self { package: package.to_string(), package_path: package.replace('.', "/"), - prefix: model.prefix.clone(), + prefix: model.prefix().to_string(), library: library.to_string(), - strip_module_prefix, types, exceptions, domains, objects, + destroys, } } - /// The Kotlin class of a user type named `ir_name` (bare or dotted). - pub(crate) fn ty(&self, ir_name: &str) -> String { - let local = local_type_name(ir_name); + /// The `_destroy` symbol of the interface `name`. + /// + /// # Panics + /// + /// Panics when no interface has that name, which validation rules out. + pub(crate) fn destroy_symbol(&self, name: &str) -> &str { + self.destroys + .get(name) + .unwrap_or_else(|| panic!("interface '{name}' is not declared")) + } + + /// The Kotlin class of the user type `name`. + pub(crate) fn ty(&self, name: &str) -> String { self.types - .get(local) + .get(name) .cloned() - .unwrap_or_else(|| kt_type_ident(local)) + .unwrap_or_else(|| kt_type_ident(name)) } /// The Kotlin exception class of an error domain. @@ -259,7 +279,8 @@ impl Names { } /// The domain index `JniBridge.error` maps a failure of `f` through: - /// the module's domain for a throwing callable, else 0 (generic). + /// the module's domain for a throwing callable, else 0 (the trap, + /// `NativeBugException`). pub(crate) fn domain(&self, f: &FnBinding, error: Option<&ErrorBinding>) -> u32 { match error { Some(eb) if f.throws => self.domains.get(&eb.c_tag).copied().unwrap_or(0), @@ -268,11 +289,11 @@ impl Names { } /// Every declared domain with its index, in index order. - pub(crate) fn domains<'a>(&self, model: &'a BindingModel) -> Vec<(u32, &'a ErrorBinding)> { + pub(crate) fn domains<'a>(&self, model: &'a Model) -> Vec<(u32, &'a ErrorBinding)> { let mut out: Vec<(u32, &ErrorBinding)> = model .modules .iter() - .filter_map(|m| m.error.as_ref().filter(|e| e.declared_here)) + .filter_map(|m| m.errors.as_ref()) .filter_map(|eb| self.domains.get(&eb.c_tag).map(|i| (*i, eb))) .collect(); out.sort_by_key(|(i, _)| *i); @@ -287,9 +308,9 @@ impl Names { .unwrap_or_else(|| pascal_case(&m.name)) } - /// The Kotlin name of a free function. - pub(crate) fn function(&self, m: &ModuleBinding, f: &FnBinding) -> String { - kt_member(&wrapper_name(&m.path, &f.name, self.strip_module_prefix)) + /// The Kotlin name of a free function: its bare name (names are global). + pub(crate) fn function(&self, f: &FnBinding) -> String { + kt_member(&f.name) } /// The `JniBridge` member name for a C symbol: the symbol without the @@ -315,15 +336,19 @@ impl Names { /// The public Kotlin type of an IR type. pub(crate) fn kt_type(&self, t: &Ty) -> String { match t { - Ty::I8 | Ty::U8 => "Byte".into(), - Ty::I16 | Ty::U16 => "Short".into(), - Ty::I32 => "Int".into(), - Ty::U32 | Ty::I64 | Ty::U64 => "Long".into(), - Ty::F32 => "Float".into(), - Ty::F64 => "Double".into(), - Ty::Bool => "Boolean".into(), - Ty::StringUtf8 => "String".into(), - Ty::Bytes => "ByteArray".into(), + Ty::Prim(Prim::I8) => "Byte".into(), + Ty::Prim(Prim::U8) => "UByte".into(), + Ty::Prim(Prim::I16) => "Short".into(), + Ty::Prim(Prim::U16) => "UShort".into(), + Ty::Prim(Prim::I32) => "Int".into(), + Ty::Prim(Prim::U32) => "UInt".into(), + Ty::Prim(Prim::I64) => "Long".into(), + Ty::Prim(Prim::U64) => "ULong".into(), + Ty::Prim(Prim::F32) => "Float".into(), + Ty::Prim(Prim::F64) => "Double".into(), + Ty::Prim(Prim::Bool) => "Boolean".into(), + Ty::Prim(Prim::String) => "String".into(), + Ty::Prim(Prim::Bytes) => "ByteArray".into(), Ty::Record(n) | Ty::RichEnum(n) | Ty::Enum(n) @@ -336,74 +361,102 @@ impl Names { } } - /// The Kotlin type a value crosses the JNI boundary as: strings, bytes, - /// and value buffers as `ByteArray`, enums as `Int`, objects and - /// iterators as their address (`0L` is none), and a callback interface - /// as the implementing object. + /// The Kotlin type a value crosses the JNI boundary as: unsigned + /// integers in the signed type of the same width (an `external fun` + /// can't take Kotlin's unsigned value classes), enums as `Int`, strings, + /// bytes, and value buffers as `ByteArray`, objects and iterators as + /// their address (`0L` is none), and a callback interface as the + /// implementing object. pub(crate) fn jni_type(&self, t: &Ty) -> String { - match t { - Ty::Enum(_) => "Int".into(), - Ty::StringUtf8 | Ty::Bytes => "ByteArray".into(), - Ty::Interface(_) | Ty::Iterator(_) => "Long".into(), - Ty::CallbackInterface(n) => self.ty(n), - _ if t.is_buffered() => "ByteArray".into(), - Ty::Optional(_) => "Long".into(), - other => self.kt_type(other), + match t.family() { + Family::Direct => jni_kind(t).into(), + Family::String | Family::Bytes | Family::Buffer => "ByteArray".into(), + Family::Object { .. } | Family::Iterator => "Long".into(), + Family::Callback { nullable } => { + let name = t + .callback_interface_name() + .expect("callback families name a callback interface"); + format!("{}{}", self.ty(name), if nullable { "?" } else { "" }) + } } } /// The JVM type descriptor of [`Names::jni_type`], for the method IDs - /// the shim caches. + /// the shim caches; `None` is `void`. pub(crate) fn jni_descriptor(&self, t: Option<&Ty>) -> String { let Some(t) = t else { return "V".into(); }; - match self.jni_type(t).as_str() { - "Byte" => "B".into(), - "Short" => "S".into(), - "Int" => "I".into(), - "Long" => "J".into(), - "Float" => "F".into(), - "Double" => "D".into(), - "Boolean" => "Z".into(), - "ByteArray" => "[B".into(), - class => format!("L{}/{class};", self.package_path), + match t.family() { + Family::Direct => match jni_kind(t) { + "Boolean" => "Z", + "Byte" => "B", + "Short" => "S", + "Int" => "I", + "Float" => "F", + "Double" => "D", + _ => "J", + } + .into(), + Family::String | Family::Bytes | Family::Buffer => "[B".into(), + Family::Object { .. } | Family::Iterator => "J".into(), + Family::Callback { .. } => { + let name = t + .callback_interface_name() + .expect("callback families name a callback interface"); + format!("L{}/{};", self.package_path, self.ty(name)) + } } } } /// The JNI C type of [`Names::jni_type`]. pub(crate) fn jni_c_type(t: &Ty) -> &'static str { - match t { - Ty::I8 | Ty::U8 => "jbyte", - Ty::I16 | Ty::U16 => "jshort", - Ty::I32 | Ty::Enum(_) => "jint", - Ty::U32 | Ty::I64 | Ty::U64 | Ty::Interface(_) | Ty::Iterator(_) => "jlong", - Ty::F32 => "jfloat", - Ty::F64 => "jdouble", - Ty::Bool => "jboolean", - Ty::StringUtf8 | Ty::Bytes => "jbyteArray", - Ty::CallbackInterface(_) => "jobject", - _ if t.is_buffered() => "jbyteArray", - Ty::Optional(_) => "jlong", - _ => unreachable!("every type is covered above"), + match t.family() { + Family::Direct => match jni_kind(t) { + "Boolean" => "jboolean", + "Byte" => "jbyte", + "Short" => "jshort", + "Int" => "jint", + "Float" => "jfloat", + "Double" => "jdouble", + _ => "jlong", + }, + Family::String | Family::Bytes | Family::Buffer => "jbyteArray", + Family::Object { .. } | Family::Iterator => "jlong", + Family::Callback { .. } => "jobject", } } -/// The `Call{Kind}Method` / `on{Kind}` stem for a direct-family or object -/// value: `Boolean`, `Byte`, `Short`, `Int`, `Long`, `Float`, or `Double`. +/// The JNI carrier of a direct-family value, which is also the +/// `Call{Kind}Method` and `on{Kind}` stem: `Boolean`, `Byte`, `Short`, +/// `Int`, `Long`, `Float`, or `Double`. Unsigned integers ride in the signed +/// type of the same width; objects ride in `Long`. pub(crate) fn jni_kind(t: &Ty) -> &'static str { match t { - Ty::Bool => "Boolean", - Ty::I8 | Ty::U8 => "Byte", - Ty::I16 | Ty::U16 => "Short", - Ty::I32 | Ty::Enum(_) => "Int", - Ty::F32 => "Float", - Ty::F64 => "Double", + Ty::Prim(Prim::Bool) => "Boolean", + Ty::Prim(Prim::I8 | Prim::U8) => "Byte", + Ty::Prim(Prim::I16 | Prim::U16) => "Short", + Ty::Prim(Prim::I32 | Prim::U32) | Ty::Enum(_) => "Int", + Ty::Prim(Prim::F32) => "Float", + Ty::Prim(Prim::F64) => "Double", _ => "Long", } } +/// For an unsigned integer type, the Kotlin conversions from its JNI carrier +/// (`toUInt`) and back (`toInt`), both of which reinterpret the bits; `None` +/// for every other type. +pub(crate) fn unsigned_conversions(t: &Ty) -> Option<(&'static str, &'static str)> { + match t { + Ty::Prim(Prim::U8) => Some(("toUByte", "toByte")), + Ty::Prim(Prim::U16) => Some(("toUShort", "toShort")), + Ty::Prim(Prim::U32) => Some(("toUInt", "toInt")), + Ty::Prim(Prim::U64) => Some(("toULong", "toLong")), + _ => None, + } +} + /// Resolve the Kotlin package: the configured one, else the identity /// prefix. A segment that is a Kotlin keyword gains a trailing underscore. pub(crate) fn resolve_package(configured: Option<&str>, prefix: &str) -> String { @@ -415,3 +468,18 @@ pub(crate) fn resolve_package(configured: Option<&str>, prefix: &str) -> String .collect::>() .join(".") } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn members_avoid_the_wrapper_and_object_members() { + assert_eq!(kt_member("close"), "close_"); + assert_eq!(kt_member("to_string"), "toString_"); + assert_eq!(kt_member("wait"), "wait_"); + assert_eq!(kt_member("notify_all"), "notifyAll_"); + assert_eq!(kt_member("dispose"), "dispose"); + assert_eq!(kt_member("object"), "object_"); + } +} diff --git a/crates/weaveffi-gen/src/targets/kotlin/package.rs b/crates/weaveffi-cli/src/targets/kotlin/package.rs similarity index 76% rename from crates/weaveffi-gen/src/targets/kotlin/package.rs rename to crates/weaveffi-cli/src/targets/kotlin/package.rs index d9cdcfb3..544c60d0 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/package.rs +++ b/crates/weaveffi-cli/src/targets/kotlin/package.rs @@ -8,7 +8,7 @@ use crate::utils::{render_prelude, render_trailer, CommentStyle}; use weaveffi_model::pkg::Identity; use crate::targets::kotlin::runtime::jni_library; -use crate::targets::kotlin::KotlinFlavor; +use crate::targets::kotlin::{KotlinConfig, KotlinFlavor}; /// The Kotlin Gradle plugin version the build files pin. const KOTLIN_VERSION: &str = "2.0.21"; @@ -25,17 +25,17 @@ pub(crate) fn kts_quote(s: &str) -> String { } /// Wrap `body` in the generated-file header and trailer. -fn wrap(style: CommentStyle, input_basename: &str, file: &str, body: &str) -> String { +fn wrap(style: CommentStyle, file: &str, body: &str) -> String { format!( "{}{body}\n{}", - render_prelude(style, input_basename), + render_prelude(style), render_trailer(style, file) ) } /// `settings.gradle.kts`: plugin and dependency repositories, so the module /// builds on its own, and the root project named after the package. -pub(crate) fn settings_gradle(id: &Identity, flavor: KotlinFlavor, input_basename: &str) -> String { +pub(crate) fn settings_gradle(id: &Identity, flavor: KotlinFlavor) -> String { let google = match flavor { KotlinFlavor::Android => " google()\n", KotlinFlavor::Jvm => "", @@ -58,22 +58,17 @@ rootProject.name = "{}" "#, kts_quote(&id.name) ); - wrap( - CommentStyle::DoubleSlash, - input_basename, - "settings.gradle.kts", - &body, - ) + wrap(CommentStyle::DoubleSlash, "settings.gradle.kts", &body) } /// `build.gradle.kts` for the configured flavor. pub(crate) fn build_gradle( id: &Identity, package: &str, - flavor: KotlinFlavor, + config: &KotlinConfig, has_async: bool, - input_basename: &str, ) -> String { + let (min_sdk, compile_sdk) = (config.min_sdk, config.compile_sdk); let group = kts_quote(package); let version = kts_quote(&id.version); let artifact = kts_quote(&id.name); @@ -82,7 +77,7 @@ pub(crate) fn build_gradle( } else { String::new() }; - let body = match flavor { + let body = match config.flavor { KotlinFlavor::Android => format!( r#"plugins {{ id("com.android.library") version "{AGP_VERSION}" @@ -93,18 +88,24 @@ pub(crate) fn build_gradle( group = "{group}" version = "{version}" +// A packaged build ships a prebuilt lib{jni}.so next to the producer +// library in src/main/jniLibs//; without one, CMake builds it. +val prebuiltJni = fileTree("src/main/jniLibs") {{ include("**/lib{jni}.so") }}.files.isNotEmpty() + android {{ namespace = "{group}" - compileSdk = 35 + compileSdk = {compile_sdk} defaultConfig {{ - minSdk = 21 + minSdk = {min_sdk} consumerProguardFiles("consumer-rules.pro") }} // Builds lib{jni}.so per ABI, linked against the producer library in // src/main/jniLibs// (see src/main/cpp/CMakeLists.txt). - externalNativeBuild {{ - cmake {{ - path = file("src/main/cpp/CMakeLists.txt") + if (!prebuiltJni) {{ + externalNativeBuild {{ + cmake {{ + path = file("src/main/cpp/CMakeLists.txt") + }} }} }} compileOptions {{ @@ -147,8 +148,10 @@ kotlin {{ group = "{group}" version = "{version}" -// The JNI shim (lib{jni}) is built separately with CMake from -// src/main/cpp and must be on java.library.path at run time. +// The JNI shim (lib{jni}) and the producer library load from the +// classpath's natives// resources (which a packaged build ships +// under src/main/resources/), else from java.library.path, where a shim +// built with CMake from src/main/cpp goes. {deps}java {{ withSourcesJar() @@ -166,32 +169,22 @@ publishing {{ jni = jni_library(&id.library), ), }; - wrap( - CommentStyle::DoubleSlash, - input_basename, - "build.gradle.kts", - &body, - ) + wrap(CommentStyle::DoubleSlash, "build.gradle.kts", &body) } /// `consumer-rules.pro`: the JNI shim finds classes and methods by name, so /// R8 must keep the package intact in consuming apps. -pub(crate) fn consumer_rules(package: &str, input_basename: &str) -> String { +pub(crate) fn consumer_rules(package: &str) -> String { let body = format!( "# The JNI shim looks classes and methods up by name.\n-keep class {package}.** {{ *; }}\n" ); - wrap( - CommentStyle::Hash, - input_basename, - "consumer-rules.pro", - &body, - ) + wrap(CommentStyle::Hash, "consumer-rules.pro", &body) } /// `src/main/cpp/CMakeLists.txt`: builds `lib{library}_jni` from the shim /// and the bundled header, linked against the producer library found in /// `{PREFIX}_LIBRARY_DIR` (on Android, `src/main/jniLibs/` by default). -pub(crate) fn cmake_lists(id: &Identity, input_basename: &str) -> String { +pub(crate) fn cmake_lists(id: &Identity) -> String { let lib = &id.library; let jni = jni_library(lib); let upper = id.macro_prefix(); @@ -224,17 +217,12 @@ if(NOT ANDROID) endif() "# ); - wrap(CommentStyle::Hash, input_basename, "CMakeLists.txt", &body) + wrap(CommentStyle::Hash, "CMakeLists.txt", &body) } /// The packaged module's README: the layout, the bundled binaries, and how /// each runtime loads them. -pub(crate) fn packaged_readme( - id: &Identity, - package: &str, - ctx: &PackageContext, - input_basename: &str, -) -> String { +pub(crate) fn packaged_readme(id: &Identity, package: &str, ctx: &PackageContext) -> String { let jni = jni_library(&id.library); let mut android = Vec::new(); let mut desktop = Vec::new(); @@ -260,28 +248,21 @@ pub(crate) fn packaged_readme( r#"# {name} (Kotlin) Kotlin bindings for `{name}` in the `{package}` package: the Kotlin sources -under `src/main/kotlin/`, the JNI shim and the C header it compiles against -under `src/main/cpp/`, and the prebuilt producer libraries. +under `src/main/kotlin/`, the JNI shim's source and the C header it compiles +against under `src/main/cpp/`, and the prebuilt native libraries. ## Android -The Android Gradle plugin builds `lib{jni}.so` for each ABI against the -bundled `src/main/jniLibs//lib{lib}.so` and packages both into the AAR. -Publish it with `./gradlew publishToMavenLocal`. +The producer library and, when prebuilt, the JNI shim `lib{jni}.so` sit in +`src/main/jniLibs//`; with the shim present, Gradle packages both into +the AAR without CMake or the NDK. Build and publish the AAR with Gradle +(`gradle assembleRelease`, `gradle publishToMavenLocal`). ## Desktop JVM -Build the shim once per platform against the bundled producer library and -place it beside that library: - -```bash -cmake -S src/main/cpp -B build -D{upper}_LIBRARY_DIR=src/main/resources/natives/ -cmake --build build -cp build/lib{jni}.* src/main/resources/natives// -``` - -At run time the bindings load both libraries from `natives/-/` on -the classpath, falling back to `java.library.path`. +The producer library and the prebuilt shim for each desktop platform sit in +`src/main/resources/natives//`. At run time the bindings extract +both from the classpath, falling back to `java.library.path`. ## Bundled Android ABIs @@ -292,14 +273,12 @@ the classpath, falling back to `java.library.path`. {desktop} "#, name = id.name, - lib = id.library, - upper = id.macro_prefix(), android = list(android), desktop = list(desktop), ); format!( "{}{body}\n{}", - render_prelude(CommentStyle::Xml, input_basename), + render_prelude(CommentStyle::Xml), render_trailer(CommentStyle::Xml, "README.md") ) } diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime.rs b/crates/weaveffi-cli/src/targets/kotlin/runtime.rs similarity index 76% rename from crates/weaveffi-gen/src/targets/kotlin/runtime.rs rename to crates/weaveffi-cli/src/targets/kotlin/runtime.rs index cc1c5edd..d310e9df 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/runtime.rs +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime.rs @@ -1,11 +1,13 @@ //! The fixed runtime sources under `runtime/`, spliced into the output with //! `{{PLACEHOLDER}}` substitution: the Kotlin runtime (`FfiException`, -//! `NativeIterator`, the loader, `NativeHandle`, the phantom-reference -//! cleaner), the value-buffer codec, the coroutine bridge, and the JNI shim's -//! helpers. +//! `NativeBugException`, `NativeIterator`, the loader, `NativeHandle`, the +//! phantom-reference cleaner), the value-buffer codec, the coroutine bridge, +//! and the JNI shim's helpers (the contract checker, thread attach and +//! detach, async completions, and callback dispatch). use crate::cabi::ABI_VERSION; -use weaveffi_model::model::BindingModel; +use weaveffi_model::model::Model; +use weaveffi_model::ty::Family; use crate::targets::kotlin::names::Names; @@ -16,6 +18,7 @@ const JNI_CORE_C: &str = include_str!("runtime/jni_core.c"); const JNI_THREADS_C: &str = include_str!("runtime/jni_threads.c"); const JNI_ASYNC_C: &str = include_str!("runtime/jni_async.c"); const JNI_CALLBACKS_C: &str = include_str!("runtime/jni_callbacks.c"); +const JNI_CALLBACK_RUNS_C: &str = include_str!("runtime/jni_callback_runs.c"); /// Replace every `{{KEY}}` in `template` with its value. fn fill(template: &str, vars: &[(&str, &str)]) -> String { @@ -60,7 +63,7 @@ pub(crate) fn async_kt(n: &Names) -> String { /// The fixed head of the JNI shim: the core helpers and `JNI_OnLoad`, plus /// the thread, async, and callback support the model needs. -pub(crate) fn jni_runtime(n: &Names, model: &BindingModel, header: &str, name: &str) -> String { +pub(crate) fn jni_runtime(n: &Names, model: &Model, header: &str, name: &str) -> String { let abi = ABI_VERSION.to_string(); let jni_class = n.jni_class(); let macro_prefix = n.prefix.to_ascii_uppercase(); @@ -86,5 +89,15 @@ pub(crate) fn jni_runtime(n: &Names, model: &BindingModel, header: &str, name: & if has_callbacks { out.push_str(&fill(JNI_CALLBACKS_C, &vars)); } + let returns_runs = model.callback_interfaces().any(|(_, cb)| { + cb.methods.iter().any(|m| { + m.ret.as_ref().is_some_and(|t| { + matches!(t.family(), Family::String | Family::Bytes | Family::Buffer) + }) + }) + }); + if returns_runs { + out.push_str(&fill(JNI_CALLBACK_RUNS_C, &vars)); + } out } diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime/Async.kt b/crates/weaveffi-cli/src/targets/kotlin/runtime/Async.kt similarity index 100% rename from crates/weaveffi-gen/src/targets/kotlin/runtime/Async.kt rename to crates/weaveffi-cli/src/targets/kotlin/runtime/Async.kt diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime/Buffers.kt b/crates/weaveffi-cli/src/targets/kotlin/runtime/Buffers.kt similarity index 92% rename from crates/weaveffi-gen/src/targets/kotlin/runtime/Buffers.kt rename to crates/weaveffi-cli/src/targets/kotlin/runtime/Buffers.kt index bb8f310c..ccf40a9d 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/runtime/Buffers.kt +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime/Buffers.kt @@ -28,7 +28,7 @@ internal class BufferWriter { buf[len++] = v } - fun writeU8(v: Byte) = writeI8(v) + fun writeU8(v: UByte) = writeI8(v.toByte()) fun writeI16(v: Short) { reserve(2) @@ -37,7 +37,7 @@ internal class BufferWriter { buf[len++] = (b shr 8).toByte() } - fun writeU16(v: Short) = writeI16(v) + fun writeU16(v: UShort) = writeI16(v.toShort()) fun writeI32(v: Int) { reserve(4) @@ -47,14 +47,14 @@ internal class BufferWriter { buf[len++] = (v shr 24).toByte() } - fun writeU32(v: Long) = writeI32(v.toInt()) + fun writeU32(v: UInt) = writeI32(v.toInt()) fun writeI64(v: Long) { writeI32(v.toInt()) writeI32((v ushr 32).toInt()) } - fun writeU64(v: Long) = writeI64(v) + fun writeU64(v: ULong) = writeI64(v.toLong()) fun writeF32(v: Float) = writeI32(v.toRawBits()) @@ -111,13 +111,13 @@ internal class BufferWriter { /** * Validating little-endian reader for the value-buffer wire format. A * truncated buffer, an invalid flag byte, a zero object token, an oversized - * length, or trailing bytes raise [FfiException] code -3. + * length, or trailing bytes raise [NativeBugException] code -3. */ internal class BufferReader(private val buf: ByteArray) { private var pos = 0 private fun malformed(detail: String): Nothing = - throw FfiException(-3, "malformed value buffer: $detail") + throw NativeBugException(-3, "malformed value buffer: $detail") private fun take(n: Int): Int { if (n > buf.size - pos) malformed("truncated buffer") @@ -134,14 +134,14 @@ internal class BufferReader(private val buf: ByteArray) { fun readI8(): Byte = buf[take(1)] - fun readU8(): Byte = readI8() + fun readU8(): UByte = readI8().toUByte() fun readI16(): Short { val at = take(2) return ((buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)).toShort() } - fun readU16(): Short = readI16() + fun readU16(): UShort = readI16().toUShort() fun readI32(): Int { val at = take(4) @@ -151,7 +151,7 @@ internal class BufferReader(private val buf: ByteArray) { ((buf[at + 3].toInt() and 0xFF) shl 24) } - fun readU32(): Long = readI32().toLong() and 0xFFFFFFFFL + fun readU32(): UInt = readI32().toUInt() fun readI64(): Long { val lo = readI32().toLong() and 0xFFFFFFFFL @@ -159,7 +159,7 @@ internal class BufferReader(private val buf: ByteArray) { return lo or (hi shl 32) } - fun readU64(): Long = readI64() + fun readU64(): ULong = readI64().toULong() fun readF32(): Float = Float.fromBits(readI32()) diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime/Runtime.kt b/crates/weaveffi-cli/src/targets/kotlin/runtime/Runtime.kt similarity index 87% rename from crates/weaveffi-gen/src/targets/kotlin/runtime/Runtime.kt rename to crates/weaveffi-cli/src/targets/kotlin/runtime/Runtime.kt index c719a7bb..22758b45 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/runtime/Runtime.kt +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime/Runtime.kt @@ -7,11 +7,28 @@ import java.util.concurrent.ConcurrentHashMap import java.util.concurrent.atomic.AtomicLong /** - * A failure reported by the native library: a producer panic (code -2), a - * marshalling failure (-3), a callback implementation that threw (-4), or any - * code these bindings don't know. Typed error domains subclass it. + * A failure a throwing call reports. Each error domain subclasses it with + * one class per declared (positive) code; a runtime failure keeps this class + * with its negative [code]: -1 a generic producer error, -2 a producer + * panic, -3 a marshalling failure, or -4 a callback implementation that + * failed. */ -open class FfiException(val code: Int, message: String) : Exception(message) +open class FfiException(val code: Int, message: String) : Exception(message) { + /** + * The fields of a typed error as the value buffer a callback reports + * with its code, or `null` when the code declares none. + */ + internal open fun encodePayload(): ByteArray? = null +} + +/** + * A call that can't fail failed anyway: a bug in the native library rather + * than an error to handle. [code] is the runtime code (-2 a producer panic, + * -3 a marshalling failure, -4 a callback failure the call couldn't report, + * and so on); the message names it and carries the producer's message. + */ +class NativeBugException(val code: Int, message: String) : + IllegalStateException("native call failed with code $code: $message") /** * A lazy sequence streamed from the native library, one element per step. diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_async.c b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_async.c similarity index 94% rename from crates/weaveffi-gen/src/targets/kotlin/runtime/jni_async.c rename to crates/weaveffi-cli/src/targets/kotlin/runtime/jni_async.c index f1fd9ec0..2fc4e1cc 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_async.c +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_async.c @@ -35,8 +35,8 @@ static jint Jni_load_async(JNIEnv* env) { /* Starts a completion on the producer thread: returns the JNIEnv with a local frame pushed, or NULL when the call failed (the failure, or the loss of the JVM, has then been handled and `context` released). */ -static JNIEnv* Jni_complete_begin(void* context, {{PREFIX}}_error* err) { - JNIEnv* env = Jni_env(); +static JNIEnv* Jni_complete_begin(void* context, {{PREFIX}}_error* err, int* detach) { + JNIEnv* env = Jni_env(detach); if (env == NULL) { {{PREFIX}}_error_free(err); return NULL; @@ -44,6 +44,7 @@ static JNIEnv* Jni_complete_begin(void* context, {{PREFIX}}_error* err) { if ((*env)->PushLocalFrame(env, 8) != 0) { (*env)->ExceptionClear(env); {{PREFIX}}_error_free(err); + Jni_env_done(*detach); return NULL; } if (err == NULL) { @@ -64,18 +65,20 @@ static JNIEnv* Jni_complete_begin(void* context, {{PREFIX}}_error* err) { } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(*detach); return NULL; } /* Finishes a successful completion: reports anything the Kotlin side threw (it has no caller to propagate to) and releases `context`. */ -static void Jni_complete_end(JNIEnv* env, void* context) { +static void Jni_complete_end(JNIEnv* env, void* context, int detach) { if ((*env)->ExceptionCheck(env)) { (*env)->ExceptionDescribe(env); (*env)->ExceptionClear(env); } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(detach); } JNIEXPORT jlong JNICALL Java_{{JNI_CLASS}}_cancel_1token_1create(JNIEnv* env, jclass cls) { diff --git a/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callback_runs.c b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callback_runs.c new file mode 100644 index 00000000..dd6e5662 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callback_runs.c @@ -0,0 +1,20 @@ + +/* Hands a string, bytes, or buffer return to the producer: a run allocated + with {{PREFIX}}_alloc in the out slots, which the producer adopts. A NULL + `value` (the shim reported a failure) leaves the slots empty. */ +static void Jni_callback_return_bytes(JNIEnv* env, jbyteArray value, uint8_t** out_ptr, size_t* out_len, {{PREFIX}}_error* out_err) { + if (value == NULL || (*env)->ExceptionCheck(env)) { + return; + } + jsize len = (*env)->GetArrayLength(env, value); + uint8_t* run = {{PREFIX}}_alloc((size_t)len); + if (len > 0 && run == NULL) { + {{PREFIX}}_error_set(out_err, -4, "out of memory"); + return; + } + if (len > 0) { + (*env)->GetByteArrayRegion(env, value, 0, len, (jbyte*)run); + } + *out_ptr = run; + *out_len = (size_t)len; +} diff --git a/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callbacks.c b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callbacks.c new file mode 100644 index 00000000..fd6bcd5b --- /dev/null +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_callbacks.c @@ -0,0 +1,74 @@ + +/* Callback interfaces: the producer holds a global reference to the Kotlin + implementation as `ctx`; trampolines call the matching static dispatch + shim on JniBridge from whatever thread the producer uses. The shim never + lets an exception escape: it reports a failure itself through `out_err` + (with the two exports below) and returns a zero value. */ + +/* Enters a trampoline: the JNIEnv with a local frame pushed, or NULL after + reporting the failure through `out_err`. */ +static JNIEnv* Jni_callback_begin({{PREFIX}}_error* out_err, int* detach) { + JNIEnv* env = Jni_env(detach); + if (env == NULL) { + {{PREFIX}}_error_set(out_err, -4, "could not attach the calling thread to the JVM"); + return NULL; + } + if ((*env)->PushLocalFrame(env, 16) != 0) { + (*env)->ExceptionClear(env); + {{PREFIX}}_error_set(out_err, -4, "JNI local frame exhausted"); + Jni_env_done(*detach); + return NULL; + } + return env; +} + +/* Leaves a trampoline. An exception still pending means the dispatch itself + failed (the shim catches everything the implementation throws), which + reaches the producer as code -4; nothing ever unwinds through its frames. */ +static void Jni_callback_end(JNIEnv* env, {{PREFIX}}_error* out_err, int detach) { + if ((*env)->ExceptionCheck(env)) { + (*env)->ExceptionClear(env); + {{PREFIX}}_error_set(out_err, -4, "the JVM failed to dispatch the callback"); + } + (*env)->PopLocalFrame(env, NULL); + Jni_env_done(detach); +} + +/* The vtable `free` entry: drops the producer's reference to the Kotlin + implementation, from whatever thread the producer releases it on. */ +static void Jni_release_callback(void* ctx) { + int detach = 0; + JNIEnv* env = Jni_env(&detach); + if (env != NULL) { + if (ctx != NULL) { + (*env)->DeleteGlobalRef(env, (jobject)ctx); + } + Jni_env_done(detach); + } +} + +/* `JniBridge.error_set`: reports a failed callback through the trampoline's + `out_err` (the message is cut at its first NUL). */ +JNIEXPORT void JNICALL Java_{{JNI_CLASS}}_error_1set(JNIEnv* env, jclass cls, jlong err, jint code, jbyteArray message) { + (void)cls; + Jni_bytes m = Jni_borrow_bytes(env, message); + char* text = (char*)malloc(m.len + 1u); + if (text != NULL) { + if (m.len > 0) { + memcpy(text, m.ptr, m.len); + } + text[m.len] = '\0'; + } + Jni_release_bytes(env, &m); + {{PREFIX}}_error_set(({{PREFIX}}_error*)(intptr_t)err, (int32_t)code, text != NULL ? text : "callback failed"); + free(text); +} + +/* `JniBridge.error_set_payload`: attaches a typed error's fields (a value + buffer) to the failure `error_set` reported. */ +JNIEXPORT void JNICALL Java_{{JNI_CLASS}}_error_1set_1payload(JNIEnv* env, jclass cls, jlong err, jbyteArray payload) { + (void)cls; + Jni_bytes p = Jni_borrow_bytes(env, payload); + {{PREFIX}}_error_set_payload(({{PREFIX}}_error*)(intptr_t)err, p.ptr, p.len); + Jni_release_bytes(env, &p); +} diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_core.c b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_core.c similarity index 69% rename from crates/weaveffi-gen/src/targets/kotlin/runtime/jni_core.c rename to crates/weaveffi-cli/src/targets/kotlin/runtime/jni_core.c index e7cc4a3e..0dc28f50 100644 --- a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_core.c +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_core.c @@ -2,6 +2,7 @@ #include #include #include +#include #include #include @@ -65,8 +66,9 @@ static inline jbyteArray Jni_take_bytes(JNIEnv* env, const uint8_t* ptr, size_t return out; } -/* Throws the Kotlin exception for `err` (mapped through domain `domain`, - 0 for the generic one) and clears `err`. */ +/* Throws the Kotlin exception for `err` (mapped through error domain + `domain`, or 0 for a call that can't fail, which raises the trap) and + clears `err`. */ static inline void Jni_throw(JNIEnv* env, {{PREFIX}}_error* err, jint domain) { const char* message = err->message != NULL ? err->message : ""; jbyteArray text = Jni_new_bytes(env, (const uint8_t*)message, strlen(message)); @@ -93,6 +95,50 @@ static jint Jni_load_error(JNIEnv* env, const char* message) { return JNI_ERR; } +/* One contract entry these bindings were generated with: a declaration's + id and signature hash, plus its dotted path for the load error. */ +typedef struct { + uint64_t id; + uint64_t hash; + const char* path; +} Jni_contract_entry; + +/* Checks that the library's contract table (from `contract`, sorted by id) + has every expected entry with an equal hash; entries only the library + has are fine. Fails loading with an error naming the first declaration + that's missing or changed. */ +static inline jint Jni_check_contract(JNIEnv* env, const {{PREFIX}}_contract_entry* (*contract)(size_t*), const Jni_contract_entry* expected, size_t count) { + size_t len = 0; + const {{PREFIX}}_contract_entry* table = contract(&len); + if (table == NULL) { + len = 0; + } + for (size_t i = 0; i < count; i++) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) { + lo = mid + 1; + } else { + hi = mid; + } + } + const char* problem = NULL; + if (lo == len || table[lo].id != expected[i].id) { + problem = "is missing from the library"; + } else if (table[lo].hash != expected[i].hash) { + problem = "changed since these bindings were generated"; + } + if (problem != NULL) { + char message[512]; + snprintf(message, sizeof message, "%s: %s %s", "{{NAME}}", expected[i].path, problem); + return Jni_load_error(env, message); + } + } + return JNI_OK; +} + /* Generated below: the contract checks and the method IDs the shim caches. */ static jint Jni_load(JNIEnv* env); @@ -111,7 +157,7 @@ JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { return JNI_ERR; } Jni_bridge = (jclass)(*env)->NewGlobalRef(env, bridge); - Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)L{{PACKAGE_PATH}}/FfiException;"); + Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ljava/lang/Throwable;"); if (Jni_error == NULL) { return JNI_ERR; } diff --git a/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_threads.c b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_threads.c new file mode 100644 index 00000000..0283466f --- /dev/null +++ b/crates/weaveffi-cli/src/targets/kotlin/runtime/jni_threads.c @@ -0,0 +1,86 @@ + +/* Producer threads (async completions, callback calls, vtable `free`) are + usually not JVM threads. The first time one calls in it attaches as a + daemon, and a thread-local destructor (a pthread key, or a fiber-local + slot on Windows) detaches it when the thread exits, so a busy producer + thread pays for one attach, not one per call. If the destructor can't be + registered, the thread detaches again as soon as the call ends. */ +#if defined(_WIN32) +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#include + +static DWORD Jni_env_slot = FLS_OUT_OF_INDEXES; +static INIT_ONCE Jni_env_once = INIT_ONCE_STATIC_INIT; + +static VOID WINAPI Jni_thread_exit(PVOID value) { + if (value != NULL && Jni_vm != NULL) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + +static BOOL CALLBACK Jni_make_env_slot(PINIT_ONCE once, PVOID param, PVOID* context) { + (void)once; + (void)param; + (void)context; + Jni_env_slot = FlsAlloc(Jni_thread_exit); + return TRUE; +} + +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { + InitOnceExecuteOnce(&Jni_env_once, Jni_make_env_slot, NULL, NULL); + return Jni_env_slot != FLS_OUT_OF_INDEXES && FlsSetValue(Jni_env_slot, (PVOID)Jni_vm); +} +#else +#include + +static pthread_key_t Jni_env_key; +static int Jni_env_key_ok = 0; +static pthread_once_t Jni_env_once = PTHREAD_ONCE_INIT; + +static void Jni_thread_exit(void* value) { + if (value != NULL && Jni_vm != NULL) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + +static void Jni_make_env_key(void) { + Jni_env_key_ok = pthread_key_create(&Jni_env_key, Jni_thread_exit) == 0; +} + +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { + pthread_once(&Jni_env_once, Jni_make_env_key); + return Jni_env_key_ok && pthread_setspecific(Jni_env_key, (void*)Jni_vm) == 0; +} +#endif + +/* The calling thread's JNIEnv, attaching the thread if needed; NULL when + the JVM is unavailable. `*detach` is set when the caller must hand it + back with Jni_env_done because the thread can't detach itself at exit. */ +static JNIEnv* Jni_env(int* detach) { + JNIEnv* env = NULL; + *detach = 0; + if (Jni_vm == NULL) { + return NULL; + } + if ((*Jni_vm)->GetEnv(Jni_vm, (void**)&env, JNI_VERSION_1_6) == JNI_OK) { + return env; + } + /* `void*` converts to the `JNIEnv**` Android declares and the `void**` + the JDK declares. */ + if ((*Jni_vm)->AttachCurrentThreadAsDaemon(Jni_vm, (void*)&env, NULL) != JNI_OK) { + return NULL; + } + *detach = !Jni_remember_attach(); + return env; +} + +/* Ends a use of Jni_env's JNIEnv, detaching the thread if Jni_env said so. */ +static void Jni_env_done(int detach) { + if (detach) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} diff --git a/crates/weaveffi-cli/src/targets/kotlin/tests.rs b/crates/weaveffi-cli/src/targets/kotlin/tests.rs new file mode 100644 index 00000000..cddea207 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/kotlin/tests.rs @@ -0,0 +1,599 @@ +//! Unit tests rendering a small IR through the generator: identity-driven +//! names and layout, configuration overrides, name escaping, the ABI 4 JNI +//! surface (ptr+len strings, the load-time contract check, cancellation), +//! unsigned types, the trap, composite codecs, object lifetime, callbacks +//! (every return family, `throws`, optional callbacks), thread detaching, +//! iterators, and packaging. + +use camino::{Utf8Path, Utf8PathBuf}; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; + +use crate::backend::LanguageBackend; +use crate::package::{FileContent, PackageContext}; +use crate::platform::{BinarySet, NativeBinary, Platform}; +use crate::targets::kotlin::{KotlinConfig, KotlinFlavor, KotlinGenerator}; + +const FIXTURE: &str = r#" +version: "0.11.0" +modules: + - name: bus + enums: + - name: Level + variants: + - { name: Low, value: 0 } + - { name: High, value: 1 } + structs: + - name: Event + fields: + - { name: id, type: i64 } + - { name: source, type: Store } + callback_interfaces: + - name: Subscriber + methods: + - name: on_event + params: + - { name: event, type: Event } + - { name: note, type: string } + return: Level + - name: label + return: string + - name: pick + params: + - { name: home, type: Store } + return: "Store?" + - name: fetch + params: + - { name: key, type: string } + return: bytes + throws: true + - name: weight + params: + - { name: count, type: u32 } + return: u64 + errors: + name: BusError + codes: + - { name: Closed, code: 1, message: "Bus is closed" } + - name: Missing + code: 2 + message: "missing" + fields: + - { name: key, type: string } + - { name: tries, type: u16 } + interfaces: + - name: Store + constructors: + - name: new + params: + - { name: name, type: string } + methods: + - name: get + params: + - { name: key, type: string } + return: "string?" + throws: true + - name: larger + params: + - { name: other, type: "Store?" } + return: "Store?" + - name: keys + params: [] + return: "iter" + - name: counts + params: [] + return: "iter" + - name: subscribe + params: + - { name: sub, type: Subscriber } + - name: watch + params: + - { name: sub, type: "Subscriber?" } + - name: sizes + params: [] + return: "iter" + - name: drain + params: [] + return: i64 + async: true + cancellable: true + throws: true + functions: + - name: get + params: + - { name: id, type: i64 } + return: Event + - name: scale + params: + - { name: by, type: u8 } + - { name: totals, type: "{u64:u16}" } + return: "[u32]" + modules: + - name: stats + structs: + - name: Stats + fields: + - { name: total, type: i64 } + functions: + - name: summarize + params: + - { name: store, type: Store } + return: Stats + - name: units + enums: + - name: Unit + variants: + - { name: Px, value: 0 } + functions: + - name: echo + params: + - { name: unit, type: Unit } + return: Unit +"#; + +fn api() -> Model { + let api = weaveffi_model::parse::parse_api_str(FIXTURE, "yaml").expect("fixture parses"); + weaveffi_model::validate::validate(&api, &Identity::named("event-bus"), None) + .expect("fixture validates") +} + +/// Render with `config` and return `(path, contents)` pairs, paths with `/`. +fn render(config: &KotlinConfig) -> Vec<(String, String)> { + let model = api(); + KotlinGenerator + .files(&model, Utf8Path::new("out"), config) + .into_iter() + .map(|f| (f.path.as_str().replace('\\', "/"), f.contents)) + .collect() +} + +fn file<'a>(files: &'a [(String, String)], suffix: &str) -> &'a str { + files + .iter() + .find(|(p, _)| p.ends_with(suffix)) + .map(|(_, c)| c.as_str()) + .unwrap_or_else(|| { + let paths: Vec<&String> = files.iter().map(|f| &f.0).collect(); + panic!("{suffix} missing from {paths:?}") + }) +} + +#[test] +fn layout_follows_the_identity() { + let files = render(&KotlinConfig::default()); + let paths: Vec<&str> = files.iter().map(|(p, _)| p.as_str()).collect(); + assert_eq!( + paths, + [ + "out/kotlin/settings.gradle.kts", + "out/kotlin/build.gradle.kts", + "out/kotlin/consumer-rules.pro", + "out/kotlin/src/main/cpp/CMakeLists.txt", + "out/kotlin/src/main/cpp/event_bus.h", + "out/kotlin/src/main/cpp/event_bus_jni.c", + "out/kotlin/src/main/kotlin/event_bus/Runtime.kt", + "out/kotlin/src/main/kotlin/event_bus/Buffers.kt", + "out/kotlin/src/main/kotlin/event_bus/Async.kt", + "out/kotlin/src/main/kotlin/event_bus/Codecs.kt", + "out/kotlin/src/main/kotlin/event_bus/JniBridge.kt", + "out/kotlin/src/main/kotlin/event_bus/Bus.kt", + "out/kotlin/src/main/kotlin/event_bus/Units.kt", + ] + ); + let runtime = file(&files, "Runtime.kt"); + assert!(runtime.contains("package event_bus\n")); + assert!(runtime.contains("private const val LIBRARY = \"event_bus\"")); + assert!(runtime.contains("private const val JNI_LIBRARY = \"event_bus_jni\"")); + assert!(runtime.contains("System.getenv(\"EVENT_BUS_LIBRARY\")")); + let settings = file(&files, "settings.gradle.kts"); + assert!(settings.contains("rootProject.name = \"event-bus\"")); + assert!(settings.contains("google()")); + let gradle = file(&files, "build.gradle.kts"); + assert!(gradle.contains("id(\"com.android.library\")")); + assert!(gradle.contains("namespace = \"event_bus\"")); + assert!(gradle.contains("minSdk = 21")); + assert!(gradle.contains("`maven-publish`")); + assert!(gradle.contains("kotlinx-coroutines-core")); + let cmake = file(&files, "CMakeLists.txt"); + assert!(cmake.contains("add_library(event_bus_jni SHARED event_bus_jni.c)")); + assert!(cmake.contains("find_library(EVENT_BUS_PRODUCER NAMES event_bus")); + assert!(cmake.contains("../jniLibs/${ANDROID_ABI}")); + // The bundled header is the C target's, verbatim. + for (path, contents) in files.iter().filter(|(p, _)| !p.ends_with(".h")) { + let body: String = contents.lines().skip(3).collect::>().join("\n"); + assert!( + !body.contains("WeaveFFI") && !body.contains("weaveffi"), + "{path} carries WeaveFFI branding" + ); + } +} + +#[test] +fn configuration_overrides_package_and_flavor() { + let config = KotlinConfig { + name: Some("com.example.bus".into()), + flavor: KotlinFlavor::Jvm, + ..KotlinConfig::default() + }; + let files = render(&config); + assert!(files + .iter() + .all(|(p, _)| !p.ends_with("consumer-rules.pro"))); + assert!(files + .iter() + .any(|(p, _)| p == "out/kotlin/src/main/kotlin/com/example/bus/JniBridge.kt")); + let gradle = file(&files, "build.gradle.kts"); + assert!(gradle.contains("kotlin(\"jvm\")")); + assert!(gradle.contains("group = \"com.example.bus\"")); + assert!(!gradle.contains("com.android.library")); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains("Java_com_example_bus_JniBridge_bus_1Store_1get(")); + assert!(c.contains("FindClass(env, \"com/example/bus/JniBridge\")")); +} + +#[test] +fn modules_are_objects_and_shadowing_names_are_escaped() { + let files = render(&KotlinConfig::default()); + let bus = file(&files, "Bus.kt"); + // Two modules each declare `get`; the module objects keep them apart, + // and the `stats` object steps aside for the `Stats` record. + assert!(bus.contains("object Bus {")); + assert!(bus.contains(" fun get(id: Long): Event = ")); + assert!(bus.contains(" object StatsModule {")); + assert!(bus.contains("fun summarize(store: Store): Stats = store.handle.borrow { _h0 ->")); + // A user type named like a Kotlin builtin gains a trailing underscore. + let units = file(&files, "Units.kt"); + assert!(units.contains("enum class Unit_(val value: Int) {")); + assert!(units.contains( + "fun echo(unit: Unit_): Unit_ = Unit_.fromValue(JniBridge.units_echo(unit.value))" + )); + // JNI natives are named after their (unique) C symbols. + let bridge = file(&files, "JniBridge.kt"); + assert!(bridge.contains("@JvmStatic external fun bus_get(id: Long): ByteArray")); + assert!(bridge.contains("@JvmStatic external fun bus_stats_summarize(store: Long): ByteArray")); +} + +#[test] +fn strings_cross_as_bytes_with_lengths() { + let files = render(&KotlinConfig::default()); + let bus = file(&files, "Bus.kt"); + assert!(bus.contains( + "fun get(key: String): String? = handle.borrow { _self ->\n decodeBuffer(JniBridge.bus_Store_get(_self, encodeUtf8(key))) { _r -> unpackOptionalOfString(_r) }" + )); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains("Jni_bytes p_key_b = Jni_borrow_bytes(env, p_key);")); + assert!(c.contains( + "event_bus_bus_Store_get((const event_bus_bus_Store*)(intptr_t)self, p_key_b.ptr, p_key_b.len, &out_len, &err);" + )); + assert!(c.contains("return Jni_take_bytes(env, rv, out_len);")); + assert!(c.contains("event_bus_free_bytes((uint8_t*)ptr, len);")); + assert!(!c.contains("free_string")); + assert!(!c.contains("GetStringUTFChars")); +} + +#[test] +fn load_checks_the_abi_revision_and_every_root_contract() { + let model = api(); + let files = render(&KotlinConfig::default()); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains("if (event_bus_abi_version() != 4u) {")); + assert!(!c.contains("checksum")); + for root in model.roots() { + let entries = weaveffi_model::contract::entries(&model, root); + let var = format!("Jni_contract_{}", root.name); + assert!(c.contains(&format!("static const Jni_contract_entry {var}[] = {{"))); + for e in &entries { + assert!(c.contains(&format!( + "{{UINT64_C(0x{:016x}), UINT64_C(0x{:016x}), \"{}\"}},", + e.id, e.hash, e.path + ))); + } + assert!(c.contains(&format!( + "if (Jni_check_contract(env, event_bus_{}_contract, {var}, sizeof {var} / sizeof {var}[0]) != JNI_OK) {{", + root.name + ))); + } + assert!(c.contains("problem = \"is missing from the library\";")); + assert!(c.contains("problem = \"changed since these bindings were generated\";")); +} + +#[test] +fn unsigned_integers_are_kotlin_unsigned_types() { + let files = render(&KotlinConfig::default()); + let bus = file(&files, "Bus.kt"); + // Public signatures use the unsigned types; the JNI natives carry the + // signed type of the same width, converted bit for bit. + assert!(bus.contains( + "fun scale(by: UByte, totals: Map): List = decodeBuffer(JniBridge.bus_scale(by.toByte(), encodeBuffer { _w -> packMapOfU64ToU16(_w, totals) })) { _r -> unpackListOfU32(_r) }" + )); + assert!(bus.contains( + "class Missing(val key: String, val tries: UShort, message: String = \"missing\")" + )); + assert!(bus.contains("fun weight(count: UInt): ULong")); + assert!(bus.contains( + "if (JniBridge.bus_Store_SizesIterator_next(_it, _slot)) _slot[0].toUInt() else NativeIterator.DONE" + )); + let bridge = file(&files, "JniBridge.kt"); + assert!(bridge + .contains("@JvmStatic external fun bus_scale(by: Byte, totals: ByteArray): ByteArray")); + assert!(bridge.contains("_impl.weight(count.toUInt()).toLong()")); + let buffers = file(&files, "Buffers.kt"); + assert!(buffers.contains("fun writeU32(v: UInt) = writeI32(v.toInt())")); + assert!(buffers.contains("fun readU64(): ULong = readI64().toULong()")); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains("JNICALL Java_event_1bus_JniBridge_bus_1scale(JNIEnv* env, jclass cls, jbyte p_by, jbyteArray p_totals)")); + assert!(c.contains("event_bus_bus_scale((uint8_t)p_by, ")); + assert!(c.contains("static uint64_t Jni_bus_Subscriber_weight(void* ctx, uint32_t p_count, event_bus_error* out_err)")); +} + +#[test] +fn composites_have_one_codec_each() { + let files = render(&KotlinConfig::default()); + let codecs = file(&files, "Codecs.kt"); + assert!(codecs.contains( + "internal fun packMapOfU64ToU16(_w: BufferWriter, _v: Map) = _w.writeMap(_v, { _w.writeU64(it) }, { _w.writeU16(it) })" + )); + assert!(codecs.contains( + "internal fun unpackOptionalOfString(_r: BufferReader): String? = _r.readOptional { _r.readString() }" + )); + assert_eq!(codecs.matches("internal fun packListOfU32(").count(), 1); + // Call sites call the codec instead of inlining a loop. + for (path, contents) in files.iter().filter(|(p, _)| p.ends_with(".kt")) { + if !path.ends_with("Codecs.kt") && !path.ends_with("Buffers.kt") { + assert!( + !contents.contains("writeList(") && !contents.contains("readOptional"), + "{path}" + ); + } + } +} + +#[test] +fn a_failed_call_that_cannot_fail_traps() { + let files = render(&KotlinConfig::default()); + let runtime = file(&files, "Runtime.kt"); + assert!(runtime.contains( + "class NativeBugException(val code: Int, message: String) :\n IllegalStateException(\"native call failed with code $code: $message\")" + )); + let bridge = file(&files, "JniBridge.kt"); + assert!(bridge.contains("1 -> BusException.fromCode(code, text, payload)")); + assert!(bridge.contains("else -> NativeBugException(code, text)")); + let c = file(&files, "event_bus_jni.c"); + // `get` throws (domain 1); `larger` can't fail, so its failure traps. + assert!(c.contains("\"(II[B[B)Ljava/lang/Throwable;\"")); + let larger = &c[c.find("bus_1Store_1larger(").unwrap()..]; + assert!(larger.contains("Jni_throw(env, &err, 0);")); + let get = &c[c.find("bus_1Store_1get(").unwrap()..]; + assert!(get.contains("Jni_throw(env, &err, 1);")); +} + +#[test] +fn objects_are_borrowed_for_every_call() { + let files = render(&KotlinConfig::default()); + let bus = file(&files, "Bus.kt"); + assert!(bus.contains("class Store private constructor(address: Long) : AutoCloseable {")); + assert!(bus.contains( + "internal val handle: NativeHandle = NativeCleaner.register(this, NativeHandle(address, JniBridge::bus_Store_destroy))" + )); + assert!(bus.contains( + "fun larger(other: Store?): Store? = handle.borrow { _self ->\n other?.handle.borrowOrNull { _h0 ->\n Store.fromHandleOrNull(JniBridge.bus_Store_larger(_self, _h0))" + )); + // Objects inside buffers are cloned on the way in and released again if + // the encoding fails. + assert!(bus.contains("_w.writeObject(_v.source.cloneHandle(), JniBridge::bus_Store_destroy)")); + let runtime = file(&files, "Runtime.kt"); + assert!(runtime.contains("PhantomReference")); + assert!(!runtime.contains("java.lang.ref.Cleaner")); +} + +#[test] +fn cancellable_async_wires_the_token() { + let files = render(&KotlinConfig::default()); + let bus = file(&files, "Bus.kt"); + assert!(bus.contains( + "suspend fun drain(): Long = awaitNative(true, 1, { _raw -> _raw as Long }) { _token, _done ->" + )); + assert!(bus.contains("JniBridge.bus_Store_drain(_self, _token, _done)")); + let runtime = file(&files, "Async.kt"); + assert!(runtime.contains("cont.invokeOnCancellation { completion.cancel() }")); + assert!(runtime.contains("if (code == -5)")); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains( + "(event_bus_cancel_token*)(intptr_t)cancel_token, Jni_done_bus_Store_drain, context);" + )); + assert!(c.contains("CallVoidMethod(env, (jobject)context, Jni_on_long, (jlong)result);")); + assert!(c.contains("Jni_complete_end(env, context, detach);")); +} + +#[test] +fn callbacks_dispatch_through_cached_static_shims() { + let files = render(&KotlinConfig::default()); + let bridge = file(&files, "JniBridge.kt"); + assert!(bridge.contains( + "fun bus_Subscriber_on_event(_impl: Subscriber, _err: Long, event: ByteArray, note: ByteArray): Int = try {\n _impl.onEvent(decodeBuffer(event) { _r -> unpackEvent(_r) }, decodeUtf8(note)).value\n } catch (_e: Throwable) {\n fail(_err, _e, null)\n 0\n }" + )); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains( + "GetStaticMethodID(env, Jni_bridge, \"bus_Subscriber_on_event\", \"(Levent_bus/Subscriber;J[B[B)I\");" + )); + assert!(c.contains("AttachCurrentThreadAsDaemon")); + assert!(c.contains( + "static const event_bus_bus_Subscriber_vtable Jni_bus_Subscriber_vtable = {sizeof(event_bus_bus_Subscriber_vtable), 0, Jni_release_callback, Jni_bus_Subscriber_on_event, Jni_bus_Subscriber_label, Jni_bus_Subscriber_pick, Jni_bus_Subscriber_fetch, Jni_bus_Subscriber_weight};" + )); + // The shim reports failures itself through these two exports. + assert!(c.contains("Java_event_1bus_JniBridge_error_1set(JNIEnv* env, jclass cls, jlong err, jint code, jbyteArray message)")); + assert!( + c.contains("event_bus_error_set_payload((event_bus_error*)(intptr_t)err, p.ptr, p.len);") + ); + assert!(!c.contains("_dealloc")); +} + +#[test] +fn callback_methods_return_every_family() { + let files = render(&KotlinConfig::default()); + let bridge = file(&files, "JniBridge.kt"); + // A string return is encoded; the trampoline copies it into an alloc'd run. + assert!(bridge.contains( + "fun bus_Subscriber_label(_impl: Subscriber, _err: Long): ByteArray? = try {\n encodeUtf8(_impl.label())" + )); + // An object parameter is adopted before anything can fail; an optional + // object return is a fresh reference the producer adopts. + assert!(bridge.contains( + "fun bus_Subscriber_pick(_impl: Subscriber, _err: Long, home: Long): Long {\n val _home = Store.fromHandle(home)\n return try {\n _impl.pick(_home)?.cloneHandle() ?: 0L" + )); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains( + "static void Jni_bus_Subscriber_label(void* ctx, uint8_t** out_ptr, size_t* out_len, event_bus_error* out_err) {" + )); + assert!(c.contains("Jni_callback_return_bytes(env, rv, out_ptr, out_len, out_err);")); + assert!(c.contains("uint8_t* run = event_bus_alloc((size_t)len);")); + assert!(c.contains("return (event_bus_bus_Store*)(intptr_t)rv;")); + // A call that can't reach the JVM still releases the adopted object. + assert!(c.contains( + " event_bus_bus_Store_destroy(p_home);\n return (event_bus_bus_Store*)0;" + )); +} + +#[test] +fn throwing_callbacks_report_their_domain_error() { + let files = render(&KotlinConfig::default()); + let bridge = file(&files, "JniBridge.kt"); + assert!(bridge.contains("fail(_err, _e, _e as? BusException)")); + assert!(bridge.contains("error_set(err, typed.code, encodeUtf8(typed.message ?: \"\"))")); + assert!(bridge.contains("typed.encodePayload()?.let { error_set_payload(err, it) }")); + assert!(bridge.contains("error_set(err, -4, encodeUtf8(error.message ?: error.toString()))")); + let bus = file(&files, "Bus.kt"); + assert!(bus.contains( + "override fun encodePayload(): ByteArray = encodeBuffer { _w ->\n _w.writeString(key)\n _w.writeU16(tries)\n }" + )); + assert!(bus.contains("Throw a [BusException] to report a typed error to the library.")); +} + +#[test] +fn optional_callbacks_pass_a_null_vtable() { + let files = render(&KotlinConfig::default()); + let bus = file(&files, "Bus.kt"); + assert!(bus.contains("fun watch(sub: Subscriber?) {")); + assert!(bus.contains("JniBridge.bus_Store_watch(_self, sub)")); + let bridge = file(&files, "JniBridge.kt"); + assert!( + bridge.contains("@JvmStatic external fun bus_Store_watch(_self: Long, sub: Subscriber?)") + ); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains( + "p_sub != NULL ? (void*)(*env)->NewGlobalRef(env, p_sub) : NULL, p_sub != NULL ? &Jni_bus_Subscriber_vtable : NULL, &err);" + )); +} + +#[test] +fn producer_threads_detach_on_every_platform() { + let files = render(&KotlinConfig::default()); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains("pthread_key_create(&Jni_env_key, Jni_thread_exit) == 0;")); + assert!(c.contains("Jni_env_slot = FlsAlloc(Jni_thread_exit);")); + assert!(c.contains("*detach = !Jni_remember_attach();")); + assert!(c.contains("Jni_env_done(detach);")); +} + +#[test] +fn iterators_stream_through_native_iterator() { + let files = render(&KotlinConfig::default()); + let bus = file(&files, "Bus.kt"); + assert!(bus.contains( + "fun keys(): NativeIterator = handle.borrow { _self ->\n NativeIterator(JniBridge.bus_Store_keys(_self), JniBridge::bus_Store_KeysIterator_destroy) { _it ->" + )); + assert!(bus.contains( + "if (JniBridge.bus_Store_CountsIterator_next(_it, _slot)) _slot[0] else NativeIterator.DONE" + )); + let c = file(&files, "event_bus_jni.c"); + assert!(c.contains("(*env)->SetIntArrayRegion(env, out, 0, 1, &value);")); +} + +#[test] +fn package_bundles_prebuilt_binaries() { + let model = api(); + let mut binaries = BinarySet::new("event_bus"); + let mut android = NativeBinary::new( + Platform::AndroidArm64, + "/prebuilt/android-arm64/libevent_bus.so", + ); + android.jni_shim = Some(Utf8PathBuf::from( + "/prebuilt/android-arm64/libevent_bus_jni.so", + )); + binaries.insert(android); + let mut mac = NativeBinary::new( + Platform::MacosArm64, + "/prebuilt/darwin-arm64/libevent_bus.dylib", + ); + mac.jni_shim = Some(Utf8PathBuf::from( + "/prebuilt/darwin-arm64/libevent_bus_jni.dylib", + )); + binaries.insert(mac); + binaries.insert(NativeBinary::new( + Platform::Wasm32, + "/prebuilt/wasm32/event_bus.wasm", + )); + let ctx = PackageContext::new(&binaries); + let artifacts = KotlinGenerator + .package(&model, &ctx, &KotlinConfig::default()) + .expect("kotlin supports packaging"); + assert_eq!(artifacts.len(), 1); + assert_eq!(artifacts[0].path, "kotlin/event-bus"); + let paths: Vec<&str> = artifacts[0].files.iter().map(|f| f.path.as_str()).collect(); + for expected in [ + "README.md", + "build.gradle.kts", + "src/main/jniLibs/arm64-v8a/libevent_bus.so", + "src/main/jniLibs/arm64-v8a/libevent_bus_jni.so", + "src/main/resources/natives/darwin-arm64/libevent_bus.dylib", + "src/main/resources/natives/darwin-arm64/libevent_bus_jni.dylib", + ] { + assert!(paths.contains(&expected), "{expected} missing: {paths:?}"); + } + assert!(!paths.iter().any(|p| p.contains("wasm"))); + let text = |name: &str| match &artifacts[0].file(name).expect(name).content { + FileContent::Text(t) => t.clone(), + _ => panic!("{name} is text"), + }; + let readme = text("README.md"); + assert!(readme.contains("- `src/main/jniLibs/arm64-v8a/libevent_bus.so`")); + assert!(readme.contains("without CMake or the NDK")); + let gradle = text("build.gradle.kts"); + assert!( + gradle.contains("include(\"**/libevent_bus_jni.so\")"), + "{gradle}" + ); + assert!(gradle.contains(" if (!prebuiltJni) {"), "{gradle}"); + + // Without a shim for every Android ABI, none ships: CMake builds them. + let mut partial = BinarySet::new("event_bus"); + partial.insert(NativeBinary::new( + Platform::AndroidX64, + "/prebuilt/android-x64/libevent_bus.so", + )); + partial.insert(binaries.get(Platform::AndroidArm64).unwrap().clone()); + let artifacts = KotlinGenerator + .package( + &model, + &PackageContext::new(&partial), + &KotlinConfig::default(), + ) + .unwrap(); + assert!(!artifacts[0] + .files + .iter() + .any(|f| f.path.as_str().ends_with("_jni.so"))); +} + +#[test] +fn android_sdk_levels_are_configurable() { + let config = KotlinConfig { + min_sdk: 24, + compile_sdk: 34, + ..KotlinConfig::default() + }; + let files = render(&config); + let gradle = file(&files, "build.gradle.kts"); + assert!(gradle.contains(" compileSdk = 34\n"), "{gradle}"); + assert!(gradle.contains(" minSdk = 24\n"), "{gradle}"); +} diff --git a/crates/weaveffi-gen/src/targets/mod.rs b/crates/weaveffi-cli/src/targets/mod.rs similarity index 87% rename from crates/weaveffi-gen/src/targets/mod.rs rename to crates/weaveffi-cli/src/targets/mod.rs index f6f16a1c..9a93101c 100644 --- a/crates/weaveffi-gen/src/targets/mod.rs +++ b/crates/weaveffi-cli/src/targets/mod.rs @@ -18,12 +18,8 @@ pub mod wasm; use crate::codegen::{ConfiguredBackend, Target}; -/// Every target's name, in canonical order (the `--target` tokens). -pub const NAMES: [&str; 11] = [ - "c", "cpp", "swift", "kotlin", "node", "wasm", "python", "dotnet", "dart", "go", "ruby", -]; - -/// Every target with its default configuration, in [`NAMES`] order. +/// Every target with its default configuration, in canonical order (the +/// order of the `--target` tokens). #[must_use] pub fn all_default() -> Vec> { vec![ diff --git a/crates/weaveffi-gen/src/targets/node/addon.rs b/crates/weaveffi-cli/src/targets/node/addon.rs similarity index 72% rename from crates/weaveffi-gen/src/targets/node/addon.rs rename to crates/weaveffi-cli/src/targets/node/addon.rs index 36226d47..e03730b5 100644 --- a/crates/weaveffi-gen/src/targets/node/addon.rs +++ b/crates/weaveffi-cli/src/targets/node/addon.rs @@ -1,20 +1,23 @@ //! The N-API addon (`{library}_node.c`): the Node.js transport. //! //! The fixed part (value conversions, error reporting, the async and -//! callback-interface machinery, and the runtime symbols) is -//! `runtime/addon.c`. This module appends one exported entry point per C -//! symbol (callables, async launchers, iterator `next`/`destroy`, interface -//! `clone`/`destroy`, and module checksums), each following the raw calling -//! convention of the shared JavaScript layer, plus a static vtable with -//! thread-hopping trampolines per callback interface. +//! callback-interface machinery, the JavaScript-thread bookkeeping, and the +//! runtime symbols) is `runtime/addon.c`. This module appends one exported +//! entry point per C symbol (callables, async launchers, iterator +//! `next`/`destroy`, interface `clone`/`destroy`, and module contract +//! tables), each following the raw calling convention of the shared +//! JavaScript layer, plus a static vtable with thread-hopping trampolines per +//! callback interface. +use weaveffi_model::abi::AbiParam; use weaveffi_model::model::{ - iterator_item_ctype, BindingModel, CallShape, CallbackInterfaceBinding, FnBinding, Prim, Ty, + contract_symbol, CallShape, CallbackInterfaceBinding, CallbackMethodBinding, FnBinding, Model, }; -use weaveffi_model::plan::{self, ArgPass, RetPass}; +use weaveffi_model::plan::{ArgPass, RetPass}; +use weaveffi_model::ty::{Prim, Ty}; use crate::codegen::CodeWriter; -use crate::targets::js::names::direct_prim; +use crate::targets::js::names::{callback_method_name, direct_prim}; use crate::utils::{render_prelude, render_trailer, CommentStyle}; /// The fixed addon runtime, with `{{PREFIX}}` (the C prefix), `{{MACRO}}` @@ -81,40 +84,39 @@ fn receive(ty: &Ty, pass: &RetPass, slots: &[String], owned: bool) -> String { } } -/// The name of the dispatcher and vtable of a callback interface by its C +/// The names of the dispatcher and vtable of a callback interface by its C /// tag. fn cb_names(c_tag: &str) -> (String, String) { (format!("dispatch_{c_tag}"), format!("vtable_{c_tag}")) } /// Render the complete addon source. -pub(crate) fn render_addon_c( - model: &BindingModel, - header: &str, - input_basename: &str, - file_name: &str, -) -> String { - let prefix = model.prefix.as_str(); - let mut out = render_prelude(CommentStyle::DoubleSlash, input_basename); - out.push_str( - &ADDON_C +pub(crate) fn render_addon_c(model: &Model, header: &str, file_name: &str) -> String { + let prefix = model.prefix(); + let mut w = CodeWriter::two_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + w.raw( + ADDON_C .replace("{{PREFIX}}", prefix) .replace("{{MACRO}}", &prefix.to_ascii_uppercase()) .replace("{{HEADER}}", header), ); - out.push('\n'); + w.blank(); - let mut w = CodeWriter::two_space(); let mut exports: Vec = Vec::new(); for root in model.roots() { - let symbol = weaveffi_model::model::checksum_symbol(prefix, &root.name); + let symbol = contract_symbol(prefix, &root.name); w.block( format!("static napi_value nx_{symbol}(napi_env env, napi_callback_info info) {{"), "}", |w| { w.line("(void)info;"); - w.line(format!("return js_new_u64(env, {symbol}());")); + w.line("size_t len = 0;"); + w.line(format!( + "const {prefix}_contract_entry* table = {symbol}(&len);" + )); + w.line("return js_new_contract(env, table, len);"); }, ); w.blank(); @@ -127,25 +129,25 @@ pub(crate) fn render_addon_c( for m in &model.modules { for i in &m.interfaces { - for (symbol, body) in [ - ( - &i.clone_symbol, - format!( - "return js_new_handle(env, {}((const {}*)h));", - i.clone_symbol, i.c_tag - ), + emit_handle_fn( + &mut w, + &i.clone_symbol, + &format!( + "js_sync_begin();\n{tag}* r = {}((const {tag}*)h);\njs_sync_end();\nreturn js_new_handle(env, r);", + i.clone_symbol, + tag = i.c_tag ), - ( - &i.destroy_symbol, - format!( - "{}(({}*)h);\nreturn js_undefined(env);", - i.destroy_symbol, i.c_tag - ), + ); + emit_handle_fn( + &mut w, + &i.destroy_symbol, + &format!( + "js_sync_begin();\n{}(({}*)h);\njs_sync_end();\nreturn js_undefined(env);", + i.destroy_symbol, i.c_tag ), - ] { - emit_handle_fn(&mut w, symbol, &body); - exports.push(symbol.clone()); - } + ); + exports.push(i.clone_symbol.clone()); + exports.push(i.destroy_symbol.clone()); } for f in m.callables() { emit_callable(&mut w, model, f, &mut exports); @@ -153,8 +155,7 @@ pub(crate) fn render_addon_c( } w.block("NAPI_MODULE_INIT() {", "}", |w| { - w.line("js_env* e = (js_env*)calloc(1, sizeof *e);"); - w.line("napi_set_instance_data(env, e, js_env_free, NULL);"); + w.line("js_env_init(env);"); w.block("napi_property_descriptor props[] = {", "};", |w| { w.line("JS_RUNTIME_EXPORTS,"); for symbol in &exports { @@ -166,10 +167,9 @@ pub(crate) fn render_addon_c( w.line("napi_define_properties(env, exports, sizeof props / sizeof props[0], props);"); w.line("return exports;"); }); - out.push_str(&w.finish()); - out.push('\n'); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, file_name)); - out + w.blank(); + w.raw(render_trailer(CommentStyle::DoubleSlash, file_name)); + w.finish() } /// An entry point taking one handle `h` and running `body`. @@ -206,8 +206,8 @@ struct Args { } /// Marshal the receiver, parameters, and cancel token of `f` from `argv`. -fn marshal(model: &BindingModel, f: &FnBinding, launch: &[weaveffi_model::abi::AbiParam]) -> Args { - let prefix = &model.prefix; +fn marshal(model: &Model, f: &FnBinding, launch: &[AbiParam]) -> Args { + let prefix = model.prefix(); let mut a = Args::default(); let mut idx = 0usize; if f.has_self { @@ -258,19 +258,22 @@ fn marshal(model: &BindingModel, f: &FnBinding, launch: &[weaveffi_model::abi::A )); a.call.push(format!("({}){v}", slot.ty.render_c(prefix))); } - ArgPass::Callback { .. } => { - let cb = find_callback( - model, + ArgPass::Callback { nullable, .. } => { + let cb = model.callback_interface( p.ty.callback_interface_name().expect("callback parameter"), ); let (dispatch, vtable) = cb_names(&cb.c_tag); a.decls.push(format!("js_cb* {v} = NULL;")); a.reads.push(format!( - "if (!js_arg_cb(env, argv[{idx}], \"{}\", {dispatch}, &{v})) goto done;", + "if (!js_arg_cb(env, argv[{idx}], \"{}\", {dispatch}, {nullable}, &{v})) goto done;", cb.c_tag )); a.call.push(format!("(void*){v}")); - a.call.push(format!("&{vtable}")); + if nullable { + a.call.push(format!("{v} != NULL ? &{vtable} : NULL")); + } else { + a.call.push(format!("&{vtable}")); + } a.transfer.push(format!("{v} = NULL;")); a.cleanup .push(format!("if ({v} != NULL) js_cb_release(env, {v});")); @@ -290,21 +293,10 @@ fn marshal(model: &BindingModel, f: &FnBinding, launch: &[weaveffi_model::abi::A a } -/// The callback interface named by the absolute dotted path `dotted`. -fn find_callback<'a>(model: &'a BindingModel, dotted: &str) -> &'a CallbackInterfaceBinding { - let (module, name) = dotted.rsplit_once('.').expect("absolute type path"); - model - .modules - .iter() - .find(|m| m.dot_path == module) - .and_then(|m| m.callback_interface(name)) - .expect("callback interface exists") -} - /// The async result kind and the statements storing the completion's /// result slots into `a`. -fn async_result(f: &FnBinding, prefix: &str) -> (&'static str, Vec) { - let pass = plan::ret_pass(f.ret.as_ref(), prefix); +fn async_result(f: &FnBinding) -> (&'static str, Vec) { + let pass = RetPass::of(f.ret.as_ref()); match pass { RetPass::Void => ("JS_R_VOID", vec![]), RetPass::Direct => { @@ -333,17 +325,12 @@ fn async_result(f: &FnBinding, prefix: &str) -> (&'static str, Vec) { } /// Emit one callable's entry point(s). -fn emit_callable( - w: &mut CodeWriter, - model: &BindingModel, - f: &FnBinding, - exports: &mut Vec, -) { - let prefix = model.prefix.as_str(); +fn emit_callable(w: &mut CodeWriter, model: &Model, f: &FnBinding, exports: &mut Vec) { + let prefix = model.prefix(); match &f.shape { CallShape::Async(ab) => { let symbol = &ab.launch.symbol; - let (kind, store) = async_result(f, prefix); + let (kind, store) = async_result(f); let params: Vec = ab .callback_params .iter() @@ -368,7 +355,9 @@ fn emit_callable( w.line(format!( "js_async* a = js_async_begin(env, {kind}, \"{symbol}\", &ret);" )); + w.line("js_sync_begin();"); w.line(format!("{symbol}({});", args.call.join(", "))); + w.line("js_sync_end();"); for t in &args.transfer { w.line(t); } @@ -377,7 +366,7 @@ fn emit_callable( } CallShape::Sync(abi) => { let args = marshal(model, f, &abi.params); - let pass = plan::ret_pass(f.ret.as_ref(), prefix); + let pass = RetPass::of(f.ret.as_ref()); let result = f.ret.as_ref().map(|ty| SyncResult { c_type: abi.ret.render_c(prefix), out_len: matches!(pass, RetPass::String | RetPass::Bytes | RetPass::Buffer), @@ -396,8 +385,8 @@ fn emit_callable( emit_sync(w, &it.launch.symbol, &args, Some(&handle)); exports.push(it.launch.symbol.clone()); - let item = iterator_item_ctype(&it.elem, "").render_c(prefix); - let protocol = it.protocol(f, prefix); + let item = it.item_ctype().render_c(prefix); + let protocol = it.protocol(f); let pair = matches!( protocol.elem, RetPass::String | RetPass::Bytes | RetPass::Buffer @@ -415,10 +404,12 @@ fn emit_callable( w.line("size_t item_len = 0;"); w.line("js_error err = {0};"); let len = if pair { "&item_len, " } else { "" }; + w.line("js_sync_begin();"); w.line(format!( "int32_t has = {symbol}(({}*)h, &item, {len}&err);", it.iter_tag )); + w.line("js_sync_end();"); if !pair { w.line("(void)item_len;"); } @@ -441,7 +432,7 @@ fn emit_callable( w, &it.destroy_symbol, &format!( - "{}(({}*)h);\nreturn js_undefined(env);", + "js_sync_begin();\n{}(({}*)h);\njs_sync_end();\nreturn js_undefined(env);", it.destroy_symbol, it.iter_tag ), ); @@ -471,10 +462,12 @@ fn emit_sync(w: &mut CodeWriter, symbol: &str, args: &Args, result: Option<&Sync } call.push("&err".into()); let call = format!("{symbol}({})", call.join(", ")); + w.line("js_sync_begin();"); match result { Some(r) => w.line(format!("{} r = {call};", r.c_type)), None => w.line(format!("{call};")), }; + w.line("js_sync_end();"); for t in &args.transfer { w.line(t); } @@ -511,30 +504,59 @@ fn emit_entry(w: &mut CodeWriter, symbol: &str, args: &Args, body: impl FnOnce(& w.blank(); } +/// The parts of one callback method's vtable entry: its C return type, the +/// parameter slots (declarations `p0`, `p1`, ...), and whether the return +/// travels through the `out_ptr`/`out_len` slots. +struct Entry<'a> { + ret_c: String, + slots: &'a [AbiParam], + run: bool, +} + +impl<'a> Entry<'a> { + fn of(method: &'a CallbackMethodBinding, pass: &RetPass, prefix: &str) -> Self { + let run = matches!(pass, RetPass::String | RetPass::Bytes | RetPass::Buffer); + let inputs: usize = method.params.iter().map(|p| p.abi.len()).sum(); + Entry { + ret_c: method.abi_ret.render_c(prefix), + slots: &method.abi_params[1..1 + inputs], + run, + } + } +} + /// Emit one callback interface: per method a frame, an invoker that runs on /// the JavaScript thread, and a trampoline (the vtable entry) that calls the /// invoker directly or hops to the JavaScript thread and waits; then the /// dispatcher of hopped calls and the static vtable. fn emit_callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, prefix: &str) { let tag = &cb.c_tag; - let protocol = cb.protocol(prefix); + let protocol = cb.protocol(); let (dispatch, vtable) = cb_names(tag); - for (method, passes) in cb.methods.iter().zip(&protocol.method_args) { + for (idx, ((method, passes), ret)) in cb + .methods + .iter() + .zip(&protocol.method_args) + .zip(&protocol.method_returns) + .enumerate() + { + let entry = Entry::of(method, ret, prefix); let frame = format!("frame_{tag}_{}", method.name); - let slots = &method.abi_params[1..method.abi_params.len() - 1]; - let ret_c = method.abi_ret.render_c(prefix); w.block("typedef struct {", format!("}} {frame};"), |w| { - w.line("js_cb_frame hdr;"); - for (i, s) in slots.iter().enumerate() { + w.line("js_error* out_err;"); + for (i, s) in entry.slots.iter().enumerate() { w.line(format!("{} p{i};", s.ty.render_c(prefix))); } - if method.ret.is_some() { - w.line(format!("{ret_c} result;")); + if entry.run { + w.line("uint8_t** out_ptr;"); + w.line("size_t* out_len;"); + } else if method.ret.is_some() { + w.line(format!("{} result;", entry.ret_c)); } }); w.blank(); - let where_ = format!("{}.{}", cb.name, method.name); + let what = format!("{}.{}", cb.name, callback_method_name(&method.name)); w.block( format!( "static void invoke_{tag}_{}(napi_env env, js_cb* cb, {frame}* f) {{", @@ -552,7 +574,10 @@ fn emit_callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, pr .map(|k| format!("f->p{k}")) .collect(); slot += p.abi.len(); - w.line(format!("argv[{i}] = {};", receive(&p.ty, pass, &names, false))); + w.line(format!( + "argv[{i}] = {};", + receive(&p.ty, pass, &names, false) + )); } w.line("napi_value result;"); w.line(format!( @@ -560,19 +585,11 @@ fn emit_callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, pr method.name )); w.line(format!( - " js_cb_report(env, f->hdr.out_err, \"{where_} failed\");" + " js_cb_report(env, f->out_err, \"{what} failed\");" )); if let Some(ty) = &method.ret { - let (reader, tmp) = arg_reader(direct_prim(ty)); w.line("} else {"); - w.line(format!(" {tmp} v = 0;")); - w.line(format!(" if ({reader}(env, result, &v)) {{")); - w.line(format!(" f->result = ({ret_c})v;")); - w.line(" } else {"); - w.line(format!( - " js_cb_report(env, f->hdr.out_err, \"{where_} returned a value of the wrong type\");" - )); - w.line(" }"); + w.scope(|w| emit_callback_return(w, ty, ret, &entry, &what)); } w.line("}"); w.line("napi_close_handle_scope(env, scope);"); @@ -581,18 +598,18 @@ fn emit_callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, pr w.blank(); let mut decls = vec!["void* ctx".to_string()]; - for (i, s) in slots.iter().enumerate() { + for (i, s) in entry.slots.iter().enumerate() { decls.push(format!("{} p{i}", s.ty.render_c(prefix))); } + if entry.run { + decls.push("uint8_t** out_ptr".into()); + decls.push("size_t* out_len".into()); + } decls.push(format!("{prefix}_error* out_err")); - let idx = cb - .methods - .iter() - .position(|m| m.name == method.name) - .unwrap_or(0); w.block( format!( - "static {ret_c} tramp_{tag}_{}({}) {{", + "static {} tramp_{tag}_{}({}) {{", + entry.ret_c, method.name, decls.join(", ") ), @@ -600,22 +617,21 @@ fn emit_callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, pr |w| { w.line(format!("{frame} f;")); w.line("memset(&f, 0, sizeof f);"); - w.line("f.hdr.out_err = out_err;"); - for i in 0..slots.len() { + w.line("f.out_err = out_err;"); + for i in 0..entry.slots.len() { w.line(format!("f.p{i} = p{i};")); } + if entry.run { + w.line("f.out_ptr = out_ptr;"); + w.line("f.out_len = out_len;"); + } w.line("js_cb* cb = (js_cb*)ctx;"); w.line("if (js_cb_on_js_thread(cb)) {"); w.line(format!(" invoke_{tag}_{}(cb->env, cb, &f);", method.name)); w.line("} else {"); - w.line(" js_cb_req req;"); - w.line(" memset(&req, 0, sizeof req);"); - w.line(" req.cb = cb;"); - w.line(format!(" req.method = {idx};")); - w.line(" req.frame = &f;"); - w.line(" js_cb_hop(&req);"); + w.line(format!(" js_cb_hop(cb, {idx}, &f, out_err, \"{what}\");")); w.line("}"); - if method.ret.is_some() { + if method.ret.is_some() && !entry.run { w.line("return f.result;"); } }, @@ -635,9 +651,10 @@ fn emit_callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, pr w.line(" free(req);"); w.line(" return;"); w.line("}"); + w.line("if (!js_cb_take(req)) return;"); w.line("if (env == NULL) {"); w.line(format!( - " {prefix}_error_set(((js_cb_frame*)req->frame)->out_err, -4, \"the JavaScript environment is shutting down\");" + " {prefix}_error_set(req->out_err, -4, \"the JavaScript environment is shutting down\");" )); w.line("} else {"); w.line(" switch (req->method) {"); @@ -654,16 +671,54 @@ fn emit_callback_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, pr }, ); w.blank(); - let entries: Vec = cb - .methods - .iter() - .map(|m| format!("tramp_{tag}_{}", m.name)) - .chain(std::iter::once("js_cb_free".to_string())) + let entries: Vec = ["js_cb_free".to_string()] + .into_iter() + .chain(cb.methods.iter().map(|m| format!("tramp_{tag}_{}", m.name))) .collect(); w.line(format!( - "static const {} {vtable} = {{{}}};", + "static const {0} {vtable} = {{sizeof({0}), 0, {1}}};", cb.vtable_tag, entries.join(", ") )); w.blank(); } + +/// Emit the conversion of a callback method's JavaScript `result` into its +/// vtable return: the frame's `result`, or a run in the out slots. A value +/// of the wrong type is reported as a failure. +fn emit_callback_return(w: &mut CodeWriter, ty: &Ty, pass: &RetPass, entry: &Entry, what: &str) { + let wrong = + format!("js_cb_report(env, f->out_err, \"{what} returned a value of the wrong type\");"); + match pass { + RetPass::Void => {} + RetPass::Direct => { + let (reader, tmp) = arg_reader(direct_prim(ty)); + w.line(format!("{tmp} v = 0;")); + w.line(format!("if ({reader}(env, result, &v)) {{")); + w.line(format!(" f->result = ({})v;", entry.ret_c)); + w.line("} else {"); + w.line(format!(" {wrong}")); + w.line("}"); + } + RetPass::String => { + w.line("if (!js_ret_str(env, result, f->out_ptr, f->out_len)) {"); + w.line(format!(" {wrong}")); + w.line("}"); + } + RetPass::Bytes | RetPass::Buffer => { + w.line("if (!js_ret_bytes(env, result, f->out_ptr, f->out_len)) {"); + w.line(format!(" {wrong}")); + w.line("}"); + } + RetPass::Object { nullable } => { + w.line("void* h = NULL;"); + w.line(format!( + "if (js_arg_handle(env, result, &h, {nullable})) {{" + )); + w.line(format!(" f->result = ({})h;", entry.ret_c)); + w.line("} else {"); + w.line(format!(" {wrong}")); + w.line("}"); + } + } +} diff --git a/crates/weaveffi-cli/src/targets/node/mod.rs b/crates/weaveffi-cli/src/targets/node/mod.rs new file mode 100644 index 00000000..5767a867 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/node/mod.rs @@ -0,0 +1,206 @@ +//! Node.js binding generator. +//! +//! Emits an npm package: the ES module (`index.js`, `index.d.ts`, +//! `runtime.js`) shared with the WebAssembly target through the shared +//! JavaScript layer, and its transport, a small N-API addon +//! (`{library}_node.c`) that calls the library's C ABI. `weaveffi build` +//! prebuilds the addon per platform and `weaveffi package` ships it in +//! per-platform packages; `npm install` compiles it with node-gyp (from +//! `binding.gyp`, against the bundled copy of the C header) only when no +//! prebuilt addon matches. Implements [`LanguageBackend`]; the shared driver +//! bridges it into the generator pipeline. + +mod addon; +mod package; + +use camino::Utf8Path; +use serde::{Deserialize, Serialize}; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::codegen::CodeWriter; +use crate::package::{Artifact, PackageContext, PackagedFile}; +use crate::platform::node_addon_name; +use crate::targets::c::render_c_header_from_model; +use crate::targets::js; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; + +use self::addon::render_addon_c; +pub(crate) use self::package::npm_tarball_name; +use self::package::{ + node_platform_tokens, render_binding_gyp, render_package_json, render_platform_package_json, + render_readme, NodeNames, +}; + +/// The fixed transport section of `index.js`: loading the addon. +const LOADER_JS: &str = include_str!("runtime/loader.js"); + +/// Per-target configuration for [`NodeGenerator`]. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct NodeConfig { + /// npm package name (default: the package identity's name). + pub name: Option, + /// The `engines.node` range of `package.json` (default `>=18`). + pub node_engine: String, +} + +impl Default for NodeConfig { + fn default() -> Self { + Self { + name: None, + node_engine: ">=18".into(), + } + } +} + +impl NodeConfig { + fn names<'a>(&'a self, identity: &'a Identity) -> NodeNames<'a> { + NodeNames { + identity, + package: self.name.clone().unwrap_or_else(|| identity.name.clone()), + addon: node_addon_name(&identity.library), + header: format!("{}.h", identity.library), + node_engine: &self.node_engine, + } + } +} + +/// Node.js backend: an ES module over an N-API addon. +pub struct NodeGenerator; + +/// The generated `index.js`: the imports, the addon loader, the load-time +/// checks, and the shared API. +fn render_index(model: &Model, names: &NodeNames<'_>) -> String { + let mut w = CodeWriter::two_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + js::render_imports(&mut w, names.identity); + w.blank(); + w.raw( + LOADER_JS + .replace("{{ADDON}}", &names.addon) + .replace("{{PACKAGE}}", &names.package), + ); + w.blank(); + js::render_contract(&mut w, model); + w.line(js::verify_call(model, names.identity)); + w.blank(); + js::render_api(&mut w, model); + w.raw(render_trailer(CommentStyle::DoubleSlash, "index.js")); + w.finish() +} + +/// Every file of the generated package, as `(name, contents)` pairs. +fn render_files( + model: &Model, + names: &NodeNames<'_>, + platforms: &[String], + default_dir: &str, +) -> Vec<(String, String)> { + let modules: Vec = model.roots().map(js::names::module_name).collect(); + vec![ + ("index.js".into(), render_index(model, names)), + ( + "index.d.ts".into(), + js::render_dts(model, names.identity, ""), + ), + ("runtime.js".into(), js::render_runtime(names.identity)), + ( + format!("{}.c", names.addon), + render_addon_c(model, &names.header, &format!("{}.c", names.addon)), + ), + ( + names.header.clone(), + render_c_header_from_model(model, &names.header), + ), + ("binding.gyp".into(), render_binding_gyp(names, default_dir)), + ("package.json".into(), render_package_json(names, platforms)), + ( + "README.md".into(), + render_readme(names, &modules, platforms), + ), + ] +} + +impl LanguageBackend for NodeGenerator { + type Config = NodeConfig; + + fn name(&self) -> &'static str { + "node" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let dir = out_dir.join("node"); + let names = config.names(&model.identity); + render_files(model, &names, &[], "path.resolve('.')") + .into_iter() + .map(|(name, contents)| OutputFile::new(dir.join(name), contents)) + .collect() + } + + /// The packaged layout follows the esbuild convention: one npm tarball + /// per platform, `{package}-{os}-{cpu}`, gated by npm `os`/`cpu` and + /// carrying the producer library and (when `weaveffi build` prebuilt it) + /// the addon, plus the main package, which lists them in + /// `optionalDependencies` and compiles the addon at install time only + /// when the installed platform package has none. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let names = config.names(&model.identity); + let id = names.identity; + let mut artifacts = Vec::new(); + let mut platform_names = Vec::new(); + for nb in &ctx.binaries.binaries { + let Some(tokens) = node_platform_tokens(nb.platform) else { + continue; + }; + let name = format!("{}-{}-{}", names.package, tokens.0, tokens.1); + let library = nb.platform.lib_filename(&id.library); + let mut contents = vec![library.clone()]; + let mut files = vec![PackagedFile::copy(library, nb.library.clone())]; + if let Some(addon) = &nb.node_addon { + let file = format!("{}.node", names.addon); + files.push(PackagedFile::copy(file.clone(), addon.clone())); + contents.push(file); + } + files.insert( + 0, + PackagedFile::text( + "package.json", + render_platform_package_json(&name, id, tokens, &contents), + ), + ); + artifacts.push(Artifact::tar_gz( + format!("node/{}", npm_tarball_name(&name, &id.version)), + "package", + files, + )); + platform_names.push(name); + } + if platform_names.is_empty() { + return Some(Vec::new()); + } + let default_dir = format!( + "path.dirname(require.resolve('{}-' + process.platform + '-' + process.arch + '/package.json'))", + names.package + ); + let files = render_files(model, &names, &platform_names, &default_dir) + .into_iter() + .map(|(name, contents)| PackagedFile::text(name, contents)) + .collect(); + artifacts.push(Artifact::tar_gz( + format!("node/{}", npm_tarball_name(&names.package, &id.version)), + "package", + files, + )); + Some(artifacts) + } +} + +#[cfg(test)] +mod tests; diff --git a/crates/weaveffi-gen/src/targets/node/package.rs b/crates/weaveffi-cli/src/targets/node/package.rs similarity index 68% rename from crates/weaveffi-gen/src/targets/node/package.rs rename to crates/weaveffi-cli/src/targets/node/package.rs index 5cf3aa7d..ca1a18c4 100644 --- a/crates/weaveffi-gen/src/targets/node/package.rs +++ b/crates/weaveffi-cli/src/targets/node/package.rs @@ -18,6 +18,8 @@ pub(crate) struct NodeNames<'a> { pub(crate) addon: String, /// The C header's file name (`{library}.h`). pub(crate) header: String, + /// The `engines.node` range. + pub(crate) node_engine: &'a str, } impl NodeNames<'_> { @@ -34,23 +36,42 @@ impl NodeNames<'_> { } } +/// The `install` script: succeed without compiling when a prebuilt addon +/// for this platform is installed (in the `{package}-{os}-{cpu}` package or +/// under `prebuilds/-/`), else compile it with node-gyp. +fn install_script(names: &NodeNames<'_>) -> String { + format!( + "node -e \"const p = process.platform + '-' + process.arch; \ + try {{ require.resolve('{package}-' + p + '/{addon}.node'); process.exit(0); }} catch {{}} \ + process.exit(require('fs').existsSync('prebuilds/' + p + '/{addon}.node') ? 0 : 1)\" \ + || node-gyp rebuild", + package = names.package, + addon = names.addon, + ) +} + +/// The npm tarball file name `npm pack` gives `name` at `version` +/// (`kvstore-1.2.0.tgz`, or `acme-kv-1.2.0.tgz` for `@acme/kv`). +pub(crate) fn npm_tarball_name(name: &str, version: &str) -> String { + format!( + "{}-{version}.tgz", + name.trim_start_matches('@').replace('/', "-") + ) +} + /// Render `package.json`. `platform_packages` lists the per-platform /// packages of the packaged layout (empty for plain generation). -pub(crate) fn render_package_json( - names: &NodeNames<'_>, - platform_packages: &[String], - input_basename: &str, -) -> String { - let mut obj = npm_metadata(names.identity, &names.package, input_basename) +pub(crate) fn render_package_json(names: &NodeNames<'_>, platform_packages: &[String]) -> String { + let mut obj = npm_metadata(names.identity, &names.package) .entry("files", JsonValue::str_array(names.files())) .entry("gypfile", JsonValue::Bool(true)) .entry( "scripts", - JsonValue::Object(JsonObject::new().str_entry("install", "node-gyp rebuild")), + JsonValue::Object(JsonObject::new().str_entry("install", install_script(names))), ) .entry( "engines", - JsonValue::Object(JsonObject::new().str_entry("node", ">=18")), + JsonValue::Object(JsonObject::new().str_entry("node", names.node_engine)), ); if !platform_packages.is_empty() { let mut deps = JsonObject::new(); @@ -69,20 +90,32 @@ pub(crate) fn node_platform_tokens(platform: Platform) -> Option<(&'static str, } /// Render a per-platform package's `package.json`, gated by npm `os` and -/// `cpu` so only the matching one installs. +/// `cpu` so only the matching one installs, listing its `files`. pub(crate) fn render_platform_package_json( name: &str, - library: &str, - version: &str, + id: &Identity, (os, cpu): (&str, &str), + files: &[String], ) -> String { - JsonObject::new() + let mut obj = JsonObject::new() .str_entry("name", name) - .str_entry("version", version) + .str_entry("version", &id.version) .str_entry( "description", - format!("Prebuilt {library} native library for {os}/{cpu}"), + format!("Prebuilt {} native library for {os}/{cpu}", id.library), ) + .opt_str_entry("license", id.license.as_deref()); + if let Some(repository) = &id.repository { + obj = obj.entry( + "repository", + JsonValue::Object( + JsonObject::new() + .str_entry("type", "git") + .str_entry("url", repository), + ), + ); + } + obj.entry("files", JsonValue::str_array(files.iter().cloned())) .entry("os", JsonValue::str_array([os])) .entry("cpu", JsonValue::str_array([cpu])) .render() @@ -94,11 +127,7 @@ pub(crate) fn render_platform_package_json( /// (`npm install --{prefix}-library=...`), else `default_dir` (a JavaScript /// expression evaluated in the package directory). An rpath to the addon's /// own directory and to `native_dir` lets the library load from either. -pub(crate) fn render_binding_gyp( - names: &NodeNames<'_>, - default_dir: &str, - input_basename: &str, -) -> String { +pub(crate) fn render_binding_gyp(names: &NodeNames<'_>, default_dir: &str) -> String { let id = names.identity; let env = id.library_env_var(); let npm = format!("npm_config_{}_library", id.prefix); @@ -135,7 +164,7 @@ pub(crate) fn render_binding_gyp( ), ) .entry("targets", JsonValue::Array(vec![JsonValue::Object(target)])); - let mut out = render_prelude(CommentStyle::Hash, input_basename); + let mut out = render_prelude(CommentStyle::Hash); out.push_str(&body.render()); out.push('\n'); out.push_str(&render_trailer(CommentStyle::Hash, "binding.gyp")); @@ -148,26 +177,27 @@ pub(crate) fn render_readme( names: &NodeNames<'_>, modules: &[String], platforms: &[String], - input_basename: &str, ) -> String { let id = names.identity; let pkg = &names.package; let env = id.library_env_var(); let lib = &id.library; - let mut out = render_prelude(CommentStyle::Xml, input_basename); + let mut out = render_prelude(CommentStyle::Xml); out.push_str(&format!("# {pkg}\n\n{}\n\n", id.description_or_default())); out.push_str(&format!( "Node.js bindings for the `{lib}` native library: an ES module over a small \ - N-API addon (`{addon}.c`) that `npm install` compiles with node-gyp. The addon \ - links `{lib}`; set `{env}` to the library file (or its directory) when \ - installing, or pass `--{prefix}-library=` to npm.\n\n", + N-API addon (`{addon}.c`). `npm install` uses a prebuilt addon when one matches \ + the platform (from a per-platform package, or under `prebuilds/-/`) \ + and otherwise compiles it with node-gyp. A compiled addon links `{lib}`; set \ + `{env}` to the library file (or its directory) when installing, or pass \ + `--{prefix}-library=` to npm.\n\n", addon = names.addon, prefix = id.prefix, )); if !platforms.is_empty() { out.push_str( - "The prebuilt library ships in per-platform packages that npm selects \ - through `optionalDependencies`:\n\n", + "The prebuilt library and addon ship in per-platform packages that npm \ + selects through `optionalDependencies`:\n\n", ); for p in platforms { out.push_str(&format!("- `{p}`\n")); diff --git a/crates/weaveffi-gen/src/targets/node/runtime/addon.c b/crates/weaveffi-cli/src/targets/node/runtime/addon.c similarity index 63% rename from crates/weaveffi-gen/src/targets/node/runtime/addon.c rename to crates/weaveffi-cli/src/targets/node/runtime/addon.c index abafe8ca..03b99b85 100644 --- a/crates/weaveffi-gen/src/targets/node/runtime/addon.c +++ b/crates/weaveffi-cli/src/targets/node/runtime/addon.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -19,8 +20,9 @@ * raw convention the shared JavaScript layer documents: direct values as * numbers, bigints, and booleans; strings as strings; bytes and value * buffers as Uint8Arrays; objects, iterators, and cancel tokens as bigint - * handles (null when absent); callback interfaces as adapter objects. A - * failure throws an instance of the runtime's `$Fault` class. + * handles (null when absent); callback interfaces as adapter objects (null + * for an absent optional one). A failure throws an instance of the + * runtime's `$Fault` class. */ typedef {{PREFIX}}_error js_error; @@ -28,23 +30,106 @@ typedef {{PREFIX}}_error js_error; /* Helpers an API may not need. */ #if defined(__GNUC__) || defined(__clang__) #define JS_HELPER static inline __attribute__((unused)) +#define JS_TLS __thread #else #define JS_HELPER static inline +#define JS_TLS __declspec(thread) #endif +/* --------------------------------------------------------------------------- + * JavaScript threads. One `js_thread` per thread that runs JavaScript and + * loaded the addon (the main thread and each worker). It counts the + * synchronous calls into the library in progress on that thread, which a + * callback made on another thread consults while it waits for the + * JavaScript thread (see js_cb_hop). Each environment and each callback + * registration holds a reference. + * ------------------------------------------------------------------------- */ + +typedef struct { + uv_mutex_t mu; + uv_cond_t cv; /* broadcast when a call begins or ends, or a hop finishes */ + unsigned depth; /* synchronous calls into the library in progress */ + uint64_t calls; /* outermost synchronous calls begun so far */ + unsigned waiters; /* producer threads waiting in js_cb_hop */ + unsigned refs; +} js_thread; + +/* The state of the JavaScript thread this is, or NULL on any other thread. */ +static JS_TLS js_thread* js_current = NULL; + +static js_thread* js_thread_new(void) { + js_thread* t = (js_thread*)calloc(1, sizeof *t); + uv_mutex_init(&t->mu); + uv_cond_init(&t->cv); + t->refs = 1; + return t; +} + +static void js_thread_retain(js_thread* t) { + uv_mutex_lock(&t->mu); + t->refs++; + uv_mutex_unlock(&t->mu); +} + +static void js_thread_release(js_thread* t) { + uv_mutex_lock(&t->mu); + bool last = --t->refs == 0; + uv_mutex_unlock(&t->mu); + if (!last) return; + uv_cond_destroy(&t->cv); + uv_mutex_destroy(&t->mu); + free(t); +} + +/* Bracket every synchronous call into the library. */ +JS_HELPER void js_sync_begin(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + if (t->depth++ == 0) t->calls++; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +JS_HELPER void js_sync_end(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + t->depth--; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + /* Per-environment state (one per main thread or worker): the `$Fault` - * constructor `index.js` registers through `$setup`. */ + * constructor `index.js` registers through `$setup`, and the thread. */ typedef struct { napi_ref fault; + js_thread* thread; } js_env; static void js_env_free(napi_env env, void* data, void* hint) { (void)hint; js_env* e = (js_env*)data; if (e->fault != NULL) napi_delete_reference(env, e->fault); + if (js_current == e->thread) js_current = NULL; + js_thread_release(e->thread); free(e); } +/* Set up the environment of the thread loading the addon. */ +static void js_env_init(napi_env env) { + js_env* e = (js_env*)calloc(1, sizeof *e); + e->thread = js_thread_new(); + js_current = e->thread; + napi_set_instance_data(env, e, js_env_free, NULL); +} + +static js_env* js_env_of(napi_env env) { + js_env* e = NULL; + napi_get_instance_data(env, (void**)&e); + return e; +} + JS_HELPER napi_value js_undefined(napi_env env) { napi_value v; napi_get_undefined(env, &v); @@ -55,13 +140,22 @@ static napi_value js_setup(napi_env env, napi_callback_info info) { size_t argc = 1; napi_value argv[1]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); + js_env* e = js_env_of(env); if (e->fault != NULL) napi_delete_reference(env, e->fault); napi_create_reference(env, argv[0], 1, &e->fault); return js_undefined(env); } +/* The `$Fault` constructor, or NULL before `$setup`. */ +static napi_value js_fault_class(napi_env env) { + js_env* e = js_env_of(env); + napi_value ctor = NULL; + if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok) { + return NULL; + } + return ctor; +} + /* --------------------------------------------------------------------------- * Value conversions * ------------------------------------------------------------------------- */ @@ -142,6 +236,21 @@ JS_HELPER napi_value js_take_bytes(napi_env env, const uint8_t* ptr, size_t len) return out; } +/* A module's contract table as a BigUint64Array of `id, hash` pairs. */ +JS_HELPER napi_value js_new_contract(napi_env env, const {{PREFIX}}_contract_entry* table, + size_t len) { + napi_value ab, out; + void* data = NULL; + napi_create_arraybuffer(env, len * 2 * sizeof(uint64_t), &data, &ab); + uint64_t* words = (uint64_t*)data; + for (size_t i = 0; i < len; i++) { + words[2 * i] = table[i].id; + words[2 * i + 1] = table[i].hash; + } + napi_create_typedarray(env, napi_biguint64_array, len * 2, ab, 0, &out); + return out; +} + JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, const uint8_t* payload, size_t payload_len) { napi_value argv[3], ctor, out = NULL; @@ -152,10 +261,8 @@ JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, } else { napi_get_null(env, &argv[2]); } - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); - if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok || - ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { + ctor = js_fault_class(env); + if (ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { napi_create_error(env, NULL, argv[1], &out); napi_set_named_property(env, out, "code", argv[0]); } @@ -421,30 +528,42 @@ JS_HELPER void js_async_done(js_async* a, js_error* err) { * producer: a reference to the adapter object, the JavaScript thread it * belongs to, and a thread-safe function that runs calls made from any other * thread on that thread while the caller waits. + * + * Waiting is what makes a call from another thread safe, and also what could + * deadlock it: if the JavaScript thread is itself inside a synchronous call + * into the library that waits for the calling thread, neither can proceed. + * A waiting call therefore watches the JavaScript thread. While that thread + * stays inside one synchronous call for JS_CB_DEADLOCK_MS without having + * started the callback, the wait is abandoned and the callback fails with + * code -4 instead of hanging (a shorter synchronous call, or one that + * returns to the event loop, just delays the callback). * ------------------------------------------------------------------------- */ +#define JS_CB_DEADLOCK_MS 1000 + typedef struct { napi_env env; napi_ref ref; napi_threadsafe_function tsfn; uv_thread_t js_thread; + js_thread* thread; } js_cb; -/* Every callback frame starts with this header so the dispatcher can reach - * `out_err` without knowing the method. */ -typedef struct { - js_error* out_err; -} js_cb_frame; +typedef enum { JS_REQ_QUEUED, JS_REQ_RUNNING, JS_REQ_DONE, JS_REQ_ABANDONED } js_req_state; /* A call queued from another thread: `method` is the vtable index, or -1 to - * release the registration. */ + * release the registration; `frame` holds the method's slots. The request is + * heap-allocated and holds its own reference to the thread state, so a + * caller that abandons it can return (and the producer can then release the + * registration) while it is still queued: the dispatcher frees it without + * touching the frame or the registration. */ typedef struct { js_cb* cb; + js_thread* thread; int method; void* frame; - uv_mutex_t mu; - uv_cond_t cv; - bool done; + js_error* out_err; + js_req_state state; } js_cb_req; JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { @@ -453,10 +572,12 @@ JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { } JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* name, - napi_threadsafe_function_call_js dispatch) { + napi_threadsafe_function_call_js dispatch) { js_cb* cb = (js_cb*)calloc(1, sizeof *cb); cb->env = env; cb->js_thread = uv_thread_self(); + cb->thread = js_env_of(env)->thread; + js_thread_retain(cb->thread); napi_create_reference(env, adapter, 1, &cb->ref); napi_value resource; napi_create_string_utf8(env, name, NAPI_AUTO_LENGTH, &resource); @@ -470,14 +591,20 @@ JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* na JS_HELPER void js_cb_release(napi_env env, js_cb* cb) { if (env != NULL) napi_delete_reference(env, cb->ref); napi_release_threadsafe_function(cb->tsfn, napi_tsfn_release); + js_thread_release(cb->thread); free(cb); } -/* A callback-interface argument: the adapter object `index.js` built. */ +/* A callback-interface argument: the adapter object `index.js` built, or + * null for an absent optional one. */ JS_HELPER bool js_arg_cb(napi_env env, napi_value v, const char* name, - napi_threadsafe_function_call_js dispatch, js_cb** out) { + napi_threadsafe_function_call_js dispatch, bool nullable, js_cb** out) { napi_valuetype t; napi_typeof(env, v, &t); + if (nullable && (t == napi_null || t == napi_undefined)) { + *out = NULL; + return true; + } if (t != napi_object) return js_type_error(env, "expected a callback interface implementation"); *out = js_cb_register(env, v, name, dispatch); return true; @@ -498,62 +625,188 @@ JS_HELPER void js_cb_free(void* ctx) { if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) free(req); } -/* Run `req` on the JavaScript thread and wait for it. */ -JS_HELPER void js_cb_hop(js_cb_req* req) { - uv_mutex_init(&req->mu); - uv_cond_init(&req->cv); - req->done = false; - if (napi_call_threadsafe_function(req->cb->tsfn, req, napi_tsfn_blocking) == napi_ok) { - uv_mutex_lock(&req->mu); - while (!req->done) uv_cond_wait(&req->cv, &req->mu); - uv_mutex_unlock(&req->mu); - } else { - {{PREFIX}}_error_set(((js_cb_frame*)req->frame)->out_err, -4, - "the callback implementation is no longer reachable"); +/* Run method `method` of `cb` on its JavaScript thread and wait for it, + * failing it with code -4 rather than deadlocking (see above). `what` names + * the method in that failure. */ +JS_HELPER void js_cb_hop(js_cb* cb, int method, void* frame, js_error* out_err, + const char* what) { + js_thread* t = cb->thread; + js_cb_req* req = (js_cb_req*)calloc(1, sizeof *req); + req->cb = cb; + req->thread = t; + js_thread_retain(t); + req->method = method; + req->frame = frame; + req->out_err = out_err; + req->state = JS_REQ_QUEUED; + uv_mutex_lock(&t->mu); + t->waiters++; + uv_mutex_unlock(&t->mu); + if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) { + uv_mutex_lock(&t->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); + {{PREFIX}}_error_set(out_err, -4, "the callback implementation is no longer reachable"); + return; } - uv_cond_destroy(&req->cv); - uv_mutex_destroy(&req->mu); + const uint64_t limit = (uint64_t)JS_CB_DEADLOCK_MS * 1000000u; + bool blocked = false; + uint64_t call = 0, since = 0; + uv_mutex_lock(&t->mu); + while (req->state != JS_REQ_DONE) { + if (req->state == JS_REQ_QUEUED && t->depth > 0) { + uint64_t now = uv_hrtime(); + if (!blocked || call != t->calls) { + blocked = true; + call = t->calls; + since = now; + } + if (now - since >= limit) { + req->state = JS_REQ_ABANDONED; /* the dispatcher frees it */ + t->waiters--; + uv_mutex_unlock(&t->mu); + char msg[512]; + snprintf(msg, sizeof msg, + "%s was called on another thread while the JavaScript thread was blocked in a " + "synchronous call into the library for %d ms; the call likely waits for this " + "callback, so waiting longer would deadlock (call it asynchronously, or call " + "back on the calling thread)", + what, JS_CB_DEADLOCK_MS); + {{PREFIX}}_error_set(out_err, -4, msg); + return; + } + uv_cond_timedwait(&t->cv, &t->mu, limit - (now - since)); + } else { + blocked = false; + uv_cond_wait(&t->cv, &t->mu); + } + } + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); +} + +/* The dispatcher's side of a hop. js_cb_take claims a queued request (false + * when its caller gave up, in which case the request is freed); + * js_cb_finish reports it done. */ +JS_HELPER bool js_cb_take(js_cb_req* req) { + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + bool abandoned = req->state == JS_REQ_ABANDONED; + if (!abandoned) req->state = JS_REQ_RUNNING; + uv_mutex_unlock(&t->mu); + if (abandoned) { + free(req); + js_thread_release(t); + } + return !abandoned; } JS_HELPER void js_cb_finish(js_cb_req* req) { - uv_mutex_lock(&req->mu); - req->done = true; - uv_cond_signal(&req->cv); - uv_mutex_unlock(&req->mu); + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + req->state = JS_REQ_DONE; + uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); } -/* Report the pending JavaScript exception (or `fallback`) through `out_err` - * as a foreign failure (code -4); nothing unwinds through the C frame. */ +/* A JavaScript value's text as a heap-allocated C string (NULL on failure). */ +JS_HELPER char* js_cstring(napi_env env, napi_value v) { + napi_value str; + size_t len = 0; + if (napi_coerce_to_string(env, v, &str) != napi_ok || + napi_get_value_string_utf8(env, str, NULL, 0, &len) != napi_ok) { + return NULL; + } + char* out = (char*)malloc(len + 1); + napi_get_value_string_utf8(env, str, out, len + 1, NULL); + return out; +} + +/* Report the pending JavaScript exception (or `fallback`) through + * `out_err`; nothing unwinds through the C frame. A `$Fault` (a domain error + * a throwing method raised) reports its code, message, and payload; any + * other exception is a foreign failure, code -4, with its message. */ JS_HELPER void js_cb_report(napi_env env, js_error* out_err, const char* fallback) { - char msg[512]; - msg[0] = 0; + int32_t code = -4; + char* msg = NULL; + const uint8_t* payload = NULL; + size_t payload_len = 0; bool pending = false; napi_is_exception_pending(env, &pending); if (pending) { - napi_value exc, text, str; + napi_value exc, text; napi_get_and_clear_last_exception(env, &exc); + napi_value fault = js_fault_class(env); + bool is_fault = false; + if (fault != NULL) napi_instanceof(env, exc, fault, &is_fault); napi_valuetype t; napi_typeof(env, exc, &t); text = exc; if (t == napi_object) napi_get_named_property(env, exc, "message", &text); - if (napi_coerce_to_string(env, text, &str) == napi_ok) { - napi_get_value_string_utf8(env, str, msg, sizeof msg, NULL); + msg = js_cstring(env, text); + if (is_fault) { + napi_value v; + napi_get_named_property(env, exc, "code", &v); + napi_get_value_int32(env, v, &code); + napi_get_named_property(env, exc, "payload", &v); + bool typed = false; + napi_is_typedarray(env, v, &typed); + if (typed) { + napi_typedarray_type type; + void* data = NULL; + napi_get_typedarray_info(env, v, &type, &payload_len, &data, NULL, NULL); + payload = (const uint8_t*)data; + } } napi_is_exception_pending(env, &pending); if (pending) napi_get_and_clear_last_exception(env, &exc); } - {{PREFIX}}_error_set(out_err, -4, msg[0] != 0 ? msg : fallback); + {{PREFIX}}_error_set(out_err, code, msg != NULL && msg[0] != 0 ? msg : fallback); + if (payload != NULL) {{PREFIX}}_error_set_payload(out_err, payload, payload_len); + free(msg); } /* Look up and call `method` on the adapter; false when it threw. */ JS_HELPER bool js_cb_call(napi_env env, js_cb* cb, const char* method, size_t argc, - const napi_value* argv, napi_value* result) { + const napi_value* argv, napi_value* result) { napi_value adapter, fn; napi_get_reference_value(env, cb->ref, &adapter); napi_get_named_property(env, adapter, method, &fn); return napi_call_function(env, adapter, fn, argc, argv, result) == napi_ok; } +/* Hand `len` bytes to the producer through a callback's out slots, as a run + * from {p}_alloc that the producer adopts. */ +JS_HELPER void js_give(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = {{PREFIX}}_alloc(len); + if (len > 0) memcpy(run, data, len); + *out_ptr = run; + *out_len = len; +} + +/* A callback's string return: false (with a pending exception) when the + * value isn't a string. */ +JS_HELPER bool js_ret_str(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + js_str s = JS_STR_INIT; + if (!js_arg_str(env, v, &s)) return false; + js_give(s.ptr, s.len, out_ptr, out_len); + js_str_free(&s); + return true; +} + +/* A callback's bytes or buffer return: a Uint8Array. */ +JS_HELPER bool js_ret_bytes(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + const uint8_t* ptr = NULL; + size_t len = 0; + if (!js_arg_bytes(env, v, &ptr, &len)) return false; + js_give(ptr, len, out_ptr, out_len); + return true; +} + /* --------------------------------------------------------------------------- * The runtime symbols every library exports. * ------------------------------------------------------------------------- */ diff --git a/crates/weaveffi-cli/src/targets/node/runtime/loader.js b/crates/weaveffi-cli/src/targets/node/runtime/loader.js new file mode 100644 index 00000000..8d3c5b90 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/node/runtime/loader.js @@ -0,0 +1,34 @@ +import { createRequire } from 'node:module'; + +// The native entry points: the N-API addon built from {{ADDON}}.c. A prebuilt +// copy comes first (from the `{{PACKAGE}}--` package `npm install` +// picks for this platform, or under prebuilds/-/), then the one +// node-gyp compiles at install time when none matches. +function $loadAddon(name) { + const require = createRequire(import.meta.url); + const platform = `${process.platform}-${process.arch}`; + const candidates = [ + `{{PACKAGE}}-${platform}/${name}`, + `./prebuilds/${platform}/${name}`, + `./build/Release/${name}`, + `./build/Debug/${name}`, + ]; + for (const candidate of candidates) { + try { + return require(candidate); + } catch (e) { + if (e.code !== 'MODULE_NOT_FOUND') throw e; + } + } + throw new Error( + `the native addon ${name} was not found (tried ${candidates.join(', ')}); ` + + 'install the package for this platform, or build the addon with `npm install` ' + + 'or `node-gyp rebuild` in this package', + ); +} + +const $raw = $loadAddon('{{ADDON}}.node'); +$raw.$setup($Fault); + +// Object tokens in value buffers are already bigint handles here. +const $token = (t) => t; diff --git a/crates/weaveffi-cli/src/targets/node/tests.rs b/crates/weaveffi-cli/src/targets/node/tests.rs new file mode 100644 index 00000000..2a1d929b --- /dev/null +++ b/crates/weaveffi-cli/src/targets/node/tests.rs @@ -0,0 +1,263 @@ +use super::*; +use crate::package::{ArtifactKind, FileContent}; +use crate::platform::{BinarySet, NativeBinary, Platform}; +use crate::targets::js::test_api; + +fn files(name: &str, config: &NodeConfig) -> Vec { + let model = test_api(name); + NodeGenerator.files(&model, Utf8Path::new("out"), config) +} + +fn file(files: &[OutputFile], name: &str) -> String { + files + .iter() + .find(|f| f.path.file_name() == Some(name)) + .unwrap_or_else(|| panic!("no {name}")) + .contents + .clone() +} + +fn has(src: &str, needle: &str) { + assert!(src.contains(needle), "missing `{needle}` in:\n{src}"); +} + +#[test] +fn the_package_is_named_after_the_identity() { + let out = files("acme-kv", &NodeConfig::default()); + let names: Vec<&str> = out.iter().filter_map(|f| f.path.file_name()).collect(); + assert_eq!( + names, + [ + "index.js", + "index.d.ts", + "runtime.js", + "acme_kv_node.c", + "acme_kv.h", + "binding.gyp", + "package.json", + "README.md" + ] + ); + has(&file(&out, "package.json"), "\"name\": \"acme-kv\","); + has( + &file(&out, "binding.gyp"), + "\"target_name\": \"acme_kv_node\",", + ); + has(&file(&out, "binding.gyp"), "\"-lacme_kv\""); + has( + &file(&out, "binding.gyp"), + "process.env.ACME_KV_LIBRARY || process.env.npm_config_acme_kv_library", + ); + has(&file(&out, "binding.gyp"), "-Wl,-rpath,@loader_path"); + has( + &file(&out, "index.js"), + "const $raw = $loadAddon('acme_kv_node.node');", + ); + for f in &out { + if f.path.file_name() != Some("package.json") { + assert!(!f.contents.contains("weaveffi_"), "{} is branded", f.path); + } + } + + let renamed = files( + "acme-kv", + &NodeConfig { + name: Some("@acme/kv".into()), + ..NodeConfig::default() + }, + ); + has(&file(&renamed, "package.json"), "\"name\": \"@acme/kv\","); +} + +#[test] +fn index_checks_the_contract_at_load() { + let js = file(&files("kv", &NodeConfig::default()), "index.js"); + has(&js, "$raw.$setup($Fault);"); + has(&js, "const $contract = [\n ['kv', [\n"); + has(&js, "'kv.Store.get'],\n"); + has(&js, "\n$verify($raw, 'kv', 'kv', 4, $contract);\n"); + has( + &js, + "export { KvError, CancelledError } from './runtime.js';", + ); + assert!(!js.contains("checksum"), "{js}"); +} + +#[test] +fn addon_exports_every_symbol_with_the_raw_convention() { + let c = file(&files("kv", &NodeConfig::default()), "kv_node.c"); + assert!(!c.contains("{{"), "unsubstituted placeholder"); + assert!(!c.contains("checksum") && !c.contains("dealloc"), "{c}"); + has(&c, "#include \"kv.h\""); + has(&c, "typedef kv_error js_error;"); + for symbol in [ + "kv_kv_contract", + "kv_kv_Store_new", + "kv_kv_Store_clone", + "kv_kv_Store_destroy", + "kv_kv_Store_ScanIterator_next", + "kv_kv_Store_ScanIterator_destroy", + "kv_kv_Store_wait", + "kv_kv_stats_count", + ] { + has( + &c, + &format!("{{\"{symbol}\", NULL, nx_{symbol}, NULL, NULL, NULL, napi_default, NULL}},"), + ); + } + has( + &c, + " const kv_contract_entry* table = kv_kv_contract(&len);\n return js_new_contract(env, table, len);", + ); + // Strings cross as (ptr, len); a returned string is freed after copying. + // Every call into the library is bracketed for the callback hop. + has( + &c, + "js_sync_begin();\n const uint8_t* r = kv_kv_Store_get((const kv_kv_Store*)self_h, JS_STR_PTR(a0), a0.len, &out_len, &err);\n js_sync_end();", + ); + has(&c, "ret = js_take_bytes(env, r, out_len);"); + // A cancellable launcher takes the token handle after its inputs. + has( + &c, + "if (!js_arg_handle(env, argv[2], &token, true)) goto done;", + ); + has(&c, "kv_kv_Store_wait((const kv_kv_Store*)self_h, a0, (kv_cancel_token*)token, done_kv_kv_Store_wait, a);"); + has( + &c, + "static void done_kv_kv_Store_wait(void* context, kv_error* err, int64_t result) {", + ); + has( + &c, + "js_async* a = js_async_begin(env, JS_R_HANDLE, \"kv_kv_Store_fetch\", &ret);", + ); + // Every declaration precedes the first read that can fail. + has( + &c, + " js_cb* a0 = NULL;\n int32_t a1 = 0;\n if (!js_arg_cb(", + ); + // An optional callback passes a null vtable for none. + has( + &c, + "if (!js_arg_cb(env, argv[0], \"kv_kv_Policy\", dispatch_kv_kv_Policy, true, &a0)) goto done;", + ); + has( + &c, + "kv_kv_install((void*)a0, a0 != NULL ? &vtable_kv_kv_Policy : NULL, &err);", + ); + has(&c, "NAPI_MODULE_INIT() {\n js_env_init(env);"); +} + +#[test] +fn callback_vtables_hop_to_the_javascript_thread() { + let c = file(&files("kv", &NodeConfig::default()), "kv_node.c"); + has(&c, "static bool tramp_kv_kv_Listener_on_message(void* ctx, const uint8_t* p0, size_t p1, uint64_t p2, kv_error* out_err) {"); + has(&c, " if (js_cb_on_js_thread(cb)) {"); + has( + &c, + " js_cb_hop(cb, 0, &f, out_err, \"Listener.onMessage\");", + ); + has(&c, "argv[0] = js_new_str(env, f->p0, f->p1);"); + has(&c, "argv[1] = js_new_u64(env, f->p2);"); + has(&c, "argv[2] = js_new_handle(env, f->p3);"); + has(&c, "static const kv_kv_Listener_vtable vtable_kv_kv_Listener = {sizeof(kv_kv_Listener_vtable), 0, js_cb_free, tramp_kv_kv_Listener_on_message, tramp_kv_kv_Listener_on_bundle};"); + has(&c, " if (!js_cb_take(req)) return;"); +} + +#[test] +fn callback_returns_of_every_family_reach_the_producer() { + let c = file(&files("kv", &NodeConfig::default()), "kv_node.c"); + // A buffer return travels as a {p}_alloc run in the out slots. + has(&c, "static void tramp_kv_kv_Policy_admit(void* ctx, const uint8_t* p0, size_t p1, uint8_t** out_ptr, size_t* out_len, kv_error* out_err) {"); + has( + &c, + " if (!js_ret_bytes(env, result, f->out_ptr, f->out_len)) {\n js_cb_report(env, f->out_err, \"Policy.admit returned a value of the wrong type\");", + ); + has( + &c, + " if (!js_ret_str(env, result, f->out_ptr, f->out_len)) {", + ); + // Objects are returned by value; `I` must be a handle, `I?` may be null. + has(&c, "static kv_kv_Store* tramp_kv_kv_Policy_pick(void* ctx, const uint8_t* p0, size_t p1, kv_error* out_err) {"); + has( + &c, + " if (js_arg_handle(env, result, &h, false)) {\n f->result = (kv_kv_Store*)h;", + ); + has(&c, " if (js_arg_handle(env, result, &h, true)) {"); + has( + &c, + "static kv_kv_Mode tramp_kv_kv_Policy_mode(void* ctx, kv_error* out_err) {", + ); + has(&c, " f->result = (kv_kv_Mode)v;"); +} + +#[test] +fn package_ships_one_npm_package_per_node_platform() { + let model = test_api("kv"); + let mut binaries = BinarySet::new("kv"); + let mut mac = NativeBinary::new(Platform::MacosArm64, "lib/libkv.dylib"); + mac.node_addon = Some("lib/kv_node.node".into()); + binaries.insert(mac); + binaries.insert(NativeBinary::new(Platform::LinuxX64, "lib/libkv.so")); + binaries.insert(NativeBinary::new(Platform::Wasm32, "lib/kv.wasm")); + let ctx = PackageContext::new(&binaries); + let artifacts = NodeGenerator + .package(&model, &ctx, &NodeConfig::default()) + .expect("node packages"); + let paths: Vec<&str> = artifacts.iter().map(|a| a.path.as_str()).collect(); + assert_eq!( + paths, + [ + "node/kv-darwin-arm64-0.1.0.tgz", + "node/kv-linux-x64-0.1.0.tgz", + "node/kv-0.1.0.tgz" + ] + ); + for artifact in &artifacts { + assert_eq!( + artifact.kind, + ArtifactKind::TarGz { + prefix: "package".into() + } + ); + } + let names = + |i: usize| -> Vec<&str> { artifacts[i].files.iter().map(|f| f.path.as_str()).collect() }; + assert_eq!(names(0), ["package.json", "libkv.dylib", "kv_node.node"]); + assert_eq!(names(1), ["package.json", "libkv.so"]); + let text = |i: usize, name: &str| match &artifacts[i].file(name).unwrap().content { + FileContent::Text(t) => t.clone(), + _ => panic!("{name} is text"), + }; + let json = |i: usize| -> serde_json::Value { + serde_json::from_str(&text(i, "package.json")).expect("valid package.json") + }; + let mac_json = json(0); + assert_eq!(mac_json["name"], "kv-darwin-arm64"); + assert_eq!(mac_json["os"], serde_json::json!(["darwin"])); + assert_eq!(mac_json["cpu"], serde_json::json!(["arm64"])); + assert_eq!( + mac_json["files"], + serde_json::json!(["libkv.dylib", "kv_node.node"]) + ); + let main_json = json(2); + assert_eq!( + main_json["optionalDependencies"], + serde_json::json!({"kv-darwin-arm64": "0.1.0", "kv-linux-x64": "0.1.0"}) + ); + let install = main_json["scripts"]["install"].as_str().unwrap(); + assert!(install.ends_with("|| node-gyp rebuild"), "{install}"); + assert!( + install.contains("require.resolve('kv-' + p + '/kv_node.node')"), + "{install}" + ); + assert_eq!(node_platform_tokens(Platform::AndroidArm64), None); +} + +#[test] +fn npm_tarball_names_follow_npm_pack() { + assert_eq!(super::npm_tarball_name("kv", "1.0.0"), "kv-1.0.0.tgz"); + assert_eq!( + super::npm_tarball_name("@acme/kv", "1.0.0"), + "acme-kv-1.0.0.tgz" + ); +} diff --git a/crates/weaveffi-cli/src/targets/python/callbacks.rs b/crates/weaveffi-cli/src/targets/python/callbacks.rs new file mode 100644 index 00000000..fc13625c --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/callbacks.rs @@ -0,0 +1,284 @@ +//! Callback interfaces: the abstract base class the consumer subclasses, +//! and the ABI side the producer calls through, satisfying +//! [`weaveffi_model::plan::CallbackProtocol`]: one `CFUNCTYPE` per method, +//! the `ctypes.Structure` mirroring the C vtable (its `size`, `flags`, and +//! `free` header first), one trampoline per method, and the single static +//! vtable whose function objects are pinned at module scope. + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding}; +use weaveffi_model::ty::{Family, Prim, Ty}; + +use crate::targets::python::calls::{adopt_expr, cfunctype, slot_params}; +use crate::targets::python::codec::{decode_expr, encode_stmts}; +use crate::targets::python::docs::{fn_docstring, with_deprecation}; +use crate::targets::python::types::{ + py_local, py_member_name, py_name, py_return_hint, py_slot_hint, py_type_hint, +}; +use crate::targets::python::Gen; + +/// Render one callback interface: the ABC, the vtable layout, the +/// trampolines, and the static vtable. +/// +/// Each trampoline looks its implementation up by the integer `ctx`, +/// converts the arguments, calls the method, and hands the return back +/// through the C return or the `out_ptr`/`out_len` slots. An exception is +/// reported through `out_err` (as a domain code when the method declares +/// errors and raised one, else `-4`) and nothing unwinds through the C +/// frame. `ctypes` acquires the GIL on entry, so the producer may call from +/// any thread. +pub(crate) fn render_callback_interface( + w: &mut CodeWriter, + g: &Gen<'_>, + cb: &CallbackInterfaceBinding, + domain: Option<&str>, +) { + let name = &cb.name; + + w.blank().blank(); + w.line(format!("class {name}(abc.ABC):")); + w.scope(|w| { + let mut usage = format!( + "Subclass it and implement every method to pass an instance where the\n\ + API takes a `{name}`. The producer may call the methods from any\n\ + thread until it releases the instance." + ); + if let Some(domain) = domain.filter(|_| cb.methods.iter().any(|m| m.throws)) { + usage.push_str(&format!( + " A method that declares errors may\n\ + raise a `{domain}` code, which reaches the producer with its fields;\n\ + any other exception reaches it as {}.FOREIGN_ERROR_CODE (-4).", + g.root_error + )); + } else { + usage.push_str(&format!( + " An exception a method raises\n\ + reaches the producer as {}.FOREIGN_ERROR_CODE (-4).", + g.root_error + )); + } + let doc = with_deprecation(cb.doc.as_deref(), cb.deprecated.as_deref()); + let doc = match doc { + Some(d) => format!("{d}\n\n{usage}"), + None => usage, + }; + fn_docstring(w, Some(&doc), &[], None); + for m in &cb.methods { + let mut sig: Vec = vec!["self".into()]; + sig.extend( + m.params + .iter() + .map(|p| format!("{}: {}", py_name(&p.name), py_type_hint(&p.ty))), + ); + w.blank(); + w.line("@abc.abstractmethod"); + w.line(format!( + "def {}({}) -> {}:", + py_member_name(&m.name), + sig.join(", "), + py_return_hint(m.ret.as_ref()) + )); + w.scope(|w| { + let doc = with_deprecation(m.doc.as_deref(), m.deprecated.as_deref()); + let raises = domain.filter(|_| m.throws).map(|d| { + ( + d, + "To report one of the domain's error codes to the producer.", + ) + }); + if doc.is_none() && raises.is_none() && m.params.iter().all(|p| p.doc.is_none()) { + w.line("..."); + } else { + fn_docstring(w, doc.as_deref(), &m.params, raises); + } + }); + } + }); + + // One function-pointer type per method, then the vtable layout: the + // header (`size`, `flags`, `free`), then the methods in declaration + // order. + w.blank().blank(); + for m in &cb.methods { + w.line(format!( + "_{name}_{}_t = {}", + m.name, + cfunctype(&m.abi_ret, &m.abi_params) + )); + } + w.blank().blank(); + w.line(format!("class _{name}Vtable(ctypes.Structure):")); + w.scope(|w| { + w.line(format!("\"\"\"The C vtable `{}`.\"\"\"", cb.vtable_tag)); + w.blank(); + w.line("_fields_ = ["); + w.scope(|w| { + w.line("(\"size\", ctypes.c_uint32),"); + w.line("(\"flags\", ctypes.c_uint32),"); + w.line("(\"free\", _CallbackFree),"); + for m in &cb.methods { + w.line(format!("(\"{0}\", _{name}_{0}_t),", m.name)); + } + }); + w.line("]"); + }); + + for m in &cb.methods { + render_trampoline(w, cb, m, domain.filter(|_| m.throws)); + } + + // The one static vtable. Each field keeps its function object alive, and + // the vtable itself lives at module scope for the process lifetime. + w.blank().blank(); + w.line(format!("_{name}_vtable = _{name}Vtable(")); + w.scope(|w| { + w.line(format!("ctypes.sizeof(_{name}Vtable),")); + w.line("0,"); + w.line("_callback_free_fn,"); + for m in &cb.methods { + w.line(format!("_{name}_{0}_t(_{name}_{0}),", m.name)); + } + }); + w.line(")"); + w.line(format!( + "_{name}_vtable_ptr = ctypes.addressof(_{name}_vtable)" + )); +} + +/// The expression converting one trampoline parameter's C slots into the +/// value handed to the implementation. Strings, bytes, and buffers are +/// borrowed for the call and copied; an object transfers one strong +/// reference, which a new wrapper adopts. +fn trampoline_arg(p: &ParamBinding) -> String { + let data = || { + format!( + "_peek_bytes({}, {})", + py_local(&format!("{}_ptr", p.name)), + py_local(&format!("{}_len", p.name)) + ) + }; + let n = py_local(&p.name); + match p.ty.family() { + Family::String => format!("{}.decode(\"utf-8\")", data()), + Family::Bytes => data(), + Family::Buffer => decode_expr(&data(), &p.ty), + Family::Object { nullable } => adopt_expr(&p.ty, &n, nullable), + Family::Direct => match &p.ty { + Ty::Enum(name) => format!("{name}({n})"), + _ => n, + }, + Family::Callback { .. } | Family::Iterator => { + unreachable!("callback interfaces and iterators are never callback arguments") + } + } +} + +/// `(coercion, default)` for a direct return: the conversion applied to the +/// implementation's result (inside the `try`, so a wrong type is reported +/// like any other failure), and the value returned after a failure. +fn direct_return(ty: &Ty) -> (&'static str, &'static str) { + match ty { + Ty::Prim(Prim::Bool) => ("bool", "False"), + Ty::Prim(Prim::F32 | Prim::F64) => ("float", "0.0"), + // Integers and C-style enums (an `IntEnum` is an `int`). + _ => ("int", "0"), + } +} + +/// Render the trampoline for one callback method: a `def` whose parameters +/// are the vtable entry's C slots (`ctx`, the parameter slots, the return's +/// out slots, `out_err`). `domain` names the error domain whose codes the +/// method may report (only for a method that declares errors). +fn render_trampoline( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, + domain: Option<&str>, +) { + w.blank().blank(); + w.line(format!( + "def _{}_{}({}) -> {}:", + cb.name, + m.name, + slot_params(&m.abi_params), + py_slot_hint(&m.abi_ret) + )); + w.scope(|w| { + w.line("try:"); + w.scope(|w| { + // Arguments carrying objects (one, or tokens in a buffer) are + // adopted first, so a failure converting another argument (or + // finding the implementation) still releases them. + let args: Vec = m + .params + .iter() + .map(|p| { + if p.ty.contains_object() { + let local = format!("_{}", py_name(&p.name)); + w.line(format!("{local} = {}", trampoline_arg(p))); + local + } else { + trampoline_arg(p) + } + }) + .collect(); + let call = format!( + "_callback_get(ctx).{}({})", + py_member_name(&m.name), + args.join(", ") + ); + let Some(ty) = &m.ret else { + w.line(call); + return; + }; + if ty.family() == Family::Direct { + w.line(format!("return {}({call})", direct_return(ty).0)); + return; + } + w.line(format!("_ret = {call}")); + match ty.family() { + Family::Object { nullable } => { + let class = ty.interface_name().expect("object names an interface"); + let helper = if nullable { + "_callback_return_object_opt" + } else { + "_callback_return_object" + }; + w.line(format!("return {helper}(_ret, {class})")); + } + Family::String => { + w.line("_callback_return_bytes(out_ptr, out_len, _ret.encode(\"utf-8\"))"); + } + Family::Bytes => { + w.line("_callback_return_bytes(out_ptr, out_len, bytes(_ret))"); + } + Family::Buffer => { + for line in encode_stmts("_w", "_ret", ty) { + w.line(line); + } + w.line("_callback_return_bytes(out_ptr, out_len, _w.finish())"); + } + Family::Direct => unreachable!("returned above"), + Family::Callback { .. } | Family::Iterator => { + unreachable!("validation rejects {ty} as a callback return") + } + } + }); + w.line("except BaseException as exc:"); + w.scope(|w| { + match domain { + Some(domain) => w.line(format!("_callback_fail(out_err, exc, {domain})")), + None => w.line("_callback_fail(out_err, exc)"), + }; + match m.ret.as_ref().map(|ty| (ty, ty.family())) { + Some((ty, Family::Direct)) => { + w.line(format!("return {}", direct_return(ty).1)); + } + Some((_, Family::Object { .. })) => { + w.line("return None"); + } + _ => {} + } + }); + }); +} diff --git a/crates/weaveffi-gen/src/targets/python/calls.rs b/crates/weaveffi-cli/src/targets/python/calls.rs similarity index 50% rename from crates/weaveffi-gen/src/targets/python/calls.rs rename to crates/weaveffi-cli/src/targets/python/calls.rs index 9e4234a2..f0a6f3b8 100644 --- a/crates/weaveffi-gen/src/targets/python/calls.rs +++ b/crates/weaveffi-cli/src/targets/python/calls.rs @@ -1,28 +1,26 @@ -//! Call rendering: sync, async, and iterator wrappers, their module-level -//! `ctypes` bindings, and the consumer-facing abstract base class, static -//! vtable, and trampolines of each callback interface. +//! Call rendering: sync, async, and iterator wrappers and their module-level +//! `ctypes` bindings. //! //! Every C prototype is bound once, at import time, from the model's lowered //! [`AbiFn`] (`argtypes` and `restype` come straight from its slots). //! Marshalling dispatch is driven by the shared plans: [`ArgPass`] decides -//! how each parameter crosses and [`Family`] how a result (or a trampoline -//! argument) is received, so this module never re-derives those shapes. +//! how each parameter crosses and [`Family`] how a result is received, so +//! this module never re-derives those shapes. use crate::codegen::CodeWriter; -use crate::utils::local_type_name; use weaveffi_model::abi::{AbiParam, CType}; use weaveffi_model::model::{ - AbiFn, CallShape, CallbackInterfaceBinding, CallbackMethodBinding, ErrorBinding, Family, - FnBinding, IteratorBinding, ModuleBinding, ParamBinding, Ty, + AbiFn, CallShape, ErrorBinding, FnBinding, IteratorBinding, ModuleBinding, ParamBinding, }; use weaveffi_model::plan::ArgPass; +use weaveffi_model::ty::{Family, Ty}; -use crate::targets::python::codec::{py_decode_expr, py_write_stmts}; -use crate::targets::python::docs::{emit_docstring, emit_fn_docstring}; -use crate::targets::python::entities::{py_checker_name, py_factory_name}; +use crate::targets::python::codec::{decode_expr, encode_stmts}; +use crate::targets::python::docs::{fn_docstring, with_deprecation}; +use crate::targets::python::entities::py_raise_factory; use crate::targets::python::types::{ - py_binding_name, py_ctype, py_local, py_member_name, py_name, py_str_literal, py_type_hint, - py_wrapper_fn_name, Slot, + py_binding_name, py_ctype, py_local, py_member_name, py_name, py_object_member, py_return_hint, + py_slot_hint, py_str_literal, py_type_hint, Slot, }; use crate::targets::python::Gen; @@ -42,13 +40,6 @@ pub(crate) enum FnScope { Init, } -impl FnScope { - /// Indentation depth of the `def` line (0 at module scope, 1 in a class). - fn depth(self) -> usize { - usize::from(!matches!(self, FnScope::Free)) - } -} - /// The lowered C function a callable's wrapper invokes. fn launcher(f: &FnBinding) -> &AbiFn { match &f.shape { @@ -78,13 +69,23 @@ fn bind_line(g: &Gen<'_>, abi: &AbiFn, callback_type: Option<&str>) -> String { } /// The `CFUNCTYPE(...)` spelling for a C function-pointer signature. -fn cfunctype(ret: &CType, params: &[AbiParam]) -> String { +pub(crate) fn cfunctype(ret: &CType, params: &[AbiParam]) -> String { let parts: Vec = std::iter::once(py_ctype(ret, Slot::Recv)) .chain(params.iter().map(|p| py_ctype(&p.ty, Slot::Recv))) .collect(); format!("ctypes.CFUNCTYPE({})", parts.join(", ")) } +/// The annotated parameter list of a function `ctypes` calls back with the +/// C slots `params` (a trampoline or an async completion). +pub(crate) fn slot_params(params: &[AbiParam]) -> String { + let all: Vec = params + .iter() + .map(|p| format!("{}: {}", py_local(&p.name), py_slot_hint(&p.ty))) + .collect(); + all.join(", ") +} + /// The module-level stem of an async function's completion objects: the /// launcher's binding name without `_c`. fn completion_stem(g: &Gen<'_>, f: &FnBinding) -> String { @@ -102,43 +103,46 @@ fn iterator_class(g: &Gen<'_>, it: &IteratorBinding) -> String { /// return, an async result, or an iterator element) from its pointer (or /// direct value) expression `p` and length expression `len`, releasing what /// it owes. -fn py_recv_expr(ty: &Ty, p: &str, len: &str) -> String { +fn recv_expr(ty: &Ty, p: &str, len: &str) -> String { match ty.family() { Family::String => format!("_take_str({p}, {len})"), Family::Bytes => format!("_take_bytes({p}, {len})"), - Family::Buffer => py_decode_expr(&format!("_take_bytes({p}, {len})"), ty), - Family::Object { nullable } => { - let class = local_type_name(ty.interface_name().expect("object names an interface")); - if nullable { - format!("({class}._adopt({p}) if {p} else None)") - } else { - format!("{class}._adopt(_required({p}))") - } - } + Family::Buffer => decode_expr(&format!("_take_bytes({p}, {len})"), ty), + Family::Object { nullable } => adopt_expr(ty, p, nullable), Family::Direct => match ty { - Ty::Enum(name) => format!("{}({p})", local_type_name(name)), + Ty::Enum(name) => format!("{name}({p})"), _ => p.to_string(), }, - Family::Callback | Family::Iterator => { + Family::Callback { .. } | Family::Iterator => { unreachable!("callback interfaces and iterators are never received as values") } } } +/// The expression adopting the object pointer `p` (one strong reference) +/// into a new wrapper of `ty`'s interface; `None` for a null `I?`. +pub(crate) fn adopt_expr(ty: &Ty, p: &str, nullable: bool) -> String { + let class = ty.interface_name().expect("object names an interface"); + if nullable { + format!("{class}._adopt({p}) if {p} else None") + } else { + format!("{class}._adopt(_required({p}))") + } +} + /// Emit every module-level piece a callable needs ahead of its wrapper: the /// bound launcher, plus the iterator class (iterator shape) or the static /// completion trampoline (async shape). `owner` names the callable in docs /// (`Store.get` for a member); `packed` appends a sync member's binding to /// the block the previous member started, with no blank lines between. pub(crate) fn render_bindings( - out: &mut String, + w: &mut CodeWriter, g: &Gen<'_>, f: &FnBinding, error: Option<&ErrorBinding>, owner: &str, packed: bool, ) { - let mut w = CodeWriter::four_space(); match &f.shape { CallShape::Sync(abi) => { if !packed { @@ -149,27 +153,19 @@ pub(crate) fn render_bindings( CallShape::Iterator(it) => { w.blank().blank(); w.line(bind_line(g, &it.launch, None)); - render_iterator_class(&mut w, g, f, it, error, owner); + render_iterator_class(w, g, f, it, error, owner); } CallShape::Async(a) => { let stem = completion_stem(g, f); let cb_type = py_binding_name(&a.callback_type, g.prefix); - let factory = match error { - Some(eb) if f.throws => py_factory_name(eb), - _ => "_error_from".to_string(), - }; - let slots: Vec = a - .callback_params - .iter() - .map(|p| py_slot_name(&p.name)) - .collect(); + let factory = py_raise_factory(f, error); let value = match &f.ret { None => "None".to_string(), Some(ty) => match ty.family() { Family::String | Family::Bytes | Family::Buffer => { - py_recv_expr(ty, "result_ptr", "result_len") + recv_expr(ty, "result_ptr", "result_len") } - _ => py_recv_expr(ty, "result", ""), + _ => recv_expr(ty, "result", ""), }, }; w.blank().blank(); @@ -181,7 +177,7 @@ pub(crate) fn render_bindings( w.blank().blank(); w.line(format!( "def {stem}_complete({}) -> None:", - slots.join(", ") + slot_params(&a.callback_params) )); w.scope(|w| { w.line("# Runs once, on a producer thread: take ownership of the result"); @@ -208,7 +204,6 @@ pub(crate) fn render_bindings( w.line(format!("{stem}_completion = {cb_type}({stem}_complete)")); } } - out.push_str(&w.finish()); } /// Render the module-level class for one iterator-returning callable, @@ -227,17 +222,8 @@ fn render_iterator_class( let class = iterator_class(g, it); let next = py_binding_name(&it.next.symbol, g.prefix); let destroy = py_binding_name(&it.destroy_symbol, g.prefix); - let checker = py_checker_name(f, error); - let item_ty = it - .next - .params - .iter() - .find(|p| p.name == "out_item") - .and_then(|p| match &p.ty { - CType::Ptr { pointee, .. } => Some(py_ctype(pointee, Slot::Recv)), - _ => None, - }) - .expect("iterator next writes through out_item"); + let factory = py_raise_factory(f, error); + let item_ty = py_ctype(it.item_ctype(), Slot::Recv); let has_len = it.next.params.iter().any(|p| p.name == "out_len"); w.line(bind_line(g, &it.next, None)); @@ -274,7 +260,10 @@ fn render_iterator_class( w.scope(|w| { w.line("self._release()"); }); - w.line(format!("{checker}(_err)")); + w.line("if _err.code:"); + w.scope(|w| { + w.line(format!("raise {factory}(*_read_error(_err))")); + }); w.line("if not _more:"); w.scope(|w| { w.line("self.close()"); @@ -282,21 +271,30 @@ fn render_iterator_class( }); w.line(format!( "return {}", - py_recv_expr(&it.elem, "_item.value", "_len.value") + recv_expr(&it.elem, "_item.value", "_len.value") )); }); }); } /// The emitted Python name for a callable in `scope`. -fn py_fn_name(g: &Gen<'_>, module: &ModuleBinding, f: &FnBinding, scope: FnScope) -> String { +fn py_fn_name(f: &FnBinding, scope: FnScope) -> String { match scope { - FnScope::Free => py_wrapper_fn_name(&module.path, &f.name, g.strip_module_prefix), FnScope::Init => "__init__".to_string(), - _ => py_member_name(&f.name), + FnScope::Factory | FnScope::Method | FnScope::Static => py_object_member(&f.name), + FnScope::Free => py_member_name(&f.name), } } +/// One object lent to a call: the local holding its pointer, the expression +/// lending it, and the statement returning it. +#[derive(Clone)] +struct Lend { + local: String, + acquire: String, + release: String, +} + /// One parameter's local preparation and its C argument expressions. struct ParamPlan { /// Statements run before any object is lent (encoding). @@ -304,13 +302,13 @@ struct ParamPlan { /// Statements run inside the innermost borrow scope, just before the /// call (finishing value buffers, registering callbacks). late: Vec, - /// The object lent for the call: `(local, acquire expr, release stmt)`. - lend: Option<(String, String, String)>, + /// The object lent for the call. + lend: Option, /// The C argument expressions, in slot order. args: Vec, } -fn plan_param(p: &ParamBinding, next: &mut usize) -> ParamPlan { +fn plan_param(p: &ParamBinding) -> ParamPlan { let n = py_name(&p.name); let mut plan = ParamPlan { prep: vec![], @@ -332,54 +330,69 @@ fn plan_param(p: &ParamBinding, next: &mut usize) -> ParamPlan { // parameter has encoded and every lent object is held, so a failure // anywhere before the call leaks no strong reference. ArgPass::Buffer { .. } => { - let mut w = CodeWriter::four_space(); - w.line(format!("_{n}_w = _Writer()")); - py_write_stmts(&mut w, &format!("_{n}_w"), &n, &p.ty, next); - plan.prep = w.finish().lines().map(str::to_string).collect(); + plan.prep + .extend(encode_stmts(&format!("_{n}_w"), &n, &p.ty)); plan.late.push(format!("_{n}_b = _{n}_w.finish()")); plan.args = vec![format!("_{n}_b"), format!("len(_{n}_b)")]; } ArgPass::Object { nullable, .. } => { - let class = local_type_name(p.ty.interface_name().expect("object names an interface")); - let (acquire, release) = if nullable { - ( - format!("_lend_opt({n}, {class})"), - format!("_release_opt({n})"), - ) + let class = p.ty.interface_name().expect("object names an interface"); + plan.lend = Some(if nullable { + Lend { + local: format!("_{n}_p"), + acquire: format!("_lend_opt({n}, {class})"), + release: format!("_release_opt({n})"), + } } else { - (format!("_lend({n}, {class})"), format!("{n}._release()")) - }; - plan.lend = Some((format!("_{n}_p"), acquire, release)); + Lend { + local: format!("_{n}_p"), + acquire: format!("_lend({n}, {class})"), + release: format!("{n}._release()"), + } + }); plan.args.push(format!("_{n}_p")); } - ArgPass::Callback { .. } => { - let cb = local_type_name( + // Registered last, so nothing between registration and the call can + // fail and strand the entry; the producer releases it with `free`. + ArgPass::Callback { nullable, .. } => { + let cb = p.ty.callback_interface_name() - .expect("callback names a callback interface"), - ); - plan.late - .push(format!("_{n}_ctx = _callback_register({n}, {cb})")); - plan.args = vec![ - format!("_{n}_ctx"), - format!("ctypes.addressof(_{cb}_vtable)"), - ]; + .expect("callback names a callback interface"); + let vtable = format!("_{cb}_vtable_ptr"); + if nullable { + plan.late + .push(format!("_{n}_ctx = _callback_register_opt({n}, {cb})")); + plan.args = vec![ + format!("_{n}_ctx"), + format!("{vtable} if {n} is not None else None"), + ]; + } else { + plan.late + .push(format!("_{n}_ctx = _callback_register({n}, {cb})")); + plan.args = vec![format!("_{n}_ctx"), vtable]; + } } } plan } /// Render one callable's wrapper `def` (its bindings are emitted separately -/// by [`render_bindings`]). A throwing callable raises `module`'s error domain. +/// by [`render_bindings`]). A throwing callable raises the error domain in +/// scope for `module`; any other failure raises the unchecked trap. pub(crate) fn render_callable( - out: &mut String, + w: &mut CodeWriter, g: &Gen<'_>, module: &ModuleBinding, f: &FnBinding, scope: FnScope, ) { - let error = module.error.as_ref(); - let depth = scope.depth(); - let raises = error.filter(|_| f.throws).map(|eb| eb.type_name.as_str()); + let error = g.model.error_domain(module); + let raises = error.filter(|_| f.throws).map(|eb| { + ( + eb.type_name.as_str(), + "If the call reports one of the domain's error codes.", + ) + }); let abi = launcher(f); let binding = py_binding_name(&abi.symbol, g.prefix); @@ -395,16 +408,11 @@ pub(crate) fn render_callable( ); let ret_hint = match scope { FnScope::Init => "None".to_string(), - _ => f - .ret - .as_ref() - .map(py_type_hint) - .unwrap_or_else(|| "None".to_string()), + _ => py_return_hint(f.ret.as_ref()), }; - let mut w = CodeWriter::four_space().with_depth(depth); w.blank(); - if depth == 0 { + if matches!(scope, FnScope::Free) { w.blank(); } match scope { @@ -418,26 +426,22 @@ pub(crate) fn render_callable( } w.line(format!( "{}def {}({}) -> {ret_hint}:", - if f.is_async { "async " } else { "" }, - py_fn_name(g, module, f, scope), + if f.is_async() { "async " } else { "" }, + py_fn_name(f, scope), sig.join(", "), )); w.indent(); - let doc = match (&f.shape, &f.doc) { - (CallShape::Iterator(_), d) => { - let streaming = "Returns a lazy iterator: each step pulls one element from the\n\ - producer. Exhaust or close() the iterator to release its native\n\ - handle (garbage collection also releases it)."; - Some(match d { - Some(d) => format!("{}\n\n{streaming}", d.trim()), - None => streaming.to_string(), - }) - } - (_, d) => d.clone(), - }; - let mut fdoc = String::new(); - emit_fn_docstring(&mut fdoc, &doc, &f.params, &w.indent_str(), raises); - w.raw(fdoc); + let mut doc = with_deprecation(f.doc.as_deref(), f.deprecated.as_deref()); + if let CallShape::Iterator(_) = f.shape { + let streaming = "Returns a lazy iterator: each step pulls one element from the\n\ + producer. Exhaust or close() the iterator to release its native\n\ + handle (garbage collection also releases it)."; + doc = Some(match doc { + Some(d) => format!("{d}\n\n{streaming}"), + None => streaming.to_string(), + }); + } + fn_docstring(w, doc.as_deref(), &f.params, raises); if let Some(msg) = &f.deprecated { w.line(format!( @@ -446,8 +450,7 @@ pub(crate) fn render_callable( )); } - let mut next = 0; - let plans: Vec = f.params.iter().map(|p| plan_param(p, &mut next)).collect(); + let plans: Vec = f.params.iter().map(plan_param).collect(); for line in plans.iter().flat_map(|p| &p.prep) { w.line(line); } @@ -476,27 +479,24 @@ pub(crate) fn render_callable( } let call = format!("{binding}({})", args.join(", ")); - let mut lends: Vec<(String, String, String)> = Vec::new(); + let mut lends: Vec = Vec::new(); if f.has_self { - lends.push(( - "_self_p".into(), - "self._acquire()".into(), - "self._release()".into(), - )); + lends.push(Lend { + local: "_self_p".into(), + acquire: "self._acquire()".into(), + release: "self._release()".into(), + }); } lends.extend(plans.iter().filter_map(|p| p.lend.clone())); let late: Vec = plans.iter().flat_map(|p| p.late.clone()).collect(); - let is_async = matches!(f.shape, CallShape::Async(_)); - if is_async { + if let CallShape::Async(_) = f.shape { w.line("_call, _future = _async_begin()"); if f.cancellable { w.line("_token = _cancel_token_create()"); } w.line("try:"); - w.indent(); - emit_lent_call(&mut w, &lends, &late, &call); - w.dedent(); + w.scope(|w| emit_lent_call(w, &lends, &late, &call)); w.line("except BaseException:"); w.scope(|w| { w.line("_async_abandon(_call)"); @@ -505,12 +505,23 @@ pub(crate) fn render_callable( } w.line("raise"); }); - if f.cancellable { - w.line("return await _async_wait_cancellable(_future, _token)"); + let wait = if f.cancellable { + "await _async_wait_cancellable(_future, _token)" } else { - w.line("return await _future"); + "await _future" + }; + // The completion built the value; the annotation hands its type to + // checkers. + match &f.ret { + Some(ty) => { + w.line(format!("_result: {} = {wait}", py_type_hint(ty))); + w.line("return _result"); + } + None => { + w.line(wait); + } } - out.push_str(&w.finish()); + w.dedent(); return; } @@ -518,14 +529,24 @@ pub(crate) fn render_callable( if abi.params.iter().any(|p| p.name == "out_len") { w.line("_out_len = ctypes.c_size_t()"); } - let has_ret = f.ret.is_some(); - let call = if has_ret { - format!("_ret = {call}") - } else { - call + // A direct result is used as returned, so its annotation tells checkers + // what the untyped `ctypes` call produced. + let call = match &f.ret { + Some(ty @ Ty::Prim(_)) if ty.family() == Family::Direct => { + format!("_ret: {} = {call}", py_type_hint(ty)) + } + Some(_) => format!("_ret = {call}"), + None if matches!(scope, FnScope::Init | FnScope::Factory) => format!("_ret = {call}"), + None => call, }; - emit_lent_call(&mut w, &lends, &late, &call); - w.line(format!("{}(_err)", py_checker_name(f, error))); + emit_lent_call(w, &lends, &late, &call); + w.line("if _err.code:"); + w.scope(|w| { + w.line(format!( + "raise {}(*_read_error(_err))", + py_raise_factory(f, error) + )); + }); match (scope, &f.shape) { (FnScope::Init, _) => { @@ -541,23 +562,18 @@ pub(crate) fn render_callable( if let Some(ty) = &f.ret { w.line(format!( "return {}", - py_recv_expr(ty, "_ret", "_out_len.value") + recv_expr(ty, "_ret", "_out_len.value") )); } } } - out.push_str(&w.finish()); + w.dedent(); } /// Emit the C call inside one `try`/`finally` per lent object (outermost /// first), with the late statements (buffer finishing, callback /// registration) just before the call. -fn emit_lent_call( - w: &mut CodeWriter, - lends: &[(String, String, String)], - late: &[String], - call: &str, -) { +fn emit_lent_call(w: &mut CodeWriter, lends: &[Lend], late: &[String], call: &str) { match lends.split_first() { None => { for line in late { @@ -565,230 +581,14 @@ fn emit_lent_call( } w.line(call); } - Some(((local, acquire, release), rest)) => { - w.line(format!("{local} = {acquire}")); + Some((lend, rest)) => { + w.line(format!("{} = {}", lend.local, lend.acquire)); w.line("try:"); - w.indent(); - emit_lent_call(w, rest, late, call); - w.dedent(); + w.scope(|w| emit_lent_call(w, rest, late, call)); w.line("finally:"); w.scope(|w| { - w.line(release); + w.line(&lend.release); }); } } } - -// ── Callback interfaces ── - -/// The Python spelling of a trampoline slot name: the ABI slot name, -/// keyword-escaped. -fn py_slot_name(slot: &str) -> String { - py_local(slot) -} - -/// The expression converting one trampoline parameter's C slots into the -/// value handed to the implementation. Strings, bytes, and buffers are -/// borrowed for the dispatch and copied; an object transfers one strong -/// reference, which the wrapper adopts. -fn py_trampoline_arg(p: &ParamBinding) -> String { - let n = py_slot_name(&p.name); - let data = || { - format!( - "_peek_bytes({}, {})", - py_slot_name(&format!("{}_ptr", p.name)), - py_slot_name(&format!("{}_len", p.name)) - ) - }; - match p.ty.family() { - Family::String => format!("{}.decode(\"utf-8\")", data()), - Family::Bytes => data(), - Family::Buffer => py_decode_expr(&data(), &p.ty), - Family::Object { nullable } => { - let class = local_type_name(p.ty.interface_name().expect("object names an interface")); - if nullable { - format!("({class}._adopt({n}) if {n} else None)") - } else { - format!("{class}._adopt(_required({n}))") - } - } - Family::Direct => match &p.ty { - Ty::Enum(name) => format!("{}({n})", local_type_name(name)), - _ => n, - }, - Family::Callback | Family::Iterator => { - unreachable!("callback interfaces and iterators are never callback arguments") - } - } -} - -/// `(coercion, default)` for a callback method's direct return: the Python -/// conversion applied to the implementation's result (inside the `try`, so -/// a wrong type is reported like any other failure), and the value returned -/// after a failure. `None` for a void method. -fn py_trampoline_return(ret: Option<&Ty>) -> Option<(&'static str, &'static str)> { - match ret { - None => None, - Some(Ty::Bool) => Some(("bool", "False")), - Some(Ty::F32 | Ty::F64) => Some(("float", "0.0")), - // Integers and C-style enums (an `IntEnum` is an `int`). - Some(_) => Some(("int", "0")), - } -} - -/// Render one callback interface: the abstract base class the consumer -/// subclasses, then the ABI side satisfying -/// [`weaveffi_model::plan::CallbackProtocol`]: one `CFUNCTYPE` per method -/// plus `free`, the `ctypes.Structure` mirroring the C vtable, one -/// trampoline per method, and the single process-wide static vtable whose -/// function objects are pinned at module scope. -/// -/// Each trampoline looks its implementation up by the integer `ctx`, -/// decodes the arguments, and calls the method. Any exception is reported -/// through `{prefix}_error_set(out_err, -4, message)` and a default value is -/// returned, so nothing unwinds through the C frame. `ctypes` acquires the -/// GIL on entry, so the producer may call from any thread. -pub(crate) fn render_callback_interface( - out: &mut String, - g: &Gen<'_>, - cb: &CallbackInterfaceBinding, -) { - let name = &cb.name; - let mut w = CodeWriter::four_space(); - - w.blank().blank(); - w.line(format!("class {name}(abc.ABC):")); - w.indent(); - let base_doc = format!( - "Consumer-implemented callback interface. Subclass it, implement every\n\ - abstract method, and pass an instance wherever the API takes a `{name}`;\n\ - the producer may call the methods from any thread until it releases the\n\ - instance. An exception raised by a method is reported to the producer,\n\ - which fails the call in progress with {}.FOREIGN_ERROR_CODE (-4).", - g.root_error - ); - let class_doc = match &cb.doc { - Some(d) if !d.trim().is_empty() => format!("{}\n\n{base_doc}", d.trim()), - _ => base_doc, - }; - let mut doc = String::new(); - emit_docstring(&mut doc, &Some(class_doc), &w.indent_str()); - w.raw(doc); - for m in &cb.methods { - let mut sig: Vec = vec!["self".into()]; - sig.extend( - m.params - .iter() - .map(|p| format!("{}: {}", py_name(&p.name), py_type_hint(&p.ty))), - ); - let ret = m - .ret - .as_ref() - .map(py_type_hint) - .unwrap_or_else(|| "None".to_string()); - w.blank(); - w.line("@abc.abstractmethod"); - w.line(format!( - "def {}({}) -> {ret}:", - py_member_name(&m.name), - sig.join(", ") - )); - w.indent(); - let mut mdoc = String::new(); - emit_fn_docstring(&mut mdoc, &m.doc, &m.params, &w.indent_str(), None); - if mdoc.is_empty() { - w.line("..."); - } else { - w.raw(mdoc); - } - w.dedent(); - } - w.dedent(); - - // One function-pointer type per vtable entry, then the vtable layout: - // methods in declaration order, then `free`. - w.blank().blank(); - for m in &cb.methods { - w.line(format!( - "_{name}_{}_t = {}", - m.name, - cfunctype(&m.abi_ret, &m.abi_params) - )); - } - w.line(format!( - "_{name}_free_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p)" - )); - w.blank().blank(); - w.line(format!("class _{name}Vtable(ctypes.Structure):")); - w.scope(|w| { - w.line(format!("\"\"\"The C vtable `{}`.\"\"\"", cb.vtable_tag)); - w.blank(); - w.line("_fields_ = ["); - w.scope(|w| { - for m in &cb.methods { - w.line(format!("(\"{0}\", _{name}_{0}_t),", m.name)); - } - w.line(format!("(\"free\", _{name}_free_t),")); - }); - w.line("]"); - }); - - for m in &cb.methods { - render_trampoline(&mut w, cb, m); - } - - // The one static vtable. Each field keeps its function object alive, and - // the vtable itself lives at module scope for the process lifetime. - w.blank().blank(); - w.line(format!("_{name}_vtable = _{name}Vtable(")); - w.scope(|w| { - for m in &cb.methods { - w.line(format!("_{name}_{0}_t(_{name}_{0}),", m.name)); - } - w.line(format!("_{name}_free_t(_callback_free),")); - }); - w.line(")"); - out.push_str(&w.finish()); -} - -/// Render the trampoline for one callback method: a `def` whose parameters -/// are the vtable entry's C slots (`ctx`, the parameter slots, `out_err`). -fn render_trampoline(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, m: &CallbackMethodBinding) { - let slots: Vec = m.abi_params.iter().map(|p| py_slot_name(&p.name)).collect(); - let args: Vec = m.params.iter().map(py_trampoline_arg).collect(); - let call = format!( - "_callback_get(ctx).{}({})", - py_member_name(&m.name), - args.join(", ") - ); - let ret = py_trampoline_return(m.ret.as_ref()); - let ret_hint = match ret { - Some((coerce, _)) => coerce, - None => "None", - }; - w.blank().blank(); - w.line(format!( - "def _{}_{}({}) -> {ret_hint}:", - cb.name, - m.name, - slots.join(", ") - )); - w.scope(|w| { - w.line("try:"); - w.scope(|w| match ret { - Some((coerce, _)) => { - w.line(format!("return {coerce}({call})")); - } - None => { - w.line(&call); - } - }); - w.line("except BaseException as exc:"); - w.scope(|w| { - w.line("_callback_fail(out_err, exc)"); - if let Some((_, default)) = ret { - w.line(format!("return {default}")); - } - }); - }); -} diff --git a/crates/weaveffi-cli/src/targets/python/codec.rs b/crates/weaveffi-cli/src/targets/python/codec.rs new file mode 100644 index 00000000..2d9ff661 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/codec.rs @@ -0,0 +1,354 @@ +//! Value-buffer codecs: one writer and one reader function per record, rich +//! enum, and distinct composite type (`[Entry]`, `{string:i64}`, `Store?`), +//! plus the read expressions and write statements that call them. +//! +//! A primitive, C-style enum, or object token is read or written inline (one +//! runtime call); everything else delegates to its named function, so no +//! loop is ever inlined at a call site. Every dispatch goes through +//! [`Ty::wire`], and primitives dispatch on [`Prim::snake`] (`write_i32`, +//! `read_string`). + +use std::collections::HashSet; + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{EnumBinding, Model, StructBinding}; +use weaveffi_model::ty::{Prim, Ty, WireType}; + +use crate::targets::python::types::{py_field, py_type_hint, py_variant}; + +/// The identifier fragment naming `ty` in its codec functions: a +/// primitive's spelling (`i64`), a user type's name (`Entry`), or the +/// composite's shape (`opt_i64`, `list_string`, `map_string_Store`). +fn mangle(ty: &Ty) -> String { + match ty { + Ty::Optional(inner) => format!("opt_{}", mangle(inner)), + Ty::List(inner) => format!("list_{}", mangle(inner)), + Ty::Map(k, v) => format!("map_{}_{}", mangle(k), mangle(v)), + Ty::Prim(p) => p.snake().to_string(), + _ => ty + .user_name() + .expect("only optionals, lists, maps, primitives, and user types are mangled") + .to_string(), + } +} + +/// `_write_{stem}`: the writer for a record, rich enum, or composite. +fn write_fn(ty: &Ty) -> String { + format!("_write_{}", mangle(ty)) +} + +/// `_read_{stem}`: the reader for a record, rich enum, or composite. +fn read_fn(ty: &Ty) -> String { + format!("_read_{}", mangle(ty)) +} + +/// The `struct` format character and byte width of a fixed-width numeric +/// primitive, for the packed list fast path. +fn numeric_format(p: Prim) -> Option<(char, usize)> { + Some(match p { + Prim::I8 => ('b', 1), + Prim::U8 => ('B', 1), + Prim::I16 => ('h', 2), + Prim::U16 => ('H', 2), + Prim::I32 => ('i', 4), + Prim::U32 => ('I', 4), + Prim::I64 => ('q', 8), + Prim::U64 => ('Q', 8), + Prim::F32 => ('f', 4), + Prim::F64 => ('d', 8), + Prim::Bool | Prim::String | Prim::Bytes => return None, + }) +} + +/// The expression reading one `ty` value from the reader `_r`. +pub(crate) fn read_expr(ty: &Ty) -> String { + match ty.wire() { + WireType::Prim(p) => format!("_r.read_{}()", p.snake()), + // An object token carries one strong reference: adopt it into a new + // wrapper. A decode that fails later in the buffer drops the + // wrapper, and its finalizer releases the adopted reference. + WireType::Object(name) => format!("{name}._adopt(_r.read_object())"), + WireType::Enum(name) => format!("_r.read_enum({name})"), + WireType::User(_) | WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{}(_r)", read_fn(ty)) + } + } +} + +/// The statement writing `expr` (one `ty` value) into the writer `_w`. +pub(crate) fn write_stmt(expr: &str, ty: &Ty) -> String { + match ty.wire() { + WireType::Prim(p) => format!("_w.write_{}({expr})", p.snake()), + // The writer clones the wrapper's reference when the buffer is + // finished, so the reader adopts a reference of its own. + WireType::Object(name) => format!("_w.write_object({expr}, {name})"), + // `IntEnum` members are ints, so the discriminant packs directly. + WireType::Enum(_) => format!("_w.write_i32({expr})"), + WireType::User(_) | WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{}(_w, {expr})", write_fn(ty)) + } + } +} + +/// The expression decoding the value buffer `data` (a `bytes` expression) +/// into one value of the buffered type `ty`, rejecting trailing data. +pub(crate) fn decode_expr(data: &str, ty: &Ty) -> String { + format!("_decode({data}, {})", read_fn(ty)) +} + +/// The statements encoding `expr` (one value of the buffered type `ty`) into +/// a new writer named `writer`. +pub(crate) fn encode_stmts(writer: &str, expr: &str, ty: &Ty) -> [String; 2] { + [ + format!("{writer} = _Writer()"), + format!("{}({writer}, {expr})", write_fn(ty)), + ] +} + +/// Append a record's writer and reader. +pub(crate) fn render_record_codecs(w: &mut CodeWriter, s: &StructBinding) { + let name = &s.name; + let ty = Ty::Record(name.clone()); + w.blank().blank(); + w.line(format!( + "def {}(_w: _Writer, value: {name}) -> None:", + write_fn(&ty) + )); + w.scope(|w| { + if s.fields.is_empty() { + w.line("pass"); + } + for f in &s.fields { + w.line(write_stmt(&format!("value.{}", py_field(&f.name)), &f.ty)); + } + }); + + w.blank().blank(); + w.line(format!("def {}(_r: _Reader) -> {name}:", read_fn(&ty))); + w.scope(|w| { + if s.fields.is_empty() { + w.line(format!("return {name}()")); + return; + } + // Keyword arguments evaluate left to right, matching the wire order + // of the record's fields. + w.line(format!("return {name}(")); + w.scope(|w| { + for f in &s.fields { + w.line(format!("{}={},", py_field(&f.name), read_expr(&f.ty))); + } + }); + w.line(")"); + }); +} + +/// Append a rich enum's writer and reader. The wire shape is an `i32` tag +/// followed by the active variant's fields. +pub(crate) fn render_rich_enum_codecs(w: &mut CodeWriter, e: &EnumBinding) { + let name = &e.name; + let ty = Ty::RichEnum(name.clone()); + w.blank().blank(); + w.line(format!( + "def {}(_w: _Writer, value: {name}) -> None:", + write_fn(&ty) + )); + w.scope(|w| { + for v in &e.variants { + let class = format!("{name}{}", py_variant(&v.name)); + w.line(format!("if isinstance(value, {class}):")); + w.scope(|w| { + w.line(format!("_w.write_i32({})", v.value)); + for f in &v.fields { + w.line(write_stmt(&format!("value.{}", py_field(&f.name)), &f.ty)); + } + w.line("return"); + }); + } + w.line(format!( + "raise TypeError(f\"expected a {name} variant, got {{type(value).__name__}}\")" + )); + }); + + w.blank().blank(); + w.line(format!("def {}(_r: _Reader) -> {name}:", read_fn(&ty))); + w.scope(|w| { + w.line("tag = _r.read_i32()"); + for v in &e.variants { + let class = format!("{name}{}", py_variant(&v.name)); + w.line(format!("if tag == {}:", v.value)); + w.scope(|w| { + if v.fields.is_empty() { + w.line(format!("return {class}()")); + return; + } + w.line(format!("return {class}(")); + w.scope(|w| { + for f in &v.fields { + w.line(format!("{}={},", py_field(&f.name), read_expr(&f.ty))); + } + }); + w.line(")"); + }); + } + w.line(format!("raise _malformed(f\"unknown {name} tag {{tag}}\")")); + }); +} + +/// Record every optional, list, and map shape inside `ty`, innermost first. +fn collect(ty: &Ty, out: &mut Vec, seen: &mut HashSet) { + match ty { + Ty::Optional(inner) | Ty::List(inner) => collect(inner, out, seen), + Ty::Map(k, v) => { + collect(k, out, seen); + collect(v, out, seen); + } + _ => return, + } + if seen.insert(ty.clone()) { + out.push(ty.clone()); + } +} + +/// Every distinct composite type a value buffer of the API carries, in +/// first-use order: inside records, rich-enum variants, and error payloads, +/// and at every buffered call boundary (parameters, returns, iterator +/// elements, and callback parameters and returns). +fn composites(model: &Model) -> Vec { + let mut out = Vec::new(); + let mut seen = HashSet::new(); + for m in &model.modules { + let fields = m + .structs + .iter() + .flat_map(|s| &s.fields) + .chain( + m.enums + .iter() + .flat_map(|e| &e.variants) + .flat_map(|v| &v.fields), + ) + .chain( + m.errors + .iter() + .flat_map(|e| &e.codes) + .flat_map(|c| &c.fields), + ); + for f in fields { + collect(&f.ty, &mut out, &mut seen); + } + } + let mut boundary = |ty: &Ty| { + let ty = ty.iterator_elem().unwrap_or(ty); + if ty.is_buffered() { + collect(ty, &mut out, &mut seen); + } + }; + for m in &model.modules { + for f in m.callables() { + f.params.iter().for_each(|p| boundary(&p.ty)); + f.ret.iter().for_each(&mut boundary); + } + for cb in &m.callback_interfaces { + for meth in &cb.methods { + meth.params.iter().for_each(|p| boundary(&p.ty)); + meth.ret.iter().for_each(&mut boundary); + } + } + } + out +} + +/// Append the writer and reader of every composite type the API uses, after +/// every module (a function body resolves the codecs it calls when it runs, +/// so definition order doesn't matter). +pub(crate) fn render_composite_codecs(w: &mut CodeWriter, model: &Model) { + let all = composites(model); + if all.is_empty() { + return; + } + w.blank().blank(); + w.line("# === Value-buffer codecs of composite types ==="); + for ty in &all { + render_composite(w, ty); + } +} + +/// Append one composite type's writer and reader. +fn render_composite(w: &mut CodeWriter, ty: &Ty) { + let hint = py_type_hint(ty); + w.blank().blank(); + w.line(format!( + "def {}(_w: _Writer, value: {hint}) -> None:", + write_fn(ty) + )); + w.scope(|w| match ty { + Ty::Optional(inner) => { + w.line("_w.write_flag(value is not None)"); + w.line("if value is not None:"); + w.scope(|w| { + w.line(write_stmt("value", inner)); + }); + } + Ty::List(inner) => match inner.wire() { + WireType::Prim(p) if numeric_format(p).is_some() => { + let (code, _) = numeric_format(p).expect("numeric"); + w.line(format!("_w.write_numbers(\"{code}\", value)")); + } + _ => { + w.line("_w.write_count(len(value))"); + w.line("for item in value:"); + w.scope(|w| { + w.line(write_stmt("item", inner)); + }); + } + }, + Ty::Map(k, v) => { + w.line("_w.write_count(len(value))"); + w.line("for key, item in value.items():"); + w.scope(|w| { + w.line(write_stmt("key", k)); + w.line(write_stmt("item", v)); + }); + } + _ => unreachable!("only optionals, lists, and maps are composites"), + }); + + w.blank().blank(); + w.line(format!("def {}(_r: _Reader) -> {hint}:", read_fn(ty))); + w.scope(|w| match ty { + Ty::Optional(inner) => { + w.line(format!( + "return {} if _r.read_flag() else None", + read_expr(inner) + )); + } + Ty::List(inner) => match inner.wire() { + WireType::Prim(p) if numeric_format(p).is_some() => { + let (code, size) = numeric_format(p).expect("numeric"); + w.line(format!("return _r.read_numbers(\"{code}\", {size})")); + } + _ => { + w.line(format!( + "return [{} for _ in range(_r.read_count())]", + read_expr(inner) + )); + } + }, + Ty::Map(k, v) => { + // A key is read before its value, and a repeated key (which + // would silently drop an entry) is malformed. + w.line("n = _r.read_count()"); + w.line(format!( + "value = {{{}: {} for _ in range(n)}}", + read_expr(k), + read_expr(v) + )); + w.line("if len(value) != n:"); + w.scope(|w| { + w.line("raise _malformed(\"repeated map key\")"); + }); + w.line("return value"); + } + _ => unreachable!("only optionals, lists, and maps are composites"), + }); +} diff --git a/crates/weaveffi-cli/src/targets/python/docs.rs b/crates/weaveffi-cli/src/targets/python/docs.rs new file mode 100644 index 00000000..faa31c9d --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/docs.rs @@ -0,0 +1,121 @@ +//! Doc emission for the generated Python: `#` comments, PEP 257 docstrings, +//! and the NumPy-style callable docstring with `Parameters` and `Raises` +//! sections. Everything writes into the one [`CodeWriter`] at its current +//! depth. + +use crate::codegen::CodeWriter; +use weaveffi_model::model::ParamBinding; + +use crate::targets::python::types::{py_name, py_type_hint}; + +/// `doc` trimmed, or `None` when it is absent or blank. +fn trimmed(doc: Option<&str>) -> Option<&str> { + doc.map(str::trim).filter(|d| !d.is_empty()) +} + +/// Escape docstring text: backslashes, and quotes that would close the +/// literal early. +fn escape(text: &str) -> String { + let mut out = text.replace('\\', "\\\\").replace("\"\"\"", "\\\"\\\"\\\""); + if out.ends_with('"') { + out.pop(); + out.push_str("\\\""); + } + out +} + +/// Join a summary with an optional deprecation note into one docstring +/// body. +pub(crate) fn with_deprecation(doc: Option<&str>, deprecated: Option<&str>) -> Option { + let note = deprecated.map(|msg| format!("Deprecated: {}", msg.trim())); + match (trimmed(doc), note) { + (Some(doc), Some(note)) => Some(format!("{doc}\n\n{note}")), + (Some(doc), None) => Some(doc.to_string()), + (None, note) => note, + } +} + +/// Emit `doc` as a PEP 257 docstring, the first statement of a class or +/// function body: one line when it fits on one, else the summary on the +/// opening line and the closing quotes on their own. No-op when `doc` is +/// absent or blank. +pub(crate) fn docstring(w: &mut CodeWriter, doc: Option<&str>) { + let Some(doc) = trimmed(doc) else { + return; + }; + let doc = escape(doc); + let mut lines = doc.lines(); + let first = lines.next().unwrap_or_default(); + if doc.contains('\n') { + w.line(format!("\"\"\"{first}")); + for line in lines { + w.line(line.trim_end()); + } + w.line("\"\"\""); + } else { + w.line(format!("\"\"\"{first}\"\"\"")); + } +} + +/// Emit `doc` as `#` comment lines (enum members and dataclass fields, which +/// have no docstrings of their own). +pub(crate) fn comment(w: &mut CodeWriter, doc: Option<&str>) { + if let Some(doc) = trimmed(doc) { + for line in doc.lines() { + w.line(format!("# {line}").trim_end()); + } + } +} + +/// Emit a callable's NumPy-style docstring: the summary, a `Parameters` +/// section for the parameters that carry docs, and a `Raises` section from +/// `raises`, a `(domain, when)` pair naming the module's error domain (for a +/// callable that declares errors) and when it's raised. Falls back to a +/// plain docstring when there are no sections. +pub(crate) fn fn_docstring( + w: &mut CodeWriter, + doc: Option<&str>, + params: &[ParamBinding], + raises: Option<(&str, &str)>, +) { + let documented: Vec<(&ParamBinding, &str)> = params + .iter() + .filter_map(|p| trimmed(p.doc.as_deref()).map(|d| (p, d))) + .collect(); + if documented.is_empty() && raises.is_none() { + docstring(w, doc); + return; + } + let mut sections: Vec = Vec::new(); + if let Some(doc) = trimmed(doc) { + sections.push(doc.to_string()); + } + if !documented.is_empty() { + let mut s = String::from("Parameters\n----------"); + for (p, d) in documented { + s.push_str(&format!("\n{} : {}", py_name(&p.name), py_type_hint(&p.ty))); + for line in d.lines() { + s.push('\n'); + if !line.trim().is_empty() { + s.push_str(" "); + s.push_str(line); + } + } + } + sections.push(s); + } + if let Some((domain, when)) = raises { + sections.push(format!("Raises\n------\n{domain}\n {when}")); + } + let body = sections.join("\n\n"); + if trimmed(doc).is_some() { + docstring(w, Some(&body)); + } else { + // No summary: the sections start on the line after the quotes. + w.line("\"\"\""); + for line in escape(&body).lines() { + w.line(line.trim_end()); + } + w.line("\"\"\""); + } +} diff --git a/crates/weaveffi-cli/src/targets/python/entities.rs b/crates/weaveffi-cli/src/targets/python/entities.rs new file mode 100644 index 00000000..e098832f --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/entities.rs @@ -0,0 +1,433 @@ +//! Entity rendering: error domains, plain and rich enums, record +//! dataclasses, and interface wrapper classes. + +use crate::codegen::CodeWriter; +use heck::ToSnakeCase; +use weaveffi_model::model::{ + CallShape, EnumBinding, ErrorBinding, FieldBinding, FnBinding, InterfaceBinding, Model, + ModuleBinding, StructBinding, +}; + +use crate::targets::python::calls::{render_bindings, render_callable, FnScope}; +use crate::targets::python::codec::{ + read_expr, render_record_codecs, render_rich_enum_codecs, write_stmt, +}; +use crate::targets::python::docs::{comment, docstring, with_deprecation}; +use crate::targets::python::types::{ + py_binding_name, py_error_field, py_field, py_field_hint, py_str_literal, py_variant, +}; +use crate::targets::python::Gen; + +// ── Errors ── + +/// The first of `candidates` that no declaration of the API (a type, or an +/// error code's class) already names. +fn free_name(model: &Model, candidates: [String; 3]) -> String { + let taken = |name: &str| { + model.modules.iter().any(|m| { + m.enums.iter().any(|e| e.name == name) + || m.structs.iter().any(|s| s.name == name) + || m.interfaces.iter().any(|i| i.name == name) + || m.callback_interfaces.iter().any(|c| c.name == name) + || m.errors.as_ref().is_some_and(|e| { + e.type_name == name + || e.codes.iter().any(|c| py_code_class_name(&c.name) == name) + }) + }) + }; + let fallback = format!("{}Root", candidates[2]); + candidates + .into_iter() + .find(|n| !taken(n)) + .unwrap_or(fallback) +} + +/// The root exception class every declared error derives from: `Error` (so +/// consumers write `except kvstore.Error`), unless the API declares a type +/// or error code with that name, in which case `{PascalName}Error`, then +/// `{PascalName}BaseError`. +pub(crate) fn root_error_name(model: &Model, pascal_name: &str) -> String { + free_name( + model, + [ + "Error".to_string(), + format!("{pascal_name}Error"), + format!("{pascal_name}BaseError"), + ], + ) +} + +/// The unchecked `RuntimeError` subclass a failed call that declares no +/// errors raises: `InternalError`, unless the API declares that name. +pub(crate) fn trap_error_name(model: &Model, pascal_name: &str) -> String { + free_name( + model, + [ + "InternalError".to_string(), + format!("{pascal_name}InternalError"), + format!("{pascal_name}RuntimeError"), + ], + ) +} + +/// `_{stem}_from`: builds the domain exception matching a code, message, +/// and payload. +pub(crate) fn py_factory_name(eb: &ErrorBinding) -> String { + format!("_{}_from", eb.type_name.to_snake_case()) +} + +/// The factory an out-err slot of `f` is raised through: the module +/// domain's typed factory when `f` declares errors, the unchecked trap +/// otherwise. +pub(crate) fn py_raise_factory(f: &FnBinding, error: Option<&ErrorBinding>) -> String { + match error { + Some(eb) if f.throws => py_factory_name(eb), + _ => "_trap_from".to_string(), + } +} + +/// The Python class name for one error code: plain PascalCase with no forced +/// suffix (`KeyNotFound`, not `KeyNotFoundError`). Each class is also +/// attached to its domain class (`KvError.KeyNotFound`). +pub(crate) fn py_code_class_name(name: &str) -> String { + weaveffi_model::errors::pascal(name) +} + +/// Render one module's declared error domain: a base exception named after +/// the domain (subclassing the root exception), one subclass per code +/// carrying its stable `CODE`, its payload fields as constructor arguments +/// and attributes, and their value-buffer encoding (a callback method that +/// declares errors raises them back to the producer), then the factory that +/// builds the exception for a code, message, and payload. Each code class +/// is also attached to the domain class, so consumers can catch +/// `KvError.KeyNotFound`. +pub(crate) fn render_error( + w: &mut CodeWriter, + g: &Gen<'_>, + module: &ModuleBinding, + eb: &ErrorBinding, +) { + let domain = &eb.type_name; + let root = &g.root_error; + + w.blank().blank(); + w.line(format!("class {domain}({root}):")); + w.scope(|w| { + docstring( + w, + Some(&format!( + "Base exception for the `{}` module's error domain.", + module.dot_path + )), + ); + // The per-code classes, attached below once they exist. + w.blank(); + for c in &eb.codes { + let class = py_code_class_name(&c.name); + w.line(format!("{class}: Type[{class}]")); + } + }); + + for c in &eb.codes { + let class = py_code_class_name(&c.name); + w.blank().blank(); + w.line(format!("class {class}({domain}):")); + w.scope(|w| { + docstring(w, Some(c.doc.as_deref().unwrap_or(&c.message))); + w.blank(); + w.line(format!("CODE = {}", c.value)); + if !c.fields.is_empty() { + w.blank(); + render_dataclass_fields(w, &c.fields, py_error_field); + } + let mut params: Vec = c + .fields + .iter() + .map(|f| { + format!( + "{}: {}", + py_error_field(&f.name), + py_field_hint(&c.fields, &f.ty) + ) + }) + .collect(); + params.push(format!("message: str = \"{}\"", py_str_literal(&c.message))); + w.blank(); + w.line(format!( + "def __init__(self, {}) -> None:", + params.join(", ") + )); + w.scope(|w| { + w.line(format!("super().__init__({}, message)", c.value)); + for f in &c.fields { + let attr = py_error_field(&f.name); + w.line(format!("self.{attr} = {attr}")); + } + }); + if !c.fields.is_empty() { + w.blank(); + w.line("def _payload(self) -> bytes:"); + w.scope(|w| { + w.line("_w = _Writer()"); + for f in &c.fields { + w.line(write_stmt( + &format!("self.{}", py_error_field(&f.name)), + &f.ty, + )); + } + w.line("return _w.finish()"); + }); + } + }); + } + + // Scoped aliases: `except KvError.KeyNotFound` names the code through its + // domain. + w.blank().blank(); + for c in &eb.codes { + let class = py_code_class_name(&c.name); + w.line(format!("{domain}.{class} = {class}")); + } + + w.blank().blank(); + w.line(format!( + "def {}(code: int, message: str, payload: bytes = b\"\") -> {root}:", + py_factory_name(eb) + )); + w.scope(|w| { + docstring( + w, + Some(&format!( + "The {domain} subclass for `code` with its payload fields, or the\n\ + plain {root} for a runtime code (a panic, a marshalling failure)." + )), + ); + for c in &eb.codes { + let class = py_code_class_name(&c.name); + w.line(format!("if code == {}:", c.value)); + w.scope(|w| { + if c.fields.is_empty() { + w.line(format!("return {class}(message)")); + return; + } + // Keyword arguments evaluate left to right, matching the + // wire order of the code's fields. + w.line(format!("return _decode(payload, lambda _r: {class}(")); + w.scope(|w| { + for f in &c.fields { + w.line(format!("{}={},", py_error_field(&f.name), read_expr(&f.ty))); + } + w.line("message=message,"); + }); + w.line("))"); + }); + } + w.line(format!("return {root}(code, message)")); + }); +} + +// ── Enums ── + +/// Render one enum: a plain `IntEnum` for a C-style enum, or the dataclass +/// sum-type hierarchy for a rich enum. +pub(crate) fn render_enum(w: &mut CodeWriter, e: &EnumBinding) { + if e.is_rich() { + render_rich_enum(w, e); + return; + } + w.blank().blank(); + w.line(format!("class {}(IntEnum):", e.name)); + w.scope(|w| { + let doc = with_deprecation(e.doc.as_deref(), e.deprecated.as_deref()); + if doc.is_some() { + docstring(w, doc.as_deref()); + w.blank(); + } + for v in &e.variants { + comment(w, v.doc.as_deref()); + w.line(format!("{} = {}", py_variant(&v.name), v.value)); + } + }); +} + +/// Render a rich (algebraic) enum as an idiomatic Python sum type: a base +/// class holding the nested `Tag` discriminant enum and a `tag` property, +/// one module-level `@dataclass` subclass per variant carrying its fields, +/// scoped aliases (`Shape.Circle` is `ShapeCircle`), and the buffer codec. +/// Consumers construct variants directly and discriminate with `isinstance` +/// (or the `tag` property). +fn render_rich_enum(w: &mut CodeWriter, e: &EnumBinding) { + let name = &e.name; + w.blank().blank(); + w.line(format!("class {name}:")); + w.scope(|w| { + let doc = with_deprecation(e.doc.as_deref(), e.deprecated.as_deref()); + if doc.is_some() { + docstring(w, doc.as_deref()); + w.blank(); + } + w.line("class Tag(IntEnum):"); + w.scope(|w| { + for v in &e.variants { + comment(w, v.doc.as_deref()); + w.line(format!("{} = {}", py_variant(&v.name), v.value)); + } + }); + w.blank(); + // The variant classes, attached below once they exist. + for v in &e.variants { + w.line(format!("{0}: Type[{name}{0}]", py_variant(&v.name))); + } + w.line(format!("TAG: {name}.Tag")); + w.blank(); + w.line("@property"); + w.line(format!("def tag(self) -> {name}.Tag:")); + w.scope(|w| { + docstring(w, Some("The discriminant of this value's active variant.")); + w.line("return type(self).TAG"); + }); + }); + + for v in &e.variants { + let class = format!("{name}{}", py_variant(&v.name)); + w.blank().blank(); + w.line("@dataclass"); + w.line(format!("class {class}({name}):")); + w.scope(|w| { + if v.doc.is_some() { + docstring(w, v.doc.as_deref()); + w.blank(); + } + w.line(format!("TAG = {name}.Tag.{}", py_variant(&v.name))); + if !v.fields.is_empty() { + w.blank(); + render_dataclass_fields(w, &v.fields, py_field); + } + }); + } + + // Scoped aliases (`Shape.Circle`), assigned once every variant class + // exists. + w.blank().blank(); + for v in &e.variants { + w.line(format!("{name}.{0} = {name}{0}", py_variant(&v.name))); + } + + render_rich_enum_codecs(w, e); +} + +// ── Records ── + +/// Render a record as a plain `@dataclass` value class plus its buffer +/// codec. Records have no C symbols: construction, equality, and repr all +/// come from the dataclass, and instances cross the ABI serialized in value +/// buffers. +pub(crate) fn render_struct(w: &mut CodeWriter, s: &StructBinding) { + w.blank().blank(); + w.line("@dataclass"); + w.line(format!("class {}:", s.name)); + w.scope(|w| { + let doc = with_deprecation(s.doc.as_deref(), s.deprecated.as_deref()); + docstring(w, doc.as_deref()); + if s.fields.is_empty() { + if doc.is_none() { + w.line("pass"); + } + return; + } + if doc.is_some() { + w.blank(); + } + render_dataclass_fields(w, &s.fields, py_field); + }); + render_record_codecs(w, s); +} + +/// Emit one annotated attribute line (`name: hint`) per field, with field +/// docs as leading comments; `spell` names each field. +fn render_dataclass_fields(w: &mut CodeWriter, fields: &[FieldBinding], spell: fn(&str) -> String) { + for f in fields { + comment(w, f.doc.as_deref()); + w.line(format!( + "{}: {}", + spell(&f.name), + py_field_hint(fields, &f.ty) + )); + } +} + +// ── Interfaces ── + +/// Render one interface as a reference-counted object wrapper class +/// (subclassing the runtime's `_Object`), preceded by the module-level +/// bindings of its members, clone, and destroy. The class holds one strong +/// reference and releases it exactly once, from `close()`, the +/// context-manager exit, or the `__del__` backstop; calls lend the pointer so +/// a concurrent `close()` defers the release until they return. A +/// constructor named `new` becomes `__init__`; every other constructor +/// becomes a `@classmethod` factory; methods lend `self` as the leading C +/// argument; statics are `@staticmethod`s. +pub(crate) fn render_interface( + w: &mut CodeWriter, + g: &Gen<'_>, + module: &ModuleBinding, + i: &InterfaceBinding, +) { + let error = g.model.error_domain(module); + let name = &i.name; + // Consecutive sync member bindings are packed into one block. + let mut packed = false; + for m in i.constructors.iter().chain(&i.methods).chain(&i.statics) { + render_bindings(w, g, m, error, &format!("{name}.{}", m.name), packed); + packed = matches!(m.shape, CallShape::Sync(_)); + } + let clone = py_binding_name(&i.clone_symbol, g.prefix); + let destroy = py_binding_name(&i.destroy_symbol, g.prefix); + if !packed { + w.blank().blank(); + } + w.line(format!( + "{clone} = _bind(\"{}\", ctypes.c_void_p, ctypes.c_void_p)", + i.clone_symbol + )); + w.line(format!( + "{destroy} = _bind(\"{}\", None, ctypes.c_void_p)", + i.destroy_symbol + )); + + w.blank().blank(); + w.line(format!("class {name}(_Object):")); + w.scope(|w| { + let doc = with_deprecation(i.doc.as_deref(), i.deprecated.as_deref()); + if doc.is_some() { + docstring(w, doc.as_deref()); + w.blank(); + } + w.line(format!("_destroy = staticmethod({destroy})")); + w.line(format!("_clone = staticmethod({clone})")); + + match i.constructors.iter().find(|c| c.name == "new") { + Some(c) => render_callable(w, g, module, c, FnScope::Init), + None => { + // No canonical constructor: instances only come from + // factories and producer returns. + w.blank(); + w.line("def __init__(self) -> None:"); + w.scope(|w| { + w.line(format!( + "raise TypeError(\"{name} cannot be instantiated directly\")" + )); + }); + } + } + for c in i.constructors.iter().filter(|c| c.name != "new") { + render_callable(w, g, module, c, FnScope::Factory); + } + for m in &i.methods { + render_callable(w, g, module, m, FnScope::Method); + } + for s in &i.statics { + render_callable(w, g, module, s, FnScope::Static); + } + }); +} diff --git a/crates/weaveffi-cli/src/targets/python/mod.rs b/crates/weaveffi-cli/src/targets/python/mod.rs new file mode 100644 index 00000000..588998b7 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/mod.rs @@ -0,0 +1,326 @@ +//! Python (`ctypes`) binding generator. +//! +//! Emits a pip-installable package of `ctypes` bindings over the C ABI +//! (revision 4): distribution `{name}`, import package `{prefix}`, and one +//! fully annotated implementation module `{prefix}/{prefix}.py` with a +//! `py.typed` marker. Every C prototype is bound once at import time, after +//! the load-time ABI revision and contract-table check. Records and rich +//! enums are dataclasses crossing the boundary as value buffers; interfaces +//! are reference-counted wrapper classes with `close()` and a `__del__` +//! backstop; callback interfaces are abstract base classes backed by one +//! static vtable of `ctypes` trampolines; async functions are coroutines +//! whose cancellation cancels the native call; `iter` returns are lazy +//! Python iterators. + +mod callbacks; +mod calls; +mod codec; +mod docs; +mod entities; +mod package; +mod runtime; +#[cfg(test)] +mod tests; +mod types; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::codegen::CodeWriter; +use crate::package::{Artifact, ArtifactKind, PackageContext, PackagedFile, WheelMeta}; +use crate::platform::{glibc_requirement, Os}; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; +use camino::Utf8Path; +use serde::{Deserialize, Serialize}; +use weaveffi_model::model::{Model, ModuleBinding}; +use weaveffi_model::pkg::Identity; + +use crate::targets::python::callbacks::render_callback_interface; +use crate::targets::python::calls::{render_bindings, render_callable, FnScope}; +use crate::targets::python::codec::render_composite_codecs; +use crate::targets::python::entities::{ + py_code_class_name, render_enum, render_error, render_interface, render_struct, + root_error_name, trap_error_name, +}; +use crate::targets::python::package::{ + render_py_typed, render_pyproject_toml, render_readme, render_wheel_readme, +}; +use crate::targets::python::runtime::render_runtime; +use crate::targets::python::types::{py_member_name, py_variant}; + +/// Per-target configuration for [`PythonGenerator`]. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct PythonConfig { + /// The distribution name published to PyPI (default: the package + /// identity's `name`). + pub name: Option, + /// The import package (default: the identity's C prefix, which is + /// always a valid Python identifier). Also names the package directory + /// and the implementation module inside it. + pub import_name: Option, + /// The `requires-python` specifier of the project and its wheels + /// (default `>=3.9`). + pub requires_python: String, +} + +impl Default for PythonConfig { + fn default() -> Self { + Self { + name: None, + import_name: None, + requires_python: ">=3.9".into(), + } + } +} + +/// The names one generation run uses, resolved from the identity and the +/// configuration. +struct Names { + /// The distribution name. + dist: String, + /// The import package and implementation module name. + import: String, +} + +impl Names { + fn new(id: &Identity, config: &PythonConfig) -> Self { + Self { + dist: config.name.clone().unwrap_or_else(|| id.name.clone()), + import: config + .import_name + .clone() + .unwrap_or_else(|| id.prefix.clone()), + } + } +} + +/// The metadata of the wheel for one platform `tag`. +fn wheel_meta(id: &Identity, names: &Names, config: &PythonConfig, tag: String) -> WheelMeta { + let mut urls = Vec::new(); + if let Some(homepage) = &id.homepage { + urls.push(("Homepage".to_string(), homepage.clone())); + } + if let Some(repository) = &id.repository { + urls.push(("Repository".to_string(), repository.clone())); + } + WheelMeta { + name: names.dist.clone(), + version: id.version.clone(), + platform: tag, + summary: id.description_or_default(), + requires_python: Some(config.requires_python.clone()), + license: id.license.clone(), + author: (!id.authors.is_empty()).then(|| id.authors.join(", ")), + urls, + readme: Some(render_wheel_readme(id, &names.dist, &names.import)), + } +} + +/// The shared context every rendering function receives. +pub(crate) struct Gen<'a> { + /// The model being rendered. + pub model: &'a Model, + /// The C symbol prefix (stripped from binding names). + pub prefix: &'a str, + /// The root exception class name (see [`root_error_name`]). + pub root_error: String, + /// The unchecked trap exception class name (see [`trap_error_name`]). + pub trap_error: String, +} + +impl<'a> Gen<'a> { + fn new(model: &'a Model) -> Self { + let pascal = model.identity.pascal_name(); + Self { + model, + prefix: model.prefix(), + root_error: root_error_name(model, &pascal), + trap_error: trap_error_name(model, &pascal), + } + } +} + +/// Python backend: emits a pip-installable package of fully annotated +/// `ctypes` bindings over the C ABI exposed by the underlying cdylib. +pub struct PythonGenerator; + +impl PythonGenerator { + /// Render the implementation module: the runtime, every module's members + /// in canonical order (error domain, enums, records, callback + /// interfaces, interfaces, functions), the composite codecs, and + /// `__all__`. + fn render_py_source(&self, model: &Model, file_name: &str) -> String { + let g = Gen::new(model); + let mut w = CodeWriter::four_space(); + w.raw(render_prelude(CommentStyle::Hash)); + render_runtime(&mut w, &g); + for m in &model.modules { + w.blank().blank(); + w.line(format!("# === Module: {} ===", m.dot_path)); + emit_module(&mut w, &g, m); + } + render_composite_codecs(&mut w, model); + render_all(&mut w, &g); + w.blank(); + w.raw(render_trailer(CommentStyle::Hash, file_name)); + w.finish() + } + + /// The `__init__.py` re-exporting the implementation module's public + /// names (its `__all__`). + fn render_init(&self, import: &str) -> String { + let hash = CommentStyle::Hash; + format!( + "{}from .{import} import * # noqa: F401,F403\n\n{}", + render_prelude(hash), + render_trailer(hash, "__init__.py"), + ) + } + + /// The import package's files, keyed by file name. + fn package_sources(&self, model: &Model, names: &Names) -> [(String, String); 3] { + let py = format!("{}.py", names.import); + [ + ("__init__.py".into(), self.render_init(&names.import)), + (py.clone(), self.render_py_source(model, &py)), + ("py.typed".into(), render_py_typed()), + ] + } +} + +/// Emit every member of `m` in canonical order. Callback interfaces precede +/// interfaces because interface members take them as parameters. +fn emit_module(w: &mut CodeWriter, g: &Gen<'_>, m: &ModuleBinding) { + let error = g.model.error_domain(m); + if let Some(e) = m.errors.as_ref() { + render_error(w, g, m, e); + } + for e in &m.enums { + render_enum(w, e); + } + for s in &m.structs { + render_struct(w, s); + } + for cb in &m.callback_interfaces { + render_callback_interface(w, g, cb, error.map(|e| e.type_name.as_str())); + } + for i in &m.interfaces { + render_interface(w, g, m, i); + } + for f in &m.functions { + render_bindings(w, g, f, error, &f.name, false); + render_callable(w, g, m, f, FnScope::Free); + } +} + +/// Emit `__all__`: the public names, in declaration order, so a star import +/// (the package's `__init__.py`) re-exports the API and nothing else. +fn render_all(w: &mut CodeWriter, g: &Gen<'_>) { + let mut names = vec![g.root_error.clone(), g.trap_error.clone()]; + for m in &g.model.modules { + if let Some(e) = &m.errors { + names.push(e.type_name.clone()); + names.extend(e.codes.iter().map(|c| py_code_class_name(&c.name))); + } + for e in &m.enums { + names.push(e.name.clone()); + if e.is_rich() { + names.extend( + e.variants + .iter() + .map(|v| format!("{}{}", e.name, py_variant(&v.name))), + ); + } + } + names.extend(m.structs.iter().map(|s| s.name.clone())); + names.extend(m.callback_interfaces.iter().map(|c| c.name.clone())); + names.extend(m.interfaces.iter().map(|i| i.name.clone())); + names.extend(m.functions.iter().map(|f| py_member_name(&f.name))); + } + w.blank().blank(); + w.line("__all__ = ["); + w.scope(|w| { + for n in &names { + w.line(format!("\"{n}\",")); + } + }); + w.line("]"); +} + +impl LanguageBackend for PythonGenerator { + type Config = PythonConfig; + + fn name(&self) -> &'static str { + "python" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let names = Names::new(&model.identity, config); + let dir = out_dir.join("python"); + let pkg_dir = dir.join(&names.import); + let mut files: Vec = self + .package_sources(model, &names) + .into_iter() + .map(|(name, contents)| OutputFile::new(pkg_dir.join(name), contents)) + .collect(); + files.push(OutputFile::new( + dir.join("pyproject.toml"), + render_pyproject_toml( + &model.identity, + &names.dist, + &names.import, + &config.requires_python, + ), + )); + files.push(OutputFile::new( + dir.join("README.md"), + render_readme(&model.identity, &names.dist, &names.import), + )); + files + } + + /// One `py3-none-{platform}` wheel per desktop platform, with the + /// library inside the import package where the loader looks first. + /// Platforms without a wheel tag (Android, iOS, `wasm32`) are skipped. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let id = &model.identity; + let names = Names::new(id, config); + let sources = self.package_sources(model, &names); + let mut artifacts = Vec::new(); + for nb in &ctx.binaries.binaries { + let platform = nb.platform; + let glibc = if platform.os() == Os::Linux { + std::fs::read(nb.library.as_std_path()) + .map(|bytes| glibc_requirement(&bytes)) + .unwrap_or((2, 17)) + } else { + (2, 17) + }; + let Some(tag) = platform.python_platform_tag(ctx.macos_deployment_target, glibc) else { + continue; + }; + let mut files: Vec = sources + .iter() + .map(|(name, contents)| { + PackagedFile::text(format!("{}/{name}", names.import), contents.clone()) + }) + .collect(); + // Bundled under the file name the loader looks for. + files.push(PackagedFile::copy( + format!("{}/{}", names.import, platform.lib_filename(&id.library)), + nb.library.clone(), + )); + let meta = wheel_meta(id, &names, config, tag); + artifacts.push(Artifact { + path: format!("python/{}", meta.file_name()).into(), + kind: ArtifactKind::Wheel(meta), + files, + }); + } + Some(artifacts) + } +} diff --git a/crates/weaveffi-gen/src/targets/python/package.rs b/crates/weaveffi-cli/src/targets/python/package.rs similarity index 52% rename from crates/weaveffi-gen/src/targets/python/package.rs rename to crates/weaveffi-cli/src/targets/python/package.rs index e3a26ec3..e68246f4 100644 --- a/crates/weaveffi-gen/src/targets/python/package.rs +++ b/crates/weaveffi-cli/src/targets/python/package.rs @@ -1,6 +1,6 @@ //! Packaging manifests: `pyproject.toml` and `README.md` for the generated -//! tree, plus the `setup.py` and README of each per-platform wheel tree the -//! `package` command assembles. +//! tree, plus the long description of each wheel the `package` command +//! writes. use crate::utils::{render_prelude, render_trailer, CommentStyle}; use weaveffi_model::pkg::Identity; @@ -11,18 +11,19 @@ fn toml_str(s: &str) -> String { } /// Render the `pyproject.toml`: a PEP 621 project built with setuptools, -/// shipping the stub, the `py.typed` marker, and any bundled library as +/// shipping the `py.typed` marker and any bundled library as /// package data. pub(crate) fn render_pyproject_toml( id: &Identity, dist: &str, import: &str, - input_basename: &str, + requires_python: &str, ) -> String { - let prelude = render_prelude(CommentStyle::Hash, input_basename); + let prelude = render_prelude(CommentStyle::Hash); let trailer = render_trailer(CommentStyle::Hash, "pyproject.toml"); let version = &id.version; let description = toml_str(&id.description_or_default()); + let requires_python = toml_str(requires_python); let mut extra = String::new(); if let Some(license) = &id.license { extra.push_str(&format!( @@ -58,13 +59,13 @@ build-backend = "setuptools.build_meta" name = "{dist}" version = "{version}" description = "{description}" -requires-python = ">=3.9" +requires-python = "{requires_python}" {extra}{urls} [tool.setuptools] packages = ["{import}"] [tool.setuptools.package-data] -"{import}" = ["py.typed", "*.pyi", "*.so", "*.dylib", "*.dll"] +"{import}" = ["py.typed", "*.so", "*.dylib", "*.dll"] {trailer}"#, ) @@ -73,18 +74,13 @@ packages = ["{import}"] /// Render the PEP 561 `py.typed` marker. Type checkers only look for the /// file's presence (its content is ignored unless it says `partial`), so it /// carries the standard prelude like every other generated file. -pub(crate) fn render_py_typed(input_basename: &str) -> String { - render_prelude(CommentStyle::Hash, input_basename) +pub(crate) fn render_py_typed() -> String { + render_prelude(CommentStyle::Hash) } /// Render the `README.md` for the generated tree. -pub(crate) fn render_readme( - id: &Identity, - dist: &str, - import: &str, - input_basename: &str, -) -> String { - let prelude = render_prelude(CommentStyle::Xml, input_basename); +pub(crate) fn render_readme(id: &Identity, dist: &str, import: &str) -> String { + let prelude = render_prelude(CommentStyle::Xml); let trailer = render_trailer(CommentStyle::Xml, "README.md"); let (darwin, linux, windows) = id.library_files(); let env = id.library_env_var(); @@ -121,75 +117,13 @@ import {import} ) } -/// Render a `setup.py` for a packaged wheel: it ships the bundled library as -/// package data and forces a non-pure (platform-tagged) wheel. -pub(crate) fn render_packaged_setup_py( - dist: &str, - version: &str, - import: &str, - input_basename: &str, -) -> String { - let prelude = render_prelude(CommentStyle::Hash, input_basename); - let trailer = render_trailer(CommentStyle::Hash, "setup.py"); - format!( - r#"{prelude}from setuptools import setup -from setuptools.dist import Distribution - - -class _BinaryDistribution(Distribution): - # Force a non-pure, platform-tagged wheel: the package bundles a native - # shared library, so it is not portable across platforms. - def has_ext_modules(self): - return True - - -setup( - name="{dist}", - version="{version}", - packages=["{import}"], - package_data={{"{import}": ["py.typed", "*.pyi", "*.so", "*.dylib", "*.dll"]}}, - include_package_data=True, - distclass=_BinaryDistribution, -) - -{trailer}"#, - ) -} - -/// README for a packaged per-platform Python wheel tree. `tag` is the -/// platform's wheel tag (`platform.python_platform_tag()`), which the caller -/// has already established exists. -pub(crate) fn render_packaged_readme( - dist: &str, - import: &str, - platform: crate::platform::Platform, - tag: &str, - input_basename: &str, -) -> String { - let prelude = render_prelude(CommentStyle::Xml, input_basename); - let trailer = render_trailer(CommentStyle::Xml, "README.md"); +/// The long description of a packaged wheel: the bundled library loads +/// automatically, so it only says how to import the package. +pub(crate) fn render_wheel_readme(id: &Identity, dist: &str, import: &str) -> String { format!( - r#"{prelude}# {dist} (Python, {plat}) - -Python bindings with the native library bundled for `{plat}`. The library -loads automatically; no external setup is required. - -## Build the wheel - -```bash -python -m build --wheel -``` - -Tag the resulting wheel for this platform with `{tag}` (for example via -`wheel tags --platform-tag {tag} dist/*.whl`) before publishing. - -## Usage - -```python -import {import} -``` - -{trailer}"#, - plat = platform.id(), + "# {dist}\n\n{}\n\nPython bindings for the `{library}` native library, which this \ + wheel bundles.\n\n```python\nimport {import}\n```\n", + id.description_or_default(), + library = id.library, ) } diff --git a/crates/weaveffi-cli/src/targets/python/runtime.rs b/crates/weaveffi-cli/src/targets/python/runtime.rs new file mode 100644 index 00000000..6dab8e1c --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/runtime.rs @@ -0,0 +1,92 @@ +//! The fixed Python runtime every generated module starts with. +//! +//! The runtime is ordinary Python kept in `runtime/*.py` and spliced in with +//! `{{PLACEHOLDER}}` substitution: `core.py` (the root and trap exceptions, +//! library loading, the load-time ABI and contract check, the owned-result +//! helpers, the handle base classes, and the value-buffer codec), `aio.py` +//! (the async completion registry), `cancel.py` (cancel tokens), and +//! `callbacks.py` (the callback-interface handle table, return hand-off, +//! and failure reporting). The last three are emitted only when the API +//! uses the feature. The contract table's entries are written into the +//! writer where `core.py` says `{{CONTRACTS}}`. + +use crate::codegen::CodeWriter; +use weaveffi_model::contract; +use weaveffi_model::model::{contract_symbol, ABI_VERSION}; + +use crate::targets::python::types::py_str_literal; +use crate::targets::python::Gen; + +const CORE: &str = include_str!("runtime/core.py"); +const AIO: &str = include_str!("runtime/aio.py"); +const CANCEL: &str = include_str!("runtime/cancel.py"); +const CALLBACKS: &str = include_str!("runtime/callbacks.py"); + +/// Write the body of the `_CONTRACTS` table: per top-level module, its +/// contract symbol and the `(id, hash, path)` entries these bindings rely +/// on, sorted by id. +fn render_contracts(w: &mut CodeWriter, g: &Gen<'_>) { + w.scope(|w| { + for root in g.model.roots() { + w.line(format!("\"{}\": [", contract_symbol(g.prefix, &root.name))); + w.scope(|w| { + for e in contract::entries(g.model, root) { + w.line(format!( + "({:#018x}, {:#018x}, \"{}\"),", + e.id, + e.hash, + py_str_literal(&e.path) + )); + } + }); + w.line("],"); + } + }); +} + +/// Substitute every scalar `{{PLACEHOLDER}}` a runtime template uses. +fn fill(template: &str, g: &Gen<'_>) -> String { + let id = &g.model.identity; + let (darwin, linux, windows) = id.library_files(); + template + .replace("{{NAME}}", &id.name) + .replace("{{ERROR}}", &g.root_error) + .replace("{{TRAP}}", &g.trap_error) + .replace("{{PREFIX}}", g.prefix) + .replace("{{LIBRARY_ENV}}", &id.library_env_var()) + .replace("{{LIB_DARWIN}}", &darwin) + .replace("{{LIB_LINUX}}", &linux) + .replace("{{LIB_WINDOWS}}", &windows) + .replace("{{ABI_VERSION}}", &ABI_VERSION.to_string()) +} + +/// Append the runtime: the core, then the feature parts the API needs. +pub(crate) fn render_runtime(w: &mut CodeWriter, g: &Gen<'_>) { + let callbacks = g.model.has_callback_interfaces(); + let is_async = g.model.has_async(); + let cancellable = g.model.callables().any(|(_, f)| f.cancellable); + let mut imports = String::new(); + if callbacks { + imports.push_str("import abc\n"); + } + if is_async { + imports.push_str("import asyncio\n"); + } + let core = fill(CORE, g).replace("{{IMPORTS}}", &imports); + let (head, tail) = core + .split_once("{{CONTRACTS}}\n") + .expect("core.py has a {{CONTRACTS}} line"); + w.raw(head); + render_contracts(w, g); + w.raw(tail); + for (wanted, template) in [ + (is_async, AIO), + (cancellable, CANCEL), + (callbacks, CALLBACKS), + ] { + if wanted { + w.blank().blank(); + w.raw(fill(template, g)); + } + } +} diff --git a/crates/weaveffi-gen/src/targets/python/runtime/aio.py b/crates/weaveffi-cli/src/targets/python/runtime/aio.py similarity index 90% rename from crates/weaveffi-gen/src/targets/python/runtime/aio.py rename to crates/weaveffi-cli/src/targets/python/runtime/aio.py index 8482835d..f83cf533 100644 --- a/crates/weaveffi-gen/src/targets/python/runtime/aio.py +++ b/crates/weaveffi-cli/src/targets/python/runtime/aio.py @@ -4,14 +4,14 @@ # The trampolines live at module scope, so nothing a pending call needs can # be garbage collected before the producer fires the completion, even when # the awaiting coroutine is gone. -_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, "asyncio.Future[Any]"]] = {} +_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, asyncio.Future[Any]]] = {} _pending_lock = threading.Lock() _pending_next = 0 _error_free = _bind("{{PREFIX}}_error_free", None, ctypes.POINTER(_ErrorStruct)) -def _async_begin() -> Tuple[int, "asyncio.Future[Any]"]: +def _async_begin() -> Tuple[int, asyncio.Future[Any]]: """Register a new pending call on the running event loop.""" global _pending_next loop = asyncio.get_running_loop() @@ -56,7 +56,7 @@ def _async_settle(key: Optional[int], exc: Optional[BaseException], value: Any) pass -def _async_resolve(future: "asyncio.Future[Any]", exc: Optional[BaseException], value: Any) -> None: +def _async_resolve(future: asyncio.Future[Any], exc: Optional[BaseException], value: Any) -> None: if future.done(): return if isinstance(exc, asyncio.CancelledError): diff --git a/crates/weaveffi-cli/src/targets/python/runtime/callbacks.py b/crates/weaveffi-cli/src/targets/python/runtime/callbacks.py new file mode 100644 index 00000000..0e59088e --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/runtime/callbacks.py @@ -0,0 +1,99 @@ +# Callback-interface implementations the producer holds, keyed by the integer +# it receives as `ctx`. An entry is added when an implementation is passed to +# a call and removed when the producer calls the vtable's `free(ctx)`, so the +# implementation lives exactly as long as the producer may call it and the +# producer never holds a raw reference to a Python object. ctypes acquires +# the GIL on entry to a trampoline, so the producer may call (and free) from +# any thread. +_callbacks: Dict[int, Any] = {} +_callbacks_lock = threading.Lock() +_callbacks_next = 0 + +_alloc = _bind("{{PREFIX}}_alloc", ctypes.c_void_p, ctypes.c_size_t) +_error_set = _bind( + "{{PREFIX}}_error_set", None, ctypes.POINTER(_ErrorStruct), ctypes.c_int32, ctypes.c_char_p +) +_error_set_payload = _bind( + "{{PREFIX}}_error_set_payload", + None, + ctypes.POINTER(_ErrorStruct), + ctypes.c_char_p, + ctypes.c_size_t, +) + + +def _callback_register(impl: Any, cls: type) -> int: + """Register an implementation for one call; returns its `ctx` key.""" + global _callbacks_next + if not isinstance(impl, cls): + raise TypeError(f"expected {cls.__name__}, got {type(impl).__name__}") + with _callbacks_lock: + _callbacks_next += 1 + ctx = _callbacks_next + _callbacks[ctx] = impl + return ctx + + +def _callback_register_opt(impl: Any, cls: type) -> Optional[int]: + """`_callback_register` for an optional callback: None passes none.""" + return None if impl is None else _callback_register(impl, cls) + + +def _callback_get(ctx: Optional[int]) -> Any: + return _callbacks[ctx or 0] + + +def _callback_free(ctx: Optional[int]) -> None: + # The producer's last reference is gone; it never passes `ctx` again. + with _callbacks_lock: + _callbacks.pop(ctx or 0, None) + + +# Every vtable's `free` entry: one function object for the process lifetime. +_CallbackFree = ctypes.CFUNCTYPE(None, ctypes.c_void_p) +_callback_free_fn = _CallbackFree(_callback_free) + + +def _callback_return_bytes(out_ptr: Any, out_len: Any, data: bytes) -> None: + """Hand a string, bytes, or value-buffer return to the producer: a run + allocated with {{PREFIX}}_alloc, which the producer adopts and frees.""" + n = len(data) + ptr = _alloc(n) if n else None + if n: + if not ptr: + raise MemoryError(f"{{PREFIX}}_alloc({n}) failed") + ctypes.memmove(ptr, data, n) + out_ptr[0] = ptr + out_len[0] = n + + +def _callback_return_object(obj: Any, cls: Type[_Object]) -> int: + """Hand an object return to the producer: one fresh strong reference it + adopts.""" + if not isinstance(obj, cls): + raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") + return obj._clone_ref() + + +def _callback_return_object_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: + """`_callback_return_object` for an optional object: None returns null.""" + return None if obj is None else _callback_return_object(obj, cls) + + +def _callback_fail(out_err: Any, exc: BaseException, domain: Optional[Type[{{ERROR}}]] = None) -> None: + """Report an implementation's exception to the producer. A method that + declares errors passes its module's error domain, and an exception of + that domain carrying a declared code travels as that code with its + fields; every other exception reaches the producer as the foreign error + code (-4). Nothing unwinds through C.""" + if domain is not None and isinstance(exc, domain) and exc.code > 0: + _error_set(out_err, exc.code, exc.message.encode("utf-8", "replace")) + payload = exc._payload() + if payload: + _error_set_payload(out_err, payload, len(payload)) + return + if isinstance(exc, ({{ERROR}}, {{TRAP}})): + message = exc.message + else: + message = str(exc) or type(exc).__name__ + _error_set(out_err, {{ERROR}}.FOREIGN_ERROR_CODE, message.encode("utf-8", "replace")) diff --git a/crates/weaveffi-gen/src/targets/python/runtime/cancel.py b/crates/weaveffi-cli/src/targets/python/runtime/cancel.py similarity index 91% rename from crates/weaveffi-gen/src/targets/python/runtime/cancel.py rename to crates/weaveffi-cli/src/targets/python/runtime/cancel.py index b87c81f3..15d9bda6 100644 --- a/crates/weaveffi-gen/src/targets/python/runtime/cancel.py +++ b/crates/weaveffi-cli/src/targets/python/runtime/cancel.py @@ -7,7 +7,7 @@ _cancel_token_destroy = _bind("{{PREFIX}}_cancel_token_destroy", None, ctypes.c_void_p) -async def _async_wait_cancellable(future: "asyncio.Future[Any]", token: int) -> Any: +async def _async_wait_cancellable(future: asyncio.Future[Any], token: int) -> Any: """Await a cancellable call. Cancelling the awaiting task cancels the native call and waits for its (cancelled) completion before raising, so nothing outlives the call.""" diff --git a/crates/weaveffi-gen/src/targets/python/runtime/core.py b/crates/weaveffi-cli/src/targets/python/runtime/core.py similarity index 74% rename from crates/weaveffi-gen/src/targets/python/runtime/core.py rename to crates/weaveffi-cli/src/targets/python/runtime/core.py index 4495ec0a..5216033d 100644 --- a/crates/weaveffi-gen/src/targets/python/runtime/core.py +++ b/crates/weaveffi-cli/src/targets/python/runtime/core.py @@ -1,4 +1,6 @@ """ctypes bindings for the {{NAME}} native library.""" +from __future__ import annotations + {{IMPORTS}}import builtins import ctypes import os @@ -8,18 +10,19 @@ import warnings from dataclasses import dataclass from enum import IntEnum -from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type +from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type, TypeVar class {{ERROR}}(Exception): - """An error reported by the native library. - - `code` is positive for one of the API's declared error codes (raised as - the declaring module's typed subclasses) and negative for a runtime trap: - a generic failure (-1), a producer panic (-2), a marshalling failure - (-3), or an exception raised by a callback-interface implementation - (-4). A cancelled async call (-5) raises `asyncio.CancelledError` - instead. + """An error a fallible call reports. + + `code` is positive for one of the API's declared error codes, raised as + the declaring module's typed subclasses, and negative for a runtime + failure of a call that declares errors: a generic failure (-1), a + producer panic (-2), a marshalling failure (-3), or an exception raised + by a callback-interface implementation (-4). A cancelled async call (-5) + raises `asyncio.CancelledError` instead. A call that declares no errors + never raises this; its failures are bugs and raise `{{TRAP}}`. """ GENERIC_ERROR_CODE = -1 @@ -33,6 +36,24 @@ def __init__(self, code: int, message: str) -> None: self.code = code self.message = message + def _payload(self) -> bytes: + """The code's fields as a value buffer (a code with fields overrides + this).""" + return b"" + + +class {{TRAP}}(RuntimeError): + """A failure of a call that declares no errors: a producer panic (-2), a + marshalling failure (-3), a callback-interface failure the producer let + through (-4), or another runtime failure (-1). These are bugs, so they + are unchecked; `code` and `message` say what went wrong. + """ + + def __init__(self, code: int, message: str) -> None: + super().__init__(f"({code}) {message}") + self.code = code + self.message = message + class _ErrorStruct(ctypes.Structure): """The C `{{PREFIX}}_error` out-parameter every call reports through.""" @@ -82,18 +103,25 @@ def _bind(name: str, restype: Any, *argtypes: Any) -> Any: return fn -# The C ABI revision these bindings were generated against, and the contract -# checksum of every top-level module: (module, checksum symbol, value). +class _ContractEntry(ctypes.Structure): + """The C `{{PREFIX}}_contract_entry`: one declaration's id and hash.""" + + _fields_ = [("id", ctypes.c_uint64), ("hash", ctypes.c_uint64)] + + +# The C ABI revision these bindings were generated against and, per +# top-level module's contract symbol, the declarations they rely on: +# (id, hash, dotted path). _ABI_VERSION = {{ABI_VERSION}} -_CHECKSUMS: List[Tuple[str, str, int]] = [ -{{CHECKSUMS}} -] +_CONTRACTS: Dict[str, List[Tuple[int, int, str]]] = { +{{CONTRACTS}} +} def _check_contract() -> None: - """Refuse to load a library built for a different ABI revision or a - different version of any module's API, before any other symbol is - bound.""" + """Refuse to load a library built for another ABI revision, or one that + lacks or changed a declaration these bindings use, before any other + symbol is bound.""" try: abi_version = _bind("{{PREFIX}}_abi_version", ctypes.c_uint32) except AttributeError: @@ -107,33 +135,40 @@ def _check_contract() -> None: f"C ABI mismatch: these bindings expect revision {_ABI_VERSION}, " f"but {_lib._name} reports revision {found}" ) - for module, symbol, expected in _CHECKSUMS: + for symbol, expected in _CONTRACTS.items(): try: - checksum = _bind(symbol, ctypes.c_uint64) - except AttributeError: - raise ImportError( - f"module {module!r}: {_lib._name} does not export {symbol}" - ) from None - found = checksum() - if found != expected: - raise ImportError( - f"module {module!r}: the native library's contract checksum " - f"{found:#018x} does not match these bindings ({expected:#018x}); " - "regenerate the bindings from the library's current API" + contract = _bind( + symbol, ctypes.POINTER(_ContractEntry), ctypes.POINTER(ctypes.c_size_t) ) + except AttributeError: + raise ImportError(f"{_lib._name} does not export {symbol}") from None + length = ctypes.c_size_t() + table = contract(ctypes.byref(length)) + hashes = {table[i].id: table[i].hash for i in range(length.value)} if table else {} + for entry_id, entry_hash, path in expected: + actual = hashes.get(entry_id) + if actual is None: + raise ImportError(f"{path} is missing from the library ({_lib._name})") + if actual != entry_hash: + raise ImportError( + f"{path} changed since these bindings were generated ({_lib._name})" + ) _check_contract() _error_clear = _bind("{{PREFIX}}_error_clear", None, ctypes.POINTER(_ErrorStruct)) _free_bytes = _bind("{{PREFIX}}_free_bytes", None, ctypes.c_void_p, ctypes.c_size_t) +_debug_live_fn = _bind("{{PREFIX}}_debug_live", ctypes.c_uint64, ctypes.c_int32) def _debug_live(kind: int) -> int: - """The producer's live-allocation counter `kind` (0 objects, 1 callbacks, - 2 iterators, 3 cancel tokens, 4 returned allocations). Always 0 unless - the producer was built with its leak counters enabled.""" - return _bind("{{PREFIX}}_debug_live", ctypes.c_uint64, ctypes.c_int32)(kind) + """The producer's live-resource counter `kind` (0 objects, 1 callbacks, + 2 iterators, 3 cancel tokens, 4 byte runs; -1 is 1 when the producer + counts at all). Always 0 unless the producer was built with its leak + counters enabled.""" + live: int = _debug_live_fn(kind) + return live def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: @@ -146,17 +181,9 @@ def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: return code, message, payload -def _error_from(code: int, message: str, payload: bytes = b"") -> {{ERROR}}: - """The exception for a code no declared error domain claims.""" - return {{ERROR}}(code, message) - - -def _check_error(err: _ErrorStruct) -> None: - """Raise for a non-zero out-err slot of a call that declares no errors: - only runtime traps (a panic, a marshalling failure, a failed callback) - land here.""" - if err.code: - raise _error_from(*_read_error(err)) +def _trap_from(code: int, message: str, payload: bytes = b"") -> {{TRAP}}: + """The exception for a failed call that declares no errors.""" + return {{TRAP}}(code, message) def _take_bytes(ptr: Optional[int], length: int) -> bytes: @@ -180,6 +207,9 @@ def _peek_bytes(ptr: Optional[int], length: int) -> bytes: return ctypes.string_at(ptr, length) if ptr and length else b"" +_H = TypeVar("_H", bound="_Handle") + + class _Handle: """Owns one native reference and releases it exactly once. @@ -193,12 +223,13 @@ class _Handle: def _init_handle(self, ptr: int) -> None: self._ptr: Optional[int] = ptr + self._key = ptr self._calls = 0 self._closing = False self._lock = threading.Lock() @classmethod - def _adopt(cls, ptr: int) -> Any: + def _adopt(cls: Type[_H], ptr: int) -> _H: """Wrap one strong reference the producer handed over.""" obj = cls.__new__(cls) obj._init_handle(ptr) @@ -208,7 +239,7 @@ def _acquire(self) -> int: with self._lock: ptr = self._ptr if ptr is None or self._closing: - raise {{ERROR}}(-1, f"{type(self).__name__} used after close()") + raise ValueError(f"{type(self).__name__} is closed") self._calls += 1 return ptr @@ -222,8 +253,8 @@ def _release(self) -> None: type(self)._destroy(ptr) def close(self) -> None: - """Release this wrapper's native reference. Idempotent; the wrapper - is unusable afterwards.""" + """Release the native reference. Idempotent; the wrapper is unusable + afterwards.""" lock = getattr(self, "_lock", None) if lock is None: return @@ -237,7 +268,7 @@ def close(self) -> None: if ptr is not None: type(self)._destroy(ptr) - def __enter__(self) -> Any: + def __enter__(self: _H) -> _H: return self def __exit__(self, *exc: object) -> None: @@ -252,27 +283,36 @@ def __del__(self) -> None: class _Object(_Handle): - """Base of every interface wrapper: a reference-counted native object.""" + """Base of every interface wrapper: a reference-counted native object. + Two wrappers are equal when they hold the same native object.""" # The native clone function, set per subclass. _clone: Callable[[int], int] def _clone_ref(self) -> int: """A new strong reference to the same object, for a receiver that - adopts it (an object token inside a value buffer).""" + adopts it (an object token, or a callback method's return).""" ptr = self._acquire() try: return type(self)._clone(ptr) finally: self._release() + def __eq__(self, other: object) -> bool: + if not isinstance(other, _Object): + return NotImplemented + return self is other or (self._ptr is not None and self._ptr == other._ptr) + + def __hash__(self) -> int: + return hash(self._key) + class _Iterator(_Handle): """Base of every lazy iterator over a producer stream: one native `next` call per step; the handle is released on exhaustion, `close()`, or garbage collection.""" - def __iter__(self) -> Any: + def __iter__(self: _H) -> _H: return self def _step(self) -> int: @@ -287,23 +327,23 @@ def _step(self) -> int: def _required(ptr: Optional[int]) -> int: """A non-null object pointer the producer returned.""" if not ptr: - raise {{ERROR}}(-1, "the producer returned a null object") + raise {{TRAP}}(-3, "the producer returned a null object") return ptr -def _lend(obj: Any, cls: "Type[_Object]") -> int: +def _lend(obj: Any, cls: Type[_Object]) -> int: """Lend `obj`'s native pointer to one call; pair with `obj._release()`.""" if not isinstance(obj, cls): raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") return obj._acquire() -def _lend_opt(obj: Any, cls: "Type[_Object]") -> Optional[int]: +def _lend_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: """`_lend` for an optional object: None lends a null pointer.""" return None if obj is None else _lend(obj, cls) -def _release_opt(obj: Any) -> None: +def _release_opt(obj: Optional[_Object]) -> None: if obj is not None: obj._release() @@ -319,9 +359,12 @@ def _release_opt(obj: Any) -> None: _F32 = struct.Struct(" {{ERROR}}: - return {{ERROR}}(-3, f"malformed value buffer: {what}") + +def _malformed(what: str) -> {{TRAP}}: + return {{TRAP}}(-3, f"malformed value buffer: {what}") class _Writer: @@ -404,7 +447,7 @@ def write_numbers(self, code: str, values: List[Any]) -> None: self._buf += _U32.pack(len(values)) self._buf += struct.pack(f"<{len(values)}{code}", *values) - def write_object(self, obj: Any, cls: "Type[_Object]") -> None: + def write_object(self, obj: Any, cls: Type[_Object]) -> None: """Reserve an object token: one strong reference to `obj` the reader adopts, cloned in `finish()`. A closed wrapper is rejected now.""" if not isinstance(obj, cls): @@ -417,7 +460,8 @@ def write_object(self, obj: Any, cls: "Type[_Object]") -> None: class _Reader: """Decodes values from the value-buffer wire format, rejecting truncated - buffers, invalid flag bytes, invalid UTF-8, and trailing data.""" + buffers, invalid flag bytes, invalid UTF-8, undeclared enum values, and + trailing data.""" __slots__ = ("_data", "_pos") @@ -490,6 +534,13 @@ def read_string(self) -> str: except UnicodeDecodeError: raise _malformed("string is not valid UTF-8") from None + def read_enum(self, cls: Type[_E]) -> _E: + value = self.read_i32() + try: + return cls(value) + except ValueError: + raise _malformed(f"undeclared {cls.__name__} value {value}") from None + def read_numbers(self, code: str, size: int) -> List[Any]: """A list of one fixed-width numeric primitive, unpacked in one call.""" n = self.read_count() @@ -508,7 +559,7 @@ def expect_end(self) -> None: raise _malformed("trailing bytes") -def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: +def _decode(data: bytes, read: Callable[[_Reader], _T]) -> _T: """Decode exactly one value from `data`.""" r = _Reader(data) value = read(r) diff --git a/crates/weaveffi-cli/src/targets/python/tests.rs b/crates/weaveffi-cli/src/targets/python/tests.rs new file mode 100644 index 00000000..50e25068 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/python/tests.rs @@ -0,0 +1,582 @@ +//! Unit tests: render a small API that exercises every ABI revision 4 shape +//! (prefixed runtime, contract table, pointer + length strings, +//! reference-counted objects, nullable objects, objects inside buffers, +//! composite codecs, typed errors with payloads, iterators, async and +//! cancellable functions, and callback interfaces with every return family, +//! a throwing method, and an optional parameter) and assert the generated +//! Python carries the pieces each contract clause requires. + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::package::{ArtifactKind, PackageContext}; +use crate::platform::{BinarySet, NativeBinary, Platform}; +use camino::Utf8Path; +use weaveffi_model::contract; +use weaveffi_model::model::Model; +use weaveffi_model::parse::parse_api_str; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; + +use crate::targets::python::{PythonConfig, PythonGenerator}; + +const KV: &str = r#" +version: "0.11.0" +modules: + - name: kv + errors: + name: KvError + codes: + - { name: NotFound, code: 1, message: "not found", fields: [{ name: key, type: string }] } + - { name: Busy, code: 2, message: "busy" } + structs: + - name: Entry + fields: + - { name: key, type: string } + - { name: store, type: Store } + - { name: mirrors, type: "[Store]" } + - { name: weights, type: "[i32]" } + - { name: labels, type: "{string:i64?}" } + interfaces: + - name: Store + doc: A key-value store. + constructors: + - { name: new, params: [{ name: path, type: string }] } + methods: + - { name: get, params: [{ name: key, type: string }], return: string, throws: true } + - { name: has, params: [{ name: key, type: string }], return: bool } + - { name: size, params: [], return: u32, deprecated: "use count" } + callback_interfaces: + - name: Watcher + doc: Observes store changes. + methods: + - name: on_change + params: + - { name: text, type: string } + - { name: entry, type: Entry } + - { name: store, type: Store } + - { name: should_stop, params: [{ name: count, type: i32 }], return: bool } + - { name: label, params: [], return: string, throws: true } + - { name: snapshot, params: [], return: Entry } + - { name: pick, params: [{ name: store, type: "Store?" }], return: "Store?" } + - { name: home, params: [], return: Store } + functions: + - name: pick + params: [{ name: store, type: "Store?" }] + return: "Store?" + - { name: watch, params: [{ name: watcher, type: Watcher }] } + - { name: maybe_watch, params: [{ name: watcher, type: "Watcher?" }] } + - { name: all_stores, params: [], return: "iter" } + - { name: fetch, params: [{ name: id, type: i64 }], return: Store, async: true } + - name: sleep + params: [{ name: ms, type: i64 }] + return: string + async: true + cancellable: true + throws: true +"#; + +fn model(yaml: &str) -> Model { + let api = parse_api_str(yaml, "yaml").expect("fixture parses"); + validate(&api, &Identity::named("kv-lib"), None) + .unwrap_or_else(|d| panic!("fixture must validate: {d:?}")) +} + +fn render_files(model: &Model, config: &PythonConfig) -> Vec { + PythonGenerator.files(model, Utf8Path::new("out"), config) +} + +/// Render `model` and return the implementation module. +fn render(model: &Model) -> String { + render_files(model, &PythonConfig::default()) + .into_iter() + .find(|f| f.path.as_str().ends_with("kv_lib.py")) + .expect("kv_lib.py") + .contents +} + +fn assert_has(hay: &str, needle: &str) { + assert!(hay.contains(needle), "missing `{needle}` in:\n{hay}"); +} + +#[test] +fn layout_follows_the_identity() { + let files = render_files(&model(KV), &PythonConfig::default()); + let paths: Vec<&str> = files.iter().map(|f| f.path.as_str()).collect(); + assert_eq!( + paths, + [ + "out/python/kv_lib/__init__.py", + "out/python/kv_lib/kv_lib.py", + "out/python/kv_lib/py.typed", + "out/python/pyproject.toml", + "out/python/README.md", + ] + ); + assert_has(&files[0].contents, "from .kv_lib import *"); + let pyproject = &files[3].contents; + assert_has(pyproject, "name = \"kv-lib\""); + assert_has(pyproject, "requires-python = \">=3.9\""); + assert_has(pyproject, "packages = [\"kv_lib\"]"); + assert_has(pyproject, "\"kv_lib\" = [\"py.typed\", \"*.so\""); + + let config = PythonConfig { + name: Some("kv-python".into()), + import_name: Some("kvpy".into()), + ..PythonConfig::default() + }; + let files = render_files(&model(KV), &config); + assert!(files.iter().any(|f| f.path == "out/python/kvpy/kvpy.py")); + let pyproject = files.iter().find(|f| f.path.ends_with("pyproject.toml")); + assert_has(&pyproject.unwrap().contents, "name = \"kv-python\""); +} + +#[test] +fn runtime_checks_the_revision_and_every_contract_entry() { + let m = model(KV); + let py = render(&m); + assert_has(&py, "from __future__ import annotations\n"); + assert_has(&py, "_ABI_VERSION = 4\n"); + assert_has(&py, " \"kv_lib_kv_contract\": [\n"); + let root = m.roots().next().unwrap(); + for e in contract::entries(&m, root) { + assert_has( + &py, + &format!("({:#018x}, {:#018x}, \"{}\"),", e.id, e.hash, e.path), + ); + } + assert_has( + &py, + "raise ImportError(f\"{path} is missing from the library", + ); + assert_has(&py, "f\"{path} changed since these bindings were generated"); + assert!(!py.contains("checksum"), "no trace of the checksum check"); + assert_has(&py, "_bind(\"kv_lib_abi_version\", ctypes.c_uint32)"); + assert_has(&py, "os.environ.get(\"KV_LIB_LIBRARY\")"); + assert_has(&py, "name = \"libkv_lib.dylib\""); + assert_has( + &py, + "_bind(\"kv_lib_free_bytes\", None, ctypes.c_void_p, ctypes.c_size_t)", + ); + assert_has( + &py, + "_bind(\"kv_lib_alloc\", ctypes.c_void_p, ctypes.c_size_t)", + ); + assert_has(&py, "\"kv_lib_error_set_payload\","); + assert!(!py.contains("dealloc")); + assert_has(&py, "class Error(Exception):"); + assert_has(&py, "class InternalError(RuntimeError):"); + // Nothing but the generated-file header names WeaveFFI. + for line in py.lines().filter(|l| !l.starts_with('#')) { + assert!( + !line.to_ascii_lowercase().contains("weaveffi"), + "branded line: {line}" + ); + } +} + +#[test] +fn failures_raise_the_domain_or_the_unchecked_trap() { + let py = render(&model(KV)); + // A throwing call raises its domain; any other failure traps. + assert_has( + &py, + " if _err.code:\n raise _kv_error_from(*_read_error(_err))\n \ + return _take_str(_ret, _out_len.value)", + ); + assert_has( + &py, + " if _err.code:\n raise _trap_from(*_read_error(_err))\n \ + return _ret", + ); + // Codes carry their payload fields as constructor arguments. + assert_has(&py, "class NotFound(KvError):"); + assert_has( + &py, + " def __init__(self, key: str, message: str = \"not found\") -> None:", + ); + assert_has( + &py, + " return _decode(payload, lambda _r: NotFound(\n \ + key=_r.read_string(),\n message=message,\n ))", + ); + assert_has(&py, " def _payload(self) -> bytes:"); + assert_has(&py, "KvError.NotFound = NotFound"); +} + +#[test] +fn prototypes_are_bound_once_with_c_bool_and_ptr_len_strings() { + let py = render(&model(KV)); + assert_has( + &py, + "_c_kv_Store_has = _bind(\"kv_lib_kv_Store_has\", ctypes.c_bool, ctypes.c_void_p, \ + ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(_ErrorStruct))", + ); + assert_eq!(py.matches("_c_kv_Store_get = _bind(").count(), 1); + assert!(!py.contains(".argtypes = ["), "no per-call binding"); + // A string crosses as UTF-8 bytes plus a length and comes back owned. + assert_has(&py, "_key_b = key.encode(\"utf-8\")"); + assert_has(&py, "_key_b, len(_key_b)"); + // A direct result is annotated for checkers. + assert_has(&py, "_ret: bool = _c_kv_Store_has("); + // Deprecation warns and is documented. + assert_has( + &py, + "warnings.warn(\"use count\", DeprecationWarning, stacklevel=2)", + ); + assert_has(&py, "Deprecated: use count"); +} + +#[test] +fn objects_are_lent_for_calls_and_adopted_from_returns() { + let py = render(&model(KV)); + assert_has(&py, "class Store(_Object):"); + assert_has(&py, " _destroy = staticmethod(_c_kv_Store_destroy)\n"); + assert_has(&py, " _clone = staticmethod(_c_kv_Store_clone)\n"); + assert_has(&py, "def __init__(self, path: str) -> None:"); + assert_has(&py, "self._init_handle(_required(_ret))"); + // `self` is lent for the call and released after it, even on error. + assert_has( + &py, + "_self_p = self._acquire()\n try:\n _ret = _c_kv_Store_get(_self_p,", + ); + assert_has(&py, " finally:\n self._release()\n"); + // Nullable objects map None to NULL both ways. + assert_has(&py, "_store_p = _lend_opt(store, Store)"); + assert_has(&py, "_release_opt(store)"); + assert_has(&py, "return Store._adopt(_ret) if _ret else None"); +} + +#[test] +fn composites_get_one_codec_function_each() { + let py = render(&model(KV)); + // Objects inside buffers are cloned when the buffer is finished and + // adopted on read. + assert_has(&py, "_w.write_object(value.store, Store)"); + assert_has(&py, "store=Store._adopt(_r.read_object()),"); + // Record fields delegate to one function per composite type. + assert_has(&py, " _write_list_Store(_w, value.mirrors)\n"); + assert_has(&py, " mirrors=_read_list_Store(_r),\n"); + assert_eq!(py.matches("def _write_list_Store(").count(), 1); + assert_eq!(py.matches("def _read_list_Store(").count(), 1); + assert_has( + &py, + "def _read_list_Store(_r: _Reader) -> List[Store]:\n \ + return [Store._adopt(_r.read_object()) for _ in range(_r.read_count())]", + ); + // Numeric lists take the packed fast path. + assert_has(&py, " _w.write_numbers(\"i\", value)\n"); + assert_has(&py, " return _r.read_numbers(\"i\", 4)\n"); + // Maps reject a repeated key; nested composites call each other. + assert_has( + &py, + "def _read_map_string_opt_i64(_r: _Reader) -> Dict[str, Optional[int]]:\n \ + n = _r.read_count()\n \ + value = {_r.read_string(): _read_opt_i64(_r) for _ in range(n)}", + ); + assert_has(&py, "raise _malformed(\"repeated map key\")"); + assert!(!py.contains("for _e0 in"), "no loop inlined at a call site"); +} + +#[test] +fn iterators_pull_lazily_through_prebound_functions() { + let py = render(&model(KV)); + assert_has(&py, "class _kv_AllStoresIterator(_Iterator):"); + assert_has( + &py, + " _destroy = staticmethod(_c_kv_AllStoresIterator_destroy)", + ); + assert_has( + &py, + "_more = _c_kv_AllStoresIterator_next(_p, ctypes.byref(_item), ctypes.byref(_err))", + ); + assert_has(&py, "return Store._adopt(_required(_item.value))"); + assert_has(&py, "return _kv_AllStoresIterator._adopt(_ret)"); + assert_has(&py, "def all_stores() -> Iterator[Store]:"); +} + +#[test] +fn async_calls_complete_through_static_trampolines() { + let py = render(&model(KV)); + assert_has(&py, "import asyncio\n"); + assert_has( + &py, + "_c_kv_fetch_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, \ + ctypes.POINTER(_ErrorStruct), ctypes.c_void_p)", + ); + assert_has( + &py, + "def _kv_fetch_complete(context: Optional[int], err: Any, result: Optional[int]) -> None:", + ); + assert_has( + &py, + "_async_settle(context, _async_error(err, _trap_from), None)", + ); + assert_has( + &py, + "_async_settle(context, None, Store._adopt(_required(result)))", + ); + assert_has( + &py, + "_kv_fetch_completion = _c_kv_fetch_callback(_kv_fetch_complete)", + ); + assert_has(&py, "_c_kv_fetch(id, _kv_fetch_completion, _call)"); + assert_has( + &py, + " _result: Store = await _future\n return _result\n", + ); + // A cancellable call owns a token for its duration. + assert_has(&py, "_token = _cancel_token_create()"); + assert_has(&py, "_c_kv_sleep(ms, _token, _kv_sleep_completion, _call)"); + assert_has( + &py, + "_result: str = await _async_wait_cancellable(_future, _token)", + ); + assert_has( + &py, + "_async_settle(context, _async_error(err, _kv_error_from), None)", + ); + assert_has( + &py, + "_async_settle(context, None, _take_str(result_ptr, result_len))", + ); + assert_has(&py, "async def sleep(ms: int) -> str:"); +} + +#[test] +fn callback_vtables_lead_with_the_header() { + let py = render(&model(KV)); + assert_has(&py, "import abc\n"); + assert_has(&py, "class Watcher(abc.ABC):"); + assert_has(&py, "class _WatcherVtable(ctypes.Structure):"); + assert_has(&py, "The C vtable `kv_lib_kv_Watcher_vtable`."); + assert_has( + &py, + " (\"size\", ctypes.c_uint32),\n (\"flags\", ctypes.c_uint32),\n \ + (\"free\", _CallbackFree),\n (\"on_change\", _Watcher_on_change_t),", + ); + assert_has( + &py, + "_Watcher_vtable = _WatcherVtable(\n ctypes.sizeof(_WatcherVtable),\n 0,\n \ + _callback_free_fn,\n _Watcher_on_change_t(_Watcher_on_change),", + ); + assert_eq!(py.matches("_Watcher_vtable = _WatcherVtable(").count(), 1); + assert_has( + &py, + "_Watcher_vtable_ptr = ctypes.addressof(_Watcher_vtable)", + ); + assert_has( + &py, + "_Watcher_on_change_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.c_void_p, \ + ctypes.c_size_t, ctypes.c_void_p, ctypes.c_size_t, ctypes.c_void_p, \ + ctypes.POINTER(_ErrorStruct))", + ); +} + +#[test] +fn callback_trampolines_adopt_objects_and_return_every_family() { + let py = render(&model(KV)); + // Object parameters are adopted before anything else can fail. + assert_has( + &py, + " _store = Store._adopt(_required(store))\n \ + _callback_get(ctx).on_change(_peek_bytes(text_ptr, text_len).decode(\"utf-8\"), \ + _decode(_peek_bytes(entry_ptr, entry_len), _read_Entry), _store)", + ); + // Direct returns by value, with a default after a failure. + assert_has(&py, "return bool(_callback_get(ctx).should_stop(count))"); + assert_has(&py, "_callback_fail(out_err, exc)\n return False"); + // Strings and buffers through the out slots, allocated for the producer. + assert_has( + &py, + "def _Watcher_label(ctx: Optional[int], out_ptr: Any, out_len: Any, out_err: Any) -> None:", + ); + assert_has( + &py, + "_callback_return_bytes(out_ptr, out_len, _ret.encode(\"utf-8\"))", + ); + assert_has( + &py, + " _w = _Writer()\n _write_Entry(_w, _ret)\n \ + _callback_return_bytes(out_ptr, out_len, _w.finish())", + ); + // Objects as a fresh reference; `I?` may be None. + assert_has(&py, "return _callback_return_object(_ret, Store)"); + assert_has(&py, "return _callback_return_object_opt(_ret, Store)"); + assert_has(&py, "_store = Store._adopt(store) if store else None"); + // A throwing method reports its domain's codes; others report -4. + assert_has(&py, "_callback_fail(out_err, exc, KvError)"); + assert_eq!( + py.matches("_callback_fail(out_err, exc, KvError)").count(), + 1 + ); +} + +#[test] +fn callback_parameters_register_and_none_passes_a_null_vtable() { + let py = render(&model(KV)); + assert_has(&py, "_watcher_ctx = _callback_register(watcher, Watcher)"); + assert_has( + &py, + "_c_kv_watch(_watcher_ctx, _Watcher_vtable_ptr, ctypes.byref(_err))", + ); + assert_has(&py, "def maybe_watch(watcher: Optional[Watcher]) -> None:"); + assert_has( + &py, + "_watcher_ctx = _callback_register_opt(watcher, Watcher)", + ); + assert_has( + &py, + "_watcher_ctx, _Watcher_vtable_ptr if watcher is not None else None,", + ); +} + +#[test] +fn public_names_are_exported_and_nothing_else() { + let py = render(&model(KV)); + let all = py + .split("__all__ = [\n") + .nth(1) + .and_then(|rest| rest.split("]\n").next()) + .expect("__all__"); + let names: Vec<&str> = all + .lines() + .map(|l| l.trim().trim_end_matches(',').trim_matches('"')) + .collect(); + assert_eq!( + names, + [ + "Error", + "InternalError", + "KvError", + "NotFound", + "Busy", + "Entry", + "Watcher", + "Store", + "pick", + "watch", + "maybe_watch", + "all_stores", + "fetch", + "sleep", + ] + ); +} + +#[test] +fn feature_runtime_is_emitted_only_when_used() { + let py = render(&model( + r#" +version: "0.11.0" +modules: + - name: math + functions: + - name: add + params: [{ name: a, type: i32 }, { name: b, type: i32 }] + return: i32 +"#, + )); + assert!(!py.contains("import abc")); + assert!(!py.contains("import asyncio")); + assert!(!py.contains("_callbacks")); + assert!(!py.contains("_cancel_token")); + assert!(!py.contains("Value-buffer codecs of composite types")); + assert_has(&py, "def add(a: int, b: int) -> int:"); +} + +#[test] +fn exception_names_avoid_user_type_names() { + let py = render(&model( + r#" +version: "0.11.0" +modules: + - name: kv + structs: + - name: Error + fields: [{ name: code, type: i32 }] + - name: InternalError + fields: [{ name: code, type: i32 }] +"#, + )); + assert_has(&py, "class KvLibError(Exception):"); + assert_has(&py, "class KvLibInternalError(RuntimeError):"); +} + +#[test] +fn generated_source_is_deterministic() { + let model = model(KV); + assert_eq!(render(&model), render(&model)); +} + +#[test] +fn package_bundles_the_identity_library_per_wheel_platform() { + let model = model(KV); + let mut binaries = BinarySet::new("kv_lib"); + for p in Platform::ALL { + binaries.insert(NativeBinary::new(p, format!("/tmp/{}/lib", p.id()))); + } + let mut ctx = PackageContext::new(&binaries); + ctx.macos_deployment_target = "12.0"; + let artifacts = PythonGenerator + .package(&model, &ctx, &PythonConfig::default()) + .expect("python packages"); + let wheels: Vec<&str> = artifacts.iter().map(|a| a.path.as_str()).collect(); + assert_eq!( + wheels, + [ + "python/kv_lib-0.1.0-py3-none-macosx_12_0_arm64.whl", + "python/kv_lib-0.1.0-py3-none-macosx_12_0_x86_64.whl", + "python/kv_lib-0.1.0-py3-none-manylinux_2_17_x86_64.whl", + "python/kv_lib-0.1.0-py3-none-manylinux_2_17_aarch64.whl", + "python/kv_lib-0.1.0-py3-none-win_amd64.whl", + ] + ); + let mac = &artifacts[0]; + let ArtifactKind::Wheel(meta) = &mac.kind else { + panic!("a wheel"); + }; + assert_eq!(meta.name, "kv-lib"); + assert_eq!(meta.requires_python.as_deref(), Some(">=3.9")); + let names: Vec<&str> = mac.files.iter().map(|f| f.path.as_str()).collect(); + assert_eq!( + names, + [ + "kv_lib/__init__.py", + "kv_lib/kv_lib.py", + "kv_lib/py.typed", + "kv_lib/libkv_lib.dylib", + ] + ); + assert!(artifacts[4].file("kv_lib/kv_lib.dll").is_some()); +} + +#[test] +fn members_never_replace_the_wrapper_close() { + let out = render(&model( + r#" +version: "0.11.0" +modules: + - name: kv + interfaces: + - name: Store + constructors: + - { name: close, params: [] } + methods: + - { name: close_all, params: [] } + statics: + - { name: closed, params: [], return: bool } + - name: Lock + methods: + - { name: close, params: [], return: bool } +"#, + )); + assert_has(&out, " def close_(cls) -> Store:"); + assert_has(&out, " def close_(self) -> bool:"); + assert_has(&out, " def close_all(self) -> None:"); + assert_has(&out, " def closed() -> bool:"); + // The runtime's `_Handle.close` is the only `close`. + assert_eq!( + out.matches("def close(").count(), + 1, + "a member replaced close" + ); +} diff --git a/crates/weaveffi-gen/src/targets/python/types.rs b/crates/weaveffi-cli/src/targets/python/types.rs similarity index 63% rename from crates/weaveffi-gen/src/targets/python/types.rs rename to crates/weaveffi-cli/src/targets/python/types.rs index 960f108f..9c5fa821 100644 --- a/crates/weaveffi-gen/src/targets/python/types.rs +++ b/crates/weaveffi-cli/src/targets/python/types.rs @@ -3,27 +3,36 @@ //! user-chosen IDL names before they land in generated Python. use crate::lang; -use crate::utils::{local_type_name, wrapper_name}; use heck::ToSnakeCase; use weaveffi_model::abi::CType; -use weaveffi_model::model::{FieldBinding, Ty}; +use weaveffi_model::model::FieldBinding; +use weaveffi_model::ty::{Prim, Ty}; -/// The Python typing hint for `ty` as it appears in signatures and stubs. -/// User types are quoted forward references to their bare local class name: -/// every module's declarations share one Python namespace, so a -/// cross-module reference (`kv.Store`) is just `"Store"`. +/// The Python typing hint for `ty` as it appears in annotations. The module +/// starts with `from __future__ import annotations`, so user types are +/// written bare even when declared later in the file; type names are +/// global, and every module's declarations share one Python namespace. pub(crate) fn py_type_hint(ty: &Ty) -> String { match ty { - Ty::I8 | Ty::I16 | Ty::I32 | Ty::U8 | Ty::U16 | Ty::U32 | Ty::I64 | Ty::U64 => "int".into(), - Ty::F32 | Ty::F64 => "float".into(), - Ty::Bool => "bool".into(), - Ty::StringUtf8 => "str".into(), - Ty::Bytes => "bytes".into(), + Ty::Prim( + Prim::I8 + | Prim::I16 + | Prim::I32 + | Prim::I64 + | Prim::U8 + | Prim::U16 + | Prim::U32 + | Prim::U64, + ) => "int".into(), + Ty::Prim(Prim::F32 | Prim::F64) => "float".into(), + Ty::Prim(Prim::Bool) => "bool".into(), + Ty::Prim(Prim::String) => "str".into(), + Ty::Prim(Prim::Bytes) => "bytes".into(), Ty::Enum(name) | Ty::Record(name) | Ty::RichEnum(name) | Ty::Interface(name) - | Ty::CallbackInterface(name) => format!("\"{}\"", local_type_name(name)), + | Ty::CallbackInterface(name) => name.clone(), Ty::Optional(inner) => format!("Optional[{}]", py_type_hint(inner)), Ty::List(inner) => format!("List[{}]", py_type_hint(inner)), Ty::Map(k, v) => format!("Dict[{}, {}]", py_type_hint(k), py_type_hint(v)), @@ -31,6 +40,11 @@ pub(crate) fn py_type_hint(ty: &Ty) -> String { } } +/// The hint for an optional return or `None`: `None` for a void callable. +pub(crate) fn py_return_hint(ty: Option<&Ty>) -> String { + ty.map_or_else(|| "None".to_string(), py_type_hint) +} + /// The builtin types an annotation names. const BUILTIN_HINTS: &[&str] = &["bool", "bytes", "float", "int", "str"]; @@ -45,10 +59,8 @@ pub(crate) fn py_field_hint(fields: &[FieldBinding], ty: &Ty) -> String { { return hint; } - // Qualify bare builtin identifiers outside quoted forward references. let mut out = String::with_capacity(hint.len()); let mut word = String::new(); - let mut quoted = false; let flush = |word: &mut String, out: &mut String| { if BUILTIN_HINTS.contains(&word.as_str()) { out.push_str("builtins."); @@ -57,14 +69,11 @@ pub(crate) fn py_field_hint(fields: &[FieldBinding], ty: &Ty) -> String { word.clear(); }; for c in hint.chars() { - if !quoted && (c.is_ascii_alphanumeric() || c == '_') { + if c.is_ascii_alphanumeric() || c == '_' { word.push(c); continue; } flush(&mut word, &mut out); - if c == '"' { - quoted = !quoted; - } out.push(c); } flush(&mut word, &mut out); @@ -125,6 +134,34 @@ pub(crate) fn py_ctype(ty: &CType, slot: Slot) -> String { } } +/// The Python type a `ctypes` callback receives for one C slot (a +/// trampoline or completion parameter), as an annotation: integers and +/// sizes are `int`, floats `float`, `bool` `bool`, an address +/// `Optional[int]` (`None` for null), and a typed pointer `Any`. +pub(crate) fn py_slot_hint(ty: &CType) -> &'static str { + match ty { + CType::Float | CType::Double => "float", + CType::Bool => "bool", + CType::Void => "None", + CType::CancelToken + | CType::StructTag { .. } + | CType::VtableTag { .. } + | CType::Named(_) => "Optional[int]", + CType::Error => "Any", + CType::Ptr { pointee, .. } => match pointee.as_ref() { + CType::Uint8 + | CType::Char + | CType::StructTag { .. } + | CType::VtableTag { .. } + | CType::CancelToken + | CType::Void + | CType::Named(_) => "Optional[int]", + _ => "Any", + }, + _ => "int", + } +} + /// Names a parameter must not take because wrapper and trampoline bodies /// refer to them: the receivers, the modules, and the builtins the /// generated code calls. Sorted for binary search. @@ -169,6 +206,31 @@ pub(crate) fn py_field(name: &str) -> String { lang::escape_ident(name, lang::PYTHON_KEYWORDS) } +/// Names an error code's payload field must not take: the attributes every +/// exception already has (or the generated error classes define), and the +/// constructor's receiver. +const EXCEPTION_ATTRS: &[&str] = &[ + "CODE", + "add_note", + "args", + "code", + "message", + "self", + "with_traceback", +]; + +/// The Python spelling of an error code's payload field: [`py_field`], +/// with a trailing `_` when it would shadow an exception attribute or the +/// receiver. +pub(crate) fn py_error_field(name: &str) -> String { + let field = py_field(name); + if EXCEPTION_ATTRS.contains(&field.as_str()) { + format!("{field}_") + } else { + field + } +} + /// The Python spelling of an enum variant, used both as the `IntEnum` member /// and as the suffix of a rich enum's per-variant dataclass. Variants are /// PascalCase, so only the capitalized keywords (`None`, `True`, `False`) @@ -177,24 +239,23 @@ pub(crate) fn py_variant(name: &str) -> String { lang::escape_ident(name, lang::PYTHON_KEYWORDS) } -/// The Python spelling of an interface member name (method, static, or -/// factory constructor): snake_case, keyword-escaped. +/// The Python spelling of a callable name (a free function, method, static, +/// or factory constructor): snake_case, keyword-escaped. Names are global, +/// so a free function needs no module prefix. pub(crate) fn py_member_name(name: &str) -> String { lang::escape_ident(&name.to_snake_case(), lang::PYTHON_KEYWORDS) } -/// The Python spelling of a module-level wrapper (free function): the -/// module-path prefix applied per config, then snake_case, then keyword -/// escaping. -pub(crate) fn py_wrapper_fn_name( - module_path: &str, - name: &str, - strip_module_prefix: bool, -) -> String { - lang::escape_ident( - &wrapper_name(module_path, name, strip_module_prefix).to_snake_case(), - lang::PYTHON_KEYWORDS, - ) +/// Names every object wrapper class already defines: `close` (from the +/// runtime's `_Handle`). The class's private helpers start with `_`, which +/// a snake-cased IDL name never does. +const OBJECT_MEMBERS: &[&str] = &["close"]; + +/// The Python spelling of an interface member (a factory constructor, +/// method, or static): [`py_member_name`], with a trailing `_` when it +/// would replace a member the wrapper class defines (`close` is `close_`). +pub(crate) fn py_object_member(name: &str) -> String { + lang::escape_member(&py_member_name(name), OBJECT_MEMBERS) } /// The module-level name a C symbol's bound `ctypes` function is stored @@ -209,5 +270,16 @@ pub(crate) fn py_binding_name(symbol: &str, prefix: &str) -> String { /// Escape a string for embedding in a double-quoted Python literal. pub(crate) fn py_str_literal(s: &str) -> String { - s.replace('\\', "\\\\").replace('"', "\\\"") + let mut out = String::with_capacity(s.len()); + for c in s.chars() { + match c { + '\\' => out.push_str("\\\\"), + '"' => out.push_str("\\\""), + '\n' => out.push_str("\\n"), + '\r' => out.push_str("\\r"), + '\t' => out.push_str("\\t"), + c => out.push(c), + } + } + out } diff --git a/crates/weaveffi-cli/src/targets/ruby/callbacks.rs b/crates/weaveffi-cli/src/targets/ruby/callbacks.rs new file mode 100644 index 00000000..d86a07e7 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/ruby/callbacks.rs @@ -0,0 +1,372 @@ +//! Callback-interface rendering: the consumer-facing Ruby module (duck-typed +//! method set with `NotImplementedError` defaults), the `FFI::Struct` vtable +//! layout with its `{size, flags, free}` header, one trampoline +//! `FFI::Function` per method plus `free`, and the single process-wide +//! vtable instance every registration passes to the producer. +//! +//! The trampolines follow [`CallbackProtocol`](weaveffi_model::plan::CallbackProtocol): +//! `ctx` is resolved through the module's implementation registry, object +//! arguments are adopted into wrappers first, borrowed string, bytes, and +//! buffer arguments are copied or decoded before the implementation runs, +//! and the return crosses back per its family (a direct value, a fresh +//! object reference, or a `{prefix}_alloc` run in the out slots). A domain +//! error raised by a method declared `throws` is reported with its code and +//! payload; any other exception is reported as `-4`. Nothing unwinds +//! through the C frame. + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use heck::ToShoutySnakeCase; +use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ErrorBinding}; +use weaveffi_model::plan::{ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::ty::{Prim, Ty}; + +use crate::targets::ruby::docs::{emit_param_docs, emit_return_doc}; +use crate::targets::ruby::entities::rb_error_payload_name; +use crate::targets::ruby::types::{rb_abi_types, rb_ffi_type, rb_param_name}; +use crate::targets::ruby::RbCtx; + +/// The `FFI::Struct` subclass laying out one callback interface's vtable: +/// `Listener` becomes `WvListenerVtable`. +fn rb_vtable_class(name: &str) -> String { + format!("Wv{name}Vtable") +} + +/// The constant holding the one process-wide vtable instance for a callback +/// interface: `Listener` becomes `WV_LISTENER_VTABLE`. +pub(crate) fn rb_vtable_const(name: &str) -> String { + format!("WV_{}_VTABLE", name.to_shouty_snake_case()) +} + +/// The constant pinning one method's trampoline `FFI::Function`: +/// `Listener.on_change` becomes `WV_LISTENER_ON_CHANGE`. +fn rb_trampoline_const(cb: &str, method: &str) -> String { + format!( + "WV_{}_{}", + cb.to_shouty_snake_case(), + method.to_shouty_snake_case() + ) +} + +/// The constant pinning the `free` trampoline. +fn rb_free_const(cb: &str) -> String { + format!("WV_{}_FREE", cb.to_shouty_snake_case()) +} + +/// The trampoline's Ruby formal parameter names for one method, position +/// for position with [`CallbackMethodBinding::abi_params`]: `wv_ctx`, the +/// value of a direct parameter under its own name, `{name}_ptr` (and +/// `{name}_len`) for every other parameter, the `wv_out_ptr`/`wv_out_len` +/// slots of a string, bytes, or buffer return, then `wv_err`. +fn rb_tramp_formals(m: &CallbackMethodBinding) -> Vec { + let mut formals = vec!["wv_ctx".to_string()]; + for p in &m.params { + let n = rb_param_name(&p.name); + match p.arg_pass() { + ArgPass::String { .. } | ArgPass::Bytes { .. } | ArgPass::Buffer { .. } => { + formals.push(format!("{n}_ptr")); + formals.push(format!("{n}_len")); + } + ArgPass::Object { .. } => formals.push(format!("{n}_ptr")), + _ => formals.push(n), + } + } + if matches!( + RetPass::of(m.ret.as_ref()), + RetPass::String | RetPass::Bytes | RetPass::Buffer + ) { + formals.push("wv_out_ptr".to_string()); + formals.push("wv_out_len".to_string()); + } + formals.push("wv_err".to_string()); + formals +} + +/// Render one callback interface: the consumer-facing module, the vtable +/// layout, the trampolines, and the static vtable instance. `error` is the +/// domain in scope for the interface's module, which its `throws` methods +/// may raise. +pub(crate) fn render_callback_interface( + w: &mut CodeWriter, + ctx: &RbCtx, + error: Option<&ErrorBinding>, + cb: &CallbackInterfaceBinding, +) { + let local = &cb.name; + let vtable_class = rb_vtable_class(&cb.name); + let vtable_const = rb_vtable_const(&cb.name); + + // The consumer-facing module: documentation of the required methods, and + // NotImplementedError defaults for consumers who include it. + w.blank(); + w.doc(&cb.doc, DocCommentStyle::Hash); + if cb.doc.is_some() { + w.line("#"); + } + w.line("# Consumer-implemented callback interface. Any object responding to the"); + w.line(format!( + "# methods below is accepted wherever a {local} is expected; include" + )); + w.line("# this module to inherit NotImplementedError defaults. The library"); + w.line("# may call the methods from any thread until it releases the"); + w.line("# implementation."); + if let Some(msg) = &cb.deprecated { + w.line(format!("# @deprecated {msg}")); + } + w.block(format!("module {local}"), "end", |w| { + for (idx, m) in cb.methods.iter().enumerate() { + if idx > 0 { + w.blank(); + } + w.doc(&m.doc, DocCommentStyle::Hash); + if let Some(msg) = &m.deprecated { + w.line(format!("# @deprecated {msg}")); + } + emit_param_docs(w, &m.params); + emit_return_doc(w, m.ret.as_ref()); + if let (ErrorStrategy::Throws, Some(eb)) = (m.error_strategy(), error) { + w.line(format!( + "# @raise [{}] reported to the library with its code and fields", + eb.type_name + )); + } + let formals: Vec = m.params.iter().map(|p| rb_param_name(&p.name)).collect(); + let open = if formals.is_empty() { + format!("def {}", m.name) + } else { + format!("def {}({})", m.name, formals.join(", ")) + }; + w.block(open, "end", |w| { + w.line(format!( + "raise NotImplementedError, \"#{{self.class}}#{} is not implemented\"", + m.name + )); + }); + } + }); + + // The vtable layout: the fixed header, then one pointer per method in + // declaration order. + w.blank(); + w.line("# @api private"); + w.line(format!( + "# The C vtable layout the library calls a {local} through." + )); + w.block(format!("class {vtable_class} < FFI::Struct"), "end", |w| { + let mut fields = vec![ + ":wv_size, :uint32".to_string(), + ":wv_flags, :uint32".to_string(), + ":wv_free, :pointer".to_string(), + ]; + fields.extend(cb.methods.iter().map(|m| format!(":{}, :pointer", m.name))); + w.line(format!("layout {}", fields.join(",\n "))); + }); + + // The free hook: drop the registry entry; the producer never touches + // ctx again after this fires. + w.blank(); + w.line("# @api private"); + w.line(format!( + "# Releases a {local} implementation when the library drops its last" + )); + w.line("# reference. May run on any library thread."); + w.block( + format!( + "{} = FFI::Function.new(:void, [:pointer]) do |ctx|", + rb_free_const(&cb.name) + ), + "end", + |w| { + w.line("_wv_cb_free(ctx)"); + }, + ); + + let protocol = cb.protocol(); + for (m, args) in cb.methods.iter().zip(&protocol.method_args) { + render_trampoline(w, ctx, error, cb, m, args); + } + + // The single static vtable instance, filled with the pinned trampolines. + w.blank(); + w.line(format!( + "# The one process-wide {local} vtable every registration hands the" + )); + w.line("# library; its entries live for the process lifetime."); + w.line(format!("{vtable_const} = {vtable_class}.new")); + w.line(format!("{vtable_const}[:wv_size] = {vtable_class}.size")); + w.line(format!("{vtable_const}[:wv_flags] = 0")); + w.line(format!( + "{vtable_const}[:wv_free] = {}", + rb_free_const(&cb.name) + )); + for m in &cb.methods { + w.line(format!( + "{vtable_const}[:{}] = {}", + m.name, + rb_trampoline_const(&cb.name, &m.name) + )); + } +} + +/// The integer-range kind (`:i32`, ...) the runtime checks a direct +/// integer return against; C-style enums cross as `i32`. +fn int_kind(ty: &Ty) -> &'static str { + match ty { + Ty::Prim(p) if p.is_integer() => p.snake(), + Ty::Enum(_) => "i32", + other => unreachable!("{other} is not an integer type"), + } +} + +/// Render one method's trampoline: an `FFI::Function` matching the vtable +/// entry's C signature that receives each argument per its [`RetPass`] +/// plan, resolves `ctx`, invokes the implementation, and hands the return +/// back per its family. Object arguments are adopted first, so their +/// references are owned by a wrapper even when a later step raises. A +/// domain error from a `throws` method goes through `_wv_cb_throw`; any +/// other exception through `_wv_cb_fail` (code `-4`); either way the +/// method's zero value is returned. +fn render_trampoline( + w: &mut CodeWriter, + ctx: &RbCtx, + error: Option<&ErrorBinding>, + cb: &CallbackInterfaceBinding, + m: &CallbackMethodBinding, + args: &[RetPass], +) { + let ret_pass = RetPass::of(m.ret.as_ref()); + let default = match (&ret_pass, m.ret.as_ref()) { + (RetPass::Direct, Some(Ty::Prim(Prim::Bool))) => "false", + (RetPass::Direct, Some(Ty::Prim(Prim::F32 | Prim::F64))) => "0.0", + (RetPass::Direct, _) => "0", + (RetPass::Object { .. }, _) => "FFI::Pointer::NULL", + _ => "nil", + }; + + w.blank(); + w.line("# @api private"); + w.line(format!("# Trampoline for {}#{}.", cb.name, m.name)); + w.line(format!( + "{} = FFI::Function.new({}, [{}]) do |{}|", + rb_trampoline_const(&cb.name, &m.name), + rb_ffi_type(&m.abi_ret), + rb_abi_types(&m.abi_params).join(", "), + rb_tramp_formals(m).join(", ") + )); + w.scope(|w| { + let params: Vec<_> = m.params.iter().zip(args).collect(); + let is_object = |pass: &RetPass| matches!(pass, RetPass::Object { .. }); + for (p, pass) in params.iter().filter(|(_, pass)| is_object(pass)) { + render_tramp_arg(w, ctx, &p.name, &p.ty, pass); + } + for (p, pass) in params.iter().filter(|(_, pass)| !is_object(pass)) { + render_tramp_arg(w, ctx, &p.name, &p.ty, pass); + } + let call_args: Vec = m.params.iter().map(|p| rb_param_name(&p.name)).collect(); + let call = if call_args.is_empty() { + format!("_wv_cb_lookup(wv_ctx).{}", m.name) + } else { + format!("_wv_cb_lookup(wv_ctx).{}({})", m.name, call_args.join(", ")) + }; + // Every conversion runs inside the rescue, so a wrong-typed return + // surfaces as a callback failure rather than escaping through ffi's + // own conversion. + match (&ret_pass, m.ret.as_ref()) { + (RetPass::Void, _) => { + w.line(call); + w.line("nil"); + } + (RetPass::Direct, Some(Ty::Prim(Prim::Bool))) => { + w.line(format!("{call} ? true : false")); + } + (RetPass::Direct, Some(Ty::Prim(Prim::F32 | Prim::F64))) => { + w.line(format!("_wv_float({call})")); + } + (RetPass::Direct, Some(ty)) => { + w.line(format!("_wv_int({call}, :{})", int_kind(ty))); + } + (RetPass::Object { .. }, Some(ty)) => { + let class = ty.interface_name().expect("object plan names an interface"); + w.line(format!("_wv_cb_return_object({call}, {class})")); + } + (RetPass::String, _) => { + w.line(format!( + "_wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_str({call}))" + )); + } + (RetPass::Bytes, _) => { + w.line(format!( + "_wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_bytes({call}))" + )); + } + (RetPass::Buffer, Some(ty)) if ctx.codecs.carries_objects(ty) => { + w.line(format!("result = {call}")); + w.line("writer = WvBufferWriter.new"); + w.line(ctx.codecs.write(ty, "writer", "result", "")); + w.line("_wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_seal(writer).first)"); + } + (RetPass::Buffer, Some(ty)) => { + w.line(format!("result = {call}")); + w.line(format!( + "_wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_encode {{ |w| {} }})", + ctx.codecs.write(ty, "w", "result", "") + )); + } + _ => unreachable!("a typed return has a type"), + } + }); + if let (ErrorStrategy::Throws, Some(eb)) = (m.error_strategy(), error) { + w.line(format!("rescue {} => e", eb.type_name)); + w.scope(|w| { + w.line(format!( + "_wv_cb_throw(wv_err, e) {{ {}(e) }}", + rb_error_payload_name(eb) + )); + w.line(default); + }); + } + // Rescue Exception, not StandardError: NotImplementedError is a + // ScriptError, and nothing may unwind through the C frame. + w.line("rescue Exception => e # rubocop:disable Lint/RescueException"); + w.scope(|w| { + w.line("_wv_cb_fail(wv_err, e)"); + w.line(default); + }); + w.line("end"); +} + +/// Emit the statement receiving one trampoline argument into the local +/// named after the parameter, per its [`RetPass`] plan. Strings, bytes, and +/// buffers are borrowed `(ptr, len)` pairs copied or decoded before the +/// implementation runs; objects transfer one strong reference that is +/// adopted into a wrapper (`nil` for a null optional slot); direct values +/// arrive under the parameter's own name already. +fn render_tramp_arg(w: &mut CodeWriter, ctx: &RbCtx, name: &str, ty: &Ty, pass: &RetPass) { + let n = rb_param_name(name); + match pass { + RetPass::Void => unreachable!("callback parameters always have a type"), + RetPass::Direct => {} + RetPass::String => { + w.line(format!("{n} = _wv_borrow_string({n}_ptr, {n}_len)")); + } + RetPass::Bytes => { + w.line(format!("{n} = _wv_borrow_bytes({n}_ptr, {n}_len)")); + } + RetPass::Buffer => { + w.line(format!( + "{n} = _wv_decode_borrowed({n}_ptr, {n}_len) {{ |r| {} }}", + ctx.codecs.read(ty, "r", "") + )); + } + RetPass::Object { nullable } => { + let class = ty.interface_name().expect("object plan names an interface"); + if *nullable { + w.line(format!( + "{n} = {n}_ptr.null? ? nil : {class}._from_ptr({n}_ptr)" + )); + } else { + w.line(format!("{n} = {class}._from_ptr({n}_ptr)")); + } + } + } +} diff --git a/crates/weaveffi-gen/src/targets/ruby/calls.rs b/crates/weaveffi-cli/src/targets/ruby/calls.rs similarity index 53% rename from crates/weaveffi-gen/src/targets/ruby/calls.rs rename to crates/weaveffi-cli/src/targets/ruby/calls.rs index 553d31a9..c6b373ac 100644 --- a/crates/weaveffi-gen/src/targets/ruby/calls.rs +++ b/crates/weaveffi-cli/src/targets/ruby/calls.rs @@ -8,81 +8,93 @@ //! borrowed `(ptr, len)` pairs, objects are borrowed as parameters (pinned //! for the call through `_wv_pin`) and adopted as returns, and a callback //! interface crosses as a handle-table key plus the interface's static -//! vtable. +//! vtable (or two NULLs for an absent optional one). +//! +//! Every call into the library except the trivial runtime helpers releases +//! the GVL (`blocking: true`): a call may run for a long time, and a +//! producer thread may need the GVL to run a callback implementation while +//! the call is in flight. use crate::codegen::common::DocCommentStyle; use crate::codegen::CodeWriter; -use crate::utils::{local_type_name, wrapper_name}; use heck::{ToShoutySnakeCase, ToSnakeCase}; -use weaveffi_model::model::{AsyncBinding, CallShape, FnBinding, IteratorBinding, ModuleBinding}; -use weaveffi_model::model::{ParamBinding, Ty}; -use weaveffi_model::plan::{self, ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::model::ParamBinding; +use weaveffi_model::model::{AsyncBinding, CallShape, ErrorBinding, FnBinding, IteratorBinding}; +use weaveffi_model::plan::{ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::ty::{Prim, Ty}; use crate::targets::ruby::callbacks::rb_vtable_const; -use crate::targets::ruby::codec::render_wv_read; -use crate::targets::ruby::codec::render_wv_write; -use crate::targets::ruby::docs::emit_param_docs; +use crate::targets::ruby::docs::{emit_param_docs, emit_return_doc}; use crate::targets::ruby::entities::{rb_checker_name, rb_error_factory_name}; use crate::targets::ruby::types::{ - rb_abi_types, rb_direct_type, rb_ffi_type, rb_param_name, rb_str_literal, + rb_abi_types, rb_direct_type, rb_ffi_type, rb_member_name, rb_param_name, rb_str_literal, }; +use crate::targets::ruby::RbCtx; -/// How a rendered Ruby callable is scoped and spelled in the generated -/// module: at module scope as a singleton method, or inside an interface -/// class as a constructor, instance method, or class method. -pub(crate) enum RbScope<'a> { +/// Where a rendered Ruby callable lives and how it's spelled. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum ScopeKind { /// A module-level free function (`def self.name` on the top-level module). - Free { - /// The owning module's underscore-joined path. - module_path: &'a str, - /// Whether the emitted name drops the module-path prefix. - strip_module_prefix: bool, - }, + Free, /// An instance method on an interface class: `def name`, borrowing the /// wrapper's own pointer as the leading C argument. - Method { - /// The top-level Ruby module name qualifying module singleton calls. - module_name: &'a str, - }, + Method, /// A static member of an interface class (`def self.name`). - Static { - /// The top-level Ruby module name qualifying module singleton calls. - module_name: &'a str, - }, + Static, /// A non-`new` constructor: a class method adopting the returned /// reference through `_from_ptr` (never re-running `initialize`). - Factory { - /// The top-level Ruby module name qualifying module singleton calls. - module_name: &'a str, - }, + Factory, /// The canonical `new` constructor, emitted as `initialize`. - Init { - /// The top-level Ruby module name qualifying module singleton calls. - module_name: &'a str, - }, + Init, +} + +/// The scope of one rendered callable: its kind, the top-level Ruby module, +/// and (for an interface member) the class it belongs to. +pub(crate) struct RbScope<'a> { + /// Where the callable lives. + pub(crate) kind: ScopeKind, + /// The top-level Ruby module name. + pub(crate) module: &'a str, + /// The interface class of a member; `None` for a free function. + pub(crate) class: Option<&'a str>, } -impl RbScope<'_> { +impl<'a> RbScope<'a> { + /// The scope of a free function. + pub(crate) fn free(module: &'a str) -> Self { + RbScope { + kind: ScopeKind::Free, + module, + class: None, + } + } + + /// The scope of an interface member of the given kind. + pub(crate) fn member(kind: ScopeKind, module: &'a str, class: &'a str) -> Self { + RbScope { + kind, + module, + class: Some(class), + } + } + /// The receiver prefix for module singleton calls (attached C symbols, /// error checkers, runtime helpers): `"{ModuleName}."` inside a class /// body, empty at module scope, where `self` already is the module. fn qualifier(&self) -> String { - match self { - RbScope::Free { .. } => String::new(), - RbScope::Method { module_name } - | RbScope::Static { module_name } - | RbScope::Factory { module_name } - | RbScope::Init { module_name } => format!("{module_name}."), + match self.kind { + ScopeKind::Free => String::new(), + _ => format!("{}.", self.module), } } - /// Two-space indent depth of the `def` line (1 at module scope, 2 inside - /// an interface class). - fn depth(&self) -> usize { - if matches!(self, RbScope::Free { .. }) { - 1 - } else { - 2 + /// The Ruby name of `f` in this scope: snake_case, escaped away from + /// the wrapper's own methods for an interface member. + fn name(&self, f: &FnBinding) -> String { + match self.kind { + ScopeKind::Method => rb_member_name(&f.name, true), + ScopeKind::Static | ScopeKind::Factory => rb_member_name(&f.name, false), + ScopeKind::Free | ScopeKind::Init => f.name.to_snake_case(), } } @@ -93,79 +105,88 @@ impl RbScope<'_> { } else { format!("({})", params.join(", ")) }; - match self { - RbScope::Free { - module_path, - strip_module_prefix, - } => format!( - "def self.{}{args}", - wrapper_name(module_path, &f.name, *strip_module_prefix).to_snake_case() - ), - RbScope::Method { .. } => format!("def {}{args}", f.name.to_snake_case()), - RbScope::Static { .. } | RbScope::Factory { .. } => { - format!("def self.{}{args}", f.name.to_snake_case()) + let name = self.name(f); + match self.kind { + ScopeKind::Method => format!("def {name}{args}"), + ScopeKind::Free | ScopeKind::Static | ScopeKind::Factory => { + format!("def self.{name}{args}") } - RbScope::Init { .. } => format!("def initialize{args}"), + ScopeKind::Init => format!("def initialize{args}"), + } + } + + /// How a caller spells `f`: `Kvstore.open_store`, `Kvstore::Store#size`, + /// `Kvstore::Store.open`, or `Kvstore::Store.new`. + fn display(&self, f: &FnBinding) -> String { + let name = self.name(f); + let module = self.module; + match (self.kind, self.class) { + (ScopeKind::Method, Some(class)) => format!("{module}::{class}#{name}"), + (ScopeKind::Init, Some(class)) => format!("{module}::{class}.new"), + (_, Some(class)) => format!("{module}::{class}.{name}"), + (_, None) => format!("{module}.{name}"), } } } -/// Render one callable: a free function or an interface member. `module` -/// supplies the error domain for throwing callables; `scope` picks the def -/// spelling, receiver, indent, and result handling. +/// Render one callable: a free function or an interface member. `error` is +/// the domain in scope for throwing callables; `scope` picks the def +/// spelling, receiver, and result handling. pub(crate) fn render_callable( - out: &mut String, - module: &ModuleBinding, + w: &mut CodeWriter, + ctx: &RbCtx, + error: Option<&ErrorBinding>, f: &FnBinding, scope: &RbScope, ) { - let mut w = CodeWriter::two_space().with_depth(scope.depth()); w.blank(); w.doc(&f.doc, DocCommentStyle::Hash); - emit_param_docs(&mut w, &f.params); + match &f.shape { + CallShape::Sync(_) => {} + CallShape::Async(_) => { + w.line("# Blocks until the call completes on a producer thread."); + } + CallShape::Iterator(_) => { + w.line("# Returns a lazy Enumerator that pulls one element per step. The"); + w.line("# producer iterator starts on the first pull and is released when"); + w.line("# iteration finishes or is abandoned early."); + } + } + emit_param_docs(w, &f.params); + if f.cancellable { + w.line("# @param cancel [CancelToken, nil] cancels the call; it then raises Cancelled"); + } + if scope.kind != ScopeKind::Init { + emit_return_doc(w, f.ret.as_ref()); + } if let Some(msg) = &f.deprecated { w.line(format!("# @deprecated {msg}")); } match &f.shape { - CallShape::Sync(abi) => render_sync(&mut w, module, f, &abi.symbol, scope), - CallShape::Async(a) => render_async(&mut w, f, a, scope), - CallShape::Iterator(it) => render_iterator(&mut w, module, f, it, scope), - } - out.push_str(&w.finish()); -} - -/// The trailing `, blocking: true` of an attachment that releases the GVL. -fn blocking_opt(blocking: bool) -> &'static str { - if blocking { - ", blocking: true" - } else { - "" + CallShape::Sync(abi) => render_sync(w, ctx, error, f, &abi.symbol, scope), + CallShape::Async(a) => render_async(w, ctx, f, a, scope), + CallShape::Iterator(it) => render_iterator(w, ctx, error, f, it, scope), } } /// Attach the C symbols for one callable: the plain symbol for a sync shape, -/// the launcher plus its completion trampoline for an async shape, and the -/// launch/next/destroy triple for an iterator. +/// the launcher for an async shape (its completion trampoline is rendered +/// by [`render_async_trampoline`]), and the launch/next/destroy triple for +/// an iterator. /// -/// `blocking` marks calls that may call back into Ruby (every call in an API -/// with callback interfaces); they release the GVL so a producer thread can -/// run a callback while the call is in flight. Async launches and iterator -/// steps always release it. -pub(crate) fn render_attach_function( - out: &mut String, - module: &ModuleBinding, - f: &FnBinding, - blocking: bool, -) { - let mut w = CodeWriter::two_space().with_depth(1); +/// Calls release the GVL (`blocking: true`) so a long call doesn't stall +/// other Ruby threads and a producer thread can run a callback +/// implementation while the call is in flight; an iterator's `_destroy`, +/// like an object's `_clone` and `_destroy`, is a trivial helper that keeps +/// it (and may therefore run from a GC finalizer). +pub(crate) fn render_attach_function(w: &mut CodeWriter, f: &FnBinding) { match &f.shape { CallShape::Sync(abi) => { w.line(format!( - "attach_function :{}, [{}], {}{}", + "attach_function :{}, [{}], {}, blocking: true", abi.symbol, rb_abi_types(&abi.params).join(", "), rb_ffi_type(&abi.ret), - blocking_opt(blocking) )); } CallShape::Async(a) => { @@ -174,7 +195,6 @@ pub(crate) fn render_attach_function( a.launch.symbol, rb_abi_types(&a.launch.params).join(", ") )); - render_async_trampoline(&mut w, module, f, a); } CallShape::Iterator(it) => { w.line(format!( @@ -188,13 +208,11 @@ pub(crate) fn render_attach_function( rb_abi_types(&it.next.params).join(", ") )); w.line(format!( - "attach_function :{}, [:pointer], :void{}", + "attach_function :{}, [:pointer], :void", it.destroy_symbol, - blocking_opt(blocking) )); } } - out.push_str(&w.finish()); } /// The constant pinning an async function's completion trampoline, named @@ -205,27 +223,29 @@ fn rb_async_const(a: &AsyncBinding) -> String { } /// The error a completion's `taken` triple becomes: the domain error for a -/// throwing function, the generic error otherwise. -fn rb_error_from_taken(module: &ModuleBinding, f: &FnBinding, q: &str) -> String { - match (f.error_strategy(), module.error.as_ref()) { - (ErrorStrategy::Throws, Some(eb)) => { - format!("{q}{}(*taken)", rb_error_factory_name(eb)) - } - _ => format!("{q}_wv_error(taken[0], taken[1])"), +/// throwing function, the trap (`NativeBugError`, or `Cancelled`) +/// otherwise. +fn rb_error_from_taken(error: Option<&ErrorBinding>, f: &FnBinding) -> String { + match (f.error_strategy(), error) { + (ErrorStrategy::Throws, Some(eb)) => format!("{}(*taken)", rb_error_factory_name(eb)), + _ => "_wv_trap(*taken)".to_string(), } } -/// Render the module-level completion trampoline of one async function. It -/// resolves the call's queue from `context`, converts the error or result -/// (copying and releasing any owned buffer), and pushes it; nothing raised -/// while converting escapes into the C frame, so the waiting call always -/// wakes up. -fn render_async_trampoline( +/// Render the module-level completion trampoline of an async function (a +/// no-op for any other shape). It resolves the call's queue from +/// `context`, converts the error or result (copying and releasing any owned +/// buffer), and pushes it; nothing raised while converting escapes into the +/// C frame, so the waiting call always wakes up. +pub(crate) fn render_async_trampoline( w: &mut CodeWriter, - module: &ModuleBinding, + ctx: &RbCtx, + error: Option<&ErrorBinding>, f: &FnBinding, - a: &AsyncBinding, ) { + let CallShape::Async(a) = &f.shape else { + return; + }; let mut formals = vec!["ctx".to_string(), "err".to_string()]; formals.extend(a.callback_params.iter().skip(2).map(|p| p.name.clone())); w.blank(); @@ -246,15 +266,16 @@ fn render_async_trampoline( w.line("taken = _wv_take_boxed_error(err)"); w.line("if taken"); w.scope(|w| { - w.line(format!("queue << {}", rb_error_from_taken(module, f, ""))); + w.line(format!("queue << {}", rb_error_from_taken(error, f))); }); w.line("else"); w.scope(|w| { - let ptr = match plan::ret_pass(f.ret.as_ref(), "") { + let ptr = match RetPass::of(f.ret.as_ref()) { RetPass::String | RetPass::Bytes | RetPass::Buffer => "result_ptr", _ => "result", }; - receive_value(w, f.ret.as_ref(), ptr, "result_len", "", "value = "); + let value = receive_value(ctx, f.ret.as_ref(), ptr, "result_len", ""); + w.line(format!("value = {value}")); w.line("queue << value"); }); w.line("end"); @@ -266,54 +287,33 @@ fn render_async_trampoline( w.line("end"); }, ); - w.blank(); } -/// Emit the statements that turn a received value into its Ruby value: a -/// sync return (`ptr` names the C result, `len` the length read from -/// `out_len`), an async result, or an iterator element. Owned strings, bytes, -/// and buffers are copied and released; objects are adopted. The final -/// expression is prefixed with `dest` (an assignment such as `"value = "`, -/// or empty to leave it as the block's value). -fn receive_value(w: &mut CodeWriter, ty: Option<&Ty>, ptr: &str, len: &str, q: &str, dest: &str) { - match plan::ret_pass(ty, "") { - RetPass::Void => { - w.line(format!("{dest}nil")); - } - RetPass::Direct => { - w.line(format!("{dest}{ptr}")); - } - RetPass::String => { - w.line(format!("{dest}{q}_wv_take_string({ptr}, {len})")); - } - RetPass::Bytes => { - w.line(format!("{dest}{q}_wv_take_bytes({ptr}, {len})")); - } +/// The expression turning a received value into its Ruby value: a sync +/// return (`ptr` names the C result, `len` the length read from `out_len`), +/// an async result, or an iterator element. Owned strings, bytes, and +/// buffers are copied and released; objects are adopted. +fn receive_value(ctx: &RbCtx, ty: Option<&Ty>, ptr: &str, len: &str, q: &str) -> String { + match RetPass::of(ty) { + RetPass::Void => "nil".to_string(), + RetPass::Direct => ptr.to_string(), + RetPass::String => format!("{q}_wv_take_string({ptr}, {len})"), + RetPass::Bytes => format!("{q}_wv_take_bytes({ptr}, {len})"), RetPass::Buffer => { let ty = ty.expect("buffered value has a type"); - w.block( - format!("{dest}{q}_wv_decode({ptr}, {len}) do |_wv_r|"), - "end", - |w| { - render_wv_read(w, "_wv_r", "_wv_value", ty, 0, q); - w.line("_wv_value"); - }, - ); + format!( + "{q}_wv_decode({ptr}, {len}) {{ |r| {} }}", + ctx.codecs.read(ty, "r", q) + ) } - RetPass::Object { nullable, .. } => { - let class = local_type_name( - ty.and_then(Ty::interface_name) - .expect("object value names an interface"), - ); + RetPass::Object { nullable } => { + let class = ty + .and_then(Ty::interface_name) + .expect("object value names an interface"); if nullable { - w.line(format!( - "{dest}{ptr}.null? ? nil : {class}._from_ptr({ptr})" - )); + format!("{ptr}.null? ? nil : {class}._from_ptr({ptr})") } else { - w.line(format!( - "raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if {ptr}.null?" - )); - w.line(format!("{dest}{class}._from_ptr({ptr})")); + format!("{class}._from_ptr({q}_wv_nonnull({ptr}))") } } } @@ -323,7 +323,7 @@ fn receive_value(w: &mut CodeWriter, ty: Option<&Ty>, ptr: &str, len: &str, q: & /// pairs: the receiver for an instance method, then each object parameter. fn pinned_objects(f: &FnBinding, scope: &RbScope) -> Vec<(String, String)> { let mut pins = Vec::new(); - if matches!(scope, RbScope::Method { .. }) { + if scope.kind == ScopeKind::Method { pins.push(("self".to_string(), "_wv_self".to_string())); } for p in &f.params { @@ -336,11 +336,13 @@ fn pinned_objects(f: &FnBinding, scope: &RbScope) -> Vec<(String, String)> { } /// Run `body` inside a `_wv_pin` block over `pins` (each object stays alive, -/// and its wrapper can't release its reference, until the block returns), or -/// directly when nothing is borrowed. +/// and its wrapper can't release its reference, until the block returns), +/// or directly when nothing is borrowed. `lead` prefixes the block opener +/// (an assignment of the block's value, or empty). fn with_pins( w: &mut CodeWriter, q: &str, + lead: &str, pins: &[(String, String)], body: impl FnOnce(&mut CodeWriter), ) { @@ -351,18 +353,24 @@ fn with_pins( let exprs: Vec<&str> = pins.iter().map(|(e, _)| e.as_str()).collect(); let vars: Vec<&str> = pins.iter().map(|(_, v)| v.as_str()).collect(); w.block( - format!("{q}_wv_pin({}) do |{}|", exprs.join(", "), vars.join(", ")), + format!( + "{lead}{q}_wv_pin({}) do |{}|", + exprs.join(", "), + vars.join(", ") + ), "end", body, ); } -/// Emit the statements converting the parameters into the locals the C call -/// borrows. Strings and bytes become private binary copies, buffered values -/// are encoded, and callback implementations are registered last, so nothing -/// that can raise runs between a registration and the call that hands it to -/// the producer. -fn render_param_prep(w: &mut CodeWriter, f: &FnBinding, q: &str) { +/// Emit the statements validating the parameters and converting them into +/// the locals the C call borrows. Strings and bytes become private binary +/// copies and buffered values are encoded; object arguments are checked +/// against their interface and a required callback implementation against +/// `nil`. Callback implementations are registered later (see +/// [`render_handoff`]), so nothing that can raise runs between a +/// registration and the call that hands it to the producer. +fn render_param_prep(w: &mut CodeWriter, ctx: &RbCtx, f: &FnBinding, q: &str) { for p in &f.params { let name = rb_param_name(&p.name); match p.arg_pass() { @@ -372,26 +380,46 @@ fn render_param_prep(w: &mut CodeWriter, f: &FnBinding, q: &str) { ArgPass::Bytes { .. } => { w.line(format!("{name}_s = {q}_wv_bytes({name})")); } - ArgPass::Buffer { .. } => { + ArgPass::Buffer { .. } if ctx.codecs.carries_objects(&p.ty) => { w.line(format!("{name}_w = WvBufferWriter.new")); - render_wv_write(w, &format!("{name}_w"), &name, &p.ty, 0, q); + w.line(ctx.codecs.write(&p.ty, &format!("{name}_w"), &name, q)); + } + ArgPass::Buffer { .. } => { + w.line(format!( + "{name}_s = {q}_wv_encode {{ |w| {} }}", + ctx.codecs.write(&p.ty, "w", &name, q) + )); + } + ArgPass::Object { nullable, .. } => { + let class = + p.ty.interface_name() + .expect("object plan names an interface"); + let optional = if nullable { ", true" } else { "" }; + w.line(format!( + "{q}_wv_object!({name}, {class}, '{name}'{optional})" + )); + } + ArgPass::Callback { + nullable: false, .. + } => { + w.line(format!("{q}_wv_present!({name}, '{name}')")); } - _ => {} + ArgPass::Callback { .. } | ArgPass::Direct { .. } => {} } } } /// Emit the statements that hand the producer what it adopts, immediately -/// before the call: seal the buffers (minting the references their object -/// tokens carry; a buffer without objects seals to its bytes as is) and then -/// register every callback-interface parameter. Registration can't raise, -/// so nothing between here and the call can strand a minted reference or a -/// registration. -fn render_handoff(w: &mut CodeWriter, f: &FnBinding, q: &str) { +/// before the call: seal the object-carrying buffers (minting the +/// references their tokens carry) and then register every callback +/// implementation. Registration can't raise, so nothing between here and +/// the call can strand a minted reference or a registration. +fn render_handoff(w: &mut CodeWriter, ctx: &RbCtx, f: &FnBinding, q: &str) { let sealed: Vec = f .params .iter() .filter(|p| matches!(p.arg_pass(), ArgPass::Buffer { .. })) + .filter(|p| ctx.codecs.carries_objects(&p.ty)) .map(|p| rb_param_name(&p.name)) .collect(); match sealed.as_slice() { @@ -400,28 +428,25 @@ fn render_handoff(w: &mut CodeWriter, f: &FnBinding, q: &str) { w.line(format!("{one}_s = {q}_wv_seal({one}_w).first")); } many => { + let locals: Vec = many.iter().map(|n| format!("{n}_s")).collect(); + let writers: Vec = many.iter().map(|n| format!("{n}_w")).collect(); w.line(format!( "{} = {q}_wv_seal({})", - many.iter() - .map(|n| format!("{n}_s")) - .collect::>() - .join(", "), - many.iter() - .map(|n| format!("{n}_w")) - .collect::>() - .join(", ") + locals.join(", "), + writers.join(", ") )); } } - render_callback_registrations(w, f, q); -} - -/// Register every callback-interface parameter (see [`render_handoff`]). -fn render_callback_registrations(w: &mut CodeWriter, f: &FnBinding, q: &str) { for p in &f.params { if matches!(p.arg_pass(), ArgPass::Callback { .. }) { let name = rb_param_name(&p.name); - w.line(format!("{name}_ctx = {q}_wv_cb_register({name})")); + let cb = + p.ty.callback_interface_name() + .expect("callback plan names a callback interface"); + w.line(format!( + "{name}_ctx, {name}_vtable = {q}_wv_cb_register({name}, {})", + rb_vtable_const(cb) + )); } } } @@ -435,16 +460,8 @@ fn rb_call_args(p: &ParamBinding) -> Vec { vec![format!("{name}_s"), format!("{name}_s.bytesize")] } ArgPass::Object { .. } => vec![format!("_wv_{name}")], - ArgPass::Callback { .. } => { - let cb = - p.ty.callback_interface_name() - .expect("callback plan names a callback interface"); - vec![ - format!("{name}_ctx"), - format!("{}.to_ptr", rb_vtable_const(cb)), - ] - } - ArgPass::Direct { .. } if matches!(p.ty, Ty::Bool) => { + ArgPass::Callback { .. } => vec![format!("{name}_ctx"), format!("{name}_vtable")], + ArgPass::Direct { .. } if matches!(p.ty, Ty::Prim(Prim::Bool)) => { vec![format!("({name} ? true : false)")] } ArgPass::Direct { .. } => vec![name], @@ -455,7 +472,7 @@ fn rb_call_args(p: &ParamBinding) -> Vec { /// parameter's slots. fn call_args(f: &FnBinding, scope: &RbScope) -> Vec { let mut args = Vec::new(); - if matches!(scope, RbScope::Method { .. }) { + if scope.kind == ScopeKind::Method { args.push("_wv_self".to_string()); } for p in &f.params { @@ -464,13 +481,39 @@ fn call_args(f: &FnBinding, scope: &RbScope) -> Vec { args } +/// The Ruby formal parameters of `f`. Trailing optional parameters (`T?`, +/// `Cb?`) default to `nil`, so callers may leave them out. fn formals(f: &FnBinding) -> Vec { - f.params.iter().map(|p| rb_param_name(&p.name)).collect() + let optional = f + .params + .iter() + .rev() + .take_while(|p| matches!(p.ty, Ty::Optional(_))) + .count(); + let first_optional = f.params.len() - optional; + f.params + .iter() + .enumerate() + .map(|(i, p)| { + let name = rb_param_name(&p.name); + if i >= first_optional { + format!("{name} = nil") + } else { + name + } + }) + .collect() } -fn render_deprecation(w: &mut CodeWriter, f: &FnBinding) { +/// Warn, pointing at the caller, when a deprecated callable is used (the +/// warning is silenced with `$VERBOSE = nil`, like any Ruby warning). +fn render_deprecation(w: &mut CodeWriter, f: &FnBinding, scope: &RbScope) { if let Some(msg) = &f.deprecated { - w.line(format!("warn '[DEPRECATED] {}'", rb_str_literal(msg))); + w.line(format!( + "warn('{} is deprecated: {}', uplevel: 1)", + scope.display(f), + rb_str_literal(msg) + )); } } @@ -480,21 +523,22 @@ fn render_deprecation(w: &mut CodeWriter, f: &FnBinding) { /// scope. fn render_sync( w: &mut CodeWriter, - module: &ModuleBinding, + ctx: &RbCtx, + error: Option<&ErrorBinding>, f: &FnBinding, symbol: &str, scope: &RbScope, ) { let q = scope.qualifier(); - let checker = rb_checker_name(f, module.error.as_ref()); + let checker = rb_checker_name(f, error); let has_out_len = matches!( - plan::ret_pass(f.ret.as_ref(), ""), + RetPass::of(f.ret.as_ref()), RetPass::String | RetPass::Bytes | RetPass::Buffer ); w.block(scope.def_open(f, &formals(f)), "end", |w| { - render_deprecation(w, f); - render_param_prep(w, f, &q); - with_pins(w, &q, &pinned_objects(f, scope), |w| { + render_deprecation(w, f, scope); + render_param_prep(w, ctx, f, &q); + with_pins(w, &q, "", &pinned_objects(f, scope), |w| { w.line("err = ErrorStruct.new"); let mut args = call_args(f, scope); if has_out_len { @@ -502,7 +546,7 @@ fn render_sync( args.push("out_len".into()); } args.push("err".into()); - render_handoff(w, f, &q); + render_handoff(w, ctx, f, &q); let call = format!("{q}{symbol}({})", args.join(", ")); if f.ret.is_some() { w.line(format!("result = {call}")); @@ -510,23 +554,25 @@ fn render_sync( w.line(call); } w.line(format!("{q}{checker}(err)")); - match scope { - RbScope::Init { .. } => { - w.line( - "raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if result.null?", - ); - w.line("_wv_init(result)"); + match scope.kind { + ScopeKind::Init => { + w.line(format!("_wv_init({q}_wv_nonnull(result))")); } - RbScope::Factory { .. } => { - w.line( - "raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if result.null?", - ); - w.line("_from_ptr(result)"); + ScopeKind::Factory => { + w.line(format!("_from_ptr({q}_wv_nonnull(result))")); } _ if f.ret.is_some() => { - receive_value(w, f.ret.as_ref(), "result", "out_len.read(:size_t)", &q, ""); + w.line(receive_value( + ctx, + f.ret.as_ref(), + "result", + "out_len.read(:size_t)", + &q, + )); + } + _ => { + w.line("nil"); } - _ => {} } }); }); @@ -541,49 +587,47 @@ fn render_sync( /// A cancellable function takes a `cancel:` keyword. Without one, the /// wrapper still passes a private token, so interrupting the wait /// (`Thread#raise`, `Timeout`) cancels the producer's work too. -fn render_async(w: &mut CodeWriter, f: &FnBinding, a: &AsyncBinding, scope: &RbScope) { +fn render_async(w: &mut CodeWriter, ctx: &RbCtx, f: &FnBinding, a: &AsyncBinding, scope: &RbScope) { let q = scope.qualifier(); let mut params = formals(f); if f.cancellable { params.push("cancel: nil".into()); } - w.line("# Blocks until the call completes on a producer thread."); - if f.cancellable { - w.line("# @param cancel [CancelToken, nil] cancels the call; it then raises Cancelled"); - } w.block(scope.def_open(f, ¶ms), "end", |w| { - render_deprecation(w, f); - render_param_prep(w, f, &q); + render_deprecation(w, f, scope); + render_param_prep(w, ctx, f, &q); if f.cancellable { w.line("own_token = CancelToken.new if cancel.nil?"); w.line("token = cancel || own_token"); } - let pins = pinned_objects(f, scope); - if !pins.is_empty() { - // Assigned inside the pin block, read after it. - w.line("queue = nil"); + let mut args = call_args(f, scope); + if f.cancellable { + args.push("token._wv_ptr".into()); } - with_pins(w, &q, &pins, |w| { - render_handoff(w, f, &q); + args.push(rb_async_const(a)); + args.push("ctx".into()); + let launch = format!("{q}{}({})", a.launch.symbol, args.join(", ")); + let pins = pinned_objects(f, scope); + if pins.is_empty() { + render_handoff(w, ctx, f, &q); w.line(format!("ctx, queue = {q}_wv_async_begin")); - let mut args = call_args(f, scope); - if f.cancellable { - args.push("token._wv_ptr".into()); - } - args.push(rb_async_const(a)); - args.push("ctx".into()); - w.line(format!("{q}{}({})", a.launch.symbol, args.join(", "))); - }); - if f.cancellable { - w.line(format!("{q}_wv_async_wait(queue, token)")); + w.line(launch); } else { - w.line(format!("{q}_wv_async_wait(queue)")); + with_pins(w, &q, "queue = ", &pins, |w| { + render_handoff(w, ctx, f, &q); + w.line(format!("ctx, pending = {q}_wv_async_begin")); + w.line(launch); + w.line("pending"); + }); } if f.cancellable { + w.line(format!("{q}_wv_async_wait(queue, token)")); w.dedent(); w.line("ensure"); w.indent(); w.line("own_token&.close"); + } else { + w.line(format!("{q}_wv_async_wait(queue)")); } }); } @@ -599,24 +643,22 @@ fn render_async(w: &mut CodeWriter, f: &FnBinding, a: &AsyncBinding, scope: &RbS /// error mid-iteration still releases the handle. fn render_iterator( w: &mut CodeWriter, - module: &ModuleBinding, + ctx: &RbCtx, + error: Option<&ErrorBinding>, f: &FnBinding, it: &IteratorBinding, scope: &RbScope, ) { let q = scope.qualifier(); - let checker = rb_checker_name(f, module.error.as_ref()); - let elem_pass = plan::ret_pass(Some(&it.elem), ""); + let checker = rb_checker_name(f, error); + let elem_pass = RetPass::of(Some(&it.elem)); let needs_len = matches!( elem_pass, RetPass::String | RetPass::Bytes | RetPass::Buffer ); - w.line("# Returns a lazy Enumerator that pulls one element per step. The"); - w.line("# producer iterator starts on the first pull and is released when"); - w.line("# iteration finishes or is abandoned early."); w.block(scope.def_open(f, &formals(f)), "end", |w| { - render_deprecation(w, f); - render_param_prep(w, f, &q); + render_deprecation(w, f, scope); + render_param_prep(w, ctx, f, &q); // The block closes over the converted arguments, so they stay // referenced until the launch call runs. w.block("Enumerator.new do |y|", "end", |w| { @@ -625,29 +667,13 @@ fn render_iterator( args.push("err".into()); let launch = format!("{q}{}({})", it.launch.symbol, args.join(", ")); let pins = pinned_objects(f, scope); - let mut handoff = CodeWriter::two_space(); - render_handoff(&mut handoff, f, &q); - let handoff = handoff.finish(); if pins.is_empty() { - w.block_raw(&handoff); + render_handoff(w, ctx, f, &q); w.line(format!("iter = {launch}")); - } else if handoff.is_empty() { - w.line(format!( - "iter = {q}_wv_pin({}) {{ |{}| {launch} }}", - pins.iter() - .map(|(e, _)| e.as_str()) - .collect::>() - .join(", "), - pins.iter() - .map(|(_, v)| v.as_str()) - .collect::>() - .join(", ") - )); } else { - w.line("iter = nil"); - with_pins(w, &q, &pins, |w| { - w.block_raw(&handoff); - w.line(format!("iter = {launch}")); + with_pins(w, &q, "iter = ", &pins, |w| { + render_handoff(w, ctx, f, &q); + w.line(launch); }); } w.line("begin"); @@ -677,14 +703,10 @@ fn render_iterator( _ => "out_item.read_pointer".to_string(), }; w.line(format!("item = {item}")); - receive_value( - w, - Some(&it.elem), - "item", - "out_len.read(:size_t)", - &q, - "value = ", - ); + w.line(format!( + "value = {}", + receive_value(ctx, Some(&it.elem), "item", "out_len.read(:size_t)", &q) + )); w.line("y << value"); }); }); diff --git a/crates/weaveffi-cli/src/targets/ruby/codec.rs b/crates/weaveffi-cli/src/targets/ruby/codec.rs new file mode 100644 index 00000000..66c87871 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/ruby/codec.rs @@ -0,0 +1,475 @@ +//! Value-buffer codecs: the expressions that pack and unpack one value, the +//! per-record and per-rich-enum codec pairs (`_wv_write_{stem}`, +//! `_wv_read_{stem}`), and one pair per distinct composite type (`[Entry?]`, +//! `{string:i64}`, `Store?`) used anywhere in the API, so no call site +//! inlines a loop. +//! +//! Every dispatch here goes through [`Ty::wire`], so this module never +//! re-derives the wire folds (records and rich enums as one user-codec +//! shape, C-style enums as `i32`, interfaces as `u64` object tokens). +//! +//! An object token carries one strong reference. Writing one reserves the +//! token (`write_object`); the runtime's `_wv_seal` mints the references +//! (the interface's `_clone` symbol) just before the call, so the wrapper +//! keeps its own reference and an abandoned encoding strands none. Reading +//! one (`read_object`) adopts the pointer into a fresh wrapper whose +//! finalizer owes the `_destroy`. + +use std::collections::{HashMap, HashSet}; + +use crate::codegen::CodeWriter; +use heck::ToSnakeCase; +use weaveffi_model::model::{ + CallShape, EnumBinding, FieldBinding, FnBinding, Model, StructBinding, +}; +use weaveffi_model::ty::{Family, Ty, WireType}; + +use crate::targets::ruby::types::rb_field_name; + +/// The indefinite article for `word` ("a" or "an"), by its first letter. +fn article(word: &str) -> &'static str { + match word.chars().next().map(|c| c.to_ascii_lowercase()) { + Some('a' | 'e' | 'i' | 'o' | 'u') => "an", + _ => "a", + } +} + +/// The longest constructor call kept on one line. +const MAX_LINE: usize = 100; + +/// The snake_case stem naming a record's or rich enum's codec pair: +/// `Contact` becomes `contact`, naming `_wv_write_contact` and +/// `_wv_read_contact`. +pub(crate) fn wv_stem(name: &str) -> String { + name.to_snake_case() +} + +/// The structural stem of a composite type before collision handling: +/// `[Entry?]` is `list_opt_entry`, `{string:i64}` is `map_string_i64`. +fn natural_stem(ty: &Ty) -> String { + match ty { + Ty::Prim(p) => p.snake().to_string(), + Ty::Record(n) | Ty::RichEnum(n) | Ty::Enum(n) | Ty::Interface(n) => wv_stem(n), + Ty::Optional(inner) => format!("opt_{}", natural_stem(inner)), + Ty::List(inner) => format!("list_{}", natural_stem(inner)), + Ty::Map(k, v) => format!("map_{}_{}", natural_stem(k), natural_stem(v)), + Ty::CallbackInterface(_) | Ty::Iterator(_) => { + unreachable!("{ty} never appears inside a value buffer") + } + } +} + +/// The records and rich enums of `model` whose fields can hold an object, +/// directly or through another record or rich enum (a fixed point, so +/// recursive types terminate). +fn user_types_with_objects(model: &Model) -> HashSet { + let mut fields: Vec<(&str, Vec<&Ty>)> = Vec::new(); + for m in &model.modules { + for s in &m.structs { + fields.push((&s.name, s.fields.iter().map(|f| &f.ty).collect())); + } + for e in m.enums.iter().filter(|e| e.is_rich()) { + let tys = e.variants.iter().flat_map(|v| &v.fields).map(|f| &f.ty); + fields.push((&e.name, tys.collect())); + } + } + let mut found: HashSet = HashSet::new(); + loop { + let before = found.len(); + for (name, tys) in &fields { + if found.contains(*name) { + continue; + } + let carries = tys.iter().any(|ty| { + ty.any(&|t| match t { + Ty::Interface(_) => true, + Ty::Record(n) | Ty::RichEnum(n) => found.contains(n), + _ => false, + }) + }); + if carries { + found.insert((*name).to_string()); + } + } + if found.len() == before { + return found; + } + } +} + +/// The composite types (optionals, lists, and maps inside value buffers) an +/// API uses, each with the stem naming its codec pair. +#[derive(Debug, Default)] +pub(crate) struct Codecs { + order: Vec, + stems: HashMap, + /// The records and rich enums whose encoding can hold an object token. + with_objects: HashSet, +} + +impl Codecs { + /// Collect every composite type `model` encodes or decodes: in + /// parameters, returns, async results, iterator elements, record + /// fields, variant fields, error payload fields, and callback + /// parameters and returns. + pub(crate) fn collect(model: &Model) -> Self { + let mut taken: HashSet = HashSet::new(); + for m in &model.modules { + taken.extend(m.structs.iter().map(|s| wv_stem(&s.name))); + taken.extend( + m.enums + .iter() + .filter(|e| e.is_rich()) + .map(|e| wv_stem(&e.name)), + ); + } + let mut codecs = Codecs { + with_objects: user_types_with_objects(model), + ..Codecs::default() + }; + let mut add = |ty: &Ty, codecs: &mut Codecs| codecs.visit(ty, &mut taken); + for m in &model.modules { + for f in m.callables() { + for p in &f.params { + codecs.position(&p.ty, &mut add); + } + codecs.result(f, &mut add); + } + for s in &m.structs { + for f in &s.fields { + add(&f.ty, &mut codecs); + } + } + for e in &m.enums { + for f in e.variants.iter().flat_map(|v| &v.fields) { + add(&f.ty, &mut codecs); + } + } + if let Some(eb) = &m.errors { + for f in eb.codes.iter().flat_map(|c| &c.fields) { + add(&f.ty, &mut codecs); + } + } + for cb in &m.callback_interfaces { + for cm in &cb.methods { + for p in &cm.params { + codecs.position(&p.ty, &mut add); + } + if let Some(ret) = &cm.ret { + codecs.position(ret, &mut add); + } + } + } + } + codecs + } + + /// Visit a parameter or return type: only a buffered one is encoded + /// (`Store?` crosses as a pointer, not a buffer). + fn position(&mut self, ty: &Ty, add: &mut impl FnMut(&Ty, &mut Codecs)) { + if ty.family() == Family::Buffer { + add(ty, self); + } + } + + /// Visit a callable's result: the return of a sync or async call, or + /// the element of an iterator. + fn result(&mut self, f: &FnBinding, add: &mut impl FnMut(&Ty, &mut Codecs)) { + match &f.shape { + CallShape::Iterator(it) => self.position(&it.elem, add), + _ => { + if let Some(ret) = &f.ret { + self.position(ret, add); + } + } + } + } + + /// Register `ty` (when it's a composite) and every composite inside it. + fn visit(&mut self, ty: &Ty, taken: &mut HashSet) { + match ty.wire() { + WireType::Optional(inner) | WireType::List(inner) => { + self.register(ty, taken); + self.visit(inner, taken); + } + WireType::Map(k, v) => { + self.register(ty, taken); + self.visit(k, taken); + self.visit(v, taken); + } + _ => {} + } + } + + fn register(&mut self, ty: &Ty, taken: &mut HashSet) { + if self.stems.contains_key(ty) { + return; + } + let natural = natural_stem(ty); + let mut stem = natural.clone(); + let mut n = 2; + while !taken.insert(stem.clone()) { + stem = format!("{natural}_{n}"); + n += 1; + } + self.order.push(ty.clone()); + self.stems.insert(ty.clone(), stem); + } + + /// Whether an encoding of `ty` can hold an object token (directly, or + /// inside a record or rich enum), so its references must be minted by + /// `_wv_seal` right before the encoding is handed over. + pub(crate) fn carries_objects(&self, ty: &Ty) -> bool { + ty.any(&|t| match t { + Ty::Interface(_) => true, + Ty::Record(n) | Ty::RichEnum(n) => self.with_objects.contains(n), + _ => false, + }) + } + + fn stem(&self, ty: &Ty) -> &str { + self.stems + .get(ty) + .unwrap_or_else(|| panic!("no codec registered for {ty}")) + } + + /// The statement appending `expr` (a value of type `ty`) to the buffer + /// writer `w`. `q` is the receiver (`"Kvstore."` or `""`) qualifying the + /// module's codec methods inside class bodies. + pub(crate) fn write(&self, ty: &Ty, w: &str, expr: &str, q: &str) -> String { + match ty.wire() { + WireType::Prim(p) => format!("{w}.write_{}({expr})", p.snake()), + WireType::Enum(_) => format!("{w}.write_i32({expr})"), + WireType::Object(n) => format!("{w}.write_object({expr}, {n})"), + WireType::User(n) => format!("{q}_wv_write_{}({w}, {expr})", wv_stem(n)), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{q}_wv_write_{}({w}, {expr})", self.stem(ty)) + } + } + } + + /// The expression reading one `ty` value from the buffer reader `r`. + pub(crate) fn read(&self, ty: &Ty, r: &str, q: &str) -> String { + match ty.wire() { + WireType::Prim(p) => format!("{r}.read_{}", p.snake()), + WireType::Enum(_) => format!("{r}.read_i32"), + WireType::Object(n) => format!("{r}.read_object({n})"), + WireType::User(n) => format!("{q}_wv_read_{}({r})", wv_stem(n)), + WireType::Optional(_) | WireType::List(_) | WireType::Map(..) => { + format!("{q}_wv_read_{}({r})", self.stem(ty)) + } + } + } + + /// Render the codec pair of every registered composite type. + pub(crate) fn render(&self, w: &mut CodeWriter) { + if self.order.is_empty() { + return; + } + w.blank(); + w.line("# === Value-buffer codecs of the composite types ==="); + for ty in &self.order { + let stem = self.stem(ty); + w.blank(); + w.line("# @api private"); + w.line(format!("# Packs `{ty}` into the value-buffer wire format.")); + w.block( + format!("def self._wv_write_{stem}(w, v)"), + "end", + |w| match ty.wire() { + WireType::Optional(inner) => { + w.line("if v.nil?"); + w.scope(|w| { + w.line("w.write_flag(false)"); + }); + w.line("else"); + w.scope(|w| { + w.line("w.write_flag(true)"); + w.line(self.write(inner, "w", "v", "")); + }); + w.line("end"); + } + WireType::List(elem) => { + w.line("w.write_len(v.length)"); + w.line(format!( + "v.each {{ |e| {} }}", + self.write(elem, "w", "e", "") + )); + } + WireType::Map(k, val) => { + w.line("w.write_len(v.length)"); + w.block("v.each do |k, e|", "end", |w| { + w.line(self.write(k, "w", "k", "")); + w.line(self.write(val, "w", "e", "")); + }); + } + _ => unreachable!("only composites are registered"), + }, + ); + w.blank(); + w.line("# @api private"); + w.line(format!( + "# Unpacks `{ty}` from the value-buffer wire format." + )); + w.block( + format!("def self._wv_read_{stem}(r)"), + "end", + |w| match ty.wire() { + WireType::Optional(inner) => { + w.line(format!("r.read_flag ? {} : nil", self.read(inner, "r", ""))); + } + WireType::List(elem) => { + w.line(format!( + "Array.new(r.read_len) {{ {} }}", + self.read(elem, "r", "") + )); + } + WireType::Map(k, val) => { + w.line(format!( + "Array.new(r.read_len) {{ [{}, {}] }}.to_h", + self.read(k, "r", ""), + self.read(val, "r", "") + )); + } + _ => unreachable!("only composites are registered"), + }, + ); + } + } + + /// Emit `{lead}{class}.new(...)` with one keyword argument per field, + /// each read from `r` in wire order (Ruby evaluates arguments left to + /// right). `positional` comes first when set (an error's message). + pub(crate) fn emit_new( + &self, + w: &mut CodeWriter, + lead: &str, + class: &str, + positional: Option<&str>, + fields: &[FieldBinding], + q: &str, + ) { + let mut args: Vec = positional.map(str::to_string).into_iter().collect(); + args.extend( + fields + .iter() + .map(|f| format!("{}: {}", rb_field_name(&f.name), self.read(&f.ty, "r", q))), + ); + let one_line = format!("{lead}{class}.new({})", args.join(", ")); + match args.as_slice() { + [] => { + w.line(format!("{lead}{class}.new")); + } + _ if w.indent_str().len() + one_line.len() <= MAX_LINE => { + w.line(one_line); + } + many => { + w.line(format!("{lead}{class}.new(")); + w.scope(|w| { + for a in many { + w.line(format!("{a},")); + } + }); + w.line(")"); + } + } + } + + /// Render the private codec pair of one record: module singleton methods + /// `_wv_write_{stem}(w, v)` and `_wv_read_{stem}(r)` serializing the + /// fields in declaration (wire) order. Writing is duck-typed: any object + /// with the field readers packs. + pub(crate) fn render_struct(&self, w: &mut CodeWriter, s: &StructBinding) { + let stem = wv_stem(&s.name); + w.blank(); + w.line("# @api private"); + w.line(format!( + "# Packs {} {} into the value-buffer wire format.", + article(&s.name), + s.name + )); + let writer = if s.fields.is_empty() { + "_w, _v" + } else { + "w, v" + }; + w.block(format!("def self._wv_write_{stem}({writer})"), "end", |w| { + for f in &s.fields { + let field = rb_field_name(&f.name); + w.line(self.write(&f.ty, "w", &format!("v.{field}"), "")); + } + }); + w.blank(); + w.line("# @api private"); + w.line(format!( + "# Unpacks {} {} from the value-buffer wire format.", + article(&s.name), + s.name + )); + let reader = if s.fields.is_empty() { "_r" } else { "r" }; + w.block(format!("def self._wv_read_{stem}({reader})"), "end", |w| { + self.emit_new(w, "", &s.name, None, &s.fields, ""); + }); + } + + /// Render the private codec pair of one rich enum: `_wv_write_{stem}` + /// dispatches on the variant class and writes the `i32` tag followed by + /// the variant's fields; `_wv_read_{stem}` switches on the decoded tag. + pub(crate) fn render_rich_enum(&self, w: &mut CodeWriter, e: &EnumBinding) { + let stem = wv_stem(&e.name); + w.blank(); + w.line("# @api private"); + w.line(format!( + "# Packs {} {} into the value-buffer wire format.", + article(&e.name), + e.name + )); + w.block(format!("def self._wv_write_{stem}(w, v)"), "end", |w| { + w.line("case v"); + for v in &e.variants { + w.line(format!("when {}::{}", e.name, v.name)); + w.scope(|w| { + w.line(format!("w.write_i32({})", v.value)); + for f in &v.fields { + let field = rb_field_name(&f.name); + w.line(self.write(&f.ty, "w", &format!("v.{field}"), "")); + } + }); + } + w.line("else"); + w.scope(|w| { + w.line(format!( + "raise TypeError, \"expected a {}, got #{{v.class}}\"", + e.name + )); + }); + w.line("end"); + }); + w.blank(); + w.line("# @api private"); + w.line(format!( + "# Unpacks {} {} from the value-buffer wire format.", + article(&e.name), + e.name + )); + w.block(format!("def self._wv_read_{stem}(r)"), "end", |w| { + w.line("tag = r.read_i32"); + w.line("case tag"); + for v in &e.variants { + w.line(format!("when {}", v.value)); + w.scope(|w| { + let class = format!("{}::{}", e.name, v.name); + self.emit_new(w, "", &class, None, &v.fields, ""); + }); + } + w.line("else"); + w.scope(|w| { + w.line(format!( + "raise _wv_malformed(\"unknown {} tag #{{tag}}\")", + e.name + )); + }); + w.line("end"); + }); + } +} diff --git a/crates/weaveffi-cli/src/targets/ruby/docs.rs b/crates/weaveffi-cli/src/targets/ruby/docs.rs new file mode 100644 index 00000000..a4380ba2 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/ruby/docs.rs @@ -0,0 +1,41 @@ +//! Doc-comment emission: YARD `@param` and `@return` tags for IDL +//! parameters and returns. + +use crate::codegen::CodeWriter; +use weaveffi_model::model::ParamBinding; +use weaveffi_model::ty::Ty; + +use crate::targets::ruby::types::{rb_doc_type, rb_param_name}; + +/// Emit one `# @param name [Type] doc` tag per documented parameter, naming +/// each parameter by its emitted Ruby spelling. Continuation lines indent +/// under the tag; blank doc lines become `#`. +pub(crate) fn emit_param_docs(w: &mut CodeWriter, params: &[ParamBinding]) { + for p in params { + let Some(doc) = p.doc.as_deref().map(str::trim).filter(|d| !d.is_empty()) else { + continue; + }; + let mut lines = doc.lines(); + if let Some(first) = lines.next() { + w.line(format!( + "# @param {} [{}] {first}", + rb_param_name(&p.name), + rb_doc_type(&p.ty) + )); + } + for line in lines { + if line.is_empty() { + w.line("#"); + } else { + w.line(format!("# {line}")); + } + } + } +} + +/// Emit the `# @return [Type]` tag of a return value. +pub(crate) fn emit_return_doc(w: &mut CodeWriter, ret: Option<&Ty>) { + if let Some(ty) = ret { + w.line(format!("# @return [{}]", rb_doc_type(ty))); + } +} diff --git a/crates/weaveffi-cli/src/targets/ruby/entities.rs b/crates/weaveffi-cli/src/targets/ruby/entities.rs new file mode 100644 index 00000000..ce294e31 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/ruby/entities.rs @@ -0,0 +1,431 @@ +//! Entity renderers: plain enums, records, rich enums, interfaces, and the +//! typed error surface of a module's declared error domain. + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use heck::{ToShoutySnakeCase, ToSnakeCase}; +use weaveffi_model::errors::pascal; +use weaveffi_model::model::{ + EnumBinding, ErrorBinding, FieldBinding, FnBinding, InterfaceBinding, ModuleBinding, + StructBinding, +}; +use weaveffi_model::plan::ErrorStrategy; + +use crate::targets::ruby::calls::{render_attach_function, render_callable, RbScope, ScopeKind}; +use crate::targets::ruby::types::{rb_field_name, rb_str_literal}; +use crate::targets::ruby::RbCtx; + +/// The snake_case stem of a domain's generated helpers: `KvError` becomes +/// `kv_error`, naming `_wv_kv_error_from`, `_wv_check_kv_error!`, and +/// `_wv_kv_error_payload`. Domain type names are globally unique +/// (validated), so the helpers can't collide. +fn rb_error_stem(eb: &ErrorBinding) -> String { + eb.type_name.to_snake_case() +} + +/// `_wv_{stem}_from`: builds the domain error matching an ABI code. +pub(crate) fn rb_error_factory_name(eb: &ErrorBinding) -> String { + format!("_wv_{}_from", rb_error_stem(eb)) +} + +/// `_wv_check_{stem}!`: raises the typed domain error for a non-zero +/// out-err slot. +fn rb_error_checker_name(eb: &ErrorBinding) -> String { + format!("_wv_check_{}!", rb_error_stem(eb)) +} + +/// `_wv_{stem}_payload`: the code and encoded payload of a domain error a +/// callback implementation raised. +pub(crate) fn rb_error_payload_name(eb: &ErrorBinding) -> String { + format!("_wv_{}_payload", rb_error_stem(eb)) +} + +/// The error-check call a callable's out-err slot goes through, per the +/// function's [`ErrorStrategy`]: the module domain's typed checker for +/// [`ErrorStrategy::Throws`], the runtime's trap (`_wv_check!`, raising +/// `NativeBugError`) for [`ErrorStrategy::Trap`]. +pub(crate) fn rb_checker_name(f: &FnBinding, error: Option<&ErrorBinding>) -> String { + match (f.error_strategy(), error) { + (ErrorStrategy::Throws, Some(eb)) => rb_error_checker_name(eb), + _ => "_wv_check!".to_string(), + } +} + +/// Emit `attr_reader` lines for `fields`, each with its doc comment, +/// separated by blank lines. +fn emit_readers(w: &mut CodeWriter, fields: &[FieldBinding]) { + for (idx, f) in fields.iter().enumerate() { + if idx > 0 { + w.blank(); + } + w.doc(&f.doc, DocCommentStyle::Hash); + w.line(format!("attr_reader :{}", rb_field_name(&f.name))); + } +} + +/// Render one module's declared error domain: a domain class subclassing +/// the root `Error`, one nested subclass per code carrying its stable +/// `CODE` constant, default message, and any declared payload fields as +/// attributes, then the private helpers: the factory and checker throwing +/// wrappers route their out-err slots through, and (when a callback method +/// may raise the domain, `payload`) the encoder reporting a raised domain +/// error back to the producer. Nesting the code classes keeps +/// `KvError::KeyNotFound` spellable and unambiguous even across domains. +/// +/// Domain codes are validated positive-only; the negative range is reserved +/// for the runtime. The factory therefore maps only declared codes onto +/// typed classes and lets everything else fall through to `_wv_error` (the +/// root `Error`, or `Cancelled`). +pub(crate) fn render_error( + w: &mut CodeWriter, + ctx: &RbCtx, + module: &ModuleBinding, + eb: &ErrorBinding, + payload: bool, +) { + let domain = &eb.type_name; + w.blank(); + w.line(format!( + "# Base error for the `{}` module's error domain.", + module.dot_path + )); + w.block(format!("class {domain} < Error"), "end", |w| { + for (idx, c) in eb.codes.iter().enumerate() { + if idx > 0 { + w.blank(); + } + let doc = c.doc.clone().unwrap_or_else(|| c.message.clone()); + w.doc(&Some(doc), DocCommentStyle::Hash); + w.block( + format!("class {} < {domain}", pascal(&c.name)), + "end", + |w| { + w.line(format!("CODE = {}", c.value)); + if !c.fields.is_empty() { + w.blank(); + emit_readers(w, &c.fields); + } + w.blank(); + let kw: String = c + .fields + .iter() + .map(|f| format!(", {}: nil", rb_field_name(&f.name))) + .collect(); + w.block(format!("def initialize(message = nil{kw})"), "end", |w| { + for f in &c.fields { + let field = rb_field_name(&f.name); + w.line(format!("@{field} = {field}")); + } + w.line(format!( + "super(CODE, message || '{}')", + rb_str_literal(&c.message) + )); + }); + }, + ); + } + }); + + w.blank(); + w.line("# @api private"); + w.line(format!( + "# The {domain} for a domain `code`, with its payload fields decoded;" + )); + w.line("# the root Error (or Cancelled) for any other code."); + w.block( + format!( + "def self.{}(code, message, payload = nil)", + rb_error_factory_name(eb) + ), + "end", + |w| { + w.line("message = nil if message.empty?"); + w.line("r = WvBufferReader.new(payload)"); + w.line("error ="); + w.scope(|w| { + w.line("case code"); + for c in &eb.codes { + w.line(format!("when {}", c.value)); + w.scope(|w| { + let class = format!("{domain}::{}", pascal(&c.name)); + ctx.codecs + .emit_new(w, "", &class, Some("message"), &c.fields, ""); + }); + } + w.line("else"); + w.scope(|w| { + w.line("return _wv_error(code, message.to_s)"); + }); + w.line("end"); + }); + w.line("r.expect_end!"); + w.line("error"); + }, + ); + + w.blank(); + w.line("# @api private"); + w.line(format!("# Raises the {domain} for a non-zero error slot.")); + w.block( + format!("def self.{}(err)", rb_error_checker_name(eb)), + "end", + |w| { + w.line("taken = _wv_take_error(err)"); + w.line(format!( + "raise {}(*taken) unless taken.nil?", + rb_error_factory_name(eb) + )); + }, + ); + + if payload { + w.blank(); + w.line("# @api private"); + w.line(format!( + "# The code and value-buffer payload of a {domain} a callback" + )); + w.line("# implementation raised, or nil for an error of no declared code."); + w.block( + format!("def self.{}(error)", rb_error_payload_name(eb)), + "end", + |w| { + w.line("w = WvBufferWriter.new"); + w.line("code ="); + w.scope(|w| { + w.line("case error"); + for c in &eb.codes { + let class = format!("{domain}::{}", pascal(&c.name)); + if c.fields.is_empty() { + w.line(format!("when {class} then {}", c.value)); + continue; + } + w.line(format!("when {class}")); + w.scope(|w| { + for f in &c.fields { + let field = rb_field_name(&f.name); + w.line(ctx.codecs.write(&f.ty, "w", &format!("error.{field}"), "")); + } + w.line(c.value.to_string()); + }); + } + w.line("end"); + }); + w.line("code && [code, w.bytes]"); + }, + ); + } +} + +/// Render one plain C-style enum as a module of integer constants, one +/// `SHOUTY_SNAKE` constant per variant. +pub(crate) fn render_enum(w: &mut CodeWriter, e: &EnumBinding) { + w.blank(); + w.doc(&e.doc, DocCommentStyle::Hash); + if let Some(msg) = &e.deprecated { + w.line(format!("# @deprecated {msg}")); + } + w.block(format!("module {}", e.name), "end", |w| { + for v in &e.variants { + w.doc(&v.doc, DocCommentStyle::Hash); + w.line(format!("{} = {}", v.name.to_shouty_snake_case(), v.value)); + } + }); +} + +/// Emit a value class body: documented `attr_reader`s, a keyword-argument +/// `initialize`, and structural `==` over `class`'s fields. +fn emit_value_class(w: &mut CodeWriter, class: &str, fields: &[FieldBinding]) { + if !fields.is_empty() { + emit_readers(w, fields); + w.blank(); + let kw: Vec = fields + .iter() + .map(|f| format!("{}:", rb_field_name(&f.name))) + .collect(); + let one_line = format!("def initialize({})", kw.join(", ")); + if w.indent_str().len() + one_line.len() <= 100 { + w.line(one_line); + } else { + w.line("def initialize("); + w.scope(|w| { + for (i, k) in kw.iter().enumerate() { + let sep = if i + 1 == kw.len() { "" } else { "," }; + w.line(format!("{k}{sep}")); + } + }); + w.line(")"); + } + w.scope(|w| { + for f in fields { + let field = rb_field_name(&f.name); + w.line(format!("@{field} = {field}")); + } + }); + w.line("end"); + w.blank(); + } + w.line("# Structural equality over every field."); + w.block("def ==(other)", "end", |w| { + let mut terms = vec![format!("other.is_a?({class})")]; + terms.extend(fields.iter().map(|f| { + let field = rb_field_name(&f.name); + format!("{field} == other.{field}") + })); + match terms.as_slice() { + [one] => { + w.line(one.clone()); + } + [first, rest @ ..] => { + w.line(format!("{first} &&")); + w.scope(|w| { + for (i, t) in rest.iter().enumerate() { + if i + 1 == rest.len() { + w.line(t.clone()); + } else { + w.line(format!("{t} &&")); + } + } + }); + } + [] => unreachable!("the class test is always present"), + } + }); +} + +/// Render one record as a plain Ruby value class: one documented +/// `attr_reader` per field, a keyword-argument `initialize`, and structural +/// `==`. Records are value types: they own no C symbols; they cross the ABI +/// packed into value buffers by the module's `_wv_write_*`/`_wv_read_*` +/// codec helpers. +pub(crate) fn render_struct_class(w: &mut CodeWriter, s: &StructBinding) { + w.blank(); + w.doc(&s.doc, DocCommentStyle::Hash); + if let Some(msg) = &s.deprecated { + w.line(format!("# @deprecated {msg}")); + } + w.block(format!("class {}", s.name), "end", |w| { + emit_value_class(w, &s.name, &s.fields); + }); +} + +/// Render one rich (algebraic) enum as a tagged class hierarchy: a base +/// class exposing `tag`, plus one nested value class per variant carrying +/// that variant's fields. Rich enums own no C symbols; they cross the ABI +/// packed into value buffers as an `i32` tag followed by the active +/// variant's fields in declaration order. +pub(crate) fn render_rich_enum_class(w: &mut CodeWriter, e: &EnumBinding) { + w.blank(); + w.doc(&e.doc, DocCommentStyle::Hash); + if let Some(msg) = &e.deprecated { + w.line(format!("# @deprecated {msg}")); + } + w.block(format!("class {}", e.name), "end", |w| { + w.line("# The active variant's integer tag."); + w.block("def tag", "end", |w| { + w.line("self.class::TAG"); + }); + for v in &e.variants { + w.blank(); + w.doc(&v.doc, DocCommentStyle::Hash); + w.block(format!("class {} < {}", v.name, e.name), "end", |w| { + w.line(format!("TAG = {}", v.value)); + w.blank(); + emit_value_class(w, &v.name, &v.fields); + }); + } + }); +} + +/// Declare the FFI bindings for one interface: the clone and destroy +/// lifecycle symbols (which keep the GVL; see [`render_attach_function`]) +/// plus every constructor, method, and static. +pub(crate) fn render_interface_ffi(w: &mut CodeWriter, i: &InterfaceBinding) { + w.line(format!( + "attach_function :{}, [:pointer], :pointer", + i.clone_symbol + )); + w.line(format!( + "attach_function :{}, [:pointer], :void", + i.destroy_symbol + )); + for f in i + .constructors + .iter() + .chain(i.methods.iter()) + .chain(i.statics.iter()) + { + render_attach_function(w, f); + } +} + +/// Render one interface as a reference-counted wrapper class on the +/// runtime's `WvObject` base (which supplies `handle`, `close`, `closed?`, +/// `dup`/`clone`, and the call pinning). A `{Name}Ptr < FFI::AutoPointer` +/// subclass owns exactly one strong reference and releases it through the +/// interface's `_destroy` symbol, either from `close` or, as a backstop, +/// from the GC finalizer. A constructor named `new` becomes `initialize`; +/// every other constructor becomes a class-method factory; methods borrow +/// the wrapper's pointer as the leading C argument; statics are class +/// methods. `_from_ptr` adopts a reference the producer handed over (a +/// return, an async result, an iterator element, a buffer token, a callback +/// argument) without re-running `initialize`. +pub(crate) fn render_interface_class( + w: &mut CodeWriter, + ctx: &RbCtx, + error: Option<&ErrorBinding>, + i: &InterfaceBinding, +) { + let ptr_class = format!("{}Ptr", i.name); + let module = ctx.module; + w.blank(); + w.line("# @api private"); + w.line(format!( + "# Owns one strong reference to a {}; releases it exactly once.", + i.name + )); + w.block( + format!("class {ptr_class} < FFI::AutoPointer"), + "end", + |w| { + w.block("def self.release(ptr)", "end", |w| { + w.line(format!("{module}.{}(ptr)", i.destroy_symbol)); + }); + }, + ); + w.blank(); + w.doc(&i.doc, DocCommentStyle::Hash); + if let Some(msg) = &i.deprecated { + w.line(format!("# @deprecated {msg}")); + } + w.block(format!("class {} < WvObject", i.name), "end", |w| { + w.line(format!("WV_PTR = {ptr_class}")); + if !i.constructors.iter().any(|c| c.name == "new") { + w.line("private_class_method :new"); + } + w.blank(); + w.line("# @api private"); + w.block("def self._wv_clone(ptr)", "end", |w| { + w.line(format!("{module}.{}(ptr)", i.clone_symbol)); + }); + // Members render at class depth through the shared callable paths, + // so sync, async, and iterator members reuse the free-function + // marshalling. + for c in &i.constructors { + let kind = if c.name == "new" { + ScopeKind::Init + } else { + ScopeKind::Factory + }; + render_callable(w, ctx, error, c, &RbScope::member(kind, module, &i.name)); + } + for f in i.methods.iter().chain(i.statics.iter()) { + let kind = if f.has_self { + ScopeKind::Method + } else { + ScopeKind::Static + }; + let scope = RbScope::member(kind, module, &i.name); + render_callable(w, ctx, error, f, &scope); + } + }); +} diff --git a/crates/weaveffi-cli/src/targets/ruby/mod.rs b/crates/weaveffi-cli/src/targets/ruby/mod.rs new file mode 100644 index 00000000..521daad5 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/ruby/mod.rs @@ -0,0 +1,305 @@ +//! Ruby (ffi gem) binding generator. +//! +//! Emits a gem that binds the C ABI (revision 4) with the `ffi` gem: +//! `lib/{prefix}.rb` (the generated bindings, required as `{prefix}`), +//! `lib/{prefix}/runtime.rb` (the fixed runtime from `runtime/runtime.rb`), +//! a `{name}.gemspec`, and a README. Everything lives in one Ruby module, +//! `PascalCase(name)` unless configured. Loading checks the ABI revision and +//! every top-level module's contract table. Interfaces become +//! reference-counted wrapper classes (`close` plus a GC finalizer backstop, +//! `dup`/`clone` for a second reference), records and rich enums become +//! value classes packed into value buffers (one codec pair per record, rich +//! enum, and composite type), async functions block on a queue fed by a +//! module-level completion trampoline (cancellable ones take a `cancel:` +//! token), iterators are lazy `Enumerator`s, and callback interfaces are +//! duck-typed modules backed by one static vtable of pinned `FFI::Function` +//! trampolines per interface. Every call except the trivial runtime helpers +//! releases the GVL. + +mod callbacks; +mod calls; +mod codec; +mod docs; +mod entities; +mod package; +mod runtime; +mod types; + +#[cfg(test)] +mod tests; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::codegen::CodeWriter; +use crate::package::{Artifact, ArtifactKind, GemSpec, PackageContext, PackagedFile}; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; +use camino::Utf8Path; +use serde::{Deserialize, Serialize}; +use weaveffi_model::contract; +use weaveffi_model::model::{contract_symbol, ErrorBinding, Model}; + +use crate::targets::ruby::calls::{ + render_async_trampoline, render_attach_function, render_callable, RbScope, +}; +use crate::targets::ruby::codec::Codecs; +use crate::targets::ruby::entities::{ + render_enum, render_error, render_interface_class, render_interface_ffi, + render_rich_enum_class, render_struct_class, +}; +use crate::targets::ruby::package::{ + gem_authors, render_gemspec, render_packaged_readme, render_readme, GemNames, FFI_REQUIREMENT, + REQUIRED_RUBY_VERSION, +}; +use crate::targets::ruby::runtime::render_runtime; +use crate::targets::ruby::types::rb_str_literal; + +/// Per-target configuration for [`RubyGenerator`]. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct RubyConfig { + /// Gem name written into the gemspec (default: the package name). + pub name: Option, + /// Top-level Ruby module name (default: the package name in PascalCase). + pub module_name: Option, +} + +impl RubyConfig { + /// The gem, module, and require names for `api`: the identity's, unless + /// overridden here. + fn names<'a>(&self, model: &'a Model) -> GemNames<'a> { + let identity = &model.identity; + let configured = |v: &Option| { + v.as_deref() + .map(str::trim) + .filter(|s| !s.is_empty()) + .map(str::to_string) + }; + GemNames { + identity, + gem: configured(&self.name).unwrap_or_else(|| identity.name.clone()), + module: configured(&self.module_name).unwrap_or_else(|| identity.pascal_name()), + } + } +} + +/// Ruby backend: emits an `ffi`-gem package binding the C ABI exposed by the +/// underlying cdylib. +pub struct RubyGenerator; + +impl LanguageBackend for RubyGenerator { + type Config = RubyConfig; + + fn name(&self) -> &'static str { + "ruby" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + let names = config.names(model); + let dir = out_dir.join("ruby"); + let lib_dir = dir.join("lib"); + let lib_file = format!("{}.rb", names.require()); + vec![ + OutputFile::new( + lib_dir.join(&lib_file), + render_bindings(model, &names, &lib_file), + ), + OutputFile::new( + lib_dir.join(names.require()).join("runtime.rb"), + render_runtime(names.identity, &names.module, "runtime.rb"), + ), + OutputFile::new(dir.join(names.gemspec_file()), render_gemspec(&names)), + OutputFile::new(dir.join("README.md"), render_readme(&names)), + ] + } + + /// One precompiled platform gem per desktop platform, with the library + /// under `lib/native/` where the runtime loads it first. Platforms + /// RubyGems has no platform string for (Android, iOS, `wasm32`) are + /// skipped. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let names = config.names(model); + let id = names.identity; + let lib_file = format!("{}.rb", names.require()); + let bindings = render_bindings(model, &names, &lib_file); + let runtime = render_runtime(id, &names.module, "runtime.rb"); + let readme = render_packaged_readme(&names); + let mut artifacts = Vec::new(); + for nb in &ctx.binaries.binaries { + let Some(ruby_platform) = nb.platform.ruby_platform() else { + continue; + }; + let files = vec![ + PackagedFile::text(format!("lib/{lib_file}"), bindings.clone()), + PackagedFile::text( + format!("lib/{}/runtime.rb", names.require()), + runtime.clone(), + ), + PackagedFile::copy( + format!("lib/native/{}", nb.platform.lib_filename(&id.library)), + nb.library.clone(), + ), + PackagedFile::text("README.md", readme.clone()), + ]; + let spec = GemSpec { + name: names.gem.clone(), + version: id.version.clone(), + platform: ruby_platform.to_string(), + summary: id.description_or_default(), + authors: gem_authors(&names), + license: id.license.clone(), + homepage: id.homepage.clone().or_else(|| id.repository.clone()), + required_ruby_version: REQUIRED_RUBY_VERSION.to_string(), + dependencies: vec![("ffi".to_string(), FFI_REQUIREMENT.to_string())], + }; + artifacts.push(Artifact { + path: format!("ruby/{}", spec.file_name()).into(), + kind: ArtifactKind::Gem(spec), + files, + }); + } + Some(artifacts) + } +} + +/// What every renderer of one bindings file shares: the model, the Ruby +/// module name, and the composite codecs the API uses. +pub(crate) struct RbCtx<'a> { + /// The validated model being rendered. + pub(crate) model: &'a Model, + /// The top-level Ruby module name, which qualifies module singleton + /// calls from inside class bodies. + pub(crate) module: &'a str, + /// The codec pairs of the API's composite types. + pub(crate) codecs: Codecs, +} + +impl RbCtx<'_> { + /// Whether a callback method declared `throws` can raise `eb`, so the + /// bindings need its payload encoder. + fn raised_by_callbacks(&self, eb: &ErrorBinding) -> bool { + self.model.modules.iter().any(|m| { + m.callback_interfaces + .iter() + .any(|cb| cb.methods.iter().any(|cm| cm.throws)) + && self + .model + .error_domain(m) + .is_some_and(|d| d.type_name == eb.type_name) + }) + } +} + +/// Render `lib/{prefix}.rb`: the runtime require, the contract check, then +/// each module's typed error surface, entities, codecs, FFI attachments, +/// callback interfaces, interface classes, and free functions, and finally +/// the composite codecs. +fn render_bindings(model: &Model, names: &GemNames, lib_file: &str) -> String { + let ctx = RbCtx { + model, + module: &names.module, + codecs: Codecs::collect(model), + }; + let mut w = CodeWriter::two_space(); + w.raw(render_prelude(CommentStyle::Hash)); + w.line("# frozen_string_literal: true"); + w.blank(); + w.line(format!("require_relative '{}/runtime'", names.require())); + w.blank(); + w.line(format!("# Ruby bindings for {}.", names.identity.name)); + w.block(format!("module {}", names.module), "end", |w| { + render_contract(w, model); + for m in &model.modules { + let error = model.error_domain(m); + w.blank(); + w.line(format!("# === Module: {} ===", m.dot_path)); + // The typed error surface comes first so the domain class exists + // before any wrapper or trampoline references it. + if let Some(eb) = m.errors.as_ref() { + render_error(w, &ctx, m, eb, ctx.raised_by_callbacks(eb)); + } + for e in &m.enums { + // A plain C-style enum is a module of integer constants; a + // rich (algebraic) enum is a tagged value-class hierarchy. + if e.is_rich() { + render_rich_enum_class(w, e); + } else { + render_enum(w, e); + } + } + for s in &m.structs { + render_struct_class(w, s); + } + for s in &m.structs { + ctx.codecs.render_struct(w, s); + } + for e in m.enums.iter().filter(|e| e.is_rich()) { + ctx.codecs.render_rich_enum(w, e); + } + if !m.interfaces.is_empty() || !m.functions.is_empty() { + w.blank(); + } + for i in &m.interfaces { + render_interface_ffi(w, i); + } + for f in &m.functions { + render_attach_function(w, f); + } + let members = m + .interfaces + .iter() + .flat_map(|i| i.constructors.iter().chain(&i.methods).chain(&i.statics)); + for f in members.chain(&m.functions) { + render_async_trampoline(w, &ctx, error, f); + } + // Callback interfaces precede interfaces and functions because + // their static vtables are what members and free functions pass. + for cb in &m.callback_interfaces { + callbacks::render_callback_interface(w, &ctx, error, cb); + } + for i in &m.interfaces { + render_interface_class(w, &ctx, error, i); + } + for f in &m.functions { + render_callable(w, &ctx, error, f, &RbScope::free(ctx.module)); + } + } + ctx.codecs.render(w); + }); + w.blank(); + w.raw(render_trailer(CommentStyle::Hash, lib_file)); + w.finish() +} + +/// The load-time contract check: for every top-level module, each +/// declaration's `[id, hash, path]` from [`contract::entries`], which the +/// runtime's `_wv_check_contract!` looks up in the library's own table. +fn render_contract(w: &mut CodeWriter, model: &Model) { + w.line("# The declarations these bindings were generated with, per top-level"); + w.line("# module, as [id, hash, path]. Loading fails, naming the path, unless"); + w.line("# the library's contract table holds each one with an equal hash."); + w.line("_wv_check_contract!("); + w.scope(|w| { + for root in model.roots() { + w.line(format!( + "{}: [", + contract_symbol(model.prefix(), &root.name) + )); + w.scope(|w| { + for e in contract::entries(model, root) { + w.line(format!( + "[0x{:016x}, 0x{:016x}, '{}'],", + e.id, + e.hash, + rb_str_literal(&e.path) + )); + } + }); + w.line("],"); + } + }); + w.line(")"); +} diff --git a/crates/weaveffi-gen/src/targets/ruby/package.rs b/crates/weaveffi-cli/src/targets/ruby/package.rs similarity index 70% rename from crates/weaveffi-gen/src/targets/ruby/package.rs rename to crates/weaveffi-cli/src/targets/ruby/package.rs index efce0fe0..03230cfb 100644 --- a/crates/weaveffi-gen/src/targets/ruby/package.rs +++ b/crates/weaveffi-cli/src/targets/ruby/package.rs @@ -33,17 +33,27 @@ impl GemNames<'_> { } } -/// The metadata lines shared by both gemspec shapes. RubyGems requires a -/// non-empty author list, so an identity without authors falls back to the -/// gem name. -fn gemspec_metadata(names: &GemNames) -> String { +/// The `required_ruby_version` every gem declares. +pub(crate) const REQUIRED_RUBY_VERSION: &str = ">= 2.7"; + +/// The `ffi` gem requirement every gem declares. +pub(crate) const FFI_REQUIREMENT: &str = "~> 1.15"; + +/// The gem's authors. RubyGems requires a non-empty list, so an identity +/// without authors falls back to the gem name. +pub(crate) fn gem_authors(names: &GemNames) -> Vec { let id = names.identity; - let authors = if id.authors.is_empty() { + if id.authors.is_empty() { vec![names.gem.clone()] } else { id.authors.clone() - }; - let authors = authors + } +} + +/// The metadata lines of the gemspec. +fn gemspec_metadata(names: &GemNames) -> String { + let id = names.identity; + let authors = gem_authors(names) .iter() .map(|a| format!("'{}'", rb_str_literal(a))) .collect::>() @@ -74,42 +84,17 @@ fn gemspec_metadata(names: &GemNames) -> String { } /// Render the source-only gemspec emitted by `generate`. -pub(crate) fn render_gemspec(names: &GemNames, input_basename: &str) -> String { - let prelude = render_prelude(CommentStyle::Hash, input_basename); +pub(crate) fn render_gemspec(names: &GemNames) -> String { + let prelude = render_prelude(CommentStyle::Hash); let trailer = render_trailer(CommentStyle::Hash, &names.gemspec_file()); let meta = gemspec_metadata(names); format!( "{prelude}Gem::Specification.new do |s| {meta} s.files = Dir['lib/**/*.rb'] s.require_paths = ['lib'] - s.required_ruby_version = '>= 2.7' + s.required_ruby_version = '{REQUIRED_RUBY_VERSION}' - s.add_dependency 'ffi', '~> 1.15' -end - -{trailer}" - ) -} - -/// Render a platform gemspec: it stamps `s.platform` with the RubyGems -/// platform string (`Platform::ruby_platform`) and ships the bundled native -/// library alongside the Ruby sources. -pub(crate) fn render_packaged_gemspec( - names: &GemNames, - ruby_platform: &str, - input_basename: &str, -) -> String { - let prelude = render_prelude(CommentStyle::Hash, input_basename); - let trailer = render_trailer(CommentStyle::Hash, &names.gemspec_file()); - let meta = gemspec_metadata(names); - format!( - "{prelude}Gem::Specification.new do |s| -{meta} s.platform = '{ruby_platform}' - s.files = Dir['lib/**/*.{{rb,so,dylib,dll}}'] - s.require_paths = ['lib'] - s.required_ruby_version = '>= 2.7' - - s.add_dependency 'ffi', '~> 1.15' + s.add_dependency 'ffi', '{FFI_REQUIREMENT}' end {trailer}" @@ -117,8 +102,8 @@ end } /// README for the source-only gem layout. -pub(crate) fn render_readme(names: &GemNames, input_basename: &str) -> String { - let prelude = render_prelude(CommentStyle::Xml, input_basename); +pub(crate) fn render_readme(names: &GemNames) -> String { + let prelude = render_prelude(CommentStyle::Xml); let trailer = render_trailer(CommentStyle::Xml, "README.md"); let gem = &names.gem; let version = &names.identity.version; @@ -157,8 +142,8 @@ require '{require}' } /// README for a packaged Ruby platform gem. -pub(crate) fn render_packaged_readme(names: &GemNames, input_basename: &str) -> String { - let prelude = render_prelude(CommentStyle::Xml, input_basename); +pub(crate) fn render_packaged_readme(names: &GemNames) -> String { + let prelude = render_prelude(CommentStyle::Xml); let trailer = render_trailer(CommentStyle::Xml, "README.md"); let gem = &names.gem; let version = &names.identity.version; @@ -173,8 +158,7 @@ automatically; no external setup is required. ## Install ```bash -gem build {gem}.gemspec -gem install {gem}-{version}-*.gem +gem install {gem} -v {version} ``` ## Usage diff --git a/crates/weaveffi-gen/src/targets/ruby/runtime.rs b/crates/weaveffi-cli/src/targets/ruby/runtime.rs similarity index 87% rename from crates/weaveffi-gen/src/targets/ruby/runtime.rs rename to crates/weaveffi-cli/src/targets/ruby/runtime.rs index af22dcfe..4ff43694 100644 --- a/crates/weaveffi-gen/src/targets/ruby/runtime.rs +++ b/crates/weaveffi-cli/src/targets/ruby/runtime.rs @@ -16,12 +16,7 @@ const RUNTIME_RB: &str = include_str!("runtime/runtime.rb"); /// Render `lib/{prefix}/runtime.rb` for `identity`, opening the Ruby module /// `module_name`. -pub(crate) fn render_runtime( - identity: &Identity, - module_name: &str, - input_basename: &str, - file_name: &str, -) -> String { +pub(crate) fn render_runtime(identity: &Identity, module_name: &str, file_name: &str) -> String { let (lib_macos, lib_linux, lib_windows) = identity.library_files(); let body = RUNTIME_RB .replace("{{MODULE}}", module_name) @@ -34,7 +29,7 @@ pub(crate) fn render_runtime( .replace("{{ABI_VERSION}}", &ABI_VERSION.to_string()); format!( "{}{body}\n{}", - render_prelude(CommentStyle::Hash, input_basename), + render_prelude(CommentStyle::Hash), render_trailer(CommentStyle::Hash, file_name) ) } diff --git a/crates/weaveffi-gen/src/targets/ruby/runtime/runtime.rb b/crates/weaveffi-cli/src/targets/ruby/runtime/runtime.rb similarity index 59% rename from crates/weaveffi-gen/src/targets/ruby/runtime/runtime.rb rename to crates/weaveffi-cli/src/targets/ruby/runtime/runtime.rb index 15f31c2f..4ef79139 100644 --- a/crates/weaveffi-gen/src/targets/ruby/runtime/runtime.rb +++ b/crates/weaveffi-cli/src/targets/ruby/runtime/runtime.rb @@ -4,8 +4,8 @@ require 'monitor' # The fixed runtime every generated {{MODULE}} binding builds on: the native -# library loader, the load-time ABI check, the error classes, the -# value-buffer reader and writer, cancel tokens, the handle table behind +# library loader, the load-time ABI and contract checks, the error classes, +# the value-buffer reader and writer, cancel tokens, the handle table behind # callback interfaces and async completions, and the object wrapper base. module {{MODULE}} extend FFI::Library @@ -52,8 +52,7 @@ class ErrorStruct < FFI::Struct end # Codes reserved by the runtime. Positive codes belong to a module's error - # domain; these negative codes are raised as a plain Error (or Cancelled) - # regardless of whether the function declares errors. + # domain; these negative codes report runtime failures. GENERIC_ERROR_CODE = -1 PANIC_ERROR_CODE = -2 MARSHAL_ERROR_CODE = -3 @@ -75,6 +74,16 @@ def initialize(code, message) end end + # Raised when a call that declares no errors fails anyway. That's a bug in + # the library (a panic, an argument it couldn't accept, a malformed value + # it returned, a callback failure it let through), never an outcome to + # handle; the message names the code and the library's message. + class NativeBugError < Error + def initialize(code, message) + super(code, "native call failed with code #{code}: #{message}") + end + end + # Raised when a cancellable call completes because its CancelToken fired. class Cancelled < Error def initialize(message = 'cancelled') @@ -83,8 +92,10 @@ def initialize(message = 'cancelled') end attach_function :{{PREFIX}}_error_set, [:pointer, :int32, :string], :void + attach_function :{{PREFIX}}_error_set_payload, [:pointer, :pointer, :size_t], :void attach_function :{{PREFIX}}_error_clear, [:pointer], :void attach_function :{{PREFIX}}_error_free, [:pointer], :void + attach_function :{{PREFIX}}_alloc, [:size_t], :pointer attach_function :{{PREFIX}}_free_bytes, [:pointer, :size_t], :void attach_function :{{PREFIX}}_cancel_token_create, [], :pointer attach_function :{{PREFIX}}_cancel_token_cancel, [:pointer], :void @@ -92,13 +103,65 @@ def initialize(message = 'cancelled') attach_function :{{PREFIX}}_cancel_token_destroy, [:pointer], :void # @api private - # The error for a code outside any declared domain. - def self._wv_error(code, message) + # One entry of a module's contract table. + class ContractEntry < FFI::Struct + layout :id, :uint64, + :hash, :uint64 + end + + # @api private + # Checks, for every top-level module, that the library's contract table + # (`{{PREFIX}}_{module}_contract`) holds each declaration these bindings + # were generated with, as `[id, hash, path]`, with an equal signature + # hash. Declarations the library adds are fine. + def self._wv_check_contract!(tables) + tables.each do |symbol, expected| + begin + attach_function symbol, [:pointer], :pointer unless respond_to?(symbol) + rescue FFI::NotFoundError + raise LoadError, "the loaded {{LIBRARY}} library does not export #{symbol}" + end + out_len = FFI::MemoryPointer.new(:size_t) + table = public_send(symbol, out_len) + count = table.null? ? 0 : out_len.read(:size_t) + actual = Array.new(count) do |i| + entry = ContractEntry.new(table + (i * ContractEntry.size)) + [entry[:id], entry[:hash]] + end.to_h + expected.each do |id, hash, path| + found = actual[id] + raise LoadError, "{{LIBRARY}}: #{path} is missing from the library" if found.nil? + next if found == hash + + raise LoadError, "{{LIBRARY}}: #{path} changed since these bindings were generated" + end + end + end + + # @api private + # The error a throwing call raises for a code outside its domain: + # Cancelled for -5, otherwise the root Error. + def self._wv_error(code, message, _payload = nil) return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE Error.new(code, message) end + # @api private + # The error a call that declares no errors raises when it fails anyway: + # Cancelled for -5, otherwise NativeBugError. + def self._wv_trap(code, message, _payload = nil) + return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE + + NativeBugError.new(code, message) + end + + # @api private + # The error for a malformed value the library handed over. + def self._wv_malformed(what) + NativeBugError.new(MARSHAL_ERROR_CODE, "malformed value buffer: #{what}") + end + # @api private # Copies the code, message, and payload out of a reported error and clears # it. Returns nil when the struct reports success. @@ -126,25 +189,71 @@ def self._wv_take_boxed_error(err_ptr) end # @api private - # Raises the generic error for a call that declares no error domain. - def self.check_error!(err) + # Raises NativeBugError for a failed call that declares no errors. + def self._wv_check!(err) taken = _wv_take_error(err) - return if taken.nil? - - raise _wv_error(taken[0], taken[1]) + raise _wv_trap(*taken) unless taken.nil? end # @api private # The UTF-8 bytes of a String argument, as a private binary copy the - # producer borrows for the call. + # library borrows for the call. def self._wv_str(value) - value.to_s.encode(Encoding::UTF_8).b + str = String.try_convert(value) + raise TypeError, "expected a String, got #{value.class}" if str.nil? + + utf8 = str.encode(Encoding::UTF_8) + raise ArgumentError, 'string is not valid UTF-8' unless utf8.valid_encoding? + + utf8.b end # @api private # A private binary copy of a bytes argument. def self._wv_bytes(value) - value.to_s.b + str = String.try_convert(value) + raise TypeError, "expected a String of bytes, got #{value.class}" if str.nil? + + str.b + end + + # The inclusive range of each fixed-width integer type. + INT_RANGES = { + i8: -0x80..0x7f, + u8: 0..0xff, + i16: -0x8000..0x7fff, + u16: 0..0xffff, + i32: -0x8000_0000..0x7fff_ffff, + u32: 0..0xffff_ffff, + i64: -0x8000_0000_0000_0000..0x7fff_ffff_ffff_ffff, + u64: 0..0xffff_ffff_ffff_ffff + }.freeze + + # @api private + # `value`, checked to be an Integer in the range of `kind` (`:i32`, ...): + # out-of-range values raise RangeError rather than wrap. + def self._wv_int(value, kind) + raise TypeError, "expected an Integer for #{kind}, got #{value.class}" unless value.is_a?(Integer) + raise RangeError, "#{value} is out of range for #{kind}" unless INT_RANGES[kind].cover?(value) + + value + end + + # @api private + # Raises TypeError unless `value` is a `cls` wrapper (or nil, when + # `optional`), so an object of the wrong interface never reaches the + # library. + def self._wv_object!(value, cls, name, optional = false) + return if value.is_a?(cls) || (optional && value.nil?) + + expected = optional ? "a #{cls.name} or nil" : "a #{cls.name}" + raise TypeError, "#{name} must be #{expected}, got #{value.class}" + end + + # @api private + # Raises TypeError for a nil callback implementation where one is required. + def self._wv_present!(value, name) + raise TypeError, "#{name} must not be nil" if value.nil? end # @api private @@ -153,6 +262,12 @@ def self._wv_borrow_bytes(ptr, len) ptr.null? || len.zero? ? ''.b : ptr.read_string(len) end + # @api private + # Copies a borrowed (ptr, len) UTF-8 run without releasing it. + def self._wv_borrow_string(ptr, len) + _wv_borrow_bytes(ptr, len).force_encoding(Encoding::UTF_8) + end + # @api private # Copies a returned (ptr, len) allocation, then releases it. def self._wv_take_bytes(ptr, len) @@ -216,21 +331,23 @@ def _wv_ptr # @api private # Appends values in the value-buffer wire format: little-endian, packed, - # no alignment. + # no alignment. Integers out of their type's range raise RangeError rather + # than wrap. class WvBufferWriter def initialize @buf = +''.b @objects = [] end - # Reserves an object token. The strong reference it carries is minted - # by #seal, just before the buffer is handed to the producer, so an - # encoding abandoned on the way never strands a reference. - def write_object(object) - raise TypeError, "expected a wrapper object, got #{object.class}" unless object.is_a?(WvObject) + # Reserves an object token for a `cls` wrapper. The strong reference it + # carries is minted by #seal, just before the buffer is handed to the + # library, so an encoding abandoned on the way never strands a + # reference. + def write_object(object, cls) + raise TypeError, "expected a #{cls.name}, got #{object.class}" unless object.is_a?(cls) @objects << [@buf.bytesize, object] - write_u64(0) + @buf << [0].pack('Q<') end # Mints the reference each reserved object token carries, recording it @@ -244,10 +361,15 @@ def seal(minted) @buf end - # Every append is a binary String, so the buffer stays binary and byte - # offsets index it directly. + # The finished encoding of a buffer that carries no objects. + def bytes + raise TypeError, 'this encoding cannot carry objects' unless @objects.empty? + + @buf + end + def write_bool(value) - @buf << [value ? 1 : 0].pack('C') + @buf << (value ? "\x01" : "\x00").b end def write_flag(value) @@ -255,27 +377,27 @@ def write_flag(value) end def write_i8(value) - @buf << [value].pack('c') + @buf << [int(value, :i8)].pack('c') end def write_u8(value) - @buf << [value].pack('C') + @buf << [int(value, :u8)].pack('C') end def write_i16(value) - @buf << [value].pack('s<') + @buf << [int(value, :i16)].pack('s<') end def write_u16(value) - @buf << [value].pack('S<') + @buf << [int(value, :u16)].pack('S<') end def write_i32(value) - @buf << [value].pack('l<') + @buf << [int(value, :i32)].pack('l<') end def write_u32(value) - @buf << [value].pack('L<') + @buf << [int(value, :u32)].pack('L<') end def write_len(value) @@ -283,11 +405,11 @@ def write_len(value) end def write_i64(value) - @buf << [value].pack('q<') + @buf << [int(value, :i64)].pack('q<') end def write_u64(value) - @buf << [value].pack('Q<') + @buf << [int(value, :u64)].pack('Q<') end def write_f32(value) @@ -299,19 +421,25 @@ def write_f64(value) end def write_string(value) - write_bytes(value.to_s.encode(Encoding::UTF_8)) + write_bytes({{MODULE}}._wv_str(value)) end def write_bytes(value) - bytes = value.to_s.b + bytes = {{MODULE}}._wv_bytes(value) write_u32(bytes.bytesize) @buf << bytes end + + private + + def int(value, kind) + {{MODULE}}._wv_int(value, kind) + end end # @api private - # Reads values in the value-buffer wire format, raising Error on any - # malformed buffer. + # Reads values in the value-buffer wire format, raising NativeBugError on + # any malformed buffer. class WvBufferReader def initialize(data) @data = data.to_s.b @@ -319,9 +447,7 @@ def initialize(data) end def take(count, what) - if @pos + count > @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, "malformed value buffer: truncated #{what}") - end + raise {{MODULE}}._wv_malformed("truncated #{what}") if @pos + count > @data.bytesize bytes = @data.byteslice(@pos, count) @pos += count @@ -330,14 +456,14 @@ def take(count, what) def read_bool byte = take(1, 'bool').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: bool byte out of range') if byte > 1 + raise {{MODULE}}._wv_malformed('bool byte out of range') if byte > 1 byte == 1 end def read_flag byte = take(1, 'option flag').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: option flag out of range') if byte > 1 + raise {{MODULE}}._wv_malformed('option flag out of range') if byte > 1 byte == 1 end @@ -391,7 +517,7 @@ def read_f64 def read_string text = read_bytes.force_encoding(Encoding::UTF_8) - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: string is not valid UTF-8') unless text.valid_encoding? + raise {{MODULE}}._wv_malformed('string is not valid UTF-8') unless text.valid_encoding? text end @@ -400,19 +526,19 @@ def read_bytes take(read_u32, 'byte run') end - # An object token: a non-zero u64 carrying one strong reference the - # caller adopts into a wrapper. - def read_object_token + # An object token: a non-zero u64 carrying one strong reference, adopted + # into a new `cls` wrapper. + def read_object(cls) addr = read_u64 - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: null object token') if addr.zero? + raise {{MODULE}}._wv_malformed('null object token') if addr.zero? - FFI::Pointer.new(addr) + cls._from_ptr(FFI::Pointer.new(addr)) end def expect_end! return if @pos == @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: trailing bytes after value') + raise {{MODULE}}._wv_malformed('trailing bytes after value') end end @@ -420,7 +546,7 @@ def expect_end! # Seals the writers of one call's object-carrying buffers (see # WvBufferWriter#seal) and returns their encodings. If any reference can't # be minted (a closed wrapper), the ones already minted are released before - # the error propagates. The producer adopts every minted reference once the + # the error propagates. The library adopts every minted reference once the # call is made. def self._wv_seal(*writers) minted = [] @@ -430,6 +556,23 @@ def self._wv_seal(*writers) raise end + # @api private + # Encodes an argument that carries no objects: yields a writer and + # returns the finished encoding. + def self._wv_encode + writer = WvBufferWriter.new + yield writer + writer.bytes + end + + # @api private + # A returned object pointer that must not be NULL. + def self._wv_nonnull(ptr) + raise NativeBugError.new(MARSHAL_ERROR_CODE, 'null object pointer') if ptr.null? + + ptr + end + # @api private # Decodes a returned value buffer: copies and releases it, yields a reader # to the block, and checks that the block consumed every byte. @@ -440,9 +583,19 @@ def self._wv_decode(ptr, len) value end - # Ruby objects the producer refers to by an integer key: callback-interface + # @api private + # Decodes a borrowed value buffer (a callback argument) without releasing + # it. + def self._wv_decode_borrowed(ptr, len) + reader = WvBufferReader.new(_wv_borrow_bytes(ptr, len)) + value = yield reader + reader.expect_end! + value + end + + # Ruby objects the library refers to by an integer key: callback-interface # implementations (the key is the vtable `ctx`) and pending async - # completions (the key is the completion `context`). The producer never + # completions (the key is the completion `context`). The library never # sees a Ruby object address. A Monitor rather than a Mutex, so a # finalizer that runs while this thread holds the lock can't deadlock it. @wv_handles = {} @@ -469,12 +622,15 @@ def self._wv_handle_take(key) end # @api private - # Registers a callback-interface implementation and returns its `ctx`. - # Never raises, so nothing can strand the registration between this and - # the call that hands it to the producer (an unusable implementation fails - # when the producer calls it, with FOREIGN_ERROR_CODE). - def self._wv_cb_register(impl) - _wv_handle_put([:callback, impl]) + # Registers a callback-interface implementation and returns its `ctx` and + # `vtable` arguments: two NULLs for a nil (optional) implementation. Never + # raises, so nothing can strand the registration between this and the + # call that hands it to the library (an unusable implementation fails + # when the library calls it, with FOREIGN_ERROR_CODE). + def self._wv_cb_register(impl, vtable) + return [nil, nil] if impl.nil? + + [_wv_handle_put([:callback, impl]), vtable.to_ptr] end # @api private @@ -487,23 +643,82 @@ def self._wv_cb_lookup(ctx) end # @api private - # Drops an implementation when the producer releases its last reference. + # Drops an implementation when the library releases its last reference. def self._wv_cb_free(ctx) _wv_handle_take(ctx) nil end # @api private - # Reports a callback implementation's exception through `out_err`. The - # producer copies the message. - def self._wv_cb_fail(out_err, exception) + # The message a callback failure reports: the exception's message (its + # class name when that's empty), as UTF-8 without NULs. + def self._wv_cb_message(exception) message = exception.message.to_s - message = exception.class.name if message.empty? - message = message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") - {{PREFIX}}_error_set(out_err, FOREIGN_ERROR_CODE, message) + message = exception.class.name.to_s if message.empty? + message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") + end + + # @api private + # Reports a callback implementation's exception through `out_err` as + # FOREIGN_ERROR_CODE. The library copies the message. + def self._wv_cb_fail(out_err, exception) + {{PREFIX}}_error_set(out_err, FOREIGN_ERROR_CODE, _wv_cb_message(exception)) nil end + # @api private + # Reports a domain error raised by a callback method declared to throw: + # the block returns its `[code, payload]`, or nil when it isn't one of the + # domain's codes, which then fails as FOREIGN_ERROR_CODE like any other + # exception. + def self._wv_cb_throw(out_err, exception) + code, payload = yield + return _wv_cb_fail(out_err, exception) if code.nil? + + {{PREFIX}}_error_set(out_err, code, _wv_cb_message(exception)) + {{PREFIX}}_error_set_payload(out_err, payload, payload.bytesize) unless payload.empty? + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(out_err, e) + end + + # @api private + # A callback method's floating-point return, checked to be numeric. + def self._wv_float(value) + raise TypeError, "expected a Float, got #{value.class}" unless value.is_a?(Numeric) + + value.to_f + end + + # @api private + # Hands a callback method's string, bytes, or buffer return to the + # library: a run allocated with {{PREFIX}}_alloc, written to the out slots. + # The library adopts and frees it. + def self._wv_cb_return_bytes(out_ptr, out_len, data) + len = data.bytesize + run = FFI::Pointer::NULL + unless len.zero? + run = {{PREFIX}}_alloc(len) + raise NoMemoryError, '{{PREFIX}}_alloc failed' if run.null? + + run.put_bytes(0, data) + end + out_ptr.write_pointer(run) + out_len.write(:size_t, len) + nil + end + + # @api private + # A callback method's object return: a fresh strong reference the library + # adopts, or NULL for nil (which the library rejects where the return + # isn't optional). + def self._wv_cb_return_object(value, cls) + return FFI::Pointer::NULL if value.nil? + raise TypeError, "expected a #{cls.name}, got #{value.class}" unless value.is_a?(cls) + + value._wv_clone_ptr + end + # @api private # Registers a pending async call; returns its completion context and the # queue its completion is delivered on. @@ -545,7 +760,7 @@ def self._wv_async_wait(queue, token = nil) # another thread only releases the reference once the call returns. class WvObject # @api private - # Adopts one strong reference the producer handed over. + # Adopts one strong reference the library handed over. def self._from_ptr(ptr) obj = allocate obj._wv_init(ptr) @@ -575,7 +790,7 @@ def closed? # Releases this wrapper's reference now rather than at GC time. # Idempotent. The object itself lives until its last reference anywhere - # (another wrapper, a record field, the producer) goes. + # (another wrapper, a record field, the library) goes. def close handle = @wv_lock.synchronize do next nil if @wv_handle.nil? || @wv_closing @@ -612,7 +827,7 @@ def _wv_call # @api private # Mints a new strong reference the caller owns (an object written into a - # value buffer carries its own reference). + # value buffer or returned from a callback carries its own reference). def _wv_clone_ptr _wv_call { |ptr| self.class._wv_clone(ptr) } end @@ -659,36 +874,16 @@ def self._wv_pin_from(objects, index, pointers, &block) end end - # @api private - # Checks each top-level module's contract checksum (`module name => - # [checksum function, expected value]`) against the library, so bindings - # generated from a different API fail at require time. - def self._wv_check_contract!(checksums) - checksums.each do |mod, (symbol, expected)| - begin - attach_function symbol, [], :uint64 - rescue FFI::NotFoundError - raise LoadError, "the loaded {{LIBRARY}} library has no module '#{mod}' (#{symbol} is missing)" - end - actual = public_send(symbol) - next if actual == expected - - raise LoadError, "{{LIBRARY}} module '#{mod}' does not match these bindings: expected contract " \ - "checksum 0x#{expected.to_s(16)}, the library reports 0x#{actual.to_s(16)}; " \ - 'regenerate the bindings' - end - end - # @api private # Live native resources of one kind (0 objects, 1 callbacks, 2 iterators, - # 3 cancel tokens, 4 byte allocations). Only libraries built with the - # `leak-check` feature export the counter. + # 3 cancel tokens, 4 byte runs; -1 reports whether the library counts at + # all). Only libraries built with the `leak-check` feature count. def self._wv_debug_live(kind) unless respond_to?(:{{PREFIX}}_debug_live) begin attach_function :{{PREFIX}}_debug_live, [:int32], :uint64 rescue FFI::NotFoundError - raise Error.new(GENERIC_ERROR_CODE, '{{LIBRARY}} was built without the leak-check feature') + raise Error.new(GENERIC_ERROR_CODE, '{{LIBRARY}} does not export {{PREFIX}}_debug_live') end end {{PREFIX}}_debug_live(kind) diff --git a/crates/weaveffi-cli/src/targets/ruby/tests.rs b/crates/weaveffi-cli/src/targets/ruby/tests.rs new file mode 100644 index 00000000..6d63f14c --- /dev/null +++ b/crates/weaveffi-cli/src/targets/ruby/tests.rs @@ -0,0 +1,595 @@ +//! Unit tests: render a small API that exercises every C ABI revision 4 +//! shape the Ruby bindings distinguish and assert the key pieces of each +//! contract. + +use camino::Utf8Path; +use weaveffi_model::contract::entries; +use weaveffi_model::ir::Api; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; + +use super::{RubyConfig, RubyGenerator}; +use crate::backend::{LanguageBackend, OutputFile}; +use crate::package::{ArtifactKind, PackageContext}; +use crate::platform::{BinarySet, NativeBinary, Platform}; + +/// One module with an error domain carrying a field; an interface with a +/// `new` constructor, a method, and a throwing static; nullable objects; an +/// iterator of objects; a cancellable async function; a record carrying +/// objects; a record whose name collides with a composite's codec stem; +/// and a callback interface whose methods take every argument family and +/// return a direct value, an enum, a string, a record, an object, and an +/// optional object, one of them `throws`, passed both required and +/// optional. +const FIXTURE: &str = r#" +version: "0.11.0" +modules: + - name: shop + errors: + name: ShopError + codes: + - { name: OutOfStock, code: 1, message: "out of stock", fields: [{ name: sku, type: string }] } + - { name: Closed, code: 2, message: "closed" } + enums: + - name: Mood + variants: [{ name: Happy, value: 0 }, { name: Grumpy, value: 1 }] + structs: + - name: Order + fields: + - { name: cart, type: Cart } + - { name: history, type: "[Cart]" } + - { name: note, type: "string?" } + - name: ListI32 + fields: + - { name: items, type: "[i32]" } + interfaces: + - name: Cart + constructors: + - { name: new, params: [{ name: owner, type: string }] } + methods: + - { name: add, params: [{ name: sku, type: string }], return: bool } + - { name: total, return: i64, deprecated: "use sum" } + statics: + - { name: restore, params: [{ name: id, type: i64 }], return: Cart, throws: true } + callback_interfaces: + - name: Watcher + methods: + - name: on_event + params: + - { name: name, type: string } + - { name: count, type: i32 } + - { name: order, type: Order } + - { name: cart, type: Cart } + return: bool + - { name: mood, return: Mood } + - { name: label, return: string, throws: true } + - { name: latest, return: "Order?" } + - { name: favorite, return: Cart } + - { name: maybe, return: "Cart?" } + - { name: on_done } + functions: + - { name: find_cart, params: [{ name: current, type: "Cart?" }], return: "Cart?" } + - { name: all_carts, return: "iter" } + - { name: watch, params: [{ name: watcher, type: Watcher }] } + - { name: maybe_watch, params: [{ name: watcher, type: "Watcher?" }] } + - { name: echo, params: [{ name: text, type: string }], return: string } + - { name: tally, params: [{ name: counts, type: "{string:[i32?]}" }], return: "[Order]" } + - { name: firsts, params: [{ name: lists, type: "[i32]" }], return: ListI32 } + - { name: wait, params: [{ name: ms, type: i64 }], return: i64, async: true, cancellable: true } +"#; + +fn model() -> Model { + let api: Api = serde_yaml::from_str(FIXTURE).unwrap(); + validate(&api, &Identity::named("shop_kit"), None).unwrap() +} + +fn generate(config: &RubyConfig) -> Vec { + RubyGenerator.files(&model(), Utf8Path::new("out"), config) +} + +fn file<'a>(files: &'a [OutputFile], path: &str) -> &'a str { + &files + .iter() + .find(|f| f.path.as_str() == path) + .unwrap_or_else(|| { + panic!( + "no {path} in {:?}", + files.iter().map(|f| &f.path).collect::>() + ) + }) + .contents +} + +fn bindings() -> String { + file( + &generate(&RubyConfig::default()), + "out/ruby/lib/shop_kit.rb", + ) + .to_string() +} + +fn runtime() -> String { + file( + &generate(&RubyConfig::default()), + "out/ruby/lib/shop_kit/runtime.rb", + ) + .to_string() +} + +#[track_caller] +fn assert_has(src: &str, needle: &str) { + assert!(src.contains(needle), "missing {needle:?} in:\n{src}"); +} + +#[track_caller] +fn assert_lacks(src: &str, needle: &str) { + assert!(!src.contains(needle), "unexpected {needle:?} in:\n{src}"); +} + +/// The text of the generated item that starts at the first line containing +/// `start`, up to its closing `end` at the same indentation. +fn item<'a>(src: &'a str, start: &str) -> &'a str { + let at = src + .find(start) + .unwrap_or_else(|| panic!("no {start:?} in:\n{src}")); + let line_start = src[..at].rfind('\n').map_or(0, |i| i + 1); + let indent = &src[line_start..at]; + let indent = &indent[..indent.len() - indent.trim_start().len()]; + let close = format!("\n{indent}end\n"); + let end = src[at..].find(&close).expect("item has an end") + at + close.len(); + &src[line_start..end] +} + +#[test] +fn layout_follows_the_identity() { + let files = generate(&RubyConfig::default()); + let paths: Vec<&str> = files.iter().map(|f| f.path.as_str()).collect(); + assert_eq!( + paths, + [ + "out/ruby/lib/shop_kit.rb", + "out/ruby/lib/shop_kit/runtime.rb", + "out/ruby/shop_kit.gemspec", + "out/ruby/README.md", + ] + ); + assert_has( + file(&files, "out/ruby/shop_kit.gemspec"), + "s.name = 'shop_kit'", + ); + let src = file(&files, "out/ruby/lib/shop_kit.rb"); + assert_has(src, "require_relative 'shop_kit/runtime'"); + assert_has(src, "module ShopKit\n"); + // Nothing but the generated-file banner mentions WeaveFFI. + for f in &files { + for line in f.contents.lines().filter(|l| { + !l.contains("Generated by WeaveFFI") && !l.contains("To regenerate: weaveffi generate") + }) { + assert!( + !line.to_ascii_lowercase().contains("weaveffi"), + "branded line in {}: {line}", + f.path + ); + } + } +} + +#[test] +fn config_overrides_gem_and_module_names() { + let config = RubyConfig { + name: Some("acme-shop".into()), + module_name: Some("Acme".into()), + }; + let files = generate(&config); + assert_has( + file(&files, "out/ruby/acme-shop.gemspec"), + "s.name = 'acme-shop'", + ); + // The require path stays the identity prefix. + let src = file(&files, "out/ruby/lib/shop_kit.rb"); + assert_has(src, "module Acme\n"); + assert_has(src, "Acme.shop_kit_shop_Cart_destroy(ptr)"); + assert_has( + file(&files, "out/ruby/lib/shop_kit/runtime.rb"), + "module Acme\n", + ); +} + +#[test] +fn runtime_speaks_abi_4() { + let rt = runtime(); + assert_has(&rt, "ENV['SHOP_KIT_LIBRARY']"); + assert_has(&rt, "'libshop_kit.dylib'"); + assert_has(&rt, "'libshop_kit.so'"); + assert_has(&rt, "'shop_kit.dll'"); + assert_has(&rt, &format!("ABI_VERSION = {}", crate::cabi::ABI_VERSION)); + assert_eq!(crate::cabi::ABI_VERSION, 4); + assert_has(&rt, "attach_function :shop_kit_abi_version, [], :uint32"); + assert_has(&rt, "attach_function :shop_kit_alloc, [:size_t], :pointer"); + assert_has( + &rt, + "attach_function :shop_kit_error_set_payload, [:pointer, :pointer, :size_t], :void", + ); + assert_has( + &rt, + "attach_function :shop_kit_free_bytes, [:pointer, :size_t], :void", + ); + assert_lacks(&rt, "dealloc"); + assert_lacks(&rt, "checksum"); + assert_lacks(&rt, "{{"); + // The load errors name the declaration path. + assert_has(&rt, "#{path} is missing from the library"); + assert_has(&rt, "#{path} changed since these bindings were generated"); + // The trap policy: a failure of a call that declares no errors is a + // NativeBugError naming the code and message; -5 is Cancelled. + assert_has(&rt, "class NativeBugError < Error"); + assert_has(&rt, "native call failed with code #{code}: #{message}"); + assert_has(&rt, "class Cancelled < Error"); + assert_has(&rt, "CANCELLED_ERROR_CODE = -5"); +} + +#[test] +fn load_checks_embed_every_contract_entry() { + let model = model(); + let src = bindings(); + assert_has( + &src, + " _wv_check_contract!(\n shop_kit_shop_contract: [\n", + ); + let root = model.roots().next().unwrap(); + let table = entries(&model, root); + assert!(table.len() > 10); + for e in &table { + assert_has( + &src, + &format!("[0x{:016x}, 0x{:016x}, '{}'],", e.id, e.hash, e.path), + ); + } + assert_lacks(&src, "checksum"); +} + +#[test] +fn every_call_but_the_trivial_helpers_releases_the_gvl() { + let src = bindings(); + for line in src.lines().filter(|l| l.contains("attach_function")) { + let trivial = line.contains("_clone,") || line.contains("_destroy,"); + assert_eq!(!trivial, line.ends_with(", blocking: true"), "{line}"); + } + assert_has( + &src, + "attach_function :shop_kit_shop_echo, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true", + ); + assert_has( + &src, + "attach_function :shop_kit_shop_AllCartsIterator_destroy, [:pointer], :void\n", + ); +} + +#[test] +fn errors_follow_the_trap_policy() { + let src = bindings(); + // A throwing call raises the domain hierarchy; any other call traps. + assert_has( + item(&src, "def self.restore(id)"), + "ShopKit._wv_check_shop_error!(err)", + ); + assert_has(item(&src, "def self.echo(text)"), "_wv_check!(err)"); + assert_has(&src, "queue << _wv_trap(*taken)"); + assert_has(&src, "class ShopError < Error"); + assert_has(&src, "class OutOfStock < ShopError"); + assert_has(&src, "def initialize(message = nil, sku: nil)"); + assert_has(&src, "super(CODE, message || 'out of stock')"); + assert_has( + &src, + "ShopError::OutOfStock.new(message, sku: r.read_string)", + ); +} + +#[test] +fn vtables_carry_the_header_and_every_return_family() { + let src = bindings(); + assert_has( + &src, + "layout :wv_size, :uint32,\n :wv_flags, :uint32,\n :wv_free, :pointer,\n :on_event, :pointer,", + ); + assert_has(&src, "WV_WATCHER_VTABLE[:wv_size] = WvWatcherVtable.size"); + assert_has(&src, "WV_WATCHER_VTABLE[:wv_flags] = 0"); + assert_has(&src, "WV_WATCHER_VTABLE[:wv_free] = WV_WATCHER_FREE"); + assert_eq!(src.matches("= WvWatcherVtable.new").count(), 1); + + // Direct: by value, coerced inside the rescue. + let on_event = item(&src, "WV_WATCHER_ON_EVENT = "); + assert_has(on_event, "FFI::Function.new(:bool, [:pointer, :pointer, :size_t, :int32, :pointer, :size_t, :pointer, :pointer])"); + assert_has( + on_event, + "do |wv_ctx, name_ptr, name_len, count, order_ptr, order_len, cart_ptr, wv_err|", + ); + // Objects are adopted before anything that can raise. + let adopt = on_event.find("cart = Cart._from_ptr(cart_ptr)").unwrap(); + assert!(adopt < on_event.find("name = _wv_borrow_string").unwrap()); + assert!(adopt < on_event.find("order = _wv_decode_borrowed").unwrap()); + assert_has( + on_event, + "_wv_cb_lookup(wv_ctx).on_event(name, count, order, cart) ? true : false", + ); + assert_has(on_event, " false\n"); + assert_has( + item(&src, "WV_WATCHER_MOOD = "), + "_wv_int(_wv_cb_lookup(wv_ctx).mood, :i32)", + ); + // String: a {prefix}_alloc run in the out slots. + let label = item(&src, "WV_WATCHER_LABEL = "); + assert_has(label, "do |wv_ctx, wv_out_ptr, wv_out_len, wv_err|"); + assert_has( + label, + "_wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_str(_wv_cb_lookup(wv_ctx).label))", + ); + // A record carrying objects: its references are minted by the seal. + let latest = item(&src, "WV_WATCHER_LATEST = "); + assert_has(latest, "_wv_write_opt_order(writer, result)"); + assert_has( + latest, + "_wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_seal(writer).first)", + ); + // Objects: a fresh reference the producer adopts (NULL for nil). + let favorite = item(&src, "WV_WATCHER_FAVORITE = "); + assert_has( + favorite, + "FFI::Function.new(:pointer, [:pointer, :pointer])", + ); + assert_has( + favorite, + "_wv_cb_return_object(_wv_cb_lookup(wv_ctx).favorite, Cart)", + ); + assert_has(favorite, "FFI::Pointer::NULL\n"); + assert_has( + item(&src, "WV_WATCHER_MAYBE = "), + "_wv_cb_return_object(_wv_cb_lookup(wv_ctx).maybe, Cart)", + ); + assert_has(&runtime(), "def self._wv_cb_return_object(value, cls)"); +} + +#[test] +fn throwing_callback_methods_report_the_domain_error() { + let src = bindings(); + let label = item(&src, "WV_WATCHER_LABEL = "); + assert_has(label, "rescue ShopError => e"); + assert_has( + label, + "_wv_cb_throw(wv_err, e) { _wv_shop_error_payload(e) }", + ); + // Every other exception is -4, after the domain rescue. + assert!(label.find("rescue ShopError").unwrap() < label.find("rescue Exception").unwrap()); + assert_has(label, "_wv_cb_fail(wv_err, e)"); + // A method without `throws` never reports a domain code. + assert_lacks(item(&src, "WV_WATCHER_MOOD = "), "rescue ShopError"); + // The payload encoder writes each code's fields. + let payload = item(&src, "def self._wv_shop_error_payload(error)"); + assert_has(payload, "when ShopError::OutOfStock\n"); + assert_has(payload, "w.write_string(error.sku)"); + assert_has(payload, "when ShopError::Closed then 2"); + assert_has(payload, "code && [code, w.bytes]"); + let rt = runtime(); + assert_has( + &rt, + "shop_kit_error_set_payload(out_err, payload, payload.bytesize)", + ); + assert_has( + &rt, + "shop_kit_error_set(out_err, FOREIGN_ERROR_CODE, _wv_cb_message(exception))", + ); +} + +#[test] +fn callback_parameters_register_their_implementation() { + let src = bindings(); + let watch = item(&src, "def self.watch(watcher)"); + assert_has(watch, "_wv_present!(watcher, 'watcher')"); + assert_has( + watch, + "watcher_ctx, watcher_vtable = _wv_cb_register(watcher, WV_WATCHER_VTABLE)", + ); + assert_has( + watch, + "shop_kit_shop_watch(watcher_ctx, watcher_vtable, err)", + ); + // An optional callback defaults to nil, which passes two NULLs. + let maybe = item(&src, "def self.maybe_watch(watcher = nil)"); + assert_lacks(maybe, "_wv_present!"); + assert_has( + maybe, + "watcher_ctx, watcher_vtable = _wv_cb_register(watcher, WV_WATCHER_VTABLE)", + ); + assert_has(&runtime(), "return [nil, nil] if impl.nil?"); +} + +#[test] +fn objects_are_checked_pinned_and_adopted() { + let src = bindings(); + assert_has(&src, "class CartPtr < FFI::AutoPointer"); + assert_has(&src, "class Cart < WvObject\n WV_PTR = CartPtr\n"); + assert_has(&src, "ShopKit.shop_kit_shop_Cart_clone(ptr)"); + let find = item(&src, "def self.find_cart(current = nil)"); + assert_has(find, "_wv_object!(current, Cart, 'current', true)"); + assert_has(find, "_wv_pin(current) do |_wv_current|"); + assert_has(find, "result.null? ? nil : Cart._from_ptr(result)"); + let add = item(&src, "def add(sku)"); + assert_has(add, "ShopKit._wv_pin(self) do |_wv_self|"); + let init = item(&src, "def initialize(owner)"); + assert_has(init, "_wv_init(ShopKit._wv_nonnull(result))"); + assert_has( + item(&src, "def self.restore(id)"), + "_from_ptr(ShopKit._wv_nonnull(result))", + ); + assert_has( + item(&src, "def total"), + "warn('ShopKit::Cart#total is deprecated: use sum', uplevel: 1)", + ); +} + +#[test] +fn composite_types_get_one_codec_pair_each() { + let src = bindings(); + for stem in [ + "map_string_list_opt_i32", + "list_opt_i32", + "opt_i32", + "list_order", + "list_cart", + "opt_string", + "opt_order", + "list_i32_2", + ] { + assert_eq!( + src.matches(&format!("def self._wv_write_{stem}(w, v)")) + .count(), + 1, + "{stem}" + ); + } + // The record `ListI32` owns the `list_i32` stem, so the `[i32]` + // composite takes the next free one. + assert_has( + &src, + "# Packs a ListI32 into the value-buffer wire format.\n def self._wv_write_list_i32(w, v)", + ); + assert_has( + &src, + "# Packs `[i32]` into the value-buffer wire format.\n def self._wv_write_list_i32_2(w, v)", + ); + assert_has(&src, "items: _wv_read_list_i32_2(r)"); + assert_has( + &src, + "counts_s = _wv_encode { |w| _wv_write_map_string_list_opt_i32(w, counts) }", + ); + assert_has( + &src, + "_wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_list_order(r) }", + ); + let map = item(&src, "def self._wv_write_map_string_list_opt_i32(w, v)"); + assert_has(map, "w.write_len(v.length)"); + assert_has(map, "v.each do |k, e|"); + assert_has(map, "_wv_write_list_opt_i32(w, e)"); + assert_has( + &src, + "Array.new(r.read_len) { [r.read_string, _wv_read_list_opt_i32(r)] }.to_h", + ); + assert_has(&src, "r.read_flag ? r.read_i32 : nil"); + // Records carrying objects reserve tokens and seal them before the call. + assert_has(&src, "w.write_object(v.cart, Cart)"); + assert_has(&src, "cart: r.read_object(Cart)"); +} + +#[test] +fn iterator_is_a_lazy_enumerator_adopting_objects() { + let src = bindings(); + let all = item(&src, "def self.all_carts"); + assert_has(all, "Enumerator.new do |y|"); + assert_has( + all, + "has_item = shop_kit_shop_AllCartsIterator_next(iter, out_item, err)", + ); + assert_has(all, "value = Cart._from_ptr(_wv_nonnull(item))"); + assert_has( + all, + "shop_kit_shop_AllCartsIterator_destroy(iter) unless iter.null?", + ); +} + +#[test] +fn cancellable_async_takes_a_cancel_token() { + let src = bindings(); + let wait = item(&src, "def self.wait(ms, cancel: nil)"); + assert_has(wait, "own_token = CancelToken.new if cancel.nil?"); + assert_has( + wait, + "shop_kit_shop_wait(ms, token._wv_ptr, SHOP_KIT_SHOP_WAIT_CALLBACK, ctx)", + ); + assert_has(wait, "_wv_async_wait(queue, token)"); + assert_has(wait, "own_token&.close"); + assert_has( + &src, + "SHOP_KIT_SHOP_WAIT_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :int64]) do |ctx, err, result|", + ); +} + +#[test] +fn rendering_is_deterministic() { + assert_eq!(bindings(), bindings()); +} + +#[test] +fn package_skips_platforms_without_a_gem_string() { + let model = model(); + let mut binaries = BinarySet::new("shop_kit"); + for (platform, path) in [ + (Platform::MacosArm64, "/tmp/darwin-arm64/libshop_kit.dylib"), + (Platform::AndroidArm64, "/tmp/android-arm64/libshop_kit.so"), + (Platform::Wasm32, "/tmp/wasm32/shop_kit.wasm"), + ] { + binaries.insert(NativeBinary::new(platform, path)); + } + let ctx = PackageContext::new(&binaries); + let artifacts = RubyGenerator + .package(&model, &ctx, &RubyConfig::default()) + .expect("ruby supports packaging"); + assert_eq!(artifacts.len(), 1, "{artifacts:?}"); + let gem = &artifacts[0]; + assert_eq!(gem.path, "ruby/shop_kit-0.1.0-arm64-darwin.gem"); + let ArtifactKind::Gem(spec) = &gem.kind else { + panic!("a gem"); + }; + assert_eq!(spec.platform, "arm64-darwin"); + assert_eq!( + spec.dependencies, + [("ffi".to_string(), "~> 1.15".to_string())] + ); + let paths: Vec<&str> = gem.files.iter().map(|f| f.path.as_str()).collect(); + assert_eq!( + paths, + [ + "lib/shop_kit.rb", + "lib/shop_kit/runtime.rb", + "lib/native/libshop_kit.dylib", + "README.md", + ] + ); +} + +#[test] +fn members_never_replace_the_wrapper_methods() { + let yaml = r#" +version: "0.11.0" +modules: + - name: kv + interfaces: + - name: Store + constructors: + - { name: allocate, params: [] } + methods: + - { name: close, params: [], return: bool } + - { name: handle, params: [], return: i64 } + - { name: initialize, params: [] } + - { name: name, params: [], return: string } + statics: + - { name: hash, params: [], return: i64 } + - { name: label, params: [], return: string } +"#; + let api: Api = serde_yaml::from_str(yaml).unwrap(); + let model = validate(&api, &Identity::named("kv"), None).unwrap(); + let files = RubyGenerator.files(&model, Utf8Path::new("out"), &RubyConfig::default()); + let rb = file(&files, "out/ruby/lib/kv.rb"); + for def in [ + "def self.allocate_", + "def close_", + "def handle_", + "def initialize_", + "def name\n", + "def self.hash\n", + "def self.label\n", + ] { + assert!(rb.contains(def), "missing `{def}` in:\n{rb}"); + } + assert!(!rb.contains("def close\n") && !rb.contains("def handle\n")); +} diff --git a/crates/weaveffi-cli/src/targets/ruby/types.rs b/crates/weaveffi-cli/src/targets/ruby/types.rs new file mode 100644 index 00000000..a3f62370 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/ruby/types.rs @@ -0,0 +1,146 @@ +//! Ruby type mapping and naming: the FFI-gem vocabulary for ABI slots, the +//! identifier policy for user-chosen IDL names, and string escaping for Ruby +//! literals. + +use crate::lang::{escape_ident, RUBY_KEYWORDS}; +use heck::ToSnakeCase; +use weaveffi_model::abi::{AbiParam, CType}; +use weaveffi_model::ty::{Prim, Ty}; + +/// The Ruby spelling of a user-chosen parameter name: snake_case via heck, +/// then keyword-escaped (a reserved name like `end` gains a trailing `_`). +/// IDL names are usually already snake, so the case conversion is a safety +/// net for camelCase inputs. +pub(crate) fn rb_param_name(name: &str) -> String { + escape_ident(&name.to_snake_case(), RUBY_KEYWORDS) +} + +/// The Ruby spelling of a user-chosen field name: verbatim unless it's a +/// Ruby reserved word, in which case it gains a trailing `_` so keyword +/// arguments and locals derived from it stay parseable. +pub(crate) fn rb_field_name(name: &str) -> String { + escape_ident(name, RUBY_KEYWORDS) +} + +/// Instance methods every interface wrapper defines or relies on: the +/// runtime's `WvObject` (`close`, `handle`, `initialize_copy`), the +/// constructor hook `initialize`, and the `Object` methods the runtime and +/// Ruby's own object model call (`class`, `dup`, `clone`, `hash`, ...). +/// The runtime's private helpers start with `_wv_`, which a snake-cased IDL +/// name never does. +const OBJECT_METHODS: &[&str] = &[ + "class", + "clone", + "close", + "dup", + "freeze", + "handle", + "hash", + "initialize", + "initialize_copy", + "object_id", + "send", +]; + +/// Class methods every interface wrapper class relies on: `allocate` +/// (adopting a returned object) and `name` (error messages). +const CLASS_METHODS: &[&str] = &["allocate", "name"]; + +/// The Ruby spelling of an interface member: snake_case, with a trailing +/// `_` when it would replace a method the wrapper relies on, an instance +/// method for a `method`, else a class method (a static or factory). +/// Ruby allows keywords as method names, so no keyword escape applies. +pub(crate) fn rb_member_name(name: &str, method: bool) -> String { + let members = if method { + OBJECT_METHODS + } else { + CLASS_METHODS + }; + crate::lang::escape_member(&name.to_snake_case(), members) +} + +/// Maps a shared ABI [`CType`] onto its Ruby FFI type symbol. Every pointer +/// (strings and buffers cross as `const uint8_t*`, objects and out-slots as +/// typed pointers) is `:pointer`; `bool` is the 1-byte C `bool`. +pub(crate) fn rb_ffi_type(ty: &CType) -> &'static str { + match ty { + CType::Bool => ":bool", + CType::Int8 => ":int8", + CType::Int16 => ":int16", + CType::Int32 | CType::Enum { .. } => ":int32", + CType::Uint8 => ":uint8", + CType::Uint16 => ":uint16", + CType::Uint32 => ":uint32", + CType::Int64 => ":int64", + CType::Uint64 => ":uint64", + CType::Float => ":float", + CType::Double => ":double", + CType::Size => ":size_t", + CType::Void => ":void", + _ => ":pointer", + } +} + +/// Map lowered ABI slots onto Ruby FFI type symbols. +pub(crate) fn rb_abi_types(params: &[AbiParam]) -> Vec { + params + .iter() + .map(|p| rb_ffi_type(&p.ty).to_string()) + .collect() +} + +/// The `FFI::MemoryPointer` type of a direct iterator element out-slot, read +/// back with `Pointer#read`. +pub(crate) fn rb_direct_type(ty: &Ty) -> &'static str { + match ty { + Ty::Prim(Prim::Bool) => ":bool", + Ty::Prim(Prim::I8) => ":int8", + Ty::Prim(Prim::I16) => ":int16", + Ty::Prim(Prim::I32) | Ty::Enum(_) => ":int32", + Ty::Prim(Prim::U8) => ":uint8", + Ty::Prim(Prim::U16) => ":uint16", + Ty::Prim(Prim::U32) => ":uint32", + Ty::Prim(Prim::I64) => ":int64", + Ty::Prim(Prim::U64) => ":uint64", + Ty::Prim(Prim::F32) => ":float", + Ty::Prim(Prim::F64) => ":double", + other => unreachable!("{other} is not a direct type"), + } +} + +/// Escape a string for embedding in a single-quoted Ruby literal (the two +/// characters with meaning there: backslash and the quote itself). +pub(crate) fn rb_str_literal(s: &str) -> String { + s.replace('\\', "\\\\").replace('\'', "\\'") +} + +/// The YARD type of a value of `ty`, for `@param` and `@return` tags: +/// `Integer`, `String`, `Array`, `Hash{String => Integer}`, +/// `Store, nil`. +pub(crate) fn rb_doc_type(ty: &Ty) -> String { + match ty { + Ty::Prim(Prim::Bool) => "Boolean".to_string(), + Ty::Prim(Prim::F32 | Prim::F64) => "Float".to_string(), + Ty::Prim(Prim::String | Prim::Bytes) => "String".to_string(), + Ty::Prim(_) | Ty::Enum(_) => "Integer".to_string(), + Ty::Record(n) | Ty::RichEnum(n) | Ty::Interface(n) | Ty::CallbackInterface(n) => n.clone(), + Ty::Optional(inner) => format!("{}, nil", rb_doc_type(inner)), + Ty::List(inner) => format!("Array<{}>", rb_doc_type(inner)), + Ty::Map(k, v) => format!("Hash{{{} => {}}}", rb_doc_type(k), rb_doc_type(v)), + Ty::Iterator(inner) => format!("Enumerator<{}>", rb_doc_type(inner)), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn interface_members_avoid_the_wrapper_methods() { + assert_eq!(rb_member_name("close", true), "close_"); + assert_eq!(rb_member_name("initialize", true), "initialize_"); + assert_eq!(rb_member_name("name", true), "name"); + assert_eq!(rb_member_name("name", false), "name_"); + assert_eq!(rb_member_name("close", false), "close"); + } +} diff --git a/crates/weaveffi-cli/src/targets/swift/callbacks.rs b/crates/weaveffi-cli/src/targets/swift/callbacks.rs new file mode 100644 index 00000000..39a129cb --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/callbacks.rs @@ -0,0 +1,299 @@ +//! Callback-interface rendering: the Swift `protocol` a consumer implements +//! and the process-wide vtable of `@convention(c)` trampolines the library +//! calls through. +//! +//! Every clause of [`CallbackProtocol`] is rendered here: one static vtable +//! per interface with the `{size, flags, free}` header, a context that +//! retains the implementation (released by `free`), arguments received per +//! [`RetPass`] (borrowed strings, bytes, and buffers are copied or decoded; +//! object arguments are adopted), returns handed back per family (a direct +//! value by value, an object as a fresh reference, a string, bytes, or +//! buffer as a `{prefix}_alloc` run in the out slots), and failures reported +//! through `out_err` instead of unwinding through the C frame: a method +//! declared `throws` reports its module's domain error with its code and +//! payload, and any other error is a callback failure (`-4`). + +use crate::cabi::c_param_name; +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use crate::lang; +use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding}; +use weaveffi_model::plan::{CallbackProtocol, ErrorStrategy, RetPass}; +use weaveffi_model::ty::{Prim, Ty}; + +use crate::targets::swift::docs::emit_fn_doc; +use crate::targets::swift::types::{callback_vtable_name, deprecated_attr, swift_ident, SwiftCtx}; + +/// The Swift spelling of one trampoline closure formal: the ABI slot name, +/// keyword-escaped (a param named `in` yields a slot named `in`, which can't +/// bind as a closure formal unescaped). +fn slot_ident(name: &str) -> String { + lang::escape_ident(name, lang::SWIFT_KEYWORDS) +} + +/// Clone a callback method with its parameter names camel-cased and +/// keyword-escaped, so the protocol requirement's argument labels and the +/// trampoline's call site agree. The ABI slot names are left untouched. +fn camel_method(m: &CallbackMethodBinding) -> CallbackMethodBinding { + let mut m = m.clone(); + for p in &mut m.params { + p.name = swift_ident(&p.name); + } + m +} + +/// The expression handing one trampoline argument to the implementation, +/// received per its [`RetPass`]: borrowed strings, bytes, and buffers are +/// copied or decoded before the implementation runs (the library owns them +/// only for the call); object arguments carry one strong reference the new +/// wrapper adopts. +fn receive_arg(p: &ParamBinding, rp: &RetPass, ctx: &SwiftCtx) -> String { + let n0 = slot_ident(&p.abi[0].name); + let n1 = || slot_ident(&p.abi[1].name); + match rp { + RetPass::Direct => match &p.ty { + Ty::Enum(name) => format!( + "wvEnumCase({}.self, numericCast({n0}.rawValue))", + ctx.ty_name(name) + ), + _ => n0, + }, + RetPass::String => format!("wvBorrowString({n0}, {})", n1()), + RetPass::Bytes => format!("wvBorrowBytes({n0}, {})", n1()), + RetPass::Buffer => format!( + "wvBorrowBuffer({n0}, {}, as: {}.self)", + n1(), + ctx.swift_type(&p.ty) + ), + RetPass::Object { nullable } => { + let wrapper = ctx.ty_name( + p.ty.interface_name() + .expect("object argument names an interface"), + ); + if *nullable { + format!("{n0}.map {{ {wrapper}(ptr: $0) }}") + } else { + format!("{wrapper}(ptr: wvNonNull({n0}))") + } + } + RetPass::Void => unreachable!("a parameter is never void"), + } +} + +/// Render the `public protocol` the consumer conforms to: one `throws` +/// requirement per method with idiomatic labels and types. Implementations +/// are classes (`AnyObject`) the library may call from any thread +/// (`Sendable`). +fn render_protocol( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + domain: Option<&str>, + ctx: &SwiftCtx, +) { + let name = ctx.ty_name(&cb.name); + if cb.doc.is_some() { + w.doc(&cb.doc, DocCommentStyle::TripleSlash); + w.line("///"); + } + w.line("/// Implement this protocol with a class and pass an instance where the API"); + w.line("/// expects it. The library keeps the instance until it no longer needs it and"); + w.line("/// may call any method from any thread. A method that throws fails the"); + w.line("/// library's call in progress: a method documented to throw a module error"); + w.line("/// reports that error, with its code and fields, and any other error reaches"); + w.line("/// the library as a callback failure (code -4)."); + if let Some(msg) = &cb.deprecated { + w.line(deprecated_attr(msg)); + } + w.line(format!("public protocol {name}: AnyObject, Sendable {{")); + w.indent(); + for (i, m) in cb.methods.iter().enumerate() { + let m = camel_method(m); + if i > 0 { + w.blank(); + } + let throws = match (m.throws, domain) { + (true, Some(ty)) => vec![format!( + "- Throws: ``{ty}`` to report a declared failure with its fields, or any other error to fail the call." + )], + _ => Vec::new(), + }; + emit_fn_doc(w, &m.doc, &m.params, &throws); + if let Some(msg) = &m.deprecated { + w.line(deprecated_attr(msg)); + } + let params = m + .params + .iter() + .map(|p| format!("{}: {}", p.name, ctx.swift_type(&p.ty))) + .collect::>() + .join(", "); + let ret = m + .ret + .as_ref() + .map(|t| format!(" -> {}", ctx.swift_type(t))) + .unwrap_or_default(); + w.line(format!( + "func {}({params}) throws{ret}", + swift_ident(&m.name) + )); + } + w.dedent(); + w.line("}"); + w.blank(); +} + +/// How one method's result crosses back: the implementation call wrapped +/// into the trampoline's return expression, and the value returned on +/// failure. +fn return_shape( + m: &CallbackMethodBinding, + rp: &RetPass, + call: &str, + ctx: &SwiftCtx, +) -> (String, String) { + let out = || { + let slots = &m.abi_params[m.abi_params.len() - 3..m.abi_params.len() - 1]; + (slot_ident(&slots[0].name), slot_ident(&slots[1].name)) + }; + match rp { + RetPass::Void => (format!("try {call}"), "()".to_string()), + RetPass::Direct => match &m.ret { + Some(Ty::Enum(name)) => { + let c_type = ctx.c_enum_type(name); + ( + format!("try {c_type}(rawValue: numericCast({call}.rawValue))"), + format!("{c_type}(rawValue: 0)"), + ) + } + Some(Ty::Prim(Prim::Bool)) => (format!("try {call}"), "false".to_string()), + _ => (format!("try {call}"), "0".to_string()), + }, + // One strong reference the library adopts. + RetPass::Object { nullable } => { + let q = if *nullable { "?" } else { "" }; + (format!("try {call}{q}.clonePtr()"), "nil".to_string()) + } + RetPass::String | RetPass::Bytes | RetPass::Buffer => { + let helper = match rp { + RetPass::String => "wvReturnString", + RetPass::Bytes => "wvReturnBytes", + _ => "wvReturnBuffer", + }; + let (ptr, len) = out(); + ( + format!("try {helper}({call}, {ptr}, {len})"), + "()".to_string(), + ) + } + } +} + +/// Render one trampoline closure literal (the vtable entry for `m`) as a +/// labeled argument of the vtable struct initializer. `i` indexes the +/// method in `cb` and `protocol`; `report` names the domain reporter a +/// throwing method passes along. +fn render_trampoline( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + protocol: &CallbackProtocol<'_>, + i: usize, + report: Option<&str>, + ctx: &SwiftCtx, +) { + let m = &cb.methods[i]; + let proto = ctx.ty_name(&cb.name); + let report = report.filter(|_| protocol.method_errors[i] == ErrorStrategy::Throws); + // No trailing comma: argument lists accept one only from Swift 6.1. + let sep = if i + 1 == cb.methods.len() { "" } else { "," }; + let camel = camel_method(m); + let formals = m + .abi_params + .iter() + .map(|s| slot_ident(&s.name)) + .collect::>() + .join(", "); + let ctx_slot = slot_ident(&m.abi_params[0].name); + let err_slot = slot_ident(&m.abi_params[m.abi_params.len() - 1].name); + let call_args = camel + .params + .iter() + .zip(&protocol.method_args[i]) + .map(|(p, rp)| format!("{}: {}", p.name, receive_arg(p, rp, ctx))) + .collect::>() + .join(", "); + let call = format!("wvImpl.{}({call_args})", swift_ident(&m.name)); + let (body, fallback) = return_shape(m, &protocol.method_returns[i], &call, ctx); + let domain = report.map(|r| format!(", domain: {r}")).unwrap_or_default(); + // The vtable field is spelled exactly as the C header declares it, so a + // method whose name is a C keyword carries the header's escape. + w.line(format!("{}: {{ {formals} in", c_param_name(&m.name))); + w.scope(|w| { + w.line(format!( + "wvInvoke({ctx_slot}, {err_slot}, as: (any {proto}).self, fallback: {fallback}{domain}) {{ wvImpl in" + )); + w.scope(|w| { + w.line(body); + }); + w.line("}"); + }); + w.line(format!("}}{sep}")); +} + +/// Render the process-wide vtable namespace: `shared` holds the C struct +/// (its header, then capture-free trampolines, implicitly `@convention(c)`) +/// at a stable address the library can hold for the process lifetime. +fn render_vtable( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + protocol: &CallbackProtocol<'_>, + report: Option<&str>, + ctx: &SwiftCtx, +) { + let proto = ctx.ty_name(&cb.name); + let vtable_name = callback_vtable_name(&cb.name); + let vtable_tag = &cb.vtable_tag; + w.line(format!( + "/// The process-wide vtable the library calls `{proto}` implementations" + )); + w.line(format!( + "/// through (`{vtable_tag}`). Every entry recovers the implementation" + )); + w.line("/// from `ctx`, copies or adopts its arguments, hands its result back, and"); + w.line("/// reports a thrown error through `out_err` instead of unwinding."); + w.line(format!("enum {vtable_name} {{")); + w.indent(); + w.line(format!("static let shared = WvVtable({vtable_tag}(")); + w.indent(); + w.line(format!("size: UInt32(MemoryLayout<{vtable_tag}>.stride),")); + w.line("flags: 0,"); + w.line("free: { ctx in"); + w.scope(|w| { + w.line(format!("wvRelease(ctx, as: (any {proto}).self)")); + }); + w.line("},"); + for i in 0..cb.methods.len() { + render_trampoline(w, cb, protocol, i, report, ctx); + } + w.dedent(); + w.line("))"); + w.dedent(); + w.line("}"); + w.blank(); +} + +/// Render everything one callback interface contributes at file scope: the +/// public protocol and the vtable namespace whose `shared.pointer` every +/// call site passing this interface hands to the library. `domain` is the +/// `(type name, stem)` of the error domain in scope for its module, which +/// throwing methods report through `wvReport{Stem}`. +pub(crate) fn render_swift_callback_interface( + w: &mut CodeWriter, + cb: &CallbackInterfaceBinding, + domain: Option<(&str, &str)>, + ctx: &SwiftCtx, +) { + let protocol = cb.protocol(); + let report = domain.map(|(_, stem)| format!("wvReport{stem}")); + render_protocol(w, cb, domain.map(|d| d.0), ctx); + render_vtable(w, cb, &protocol, report.as_deref(), ctx); +} diff --git a/crates/weaveffi-gen/src/targets/swift/calls.rs b/crates/weaveffi-cli/src/targets/swift/calls.rs similarity index 76% rename from crates/weaveffi-gen/src/targets/swift/calls.rs rename to crates/weaveffi-cli/src/targets/swift/calls.rs index 68ec473b..ca4bfaf8 100644 --- a/crates/weaveffi-gen/src/targets/swift/calls.rs +++ b/crates/weaveffi-cli/src/targets/swift/calls.rs @@ -5,27 +5,24 @@ //! return handling on [`RetPass`], so how a value crosses the ABI is decided //! centrally; this module only renders the Swift spelling. -use std::fmt::Write; - use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use weaveffi_model::model::{CallShape, FnBinding, IteratorBinding, ParamBinding, Ty}; -use weaveffi_model::plan::{ret_pass, ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::model::{CallShape, FnBinding, IteratorBinding, ParamBinding}; +use weaveffi_model::plan::{ArgPass, ErrorStrategy, RetPass}; +use weaveffi_model::ty::{Prim, Ty}; -use crate::targets::swift::codec::{decode_closure, write_value_stmts}; use crate::targets::swift::docs::emit_fn_doc; use crate::targets::swift::types::{ - callback_box_name, callback_vtable_name, deprecated_attr, iterator_class_name, - scalar_swift_type, swift_ident, SwiftCtx, + callback_vtable_name, deprecated_attr, iterator_class_name, scalar_swift_type, swift_ident, + SwiftCtx, }; /// How a wrapper reports a filled error slot. /// /// A callable declaring `throws` maps codes through its domain's typed -/// mapper (`checkKv`/`mapKv`) and throws; a cancellable async callable also -/// throws (`CancellationError` for code `-5`). Any other callable has a plain -/// signature and stops the process with `fatalError`, naming the code and -/// message, since a reported error can only be a runtime trap. +/// mapper (`wvCheckKv`/`wvMapKv`) and throws; a cancellable async callable +/// also throws (`CancellationError` for code `-5`). Any other callable has a +/// plain signature and stops the process with `fatalError`, naming the code +/// and message, since a reported error can only be a producer bug. #[derive(Clone, Copy)] pub(crate) struct ErrCtx<'a> { /// `true` when the Swift signature `throws`. @@ -34,30 +31,33 @@ pub(crate) struct ErrCtx<'a> { typed: bool, /// `true` for a cancellable async callable. cancellable: bool, - /// PascalCase stem of the domain in effect (`Kv` names `checkKv` and - /// `mapKv`); `None` falls back to the runtime helpers. + /// The domain error type in effect, when the callable is typed. + domain_type: Option<&'a str>, + /// PascalCase stem of the domain in effect (`Kv` names `wvCheckKv` and + /// `wvMapKv`); `None` falls back to the runtime helpers. domain: Option<&'a str>, } impl<'a> ErrCtx<'a> { /// Build the error context for `f` from its [`ErrorStrategy`] and - /// cancellability. - pub(crate) fn for_fn(f: &FnBinding, domain: Option<&'a str>) -> Self { + /// cancellability, given the `(type name, stem)` of the domain in scope. + pub(crate) fn for_fn(f: &FnBinding, domain: Option<(&'a str, &'a str)>) -> Self { let typed = f.error_strategy() == ErrorStrategy::Throws; - let cancellable = f.is_async && f.cancellable; + let cancellable = f.is_async() && f.cancellable; Self { throws: typed || cancellable, typed, cancellable, - domain, + domain_type: domain.map(|d| d.0), + domain: domain.map(|d| d.1), } } /// The mapper turning an error slot into a Swift error, when the wrapper - /// throws: the domain's `map{Stem}`, else the runtime's. + /// throws: the domain's `wvMap{Stem}`, else the runtime's. fn mapper(&self) -> String { match (self.typed, self.domain) { - (true, Some(stem)) => format!("map{stem}"), + (true, Some(stem)) => format!("wvMap{stem}"), (true, None) => "wvRuntimeError".to_string(), (false, _) => "wvCancelledOrTrap".to_string(), } @@ -66,11 +66,33 @@ impl<'a> ErrCtx<'a> { /// The statement checking the error slot of a synchronous call. fn check_stmt(&self) -> String { match (self.typed, self.domain) { - (true, Some(stem)) => format!("try check{stem}(&err)"), + (true, Some(stem)) => format!("try wvCheck{stem}(&err)"), (true, None) => "try wvCheck(&err)".to_string(), (false, _) => "wvTrap(&err)".to_string(), } } + + /// The `- Throws:` doc callout of a throwing wrapper. + fn throws_doc(&self, runtime_error: &str) -> Option { + if !self.throws { + return None; + } + let mut doc = match (self.typed, self.domain_type) { + (true, Some(ty)) => { + format!("- Throws: ``{ty}`` for a declared failure, ``{runtime_error}`` otherwise") + } + (true, None) => format!("- Throws: ``{runtime_error}``"), + (false, _) => "- Throws: `CancellationError`".to_string(), + }; + if self.cancellable && self.typed { + doc.push_str(", and `CancellationError`"); + } + if self.cancellable { + doc.push_str(" when the calling task is cancelled, which cancels the native call"); + } + doc.push('.'); + Some(doc) + } } /// Clone a callable with its parameter names camel-cased and keyword-escaped, @@ -86,10 +108,10 @@ pub(crate) fn camel_params(f: &FnBinding) -> FnBinding { } /// Where a wrapper sits, which decides how it reaches its object and whether -/// it runs the contract check. +/// it runs the load-time checks. #[derive(Clone, Copy, PartialEq, Eq)] pub(crate) enum Receiver { - /// A free function or static member: runs the contract check first. + /// A free function or static member: runs the load-time checks first. Static, /// An instance method passing its own object as the leading argument. Instance, @@ -98,41 +120,33 @@ pub(crate) enum Receiver { } /// Render one wrapper for `f` (sync, async, or iterator-returning) named -/// `swift_name`, at the indentation depth `depth`. +/// `swift_name`. pub(crate) fn render_callable( - out: &mut String, + w: &mut CodeWriter, f: &FnBinding, swift_name: &str, receiver: Receiver, err: ErrCtx, ctx: &SwiftCtx, - depth: usize, ) { - let mut w = CodeWriter::four_space().with_depth(depth); - { - let mut tmp = String::new(); - emit_fn_doc(&mut tmp, &f.doc, &f.params, &w.indent_str()); - w.raw(tmp); - } + let mut callouts = Vec::new(); if let CallShape::Iterator(_) = &f.shape { - w.line("/// - Returns: A lazy sequence that pulls one element per step from the"); - w.line("/// library; the underlying iterator is destroyed when the sequence is"); - w.line("/// exhausted or deinitialized."); - } - if err.cancellable { - w.line("/// - Throws: `CancellationError` when the calling task is cancelled, which"); - w.line("/// also cancels the native call."); + callouts.push( + "- Returns: A lazy sequence that pulls one element per step from the library." + .to_string(), + ); } + callouts.extend(err.throws_doc(&ctx.runtime_error)); + emit_fn_doc(w, &f.doc, &f.params, &callouts); if let Some(msg) = &f.deprecated { w.line(deprecated_attr(msg)); } - let mut sig = String::new(); - for (i, p) in f.params.iter().enumerate() { - if i > 0 { - sig.push_str(", "); - } - let _ = write!(sig, "{}: {}", p.name, ctx.swift_type(&p.ty)); - } + let sig = f + .params + .iter() + .map(|p| format!("{}: {}", p.name, ctx.swift_type(&p.ty))) + .collect::>() + .join(", "); let is_async = matches!(f.shape, CallShape::Async(_)); let effects = match (is_async, err.throws) { (true, true) => " async throws", @@ -165,13 +179,12 @@ pub(crate) fn render_callable( w.line("wvLoad()"); } if is_async { - render_async_body(&mut w, f, receiver, err, ctx); + render_async_body(w, f, receiver, err, ctx); } else { - render_sync_body(&mut w, f, receiver, err, ctx); + render_sync_body(w, f, receiver, err, ctx); } w.dedent(); w.line("}"); - out.push_str(&w.finish()); } /// The C argument list for `params`, prefixed with the receiver's object for @@ -194,12 +207,17 @@ fn c_call_args(params: &[ParamBinding], self_arg: Option<&str>, ctx: &SwiftCtx) format!("{n}.ptr") }); } - ArgPass::Callback { .. } => { + ArgPass::Callback { nullable, .. } => { let cb = p.ty.callback_interface_name() .expect("callback family names a callback interface"); + let vtable = format!("{}.shared.pointer", callback_vtable_name(cb)); args.push(format!("{n}_ctx")); - args.push(format!("{}.shared.pointer", callback_vtable_name(cb))); + args.push(if nullable { + format!("{n} == nil ? nil : {vtable}") + } else { + vtable + }); } ArgPass::Direct { .. } => match &p.ty { Ty::Enum(name) => args.push(format!( @@ -213,29 +231,18 @@ fn c_call_args(params: &[ParamBinding], self_arg: Option<&str>, ctx: &SwiftCtx) args } -/// Emit the pre-call staging: buffered params are packed into a writer, and -/// callback interfaces are boxed with a +1 retain the library's `free` entry -/// releases. +/// Emit the pre-call staging of callback parameters: each implementation is +/// retained for the library, whose vtable `free` entry releases it (also +/// when the call fails). fn render_arg_staging(w: &mut CodeWriter, params: &[ParamBinding]) { - let mut counter = 0usize; for p in params { - match p.arg_pass() { - ArgPass::Buffer { .. } => { - let writer = format!("{}Writer", p.name); - w.line(format!("var {writer} = WvWriter()")); - write_value_stmts(w, &p.ty, &p.name, &writer, &mut counter); - } - ArgPass::Callback { .. } => { - let cb = - p.ty.callback_interface_name() - .expect("callback family names a callback interface"); - w.line(format!( - "let {n}_ctx = Unmanaged.passRetained({}({n})).toOpaque()", - callback_box_name(cb), - n = p.name - )); + if let ArgPass::Callback { nullable, .. } = p.arg_pass() { + let n = &p.name; + if nullable { + w.line(format!("let {n}_ctx = {n}.map {{ wvRetain($0) }}")); + } else { + w.line(format!("let {n}_ctx = wvRetain({n})")); } - _ => {} } } } @@ -251,7 +258,7 @@ fn open_lending_closures(w: &mut CodeWriter, params: &[ParamBinding], bind: &str let open = match p.arg_pass() { ArgPass::String { .. } => format!("wvWithUTF8({n})"), ArgPass::Bytes { .. } => format!("wvWithBytes({n})"), - ArgPass::Buffer { .. } => format!("wvWithBuffer({n}Writer)"), + ArgPass::Buffer { .. } => format!("wvWithEncoded({n})"), _ => continue, }; let prefix = if opened == 0 { bind } else { "" }; @@ -287,16 +294,9 @@ fn raw_return_swift(f: &FnBinding, rp: Option<&RetPass>, ctx: &SwiftCtx) -> Stri } } -/// The interface name behind an object-returning callable (direct or -/// nullable). -fn ret_interface_name(ty: Option<&Ty>) -> &str { - ty.and_then(Ty::interface_name) - .expect("an object return names an interface") -} - /// Render the body of a synchronous (or iterator-returning) callable: the -/// error slot, input staging, the C call wrapped in whatever lending closures -/// the inputs need, the error check, and the return conversion. +/// error slot, callback staging, the C call wrapped in whatever lending +/// closures the inputs need, the error check, and the return conversion. fn render_sync_body( w: &mut CodeWriter, f: &FnBinding, @@ -311,7 +311,7 @@ fn render_sync_body( // protocol rather than a `RetPass`. let (symbol, rp) = match &f.shape { CallShape::Iterator(it) => (&it.launch.symbol, None), - CallShape::Sync(abi) => (&abi.symbol, Some(ret_pass(f.ret.as_ref(), ctx.c_prefix))), + CallShape::Sync(abi) => (&abi.symbol, Some(RetPass::of(f.ret.as_ref()))), CallShape::Async(_) => unreachable!("async callables render through render_async_body"), }; let needs_out_len = matches!(rp, Some(RetPass::String | RetPass::Bytes | RetPass::Buffer)); @@ -360,7 +360,7 @@ fn render_sync_body( w.line("self.ptr = wvNonNull(rv)"); } Some(rp) => { - let value = receive_value(w, f.ret.as_ref(), &rp, "rv", "outLen", ctx); + let value = receive_value(f.ret.as_ref(), &rp, "rv", "outLen", ctx); w.line(format!("return {value}")); } } @@ -369,20 +369,13 @@ fn render_sync_body( /// The Swift expression converting a value the library returned (an owned /// result) into its Swift form, releasing what the conversion copied. /// `raw` names the result slot and `len` its length slot, if any. -fn receive_value( - w: &CodeWriter, - ty: Option<&Ty>, - rp: &RetPass, - raw: &str, - len: &str, - ctx: &SwiftCtx, -) -> String { +fn receive_value(ty: Option<&Ty>, rp: &RetPass, raw: &str, len: &str, ctx: &SwiftCtx) -> String { match rp { RetPass::Void => "()".to_string(), RetPass::Direct => match ty { Some(Ty::Enum(name)) => format!( "wvEnumCase({}.self, numericCast({raw}.rawValue))", - ctx.ty_name(local_type_name(name)) + ctx.ty_name(name) ), _ => raw.to_string(), }, @@ -390,15 +383,16 @@ fn receive_value( RetPass::Bytes => format!("wvTakeBytes({raw}, {len})"), RetPass::Buffer => { let ty = ty.expect("a buffered value carries a type"); - let decode = decode_closure(w, ty, ctx); - if decode.starts_with('{') { - format!("wvTakeBuffer({raw}, {len}) {decode}") - } else { - format!("wvTakeBuffer({raw}, {len}, {decode})") - } + format!( + "wvTakeBuffer({raw}, {len}, as: {}.self)", + ctx.swift_type(ty) + ) } - RetPass::Object { nullable, .. } => { - let class = ctx.ty_name(local_type_name(ret_interface_name(ty))); + RetPass::Object { nullable } => { + let class = ctx.ty_name( + ty.and_then(Ty::interface_name) + .expect("an object return names an interface"), + ); if *nullable { format!("{raw}.map {{ {class}(ptr: $0) }}") } else { @@ -421,7 +415,7 @@ fn render_async_body( let CallShape::Async(binding) = &f.shape else { unreachable!("render_async_body renders async callables") }; - let protocol = binding.protocol(f, ctx.c_prefix); + let protocol = binding.protocol(f); let ret = f .ret .as_ref() @@ -456,10 +450,12 @@ fn render_async_body( if protocol.cancellable { args.push("token.raw".to_string()); } - let slots = match &protocol.result { - RetPass::Void => "context, err", - RetPass::String | RetPass::Bytes | RetPass::Buffer => "context, err, resultPtr, resultLen", - _ => "context, err, result", + let (slots, raw) = match &protocol.result { + RetPass::Void => ("context, err", "result"), + RetPass::String | RetPass::Bytes | RetPass::Buffer => { + ("context, err, resultPtr, resultLen", "resultPtr") + } + _ => ("context, err, result", "result"), }; let mut launch = args.join(", "); if !launch.is_empty() { @@ -483,12 +479,7 @@ fn render_async_body( } w.dedent(); w.line("}"); - let raw = if slots.ends_with("resultLen") { - "resultPtr" - } else { - "result" - }; - let value = receive_value(w, f.ret.as_ref(), &protocol.result, raw, "resultLen", ctx); + let value = receive_value(f.ret.as_ref(), &protocol.result, raw, "resultLen", ctx); w.line(format!("cont.resume(returning: {value})")); w.dedent(); w.line("}, context)"); @@ -518,13 +509,13 @@ fn render_async_body( /// `error` property; for a non-throwing one a reported error stops the /// process. pub(crate) fn render_swift_iterator_class( - out: &mut String, + w: &mut CodeWriter, f: &FnBinding, it: &IteratorBinding, err: ErrCtx, ctx: &SwiftCtx, ) { - let protocol = it.protocol(f, ctx.c_prefix); + let protocol = it.protocol(f); let class_name = iterator_class_name(it, ctx.c_prefix); let elem = &it.elem; let elem_swift = ctx.swift_type(elem); @@ -541,13 +532,12 @@ pub(crate) fn render_swift_iterator_class( RetPass::Object { .. } => "var item: OpaquePointer? = nil".to_string(), RetPass::Direct => match elem { Ty::Enum(name) => format!("var item = {}(rawValue: 0)", ctx.c_enum_type(name)), - Ty::Bool => "var item = false".to_string(), + Ty::Prim(Prim::Bool) => "var item = false".to_string(), other => format!("var item: {} = 0", scalar_swift_type(other)), }, RetPass::Void => unreachable!("an iterator element is never void"), }; - let mut w = CodeWriter::four_space(); w.line(format!( "/// A lazy sequence over the `{elem_swift}` elements streamed by `{}`.", it.launch.symbol @@ -621,12 +611,11 @@ pub(crate) fn render_swift_iterator_class( w.line("return nil"); w.dedent(); w.line("}"); - let value = receive_value(&w, Some(elem), &protocol.elem, "item", "itemLen", ctx); + let value = receive_value(Some(elem), &protocol.elem, "item", "itemLen", ctx); w.line(format!("return {value}")); w.dedent(); w.line("}"); w.dedent(); w.line("}"); w.blank(); - out.push_str(&w.finish()); } diff --git a/crates/weaveffi-cli/src/targets/swift/codec.rs b/crates/weaveffi-cli/src/targets/swift/codec.rs new file mode 100644 index 00000000..256bc22a --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/codec.rs @@ -0,0 +1,149 @@ +//! Value-buffer codecs: each user type's `WvCodable` conformance. +//! +//! The runtime's `WvCodable` protocol covers every primitive and, through +//! conditional conformances, every optional, list, and map whose elements +//! conform, so a composite type like `[String: [Entry?]]` has exactly one +//! encoder and one decoder (the generic ones) instead of a loop inlined at +//! each use. This module renders the conformances of the declared types: a +//! record reads and writes its fields in order, a rich enum its `i32` tag and +//! then the variant's fields, a C-style enum its discriminant, and an +//! interface an object token carrying one strong reference. + +use crate::codegen::CodeWriter; +use weaveffi_model::model::{EnumBinding, InterfaceBinding, StructBinding}; + +use crate::targets::swift::types::{deprecated_attr, swift_ident, SwiftCtx}; + +/// Open a conformance extension of `ty`, marked deprecated with the type +/// itself so it can name it without a warning. +fn open_extension(w: &mut CodeWriter, ty: &str, protocol: &str, deprecated: Option<&str>) { + if let Some(msg) = deprecated { + w.line(deprecated_attr(msg)); + } + w.line(format!("extension {ty}: {protocol} {{")); +} + +/// Render a C-style enum's conformance: its discriminant, via the runtime's +/// `WvCEnum` defaults. +pub(crate) fn render_c_enum_codec(w: &mut CodeWriter, e: &EnumBinding, ctx: &SwiftCtx) { + if let Some(msg) = &e.deprecated { + w.line(deprecated_attr(msg)); + } + w.line(format!("extension {}: WvCEnum {{}}", ctx.ty_name(&e.name))); + w.blank(); +} + +/// Render a record's conformance: its fields in declaration order. +pub(crate) fn render_record_codec(w: &mut CodeWriter, s: &StructBinding, ctx: &SwiftCtx) { + let ty = ctx.ty_name(&s.name); + open_extension(w, &ty, "WvCodable", s.deprecated.as_deref()); + w.scope(|w| { + let reader = if s.fields.is_empty() { "_" } else { "r" }; + let args = s + .fields + .iter() + .map(|f| format!("{}: r.read()", swift_ident(&f.name))) + .collect::>() + .join(", "); + w.line(format!( + "static func wvRead(_ {reader}: inout WvReader) -> {ty} {{" + )); + w.scope(|w| { + w.line(format!("{ty}({args})")); + }); + w.line("}"); + w.blank(); + let writer = if s.fields.is_empty() { "_" } else { "w" }; + w.line(format!("func wvWrite(_ {writer}: inout WvWriter) {{")); + w.scope(|w| { + for f in &s.fields { + w.line(format!("w.write(self.{})", swift_ident(&f.name))); + } + }); + w.line("}"); + }); + w.line("}"); + w.blank(); +} + +/// Render a rich enum's conformance: the `i32` tag, then the active +/// variant's fields in order. Reading an undeclared tag is a malformed +/// buffer. +pub(crate) fn render_rich_enum_codec(w: &mut CodeWriter, e: &EnumBinding, ctx: &SwiftCtx) { + let ty = ctx.ty_name(&e.name); + open_extension(w, &ty, "WvCodable", e.deprecated.as_deref()); + w.scope(|w| { + w.line(format!( + "static func wvRead(_ r: inout WvReader) -> {ty} {{" + )); + w.scope(|w| { + w.line("let tag = r.readI32()"); + w.line("switch tag {"); + for v in &e.variants { + let case = swift_ident(&v.name); + if v.fields.is_empty() { + w.line(format!("case {}: return .{case}", v.value)); + } else { + let args = v + .fields + .iter() + .map(|f| format!("{}: r.read()", swift_ident(&f.name))) + .collect::>() + .join(", "); + w.line(format!("case {}: return .{case}({args})", v.value)); + } + } + w.line(format!( + "default: wvDecodeFailure(\"unknown {} tag \\(tag)\")", + e.name + )); + w.line("}"); + }); + w.line("}"); + w.blank(); + w.line("func wvWrite(_ w: inout WvWriter) {"); + w.scope(|w| { + w.line("switch self {"); + for v in &e.variants { + let case = swift_ident(&v.name); + if v.fields.is_empty() { + w.line(format!("case .{case}:")); + w.scope(|w| { + w.line(format!("w.writeI32({})", v.value)); + }); + } else { + let binds = (0..v.fields.len()) + .map(|i| format!("v{i}")) + .collect::>(); + w.line(format!("case let .{case}({}):", binds.join(", "))); + w.scope(|w| { + w.line(format!("w.writeI32({})", v.value)); + for b in &binds { + w.line(format!("w.write({b})")); + } + }); + } + } + w.line("}"); + }); + w.line("}"); + }); + w.line("}"); + w.blank(); +} + +/// Render an interface's conformance: an object token. Reading adopts the +/// token's reference into a new wrapper; writing mints a fresh one, so the +/// wrapper keeps its own. +pub(crate) fn render_interface_codec(w: &mut CodeWriter, iface: &InterfaceBinding, ctx: &SwiftCtx) { + let ty = ctx.ty_name(&iface.name); + open_extension(w, &ty, "WvCodable", iface.deprecated.as_deref()); + w.scope(|w| { + w.line(format!( + "static func wvRead(_ r: inout WvReader) -> {ty} {{ {ty}(ptr: r.readObject()) }}" + )); + w.line("func wvWrite(_ w: inout WvWriter) { w.writeObject(clonePtr()) }"); + }); + w.line("}"); + w.blank(); +} diff --git a/crates/weaveffi-cli/src/targets/swift/docs.rs b/crates/weaveffi-cli/src/targets/swift/docs.rs new file mode 100644 index 00000000..4f6c4832 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/docs.rs @@ -0,0 +1,47 @@ +//! Swift doc-comment emission in Swift's markup: an item's summary and +//! discussion, then its callouts (`- Parameter`, `- Returns:`, `- Throws:`). + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use weaveffi_model::model::ParamBinding; + +/// Emit the doc comment of a callable: its own doc, then a +/// `/// - Parameter name: ...` callout for each documented parameter, then +/// `callouts` (each a `- Returns:` or `- Throws:` line without the `///`), +/// separated from the discussion by an empty doc line. Callers pass params +/// whose names are already camel-cased and keyword-escaped (see +/// `calls::camel_params`), so the labels match the emitted signature. +pub(crate) fn emit_fn_doc( + w: &mut CodeWriter, + doc: &Option, + params: &[ParamBinding], + callouts: &[String], +) { + let mut lines = Vec::new(); + for p in params { + let Some(pdoc) = p.doc.as_deref().map(str::trim) else { + continue; + }; + let mut pdoc = pdoc.lines(); + let Some(first) = pdoc.next() else { + continue; + }; + lines.push(format!("/// - Parameter {}: {first}", p.name)); + for line in pdoc { + lines.push(if line.is_empty() { + "///".to_string() + } else { + format!("/// {line}") + }); + } + } + lines.extend(callouts.iter().map(|c| format!("/// {c}"))); + let has_doc = doc.as_deref().is_some_and(|d| !d.trim().is_empty()); + w.doc(doc, DocCommentStyle::TripleSlash); + if has_doc && !lines.is_empty() { + w.line("///"); + } + for line in lines { + w.line(line); + } +} diff --git a/crates/weaveffi-cli/src/targets/swift/entities.rs b/crates/weaveffi-cli/src/targets/swift/entities.rs new file mode 100644 index 00000000..9f3d0693 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/entities.rs @@ -0,0 +1,476 @@ +//! Entity rendering: enums, rich enums, records, interfaces, typed error +//! domains, and the recursive module walk emitting them (callback interfaces +//! render through [`crate::targets::swift::callbacks`], value-buffer codecs +//! through [`crate::targets::swift::codec`]). + +use crate::codegen::common::DocCommentStyle; +use crate::codegen::CodeWriter; +use heck::ToUpperCamelCase; +use weaveffi_model::model::{ + CallShape, EnumBinding, ErrorBinding, FieldBinding, FnBinding, InterfaceBinding, Model, + ModuleBinding, StructBinding, +}; + +use crate::targets::swift::callbacks::render_swift_callback_interface; +use crate::targets::swift::calls::{ + camel_params, render_callable, render_swift_iterator_class, ErrCtx, Receiver, +}; +use crate::targets::swift::codec::{ + render_c_enum_codec, render_interface_codec, render_record_codec, render_rich_enum_codec, +}; +use crate::targets::swift::types::{ + deprecated_attr, swift_ident, swift_member, swift_str, SwiftCtx, +}; + +/// The PascalCase helper stem of an error domain, naming its +/// `wvCheck{Stem}`/`wvMap{Stem}`/`wvReport{Stem}` helpers (derived from the +/// *declaring* module's path, so inheriting submodules reference the +/// ancestor's helpers). +fn stem(eb: &ErrorBinding) -> String { + eb.owner_path.to_upper_camel_case() +} + +/// The `(type name, stem)` of the error domain in scope for `module`. +fn domain_of(model: &Model, module: &ModuleBinding) -> Option<(String, String)> { + model + .error_domain(module) + .map(|e| (e.type_name.clone(), stem(e))) +} + +/// Borrow a `(type name, stem)` pair. +fn as_refs(d: &Option<(String, String)>) -> Option<(&str, &str)> { + d.as_ref().map(|(t, s)| (t.as_str(), s.as_str())) +} + +/// The conformance list of a value type: `Hashable` (which implies +/// `Equatable`) when every field is, and always `Sendable`. +fn value_conformances<'t>( + ctx: &SwiftCtx, + fields: impl IntoIterator, +) -> &'static str { + if fields.into_iter().all(|f| ctx.is_hashable(&f.ty)) { + "Hashable, Sendable" + } else { + "Sendable" + } +} + +/// Render a C-style enum as an `Int32` raw-value Swift enum, one case per +/// variant, plus its codec. Wrappers convert to and from the imported C +/// type at the call. +fn render_swift_enum(w: &mut CodeWriter, e: &EnumBinding, ctx: &SwiftCtx) { + w.doc(&e.doc, DocCommentStyle::TripleSlash); + if let Some(msg) = &e.deprecated { + w.line(deprecated_attr(msg)); + } + w.line(format!("public enum {}: Int32, Sendable {{", e.name)); + w.scope(|w| { + for v in &e.variants { + w.doc(&v.doc, DocCommentStyle::TripleSlash); + w.line(format!("case {} = {}", swift_ident(&v.name), v.value)); + } + }); + w.line("}"); + w.blank(); + render_c_enum_codec(w, e, ctx); +} + +/// Render a rich (algebraic) enum as a native Swift enum with associated +/// values: one case per variant, with labeled associated values matching the +/// variant's field names, plus its codec. +fn render_swift_rich_enum(w: &mut CodeWriter, e: &EnumBinding, ctx: &SwiftCtx) { + let fields = e.variants.iter().flat_map(|v| v.fields.iter()); + w.doc(&e.doc, DocCommentStyle::TripleSlash); + if let Some(msg) = &e.deprecated { + w.line(deprecated_attr(msg)); + } + w.line(format!( + "public enum {}: {} {{", + e.name, + value_conformances(ctx, fields) + )); + w.scope(|w| { + for v in &e.variants { + w.doc(&v.doc, DocCommentStyle::TripleSlash); + let case_name = swift_ident(&v.name); + if v.fields.is_empty() { + w.line(format!("case {case_name}")); + } else { + let assoc = v + .fields + .iter() + .map(|f| format!("{}: {}", swift_ident(&f.name), ctx.swift_type(&f.ty))) + .collect::>() + .join(", "); + w.line(format!("case {case_name}({assoc})")); + } + } + }); + w.line("}"); + w.blank(); + render_rich_enum_codec(w, e, ctx); +} + +/// Render a record as a plain Swift struct with one `public var` per field +/// and an explicit public memberwise initializer (the synthesized one is +/// internal), plus its codec. +fn render_swift_struct(w: &mut CodeWriter, s: &StructBinding, ctx: &SwiftCtx) { + w.doc(&s.doc, DocCommentStyle::TripleSlash); + if let Some(msg) = &s.deprecated { + w.line(deprecated_attr(msg)); + } + w.line(format!( + "public struct {}: {} {{", + s.name, + value_conformances(ctx, &s.fields) + )); + w.indent(); + for f in &s.fields { + w.doc(&f.doc, DocCommentStyle::TripleSlash); + w.line(format!( + "public var {}: {}", + swift_ident(&f.name), + ctx.swift_type(&f.ty) + )); + } + if !s.fields.is_empty() { + w.blank(); + } + let params = s + .fields + .iter() + .map(|f| format!("{}: {}", swift_ident(&f.name), ctx.swift_type(&f.ty))) + .collect::>() + .join(", "); + w.line("/// Creates a value from its fields."); + w.line(format!("public init({params}) {{")); + w.scope(|w| { + for f in &s.fields { + let prop = swift_ident(&f.name); + w.line(format!("self.{prop} = {prop}")); + } + }); + w.line("}"); + w.dedent(); + w.line("}"); + w.blank(); + render_record_codec(w, s, ctx); +} + +/// Render one declaring module's typed error surface: a `public enum +/// {TypeName}: Error` whose lowerCamel cases carry the message plus, for +/// codes that declare payload fields, one labeled associated value per +/// field. Also emits the file-scope helpers: `wvMap{Stem}` and +/// `wvCheck{Stem}` convert a filled error slot into it (decoding the payload +/// for codes with fields), and, when a callback method can throw it, +/// `wvReport{Stem}` reports it back through a callback's error slot. +/// +/// Only declared codes get typed cases. Domain codes are validated positive, +/// so the mapper's `default` arm is what every reserved negative runtime +/// code falls through to: the runtime error, or `CancellationError` for `-5`. +fn render_swift_error( + w: &mut CodeWriter, + module: &ModuleBinding, + eb: &ErrorBinding, + ctx: &SwiftCtx, +) { + let stem = stem(eb); + let ty = &eb.type_name; + + w.doc( + &Some(format!( + "Typed errors reported by the `{}` module.", + module.segments.join(".") + )), + DocCommentStyle::TripleSlash, + ); + w.line(format!( + "public enum {ty}: Error, LocalizedError, Sendable {{" + )); + w.indent(); + for c in &eb.codes { + w.doc(&c.doc, DocCommentStyle::TripleSlash); + let mut parts = vec!["message: String".to_string()]; + parts.extend( + c.fields + .iter() + .map(|f| format!("{}: {}", swift_ident(&f.name), ctx.swift_type(&f.ty))), + ); + w.line(format!( + "case {}({})", + swift_ident(&c.name), + parts.join(", ") + )); + } + w.blank(); + w.line("/// The message the library reported, or the code's documented default."); + w.line("public var errorDescription: String? {"); + w.scope(|w| { + w.line("switch self {"); + for c in &eb.codes { + // Bind only the message; wildcard the payload fields. + let mut binds = vec!["message".to_string()]; + binds.extend(c.fields.iter().map(|_| "_".to_string())); + w.line(format!( + "case let .{}({}): return message", + swift_ident(&c.name), + binds.join(", ") + )); + } + w.line("}"); + }); + w.line("}"); + w.blank(); + w.line("/// The numeric ABI code carried by this error."); + w.line("public var errorCode: Int32 {"); + w.scope(|w| { + w.line("switch self {"); + for c in &eb.codes { + w.line(format!( + "case .{}: return {}", + swift_ident(&c.name), + c.value + )); + } + w.line("}"); + }); + w.line("}"); + w.dedent(); + w.line("}"); + w.blank(); + + // `wvMap{Stem}`: code -> typed case (default message when the slot + // carried none), decoding the payload buffer for codes with fields. + w.line(format!("func wvMap{stem}(_ err: WvError) -> Error {{")); + w.indent(); + w.line("let message = wvErrorMessage(err)"); + w.line("switch err.code {"); + for c in &eb.codes { + let case_name = swift_ident(&c.name); + let message_arg = format!( + "message: message.isEmpty ? \"{}\" : message", + swift_str(&c.message) + ); + if c.fields.is_empty() { + w.line(format!( + "case {}: return {ty}.{case_name}({message_arg})", + c.value + )); + } else { + let mut args = vec![message_arg]; + args.extend( + c.fields + .iter() + .map(|f| format!("{}: r.read()", swift_ident(&f.name))), + ); + w.line(format!("case {}:", c.value)); + w.scope(|w| { + w.line("var r = WvReader(err.payload_ptr, err.payload_len)"); + w.line(format!("let error = {ty}.{case_name}({})", args.join(", "))); + w.line("r.finish()"); + w.line("return error"); + }); + } + } + w.line("default: return wvRuntimeError(err)"); + w.line("}"); + w.dedent(); + w.line("}"); + w.blank(); + + w.line(format!( + "func wvCheck{stem}(_ err: inout WvError) throws {{" + )); + w.scope(|w| { + w.line("guard err.code != 0 else { return }"); + w.line(format!("let error = wvMap{stem}(err)")); + w.line(format!("{}_error_clear(&err)", ctx.c_prefix)); + w.line("throw error"); + }); + w.line("}"); + w.blank(); + + if !ctx.reports(eb) { + return; + } + w.line(format!( + "/// Reports a ``{ty}`` thrown by a callback method declared to throw it: its" + )); + w.line("/// code, its message, and its fields as the payload. Any other error is left"); + w.line("/// to the caller, which reports a callback failure."); + w.line(format!( + "func wvReport{stem}(_ error: Error, _ outErr: UnsafeMutablePointer?) -> Bool {{" + )); + w.scope(|w| { + w.line(format!( + "guard let error = error as? {ty} else {{ return false }}" + )); + w.line("switch error {"); + for c in &eb.codes { + let case_name = swift_ident(&c.name); + if c.fields.is_empty() { + w.line(format!("case let .{case_name}(message):")); + w.scope(|w| { + w.line(format!("wvSetError(outErr, {}, message)", c.value)); + }); + } else { + let binds = (0..c.fields.len()) + .map(|i| format!("v{i}")) + .collect::>(); + w.line(format!( + "case let .{case_name}(message, {}):", + binds.join(", ") + )); + w.scope(|w| { + w.line("var payload = WvWriter()"); + for b in &binds { + w.line(format!("payload.write({b})")); + } + w.line(format!("wvSetError(outErr, {}, message, payload)", c.value)); + }); + } + } + w.line("}"); + w.line("return true"); + }); + w.line("}"); + w.blank(); +} + +/// Render one interface as a `public final class` owning one strong +/// reference to its object: a stored `ptr`, an internal reference-adopting +/// `init(ptr:)`, a `deinit` that releases the reference exactly once through +/// the destroy symbol, and an internal `clonePtr()` that mints a second +/// reference for positions that take ownership (an object token inside a +/// value buffer, a callback method's return). The native object is +/// thread-safe, so the class is `@unchecked Sendable`. +/// +/// The constructor named `new` surfaces as `public init`; every other +/// constructor becomes a `public static func` factory. Methods are instance +/// funcs passing `ptr` as the leading C argument; statics are plain `public +/// static func`s. +fn render_swift_interface( + w: &mut CodeWriter, + module: &ModuleBinding, + iface: &InterfaceBinding, + ctx: &SwiftCtx, +) { + let domain = domain_of(ctx.model, module); + w.doc(&iface.doc, DocCommentStyle::TripleSlash); + if let Some(msg) = &iface.deprecated { + w.line(deprecated_attr(msg)); + } + w.line(format!( + "public final class {}: @unchecked Sendable {{", + iface.name + )); + w.indent(); + w.line("let ptr: OpaquePointer"); + w.blank(); + w.line("init(ptr: OpaquePointer) {"); + w.scope(|w| { + w.line("self.ptr = ptr"); + }); + w.line("}"); + w.blank(); + w.line("deinit {"); + w.scope(|w| { + w.line(format!("{}(ptr)", iface.destroy_symbol)); + }); + w.line("}"); + w.blank(); + w.line("/// Returns a new strong reference to the same object, for a position that"); + w.line("/// takes ownership of it."); + w.line("func clonePtr() -> OpaquePointer {"); + w.scope(|w| { + w.line(format!("wvNonNull({}(ptr))", iface.clone_symbol)); + }); + w.line("}"); + + let constructors = iface.constructors.iter().map(|f| { + let receiver = if f.name == "new" { + Receiver::Init + } else { + Receiver::Static + }; + (f, receiver) + }); + let members = constructors + .chain(iface.methods.iter().map(|f| (f, Receiver::Instance))) + .chain(iface.statics.iter().map(|f| (f, Receiver::Static))); + for (f, receiver) in members { + let f = camel_params(f); + let err = ErrCtx::for_fn(&f, as_refs(&domain)); + w.blank(); + render_callable(w, &f, &swift_member(&f.name), receiver, err, ctx); + } + w.dedent(); + w.line("}"); + w.blank(); + render_interface_codec(w, iface, ctx); +} + +/// Emit every file-scope type one module contributes: the typed error +/// surface, enums and records with their codecs, callback-interface +/// protocols with their vtables, interface classes, and the sequence classes +/// backing `iter` callables. +pub(crate) fn render_swift_module_types(w: &mut CodeWriter, mb: &ModuleBinding, ctx: &SwiftCtx) { + if let Some(eb) = mb.errors.as_ref() { + render_swift_error(w, mb, eb, ctx); + } + for e in &mb.enums { + if e.is_rich() { + render_swift_rich_enum(w, e, ctx); + } else { + render_swift_enum(w, e, ctx); + } + } + for s in &mb.structs { + render_swift_struct(w, s, ctx); + } + let domain = domain_of(ctx.model, mb); + for cb in &mb.callback_interfaces { + render_swift_callback_interface(w, cb, as_refs(&domain), ctx); + } + for i in &mb.interfaces { + render_swift_interface(w, mb, i, ctx); + } + for f in mb.callables() { + if let CallShape::Iterator(it) = &f.shape { + render_swift_iterator_class(w, f, it, ErrCtx::for_fn(f, as_refs(&domain)), ctx); + } + } +} + +/// Emit one module's namespace `enum`: its function wrappers, then one +/// nested namespace per submodule. +pub(crate) fn render_swift_namespace( + w: &mut CodeWriter, + model: &Model, + mb: &ModuleBinding, + ctx: &SwiftCtx, +) { + let domain = domain_of(model, mb); + w.doc(&mb.doc, DocCommentStyle::TripleSlash); + w.line(format!("public enum {} {{", mb.name.to_upper_camel_case())); + w.indent(); + let mut first = true; + for f in &mb.functions { + if !first { + w.blank(); + } + first = false; + let f: FnBinding = camel_params(f); + let err = ErrCtx::for_fn(&f, as_refs(&domain)); + render_callable(w, &f, &swift_ident(&f.name), Receiver::Static, err, ctx); + } + for sub in model.children(mb) { + if !first { + w.blank(); + } + first = false; + render_swift_namespace(w, model, sub, ctx); + } + w.dedent(); + w.line("}"); +} diff --git a/crates/weaveffi-cli/src/targets/swift/mod.rs b/crates/weaveffi-cli/src/targets/swift/mod.rs new file mode 100644 index 00000000..c79201a6 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/mod.rs @@ -0,0 +1,279 @@ +//! Swift binding generator. +//! +//! Emits a standalone SwiftPM package: the Swift wrapper module over the C +//! ABI, plus a `C{Module}` system-library target holding a copy of the C +//! header and a module map that links the native library. Implements +//! [`LanguageBackend`]; the shared driver bridges it into the generator +//! pipeline. +//! +//! Records, rich enums, optionals, lists, and maps cross the C ABI as value +//! buffers. The wrapper ships a small private writer/reader pair +//! (`WvWriter`/`WvReader`, decoding in place from the library's memory) and a +//! `WvCodable` protocol: every primitive conforms, optionals, arrays, and +//! dictionaries conform generically, and each record, enum, and interface +//! conforms where it's declared, so records surface as plain Swift structs +//! and rich enums as native Swift enums with associated values, and every +//! composite type has one codec. Objects surface as `final class` wrappers +//! owning one strong reference each, callback interfaces as class-bound +//! `Sendable` protocols whose implementations cross the boundary through a +//! process-wide vtable of `@convention(c)` trampolines, and async functions +//! as `async` functions over checked continuations, with task cancellation +//! wired to the native cancel token. Before the first call the wrapper checks +//! the ABI revision and every top-level module's contract table. +//! +//! The fixed Swift sources (runtime, manifest, module map, README) live +//! under `runtime/` and are spliced with the package's names. + +mod callbacks; +mod calls; +mod codec; +mod docs; +mod entities; +mod package; +mod runtime; +#[cfg(test)] +mod tests; +mod types; + +use crate::backend::{LanguageBackend, OutputFile}; +use crate::codegen::CodeWriter; +use crate::package::{Artifact, PackageContext, PackagedFile}; +use crate::platform::Os; +use crate::targets::c::{header_name, render_c_header_from_model}; +use crate::utils::{render_prelude, render_trailer, CommentStyle}; +use camino::{Utf8Path, Utf8PathBuf}; +use serde::{Deserialize, Serialize}; +use weaveffi_model::model::Model; + +use crate::targets::swift::entities::{render_swift_module_types, render_swift_namespace}; +use crate::targets::swift::package::{ + render_modulemap, render_package_swift, render_packaged_readme, CSource, +}; +use crate::targets::swift::runtime::{render_load_checks, render_runtime, Names}; +use crate::targets::swift::types::SwiftCtx; + +/// The URL a packaged manifest points its binary target at when +/// `xcframework_url` isn't set: a placeholder to replace before publishing. +pub const PLACEHOLDER_XCFRAMEWORK_URL: &str = + "https://example.invalid/set-generators.swift.xcframework_url/{file}"; + +/// Per-target configuration for [`SwiftGenerator`]. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct SwiftConfig { + /// SwiftPM package, product, and module name. Defaults to the package + /// name in PascalCase (`kvstore` becomes `Kvstore`); the C module is + /// always this name with a `C` prefix. + pub name: Option, + /// The minimum macOS version the manifest declares (default `11.0`, the + /// `[build]` default deployment target). + pub min_macos: String, + /// The minimum iOS version the manifest declares (default `13.0`, the + /// `[build]` default deployment target). + pub min_ios: String, + /// The URL `weaveffi package` points the packaged manifest's binary + /// target at, where you upload `C{Module}.xcframework.zip`. `{version}` + /// is replaced by the package version and `{file}` by the archive's + /// file name. Defaults to [`PLACEHOLDER_XCFRAMEWORK_URL`]. + pub xcframework_url: Option, +} + +impl Default for SwiftConfig { + fn default() -> Self { + Self { + name: None, + min_macos: "11.0".into(), + min_ios: "13.0".into(), + xcframework_url: None, + } + } +} + +impl SwiftConfig { + /// The Swift module (and package and product) name for `model`. + #[must_use] + pub fn module_name(&self, model: &Model) -> String { + self.name + .clone() + .unwrap_or_else(|| model.identity.pascal_name()) + } + + /// The C module (and binary target) name: the module name with a `C` + /// prefix. + #[must_use] + pub fn c_module_name(&self, model: &Model) -> String { + format!("C{}", self.module_name(model)) + } + + /// The URL of the `XCFramework` archive `file` for `version`. + #[must_use] + pub fn xcframework_url(&self, version: &str, file: &str) -> String { + self.xcframework_url + .as_deref() + .unwrap_or(PLACEHOLDER_XCFRAMEWORK_URL) + .replace("{version}", version) + .replace("{file}", file) + } +} + +/// The resolved names of one generated package. +struct Layout { + /// The Swift module (and package and product) name. + module: String, + /// The C module name, `C{module}`. + c_module: String, + /// The native library base name. + library: String, + /// The C header file name. + header: String, +} + +impl Layout { + fn new(model: &Model, config: &SwiftConfig) -> Self { + let identity = &model.identity; + let module = config.module_name(model); + Self { + c_module: format!("C{module}"), + module, + library: identity.library.clone(), + header: header_name(model), + } + } + + fn names(&self) -> Names<'_> { + Names { + module: &self.module, + c_module: &self.c_module, + library: &self.library, + header: &self.header, + } + } + + /// The `(path, contents)` of every source file of the package rooted at + /// `dir`. + fn sources( + &self, + model: &Model, + dir: &Utf8Path, + config: &SwiftConfig, + ) -> Vec<(Utf8PathBuf, String)> { + let c_dir = dir.join("Sources").join(&self.c_module); + let swift_file = format!("{}.swift", self.module); + vec![ + ( + dir.join("Package.swift"), + render_package_swift(&self.names(), config, &CSource::SystemLibrary), + ), + ( + c_dir.join("module.modulemap"), + render_modulemap(&self.names()), + ), + ( + c_dir.join(&self.header), + render_c_header_from_model(model, &self.header), + ), + ( + dir.join("Sources").join(&self.module).join(&swift_file), + render_swift_wrapper(self, model, &swift_file), + ), + ] + } +} + +/// Swift backend: emits a standalone SwiftPM package wrapping the C ABI. +pub struct SwiftGenerator; + +impl LanguageBackend for SwiftGenerator { + type Config = SwiftConfig; + + fn name(&self) -> &'static str { + "swift" + } + + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { + Layout::new(model, config) + .sources(model, &out_dir.join("swift"), config) + .into_iter() + .map(|(path, contents)| OutputFile::new(path, contents)) + .collect() + } + + /// The SwiftPM package at `swift/{Module}/` whose binary target is the + /// `C{Module}.xcframework.zip` archive `weaveffi package` assembles from + /// the Apple static libraries (at the configured `xcframework_url`, with + /// its checksum). Without an archive (no Apple platform built, or not on + /// macOS) there is nothing to package. + fn package( + &self, + model: &Model, + ctx: &PackageContext, + config: &Self::Config, + ) -> Option> { + let Some(archive) = ctx.xcframework else { + return Some(Vec::new()); + }; + let layout = Layout::new(model, config); + let names = layout.names(); + let url = config.xcframework_url(&model.identity.version, &archive.file_name); + let swift_file = format!("{}.swift", layout.module); + let slices = ctx + .binaries + .binaries + .iter() + .filter(|nb| matches!(nb.platform.os(), Os::MacOs | Os::Ios) && nb.staticlib.is_some()) + .map(|nb| format!("`{}`", nb.platform.id())) + .collect::>() + .join(", "); + let files = vec![ + PackagedFile::text( + "Package.swift", + render_package_swift( + &names, + config, + &CSource::Remote { + url: &url, + checksum: &archive.checksum, + }, + ), + ), + PackagedFile::text( + format!("Sources/{}/{swift_file}", layout.module), + render_swift_wrapper(&layout, model, &swift_file), + ), + PackagedFile::text( + "README.md", + render_packaged_readme(&names, &slices, &url, &archive.checksum), + ), + ]; + Some(vec![Artifact::directory( + format!("swift/{}", layout.module), + files, + )]) + } +} + +/// Render the complete Swift wrapper file: prelude and imports, the private +/// runtime, the load-time checks, every module's file-scope types, and one +/// namespace `enum` per top-level module. +fn render_swift_wrapper(layout: &Layout, model: &Model, filename: &str) -> String { + let ctx = SwiftCtx::new(model, &layout.module); + let mut w = CodeWriter::four_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + w.line(format!("import {}", layout.c_module)); + w.line("import Foundation"); + w.blank(); + w.raw(render_runtime(model, &layout.names(), &ctx.runtime_error)); + w.blank(); + render_load_checks(&mut w, model); + w.line("// MARK: - API"); + w.blank(); + for mb in &model.modules { + render_swift_module_types(&mut w, mb, &ctx); + } + for root in model.roots() { + render_swift_namespace(&mut w, model, root, &ctx); + w.blank(); + } + w.raw(render_trailer(CommentStyle::DoubleSlash, filename)); + w.finish() +} diff --git a/crates/weaveffi-cli/src/targets/swift/package.rs b/crates/weaveffi-cli/src/targets/swift/package.rs new file mode 100644 index 00000000..994793f4 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/package.rs @@ -0,0 +1,104 @@ +//! SwiftPM packaging surfaces: `Package.swift` (generated and packaged), the +//! system-library module map, and the packaged README, framed with the +//! generated-file prelude and trailer. + +use crate::utils::{render_prelude, render_trailer, CommentStyle}; + +use crate::targets::swift::runtime::{render_module_map, render_package, render_readme, Names}; +use crate::targets::swift::SwiftConfig; + +/// The SwiftPM tools version the manifest declares (Swift 5 language mode). +const TOOLS_VERSION: &str = "5.9"; + +/// Where a manifest's C module comes from when no `XCFramework` sits next to +/// it. +pub(crate) enum CSource<'a> { + /// The generated `Sources/C{Module}` system-library target, linking the + /// library from the linker search path. + SystemLibrary, + /// A published `XCFramework` archive. + Remote { + /// The archive's URL. + url: &'a str, + /// The archive's SHA-256 checksum. + checksum: &'a str, + }, +} + +/// The manifest's `platforms:` list from the configured minimum versions. +pub(crate) fn platforms(config: &SwiftConfig) -> String { + format!( + ".macOS(\"{}\"), .iOS(\"{}\")", + config.min_macos, config.min_ios + ) +} + +/// Render `Package.swift`. The manifest picks the C module's source at +/// resolution time: a prebuilt `C{Module}.xcframework` next to it when one +/// exists, otherwise `source`. +pub(crate) fn render_package_swift( + names: &Names, + config: &SwiftConfig, + source: &CSource, +) -> String { + let c = names.c_module; + let (comment, fallback) = match source { + CSource::SystemLibrary => ( + format!( + "// The C module `{c}` comes from one of two places. Next to this\n\ + // manifest, a prebuilt `{c}.xcframework` (from `weaveffi package`) wins.\n\ + // Otherwise `Sources/{c}` declares the generated header as a system\n\ + // library that links `{}` from the linker search path, e.g.\n\ + // `swift build -Xlinker -L/path/to/lib`.", + names.library + ), + format!(".systemLibrary(name: \"{c}\")"), + ), + CSource::Remote { url, checksum } => ( + format!( + "// The C module `{c}` is the prebuilt `{c}.xcframework`, published as\n\ + // `{c}.xcframework.zip`. A copy unzipped next to this manifest wins,\n\ + // for local development." + ), + format!( + ".binaryTarget(\n name: \"{c}\",\n url: \"{url}\",\n \ + checksum: \"{checksum}\"\n )" + ), + ), + }; + // `swift-tools-version` must be the very first line of the manifest, so + // the generated-file prelude follows it. + format!( + "// swift-tools-version:{TOOLS_VERSION}\n{}{}\n{}", + render_prelude(CommentStyle::DoubleSlash), + render_package(names, &comment, &fallback, &platforms(config)), + render_trailer(CommentStyle::DoubleSlash, "Package.swift"), + ) +} + +/// Render the module map of the `C{Module}` system-library target, which +/// sits next to its copy of the C header. +pub(crate) fn render_modulemap(names: &Names) -> String { + format!( + "{}{}\n{}", + render_prelude(CommentStyle::DoubleSlash), + render_module_map(names), + render_trailer(CommentStyle::DoubleSlash, "module.modulemap"), + ) +} + +/// README for the packaged Swift package: the `XCFramework` slices and how +/// to publish the archive its binary target points at. +pub(crate) fn render_packaged_readme( + names: &Names, + slices: &str, + url: &str, + checksum: &str, +) -> String { + format!( + "{}{}\n{}", + render_prelude(CommentStyle::Xml), + render_readme(names, slices, url, checksum), + render_trailer(CommentStyle::Xml, "README.md"), + ) +} diff --git a/crates/weaveffi-cli/src/targets/swift/runtime.rs b/crates/weaveffi-cli/src/targets/swift/runtime.rs new file mode 100644 index 00000000..98d01f2c --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/runtime.rs @@ -0,0 +1,140 @@ +//! The fixed Swift sources the generator splices names into (the private +//! runtime with its codec, error helpers, and async and callback support; +//! the `Package.swift` manifest; the C module map; and the packaged README), +//! plus the load-time checks rendered from the model. Each fixed source +//! lives as a real file under `runtime/` with `{{PLACEHOLDER}}` markers. + +use crate::codegen::CodeWriter; +use weaveffi_model::contract::entries; +use weaveffi_model::model::{contract_symbol, Model, ABI_VERSION}; + +const RUNTIME: &str = include_str!("runtime/Runtime.swift"); +const PACKAGE: &str = include_str!("runtime/Package.swift"); +const MODULE_MAP: &str = include_str!("runtime/module.modulemap"); +const README: &str = include_str!("runtime/README.md"); + +/// Replace every `{{KEY}}` in `template` with its value. +/// +/// # Panics +/// +/// Panics (in debug builds) when a placeholder is left unfilled, which is a +/// bug in the caller's substitution list. +fn fill(template: &str, vars: &[(&str, &str)]) -> String { + let mut out = template.to_string(); + for (key, value) in vars { + out = out.replace(&format!("{{{{{key}}}}}"), value); + } + debug_assert!(!out.contains("{{"), "unfilled placeholder in:\n{out}"); + out +} + +/// The names the fixed sources are spliced with. +pub(crate) struct Names<'a> { + /// The Swift module (and package and product) name. + pub(crate) module: &'a str, + /// The C module name, `C{module}`. + pub(crate) c_module: &'a str, + /// The native library base name. + pub(crate) library: &'a str, + /// The C header file name. + pub(crate) header: &'a str, +} + +/// The private runtime the wrapper file starts with: error helpers, the +/// value-buffer codec, and async and callback support. +pub(crate) fn render_runtime(model: &Model, names: &Names, runtime_error: &str) -> String { + fill( + RUNTIME, + &[ + ("RUNTIME_ERROR", runtime_error), + ("PREFIX", model.prefix()), + ("MODULE", names.module), + ("LIBRARY", names.library), + ("ABI_VERSION", &ABI_VERSION.to_string()), + ], + ) +} + +/// Render the load-time checks the runtime's `wvLoad()` runs once: the ABI +/// revision, then every top-level module's contract table against the +/// entries these bindings were generated with. +pub(crate) fn render_load_checks(w: &mut CodeWriter, model: &Model) { + w.line("// MARK: - Load-time checks"); + w.blank(); + w.line("/// The load-time checks, run once before the first native call: the library"); + w.line(format!( + "/// must implement C ABI revision {ABI_VERSION} and carry every declaration these" + )); + w.line("/// bindings were generated with, unchanged."); + w.line("let wvContract: Void = {"); + w.scope(|w| { + w.line("wvCheckAbiVersion()"); + for root in model.roots() { + w.line(format!( + "wvCheckContract({}, [", + contract_symbol(model.prefix(), &root.name) + )); + w.scope(|w| { + for e in entries(model, root) { + w.line(format!( + "(0x{:016x}, 0x{:016x}, \"{}\"),", + e.id, e.hash, e.path + )); + } + }); + w.line("])"); + } + }); + w.line("}()"); + w.blank(); +} + +/// The `Package.swift` body (after the tools-version line and prelude): +/// `fallback` is the C target used when no `XCFramework` sits next to the +/// manifest, explained by `source_comment`, and `platforms` lists the +/// minimum OS versions. +pub(crate) fn render_package( + names: &Names, + source_comment: &str, + fallback: &str, + platforms: &str, +) -> String { + fill( + PACKAGE, + &[ + ("SOURCE_COMMENT", source_comment), + ("FALLBACK", fallback), + ("PLATFORMS", platforms), + ("MODULE", names.module), + ("C_MODULE", names.c_module), + ], + ) +} + +/// The system-library module map declaring the bundled header. +pub(crate) fn render_module_map(names: &Names) -> String { + fill( + MODULE_MAP, + &[ + ("C_MODULE", names.c_module), + ("HEADER", names.header), + ("LIBRARY", names.library), + ], + ) +} + +/// The packaged README: the `XCFramework`'s `slices`, and the `url` and +/// `checksum` its binary target resolves from. +pub(crate) fn render_readme(names: &Names, slices: &str, url: &str, checksum: &str) -> String { + fill( + README, + &[ + ("MODULE", names.module), + ("C_MODULE", names.c_module), + ("LIBRARY", names.library), + ("SLICES", slices), + ("URL", url), + ("CHECKSUM", checksum), + ], + ) +} diff --git a/crates/weaveffi-gen/src/targets/swift/runtime/Package.swift b/crates/weaveffi-cli/src/targets/swift/runtime/Package.swift similarity index 50% rename from crates/weaveffi-gen/src/targets/swift/runtime/Package.swift rename to crates/weaveffi-cli/src/targets/swift/runtime/Package.swift index 189fe68b..5359c3dd 100644 --- a/crates/weaveffi-gen/src/targets/swift/runtime/Package.swift +++ b/crates/weaveffi-cli/src/targets/swift/runtime/Package.swift @@ -1,19 +1,15 @@ import Foundation import PackageDescription -// The C module `{{C_MODULE}}` comes from one of two places. Next to this -// manifest, a prebuilt `{{C_MODULE}}.xcframework` (assembled from the native -// libraries, see the README) wins. Otherwise `Sources/{{C_MODULE}}` declares -// the generated header as a system library that links `{{LIBRARY}}` from the -// linker search path, e.g. `swift build -Xlinker -L/path/to/lib`. +{{SOURCE_COMMENT}} let xcframework = "{{C_MODULE}}.xcframework" let cModule: Target = FileManager.default.fileExists(atPath: "\(Context.packageDirectory)/\(xcframework)") ? .binaryTarget(name: "{{C_MODULE}}", path: xcframework) - : .systemLibrary(name: "{{C_MODULE}}") + : {{FALLBACK}} let package = Package( name: "{{MODULE}}", - platforms: [.macOS(.v10_15), .iOS(.v13), .tvOS(.v13), .watchOS(.v6)], + platforms: [{{PLATFORMS}}], products: [ .library(name: "{{MODULE}}", targets: ["{{MODULE}}"]), ], diff --git a/crates/weaveffi-cli/src/targets/swift/runtime/README.md b/crates/weaveffi-cli/src/targets/swift/runtime/README.md new file mode 100644 index 00000000..01eee375 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/runtime/README.md @@ -0,0 +1,28 @@ +# {{MODULE}} (Swift) + +A SwiftPM package (product and module `{{MODULE}}`) over the native library +`{{LIBRARY}}`. Its C declarations are the binary target `{{C_MODULE}}`: a +prebuilt `{{C_MODULE}}.xcframework` of static libraries ({{SLICES}}), so apps +link the library with no flags or search paths. + +## Publish + +`Package.swift` resolves the binary target from + +```text +{{URL}} +``` + +with checksum `{{CHECKSUM}}`. Upload `{{C_MODULE}}.xcframework.zip` (next to +this directory in the dist folder) to that URL, then commit this directory to +the repository your consumers depend on and tag the release. Set +`[generators.swift] xcframework_url` before packaging to choose the URL. + +## Local development + +A `{{C_MODULE}}.xcframework` next to `Package.swift` takes precedence over the +URL, so unzipping the archive here makes the package usable by path: + +```bash +unzip ../{{C_MODULE}}.xcframework.zip -d . +``` diff --git a/crates/weaveffi-gen/src/targets/swift/runtime/Runtime.swift b/crates/weaveffi-cli/src/targets/swift/runtime/Runtime.swift similarity index 56% rename from crates/weaveffi-gen/src/targets/swift/runtime/Runtime.swift rename to crates/weaveffi-cli/src/targets/swift/runtime/Runtime.swift index 0d306318..560b7527 100644 --- a/crates/weaveffi-gen/src/targets/swift/runtime/Runtime.swift +++ b/crates/weaveffi-cli/src/targets/swift/runtime/Runtime.swift @@ -1,7 +1,7 @@ // MARK: - Runtime support /// A failure the native library reports outside any declared error domain: -/// an unknown error code or a runtime trap. Throwing wrappers raise it; +/// an unknown error code or a runtime failure. Throwing wrappers raise it; /// non-throwing wrappers stop the process with `fatalError` instead, since /// Swift has no unchecked errors. public struct {{RUNTIME_ERROR}}: Error, LocalizedError, Hashable, Sendable { @@ -34,27 +34,41 @@ typealias WvError = {{PREFIX}}_error /// The C ABI revision these bindings were generated against. let wvAbiVersion: UInt32 = {{ABI_VERSION}} -/// The contract check, run once before the first native call: the library -/// must implement this ABI revision and every top-level module must carry -/// the checksum these bindings were generated from. -let wvContract: Void = { +/// Runs the load-time checks if they haven't run yet. +@inline(__always) +func wvLoad() { + _ = wvContract +} + +/// Stops the process unless the library implements the ABI revision these +/// bindings were generated against. +func wvCheckAbiVersion() { let abi = {{PREFIX}}_abi_version() guard abi == wvAbiVersion else { fatalError("{{MODULE}}: the native library '{{LIBRARY}}' implements C ABI revision \(abi), but these bindings need revision \(wvAbiVersion)") } -{{CONTRACT_CHECKS}} -}() - -/// Runs the contract check if it hasn't run yet. -@inline(__always) -func wvLoad() { - _ = wvContract } -/// Stops the process when a module's contract checksum doesn't match. -func wvCheckModule(_ module: StaticString, _ actual: UInt64, _ expected: UInt64) { - guard actual == expected else { - fatalError("{{MODULE}}: module '\(module)' of the native library '{{LIBRARY}}' doesn't match these bindings (checksum 0x\(String(actual, radix: 16)), expected 0x\(String(expected, radix: 16))); regenerate the bindings or rebuild the library") +/// Stops the process unless every declaration these bindings were generated +/// with is in a top-level module's contract table with an equal signature +/// hash. Declarations the library adds are fine. +func wvCheckContract( + _ table: (UnsafeMutablePointer?) -> UnsafePointer<{{PREFIX}}_contract_entry>?, + _ expected: [(id: UInt64, hash: UInt64, path: StaticString)] +) { + var len = 0 + let entries = UnsafeBufferPointer(start: table(&len), count: len) + var hashes: [UInt64: UInt64] = [:] + for entry in entries { + hashes[entry.id] = entry.hash + } + for (id, hash, path) in expected { + guard let actual = hashes[id] else { + fatalError("{{MODULE}}: \(path) is missing from the library '{{LIBRARY}}'; regenerate the bindings or rebuild the library") + } + guard actual == hash else { + fatalError("{{MODULE}}: \(path) changed since these bindings were generated; regenerate the bindings or rebuild the library '{{LIBRARY}}'") + } } } @@ -150,10 +164,12 @@ func wvWithBytes(_ d: Data, _ body: (UnsafePointer?, Int) throws -> R) try d.withUnsafeBytes { try body($0.baseAddress?.assumingMemoryBound(to: UInt8.self), $0.count) } } -/// Lends an encoded value buffer for one call. +/// Encodes `value` as a value buffer and lends it for one call. @inline(__always) -func wvWithBuffer(_ w: WvWriter, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { - try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } +func wvWithEncoded(_ value: T, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { + var w = WvWriter() + w.write(value) + return try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } } /// Copies a returned string, then releases the library's allocation. @@ -171,11 +187,11 @@ func wvTakeBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a returned value buffer in place, then releases it. -func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { defer { if let ptr = ptr { {{PREFIX}}_free_bytes(UnsafeMutablePointer(mutating: ptr), len) } } - return wvBorrowBuffer(ptr, len, decode) + return wvBorrowBuffer(ptr, len, as: type) } /// Copies a borrowed string. @@ -191,9 +207,9 @@ func wvBorrowBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a borrowed value buffer in place. -func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { var r = WvReader(UnsafeRawPointer(ptr), len) - let value = decode(&r) + let value: T = r.read() r.finish() return value } @@ -252,6 +268,12 @@ struct WvWriter { /// Writes an object token. `p` must be a strong reference the buffer now /// owns (a freshly cloned pointer), never one a wrapper still holds. mutating func writeObject(_ p: OpaquePointer) { append(UInt64(UInt(bitPattern: p))) } + + /// Writes any value that crosses inside a value buffer. + @inline(__always) + mutating func write(_ value: T) { + value.wvWrite(&self) + } } /// Deserializes values from the value-buffer wire format directly from the @@ -345,11 +367,154 @@ struct WvReader { return p } + /// Reads any value that crosses inside a value buffer, typed by context. + @inline(__always) + mutating func read() -> T { + T.wvRead(&self) + } + func finish() { if remaining != 0 { wvDecodeFailure("trailing bytes after value") } } } +// MARK: Value-buffer codecs + +/// A type that crosses the C ABI inside a value buffer. Every primitive, +/// optional, list, and map conforms here; each record, enum, and interface +/// conforms where it's declared. +protocol WvCodable { + /// Reads one value, adopting the reference of any object token. + static func wvRead(_ r: inout WvReader) -> Self + /// Writes this value, writing a fresh reference for any object. + func wvWrite(_ w: inout WvWriter) +} + +/// A C-style enum, which crosses as its `i32` discriminant. +protocol WvCEnum: WvCodable, RawRepresentable where RawValue == Int32 {} + +extension WvCEnum { + static func wvRead(_ r: inout WvReader) -> Self { wvEnumCase(Self.self, r.readI32()) } + func wvWrite(_ w: inout WvWriter) { w.writeI32(rawValue) } +} + +extension Bool: WvCodable { + static func wvRead(_ r: inout WvReader) -> Bool { r.readBool() } + func wvWrite(_ w: inout WvWriter) { w.writeBool(self) } +} + +extension Int8: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int8 { r.readI8() } + func wvWrite(_ w: inout WvWriter) { w.writeI8(self) } +} + +extension UInt8: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt8 { r.readU8() } + func wvWrite(_ w: inout WvWriter) { w.writeU8(self) } +} + +extension Int16: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int16 { r.readI16() } + func wvWrite(_ w: inout WvWriter) { w.writeI16(self) } +} + +extension UInt16: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt16 { r.readU16() } + func wvWrite(_ w: inout WvWriter) { w.writeU16(self) } +} + +extension Int32: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int32 { r.readI32() } + func wvWrite(_ w: inout WvWriter) { w.writeI32(self) } +} + +extension UInt32: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt32 { r.readU32() } + func wvWrite(_ w: inout WvWriter) { w.writeU32(self) } +} + +extension Int64: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int64 { r.readI64() } + func wvWrite(_ w: inout WvWriter) { w.writeI64(self) } +} + +extension UInt64: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt64 { r.readU64() } + func wvWrite(_ w: inout WvWriter) { w.writeU64(self) } +} + +extension Float: WvCodable { + static func wvRead(_ r: inout WvReader) -> Float { r.readF32() } + func wvWrite(_ w: inout WvWriter) { w.writeF32(self) } +} + +extension Double: WvCodable { + static func wvRead(_ r: inout WvReader) -> Double { r.readF64() } + func wvWrite(_ w: inout WvWriter) { w.writeF64(self) } +} + +extension String: WvCodable { + static func wvRead(_ r: inout WvReader) -> String { r.readString() } + func wvWrite(_ w: inout WvWriter) { w.writeString(self) } +} + +extension Data: WvCodable { + static func wvRead(_ r: inout WvReader) -> Data { r.readBytes() } + func wvWrite(_ w: inout WvWriter) { w.writeBytes(self) } +} + +extension Optional: WvCodable where Wrapped: WvCodable { + static func wvRead(_ r: inout WvReader) -> Wrapped? { + r.readOptionFlag() ? .some(r.read()) : nil + } + + func wvWrite(_ w: inout WvWriter) { + w.writeOptionFlag(self != nil) + if let value = self { w.write(value) } + } +} + +extension Array: WvCodable where Element: WvCodable { + static func wvRead(_ r: inout WvReader) -> [Element] { + let n = r.readCount() + var items: [Element] = [] + items.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0.. [Key: Value] { + let n = r.readCount() + var map: [Key: Value] = [:] + map.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0..: @unchecked Sendable { } } -/// Reports a thrown Swift error to the library as a callback failure -/// (code -4); the library aborts its current call with that code and message. -func wvForeignError(_ outErr: UnsafeMutablePointer?, _ error: Error) { - let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) - message.withCString { {{PREFIX}}_error_set(outErr, {{RUNTIME_ERROR}}.foreignCode, $0) } +/// Holds one callback implementation behind the `ctx` the library keeps. +final class WvBox: @unchecked Sendable { + let value: T + + init(_ value: T) { + self.value = value + } +} + +/// Retains `impl` for the library: the `ctx` of a callback parameter, which +/// the vtable's `free` entry releases with ``wvRelease(_:as:)``. +func wvRetain(_ impl: T) -> UnsafeMutableRawPointer { + Unmanaged.passRetained(WvBox(impl)).toOpaque() +} + +/// Releases the implementation behind `ctx`, from the vtable's `free` entry. +func wvRelease(_ ctx: UnsafeMutableRawPointer?, as type: T.Type) { + Unmanaged>.fromOpaque(ctx!).release() +} + +/// Runs one callback method on the implementation behind `ctx`. A thrown +/// error never unwinds through the library: `domain` reports it when it's +/// the method's declared error, anything else is reported as a callback +/// failure (`-4`), and the method returns `fallback`. +func wvInvoke( + _ ctx: UnsafeMutableRawPointer?, + _ outErr: UnsafeMutablePointer?, + as type: T.Type, + fallback: R, + domain: (Error, UnsafeMutablePointer?) -> Bool = { _, _ in false }, + _ body: (T) throws -> R +) -> R { + let impl = Unmanaged>.fromOpaque(ctx!).takeUnretainedValue().value + do { + return try body(impl) + } catch { + if !domain(error, outErr) { + let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) + message.withCString { {{PREFIX}}_error_set(outErr, {{RUNTIME_ERROR}}.foreignCode, $0) } + } + return fallback + } +} + +/// Reports a declared domain error from a callback method: its code, its +/// message, and its fields as the payload. +func wvSetError(_ outErr: UnsafeMutablePointer?, _ code: Int32, _ message: String, _ payload: WvWriter? = nil) { + message.withCString { {{PREFIX}}_error_set(outErr, code, $0) } + if let payload = payload { + payload.bytes.withUnsafeBufferPointer { {{PREFIX}}_error_set_payload(outErr, $0.baseAddress, $0.count) } + } +} + +/// Hands `bytes` to the library as a callback method's return: a run +/// allocated with `{{PREFIX}}_alloc` that the library adopts. +func wvHandOver(_ bytes: UnsafeRawBufferPointer, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + let run = {{PREFIX}}_alloc(bytes.count) + if let run = run, let base = bytes.baseAddress { + run.initialize(from: base.assumingMemoryBound(to: UInt8.self), count: bytes.count) + } + outPtr?.pointee = run + outLen?.pointee = run == nil ? 0 : bytes.count +} + +/// Returns a string from a callback method through its out slots. +func wvReturnString(_ value: String, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var value = value + value.withUTF8 { wvHandOver(UnsafeRawBufferPointer($0), outPtr, outLen) } +} + +/// Returns bytes from a callback method through its out slots. +func wvReturnBytes(_ value: Data, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + value.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +/// Returns a value buffer from a callback method through its out slots. +func wvReturnBuffer(_ value: T, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var w = WvWriter() + w.write(value) + w.bytes.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } } diff --git a/crates/weaveffi-gen/src/targets/swift/runtime/module.modulemap b/crates/weaveffi-cli/src/targets/swift/runtime/module.modulemap similarity index 100% rename from crates/weaveffi-gen/src/targets/swift/runtime/module.modulemap rename to crates/weaveffi-cli/src/targets/swift/runtime/module.modulemap diff --git a/crates/weaveffi-cli/src/targets/swift/tests.rs b/crates/weaveffi-cli/src/targets/swift/tests.rs new file mode 100644 index 00000000..8c202195 --- /dev/null +++ b/crates/weaveffi-cli/src/targets/swift/tests.rs @@ -0,0 +1,429 @@ +//! Unit tests: render a small API that exercises every C ABI revision 4 +//! shape the Swift wrapper distinguishes and assert the key pieces of each +//! contract. + +use camino::Utf8Path; +use weaveffi_model::contract::entries; +use weaveffi_model::ir::Api; +use weaveffi_model::model::Model; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; + +use super::{render_swift_wrapper, Layout, SwiftConfig}; + +/// One module with an error domain carrying a field; an interface with a +/// `new` constructor, a method, and a throwing static; nullable objects; an +/// iterator of objects; a cancellable async function; a record carrying +/// objects; and a callback interface whose methods take every argument +/// family and return a direct value, an enum, a string, a record, an object, +/// and an optional object, one of them `throws`, passed both required and +/// optional. +const FIXTURE: &str = r#" +version: "0.11.0" +modules: + - name: shop + errors: + name: ShopError + codes: + - { name: OutOfStock, code: 1, message: "out of stock", fields: [{ name: sku, type: string }] } + - { name: Closed, code: 2, message: "closed" } + enums: + - name: Mood + variants: [{ name: Happy, value: 0 }, { name: Grumpy, value: 1 }] + structs: + - name: Order + fields: + - { name: cart, type: Cart } + - { name: history, type: "[Cart]" } + - { name: note, type: "string?" } + interfaces: + - name: Cart + constructors: + - { name: new, params: [{ name: owner, type: string }] } + methods: + - { name: add, params: [{ name: sku, type: string }], return: bool } + statics: + - { name: restore, params: [{ name: id, type: i64 }], return: Cart, throws: true } + callback_interfaces: + - name: Watcher + methods: + - name: on_event + params: + - { name: name, type: string } + - { name: count, type: i32 } + - { name: order, type: Order } + - { name: cart, type: Cart } + return: bool + - { name: mood, return: Mood } + - { name: label, return: string, throws: true } + - { name: latest, return: "Order?" } + - { name: favorite, return: Cart } + - { name: maybe, return: "Cart?" } + - { name: on_done } + functions: + - { name: find_cart, params: [{ name: current, type: "Cart?" }], return: "Cart?" } + - { name: all_carts, return: "iter" } + - { name: watch, params: [{ name: watcher, type: Watcher }] } + - { name: maybe_watch, params: [{ name: watcher, type: "Watcher?" }] } + - { name: echo, params: [{ name: text, type: string }], return: string } + - { name: tally, params: [{ name: counts, type: "{string:[i32?]}" }], return: "[Order]" } + - { name: wait, params: [{ name: ms, type: i64 }], return: i64, async: true, cancellable: true } +"#; + +fn model() -> Model { + let api: Api = serde_yaml::from_str(FIXTURE).unwrap(); + validate(&api, &Identity::named("shop_kit"), None).unwrap() +} + +fn render() -> String { + let model = model(); + let config = SwiftConfig::default(); + render_swift_wrapper(&Layout::new(&model, &config), &model, "ShopKit.swift") +} + +#[track_caller] +fn assert_has(out: &str, needle: &str) { + assert!(out.contains(needle), "missing {needle:?} in:\n{out}"); +} + +#[test] +fn names_come_from_the_identity() { + let model = model(); + let config = SwiftConfig::default(); + let files = Layout::new(&model, &config).sources(&model, Utf8Path::new("out"), &config); + let paths: Vec = files + .iter() + .map(|(p, _)| p.as_str().replace('\\', "/")) + .collect(); + assert_eq!( + paths, + [ + "out/Package.swift", + "out/Sources/CShopKit/module.modulemap", + "out/Sources/CShopKit/shop_kit.h", + "out/Sources/ShopKit/ShopKit.swift", + ] + ); + assert_has(&files[0].1, "name: \"ShopKit\""); + assert_has(&files[0].1, ".systemLibrary(name: \"CShopKit\")"); + assert_has( + &files[0].1, + ".binaryTarget(name: \"CShopKit\", path: xcframework)", + ); + assert_has(&files[1].1, "header \"shop_kit.h\""); + assert_has(&files[1].1, "link \"shop_kit\""); + let all = files.iter().map(|(_, c)| c.as_str()).collect::(); + assert!(!all.contains("weaveffi_"), "{all}"); + assert!(!all.contains("WeaveFFIError"), "{all}"); +} + +#[test] +fn module_name_override_wins() { + let model = model(); + let config = SwiftConfig { + name: Some("Shop".into()), + ..SwiftConfig::default() + }; + let layout = Layout::new(&model, &config); + assert_eq!(layout.module, "Shop"); + assert_eq!(layout.c_module, "CShop"); + assert_eq!(layout.library, "shop_kit"); +} + +#[test] +fn load_checks_embed_every_contract_entry() { + let model = model(); + let out = render(); + assert_has(&out, "let wvAbiVersion: UInt32 = 4"); + assert_has(&out, "let abi = shop_kit_abi_version()"); + assert_has( + &out, + " wvCheckAbiVersion()\n wvCheckContract(shop_kit_shop_contract, [\n", + ); + let root = model.roots().next().unwrap(); + let table = entries(&model, root); + assert!(table.len() > 10); + for e in &table { + assert_has( + &out, + &format!("(0x{:016x}, 0x{:016x}, \"{}\"),", e.id, e.hash, e.path), + ); + } + assert_has(&out, "\\(path) is missing from the library 'shop_kit'"); + assert_has(&out, "\\(path) changed since these bindings were generated"); + assert!(!out.contains("checksum"), "{out}"); + // Static entry points run the checks before the first native call. + assert_has( + &out, + "public static func echo(text: String) -> String {\n wvLoad()", + ); +} + +#[test] +fn strings_cross_as_pointer_and_length() { + let out = render(); + assert_has(&out, "wvWithUTF8(text) { text_ptr, text_len in"); + assert_has( + &out, + "shop_kit_shop_echo(text_ptr, text_len, &outLen, &err)", + ); + assert_has(&out, "return wvTakeString(rv, outLen)"); +} + +#[test] +fn composites_share_one_generic_codec() { + let out = render(); + // Records conform once; every composite reuses the runtime's generic + // Optional, Array, and Dictionary conformances instead of inlined loops. + assert_has(&out, "extension Order: WvCodable {"); + assert_has( + &out, + "Order(cart: r.read(), history: r.read(), note: r.read())", + ); + assert_has(&out, " w.write(self.history)\n"); + assert_has( + &out, + "extension Array: WvCodable where Element: WvCodable {", + ); + assert_has( + &out, + "extension Dictionary: WvCodable where Key: WvCodable, Value: WvCodable {", + ); + assert_has(&out, "extension Mood: WvCEnum {}"); + assert_has(&out, "wvWithEncoded(counts) { counts_ptr, counts_len in"); + assert_has(&out, "return wvTakeBuffer(rv, outLen, as: [Order].self)"); + assert!(!out.contains("for v"), "{out}"); +} + +#[test] +fn interface_class_owns_one_reference() { + let out = render(); + assert_has(&out, "public final class Cart: @unchecked Sendable {"); + assert_has(&out, "let ptr: OpaquePointer"); + assert_has(&out, "shop_kit_shop_Cart_destroy(ptr)"); + assert_has(&out, "wvNonNull(shop_kit_shop_Cart_clone(ptr))"); + assert_eq!(out.matches("shop_kit_shop_Cart_destroy(").count(), 1); + assert_has(&out, "public init(owner: String) {"); + assert_has(&out, "self.ptr = wvNonNull(rv)"); + assert_has(&out, "public func add(sku: String) -> Bool {"); + assert_has(&out, "shop_kit_shop_Cart_add(ptr, sku_ptr, sku_len, &err)"); + assert_has( + &out, + "static func wvRead(_ r: inout WvReader) -> Cart { Cart(ptr: r.readObject()) }", + ); + assert_has( + &out, + "func wvWrite(_ w: inout WvWriter) { w.writeObject(clonePtr()) }", + ); +} + +#[test] +fn throwing_calls_map_the_domain_and_others_trap() { + let out = render(); + assert_has( + &out, + "public enum ShopError: Error, LocalizedError, Sendable {", + ); + assert_has(&out, "case outOfStock(message: String, sku: String)"); + assert_has( + &out, + "/// - Throws: ``ShopError`` for a declared failure, ``ShopKitRuntimeError`` otherwise.", + ); + assert_has( + &out, + "public static func restore(id: Int64) throws -> Cart {", + ); + assert_has(&out, "try wvCheckShop(&err)"); + assert_has( + &out, + "let error = ShopError.outOfStock(message: message.isEmpty ? \"out of stock\" : message, sku: r.read())", + ); + // A call that can't throw stops the process, naming code and message. + assert_has(&out, "wvTrap(&err)"); + assert_has( + &out, + "fatalError(\"ShopKit.\\(function) failed with code \\(code): \\(message)\")", + ); +} + +#[test] +fn nullable_objects_map_to_optional_wrappers() { + let out = render(); + assert_has( + &out, + "public static func findCart(current: Cart?) -> Cart? {", + ); + assert_has(&out, "shop_kit_shop_find_cart(current?.ptr, &err)"); + assert_has(&out, "return rv.map { Cart(ptr: $0) }"); +} + +#[test] +fn object_iterator_adopts_each_element() { + let out = render(); + assert_has( + &out, + "public final class ShopAllCartsIterator: Sequence, IteratorProtocol {", + ); + assert_has(&out, "var item: OpaquePointer? = nil"); + assert_has( + &out, + "shop_kit_shop_AllCartsIterator_next(handle, &item, &err)", + ); + assert_has(&out, "shop_kit_shop_AllCartsIterator_destroy(handle)"); + assert_has(&out, "return Cart(ptr: wvNonNull(item))"); +} + +#[test] +fn callback_vtable_starts_with_the_header() { + let out = render(); + assert_has(&out, "public protocol Watcher: AnyObject, Sendable {"); + assert_has( + &out, + "static let shared = WvVtable(shop_kit_shop_Watcher_vtable(\n size: UInt32(MemoryLayout.stride),\n flags: 0,\n free: { ctx in\n wvRelease(ctx, as: (any Watcher).self)\n },\n on_event: {", + ); + // No trailing comma after the last entry (Swift before 6.1 rejects it). + assert_has(&out, " }\n ))\n}"); +} + +#[test] +fn callback_arguments_are_copied_or_adopted() { + let out = render(); + assert_has( + &out, + "func onEvent(name: String, count: Int32, order: Order, cart: Cart) throws -> Bool", + ); + assert_has( + &out, + "on_event: { ctx, name_ptr, name_len, count, order_ptr, order_len, cart, out_err in", + ); + assert_has( + &out, + "wvInvoke(ctx, out_err, as: (any Watcher).self, fallback: false) { wvImpl in", + ); + assert_has( + &out, + "try wvImpl.onEvent(name: wvBorrowString(name_ptr, name_len), count: count, order: wvBorrowBuffer(order_ptr, order_len, as: Order.self), cart: Cart(ptr: wvNonNull(cart)))", + ); +} + +#[test] +fn callback_returns_cover_every_family() { + let out = render(); + // An enum by value. + assert_has(&out, "func mood() throws -> Mood"); + assert_has( + &out, + "fallback: shop_kit_shop_Mood(rawValue: 0)) { wvImpl in\n try shop_kit_shop_Mood(rawValue: numericCast(wvImpl.mood().rawValue))", + ); + // A string and a buffer through the out slots, as `{p}_alloc` runs. + assert_has(&out, "label: { ctx, out_ptr, out_len, out_err in"); + assert_has(&out, "try wvReturnString(wvImpl.label(), out_ptr, out_len)"); + assert_has( + &out, + "try wvReturnBuffer(wvImpl.latest(), out_ptr, out_len)", + ); + assert_has(&out, "let run = shop_kit_alloc(bytes.count)"); + assert!(!out.contains("_dealloc"), "{out}"); + // Objects as a fresh reference the library adopts; `I?` may be nil. + assert_has(&out, "func favorite() throws -> Cart"); + assert_has( + &out, + "fallback: nil) { wvImpl in\n try wvImpl.favorite().clonePtr()", + ); + assert_has(&out, "func maybe() throws -> Cart?"); + assert_has(&out, "try wvImpl.maybe()?.clonePtr()"); + assert_has( + &out, + "fallback: ()) { wvImpl in\n try wvImpl.onDone()", + ); +} + +#[test] +fn throwing_callback_methods_report_the_domain_error() { + let out = render(); + assert_has(&out, "func label() throws -> String"); + assert_has( + &out, + "/// - Throws: ``ShopError`` to report a declared failure with its fields", + ); + assert_has( + &out, + "wvInvoke(ctx, out_err, as: (any Watcher).self, fallback: (), domain: wvReportShop) { wvImpl in", + ); + assert_eq!(out.matches("domain: wvReportShop").count(), 1, "{out}"); + assert_has( + &out, + "func wvReportShop(_ error: Error, _ outErr: UnsafeMutablePointer?) -> Bool {", + ); + assert_has( + &out, + " case let .outOfStock(message, v0):\n var payload = WvWriter()\n payload.write(v0)\n wvSetError(outErr, 1, message, payload)", + ); + assert_has( + &out, + " case let .closed(message):\n wvSetError(outErr, 2, message)", + ); + assert_has( + &out, + "shop_kit_error_set_payload(outErr, $0.baseAddress, $0.count)", + ); + // Any other error is a callback failure. + assert_has( + &out, + "shop_kit_error_set(outErr, ShopKitRuntimeError.foreignCode, $0)", + ); +} + +#[test] +fn callback_parameters_retain_the_implementation() { + let out = render(); + assert_has(&out, "public static func watch(watcher: any Watcher) {"); + assert_has(&out, "let watcher_ctx = wvRetain(watcher)"); + assert_has( + &out, + "shop_kit_shop_watch(watcher_ctx, WvWatcherVtable.shared.pointer, &err)", + ); + // An optional callback passes a null vtable for nil. + assert_has( + &out, + "public static func maybeWatch(watcher: (any Watcher)?) {", + ); + assert_has(&out, "let watcher_ctx = watcher.map { wvRetain($0) }"); + assert_has( + &out, + "shop_kit_shop_maybe_watch(watcher_ctx, watcher == nil ? nil : WvWatcherVtable.shared.pointer, &err)", + ); +} + +#[test] +fn cancellable_async_cancels_the_native_token() { + let out = render(); + assert_has( + &out, + "public static func wait(ms: Int64) async throws -> Int64 {", + ); + assert_has(&out, "let token = WvCancelToken()"); + assert_has(&out, "return try await withTaskCancellationHandler {"); + assert_has( + &out, + "shop_kit_shop_wait(ms, token.raw, { context, err, result in", + ); + assert_has( + &out, + "cont.resume(throwing: wvTakeError(err, wvCancelledOrTrap))", + ); + assert_has(&out, "} onCancel: {\n token.cancel()"); +} + +#[test] +fn every_line_is_free_of_trailing_whitespace() { + let out = render(); + for (i, line) in out.lines().enumerate() { + assert_eq!( + line, + line.trim_end(), + "line {} has trailing whitespace", + i + 1 + ); + } +} diff --git a/crates/weaveffi-gen/src/targets/swift/types.rs b/crates/weaveffi-cli/src/targets/swift/types.rs similarity index 60% rename from crates/weaveffi-gen/src/targets/swift/types.rs rename to crates/weaveffi-cli/src/targets/swift/types.rs index a2c74181..22812933 100644 --- a/crates/weaveffi-gen/src/targets/swift/types.rs +++ b/crates/weaveffi-cli/src/targets/swift/types.rs @@ -1,12 +1,12 @@ //! Swift type spellings and identifier policy: how model types and //! user-chosen names render in Swift source. -use std::collections::{HashMap, HashSet}; +use std::collections::HashSet; use crate::lang; -use crate::utils::local_type_name; use heck::{ToLowerCamelCase, ToUpperCamelCase}; -use weaveffi_model::model::{BindingModel, IteratorBinding, Ty}; +use weaveffi_model::model::{ErrorBinding, IteratorBinding, Model}; +use weaveffi_model::ty::{Prim, Ty}; /// The Swift spelling of a user-chosen identifier in a lowerCamel position /// (parameters, fields, enum cases, wrapper names): camel-cased, then @@ -16,6 +16,20 @@ pub(crate) fn swift_ident(name: &str) -> String { lang::escape_ident(&name.to_lower_camel_case(), lang::SWIFT_KEYWORDS) } +/// Members every interface wrapper class declares: the stored `ptr`, the +/// internal `clonePtr()`, and the `WvCodable` conformance's `wvRead` and +/// `wvWrite`. (`init` and `deinit` are keywords, so [`swift_ident`] already +/// escapes them.) +const OBJECT_MEMBERS: &[&str] = &["clonePtr", "ptr", "wvRead", "wvWrite"]; + +/// The Swift spelling of an interface member (a factory constructor, +/// method, or static): [`swift_ident`], with a trailing `_` when it would +/// redeclare a member the wrapper class declares (`clone_ptr` is +/// `clonePtr_`). +pub(crate) fn swift_member(name: &str) -> String { + lang::escape_member(&swift_ident(name), OBJECT_MEMBERS) +} + /// Escape a string for embedding inside a Swift double-quoted literal. pub(crate) fn swift_str(s: &str) -> String { s.replace('\\', "\\\\").replace('"', "\\\"") @@ -31,47 +45,60 @@ pub(crate) fn deprecated_attr(msg: &str) -> String { /// namespace collisions, C enum type names, and which value types can be /// `Hashable`. pub(crate) struct SwiftCtx<'a> { + /// The model being rendered. + pub(crate) model: &'a Model, /// C ABI symbol prefix (e.g. `kvstore`). pub(crate) c_prefix: &'a str, /// SwiftPM module name (e.g. `Kvstore`). pub(crate) swift_module: &'a str, + /// The error type for codes outside any declared domain + /// (`KvstoreRuntimeError`). + pub(crate) runtime_error: String, /// Every module name in the API, PascalCased: the namespace `enum` /// names a wrapper-type reference can be shadowed by. module_names: HashSet, - /// The C type of every C-style enum, keyed by its absolute name. - enum_c_types: HashMap, - /// Absolute names of the records and rich enums that are `Hashable` - /// (every field is). + /// The records and rich enums that are `Hashable` (every field is). hashable: HashSet, + /// The declaring module paths of the error domains a callback method + /// may throw, which need a `wvReport{Stem}` helper. + reported: HashSet, } impl<'a> SwiftCtx<'a> { - /// Collect the API-wide facts from the binding model. - pub(crate) fn new(model: &'a BindingModel, swift_module: &'a str) -> SwiftCtx<'a> { + /// Collect the API-wide facts from the model. + pub(crate) fn new(model: &'a Model, swift_module: &'a str) -> SwiftCtx<'a> { let module_names = model .modules .iter() .map(|m| m.name.to_upper_camel_case()) .collect(); - let enum_c_types = model + let reported = model .modules .iter() - .flat_map(|m| { - m.enums + .filter(|m| { + m.callback_interfaces .iter() - .filter(|e| !e.is_rich()) - .map(move |e| (format!("{}.{}", m.dot_path, e.name), e.c_tag.clone())) + .any(|cb| cb.methods.iter().any(|f| f.throws)) }) + .filter_map(|m| model.error_domain(m)) + .map(|e| e.owner_path.clone()) .collect(); SwiftCtx { - c_prefix: model.prefix.as_str(), + model, + c_prefix: model.prefix(), swift_module, + runtime_error: format!("{swift_module}RuntimeError"), module_names, - enum_c_types, hashable: hashable_types(model), + reported, } } + /// `true` when a callback method may throw the domain `eb`. + pub(crate) fn reports(&self, eb: &ErrorBinding) -> bool { + self.reported.contains(&eb.owner_path) + } + /// Qualify a top-level wrapper type name with the Swift module when its /// name collides with a namespace `enum`. Inside `enum Kv { enum Stats { /// ... } }` the bare name `Stats` resolves to the namespace, not the @@ -84,7 +111,8 @@ impl<'a> SwiftCtx<'a> { } } - /// The Swift surface type of a model type. + /// The Swift surface type of a model type. A callback interface is an + /// existential (`any Listener`). /// /// # Panics /// @@ -92,11 +120,13 @@ impl<'a> SwiftCtx<'a> { /// renders as its per-function sequence class instead. pub(crate) fn swift_type(&self, t: &Ty) -> String { match t { - Ty::Enum(name) - | Ty::Record(name) - | Ty::RichEnum(name) - | Ty::Interface(name) - | Ty::CallbackInterface(name) => self.ty_name(local_type_name(name)), + Ty::Enum(name) | Ty::Record(name) | Ty::RichEnum(name) | Ty::Interface(name) => { + self.ty_name(name) + } + Ty::CallbackInterface(name) => format!("any {}", self.ty_name(name)), + Ty::Optional(inner) if matches!(**inner, Ty::CallbackInterface(_)) => { + format!("({})?", self.swift_type(inner)) + } Ty::Optional(inner) => format!("{}?", self.swift_type(inner)), Ty::List(inner) => format!("[{}]", self.swift_type(inner)), Ty::Map(k, v) => format!("[{}: {}]", self.swift_type(k), self.swift_type(v)), @@ -108,11 +138,9 @@ impl<'a> SwiftCtx<'a> { /// /// # Panics /// - /// Panics when no C-style enum has that name, which validation rules out. + /// Panics when no enum has that name, which validation rules out. pub(crate) fn c_enum_type(&self, name: &str) -> &str { - self.enum_c_types - .get(name) - .unwrap_or_else(|| panic!("unknown C-style enum `{name}`")) + &self.model.enumeration(name).c_tag } /// `true` when values of `t` can be `Hashable` (and so `Equatable`). @@ -128,19 +156,19 @@ impl<'a> SwiftCtx<'a> { /// Panics on a user, container, or iterator type. pub(crate) fn scalar_swift_type(t: &Ty) -> &'static str { match t { - Ty::I8 => "Int8", - Ty::I16 => "Int16", - Ty::I32 => "Int32", - Ty::I64 => "Int64", - Ty::U8 => "UInt8", - Ty::U16 => "UInt16", - Ty::U32 => "UInt32", - Ty::U64 => "UInt64", - Ty::F32 => "Float", - Ty::F64 => "Double", - Ty::Bool => "Bool", - Ty::StringUtf8 => "String", - Ty::Bytes => "Data", + Ty::Prim(Prim::I8) => "Int8", + Ty::Prim(Prim::I16) => "Int16", + Ty::Prim(Prim::I32) => "Int32", + Ty::Prim(Prim::I64) => "Int64", + Ty::Prim(Prim::U8) => "UInt8", + Ty::Prim(Prim::U16) => "UInt16", + Ty::Prim(Prim::U32) => "UInt32", + Ty::Prim(Prim::U64) => "UInt64", + Ty::Prim(Prim::F32) => "Float", + Ty::Prim(Prim::F64) => "Double", + Ty::Prim(Prim::Bool) => "Bool", + Ty::Prim(Prim::String) => "String", + Ty::Prim(Prim::Bytes) => "Data", other => unreachable!("`{other}` isn't a scalar type"), } } @@ -157,12 +185,12 @@ fn type_is_hashable(t: &Ty, hashable: &HashSet) -> bool { /// The records and rich enums whose every field is `Hashable`, as a greatest /// fixed point so mutually referencing types resolve. -fn hashable_types(model: &BindingModel) -> HashSet { +fn hashable_types(model: &Model) -> HashSet { let mut types: Vec<(String, Vec<&Ty>)> = Vec::new(); for m in &model.modules { for s in &m.structs { let fields = s.fields.iter().map(|f| &f.ty).collect(); - types.push((format!("{}.{}", m.dot_path, s.name), fields)); + types.push((s.name.clone(), fields)); } for e in m.enums.iter().filter(|e| e.is_rich()) { let fields = e @@ -170,7 +198,7 @@ fn hashable_types(model: &BindingModel) -> HashSet { .iter() .flat_map(|v| v.fields.iter().map(|f| &f.ty)) .collect(); - types.push((format!("{}.{}", m.dot_path, e.name), fields)); + types.push((e.name.clone(), fields)); } } let mut hashable: HashSet = types.iter().map(|(n, _)| n.clone()).collect(); @@ -197,14 +225,22 @@ pub(crate) fn iterator_class_name(it: &IteratorBinding, c_prefix: &str) -> Strin .to_upper_camel_case() } -/// The internal box class retaining one implementation of the callback -/// interface named `name` across the C boundary: `Wv{Name}Box`. -pub(crate) fn callback_box_name(name: &str) -> String { - format!("Wv{}Box", local_type_name(name)) -} - /// The internal namespace holding the process-wide vtable for the callback /// interface named `name`: `Wv{Name}Vtable`. pub(crate) fn callback_vtable_name(name: &str) -> String { - format!("Wv{}Vtable", local_type_name(name)) + format!("Wv{name}Vtable") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn interface_members_avoid_the_wrapper_members() { + assert_eq!(swift_member("clone_ptr"), "clonePtr_"); + assert_eq!(swift_member("ptr"), "ptr_"); + assert_eq!(swift_member("wv_write"), "wvWrite_"); + assert_eq!(swift_member("init"), "init_"); + assert_eq!(swift_member("close"), "close"); + } } diff --git a/crates/weaveffi-gen/src/targets/wasm/glue.rs b/crates/weaveffi-cli/src/targets/wasm/glue.rs similarity index 75% rename from crates/weaveffi-gen/src/targets/wasm/glue.rs rename to crates/weaveffi-cli/src/targets/wasm/glue.rs index 19b4d963..c44f2174 100644 --- a/crates/weaveffi-gen/src/targets/wasm/glue.rs +++ b/crates/weaveffi-cli/src/targets/wasm/glue.rs @@ -8,13 +8,15 @@ //! and turns results back into JavaScript values (copying and freeing //! returned strings and buffers). Callback-interface vtables and async //! completions are JavaScript functions installed in the module's function -//! table. +//! table; a callback's string, bytes, or buffer return is copied into a run +//! from `{prefix}_alloc` that the producer adopts. use weaveffi_model::abi::{AbiParam, CType}; use weaveffi_model::model::{ - BindingModel, CallShape, CallbackInterfaceBinding, FnBinding, Prim, Ty, + contract_symbol, CallShape, CallbackInterfaceBinding, FnBinding, Model, }; -use weaveffi_model::plan::{self, ArgPass, RetPass}; +use weaveffi_model::plan::{ArgPass, RetPass}; +use weaveffi_model::ty::{Prim, Ty}; use crate::codegen::CodeWriter; use crate::targets::js::names::direct_prim; @@ -135,7 +137,7 @@ struct Args { call: Vec, } -fn marshal(model: &BindingModel, f: &FnBinding) -> Args { +fn marshal(model: &Model, f: &FnBinding) -> Args { let mut a = Args::default(); if f.has_self { a.params.push("self".into()); @@ -148,7 +150,7 @@ fn marshal(model: &BindingModel, f: &FnBinding) -> Args { ArgPass::Direct { .. } => a.call.push(direct_arg(direct_prim(&p.ty), &v)), ArgPass::String { .. } | ArgPass::Bytes { .. } | ArgPass::Buffer { .. } => { let s = format!("s{n}"); - let stage = if matches!(p.ty, Ty::StringUtf8) { + let stage = if matches!(p.ty, Ty::Prim(Prim::String)) { "str" } else { "data" @@ -162,16 +164,19 @@ fn marshal(model: &BindingModel, f: &FnBinding) -> Args { "m.{}({v})", if nullable { "handleOpt" } else { "handle" } )), - ArgPass::Callback { .. } => { - let cb = find_callback( - model, + ArgPass::Callback { nullable, .. } => { + let cb = model.callback_interface( p.ty.callback_interface_name().expect("callback parameter"), ); a.call.push(format!("m.register({v})")); - a.call.push(format!( - "m.vtable('{}', () => $vt_{}(m))", - cb.c_tag, cb.c_tag - )); + a.call.push(if nullable { + format!( + "m.vtableOpt({v}, '{}', () => $vt_{}(m))", + cb.c_tag, cb.c_tag + ) + } else { + format!("m.vtable('{}', () => $vt_{}(m))", cb.c_tag, cb.c_tag) + }); } } } @@ -182,17 +187,6 @@ fn marshal(model: &BindingModel, f: &FnBinding) -> Args { a } -/// The callback interface named by the absolute dotted path `dotted`. -fn find_callback<'a>(model: &'a BindingModel, dotted: &str) -> &'a CallbackInterfaceBinding { - let (module, name) = dotted.rsplit_once('.').expect("absolute type path"); - model - .modules - .iter() - .find(|m| m.dot_path == module) - .and_then(|m| m.callback_interface(name)) - .expect("callback interface exists") -} - /// Emit `body` (the lines of an entry point) wrapped in the staging /// `try`/`finally` when anything is staged. fn emit_staged(w: &mut CodeWriter, a: &Args, body: &[String]) { @@ -222,31 +216,59 @@ fn emit_staged(w: &mut CodeWriter, a: &Args, body: &[String]) { w.line("}"); } -/// Render `$bind(m)`, which builds the raw entry points over a loaded -/// `$Linear`, plus the callback-interface vtable builders, and return it -/// with every C symbol the glue calls (Emscripten mode binds them by name). -pub(crate) fn render_bind(model: &BindingModel) -> (String, Vec) { - let prefix = &model.prefix; - let mut symbols: Vec = [ - "abi_version", - "alloc", - "dealloc", - "free_bytes", - "error_set", - "error_clear", - "error_free", - "debug_live", - "cancel_token_create", - "cancel_token_cancel", - "cancel_token_destroy", - ] - .iter() - .map(|s| format!("{prefix}_{s}")) - .collect(); - let mut w = CodeWriter::two_space(); +/// The runtime symbols the glue calls. +const RUNTIME_SYMBOLS: &[&str] = &[ + "abi_version", + "alloc", + "free_bytes", + "error_set", + "error_set_payload", + "error_clear", + "error_free", + "debug_live", + "cancel_token_create", + "cancel_token_cancel", + "cancel_token_destroy", +]; + +/// Every C symbol the glue calls, which Emscripten mode binds by name. +pub(crate) fn bound_symbols(model: &Model) -> Vec { + let prefix = model.prefix(); + let mut out: Vec = RUNTIME_SYMBOLS + .iter() + .map(|s| format!("{prefix}_{s}")) + .collect(); + out.extend( + model + .roots() + .map(|root| contract_symbol(prefix, &root.name)), + ); + for m in &model.modules { + for i in &m.interfaces { + out.push(i.clone_symbol.clone()); + out.push(i.destroy_symbol.clone()); + } + for f in m.callables() { + match &f.shape { + CallShape::Sync(abi) => out.push(abi.symbol.clone()), + CallShape::Async(ab) => out.push(ab.launch.symbol.clone()), + CallShape::Iterator(it) => out.extend([ + it.launch.symbol.clone(), + it.next.symbol.clone(), + it.destroy_symbol.clone(), + ]), + } + } + } + out +} +/// Render `$bind(m)`, which builds the raw entry points over a loaded +/// `$Linear`, preceded by the callback-interface vtable builders. +pub(crate) fn render_bind(w: &mut CodeWriter, model: &Model) { + let prefix = model.prefix(); for (_, cb) in model.callback_interfaces() { - emit_vtable(&mut w, cb, prefix); + emit_vtable(w, cb); } w.line("// The raw entry points, one per C symbol."); @@ -257,7 +279,7 @@ pub(crate) fn render_bind(model: &BindingModel) -> (String, Vec) { "{prefix}_abi_version: () => x.{prefix}_abi_version() >>> 0," )); w.line(format!( - "{prefix}_debug_live: (kind) => BigInt.asUintN(64, x.{prefix}_debug_live(m.num(kind)))," + "{prefix}_debug_live: (kind) => m.live(x.{prefix}_debug_live(m.num(kind)), kind)," )); w.line(format!( "{prefix}_cancel_token_create: () => x.{prefix}_cancel_token_create()," @@ -268,38 +290,29 @@ pub(crate) fn render_bind(model: &BindingModel) -> (String, Vec) { )); } for root in model.roots() { - let sym = weaveffi_model::model::checksum_symbol(prefix, &root.name); - w.line(format!("{sym}: () => BigInt.asUintN(64, x.{sym}()),")); - symbols.push(sym); + let sym = contract_symbol(prefix, &root.name); + w.line(format!("{sym}: () => m.contract(x.{sym}),")); } for m in &model.modules { for i in &m.interfaces { for sym in [&i.clone_symbol, &i.destroy_symbol] { w.line(format!("{sym}: (h) => x.{sym}(m.handle(h)),")); - symbols.push(sym.clone()); } } for f in m.callables() { - emit_callable(w, model, f, &mut symbols); + emit_callable(w, model, f); } } }); }); - (w.finish(), symbols) } -fn emit_callable( - w: &mut CodeWriter, - model: &BindingModel, - f: &FnBinding, - symbols: &mut Vec, -) { - let prefix = &model.prefix; +fn emit_callable(w: &mut CodeWriter, model: &Model, f: &FnBinding) { let a = marshal(model, f); let head = |sym: &str| format!("{sym}: ({}) => {{", a.params.join(", ")); match &f.shape { CallShape::Sync(abi) => { - let pass = plan::ret_pass(f.ret.as_ref(), prefix); + let pass = RetPass::of(f.ret.as_ref()); let pair = matches!(pass, RetPass::String | RetPass::Bytes | RetPass::Buffer); let mut call = a.call.clone(); if pair { @@ -324,7 +337,6 @@ fn emit_callable( } } w.block(head(&abi.symbol), "},", |w| emit_staged(w, &a, &body)); - symbols.push(abi.symbol.clone()); } CallShape::Iterator(it) => { let mut call = a.call.clone(); @@ -335,9 +347,8 @@ fn emit_callable( "return r;".into(), ]; w.block(head(&it.launch.symbol), "},", |w| emit_staged(w, &a, &body)); - symbols.push(it.launch.symbol.clone()); - let protocol = it.protocol(f, prefix); + let protocol = it.protocol(f); let pair = matches!( protocol.elem, RetPass::String | RetPass::Bytes | RetPass::Buffer @@ -367,11 +378,9 @@ fn emit_callable( "{0}: (h) => x.{0}(m.handle(h)),", it.destroy_symbol )); - symbols.push(it.next.symbol.clone()); - symbols.push(it.destroy_symbol.clone()); } CallShape::Async(ab) => { - let pass = plan::ret_pass(f.ret.as_ref(), prefix); + let pass = RetPass::of(f.ret.as_ref()); let names: Vec = ab.callback_params[2..] .iter() .map(|p| p.name.clone()) @@ -395,22 +404,28 @@ fn emit_callable( call.join(", ") )]; w.block(head(&ab.launch.symbol), "},", |w| emit_staged(w, &a, &body)); - symbols.push(ab.launch.symbol.clone()); } } } /// Emit `$vt_{tag}(m)`, the table functions of one callback interface's -/// vtable: each converts the raw slots into the values the adapter takes, -/// calls it, and converts the result back, reporting an exception through -/// `out_err` (code -4) instead of letting it unwind into the module. -fn emit_vtable(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, prefix: &str) { - let protocol = cb.protocol(prefix); +/// vtable methods: each converts the raw slots into the values the adapter +/// takes, calls it, and hands the result back (a direct value or object +/// handle as the return value, a string, bytes, or buffer as a run in the +/// `out_ptr`/`out_len` slots), reporting an exception through `out_err` +/// instead of letting it unwind into the module. +fn emit_vtable(w: &mut CodeWriter, cb: &CallbackInterfaceBinding) { + let protocol = cb.protocol(); w.block(format!("function $vt_{}(m) {{", cb.c_tag), "}", |w| { w.block("return [", "];", |w| { - for (method, passes) in cb.methods.iter().zip(&protocol.method_args) { - let slots = &method.abi_params[1..method.abi_params.len() - 1]; - let names: Vec = (0..slots.len()).map(|i| format!("p{i}")).collect(); + for ((method, passes), ret) in cb + .methods + .iter() + .zip(&protocol.method_args) + .zip(&protocol.method_returns) + { + let inputs: usize = method.params.iter().map(|p| p.abi.len()).sum(); + let names: Vec = (0..inputs).map(|i| format!("p{i}")).collect(); let mut k = 0usize; let args: Vec = method .params @@ -426,16 +441,27 @@ fn emit_vtable(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, prefix: &str) }) .collect(); let call = format!("m.adapter(ctx).{}({})", method.name, args.join(", ")); - let (ret, fallback) = match &method.ret { - None => (format!("{call};"), None), - Some(ty) => match direct_prim(ty) { + let mut params = vec!["ctx".to_string()]; + params.extend(names.iter().cloned()); + let (body, fallback) = match (ret, &method.ret) { + (RetPass::Void, _) | (_, None) => (format!("{call};"), None), + (RetPass::Direct, Some(ty)) => match direct_prim(ty) { Prim::Bool => (format!("return {call} ? 1 : 0;"), Some("return 0;")), Prim::I64 | Prim::U64 => (format!("return {call};"), Some("return 0n;")), _ => (format!("return {call};"), Some("return 0;")), }, + (RetPass::Object { .. }, _) => { + (format!("return m.handleOpt({call});"), Some("return 0;")) + } + (RetPass::String, _) => { + params.extend(["out".to_string(), "outLen".to_string()]); + (format!("m.giveStr(out, outLen, {call});"), None) + } + (RetPass::Bytes | RetPass::Buffer, _) => { + params.extend(["out".to_string(), "outLen".to_string()]); + (format!("m.give(out, outLen, {call});"), None) + } }; - let mut params = vec!["ctx".to_string()]; - params.extend(names.iter().cloned()); params.push("err".into()); w.line(format!( "[{}, ({}) => {{", @@ -445,7 +471,7 @@ fn emit_vtable(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, prefix: &str) w.scope(|w| { w.line("try {"); w.scope(|w| { - w.line(&ret); + w.line(&body); }); w.line("} catch (e) {"); w.scope(|w| { diff --git a/crates/weaveffi-gen/src/targets/wasm/mod.rs b/crates/weaveffi-cli/src/targets/wasm/mod.rs similarity index 54% rename from crates/weaveffi-gen/src/targets/wasm/mod.rs rename to crates/weaveffi-cli/src/targets/wasm/mod.rs index 10af5ada..ad9967b9 100644 --- a/crates/weaveffi-gen/src/targets/wasm/mod.rs +++ b/crates/weaveffi-cli/src/targets/wasm/mod.rs @@ -7,7 +7,7 @@ //! (`linear.js` plus one generated entry point per C symbol) that stages //! arguments in the module's memory and installs async completions and //! callback-interface vtables in its function table. `init()` loads the -//! module and checks its ABI revision and contract checksums before any +//! module and checks its ABI revision and contract tables before any //! other export works. Implements [`LanguageBackend`]; the shared driver //! bridges it into the generator pipeline. @@ -16,18 +16,18 @@ mod package; use camino::Utf8Path; use serde::{Deserialize, Serialize}; -use weaveffi_model::model::BindingModel; +use weaveffi_model::model::Model; use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; +use crate::codegen::CodeWriter; +use crate::package::{Artifact, PackageContext, PackagedFile}; use crate::platform::Platform; use crate::targets::js; +use crate::targets::node::npm_tarball_name; use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use self::glue::render_bind; +use self::glue::{bound_symbols, render_bind}; use self::package::{render_package_json, render_readme}; /// The fixed transport: loading, staging, and table functions. @@ -38,108 +38,112 @@ const LINEAR_JS: &str = include_str!("runtime/linear.js"); #[serde(default, deny_unknown_fields)] pub struct WasmConfig { /// npm package name (default: the package identity's name). - pub package_name: Option, + pub name: Option, /// Target an Emscripten build instead of a `wasm32-unknown-unknown` /// one: `init` then takes the initialized Emscripten module (or the /// promise its `MODULARIZE` factory returns), binds its /// underscore-prefixed exports, and installs table functions with /// `addFunction`. pub emscripten: bool, - /// Basename of the IDL the CLI was invoked with. - #[serde(skip)] - pub input_basename: Option, } impl WasmConfig { - fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } - fn package_name(&self, identity: &Identity) -> String { - self.package_name - .clone() - .unwrap_or_else(|| identity.name.clone()) + self.name.clone().unwrap_or_else(|| identity.name.clone()) } } /// WebAssembly backend: an ES module over linear-memory glue. pub struct WasmGenerator; -/// The generated `index.js`. -fn render_index( - model: &BindingModel, - identity: &Identity, - emscripten: bool, - input_basename: &str, -) -> String { +/// The generated `index.js`: the imports, `init()` with the load-time +/// checks, the raw entry points, and the shared API. +fn render_index(model: &Model, identity: &Identity, emscripten: bool) -> String { let name = js::names::js_string(&identity.name); - let prefix = &model.prefix; - let (bind, symbols) = render_bind(model); - let mut out = render_prelude(CommentStyle::DoubleSlash, input_basename); - out.push_str(&js::render_imports(identity)); + let prefix = model.prefix(); + let mut w = CodeWriter::two_space(); + w.raw(render_prelude(CommentStyle::DoubleSlash)); + js::render_imports(&mut w, identity); let loader = if emscripten { "$loadEmscripten" } else { "$loadWasm" }; - out.push_str(&format!( - "import {{ {loader}, $unloaded }} from './linear.js';\n\n" + w.line(format!( + "import {{ {loader}, $unloaded }} from './linear.js';" )); - out.push_str(&format!("let $raw = $unloaded({name});\n")); - out.push_str("// Handles are linear-memory addresses; object tokens are u64s.\n"); - out.push_str("const $token = Number;\n"); - out.push_str("let $loading = null;\n\n"); + w.blank(); + w.line(format!("let $raw = $unloaded({name});")); + w.line("// Handles are linear-memory addresses; object tokens are u64s."); + w.line("const $token = Number;"); + w.line("let $loading = null;"); + w.blank(); + js::render_contract(&mut w, model); + w.blank(); let load = if emscripten { - let list: Vec = symbols.iter().map(|s| format!("'{s}'")).collect(); - out.push_str(&format!("const $SYMBOLS = [{}];\n\n", list.join(", "))); - out.push_str("/**\n"); - out.push_str(" * Adopt the initialized Emscripten module (or the promise its factory\n"); - out.push_str(" * returns) and check that it matches these bindings. Every other export\n"); - out.push_str(" * throws until the returned promise resolves.\n"); - out.push_str(" */\n"); - out.push_str("export function init(module) {\n"); + let list: Vec = bound_symbols(model) + .iter() + .map(|s| format!("'{s}'")) + .collect(); + w.line(format!("const $SYMBOLS = [{}];", list.join(", "))); + w.blank(); + w.line("/**"); + w.line(" * Adopt the initialized Emscripten module (or the promise its factory"); + w.line(" * returns) and check that it matches these bindings. Every other export"); + w.line(" * throws until the returned promise resolves."); + w.line(" */"); + w.line("export function init(module) {"); format!("$loadEmscripten(module, '{prefix}', $SYMBOLS)") } else { - out.push_str("/**\n"); - out.push_str(&format!( - " * Load the WebAssembly module (by default `{}.wasm` next to this file, or\n", + w.line("/**"); + w.line(format!( + " * Load the WebAssembly module (by default `{}.wasm` next to this file, or", identity.library )); - out.push_str(&format!( - " * the file named by `{}` on Node.js) and check that it matches\n", + w.line(format!( + " * the file named by `{}` on Node.js) and check that it matches", identity.library_env_var() )); - out.push_str(" * these bindings. Every other export throws until the returned promise\n"); - out.push_str(" * resolves.\n"); - out.push_str(" */\n"); - out.push_str("export function init(source) {\n"); + w.line(" * these bindings. Every other export throws until the returned promise"); + w.line(" * resolves."); + w.line(" */"); + w.line("export function init(source) {"); format!( "$loadWasm(source, '{prefix}', '{}', new URL('./{}.wasm', import.meta.url))", identity.library_env_var(), identity.library ) }; - out.push_str(&format!(" $loading ??= {load}\n")); - out.push_str(" .then((m) => {\n"); - out.push_str(" $raw = $bind(m);\n"); - out.push_str(" try {\n"); - out.push_str(&format!(" {}", js::render_verify(model, identity))); - out.push_str(" } catch (e) {\n"); - out.push_str(&format!(" $raw = $unloaded({name});\n")); - out.push_str(" throw e;\n"); - out.push_str(" }\n"); - out.push_str(" })\n"); - out.push_str(" .catch((e) => {\n"); - out.push_str(" $loading = null;\n"); - out.push_str(" throw e;\n"); - out.push_str(" });\n"); - out.push_str(" return $loading;\n"); - out.push_str("}\n\n"); - out.push_str(&bind); - out.push('\n'); - out.push_str(&js::render_api(model)); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, "index.js")); - out + w.scope(|w| { + w.line(format!("$loading ??= {load}")); + w.scope(|w| { + w.block(".then((m) => {", "})", |w| { + w.line("$raw = $bind(m);"); + w.line("try {"); + w.scope(|w| { + w.line(js::verify_call(model, identity)); + }); + w.line("} catch (e) {"); + w.scope(|w| { + w.line(format!("$raw = $unloaded({name});")); + w.line("throw e;"); + }); + w.line("}"); + }); + w.block(".catch((e) => {", "});", |w| { + w.line("$loading = null;"); + w.line("throw e;"); + }); + }); + w.line("return $loading;"); + }); + w.line("}"); + w.blank(); + render_bind(&mut w, model); + w.blank(); + js::render_api(&mut w, model); + w.raw(render_trailer(CommentStyle::DoubleSlash, "index.js")); + w.finish() } /// The transport's own declarations in `index.d.ts`. @@ -162,36 +166,35 @@ fn dts_extra(emscripten: bool) -> String { /// Every file of the generated package, as `(name, contents)` pairs. fn render_files( - model: &BindingModel, + model: &Model, identity: &Identity, config: &WasmConfig, wasm: Option<&str>, ) -> Vec<(String, String)> { - let input = config.input_basename(); let name = config.package_name(identity); let modules: Vec = model.roots().map(js::names::module_name).collect(); - let mut linear = render_prelude(CommentStyle::DoubleSlash, input); + let mut linear = render_prelude(CommentStyle::DoubleSlash); linear.push_str(LINEAR_JS); linear.push('\n'); linear.push_str(&render_trailer(CommentStyle::DoubleSlash, "linear.js")); vec![ ( "index.js".into(), - render_index(model, identity, config.emscripten, input), + render_index(model, identity, config.emscripten), ), ( "index.d.ts".into(), - js::render_dts(model, identity, input, &dts_extra(config.emscripten)), + js::render_dts(model, identity, &dts_extra(config.emscripten)), ), - ("runtime.js".into(), js::render_runtime(identity, input)), + ("runtime.js".into(), js::render_runtime(identity)), ("linear.js".into(), linear), ( "package.json".into(), - render_package_json(identity, &name, wasm, input), + render_package_json(identity, &name, wasm), ), ( "README.md".into(), - render_readme(identity, &name, &modules, config.emscripten, input), + render_readme(identity, &name, &modules, config.emscripten), ), ] } @@ -208,53 +211,48 @@ impl LanguageBackend for WasmGenerator { /// table. The target is single-threaded, so a callback only ever runs /// while a call into the module is on the stack, and an async call /// usually completes before its launcher returns. - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { + fn files(&self, model: &Model, out_dir: &Utf8Path, config: &Self::Config) -> Vec { let dir = out_dir.join("wasm"); - render_files(model, api.identity(), config, None) + render_files(model, &model.identity, config, None) .into_iter() .map(|(name, contents)| OutputFile::new(dir.join(name), contents)) .collect() } - /// The npm package with the prebuilt `wasm32-unknown-unknown` module + /// The npm tarball with the prebuilt `wasm32-unknown-unknown` module /// bundled as `{library}.wasm`, where `init()` finds it by default. /// Emscripten mode ships glue only (the consumer builds the module), and - /// without a wasm32 binary there is nothing to package. + /// without a `wasm32` build there is nothing to package. fn package( &self, - api: &ResolvedApi, - model: &BindingModel, + model: &Model, ctx: &PackageContext, - out_dir: &Utf8Path, config: &Self::Config, - ) -> Option> { - let dir = out_dir.join("wasm"); - let identity = api.identity(); + ) -> Option> { + let identity = &model.identity; let wasm_name = format!("{}.wasm", identity.library); let binary = if config.emscripten { None } else { - Some(ctx.binaries.get(Platform::Wasm32)?) + match ctx.binaries.get(Platform::Wasm32) { + Some(binary) => Some(binary), + None => return Some(Vec::new()), + } }; let mut files: Vec = render_files(model, identity, config, binary.map(|_| wasm_name.as_str())) .into_iter() - .map(|(name, contents)| PackagedFile::text(dir.join(name), contents)) + .map(|(name, contents)| PackagedFile::text(name, contents)) .collect(); if let Some(b) = binary { - files.push(PackagedFile::copy(dir.join(&wasm_name), b.source.clone())); + files.push(PackagedFile::copy(wasm_name, b.library.clone())); } - Some(files) + let name = config.package_name(identity); + Some(vec![Artifact::tar_gz( + format!("wasm/{}", npm_tarball_name(&name, &identity.version)), + "package", + files, + )]) } } diff --git a/crates/weaveffi-gen/src/targets/wasm/package.rs b/crates/weaveffi-cli/src/targets/wasm/package.rs similarity index 87% rename from crates/weaveffi-gen/src/targets/wasm/package.rs rename to crates/weaveffi-cli/src/targets/wasm/package.rs index c6d9d7ee..ff07ba19 100644 --- a/crates/weaveffi-gen/src/targets/wasm/package.rs +++ b/crates/weaveffi-cli/src/targets/wasm/package.rs @@ -8,17 +8,12 @@ use crate::utils::{render_prelude, render_trailer, CommentStyle}; /// Render `package.json`. `wasm` is the bundled module's file name, when /// the package carries one. -pub(crate) fn render_package_json( - identity: &Identity, - name: &str, - wasm: Option<&str>, - input_basename: &str, -) -> String { +pub(crate) fn render_package_json(identity: &Identity, name: &str, wasm: Option<&str>) -> String { let mut files = vec!["index.js", "index.d.ts", "runtime.js", "linear.js"]; if let Some(w) = wasm { files.push(w); } - npm_metadata(identity, name, input_basename) + npm_metadata(identity, name) .entry("files", JsonValue::str_array(files)) .render() } @@ -29,11 +24,10 @@ pub(crate) fn render_readme( name: &str, modules: &[String], emscripten: bool, - input_basename: &str, ) -> String { let lib = &identity.library; let env = identity.library_env_var(); - let mut out = render_prelude(CommentStyle::Xml, input_basename); + let mut out = render_prelude(CommentStyle::Xml); out.push_str(&format!( "# {name}\n\n{}\n\n", identity.description_or_default() diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__linear_js.snap b/crates/weaveffi-cli/src/targets/wasm/runtime/linear.js similarity index 67% rename from crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__linear_js.snap rename to crates/weaveffi-cli/src/targets/wasm/runtime/linear.js index 608de11a..ec79ab51 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__linear_js.snap +++ b/crates/weaveffi-cli/src/targets/wasm/runtime/linear.js @@ -1,15 +1,14 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - // The WebAssembly transport: loading the module, staging arguments in its // linear memory, reading results and errors back, and installing JavaScript // functions in its function table (async completions and callback // interface vtables). The generated `index.js` builds one native entry point // per C symbol on top of this. +// +// Byte runs from `{prefix}_alloc` have alignment 1, so every multi-byte +// read and write goes through a `DataView`, and the slots the producer +// writes through typed pointers (the error struct, `out_len`, iterator +// items, vtables) live in 8-byte-aligned blocks carved from the transport's +// own arenas. import { $Fault } from './runtime.js'; @@ -33,6 +32,9 @@ function $bigint(v, bits, what) { return b; } +/** The size of each arena the transport carves its permanent blocks from. */ +const $ARENA = 4096; + /** * One loaded module's linear memory and runtime symbols. `x` holds the * module's exports by C symbol name; `memory()` returns the current @@ -45,20 +47,25 @@ export class $Linear { this.memory = memory; this.fnptr = fnptr; this.allocFn = x[`${prefix}_alloc`]; - this.deallocFn = x[`${prefix}_dealloc`]; this.freeBytesFn = x[`${prefix}_free_bytes`]; this.errorClearFn = x[`${prefix}_error_clear`]; this.errorFreeFn = x[`${prefix}_error_free`]; this.errorSetFn = x[`${prefix}_error_set`]; - if (typeof this.allocFn !== 'function' || typeof this.deallocFn !== 'function') { - throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_dealloc`); + this.errorSetPayloadFn = x[`${prefix}_error_set_payload`]; + if (typeof this.allocFn !== 'function' || typeof this.freeBytesFn !== 'function') { + throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_free_bytes`); } + // Permanent blocks (the scratch slots and the vtables) come from + // arenas allocated with {prefix}_alloc and never freed; `arenas` counts + // them so the leak counter of byte runs can leave them out. + this.arenas = 0; + this.arenaNext = 0; + this.arenaEnd = 0; // One scratch block reused by every call: the error slot (16 bytes), // the out_len slot (4), and the iterator out_item slot (8). The producer // only writes them as a call returns, so a call made from a callback in // the middle of another call can reuse them safely. - const scratch = this.allocFn(32); - this.bytes().fill(0, scratch, scratch + 32); + const scratch = this.reserve(32); this.err = scratch; this.len = scratch + 16; this.item = scratch + 24; @@ -85,6 +92,38 @@ export class $Linear { return this.dv; } + /** A run of `size` bytes from {prefix}_alloc (`0` for an empty one). */ + alloc(size) { + if (size === 0) return 0; + const ptr = this.allocFn(size) >>> 0; + if (ptr === 0) throw new RangeError(`the WebAssembly module could not allocate ${size} bytes`); + return ptr; + } + + /** A permanent, zero-filled, 8-byte-aligned block of `size` bytes. */ + reserve(size) { + const need = Math.ceil(size / 8) * 8; + if (this.arenaNext + need > this.arenaEnd) { + const length = Math.max($ARENA, need + 8); + const base = this.alloc(length); + this.arenas++; + this.arenaNext = Math.ceil(base / 8) * 8; + this.arenaEnd = base + length; + } + const at = this.arenaNext; + this.arenaNext += need; + return at; + } + + /** + * A `{prefix}_debug_live(kind)` result as a bigint, leaving out the + * transport's own arenas from the count of byte runs (kind 4). + */ + live(raw, kind) { + const n = BigInt.asUintN(64, raw); + return kind === 4 && n > 0n ? n - BigInt(this.arenas) : n; + } + // Argument checks and conversions for by-value slots. num(v) { if (typeof v !== 'number') throw $expected('a number', v); @@ -113,13 +152,13 @@ export class $Linear { return v === null || v === undefined ? 0 : this.handle(v); } - // Staging: copy a string or bytes into linear memory for one call. A - // staged value is [ptr, len, size]; release it with unstage(). + // Staging: copy a string or bytes into a run for one call. A staged value + // is [ptr, len, size]; release it with unstage(). str(s) { if (typeof s !== 'string') throw $expected('a string', s); if (s.length === 0) return [0, 0, 0]; const size = s.length * 3; - const ptr = this.allocFn(size); + const ptr = this.alloc(size); const { written } = $enc.encodeInto(s, this.bytes().subarray(ptr, ptr + size)); return [ptr, written, size]; } @@ -127,13 +166,13 @@ export class $Linear { data(b) { if (!(b instanceof Uint8Array)) throw $expected('a Uint8Array', b); if (b.length === 0) return [0, 0, 0]; - const ptr = this.allocFn(b.length); + const ptr = this.alloc(b.length); this.bytes().set(b, ptr); return [ptr, b.length, b.length]; } unstage(staged) { - if (staged !== null && staged[2] !== 0) this.deallocFn(staged[0], staged[2]); + if (staged !== null && staged[2] !== 0) this.freeBytesFn(staged[0], staged[2]); } // Results. @@ -142,10 +181,12 @@ export class $Linear { } readStr(ptr, len) { + ptr >>>= 0; return len === 0 ? '' : $dec.decode(this.bytes().subarray(ptr, ptr + len)); } readData(ptr, len) { + ptr >>>= 0; return len === 0 ? new Uint8Array(0) : this.bytes().slice(ptr, ptr + len); } @@ -162,6 +203,7 @@ export class $Linear { } cstr(ptr) { + ptr >>>= 0; if (ptr === 0) return ''; const u8 = this.bytes(); let end = ptr; @@ -178,7 +220,7 @@ export class $Linear { return new $Fault( code, this.cstr(dv.getUint32(errPtr + 4, true)), - payloadPtr === 0 ? null : this.bytes().slice(payloadPtr, payloadPtr + payloadLen), + payloadPtr === 0 ? null : this.readData(payloadPtr, payloadLen), ); } @@ -190,9 +232,21 @@ export class $Linear { throw fault; } + /** A module's contract table as a `BigUint64Array` of `id, hash` pairs. */ + contract(table) { + const ptr = table(this.len) >>> 0; + const n = this.outLen() * 2; + const dv = this.view(); + const out = new BigUint64Array(n); + for (let i = 0; i < n; i++) out[i] = dv.getBigUint64(ptr + i * 8, true); + return out; + } + // Callback interfaces: implementations live in a table keyed by the // integer passed as `ctx`, until the producer calls the vtable's `free`. + // An absent optional implementation is ctx 0 with a null vtable. register(adapter) { + if (adapter === null) return 0; const id = this.nextCallback++; this.callbacks.set(id, adapter); return id; @@ -208,36 +262,91 @@ export class $Linear { this.callbacks.delete(id); } - /** Report a callback implementation's exception through `errPtr` (code -4). */ + /** Stage `text` as a NUL-terminated C string (release it with unstage()). */ + cstrArg(text) { + const bytes = $enc.encode(text); + const ptr = this.alloc(bytes.length + 1); + this.bytes().set(bytes, ptr); + return [ptr, bytes.length, bytes.length + 1]; + } + + /** + * Report a callback implementation's exception through `errPtr`: a + * `$Fault` (a domain error a throwing method raised) with its code, + * message, and payload, anything else as code -4 with its message. + */ foreign(errPtr, e) { - const message = e instanceof Error ? e.message || e.name : String(e); - const bytes = $enc.encode(message); - const ptr = this.allocFn(bytes.length + 1); - const u8 = this.bytes(); - u8.set(bytes, ptr); - u8[ptr + bytes.length] = 0; - this.errorSetFn(errPtr, -4, ptr); - this.deallocFn(ptr, bytes.length + 1); + let code = -4; + let message; + let payload = null; + if (e instanceof $Fault) { + code = e.code; + message = e.message; + payload = e.payload; + } else { + message = e instanceof Error ? e.message || e.name : String(e); + } + const text = this.cstrArg(message); + try { + this.errorSetFn(errPtr, code, text[0]); + } finally { + this.unstage(text); + } + if (payload !== null && payload.length > 0) { + const staged = this.data(payload); + try { + this.errorSetPayloadFn(errPtr, staged[0], staged[1]); + } finally { + this.unstage(staged); + } + } + } + + /** + * Hand `bytes` (a `Uint8Array`) to the producer through a callback's + * `out_ptr`/`out_len` slots, as a run from {prefix}_alloc it adopts. + */ + give(outPtr, outLen, bytes) { + if (!(bytes instanceof Uint8Array)) throw $expected('a Uint8Array', bytes); + const run = this.alloc(bytes.length); + if (bytes.length > 0) this.bytes().set(bytes, run); + const dv = this.view(); + dv.setUint32(outPtr >>> 0, run, true); + dv.setUint32(outLen >>> 0, bytes.length, true); + } + + /** `give` for a string, as UTF-8. */ + giveStr(outPtr, outLen, s) { + if (typeof s !== 'string') throw $expected('a string', s); + this.give(outPtr, outLen, $enc.encode(s)); } /** - * The static vtable of one callback interface: one table entry per - * method (`methods()` returns each as `[params, results, sig, fn]`), then - * `free`. Built on first use. + * The static vtable of one callback interface, built on first use: the + * header (`size`, `flags`, then the `free` entry), then one table entry + * per method (`methods()` returns each as `[params, results, sig, fn]`). */ vtable(name, methods) { let ptr = this.vtables.get(name); if (ptr !== undefined) return ptr; - const entries = [...methods(), [['i32'], [], 'vi', (ctx) => this.unregister(ctx)]]; - ptr = this.allocFn(entries.length * 4); + const entries = [[['i32'], [], 'vi', (ctx) => this.unregister(ctx)], ...methods()]; + const size = 8 + entries.length * 4; + ptr = this.reserve(size); const dv = this.view(); + dv.setUint32(ptr, size, true); + dv.setUint32(ptr + 4, 0, true); entries.forEach(([params, results, sig, fn], i) => { - dv.setUint32(ptr + i * 4, this.fnptr(params, results, sig, fn), true); + dv.setUint32(ptr + 8 + i * 4, this.fnptr(params, results, sig, fn), true); }); this.vtables.set(name, ptr); return ptr; } + /** `vtable`, or a null vtable for an absent optional implementation. */ + vtableOpt(adapter, name, methods) { + return adapter === null ? 0 : this.vtable(name, methods); + } + /** * Launch an async call. `call(callback, context)` invokes the launcher; * the completion (one table entry per signature, shared by every call) @@ -269,7 +378,7 @@ export class $Linear { if (pending === undefined) return; this.pending.delete(id); try { - const fault = errPtr === 0 ? null : this.fault(errPtr); + const fault = errPtr === 0 ? null : this.fault(errPtr >>> 0); if (errPtr !== 0) this.errorFreeFn(errPtr); if (fault !== null) { pending.reject(fault); @@ -399,5 +508,3 @@ export async function $loadEmscripten(module, prefix, symbols) { }; return new $Linear(x, prefix, () => m.HEAPU8.buffer, fnptr); } - -// END linear.js diff --git a/crates/weaveffi-gen/src/targets/wasm/tests.rs b/crates/weaveffi-cli/src/targets/wasm/tests.rs similarity index 61% rename from crates/weaveffi-gen/src/targets/wasm/tests.rs rename to crates/weaveffi-cli/src/targets/wasm/tests.rs index 40d46d3c..8817962c 100644 --- a/crates/weaveffi-gen/src/targets/wasm/tests.rs +++ b/crates/weaveffi-cli/src/targets/wasm/tests.rs @@ -1,12 +1,11 @@ use super::*; -use crate::platform::BinarySet; +use crate::platform::{BinarySet, NativeBinary}; use crate::targets::js::test_api; fn index(config: &WasmConfig) -> String { - let api = test_api("acme-kv"); - let model = BindingModel::build(&api); + let model = test_api("acme-kv"); WasmGenerator - .files(&api, &model, Utf8Path::new("out"), config) + .files(&model, Utf8Path::new("out"), config) .into_iter() .find(|f| f.path.file_name() == Some("index.js")) .expect("index.js") @@ -19,9 +18,8 @@ fn has(src: &str, needle: &str) { #[test] fn files_are_named_for_the_package() { - let api = test_api("acme-kv"); - let model = BindingModel::build(&api); - let out = WasmGenerator.files(&api, &model, Utf8Path::new("out"), &WasmConfig::default()); + let model = test_api("acme-kv"); + let out = WasmGenerator.files(&model, Utf8Path::new("out"), &WasmConfig::default()); let names: Vec<&str> = out.iter().filter_map(|f| f.path.file_name()).collect(); assert_eq!( names, @@ -49,8 +47,21 @@ fn init_loads_and_verifies_the_module() { &js, "$loading ??= $loadWasm(source, 'acme_kv', 'ACME_KV_LIBRARY', new URL('./acme_kv.wasm', import.meta.url))", ); - has(&js, "$verify($raw, 'acme-kv', 'acme_kv', 3, [['kv', 0x"); + has(&js, "const $contract = [\n ['kv', [\n"); + has( + &js, + " try {\n $verify($raw, 'acme-kv', 'acme_kv', 4, $contract);\n } catch (e) {", + ); + has( + &js, + "acme_kv_kv_contract: () => m.contract(x.acme_kv_kv_contract),", + ); + has( + &js, + "acme_kv_debug_live: (kind) => m.live(x.acme_kv_debug_live(m.num(kind)), kind),", + ); assert!(!js.contains("weaveffi_"), "{js}"); + assert!(!js.contains("checksum"), "{js}"); } #[test] @@ -84,6 +95,10 @@ fn async_and_callbacks_install_table_functions() { &js, "m.register(a0), m.vtable('acme_kv_kv_Listener', () => $vt_acme_kv_kv_Listener(m))", ); + has( + &js, + "x.acme_kv_kv_install(m.register(a0), m.vtableOpt(a0, 'acme_kv_kv_Policy', () => $vt_acme_kv_kv_Policy(m)), m.err);", + ); has( &js, "[['i32', 'i32', 'i32', 'i64', 'i32'], ['i32'], 'iiiiji', (ctx, p0, p1, p2, err) => {", @@ -99,6 +114,28 @@ fn async_and_callbacks_install_table_functions() { ); } +#[test] +fn callback_returns_of_every_family_reach_the_producer() { + let js = index(&WasmConfig::default()); + // Strings, bytes, and buffers go out through the out slots as runs. + has( + &js, + "[['i32', 'i32', 'i32', 'i32', 'i32', 'i32'], [], 'viiiiii', (ctx, p0, p1, out, outLen, err) => {\n try {\n m.give(out, outLen, m.adapter(ctx).admit(m.readData(p0, p1)));", + ); + has(&js, "m.giveStr(out, outLen, m.adapter(ctx).label());"); + has(&js, "m.give(out, outLen, m.adapter(ctx).blob());"); + // Objects and direct values are the table function's result. + has( + &js, + "return m.handleOpt(m.adapter(ctx).pick(m.readStr(p0, p1)));\n } catch (e) {\n m.foreign(err, e);\n return 0;\n }", + ); + has(&js, "return m.handleOpt(m.adapter(ctx).maybe());"); + has( + &js, + "return m.adapter(ctx).weight();\n } catch (e) {\n m.foreign(err, e);\n return 0n;", + ); +} + #[test] fn iterators_read_the_item_slot() { let js = index(&WasmConfig::default()); @@ -125,9 +162,11 @@ fn emscripten_mode_binds_underscored_exports() { ); has( &js, - "const $SYMBOLS = ['acme_kv_abi_version', 'acme_kv_alloc',", + "const $SYMBOLS = ['acme_kv_abi_version', 'acme_kv_alloc', 'acme_kv_free_bytes', 'acme_kv_error_set', 'acme_kv_error_set_payload',", ); + has(&js, "'acme_kv_kv_contract'"); has(&js, "'acme_kv_kv_Store_wait'"); + assert!(!js.contains("dealloc"), "{js}"); has( &js, "$loading ??= $loadEmscripten(module, 'acme_kv', $SYMBOLS)", @@ -136,57 +175,35 @@ fn emscripten_mode_binds_underscored_exports() { #[test] fn package_bundles_the_wasm_module() { - let api = test_api("kv"); - let model = BindingModel::build(&api); + let model = test_api("kv"); let mut binaries = BinarySet::new("kv"); - binaries.insert(Platform::Wasm32, "target/kv.wasm"); - let ctx = PackageContext { - binaries: &binaries, - input_basename: None, - }; - let files = WasmGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &WasmConfig::default(), - ) + binaries.insert(NativeBinary::new(Platform::Wasm32, "target/kv.wasm")); + let ctx = PackageContext::new(&binaries); + let artifacts = WasmGenerator + .package(&model, &ctx, &WasmConfig::default()) .expect("a wasm32 binary packages"); - let wasm = files - .iter() - .find(|f| f.path == "out/wasm/kv.wasm") - .expect("bundled module"); - assert!(wasm.is_binary()); - let PackagedFile { content, .. } = files - .iter() - .find(|f| f.path == "out/wasm/package.json") - .expect("manifest"); + assert_eq!(artifacts.len(), 1); + let tgz = &artifacts[0]; + assert_eq!(tgz.path, "wasm/kv-0.1.0.tgz"); + assert!(tgz.file("kv.wasm").expect("bundled module").is_binary()); + let PackagedFile { content, .. } = tgz.file("package.json").expect("manifest"); let crate::package::FileContent::Text(manifest) = content else { panic!("manifest is text") }; has(manifest, "\"kv.wasm\""); let none = BinarySet::new("kv"); - let ctx = PackageContext { - binaries: &none, - input_basename: None, - }; + let ctx = PackageContext::new(&none); assert!(WasmGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &WasmConfig::default() - ) - .is_none()); + .package(&model, &ctx, &WasmConfig::default()) + .unwrap() + .is_empty()); let emscripten = WasmConfig { emscripten: true, ..WasmConfig::default() }; let glue = WasmGenerator - .package(&api, &model, &ctx, Utf8Path::new("out"), &emscripten) + .package(&model, &ctx, &emscripten) .expect("Emscripten mode ships glue only"); - assert!(glue.iter().all(|f| !f.is_binary())); + assert!(glue[0].files.iter().all(|f| !f.is_binary())); } diff --git a/crates/weaveffi-cli/src/utils.rs b/crates/weaveffi-cli/src/utils.rs new file mode 100644 index 00000000..bddfdd30 --- /dev/null +++ b/crates/weaveffi-cli/src/utils.rs @@ -0,0 +1,103 @@ +//! Shared text helpers for the generators: the standard generated-file +//! prelude and trailer. +//! +//! These are the small string-building routines every backend reaches for +//! when stamping the "generated by WeaveFFI" banner onto a file. Centralizing +//! them keeps the banner identical across all of the language generators. + +/// Comment syntax used to emit the standard prelude/trailer in generated files. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum CommentStyle { + /// `// ...` line comments (C, C++, Swift, Kotlin, JS/TS, C#, Dart, Go). + DoubleSlash, + /// `# ...` line comments (Python, Ruby, YAML, TOML, CMake, GYP, gradle.properties). + Hash, + /// `` block comments (XML, HTML, Markdown). + Xml, +} + +impl CommentStyle { + fn open(self) -> &'static str { + match self { + Self::DoubleSlash => "// ", + Self::Hash => "# ", + Self::Xml => "", + _ => "", + } + } +} + +/// Renders the standard `Generated by WeaveFFI {VERSION}` prelude followed +/// by the `DO NOT EDIT` warning and the regenerate command. Trailing blank +/// line included so generators can append their content directly. +pub fn render_prelude(style: CommentStyle) -> String { + let version = env!("CARGO_PKG_VERSION"); + let o = style.open(); + let c = style.close(); + format!( + "{o}Generated by WeaveFFI {version}{c}\n\ + {o}DO NOT EDIT. Your changes will be overwritten.{c}\n\ + {o}To regenerate: weaveffi generate{c}\n\n" + ) +} + +/// Renders the closing `END {filename}` marker. Caller is responsible for any +/// preceding newline; the returned string ends with `\n`. +pub fn render_trailer(style: CommentStyle, filename: &str) -> String { + let o = style.open(); + let c = style.close(); + format!("{o}END {filename}{c}\n") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn prelude_double_slash_carries_required_phrases() { + let p = render_prelude(CommentStyle::DoubleSlash); + assert!(p.starts_with("// Generated by WeaveFFI ")); + assert!(p.contains("// DO NOT EDIT")); + assert!(p.contains("// To regenerate: weaveffi generate\n")); + assert!(p.ends_with("\n\n")); + } + + #[test] + fn prelude_hash_uses_hash_marker() { + let p = render_prelude(CommentStyle::Hash); + assert!(p.starts_with("# Generated by WeaveFFI ")); + assert!(p.contains("# DO NOT EDIT")); + assert!(p.contains("# To regenerate: weaveffi generate\n")); + } + + #[test] + fn prelude_xml_wraps_lines_in_brackets() { + let p = render_prelude(CommentStyle::Xml); + assert!(p.starts_with("\n")); + } + + #[test] + fn trailer_has_correct_marker_per_style() { + assert_eq!( + render_trailer(CommentStyle::DoubleSlash, "lib.rs"), + "// END lib.rs\n" + ); + assert_eq!( + render_trailer(CommentStyle::Hash, "build.toml"), + "# END build.toml\n" + ); + assert_eq!( + render_trailer(CommentStyle::Xml, "package.csproj"), + "\n" + ); + } +} diff --git a/crates/weaveffi-cli/tests/abi_version.rs b/crates/weaveffi-cli/tests/abi_version.rs new file mode 100644 index 00000000..24e702c5 --- /dev/null +++ b/crates/weaveffi-cli/tests/abi_version.rs @@ -0,0 +1,9 @@ +//! The ABI revision is declared twice on purpose: once in the `weaveffi` +//! runtime (which producers link) and once in the model the generators read. +//! The runtime must not depend on the generator stack, so this test is what +//! keeps the two numbers equal. + +#[test] +fn runtime_and_generator_abi_revisions_agree() { + assert_eq!(weaveffi::abi::ABI_VERSION, weaveffi_cli::cabi::ABI_VERSION); +} diff --git a/crates/weaveffi-gen/tests/c_buffer.rs b/crates/weaveffi-cli/tests/c_buffer.rs similarity index 90% rename from crates/weaveffi-gen/tests/c_buffer.rs rename to crates/weaveffi-cli/tests/c_buffer.rs index eb39a910..5da40b62 100644 --- a/crates/weaveffi-gen/tests/c_buffer.rs +++ b/crates/weaveffi-cli/tests/c_buffer.rs @@ -1,21 +1,21 @@ //! Checks for the C target's `{library}_buffer.h` value-buffer helper -//! header: which files the generator emits, the shapes it renders, and (when -//! a C compiler is available) that its codecs round-trip values and reject -//! malformed buffers. +//! header: which files the generator emits, the shapes it renders, and that +//! its codecs round-trip values and reject malformed buffers when compiled as +//! C and as C++. A missing compiler skips its half of the round trip locally +//! and fails the test under CI (`CI=true`). use std::process::Command; use camino::Utf8Path; -use weaveffi_gen::backend::LanguageBackend; -use weaveffi_gen::targets::c::{CConfig, CGenerator}; -use weaveffi_model::model::BindingModel; +use weaveffi_cli::backend::LanguageBackend; +use weaveffi_cli::targets::c::{CConfig, CGenerator}; +use weaveffi_model::model::Model; use weaveffi_model::parse::parse_api_str; use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; -use weaveffi_model::validate::validate_api; +use weaveffi_model::validate::validate; const SHOP: &str = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: shop enums: @@ -62,7 +62,7 @@ modules: "#; const PLAIN: &str = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: math functions: @@ -73,17 +73,14 @@ modules: return: i32 "#; -fn load(yaml: &str, name: &str) -> ResolvedApi { +fn load(yaml: &str, name: &str) -> Model { let api = parse_api_str(yaml, "yaml").expect("parse"); - validate_api(api, None) - .expect("validate") - .with_identity(Identity::named(name)) + validate(&api, &Identity::named(name), None).expect("validate") } -fn generate(api: &ResolvedApi, config: &CConfig, dir: &Utf8Path) -> Vec<(String, String)> { - let model = BindingModel::build(api); +fn generate(model: &Model, config: &CConfig, dir: &Utf8Path) -> Vec<(String, String)> { CGenerator - .files(api, &model, dir, config) + .files(model, dir, config) .into_iter() .map(|f| { let name = f.path.file_name().unwrap_or_default().to_string(); @@ -113,7 +110,6 @@ fn emits_the_helper_header_only_when_buffers_cross_the_abi() { let off = CConfig { buffer_helpers: false, - ..CConfig::default() }; let names: Vec = generate(&api, &off, Utf8Path::new("out")) .into_iter() @@ -261,20 +257,26 @@ int main(void) { } "#; +/// Whether `tool` runs. A missing tool is a printed skip locally and a +/// failure under CI, where every runner has both compilers. fn have(tool: &str) -> bool { - Command::new(tool) + let found = Command::new(tool) .arg("--version") .output() - .is_ok_and(|o| o.status.success()) + .is_ok_and(|o| o.status.success()); + if !found { + assert!( + std::env::var("CI").as_deref() != Ok("true"), + "{tool} not found (required when CI=true)", + ); + eprintln!("{tool} not found; skipping its round trip"); + } + found } #[test] fn codecs_round_trip_in_c_and_cpp() { let compilers = [("cc", "c", "-std=c11"), ("c++", "cpp", "-std=c++17")]; - if !compilers.iter().any(|(cc, _, _)| have(cc)) { - eprintln!("no C compiler found; skipping"); - return; - } let tmp = tempfile::tempdir().expect("tempdir"); let dir = Utf8Path::from_path(tmp.path()).expect("utf8"); let api = load(SHOP, "shop"); diff --git a/crates/weaveffi-cli/tests/cli/check.rs b/crates/weaveffi-cli/tests/cli/check.rs index 7952cbed..fc510b32 100644 --- a/crates/weaveffi-cli/tests/cli/check.rs +++ b/crates/weaveffi-cli/tests/cli/check.rs @@ -15,21 +15,21 @@ fn write_file(path: &Path, contents: &str) { f.write_all(contents.as_bytes()).unwrap(); } -fn calculator_idl() -> std::path::PathBuf { +fn calculator_crate() -> std::path::PathBuf { let manifest_dir = env!("CARGO_MANIFEST_DIR"); Path::new(manifest_dir) .parent() .unwrap() .parent() .unwrap() - .join("samples/calculator/src/lib.rs") + .join("samples/calculator") } #[test] fn diff_check_passes_when_output_matches() { let tmp = tempfile::tempdir().unwrap(); let out = tmp.path().join("generated"); - let input = calculator_idl(); + let input = calculator_crate(); cargo_bin() .args([ @@ -76,7 +76,7 @@ fn diff_check_fails_when_idl_changed() { write_file( &idl, concat!( - "version: \"0.10.0\"\n", + "version: \"0.11.0\"\n", "modules:\n", " - name: calc\n", " functions:\n", @@ -102,7 +102,7 @@ fn diff_check_fails_when_idl_changed() { write_file( &idl, concat!( - "version: \"0.10.0\"\n", + "version: \"0.11.0\"\n", "modules:\n", " - name: calc\n", " functions:\n", @@ -156,7 +156,7 @@ fn diff_check_fails_when_idl_changed() { #[test] fn validate_json_format_outputs_object() { - let input = calculator_idl(); + let input = calculator_crate(); let output = cargo_bin() .args(["validate", input.to_str().unwrap(), "--format", "json"]) @@ -189,7 +189,7 @@ fn validate_warn_json_format_outputs_warnings_array() { write_file( &path, concat!( - "version: \"0.10.0\"\n", + "version: \"0.11.0\"\n", "modules:\n", " - name: nodocs\n", " functions:\n", @@ -236,3 +236,45 @@ fn validate_warn_json_format_outputs_warnings_array() { ); assert!(output.status.success(), "warnings must not fail validate"); } + +#[test] +fn validate_json_failures_carry_the_code_and_fields() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("global.yml"); + write_file( + &path, + concat!( + "version: \"0.11.0\"\n", + "modules:\n", + " - name: a\n", + " structs: [{ name: Item, fields: [{ name: n, type: usize }] }]\n", + " functions: [{ name: open, params: [{ name: i, type: a.Item }] }]\n", + " - name: b\n", + " functions: [{ name: open }]\n", + ), + ); + + let output = cargo_bin() + .args(["validate", path.to_str().unwrap(), "--format", "json"]) + .output() + .expect("failed to run weaveffi validate --format json"); + assert!(!output.status.success(), "global name clashes must fail"); + + let stdout = String::from_utf8_lossy(&output.stdout); + let parsed: serde_json::Value = + serde_json::from_str(stdout.trim()).expect("stdout must be valid JSON"); + let errors = parsed["errors"].as_array().expect("errors array"); + let find = |code: &str| { + errors + .iter() + .find(|e| e["code"] == code) + .unwrap_or_else(|| panic!("no {code} in {parsed}")) + }; + assert_eq!(find("UnsupportedPrimitive")["name"], "usize"); + assert_eq!(find("QualifiedTypeRef")["name"], "a.Item"); + let dup = find("DuplicateFunctionName"); + assert_eq!(dup["name"], "open"); + assert_eq!(dup["first"], "a"); + assert_eq!(dup["second"], "b"); + assert!(dup["suggestion"].as_str().is_some_and(|s| !s.is_empty())); +} diff --git a/crates/weaveffi-cli/tests/cli/determinism.rs b/crates/weaveffi-cli/tests/cli/determinism.rs index b5e9fc36..4a396a3e 100644 --- a/crates/weaveffi-cli/tests/cli/determinism.rs +++ b/crates/weaveffi-cli/tests/cli/determinism.rs @@ -6,24 +6,27 @@ use std::path::Path; use camino::Utf8Path; +use weaveffi_model::model::Model; use weaveffi_model::parse::parse_api_str; -use weaveffi_model::resolved::ResolvedApi; -use weaveffi_model::validate::validate_api; +use weaveffi_model::validate::validate; -fn load_kitchen_sink() -> ResolvedApi { +fn load_kitchen_sink() -> Model { let path = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/kitchen_sink.yml"); let contents = std::fs::read_to_string(&path).expect("read fixture"); let api = parse_api_str(&contents, "yaml").expect("parse fixture"); - validate_api(api, None) - .expect("validate fixture") - .with_identity(weaveffi_model::pkg::Identity::named("kitchen_sink")) + validate( + &api, + &weaveffi_model::pkg::Identity::named("kitchen_sink"), + None, + ) + .expect("validate fixture") } #[test] fn generator_output_is_byte_identical_across_runs() { let api = load_kitchen_sink(); let out = Utf8Path::new("out"); - for target in weaveffi_gen::targets::all_default() { + for target in weaveffi_cli::targets::all_default() { let a = target.render(&api, out); let b = target.render(&api, out); assert!(!a.is_empty(), "{} produced no files", target.name()); diff --git a/crates/weaveffi-cli/tests/cli/diagnostics.rs b/crates/weaveffi-cli/tests/cli/diagnostics.rs index 8ad1175e..8b819400 100644 --- a/crates/weaveffi-cli/tests/cli/diagnostics.rs +++ b/crates/weaveffi-cli/tests/cli/diagnostics.rs @@ -25,7 +25,7 @@ fn write_temp_file(dir: &tempfile::TempDir, name: &str, contents: &str) -> std:: #[test] fn parse_yaml_error_includes_line() { let dir = tempfile::tempdir().unwrap(); - let yaml = "version: \"0.10.0\"\nmodules:\n - name: [bad\n"; + let yaml = "version: \"0.11.0\"\nmodules:\n - name: [bad\n"; let path = write_temp_file(&dir, "broken.yml", yaml); let output = cargo_bin() @@ -56,7 +56,7 @@ fn parse_yaml_error_includes_line() { #[test] fn validate_unknown_typeref_includes_offending_name() { let dir = tempfile::tempdir().unwrap(); - let yaml = r#"version: "0.10.0" + let yaml = r#"version: "0.11.0" modules: - name: m functions: @@ -94,7 +94,7 @@ modules: #[test] fn validate_duplicate_module_includes_offending_name() { let dir = tempfile::tempdir().unwrap(); - let yaml = r#"version: "0.10.0" + let yaml = r#"version: "0.11.0" modules: - name: alpha functions: diff --git a/crates/weaveffi-cli/tests/cli/diff.rs b/crates/weaveffi-cli/tests/cli/diff.rs index dd527398..86dd4fb4 100644 --- a/crates/weaveffi-cli/tests/cli/diff.rs +++ b/crates/weaveffi-cli/tests/cli/diff.rs @@ -6,7 +6,7 @@ use predicates::prelude::*; fn diff_against_empty_dir() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/calculator/src/lib.rs"); + let input = repo_root.join("samples/calculator"); let tmp = tempfile::tempdir().expect("failed to create temp dir"); let empty_out = tmp.path().join("empty"); @@ -39,7 +39,7 @@ fn diff_against_empty_dir() { fn diff_no_changes() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/calculator/src/lib.rs"); + let input = repo_root.join("samples/calculator"); let tmp = tempfile::tempdir().expect("failed to create temp dir"); let out_path = tmp.path().join("generated"); @@ -76,7 +76,7 @@ fn diff_no_changes() { fn diff_check_honors_inline_generators() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/kvstore/src/lib.rs"); + let input = repo_root.join("samples/kvstore"); let tmp = tempfile::tempdir().expect("failed to create temp dir"); let out_path = tmp.path().join("generated"); diff --git a/crates/weaveffi-cli/tests/cli/errors.rs b/crates/weaveffi-cli/tests/cli/errors.rs index 0b078cf7..2373a27b 100644 --- a/crates/weaveffi-cli/tests/cli/errors.rs +++ b/crates/weaveffi-cli/tests/cli/errors.rs @@ -85,7 +85,7 @@ fn parse_error_toml_shows_filename() { fn validation_error_duplicate_module_shows_suggestion() { let dir = tempfile::tempdir().unwrap(); let yaml = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: foo functions: @@ -123,7 +123,7 @@ modules: fn async_function_validates_successfully() { let dir = tempfile::tempdir().unwrap(); let yaml = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: mymod functions: @@ -149,7 +149,7 @@ modules: fn validation_error_duplicate_function_shows_suggestion() { let dir = tempfile::tempdir().unwrap(); let yaml = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: mymod functions: @@ -209,7 +209,7 @@ fn generate_parse_error_shows_filename_and_suggestion() { fn generate_validation_error_shows_suggestion() { let dir = tempfile::tempdir().unwrap(); let yaml = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: dup functions: @@ -249,7 +249,7 @@ modules: fn validate_with_warnings() { let dir = tempfile::tempdir().unwrap(); let yaml = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: nodocs functions: diff --git a/crates/weaveffi-cli/tests/cli/extract.rs b/crates/weaveffi-cli/tests/cli/extract.rs index f271bc10..22e39931 100644 --- a/crates/weaveffi-cli/tests/cli/extract.rs +++ b/crates/weaveffi-cli/tests/cli/extract.rs @@ -1,708 +1,152 @@ -//! End-to-end tests for `weaveffi extract`, which reads annotated Rust source -//! (via the shared `weaveffi_model::rust`) and emits the IDL. The annotation scheme -//! is the one the `#[weaveffi::module]` macro uses: a `#[weaveffi::module]` -//! marks the exported namespace, and item markers (`#[weaveffi::export]`, -//! `#[weaveffi::record]`, `#[weaveffi::enumeration]`, ...) tag the surface. +//! Library mode end to end: `weaveffi extract` and `weaveffi generate` read +//! a Rust producer's API from the library it builds. The producer is the +//! fixture crate at `tests/fixtures/producer`, whose expected API is +//! `tests/fixtures/producer/expected.yml`. -use std::io::Write; +use std::path::{Path, PathBuf}; -fn write_src(contents: &str) -> (tempfile::TempDir, std::path::PathBuf) { - let dir = tempfile::tempdir().expect("failed to create temp dir"); - let src_path = dir.path().join("lib.rs"); - let mut f = std::fs::File::create(&src_path).unwrap(); - f.write_all(contents.as_bytes()).unwrap(); - (dir, src_path) +fn fixture() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/producer") } -#[test] -fn extract_basic_rust_file() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod math { - #[weaveffi::export] - fn add(a: i32, b: i32) -> i32 { - a + b - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract command failed"); - - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - let modules = api["modules"] - .as_sequence() - .expect("should have modules array"); - assert_eq!(modules.len(), 1); - - let module = &modules[0]; - assert_eq!(module["name"].as_str().unwrap(), "math"); - - let functions = module["functions"] - .as_sequence() - .expect("should have functions array"); - assert_eq!(functions.len(), 1); - - let func = &functions[0]; - assert_eq!(func["name"].as_str().unwrap(), "add"); - - let params = func["params"] - .as_sequence() - .expect("should have params array"); - assert_eq!(params.len(), 2); - assert_eq!(params[0]["name"].as_str().unwrap(), "a"); - assert_eq!(params[0]["type"].as_str().unwrap(), "i32"); - assert_eq!(params[1]["name"].as_str().unwrap(), "b"); - assert_eq!(params[1]["type"].as_str().unwrap(), "i32"); - - assert_eq!(func["return"].as_str().unwrap(), "i32"); -} - -#[test] -fn unmarked_module_is_ignored() { - // A module without #[weaveffi::module] exports nothing, even if its items - // carry item markers: the module attribute is the opt-in. - let (_dir, src_path) = write_src( - r#" -mod plain { - #[weaveffi::export] - fn add(a: i32) -> i32 { a } +fn weaveffi() -> assert_cmd::Command { + assert_cmd::Command::cargo_bin("weaveffi").expect("binary not found") } -"#, - ); - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract command failed"); - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); +fn stdout_of(output: &std::process::Output) -> String { assert!( - api["modules"] - .as_sequence() - .map(|m| m.is_empty()) - .unwrap_or(true), - "an unmarked module should produce no exported modules: {stdout}" - ); -} - -#[test] -fn extract_with_struct_and_enum() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod shapes { - #[weaveffi::record] - struct Point { - x: f64, - y: f64, - } - - #[weaveffi::enumeration] - #[repr(i32)] - enum Color { - Red = 0, - Green = 1, - Blue = 2, - } - - #[weaveffi::export] - fn create_point(x: f64, y: f64) -> Point { - todo!() - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract command failed"); - - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - - let module = &api["modules"].as_sequence().unwrap()[0]; - assert_eq!(module["name"].as_str().unwrap(), "shapes"); - - let structs = module["structs"].as_sequence().unwrap(); - assert_eq!(structs.len(), 1); - assert_eq!(structs[0]["name"].as_str().unwrap(), "Point"); - let fields = structs[0]["fields"].as_sequence().unwrap(); - assert_eq!(fields.len(), 2); - assert_eq!(fields[0]["name"].as_str().unwrap(), "x"); - assert_eq!(fields[1]["name"].as_str().unwrap(), "y"); - - let enums = module["enums"].as_sequence().unwrap(); - assert_eq!(enums.len(), 1); - assert_eq!(enums[0]["name"].as_str().unwrap(), "Color"); - let variants = enums[0]["variants"].as_sequence().unwrap(); - assert_eq!(variants.len(), 3); - assert_eq!(variants[0]["name"].as_str().unwrap(), "Red"); - assert_eq!(variants[1]["name"].as_str().unwrap(), "Green"); - assert_eq!(variants[2]["name"].as_str().unwrap(), "Blue"); - - let functions = module["functions"].as_sequence().unwrap(); - assert_eq!(functions.len(), 1); - assert_eq!(functions[0]["name"].as_str().unwrap(), "create_point"); -} - -#[test] -fn extract_with_optional_and_list() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod collections { - #[weaveffi::export] - fn process(items: Vec, label: Option) -> Option> { - todo!() - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract command failed"); - - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - - let func = &api["modules"].as_sequence().unwrap()[0]["functions"] - .as_sequence() - .unwrap()[0]; - assert_eq!(func["name"].as_str().unwrap(), "process"); - - let params = func["params"].as_sequence().unwrap(); - assert_eq!(params.len(), 2); - assert_eq!(params[0]["name"].as_str().unwrap(), "items"); - assert_eq!(params[0]["type"].as_str().unwrap(), "[i32]"); - assert_eq!(params[1]["name"].as_str().unwrap(), "label"); - assert_eq!(params[1]["type"].as_str().unwrap(), "string?"); - - assert_eq!(func["return"].as_str().unwrap(), "[i32]?"); -} - -#[test] -fn extract_to_json_format() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod math { - #[weaveffi::export] - fn add(a: i32, b: i32) -> i32 { - a + b - } -} -"#, + output.status.success(), + "stdout={}\nstderr={}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap(), "--format", "json"]) - .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract --format json failed"); - - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_json::Value = - serde_json::from_str(&stdout).expect("output should be valid JSON"); - - let modules = api["modules"] - .as_array() - .expect("should have modules array"); - assert_eq!(modules.len(), 1); - assert_eq!(modules[0]["name"].as_str().unwrap(), "math"); - - let functions = modules[0]["functions"] - .as_array() - .expect("should have functions array"); - assert_eq!(functions.len(), 1); - assert_eq!(functions[0]["name"].as_str().unwrap(), "add"); + String::from_utf8(output.stdout.clone()).unwrap() } +/// The extracted IDL is exactly the expected one: every declaration kind, +/// declaration order, docs, the type alias resolved, the sibling tree, and +/// only the declarations the default build compiles (`list_price`, not the +/// `#[cfg(feature = "extra")]` function, `impl` block, or module). #[test] -fn extract_async_function() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod async_demo { - /// Fetch data asynchronously. - #[weaveffi::export] - async fn fetch(url: String) -> String { - todo!() - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) +fn extract_reads_the_api_from_the_built_library() { + let output = weaveffi() + .args(["-q", "extract"]) + .arg(fixture()) .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract command failed"); - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - let func = &api["modules"].as_sequence().unwrap()[0]["functions"] - .as_sequence() - .unwrap()[0]; - assert_eq!(func["name"].as_str().unwrap(), "fetch"); - assert_eq!( - func["async"].as_bool(), - Some(true), - "async flag should be set" - ); + .unwrap(); + let expected = std::fs::read_to_string(fixture().join("expected.yml")).unwrap(); + assert_eq!(stdout_of(&output), expected); + assert!(expected.contains("list_price") && !expected.contains("discount")); } +/// Bindings generated from the library are byte-identical to bindings +/// generated from its extracted IDL under the same identity. #[test] -fn extract_fails_loud_on_invalid_api_without_warn() { - // An undeclared object type makes the extracted API fail validation. - // Without `--warn`, `extract` must abort instead of emitting broken IDL. - let (_dir, src_path) = write_src( - r#" -use std::sync::Arc; - -#[weaveffi::module] -mod sessions { - use std::sync::Arc; - - #[weaveffi::export] - fn close(session: Arc) { - todo!() - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!( - !output.status.success(), - "extract should fail loudly on an API that does not validate" - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("--lenient"), - "the error should point users at --lenient: {stderr}" - ); -} - +fn generating_from_the_library_matches_generating_from_its_idl() { + let dir = tempfile::tempdir().unwrap(); + let from_library = dir.path().join("library"); + weaveffi() + .args(["-q", "generate"]) + .arg(fixture()) + .args(["--target", "c", "-o"]) + .arg(&from_library) + .assert() + .success(); + + let idl_dir = dir.path().join("idl"); + std::fs::create_dir_all(&idl_dir).unwrap(); + std::fs::copy(fixture().join("expected.yml"), idl_dir.join("api.yml")).unwrap(); + std::fs::write( + idl_dir.join("weaveffi.toml"), + concat!( + "[package]\n", + "name = \"producer-fixture\"\n", + "version = \"0.3.0\"\n", + "c_prefix = \"producer_fixture\"\n", + "library = \"producer_fixture\"\n", + ), + ) + .unwrap(); + let from_idl = dir.path().join("from-idl"); + weaveffi() + .args(["-q", "generate"]) + .arg(idl_dir.join("api.yml")) + .args(["--target", "c", "-o"]) + .arg(&from_idl) + .assert() + .success(); + + let header = "c/producer_fixture.h"; + let a = std::fs::read_to_string(from_library.join(header)).unwrap(); + let b = std::fs::read_to_string(from_idl.join(header)).unwrap(); + assert!(a.contains("producer_fixture_shop_Cart_total"), "{a}"); + assert_eq!(a, b); +} + +/// `--library` reads a library standing alone (no crate, no IDL), and the +/// output formats round-trip. #[test] -fn extract_callback_interface_definition() { - let (_dir, src_path) = write_src( - r#" -use std::sync::Arc; - -#[weaveffi::module] -mod events { - use std::sync::Arc; - - #[weaveffi::interface] - struct Ticker; - - impl Ticker { - pub fn new() -> Self { Ticker } - } - - /// Consumer-implemented sink for data events. - #[weaveffi::callback_interface] - trait DataListener: Send + Sync { - /// Fired when data arrives. - fn on_data(&self, payload: String, source: Arc); - /// Asks whether to keep delivering. - fn keep_going(&self, n: i32) -> bool; - } - - #[weaveffi::export] - fn subscribe(listener: Arc) {} -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!( - output.status.success(), - "extract command failed: {}", - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - let module = &api["modules"].as_sequence().unwrap()[0]; - - let interfaces = module["callback_interfaces"].as_sequence().unwrap(); - assert_eq!(interfaces.len(), 1); - let listener = &interfaces[0]; - assert_eq!(listener["name"].as_str().unwrap(), "DataListener"); - assert_eq!( - listener["doc"].as_str(), - Some("Consumer-implemented sink for data events.") - ); - - let methods = listener["methods"].as_sequence().unwrap(); - assert_eq!(methods.len(), 2); - assert_eq!(methods[0]["name"].as_str().unwrap(), "on_data"); - assert_eq!(methods[0]["doc"].as_str(), Some("Fired when data arrives.")); - let params = methods[0]["params"].as_sequence().unwrap(); - assert_eq!(params[0]["type"].as_str().unwrap(), "string"); - assert_eq!( - params[1]["type"].as_str().unwrap(), - "Ticker", - "an `Arc` parameter extracts as the bare interface type" - ); - assert_eq!(methods[1]["name"].as_str().unwrap(), "keep_going"); - assert_eq!(methods[1]["return"].as_str().unwrap(), "bool"); - - let subscribe = &module["functions"].as_sequence().unwrap()[0]; - assert_eq!( - subscribe["params"].as_sequence().unwrap()[0]["type"] - .as_str() +fn extract_formats_and_a_library_alone() { + let yaml = stdout_of( + &weaveffi() + .args(["-q", "extract"]) + .arg(fixture()) + .output() .unwrap(), - "DataListener", - "an `Arc` parameter extracts as the callback interface" ); -} - -#[test] -fn extract_interface_with_error_domain() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod store { - #[weaveffi::error] - enum StoreError { - /// resource is missing - Missing = 1, - } - - /// A tiny stateful object. - #[weaveffi::interface] - struct Session { - id: i64, - } - - impl Session { - /// Open a session, failing when the id is invalid. - pub fn open(id: i64) -> Result { - Ok(Session { id }) - } - - /// Fetch a value, failing when it is missing. - pub fn fetch(&self, key: String) -> Result { - Err(StoreError::Missing) - } - - /// The session subsystem version. - pub fn version() -> i32 { - 1 - } - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract command failed"); - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - let module = &api["modules"].as_sequence().unwrap()[0]; - - let errors = &module["errors"]; - assert_eq!(errors["name"].as_str().unwrap(), "StoreError"); - let codes = errors["codes"].as_sequence().unwrap(); - assert_eq!(codes.len(), 1); - assert_eq!(codes[0]["name"].as_str().unwrap(), "Missing"); - assert_eq!(codes[0]["code"].as_i64(), Some(1)); - assert_eq!( - codes[0]["message"].as_str().unwrap(), - "resource is missing", - "the variant's doc comment becomes the default message" + let api: weaveffi_model::ir::Api = serde_yaml::from_str(&yaml).unwrap(); + let json = stdout_of( + &weaveffi() + .args(["-q", "extract", "-f", "json"]) + .arg(fixture()) + .output() + .unwrap(), ); - - let interfaces = module["interfaces"].as_sequence().unwrap(); - assert_eq!(interfaces.len(), 1); - let session = &interfaces[0]; - assert_eq!(session["name"].as_str().unwrap(), "Session"); - assert_eq!(session["doc"].as_str(), Some("A tiny stateful object.")); - - let constructors = session["constructors"].as_sequence().unwrap(); - assert_eq!(constructors.len(), 1); - assert_eq!(constructors[0]["name"].as_str().unwrap(), "open"); assert_eq!( - constructors[0]["throws"].as_bool(), - Some(true), - "a Result constructor should extract as throws: true" + serde_json::from_str::(&json).unwrap(), + api ); - - let methods = session["methods"].as_sequence().unwrap(); - assert_eq!(methods.len(), 1); - assert_eq!(methods[0]["name"].as_str().unwrap(), "fetch"); - assert_eq!(methods[0]["throws"].as_bool(), Some(true)); - assert_eq!( - methods[0]["params"].as_sequence().unwrap()[0]["type"] - .as_str() + let toml_text = stdout_of( + &weaveffi() + .args(["-q", "extract", "-f", "toml"]) + .arg(fixture()) + .output() .unwrap(), - "string" - ); - assert_eq!(methods[0]["return"].as_str().unwrap(), "i64"); - - let statics = session["statics"].as_sequence().unwrap(); - assert_eq!(statics.len(), 1); - assert_eq!(statics[0]["name"].as_str().unwrap(), "version"); - assert_eq!(statics[0]["return"].as_str().unwrap(), "i32"); -} - -#[test] -fn extract_deprecated_attribute_to_note() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod legacy { - /// Legacy add. - #[weaveffi::export] - #[deprecated(since = "0.2.0", note = "Use add_v2 instead")] - fn add_old(a: i32, b: i32) -> i32 { - a + b - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract command failed"); - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - let func = &api["modules"].as_sequence().unwrap()[0]["functions"] - .as_sequence() - .unwrap()[0]; - assert_eq!(func["name"].as_str().unwrap(), "add_old"); - assert_eq!(func["deprecated"].as_str().unwrap(), "Use add_v2 instead"); - assert!( - func.get("since").is_none(), - "schema 0.10.0 has no `since`; the version rides in the deprecation note only" ); -} - -#[test] -fn extract_rejects_mutable_reference_params() { - // Write-back parameters were removed with schema 0.10.0: every parameter - // is borrowed or moved across the boundary, so `&mut T` has no lowering - // and the extractor must refuse it with an actionable message. - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod buffers { - #[weaveffi::export] - fn fill(buf: &mut String, value: i32) { - todo!() - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!( - !output.status.success(), - "extract should reject `&mut` parameters" - ); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains("`&mut` parameters cannot cross the FFI boundary"), - "unexpected diagnostic: {stderr}" + assert_eq!( + toml::from_str::(&toml_text).unwrap(), + api ); } +/// What isn't a Rust producer's library is refused with a reason: an IDL, +/// a binary without WeaveFFI metadata, and Rust source. #[test] -fn extract_rejects_raw_pointers_and_box() { - for (ty, needle) in [ - ("*mut Widget", "raw pointers cannot cross the FFI boundary"), - ("Box", "`Box` cannot cross the FFI boundary"), - ] { - let (_dir, src_path) = write_src(&format!( - r#" -#[weaveffi::module] -mod objects {{ - #[weaveffi::interface] - struct Widget; - - impl Widget {{ - pub fn new() -> Self {{ Widget }} - }} - - #[weaveffi::export] - fn take(w: {ty}) {{}} -}} -"# - )); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) - .output() - .expect("failed to run extract"); - - assert!(!output.status.success(), "extract should reject `{ty}`"); - let stderr = String::from_utf8_lossy(&output.stderr); - assert!( - stderr.contains(needle), - "unexpected diagnostic for `{ty}`: {stderr}" - ); - } -} - -#[test] -fn extract_arc_object_shapes() { - // `Arc` in every position lowers to the bare interface type, with - // `Option`, `Vec`, and record fields composing on top. - let (_dir, src_path) = write_src( - r#" -use std::sync::Arc; - -#[weaveffi::module] -mod graph { - use std::sync::Arc; - - #[weaveffi::interface] - struct Node; - - #[weaveffi::record] - struct Edge { - from: Arc, - to: Option>, - } - - impl Node { - pub fn new() -> Self { Node } - pub fn me(self: Arc) -> Arc { self } - pub fn parent(&self) -> Option> { None } - pub fn children(&self) -> Vec> { vec![] } - pub fn edges(&self) -> weaveffi::Iter> { weaveffi::Iter::new(vec![]) } - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap()]) +fn extract_explains_what_it_cannot_read() { + let idl = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/kitchen_sink.yml"); + let output = weaveffi().arg("extract").arg(&idl).output().unwrap(); + assert!(!output.status.success()); + let err = String::from_utf8_lossy(&output.stderr); + assert!(err.contains("already an IDL"), "{err}"); + + // The CLI binary itself is a real executable with no metadata. + let bin = assert_cmd::cargo::cargo_bin("weaveffi"); + let output = weaveffi() + .args(["extract", "--library"]) + .arg(&bin) .output() - .expect("failed to run extract"); - - assert!( - output.status.success(), - "extract command failed: {}", - String::from_utf8_lossy(&output.stderr) - ); - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: serde_yaml::Value = - serde_yaml::from_str(&stdout).expect("output should be valid YAML"); - let module = &api["modules"].as_sequence().unwrap()[0]; - - let edge = &module["structs"].as_sequence().unwrap()[0]; - let fields = edge["fields"].as_sequence().unwrap(); - assert_eq!(fields[0]["type"].as_str().unwrap(), "Node"); - assert_eq!(fields[1]["type"].as_str().unwrap(), "Node?"); - - let node = &module["interfaces"].as_sequence().unwrap()[0]; - let methods = node["methods"].as_sequence().unwrap(); - let by_name = |n: &str| { - methods - .iter() - .find(|m| m["name"].as_str() == Some(n)) - .unwrap_or_else(|| panic!("missing method {n}")) - }; - assert_eq!(by_name("me")["return"].as_str().unwrap(), "Node"); - assert_eq!(by_name("parent")["return"].as_str().unwrap(), "Node?"); - assert_eq!(by_name("children")["return"].as_str().unwrap(), "[Node]"); - assert_eq!(by_name("edges")["return"].as_str().unwrap(), "iter"); -} - -#[test] -fn extract_to_toml_format() { - let (_dir, src_path) = write_src( - r#" -#[weaveffi::module] -mod math { - #[weaveffi::export] - fn add(a: i32, b: i32) -> i32 { - a + b - } -} -"#, - ); - - let output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args(["extract", src_path.to_str().unwrap(), "--format", "toml"]) + .unwrap(); + assert!(!output.status.success()); + let err = String::from_utf8_lossy(&output.stderr); + assert!(err.contains("no WeaveFFI metadata"), "{err}"); + assert!(err.contains("export_runtime!"), "{err}"); + + let output = weaveffi() + .arg("extract") + .arg(fixture().join("src/lib.rs")) .output() - .expect("failed to run extract"); - - assert!(output.status.success(), "extract --format toml failed"); - - let stdout = String::from_utf8(output.stdout).unwrap(); - let api: toml::Value = stdout.parse().expect("output should be valid TOML"); - - let modules = api["modules"] - .as_array() - .expect("should have modules array"); - assert_eq!(modules.len(), 1); - assert_eq!(modules[0]["name"].as_str().unwrap(), "math"); - - let functions = modules[0]["functions"] - .as_array() - .expect("should have functions array"); - assert_eq!(functions.len(), 1); - assert_eq!(functions[0]["name"].as_str().unwrap(), "add"); + .unwrap(); + assert!(!output.status.success()); + let err = String::from_utf8_lossy(&output.stderr); + assert!(err.contains("pass the crate"), "{err}"); } diff --git a/crates/weaveffi-cli/tests/cli/extract_roundtrip.rs b/crates/weaveffi-cli/tests/cli/extract_roundtrip.rs deleted file mode 100644 index c952c6e3..00000000 --- a/crates/weaveffi-cli/tests/cli/extract_roundtrip.rs +++ /dev/null @@ -1,322 +0,0 @@ -//! Round-trip tests for `weaveffi extract`. -//! -//! `roundtrip_kitchen_sink` proves the extractor recovers the same shape -//! as the original kitchen-sink IDL when run on the hand-annotated Rust -//! file at `crates/weaveffi-cli/tests/fixtures/kitchen_sink_annotated.rs`, -//! including the full surface: the `Gadget` interface, the `KitchenErrors` -//! domain, the `ReadyListener` callback interface, and per-function -//! `throws`. Lossy fields (struct field defaults, callback param docs, -//! error-code `doc:` separate from `message:`) are documented in -//! `docs/src/guides/extract.md` and skipped by name in the assertions below. - -use std::collections::BTreeMap; - -use weaveffi_model::ir::{Api, Function, InterfaceDef, Module, StructDef, TypeRef}; -use weaveffi_model::parse::parse_api_str; - -/// Compare user-type references by their final path segment so a -/// module-qualified spelling (`shared.Token`) matches the bare one the -/// extractor emits for an in-scope Rust type. -fn normalize(ty: &TypeRef) -> TypeRef { - fn last_segment(name: &str) -> String { - name.rsplit('.').next().unwrap_or(name).to_string() - } - match ty { - TypeRef::Named(name) => TypeRef::Named(last_segment(name)), - TypeRef::Optional(inner) => TypeRef::Optional(Box::new(normalize(inner))), - TypeRef::List(inner) => TypeRef::List(Box::new(normalize(inner))), - TypeRef::Iterator(inner) => TypeRef::Iterator(Box::new(normalize(inner))), - TypeRef::Map(k, v) => TypeRef::Map(Box::new(normalize(k)), Box::new(normalize(v))), - other => other.clone(), - } -} - -fn assert_types_equivalent(a: &TypeRef, b: &TypeRef, ctx: &str) { - assert_eq!(normalize(a), normalize(b), "{ctx} type mismatch"); -} - -fn module_by_name<'a>(api: &'a Api, name: &str) -> &'a Module { - api.modules - .iter() - .find(|m| m.name == name) - .unwrap_or_else(|| panic!("module {name} missing; modules: {:?}", module_names(api))) -} - -fn module_names(api: &Api) -> Vec<&str> { - api.modules.iter().map(|m| m.name.as_str()).collect() -} - -fn struct_by_name<'a>(module: &'a Module, name: &str) -> &'a StructDef { - module - .structs - .iter() - .find(|s| s.name == name) - .unwrap_or_else(|| panic!("struct {name} missing in module {}", module.name)) -} - -fn function_by_name<'a>(module: &'a Module, name: &str) -> &'a Function { - module - .functions - .iter() - .find(|f| f.name == name) - .unwrap_or_else(|| panic!("function {name} missing in module {}", module.name)) -} - -fn interface_by_name<'a>(module: &'a Module, name: &str) -> &'a InterfaceDef { - module - .interfaces - .iter() - .find(|i| i.name == name) - .unwrap_or_else(|| panic!("interface {name} missing in module {}", module.name)) -} - -/// Assert one extracted callable matches its original: name, params (name + -/// type), return, throws, async, cancellable, and doc. -fn assert_functions_equivalent(extracted: &Function, orig: &Function, ctx: &str) { - assert_eq!(extracted.name, orig.name, "{ctx} name mismatch"); - assert_eq!( - extracted.params.len(), - orig.params.len(), - "{ctx} param count mismatch" - ); - for (a, b) in extracted.params.iter().zip(orig.params.iter()) { - assert_eq!(a.name, b.name, "{ctx} param name mismatch"); - assert_types_equivalent(&a.ty, &b.ty, &format!("{ctx} param {}", b.name)); - } - match (&extracted.returns, &orig.returns) { - (Some(a), Some(b)) => assert_types_equivalent(a, b, &format!("{ctx} return")), - (None, None) => {} - (a, b) => panic!("{ctx} return mismatch: {a:?} vs {b:?}"), - } - assert_eq!(extracted.throws, orig.throws, "{ctx} throws mismatch"); - assert_eq!(extracted.r#async, orig.r#async, "{ctx} async mismatch"); - assert_eq!( - extracted.cancellable, orig.cancellable, - "{ctx} cancellable mismatch" - ); - assert_eq!(extracted.doc, orig.doc, "{ctx} doc mismatch"); -} - -#[test] -fn roundtrip_kitchen_sink() { - let manifest = env!("CARGO_MANIFEST_DIR"); - let idl_path = format!("{manifest}/tests/fixtures/kitchen_sink.yml"); - let annotated_path = format!("{manifest}/tests/fixtures/kitchen_sink_annotated.rs"); - - let original_src = std::fs::read_to_string(&idl_path).expect("read kitchen-sink IDL"); - let original = parse_api_str(&original_src, "yaml").expect("parse kitchen-sink IDL"); - - let extract_output = assert_cmd::Command::cargo_bin("weaveffi") - .expect("weaveffi binary not found") - .args(["extract", &annotated_path, "-f", "yaml"]) - .output() - .expect("failed to run weaveffi extract"); - assert!( - extract_output.status.success(), - "extract failed: stdout={}, stderr={}", - String::from_utf8_lossy(&extract_output.stdout), - String::from_utf8_lossy(&extract_output.stderr), - ); - let extracted_yaml = String::from_utf8(extract_output.stdout).expect("extract stdout utf-8"); - let extracted = parse_api_str(&extracted_yaml, "yaml").expect("parse extracted YAML"); - - let original_modules: BTreeMap<&str, &Module> = original - .modules - .iter() - .map(|m| (m.name.as_str(), m)) - .collect(); - let extracted_modules: BTreeMap<&str, &Module> = extracted - .modules - .iter() - .map(|m| (m.name.as_str(), m)) - .collect(); - assert_eq!( - original_modules.keys().collect::>(), - extracted_modules.keys().collect::>(), - "top-level modules differ" - ); - - // shared module - let shared_orig = module_by_name(&original, "shared"); - let shared_ex = module_by_name(&extracted, "shared"); - let token_orig = struct_by_name(shared_orig, "Token"); - let token_ex = struct_by_name(shared_ex, "Token"); - assert_eq!(token_ex.name, token_orig.name); - assert_eq!(token_ex.fields.len(), token_orig.fields.len()); - for (a, b) in token_ex.fields.iter().zip(token_orig.fields.iter()) { - assert_eq!(a.name, b.name); - assert_types_equivalent(&a.ty, &b.ty, &format!("Token.{}", b.name)); - } - let ping_orig = function_by_name(shared_orig, "ping"); - let ping_ex = function_by_name(shared_ex, "ping"); - assert_types_equivalent( - ping_ex.returns.as_ref().unwrap(), - ping_orig.returns.as_ref().unwrap(), - "ping return", - ); - assert_eq!(ping_ex.doc, ping_orig.doc); - - // kitchen module - let kitchen_orig = module_by_name(&original, "kitchen"); - let kitchen_ex = module_by_name(&extracted, "kitchen"); - - // Enums (with variant docs) - assert_eq!(kitchen_ex.enums.len(), kitchen_orig.enums.len()); - let prio_orig = &kitchen_orig.enums[0]; - let prio_ex = &kitchen_ex.enums[0]; - assert_eq!(prio_ex.name, prio_orig.name); - assert_eq!(prio_ex.doc, prio_orig.doc); - assert_eq!(prio_ex.variants.len(), prio_orig.variants.len()); - for (a, b) in prio_ex.variants.iter().zip(prio_orig.variants.iter()) { - assert_eq!(a.name, b.name); - assert_eq!(a.value, b.value); - assert_eq!(a.doc, b.doc, "variant {} doc mismatch", b.name); - } - - // Record struct - let item_orig = struct_by_name(kitchen_orig, "Item"); - let item_ex = struct_by_name(kitchen_ex, "Item"); - assert_eq!(item_ex.doc, item_orig.doc); - assert_eq!(item_ex.fields.len(), item_orig.fields.len()); - for (a, b) in item_ex.fields.iter().zip(item_orig.fields.iter()) { - assert_eq!(a.name, b.name); - assert_types_equivalent(&a.ty, &b.ty, &format!("Item.{}", b.name)); - assert_eq!(a.doc, b.doc, "Item.{} doc mismatch", b.name); - // a.default may legitimately differ; the fixture cannot recover - // struct field defaults from Rust syntax (documented gap). - } - - // Callback interfaces - assert_eq!( - kitchen_ex.callback_interfaces.len(), - kitchen_orig.callback_interfaces.len() - ); - let listener_orig = &kitchen_orig.callback_interfaces[0]; - let listener_ex = &kitchen_ex.callback_interfaces[0]; - assert_eq!(listener_ex.name, listener_orig.name); - assert_eq!(listener_ex.doc, listener_orig.doc); - assert_eq!(listener_ex.methods.len(), listener_orig.methods.len()); - for (a, b) in listener_ex.methods.iter().zip(listener_orig.methods.iter()) { - assert_functions_equivalent(a, b, &format!("ReadyListener method {}", b.name)); - } - - // Functions: every original IDL function must reappear in extracted - // output with matching shape. - for orig in &kitchen_orig.functions { - let extracted = function_by_name(kitchen_ex, &orig.name); - assert_eq!( - extracted.params.len(), - orig.params.len(), - "{} param count mismatch", - orig.name - ); - for (a, b) in extracted.params.iter().zip(orig.params.iter()) { - assert_eq!(a.name, b.name, "{} param name mismatch", orig.name); - assert_types_equivalent(&a.ty, &b.ty, &format!("{} param {}", orig.name, b.name)); - // Param.doc is allowed to differ (documented gap). - } - match (&extracted.returns, &orig.returns) { - (Some(a), Some(b)) => assert_types_equivalent(a, b, &format!("{} return", orig.name)), - (None, None) => {} - (a, b) => panic!("{} return mismatch: {:?} vs {:?}", orig.name, a, b), - } - assert_eq!(extracted.doc, orig.doc, "{} doc mismatch", orig.name); - assert_eq!( - extracted.r#async, orig.r#async, - "{} async mismatch", - orig.name - ); - assert_eq!( - extracted.cancellable, orig.cancellable, - "{} cancellable mismatch", - orig.name - ); - assert_eq!( - extracted.throws, orig.throws, - "{} throws mismatch", - orig.name - ); - assert_eq!( - extracted.deprecated, orig.deprecated, - "{} deprecated mismatch", - orig.name - ); - } - - // Error domain: `#[weaveffi::error]` makes the domain derivable, so name, - // code names, values, and messages must all round-trip. A code's `doc:` is - // skipped: Rust spells the message as the first doc line, so the extracted - // doc always includes the message and cannot match an IDL doc that omits it. - let errors_orig = kitchen_orig - .errors - .as_ref() - .expect("KitchenErrors in original"); - let errors_ex = kitchen_ex - .errors - .as_ref() - .expect("KitchenErrors in extracted output"); - assert_eq!(errors_ex.name, errors_orig.name); - assert_eq!(errors_ex.codes.len(), errors_orig.codes.len()); - for (a, b) in errors_ex.codes.iter().zip(errors_orig.codes.iter()) { - assert_eq!(a.name, b.name, "error code name mismatch"); - assert_eq!(a.code, b.code, "error code {} value mismatch", b.name); - assert_eq!( - a.message, b.message, - "error code {} message mismatch", - b.name - ); - } - - // The `maybe_item` throws flag rides through the loop above; assert it - // explicitly too so a regression names the feature, not just the function. - assert!( - function_by_name(kitchen_ex, "maybe_item").throws, - "maybe_item should round-trip as throws: true" - ); - - // Interface: constructors, methods, and statics with their shapes. - let gadget_orig = interface_by_name(kitchen_orig, "Gadget"); - let gadget_ex = interface_by_name(kitchen_ex, "Gadget"); - assert_eq!(gadget_ex.doc, gadget_orig.doc, "Gadget doc mismatch"); - for (kind, ex, orig) in [ - ( - "constructor", - &gadget_ex.constructors, - &gadget_orig.constructors, - ), - ("method", &gadget_ex.methods, &gadget_orig.methods), - ("static", &gadget_ex.statics, &gadget_orig.statics), - ] { - assert_eq!( - ex.len(), - orig.len(), - "Gadget {kind} count mismatch: {:?} vs {:?}", - ex.iter().map(|f| &f.name).collect::>(), - orig.iter().map(|f| &f.name).collect::>(), - ); - for (a, b) in ex.iter().zip(orig.iter()) { - assert_functions_equivalent(a, b, &format!("Gadget {kind} {}", b.name)); - } - } - - // Nested module - let nested_orig = kitchen_orig - .modules - .iter() - .find(|m| m.name == "nested") - .expect("nested submodule in original"); - let nested_ex = kitchen_ex - .modules - .iter() - .find(|m| m.name == "nested") - .expect("nested submodule in extracted output"); - assert_eq!(nested_ex.functions.len(), nested_orig.functions.len()); - let hello_orig = function_by_name(nested_orig, "hello"); - let hello_ex = function_by_name(nested_ex, "hello"); - assert_types_equivalent( - hello_ex.returns.as_ref().unwrap(), - hello_orig.returns.as_ref().unwrap(), - "hello return", - ); - assert_eq!(hello_ex.doc, hello_orig.doc); -} diff --git a/crates/weaveffi-cli/tests/cli/formats.rs b/crates/weaveffi-cli/tests/cli/formats.rs index 744b3636..0f5296c9 100644 --- a/crates/weaveffi-cli/tests/cli/formats.rs +++ b/crates/weaveffi-cli/tests/cli/formats.rs @@ -1,16 +1,25 @@ -use std::path::Path; +//! The JSON and TOML spellings of an IDL generate and validate like YAML. use weaveffi_model::ir::Api; -fn repo_root() -> &'static Path { - let manifest_dir = env!("CARGO_MANIFEST_DIR"); - Path::new(manifest_dir).parent().unwrap().parent().unwrap() -} - +/// The calculator sample's API, read from YAML. fn load_calculator_api() -> Api { - let src = std::fs::read_to_string(repo_root().join("samples/calculator/src/lib.rs")) - .expect("failed to read calculator lib.rs"); - weaveffi_model::rust::api_from_src_stringly(&src).expect("failed to extract calculator API") + weaveffi_model::parse::parse_api_str( + concat!( + "version: \"0.11.0\"\n", + "modules:\n", + " - name: calculator\n", + " errors:\n", + " name: CalcError\n", + " codes: [{ name: DivisionByZero, code: 1, message: division by zero }]\n", + " functions:\n", + " - { name: add, params: [{ name: a, type: i32 }, { name: b, type: i32 }], return: i32 }\n", + " - { name: divide, params: [{ name: a, type: i32 }, { name: b, type: i32 }], return: i32, throws: true }\n", + " - { name: greet, params: [{ name: name, type: string }], return: string }\n", + ), + "yaml", + ) + .expect("failed to parse the calculator API") } #[test] diff --git a/crates/weaveffi-cli/tests/cli/generate.rs b/crates/weaveffi-cli/tests/cli/generate.rs index 57d3bd2a..60dfc6a7 100644 --- a/crates/weaveffi-cli/tests/cli/generate.rs +++ b/crates/weaveffi-cli/tests/cli/generate.rs @@ -5,7 +5,7 @@ use std::path::Path; fn generate_produces_expected_files() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/calculator/src/lib.rs"); + let input = repo_root.join("samples/calculator"); let out_dir = tempfile::tempdir().expect("failed to create temp dir"); let out_path = out_dir.path(); @@ -43,7 +43,7 @@ fn generate_produces_expected_files() { fn generate_with_target_filter() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/calculator/src/lib.rs"); + let input = repo_root.join("samples/calculator"); let out_dir = tempfile::tempdir().expect("failed to create temp dir"); let out_path = out_dir.path(); @@ -75,7 +75,7 @@ fn generate_with_target_filter() { fn generate_cpp_target_filter() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/calculator/src/lib.rs"); + let input = repo_root.join("samples/calculator"); let out_dir = tempfile::tempdir().expect("failed to create temp dir"); let out_path = out_dir.path(); @@ -107,7 +107,7 @@ fn generate_cpp_target_filter() { fn validate_command_succeeds() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/calculator/src/lib.rs"); + let input = repo_root.join("samples/calculator"); assert_cmd::Command::cargo_bin("weaveffi") .expect("binary not found") @@ -121,7 +121,7 @@ fn validate_command_succeeds() { fn quiet_flag_suppresses_output() { let manifest_dir = env!("CARGO_MANIFEST_DIR"); let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/calculator/src/lib.rs"); + let input = repo_root.join("samples/calculator"); let out_dir = tempfile::tempdir().expect("failed to create temp dir"); let out_path = out_dir.path(); @@ -144,3 +144,45 @@ fn quiet_flag_suppresses_output() { "files should still be generated with --quiet" ); } + +/// `weaveffi dev` builds the debug library, generates, and copies the +/// library into the Python package, which loads a bundled copy first; the +/// targets that can't bundle it are told which variable to set. +#[test] +fn dev_bundles_the_debug_library_into_the_python_package() { + let manifest_dir = env!("CARGO_MANIFEST_DIR"); + let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); + let input = repo_root.join("samples/calculator"); + let out_dir = tempfile::tempdir().expect("failed to create temp dir"); + let out_path = out_dir.path(); + + let output = assert_cmd::Command::cargo_bin("weaveffi") + .expect("binary not found") + .args(["dev", input.to_str().unwrap(), "-o"]) + .arg(out_path) + .args(["--target", "c,python,dotnet"]) + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&output.stdout); + assert!( + output.status.success(), + "{stdout}{}", + String::from_utf8_lossy(&output.stderr) + ); + let library = if cfg!(target_os = "macos") { + "libcalculator.dylib" + } else if cfg!(windows) { + "calculator.dll" + } else { + "libcalculator.so" + }; + assert!( + out_path.join("python/calculator").join(library).is_file(), + "{stdout}" + ); + assert!( + stdout.contains("dotnet: export CALCULATOR_LIBRARY="), + "{stdout}" + ); + assert!(stdout.contains("c: link with"), "{stdout}"); +} diff --git a/crates/weaveffi-cli/tests/cli/main.rs b/crates/weaveffi-cli/tests/cli/main.rs index 13057578..9df62011 100644 --- a/crates/weaveffi-cli/tests/cli/main.rs +++ b/crates/weaveffi-cli/tests/cli/main.rs @@ -8,7 +8,6 @@ mod diagnostics; mod diff; mod errors; mod extract; -mod extract_roundtrip; mod formats; mod generate; mod no_silent_stubs; diff --git a/crates/weaveffi-cli/tests/cli/no_silent_stubs.rs b/crates/weaveffi-cli/tests/cli/no_silent_stubs.rs index 1a04eca2..8479ab3e 100644 --- a/crates/weaveffi-cli/tests/cli/no_silent_stubs.rs +++ b/crates/weaveffi-cli/tests/cli/no_silent_stubs.rs @@ -5,12 +5,12 @@ //! paths for whole call shapes, features skipped without a word. These tests //! make that class of regression a build failure: //! -//! 1. Generator and model crate sources must not contain `unimplemented!(` / `todo!(`. +//! 1. Generator and model sources must not contain `unimplemented!(` / `todo!(`. //! (`unreachable!` stays allowed: it documents genuinely impossible states.) //! 2. The full generated output for the feature-complete sample IDLs must not -//! contain stub markers. A target that cannot support a feature must either -//! fail generation loudly (the capability gate) or emit an *explicit* -//! "not supported by this target" surface, never a fake implementation. +//! contain stub markers. A target that cannot support a feature in some +//! mode must emit an *explicit* "not supported by this target" surface, +//! never a fake implementation. use std::fs; use std::path::{Path, PathBuf}; @@ -33,7 +33,7 @@ fn walk_files(dir: &Path, out: &mut Vec) { } } -/// Generator crate sources must not punt with panicking placeholder macros. +/// Generator and model sources must not punt with panicking placeholder macros. #[test] fn generator_sources_ban_unimplemented_and_todo() { let crates_dir = workspace_root().join("crates"); @@ -43,10 +43,10 @@ fn generator_sources_ban_unimplemented_and_todo() { for entry in fs::read_dir(&crates_dir).expect("read crates/") { let path = entry.expect("dir entry").path(); let name = path.file_name().unwrap().to_string_lossy().to_string(); - // Only the generator and model crates (the code that produces - // consumer bindings) are in scope; test helpers and the CLI are free - // to use placeholder macros. - if !(name == "weaveffi-gen" || name == "weaveffi-model") { + // Only the CLI (which holds the generators) and the model crate (the + // code that produces consumer bindings) are in scope; test helpers + // are free to use placeholder macros. + if !(name == "weaveffi-cli" || name == "weaveffi-model") { continue; } let src = path.join("src"); @@ -77,9 +77,8 @@ fn generator_sources_ban_unimplemented_and_todo() { ); assert!( violations.is_empty(), - "panicking placeholder macros in generator crates (implement the path, \ - fail generation via the capability gate, or use unreachable! for \ - impossible states):\n{}", + "panicking placeholder macros in generator sources (implement the path \ + or use unreachable! for impossible states):\n{}", violations.join("\n") ); } @@ -87,10 +86,13 @@ fn generator_sources_ban_unimplemented_and_todo() { /// The Ruby callback-interface mixin's abstract-method default, which is the /// one legitimate "not implemented" in generated output (see the exemption in /// [`generated_output_has_no_stub_markers`]). -fn is_abstract_callback_default(line: &str) -> bool { +fn is_exempt_idiom(line: &str) -> bool { let t = line.trim_start(); t.starts_with("raise NotImplementedError, \"#{self.class}#") || (t.starts_with('#') && t.contains("NotImplementedError")) + // Python's rich-comparison protocol: `__eq__` returns the + // `NotImplemented` singleton for foreign types. + || t == "return NotImplemented" } /// Every file generated for the feature-complete samples (structs, builders, @@ -100,22 +102,16 @@ fn is_abstract_callback_default(line: &str) -> bool { #[test] fn generated_output_has_no_stub_markers() { let root = workspace_root(); - let samples = [ - root.join("samples/contacts/src/lib.rs"), - root.join("samples/events/src/lib.rs"), - root.join("samples/kvstore/src/lib.rs"), - root.join("samples/codec/src/lib.rs"), - ]; + let samples = [root.join("samples/kvstore"), root.join("samples/codec")]; // Case-insensitive marker list. Bare "not supported" is deliberately // absent: an explicit, permanently declared unsupported-feature surface // (e.g. wasm async/listener stubs in Emscripten mode, which throw "is // not supported in Emscripten mode") is the *correct* loud behavior. The - // "yet" in "not yet supported" is what distinguishes capability drift: a - // target that declares a feature `true` (so the gate lets generation - // through) but then emits a runtime-throwing TODO stub for it. That is - // exactly the Kotlin `iter` regression the capability gate exists to - // prevent, so it must stay a hard build failure. + // "yet" in "not yet supported" is what marks a runtime-throwing TODO stub + // for a feature the target claims to implement, the Kotlin `iter` + // regression this test exists to prevent, so it must stay a hard build + // failure. let banned = [ "unimplemented", "notimplemented", @@ -138,7 +134,6 @@ fn generated_output_has_no_stub_markers() { idl.to_str().unwrap(), "-o", out.to_str().unwrap(), - "--force", ]) .assert() .success(); @@ -163,7 +158,7 @@ fn generated_output_has_no_stub_markers() { // stub, so those lines are exempt from the scan. let lower: String = text .lines() - .filter(|l| !is_abstract_callback_default(l)) + .filter(|l| !is_exempt_idiom(l)) .map(str::to_lowercase) .collect::>() .join("\n"); diff --git a/crates/weaveffi-cli/tests/cli/package.rs b/crates/weaveffi-cli/tests/cli/package.rs index 39e491bb..f0162b15 100644 --- a/crates/weaveffi-cli/tests/cli/package.rs +++ b/crates/weaveffi-cli/tests/cli/package.rs @@ -1,7 +1,22 @@ -use std::path::Path; +//! `weaveffi build` and `weaveffi package` end to end: artifacts written +//! from a `--binaries` tree, and the error messages for the setups that +//! can't build or package. -/// Lay out a fake `--binaries` tree (`//`) covering the full -/// v1 matrix, with placeholder library bytes so packaging has something to copy. +use std::io::Read; +use std::path::{Path, PathBuf}; + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .unwrap() + .parent() + .unwrap() + .to_path_buf() +} + +/// Lay out a fake `--binaries` tree (`//`) covering the +/// desktop matrix, with placeholder library bytes so packaging has something +/// to copy. fn write_prebuilt(root: &Path, lib_base: &str) { let entries = [ ("darwin-arm64", format!("lib{lib_base}.dylib")), @@ -15,118 +30,248 @@ fn write_prebuilt(root: &Path, lib_base: &str) { std::fs::create_dir_all(&dir).expect("create platform dir"); std::fs::write(dir.join(&lib), b"\x00fake-native\x01").expect("write fake lib"); } + std::fs::write( + root.join("darwin-arm64") + .join(format!("{lib_base}_node.node")), + b"addon", + ) + .unwrap(); } -#[test] -fn package_bundles_native_libraries_per_ecosystem() { - let manifest_dir = env!("CARGO_MANIFEST_DIR"); - let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/contacts/src/lib.rs"); +/// An IDL project for the calculator API (`calculator` 1.0.0) in `dir`, +/// returning its IDL: what a producer packaged from `--binaries` declares. +fn calculator_idl(dir: &Path) -> PathBuf { + let idl = dir.join("calculator.yml"); + std::fs::write( + &idl, + concat!( + "version: \"0.11.0\"\n", + "modules:\n", + " - name: calculator\n", + " functions:\n", + " - { name: add, params: [{ name: a, type: i32 }, { name: b, type: i32 }], return: i32 }\n", + ), + ) + .unwrap(); + std::fs::write( + dir.join("weaveffi.toml"), + "[package]\nname = \"calculator\"\nversion = \"1.0.0\"\nlicense = \"MIT\"\n", + ) + .unwrap(); + idl +} - let bins = tempfile::tempdir().expect("temp bins dir"); - write_prebuilt(bins.path(), "contacts"); +fn weaveffi() -> assert_cmd::Command { + assert_cmd::Command::cargo_bin("weaveffi").expect("binary not found") +} - let out_dir = tempfile::tempdir().expect("temp out dir"); - let out_path = out_dir.path(); +/// The command's stderr with miette's wrapping undone: box-drawing gutters +/// removed and whitespace runs collapsed, so messages match as one line. +fn stderr(output: &std::process::Output) -> String { + String::from_utf8_lossy(&output.stderr) + .replace(['│', '×'], " ") + .split_whitespace() + .collect::>() + .join(" ") +} - assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args([ - "package", - input.to_str().unwrap(), - "--binaries", - bins.path().to_str().unwrap(), - "--target", - "dotnet,python,node", - "-o", - out_path.to_str().unwrap(), - ]) +/// The `(path, bytes)` entries of a gzipped tarball. +fn untar_gz(path: &Path) -> Vec<(String, Vec)> { + let file = std::fs::File::open(path).unwrap_or_else(|e| panic!("{}: {e}", path.display())); + let mut archive = tar::Archive::new(flate2::read::GzDecoder::new(file)); + archive + .entries() + .unwrap() + .map(|e| { + let mut e = e.unwrap(); + let name = e.path().unwrap().to_string_lossy().into_owned(); + let mut data = Vec::new(); + e.read_to_end(&mut data).unwrap(); + (name, data) + }) + .collect() +} + +#[test] +fn package_writes_installable_artifacts_from_binaries() { + let project = tempfile::tempdir().expect("temp project dir"); + let input = calculator_idl(project.path()); + let bins = tempfile::tempdir().expect("temp bins dir"); + write_prebuilt(bins.path(), "calculator"); + let out = tempfile::tempdir().expect("temp out dir"); + let dist = out.path(); + + weaveffi() + .args(["package", input.to_str().unwrap(), "--binaries"]) + .arg(bins.path()) + .args(["--target", "python,node,ruby,c,go", "-o"]) + .arg(dist) .assert() .success(); - // .NET: NuGet runtimes//native/ layout. - assert!( - out_path - .join("dotnet/runtimes/osx-arm64/native/libcontacts.dylib") - .exists(), - "missing macOS arm64 runtime library" + // Python: one platform-tagged wheel per desktop platform. + for wheel in [ + "calculator-1.0.0-py3-none-macosx_11_0_arm64.whl", + "calculator-1.0.0-py3-none-macosx_11_0_x86_64.whl", + "calculator-1.0.0-py3-none-manylinux_2_17_x86_64.whl", + "calculator-1.0.0-py3-none-manylinux_2_17_aarch64.whl", + "calculator-1.0.0-py3-none-win_amd64.whl", + ] { + assert!(dist.join("python").join(wheel).is_file(), "missing {wheel}"); + } + + // Node: per-platform tarballs with the library (and the prebuilt addon + // where one was built), plus the main package. + let mac = untar_gz(&dist.join("node/calculator-darwin-arm64-1.0.0.tgz")); + let names: Vec<&str> = mac.iter().map(|(n, _)| n.as_str()).collect(); + assert_eq!( + names, + [ + "package/package.json", + "package/libcalculator.dylib", + "package/calculator_node.node" + ] ); + let linux = untar_gz(&dist.join("node/calculator-linux-x64-1.0.0.tgz")); + assert_eq!(linux.len(), 2, "no addon was prebuilt for linux-x64"); + let main = untar_gz(&dist.join("node/calculator-1.0.0.tgz")); + let package_json = main + .iter() + .find(|(n, _)| n == "package/package.json") + .map(|(_, d)| String::from_utf8_lossy(d).into_owned()) + .expect("main package.json"); assert!( - out_path - .join("dotnet/runtimes/win-x64/native/contacts.dll") - .exists(), - "missing Windows runtime library" + package_json.contains("\"calculator-darwin-arm64\": \"1.0.0\""), + "{package_json}" ); - // Python: a per-platform wheel tree with the library bundled in-package. + // Ruby: one precompiled gem per desktop platform. + for gem in [ + "calculator-1.0.0-arm64-darwin.gem", + "calculator-1.0.0-x86_64-linux.gem", + "calculator-1.0.0-x64-mingw-ucrt.gem", + ] { + assert!(dist.join("ruby").join(gem).is_file(), "missing {gem}"); + } + + // C: headers and every platform's library in one tarball. + let c = untar_gz(&dist.join("c/calculator-1.0.0-c.tar.gz")); + let names: Vec<&str> = c.iter().map(|(n, _)| n.as_str()).collect(); assert!( - out_path - .join("python/linux-x64/contacts/libcontacts.so") - .exists(), - "missing bundled Linux library in the Python package" + names.contains(&"calculator-1.0.0/include/calculator.h"), + "{names:?}" ); - - // Node: optionalDependencies plus per-platform packages bundling the lib. - let node_pkg = std::fs::read_to_string(out_path.join("node/package.json")) - .expect("missing node/package.json"); assert!( - node_pkg.contains("\"optionalDependencies\""), - "node package.json should declare optionalDependencies" + names.contains(&"calculator-1.0.0/lib/windows-x64/calculator.dll"), + "{names:?}" ); + + // Go: the module directory with per-platform libraries and cgo flags. + let bindings = std::fs::read_to_string(dist.join("go/calculator/bindings.go")).unwrap(); assert!( - out_path - .join("node/npm/contacts-darwin-arm64/libcontacts.dylib") - .exists(), - "missing per-platform npm package library" + bindings.contains("#cgo linux,amd64 LDFLAGS: -L${SRCDIR}/lib/linux-x64"), + "{bindings}" ); + assert!(dist + .join("go/calculator/lib/darwin-arm64/libcalculator.dylib") + .is_file()); } #[test] -fn package_skips_unsupported_targets() { - let manifest_dir = env!("CARGO_MANIFEST_DIR"); - let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/contacts/src/lib.rs"); - +fn package_reports_targets_with_no_platform() { + let project = tempfile::tempdir().expect("temp project dir"); + let input = calculator_idl(project.path()); let bins = tempfile::tempdir().expect("temp bins dir"); - write_prebuilt(bins.path(), "contacts"); + write_prebuilt(bins.path(), "calculator"); + let out = tempfile::tempdir().expect("temp out dir"); - let out_dir = tempfile::tempdir().expect("temp out dir"); + // The tree holds only desktop libraries, so a wasm-only package has no + // `wasm32` module to carry and must fail rather than write nothing. + let output = weaveffi() + .args(["package", input.to_str().unwrap(), "--binaries"]) + .arg(bins.path()) + .args(["--target", "wasm", "-o"]) + .arg(out.path()) + .output() + .unwrap(); + assert!(!output.status.success()); + let err = stderr(&output); + assert!(err.contains("no artifacts for wasm"), "{err}"); + assert!( + err.contains("none of the selected targets produced"), + "{err}" + ); +} - // The prebuilt tree holds only desktop cdylibs, so a wasm-only package has - // no `wasm32` module to carry and must fail with a clear message rather - // than producing an empty artifact. - assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") - .args([ - "package", - input.to_str().unwrap(), - "--binaries", - bins.path().to_str().unwrap(), - "--target", - "wasm", - "-o", - out_dir.path().to_str().unwrap(), - ]) - .assert() - .failure(); +#[test] +fn package_names_the_missing_platform_library() { + let input = repo_root().join("samples/calculator"); + let bins = tempfile::tempdir().expect("temp bins dir"); + write_prebuilt(bins.path(), "calculator"); + let output = weaveffi() + .args(["package", input.to_str().unwrap(), "--binaries"]) + .arg(bins.path()) + .args(["--platforms", "android-arm64", "--target", "kotlin"]) + .output() + .unwrap(); + assert!(!output.status.success()); + let err = stderr(&output); + assert!(err.contains("libcalculator.so"), "{err}"); + assert!( + err.contains("weaveffi build --platforms android-arm64"), + "{err}" + ); +} + +#[test] +fn build_without_a_cargo_toml_explains_the_alternatives() { + let dir = tempfile::tempdir().unwrap(); + let idl = dir.path().join("greeter.yml"); + std::fs::write( + &idl, + "version: \"0.11.0\"\nmodules:\n - name: greeter\n functions:\n - { name: hi, params: [], return: i32 }\n", + ) + .unwrap(); + for command in ["build", "package"] { + let output = weaveffi() + .args([command, idl.to_str().unwrap()]) + .output() + .unwrap(); + assert!(!output.status.success(), "{command} succeeded"); + let err = stderr(&output); + assert!(err.contains("no Cargo.toml at"), "{command}: {err}"); + assert!(err.contains("--binaries"), "{command}: {err}"); + } } #[test] -fn package_requires_a_binary_source() { - let manifest_dir = env!("CARGO_MANIFEST_DIR"); - let repo_root = Path::new(manifest_dir).parent().unwrap().parent().unwrap(); - let input = repo_root.join("samples/contacts/src/lib.rs"); - let out_dir = tempfile::tempdir().expect("temp out dir"); - - // Neither --binaries nor --build: nothing to bundle. - assert_cmd::Command::cargo_bin("weaveffi") - .expect("binary not found") +fn build_rejects_unknown_and_foreign_platforms_before_compiling() { + let input = repo_root().join("samples/calculator"); + let output = weaveffi() .args([ - "package", + "build", input.to_str().unwrap(), - "-o", - out_dir.path().to_str().unwrap(), + "--platforms", + "solaris-sparc", ]) - .assert() - .failure(); + .output() + .unwrap(); + assert!(!output.status.success()); + let err = stderr(&output); + assert!(err.contains("unknown platform `solaris-sparc`"), "{err}"); + assert!(err.contains("darwin-arm64"), "{err}"); + + let foreign = if cfg!(target_os = "windows") { + "darwin-arm64" + } else { + "windows-x64" + }; + let output = weaveffi() + .args(["build", input.to_str().unwrap(), "--platforms", foreign]) + .output() + .unwrap(); + assert!(!output.status.success()); + let err = stderr(&output); + assert!(err.contains("can only be built on a"), "{err}"); + assert!(!err.contains("Compiling"), "checks run before cargo: {err}"); } diff --git a/crates/weaveffi-cli/tests/cli/project_config.rs b/crates/weaveffi-cli/tests/cli/project_config.rs index ef03d134..b9883a28 100644 --- a/crates/weaveffi-cli/tests/cli/project_config.rs +++ b/crates/weaveffi-cli/tests/cli/project_config.rs @@ -55,12 +55,12 @@ const ALL_TARGETS: [&str; 11] = [ "c", "cpp", "swift", "kotlin", "node", "wasm", "python", "dotnet", "dart", "go", "ruby", ]; -/// The kvstore sample's `weaveffi.toml` sits beside its `src/lib.rs`; running -/// on the Rust source must pick it up without `--config` and honor every -/// per-target table plus the `[package]` identity. +/// The kvstore sample's `weaveffi.toml` sits beside its `Cargo.toml`; +/// generating from the crate must pick it up without `--config` and honor +/// every per-target table plus the `[package]` identity. #[test] fn discovers_weaveffi_toml_above_the_input() { - let input = repo_root().join("samples/kvstore/src/lib.rs"); + let input = repo_root().join("samples/kvstore"); let dir = tempfile::tempdir().unwrap(); let out = dir.path(); generate(&input, out, &[]); @@ -101,7 +101,7 @@ fn discovers_weaveffi_toml_above_the_input() { /// `--config` names the file explicitly and wins over discovery. #[test] fn explicit_config_overrides_discovery() { - let input = repo_root().join("samples/kvstore/src/lib.rs"); + let input = repo_root().join("samples/kvstore"); let dir = tempfile::tempdir().unwrap(); let cfg = dir.path().join("other.toml"); fs::write( @@ -111,7 +111,7 @@ fn explicit_config_overrides_discovery() { "name = \"renamed\"\n", "version = \"9.9.9\"\n", "[generators.cpp]\n", - "namespace = \"elsewhere\"\n", + "name = \"elsewhere\"\n", ), ) .unwrap(); @@ -134,22 +134,19 @@ fn explicit_config_overrides_discovery() { ); } -/// Without any config, a `lib.rs` input names its package after the crate -/// directory (Cargo's convention), not after the file stem. +/// Rust source is never read: a `.rs` input fails and points at the crate, +/// whose API is read from its built library. #[test] -fn lib_rs_without_config_is_named_after_its_crate_directory() { - let dir = tempfile::tempdir().unwrap(); - let crate_dir = dir.path().join("mycrate").join("src"); - fs::create_dir_all(&crate_dir).unwrap(); - let input = crate_dir.join("lib.rs"); - fs::copy(repo_root().join("samples/calculator/src/lib.rs"), &input).unwrap(); - let out = dir.path().join("out"); - generate(&input, &out, &["--target", "node"]); - let package_json = read(&out.join("node/package.json")); - assert!( - package_json.contains("\"name\": \"mycrate\""), - "{package_json}" - ); +fn rust_source_input_points_at_the_crate() { + let input = repo_root().join("samples/calculator/src/lib.rs"); + let output = assert_cmd::Command::cargo_bin("weaveffi") + .expect("binary not found") + .args(["generate", input.to_str().unwrap()]) + .output() + .unwrap(); + assert!(!output.status.success()); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("pass the crate"), "{stderr}"); } /// A stale `package:` or `generators:` block in an IDL document is rejected @@ -162,7 +159,7 @@ fn inline_package_block_in_idl_is_an_error() { fs::write( &idl, concat!( - "version: \"0.10.0\"\n", + "version: \"0.11.0\"\n", "package:\n", " name: legacy\n", "modules:\n", @@ -185,7 +182,7 @@ fn inline_package_block_in_idl_is_an_error() { /// spelling) fail loudly so a misplaced option cannot be silently dropped. #[test] fn unknown_config_table_is_an_error() { - let input = repo_root().join("samples/calculator/src/lib.rs"); + let input = repo_root().join("samples/calculator"); let dir = tempfile::tempdir().unwrap(); let cfg = dir.path().join("cfg.toml"); fs::write(&cfg, "[swift]\nmodule_name = \"Old\"\n").unwrap(); @@ -210,7 +207,7 @@ fn unknown_config_table_is_an_error() { /// An unknown `--target` fails instead of silently generating nothing. #[test] fn unknown_target_is_an_error() { - let input = repo_root().join("samples/calculator/src/lib.rs"); + let input = repo_root().join("samples/calculator"); let dir = tempfile::tempdir().unwrap(); let output = assert_cmd::Command::cargo_bin("weaveffi") .expect("binary not found") @@ -240,7 +237,7 @@ fn idl_c_prefix_reaches_every_target() { fs::write( &idl, concat!( - "version: \"0.10.0\"\n", + "version: \"0.11.0\"\n", "modules:\n", " - name: calculator\n", " functions:\n", @@ -271,11 +268,11 @@ fn idl_c_prefix_reaches_every_target() { } /// A Rust producer's prefix is its crate name, so `[package] c_prefix` is -/// rejected for a `.rs` input instead of generating bindings that call +/// rejected for a crate input instead of generating bindings that call /// symbols the producer never exports. #[test] fn rust_producer_rejects_c_prefix() { - let input = repo_root().join("samples/calculator/src/lib.rs"); + let input = repo_root().join("samples/calculator"); let dir = tempfile::tempdir().unwrap(); let cfg = dir.path().join("cfg.toml"); fs::write(&cfg, "[package]\nc_prefix = \"myffi\"\n").unwrap(); diff --git a/crates/weaveffi-cli/tests/fixtures/docs_everywhere.yml b/crates/weaveffi-cli/tests/fixtures/docs_everywhere.yml index a770ead1..dd079ce4 100644 --- a/crates/weaveffi-cli/tests/fixtures/docs_everywhere.yml +++ b/crates/weaveffi-cli/tests/fixtures/docs_everywhere.yml @@ -1,6 +1,8 @@ -version: "0.10.0" +version: "0.11.0" modules: - name: docs + doc: | + Document storage: create, publish, and observe documents. enums: - name: Status doc: | diff --git a/crates/weaveffi-cli/tests/fixtures/edge_cases.yml b/crates/weaveffi-cli/tests/fixtures/edge_cases.yml index 0e7f717b..87505855 100644 --- a/crates/weaveffi-cli/tests/fixtures/edge_cases.yml +++ b/crates/weaveffi-cli/tests/fixtures/edge_cases.yml @@ -1,9 +1,11 @@ -version: "0.10.0" +version: "0.11.0" # Shapes the mainstream fixtures do not reach: identifiers that are reserved -# words in one or more target languages, deeply nested composites, optional -# parameters, interfaces in every legal position (including inside records, -# lists, maps, and iterators), async functions with non-string results, -# type-level deprecation, scalar and string iterators, and callback interfaces +# words in one or more target languages, interface members spelled like the +# members object wrappers declare or inherit (close, dispose, handle, +# to_string), deeply nested composites, optional parameters, interfaces in +# every legal position (including inside records, lists, maps, and +# iterators), async functions with non-string results, type-level +# deprecation, scalar and string iterators, and callback interfaces # carrying composite payloads and objects. Snapshots pin how every generator # renders each of these, so a regression in keyword escaping or a codec edge # shows up as a diff instead of as a compile error in a downstream project. @@ -182,6 +184,14 @@ modules: deprecated: "Use rename instead" params: [] return: string + - name: close + doc: Spelled like the release method of several wrappers + params: [] + return: bool + - name: dispose + doc: Spelled like the release method of the Dart and .NET wrappers + params: + - { name: force, type: bool } - name: Vault doc: Deprecated interface with a throwing constructor deprecated: "Use Widget instead" @@ -195,12 +205,20 @@ modules: - name: peek params: [] return: "Widget?" + - name: handle + doc: Spelled like the handle accessor of several wrappers + params: [] + return: i64 statics: - name: probe doc: Throwing static params: [] return: bool throws: true + - name: to_string + doc: Static spelled like an inherited object member + params: [] + return: string functions: - name: noop doc: No parameters and no result diff --git a/crates/weaveffi-cli/tests/fixtures/kitchen_sink.yml b/crates/weaveffi-cli/tests/fixtures/kitchen_sink.yml index 32dcb996..6ff3b8da 100644 --- a/crates/weaveffi-cli/tests/fixtures/kitchen_sink.yml +++ b/crates/weaveffi-cli/tests/fixtures/kitchen_sink.yml @@ -1,4 +1,4 @@ -version: "0.10.0" +version: "0.11.0" modules: - name: shared structs: @@ -52,6 +52,19 @@ modules: - { name: item, type: Item } - { name: gadget, type: Gadget } return: bool + - name: label + doc: A display label, failing with a kitchen error + params: [] + return: string + throws: true + - name: latest + doc: The most recent item the listener kept, if any + params: [] + return: "Item?" + - name: favorite + doc: The listener's favorite gadget + params: [] + return: Gadget errors: name: KitchenErrors codes: @@ -116,6 +129,11 @@ modules: params: - { name: listener, type: ReadyListener } return: i32 + - name: maybe_subscribe + doc: Register a listener if one is given + params: + - { name: listener, type: "ReadyListener?" } + return: bool - name: maybe_gadget doc: Optional object in both directions params: diff --git a/crates/weaveffi-cli/tests/fixtures/kitchen_sink_annotated.rs b/crates/weaveffi-cli/tests/fixtures/kitchen_sink_annotated.rs deleted file mode 100644 index 63d2ed2d..00000000 --- a/crates/weaveffi-cli/tests/fixtures/kitchen_sink_annotated.rs +++ /dev/null @@ -1,248 +0,0 @@ -//! Hand-annotated form of the kitchen-sink IDL, used by the -//! `extract_roundtrip::roundtrip_kitchen_sink` test. Each item carries the -//! WeaveFFI marker attributes that `weaveffi extract` recognizes so the -//! file round-trips back to the original IR shape. -//! -//! The file is parse-only; it is never compiled. The marker attributes -//! (`#[weaveffi::*]`) resolve by their final path segment, so the bare -//! syntactic markers here match the same names the proc-macro re-emits. -//! -//! Round-trip gaps documented in `docs/src/guides/extract.md`: -//! * An error code's yml `doc:` cannot be expressed separately from its -//! `message:` in Rust (the message is the first doc line), so the -//! round-trip test compares codes by name/code/message only. - -#![allow(dead_code)] -#![allow(unused_attributes)] -#![allow(unused_variables)] - -use std::sync::Arc; - -#[weaveffi::module] -mod shared { - /// A reusable token referenced across modules - #[weaveffi::record] - struct Token { - id: i64, - label: String, - } - - /// Trivial cross-module helper - #[weaveffi::export] - fn ping() -> String { - String::new() - } -} - -#[weaveffi::module] -mod kitchen { - /// Task priority levels - #[weaveffi::enumeration] - #[repr(i32)] - enum Priority { - /// Low priority - Low = 0, - /// Normal priority - Normal = 1, - /// High priority - High = 2, - } - - /// Kitchen sink struct exercising every field feature - #[weaveffi::record] - struct Item { - /// Stable identifier - id: i64, - /// Display name - name: String, - /// Initial count - count: i32, - /// Whether the item is enabled - enabled: bool, - /// Scaling ratio - ratio: f64, - nick: u32, - payload: Vec, - tags: Vec, - attrs: HashMap, - parent: Option, - /// Cross-module struct reference - token: Token, - priority: Priority, - /// An optional object inside a record - gadget: Option>, - } - - /// Consumer-implemented observer of ready items - #[weaveffi::callback_interface] - trait ReadyListener: Send + Sync { - /// Fires when an item is ready - fn on_ready(&self, code: i32, msg: String); - /// Receives the item itself and says whether to keep listening - fn on_item(&self, item: &Item, gadget: Arc) -> bool; - } - - #[weaveffi::error] - enum KitchenErrors { - /// Item not found - NotFound = 1, - /// Invalid input - InvalidInput = 2, - } - - /// A pocket-sized interface exercising the object surface - #[weaveffi::interface] - struct Gadget { - id: i64, - } - - impl Gadget { - /// Create a gadget with the given id - pub fn new(id: i64) -> Self { - Gadget { id } - } - - /// Render the gadget as a human-readable string - pub fn describe(&self) -> String { - format!("gadget {}", self.id) - } - - /// Poke the gadget a number of times, failing on a negative count - pub fn poke(&self, times: i32) -> Result { - if times < 0 { - return Err(KitchenErrors::InvalidInput); - } - Ok(times) - } - - /// The gadget subsystem version string - pub fn version() -> String { - String::new() - } - } - - /// Identity for bool - #[weaveffi::export] - fn bool_id(x: bool) -> bool { - x - } - - #[weaveffi::export] - fn i32_id(x: i32) -> i32 { - x - } - - #[weaveffi::export] - fn u32_id(x: u32) -> u32 { - x - } - - #[weaveffi::export] - fn i64_id(x: i64) -> i64 { - x - } - - #[weaveffi::export] - fn f64_id(x: f64) -> f64 { - x - } - - #[weaveffi::export] - fn echo_string(s: String) -> String { - s - } - - #[weaveffi::export] - fn echo_bytes(b: Vec) -> Vec { - b - } - - /// Register a listener, returning how many are attached - #[weaveffi::export] - fn subscribe(listener: Arc) -> i32 { - 1 - } - - /// Optional object in both directions - #[weaveffi::export] - fn maybe_gadget(fallback: Option>) -> Option> { - fallback - } - - /// A list of objects - #[weaveffi::export] - fn gadgets() -> Vec> { - Vec::new() - } - - /// An iterator over objects - #[weaveffi::export] - fn stream_gadgets() -> weaveffi::Iter> { - weaveffi::Iter::new(Vec::new()) - } - - /// Iterator return type - #[weaveffi::export] - fn stream_items() -> weaveffi::Iter { - weaveffi::Iter::new(Vec::new()) - } - - /// Optional struct return - #[weaveffi::export] - fn maybe_item(id: i64) -> Result, KitchenErrors> { - Ok(None) - } - - /// List of structs - #[weaveffi::export] - fn list_items() -> Vec { - Vec::new() - } - - /// Map return type - #[weaveffi::export] - fn get_attrs() -> HashMap { - HashMap::new() - } - - /// Returns the shared Token type from another module - #[weaveffi::export] - fn cross_module_token() -> Token { - todo!() - } - - /// Async operation - #[weaveffi::export] - async fn do_async(input: String) -> String { - input - } - - /// Cancellable async operation - #[weaveffi::export] - #[weaveffi::cancellable] - async fn do_cancellable(input: String) -> String { - input - } - - /// Legacy operation kept for compatibility - #[weaveffi::export] - #[deprecated(since = "0.1.0", note = "Use new_op instead")] - fn legacy_op() -> i32 { - 0 - } - - /// Replacement for legacy_op - #[weaveffi::export] - fn new_op() -> i32 { - 0 - } - - #[weaveffi::module] - mod nested { - /// Trivial nested-module function - #[weaveffi::export] - fn hello() -> String { - String::new() - } - } -} diff --git a/crates/weaveffi-cli/tests/fixtures/nested_modules.yml b/crates/weaveffi-cli/tests/fixtures/nested_modules.yml index 012ed7a6..136e3791 100644 --- a/crates/weaveffi-cli/tests/fixtures/nested_modules.yml +++ b/crates/weaveffi-cli/tests/fixtures/nested_modules.yml @@ -1,4 +1,4 @@ -version: "0.10.0" +version: "0.11.0" # Exercises multi-level nested modules and type references that cross nesting # boundaries in every direction: # * parent -> nested child (graphics.Canvas.root : shapes.Shape) @@ -6,9 +6,9 @@ version: "0.10.0" # * nested -> deeper nested (graphics.shapes.Shape.detail : geometry.Detail) # * nested -> ancestor return (graphics.shapes.describe -> Canvas) # * deeper -> ancestor param (geometry.compute(shape: Shape)) -# All bare names below are auto-qualified by the resolver to their full -# dot-joined module path, which the C ABI then flattens to underscores -# (e.g. graphics.shapes.geometry.Detail -> weaveffi_graphics_shapes_geometry_Detail). +# Type names are global, so every reference is the bare name; the C ABI +# names each type after its declaring module's underscore-joined path +# (e.g. Detail in graphics.shapes.geometry -> {prefix}_graphics_shapes_geometry_Detail). # The root module also declares an error domain that geometry.compute (two # levels down) relies on through domain inheritance for its `throws: true`. modules: diff --git a/crates/weaveffi-cli/tests/fixtures/producer/Cargo.toml b/crates/weaveffi-cli/tests/fixtures/producer/Cargo.toml new file mode 100644 index 00000000..05c44ae4 --- /dev/null +++ b/crates/weaveffi-cli/tests/fixtures/producer/Cargo.toml @@ -0,0 +1,20 @@ +# A small Rust producer for the CLI's library-mode tests: `weaveffi extract` +# and `weaveffi generate` read its API from the library it builds. +[package] +name = "producer-fixture" +version = "0.3.0" +edition.workspace = true +publish = false + +[lib] +crate-type = ["cdylib", "rlib"] + +[features] +# Compiles in the declarations marked `#[cfg(feature = "extra")]`. +extra = [] + +[dependencies] +weaveffi = { path = "../../../../weaveffi" } + +[lints] +workspace = true diff --git a/crates/weaveffi-cli/tests/fixtures/producer/expected.yml b/crates/weaveffi-cli/tests/fixtures/producer/expected.yml new file mode 100644 index 00000000..36c7d6f9 --- /dev/null +++ b/crates/weaveffi-cli/tests/fixtures/producer/expected.yml @@ -0,0 +1,102 @@ +version: 0.11.0 +modules: +- name: receipt + doc: Receipts, in a sibling tree that uses the shop's record. + functions: + - name: line + params: + - name: item + type: Item + return: string + doc: One line describing `item`. +- name: shop + doc: The shop. + functions: + - name: price_of + params: + - name: item + type: Item + return: u64 + doc: What `item` costs. + - name: buy + params: + - name: name + type: string + return: Item + doc: |- + Buy one `name`. Fails with `ShopError::SoldOut` for anything but + `"tea"`. + throws: true + - name: list_price + return: u64 + doc: The list price of everything. + interfaces: + - name: Cart + doc: A shopping cart. + constructors: + - name: new + doc: An empty cart. + methods: + - name: add + params: + - name: item + type: Item + - name: watcher + type: Watcher? + doc: Add `item`, telling `watcher`. + - name: total + return: u64 + doc: The sum of every item's price. + callback_interfaces: + - name: Watcher + doc: Told about every item added to a cart. + methods: + - name: added + params: + - name: item + type: Item + doc: '`item` was added.' + structs: + - name: Item + doc: Something for sale. + fields: + - name: name + type: string + doc: The item's name. + - name: price + type: u64 + doc: What it costs. + - name: size + type: Size + doc: How big it is. + enums: + - name: Size + doc: How big an item is. + variants: + - name: Small + value: 0 + doc: Small. + - name: Large + value: 1 + doc: Large. + errors: + name: ShopError + codes: + - name: SoldOut + code: 1 + message: sold out + doc: sold out + fields: + - name: item + type: string + doc: The item that ran out. + modules: + - name: stock + doc: Stock levels. + functions: + - name: count + params: + - name: cart + type: Cart? + return: u32 + doc: How many items `cart` holds, if there is one. diff --git a/crates/weaveffi-cli/tests/fixtures/producer/src/lib.rs b/crates/weaveffi-cli/tests/fixtures/producer/src/lib.rs new file mode 100644 index 00000000..6b3951d3 --- /dev/null +++ b/crates/weaveffi-cli/tests/fixtures/producer/src/lib.rs @@ -0,0 +1,177 @@ +//! A small producer exercising every kind of declaration the library +//! metadata carries: a module tree with a nested module, an error domain +//! with a payload, a type alias, a C-style enum, a record, a callback +//! interface, an interface whose members span two `impl` blocks, and a +//! sibling tree that uses the first tree's record. Declarations marked +//! `#[cfg(feature = "extra")]` (an item, an `impl` block, and a nested +//! module) exist only in a build with the `extra` feature, and so only in +//! that build's metadata. + +/// The shop. +#[weaveffi::module] +pub mod shop { + use std::sync::{Arc, Mutex}; + + use weaveffi::ForeignError; + + /// The shop's error domain. + #[weaveffi::error] + #[derive(Debug)] + #[repr(i32)] + pub enum ShopError { + /// sold out + SoldOut { + /// The item that ran out. + item: String, + } = 1, + } + + impl std::fmt::Display for ShopError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::SoldOut { item } => write!(f, "{item} is sold out"), + } + } + } + + /// A price in cents. + pub type Price = u64; + + /// How big an item is. + #[weaveffi::enumeration] + #[repr(i32)] + #[derive(Clone, Copy, Debug)] + pub enum Size { + /// Small. + Small = 0, + /// Large. + Large = 1, + } + + /// Something for sale. + #[weaveffi::record] + #[derive(Clone, Debug)] + pub struct Item { + /// The item's name. + pub name: String, + /// What it costs. + pub price: Price, + /// How big it is. + pub size: Size, + } + + /// Told about every item added to a cart. + #[weaveffi::callback_interface] + pub trait Watcher: Send + Sync { + /// `item` was added. + fn added(&self, item: &Item) -> Result<(), ForeignError>; + } + + /// A shopping cart. + #[weaveffi::interface] + #[derive(Default)] + pub struct Cart { + items: Mutex>, + } + + impl Cart { + /// An empty cart. + pub fn new() -> Cart { + Cart::default() + } + + /// Add `item`, telling `watcher`. + pub fn add(&self, item: Item, watcher: Option>) { + if let Some(w) = watcher { + let _ = w.added(&item); + } + self.items.lock().unwrap().push(item); + } + + /// The sum of every item's price. + pub fn total(&self) -> Price { + self.items.lock().unwrap().iter().map(|i| i.price).sum() + } + } + + #[cfg(feature = "extra")] + impl Cart { + /// Empty the cart. + pub fn clear(&self) { + self.items.lock().unwrap().clear(); + } + } + + /// What `item` costs. + #[weaveffi::export] + pub fn price_of(item: Item) -> Price { + item.price + } + + /// Buy one `name`. Fails with [`ShopError::SoldOut`] for anything but + /// `"tea"`. + #[weaveffi::export] + pub fn buy(name: String) -> Result { + if name != "tea" { + return Err(ShopError::SoldOut { item: name }); + } + Ok(Item { + name, + price: 250, + size: Size::Small, + }) + } + + /// The discount in percent. + #[cfg(feature = "extra")] + #[weaveffi::export] + pub fn discount() -> u32 { + 10 + } + + /// The list price of everything. + #[cfg(not(feature = "extra"))] + #[weaveffi::export] + pub fn list_price() -> Price { + 250 + } + + /// Stock levels. + #[weaveffi::module] + pub mod stock { + use std::sync::Arc; + + use super::Cart; + + /// How many items `cart` holds, if there is one. + #[weaveffi::export] + pub fn count(cart: Option>) -> u32 { + cart.map_or(0, |c| c.items.lock().unwrap().len() as u32) + } + } + + /// Promotions. + #[cfg(feature = "extra")] + #[weaveffi::module] + pub mod promo { + /// Today's code. + #[weaveffi::export] + pub fn code() -> String { + "TEA10".to_string() + } + } +} + +/// Receipts, in a sibling tree that uses the shop's record. +#[weaveffi::module] +pub mod receipt { + use super::shop::Item; + + /// One line describing `item`. + #[weaveffi::export] + pub fn line(item: Item) -> String { + format!("{}: {}", item.name, item.price) + } +} + +weaveffi::export_runtime!(); diff --git a/crates/weaveffi-cli/tests/fixtures/shapes.yml b/crates/weaveffi-cli/tests/fixtures/shapes.yml index 3fdef3a8..f2b3e653 100644 --- a/crates/weaveffi-cli/tests/fixtures/shapes.yml +++ b/crates/weaveffi-cli/tests/fixtures/shapes.yml @@ -1,4 +1,4 @@ -version: "0.10.0" +version: "0.11.0" modules: - name: shapes doc: Rich (algebraic) enums and the expanded numeric primitives. diff --git a/crates/weaveffi-cli/tests/snapshots.rs b/crates/weaveffi-cli/tests/snapshots.rs index 47c21256..317f04d6 100644 --- a/crates/weaveffi-cli/tests/snapshots.rs +++ b/crates/weaveffi-cli/tests/snapshots.rs @@ -12,32 +12,43 @@ //! deeply nested composites, optional parameters, interfaces and callback //! interfaces in every legal position, async functions with non-string //! results, type-level deprecation, and scalar and string iterators). One -//! test per generator runs it over all -//! five fixtures into a fresh tempdir, walks the -//! resulting files in sorted order, and snapshots each file under -//! `tests/snapshots/`. Regressions in any generator's output fail the -//! affected `cargo insta test` job; behavioral regressions are the -//! conformance harness's job. +//! test per generator renders all five fixtures and checks each file, in +//! sorted order, by one of three rules: +//! +//! - A file that shares its name with a C target output is that target's +//! copy of the C header. It must be byte-equal to the C target's own +//! header, which `snapshot_c` snapshots, so it gets no snapshot of its own. +//! - A file matching the target's entry in [`FIXED_FILES`] (fixed runtimes, +//! package manifests, READMEs) is the same for every fixture apart from +//! its name, so it's snapshotted once, from [`FIXED_FROM`]. +//! - Every other file depends on the fixture and is snapshotted for every +//! fixture. +//! +//! Snapshots live under `tests/snapshots/`. Regressions in any generator's +//! output fail the affected `cargo insta test` job; behavioral regressions +//! are the conformance harness's job. +use std::collections::{BTreeMap, BTreeSet}; use std::fs; use std::path::Path; use camino::Utf8Path; -use weaveffi_gen::codegen::{ConfiguredBackend, Target}; -use weaveffi_gen::targets::c::{CConfig, CGenerator}; -use weaveffi_gen::targets::cpp::{CppConfig, CppGenerator}; -use weaveffi_gen::targets::dart::{DartConfig, DartGenerator}; -use weaveffi_gen::targets::dotnet::{DotnetConfig, DotnetGenerator}; -use weaveffi_gen::targets::go::{GoConfig, GoGenerator}; -use weaveffi_gen::targets::kotlin::{KotlinConfig, KotlinGenerator}; -use weaveffi_gen::targets::node::{NodeConfig, NodeGenerator}; -use weaveffi_gen::targets::python::{PythonConfig, PythonGenerator}; -use weaveffi_gen::targets::ruby::{RubyConfig, RubyGenerator}; -use weaveffi_gen::targets::swift::{SwiftConfig, SwiftGenerator}; -use weaveffi_gen::targets::wasm::{WasmConfig, WasmGenerator}; +use weaveffi_cli::codegen::{ConfiguredBackend, Target}; +use weaveffi_cli::targets::c::{CConfig, CGenerator}; +use weaveffi_cli::targets::cpp::{CppConfig, CppGenerator}; +use weaveffi_cli::targets::dart::{DartConfig, DartGenerator}; +use weaveffi_cli::targets::dotnet::{DotnetConfig, DotnetGenerator}; +use weaveffi_cli::targets::go::{GoConfig, GoGenerator}; +use weaveffi_cli::targets::kotlin::{KotlinConfig, KotlinGenerator}; +use weaveffi_cli::targets::node::{NodeConfig, NodeGenerator}; +use weaveffi_cli::targets::python::{PythonConfig, PythonGenerator}; +use weaveffi_cli::targets::ruby::{RubyConfig, RubyGenerator}; +use weaveffi_cli::targets::swift::{SwiftConfig, SwiftGenerator}; +use weaveffi_cli::targets::wasm::{WasmConfig, WasmGenerator}; +use weaveffi_model::model::Model; use weaveffi_model::parse::parse_api_str; -use weaveffi_model::resolved::ResolvedApi; -use weaveffi_model::validate::validate_api; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; const FIXTURES: [&str; 5] = [ "kitchen_sink", @@ -47,7 +58,82 @@ const FIXTURES: [&str; 5] = [ "edge_cases", ]; -fn load_api(stem: &str) -> ResolvedApi { +/// The fixture that fixed files are snapshotted from. It exercises every +/// feature, so it also emits every optional fixed file (such as Kotlin's +/// `Async.kt`). +const FIXED_FROM: &str = "kitchen_sink"; + +/// Per target, the files whose content doesn't depend on the fixture beyond +/// its name: fixed runtimes, package manifests, and READMEs. (A README's +/// usage example names the top-level modules, and Kotlin's +/// `build.gradle.kts` adds the coroutines dependency only for async APIs; +/// [`FIXED_FROM`] covers both.) A pattern starting with `*` matches a +/// file-name suffix; any other pattern matches the whole file name. Every +/// pattern must match a [`FIXED_FROM`] output, so a renamed or removed file +/// can't leave a stale entry behind. +const FIXED_FILES: &[(&str, &[&str])] = &[ + ("c", &[]), + ("cpp", &["CMakeLists.txt", "README.md"]), + ("swift", &["Package.swift", "module.modulemap"]), + ( + "kotlin", + &[ + "Async.kt", + "Buffers.kt", + "CMakeLists.txt", + "Runtime.kt", + "build.gradle.kts", + "consumer-rules.pro", + "settings.gradle.kts", + ], + ), + ( + "node", + &["README.md", "binding.gyp", "package.json", "runtime.js"], + ), + ( + "wasm", + &["README.md", "linear.js", "package.json", "runtime.js"], + ), + ( + "python", + &["README.md", "__init__.py", "py.typed", "pyproject.toml"], + ), + ("dotnet", &["*.csproj", "README.md", "Runtime.cs"]), + ("dart", &["README.md", "pubspec.yaml"]), + ("go", &["README.md", "codec.go", "go.mod", "runtime.go"]), + ("ruby", &["*.gemspec", "README.md", "runtime.rb"]), +]; + +fn fixed_patterns(target: &str) -> &'static [&'static str] { + FIXED_FILES + .iter() + .find(|(name, _)| *name == target) + .map(|(_, patterns)| *patterns) + .unwrap_or_else(|| panic!("target {target} has no FIXED_FILES entry")) +} + +fn matches_pattern(pattern: &str, file_name: &str) -> bool { + match pattern.strip_prefix('*') { + Some(suffix) => file_name.ends_with(suffix), + None => file_name == pattern, + } +} + +/// The C target's outputs for `api`, keyed by file name. Other targets copy +/// the C header verbatim, under the same file name. +fn c_outputs(model: &Model) -> BTreeMap { + ConfiguredBackend::new(CGenerator, CConfig::default()) + .render(model, Utf8Path::new("out")) + .into_iter() + .map(|file| { + let name = file.path.file_name().expect("file name").to_owned(); + (name, file.contents) + }) + .collect() +} + +fn load_model(stem: &str) -> Model { let path = Path::new(env!("CARGO_MANIFEST_DIR")) .join("tests/fixtures") .join(format!("{stem}.yml")); @@ -55,9 +141,8 @@ fn load_api(stem: &str) -> ResolvedApi { .unwrap_or_else(|e| panic!("read fixture {}: {e}", path.display())); let api = parse_api_str(&contents, "yaml") .unwrap_or_else(|e| panic!("parse fixture {}: {e}", path.display())); - validate_api(api, None) + validate(&api, &Identity::named(stem), None) .unwrap_or_else(|e| panic!("validate fixture {}: {e}", path.display())) - .with_identity(weaveffi_model::pkg::Identity::named(stem)) } fn sanitize(rel: &Path) -> String { @@ -98,9 +183,16 @@ fn assert_prelude_present(contents: &str, file: &Path) { fn run_snapshots(target: &dyn Target) { let out_dir = Utf8Path::new("out"); + let fixed = fixed_patterns(target.name()); + let mut unmatched: BTreeSet<&str> = fixed.iter().copied().collect(); for stem in FIXTURES { - let api = load_api(stem); - let mut files = target.render(&api, out_dir); + let model = load_model(stem); + let c_headers = if target.name() == "c" { + BTreeMap::new() + } else { + c_outputs(&model) + }; + let mut files = target.render(&model, out_dir); files.sort_by(|a, b| a.path.cmp(&b.path)); assert!( !files.is_empty(), @@ -115,16 +207,36 @@ fn run_snapshots(target: &dyn Target) { omit_expression => true, }, { for file in files { + assert_prelude_present(&file.contents, file.path.as_std_path()); + let file_name = file.path.file_name().expect("file name"); + if let Some(header) = c_headers.get(file_name) { + assert!( + file.contents == *header, + "{} in fixture {stem} isn't a byte-equal copy of the C target's {file_name}", + file.path, + ); + continue; + } + if let Some(pattern) = fixed.iter().find(|p| matches_pattern(p, file_name)) { + if stem != FIXED_FROM { + continue; + } + unmatched.remove(pattern); + } let rel = file .path .strip_prefix(&root) .expect("file under generator root"); let name = format!("{}_{stem}__{}", target.name(), sanitize(rel.as_std_path())); insta::assert_snapshot!(name, redact_version(&file.contents)); - assert_prelude_present(&file.contents, file.path.as_std_path()); } }); } + assert!( + unmatched.is_empty(), + "FIXED_FILES patterns for {} match no {FIXED_FROM} output: {unmatched:?}", + target.name(), + ); } macro_rules! snapshot_tests { diff --git a/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_buffer_h.snap b/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_buffer_h.snap index 2ee73664..c31d6ffa 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_buffer_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_buffer_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef DOCS_EVERYWHERE_BUFFER_H #define DOCS_EVERYWHERE_BUFFER_H diff --git a/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_h.snap b/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_h.snap index 31ab85e2..116d8703 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_docs_everywhere__docs_everywhere_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef DOCS_EVERYWHERE_H #define DOCS_EVERYWHERE_H @@ -45,7 +45,7 @@ extern "C" { /* The C ABI revision this header was generated against. The producer exports docs_everywhere_abi_version() so a consumer can refuse to load a library built for a different revision. */ -#define DOCS_EVERYWHERE_ABI_VERSION 3u +#define DOCS_EVERYWHERE_ABI_VERSION 4u DOCS_EVERYWHERE_API uint32_t docs_everywhere_abi_version(void); /* Error slot written by every fallible call. `message` is NUL-terminated @@ -63,9 +63,13 @@ typedef struct docs_everywhere_error { } docs_everywhere_error; /* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ + interface implementations call this to report a failure without + allocating with a foreign allocator. */ DOCS_EVERYWHERE_API void docs_everywhere_error_set(docs_everywhere_error* err, int32_t code, const char* message); +/* Replace `err`'s payload with a producer-owned copy of `len` bytes: the + fields of a declared error code, which a callback method that throws + attaches after docs_everywhere_error_set. */ +DOCS_EVERYWHERE_API void docs_everywhere_error_set_payload(docs_everywhere_error* err, const uint8_t* ptr, size_t len); DOCS_EVERYWHERE_API void docs_everywhere_error_clear(docs_everywhere_error* err); /* Async completions receive a heap-boxed error the consumer owns; @@ -73,7 +77,12 @@ DOCS_EVERYWHERE_API void docs_everywhere_error_clear(docs_everywhere_error* err) NULL is a no-op. */ DOCS_EVERYWHERE_API void docs_everywhere_error_free(docs_everywhere_error* err); -/* Release a string, bytes, or value buffer the producer returned. */ +/* Allocate a zero-filled run of `len` bytes (NULL for 0). A callback + method returns a string, bytes, or buffer as such a run, which the + producer adopts; any other run is released with docs_everywhere_free_bytes. */ +DOCS_EVERYWHERE_API uint8_t* docs_everywhere_alloc(size_t len); +/* Release a string, bytes, or value buffer the producer returned, or a + run from docs_everywhere_alloc, with its exact length. NULL or 0 is a no-op. */ DOCS_EVERYWHERE_API void docs_everywhere_free_bytes(uint8_t* ptr, size_t len); /* Cancellation for `cancellable` async functions. A token starts with @@ -86,21 +95,59 @@ DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_cancel(docs_everywhere_can DOCS_EVERYWHERE_API bool docs_everywhere_cancel_token_is_cancelled(const docs_everywhere_cancel_token* token); DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_destroy(docs_everywhere_cancel_token* token); -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -DOCS_EVERYWHERE_API uint8_t* docs_everywhere_alloc(uint32_t size); -DOCS_EVERYWHERE_API void docs_everywhere_dealloc(uint8_t* ptr, uint32_t size); - /* Live-allocation counters for leak checks in tests: 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ + Kind -1 is 1 when the producer counts at all (its `leak-check` + feature) and 0 when every counter reads 0 regardless. */ DOCS_EVERYWHERE_API uint64_t docs_everywhere_debug_live(int32_t kind); -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define DOCS_EVERYWHERE_DOCS_CHECKSUM 0x7950eec1e9cfc819ull -DOCS_EVERYWHERE_API uint64_t docs_everywhere_docs_checksum(void); +/* One declaration's fingerprint in a module's contract table: FNV-1a 64 + of its dotted path and of its canonical signature. */ +typedef struct docs_everywhere_contract_entry { + uint64_t id; + uint64_t hash; +} docs_everywhere_contract_entry; + +/* The contract of module `docs`: one entry per declaration, sorted by + id. The producer's table comes from the function below; a consumer + checks each entry it was generated with against it once, before any + other call (producer entries it doesn't know are fine). */ +DOCS_EVERYWHERE_API const docs_everywhere_contract_entry* docs_everywhere_docs_contract(size_t* out_len); +#define DOCS_EVERYWHERE_DOCS_CONTRACT_LEN 14 +#define DOCS_EVERYWHERE_DOCS_CONTRACT { \ + {0x00b2187fd75634ecull, 0x2263c6f787bf6081ull}, /* docs.publish_async */ \ + {0x12fc242b9dc6444cull, 0x96405f484a642b31ull}, /* docs.Notebook.insert */ \ + {0x2213822ed05cd5bdull, 0x26c5833392aa63ceull}, /* docs.Notebook.new */ \ + {0x2bd762bcdaa207ffull, 0x1b0ebab2d04c49b0ull}, /* docs.SavedListener */ \ + {0x2e086c45e9d617deull, 0x4d43bf32358c01ffull}, /* docs.Notebook.find */ \ + {0x4817dc98197ad960ull, 0xf8017b5a7d58b2c7ull}, /* docs.Notebook.size */ \ + {0x614de47452d5a55cull, 0xd9aae1472494ffefull}, /* docs.get_document */ \ + {0x67c5cb9fc119bcd5ull, 0x08d5b25a1a22a7faull}, /* docs.Document */ \ + {0x8d484bf2d78c9b70ull, 0xe9093cf96912194cull}, /* docs.Notebook.max_documents */ \ + {0xaea61d6451691c67ull, 0xe4704128b3524b3dull}, /* docs.watch */ \ + {0xd68d6ad1540cdec5ull, 0x01afeb40b0a8bfe1ull}, /* docs.Notebook */ \ + {0xdb0601444a4760ccull, 0xe56c69629b444797ull}, /* docs.create_document */ \ + {0xe12d6f4838347b3eull, 0xa9bab86646d84f51ull}, /* docs.Status */ \ + {0xee2f8d25b3313b50ull, 0x9691232f35439a2bull}, /* docs.DocsErrors */ \ +} +/* The id of the first expected entry the loaded library lacks or + declares differently, or 0 when these bindings match it. */ +static inline uint64_t docs_everywhere_docs_contract_check(void) { + static const docs_everywhere_contract_entry expected[] = DOCS_EVERYWHERE_DOCS_CONTRACT; + size_t len = 0; + const docs_everywhere_contract_entry* table = docs_everywhere_docs_contract(&len); + for (size_t i = 0; i < DOCS_EVERYWHERE_DOCS_CONTRACT_LEN; i++) { + size_t lo = 0, hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) lo = mid + 1; else hi = mid; + } + if (lo == len || table[lo].id != expected[i].id || table[lo].hash != expected[i].hash) { + return expected[i].id; + } + } + return 0; +} /* * Strings, bytes, and value buffers cross as (ptr, len) pairs. A @@ -142,12 +189,14 @@ typedef struct docs_everywhere_docs_Notebook docs_everywhere_docs_Notebook; * Implementations must be safe to call from any thread. * * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. + * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable, with `size` set to `sizeof` the vtable and `flags` to 0 (the producer rejects a smaller vtable). The producer may call any entry from any thread until it calls `free(ctx)` exactly once, also from any thread. A method reports failure with `error_set(out_err, ...)`; a string, bytes, or buffer return is a run the consumer allocates with `alloc` and writes to `out_ptr` and `out_len`, which the producer adopts. */ typedef struct docs_everywhere_docs_SavedListener_vtable { + uint32_t size; + uint32_t flags; + void (*free)(void* ctx); /** Fires after a document has been persisted successfully */ void (*on_saved)(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err); - void (*free)(void* ctx); } docs_everywhere_docs_SavedListener_vtable; typedef void (*docs_everywhere_docs_publish_async_callback)(void* context, docs_everywhere_error* err, bool result); diff --git a/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_buffer_h.snap b/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_buffer_h.snap index 45cc3993..294a06e1 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_buffer_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_buffer_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef EDGE_CASES_BUFFER_H #define EDGE_CASES_BUFFER_H @@ -385,20 +385,20 @@ typedef struct edge_cases_list_opt_edge_cases_Outer edge_cases_list_opt_edge_cas typedef struct edge_cases_list_opt_edge_cases_Widget edge_cases_list_opt_edge_cases_Widget; typedef struct edge_cases_list_edge_cases_Outer edge_cases_list_edge_cases_Outer; -/** `[edge_cases.Leaf]` */ +/** `[Leaf]` */ struct edge_cases_list_edge_cases_Leaf { edge_cases_edge_cases_Leaf* items; size_t len; }; -/** `{string:edge_cases.Inner}` */ +/** `{string:Inner}` */ struct edge_cases_map_string_edge_cases_Inner { edge_cases_str* keys; edge_cases_edge_cases_Inner* values; size_t len; }; -/** `[edge_cases.Mode]` */ +/** `[Mode]` */ struct edge_cases_list_edge_cases_Mode { edge_cases_edge_cases_Mode* items; size_t len; @@ -429,13 +429,13 @@ struct edge_cases_map_i64_string { size_t len; }; -/** `[edge_cases.Widget]` */ +/** `[Widget]` */ struct edge_cases_list_edge_cases_Widget { edge_cases_edge_cases_Widget** items; size_t len; }; -/** `{string:edge_cases.Widget}` */ +/** `{string:Widget}` */ struct edge_cases_map_string_edge_cases_Widget { edge_cases_str* keys; edge_cases_edge_cases_Widget** values; @@ -448,19 +448,19 @@ struct edge_cases_list_string { size_t len; }; -/** `[edge_cases.Outer?]` */ +/** `[Outer?]` */ struct edge_cases_list_opt_edge_cases_Outer { edge_cases_edge_cases_Outer** items; size_t len; }; -/** `[edge_cases.Widget?]` */ +/** `[Widget?]` */ struct edge_cases_list_opt_edge_cases_Widget { edge_cases_edge_cases_Widget** items; size_t len; }; -/** `[edge_cases.Outer]` */ +/** `[Outer]` */ struct edge_cases_list_edge_cases_Outer { edge_cases_edge_cases_Outer* items; size_t len; diff --git a/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_h.snap b/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_h.snap index 94864086..fa3c9950 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_edge_cases__edge_cases_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef EDGE_CASES_H #define EDGE_CASES_H @@ -45,7 +45,7 @@ extern "C" { /* The C ABI revision this header was generated against. The producer exports edge_cases_abi_version() so a consumer can refuse to load a library built for a different revision. */ -#define EDGE_CASES_ABI_VERSION 3u +#define EDGE_CASES_ABI_VERSION 4u EDGE_CASES_API uint32_t edge_cases_abi_version(void); /* Error slot written by every fallible call. `message` is NUL-terminated @@ -63,9 +63,13 @@ typedef struct edge_cases_error { } edge_cases_error; /* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ + interface implementations call this to report a failure without + allocating with a foreign allocator. */ EDGE_CASES_API void edge_cases_error_set(edge_cases_error* err, int32_t code, const char* message); +/* Replace `err`'s payload with a producer-owned copy of `len` bytes: the + fields of a declared error code, which a callback method that throws + attaches after edge_cases_error_set. */ +EDGE_CASES_API void edge_cases_error_set_payload(edge_cases_error* err, const uint8_t* ptr, size_t len); EDGE_CASES_API void edge_cases_error_clear(edge_cases_error* err); /* Async completions receive a heap-boxed error the consumer owns; @@ -73,7 +77,12 @@ EDGE_CASES_API void edge_cases_error_clear(edge_cases_error* err); NULL is a no-op. */ EDGE_CASES_API void edge_cases_error_free(edge_cases_error* err); -/* Release a string, bytes, or value buffer the producer returned. */ +/* Allocate a zero-filled run of `len` bytes (NULL for 0). A callback + method returns a string, bytes, or buffer as such a run, which the + producer adopts; any other run is released with edge_cases_free_bytes. */ +EDGE_CASES_API uint8_t* edge_cases_alloc(size_t len); +/* Release a string, bytes, or value buffer the producer returned, or a + run from edge_cases_alloc, with its exact length. NULL or 0 is a no-op. */ EDGE_CASES_API void edge_cases_free_bytes(uint8_t* ptr, size_t len); /* Cancellation for `cancellable` async functions. A token starts with @@ -86,21 +95,92 @@ EDGE_CASES_API void edge_cases_cancel_token_cancel(edge_cases_cancel_token* toke EDGE_CASES_API bool edge_cases_cancel_token_is_cancelled(const edge_cases_cancel_token* token); EDGE_CASES_API void edge_cases_cancel_token_destroy(edge_cases_cancel_token* token); -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -EDGE_CASES_API uint8_t* edge_cases_alloc(uint32_t size); -EDGE_CASES_API void edge_cases_dealloc(uint8_t* ptr, uint32_t size); - /* Live-allocation counters for leak checks in tests: 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ + Kind -1 is 1 when the producer counts at all (its `leak-check` + feature) and 0 when every counter reads 0 regardless. */ EDGE_CASES_API uint64_t edge_cases_debug_live(int32_t kind); -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define EDGE_CASES_EDGE_CASES_CHECKSUM 0x1567f441073bd4aaull -EDGE_CASES_API uint64_t edge_cases_edge_cases_checksum(void); +/* One declaration's fingerprint in a module's contract table: FNV-1a 64 + of its dotted path and of its canonical signature. */ +typedef struct edge_cases_contract_entry { + uint64_t id; + uint64_t hash; +} edge_cases_contract_entry; + +/* The contract of module `edge_cases`: one entry per declaration, sorted by + id. The producer's table comes from the function below; a consumer + checks each entry it was generated with against it once, before any + other call (producer entries it doesn't know are fine). */ +EDGE_CASES_API const edge_cases_contract_entry* edge_cases_edge_cases_contract(size_t* out_len); +#define EDGE_CASES_EDGE_CASES_CONTRACT_LEN 47 +#define EDGE_CASES_EDGE_CASES_CONTRACT { \ + {0x056a86511785ba2bull, 0x5cba3f16d57c6cdbull}, /* edge_cases.make_widget */ \ + {0x07bfa9ecb2b720d2ull, 0x3813b3f57c64bf35ull}, /* edge_cases.EdgeErrors */ \ + {0x0aaa11867e37f991ull, 0xf090dd52146ebdf0ull}, /* edge_cases.Widget.refresh */ \ + {0x0f9701493898ae4cull, 0x6944f47a275c96e8ull}, /* edge_cases.Inner */ \ + {0x1226b1adac179f7dull, 0x0e95a00ab0bef637ull}, /* edge_cases.async_keywords */ \ + {0x16d36bef1541136cull, 0x5ea7021ec324298full}, /* edge_cases.find_widget */ \ + {0x1b7324c55e980febull, 0xabf87fe372358f42ull}, /* edge_cases.Widget.old_name */ \ + {0x1cbd731467c2bab8ull, 0x4915b9989c2c9074ull}, /* edge_cases.Vault */ \ + {0x2c2a6de67d894b0aull, 0x50a4496b1e7191caull}, /* edge_cases.Vault.open */ \ + {0x322b4cfcd2d6861cull, 0xca7527a379ae8465ull}, /* edge_cases.Keywords */ \ + {0x3f8e23a5f32906f4ull, 0x8d4c908d796f130dull}, /* edge_cases.Leaf */ \ + {0x42ab8a685457b97cull, 0xe5ce30500cadb32dull}, /* edge_cases.Vault.handle */ \ + {0x4a68540e2836d3d0ull, 0xa20e44c71e420ed2ull}, /* edge_cases.mode_stream */ \ + {0x4e030580ac0b111dull, 0xb68b7a18d910e175ull}, /* edge_cases.legacy_value */ \ + {0x4f5e4e5291864ef3ull, 0x55a1a88b21a08f8bull}, /* edge_cases.Legacy */ \ + {0x4f84b06171b4213dull, 0xbaedcfdebb6a2820ull}, /* edge_cases.scalar_stream */ \ + {0x5067a5ff2d9ef910ull, 0xbfc79b18fc4714fcull}, /* edge_cases.Widget */ \ + {0x59aff159112fbf5bull, 0x18e4def57a8cfa49ull}, /* edge_cases.optional_params */ \ + {0x5daf65b98fa77c48ull, 0xc817a8cb202e59c2ull}, /* edge_cases.keywords_roundtrip */ \ + {0x5de8a5c9726c2c5aull, 0x7c50b5b65f7a896dull}, /* edge_cases.register_batch */ \ + {0x618a620309994c3cull, 0x826ad0b1d208d723ull}, /* edge_cases.async_bytes */ \ + {0x648285cdba0a7b8eull, 0xcf6e80e783a4d8beull}, /* edge_cases.WidgetHolder */ \ + {0x649c78079824b10dull, 0x5b12dd26e0634b69ull}, /* edge_cases.async_widget */ \ + {0x7da7a7357d122a37ull, 0xf544e025ffaa3d20ull}, /* edge_cases.async_void */ \ + {0x7e4015bc3a472e2dull, 0xfe22291d90b82e4aull}, /* edge_cases.widget_stream */ \ + {0x805276e5d81fdba7ull, 0xd9b64e360bd03b2bull}, /* edge_cases.Outer */ \ + {0x80e1183326542df4ull, 0xe8a57e12cc6ed276ull}, /* edge_cases.select */ \ + {0x923ce5e9bcd769a1ull, 0x0b973f5dc07dda8bull}, /* edge_cases.Vault.to_string */ \ + {0x98ca8c01f5239404ull, 0xfe11c93eb6b303efull}, /* edge_cases.Widget.rename */ \ + {0xa13e22e98ea278d4ull, 0xf1fae215d7387800ull}, /* edge_cases.widgets_everywhere */ \ + {0xa84a8c9bf2a881e1ull, 0xbe1dc840bf221996ull}, /* edge_cases.async_optional */ \ + {0xad958c478fbf205dull, 0xe386b0716d24f19bull}, /* edge_cases.Retired */ \ + {0xb2fc3964ef56ab39ull, 0x73c325c8bc351236ull}, /* edge_cases.async_list */ \ + {0xb3403af521cf0006ull, 0x6abc30d48206a3baull}, /* edge_cases.Widget.close */ \ + {0xb794304a22a406b8ull, 0xf6fb98711c516105ull}, /* edge_cases.async_struct */ \ + {0xba3936106a49c47dull, 0xb7025f2fadd6288eull}, /* edge_cases.Vault.peek */ \ + {0xc155b0c3cfb164f4ull, 0xc777ae57d92ebc47ull}, /* edge_cases.noop */ \ + {0xcdc18607777019ddull, 0xb81a31e5c20aa7a8ull}, /* edge_cases.inspect_widget */ \ + {0xd164a82aeb64b8ccull, 0xded02c68005c4737ull}, /* edge_cases.nested_optional */ \ + {0xd978ccad2468038full, 0x30c39c65d6235153ull}, /* edge_cases.Mode */ \ + {0xdbe9d947431c9124ull, 0x39537fb848fdf980ull}, /* edge_cases.BatchListener */ \ + {0xeae8b7862c08bb07ull, 0xfe12eeaa830452eaull}, /* edge_cases.Keyword */ \ + {0xf1e155a05ae262b8ull, 0xb575eb818a423d42ull}, /* edge_cases.async_enum */ \ + {0xf22e210bdbce7343ull, 0x2a5e0f801bc8c768ull}, /* edge_cases.Widget.dispose */ \ + {0xf3c7cf675563ee6cull, 0x0ecffeef399facc8ull}, /* edge_cases.Vault.probe */ \ + {0xfd34bc67acac379eull, 0x576caa14f3c0901aull}, /* edge_cases.Widget.fetch */ \ + {0xff2e9d2bff941786ull, 0x7e96764ef1f5fbaaull}, /* edge_cases.string_stream */ \ +} +/* The id of the first expected entry the loaded library lacks or + declares differently, or 0 when these bindings match it. */ +static inline uint64_t edge_cases_edge_cases_contract_check(void) { + static const edge_cases_contract_entry expected[] = EDGE_CASES_EDGE_CASES_CONTRACT; + size_t len = 0; + const edge_cases_contract_entry* table = edge_cases_edge_cases_contract(&len); + for (size_t i = 0; i < EDGE_CASES_EDGE_CASES_CONTRACT_LEN; i++) { + size_t lo = 0, hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) lo = mid + 1; else hi = mid; + } + if (lo == len || table[lo].id != expected[i].id || table[lo].hash != expected[i].hash) { + return expected[i].id; + } + } + return 0; +} /* * Strings, bytes, and value buffers cross as (ptr, len) pairs. A @@ -129,9 +209,12 @@ typedef struct edge_cases_edge_cases_ModeStreamIterator edge_cases_edge_cases_Mo /** * Callback interface carrying composite payloads and objects * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. + * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable, with `size` set to `sizeof` the vtable and `flags` to 0 (the producer rejects a smaller vtable). The producer may call any entry from any thread until it calls `free(ctx)` exactly once, also from any thread. A method reports failure with `error_set(out_err, ...)`; a string, bytes, or buffer return is a run the consumer allocates with `alloc` and writes to `out_ptr` and `out_len`, which the producer adopts. */ typedef struct edge_cases_edge_cases_BatchListener_vtable { + uint32_t size; + uint32_t flags; + void (*free)(void* ctx); /** Composite payloads cross as value buffers */ void (*on_batch)(void* ctx, const uint8_t* items_ptr, size_t items_len, const uint8_t* raw_ptr, size_t raw_len, const uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err); /** Objects cross as strong references the consumer must release */ @@ -141,17 +224,18 @@ typedef struct edge_cases_edge_cases_BatchListener_vtable { bool (*should_continue)(void* ctx, int32_t class_, edge_cases_error* out_err); /** Enum return travels back as its integer discriminant */ edge_cases_edge_cases_Mode (*pick_mode)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); } edge_cases_edge_cases_BatchListener_vtable; /** * Callback interface whose name and methods are keywords * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. + * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable, with `size` set to `sizeof` the vtable and `flags` to 0 (the producer rejects a smaller vtable). The producer may call any entry from any thread until it calls `free(ctx)` exactly once, also from any thread. A method reports failure with `error_set(out_err, ...)`; a string, bytes, or buffer return is a run the consumer allocates with `alloc` and writes to `out_ptr` and `out_len`, which the producer adopts. */ typedef struct edge_cases_edge_cases_Keyword_vtable { + uint32_t size; + uint32_t flags; + void (*free)(void* ctx); void (*delete_)(void* ctx, int32_t self, edge_cases_error* out_err); int32_t (*import)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); } edge_cases_edge_cases_Keyword_vtable; typedef void (*edge_cases_edge_cases_async_widget_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result); typedef void (*edge_cases_edge_cases_async_void_callback)(void* context, edge_cases_error* err); @@ -172,6 +256,10 @@ EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_rename(const edge_cas EDGE_CASES_API void edge_cases_edge_cases_Widget_fetch(const edge_cases_edge_cases_Widget* self, int64_t id, edge_cases_edge_cases_Widget_fetch_callback callback, void* context); EDGE_CASES_DEPRECATED("Use rename instead") EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_old_name(const edge_cases_edge_cases_Widget* self, size_t* out_len, edge_cases_error* out_err); +/** Spelled like the release method of several wrappers */ +EDGE_CASES_API bool edge_cases_edge_cases_Widget_close(const edge_cases_edge_cases_Widget* self, edge_cases_error* out_err); +/** Spelled like the release method of the Dart and .NET wrappers */ +EDGE_CASES_API void edge_cases_edge_cases_Widget_dispose(const edge_cases_edge_cases_Widget* self, bool force, edge_cases_error* out_err); /** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Widget_clone(const edge_cases_edge_cases_Widget* self); /** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ @@ -182,7 +270,11 @@ EDGE_CASES_API void edge_cases_edge_cases_Widget_destroy(edge_cases_edge_cases_W EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_open(const uint8_t* path_ptr, size_t path_len, edge_cases_error* out_err); /** Throwing static */ EDGE_CASES_API bool edge_cases_edge_cases_Vault_probe(edge_cases_error* out_err); +/** Static spelled like an inherited object member */ +EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Vault_to_string(size_t* out_len, edge_cases_error* out_err); EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Vault_peek(const edge_cases_edge_cases_Vault* self, edge_cases_error* out_err); +/** Spelled like the handle accessor of several wrappers */ +EDGE_CASES_API int64_t edge_cases_edge_cases_Vault_handle(const edge_cases_edge_cases_Vault* self, edge_cases_error* out_err); /** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_clone(const edge_cases_edge_cases_Vault* self); /** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ diff --git a/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_buffer_h.snap b/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_buffer_h.snap index 0737cc41..390ef0a9 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_buffer_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_buffer_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef KITCHEN_SINK_BUFFER_H #define KITCHEN_SINK_BUFFER_H @@ -386,13 +386,13 @@ struct kitchen_sink_map_string_string { size_t len; }; -/** `[kitchen.Gadget]` */ +/** `[Gadget]` */ struct kitchen_sink_list_kitchen_Gadget { kitchen_sink_kitchen_Gadget** items; size_t len; }; -/** `[kitchen.Item]` */ +/** `[Item]` */ struct kitchen_sink_list_kitchen_Item { kitchen_sink_kitchen_Item* items; size_t len; diff --git a/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_h.snap b/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_h.snap index 14faeffb..ae1a6fe2 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_kitchen_sink__kitchen_sink_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef KITCHEN_SINK_H #define KITCHEN_SINK_H @@ -45,7 +45,7 @@ extern "C" { /* The C ABI revision this header was generated against. The producer exports kitchen_sink_abi_version() so a consumer can refuse to load a library built for a different revision. */ -#define KITCHEN_SINK_ABI_VERSION 3u +#define KITCHEN_SINK_ABI_VERSION 4u KITCHEN_SINK_API uint32_t kitchen_sink_abi_version(void); /* Error slot written by every fallible call. `message` is NUL-terminated @@ -63,9 +63,13 @@ typedef struct kitchen_sink_error { } kitchen_sink_error; /* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ + interface implementations call this to report a failure without + allocating with a foreign allocator. */ KITCHEN_SINK_API void kitchen_sink_error_set(kitchen_sink_error* err, int32_t code, const char* message); +/* Replace `err`'s payload with a producer-owned copy of `len` bytes: the + fields of a declared error code, which a callback method that throws + attaches after kitchen_sink_error_set. */ +KITCHEN_SINK_API void kitchen_sink_error_set_payload(kitchen_sink_error* err, const uint8_t* ptr, size_t len); KITCHEN_SINK_API void kitchen_sink_error_clear(kitchen_sink_error* err); /* Async completions receive a heap-boxed error the consumer owns; @@ -73,7 +77,12 @@ KITCHEN_SINK_API void kitchen_sink_error_clear(kitchen_sink_error* err); NULL is a no-op. */ KITCHEN_SINK_API void kitchen_sink_error_free(kitchen_sink_error* err); -/* Release a string, bytes, or value buffer the producer returned. */ +/* Allocate a zero-filled run of `len` bytes (NULL for 0). A callback + method returns a string, bytes, or buffer as such a run, which the + producer adopts; any other run is released with kitchen_sink_free_bytes. */ +KITCHEN_SINK_API uint8_t* kitchen_sink_alloc(size_t len); +/* Release a string, bytes, or value buffer the producer returned, or a + run from kitchen_sink_alloc, with its exact length. NULL or 0 is a no-op. */ KITCHEN_SINK_API void kitchen_sink_free_bytes(uint8_t* ptr, size_t len); /* Cancellation for `cancellable` async functions. A token starts with @@ -86,23 +95,105 @@ KITCHEN_SINK_API void kitchen_sink_cancel_token_cancel(kitchen_sink_cancel_token KITCHEN_SINK_API bool kitchen_sink_cancel_token_is_cancelled(const kitchen_sink_cancel_token* token); KITCHEN_SINK_API void kitchen_sink_cancel_token_destroy(kitchen_sink_cancel_token* token); -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -KITCHEN_SINK_API uint8_t* kitchen_sink_alloc(uint32_t size); -KITCHEN_SINK_API void kitchen_sink_dealloc(uint8_t* ptr, uint32_t size); - /* Live-allocation counters for leak checks in tests: 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ + Kind -1 is 1 when the producer counts at all (its `leak-check` + feature) and 0 when every counter reads 0 regardless. */ KITCHEN_SINK_API uint64_t kitchen_sink_debug_live(int32_t kind); -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define KITCHEN_SINK_SHARED_CHECKSUM 0x42c4ce2c8d0af052ull -KITCHEN_SINK_API uint64_t kitchen_sink_shared_checksum(void); -#define KITCHEN_SINK_KITCHEN_CHECKSUM 0xd83da75b24b545ffull -KITCHEN_SINK_API uint64_t kitchen_sink_kitchen_checksum(void); +/* One declaration's fingerprint in a module's contract table: FNV-1a 64 + of its dotted path and of its canonical signature. */ +typedef struct kitchen_sink_contract_entry { + uint64_t id; + uint64_t hash; +} kitchen_sink_contract_entry; + +/* The contract of module `shared`: one entry per declaration, sorted by + id. The producer's table comes from the function below; a consumer + checks each entry it was generated with against it once, before any + other call (producer entries it doesn't know are fine). */ +KITCHEN_SINK_API const kitchen_sink_contract_entry* kitchen_sink_shared_contract(size_t* out_len); +#define KITCHEN_SINK_SHARED_CONTRACT_LEN 2 +#define KITCHEN_SINK_SHARED_CONTRACT { \ + {0x709e234caa962446ull, 0x3894f2ca452e9f98ull}, /* shared.ping */ \ + {0xcfb306fa5801ee5bull, 0x1cbd9e10983cac2cull}, /* shared.Token */ \ +} +/* The id of the first expected entry the loaded library lacks or + declares differently, or 0 when these bindings match it. */ +static inline uint64_t kitchen_sink_shared_contract_check(void) { + static const kitchen_sink_contract_entry expected[] = KITCHEN_SINK_SHARED_CONTRACT; + size_t len = 0; + const kitchen_sink_contract_entry* table = kitchen_sink_shared_contract(&len); + for (size_t i = 0; i < KITCHEN_SINK_SHARED_CONTRACT_LEN; i++) { + size_t lo = 0, hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) lo = mid + 1; else hi = mid; + } + if (lo == len || table[lo].id != expected[i].id || table[lo].hash != expected[i].hash) { + return expected[i].id; + } + } + return 0; +} + +/* The contract of module `kitchen`: one entry per declaration, sorted by + id. The producer's table comes from the function below; a consumer + checks each entry it was generated with against it once, before any + other call (producer entries it doesn't know are fine). */ +KITCHEN_SINK_API const kitchen_sink_contract_entry* kitchen_sink_kitchen_contract(size_t* out_len); +#define KITCHEN_SINK_KITCHEN_CONTRACT_LEN 31 +#define KITCHEN_SINK_KITCHEN_CONTRACT { \ + {0x01d4f09bbddc986full, 0x29eb88fecc66544dull}, /* kitchen.Gadget.new */ \ + {0x10472d8fe748c2eeull, 0xb265832b6a759fc4ull}, /* kitchen.maybe_gadget */ \ + {0x11368eec07a74078ull, 0xfa049f78b77e146aull}, /* kitchen.gadgets */ \ + {0x137f08a29cde6021ull, 0xda8de7aeb2dbb8a3ull}, /* kitchen.stream_gadgets */ \ + {0x2111b54980826dd7ull, 0x452783f3e6ca6e1dull}, /* kitchen.do_cancellable */ \ + {0x29dddbc60bf8d7dbull, 0x452a3e573d57f6dbull}, /* kitchen.Priority */ \ + {0x2b227003b74a0d16ull, 0x12332957470e9782ull}, /* kitchen.maybe_subscribe */ \ + {0x2dd66cb0064f0494ull, 0x60197363fefd92ddull}, /* kitchen.Item */ \ + {0x33c265bbf079b0e9ull, 0xc3dcd0927e6c1debull}, /* kitchen.u32_id */ \ + {0x33e6381d01660dfeull, 0x077c63c31fd7f90aull}, /* kitchen.KitchenErrors */ \ + {0x3f5fc99d73504b76ull, 0x2df88d710f1ea10eull}, /* kitchen.Gadget.poke */ \ + {0x4c5764cf109528d1ull, 0x2a65529af4338a58ull}, /* kitchen.maybe_item */ \ + {0x4de17dfb93fd494dull, 0xba0ad0dd52ed74afull}, /* kitchen.i32_id */ \ + {0x504e95b7f7e65570ull, 0x01b3e8e0c9a10e23ull}, /* kitchen.stream_items */ \ + {0x68cdb9a4bd3e3dd5ull, 0xc678b9164348d497ull}, /* kitchen.bool_id */ \ + {0x69e7cd703d16eb85ull, 0xf3b28dd22ae9e689ull}, /* kitchen.do_async */ \ + {0x70865079ce41be82ull, 0xbe3b69464672af46ull}, /* kitchen.Gadget.describe */ \ + {0x71c5a1116cb7c49aull, 0xcba1837108a89b84ull}, /* kitchen.echo_bytes */ \ + {0x735ee1e0049ee91aull, 0xca74320bc53eddabull}, /* kitchen.echo_string */ \ + {0x76d7fc58f4e41d84ull, 0x7373a0e47d30e690ull}, /* kitchen.i64_id */ \ + {0x79abfe1aa960fd6aull, 0xbca7a9e5eb26f3a6ull}, /* kitchen.nested.hello */ \ + {0x7d3dd7d5cf6749e3ull, 0xa192f5a7298adb39ull}, /* kitchen.new_op */ \ + {0x89a5e664bc25c67bull, 0x5764a95e81ecfd31ull}, /* kitchen.f64_id */ \ + {0x8cfdbdcecac5f7f1ull, 0x07e272f09c41fffcull}, /* kitchen.Gadget.version */ \ + {0x974b9f3e823c1738ull, 0xb81b91e975695f27ull}, /* kitchen.cross_module_token */ \ + {0xaca0655fac15d230ull, 0x11c4d2b28abe07bbull}, /* kitchen.list_items */ \ + {0xc883b4afae20543eull, 0xc40560e5411ff392ull}, /* kitchen.legacy_op */ \ + {0xf2f7753e6e1dffc2ull, 0xc9047a95bdf49e93ull}, /* kitchen.get_attrs */ \ + {0xfae25d3391922eadull, 0x1c7f95739f3210b8ull}, /* kitchen.subscribe */ \ + {0xfd0059b05cb2fcfbull, 0x248bf8404a5823b4ull}, /* kitchen.Gadget */ \ + {0xfed9c551fb9fd2d6ull, 0x59aec980ef3f3291ull}, /* kitchen.ReadyListener */ \ +} +/* The id of the first expected entry the loaded library lacks or + declares differently, or 0 when these bindings match it. */ +static inline uint64_t kitchen_sink_kitchen_contract_check(void) { + static const kitchen_sink_contract_entry expected[] = KITCHEN_SINK_KITCHEN_CONTRACT; + size_t len = 0; + const kitchen_sink_contract_entry* table = kitchen_sink_kitchen_contract(&len); + for (size_t i = 0; i < KITCHEN_SINK_KITCHEN_CONTRACT_LEN; i++) { + size_t lo = 0, hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) lo = mid + 1; else hi = mid; + } + if (lo == len || table[lo].id != expected[i].id || table[lo].hash != expected[i].hash) { + return expected[i].id; + } + } + return 0; +} /* * Strings, bytes, and value buffers cross as (ptr, len) pairs. A @@ -134,14 +225,26 @@ typedef struct kitchen_sink_kitchen_StreamItemsIterator kitchen_sink_kitchen_Str /** * Consumer-implemented observer of ready items * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. + * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable, with `size` set to `sizeof` the vtable and `flags` to 0 (the producer rejects a smaller vtable). The producer may call any entry from any thread until it calls `free(ctx)` exactly once, also from any thread. A method reports failure with `error_set(out_err, ...)`; a string, bytes, or buffer return is a run the consumer allocates with `alloc` and writes to `out_ptr` and `out_len`, which the producer adopts. */ typedef struct kitchen_sink_kitchen_ReadyListener_vtable { + uint32_t size; + uint32_t flags; + void (*free)(void* ctx); /** Fires when an item is ready */ void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); /** Receives the item itself and says whether to keep listening */ bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); + /** + * A display label, failing with a kitchen error + * + * May fail with a `kitchen_sink_kitchen_KitchenErrors` code (its fields attached with `kitchen_sink_error_set_payload`). + */ + void (*label)(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); + /** The most recent item the listener kept, if any */ + void (*latest)(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); + /** The listener's favorite gadget */ + kitchen_sink_kitchen_Gadget* (*favorite)(void* ctx, kitchen_sink_error* out_err); } kitchen_sink_kitchen_ReadyListener_vtable; typedef void (*kitchen_sink_kitchen_do_async_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); typedef void (*kitchen_sink_kitchen_do_cancellable_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); @@ -175,6 +278,8 @@ KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_string(const uint8_t* KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_bytes(const uint8_t* b_ptr, size_t b_len, size_t* out_len, kitchen_sink_error* out_err); /** Register a listener, returning how many are attached */ KITCHEN_SINK_API int32_t kitchen_sink_kitchen_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); +/** Register a listener if one is given */ +KITCHEN_SINK_API bool kitchen_sink_kitchen_maybe_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); /** Optional object in both directions */ KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_maybe_gadget(const kitchen_sink_kitchen_Gadget* fallback, kitchen_sink_error* out_err); /** A list of objects */ diff --git a/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_buffer_h.snap b/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_buffer_h.snap index f541207a..320c6526 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_buffer_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_buffer_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef NESTED_MODULES_BUFFER_H #define NESTED_MODULES_BUFFER_H diff --git a/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_h.snap b/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_h.snap index 27c97f2b..49955709 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_nested_modules__nested_modules_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef NESTED_MODULES_H #define NESTED_MODULES_H @@ -45,7 +45,7 @@ extern "C" { /* The C ABI revision this header was generated against. The producer exports nested_modules_abi_version() so a consumer can refuse to load a library built for a different revision. */ -#define NESTED_MODULES_ABI_VERSION 3u +#define NESTED_MODULES_ABI_VERSION 4u NESTED_MODULES_API uint32_t nested_modules_abi_version(void); /* Error slot written by every fallible call. `message` is NUL-terminated @@ -63,9 +63,13 @@ typedef struct nested_modules_error { } nested_modules_error; /* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ + interface implementations call this to report a failure without + allocating with a foreign allocator. */ NESTED_MODULES_API void nested_modules_error_set(nested_modules_error* err, int32_t code, const char* message); +/* Replace `err`'s payload with a producer-owned copy of `len` bytes: the + fields of a declared error code, which a callback method that throws + attaches after nested_modules_error_set. */ +NESTED_MODULES_API void nested_modules_error_set_payload(nested_modules_error* err, const uint8_t* ptr, size_t len); NESTED_MODULES_API void nested_modules_error_clear(nested_modules_error* err); /* Async completions receive a heap-boxed error the consumer owns; @@ -73,7 +77,12 @@ NESTED_MODULES_API void nested_modules_error_clear(nested_modules_error* err); NULL is a no-op. */ NESTED_MODULES_API void nested_modules_error_free(nested_modules_error* err); -/* Release a string, bytes, or value buffer the producer returned. */ +/* Allocate a zero-filled run of `len` bytes (NULL for 0). A callback + method returns a string, bytes, or buffer as such a run, which the + producer adopts; any other run is released with nested_modules_free_bytes. */ +NESTED_MODULES_API uint8_t* nested_modules_alloc(size_t len); +/* Release a string, bytes, or value buffer the producer returned, or a + run from nested_modules_alloc, with its exact length. NULL or 0 is a no-op. */ NESTED_MODULES_API void nested_modules_free_bytes(uint8_t* ptr, size_t len); /* Cancellation for `cancellable` async functions. A token starts with @@ -86,21 +95,54 @@ NESTED_MODULES_API void nested_modules_cancel_token_cancel(nested_modules_cancel NESTED_MODULES_API bool nested_modules_cancel_token_is_cancelled(const nested_modules_cancel_token* token); NESTED_MODULES_API void nested_modules_cancel_token_destroy(nested_modules_cancel_token* token); -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -NESTED_MODULES_API uint8_t* nested_modules_alloc(uint32_t size); -NESTED_MODULES_API void nested_modules_dealloc(uint8_t* ptr, uint32_t size); - /* Live-allocation counters for leak checks in tests: 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ + Kind -1 is 1 when the producer counts at all (its `leak-check` + feature) and 0 when every counter reads 0 regardless. */ NESTED_MODULES_API uint64_t nested_modules_debug_live(int32_t kind); -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define NESTED_MODULES_GRAPHICS_CHECKSUM 0x44d31b5b10fdb90dull -NESTED_MODULES_API uint64_t nested_modules_graphics_checksum(void); +/* One declaration's fingerprint in a module's contract table: FNV-1a 64 + of its dotted path and of its canonical signature. */ +typedef struct nested_modules_contract_entry { + uint64_t id; + uint64_t hash; +} nested_modules_contract_entry; + +/* The contract of module `graphics`: one entry per declaration, sorted by + id. The producer's table comes from the function below; a consumer + checks each entry it was generated with against it once, before any + other call (producer entries it doesn't know are fine). */ +NESTED_MODULES_API const nested_modules_contract_entry* nested_modules_graphics_contract(size_t* out_len); +#define NESTED_MODULES_GRAPHICS_CONTRACT_LEN 9 +#define NESTED_MODULES_GRAPHICS_CONTRACT { \ + {0x2c29eef9327a6bd4ull, 0x12983d5271651389ull}, /* graphics.Unit */ \ + {0x304088bbc28f0015ull, 0x97e6e809913be6b6ull}, /* graphics.new_canvas */ \ + {0x49f8837532df9ac6ull, 0x32e5b27e81ee0957ull}, /* graphics.Canvas */ \ + {0x55717fa027336b47ull, 0xb2bd3a6bdcce48c8ull}, /* graphics.shapes.describe */ \ + {0x7626b91b8f1f53cdull, 0x867bbf7cf0f41c74ull}, /* graphics.shapes.geometry.compute */ \ + {0xb3efcb50d9a61ab7ull, 0x3cb0207dfad2a149ull}, /* graphics.GraphicsError */ \ + {0xc62b16a9591b744bull, 0x7895610c8487eb1bull}, /* graphics.shapes.Shape */ \ + {0xe13001f4983ccd35ull, 0x16e7de070b4a4cfcull}, /* graphics.shapes.geometry.Detail */ \ + {0xf3f658bcfc5dba62ull, 0x2665ccb47356e746ull}, /* graphics.shapes.make_shape */ \ +} +/* The id of the first expected entry the loaded library lacks or + declares differently, or 0 when these bindings match it. */ +static inline uint64_t nested_modules_graphics_contract_check(void) { + static const nested_modules_contract_entry expected[] = NESTED_MODULES_GRAPHICS_CONTRACT; + size_t len = 0; + const nested_modules_contract_entry* table = nested_modules_graphics_contract(&len); + for (size_t i = 0; i < NESTED_MODULES_GRAPHICS_CONTRACT_LEN; i++) { + size_t lo = 0, hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) lo = mid + 1; else hi = mid; + } + if (lo == len || table[lo].id != expected[i].id || table[lo].hash != expected[i].hash) { + return expected[i].id; + } + } + return 0; +} /* * Strings, bytes, and value buffers cross as (ptr, len) pairs. A diff --git a/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_buffer_h.snap b/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_buffer_h.snap index 25549e50..b9aca031 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_buffer_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_buffer_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef SHAPES_BUFFER_H #define SHAPES_BUFFER_H diff --git a/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_h.snap b/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_h.snap index df20c3f5..968d33c1 100644 --- a/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_h.snap +++ b/crates/weaveffi-cli/tests/snapshots/c_shapes__shapes_h.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef SHAPES_H #define SHAPES_H @@ -45,7 +45,7 @@ extern "C" { /* The C ABI revision this header was generated against. The producer exports shapes_abi_version() so a consumer can refuse to load a library built for a different revision. */ -#define SHAPES_ABI_VERSION 3u +#define SHAPES_ABI_VERSION 4u SHAPES_API uint32_t shapes_abi_version(void); /* Error slot written by every fallible call. `message` is NUL-terminated @@ -63,9 +63,13 @@ typedef struct shapes_error { } shapes_error; /* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ + interface implementations call this to report a failure without + allocating with a foreign allocator. */ SHAPES_API void shapes_error_set(shapes_error* err, int32_t code, const char* message); +/* Replace `err`'s payload with a producer-owned copy of `len` bytes: the + fields of a declared error code, which a callback method that throws + attaches after shapes_error_set. */ +SHAPES_API void shapes_error_set_payload(shapes_error* err, const uint8_t* ptr, size_t len); SHAPES_API void shapes_error_clear(shapes_error* err); /* Async completions receive a heap-boxed error the consumer owns; @@ -73,7 +77,12 @@ SHAPES_API void shapes_error_clear(shapes_error* err); NULL is a no-op. */ SHAPES_API void shapes_error_free(shapes_error* err); -/* Release a string, bytes, or value buffer the producer returned. */ +/* Allocate a zero-filled run of `len` bytes (NULL for 0). A callback + method returns a string, bytes, or buffer as such a run, which the + producer adopts; any other run is released with shapes_free_bytes. */ +SHAPES_API uint8_t* shapes_alloc(size_t len); +/* Release a string, bytes, or value buffer the producer returned, or a + run from shapes_alloc, with its exact length. NULL or 0 is a no-op. */ SHAPES_API void shapes_free_bytes(uint8_t* ptr, size_t len); /* Cancellation for `cancellable` async functions. A token starts with @@ -86,21 +95,52 @@ SHAPES_API void shapes_cancel_token_cancel(shapes_cancel_token* token); SHAPES_API bool shapes_cancel_token_is_cancelled(const shapes_cancel_token* token); SHAPES_API void shapes_cancel_token_destroy(shapes_cancel_token* token); -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -SHAPES_API uint8_t* shapes_alloc(uint32_t size); -SHAPES_API void shapes_dealloc(uint8_t* ptr, uint32_t size); - /* Live-allocation counters for leak checks in tests: 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ + Kind -1 is 1 when the producer counts at all (its `leak-check` + feature) and 0 when every counter reads 0 regardless. */ SHAPES_API uint64_t shapes_debug_live(int32_t kind); -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define SHAPES_SHAPES_CHECKSUM 0x06acf433ffa2a0f0ull -SHAPES_API uint64_t shapes_shapes_checksum(void); +/* One declaration's fingerprint in a module's contract table: FNV-1a 64 + of its dotted path and of its canonical signature. */ +typedef struct shapes_contract_entry { + uint64_t id; + uint64_t hash; +} shapes_contract_entry; + +/* The contract of module `shapes`: one entry per declaration, sorted by + id. The producer's table comes from the function below; a consumer + checks each entry it was generated with against it once, before any + other call (producer entries it doesn't know are fine). */ +SHAPES_API const shapes_contract_entry* shapes_shapes_contract(size_t* out_len); +#define SHAPES_SHAPES_CONTRACT_LEN 7 +#define SHAPES_SHAPES_CONTRACT { \ + {0x0d7b3eec49243e7aull, 0x5f232311808e1b1full}, /* shapes.Channel */ \ + {0x183fa8198100860eull, 0xe40c6993cb31d359ull}, /* shapes.describe */ \ + {0x2d1395eb837b848eull, 0x42202a0c12f3a16eull}, /* shapes.sum_bytes */ \ + {0x7f9a5b87ba09b86bull, 0xc5b32fb0efbbaa17ull}, /* shapes.scale */ \ + {0xc8edd779240fda17ull, 0x68e1be78fddb20e4ull}, /* shapes.Numerics */ \ + {0xd3d3881b2fd18a62ull, 0xf24d0eb76a99d918ull}, /* shapes.make_numerics */ \ + {0xf010844afff5ac3cull, 0x50de28ae9b41ff15ull}, /* shapes.Shape */ \ +} +/* The id of the first expected entry the loaded library lacks or + declares differently, or 0 when these bindings match it. */ +static inline uint64_t shapes_shapes_contract_check(void) { + static const shapes_contract_entry expected[] = SHAPES_SHAPES_CONTRACT; + size_t len = 0; + const shapes_contract_entry* table = shapes_shapes_contract(&len); + for (size_t i = 0; i < SHAPES_SHAPES_CONTRACT_LEN; i++) { + size_t lo = 0, hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) lo = mid + 1; else hi = mid; + } + if (lo == len || table[lo].id != expected[i].id || table[lo].hash != expected[i].hash) { + return expected[i].id; + } + } + return 0; +} /* * Strings, bytes, and value buffers cross as (ptr, len) pairs. A diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__CMakeLists_txt.snap deleted file mode 100644 index b817eb9d..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__CMakeLists_txt.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.14) -project(docs_everywhere_cpp VERSION 0.1.0 LANGUAGES CXX) - -# The producer library to link: a full path from the DOCS_EVERYWHERE_LIBRARY cache or -# environment variable, or else the library name `docs_everywhere` (a CMake target -# of that name, or libdocs_everywhere on the linker search path). -set(DOCS_EVERYWHERE_LIBRARY "$ENV{DOCS_EVERYWHERE_LIBRARY}" CACHE FILEPATH "Path to the docs_everywhere native library") - -add_library(docs_everywhere_cpp INTERFACE) -add_library(docs_everywhere::cpp ALIAS docs_everywhere_cpp) -target_include_directories(docs_everywhere_cpp INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}) -target_compile_features(docs_everywhere_cpp INTERFACE cxx_std_17) -if(DOCS_EVERYWHERE_LIBRARY) - target_link_libraries(docs_everywhere_cpp INTERFACE "${DOCS_EVERYWHERE_LIBRARY}") -else() - target_link_libraries(docs_everywhere_cpp INTERFACE docs_everywhere) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__README_md.snap deleted file mode 100644 index e1d8ad02..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__README_md.snap +++ /dev/null @@ -1,43 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# docs_everywhere C++ bindings - -A header-only C++17 wrapper over the `docs_everywhere` C ABI. `docs_everywhere.hpp` -includes the C header beside it; every declaration lives in -`namespace docs_everywhere`. - -## Prerequisites - -- CMake 3.14 or later -- A C++17 compiler -- The `docs_everywhere` native library (`libdocs_everywhere.so`, `libdocs_everywhere.dylib`, or - `docs_everywhere.dll`) - -## Use with CMake - -```cmake -add_subdirectory(path/to/cpp) -add_executable(app main.cpp) -target_link_libraries(app PRIVATE docs_everywhere::cpp) -``` - -`docs_everywhere::cpp` adds this directory to the include path, requires -C++17, and links the native library: the file named by the `DOCS_EVERYWHERE_LIBRARY` -CMake cache variable or environment variable when set, or else -`docs_everywhere` by name. - -```cpp -#include "docs_everywhere.hpp" -``` - -The first free function, constructor, or static member you call checks -that the linked library matches these headers and throws -`docs_everywhere::LoadError` if it doesn't; call `docs_everywhere::check_library()` -at startup to check eagerly. - - diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_h.snap b/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_h.snap deleted file mode 100644 index 31ab85e2..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_h.snap +++ /dev/null @@ -1,196 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef DOCS_EVERYWHERE_H -#define DOCS_EVERYWHERE_H - -#include -#include -#include - -#ifndef DOCS_EVERYWHERE_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef DOCS_EVERYWHERE_BUILD -# define DOCS_EVERYWHERE_API __declspec(dllexport) -# else -# define DOCS_EVERYWHERE_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define DOCS_EVERYWHERE_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define DOCS_EVERYWHERE_API __attribute__((visibility("default"))) -# else -# define DOCS_EVERYWHERE_API -# endif -#endif - -#ifndef DOCS_EVERYWHERE_DEPRECATED -# if defined(_MSC_VER) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define DOCS_EVERYWHERE_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports docs_everywhere_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define DOCS_EVERYWHERE_ABI_VERSION 3u -DOCS_EVERYWHERE_API uint32_t docs_everywhere_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by docs_everywhere_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct docs_everywhere_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} docs_everywhere_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_set(docs_everywhere_error* err, int32_t code, const char* message); -DOCS_EVERYWHERE_API void docs_everywhere_error_clear(docs_everywhere_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - docs_everywhere_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_free(docs_everywhere_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -DOCS_EVERYWHERE_API void docs_everywhere_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct docs_everywhere_cancel_token docs_everywhere_cancel_token; -DOCS_EVERYWHERE_API docs_everywhere_cancel_token* docs_everywhere_cancel_token_create(void); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_cancel(docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API bool docs_everywhere_cancel_token_is_cancelled(const docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_destroy(docs_everywhere_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -DOCS_EVERYWHERE_API uint8_t* docs_everywhere_alloc(uint32_t size); -DOCS_EVERYWHERE_API void docs_everywhere_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -DOCS_EVERYWHERE_API uint64_t docs_everywhere_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define DOCS_EVERYWHERE_DOCS_CHECKSUM 0x7950eec1e9cfc819ull -DOCS_EVERYWHERE_API uint64_t docs_everywhere_docs_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with docs_everywhere_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * docs_everywhere_buffer.h declares a C struct and codec for each. - */ - -/** - * Lifecycle status of a document. - * - * Used to drive UI badges and filters. - */ -typedef enum { - /** Document is being edited */ - docs_everywhere_docs_Status_Draft = 0, - /** Document is publicly visible */ - docs_everywhere_docs_Status_Published = 1, - /** Document is hidden from listings */ - docs_everywhere_docs_Status_Archived = 2 -} docs_everywhere_docs_Status; -/** Error codes reported by throwing functions in the `docs` module tree. */ -typedef enum { - /** Returned when no document matches the requested id */ - docs_everywhere_docs_DocsErrors_not_found = 1, - /** Raised when attempting to move into a forbidden state */ - docs_everywhere_docs_DocsErrors_invalid_status = 2 -} docs_everywhere_docs_DocsErrors; -typedef struct docs_everywhere_docs_Notebook docs_everywhere_docs_Notebook; -/** - * Consumer-implemented observer of document-saved notifications. - * - * Implementations must be safe to call from any thread. - * - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct docs_everywhere_docs_SavedListener_vtable { - /** Fires after a document has been persisted successfully */ - void (*on_saved)(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err); - void (*free)(void* ctx); -} docs_everywhere_docs_SavedListener_vtable; -typedef void (*docs_everywhere_docs_publish_async_callback)(void* context, docs_everywhere_error* err, bool result); - -// Module: docs -/** - * An ordered collection of documents. - * - * Notebooks own their documents and release them when destroyed. - */ -/** Create an empty notebook */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_new(docs_everywhere_error* out_err); -/** The largest number of documents one notebook will hold */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_max_documents(docs_everywhere_error* out_err); -/** Add a document, returning its position in the notebook */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_insert(const docs_everywhere_docs_Notebook* self, const uint8_t* document_ptr, size_t document_len, docs_everywhere_error* out_err); -/** Look up a document by identifier, failing when none matches */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_Notebook_find(const docs_everywhere_docs_Notebook* self, int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** The number of documents currently held */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_size(const docs_everywhere_docs_Notebook* self, docs_everywhere_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_clone(const docs_everywhere_docs_Notebook* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_Notebook_destroy(docs_everywhere_docs_Notebook* self); - -/** - * Create a brand new document. - * - * The document starts in `Draft` status; callers must publish - * it explicitly to make it visible. - */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_create_document(const uint8_t* title_ptr, size_t title_len, const uint8_t* tags_ptr, size_t tags_len, size_t* out_len, docs_everywhere_error* out_err); -/** Look up a document by identifier */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_get_document(int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** Publish a document in the background */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_publish_async(int64_t id, docs_everywhere_docs_publish_async_callback callback, void* context); -/** Register a listener for saved notifications */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_watch(void* listener_ctx, const docs_everywhere_docs_SavedListener_vtable* listener_vtable, docs_everywhere_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // DOCS_EVERYWHERE_H - -// END docs_everywhere.h diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_hpp.snap b/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_hpp.snap index 30368ff1..7e11b634 100644 --- a/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_hpp.snap +++ b/crates/weaveffi-cli/tests/snapshots/cpp_docs_everywhere__docs_everywhere_hpp.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #pragma once @@ -27,13 +27,13 @@ source: crates/weaveffi-cli/tests/snapshots.rs namespace docs_everywhere { /** - * Base exception for every error reported through the C ABI. + * An error a call that declares errors reports. * - * Positive codes are the declared error codes (see the typed domain - * exceptions); negative codes are runtime traps: -1 a generic producer - * error, -2 a producer panic, -3 a marshalling failure, -4 a - * callback-interface implementation that threw, and -5 a cancelled call - * (thrown as Cancelled). + * A positive code is a declared code of the module's error domain, thrown as + * that domain's typed subclass. A negative code is a runtime failure: -1 a + * generic producer error, -2 a producer panic, -3 a marshalling failure, and + * -4 a callback-interface implementation that failed. A cancelled async call + * (-5) throws Cancelled. */ class Error : public std::runtime_error { int32_t code_; @@ -46,7 +46,7 @@ public: int32_t code() const noexcept { return code_; } }; -/** Thrown by an async call whose CancelToken was cancelled (code -5). */ +/** Thrown by an async call that was cancelled through its CancelToken (code -5). */ class Cancelled : public Error { public: /** Builds a cancellation error carrying the producer's message. */ @@ -54,13 +54,39 @@ public: }; /** - * Thrown on first use when the linked docs_everywhere library was built for a - * different C ABI revision or a different API contract than this header. + * Thrown by a call that declares no errors when it fails anyway: a producer + * panic (-2), a marshalling failure (-3), a callback-interface failure the + * producer let through (-4), or another runtime failure (-1). These are bugs, + * so no signature declares them; `what()` names the code and the producer's + * message. */ -class LoadError : public Error { +class InternalError : public std::runtime_error { + int32_t code_; + std::string message_; + +public: + /** Builds an internal error carrying `code` and the producer's `message`. */ + InternalError(int32_t code, const std::string& message) + : std::runtime_error("docs_everywhere: runtime error " + std::to_string(code) + ": " + message), + code_(code), + message_(message) {} + + /** The runtime error code (negative). */ + int32_t code() const noexcept { return code_; } + + /** The producer's message. */ + const std::string& message() const noexcept { return message_; } +}; + +/** + * Thrown on first use when the linked docs_everywhere library implements a + * different C ABI revision than this header, or lacks or changed a + * declaration this header was generated with. + */ +class LoadError : public std::runtime_error { public: /** Builds a load error describing the mismatch. */ - explicit LoadError(const std::string& message) : Error(-1, message) {} + explicit LoadError(const std::string& message) : std::runtime_error(message) {} }; /** @@ -71,7 +97,7 @@ struct adopt_t { explicit adopt_t() = default; }; -/** The adopt_t tag value: `Gadget g(adopt, raw)`. */ +/** The adopt_t tag value: `Store s(adopt, raw)`. */ inline constexpr adopt_t adopt{}; /** @@ -137,10 +163,10 @@ public: namespace detail { -/** The exception for a runtime (non-domain) error code. */ -inline std::exception_ptr make_error(int32_t code, const std::string& message) { +/** The exception for a failed call that declares no errors: Cancelled for -5, else InternalError. */ +inline std::exception_ptr make_internal_error(int32_t code, const std::string& message) { if (code == -5) return std::make_exception_ptr(Cancelled(message)); - return std::make_exception_ptr(Error(code, message)); + return std::make_exception_ptr(InternalError(code, message)); } /** The message of a nonzero error slot. */ @@ -148,10 +174,10 @@ inline std::string error_message(const docs_everywhere_error& err) { return std::string(err.message != nullptr ? err.message : "unknown error"); } -/** Throws the generic Error (or Cancelled) if `err` carries a nonzero code. */ +/** Throws InternalError if `err` carries a nonzero code (a call that declares no errors). */ inline void check(docs_everywhere_error& err) { if (err.code == 0) return; - std::exception_ptr ex = make_error(err.code, error_message(err)); + std::exception_ptr ex = make_internal_error(err.code, error_message(err)); docs_everywhere_error_clear(&err); std::rethrow_exception(ex); } @@ -184,7 +210,7 @@ inline std::string_view borrow_string(const uint8_t* ptr, size_t len) noexcept { return std::string_view(reinterpret_cast(ptr), len); } -/** Views a borrowed byte run (null when empty) as an owned vector. */ +/** Copies a borrowed byte run (null when empty) into a vector. */ inline std::vector borrow_bytes(const uint8_t* ptr, size_t len) { if (ptr == nullptr || len == 0) return std::vector(); return std::vector(ptr, ptr + len); @@ -195,24 +221,101 @@ inline const uint8_t* utf8(std::string_view s) noexcept { return reinterpret_cast(s.data()); } +/** + * Hands `len` bytes from `data` to the producer through a callback method's + * out slots: copies them into a run from docs_everywhere_alloc, which the + * producer adopts and frees. + */ +inline void hand_over(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = nullptr; + if (len > 0) { + run = docs_everywhere_alloc(len); + if (run == nullptr) throw std::bad_alloc(); + std::memcpy(run, data, len); + } + *out_ptr = run; + *out_len = len; +} + +/** One declaration this header was generated with: its contract id and hash, and its dotted path. */ +struct ContractEntry { + /** FNV-1a 64 of the declaration's dotted path. */ + uint64_t id; + /** FNV-1a 64 of the declaration's canonical signature. */ + uint64_t hash; + /** The declaration's dotted path (`kv.Store.put`). */ + const char* path; +}; + +/** + * Why the contract table `contract` returns doesn't satisfy `expected`, + * naming the first declaration it lacks or declares differently, or an empty + * string when it has them all. Entries the producer has and `expected` + * doesn't are fine. + */ +template +std::string contract_mismatch(const docs_everywhere_contract_entry* (*contract)(size_t*), + const ContractEntry (&expected)[N]) { + size_t len = 0; + const docs_everywhere_contract_entry* table = contract(&len); + if (table == nullptr) len = 0; + for (const ContractEntry& want : expected) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < want.id) { + lo = mid + 1; + } else { + hi = mid; + } + } + if (lo == len || table[lo].id != want.id) { + return std::string(want.path) + " is missing from the library"; + } + if (table[lo].hash != want.hash) { + return std::string(want.path) + " changed since these bindings were generated"; + } + } + return std::string(); +} + +/** The declarations of module `docs` this header was generated with. */ +inline constexpr ContractEntry docs_contract[] = { + {0x00b2187fd75634ecull, 0x2263c6f787bf6081ull, "docs.publish_async"}, + {0x12fc242b9dc6444cull, 0x96405f484a642b31ull, "docs.Notebook.insert"}, + {0x2213822ed05cd5bdull, 0x26c5833392aa63ceull, "docs.Notebook.new"}, + {0x2bd762bcdaa207ffull, 0x1b0ebab2d04c49b0ull, "docs.SavedListener"}, + {0x2e086c45e9d617deull, 0x4d43bf32358c01ffull, "docs.Notebook.find"}, + {0x4817dc98197ad960ull, 0xf8017b5a7d58b2c7ull, "docs.Notebook.size"}, + {0x614de47452d5a55cull, 0xd9aae1472494ffefull, "docs.get_document"}, + {0x67c5cb9fc119bcd5ull, 0x08d5b25a1a22a7faull, "docs.Document"}, + {0x8d484bf2d78c9b70ull, 0xe9093cf96912194cull, "docs.Notebook.max_documents"}, + {0xaea61d6451691c67ull, 0xe4704128b3524b3dull, "docs.watch"}, + {0xd68d6ad1540cdec5ull, 0x01afeb40b0a8bfe1ull, "docs.Notebook"}, + {0xdb0601444a4760ccull, 0xe56c69629b444797ull, "docs.create_document"}, + {0xe12d6f4838347b3eull, 0xa9bab86646d84f51ull, "docs.Status"}, + {0xee2f8d25b3313b50ull, 0x9691232f35439a2bull, "docs.DocsErrors"}, +}; + } // namespace detail /** - * Verifies that the linked docs_everywhere library implements the C ABI revision - * and the module contracts this header was generated against, throwing + * Verifies that the linked docs_everywhere library implements C ABI revision + * 4 and every declaration this header was generated with, throwing * LoadError naming the first mismatch. The check runs once; every free * function, constructor, and static member calls this before its first * native call, so calling it at startup is optional. */ inline void check_library() { - static const std::string failure = [] { + static const std::string failure = []() -> std::string { uint32_t abi = docs_everywhere_abi_version(); if (abi != DOCS_EVERYWHERE_ABI_VERSION) { return "docs_everywhere: C ABI revision mismatch (header " + std::to_string(DOCS_EVERYWHERE_ABI_VERSION) + ", library " + std::to_string(abi) + ")"; } - if (docs_everywhere_docs_checksum() != 0x7950eec1e9cfc819ull) { - return std::string("docs_everywhere: module 'docs' does not match the linked library (contract checksum mismatch); regenerate the bindings for this build"); + if (std::string why = detail::contract_mismatch(docs_everywhere_docs_contract, detail::docs_contract); !why.empty()) { + return "docs_everywhere: " + why; } return std::string(); }(); @@ -266,7 +369,7 @@ public: /** Writes a byte length or an element count as a u32. */ void write_len(size_t n) { - if (static_cast(n) > UINT32_MAX) throw Error(-3, "value buffer length exceeds u32"); + if (static_cast(n) > UINT32_MAX) throw std::length_error("value buffer length exceeds u32"); append_le(static_cast(n)); } @@ -282,24 +385,29 @@ public: /** Writes an optional's presence flag: 0 absent, 1 present. */ void write_option_flag(bool present) { buf_.push_back(present ? 1 : 0); } + + /** + * Writes an object token: a fresh strong reference from the wrapper's + * `_clone`, which the reader adopts while `v` keeps its own. + */ + template + void write_object(const T& v) { + write_u64(static_cast(reinterpret_cast(v.clone_handle()))); + } }; /** * Decodes values from the value-buffer wire format. A malformed buffer is a - * producer/consumer contract violation (both sides are generated from one - * API), so every decode failure throws the generic Error with the - * marshalling code -3. Object tokens adopted before a failure live in RAII - * wrappers that release them on unwind. + * producer bug (both sides are generated from one API), so every decode + * failure throws InternalError with the marshalling code -3. Object tokens + * adopted before a failure live in RAII wrappers that release them on + * unwind. */ class BufferReader { const uint8_t* data_; size_t len_; size_t pos_; - [[noreturn]] static void fail(const char* what) { - throw Error(-3, std::string("malformed value buffer: ") + what); - } - void require(size_t n, const char* what) const { if (len_ - pos_ < n) fail(what); } @@ -316,6 +424,12 @@ class BufferReader { } public: + /** Throws the marshalling failure (-3) for a malformed buffer. */ + [[noreturn]] static void fail(const char* what) { + throw InternalError(-3, std::string("malformed value buffer: ") + what); + } + + /** Reads from the `len` bytes at `data` (null when empty). */ BufferReader(const uint8_t* data, size_t len) noexcept : data_(data), len_(data != nullptr ? len : 0), pos_(0) {} /** Bytes not yet consumed. */ @@ -387,6 +501,14 @@ public: return b != 0; } + /** Reads an object token, adopting the strong reference it carries into a `T` wrapper. */ + template + T read_object() { + auto* raw = reinterpret_cast(static_cast(read_u64())); + if (raw == nullptr) fail("null object token"); + return T(adopt, raw); + } + /** Rejects unconsumed bytes after decoding a complete value. */ void expect_end() const { if (pos_ != len_) fail("trailing bytes after value"); @@ -409,6 +531,22 @@ struct BufferGuard { } }; +/** Decodes one complete `T` from a borrowed buffer with `read`, rejecting trailing bytes. */ +template +T decode(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferReader r(ptr, len); + T value = read(r); + r.expect_end(); + return value; +} + +/** Decodes one complete `T` from a producer-returned buffer with `read`, then releases the buffer. */ +template +T take(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferGuard guard(ptr, len); + return decode(ptr, len, read); +} + } // namespace detail /** @@ -422,7 +560,7 @@ enum class Status : int32_t { /** Document is publicly visible */ Published = 1, /** Document is hidden from listings */ - Archived = 2 + Archived = 2, }; struct Document; @@ -435,58 +573,68 @@ class SavedListener; * Notebooks own their documents and release them when destroyed. */ class Notebook { - docs_everywhere_docs_Notebook* handle_; + docs_everywhere_docs_Notebook* raw_; public: /** The C type this class wraps. */ using raw_type = docs_everywhere_docs_Notebook; /** Adopts one strong reference to a producer object: `Notebook(adopt, raw)`. */ - explicit Notebook(adopt_t, docs_everywhere_docs_Notebook* h) noexcept : handle_(h) {} + explicit Notebook(adopt_t, docs_everywhere_docs_Notebook* h) noexcept : raw_(h) {} /** Releases this wrapper's reference; the object is dropped with its last one. */ ~Notebook() { - if (handle_) docs_everywhere_docs_Notebook_destroy(handle_); + if (raw_ != nullptr) docs_everywhere_docs_Notebook_destroy(raw_); } /** Copies share the object: the copy takes a new strong reference. */ - Notebook(const Notebook& other) : handle_(docs_everywhere_docs_Notebook_clone(other.handle_)) {} + Notebook(const Notebook& other) : raw_(docs_everywhere_docs_Notebook_clone(other.raw_)) {} + /** Takes a new reference to `other`'s object and releases the current one. */ Notebook& operator=(const Notebook& other) { if (this != &other) { - docs_everywhere_docs_Notebook* h = docs_everywhere_docs_Notebook_clone(other.handle_); - if (handle_) docs_everywhere_docs_Notebook_destroy(handle_); - handle_ = h; + docs_everywhere_docs_Notebook* h = docs_everywhere_docs_Notebook_clone(other.raw_); + if (raw_ != nullptr) docs_everywhere_docs_Notebook_destroy(raw_); + raw_ = h; } return *this; } - Notebook(Notebook&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s reference; `other` becomes empty. */ + Notebook(Notebook&& other) noexcept : raw_(other.raw_) { other.raw_ = nullptr; } + /** Releases the current reference and takes over `other`'s. */ Notebook& operator=(Notebook&& other) noexcept { if (this != &other) { - if (handle_) docs_everywhere_docs_Notebook_destroy(handle_); - handle_ = other.handle_; - other.handle_ = nullptr; + if (raw_ != nullptr) docs_everywhere_docs_Notebook_destroy(raw_); + raw_ = other.raw_; + other.raw_ = nullptr; } return *this; } - /** The wrapped pointer, borrowed: this wrapper keeps its reference. */ - const docs_everywhere_docs_Notebook* handle() const { return handle_; } + /** The wrapped pointer, borrowed (null after a move): this wrapper keeps its reference. */ + const docs_everywhere_docs_Notebook* handle() const noexcept { return raw_; } - /** A new strong reference the caller owns (for example to write into a value buffer). */ - docs_everywhere_docs_Notebook* clone_handle() const { return docs_everywhere_docs_Notebook_clone(handle_); } + /** A new strong reference to the object, which the caller owns. */ + docs_everywhere_docs_Notebook* clone_handle() const { return docs_everywhere_docs_Notebook_clone(raw_); } /** Create an empty notebook */ - explicit Notebook(); + Notebook(); - /** Add a document, returning its position in the notebook */ + /** + * Add a document, returning its position in the notebook + * + * @param document The document to add + */ int32_t insert(const Document& document) const; - /** Look up a document by identifier, failing when none matches */ + /** + * Look up a document by identifier, failing when none matches + * + * @param id Identifier of the document to find + * @throws DocsErrorsError when the call fails with one of the domain's codes. + */ Document find(int64_t id) const; /** The number of documents currently held */ @@ -515,73 +663,108 @@ struct Document { namespace detail { -/** Encodes a `Document` in the value-buffer wire format. */ +inline void write_Document(BufferWriter& w, const Document& v); +inline Document read_Document(BufferReader& r); +inline void write_list_string(BufferWriter& w, const std::vector& v); +inline std::vector read_list_string(BufferReader& r); +inline void write_opt_Document(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_Document(BufferReader& r); + +/** Writes one `Document` to a value buffer. */ inline void write_Document(BufferWriter& w, const Document& v) { w.write_i64(v.id); w.write_string(v.title); w.write_i32(static_cast(v.status)); - w.write_len(v.tags.size()); - for (const auto& item0 : v.tags) { - w.write_string(item0); - } + detail::write_list_string(w, v.tags); } -/** Decodes a `Document` from the value-buffer wire format. */ +/** Reads one `Document` from a value buffer. */ inline Document read_Document(BufferReader& r) { - int64_t f_id = r.read_i64(); - std::string f_title = r.read_string(); - Status f_status = static_cast(r.read_i32()); - std::vector f_tags{}; - { - size_t f_tags_n = r.read_count(); - f_tags.reserve(r.reserve_hint(f_tags_n)); - for (size_t f_tags_i = 0; f_tags_i < f_tags_n; ++f_tags_i) { - std::string f_tags_item = r.read_string(); - f_tags.push_back(std::move(f_tags_item)); - } - } - return Document{std::move(f_id), std::move(f_title), std::move(f_status), std::move(f_tags)}; + return Document{ + r.read_i64(), + r.read_string(), + static_cast(r.read_i32()), + detail::read_list_string(r), + }; +} + +/** Writes one `[string]` to a value buffer. */ +inline void write_list_string(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) w.write_string(item); +} + +/** Reads one `[string]` from a value buffer. */ +inline std::vector read_list_string(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(r.read_string()); + return v; +} + +/** Writes one `Document?` to a value buffer. */ +inline void write_opt_Document(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) detail::write_Document(w, *v); +} + +/** Reads one `Document?` from a value buffer. */ +inline std::optional read_opt_Document(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(detail::read_Document(r)); } } // namespace detail -/** Typed errors reported by the `docs` module's throwing functions. */ +/** The errors the `docs` module's throwing calls report; catch a code's subclass or this. */ class DocsErrorsError : public Error { public: - DocsErrorsError(int32_t code, const std::string& msg) : Error(code, msg) {} + /** Builds an error carrying a domain `code` and `message`. */ + DocsErrorsError(int32_t code, const std::string& message) : Error(code, message) {} }; -/** Returned when no document matches the requested id */ +/** + * Returned when no document matches the requested id + * + * Code 1. + */ class NotFoundError : public DocsErrorsError { public: - explicit NotFoundError(const std::string& msg) : DocsErrorsError(1, msg) {} + /** Builds the error with the producer's `message` and the code's fields. */ + explicit NotFoundError(const std::string& message) : DocsErrorsError(1, message) {} }; -/** Raised when attempting to move into a forbidden state */ +/** + * Raised when attempting to move into a forbidden state + * + * Code 2. + */ class InvalidStatusError : public DocsErrorsError { public: - explicit InvalidStatusError(const std::string& msg) : DocsErrorsError(2, msg) {} + /** Builds the error with the producer's `message` and the code's fields. */ + explicit InvalidStatusError(const std::string& message) : DocsErrorsError(2, message) {} }; namespace detail { -/** Map a `docs` error code and payload to its typed exception (the runtime exception for negative codes, DocsErrorsError for unknown codes). */ -inline std::exception_ptr make_docs_error(int32_t code, const std::string& msg, const uint8_t* payload_ptr, size_t payload_len) { - if (code < 0) return make_error(code, msg); +/** The exception for a failed `DocsErrorsError` call: its code's subclass, Cancelled, or the root Error. */ +inline std::exception_ptr make_docs_error(int32_t code, const std::string& message, const uint8_t* payload_ptr, size_t payload_len) { + if (code == -5) return std::make_exception_ptr(Cancelled(message)); + if (code < 0) return std::make_exception_ptr(Error(code, message)); (void)payload_ptr; (void)payload_len; switch (code) { - case 1: return std::make_exception_ptr(NotFoundError(msg)); - case 2: return std::make_exception_ptr(InvalidStatusError(msg)); - default: return std::make_exception_ptr(DocsErrorsError(code, msg)); + case 1: return std::make_exception_ptr(NotFoundError(message)); + case 2: return std::make_exception_ptr(InvalidStatusError(message)); + default: return std::make_exception_ptr(DocsErrorsError(code, message)); } } -/** Throw the typed `docs` exception if `err` carries a nonzero code. */ +/** Throws the typed `DocsErrorsError` exception if `err` carries a nonzero code. */ inline void check_docs(docs_everywhere_error& err) { if (err.code == 0) return; - std::string msg = error_message(err); - std::exception_ptr ex = make_docs_error(err.code, msg, err.payload_ptr, err.payload_len); + std::exception_ptr ex = make_docs_error(err.code, error_message(err), err.payload_ptr, err.payload_len); docs_everywhere_error_clear(&err); std::rethrow_exception(ex); } @@ -594,7 +777,7 @@ inline void check_docs(docs_everywhere_error& err) { * Implementations must be safe to call from any thread. * * - * Implement this interface and pass a `std::shared_ptr` to any function that accepts one. The producer may call the methods from any thread until it releases its last reference to the implementation. An exception thrown by a method is reported to the producer as a foreign error (code -4) and aborts the call that triggered it. + * Subclass it and pass a `std::shared_ptr` where the API takes a `SavedListener`. The producer may call the methods from any thread, so they must be thread-safe, until it releases the implementation (also from any thread). An exception a method throws reaches the producer as a callback failure (code -4). */ class SavedListener { public: @@ -606,13 +789,12 @@ public: namespace detail { -/** Vtable trampolines adapting a `SavedListener` implementation to `docs_everywhere_docs_SavedListener_vtable`. */ +/** Trampolines adapting a `SavedListener` implementation to `docs_everywhere_docs_SavedListener_vtable`. */ struct SavedListener_trampolines { static void on_saved(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err) { try { SavedListener& impl = **static_cast*>(ctx); impl.on_saved(doc_id, revision); - return; } catch (const std::exception& e) { docs_everywhere_error_set(out_err, -4, e.what()); } catch (...) { @@ -620,36 +802,38 @@ struct SavedListener_trampolines { } } - /** Releases the implementation box once the producer drops its last reference. */ + /** Deletes the implementation box once the producer releases it. */ static void free_ctx(void* ctx) { delete static_cast*>(ctx); } }; -/** The process-wide vtable every `SavedListener` implementation is passed with. */ +/** The vtable every `SavedListener` implementation is passed with. */ inline const docs_everywhere_docs_SavedListener_vtable& SavedListener_vtable() { static const docs_everywhere_docs_SavedListener_vtable vtable = { - &SavedListener_trampolines::on_saved, + static_cast(sizeof(docs_everywhere_docs_SavedListener_vtable)), + 0, &SavedListener_trampolines::free_ctx, + &SavedListener_trampolines::on_saved, }; return vtable; } } // namespace detail -inline Notebook::Notebook() : handle_(nullptr) { +inline Notebook::Notebook() : raw_(nullptr) { check_library(); docs_everywhere_error err{}; auto result = docs_everywhere_docs_Notebook_new(&err); detail::check(err); - handle_ = result; + raw_ = result; } inline int32_t Notebook::insert(const Document& document) const { detail::BufferWriter document_buf; detail::write_Document(document_buf, document); docs_everywhere_error err{}; - auto result = docs_everywhere_docs_Notebook_insert(handle_, document_buf.data(), document_buf.size(), &err); + auto result = docs_everywhere_docs_Notebook_insert(raw_, document_buf.data(), document_buf.size(), &err); detail::check(err); return result; } @@ -657,18 +841,14 @@ inline int32_t Notebook::insert(const Document& document) const { inline Document Notebook::find(int64_t id) const { size_t out_len = 0; docs_everywhere_error err{}; - auto result = docs_everywhere_docs_Notebook_find(handle_, id, &out_len, &err); + auto result = docs_everywhere_docs_Notebook_find(raw_, id, &out_len, &err); detail::check_docs(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Document ret = detail::read_Document(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Document); } inline int32_t Notebook::size() const { docs_everywhere_error err{}; - auto result = docs_everywhere_docs_Notebook_size(handle_, &err); + auto result = docs_everywhere_docs_Notebook_size(raw_, &err); detail::check(err); return result; } @@ -688,44 +868,42 @@ namespace docs { * * The document starts in `Draft` status; callers must publish * it explicitly to make it visible. + * + * + * @param title Human-readable title for the new document + * @param tags Initial set of tags (may be empty) */ inline Document create_document(std::string_view title, const std::vector& tags) { check_library(); detail::BufferWriter tags_buf; - tags_buf.write_len(tags.size()); - for (const auto& item0 : tags) { - tags_buf.write_string(item0); - } + detail::write_list_string(tags_buf, tags); size_t out_len = 0; docs_everywhere_error err{}; auto result = docs_everywhere_docs_create_document(detail::utf8(title), title.size(), tags_buf.data(), tags_buf.size(), &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Document ret = detail::read_Document(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Document); } -/** Look up a document by identifier */ +/** + * Look up a document by identifier + * + * @param id Identifier returned from create_document + */ inline std::optional get_document(int64_t id) { check_library(); size_t out_len = 0; docs_everywhere_error err{}; auto result = docs_everywhere_docs_get_document(id, &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - std::optional ret{}; - if (result_r.read_option_flag()) { - Document ret_v = detail::read_Document(result_r); - ret = std::move(ret_v); - } - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_opt_Document); } -/** Publish a document in the background */ +/** + * Publish a document in the background + * + * @param id Identifier of the document to publish + * @return A future settled from a producer thread; get() rethrows the call's error. + */ inline std::future publish_async(int64_t id) { check_library(); auto promise = std::make_unique>(); @@ -734,8 +912,8 @@ inline std::future publish_async(int64_t id) { std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { p->set_value(result); } @@ -747,7 +925,11 @@ inline std::future publish_async(int64_t id) { return future; } -/** Register a listener for saved notifications */ +/** + * Register a listener for saved notifications + * + * @param listener Receiver of saved notifications + */ inline void watch(std::shared_ptr listener) { check_library(); if (!listener) throw std::invalid_argument("listener: null callback interface"); diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__CMakeLists_txt.snap deleted file mode 100644 index 84261aa3..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__CMakeLists_txt.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.14) -project(edge_cases_cpp VERSION 0.1.0 LANGUAGES CXX) - -# The producer library to link: a full path from the EDGE_CASES_LIBRARY cache or -# environment variable, or else the library name `edge_cases` (a CMake target -# of that name, or libedge_cases on the linker search path). -set(EDGE_CASES_LIBRARY "$ENV{EDGE_CASES_LIBRARY}" CACHE FILEPATH "Path to the edge_cases native library") - -add_library(edge_cases_cpp INTERFACE) -add_library(edge_cases::cpp ALIAS edge_cases_cpp) -target_include_directories(edge_cases_cpp INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}) -target_compile_features(edge_cases_cpp INTERFACE cxx_std_17) -if(EDGE_CASES_LIBRARY) - target_link_libraries(edge_cases_cpp INTERFACE "${EDGE_CASES_LIBRARY}") -else() - target_link_libraries(edge_cases_cpp INTERFACE edge_cases) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__README_md.snap deleted file mode 100644 index 1c7b58b7..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__README_md.snap +++ /dev/null @@ -1,43 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# edge_cases C++ bindings - -A header-only C++17 wrapper over the `edge_cases` C ABI. `edge_cases.hpp` -includes the C header beside it; every declaration lives in -`namespace edge_cases`. - -## Prerequisites - -- CMake 3.14 or later -- A C++17 compiler -- The `edge_cases` native library (`libedge_cases.so`, `libedge_cases.dylib`, or - `edge_cases.dll`) - -## Use with CMake - -```cmake -add_subdirectory(path/to/cpp) -add_executable(app main.cpp) -target_link_libraries(app PRIVATE edge_cases::cpp) -``` - -`edge_cases::cpp` adds this directory to the include path, requires -C++17, and links the native library: the file named by the `EDGE_CASES_LIBRARY` -CMake cache variable or environment variable when set, or else -`edge_cases` by name. - -```cpp -#include "edge_cases.hpp" -``` - -The first free function, constructor, or static member you call checks -that the linked library matches these headers and throws -`edge_cases::LoadError` if it doesn't; call `edge_cases::check_library()` -at startup to check eagerly. - - diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_h.snap b/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_h.snap deleted file mode 100644 index 94864086..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_h.snap +++ /dev/null @@ -1,245 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef EDGE_CASES_H -#define EDGE_CASES_H - -#include -#include -#include - -#ifndef EDGE_CASES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef EDGE_CASES_BUILD -# define EDGE_CASES_API __declspec(dllexport) -# else -# define EDGE_CASES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define EDGE_CASES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define EDGE_CASES_API __attribute__((visibility("default"))) -# else -# define EDGE_CASES_API -# endif -#endif - -#ifndef EDGE_CASES_DEPRECATED -# if defined(_MSC_VER) -# define EDGE_CASES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define EDGE_CASES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define EDGE_CASES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports edge_cases_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define EDGE_CASES_ABI_VERSION 3u -EDGE_CASES_API uint32_t edge_cases_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by edge_cases_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct edge_cases_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} edge_cases_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -EDGE_CASES_API void edge_cases_error_set(edge_cases_error* err, int32_t code, const char* message); -EDGE_CASES_API void edge_cases_error_clear(edge_cases_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - edge_cases_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -EDGE_CASES_API void edge_cases_error_free(edge_cases_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -EDGE_CASES_API void edge_cases_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct edge_cases_cancel_token edge_cases_cancel_token; -EDGE_CASES_API edge_cases_cancel_token* edge_cases_cancel_token_create(void); -EDGE_CASES_API void edge_cases_cancel_token_cancel(edge_cases_cancel_token* token); -EDGE_CASES_API bool edge_cases_cancel_token_is_cancelled(const edge_cases_cancel_token* token); -EDGE_CASES_API void edge_cases_cancel_token_destroy(edge_cases_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -EDGE_CASES_API uint8_t* edge_cases_alloc(uint32_t size); -EDGE_CASES_API void edge_cases_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -EDGE_CASES_API uint64_t edge_cases_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define EDGE_CASES_EDGE_CASES_CHECKSUM 0x1567f441073bd4aaull -EDGE_CASES_API uint64_t edge_cases_edge_cases_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with edge_cases_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * edge_cases_buffer.h declares a C struct and codec for each. - */ - -/** Deprecated enum with non-contiguous values and keyword-like variants */ -typedef enum { edge_cases_edge_cases_Legacy_None = 0, edge_cases_edge_cases_Legacy_Default = 5, edge_cases_edge_cases_Legacy_Class = 10 } edge_cases_edge_cases_Legacy; -typedef enum { edge_cases_edge_cases_Mode_Fast = 1, edge_cases_edge_cases_Mode_Safe = 2 } edge_cases_edge_cases_Mode; -/** Error codes reported by throwing functions in the `edge_cases` module tree. */ -typedef enum { edge_cases_edge_cases_EdgeErrors_Busy = 7 } edge_cases_edge_cases_EdgeErrors; -typedef struct edge_cases_edge_cases_Widget edge_cases_edge_cases_Widget; -typedef struct edge_cases_edge_cases_Vault edge_cases_edge_cases_Vault; -typedef struct edge_cases_edge_cases_WidgetStreamIterator edge_cases_edge_cases_WidgetStreamIterator; -typedef struct edge_cases_edge_cases_StringStreamIterator edge_cases_edge_cases_StringStreamIterator; -typedef struct edge_cases_edge_cases_ScalarStreamIterator edge_cases_edge_cases_ScalarStreamIterator; -typedef struct edge_cases_edge_cases_ModeStreamIterator edge_cases_edge_cases_ModeStreamIterator; -/** - * Callback interface carrying composite payloads and objects - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_BatchListener_vtable { - /** Composite payloads cross as value buffers */ - void (*on_batch)(void* ctx, const uint8_t* items_ptr, size_t items_len, const uint8_t* raw_ptr, size_t raw_len, const uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err); - /** Objects cross as strong references the consumer must release */ - void (*on_widget)(void* ctx, edge_cases_edge_cases_Widget* widget, edge_cases_edge_cases_Widget* fallback, const uint8_t* children_ptr, size_t children_len, edge_cases_error* out_err); - void (*on_tick)(void* ctx, edge_cases_error* out_err); - /** Scalar return travels back through the vtable */ - bool (*should_continue)(void* ctx, int32_t class_, edge_cases_error* out_err); - /** Enum return travels back as its integer discriminant */ - edge_cases_edge_cases_Mode (*pick_mode)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_BatchListener_vtable; -/** - * Callback interface whose name and methods are keywords - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_Keyword_vtable { - void (*delete_)(void* ctx, int32_t self, edge_cases_error* out_err); - int32_t (*import)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_Keyword_vtable; -typedef void (*edge_cases_edge_cases_async_widget_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result); -typedef void (*edge_cases_edge_cases_async_void_callback)(void* context, edge_cases_error* err); -typedef void (*edge_cases_edge_cases_async_bytes_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_struct_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_list_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_optional_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_enum_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Mode result); -typedef void (*edge_cases_edge_cases_async_keywords_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_Widget_fetch_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: edge_cases -/** Interface with no constructor; instances come from make_widget */ -/** Void method that can fail */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_refresh(const edge_cases_edge_cases_Widget* self, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_rename(const edge_cases_edge_cases_Widget* self, const uint8_t* class_ptr, size_t class_len, const uint8_t* import_ptr, size_t import_len, size_t* out_len, edge_cases_error* out_err); -/** Async method with a struct result */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_fetch(const edge_cases_edge_cases_Widget* self, int64_t id, edge_cases_edge_cases_Widget_fetch_callback callback, void* context); -EDGE_CASES_DEPRECATED("Use rename instead") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_old_name(const edge_cases_edge_cases_Widget* self, size_t* out_len, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Widget_clone(const edge_cases_edge_cases_Widget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_destroy(edge_cases_edge_cases_Widget* self); - -/** Deprecated interface with a throwing constructor */ -/** Fails when the vault is busy */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_open(const uint8_t* path_ptr, size_t path_len, edge_cases_error* out_err); -/** Throwing static */ -EDGE_CASES_API bool edge_cases_edge_cases_Vault_probe(edge_cases_error* out_err); -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Vault_peek(const edge_cases_edge_cases_Vault* self, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_clone(const edge_cases_edge_cases_Vault* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Vault_destroy(edge_cases_edge_cases_Vault* self); - -/** No parameters and no result */ -EDGE_CASES_API void edge_cases_edge_cases_noop(edge_cases_error* out_err); -/** Function named after a keyword whose parameters are all keywords */ -EDGE_CASES_API int32_t edge_cases_edge_cases_select(int32_t class_, const uint8_t* from_ptr, size_t from_len, const uint8_t* func_ptr, size_t func_len, const uint8_t* range_ptr, size_t range_len, bool type_, edge_cases_error* out_err); -/** Optional list of optional structs, the deepest legal nesting */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_nested_optional(const uint8_t* items_ptr, size_t items_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_optional_params(const uint8_t* text_ptr, size_t text_len, const uint8_t* count_ptr, size_t count_len, const uint8_t* outer_ptr, size_t outer_len, const uint8_t* tags_ptr, size_t tags_len, const uint8_t* blob_ptr, size_t blob_len, const uint8_t* mode_ptr, size_t mode_len, size_t* out_len, edge_cases_error* out_err); -/** Two callback interface parameters on one function */ -EDGE_CASES_API void edge_cases_edge_cases_register_batch(void* listener_ctx, const edge_cases_edge_cases_BatchListener_vtable* listener_vtable, void* keyword_ctx, const edge_cases_edge_cases_Keyword_vtable* keyword_vtable, edge_cases_error* out_err); -/** Objects in every buffered position */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_widgets_everywhere(const uint8_t* list_ptr, size_t list_len, const uint8_t* by_name_ptr, size_t by_name_len, const uint8_t* sparse_ptr, size_t sparse_len, const uint8_t* holder_ptr, size_t holder_len, size_t* out_len, edge_cases_error* out_err); -/** Iterator of objects */ -EDGE_CASES_API edge_cases_edge_cases_WidgetStreamIterator* edge_cases_edge_cases_widget_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_WidgetStreamIterator_next(edge_cases_edge_cases_WidgetStreamIterator* iter, edge_cases_edge_cases_Widget** out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_WidgetStreamIterator_destroy(edge_cases_edge_cases_WidgetStreamIterator* iter); -/** Async returning an object */ -EDGE_CASES_API void edge_cases_edge_cases_async_widget(edge_cases_edge_cases_async_widget_callback callback, void* context); -/** Factory for an interface with no constructor */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_make_widget(const uint8_t* label_ptr, size_t label_len, edge_cases_error* out_err); -/** Optional interface return */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_find_widget(int64_t id, edge_cases_error* out_err); -/** Interface parameter, including an optional one */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_inspect_widget(const edge_cases_edge_cases_Widget* widget, const edge_cases_edge_cases_Widget* fallback, size_t* out_len, edge_cases_error* out_err); -/** Iterator of strings */ -EDGE_CASES_API edge_cases_edge_cases_StringStreamIterator* edge_cases_edge_cases_string_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_StringStreamIterator_next(edge_cases_edge_cases_StringStreamIterator* iter, const uint8_t** out_item, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_StringStreamIterator_destroy(edge_cases_edge_cases_StringStreamIterator* iter); -/** Iterator of scalars */ -EDGE_CASES_API edge_cases_edge_cases_ScalarStreamIterator* edge_cases_edge_cases_scalar_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ScalarStreamIterator_next(edge_cases_edge_cases_ScalarStreamIterator* iter, int32_t* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ScalarStreamIterator_destroy(edge_cases_edge_cases_ScalarStreamIterator* iter); -/** Iterator of enums */ -EDGE_CASES_API edge_cases_edge_cases_ModeStreamIterator* edge_cases_edge_cases_mode_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ModeStreamIterator_next(edge_cases_edge_cases_ModeStreamIterator* iter, edge_cases_edge_cases_Mode* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ModeStreamIterator_destroy(edge_cases_edge_cases_ModeStreamIterator* iter); -/** Async with no result */ -EDGE_CASES_API void edge_cases_edge_cases_async_void(edge_cases_edge_cases_async_void_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_bytes(uint32_t size, edge_cases_edge_cases_async_bytes_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_struct(edge_cases_edge_cases_async_struct_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_list(edge_cases_cancel_token* cancel_token, edge_cases_edge_cases_async_list_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_optional(edge_cases_edge_cases_async_optional_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_enum(edge_cases_edge_cases_async_enum_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_keywords(const uint8_t* class_ptr, size_t class_len, int32_t import, edge_cases_edge_cases_async_keywords_callback callback, void* context); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_keywords_roundtrip(const uint8_t* value_ptr, size_t value_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_DEPRECATED("Use Mode-based APIs") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_legacy_value(edge_cases_edge_cases_Legacy value, size_t* out_len, edge_cases_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // EDGE_CASES_H - -// END edge_cases.h diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_hpp.snap b/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_hpp.snap index 154c3b48..defce99c 100644 --- a/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_hpp.snap +++ b/crates/weaveffi-cli/tests/snapshots/cpp_edge_cases__edge_cases_hpp.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #pragma once @@ -28,13 +28,13 @@ source: crates/weaveffi-cli/tests/snapshots.rs namespace edge_cases { /** - * Base exception for every error reported through the C ABI. + * An error a call that declares errors reports. * - * Positive codes are the declared error codes (see the typed domain - * exceptions); negative codes are runtime traps: -1 a generic producer - * error, -2 a producer panic, -3 a marshalling failure, -4 a - * callback-interface implementation that threw, and -5 a cancelled call - * (thrown as Cancelled). + * A positive code is a declared code of the module's error domain, thrown as + * that domain's typed subclass. A negative code is a runtime failure: -1 a + * generic producer error, -2 a producer panic, -3 a marshalling failure, and + * -4 a callback-interface implementation that failed. A cancelled async call + * (-5) throws Cancelled. */ class Error : public std::runtime_error { int32_t code_; @@ -47,7 +47,7 @@ public: int32_t code() const noexcept { return code_; } }; -/** Thrown by an async call whose CancelToken was cancelled (code -5). */ +/** Thrown by an async call that was cancelled through its CancelToken (code -5). */ class Cancelled : public Error { public: /** Builds a cancellation error carrying the producer's message. */ @@ -55,13 +55,39 @@ public: }; /** - * Thrown on first use when the linked edge_cases library was built for a - * different C ABI revision or a different API contract than this header. + * Thrown by a call that declares no errors when it fails anyway: a producer + * panic (-2), a marshalling failure (-3), a callback-interface failure the + * producer let through (-4), or another runtime failure (-1). These are bugs, + * so no signature declares them; `what()` names the code and the producer's + * message. */ -class LoadError : public Error { +class InternalError : public std::runtime_error { + int32_t code_; + std::string message_; + +public: + /** Builds an internal error carrying `code` and the producer's `message`. */ + InternalError(int32_t code, const std::string& message) + : std::runtime_error("edge_cases: runtime error " + std::to_string(code) + ": " + message), + code_(code), + message_(message) {} + + /** The runtime error code (negative). */ + int32_t code() const noexcept { return code_; } + + /** The producer's message. */ + const std::string& message() const noexcept { return message_; } +}; + +/** + * Thrown on first use when the linked edge_cases library implements a + * different C ABI revision than this header, or lacks or changed a + * declaration this header was generated with. + */ +class LoadError : public std::runtime_error { public: /** Builds a load error describing the mismatch. */ - explicit LoadError(const std::string& message) : Error(-1, message) {} + explicit LoadError(const std::string& message) : std::runtime_error(message) {} }; /** @@ -72,7 +98,7 @@ struct adopt_t { explicit adopt_t() = default; }; -/** The adopt_t tag value: `Gadget g(adopt, raw)`. */ +/** The adopt_t tag value: `Store s(adopt, raw)`. */ inline constexpr adopt_t adopt{}; /** @@ -138,10 +164,10 @@ public: namespace detail { -/** The exception for a runtime (non-domain) error code. */ -inline std::exception_ptr make_error(int32_t code, const std::string& message) { +/** The exception for a failed call that declares no errors: Cancelled for -5, else InternalError. */ +inline std::exception_ptr make_internal_error(int32_t code, const std::string& message) { if (code == -5) return std::make_exception_ptr(Cancelled(message)); - return std::make_exception_ptr(Error(code, message)); + return std::make_exception_ptr(InternalError(code, message)); } /** The message of a nonzero error slot. */ @@ -149,10 +175,10 @@ inline std::string error_message(const edge_cases_error& err) { return std::string(err.message != nullptr ? err.message : "unknown error"); } -/** Throws the generic Error (or Cancelled) if `err` carries a nonzero code. */ +/** Throws InternalError if `err` carries a nonzero code (a call that declares no errors). */ inline void check(edge_cases_error& err) { if (err.code == 0) return; - std::exception_ptr ex = make_error(err.code, error_message(err)); + std::exception_ptr ex = make_internal_error(err.code, error_message(err)); edge_cases_error_clear(&err); std::rethrow_exception(ex); } @@ -185,7 +211,7 @@ inline std::string_view borrow_string(const uint8_t* ptr, size_t len) noexcept { return std::string_view(reinterpret_cast(ptr), len); } -/** Views a borrowed byte run (null when empty) as an owned vector. */ +/** Copies a borrowed byte run (null when empty) into a vector. */ inline std::vector borrow_bytes(const uint8_t* ptr, size_t len) { if (ptr == nullptr || len == 0) return std::vector(); return std::vector(ptr, ptr + len); @@ -196,24 +222,134 @@ inline const uint8_t* utf8(std::string_view s) noexcept { return reinterpret_cast(s.data()); } +/** + * Hands `len` bytes from `data` to the producer through a callback method's + * out slots: copies them into a run from edge_cases_alloc, which the + * producer adopts and frees. + */ +inline void hand_over(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = nullptr; + if (len > 0) { + run = edge_cases_alloc(len); + if (run == nullptr) throw std::bad_alloc(); + std::memcpy(run, data, len); + } + *out_ptr = run; + *out_len = len; +} + +/** One declaration this header was generated with: its contract id and hash, and its dotted path. */ +struct ContractEntry { + /** FNV-1a 64 of the declaration's dotted path. */ + uint64_t id; + /** FNV-1a 64 of the declaration's canonical signature. */ + uint64_t hash; + /** The declaration's dotted path (`kv.Store.put`). */ + const char* path; +}; + +/** + * Why the contract table `contract` returns doesn't satisfy `expected`, + * naming the first declaration it lacks or declares differently, or an empty + * string when it has them all. Entries the producer has and `expected` + * doesn't are fine. + */ +template +std::string contract_mismatch(const edge_cases_contract_entry* (*contract)(size_t*), + const ContractEntry (&expected)[N]) { + size_t len = 0; + const edge_cases_contract_entry* table = contract(&len); + if (table == nullptr) len = 0; + for (const ContractEntry& want : expected) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < want.id) { + lo = mid + 1; + } else { + hi = mid; + } + } + if (lo == len || table[lo].id != want.id) { + return std::string(want.path) + " is missing from the library"; + } + if (table[lo].hash != want.hash) { + return std::string(want.path) + " changed since these bindings were generated"; + } + } + return std::string(); +} + +/** The declarations of module `edge_cases` this header was generated with. */ +inline constexpr ContractEntry edge_cases_contract[] = { + {0x056a86511785ba2bull, 0x5cba3f16d57c6cdbull, "edge_cases.make_widget"}, + {0x07bfa9ecb2b720d2ull, 0x3813b3f57c64bf35ull, "edge_cases.EdgeErrors"}, + {0x0aaa11867e37f991ull, 0xf090dd52146ebdf0ull, "edge_cases.Widget.refresh"}, + {0x0f9701493898ae4cull, 0x6944f47a275c96e8ull, "edge_cases.Inner"}, + {0x1226b1adac179f7dull, 0x0e95a00ab0bef637ull, "edge_cases.async_keywords"}, + {0x16d36bef1541136cull, 0x5ea7021ec324298full, "edge_cases.find_widget"}, + {0x1b7324c55e980febull, 0xabf87fe372358f42ull, "edge_cases.Widget.old_name"}, + {0x1cbd731467c2bab8ull, 0x4915b9989c2c9074ull, "edge_cases.Vault"}, + {0x2c2a6de67d894b0aull, 0x50a4496b1e7191caull, "edge_cases.Vault.open"}, + {0x322b4cfcd2d6861cull, 0xca7527a379ae8465ull, "edge_cases.Keywords"}, + {0x3f8e23a5f32906f4ull, 0x8d4c908d796f130dull, "edge_cases.Leaf"}, + {0x42ab8a685457b97cull, 0xe5ce30500cadb32dull, "edge_cases.Vault.handle"}, + {0x4a68540e2836d3d0ull, 0xa20e44c71e420ed2ull, "edge_cases.mode_stream"}, + {0x4e030580ac0b111dull, 0xb68b7a18d910e175ull, "edge_cases.legacy_value"}, + {0x4f5e4e5291864ef3ull, 0x55a1a88b21a08f8bull, "edge_cases.Legacy"}, + {0x4f84b06171b4213dull, 0xbaedcfdebb6a2820ull, "edge_cases.scalar_stream"}, + {0x5067a5ff2d9ef910ull, 0xbfc79b18fc4714fcull, "edge_cases.Widget"}, + {0x59aff159112fbf5bull, 0x18e4def57a8cfa49ull, "edge_cases.optional_params"}, + {0x5daf65b98fa77c48ull, 0xc817a8cb202e59c2ull, "edge_cases.keywords_roundtrip"}, + {0x5de8a5c9726c2c5aull, 0x7c50b5b65f7a896dull, "edge_cases.register_batch"}, + {0x618a620309994c3cull, 0x826ad0b1d208d723ull, "edge_cases.async_bytes"}, + {0x648285cdba0a7b8eull, 0xcf6e80e783a4d8beull, "edge_cases.WidgetHolder"}, + {0x649c78079824b10dull, 0x5b12dd26e0634b69ull, "edge_cases.async_widget"}, + {0x7da7a7357d122a37ull, 0xf544e025ffaa3d20ull, "edge_cases.async_void"}, + {0x7e4015bc3a472e2dull, 0xfe22291d90b82e4aull, "edge_cases.widget_stream"}, + {0x805276e5d81fdba7ull, 0xd9b64e360bd03b2bull, "edge_cases.Outer"}, + {0x80e1183326542df4ull, 0xe8a57e12cc6ed276ull, "edge_cases.select"}, + {0x923ce5e9bcd769a1ull, 0x0b973f5dc07dda8bull, "edge_cases.Vault.to_string"}, + {0x98ca8c01f5239404ull, 0xfe11c93eb6b303efull, "edge_cases.Widget.rename"}, + {0xa13e22e98ea278d4ull, 0xf1fae215d7387800ull, "edge_cases.widgets_everywhere"}, + {0xa84a8c9bf2a881e1ull, 0xbe1dc840bf221996ull, "edge_cases.async_optional"}, + {0xad958c478fbf205dull, 0xe386b0716d24f19bull, "edge_cases.Retired"}, + {0xb2fc3964ef56ab39ull, 0x73c325c8bc351236ull, "edge_cases.async_list"}, + {0xb3403af521cf0006ull, 0x6abc30d48206a3baull, "edge_cases.Widget.close"}, + {0xb794304a22a406b8ull, 0xf6fb98711c516105ull, "edge_cases.async_struct"}, + {0xba3936106a49c47dull, 0xb7025f2fadd6288eull, "edge_cases.Vault.peek"}, + {0xc155b0c3cfb164f4ull, 0xc777ae57d92ebc47ull, "edge_cases.noop"}, + {0xcdc18607777019ddull, 0xb81a31e5c20aa7a8ull, "edge_cases.inspect_widget"}, + {0xd164a82aeb64b8ccull, 0xded02c68005c4737ull, "edge_cases.nested_optional"}, + {0xd978ccad2468038full, 0x30c39c65d6235153ull, "edge_cases.Mode"}, + {0xdbe9d947431c9124ull, 0x39537fb848fdf980ull, "edge_cases.BatchListener"}, + {0xeae8b7862c08bb07ull, 0xfe12eeaa830452eaull, "edge_cases.Keyword"}, + {0xf1e155a05ae262b8ull, 0xb575eb818a423d42ull, "edge_cases.async_enum"}, + {0xf22e210bdbce7343ull, 0x2a5e0f801bc8c768ull, "edge_cases.Widget.dispose"}, + {0xf3c7cf675563ee6cull, 0x0ecffeef399facc8ull, "edge_cases.Vault.probe"}, + {0xfd34bc67acac379eull, 0x576caa14f3c0901aull, "edge_cases.Widget.fetch"}, + {0xff2e9d2bff941786ull, 0x7e96764ef1f5fbaaull, "edge_cases.string_stream"}, +}; + } // namespace detail /** - * Verifies that the linked edge_cases library implements the C ABI revision - * and the module contracts this header was generated against, throwing + * Verifies that the linked edge_cases library implements C ABI revision + * 4 and every declaration this header was generated with, throwing * LoadError naming the first mismatch. The check runs once; every free * function, constructor, and static member calls this before its first * native call, so calling it at startup is optional. */ inline void check_library() { - static const std::string failure = [] { + static const std::string failure = []() -> std::string { uint32_t abi = edge_cases_abi_version(); if (abi != EDGE_CASES_ABI_VERSION) { return "edge_cases: C ABI revision mismatch (header " + std::to_string(EDGE_CASES_ABI_VERSION) + ", library " + std::to_string(abi) + ")"; } - if (edge_cases_edge_cases_checksum() != 0x1567f441073bd4aaull) { - return std::string("edge_cases: module 'edge_cases' does not match the linked library (contract checksum mismatch); regenerate the bindings for this build"); + if (std::string why = detail::contract_mismatch(edge_cases_edge_cases_contract, detail::edge_cases_contract); !why.empty()) { + return "edge_cases: " + why; } return std::string(); }(); @@ -267,7 +403,7 @@ public: /** Writes a byte length or an element count as a u32. */ void write_len(size_t n) { - if (static_cast(n) > UINT32_MAX) throw Error(-3, "value buffer length exceeds u32"); + if (static_cast(n) > UINT32_MAX) throw std::length_error("value buffer length exceeds u32"); append_le(static_cast(n)); } @@ -283,24 +419,29 @@ public: /** Writes an optional's presence flag: 0 absent, 1 present. */ void write_option_flag(bool present) { buf_.push_back(present ? 1 : 0); } + + /** + * Writes an object token: a fresh strong reference from the wrapper's + * `_clone`, which the reader adopts while `v` keeps its own. + */ + template + void write_object(const T& v) { + write_u64(static_cast(reinterpret_cast(v.clone_handle()))); + } }; /** * Decodes values from the value-buffer wire format. A malformed buffer is a - * producer/consumer contract violation (both sides are generated from one - * API), so every decode failure throws the generic Error with the - * marshalling code -3. Object tokens adopted before a failure live in RAII - * wrappers that release them on unwind. + * producer bug (both sides are generated from one API), so every decode + * failure throws InternalError with the marshalling code -3. Object tokens + * adopted before a failure live in RAII wrappers that release them on + * unwind. */ class BufferReader { const uint8_t* data_; size_t len_; size_t pos_; - [[noreturn]] static void fail(const char* what) { - throw Error(-3, std::string("malformed value buffer: ") + what); - } - void require(size_t n, const char* what) const { if (len_ - pos_ < n) fail(what); } @@ -317,6 +458,12 @@ class BufferReader { } public: + /** Throws the marshalling failure (-3) for a malformed buffer. */ + [[noreturn]] static void fail(const char* what) { + throw InternalError(-3, std::string("malformed value buffer: ") + what); + } + + /** Reads from the `len` bytes at `data` (null when empty). */ BufferReader(const uint8_t* data, size_t len) noexcept : data_(data), len_(data != nullptr ? len : 0), pos_(0) {} /** Bytes not yet consumed. */ @@ -388,6 +535,14 @@ public: return b != 0; } + /** Reads an object token, adopting the strong reference it carries into a `T` wrapper. */ + template + T read_object() { + auto* raw = reinterpret_cast(static_cast(read_u64())); + if (raw == nullptr) fail("null object token"); + return T(adopt, raw); + } + /** Rejects unconsumed bytes after decoding a complete value. */ void expect_end() const { if (pos_ != len_) fail("trailing bytes after value"); @@ -410,18 +565,38 @@ struct BufferGuard { } }; +/** Decodes one complete `T` from a borrowed buffer with `read`, rejecting trailing bytes. */ +template +T decode(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferReader r(ptr, len); + T value = read(r); + r.expect_end(); + return value; +} + +/** Decodes one complete `T` from a producer-returned buffer with `read`, then releases the buffer. */ +template +T take(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferGuard guard(ptr, len); + return decode(ptr, len, read); +} + } // namespace detail -/** Deprecated enum with non-contiguous values and keyword-like variants */ +/** + * Deprecated enum with non-contiguous values and keyword-like variants + * + * @deprecated Use Mode instead + */ enum class Legacy : int32_t { None = 0, Default = 5, - Class = 10 + Class = 10, }; enum class Mode : int32_t { Fast = 1, - Safe = 2 + Safe = 2, }; struct Leaf; @@ -437,119 +612,154 @@ class Keyword; /** Interface with no constructor; instances come from make_widget */ class Widget { - edge_cases_edge_cases_Widget* handle_; + edge_cases_edge_cases_Widget* raw_; public: /** The C type this class wraps. */ using raw_type = edge_cases_edge_cases_Widget; /** Adopts one strong reference to a producer object: `Widget(adopt, raw)`. */ - explicit Widget(adopt_t, edge_cases_edge_cases_Widget* h) noexcept : handle_(h) {} + explicit Widget(adopt_t, edge_cases_edge_cases_Widget* h) noexcept : raw_(h) {} /** Releases this wrapper's reference; the object is dropped with its last one. */ ~Widget() { - if (handle_) edge_cases_edge_cases_Widget_destroy(handle_); + if (raw_ != nullptr) edge_cases_edge_cases_Widget_destroy(raw_); } /** Copies share the object: the copy takes a new strong reference. */ - Widget(const Widget& other) : handle_(edge_cases_edge_cases_Widget_clone(other.handle_)) {} + Widget(const Widget& other) : raw_(edge_cases_edge_cases_Widget_clone(other.raw_)) {} + /** Takes a new reference to `other`'s object and releases the current one. */ Widget& operator=(const Widget& other) { if (this != &other) { - edge_cases_edge_cases_Widget* h = edge_cases_edge_cases_Widget_clone(other.handle_); - if (handle_) edge_cases_edge_cases_Widget_destroy(handle_); - handle_ = h; + edge_cases_edge_cases_Widget* h = edge_cases_edge_cases_Widget_clone(other.raw_); + if (raw_ != nullptr) edge_cases_edge_cases_Widget_destroy(raw_); + raw_ = h; } return *this; } - Widget(Widget&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s reference; `other` becomes empty. */ + Widget(Widget&& other) noexcept : raw_(other.raw_) { other.raw_ = nullptr; } + /** Releases the current reference and takes over `other`'s. */ Widget& operator=(Widget&& other) noexcept { if (this != &other) { - if (handle_) edge_cases_edge_cases_Widget_destroy(handle_); - handle_ = other.handle_; - other.handle_ = nullptr; + if (raw_ != nullptr) edge_cases_edge_cases_Widget_destroy(raw_); + raw_ = other.raw_; + other.raw_ = nullptr; } return *this; } - /** The wrapped pointer, borrowed: this wrapper keeps its reference. */ - const edge_cases_edge_cases_Widget* handle() const { return handle_; } + /** The wrapped pointer, borrowed (null after a move): this wrapper keeps its reference. */ + const edge_cases_edge_cases_Widget* handle() const noexcept { return raw_; } - /** A new strong reference the caller owns (for example to write into a value buffer). */ - edge_cases_edge_cases_Widget* clone_handle() const { return edge_cases_edge_cases_Widget_clone(handle_); } + /** A new strong reference to the object, which the caller owns. */ + edge_cases_edge_cases_Widget* clone_handle() const { return edge_cases_edge_cases_Widget_clone(raw_); } - /** Void method that can fail */ + /** + * Void method that can fail + * + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ void refresh() const; std::string rename(std::string_view class_, const std::optional& import) const; - /** Async method with a struct result */ + /** + * Async method with a struct result + * + * @return A future settled from a producer thread; get() rethrows the call's error. + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ std::future fetch(int64_t id) const; [[deprecated("Use rename instead")]] std::string old_name() const; + /** Spelled like the release method of several wrappers */ + bool close() const; + + /** Spelled like the release method of the Dart and .NET wrappers */ + void dispose(bool force) const; + }; -/** Deprecated interface with a throwing constructor */ +/** + * Deprecated interface with a throwing constructor + * + * @deprecated Use Widget instead + */ class Vault { - edge_cases_edge_cases_Vault* handle_; + edge_cases_edge_cases_Vault* raw_; public: /** The C type this class wraps. */ using raw_type = edge_cases_edge_cases_Vault; /** Adopts one strong reference to a producer object: `Vault(adopt, raw)`. */ - explicit Vault(adopt_t, edge_cases_edge_cases_Vault* h) noexcept : handle_(h) {} + explicit Vault(adopt_t, edge_cases_edge_cases_Vault* h) noexcept : raw_(h) {} /** Releases this wrapper's reference; the object is dropped with its last one. */ ~Vault() { - if (handle_) edge_cases_edge_cases_Vault_destroy(handle_); + if (raw_ != nullptr) edge_cases_edge_cases_Vault_destroy(raw_); } /** Copies share the object: the copy takes a new strong reference. */ - Vault(const Vault& other) : handle_(edge_cases_edge_cases_Vault_clone(other.handle_)) {} + Vault(const Vault& other) : raw_(edge_cases_edge_cases_Vault_clone(other.raw_)) {} + /** Takes a new reference to `other`'s object and releases the current one. */ Vault& operator=(const Vault& other) { if (this != &other) { - edge_cases_edge_cases_Vault* h = edge_cases_edge_cases_Vault_clone(other.handle_); - if (handle_) edge_cases_edge_cases_Vault_destroy(handle_); - handle_ = h; + edge_cases_edge_cases_Vault* h = edge_cases_edge_cases_Vault_clone(other.raw_); + if (raw_ != nullptr) edge_cases_edge_cases_Vault_destroy(raw_); + raw_ = h; } return *this; } - Vault(Vault&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s reference; `other` becomes empty. */ + Vault(Vault&& other) noexcept : raw_(other.raw_) { other.raw_ = nullptr; } + /** Releases the current reference and takes over `other`'s. */ Vault& operator=(Vault&& other) noexcept { if (this != &other) { - if (handle_) edge_cases_edge_cases_Vault_destroy(handle_); - handle_ = other.handle_; - other.handle_ = nullptr; + if (raw_ != nullptr) edge_cases_edge_cases_Vault_destroy(raw_); + raw_ = other.raw_; + other.raw_ = nullptr; } return *this; } - /** The wrapped pointer, borrowed: this wrapper keeps its reference. */ - const edge_cases_edge_cases_Vault* handle() const { return handle_; } + /** The wrapped pointer, borrowed (null after a move): this wrapper keeps its reference. */ + const edge_cases_edge_cases_Vault* handle() const noexcept { return raw_; } - /** A new strong reference the caller owns (for example to write into a value buffer). */ - edge_cases_edge_cases_Vault* clone_handle() const { return edge_cases_edge_cases_Vault_clone(handle_); } + /** A new strong reference to the object, which the caller owns. */ + edge_cases_edge_cases_Vault* clone_handle() const { return edge_cases_edge_cases_Vault_clone(raw_); } - /** Fails when the vault is busy */ + /** + * Fails when the vault is busy + * + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ static Vault open(std::string_view path); std::optional peek() const; - /** Throwing static */ + /** Spelled like the handle accessor of several wrappers */ + int64_t handle_() const; + + /** + * Throwing static + * + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ static bool probe(); + /** Static spelled like an inherited object member */ + static std::string to_string(); + }; struct Leaf { @@ -639,7 +849,11 @@ struct Keywords { int32_t super; }; -/** Deprecated struct */ +/** + * Deprecated struct + * + * @deprecated Use Outer instead + */ struct Retired { int64_t id; }; @@ -654,169 +868,124 @@ struct WidgetHolder { namespace detail { -/** Encodes a `Leaf` in the value-buffer wire format. */ +inline void write_Leaf(BufferWriter& w, const Leaf& v); +inline Leaf read_Leaf(BufferReader& r); +inline void write_Inner(BufferWriter& w, const Inner& v); +inline Inner read_Inner(BufferReader& r); +inline void write_Outer(BufferWriter& w, const Outer& v); +inline Outer read_Outer(BufferReader& r); +inline void write_Keywords(BufferWriter& w, const Keywords& v); +inline Keywords read_Keywords(BufferReader& r); +inline void write_Retired(BufferWriter& w, const Retired& v); +inline Retired read_Retired(BufferReader& r); +inline void write_WidgetHolder(BufferWriter& w, const WidgetHolder& v); +inline WidgetHolder read_WidgetHolder(BufferReader& r); +inline void write_list_Leaf(BufferWriter& w, const std::vector& v); +inline std::vector read_list_Leaf(BufferReader& r); +inline void write_opt_Inner(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_Inner(BufferReader& r); +inline void write_map_string_Inner(BufferWriter& w, const std::unordered_map& v); +inline std::unordered_map read_map_string_Inner(BufferReader& r); +inline void write_list_Mode(BufferWriter& w, const std::vector& v); +inline std::vector read_list_Mode(BufferReader& r); +inline void write_list_i32(BufferWriter& w, const std::vector& v); +inline std::vector read_list_i32(BufferReader& r); +inline void write_list_list_i32(BufferWriter& w, const std::vector>& v); +inline std::vector> read_list_list_i32(BufferReader& r); +inline void write_opt_i64(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_i64(BufferReader& r); +inline void write_list_opt_i64(BufferWriter& w, const std::vector>& v); +inline std::vector> read_list_opt_i64(BufferReader& r); +inline void write_opt_bytes(BufferWriter& w, const std::optional>& v); +inline std::optional> read_opt_bytes(BufferReader& r); +inline void write_opt_string(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_string(BufferReader& r); +inline void write_map_i64_string(BufferWriter& w, const std::unordered_map& v); +inline std::unordered_map read_map_i64_string(BufferReader& r); +inline void write_opt_Widget(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_Widget(BufferReader& r); +inline void write_list_Widget(BufferWriter& w, const std::vector& v); +inline std::vector read_list_Widget(BufferReader& r); +inline void write_map_string_Widget(BufferWriter& w, const std::unordered_map& v); +inline std::unordered_map read_map_string_Widget(BufferReader& r); +inline void write_list_string(BufferWriter& w, const std::vector& v); +inline std::vector read_list_string(BufferReader& r); +inline void write_opt_i32(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_i32(BufferReader& r); +inline void write_opt_Outer(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_Outer(BufferReader& r); +inline void write_list_opt_Outer(BufferWriter& w, const std::vector>& v); +inline std::vector> read_list_opt_Outer(BufferReader& r); +inline void write_opt_list_opt_Outer(BufferWriter& w, const std::optional>>& v); +inline std::optional>> read_opt_list_opt_Outer(BufferReader& r); +inline void write_opt_list_string(BufferWriter& w, const std::optional>& v); +inline std::optional> read_opt_list_string(BufferReader& r); +inline void write_opt_Mode(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_Mode(BufferReader& r); +inline void write_list_opt_Widget(BufferWriter& w, const std::vector>& v); +inline std::vector> read_list_opt_Widget(BufferReader& r); +inline void write_list_Outer(BufferWriter& w, const std::vector& v); +inline std::vector read_list_Outer(BufferReader& r); + +/** Writes one `Leaf` to a value buffer. */ inline void write_Leaf(BufferWriter& w, const Leaf& v) { w.write_f32(v.weight); } -/** Decodes a `Leaf` from the value-buffer wire format. */ +/** Reads one `Leaf` from a value buffer. */ inline Leaf read_Leaf(BufferReader& r) { - float f_weight = r.read_f32(); - return Leaf{std::move(f_weight)}; + return Leaf{ + r.read_f32(), + }; } -/** Encodes a `Inner` in the value-buffer wire format. */ +/** Writes one `Inner` to a value buffer. */ inline void write_Inner(BufferWriter& w, const Inner& v) { detail::write_Leaf(w, v.leaf); - w.write_len(v.leaves.size()); - for (const auto& item0 : v.leaves) { - detail::write_Leaf(w, item0); - } + detail::write_list_Leaf(w, v.leaves); } -/** Decodes a `Inner` from the value-buffer wire format. */ +/** Reads one `Inner` from a value buffer. */ inline Inner read_Inner(BufferReader& r) { - Leaf f_leaf = detail::read_Leaf(r); - std::vector f_leaves{}; - { - size_t f_leaves_n = r.read_count(); - f_leaves.reserve(r.reserve_hint(f_leaves_n)); - for (size_t f_leaves_i = 0; f_leaves_i < f_leaves_n; ++f_leaves_i) { - Leaf f_leaves_item = detail::read_Leaf(r); - f_leaves.push_back(std::move(f_leaves_item)); - } - } - return Inner{std::move(f_leaf), std::move(f_leaves)}; + return Inner{ + detail::read_Leaf(r), + detail::read_list_Leaf(r), + }; } -/** Encodes a `Outer` in the value-buffer wire format. */ +/** Writes one `Outer` to a value buffer. */ inline void write_Outer(BufferWriter& w, const Outer& v) { detail::write_Inner(w, v.inner); - w.write_option_flag(v.maybe_inner.has_value()); - if (v.maybe_inner.has_value()) { - detail::write_Inner(w, (*v.maybe_inner)); - } - w.write_len(v.by_name.size()); - for (const auto& kv0 : v.by_name) { - w.write_string(kv0.first); - detail::write_Inner(w, kv0.second); - } - w.write_len(v.modes.size()); - for (const auto& item0 : v.modes) { - w.write_i32(static_cast(item0)); - } - w.write_len(v.matrix.size()); - for (const auto& item0 : v.matrix) { - w.write_len(item0.size()); - for (const auto& item1 : item0) { - w.write_i32(item1); - } - } - w.write_len(v.sparse.size()); - for (const auto& item0 : v.sparse) { - w.write_option_flag(item0.has_value()); - if (item0.has_value()) { - w.write_i64((*item0)); - } - } - w.write_option_flag(v.blob.has_value()); - if (v.blob.has_value()) { - w.write_bytes((*v.blob)); - } - w.write_option_flag(v.note.has_value()); - if (v.note.has_value()) { - w.write_string((*v.note)); - } - w.write_len(v.by_id.size()); - for (const auto& kv0 : v.by_id) { - w.write_i64(kv0.first); - w.write_string(kv0.second); - } + detail::write_opt_Inner(w, v.maybe_inner); + detail::write_map_string_Inner(w, v.by_name); + detail::write_list_Mode(w, v.modes); + detail::write_list_list_i32(w, v.matrix); + detail::write_list_opt_i64(w, v.sparse); + detail::write_opt_bytes(w, v.blob); + detail::write_opt_string(w, v.note); + detail::write_map_i64_string(w, v.by_id); w.write_bool(v._hidden); w.write_u8(v.v2); } -/** Decodes a `Outer` from the value-buffer wire format. */ +/** Reads one `Outer` from a value buffer. */ inline Outer read_Outer(BufferReader& r) { - Inner f_inner = detail::read_Inner(r); - std::optional f_maybe_inner{}; - if (r.read_option_flag()) { - Inner f_maybe_inner_v = detail::read_Inner(r); - f_maybe_inner = std::move(f_maybe_inner_v); - } - std::unordered_map f_by_name{}; - { - size_t f_by_name_n = r.read_count(); - for (size_t f_by_name_i = 0; f_by_name_i < f_by_name_n; ++f_by_name_i) { - std::string f_by_name_key = r.read_string(); - Inner f_by_name_val = detail::read_Inner(r); - f_by_name.emplace(std::move(f_by_name_key), std::move(f_by_name_val)); - } - } - std::vector f_modes{}; - { - size_t f_modes_n = r.read_count(); - f_modes.reserve(r.reserve_hint(f_modes_n)); - for (size_t f_modes_i = 0; f_modes_i < f_modes_n; ++f_modes_i) { - Mode f_modes_item = static_cast(r.read_i32()); - f_modes.push_back(std::move(f_modes_item)); - } - } - std::vector> f_matrix{}; - { - size_t f_matrix_n = r.read_count(); - f_matrix.reserve(r.reserve_hint(f_matrix_n)); - for (size_t f_matrix_i = 0; f_matrix_i < f_matrix_n; ++f_matrix_i) { - std::vector f_matrix_item{}; - { - size_t f_matrix_item_n = r.read_count(); - f_matrix_item.reserve(r.reserve_hint(f_matrix_item_n)); - for (size_t f_matrix_item_i = 0; f_matrix_item_i < f_matrix_item_n; ++f_matrix_item_i) { - int32_t f_matrix_item_item = r.read_i32(); - f_matrix_item.push_back(std::move(f_matrix_item_item)); - } - } - f_matrix.push_back(std::move(f_matrix_item)); - } - } - std::vector> f_sparse{}; - { - size_t f_sparse_n = r.read_count(); - f_sparse.reserve(r.reserve_hint(f_sparse_n)); - for (size_t f_sparse_i = 0; f_sparse_i < f_sparse_n; ++f_sparse_i) { - std::optional f_sparse_item{}; - if (r.read_option_flag()) { - int64_t f_sparse_item_v = r.read_i64(); - f_sparse_item = std::move(f_sparse_item_v); - } - f_sparse.push_back(std::move(f_sparse_item)); - } - } - std::optional> f_blob{}; - if (r.read_option_flag()) { - std::vector f_blob_v = r.read_bytes(); - f_blob = std::move(f_blob_v); - } - std::optional f_note{}; - if (r.read_option_flag()) { - std::string f_note_v = r.read_string(); - f_note = std::move(f_note_v); - } - std::unordered_map f_by_id{}; - { - size_t f_by_id_n = r.read_count(); - for (size_t f_by_id_i = 0; f_by_id_i < f_by_id_n; ++f_by_id_i) { - int64_t f_by_id_key = r.read_i64(); - std::string f_by_id_val = r.read_string(); - f_by_id.emplace(std::move(f_by_id_key), std::move(f_by_id_val)); - } - } - bool f__hidden = r.read_bool(); - uint8_t f_v2 = r.read_u8(); - return Outer{std::move(f_inner), std::move(f_maybe_inner), std::move(f_by_name), std::move(f_modes), std::move(f_matrix), std::move(f_sparse), std::move(f_blob), std::move(f_note), std::move(f_by_id), std::move(f__hidden), std::move(f_v2)}; + return Outer{ + detail::read_Inner(r), + detail::read_opt_Inner(r), + detail::read_map_string_Inner(r), + detail::read_list_Mode(r), + detail::read_list_list_i32(r), + detail::read_list_opt_i64(r), + detail::read_opt_bytes(r), + detail::read_opt_string(r), + detail::read_map_i64_string(r), + r.read_bool(), + r.read_u8(), + }; } -/** Encodes a `Keywords` in the value-buffer wire format. */ +/** Writes one `Keywords` to a value buffer. */ inline void write_Keywords(BufferWriter& w, const Keywords& v) { w.write_i32(v.class_); w.write_i32(v.def); @@ -879,160 +1048,475 @@ inline void write_Keywords(BufferWriter& w, const Keywords& v) { w.write_i32(v.super); } -/** Decodes a `Keywords` from the value-buffer wire format. */ +/** Reads one `Keywords` from a value buffer. */ inline Keywords read_Keywords(BufferReader& r) { - int32_t f_class = r.read_i32(); - int32_t f_def = r.read_i32(); - int32_t f_import = r.read_i32(); - int32_t f_from = r.read_i32(); - int32_t f_func = r.read_i32(); - int32_t f_range = r.read_i32(); - int32_t f_register = r.read_i32(); - int32_t f_volatile = r.read_i32(); - int32_t f_new = r.read_i32(); - int32_t f_default = r.read_i32(); - int32_t f_end = r.read_i32(); - int32_t f_in = r.read_i32(); - int32_t f_is = r.read_i32(); - int32_t f_int = r.read_i32(); - int32_t f_var = r.read_i32(); - int32_t f_val = r.read_i32(); - int32_t f_self = r.read_i32(); - int32_t f_this = r.read_i32(); - int32_t f_object = r.read_i32(); - int32_t f_string = r.read_i32(); - int32_t f_case = r.read_i32(); - int32_t f_select = r.read_i32(); - int32_t f_switch = r.read_i32(); - int32_t f_void = r.read_i32(); - int32_t f_union = r.read_i32(); - int32_t f_namespace = r.read_i32(); - int32_t f_operator = r.read_i32(); - int32_t f_template = r.read_i32(); - int32_t f_package = r.read_i32(); - int32_t f_protocol = r.read_i32(); - int32_t f_guard = r.read_i32(); - int32_t f_defer = r.read_i32(); - int32_t f_inout = r.read_i32(); - int32_t f_internal = r.read_i32(); - int32_t f_when = r.read_i32(); - int32_t f_fun = r.read_i32(); - int32_t f_data = r.read_i32(); - int32_t f_late = r.read_i32(); - int32_t f_part = r.read_i32(); - int32_t f_out = r.read_i32(); - int32_t f_ref = r.read_i32(); - int32_t f_event = r.read_i32(); - int32_t f_base = r.read_i32(); - int32_t f_lambda = r.read_i32(); - int32_t f_yield = r.read_i32(); - int32_t f_nil = r.read_i32(); - int32_t f_null = r.read_i32(); - int32_t f_begin = r.read_i32(); - int32_t f_rescue = r.read_i32(); - int32_t f_module = r.read_i32(); - int32_t f_chan = r.read_i32(); - int32_t f_go = r.read_i32(); - int32_t f_let = r.read_i32(); - int32_t f_export = r.read_i32(); - int32_t f_delete = r.read_i32(); - int32_t f_typedef = r.read_i32(); - int32_t f_sizeof = r.read_i32(); - int32_t f_static = r.read_i32(); - int32_t f_super = r.read_i32(); - return Keywords{std::move(f_class), std::move(f_def), std::move(f_import), std::move(f_from), std::move(f_func), std::move(f_range), std::move(f_register), std::move(f_volatile), std::move(f_new), std::move(f_default), std::move(f_end), std::move(f_in), std::move(f_is), std::move(f_int), std::move(f_var), std::move(f_val), std::move(f_self), std::move(f_this), std::move(f_object), std::move(f_string), std::move(f_case), std::move(f_select), std::move(f_switch), std::move(f_void), std::move(f_union), std::move(f_namespace), std::move(f_operator), std::move(f_template), std::move(f_package), std::move(f_protocol), std::move(f_guard), std::move(f_defer), std::move(f_inout), std::move(f_internal), std::move(f_when), std::move(f_fun), std::move(f_data), std::move(f_late), std::move(f_part), std::move(f_out), std::move(f_ref), std::move(f_event), std::move(f_base), std::move(f_lambda), std::move(f_yield), std::move(f_nil), std::move(f_null), std::move(f_begin), std::move(f_rescue), std::move(f_module), std::move(f_chan), std::move(f_go), std::move(f_let), std::move(f_export), std::move(f_delete), std::move(f_typedef), std::move(f_sizeof), std::move(f_static), std::move(f_super)}; -} - -/** Encodes a `Retired` in the value-buffer wire format. */ + return Keywords{ + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + r.read_i32(), + }; +} + +/** Writes one `Retired` to a value buffer. */ inline void write_Retired(BufferWriter& w, const Retired& v) { w.write_i64(v.id); } -/** Decodes a `Retired` from the value-buffer wire format. */ +/** Reads one `Retired` from a value buffer. */ inline Retired read_Retired(BufferReader& r) { - int64_t f_id = r.read_i64(); - return Retired{std::move(f_id)}; + return Retired{ + r.read_i64(), + }; } -/** Encodes a `WidgetHolder` in the value-buffer wire format. */ +/** Writes one `WidgetHolder` to a value buffer. */ inline void write_WidgetHolder(BufferWriter& w, const WidgetHolder& v) { - w.write_u64(static_cast(reinterpret_cast(v.primary.clone_handle()))); - w.write_option_flag(v.fallback.has_value()); - if (v.fallback.has_value()) { - w.write_u64(static_cast(reinterpret_cast((*v.fallback).clone_handle()))); - } - w.write_len(v.extras.size()); - for (const auto& item0 : v.extras) { - w.write_u64(static_cast(reinterpret_cast(item0.clone_handle()))); - } - w.write_len(v.by_name.size()); - for (const auto& kv0 : v.by_name) { - w.write_string(kv0.first); - w.write_u64(static_cast(reinterpret_cast(kv0.second.clone_handle()))); - } + w.write_object(v.primary); + detail::write_opt_Widget(w, v.fallback); + detail::write_list_Widget(w, v.extras); + detail::write_map_string_Widget(w, v.by_name); } -/** Decodes a `WidgetHolder` from the value-buffer wire format. */ +/** Reads one `WidgetHolder` from a value buffer. */ inline WidgetHolder read_WidgetHolder(BufferReader& r) { - Widget f_primary = Widget(adopt, reinterpret_cast(static_cast(r.read_u64()))); - std::optional f_fallback{}; - if (r.read_option_flag()) { - Widget f_fallback_v = Widget(adopt, reinterpret_cast(static_cast(r.read_u64()))); - f_fallback = std::move(f_fallback_v); + return WidgetHolder{ + r.read_object(), + detail::read_opt_Widget(r), + detail::read_list_Widget(r), + detail::read_map_string_Widget(r), + }; +} + +/** Writes one `[Leaf]` to a value buffer. */ +inline void write_list_Leaf(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) detail::write_Leaf(w, item); +} + +/** Reads one `[Leaf]` from a value buffer. */ +inline std::vector read_list_Leaf(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(detail::read_Leaf(r)); + return v; +} + +/** Writes one `Inner?` to a value buffer. */ +inline void write_opt_Inner(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) detail::write_Inner(w, *v); +} + +/** Reads one `Inner?` from a value buffer. */ +inline std::optional read_opt_Inner(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(detail::read_Inner(r)); +} + +/** Writes one `{string:Inner}` to a value buffer. */ +inline void write_map_string_Inner(BufferWriter& w, const std::unordered_map& v) { + w.write_len(v.size()); + for (const auto& entry : v) { + w.write_string(entry.first); + detail::write_Inner(w, entry.second); } - std::vector f_extras{}; - { - size_t f_extras_n = r.read_count(); - f_extras.reserve(r.reserve_hint(f_extras_n)); - for (size_t f_extras_i = 0; f_extras_i < f_extras_n; ++f_extras_i) { - Widget f_extras_item = Widget(adopt, reinterpret_cast(static_cast(r.read_u64()))); - f_extras.push_back(std::move(f_extras_item)); - } +} + +/** Reads one `{string:Inner}` from a value buffer. */ +inline std::unordered_map read_map_string_Inner(BufferReader& r) { + size_t n = r.read_count(); + std::unordered_map v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) { + std::string key = r.read_string(); + Inner value = detail::read_Inner(r); + if (!v.emplace(std::move(key), std::move(value)).second) BufferReader::fail("repeated map key"); + } + return v; +} + +/** Writes one `[Mode]` to a value buffer. */ +inline void write_list_Mode(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) w.write_i32(static_cast(item)); +} + +/** Reads one `[Mode]` from a value buffer. */ +inline std::vector read_list_Mode(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(static_cast(r.read_i32())); + return v; +} + +/** Writes one `[i32]` to a value buffer. */ +inline void write_list_i32(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) w.write_i32(item); +} + +/** Reads one `[i32]` from a value buffer. */ +inline std::vector read_list_i32(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(r.read_i32()); + return v; +} + +/** Writes one `[[i32]]` to a value buffer. */ +inline void write_list_list_i32(BufferWriter& w, const std::vector>& v) { + w.write_len(v.size()); + for (const auto& item : v) detail::write_list_i32(w, item); +} + +/** Reads one `[[i32]]` from a value buffer. */ +inline std::vector> read_list_list_i32(BufferReader& r) { + size_t n = r.read_count(); + std::vector> v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(detail::read_list_i32(r)); + return v; +} + +/** Writes one `i64?` to a value buffer. */ +inline void write_opt_i64(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_i64(*v); +} + +/** Reads one `i64?` from a value buffer. */ +inline std::optional read_opt_i64(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(r.read_i64()); +} + +/** Writes one `[i64?]` to a value buffer. */ +inline void write_list_opt_i64(BufferWriter& w, const std::vector>& v) { + w.write_len(v.size()); + for (const auto& item : v) detail::write_opt_i64(w, item); +} + +/** Reads one `[i64?]` from a value buffer. */ +inline std::vector> read_list_opt_i64(BufferReader& r) { + size_t n = r.read_count(); + std::vector> v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(detail::read_opt_i64(r)); + return v; +} + +/** Writes one `bytes?` to a value buffer. */ +inline void write_opt_bytes(BufferWriter& w, const std::optional>& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_bytes(*v); +} + +/** Reads one `bytes?` from a value buffer. */ +inline std::optional> read_opt_bytes(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional>(r.read_bytes()); +} + +/** Writes one `string?` to a value buffer. */ +inline void write_opt_string(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_string(*v); +} + +/** Reads one `string?` from a value buffer. */ +inline std::optional read_opt_string(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(r.read_string()); +} + +/** Writes one `{i64:string}` to a value buffer. */ +inline void write_map_i64_string(BufferWriter& w, const std::unordered_map& v) { + w.write_len(v.size()); + for (const auto& entry : v) { + w.write_i64(entry.first); + w.write_string(entry.second); } - std::unordered_map f_by_name{}; - { - size_t f_by_name_n = r.read_count(); - for (size_t f_by_name_i = 0; f_by_name_i < f_by_name_n; ++f_by_name_i) { - std::string f_by_name_key = r.read_string(); - Widget f_by_name_val = Widget(adopt, reinterpret_cast(static_cast(r.read_u64()))); - f_by_name.emplace(std::move(f_by_name_key), std::move(f_by_name_val)); - } +} + +/** Reads one `{i64:string}` from a value buffer. */ +inline std::unordered_map read_map_i64_string(BufferReader& r) { + size_t n = r.read_count(); + std::unordered_map v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) { + int64_t key = r.read_i64(); + std::string value = r.read_string(); + if (!v.emplace(std::move(key), std::move(value)).second) BufferReader::fail("repeated map key"); + } + return v; +} + +/** Writes one `Widget?` to a value buffer. */ +inline void write_opt_Widget(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_object(*v); +} + +/** Reads one `Widget?` from a value buffer. */ +inline std::optional read_opt_Widget(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(r.read_object()); +} + +/** Writes one `[Widget]` to a value buffer. */ +inline void write_list_Widget(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) w.write_object(item); +} + +/** Reads one `[Widget]` from a value buffer. */ +inline std::vector read_list_Widget(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(r.read_object()); + return v; +} + +/** Writes one `{string:Widget}` to a value buffer. */ +inline void write_map_string_Widget(BufferWriter& w, const std::unordered_map& v) { + w.write_len(v.size()); + for (const auto& entry : v) { + w.write_string(entry.first); + w.write_object(entry.second); } - return WidgetHolder{std::move(f_primary), std::move(f_fallback), std::move(f_extras), std::move(f_by_name)}; +} + +/** Reads one `{string:Widget}` from a value buffer. */ +inline std::unordered_map read_map_string_Widget(BufferReader& r) { + size_t n = r.read_count(); + std::unordered_map v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) { + std::string key = r.read_string(); + Widget value = r.read_object(); + if (!v.emplace(std::move(key), std::move(value)).second) BufferReader::fail("repeated map key"); + } + return v; +} + +/** Writes one `[string]` to a value buffer. */ +inline void write_list_string(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) w.write_string(item); +} + +/** Reads one `[string]` from a value buffer. */ +inline std::vector read_list_string(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(r.read_string()); + return v; +} + +/** Writes one `i32?` to a value buffer. */ +inline void write_opt_i32(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_i32(*v); +} + +/** Reads one `i32?` from a value buffer. */ +inline std::optional read_opt_i32(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(r.read_i32()); +} + +/** Writes one `Outer?` to a value buffer. */ +inline void write_opt_Outer(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) detail::write_Outer(w, *v); +} + +/** Reads one `Outer?` from a value buffer. */ +inline std::optional read_opt_Outer(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(detail::read_Outer(r)); +} + +/** Writes one `[Outer?]` to a value buffer. */ +inline void write_list_opt_Outer(BufferWriter& w, const std::vector>& v) { + w.write_len(v.size()); + for (const auto& item : v) detail::write_opt_Outer(w, item); +} + +/** Reads one `[Outer?]` from a value buffer. */ +inline std::vector> read_list_opt_Outer(BufferReader& r) { + size_t n = r.read_count(); + std::vector> v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(detail::read_opt_Outer(r)); + return v; +} + +/** Writes one `[Outer?]?` to a value buffer. */ +inline void write_opt_list_opt_Outer(BufferWriter& w, const std::optional>>& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) detail::write_list_opt_Outer(w, *v); +} + +/** Reads one `[Outer?]?` from a value buffer. */ +inline std::optional>> read_opt_list_opt_Outer(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional>>(detail::read_list_opt_Outer(r)); +} + +/** Writes one `[string]?` to a value buffer. */ +inline void write_opt_list_string(BufferWriter& w, const std::optional>& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) detail::write_list_string(w, *v); +} + +/** Reads one `[string]?` from a value buffer. */ +inline std::optional> read_opt_list_string(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional>(detail::read_list_string(r)); +} + +/** Writes one `Mode?` to a value buffer. */ +inline void write_opt_Mode(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_i32(static_cast(*v)); +} + +/** Reads one `Mode?` from a value buffer. */ +inline std::optional read_opt_Mode(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(static_cast(r.read_i32())); +} + +/** Writes one `[Widget?]` to a value buffer. */ +inline void write_list_opt_Widget(BufferWriter& w, const std::vector>& v) { + w.write_len(v.size()); + for (const auto& item : v) detail::write_opt_Widget(w, item); +} + +/** Reads one `[Widget?]` from a value buffer. */ +inline std::vector> read_list_opt_Widget(BufferReader& r) { + size_t n = r.read_count(); + std::vector> v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(detail::read_opt_Widget(r)); + return v; +} + +/** Writes one `[Outer]` to a value buffer. */ +inline void write_list_Outer(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) detail::write_Outer(w, item); +} + +/** Reads one `[Outer]` from a value buffer. */ +inline std::vector read_list_Outer(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(detail::read_Outer(r)); + return v; } } // namespace detail -/** Typed errors reported by the `edge_cases` module's throwing functions. */ +/** The errors the `edge_cases` module's throwing calls report; catch a code's subclass or this. */ class EdgeErrorsError : public Error { public: - EdgeErrorsError(int32_t code, const std::string& msg) : Error(code, msg) {} + /** Builds an error carrying a domain `code` and `message`. */ + EdgeErrorsError(int32_t code, const std::string& message) : Error(code, message) {} }; -/** Resource is busy */ +/** + * Resource is busy + * + * Code 7. + */ class BusyError : public EdgeErrorsError { public: - explicit BusyError(const std::string& msg) : EdgeErrorsError(7, msg) {} + /** Builds the error with the producer's `message` and the code's fields. */ + explicit BusyError(const std::string& message) : EdgeErrorsError(7, message) {} }; namespace detail { -/** Map a `edge_cases` error code and payload to its typed exception (the runtime exception for negative codes, EdgeErrorsError for unknown codes). */ -inline std::exception_ptr make_edge_cases_error(int32_t code, const std::string& msg, const uint8_t* payload_ptr, size_t payload_len) { - if (code < 0) return make_error(code, msg); +/** The exception for a failed `EdgeErrorsError` call: its code's subclass, Cancelled, or the root Error. */ +inline std::exception_ptr make_edge_cases_error(int32_t code, const std::string& message, const uint8_t* payload_ptr, size_t payload_len) { + if (code == -5) return std::make_exception_ptr(Cancelled(message)); + if (code < 0) return std::make_exception_ptr(Error(code, message)); (void)payload_ptr; (void)payload_len; switch (code) { - case 7: return std::make_exception_ptr(BusyError(msg)); - default: return std::make_exception_ptr(EdgeErrorsError(code, msg)); + case 7: return std::make_exception_ptr(BusyError(message)); + default: return std::make_exception_ptr(EdgeErrorsError(code, message)); } } -/** Throw the typed `edge_cases` exception if `err` carries a nonzero code. */ +/** Throws the typed `EdgeErrorsError` exception if `err` carries a nonzero code. */ inline void check_edge_cases(edge_cases_error& err) { if (err.code == 0) return; - std::string msg = error_message(err); - std::exception_ptr ex = make_edge_cases_error(err.code, msg, err.payload_ptr, err.payload_len); + std::exception_ptr ex = make_edge_cases_error(err.code, error_message(err), err.payload_ptr, err.payload_len); edge_cases_error_clear(&err); std::rethrow_exception(ex); } @@ -1042,7 +1526,7 @@ inline void check_edge_cases(edge_cases_error& err) { /** * Callback interface carrying composite payloads and objects * - * Implement this interface and pass a `std::shared_ptr` to any function that accepts one. The producer may call the methods from any thread until it releases its last reference to the implementation. An exception thrown by a method is reported to the producer as a foreign error (code -4) and aborts the call that triggered it. + * Subclass it and pass a `std::shared_ptr` where the API takes a `BatchListener`. The producer may call the methods from any thread, so they must be thread-safe, until it releases the implementation (also from any thread). An exception a method throws reaches the producer as a callback failure (code -4). */ class BatchListener { public: @@ -1050,43 +1534,29 @@ public: /** Composite payloads cross as value buffers */ virtual void on_batch(const std::vector& items, const std::vector& raw, const std::optional& first, Mode mode) = 0; + /** Objects cross as strong references the consumer must release */ virtual void on_widget(Widget widget, std::optional fallback, const std::vector& children) = 0; + virtual void on_tick() = 0; + /** Scalar return travels back through the vtable */ virtual bool should_continue(int32_t class_) = 0; + /** Enum return travels back as its integer discriminant */ virtual Mode pick_mode() = 0; }; namespace detail { -/** Vtable trampolines adapting a `BatchListener` implementation to `edge_cases_edge_cases_BatchListener_vtable`. */ +/** Trampolines adapting a `BatchListener` implementation to `edge_cases_edge_cases_BatchListener_vtable`. */ struct BatchListener_trampolines { static void on_batch(void* ctx, const uint8_t* items_ptr, size_t items_len, const uint8_t* raw_ptr, size_t raw_len, const uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err) { try { BatchListener& impl = **static_cast*>(ctx); - detail::BufferReader items_r(items_ptr, items_len); - std::vector items_val{}; - { - size_t items_val_n = items_r.read_count(); - items_val.reserve(items_r.reserve_hint(items_val_n)); - for (size_t items_val_i = 0; items_val_i < items_val_n; ++items_val_i) { - Outer items_val_item = detail::read_Outer(items_r); - items_val.push_back(std::move(items_val_item)); - } - } - items_r.expect_end(); - std::vector raw_val = detail::borrow_bytes(raw_ptr, raw_len); - detail::BufferReader first_r(first_ptr, first_len); - std::optional first_val{}; - if (first_r.read_option_flag()) { - Outer first_val_v = detail::read_Outer(first_r); - first_val = std::move(first_val_v); - } - first_r.expect_end(); - impl.on_batch(items_val, raw_val, first_val, static_cast(static_cast(mode))); - return; + std::vector items_arg = detail::decode(items_ptr, items_len, &detail::read_list_Outer); + std::optional first_arg = detail::decode(first_ptr, first_len, &detail::read_opt_Outer); + impl.on_batch(std::move(items_arg), detail::borrow_bytes(raw_ptr, raw_len), std::move(first_arg), static_cast(static_cast(mode))); } catch (const std::exception& e) { edge_cases_error_set(out_err, -4, e.what()); } catch (...) { @@ -1095,24 +1565,13 @@ struct BatchListener_trampolines { } static void on_widget(void* ctx, edge_cases_edge_cases_Widget* widget, edge_cases_edge_cases_Widget* fallback, const uint8_t* children_ptr, size_t children_len, edge_cases_error* out_err) { - Widget widget_val(adopt, widget); - std::optional fallback_val; - if (fallback) fallback_val.emplace(adopt, fallback); + Widget widget_arg(adopt, widget); + std::optional fallback_arg; + if (fallback != nullptr) fallback_arg.emplace(adopt, fallback); try { BatchListener& impl = **static_cast*>(ctx); - detail::BufferReader children_r(children_ptr, children_len); - std::vector children_val{}; - { - size_t children_val_n = children_r.read_count(); - children_val.reserve(children_r.reserve_hint(children_val_n)); - for (size_t children_val_i = 0; children_val_i < children_val_n; ++children_val_i) { - Widget children_val_item = Widget(adopt, reinterpret_cast(static_cast(children_r.read_u64()))); - children_val.push_back(std::move(children_val_item)); - } - } - children_r.expect_end(); - impl.on_widget(std::move(widget_val), std::move(fallback_val), children_val); - return; + std::vector children_arg = detail::decode(children_ptr, children_len, &detail::read_list_Widget); + impl.on_widget(std::move(widget_arg), std::move(fallback_arg), std::move(children_arg)); } catch (const std::exception& e) { edge_cases_error_set(out_err, -4, e.what()); } catch (...) { @@ -1124,7 +1583,6 @@ struct BatchListener_trampolines { try { BatchListener& impl = **static_cast*>(ctx); impl.on_tick(); - return; } catch (const std::exception& e) { edge_cases_error_set(out_err, -4, e.what()); } catch (...) { @@ -1156,21 +1614,23 @@ struct BatchListener_trampolines { return edge_cases_edge_cases_Mode{}; } - /** Releases the implementation box once the producer drops its last reference. */ + /** Deletes the implementation box once the producer releases it. */ static void free_ctx(void* ctx) { delete static_cast*>(ctx); } }; -/** The process-wide vtable every `BatchListener` implementation is passed with. */ +/** The vtable every `BatchListener` implementation is passed with. */ inline const edge_cases_edge_cases_BatchListener_vtable& BatchListener_vtable() { static const edge_cases_edge_cases_BatchListener_vtable vtable = { + static_cast(sizeof(edge_cases_edge_cases_BatchListener_vtable)), + 0, + &BatchListener_trampolines::free_ctx, &BatchListener_trampolines::on_batch, &BatchListener_trampolines::on_widget, &BatchListener_trampolines::on_tick, &BatchListener_trampolines::should_continue, &BatchListener_trampolines::pick_mode, - &BatchListener_trampolines::free_ctx, }; return vtable; } @@ -1180,25 +1640,25 @@ inline const edge_cases_edge_cases_BatchListener_vtable& BatchListener_vtable() /** * Callback interface whose name and methods are keywords * - * Implement this interface and pass a `std::shared_ptr` to any function that accepts one. The producer may call the methods from any thread until it releases its last reference to the implementation. An exception thrown by a method is reported to the producer as a foreign error (code -4) and aborts the call that triggered it. + * Subclass it and pass a `std::shared_ptr` where the API takes a `Keyword`. The producer may call the methods from any thread, so they must be thread-safe, until it releases the implementation (also from any thread). An exception a method throws reaches the producer as a callback failure (code -4). */ class Keyword { public: virtual ~Keyword() = default; virtual void delete_(int32_t self) = 0; + virtual int32_t import() = 0; }; namespace detail { -/** Vtable trampolines adapting a `Keyword` implementation to `edge_cases_edge_cases_Keyword_vtable`. */ +/** Trampolines adapting a `Keyword` implementation to `edge_cases_edge_cases_Keyword_vtable`. */ struct Keyword_trampolines { static void delete_(void* ctx, int32_t self, edge_cases_error* out_err) { try { Keyword& impl = **static_cast*>(ctx); impl.delete_(self); - return; } catch (const std::exception& e) { edge_cases_error_set(out_err, -4, e.what()); } catch (...) { @@ -1218,18 +1678,20 @@ struct Keyword_trampolines { return int32_t{}; } - /** Releases the implementation box once the producer drops its last reference. */ + /** Deletes the implementation box once the producer releases it. */ static void free_ctx(void* ctx) { delete static_cast*>(ctx); } }; -/** The process-wide vtable every `Keyword` implementation is passed with. */ +/** The vtable every `Keyword` implementation is passed with. */ inline const edge_cases_edge_cases_Keyword_vtable& Keyword_vtable() { static const edge_cases_edge_cases_Keyword_vtable vtable = { + static_cast(sizeof(edge_cases_edge_cases_Keyword_vtable)), + 0, + &Keyword_trampolines::free_ctx, &Keyword_trampolines::delete_, &Keyword_trampolines::import, - &Keyword_trampolines::free_ctx, }; return vtable; } @@ -1238,19 +1700,16 @@ inline const edge_cases_edge_cases_Keyword_vtable& Keyword_vtable() { inline void Widget::refresh() const { edge_cases_error err{}; - edge_cases_edge_cases_Widget_refresh(handle_, &err); + edge_cases_edge_cases_Widget_refresh(raw_, &err); detail::check_edge_cases(err); } inline std::string Widget::rename(std::string_view class_, const std::optional& import) const { detail::BufferWriter import_buf; - import_buf.write_option_flag(import.has_value()); - if (import.has_value()) { - import_buf.write_string((*import)); - } + detail::write_opt_string(import_buf, import); size_t out_len = 0; edge_cases_error err{}; - auto result = edge_cases_edge_cases_Widget_rename(handle_, detail::utf8(class_), class_.size(), import_buf.data(), import_buf.size(), &out_len, &err); + auto result = edge_cases_edge_cases_Widget_rename(raw_, detail::utf8(class_), class_.size(), import_buf.data(), import_buf.size(), &out_len, &err); detail::check(err); return detail::take_string(result, out_len); } @@ -1258,18 +1717,14 @@ inline std::string Widget::rename(std::string_view class_, const std::optional Widget::fetch(int64_t id) const { auto promise = std::make_unique>(); auto future = promise->get_future(); - edge_cases_edge_cases_Widget_fetch(handle_, id, [](void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { + edge_cases_edge_cases_Widget_fetch(raw_, id, [](void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_edge_cases_error(err->code, msg, err->payload_ptr, err->payload_len)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_edge_cases_error(err->code, message, err->payload_ptr, err->payload_len)); } else { - detail::BufferGuard result_guard(result_ptr, result_len); - detail::BufferReader result_r(result_ptr, result_len); - Outer value = detail::read_Outer(result_r); - result_r.expect_end(); - p->set_value(std::move(value)); + p->set_value(detail::take(result_ptr, result_len, &detail::read_Outer)); } } catch (...) { p->set_exception(std::current_exception()); @@ -1282,11 +1737,24 @@ inline std::future Widget::fetch(int64_t id) const { inline std::string Widget::old_name() const { size_t out_len = 0; edge_cases_error err{}; - auto result = edge_cases_edge_cases_Widget_old_name(handle_, &out_len, &err); + auto result = edge_cases_edge_cases_Widget_old_name(raw_, &out_len, &err); detail::check(err); return detail::take_string(result, out_len); } +inline bool Widget::close() const { + edge_cases_error err{}; + auto result = edge_cases_edge_cases_Widget_close(raw_, &err); + detail::check(err); + return result; +} + +inline void Widget::dispose(bool force) const { + edge_cases_error err{}; + edge_cases_edge_cases_Widget_dispose(raw_, force, &err); + detail::check(err); +} + inline Vault Vault::open(std::string_view path) { check_library(); edge_cases_error err{}; @@ -1297,10 +1765,16 @@ inline Vault Vault::open(std::string_view path) { inline std::optional Vault::peek() const { edge_cases_error err{}; - auto result = edge_cases_edge_cases_Vault_peek(handle_, &err); + auto result = edge_cases_edge_cases_Vault_peek(raw_, &err); detail::check(err); - if (!result) return std::nullopt; - return Widget(adopt, result); + return result != nullptr ? std::optional(std::in_place, adopt, result) : std::nullopt; +} + +inline int64_t Vault::handle_() const { + edge_cases_error err{}; + auto result = edge_cases_edge_cases_Vault_handle(raw_, &err); + detail::check(err); + return result; } inline bool Vault::probe() { @@ -1311,6 +1785,15 @@ inline bool Vault::probe() { return result; } +inline std::string Vault::to_string() { + check_library(); + size_t out_len = 0; + edge_cases_error err{}; + auto result = edge_cases_edge_cases_Vault_to_string(&out_len, &err); + detail::check(err); + return detail::take_string(result, out_len); +} + namespace edge_cases { /** No parameters and no result */ @@ -1325,15 +1808,9 @@ inline void noop() { inline int32_t select(int32_t class_, std::string_view from, const std::vector& func, const std::optional& range, bool type_) { check_library(); detail::BufferWriter func_buf; - func_buf.write_len(func.size()); - for (const auto& item0 : func) { - func_buf.write_string(item0); - } + detail::write_list_string(func_buf, func); detail::BufferWriter range_buf; - range_buf.write_option_flag(range.has_value()); - if (range.has_value()) { - range_buf.write_i32((*range)); - } + detail::write_opt_i32(range_buf, range); edge_cases_error err{}; auto result = edge_cases_edge_cases_select(class_, detail::utf8(from), from.size(), func_buf.data(), func_buf.size(), range_buf.data(), range_buf.size(), type_, &err); detail::check(err); @@ -1344,91 +1821,33 @@ inline int32_t select(int32_t class_, std::string_view from, const std::vector>> nested_optional(const std::optional>>& items) { check_library(); detail::BufferWriter items_buf; - items_buf.write_option_flag(items.has_value()); - if (items.has_value()) { - items_buf.write_len((*items).size()); - for (const auto& item0 : (*items)) { - items_buf.write_option_flag(item0.has_value()); - if (item0.has_value()) { - detail::write_Outer(items_buf, (*item0)); - } - } - } + detail::write_opt_list_opt_Outer(items_buf, items); size_t out_len = 0; edge_cases_error err{}; auto result = edge_cases_edge_cases_nested_optional(items_buf.data(), items_buf.size(), &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - std::optional>> ret{}; - if (result_r.read_option_flag()) { - std::vector> ret_v{}; - { - size_t ret_v_n = result_r.read_count(); - ret_v.reserve(result_r.reserve_hint(ret_v_n)); - for (size_t ret_v_i = 0; ret_v_i < ret_v_n; ++ret_v_i) { - std::optional ret_v_item{}; - if (result_r.read_option_flag()) { - Outer ret_v_item_v = detail::read_Outer(result_r); - ret_v_item = std::move(ret_v_item_v); - } - ret_v.push_back(std::move(ret_v_item)); - } - } - ret = std::move(ret_v); - } - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_opt_list_opt_Outer); } inline std::optional optional_params(const std::optional& text, const std::optional& count, const std::optional& outer, const std::optional>& tags, const std::optional>& blob, const std::optional& mode) { check_library(); detail::BufferWriter text_buf; - text_buf.write_option_flag(text.has_value()); - if (text.has_value()) { - text_buf.write_string((*text)); - } + detail::write_opt_string(text_buf, text); detail::BufferWriter count_buf; - count_buf.write_option_flag(count.has_value()); - if (count.has_value()) { - count_buf.write_i32((*count)); - } + detail::write_opt_i32(count_buf, count); detail::BufferWriter outer_buf; - outer_buf.write_option_flag(outer.has_value()); - if (outer.has_value()) { - detail::write_Outer(outer_buf, (*outer)); - } + detail::write_opt_Outer(outer_buf, outer); detail::BufferWriter tags_buf; - tags_buf.write_option_flag(tags.has_value()); - if (tags.has_value()) { - tags_buf.write_len((*tags).size()); - for (const auto& item0 : (*tags)) { - tags_buf.write_string(item0); - } - } + detail::write_opt_list_string(tags_buf, tags); detail::BufferWriter blob_buf; - blob_buf.write_option_flag(blob.has_value()); - if (blob.has_value()) { - blob_buf.write_bytes((*blob)); - } + detail::write_opt_bytes(blob_buf, blob); detail::BufferWriter mode_buf; - mode_buf.write_option_flag(mode.has_value()); - if (mode.has_value()) { - mode_buf.write_i32(static_cast((*mode))); - } + detail::write_opt_Mode(mode_buf, mode); size_t out_len = 0; edge_cases_error err{}; auto result = edge_cases_edge_cases_optional_params(text_buf.data(), text_buf.size(), count_buf.data(), count_buf.size(), outer_buf.data(), outer_buf.size(), tags_buf.data(), tags_buf.size(), blob_buf.data(), blob_buf.size(), mode_buf.data(), mode_buf.size(), &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - std::optional ret{}; - if (result_r.read_option_flag()) { - Mode ret_v = static_cast(result_r.read_i32()); - ret = std::move(ret_v); - } - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_opt_Mode); } /** Two callback interface parameters on one function */ @@ -1447,92 +1866,72 @@ inline void register_batch(std::shared_ptr listener, std::shared_ inline WidgetHolder widgets_everywhere(const std::vector& list, const std::unordered_map& by_name, const std::vector>& sparse, const WidgetHolder& holder) { check_library(); detail::BufferWriter list_buf; - list_buf.write_len(list.size()); - for (const auto& item0 : list) { - list_buf.write_u64(static_cast(reinterpret_cast(item0.clone_handle()))); - } + detail::write_list_Widget(list_buf, list); detail::BufferWriter by_name_buf; - by_name_buf.write_len(by_name.size()); - for (const auto& kv0 : by_name) { - by_name_buf.write_string(kv0.first); - by_name_buf.write_u64(static_cast(reinterpret_cast(kv0.second.clone_handle()))); - } + detail::write_map_string_Widget(by_name_buf, by_name); detail::BufferWriter sparse_buf; - sparse_buf.write_len(sparse.size()); - for (const auto& item0 : sparse) { - sparse_buf.write_option_flag(item0.has_value()); - if (item0.has_value()) { - sparse_buf.write_u64(static_cast(reinterpret_cast((*item0).clone_handle()))); - } - } + detail::write_list_opt_Widget(sparse_buf, sparse); detail::BufferWriter holder_buf; detail::write_WidgetHolder(holder_buf, holder); size_t out_len = 0; edge_cases_error err{}; auto result = edge_cases_edge_cases_widgets_everywhere(list_buf.data(), list_buf.size(), by_name_buf.data(), by_name_buf.size(), sparse_buf.data(), sparse_buf.size(), holder_buf.data(), holder_buf.size(), &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - WidgetHolder ret = detail::read_WidgetHolder(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_WidgetHolder); } /** - * A lazy, move-only range over the `Widget` elements produced by `widget_stream()`. + * A lazy, move-only range over the `Widget` elements `widget_stream()` produces. * - * Each iteration step pulls exactly one element from the producer, so results stream in constant memory. The range owns the producer-side iterator and releases it exactly once: eagerly when the range is exhausted, or from the destructor when iteration is abandoned early. + * Each step pulls one element from the producer, so results stream in constant memory. The range releases the producer iterator exactly once: when it's exhausted, or from the destructor when iteration stops early. */ class WidgetStreamIterator { edge_cases_edge_cases_WidgetStreamIterator* handle_; public: - /** Adopts ownership of a raw producer iterator. */ + /** Adopts a producer iterator. */ explicit WidgetStreamIterator(adopt_t, edge_cases_edge_cases_WidgetStreamIterator* h) noexcept : handle_(h) {} + /** Releases the producer iterator if it's still held. */ ~WidgetStreamIterator() { - if (handle_) edge_cases_edge_cases_WidgetStreamIterator_destroy(handle_); + if (handle_ != nullptr) edge_cases_edge_cases_WidgetStreamIterator_destroy(handle_); } WidgetStreamIterator(const WidgetStreamIterator&) = delete; WidgetStreamIterator& operator=(const WidgetStreamIterator&) = delete; - WidgetStreamIterator(WidgetStreamIterator&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s iterator; `other` becomes empty. */ + WidgetStreamIterator(WidgetStreamIterator&& other) noexcept : handle_(other.handle_) { other.handle_ = nullptr; } + /** Releases the current iterator and takes over `other`'s. */ WidgetStreamIterator& operator=(WidgetStreamIterator&& other) noexcept { if (this != &other) { - if (handle_) edge_cases_edge_cases_WidgetStreamIterator_destroy(handle_); + if (handle_ != nullptr) edge_cases_edge_cases_WidgetStreamIterator_destroy(handle_); handle_ = other.handle_; other.handle_ = nullptr; } return *this; } - /** Pulls the next element from the producer, or `std::nullopt` once exhausted (which releases the producer iterator eagerly). A producer error throws the generic Error after releasing the iterator. */ + /** Pulls the next element, or `std::nullopt` once exhausted (which releases the producer iterator). A producer error releases the iterator and throws InternalError. */ std::optional next() { - if (!handle_) return std::nullopt; + if (handle_ == nullptr) return std::nullopt; edge_cases_error err{}; edge_cases_edge_cases_Widget* item{}; int32_t has_item = edge_cases_edge_cases_WidgetStreamIterator_next(handle_, &item, &err); - if (err.code != 0) { + if (err.code != 0 || has_item == 0) { edge_cases_edge_cases_WidgetStreamIterator_destroy(handle_); handle_ = nullptr; detail::check(err); - } - if (has_item == 0) { - edge_cases_edge_cases_WidgetStreamIterator_destroy(handle_); - handle_ = nullptr; return std::nullopt; } - return Widget(adopt, item); + return std::optional(std::in_place, Widget(adopt, item)); } /** Sentinel type marking the end of the range. */ struct sentinel {}; - /** Single-pass input iterator; each increment pulls one element from the producer. */ + /** Single-pass input iterator; each increment pulls one element. */ class iterator { WidgetStreamIterator* range_; std::optional current_; @@ -1564,7 +1963,7 @@ public: /** * Iterator of objects * - * @return A lazy `WidgetStreamIterator` range that streams one element per iteration step and releases the producer iterator when exhausted or destroyed. + * @return A lazy `WidgetStreamIterator` range that pulls one element per step and releases the producer iterator when exhausted or destroyed. */ inline WidgetStreamIterator widget_stream() { check_library(); @@ -1574,7 +1973,12 @@ inline WidgetStreamIterator widget_stream() { return WidgetStreamIterator(adopt, iter); } -/** Async returning an object */ +/** + * Async returning an object + * + * @return A future settled from a producer thread; get() rethrows the call's error. + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ inline std::future async_widget() { check_library(); auto promise = std::make_unique>(); @@ -1583,8 +1987,8 @@ inline std::future async_widget() { std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_edge_cases_error(err->code, msg, err->payload_ptr, err->payload_len)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_edge_cases_error(err->code, message, err->payload_ptr, err->payload_len)); } else { p->set_value(Widget(adopt, result)); } @@ -1605,14 +2009,17 @@ inline Widget make_widget(std::string_view label) { return Widget(adopt, result); } -/** Optional interface return */ +/** + * Optional interface return + * + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ inline std::optional find_widget(int64_t id) { check_library(); edge_cases_error err{}; auto result = edge_cases_edge_cases_find_widget(id, &err); detail::check_edge_cases(err); - if (!result) return std::nullopt; - return Widget(adopt, result); + return result != nullptr ? std::optional(std::in_place, adopt, result) : std::nullopt; } /** Interface parameter, including an optional one */ @@ -1626,61 +2033,58 @@ inline std::string inspect_widget(const Widget& widget, const std::optional next() { - if (!handle_) return std::nullopt; + if (handle_ == nullptr) return std::nullopt; edge_cases_error err{}; const uint8_t* item{}; size_t item_len = 0; int32_t has_item = edge_cases_edge_cases_StringStreamIterator_next(handle_, &item, &item_len, &err); - if (err.code != 0) { + if (err.code != 0 || has_item == 0) { edge_cases_edge_cases_StringStreamIterator_destroy(handle_); handle_ = nullptr; detail::check(err); - } - if (has_item == 0) { - edge_cases_edge_cases_StringStreamIterator_destroy(handle_); - handle_ = nullptr; return std::nullopt; } - return detail::take_string(item, item_len); + return std::optional(std::in_place, detail::take_string(item, item_len)); } /** Sentinel type marking the end of the range. */ struct sentinel {}; - /** Single-pass input iterator; each increment pulls one element from the producer. */ + /** Single-pass input iterator; each increment pulls one element. */ class iterator { StringStreamIterator* range_; std::optional current_; @@ -1712,7 +2116,7 @@ public: /** * Iterator of strings * - * @return A lazy `StringStreamIterator` range that streams one element per iteration step and releases the producer iterator when exhausted or destroyed. + * @return A lazy `StringStreamIterator` range that pulls one element per step and releases the producer iterator when exhausted or destroyed. */ inline StringStreamIterator string_stream() { check_library(); @@ -1723,60 +2127,57 @@ inline StringStreamIterator string_stream() { } /** - * A lazy, move-only range over the `int32_t` elements produced by `scalar_stream()`. + * A lazy, move-only range over the `int32_t` elements `scalar_stream()` produces. * - * Each iteration step pulls exactly one element from the producer, so results stream in constant memory. The range owns the producer-side iterator and releases it exactly once: eagerly when the range is exhausted, or from the destructor when iteration is abandoned early. + * Each step pulls one element from the producer, so results stream in constant memory. The range releases the producer iterator exactly once: when it's exhausted, or from the destructor when iteration stops early. */ class ScalarStreamIterator { edge_cases_edge_cases_ScalarStreamIterator* handle_; public: - /** Adopts ownership of a raw producer iterator. */ + /** Adopts a producer iterator. */ explicit ScalarStreamIterator(adopt_t, edge_cases_edge_cases_ScalarStreamIterator* h) noexcept : handle_(h) {} + /** Releases the producer iterator if it's still held. */ ~ScalarStreamIterator() { - if (handle_) edge_cases_edge_cases_ScalarStreamIterator_destroy(handle_); + if (handle_ != nullptr) edge_cases_edge_cases_ScalarStreamIterator_destroy(handle_); } ScalarStreamIterator(const ScalarStreamIterator&) = delete; ScalarStreamIterator& operator=(const ScalarStreamIterator&) = delete; - ScalarStreamIterator(ScalarStreamIterator&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s iterator; `other` becomes empty. */ + ScalarStreamIterator(ScalarStreamIterator&& other) noexcept : handle_(other.handle_) { other.handle_ = nullptr; } + /** Releases the current iterator and takes over `other`'s. */ ScalarStreamIterator& operator=(ScalarStreamIterator&& other) noexcept { if (this != &other) { - if (handle_) edge_cases_edge_cases_ScalarStreamIterator_destroy(handle_); + if (handle_ != nullptr) edge_cases_edge_cases_ScalarStreamIterator_destroy(handle_); handle_ = other.handle_; other.handle_ = nullptr; } return *this; } - /** Pulls the next element from the producer, or `std::nullopt` once exhausted (which releases the producer iterator eagerly). A producer error throws the generic Error after releasing the iterator. */ + /** Pulls the next element, or `std::nullopt` once exhausted (which releases the producer iterator). A producer error releases the iterator and throws InternalError. */ std::optional next() { - if (!handle_) return std::nullopt; + if (handle_ == nullptr) return std::nullopt; edge_cases_error err{}; int32_t item{}; int32_t has_item = edge_cases_edge_cases_ScalarStreamIterator_next(handle_, &item, &err); - if (err.code != 0) { + if (err.code != 0 || has_item == 0) { edge_cases_edge_cases_ScalarStreamIterator_destroy(handle_); handle_ = nullptr; detail::check(err); - } - if (has_item == 0) { - edge_cases_edge_cases_ScalarStreamIterator_destroy(handle_); - handle_ = nullptr; return std::nullopt; } - return item; + return std::optional(std::in_place, item); } /** Sentinel type marking the end of the range. */ struct sentinel {}; - /** Single-pass input iterator; each increment pulls one element from the producer. */ + /** Single-pass input iterator; each increment pulls one element. */ class iterator { ScalarStreamIterator* range_; std::optional current_; @@ -1808,7 +2209,7 @@ public: /** * Iterator of scalars * - * @return A lazy `ScalarStreamIterator` range that streams one element per iteration step and releases the producer iterator when exhausted or destroyed. + * @return A lazy `ScalarStreamIterator` range that pulls one element per step and releases the producer iterator when exhausted or destroyed. */ inline ScalarStreamIterator scalar_stream() { check_library(); @@ -1819,60 +2220,57 @@ inline ScalarStreamIterator scalar_stream() { } /** - * A lazy, move-only range over the `Mode` elements produced by `mode_stream()`. + * A lazy, move-only range over the `Mode` elements `mode_stream()` produces. * - * Each iteration step pulls exactly one element from the producer, so results stream in constant memory. The range owns the producer-side iterator and releases it exactly once: eagerly when the range is exhausted, or from the destructor when iteration is abandoned early. + * Each step pulls one element from the producer, so results stream in constant memory. The range releases the producer iterator exactly once: when it's exhausted, or from the destructor when iteration stops early. */ class ModeStreamIterator { edge_cases_edge_cases_ModeStreamIterator* handle_; public: - /** Adopts ownership of a raw producer iterator. */ + /** Adopts a producer iterator. */ explicit ModeStreamIterator(adopt_t, edge_cases_edge_cases_ModeStreamIterator* h) noexcept : handle_(h) {} + /** Releases the producer iterator if it's still held. */ ~ModeStreamIterator() { - if (handle_) edge_cases_edge_cases_ModeStreamIterator_destroy(handle_); + if (handle_ != nullptr) edge_cases_edge_cases_ModeStreamIterator_destroy(handle_); } ModeStreamIterator(const ModeStreamIterator&) = delete; ModeStreamIterator& operator=(const ModeStreamIterator&) = delete; - ModeStreamIterator(ModeStreamIterator&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s iterator; `other` becomes empty. */ + ModeStreamIterator(ModeStreamIterator&& other) noexcept : handle_(other.handle_) { other.handle_ = nullptr; } + /** Releases the current iterator and takes over `other`'s. */ ModeStreamIterator& operator=(ModeStreamIterator&& other) noexcept { if (this != &other) { - if (handle_) edge_cases_edge_cases_ModeStreamIterator_destroy(handle_); + if (handle_ != nullptr) edge_cases_edge_cases_ModeStreamIterator_destroy(handle_); handle_ = other.handle_; other.handle_ = nullptr; } return *this; } - /** Pulls the next element from the producer, or `std::nullopt` once exhausted (which releases the producer iterator eagerly). A producer error throws the generic Error after releasing the iterator. */ + /** Pulls the next element, or `std::nullopt` once exhausted (which releases the producer iterator). A producer error releases the iterator and throws InternalError. */ std::optional next() { - if (!handle_) return std::nullopt; + if (handle_ == nullptr) return std::nullopt; edge_cases_error err{}; edge_cases_edge_cases_Mode item{}; int32_t has_item = edge_cases_edge_cases_ModeStreamIterator_next(handle_, &item, &err); - if (err.code != 0) { + if (err.code != 0 || has_item == 0) { edge_cases_edge_cases_ModeStreamIterator_destroy(handle_); handle_ = nullptr; detail::check(err); - } - if (has_item == 0) { - edge_cases_edge_cases_ModeStreamIterator_destroy(handle_); - handle_ = nullptr; return std::nullopt; } - return static_cast(item); + return std::optional(std::in_place, static_cast(item)); } /** Sentinel type marking the end of the range. */ struct sentinel {}; - /** Single-pass input iterator; each increment pulls one element from the producer. */ + /** Single-pass input iterator; each increment pulls one element. */ class iterator { ModeStreamIterator* range_; std::optional current_; @@ -1904,7 +2302,7 @@ public: /** * Iterator of enums * - * @return A lazy `ModeStreamIterator` range that streams one element per iteration step and releases the producer iterator when exhausted or destroyed. + * @return A lazy `ModeStreamIterator` range that pulls one element per step and releases the producer iterator when exhausted or destroyed. */ inline ModeStreamIterator mode_stream() { check_library(); @@ -1914,7 +2312,11 @@ inline ModeStreamIterator mode_stream() { return ModeStreamIterator(adopt, iter); } -/** Async with no result */ +/** + * Async with no result + * + * @return A future settled from a producer thread; get() rethrows the call's error. + */ inline std::future async_void() { check_library(); auto promise = std::make_unique>(); @@ -1923,8 +2325,8 @@ inline std::future async_void() { std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { p->set_value(); } @@ -1936,6 +2338,7 @@ inline std::future async_void() { return future; } +/** @return A future settled from a producer thread; get() rethrows the call's error. */ inline std::future> async_bytes(uint32_t size) { check_library(); auto promise = std::make_unique>>(); @@ -1944,8 +2347,8 @@ inline std::future> async_bytes(uint32_t size) { std::unique_ptr>> p(static_cast>*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { p->set_value(detail::take_bytes(result_ptr, result_len)); } @@ -1957,6 +2360,10 @@ inline std::future> async_bytes(uint32_t size) { return future; } +/** + * @return A future settled from a producer thread; get() rethrows the call's error. + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ inline std::future async_struct() { check_library(); auto promise = std::make_unique>(); @@ -1965,14 +2372,10 @@ inline std::future async_struct() { std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_edge_cases_error(err->code, msg, err->payload_ptr, err->payload_len)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_edge_cases_error(err->code, message, err->payload_ptr, err->payload_len)); } else { - detail::BufferGuard result_guard(result_ptr, result_len); - detail::BufferReader result_r(result_ptr, result_len); - Outer value = detail::read_Outer(result_r); - result_r.expect_end(); - p->set_value(std::move(value)); + p->set_value(detail::take(result_ptr, result_len, &detail::read_Outer)); } } catch (...) { p->set_exception(std::current_exception()); @@ -1982,6 +2385,11 @@ inline std::future async_struct() { return future; } +/** + * @param cancel_token Cancels the call; the future then throws Cancelled. + * @return A future settled from a producer thread; get() rethrows the call's error. + * @throws EdgeErrorsError when the call fails with one of the domain's codes. + */ inline std::future> async_list(const CancelToken& cancel_token = CancelToken::none()) { check_library(); auto promise = std::make_unique>>(); @@ -1990,22 +2398,10 @@ inline std::future> async_list(const CancelToken& cancel_toke std::unique_ptr>> p(static_cast>*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_edge_cases_error(err->code, msg, err->payload_ptr, err->payload_len)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_edge_cases_error(err->code, message, err->payload_ptr, err->payload_len)); } else { - detail::BufferGuard result_guard(result_ptr, result_len); - detail::BufferReader result_r(result_ptr, result_len); - std::vector value{}; - { - size_t value_n = result_r.read_count(); - value.reserve(result_r.reserve_hint(value_n)); - for (size_t value_i = 0; value_i < value_n; ++value_i) { - Outer value_item = detail::read_Outer(result_r); - value.push_back(std::move(value_item)); - } - } - result_r.expect_end(); - p->set_value(std::move(value)); + p->set_value(detail::take(result_ptr, result_len, &detail::read_list_Outer)); } } catch (...) { p->set_exception(std::current_exception()); @@ -2015,6 +2411,7 @@ inline std::future> async_list(const CancelToken& cancel_toke return future; } +/** @return A future settled from a producer thread; get() rethrows the call's error. */ inline std::future> async_optional() { check_library(); auto promise = std::make_unique>>(); @@ -2023,18 +2420,10 @@ inline std::future> async_optional() { std::unique_ptr>> p(static_cast>*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { - detail::BufferGuard result_guard(result_ptr, result_len); - detail::BufferReader result_r(result_ptr, result_len); - std::optional value{}; - if (result_r.read_option_flag()) { - int64_t value_v = result_r.read_i64(); - value = std::move(value_v); - } - result_r.expect_end(); - p->set_value(std::move(value)); + p->set_value(detail::take(result_ptr, result_len, &detail::read_opt_i64)); } } catch (...) { p->set_exception(std::current_exception()); @@ -2044,6 +2433,7 @@ inline std::future> async_optional() { return future; } +/** @return A future settled from a producer thread; get() rethrows the call's error. */ inline std::future async_enum() { check_library(); auto promise = std::make_unique>(); @@ -2052,8 +2442,8 @@ inline std::future async_enum() { std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { p->set_value(static_cast(result)); } @@ -2065,6 +2455,7 @@ inline std::future async_enum() { return future; } +/** @return A future settled from a producer thread; get() rethrows the call's error. */ inline std::future async_keywords(std::string_view class_, int32_t import) { check_library(); auto promise = std::make_unique>(); @@ -2073,8 +2464,8 @@ inline std::future async_keywords(std::string_view class_, int32_t std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { p->set_value(detail::take_string(result_ptr, result_len)); } @@ -2094,11 +2485,7 @@ inline Keywords keywords_roundtrip(const Keywords& value) { edge_cases_error err{}; auto result = edge_cases_edge_cases_keywords_roundtrip(value_buf.data(), value_buf.size(), &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Keywords ret = detail::read_Keywords(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Keywords); } [[deprecated("Use Mode-based APIs")]] @@ -2108,11 +2495,7 @@ inline Retired legacy_value(Legacy value) { edge_cases_error err{}; auto result = edge_cases_edge_cases_legacy_value(static_cast(static_cast(value)), &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Retired ret = detail::read_Retired(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Retired); } } // namespace edge_cases diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__CMakeLists_txt.snap index 6eee449a..fc660879 100644 --- a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__CMakeLists_txt.snap +++ b/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__CMakeLists_txt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate cmake_minimum_required(VERSION 3.14) project(kitchen_sink_cpp VERSION 0.1.0 LANGUAGES CXX) diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__README_md.snap index e9bf90b7..734d3c47 100644 --- a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__README_md.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # kitchen_sink C++ bindings diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_h.snap b/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_h.snap deleted file mode 100644 index 14faeffb..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_h.snap +++ /dev/null @@ -1,219 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef KITCHEN_SINK_H -#define KITCHEN_SINK_H - -#include -#include -#include - -#ifndef KITCHEN_SINK_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef KITCHEN_SINK_BUILD -# define KITCHEN_SINK_API __declspec(dllexport) -# else -# define KITCHEN_SINK_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define KITCHEN_SINK_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define KITCHEN_SINK_API __attribute__((visibility("default"))) -# else -# define KITCHEN_SINK_API -# endif -#endif - -#ifndef KITCHEN_SINK_DEPRECATED -# if defined(_MSC_VER) -# define KITCHEN_SINK_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define KITCHEN_SINK_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define KITCHEN_SINK_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports kitchen_sink_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define KITCHEN_SINK_ABI_VERSION 3u -KITCHEN_SINK_API uint32_t kitchen_sink_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by kitchen_sink_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct kitchen_sink_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} kitchen_sink_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -KITCHEN_SINK_API void kitchen_sink_error_set(kitchen_sink_error* err, int32_t code, const char* message); -KITCHEN_SINK_API void kitchen_sink_error_clear(kitchen_sink_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - kitchen_sink_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_error_free(kitchen_sink_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -KITCHEN_SINK_API void kitchen_sink_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct kitchen_sink_cancel_token kitchen_sink_cancel_token; -KITCHEN_SINK_API kitchen_sink_cancel_token* kitchen_sink_cancel_token_create(void); -KITCHEN_SINK_API void kitchen_sink_cancel_token_cancel(kitchen_sink_cancel_token* token); -KITCHEN_SINK_API bool kitchen_sink_cancel_token_is_cancelled(const kitchen_sink_cancel_token* token); -KITCHEN_SINK_API void kitchen_sink_cancel_token_destroy(kitchen_sink_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -KITCHEN_SINK_API uint8_t* kitchen_sink_alloc(uint32_t size); -KITCHEN_SINK_API void kitchen_sink_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -KITCHEN_SINK_API uint64_t kitchen_sink_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define KITCHEN_SINK_SHARED_CHECKSUM 0x42c4ce2c8d0af052ull -KITCHEN_SINK_API uint64_t kitchen_sink_shared_checksum(void); -#define KITCHEN_SINK_KITCHEN_CHECKSUM 0xd83da75b24b545ffull -KITCHEN_SINK_API uint64_t kitchen_sink_kitchen_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with kitchen_sink_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * kitchen_sink_buffer.h declares a C struct and codec for each. - */ - -/** Task priority levels */ -typedef enum { - /** Low priority */ - kitchen_sink_kitchen_Priority_Low = 0, - /** Normal priority */ - kitchen_sink_kitchen_Priority_Normal = 1, - /** High priority */ - kitchen_sink_kitchen_Priority_High = 2 -} kitchen_sink_kitchen_Priority; -/** Error codes reported by throwing functions in the `kitchen` module tree. */ -typedef enum { kitchen_sink_kitchen_KitchenErrors_NotFound = 1, kitchen_sink_kitchen_KitchenErrors_InvalidInput = 2 } kitchen_sink_kitchen_KitchenErrors; -typedef struct kitchen_sink_kitchen_Gadget kitchen_sink_kitchen_Gadget; -typedef struct kitchen_sink_kitchen_StreamGadgetsIterator kitchen_sink_kitchen_StreamGadgetsIterator; -typedef struct kitchen_sink_kitchen_StreamItemsIterator kitchen_sink_kitchen_StreamItemsIterator; -/** - * Consumer-implemented observer of ready items - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct kitchen_sink_kitchen_ReadyListener_vtable { - /** Fires when an item is ready */ - void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); - /** Receives the item itself and says whether to keep listening */ - bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); -} kitchen_sink_kitchen_ReadyListener_vtable; -typedef void (*kitchen_sink_kitchen_do_async_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*kitchen_sink_kitchen_do_cancellable_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: shared -/** Trivial cross-module helper */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_shared_ping(size_t* out_len, kitchen_sink_error* out_err); - -// Module: kitchen -/** A pocket-sized interface exercising the object surface */ -/** Create a gadget with the given id */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_new(int64_t id, kitchen_sink_error* out_err); -/** The gadget subsystem version string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_version(size_t* out_len, kitchen_sink_error* out_err); -/** Render the gadget as a human-readable string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_describe(const kitchen_sink_kitchen_Gadget* self, size_t* out_len, kitchen_sink_error* out_err); -/** Poke the gadget a number of times, failing on a negative count */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_Gadget_poke(const kitchen_sink_kitchen_Gadget* self, int32_t times, kitchen_sink_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_clone(const kitchen_sink_kitchen_Gadget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_kitchen_Gadget_destroy(kitchen_sink_kitchen_Gadget* self); - -/** Identity for bool */ -KITCHEN_SINK_API bool kitchen_sink_kitchen_bool_id(bool x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_i32_id(int32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API uint32_t kitchen_sink_kitchen_u32_id(uint32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int64_t kitchen_sink_kitchen_i64_id(int64_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API double kitchen_sink_kitchen_f64_id(double x, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_string(const uint8_t* s_ptr, size_t s_len, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_bytes(const uint8_t* b_ptr, size_t b_len, size_t* out_len, kitchen_sink_error* out_err); -/** Register a listener, returning how many are attached */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); -/** Optional object in both directions */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_maybe_gadget(const kitchen_sink_kitchen_Gadget* fallback, kitchen_sink_error* out_err); -/** A list of objects */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_gadgets(size_t* out_len, kitchen_sink_error* out_err); -/** An iterator over objects */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamGadgetsIterator* kitchen_sink_kitchen_stream_gadgets(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamGadgetsIterator_next(kitchen_sink_kitchen_StreamGadgetsIterator* iter, kitchen_sink_kitchen_Gadget** out_item, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamGadgetsIterator_destroy(kitchen_sink_kitchen_StreamGadgetsIterator* iter); -/** Optional struct return */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_maybe_item(int64_t id, size_t* out_len, kitchen_sink_error* out_err); -/** List of structs */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_list_items(size_t* out_len, kitchen_sink_error* out_err); -/** Map return type */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_get_attrs(size_t* out_len, kitchen_sink_error* out_err); -/** Iterator return type */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamItemsIterator* kitchen_sink_kitchen_stream_items(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamItemsIterator_next(kitchen_sink_kitchen_StreamItemsIterator* iter, const uint8_t** out_item, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamItemsIterator_destroy(kitchen_sink_kitchen_StreamItemsIterator* iter); -/** Returns the shared Token type from another module */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_cross_module_token(size_t* out_len, kitchen_sink_error* out_err); -/** Async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_async(const uint8_t* input_ptr, size_t input_len, kitchen_sink_kitchen_do_async_callback callback, void* context); -/** Cancellable async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_cancellable(const uint8_t* input_ptr, size_t input_len, kitchen_sink_cancel_token* cancel_token, kitchen_sink_kitchen_do_cancellable_callback callback, void* context); -/** Legacy operation kept for compatibility */ -KITCHEN_SINK_DEPRECATED("Use new_op instead") -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_legacy_op(kitchen_sink_error* out_err); -/** Replacement for legacy_op */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_new_op(kitchen_sink_error* out_err); - -// Module: kitchen_nested -/** Trivial nested-module function */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_nested_hello(size_t* out_len, kitchen_sink_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // KITCHEN_SINK_H - -// END kitchen_sink.h diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_hpp.snap b/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_hpp.snap index 3ba874d8..fdfc2800 100644 --- a/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_hpp.snap +++ b/crates/weaveffi-cli/tests/snapshots/cpp_kitchen_sink__kitchen_sink_hpp.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #pragma once @@ -28,13 +28,13 @@ source: crates/weaveffi-cli/tests/snapshots.rs namespace kitchen_sink { /** - * Base exception for every error reported through the C ABI. + * An error a call that declares errors reports. * - * Positive codes are the declared error codes (see the typed domain - * exceptions); negative codes are runtime traps: -1 a generic producer - * error, -2 a producer panic, -3 a marshalling failure, -4 a - * callback-interface implementation that threw, and -5 a cancelled call - * (thrown as Cancelled). + * A positive code is a declared code of the module's error domain, thrown as + * that domain's typed subclass. A negative code is a runtime failure: -1 a + * generic producer error, -2 a producer panic, -3 a marshalling failure, and + * -4 a callback-interface implementation that failed. A cancelled async call + * (-5) throws Cancelled. */ class Error : public std::runtime_error { int32_t code_; @@ -47,7 +47,7 @@ public: int32_t code() const noexcept { return code_; } }; -/** Thrown by an async call whose CancelToken was cancelled (code -5). */ +/** Thrown by an async call that was cancelled through its CancelToken (code -5). */ class Cancelled : public Error { public: /** Builds a cancellation error carrying the producer's message. */ @@ -55,13 +55,39 @@ public: }; /** - * Thrown on first use when the linked kitchen_sink library was built for a - * different C ABI revision or a different API contract than this header. + * Thrown by a call that declares no errors when it fails anyway: a producer + * panic (-2), a marshalling failure (-3), a callback-interface failure the + * producer let through (-4), or another runtime failure (-1). These are bugs, + * so no signature declares them; `what()` names the code and the producer's + * message. */ -class LoadError : public Error { +class InternalError : public std::runtime_error { + int32_t code_; + std::string message_; + +public: + /** Builds an internal error carrying `code` and the producer's `message`. */ + InternalError(int32_t code, const std::string& message) + : std::runtime_error("kitchen_sink: runtime error " + std::to_string(code) + ": " + message), + code_(code), + message_(message) {} + + /** The runtime error code (negative). */ + int32_t code() const noexcept { return code_; } + + /** The producer's message. */ + const std::string& message() const noexcept { return message_; } +}; + +/** + * Thrown on first use when the linked kitchen_sink library implements a + * different C ABI revision than this header, or lacks or changed a + * declaration this header was generated with. + */ +class LoadError : public std::runtime_error { public: /** Builds a load error describing the mismatch. */ - explicit LoadError(const std::string& message) : Error(-1, message) {} + explicit LoadError(const std::string& message) : std::runtime_error(message) {} }; /** @@ -72,7 +98,7 @@ struct adopt_t { explicit adopt_t() = default; }; -/** The adopt_t tag value: `Gadget g(adopt, raw)`. */ +/** The adopt_t tag value: `Store s(adopt, raw)`. */ inline constexpr adopt_t adopt{}; /** @@ -138,10 +164,10 @@ public: namespace detail { -/** The exception for a runtime (non-domain) error code. */ -inline std::exception_ptr make_error(int32_t code, const std::string& message) { +/** The exception for a failed call that declares no errors: Cancelled for -5, else InternalError. */ +inline std::exception_ptr make_internal_error(int32_t code, const std::string& message) { if (code == -5) return std::make_exception_ptr(Cancelled(message)); - return std::make_exception_ptr(Error(code, message)); + return std::make_exception_ptr(InternalError(code, message)); } /** The message of a nonzero error slot. */ @@ -149,10 +175,10 @@ inline std::string error_message(const kitchen_sink_error& err) { return std::string(err.message != nullptr ? err.message : "unknown error"); } -/** Throws the generic Error (or Cancelled) if `err` carries a nonzero code. */ +/** Throws InternalError if `err` carries a nonzero code (a call that declares no errors). */ inline void check(kitchen_sink_error& err) { if (err.code == 0) return; - std::exception_ptr ex = make_error(err.code, error_message(err)); + std::exception_ptr ex = make_internal_error(err.code, error_message(err)); kitchen_sink_error_clear(&err); std::rethrow_exception(ex); } @@ -185,7 +211,7 @@ inline std::string_view borrow_string(const uint8_t* ptr, size_t len) noexcept { return std::string_view(reinterpret_cast(ptr), len); } -/** Views a borrowed byte run (null when empty) as an owned vector. */ +/** Copies a borrowed byte run (null when empty) into a vector. */ inline std::vector borrow_bytes(const uint8_t* ptr, size_t len) { if (ptr == nullptr || len == 0) return std::vector(); return std::vector(ptr, ptr + len); @@ -196,27 +222,127 @@ inline const uint8_t* utf8(std::string_view s) noexcept { return reinterpret_cast(s.data()); } +/** + * Hands `len` bytes from `data` to the producer through a callback method's + * out slots: copies them into a run from kitchen_sink_alloc, which the + * producer adopts and frees. + */ +inline void hand_over(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = nullptr; + if (len > 0) { + run = kitchen_sink_alloc(len); + if (run == nullptr) throw std::bad_alloc(); + std::memcpy(run, data, len); + } + *out_ptr = run; + *out_len = len; +} + +/** One declaration this header was generated with: its contract id and hash, and its dotted path. */ +struct ContractEntry { + /** FNV-1a 64 of the declaration's dotted path. */ + uint64_t id; + /** FNV-1a 64 of the declaration's canonical signature. */ + uint64_t hash; + /** The declaration's dotted path (`kv.Store.put`). */ + const char* path; +}; + +/** + * Why the contract table `contract` returns doesn't satisfy `expected`, + * naming the first declaration it lacks or declares differently, or an empty + * string when it has them all. Entries the producer has and `expected` + * doesn't are fine. + */ +template +std::string contract_mismatch(const kitchen_sink_contract_entry* (*contract)(size_t*), + const ContractEntry (&expected)[N]) { + size_t len = 0; + const kitchen_sink_contract_entry* table = contract(&len); + if (table == nullptr) len = 0; + for (const ContractEntry& want : expected) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < want.id) { + lo = mid + 1; + } else { + hi = mid; + } + } + if (lo == len || table[lo].id != want.id) { + return std::string(want.path) + " is missing from the library"; + } + if (table[lo].hash != want.hash) { + return std::string(want.path) + " changed since these bindings were generated"; + } + } + return std::string(); +} + +/** The declarations of module `shared` this header was generated with. */ +inline constexpr ContractEntry shared_contract[] = { + {0x709e234caa962446ull, 0x3894f2ca452e9f98ull, "shared.ping"}, + {0xcfb306fa5801ee5bull, 0x1cbd9e10983cac2cull, "shared.Token"}, +}; + +/** The declarations of module `kitchen` this header was generated with. */ +inline constexpr ContractEntry kitchen_contract[] = { + {0x01d4f09bbddc986full, 0x29eb88fecc66544dull, "kitchen.Gadget.new"}, + {0x10472d8fe748c2eeull, 0xb265832b6a759fc4ull, "kitchen.maybe_gadget"}, + {0x11368eec07a74078ull, 0xfa049f78b77e146aull, "kitchen.gadgets"}, + {0x137f08a29cde6021ull, 0xda8de7aeb2dbb8a3ull, "kitchen.stream_gadgets"}, + {0x2111b54980826dd7ull, 0x452783f3e6ca6e1dull, "kitchen.do_cancellable"}, + {0x29dddbc60bf8d7dbull, 0x452a3e573d57f6dbull, "kitchen.Priority"}, + {0x2b227003b74a0d16ull, 0x12332957470e9782ull, "kitchen.maybe_subscribe"}, + {0x2dd66cb0064f0494ull, 0x60197363fefd92ddull, "kitchen.Item"}, + {0x33c265bbf079b0e9ull, 0xc3dcd0927e6c1debull, "kitchen.u32_id"}, + {0x33e6381d01660dfeull, 0x077c63c31fd7f90aull, "kitchen.KitchenErrors"}, + {0x3f5fc99d73504b76ull, 0x2df88d710f1ea10eull, "kitchen.Gadget.poke"}, + {0x4c5764cf109528d1ull, 0x2a65529af4338a58ull, "kitchen.maybe_item"}, + {0x4de17dfb93fd494dull, 0xba0ad0dd52ed74afull, "kitchen.i32_id"}, + {0x504e95b7f7e65570ull, 0x01b3e8e0c9a10e23ull, "kitchen.stream_items"}, + {0x68cdb9a4bd3e3dd5ull, 0xc678b9164348d497ull, "kitchen.bool_id"}, + {0x69e7cd703d16eb85ull, 0xf3b28dd22ae9e689ull, "kitchen.do_async"}, + {0x70865079ce41be82ull, 0xbe3b69464672af46ull, "kitchen.Gadget.describe"}, + {0x71c5a1116cb7c49aull, 0xcba1837108a89b84ull, "kitchen.echo_bytes"}, + {0x735ee1e0049ee91aull, 0xca74320bc53eddabull, "kitchen.echo_string"}, + {0x76d7fc58f4e41d84ull, 0x7373a0e47d30e690ull, "kitchen.i64_id"}, + {0x79abfe1aa960fd6aull, 0xbca7a9e5eb26f3a6ull, "kitchen.nested.hello"}, + {0x7d3dd7d5cf6749e3ull, 0xa192f5a7298adb39ull, "kitchen.new_op"}, + {0x89a5e664bc25c67bull, 0x5764a95e81ecfd31ull, "kitchen.f64_id"}, + {0x8cfdbdcecac5f7f1ull, 0x07e272f09c41fffcull, "kitchen.Gadget.version"}, + {0x974b9f3e823c1738ull, 0xb81b91e975695f27ull, "kitchen.cross_module_token"}, + {0xaca0655fac15d230ull, 0x11c4d2b28abe07bbull, "kitchen.list_items"}, + {0xc883b4afae20543eull, 0xc40560e5411ff392ull, "kitchen.legacy_op"}, + {0xf2f7753e6e1dffc2ull, 0xc9047a95bdf49e93ull, "kitchen.get_attrs"}, + {0xfae25d3391922eadull, 0x1c7f95739f3210b8ull, "kitchen.subscribe"}, + {0xfd0059b05cb2fcfbull, 0x248bf8404a5823b4ull, "kitchen.Gadget"}, + {0xfed9c551fb9fd2d6ull, 0x59aec980ef3f3291ull, "kitchen.ReadyListener"}, +}; + } // namespace detail /** - * Verifies that the linked kitchen_sink library implements the C ABI revision - * and the module contracts this header was generated against, throwing + * Verifies that the linked kitchen_sink library implements C ABI revision + * 4 and every declaration this header was generated with, throwing * LoadError naming the first mismatch. The check runs once; every free * function, constructor, and static member calls this before its first * native call, so calling it at startup is optional. */ inline void check_library() { - static const std::string failure = [] { + static const std::string failure = []() -> std::string { uint32_t abi = kitchen_sink_abi_version(); if (abi != KITCHEN_SINK_ABI_VERSION) { return "kitchen_sink: C ABI revision mismatch (header " + std::to_string(KITCHEN_SINK_ABI_VERSION) + ", library " + std::to_string(abi) + ")"; } - if (kitchen_sink_shared_checksum() != 0x42c4ce2c8d0af052ull) { - return std::string("kitchen_sink: module 'shared' does not match the linked library (contract checksum mismatch); regenerate the bindings for this build"); + if (std::string why = detail::contract_mismatch(kitchen_sink_shared_contract, detail::shared_contract); !why.empty()) { + return "kitchen_sink: " + why; } - if (kitchen_sink_kitchen_checksum() != 0xd83da75b24b545ffull) { - return std::string("kitchen_sink: module 'kitchen' does not match the linked library (contract checksum mismatch); regenerate the bindings for this build"); + if (std::string why = detail::contract_mismatch(kitchen_sink_kitchen_contract, detail::kitchen_contract); !why.empty()) { + return "kitchen_sink: " + why; } return std::string(); }(); @@ -270,7 +396,7 @@ public: /** Writes a byte length or an element count as a u32. */ void write_len(size_t n) { - if (static_cast(n) > UINT32_MAX) throw Error(-3, "value buffer length exceeds u32"); + if (static_cast(n) > UINT32_MAX) throw std::length_error("value buffer length exceeds u32"); append_le(static_cast(n)); } @@ -286,24 +412,29 @@ public: /** Writes an optional's presence flag: 0 absent, 1 present. */ void write_option_flag(bool present) { buf_.push_back(present ? 1 : 0); } + + /** + * Writes an object token: a fresh strong reference from the wrapper's + * `_clone`, which the reader adopts while `v` keeps its own. + */ + template + void write_object(const T& v) { + write_u64(static_cast(reinterpret_cast(v.clone_handle()))); + } }; /** * Decodes values from the value-buffer wire format. A malformed buffer is a - * producer/consumer contract violation (both sides are generated from one - * API), so every decode failure throws the generic Error with the - * marshalling code -3. Object tokens adopted before a failure live in RAII - * wrappers that release them on unwind. + * producer bug (both sides are generated from one API), so every decode + * failure throws InternalError with the marshalling code -3. Object tokens + * adopted before a failure live in RAII wrappers that release them on + * unwind. */ class BufferReader { const uint8_t* data_; size_t len_; size_t pos_; - [[noreturn]] static void fail(const char* what) { - throw Error(-3, std::string("malformed value buffer: ") + what); - } - void require(size_t n, const char* what) const { if (len_ - pos_ < n) fail(what); } @@ -320,6 +451,12 @@ class BufferReader { } public: + /** Throws the marshalling failure (-3) for a malformed buffer. */ + [[noreturn]] static void fail(const char* what) { + throw InternalError(-3, std::string("malformed value buffer: ") + what); + } + + /** Reads from the `len` bytes at `data` (null when empty). */ BufferReader(const uint8_t* data, size_t len) noexcept : data_(data), len_(data != nullptr ? len : 0), pos_(0) {} /** Bytes not yet consumed. */ @@ -391,6 +528,14 @@ public: return b != 0; } + /** Reads an object token, adopting the strong reference it carries into a `T` wrapper. */ + template + T read_object() { + auto* raw = reinterpret_cast(static_cast(read_u64())); + if (raw == nullptr) fail("null object token"); + return T(adopt, raw); + } + /** Rejects unconsumed bytes after decoding a complete value. */ void expect_end() const { if (pos_ != len_) fail("trailing bytes after value"); @@ -413,6 +558,22 @@ struct BufferGuard { } }; +/** Decodes one complete `T` from a borrowed buffer with `read`, rejecting trailing bytes. */ +template +T decode(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferReader r(ptr, len); + T value = read(r); + r.expect_end(); + return value; +} + +/** Decodes one complete `T` from a producer-returned buffer with `read`, then releases the buffer. */ +template +T take(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferGuard guard(ptr, len); + return decode(ptr, len, read); +} + } // namespace detail /** Task priority levels */ @@ -422,7 +583,7 @@ enum class Priority : int32_t { /** Normal priority */ Normal = 1, /** High priority */ - High = 2 + High = 2, }; struct Token; @@ -432,50 +593,51 @@ class ReadyListener; /** A pocket-sized interface exercising the object surface */ class Gadget { - kitchen_sink_kitchen_Gadget* handle_; + kitchen_sink_kitchen_Gadget* raw_; public: /** The C type this class wraps. */ using raw_type = kitchen_sink_kitchen_Gadget; /** Adopts one strong reference to a producer object: `Gadget(adopt, raw)`. */ - explicit Gadget(adopt_t, kitchen_sink_kitchen_Gadget* h) noexcept : handle_(h) {} + explicit Gadget(adopt_t, kitchen_sink_kitchen_Gadget* h) noexcept : raw_(h) {} /** Releases this wrapper's reference; the object is dropped with its last one. */ ~Gadget() { - if (handle_) kitchen_sink_kitchen_Gadget_destroy(handle_); + if (raw_ != nullptr) kitchen_sink_kitchen_Gadget_destroy(raw_); } /** Copies share the object: the copy takes a new strong reference. */ - Gadget(const Gadget& other) : handle_(kitchen_sink_kitchen_Gadget_clone(other.handle_)) {} + Gadget(const Gadget& other) : raw_(kitchen_sink_kitchen_Gadget_clone(other.raw_)) {} + /** Takes a new reference to `other`'s object and releases the current one. */ Gadget& operator=(const Gadget& other) { if (this != &other) { - kitchen_sink_kitchen_Gadget* h = kitchen_sink_kitchen_Gadget_clone(other.handle_); - if (handle_) kitchen_sink_kitchen_Gadget_destroy(handle_); - handle_ = h; + kitchen_sink_kitchen_Gadget* h = kitchen_sink_kitchen_Gadget_clone(other.raw_); + if (raw_ != nullptr) kitchen_sink_kitchen_Gadget_destroy(raw_); + raw_ = h; } return *this; } - Gadget(Gadget&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s reference; `other` becomes empty. */ + Gadget(Gadget&& other) noexcept : raw_(other.raw_) { other.raw_ = nullptr; } + /** Releases the current reference and takes over `other`'s. */ Gadget& operator=(Gadget&& other) noexcept { if (this != &other) { - if (handle_) kitchen_sink_kitchen_Gadget_destroy(handle_); - handle_ = other.handle_; - other.handle_ = nullptr; + if (raw_ != nullptr) kitchen_sink_kitchen_Gadget_destroy(raw_); + raw_ = other.raw_; + other.raw_ = nullptr; } return *this; } - /** The wrapped pointer, borrowed: this wrapper keeps its reference. */ - const kitchen_sink_kitchen_Gadget* handle() const { return handle_; } + /** The wrapped pointer, borrowed (null after a move): this wrapper keeps its reference. */ + const kitchen_sink_kitchen_Gadget* handle() const noexcept { return raw_; } - /** A new strong reference the caller owns (for example to write into a value buffer). */ - kitchen_sink_kitchen_Gadget* clone_handle() const { return kitchen_sink_kitchen_Gadget_clone(handle_); } + /** A new strong reference to the object, which the caller owns. */ + kitchen_sink_kitchen_Gadget* clone_handle() const { return kitchen_sink_kitchen_Gadget_clone(raw_); } /** Create a gadget with the given id */ explicit Gadget(int64_t id); @@ -483,7 +645,11 @@ public: /** Render the gadget as a human-readable string */ std::string describe() const; - /** Poke the gadget a number of times, failing on a negative count */ + /** + * Poke the gadget a number of times, failing on a negative count + * + * @throws KitchenErrorsError when the call fails with one of the domain's codes. + */ int32_t poke(int32_t times) const; /** The gadget subsystem version string */ @@ -523,20 +689,42 @@ struct Item { namespace detail { -/** Encodes a `Token` in the value-buffer wire format. */ +inline void write_Token(BufferWriter& w, const Token& v); +inline Token read_Token(BufferReader& r); +inline void write_Item(BufferWriter& w, const Item& v); +inline Item read_Item(BufferReader& r); +inline void write_list_string(BufferWriter& w, const std::vector& v); +inline std::vector read_list_string(BufferReader& r); +inline void write_map_string_string(BufferWriter& w, const std::unordered_map& v); +inline std::unordered_map read_map_string_string(BufferReader& r); +inline void write_opt_i64(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_i64(BufferReader& r); +inline void write_opt_Gadget(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_Gadget(BufferReader& r); +inline void write_list_Gadget(BufferWriter& w, const std::vector& v); +inline std::vector read_list_Gadget(BufferReader& r); +inline void write_opt_Item(BufferWriter& w, const std::optional& v); +inline std::optional read_opt_Item(BufferReader& r); +inline void write_list_Item(BufferWriter& w, const std::vector& v); +inline std::vector read_list_Item(BufferReader& r); +inline void write_map_string_i32(BufferWriter& w, const std::unordered_map& v); +inline std::unordered_map read_map_string_i32(BufferReader& r); + +/** Writes one `Token` to a value buffer. */ inline void write_Token(BufferWriter& w, const Token& v) { w.write_i64(v.id); w.write_string(v.label); } -/** Decodes a `Token` from the value-buffer wire format. */ +/** Reads one `Token` from a value buffer. */ inline Token read_Token(BufferReader& r) { - int64_t f_id = r.read_i64(); - std::string f_label = r.read_string(); - return Token{std::move(f_id), std::move(f_label)}; + return Token{ + r.read_i64(), + r.read_string(), + }; } -/** Encodes a `Item` in the value-buffer wire format. */ +/** Writes one `Item` to a value buffer. */ inline void write_Item(BufferWriter& w, const Item& v) { w.write_i64(v.id); w.write_string(v.name); @@ -545,118 +733,228 @@ inline void write_Item(BufferWriter& w, const Item& v) { w.write_f64(v.ratio); w.write_u32(v.nick); w.write_bytes(v.payload); - w.write_len(v.tags.size()); - for (const auto& item0 : v.tags) { - w.write_string(item0); - } - w.write_len(v.attrs.size()); - for (const auto& kv0 : v.attrs) { - w.write_string(kv0.first); - w.write_string(kv0.second); - } - w.write_option_flag(v.parent.has_value()); - if (v.parent.has_value()) { - w.write_i64((*v.parent)); - } + detail::write_list_string(w, v.tags); + detail::write_map_string_string(w, v.attrs); + detail::write_opt_i64(w, v.parent); detail::write_Token(w, v.token); w.write_i32(static_cast(v.priority)); - w.write_option_flag(v.gadget.has_value()); - if (v.gadget.has_value()) { - w.write_u64(static_cast(reinterpret_cast((*v.gadget).clone_handle()))); - } + detail::write_opt_Gadget(w, v.gadget); } -/** Decodes a `Item` from the value-buffer wire format. */ +/** Reads one `Item` from a value buffer. */ inline Item read_Item(BufferReader& r) { - int64_t f_id = r.read_i64(); - std::string f_name = r.read_string(); - int32_t f_count = r.read_i32(); - bool f_enabled = r.read_bool(); - double f_ratio = r.read_f64(); - uint32_t f_nick = r.read_u32(); - std::vector f_payload = r.read_bytes(); - std::vector f_tags{}; - { - size_t f_tags_n = r.read_count(); - f_tags.reserve(r.reserve_hint(f_tags_n)); - for (size_t f_tags_i = 0; f_tags_i < f_tags_n; ++f_tags_i) { - std::string f_tags_item = r.read_string(); - f_tags.push_back(std::move(f_tags_item)); - } + return Item{ + r.read_i64(), + r.read_string(), + r.read_i32(), + r.read_bool(), + r.read_f64(), + r.read_u32(), + r.read_bytes(), + detail::read_list_string(r), + detail::read_map_string_string(r), + detail::read_opt_i64(r), + detail::read_Token(r), + static_cast(r.read_i32()), + detail::read_opt_Gadget(r), + }; +} + +/** Writes one `[string]` to a value buffer. */ +inline void write_list_string(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) w.write_string(item); +} + +/** Reads one `[string]` from a value buffer. */ +inline std::vector read_list_string(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(r.read_string()); + return v; +} + +/** Writes one `{string:string}` to a value buffer. */ +inline void write_map_string_string(BufferWriter& w, const std::unordered_map& v) { + w.write_len(v.size()); + for (const auto& entry : v) { + w.write_string(entry.first); + w.write_string(entry.second); } - std::unordered_map f_attrs{}; - { - size_t f_attrs_n = r.read_count(); - for (size_t f_attrs_i = 0; f_attrs_i < f_attrs_n; ++f_attrs_i) { - std::string f_attrs_key = r.read_string(); - std::string f_attrs_val = r.read_string(); - f_attrs.emplace(std::move(f_attrs_key), std::move(f_attrs_val)); - } +} + +/** Reads one `{string:string}` from a value buffer. */ +inline std::unordered_map read_map_string_string(BufferReader& r) { + size_t n = r.read_count(); + std::unordered_map v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) { + std::string key = r.read_string(); + std::string value = r.read_string(); + if (!v.emplace(std::move(key), std::move(value)).second) BufferReader::fail("repeated map key"); } - std::optional f_parent{}; - if (r.read_option_flag()) { - int64_t f_parent_v = r.read_i64(); - f_parent = std::move(f_parent_v); + return v; +} + +/** Writes one `i64?` to a value buffer. */ +inline void write_opt_i64(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_i64(*v); +} + +/** Reads one `i64?` from a value buffer. */ +inline std::optional read_opt_i64(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(r.read_i64()); +} + +/** Writes one `Gadget?` to a value buffer. */ +inline void write_opt_Gadget(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) w.write_object(*v); +} + +/** Reads one `Gadget?` from a value buffer. */ +inline std::optional read_opt_Gadget(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(r.read_object()); +} + +/** Writes one `[Gadget]` to a value buffer. */ +inline void write_list_Gadget(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) w.write_object(item); +} + +/** Reads one `[Gadget]` from a value buffer. */ +inline std::vector read_list_Gadget(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(r.read_object()); + return v; +} + +/** Writes one `Item?` to a value buffer. */ +inline void write_opt_Item(BufferWriter& w, const std::optional& v) { + w.write_option_flag(v.has_value()); + if (v.has_value()) detail::write_Item(w, *v); +} + +/** Reads one `Item?` from a value buffer. */ +inline std::optional read_opt_Item(BufferReader& r) { + if (!r.read_option_flag()) return std::nullopt; + return std::optional(detail::read_Item(r)); +} + +/** Writes one `[Item]` to a value buffer. */ +inline void write_list_Item(BufferWriter& w, const std::vector& v) { + w.write_len(v.size()); + for (const auto& item : v) detail::write_Item(w, item); +} + +/** Reads one `[Item]` from a value buffer. */ +inline std::vector read_list_Item(BufferReader& r) { + size_t n = r.read_count(); + std::vector v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) v.push_back(detail::read_Item(r)); + return v; +} + +/** Writes one `{string:i32}` to a value buffer. */ +inline void write_map_string_i32(BufferWriter& w, const std::unordered_map& v) { + w.write_len(v.size()); + for (const auto& entry : v) { + w.write_string(entry.first); + w.write_i32(entry.second); } - Token f_token = detail::read_Token(r); - Priority f_priority = static_cast(r.read_i32()); - std::optional f_gadget{}; - if (r.read_option_flag()) { - Gadget f_gadget_v = Gadget(adopt, reinterpret_cast(static_cast(r.read_u64()))); - f_gadget = std::move(f_gadget_v); +} + +/** Reads one `{string:i32}` from a value buffer. */ +inline std::unordered_map read_map_string_i32(BufferReader& r) { + size_t n = r.read_count(); + std::unordered_map v; + v.reserve(r.reserve_hint(n)); + for (size_t i = 0; i < n; ++i) { + std::string key = r.read_string(); + int32_t value = r.read_i32(); + if (!v.emplace(std::move(key), std::move(value)).second) BufferReader::fail("repeated map key"); } - return Item{std::move(f_id), std::move(f_name), std::move(f_count), std::move(f_enabled), std::move(f_ratio), std::move(f_nick), std::move(f_payload), std::move(f_tags), std::move(f_attrs), std::move(f_parent), std::move(f_token), std::move(f_priority), std::move(f_gadget)}; + return v; } } // namespace detail -/** Typed errors reported by the `kitchen` module's throwing functions. */ +/** The errors the `kitchen` module's throwing calls report; catch a code's subclass or this. */ class KitchenErrorsError : public Error { public: - KitchenErrorsError(int32_t code, const std::string& msg) : Error(code, msg) {} + /** Builds an error carrying a domain `code` and `message`. */ + KitchenErrorsError(int32_t code, const std::string& message) : Error(code, message) {} }; -/** Item not found */ +/** + * Item not found + * + * Code 1. + */ class NotFoundError : public KitchenErrorsError { public: - explicit NotFoundError(const std::string& msg) : KitchenErrorsError(1, msg) {} + /** Builds the error with the producer's `message` and the code's fields. */ + explicit NotFoundError(const std::string& message) : KitchenErrorsError(1, message) {} }; -/** Invalid input */ +/** + * Invalid input + * + * Code 2. + */ class InvalidInputError : public KitchenErrorsError { public: - explicit InvalidInputError(const std::string& msg) : KitchenErrorsError(2, msg) {} + /** Builds the error with the producer's `message` and the code's fields. */ + explicit InvalidInputError(const std::string& message) : KitchenErrorsError(2, message) {} }; namespace detail { -/** Map a `kitchen` error code and payload to its typed exception (the runtime exception for negative codes, KitchenErrorsError for unknown codes). */ -inline std::exception_ptr make_kitchen_error(int32_t code, const std::string& msg, const uint8_t* payload_ptr, size_t payload_len) { - if (code < 0) return make_error(code, msg); +/** The exception for a failed `KitchenErrorsError` call: its code's subclass, Cancelled, or the root Error. */ +inline std::exception_ptr make_kitchen_error(int32_t code, const std::string& message, const uint8_t* payload_ptr, size_t payload_len) { + if (code == -5) return std::make_exception_ptr(Cancelled(message)); + if (code < 0) return std::make_exception_ptr(Error(code, message)); (void)payload_ptr; (void)payload_len; switch (code) { - case 1: return std::make_exception_ptr(NotFoundError(msg)); - case 2: return std::make_exception_ptr(InvalidInputError(msg)); - default: return std::make_exception_ptr(KitchenErrorsError(code, msg)); + case 1: return std::make_exception_ptr(NotFoundError(message)); + case 2: return std::make_exception_ptr(InvalidInputError(message)); + default: return std::make_exception_ptr(KitchenErrorsError(code, message)); } } -/** Throw the typed `kitchen` exception if `err` carries a nonzero code. */ +/** Throws the typed `KitchenErrorsError` exception if `err` carries a nonzero code. */ inline void check_kitchen(kitchen_sink_error& err) { if (err.code == 0) return; - std::string msg = error_message(err); - std::exception_ptr ex = make_kitchen_error(err.code, msg, err.payload_ptr, err.payload_len); + std::exception_ptr ex = make_kitchen_error(err.code, error_message(err), err.payload_ptr, err.payload_len); kitchen_sink_error_clear(&err); std::rethrow_exception(ex); } +/** Reports a `KitchenErrorsError` a callback implementation threw through the vtable entry's `out_err`. */ +inline void report_kitchen_error(const KitchenErrorsError& e, kitchen_sink_error* out_err) { + switch (e.code()) { + case 1: kitchen_sink_error_set(out_err, 1, e.what()); return; + case 2: kitchen_sink_error_set(out_err, 2, e.what()); return; + default: break; + } + kitchen_sink_error_set(out_err, -4, e.what()); +} + } // namespace detail /** * Consumer-implemented observer of ready items * - * Implement this interface and pass a `std::shared_ptr` to any function that accepts one. The producer may call the methods from any thread until it releases its last reference to the implementation. An exception thrown by a method is reported to the producer as a foreign error (code -4) and aborts the call that triggered it. + * Subclass it and pass a `std::shared_ptr` where the API takes a `ReadyListener`. The producer may call the methods from any thread, so they must be thread-safe, until it releases the implementation (also from any thread). A method that declares errors may throw a `KitchenErrorsError` code's exception, which reaches the producer with its fields; any other exception reaches it as a callback failure (code -4). */ class ReadyListener { public: @@ -664,20 +962,32 @@ public: /** Fires when an item is ready */ virtual void on_ready(int32_t code, std::string_view msg) = 0; + /** Receives the item itself and says whether to keep listening */ virtual bool on_item(const Item& item, Gadget gadget) = 0; + + /** + * A display label, failing with a kitchen error + * + * @throws KitchenErrorsError to report one of the domain's codes to the producer. + */ + virtual std::string label() = 0; + + /** The most recent item the listener kept, if any */ + virtual std::optional latest() = 0; + + /** The listener's favorite gadget */ + virtual Gadget favorite() = 0; }; namespace detail { -/** Vtable trampolines adapting a `ReadyListener` implementation to `kitchen_sink_kitchen_ReadyListener_vtable`. */ +/** Trampolines adapting a `ReadyListener` implementation to `kitchen_sink_kitchen_ReadyListener_vtable`. */ struct ReadyListener_trampolines { static void on_ready(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err) { try { ReadyListener& impl = **static_cast*>(ctx); - std::string_view msg_val = detail::borrow_string(msg_ptr, msg_len); - impl.on_ready(code, msg_val); - return; + impl.on_ready(code, detail::borrow_string(msg_ptr, msg_len)); } catch (const std::exception& e) { kitchen_sink_error_set(out_err, -4, e.what()); } catch (...) { @@ -686,13 +996,11 @@ struct ReadyListener_trampolines { } static bool on_item(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err) { - Gadget gadget_val(adopt, gadget); + Gadget gadget_arg(adopt, gadget); try { ReadyListener& impl = **static_cast*>(ctx); - detail::BufferReader item_r(item_ptr, item_len); - Item item_val = detail::read_Item(item_r); - item_r.expect_end(); - return impl.on_item(item_val, std::move(gadget_val)); + Item item_arg = detail::decode(item_ptr, item_len, &detail::read_Item); + return impl.on_item(std::move(item_arg), std::move(gadget_arg)); } catch (const std::exception& e) { kitchen_sink_error_set(out_err, -4, e.what()); } catch (...) { @@ -701,43 +1009,88 @@ struct ReadyListener_trampolines { return bool{}; } - /** Releases the implementation box once the producer drops its last reference. */ + static void label(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err) { + try { + ReadyListener& impl = **static_cast*>(ctx); + std::string ret = impl.label(); + detail::hand_over(ret.data(), ret.size(), out_ptr, out_len); + } catch (const KitchenErrorsError& e) { + detail::report_kitchen_error(e, out_err); + } catch (const std::exception& e) { + kitchen_sink_error_set(out_err, -4, e.what()); + } catch (...) { + kitchen_sink_error_set(out_err, -4, "ReadyListener::label threw a non-standard exception"); + } + } + + static void latest(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err) { + try { + ReadyListener& impl = **static_cast*>(ctx); + std::optional ret = impl.latest(); + detail::BufferWriter ret_buf; + detail::write_opt_Item(ret_buf, ret); + detail::hand_over(ret_buf.data(), ret_buf.size(), out_ptr, out_len); + } catch (const std::exception& e) { + kitchen_sink_error_set(out_err, -4, e.what()); + } catch (...) { + kitchen_sink_error_set(out_err, -4, "ReadyListener::latest threw a non-standard exception"); + } + } + + static kitchen_sink_kitchen_Gadget* favorite(void* ctx, kitchen_sink_error* out_err) { + try { + ReadyListener& impl = **static_cast*>(ctx); + return impl.favorite().clone_handle(); + } catch (const std::exception& e) { + kitchen_sink_error_set(out_err, -4, e.what()); + } catch (...) { + kitchen_sink_error_set(out_err, -4, "ReadyListener::favorite threw a non-standard exception"); + } + return nullptr; + } + + /** Deletes the implementation box once the producer releases it. */ static void free_ctx(void* ctx) { delete static_cast*>(ctx); } }; -/** The process-wide vtable every `ReadyListener` implementation is passed with. */ +/** The vtable every `ReadyListener` implementation is passed with. */ inline const kitchen_sink_kitchen_ReadyListener_vtable& ReadyListener_vtable() { static const kitchen_sink_kitchen_ReadyListener_vtable vtable = { + static_cast(sizeof(kitchen_sink_kitchen_ReadyListener_vtable)), + 0, + &ReadyListener_trampolines::free_ctx, &ReadyListener_trampolines::on_ready, &ReadyListener_trampolines::on_item, - &ReadyListener_trampolines::free_ctx, + &ReadyListener_trampolines::label, + &ReadyListener_trampolines::latest, + &ReadyListener_trampolines::favorite, }; return vtable; } } // namespace detail -inline Gadget::Gadget(int64_t id) : handle_(nullptr) { +inline Gadget::Gadget(int64_t id) : raw_(nullptr) { check_library(); kitchen_sink_error err{}; auto result = kitchen_sink_kitchen_Gadget_new(id, &err); detail::check(err); - handle_ = result; + raw_ = result; } inline std::string Gadget::describe() const { size_t out_len = 0; kitchen_sink_error err{}; - auto result = kitchen_sink_kitchen_Gadget_describe(handle_, &out_len, &err); + auto result = kitchen_sink_kitchen_Gadget_describe(raw_, &out_len, &err); detail::check(err); return detail::take_string(result, out_len); } inline int32_t Gadget::poke(int32_t times) const { kitchen_sink_error err{}; - auto result = kitchen_sink_kitchen_Gadget_poke(handle_, times, &err); + auto result = kitchen_sink_kitchen_Gadget_poke(raw_, times, &err); detail::check_kitchen(err); return result; } @@ -837,14 +1190,24 @@ inline int32_t subscribe(std::shared_ptr listener) { return result; } +/** Register a listener if one is given */ +inline bool maybe_subscribe(std::shared_ptr listener) { + check_library(); + const auto* listener_vtable = listener ? &detail::ReadyListener_vtable() : nullptr; + std::unique_ptr> listener_ctx(listener ? new std::shared_ptr(std::move(listener)) : nullptr); + kitchen_sink_error err{}; + auto result = kitchen_sink_kitchen_maybe_subscribe(static_cast(listener_ctx.release()), listener_vtable, &err); + detail::check(err); + return result; +} + /** Optional object in both directions */ inline std::optional maybe_gadget(const std::optional& fallback) { check_library(); kitchen_sink_error err{}; auto result = kitchen_sink_kitchen_maybe_gadget(fallback.has_value() ? fallback->handle() : nullptr, &err); detail::check(err); - if (!result) return std::nullopt; - return Gadget(adopt, result); + return result != nullptr ? std::optional(std::in_place, adopt, result) : std::nullopt; } /** A list of objects */ @@ -854,76 +1217,61 @@ inline std::vector gadgets() { kitchen_sink_error err{}; auto result = kitchen_sink_kitchen_gadgets(&out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - std::vector ret{}; - { - size_t ret_n = result_r.read_count(); - ret.reserve(result_r.reserve_hint(ret_n)); - for (size_t ret_i = 0; ret_i < ret_n; ++ret_i) { - Gadget ret_item = Gadget(adopt, reinterpret_cast(static_cast(result_r.read_u64()))); - ret.push_back(std::move(ret_item)); - } - } - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_list_Gadget); } /** - * A lazy, move-only range over the `Gadget` elements produced by `stream_gadgets()`. + * A lazy, move-only range over the `Gadget` elements `stream_gadgets()` produces. * - * Each iteration step pulls exactly one element from the producer, so results stream in constant memory. The range owns the producer-side iterator and releases it exactly once: eagerly when the range is exhausted, or from the destructor when iteration is abandoned early. + * Each step pulls one element from the producer, so results stream in constant memory. The range releases the producer iterator exactly once: when it's exhausted, or from the destructor when iteration stops early. */ class StreamGadgetsIterator { kitchen_sink_kitchen_StreamGadgetsIterator* handle_; public: - /** Adopts ownership of a raw producer iterator. */ + /** Adopts a producer iterator. */ explicit StreamGadgetsIterator(adopt_t, kitchen_sink_kitchen_StreamGadgetsIterator* h) noexcept : handle_(h) {} + /** Releases the producer iterator if it's still held. */ ~StreamGadgetsIterator() { - if (handle_) kitchen_sink_kitchen_StreamGadgetsIterator_destroy(handle_); + if (handle_ != nullptr) kitchen_sink_kitchen_StreamGadgetsIterator_destroy(handle_); } StreamGadgetsIterator(const StreamGadgetsIterator&) = delete; StreamGadgetsIterator& operator=(const StreamGadgetsIterator&) = delete; - StreamGadgetsIterator(StreamGadgetsIterator&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s iterator; `other` becomes empty. */ + StreamGadgetsIterator(StreamGadgetsIterator&& other) noexcept : handle_(other.handle_) { other.handle_ = nullptr; } + /** Releases the current iterator and takes over `other`'s. */ StreamGadgetsIterator& operator=(StreamGadgetsIterator&& other) noexcept { if (this != &other) { - if (handle_) kitchen_sink_kitchen_StreamGadgetsIterator_destroy(handle_); + if (handle_ != nullptr) kitchen_sink_kitchen_StreamGadgetsIterator_destroy(handle_); handle_ = other.handle_; other.handle_ = nullptr; } return *this; } - /** Pulls the next element from the producer, or `std::nullopt` once exhausted (which releases the producer iterator eagerly). A producer error throws the generic Error after releasing the iterator. */ + /** Pulls the next element, or `std::nullopt` once exhausted (which releases the producer iterator). A producer error releases the iterator and throws InternalError. */ std::optional next() { - if (!handle_) return std::nullopt; + if (handle_ == nullptr) return std::nullopt; kitchen_sink_error err{}; kitchen_sink_kitchen_Gadget* item{}; int32_t has_item = kitchen_sink_kitchen_StreamGadgetsIterator_next(handle_, &item, &err); - if (err.code != 0) { + if (err.code != 0 || has_item == 0) { kitchen_sink_kitchen_StreamGadgetsIterator_destroy(handle_); handle_ = nullptr; detail::check(err); - } - if (has_item == 0) { - kitchen_sink_kitchen_StreamGadgetsIterator_destroy(handle_); - handle_ = nullptr; return std::nullopt; } - return Gadget(adopt, item); + return std::optional(std::in_place, Gadget(adopt, item)); } /** Sentinel type marking the end of the range. */ struct sentinel {}; - /** Single-pass input iterator; each increment pulls one element from the producer. */ + /** Single-pass input iterator; each increment pulls one element. */ class iterator { StreamGadgetsIterator* range_; std::optional current_; @@ -955,7 +1303,7 @@ public: /** * An iterator over objects * - * @return A lazy `StreamGadgetsIterator` range that streams one element per iteration step and releases the producer iterator when exhausted or destroyed. + * @return A lazy `StreamGadgetsIterator` range that pulls one element per step and releases the producer iterator when exhausted or destroyed. */ inline StreamGadgetsIterator stream_gadgets() { check_library(); @@ -965,22 +1313,18 @@ inline StreamGadgetsIterator stream_gadgets() { return StreamGadgetsIterator(adopt, iter); } -/** Optional struct return */ +/** + * Optional struct return + * + * @throws KitchenErrorsError when the call fails with one of the domain's codes. + */ inline std::optional maybe_item(int64_t id) { check_library(); size_t out_len = 0; kitchen_sink_error err{}; auto result = kitchen_sink_kitchen_maybe_item(id, &out_len, &err); detail::check_kitchen(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - std::optional ret{}; - if (result_r.read_option_flag()) { - Item ret_v = detail::read_Item(result_r); - ret = std::move(ret_v); - } - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_opt_Item); } /** List of structs */ @@ -990,19 +1334,7 @@ inline std::vector list_items() { kitchen_sink_error err{}; auto result = kitchen_sink_kitchen_list_items(&out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - std::vector ret{}; - { - size_t ret_n = result_r.read_count(); - ret.reserve(result_r.reserve_hint(ret_n)); - for (size_t ret_i = 0; ret_i < ret_n; ++ret_i) { - Item ret_item = detail::read_Item(result_r); - ret.push_back(std::move(ret_item)); - } - } - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_list_Item); } /** Map return type */ @@ -1012,81 +1344,62 @@ inline std::unordered_map get_attrs() { kitchen_sink_error err{}; auto result = kitchen_sink_kitchen_get_attrs(&out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - std::unordered_map ret{}; - { - size_t ret_n = result_r.read_count(); - for (size_t ret_i = 0; ret_i < ret_n; ++ret_i) { - std::string ret_key = result_r.read_string(); - int32_t ret_val = result_r.read_i32(); - ret.emplace(std::move(ret_key), std::move(ret_val)); - } - } - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_map_string_i32); } /** - * A lazy, move-only range over the `Item` elements produced by `stream_items()`. + * A lazy, move-only range over the `Item` elements `stream_items()` produces. * - * Each iteration step pulls exactly one element from the producer, so results stream in constant memory. The range owns the producer-side iterator and releases it exactly once: eagerly when the range is exhausted, or from the destructor when iteration is abandoned early. + * Each step pulls one element from the producer, so results stream in constant memory. The range releases the producer iterator exactly once: when it's exhausted, or from the destructor when iteration stops early. */ class StreamItemsIterator { kitchen_sink_kitchen_StreamItemsIterator* handle_; public: - /** Adopts ownership of a raw producer iterator. */ + /** Adopts a producer iterator. */ explicit StreamItemsIterator(adopt_t, kitchen_sink_kitchen_StreamItemsIterator* h) noexcept : handle_(h) {} + /** Releases the producer iterator if it's still held. */ ~StreamItemsIterator() { - if (handle_) kitchen_sink_kitchen_StreamItemsIterator_destroy(handle_); + if (handle_ != nullptr) kitchen_sink_kitchen_StreamItemsIterator_destroy(handle_); } StreamItemsIterator(const StreamItemsIterator&) = delete; StreamItemsIterator& operator=(const StreamItemsIterator&) = delete; - StreamItemsIterator(StreamItemsIterator&& other) noexcept : handle_(other.handle_) { - other.handle_ = nullptr; - } + /** Transfers `other`'s iterator; `other` becomes empty. */ + StreamItemsIterator(StreamItemsIterator&& other) noexcept : handle_(other.handle_) { other.handle_ = nullptr; } + /** Releases the current iterator and takes over `other`'s. */ StreamItemsIterator& operator=(StreamItemsIterator&& other) noexcept { if (this != &other) { - if (handle_) kitchen_sink_kitchen_StreamItemsIterator_destroy(handle_); + if (handle_ != nullptr) kitchen_sink_kitchen_StreamItemsIterator_destroy(handle_); handle_ = other.handle_; other.handle_ = nullptr; } return *this; } - /** Pulls the next element from the producer, or `std::nullopt` once exhausted (which releases the producer iterator eagerly). A producer error throws the generic Error after releasing the iterator. */ + /** Pulls the next element, or `std::nullopt` once exhausted (which releases the producer iterator). A producer error releases the iterator and throws InternalError. */ std::optional next() { - if (!handle_) return std::nullopt; + if (handle_ == nullptr) return std::nullopt; kitchen_sink_error err{}; const uint8_t* item{}; size_t item_len = 0; int32_t has_item = kitchen_sink_kitchen_StreamItemsIterator_next(handle_, &item, &item_len, &err); - if (err.code != 0) { + if (err.code != 0 || has_item == 0) { kitchen_sink_kitchen_StreamItemsIterator_destroy(handle_); handle_ = nullptr; detail::check(err); - } - if (has_item == 0) { - kitchen_sink_kitchen_StreamItemsIterator_destroy(handle_); - handle_ = nullptr; return std::nullopt; } - detail::BufferGuard item_guard(item, item_len); - detail::BufferReader item_r(item, item_len); - Item value = detail::read_Item(item_r); - item_r.expect_end(); - return value; + return std::optional(std::in_place, detail::take(item, item_len, &detail::read_Item)); } /** Sentinel type marking the end of the range. */ struct sentinel {}; - /** Single-pass input iterator; each increment pulls one element from the producer. */ + /** Single-pass input iterator; each increment pulls one element. */ class iterator { StreamItemsIterator* range_; std::optional current_; @@ -1118,7 +1431,7 @@ public: /** * Iterator return type * - * @return A lazy `StreamItemsIterator` range that streams one element per iteration step and releases the producer iterator when exhausted or destroyed. + * @return A lazy `StreamItemsIterator` range that pulls one element per step and releases the producer iterator when exhausted or destroyed. */ inline StreamItemsIterator stream_items() { check_library(); @@ -1135,14 +1448,14 @@ inline Token cross_module_token() { kitchen_sink_error err{}; auto result = kitchen_sink_kitchen_cross_module_token(&out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Token ret = detail::read_Token(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Token); } -/** Async operation */ +/** + * Async operation + * + * @return A future settled from a producer thread; get() rethrows the call's error. + */ inline std::future do_async(std::string_view input) { check_library(); auto promise = std::make_unique>(); @@ -1151,8 +1464,8 @@ inline std::future do_async(std::string_view input) { std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { p->set_value(detail::take_string(result_ptr, result_len)); } @@ -1164,7 +1477,12 @@ inline std::future do_async(std::string_view input) { return future; } -/** Cancellable async operation */ +/** + * Cancellable async operation + * + * @param cancel_token Cancels the call; the future then throws Cancelled. + * @return A future settled from a producer thread; get() rethrows the call's error. + */ inline std::future do_cancellable(std::string_view input, const CancelToken& cancel_token = CancelToken::none()) { check_library(); auto promise = std::make_unique>(); @@ -1173,8 +1491,8 @@ inline std::future do_cancellable(std::string_view input, const Can std::unique_ptr> p(static_cast*>(context)); try { if (err != nullptr && err->code != 0) { - std::string msg = detail::error_message(*err); - p->set_exception(detail::make_error(err->code, msg)); + std::string message = detail::error_message(*err); + p->set_exception(detail::make_internal_error(err->code, message)); } else { p->set_value(detail::take_string(result_ptr, result_len)); } diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__CMakeLists_txt.snap deleted file mode 100644 index c1ee2d9a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__CMakeLists_txt.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.14) -project(nested_modules_cpp VERSION 0.1.0 LANGUAGES CXX) - -# The producer library to link: a full path from the NESTED_MODULES_LIBRARY cache or -# environment variable, or else the library name `nested_modules` (a CMake target -# of that name, or libnested_modules on the linker search path). -set(NESTED_MODULES_LIBRARY "$ENV{NESTED_MODULES_LIBRARY}" CACHE FILEPATH "Path to the nested_modules native library") - -add_library(nested_modules_cpp INTERFACE) -add_library(nested_modules::cpp ALIAS nested_modules_cpp) -target_include_directories(nested_modules_cpp INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}) -target_compile_features(nested_modules_cpp INTERFACE cxx_std_17) -if(NESTED_MODULES_LIBRARY) - target_link_libraries(nested_modules_cpp INTERFACE "${NESTED_MODULES_LIBRARY}") -else() - target_link_libraries(nested_modules_cpp INTERFACE nested_modules) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__README_md.snap deleted file mode 100644 index 2e417184..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__README_md.snap +++ /dev/null @@ -1,43 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# nested_modules C++ bindings - -A header-only C++17 wrapper over the `nested_modules` C ABI. `nested_modules.hpp` -includes the C header beside it; every declaration lives in -`namespace nested_modules`. - -## Prerequisites - -- CMake 3.14 or later -- A C++17 compiler -- The `nested_modules` native library (`libnested_modules.so`, `libnested_modules.dylib`, or - `nested_modules.dll`) - -## Use with CMake - -```cmake -add_subdirectory(path/to/cpp) -add_executable(app main.cpp) -target_link_libraries(app PRIVATE nested_modules::cpp) -``` - -`nested_modules::cpp` adds this directory to the include path, requires -C++17, and links the native library: the file named by the `NESTED_MODULES_LIBRARY` -CMake cache variable or environment variable when set, or else -`nested_modules` by name. - -```cpp -#include "nested_modules.hpp" -``` - -The first free function, constructor, or static member you call checks -that the linked library matches these headers and throws -`nested_modules::LoadError` if it doesn't; call `nested_modules::check_library()` -at startup to check eagerly. - - diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_h.snap b/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_h.snap deleted file mode 100644 index 27c97f2b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_h.snap +++ /dev/null @@ -1,144 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef NESTED_MODULES_H -#define NESTED_MODULES_H - -#include -#include -#include - -#ifndef NESTED_MODULES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef NESTED_MODULES_BUILD -# define NESTED_MODULES_API __declspec(dllexport) -# else -# define NESTED_MODULES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define NESTED_MODULES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define NESTED_MODULES_API __attribute__((visibility("default"))) -# else -# define NESTED_MODULES_API -# endif -#endif - -#ifndef NESTED_MODULES_DEPRECATED -# if defined(_MSC_VER) -# define NESTED_MODULES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define NESTED_MODULES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define NESTED_MODULES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports nested_modules_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define NESTED_MODULES_ABI_VERSION 3u -NESTED_MODULES_API uint32_t nested_modules_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by nested_modules_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct nested_modules_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} nested_modules_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -NESTED_MODULES_API void nested_modules_error_set(nested_modules_error* err, int32_t code, const char* message); -NESTED_MODULES_API void nested_modules_error_clear(nested_modules_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - nested_modules_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -NESTED_MODULES_API void nested_modules_error_free(nested_modules_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -NESTED_MODULES_API void nested_modules_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct nested_modules_cancel_token nested_modules_cancel_token; -NESTED_MODULES_API nested_modules_cancel_token* nested_modules_cancel_token_create(void); -NESTED_MODULES_API void nested_modules_cancel_token_cancel(nested_modules_cancel_token* token); -NESTED_MODULES_API bool nested_modules_cancel_token_is_cancelled(const nested_modules_cancel_token* token); -NESTED_MODULES_API void nested_modules_cancel_token_destroy(nested_modules_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -NESTED_MODULES_API uint8_t* nested_modules_alloc(uint32_t size); -NESTED_MODULES_API void nested_modules_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -NESTED_MODULES_API uint64_t nested_modules_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define NESTED_MODULES_GRAPHICS_CHECKSUM 0x44d31b5b10fdb90dull -NESTED_MODULES_API uint64_t nested_modules_graphics_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with nested_modules_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * nested_modules_buffer.h declares a C struct and codec for each. - */ - -/** A measurement unit shared with nested submodules. */ -typedef enum { nested_modules_graphics_Unit_pixels = 0, nested_modules_graphics_Unit_points = 1 } nested_modules_graphics_Unit; -/** Error codes reported by throwing functions in the `graphics` module tree. */ -typedef enum { nested_modules_graphics_GraphicsError_OutOfBounds = 1 } nested_modules_graphics_GraphicsError; - -// Module: graphics -/** Construct a Canvas. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_new_canvas(int32_t width, size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes -/** Construct a Shape. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_make_shape(int32_t sides, size_t* out_len, nested_modules_error* out_err); -/** Nested function returning a struct owned by an ancestor module. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_describe(size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes_geometry -/** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_geometry_compute(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, nested_modules_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // NESTED_MODULES_H - -// END nested_modules.h diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_hpp.snap b/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_hpp.snap index 1cd1bbd7..11f61ed0 100644 --- a/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_hpp.snap +++ b/crates/weaveffi-cli/tests/snapshots/cpp_nested_modules__nested_modules_hpp.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #pragma once @@ -26,13 +26,13 @@ source: crates/weaveffi-cli/tests/snapshots.rs namespace nested_modules { /** - * Base exception for every error reported through the C ABI. + * An error a call that declares errors reports. * - * Positive codes are the declared error codes (see the typed domain - * exceptions); negative codes are runtime traps: -1 a generic producer - * error, -2 a producer panic, -3 a marshalling failure, -4 a - * callback-interface implementation that threw, and -5 a cancelled call - * (thrown as Cancelled). + * A positive code is a declared code of the module's error domain, thrown as + * that domain's typed subclass. A negative code is a runtime failure: -1 a + * generic producer error, -2 a producer panic, -3 a marshalling failure, and + * -4 a callback-interface implementation that failed. A cancelled async call + * (-5) throws Cancelled. */ class Error : public std::runtime_error { int32_t code_; @@ -45,7 +45,7 @@ public: int32_t code() const noexcept { return code_; } }; -/** Thrown by an async call whose CancelToken was cancelled (code -5). */ +/** Thrown by an async call that was cancelled through its CancelToken (code -5). */ class Cancelled : public Error { public: /** Builds a cancellation error carrying the producer's message. */ @@ -53,13 +53,39 @@ public: }; /** - * Thrown on first use when the linked nested_modules library was built for a - * different C ABI revision or a different API contract than this header. + * Thrown by a call that declares no errors when it fails anyway: a producer + * panic (-2), a marshalling failure (-3), a callback-interface failure the + * producer let through (-4), or another runtime failure (-1). These are bugs, + * so no signature declares them; `what()` names the code and the producer's + * message. */ -class LoadError : public Error { +class InternalError : public std::runtime_error { + int32_t code_; + std::string message_; + +public: + /** Builds an internal error carrying `code` and the producer's `message`. */ + InternalError(int32_t code, const std::string& message) + : std::runtime_error("nested_modules: runtime error " + std::to_string(code) + ": " + message), + code_(code), + message_(message) {} + + /** The runtime error code (negative). */ + int32_t code() const noexcept { return code_; } + + /** The producer's message. */ + const std::string& message() const noexcept { return message_; } +}; + +/** + * Thrown on first use when the linked nested_modules library implements a + * different C ABI revision than this header, or lacks or changed a + * declaration this header was generated with. + */ +class LoadError : public std::runtime_error { public: /** Builds a load error describing the mismatch. */ - explicit LoadError(const std::string& message) : Error(-1, message) {} + explicit LoadError(const std::string& message) : std::runtime_error(message) {} }; /** @@ -70,7 +96,7 @@ struct adopt_t { explicit adopt_t() = default; }; -/** The adopt_t tag value: `Gadget g(adopt, raw)`. */ +/** The adopt_t tag value: `Store s(adopt, raw)`. */ inline constexpr adopt_t adopt{}; /** @@ -136,10 +162,10 @@ public: namespace detail { -/** The exception for a runtime (non-domain) error code. */ -inline std::exception_ptr make_error(int32_t code, const std::string& message) { +/** The exception for a failed call that declares no errors: Cancelled for -5, else InternalError. */ +inline std::exception_ptr make_internal_error(int32_t code, const std::string& message) { if (code == -5) return std::make_exception_ptr(Cancelled(message)); - return std::make_exception_ptr(Error(code, message)); + return std::make_exception_ptr(InternalError(code, message)); } /** The message of a nonzero error slot. */ @@ -147,10 +173,10 @@ inline std::string error_message(const nested_modules_error& err) { return std::string(err.message != nullptr ? err.message : "unknown error"); } -/** Throws the generic Error (or Cancelled) if `err` carries a nonzero code. */ +/** Throws InternalError if `err` carries a nonzero code (a call that declares no errors). */ inline void check(nested_modules_error& err) { if (err.code == 0) return; - std::exception_ptr ex = make_error(err.code, error_message(err)); + std::exception_ptr ex = make_internal_error(err.code, error_message(err)); nested_modules_error_clear(&err); std::rethrow_exception(ex); } @@ -183,7 +209,7 @@ inline std::string_view borrow_string(const uint8_t* ptr, size_t len) noexcept { return std::string_view(reinterpret_cast(ptr), len); } -/** Views a borrowed byte run (null when empty) as an owned vector. */ +/** Copies a borrowed byte run (null when empty) into a vector. */ inline std::vector borrow_bytes(const uint8_t* ptr, size_t len) { if (ptr == nullptr || len == 0) return std::vector(); return std::vector(ptr, ptr + len); @@ -194,24 +220,96 @@ inline const uint8_t* utf8(std::string_view s) noexcept { return reinterpret_cast(s.data()); } +/** + * Hands `len` bytes from `data` to the producer through a callback method's + * out slots: copies them into a run from nested_modules_alloc, which the + * producer adopts and frees. + */ +inline void hand_over(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = nullptr; + if (len > 0) { + run = nested_modules_alloc(len); + if (run == nullptr) throw std::bad_alloc(); + std::memcpy(run, data, len); + } + *out_ptr = run; + *out_len = len; +} + +/** One declaration this header was generated with: its contract id and hash, and its dotted path. */ +struct ContractEntry { + /** FNV-1a 64 of the declaration's dotted path. */ + uint64_t id; + /** FNV-1a 64 of the declaration's canonical signature. */ + uint64_t hash; + /** The declaration's dotted path (`kv.Store.put`). */ + const char* path; +}; + +/** + * Why the contract table `contract` returns doesn't satisfy `expected`, + * naming the first declaration it lacks or declares differently, or an empty + * string when it has them all. Entries the producer has and `expected` + * doesn't are fine. + */ +template +std::string contract_mismatch(const nested_modules_contract_entry* (*contract)(size_t*), + const ContractEntry (&expected)[N]) { + size_t len = 0; + const nested_modules_contract_entry* table = contract(&len); + if (table == nullptr) len = 0; + for (const ContractEntry& want : expected) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < want.id) { + lo = mid + 1; + } else { + hi = mid; + } + } + if (lo == len || table[lo].id != want.id) { + return std::string(want.path) + " is missing from the library"; + } + if (table[lo].hash != want.hash) { + return std::string(want.path) + " changed since these bindings were generated"; + } + } + return std::string(); +} + +/** The declarations of module `graphics` this header was generated with. */ +inline constexpr ContractEntry graphics_contract[] = { + {0x2c29eef9327a6bd4ull, 0x12983d5271651389ull, "graphics.Unit"}, + {0x304088bbc28f0015ull, 0x97e6e809913be6b6ull, "graphics.new_canvas"}, + {0x49f8837532df9ac6ull, 0x32e5b27e81ee0957ull, "graphics.Canvas"}, + {0x55717fa027336b47ull, 0xb2bd3a6bdcce48c8ull, "graphics.shapes.describe"}, + {0x7626b91b8f1f53cdull, 0x867bbf7cf0f41c74ull, "graphics.shapes.geometry.compute"}, + {0xb3efcb50d9a61ab7ull, 0x3cb0207dfad2a149ull, "graphics.GraphicsError"}, + {0xc62b16a9591b744bull, 0x7895610c8487eb1bull, "graphics.shapes.Shape"}, + {0xe13001f4983ccd35ull, 0x16e7de070b4a4cfcull, "graphics.shapes.geometry.Detail"}, + {0xf3f658bcfc5dba62ull, 0x2665ccb47356e746ull, "graphics.shapes.make_shape"}, +}; + } // namespace detail /** - * Verifies that the linked nested_modules library implements the C ABI revision - * and the module contracts this header was generated against, throwing + * Verifies that the linked nested_modules library implements C ABI revision + * 4 and every declaration this header was generated with, throwing * LoadError naming the first mismatch. The check runs once; every free * function, constructor, and static member calls this before its first * native call, so calling it at startup is optional. */ inline void check_library() { - static const std::string failure = [] { + static const std::string failure = []() -> std::string { uint32_t abi = nested_modules_abi_version(); if (abi != NESTED_MODULES_ABI_VERSION) { return "nested_modules: C ABI revision mismatch (header " + std::to_string(NESTED_MODULES_ABI_VERSION) + ", library " + std::to_string(abi) + ")"; } - if (nested_modules_graphics_checksum() != 0x44d31b5b10fdb90dull) { - return std::string("nested_modules: module 'graphics' does not match the linked library (contract checksum mismatch); regenerate the bindings for this build"); + if (std::string why = detail::contract_mismatch(nested_modules_graphics_contract, detail::graphics_contract); !why.empty()) { + return "nested_modules: " + why; } return std::string(); }(); @@ -265,7 +363,7 @@ public: /** Writes a byte length or an element count as a u32. */ void write_len(size_t n) { - if (static_cast(n) > UINT32_MAX) throw Error(-3, "value buffer length exceeds u32"); + if (static_cast(n) > UINT32_MAX) throw std::length_error("value buffer length exceeds u32"); append_le(static_cast(n)); } @@ -281,24 +379,29 @@ public: /** Writes an optional's presence flag: 0 absent, 1 present. */ void write_option_flag(bool present) { buf_.push_back(present ? 1 : 0); } + + /** + * Writes an object token: a fresh strong reference from the wrapper's + * `_clone`, which the reader adopts while `v` keeps its own. + */ + template + void write_object(const T& v) { + write_u64(static_cast(reinterpret_cast(v.clone_handle()))); + } }; /** * Decodes values from the value-buffer wire format. A malformed buffer is a - * producer/consumer contract violation (both sides are generated from one - * API), so every decode failure throws the generic Error with the - * marshalling code -3. Object tokens adopted before a failure live in RAII - * wrappers that release them on unwind. + * producer bug (both sides are generated from one API), so every decode + * failure throws InternalError with the marshalling code -3. Object tokens + * adopted before a failure live in RAII wrappers that release them on + * unwind. */ class BufferReader { const uint8_t* data_; size_t len_; size_t pos_; - [[noreturn]] static void fail(const char* what) { - throw Error(-3, std::string("malformed value buffer: ") + what); - } - void require(size_t n, const char* what) const { if (len_ - pos_ < n) fail(what); } @@ -315,6 +418,12 @@ class BufferReader { } public: + /** Throws the marshalling failure (-3) for a malformed buffer. */ + [[noreturn]] static void fail(const char* what) { + throw InternalError(-3, std::string("malformed value buffer: ") + what); + } + + /** Reads from the `len` bytes at `data` (null when empty). */ BufferReader(const uint8_t* data, size_t len) noexcept : data_(data), len_(data != nullptr ? len : 0), pos_(0) {} /** Bytes not yet consumed. */ @@ -386,6 +495,14 @@ public: return b != 0; } + /** Reads an object token, adopting the strong reference it carries into a `T` wrapper. */ + template + T read_object() { + auto* raw = reinterpret_cast(static_cast(read_u64())); + if (raw == nullptr) fail("null object token"); + return T(adopt, raw); + } + /** Rejects unconsumed bytes after decoding a complete value. */ void expect_end() const { if (pos_ != len_) fail("trailing bytes after value"); @@ -408,12 +525,28 @@ struct BufferGuard { } }; +/** Decodes one complete `T` from a borrowed buffer with `read`, rejecting trailing bytes. */ +template +T decode(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferReader r(ptr, len); + T value = read(r); + r.expect_end(); + return value; +} + +/** Decodes one complete `T` from a producer-returned buffer with `read`, then releases the buffer. */ +template +T take(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferGuard guard(ptr, len); + return decode(ptr, len, read); +} + } // namespace detail /** A measurement unit shared with nested submodules. */ enum class Unit : int32_t { pixels = 0, - points = 1 + points = 1, }; struct Canvas; @@ -443,77 +576,93 @@ struct Canvas { namespace detail { -/** Encodes a `Detail` in the value-buffer wire format. */ -inline void write_Detail(BufferWriter& w, const Detail& v) { - w.write_f64(v.area); +inline void write_Canvas(BufferWriter& w, const Canvas& v); +inline Canvas read_Canvas(BufferReader& r); +inline void write_Shape(BufferWriter& w, const Shape& v); +inline Shape read_Shape(BufferReader& r); +inline void write_Detail(BufferWriter& w, const Detail& v); +inline Detail read_Detail(BufferReader& r); + +/** Writes one `Canvas` to a value buffer. */ +inline void write_Canvas(BufferWriter& w, const Canvas& v) { + w.write_i32(v.width); + detail::write_Shape(w, v.root); } -/** Decodes a `Detail` from the value-buffer wire format. */ -inline Detail read_Detail(BufferReader& r) { - double f_area = r.read_f64(); - return Detail{std::move(f_area)}; +/** Reads one `Canvas` from a value buffer. */ +inline Canvas read_Canvas(BufferReader& r) { + return Canvas{ + r.read_i32(), + detail::read_Shape(r), + }; } -/** Encodes a `Shape` in the value-buffer wire format. */ +/** Writes one `Shape` to a value buffer. */ inline void write_Shape(BufferWriter& w, const Shape& v) { w.write_i32(v.sides); w.write_i32(static_cast(v.unit)); detail::write_Detail(w, v.detail); } -/** Decodes a `Shape` from the value-buffer wire format. */ +/** Reads one `Shape` from a value buffer. */ inline Shape read_Shape(BufferReader& r) { - int32_t f_sides = r.read_i32(); - Unit f_unit = static_cast(r.read_i32()); - Detail f_detail = detail::read_Detail(r); - return Shape{std::move(f_sides), std::move(f_unit), std::move(f_detail)}; + return Shape{ + r.read_i32(), + static_cast(r.read_i32()), + detail::read_Detail(r), + }; } -/** Encodes a `Canvas` in the value-buffer wire format. */ -inline void write_Canvas(BufferWriter& w, const Canvas& v) { - w.write_i32(v.width); - detail::write_Shape(w, v.root); +/** Writes one `Detail` to a value buffer. */ +inline void write_Detail(BufferWriter& w, const Detail& v) { + w.write_f64(v.area); } -/** Decodes a `Canvas` from the value-buffer wire format. */ -inline Canvas read_Canvas(BufferReader& r) { - int32_t f_width = r.read_i32(); - Shape f_root = detail::read_Shape(r); - return Canvas{std::move(f_width), std::move(f_root)}; +/** Reads one `Detail` from a value buffer. */ +inline Detail read_Detail(BufferReader& r) { + return Detail{ + r.read_f64(), + }; } } // namespace detail -/** Typed errors reported by the `graphics` module's throwing functions. */ +/** The errors the `graphics` module's throwing calls report; catch a code's subclass or this. */ class GraphicsError : public Error { public: - GraphicsError(int32_t code, const std::string& msg) : Error(code, msg) {} + /** Builds an error carrying a domain `code` and `message`. */ + GraphicsError(int32_t code, const std::string& message) : Error(code, message) {} }; -/** coordinate out of bounds */ +/** + * coordinate out of bounds + * + * Code 1. + */ class OutOfBoundsError : public GraphicsError { public: - explicit OutOfBoundsError(const std::string& msg) : GraphicsError(1, msg) {} + /** Builds the error with the producer's `message` and the code's fields. */ + explicit OutOfBoundsError(const std::string& message) : GraphicsError(1, message) {} }; namespace detail { -/** Map a `graphics` error code and payload to its typed exception (the runtime exception for negative codes, GraphicsError for unknown codes). */ -inline std::exception_ptr make_graphics_error(int32_t code, const std::string& msg, const uint8_t* payload_ptr, size_t payload_len) { - if (code < 0) return make_error(code, msg); +/** The exception for a failed `GraphicsError` call: its code's subclass, Cancelled, or the root Error. */ +inline std::exception_ptr make_graphics_error(int32_t code, const std::string& message, const uint8_t* payload_ptr, size_t payload_len) { + if (code == -5) return std::make_exception_ptr(Cancelled(message)); + if (code < 0) return std::make_exception_ptr(Error(code, message)); (void)payload_ptr; (void)payload_len; switch (code) { - case 1: return std::make_exception_ptr(OutOfBoundsError(msg)); - default: return std::make_exception_ptr(GraphicsError(code, msg)); + case 1: return std::make_exception_ptr(OutOfBoundsError(message)); + default: return std::make_exception_ptr(GraphicsError(code, message)); } } -/** Throw the typed `graphics` exception if `err` carries a nonzero code. */ +/** Throws the typed `GraphicsError` exception if `err` carries a nonzero code. */ inline void check_graphics(nested_modules_error& err) { if (err.code == 0) return; - std::string msg = error_message(err); - std::exception_ptr ex = make_graphics_error(err.code, msg, err.payload_ptr, err.payload_len); + std::exception_ptr ex = make_graphics_error(err.code, error_message(err), err.payload_ptr, err.payload_len); nested_modules_error_clear(&err); std::rethrow_exception(ex); } @@ -529,11 +678,7 @@ inline Canvas new_canvas(int32_t width) { nested_modules_error err{}; auto result = nested_modules_graphics_new_canvas(width, &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Canvas ret = detail::read_Canvas(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Canvas); } } // namespace graphics @@ -547,11 +692,7 @@ inline Shape make_shape(int32_t sides) { nested_modules_error err{}; auto result = nested_modules_graphics_shapes_make_shape(sides, &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Shape ret = detail::read_Shape(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Shape); } /** Nested function returning a struct owned by an ancestor module. */ @@ -561,18 +702,18 @@ inline Canvas describe() { nested_modules_error err{}; auto result = nested_modules_graphics_shapes_describe(&out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Canvas ret = detail::read_Canvas(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Canvas); } } // namespace graphics::shapes namespace graphics::shapes::geometry { -/** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ +/** + * Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. + * + * @throws GraphicsError when the call fails with one of the domain's codes. + */ inline Detail compute(const Shape& shape) { check_library(); detail::BufferWriter shape_buf; @@ -581,11 +722,7 @@ inline Detail compute(const Shape& shape) { nested_modules_error err{}; auto result = nested_modules_graphics_shapes_geometry_compute(shape_buf.data(), shape_buf.size(), &out_len, &err); detail::check_graphics(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Detail ret = detail::read_Detail(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Detail); } } // namespace graphics::shapes::geometry diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/cpp_shapes__CMakeLists_txt.snap deleted file mode 100644 index 544a78cd..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__CMakeLists_txt.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.14) -project(shapes_cpp VERSION 0.1.0 LANGUAGES CXX) - -# The producer library to link: a full path from the SHAPES_LIBRARY cache or -# environment variable, or else the library name `shapes` (a CMake target -# of that name, or libshapes on the linker search path). -set(SHAPES_LIBRARY "$ENV{SHAPES_LIBRARY}" CACHE FILEPATH "Path to the shapes native library") - -add_library(shapes_cpp INTERFACE) -add_library(shapes::cpp ALIAS shapes_cpp) -target_include_directories(shapes_cpp INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}) -target_compile_features(shapes_cpp INTERFACE cxx_std_17) -if(SHAPES_LIBRARY) - target_link_libraries(shapes_cpp INTERFACE "${SHAPES_LIBRARY}") -else() - target_link_libraries(shapes_cpp INTERFACE shapes) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/cpp_shapes__README_md.snap deleted file mode 100644 index c36d04f3..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__README_md.snap +++ /dev/null @@ -1,43 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# shapes C++ bindings - -A header-only C++17 wrapper over the `shapes` C ABI. `shapes.hpp` -includes the C header beside it; every declaration lives in -`namespace shapes`. - -## Prerequisites - -- CMake 3.14 or later -- A C++17 compiler -- The `shapes` native library (`libshapes.so`, `libshapes.dylib`, or - `shapes.dll`) - -## Use with CMake - -```cmake -add_subdirectory(path/to/cpp) -add_executable(app main.cpp) -target_link_libraries(app PRIVATE shapes::cpp) -``` - -`shapes::cpp` adds this directory to the include path, requires -C++17, and links the native library: the file named by the `SHAPES_LIBRARY` -CMake cache variable or environment variable when set, or else -`shapes` by name. - -```cpp -#include "shapes.hpp" -``` - -The first free function, constructor, or static member you call checks -that the linked library matches these headers and throws -`shapes::LoadError` if it doesn't; call `shapes::check_library()` -at startup to check eagerly. - - diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_h.snap b/crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_h.snap deleted file mode 100644 index df20c3f5..00000000 --- a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_h.snap +++ /dev/null @@ -1,151 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef SHAPES_H -#define SHAPES_H - -#include -#include -#include - -#ifndef SHAPES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef SHAPES_BUILD -# define SHAPES_API __declspec(dllexport) -# else -# define SHAPES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define SHAPES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define SHAPES_API __attribute__((visibility("default"))) -# else -# define SHAPES_API -# endif -#endif - -#ifndef SHAPES_DEPRECATED -# if defined(_MSC_VER) -# define SHAPES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define SHAPES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define SHAPES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports shapes_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define SHAPES_ABI_VERSION 3u -SHAPES_API uint32_t shapes_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by shapes_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct shapes_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} shapes_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -SHAPES_API void shapes_error_set(shapes_error* err, int32_t code, const char* message); -SHAPES_API void shapes_error_clear(shapes_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - shapes_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -SHAPES_API void shapes_error_free(shapes_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -SHAPES_API void shapes_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct shapes_cancel_token shapes_cancel_token; -SHAPES_API shapes_cancel_token* shapes_cancel_token_create(void); -SHAPES_API void shapes_cancel_token_cancel(shapes_cancel_token* token); -SHAPES_API bool shapes_cancel_token_is_cancelled(const shapes_cancel_token* token); -SHAPES_API void shapes_cancel_token_destroy(shapes_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -SHAPES_API uint8_t* shapes_alloc(uint32_t size); -SHAPES_API void shapes_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -SHAPES_API uint64_t shapes_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define SHAPES_SHAPES_CHECKSUM 0x06acf433ffa2a0f0ull -SHAPES_API uint64_t shapes_shapes_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with shapes_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * shapes_buffer.h declares a C struct and codec for each. - */ - -/** An algebraic shape (sum type with associated data) */ -typedef enum { - /** The empty shape (a unit variant) */ - shapes_shapes_Shape_Empty = 0, - /** A circle with a radius */ - shapes_shapes_Shape_Circle = 1, - /** An axis-aligned rectangle */ - shapes_shapes_Shape_Rectangle = 2, - /** An integer-bounded shape (exercises narrow + wide ints) */ - shapes_shapes_Shape_Bounded = 3, - /** A labeled shape with a small count */ - shapes_shapes_Shape_Labeled = 4 -} shapes_shapes_Shape_Tag; -/** A plain C-style enum (no payloads) */ -typedef enum { shapes_shapes_Channel_Red = 0, shapes_shapes_Channel_Green = 1, shapes_shapes_Channel_Blue = 2 } shapes_shapes_Channel; - -// Module: shapes -/** Render a shape to a string */ -SHAPES_API const uint8_t* shapes_shapes_describe(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, shapes_error* out_err); -/** Scale a shape by a factor, returning a new shape */ -SHAPES_API const uint8_t* shapes_shapes_scale(const uint8_t* shape_ptr, size_t shape_len, double factor, size_t* out_len, shapes_error* out_err); -/** Sum a list of bytes into a wide integer (numerics smoke) */ -SHAPES_API uint64_t shapes_shapes_sum_bytes(const uint8_t* values_ptr, size_t values_len, shapes_error* out_err); -/** Build a Numerics struct from every numeric primitive */ -SHAPES_API const uint8_t* shapes_shapes_make_numerics(int8_t a, int16_t b, int32_t c, int64_t d, uint8_t e, uint16_t f, uint32_t g, uint64_t h, float i, double j, size_t* out_len, shapes_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // SHAPES_H - -// END shapes.h diff --git a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_hpp.snap b/crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_hpp.snap index 0dc0cdfb..a96adda5 100644 --- a/crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_hpp.snap +++ b/crates/weaveffi-cli/tests/snapshots/cpp_shapes__shapes_hpp.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #pragma once @@ -27,13 +27,13 @@ source: crates/weaveffi-cli/tests/snapshots.rs namespace shapes { /** - * Base exception for every error reported through the C ABI. + * An error a call that declares errors reports. * - * Positive codes are the declared error codes (see the typed domain - * exceptions); negative codes are runtime traps: -1 a generic producer - * error, -2 a producer panic, -3 a marshalling failure, -4 a - * callback-interface implementation that threw, and -5 a cancelled call - * (thrown as Cancelled). + * A positive code is a declared code of the module's error domain, thrown as + * that domain's typed subclass. A negative code is a runtime failure: -1 a + * generic producer error, -2 a producer panic, -3 a marshalling failure, and + * -4 a callback-interface implementation that failed. A cancelled async call + * (-5) throws Cancelled. */ class Error : public std::runtime_error { int32_t code_; @@ -46,7 +46,7 @@ public: int32_t code() const noexcept { return code_; } }; -/** Thrown by an async call whose CancelToken was cancelled (code -5). */ +/** Thrown by an async call that was cancelled through its CancelToken (code -5). */ class Cancelled : public Error { public: /** Builds a cancellation error carrying the producer's message. */ @@ -54,13 +54,39 @@ public: }; /** - * Thrown on first use when the linked shapes library was built for a - * different C ABI revision or a different API contract than this header. + * Thrown by a call that declares no errors when it fails anyway: a producer + * panic (-2), a marshalling failure (-3), a callback-interface failure the + * producer let through (-4), or another runtime failure (-1). These are bugs, + * so no signature declares them; `what()` names the code and the producer's + * message. */ -class LoadError : public Error { +class InternalError : public std::runtime_error { + int32_t code_; + std::string message_; + +public: + /** Builds an internal error carrying `code` and the producer's `message`. */ + InternalError(int32_t code, const std::string& message) + : std::runtime_error("shapes: runtime error " + std::to_string(code) + ": " + message), + code_(code), + message_(message) {} + + /** The runtime error code (negative). */ + int32_t code() const noexcept { return code_; } + + /** The producer's message. */ + const std::string& message() const noexcept { return message_; } +}; + +/** + * Thrown on first use when the linked shapes library implements a + * different C ABI revision than this header, or lacks or changed a + * declaration this header was generated with. + */ +class LoadError : public std::runtime_error { public: /** Builds a load error describing the mismatch. */ - explicit LoadError(const std::string& message) : Error(-1, message) {} + explicit LoadError(const std::string& message) : std::runtime_error(message) {} }; /** @@ -71,7 +97,7 @@ struct adopt_t { explicit adopt_t() = default; }; -/** The adopt_t tag value: `Gadget g(adopt, raw)`. */ +/** The adopt_t tag value: `Store s(adopt, raw)`. */ inline constexpr adopt_t adopt{}; /** @@ -137,10 +163,10 @@ public: namespace detail { -/** The exception for a runtime (non-domain) error code. */ -inline std::exception_ptr make_error(int32_t code, const std::string& message) { +/** The exception for a failed call that declares no errors: Cancelled for -5, else InternalError. */ +inline std::exception_ptr make_internal_error(int32_t code, const std::string& message) { if (code == -5) return std::make_exception_ptr(Cancelled(message)); - return std::make_exception_ptr(Error(code, message)); + return std::make_exception_ptr(InternalError(code, message)); } /** The message of a nonzero error slot. */ @@ -148,10 +174,10 @@ inline std::string error_message(const shapes_error& err) { return std::string(err.message != nullptr ? err.message : "unknown error"); } -/** Throws the generic Error (or Cancelled) if `err` carries a nonzero code. */ +/** Throws InternalError if `err` carries a nonzero code (a call that declares no errors). */ inline void check(shapes_error& err) { if (err.code == 0) return; - std::exception_ptr ex = make_error(err.code, error_message(err)); + std::exception_ptr ex = make_internal_error(err.code, error_message(err)); shapes_error_clear(&err); std::rethrow_exception(ex); } @@ -184,7 +210,7 @@ inline std::string_view borrow_string(const uint8_t* ptr, size_t len) noexcept { return std::string_view(reinterpret_cast(ptr), len); } -/** Views a borrowed byte run (null when empty) as an owned vector. */ +/** Copies a borrowed byte run (null when empty) into a vector. */ inline std::vector borrow_bytes(const uint8_t* ptr, size_t len) { if (ptr == nullptr || len == 0) return std::vector(); return std::vector(ptr, ptr + len); @@ -195,24 +221,94 @@ inline const uint8_t* utf8(std::string_view s) noexcept { return reinterpret_cast(s.data()); } +/** + * Hands `len` bytes from `data` to the producer through a callback method's + * out slots: copies them into a run from shapes_alloc, which the + * producer adopts and frees. + */ +inline void hand_over(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = nullptr; + if (len > 0) { + run = shapes_alloc(len); + if (run == nullptr) throw std::bad_alloc(); + std::memcpy(run, data, len); + } + *out_ptr = run; + *out_len = len; +} + +/** One declaration this header was generated with: its contract id and hash, and its dotted path. */ +struct ContractEntry { + /** FNV-1a 64 of the declaration's dotted path. */ + uint64_t id; + /** FNV-1a 64 of the declaration's canonical signature. */ + uint64_t hash; + /** The declaration's dotted path (`kv.Store.put`). */ + const char* path; +}; + +/** + * Why the contract table `contract` returns doesn't satisfy `expected`, + * naming the first declaration it lacks or declares differently, or an empty + * string when it has them all. Entries the producer has and `expected` + * doesn't are fine. + */ +template +std::string contract_mismatch(const shapes_contract_entry* (*contract)(size_t*), + const ContractEntry (&expected)[N]) { + size_t len = 0; + const shapes_contract_entry* table = contract(&len); + if (table == nullptr) len = 0; + for (const ContractEntry& want : expected) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < want.id) { + lo = mid + 1; + } else { + hi = mid; + } + } + if (lo == len || table[lo].id != want.id) { + return std::string(want.path) + " is missing from the library"; + } + if (table[lo].hash != want.hash) { + return std::string(want.path) + " changed since these bindings were generated"; + } + } + return std::string(); +} + +/** The declarations of module `shapes` this header was generated with. */ +inline constexpr ContractEntry shapes_contract[] = { + {0x0d7b3eec49243e7aull, 0x5f232311808e1b1full, "shapes.Channel"}, + {0x183fa8198100860eull, 0xe40c6993cb31d359ull, "shapes.describe"}, + {0x2d1395eb837b848eull, 0x42202a0c12f3a16eull, "shapes.sum_bytes"}, + {0x7f9a5b87ba09b86bull, 0xc5b32fb0efbbaa17ull, "shapes.scale"}, + {0xc8edd779240fda17ull, 0x68e1be78fddb20e4ull, "shapes.Numerics"}, + {0xd3d3881b2fd18a62ull, 0xf24d0eb76a99d918ull, "shapes.make_numerics"}, + {0xf010844afff5ac3cull, 0x50de28ae9b41ff15ull, "shapes.Shape"}, +}; + } // namespace detail /** - * Verifies that the linked shapes library implements the C ABI revision - * and the module contracts this header was generated against, throwing + * Verifies that the linked shapes library implements C ABI revision + * 4 and every declaration this header was generated with, throwing * LoadError naming the first mismatch. The check runs once; every free * function, constructor, and static member calls this before its first * native call, so calling it at startup is optional. */ inline void check_library() { - static const std::string failure = [] { + static const std::string failure = []() -> std::string { uint32_t abi = shapes_abi_version(); if (abi != SHAPES_ABI_VERSION) { return "shapes: C ABI revision mismatch (header " + std::to_string(SHAPES_ABI_VERSION) + ", library " + std::to_string(abi) + ")"; } - if (shapes_shapes_checksum() != 0x06acf433ffa2a0f0ull) { - return std::string("shapes: module 'shapes' does not match the linked library (contract checksum mismatch); regenerate the bindings for this build"); + if (std::string why = detail::contract_mismatch(shapes_shapes_contract, detail::shapes_contract); !why.empty()) { + return "shapes: " + why; } return std::string(); }(); @@ -266,7 +362,7 @@ public: /** Writes a byte length or an element count as a u32. */ void write_len(size_t n) { - if (static_cast(n) > UINT32_MAX) throw Error(-3, "value buffer length exceeds u32"); + if (static_cast(n) > UINT32_MAX) throw std::length_error("value buffer length exceeds u32"); append_le(static_cast(n)); } @@ -282,24 +378,29 @@ public: /** Writes an optional's presence flag: 0 absent, 1 present. */ void write_option_flag(bool present) { buf_.push_back(present ? 1 : 0); } + + /** + * Writes an object token: a fresh strong reference from the wrapper's + * `_clone`, which the reader adopts while `v` keeps its own. + */ + template + void write_object(const T& v) { + write_u64(static_cast(reinterpret_cast(v.clone_handle()))); + } }; /** * Decodes values from the value-buffer wire format. A malformed buffer is a - * producer/consumer contract violation (both sides are generated from one - * API), so every decode failure throws the generic Error with the - * marshalling code -3. Object tokens adopted before a failure live in RAII - * wrappers that release them on unwind. + * producer bug (both sides are generated from one API), so every decode + * failure throws InternalError with the marshalling code -3. Object tokens + * adopted before a failure live in RAII wrappers that release them on + * unwind. */ class BufferReader { const uint8_t* data_; size_t len_; size_t pos_; - [[noreturn]] static void fail(const char* what) { - throw Error(-3, std::string("malformed value buffer: ") + what); - } - void require(size_t n, const char* what) const { if (len_ - pos_ < n) fail(what); } @@ -316,6 +417,12 @@ class BufferReader { } public: + /** Throws the marshalling failure (-3) for a malformed buffer. */ + [[noreturn]] static void fail(const char* what) { + throw InternalError(-3, std::string("malformed value buffer: ") + what); + } + + /** Reads from the `len` bytes at `data` (null when empty). */ BufferReader(const uint8_t* data, size_t len) noexcept : data_(data), len_(data != nullptr ? len : 0), pos_(0) {} /** Bytes not yet consumed. */ @@ -387,6 +494,14 @@ public: return b != 0; } + /** Reads an object token, adopting the strong reference it carries into a `T` wrapper. */ + template + T read_object() { + auto* raw = reinterpret_cast(static_cast(read_u64())); + if (raw == nullptr) fail("null object token"); + return T(adopt, raw); + } + /** Rejects unconsumed bytes after decoding a complete value. */ void expect_end() const { if (pos_ != len_) fail("trailing bytes after value"); @@ -409,13 +524,29 @@ struct BufferGuard { } }; +/** Decodes one complete `T` from a borrowed buffer with `read`, rejecting trailing bytes. */ +template +T decode(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferReader r(ptr, len); + T value = read(r); + r.expect_end(); + return value; +} + +/** Decodes one complete `T` from a producer-returned buffer with `read`, then releases the buffer. */ +template +T take(const uint8_t* ptr, size_t len, T (*read)(BufferReader&)) { + BufferGuard guard(ptr, len); + return decode(ptr, len, read); +} + } // namespace detail /** A plain C-style enum (no payloads) */ enum class Channel : int32_t { Red = 0, Green = 1, - Blue = 2 + Blue = 2, }; struct Numerics; @@ -443,7 +574,7 @@ struct Shape { Circle = 1, Rectangle = 2, Bounded = 3, - Labeled = 4 + Labeled = 4, }; /** The empty shape (a unit variant) */ @@ -479,21 +610,20 @@ struct Shape { std::variant value; /** The tag of the active variant. */ - Tag tag() const { - static constexpr Tag tags[] = { - Tag::Empty, - Tag::Circle, - Tag::Rectangle, - Tag::Bounded, - Tag::Labeled - }; + Tag tag() const noexcept { + static constexpr Tag tags[] = {Tag::Empty, Tag::Circle, Tag::Rectangle, Tag::Bounded, Tag::Labeled}; return tags[value.index()]; } }; namespace detail { -/** Encodes a `Numerics` in the value-buffer wire format. */ +inline void write_Numerics(BufferWriter& w, const Numerics& v); +inline Numerics read_Numerics(BufferReader& r); +inline void write_Shape(BufferWriter& w, const Shape& v); +inline Shape read_Shape(BufferReader& r); + +/** Writes one `Numerics` to a value buffer. */ inline void write_Numerics(BufferWriter& w, const Numerics& v) { w.write_i8(v.a); w.write_i16(v.b); @@ -507,90 +637,88 @@ inline void write_Numerics(BufferWriter& w, const Numerics& v) { w.write_f64(v.j); } -/** Decodes a `Numerics` from the value-buffer wire format. */ +/** Reads one `Numerics` from a value buffer. */ inline Numerics read_Numerics(BufferReader& r) { - int8_t f_a = r.read_i8(); - int16_t f_b = r.read_i16(); - int32_t f_c = r.read_i32(); - int64_t f_d = r.read_i64(); - uint8_t f_e = r.read_u8(); - uint16_t f_f = r.read_u16(); - uint32_t f_g = r.read_u32(); - uint64_t f_h = r.read_u64(); - float f_i = r.read_f32(); - double f_j = r.read_f64(); - return Numerics{std::move(f_a), std::move(f_b), std::move(f_c), std::move(f_d), std::move(f_e), std::move(f_f), std::move(f_g), std::move(f_h), std::move(f_i), std::move(f_j)}; + return Numerics{ + r.read_i8(), + r.read_i16(), + r.read_i32(), + r.read_i64(), + r.read_u8(), + r.read_u16(), + r.read_u32(), + r.read_u64(), + r.read_f32(), + r.read_f64(), + }; } -/** Encodes a `Shape` in the value-buffer wire format. */ +/** Writes one `Shape` to a value buffer. */ inline void write_Shape(BufferWriter& w, const Shape& v) { switch (v.value.index()) { - case 0: { - w.write_i32(0); - break; - } - case 1: { - w.write_i32(1); - const Shape::Circle& p = std::get<1>(v.value); - w.write_f64(p.radius); - break; - } - case 2: { - w.write_i32(2); - const Shape::Rectangle& p = std::get<2>(v.value); - w.write_f32(p.width); - w.write_f32(p.height); - break; - } - case 3: { - w.write_i32(3); - const Shape::Bounded& p = std::get<3>(v.value); - w.write_i16(p.min); - w.write_i16(p.max); - w.write_u64(p.weight); - break; - } - case 4: { - w.write_i32(4); - const Shape::Labeled& p = std::get<4>(v.value); - w.write_string(p.label); - w.write_u8(p.count); - break; - } + case 0: { + w.write_i32(0); + break; + } + case 1: { + w.write_i32(1); + const Shape::Circle& p = std::get<1>(v.value); + w.write_f64(p.radius); + break; + } + case 2: { + w.write_i32(2); + const Shape::Rectangle& p = std::get<2>(v.value); + w.write_f32(p.width); + w.write_f32(p.height); + break; + } + case 3: { + w.write_i32(3); + const Shape::Bounded& p = std::get<3>(v.value); + w.write_i16(p.min); + w.write_i16(p.max); + w.write_u64(p.weight); + break; + } + case 4: { + w.write_i32(4); + const Shape::Labeled& p = std::get<4>(v.value); + w.write_string(p.label); + w.write_u8(p.count); + break; + } } } -/** Decodes a `Shape` from the value-buffer wire format. */ +/** Reads one `Shape` from a value buffer. */ inline Shape read_Shape(BufferReader& r) { int32_t tag = r.read_i32(); switch (tag) { - case 0: { - return Shape{Shape::Empty{}}; - } - case 1: { - double f_radius = r.read_f64(); - return Shape{Shape::Circle{std::move(f_radius)}}; - } - case 2: { - float f_width = r.read_f32(); - float f_height = r.read_f32(); - return Shape{Shape::Rectangle{std::move(f_width), std::move(f_height)}}; - } - case 3: { - int16_t f_min = r.read_i16(); - int16_t f_max = r.read_i16(); - uint64_t f_weight = r.read_u64(); - return Shape{Shape::Bounded{std::move(f_min), std::move(f_max), std::move(f_weight)}}; - } - case 4: { - std::string f_label = r.read_string(); - uint8_t f_count = r.read_u8(); - return Shape{Shape::Labeled{std::move(f_label), std::move(f_count)}}; - } - default: - break; - } - throw Error(-3, "malformed value buffer: unknown Shape tag"); + case 0: return Shape{Shape::Empty{}}; + case 1: + return Shape{Shape::Circle{ + r.read_f64(), + }}; + case 2: + return Shape{Shape::Rectangle{ + r.read_f32(), + r.read_f32(), + }}; + case 3: + return Shape{Shape::Bounded{ + r.read_i16(), + r.read_i16(), + r.read_u64(), + }}; + case 4: + return Shape{Shape::Labeled{ + r.read_string(), + r.read_u8(), + }}; + default: break; + } + BufferReader::fail("unknown Shape tag"); } } // namespace detail @@ -618,11 +746,7 @@ inline Shape scale(const Shape& shape, double factor) { shapes_error err{}; auto result = shapes_shapes_scale(shape_buf.data(), shape_buf.size(), factor, &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Shape ret = detail::read_Shape(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Shape); } /** Sum a list of bytes into a wide integer (numerics smoke) */ @@ -641,11 +765,7 @@ inline Numerics make_numerics(int8_t a, int16_t b, int32_t c, int64_t d, uint8_t shapes_error err{}; auto result = shapes_shapes_make_numerics(a, b, c, d, e, f, g, h, i, j, &out_len, &err); detail::check(err); - detail::BufferGuard result_guard(result, out_len); - detail::BufferReader result_r(result, out_len); - Numerics ret = detail::read_Numerics(result_r); - result_r.expect_end(); - return ret; + return detail::take(result, out_len, &detail::read_Numerics); } } // namespace shapes diff --git a/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__README_md.snap deleted file mode 100644 index 794b90af..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__README_md.snap +++ /dev/null @@ -1,39 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# docs_everywhere (Dart) - -`dart:ffi` bindings for the `docs_everywhere` native library. - -## Usage - -Add this package as a dependency (a `path:` dependency works) and import it: - -```dart -import 'package:docs_everywhere/docs_everywhere.dart'; -``` - -The library loads `libdocs_everywhere.dylib` (macOS), `libdocs_everywhere.so` (Linux and Android), or -`docs_everywhere.dll` (Windows) from the platform's search path on first use. Set -`DOCS_EVERYWHERE_LIBRARY` to a full path to load a specific build. On iOS, and as a macOS -fallback, the bindings look the symbols up in the running executable. - -## Objects and callbacks - -Each interface wrapper holds one native reference: call `dispose()` when -you're done, or let the garbage collector's finalizer release it. - -Callback-interface methods that return a value run synchronously on the -isolate's thread; void methods may be called from any thread and are -delivered on the event loop. - -## Requirements - -- Dart SDK `>=3.12.0 <4.0.0` -- The `ffi` package - - diff --git a/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__lib_docs_everywhere_dart.snap b/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__lib_docs_everywhere_dart.snap index 759e4eb5..200bd830 100644 --- a/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__lib_docs_everywhere_dart.snap +++ b/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__lib_docs_everywhere_dart.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // ignore_for_file: camel_case_types, non_constant_identifier_names, unused_element @@ -18,45 +18,69 @@ import 'package:ffi/ffi.dart'; // ── Library loading ── // The native library opens on first use and is verified before any other -// lookup: its C ABI revision and every module's contract checksum must match -// the values these bindings were generated against. +// lookup: it must implement the C ABI revision these bindings were generated +// against and carry every declaration they were generated with, unchanged. /// The native library every binding in this file calls. final DynamicLibrary _lib = _verifyLibrary(_openLibrary()); +/// The native library couldn't be loaded, or it doesn't match these +/// bindings: a different C ABI revision, or a declaration that's missing or +/// changed. Regenerate the bindings from the library you ship. +final class NativeLibraryError extends Error { + /// Creates a load failure described by [message]. + NativeLibraryError(this.message); + + /// What failed, naming the library or declaration. + final String message; + + @override + String toString() => 'NativeLibraryError: $message'; +} + +/// The platform file name of the library on the system search path. +String get _libraryFile => Platform.isMacOS + ? 'libdocs_everywhere.dylib' + : Platform.isWindows + ? 'docs_everywhere.dll' + : 'libdocs_everywhere.so'; + DynamicLibrary _openLibrary() { // An explicit path wins, so a caller can point at any build artifact. final override = Platform.environment['DOCS_EVERYWHERE_LIBRARY']; if (override != null && override.isNotEmpty) { - return DynamicLibrary.open(override); + return _open(override); } // iOS links native code into the app binary. if (Platform.isIOS) return DynamicLibrary.process(); - final candidates = Platform.isMacOS - ? const ['libdocs_everywhere.dylib'] - : Platform.isWindows - ? const ['docs_everywhere.dll'] - : const ['libdocs_everywhere.so']; - ArgumentError? failure; - for (final candidate in candidates) { - try { - return DynamicLibrary.open(candidate); - } on ArgumentError catch (e) { - failure ??= e; - } + try { + return DynamicLibrary.open(_libraryFile); + } on ArgumentError catch (e) { + // A macOS app may link the library into its executable instead. + if (Platform.isMacOS) return DynamicLibrary.process(); + throw NativeLibraryError("can't open '$_libraryFile' ($e); set " + 'DOCS_EVERYWHERE_LIBRARY to its path'); } - // A macOS app may link the library into its executable instead. - if (Platform.isMacOS) return DynamicLibrary.process(); - throw failure!; } -/// The C ABI revision these bindings were generated against. -const int _abiVersion = 3; +DynamicLibrary _open(String path) { + try { + return DynamicLibrary.open(path); + } on ArgumentError catch (e) { + throw NativeLibraryError("can't open '$path': $e"); + } +} -/// Each top-level module's name, checksum symbol, and expected checksum. -const List<(String, String, int)> _contracts = <(String, String, int)>[ - ('docs', 'docs_everywhere_docs_checksum', 0x7950eec1e9cfc819), -]; +/// The C ABI revision these bindings were generated against. +const int _abiVersion = 4; + +/// One `docs_everywhere_contract_entry` of a module's contract table. +final class _ContractEntry extends Struct { + @Uint64() + external int id; + @Uint64() + external int hash; +} DynamicLibrary _verifyLibrary(DynamicLibrary lib) { final int Function() abiVersion; @@ -64,33 +88,76 @@ DynamicLibrary _verifyLibrary(DynamicLibrary lib) { abiVersion = lib.lookupFunction( 'docs_everywhere_abi_version'); } on ArgumentError { - throw StateError( - "the native library 'docs_everywhere' isn't loaded; set DOCS_EVERYWHERE_LIBRARY to " - 'its path'); + throw NativeLibraryError( + "the native library 'docs_everywhere' isn't loaded; set DOCS_EVERYWHERE_LIBRARY " + 'to its path'); } final found = abiVersion(); if (found != _abiVersion) { - throw StateError('docs_everywhere: these bindings expect C ABI revision ' - '$_abiVersion, but the loaded library reports revision $found'); + throw NativeLibraryError('docs_everywhere: these bindings expect C ABI revision ' + '$_abiVersion, but the loaded library implements revision $found'); } - for (final (module, symbol, expected) in _contracts) { - final int actual; - try { - actual = lib.lookupFunction(symbol)(); - } on ArgumentError { - throw StateError( - "docs_everywhere: the loaded library doesn't export module '$module'"); + for (final (symbol, expected) in _contracts) { + _checkContract(lib, symbol, expected); + } + return lib; +} + +/// Checks that every `(id, hash, path)` entry in [expected] is in the table +/// [symbol] returns with an equal hash. Entries only the library has are +/// fine: they're declarations added after these bindings were generated. +void _checkContract( + DynamicLibrary lib, String symbol, List<(int, int, String)> expected) { + final Pointer<_ContractEntry> Function(Pointer) table; + try { + table = lib.lookupFunction Function(Pointer), + Pointer<_ContractEntry> Function(Pointer)>(symbol); + } on ArgumentError { + throw NativeLibraryError( + "docs_everywhere: the loaded library doesn't export '$symbol'"); + } + final hashes = {}; + final len = calloc(); + try { + final entries = table(len); + for (var i = 0; i < len.value; i++) { + hashes[entries[i].id] = entries[i].hash; + } + } finally { + calloc.free(len); + } + for (final (id, hash, path) in expected) { + final found = hashes[id]; + if (found == null) { + throw NativeLibraryError('docs_everywhere: $path is missing from the library'); } - if (actual != expected) { - throw StateError("docs_everywhere: module '$module' doesn't match the loaded " - 'library (contract checksum ${_hex(actual)}, expected ' - '${_hex(expected)}); regenerate the bindings'); + if (found != hash) { + throw NativeLibraryError( + 'docs_everywhere: $path changed since these bindings were generated'); } } - return lib; } -String _hex(int v) => '0x${v.toUnsigned(64).toRadixString(16).padLeft(16, '0')}'; +/// Each top-level module's contract-table symbol and the declarations +/// these bindings were generated with: (id, signature hash, path). +const List<(String, List<(int, int, String)>)> _contracts = [ + ('docs_everywhere_docs_contract', [ + (0x00b2187fd75634ec, 0x2263c6f787bf6081, 'docs.publish_async'), + (0x12fc242b9dc6444c, 0x96405f484a642b31, 'docs.Notebook.insert'), + (0x2213822ed05cd5bd, 0x26c5833392aa63ce, 'docs.Notebook.new'), + (0x2bd762bcdaa207ff, 0x1b0ebab2d04c49b0, 'docs.SavedListener'), + (0x2e086c45e9d617de, 0x4d43bf32358c01ff, 'docs.Notebook.find'), + (0x4817dc98197ad960, 0xf8017b5a7d58b2c7, 'docs.Notebook.size'), + (0x614de47452d5a55c, 0xd9aae1472494ffef, 'docs.get_document'), + (0x67c5cb9fc119bcd5, 0x08d5b25a1a22a7fa, 'docs.Document'), + (0x8d484bf2d78c9b70, 0xe9093cf96912194c, 'docs.Notebook.max_documents'), + (0xaea61d6451691c67, 0xe4704128b3524b3d, 'docs.watch'), + (0xd68d6ad1540cdec5, 0x01afeb40b0a8bfe1, 'docs.Notebook'), + (0xdb0601444a4760cc, 0xe56c69629b444797, 'docs.create_document'), + (0xe12d6f4838347b3e, 0xa9bab86646d84f51, 'docs.Status'), + (0xee2f8d25b3313b50, 0x9691232f35439a2b, 'docs.DocsErrors'), + ]), +]; // ── Errors and memory ── @@ -106,10 +173,8 @@ final class _Error extends Struct { external int payloadLen; } -final _errorClear = - _lib.lookupFunction), void Function(Pointer<_Error>)>( - 'docs_everywhere_error_clear', - isLeaf: true); +final _errorClear = _lib.lookupFunction), + void Function(Pointer<_Error>)>('docs_everywhere_error_clear', isLeaf: true); final _freeBytes = _lib.lookupFunction, Size), void Function(Pointer, int)>('docs_everywhere_free_bytes', isLeaf: true); @@ -120,9 +185,9 @@ final _freeBytes = _lib.lookupFunction, Size), final Pointer<_Error> _err = calloc<_Error>(); final Pointer _outLen = calloc(); -/// A failure the native library reported. Domain errors extend this class -/// with a positive [code]; the negative runtime codes are the constants -/// below. +/// A failure the native library reported to a function declared to throw. +/// Domain errors extend this class with a positive [code]; a negative code is +/// one of the runtime constants below. class NativeException implements Exception { /// Creates an exception carrying [code] and [message]. NativeException(this.code, this.message); @@ -133,10 +198,10 @@ class NativeException implements Exception { /// The producer panicked; [message] carries the panic text. static const int panicCode = -2; - /// An argument couldn't be lifted by the producer. + /// An argument or a callback's return couldn't be lifted by the producer. static const int marshalCode = -3; - /// A callback implementation threw; [message] carries its text. + /// A callback implementation failed; [message] carries its text. static const int foreignCode = -4; /// The call was cancelled (see [CancelledException]). @@ -160,24 +225,51 @@ class CancelledException extends NativeException { : super(NativeException.cancelledCode, message); } -typedef _ErrorMapper = NativeException Function( +/// A function that isn't declared to throw failed anyway. That's a bug in +/// the native library (a panic, an argument it couldn't lift, a callback +/// failure it let through), so it's an [Error], not an [Exception]. +final class NativeError extends Error { + /// Creates the error a failed call reported. + NativeError(this.code, this.message); + + /// The runtime code the producer reported (see [NativeException]). + final int code; + + /// The producer's message. + final String message; + + @override + String toString() => 'NativeError($code): $message'; +} + +/// Maps a reported code, message, and payload onto the object to throw. +typedef _ErrorMapper = Object Function( int code, String message, Uint8List payload); -/// Maps a runtime failure (or an undeclared code) onto its exception. -NativeException _runtimeError(int code, String message, Uint8List payload) => +/// The mapper of a function that can't fail: every code but cancellation +/// is a [NativeError]. +Object _trap(int code, String message, Uint8List payload) => + code == NativeException.cancelledCode + ? CancelledException(message) + : NativeError(code, message); + +/// Maps a runtime code (or one the domain doesn't declare) onto its +/// exception, for a function declared to throw. +NativeException _runtimeException(int code, String message, Uint8List payload) => code == NativeException.cancelledCode ? CancelledException(message) : NativeException(code, message); -/// Builds the exception [err] reports, using [map] for domain codes. -NativeException _readError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error [err] reports, using [map]. +Object _readError(Pointer<_Error> err, _ErrorMapper map) { final e = err.ref; final message = e.message == nullptr ? '' : e.message.toDartString(); return map(e.code, message, _copyBytes(e.payloadPtr, e.payloadLen)); } /// Throws the error a synchronous call left in [err], clearing the slot. -void _check(Pointer<_Error> err, [_ErrorMapper map = _runtimeError]) { +/// [map] is the domain's mapper for a function declared to throw. +void _check(Pointer<_Error> err, [_ErrorMapper map = _trap]) { if (err.ref.code == 0) return; final error = _readError(err, map); _errorClear(err); @@ -379,6 +471,27 @@ final class _BufferWriter { _buf.setRange(_len, _len + v.length, v); _len += v.length; } + + void writeOptional(T? v, void Function(T) write) { + writeFlag(v != null); + if (v != null) write(v); + } + + void writeList(List v, void Function(T) write) { + writeLength(v.length); + for (final e in v) { + write(e); + } + } + + void writeMap( + Map v, void Function(K) writeKey, void Function(V) writeValue) { + writeLength(v.length); + for (final e in v.entries) { + writeKey(e.key); + writeValue(e.value); + } + } } final class _BufferReader { @@ -442,6 +555,24 @@ final class _BufferReader { return Uint8List.fromList(Uint8List.sublistView(_buf, at, at + n)); } + T? readOptional(T Function() read) => + readFlag() ? read() : null; + + // Collections grow as they decode, so a corrupt count can't force a huge + // allocation up front. + List readList(T Function() read) => + [for (var i = readLength(); i > 0; i--) read()]; + + Map readMap(K Function() readKey, V Function() readValue) { + final map = {}; + for (var i = readLength(); i > 0; i--) { + final key = readKey(); + if (map.containsKey(key)) _bufferError('duplicate map key $key'); + map[key] = readValue(); + } + return map; + } + void expectEnd() { if (_pos != _buf.length) _bufferError('trailing bytes'); } @@ -462,6 +593,36 @@ T _decode(Uint8List bytes, T Function(_BufferReader) read) { return value; } +// Records and rich enums compare by value: lists, maps, and byte arrays +// element by element, and everything else with `==` (interface wrappers by +// identity). + +bool _deepEquals(Object? a, Object? b) { + if (identical(a, b)) return true; + if (a is List && b is List) { + if (a.length != b.length) return false; + for (var i = 0; i < a.length; i++) { + if (!_deepEquals(a[i], b[i])) return false; + } + return true; + } + if (a is Map && b is Map) { + if (a.length != b.length) return false; + for (final key in a.keys) { + if (!b.containsKey(key) || !_deepEquals(a[key], b[key])) return false; + } + return true; + } + return a == b; +} + +int _deepHash(Object? v) => switch (v) { + List() => Object.hashAll(v.map(_deepHash)), + Map() => Object.hashAllUnordered( + v.entries.map((e) => Object.hash(_deepHash(e.key), _deepHash(e.value)))), + _ => v.hashCode, + }; + // ── Async completion ── // An async call completes through a `NativeCallable.listener` the producer // invokes exactly once, from any thread. Everything the completion carries is @@ -472,8 +633,8 @@ final _errorFree = _lib.lookupFunction), void Function(Pointer<_Error>)>( 'docs_everywhere_error_free'); -/// Builds the exception a completion reports and frees its boxed error. -NativeException _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error a completion reports and frees its boxed error. +Object _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { final error = _readError(err, map); _errorFree(err); return error; @@ -488,11 +649,14 @@ NativeException _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { // // Methods that return a value are `NativeCallable.isolateLocal` trampolines: // they run synchronously, so the producer must call them on this isolate's -// thread while a call from Dart is in progress. Void methods and `free` are -// `NativeCallable.isolateGroupBound` forwarders, which may run on any thread: -// they copy their arguments into a message for this isolate (borrowed bytes -// are copied before the forwarder returns) and the implementation runs later -// on the event loop, in the zone that passed the callback. +// thread while a call from Dart is in progress (the VM aborts the process +// otherwise, and an entry point that may run on any thread can't re-enter +// the isolate to run the method). Void methods and `free` are +// `NativeCallable.isolateGroupBound` forwarders, which may run on any +// thread: they copy their arguments into a message for this isolate +// (borrowed bytes are copied before the forwarder returns) and the +// implementation runs later on the event loop, in the zone that passed the +// callback. /// Where forwarders send messages: the allocator they build a message with /// and the isolate port that receives it. Forwarders run outside the @@ -700,35 +864,86 @@ final _errorSet = _lib.lookupFunction< Void Function(Pointer<_Error>, Int32, Pointer), void Function(Pointer<_Error>, int, Pointer)>('docs_everywhere_error_set'); -/// Reports an implementation's exception to the producer through -/// `docs_everywhere_error_set` with the foreign code (-4). Never throws: it runs -/// in the catch path of a trampoline the exception must not unwind through. -void _foreignError(Pointer<_Error> outErr, Object error) { +final _errorSetPayload = _lib.lookupFunction< + Void Function(Pointer<_Error>, Pointer, Size), + void Function( + Pointer<_Error>, Pointer, int)>('docs_everywhere_error_set_payload'); + +final _alloc = _lib.lookupFunction Function(Size), + Pointer Function(int)>('docs_everywhere_alloc', isLeaf: true); + +/// Reports a failed callback method to the producer through [outErr]: +/// `docs_everywhere_error_set` copies [message], and `docs_everywhere_error_set_payload` +/// copies a domain code's encoded fields. Never throws: it runs in the catch +/// path of a trampoline an exception must not unwind through. +void _failCallback( + Pointer<_Error> outErr, int code, String message, Uint8List? payload) { if (outErr == nullptr) return; - Pointer message = nullptr; + final text = message.toNativeUtf8(allocator: calloc); + try { + _errorSet(outErr, code, text); + } finally { + calloc.free(text); + } + if (payload == null || payload.isEmpty) return; + final bytes = calloc(payload.length); + try { + bytes.asTypedList(payload.length).setAll(0, payload); + _errorSetPayload(outErr, bytes, payload.length); + } finally { + calloc.free(bytes); + } +} + +/// Reports an implementation's exception that isn't a domain error of the +/// method with the foreign code (-4) and the exception's text. +void _foreignError(Pointer<_Error> outErr, Object error) { try { - message = error.toString().toNativeUtf8(allocator: calloc); - _errorSet(outErr, NativeException.foreignCode, message); + _failCallback(outErr, NativeException.foreignCode, '$error', null); } catch (_) { _errorSet(outErr, NativeException.foreignCode, nullptr); - } finally { - if (message != nullptr) calloc.free(message); } } -/// The `DocsErrors` error domain of module `docs`. -class DocsErrorsException extends NativeException { +/// Hands [bytes] to the producer through a method's `out_ptr`/`out_len` +/// slots as a run allocated with `docs_everywhere_alloc`, which the producer +/// adopts and frees. An empty run is null with length 0. +void _handOver( + List bytes, Pointer> outPtr, Pointer outLen) { + final run = bytes.isEmpty ? nullptr : _alloc(bytes.length); + if (run != nullptr) run.asTypedList(bytes.length).setAll(0, bytes); + outPtr.value = run; + outLen.value = bytes.length; +} + +void _packListOfString(_BufferWriter w, List v) => + w.writeList(v, (e) => w.writeString(e)); + +List _unpackListOfString(_BufferReader r) => + r.readList(() => r.readString()); + +void _packOptionalOfDocument(_BufferWriter w, Document? v) => + w.writeOptional(v, (e) => _packDocument(w, e)); + +Document? _unpackOptionalOfDocument(_BufferReader r) => + r.readOptional(() => _unpackDocument(r)); + +/// The `DocsErrors` error domain of module `docs`: catch it to handle any of +/// its codes, or a subclass for one code. +sealed class DocsErrorsException extends NativeException { /// Creates a domain error carrying [code] and [message]. DocsErrorsException(super.code, super.message); } /// Returned when no document matches the requested id -class NotFoundException extends DocsErrorsException { +final class NotFoundException extends DocsErrorsException { + /// Creates a [NotFoundException] (code 1). NotFoundException([String message = 'Document not found']) : super(1, message); } /// Raised when attempting to move into a forbidden state -class InvalidStatusException extends DocsErrorsException { +final class InvalidStatusException extends DocsErrorsException { + /// Creates an [InvalidStatusException] (code 2). InvalidStatusException([String message = 'Invalid status transition']) : super(2, message); } @@ -739,7 +954,7 @@ NativeException _mapDocsErrorsException(int code, String message, Uint8List payl case 2: return InvalidStatusException(message); default: - return _runtimeError(code, message, payload); + return _runtimeException(code, message, payload); } } @@ -768,7 +983,7 @@ enum Status { /// Represents a single document tracked by the system. /// /// Documents are persisted to disk and exposed via the public API. -class Document { +final class Document { /// Creates a [Document]. Document({required this.id, required this.title, required this.status, required this.tags}); @@ -780,25 +995,36 @@ class Document { final Status status; /// Free-form tags for grouping final List tags; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Document && + id == other.id && + title == other.title && + status == other.status && + _deepEquals(tags, other.tags); + + @override + int get hashCode => Object.hashAll([Document, id, title, status, _deepHash(tags)]); + + @override + String toString() => 'Document(id: $id, title: $title, status: $status, tags: $tags)'; } void _packDocument(_BufferWriter w, Document v) { w.writeI64(v.id); w.writeString(v.title); w.writeI32(v.status.value); - final _t0 = v.tags; - w.writeLength(_t0.length); - for (final _t1 in _t0) { - w.writeString(_t1); - } + _packListOfString(w, v.tags); } Document _unpackDocument(_BufferReader r) => Document( - id: r.readI64(), - title: r.readString(), - status: Status.fromValue(r.readI32()), - tags: [for (var i = r.readLength(); i > 0; i--) r.readString()], -); + id: r.readI64(), + title: r.readString(), + status: Status.fromValue(r.readI32()), + tags: _unpackListOfString(r), + ); /// Consumer-implemented observer of document-saved notifications. /// @@ -810,11 +1036,13 @@ Document _unpackDocument(_BufferReader r) => Document( /// /// Methods that return a value run synchronously and must be called by /// the producer on this isolate's thread during a call from Dart; a -/// thrown exception fails that call with [NativeException.foreignCode]. -/// Void methods may be called from any thread: they're delivered -/// asynchronously on this isolate's event loop, in the zone that passed -/// the instance, where a thrown exception is an uncaught error of that -/// zone. Object arguments are owned by the implementation. +/// thrown exception fails that call with [NativeException.foreignCode] +/// (or, from a method that throws, the domain exception's code and +/// fields). Void methods may be called from any thread: they're +/// delivered asynchronously on this isolate's event loop, in the zone +/// that passed the instance, where a thrown exception is an uncaught +/// error of that zone. Object arguments are owned by the +/// implementation. abstract class SavedListener { /// Fires after a document has been persisted successfully void onSaved(int docId, int revision); @@ -824,8 +1052,12 @@ typedef _DocsEverywhereDocsSavedListenerOnSaved = Void Function(Pointer, I // The C `docs_everywhere_docs_SavedListener_vtable` layout. final class _SavedListenerVtable extends Struct { - external Pointer> onSaved; + @Uint32() + external int size; + @Uint32() + external int flags; external Pointer)>> free; + external Pointer> onSaved; } // Vtable entry `docs_everywhere_docs_SavedListener_vtable.on_saved`: forwarded to the isolate. @@ -840,9 +1072,9 @@ void _savedListenerDispatch(Object impl, int method, List message) { final target = impl as SavedListener; switch (method) { case 0: { - final _a2 = message[2]! as int; - final _a3 = message[3]! as int; - target.onSaved(_a2, _a3); + final a2 = message[2]! as int; + final a3 = message[3]! as int; + target.onSaved(a2, a3); } } } @@ -850,8 +1082,10 @@ void _savedListenerDispatch(Object impl, int method, List message) { final Pointer<_SavedListenerVtable> _savedListenerVtable = () { final vtable = calloc<_SavedListenerVtable>(); vtable.ref - ..onSaved = _pin(NativeCallable<_DocsEverywhereDocsSavedListenerOnSaved>.isolateGroupBound(_savedListenerOnSaved)) - ..free = _callbackFree; + ..size = sizeOf<_SavedListenerVtable>() + ..flags = 0 + ..free = _callbackFree + ..onSaved = _pin(NativeCallable<_DocsEverywhereDocsSavedListenerOnSaved>.isolateGroupBound(_savedListenerOnSaved)); return vtable; }(); @@ -919,9 +1153,7 @@ final class Notebook extends _NativeObject { final _self = _enter(); final _arena = Arena(); try { - final _documentWriter = _BufferWriter(); - _packDocument(_documentWriter, document); - final _documentBytes = _documentWriter.takeBytes(); + final _documentBytes = _encode(document, _packDocument); final _result = _docsEverywhereDocsNotebookInsert(_self, _stage(_arena, _documentBytes), _documentBytes.length, _err); _check(_err); return _result; @@ -977,13 +1209,7 @@ Document createDocument(String title, List tags) { final _arena = Arena(); try { final _titleBytes = utf8.encode(title); - final _tagsWriter = _BufferWriter(); - final _t0 = tags; - _tagsWriter.writeLength(_t0.length); - for (final _t1 in _t0) { - _tagsWriter.writeString(_t1); - } - final _tagsBytes = _tagsWriter.takeBytes(); + final _tagsBytes = _encode(tags, _packListOfString); final _result = _docsEverywhereDocsCreateDocument(_stage(_arena, _titleBytes), _titleBytes.length, _stage(_arena, _tagsBytes), _tagsBytes.length, _outLen, _err); _check(_err); return _decode(_takeBytes(_result, _outLen.value), _unpackDocument); @@ -1000,7 +1226,7 @@ final _docsEverywhereDocsGetDocument = _lib.lookupFunction< Document? getDocument(int id) { final _result = _docsEverywhereDocsGetDocument(id, _outLen, _err); _check(_err); - return _decode(_takeBytes(_result, _outLen.value), (r) => (r.readFlag() ? _unpackDocument(r) : null)); + return _decode(_takeBytes(_result, _outLen.value), _unpackOptionalOfDocument); } typedef _DocsEverywhereDocsPublishAsyncCallback = Void Function(Pointer, Pointer<_Error>, Bool); @@ -1014,11 +1240,11 @@ Future publishAsync(int id) { final _completer = Completer(); late final NativeCallable<_DocsEverywhereDocsPublishAsyncCallback> _callback; _callback = NativeCallable<_DocsEverywhereDocsPublishAsyncCallback>.listener(( - Pointer _, Pointer<_Error> _error, bool _value) { + Pointer _, Pointer<_Error> error, bool value) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); - _completer.complete(_value); + if (error != nullptr) throw _takeAsyncError(error, _trap); + _completer.complete(value); } catch (e, s) { _completer.completeError(e, s); } @@ -1039,7 +1265,7 @@ final _docsEverywhereDocsWatch = _lib.lookupFunction< /// Register a listener for saved notifications void watch(SavedListener listener) { final _listenerCtx = _registerCallback(listener, _savedListenerDispatch); - _docsEverywhereDocsWatch(_listenerCtx, _savedListenerVtable.cast(), _err); + _docsEverywhereDocsWatch(_listenerCtx, _savedListenerVtable.cast(), _err); _check(_err); } diff --git a/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__pubspec_yaml.snap b/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__pubspec_yaml.snap deleted file mode 100644 index 973243cc..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_docs_everywhere__pubspec_yaml.snap +++ /dev/null @@ -1,18 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -name: docs_everywhere -version: 0.1.0 -description: 'Native bindings for docs_everywhere' - -environment: - sdk: '>=3.12.0 <4.0.0' - -dependencies: - ffi: ^2.1.0 - -# END pubspec.yaml diff --git a/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__README_md.snap deleted file mode 100644 index 20d5449a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__README_md.snap +++ /dev/null @@ -1,39 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# edge_cases (Dart) - -`dart:ffi` bindings for the `edge_cases` native library. - -## Usage - -Add this package as a dependency (a `path:` dependency works) and import it: - -```dart -import 'package:edge_cases/edge_cases.dart'; -``` - -The library loads `libedge_cases.dylib` (macOS), `libedge_cases.so` (Linux and Android), or -`edge_cases.dll` (Windows) from the platform's search path on first use. Set -`EDGE_CASES_LIBRARY` to a full path to load a specific build. On iOS, and as a macOS -fallback, the bindings look the symbols up in the running executable. - -## Objects and callbacks - -Each interface wrapper holds one native reference: call `dispose()` when -you're done, or let the garbage collector's finalizer release it. - -Callback-interface methods that return a value run synchronously on the -isolate's thread; void methods may be called from any thread and are -delivered on the event loop. - -## Requirements - -- Dart SDK `>=3.12.0 <4.0.0` -- The `ffi` package - - diff --git a/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__lib_edge_cases_dart.snap b/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__lib_edge_cases_dart.snap index 0ba27b38..c1bfe078 100644 --- a/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__lib_edge_cases_dart.snap +++ b/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__lib_edge_cases_dart.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // ignore_for_file: camel_case_types, non_constant_identifier_names, unused_element @@ -18,45 +18,69 @@ import 'package:ffi/ffi.dart'; // ── Library loading ── // The native library opens on first use and is verified before any other -// lookup: its C ABI revision and every module's contract checksum must match -// the values these bindings were generated against. +// lookup: it must implement the C ABI revision these bindings were generated +// against and carry every declaration they were generated with, unchanged. /// The native library every binding in this file calls. final DynamicLibrary _lib = _verifyLibrary(_openLibrary()); +/// The native library couldn't be loaded, or it doesn't match these +/// bindings: a different C ABI revision, or a declaration that's missing or +/// changed. Regenerate the bindings from the library you ship. +final class NativeLibraryError extends Error { + /// Creates a load failure described by [message]. + NativeLibraryError(this.message); + + /// What failed, naming the library or declaration. + final String message; + + @override + String toString() => 'NativeLibraryError: $message'; +} + +/// The platform file name of the library on the system search path. +String get _libraryFile => Platform.isMacOS + ? 'libedge_cases.dylib' + : Platform.isWindows + ? 'edge_cases.dll' + : 'libedge_cases.so'; + DynamicLibrary _openLibrary() { // An explicit path wins, so a caller can point at any build artifact. final override = Platform.environment['EDGE_CASES_LIBRARY']; if (override != null && override.isNotEmpty) { - return DynamicLibrary.open(override); + return _open(override); } // iOS links native code into the app binary. if (Platform.isIOS) return DynamicLibrary.process(); - final candidates = Platform.isMacOS - ? const ['libedge_cases.dylib'] - : Platform.isWindows - ? const ['edge_cases.dll'] - : const ['libedge_cases.so']; - ArgumentError? failure; - for (final candidate in candidates) { - try { - return DynamicLibrary.open(candidate); - } on ArgumentError catch (e) { - failure ??= e; - } + try { + return DynamicLibrary.open(_libraryFile); + } on ArgumentError catch (e) { + // A macOS app may link the library into its executable instead. + if (Platform.isMacOS) return DynamicLibrary.process(); + throw NativeLibraryError("can't open '$_libraryFile' ($e); set " + 'EDGE_CASES_LIBRARY to its path'); + } +} + +DynamicLibrary _open(String path) { + try { + return DynamicLibrary.open(path); + } on ArgumentError catch (e) { + throw NativeLibraryError("can't open '$path': $e"); } - // A macOS app may link the library into its executable instead. - if (Platform.isMacOS) return DynamicLibrary.process(); - throw failure!; } /// The C ABI revision these bindings were generated against. -const int _abiVersion = 3; +const int _abiVersion = 4; -/// Each top-level module's name, checksum symbol, and expected checksum. -const List<(String, String, int)> _contracts = <(String, String, int)>[ - ('edge_cases', 'edge_cases_edge_cases_checksum', 0x1567f441073bd4aa), -]; +/// One `edge_cases_contract_entry` of a module's contract table. +final class _ContractEntry extends Struct { + @Uint64() + external int id; + @Uint64() + external int hash; +} DynamicLibrary _verifyLibrary(DynamicLibrary lib) { final int Function() abiVersion; @@ -64,33 +88,109 @@ DynamicLibrary _verifyLibrary(DynamicLibrary lib) { abiVersion = lib.lookupFunction( 'edge_cases_abi_version'); } on ArgumentError { - throw StateError( - "the native library 'edge_cases' isn't loaded; set EDGE_CASES_LIBRARY to " - 'its path'); + throw NativeLibraryError( + "the native library 'edge_cases' isn't loaded; set EDGE_CASES_LIBRARY " + 'to its path'); } final found = abiVersion(); if (found != _abiVersion) { - throw StateError('edge_cases: these bindings expect C ABI revision ' - '$_abiVersion, but the loaded library reports revision $found'); + throw NativeLibraryError('edge_cases: these bindings expect C ABI revision ' + '$_abiVersion, but the loaded library implements revision $found'); } - for (final (module, symbol, expected) in _contracts) { - final int actual; - try { - actual = lib.lookupFunction(symbol)(); - } on ArgumentError { - throw StateError( - "edge_cases: the loaded library doesn't export module '$module'"); + for (final (symbol, expected) in _contracts) { + _checkContract(lib, symbol, expected); + } + return lib; +} + +/// Checks that every `(id, hash, path)` entry in [expected] is in the table +/// [symbol] returns with an equal hash. Entries only the library has are +/// fine: they're declarations added after these bindings were generated. +void _checkContract( + DynamicLibrary lib, String symbol, List<(int, int, String)> expected) { + final Pointer<_ContractEntry> Function(Pointer) table; + try { + table = lib.lookupFunction Function(Pointer), + Pointer<_ContractEntry> Function(Pointer)>(symbol); + } on ArgumentError { + throw NativeLibraryError( + "edge_cases: the loaded library doesn't export '$symbol'"); + } + final hashes = {}; + final len = calloc(); + try { + final entries = table(len); + for (var i = 0; i < len.value; i++) { + hashes[entries[i].id] = entries[i].hash; } - if (actual != expected) { - throw StateError("edge_cases: module '$module' doesn't match the loaded " - 'library (contract checksum ${_hex(actual)}, expected ' - '${_hex(expected)}); regenerate the bindings'); + } finally { + calloc.free(len); + } + for (final (id, hash, path) in expected) { + final found = hashes[id]; + if (found == null) { + throw NativeLibraryError('edge_cases: $path is missing from the library'); + } + if (found != hash) { + throw NativeLibraryError( + 'edge_cases: $path changed since these bindings were generated'); } } - return lib; } -String _hex(int v) => '0x${v.toUnsigned(64).toRadixString(16).padLeft(16, '0')}'; +/// Each top-level module's contract-table symbol and the declarations +/// these bindings were generated with: (id, signature hash, path). +const List<(String, List<(int, int, String)>)> _contracts = [ + ('edge_cases_edge_cases_contract', [ + (0x056a86511785ba2b, 0x5cba3f16d57c6cdb, 'edge_cases.make_widget'), + (0x07bfa9ecb2b720d2, 0x3813b3f57c64bf35, 'edge_cases.EdgeErrors'), + (0x0aaa11867e37f991, 0xf090dd52146ebdf0, 'edge_cases.Widget.refresh'), + (0x0f9701493898ae4c, 0x6944f47a275c96e8, 'edge_cases.Inner'), + (0x1226b1adac179f7d, 0x0e95a00ab0bef637, 'edge_cases.async_keywords'), + (0x16d36bef1541136c, 0x5ea7021ec324298f, 'edge_cases.find_widget'), + (0x1b7324c55e980feb, 0xabf87fe372358f42, 'edge_cases.Widget.old_name'), + (0x1cbd731467c2bab8, 0x4915b9989c2c9074, 'edge_cases.Vault'), + (0x2c2a6de67d894b0a, 0x50a4496b1e7191ca, 'edge_cases.Vault.open'), + (0x322b4cfcd2d6861c, 0xca7527a379ae8465, 'edge_cases.Keywords'), + (0x3f8e23a5f32906f4, 0x8d4c908d796f130d, 'edge_cases.Leaf'), + (0x42ab8a685457b97c, 0xe5ce30500cadb32d, 'edge_cases.Vault.handle'), + (0x4a68540e2836d3d0, 0xa20e44c71e420ed2, 'edge_cases.mode_stream'), + (0x4e030580ac0b111d, 0xb68b7a18d910e175, 'edge_cases.legacy_value'), + (0x4f5e4e5291864ef3, 0x55a1a88b21a08f8b, 'edge_cases.Legacy'), + (0x4f84b06171b4213d, 0xbaedcfdebb6a2820, 'edge_cases.scalar_stream'), + (0x5067a5ff2d9ef910, 0xbfc79b18fc4714fc, 'edge_cases.Widget'), + (0x59aff159112fbf5b, 0x18e4def57a8cfa49, 'edge_cases.optional_params'), + (0x5daf65b98fa77c48, 0xc817a8cb202e59c2, 'edge_cases.keywords_roundtrip'), + (0x5de8a5c9726c2c5a, 0x7c50b5b65f7a896d, 'edge_cases.register_batch'), + (0x618a620309994c3c, 0x826ad0b1d208d723, 'edge_cases.async_bytes'), + (0x648285cdba0a7b8e, 0xcf6e80e783a4d8be, 'edge_cases.WidgetHolder'), + (0x649c78079824b10d, 0x5b12dd26e0634b69, 'edge_cases.async_widget'), + (0x7da7a7357d122a37, 0xf544e025ffaa3d20, 'edge_cases.async_void'), + (0x7e4015bc3a472e2d, 0xfe22291d90b82e4a, 'edge_cases.widget_stream'), + (0x805276e5d81fdba7, 0xd9b64e360bd03b2b, 'edge_cases.Outer'), + (0x80e1183326542df4, 0xe8a57e12cc6ed276, 'edge_cases.select'), + (0x923ce5e9bcd769a1, 0x0b973f5dc07dda8b, 'edge_cases.Vault.to_string'), + (0x98ca8c01f5239404, 0xfe11c93eb6b303ef, 'edge_cases.Widget.rename'), + (0xa13e22e98ea278d4, 0xf1fae215d7387800, 'edge_cases.widgets_everywhere'), + (0xa84a8c9bf2a881e1, 0xbe1dc840bf221996, 'edge_cases.async_optional'), + (0xad958c478fbf205d, 0xe386b0716d24f19b, 'edge_cases.Retired'), + (0xb2fc3964ef56ab39, 0x73c325c8bc351236, 'edge_cases.async_list'), + (0xb3403af521cf0006, 0x6abc30d48206a3ba, 'edge_cases.Widget.close'), + (0xb794304a22a406b8, 0xf6fb98711c516105, 'edge_cases.async_struct'), + (0xba3936106a49c47d, 0xb7025f2fadd6288e, 'edge_cases.Vault.peek'), + (0xc155b0c3cfb164f4, 0xc777ae57d92ebc47, 'edge_cases.noop'), + (0xcdc18607777019dd, 0xb81a31e5c20aa7a8, 'edge_cases.inspect_widget'), + (0xd164a82aeb64b8cc, 0xded02c68005c4737, 'edge_cases.nested_optional'), + (0xd978ccad2468038f, 0x30c39c65d6235153, 'edge_cases.Mode'), + (0xdbe9d947431c9124, 0x39537fb848fdf980, 'edge_cases.BatchListener'), + (0xeae8b7862c08bb07, 0xfe12eeaa830452ea, 'edge_cases.Keyword'), + (0xf1e155a05ae262b8, 0xb575eb818a423d42, 'edge_cases.async_enum'), + (0xf22e210bdbce7343, 0x2a5e0f801bc8c768, 'edge_cases.Widget.dispose'), + (0xf3c7cf675563ee6c, 0x0ecffeef399facc8, 'edge_cases.Vault.probe'), + (0xfd34bc67acac379e, 0x576caa14f3c0901a, 'edge_cases.Widget.fetch'), + (0xff2e9d2bff941786, 0x7e96764ef1f5fbaa, 'edge_cases.string_stream'), + ]), +]; // ── Errors and memory ── @@ -106,10 +206,8 @@ final class _Error extends Struct { external int payloadLen; } -final _errorClear = - _lib.lookupFunction), void Function(Pointer<_Error>)>( - 'edge_cases_error_clear', - isLeaf: true); +final _errorClear = _lib.lookupFunction), + void Function(Pointer<_Error>)>('edge_cases_error_clear', isLeaf: true); final _freeBytes = _lib.lookupFunction, Size), void Function(Pointer, int)>('edge_cases_free_bytes', isLeaf: true); @@ -120,9 +218,9 @@ final _freeBytes = _lib.lookupFunction, Size), final Pointer<_Error> _err = calloc<_Error>(); final Pointer _outLen = calloc(); -/// A failure the native library reported. Domain errors extend this class -/// with a positive [code]; the negative runtime codes are the constants -/// below. +/// A failure the native library reported to a function declared to throw. +/// Domain errors extend this class with a positive [code]; a negative code is +/// one of the runtime constants below. class NativeException implements Exception { /// Creates an exception carrying [code] and [message]. NativeException(this.code, this.message); @@ -133,10 +231,10 @@ class NativeException implements Exception { /// The producer panicked; [message] carries the panic text. static const int panicCode = -2; - /// An argument couldn't be lifted by the producer. + /// An argument or a callback's return couldn't be lifted by the producer. static const int marshalCode = -3; - /// A callback implementation threw; [message] carries its text. + /// A callback implementation failed; [message] carries its text. static const int foreignCode = -4; /// The call was cancelled (see [CancelledException]). @@ -160,24 +258,51 @@ class CancelledException extends NativeException { : super(NativeException.cancelledCode, message); } -typedef _ErrorMapper = NativeException Function( +/// A function that isn't declared to throw failed anyway. That's a bug in +/// the native library (a panic, an argument it couldn't lift, a callback +/// failure it let through), so it's an [Error], not an [Exception]. +final class NativeError extends Error { + /// Creates the error a failed call reported. + NativeError(this.code, this.message); + + /// The runtime code the producer reported (see [NativeException]). + final int code; + + /// The producer's message. + final String message; + + @override + String toString() => 'NativeError($code): $message'; +} + +/// Maps a reported code, message, and payload onto the object to throw. +typedef _ErrorMapper = Object Function( int code, String message, Uint8List payload); -/// Maps a runtime failure (or an undeclared code) onto its exception. -NativeException _runtimeError(int code, String message, Uint8List payload) => +/// The mapper of a function that can't fail: every code but cancellation +/// is a [NativeError]. +Object _trap(int code, String message, Uint8List payload) => + code == NativeException.cancelledCode + ? CancelledException(message) + : NativeError(code, message); + +/// Maps a runtime code (or one the domain doesn't declare) onto its +/// exception, for a function declared to throw. +NativeException _runtimeException(int code, String message, Uint8List payload) => code == NativeException.cancelledCode ? CancelledException(message) : NativeException(code, message); -/// Builds the exception [err] reports, using [map] for domain codes. -NativeException _readError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error [err] reports, using [map]. +Object _readError(Pointer<_Error> err, _ErrorMapper map) { final e = err.ref; final message = e.message == nullptr ? '' : e.message.toDartString(); return map(e.code, message, _copyBytes(e.payloadPtr, e.payloadLen)); } /// Throws the error a synchronous call left in [err], clearing the slot. -void _check(Pointer<_Error> err, [_ErrorMapper map = _runtimeError]) { +/// [map] is the domain's mapper for a function declared to throw. +void _check(Pointer<_Error> err, [_ErrorMapper map = _trap]) { if (err.ref.code == 0) return; final error = _readError(err, map); _errorClear(err); @@ -379,6 +504,27 @@ final class _BufferWriter { _buf.setRange(_len, _len + v.length, v); _len += v.length; } + + void writeOptional(T? v, void Function(T) write) { + writeFlag(v != null); + if (v != null) write(v); + } + + void writeList(List v, void Function(T) write) { + writeLength(v.length); + for (final e in v) { + write(e); + } + } + + void writeMap( + Map v, void Function(K) writeKey, void Function(V) writeValue) { + writeLength(v.length); + for (final e in v.entries) { + writeKey(e.key); + writeValue(e.value); + } + } } final class _BufferReader { @@ -442,6 +588,24 @@ final class _BufferReader { return Uint8List.fromList(Uint8List.sublistView(_buf, at, at + n)); } + T? readOptional(T Function() read) => + readFlag() ? read() : null; + + // Collections grow as they decode, so a corrupt count can't force a huge + // allocation up front. + List readList(T Function() read) => + [for (var i = readLength(); i > 0; i--) read()]; + + Map readMap(K Function() readKey, V Function() readValue) { + final map = {}; + for (var i = readLength(); i > 0; i--) { + final key = readKey(); + if (map.containsKey(key)) _bufferError('duplicate map key $key'); + map[key] = readValue(); + } + return map; + } + void expectEnd() { if (_pos != _buf.length) _bufferError('trailing bytes'); } @@ -462,6 +626,36 @@ T _decode(Uint8List bytes, T Function(_BufferReader) read) { return value; } +// Records and rich enums compare by value: lists, maps, and byte arrays +// element by element, and everything else with `==` (interface wrappers by +// identity). + +bool _deepEquals(Object? a, Object? b) { + if (identical(a, b)) return true; + if (a is List && b is List) { + if (a.length != b.length) return false; + for (var i = 0; i < a.length; i++) { + if (!_deepEquals(a[i], b[i])) return false; + } + return true; + } + if (a is Map && b is Map) { + if (a.length != b.length) return false; + for (final key in a.keys) { + if (!b.containsKey(key) || !_deepEquals(a[key], b[key])) return false; + } + return true; + } + return a == b; +} + +int _deepHash(Object? v) => switch (v) { + List() => Object.hashAll(v.map(_deepHash)), + Map() => Object.hashAllUnordered( + v.entries.map((e) => Object.hash(_deepHash(e.key), _deepHash(e.value)))), + _ => v.hashCode, + }; + // ── Async completion ── // An async call completes through a `NativeCallable.listener` the producer // invokes exactly once, from any thread. Everything the completion carries is @@ -472,8 +666,8 @@ final _errorFree = _lib.lookupFunction), void Function(Pointer<_Error>)>( 'edge_cases_error_free'); -/// Builds the exception a completion reports and frees its boxed error. -NativeException _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error a completion reports and frees its boxed error. +Object _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { final error = _readError(err, map); _errorFree(err); return error; @@ -552,11 +746,14 @@ final class _NativeCancel { // // Methods that return a value are `NativeCallable.isolateLocal` trampolines: // they run synchronously, so the producer must call them on this isolate's -// thread while a call from Dart is in progress. Void methods and `free` are -// `NativeCallable.isolateGroupBound` forwarders, which may run on any thread: -// they copy their arguments into a message for this isolate (borrowed bytes -// are copied before the forwarder returns) and the implementation runs later -// on the event loop, in the zone that passed the callback. +// thread while a call from Dart is in progress (the VM aborts the process +// otherwise, and an entry point that may run on any thread can't re-enter +// the isolate to run the method). Void methods and `free` are +// `NativeCallable.isolateGroupBound` forwarders, which may run on any +// thread: they copy their arguments into a message for this isolate +// (borrowed bytes are copied before the forwarder returns) and the +// implementation runs later on the event loop, in the zone that passed the +// callback. /// Where forwarders send messages: the allocator they build a message with /// and the isolate port that receives it. Forwarders run outside the @@ -764,22 +961,58 @@ final _errorSet = _lib.lookupFunction< Void Function(Pointer<_Error>, Int32, Pointer), void Function(Pointer<_Error>, int, Pointer)>('edge_cases_error_set'); -/// Reports an implementation's exception to the producer through -/// `edge_cases_error_set` with the foreign code (-4). Never throws: it runs -/// in the catch path of a trampoline the exception must not unwind through. -void _foreignError(Pointer<_Error> outErr, Object error) { +final _errorSetPayload = _lib.lookupFunction< + Void Function(Pointer<_Error>, Pointer, Size), + void Function( + Pointer<_Error>, Pointer, int)>('edge_cases_error_set_payload'); + +final _alloc = _lib.lookupFunction Function(Size), + Pointer Function(int)>('edge_cases_alloc', isLeaf: true); + +/// Reports a failed callback method to the producer through [outErr]: +/// `edge_cases_error_set` copies [message], and `edge_cases_error_set_payload` +/// copies a domain code's encoded fields. Never throws: it runs in the catch +/// path of a trampoline an exception must not unwind through. +void _failCallback( + Pointer<_Error> outErr, int code, String message, Uint8List? payload) { if (outErr == nullptr) return; - Pointer message = nullptr; + final text = message.toNativeUtf8(allocator: calloc); + try { + _errorSet(outErr, code, text); + } finally { + calloc.free(text); + } + if (payload == null || payload.isEmpty) return; + final bytes = calloc(payload.length); + try { + bytes.asTypedList(payload.length).setAll(0, payload); + _errorSetPayload(outErr, bytes, payload.length); + } finally { + calloc.free(bytes); + } +} + +/// Reports an implementation's exception that isn't a domain error of the +/// method with the foreign code (-4) and the exception's text. +void _foreignError(Pointer<_Error> outErr, Object error) { try { - message = error.toString().toNativeUtf8(allocator: calloc); - _errorSet(outErr, NativeException.foreignCode, message); + _failCallback(outErr, NativeException.foreignCode, '$error', null); } catch (_) { _errorSet(outErr, NativeException.foreignCode, nullptr); - } finally { - if (message != nullptr) calloc.free(message); } } +/// Hands [bytes] to the producer through a method's `out_ptr`/`out_len` +/// slots as a run allocated with `edge_cases_alloc`, which the producer +/// adopts and frees. An empty run is null with length 0. +void _handOver( + List bytes, Pointer> outPtr, Pointer outLen) { + final run = bytes.isEmpty ? nullptr : _alloc(bytes.length); + if (run != nullptr) run.asTypedList(bytes.length).setAll(0, bytes); + outPtr.value = run; + outLen.value = bytes.length; +} + // ── Iterators ── /// Anchors one live native iterator for its GC-finalizer backstop. A @@ -792,14 +1025,154 @@ final class _IteratorAnchor implements Finalizable {} /// The `out_item` slot iterator steps write, wide enough for any element. final Pointer _outItem = calloc(); -/// The `EdgeErrors` error domain of module `edge_cases`. -class EdgeErrorsException extends NativeException { +void _packListOfI32(_BufferWriter w, List v) => + w.writeList(v, (e) => w.writeI32(e)); + +List _unpackListOfI32(_BufferReader r) => + r.readList(() => r.readI32()); + +void _packListOfLeaf(_BufferWriter w, List v) => + w.writeList(v, (e) => _packLeaf(w, e)); + +List _unpackListOfLeaf(_BufferReader r) => + r.readList(() => _unpackLeaf(r)); + +void _packListOfListOfI32(_BufferWriter w, List> v) => + w.writeList(v, (e) => _packListOfI32(w, e)); + +List> _unpackListOfListOfI32(_BufferReader r) => + r.readList(() => _unpackListOfI32(r)); + +void _packListOfMode(_BufferWriter w, List v) => + w.writeList(v, (e) => w.writeI32(e.value)); + +List _unpackListOfMode(_BufferReader r) => + r.readList(() => Mode.fromValue(r.readI32())); + +void _packListOfOptionalOfI64(_BufferWriter w, List v) => + w.writeList(v, (e) => _packOptionalOfI64(w, e)); + +List _unpackListOfOptionalOfI64(_BufferReader r) => + r.readList(() => _unpackOptionalOfI64(r)); + +void _packListOfOptionalOfOuter(_BufferWriter w, List v) => + w.writeList(v, (e) => _packOptionalOfOuter(w, e)); + +List _unpackListOfOptionalOfOuter(_BufferReader r) => + r.readList(() => _unpackOptionalOfOuter(r)); + +void _packListOfOptionalOfWidget(_BufferWriter w, List v) => + w.writeList(v, (e) => _packOptionalOfWidget(w, e)); + +List _unpackListOfOptionalOfWidget(_BufferReader r) => + r.readList(() => _unpackOptionalOfWidget(r)); + +void _packListOfOuter(_BufferWriter w, List v) => + w.writeList(v, (e) => _packOuter(w, e)); + +List _unpackListOfOuter(_BufferReader r) => + r.readList(() => _unpackOuter(r)); + +void _packListOfString(_BufferWriter w, List v) => + w.writeList(v, (e) => w.writeString(e)); + +List _unpackListOfString(_BufferReader r) => + r.readList(() => r.readString()); + +void _packListOfWidget(_BufferWriter w, List v) => + w.writeList(v, (e) => w.writeU64(e._cloneRef().address)); + +List _unpackListOfWidget(_BufferReader r) => + r.readList(() => Widget._(Pointer.fromAddress(r.readU64()))); + +void _packMapOfI64ToString(_BufferWriter w, Map v) => + w.writeMap(v, (k) => w.writeI64(k), (e) => w.writeString(e)); + +Map _unpackMapOfI64ToString(_BufferReader r) => + r.readMap(() => r.readI64(), () => r.readString()); + +void _packMapOfStringToInner(_BufferWriter w, Map v) => + w.writeMap(v, (k) => w.writeString(k), (e) => _packInner(w, e)); + +Map _unpackMapOfStringToInner(_BufferReader r) => + r.readMap(() => r.readString(), () => _unpackInner(r)); + +void _packMapOfStringToWidget(_BufferWriter w, Map v) => + w.writeMap(v, (k) => w.writeString(k), (e) => w.writeU64(e._cloneRef().address)); + +Map _unpackMapOfStringToWidget(_BufferReader r) => + r.readMap(() => r.readString(), () => Widget._(Pointer.fromAddress(r.readU64()))); + +void _packOptionalOfBytes(_BufferWriter w, List? v) => + w.writeOptional(v, (e) => w.writeBytes(e)); + +List? _unpackOptionalOfBytes(_BufferReader r) => + r.readOptional(() => r.readBytes()); + +void _packOptionalOfI32(_BufferWriter w, int? v) => + w.writeOptional(v, (e) => w.writeI32(e)); + +int? _unpackOptionalOfI32(_BufferReader r) => + r.readOptional(() => r.readI32()); + +void _packOptionalOfI64(_BufferWriter w, int? v) => + w.writeOptional(v, (e) => w.writeI64(e)); + +int? _unpackOptionalOfI64(_BufferReader r) => + r.readOptional(() => r.readI64()); + +void _packOptionalOfInner(_BufferWriter w, Inner? v) => + w.writeOptional(v, (e) => _packInner(w, e)); + +Inner? _unpackOptionalOfInner(_BufferReader r) => + r.readOptional(() => _unpackInner(r)); + +void _packOptionalOfListOfOptionalOfOuter(_BufferWriter w, List? v) => + w.writeOptional(v, (e) => _packListOfOptionalOfOuter(w, e)); + +List? _unpackOptionalOfListOfOptionalOfOuter(_BufferReader r) => + r.readOptional(() => _unpackListOfOptionalOfOuter(r)); + +void _packOptionalOfListOfString(_BufferWriter w, List? v) => + w.writeOptional(v, (e) => _packListOfString(w, e)); + +List? _unpackOptionalOfListOfString(_BufferReader r) => + r.readOptional(() => _unpackListOfString(r)); + +void _packOptionalOfMode(_BufferWriter w, Mode? v) => + w.writeOptional(v, (e) => w.writeI32(e.value)); + +Mode? _unpackOptionalOfMode(_BufferReader r) => + r.readOptional(() => Mode.fromValue(r.readI32())); + +void _packOptionalOfOuter(_BufferWriter w, Outer? v) => + w.writeOptional(v, (e) => _packOuter(w, e)); + +Outer? _unpackOptionalOfOuter(_BufferReader r) => + r.readOptional(() => _unpackOuter(r)); + +void _packOptionalOfString(_BufferWriter w, String? v) => + w.writeOptional(v, (e) => w.writeString(e)); + +String? _unpackOptionalOfString(_BufferReader r) => + r.readOptional(() => r.readString()); + +void _packOptionalOfWidget(_BufferWriter w, Widget? v) => + w.writeOptional(v, (e) => w.writeU64(e._cloneRef().address)); + +Widget? _unpackOptionalOfWidget(_BufferReader r) => + r.readOptional(() => Widget._(Pointer.fromAddress(r.readU64()))); + +/// The `EdgeErrors` error domain of module `edge_cases`: catch it to handle any of +/// its codes, or a subclass for one code. +sealed class EdgeErrorsException extends NativeException { /// Creates a domain error carrying [code] and [message]. EdgeErrorsException(super.code, super.message); } /// Resource is busy -class BusyException extends EdgeErrorsException { +final class BusyException extends EdgeErrorsException { + /// Creates a [BusyException] (code 7). BusyException([String message = 'Resource is busy']) : super(7, message); } @@ -808,7 +1181,7 @@ NativeException _mapEdgeErrorsException(int code, String message, Uint8List payl case 7: return BusyException(message); default: - return _runtimeError(code, message, payload); + return _runtimeException(code, message, payload); } } @@ -845,11 +1218,23 @@ enum Mode { orElse: () => throw ArgumentError.value(value, 'value', 'not a Mode discriminant')); } -class Leaf { +final class Leaf { /// Creates a [Leaf]. Leaf({required this.weight}); final double weight; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Leaf && + weight == other.weight; + + @override + int get hashCode => Object.hashAll([Leaf, weight]); + + @override + String toString() => 'Leaf(weight: $weight)'; } void _packLeaf(_BufferWriter w, Leaf v) { @@ -857,33 +1242,42 @@ void _packLeaf(_BufferWriter w, Leaf v) { } Leaf _unpackLeaf(_BufferReader r) => Leaf( - weight: r.readF32(), -); + weight: r.readF32(), + ); /// Middle layer of a three-deep struct nesting -class Inner { +final class Inner { /// Creates an [Inner]. Inner({required this.leaf, required this.leaves}); final Leaf leaf; final List leaves; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Inner && + leaf == other.leaf && + _deepEquals(leaves, other.leaves); + + @override + int get hashCode => Object.hashAll([Inner, leaf, _deepHash(leaves)]); + + @override + String toString() => 'Inner(leaf: $leaf, leaves: $leaves)'; } void _packInner(_BufferWriter w, Inner v) { _packLeaf(w, v.leaf); - final _t0 = v.leaves; - w.writeLength(_t0.length); - for (final _t1 in _t0) { - _packLeaf(w, _t1); - } + _packListOfLeaf(w, v.leaves); } Inner _unpackInner(_BufferReader r) => Inner( - leaf: _unpackLeaf(r), - leaves: [for (var i = r.readLength(); i > 0; i--) _unpackLeaf(r)], -); + leaf: _unpackLeaf(r), + leaves: _unpackListOfLeaf(r), + ); -class Outer { +final class Outer { /// Creates an [Outer]. Outer({required this.inner, this.maybeInner, required this.byName, required this.modes, required this.matrix, required this.sparse, this.blob, this.note, required this.byId, required this.hidden, required this.v2}); @@ -898,80 +1292,60 @@ class Outer { final Map byId; final bool hidden; final int v2; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Outer && + inner == other.inner && + maybeInner == other.maybeInner && + _deepEquals(byName, other.byName) && + _deepEquals(modes, other.modes) && + _deepEquals(matrix, other.matrix) && + _deepEquals(sparse, other.sparse) && + _deepEquals(blob, other.blob) && + note == other.note && + _deepEquals(byId, other.byId) && + hidden == other.hidden && + v2 == other.v2; + + @override + int get hashCode => Object.hashAll([Outer, inner, maybeInner, _deepHash(byName), _deepHash(modes), _deepHash(matrix), _deepHash(sparse), _deepHash(blob), note, _deepHash(byId), hidden, v2]); + + @override + String toString() => 'Outer(inner: $inner, maybeInner: $maybeInner, byName: $byName, modes: $modes, matrix: $matrix, sparse: $sparse, blob: $blob, note: $note, byId: $byId, hidden: $hidden, v2: $v2)'; } void _packOuter(_BufferWriter w, Outer v) { _packInner(w, v.inner); - final _t0 = v.maybeInner; - w.writeFlag(_t0 != null); - if (_t0 != null) { - _packInner(w, _t0); - } - final _t1 = v.byName; - w.writeLength(_t1.length); - for (final _t2 in _t1.entries) { - w.writeString(_t2.key); - _packInner(w, _t2.value); - } - final _t3 = v.modes; - w.writeLength(_t3.length); - for (final _t4 in _t3) { - w.writeI32(_t4.value); - } - final _t5 = v.matrix; - w.writeLength(_t5.length); - for (final _t6 in _t5) { - final _t7 = _t6; - w.writeLength(_t7.length); - for (final _t8 in _t7) { - w.writeI32(_t8); - } - } - final _t9 = v.sparse; - w.writeLength(_t9.length); - for (final _t10 in _t9) { - final _t11 = _t10; - w.writeFlag(_t11 != null); - if (_t11 != null) { - w.writeI64(_t11); - } - } - final _t12 = v.blob; - w.writeFlag(_t12 != null); - if (_t12 != null) { - w.writeBytes(_t12); - } - final _t13 = v.note; - w.writeFlag(_t13 != null); - if (_t13 != null) { - w.writeString(_t13); - } - final _t14 = v.byId; - w.writeLength(_t14.length); - for (final _t15 in _t14.entries) { - w.writeI64(_t15.key); - w.writeString(_t15.value); - } + _packOptionalOfInner(w, v.maybeInner); + _packMapOfStringToInner(w, v.byName); + _packListOfMode(w, v.modes); + _packListOfListOfI32(w, v.matrix); + _packListOfOptionalOfI64(w, v.sparse); + _packOptionalOfBytes(w, v.blob); + _packOptionalOfString(w, v.note); + _packMapOfI64ToString(w, v.byId); w.writeBool(v.hidden); w.writeU8(v.v2); } Outer _unpackOuter(_BufferReader r) => Outer( - inner: _unpackInner(r), - maybeInner: (r.readFlag() ? _unpackInner(r) : null), - byName: {for (var i = r.readLength(); i > 0; i--) r.readString(): _unpackInner(r)}, - modes: [for (var i = r.readLength(); i > 0; i--) Mode.fromValue(r.readI32())], - matrix: >[for (var i = r.readLength(); i > 0; i--) [for (var i = r.readLength(); i > 0; i--) r.readI32()]], - sparse: [for (var i = r.readLength(); i > 0; i--) (r.readFlag() ? r.readI64() : null)], - blob: (r.readFlag() ? r.readBytes() : null), - note: (r.readFlag() ? r.readString() : null), - byId: {for (var i = r.readLength(); i > 0; i--) r.readI64(): r.readString()}, - hidden: r.readBool(), - v2: r.readU8(), -); + inner: _unpackInner(r), + maybeInner: _unpackOptionalOfInner(r), + byName: _unpackMapOfStringToInner(r), + modes: _unpackListOfMode(r), + matrix: _unpackListOfListOfI32(r), + sparse: _unpackListOfOptionalOfI64(r), + blob: _unpackOptionalOfBytes(r), + note: _unpackOptionalOfString(r), + byId: _unpackMapOfI64ToString(r), + hidden: r.readBool(), + v2: r.readU8(), + ); /// Every field is a reserved word somewhere -class Keywords { +final class Keywords { /// Creates a [Keywords]. Keywords({required this.class_, required this.def, required this.import, required this.from, required this.func, required this.range, required this.register, required this.volatile, required this.new_, required this.default_, required this.end, required this.in_, required this.is_, required this.int_, required this.var_, required this.val, required this.self, required this.this_, required this.object, required this.string, required this.case_, required this.select, required this.switch_, required this.void_, required this.union, required this.namespace, required this.operator, required this.template, required this.package, required this.protocol, required this.guard, required this.defer, required this.inout, required this.internal, required this.when, required this.fun, required this.data, required this.late, required this.part, required this.out, required this.ref, required this.event, required this.base, required this.lambda, required this.yield, required this.nil, required this.null_, required this.begin, required this.rescue, required this.module, required this.chan, required this.go, required this.let, required this.export, required this.delete, required this.typedef, required this.sizeof, required this.static, required this.super_}); @@ -1034,6 +1408,76 @@ class Keywords { final int sizeof; final int static; final int super_; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Keywords && + class_ == other.class_ && + def == other.def && + import == other.import && + from == other.from && + func == other.func && + range == other.range && + register == other.register && + volatile == other.volatile && + new_ == other.new_ && + default_ == other.default_ && + end == other.end && + in_ == other.in_ && + is_ == other.is_ && + int_ == other.int_ && + var_ == other.var_ && + val == other.val && + self == other.self && + this_ == other.this_ && + object == other.object && + string == other.string && + case_ == other.case_ && + select == other.select && + switch_ == other.switch_ && + void_ == other.void_ && + union == other.union && + namespace == other.namespace && + operator == other.operator && + template == other.template && + package == other.package && + protocol == other.protocol && + guard == other.guard && + defer == other.defer && + inout == other.inout && + internal == other.internal && + when == other.when && + fun == other.fun && + data == other.data && + late == other.late && + part == other.part && + out == other.out && + ref == other.ref && + event == other.event && + base == other.base && + lambda == other.lambda && + yield == other.yield && + nil == other.nil && + null_ == other.null_ && + begin == other.begin && + rescue == other.rescue && + module == other.module && + chan == other.chan && + go == other.go && + let == other.let && + export == other.export && + delete == other.delete && + typedef == other.typedef && + sizeof == other.sizeof && + static == other.static && + super_ == other.super_; + + @override + int get hashCode => Object.hashAll([Keywords, class_, def, import, from, func, range, register, volatile, new_, default_, end, in_, is_, int_, var_, val, self, this_, object, string, case_, select, switch_, void_, union, namespace, operator, template, package, protocol, guard, defer, inout, internal, when, fun, data, late, part, out, ref, event, base, lambda, yield, nil, null_, begin, rescue, module, chan, go, let, export, delete, typedef, sizeof, static, super_]); + + @override + String toString() => 'Keywords(class_: $class_, def: $def, import: $import, from: $from, func: $func, range: $range, register: $register, volatile: $volatile, new_: $new_, default_: $default_, end: $end, in_: $in_, is_: $is_, int_: $int_, var_: $var_, val: $val, self: $self, this_: $this_, object: $object, string: $string, case_: $case_, select: $select, switch_: $switch_, void_: $void_, union: $union, namespace: $namespace, operator: $operator, template: $template, package: $package, protocol: $protocol, guard: $guard, defer: $defer, inout: $inout, internal: $internal, when: $when, fun: $fun, data: $data, late: $late, part: $part, out: $out, ref: $ref, event: $event, base: $base, lambda: $lambda, yield: $yield, nil: $nil, null_: $null_, begin: $begin, rescue: $rescue, module: $module, chan: $chan, go: $go, let: $let, export: $export, delete: $delete, typedef: $typedef, sizeof: $sizeof, static: $static, super_: $super_)'; } void _packKeywords(_BufferWriter w, Keywords v) { @@ -1099,74 +1543,86 @@ void _packKeywords(_BufferWriter w, Keywords v) { } Keywords _unpackKeywords(_BufferReader r) => Keywords( - class_: r.readI32(), - def: r.readI32(), - import: r.readI32(), - from: r.readI32(), - func: r.readI32(), - range: r.readI32(), - register: r.readI32(), - volatile: r.readI32(), - new_: r.readI32(), - default_: r.readI32(), - end: r.readI32(), - in_: r.readI32(), - is_: r.readI32(), - int_: r.readI32(), - var_: r.readI32(), - val: r.readI32(), - self: r.readI32(), - this_: r.readI32(), - object: r.readI32(), - string: r.readI32(), - case_: r.readI32(), - select: r.readI32(), - switch_: r.readI32(), - void_: r.readI32(), - union: r.readI32(), - namespace: r.readI32(), - operator: r.readI32(), - template: r.readI32(), - package: r.readI32(), - protocol: r.readI32(), - guard: r.readI32(), - defer: r.readI32(), - inout: r.readI32(), - internal: r.readI32(), - when: r.readI32(), - fun: r.readI32(), - data: r.readI32(), - late: r.readI32(), - part: r.readI32(), - out: r.readI32(), - ref: r.readI32(), - event: r.readI32(), - base: r.readI32(), - lambda: r.readI32(), - yield: r.readI32(), - nil: r.readI32(), - null_: r.readI32(), - begin: r.readI32(), - rescue: r.readI32(), - module: r.readI32(), - chan: r.readI32(), - go: r.readI32(), - let: r.readI32(), - export: r.readI32(), - delete: r.readI32(), - typedef: r.readI32(), - sizeof: r.readI32(), - static: r.readI32(), - super_: r.readI32(), -); + class_: r.readI32(), + def: r.readI32(), + import: r.readI32(), + from: r.readI32(), + func: r.readI32(), + range: r.readI32(), + register: r.readI32(), + volatile: r.readI32(), + new_: r.readI32(), + default_: r.readI32(), + end: r.readI32(), + in_: r.readI32(), + is_: r.readI32(), + int_: r.readI32(), + var_: r.readI32(), + val: r.readI32(), + self: r.readI32(), + this_: r.readI32(), + object: r.readI32(), + string: r.readI32(), + case_: r.readI32(), + select: r.readI32(), + switch_: r.readI32(), + void_: r.readI32(), + union: r.readI32(), + namespace: r.readI32(), + operator: r.readI32(), + template: r.readI32(), + package: r.readI32(), + protocol: r.readI32(), + guard: r.readI32(), + defer: r.readI32(), + inout: r.readI32(), + internal: r.readI32(), + when: r.readI32(), + fun: r.readI32(), + data: r.readI32(), + late: r.readI32(), + part: r.readI32(), + out: r.readI32(), + ref: r.readI32(), + event: r.readI32(), + base: r.readI32(), + lambda: r.readI32(), + yield: r.readI32(), + nil: r.readI32(), + null_: r.readI32(), + begin: r.readI32(), + rescue: r.readI32(), + module: r.readI32(), + chan: r.readI32(), + go: r.readI32(), + let: r.readI32(), + export: r.readI32(), + delete: r.readI32(), + typedef: r.readI32(), + sizeof: r.readI32(), + static: r.readI32(), + super_: r.readI32(), + ); /// Deprecated struct @Deprecated('Use Outer instead') -class Retired { +final class Retired { /// Creates a [Retired]. Retired({required this.id}); final int id; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Retired && + id == other.id; + + @override + int get hashCode => Object.hashAll([Retired, id]); + + @override + String toString() => 'Retired(id: $id)'; } void _packRetired(_BufferWriter w, Retired v) { @@ -1174,11 +1630,11 @@ void _packRetired(_BufferWriter w, Retired v) { } Retired _unpackRetired(_BufferReader r) => Retired( - id: r.readI64(), -); + id: r.readI64(), + ); /// Record carrying objects in every buffered position -class WidgetHolder { +final class WidgetHolder { /// Creates a [WidgetHolder]. WidgetHolder({required this.primary, this.fallback, required this.extras, required this.byName}); @@ -1186,34 +1642,36 @@ class WidgetHolder { final Widget? fallback; final List extras; final Map byName; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is WidgetHolder && + primary == other.primary && + fallback == other.fallback && + _deepEquals(extras, other.extras) && + _deepEquals(byName, other.byName); + + @override + int get hashCode => Object.hashAll([WidgetHolder, primary, fallback, _deepHash(extras), _deepHash(byName)]); + + @override + String toString() => 'WidgetHolder(primary: $primary, fallback: $fallback, extras: $extras, byName: $byName)'; } void _packWidgetHolder(_BufferWriter w, WidgetHolder v) { w.writeU64(v.primary._cloneRef().address); - final _t0 = v.fallback; - w.writeFlag(_t0 != null); - if (_t0 != null) { - w.writeU64(_t0._cloneRef().address); - } - final _t1 = v.extras; - w.writeLength(_t1.length); - for (final _t2 in _t1) { - w.writeU64(_t2._cloneRef().address); - } - final _t3 = v.byName; - w.writeLength(_t3.length); - for (final _t4 in _t3.entries) { - w.writeString(_t4.key); - w.writeU64(_t4.value._cloneRef().address); - } + _packOptionalOfWidget(w, v.fallback); + _packListOfWidget(w, v.extras); + _packMapOfStringToWidget(w, v.byName); } WidgetHolder _unpackWidgetHolder(_BufferReader r) => WidgetHolder( - primary: Widget._(Pointer.fromAddress(r.readU64())), - fallback: (r.readFlag() ? Widget._(Pointer.fromAddress(r.readU64())) : null), - extras: [for (var i = r.readLength(); i > 0; i--) Widget._(Pointer.fromAddress(r.readU64()))], - byName: {for (var i = r.readLength(); i > 0; i--) r.readString(): Widget._(Pointer.fromAddress(r.readU64()))}, -); + primary: Widget._(Pointer.fromAddress(r.readU64())), + fallback: _unpackOptionalOfWidget(r), + extras: _unpackListOfWidget(r), + byName: _unpackMapOfStringToWidget(r), + ); /// Callback interface carrying composite payloads and objects /// @@ -1223,11 +1681,13 @@ WidgetHolder _unpackWidgetHolder(_BufferReader r) => WidgetHolder( /// /// Methods that return a value run synchronously and must be called by /// the producer on this isolate's thread during a call from Dart; a -/// thrown exception fails that call with [NativeException.foreignCode]. -/// Void methods may be called from any thread: they're delivered -/// asynchronously on this isolate's event loop, in the zone that passed -/// the instance, where a thrown exception is an uncaught error of that -/// zone. Object arguments are owned by the implementation. +/// thrown exception fails that call with [NativeException.foreignCode] +/// (or, from a method that throws, the domain exception's code and +/// fields). Void methods may be called from any thread: they're +/// delivered asynchronously on this isolate's event loop, in the zone +/// that passed the instance, where a thrown exception is an uncaught +/// error of that zone. Object arguments are owned by the +/// implementation. abstract class BatchListener { /// Composite payloads cross as value buffers void onBatch(List items, List raw, Outer? first, Mode mode); @@ -1252,12 +1712,16 @@ typedef _EdgeCasesEdgeCasesBatchListenerPickMode = Int32 Function(Pointer, // The C `edge_cases_edge_cases_BatchListener_vtable` layout. final class _BatchListenerVtable extends Struct { + @Uint32() + external int size; + @Uint32() + external int flags; + external Pointer)>> free; external Pointer> onBatch; external Pointer> onWidget; external Pointer> onTick; external Pointer> shouldContinue; external Pointer> pickMode; - external Pointer)>> free; } // Vtable entry `edge_cases_edge_cases_BatchListener_vtable.on_batch`: forwarded to the isolate. @@ -1291,8 +1755,8 @@ bool _batchListenerShouldContinue(Pointer ctx, int class_, Pointer<_Error> return (_callbackTarget(ctx) as BatchListener).shouldContinue(class_); } catch (e) { _foreignError(outErr, e); - return false; } + return false; } // Vtable entry `edge_cases_edge_cases_BatchListener_vtable.pick_mode`. @@ -1301,27 +1765,27 @@ int _batchListenerPickMode(Pointer ctx, Pointer<_Error> outErr) { return (_callbackTarget(ctx) as BatchListener).pickMode().value; } catch (e) { _foreignError(outErr, e); - return 0; } + return 0; } void _batchListenerDispatch(Object impl, int method, List message) { final target = impl as BatchListener; switch (method) { case 0: { - final _a2 = _decode(message[2]! as Uint8List, (r) => [for (var i = r.readLength(); i > 0; i--) _unpackOuter(r)]); - final _a3 = message[3]! as Uint8List; - final _a4 = _decode(message[4]! as Uint8List, (r) => (r.readFlag() ? _unpackOuter(r) : null)); - final _a5 = Mode.fromValue(message[5]! as int); - target.onBatch(_a2, _a3, _a4, _a5); + final a2 = _decode(message[2]! as Uint8List, _unpackListOfOuter); + final a3 = message[3]! as Uint8List; + final a4 = _decode(message[4]! as Uint8List, _unpackOptionalOfOuter); + final a5 = Mode.fromValue(message[5]! as int); + target.onBatch(a2, a3, a4, a5); } case 1: { - final _a2Address = message[2]! as int; - final _a2 = Widget._(Pointer.fromAddress(_a2Address)); - final _a3Address = message[3]! as int; - final _a3 = _a3Address == 0 ? null : Widget._(Pointer.fromAddress(_a3Address)); - final _a4 = _decode(message[4]! as Uint8List, (r) => [for (var i = r.readLength(); i > 0; i--) Widget._(Pointer.fromAddress(r.readU64()))]); - target.onWidget(_a2, _a3, _a4); + final a2Address = message[2]! as int; + final a2 = Widget._(Pointer.fromAddress(a2Address)); + final a3Address = message[3]! as int; + final a3 = a3Address == 0 ? null : Widget._(Pointer.fromAddress(a3Address)); + final a4 = _decode(message[4]! as Uint8List, _unpackListOfWidget); + target.onWidget(a2, a3, a4); } case 2: { target.onTick(); @@ -1332,12 +1796,14 @@ void _batchListenerDispatch(Object impl, int method, List message) { final Pointer<_BatchListenerVtable> _batchListenerVtable = () { final vtable = calloc<_BatchListenerVtable>(); vtable.ref + ..size = sizeOf<_BatchListenerVtable>() + ..flags = 0 + ..free = _callbackFree ..onBatch = _pin(NativeCallable<_EdgeCasesEdgeCasesBatchListenerOnBatch>.isolateGroupBound(_batchListenerOnBatch)) ..onWidget = _pin(NativeCallable<_EdgeCasesEdgeCasesBatchListenerOnWidget>.isolateGroupBound(_batchListenerOnWidget)) ..onTick = _pin(NativeCallable<_EdgeCasesEdgeCasesBatchListenerOnTick>.isolateGroupBound(_batchListenerOnTick)) ..shouldContinue = _pin(NativeCallable<_EdgeCasesEdgeCasesBatchListenerShouldContinue>.isolateLocal(_batchListenerShouldContinue, exceptionalReturn: false)) - ..pickMode = _pin(NativeCallable<_EdgeCasesEdgeCasesBatchListenerPickMode>.isolateLocal(_batchListenerPickMode, exceptionalReturn: 0)) - ..free = _callbackFree; + ..pickMode = _pin(NativeCallable<_EdgeCasesEdgeCasesBatchListenerPickMode>.isolateLocal(_batchListenerPickMode, exceptionalReturn: 0)); return vtable; }(); @@ -1349,11 +1815,13 @@ final Pointer<_BatchListenerVtable> _batchListenerVtable = () { /// /// Methods that return a value run synchronously and must be called by /// the producer on this isolate's thread during a call from Dart; a -/// thrown exception fails that call with [NativeException.foreignCode]. -/// Void methods may be called from any thread: they're delivered -/// asynchronously on this isolate's event loop, in the zone that passed -/// the instance, where a thrown exception is an uncaught error of that -/// zone. Object arguments are owned by the implementation. +/// thrown exception fails that call with [NativeException.foreignCode] +/// (or, from a method that throws, the domain exception's code and +/// fields). Void methods may be called from any thread: they're +/// delivered asynchronously on this isolate's event loop, in the zone +/// that passed the instance, where a thrown exception is an uncaught +/// error of that zone. Object arguments are owned by the +/// implementation. abstract class Keyword { void delete(int self); @@ -1365,9 +1833,13 @@ typedef _EdgeCasesEdgeCasesKeywordImport = Int32 Function(Pointer, Pointer // The C `edge_cases_edge_cases_Keyword_vtable` layout. final class _KeywordVtable extends Struct { + @Uint32() + external int size; + @Uint32() + external int flags; + external Pointer)>> free; external Pointer> delete; external Pointer> import; - external Pointer)>> free; } // Vtable entry `edge_cases_edge_cases_Keyword_vtable.delete`: forwarded to the isolate. @@ -1383,16 +1855,16 @@ int _keywordImport(Pointer ctx, Pointer<_Error> outErr) { return (_callbackTarget(ctx) as Keyword).import(); } catch (e) { _foreignError(outErr, e); - return 0; } + return 0; } void _keywordDispatch(Object impl, int method, List message) { final target = impl as Keyword; switch (method) { case 0: { - final _a2 = message[2]! as int; - target.delete(_a2); + final a2 = message[2]! as int; + target.delete(a2); } } } @@ -1400,9 +1872,11 @@ void _keywordDispatch(Object impl, int method, List message) { final Pointer<_KeywordVtable> _keywordVtable = () { final vtable = calloc<_KeywordVtable>(); vtable.ref + ..size = sizeOf<_KeywordVtable>() + ..flags = 0 + ..free = _callbackFree ..delete = _pin(NativeCallable<_EdgeCasesEdgeCasesKeywordDelete>.isolateGroupBound(_keywordDelete)) - ..import = _pin(NativeCallable<_EdgeCasesEdgeCasesKeywordImport>.isolateLocal(_keywordImport, exceptionalReturn: 0)) - ..free = _callbackFree; + ..import = _pin(NativeCallable<_EdgeCasesEdgeCasesKeywordImport>.isolateLocal(_keywordImport, exceptionalReturn: 0)); return vtable; }(); @@ -1434,6 +1908,14 @@ final _edgeCasesEdgeCasesWidgetOldName = _lib.lookupFunction< Pointer Function(Pointer, Pointer, Pointer<_Error>), Pointer Function(Pointer, Pointer, Pointer<_Error>)>('edge_cases_edge_cases_Widget_old_name'); +final _edgeCasesEdgeCasesWidgetClose = _lib.lookupFunction< + Bool Function(Pointer, Pointer<_Error>), + bool Function(Pointer, Pointer<_Error>)>('edge_cases_edge_cases_Widget_close'); + +final _edgeCasesEdgeCasesWidgetDispose = _lib.lookupFunction< + Void Function(Pointer, Bool, Pointer<_Error>), + void Function(Pointer, bool, Pointer<_Error>)>('edge_cases_edge_cases_Widget_dispose'); + /// Interface with no constructor; instances come from make_widget /// /// A reference-counted native object. Each instance holds one strong @@ -1469,13 +1951,7 @@ final class Widget extends _NativeObject { final _arena = Arena(); try { final _class_Bytes = utf8.encode(class_); - final _importWriter = _BufferWriter(); - final _t0 = import; - _importWriter.writeFlag(_t0 != null); - if (_t0 != null) { - _importWriter.writeString(_t0); - } - final _importBytes = _importWriter.takeBytes(); + final _importBytes = _encode(import, _packOptionalOfString); final _result = _edgeCasesEdgeCasesWidgetRename(_self, _stage(_arena, _class_Bytes), _class_Bytes.length, _stage(_arena, _importBytes), _importBytes.length, _outLen, _err); _check(_err); return _takeString(_result, _outLen.value); @@ -1493,11 +1969,11 @@ final class Widget extends _NativeObject { final _completer = Completer(); late final NativeCallable<_EdgeCasesEdgeCasesWidgetFetchCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesWidgetFetchCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _mapEdgeErrorsException); - _completer.complete(_decode(_takeBytes(_value, _valueLen), _unpackOuter)); + if (error != nullptr) throw _takeAsyncError(error, _mapEdgeErrorsException); + _completer.complete(_decode(_takeBytes(value, valueLen), _unpackOuter)); } catch (e, s) { _completer.completeError(e, s); } @@ -1524,6 +2000,29 @@ final class Widget extends _NativeObject { _leave(); } } + + /// Spelled like the release method of several wrappers + bool close() { + final _self = _enter(); + try { + final _result = _edgeCasesEdgeCasesWidgetClose(_self, _err); + _check(_err); + return _result; + } finally { + _leave(); + } + } + + /// Spelled like the release method of the Dart and .NET wrappers + void dispose_(bool force) { + final _self = _enter(); + try { + _edgeCasesEdgeCasesWidgetDispose(_self, force, _err); + _check(_err); + } finally { + _leave(); + } + } } final _edgeCasesEdgeCasesVaultClone = _lib.lookupFunction< @@ -1544,10 +2043,18 @@ final _edgeCasesEdgeCasesVaultPeek = _lib.lookupFunction< Pointer Function(Pointer, Pointer<_Error>), Pointer Function(Pointer, Pointer<_Error>)>('edge_cases_edge_cases_Vault_peek'); +final _edgeCasesEdgeCasesVaultHandle = _lib.lookupFunction< + Int64 Function(Pointer, Pointer<_Error>), + int Function(Pointer, Pointer<_Error>)>('edge_cases_edge_cases_Vault_handle'); + final _edgeCasesEdgeCasesVaultProbe = _lib.lookupFunction< Bool Function(Pointer<_Error>), bool Function(Pointer<_Error>)>('edge_cases_edge_cases_Vault_probe'); +final _edgeCasesEdgeCasesVaultToString = _lib.lookupFunction< + Pointer Function(Pointer, Pointer<_Error>), + Pointer Function(Pointer, Pointer<_Error>)>('edge_cases_edge_cases_Vault_to_string'); + /// Deprecated interface with a throwing constructor /// /// A reference-counted native object. Each instance holds one strong @@ -1597,6 +2104,18 @@ final class Vault extends _NativeObject { } } + /// Spelled like the handle accessor of several wrappers + int handle() { + final _self = _enter(); + try { + final _result = _edgeCasesEdgeCasesVaultHandle(_self, _err); + _check(_err); + return _result; + } finally { + _leave(); + } + } + /// Throwing static /// /// Throws [EdgeErrorsException] on domain errors. @@ -1605,6 +2124,13 @@ final class Vault extends _NativeObject { _check(_err, _mapEdgeErrorsException); return _result; } + + /// Static spelled like an inherited object member + static String toString_() { + final _result = _edgeCasesEdgeCasesVaultToString(_outLen, _err); + _check(_err); + return _takeString(_result, _outLen.value); + } } final _edgeCasesEdgeCasesNoop = _lib.lookupFunction< @@ -1626,20 +2152,8 @@ int select(int class_, String from, List func, int? range, bool type) { final _arena = Arena(); try { final _fromBytes = utf8.encode(from); - final _funcWriter = _BufferWriter(); - final _t0 = func; - _funcWriter.writeLength(_t0.length); - for (final _t1 in _t0) { - _funcWriter.writeString(_t1); - } - final _funcBytes = _funcWriter.takeBytes(); - final _rangeWriter = _BufferWriter(); - final _t2 = range; - _rangeWriter.writeFlag(_t2 != null); - if (_t2 != null) { - _rangeWriter.writeI32(_t2); - } - final _rangeBytes = _rangeWriter.takeBytes(); + final _funcBytes = _encode(func, _packListOfString); + final _rangeBytes = _encode(range, _packOptionalOfI32); final _result = _edgeCasesEdgeCasesSelect(class_, _stage(_arena, _fromBytes), _fromBytes.length, _stage(_arena, _funcBytes), _funcBytes.length, _stage(_arena, _rangeBytes), _rangeBytes.length, type, _err); _check(_err); return _result; @@ -1656,24 +2170,10 @@ final _edgeCasesEdgeCasesNestedOptional = _lib.lookupFunction< List? nestedOptional(List? items) { final _arena = Arena(); try { - final _itemsWriter = _BufferWriter(); - final _t0 = items; - _itemsWriter.writeFlag(_t0 != null); - if (_t0 != null) { - final _t1 = _t0; - _itemsWriter.writeLength(_t1.length); - for (final _t2 in _t1) { - final _t3 = _t2; - _itemsWriter.writeFlag(_t3 != null); - if (_t3 != null) { - _packOuter(_itemsWriter, _t3); - } - } - } - final _itemsBytes = _itemsWriter.takeBytes(); + final _itemsBytes = _encode(items, _packOptionalOfListOfOptionalOfOuter); final _result = _edgeCasesEdgeCasesNestedOptional(_stage(_arena, _itemsBytes), _itemsBytes.length, _outLen, _err); _check(_err); - return _decode(_takeBytes(_result, _outLen.value), (r) => (r.readFlag() ? [for (var i = r.readLength(); i > 0; i--) (r.readFlag() ? _unpackOuter(r) : null)] : null)); + return _decode(_takeBytes(_result, _outLen.value), _unpackOptionalOfListOfOptionalOfOuter); } finally { _arena.releaseAll(); } @@ -1686,55 +2186,15 @@ final _edgeCasesEdgeCasesOptionalParams = _lib.lookupFunction< Mode? optionalParams(String? text, int? count, Outer? outer, List? tags, List? blob, Mode? mode) { final _arena = Arena(); try { - final _textWriter = _BufferWriter(); - final _t0 = text; - _textWriter.writeFlag(_t0 != null); - if (_t0 != null) { - _textWriter.writeString(_t0); - } - final _textBytes = _textWriter.takeBytes(); - final _countWriter = _BufferWriter(); - final _t1 = count; - _countWriter.writeFlag(_t1 != null); - if (_t1 != null) { - _countWriter.writeI32(_t1); - } - final _countBytes = _countWriter.takeBytes(); - final _outerWriter = _BufferWriter(); - final _t2 = outer; - _outerWriter.writeFlag(_t2 != null); - if (_t2 != null) { - _packOuter(_outerWriter, _t2); - } - final _outerBytes = _outerWriter.takeBytes(); - final _tagsWriter = _BufferWriter(); - final _t3 = tags; - _tagsWriter.writeFlag(_t3 != null); - if (_t3 != null) { - final _t4 = _t3; - _tagsWriter.writeLength(_t4.length); - for (final _t5 in _t4) { - _tagsWriter.writeString(_t5); - } - } - final _tagsBytes = _tagsWriter.takeBytes(); - final _blobWriter = _BufferWriter(); - final _t6 = blob; - _blobWriter.writeFlag(_t6 != null); - if (_t6 != null) { - _blobWriter.writeBytes(_t6); - } - final _blobBytes = _blobWriter.takeBytes(); - final _modeWriter = _BufferWriter(); - final _t7 = mode; - _modeWriter.writeFlag(_t7 != null); - if (_t7 != null) { - _modeWriter.writeI32(_t7.value); - } - final _modeBytes = _modeWriter.takeBytes(); + final _textBytes = _encode(text, _packOptionalOfString); + final _countBytes = _encode(count, _packOptionalOfI32); + final _outerBytes = _encode(outer, _packOptionalOfOuter); + final _tagsBytes = _encode(tags, _packOptionalOfListOfString); + final _blobBytes = _encode(blob, _packOptionalOfBytes); + final _modeBytes = _encode(mode, _packOptionalOfMode); final _result = _edgeCasesEdgeCasesOptionalParams(_stage(_arena, _textBytes), _textBytes.length, _stage(_arena, _countBytes), _countBytes.length, _stage(_arena, _outerBytes), _outerBytes.length, _stage(_arena, _tagsBytes), _tagsBytes.length, _stage(_arena, _blobBytes), _blobBytes.length, _stage(_arena, _modeBytes), _modeBytes.length, _outLen, _err); _check(_err); - return _decode(_takeBytes(_result, _outLen.value), (r) => (r.readFlag() ? Mode.fromValue(r.readI32()) : null)); + return _decode(_takeBytes(_result, _outLen.value), _unpackOptionalOfMode); } finally { _arena.releaseAll(); } @@ -1748,7 +2208,7 @@ final _edgeCasesEdgeCasesRegisterBatch = _lib.lookupFunction< void registerBatch(BatchListener listener, Keyword keyword) { final _listenerCtx = _registerCallback(listener, _batchListenerDispatch); final _keywordCtx = _registerCallback(keyword, _keywordDispatch); - _edgeCasesEdgeCasesRegisterBatch(_listenerCtx, _batchListenerVtable.cast(), _keywordCtx, _keywordVtable.cast(), _err); + _edgeCasesEdgeCasesRegisterBatch(_listenerCtx, _batchListenerVtable.cast(), _keywordCtx, _keywordVtable.cast(), _err); _check(_err); } @@ -1760,35 +2220,10 @@ final _edgeCasesEdgeCasesWidgetsEverywhere = _lib.lookupFunction< WidgetHolder widgetsEverywhere(List list, Map byName, List sparse, WidgetHolder holder) { final _arena = Arena(); try { - final _listWriter = _BufferWriter(); - final _t0 = list; - _listWriter.writeLength(_t0.length); - for (final _t1 in _t0) { - _listWriter.writeU64(_t1._cloneRef().address); - } - final _listBytes = _listWriter.takeBytes(); - final _byNameWriter = _BufferWriter(); - final _t2 = byName; - _byNameWriter.writeLength(_t2.length); - for (final _t3 in _t2.entries) { - _byNameWriter.writeString(_t3.key); - _byNameWriter.writeU64(_t3.value._cloneRef().address); - } - final _byNameBytes = _byNameWriter.takeBytes(); - final _sparseWriter = _BufferWriter(); - final _t4 = sparse; - _sparseWriter.writeLength(_t4.length); - for (final _t5 in _t4) { - final _t6 = _t5; - _sparseWriter.writeFlag(_t6 != null); - if (_t6 != null) { - _sparseWriter.writeU64(_t6._cloneRef().address); - } - } - final _sparseBytes = _sparseWriter.takeBytes(); - final _holderWriter = _BufferWriter(); - _packWidgetHolder(_holderWriter, holder); - final _holderBytes = _holderWriter.takeBytes(); + final _listBytes = _encode(list, _packListOfWidget); + final _byNameBytes = _encode(byName, _packMapOfStringToWidget); + final _sparseBytes = _encode(sparse, _packListOfOptionalOfWidget); + final _holderBytes = _encode(holder, _packWidgetHolder); final _result = _edgeCasesEdgeCasesWidgetsEverywhere(_stage(_arena, _listBytes), _listBytes.length, _stage(_arena, _byNameBytes), _byNameBytes.length, _stage(_arena, _sparseBytes), _sparseBytes.length, _stage(_arena, _holderBytes), _holderBytes.length, _outLen, _err); _check(_err); return _decode(_takeBytes(_result, _outLen.value), _unpackWidgetHolder); @@ -1855,11 +2290,11 @@ Future asyncWidget() { final _completer = Completer(); late final NativeCallable<_EdgeCasesEdgeCasesAsyncWidgetCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncWidgetCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value) { + Pointer _, Pointer<_Error> error, Pointer value) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _mapEdgeErrorsException); - _completer.complete(Widget._(_value)); + if (error != nullptr) throw _takeAsyncError(error, _mapEdgeErrorsException); + _completer.complete(Widget._(value)); } catch (e, s) { _completer.completeError(e, s); } @@ -2055,10 +2490,10 @@ Future asyncVoid() { final _completer = Completer(); late final NativeCallable<_EdgeCasesEdgeCasesAsyncVoidCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncVoidCallback>.listener(( - Pointer _, Pointer<_Error> _error) { + Pointer _, Pointer<_Error> error) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); + if (error != nullptr) throw _takeAsyncError(error, _trap); _completer.complete(); } catch (e, s) { _completer.completeError(e, s); @@ -2083,11 +2518,11 @@ Future> asyncBytes(int size) { final _completer = Completer>(); late final NativeCallable<_EdgeCasesEdgeCasesAsyncBytesCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncBytesCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); - _completer.complete(_takeBytes(_value, _valueLen)); + if (error != nullptr) throw _takeAsyncError(error, _trap); + _completer.complete(_takeBytes(value, valueLen)); } catch (e, s) { _completer.completeError(e, s); } @@ -2112,11 +2547,11 @@ Future asyncStruct() { final _completer = Completer(); late final NativeCallable<_EdgeCasesEdgeCasesAsyncStructCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncStructCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _mapEdgeErrorsException); - _completer.complete(_decode(_takeBytes(_value, _valueLen), _unpackOuter)); + if (error != nullptr) throw _takeAsyncError(error, _mapEdgeErrorsException); + _completer.complete(_decode(_takeBytes(value, valueLen), _unpackOuter)); } catch (e, s) { _completer.completeError(e, s); } @@ -2145,12 +2580,12 @@ Future> asyncList({CancelToken? cancelToken}) { final _cancel = _NativeCancel.bind(cancelToken); late final NativeCallable<_EdgeCasesEdgeCasesAsyncListCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncListCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); _cancel?.release(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _mapEdgeErrorsException); - _completer.complete(_decode(_takeBytes(_value, _valueLen), (r) => [for (var i = r.readLength(); i > 0; i--) _unpackOuter(r)])); + if (error != nullptr) throw _takeAsyncError(error, _mapEdgeErrorsException); + _completer.complete(_decode(_takeBytes(value, valueLen), _unpackListOfOuter)); } catch (e, s) { _completer.completeError(e, s); } @@ -2175,11 +2610,11 @@ Future asyncOptional() { final _completer = Completer(); late final NativeCallable<_EdgeCasesEdgeCasesAsyncOptionalCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncOptionalCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); - _completer.complete(_decode(_takeBytes(_value, _valueLen), (r) => (r.readFlag() ? r.readI64() : null))); + if (error != nullptr) throw _takeAsyncError(error, _trap); + _completer.complete(_decode(_takeBytes(value, valueLen), _unpackOptionalOfI64)); } catch (e, s) { _completer.completeError(e, s); } @@ -2203,11 +2638,11 @@ Future asyncEnum() { final _completer = Completer(); late final NativeCallable<_EdgeCasesEdgeCasesAsyncEnumCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncEnumCallback>.listener(( - Pointer _, Pointer<_Error> _error, int _value) { + Pointer _, Pointer<_Error> error, int value) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); - _completer.complete(Mode.fromValue(_value)); + if (error != nullptr) throw _takeAsyncError(error, _trap); + _completer.complete(Mode.fromValue(value)); } catch (e, s) { _completer.completeError(e, s); } @@ -2232,11 +2667,11 @@ Future asyncKeywords(String class_, int import) { final _completer = Completer(); late final NativeCallable<_EdgeCasesEdgeCasesAsyncKeywordsCallback> _callback; _callback = NativeCallable<_EdgeCasesEdgeCasesAsyncKeywordsCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); - _completer.complete(_takeString(_value, _valueLen)); + if (error != nullptr) throw _takeAsyncError(error, _trap); + _completer.complete(_takeString(value, valueLen)); } catch (e, s) { _completer.completeError(e, s); } @@ -2260,9 +2695,7 @@ final _edgeCasesEdgeCasesKeywordsRoundtrip = _lib.lookupFunction< Keywords keywordsRoundtrip(Keywords value) { final _arena = Arena(); try { - final _valueWriter = _BufferWriter(); - _packKeywords(_valueWriter, value); - final _valueBytes = _valueWriter.takeBytes(); + final _valueBytes = _encode(value, _packKeywords); final _result = _edgeCasesEdgeCasesKeywordsRoundtrip(_stage(_arena, _valueBytes), _valueBytes.length, _outLen, _err); _check(_err); return _decode(_takeBytes(_result, _outLen.value), _unpackKeywords); diff --git a/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__pubspec_yaml.snap b/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__pubspec_yaml.snap deleted file mode 100644 index 40ccfab8..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_edge_cases__pubspec_yaml.snap +++ /dev/null @@ -1,18 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -name: edge_cases -version: 0.1.0 -description: 'Native bindings for edge_cases' - -environment: - sdk: '>=3.12.0 <4.0.0' - -dependencies: - ffi: ^2.1.0 - -# END pubspec.yaml diff --git a/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__README_md.snap index 79832093..7cf53204 100644 --- a/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__README_md.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # kitchen_sink (Dart) @@ -27,13 +27,14 @@ fallback, the bindings look the symbols up in the running executable. Each interface wrapper holds one native reference: call `dispose()` when you're done, or let the garbage collector's finalizer release it. -Callback-interface methods that return a value run synchronously on the -isolate's thread; void methods may be called from any thread and are -delivered on the event loop. +Callback-interface methods that return a value run synchronously and must +be called by the native library on the isolate's thread during a call from +Dart; void methods may be called from any thread and are delivered on the +event loop. ## Requirements -- Dart SDK `>=3.12.0 <4.0.0` +- Dart SDK `>=3.10.0 <4.0.0` - The `ffi` package diff --git a/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__lib_kitchen_sink_dart.snap b/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__lib_kitchen_sink_dart.snap index b5eb029d..02c7ded1 100644 --- a/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__lib_kitchen_sink_dart.snap +++ b/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__lib_kitchen_sink_dart.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // ignore_for_file: camel_case_types, non_constant_identifier_names, unused_element @@ -18,46 +18,69 @@ import 'package:ffi/ffi.dart'; // ── Library loading ── // The native library opens on first use and is verified before any other -// lookup: its C ABI revision and every module's contract checksum must match -// the values these bindings were generated against. +// lookup: it must implement the C ABI revision these bindings were generated +// against and carry every declaration they were generated with, unchanged. /// The native library every binding in this file calls. final DynamicLibrary _lib = _verifyLibrary(_openLibrary()); +/// The native library couldn't be loaded, or it doesn't match these +/// bindings: a different C ABI revision, or a declaration that's missing or +/// changed. Regenerate the bindings from the library you ship. +final class NativeLibraryError extends Error { + /// Creates a load failure described by [message]. + NativeLibraryError(this.message); + + /// What failed, naming the library or declaration. + final String message; + + @override + String toString() => 'NativeLibraryError: $message'; +} + +/// The platform file name of the library on the system search path. +String get _libraryFile => Platform.isMacOS + ? 'libkitchen_sink.dylib' + : Platform.isWindows + ? 'kitchen_sink.dll' + : 'libkitchen_sink.so'; + DynamicLibrary _openLibrary() { // An explicit path wins, so a caller can point at any build artifact. final override = Platform.environment['KITCHEN_SINK_LIBRARY']; if (override != null && override.isNotEmpty) { - return DynamicLibrary.open(override); + return _open(override); } // iOS links native code into the app binary. if (Platform.isIOS) return DynamicLibrary.process(); - final candidates = Platform.isMacOS - ? const ['libkitchen_sink.dylib'] - : Platform.isWindows - ? const ['kitchen_sink.dll'] - : const ['libkitchen_sink.so']; - ArgumentError? failure; - for (final candidate in candidates) { - try { - return DynamicLibrary.open(candidate); - } on ArgumentError catch (e) { - failure ??= e; - } + try { + return DynamicLibrary.open(_libraryFile); + } on ArgumentError catch (e) { + // A macOS app may link the library into its executable instead. + if (Platform.isMacOS) return DynamicLibrary.process(); + throw NativeLibraryError("can't open '$_libraryFile' ($e); set " + 'KITCHEN_SINK_LIBRARY to its path'); + } +} + +DynamicLibrary _open(String path) { + try { + return DynamicLibrary.open(path); + } on ArgumentError catch (e) { + throw NativeLibraryError("can't open '$path': $e"); } - // A macOS app may link the library into its executable instead. - if (Platform.isMacOS) return DynamicLibrary.process(); - throw failure!; } /// The C ABI revision these bindings were generated against. -const int _abiVersion = 3; +const int _abiVersion = 4; -/// Each top-level module's name, checksum symbol, and expected checksum. -const List<(String, String, int)> _contracts = <(String, String, int)>[ - ('shared', 'kitchen_sink_shared_checksum', 0x42c4ce2c8d0af052), - ('kitchen', 'kitchen_sink_kitchen_checksum', 0xd83da75b24b545ff), -]; +/// One `kitchen_sink_contract_entry` of a module's contract table. +final class _ContractEntry extends Struct { + @Uint64() + external int id; + @Uint64() + external int hash; +} DynamicLibrary _verifyLibrary(DynamicLibrary lib) { final int Function() abiVersion; @@ -65,33 +88,97 @@ DynamicLibrary _verifyLibrary(DynamicLibrary lib) { abiVersion = lib.lookupFunction( 'kitchen_sink_abi_version'); } on ArgumentError { - throw StateError( - "the native library 'kitchen_sink' isn't loaded; set KITCHEN_SINK_LIBRARY to " - 'its path'); + throw NativeLibraryError( + "the native library 'kitchen_sink' isn't loaded; set KITCHEN_SINK_LIBRARY " + 'to its path'); } final found = abiVersion(); if (found != _abiVersion) { - throw StateError('kitchen_sink: these bindings expect C ABI revision ' - '$_abiVersion, but the loaded library reports revision $found'); + throw NativeLibraryError('kitchen_sink: these bindings expect C ABI revision ' + '$_abiVersion, but the loaded library implements revision $found'); } - for (final (module, symbol, expected) in _contracts) { - final int actual; - try { - actual = lib.lookupFunction(symbol)(); - } on ArgumentError { - throw StateError( - "kitchen_sink: the loaded library doesn't export module '$module'"); + for (final (symbol, expected) in _contracts) { + _checkContract(lib, symbol, expected); + } + return lib; +} + +/// Checks that every `(id, hash, path)` entry in [expected] is in the table +/// [symbol] returns with an equal hash. Entries only the library has are +/// fine: they're declarations added after these bindings were generated. +void _checkContract( + DynamicLibrary lib, String symbol, List<(int, int, String)> expected) { + final Pointer<_ContractEntry> Function(Pointer) table; + try { + table = lib.lookupFunction Function(Pointer), + Pointer<_ContractEntry> Function(Pointer)>(symbol); + } on ArgumentError { + throw NativeLibraryError( + "kitchen_sink: the loaded library doesn't export '$symbol'"); + } + final hashes = {}; + final len = calloc(); + try { + final entries = table(len); + for (var i = 0; i < len.value; i++) { + hashes[entries[i].id] = entries[i].hash; + } + } finally { + calloc.free(len); + } + for (final (id, hash, path) in expected) { + final found = hashes[id]; + if (found == null) { + throw NativeLibraryError('kitchen_sink: $path is missing from the library'); } - if (actual != expected) { - throw StateError("kitchen_sink: module '$module' doesn't match the loaded " - 'library (contract checksum ${_hex(actual)}, expected ' - '${_hex(expected)}); regenerate the bindings'); + if (found != hash) { + throw NativeLibraryError( + 'kitchen_sink: $path changed since these bindings were generated'); } } - return lib; } -String _hex(int v) => '0x${v.toUnsigned(64).toRadixString(16).padLeft(16, '0')}'; +/// Each top-level module's contract-table symbol and the declarations +/// these bindings were generated with: (id, signature hash, path). +const List<(String, List<(int, int, String)>)> _contracts = [ + ('kitchen_sink_shared_contract', [ + (0x709e234caa962446, 0x3894f2ca452e9f98, 'shared.ping'), + (0xcfb306fa5801ee5b, 0x1cbd9e10983cac2c, 'shared.Token'), + ]), + ('kitchen_sink_kitchen_contract', [ + (0x01d4f09bbddc986f, 0x29eb88fecc66544d, 'kitchen.Gadget.new'), + (0x10472d8fe748c2ee, 0xb265832b6a759fc4, 'kitchen.maybe_gadget'), + (0x11368eec07a74078, 0xfa049f78b77e146a, 'kitchen.gadgets'), + (0x137f08a29cde6021, 0xda8de7aeb2dbb8a3, 'kitchen.stream_gadgets'), + (0x2111b54980826dd7, 0x452783f3e6ca6e1d, 'kitchen.do_cancellable'), + (0x29dddbc60bf8d7db, 0x452a3e573d57f6db, 'kitchen.Priority'), + (0x2b227003b74a0d16, 0x12332957470e9782, 'kitchen.maybe_subscribe'), + (0x2dd66cb0064f0494, 0x60197363fefd92dd, 'kitchen.Item'), + (0x33c265bbf079b0e9, 0xc3dcd0927e6c1deb, 'kitchen.u32_id'), + (0x33e6381d01660dfe, 0x077c63c31fd7f90a, 'kitchen.KitchenErrors'), + (0x3f5fc99d73504b76, 0x2df88d710f1ea10e, 'kitchen.Gadget.poke'), + (0x4c5764cf109528d1, 0x2a65529af4338a58, 'kitchen.maybe_item'), + (0x4de17dfb93fd494d, 0xba0ad0dd52ed74af, 'kitchen.i32_id'), + (0x504e95b7f7e65570, 0x01b3e8e0c9a10e23, 'kitchen.stream_items'), + (0x68cdb9a4bd3e3dd5, 0xc678b9164348d497, 'kitchen.bool_id'), + (0x69e7cd703d16eb85, 0xf3b28dd22ae9e689, 'kitchen.do_async'), + (0x70865079ce41be82, 0xbe3b69464672af46, 'kitchen.Gadget.describe'), + (0x71c5a1116cb7c49a, 0xcba1837108a89b84, 'kitchen.echo_bytes'), + (0x735ee1e0049ee91a, 0xca74320bc53eddab, 'kitchen.echo_string'), + (0x76d7fc58f4e41d84, 0x7373a0e47d30e690, 'kitchen.i64_id'), + (0x79abfe1aa960fd6a, 0xbca7a9e5eb26f3a6, 'kitchen.nested.hello'), + (0x7d3dd7d5cf6749e3, 0xa192f5a7298adb39, 'kitchen.new_op'), + (0x89a5e664bc25c67b, 0x5764a95e81ecfd31, 'kitchen.f64_id'), + (0x8cfdbdcecac5f7f1, 0x07e272f09c41fffc, 'kitchen.Gadget.version'), + (0x974b9f3e823c1738, 0xb81b91e975695f27, 'kitchen.cross_module_token'), + (0xaca0655fac15d230, 0x11c4d2b28abe07bb, 'kitchen.list_items'), + (0xc883b4afae20543e, 0xc40560e5411ff392, 'kitchen.legacy_op'), + (0xf2f7753e6e1dffc2, 0xc9047a95bdf49e93, 'kitchen.get_attrs'), + (0xfae25d3391922ead, 0x1c7f95739f3210b8, 'kitchen.subscribe'), + (0xfd0059b05cb2fcfb, 0x248bf8404a5823b4, 'kitchen.Gadget'), + (0xfed9c551fb9fd2d6, 0x59aec980ef3f3291, 'kitchen.ReadyListener'), + ]), +]; // ── Errors and memory ── @@ -107,10 +194,8 @@ final class _Error extends Struct { external int payloadLen; } -final _errorClear = - _lib.lookupFunction), void Function(Pointer<_Error>)>( - 'kitchen_sink_error_clear', - isLeaf: true); +final _errorClear = _lib.lookupFunction), + void Function(Pointer<_Error>)>('kitchen_sink_error_clear', isLeaf: true); final _freeBytes = _lib.lookupFunction, Size), void Function(Pointer, int)>('kitchen_sink_free_bytes', isLeaf: true); @@ -121,9 +206,9 @@ final _freeBytes = _lib.lookupFunction, Size), final Pointer<_Error> _err = calloc<_Error>(); final Pointer _outLen = calloc(); -/// A failure the native library reported. Domain errors extend this class -/// with a positive [code]; the negative runtime codes are the constants -/// below. +/// A failure the native library reported to a function declared to throw. +/// Domain errors extend this class with a positive [code]; a negative code is +/// one of the runtime constants below. class NativeException implements Exception { /// Creates an exception carrying [code] and [message]. NativeException(this.code, this.message); @@ -134,10 +219,10 @@ class NativeException implements Exception { /// The producer panicked; [message] carries the panic text. static const int panicCode = -2; - /// An argument couldn't be lifted by the producer. + /// An argument or a callback's return couldn't be lifted by the producer. static const int marshalCode = -3; - /// A callback implementation threw; [message] carries its text. + /// A callback implementation failed; [message] carries its text. static const int foreignCode = -4; /// The call was cancelled (see [CancelledException]). @@ -161,24 +246,51 @@ class CancelledException extends NativeException { : super(NativeException.cancelledCode, message); } -typedef _ErrorMapper = NativeException Function( +/// A function that isn't declared to throw failed anyway. That's a bug in +/// the native library (a panic, an argument it couldn't lift, a callback +/// failure it let through), so it's an [Error], not an [Exception]. +final class NativeError extends Error { + /// Creates the error a failed call reported. + NativeError(this.code, this.message); + + /// The runtime code the producer reported (see [NativeException]). + final int code; + + /// The producer's message. + final String message; + + @override + String toString() => 'NativeError($code): $message'; +} + +/// Maps a reported code, message, and payload onto the object to throw. +typedef _ErrorMapper = Object Function( int code, String message, Uint8List payload); -/// Maps a runtime failure (or an undeclared code) onto its exception. -NativeException _runtimeError(int code, String message, Uint8List payload) => +/// The mapper of a function that can't fail: every code but cancellation +/// is a [NativeError]. +Object _trap(int code, String message, Uint8List payload) => + code == NativeException.cancelledCode + ? CancelledException(message) + : NativeError(code, message); + +/// Maps a runtime code (or one the domain doesn't declare) onto its +/// exception, for a function declared to throw. +NativeException _runtimeException(int code, String message, Uint8List payload) => code == NativeException.cancelledCode ? CancelledException(message) : NativeException(code, message); -/// Builds the exception [err] reports, using [map] for domain codes. -NativeException _readError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error [err] reports, using [map]. +Object _readError(Pointer<_Error> err, _ErrorMapper map) { final e = err.ref; final message = e.message == nullptr ? '' : e.message.toDartString(); return map(e.code, message, _copyBytes(e.payloadPtr, e.payloadLen)); } /// Throws the error a synchronous call left in [err], clearing the slot. -void _check(Pointer<_Error> err, [_ErrorMapper map = _runtimeError]) { +/// [map] is the domain's mapper for a function declared to throw. +void _check(Pointer<_Error> err, [_ErrorMapper map = _trap]) { if (err.ref.code == 0) return; final error = _readError(err, map); _errorClear(err); @@ -380,6 +492,27 @@ final class _BufferWriter { _buf.setRange(_len, _len + v.length, v); _len += v.length; } + + void writeOptional(T? v, void Function(T) write) { + writeFlag(v != null); + if (v != null) write(v); + } + + void writeList(List v, void Function(T) write) { + writeLength(v.length); + for (final e in v) { + write(e); + } + } + + void writeMap( + Map v, void Function(K) writeKey, void Function(V) writeValue) { + writeLength(v.length); + for (final e in v.entries) { + writeKey(e.key); + writeValue(e.value); + } + } } final class _BufferReader { @@ -443,6 +576,24 @@ final class _BufferReader { return Uint8List.fromList(Uint8List.sublistView(_buf, at, at + n)); } + T? readOptional(T Function() read) => + readFlag() ? read() : null; + + // Collections grow as they decode, so a corrupt count can't force a huge + // allocation up front. + List readList(T Function() read) => + [for (var i = readLength(); i > 0; i--) read()]; + + Map readMap(K Function() readKey, V Function() readValue) { + final map = {}; + for (var i = readLength(); i > 0; i--) { + final key = readKey(); + if (map.containsKey(key)) _bufferError('duplicate map key $key'); + map[key] = readValue(); + } + return map; + } + void expectEnd() { if (_pos != _buf.length) _bufferError('trailing bytes'); } @@ -463,6 +614,36 @@ T _decode(Uint8List bytes, T Function(_BufferReader) read) { return value; } +// Records and rich enums compare by value: lists, maps, and byte arrays +// element by element, and everything else with `==` (interface wrappers by +// identity). + +bool _deepEquals(Object? a, Object? b) { + if (identical(a, b)) return true; + if (a is List && b is List) { + if (a.length != b.length) return false; + for (var i = 0; i < a.length; i++) { + if (!_deepEquals(a[i], b[i])) return false; + } + return true; + } + if (a is Map && b is Map) { + if (a.length != b.length) return false; + for (final key in a.keys) { + if (!b.containsKey(key) || !_deepEquals(a[key], b[key])) return false; + } + return true; + } + return a == b; +} + +int _deepHash(Object? v) => switch (v) { + List() => Object.hashAll(v.map(_deepHash)), + Map() => Object.hashAllUnordered( + v.entries.map((e) => Object.hash(_deepHash(e.key), _deepHash(e.value)))), + _ => v.hashCode, + }; + // ── Async completion ── // An async call completes through a `NativeCallable.listener` the producer // invokes exactly once, from any thread. Everything the completion carries is @@ -473,8 +654,8 @@ final _errorFree = _lib.lookupFunction), void Function(Pointer<_Error>)>( 'kitchen_sink_error_free'); -/// Builds the exception a completion reports and frees its boxed error. -NativeException _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error a completion reports and frees its boxed error. +Object _takeAsyncError(Pointer<_Error> err, _ErrorMapper map) { final error = _readError(err, map); _errorFree(err); return error; @@ -553,11 +734,14 @@ final class _NativeCancel { // // Methods that return a value are `NativeCallable.isolateLocal` trampolines: // they run synchronously, so the producer must call them on this isolate's -// thread while a call from Dart is in progress. Void methods and `free` are -// `NativeCallable.isolateGroupBound` forwarders, which may run on any thread: -// they copy their arguments into a message for this isolate (borrowed bytes -// are copied before the forwarder returns) and the implementation runs later -// on the event loop, in the zone that passed the callback. +// thread while a call from Dart is in progress (the VM aborts the process +// otherwise, and an entry point that may run on any thread can't re-enter +// the isolate to run the method). Void methods and `free` are +// `NativeCallable.isolateGroupBound` forwarders, which may run on any +// thread: they copy their arguments into a message for this isolate +// (borrowed bytes are copied before the forwarder returns) and the +// implementation runs later on the event loop, in the zone that passed the +// callback. /// Where forwarders send messages: the allocator they build a message with /// and the isolate port that receives it. Forwarders run outside the @@ -765,22 +949,58 @@ final _errorSet = _lib.lookupFunction< Void Function(Pointer<_Error>, Int32, Pointer), void Function(Pointer<_Error>, int, Pointer)>('kitchen_sink_error_set'); -/// Reports an implementation's exception to the producer through -/// `kitchen_sink_error_set` with the foreign code (-4). Never throws: it runs -/// in the catch path of a trampoline the exception must not unwind through. -void _foreignError(Pointer<_Error> outErr, Object error) { +final _errorSetPayload = _lib.lookupFunction< + Void Function(Pointer<_Error>, Pointer, Size), + void Function( + Pointer<_Error>, Pointer, int)>('kitchen_sink_error_set_payload'); + +final _alloc = _lib.lookupFunction Function(Size), + Pointer Function(int)>('kitchen_sink_alloc', isLeaf: true); + +/// Reports a failed callback method to the producer through [outErr]: +/// `kitchen_sink_error_set` copies [message], and `kitchen_sink_error_set_payload` +/// copies a domain code's encoded fields. Never throws: it runs in the catch +/// path of a trampoline an exception must not unwind through. +void _failCallback( + Pointer<_Error> outErr, int code, String message, Uint8List? payload) { if (outErr == nullptr) return; - Pointer message = nullptr; + final text = message.toNativeUtf8(allocator: calloc); + try { + _errorSet(outErr, code, text); + } finally { + calloc.free(text); + } + if (payload == null || payload.isEmpty) return; + final bytes = calloc(payload.length); try { - message = error.toString().toNativeUtf8(allocator: calloc); - _errorSet(outErr, NativeException.foreignCode, message); + bytes.asTypedList(payload.length).setAll(0, payload); + _errorSetPayload(outErr, bytes, payload.length); + } finally { + calloc.free(bytes); + } +} + +/// Reports an implementation's exception that isn't a domain error of the +/// method with the foreign code (-4) and the exception's text. +void _foreignError(Pointer<_Error> outErr, Object error) { + try { + _failCallback(outErr, NativeException.foreignCode, '$error', null); } catch (_) { _errorSet(outErr, NativeException.foreignCode, nullptr); - } finally { - if (message != nullptr) calloc.free(message); } } +/// Hands [bytes] to the producer through a method's `out_ptr`/`out_len` +/// slots as a run allocated with `kitchen_sink_alloc`, which the producer +/// adopts and frees. An empty run is null with length 0. +void _handOver( + List bytes, Pointer> outPtr, Pointer outLen) { + final run = bytes.isEmpty ? nullptr : _alloc(bytes.length); + if (run != nullptr) run.asTypedList(bytes.length).setAll(0, bytes); + outPtr.value = run; + outLen.value = bytes.length; +} + // ── Iterators ── /// Anchors one live native iterator for its GC-finalizer backstop. A @@ -793,13 +1013,74 @@ final class _IteratorAnchor implements Finalizable {} /// The `out_item` slot iterator steps write, wide enough for any element. final Pointer _outItem = calloc(); +void _packListOfGadget(_BufferWriter w, List v) => + w.writeList(v, (e) => w.writeU64(e._cloneRef().address)); + +List _unpackListOfGadget(_BufferReader r) => + r.readList(() => Gadget._(Pointer.fromAddress(r.readU64()))); + +void _packListOfItem(_BufferWriter w, List v) => + w.writeList(v, (e) => _packItem(w, e)); + +List _unpackListOfItem(_BufferReader r) => + r.readList(() => _unpackItem(r)); + +void _packListOfString(_BufferWriter w, List v) => + w.writeList(v, (e) => w.writeString(e)); + +List _unpackListOfString(_BufferReader r) => + r.readList(() => r.readString()); + +void _packMapOfStringToI32(_BufferWriter w, Map v) => + w.writeMap(v, (k) => w.writeString(k), (e) => w.writeI32(e)); + +Map _unpackMapOfStringToI32(_BufferReader r) => + r.readMap(() => r.readString(), () => r.readI32()); + +void _packMapOfStringToString(_BufferWriter w, Map v) => + w.writeMap(v, (k) => w.writeString(k), (e) => w.writeString(e)); + +Map _unpackMapOfStringToString(_BufferReader r) => + r.readMap(() => r.readString(), () => r.readString()); + +void _packOptionalOfGadget(_BufferWriter w, Gadget? v) => + w.writeOptional(v, (e) => w.writeU64(e._cloneRef().address)); + +Gadget? _unpackOptionalOfGadget(_BufferReader r) => + r.readOptional(() => Gadget._(Pointer.fromAddress(r.readU64()))); + +void _packOptionalOfI64(_BufferWriter w, int? v) => + w.writeOptional(v, (e) => w.writeI64(e)); + +int? _unpackOptionalOfI64(_BufferReader r) => + r.readOptional(() => r.readI64()); + +void _packOptionalOfItem(_BufferWriter w, Item? v) => + w.writeOptional(v, (e) => _packItem(w, e)); + +Item? _unpackOptionalOfItem(_BufferReader r) => + r.readOptional(() => _unpackItem(r)); + /// A reusable token referenced across modules -class Token { +final class Token { /// Creates a [Token]. Token({required this.id, required this.label}); final int id; final String label; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Token && + id == other.id && + label == other.label; + + @override + int get hashCode => Object.hashAll([Token, id, label]); + + @override + String toString() => 'Token(id: $id, label: $label)'; } void _packToken(_BufferWriter w, Token v) { @@ -808,9 +1089,9 @@ void _packToken(_BufferWriter w, Token v) { } Token _unpackToken(_BufferReader r) => Token( - id: r.readI64(), - label: r.readString(), -); + id: r.readI64(), + label: r.readString(), + ); final _kitchenSinkSharedPing = _lib.lookupFunction< Pointer Function(Pointer, Pointer<_Error>), @@ -823,19 +1104,22 @@ String ping() { return _takeString(_result, _outLen.value); } -/// The `KitchenErrors` error domain of module `kitchen`. -class KitchenErrorsException extends NativeException { +/// The `KitchenErrors` error domain of module `kitchen`: catch it to handle any of +/// its codes, or a subclass for one code. +sealed class KitchenErrorsException extends NativeException { /// Creates a domain error carrying [code] and [message]. KitchenErrorsException(super.code, super.message); } /// Item not found -class NotFoundException extends KitchenErrorsException { +final class NotFoundException extends KitchenErrorsException { + /// Creates a [NotFoundException] (code 1). NotFoundException([String message = 'Item not found']) : super(1, message); } /// Invalid input -class InvalidInputException extends KitchenErrorsException { +final class InvalidInputException extends KitchenErrorsException { + /// Creates an [InvalidInputException] (code 2). InvalidInputException([String message = 'Invalid input']) : super(2, message); } @@ -846,7 +1130,17 @@ NativeException _mapKitchenErrorsException(int code, String message, Uint8List p case 2: return InvalidInputException(message); default: - return _runtimeError(code, message, payload); + return _runtimeException(code, message, payload); + } +} + +/// The value buffer of [e]'s fields, or null for a code without any. +Uint8List? _payloadOfKitchenErrorsException(KitchenErrorsException e) { + switch (e) { + case NotFoundException(): + return null; + case InvalidInputException(): + return null; } } @@ -871,7 +1165,7 @@ enum Priority { } /// Kitchen sink struct exercising every field feature -class Item { +final class Item { /// Creates an [Item]. Item({required this.id, required this.name, required this.count, required this.enabled, required this.ratio, required this.nick, required this.payload, required this.tags, required this.attrs, this.parent, required this.token, required this.priority, this.gadget}); @@ -895,6 +1189,30 @@ class Item { final Priority priority; /// An optional object inside a record final Gadget? gadget; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Item && + id == other.id && + name == other.name && + count == other.count && + enabled == other.enabled && + ratio == other.ratio && + nick == other.nick && + _deepEquals(payload, other.payload) && + _deepEquals(tags, other.tags) && + _deepEquals(attrs, other.attrs) && + parent == other.parent && + token == other.token && + priority == other.priority && + gadget == other.gadget; + + @override + int get hashCode => Object.hashAll([Item, id, name, count, enabled, ratio, nick, _deepHash(payload), _deepHash(tags), _deepHash(attrs), parent, token, priority, gadget]); + + @override + String toString() => 'Item(id: $id, name: $name, count: $count, enabled: $enabled, ratio: $ratio, nick: $nick, payload: $payload, tags: $tags, attrs: $attrs, parent: $parent, token: $token, priority: $priority, gadget: $gadget)'; } void _packItem(_BufferWriter w, Item v) { @@ -905,46 +1223,29 @@ void _packItem(_BufferWriter w, Item v) { w.writeF64(v.ratio); w.writeU32(v.nick); w.writeBytes(v.payload); - final _t0 = v.tags; - w.writeLength(_t0.length); - for (final _t1 in _t0) { - w.writeString(_t1); - } - final _t2 = v.attrs; - w.writeLength(_t2.length); - for (final _t3 in _t2.entries) { - w.writeString(_t3.key); - w.writeString(_t3.value); - } - final _t4 = v.parent; - w.writeFlag(_t4 != null); - if (_t4 != null) { - w.writeI64(_t4); - } + _packListOfString(w, v.tags); + _packMapOfStringToString(w, v.attrs); + _packOptionalOfI64(w, v.parent); _packToken(w, v.token); w.writeI32(v.priority.value); - final _t5 = v.gadget; - w.writeFlag(_t5 != null); - if (_t5 != null) { - w.writeU64(_t5._cloneRef().address); - } + _packOptionalOfGadget(w, v.gadget); } Item _unpackItem(_BufferReader r) => Item( - id: r.readI64(), - name: r.readString(), - count: r.readI32(), - enabled: r.readBool(), - ratio: r.readF64(), - nick: r.readU32(), - payload: r.readBytes(), - tags: [for (var i = r.readLength(); i > 0; i--) r.readString()], - attrs: {for (var i = r.readLength(); i > 0; i--) r.readString(): r.readString()}, - parent: (r.readFlag() ? r.readI64() : null), - token: _unpackToken(r), - priority: Priority.fromValue(r.readI32()), - gadget: (r.readFlag() ? Gadget._(Pointer.fromAddress(r.readU64())) : null), -); + id: r.readI64(), + name: r.readString(), + count: r.readI32(), + enabled: r.readBool(), + ratio: r.readF64(), + nick: r.readU32(), + payload: r.readBytes(), + tags: _unpackListOfString(r), + attrs: _unpackMapOfStringToString(r), + parent: _unpackOptionalOfI64(r), + token: _unpackToken(r), + priority: Priority.fromValue(r.readI32()), + gadget: _unpackOptionalOfGadget(r), + ); /// Consumer-implemented observer of ready items /// @@ -954,27 +1255,50 @@ Item _unpackItem(_BufferReader r) => Item( /// /// Methods that return a value run synchronously and must be called by /// the producer on this isolate's thread during a call from Dart; a -/// thrown exception fails that call with [NativeException.foreignCode]. -/// Void methods may be called from any thread: they're delivered -/// asynchronously on this isolate's event loop, in the zone that passed -/// the instance, where a thrown exception is an uncaught error of that -/// zone. Object arguments are owned by the implementation. +/// thrown exception fails that call with [NativeException.foreignCode] +/// (or, from a method that throws, the domain exception's code and +/// fields). Void methods may be called from any thread: they're +/// delivered asynchronously on this isolate's event loop, in the zone +/// that passed the instance, where a thrown exception is an uncaught +/// error of that zone. Object arguments are owned by the +/// implementation. abstract class ReadyListener { /// Fires when an item is ready void onReady(int code, String msg); /// Receives the item itself and says whether to keep listening bool onItem(Item item, Gadget gadget); + + /// A display label, failing with a kitchen error + /// + /// Throw a [KitchenErrorsException] to fail the producer's call with it. + String label(); + + /// The most recent item the listener kept, if any + Item? latest(); + + /// The listener's favorite gadget + Gadget favorite(); } typedef _KitchenSinkKitchenReadyListenerOnReady = Void Function(Pointer, Int32, Pointer, Size, Pointer<_Error>); typedef _KitchenSinkKitchenReadyListenerOnItem = Bool Function(Pointer, Pointer, Size, Pointer, Pointer<_Error>); +typedef _KitchenSinkKitchenReadyListenerLabel = Void Function(Pointer, Pointer>, Pointer, Pointer<_Error>); +typedef _KitchenSinkKitchenReadyListenerLatest = Void Function(Pointer, Pointer>, Pointer, Pointer<_Error>); +typedef _KitchenSinkKitchenReadyListenerFavorite = Pointer Function(Pointer, Pointer<_Error>); // The C `kitchen_sink_kitchen_ReadyListener_vtable` layout. final class _ReadyListenerVtable extends Struct { + @Uint32() + external int size; + @Uint32() + external int flags; + external Pointer)>> free; external Pointer> onReady; external Pointer> onItem; - external Pointer)>> free; + external Pointer> label; + external Pointer> latest; + external Pointer> favorite; } // Vtable entry `kitchen_sink_kitchen_ReadyListener_vtable.on_ready`: forwarded to the isolate. @@ -989,20 +1313,54 @@ void _readyListenerOnReady(Pointer ctx, int code, Pointer msgPtr, i bool _readyListenerOnItem(Pointer ctx, Pointer itemPtr, int itemLen, Pointer gadget, Pointer<_Error> outErr) { try { final _gadget = Gadget._(gadget); - return (_callbackTarget(ctx) as ReadyListener).onItem(_decode(_copyBytes(itemPtr, itemLen), _unpackItem), _gadget); + final _item = _decode(_copyBytes(itemPtr, itemLen), _unpackItem); + return (_callbackTarget(ctx) as ReadyListener).onItem(_item, _gadget); + } catch (e) { + _foreignError(outErr, e); + } + return false; +} + +// Vtable entry `kitchen_sink_kitchen_ReadyListener_vtable.label`. +void _readyListenerLabel(Pointer ctx, Pointer> outPtr, Pointer outLen, Pointer<_Error> outErr) { + try { + final result = (_callbackTarget(ctx) as ReadyListener).label(); + _handOver(utf8.encode(result), outPtr, outLen); + } on KitchenErrorsException catch (e) { + _failCallback(outErr, e.code, e.message, _payloadOfKitchenErrorsException(e)); } catch (e) { _foreignError(outErr, e); - return false; } } +// Vtable entry `kitchen_sink_kitchen_ReadyListener_vtable.latest`. +void _readyListenerLatest(Pointer ctx, Pointer> outPtr, Pointer outLen, Pointer<_Error> outErr) { + try { + final result = (_callbackTarget(ctx) as ReadyListener).latest(); + _handOver(_encode(result, _packOptionalOfItem), outPtr, outLen); + } catch (e) { + _foreignError(outErr, e); + } +} + +// Vtable entry `kitchen_sink_kitchen_ReadyListener_vtable.favorite`. +Pointer _readyListenerFavorite(Pointer ctx, Pointer<_Error> outErr) { + try { + final result = (_callbackTarget(ctx) as ReadyListener).favorite(); + return result._cloneRef(); + } catch (e) { + _foreignError(outErr, e); + } + return nullptr; +} + void _readyListenerDispatch(Object impl, int method, List message) { final target = impl as ReadyListener; switch (method) { case 0: { - final _a2 = message[2]! as int; - final _a3 = utf8.decode(message[3]! as Uint8List); - target.onReady(_a2, _a3); + final a2 = message[2]! as int; + final a3 = utf8.decode(message[3]! as Uint8List); + target.onReady(a2, a3); } } } @@ -1010,9 +1368,14 @@ void _readyListenerDispatch(Object impl, int method, List message) { final Pointer<_ReadyListenerVtable> _readyListenerVtable = () { final vtable = calloc<_ReadyListenerVtable>(); vtable.ref + ..size = sizeOf<_ReadyListenerVtable>() + ..flags = 0 + ..free = _callbackFree ..onReady = _pin(NativeCallable<_KitchenSinkKitchenReadyListenerOnReady>.isolateGroupBound(_readyListenerOnReady)) ..onItem = _pin(NativeCallable<_KitchenSinkKitchenReadyListenerOnItem>.isolateLocal(_readyListenerOnItem, exceptionalReturn: false)) - ..free = _callbackFree; + ..label = _pin(NativeCallable<_KitchenSinkKitchenReadyListenerLabel>.isolateLocal(_readyListenerLabel)) + ..latest = _pin(NativeCallable<_KitchenSinkKitchenReadyListenerLatest>.isolateLocal(_readyListenerLatest)) + ..favorite = _pin(NativeCallable<_KitchenSinkKitchenReadyListenerFavorite>.isolateLocal(_readyListenerFavorite)); return vtable; }(); @@ -1192,7 +1555,19 @@ final _kitchenSinkKitchenSubscribe = _lib.lookupFunction< /// Register a listener, returning how many are attached int subscribe(ReadyListener listener) { final _listenerCtx = _registerCallback(listener, _readyListenerDispatch); - final _result = _kitchenSinkKitchenSubscribe(_listenerCtx, _readyListenerVtable.cast(), _err); + final _result = _kitchenSinkKitchenSubscribe(_listenerCtx, _readyListenerVtable.cast(), _err); + _check(_err); + return _result; +} + +final _kitchenSinkKitchenMaybeSubscribe = _lib.lookupFunction< + Bool Function(Pointer, Pointer, Pointer<_Error>), + bool Function(Pointer, Pointer, Pointer<_Error>)>('kitchen_sink_kitchen_maybe_subscribe'); + +/// Register a listener if one is given +bool maybeSubscribe(ReadyListener? listener) { + final _listenerCtx = listener == null ? nullptr : _registerCallback(listener, _readyListenerDispatch); + final _result = _kitchenSinkKitchenMaybeSubscribe(_listenerCtx, listener == null ? nullptr : _readyListenerVtable.cast(), _err); _check(_err); return _result; } @@ -1225,7 +1600,7 @@ final _kitchenSinkKitchenGadgets = _lib.lookupFunction< List gadgets() { final _result = _kitchenSinkKitchenGadgets(_outLen, _err); _check(_err); - return _decode(_takeBytes(_result, _outLen.value), (r) => [for (var i = r.readLength(); i > 0; i--) Gadget._(Pointer.fromAddress(r.readU64()))]); + return _decode(_takeBytes(_result, _outLen.value), _unpackListOfGadget); } final _kitchenSinkKitchenStreamGadgets = _lib.lookupFunction< @@ -1280,7 +1655,7 @@ final _kitchenSinkKitchenMaybeItem = _lib.lookupFunction< Item? maybeItem(int id) { final _result = _kitchenSinkKitchenMaybeItem(id, _outLen, _err); _check(_err, _mapKitchenErrorsException); - return _decode(_takeBytes(_result, _outLen.value), (r) => (r.readFlag() ? _unpackItem(r) : null)); + return _decode(_takeBytes(_result, _outLen.value), _unpackOptionalOfItem); } final _kitchenSinkKitchenListItems = _lib.lookupFunction< @@ -1291,7 +1666,7 @@ final _kitchenSinkKitchenListItems = _lib.lookupFunction< List listItems() { final _result = _kitchenSinkKitchenListItems(_outLen, _err); _check(_err); - return _decode(_takeBytes(_result, _outLen.value), (r) => [for (var i = r.readLength(); i > 0; i--) _unpackItem(r)]); + return _decode(_takeBytes(_result, _outLen.value), _unpackListOfItem); } final _kitchenSinkKitchenGetAttrs = _lib.lookupFunction< @@ -1302,7 +1677,7 @@ final _kitchenSinkKitchenGetAttrs = _lib.lookupFunction< Map getAttrs() { final _result = _kitchenSinkKitchenGetAttrs(_outLen, _err); _check(_err); - return _decode(_takeBytes(_result, _outLen.value), (r) => {for (var i = r.readLength(); i > 0; i--) r.readString(): r.readI32()}); + return _decode(_takeBytes(_result, _outLen.value), _unpackMapOfStringToI32); } final _kitchenSinkKitchenStreamItems = _lib.lookupFunction< @@ -1367,11 +1742,11 @@ Future doAsync(String input) { final _completer = Completer(); late final NativeCallable<_KitchenSinkKitchenDoAsyncCallback> _callback; _callback = NativeCallable<_KitchenSinkKitchenDoAsyncCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); - _completer.complete(_takeString(_value, _valueLen)); + if (error != nullptr) throw _takeAsyncError(error, _trap); + _completer.complete(_takeString(value, valueLen)); } catch (e, s) { _completer.completeError(e, s); } @@ -1404,12 +1779,12 @@ Future doCancellable(String input, {CancelToken? cancelToken}) { final _cancel = _NativeCancel.bind(cancelToken); late final NativeCallable<_KitchenSinkKitchenDoCancellableCallback> _callback; _callback = NativeCallable<_KitchenSinkKitchenDoCancellableCallback>.listener(( - Pointer _, Pointer<_Error> _error, Pointer _value, int _valueLen) { + Pointer _, Pointer<_Error> error, Pointer value, int valueLen) { _callback.close(); _cancel?.release(); try { - if (_error != nullptr) throw _takeAsyncError(_error, _runtimeError); - _completer.complete(_takeString(_value, _valueLen)); + if (error != nullptr) throw _takeAsyncError(error, _trap); + _completer.complete(_takeString(value, valueLen)); } catch (e, s) { _completer.completeError(e, s); } diff --git a/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__pubspec_yaml.snap b/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__pubspec_yaml.snap index 6d542a54..6fd0da3f 100644 --- a/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__pubspec_yaml.snap +++ b/crates/weaveffi-cli/tests/snapshots/dart_kitchen_sink__pubspec_yaml.snap @@ -1,16 +1,16 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate name: kitchen_sink version: 0.1.0 description: 'Native bindings for kitchen_sink' environment: - sdk: '>=3.12.0 <4.0.0' + sdk: '>=3.10.0 <4.0.0' dependencies: ffi: ^2.1.0 diff --git a/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__README_md.snap deleted file mode 100644 index bf98166d..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__README_md.snap +++ /dev/null @@ -1,39 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# nested_modules (Dart) - -`dart:ffi` bindings for the `nested_modules` native library. - -## Usage - -Add this package as a dependency (a `path:` dependency works) and import it: - -```dart -import 'package:nested_modules/nested_modules.dart'; -``` - -The library loads `libnested_modules.dylib` (macOS), `libnested_modules.so` (Linux and Android), or -`nested_modules.dll` (Windows) from the platform's search path on first use. Set -`NESTED_MODULES_LIBRARY` to a full path to load a specific build. On iOS, and as a macOS -fallback, the bindings look the symbols up in the running executable. - -## Objects and callbacks - -Each interface wrapper holds one native reference: call `dispose()` when -you're done, or let the garbage collector's finalizer release it. - -Callback-interface methods that return a value run synchronously on the -isolate's thread; void methods may be called from any thread and are -delivered on the event loop. - -## Requirements - -- Dart SDK `>=3.12.0 <4.0.0` -- The `ffi` package - - diff --git a/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__lib_nested_modules_dart.snap b/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__lib_nested_modules_dart.snap index e076f8be..9cf72fa1 100644 --- a/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__lib_nested_modules_dart.snap +++ b/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__lib_nested_modules_dart.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // ignore_for_file: camel_case_types, non_constant_identifier_names, unused_element @@ -16,45 +16,69 @@ import 'package:ffi/ffi.dart'; // ── Library loading ── // The native library opens on first use and is verified before any other -// lookup: its C ABI revision and every module's contract checksum must match -// the values these bindings were generated against. +// lookup: it must implement the C ABI revision these bindings were generated +// against and carry every declaration they were generated with, unchanged. /// The native library every binding in this file calls. final DynamicLibrary _lib = _verifyLibrary(_openLibrary()); +/// The native library couldn't be loaded, or it doesn't match these +/// bindings: a different C ABI revision, or a declaration that's missing or +/// changed. Regenerate the bindings from the library you ship. +final class NativeLibraryError extends Error { + /// Creates a load failure described by [message]. + NativeLibraryError(this.message); + + /// What failed, naming the library or declaration. + final String message; + + @override + String toString() => 'NativeLibraryError: $message'; +} + +/// The platform file name of the library on the system search path. +String get _libraryFile => Platform.isMacOS + ? 'libnested_modules.dylib' + : Platform.isWindows + ? 'nested_modules.dll' + : 'libnested_modules.so'; + DynamicLibrary _openLibrary() { // An explicit path wins, so a caller can point at any build artifact. final override = Platform.environment['NESTED_MODULES_LIBRARY']; if (override != null && override.isNotEmpty) { - return DynamicLibrary.open(override); + return _open(override); } // iOS links native code into the app binary. if (Platform.isIOS) return DynamicLibrary.process(); - final candidates = Platform.isMacOS - ? const ['libnested_modules.dylib'] - : Platform.isWindows - ? const ['nested_modules.dll'] - : const ['libnested_modules.so']; - ArgumentError? failure; - for (final candidate in candidates) { - try { - return DynamicLibrary.open(candidate); - } on ArgumentError catch (e) { - failure ??= e; - } + try { + return DynamicLibrary.open(_libraryFile); + } on ArgumentError catch (e) { + // A macOS app may link the library into its executable instead. + if (Platform.isMacOS) return DynamicLibrary.process(); + throw NativeLibraryError("can't open '$_libraryFile' ($e); set " + 'NESTED_MODULES_LIBRARY to its path'); } - // A macOS app may link the library into its executable instead. - if (Platform.isMacOS) return DynamicLibrary.process(); - throw failure!; } -/// The C ABI revision these bindings were generated against. -const int _abiVersion = 3; +DynamicLibrary _open(String path) { + try { + return DynamicLibrary.open(path); + } on ArgumentError catch (e) { + throw NativeLibraryError("can't open '$path': $e"); + } +} -/// Each top-level module's name, checksum symbol, and expected checksum. -const List<(String, String, int)> _contracts = <(String, String, int)>[ - ('graphics', 'nested_modules_graphics_checksum', 0x44d31b5b10fdb90d), -]; +/// The C ABI revision these bindings were generated against. +const int _abiVersion = 4; + +/// One `nested_modules_contract_entry` of a module's contract table. +final class _ContractEntry extends Struct { + @Uint64() + external int id; + @Uint64() + external int hash; +} DynamicLibrary _verifyLibrary(DynamicLibrary lib) { final int Function() abiVersion; @@ -62,33 +86,71 @@ DynamicLibrary _verifyLibrary(DynamicLibrary lib) { abiVersion = lib.lookupFunction( 'nested_modules_abi_version'); } on ArgumentError { - throw StateError( - "the native library 'nested_modules' isn't loaded; set NESTED_MODULES_LIBRARY to " - 'its path'); + throw NativeLibraryError( + "the native library 'nested_modules' isn't loaded; set NESTED_MODULES_LIBRARY " + 'to its path'); } final found = abiVersion(); if (found != _abiVersion) { - throw StateError('nested_modules: these bindings expect C ABI revision ' - '$_abiVersion, but the loaded library reports revision $found'); - } - for (final (module, symbol, expected) in _contracts) { - final int actual; - try { - actual = lib.lookupFunction(symbol)(); - } on ArgumentError { - throw StateError( - "nested_modules: the loaded library doesn't export module '$module'"); + throw NativeLibraryError('nested_modules: these bindings expect C ABI revision ' + '$_abiVersion, but the loaded library implements revision $found'); + } + for (final (symbol, expected) in _contracts) { + _checkContract(lib, symbol, expected); + } + return lib; +} + +/// Checks that every `(id, hash, path)` entry in [expected] is in the table +/// [symbol] returns with an equal hash. Entries only the library has are +/// fine: they're declarations added after these bindings were generated. +void _checkContract( + DynamicLibrary lib, String symbol, List<(int, int, String)> expected) { + final Pointer<_ContractEntry> Function(Pointer) table; + try { + table = lib.lookupFunction Function(Pointer), + Pointer<_ContractEntry> Function(Pointer)>(symbol); + } on ArgumentError { + throw NativeLibraryError( + "nested_modules: the loaded library doesn't export '$symbol'"); + } + final hashes = {}; + final len = calloc(); + try { + final entries = table(len); + for (var i = 0; i < len.value; i++) { + hashes[entries[i].id] = entries[i].hash; } - if (actual != expected) { - throw StateError("nested_modules: module '$module' doesn't match the loaded " - 'library (contract checksum ${_hex(actual)}, expected ' - '${_hex(expected)}); regenerate the bindings'); + } finally { + calloc.free(len); + } + for (final (id, hash, path) in expected) { + final found = hashes[id]; + if (found == null) { + throw NativeLibraryError('nested_modules: $path is missing from the library'); + } + if (found != hash) { + throw NativeLibraryError( + 'nested_modules: $path changed since these bindings were generated'); } } - return lib; } -String _hex(int v) => '0x${v.toUnsigned(64).toRadixString(16).padLeft(16, '0')}'; +/// Each top-level module's contract-table symbol and the declarations +/// these bindings were generated with: (id, signature hash, path). +const List<(String, List<(int, int, String)>)> _contracts = [ + ('nested_modules_graphics_contract', [ + (0x2c29eef9327a6bd4, 0x12983d5271651389, 'graphics.Unit'), + (0x304088bbc28f0015, 0x97e6e809913be6b6, 'graphics.new_canvas'), + (0x49f8837532df9ac6, 0x32e5b27e81ee0957, 'graphics.Canvas'), + (0x55717fa027336b47, 0xb2bd3a6bdcce48c8, 'graphics.shapes.describe'), + (0x7626b91b8f1f53cd, 0x867bbf7cf0f41c74, 'graphics.shapes.geometry.compute'), + (0xb3efcb50d9a61ab7, 0x3cb0207dfad2a149, 'graphics.GraphicsError'), + (0xc62b16a9591b744b, 0x7895610c8487eb1b, 'graphics.shapes.Shape'), + (0xe13001f4983ccd35, 0x16e7de070b4a4cfc, 'graphics.shapes.geometry.Detail'), + (0xf3f658bcfc5dba62, 0x2665ccb47356e746, 'graphics.shapes.make_shape'), + ]), +]; // ── Errors and memory ── @@ -104,10 +166,8 @@ final class _Error extends Struct { external int payloadLen; } -final _errorClear = - _lib.lookupFunction), void Function(Pointer<_Error>)>( - 'nested_modules_error_clear', - isLeaf: true); +final _errorClear = _lib.lookupFunction), + void Function(Pointer<_Error>)>('nested_modules_error_clear', isLeaf: true); final _freeBytes = _lib.lookupFunction, Size), void Function(Pointer, int)>('nested_modules_free_bytes', isLeaf: true); @@ -118,9 +178,9 @@ final _freeBytes = _lib.lookupFunction, Size), final Pointer<_Error> _err = calloc<_Error>(); final Pointer _outLen = calloc(); -/// A failure the native library reported. Domain errors extend this class -/// with a positive [code]; the negative runtime codes are the constants -/// below. +/// A failure the native library reported to a function declared to throw. +/// Domain errors extend this class with a positive [code]; a negative code is +/// one of the runtime constants below. class NativeException implements Exception { /// Creates an exception carrying [code] and [message]. NativeException(this.code, this.message); @@ -131,10 +191,10 @@ class NativeException implements Exception { /// The producer panicked; [message] carries the panic text. static const int panicCode = -2; - /// An argument couldn't be lifted by the producer. + /// An argument or a callback's return couldn't be lifted by the producer. static const int marshalCode = -3; - /// A callback implementation threw; [message] carries its text. + /// A callback implementation failed; [message] carries its text. static const int foreignCode = -4; /// The call was cancelled (see [CancelledException]). @@ -158,24 +218,51 @@ class CancelledException extends NativeException { : super(NativeException.cancelledCode, message); } -typedef _ErrorMapper = NativeException Function( +/// A function that isn't declared to throw failed anyway. That's a bug in +/// the native library (a panic, an argument it couldn't lift, a callback +/// failure it let through), so it's an [Error], not an [Exception]. +final class NativeError extends Error { + /// Creates the error a failed call reported. + NativeError(this.code, this.message); + + /// The runtime code the producer reported (see [NativeException]). + final int code; + + /// The producer's message. + final String message; + + @override + String toString() => 'NativeError($code): $message'; +} + +/// Maps a reported code, message, and payload onto the object to throw. +typedef _ErrorMapper = Object Function( int code, String message, Uint8List payload); -/// Maps a runtime failure (or an undeclared code) onto its exception. -NativeException _runtimeError(int code, String message, Uint8List payload) => +/// The mapper of a function that can't fail: every code but cancellation +/// is a [NativeError]. +Object _trap(int code, String message, Uint8List payload) => + code == NativeException.cancelledCode + ? CancelledException(message) + : NativeError(code, message); + +/// Maps a runtime code (or one the domain doesn't declare) onto its +/// exception, for a function declared to throw. +NativeException _runtimeException(int code, String message, Uint8List payload) => code == NativeException.cancelledCode ? CancelledException(message) : NativeException(code, message); -/// Builds the exception [err] reports, using [map] for domain codes. -NativeException _readError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error [err] reports, using [map]. +Object _readError(Pointer<_Error> err, _ErrorMapper map) { final e = err.ref; final message = e.message == nullptr ? '' : e.message.toDartString(); return map(e.code, message, _copyBytes(e.payloadPtr, e.payloadLen)); } /// Throws the error a synchronous call left in [err], clearing the slot. -void _check(Pointer<_Error> err, [_ErrorMapper map = _runtimeError]) { +/// [map] is the domain's mapper for a function declared to throw. +void _check(Pointer<_Error> err, [_ErrorMapper map = _trap]) { if (err.ref.code == 0) return; final error = _readError(err, map); _errorClear(err); @@ -311,6 +398,27 @@ final class _BufferWriter { _buf.setRange(_len, _len + v.length, v); _len += v.length; } + + void writeOptional(T? v, void Function(T) write) { + writeFlag(v != null); + if (v != null) write(v); + } + + void writeList(List v, void Function(T) write) { + writeLength(v.length); + for (final e in v) { + write(e); + } + } + + void writeMap( + Map v, void Function(K) writeKey, void Function(V) writeValue) { + writeLength(v.length); + for (final e in v.entries) { + writeKey(e.key); + writeValue(e.value); + } + } } final class _BufferReader { @@ -374,6 +482,24 @@ final class _BufferReader { return Uint8List.fromList(Uint8List.sublistView(_buf, at, at + n)); } + T? readOptional(T Function() read) => + readFlag() ? read() : null; + + // Collections grow as they decode, so a corrupt count can't force a huge + // allocation up front. + List readList(T Function() read) => + [for (var i = readLength(); i > 0; i--) read()]; + + Map readMap(K Function() readKey, V Function() readValue) { + final map = {}; + for (var i = readLength(); i > 0; i--) { + final key = readKey(); + if (map.containsKey(key)) _bufferError('duplicate map key $key'); + map[key] = readValue(); + } + return map; + } + void expectEnd() { if (_pos != _buf.length) _bufferError('trailing bytes'); } @@ -394,14 +520,46 @@ T _decode(Uint8List bytes, T Function(_BufferReader) read) { return value; } -/// The `GraphicsError` error domain of module `graphics`. -class GraphicsException extends NativeException { +// Records and rich enums compare by value: lists, maps, and byte arrays +// element by element, and everything else with `==` (interface wrappers by +// identity). + +bool _deepEquals(Object? a, Object? b) { + if (identical(a, b)) return true; + if (a is List && b is List) { + if (a.length != b.length) return false; + for (var i = 0; i < a.length; i++) { + if (!_deepEquals(a[i], b[i])) return false; + } + return true; + } + if (a is Map && b is Map) { + if (a.length != b.length) return false; + for (final key in a.keys) { + if (!b.containsKey(key) || !_deepEquals(a[key], b[key])) return false; + } + return true; + } + return a == b; +} + +int _deepHash(Object? v) => switch (v) { + List() => Object.hashAll(v.map(_deepHash)), + Map() => Object.hashAllUnordered( + v.entries.map((e) => Object.hash(_deepHash(e.key), _deepHash(e.value)))), + _ => v.hashCode, + }; + +/// The `GraphicsError` error domain of module `graphics`: catch it to handle any of +/// its codes, or a subclass for one code. +sealed class GraphicsException extends NativeException { /// Creates a domain error carrying [code] and [message]. GraphicsException(super.code, super.message); } /// coordinate out of bounds -class OutOfBoundsException extends GraphicsException { +final class OutOfBoundsException extends GraphicsException { + /// Creates an [OutOfBoundsException] (code 1). OutOfBoundsException([String message = 'coordinate out of bounds']) : super(1, message); } @@ -410,7 +568,7 @@ NativeException _mapGraphicsException(int code, String message, Uint8List payloa case 1: return OutOfBoundsException(message); default: - return _runtimeError(code, message, payload); + return _runtimeException(code, message, payload); } } @@ -427,17 +585,30 @@ enum Unit { /// The variant whose discriminant is [value]. static Unit fromValue(int value) => values.firstWhere( (e) => e.value == value, - orElse: () => throw ArgumentError.value(value, 'value', 'not a Unit discriminant')); + orElse: () => throw ArgumentError.value(value, 'value', 'not an Unit discriminant')); } /// Top-level struct that references a struct defined one level deeper. -class Canvas { +final class Canvas { /// Creates a [Canvas]. Canvas({required this.width, required this.root}); final int width; /// Reference down into a nested module final Shape root; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Canvas && + width == other.width && + root == other.root; + + @override + int get hashCode => Object.hashAll([Canvas, width, root]); + + @override + String toString() => 'Canvas(width: $width, root: $root)'; } void _packCanvas(_BufferWriter w, Canvas v) { @@ -446,9 +617,9 @@ void _packCanvas(_BufferWriter w, Canvas v) { } Canvas _unpackCanvas(_BufferReader r) => Canvas( - width: r.readI32(), - root: _unpackShape(r), -); + width: r.readI32(), + root: _unpackShape(r), + ); final _nestedModulesGraphicsNewCanvas = _lib.lookupFunction< Pointer Function(Int32, Pointer, Pointer<_Error>), @@ -462,7 +633,7 @@ Canvas newCanvas(int width) { } /// A struct one level deep that references both an ancestor enum and a deeper struct. -class Shape { +final class Shape { /// Creates a [Shape]. Shape({required this.sides, required this.unit, required this.detail}); @@ -471,6 +642,20 @@ class Shape { final Unit unit; /// Reference down into a deeper module final Detail detail; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Shape && + sides == other.sides && + unit == other.unit && + detail == other.detail; + + @override + int get hashCode => Object.hashAll([Shape, sides, unit, detail]); + + @override + String toString() => 'Shape(sides: $sides, unit: $unit, detail: $detail)'; } void _packShape(_BufferWriter w, Shape v) { @@ -480,10 +665,10 @@ void _packShape(_BufferWriter w, Shape v) { } Shape _unpackShape(_BufferReader r) => Shape( - sides: r.readI32(), - unit: Unit.fromValue(r.readI32()), - detail: _unpackDetail(r), -); + sides: r.readI32(), + unit: Unit.fromValue(r.readI32()), + detail: _unpackDetail(r), + ); final _nestedModulesGraphicsShapesMakeShape = _lib.lookupFunction< Pointer Function(Int32, Pointer, Pointer<_Error>), @@ -508,11 +693,23 @@ Canvas describe() { } /// A struct two levels deep. -class Detail { +final class Detail { /// Creates a [Detail]. Detail({required this.area}); final double area; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Detail && + area == other.area; + + @override + int get hashCode => Object.hashAll([Detail, area]); + + @override + String toString() => 'Detail(area: $area)'; } void _packDetail(_BufferWriter w, Detail v) { @@ -520,8 +717,8 @@ void _packDetail(_BufferWriter w, Detail v) { } Detail _unpackDetail(_BufferReader r) => Detail( - area: r.readF64(), -); + area: r.readF64(), + ); final _nestedModulesGraphicsShapesGeometryCompute = _lib.lookupFunction< Pointer Function(Pointer, Size, Pointer, Pointer<_Error>), @@ -533,9 +730,7 @@ final _nestedModulesGraphicsShapesGeometryCompute = _lib.lookupFunction< Detail compute(Shape shape) { final _arena = Arena(); try { - final _shapeWriter = _BufferWriter(); - _packShape(_shapeWriter, shape); - final _shapeBytes = _shapeWriter.takeBytes(); + final _shapeBytes = _encode(shape, _packShape); final _result = _nestedModulesGraphicsShapesGeometryCompute(_stage(_arena, _shapeBytes), _shapeBytes.length, _outLen, _err); _check(_err, _mapGraphicsException); return _decode(_takeBytes(_result, _outLen.value), _unpackDetail); diff --git a/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__pubspec_yaml.snap b/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__pubspec_yaml.snap deleted file mode 100644 index 3c64734f..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_nested_modules__pubspec_yaml.snap +++ /dev/null @@ -1,18 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -name: nested_modules -version: 0.1.0 -description: 'Native bindings for nested_modules' - -environment: - sdk: '>=3.12.0 <4.0.0' - -dependencies: - ffi: ^2.1.0 - -# END pubspec.yaml diff --git a/crates/weaveffi-cli/tests/snapshots/dart_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dart_shapes__README_md.snap deleted file mode 100644 index 56579e29..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_shapes__README_md.snap +++ /dev/null @@ -1,39 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# shapes (Dart) - -`dart:ffi` bindings for the `shapes` native library. - -## Usage - -Add this package as a dependency (a `path:` dependency works) and import it: - -```dart -import 'package:shapes/shapes.dart'; -``` - -The library loads `libshapes.dylib` (macOS), `libshapes.so` (Linux and Android), or -`shapes.dll` (Windows) from the platform's search path on first use. Set -`SHAPES_LIBRARY` to a full path to load a specific build. On iOS, and as a macOS -fallback, the bindings look the symbols up in the running executable. - -## Objects and callbacks - -Each interface wrapper holds one native reference: call `dispose()` when -you're done, or let the garbage collector's finalizer release it. - -Callback-interface methods that return a value run synchronously on the -isolate's thread; void methods may be called from any thread and are -delivered on the event loop. - -## Requirements - -- Dart SDK `>=3.12.0 <4.0.0` -- The `ffi` package - - diff --git a/crates/weaveffi-cli/tests/snapshots/dart_shapes__lib_shapes_dart.snap b/crates/weaveffi-cli/tests/snapshots/dart_shapes__lib_shapes_dart.snap index 4009a59c..7fec7356 100644 --- a/crates/weaveffi-cli/tests/snapshots/dart_shapes__lib_shapes_dart.snap +++ b/crates/weaveffi-cli/tests/snapshots/dart_shapes__lib_shapes_dart.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // ignore_for_file: camel_case_types, non_constant_identifier_names, unused_element @@ -16,45 +16,69 @@ import 'package:ffi/ffi.dart'; // ── Library loading ── // The native library opens on first use and is verified before any other -// lookup: its C ABI revision and every module's contract checksum must match -// the values these bindings were generated against. +// lookup: it must implement the C ABI revision these bindings were generated +// against and carry every declaration they were generated with, unchanged. /// The native library every binding in this file calls. final DynamicLibrary _lib = _verifyLibrary(_openLibrary()); +/// The native library couldn't be loaded, or it doesn't match these +/// bindings: a different C ABI revision, or a declaration that's missing or +/// changed. Regenerate the bindings from the library you ship. +final class NativeLibraryError extends Error { + /// Creates a load failure described by [message]. + NativeLibraryError(this.message); + + /// What failed, naming the library or declaration. + final String message; + + @override + String toString() => 'NativeLibraryError: $message'; +} + +/// The platform file name of the library on the system search path. +String get _libraryFile => Platform.isMacOS + ? 'libshapes.dylib' + : Platform.isWindows + ? 'shapes.dll' + : 'libshapes.so'; + DynamicLibrary _openLibrary() { // An explicit path wins, so a caller can point at any build artifact. final override = Platform.environment['SHAPES_LIBRARY']; if (override != null && override.isNotEmpty) { - return DynamicLibrary.open(override); + return _open(override); } // iOS links native code into the app binary. if (Platform.isIOS) return DynamicLibrary.process(); - final candidates = Platform.isMacOS - ? const ['libshapes.dylib'] - : Platform.isWindows - ? const ['shapes.dll'] - : const ['libshapes.so']; - ArgumentError? failure; - for (final candidate in candidates) { - try { - return DynamicLibrary.open(candidate); - } on ArgumentError catch (e) { - failure ??= e; - } + try { + return DynamicLibrary.open(_libraryFile); + } on ArgumentError catch (e) { + // A macOS app may link the library into its executable instead. + if (Platform.isMacOS) return DynamicLibrary.process(); + throw NativeLibraryError("can't open '$_libraryFile' ($e); set " + 'SHAPES_LIBRARY to its path'); } - // A macOS app may link the library into its executable instead. - if (Platform.isMacOS) return DynamicLibrary.process(); - throw failure!; } -/// The C ABI revision these bindings were generated against. -const int _abiVersion = 3; +DynamicLibrary _open(String path) { + try { + return DynamicLibrary.open(path); + } on ArgumentError catch (e) { + throw NativeLibraryError("can't open '$path': $e"); + } +} -/// Each top-level module's name, checksum symbol, and expected checksum. -const List<(String, String, int)> _contracts = <(String, String, int)>[ - ('shapes', 'shapes_shapes_checksum', 0x06acf433ffa2a0f0), -]; +/// The C ABI revision these bindings were generated against. +const int _abiVersion = 4; + +/// One `shapes_contract_entry` of a module's contract table. +final class _ContractEntry extends Struct { + @Uint64() + external int id; + @Uint64() + external int hash; +} DynamicLibrary _verifyLibrary(DynamicLibrary lib) { final int Function() abiVersion; @@ -62,33 +86,69 @@ DynamicLibrary _verifyLibrary(DynamicLibrary lib) { abiVersion = lib.lookupFunction( 'shapes_abi_version'); } on ArgumentError { - throw StateError( - "the native library 'shapes' isn't loaded; set SHAPES_LIBRARY to " - 'its path'); + throw NativeLibraryError( + "the native library 'shapes' isn't loaded; set SHAPES_LIBRARY " + 'to its path'); } final found = abiVersion(); if (found != _abiVersion) { - throw StateError('shapes: these bindings expect C ABI revision ' - '$_abiVersion, but the loaded library reports revision $found'); - } - for (final (module, symbol, expected) in _contracts) { - final int actual; - try { - actual = lib.lookupFunction(symbol)(); - } on ArgumentError { - throw StateError( - "shapes: the loaded library doesn't export module '$module'"); + throw NativeLibraryError('shapes: these bindings expect C ABI revision ' + '$_abiVersion, but the loaded library implements revision $found'); + } + for (final (symbol, expected) in _contracts) { + _checkContract(lib, symbol, expected); + } + return lib; +} + +/// Checks that every `(id, hash, path)` entry in [expected] is in the table +/// [symbol] returns with an equal hash. Entries only the library has are +/// fine: they're declarations added after these bindings were generated. +void _checkContract( + DynamicLibrary lib, String symbol, List<(int, int, String)> expected) { + final Pointer<_ContractEntry> Function(Pointer) table; + try { + table = lib.lookupFunction Function(Pointer), + Pointer<_ContractEntry> Function(Pointer)>(symbol); + } on ArgumentError { + throw NativeLibraryError( + "shapes: the loaded library doesn't export '$symbol'"); + } + final hashes = {}; + final len = calloc(); + try { + final entries = table(len); + for (var i = 0; i < len.value; i++) { + hashes[entries[i].id] = entries[i].hash; } - if (actual != expected) { - throw StateError("shapes: module '$module' doesn't match the loaded " - 'library (contract checksum ${_hex(actual)}, expected ' - '${_hex(expected)}); regenerate the bindings'); + } finally { + calloc.free(len); + } + for (final (id, hash, path) in expected) { + final found = hashes[id]; + if (found == null) { + throw NativeLibraryError('shapes: $path is missing from the library'); + } + if (found != hash) { + throw NativeLibraryError( + 'shapes: $path changed since these bindings were generated'); } } - return lib; } -String _hex(int v) => '0x${v.toUnsigned(64).toRadixString(16).padLeft(16, '0')}'; +/// Each top-level module's contract-table symbol and the declarations +/// these bindings were generated with: (id, signature hash, path). +const List<(String, List<(int, int, String)>)> _contracts = [ + ('shapes_shapes_contract', [ + (0x0d7b3eec49243e7a, 0x5f232311808e1b1f, 'shapes.Channel'), + (0x183fa8198100860e, 0xe40c6993cb31d359, 'shapes.describe'), + (0x2d1395eb837b848e, 0x42202a0c12f3a16e, 'shapes.sum_bytes'), + (0x7f9a5b87ba09b86b, 0xc5b32fb0efbbaa17, 'shapes.scale'), + (0xc8edd779240fda17, 0x68e1be78fddb20e4, 'shapes.Numerics'), + (0xd3d3881b2fd18a62, 0xf24d0eb76a99d918, 'shapes.make_numerics'), + (0xf010844afff5ac3c, 0x50de28ae9b41ff15, 'shapes.Shape'), + ]), +]; // ── Errors and memory ── @@ -104,10 +164,8 @@ final class _Error extends Struct { external int payloadLen; } -final _errorClear = - _lib.lookupFunction), void Function(Pointer<_Error>)>( - 'shapes_error_clear', - isLeaf: true); +final _errorClear = _lib.lookupFunction), + void Function(Pointer<_Error>)>('shapes_error_clear', isLeaf: true); final _freeBytes = _lib.lookupFunction, Size), void Function(Pointer, int)>('shapes_free_bytes', isLeaf: true); @@ -118,9 +176,9 @@ final _freeBytes = _lib.lookupFunction, Size), final Pointer<_Error> _err = calloc<_Error>(); final Pointer _outLen = calloc(); -/// A failure the native library reported. Domain errors extend this class -/// with a positive [code]; the negative runtime codes are the constants -/// below. +/// A failure the native library reported to a function declared to throw. +/// Domain errors extend this class with a positive [code]; a negative code is +/// one of the runtime constants below. class NativeException implements Exception { /// Creates an exception carrying [code] and [message]. NativeException(this.code, this.message); @@ -131,10 +189,10 @@ class NativeException implements Exception { /// The producer panicked; [message] carries the panic text. static const int panicCode = -2; - /// An argument couldn't be lifted by the producer. + /// An argument or a callback's return couldn't be lifted by the producer. static const int marshalCode = -3; - /// A callback implementation threw; [message] carries its text. + /// A callback implementation failed; [message] carries its text. static const int foreignCode = -4; /// The call was cancelled (see [CancelledException]). @@ -158,24 +216,51 @@ class CancelledException extends NativeException { : super(NativeException.cancelledCode, message); } -typedef _ErrorMapper = NativeException Function( +/// A function that isn't declared to throw failed anyway. That's a bug in +/// the native library (a panic, an argument it couldn't lift, a callback +/// failure it let through), so it's an [Error], not an [Exception]. +final class NativeError extends Error { + /// Creates the error a failed call reported. + NativeError(this.code, this.message); + + /// The runtime code the producer reported (see [NativeException]). + final int code; + + /// The producer's message. + final String message; + + @override + String toString() => 'NativeError($code): $message'; +} + +/// Maps a reported code, message, and payload onto the object to throw. +typedef _ErrorMapper = Object Function( int code, String message, Uint8List payload); -/// Maps a runtime failure (or an undeclared code) onto its exception. -NativeException _runtimeError(int code, String message, Uint8List payload) => +/// The mapper of a function that can't fail: every code but cancellation +/// is a [NativeError]. +Object _trap(int code, String message, Uint8List payload) => + code == NativeException.cancelledCode + ? CancelledException(message) + : NativeError(code, message); + +/// Maps a runtime code (or one the domain doesn't declare) onto its +/// exception, for a function declared to throw. +NativeException _runtimeException(int code, String message, Uint8List payload) => code == NativeException.cancelledCode ? CancelledException(message) : NativeException(code, message); -/// Builds the exception [err] reports, using [map] for domain codes. -NativeException _readError(Pointer<_Error> err, _ErrorMapper map) { +/// Builds the error [err] reports, using [map]. +Object _readError(Pointer<_Error> err, _ErrorMapper map) { final e = err.ref; final message = e.message == nullptr ? '' : e.message.toDartString(); return map(e.code, message, _copyBytes(e.payloadPtr, e.payloadLen)); } /// Throws the error a synchronous call left in [err], clearing the slot. -void _check(Pointer<_Error> err, [_ErrorMapper map = _runtimeError]) { +/// [map] is the domain's mapper for a function declared to throw. +void _check(Pointer<_Error> err, [_ErrorMapper map = _trap]) { if (err.ref.code == 0) return; final error = _readError(err, map); _errorClear(err); @@ -311,6 +396,27 @@ final class _BufferWriter { _buf.setRange(_len, _len + v.length, v); _len += v.length; } + + void writeOptional(T? v, void Function(T) write) { + writeFlag(v != null); + if (v != null) write(v); + } + + void writeList(List v, void Function(T) write) { + writeLength(v.length); + for (final e in v) { + write(e); + } + } + + void writeMap( + Map v, void Function(K) writeKey, void Function(V) writeValue) { + writeLength(v.length); + for (final e in v.entries) { + writeKey(e.key); + writeValue(e.value); + } + } } final class _BufferReader { @@ -374,6 +480,24 @@ final class _BufferReader { return Uint8List.fromList(Uint8List.sublistView(_buf, at, at + n)); } + T? readOptional(T Function() read) => + readFlag() ? read() : null; + + // Collections grow as they decode, so a corrupt count can't force a huge + // allocation up front. + List readList(T Function() read) => + [for (var i = readLength(); i > 0; i--) read()]; + + Map readMap(K Function() readKey, V Function() readValue) { + final map = {}; + for (var i = readLength(); i > 0; i--) { + final key = readKey(); + if (map.containsKey(key)) _bufferError('duplicate map key $key'); + map[key] = readValue(); + } + return map; + } + void expectEnd() { if (_pos != _buf.length) _bufferError('trailing bytes'); } @@ -394,6 +518,36 @@ T _decode(Uint8List bytes, T Function(_BufferReader) read) { return value; } +// Records and rich enums compare by value: lists, maps, and byte arrays +// element by element, and everything else with `==` (interface wrappers by +// identity). + +bool _deepEquals(Object? a, Object? b) { + if (identical(a, b)) return true; + if (a is List && b is List) { + if (a.length != b.length) return false; + for (var i = 0; i < a.length; i++) { + if (!_deepEquals(a[i], b[i])) return false; + } + return true; + } + if (a is Map && b is Map) { + if (a.length != b.length) return false; + for (final key in a.keys) { + if (!b.containsKey(key) || !_deepEquals(a[key], b[key])) return false; + } + return true; + } + return a == b; +} + +int _deepHash(Object? v) => switch (v) { + List() => Object.hashAll(v.map(_deepHash)), + Map() => Object.hashAllUnordered( + v.entries.map((e) => Object.hash(_deepHash(e.key), _deepHash(e.value)))), + _ => v.hashCode, + }; + /// An algebraic shape (sum type with associated data) sealed class Shape { const Shape(); @@ -403,6 +557,15 @@ sealed class Shape { final class ShapeEmpty extends Shape { /// Creates a [ShapeEmpty]. const ShapeEmpty(); + + @override + bool operator ==(Object other) => other is ShapeEmpty; + + @override + int get hashCode => (ShapeEmpty).hashCode; + + @override + String toString() => 'ShapeEmpty()'; } /// A circle with a radius @@ -412,6 +575,18 @@ final class ShapeCircle extends Shape { /// Radius in points final double radius; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is ShapeCircle && + radius == other.radius; + + @override + int get hashCode => Object.hashAll([ShapeCircle, radius]); + + @override + String toString() => 'ShapeCircle(radius: $radius)'; } /// An axis-aligned rectangle @@ -421,6 +596,19 @@ final class ShapeRectangle extends Shape { final double width; final double height; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is ShapeRectangle && + width == other.width && + height == other.height; + + @override + int get hashCode => Object.hashAll([ShapeRectangle, width, height]); + + @override + String toString() => 'ShapeRectangle(width: $width, height: $height)'; } /// An integer-bounded shape (exercises narrow + wide ints) @@ -431,6 +619,20 @@ final class ShapeBounded extends Shape { final int min; final int max; final int weight; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is ShapeBounded && + min == other.min && + max == other.max && + weight == other.weight; + + @override + int get hashCode => Object.hashAll([ShapeBounded, min, max, weight]); + + @override + String toString() => 'ShapeBounded(min: $min, max: $max, weight: $weight)'; } /// A labeled shape with a small count @@ -440,28 +642,41 @@ final class ShapeLabeled extends Shape { final String label; final int count; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is ShapeLabeled && + label == other.label && + count == other.count; + + @override + int get hashCode => Object.hashAll([ShapeLabeled, label, count]); + + @override + String toString() => 'ShapeLabeled(label: $label, count: $count)'; } void _packShape(_BufferWriter w, Shape v) { switch (v) { case ShapeEmpty(): w.writeI32(0); - case final ShapeCircle _t0: + case ShapeCircle(): w.writeI32(1); - w.writeF64(_t0.radius); - case final ShapeRectangle _t1: + w.writeF64(v.radius); + case ShapeRectangle(): w.writeI32(2); - w.writeF32(_t1.width); - w.writeF32(_t1.height); - case final ShapeBounded _t2: + w.writeF32(v.width); + w.writeF32(v.height); + case ShapeBounded(): w.writeI32(3); - w.writeI16(_t2.min); - w.writeI16(_t2.max); - w.writeU64(_t2.weight); - case final ShapeLabeled _t3: + w.writeI16(v.min); + w.writeI16(v.max); + w.writeU64(v.weight); + case ShapeLabeled(): w.writeI32(4); - w.writeString(_t3.label); - w.writeU8(_t3.count); + w.writeString(v.label); + w.writeU8(v.count); } } @@ -501,7 +716,7 @@ enum Channel { } /// One field per supported numeric primitive -class Numerics { +final class Numerics { /// Creates a [Numerics]. Numerics({required this.a, required this.b, required this.c, required this.d, required this.e, required this.f, required this.g, required this.h, required this.i, required this.j}); @@ -515,6 +730,27 @@ class Numerics { final int h; final double i; final double j; + + @override + bool operator ==(Object other) => + identical(this, other) || + other is Numerics && + a == other.a && + b == other.b && + c == other.c && + d == other.d && + e == other.e && + f == other.f && + g == other.g && + h == other.h && + i == other.i && + j == other.j; + + @override + int get hashCode => Object.hashAll([Numerics, a, b, c, d, e, f, g, h, i, j]); + + @override + String toString() => 'Numerics(a: $a, b: $b, c: $c, d: $d, e: $e, f: $f, g: $g, h: $h, i: $i, j: $j)'; } void _packNumerics(_BufferWriter w, Numerics v) { @@ -531,17 +767,17 @@ void _packNumerics(_BufferWriter w, Numerics v) { } Numerics _unpackNumerics(_BufferReader r) => Numerics( - a: r.readI8(), - b: r.readI16(), - c: r.readI32(), - d: r.readI64(), - e: r.readU8(), - f: r.readU16(), - g: r.readU32(), - h: r.readU64(), - i: r.readF32(), - j: r.readF64(), -); + a: r.readI8(), + b: r.readI16(), + c: r.readI32(), + d: r.readI64(), + e: r.readU8(), + f: r.readU16(), + g: r.readU32(), + h: r.readU64(), + i: r.readF32(), + j: r.readF64(), + ); final _shapesShapesDescribe = _lib.lookupFunction< Pointer Function(Pointer, Size, Pointer, Pointer<_Error>), @@ -551,9 +787,7 @@ final _shapesShapesDescribe = _lib.lookupFunction< String describe(Shape shape) { final _arena = Arena(); try { - final _shapeWriter = _BufferWriter(); - _packShape(_shapeWriter, shape); - final _shapeBytes = _shapeWriter.takeBytes(); + final _shapeBytes = _encode(shape, _packShape); final _result = _shapesShapesDescribe(_stage(_arena, _shapeBytes), _shapeBytes.length, _outLen, _err); _check(_err); return _takeString(_result, _outLen.value); @@ -570,9 +804,7 @@ final _shapesShapesScale = _lib.lookupFunction< Shape scale(Shape shape, double factor) { final _arena = Arena(); try { - final _shapeWriter = _BufferWriter(); - _packShape(_shapeWriter, shape); - final _shapeBytes = _shapeWriter.takeBytes(); + final _shapeBytes = _encode(shape, _packShape); final _result = _shapesShapesScale(_stage(_arena, _shapeBytes), _shapeBytes.length, factor, _outLen, _err); _check(_err); return _decode(_takeBytes(_result, _outLen.value), _unpackShape); diff --git a/crates/weaveffi-cli/tests/snapshots/dart_shapes__pubspec_yaml.snap b/crates/weaveffi-cli/tests/snapshots/dart_shapes__pubspec_yaml.snap deleted file mode 100644 index 6b354df6..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dart_shapes__pubspec_yaml.snap +++ /dev/null @@ -1,18 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -name: shapes -version: 0.1.0 -description: 'Native bindings for shapes' - -environment: - sdk: '>=3.12.0 <4.0.0' - -dependencies: - ffi: ^2.1.0 - -# END pubspec.yaml diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_cs.snap index 192249f0..a29040ab 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_cs.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_cs.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #nullable enable @@ -16,28 +16,56 @@ using System.Threading.Tasks; namespace DocsEverywhere; -/// Typed exception for the DocsErrorsError error domain (module docs). -public class DocsErrorsException : NativeException +/// The errors of the DocsErrors domain, declared by module docs. +/// Catch a nested class for one code, or this class for any of them. +public abstract class DocsErrorsException : NativeException { + private protected DocsErrorsException(int code, string message) : base(code, message) + { + } + + /// Writes this code's fields, the error payload a callback + /// reports with it. + internal virtual void WritePayload(FfiBufferWriter writer) + { + } + /// Returned when no document matches the requested id - public const int NotFound = 1; + public sealed class NotFound : DocsErrorsException + { + /// The domain code, 1. + public const int ErrorCode = 1; - /// Raised when attempting to move into a forbidden state - public const int InvalidStatus = 2; + /// Creates the error with , or the + /// default message "Document not found" when it's null. + public NotFound(string? message = null) : base(ErrorCode, message ?? "Document not found") + { + } + } - /// Creates an exception carrying a domain error code. - public DocsErrorsException(int code, string message) : base(code, message) + /// Raised when attempting to move into a forbidden state + public sealed class InvalidStatus : DocsErrorsException { + /// The domain code, 2. + public const int ErrorCode = 2; + + /// Creates the error with , or the + /// default message "Invalid status transition" when it's null. + public InvalidStatus(string? message = null) : base(ErrorCode, message ?? "Invalid status transition") + { + } } + /// The exception for a raw code and its serialized payload. internal static new Exception FromError(int code, string message, byte[]? payload) { + var given = message.Length == 0 ? null : message; switch (code) { - case NotFound: - return new DocsErrorsException(code, string.IsNullOrEmpty(message) ? "Document not found" : message); - case InvalidStatus: - return new DocsErrorsException(code, string.IsNullOrEmpty(message) ? "Invalid status transition" : message); + case NotFound.ErrorCode: + return new NotFound(given); + case InvalidStatus.ErrorCode: + return new InvalidStatus(given); default: return global::DocsEverywhere.NativeException.FromError(code, message, payload); } @@ -92,27 +120,16 @@ public sealed class Document writer.WriteI64(Id); writer.WriteString(Title); writer.WriteI32((int)Status); - writer.WriteLen(Tags.Length); - foreach (var item0 in Tags) - { - writer.WriteString(item0); - } + FfiCodecs.WriteListOfString(writer, Tags); } internal static Document ReadFrom(FfiBufferReader reader) { - var fId = reader.ReadI64(); - var fTitle = reader.ReadString(); - var fStatus = (Status)reader.ReadI32(); - var fTagsCount = reader.ReadLen(); - var fTagsList = new List(reader.Capacity(fTagsCount)); - for (var i0 = 0; i0 < fTagsCount; i0++) - { - var fTagsItem = reader.ReadString(); - fTagsList.Add(fTagsItem); - } - var fTags = fTagsList.ToArray(); - return new Document(fId, fTitle, fStatus, fTags); + return new Document( + reader.ReadI64(), + reader.ReadString(), + (global::DocsEverywhere.Status)reader.ReadI32(), + FfiCodecs.ReadListOfString(reader)); } } @@ -123,8 +140,10 @@ public sealed class Document /// /// The native library may call any method from any thread until /// it releases its last reference to the implementation. Object arguments -/// belong to the implementation. An exception thrown by a method reaches -/// the native caller as . +/// belong to the implementation. An exception thrown by a method fails the +/// native library's call in progress: a method documented to throw a domain +/// exception reports it with its code and fields, and any other exception +/// reaches the library as . public interface ISavedListener { /// Fires after a document has been persisted successfully @@ -140,8 +159,10 @@ internal static unsafe class FfiVtable_docs_SavedListener [StructLayout(LayoutKind.Sequential)] private struct Layout { + public uint Size; + public uint Flags; + public delegate* unmanaged[Cdecl] Free; public delegate* unmanaged[Cdecl] on_saved; - public delegate* unmanaged[Cdecl] free; } /// The vtable's address, valid for the process lifetime. @@ -149,12 +170,20 @@ internal static unsafe class FfiVtable_docs_SavedListener private static IntPtr Allocate() { - var vtable = (Layout*)NativeMemory.Alloc((nuint)sizeof(Layout)); + var vtable = (Layout*)NativeMemory.AllocZeroed((nuint)sizeof(Layout)); + vtable->Size = (uint)sizeof(Layout); + vtable->Flags = 0; + vtable->Free = &FreeTrampoline; vtable->on_saved = &OnSavedTrampoline; - vtable->free = &FreeTrampoline; return (IntPtr)vtable; } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] + private static void FreeTrampoline(IntPtr ctx) + { + Ffi.Unregister(ctx); + } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] private static void OnSavedTrampoline(IntPtr ctx, long doc_id, int revision, FfiError* out_err) { @@ -167,12 +196,6 @@ internal static unsafe class FfiVtable_docs_SavedListener Ffi.SetForeignError(out_err, e); } } - - [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] - private static void FreeTrampoline(IntPtr ctx) - { - Ffi.Unregister(ctx); - } } /// @@ -208,7 +231,7 @@ public sealed unsafe class Notebook : IDisposable, IEquatable { var ffiErr = default(FfiError); var ffiResult = NativeMethods.docs_everywhere_docs_Notebook_new(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeBugException.FromError); Handle = new NativeHandle(ffiResult); } @@ -222,24 +245,21 @@ public sealed unsafe class Notebook : IDisposable, IEquatable fixed (byte* documentPtr = documentWriter.Written) { var ffiResult = NativeMethods.docs_everywhere_docs_Notebook_insert(Handle, documentPtr, (nuint)documentWriter.Length, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeBugException.FromError); return ffiResult; } } /// Look up a document by identifier, failing when none matches /// Identifier of the document to find - /// The call reported a DocsErrorsError code. + /// The call reported a DocsErrors code. public Document Find(long id) { var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.docs_everywhere_docs_Notebook_find(Handle, id, &ffiOutLen, &ffiErr); if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.DocsErrorsException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::DocsEverywhere.Document.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::DocsEverywhere.Document.ReadFrom(r)); } /// The number of documents currently held @@ -247,7 +267,7 @@ public sealed unsafe class Notebook : IDisposable, IEquatable { var ffiErr = default(FfiError); var ffiResult = NativeMethods.docs_everywhere_docs_Notebook_size(Handle, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeBugException.FromError); return ffiResult; } @@ -256,7 +276,7 @@ public sealed unsafe class Notebook : IDisposable, IEquatable { var ffiErr = default(FfiError); var ffiResult = NativeMethods.docs_everywhere_docs_Notebook_max_documents(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeBugException.FromError); return ffiResult; } @@ -312,6 +332,7 @@ public sealed unsafe class Notebook : IDisposable, IEquatable } } +/// Document storage: create, publish, and observe documents. public static unsafe class Docs { /// @@ -326,21 +347,14 @@ public static unsafe class Docs { var titleBytes = Ffi.Utf8(title); var tagsWriter = new FfiBufferWriter(); - tagsWriter.WriteLen(tags.Length); - foreach (var item0 in tags) - { - tagsWriter.WriteString(item0); - } + FfiCodecs.WriteListOfString(tagsWriter, tags); var ffiErr = default(FfiError); nuint ffiOutLen = 0; fixed (byte* titlePtr = titleBytes, tagsPtr = tagsWriter.Written) { var ffiResult = NativeMethods.docs_everywhere_docs_create_document(titlePtr, (nuint)titleBytes.Length, tagsPtr, (nuint)tagsWriter.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::DocsEverywhere.Document.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::DocsEverywhere.Document.ReadFrom(r)); } } @@ -351,16 +365,8 @@ public static unsafe class Docs var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.docs_everywhere_docs_get_document(id, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - Document? ffiDecoded = null; - if (ffiDecodedReader.ReadBool()) - { - var ffiDecodedValue = global::DocsEverywhere.Document.ReadFrom(ffiDecodedReader); - ffiDecoded = ffiDecodedValue; - } - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => FfiCodecs.ReadOptionalOfDocument(r)); } /// Publish a document in the background @@ -388,7 +394,7 @@ public static unsafe class Docs { if (err != null) { - ffiCall.SetError(err, global::DocsEverywhere.NativeException.FromError); + ffiCall.SetError(err, global::DocsEverywhere.NativeBugException.FromError); return; } ffiCall.SetResult(result != 0); @@ -414,20 +420,83 @@ public static unsafe class Docs Ffi.Unregister(listenerCtx); throw; } - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::DocsEverywhere.NativeBugException.FromError); + } +} + +/// The value-buffer codec of every composite type the API +/// carries. +internal static class FfiCodecs +{ + /// Reads one value that must fill the whole buffer. + internal static T Decode(FfiBufferReader reader, Func read) + { + var value = read(reader); + reader.ExpectEnd(); + return value; + } + + internal static void WriteListOfString(FfiBufferWriter writer, string[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + writer.WriteString(item); + } + } + + internal static string[] ReadListOfString(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(reader.ReadString()); + } + return list.ToArray(); + } + + internal static void WriteOptionalOfDocument(FfiBufferWriter writer, Document? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + present.WriteTo(writer); + } + } + + internal static Document? ReadOptionalOfDocument(FfiBufferReader reader) + { + return reader.ReadBool() ? global::DocsEverywhere.Document.ReadFrom(reader) : null; } } internal static unsafe partial class NativeMethods { - static partial void VerifyChecksums() + static partial void VerifyContracts() { - VerifyChecksum("docs", &docs_everywhere_docs_checksum, 0x7950eec1e9cfc819UL); + VerifyContract("docs_everywhere_docs_contract", &docs_everywhere_docs_contract, new (ulong, ulong, string)[] + { + (0x00b2187fd75634ecUL, 0x2263c6f787bf6081UL, "docs.publish_async"), + (0x12fc242b9dc6444cUL, 0x96405f484a642b31UL, "docs.Notebook.insert"), + (0x2213822ed05cd5bdUL, 0x26c5833392aa63ceUL, "docs.Notebook.new"), + (0x2bd762bcdaa207ffUL, 0x1b0ebab2d04c49b0UL, "docs.SavedListener"), + (0x2e086c45e9d617deUL, 0x4d43bf32358c01ffUL, "docs.Notebook.find"), + (0x4817dc98197ad960UL, 0xf8017b5a7d58b2c7UL, "docs.Notebook.size"), + (0x614de47452d5a55cUL, 0xd9aae1472494ffefUL, "docs.get_document"), + (0x67c5cb9fc119bcd5UL, 0x08d5b25a1a22a7faUL, "docs.Document"), + (0x8d484bf2d78c9b70UL, 0xe9093cf96912194cUL, "docs.Notebook.max_documents"), + (0xaea61d6451691c67UL, 0xe4704128b3524b3dUL, "docs.watch"), + (0xd68d6ad1540cdec5UL, 0x01afeb40b0a8bfe1UL, "docs.Notebook"), + (0xdb0601444a4760ccUL, 0xe56c69629b444797UL, "docs.create_document"), + (0xe12d6f4838347b3eUL, 0xa9bab86646d84f51UL, "docs.Status"), + (0xee2f8d25b3313b50UL, 0x9691232f35439a2bUL, "docs.DocsErrors"), + }); } - [LibraryImport(LibName, EntryPoint = "docs_everywhere_docs_checksum")] + [LibraryImport(LibName, EntryPoint = "docs_everywhere_docs_contract")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial ulong docs_everywhere_docs_checksum(); + internal static partial FfiContractEntry* docs_everywhere_docs_contract(nuint* out_len); [LibraryImport(LibName, EntryPoint = "docs_everywhere_docs_Notebook_clone")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_csproj.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_csproj.snap deleted file mode 100644 index 45661135..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__DocsEverywhere_csproj.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - - - - - net8.0 - DocsEverywhere - DocsEverywhere - DocsEverywhere - 0.1.0 - Native bindings for docs_everywhere - enable - true - true - - $(NoWarn);CS0618 - - - - - diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__README_md.snap deleted file mode 100644 index 0e5f6afb..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__README_md.snap +++ /dev/null @@ -1,32 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# DocsEverywhere (.NET) - -Native bindings for docs_everywhere - -The `DocsEverywhere` namespace binds the `docs_everywhere` native library through -source-generated P/Invoke (`[LibraryImport]`), so it's trim- and -AOT-friendly. - -## Build - -```bash -dotnet build -``` - -Reference this project from yours (`dotnet add reference DocsEverywhere.csproj`) -or pack it (`dotnet pack -c Release`). - -## Loading the native library - -The bindings load `docs_everywhere` with the platform's normal rules -(`libdocs_everywhere.dylib`, `libdocs_everywhere.so`, or `docs_everywhere.dll` next to the app or on the -library search path). Set `DOCS_EVERYWHERE_LIBRARY` to a full path to load a -specific file instead. - - diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__Runtime_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__Runtime_cs.snap deleted file mode 100644 index fa8342d0..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_docs_everywhere__Runtime_cs.snap +++ /dev/null @@ -1,698 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#nullable enable - -using System; -using System.Collections; -using System.Collections.Generic; -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using System.Text; -using System.Threading; -using System.Threading.Tasks; - -namespace DocsEverywhere; - -/// Raised for a failure the native library reports that isn't a typed -/// domain error: producer bugs, panics, marshalling failures, and exceptions -/// thrown by a callback-interface implementation. Every typed domain -/// exception derives from it. -public class NativeException : Exception -{ - /// An untyped producer error. - public const int GenericErrorCode = -1; - /// The producer panicked; the message carries the panic text. - public const int PanicErrorCode = -2; - /// An argument couldn't be lifted by the producer. - public const int MarshalErrorCode = -3; - /// A callback-interface implementation threw; the message carries - /// the original exception's message. - public const int ForeignErrorCode = -4; - /// The call was cancelled. Surfaces as - /// , never as this type. - public const int CancelledErrorCode = -5; - - /// The error code the native library reported. - public int Code { get; } - - /// Creates an exception carrying a native error code. - public NativeException(int code, string message) : base(message) - { - Code = code; - } - - /// Maps a runtime error code to its exception: cancellation to - /// , anything else to this - /// type. - internal static Exception FromError(int code, string message, byte[]? payload) - { - if (code == CancelledErrorCode) - { - return new OperationCanceledException(message); - } - return new NativeException(code, message); - } -} - -/// Maps a native error code, message, and serialized payload to the -/// exception a call throws. -internal delegate Exception FfiErrorMap(int code, string message, byte[]? payload); - -/// The native docs_everywhere_error slot. -[StructLayout(LayoutKind.Sequential)] -internal unsafe struct FfiError -{ - public int Code; - public byte* Message; - public byte* PayloadPtr; - public nuint PayloadLen; -} - -/// Marshalling helpers shared by every generated call. -internal static unsafe class Ffi -{ - /// The UTF-8 bytes of a string argument. The native side borrows - /// them as a (pointer, length) pair, so interior NULs survive. - internal static byte[] Utf8(string value) - { - return Encoding.UTF8.GetBytes(value); - } - - /// Decodes a borrowed UTF-8 run; the native side keeps - /// ownership. - internal static string ReadString(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? "" : Encoding.UTF8.GetString(ptr, checked((int)len)); - } - - /// Copies a borrowed byte run; the native side keeps - /// ownership. - internal static byte[] ReadBytes(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? Array.Empty() : new ReadOnlySpan(ptr, checked((int)len)).ToArray(); - } - - /// Decodes a returned UTF-8 string and releases it. - internal static string TakeString(byte* ptr, nuint len) - { - try - { - return ReadString(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies returned bytes and releases them. - internal static byte[] TakeBytes(byte* ptr, nuint len) - { - try - { - return ReadBytes(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies a returned value buffer, releases it, and returns a - /// reader over the copy. - internal static FfiBufferReader TakeBuffer(byte* ptr, nuint len) - { - return new FfiBufferReader(TakeBytes(ptr, len)); - } - - /// Releases a caller-owned error slot and returns the exception - /// it described. - internal static Exception TakeError(FfiError* err, FfiErrorMap map) - { - var code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorClear(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - /// Releases a heap-boxed async error and returns the exception it - /// described. - internal static Exception TakeBoxedError(FfiError* err, FfiErrorMap map, out int code) - { - code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorFree(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - private static string ErrorMessage(FfiError* err) - { - return Marshal.PtrToStringUTF8((IntPtr)err->Message) ?? ""; - } - - /// Reports a callback-interface failure to the producer through - /// out_err. The producer copies the message. - internal static void SetForeignError(FfiError* err, Exception e) - { - var bytes = Encoding.UTF8.GetBytes(e.Message + "\0"); - fixed (byte* message = bytes) - { - NativeMethods.ErrorSet(err, NativeException.ForeignErrorCode, message); - } - } - - /// Pins a callback-interface implementation for the producer and - /// returns the ctx it hands back to every trampoline. The - /// producer releases it through the vtable's free. - internal static IntPtr Register(object implementation) - { - return GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); - } - - /// Releases a ctx the producer never received (the call - /// failed before reaching it) or no longer holds. - internal static void Unregister(IntPtr ctx) - { - GCHandle.FromIntPtr(ctx).Free(); - } - - /// The implementation behind a trampoline's ctx. - internal static T Target(IntPtr ctx) where T : class - { - return (T)GCHandle.FromIntPtr(ctx).Target!; - } -} - -/// Imports of the native runtime surface plus the load-time contract -/// check. The generated API declares its own imports in the other half of -/// this partial class. -internal static unsafe partial class NativeMethods -{ - /// The native library's base name. - internal const string LibName = "docs_everywhere"; - - /// The environment variable naming an explicit library - /// path. - internal const string LibraryEnvVar = "DOCS_EVERYWHERE_LIBRARY"; - - /// The C ABI revision these bindings were generated - /// against. - internal const uint AbiVersion = 3; - - // Runs before the first native call through this class, so a library - // built for another ABI revision or another contract fails here instead - // of misreading memory later. - static NativeMethods() - { - try - { - NativeLibrary.SetDllImportResolver(typeof(NativeMethods).Assembly, Resolve); - } - catch (InvalidOperationException) - { - // The assembly already has a resolver; default probing still applies. - } - uint found; - try - { - found = AbiVersionOf(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't export docs_everywhere_abi_version; these bindings expect ABI revision {AbiVersion}", e); - } - if (found != AbiVersion) - { - throw new InvalidOperationException( - $"ABI mismatch: these bindings expect revision {AbiVersion} but the loaded {LibName} library reports revision {found}"); - } - VerifyChecksums(); - } - - static partial void VerifyChecksums(); - - private static IntPtr Resolve(string name, Assembly assembly, DllImportSearchPath? searchPath) - { - if (name != LibName) - { - return IntPtr.Zero; - } - var path = Environment.GetEnvironmentVariable(LibraryEnvVar); - return string.IsNullOrEmpty(path) ? IntPtr.Zero : NativeLibrary.Load(path); - } - - /// Fails the load when module was - /// generated from a different contract than the loaded library - /// implements. - private static void VerifyChecksum(string module, delegate* checksum, ulong expected) - { - ulong found; - try - { - found = checksum(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't implement module '{module}'", e); - } - if (found != expected) - { - throw new InvalidOperationException( - $"contract mismatch for module '{module}': these bindings expect checksum 0x{expected:x16} but the loaded {LibName} library reports 0x{found:x16}; regenerate the bindings for this library"); - } - } - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_abi_version")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial uint AbiVersionOf(); - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_error_set")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorSet(FfiError* err, int code, byte* message); - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_error_clear")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorClear(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_error_free")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorFree(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_free_bytes")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void FreeBytes(byte* ptr, nuint len); - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_cancel_token_create")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial IntPtr CancelTokenCreate(); - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_cancel_token_cancel")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenCancel(IntPtr token); - - [LibraryImport(LibName, EntryPoint = "docs_everywhere_cancel_token_destroy")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenDestroy(IntPtr token); -} - -/// One pending async call: the task it completes, the native -/// context that finds it again, and, for a cancellable function, the -/// native cancel token linked to the caller's -/// . -internal sealed class FfiCall -{ - private readonly TaskCompletionSource _tcs = - new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - private readonly CancellationToken _cancellation; - private readonly object _gate = new object(); - private GCHandle _self; - private IntPtr _token; - private CancellationTokenRegistration _registration; - private bool _completed; - private bool _cancelling; - - internal FfiCall(CancellationToken cancellation = default) - { - _cancellation = cancellation; - _self = GCHandle.Alloc(this); - } - - /// The task the completion resolves. - internal Task Task => _tcs.Task; - - /// The context argument for the launcher. - internal IntPtr Context => GCHandle.ToIntPtr(_self); - - /// Creates the native cancel token for the launcher and cancels - /// it when the caller's fires. - internal IntPtr CancelToken() - { - _token = NativeMethods.CancelTokenCreate(); - if (_cancellation.CanBeCanceled) - { - _registration = _cancellation.Register(static state => ((FfiCall)state!).Cancel(), this); - } - return _token; - } - - /// Recovers the call behind a completion's context. Runs - /// exactly once per launch, on a producer thread. - internal static FfiCall Complete(IntPtr context) - { - var handle = GCHandle.FromIntPtr(context); - var call = (FfiCall)handle.Target!; - handle.Free(); - call.Release(); - return call; - } - - /// Releases everything when the launcher threw before the - /// producer took the call, so no completion will arrive. - internal void Abandon() - { - _self.Free(); - Release(); - } - - internal void SetResult(T value) - { - _tcs.TrySetResult(value); - } - - internal void SetException(Exception e) - { - _tcs.TrySetException(e); - } - - /// Faults (or cancels, for the cancelled code) the task from a - /// heap-boxed error, releasing the box. - internal unsafe void SetError(FfiError* err, FfiErrorMap map) - { - var e = Ffi.TakeBoxedError(err, map, out var code); - if (code == NativeException.CancelledErrorCode) - { - _tcs.TrySetCanceled(_cancellation); - } - else - { - _tcs.TrySetException(e); - } - } - - // The registration callback and the completion race: whichever runs - // last destroys the consumer's token reference, so a cancel never - // touches a destroyed token. - private void Cancel() - { - lock (_gate) - { - if (_completed) - { - return; - } - _cancelling = true; - } - NativeMethods.CancelTokenCancel(_token); - bool destroy; - lock (_gate) - { - _cancelling = false; - destroy = _completed; - } - if (destroy) - { - NativeMethods.CancelTokenDestroy(_token); - } - } - - private void Release() - { - if (_token == IntPtr.Zero) - { - return; - } - bool destroy; - lock (_gate) - { - _completed = true; - destroy = !_cancelling; - } - if (destroy) - { - _registration.Dispose(); - NativeMethods.CancelTokenDestroy(_token); - } - } -} - -/// Owns one native iterator and destroys it exactly once, when -/// enumeration ends, the enumerator is disposed, or the garbage collector -/// finalizes an abandoned one. -internal sealed unsafe class FfiIteratorHandle : SafeHandle -{ - private readonly delegate* _destroy; - - internal FfiIteratorHandle(IntPtr iterator, delegate* destroy) - : base(IntPtr.Zero, true) - { - _destroy = destroy; - SetHandle(iterator); - } - - public override bool IsInvalid => handle == IntPtr.Zero; - - protected override bool ReleaseHandle() - { - _destroy(handle); - return true; - } -} - -/// A lazily streamed sequence backed by a native iterator. It can be -/// enumerated once; each MoveNext pulls one item. -internal sealed unsafe class FfiSequence : IEnumerable, IEnumerator -{ - private readonly FfiIteratorHandle _iterator; - private readonly delegate* _next; - private int _started; - private T _current = default!; - - internal FfiSequence(FfiIteratorHandle iterator, delegate* next) - { - _iterator = iterator; - _next = next; - } - - public T Current => _current; - - object? IEnumerator.Current => _current; - - public IEnumerator GetEnumerator() - { - if (Interlocked.Exchange(ref _started, 1) != 0) - { - throw new InvalidOperationException("this sequence can be enumerated only once"); - } - return this; - } - - IEnumerator IEnumerable.GetEnumerator() - { - return GetEnumerator(); - } - - public bool MoveNext() - { - if (_iterator.IsClosed) - { - return false; - } - if (_next(_iterator, out _current)) - { - return true; - } - _iterator.Dispose(); - return false; - } - - public void Reset() - { - throw new NotSupportedException("a native iterator can't be reset"); - } - - public void Dispose() - { - _iterator.Dispose(); - } -} - -/// Serializes values into the value-buffer wire format -/// (little-endian, packed). -internal sealed class FfiBufferWriter -{ - private byte[] _buf = new byte[64]; - private int _len; - - /// The bytes written so far. - internal ReadOnlySpan Written => _buf.AsSpan(0, _len); - - /// The number of bytes written so far. - internal int Length => _len; - - private Span Grow(int extra) - { - if (_len + extra > _buf.Length) - { - Array.Resize(ref _buf, Math.Max(_buf.Length * 2, _len + extra)); - } - var span = _buf.AsSpan(_len, extra); - _len += extra; - return span; - } - - internal void WriteBool(bool v) => WriteU8(v ? (byte)1 : (byte)0); - - internal void WriteI8(sbyte v) => WriteU8((byte)v); - - internal void WriteU8(byte v) => Grow(1)[0] = v; - - internal void WriteI16(short v) => System.Buffers.Binary.BinaryPrimitives.WriteInt16LittleEndian(Grow(2), v); - - internal void WriteU16(ushort v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt16LittleEndian(Grow(2), v); - - internal void WriteI32(int v) => System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(Grow(4), v); - - internal void WriteU32(uint v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(Grow(4), v); - - internal void WriteI64(long v) => System.Buffers.Binary.BinaryPrimitives.WriteInt64LittleEndian(Grow(8), v); - - internal void WriteU64(ulong v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt64LittleEndian(Grow(8), v); - - internal void WriteF32(float v) => System.Buffers.Binary.BinaryPrimitives.WriteSingleLittleEndian(Grow(4), v); - - internal void WriteF64(double v) => System.Buffers.Binary.BinaryPrimitives.WriteDoubleLittleEndian(Grow(8), v); - - internal void WriteLen(int len) => WriteU32((uint)len); - - internal void WriteString(string v) - { - var count = Encoding.UTF8.GetByteCount(v); - WriteLen(count); - Encoding.UTF8.GetBytes(v, Grow(count)); - } - - internal void WriteBytes(byte[] v) - { - WriteLen(v.Length); - v.CopyTo(Grow(v.Length)); - } - - /// Writes an object token. The token must carry its own strong - /// reference (a fresh clone), which the reader adopts. - internal void WriteObject(IntPtr token) => WriteU64((ulong)(nuint)token); -} - -/// Decodes values from the value-buffer wire format. A malformed -/// buffer is a producer or consumer bug and throws -/// . -internal sealed class FfiBufferReader -{ - private static readonly Encoding Utf8Strict = new UTF8Encoding(false, true); - - private readonly byte[] _data; - private int _pos; - - internal FfiBufferReader(byte[] data) - { - _data = data; - } - - private static InvalidOperationException Malformed(string what) - { - return new InvalidOperationException("malformed value buffer: " + what); - } - - private ReadOnlySpan Take(int n) - { - if (_data.Length - _pos < n) - { - throw Malformed("buffer exhausted"); - } - var span = _data.AsSpan(_pos, n); - _pos += n; - return span; - } - - internal bool ReadBool() - { - var b = ReadU8(); - if (b > 1) - { - throw Malformed("invalid bool byte"); - } - return b == 1; - } - - internal sbyte ReadI8() => (sbyte)ReadU8(); - - internal byte ReadU8() => Take(1)[0]; - - internal short ReadI16() => System.Buffers.Binary.BinaryPrimitives.ReadInt16LittleEndian(Take(2)); - - internal ushort ReadU16() => System.Buffers.Binary.BinaryPrimitives.ReadUInt16LittleEndian(Take(2)); - - internal int ReadI32() => System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(Take(4)); - - internal uint ReadU32() => System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(Take(4)); - - internal long ReadI64() => System.Buffers.Binary.BinaryPrimitives.ReadInt64LittleEndian(Take(8)); - - internal ulong ReadU64() => System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(Take(8)); - - internal float ReadF32() => System.Buffers.Binary.BinaryPrimitives.ReadSingleLittleEndian(Take(4)); - - internal double ReadF64() => System.Buffers.Binary.BinaryPrimitives.ReadDoubleLittleEndian(Take(8)); - - /// A collection count. Elements can be zero-sized, so the count - /// isn't checked against the remaining bytes; callers cap their - /// preallocation with . - internal int ReadLen() - { - var len = ReadU32(); - if (len > int.MaxValue) - { - throw Malformed("length prefix out of range"); - } - return (int)len; - } - - /// A preallocation size for elements - /// that a malformed count can't inflate. - internal int Capacity(int count) => Math.Min(count, _data.Length - _pos); - - internal string ReadString() - { - var bytes = Take(ReadLen()); - try - { - return Utf8Strict.GetString(bytes); - } - catch (ArgumentException) - { - throw Malformed("string is not valid UTF-8"); - } - } - - internal byte[] ReadBytes() => Take(ReadLen()).ToArray(); - - /// Reads an object token carrying one strong reference. - internal IntPtr ReadObject() - { - var token = ReadU64(); - if (token == 0) - { - throw Malformed("null object token"); - } - return (IntPtr)(nint)token; - } - - internal void ExpectEnd() - { - if (_pos != _data.Length) - { - throw Malformed("trailing bytes"); - } - } -} - -// END Runtime.cs diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_cs.snap index afc7db45..761df884 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_cs.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_cs.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #nullable enable @@ -16,23 +16,41 @@ using System.Threading.Tasks; namespace EdgeCases; -/// Typed exception for the EdgeErrorsError error domain (module edge.cases). -public class EdgeErrorsException : NativeException +/// The errors of the EdgeErrors domain, declared by module edge_cases. +/// Catch a nested class for one code, or this class for any of them. +public abstract class EdgeErrorsException : NativeException { - /// Resource is busy - public const int Busy = 7; + private protected EdgeErrorsException(int code, string message) : base(code, message) + { + } - /// Creates an exception carrying a domain error code. - public EdgeErrorsException(int code, string message) : base(code, message) + /// Writes this code's fields, the error payload a callback + /// reports with it. + internal virtual void WritePayload(FfiBufferWriter writer) { } + /// Resource is busy + public sealed class Busy : EdgeErrorsException + { + /// The domain code, 7. + public const int ErrorCode = 7; + + /// Creates the error with , or the + /// default message "Resource is busy" when it's null. + public Busy(string? message = null) : base(ErrorCode, message ?? "Resource is busy") + { + } + } + + /// The exception for a raw code and its serialized payload. internal static new Exception FromError(int code, string message, byte[]? payload) { + var given = message.Length == 0 ? null : message; switch (code) { - case Busy: - return new EdgeErrorsException(code, string.IsNullOrEmpty(message) ? "Resource is busy" : message); + case Busy.ErrorCode: + return new Busy(given); default: return global::EdgeCases.NativeException.FromError(code, message, payload); } @@ -71,8 +89,8 @@ public sealed class Leaf internal static Leaf ReadFrom(FfiBufferReader reader) { - var fWeight = reader.ReadF32(); - return new Leaf(fWeight); + return new Leaf( + reader.ReadF32()); } } @@ -93,25 +111,14 @@ public sealed class Inner internal void WriteTo(FfiBufferWriter writer) { Leaf.WriteTo(writer); - writer.WriteLen(Leaves.Length); - foreach (var item0 in Leaves) - { - item0.WriteTo(writer); - } + FfiCodecs.WriteListOfLeaf(writer, Leaves); } internal static Inner ReadFrom(FfiBufferReader reader) { - var fLeaf = global::EdgeCases.Leaf.ReadFrom(reader); - var fLeavesCount = reader.ReadLen(); - var fLeavesList = new List(reader.Capacity(fLeavesCount)); - for (var i0 = 0; i0 < fLeavesCount; i0++) - { - var fLeavesItem = global::EdgeCases.Leaf.ReadFrom(reader); - fLeavesList.Add(fLeavesItem); - } - var fLeaves = fLeavesList.ToArray(); - return new Inner(fLeaf, fLeaves); + return new Inner( + global::EdgeCases.Leaf.ReadFrom(reader), + FfiCodecs.ReadListOfLeaf(reader)); } } @@ -158,136 +165,32 @@ public sealed class Outer internal void WriteTo(FfiBufferWriter writer) { Inner.WriteTo(writer); - writer.WriteBool(MaybeInner != null); - if (MaybeInner != null) - { - MaybeInner!.WriteTo(writer); - } - writer.WriteLen(ByName.Count); - foreach (var entry0 in ByName) - { - writer.WriteString(entry0.Key); - entry0.Value.WriteTo(writer); - } - writer.WriteLen(Modes.Length); - foreach (var item0 in Modes) - { - writer.WriteI32((int)item0); - } - writer.WriteLen(Matrix.Length); - foreach (var item0 in Matrix) - { - writer.WriteLen(item0.Length); - foreach (var item1 in item0) - { - writer.WriteI32(item1); - } - } - writer.WriteLen(Sparse.Length); - foreach (var item0 in Sparse) - { - writer.WriteBool(item0 != null); - if (item0 != null) - { - writer.WriteI64(item0.Value); - } - } - writer.WriteBool(Blob != null); - if (Blob != null) - { - writer.WriteBytes(Blob!); - } - writer.WriteBool(Note != null); - if (Note != null) - { - writer.WriteString(Note!); - } - writer.WriteLen(ById.Count); - foreach (var entry0 in ById) - { - writer.WriteI64(entry0.Key); - writer.WriteString(entry0.Value); - } + FfiCodecs.WriteOptionalOfInner(writer, MaybeInner); + FfiCodecs.WriteMapOfStringToInner(writer, ByName); + FfiCodecs.WriteListOfMode(writer, Modes); + FfiCodecs.WriteListOfListOfI32(writer, Matrix); + FfiCodecs.WriteListOfOptionalOfI64(writer, Sparse); + FfiCodecs.WriteOptionalOfBytes(writer, Blob); + FfiCodecs.WriteOptionalOfString(writer, Note); + FfiCodecs.WriteMapOfI64ToString(writer, ById); writer.WriteBool(Hidden); writer.WriteU8(V2); } internal static Outer ReadFrom(FfiBufferReader reader) { - var fInner = global::EdgeCases.Inner.ReadFrom(reader); - Inner? fMaybeInner = null; - if (reader.ReadBool()) - { - var fMaybeInnerValue = global::EdgeCases.Inner.ReadFrom(reader); - fMaybeInner = fMaybeInnerValue; - } - var fByNameCount = reader.ReadLen(); - var fByName = new Dictionary(reader.Capacity(fByNameCount)); - for (var i0 = 0; i0 < fByNameCount; i0++) - { - var fByNameKey = reader.ReadString(); - var fByNameVal = global::EdgeCases.Inner.ReadFrom(reader); - fByName[fByNameKey] = fByNameVal; - } - var fModesCount = reader.ReadLen(); - var fModesList = new List(reader.Capacity(fModesCount)); - for (var i0 = 0; i0 < fModesCount; i0++) - { - var fModesItem = (Mode)reader.ReadI32(); - fModesList.Add(fModesItem); - } - var fModes = fModesList.ToArray(); - var fMatrixCount = reader.ReadLen(); - var fMatrixList = new List(reader.Capacity(fMatrixCount)); - for (var i0 = 0; i0 < fMatrixCount; i0++) - { - var fMatrixItemCount = reader.ReadLen(); - var fMatrixItemList = new List(reader.Capacity(fMatrixItemCount)); - for (var i1 = 0; i1 < fMatrixItemCount; i1++) - { - var fMatrixItemItem = reader.ReadI32(); - fMatrixItemList.Add(fMatrixItemItem); - } - var fMatrixItem = fMatrixItemList.ToArray(); - fMatrixList.Add(fMatrixItem); - } - var fMatrix = fMatrixList.ToArray(); - var fSparseCount = reader.ReadLen(); - var fSparseList = new List(reader.Capacity(fSparseCount)); - for (var i0 = 0; i0 < fSparseCount; i0++) - { - long? fSparseItem = null; - if (reader.ReadBool()) - { - var fSparseItemValue = reader.ReadI64(); - fSparseItem = fSparseItemValue; - } - fSparseList.Add(fSparseItem); - } - var fSparse = fSparseList.ToArray(); - byte[]? fBlob = null; - if (reader.ReadBool()) - { - var fBlobValue = reader.ReadBytes(); - fBlob = fBlobValue; - } - string? fNote = null; - if (reader.ReadBool()) - { - var fNoteValue = reader.ReadString(); - fNote = fNoteValue; - } - var fByIdCount = reader.ReadLen(); - var fById = new Dictionary(reader.Capacity(fByIdCount)); - for (var i0 = 0; i0 < fByIdCount; i0++) - { - var fByIdKey = reader.ReadI64(); - var fByIdVal = reader.ReadString(); - fById[fByIdKey] = fByIdVal; - } - var fHidden = reader.ReadBool(); - var fV2 = reader.ReadU8(); - return new Outer(fInner, fMaybeInner, fByName, fModes, fMatrix, fSparse, fBlob, fNote, fById, fHidden, fV2); + return new Outer( + global::EdgeCases.Inner.ReadFrom(reader), + FfiCodecs.ReadOptionalOfInner(reader), + FfiCodecs.ReadMapOfStringToInner(reader), + FfiCodecs.ReadListOfMode(reader), + FfiCodecs.ReadListOfListOfI32(reader), + FfiCodecs.ReadListOfOptionalOfI64(reader), + FfiCodecs.ReadOptionalOfBytes(reader), + FfiCodecs.ReadOptionalOfString(reader), + FfiCodecs.ReadMapOfI64ToString(reader), + reader.ReadBool(), + reader.ReadU8()); } } @@ -541,66 +444,66 @@ public sealed class Keywords internal static Keywords ReadFrom(FfiBufferReader reader) { - var fClass = reader.ReadI32(); - var fDef = reader.ReadI32(); - var fImport = reader.ReadI32(); - var fFrom = reader.ReadI32(); - var fFunc = reader.ReadI32(); - var fRange = reader.ReadI32(); - var fRegister = reader.ReadI32(); - var fVolatile = reader.ReadI32(); - var fNew = reader.ReadI32(); - var fDefault = reader.ReadI32(); - var fEnd = reader.ReadI32(); - var fIn = reader.ReadI32(); - var fIs = reader.ReadI32(); - var fInt = reader.ReadI32(); - var fVar = reader.ReadI32(); - var fVal = reader.ReadI32(); - var fSelf = reader.ReadI32(); - var fThis = reader.ReadI32(); - var fObject = reader.ReadI32(); - var fString = reader.ReadI32(); - var fCase = reader.ReadI32(); - var fSelect = reader.ReadI32(); - var fSwitch = reader.ReadI32(); - var fVoid = reader.ReadI32(); - var fUnion = reader.ReadI32(); - var fNamespace = reader.ReadI32(); - var fOperator = reader.ReadI32(); - var fTemplate = reader.ReadI32(); - var fPackage = reader.ReadI32(); - var fProtocol = reader.ReadI32(); - var fGuard = reader.ReadI32(); - var fDefer = reader.ReadI32(); - var fInout = reader.ReadI32(); - var fInternal = reader.ReadI32(); - var fWhen = reader.ReadI32(); - var fFun = reader.ReadI32(); - var fData = reader.ReadI32(); - var fLate = reader.ReadI32(); - var fPart = reader.ReadI32(); - var fOut = reader.ReadI32(); - var fRef = reader.ReadI32(); - var fEvent = reader.ReadI32(); - var fBase = reader.ReadI32(); - var fLambda = reader.ReadI32(); - var fYield = reader.ReadI32(); - var fNil = reader.ReadI32(); - var fNull = reader.ReadI32(); - var fBegin = reader.ReadI32(); - var fRescue = reader.ReadI32(); - var fModule = reader.ReadI32(); - var fChan = reader.ReadI32(); - var fGo = reader.ReadI32(); - var fLet = reader.ReadI32(); - var fExport = reader.ReadI32(); - var fDelete = reader.ReadI32(); - var fTypedef = reader.ReadI32(); - var fSizeof = reader.ReadI32(); - var fStatic = reader.ReadI32(); - var fSuper = reader.ReadI32(); - return new Keywords(fClass, fDef, fImport, fFrom, fFunc, fRange, fRegister, fVolatile, fNew, fDefault, fEnd, fIn, fIs, fInt, fVar, fVal, fSelf, fThis, fObject, fString, fCase, fSelect, fSwitch, fVoid, fUnion, fNamespace, fOperator, fTemplate, fPackage, fProtocol, fGuard, fDefer, fInout, fInternal, fWhen, fFun, fData, fLate, fPart, fOut, fRef, fEvent, fBase, fLambda, fYield, fNil, fNull, fBegin, fRescue, fModule, fChan, fGo, fLet, fExport, fDelete, fTypedef, fSizeof, fStatic, fSuper); + return new Keywords( + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32(), + reader.ReadI32()); } } @@ -623,8 +526,8 @@ public sealed class Retired internal static Retired ReadFrom(FfiBufferReader reader) { - var fId = reader.ReadI64(); - return new Retired(fId); + return new Retired( + reader.ReadI64()); } } @@ -651,67 +554,41 @@ public sealed class WidgetHolder internal void WriteTo(FfiBufferWriter writer) { writer.WriteObject(Primary.CloneHandle()); - writer.WriteBool(Fallback != null); - if (Fallback != null) - { - writer.WriteObject(Fallback!.CloneHandle()); - } - writer.WriteLen(Extras.Length); - foreach (var item0 in Extras) - { - writer.WriteObject(item0.CloneHandle()); - } - writer.WriteLen(ByName.Count); - foreach (var entry0 in ByName) - { - writer.WriteString(entry0.Key); - writer.WriteObject(entry0.Value.CloneHandle()); - } + FfiCodecs.WriteOptionalOfWidget(writer, Fallback); + FfiCodecs.WriteListOfWidget(writer, Extras); + FfiCodecs.WriteMapOfStringToWidget(writer, ByName); } internal static WidgetHolder ReadFrom(FfiBufferReader reader) { - var fPrimary = global::EdgeCases.Widget.Adopt(reader.ReadObject()); - Widget? fFallback = null; - if (reader.ReadBool()) - { - var fFallbackValue = global::EdgeCases.Widget.Adopt(reader.ReadObject()); - fFallback = fFallbackValue; - } - var fExtrasCount = reader.ReadLen(); - var fExtrasList = new List(reader.Capacity(fExtrasCount)); - for (var i0 = 0; i0 < fExtrasCount; i0++) - { - var fExtrasItem = global::EdgeCases.Widget.Adopt(reader.ReadObject()); - fExtrasList.Add(fExtrasItem); - } - var fExtras = fExtrasList.ToArray(); - var fByNameCount = reader.ReadLen(); - var fByName = new Dictionary(reader.Capacity(fByNameCount)); - for (var i0 = 0; i0 < fByNameCount; i0++) - { - var fByNameKey = reader.ReadString(); - var fByNameVal = global::EdgeCases.Widget.Adopt(reader.ReadObject()); - fByName[fByNameKey] = fByNameVal; - } - return new WidgetHolder(fPrimary, fFallback, fExtras, fByName); + return new WidgetHolder( + global::EdgeCases.Widget.Adopt(reader.ReadObject()), + FfiCodecs.ReadOptionalOfWidget(reader), + FfiCodecs.ReadListOfWidget(reader), + FfiCodecs.ReadMapOfStringToWidget(reader)); } } /// Callback interface carrying composite payloads and objects /// The native library may call any method from any thread until /// it releases its last reference to the implementation. Object arguments -/// belong to the implementation. An exception thrown by a method reaches -/// the native caller as . +/// belong to the implementation. An exception thrown by a method fails the +/// native library's call in progress: a method documented to throw a domain +/// exception reports it with its code and fields, and any other exception +/// reaches the library as . public interface IBatchListener { /// Composite payloads cross as value buffers void OnBatch(Outer[] items, byte[] raw, Outer? first, Mode mode); + /// Objects cross as strong references the consumer must release void OnWidget(Widget widget, Widget? fallback, Widget[] children); + void OnTick(); + /// Scalar return travels back through the vtable bool ShouldContinue(int @class); + /// Enum return travels back as its integer discriminant Mode PickMode(); } @@ -723,12 +600,14 @@ internal static unsafe class FfiVtable_edge_cases_BatchListener [StructLayout(LayoutKind.Sequential)] private struct Layout { + public uint Size; + public uint Flags; + public delegate* unmanaged[Cdecl] Free; public delegate* unmanaged[Cdecl] on_batch; public delegate* unmanaged[Cdecl] on_widget; public delegate* unmanaged[Cdecl] on_tick; public delegate* unmanaged[Cdecl] should_continue; public delegate* unmanaged[Cdecl] pick_mode; - public delegate* unmanaged[Cdecl] free; } /// The vtable's address, valid for the process lifetime. @@ -736,40 +615,30 @@ internal static unsafe class FfiVtable_edge_cases_BatchListener private static IntPtr Allocate() { - var vtable = (Layout*)NativeMemory.Alloc((nuint)sizeof(Layout)); + var vtable = (Layout*)NativeMemory.AllocZeroed((nuint)sizeof(Layout)); + vtable->Size = (uint)sizeof(Layout); + vtable->Flags = 0; + vtable->Free = &FreeTrampoline; vtable->on_batch = &OnBatchTrampoline; vtable->on_widget = &OnWidgetTrampoline; vtable->on_tick = &OnTickTrampoline; vtable->should_continue = &ShouldContinueTrampoline; vtable->pick_mode = &PickModeTrampoline; - vtable->free = &FreeTrampoline; return (IntPtr)vtable; } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] + private static void FreeTrampoline(IntPtr ctx) + { + Ffi.Unregister(ctx); + } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] private static void OnBatchTrampoline(IntPtr ctx, byte* items_ptr, nuint items_len, byte* raw_ptr, nuint raw_len, byte* first_ptr, nuint first_len, int mode, FfiError* out_err) { try { - var arg0Reader = new FfiBufferReader(Ffi.ReadBytes(items_ptr, items_len)); - var arg0Count = arg0Reader.ReadLen(); - var arg0List = new List(arg0Reader.Capacity(arg0Count)); - for (var i0 = 0; i0 < arg0Count; i0++) - { - var arg0Item = global::EdgeCases.Outer.ReadFrom(arg0Reader); - arg0List.Add(arg0Item); - } - var arg0 = arg0List.ToArray(); - arg0Reader.ExpectEnd(); - var arg2Reader = new FfiBufferReader(Ffi.ReadBytes(first_ptr, first_len)); - Outer? arg2 = null; - if (arg2Reader.ReadBool()) - { - var arg2Value = global::EdgeCases.Outer.ReadFrom(arg2Reader); - arg2 = arg2Value; - } - arg2Reader.ExpectEnd(); - Ffi.Target(ctx).OnBatch(arg0, Ffi.ReadBytes(raw_ptr, raw_len), arg2, (Mode)mode); + Ffi.Target(ctx).OnBatch(FfiCodecs.Decode(Ffi.ReadBuffer(items_ptr, items_len), static r => FfiCodecs.ReadListOfOuter(r)), Ffi.ReadBytes(raw_ptr, raw_len), FfiCodecs.Decode(Ffi.ReadBuffer(first_ptr, first_len), static r => FfiCodecs.ReadOptionalOfOuter(r)), (global::EdgeCases.Mode)mode); } catch (Exception e) { @@ -782,17 +651,7 @@ internal static unsafe class FfiVtable_edge_cases_BatchListener { try { - var arg2Reader = new FfiBufferReader(Ffi.ReadBytes(children_ptr, children_len)); - var arg2Count = arg2Reader.ReadLen(); - var arg2List = new List(arg2Reader.Capacity(arg2Count)); - for (var i0 = 0; i0 < arg2Count; i0++) - { - var arg2Item = global::EdgeCases.Widget.Adopt(arg2Reader.ReadObject()); - arg2List.Add(arg2Item); - } - var arg2 = arg2List.ToArray(); - arg2Reader.ExpectEnd(); - Ffi.Target(ctx).OnWidget(global::EdgeCases.Widget.Adopt(widget), fallback == IntPtr.Zero ? null : global::EdgeCases.Widget.Adopt(fallback), arg2); + Ffi.Target(ctx).OnWidget(global::EdgeCases.Widget.Adopt(widget), fallback == IntPtr.Zero ? null : global::EdgeCases.Widget.Adopt(fallback), FfiCodecs.Decode(Ffi.ReadBuffer(children_ptr, children_len), static r => FfiCodecs.ReadListOfWidget(r))); } catch (Exception e) { @@ -840,22 +699,19 @@ internal static unsafe class FfiVtable_edge_cases_BatchListener return default; } } - - [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] - private static void FreeTrampoline(IntPtr ctx) - { - Ffi.Unregister(ctx); - } } /// Callback interface whose name and methods are keywords /// The native library may call any method from any thread until /// it releases its last reference to the implementation. Object arguments -/// belong to the implementation. An exception thrown by a method reaches -/// the native caller as . +/// belong to the implementation. An exception thrown by a method fails the +/// native library's call in progress: a method documented to throw a domain +/// exception reports it with its code and fields, and any other exception +/// reaches the library as . public interface IKeyword { void Delete(int self); + int Import(); } @@ -866,9 +722,11 @@ internal static unsafe class FfiVtable_edge_cases_Keyword [StructLayout(LayoutKind.Sequential)] private struct Layout { + public uint Size; + public uint Flags; + public delegate* unmanaged[Cdecl] Free; public delegate* unmanaged[Cdecl] delete; public delegate* unmanaged[Cdecl] import; - public delegate* unmanaged[Cdecl] free; } /// The vtable's address, valid for the process lifetime. @@ -876,13 +734,21 @@ internal static unsafe class FfiVtable_edge_cases_Keyword private static IntPtr Allocate() { - var vtable = (Layout*)NativeMemory.Alloc((nuint)sizeof(Layout)); + var vtable = (Layout*)NativeMemory.AllocZeroed((nuint)sizeof(Layout)); + vtable->Size = (uint)sizeof(Layout); + vtable->Flags = 0; + vtable->Free = &FreeTrampoline; vtable->delete = &DeleteTrampoline; vtable->import = &ImportTrampoline; - vtable->free = &FreeTrampoline; return (IntPtr)vtable; } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] + private static void FreeTrampoline(IntPtr ctx) + { + Ffi.Unregister(ctx); + } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] private static void DeleteTrampoline(IntPtr ctx, int self, FfiError* out_err) { @@ -909,12 +775,6 @@ internal static unsafe class FfiVtable_edge_cases_Keyword return default; } } - - [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] - private static void FreeTrampoline(IntPtr ctx) - { - Ffi.Unregister(ctx); - } } /// Interface with no constructor; instances come from make_widget @@ -942,7 +802,7 @@ public sealed unsafe class Widget : IDisposable, IEquatable } /// Void method that can fail - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. public void Refresh() { var ffiErr = default(FfiError); @@ -954,23 +814,19 @@ public sealed unsafe class Widget : IDisposable, IEquatable { var classBytes = Ffi.Utf8(@class); var importWriter = new FfiBufferWriter(); - importWriter.WriteBool(import != null); - if (import != null) - { - importWriter.WriteString(import!); - } + FfiCodecs.WriteOptionalOfString(importWriter, import); var ffiErr = default(FfiError); nuint ffiOutLen = 0; fixed (byte* classPtr = classBytes, importPtr = importWriter.Written) { var ffiResult = NativeMethods.edge_cases_edge_cases_Widget_rename(Handle, classPtr, (nuint)classBytes.Length, importPtr, (nuint)importWriter.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } } /// Async method with a struct result - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. public Task Fetch(long id) { var ffiCall = new FfiCall(); @@ -997,10 +853,7 @@ public sealed unsafe class Widget : IDisposable, IEquatable ffiCall.SetError(err, global::EdgeCases.EdgeErrorsException.FromError); return; } - var ffiDecodedReader = Ffi.TakeBuffer(result_ptr, result_len); - var ffiDecoded = global::EdgeCases.Outer.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - ffiCall.SetResult(ffiDecoded); + ffiCall.SetResult(FfiCodecs.Decode(Ffi.TakeBuffer(result_ptr, result_len), static r => global::EdgeCases.Outer.ReadFrom(r))); } catch (Exception e) { @@ -1014,10 +867,27 @@ public sealed unsafe class Widget : IDisposable, IEquatable var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.edge_cases_edge_cases_Widget_old_name(Handle, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } + /// Spelled like the release method of several wrappers + public bool Close() + { + var ffiErr = default(FfiError); + var ffiResult = NativeMethods.edge_cases_edge_cases_Widget_close(Handle, &ffiErr); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return ffiResult != 0; + } + + /// Spelled like the release method of the Dart and .NET wrappers + public void Dispose_(bool force) + { + var ffiErr = default(FfiError); + NativeMethods.edge_cases_edge_cases_Widget_dispose(Handle, (byte)(force ? 1 : 0), &ffiErr); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + } + /// Releases this wrapper's reference. The native object is /// dropped once its last reference is released, and an in-flight /// call finishes first. @@ -1096,7 +966,7 @@ public sealed unsafe class Vault : IDisposable, IEquatable } /// Fails when the vault is busy - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. public static Vault Open(string path) { var pathBytes = Ffi.Utf8(path); @@ -1113,12 +983,21 @@ public sealed unsafe class Vault : IDisposable, IEquatable { var ffiErr = default(FfiError); var ffiResult = NativeMethods.edge_cases_edge_cases_Vault_peek(Handle, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return ffiResult == IntPtr.Zero ? null : global::EdgeCases.Widget.Adopt(ffiResult); } + /// Spelled like the handle accessor of several wrappers + public long Handle_() + { + var ffiErr = default(FfiError); + var ffiResult = NativeMethods.edge_cases_edge_cases_Vault_handle(Handle, &ffiErr); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return ffiResult; + } + /// Throwing static - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. public static bool Probe() { var ffiErr = default(FfiError); @@ -1127,6 +1006,16 @@ public sealed unsafe class Vault : IDisposable, IEquatable return ffiResult != 0; } + /// Static spelled like an inherited object member + public static string ToString_() + { + var ffiErr = default(FfiError); + nuint ffiOutLen = 0; + var ffiResult = NativeMethods.edge_cases_edge_cases_Vault_to_string(&ffiOutLen, &ffiErr); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return Ffi.TakeString(ffiResult, ffiOutLen); + } + /// Releases this wrapper's reference. The native object is /// dropped once its last reference is released, and an in-flight /// call finishes first. @@ -1186,7 +1075,7 @@ public static unsafe class EdgeCases { var ffiErr = default(FfiError); NativeMethods.edge_cases_edge_cases_noop(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); } /// Function named after a keyword whose parameters are all keywords @@ -1194,22 +1083,14 @@ public static unsafe class EdgeCases { var fromBytes = Ffi.Utf8(from); var funcWriter = new FfiBufferWriter(); - funcWriter.WriteLen(func.Length); - foreach (var item0 in func) - { - funcWriter.WriteString(item0); - } + FfiCodecs.WriteListOfString(funcWriter, func); var rangeWriter = new FfiBufferWriter(); - rangeWriter.WriteBool(range != null); - if (range != null) - { - rangeWriter.WriteI32(range.Value); - } + FfiCodecs.WriteOptionalOfI32(rangeWriter, range); var ffiErr = default(FfiError); fixed (byte* fromPtr = fromBytes, funcPtr = funcWriter.Written, rangePtr = rangeWriter.Written) { var ffiResult = NativeMethods.edge_cases_edge_cases_select(@class, fromPtr, (nuint)fromBytes.Length, funcPtr, (nuint)funcWriter.Length, rangePtr, (nuint)rangeWriter.Length, (byte)(type ? 1 : 0), &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return ffiResult; } } @@ -1218,106 +1099,38 @@ public static unsafe class EdgeCases public static Outer?[]? NestedOptional(Outer?[]? items) { var itemsWriter = new FfiBufferWriter(); - itemsWriter.WriteBool(items != null); - if (items != null) - { - itemsWriter.WriteLen(items!.Length); - foreach (var item0 in items!) - { - itemsWriter.WriteBool(item0 != null); - if (item0 != null) - { - item0!.WriteTo(itemsWriter); - } - } - } + FfiCodecs.WriteOptionalOfListOfOptionalOfOuter(itemsWriter, items); var ffiErr = default(FfiError); nuint ffiOutLen = 0; fixed (byte* itemsPtr = itemsWriter.Written) { var ffiResult = NativeMethods.edge_cases_edge_cases_nested_optional(itemsPtr, (nuint)itemsWriter.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - Outer?[]? ffiDecoded = null; - if (ffiDecodedReader.ReadBool()) - { - var ffiDecodedValueCount = ffiDecodedReader.ReadLen(); - var ffiDecodedValueList = new List(ffiDecodedReader.Capacity(ffiDecodedValueCount)); - for (var i0 = 0; i0 < ffiDecodedValueCount; i0++) - { - Outer? ffiDecodedValueItem = null; - if (ffiDecodedReader.ReadBool()) - { - var ffiDecodedValueItemValue = global::EdgeCases.Outer.ReadFrom(ffiDecodedReader); - ffiDecodedValueItem = ffiDecodedValueItemValue; - } - ffiDecodedValueList.Add(ffiDecodedValueItem); - } - var ffiDecodedValue = ffiDecodedValueList.ToArray(); - ffiDecoded = ffiDecodedValue; - } - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => FfiCodecs.ReadOptionalOfListOfOptionalOfOuter(r)); } } public static Mode? OptionalParams(string? text, int? count, Outer? outer, string[]? tags, byte[]? blob, Mode? mode) { var textWriter = new FfiBufferWriter(); - textWriter.WriteBool(text != null); - if (text != null) - { - textWriter.WriteString(text!); - } + FfiCodecs.WriteOptionalOfString(textWriter, text); var countWriter = new FfiBufferWriter(); - countWriter.WriteBool(count != null); - if (count != null) - { - countWriter.WriteI32(count.Value); - } + FfiCodecs.WriteOptionalOfI32(countWriter, count); var outerWriter = new FfiBufferWriter(); - outerWriter.WriteBool(outer != null); - if (outer != null) - { - outer!.WriteTo(outerWriter); - } + FfiCodecs.WriteOptionalOfOuter(outerWriter, outer); var tagsWriter = new FfiBufferWriter(); - tagsWriter.WriteBool(tags != null); - if (tags != null) - { - tagsWriter.WriteLen(tags!.Length); - foreach (var item0 in tags!) - { - tagsWriter.WriteString(item0); - } - } + FfiCodecs.WriteOptionalOfListOfString(tagsWriter, tags); var blobWriter = new FfiBufferWriter(); - blobWriter.WriteBool(blob != null); - if (blob != null) - { - blobWriter.WriteBytes(blob!); - } + FfiCodecs.WriteOptionalOfBytes(blobWriter, blob); var modeWriter = new FfiBufferWriter(); - modeWriter.WriteBool(mode != null); - if (mode != null) - { - modeWriter.WriteI32((int)mode.Value); - } + FfiCodecs.WriteOptionalOfMode(modeWriter, mode); var ffiErr = default(FfiError); nuint ffiOutLen = 0; fixed (byte* textPtr = textWriter.Written, countPtr = countWriter.Written, outerPtr = outerWriter.Written, tagsPtr = tagsWriter.Written, blobPtr = blobWriter.Written, modePtr = modeWriter.Written) { var ffiResult = NativeMethods.edge_cases_edge_cases_optional_params(textPtr, (nuint)textWriter.Length, countPtr, (nuint)countWriter.Length, outerPtr, (nuint)outerWriter.Length, tagsPtr, (nuint)tagsWriter.Length, blobPtr, (nuint)blobWriter.Length, modePtr, (nuint)modeWriter.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - Mode? ffiDecoded = null; - if (ffiDecodedReader.ReadBool()) - { - var ffiDecodedValue = (Mode)ffiDecodedReader.ReadI32(); - ffiDecoded = ffiDecodedValue; - } - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => FfiCodecs.ReadOptionalOfMode(r)); } } @@ -1337,35 +1150,18 @@ public static unsafe class EdgeCases Ffi.Unregister(keywordCtx); throw; } - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); } /// Objects in every buffered position public static WidgetHolder WidgetsEverywhere(Widget[] list, Dictionary byName, Widget?[] sparse, WidgetHolder holder) { var listWriter = new FfiBufferWriter(); - listWriter.WriteLen(list.Length); - foreach (var item0 in list) - { - listWriter.WriteObject(item0.CloneHandle()); - } + FfiCodecs.WriteListOfWidget(listWriter, list); var byNameWriter = new FfiBufferWriter(); - byNameWriter.WriteLen(byName.Count); - foreach (var entry0 in byName) - { - byNameWriter.WriteString(entry0.Key); - byNameWriter.WriteObject(entry0.Value.CloneHandle()); - } + FfiCodecs.WriteMapOfStringToWidget(byNameWriter, byName); var sparseWriter = new FfiBufferWriter(); - sparseWriter.WriteLen(sparse.Length); - foreach (var item0 in sparse) - { - sparseWriter.WriteBool(item0 != null); - if (item0 != null) - { - sparseWriter.WriteObject(item0!.CloneHandle()); - } - } + FfiCodecs.WriteListOfOptionalOfWidget(sparseWriter, sparse); var holderWriter = new FfiBufferWriter(); holder.WriteTo(holderWriter); var ffiErr = default(FfiError); @@ -1373,11 +1169,8 @@ public static unsafe class EdgeCases fixed (byte* listPtr = listWriter.Written, byNamePtr = byNameWriter.Written, sparsePtr = sparseWriter.Written, holderPtr = holderWriter.Written) { var ffiResult = NativeMethods.edge_cases_edge_cases_widgets_everywhere(listPtr, (nuint)listWriter.Length, byNamePtr, (nuint)byNameWriter.Length, sparsePtr, (nuint)sparseWriter.Length, holderPtr, (nuint)holderWriter.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::EdgeCases.WidgetHolder.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::EdgeCases.WidgetHolder.ReadFrom(r)); } } @@ -1389,7 +1182,7 @@ public static unsafe class EdgeCases { var ffiErr = default(FfiError); var ffiResult = NativeMethods.edge_cases_edge_cases_widget_stream(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.edge_cases_edge_cases_WidgetStreamIterator_destroy), &NextWidgetStream); } @@ -1398,7 +1191,7 @@ public static unsafe class EdgeCases IntPtr ffiItem = default; var ffiErr = default(FfiError); var ffiMore = NativeMethods.edge_cases_edge_cases_WidgetStreamIterator_next(iter, &ffiItem, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); if (ffiMore == 0) { item = default!; @@ -1409,7 +1202,7 @@ public static unsafe class EdgeCases } /// Async returning an object - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. public static Task AsyncWidget() { var ffiCall = new FfiCall(); @@ -1452,13 +1245,13 @@ public static unsafe class EdgeCases fixed (byte* labelPtr = labelBytes) { var ffiResult = NativeMethods.edge_cases_edge_cases_make_widget(labelPtr, (nuint)labelBytes.Length, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return global::EdgeCases.Widget.Adopt(ffiResult); } } /// Optional interface return - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. public static Widget? FindWidget(long id) { var ffiErr = default(FfiError); @@ -1473,7 +1266,7 @@ public static unsafe class EdgeCases var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.edge_cases_edge_cases_inspect_widget(widget.Handle, fallback?.Handle ?? global::EdgeCases.Widget.NativeHandle.Null, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } @@ -1485,7 +1278,7 @@ public static unsafe class EdgeCases { var ffiErr = default(FfiError); var ffiResult = NativeMethods.edge_cases_edge_cases_string_stream(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.edge_cases_edge_cases_StringStreamIterator_destroy), &NextStringStream); } @@ -1495,7 +1288,7 @@ public static unsafe class EdgeCases nuint ffiLen = default; var ffiErr = default(FfiError); var ffiMore = NativeMethods.edge_cases_edge_cases_StringStreamIterator_next(iter, &ffiItem, &ffiLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); if (ffiMore == 0) { item = default!; @@ -1513,7 +1306,7 @@ public static unsafe class EdgeCases { var ffiErr = default(FfiError); var ffiResult = NativeMethods.edge_cases_edge_cases_scalar_stream(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.edge_cases_edge_cases_ScalarStreamIterator_destroy), &NextScalarStream); } @@ -1522,7 +1315,7 @@ public static unsafe class EdgeCases int ffiItem = default; var ffiErr = default(FfiError); var ffiMore = NativeMethods.edge_cases_edge_cases_ScalarStreamIterator_next(iter, &ffiItem, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); if (ffiMore == 0) { item = default!; @@ -1540,7 +1333,7 @@ public static unsafe class EdgeCases { var ffiErr = default(FfiError); var ffiResult = NativeMethods.edge_cases_edge_cases_mode_stream(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.edge_cases_edge_cases_ModeStreamIterator_destroy), &NextModeStream); } @@ -1549,13 +1342,13 @@ public static unsafe class EdgeCases int ffiItem = default; var ffiErr = default(FfiError); var ffiMore = NativeMethods.edge_cases_edge_cases_ModeStreamIterator_next(iter, &ffiItem, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); if (ffiMore == 0) { item = default!; return false; } - item = (Mode)ffiItem; + item = (global::EdgeCases.Mode)ffiItem; return true; } @@ -1583,7 +1376,7 @@ public static unsafe class EdgeCases { if (err != null) { - ffiCall.SetError(err, global::EdgeCases.NativeException.FromError); + ffiCall.SetError(err, global::EdgeCases.NativeBugException.FromError); return; } ffiCall.SetResult(true); @@ -1617,7 +1410,7 @@ public static unsafe class EdgeCases { if (err != null) { - ffiCall.SetError(err, global::EdgeCases.NativeException.FromError); + ffiCall.SetError(err, global::EdgeCases.NativeBugException.FromError); return; } ffiCall.SetResult(Ffi.TakeBytes(result_ptr, result_len)); @@ -1628,7 +1421,7 @@ public static unsafe class EdgeCases } } - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. public static Task AsyncStruct() { var ffiCall = new FfiCall(); @@ -1655,10 +1448,7 @@ public static unsafe class EdgeCases ffiCall.SetError(err, global::EdgeCases.EdgeErrorsException.FromError); return; } - var ffiDecodedReader = Ffi.TakeBuffer(result_ptr, result_len); - var ffiDecoded = global::EdgeCases.Outer.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - ffiCall.SetResult(ffiDecoded); + ffiCall.SetResult(FfiCodecs.Decode(Ffi.TakeBuffer(result_ptr, result_len), static r => global::EdgeCases.Outer.ReadFrom(r))); } catch (Exception e) { @@ -1666,7 +1456,7 @@ public static unsafe class EdgeCases } } - /// The call reported a EdgeErrorsError code. + /// The call reported a EdgeErrors code. /// Cancels the native call; the task /// then completes as canceled. public static Task AsyncList(CancellationToken cancellationToken = default) @@ -1699,17 +1489,7 @@ public static unsafe class EdgeCases ffiCall.SetError(err, global::EdgeCases.EdgeErrorsException.FromError); return; } - var ffiDecodedReader = Ffi.TakeBuffer(result_ptr, result_len); - var ffiDecodedCount = ffiDecodedReader.ReadLen(); - var ffiDecodedList = new List(ffiDecodedReader.Capacity(ffiDecodedCount)); - for (var i0 = 0; i0 < ffiDecodedCount; i0++) - { - var ffiDecodedItem = global::EdgeCases.Outer.ReadFrom(ffiDecodedReader); - ffiDecodedList.Add(ffiDecodedItem); - } - var ffiDecoded = ffiDecodedList.ToArray(); - ffiDecodedReader.ExpectEnd(); - ffiCall.SetResult(ffiDecoded); + ffiCall.SetResult(FfiCodecs.Decode(Ffi.TakeBuffer(result_ptr, result_len), static r => FfiCodecs.ReadListOfOuter(r))); } catch (Exception e) { @@ -1740,18 +1520,10 @@ public static unsafe class EdgeCases { if (err != null) { - ffiCall.SetError(err, global::EdgeCases.NativeException.FromError); + ffiCall.SetError(err, global::EdgeCases.NativeBugException.FromError); return; } - var ffiDecodedReader = Ffi.TakeBuffer(result_ptr, result_len); - long? ffiDecoded = null; - if (ffiDecodedReader.ReadBool()) - { - var ffiDecodedValue = ffiDecodedReader.ReadI64(); - ffiDecoded = ffiDecodedValue; - } - ffiDecodedReader.ExpectEnd(); - ffiCall.SetResult(ffiDecoded); + ffiCall.SetResult(FfiCodecs.Decode(Ffi.TakeBuffer(result_ptr, result_len), static r => FfiCodecs.ReadOptionalOfI64(r))); } catch (Exception e) { @@ -1782,10 +1554,10 @@ public static unsafe class EdgeCases { if (err != null) { - ffiCall.SetError(err, global::EdgeCases.NativeException.FromError); + ffiCall.SetError(err, global::EdgeCases.NativeBugException.FromError); return; } - ffiCall.SetResult((Mode)result); + ffiCall.SetResult((global::EdgeCases.Mode)result); } catch (Exception e) { @@ -1820,7 +1592,7 @@ public static unsafe class EdgeCases { if (err != null) { - ffiCall.SetError(err, global::EdgeCases.NativeException.FromError); + ffiCall.SetError(err, global::EdgeCases.NativeBugException.FromError); return; } ffiCall.SetResult(Ffi.TakeString(result_ptr, result_len)); @@ -1840,11 +1612,8 @@ public static unsafe class EdgeCases fixed (byte* valuePtr = valueWriter.Written) { var ffiResult = NativeMethods.edge_cases_edge_cases_keywords_roundtrip(valuePtr, (nuint)valueWriter.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::EdgeCases.Keywords.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::EdgeCases.Keywords.ReadFrom(r)); } } @@ -1854,24 +1623,498 @@ public static unsafe class EdgeCases var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.edge_cases_edge_cases_legacy_value((int)value, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::EdgeCases.Retired.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::EdgeCases.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::EdgeCases.Retired.ReadFrom(r)); } } -internal static unsafe partial class NativeMethods +/// The value-buffer codec of every composite type the API +/// carries. +internal static class FfiCodecs { - static partial void VerifyChecksums() + /// Reads one value that must fill the whole buffer. + internal static T Decode(FfiBufferReader reader, Func read) + { + var value = read(reader); + reader.ExpectEnd(); + return value; + } + + internal static void WriteListOfI32(FfiBufferWriter writer, int[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + writer.WriteI32(item); + } + } + + internal static int[] ReadListOfI32(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(reader.ReadI32()); + } + return list.ToArray(); + } + + internal static void WriteListOfLeaf(FfiBufferWriter writer, Leaf[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + item.WriteTo(writer); + } + } + + internal static Leaf[] ReadListOfLeaf(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(global::EdgeCases.Leaf.ReadFrom(reader)); + } + return list.ToArray(); + } + + internal static void WriteListOfListOfI32(FfiBufferWriter writer, int[][] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + FfiCodecs.WriteListOfI32(writer, item); + } + } + + internal static int[][] ReadListOfListOfI32(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(FfiCodecs.ReadListOfI32(reader)); + } + return list.ToArray(); + } + + internal static void WriteListOfMode(FfiBufferWriter writer, Mode[] value) { - VerifyChecksum("edge_cases", &edge_cases_edge_cases_checksum, 0x1567f441073bd4aaUL); + writer.WriteLen(value.Length); + foreach (var item in value) + { + writer.WriteI32((int)item); + } + } + + internal static Mode[] ReadListOfMode(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add((global::EdgeCases.Mode)reader.ReadI32()); + } + return list.ToArray(); + } + + internal static void WriteListOfOptionalOfI64(FfiBufferWriter writer, long?[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + FfiCodecs.WriteOptionalOfI64(writer, item); + } + } + + internal static long?[] ReadListOfOptionalOfI64(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(FfiCodecs.ReadOptionalOfI64(reader)); + } + return list.ToArray(); + } + + internal static void WriteListOfOptionalOfOuter(FfiBufferWriter writer, Outer?[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + FfiCodecs.WriteOptionalOfOuter(writer, item); + } + } + + internal static Outer?[] ReadListOfOptionalOfOuter(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(FfiCodecs.ReadOptionalOfOuter(reader)); + } + return list.ToArray(); + } + + internal static void WriteListOfOptionalOfWidget(FfiBufferWriter writer, Widget?[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + FfiCodecs.WriteOptionalOfWidget(writer, item); + } + } + + internal static Widget?[] ReadListOfOptionalOfWidget(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(FfiCodecs.ReadOptionalOfWidget(reader)); + } + return list.ToArray(); + } + + internal static void WriteListOfOuter(FfiBufferWriter writer, Outer[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + item.WriteTo(writer); + } + } + + internal static Outer[] ReadListOfOuter(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(global::EdgeCases.Outer.ReadFrom(reader)); + } + return list.ToArray(); + } + + internal static void WriteListOfString(FfiBufferWriter writer, string[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + writer.WriteString(item); + } + } + + internal static string[] ReadListOfString(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(reader.ReadString()); + } + return list.ToArray(); + } + + internal static void WriteListOfWidget(FfiBufferWriter writer, Widget[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + writer.WriteObject(item.CloneHandle()); + } + } + + internal static Widget[] ReadListOfWidget(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(global::EdgeCases.Widget.Adopt(reader.ReadObject())); + } + return list.ToArray(); + } + + internal static void WriteMapOfI64ToString(FfiBufferWriter writer, Dictionary value) + { + writer.WriteLen(value.Count); + foreach (var entry in value) + { + writer.WriteI64(entry.Key); + writer.WriteString(entry.Value); + } + } + + internal static Dictionary ReadMapOfI64ToString(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var map = new Dictionary(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + var key = reader.ReadI64(); + if (!map.TryAdd(key, reader.ReadString())) + { + throw FfiBufferReader.Malformed("repeated map key"); + } + } + return map; + } + + internal static void WriteMapOfStringToInner(FfiBufferWriter writer, Dictionary value) + { + writer.WriteLen(value.Count); + foreach (var entry in value) + { + writer.WriteString(entry.Key); + entry.Value.WriteTo(writer); + } + } + + internal static Dictionary ReadMapOfStringToInner(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var map = new Dictionary(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + var key = reader.ReadString(); + if (!map.TryAdd(key, global::EdgeCases.Inner.ReadFrom(reader))) + { + throw FfiBufferReader.Malformed("repeated map key"); + } + } + return map; + } + + internal static void WriteMapOfStringToWidget(FfiBufferWriter writer, Dictionary value) + { + writer.WriteLen(value.Count); + foreach (var entry in value) + { + writer.WriteString(entry.Key); + writer.WriteObject(entry.Value.CloneHandle()); + } + } + + internal static Dictionary ReadMapOfStringToWidget(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var map = new Dictionary(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + var key = reader.ReadString(); + if (!map.TryAdd(key, global::EdgeCases.Widget.Adopt(reader.ReadObject()))) + { + throw FfiBufferReader.Malformed("repeated map key"); + } + } + return map; } - [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_checksum")] + internal static void WriteOptionalOfBytes(FfiBufferWriter writer, byte[]? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteBytes(present); + } + } + + internal static byte[]? ReadOptionalOfBytes(FfiBufferReader reader) + { + return reader.ReadBool() ? reader.ReadBytes() : null; + } + + internal static void WriteOptionalOfI32(FfiBufferWriter writer, int? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteI32(present); + } + } + + internal static int? ReadOptionalOfI32(FfiBufferReader reader) + { + return reader.ReadBool() ? reader.ReadI32() : null; + } + + internal static void WriteOptionalOfI64(FfiBufferWriter writer, long? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteI64(present); + } + } + + internal static long? ReadOptionalOfI64(FfiBufferReader reader) + { + return reader.ReadBool() ? reader.ReadI64() : null; + } + + internal static void WriteOptionalOfInner(FfiBufferWriter writer, Inner? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + present.WriteTo(writer); + } + } + + internal static Inner? ReadOptionalOfInner(FfiBufferReader reader) + { + return reader.ReadBool() ? global::EdgeCases.Inner.ReadFrom(reader) : null; + } + + internal static void WriteOptionalOfListOfOptionalOfOuter(FfiBufferWriter writer, Outer?[]? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + FfiCodecs.WriteListOfOptionalOfOuter(writer, present); + } + } + + internal static Outer?[]? ReadOptionalOfListOfOptionalOfOuter(FfiBufferReader reader) + { + return reader.ReadBool() ? FfiCodecs.ReadListOfOptionalOfOuter(reader) : null; + } + + internal static void WriteOptionalOfListOfString(FfiBufferWriter writer, string[]? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + FfiCodecs.WriteListOfString(writer, present); + } + } + + internal static string[]? ReadOptionalOfListOfString(FfiBufferReader reader) + { + return reader.ReadBool() ? FfiCodecs.ReadListOfString(reader) : null; + } + + internal static void WriteOptionalOfMode(FfiBufferWriter writer, Mode? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteI32((int)present); + } + } + + internal static Mode? ReadOptionalOfMode(FfiBufferReader reader) + { + return reader.ReadBool() ? (global::EdgeCases.Mode)reader.ReadI32() : null; + } + + internal static void WriteOptionalOfOuter(FfiBufferWriter writer, Outer? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + present.WriteTo(writer); + } + } + + internal static Outer? ReadOptionalOfOuter(FfiBufferReader reader) + { + return reader.ReadBool() ? global::EdgeCases.Outer.ReadFrom(reader) : null; + } + + internal static void WriteOptionalOfString(FfiBufferWriter writer, string? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteString(present); + } + } + + internal static string? ReadOptionalOfString(FfiBufferReader reader) + { + return reader.ReadBool() ? reader.ReadString() : null; + } + + internal static void WriteOptionalOfWidget(FfiBufferWriter writer, Widget? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteObject(present.CloneHandle()); + } + } + + internal static Widget? ReadOptionalOfWidget(FfiBufferReader reader) + { + return reader.ReadBool() ? global::EdgeCases.Widget.Adopt(reader.ReadObject()) : null; + } +} + +internal static unsafe partial class NativeMethods +{ + static partial void VerifyContracts() + { + VerifyContract("edge_cases_edge_cases_contract", &edge_cases_edge_cases_contract, new (ulong, ulong, string)[] + { + (0x056a86511785ba2bUL, 0x5cba3f16d57c6cdbUL, "edge_cases.make_widget"), + (0x07bfa9ecb2b720d2UL, 0x3813b3f57c64bf35UL, "edge_cases.EdgeErrors"), + (0x0aaa11867e37f991UL, 0xf090dd52146ebdf0UL, "edge_cases.Widget.refresh"), + (0x0f9701493898ae4cUL, 0x6944f47a275c96e8UL, "edge_cases.Inner"), + (0x1226b1adac179f7dUL, 0x0e95a00ab0bef637UL, "edge_cases.async_keywords"), + (0x16d36bef1541136cUL, 0x5ea7021ec324298fUL, "edge_cases.find_widget"), + (0x1b7324c55e980febUL, 0xabf87fe372358f42UL, "edge_cases.Widget.old_name"), + (0x1cbd731467c2bab8UL, 0x4915b9989c2c9074UL, "edge_cases.Vault"), + (0x2c2a6de67d894b0aUL, 0x50a4496b1e7191caUL, "edge_cases.Vault.open"), + (0x322b4cfcd2d6861cUL, 0xca7527a379ae8465UL, "edge_cases.Keywords"), + (0x3f8e23a5f32906f4UL, 0x8d4c908d796f130dUL, "edge_cases.Leaf"), + (0x42ab8a685457b97cUL, 0xe5ce30500cadb32dUL, "edge_cases.Vault.handle"), + (0x4a68540e2836d3d0UL, 0xa20e44c71e420ed2UL, "edge_cases.mode_stream"), + (0x4e030580ac0b111dUL, 0xb68b7a18d910e175UL, "edge_cases.legacy_value"), + (0x4f5e4e5291864ef3UL, 0x55a1a88b21a08f8bUL, "edge_cases.Legacy"), + (0x4f84b06171b4213dUL, 0xbaedcfdebb6a2820UL, "edge_cases.scalar_stream"), + (0x5067a5ff2d9ef910UL, 0xbfc79b18fc4714fcUL, "edge_cases.Widget"), + (0x59aff159112fbf5bUL, 0x18e4def57a8cfa49UL, "edge_cases.optional_params"), + (0x5daf65b98fa77c48UL, 0xc817a8cb202e59c2UL, "edge_cases.keywords_roundtrip"), + (0x5de8a5c9726c2c5aUL, 0x7c50b5b65f7a896dUL, "edge_cases.register_batch"), + (0x618a620309994c3cUL, 0x826ad0b1d208d723UL, "edge_cases.async_bytes"), + (0x648285cdba0a7b8eUL, 0xcf6e80e783a4d8beUL, "edge_cases.WidgetHolder"), + (0x649c78079824b10dUL, 0x5b12dd26e0634b69UL, "edge_cases.async_widget"), + (0x7da7a7357d122a37UL, 0xf544e025ffaa3d20UL, "edge_cases.async_void"), + (0x7e4015bc3a472e2dUL, 0xfe22291d90b82e4aUL, "edge_cases.widget_stream"), + (0x805276e5d81fdba7UL, 0xd9b64e360bd03b2bUL, "edge_cases.Outer"), + (0x80e1183326542df4UL, 0xe8a57e12cc6ed276UL, "edge_cases.select"), + (0x923ce5e9bcd769a1UL, 0x0b973f5dc07dda8bUL, "edge_cases.Vault.to_string"), + (0x98ca8c01f5239404UL, 0xfe11c93eb6b303efUL, "edge_cases.Widget.rename"), + (0xa13e22e98ea278d4UL, 0xf1fae215d7387800UL, "edge_cases.widgets_everywhere"), + (0xa84a8c9bf2a881e1UL, 0xbe1dc840bf221996UL, "edge_cases.async_optional"), + (0xad958c478fbf205dUL, 0xe386b0716d24f19bUL, "edge_cases.Retired"), + (0xb2fc3964ef56ab39UL, 0x73c325c8bc351236UL, "edge_cases.async_list"), + (0xb3403af521cf0006UL, 0x6abc30d48206a3baUL, "edge_cases.Widget.close"), + (0xb794304a22a406b8UL, 0xf6fb98711c516105UL, "edge_cases.async_struct"), + (0xba3936106a49c47dUL, 0xb7025f2fadd6288eUL, "edge_cases.Vault.peek"), + (0xc155b0c3cfb164f4UL, 0xc777ae57d92ebc47UL, "edge_cases.noop"), + (0xcdc18607777019ddUL, 0xb81a31e5c20aa7a8UL, "edge_cases.inspect_widget"), + (0xd164a82aeb64b8ccUL, 0xded02c68005c4737UL, "edge_cases.nested_optional"), + (0xd978ccad2468038fUL, 0x30c39c65d6235153UL, "edge_cases.Mode"), + (0xdbe9d947431c9124UL, 0x39537fb848fdf980UL, "edge_cases.BatchListener"), + (0xeae8b7862c08bb07UL, 0xfe12eeaa830452eaUL, "edge_cases.Keyword"), + (0xf1e155a05ae262b8UL, 0xb575eb818a423d42UL, "edge_cases.async_enum"), + (0xf22e210bdbce7343UL, 0x2a5e0f801bc8c768UL, "edge_cases.Widget.dispose"), + (0xf3c7cf675563ee6cUL, 0x0ecffeef399facc8UL, "edge_cases.Vault.probe"), + (0xfd34bc67acac379eUL, 0x576caa14f3c0901aUL, "edge_cases.Widget.fetch"), + (0xff2e9d2bff941786UL, 0x7e96764ef1f5fbaaUL, "edge_cases.string_stream"), + }); + } + + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_contract")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial ulong edge_cases_edge_cases_checksum(); + internal static partial FfiContractEntry* edge_cases_edge_cases_contract(nuint* out_len); [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_Widget_clone")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] @@ -1897,6 +2140,14 @@ internal static unsafe partial class NativeMethods [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial byte* edge_cases_edge_cases_Widget_old_name(Widget.NativeHandle self, nuint* out_len, FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_Widget_close")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial byte edge_cases_edge_cases_Widget_close(Widget.NativeHandle self, FfiError* out_err); + + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_Widget_dispose")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial void edge_cases_edge_cases_Widget_dispose(Widget.NativeHandle self, byte force, FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_Vault_clone")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial IntPtr edge_cases_edge_cases_Vault_clone(Vault.NativeHandle self); @@ -1913,10 +2164,18 @@ internal static unsafe partial class NativeMethods [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial IntPtr edge_cases_edge_cases_Vault_peek(Vault.NativeHandle self, FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_Vault_handle")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial long edge_cases_edge_cases_Vault_handle(Vault.NativeHandle self, FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_Vault_probe")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial byte edge_cases_edge_cases_Vault_probe(FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_Vault_to_string")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial byte* edge_cases_edge_cases_Vault_to_string(nuint* out_len, FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "edge_cases_edge_cases_noop")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void edge_cases_edge_cases_noop(FfiError* out_err); diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_csproj.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_csproj.snap deleted file mode 100644 index 7b8b128d..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__EdgeCases_csproj.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - - - - - net8.0 - EdgeCases - EdgeCases - EdgeCases - 0.1.0 - Native bindings for edge_cases - enable - true - true - - $(NoWarn);CS0618 - - - - - diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__README_md.snap deleted file mode 100644 index 83f75ad4..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__README_md.snap +++ /dev/null @@ -1,32 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# EdgeCases (.NET) - -Native bindings for edge_cases - -The `EdgeCases` namespace binds the `edge_cases` native library through -source-generated P/Invoke (`[LibraryImport]`), so it's trim- and -AOT-friendly. - -## Build - -```bash -dotnet build -``` - -Reference this project from yours (`dotnet add reference EdgeCases.csproj`) -or pack it (`dotnet pack -c Release`). - -## Loading the native library - -The bindings load `edge_cases` with the platform's normal rules -(`libedge_cases.dylib`, `libedge_cases.so`, or `edge_cases.dll` next to the app or on the -library search path). Set `EDGE_CASES_LIBRARY` to a full path to load a -specific file instead. - - diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__Runtime_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__Runtime_cs.snap deleted file mode 100644 index 51cdaa58..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_edge_cases__Runtime_cs.snap +++ /dev/null @@ -1,698 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#nullable enable - -using System; -using System.Collections; -using System.Collections.Generic; -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using System.Text; -using System.Threading; -using System.Threading.Tasks; - -namespace EdgeCases; - -/// Raised for a failure the native library reports that isn't a typed -/// domain error: producer bugs, panics, marshalling failures, and exceptions -/// thrown by a callback-interface implementation. Every typed domain -/// exception derives from it. -public class NativeException : Exception -{ - /// An untyped producer error. - public const int GenericErrorCode = -1; - /// The producer panicked; the message carries the panic text. - public const int PanicErrorCode = -2; - /// An argument couldn't be lifted by the producer. - public const int MarshalErrorCode = -3; - /// A callback-interface implementation threw; the message carries - /// the original exception's message. - public const int ForeignErrorCode = -4; - /// The call was cancelled. Surfaces as - /// , never as this type. - public const int CancelledErrorCode = -5; - - /// The error code the native library reported. - public int Code { get; } - - /// Creates an exception carrying a native error code. - public NativeException(int code, string message) : base(message) - { - Code = code; - } - - /// Maps a runtime error code to its exception: cancellation to - /// , anything else to this - /// type. - internal static Exception FromError(int code, string message, byte[]? payload) - { - if (code == CancelledErrorCode) - { - return new OperationCanceledException(message); - } - return new NativeException(code, message); - } -} - -/// Maps a native error code, message, and serialized payload to the -/// exception a call throws. -internal delegate Exception FfiErrorMap(int code, string message, byte[]? payload); - -/// The native edge_cases_error slot. -[StructLayout(LayoutKind.Sequential)] -internal unsafe struct FfiError -{ - public int Code; - public byte* Message; - public byte* PayloadPtr; - public nuint PayloadLen; -} - -/// Marshalling helpers shared by every generated call. -internal static unsafe class Ffi -{ - /// The UTF-8 bytes of a string argument. The native side borrows - /// them as a (pointer, length) pair, so interior NULs survive. - internal static byte[] Utf8(string value) - { - return Encoding.UTF8.GetBytes(value); - } - - /// Decodes a borrowed UTF-8 run; the native side keeps - /// ownership. - internal static string ReadString(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? "" : Encoding.UTF8.GetString(ptr, checked((int)len)); - } - - /// Copies a borrowed byte run; the native side keeps - /// ownership. - internal static byte[] ReadBytes(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? Array.Empty() : new ReadOnlySpan(ptr, checked((int)len)).ToArray(); - } - - /// Decodes a returned UTF-8 string and releases it. - internal static string TakeString(byte* ptr, nuint len) - { - try - { - return ReadString(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies returned bytes and releases them. - internal static byte[] TakeBytes(byte* ptr, nuint len) - { - try - { - return ReadBytes(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies a returned value buffer, releases it, and returns a - /// reader over the copy. - internal static FfiBufferReader TakeBuffer(byte* ptr, nuint len) - { - return new FfiBufferReader(TakeBytes(ptr, len)); - } - - /// Releases a caller-owned error slot and returns the exception - /// it described. - internal static Exception TakeError(FfiError* err, FfiErrorMap map) - { - var code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorClear(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - /// Releases a heap-boxed async error and returns the exception it - /// described. - internal static Exception TakeBoxedError(FfiError* err, FfiErrorMap map, out int code) - { - code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorFree(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - private static string ErrorMessage(FfiError* err) - { - return Marshal.PtrToStringUTF8((IntPtr)err->Message) ?? ""; - } - - /// Reports a callback-interface failure to the producer through - /// out_err. The producer copies the message. - internal static void SetForeignError(FfiError* err, Exception e) - { - var bytes = Encoding.UTF8.GetBytes(e.Message + "\0"); - fixed (byte* message = bytes) - { - NativeMethods.ErrorSet(err, NativeException.ForeignErrorCode, message); - } - } - - /// Pins a callback-interface implementation for the producer and - /// returns the ctx it hands back to every trampoline. The - /// producer releases it through the vtable's free. - internal static IntPtr Register(object implementation) - { - return GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); - } - - /// Releases a ctx the producer never received (the call - /// failed before reaching it) or no longer holds. - internal static void Unregister(IntPtr ctx) - { - GCHandle.FromIntPtr(ctx).Free(); - } - - /// The implementation behind a trampoline's ctx. - internal static T Target(IntPtr ctx) where T : class - { - return (T)GCHandle.FromIntPtr(ctx).Target!; - } -} - -/// Imports of the native runtime surface plus the load-time contract -/// check. The generated API declares its own imports in the other half of -/// this partial class. -internal static unsafe partial class NativeMethods -{ - /// The native library's base name. - internal const string LibName = "edge_cases"; - - /// The environment variable naming an explicit library - /// path. - internal const string LibraryEnvVar = "EDGE_CASES_LIBRARY"; - - /// The C ABI revision these bindings were generated - /// against. - internal const uint AbiVersion = 3; - - // Runs before the first native call through this class, so a library - // built for another ABI revision or another contract fails here instead - // of misreading memory later. - static NativeMethods() - { - try - { - NativeLibrary.SetDllImportResolver(typeof(NativeMethods).Assembly, Resolve); - } - catch (InvalidOperationException) - { - // The assembly already has a resolver; default probing still applies. - } - uint found; - try - { - found = AbiVersionOf(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't export edge_cases_abi_version; these bindings expect ABI revision {AbiVersion}", e); - } - if (found != AbiVersion) - { - throw new InvalidOperationException( - $"ABI mismatch: these bindings expect revision {AbiVersion} but the loaded {LibName} library reports revision {found}"); - } - VerifyChecksums(); - } - - static partial void VerifyChecksums(); - - private static IntPtr Resolve(string name, Assembly assembly, DllImportSearchPath? searchPath) - { - if (name != LibName) - { - return IntPtr.Zero; - } - var path = Environment.GetEnvironmentVariable(LibraryEnvVar); - return string.IsNullOrEmpty(path) ? IntPtr.Zero : NativeLibrary.Load(path); - } - - /// Fails the load when module was - /// generated from a different contract than the loaded library - /// implements. - private static void VerifyChecksum(string module, delegate* checksum, ulong expected) - { - ulong found; - try - { - found = checksum(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't implement module '{module}'", e); - } - if (found != expected) - { - throw new InvalidOperationException( - $"contract mismatch for module '{module}': these bindings expect checksum 0x{expected:x16} but the loaded {LibName} library reports 0x{found:x16}; regenerate the bindings for this library"); - } - } - - [LibraryImport(LibName, EntryPoint = "edge_cases_abi_version")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial uint AbiVersionOf(); - - [LibraryImport(LibName, EntryPoint = "edge_cases_error_set")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorSet(FfiError* err, int code, byte* message); - - [LibraryImport(LibName, EntryPoint = "edge_cases_error_clear")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorClear(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "edge_cases_error_free")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorFree(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "edge_cases_free_bytes")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void FreeBytes(byte* ptr, nuint len); - - [LibraryImport(LibName, EntryPoint = "edge_cases_cancel_token_create")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial IntPtr CancelTokenCreate(); - - [LibraryImport(LibName, EntryPoint = "edge_cases_cancel_token_cancel")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenCancel(IntPtr token); - - [LibraryImport(LibName, EntryPoint = "edge_cases_cancel_token_destroy")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenDestroy(IntPtr token); -} - -/// One pending async call: the task it completes, the native -/// context that finds it again, and, for a cancellable function, the -/// native cancel token linked to the caller's -/// . -internal sealed class FfiCall -{ - private readonly TaskCompletionSource _tcs = - new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - private readonly CancellationToken _cancellation; - private readonly object _gate = new object(); - private GCHandle _self; - private IntPtr _token; - private CancellationTokenRegistration _registration; - private bool _completed; - private bool _cancelling; - - internal FfiCall(CancellationToken cancellation = default) - { - _cancellation = cancellation; - _self = GCHandle.Alloc(this); - } - - /// The task the completion resolves. - internal Task Task => _tcs.Task; - - /// The context argument for the launcher. - internal IntPtr Context => GCHandle.ToIntPtr(_self); - - /// Creates the native cancel token for the launcher and cancels - /// it when the caller's fires. - internal IntPtr CancelToken() - { - _token = NativeMethods.CancelTokenCreate(); - if (_cancellation.CanBeCanceled) - { - _registration = _cancellation.Register(static state => ((FfiCall)state!).Cancel(), this); - } - return _token; - } - - /// Recovers the call behind a completion's context. Runs - /// exactly once per launch, on a producer thread. - internal static FfiCall Complete(IntPtr context) - { - var handle = GCHandle.FromIntPtr(context); - var call = (FfiCall)handle.Target!; - handle.Free(); - call.Release(); - return call; - } - - /// Releases everything when the launcher threw before the - /// producer took the call, so no completion will arrive. - internal void Abandon() - { - _self.Free(); - Release(); - } - - internal void SetResult(T value) - { - _tcs.TrySetResult(value); - } - - internal void SetException(Exception e) - { - _tcs.TrySetException(e); - } - - /// Faults (or cancels, for the cancelled code) the task from a - /// heap-boxed error, releasing the box. - internal unsafe void SetError(FfiError* err, FfiErrorMap map) - { - var e = Ffi.TakeBoxedError(err, map, out var code); - if (code == NativeException.CancelledErrorCode) - { - _tcs.TrySetCanceled(_cancellation); - } - else - { - _tcs.TrySetException(e); - } - } - - // The registration callback and the completion race: whichever runs - // last destroys the consumer's token reference, so a cancel never - // touches a destroyed token. - private void Cancel() - { - lock (_gate) - { - if (_completed) - { - return; - } - _cancelling = true; - } - NativeMethods.CancelTokenCancel(_token); - bool destroy; - lock (_gate) - { - _cancelling = false; - destroy = _completed; - } - if (destroy) - { - NativeMethods.CancelTokenDestroy(_token); - } - } - - private void Release() - { - if (_token == IntPtr.Zero) - { - return; - } - bool destroy; - lock (_gate) - { - _completed = true; - destroy = !_cancelling; - } - if (destroy) - { - _registration.Dispose(); - NativeMethods.CancelTokenDestroy(_token); - } - } -} - -/// Owns one native iterator and destroys it exactly once, when -/// enumeration ends, the enumerator is disposed, or the garbage collector -/// finalizes an abandoned one. -internal sealed unsafe class FfiIteratorHandle : SafeHandle -{ - private readonly delegate* _destroy; - - internal FfiIteratorHandle(IntPtr iterator, delegate* destroy) - : base(IntPtr.Zero, true) - { - _destroy = destroy; - SetHandle(iterator); - } - - public override bool IsInvalid => handle == IntPtr.Zero; - - protected override bool ReleaseHandle() - { - _destroy(handle); - return true; - } -} - -/// A lazily streamed sequence backed by a native iterator. It can be -/// enumerated once; each MoveNext pulls one item. -internal sealed unsafe class FfiSequence : IEnumerable, IEnumerator -{ - private readonly FfiIteratorHandle _iterator; - private readonly delegate* _next; - private int _started; - private T _current = default!; - - internal FfiSequence(FfiIteratorHandle iterator, delegate* next) - { - _iterator = iterator; - _next = next; - } - - public T Current => _current; - - object? IEnumerator.Current => _current; - - public IEnumerator GetEnumerator() - { - if (Interlocked.Exchange(ref _started, 1) != 0) - { - throw new InvalidOperationException("this sequence can be enumerated only once"); - } - return this; - } - - IEnumerator IEnumerable.GetEnumerator() - { - return GetEnumerator(); - } - - public bool MoveNext() - { - if (_iterator.IsClosed) - { - return false; - } - if (_next(_iterator, out _current)) - { - return true; - } - _iterator.Dispose(); - return false; - } - - public void Reset() - { - throw new NotSupportedException("a native iterator can't be reset"); - } - - public void Dispose() - { - _iterator.Dispose(); - } -} - -/// Serializes values into the value-buffer wire format -/// (little-endian, packed). -internal sealed class FfiBufferWriter -{ - private byte[] _buf = new byte[64]; - private int _len; - - /// The bytes written so far. - internal ReadOnlySpan Written => _buf.AsSpan(0, _len); - - /// The number of bytes written so far. - internal int Length => _len; - - private Span Grow(int extra) - { - if (_len + extra > _buf.Length) - { - Array.Resize(ref _buf, Math.Max(_buf.Length * 2, _len + extra)); - } - var span = _buf.AsSpan(_len, extra); - _len += extra; - return span; - } - - internal void WriteBool(bool v) => WriteU8(v ? (byte)1 : (byte)0); - - internal void WriteI8(sbyte v) => WriteU8((byte)v); - - internal void WriteU8(byte v) => Grow(1)[0] = v; - - internal void WriteI16(short v) => System.Buffers.Binary.BinaryPrimitives.WriteInt16LittleEndian(Grow(2), v); - - internal void WriteU16(ushort v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt16LittleEndian(Grow(2), v); - - internal void WriteI32(int v) => System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(Grow(4), v); - - internal void WriteU32(uint v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(Grow(4), v); - - internal void WriteI64(long v) => System.Buffers.Binary.BinaryPrimitives.WriteInt64LittleEndian(Grow(8), v); - - internal void WriteU64(ulong v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt64LittleEndian(Grow(8), v); - - internal void WriteF32(float v) => System.Buffers.Binary.BinaryPrimitives.WriteSingleLittleEndian(Grow(4), v); - - internal void WriteF64(double v) => System.Buffers.Binary.BinaryPrimitives.WriteDoubleLittleEndian(Grow(8), v); - - internal void WriteLen(int len) => WriteU32((uint)len); - - internal void WriteString(string v) - { - var count = Encoding.UTF8.GetByteCount(v); - WriteLen(count); - Encoding.UTF8.GetBytes(v, Grow(count)); - } - - internal void WriteBytes(byte[] v) - { - WriteLen(v.Length); - v.CopyTo(Grow(v.Length)); - } - - /// Writes an object token. The token must carry its own strong - /// reference (a fresh clone), which the reader adopts. - internal void WriteObject(IntPtr token) => WriteU64((ulong)(nuint)token); -} - -/// Decodes values from the value-buffer wire format. A malformed -/// buffer is a producer or consumer bug and throws -/// . -internal sealed class FfiBufferReader -{ - private static readonly Encoding Utf8Strict = new UTF8Encoding(false, true); - - private readonly byte[] _data; - private int _pos; - - internal FfiBufferReader(byte[] data) - { - _data = data; - } - - private static InvalidOperationException Malformed(string what) - { - return new InvalidOperationException("malformed value buffer: " + what); - } - - private ReadOnlySpan Take(int n) - { - if (_data.Length - _pos < n) - { - throw Malformed("buffer exhausted"); - } - var span = _data.AsSpan(_pos, n); - _pos += n; - return span; - } - - internal bool ReadBool() - { - var b = ReadU8(); - if (b > 1) - { - throw Malformed("invalid bool byte"); - } - return b == 1; - } - - internal sbyte ReadI8() => (sbyte)ReadU8(); - - internal byte ReadU8() => Take(1)[0]; - - internal short ReadI16() => System.Buffers.Binary.BinaryPrimitives.ReadInt16LittleEndian(Take(2)); - - internal ushort ReadU16() => System.Buffers.Binary.BinaryPrimitives.ReadUInt16LittleEndian(Take(2)); - - internal int ReadI32() => System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(Take(4)); - - internal uint ReadU32() => System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(Take(4)); - - internal long ReadI64() => System.Buffers.Binary.BinaryPrimitives.ReadInt64LittleEndian(Take(8)); - - internal ulong ReadU64() => System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(Take(8)); - - internal float ReadF32() => System.Buffers.Binary.BinaryPrimitives.ReadSingleLittleEndian(Take(4)); - - internal double ReadF64() => System.Buffers.Binary.BinaryPrimitives.ReadDoubleLittleEndian(Take(8)); - - /// A collection count. Elements can be zero-sized, so the count - /// isn't checked against the remaining bytes; callers cap their - /// preallocation with . - internal int ReadLen() - { - var len = ReadU32(); - if (len > int.MaxValue) - { - throw Malformed("length prefix out of range"); - } - return (int)len; - } - - /// A preallocation size for elements - /// that a malformed count can't inflate. - internal int Capacity(int count) => Math.Min(count, _data.Length - _pos); - - internal string ReadString() - { - var bytes = Take(ReadLen()); - try - { - return Utf8Strict.GetString(bytes); - } - catch (ArgumentException) - { - throw Malformed("string is not valid UTF-8"); - } - } - - internal byte[] ReadBytes() => Take(ReadLen()).ToArray(); - - /// Reads an object token carrying one strong reference. - internal IntPtr ReadObject() - { - var token = ReadU64(); - if (token == 0) - { - throw Malformed("null object token"); - } - return (IntPtr)(nint)token; - } - - internal void ExpectEnd() - { - if (_pos != _data.Length) - { - throw Malformed("trailing bytes"); - } - } -} - -// END Runtime.cs diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_cs.snap index 8586ed9e..6b7ff4ed 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_cs.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_cs.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #nullable enable @@ -16,28 +16,56 @@ using System.Threading.Tasks; namespace KitchenSink; -/// Typed exception for the KitchenErrorsError error domain (module kitchen). -public class KitchenErrorsException : NativeException +/// The errors of the KitchenErrors domain, declared by module kitchen. +/// Catch a nested class for one code, or this class for any of them. +public abstract class KitchenErrorsException : NativeException { + private protected KitchenErrorsException(int code, string message) : base(code, message) + { + } + + /// Writes this code's fields, the error payload a callback + /// reports with it. + internal virtual void WritePayload(FfiBufferWriter writer) + { + } + /// Item not found - public const int NotFound = 1; + public sealed class NotFound : KitchenErrorsException + { + /// The domain code, 1. + public const int ErrorCode = 1; - /// Invalid input - public const int InvalidInput = 2; + /// Creates the error with , or the + /// default message "Item not found" when it's null. + public NotFound(string? message = null) : base(ErrorCode, message ?? "Item not found") + { + } + } - /// Creates an exception carrying a domain error code. - public KitchenErrorsException(int code, string message) : base(code, message) + /// Invalid input + public sealed class InvalidInput : KitchenErrorsException { + /// The domain code, 2. + public const int ErrorCode = 2; + + /// Creates the error with , or the + /// default message "Invalid input" when it's null. + public InvalidInput(string? message = null) : base(ErrorCode, message ?? "Invalid input") + { + } } + /// The exception for a raw code and its serialized payload. internal static new Exception FromError(int code, string message, byte[]? payload) { + var given = message.Length == 0 ? null : message; switch (code) { - case NotFound: - return new KitchenErrorsException(code, string.IsNullOrEmpty(message) ? "Item not found" : message); - case InvalidInput: - return new KitchenErrorsException(code, string.IsNullOrEmpty(message) ? "Invalid input" : message); + case NotFound.ErrorCode: + return new NotFound(given); + case InvalidInput.ErrorCode: + return new InvalidInput(given); default: return global::KitchenSink.NativeException.FromError(code, message, payload); } @@ -66,9 +94,9 @@ public sealed class Token internal static Token ReadFrom(FfiBufferReader reader) { - var fId = reader.ReadI64(); - var fLabel = reader.ReadString(); - return new Token(fId, fLabel); + return new Token( + reader.ReadI64(), + reader.ReadString()); } } @@ -146,85 +174,57 @@ public sealed class Item writer.WriteF64(Ratio); writer.WriteU32(Nick); writer.WriteBytes(Payload); - writer.WriteLen(Tags.Length); - foreach (var item0 in Tags) - { - writer.WriteString(item0); - } - writer.WriteLen(Attrs.Count); - foreach (var entry0 in Attrs) - { - writer.WriteString(entry0.Key); - writer.WriteString(entry0.Value); - } - writer.WriteBool(Parent != null); - if (Parent != null) - { - writer.WriteI64(Parent.Value); - } + FfiCodecs.WriteListOfString(writer, Tags); + FfiCodecs.WriteMapOfStringToString(writer, Attrs); + FfiCodecs.WriteOptionalOfI64(writer, Parent); Token.WriteTo(writer); writer.WriteI32((int)Priority); - writer.WriteBool(Gadget != null); - if (Gadget != null) - { - writer.WriteObject(Gadget!.CloneHandle()); - } + FfiCodecs.WriteOptionalOfGadget(writer, Gadget); } internal static Item ReadFrom(FfiBufferReader reader) { - var fId = reader.ReadI64(); - var fName = reader.ReadString(); - var fCount = reader.ReadI32(); - var fEnabled = reader.ReadBool(); - var fRatio = reader.ReadF64(); - var fNick = reader.ReadU32(); - var fPayload = reader.ReadBytes(); - var fTagsCount = reader.ReadLen(); - var fTagsList = new List(reader.Capacity(fTagsCount)); - for (var i0 = 0; i0 < fTagsCount; i0++) - { - var fTagsItem = reader.ReadString(); - fTagsList.Add(fTagsItem); - } - var fTags = fTagsList.ToArray(); - var fAttrsCount = reader.ReadLen(); - var fAttrs = new Dictionary(reader.Capacity(fAttrsCount)); - for (var i0 = 0; i0 < fAttrsCount; i0++) - { - var fAttrsKey = reader.ReadString(); - var fAttrsVal = reader.ReadString(); - fAttrs[fAttrsKey] = fAttrsVal; - } - long? fParent = null; - if (reader.ReadBool()) - { - var fParentValue = reader.ReadI64(); - fParent = fParentValue; - } - var fToken = global::KitchenSink.Token.ReadFrom(reader); - var fPriority = (Priority)reader.ReadI32(); - Gadget? fGadget = null; - if (reader.ReadBool()) - { - var fGadgetValue = global::KitchenSink.Gadget.Adopt(reader.ReadObject()); - fGadget = fGadgetValue; - } - return new Item(fId, fName, fCount, fEnabled, fRatio, fNick, fPayload, fTags, fAttrs, fParent, fToken, fPriority, fGadget); + return new Item( + reader.ReadI64(), + reader.ReadString(), + reader.ReadI32(), + reader.ReadBool(), + reader.ReadF64(), + reader.ReadU32(), + reader.ReadBytes(), + FfiCodecs.ReadListOfString(reader), + FfiCodecs.ReadMapOfStringToString(reader), + FfiCodecs.ReadOptionalOfI64(reader), + global::KitchenSink.Token.ReadFrom(reader), + (global::KitchenSink.Priority)reader.ReadI32(), + FfiCodecs.ReadOptionalOfGadget(reader)); } } /// Consumer-implemented observer of ready items /// The native library may call any method from any thread until /// it releases its last reference to the implementation. Object arguments -/// belong to the implementation. An exception thrown by a method reaches -/// the native caller as . +/// belong to the implementation. An exception thrown by a method fails the +/// native library's call in progress: a method documented to throw a domain +/// exception reports it with its code and fields, and any other exception +/// reaches the library as . public interface IReadyListener { /// Fires when an item is ready void OnReady(int code, string msg); + /// Receives the item itself and says whether to keep listening bool OnItem(Item item, Gadget gadget); + + /// A display label, failing with a kitchen error + /// Reported to the library with its code and fields. + string Label(); + + /// The most recent item the listener kept, if any + Item? Latest(); + + /// The listener's favorite gadget + Gadget Favorite(); } /// The process-wide kitchen_sink_kitchen_ReadyListener_vtable and the trampolines behind it, @@ -234,9 +234,14 @@ internal static unsafe class FfiVtable_kitchen_ReadyListener [StructLayout(LayoutKind.Sequential)] private struct Layout { + public uint Size; + public uint Flags; + public delegate* unmanaged[Cdecl] Free; public delegate* unmanaged[Cdecl] on_ready; public delegate* unmanaged[Cdecl] on_item; - public delegate* unmanaged[Cdecl] free; + public delegate* unmanaged[Cdecl] label; + public delegate* unmanaged[Cdecl] latest; + public delegate* unmanaged[Cdecl] favorite; } /// The vtable's address, valid for the process lifetime. @@ -244,13 +249,24 @@ internal static unsafe class FfiVtable_kitchen_ReadyListener private static IntPtr Allocate() { - var vtable = (Layout*)NativeMemory.Alloc((nuint)sizeof(Layout)); + var vtable = (Layout*)NativeMemory.AllocZeroed((nuint)sizeof(Layout)); + vtable->Size = (uint)sizeof(Layout); + vtable->Flags = 0; + vtable->Free = &FreeTrampoline; vtable->on_ready = &OnReadyTrampoline; vtable->on_item = &OnItemTrampoline; - vtable->free = &FreeTrampoline; + vtable->label = &LabelTrampoline; + vtable->latest = &LatestTrampoline; + vtable->favorite = &FavoriteTrampoline; return (IntPtr)vtable; } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] + private static void FreeTrampoline(IntPtr ctx) + { + Ffi.Unregister(ctx); + } + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] private static void OnReadyTrampoline(IntPtr ctx, int code, byte* msg_ptr, nuint msg_len, FfiError* out_err) { @@ -269,10 +285,7 @@ internal static unsafe class FfiVtable_kitchen_ReadyListener { try { - var arg0Reader = new FfiBufferReader(Ffi.ReadBytes(item_ptr, item_len)); - var arg0 = global::KitchenSink.Item.ReadFrom(arg0Reader); - arg0Reader.ExpectEnd(); - return (byte)(Ffi.Target(ctx).OnItem(arg0, global::KitchenSink.Gadget.Adopt(gadget)) ? 1 : 0); + return (byte)(Ffi.Target(ctx).OnItem(FfiCodecs.Decode(Ffi.ReadBuffer(item_ptr, item_len), static r => global::KitchenSink.Item.ReadFrom(r)), global::KitchenSink.Gadget.Adopt(gadget)) ? 1 : 0); } catch (Exception e) { @@ -282,9 +295,50 @@ internal static unsafe class FfiVtable_kitchen_ReadyListener } [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] - private static void FreeTrampoline(IntPtr ctx) + private static void LabelTrampoline(IntPtr ctx, byte** out_ptr, nuint* out_len, FfiError* out_err) { - Ffi.Unregister(ctx); + try + { + Ffi.ReturnString(Ffi.Target(ctx).Label(), out_ptr, out_len); + } + catch (global::KitchenSink.KitchenErrorsException e) + { + Ffi.SetDomainError(out_err, e.Code, e, e.WritePayload); + } + catch (Exception e) + { + Ffi.SetForeignError(out_err, e); + } + } + + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] + private static void LatestTrampoline(IntPtr ctx, byte** out_ptr, nuint* out_len, FfiError* out_err) + { + try + { + var ffiResult = Ffi.Target(ctx).Latest(); + var ffiWriter = new FfiBufferWriter(); + FfiCodecs.WriteOptionalOfItem(ffiWriter, ffiResult); + Ffi.ReturnBuffer(ffiWriter, out_ptr, out_len); + } + catch (Exception e) + { + Ffi.SetForeignError(out_err, e); + } + } + + [UnmanagedCallersOnly(CallConvs = new[] { typeof(CallConvCdecl) })] + private static IntPtr FavoriteTrampoline(IntPtr ctx, FfiError* out_err) + { + try + { + return Ffi.Target(ctx).Favorite()?.CloneHandle() ?? IntPtr.Zero; + } + catch (Exception e) + { + Ffi.SetForeignError(out_err, e); + return default; + } } } @@ -317,7 +371,7 @@ public sealed unsafe class Gadget : IDisposable, IEquatable { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_Gadget_new(id, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); Handle = new NativeHandle(ffiResult); } @@ -327,12 +381,12 @@ public sealed unsafe class Gadget : IDisposable, IEquatable var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_Gadget_describe(Handle, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } /// Poke the gadget a number of times, failing on a negative count - /// The call reported a KitchenErrorsError code. + /// The call reported a KitchenErrors code. public int Poke(int times) { var ffiErr = default(FfiError); @@ -347,7 +401,7 @@ public sealed unsafe class Gadget : IDisposable, IEquatable var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_Gadget_version(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } @@ -411,7 +465,7 @@ public static unsafe class Shared var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_shared_ping(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } } @@ -423,7 +477,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_bool_id((byte)(x ? 1 : 0), &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult != 0; } @@ -431,7 +485,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_i32_id(x, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult; } @@ -439,7 +493,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_u32_id(x, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult; } @@ -447,7 +501,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_i64_id(x, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult; } @@ -455,7 +509,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_f64_id(x, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult; } @@ -467,7 +521,7 @@ public static unsafe class Kitchen fixed (byte* sPtr = sBytes) { var ffiResult = NativeMethods.kitchen_sink_kitchen_echo_string(sPtr, (nuint)sBytes.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } } @@ -479,7 +533,7 @@ public static unsafe class Kitchen fixed (byte* bPtr = b) { var ffiResult = NativeMethods.kitchen_sink_kitchen_echo_bytes(bPtr, (nuint)b.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return Ffi.TakeBytes(ffiResult, ffiOutLen); } } @@ -501,16 +555,35 @@ public static unsafe class Kitchen Ffi.Unregister(listenerCtx); throw; } - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult; } + /// Register a listener if one is given + public static bool MaybeSubscribe(IReadyListener? listener) + { + var listenerCtx = Ffi.Register(listener); + var ffiErr = default(FfiError); + byte ffiResult; + try + { + ffiResult = NativeMethods.kitchen_sink_kitchen_maybe_subscribe(listenerCtx, listener == null ? IntPtr.Zero : FfiVtable_kitchen_ReadyListener.Pointer, &ffiErr); + } + catch + { + Ffi.Unregister(listenerCtx); + throw; + } + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); + return ffiResult != 0; + } + /// Optional object in both directions public static Gadget? MaybeGadget(Gadget? fallback) { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_maybe_gadget(fallback?.Handle ?? global::KitchenSink.Gadget.NativeHandle.Null, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult == IntPtr.Zero ? null : global::KitchenSink.Gadget.Adopt(ffiResult); } @@ -520,18 +593,8 @@ public static unsafe class Kitchen var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_gadgets(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecodedCount = ffiDecodedReader.ReadLen(); - var ffiDecodedList = new List(ffiDecodedReader.Capacity(ffiDecodedCount)); - for (var i0 = 0; i0 < ffiDecodedCount; i0++) - { - var ffiDecodedItem = global::KitchenSink.Gadget.Adopt(ffiDecodedReader.ReadObject()); - ffiDecodedList.Add(ffiDecodedItem); - } - var ffiDecoded = ffiDecodedList.ToArray(); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => FfiCodecs.ReadListOfGadget(r)); } /// An iterator over objects @@ -542,7 +605,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_stream_gadgets(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.kitchen_sink_kitchen_StreamGadgetsIterator_destroy), &NextStreamGadgets); } @@ -551,7 +614,7 @@ public static unsafe class Kitchen IntPtr ffiItem = default; var ffiErr = default(FfiError); var ffiMore = NativeMethods.kitchen_sink_kitchen_StreamGadgetsIterator_next(iter, &ffiItem, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); if (ffiMore == 0) { item = default!; @@ -562,22 +625,14 @@ public static unsafe class Kitchen } /// Optional struct return - /// The call reported a KitchenErrorsError code. + /// The call reported a KitchenErrors code. public static Item? MaybeItem(long id) { var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_maybe_item(id, &ffiOutLen, &ffiErr); if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.KitchenErrorsException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - Item? ffiDecoded = null; - if (ffiDecodedReader.ReadBool()) - { - var ffiDecodedValue = global::KitchenSink.Item.ReadFrom(ffiDecodedReader); - ffiDecoded = ffiDecodedValue; - } - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => FfiCodecs.ReadOptionalOfItem(r)); } /// List of structs @@ -586,18 +641,8 @@ public static unsafe class Kitchen var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_list_items(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecodedCount = ffiDecodedReader.ReadLen(); - var ffiDecodedList = new List(ffiDecodedReader.Capacity(ffiDecodedCount)); - for (var i0 = 0; i0 < ffiDecodedCount; i0++) - { - var ffiDecodedItem = global::KitchenSink.Item.ReadFrom(ffiDecodedReader); - ffiDecodedList.Add(ffiDecodedItem); - } - var ffiDecoded = ffiDecodedList.ToArray(); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => FfiCodecs.ReadListOfItem(r)); } /// Map return type @@ -606,18 +651,8 @@ public static unsafe class Kitchen var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_get_attrs(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecodedCount = ffiDecodedReader.ReadLen(); - var ffiDecoded = new Dictionary(ffiDecodedReader.Capacity(ffiDecodedCount)); - for (var i0 = 0; i0 < ffiDecodedCount; i0++) - { - var ffiDecodedKey = ffiDecodedReader.ReadString(); - var ffiDecodedVal = ffiDecodedReader.ReadI32(); - ffiDecoded[ffiDecodedKey] = ffiDecodedVal; - } - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => FfiCodecs.ReadMapOfStringToI32(r)); } /// Iterator return type @@ -628,7 +663,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_stream_items(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.kitchen_sink_kitchen_StreamItemsIterator_destroy), &NextStreamItems); } @@ -638,16 +673,13 @@ public static unsafe class Kitchen nuint ffiLen = default; var ffiErr = default(FfiError); var ffiMore = NativeMethods.kitchen_sink_kitchen_StreamItemsIterator_next(iter, &ffiItem, &ffiLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); if (ffiMore == 0) { item = default!; return false; } - var ffiDecodedReader = Ffi.TakeBuffer(ffiItem, ffiLen); - var ffiDecoded = global::KitchenSink.Item.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - item = ffiDecoded; + item = FfiCodecs.Decode(Ffi.TakeBuffer(ffiItem, ffiLen), static r => global::KitchenSink.Item.ReadFrom(r)); return true; } @@ -657,11 +689,8 @@ public static unsafe class Kitchen var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_cross_module_token(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::KitchenSink.Token.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::KitchenSink.Token.ReadFrom(r)); } /// Async operation @@ -692,7 +721,7 @@ public static unsafe class Kitchen { if (err != null) { - ffiCall.SetError(err, global::KitchenSink.NativeException.FromError); + ffiCall.SetError(err, global::KitchenSink.NativeBugException.FromError); return; } ffiCall.SetResult(Ffi.TakeString(result_ptr, result_len)); @@ -737,7 +766,7 @@ public static unsafe class Kitchen { if (err != null) { - ffiCall.SetError(err, global::KitchenSink.NativeException.FromError); + ffiCall.SetError(err, global::KitchenSink.NativeBugException.FromError); return; } ffiCall.SetResult(Ffi.TakeString(result_ptr, result_len)); @@ -754,7 +783,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_legacy_op(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult; } @@ -763,7 +792,7 @@ public static unsafe class Kitchen { var ffiErr = default(FfiError); var ffiResult = NativeMethods.kitchen_sink_kitchen_new_op(&ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return ffiResult; } } @@ -776,26 +805,228 @@ public static unsafe class KitchenNested var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.kitchen_sink_kitchen_nested_hello(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } } +/// The value-buffer codec of every composite type the API +/// carries. +internal static class FfiCodecs +{ + /// Reads one value that must fill the whole buffer. + internal static T Decode(FfiBufferReader reader, Func read) + { + var value = read(reader); + reader.ExpectEnd(); + return value; + } + + internal static void WriteListOfGadget(FfiBufferWriter writer, Gadget[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + writer.WriteObject(item.CloneHandle()); + } + } + + internal static Gadget[] ReadListOfGadget(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(global::KitchenSink.Gadget.Adopt(reader.ReadObject())); + } + return list.ToArray(); + } + + internal static void WriteListOfItem(FfiBufferWriter writer, Item[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + item.WriteTo(writer); + } + } + + internal static Item[] ReadListOfItem(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(global::KitchenSink.Item.ReadFrom(reader)); + } + return list.ToArray(); + } + + internal static void WriteListOfString(FfiBufferWriter writer, string[] value) + { + writer.WriteLen(value.Length); + foreach (var item in value) + { + writer.WriteString(item); + } + } + + internal static string[] ReadListOfString(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var list = new List(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + list.Add(reader.ReadString()); + } + return list.ToArray(); + } + + internal static void WriteMapOfStringToI32(FfiBufferWriter writer, Dictionary value) + { + writer.WriteLen(value.Count); + foreach (var entry in value) + { + writer.WriteString(entry.Key); + writer.WriteI32(entry.Value); + } + } + + internal static Dictionary ReadMapOfStringToI32(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var map = new Dictionary(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + var key = reader.ReadString(); + if (!map.TryAdd(key, reader.ReadI32())) + { + throw FfiBufferReader.Malformed("repeated map key"); + } + } + return map; + } + + internal static void WriteMapOfStringToString(FfiBufferWriter writer, Dictionary value) + { + writer.WriteLen(value.Count); + foreach (var entry in value) + { + writer.WriteString(entry.Key); + writer.WriteString(entry.Value); + } + } + + internal static Dictionary ReadMapOfStringToString(FfiBufferReader reader) + { + var count = reader.ReadLen(); + var map = new Dictionary(reader.Capacity(count)); + for (var i = 0; i < count; i++) + { + var key = reader.ReadString(); + if (!map.TryAdd(key, reader.ReadString())) + { + throw FfiBufferReader.Malformed("repeated map key"); + } + } + return map; + } + + internal static void WriteOptionalOfGadget(FfiBufferWriter writer, Gadget? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteObject(present.CloneHandle()); + } + } + + internal static Gadget? ReadOptionalOfGadget(FfiBufferReader reader) + { + return reader.ReadBool() ? global::KitchenSink.Gadget.Adopt(reader.ReadObject()) : null; + } + + internal static void WriteOptionalOfI64(FfiBufferWriter writer, long? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + writer.WriteI64(present); + } + } + + internal static long? ReadOptionalOfI64(FfiBufferReader reader) + { + return reader.ReadBool() ? reader.ReadI64() : null; + } + + internal static void WriteOptionalOfItem(FfiBufferWriter writer, Item? value) + { + writer.WriteBool(value != null); + if (value is { } present) + { + present.WriteTo(writer); + } + } + + internal static Item? ReadOptionalOfItem(FfiBufferReader reader) + { + return reader.ReadBool() ? global::KitchenSink.Item.ReadFrom(reader) : null; + } +} + internal static unsafe partial class NativeMethods { - static partial void VerifyChecksums() + static partial void VerifyContracts() { - VerifyChecksum("shared", &kitchen_sink_shared_checksum, 0x42c4ce2c8d0af052UL); - VerifyChecksum("kitchen", &kitchen_sink_kitchen_checksum, 0xd83da75b24b545ffUL); + VerifyContract("kitchen_sink_shared_contract", &kitchen_sink_shared_contract, new (ulong, ulong, string)[] + { + (0x709e234caa962446UL, 0x3894f2ca452e9f98UL, "shared.ping"), + (0xcfb306fa5801ee5bUL, 0x1cbd9e10983cac2cUL, "shared.Token"), + }); + VerifyContract("kitchen_sink_kitchen_contract", &kitchen_sink_kitchen_contract, new (ulong, ulong, string)[] + { + (0x01d4f09bbddc986fUL, 0x29eb88fecc66544dUL, "kitchen.Gadget.new"), + (0x10472d8fe748c2eeUL, 0xb265832b6a759fc4UL, "kitchen.maybe_gadget"), + (0x11368eec07a74078UL, 0xfa049f78b77e146aUL, "kitchen.gadgets"), + (0x137f08a29cde6021UL, 0xda8de7aeb2dbb8a3UL, "kitchen.stream_gadgets"), + (0x2111b54980826dd7UL, 0x452783f3e6ca6e1dUL, "kitchen.do_cancellable"), + (0x29dddbc60bf8d7dbUL, 0x452a3e573d57f6dbUL, "kitchen.Priority"), + (0x2b227003b74a0d16UL, 0x12332957470e9782UL, "kitchen.maybe_subscribe"), + (0x2dd66cb0064f0494UL, 0x60197363fefd92ddUL, "kitchen.Item"), + (0x33c265bbf079b0e9UL, 0xc3dcd0927e6c1debUL, "kitchen.u32_id"), + (0x33e6381d01660dfeUL, 0x077c63c31fd7f90aUL, "kitchen.KitchenErrors"), + (0x3f5fc99d73504b76UL, 0x2df88d710f1ea10eUL, "kitchen.Gadget.poke"), + (0x4c5764cf109528d1UL, 0x2a65529af4338a58UL, "kitchen.maybe_item"), + (0x4de17dfb93fd494dUL, 0xba0ad0dd52ed74afUL, "kitchen.i32_id"), + (0x504e95b7f7e65570UL, 0x01b3e8e0c9a10e23UL, "kitchen.stream_items"), + (0x68cdb9a4bd3e3dd5UL, 0xc678b9164348d497UL, "kitchen.bool_id"), + (0x69e7cd703d16eb85UL, 0xf3b28dd22ae9e689UL, "kitchen.do_async"), + (0x70865079ce41be82UL, 0xbe3b69464672af46UL, "kitchen.Gadget.describe"), + (0x71c5a1116cb7c49aUL, 0xcba1837108a89b84UL, "kitchen.echo_bytes"), + (0x735ee1e0049ee91aUL, 0xca74320bc53eddabUL, "kitchen.echo_string"), + (0x76d7fc58f4e41d84UL, 0x7373a0e47d30e690UL, "kitchen.i64_id"), + (0x79abfe1aa960fd6aUL, 0xbca7a9e5eb26f3a6UL, "kitchen.nested.hello"), + (0x7d3dd7d5cf6749e3UL, 0xa192f5a7298adb39UL, "kitchen.new_op"), + (0x89a5e664bc25c67bUL, 0x5764a95e81ecfd31UL, "kitchen.f64_id"), + (0x8cfdbdcecac5f7f1UL, 0x07e272f09c41fffcUL, "kitchen.Gadget.version"), + (0x974b9f3e823c1738UL, 0xb81b91e975695f27UL, "kitchen.cross_module_token"), + (0xaca0655fac15d230UL, 0x11c4d2b28abe07bbUL, "kitchen.list_items"), + (0xc883b4afae20543eUL, 0xc40560e5411ff392UL, "kitchen.legacy_op"), + (0xf2f7753e6e1dffc2UL, 0xc9047a95bdf49e93UL, "kitchen.get_attrs"), + (0xfae25d3391922eadUL, 0x1c7f95739f3210b8UL, "kitchen.subscribe"), + (0xfd0059b05cb2fcfbUL, 0x248bf8404a5823b4UL, "kitchen.Gadget"), + (0xfed9c551fb9fd2d6UL, 0x59aec980ef3f3291UL, "kitchen.ReadyListener"), + }); } - [LibraryImport(LibName, EntryPoint = "kitchen_sink_shared_checksum")] + [LibraryImport(LibName, EntryPoint = "kitchen_sink_shared_contract")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial ulong kitchen_sink_shared_checksum(); + internal static partial FfiContractEntry* kitchen_sink_shared_contract(nuint* out_len); - [LibraryImport(LibName, EntryPoint = "kitchen_sink_kitchen_checksum")] + [LibraryImport(LibName, EntryPoint = "kitchen_sink_kitchen_contract")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial ulong kitchen_sink_kitchen_checksum(); + internal static partial FfiContractEntry* kitchen_sink_kitchen_contract(nuint* out_len); [LibraryImport(LibName, EntryPoint = "kitchen_sink_shared_ping")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] @@ -857,6 +1088,10 @@ internal static unsafe partial class NativeMethods [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial int kitchen_sink_kitchen_subscribe(IntPtr listener_ctx, IntPtr listener_vtable, FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "kitchen_sink_kitchen_maybe_subscribe")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial byte kitchen_sink_kitchen_maybe_subscribe(IntPtr listener_ctx, IntPtr listener_vtable, FfiError* out_err); + [LibraryImport(LibName, EntryPoint = "kitchen_sink_kitchen_maybe_gadget")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial IntPtr kitchen_sink_kitchen_maybe_gadget(Gadget.NativeHandle fallback, FfiError* out_err); diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_csproj.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_csproj.snap index b5955816..3b5dda59 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_csproj.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__KitchenSink_csproj.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__README_md.snap index cbcaac4f..555101d6 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__README_md.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # KitchenSink (.NET) diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__Runtime_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__Runtime_cs.snap index a837fb34..0d4f3fa1 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__Runtime_cs.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_kitchen_sink__Runtime_cs.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #nullable enable @@ -19,10 +19,11 @@ using System.Threading.Tasks; namespace KitchenSink; -/// Raised for a failure the native library reports that isn't a typed -/// domain error: producer bugs, panics, marshalling failures, and exceptions -/// thrown by a callback-interface implementation. Every typed domain -/// exception derives from it. +/// The root of every error a throwing call reports. Each typed domain +/// exception derives from it, and a runtime failure of a throwing call (an +/// untyped producer error, a panic, a marshalling failure, or a failed +/// callback-interface implementation) surfaces as this type itself, with its +/// negative . public class NativeException : Exception { /// An untyped producer error. @@ -31,14 +32,15 @@ public class NativeException : Exception public const int PanicErrorCode = -2; /// An argument couldn't be lifted by the producer. public const int MarshalErrorCode = -3; - /// A callback-interface implementation threw; the message carries - /// the original exception's message. + /// A callback-interface implementation failed; the message carries + /// the implementation's message. public const int ForeignErrorCode = -4; /// The call was cancelled. Surfaces as /// , never as this type. public const int CancelledErrorCode = -5; - /// The error code the native library reported. + /// The error code the native library reported: positive for a + /// domain error, negative for a runtime failure. public int Code { get; } /// Creates an exception carrying a native error code. @@ -47,9 +49,9 @@ public class NativeException : Exception Code = code; } - /// Maps a runtime error code to its exception: cancellation to - /// , anything else to this - /// type. + /// Maps a code outside every declared domain to its exception: + /// cancellation to , anything else + /// to this type. internal static Exception FromError(int code, string message, byte[]? payload) { if (code == CancelledErrorCode) @@ -60,6 +62,37 @@ public class NativeException : Exception } } +/// A call that can't fail failed anyway: a bug in the native library, +/// or a callback failure the call couldn't report, rather than an error to +/// handle. is the runtime code (-2 a producer panic, -3 a +/// marshalling failure, -4 a failed callback, and so on), and the message +/// names it along with the producer's message. +public sealed class NativeBugException : InvalidOperationException +{ + /// The error code the native library reported. + public int Code { get; } + + /// Creates the exception for a failed call that can't + /// fail. + public NativeBugException(int code, string message) + : base($"native call failed with code {code}: {message}") + { + Code = code; + } + + /// Maps a failure of a call that isn't declared to throw: + /// cancellation to , anything else + /// to this type. + internal static Exception FromError(int code, string message, byte[]? payload) + { + if (code == NativeException.CancelledErrorCode) + { + return new OperationCanceledException(message); + } + return new NativeBugException(code, message); + } +} + /// Maps a native error code, message, and serialized payload to the /// exception a call throws. internal delegate Exception FfiErrorMap(int code, string message, byte[]? payload); @@ -74,6 +107,15 @@ internal unsafe struct FfiError public nuint PayloadLen; } +/// The native kitchen_sink_contract_entry: one declaration's +/// fingerprint in a module's contract table. +[StructLayout(LayoutKind.Sequential)] +internal struct FfiContractEntry +{ + public ulong Id; + public ulong Hash; +} + /// Marshalling helpers shared by every generated call. internal static unsafe class Ffi { @@ -98,6 +140,13 @@ internal static unsafe class Ffi return ptr == null || len == 0 ? Array.Empty() : new ReadOnlySpan(ptr, checked((int)len)).ToArray(); } + /// A reader over a copy of a borrowed value buffer; the native + /// side keeps ownership. + internal static FfiBufferReader ReadBuffer(byte* ptr, nuint len) + { + return new FfiBufferReader(ReadBytes(ptr, len)); + } + /// Decodes a returned UTF-8 string and releases it. internal static string TakeString(byte* ptr, nuint len) { @@ -158,30 +207,103 @@ internal static unsafe class Ffi return Marshal.PtrToStringUTF8((IntPtr)err->Message) ?? ""; } - /// Reports a callback-interface failure to the producer through - /// out_err. The producer copies the message. - internal static void SetForeignError(FfiError* err, Exception e) + /// Reports a failed callback-interface method to the producer + /// through out_err: with the exception's + /// message and, for a domain error with fields, the encoded fields as + /// its payload. The producer copies both. + internal static void SetError(FfiError* err, int code, Exception e, FfiBufferWriter? payload = null) { var bytes = Encoding.UTF8.GetBytes(e.Message + "\0"); fixed (byte* message = bytes) { - NativeMethods.ErrorSet(err, NativeException.ForeignErrorCode, message); + NativeMethods.ErrorSet(err, code, message); + } + if (payload != null && payload.Length > 0) + { + fixed (byte* ptr = payload.Written) + { + NativeMethods.ErrorSetPayload(err, ptr, (nuint)payload.Length); + } + } + } + + /// Reports a domain exception a throwing callback-interface + /// method raised: its positive , its message, and + /// the fields encodes. A failure to + /// encode them is reported as a foreign error instead. + internal static void SetDomainError(FfiError* err, int code, Exception e, Action writePayload) + { + var payload = new FfiBufferWriter(); + try + { + writePayload(payload); + } + catch (Exception inner) + { + SetForeignError(err, inner); + return; + } + SetError(err, code, e, payload); + } + + /// Reports a failed callback-interface method that isn't a + /// declared domain error: the producer receives the foreign error code + /// with the exception's message. + internal static void SetForeignError(FfiError* err, Exception e) + { + SetError(err, NativeException.ForeignErrorCode, e); + } + + /// Hands to the producer through a + /// callback method's out slots, as a run allocated with + /// kitchen_sink_alloc that the producer adopts. + internal static void Return(ReadOnlySpan bytes, byte** outPtr, nuint* outLen) + { + var run = NativeMethods.Alloc((nuint)bytes.Length); + if (run == null && bytes.Length > 0) + { + throw new OutOfMemoryException("the native allocator returned null"); } + bytes.CopyTo(new Span(run, bytes.Length)); + *outPtr = run; + *outLen = (nuint)bytes.Length; + } + + /// Returns a string from a callback method. + internal static void ReturnString(string value, byte** outPtr, nuint* outLen) + { + Return(Encoding.UTF8.GetBytes(value), outPtr, outLen); + } + + /// Returns bytes from a callback method. + internal static void ReturnBytes(byte[] value, byte** outPtr, nuint* outLen) + { + Return(value, outPtr, outLen); + } + + /// Returns an encoded value buffer from a callback method. + internal static void ReturnBuffer(FfiBufferWriter writer, byte** outPtr, nuint* outLen) + { + Return(writer.Written, outPtr, outLen); } /// Pins a callback-interface implementation for the producer and /// returns the ctx it hands back to every trampoline. The - /// producer releases it through the vtable's free. - internal static IntPtr Register(object implementation) + /// producer releases it through the vtable's free. An absent + /// optional implementation is a null ctx. + internal static IntPtr Register(object? implementation) { - return GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); + return implementation == null ? IntPtr.Zero : GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); } /// Releases a ctx the producer never received (the call /// failed before reaching it) or no longer holds. internal static void Unregister(IntPtr ctx) { - GCHandle.FromIntPtr(ctx).Free(); + if (ctx != IntPtr.Zero) + { + GCHandle.FromIntPtr(ctx).Free(); + } } /// The implementation behind a trampoline's ctx. @@ -191,9 +313,9 @@ internal static unsafe class Ffi } } -/// Imports of the native runtime surface plus the load-time contract -/// check. The generated API declares its own imports in the other half of -/// this partial class. +/// Imports of the native runtime surface plus the load-time checks. +/// The generated API declares its own imports, and the contract it was +/// generated with, in the other half of this partial class. internal static unsafe partial class NativeMethods { /// The native library's base name. @@ -205,11 +327,11 @@ internal static unsafe partial class NativeMethods /// The C ABI revision these bindings were generated /// against. - internal const uint AbiVersion = 3; + internal const uint AbiVersion = 4; // Runs before the first native call through this class, so a library - // built for another ABI revision or another contract fails here instead - // of misreading memory later. + // built for another ABI revision or missing a declaration these bindings + // use fails here instead of misreading memory later. static NativeMethods() { try @@ -235,10 +357,10 @@ internal static unsafe partial class NativeMethods throw new InvalidOperationException( $"ABI mismatch: these bindings expect revision {AbiVersion} but the loaded {LibName} library reports revision {found}"); } - VerifyChecksums(); + VerifyContracts(); } - static partial void VerifyChecksums(); + static partial void VerifyContracts(); private static IntPtr Resolve(string name, Assembly assembly, DllImportSearchPath? searchPath) { @@ -250,25 +372,44 @@ internal static unsafe partial class NativeMethods return string.IsNullOrEmpty(path) ? IntPtr.Zero : NativeLibrary.Load(path); } - /// Fails the load when module was - /// generated from a different contract than the loaded library - /// implements. - private static void VerifyChecksum(string module, delegate* checksum, ulong expected) + /// Fails the load unless the loaded library's contract table, + /// read through (the import of + /// ), carries every + /// declaration with an unchanged signature hash. Declarations the + /// library has and these bindings don't are fine. + private static void VerifyContract( + string symbol, + delegate* table, + (ulong Id, ulong Hash, string Path)[] expected) { - ulong found; + nuint len = 0; + FfiContractEntry* entries; try { - found = checksum(); + entries = table(&len); } catch (EntryPointNotFoundException e) { throw new InvalidOperationException( - $"the loaded {LibName} library doesn't implement module '{module}'", e); + $"the loaded {LibName} library doesn't export {symbol}; regenerate the bindings or rebuild the library", e); } - if (found != expected) + var hashes = new Dictionary(checked((int)len)); + for (nuint i = 0; i < len; i++) { - throw new InvalidOperationException( - $"contract mismatch for module '{module}': these bindings expect checksum 0x{expected:x16} but the loaded {LibName} library reports 0x{found:x16}; regenerate the bindings for this library"); + hashes[entries[i].Id] = entries[i].Hash; + } + foreach (var (id, hash, path) in expected) + { + if (!hashes.TryGetValue(id, out var actual)) + { + throw new InvalidOperationException( + $"{path} is missing from the library {LibName}; regenerate the bindings or rebuild the library"); + } + if (actual != hash) + { + throw new InvalidOperationException( + $"{path} changed since these bindings were generated; regenerate the bindings or rebuild the library {LibName}"); + } } } @@ -280,6 +421,10 @@ internal static unsafe partial class NativeMethods [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void ErrorSet(FfiError* err, int code, byte* message); + [LibraryImport(LibName, EntryPoint = "kitchen_sink_error_set_payload")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial void ErrorSetPayload(FfiError* err, byte* ptr, nuint len); + [LibraryImport(LibName, EntryPoint = "kitchen_sink_error_clear")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void ErrorClear(FfiError* err); @@ -288,6 +433,10 @@ internal static unsafe partial class NativeMethods [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void ErrorFree(FfiError* err); + [LibraryImport(LibName, EntryPoint = "kitchen_sink_alloc")] + [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] + internal static partial byte* Alloc(nuint len); + [LibraryImport(LibName, EntryPoint = "kitchen_sink_free_bytes")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] internal static partial void FreeBytes(byte* ptr, nuint len); @@ -583,8 +732,8 @@ internal sealed class FfiBufferWriter } /// Decodes values from the value-buffer wire format. A malformed -/// buffer is a producer or consumer bug and throws -/// . +/// buffer is a bug on the side that encoded it and throws +/// with the marshalling code. internal sealed class FfiBufferReader { private static readonly Encoding Utf8Strict = new UTF8Encoding(false, true); @@ -597,9 +746,11 @@ internal sealed class FfiBufferReader _data = data; } - private static InvalidOperationException Malformed(string what) + /// The exception for a buffer that breaks the wire + /// format. + internal static NativeBugException Malformed(string what) { - return new InvalidOperationException("malformed value buffer: " + what); + return new NativeBugException(NativeException.MarshalErrorCode, "malformed value buffer: " + what); } private ReadOnlySpan Take(int n) diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_cs.snap index 883176ad..a0473c7c 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_cs.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_cs.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #nullable enable @@ -16,23 +16,41 @@ using System.Threading.Tasks; namespace NestedModules; -/// Typed exception for the GraphicsError error domain (module graphics). -public class GraphicsException : NativeException +/// The errors of the GraphicsError domain, declared by module graphics. +/// Catch a nested class for one code, or this class for any of them. +public abstract class GraphicsException : NativeException { - /// coordinate out of bounds - public const int OutOfBounds = 1; + private protected GraphicsException(int code, string message) : base(code, message) + { + } + + /// Writes this code's fields, the error payload a callback + /// reports with it. + internal virtual void WritePayload(FfiBufferWriter writer) + { + } - /// Creates an exception carrying a domain error code. - public GraphicsException(int code, string message) : base(code, message) + /// coordinate out of bounds + public sealed class OutOfBounds : GraphicsException { + /// The domain code, 1. + public const int ErrorCode = 1; + + /// Creates the error with , or the + /// default message "coordinate out of bounds" when it's null. + public OutOfBounds(string? message = null) : base(ErrorCode, message ?? "coordinate out of bounds") + { + } } + /// The exception for a raw code and its serialized payload. internal static new Exception FromError(int code, string message, byte[]? payload) { + var given = message.Length == 0 ? null : message; switch (code) { - case OutOfBounds: - return new GraphicsException(code, string.IsNullOrEmpty(message) ? "coordinate out of bounds" : message); + case OutOfBounds.ErrorCode: + return new OutOfBounds(given); default: return global::NestedModules.NativeException.FromError(code, message, payload); } @@ -69,9 +87,9 @@ public sealed class Canvas internal static Canvas ReadFrom(FfiBufferReader reader) { - var fWidth = reader.ReadI32(); - var fRoot = global::NestedModules.Shape.ReadFrom(reader); - return new Canvas(fWidth, fRoot); + return new Canvas( + reader.ReadI32(), + global::NestedModules.Shape.ReadFrom(reader)); } } @@ -103,10 +121,10 @@ public sealed class Shape internal static Shape ReadFrom(FfiBufferReader reader) { - var fSides = reader.ReadI32(); - var fUnit = (Unit)reader.ReadI32(); - var fDetail = global::NestedModules.Detail.ReadFrom(reader); - return new Shape(fSides, fUnit, fDetail); + return new Shape( + reader.ReadI32(), + (global::NestedModules.Unit)reader.ReadI32(), + global::NestedModules.Detail.ReadFrom(reader)); } } @@ -128,8 +146,8 @@ public sealed class Detail internal static Detail ReadFrom(FfiBufferReader reader) { - var fArea = reader.ReadF64(); - return new Detail(fArea); + return new Detail( + reader.ReadF64()); } } @@ -141,11 +159,8 @@ public static unsafe class Graphics var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.nested_modules_graphics_new_canvas(width, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::NestedModules.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::NestedModules.Canvas.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::NestedModules.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::NestedModules.Canvas.ReadFrom(r)); } } @@ -157,11 +172,8 @@ public static unsafe class GraphicsShapes var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.nested_modules_graphics_shapes_make_shape(sides, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::NestedModules.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::NestedModules.Shape.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::NestedModules.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::NestedModules.Shape.ReadFrom(r)); } /// Nested function returning a struct owned by an ancestor module. @@ -170,11 +182,8 @@ public static unsafe class GraphicsShapes var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.nested_modules_graphics_shapes_describe(&ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::NestedModules.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::NestedModules.Canvas.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::NestedModules.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::NestedModules.Canvas.ReadFrom(r)); } } @@ -192,24 +201,45 @@ public static unsafe class GraphicsShapesGeometry { var ffiResult = NativeMethods.nested_modules_graphics_shapes_geometry_compute(shapePtr, (nuint)shapeWriter.Length, &ffiOutLen, &ffiErr); if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::NestedModules.GraphicsException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::NestedModules.Detail.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::NestedModules.Detail.ReadFrom(r)); } } } +/// The value-buffer codec of every composite type the API +/// carries. +internal static class FfiCodecs +{ + /// Reads one value that must fill the whole buffer. + internal static T Decode(FfiBufferReader reader, Func read) + { + var value = read(reader); + reader.ExpectEnd(); + return value; + } +} + internal static unsafe partial class NativeMethods { - static partial void VerifyChecksums() + static partial void VerifyContracts() { - VerifyChecksum("graphics", &nested_modules_graphics_checksum, 0x44d31b5b10fdb90dUL); + VerifyContract("nested_modules_graphics_contract", &nested_modules_graphics_contract, new (ulong, ulong, string)[] + { + (0x2c29eef9327a6bd4UL, 0x12983d5271651389UL, "graphics.Unit"), + (0x304088bbc28f0015UL, 0x97e6e809913be6b6UL, "graphics.new_canvas"), + (0x49f8837532df9ac6UL, 0x32e5b27e81ee0957UL, "graphics.Canvas"), + (0x55717fa027336b47UL, 0xb2bd3a6bdcce48c8UL, "graphics.shapes.describe"), + (0x7626b91b8f1f53cdUL, 0x867bbf7cf0f41c74UL, "graphics.shapes.geometry.compute"), + (0xb3efcb50d9a61ab7UL, 0x3cb0207dfad2a149UL, "graphics.GraphicsError"), + (0xc62b16a9591b744bUL, 0x7895610c8487eb1bUL, "graphics.shapes.Shape"), + (0xe13001f4983ccd35UL, 0x16e7de070b4a4cfcUL, "graphics.shapes.geometry.Detail"), + (0xf3f658bcfc5dba62UL, 0x2665ccb47356e746UL, "graphics.shapes.make_shape"), + }); } - [LibraryImport(LibName, EntryPoint = "nested_modules_graphics_checksum")] + [LibraryImport(LibName, EntryPoint = "nested_modules_graphics_contract")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial ulong nested_modules_graphics_checksum(); + internal static partial FfiContractEntry* nested_modules_graphics_contract(nuint* out_len); [LibraryImport(LibName, EntryPoint = "nested_modules_graphics_new_canvas")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_csproj.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_csproj.snap deleted file mode 100644 index b98ed2a9..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__NestedModules_csproj.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - - - - - net8.0 - NestedModules - NestedModules - NestedModules - 0.1.0 - Native bindings for nested_modules - enable - true - true - - $(NoWarn);CS0618 - - - - - diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__README_md.snap deleted file mode 100644 index 1cbc635b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__README_md.snap +++ /dev/null @@ -1,32 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# NestedModules (.NET) - -Native bindings for nested_modules - -The `NestedModules` namespace binds the `nested_modules` native library through -source-generated P/Invoke (`[LibraryImport]`), so it's trim- and -AOT-friendly. - -## Build - -```bash -dotnet build -``` - -Reference this project from yours (`dotnet add reference NestedModules.csproj`) -or pack it (`dotnet pack -c Release`). - -## Loading the native library - -The bindings load `nested_modules` with the platform's normal rules -(`libnested_modules.dylib`, `libnested_modules.so`, or `nested_modules.dll` next to the app or on the -library search path). Set `NESTED_MODULES_LIBRARY` to a full path to load a -specific file instead. - - diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__Runtime_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__Runtime_cs.snap deleted file mode 100644 index 66aae836..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_nested_modules__Runtime_cs.snap +++ /dev/null @@ -1,698 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#nullable enable - -using System; -using System.Collections; -using System.Collections.Generic; -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using System.Text; -using System.Threading; -using System.Threading.Tasks; - -namespace NestedModules; - -/// Raised for a failure the native library reports that isn't a typed -/// domain error: producer bugs, panics, marshalling failures, and exceptions -/// thrown by a callback-interface implementation. Every typed domain -/// exception derives from it. -public class NativeException : Exception -{ - /// An untyped producer error. - public const int GenericErrorCode = -1; - /// The producer panicked; the message carries the panic text. - public const int PanicErrorCode = -2; - /// An argument couldn't be lifted by the producer. - public const int MarshalErrorCode = -3; - /// A callback-interface implementation threw; the message carries - /// the original exception's message. - public const int ForeignErrorCode = -4; - /// The call was cancelled. Surfaces as - /// , never as this type. - public const int CancelledErrorCode = -5; - - /// The error code the native library reported. - public int Code { get; } - - /// Creates an exception carrying a native error code. - public NativeException(int code, string message) : base(message) - { - Code = code; - } - - /// Maps a runtime error code to its exception: cancellation to - /// , anything else to this - /// type. - internal static Exception FromError(int code, string message, byte[]? payload) - { - if (code == CancelledErrorCode) - { - return new OperationCanceledException(message); - } - return new NativeException(code, message); - } -} - -/// Maps a native error code, message, and serialized payload to the -/// exception a call throws. -internal delegate Exception FfiErrorMap(int code, string message, byte[]? payload); - -/// The native nested_modules_error slot. -[StructLayout(LayoutKind.Sequential)] -internal unsafe struct FfiError -{ - public int Code; - public byte* Message; - public byte* PayloadPtr; - public nuint PayloadLen; -} - -/// Marshalling helpers shared by every generated call. -internal static unsafe class Ffi -{ - /// The UTF-8 bytes of a string argument. The native side borrows - /// them as a (pointer, length) pair, so interior NULs survive. - internal static byte[] Utf8(string value) - { - return Encoding.UTF8.GetBytes(value); - } - - /// Decodes a borrowed UTF-8 run; the native side keeps - /// ownership. - internal static string ReadString(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? "" : Encoding.UTF8.GetString(ptr, checked((int)len)); - } - - /// Copies a borrowed byte run; the native side keeps - /// ownership. - internal static byte[] ReadBytes(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? Array.Empty() : new ReadOnlySpan(ptr, checked((int)len)).ToArray(); - } - - /// Decodes a returned UTF-8 string and releases it. - internal static string TakeString(byte* ptr, nuint len) - { - try - { - return ReadString(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies returned bytes and releases them. - internal static byte[] TakeBytes(byte* ptr, nuint len) - { - try - { - return ReadBytes(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies a returned value buffer, releases it, and returns a - /// reader over the copy. - internal static FfiBufferReader TakeBuffer(byte* ptr, nuint len) - { - return new FfiBufferReader(TakeBytes(ptr, len)); - } - - /// Releases a caller-owned error slot and returns the exception - /// it described. - internal static Exception TakeError(FfiError* err, FfiErrorMap map) - { - var code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorClear(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - /// Releases a heap-boxed async error and returns the exception it - /// described. - internal static Exception TakeBoxedError(FfiError* err, FfiErrorMap map, out int code) - { - code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorFree(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - private static string ErrorMessage(FfiError* err) - { - return Marshal.PtrToStringUTF8((IntPtr)err->Message) ?? ""; - } - - /// Reports a callback-interface failure to the producer through - /// out_err. The producer copies the message. - internal static void SetForeignError(FfiError* err, Exception e) - { - var bytes = Encoding.UTF8.GetBytes(e.Message + "\0"); - fixed (byte* message = bytes) - { - NativeMethods.ErrorSet(err, NativeException.ForeignErrorCode, message); - } - } - - /// Pins a callback-interface implementation for the producer and - /// returns the ctx it hands back to every trampoline. The - /// producer releases it through the vtable's free. - internal static IntPtr Register(object implementation) - { - return GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); - } - - /// Releases a ctx the producer never received (the call - /// failed before reaching it) or no longer holds. - internal static void Unregister(IntPtr ctx) - { - GCHandle.FromIntPtr(ctx).Free(); - } - - /// The implementation behind a trampoline's ctx. - internal static T Target(IntPtr ctx) where T : class - { - return (T)GCHandle.FromIntPtr(ctx).Target!; - } -} - -/// Imports of the native runtime surface plus the load-time contract -/// check. The generated API declares its own imports in the other half of -/// this partial class. -internal static unsafe partial class NativeMethods -{ - /// The native library's base name. - internal const string LibName = "nested_modules"; - - /// The environment variable naming an explicit library - /// path. - internal const string LibraryEnvVar = "NESTED_MODULES_LIBRARY"; - - /// The C ABI revision these bindings were generated - /// against. - internal const uint AbiVersion = 3; - - // Runs before the first native call through this class, so a library - // built for another ABI revision or another contract fails here instead - // of misreading memory later. - static NativeMethods() - { - try - { - NativeLibrary.SetDllImportResolver(typeof(NativeMethods).Assembly, Resolve); - } - catch (InvalidOperationException) - { - // The assembly already has a resolver; default probing still applies. - } - uint found; - try - { - found = AbiVersionOf(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't export nested_modules_abi_version; these bindings expect ABI revision {AbiVersion}", e); - } - if (found != AbiVersion) - { - throw new InvalidOperationException( - $"ABI mismatch: these bindings expect revision {AbiVersion} but the loaded {LibName} library reports revision {found}"); - } - VerifyChecksums(); - } - - static partial void VerifyChecksums(); - - private static IntPtr Resolve(string name, Assembly assembly, DllImportSearchPath? searchPath) - { - if (name != LibName) - { - return IntPtr.Zero; - } - var path = Environment.GetEnvironmentVariable(LibraryEnvVar); - return string.IsNullOrEmpty(path) ? IntPtr.Zero : NativeLibrary.Load(path); - } - - /// Fails the load when module was - /// generated from a different contract than the loaded library - /// implements. - private static void VerifyChecksum(string module, delegate* checksum, ulong expected) - { - ulong found; - try - { - found = checksum(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't implement module '{module}'", e); - } - if (found != expected) - { - throw new InvalidOperationException( - $"contract mismatch for module '{module}': these bindings expect checksum 0x{expected:x16} but the loaded {LibName} library reports 0x{found:x16}; regenerate the bindings for this library"); - } - } - - [LibraryImport(LibName, EntryPoint = "nested_modules_abi_version")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial uint AbiVersionOf(); - - [LibraryImport(LibName, EntryPoint = "nested_modules_error_set")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorSet(FfiError* err, int code, byte* message); - - [LibraryImport(LibName, EntryPoint = "nested_modules_error_clear")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorClear(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "nested_modules_error_free")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorFree(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "nested_modules_free_bytes")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void FreeBytes(byte* ptr, nuint len); - - [LibraryImport(LibName, EntryPoint = "nested_modules_cancel_token_create")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial IntPtr CancelTokenCreate(); - - [LibraryImport(LibName, EntryPoint = "nested_modules_cancel_token_cancel")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenCancel(IntPtr token); - - [LibraryImport(LibName, EntryPoint = "nested_modules_cancel_token_destroy")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenDestroy(IntPtr token); -} - -/// One pending async call: the task it completes, the native -/// context that finds it again, and, for a cancellable function, the -/// native cancel token linked to the caller's -/// . -internal sealed class FfiCall -{ - private readonly TaskCompletionSource _tcs = - new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - private readonly CancellationToken _cancellation; - private readonly object _gate = new object(); - private GCHandle _self; - private IntPtr _token; - private CancellationTokenRegistration _registration; - private bool _completed; - private bool _cancelling; - - internal FfiCall(CancellationToken cancellation = default) - { - _cancellation = cancellation; - _self = GCHandle.Alloc(this); - } - - /// The task the completion resolves. - internal Task Task => _tcs.Task; - - /// The context argument for the launcher. - internal IntPtr Context => GCHandle.ToIntPtr(_self); - - /// Creates the native cancel token for the launcher and cancels - /// it when the caller's fires. - internal IntPtr CancelToken() - { - _token = NativeMethods.CancelTokenCreate(); - if (_cancellation.CanBeCanceled) - { - _registration = _cancellation.Register(static state => ((FfiCall)state!).Cancel(), this); - } - return _token; - } - - /// Recovers the call behind a completion's context. Runs - /// exactly once per launch, on a producer thread. - internal static FfiCall Complete(IntPtr context) - { - var handle = GCHandle.FromIntPtr(context); - var call = (FfiCall)handle.Target!; - handle.Free(); - call.Release(); - return call; - } - - /// Releases everything when the launcher threw before the - /// producer took the call, so no completion will arrive. - internal void Abandon() - { - _self.Free(); - Release(); - } - - internal void SetResult(T value) - { - _tcs.TrySetResult(value); - } - - internal void SetException(Exception e) - { - _tcs.TrySetException(e); - } - - /// Faults (or cancels, for the cancelled code) the task from a - /// heap-boxed error, releasing the box. - internal unsafe void SetError(FfiError* err, FfiErrorMap map) - { - var e = Ffi.TakeBoxedError(err, map, out var code); - if (code == NativeException.CancelledErrorCode) - { - _tcs.TrySetCanceled(_cancellation); - } - else - { - _tcs.TrySetException(e); - } - } - - // The registration callback and the completion race: whichever runs - // last destroys the consumer's token reference, so a cancel never - // touches a destroyed token. - private void Cancel() - { - lock (_gate) - { - if (_completed) - { - return; - } - _cancelling = true; - } - NativeMethods.CancelTokenCancel(_token); - bool destroy; - lock (_gate) - { - _cancelling = false; - destroy = _completed; - } - if (destroy) - { - NativeMethods.CancelTokenDestroy(_token); - } - } - - private void Release() - { - if (_token == IntPtr.Zero) - { - return; - } - bool destroy; - lock (_gate) - { - _completed = true; - destroy = !_cancelling; - } - if (destroy) - { - _registration.Dispose(); - NativeMethods.CancelTokenDestroy(_token); - } - } -} - -/// Owns one native iterator and destroys it exactly once, when -/// enumeration ends, the enumerator is disposed, or the garbage collector -/// finalizes an abandoned one. -internal sealed unsafe class FfiIteratorHandle : SafeHandle -{ - private readonly delegate* _destroy; - - internal FfiIteratorHandle(IntPtr iterator, delegate* destroy) - : base(IntPtr.Zero, true) - { - _destroy = destroy; - SetHandle(iterator); - } - - public override bool IsInvalid => handle == IntPtr.Zero; - - protected override bool ReleaseHandle() - { - _destroy(handle); - return true; - } -} - -/// A lazily streamed sequence backed by a native iterator. It can be -/// enumerated once; each MoveNext pulls one item. -internal sealed unsafe class FfiSequence : IEnumerable, IEnumerator -{ - private readonly FfiIteratorHandle _iterator; - private readonly delegate* _next; - private int _started; - private T _current = default!; - - internal FfiSequence(FfiIteratorHandle iterator, delegate* next) - { - _iterator = iterator; - _next = next; - } - - public T Current => _current; - - object? IEnumerator.Current => _current; - - public IEnumerator GetEnumerator() - { - if (Interlocked.Exchange(ref _started, 1) != 0) - { - throw new InvalidOperationException("this sequence can be enumerated only once"); - } - return this; - } - - IEnumerator IEnumerable.GetEnumerator() - { - return GetEnumerator(); - } - - public bool MoveNext() - { - if (_iterator.IsClosed) - { - return false; - } - if (_next(_iterator, out _current)) - { - return true; - } - _iterator.Dispose(); - return false; - } - - public void Reset() - { - throw new NotSupportedException("a native iterator can't be reset"); - } - - public void Dispose() - { - _iterator.Dispose(); - } -} - -/// Serializes values into the value-buffer wire format -/// (little-endian, packed). -internal sealed class FfiBufferWriter -{ - private byte[] _buf = new byte[64]; - private int _len; - - /// The bytes written so far. - internal ReadOnlySpan Written => _buf.AsSpan(0, _len); - - /// The number of bytes written so far. - internal int Length => _len; - - private Span Grow(int extra) - { - if (_len + extra > _buf.Length) - { - Array.Resize(ref _buf, Math.Max(_buf.Length * 2, _len + extra)); - } - var span = _buf.AsSpan(_len, extra); - _len += extra; - return span; - } - - internal void WriteBool(bool v) => WriteU8(v ? (byte)1 : (byte)0); - - internal void WriteI8(sbyte v) => WriteU8((byte)v); - - internal void WriteU8(byte v) => Grow(1)[0] = v; - - internal void WriteI16(short v) => System.Buffers.Binary.BinaryPrimitives.WriteInt16LittleEndian(Grow(2), v); - - internal void WriteU16(ushort v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt16LittleEndian(Grow(2), v); - - internal void WriteI32(int v) => System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(Grow(4), v); - - internal void WriteU32(uint v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(Grow(4), v); - - internal void WriteI64(long v) => System.Buffers.Binary.BinaryPrimitives.WriteInt64LittleEndian(Grow(8), v); - - internal void WriteU64(ulong v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt64LittleEndian(Grow(8), v); - - internal void WriteF32(float v) => System.Buffers.Binary.BinaryPrimitives.WriteSingleLittleEndian(Grow(4), v); - - internal void WriteF64(double v) => System.Buffers.Binary.BinaryPrimitives.WriteDoubleLittleEndian(Grow(8), v); - - internal void WriteLen(int len) => WriteU32((uint)len); - - internal void WriteString(string v) - { - var count = Encoding.UTF8.GetByteCount(v); - WriteLen(count); - Encoding.UTF8.GetBytes(v, Grow(count)); - } - - internal void WriteBytes(byte[] v) - { - WriteLen(v.Length); - v.CopyTo(Grow(v.Length)); - } - - /// Writes an object token. The token must carry its own strong - /// reference (a fresh clone), which the reader adopts. - internal void WriteObject(IntPtr token) => WriteU64((ulong)(nuint)token); -} - -/// Decodes values from the value-buffer wire format. A malformed -/// buffer is a producer or consumer bug and throws -/// . -internal sealed class FfiBufferReader -{ - private static readonly Encoding Utf8Strict = new UTF8Encoding(false, true); - - private readonly byte[] _data; - private int _pos; - - internal FfiBufferReader(byte[] data) - { - _data = data; - } - - private static InvalidOperationException Malformed(string what) - { - return new InvalidOperationException("malformed value buffer: " + what); - } - - private ReadOnlySpan Take(int n) - { - if (_data.Length - _pos < n) - { - throw Malformed("buffer exhausted"); - } - var span = _data.AsSpan(_pos, n); - _pos += n; - return span; - } - - internal bool ReadBool() - { - var b = ReadU8(); - if (b > 1) - { - throw Malformed("invalid bool byte"); - } - return b == 1; - } - - internal sbyte ReadI8() => (sbyte)ReadU8(); - - internal byte ReadU8() => Take(1)[0]; - - internal short ReadI16() => System.Buffers.Binary.BinaryPrimitives.ReadInt16LittleEndian(Take(2)); - - internal ushort ReadU16() => System.Buffers.Binary.BinaryPrimitives.ReadUInt16LittleEndian(Take(2)); - - internal int ReadI32() => System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(Take(4)); - - internal uint ReadU32() => System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(Take(4)); - - internal long ReadI64() => System.Buffers.Binary.BinaryPrimitives.ReadInt64LittleEndian(Take(8)); - - internal ulong ReadU64() => System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(Take(8)); - - internal float ReadF32() => System.Buffers.Binary.BinaryPrimitives.ReadSingleLittleEndian(Take(4)); - - internal double ReadF64() => System.Buffers.Binary.BinaryPrimitives.ReadDoubleLittleEndian(Take(8)); - - /// A collection count. Elements can be zero-sized, so the count - /// isn't checked against the remaining bytes; callers cap their - /// preallocation with . - internal int ReadLen() - { - var len = ReadU32(); - if (len > int.MaxValue) - { - throw Malformed("length prefix out of range"); - } - return (int)len; - } - - /// A preallocation size for elements - /// that a malformed count can't inflate. - internal int Capacity(int count) => Math.Min(count, _data.Length - _pos); - - internal string ReadString() - { - var bytes = Take(ReadLen()); - try - { - return Utf8Strict.GetString(bytes); - } - catch (ArgumentException) - { - throw Malformed("string is not valid UTF-8"); - } - } - - internal byte[] ReadBytes() => Take(ReadLen()).ToArray(); - - /// Reads an object token carrying one strong reference. - internal IntPtr ReadObject() - { - var token = ReadU64(); - if (token == 0) - { - throw Malformed("null object token"); - } - return (IntPtr)(nint)token; - } - - internal void ExpectEnd() - { - if (_pos != _data.Length) - { - throw Malformed("trailing bytes"); - } - } -} - -// END Runtime.cs diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__README_md.snap deleted file mode 100644 index d46dda8a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__README_md.snap +++ /dev/null @@ -1,32 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# Shapes (.NET) - -Native bindings for shapes - -The `Shapes` namespace binds the `shapes` native library through -source-generated P/Invoke (`[LibraryImport]`), so it's trim- and -AOT-friendly. - -## Build - -```bash -dotnet build -``` - -Reference this project from yours (`dotnet add reference Shapes.csproj`) -or pack it (`dotnet pack -c Release`). - -## Loading the native library - -The bindings load `shapes` with the platform's normal rules -(`libshapes.dylib`, `libshapes.so`, or `shapes.dll` next to the app or on the -library search path). Set `SHAPES_LIBRARY` to a full path to load a -specific file instead. - - diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Runtime_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Runtime_cs.snap deleted file mode 100644 index c73f86c0..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Runtime_cs.snap +++ /dev/null @@ -1,698 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#nullable enable - -using System; -using System.Collections; -using System.Collections.Generic; -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using System.Text; -using System.Threading; -using System.Threading.Tasks; - -namespace Shapes; - -/// Raised for a failure the native library reports that isn't a typed -/// domain error: producer bugs, panics, marshalling failures, and exceptions -/// thrown by a callback-interface implementation. Every typed domain -/// exception derives from it. -public class NativeException : Exception -{ - /// An untyped producer error. - public const int GenericErrorCode = -1; - /// The producer panicked; the message carries the panic text. - public const int PanicErrorCode = -2; - /// An argument couldn't be lifted by the producer. - public const int MarshalErrorCode = -3; - /// A callback-interface implementation threw; the message carries - /// the original exception's message. - public const int ForeignErrorCode = -4; - /// The call was cancelled. Surfaces as - /// , never as this type. - public const int CancelledErrorCode = -5; - - /// The error code the native library reported. - public int Code { get; } - - /// Creates an exception carrying a native error code. - public NativeException(int code, string message) : base(message) - { - Code = code; - } - - /// Maps a runtime error code to its exception: cancellation to - /// , anything else to this - /// type. - internal static Exception FromError(int code, string message, byte[]? payload) - { - if (code == CancelledErrorCode) - { - return new OperationCanceledException(message); - } - return new NativeException(code, message); - } -} - -/// Maps a native error code, message, and serialized payload to the -/// exception a call throws. -internal delegate Exception FfiErrorMap(int code, string message, byte[]? payload); - -/// The native shapes_error slot. -[StructLayout(LayoutKind.Sequential)] -internal unsafe struct FfiError -{ - public int Code; - public byte* Message; - public byte* PayloadPtr; - public nuint PayloadLen; -} - -/// Marshalling helpers shared by every generated call. -internal static unsafe class Ffi -{ - /// The UTF-8 bytes of a string argument. The native side borrows - /// them as a (pointer, length) pair, so interior NULs survive. - internal static byte[] Utf8(string value) - { - return Encoding.UTF8.GetBytes(value); - } - - /// Decodes a borrowed UTF-8 run; the native side keeps - /// ownership. - internal static string ReadString(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? "" : Encoding.UTF8.GetString(ptr, checked((int)len)); - } - - /// Copies a borrowed byte run; the native side keeps - /// ownership. - internal static byte[] ReadBytes(byte* ptr, nuint len) - { - return ptr == null || len == 0 ? Array.Empty() : new ReadOnlySpan(ptr, checked((int)len)).ToArray(); - } - - /// Decodes a returned UTF-8 string and releases it. - internal static string TakeString(byte* ptr, nuint len) - { - try - { - return ReadString(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies returned bytes and releases them. - internal static byte[] TakeBytes(byte* ptr, nuint len) - { - try - { - return ReadBytes(ptr, len); - } - finally - { - NativeMethods.FreeBytes(ptr, len); - } - } - - /// Copies a returned value buffer, releases it, and returns a - /// reader over the copy. - internal static FfiBufferReader TakeBuffer(byte* ptr, nuint len) - { - return new FfiBufferReader(TakeBytes(ptr, len)); - } - - /// Releases a caller-owned error slot and returns the exception - /// it described. - internal static Exception TakeError(FfiError* err, FfiErrorMap map) - { - var code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorClear(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - /// Releases a heap-boxed async error and returns the exception it - /// described. - internal static Exception TakeBoxedError(FfiError* err, FfiErrorMap map, out int code) - { - code = err->Code; - var message = ErrorMessage(err); - var payload = ReadBytes(err->PayloadPtr, err->PayloadLen); - NativeMethods.ErrorFree(err); - return map(code, message, payload.Length == 0 ? null : payload); - } - - private static string ErrorMessage(FfiError* err) - { - return Marshal.PtrToStringUTF8((IntPtr)err->Message) ?? ""; - } - - /// Reports a callback-interface failure to the producer through - /// out_err. The producer copies the message. - internal static void SetForeignError(FfiError* err, Exception e) - { - var bytes = Encoding.UTF8.GetBytes(e.Message + "\0"); - fixed (byte* message = bytes) - { - NativeMethods.ErrorSet(err, NativeException.ForeignErrorCode, message); - } - } - - /// Pins a callback-interface implementation for the producer and - /// returns the ctx it hands back to every trampoline. The - /// producer releases it through the vtable's free. - internal static IntPtr Register(object implementation) - { - return GCHandle.ToIntPtr(GCHandle.Alloc(implementation)); - } - - /// Releases a ctx the producer never received (the call - /// failed before reaching it) or no longer holds. - internal static void Unregister(IntPtr ctx) - { - GCHandle.FromIntPtr(ctx).Free(); - } - - /// The implementation behind a trampoline's ctx. - internal static T Target(IntPtr ctx) where T : class - { - return (T)GCHandle.FromIntPtr(ctx).Target!; - } -} - -/// Imports of the native runtime surface plus the load-time contract -/// check. The generated API declares its own imports in the other half of -/// this partial class. -internal static unsafe partial class NativeMethods -{ - /// The native library's base name. - internal const string LibName = "shapes"; - - /// The environment variable naming an explicit library - /// path. - internal const string LibraryEnvVar = "SHAPES_LIBRARY"; - - /// The C ABI revision these bindings were generated - /// against. - internal const uint AbiVersion = 3; - - // Runs before the first native call through this class, so a library - // built for another ABI revision or another contract fails here instead - // of misreading memory later. - static NativeMethods() - { - try - { - NativeLibrary.SetDllImportResolver(typeof(NativeMethods).Assembly, Resolve); - } - catch (InvalidOperationException) - { - // The assembly already has a resolver; default probing still applies. - } - uint found; - try - { - found = AbiVersionOf(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't export shapes_abi_version; these bindings expect ABI revision {AbiVersion}", e); - } - if (found != AbiVersion) - { - throw new InvalidOperationException( - $"ABI mismatch: these bindings expect revision {AbiVersion} but the loaded {LibName} library reports revision {found}"); - } - VerifyChecksums(); - } - - static partial void VerifyChecksums(); - - private static IntPtr Resolve(string name, Assembly assembly, DllImportSearchPath? searchPath) - { - if (name != LibName) - { - return IntPtr.Zero; - } - var path = Environment.GetEnvironmentVariable(LibraryEnvVar); - return string.IsNullOrEmpty(path) ? IntPtr.Zero : NativeLibrary.Load(path); - } - - /// Fails the load when module was - /// generated from a different contract than the loaded library - /// implements. - private static void VerifyChecksum(string module, delegate* checksum, ulong expected) - { - ulong found; - try - { - found = checksum(); - } - catch (EntryPointNotFoundException e) - { - throw new InvalidOperationException( - $"the loaded {LibName} library doesn't implement module '{module}'", e); - } - if (found != expected) - { - throw new InvalidOperationException( - $"contract mismatch for module '{module}': these bindings expect checksum 0x{expected:x16} but the loaded {LibName} library reports 0x{found:x16}; regenerate the bindings for this library"); - } - } - - [LibraryImport(LibName, EntryPoint = "shapes_abi_version")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial uint AbiVersionOf(); - - [LibraryImport(LibName, EntryPoint = "shapes_error_set")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorSet(FfiError* err, int code, byte* message); - - [LibraryImport(LibName, EntryPoint = "shapes_error_clear")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorClear(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "shapes_error_free")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void ErrorFree(FfiError* err); - - [LibraryImport(LibName, EntryPoint = "shapes_free_bytes")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void FreeBytes(byte* ptr, nuint len); - - [LibraryImport(LibName, EntryPoint = "shapes_cancel_token_create")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial IntPtr CancelTokenCreate(); - - [LibraryImport(LibName, EntryPoint = "shapes_cancel_token_cancel")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenCancel(IntPtr token); - - [LibraryImport(LibName, EntryPoint = "shapes_cancel_token_destroy")] - [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial void CancelTokenDestroy(IntPtr token); -} - -/// One pending async call: the task it completes, the native -/// context that finds it again, and, for a cancellable function, the -/// native cancel token linked to the caller's -/// . -internal sealed class FfiCall -{ - private readonly TaskCompletionSource _tcs = - new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - private readonly CancellationToken _cancellation; - private readonly object _gate = new object(); - private GCHandle _self; - private IntPtr _token; - private CancellationTokenRegistration _registration; - private bool _completed; - private bool _cancelling; - - internal FfiCall(CancellationToken cancellation = default) - { - _cancellation = cancellation; - _self = GCHandle.Alloc(this); - } - - /// The task the completion resolves. - internal Task Task => _tcs.Task; - - /// The context argument for the launcher. - internal IntPtr Context => GCHandle.ToIntPtr(_self); - - /// Creates the native cancel token for the launcher and cancels - /// it when the caller's fires. - internal IntPtr CancelToken() - { - _token = NativeMethods.CancelTokenCreate(); - if (_cancellation.CanBeCanceled) - { - _registration = _cancellation.Register(static state => ((FfiCall)state!).Cancel(), this); - } - return _token; - } - - /// Recovers the call behind a completion's context. Runs - /// exactly once per launch, on a producer thread. - internal static FfiCall Complete(IntPtr context) - { - var handle = GCHandle.FromIntPtr(context); - var call = (FfiCall)handle.Target!; - handle.Free(); - call.Release(); - return call; - } - - /// Releases everything when the launcher threw before the - /// producer took the call, so no completion will arrive. - internal void Abandon() - { - _self.Free(); - Release(); - } - - internal void SetResult(T value) - { - _tcs.TrySetResult(value); - } - - internal void SetException(Exception e) - { - _tcs.TrySetException(e); - } - - /// Faults (or cancels, for the cancelled code) the task from a - /// heap-boxed error, releasing the box. - internal unsafe void SetError(FfiError* err, FfiErrorMap map) - { - var e = Ffi.TakeBoxedError(err, map, out var code); - if (code == NativeException.CancelledErrorCode) - { - _tcs.TrySetCanceled(_cancellation); - } - else - { - _tcs.TrySetException(e); - } - } - - // The registration callback and the completion race: whichever runs - // last destroys the consumer's token reference, so a cancel never - // touches a destroyed token. - private void Cancel() - { - lock (_gate) - { - if (_completed) - { - return; - } - _cancelling = true; - } - NativeMethods.CancelTokenCancel(_token); - bool destroy; - lock (_gate) - { - _cancelling = false; - destroy = _completed; - } - if (destroy) - { - NativeMethods.CancelTokenDestroy(_token); - } - } - - private void Release() - { - if (_token == IntPtr.Zero) - { - return; - } - bool destroy; - lock (_gate) - { - _completed = true; - destroy = !_cancelling; - } - if (destroy) - { - _registration.Dispose(); - NativeMethods.CancelTokenDestroy(_token); - } - } -} - -/// Owns one native iterator and destroys it exactly once, when -/// enumeration ends, the enumerator is disposed, or the garbage collector -/// finalizes an abandoned one. -internal sealed unsafe class FfiIteratorHandle : SafeHandle -{ - private readonly delegate* _destroy; - - internal FfiIteratorHandle(IntPtr iterator, delegate* destroy) - : base(IntPtr.Zero, true) - { - _destroy = destroy; - SetHandle(iterator); - } - - public override bool IsInvalid => handle == IntPtr.Zero; - - protected override bool ReleaseHandle() - { - _destroy(handle); - return true; - } -} - -/// A lazily streamed sequence backed by a native iterator. It can be -/// enumerated once; each MoveNext pulls one item. -internal sealed unsafe class FfiSequence : IEnumerable, IEnumerator -{ - private readonly FfiIteratorHandle _iterator; - private readonly delegate* _next; - private int _started; - private T _current = default!; - - internal FfiSequence(FfiIteratorHandle iterator, delegate* next) - { - _iterator = iterator; - _next = next; - } - - public T Current => _current; - - object? IEnumerator.Current => _current; - - public IEnumerator GetEnumerator() - { - if (Interlocked.Exchange(ref _started, 1) != 0) - { - throw new InvalidOperationException("this sequence can be enumerated only once"); - } - return this; - } - - IEnumerator IEnumerable.GetEnumerator() - { - return GetEnumerator(); - } - - public bool MoveNext() - { - if (_iterator.IsClosed) - { - return false; - } - if (_next(_iterator, out _current)) - { - return true; - } - _iterator.Dispose(); - return false; - } - - public void Reset() - { - throw new NotSupportedException("a native iterator can't be reset"); - } - - public void Dispose() - { - _iterator.Dispose(); - } -} - -/// Serializes values into the value-buffer wire format -/// (little-endian, packed). -internal sealed class FfiBufferWriter -{ - private byte[] _buf = new byte[64]; - private int _len; - - /// The bytes written so far. - internal ReadOnlySpan Written => _buf.AsSpan(0, _len); - - /// The number of bytes written so far. - internal int Length => _len; - - private Span Grow(int extra) - { - if (_len + extra > _buf.Length) - { - Array.Resize(ref _buf, Math.Max(_buf.Length * 2, _len + extra)); - } - var span = _buf.AsSpan(_len, extra); - _len += extra; - return span; - } - - internal void WriteBool(bool v) => WriteU8(v ? (byte)1 : (byte)0); - - internal void WriteI8(sbyte v) => WriteU8((byte)v); - - internal void WriteU8(byte v) => Grow(1)[0] = v; - - internal void WriteI16(short v) => System.Buffers.Binary.BinaryPrimitives.WriteInt16LittleEndian(Grow(2), v); - - internal void WriteU16(ushort v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt16LittleEndian(Grow(2), v); - - internal void WriteI32(int v) => System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(Grow(4), v); - - internal void WriteU32(uint v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(Grow(4), v); - - internal void WriteI64(long v) => System.Buffers.Binary.BinaryPrimitives.WriteInt64LittleEndian(Grow(8), v); - - internal void WriteU64(ulong v) => System.Buffers.Binary.BinaryPrimitives.WriteUInt64LittleEndian(Grow(8), v); - - internal void WriteF32(float v) => System.Buffers.Binary.BinaryPrimitives.WriteSingleLittleEndian(Grow(4), v); - - internal void WriteF64(double v) => System.Buffers.Binary.BinaryPrimitives.WriteDoubleLittleEndian(Grow(8), v); - - internal void WriteLen(int len) => WriteU32((uint)len); - - internal void WriteString(string v) - { - var count = Encoding.UTF8.GetByteCount(v); - WriteLen(count); - Encoding.UTF8.GetBytes(v, Grow(count)); - } - - internal void WriteBytes(byte[] v) - { - WriteLen(v.Length); - v.CopyTo(Grow(v.Length)); - } - - /// Writes an object token. The token must carry its own strong - /// reference (a fresh clone), which the reader adopts. - internal void WriteObject(IntPtr token) => WriteU64((ulong)(nuint)token); -} - -/// Decodes values from the value-buffer wire format. A malformed -/// buffer is a producer or consumer bug and throws -/// . -internal sealed class FfiBufferReader -{ - private static readonly Encoding Utf8Strict = new UTF8Encoding(false, true); - - private readonly byte[] _data; - private int _pos; - - internal FfiBufferReader(byte[] data) - { - _data = data; - } - - private static InvalidOperationException Malformed(string what) - { - return new InvalidOperationException("malformed value buffer: " + what); - } - - private ReadOnlySpan Take(int n) - { - if (_data.Length - _pos < n) - { - throw Malformed("buffer exhausted"); - } - var span = _data.AsSpan(_pos, n); - _pos += n; - return span; - } - - internal bool ReadBool() - { - var b = ReadU8(); - if (b > 1) - { - throw Malformed("invalid bool byte"); - } - return b == 1; - } - - internal sbyte ReadI8() => (sbyte)ReadU8(); - - internal byte ReadU8() => Take(1)[0]; - - internal short ReadI16() => System.Buffers.Binary.BinaryPrimitives.ReadInt16LittleEndian(Take(2)); - - internal ushort ReadU16() => System.Buffers.Binary.BinaryPrimitives.ReadUInt16LittleEndian(Take(2)); - - internal int ReadI32() => System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(Take(4)); - - internal uint ReadU32() => System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(Take(4)); - - internal long ReadI64() => System.Buffers.Binary.BinaryPrimitives.ReadInt64LittleEndian(Take(8)); - - internal ulong ReadU64() => System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(Take(8)); - - internal float ReadF32() => System.Buffers.Binary.BinaryPrimitives.ReadSingleLittleEndian(Take(4)); - - internal double ReadF64() => System.Buffers.Binary.BinaryPrimitives.ReadDoubleLittleEndian(Take(8)); - - /// A collection count. Elements can be zero-sized, so the count - /// isn't checked against the remaining bytes; callers cap their - /// preallocation with . - internal int ReadLen() - { - var len = ReadU32(); - if (len > int.MaxValue) - { - throw Malformed("length prefix out of range"); - } - return (int)len; - } - - /// A preallocation size for elements - /// that a malformed count can't inflate. - internal int Capacity(int count) => Math.Min(count, _data.Length - _pos); - - internal string ReadString() - { - var bytes = Take(ReadLen()); - try - { - return Utf8Strict.GetString(bytes); - } - catch (ArgumentException) - { - throw Malformed("string is not valid UTF-8"); - } - } - - internal byte[] ReadBytes() => Take(ReadLen()).ToArray(); - - /// Reads an object token carrying one strong reference. - internal IntPtr ReadObject() - { - var token = ReadU64(); - if (token == 0) - { - throw Malformed("null object token"); - } - return (IntPtr)(nint)token; - } - - internal void ExpectEnd() - { - if (_pos != _data.Length) - { - throw Malformed("trailing bytes"); - } - } -} - -// END Runtime.cs diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_cs.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_cs.snap index 97a4573d..9dfb635f 100644 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_cs.snap +++ b/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_cs.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #nullable enable @@ -132,30 +132,30 @@ public abstract class Shape } case 1: { - var fRadius = reader.ReadF64(); - return new Circle(fRadius); + return new Circle( + reader.ReadF64()); } case 2: { - var fWidth = reader.ReadF32(); - var fHeight = reader.ReadF32(); - return new Rectangle(fWidth, fHeight); + return new Rectangle( + reader.ReadF32(), + reader.ReadF32()); } case 3: { - var fMin = reader.ReadI16(); - var fMax = reader.ReadI16(); - var fWeight = reader.ReadU64(); - return new Bounded(fMin, fMax, fWeight); + return new Bounded( + reader.ReadI16(), + reader.ReadI16(), + reader.ReadU64()); } case 4: { - var fLabel = reader.ReadString(); - var fCount = reader.ReadU8(); - return new Labeled(fLabel, fCount); + return new Labeled( + reader.ReadString(), + reader.ReadU8()); } default: - throw new InvalidOperationException("malformed value buffer: unknown Shape tag " + tag); + throw FfiBufferReader.Malformed("unknown Shape tag " + tag); } } } @@ -222,17 +222,17 @@ public sealed class Numerics internal static Numerics ReadFrom(FfiBufferReader reader) { - var fA = reader.ReadI8(); - var fB = reader.ReadI16(); - var fC = reader.ReadI32(); - var fD = reader.ReadI64(); - var fE = reader.ReadU8(); - var fF = reader.ReadU16(); - var fG = reader.ReadU32(); - var fH = reader.ReadU64(); - var fI = reader.ReadF32(); - var fJ = reader.ReadF64(); - return new Numerics(fA, fB, fC, fD, fE, fF, fG, fH, fI, fJ); + return new Numerics( + reader.ReadI8(), + reader.ReadI16(), + reader.ReadI32(), + reader.ReadI64(), + reader.ReadU8(), + reader.ReadU16(), + reader.ReadU32(), + reader.ReadU64(), + reader.ReadF32(), + reader.ReadF64()); } } @@ -249,7 +249,7 @@ public static unsafe class Shapes fixed (byte* shapePtr = shapeWriter.Written) { var ffiResult = NativeMethods.shapes_shapes_describe(shapePtr, (nuint)shapeWriter.Length, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeBugException.FromError); return Ffi.TakeString(ffiResult, ffiOutLen); } } @@ -264,11 +264,8 @@ public static unsafe class Shapes fixed (byte* shapePtr = shapeWriter.Written) { var ffiResult = NativeMethods.shapes_shapes_scale(shapePtr, (nuint)shapeWriter.Length, factor, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::Shapes.Shape.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::Shapes.Shape.ReadFrom(r)); } } @@ -279,7 +276,7 @@ public static unsafe class Shapes fixed (byte* valuesPtr = values) { var ffiResult = NativeMethods.shapes_shapes_sum_bytes(valuesPtr, (nuint)values.Length, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeException.FromError); + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeBugException.FromError); return ffiResult; } } @@ -293,24 +290,43 @@ public static unsafe class Shapes var ffiErr = default(FfiError); nuint ffiOutLen = 0; var ffiResult = NativeMethods.shapes_shapes_make_numerics(a, b, c, d, e, f, g, h, i, j, &ffiOutLen, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeException.FromError); - var ffiDecodedReader = Ffi.TakeBuffer(ffiResult, ffiOutLen); - var ffiDecoded = global::Shapes.Numerics.ReadFrom(ffiDecodedReader); - ffiDecodedReader.ExpectEnd(); - return ffiDecoded; + if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::Shapes.NativeBugException.FromError); + return FfiCodecs.Decode(Ffi.TakeBuffer(ffiResult, ffiOutLen), static r => global::Shapes.Numerics.ReadFrom(r)); + } +} + +/// The value-buffer codec of every composite type the API +/// carries. +internal static class FfiCodecs +{ + /// Reads one value that must fill the whole buffer. + internal static T Decode(FfiBufferReader reader, Func read) + { + var value = read(reader); + reader.ExpectEnd(); + return value; } } internal static unsafe partial class NativeMethods { - static partial void VerifyChecksums() + static partial void VerifyContracts() { - VerifyChecksum("shapes", &shapes_shapes_checksum, 0x06acf433ffa2a0f0UL); + VerifyContract("shapes_shapes_contract", &shapes_shapes_contract, new (ulong, ulong, string)[] + { + (0x0d7b3eec49243e7aUL, 0x5f232311808e1b1fUL, "shapes.Channel"), + (0x183fa8198100860eUL, 0xe40c6993cb31d359UL, "shapes.describe"), + (0x2d1395eb837b848eUL, 0x42202a0c12f3a16eUL, "shapes.sum_bytes"), + (0x7f9a5b87ba09b86bUL, 0xc5b32fb0efbbaa17UL, "shapes.scale"), + (0xc8edd779240fda17UL, 0x68e1be78fddb20e4UL, "shapes.Numerics"), + (0xd3d3881b2fd18a62UL, 0xf24d0eb76a99d918UL, "shapes.make_numerics"), + (0xf010844afff5ac3cUL, 0x50de28ae9b41ff15UL, "shapes.Shape"), + }); } - [LibraryImport(LibName, EntryPoint = "shapes_shapes_checksum")] + [LibraryImport(LibName, EntryPoint = "shapes_shapes_contract")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] - internal static partial ulong shapes_shapes_checksum(); + internal static partial FfiContractEntry* shapes_shapes_contract(nuint* out_len); [LibraryImport(LibName, EntryPoint = "shapes_shapes_describe")] [UnmanagedCallConv(CallConvs = new[] { typeof(CallConvCdecl) })] diff --git a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_csproj.snap b/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_csproj.snap deleted file mode 100644 index 1b6f1159..00000000 --- a/crates/weaveffi-cli/tests/snapshots/dotnet_shapes__Shapes_csproj.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - - - - - net8.0 - Shapes - Shapes - Shapes - 0.1.0 - Native bindings for shapes - enable - true - true - - $(NoWarn);CS0618 - - - - - diff --git a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__README_md.snap deleted file mode 100644 index a562400c..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__README_md.snap +++ /dev/null @@ -1,49 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# docs_everywhere (Go) - -cgo bindings for the `docs_everywhere` native library. - -## Build - -The module ships its own copy of the C header (`docs_everywhere.h`) and links with -`-ldocs_everywhere`. Point the linker at the directory holding `libdocs_everywhere.so`, -`libdocs_everywhere.dylib`, or `docs_everywhere.dll`, and make the library findable at -run time: - -```sh -export CGO_LDFLAGS="-L/path/to/lib" -export LD_LIBRARY_PATH="/path/to/lib" # DYLD_LIBRARY_PATH on macOS -go build ./... -``` - -Depend on the module from your own project with a `require` (and, for a -local checkout, a `replace`) of `docs_everywhere`, then import it: - -```go -import "docs_everywhere" -``` - -Go 1.23 or newer is required. - -## Usage notes - -- Records are plain structs and rich enums are sealed interfaces; both cross - the boundary by value. -- Interfaces are reference-counted objects. Call `Close` when you're done - with one (a finalizer releases it otherwise); it's safe to call twice or - while a call is in flight. -- Async functions take a `context.Context` first. Cancellable ones cancel - the native call when the context is done. -- Callback interfaces are Go interfaces you implement. The library may call - them from any thread, and a panic is reported to the native caller as a - foreign error instead of crashing. -- Loading checks the library's ABI revision and contract checksums and - panics with the stale module's name on a mismatch. - - diff --git a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__bindings_go.snap b/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__bindings_go.snap index 4a756837..fa245077 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__bindings_go.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__bindings_go.snap @@ -1,77 +1,129 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate +// Package docs_everywhere binds the docs_everywhere native library. package docs_everywhere /* #cgo LDFLAGS: -ldocs_everywhere #include "docs_everywhere.h" -__attribute__((unused)) static void* wvHandlePtr(uintptr_t h) { return (void*)h; } extern void goWv_docs_everywhere_docs_SavedListener_on_saved(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err); extern void goWv_docs_everywhere_docs_SavedListener_free(void* ctx); static const docs_everywhere_docs_SavedListener_vtable wvVtable_docs_everywhere_docs_SavedListener_vtable = { - goWv_docs_everywhere_docs_SavedListener_on_saved, - goWv_docs_everywhere_docs_SavedListener_free, + .size = sizeof(docs_everywhere_docs_SavedListener_vtable), + .flags = 0, + .free = goWv_docs_everywhere_docs_SavedListener_free, + .on_saved = goWv_docs_everywhere_docs_SavedListener_on_saved, }; __attribute__((unused)) static const docs_everywhere_docs_SavedListener_vtable* wvVtablePtr_docs_everywhere_docs_SavedListener_vtable(void) { return &wvVtable_docs_everywhere_docs_SavedListener_vtable; } -extern void goWv_docs_everywhere_docs_publish_async_callback(void* context_, docs_everywhere_error* err, bool result); +extern void goWv_docs_everywhere_docs_publish_async_callback(void* context, docs_everywhere_error* err, bool result); */ import "C" import ( "context" - "fmt" "runtime" - "runtime/cgo" "unsafe" ) func init() { wvCheckABI() - wvCheckModule("docs", 0x7950eec1e9cfc819, uint64(C.docs_everywhere_docs_checksum())) + var n C.size_t + table := C.docs_everywhere_docs_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0x00b2187fd75634ec, 0x2263c6f787bf6081, "docs.publish_async"}, + {0x12fc242b9dc6444c, 0x96405f484a642b31, "docs.Notebook.insert"}, + {0x2213822ed05cd5bd, 0x26c5833392aa63ce, "docs.Notebook.new"}, + {0x2bd762bcdaa207ff, 0x1b0ebab2d04c49b0, "docs.SavedListener"}, + {0x2e086c45e9d617de, 0x4d43bf32358c01ff, "docs.Notebook.find"}, + {0x4817dc98197ad960, 0xf8017b5a7d58b2c7, "docs.Notebook.size"}, + {0x614de47452d5a55c, 0xd9aae1472494ffef, "docs.get_document"}, + {0x67c5cb9fc119bcd5, 0x08d5b25a1a22a7fa, "docs.Document"}, + {0x8d484bf2d78c9b70, 0xe9093cf96912194c, "docs.Notebook.max_documents"}, + {0xaea61d6451691c67, 0xe4704128b3524b3d, "docs.watch"}, + {0xd68d6ad1540cdec5, 0x01afeb40b0a8bfe1, "docs.Notebook"}, + {0xdb0601444a4760cc, 0xe56c69629b444797, "docs.create_document"}, + {0xe12d6f4838347b3e, 0xa9bab86646d84f51, "docs.Status"}, + {0xee2f8d25b3313b50, 0x9691232f35439a2b, "docs.DocsErrors"}, + }) +} + +// DocsErrorsError is an error the docs module reports: *NotFoundError, or +// *InvalidStatusError. +// +// Match one code with errors.As and its type, or any of them with a +// DocsErrorsError. +type DocsErrorsError interface { + error + // Code returns the error's numeric code. + Code() int32 + isDocsErrorsError() } -// DocsErrorsError is a typed error reported by the `docs` module. -type DocsErrorsError struct { - // Code is the numeric ABI error code (one of the DocsErrorsError constants). - Code int32 - // Message is the human-readable error message. +// NotFoundError: Returned when no document matches the requested id +type NotFoundError struct { + // Message describes the failure; when it's empty, Error returns the + // code's default message, "Document not found". Message string } -func (e *DocsErrorsError) Error() string { - return fmt.Sprintf("docs: %s (code %d)", e.Message, e.Code) +// Error returns the error's message. +func (e *NotFoundError) Error() string { + if e.Message == "" { + return "Document not found" + } + return e.Message } -// DocsErrorsError codes. -const ( - // DocsErrorsErrorNotFound: Returned when no document matches the requested id - DocsErrorsErrorNotFound int32 = 1 - // DocsErrorsErrorInvalidStatus: Raised when attempting to move into a forbidden state - DocsErrorsErrorInvalidStatus int32 = 2 -) +// Code returns 1. +func (*NotFoundError) Code() int32 { + return 1 +} + +func (*NotFoundError) isDocsErrorsError() {} -// wvMapDocs converts a non-zero code from the `docs` domain into a -// *DocsErrorsError, falling back to the generic *Error for unknown codes. -func wvMapDocs(f wvFailure) error { +func (*NotFoundError) wvPayload() []byte { + return nil +} + +// InvalidStatusError: Raised when attempting to move into a forbidden state +type InvalidStatusError struct { + // Message describes the failure; when it's empty, Error returns the + // code's default message, "Invalid status transition". + Message string +} + +// Error returns the error's message. +func (e *InvalidStatusError) Error() string { + if e.Message == "" { + return "Invalid status transition" + } + return e.Message +} + +// Code returns 2. +func (*InvalidStatusError) Code() int32 { + return 2 +} + +func (*InvalidStatusError) isDocsErrorsError() {} + +func (*InvalidStatusError) wvPayload() []byte { + return nil +} + +func wvDocsErrorsError(f wvFailure) error { switch f.code { - case DocsErrorsErrorNotFound: - if f.message == "" { - f.message = "Document not found" - } - return &DocsErrorsError{Code: f.code, Message: f.message} - case DocsErrorsErrorInvalidStatus: - if f.message == "" { - f.message = "Invalid status transition" - } - return &DocsErrorsError{Code: f.code, Message: f.message} - default: - return f.err() + case 1: + return &NotFoundError{Message: f.message} + case 2: + return &InvalidStatusError{Message: f.message} } + return f.err() } // Status: Lifecycle status of a document. @@ -88,12 +140,20 @@ const ( StatusArchived Status = 2 ) -// Document: Represents a single document tracked by the system. +func wvWriteStatus(w *wvWriter, v Status) { + w.writeI32(int32(v)) +} + +func wvReadStatus(r *wvReader) Status { + return Status(r.readI32()) +} + +// Document represents a single document tracked by the system. // // Documents are persisted to disk and exposed via the public API. type Document struct { - // Id: Stable opaque identifier - Id int64 + // ID: Stable opaque identifier + ID int64 // Title: Human-readable title shown in lists Title string // Status: Current lifecycle status @@ -102,30 +162,19 @@ type Document struct { Tags []string } -// wvPackDocument appends v to w in the value-buffer wire format. -func wvPackDocument(w *wvWriter, v Document) { - w.writeI64(v.Id) +func wvWriteDocument(w *wvWriter, v Document) { + w.writeI64(v.ID) w.writeString(v.Title) - w.writeI32(int32(v.Status)) - w.writeLen(len(v.Tags)) - for _, eTags0 := range v.Tags { - w.writeString(eTags0) - } + wvWriteStatus(w, v.Status) + wvWriteListString(w, v.Tags) } -// wvUnpackDocument decodes one Document from r. -func wvUnpackDocument(r *wvReader) Document { +func wvReadDocument(r *wvReader) Document { var v Document - v.Id = r.readI64() + v.ID = r.readI64() v.Title = r.readString() - v.Status = Status(r.readI32()) - nTags0 := r.readLen() - v.Tags = make([]string, 0, r.capHint(nTags0)) - for range nTags0 { - var eTags0 string - eTags0 = r.readString() - v.Tags = append(v.Tags, eTags0) - } + v.Status = wvReadStatus(r) + v.Tags = wvReadListString(r) return v } @@ -133,45 +182,38 @@ func wvUnpackDocument(r *wvReader) Document { // // Implementations must be safe to call from any thread. // -// Implement this interface in Go and pass the value to native functions -// that accept it. -// -// The native library may call any method from any thread until it releases -// the implementation. A panic in a method is reported to the native caller -// as a foreign error (code -4) instead of crashing the process. +// Implement SavedListener in Go and pass the value to the functions that take +// it. The native library may call its methods from any thread until it +// releases the implementation. A method that panics, or a throwing method that +// returns an error outside its domain, fails the native call in progress as a +// callback failure (code -4). type SavedListener interface { - // OnSaved: Fires after a document has been persisted successfully + // OnSaved fires after a document has been persisted successfully // // Parameters: - // - doc_id: Identifier of the saved document + // + // - docID: Identifier of the saved document // - revision: Monotonically increasing revision - OnSaved(docId int64, revision int32) + OnSaved(docID int64, revision int32) } //export goWv_docs_everywhere_docs_SavedListener_on_saved func goWv_docs_everywhere_docs_SavedListener_on_saved(ctx unsafe.Pointer, doc_id C.int64_t, revision C.int32_t, out_err *C.docs_everywhere_error) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(SavedListener) - arg0 := int64(doc_id) - arg1 := int32(revision) - impl.OnSaved(arg0, arg1) + defer wvRecoverCallback(out_err) + wvCallback[SavedListener](ctx).OnSaved(int64(doc_id), int32(revision)) } //export goWv_docs_everywhere_docs_SavedListener_free func goWv_docs_everywhere_docs_SavedListener_free(ctx unsafe.Pointer) { - cgo.Handle(uintptr(ctx)).Delete() + wvFreeCallback(ctx) } -// Notebook: An ordered collection of documents. +// Notebook is an ordered collection of documents. // // Notebooks own their documents and release them when destroyed. // -// Each wrapper holds one strong reference; Close releases it, and a -// finalizer releases it if the wrapper is garbage collected first. +// Each wrapper holds one strong reference; Close releases it, and a finalizer +// releases it if the wrapper is garbage collected first. type Notebook struct { ref wvRef } @@ -180,8 +222,7 @@ func wvDestroyNotebook(ptr unsafe.Pointer) { C.docs_everywhere_docs_Notebook_destroy((*C.docs_everywhere_docs_Notebook)(ptr)) } -// wvAdoptNotebook adopts one owned strong reference into a new wrapper. A null -// pointer adopts to nil. +// wvAdoptNotebook wraps one owned strong reference. A null pointer adopts to nil. func wvAdoptNotebook(ptr *C.docs_everywhere_docs_Notebook) *Notebook { if ptr == nil { return nil @@ -201,28 +242,22 @@ func (s *Notebook) native() *C.docs_everywhere_docs_Notebook { return (*C.docs_everywhere_docs_Notebook)(s.ref.acquire("Notebook")) } -// wvTokenNotebook clones o's reference into a value-buffer object token. The -// wrapper keeps its own reference; the token carries the new one. -func wvTokenNotebook(o *Notebook) uint64 { - ptr := o.native() - defer o.ref.release() - return uint64(uintptr(unsafe.Pointer(C.docs_everywhere_docs_Notebook_clone(ptr)))) -} - -// wvUntokenNotebook adopts the strong reference carried by a value-buffer object -// token. -func wvUntokenNotebook(token uint64) *Notebook { - if token == 0 { - panic("docs_everywhere: malformed value buffer: null object token") +// share returns a new strong reference to the wrapper's object for the +// native library to adopt (the wrapper keeps its own), or null for nil. +func (s *Notebook) share() *C.docs_everywhere_docs_Notebook { + if s == nil { + return nil } - return wvAdoptNotebook((*C.docs_everywhere_docs_Notebook)(C.wvHandlePtr(C.uintptr_t(token)))) + ptr := s.native() + defer s.ref.release() + return C.docs_everywhere_docs_Notebook_clone(ptr) } // Close releases the wrapper's strong reference and always returns nil. // It's idempotent and safe to call from any goroutine, even while a call // on the wrapper is in flight: the reference is then released when that // call returns. The object itself is dropped once its last reference, from -// any wrapper, record, or the native side, is gone. +// any wrapper, record, or the native library, is gone. func (s *Notebook) Close() error { runtime.SetFinalizer(s, nil) s.ref.close() @@ -240,13 +275,12 @@ func NewNotebook() *Notebook { // Insert: Add a document, returning its position in the notebook // // Parameters: +// // - document: The document to add func (s *Notebook) Insert(document Document) int32 { cSelf := s.native() defer s.ref.release() - wDocument := &wvWriter{} - wvPackDocument(wDocument, document) - cDocumentPtr, cDocumentLen := wvBytes(wDocument.buf) + cDocumentPtr, cDocumentLen := wvBytes(wvEncode(document, wvWriteDocument)) var cErr C.docs_everywhere_error cRet := C.docs_everywhere_docs_Notebook_insert(cSelf, cDocumentPtr, cDocumentLen, &cErr) wvTrap(&cErr) @@ -256,6 +290,7 @@ func (s *Notebook) Insert(document Document) int32 { // Find: Look up a document by identifier, failing when none matches // // Parameters: +// // - id: Identifier of the document to find func (s *Notebook) Find(id int64) (Document, error) { cSelf := s.native() @@ -264,16 +299,12 @@ func (s *Notebook) Find(id int64) (Document, error) { var cErr C.docs_everywhere_error cRet := C.docs_everywhere_docs_Notebook_find(cSelf, C.int64_t(id), &cRetLen, &cErr) if cErr.code != 0 { - return Document{}, wvMapDocs(wvTakeError(&cErr)) + return Document{}, wvDocsErrorsError(wvTakeError(&cErr)) } - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Document - ret = wvUnpackDocument(rRet) - rRet.expectEnd() - return ret, nil + return wvDecode(cRet, cRetLen, wvReadDocument), nil } -// Size: The number of documents currently held +// Size returns the number of documents currently held func (s *Notebook) Size() int32 { cSelf := s.native() defer s.ref.release() @@ -283,7 +314,7 @@ func (s *Notebook) Size() int32 { return int32(cRet) } -// NotebookMaxDocuments: The largest number of documents one notebook will hold +// NotebookMaxDocuments returns the largest number of documents one notebook will hold func NotebookMaxDocuments() int32 { var cErr C.docs_everywhere_error cRet := C.docs_everywhere_docs_Notebook_max_documents(&cErr) @@ -297,50 +328,35 @@ func NotebookMaxDocuments() int32 { // it explicitly to make it visible. // // Parameters: +// // - title: Human-readable title for the new document // - tags: Initial set of tags (may be empty) func CreateDocument(title string, tags []string) Document { cTitlePtr, cTitleLen := wvStr(title) - wTags := &wvWriter{} - wTags.writeLen(len(tags)) - for _, eTags0 := range tags { - wTags.writeString(eTags0) - } - cTagsPtr, cTagsLen := wvBytes(wTags.buf) + cTagsPtr, cTagsLen := wvBytes(wvEncode(tags, wvWriteListString)) var cRetLen C.size_t var cErr C.docs_everywhere_error cRet := C.docs_everywhere_docs_create_document(cTitlePtr, cTitleLen, cTagsPtr, cTagsLen, &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Document - ret = wvUnpackDocument(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadDocument) } // GetDocument: Look up a document by identifier // // Parameters: +// // - id: Identifier returned from create_document func GetDocument(id int64) *Document { var cRetLen C.size_t var cErr C.docs_everywhere_error cRet := C.docs_everywhere_docs_get_document(C.int64_t(id), &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret *Document - if rRet.readOptionFlag() { - var oRet0 Document - oRet0 = wvUnpackDocument(rRet) - ret = &oRet0 - } - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadOptDocument) } //export goWv_docs_everywhere_docs_publish_async_callback -func goWv_docs_everywhere_docs_publish_async_callback(context_ unsafe.Pointer, err *C.docs_everywhere_error, result C._Bool) { - wvComplete(context_, err, func() bool { +func goWv_docs_everywhere_docs_publish_async_callback(context_ unsafe.Pointer, err_ *C.docs_everywhere_error, result C._Bool) { + wvComplete(context_, err_, func() bool { return bool(result) }) } @@ -348,6 +364,7 @@ func goWv_docs_everywhere_docs_publish_async_callback(context_ unsafe.Pointer, e // PublishAsync: Publish a document in the background // // Parameters: +// // - id: Identifier of the document to publish // // It blocks until the call completes or ctx is done; in the latter case it @@ -356,10 +373,9 @@ func PublishAsync(ctx context.Context, id int64) (bool, error) { if err := ctx.Err(); err != nil { return false, err } - wvDone := make(chan wvOutcome[bool], 1) - wvHandle := cgo.NewHandle(wvDone) - C.docs_everywhere_docs_publish_async(C.int64_t(id), C.docs_everywhere_docs_publish_async_callback(unsafe.Pointer(C.goWv_docs_everywhere_docs_publish_async_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[bool]() + C.docs_everywhere_docs_publish_async(C.int64_t(id), C.docs_everywhere_docs_publish_async_callback(unsafe.Pointer(C.goWv_docs_everywhere_docs_publish_async_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return false, err } @@ -372,12 +388,28 @@ func PublishAsync(ctx context.Context, id int64) (bool, error) { // Watch: Register a listener for saved notifications // // Parameters: +// // - listener: Receiver of saved notifications func Watch(listener SavedListener) { - hListener := cgo.NewHandle(listener) var cErr C.docs_everywhere_error - C.docs_everywhere_docs_watch(C.wvHandlePtr(C.uintptr_t(hListener)), C.wvVtablePtr_docs_everywhere_docs_SavedListener_vtable(), &cErr) + C.docs_everywhere_docs_watch(wvNewCallback(listener), C.wvVtablePtr_docs_everywhere_docs_SavedListener_vtable(), &cErr) wvTrap(&cErr) } +func wvWriteListString(w *wvWriter, v []string) { + wvWriteList(w, v, (*wvWriter).writeString) +} + +func wvReadListString(r *wvReader) []string { + return wvReadList(r, (*wvReader).readString) +} + +func wvWriteOptDocument(w *wvWriter, v *Document) { + wvWriteOptional(w, v, wvWriteDocument) +} + +func wvReadOptDocument(r *wvReader) *Document { + return wvReadOptional(r, wvReadDocument) +} + // END bindings.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__codec_go.snap b/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__codec_go.snap deleted file mode 100644 index 0db665ad..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__codec_go.snap +++ /dev/null @@ -1,205 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package docs_everywhere - -import ( - "encoding/binary" - "math" - "unicode/utf8" -) - -// wvWriter serializes values into the value-buffer format: little-endian, -// packed, u32 length prefixes. -type wvWriter struct { - buf []byte -} - -func (w *wvWriter) writeBool(v bool) { - if v { - w.buf = append(w.buf, 1) - } else { - w.buf = append(w.buf, 0) - } -} - -func (w *wvWriter) writeI8(v int8) { - w.buf = append(w.buf, byte(v)) -} - -func (w *wvWriter) writeU8(v uint8) { - w.buf = append(w.buf, v) -} - -func (w *wvWriter) writeI16(v int16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, uint16(v)) -} - -func (w *wvWriter) writeU16(v uint16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, v) -} - -func (w *wvWriter) writeI32(v int32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, uint32(v)) -} - -func (w *wvWriter) writeU32(v uint32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, v) -} - -func (w *wvWriter) writeI64(v int64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, uint64(v)) -} - -func (w *wvWriter) writeU64(v uint64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, v) -} - -func (w *wvWriter) writeF32(v float32) { - w.writeU32(math.Float32bits(v)) -} - -func (w *wvWriter) writeF64(v float64) { - w.writeU64(math.Float64bits(v)) -} - -func (w *wvWriter) writeLen(n int) { - if n < 0 || uint64(n) > math.MaxUint32 { - panic("docs_everywhere: value-buffer length exceeds u32 range") - } - w.writeU32(uint32(n)) -} - -func (w *wvWriter) writeString(v string) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeBytes(v []byte) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeOptionFlag(present bool) { - w.writeBool(present) -} - -// wvReader decodes values from the value-buffer format. A malformed buffer -// is a producer/consumer contract violation, so every read panics instead of -// returning a typed domain error. -type wvReader struct { - buf []byte - pos int -} - -func wvMalformed(context string) { - panic("docs_everywhere: malformed value buffer: " + context) -} - -func (r *wvReader) take(n int, context string) []byte { - if n < 0 || len(r.buf)-r.pos < n { - wvMalformed(context) - } - b := r.buf[r.pos : r.pos+n] - r.pos += n - return b -} - -func (r *wvReader) readBool() bool { - switch r.take(1, "bool")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("bool byte out of range") - return false -} - -func (r *wvReader) readI8() int8 { - return int8(r.take(1, "i8")[0]) -} - -func (r *wvReader) readU8() uint8 { - return r.take(1, "u8")[0] -} - -func (r *wvReader) readI16() int16 { - return int16(binary.LittleEndian.Uint16(r.take(2, "i16"))) -} - -func (r *wvReader) readU16() uint16 { - return binary.LittleEndian.Uint16(r.take(2, "u16")) -} - -func (r *wvReader) readI32() int32 { - return int32(binary.LittleEndian.Uint32(r.take(4, "i32"))) -} - -func (r *wvReader) readU32() uint32 { - return binary.LittleEndian.Uint32(r.take(4, "u32")) -} - -func (r *wvReader) readI64() int64 { - return int64(binary.LittleEndian.Uint64(r.take(8, "i64"))) -} - -func (r *wvReader) readU64() uint64 { - return binary.LittleEndian.Uint64(r.take(8, "u64")) -} - -func (r *wvReader) readF32() float32 { - return math.Float32frombits(r.readU32()) -} - -func (r *wvReader) readF64() float64 { - return math.Float64frombits(r.readU64()) -} - -// readLen reads a u32 element count. Elements may encode to zero bytes, so -// the count isn't checked against the remaining buffer; callers cap their -// preallocation with capHint instead. -func (r *wvReader) readLen() int { - return int(r.readU32()) -} - -// capHint bounds a preallocation for n decoded elements by the bytes left in -// the buffer, so a corrupt count can't force a huge allocation. -func (r *wvReader) capHint(n int) int { - return min(n, len(r.buf)-r.pos) -} - -func (r *wvReader) readString() string { - b := r.take(r.readLen(), "string bytes") - if !utf8.Valid(b) { - wvMalformed("string is not valid UTF-8") - } - return string(b) -} - -func (r *wvReader) readBytes() []byte { - return append([]byte{}, r.take(r.readLen(), "byte buffer")...) -} - -func (r *wvReader) readOptionFlag() bool { - switch r.take(1, "option flag")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("option flag byte out of range") - return false -} - -func (r *wvReader) expectEnd() { - if r.pos != len(r.buf) { - wvMalformed("trailing bytes after value") - } -} - -// END codec.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__docs_everywhere_h.snap b/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__docs_everywhere_h.snap deleted file mode 100644 index 31ab85e2..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__docs_everywhere_h.snap +++ /dev/null @@ -1,196 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef DOCS_EVERYWHERE_H -#define DOCS_EVERYWHERE_H - -#include -#include -#include - -#ifndef DOCS_EVERYWHERE_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef DOCS_EVERYWHERE_BUILD -# define DOCS_EVERYWHERE_API __declspec(dllexport) -# else -# define DOCS_EVERYWHERE_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define DOCS_EVERYWHERE_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define DOCS_EVERYWHERE_API __attribute__((visibility("default"))) -# else -# define DOCS_EVERYWHERE_API -# endif -#endif - -#ifndef DOCS_EVERYWHERE_DEPRECATED -# if defined(_MSC_VER) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define DOCS_EVERYWHERE_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports docs_everywhere_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define DOCS_EVERYWHERE_ABI_VERSION 3u -DOCS_EVERYWHERE_API uint32_t docs_everywhere_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by docs_everywhere_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct docs_everywhere_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} docs_everywhere_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_set(docs_everywhere_error* err, int32_t code, const char* message); -DOCS_EVERYWHERE_API void docs_everywhere_error_clear(docs_everywhere_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - docs_everywhere_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_free(docs_everywhere_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -DOCS_EVERYWHERE_API void docs_everywhere_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct docs_everywhere_cancel_token docs_everywhere_cancel_token; -DOCS_EVERYWHERE_API docs_everywhere_cancel_token* docs_everywhere_cancel_token_create(void); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_cancel(docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API bool docs_everywhere_cancel_token_is_cancelled(const docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_destroy(docs_everywhere_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -DOCS_EVERYWHERE_API uint8_t* docs_everywhere_alloc(uint32_t size); -DOCS_EVERYWHERE_API void docs_everywhere_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -DOCS_EVERYWHERE_API uint64_t docs_everywhere_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define DOCS_EVERYWHERE_DOCS_CHECKSUM 0x7950eec1e9cfc819ull -DOCS_EVERYWHERE_API uint64_t docs_everywhere_docs_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with docs_everywhere_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * docs_everywhere_buffer.h declares a C struct and codec for each. - */ - -/** - * Lifecycle status of a document. - * - * Used to drive UI badges and filters. - */ -typedef enum { - /** Document is being edited */ - docs_everywhere_docs_Status_Draft = 0, - /** Document is publicly visible */ - docs_everywhere_docs_Status_Published = 1, - /** Document is hidden from listings */ - docs_everywhere_docs_Status_Archived = 2 -} docs_everywhere_docs_Status; -/** Error codes reported by throwing functions in the `docs` module tree. */ -typedef enum { - /** Returned when no document matches the requested id */ - docs_everywhere_docs_DocsErrors_not_found = 1, - /** Raised when attempting to move into a forbidden state */ - docs_everywhere_docs_DocsErrors_invalid_status = 2 -} docs_everywhere_docs_DocsErrors; -typedef struct docs_everywhere_docs_Notebook docs_everywhere_docs_Notebook; -/** - * Consumer-implemented observer of document-saved notifications. - * - * Implementations must be safe to call from any thread. - * - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct docs_everywhere_docs_SavedListener_vtable { - /** Fires after a document has been persisted successfully */ - void (*on_saved)(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err); - void (*free)(void* ctx); -} docs_everywhere_docs_SavedListener_vtable; -typedef void (*docs_everywhere_docs_publish_async_callback)(void* context, docs_everywhere_error* err, bool result); - -// Module: docs -/** - * An ordered collection of documents. - * - * Notebooks own their documents and release them when destroyed. - */ -/** Create an empty notebook */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_new(docs_everywhere_error* out_err); -/** The largest number of documents one notebook will hold */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_max_documents(docs_everywhere_error* out_err); -/** Add a document, returning its position in the notebook */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_insert(const docs_everywhere_docs_Notebook* self, const uint8_t* document_ptr, size_t document_len, docs_everywhere_error* out_err); -/** Look up a document by identifier, failing when none matches */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_Notebook_find(const docs_everywhere_docs_Notebook* self, int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** The number of documents currently held */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_size(const docs_everywhere_docs_Notebook* self, docs_everywhere_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_clone(const docs_everywhere_docs_Notebook* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_Notebook_destroy(docs_everywhere_docs_Notebook* self); - -/** - * Create a brand new document. - * - * The document starts in `Draft` status; callers must publish - * it explicitly to make it visible. - */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_create_document(const uint8_t* title_ptr, size_t title_len, const uint8_t* tags_ptr, size_t tags_len, size_t* out_len, docs_everywhere_error* out_err); -/** Look up a document by identifier */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_get_document(int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** Publish a document in the background */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_publish_async(int64_t id, docs_everywhere_docs_publish_async_callback callback, void* context); -/** Register a listener for saved notifications */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_watch(void* listener_ctx, const docs_everywhere_docs_SavedListener_vtable* listener_vtable, docs_everywhere_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // DOCS_EVERYWHERE_H - -// END docs_everywhere.h diff --git a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__go_mod.snap b/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__go_mod.snap deleted file mode 100644 index 3e527b48..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__go_mod.snap +++ /dev/null @@ -1,12 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module docs_everywhere - -go 1.23 - -// END go.mod diff --git a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__runtime_go.snap b/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__runtime_go.snap deleted file mode 100644 index 6fd7b814..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_docs_everywhere__runtime_go.snap +++ /dev/null @@ -1,305 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package docs_everywhere - -/* -#include -#include "docs_everywhere.h" -*/ -import "C" - -import ( - "context" - "fmt" - "runtime/cgo" - "sync/atomic" - "unsafe" -) - -// wvABIVersion is the C ABI revision these bindings were generated against. -const wvABIVersion uint32 = 3 - -// The runtime error codes this package produces or interprets itself. -const ( - wvCodeMarshal int32 = -3 - wvCodeForeign int32 = -4 - wvCodeCancelled int32 = -5 -) - -// Error reports a failure that no typed error domain claims: an unknown -// code, a generic failure (code -1), a producer panic (-2), a marshalling -// failure (-3), or a Go callback-interface implementation that panicked -// (-4). The code is preserved so callers can tell them apart. -type Error struct { - // Code is the numeric ABI error code. - Code int32 - // Message is the human-readable error message. - Message string -} - -func (e *Error) Error() string { - return fmt.Sprintf("docs_everywhere: %s (code %d)", e.Message, e.Code) -} - -// DebugLive reports how many resources of one kind the native library holds: -// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned -// allocations. It's always 0 unless the library was built with leak checks -// enabled; tests use it to assert that nothing leaked. -func DebugLive(kind int32) uint64 { - return uint64(C.docs_everywhere_debug_live(C.int32_t(kind))) -} - -// wvCheckABI panics when the linked library implements a different C ABI -// revision than these bindings were generated for. -func wvCheckABI() { - if found := uint32(C.docs_everywhere_abi_version()); found != wvABIVersion { - panic(fmt.Sprintf("docs_everywhere: ABI mismatch: these bindings expect revision %d but the linked library reports revision %d", wvABIVersion, found)) - } -} - -// wvCheckModule panics when the linked library's contract checksum for a -// top-level module differs from the one these bindings were generated -// against, which means the bindings are stale. -func wvCheckModule(module string, want, found uint64) { - if found != want { - panic(fmt.Sprintf("docs_everywhere: module %q doesn't match the linked library (bindings checksum %#016x, library checksum %#016x); regenerate the bindings", module, want, found)) - } -} - -// ── Errors ── - -// wvFailure is a non-zero error slot copied into Go memory. -type wvFailure struct { - code int32 - message string - payload []byte -} - -// err converts the failure into the generic *Error. -func (f wvFailure) err() error { - return &Error{Code: f.code, Message: f.message} -} - -func wvCopyFailure(cErr *C.docs_everywhere_error) wvFailure { - f := wvFailure{code: int32(cErr.code)} - if cErr.message != nil { - f.message = C.GoString(cErr.message) - } - if cErr.payload_ptr != nil { - f.payload = wvBorrowBytes(cErr.payload_ptr, cErr.payload_len) - } - return f -} - -// wvTakeError copies a non-zero error slot and clears it. -func wvTakeError(cErr *C.docs_everywhere_error) wvFailure { - f := wvCopyFailure(cErr) - C.docs_everywhere_error_clear(cErr) - return f -} - -// wvTakeBoxedError copies the heap-boxed error an async completion receives -// and frees the box. -func wvTakeBoxedError(cErr *C.docs_everywhere_error) wvFailure { - f := wvCopyFailure(cErr) - C.docs_everywhere_error_free(cErr) - return f -} - -// wvTrap panics with the generic *Error when the slot reports a failure. -// Wrappers of functions that declare no error domain check their slot with -// it: a failure there can only be a producer bug, a marshalling failure, or -// a callback-interface implementation that panicked. -func wvTrap(cErr *C.docs_everywhere_error) { - if cErr.code != 0 { - panic(wvTakeError(cErr).err()) - } -} - -// wvForeignError reports a panic recovered inside a callback-interface -// trampoline through the producer's error slot (code -4). The producer -// copies the message, so it's only borrowed for the call. -func wvForeignError(outErr *C.docs_everywhere_error, recovered any) { - msg := C.CString(fmt.Sprint(recovered)) - defer C.free(unsafe.Pointer(msg)) - C.docs_everywhere_error_set(outErr, C.int32_t(wvCodeForeign), msg) -} - -// ── Strings and bytes ── - -// wvStr returns a borrowed (pointer, length) view of s's UTF-8 bytes for the -// duration of one call. Strings may contain NUL bytes. -func wvStr(s string) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.StringData(s))), C.size_t(len(s)) -} - -// wvBytes returns a borrowed (pointer, length) view of b for the duration of -// one call. -func wvBytes(b []byte) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.SliceData(b))), C.size_t(len(b)) -} - -// wvBorrowString copies a borrowed UTF-8 run into a Go string. -func wvBorrowString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - return string(unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) -} - -// wvTakeString copies a returned UTF-8 run into a Go string and releases the -// producer's allocation. -func wvTakeString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - s := wvBorrowString(ptr, n) - C.docs_everywhere_free_bytes(ptr, n) - return s -} - -// wvBorrowBytes copies a borrowed byte run into Go memory. A null pointer -// copies to nil. -func wvBorrowBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - return append([]byte{}, unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)...) -} - -// wvTakeBytes copies a returned byte run (raw bytes or a value buffer) into -// Go memory and releases the producer's allocation. -func wvTakeBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - b := wvBorrowBytes(ptr, n) - C.docs_everywhere_free_bytes(ptr, n) - return b -} - -// ── Objects ── - -// wvRef owns one strong reference to a native object on behalf of a wrapper. -// Calls borrow the pointer between acquire and release. Close may race them: -// the reference is destroyed exactly once, by close when no call is in -// flight, or else by the last in-flight call to finish. -type wvRef struct { - ptr unsafe.Pointer - destroy func(unsafe.Pointer) - // state counts in-flight calls in its upper bits; the low bit is set - // once close has run. - state atomic.Uint64 -} - -func (r *wvRef) init(ptr unsafe.Pointer, destroy func(unsafe.Pointer)) { - r.ptr = ptr - r.destroy = destroy -} - -// acquire borrows the pointer for one call, panicking when the wrapper was -// already closed. -func (r *wvRef) acquire(typeName string) unsafe.Pointer { - for { - s := r.state.Load() - if s&1 != 0 { - panic("docs_everywhere: " + typeName + " used after Close") - } - if r.state.CompareAndSwap(s, s+2) { - return r.ptr - } - } -} - -// release ends one call's borrow. -func (r *wvRef) release() { - if r.state.Add(^uint64(1)) == 1 { - r.destroy(r.ptr) - } -} - -// close releases the wrapper's reference once no call is in flight. Calling -// it again is a no-op. -func (r *wvRef) close() { - for { - s := r.state.Load() - if s&1 != 0 { - return - } - if r.state.CompareAndSwap(s, s|1) { - if s == 0 { - r.destroy(r.ptr) - } - return - } - } -} - -// ── Async ── - -// wvOutcome is one async completion: a converted result, or the failure the -// producer reported. -type wvOutcome[T any] struct { - val T - fail *wvFailure -} - -// wvComplete delivers one async completion to the waiting wrapper. It runs on -// a producer thread inside an exported trampoline, so it never panics: a -// result that fails to decode is delivered as a marshalling failure. -func wvComplete[T any](handle unsafe.Pointer, cErr *C.docs_everywhere_error, convert func() T) { - h := cgo.Handle(uintptr(handle)) - done := h.Value().(chan wvOutcome[T]) - h.Delete() - done <- wvSettle(cErr, convert) -} - -func wvSettle[T any](cErr *C.docs_everywhere_error, convert func() T) (o wvOutcome[T]) { - if cErr != nil { - if f := wvTakeBoxedError(cErr); f.code != 0 { - o.fail = &f - return o - } - } - defer func() { - if r := recover(); r != nil { - o = wvOutcome[T]{fail: &wvFailure{code: wvCodeMarshal, message: fmt.Sprint(r)}} - } - }() - o.val = convert() - return o -} - -// wvAwait blocks until an async call completes or ctx is done. With a cancel -// token, ctx's cancellation cancels the native call and waits for its -// completion, which the producer delivers promptly; without one, the call is -// abandoned and ctx.Err() returned at once. A cancelled completion (code -5) -// returns ctx.Err(), or context.Canceled when ctx itself isn't done. -func wvAwait[T any](ctx context.Context, done chan wvOutcome[T], token *C.docs_everywhere_cancel_token) (T, *wvFailure, error) { - var o wvOutcome[T] - select { - case o = <-done: - case <-ctx.Done(): - if token == nil { - var zero T - return zero, nil, ctx.Err() - } - C.docs_everywhere_cancel_token_cancel(token) - o = <-done - } - if o.fail != nil && o.fail.code == wvCodeCancelled { - var zero T - if err := ctx.Err(); err != nil { - return zero, nil, err - } - return zero, nil, context.Canceled - } - return o.val, o.fail, nil -} - -// END runtime.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/go_edge_cases__README_md.snap deleted file mode 100644 index e837672a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__README_md.snap +++ /dev/null @@ -1,49 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# edge_cases (Go) - -cgo bindings for the `edge_cases` native library. - -## Build - -The module ships its own copy of the C header (`edge_cases.h`) and links with -`-ledge_cases`. Point the linker at the directory holding `libedge_cases.so`, -`libedge_cases.dylib`, or `edge_cases.dll`, and make the library findable at -run time: - -```sh -export CGO_LDFLAGS="-L/path/to/lib" -export LD_LIBRARY_PATH="/path/to/lib" # DYLD_LIBRARY_PATH on macOS -go build ./... -``` - -Depend on the module from your own project with a `require` (and, for a -local checkout, a `replace`) of `edge_cases`, then import it: - -```go -import "edge_cases" -``` - -Go 1.23 or newer is required. - -## Usage notes - -- Records are plain structs and rich enums are sealed interfaces; both cross - the boundary by value. -- Interfaces are reference-counted objects. Call `Close` when you're done - with one (a finalizer releases it otherwise); it's safe to call twice or - while a call is in flight. -- Async functions take a `context.Context` first. Cancellable ones cancel - the native call when the context is done. -- Callback interfaces are Go interfaces you implement. The library may call - them from any thread, and a panic is reported to the native caller as a - foreign error instead of crashing. -- Loading checks the library's ABI revision and contract checksums and - panics with the stale module's name on a mismatch. - - diff --git a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__bindings_go.snap b/crates/weaveffi-cli/tests/snapshots/go_edge_cases__bindings_go.snap index 9ed0f38d..9ec90eb1 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__bindings_go.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_edge_cases__bindings_go.snap @@ -1,10 +1,11 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate +// Package edge_cases binds the edge_cases native library. package edge_cases /* @@ -15,86 +16,153 @@ __attribute__((unused)) static void* wvHandlePtr(uintptr_t h) { return (void*)h; extern void goWv_edge_cases_edge_cases_BatchListener_on_batch(void* ctx, uint8_t* items_ptr, size_t items_len, uint8_t* raw_ptr, size_t raw_len, uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err); extern void goWv_edge_cases_edge_cases_BatchListener_on_widget(void* ctx, edge_cases_edge_cases_Widget* widget, edge_cases_edge_cases_Widget* fallback, uint8_t* children_ptr, size_t children_len, edge_cases_error* out_err); extern void goWv_edge_cases_edge_cases_BatchListener_on_tick(void* ctx, edge_cases_error* out_err); -extern bool goWv_edge_cases_edge_cases_BatchListener_should_continue(void* ctx, int32_t class, edge_cases_error* out_err); +extern bool goWv_edge_cases_edge_cases_BatchListener_should_continue(void* ctx, int32_t class_, edge_cases_error* out_err); extern edge_cases_edge_cases_Mode goWv_edge_cases_edge_cases_BatchListener_pick_mode(void* ctx, edge_cases_error* out_err); extern void goWv_edge_cases_edge_cases_BatchListener_free(void* ctx); static const edge_cases_edge_cases_BatchListener_vtable wvVtable_edge_cases_edge_cases_BatchListener_vtable = { - (void (*)(void*, const uint8_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t, edge_cases_edge_cases_Mode, edge_cases_error*))goWv_edge_cases_edge_cases_BatchListener_on_batch, - (void (*)(void*, edge_cases_edge_cases_Widget*, edge_cases_edge_cases_Widget*, const uint8_t*, size_t, edge_cases_error*))goWv_edge_cases_edge_cases_BatchListener_on_widget, - goWv_edge_cases_edge_cases_BatchListener_on_tick, - goWv_edge_cases_edge_cases_BatchListener_should_continue, - goWv_edge_cases_edge_cases_BatchListener_pick_mode, - goWv_edge_cases_edge_cases_BatchListener_free, + .size = sizeof(edge_cases_edge_cases_BatchListener_vtable), + .flags = 0, + .free = goWv_edge_cases_edge_cases_BatchListener_free, + .on_batch = (void (*)(void*, const uint8_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t, edge_cases_edge_cases_Mode, edge_cases_error*))goWv_edge_cases_edge_cases_BatchListener_on_batch, + .on_widget = (void (*)(void*, edge_cases_edge_cases_Widget*, edge_cases_edge_cases_Widget*, const uint8_t*, size_t, edge_cases_error*))goWv_edge_cases_edge_cases_BatchListener_on_widget, + .on_tick = goWv_edge_cases_edge_cases_BatchListener_on_tick, + .should_continue = goWv_edge_cases_edge_cases_BatchListener_should_continue, + .pick_mode = goWv_edge_cases_edge_cases_BatchListener_pick_mode, }; __attribute__((unused)) static const edge_cases_edge_cases_BatchListener_vtable* wvVtablePtr_edge_cases_edge_cases_BatchListener_vtable(void) { return &wvVtable_edge_cases_edge_cases_BatchListener_vtable; } extern void goWv_edge_cases_edge_cases_Keyword_delete(void* ctx, int32_t self, edge_cases_error* out_err); extern int32_t goWv_edge_cases_edge_cases_Keyword_import(void* ctx, edge_cases_error* out_err); extern void goWv_edge_cases_edge_cases_Keyword_free(void* ctx); static const edge_cases_edge_cases_Keyword_vtable wvVtable_edge_cases_edge_cases_Keyword_vtable = { - goWv_edge_cases_edge_cases_Keyword_delete, - goWv_edge_cases_edge_cases_Keyword_import, - goWv_edge_cases_edge_cases_Keyword_free, + .size = sizeof(edge_cases_edge_cases_Keyword_vtable), + .flags = 0, + .free = goWv_edge_cases_edge_cases_Keyword_free, + .delete_ = goWv_edge_cases_edge_cases_Keyword_delete, + .import = goWv_edge_cases_edge_cases_Keyword_import, }; __attribute__((unused)) static const edge_cases_edge_cases_Keyword_vtable* wvVtablePtr_edge_cases_edge_cases_Keyword_vtable(void) { return &wvVtable_edge_cases_edge_cases_Keyword_vtable; } -extern void goWv_edge_cases_edge_cases_async_widget_callback(void* context_, edge_cases_error* err, edge_cases_edge_cases_Widget* result); -extern void goWv_edge_cases_edge_cases_async_void_callback(void* context_, edge_cases_error* err); -extern void goWv_edge_cases_edge_cases_async_bytes_callback(void* context_, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); -extern void goWv_edge_cases_edge_cases_async_struct_callback(void* context_, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); -extern void goWv_edge_cases_edge_cases_async_list_callback(void* context_, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); -extern void goWv_edge_cases_edge_cases_async_optional_callback(void* context_, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); -extern void goWv_edge_cases_edge_cases_async_enum_callback(void* context_, edge_cases_error* err, edge_cases_edge_cases_Mode result); -extern void goWv_edge_cases_edge_cases_async_keywords_callback(void* context_, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); -extern void goWv_edge_cases_edge_cases_Widget_fetch_callback(void* context_, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_edge_cases_edge_cases_async_widget_callback(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result); +extern void goWv_edge_cases_edge_cases_async_void_callback(void* context, edge_cases_error* err); +extern void goWv_edge_cases_edge_cases_async_bytes_callback(void* context, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_edge_cases_edge_cases_async_struct_callback(void* context, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_edge_cases_edge_cases_async_list_callback(void* context, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_edge_cases_edge_cases_async_optional_callback(void* context, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_edge_cases_edge_cases_async_enum_callback(void* context, edge_cases_error* err, edge_cases_edge_cases_Mode result); +extern void goWv_edge_cases_edge_cases_async_keywords_callback(void* context, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_edge_cases_edge_cases_Widget_fetch_callback(void* context, edge_cases_error* err, uint8_t* result_ptr, size_t result_len); */ import "C" import ( "context" - "fmt" "iter" "runtime" - "runtime/cgo" "unsafe" ) func init() { wvCheckABI() - wvCheckModule("edge_cases", 0x1567f441073bd4aa, uint64(C.edge_cases_edge_cases_checksum())) + var n C.size_t + table := C.edge_cases_edge_cases_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0x056a86511785ba2b, 0x5cba3f16d57c6cdb, "edge_cases.make_widget"}, + {0x07bfa9ecb2b720d2, 0x3813b3f57c64bf35, "edge_cases.EdgeErrors"}, + {0x0aaa11867e37f991, 0xf090dd52146ebdf0, "edge_cases.Widget.refresh"}, + {0x0f9701493898ae4c, 0x6944f47a275c96e8, "edge_cases.Inner"}, + {0x1226b1adac179f7d, 0x0e95a00ab0bef637, "edge_cases.async_keywords"}, + {0x16d36bef1541136c, 0x5ea7021ec324298f, "edge_cases.find_widget"}, + {0x1b7324c55e980feb, 0xabf87fe372358f42, "edge_cases.Widget.old_name"}, + {0x1cbd731467c2bab8, 0x4915b9989c2c9074, "edge_cases.Vault"}, + {0x2c2a6de67d894b0a, 0x50a4496b1e7191ca, "edge_cases.Vault.open"}, + {0x322b4cfcd2d6861c, 0xca7527a379ae8465, "edge_cases.Keywords"}, + {0x3f8e23a5f32906f4, 0x8d4c908d796f130d, "edge_cases.Leaf"}, + {0x42ab8a685457b97c, 0xe5ce30500cadb32d, "edge_cases.Vault.handle"}, + {0x4a68540e2836d3d0, 0xa20e44c71e420ed2, "edge_cases.mode_stream"}, + {0x4e030580ac0b111d, 0xb68b7a18d910e175, "edge_cases.legacy_value"}, + {0x4f5e4e5291864ef3, 0x55a1a88b21a08f8b, "edge_cases.Legacy"}, + {0x4f84b06171b4213d, 0xbaedcfdebb6a2820, "edge_cases.scalar_stream"}, + {0x5067a5ff2d9ef910, 0xbfc79b18fc4714fc, "edge_cases.Widget"}, + {0x59aff159112fbf5b, 0x18e4def57a8cfa49, "edge_cases.optional_params"}, + {0x5daf65b98fa77c48, 0xc817a8cb202e59c2, "edge_cases.keywords_roundtrip"}, + {0x5de8a5c9726c2c5a, 0x7c50b5b65f7a896d, "edge_cases.register_batch"}, + {0x618a620309994c3c, 0x826ad0b1d208d723, "edge_cases.async_bytes"}, + {0x648285cdba0a7b8e, 0xcf6e80e783a4d8be, "edge_cases.WidgetHolder"}, + {0x649c78079824b10d, 0x5b12dd26e0634b69, "edge_cases.async_widget"}, + {0x7da7a7357d122a37, 0xf544e025ffaa3d20, "edge_cases.async_void"}, + {0x7e4015bc3a472e2d, 0xfe22291d90b82e4a, "edge_cases.widget_stream"}, + {0x805276e5d81fdba7, 0xd9b64e360bd03b2b, "edge_cases.Outer"}, + {0x80e1183326542df4, 0xe8a57e12cc6ed276, "edge_cases.select"}, + {0x923ce5e9bcd769a1, 0x0b973f5dc07dda8b, "edge_cases.Vault.to_string"}, + {0x98ca8c01f5239404, 0xfe11c93eb6b303ef, "edge_cases.Widget.rename"}, + {0xa13e22e98ea278d4, 0xf1fae215d7387800, "edge_cases.widgets_everywhere"}, + {0xa84a8c9bf2a881e1, 0xbe1dc840bf221996, "edge_cases.async_optional"}, + {0xad958c478fbf205d, 0xe386b0716d24f19b, "edge_cases.Retired"}, + {0xb2fc3964ef56ab39, 0x73c325c8bc351236, "edge_cases.async_list"}, + {0xb3403af521cf0006, 0x6abc30d48206a3ba, "edge_cases.Widget.close"}, + {0xb794304a22a406b8, 0xf6fb98711c516105, "edge_cases.async_struct"}, + {0xba3936106a49c47d, 0xb7025f2fadd6288e, "edge_cases.Vault.peek"}, + {0xc155b0c3cfb164f4, 0xc777ae57d92ebc47, "edge_cases.noop"}, + {0xcdc18607777019dd, 0xb81a31e5c20aa7a8, "edge_cases.inspect_widget"}, + {0xd164a82aeb64b8cc, 0xded02c68005c4737, "edge_cases.nested_optional"}, + {0xd978ccad2468038f, 0x30c39c65d6235153, "edge_cases.Mode"}, + {0xdbe9d947431c9124, 0x39537fb848fdf980, "edge_cases.BatchListener"}, + {0xeae8b7862c08bb07, 0xfe12eeaa830452ea, "edge_cases.Keyword"}, + {0xf1e155a05ae262b8, 0xb575eb818a423d42, "edge_cases.async_enum"}, + {0xf22e210bdbce7343, 0x2a5e0f801bc8c768, "edge_cases.Widget.dispose"}, + {0xf3c7cf675563ee6c, 0x0ecffeef399facc8, "edge_cases.Vault.probe"}, + {0xfd34bc67acac379e, 0x576caa14f3c0901a, "edge_cases.Widget.fetch"}, + {0xff2e9d2bff941786, 0x7e96764ef1f5fbaa, "edge_cases.string_stream"}, + }) } -// EdgeErrorsError is a typed error reported by the `edge_cases` module. -type EdgeErrorsError struct { - // Code is the numeric ABI error code (one of the EdgeErrorsError constants). - Code int32 - // Message is the human-readable error message. +// EdgeErrorsError is an error the edge_cases module reports: *BusyError. +// +// Match one code with errors.As and its type, or any of them with a +// EdgeErrorsError. +type EdgeErrorsError interface { + error + // Code returns the error's numeric code. + Code() int32 + isEdgeErrorsError() +} + +// BusyError is the EdgeErrorsError code Busy (7): Resource is busy. +type BusyError struct { + // Message describes the failure; when it's empty, Error returns the + // code's default message, "Resource is busy". Message string } -func (e *EdgeErrorsError) Error() string { - return fmt.Sprintf("edge_cases: %s (code %d)", e.Message, e.Code) +// Error returns the error's message. +func (e *BusyError) Error() string { + if e.Message == "" { + return "Resource is busy" + } + return e.Message } -// EdgeErrorsError codes. -const ( - // EdgeErrorsErrorBusy: Resource is busy - EdgeErrorsErrorBusy int32 = 7 -) +// Code returns 7. +func (*BusyError) Code() int32 { + return 7 +} + +func (*BusyError) isEdgeErrorsError() {} -// wvMapEdgeCases converts a non-zero code from the `edge_cases` domain into a -// *EdgeErrorsError, falling back to the generic *Error for unknown codes. -func wvMapEdgeCases(f wvFailure) error { +func (*BusyError) wvPayload() []byte { + return nil +} + +func wvEdgeErrorsError(f wvFailure) error { switch f.code { - case EdgeErrorsErrorBusy: - if f.message == "" { - f.message = "Resource is busy" - } - return &EdgeErrorsError{Code: f.code, Message: f.message} - default: - return f.err() + case 7: + return &BusyError{Message: f.message} } + return f.err() } // Legacy: Deprecated enum with non-contiguous values and keyword-like variants +// +// Deprecated: Use Mode instead type Legacy int32 const ( @@ -110,17 +178,23 @@ const ( ModeSafe Mode = 2 ) +func wvWriteMode(w *wvWriter, v Mode) { + w.writeI32(int32(v)) +} + +func wvReadMode(r *wvReader) Mode { + return Mode(r.readI32()) +} + type Leaf struct { Weight float32 } -// wvPackLeaf appends v to w in the value-buffer wire format. -func wvPackLeaf(w *wvWriter, v Leaf) { +func wvWriteLeaf(w *wvWriter, v Leaf) { w.writeF32(v.Weight) } -// wvUnpackLeaf decodes one Leaf from r. -func wvUnpackLeaf(r *wvReader) Leaf { +func wvReadLeaf(r *wvReader) Leaf { var v Leaf v.Weight = r.readF32() return v @@ -132,26 +206,15 @@ type Inner struct { Leaves []Leaf } -// wvPackInner appends v to w in the value-buffer wire format. -func wvPackInner(w *wvWriter, v Inner) { - wvPackLeaf(w, v.Leaf) - w.writeLen(len(v.Leaves)) - for _, eLeaves0 := range v.Leaves { - wvPackLeaf(w, eLeaves0) - } +func wvWriteInner(w *wvWriter, v Inner) { + wvWriteLeaf(w, v.Leaf) + wvWriteListLeaf(w, v.Leaves) } -// wvUnpackInner decodes one Inner from r. -func wvUnpackInner(r *wvReader) Inner { +func wvReadInner(r *wvReader) Inner { var v Inner - v.Leaf = wvUnpackLeaf(r) - nLeaves0 := r.readLen() - v.Leaves = make([]Leaf, 0, r.capHint(nLeaves0)) - for range nLeaves0 { - var eLeaves0 Leaf - eLeaves0 = wvUnpackLeaf(r) - v.Leaves = append(v.Leaves, eLeaves0) - } + v.Leaf = wvReadLeaf(r) + v.Leaves = wvReadListLeaf(r) return v } @@ -164,134 +227,36 @@ type Outer struct { Sparse []*int64 Blob []byte Note *string - ById map[int64]string + ByID map[int64]string Hidden bool V2 uint8 } -// wvPackOuter appends v to w in the value-buffer wire format. -func wvPackOuter(w *wvWriter, v Outer) { - wvPackInner(w, v.Inner) - if v.MaybeInner == nil { - w.writeOptionFlag(false) - } else { - w.writeOptionFlag(true) - wvPackInner(w, *v.MaybeInner) - } - w.writeLen(len(v.ByName)) - for kByName0, vByName0 := range v.ByName { - w.writeString(kByName0) - wvPackInner(w, vByName0) - } - w.writeLen(len(v.Modes)) - for _, eModes0 := range v.Modes { - w.writeI32(int32(eModes0)) - } - w.writeLen(len(v.Matrix)) - for _, eMatrix0 := range v.Matrix { - w.writeLen(len(eMatrix0)) - for _, eMatrix1 := range eMatrix0 { - w.writeI32(eMatrix1) - } - } - w.writeLen(len(v.Sparse)) - for _, eSparse0 := range v.Sparse { - if eSparse0 == nil { - w.writeOptionFlag(false) - } else { - w.writeOptionFlag(true) - w.writeI64(*eSparse0) - } - } - if v.Blob == nil { - w.writeOptionFlag(false) - } else { - w.writeOptionFlag(true) - w.writeBytes(v.Blob) - } - if v.Note == nil { - w.writeOptionFlag(false) - } else { - w.writeOptionFlag(true) - w.writeString(*v.Note) - } - w.writeLen(len(v.ById)) - for kById0, vById0 := range v.ById { - w.writeI64(kById0) - w.writeString(vById0) - } +func wvWriteOuter(w *wvWriter, v Outer) { + wvWriteInner(w, v.Inner) + wvWriteOptInner(w, v.MaybeInner) + wvWriteMapStringInner(w, v.ByName) + wvWriteListMode(w, v.Modes) + wvWriteListListI32(w, v.Matrix) + wvWriteListOptI64(w, v.Sparse) + wvWriteOptBytes(w, v.Blob) + wvWriteOptString(w, v.Note) + wvWriteMapI64String(w, v.ByID) w.writeBool(v.Hidden) w.writeU8(v.V2) } -// wvUnpackOuter decodes one Outer from r. -func wvUnpackOuter(r *wvReader) Outer { +func wvReadOuter(r *wvReader) Outer { var v Outer - v.Inner = wvUnpackInner(r) - if r.readOptionFlag() { - var oMaybeInner0 Inner - oMaybeInner0 = wvUnpackInner(r) - v.MaybeInner = &oMaybeInner0 - } - nByName0 := r.readLen() - v.ByName = make(map[string]Inner, r.capHint(nByName0)) - for range nByName0 { - var kByName0 string - kByName0 = r.readString() - var vByName0 Inner - vByName0 = wvUnpackInner(r) - v.ByName[kByName0] = vByName0 - } - nModes0 := r.readLen() - v.Modes = make([]Mode, 0, r.capHint(nModes0)) - for range nModes0 { - var eModes0 Mode - eModes0 = Mode(r.readI32()) - v.Modes = append(v.Modes, eModes0) - } - nMatrix0 := r.readLen() - v.Matrix = make([][]int32, 0, r.capHint(nMatrix0)) - for range nMatrix0 { - var eMatrix0 []int32 - nMatrix1 := r.readLen() - eMatrix0 = make([]int32, 0, r.capHint(nMatrix1)) - for range nMatrix1 { - var eMatrix1 int32 - eMatrix1 = r.readI32() - eMatrix0 = append(eMatrix0, eMatrix1) - } - v.Matrix = append(v.Matrix, eMatrix0) - } - nSparse0 := r.readLen() - v.Sparse = make([]*int64, 0, r.capHint(nSparse0)) - for range nSparse0 { - var eSparse0 *int64 - if r.readOptionFlag() { - var oSparse1 int64 - oSparse1 = r.readI64() - eSparse0 = &oSparse1 - } - v.Sparse = append(v.Sparse, eSparse0) - } - if r.readOptionFlag() { - var oBlob0 []byte - oBlob0 = r.readBytes() - v.Blob = oBlob0 - } - if r.readOptionFlag() { - var oNote0 string - oNote0 = r.readString() - v.Note = &oNote0 - } - nById0 := r.readLen() - v.ById = make(map[int64]string, r.capHint(nById0)) - for range nById0 { - var kById0 int64 - kById0 = r.readI64() - var vById0 string - vById0 = r.readString() - v.ById[kById0] = vById0 - } + v.Inner = wvReadInner(r) + v.MaybeInner = wvReadOptInner(r) + v.ByName = wvReadMapStringInner(r) + v.Modes = wvReadListMode(r) + v.Matrix = wvReadListListI32(r) + v.Sparse = wvReadListOptI64(r) + v.Blob = wvReadOptBytes(r) + v.Note = wvReadOptString(r) + v.ByID = wvReadMapI64String(r) v.Hidden = r.readBool() v.V2 = r.readU8() return v @@ -360,8 +325,7 @@ type Keywords struct { Super int32 } -// wvPackKeywords appends v to w in the value-buffer wire format. -func wvPackKeywords(w *wvWriter, v Keywords) { +func wvWriteKeywords(w *wvWriter, v Keywords) { w.writeI32(v.Class) w.writeI32(v.Def) w.writeI32(v.Import) @@ -423,8 +387,7 @@ func wvPackKeywords(w *wvWriter, v Keywords) { w.writeI32(v.Super) } -// wvUnpackKeywords decodes one Keywords from r. -func wvUnpackKeywords(r *wvReader) Keywords { +func wvReadKeywords(r *wvReader) Keywords { var v Keywords v.Class = r.readI32() v.Def = r.readI32() @@ -489,19 +452,19 @@ func wvUnpackKeywords(r *wvReader) Keywords { } // Retired: Deprecated struct +// +// Deprecated: Use Outer instead type Retired struct { - Id int64 + ID int64 } -// wvPackRetired appends v to w in the value-buffer wire format. -func wvPackRetired(w *wvWriter, v Retired) { - w.writeI64(v.Id) +func wvWriteRetired(w *wvWriter, v Retired) { + w.writeI64(v.ID) } -// wvUnpackRetired decodes one Retired from r. -func wvUnpackRetired(r *wvReader) Retired { +func wvReadRetired(r *wvReader) Retired { var v Retired - v.Id = r.readI64() + v.ID = r.readI64() return v } @@ -513,62 +476,29 @@ type WidgetHolder struct { ByName map[string]*Widget } -// wvPackWidgetHolder appends v to w in the value-buffer wire format. -func wvPackWidgetHolder(w *wvWriter, v WidgetHolder) { - w.writeU64(wvTokenWidget(v.Primary)) - if v.Fallback == nil { - w.writeOptionFlag(false) - } else { - w.writeOptionFlag(true) - w.writeU64(wvTokenWidget(v.Fallback)) - } - w.writeLen(len(v.Extras)) - for _, eExtras0 := range v.Extras { - w.writeU64(wvTokenWidget(eExtras0)) - } - w.writeLen(len(v.ByName)) - for kByName0, vByName0 := range v.ByName { - w.writeString(kByName0) - w.writeU64(wvTokenWidget(vByName0)) - } +func wvWriteWidgetHolder(w *wvWriter, v WidgetHolder) { + wvWriteWidget(w, v.Primary) + wvWriteOptWidget(w, v.Fallback) + wvWriteListWidget(w, v.Extras) + wvWriteMapStringWidget(w, v.ByName) } -// wvUnpackWidgetHolder decodes one WidgetHolder from r. -func wvUnpackWidgetHolder(r *wvReader) WidgetHolder { +func wvReadWidgetHolder(r *wvReader) WidgetHolder { var v WidgetHolder - v.Primary = wvUntokenWidget(r.readU64()) - if r.readOptionFlag() { - var oFallback0 *Widget - oFallback0 = wvUntokenWidget(r.readU64()) - v.Fallback = oFallback0 - } - nExtras0 := r.readLen() - v.Extras = make([]*Widget, 0, r.capHint(nExtras0)) - for range nExtras0 { - var eExtras0 *Widget - eExtras0 = wvUntokenWidget(r.readU64()) - v.Extras = append(v.Extras, eExtras0) - } - nByName0 := r.readLen() - v.ByName = make(map[string]*Widget, r.capHint(nByName0)) - for range nByName0 { - var kByName0 string - kByName0 = r.readString() - var vByName0 *Widget - vByName0 = wvUntokenWidget(r.readU64()) - v.ByName[kByName0] = vByName0 - } + v.Primary = wvReadWidget(r) + v.Fallback = wvReadOptWidget(r) + v.Extras = wvReadListWidget(r) + v.ByName = wvReadMapStringWidget(r) return v } // BatchListener: Callback interface carrying composite payloads and objects // -// Implement this interface in Go and pass the value to native functions -// that accept it. -// -// The native library may call any method from any thread until it releases -// the implementation. A panic in a method is reported to the native caller -// as a foreign error (code -4) instead of crashing the process. +// Implement BatchListener in Go and pass the value to the functions that take +// it. The native library may call its methods from any thread until it +// releases the implementation. A method that panics, or a throwing method that +// returns an error outside its domain, fails the native call in progress as a +// callback failure (code -4). type BatchListener interface { // OnBatch: Composite payloads cross as value buffers OnBatch(items []Outer, raw []byte, first *Outer, mode Mode) @@ -583,107 +513,48 @@ type BatchListener interface { //export goWv_edge_cases_edge_cases_BatchListener_on_batch func goWv_edge_cases_edge_cases_BatchListener_on_batch(ctx unsafe.Pointer, items_ptr *C.uint8_t, items_len C.size_t, raw_ptr *C.uint8_t, raw_len C.size_t, first_ptr *C.uint8_t, first_len C.size_t, mode C.edge_cases_edge_cases_Mode, out_err *C.edge_cases_error) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(BatchListener) - rArg0 := &wvReader{buf: wvBorrowBytes(items_ptr, items_len)} - var arg0 []Outer - nArg00 := rArg0.readLen() - arg0 = make([]Outer, 0, rArg0.capHint(nArg00)) - for range nArg00 { - var eArg00 Outer - eArg00 = wvUnpackOuter(rArg0) - arg0 = append(arg0, eArg00) - } - rArg0.expectEnd() - arg1 := wvBorrowBytes(raw_ptr, raw_len) - rArg2 := &wvReader{buf: wvBorrowBytes(first_ptr, first_len)} - var arg2 *Outer - if rArg2.readOptionFlag() { - var oArg20 Outer - oArg20 = wvUnpackOuter(rArg2) - arg2 = &oArg20 - } - rArg2.expectEnd() - arg3 := Mode(mode) - impl.OnBatch(arg0, arg1, arg2, arg3) + defer wvRecoverCallback(out_err) + wvCallback[BatchListener](ctx).OnBatch(wvDecodeBorrowed(items_ptr, items_len, wvReadListOuter), wvBorrowBytes(raw_ptr, raw_len), wvDecodeBorrowed(first_ptr, first_len, wvReadOptOuter), Mode(mode)) } //export goWv_edge_cases_edge_cases_BatchListener_on_widget func goWv_edge_cases_edge_cases_BatchListener_on_widget(ctx unsafe.Pointer, widget *C.edge_cases_edge_cases_Widget, fallback *C.edge_cases_edge_cases_Widget, children_ptr *C.uint8_t, children_len C.size_t, out_err *C.edge_cases_error) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(BatchListener) - arg0 := wvAdoptWidget(widget) - arg1 := wvAdoptWidget(fallback) - rArg2 := &wvReader{buf: wvBorrowBytes(children_ptr, children_len)} - var arg2 []*Widget - nArg20 := rArg2.readLen() - arg2 = make([]*Widget, 0, rArg2.capHint(nArg20)) - for range nArg20 { - var eArg20 *Widget - eArg20 = wvUntokenWidget(rArg2.readU64()) - arg2 = append(arg2, eArg20) - } - rArg2.expectEnd() - impl.OnWidget(arg0, arg1, arg2) + defer wvRecoverCallback(out_err) + wvCallback[BatchListener](ctx).OnWidget(wvAdoptWidget(widget), wvAdoptWidget(fallback), wvDecodeBorrowed(children_ptr, children_len, wvReadListWidget)) } //export goWv_edge_cases_edge_cases_BatchListener_on_tick func goWv_edge_cases_edge_cases_BatchListener_on_tick(ctx unsafe.Pointer, out_err *C.edge_cases_error) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(BatchListener) - impl.OnTick() + defer wvRecoverCallback(out_err) + wvCallback[BatchListener](ctx).OnTick() } //export goWv_edge_cases_edge_cases_BatchListener_should_continue -func goWv_edge_cases_edge_cases_BatchListener_should_continue(ctx unsafe.Pointer, class C.int32_t, out_err *C.edge_cases_error) (ret C._Bool) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(BatchListener) - arg0 := int32(class) - ret = C._Bool(impl.ShouldContinue(arg0)) - return +func goWv_edge_cases_edge_cases_BatchListener_should_continue(ctx unsafe.Pointer, class C.int32_t, out_err *C.edge_cases_error) (cRet C._Bool) { + defer wvRecoverCallback(out_err) + ret := wvCallback[BatchListener](ctx).ShouldContinue(int32(class)) + return C._Bool(ret) } //export goWv_edge_cases_edge_cases_BatchListener_pick_mode -func goWv_edge_cases_edge_cases_BatchListener_pick_mode(ctx unsafe.Pointer, out_err *C.edge_cases_error) (ret C.edge_cases_edge_cases_Mode) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(BatchListener) - ret = C.edge_cases_edge_cases_Mode(impl.PickMode()) - return +func goWv_edge_cases_edge_cases_BatchListener_pick_mode(ctx unsafe.Pointer, out_err *C.edge_cases_error) (cRet C.edge_cases_edge_cases_Mode) { + defer wvRecoverCallback(out_err) + ret := wvCallback[BatchListener](ctx).PickMode() + return C.edge_cases_edge_cases_Mode(ret) } //export goWv_edge_cases_edge_cases_BatchListener_free func goWv_edge_cases_edge_cases_BatchListener_free(ctx unsafe.Pointer) { - cgo.Handle(uintptr(ctx)).Delete() + wvFreeCallback(ctx) } // Keyword: Callback interface whose name and methods are keywords // -// Implement this interface in Go and pass the value to native functions -// that accept it. -// -// The native library may call any method from any thread until it releases -// the implementation. A panic in a method is reported to the native caller -// as a foreign error (code -4) instead of crashing the process. +// Implement Keyword in Go and pass the value to the functions that take it. +// The native library may call its methods from any thread until it releases +// the implementation. A method that panics, or a throwing method that returns +// an error outside its domain, fails the native call in progress as a callback +// failure (code -4). type Keyword interface { Delete(self int32) Import() int32 @@ -691,37 +562,26 @@ type Keyword interface { //export goWv_edge_cases_edge_cases_Keyword_delete func goWv_edge_cases_edge_cases_Keyword_delete(ctx unsafe.Pointer, self C.int32_t, out_err *C.edge_cases_error) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(Keyword) - arg0 := int32(self) - impl.Delete(arg0) + defer wvRecoverCallback(out_err) + wvCallback[Keyword](ctx).Delete(int32(self)) } //export goWv_edge_cases_edge_cases_Keyword_import -func goWv_edge_cases_edge_cases_Keyword_import(ctx unsafe.Pointer, out_err *C.edge_cases_error) (ret C.int32_t) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(Keyword) - ret = C.int32_t(impl.Import()) - return +func goWv_edge_cases_edge_cases_Keyword_import(ctx unsafe.Pointer, out_err *C.edge_cases_error) (cRet C.int32_t) { + defer wvRecoverCallback(out_err) + ret := wvCallback[Keyword](ctx).Import() + return C.int32_t(ret) } //export goWv_edge_cases_edge_cases_Keyword_free func goWv_edge_cases_edge_cases_Keyword_free(ctx unsafe.Pointer) { - cgo.Handle(uintptr(ctx)).Delete() + wvFreeCallback(ctx) } // Widget: Interface with no constructor; instances come from make_widget // -// Each wrapper holds one strong reference; Close releases it, and a -// finalizer releases it if the wrapper is garbage collected first. +// Each wrapper holds one strong reference; Close releases it, and a finalizer +// releases it if the wrapper is garbage collected first. type Widget struct { ref wvRef } @@ -730,8 +590,7 @@ func wvDestroyWidget(ptr unsafe.Pointer) { C.edge_cases_edge_cases_Widget_destroy((*C.edge_cases_edge_cases_Widget)(ptr)) } -// wvAdoptWidget adopts one owned strong reference into a new wrapper. A null -// pointer adopts to nil. +// wvAdoptWidget wraps one owned strong reference. A null pointer adopts to nil. func wvAdoptWidget(ptr *C.edge_cases_edge_cases_Widget) *Widget { if ptr == nil { return nil @@ -751,19 +610,25 @@ func (s *Widget) native() *C.edge_cases_edge_cases_Widget { return (*C.edge_cases_edge_cases_Widget)(s.ref.acquire("Widget")) } -// wvTokenWidget clones o's reference into a value-buffer object token. The -// wrapper keeps its own reference; the token carries the new one. -func wvTokenWidget(o *Widget) uint64 { - ptr := o.native() - defer o.ref.release() - return uint64(uintptr(unsafe.Pointer(C.edge_cases_edge_cases_Widget_clone(ptr)))) +// share returns a new strong reference to the wrapper's object for the +// native library to adopt (the wrapper keeps its own), or null for nil. +func (s *Widget) share() *C.edge_cases_edge_cases_Widget { + if s == nil { + return nil + } + ptr := s.native() + defer s.ref.release() + return C.edge_cases_edge_cases_Widget_clone(ptr) +} + +func wvWriteWidget(w *wvWriter, v *Widget) { + w.writeU64(uint64(uintptr(unsafe.Pointer(v.share())))) } -// wvUntokenWidget adopts the strong reference carried by a value-buffer object -// token. -func wvUntokenWidget(token uint64) *Widget { +func wvReadWidget(r *wvReader) *Widget { + token := r.readU64() if token == 0 { - panic("edge_cases: malformed value buffer: null object token") + wvMalformed("null object token") } return wvAdoptWidget((*C.edge_cases_edge_cases_Widget)(C.wvHandlePtr(C.uintptr_t(token)))) } @@ -772,7 +637,7 @@ func wvUntokenWidget(token uint64) *Widget { // It's idempotent and safe to call from any goroutine, even while a call // on the wrapper is in flight: the reference is then released when that // call returns. The object itself is dropped once its last reference, from -// any wrapper, record, or the native side, is gone. +// any wrapper, record, or the native library, is gone. func (s *Widget) Close() error { runtime.SetFinalizer(s, nil) s.ref.close() @@ -786,7 +651,7 @@ func (s *Widget) Refresh() error { var cErr C.edge_cases_error C.edge_cases_edge_cases_Widget_refresh(cSelf, &cErr) if cErr.code != 0 { - return wvMapEdgeCases(wvTakeError(&cErr)) + return wvEdgeErrorsError(wvTakeError(&cErr)) } return nil } @@ -795,14 +660,7 @@ func (s *Widget) Rename(class string, import_ *string) string { cSelf := s.native() defer s.ref.release() cClassPtr, cClassLen := wvStr(class) - wImport := &wvWriter{} - if import_ == nil { - wImport.writeOptionFlag(false) - } else { - wImport.writeOptionFlag(true) - wImport.writeString(*import_) - } - cImportPtr, cImportLen := wvBytes(wImport.buf) + cImportPtr, cImportLen := wvBytes(wvEncode(import_, wvWriteOptString)) var cRetLen C.size_t var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_Widget_rename(cSelf, cClassPtr, cClassLen, cImportPtr, cImportLen, &cRetLen, &cErr) @@ -811,13 +669,9 @@ func (s *Widget) Rename(class string, import_ *string) string { } //export goWv_edge_cases_edge_cases_Widget_fetch_callback -func goWv_edge_cases_edge_cases_Widget_fetch_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() Outer { - rRet := &wvReader{buf: wvTakeBytes(result_ptr, result_len)} - var ret Outer - ret = wvUnpackOuter(rRet) - rRet.expectEnd() - return ret +func goWv_edge_cases_edge_cases_Widget_fetch_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() Outer { + return wvDecode(result_ptr, result_len, wvReadOuter) }) } @@ -831,15 +685,14 @@ func (s *Widget) Fetch(ctx context.Context, id int64) (Outer, error) { } cSelf := s.native() defer s.ref.release() - wvDone := make(chan wvOutcome[Outer], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_Widget_fetch(cSelf, C.int64_t(id), C.edge_cases_edge_cases_Widget_fetch_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_Widget_fetch_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[Outer]() + C.edge_cases_edge_cases_Widget_fetch(cSelf, C.int64_t(id), C.edge_cases_edge_cases_Widget_fetch_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_Widget_fetch_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return Outer{}, err } if fail != nil { - return Outer{}, wvMapEdgeCases(*fail) + return Outer{}, wvEdgeErrorsError(*fail) } return res, nil } @@ -855,10 +708,29 @@ func (s *Widget) OldName() string { return wvTakeString(cRet, cRetLen) } +// Close_: Spelled like the release method of several wrappers +func (s *Widget) Close_() bool { + cSelf := s.native() + defer s.ref.release() + var cErr C.edge_cases_error + cRet := C.edge_cases_edge_cases_Widget_close(cSelf, &cErr) + wvTrap(&cErr) + return bool(cRet) +} + +// Dispose: Spelled like the release method of the Dart and .NET wrappers +func (s *Widget) Dispose(force bool) { + cSelf := s.native() + defer s.ref.release() + var cErr C.edge_cases_error + C.edge_cases_edge_cases_Widget_dispose(cSelf, C._Bool(force), &cErr) + wvTrap(&cErr) +} + // Vault: Deprecated interface with a throwing constructor // -// Each wrapper holds one strong reference; Close releases it, and a -// finalizer releases it if the wrapper is garbage collected first. +// Each wrapper holds one strong reference; Close releases it, and a finalizer +// releases it if the wrapper is garbage collected first. // // Deprecated: Use Widget instead type Vault struct { @@ -869,8 +741,7 @@ func wvDestroyVault(ptr unsafe.Pointer) { C.edge_cases_edge_cases_Vault_destroy((*C.edge_cases_edge_cases_Vault)(ptr)) } -// wvAdoptVault adopts one owned strong reference into a new wrapper. A null -// pointer adopts to nil. +// wvAdoptVault wraps one owned strong reference. A null pointer adopts to nil. func wvAdoptVault(ptr *C.edge_cases_edge_cases_Vault) *Vault { if ptr == nil { return nil @@ -890,28 +761,22 @@ func (s *Vault) native() *C.edge_cases_edge_cases_Vault { return (*C.edge_cases_edge_cases_Vault)(s.ref.acquire("Vault")) } -// wvTokenVault clones o's reference into a value-buffer object token. The -// wrapper keeps its own reference; the token carries the new one. -func wvTokenVault(o *Vault) uint64 { - ptr := o.native() - defer o.ref.release() - return uint64(uintptr(unsafe.Pointer(C.edge_cases_edge_cases_Vault_clone(ptr)))) -} - -// wvUntokenVault adopts the strong reference carried by a value-buffer object -// token. -func wvUntokenVault(token uint64) *Vault { - if token == 0 { - panic("edge_cases: malformed value buffer: null object token") +// share returns a new strong reference to the wrapper's object for the +// native library to adopt (the wrapper keeps its own), or null for nil. +func (s *Vault) share() *C.edge_cases_edge_cases_Vault { + if s == nil { + return nil } - return wvAdoptVault((*C.edge_cases_edge_cases_Vault)(C.wvHandlePtr(C.uintptr_t(token)))) + ptr := s.native() + defer s.ref.release() + return C.edge_cases_edge_cases_Vault_clone(ptr) } // Close releases the wrapper's strong reference and always returns nil. // It's idempotent and safe to call from any goroutine, even while a call // on the wrapper is in flight: the reference is then released when that // call returns. The object itself is dropped once its last reference, from -// any wrapper, record, or the native side, is gone. +// any wrapper, record, or the native library, is gone. func (s *Vault) Close() error { runtime.SetFinalizer(s, nil) s.ref.close() @@ -924,7 +789,7 @@ func OpenVault(path string) (*Vault, error) { var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_Vault_open(cPathPtr, cPathLen, &cErr) if cErr.code != 0 { - return nil, wvMapEdgeCases(wvTakeError(&cErr)) + return nil, wvEdgeErrorsError(wvTakeError(&cErr)) } return wvAdoptVault(cRet), nil } @@ -938,16 +803,35 @@ func (s *Vault) Peek() *Widget { return wvAdoptWidget(cRet) } +// Handle: Spelled like the handle accessor of several wrappers +func (s *Vault) Handle() int64 { + cSelf := s.native() + defer s.ref.release() + var cErr C.edge_cases_error + cRet := C.edge_cases_edge_cases_Vault_handle(cSelf, &cErr) + wvTrap(&cErr) + return int64(cRet) +} + // VaultProbe: Throwing static func VaultProbe() (bool, error) { var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_Vault_probe(&cErr) if cErr.code != 0 { - return false, wvMapEdgeCases(wvTakeError(&cErr)) + return false, wvEdgeErrorsError(wvTakeError(&cErr)) } return bool(cRet), nil } +// VaultToString: Static spelled like an inherited object member +func VaultToString() string { + var cRetLen C.size_t + var cErr C.edge_cases_error + cRet := C.edge_cases_edge_cases_Vault_to_string(&cRetLen, &cErr) + wvTrap(&cErr) + return wvTakeString(cRet, cRetLen) +} + // Noop: No parameters and no result func Noop() { var cErr C.edge_cases_error @@ -958,20 +842,8 @@ func Noop() { // Select: Function named after a keyword whose parameters are all keywords func Select(class int32, from string, func_ []string, range_ *int32, type_ bool) int32 { cFromPtr, cFromLen := wvStr(from) - wFunc := &wvWriter{} - wFunc.writeLen(len(func_)) - for _, eFunc0 := range func_ { - wFunc.writeString(eFunc0) - } - cFuncPtr, cFuncLen := wvBytes(wFunc.buf) - wRange := &wvWriter{} - if range_ == nil { - wRange.writeOptionFlag(false) - } else { - wRange.writeOptionFlag(true) - wRange.writeI32(*range_) - } - cRangePtr, cRangeLen := wvBytes(wRange.buf) + cFuncPtr, cFuncLen := wvBytes(wvEncode(func_, wvWriteListString)) + cRangePtr, cRangeLen := wvBytes(wvEncode(range_, wvWriteOptI32)) var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_select(C.int32_t(class), cFromPtr, cFromLen, cFuncPtr, cFuncLen, cRangePtr, cRangeLen, C._Bool(type_), &cErr) wvTrap(&cErr) @@ -980,167 +852,53 @@ func Select(class int32, from string, func_ []string, range_ *int32, type_ bool) // NestedOptional: Optional list of optional structs, the deepest legal nesting func NestedOptional(items []*Outer) []*Outer { - wItems := &wvWriter{} - if items == nil { - wItems.writeOptionFlag(false) - } else { - wItems.writeOptionFlag(true) - wItems.writeLen(len(items)) - for _, eItems1 := range items { - if eItems1 == nil { - wItems.writeOptionFlag(false) - } else { - wItems.writeOptionFlag(true) - wvPackOuter(wItems, *eItems1) - } - } - } - cItemsPtr, cItemsLen := wvBytes(wItems.buf) + cItemsPtr, cItemsLen := wvBytes(wvEncode(items, wvWriteOptListOptOuter)) var cRetLen C.size_t var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_nested_optional(cItemsPtr, cItemsLen, &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret []*Outer - if rRet.readOptionFlag() { - var oRet0 []*Outer - nRet1 := rRet.readLen() - oRet0 = make([]*Outer, 0, rRet.capHint(nRet1)) - for range nRet1 { - var eRet1 *Outer - if rRet.readOptionFlag() { - var oRet2 Outer - oRet2 = wvUnpackOuter(rRet) - eRet1 = &oRet2 - } - oRet0 = append(oRet0, eRet1) - } - ret = oRet0 - } - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadOptListOptOuter) } func OptionalParams(text *string, count *int32, outer *Outer, tags []string, blob []byte, mode *Mode) *Mode { - wText := &wvWriter{} - if text == nil { - wText.writeOptionFlag(false) - } else { - wText.writeOptionFlag(true) - wText.writeString(*text) - } - cTextPtr, cTextLen := wvBytes(wText.buf) - wCount := &wvWriter{} - if count == nil { - wCount.writeOptionFlag(false) - } else { - wCount.writeOptionFlag(true) - wCount.writeI32(*count) - } - cCountPtr, cCountLen := wvBytes(wCount.buf) - wOuter := &wvWriter{} - if outer == nil { - wOuter.writeOptionFlag(false) - } else { - wOuter.writeOptionFlag(true) - wvPackOuter(wOuter, *outer) - } - cOuterPtr, cOuterLen := wvBytes(wOuter.buf) - wTags := &wvWriter{} - if tags == nil { - wTags.writeOptionFlag(false) - } else { - wTags.writeOptionFlag(true) - wTags.writeLen(len(tags)) - for _, eTags1 := range tags { - wTags.writeString(eTags1) - } - } - cTagsPtr, cTagsLen := wvBytes(wTags.buf) - wBlob := &wvWriter{} - if blob == nil { - wBlob.writeOptionFlag(false) - } else { - wBlob.writeOptionFlag(true) - wBlob.writeBytes(blob) - } - cBlobPtr, cBlobLen := wvBytes(wBlob.buf) - wMode := &wvWriter{} - if mode == nil { - wMode.writeOptionFlag(false) - } else { - wMode.writeOptionFlag(true) - wMode.writeI32(int32(*mode)) - } - cModePtr, cModeLen := wvBytes(wMode.buf) + cTextPtr, cTextLen := wvBytes(wvEncode(text, wvWriteOptString)) + cCountPtr, cCountLen := wvBytes(wvEncode(count, wvWriteOptI32)) + cOuterPtr, cOuterLen := wvBytes(wvEncode(outer, wvWriteOptOuter)) + cTagsPtr, cTagsLen := wvBytes(wvEncode(tags, wvWriteOptListString)) + cBlobPtr, cBlobLen := wvBytes(wvEncode(blob, wvWriteOptBytes)) + cModePtr, cModeLen := wvBytes(wvEncode(mode, wvWriteOptMode)) var cRetLen C.size_t var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_optional_params(cTextPtr, cTextLen, cCountPtr, cCountLen, cOuterPtr, cOuterLen, cTagsPtr, cTagsLen, cBlobPtr, cBlobLen, cModePtr, cModeLen, &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret *Mode - if rRet.readOptionFlag() { - var oRet0 Mode - oRet0 = Mode(rRet.readI32()) - ret = &oRet0 - } - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadOptMode) } // RegisterBatch: Two callback interface parameters on one function func RegisterBatch(listener BatchListener, keyword Keyword) { - hListener := cgo.NewHandle(listener) - hKeyword := cgo.NewHandle(keyword) var cErr C.edge_cases_error - C.edge_cases_edge_cases_register_batch(C.wvHandlePtr(C.uintptr_t(hListener)), C.wvVtablePtr_edge_cases_edge_cases_BatchListener_vtable(), C.wvHandlePtr(C.uintptr_t(hKeyword)), C.wvVtablePtr_edge_cases_edge_cases_Keyword_vtable(), &cErr) + C.edge_cases_edge_cases_register_batch(wvNewCallback(listener), C.wvVtablePtr_edge_cases_edge_cases_BatchListener_vtable(), wvNewCallback(keyword), C.wvVtablePtr_edge_cases_edge_cases_Keyword_vtable(), &cErr) wvTrap(&cErr) } // WidgetsEverywhere: Objects in every buffered position func WidgetsEverywhere(list []*Widget, byName map[string]*Widget, sparse []*Widget, holder WidgetHolder) WidgetHolder { - wList := &wvWriter{} - wList.writeLen(len(list)) - for _, eList0 := range list { - wList.writeU64(wvTokenWidget(eList0)) - } - cListPtr, cListLen := wvBytes(wList.buf) - wByName := &wvWriter{} - wByName.writeLen(len(byName)) - for kByName0, vByName0 := range byName { - wByName.writeString(kByName0) - wByName.writeU64(wvTokenWidget(vByName0)) - } - cByNamePtr, cByNameLen := wvBytes(wByName.buf) - wSparse := &wvWriter{} - wSparse.writeLen(len(sparse)) - for _, eSparse0 := range sparse { - if eSparse0 == nil { - wSparse.writeOptionFlag(false) - } else { - wSparse.writeOptionFlag(true) - wSparse.writeU64(wvTokenWidget(eSparse0)) - } - } - cSparsePtr, cSparseLen := wvBytes(wSparse.buf) - wHolder := &wvWriter{} - wvPackWidgetHolder(wHolder, holder) - cHolderPtr, cHolderLen := wvBytes(wHolder.buf) + cListPtr, cListLen := wvBytes(wvEncode(list, wvWriteListWidget)) + cByNamePtr, cByNameLen := wvBytes(wvEncode(byName, wvWriteMapStringWidget)) + cSparsePtr, cSparseLen := wvBytes(wvEncode(sparse, wvWriteListOptWidget)) + cHolderPtr, cHolderLen := wvBytes(wvEncode(holder, wvWriteWidgetHolder)) var cRetLen C.size_t var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_widgets_everywhere(cListPtr, cListLen, cByNamePtr, cByNameLen, cSparsePtr, cSparseLen, cHolderPtr, cHolderLen, &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret WidgetHolder - ret = wvUnpackWidgetHolder(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadWidgetHolder) } // WidgetStream: Iterator of objects // -// The sequence launches the native iterator on each range and calls next once -// per element; the iterator is destroyed when the range ends, early or not. +// The sequence launches the native iterator each time it's ranged over and +// pulls one element per step; the iterator is released when the range ends, +// early or not. func WidgetStream() iter.Seq[*Widget] { return func(yield func(*Widget) bool) { var cErr C.edge_cases_error @@ -1155,8 +913,7 @@ func WidgetStream() iter.Seq[*Widget] { if !more { return } - item := wvAdoptWidget(cItem) - if !yield(item) { + if !yield(wvAdoptWidget(cItem)) { return } } @@ -1164,8 +921,8 @@ func WidgetStream() iter.Seq[*Widget] { } //export goWv_edge_cases_edge_cases_async_widget_callback -func goWv_edge_cases_edge_cases_async_widget_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result *C.edge_cases_edge_cases_Widget) { - wvComplete(context_, err, func() *Widget { +func goWv_edge_cases_edge_cases_async_widget_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result *C.edge_cases_edge_cases_Widget) { + wvComplete(context_, err_, func() *Widget { return wvAdoptWidget(result) }) } @@ -1178,15 +935,14 @@ func AsyncWidget(ctx context.Context) (*Widget, error) { if err := ctx.Err(); err != nil { return nil, err } - wvDone := make(chan wvOutcome[*Widget], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_widget(C.edge_cases_edge_cases_async_widget_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_widget_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[*Widget]() + C.edge_cases_edge_cases_async_widget(C.edge_cases_edge_cases_async_widget_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_widget_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return nil, err } if fail != nil { - return nil, wvMapEdgeCases(*fail) + return nil, wvEdgeErrorsError(*fail) } return res, nil } @@ -1205,7 +961,7 @@ func FindWidget(id int64) (*Widget, error) { var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_find_widget(C.int64_t(id), &cErr) if cErr.code != 0 { - return nil, wvMapEdgeCases(wvTakeError(&cErr)) + return nil, wvEdgeErrorsError(wvTakeError(&cErr)) } return wvAdoptWidget(cRet), nil } @@ -1228,8 +984,9 @@ func InspectWidget(widget *Widget, fallback *Widget) string { // StringStream: Iterator of strings // -// The sequence launches the native iterator on each range and calls next once -// per element; the iterator is destroyed when the range ends, early or not. +// The sequence launches the native iterator each time it's ranged over and +// pulls one element per step; the iterator is released when the range ends, +// early or not. func StringStream() iter.Seq[string] { return func(yield func(string) bool) { var cErr C.edge_cases_error @@ -1245,8 +1002,7 @@ func StringStream() iter.Seq[string] { if !more { return } - item := wvTakeString(cItem, cItemLen) - if !yield(item) { + if !yield(wvTakeString(cItem, cItemLen)) { return } } @@ -1255,8 +1011,9 @@ func StringStream() iter.Seq[string] { // ScalarStream: Iterator of scalars // -// The sequence launches the native iterator on each range and calls next once -// per element; the iterator is destroyed when the range ends, early or not. +// The sequence launches the native iterator each time it's ranged over and +// pulls one element per step; the iterator is released when the range ends, +// early or not. func ScalarStream() iter.Seq[int32] { return func(yield func(int32) bool) { var cErr C.edge_cases_error @@ -1271,8 +1028,7 @@ func ScalarStream() iter.Seq[int32] { if !more { return } - item := int32(cItem) - if !yield(item) { + if !yield(int32(cItem)) { return } } @@ -1281,8 +1037,9 @@ func ScalarStream() iter.Seq[int32] { // ModeStream: Iterator of enums // -// The sequence launches the native iterator on each range and calls next once -// per element; the iterator is destroyed when the range ends, early or not. +// The sequence launches the native iterator each time it's ranged over and +// pulls one element per step; the iterator is released when the range ends, +// early or not. func ModeStream() iter.Seq[Mode] { return func(yield func(Mode) bool) { var cErr C.edge_cases_error @@ -1297,8 +1054,7 @@ func ModeStream() iter.Seq[Mode] { if !more { return } - item := Mode(cItem) - if !yield(item) { + if !yield(Mode(cItem)) { return } } @@ -1306,8 +1062,8 @@ func ModeStream() iter.Seq[Mode] { } //export goWv_edge_cases_edge_cases_async_void_callback -func goWv_edge_cases_edge_cases_async_void_callback(context_ unsafe.Pointer, err *C.edge_cases_error) { - wvComplete(context_, err, func() struct{} { +func goWv_edge_cases_edge_cases_async_void_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error) { + wvComplete(context_, err_, func() struct{} { return struct{}{} }) } @@ -1320,10 +1076,9 @@ func AsyncVoid(ctx context.Context) error { if err := ctx.Err(); err != nil { return err } - wvDone := make(chan wvOutcome[struct{}], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_void(C.edge_cases_edge_cases_async_void_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_void_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - _, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[struct{}]() + C.edge_cases_edge_cases_async_void(C.edge_cases_edge_cases_async_void_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_void_callback)), call.context()) + _, fail, err := call.wait(ctx, nil) if err != nil { return err } @@ -1334,8 +1089,8 @@ func AsyncVoid(ctx context.Context) error { } //export goWv_edge_cases_edge_cases_async_bytes_callback -func goWv_edge_cases_edge_cases_async_bytes_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() []byte { +func goWv_edge_cases_edge_cases_async_bytes_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() []byte { return wvTakeBytes(result_ptr, result_len) }) } @@ -1348,10 +1103,9 @@ func AsyncBytes(ctx context.Context, size uint32) ([]byte, error) { if err := ctx.Err(); err != nil { return nil, err } - wvDone := make(chan wvOutcome[[]byte], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_bytes(C.uint32_t(size), C.edge_cases_edge_cases_async_bytes_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_bytes_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[[]byte]() + C.edge_cases_edge_cases_async_bytes(C.uint32_t(size), C.edge_cases_edge_cases_async_bytes_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_bytes_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return nil, err } @@ -1362,13 +1116,9 @@ func AsyncBytes(ctx context.Context, size uint32) ([]byte, error) { } //export goWv_edge_cases_edge_cases_async_struct_callback -func goWv_edge_cases_edge_cases_async_struct_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() Outer { - rRet := &wvReader{buf: wvTakeBytes(result_ptr, result_len)} - var ret Outer - ret = wvUnpackOuter(rRet) - rRet.expectEnd() - return ret +func goWv_edge_cases_edge_cases_async_struct_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() Outer { + return wvDecode(result_ptr, result_len, wvReadOuter) }) } @@ -1380,71 +1130,51 @@ func AsyncStruct(ctx context.Context) (Outer, error) { if err := ctx.Err(); err != nil { return Outer{}, err } - wvDone := make(chan wvOutcome[Outer], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_struct(C.edge_cases_edge_cases_async_struct_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_struct_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[Outer]() + C.edge_cases_edge_cases_async_struct(C.edge_cases_edge_cases_async_struct_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_struct_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return Outer{}, err } if fail != nil { - return Outer{}, wvMapEdgeCases(*fail) + return Outer{}, wvEdgeErrorsError(*fail) } return res, nil } //export goWv_edge_cases_edge_cases_async_list_callback -func goWv_edge_cases_edge_cases_async_list_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() []Outer { - rRet := &wvReader{buf: wvTakeBytes(result_ptr, result_len)} - var ret []Outer - nRet0 := rRet.readLen() - ret = make([]Outer, 0, rRet.capHint(nRet0)) - for range nRet0 { - var eRet0 Outer - eRet0 = wvUnpackOuter(rRet) - ret = append(ret, eRet0) - } - rRet.expectEnd() - return ret +func goWv_edge_cases_edge_cases_async_list_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() []Outer { + return wvDecode(result_ptr, result_len, wvReadListOuter) }) } // AsyncList calls the native async function. // -// It blocks until the call completes. Cancelling ctx cancels the native -// call, which then returns ctx.Err(). +// It blocks until the call completes. Cancelling ctx cancels the native call, +// which then returns ctx.Err(). func AsyncList(ctx context.Context) ([]Outer, error) { if err := ctx.Err(); err != nil { return nil, err } - wvToken := C.edge_cases_cancel_token_create() - defer C.edge_cases_cancel_token_destroy(wvToken) - wvDone := make(chan wvOutcome[[]Outer], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_list(wvToken, C.edge_cases_edge_cases_async_list_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_list_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, wvToken) + token := C.edge_cases_cancel_token_create() + defer C.edge_cases_cancel_token_destroy(token) + call := wvNewAsyncCall[[]Outer]() + C.edge_cases_edge_cases_async_list(token, C.edge_cases_edge_cases_async_list_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_list_callback)), call.context()) + res, fail, err := call.wait(ctx, token) if err != nil { return nil, err } if fail != nil { - return nil, wvMapEdgeCases(*fail) + return nil, wvEdgeErrorsError(*fail) } return res, nil } //export goWv_edge_cases_edge_cases_async_optional_callback -func goWv_edge_cases_edge_cases_async_optional_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() *int64 { - rRet := &wvReader{buf: wvTakeBytes(result_ptr, result_len)} - var ret *int64 - if rRet.readOptionFlag() { - var oRet0 int64 - oRet0 = rRet.readI64() - ret = &oRet0 - } - rRet.expectEnd() - return ret +func goWv_edge_cases_edge_cases_async_optional_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() *int64 { + return wvDecode(result_ptr, result_len, wvReadOptI64) }) } @@ -1456,10 +1186,9 @@ func AsyncOptional(ctx context.Context) (*int64, error) { if err := ctx.Err(); err != nil { return nil, err } - wvDone := make(chan wvOutcome[*int64], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_optional(C.edge_cases_edge_cases_async_optional_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_optional_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[*int64]() + C.edge_cases_edge_cases_async_optional(C.edge_cases_edge_cases_async_optional_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_optional_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return nil, err } @@ -1470,8 +1199,8 @@ func AsyncOptional(ctx context.Context) (*int64, error) { } //export goWv_edge_cases_edge_cases_async_enum_callback -func goWv_edge_cases_edge_cases_async_enum_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result C.edge_cases_edge_cases_Mode) { - wvComplete(context_, err, func() Mode { +func goWv_edge_cases_edge_cases_async_enum_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result C.edge_cases_edge_cases_Mode) { + wvComplete(context_, err_, func() Mode { return Mode(result) }) } @@ -1484,10 +1213,9 @@ func AsyncEnum(ctx context.Context) (Mode, error) { if err := ctx.Err(); err != nil { return 0, err } - wvDone := make(chan wvOutcome[Mode], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_enum(C.edge_cases_edge_cases_async_enum_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_enum_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[Mode]() + C.edge_cases_edge_cases_async_enum(C.edge_cases_edge_cases_async_enum_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_enum_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return 0, err } @@ -1498,8 +1226,8 @@ func AsyncEnum(ctx context.Context) (Mode, error) { } //export goWv_edge_cases_edge_cases_async_keywords_callback -func goWv_edge_cases_edge_cases_async_keywords_callback(context_ unsafe.Pointer, err *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() string { +func goWv_edge_cases_edge_cases_async_keywords_callback(context_ unsafe.Pointer, err_ *C.edge_cases_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() string { return wvTakeString(result_ptr, result_len) }) } @@ -1513,10 +1241,9 @@ func AsyncKeywords(ctx context.Context, class string, import_ int32) (string, er return "", err } cClassPtr, cClassLen := wvStr(class) - wvDone := make(chan wvOutcome[string], 1) - wvHandle := cgo.NewHandle(wvDone) - C.edge_cases_edge_cases_async_keywords(cClassPtr, cClassLen, C.int32_t(import_), C.edge_cases_edge_cases_async_keywords_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_keywords_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[string]() + C.edge_cases_edge_cases_async_keywords(cClassPtr, cClassLen, C.int32_t(import_), C.edge_cases_edge_cases_async_keywords_callback(unsafe.Pointer(C.goWv_edge_cases_edge_cases_async_keywords_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return "", err } @@ -1527,18 +1254,12 @@ func AsyncKeywords(ctx context.Context, class string, import_ int32) (string, er } func KeywordsRoundtrip(value Keywords) Keywords { - wValue := &wvWriter{} - wvPackKeywords(wValue, value) - cValuePtr, cValueLen := wvBytes(wValue.buf) + cValuePtr, cValueLen := wvBytes(wvEncode(value, wvWriteKeywords)) var cRetLen C.size_t var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_keywords_roundtrip(cValuePtr, cValueLen, &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Keywords - ret = wvUnpackKeywords(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadKeywords) } // Deprecated: Use Mode-based APIs @@ -1547,11 +1268,191 @@ func LegacyValue(value Legacy) Retired { var cErr C.edge_cases_error cRet := C.edge_cases_edge_cases_legacy_value(C.edge_cases_edge_cases_Legacy(value), &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Retired - ret = wvUnpackRetired(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadRetired) +} + +func wvWriteListI32(w *wvWriter, v []int32) { + wvWriteList(w, v, (*wvWriter).writeI32) +} + +func wvReadListI32(r *wvReader) []int32 { + return wvReadList(r, (*wvReader).readI32) +} + +func wvWriteListLeaf(w *wvWriter, v []Leaf) { + wvWriteList(w, v, wvWriteLeaf) +} + +func wvReadListLeaf(r *wvReader) []Leaf { + return wvReadList(r, wvReadLeaf) +} + +func wvWriteListListI32(w *wvWriter, v [][]int32) { + wvWriteList(w, v, wvWriteListI32) +} + +func wvReadListListI32(r *wvReader) [][]int32 { + return wvReadList(r, wvReadListI32) +} + +func wvWriteListMode(w *wvWriter, v []Mode) { + wvWriteList(w, v, wvWriteMode) +} + +func wvReadListMode(r *wvReader) []Mode { + return wvReadList(r, wvReadMode) +} + +func wvWriteListOptI64(w *wvWriter, v []*int64) { + wvWriteList(w, v, wvWriteOptI64) +} + +func wvReadListOptI64(r *wvReader) []*int64 { + return wvReadList(r, wvReadOptI64) +} + +func wvWriteListOptOuter(w *wvWriter, v []*Outer) { + wvWriteList(w, v, wvWriteOptOuter) +} + +func wvReadListOptOuter(r *wvReader) []*Outer { + return wvReadList(r, wvReadOptOuter) +} + +func wvWriteListOptWidget(w *wvWriter, v []*Widget) { + wvWriteList(w, v, wvWriteOptWidget) +} + +func wvReadListOptWidget(r *wvReader) []*Widget { + return wvReadList(r, wvReadOptWidget) +} + +func wvWriteListOuter(w *wvWriter, v []Outer) { + wvWriteList(w, v, wvWriteOuter) +} + +func wvReadListOuter(r *wvReader) []Outer { + return wvReadList(r, wvReadOuter) +} + +func wvWriteListString(w *wvWriter, v []string) { + wvWriteList(w, v, (*wvWriter).writeString) +} + +func wvReadListString(r *wvReader) []string { + return wvReadList(r, (*wvReader).readString) +} + +func wvWriteListWidget(w *wvWriter, v []*Widget) { + wvWriteList(w, v, wvWriteWidget) +} + +func wvReadListWidget(r *wvReader) []*Widget { + return wvReadList(r, wvReadWidget) +} + +func wvWriteMapI64String(w *wvWriter, v map[int64]string) { + wvWriteMap(w, v, (*wvWriter).writeI64, (*wvWriter).writeString) +} + +func wvReadMapI64String(r *wvReader) map[int64]string { + return wvReadMap(r, (*wvReader).readI64, (*wvReader).readString) +} + +func wvWriteMapStringInner(w *wvWriter, v map[string]Inner) { + wvWriteMap(w, v, (*wvWriter).writeString, wvWriteInner) +} + +func wvReadMapStringInner(r *wvReader) map[string]Inner { + return wvReadMap(r, (*wvReader).readString, wvReadInner) +} + +func wvWriteMapStringWidget(w *wvWriter, v map[string]*Widget) { + wvWriteMap(w, v, (*wvWriter).writeString, wvWriteWidget) +} + +func wvReadMapStringWidget(r *wvReader) map[string]*Widget { + return wvReadMap(r, (*wvReader).readString, wvReadWidget) +} + +func wvWriteOptBytes(w *wvWriter, v []byte) { + wvWriteNilable(w, v, v != nil, (*wvWriter).writeBytes) +} + +func wvReadOptBytes(r *wvReader) []byte { + return wvReadNilable(r, (*wvReader).readBytes) +} + +func wvWriteOptI32(w *wvWriter, v *int32) { + wvWriteOptional(w, v, (*wvWriter).writeI32) +} + +func wvReadOptI32(r *wvReader) *int32 { + return wvReadOptional(r, (*wvReader).readI32) +} + +func wvWriteOptI64(w *wvWriter, v *int64) { + wvWriteOptional(w, v, (*wvWriter).writeI64) +} + +func wvReadOptI64(r *wvReader) *int64 { + return wvReadOptional(r, (*wvReader).readI64) +} + +func wvWriteOptInner(w *wvWriter, v *Inner) { + wvWriteOptional(w, v, wvWriteInner) +} + +func wvReadOptInner(r *wvReader) *Inner { + return wvReadOptional(r, wvReadInner) +} + +func wvWriteOptListOptOuter(w *wvWriter, v []*Outer) { + wvWriteNilable(w, v, v != nil, wvWriteListOptOuter) +} + +func wvReadOptListOptOuter(r *wvReader) []*Outer { + return wvReadNilable(r, wvReadListOptOuter) +} + +func wvWriteOptListString(w *wvWriter, v []string) { + wvWriteNilable(w, v, v != nil, wvWriteListString) +} + +func wvReadOptListString(r *wvReader) []string { + return wvReadNilable(r, wvReadListString) +} + +func wvWriteOptMode(w *wvWriter, v *Mode) { + wvWriteOptional(w, v, wvWriteMode) +} + +func wvReadOptMode(r *wvReader) *Mode { + return wvReadOptional(r, wvReadMode) +} + +func wvWriteOptOuter(w *wvWriter, v *Outer) { + wvWriteOptional(w, v, wvWriteOuter) +} + +func wvReadOptOuter(r *wvReader) *Outer { + return wvReadOptional(r, wvReadOuter) +} + +func wvWriteOptString(w *wvWriter, v *string) { + wvWriteOptional(w, v, (*wvWriter).writeString) +} + +func wvReadOptString(r *wvReader) *string { + return wvReadOptional(r, (*wvReader).readString) +} + +func wvWriteOptWidget(w *wvWriter, v *Widget) { + wvWriteNilable(w, v, v != nil, wvWriteWidget) +} + +func wvReadOptWidget(r *wvReader) *Widget { + return wvReadNilable(r, wvReadWidget) } // END bindings.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__codec_go.snap b/crates/weaveffi-cli/tests/snapshots/go_edge_cases__codec_go.snap deleted file mode 100644 index 16ddb647..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__codec_go.snap +++ /dev/null @@ -1,205 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package edge_cases - -import ( - "encoding/binary" - "math" - "unicode/utf8" -) - -// wvWriter serializes values into the value-buffer format: little-endian, -// packed, u32 length prefixes. -type wvWriter struct { - buf []byte -} - -func (w *wvWriter) writeBool(v bool) { - if v { - w.buf = append(w.buf, 1) - } else { - w.buf = append(w.buf, 0) - } -} - -func (w *wvWriter) writeI8(v int8) { - w.buf = append(w.buf, byte(v)) -} - -func (w *wvWriter) writeU8(v uint8) { - w.buf = append(w.buf, v) -} - -func (w *wvWriter) writeI16(v int16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, uint16(v)) -} - -func (w *wvWriter) writeU16(v uint16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, v) -} - -func (w *wvWriter) writeI32(v int32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, uint32(v)) -} - -func (w *wvWriter) writeU32(v uint32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, v) -} - -func (w *wvWriter) writeI64(v int64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, uint64(v)) -} - -func (w *wvWriter) writeU64(v uint64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, v) -} - -func (w *wvWriter) writeF32(v float32) { - w.writeU32(math.Float32bits(v)) -} - -func (w *wvWriter) writeF64(v float64) { - w.writeU64(math.Float64bits(v)) -} - -func (w *wvWriter) writeLen(n int) { - if n < 0 || uint64(n) > math.MaxUint32 { - panic("edge_cases: value-buffer length exceeds u32 range") - } - w.writeU32(uint32(n)) -} - -func (w *wvWriter) writeString(v string) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeBytes(v []byte) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeOptionFlag(present bool) { - w.writeBool(present) -} - -// wvReader decodes values from the value-buffer format. A malformed buffer -// is a producer/consumer contract violation, so every read panics instead of -// returning a typed domain error. -type wvReader struct { - buf []byte - pos int -} - -func wvMalformed(context string) { - panic("edge_cases: malformed value buffer: " + context) -} - -func (r *wvReader) take(n int, context string) []byte { - if n < 0 || len(r.buf)-r.pos < n { - wvMalformed(context) - } - b := r.buf[r.pos : r.pos+n] - r.pos += n - return b -} - -func (r *wvReader) readBool() bool { - switch r.take(1, "bool")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("bool byte out of range") - return false -} - -func (r *wvReader) readI8() int8 { - return int8(r.take(1, "i8")[0]) -} - -func (r *wvReader) readU8() uint8 { - return r.take(1, "u8")[0] -} - -func (r *wvReader) readI16() int16 { - return int16(binary.LittleEndian.Uint16(r.take(2, "i16"))) -} - -func (r *wvReader) readU16() uint16 { - return binary.LittleEndian.Uint16(r.take(2, "u16")) -} - -func (r *wvReader) readI32() int32 { - return int32(binary.LittleEndian.Uint32(r.take(4, "i32"))) -} - -func (r *wvReader) readU32() uint32 { - return binary.LittleEndian.Uint32(r.take(4, "u32")) -} - -func (r *wvReader) readI64() int64 { - return int64(binary.LittleEndian.Uint64(r.take(8, "i64"))) -} - -func (r *wvReader) readU64() uint64 { - return binary.LittleEndian.Uint64(r.take(8, "u64")) -} - -func (r *wvReader) readF32() float32 { - return math.Float32frombits(r.readU32()) -} - -func (r *wvReader) readF64() float64 { - return math.Float64frombits(r.readU64()) -} - -// readLen reads a u32 element count. Elements may encode to zero bytes, so -// the count isn't checked against the remaining buffer; callers cap their -// preallocation with capHint instead. -func (r *wvReader) readLen() int { - return int(r.readU32()) -} - -// capHint bounds a preallocation for n decoded elements by the bytes left in -// the buffer, so a corrupt count can't force a huge allocation. -func (r *wvReader) capHint(n int) int { - return min(n, len(r.buf)-r.pos) -} - -func (r *wvReader) readString() string { - b := r.take(r.readLen(), "string bytes") - if !utf8.Valid(b) { - wvMalformed("string is not valid UTF-8") - } - return string(b) -} - -func (r *wvReader) readBytes() []byte { - return append([]byte{}, r.take(r.readLen(), "byte buffer")...) -} - -func (r *wvReader) readOptionFlag() bool { - switch r.take(1, "option flag")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("option flag byte out of range") - return false -} - -func (r *wvReader) expectEnd() { - if r.pos != len(r.buf) { - wvMalformed("trailing bytes after value") - } -} - -// END codec.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__edge_cases_h.snap b/crates/weaveffi-cli/tests/snapshots/go_edge_cases__edge_cases_h.snap deleted file mode 100644 index 94864086..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__edge_cases_h.snap +++ /dev/null @@ -1,245 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef EDGE_CASES_H -#define EDGE_CASES_H - -#include -#include -#include - -#ifndef EDGE_CASES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef EDGE_CASES_BUILD -# define EDGE_CASES_API __declspec(dllexport) -# else -# define EDGE_CASES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define EDGE_CASES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define EDGE_CASES_API __attribute__((visibility("default"))) -# else -# define EDGE_CASES_API -# endif -#endif - -#ifndef EDGE_CASES_DEPRECATED -# if defined(_MSC_VER) -# define EDGE_CASES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define EDGE_CASES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define EDGE_CASES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports edge_cases_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define EDGE_CASES_ABI_VERSION 3u -EDGE_CASES_API uint32_t edge_cases_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by edge_cases_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct edge_cases_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} edge_cases_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -EDGE_CASES_API void edge_cases_error_set(edge_cases_error* err, int32_t code, const char* message); -EDGE_CASES_API void edge_cases_error_clear(edge_cases_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - edge_cases_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -EDGE_CASES_API void edge_cases_error_free(edge_cases_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -EDGE_CASES_API void edge_cases_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct edge_cases_cancel_token edge_cases_cancel_token; -EDGE_CASES_API edge_cases_cancel_token* edge_cases_cancel_token_create(void); -EDGE_CASES_API void edge_cases_cancel_token_cancel(edge_cases_cancel_token* token); -EDGE_CASES_API bool edge_cases_cancel_token_is_cancelled(const edge_cases_cancel_token* token); -EDGE_CASES_API void edge_cases_cancel_token_destroy(edge_cases_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -EDGE_CASES_API uint8_t* edge_cases_alloc(uint32_t size); -EDGE_CASES_API void edge_cases_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -EDGE_CASES_API uint64_t edge_cases_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define EDGE_CASES_EDGE_CASES_CHECKSUM 0x1567f441073bd4aaull -EDGE_CASES_API uint64_t edge_cases_edge_cases_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with edge_cases_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * edge_cases_buffer.h declares a C struct and codec for each. - */ - -/** Deprecated enum with non-contiguous values and keyword-like variants */ -typedef enum { edge_cases_edge_cases_Legacy_None = 0, edge_cases_edge_cases_Legacy_Default = 5, edge_cases_edge_cases_Legacy_Class = 10 } edge_cases_edge_cases_Legacy; -typedef enum { edge_cases_edge_cases_Mode_Fast = 1, edge_cases_edge_cases_Mode_Safe = 2 } edge_cases_edge_cases_Mode; -/** Error codes reported by throwing functions in the `edge_cases` module tree. */ -typedef enum { edge_cases_edge_cases_EdgeErrors_Busy = 7 } edge_cases_edge_cases_EdgeErrors; -typedef struct edge_cases_edge_cases_Widget edge_cases_edge_cases_Widget; -typedef struct edge_cases_edge_cases_Vault edge_cases_edge_cases_Vault; -typedef struct edge_cases_edge_cases_WidgetStreamIterator edge_cases_edge_cases_WidgetStreamIterator; -typedef struct edge_cases_edge_cases_StringStreamIterator edge_cases_edge_cases_StringStreamIterator; -typedef struct edge_cases_edge_cases_ScalarStreamIterator edge_cases_edge_cases_ScalarStreamIterator; -typedef struct edge_cases_edge_cases_ModeStreamIterator edge_cases_edge_cases_ModeStreamIterator; -/** - * Callback interface carrying composite payloads and objects - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_BatchListener_vtable { - /** Composite payloads cross as value buffers */ - void (*on_batch)(void* ctx, const uint8_t* items_ptr, size_t items_len, const uint8_t* raw_ptr, size_t raw_len, const uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err); - /** Objects cross as strong references the consumer must release */ - void (*on_widget)(void* ctx, edge_cases_edge_cases_Widget* widget, edge_cases_edge_cases_Widget* fallback, const uint8_t* children_ptr, size_t children_len, edge_cases_error* out_err); - void (*on_tick)(void* ctx, edge_cases_error* out_err); - /** Scalar return travels back through the vtable */ - bool (*should_continue)(void* ctx, int32_t class_, edge_cases_error* out_err); - /** Enum return travels back as its integer discriminant */ - edge_cases_edge_cases_Mode (*pick_mode)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_BatchListener_vtable; -/** - * Callback interface whose name and methods are keywords - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_Keyword_vtable { - void (*delete_)(void* ctx, int32_t self, edge_cases_error* out_err); - int32_t (*import)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_Keyword_vtable; -typedef void (*edge_cases_edge_cases_async_widget_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result); -typedef void (*edge_cases_edge_cases_async_void_callback)(void* context, edge_cases_error* err); -typedef void (*edge_cases_edge_cases_async_bytes_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_struct_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_list_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_optional_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_enum_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Mode result); -typedef void (*edge_cases_edge_cases_async_keywords_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_Widget_fetch_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: edge_cases -/** Interface with no constructor; instances come from make_widget */ -/** Void method that can fail */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_refresh(const edge_cases_edge_cases_Widget* self, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_rename(const edge_cases_edge_cases_Widget* self, const uint8_t* class_ptr, size_t class_len, const uint8_t* import_ptr, size_t import_len, size_t* out_len, edge_cases_error* out_err); -/** Async method with a struct result */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_fetch(const edge_cases_edge_cases_Widget* self, int64_t id, edge_cases_edge_cases_Widget_fetch_callback callback, void* context); -EDGE_CASES_DEPRECATED("Use rename instead") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_old_name(const edge_cases_edge_cases_Widget* self, size_t* out_len, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Widget_clone(const edge_cases_edge_cases_Widget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_destroy(edge_cases_edge_cases_Widget* self); - -/** Deprecated interface with a throwing constructor */ -/** Fails when the vault is busy */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_open(const uint8_t* path_ptr, size_t path_len, edge_cases_error* out_err); -/** Throwing static */ -EDGE_CASES_API bool edge_cases_edge_cases_Vault_probe(edge_cases_error* out_err); -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Vault_peek(const edge_cases_edge_cases_Vault* self, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_clone(const edge_cases_edge_cases_Vault* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Vault_destroy(edge_cases_edge_cases_Vault* self); - -/** No parameters and no result */ -EDGE_CASES_API void edge_cases_edge_cases_noop(edge_cases_error* out_err); -/** Function named after a keyword whose parameters are all keywords */ -EDGE_CASES_API int32_t edge_cases_edge_cases_select(int32_t class_, const uint8_t* from_ptr, size_t from_len, const uint8_t* func_ptr, size_t func_len, const uint8_t* range_ptr, size_t range_len, bool type_, edge_cases_error* out_err); -/** Optional list of optional structs, the deepest legal nesting */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_nested_optional(const uint8_t* items_ptr, size_t items_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_optional_params(const uint8_t* text_ptr, size_t text_len, const uint8_t* count_ptr, size_t count_len, const uint8_t* outer_ptr, size_t outer_len, const uint8_t* tags_ptr, size_t tags_len, const uint8_t* blob_ptr, size_t blob_len, const uint8_t* mode_ptr, size_t mode_len, size_t* out_len, edge_cases_error* out_err); -/** Two callback interface parameters on one function */ -EDGE_CASES_API void edge_cases_edge_cases_register_batch(void* listener_ctx, const edge_cases_edge_cases_BatchListener_vtable* listener_vtable, void* keyword_ctx, const edge_cases_edge_cases_Keyword_vtable* keyword_vtable, edge_cases_error* out_err); -/** Objects in every buffered position */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_widgets_everywhere(const uint8_t* list_ptr, size_t list_len, const uint8_t* by_name_ptr, size_t by_name_len, const uint8_t* sparse_ptr, size_t sparse_len, const uint8_t* holder_ptr, size_t holder_len, size_t* out_len, edge_cases_error* out_err); -/** Iterator of objects */ -EDGE_CASES_API edge_cases_edge_cases_WidgetStreamIterator* edge_cases_edge_cases_widget_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_WidgetStreamIterator_next(edge_cases_edge_cases_WidgetStreamIterator* iter, edge_cases_edge_cases_Widget** out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_WidgetStreamIterator_destroy(edge_cases_edge_cases_WidgetStreamIterator* iter); -/** Async returning an object */ -EDGE_CASES_API void edge_cases_edge_cases_async_widget(edge_cases_edge_cases_async_widget_callback callback, void* context); -/** Factory for an interface with no constructor */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_make_widget(const uint8_t* label_ptr, size_t label_len, edge_cases_error* out_err); -/** Optional interface return */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_find_widget(int64_t id, edge_cases_error* out_err); -/** Interface parameter, including an optional one */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_inspect_widget(const edge_cases_edge_cases_Widget* widget, const edge_cases_edge_cases_Widget* fallback, size_t* out_len, edge_cases_error* out_err); -/** Iterator of strings */ -EDGE_CASES_API edge_cases_edge_cases_StringStreamIterator* edge_cases_edge_cases_string_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_StringStreamIterator_next(edge_cases_edge_cases_StringStreamIterator* iter, const uint8_t** out_item, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_StringStreamIterator_destroy(edge_cases_edge_cases_StringStreamIterator* iter); -/** Iterator of scalars */ -EDGE_CASES_API edge_cases_edge_cases_ScalarStreamIterator* edge_cases_edge_cases_scalar_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ScalarStreamIterator_next(edge_cases_edge_cases_ScalarStreamIterator* iter, int32_t* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ScalarStreamIterator_destroy(edge_cases_edge_cases_ScalarStreamIterator* iter); -/** Iterator of enums */ -EDGE_CASES_API edge_cases_edge_cases_ModeStreamIterator* edge_cases_edge_cases_mode_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ModeStreamIterator_next(edge_cases_edge_cases_ModeStreamIterator* iter, edge_cases_edge_cases_Mode* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ModeStreamIterator_destroy(edge_cases_edge_cases_ModeStreamIterator* iter); -/** Async with no result */ -EDGE_CASES_API void edge_cases_edge_cases_async_void(edge_cases_edge_cases_async_void_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_bytes(uint32_t size, edge_cases_edge_cases_async_bytes_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_struct(edge_cases_edge_cases_async_struct_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_list(edge_cases_cancel_token* cancel_token, edge_cases_edge_cases_async_list_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_optional(edge_cases_edge_cases_async_optional_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_enum(edge_cases_edge_cases_async_enum_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_keywords(const uint8_t* class_ptr, size_t class_len, int32_t import, edge_cases_edge_cases_async_keywords_callback callback, void* context); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_keywords_roundtrip(const uint8_t* value_ptr, size_t value_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_DEPRECATED("Use Mode-based APIs") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_legacy_value(edge_cases_edge_cases_Legacy value, size_t* out_len, edge_cases_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // EDGE_CASES_H - -// END edge_cases.h diff --git a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__go_mod.snap b/crates/weaveffi-cli/tests/snapshots/go_edge_cases__go_mod.snap deleted file mode 100644 index 66173700..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__go_mod.snap +++ /dev/null @@ -1,12 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module edge_cases - -go 1.23 - -// END go.mod diff --git a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__runtime_go.snap b/crates/weaveffi-cli/tests/snapshots/go_edge_cases__runtime_go.snap deleted file mode 100644 index ab9dfdca..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_edge_cases__runtime_go.snap +++ /dev/null @@ -1,305 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package edge_cases - -/* -#include -#include "edge_cases.h" -*/ -import "C" - -import ( - "context" - "fmt" - "runtime/cgo" - "sync/atomic" - "unsafe" -) - -// wvABIVersion is the C ABI revision these bindings were generated against. -const wvABIVersion uint32 = 3 - -// The runtime error codes this package produces or interprets itself. -const ( - wvCodeMarshal int32 = -3 - wvCodeForeign int32 = -4 - wvCodeCancelled int32 = -5 -) - -// Error reports a failure that no typed error domain claims: an unknown -// code, a generic failure (code -1), a producer panic (-2), a marshalling -// failure (-3), or a Go callback-interface implementation that panicked -// (-4). The code is preserved so callers can tell them apart. -type Error struct { - // Code is the numeric ABI error code. - Code int32 - // Message is the human-readable error message. - Message string -} - -func (e *Error) Error() string { - return fmt.Sprintf("edge_cases: %s (code %d)", e.Message, e.Code) -} - -// DebugLive reports how many resources of one kind the native library holds: -// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned -// allocations. It's always 0 unless the library was built with leak checks -// enabled; tests use it to assert that nothing leaked. -func DebugLive(kind int32) uint64 { - return uint64(C.edge_cases_debug_live(C.int32_t(kind))) -} - -// wvCheckABI panics when the linked library implements a different C ABI -// revision than these bindings were generated for. -func wvCheckABI() { - if found := uint32(C.edge_cases_abi_version()); found != wvABIVersion { - panic(fmt.Sprintf("edge_cases: ABI mismatch: these bindings expect revision %d but the linked library reports revision %d", wvABIVersion, found)) - } -} - -// wvCheckModule panics when the linked library's contract checksum for a -// top-level module differs from the one these bindings were generated -// against, which means the bindings are stale. -func wvCheckModule(module string, want, found uint64) { - if found != want { - panic(fmt.Sprintf("edge_cases: module %q doesn't match the linked library (bindings checksum %#016x, library checksum %#016x); regenerate the bindings", module, want, found)) - } -} - -// ── Errors ── - -// wvFailure is a non-zero error slot copied into Go memory. -type wvFailure struct { - code int32 - message string - payload []byte -} - -// err converts the failure into the generic *Error. -func (f wvFailure) err() error { - return &Error{Code: f.code, Message: f.message} -} - -func wvCopyFailure(cErr *C.edge_cases_error) wvFailure { - f := wvFailure{code: int32(cErr.code)} - if cErr.message != nil { - f.message = C.GoString(cErr.message) - } - if cErr.payload_ptr != nil { - f.payload = wvBorrowBytes(cErr.payload_ptr, cErr.payload_len) - } - return f -} - -// wvTakeError copies a non-zero error slot and clears it. -func wvTakeError(cErr *C.edge_cases_error) wvFailure { - f := wvCopyFailure(cErr) - C.edge_cases_error_clear(cErr) - return f -} - -// wvTakeBoxedError copies the heap-boxed error an async completion receives -// and frees the box. -func wvTakeBoxedError(cErr *C.edge_cases_error) wvFailure { - f := wvCopyFailure(cErr) - C.edge_cases_error_free(cErr) - return f -} - -// wvTrap panics with the generic *Error when the slot reports a failure. -// Wrappers of functions that declare no error domain check their slot with -// it: a failure there can only be a producer bug, a marshalling failure, or -// a callback-interface implementation that panicked. -func wvTrap(cErr *C.edge_cases_error) { - if cErr.code != 0 { - panic(wvTakeError(cErr).err()) - } -} - -// wvForeignError reports a panic recovered inside a callback-interface -// trampoline through the producer's error slot (code -4). The producer -// copies the message, so it's only borrowed for the call. -func wvForeignError(outErr *C.edge_cases_error, recovered any) { - msg := C.CString(fmt.Sprint(recovered)) - defer C.free(unsafe.Pointer(msg)) - C.edge_cases_error_set(outErr, C.int32_t(wvCodeForeign), msg) -} - -// ── Strings and bytes ── - -// wvStr returns a borrowed (pointer, length) view of s's UTF-8 bytes for the -// duration of one call. Strings may contain NUL bytes. -func wvStr(s string) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.StringData(s))), C.size_t(len(s)) -} - -// wvBytes returns a borrowed (pointer, length) view of b for the duration of -// one call. -func wvBytes(b []byte) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.SliceData(b))), C.size_t(len(b)) -} - -// wvBorrowString copies a borrowed UTF-8 run into a Go string. -func wvBorrowString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - return string(unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) -} - -// wvTakeString copies a returned UTF-8 run into a Go string and releases the -// producer's allocation. -func wvTakeString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - s := wvBorrowString(ptr, n) - C.edge_cases_free_bytes(ptr, n) - return s -} - -// wvBorrowBytes copies a borrowed byte run into Go memory. A null pointer -// copies to nil. -func wvBorrowBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - return append([]byte{}, unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)...) -} - -// wvTakeBytes copies a returned byte run (raw bytes or a value buffer) into -// Go memory and releases the producer's allocation. -func wvTakeBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - b := wvBorrowBytes(ptr, n) - C.edge_cases_free_bytes(ptr, n) - return b -} - -// ── Objects ── - -// wvRef owns one strong reference to a native object on behalf of a wrapper. -// Calls borrow the pointer between acquire and release. Close may race them: -// the reference is destroyed exactly once, by close when no call is in -// flight, or else by the last in-flight call to finish. -type wvRef struct { - ptr unsafe.Pointer - destroy func(unsafe.Pointer) - // state counts in-flight calls in its upper bits; the low bit is set - // once close has run. - state atomic.Uint64 -} - -func (r *wvRef) init(ptr unsafe.Pointer, destroy func(unsafe.Pointer)) { - r.ptr = ptr - r.destroy = destroy -} - -// acquire borrows the pointer for one call, panicking when the wrapper was -// already closed. -func (r *wvRef) acquire(typeName string) unsafe.Pointer { - for { - s := r.state.Load() - if s&1 != 0 { - panic("edge_cases: " + typeName + " used after Close") - } - if r.state.CompareAndSwap(s, s+2) { - return r.ptr - } - } -} - -// release ends one call's borrow. -func (r *wvRef) release() { - if r.state.Add(^uint64(1)) == 1 { - r.destroy(r.ptr) - } -} - -// close releases the wrapper's reference once no call is in flight. Calling -// it again is a no-op. -func (r *wvRef) close() { - for { - s := r.state.Load() - if s&1 != 0 { - return - } - if r.state.CompareAndSwap(s, s|1) { - if s == 0 { - r.destroy(r.ptr) - } - return - } - } -} - -// ── Async ── - -// wvOutcome is one async completion: a converted result, or the failure the -// producer reported. -type wvOutcome[T any] struct { - val T - fail *wvFailure -} - -// wvComplete delivers one async completion to the waiting wrapper. It runs on -// a producer thread inside an exported trampoline, so it never panics: a -// result that fails to decode is delivered as a marshalling failure. -func wvComplete[T any](handle unsafe.Pointer, cErr *C.edge_cases_error, convert func() T) { - h := cgo.Handle(uintptr(handle)) - done := h.Value().(chan wvOutcome[T]) - h.Delete() - done <- wvSettle(cErr, convert) -} - -func wvSettle[T any](cErr *C.edge_cases_error, convert func() T) (o wvOutcome[T]) { - if cErr != nil { - if f := wvTakeBoxedError(cErr); f.code != 0 { - o.fail = &f - return o - } - } - defer func() { - if r := recover(); r != nil { - o = wvOutcome[T]{fail: &wvFailure{code: wvCodeMarshal, message: fmt.Sprint(r)}} - } - }() - o.val = convert() - return o -} - -// wvAwait blocks until an async call completes or ctx is done. With a cancel -// token, ctx's cancellation cancels the native call and waits for its -// completion, which the producer delivers promptly; without one, the call is -// abandoned and ctx.Err() returned at once. A cancelled completion (code -5) -// returns ctx.Err(), or context.Canceled when ctx itself isn't done. -func wvAwait[T any](ctx context.Context, done chan wvOutcome[T], token *C.edge_cases_cancel_token) (T, *wvFailure, error) { - var o wvOutcome[T] - select { - case o = <-done: - case <-ctx.Done(): - if token == nil { - var zero T - return zero, nil, ctx.Err() - } - C.edge_cases_cancel_token_cancel(token) - o = <-done - } - if o.fail != nil && o.fail.code == wvCodeCancelled { - var zero T - if err := ctx.Err(); err != nil { - return zero, nil, err - } - return zero, nil, context.Canceled - } - return o.val, o.fail, nil -} - -// END runtime.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__README_md.snap index 3a991a52..d4e7d167 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__README_md.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # kitchen_sink (Go) @@ -22,6 +22,12 @@ export LD_LIBRARY_PATH="/path/to/lib" # DYLD_LIBRARY_PATH on macOS go build ./... ``` +cgo links the library when the program is built and the platform loader +finds it when the program starts, so the library-path environment variable +the runtime-loading bindings (Python, .NET, and others) honor doesn't apply. +Add `-Wl,-rpath,/path/to/lib` to `CGO_LDFLAGS` to record the directory in the +binary instead of setting the loader path. + Depend on the module from your own project with a `require` (and, for a local checkout, a `replace`) of `kitchen_sink`, then import it: @@ -38,12 +44,16 @@ Go 1.23 or newer is required. - Interfaces are reference-counted objects. Call `Close` when you're done with one (a finalizer releases it otherwise); it's safe to call twice or while a call is in flight. +- A call that declares errors returns `error` values you match with + `errors.As`; any other call panics with an `*Error` when the library + reports a failure, since that's a bug. - Async functions take a `context.Context` first. Cancellable ones cancel the native call when the context is done. - Callback interfaces are Go interfaces you implement. The library may call - them from any thread, and a panic is reported to the native caller as a - foreign error instead of crashing. -- Loading checks the library's ABI revision and contract checksums and - panics with the stale module's name on a mismatch. + them from any thread; a panic is reported to the native caller as a + callback failure instead of crashing. +- Importing the package checks the library's ABI revision and every + module's contract table, and panics naming the first declaration that's + missing from the library or changed since the bindings were generated. diff --git a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__bindings_go.snap b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__bindings_go.snap index b727ce48..d2ad043f 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__bindings_go.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__bindings_go.snap @@ -1,10 +1,11 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate +// Package kitchen_sink binds the kitchen_sink native library. package kitchen_sink /* @@ -14,49 +15,91 @@ package kitchen_sink __attribute__((unused)) static void* wvHandlePtr(uintptr_t h) { return (void*)h; } extern void goWv_kitchen_sink_kitchen_ReadyListener_on_ready(void* ctx, int32_t code, uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); extern bool goWv_kitchen_sink_kitchen_ReadyListener_on_item(void* ctx, uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); +extern void goWv_kitchen_sink_kitchen_ReadyListener_label(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); +extern void goWv_kitchen_sink_kitchen_ReadyListener_latest(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); +extern kitchen_sink_kitchen_Gadget* goWv_kitchen_sink_kitchen_ReadyListener_favorite(void* ctx, kitchen_sink_error* out_err); extern void goWv_kitchen_sink_kitchen_ReadyListener_free(void* ctx); static const kitchen_sink_kitchen_ReadyListener_vtable wvVtable_kitchen_sink_kitchen_ReadyListener_vtable = { - (void (*)(void*, int32_t, const uint8_t*, size_t, kitchen_sink_error*))goWv_kitchen_sink_kitchen_ReadyListener_on_ready, - (bool (*)(void*, const uint8_t*, size_t, kitchen_sink_kitchen_Gadget*, kitchen_sink_error*))goWv_kitchen_sink_kitchen_ReadyListener_on_item, - goWv_kitchen_sink_kitchen_ReadyListener_free, + .size = sizeof(kitchen_sink_kitchen_ReadyListener_vtable), + .flags = 0, + .free = goWv_kitchen_sink_kitchen_ReadyListener_free, + .on_ready = (void (*)(void*, int32_t, const uint8_t*, size_t, kitchen_sink_error*))goWv_kitchen_sink_kitchen_ReadyListener_on_ready, + .on_item = (bool (*)(void*, const uint8_t*, size_t, kitchen_sink_kitchen_Gadget*, kitchen_sink_error*))goWv_kitchen_sink_kitchen_ReadyListener_on_item, + .label = goWv_kitchen_sink_kitchen_ReadyListener_label, + .latest = goWv_kitchen_sink_kitchen_ReadyListener_latest, + .favorite = goWv_kitchen_sink_kitchen_ReadyListener_favorite, }; __attribute__((unused)) static const kitchen_sink_kitchen_ReadyListener_vtable* wvVtablePtr_kitchen_sink_kitchen_ReadyListener_vtable(void) { return &wvVtable_kitchen_sink_kitchen_ReadyListener_vtable; } -extern void goWv_kitchen_sink_kitchen_do_async_callback(void* context_, kitchen_sink_error* err, uint8_t* result_ptr, size_t result_len); -extern void goWv_kitchen_sink_kitchen_do_cancellable_callback(void* context_, kitchen_sink_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_kitchen_sink_kitchen_do_async_callback(void* context, kitchen_sink_error* err, uint8_t* result_ptr, size_t result_len); +extern void goWv_kitchen_sink_kitchen_do_cancellable_callback(void* context, kitchen_sink_error* err, uint8_t* result_ptr, size_t result_len); */ import "C" import ( "context" - "fmt" "iter" "runtime" - "runtime/cgo" "unsafe" ) func init() { wvCheckABI() - wvCheckModule("shared", 0x42c4ce2c8d0af052, uint64(C.kitchen_sink_shared_checksum())) - wvCheckModule("kitchen", 0xd83da75b24b545ff, uint64(C.kitchen_sink_kitchen_checksum())) + var n C.size_t + table := C.kitchen_sink_shared_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0x709e234caa962446, 0x3894f2ca452e9f98, "shared.ping"}, + {0xcfb306fa5801ee5b, 0x1cbd9e10983cac2c, "shared.Token"}, + }) + table = C.kitchen_sink_kitchen_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0x01d4f09bbddc986f, 0x29eb88fecc66544d, "kitchen.Gadget.new"}, + {0x10472d8fe748c2ee, 0xb265832b6a759fc4, "kitchen.maybe_gadget"}, + {0x11368eec07a74078, 0xfa049f78b77e146a, "kitchen.gadgets"}, + {0x137f08a29cde6021, 0xda8de7aeb2dbb8a3, "kitchen.stream_gadgets"}, + {0x2111b54980826dd7, 0x452783f3e6ca6e1d, "kitchen.do_cancellable"}, + {0x29dddbc60bf8d7db, 0x452a3e573d57f6db, "kitchen.Priority"}, + {0x2b227003b74a0d16, 0x12332957470e9782, "kitchen.maybe_subscribe"}, + {0x2dd66cb0064f0494, 0x60197363fefd92dd, "kitchen.Item"}, + {0x33c265bbf079b0e9, 0xc3dcd0927e6c1deb, "kitchen.u32_id"}, + {0x33e6381d01660dfe, 0x077c63c31fd7f90a, "kitchen.KitchenErrors"}, + {0x3f5fc99d73504b76, 0x2df88d710f1ea10e, "kitchen.Gadget.poke"}, + {0x4c5764cf109528d1, 0x2a65529af4338a58, "kitchen.maybe_item"}, + {0x4de17dfb93fd494d, 0xba0ad0dd52ed74af, "kitchen.i32_id"}, + {0x504e95b7f7e65570, 0x01b3e8e0c9a10e23, "kitchen.stream_items"}, + {0x68cdb9a4bd3e3dd5, 0xc678b9164348d497, "kitchen.bool_id"}, + {0x69e7cd703d16eb85, 0xf3b28dd22ae9e689, "kitchen.do_async"}, + {0x70865079ce41be82, 0xbe3b69464672af46, "kitchen.Gadget.describe"}, + {0x71c5a1116cb7c49a, 0xcba1837108a89b84, "kitchen.echo_bytes"}, + {0x735ee1e0049ee91a, 0xca74320bc53eddab, "kitchen.echo_string"}, + {0x76d7fc58f4e41d84, 0x7373a0e47d30e690, "kitchen.i64_id"}, + {0x79abfe1aa960fd6a, 0xbca7a9e5eb26f3a6, "kitchen.nested.hello"}, + {0x7d3dd7d5cf6749e3, 0xa192f5a7298adb39, "kitchen.new_op"}, + {0x89a5e664bc25c67b, 0x5764a95e81ecfd31, "kitchen.f64_id"}, + {0x8cfdbdcecac5f7f1, 0x07e272f09c41fffc, "kitchen.Gadget.version"}, + {0x974b9f3e823c1738, 0xb81b91e975695f27, "kitchen.cross_module_token"}, + {0xaca0655fac15d230, 0x11c4d2b28abe07bb, "kitchen.list_items"}, + {0xc883b4afae20543e, 0xc40560e5411ff392, "kitchen.legacy_op"}, + {0xf2f7753e6e1dffc2, 0xc9047a95bdf49e93, "kitchen.get_attrs"}, + {0xfae25d3391922ead, 0x1c7f95739f3210b8, "kitchen.subscribe"}, + {0xfd0059b05cb2fcfb, 0x248bf8404a5823b4, "kitchen.Gadget"}, + {0xfed9c551fb9fd2d6, 0x59aec980ef3f3291, "kitchen.ReadyListener"}, + }) } -// Token: A reusable token referenced across modules +// Token is a reusable token referenced across modules type Token struct { - Id int64 + ID int64 Label string } -// wvPackToken appends v to w in the value-buffer wire format. -func wvPackToken(w *wvWriter, v Token) { - w.writeI64(v.Id) +func wvWriteToken(w *wvWriter, v Token) { + w.writeI64(v.ID) w.writeString(v.Label) } -// wvUnpackToken decodes one Token from r. -func wvUnpackToken(r *wvReader) Token { +func wvReadToken(r *wvReader) Token { var v Token - v.Id = r.readI64() + v.ID = r.readI64() v.Label = r.readString() return v } @@ -70,43 +113,80 @@ func Ping() string { return wvTakeString(cRet, cRetLen) } -// KitchenErrorsError is a typed error reported by the `kitchen` module. -type KitchenErrorsError struct { - // Code is the numeric ABI error code (one of the KitchenErrorsError constants). - Code int32 - // Message is the human-readable error message. +// KitchenErrorsError is an error the kitchen module reports: *NotFoundError, +// or *InvalidInputError. +// +// Match one code with errors.As and its type, or any of them with a +// KitchenErrorsError. A throwing callback method returns one to report that +// code. +type KitchenErrorsError interface { + error + // Code returns the error's numeric code. + Code() int32 + isKitchenErrorsError() +} + +// NotFoundError is the KitchenErrorsError code NotFound (1): Item not found. +type NotFoundError struct { + // Message describes the failure; when it's empty, Error returns the + // code's default message, "Item not found". Message string } -func (e *KitchenErrorsError) Error() string { - return fmt.Sprintf("kitchen: %s (code %d)", e.Message, e.Code) +// Error returns the error's message. +func (e *NotFoundError) Error() string { + if e.Message == "" { + return "Item not found" + } + return e.Message } -// KitchenErrorsError codes. -const ( - // KitchenErrorsErrorNotFound: Item not found - KitchenErrorsErrorNotFound int32 = 1 - // KitchenErrorsErrorInvalidInput: Invalid input - KitchenErrorsErrorInvalidInput int32 = 2 -) +// Code returns 1. +func (*NotFoundError) Code() int32 { + return 1 +} + +func (*NotFoundError) isKitchenErrorsError() {} + +func (*NotFoundError) wvPayload() []byte { + return nil +} + +// InvalidInputError is the KitchenErrorsError code InvalidInput (2): Invalid +// input. +type InvalidInputError struct { + // Message describes the failure; when it's empty, Error returns the + // code's default message, "Invalid input". + Message string +} + +// Error returns the error's message. +func (e *InvalidInputError) Error() string { + if e.Message == "" { + return "Invalid input" + } + return e.Message +} + +// Code returns 2. +func (*InvalidInputError) Code() int32 { + return 2 +} + +func (*InvalidInputError) isKitchenErrorsError() {} + +func (*InvalidInputError) wvPayload() []byte { + return nil +} -// wvMapKitchen converts a non-zero code from the `kitchen` domain into a -// *KitchenErrorsError, falling back to the generic *Error for unknown codes. -func wvMapKitchen(f wvFailure) error { +func wvKitchenErrorsError(f wvFailure) error { switch f.code { - case KitchenErrorsErrorNotFound: - if f.message == "" { - f.message = "Item not found" - } - return &KitchenErrorsError{Code: f.code, Message: f.message} - case KitchenErrorsErrorInvalidInput: - if f.message == "" { - f.message = "Invalid input" - } - return &KitchenErrorsError{Code: f.code, Message: f.message} - default: - return f.err() + case 1: + return &NotFoundError{Message: f.message} + case 2: + return &InvalidInputError{Message: f.message} } + return f.err() } // Priority: Task priority levels @@ -121,15 +201,23 @@ const ( PriorityHigh Priority = 2 ) +func wvWritePriority(w *wvWriter, v Priority) { + w.writeI32(int32(v)) +} + +func wvReadPriority(r *wvReader) Priority { + return Priority(r.readI32()) +} + // Item: Kitchen sink struct exercising every field feature type Item struct { - // Id: Stable identifier - Id int64 + // ID: Stable identifier + ID int64 // Name: Display name Name string // Count: Initial count Count int32 - // Enabled: Whether the item is enabled + // Enabled reports whether the item is enabled Enabled bool // Ratio: Scaling ratio Ratio float64 @@ -141,139 +229,114 @@ type Item struct { // Token: Cross-module struct reference Token Token Priority Priority - // Gadget: An optional object inside a record + // Gadget is an optional object inside a record Gadget *Gadget } -// wvPackItem appends v to w in the value-buffer wire format. -func wvPackItem(w *wvWriter, v Item) { - w.writeI64(v.Id) +func wvWriteItem(w *wvWriter, v Item) { + w.writeI64(v.ID) w.writeString(v.Name) w.writeI32(v.Count) w.writeBool(v.Enabled) w.writeF64(v.Ratio) w.writeU32(v.Nick) w.writeBytes(v.Payload) - w.writeLen(len(v.Tags)) - for _, eTags0 := range v.Tags { - w.writeString(eTags0) - } - w.writeLen(len(v.Attrs)) - for kAttrs0, vAttrs0 := range v.Attrs { - w.writeString(kAttrs0) - w.writeString(vAttrs0) - } - if v.Parent == nil { - w.writeOptionFlag(false) - } else { - w.writeOptionFlag(true) - w.writeI64(*v.Parent) - } - wvPackToken(w, v.Token) - w.writeI32(int32(v.Priority)) - if v.Gadget == nil { - w.writeOptionFlag(false) - } else { - w.writeOptionFlag(true) - w.writeU64(wvTokenGadget(v.Gadget)) - } + wvWriteListString(w, v.Tags) + wvWriteMapStringString(w, v.Attrs) + wvWriteOptI64(w, v.Parent) + wvWriteToken(w, v.Token) + wvWritePriority(w, v.Priority) + wvWriteOptGadget(w, v.Gadget) } -// wvUnpackItem decodes one Item from r. -func wvUnpackItem(r *wvReader) Item { +func wvReadItem(r *wvReader) Item { var v Item - v.Id = r.readI64() + v.ID = r.readI64() v.Name = r.readString() v.Count = r.readI32() v.Enabled = r.readBool() v.Ratio = r.readF64() v.Nick = r.readU32() v.Payload = r.readBytes() - nTags0 := r.readLen() - v.Tags = make([]string, 0, r.capHint(nTags0)) - for range nTags0 { - var eTags0 string - eTags0 = r.readString() - v.Tags = append(v.Tags, eTags0) - } - nAttrs0 := r.readLen() - v.Attrs = make(map[string]string, r.capHint(nAttrs0)) - for range nAttrs0 { - var kAttrs0 string - kAttrs0 = r.readString() - var vAttrs0 string - vAttrs0 = r.readString() - v.Attrs[kAttrs0] = vAttrs0 - } - if r.readOptionFlag() { - var oParent0 int64 - oParent0 = r.readI64() - v.Parent = &oParent0 - } - v.Token = wvUnpackToken(r) - v.Priority = Priority(r.readI32()) - if r.readOptionFlag() { - var oGadget0 *Gadget - oGadget0 = wvUntokenGadget(r.readU64()) - v.Gadget = oGadget0 - } + v.Tags = wvReadListString(r) + v.Attrs = wvReadMapStringString(r) + v.Parent = wvReadOptI64(r) + v.Token = wvReadToken(r) + v.Priority = wvReadPriority(r) + v.Gadget = wvReadOptGadget(r) return v } // ReadyListener: Consumer-implemented observer of ready items // -// Implement this interface in Go and pass the value to native functions -// that accept it. -// -// The native library may call any method from any thread until it releases -// the implementation. A panic in a method is reported to the native caller -// as a foreign error (code -4) instead of crashing the process. +// Implement ReadyListener in Go and pass the value to the functions that take +// it. The native library may call its methods from any thread until it +// releases the implementation. A method that panics, or a throwing method that +// returns an error outside its domain, fails the native call in progress as a +// callback failure (code -4). type ReadyListener interface { - // OnReady: Fires when an item is ready + // OnReady fires when an item is ready OnReady(code int32, msg string) - // OnItem: Receives the item itself and says whether to keep listening + // OnItem receives the item itself and says whether to keep listening OnItem(item Item, gadget *Gadget) bool + // Label returns a display label, failing with a kitchen error + // + // Return a KitchenErrorsError to report that code, with its fields, to the + // native caller. + Label() (string, error) + // Latest returns the most recent item the listener kept, if any + Latest() *Item + // Favorite returns the listener's favorite gadget + Favorite() *Gadget } //export goWv_kitchen_sink_kitchen_ReadyListener_on_ready func goWv_kitchen_sink_kitchen_ReadyListener_on_ready(ctx unsafe.Pointer, code C.int32_t, msg_ptr *C.uint8_t, msg_len C.size_t, out_err *C.kitchen_sink_error) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(ReadyListener) - arg0 := int32(code) - arg1 := wvBorrowString(msg_ptr, msg_len) - impl.OnReady(arg0, arg1) + defer wvRecoverCallback(out_err) + wvCallback[ReadyListener](ctx).OnReady(int32(code), wvBorrowString(msg_ptr, msg_len)) } //export goWv_kitchen_sink_kitchen_ReadyListener_on_item -func goWv_kitchen_sink_kitchen_ReadyListener_on_item(ctx unsafe.Pointer, item_ptr *C.uint8_t, item_len C.size_t, gadget *C.kitchen_sink_kitchen_Gadget, out_err *C.kitchen_sink_error) (ret C._Bool) { - defer func() { - if r := recover(); r != nil { - wvForeignError(out_err, r) - } - }() - impl := cgo.Handle(uintptr(ctx)).Value().(ReadyListener) - rArg0 := &wvReader{buf: wvBorrowBytes(item_ptr, item_len)} - var arg0 Item - arg0 = wvUnpackItem(rArg0) - rArg0.expectEnd() - arg1 := wvAdoptGadget(gadget) - ret = C._Bool(impl.OnItem(arg0, arg1)) - return +func goWv_kitchen_sink_kitchen_ReadyListener_on_item(ctx unsafe.Pointer, item_ptr *C.uint8_t, item_len C.size_t, gadget *C.kitchen_sink_kitchen_Gadget, out_err *C.kitchen_sink_error) (cRet C._Bool) { + defer wvRecoverCallback(out_err) + ret := wvCallback[ReadyListener](ctx).OnItem(wvDecodeBorrowed(item_ptr, item_len, wvReadItem), wvAdoptGadget(gadget)) + return C._Bool(ret) +} + +//export goWv_kitchen_sink_kitchen_ReadyListener_label +func goWv_kitchen_sink_kitchen_ReadyListener_label(ctx unsafe.Pointer, out_ptr **C.uint8_t, out_len *C.size_t, out_err *C.kitchen_sink_error) { + defer wvRecoverCallback(out_err) + ret, err := wvCallback[ReadyListener](ctx).Label() + if err != nil { + wvCallbackFailed[KitchenErrorsError](out_err, err) + return + } + wvHandOverString(ret, out_ptr, out_len) +} + +//export goWv_kitchen_sink_kitchen_ReadyListener_latest +func goWv_kitchen_sink_kitchen_ReadyListener_latest(ctx unsafe.Pointer, out_ptr **C.uint8_t, out_len *C.size_t, out_err *C.kitchen_sink_error) { + defer wvRecoverCallback(out_err) + ret := wvCallback[ReadyListener](ctx).Latest() + wvHandOverBytes(wvEncode(ret, wvWriteOptItem), out_ptr, out_len) +} + +//export goWv_kitchen_sink_kitchen_ReadyListener_favorite +func goWv_kitchen_sink_kitchen_ReadyListener_favorite(ctx unsafe.Pointer, out_err *C.kitchen_sink_error) (cRet *C.kitchen_sink_kitchen_Gadget) { + defer wvRecoverCallback(out_err) + ret := wvCallback[ReadyListener](ctx).Favorite() + return ret.share() } //export goWv_kitchen_sink_kitchen_ReadyListener_free func goWv_kitchen_sink_kitchen_ReadyListener_free(ctx unsafe.Pointer) { - cgo.Handle(uintptr(ctx)).Delete() + wvFreeCallback(ctx) } -// Gadget: A pocket-sized interface exercising the object surface +// Gadget is a pocket-sized interface exercising the object surface // -// Each wrapper holds one strong reference; Close releases it, and a -// finalizer releases it if the wrapper is garbage collected first. +// Each wrapper holds one strong reference; Close releases it, and a finalizer +// releases it if the wrapper is garbage collected first. type Gadget struct { ref wvRef } @@ -282,8 +345,7 @@ func wvDestroyGadget(ptr unsafe.Pointer) { C.kitchen_sink_kitchen_Gadget_destroy((*C.kitchen_sink_kitchen_Gadget)(ptr)) } -// wvAdoptGadget adopts one owned strong reference into a new wrapper. A null -// pointer adopts to nil. +// wvAdoptGadget wraps one owned strong reference. A null pointer adopts to nil. func wvAdoptGadget(ptr *C.kitchen_sink_kitchen_Gadget) *Gadget { if ptr == nil { return nil @@ -303,19 +365,25 @@ func (s *Gadget) native() *C.kitchen_sink_kitchen_Gadget { return (*C.kitchen_sink_kitchen_Gadget)(s.ref.acquire("Gadget")) } -// wvTokenGadget clones o's reference into a value-buffer object token. The -// wrapper keeps its own reference; the token carries the new one. -func wvTokenGadget(o *Gadget) uint64 { - ptr := o.native() - defer o.ref.release() - return uint64(uintptr(unsafe.Pointer(C.kitchen_sink_kitchen_Gadget_clone(ptr)))) +// share returns a new strong reference to the wrapper's object for the +// native library to adopt (the wrapper keeps its own), or null for nil. +func (s *Gadget) share() *C.kitchen_sink_kitchen_Gadget { + if s == nil { + return nil + } + ptr := s.native() + defer s.ref.release() + return C.kitchen_sink_kitchen_Gadget_clone(ptr) +} + +func wvWriteGadget(w *wvWriter, v *Gadget) { + w.writeU64(uint64(uintptr(unsafe.Pointer(v.share())))) } -// wvUntokenGadget adopts the strong reference carried by a value-buffer object -// token. -func wvUntokenGadget(token uint64) *Gadget { +func wvReadGadget(r *wvReader) *Gadget { + token := r.readU64() if token == 0 { - panic("kitchen_sink: malformed value buffer: null object token") + wvMalformed("null object token") } return wvAdoptGadget((*C.kitchen_sink_kitchen_Gadget)(C.wvHandlePtr(C.uintptr_t(token)))) } @@ -324,7 +392,7 @@ func wvUntokenGadget(token uint64) *Gadget { // It's idempotent and safe to call from any goroutine, even while a call // on the wrapper is in flight: the reference is then released when that // call returns. The object itself is dropped once its last reference, from -// any wrapper, record, or the native side, is gone. +// any wrapper, record, or the native library, is gone. func (s *Gadget) Close() error { runtime.SetFinalizer(s, nil) s.ref.close() @@ -350,19 +418,19 @@ func (s *Gadget) Describe() string { return wvTakeString(cRet, cRetLen) } -// Poke: Poke the gadget a number of times, failing on a negative count +// Poke the gadget a number of times, failing on a negative count func (s *Gadget) Poke(times int32) (int32, error) { cSelf := s.native() defer s.ref.release() var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_Gadget_poke(cSelf, C.int32_t(times), &cErr) if cErr.code != 0 { - return 0, wvMapKitchen(wvTakeError(&cErr)) + return 0, wvKitchenErrorsError(wvTakeError(&cErr)) } return int32(cRet), nil } -// GadgetVersion: The gadget subsystem version string +// GadgetVersion returns the gadget subsystem version string func GadgetVersion() string { var cRetLen C.size_t var cErr C.kitchen_sink_error @@ -371,36 +439,36 @@ func GadgetVersion() string { return wvTakeString(cRet, cRetLen) } -// BoolId: Identity for bool -func BoolId(x bool) bool { +// BoolID: Identity for bool +func BoolID(x bool) bool { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_bool_id(C._Bool(x), &cErr) wvTrap(&cErr) return bool(cRet) } -func I32Id(x int32) int32 { +func I32ID(x int32) int32 { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_i32_id(C.int32_t(x), &cErr) wvTrap(&cErr) return int32(cRet) } -func U32Id(x uint32) uint32 { +func U32ID(x uint32) uint32 { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_u32_id(C.uint32_t(x), &cErr) wvTrap(&cErr) return uint32(cRet) } -func I64Id(x int64) int64 { +func I64ID(x int64) int64 { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_i64_id(C.int64_t(x), &cErr) wvTrap(&cErr) return int64(cRet) } -func F64Id(x float64) float64 { +func F64ID(x float64) float64 { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_f64_id(C.double(x), &cErr) wvTrap(&cErr) @@ -427,13 +495,25 @@ func EchoBytes(b []byte) []byte { // Subscribe: Register a listener, returning how many are attached func Subscribe(listener ReadyListener) int32 { - hListener := cgo.NewHandle(listener) var cErr C.kitchen_sink_error - cRet := C.kitchen_sink_kitchen_subscribe(C.wvHandlePtr(C.uintptr_t(hListener)), C.wvVtablePtr_kitchen_sink_kitchen_ReadyListener_vtable(), &cErr) + cRet := C.kitchen_sink_kitchen_subscribe(wvNewCallback(listener), C.wvVtablePtr_kitchen_sink_kitchen_ReadyListener_vtable(), &cErr) wvTrap(&cErr) return int32(cRet) } +// MaybeSubscribe: Register a listener if one is given +func MaybeSubscribe(listener ReadyListener) bool { + var cListenerCtx unsafe.Pointer + var cListenerVtable *C.kitchen_sink_kitchen_ReadyListener_vtable + if listener != nil { + cListenerCtx, cListenerVtable = wvNewCallback(listener), C.wvVtablePtr_kitchen_sink_kitchen_ReadyListener_vtable() + } + var cErr C.kitchen_sink_error + cRet := C.kitchen_sink_kitchen_maybe_subscribe(cListenerCtx, cListenerVtable, &cErr) + wvTrap(&cErr) + return bool(cRet) +} + // MaybeGadget: Optional object in both directions func MaybeGadget(fallback *Gadget) *Gadget { var cFallback *C.kitchen_sink_kitchen_Gadget @@ -447,29 +527,20 @@ func MaybeGadget(fallback *Gadget) *Gadget { return wvAdoptGadget(cRet) } -// Gadgets: A list of objects +// Gadgets returns a list of objects func Gadgets() []*Gadget { var cRetLen C.size_t var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_gadgets(&cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret []*Gadget - nRet0 := rRet.readLen() - ret = make([]*Gadget, 0, rRet.capHint(nRet0)) - for range nRet0 { - var eRet0 *Gadget - eRet0 = wvUntokenGadget(rRet.readU64()) - ret = append(ret, eRet0) - } - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadListGadget) } -// StreamGadgets: An iterator over objects +// StreamGadgets returns an iterator over objects // -// The sequence launches the native iterator on each range and calls next once -// per element; the iterator is destroyed when the range ends, early or not. +// The sequence launches the native iterator each time it's ranged over and +// pulls one element per step; the iterator is released when the range ends, +// early or not. func StreamGadgets() iter.Seq[*Gadget] { return func(yield func(*Gadget) bool) { var cErr C.kitchen_sink_error @@ -484,8 +555,7 @@ func StreamGadgets() iter.Seq[*Gadget] { if !more { return } - item := wvAdoptGadget(cItem) - if !yield(item) { + if !yield(wvAdoptGadget(cItem)) { return } } @@ -498,17 +568,9 @@ func MaybeItem(id int64) (*Item, error) { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_maybe_item(C.int64_t(id), &cRetLen, &cErr) if cErr.code != 0 { - return nil, wvMapKitchen(wvTakeError(&cErr)) - } - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret *Item - if rRet.readOptionFlag() { - var oRet0 Item - oRet0 = wvUnpackItem(rRet) - ret = &oRet0 + return nil, wvKitchenErrorsError(wvTakeError(&cErr)) } - rRet.expectEnd() - return ret, nil + return wvDecode(cRet, cRetLen, wvReadOptItem), nil } // ListItems: List of structs @@ -517,17 +579,7 @@ func ListItems() []Item { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_list_items(&cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret []Item - nRet0 := rRet.readLen() - ret = make([]Item, 0, rRet.capHint(nRet0)) - for range nRet0 { - var eRet0 Item - eRet0 = wvUnpackItem(rRet) - ret = append(ret, eRet0) - } - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadListItem) } // GetAttrs: Map return type @@ -536,25 +588,14 @@ func GetAttrs() map[string]int32 { var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_get_attrs(&cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret map[string]int32 - nRet0 := rRet.readLen() - ret = make(map[string]int32, rRet.capHint(nRet0)) - for range nRet0 { - var kRet0 string - kRet0 = rRet.readString() - var vRet0 int32 - vRet0 = rRet.readI32() - ret[kRet0] = vRet0 - } - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadMapStringI32) } // StreamItems: Iterator return type // -// The sequence launches the native iterator on each range and calls next once -// per element; the iterator is destroyed when the range ends, early or not. +// The sequence launches the native iterator each time it's ranged over and +// pulls one element per step; the iterator is released when the range ends, +// early or not. func StreamItems() iter.Seq[Item] { return func(yield func(Item) bool) { var cErr C.kitchen_sink_error @@ -570,33 +611,25 @@ func StreamItems() iter.Seq[Item] { if !more { return } - rItem := &wvReader{buf: wvTakeBytes(cItem, cItemLen)} - var item Item - item = wvUnpackItem(rItem) - rItem.expectEnd() - if !yield(item) { + if !yield(wvDecode(cItem, cItemLen, wvReadItem)) { return } } } } -// CrossModuleToken: Returns the shared Token type from another module +// CrossModuleToken returns the shared Token type from another module func CrossModuleToken() Token { var cRetLen C.size_t var cErr C.kitchen_sink_error cRet := C.kitchen_sink_kitchen_cross_module_token(&cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Token - ret = wvUnpackToken(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadToken) } //export goWv_kitchen_sink_kitchen_do_async_callback -func goWv_kitchen_sink_kitchen_do_async_callback(context_ unsafe.Pointer, err *C.kitchen_sink_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() string { +func goWv_kitchen_sink_kitchen_do_async_callback(context_ unsafe.Pointer, err_ *C.kitchen_sink_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() string { return wvTakeString(result_ptr, result_len) }) } @@ -610,10 +643,9 @@ func DoAsync(ctx context.Context, input string) (string, error) { return "", err } cInputPtr, cInputLen := wvStr(input) - wvDone := make(chan wvOutcome[string], 1) - wvHandle := cgo.NewHandle(wvDone) - C.kitchen_sink_kitchen_do_async(cInputPtr, cInputLen, C.kitchen_sink_kitchen_do_async_callback(unsafe.Pointer(C.goWv_kitchen_sink_kitchen_do_async_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, nil) + call := wvNewAsyncCall[string]() + C.kitchen_sink_kitchen_do_async(cInputPtr, cInputLen, C.kitchen_sink_kitchen_do_async_callback(unsafe.Pointer(C.goWv_kitchen_sink_kitchen_do_async_callback)), call.context()) + res, fail, err := call.wait(ctx, nil) if err != nil { return "", err } @@ -624,27 +656,26 @@ func DoAsync(ctx context.Context, input string) (string, error) { } //export goWv_kitchen_sink_kitchen_do_cancellable_callback -func goWv_kitchen_sink_kitchen_do_cancellable_callback(context_ unsafe.Pointer, err *C.kitchen_sink_error, result_ptr *C.uint8_t, result_len C.size_t) { - wvComplete(context_, err, func() string { +func goWv_kitchen_sink_kitchen_do_cancellable_callback(context_ unsafe.Pointer, err_ *C.kitchen_sink_error, result_ptr *C.uint8_t, result_len C.size_t) { + wvComplete(context_, err_, func() string { return wvTakeString(result_ptr, result_len) }) } // DoCancellable: Cancellable async operation // -// It blocks until the call completes. Cancelling ctx cancels the native -// call, which then returns ctx.Err(). +// It blocks until the call completes. Cancelling ctx cancels the native call, +// which then returns ctx.Err(). func DoCancellable(ctx context.Context, input string) (string, error) { if err := ctx.Err(); err != nil { return "", err } cInputPtr, cInputLen := wvStr(input) - wvToken := C.kitchen_sink_cancel_token_create() - defer C.kitchen_sink_cancel_token_destroy(wvToken) - wvDone := make(chan wvOutcome[string], 1) - wvHandle := cgo.NewHandle(wvDone) - C.kitchen_sink_kitchen_do_cancellable(cInputPtr, cInputLen, wvToken, C.kitchen_sink_kitchen_do_cancellable_callback(unsafe.Pointer(C.goWv_kitchen_sink_kitchen_do_cancellable_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle))) - res, fail, err := wvAwait(ctx, wvDone, wvToken) + token := C.kitchen_sink_cancel_token_create() + defer C.kitchen_sink_cancel_token_destroy(token) + call := wvNewAsyncCall[string]() + C.kitchen_sink_kitchen_do_cancellable(cInputPtr, cInputLen, token, C.kitchen_sink_kitchen_do_cancellable_callback(unsafe.Pointer(C.goWv_kitchen_sink_kitchen_do_cancellable_callback)), call.context()) + res, fail, err := call.wait(ctx, token) if err != nil { return "", err } @@ -681,4 +712,68 @@ func Hello() string { return wvTakeString(cRet, cRetLen) } +func wvWriteListGadget(w *wvWriter, v []*Gadget) { + wvWriteList(w, v, wvWriteGadget) +} + +func wvReadListGadget(r *wvReader) []*Gadget { + return wvReadList(r, wvReadGadget) +} + +func wvWriteListItem(w *wvWriter, v []Item) { + wvWriteList(w, v, wvWriteItem) +} + +func wvReadListItem(r *wvReader) []Item { + return wvReadList(r, wvReadItem) +} + +func wvWriteListString(w *wvWriter, v []string) { + wvWriteList(w, v, (*wvWriter).writeString) +} + +func wvReadListString(r *wvReader) []string { + return wvReadList(r, (*wvReader).readString) +} + +func wvWriteMapStringI32(w *wvWriter, v map[string]int32) { + wvWriteMap(w, v, (*wvWriter).writeString, (*wvWriter).writeI32) +} + +func wvReadMapStringI32(r *wvReader) map[string]int32 { + return wvReadMap(r, (*wvReader).readString, (*wvReader).readI32) +} + +func wvWriteMapStringString(w *wvWriter, v map[string]string) { + wvWriteMap(w, v, (*wvWriter).writeString, (*wvWriter).writeString) +} + +func wvReadMapStringString(r *wvReader) map[string]string { + return wvReadMap(r, (*wvReader).readString, (*wvReader).readString) +} + +func wvWriteOptGadget(w *wvWriter, v *Gadget) { + wvWriteNilable(w, v, v != nil, wvWriteGadget) +} + +func wvReadOptGadget(r *wvReader) *Gadget { + return wvReadNilable(r, wvReadGadget) +} + +func wvWriteOptI64(w *wvWriter, v *int64) { + wvWriteOptional(w, v, (*wvWriter).writeI64) +} + +func wvReadOptI64(r *wvReader) *int64 { + return wvReadOptional(r, (*wvReader).readI64) +} + +func wvWriteOptItem(w *wvWriter, v *Item) { + wvWriteOptional(w, v, wvWriteItem) +} + +func wvReadOptItem(r *wvReader) *Item { + return wvReadOptional(r, wvReadItem) +} + // END bindings.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__codec_go.snap b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__codec_go.snap index cf805789..bdeeb6a1 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__codec_go.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__codec_go.snap @@ -1,18 +1,129 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink +/* +#include "kitchen_sink.h" +*/ +import "C" + import ( "encoding/binary" "math" "unicode/utf8" ) +// wvEncode encodes v with write into a new value buffer. +func wvEncode[T any](v T, write func(*wvWriter, T)) []byte { + var w wvWriter + write(&w, v) + return w.buf +} + +// wvDecode decodes one value from a returned value buffer, checks that +// nothing trails it, and releases the native allocation. +func wvDecode[T any](ptr *C.uint8_t, n C.size_t, read func(*wvReader) T) T { + return wvDecodeBytes(wvTakeBytes(ptr, n), read) +} + +// wvDecodeBorrowed is wvDecode for a buffer the native library only lends +// (a callback argument). +func wvDecodeBorrowed[T any](ptr *C.uint8_t, n C.size_t, read func(*wvReader) T) T { + return wvDecodeBytes(wvBorrowBytes(ptr, n), read) +} + +func wvDecodeBytes[T any](b []byte, read func(*wvReader) T) T { + r := wvReader{buf: b} + v := read(&r) + r.expectEnd() + return v +} + +// wvWriteList writes a u32 count, then each element. +func wvWriteList[T any](w *wvWriter, v []T, elem func(*wvWriter, T)) { + w.writeLen(len(v)) + for _, e := range v { + elem(w, e) + } +} + +// wvReadList reads a u32 count, then that many elements. +func wvReadList[T any](r *wvReader, elem func(*wvReader) T) []T { + n := r.readLen() + v := make([]T, 0, r.capHint(n)) + for range n { + v = append(v, elem(r)) + } + return v +} + +// wvWriteMap writes a u32 count, then each key and value. Entry order +// carries no meaning. +func wvWriteMap[K comparable, V any](w *wvWriter, v map[K]V, key func(*wvWriter, K), val func(*wvWriter, V)) { + w.writeLen(len(v)) + for k, e := range v { + key(w, k) + val(w, e) + } +} + +// wvReadMap reads a u32 count, then that many keys and values. A repeated +// key is a malformed buffer. +func wvReadMap[K comparable, V any](r *wvReader, key func(*wvReader) K, val func(*wvReader) V) map[K]V { + n := r.readLen() + v := make(map[K]V, r.capHint(n)) + for range n { + k := key(r) + if _, dup := v[k]; dup { + wvMalformed("repeated map key") + } + v[k] = val(r) + } + return v +} + +// wvWriteOptional writes an optional held as a pointer: a presence flag, +// then the value when present. +func wvWriteOptional[T any](w *wvWriter, v *T, elem func(*wvWriter, T)) { + w.writeBool(v != nil) + if v != nil { + elem(w, *v) + } +} + +// wvReadOptional reads an optional held as a pointer. +func wvReadOptional[T any](r *wvReader, elem func(*wvReader) T) *T { + if !r.readOptionFlag() { + return nil + } + v := elem(r) + return &v +} + +// wvWriteNilable writes an optional whose Go type is already nil-able (a +// slice, map, interface, or wrapper pointer): present says it isn't nil. +func wvWriteNilable[T any](w *wvWriter, v T, present bool, elem func(*wvWriter, T)) { + w.writeBool(present) + if present { + elem(w, v) + } +} + +// wvReadNilable reads an optional whose Go type is already nil-able; absent +// is the zero value (nil). +func wvReadNilable[T any](r *wvReader, elem func(*wvReader) T) T { + if !r.readOptionFlag() { + var zero T + return zero + } + return elem(r) +} + // wvWriter serializes values into the value-buffer format: little-endian, // packed, u32 length prefixes. type wvWriter struct { @@ -84,10 +195,6 @@ func (w *wvWriter) writeBytes(v []byte) { w.buf = append(w.buf, v...) } -func (w *wvWriter) writeOptionFlag(present bool) { - w.writeBool(present) -} - // wvReader decodes values from the value-buffer format. A malformed buffer // is a producer/consumer contract violation, so every read panics instead of // returning a typed domain error. diff --git a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__go_mod.snap b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__go_mod.snap index 4a40225f..895b9e79 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__go_mod.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__go_mod.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate module kitchen_sink diff --git a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__kitchen_sink_h.snap b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__kitchen_sink_h.snap deleted file mode 100644 index 14faeffb..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__kitchen_sink_h.snap +++ /dev/null @@ -1,219 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef KITCHEN_SINK_H -#define KITCHEN_SINK_H - -#include -#include -#include - -#ifndef KITCHEN_SINK_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef KITCHEN_SINK_BUILD -# define KITCHEN_SINK_API __declspec(dllexport) -# else -# define KITCHEN_SINK_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define KITCHEN_SINK_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define KITCHEN_SINK_API __attribute__((visibility("default"))) -# else -# define KITCHEN_SINK_API -# endif -#endif - -#ifndef KITCHEN_SINK_DEPRECATED -# if defined(_MSC_VER) -# define KITCHEN_SINK_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define KITCHEN_SINK_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define KITCHEN_SINK_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports kitchen_sink_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define KITCHEN_SINK_ABI_VERSION 3u -KITCHEN_SINK_API uint32_t kitchen_sink_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by kitchen_sink_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct kitchen_sink_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} kitchen_sink_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -KITCHEN_SINK_API void kitchen_sink_error_set(kitchen_sink_error* err, int32_t code, const char* message); -KITCHEN_SINK_API void kitchen_sink_error_clear(kitchen_sink_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - kitchen_sink_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_error_free(kitchen_sink_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -KITCHEN_SINK_API void kitchen_sink_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct kitchen_sink_cancel_token kitchen_sink_cancel_token; -KITCHEN_SINK_API kitchen_sink_cancel_token* kitchen_sink_cancel_token_create(void); -KITCHEN_SINK_API void kitchen_sink_cancel_token_cancel(kitchen_sink_cancel_token* token); -KITCHEN_SINK_API bool kitchen_sink_cancel_token_is_cancelled(const kitchen_sink_cancel_token* token); -KITCHEN_SINK_API void kitchen_sink_cancel_token_destroy(kitchen_sink_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -KITCHEN_SINK_API uint8_t* kitchen_sink_alloc(uint32_t size); -KITCHEN_SINK_API void kitchen_sink_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -KITCHEN_SINK_API uint64_t kitchen_sink_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define KITCHEN_SINK_SHARED_CHECKSUM 0x42c4ce2c8d0af052ull -KITCHEN_SINK_API uint64_t kitchen_sink_shared_checksum(void); -#define KITCHEN_SINK_KITCHEN_CHECKSUM 0xd83da75b24b545ffull -KITCHEN_SINK_API uint64_t kitchen_sink_kitchen_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with kitchen_sink_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * kitchen_sink_buffer.h declares a C struct and codec for each. - */ - -/** Task priority levels */ -typedef enum { - /** Low priority */ - kitchen_sink_kitchen_Priority_Low = 0, - /** Normal priority */ - kitchen_sink_kitchen_Priority_Normal = 1, - /** High priority */ - kitchen_sink_kitchen_Priority_High = 2 -} kitchen_sink_kitchen_Priority; -/** Error codes reported by throwing functions in the `kitchen` module tree. */ -typedef enum { kitchen_sink_kitchen_KitchenErrors_NotFound = 1, kitchen_sink_kitchen_KitchenErrors_InvalidInput = 2 } kitchen_sink_kitchen_KitchenErrors; -typedef struct kitchen_sink_kitchen_Gadget kitchen_sink_kitchen_Gadget; -typedef struct kitchen_sink_kitchen_StreamGadgetsIterator kitchen_sink_kitchen_StreamGadgetsIterator; -typedef struct kitchen_sink_kitchen_StreamItemsIterator kitchen_sink_kitchen_StreamItemsIterator; -/** - * Consumer-implemented observer of ready items - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct kitchen_sink_kitchen_ReadyListener_vtable { - /** Fires when an item is ready */ - void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); - /** Receives the item itself and says whether to keep listening */ - bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); -} kitchen_sink_kitchen_ReadyListener_vtable; -typedef void (*kitchen_sink_kitchen_do_async_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*kitchen_sink_kitchen_do_cancellable_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: shared -/** Trivial cross-module helper */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_shared_ping(size_t* out_len, kitchen_sink_error* out_err); - -// Module: kitchen -/** A pocket-sized interface exercising the object surface */ -/** Create a gadget with the given id */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_new(int64_t id, kitchen_sink_error* out_err); -/** The gadget subsystem version string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_version(size_t* out_len, kitchen_sink_error* out_err); -/** Render the gadget as a human-readable string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_describe(const kitchen_sink_kitchen_Gadget* self, size_t* out_len, kitchen_sink_error* out_err); -/** Poke the gadget a number of times, failing on a negative count */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_Gadget_poke(const kitchen_sink_kitchen_Gadget* self, int32_t times, kitchen_sink_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_clone(const kitchen_sink_kitchen_Gadget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_kitchen_Gadget_destroy(kitchen_sink_kitchen_Gadget* self); - -/** Identity for bool */ -KITCHEN_SINK_API bool kitchen_sink_kitchen_bool_id(bool x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_i32_id(int32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API uint32_t kitchen_sink_kitchen_u32_id(uint32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int64_t kitchen_sink_kitchen_i64_id(int64_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API double kitchen_sink_kitchen_f64_id(double x, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_string(const uint8_t* s_ptr, size_t s_len, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_bytes(const uint8_t* b_ptr, size_t b_len, size_t* out_len, kitchen_sink_error* out_err); -/** Register a listener, returning how many are attached */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); -/** Optional object in both directions */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_maybe_gadget(const kitchen_sink_kitchen_Gadget* fallback, kitchen_sink_error* out_err); -/** A list of objects */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_gadgets(size_t* out_len, kitchen_sink_error* out_err); -/** An iterator over objects */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamGadgetsIterator* kitchen_sink_kitchen_stream_gadgets(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamGadgetsIterator_next(kitchen_sink_kitchen_StreamGadgetsIterator* iter, kitchen_sink_kitchen_Gadget** out_item, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamGadgetsIterator_destroy(kitchen_sink_kitchen_StreamGadgetsIterator* iter); -/** Optional struct return */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_maybe_item(int64_t id, size_t* out_len, kitchen_sink_error* out_err); -/** List of structs */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_list_items(size_t* out_len, kitchen_sink_error* out_err); -/** Map return type */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_get_attrs(size_t* out_len, kitchen_sink_error* out_err); -/** Iterator return type */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamItemsIterator* kitchen_sink_kitchen_stream_items(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamItemsIterator_next(kitchen_sink_kitchen_StreamItemsIterator* iter, const uint8_t** out_item, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamItemsIterator_destroy(kitchen_sink_kitchen_StreamItemsIterator* iter); -/** Returns the shared Token type from another module */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_cross_module_token(size_t* out_len, kitchen_sink_error* out_err); -/** Async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_async(const uint8_t* input_ptr, size_t input_len, kitchen_sink_kitchen_do_async_callback callback, void* context); -/** Cancellable async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_cancellable(const uint8_t* input_ptr, size_t input_len, kitchen_sink_cancel_token* cancel_token, kitchen_sink_kitchen_do_cancellable_callback callback, void* context); -/** Legacy operation kept for compatibility */ -KITCHEN_SINK_DEPRECATED("Use new_op instead") -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_legacy_op(kitchen_sink_error* out_err); -/** Replacement for legacy_op */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_new_op(kitchen_sink_error* out_err); - -// Module: kitchen_nested -/** Trivial nested-module function */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_nested_hello(size_t* out_len, kitchen_sink_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // KITCHEN_SINK_H - -// END kitchen_sink.h diff --git a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__runtime_go.snap b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__runtime_go.snap index 1bebbcb4..8b78a350 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__runtime_go.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_kitchen_sink__runtime_go.snap @@ -1,44 +1,49 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink /* #include #include "kitchen_sink.h" + +static void* wvRuntimeHandlePtr(uintptr_t h) { return (void*)h; } */ import "C" import ( + "cmp" "context" + "errors" "fmt" "runtime/cgo" + "slices" "sync/atomic" "unsafe" ) // wvABIVersion is the C ABI revision these bindings were generated against. -const wvABIVersion uint32 = 3 +const wvABIVersion uint32 = 4 // The runtime error codes this package produces or interprets itself. const ( - wvCodeMarshal int32 = -3 wvCodeForeign int32 = -4 wvCodeCancelled int32 = -5 ) -// Error reports a failure that no typed error domain claims: an unknown -// code, a generic failure (code -1), a producer panic (-2), a marshalling -// failure (-3), or a Go callback-interface implementation that panicked -// (-4). The code is preserved so callers can tell them apart. +// Error is a failure no error domain claims: a generic failure (code -1), a +// panic in the native library (-2), an argument or result that couldn't be +// marshalled (-3), a callback implementation that failed (-4), or a code +// outside the domain. A call that doesn't declare errors panics with an +// *Error, since its failure is a bug; a call that does returns one. type Error struct { - // Code is the numeric ABI error code. + // Code is the runtime code. Code int32 - // Message is the human-readable error message. + // Message is the native library's message. Message string } @@ -47,27 +52,46 @@ func (e *Error) Error() string { } // DebugLive reports how many resources of one kind the native library holds: -// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned -// allocations. It's always 0 unless the library was built with leak checks -// enabled; tests use it to assert that nothing leaked. +// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 byte runs. Kind -1 +// reports 1 when the library counts at all. Every count is 0 unless the +// library was built with leak checks enabled; tests use it to assert that +// nothing leaked. func DebugLive(kind int32) uint64 { return uint64(C.kitchen_sink_debug_live(C.int32_t(kind))) } +// ── Load-time checks ── + // wvCheckABI panics when the linked library implements a different C ABI // revision than these bindings were generated for. func wvCheckABI() { if found := uint32(C.kitchen_sink_abi_version()); found != wvABIVersion { - panic(fmt.Sprintf("kitchen_sink: ABI mismatch: these bindings expect revision %d but the linked library reports revision %d", wvABIVersion, found)) + panic(fmt.Errorf("kitchen_sink: the linked library implements C ABI revision %d, but these bindings were generated for revision %d", found, wvABIVersion)) } } -// wvCheckModule panics when the linked library's contract checksum for a -// top-level module differs from the one these bindings were generated -// against, which means the bindings are stale. -func wvCheckModule(module string, want, found uint64) { - if found != want { - panic(fmt.Sprintf("kitchen_sink: module %q doesn't match the linked library (bindings checksum %#016x, library checksum %#016x); regenerate the bindings", module, want, found)) +// wvContractEntry is one declaration these bindings were generated with: +// the FNV-1a hashes of its dotted path and of its signature. +type wvContractEntry struct { + id, hash uint64 + path string +} + +// wvCheckContract panics unless every entry in want is in the library's +// contract table for one top-level module (n entries sorted by id) with an +// equal signature hash. Entries only the library has are fine. +func wvCheckContract(table *C.kitchen_sink_contract_entry, n C.size_t, want []wvContractEntry) { + have := unsafe.Slice(table, int(n)) + for _, w := range want { + i, found := slices.BinarySearchFunc(have, w.id, func(e C.kitchen_sink_contract_entry, id uint64) int { + return cmp.Compare(uint64(e.id), id) + }) + if !found { + panic(fmt.Errorf("kitchen_sink: %s is missing from the library", w.path)) + } + if uint64(have[i].hash) != w.hash { + panic(fmt.Errorf("kitchen_sink: %s changed since these bindings were generated", w.path)) + } } } @@ -111,23 +135,85 @@ func wvTakeBoxedError(cErr *C.kitchen_sink_error) wvFailure { return f } -// wvTrap panics with the generic *Error when the slot reports a failure. -// Wrappers of functions that declare no error domain check their slot with -// it: a failure there can only be a producer bug, a marshalling failure, or -// a callback-interface implementation that panicked. +// wvTrap panics with an *Error when the slot reports a failure. Wrappers of +// calls that declare no errors check their slot with it: a failure there is +// a bug in the native library, an argument it couldn't take, or a callback +// failure it let through. func wvTrap(cErr *C.kitchen_sink_error) { if cErr.code != 0 { panic(wvTakeError(cErr).err()) } } -// wvForeignError reports a panic recovered inside a callback-interface -// trampoline through the producer's error slot (code -4). The producer -// copies the message, so it's only borrowed for the call. -func wvForeignError(outErr *C.kitchen_sink_error, recovered any) { - msg := C.CString(fmt.Sprint(recovered)) +// wvDomainError is implemented by every error-code type: its code and its +// fields encoded as a value buffer (nil when it has none). +type wvDomainError interface { + error + Code() int32 + wvPayload() []byte +} + +// wvSetError reports a failure through a callback's error slot. The native +// library copies the message and payload, so both are only borrowed. +func wvSetError(outErr *C.kitchen_sink_error, code int32, message string, payload []byte) { + msg := C.CString(message) defer C.free(unsafe.Pointer(msg)) - C.kitchen_sink_error_set(outErr, C.int32_t(wvCodeForeign), msg) + C.kitchen_sink_error_set(outErr, C.int32_t(code), msg) + if len(payload) > 0 { + ptr, n := wvBytes(payload) + C.kitchen_sink_error_set_payload(outErr, ptr, n) + } +} + +// wvCallbackFailed reports the error a throwing callback method returned: +// an error of the domain D in scope for the method reports its code and +// payload, and anything else is a callback failure (code -4). +func wvCallbackFailed[D error](outErr *C.kitchen_sink_error, err error) { + var domain D + if errors.As(err, &domain) { + if d, ok := any(domain).(wvDomainError); ok { + wvSetError(outErr, d.Code(), d.Error(), d.wvPayload()) + return + } + } + wvSetError(outErr, wvCodeForeign, err.Error(), nil) +} + +// wvRecoverCallback, deferred by every callback trampoline, reports a panic +// in the implementation as a callback failure (code -4) instead of letting +// it unwind into the native caller. +func wvRecoverCallback(outErr *C.kitchen_sink_error) { + if r := recover(); r != nil { + msg := fmt.Sprint(r) + if err, ok := r.(error); ok { + msg = err.Error() + } + wvSetError(outErr, wvCodeForeign, msg, nil) + } +} + +// ── Callback contexts ── + +// wvNewCallback keeps impl alive in a handle table for as long as the +// native library holds it and returns the handle as the callback's context +// pointer; the vtable's free entry deletes it (see wvFreeCallback). A nil +// implementation of a required callback panics. +func wvNewCallback(impl any) unsafe.Pointer { + if impl == nil { + panic("kitchen_sink: nil callback implementation") + } + return C.wvRuntimeHandlePtr(C.uintptr_t(cgo.NewHandle(impl))) +} + +// wvCallback recovers the implementation behind a callback context. +func wvCallback[T any](ctx unsafe.Pointer) T { + return cgo.Handle(uintptr(ctx)).Value().(T) +} + +// wvFreeCallback releases the implementation behind a callback context once +// the native library is done with it. +func wvFreeCallback(ctx unsafe.Pointer) { + cgo.Handle(uintptr(ctx)).Delete() } // ── Strings and bytes ── @@ -153,36 +239,53 @@ func wvBorrowString(ptr *C.uint8_t, n C.size_t) string { } // wvTakeString copies a returned UTF-8 run into a Go string and releases the -// producer's allocation. +// native allocation. func wvTakeString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } s := wvBorrowString(ptr, n) C.kitchen_sink_free_bytes(ptr, n) return s } // wvBorrowBytes copies a borrowed byte run into Go memory. A null pointer -// copies to nil. +// copies to an empty slice. func wvBorrowBytes(ptr *C.uint8_t, n C.size_t) []byte { if ptr == nil { - return nil + return []byte{} } return append([]byte{}, unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)...) } -// wvTakeBytes copies a returned byte run (raw bytes or a value buffer) into -// Go memory and releases the producer's allocation. +// wvTakeBytes copies a returned byte run into Go memory and releases the +// native allocation. func wvTakeBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } b := wvBorrowBytes(ptr, n) C.kitchen_sink_free_bytes(ptr, n) return b } +// wvHandOverString copies s into a run allocated with the native library's +// allocator and writes it to a callback's return slots; the library adopts +// the run. +func wvHandOverString(s string, outPtr **C.uint8_t, outLen *C.size_t) { + ptr, n := wvStr(s) + wvHandOver(ptr, n, outPtr, outLen) +} + +// wvHandOverBytes is wvHandOverString for a byte slice (raw bytes or an +// encoded value buffer). +func wvHandOverBytes(b []byte, outPtr **C.uint8_t, outLen *C.size_t) { + ptr, n := wvBytes(b) + wvHandOver(ptr, n, outPtr, outLen) +} + +func wvHandOver(ptr *C.uint8_t, n C.size_t, outPtr **C.uint8_t, outLen *C.size_t) { + run := C.kitchen_sink_alloc(n) + if n > 0 { + copy(unsafe.Slice((*byte)(unsafe.Pointer(run)), n), unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) + } + *outPtr, *outLen = run, n +} + // ── Objects ── // wvRef owns one strong reference to a native object on behalf of a wrapper. @@ -243,17 +346,35 @@ func (r *wvRef) close() { // ── Async ── // wvOutcome is one async completion: a converted result, or the failure the -// producer reported. +// native library reported. type wvOutcome[T any] struct { val T fail *wvFailure } +// wvAsyncCall is one pending async call: the channel its completion is +// delivered on, kept alive in a handle table while the call is in flight. +type wvAsyncCall[T any] struct { + done chan wvOutcome[T] + handle cgo.Handle +} + +func wvNewAsyncCall[T any]() *wvAsyncCall[T] { + c := &wvAsyncCall[T]{done: make(chan wvOutcome[T], 1)} + c.handle = cgo.NewHandle(c.done) + return c +} + +// context is the completion's context pointer. +func (c *wvAsyncCall[T]) context() unsafe.Pointer { + return C.wvRuntimeHandlePtr(C.uintptr_t(c.handle)) +} + // wvComplete delivers one async completion to the waiting wrapper. It runs on -// a producer thread inside an exported trampoline, so it never panics: a -// result that fails to decode is delivered as a marshalling failure. -func wvComplete[T any](handle unsafe.Pointer, cErr *C.kitchen_sink_error, convert func() T) { - h := cgo.Handle(uintptr(handle)) +// a native thread inside an exported trampoline, so it never panics: a result +// that fails to decode is delivered as a marshalling failure. +func wvComplete[T any](context unsafe.Pointer, cErr *C.kitchen_sink_error, convert func() T) { + h := cgo.Handle(uintptr(context)) done := h.Value().(chan wvOutcome[T]) h.Delete() done <- wvSettle(cErr, convert) @@ -268,29 +389,29 @@ func wvSettle[T any](cErr *C.kitchen_sink_error, convert func() T) (o wvOutcome[ } defer func() { if r := recover(); r != nil { - o = wvOutcome[T]{fail: &wvFailure{code: wvCodeMarshal, message: fmt.Sprint(r)}} + o = wvOutcome[T]{fail: &wvFailure{code: -3, message: fmt.Sprint(r)}} } }() o.val = convert() return o } -// wvAwait blocks until an async call completes or ctx is done. With a cancel -// token, ctx's cancellation cancels the native call and waits for its -// completion, which the producer delivers promptly; without one, the call is +// wait blocks until the call completes or ctx is done. With a cancel token, +// ctx's cancellation cancels the native call and waits for its completion, +// which the native library delivers promptly; without one, the call is // abandoned and ctx.Err() returned at once. A cancelled completion (code -5) // returns ctx.Err(), or context.Canceled when ctx itself isn't done. -func wvAwait[T any](ctx context.Context, done chan wvOutcome[T], token *C.kitchen_sink_cancel_token) (T, *wvFailure, error) { +func (c *wvAsyncCall[T]) wait(ctx context.Context, token *C.kitchen_sink_cancel_token) (T, *wvFailure, error) { var o wvOutcome[T] select { - case o = <-done: + case o = <-c.done: case <-ctx.Done(): if token == nil { var zero T return zero, nil, ctx.Err() } C.kitchen_sink_cancel_token_cancel(token) - o = <-done + o = <-c.done } if o.fail != nil && o.fail.code == wvCodeCancelled { var zero T diff --git a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/go_nested_modules__README_md.snap deleted file mode 100644 index 0a167f95..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__README_md.snap +++ /dev/null @@ -1,49 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# nested_modules (Go) - -cgo bindings for the `nested_modules` native library. - -## Build - -The module ships its own copy of the C header (`nested_modules.h`) and links with -`-lnested_modules`. Point the linker at the directory holding `libnested_modules.so`, -`libnested_modules.dylib`, or `nested_modules.dll`, and make the library findable at -run time: - -```sh -export CGO_LDFLAGS="-L/path/to/lib" -export LD_LIBRARY_PATH="/path/to/lib" # DYLD_LIBRARY_PATH on macOS -go build ./... -``` - -Depend on the module from your own project with a `require` (and, for a -local checkout, a `replace`) of `nested_modules`, then import it: - -```go -import "nested_modules" -``` - -Go 1.23 or newer is required. - -## Usage notes - -- Records are plain structs and rich enums are sealed interfaces; both cross - the boundary by value. -- Interfaces are reference-counted objects. Call `Close` when you're done - with one (a finalizer releases it otherwise); it's safe to call twice or - while a call is in flight. -- Async functions take a `context.Context` first. Cancellable ones cancel - the native call when the context is done. -- Callback interfaces are Go interfaces you implement. The library may call - them from any thread, and a panic is reported to the native caller as a - foreign error instead of crashing. -- Loading checks the library's ABI revision and contract checksums and - panics with the stale module's name on a mismatch. - - diff --git a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__bindings_go.snap b/crates/weaveffi-cli/tests/snapshots/go_nested_modules__bindings_go.snap index a20a9b6d..38d54ed4 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__bindings_go.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_nested_modules__bindings_go.snap @@ -1,10 +1,11 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate +// Package nested_modules binds the nested_modules native library. package nested_modules /* @@ -13,48 +14,70 @@ package nested_modules */ import "C" -import ( - "fmt" -) - func init() { wvCheckABI() - wvCheckModule("graphics", 0x44d31b5b10fdb90d, uint64(C.nested_modules_graphics_checksum())) + var n C.size_t + table := C.nested_modules_graphics_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0x2c29eef9327a6bd4, 0x12983d5271651389, "graphics.Unit"}, + {0x304088bbc28f0015, 0x97e6e809913be6b6, "graphics.new_canvas"}, + {0x49f8837532df9ac6, 0x32e5b27e81ee0957, "graphics.Canvas"}, + {0x55717fa027336b47, 0xb2bd3a6bdcce48c8, "graphics.shapes.describe"}, + {0x7626b91b8f1f53cd, 0x867bbf7cf0f41c74, "graphics.shapes.geometry.compute"}, + {0xb3efcb50d9a61ab7, 0x3cb0207dfad2a149, "graphics.GraphicsError"}, + {0xc62b16a9591b744b, 0x7895610c8487eb1b, "graphics.shapes.Shape"}, + {0xe13001f4983ccd35, 0x16e7de070b4a4cfc, "graphics.shapes.geometry.Detail"}, + {0xf3f658bcfc5dba62, 0x2665ccb47356e746, "graphics.shapes.make_shape"}, + }) +} + +// GraphicsError is an error the graphics module reports: *OutOfBoundsError. +// +// Match one code with errors.As and its type, or any of them with a +// GraphicsError. +type GraphicsError interface { + error + // Code returns the error's numeric code. + Code() int32 + isGraphicsError() +} + +// OutOfBoundsError is the GraphicsError code OutOfBounds (1): coordinate out +// of bounds. +type OutOfBoundsError struct { + // Message describes the failure; when it's empty, Error returns the + // code's default message, "coordinate out of bounds". + Message string } -// GraphicsError is a typed error reported by the `graphics` module. -type GraphicsError struct { - // Code is the numeric ABI error code (one of the GraphicsError constants). - Code int32 - // Message is the human-readable error message. - Message string +// Error returns the error's message. +func (e *OutOfBoundsError) Error() string { + if e.Message == "" { + return "coordinate out of bounds" + } + return e.Message } -func (e *GraphicsError) Error() string { - return fmt.Sprintf("graphics: %s (code %d)", e.Message, e.Code) +// Code returns 1. +func (*OutOfBoundsError) Code() int32 { + return 1 } -// GraphicsError codes. -const ( - // GraphicsErrorOutOfBounds coordinate out of bounds - GraphicsErrorOutOfBounds int32 = 1 -) +func (*OutOfBoundsError) isGraphicsError() {} -// wvMapGraphics converts a non-zero code from the `graphics` domain into a -// *GraphicsError, falling back to the generic *Error for unknown codes. -func wvMapGraphics(f wvFailure) error { +func (*OutOfBoundsError) wvPayload() []byte { + return nil +} + +func wvGraphicsError(f wvFailure) error { switch f.code { - case GraphicsErrorOutOfBounds: - if f.message == "" { - f.message = "coordinate out of bounds" - } - return &GraphicsError{Code: f.code, Message: f.message} - default: - return f.err() + case 1: + return &OutOfBoundsError{Message: f.message} } + return f.err() } -// Unit: A measurement unit shared with nested submodules. +// Unit is a measurement unit shared with nested submodules. type Unit int32 const ( @@ -62,6 +85,14 @@ const ( UnitPoints Unit = 1 ) +func wvWriteUnit(w *wvWriter, v Unit) { + w.writeI32(int32(v)) +} + +func wvReadUnit(r *wvReader) Unit { + return Unit(r.readI32()) +} + // Canvas: Top-level struct that references a struct defined one level deeper. type Canvas struct { Width int32 @@ -69,17 +100,15 @@ type Canvas struct { Root Shape } -// wvPackCanvas appends v to w in the value-buffer wire format. -func wvPackCanvas(w *wvWriter, v Canvas) { +func wvWriteCanvas(w *wvWriter, v Canvas) { w.writeI32(v.Width) - wvPackShape(w, v.Root) + wvWriteShape(w, v.Root) } -// wvUnpackCanvas decodes one Canvas from r. -func wvUnpackCanvas(r *wvReader) Canvas { +func wvReadCanvas(r *wvReader) Canvas { var v Canvas v.Width = r.readI32() - v.Root = wvUnpackShape(r) + v.Root = wvReadShape(r) return v } @@ -89,14 +118,10 @@ func NewCanvas(width int32) Canvas { var cErr C.nested_modules_error cRet := C.nested_modules_graphics_new_canvas(C.int32_t(width), &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Canvas - ret = wvUnpackCanvas(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadCanvas) } -// Shape: A struct one level deep that references both an ancestor enum and a deeper struct. +// Shape is a struct one level deep that references both an ancestor enum and a deeper struct. type Shape struct { Sides int32 // Unit: Reference up to the parent module's enum @@ -105,19 +130,17 @@ type Shape struct { Detail Detail } -// wvPackShape appends v to w in the value-buffer wire format. -func wvPackShape(w *wvWriter, v Shape) { +func wvWriteShape(w *wvWriter, v Shape) { w.writeI32(v.Sides) - w.writeI32(int32(v.Unit)) - wvPackDetail(w, v.Detail) + wvWriteUnit(w, v.Unit) + wvWriteDetail(w, v.Detail) } -// wvUnpackShape decodes one Shape from r. -func wvUnpackShape(r *wvReader) Shape { +func wvReadShape(r *wvReader) Shape { var v Shape v.Sides = r.readI32() - v.Unit = Unit(r.readI32()) - v.Detail = wvUnpackDetail(r) + v.Unit = wvReadUnit(r) + v.Detail = wvReadDetail(r) return v } @@ -127,11 +150,7 @@ func MakeShape(sides int32) Shape { var cErr C.nested_modules_error cRet := C.nested_modules_graphics_shapes_make_shape(C.int32_t(sides), &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Shape - ret = wvUnpackShape(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadShape) } // Describe: Nested function returning a struct owned by an ancestor module. @@ -140,25 +159,19 @@ func Describe() Canvas { var cErr C.nested_modules_error cRet := C.nested_modules_graphics_shapes_describe(&cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Canvas - ret = wvUnpackCanvas(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadCanvas) } -// Detail: A struct two levels deep. +// Detail is a struct two levels deep. type Detail struct { Area float64 } -// wvPackDetail appends v to w in the value-buffer wire format. -func wvPackDetail(w *wvWriter, v Detail) { +func wvWriteDetail(w *wvWriter, v Detail) { w.writeF64(v.Area) } -// wvUnpackDetail decodes one Detail from r. -func wvUnpackDetail(r *wvReader) Detail { +func wvReadDetail(r *wvReader) Detail { var v Detail v.Area = r.readF64() return v @@ -166,20 +179,14 @@ func wvUnpackDetail(r *wvReader) Detail { // Compute: Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. func Compute(shape Shape) (Detail, error) { - wShape := &wvWriter{} - wvPackShape(wShape, shape) - cShapePtr, cShapeLen := wvBytes(wShape.buf) + cShapePtr, cShapeLen := wvBytes(wvEncode(shape, wvWriteShape)) var cRetLen C.size_t var cErr C.nested_modules_error cRet := C.nested_modules_graphics_shapes_geometry_compute(cShapePtr, cShapeLen, &cRetLen, &cErr) if cErr.code != 0 { - return Detail{}, wvMapGraphics(wvTakeError(&cErr)) + return Detail{}, wvGraphicsError(wvTakeError(&cErr)) } - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Detail - ret = wvUnpackDetail(rRet) - rRet.expectEnd() - return ret, nil + return wvDecode(cRet, cRetLen, wvReadDetail), nil } // END bindings.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__codec_go.snap b/crates/weaveffi-cli/tests/snapshots/go_nested_modules__codec_go.snap deleted file mode 100644 index a9534aec..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__codec_go.snap +++ /dev/null @@ -1,205 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package nested_modules - -import ( - "encoding/binary" - "math" - "unicode/utf8" -) - -// wvWriter serializes values into the value-buffer format: little-endian, -// packed, u32 length prefixes. -type wvWriter struct { - buf []byte -} - -func (w *wvWriter) writeBool(v bool) { - if v { - w.buf = append(w.buf, 1) - } else { - w.buf = append(w.buf, 0) - } -} - -func (w *wvWriter) writeI8(v int8) { - w.buf = append(w.buf, byte(v)) -} - -func (w *wvWriter) writeU8(v uint8) { - w.buf = append(w.buf, v) -} - -func (w *wvWriter) writeI16(v int16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, uint16(v)) -} - -func (w *wvWriter) writeU16(v uint16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, v) -} - -func (w *wvWriter) writeI32(v int32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, uint32(v)) -} - -func (w *wvWriter) writeU32(v uint32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, v) -} - -func (w *wvWriter) writeI64(v int64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, uint64(v)) -} - -func (w *wvWriter) writeU64(v uint64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, v) -} - -func (w *wvWriter) writeF32(v float32) { - w.writeU32(math.Float32bits(v)) -} - -func (w *wvWriter) writeF64(v float64) { - w.writeU64(math.Float64bits(v)) -} - -func (w *wvWriter) writeLen(n int) { - if n < 0 || uint64(n) > math.MaxUint32 { - panic("nested_modules: value-buffer length exceeds u32 range") - } - w.writeU32(uint32(n)) -} - -func (w *wvWriter) writeString(v string) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeBytes(v []byte) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeOptionFlag(present bool) { - w.writeBool(present) -} - -// wvReader decodes values from the value-buffer format. A malformed buffer -// is a producer/consumer contract violation, so every read panics instead of -// returning a typed domain error. -type wvReader struct { - buf []byte - pos int -} - -func wvMalformed(context string) { - panic("nested_modules: malformed value buffer: " + context) -} - -func (r *wvReader) take(n int, context string) []byte { - if n < 0 || len(r.buf)-r.pos < n { - wvMalformed(context) - } - b := r.buf[r.pos : r.pos+n] - r.pos += n - return b -} - -func (r *wvReader) readBool() bool { - switch r.take(1, "bool")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("bool byte out of range") - return false -} - -func (r *wvReader) readI8() int8 { - return int8(r.take(1, "i8")[0]) -} - -func (r *wvReader) readU8() uint8 { - return r.take(1, "u8")[0] -} - -func (r *wvReader) readI16() int16 { - return int16(binary.LittleEndian.Uint16(r.take(2, "i16"))) -} - -func (r *wvReader) readU16() uint16 { - return binary.LittleEndian.Uint16(r.take(2, "u16")) -} - -func (r *wvReader) readI32() int32 { - return int32(binary.LittleEndian.Uint32(r.take(4, "i32"))) -} - -func (r *wvReader) readU32() uint32 { - return binary.LittleEndian.Uint32(r.take(4, "u32")) -} - -func (r *wvReader) readI64() int64 { - return int64(binary.LittleEndian.Uint64(r.take(8, "i64"))) -} - -func (r *wvReader) readU64() uint64 { - return binary.LittleEndian.Uint64(r.take(8, "u64")) -} - -func (r *wvReader) readF32() float32 { - return math.Float32frombits(r.readU32()) -} - -func (r *wvReader) readF64() float64 { - return math.Float64frombits(r.readU64()) -} - -// readLen reads a u32 element count. Elements may encode to zero bytes, so -// the count isn't checked against the remaining buffer; callers cap their -// preallocation with capHint instead. -func (r *wvReader) readLen() int { - return int(r.readU32()) -} - -// capHint bounds a preallocation for n decoded elements by the bytes left in -// the buffer, so a corrupt count can't force a huge allocation. -func (r *wvReader) capHint(n int) int { - return min(n, len(r.buf)-r.pos) -} - -func (r *wvReader) readString() string { - b := r.take(r.readLen(), "string bytes") - if !utf8.Valid(b) { - wvMalformed("string is not valid UTF-8") - } - return string(b) -} - -func (r *wvReader) readBytes() []byte { - return append([]byte{}, r.take(r.readLen(), "byte buffer")...) -} - -func (r *wvReader) readOptionFlag() bool { - switch r.take(1, "option flag")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("option flag byte out of range") - return false -} - -func (r *wvReader) expectEnd() { - if r.pos != len(r.buf) { - wvMalformed("trailing bytes after value") - } -} - -// END codec.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__go_mod.snap b/crates/weaveffi-cli/tests/snapshots/go_nested_modules__go_mod.snap deleted file mode 100644 index d61da40e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__go_mod.snap +++ /dev/null @@ -1,12 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module nested_modules - -go 1.23 - -// END go.mod diff --git a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__nested_modules_h.snap b/crates/weaveffi-cli/tests/snapshots/go_nested_modules__nested_modules_h.snap deleted file mode 100644 index 27c97f2b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__nested_modules_h.snap +++ /dev/null @@ -1,144 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef NESTED_MODULES_H -#define NESTED_MODULES_H - -#include -#include -#include - -#ifndef NESTED_MODULES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef NESTED_MODULES_BUILD -# define NESTED_MODULES_API __declspec(dllexport) -# else -# define NESTED_MODULES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define NESTED_MODULES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define NESTED_MODULES_API __attribute__((visibility("default"))) -# else -# define NESTED_MODULES_API -# endif -#endif - -#ifndef NESTED_MODULES_DEPRECATED -# if defined(_MSC_VER) -# define NESTED_MODULES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define NESTED_MODULES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define NESTED_MODULES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports nested_modules_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define NESTED_MODULES_ABI_VERSION 3u -NESTED_MODULES_API uint32_t nested_modules_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by nested_modules_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct nested_modules_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} nested_modules_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -NESTED_MODULES_API void nested_modules_error_set(nested_modules_error* err, int32_t code, const char* message); -NESTED_MODULES_API void nested_modules_error_clear(nested_modules_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - nested_modules_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -NESTED_MODULES_API void nested_modules_error_free(nested_modules_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -NESTED_MODULES_API void nested_modules_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct nested_modules_cancel_token nested_modules_cancel_token; -NESTED_MODULES_API nested_modules_cancel_token* nested_modules_cancel_token_create(void); -NESTED_MODULES_API void nested_modules_cancel_token_cancel(nested_modules_cancel_token* token); -NESTED_MODULES_API bool nested_modules_cancel_token_is_cancelled(const nested_modules_cancel_token* token); -NESTED_MODULES_API void nested_modules_cancel_token_destroy(nested_modules_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -NESTED_MODULES_API uint8_t* nested_modules_alloc(uint32_t size); -NESTED_MODULES_API void nested_modules_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -NESTED_MODULES_API uint64_t nested_modules_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define NESTED_MODULES_GRAPHICS_CHECKSUM 0x44d31b5b10fdb90dull -NESTED_MODULES_API uint64_t nested_modules_graphics_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with nested_modules_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * nested_modules_buffer.h declares a C struct and codec for each. - */ - -/** A measurement unit shared with nested submodules. */ -typedef enum { nested_modules_graphics_Unit_pixels = 0, nested_modules_graphics_Unit_points = 1 } nested_modules_graphics_Unit; -/** Error codes reported by throwing functions in the `graphics` module tree. */ -typedef enum { nested_modules_graphics_GraphicsError_OutOfBounds = 1 } nested_modules_graphics_GraphicsError; - -// Module: graphics -/** Construct a Canvas. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_new_canvas(int32_t width, size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes -/** Construct a Shape. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_make_shape(int32_t sides, size_t* out_len, nested_modules_error* out_err); -/** Nested function returning a struct owned by an ancestor module. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_describe(size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes_geometry -/** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_geometry_compute(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, nested_modules_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // NESTED_MODULES_H - -// END nested_modules.h diff --git a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__runtime_go.snap b/crates/weaveffi-cli/tests/snapshots/go_nested_modules__runtime_go.snap deleted file mode 100644 index 3a7876cf..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_nested_modules__runtime_go.snap +++ /dev/null @@ -1,305 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package nested_modules - -/* -#include -#include "nested_modules.h" -*/ -import "C" - -import ( - "context" - "fmt" - "runtime/cgo" - "sync/atomic" - "unsafe" -) - -// wvABIVersion is the C ABI revision these bindings were generated against. -const wvABIVersion uint32 = 3 - -// The runtime error codes this package produces or interprets itself. -const ( - wvCodeMarshal int32 = -3 - wvCodeForeign int32 = -4 - wvCodeCancelled int32 = -5 -) - -// Error reports a failure that no typed error domain claims: an unknown -// code, a generic failure (code -1), a producer panic (-2), a marshalling -// failure (-3), or a Go callback-interface implementation that panicked -// (-4). The code is preserved so callers can tell them apart. -type Error struct { - // Code is the numeric ABI error code. - Code int32 - // Message is the human-readable error message. - Message string -} - -func (e *Error) Error() string { - return fmt.Sprintf("nested_modules: %s (code %d)", e.Message, e.Code) -} - -// DebugLive reports how many resources of one kind the native library holds: -// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned -// allocations. It's always 0 unless the library was built with leak checks -// enabled; tests use it to assert that nothing leaked. -func DebugLive(kind int32) uint64 { - return uint64(C.nested_modules_debug_live(C.int32_t(kind))) -} - -// wvCheckABI panics when the linked library implements a different C ABI -// revision than these bindings were generated for. -func wvCheckABI() { - if found := uint32(C.nested_modules_abi_version()); found != wvABIVersion { - panic(fmt.Sprintf("nested_modules: ABI mismatch: these bindings expect revision %d but the linked library reports revision %d", wvABIVersion, found)) - } -} - -// wvCheckModule panics when the linked library's contract checksum for a -// top-level module differs from the one these bindings were generated -// against, which means the bindings are stale. -func wvCheckModule(module string, want, found uint64) { - if found != want { - panic(fmt.Sprintf("nested_modules: module %q doesn't match the linked library (bindings checksum %#016x, library checksum %#016x); regenerate the bindings", module, want, found)) - } -} - -// ── Errors ── - -// wvFailure is a non-zero error slot copied into Go memory. -type wvFailure struct { - code int32 - message string - payload []byte -} - -// err converts the failure into the generic *Error. -func (f wvFailure) err() error { - return &Error{Code: f.code, Message: f.message} -} - -func wvCopyFailure(cErr *C.nested_modules_error) wvFailure { - f := wvFailure{code: int32(cErr.code)} - if cErr.message != nil { - f.message = C.GoString(cErr.message) - } - if cErr.payload_ptr != nil { - f.payload = wvBorrowBytes(cErr.payload_ptr, cErr.payload_len) - } - return f -} - -// wvTakeError copies a non-zero error slot and clears it. -func wvTakeError(cErr *C.nested_modules_error) wvFailure { - f := wvCopyFailure(cErr) - C.nested_modules_error_clear(cErr) - return f -} - -// wvTakeBoxedError copies the heap-boxed error an async completion receives -// and frees the box. -func wvTakeBoxedError(cErr *C.nested_modules_error) wvFailure { - f := wvCopyFailure(cErr) - C.nested_modules_error_free(cErr) - return f -} - -// wvTrap panics with the generic *Error when the slot reports a failure. -// Wrappers of functions that declare no error domain check their slot with -// it: a failure there can only be a producer bug, a marshalling failure, or -// a callback-interface implementation that panicked. -func wvTrap(cErr *C.nested_modules_error) { - if cErr.code != 0 { - panic(wvTakeError(cErr).err()) - } -} - -// wvForeignError reports a panic recovered inside a callback-interface -// trampoline through the producer's error slot (code -4). The producer -// copies the message, so it's only borrowed for the call. -func wvForeignError(outErr *C.nested_modules_error, recovered any) { - msg := C.CString(fmt.Sprint(recovered)) - defer C.free(unsafe.Pointer(msg)) - C.nested_modules_error_set(outErr, C.int32_t(wvCodeForeign), msg) -} - -// ── Strings and bytes ── - -// wvStr returns a borrowed (pointer, length) view of s's UTF-8 bytes for the -// duration of one call. Strings may contain NUL bytes. -func wvStr(s string) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.StringData(s))), C.size_t(len(s)) -} - -// wvBytes returns a borrowed (pointer, length) view of b for the duration of -// one call. -func wvBytes(b []byte) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.SliceData(b))), C.size_t(len(b)) -} - -// wvBorrowString copies a borrowed UTF-8 run into a Go string. -func wvBorrowString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - return string(unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) -} - -// wvTakeString copies a returned UTF-8 run into a Go string and releases the -// producer's allocation. -func wvTakeString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - s := wvBorrowString(ptr, n) - C.nested_modules_free_bytes(ptr, n) - return s -} - -// wvBorrowBytes copies a borrowed byte run into Go memory. A null pointer -// copies to nil. -func wvBorrowBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - return append([]byte{}, unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)...) -} - -// wvTakeBytes copies a returned byte run (raw bytes or a value buffer) into -// Go memory and releases the producer's allocation. -func wvTakeBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - b := wvBorrowBytes(ptr, n) - C.nested_modules_free_bytes(ptr, n) - return b -} - -// ── Objects ── - -// wvRef owns one strong reference to a native object on behalf of a wrapper. -// Calls borrow the pointer between acquire and release. Close may race them: -// the reference is destroyed exactly once, by close when no call is in -// flight, or else by the last in-flight call to finish. -type wvRef struct { - ptr unsafe.Pointer - destroy func(unsafe.Pointer) - // state counts in-flight calls in its upper bits; the low bit is set - // once close has run. - state atomic.Uint64 -} - -func (r *wvRef) init(ptr unsafe.Pointer, destroy func(unsafe.Pointer)) { - r.ptr = ptr - r.destroy = destroy -} - -// acquire borrows the pointer for one call, panicking when the wrapper was -// already closed. -func (r *wvRef) acquire(typeName string) unsafe.Pointer { - for { - s := r.state.Load() - if s&1 != 0 { - panic("nested_modules: " + typeName + " used after Close") - } - if r.state.CompareAndSwap(s, s+2) { - return r.ptr - } - } -} - -// release ends one call's borrow. -func (r *wvRef) release() { - if r.state.Add(^uint64(1)) == 1 { - r.destroy(r.ptr) - } -} - -// close releases the wrapper's reference once no call is in flight. Calling -// it again is a no-op. -func (r *wvRef) close() { - for { - s := r.state.Load() - if s&1 != 0 { - return - } - if r.state.CompareAndSwap(s, s|1) { - if s == 0 { - r.destroy(r.ptr) - } - return - } - } -} - -// ── Async ── - -// wvOutcome is one async completion: a converted result, or the failure the -// producer reported. -type wvOutcome[T any] struct { - val T - fail *wvFailure -} - -// wvComplete delivers one async completion to the waiting wrapper. It runs on -// a producer thread inside an exported trampoline, so it never panics: a -// result that fails to decode is delivered as a marshalling failure. -func wvComplete[T any](handle unsafe.Pointer, cErr *C.nested_modules_error, convert func() T) { - h := cgo.Handle(uintptr(handle)) - done := h.Value().(chan wvOutcome[T]) - h.Delete() - done <- wvSettle(cErr, convert) -} - -func wvSettle[T any](cErr *C.nested_modules_error, convert func() T) (o wvOutcome[T]) { - if cErr != nil { - if f := wvTakeBoxedError(cErr); f.code != 0 { - o.fail = &f - return o - } - } - defer func() { - if r := recover(); r != nil { - o = wvOutcome[T]{fail: &wvFailure{code: wvCodeMarshal, message: fmt.Sprint(r)}} - } - }() - o.val = convert() - return o -} - -// wvAwait blocks until an async call completes or ctx is done. With a cancel -// token, ctx's cancellation cancels the native call and waits for its -// completion, which the producer delivers promptly; without one, the call is -// abandoned and ctx.Err() returned at once. A cancelled completion (code -5) -// returns ctx.Err(), or context.Canceled when ctx itself isn't done. -func wvAwait[T any](ctx context.Context, done chan wvOutcome[T], token *C.nested_modules_cancel_token) (T, *wvFailure, error) { - var o wvOutcome[T] - select { - case o = <-done: - case <-ctx.Done(): - if token == nil { - var zero T - return zero, nil, ctx.Err() - } - C.nested_modules_cancel_token_cancel(token) - o = <-done - } - if o.fail != nil && o.fail.code == wvCodeCancelled { - var zero T - if err := ctx.Err(); err != nil { - return zero, nil, err - } - return zero, nil, context.Canceled - } - return o.val, o.fail, nil -} - -// END runtime.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/go_shapes__README_md.snap deleted file mode 100644 index 9bf2eb9d..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_shapes__README_md.snap +++ /dev/null @@ -1,49 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# shapes (Go) - -cgo bindings for the `shapes` native library. - -## Build - -The module ships its own copy of the C header (`shapes.h`) and links with -`-lshapes`. Point the linker at the directory holding `libshapes.so`, -`libshapes.dylib`, or `shapes.dll`, and make the library findable at -run time: - -```sh -export CGO_LDFLAGS="-L/path/to/lib" -export LD_LIBRARY_PATH="/path/to/lib" # DYLD_LIBRARY_PATH on macOS -go build ./... -``` - -Depend on the module from your own project with a `require` (and, for a -local checkout, a `replace`) of `shapes`, then import it: - -```go -import "shapes" -``` - -Go 1.23 or newer is required. - -## Usage notes - -- Records are plain structs and rich enums are sealed interfaces; both cross - the boundary by value. -- Interfaces are reference-counted objects. Call `Close` when you're done - with one (a finalizer releases it otherwise); it's safe to call twice or - while a call is in flight. -- Async functions take a `context.Context` first. Cancellable ones cancel - the native call when the context is done. -- Callback interfaces are Go interfaces you implement. The library may call - them from any thread, and a panic is reported to the native caller as a - foreign error instead of crashing. -- Loading checks the library's ABI revision and contract checksums and - panics with the stale module's name on a mismatch. - - diff --git a/crates/weaveffi-cli/tests/snapshots/go_shapes__bindings_go.snap b/crates/weaveffi-cli/tests/snapshots/go_shapes__bindings_go.snap index 8df9b2ec..61c62ec8 100644 --- a/crates/weaveffi-cli/tests/snapshots/go_shapes__bindings_go.snap +++ b/crates/weaveffi-cli/tests/snapshots/go_shapes__bindings_go.snap @@ -1,10 +1,11 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate +// Package shapes binds the shapes native library. package shapes /* @@ -15,13 +16,23 @@ import "C" func init() { wvCheckABI() - wvCheckModule("shapes", 0x06acf433ffa2a0f0, uint64(C.shapes_shapes_checksum())) + var n C.size_t + table := C.shapes_shapes_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0x0d7b3eec49243e7a, 0x5f232311808e1b1f, "shapes.Channel"}, + {0x183fa8198100860e, 0xe40c6993cb31d359, "shapes.describe"}, + {0x2d1395eb837b848e, 0x42202a0c12f3a16e, "shapes.sum_bytes"}, + {0x7f9a5b87ba09b86b, 0xc5b32fb0efbbaa17, "shapes.scale"}, + {0xc8edd779240fda17, 0x68e1be78fddb20e4, "shapes.Numerics"}, + {0xd3d3881b2fd18a62, 0xf24d0eb76a99d918, "shapes.make_numerics"}, + {0xf010844afff5ac3c, 0x50de28ae9b41ff15, "shapes.Shape"}, + }) } -// Shape: An algebraic shape (sum type with associated data) +// Shape is an algebraic shape (sum type with associated data) // -// Shape is a sealed sum type: exactly one of its variant structs is the -// value at a time. +// Shape is a sealed sum type: exactly one of its variant structs is the value +// at a time. type Shape interface { isShape() } @@ -33,7 +44,7 @@ func (ShapeEmpty) isShape() {} // ShapeCircle: A circle with a radius type ShapeCircle struct { - // Radius: Radius in points + // Radius in points Radius float64 } @@ -64,8 +75,7 @@ type ShapeLabeled struct { func (ShapeLabeled) isShape() {} -// wvPackShape appends v to w in the value-buffer wire format. -func wvPackShape(w *wvWriter, v Shape) { +func wvWriteShape(w *wvWriter, v Shape) { switch x := v.(type) { case ShapeEmpty: w.writeI32(0) @@ -90,8 +100,7 @@ func wvPackShape(w *wvWriter, v Shape) { } } -// wvUnpackShape decodes one Shape from r. -func wvUnpackShape(r *wvReader) Shape { +func wvReadShape(r *wvReader) Shape { switch r.readI32() { case 0: return ShapeEmpty{} @@ -115,12 +124,12 @@ func wvUnpackShape(r *wvReader) Shape { x.Label = r.readString() x.Count = r.readU8() return x - default: - panic("shapes: malformed value buffer: Shape tag out of range") } + wvMalformed("Shape tag out of range") + return nil } -// Channel: A plain C-style enum (no payloads) +// Channel is a plain C-style enum (no payloads) type Channel int32 const ( @@ -143,8 +152,7 @@ type Numerics struct { J float64 } -// wvPackNumerics appends v to w in the value-buffer wire format. -func wvPackNumerics(w *wvWriter, v Numerics) { +func wvWriteNumerics(w *wvWriter, v Numerics) { w.writeI8(v.A) w.writeI16(v.B) w.writeI32(v.C) @@ -157,8 +165,7 @@ func wvPackNumerics(w *wvWriter, v Numerics) { w.writeF64(v.J) } -// wvUnpackNumerics decodes one Numerics from r. -func wvUnpackNumerics(r *wvReader) Numerics { +func wvReadNumerics(r *wvReader) Numerics { var v Numerics v.A = r.readI8() v.B = r.readI16() @@ -175,9 +182,7 @@ func wvUnpackNumerics(r *wvReader) Numerics { // Describe: Render a shape to a string func Describe(shape Shape) string { - wShape := &wvWriter{} - wvPackShape(wShape, shape) - cShapePtr, cShapeLen := wvBytes(wShape.buf) + cShapePtr, cShapeLen := wvBytes(wvEncode(shape, wvWriteShape)) var cRetLen C.size_t var cErr C.shapes_error cRet := C.shapes_shapes_describe(cShapePtr, cShapeLen, &cRetLen, &cErr) @@ -185,20 +190,14 @@ func Describe(shape Shape) string { return wvTakeString(cRet, cRetLen) } -// Scale: Scale a shape by a factor, returning a new shape +// Scale a shape by a factor, returning a new shape func Scale(shape Shape, factor float64) Shape { - wShape := &wvWriter{} - wvPackShape(wShape, shape) - cShapePtr, cShapeLen := wvBytes(wShape.buf) + cShapePtr, cShapeLen := wvBytes(wvEncode(shape, wvWriteShape)) var cRetLen C.size_t var cErr C.shapes_error cRet := C.shapes_shapes_scale(cShapePtr, cShapeLen, C.double(factor), &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Shape - ret = wvUnpackShape(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadShape) } // SumBytes: Sum a list of bytes into a wide integer (numerics smoke) @@ -216,11 +215,7 @@ func MakeNumerics(a int8, b int16, c int32, d int64, e uint8, f uint16, g uint32 var cErr C.shapes_error cRet := C.shapes_shapes_make_numerics(C.int8_t(a), C.int16_t(b), C.int32_t(c), C.int64_t(d), C.uint8_t(e), C.uint16_t(f), C.uint32_t(g), C.uint64_t(h), C.float(i), C.double(j), &cRetLen, &cErr) wvTrap(&cErr) - rRet := &wvReader{buf: wvTakeBytes(cRet, cRetLen)} - var ret Numerics - ret = wvUnpackNumerics(rRet) - rRet.expectEnd() - return ret + return wvDecode(cRet, cRetLen, wvReadNumerics) } // END bindings.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_shapes__codec_go.snap b/crates/weaveffi-cli/tests/snapshots/go_shapes__codec_go.snap deleted file mode 100644 index abdff52a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_shapes__codec_go.snap +++ /dev/null @@ -1,205 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package shapes - -import ( - "encoding/binary" - "math" - "unicode/utf8" -) - -// wvWriter serializes values into the value-buffer format: little-endian, -// packed, u32 length prefixes. -type wvWriter struct { - buf []byte -} - -func (w *wvWriter) writeBool(v bool) { - if v { - w.buf = append(w.buf, 1) - } else { - w.buf = append(w.buf, 0) - } -} - -func (w *wvWriter) writeI8(v int8) { - w.buf = append(w.buf, byte(v)) -} - -func (w *wvWriter) writeU8(v uint8) { - w.buf = append(w.buf, v) -} - -func (w *wvWriter) writeI16(v int16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, uint16(v)) -} - -func (w *wvWriter) writeU16(v uint16) { - w.buf = binary.LittleEndian.AppendUint16(w.buf, v) -} - -func (w *wvWriter) writeI32(v int32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, uint32(v)) -} - -func (w *wvWriter) writeU32(v uint32) { - w.buf = binary.LittleEndian.AppendUint32(w.buf, v) -} - -func (w *wvWriter) writeI64(v int64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, uint64(v)) -} - -func (w *wvWriter) writeU64(v uint64) { - w.buf = binary.LittleEndian.AppendUint64(w.buf, v) -} - -func (w *wvWriter) writeF32(v float32) { - w.writeU32(math.Float32bits(v)) -} - -func (w *wvWriter) writeF64(v float64) { - w.writeU64(math.Float64bits(v)) -} - -func (w *wvWriter) writeLen(n int) { - if n < 0 || uint64(n) > math.MaxUint32 { - panic("shapes: value-buffer length exceeds u32 range") - } - w.writeU32(uint32(n)) -} - -func (w *wvWriter) writeString(v string) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeBytes(v []byte) { - w.writeLen(len(v)) - w.buf = append(w.buf, v...) -} - -func (w *wvWriter) writeOptionFlag(present bool) { - w.writeBool(present) -} - -// wvReader decodes values from the value-buffer format. A malformed buffer -// is a producer/consumer contract violation, so every read panics instead of -// returning a typed domain error. -type wvReader struct { - buf []byte - pos int -} - -func wvMalformed(context string) { - panic("shapes: malformed value buffer: " + context) -} - -func (r *wvReader) take(n int, context string) []byte { - if n < 0 || len(r.buf)-r.pos < n { - wvMalformed(context) - } - b := r.buf[r.pos : r.pos+n] - r.pos += n - return b -} - -func (r *wvReader) readBool() bool { - switch r.take(1, "bool")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("bool byte out of range") - return false -} - -func (r *wvReader) readI8() int8 { - return int8(r.take(1, "i8")[0]) -} - -func (r *wvReader) readU8() uint8 { - return r.take(1, "u8")[0] -} - -func (r *wvReader) readI16() int16 { - return int16(binary.LittleEndian.Uint16(r.take(2, "i16"))) -} - -func (r *wvReader) readU16() uint16 { - return binary.LittleEndian.Uint16(r.take(2, "u16")) -} - -func (r *wvReader) readI32() int32 { - return int32(binary.LittleEndian.Uint32(r.take(4, "i32"))) -} - -func (r *wvReader) readU32() uint32 { - return binary.LittleEndian.Uint32(r.take(4, "u32")) -} - -func (r *wvReader) readI64() int64 { - return int64(binary.LittleEndian.Uint64(r.take(8, "i64"))) -} - -func (r *wvReader) readU64() uint64 { - return binary.LittleEndian.Uint64(r.take(8, "u64")) -} - -func (r *wvReader) readF32() float32 { - return math.Float32frombits(r.readU32()) -} - -func (r *wvReader) readF64() float64 { - return math.Float64frombits(r.readU64()) -} - -// readLen reads a u32 element count. Elements may encode to zero bytes, so -// the count isn't checked against the remaining buffer; callers cap their -// preallocation with capHint instead. -func (r *wvReader) readLen() int { - return int(r.readU32()) -} - -// capHint bounds a preallocation for n decoded elements by the bytes left in -// the buffer, so a corrupt count can't force a huge allocation. -func (r *wvReader) capHint(n int) int { - return min(n, len(r.buf)-r.pos) -} - -func (r *wvReader) readString() string { - b := r.take(r.readLen(), "string bytes") - if !utf8.Valid(b) { - wvMalformed("string is not valid UTF-8") - } - return string(b) -} - -func (r *wvReader) readBytes() []byte { - return append([]byte{}, r.take(r.readLen(), "byte buffer")...) -} - -func (r *wvReader) readOptionFlag() bool { - switch r.take(1, "option flag")[0] { - case 0: - return false - case 1: - return true - } - wvMalformed("option flag byte out of range") - return false -} - -func (r *wvReader) expectEnd() { - if r.pos != len(r.buf) { - wvMalformed("trailing bytes after value") - } -} - -// END codec.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_shapes__go_mod.snap b/crates/weaveffi-cli/tests/snapshots/go_shapes__go_mod.snap deleted file mode 100644 index f9f7bd34..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_shapes__go_mod.snap +++ /dev/null @@ -1,12 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module shapes - -go 1.23 - -// END go.mod diff --git a/crates/weaveffi-cli/tests/snapshots/go_shapes__runtime_go.snap b/crates/weaveffi-cli/tests/snapshots/go_shapes__runtime_go.snap deleted file mode 100644 index e8f30115..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_shapes__runtime_go.snap +++ /dev/null @@ -1,305 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package shapes - -/* -#include -#include "shapes.h" -*/ -import "C" - -import ( - "context" - "fmt" - "runtime/cgo" - "sync/atomic" - "unsafe" -) - -// wvABIVersion is the C ABI revision these bindings were generated against. -const wvABIVersion uint32 = 3 - -// The runtime error codes this package produces or interprets itself. -const ( - wvCodeMarshal int32 = -3 - wvCodeForeign int32 = -4 - wvCodeCancelled int32 = -5 -) - -// Error reports a failure that no typed error domain claims: an unknown -// code, a generic failure (code -1), a producer panic (-2), a marshalling -// failure (-3), or a Go callback-interface implementation that panicked -// (-4). The code is preserved so callers can tell them apart. -type Error struct { - // Code is the numeric ABI error code. - Code int32 - // Message is the human-readable error message. - Message string -} - -func (e *Error) Error() string { - return fmt.Sprintf("shapes: %s (code %d)", e.Message, e.Code) -} - -// DebugLive reports how many resources of one kind the native library holds: -// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned -// allocations. It's always 0 unless the library was built with leak checks -// enabled; tests use it to assert that nothing leaked. -func DebugLive(kind int32) uint64 { - return uint64(C.shapes_debug_live(C.int32_t(kind))) -} - -// wvCheckABI panics when the linked library implements a different C ABI -// revision than these bindings were generated for. -func wvCheckABI() { - if found := uint32(C.shapes_abi_version()); found != wvABIVersion { - panic(fmt.Sprintf("shapes: ABI mismatch: these bindings expect revision %d but the linked library reports revision %d", wvABIVersion, found)) - } -} - -// wvCheckModule panics when the linked library's contract checksum for a -// top-level module differs from the one these bindings were generated -// against, which means the bindings are stale. -func wvCheckModule(module string, want, found uint64) { - if found != want { - panic(fmt.Sprintf("shapes: module %q doesn't match the linked library (bindings checksum %#016x, library checksum %#016x); regenerate the bindings", module, want, found)) - } -} - -// ── Errors ── - -// wvFailure is a non-zero error slot copied into Go memory. -type wvFailure struct { - code int32 - message string - payload []byte -} - -// err converts the failure into the generic *Error. -func (f wvFailure) err() error { - return &Error{Code: f.code, Message: f.message} -} - -func wvCopyFailure(cErr *C.shapes_error) wvFailure { - f := wvFailure{code: int32(cErr.code)} - if cErr.message != nil { - f.message = C.GoString(cErr.message) - } - if cErr.payload_ptr != nil { - f.payload = wvBorrowBytes(cErr.payload_ptr, cErr.payload_len) - } - return f -} - -// wvTakeError copies a non-zero error slot and clears it. -func wvTakeError(cErr *C.shapes_error) wvFailure { - f := wvCopyFailure(cErr) - C.shapes_error_clear(cErr) - return f -} - -// wvTakeBoxedError copies the heap-boxed error an async completion receives -// and frees the box. -func wvTakeBoxedError(cErr *C.shapes_error) wvFailure { - f := wvCopyFailure(cErr) - C.shapes_error_free(cErr) - return f -} - -// wvTrap panics with the generic *Error when the slot reports a failure. -// Wrappers of functions that declare no error domain check their slot with -// it: a failure there can only be a producer bug, a marshalling failure, or -// a callback-interface implementation that panicked. -func wvTrap(cErr *C.shapes_error) { - if cErr.code != 0 { - panic(wvTakeError(cErr).err()) - } -} - -// wvForeignError reports a panic recovered inside a callback-interface -// trampoline through the producer's error slot (code -4). The producer -// copies the message, so it's only borrowed for the call. -func wvForeignError(outErr *C.shapes_error, recovered any) { - msg := C.CString(fmt.Sprint(recovered)) - defer C.free(unsafe.Pointer(msg)) - C.shapes_error_set(outErr, C.int32_t(wvCodeForeign), msg) -} - -// ── Strings and bytes ── - -// wvStr returns a borrowed (pointer, length) view of s's UTF-8 bytes for the -// duration of one call. Strings may contain NUL bytes. -func wvStr(s string) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.StringData(s))), C.size_t(len(s)) -} - -// wvBytes returns a borrowed (pointer, length) view of b for the duration of -// one call. -func wvBytes(b []byte) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.SliceData(b))), C.size_t(len(b)) -} - -// wvBorrowString copies a borrowed UTF-8 run into a Go string. -func wvBorrowString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - return string(unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) -} - -// wvTakeString copies a returned UTF-8 run into a Go string and releases the -// producer's allocation. -func wvTakeString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - s := wvBorrowString(ptr, n) - C.shapes_free_bytes(ptr, n) - return s -} - -// wvBorrowBytes copies a borrowed byte run into Go memory. A null pointer -// copies to nil. -func wvBorrowBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - return append([]byte{}, unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)...) -} - -// wvTakeBytes copies a returned byte run (raw bytes or a value buffer) into -// Go memory and releases the producer's allocation. -func wvTakeBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - b := wvBorrowBytes(ptr, n) - C.shapes_free_bytes(ptr, n) - return b -} - -// ── Objects ── - -// wvRef owns one strong reference to a native object on behalf of a wrapper. -// Calls borrow the pointer between acquire and release. Close may race them: -// the reference is destroyed exactly once, by close when no call is in -// flight, or else by the last in-flight call to finish. -type wvRef struct { - ptr unsafe.Pointer - destroy func(unsafe.Pointer) - // state counts in-flight calls in its upper bits; the low bit is set - // once close has run. - state atomic.Uint64 -} - -func (r *wvRef) init(ptr unsafe.Pointer, destroy func(unsafe.Pointer)) { - r.ptr = ptr - r.destroy = destroy -} - -// acquire borrows the pointer for one call, panicking when the wrapper was -// already closed. -func (r *wvRef) acquire(typeName string) unsafe.Pointer { - for { - s := r.state.Load() - if s&1 != 0 { - panic("shapes: " + typeName + " used after Close") - } - if r.state.CompareAndSwap(s, s+2) { - return r.ptr - } - } -} - -// release ends one call's borrow. -func (r *wvRef) release() { - if r.state.Add(^uint64(1)) == 1 { - r.destroy(r.ptr) - } -} - -// close releases the wrapper's reference once no call is in flight. Calling -// it again is a no-op. -func (r *wvRef) close() { - for { - s := r.state.Load() - if s&1 != 0 { - return - } - if r.state.CompareAndSwap(s, s|1) { - if s == 0 { - r.destroy(r.ptr) - } - return - } - } -} - -// ── Async ── - -// wvOutcome is one async completion: a converted result, or the failure the -// producer reported. -type wvOutcome[T any] struct { - val T - fail *wvFailure -} - -// wvComplete delivers one async completion to the waiting wrapper. It runs on -// a producer thread inside an exported trampoline, so it never panics: a -// result that fails to decode is delivered as a marshalling failure. -func wvComplete[T any](handle unsafe.Pointer, cErr *C.shapes_error, convert func() T) { - h := cgo.Handle(uintptr(handle)) - done := h.Value().(chan wvOutcome[T]) - h.Delete() - done <- wvSettle(cErr, convert) -} - -func wvSettle[T any](cErr *C.shapes_error, convert func() T) (o wvOutcome[T]) { - if cErr != nil { - if f := wvTakeBoxedError(cErr); f.code != 0 { - o.fail = &f - return o - } - } - defer func() { - if r := recover(); r != nil { - o = wvOutcome[T]{fail: &wvFailure{code: wvCodeMarshal, message: fmt.Sprint(r)}} - } - }() - o.val = convert() - return o -} - -// wvAwait blocks until an async call completes or ctx is done. With a cancel -// token, ctx's cancellation cancels the native call and waits for its -// completion, which the producer delivers promptly; without one, the call is -// abandoned and ctx.Err() returned at once. A cancelled completion (code -5) -// returns ctx.Err(), or context.Canceled when ctx itself isn't done. -func wvAwait[T any](ctx context.Context, done chan wvOutcome[T], token *C.shapes_cancel_token) (T, *wvFailure, error) { - var o wvOutcome[T] - select { - case o = <-done: - case <-ctx.Done(): - if token == nil { - var zero T - return zero, nil, ctx.Err() - } - C.shapes_cancel_token_cancel(token) - o = <-done - } - if o.fail != nil && o.fail.code == wvCodeCancelled { - var zero T - if err := ctx.Err(); err != nil { - return zero, nil, err - } - return zero, nil, context.Canceled - } - return o.val, o.fail, nil -} - -// END runtime.go diff --git a/crates/weaveffi-cli/tests/snapshots/go_shapes__shapes_h.snap b/crates/weaveffi-cli/tests/snapshots/go_shapes__shapes_h.snap deleted file mode 100644 index df20c3f5..00000000 --- a/crates/weaveffi-cli/tests/snapshots/go_shapes__shapes_h.snap +++ /dev/null @@ -1,151 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef SHAPES_H -#define SHAPES_H - -#include -#include -#include - -#ifndef SHAPES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef SHAPES_BUILD -# define SHAPES_API __declspec(dllexport) -# else -# define SHAPES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define SHAPES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define SHAPES_API __attribute__((visibility("default"))) -# else -# define SHAPES_API -# endif -#endif - -#ifndef SHAPES_DEPRECATED -# if defined(_MSC_VER) -# define SHAPES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define SHAPES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define SHAPES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports shapes_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define SHAPES_ABI_VERSION 3u -SHAPES_API uint32_t shapes_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by shapes_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct shapes_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} shapes_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -SHAPES_API void shapes_error_set(shapes_error* err, int32_t code, const char* message); -SHAPES_API void shapes_error_clear(shapes_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - shapes_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -SHAPES_API void shapes_error_free(shapes_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -SHAPES_API void shapes_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct shapes_cancel_token shapes_cancel_token; -SHAPES_API shapes_cancel_token* shapes_cancel_token_create(void); -SHAPES_API void shapes_cancel_token_cancel(shapes_cancel_token* token); -SHAPES_API bool shapes_cancel_token_is_cancelled(const shapes_cancel_token* token); -SHAPES_API void shapes_cancel_token_destroy(shapes_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -SHAPES_API uint8_t* shapes_alloc(uint32_t size); -SHAPES_API void shapes_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -SHAPES_API uint64_t shapes_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define SHAPES_SHAPES_CHECKSUM 0x06acf433ffa2a0f0ull -SHAPES_API uint64_t shapes_shapes_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with shapes_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * shapes_buffer.h declares a C struct and codec for each. - */ - -/** An algebraic shape (sum type with associated data) */ -typedef enum { - /** The empty shape (a unit variant) */ - shapes_shapes_Shape_Empty = 0, - /** A circle with a radius */ - shapes_shapes_Shape_Circle = 1, - /** An axis-aligned rectangle */ - shapes_shapes_Shape_Rectangle = 2, - /** An integer-bounded shape (exercises narrow + wide ints) */ - shapes_shapes_Shape_Bounded = 3, - /** A labeled shape with a small count */ - shapes_shapes_Shape_Labeled = 4 -} shapes_shapes_Shape_Tag; -/** A plain C-style enum (no payloads) */ -typedef enum { shapes_shapes_Channel_Red = 0, shapes_shapes_Channel_Green = 1, shapes_shapes_Channel_Blue = 2 } shapes_shapes_Channel; - -// Module: shapes -/** Render a shape to a string */ -SHAPES_API const uint8_t* shapes_shapes_describe(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, shapes_error* out_err); -/** Scale a shape by a factor, returning a new shape */ -SHAPES_API const uint8_t* shapes_shapes_scale(const uint8_t* shape_ptr, size_t shape_len, double factor, size_t* out_len, shapes_error* out_err); -/** Sum a list of bytes into a wide integer (numerics smoke) */ -SHAPES_API uint64_t shapes_shapes_sum_bytes(const uint8_t* values_ptr, size_t values_len, shapes_error* out_err); -/** Build a Numerics struct from every numeric primitive */ -SHAPES_API const uint8_t* shapes_shapes_make_numerics(int8_t a, int16_t b, int32_t c, int64_t d, uint8_t e, uint16_t f, uint32_t g, uint64_t h, float i, double j, size_t* out_len, shapes_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // SHAPES_H - -// END shapes.h diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__build_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__build_gradle_kts.snap deleted file mode 100644 index d2bbb125..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__build_gradle_kts.snap +++ /dev/null @@ -1,63 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -plugins { - id("com.android.library") version "8.7.3" - id("org.jetbrains.kotlin.android") version "2.0.21" - `maven-publish` -} - -group = "docs_everywhere" -version = "0.1.0" - -android { - namespace = "docs_everywhere" - compileSdk = 35 - defaultConfig { - minSdk = 21 - consumerProguardFiles("consumer-rules.pro") - } - // Builds libdocs_everywhere_jni.so per ABI, linked against the producer library in - // src/main/jniLibs// (see src/main/cpp/CMakeLists.txt). - externalNativeBuild { - cmake { - path = file("src/main/cpp/CMakeLists.txt") - } - } - compileOptions { - sourceCompatibility = JavaVersion.VERSION_1_8 - targetCompatibility = JavaVersion.VERSION_1_8 - } - publishing { - singleVariant("release") { - withSourcesJar() - } - } -} - -kotlin { - compilerOptions { - jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_1_8) - } -} - -dependencies { - implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.9.0") -} - -publishing { - publications { - register("release") { - artifactId = "docs_everywhere" - afterEvaluate { - from(components["release"]) - } - } - } -} - -// END build.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__consumer_rules_pro.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__consumer_rules_pro.snap deleted file mode 100644 index bd595a8f..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__consumer_rules_pro.snap +++ /dev/null @@ -1,11 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# The JNI shim looks classes and methods up by name. --keep class docs_everywhere.** { *; } - -# END consumer-rules.pro diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__settings_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__settings_gradle_kts.snap deleted file mode 100644 index 96cacab4..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__settings_gradle_kts.snap +++ /dev/null @@ -1,25 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -pluginManagement { - repositories { - google() - mavenCentral() - gradlePluginPortal() - } -} - -dependencyResolutionManagement { - repositories { - google() - mavenCentral() - } -} - -rootProject.name = "docs_everywhere" - -// END settings.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_CMakeLists_txt.snap deleted file mode 100644 index 0fccae2a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_CMakeLists_txt.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.18) -project(docs_everywhere_jni C) - -# The directory holding the producer library (libdocs_everywhere). Android builds -# default to the per-ABI copy under src/main/jniLibs/. -set(DOCS_EVERYWHERE_LIBRARY_DIR "" CACHE PATH "Directory holding the docs_everywhere producer library") -if(NOT DOCS_EVERYWHERE_LIBRARY_DIR AND ANDROID) - set(DOCS_EVERYWHERE_LIBRARY_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../jniLibs/${ANDROID_ABI}") -endif() -if(DOCS_EVERYWHERE_LIBRARY_DIR) - find_library(DOCS_EVERYWHERE_PRODUCER NAMES docs_everywhere PATHS "${DOCS_EVERYWHERE_LIBRARY_DIR}" NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH REQUIRED) -else() - set(CMAKE_FIND_FRAMEWORK NEVER) - find_library(DOCS_EVERYWHERE_PRODUCER NAMES docs_everywhere REQUIRED) -endif() - -add_library(docs_everywhere_jni SHARED docs_everywhere_jni.c) -target_include_directories(docs_everywhere_jni PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}") -target_link_libraries(docs_everywhere_jni PRIVATE "${DOCS_EVERYWHERE_PRODUCER}") -if(NOT ANDROID) - find_package(JNI REQUIRED) - find_package(Threads REQUIRED) - target_include_directories(docs_everywhere_jni PRIVATE ${JNI_INCLUDE_DIRS}) - target_link_libraries(docs_everywhere_jni PRIVATE Threads::Threads) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_h.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_h.snap deleted file mode 100644 index 31ab85e2..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_h.snap +++ /dev/null @@ -1,196 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef DOCS_EVERYWHERE_H -#define DOCS_EVERYWHERE_H - -#include -#include -#include - -#ifndef DOCS_EVERYWHERE_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef DOCS_EVERYWHERE_BUILD -# define DOCS_EVERYWHERE_API __declspec(dllexport) -# else -# define DOCS_EVERYWHERE_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define DOCS_EVERYWHERE_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define DOCS_EVERYWHERE_API __attribute__((visibility("default"))) -# else -# define DOCS_EVERYWHERE_API -# endif -#endif - -#ifndef DOCS_EVERYWHERE_DEPRECATED -# if defined(_MSC_VER) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define DOCS_EVERYWHERE_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports docs_everywhere_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define DOCS_EVERYWHERE_ABI_VERSION 3u -DOCS_EVERYWHERE_API uint32_t docs_everywhere_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by docs_everywhere_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct docs_everywhere_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} docs_everywhere_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_set(docs_everywhere_error* err, int32_t code, const char* message); -DOCS_EVERYWHERE_API void docs_everywhere_error_clear(docs_everywhere_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - docs_everywhere_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_free(docs_everywhere_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -DOCS_EVERYWHERE_API void docs_everywhere_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct docs_everywhere_cancel_token docs_everywhere_cancel_token; -DOCS_EVERYWHERE_API docs_everywhere_cancel_token* docs_everywhere_cancel_token_create(void); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_cancel(docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API bool docs_everywhere_cancel_token_is_cancelled(const docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_destroy(docs_everywhere_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -DOCS_EVERYWHERE_API uint8_t* docs_everywhere_alloc(uint32_t size); -DOCS_EVERYWHERE_API void docs_everywhere_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -DOCS_EVERYWHERE_API uint64_t docs_everywhere_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define DOCS_EVERYWHERE_DOCS_CHECKSUM 0x7950eec1e9cfc819ull -DOCS_EVERYWHERE_API uint64_t docs_everywhere_docs_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with docs_everywhere_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * docs_everywhere_buffer.h declares a C struct and codec for each. - */ - -/** - * Lifecycle status of a document. - * - * Used to drive UI badges and filters. - */ -typedef enum { - /** Document is being edited */ - docs_everywhere_docs_Status_Draft = 0, - /** Document is publicly visible */ - docs_everywhere_docs_Status_Published = 1, - /** Document is hidden from listings */ - docs_everywhere_docs_Status_Archived = 2 -} docs_everywhere_docs_Status; -/** Error codes reported by throwing functions in the `docs` module tree. */ -typedef enum { - /** Returned when no document matches the requested id */ - docs_everywhere_docs_DocsErrors_not_found = 1, - /** Raised when attempting to move into a forbidden state */ - docs_everywhere_docs_DocsErrors_invalid_status = 2 -} docs_everywhere_docs_DocsErrors; -typedef struct docs_everywhere_docs_Notebook docs_everywhere_docs_Notebook; -/** - * Consumer-implemented observer of document-saved notifications. - * - * Implementations must be safe to call from any thread. - * - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct docs_everywhere_docs_SavedListener_vtable { - /** Fires after a document has been persisted successfully */ - void (*on_saved)(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err); - void (*free)(void* ctx); -} docs_everywhere_docs_SavedListener_vtable; -typedef void (*docs_everywhere_docs_publish_async_callback)(void* context, docs_everywhere_error* err, bool result); - -// Module: docs -/** - * An ordered collection of documents. - * - * Notebooks own their documents and release them when destroyed. - */ -/** Create an empty notebook */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_new(docs_everywhere_error* out_err); -/** The largest number of documents one notebook will hold */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_max_documents(docs_everywhere_error* out_err); -/** Add a document, returning its position in the notebook */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_insert(const docs_everywhere_docs_Notebook* self, const uint8_t* document_ptr, size_t document_len, docs_everywhere_error* out_err); -/** Look up a document by identifier, failing when none matches */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_Notebook_find(const docs_everywhere_docs_Notebook* self, int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** The number of documents currently held */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_size(const docs_everywhere_docs_Notebook* self, docs_everywhere_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_clone(const docs_everywhere_docs_Notebook* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_Notebook_destroy(docs_everywhere_docs_Notebook* self); - -/** - * Create a brand new document. - * - * The document starts in `Draft` status; callers must publish - * it explicitly to make it visible. - */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_create_document(const uint8_t* title_ptr, size_t title_len, const uint8_t* tags_ptr, size_t tags_len, size_t* out_len, docs_everywhere_error* out_err); -/** Look up a document by identifier */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_get_document(int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** Publish a document in the background */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_publish_async(int64_t id, docs_everywhere_docs_publish_async_callback callback, void* context); -/** Register a listener for saved notifications */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_watch(void* listener_ctx, const docs_everywhere_docs_SavedListener_vtable* listener_vtable, docs_everywhere_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // DOCS_EVERYWHERE_H - -// END docs_everywhere.h diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_jni_c.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_jni_c.snap index 7be6f786..3b5e714b 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_jni_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_cpp_docs_everywhere_jni_c.snap @@ -1,14 +1,15 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #include #include #include #include +#include #include #include @@ -72,8 +73,9 @@ static inline jbyteArray Jni_take_bytes(JNIEnv* env, const uint8_t* ptr, size_t return out; } -/* Throws the Kotlin exception for `err` (mapped through domain `domain`, - 0 for the generic one) and clears `err`. */ +/* Throws the Kotlin exception for `err` (mapped through error domain + `domain`, or 0 for a call that can't fail, which raises the trap) and + clears `err`. */ static inline void Jni_throw(JNIEnv* env, docs_everywhere_error* err, jint domain) { const char* message = err->message != NULL ? err->message : ""; jbyteArray text = Jni_new_bytes(env, (const uint8_t*)message, strlen(message)); @@ -100,6 +102,50 @@ static jint Jni_load_error(JNIEnv* env, const char* message) { return JNI_ERR; } +/* One contract entry these bindings were generated with: a declaration's + id and signature hash, plus its dotted path for the load error. */ +typedef struct { + uint64_t id; + uint64_t hash; + const char* path; +} Jni_contract_entry; + +/* Checks that the library's contract table (from `contract`, sorted by id) + has every expected entry with an equal hash; entries only the library + has are fine. Fails loading with an error naming the first declaration + that's missing or changed. */ +static inline jint Jni_check_contract(JNIEnv* env, const docs_everywhere_contract_entry* (*contract)(size_t*), const Jni_contract_entry* expected, size_t count) { + size_t len = 0; + const docs_everywhere_contract_entry* table = contract(&len); + if (table == NULL) { + len = 0; + } + for (size_t i = 0; i < count; i++) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) { + lo = mid + 1; + } else { + hi = mid; + } + } + const char* problem = NULL; + if (lo == len || table[lo].id != expected[i].id) { + problem = "is missing from the library"; + } else if (table[lo].hash != expected[i].hash) { + problem = "changed since these bindings were generated"; + } + if (problem != NULL) { + char message[512]; + snprintf(message, sizeof message, "%s: %s %s", "docs_everywhere", expected[i].path, problem); + return Jni_load_error(env, message); + } + } + return JNI_OK; +} + /* Generated below: the contract checks and the method IDs the shim caches. */ static jint Jni_load(JNIEnv* env); @@ -110,15 +156,15 @@ JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { if ((*vm)->GetEnv(vm, (void**)&env, JNI_VERSION_1_6) != JNI_OK) { return JNI_ERR; } - if (docs_everywhere_abi_version() != 3u) { - return Jni_load_error(env, "docs_everywhere: the native library implements a different C ABI revision than these bindings (expected 3)"); + if (docs_everywhere_abi_version() != 4u) { + return Jni_load_error(env, "docs_everywhere: the native library implements a different C ABI revision than these bindings (expected 4)"); } jclass bridge = (*env)->FindClass(env, "docs_everywhere/JniBridge"); if (bridge == NULL) { return JNI_ERR; } Jni_bridge = (jclass)(*env)->NewGlobalRef(env, bridge); - Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ldocs_everywhere/FfiException;"); + Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ljava/lang/Throwable;"); if (Jni_error == NULL) { return JNI_ERR; } @@ -136,38 +182,68 @@ JNIEXPORT jlong JNICALL Java_docs_1everywhere_JniBridge_debug_1live(JNIEnv* env, /* Producer threads (async completions, callback calls, vtable `free`) are usually not JVM threads. The first time one calls in it attaches as a - daemon and keeps its JNIEnv; a thread-local destructor detaches it when - the thread exits, so a busy producer thread pays for one attach, not one - per call. */ + daemon, and a thread-local destructor (a pthread key, or a fiber-local + slot on Windows) detaches it when the thread exits, so a busy producer + thread pays for one attach, not one per call. If the destructor can't be + registered, the thread detaches again as soon as the call ends. */ #if defined(_WIN32) -static void Jni_remember_attach(void) {} +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#include + +static DWORD Jni_env_slot = FLS_OUT_OF_INDEXES; +static INIT_ONCE Jni_env_once = INIT_ONCE_STATIC_INIT; + +static VOID WINAPI Jni_thread_exit(PVOID value) { + if (value != NULL && Jni_vm != NULL) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + +static BOOL CALLBACK Jni_make_env_slot(PINIT_ONCE once, PVOID param, PVOID* context) { + (void)once; + (void)param; + (void)context; + Jni_env_slot = FlsAlloc(Jni_thread_exit); + return TRUE; +} + +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { + InitOnceExecuteOnce(&Jni_env_once, Jni_make_env_slot, NULL, NULL); + return Jni_env_slot != FLS_OUT_OF_INDEXES && FlsSetValue(Jni_env_slot, (PVOID)Jni_vm); +} #else #include static pthread_key_t Jni_env_key; +static int Jni_env_key_ok = 0; static pthread_once_t Jni_env_once = PTHREAD_ONCE_INIT; -static void Jni_detach(void* env) { - (void)env; - if (Jni_vm != NULL) { +static void Jni_thread_exit(void* value) { + if (value != NULL && Jni_vm != NULL) { (*Jni_vm)->DetachCurrentThread(Jni_vm); } } static void Jni_make_env_key(void) { - pthread_key_create(&Jni_env_key, Jni_detach); + Jni_env_key_ok = pthread_key_create(&Jni_env_key, Jni_thread_exit) == 0; } -static void Jni_remember_attach(void) { +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { pthread_once(&Jni_env_once, Jni_make_env_key); - pthread_setspecific(Jni_env_key, (void*)Jni_vm); + return Jni_env_key_ok && pthread_setspecific(Jni_env_key, (void*)Jni_vm) == 0; } #endif /* The calling thread's JNIEnv, attaching the thread if needed; NULL when - the JVM is unavailable. */ -static JNIEnv* Jni_env(void) { + the JVM is unavailable. `*detach` is set when the caller must hand it + back with Jni_env_done because the thread can't detach itself at exit. */ +static JNIEnv* Jni_env(int* detach) { JNIEnv* env = NULL; + *detach = 0; if (Jni_vm == NULL) { return NULL; } @@ -179,10 +255,17 @@ static JNIEnv* Jni_env(void) { if ((*Jni_vm)->AttachCurrentThreadAsDaemon(Jni_vm, (void*)&env, NULL) != JNI_OK) { return NULL; } - Jni_remember_attach(); + *detach = !Jni_remember_attach(); return env; } +/* Ends a use of Jni_env's JNIEnv, detaching the thread if Jni_env said so. */ +static void Jni_env_done(int detach) { + if (detach) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + /* Async completions deliver to the pinned Kotlin `NativeCompletion`. */ static jmethodID Jni_on_unit = NULL; static jmethodID Jni_on_boolean = NULL; @@ -219,8 +302,8 @@ static jint Jni_load_async(JNIEnv* env) { /* Starts a completion on the producer thread: returns the JNIEnv with a local frame pushed, or NULL when the call failed (the failure, or the loss of the JVM, has then been handled and `context` released). */ -static JNIEnv* Jni_complete_begin(void* context, docs_everywhere_error* err) { - JNIEnv* env = Jni_env(); +static JNIEnv* Jni_complete_begin(void* context, docs_everywhere_error* err, int* detach) { + JNIEnv* env = Jni_env(detach); if (env == NULL) { docs_everywhere_error_free(err); return NULL; @@ -228,6 +311,7 @@ static JNIEnv* Jni_complete_begin(void* context, docs_everywhere_error* err) { if ((*env)->PushLocalFrame(env, 8) != 0) { (*env)->ExceptionClear(env); docs_everywhere_error_free(err); + Jni_env_done(*detach); return NULL; } if (err == NULL) { @@ -248,18 +332,20 @@ static JNIEnv* Jni_complete_begin(void* context, docs_everywhere_error* err) { } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(*detach); return NULL; } /* Finishes a successful completion: reports anything the Kotlin side threw (it has no caller to propagate to) and releases `context`. */ -static void Jni_complete_end(JNIEnv* env, void* context) { +static void Jni_complete_end(JNIEnv* env, void* context, int detach) { if ((*env)->ExceptionCheck(env)) { (*env)->ExceptionDescribe(env); (*env)->ExceptionClear(env); } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(detach); } JNIEXPORT jlong JNICALL Java_docs_1everywhere_JniBridge_cancel_1token_1create(JNIEnv* env, jclass cls) { @@ -282,18 +368,14 @@ JNIEXPORT void JNICALL Java_docs_1everywhere_JniBridge_cancel_1token_1destroy(JN /* Callback interfaces: the producer holds a global reference to the Kotlin implementation as `ctx`; trampolines call the matching static dispatch - method on JniBridge from whatever thread the producer uses. */ -static jmethodID Jni_describe = NULL; - -static jint Jni_load_callbacks(JNIEnv* env) { - Jni_describe = (*env)->GetStaticMethodID(env, Jni_bridge, "describe", "(Ljava/lang/Throwable;)[B"); - return Jni_describe != NULL ? JNI_OK : JNI_ERR; -} + shim on JniBridge from whatever thread the producer uses. The shim never + lets an exception escape: it reports a failure itself through `out_err` + (with the two exports below) and returns a zero value. */ /* Enters a trampoline: the JNIEnv with a local frame pushed, or NULL after reporting the failure through `out_err`. */ -static JNIEnv* Jni_callback_begin(docs_everywhere_error* out_err) { - JNIEnv* env = Jni_env(); +static JNIEnv* Jni_callback_begin(docs_everywhere_error* out_err, int* detach) { + JNIEnv* env = Jni_env(detach); if (env == NULL) { docs_everywhere_error_set(out_err, -4, "could not attach the calling thread to the JVM"); return NULL; @@ -301,73 +383,103 @@ static JNIEnv* Jni_callback_begin(docs_everywhere_error* out_err) { if ((*env)->PushLocalFrame(env, 16) != 0) { (*env)->ExceptionClear(env); docs_everywhere_error_set(out_err, -4, "JNI local frame exhausted"); + Jni_env_done(*detach); return NULL; } return env; } -/* Leaves a trampoline. When the implementation threw, reports its - `toString()` through `out_err` with code -4 and returns 1; nothing ever - unwinds through the producer's frames. */ -static int Jni_callback_end(JNIEnv* env, docs_everywhere_error* out_err) { - int failed = 0; +/* Leaves a trampoline. An exception still pending means the dispatch itself + failed (the shim catches everything the implementation throws), which + reaches the producer as code -4; nothing ever unwinds through its frames. */ +static void Jni_callback_end(JNIEnv* env, docs_everywhere_error* out_err, int detach) { if ((*env)->ExceptionCheck(env)) { - failed = 1; - jthrowable ex = (*env)->ExceptionOccurred(env); (*env)->ExceptionClear(env); - jbyteArray text = (jbyteArray)(*env)->CallStaticObjectMethod(env, Jni_bridge, Jni_describe, ex); - char* message = NULL; - if ((*env)->ExceptionCheck(env)) { - (*env)->ExceptionClear(env); - } else if (text != NULL) { - jsize len = (*env)->GetArrayLength(env, text); - message = (char*)malloc((size_t)len + 1u); - if (message != NULL) { - (*env)->GetByteArrayRegion(env, text, 0, len, (jbyte*)message); - message[len] = '\0'; - } - } - docs_everywhere_error_set(out_err, -4, message != NULL ? message : "callback implementation threw"); - free(message); + docs_everywhere_error_set(out_err, -4, "the JVM failed to dispatch the callback"); } (*env)->PopLocalFrame(env, NULL); - return failed; + Jni_env_done(detach); } /* The vtable `free` entry: drops the producer's reference to the Kotlin - implementation. */ + implementation, from whatever thread the producer releases it on. */ static void Jni_release_callback(void* ctx) { - JNIEnv* env = Jni_env(); - if (env != NULL && ctx != NULL) { - (*env)->DeleteGlobalRef(env, (jobject)ctx); + int detach = 0; + JNIEnv* env = Jni_env(&detach); + if (env != NULL) { + if (ctx != NULL) { + (*env)->DeleteGlobalRef(env, (jobject)ctx); + } + Jni_env_done(detach); } } +/* `JniBridge.error_set`: reports a failed callback through the trampoline's + `out_err` (the message is cut at its first NUL). */ +JNIEXPORT void JNICALL Java_docs_1everywhere_JniBridge_error_1set(JNIEnv* env, jclass cls, jlong err, jint code, jbyteArray message) { + (void)cls; + Jni_bytes m = Jni_borrow_bytes(env, message); + char* text = (char*)malloc(m.len + 1u); + if (text != NULL) { + if (m.len > 0) { + memcpy(text, m.ptr, m.len); + } + text[m.len] = '\0'; + } + Jni_release_bytes(env, &m); + docs_everywhere_error_set((docs_everywhere_error*)(intptr_t)err, (int32_t)code, text != NULL ? text : "callback failed"); + free(text); +} + +/* `JniBridge.error_set_payload`: attaches a typed error's fields (a value + buffer) to the failure `error_set` reported. */ +JNIEXPORT void JNICALL Java_docs_1everywhere_JniBridge_error_1set_1payload(JNIEnv* env, jclass cls, jlong err, jbyteArray payload) { + (void)cls; + Jni_bytes p = Jni_borrow_bytes(env, payload); + docs_everywhere_error_set_payload((docs_everywhere_error*)(intptr_t)err, p.ptr, p.len); + Jni_release_bytes(env, &p); +} + static jmethodID Jni_mid_docs_SavedListener_on_saved = NULL; static void Jni_docs_SavedListener_on_saved(void* ctx, int64_t p_doc_id, int32_t p_revision, docs_everywhere_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return; } - (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_docs_SavedListener_on_saved, (jobject)ctx, (jlong)p_doc_id, (jint)p_revision); - Jni_callback_end(env, out_err); -} - -static const docs_everywhere_docs_SavedListener_vtable Jni_docs_SavedListener_vtable = {Jni_docs_SavedListener_on_saved, Jni_release_callback}; + (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_docs_SavedListener_on_saved, (jobject)ctx, (jlong)(intptr_t)out_err, (jlong)p_doc_id, (jint)p_revision); + Jni_callback_end(env, out_err, detach); +} + +static const docs_everywhere_docs_SavedListener_vtable Jni_docs_SavedListener_vtable = {sizeof(docs_everywhere_docs_SavedListener_vtable), 0, Jni_release_callback, Jni_docs_SavedListener_on_saved}; + +static const Jni_contract_entry Jni_contract_docs[] = { + {UINT64_C(0x00b2187fd75634ec), UINT64_C(0x2263c6f787bf6081), "docs.publish_async"}, + {UINT64_C(0x12fc242b9dc6444c), UINT64_C(0x96405f484a642b31), "docs.Notebook.insert"}, + {UINT64_C(0x2213822ed05cd5bd), UINT64_C(0x26c5833392aa63ce), "docs.Notebook.new"}, + {UINT64_C(0x2bd762bcdaa207ff), UINT64_C(0x1b0ebab2d04c49b0), "docs.SavedListener"}, + {UINT64_C(0x2e086c45e9d617de), UINT64_C(0x4d43bf32358c01ff), "docs.Notebook.find"}, + {UINT64_C(0x4817dc98197ad960), UINT64_C(0xf8017b5a7d58b2c7), "docs.Notebook.size"}, + {UINT64_C(0x614de47452d5a55c), UINT64_C(0xd9aae1472494ffef), "docs.get_document"}, + {UINT64_C(0x67c5cb9fc119bcd5), UINT64_C(0x08d5b25a1a22a7fa), "docs.Document"}, + {UINT64_C(0x8d484bf2d78c9b70), UINT64_C(0xe9093cf96912194c), "docs.Notebook.max_documents"}, + {UINT64_C(0xaea61d6451691c67), UINT64_C(0xe4704128b3524b3d), "docs.watch"}, + {UINT64_C(0xd68d6ad1540cdec5), UINT64_C(0x01afeb40b0a8bfe1), "docs.Notebook"}, + {UINT64_C(0xdb0601444a4760cc), UINT64_C(0xe56c69629b444797), "docs.create_document"}, + {UINT64_C(0xe12d6f4838347b3e), UINT64_C(0xa9bab86646d84f51), "docs.Status"}, + {UINT64_C(0xee2f8d25b3313b50), UINT64_C(0x9691232f35439a2b), "docs.DocsErrors"}, +}; static jint Jni_load(JNIEnv* env) { (void)env; - if (docs_everywhere_docs_checksum() != UINT64_C(0x7950eec1e9cfc819)) { - return Jni_load_error(env, "docs_everywhere: module 'docs' does not match these bindings (its contract checksum differs); regenerate the bindings or rebuild the library"); - } - if (Jni_load_async(env) != JNI_OK) { + if (Jni_check_contract(env, docs_everywhere_docs_contract, Jni_contract_docs, sizeof Jni_contract_docs / sizeof Jni_contract_docs[0]) != JNI_OK) { return JNI_ERR; } - if (Jni_load_callbacks(env) != JNI_OK) { + if (Jni_load_async(env) != JNI_OK) { return JNI_ERR; } - Jni_mid_docs_SavedListener_on_saved = (*env)->GetStaticMethodID(env, Jni_bridge, "docs_SavedListener_on_saved", "(Ldocs_everywhere/SavedListener;JI)V"); + Jni_mid_docs_SavedListener_on_saved = (*env)->GetStaticMethodID(env, Jni_bridge, "docs_SavedListener_on_saved", "(Ldocs_everywhere/SavedListener;JJI)V"); if (Jni_mid_docs_SavedListener_on_saved == NULL) { return JNI_ERR; } @@ -415,12 +527,13 @@ JNIEXPORT jbyteArray JNICALL Java_docs_1everywhere_JniBridge_docs_1get_1document } static void Jni_done_docs_publish_async(void* context, docs_everywhere_error* err, bool result) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_boolean, result ? JNI_TRUE : JNI_FALSE); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_docs_1everywhere_JniBridge_docs_1publish_1async(JNIEnv* env, jclass cls, jlong p_id, jobject completion) { diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Async_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Async_kt.snap deleted file mode 100644 index dac0ddce..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Async_kt.snap +++ /dev/null @@ -1,124 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package docs_everywhere - -import kotlin.coroutines.cancellation.CancellationException -import kotlin.coroutines.resume -import kotlin.coroutines.resumeWithException -import kotlinx.coroutines.CancellableContinuation -import kotlinx.coroutines.suspendCancellableCoroutine - -/** - * Bridges one native async call to a suspended coroutine. The JNI shim pins - * this object with a global reference and calls exactly one `on*` method, - * from whichever producer thread completes the call. - */ -internal class NativeCompletion( - private val cont: CancellableContinuation, - private val domain: Int, - private val convert: (Any?) -> T, - cancellable: Boolean, -) { - // The native cancel token, or 0 when the call isn't cancellable or has - // completed. Guarded by `this` so a late cancel never touches a freed - // token; a completion that lands inside `cancel` (on the same thread) - // defers the release until the cancel returns. - private var token: Long = if (cancellable) JniBridge.cancel_token_create() else 0L - private var cancelling = false - private var released = false - - val cancelToken: Long - get() = synchronized(this) { token } - - fun cancel() { - synchronized(this) { - if (token == 0L) return - cancelling = true - try { - JniBridge.cancel_token_cancel(token) - } finally { - cancelling = false - } - if (released) destroyToken() - } - } - - fun releaseToken() { - synchronized(this) { - if (cancelling) released = true else destroyToken() - } - } - - private fun destroyToken() { - if (token != 0L) JniBridge.cancel_token_destroy(token) - token = 0L - } - - fun onUnit() = succeed(Unit) - - fun onBoolean(value: Boolean) = succeed(value) - - fun onByte(value: Byte) = succeed(value) - - fun onShort(value: Short) = succeed(value) - - fun onInt(value: Int) = succeed(value) - - fun onLong(value: Long) = succeed(value) - - fun onFloat(value: Float) = succeed(value) - - fun onDouble(value: Double) = succeed(value) - - fun onBytes(value: ByteArray) = succeed(value) - - fun onError(code: Int, message: ByteArray, payload: ByteArray?) { - releaseToken() - val error = if (code == -5) { - CancellationException(decodeUtf8(message)) - } else { - JniBridge.error(domain, code, message, payload) - } - cont.resumeWithException(error) - } - - private fun succeed(raw: Any?) { - releaseToken() - val result = try { - convert(raw) - } catch (t: Throwable) { - cont.resumeWithException(t) - return - } - cont.resume(result) - } -} - -/** - * Starts a native async call and suspends until it completes. [launch] gets - * the cancel token (0 unless [cancellable]) and the completion to hand to the - * shim. Cancelling the coroutine cancels the native call; a cancelled call - * (code -5) resumes with [CancellationException]. - */ -internal suspend fun awaitNative( - cancellable: Boolean, - domain: Int, - convert: (Any?) -> T, - launch: (Long, NativeCompletion) -> Unit, -): T = suspendCancellableCoroutine { cont -> - val completion = NativeCompletion(cont, domain, convert, cancellable) - cont.invokeOnCancellation { completion.cancel() } - try { - launch(completion.cancelToken, completion) - } catch (t: Throwable) { - completion.releaseToken() - throw t - } -} - -// END Async.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Buffers_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Buffers_kt.snap deleted file mode 100644 index 2abe38cb..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Buffers_kt.snap +++ /dev/null @@ -1,255 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package docs_everywhere - -/** - * Growable little-endian writer for the value-buffer wire format that - * records, rich enums, optionals, lists, and maps cross the ABI in. - */ -internal class BufferWriter { - private var buf = ByteArray(64) - private var len = 0 - private var minted: ArrayList Unit>>? = null - - private fun reserve(extra: Int) { - if (len + extra <= buf.size) return - var cap = buf.size * 2 - while (cap < len + extra) cap *= 2 - buf = buf.copyOf(cap) - } - - fun toByteArray(): ByteArray = buf.copyOf(len) - - fun writeBool(v: Boolean) { - reserve(1) - buf[len++] = if (v) 1 else 0 - } - - fun writeI8(v: Byte) { - reserve(1) - buf[len++] = v - } - - fun writeU8(v: Byte) = writeI8(v) - - fun writeI16(v: Short) { - reserve(2) - val b = v.toInt() - buf[len++] = b.toByte() - buf[len++] = (b shr 8).toByte() - } - - fun writeU16(v: Short) = writeI16(v) - - fun writeI32(v: Int) { - reserve(4) - buf[len++] = v.toByte() - buf[len++] = (v shr 8).toByte() - buf[len++] = (v shr 16).toByte() - buf[len++] = (v shr 24).toByte() - } - - fun writeU32(v: Long) = writeI32(v.toInt()) - - fun writeI64(v: Long) { - writeI32(v.toInt()) - writeI32((v ushr 32).toInt()) - } - - fun writeU64(v: Long) = writeI64(v) - - fun writeF32(v: Float) = writeI32(v.toRawBits()) - - fun writeF64(v: Double) = writeI64(v.toRawBits()) - - fun writeString(v: String) = writeBytes(encodeUtf8(v)) - - fun writeBytes(v: ByteArray) { - writeI32(v.size) - reserve(v.size) - v.copyInto(buf, len) - len += v.size - } - - /** - * An object token: one new strong reference the reader adopts, which - * [abandon] releases with [release] if the encoding fails. - */ - fun writeObject(token: Long, release: (Long) -> Unit) { - val list = minted ?: ArrayList Unit>>().also { minted = it } - list.add(token to release) - writeI64(token) - } - - /** Releases every object token written so far, for an encoding that failed midway. */ - fun abandon() { - minted?.forEach { (token, release) -> release(token) } - minted = null - } - - inline fun writeOptional(v: T?, write: (T) -> Unit) { - if (v == null) { - writeBool(false) - } else { - writeBool(true) - write(v) - } - } - - inline fun writeList(v: List, write: (T) -> Unit) { - writeI32(v.size) - for (e in v) write(e) - } - - inline fun writeMap(v: Map, writeKey: (K) -> Unit, writeValue: (V) -> Unit) { - writeI32(v.size) - for ((k, e) in v) { - writeKey(k) - writeValue(e) - } - } -} - -/** - * Validating little-endian reader for the value-buffer wire format. A - * truncated buffer, an invalid flag byte, a zero object token, an oversized - * length, or trailing bytes raise [FfiException] code -3. - */ -internal class BufferReader(private val buf: ByteArray) { - private var pos = 0 - - private fun malformed(detail: String): Nothing = - throw FfiException(-3, "malformed value buffer: $detail") - - private fun take(n: Int): Int { - if (n > buf.size - pos) malformed("truncated buffer") - val at = pos - pos += n - return at - } - - fun readBool(): Boolean = when (buf[take(1)].toInt()) { - 0 -> false - 1 -> true - else -> malformed("invalid bool byte") - } - - fun readI8(): Byte = buf[take(1)] - - fun readU8(): Byte = readI8() - - fun readI16(): Short { - val at = take(2) - return ((buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)).toShort() - } - - fun readU16(): Short = readI16() - - fun readI32(): Int { - val at = take(4) - return (buf[at].toInt() and 0xFF) or - ((buf[at + 1].toInt() and 0xFF) shl 8) or - ((buf[at + 2].toInt() and 0xFF) shl 16) or - ((buf[at + 3].toInt() and 0xFF) shl 24) - } - - fun readU32(): Long = readI32().toLong() and 0xFFFFFFFFL - - fun readI64(): Long { - val lo = readI32().toLong() and 0xFFFFFFFFL - val hi = readI32().toLong() - return lo or (hi shl 32) - } - - fun readU64(): Long = readI64() - - fun readF32(): Float = Float.fromBits(readI32()) - - fun readF64(): Double = Double.fromBits(readI64()) - - private fun readLen(): Int { - val n = readI32() - if (n < 0 || n > buf.size - pos) malformed("length prefix exceeds remaining bytes") - return n - } - - fun readString(): String { - val n = readLen() - val at = take(n) - return try { - Charsets.UTF_8.newDecoder().decode(java.nio.ByteBuffer.wrap(buf, at, n)).toString() - } catch (e: java.nio.charset.CharacterCodingException) { - malformed("string is not valid UTF-8") - } - } - - fun readBytes(): ByteArray { - val n = readLen() - val at = take(n) - return buf.copyOfRange(at, at + n) - } - - /** An object token: one strong reference the caller adopts. */ - fun readObject(): Long { - val token = readI64() - if (token == 0L) malformed("null object token") - return token - } - - inline fun readOptional(read: () -> T): T? = if (readBool()) read() else null - - inline fun readList(read: () -> T): List { - val n = readCount() - val out = ArrayList(minOf(n, 4096)) - repeat(n) { out.add(read()) } - return out - } - - inline fun readMap(readKey: () -> K, readValue: () -> V): Map { - val n = readCount() - val out = LinkedHashMap(minOf(n, 4096)) - repeat(n) { - val k = readKey() - out[k] = readValue() - } - return out - } - - /** An element count; elements may encode to zero bytes, so only the sign is checked. */ - fun readCount(): Int { - val n = readI32() - if (n < 0) malformed("negative element count") - return n - } - - fun expectEnd() { - if (pos != buf.size) malformed("trailing bytes after value") - } -} - -/** Encodes one value into a fresh value buffer. */ -internal inline fun encodeBuffer(write: (BufferWriter) -> Unit): ByteArray { - val w = BufferWriter() - try { - write(w) - } catch (t: Throwable) { - w.abandon() - throw t - } - return w.toByteArray() -} - -/** Decodes one value from [bytes], rejecting malformed or trailing bytes. */ -internal inline fun decodeBuffer(bytes: ByteArray, read: (BufferReader) -> T): T { - val r = BufferReader(bytes) - val v = read(r) - r.expectEnd() - return v -} - -// END Buffers.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Codecs_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Codecs_kt.snap new file mode 100644 index 00000000..59e7bd23 --- /dev/null +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Codecs_kt.snap @@ -0,0 +1,18 @@ +--- +source: crates/weaveffi-cli/tests/snapshots.rs +--- +// Generated by WeaveFFI [VERSION] +// DO NOT EDIT. Your changes will be overwritten. +// To regenerate: weaveffi generate + +package docs_everywhere + +internal fun packListOfString(_w: BufferWriter, _v: List) = _w.writeList(_v) { _w.writeString(it) } + +internal fun unpackListOfString(_r: BufferReader): List = _r.readList { _r.readString() } + +internal fun packOptionalOfDocument(_w: BufferWriter, _v: Document?) = _w.writeOptional(_v) { packDocument(_w, it) } + +internal fun unpackOptionalOfDocument(_r: BufferReader): Document? = _r.readOptional { unpackDocument(_r) } + +// END Codecs.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Docs_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Docs_kt.snap index c6802312..aa433be9 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Docs_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Docs_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package docs_everywhere @@ -40,7 +40,7 @@ enum class Status(val value: Int) { companion object { /** The variant whose ABI value is [value]. */ fun fromValue(value: Int): Status = entries.firstOrNull { it.value == value } - ?: throw FfiException(-3, "unknown Status value $value") + ?: throw NativeBugException(-3, "unknown Status value $value") } } @@ -64,14 +64,14 @@ internal fun packDocument(_w: BufferWriter, _v: Document) { _w.writeI64(_v.id) _w.writeString(_v.title) _w.writeI32(_v.status.value) - _w.writeList(_v.tags) { _v0 -> _w.writeString(_v0) } + packListOfString(_w, _v.tags) } internal fun unpackDocument(_r: BufferReader): Document = Document( _r.readI64(), _r.readString(), Status.fromValue(_r.readI32()), - _r.readList { _r.readString() }, + unpackListOfString(_r), ) /** @@ -138,6 +138,7 @@ class Notebook private constructor(address: Long) : AutoCloseable { } } +/** Document storage: create, publish, and observe documents. */ object Docs { /** * Create a brand new document. @@ -147,19 +148,19 @@ object Docs { * @param title Human-readable title for the new document * @param tags Initial set of tags (may be empty) */ - fun createDocument(title: String, tags: List): Document = decodeBuffer(JniBridge.docs_create_document(encodeUtf8(title), encodeBuffer { _w -> _w.writeList(tags) { _v0 -> _w.writeString(_v0) } })) { _r -> unpackDocument(_r) } + fun createDocument(title: String, tags: List): Document = decodeBuffer(JniBridge.docs_create_document(encodeUtf8(title), encodeBuffer { _w -> packListOfString(_w, tags) })) { _r -> unpackDocument(_r) } /** * Look up a document by identifier * @param id Identifier returned from create_document */ - fun getDocument(id: Long): Document? = decodeBuffer(JniBridge.docs_get_document(id)) { _r -> _r.readOptional { unpackDocument(_r) } } + fun getDocument(id: Long): Document? = decodeBuffer(JniBridge.docs_get_document(id)) { _r -> unpackOptionalOfDocument(_r) } /** * Publish a document in the background * @param id Identifier of the document to publish */ - suspend fun publishAsync(id: Long): Boolean = awaitNative(false, 0, { it as Boolean }) { _, _done -> + suspend fun publishAsync(id: Long): Boolean = awaitNative(false, 0, { _raw -> _raw as Boolean }) { _, _done -> JniBridge.docs_publish_async(id, _done) } diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_JniBridge_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_JniBridge_kt.snap index 99527dfc..b671a538 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_JniBridge_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_JniBridge_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package docs_everywhere @@ -17,20 +17,35 @@ internal object JniBridge { NativeLibrary.ensureLoaded() } - /** The exception for a native failure; domain 0 is the generic one. */ + /** + * The exception for a native failure: through error domain [domain] for a + * throwing call, or a [NativeBugException] (domain 0) for one that can't fail. + */ @JvmStatic - @Suppress("UNUSED_PARAMETER") - fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): FfiException { + fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): Throwable { val text = decodeUtf8(message) return when (domain) { 1 -> DocsErrorsException.fromCode(code, text, payload) - else -> FfiException(code, text) + else -> NativeBugException(code, text) } } - /** The message a failed callback reports to the producer (code -4). */ - @JvmStatic - fun describe(error: Throwable): ByteArray = encodeUtf8(error.toString()) + /** + * Reports a failed callback through its vtable entry's `out_err` ([err]): + * [typed], a domain error the method is declared to throw, with its code and + * payload fields, and anything else as code -4 with its message. + */ + private fun fail(err: Long, error: Throwable, typed: FfiException?) { + if (typed != null) { + error_set(err, typed.code, encodeUtf8(typed.message ?: "")) + typed.encodePayload()?.let { error_set_payload(err, it) } + } else { + error_set(err, -4, encodeUtf8(error.message ?: error.toString())) + } + } + + @JvmStatic external fun error_set(err: Long, code: Int, message: ByteArray) + @JvmStatic external fun error_set_payload(err: Long, payload: ByteArray) @JvmStatic external fun debug_live(kind: Int): Long @JvmStatic external fun cancel_token_create(): Long @@ -49,7 +64,13 @@ internal object JniBridge { @JvmStatic external fun docs_Notebook_max_documents(): Int @JvmStatic - fun docs_SavedListener_on_saved(_impl: SavedListener, docId: Long, revision: Int) { _impl.onSaved(docId, revision) } + fun docs_SavedListener_on_saved(_impl: SavedListener, _err: Long, docId: Long, revision: Int) { + try { + _impl.onSaved(docId, revision) + } catch (_e: Throwable) { + fail(_err, _e, null) + } + } } // END JniBridge.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Runtime_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Runtime_kt.snap deleted file mode 100644 index 722c2842..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_docs_everywhere__src_main_kotlin_docs_everywhere_Runtime_kt.snap +++ /dev/null @@ -1,212 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package docs_everywhere - -import java.lang.ref.PhantomReference -import java.lang.ref.ReferenceQueue -import java.util.Collections -import java.util.concurrent.ConcurrentHashMap -import java.util.concurrent.atomic.AtomicLong - -/** - * A failure reported by the native library: a producer panic (code -2), a - * marshalling failure (-3), a callback implementation that threw (-4), or any - * code these bindings don't know. Typed error domains subclass it. - */ -open class FfiException(val code: Int, message: String) : Exception(message) - -/** - * A lazy sequence streamed from the native library, one element per step. - * The native iterator is released when the sequence is exhausted, when - * [close] is called, or once this object becomes unreachable, whichever - * comes first. - */ -class NativeIterator internal constructor( - address: Long, - release: (Long) -> Unit, - private val step: (Long) -> Any?, -) : Iterator, AutoCloseable { - private val handle = NativeCleaner.register(this, NativeHandle(address, release)) - private var lookahead: Any? = EMPTY - - override fun hasNext(): Boolean { - if (lookahead !== EMPTY) return true - if (handle.isClosed) return false - val item = handle.borrow(step) - if (item === DONE) { - close() - return false - } - lookahead = item - return true - } - - override fun next(): T { - if (!hasNext()) throw NoSuchElementException() - val item = lookahead - lookahead = EMPTY - @Suppress("UNCHECKED_CAST") - return item as T - } - - /** Releases the native iterator; safe to call more than once. */ - override fun close() = handle.close() - - internal companion object { - /** What a step function returns once the native sequence is exhausted. */ - val DONE = Any() - private val EMPTY = Any() - } -} - -/** Loads the producer library and then the JNI shim, exactly once. */ -internal object NativeLibrary { - private const val LIBRARY = "docs_everywhere" - private const val JNI_LIBRARY = "docs_everywhere_jni" - - init { - val explicit = System.getenv("DOCS_EVERYWHERE_LIBRARY") - if (!explicit.isNullOrEmpty()) { - System.load(explicit) - } else if (!loadBundled(LIBRARY)) { - // The shim's own link dependency usually finds the producer when - // it isn't on java.library.path, so a miss here is only fatal if - // loading the shim fails too. - try { - System.loadLibrary(LIBRARY) - } catch (ignored: UnsatisfiedLinkError) { - } - } - if (!loadBundled(JNI_LIBRARY)) System.loadLibrary(JNI_LIBRARY) - } - - fun ensureLoaded() {} - - /** - * Loads `base` from the `natives/-/` classpath resources a - * packaged desktop build bundles; false when it isn't bundled. - */ - private fun loadBundled(base: String): Boolean { - val os = System.getProperty("os.name").lowercase() - val arch = System.getProperty("os.arch").lowercase() - val osId = when { - os.contains("mac") || os.contains("darwin") -> "darwin" - os.contains("win") -> "windows" - else -> "linux" - } - val archId = if (arch == "aarch64" || arch == "arm64") "arm64" else "x64" - val file = when (osId) { - "darwin" -> "lib$base.dylib" - "windows" -> "$base.dll" - else -> "lib$base.so" - } - val stream = NativeLibrary::class.java.getResourceAsStream("/natives/$osId-$archId/$file") ?: return false - val dir = java.nio.file.Files.createTempDirectory("$base-natives") - val target = dir.resolve(file) - stream.use { java.nio.file.Files.copy(it, target) } - target.toFile().deleteOnExit() - dir.toFile().deleteOnExit() - System.load(target.toAbsolutePath().toString()) - return true - } -} - -/** - * One strong reference to a native object, released exactly once: by - * [close], or by [NativeCleaner] once the owning wrapper is unreachable. - * Every call borrows the address for its duration, and a close that races an - * in-flight call defers the release until the last borrow ends, so the object - * is never freed while native code is using it. - */ -internal class NativeHandle(val address: Long, private val release: (Long) -> Unit) { - // Bit 0 is the closed flag; the remaining bits count in-flight borrows. - private val state = AtomicLong(0L) - - val isClosed: Boolean get() = state.get() and 1L != 0L - - fun acquire(): Long { - while (true) { - val s = state.get() - check(s and 1L == 0L) { "native object used after close()" } - if (state.compareAndSet(s, s + 2L)) return address - } - } - - fun unacquire() { - if (state.addAndGet(-2L) == 1L) release(address) - } - - fun close() { - while (true) { - val s = state.get() - if (s and 1L != 0L) return - if (state.compareAndSet(s, s or 1L)) { - if (s == 0L) release(address) - return - } - } - } -} - -/** Runs [block] with the native address borrowed for its duration. */ -internal inline fun NativeHandle.borrow(block: (Long) -> R): R { - val address = acquire() - try { - return block(address) - } finally { - unacquire() - } -} - -/** [borrow] for an optional object: a missing object lends address 0. */ -internal inline fun NativeHandle?.borrowOrNull(block: (Long) -> R): R = - if (this == null) block(0L) else borrow(block) - -/** - * Releases the native reference of every wrapper that becomes unreachable - * without being closed. A daemon thread drains a phantom-reference queue, so - * this works on every Android API level and every JVM. - */ -internal object NativeCleaner { - private val queue = ReferenceQueue() - private val pending: MutableSet = Collections.newSetFromMap(ConcurrentHashMap()) - - private class Entry(owner: Any, val handle: NativeHandle, queue: ReferenceQueue) : - PhantomReference(owner, queue) - - init { - val thread = Thread({ drain() }, "docs_everywhere-cleaner") - thread.isDaemon = true - thread.start() - } - - /** Releases [handle] once [owner] is unreachable; returns [handle]. */ - fun register(owner: Any, handle: NativeHandle): NativeHandle { - pending.add(Entry(owner, handle, queue)) - return handle - } - - private fun drain() { - while (true) { - val entry = queue.remove() as Entry - pending.remove(entry) - try { - entry.handle.close() - } catch (_: Throwable) { - } - } - } -} - -/** UTF-8 bytes of [value], for a string crossing into native code. */ -internal fun encodeUtf8(value: String): ByteArray = value.toByteArray(Charsets.UTF_8) - -/** The string a native UTF-8 byte sequence spells. */ -internal fun decodeUtf8(bytes: ByteArray): String = String(bytes, Charsets.UTF_8) - -// END Runtime.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__build_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__build_gradle_kts.snap deleted file mode 100644 index fecef99e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__build_gradle_kts.snap +++ /dev/null @@ -1,63 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -plugins { - id("com.android.library") version "8.7.3" - id("org.jetbrains.kotlin.android") version "2.0.21" - `maven-publish` -} - -group = "edge_cases" -version = "0.1.0" - -android { - namespace = "edge_cases" - compileSdk = 35 - defaultConfig { - minSdk = 21 - consumerProguardFiles("consumer-rules.pro") - } - // Builds libedge_cases_jni.so per ABI, linked against the producer library in - // src/main/jniLibs// (see src/main/cpp/CMakeLists.txt). - externalNativeBuild { - cmake { - path = file("src/main/cpp/CMakeLists.txt") - } - } - compileOptions { - sourceCompatibility = JavaVersion.VERSION_1_8 - targetCompatibility = JavaVersion.VERSION_1_8 - } - publishing { - singleVariant("release") { - withSourcesJar() - } - } -} - -kotlin { - compilerOptions { - jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_1_8) - } -} - -dependencies { - implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.9.0") -} - -publishing { - publications { - register("release") { - artifactId = "edge_cases" - afterEvaluate { - from(components["release"]) - } - } - } -} - -// END build.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__consumer_rules_pro.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__consumer_rules_pro.snap deleted file mode 100644 index f9bfb9c5..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__consumer_rules_pro.snap +++ /dev/null @@ -1,11 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# The JNI shim looks classes and methods up by name. --keep class edge_cases.** { *; } - -# END consumer-rules.pro diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__settings_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__settings_gradle_kts.snap deleted file mode 100644 index 5ae1f746..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__settings_gradle_kts.snap +++ /dev/null @@ -1,25 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -pluginManagement { - repositories { - google() - mavenCentral() - gradlePluginPortal() - } -} - -dependencyResolutionManagement { - repositories { - google() - mavenCentral() - } -} - -rootProject.name = "edge_cases" - -// END settings.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_CMakeLists_txt.snap deleted file mode 100644 index 5bb79e2a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_CMakeLists_txt.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.18) -project(edge_cases_jni C) - -# The directory holding the producer library (libedge_cases). Android builds -# default to the per-ABI copy under src/main/jniLibs/. -set(EDGE_CASES_LIBRARY_DIR "" CACHE PATH "Directory holding the edge_cases producer library") -if(NOT EDGE_CASES_LIBRARY_DIR AND ANDROID) - set(EDGE_CASES_LIBRARY_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../jniLibs/${ANDROID_ABI}") -endif() -if(EDGE_CASES_LIBRARY_DIR) - find_library(EDGE_CASES_PRODUCER NAMES edge_cases PATHS "${EDGE_CASES_LIBRARY_DIR}" NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH REQUIRED) -else() - set(CMAKE_FIND_FRAMEWORK NEVER) - find_library(EDGE_CASES_PRODUCER NAMES edge_cases REQUIRED) -endif() - -add_library(edge_cases_jni SHARED edge_cases_jni.c) -target_include_directories(edge_cases_jni PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}") -target_link_libraries(edge_cases_jni PRIVATE "${EDGE_CASES_PRODUCER}") -if(NOT ANDROID) - find_package(JNI REQUIRED) - find_package(Threads REQUIRED) - target_include_directories(edge_cases_jni PRIVATE ${JNI_INCLUDE_DIRS}) - target_link_libraries(edge_cases_jni PRIVATE Threads::Threads) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_h.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_h.snap deleted file mode 100644 index 94864086..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_h.snap +++ /dev/null @@ -1,245 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef EDGE_CASES_H -#define EDGE_CASES_H - -#include -#include -#include - -#ifndef EDGE_CASES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef EDGE_CASES_BUILD -# define EDGE_CASES_API __declspec(dllexport) -# else -# define EDGE_CASES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define EDGE_CASES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define EDGE_CASES_API __attribute__((visibility("default"))) -# else -# define EDGE_CASES_API -# endif -#endif - -#ifndef EDGE_CASES_DEPRECATED -# if defined(_MSC_VER) -# define EDGE_CASES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define EDGE_CASES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define EDGE_CASES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports edge_cases_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define EDGE_CASES_ABI_VERSION 3u -EDGE_CASES_API uint32_t edge_cases_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by edge_cases_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct edge_cases_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} edge_cases_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -EDGE_CASES_API void edge_cases_error_set(edge_cases_error* err, int32_t code, const char* message); -EDGE_CASES_API void edge_cases_error_clear(edge_cases_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - edge_cases_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -EDGE_CASES_API void edge_cases_error_free(edge_cases_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -EDGE_CASES_API void edge_cases_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct edge_cases_cancel_token edge_cases_cancel_token; -EDGE_CASES_API edge_cases_cancel_token* edge_cases_cancel_token_create(void); -EDGE_CASES_API void edge_cases_cancel_token_cancel(edge_cases_cancel_token* token); -EDGE_CASES_API bool edge_cases_cancel_token_is_cancelled(const edge_cases_cancel_token* token); -EDGE_CASES_API void edge_cases_cancel_token_destroy(edge_cases_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -EDGE_CASES_API uint8_t* edge_cases_alloc(uint32_t size); -EDGE_CASES_API void edge_cases_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -EDGE_CASES_API uint64_t edge_cases_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define EDGE_CASES_EDGE_CASES_CHECKSUM 0x1567f441073bd4aaull -EDGE_CASES_API uint64_t edge_cases_edge_cases_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with edge_cases_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * edge_cases_buffer.h declares a C struct and codec for each. - */ - -/** Deprecated enum with non-contiguous values and keyword-like variants */ -typedef enum { edge_cases_edge_cases_Legacy_None = 0, edge_cases_edge_cases_Legacy_Default = 5, edge_cases_edge_cases_Legacy_Class = 10 } edge_cases_edge_cases_Legacy; -typedef enum { edge_cases_edge_cases_Mode_Fast = 1, edge_cases_edge_cases_Mode_Safe = 2 } edge_cases_edge_cases_Mode; -/** Error codes reported by throwing functions in the `edge_cases` module tree. */ -typedef enum { edge_cases_edge_cases_EdgeErrors_Busy = 7 } edge_cases_edge_cases_EdgeErrors; -typedef struct edge_cases_edge_cases_Widget edge_cases_edge_cases_Widget; -typedef struct edge_cases_edge_cases_Vault edge_cases_edge_cases_Vault; -typedef struct edge_cases_edge_cases_WidgetStreamIterator edge_cases_edge_cases_WidgetStreamIterator; -typedef struct edge_cases_edge_cases_StringStreamIterator edge_cases_edge_cases_StringStreamIterator; -typedef struct edge_cases_edge_cases_ScalarStreamIterator edge_cases_edge_cases_ScalarStreamIterator; -typedef struct edge_cases_edge_cases_ModeStreamIterator edge_cases_edge_cases_ModeStreamIterator; -/** - * Callback interface carrying composite payloads and objects - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_BatchListener_vtable { - /** Composite payloads cross as value buffers */ - void (*on_batch)(void* ctx, const uint8_t* items_ptr, size_t items_len, const uint8_t* raw_ptr, size_t raw_len, const uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err); - /** Objects cross as strong references the consumer must release */ - void (*on_widget)(void* ctx, edge_cases_edge_cases_Widget* widget, edge_cases_edge_cases_Widget* fallback, const uint8_t* children_ptr, size_t children_len, edge_cases_error* out_err); - void (*on_tick)(void* ctx, edge_cases_error* out_err); - /** Scalar return travels back through the vtable */ - bool (*should_continue)(void* ctx, int32_t class_, edge_cases_error* out_err); - /** Enum return travels back as its integer discriminant */ - edge_cases_edge_cases_Mode (*pick_mode)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_BatchListener_vtable; -/** - * Callback interface whose name and methods are keywords - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_Keyword_vtable { - void (*delete_)(void* ctx, int32_t self, edge_cases_error* out_err); - int32_t (*import)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_Keyword_vtable; -typedef void (*edge_cases_edge_cases_async_widget_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result); -typedef void (*edge_cases_edge_cases_async_void_callback)(void* context, edge_cases_error* err); -typedef void (*edge_cases_edge_cases_async_bytes_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_struct_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_list_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_optional_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_enum_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Mode result); -typedef void (*edge_cases_edge_cases_async_keywords_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_Widget_fetch_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: edge_cases -/** Interface with no constructor; instances come from make_widget */ -/** Void method that can fail */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_refresh(const edge_cases_edge_cases_Widget* self, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_rename(const edge_cases_edge_cases_Widget* self, const uint8_t* class_ptr, size_t class_len, const uint8_t* import_ptr, size_t import_len, size_t* out_len, edge_cases_error* out_err); -/** Async method with a struct result */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_fetch(const edge_cases_edge_cases_Widget* self, int64_t id, edge_cases_edge_cases_Widget_fetch_callback callback, void* context); -EDGE_CASES_DEPRECATED("Use rename instead") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_old_name(const edge_cases_edge_cases_Widget* self, size_t* out_len, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Widget_clone(const edge_cases_edge_cases_Widget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_destroy(edge_cases_edge_cases_Widget* self); - -/** Deprecated interface with a throwing constructor */ -/** Fails when the vault is busy */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_open(const uint8_t* path_ptr, size_t path_len, edge_cases_error* out_err); -/** Throwing static */ -EDGE_CASES_API bool edge_cases_edge_cases_Vault_probe(edge_cases_error* out_err); -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Vault_peek(const edge_cases_edge_cases_Vault* self, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_clone(const edge_cases_edge_cases_Vault* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Vault_destroy(edge_cases_edge_cases_Vault* self); - -/** No parameters and no result */ -EDGE_CASES_API void edge_cases_edge_cases_noop(edge_cases_error* out_err); -/** Function named after a keyword whose parameters are all keywords */ -EDGE_CASES_API int32_t edge_cases_edge_cases_select(int32_t class_, const uint8_t* from_ptr, size_t from_len, const uint8_t* func_ptr, size_t func_len, const uint8_t* range_ptr, size_t range_len, bool type_, edge_cases_error* out_err); -/** Optional list of optional structs, the deepest legal nesting */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_nested_optional(const uint8_t* items_ptr, size_t items_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_optional_params(const uint8_t* text_ptr, size_t text_len, const uint8_t* count_ptr, size_t count_len, const uint8_t* outer_ptr, size_t outer_len, const uint8_t* tags_ptr, size_t tags_len, const uint8_t* blob_ptr, size_t blob_len, const uint8_t* mode_ptr, size_t mode_len, size_t* out_len, edge_cases_error* out_err); -/** Two callback interface parameters on one function */ -EDGE_CASES_API void edge_cases_edge_cases_register_batch(void* listener_ctx, const edge_cases_edge_cases_BatchListener_vtable* listener_vtable, void* keyword_ctx, const edge_cases_edge_cases_Keyword_vtable* keyword_vtable, edge_cases_error* out_err); -/** Objects in every buffered position */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_widgets_everywhere(const uint8_t* list_ptr, size_t list_len, const uint8_t* by_name_ptr, size_t by_name_len, const uint8_t* sparse_ptr, size_t sparse_len, const uint8_t* holder_ptr, size_t holder_len, size_t* out_len, edge_cases_error* out_err); -/** Iterator of objects */ -EDGE_CASES_API edge_cases_edge_cases_WidgetStreamIterator* edge_cases_edge_cases_widget_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_WidgetStreamIterator_next(edge_cases_edge_cases_WidgetStreamIterator* iter, edge_cases_edge_cases_Widget** out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_WidgetStreamIterator_destroy(edge_cases_edge_cases_WidgetStreamIterator* iter); -/** Async returning an object */ -EDGE_CASES_API void edge_cases_edge_cases_async_widget(edge_cases_edge_cases_async_widget_callback callback, void* context); -/** Factory for an interface with no constructor */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_make_widget(const uint8_t* label_ptr, size_t label_len, edge_cases_error* out_err); -/** Optional interface return */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_find_widget(int64_t id, edge_cases_error* out_err); -/** Interface parameter, including an optional one */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_inspect_widget(const edge_cases_edge_cases_Widget* widget, const edge_cases_edge_cases_Widget* fallback, size_t* out_len, edge_cases_error* out_err); -/** Iterator of strings */ -EDGE_CASES_API edge_cases_edge_cases_StringStreamIterator* edge_cases_edge_cases_string_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_StringStreamIterator_next(edge_cases_edge_cases_StringStreamIterator* iter, const uint8_t** out_item, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_StringStreamIterator_destroy(edge_cases_edge_cases_StringStreamIterator* iter); -/** Iterator of scalars */ -EDGE_CASES_API edge_cases_edge_cases_ScalarStreamIterator* edge_cases_edge_cases_scalar_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ScalarStreamIterator_next(edge_cases_edge_cases_ScalarStreamIterator* iter, int32_t* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ScalarStreamIterator_destroy(edge_cases_edge_cases_ScalarStreamIterator* iter); -/** Iterator of enums */ -EDGE_CASES_API edge_cases_edge_cases_ModeStreamIterator* edge_cases_edge_cases_mode_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ModeStreamIterator_next(edge_cases_edge_cases_ModeStreamIterator* iter, edge_cases_edge_cases_Mode* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ModeStreamIterator_destroy(edge_cases_edge_cases_ModeStreamIterator* iter); -/** Async with no result */ -EDGE_CASES_API void edge_cases_edge_cases_async_void(edge_cases_edge_cases_async_void_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_bytes(uint32_t size, edge_cases_edge_cases_async_bytes_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_struct(edge_cases_edge_cases_async_struct_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_list(edge_cases_cancel_token* cancel_token, edge_cases_edge_cases_async_list_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_optional(edge_cases_edge_cases_async_optional_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_enum(edge_cases_edge_cases_async_enum_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_keywords(const uint8_t* class_ptr, size_t class_len, int32_t import, edge_cases_edge_cases_async_keywords_callback callback, void* context); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_keywords_roundtrip(const uint8_t* value_ptr, size_t value_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_DEPRECATED("Use Mode-based APIs") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_legacy_value(edge_cases_edge_cases_Legacy value, size_t* out_len, edge_cases_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // EDGE_CASES_H - -// END edge_cases.h diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_jni_c.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_jni_c.snap index 415540fc..70b317e5 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_jni_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_cpp_edge_cases_jni_c.snap @@ -1,14 +1,15 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #include #include #include #include +#include #include #include @@ -72,8 +73,9 @@ static inline jbyteArray Jni_take_bytes(JNIEnv* env, const uint8_t* ptr, size_t return out; } -/* Throws the Kotlin exception for `err` (mapped through domain `domain`, - 0 for the generic one) and clears `err`. */ +/* Throws the Kotlin exception for `err` (mapped through error domain + `domain`, or 0 for a call that can't fail, which raises the trap) and + clears `err`. */ static inline void Jni_throw(JNIEnv* env, edge_cases_error* err, jint domain) { const char* message = err->message != NULL ? err->message : ""; jbyteArray text = Jni_new_bytes(env, (const uint8_t*)message, strlen(message)); @@ -100,6 +102,50 @@ static jint Jni_load_error(JNIEnv* env, const char* message) { return JNI_ERR; } +/* One contract entry these bindings were generated with: a declaration's + id and signature hash, plus its dotted path for the load error. */ +typedef struct { + uint64_t id; + uint64_t hash; + const char* path; +} Jni_contract_entry; + +/* Checks that the library's contract table (from `contract`, sorted by id) + has every expected entry with an equal hash; entries only the library + has are fine. Fails loading with an error naming the first declaration + that's missing or changed. */ +static inline jint Jni_check_contract(JNIEnv* env, const edge_cases_contract_entry* (*contract)(size_t*), const Jni_contract_entry* expected, size_t count) { + size_t len = 0; + const edge_cases_contract_entry* table = contract(&len); + if (table == NULL) { + len = 0; + } + for (size_t i = 0; i < count; i++) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) { + lo = mid + 1; + } else { + hi = mid; + } + } + const char* problem = NULL; + if (lo == len || table[lo].id != expected[i].id) { + problem = "is missing from the library"; + } else if (table[lo].hash != expected[i].hash) { + problem = "changed since these bindings were generated"; + } + if (problem != NULL) { + char message[512]; + snprintf(message, sizeof message, "%s: %s %s", "edge_cases", expected[i].path, problem); + return Jni_load_error(env, message); + } + } + return JNI_OK; +} + /* Generated below: the contract checks and the method IDs the shim caches. */ static jint Jni_load(JNIEnv* env); @@ -110,15 +156,15 @@ JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { if ((*vm)->GetEnv(vm, (void**)&env, JNI_VERSION_1_6) != JNI_OK) { return JNI_ERR; } - if (edge_cases_abi_version() != 3u) { - return Jni_load_error(env, "edge_cases: the native library implements a different C ABI revision than these bindings (expected 3)"); + if (edge_cases_abi_version() != 4u) { + return Jni_load_error(env, "edge_cases: the native library implements a different C ABI revision than these bindings (expected 4)"); } jclass bridge = (*env)->FindClass(env, "edge_cases/JniBridge"); if (bridge == NULL) { return JNI_ERR; } Jni_bridge = (jclass)(*env)->NewGlobalRef(env, bridge); - Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ledge_cases/FfiException;"); + Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ljava/lang/Throwable;"); if (Jni_error == NULL) { return JNI_ERR; } @@ -136,38 +182,68 @@ JNIEXPORT jlong JNICALL Java_edge_1cases_JniBridge_debug_1live(JNIEnv* env, jcla /* Producer threads (async completions, callback calls, vtable `free`) are usually not JVM threads. The first time one calls in it attaches as a - daemon and keeps its JNIEnv; a thread-local destructor detaches it when - the thread exits, so a busy producer thread pays for one attach, not one - per call. */ + daemon, and a thread-local destructor (a pthread key, or a fiber-local + slot on Windows) detaches it when the thread exits, so a busy producer + thread pays for one attach, not one per call. If the destructor can't be + registered, the thread detaches again as soon as the call ends. */ #if defined(_WIN32) -static void Jni_remember_attach(void) {} +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#include + +static DWORD Jni_env_slot = FLS_OUT_OF_INDEXES; +static INIT_ONCE Jni_env_once = INIT_ONCE_STATIC_INIT; + +static VOID WINAPI Jni_thread_exit(PVOID value) { + if (value != NULL && Jni_vm != NULL) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + +static BOOL CALLBACK Jni_make_env_slot(PINIT_ONCE once, PVOID param, PVOID* context) { + (void)once; + (void)param; + (void)context; + Jni_env_slot = FlsAlloc(Jni_thread_exit); + return TRUE; +} + +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { + InitOnceExecuteOnce(&Jni_env_once, Jni_make_env_slot, NULL, NULL); + return Jni_env_slot != FLS_OUT_OF_INDEXES && FlsSetValue(Jni_env_slot, (PVOID)Jni_vm); +} #else #include static pthread_key_t Jni_env_key; +static int Jni_env_key_ok = 0; static pthread_once_t Jni_env_once = PTHREAD_ONCE_INIT; -static void Jni_detach(void* env) { - (void)env; - if (Jni_vm != NULL) { +static void Jni_thread_exit(void* value) { + if (value != NULL && Jni_vm != NULL) { (*Jni_vm)->DetachCurrentThread(Jni_vm); } } static void Jni_make_env_key(void) { - pthread_key_create(&Jni_env_key, Jni_detach); + Jni_env_key_ok = pthread_key_create(&Jni_env_key, Jni_thread_exit) == 0; } -static void Jni_remember_attach(void) { +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { pthread_once(&Jni_env_once, Jni_make_env_key); - pthread_setspecific(Jni_env_key, (void*)Jni_vm); + return Jni_env_key_ok && pthread_setspecific(Jni_env_key, (void*)Jni_vm) == 0; } #endif /* The calling thread's JNIEnv, attaching the thread if needed; NULL when - the JVM is unavailable. */ -static JNIEnv* Jni_env(void) { + the JVM is unavailable. `*detach` is set when the caller must hand it + back with Jni_env_done because the thread can't detach itself at exit. */ +static JNIEnv* Jni_env(int* detach) { JNIEnv* env = NULL; + *detach = 0; if (Jni_vm == NULL) { return NULL; } @@ -179,10 +255,17 @@ static JNIEnv* Jni_env(void) { if ((*Jni_vm)->AttachCurrentThreadAsDaemon(Jni_vm, (void*)&env, NULL) != JNI_OK) { return NULL; } - Jni_remember_attach(); + *detach = !Jni_remember_attach(); return env; } +/* Ends a use of Jni_env's JNIEnv, detaching the thread if Jni_env said so. */ +static void Jni_env_done(int detach) { + if (detach) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + /* Async completions deliver to the pinned Kotlin `NativeCompletion`. */ static jmethodID Jni_on_unit = NULL; static jmethodID Jni_on_boolean = NULL; @@ -219,8 +302,8 @@ static jint Jni_load_async(JNIEnv* env) { /* Starts a completion on the producer thread: returns the JNIEnv with a local frame pushed, or NULL when the call failed (the failure, or the loss of the JVM, has then been handled and `context` released). */ -static JNIEnv* Jni_complete_begin(void* context, edge_cases_error* err) { - JNIEnv* env = Jni_env(); +static JNIEnv* Jni_complete_begin(void* context, edge_cases_error* err, int* detach) { + JNIEnv* env = Jni_env(detach); if (env == NULL) { edge_cases_error_free(err); return NULL; @@ -228,6 +311,7 @@ static JNIEnv* Jni_complete_begin(void* context, edge_cases_error* err) { if ((*env)->PushLocalFrame(env, 8) != 0) { (*env)->ExceptionClear(env); edge_cases_error_free(err); + Jni_env_done(*detach); return NULL; } if (err == NULL) { @@ -248,18 +332,20 @@ static JNIEnv* Jni_complete_begin(void* context, edge_cases_error* err) { } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(*detach); return NULL; } /* Finishes a successful completion: reports anything the Kotlin side threw (it has no caller to propagate to) and releases `context`. */ -static void Jni_complete_end(JNIEnv* env, void* context) { +static void Jni_complete_end(JNIEnv* env, void* context, int detach) { if ((*env)->ExceptionCheck(env)) { (*env)->ExceptionDescribe(env); (*env)->ExceptionClear(env); } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(detach); } JNIEXPORT jlong JNICALL Java_edge_1cases_JniBridge_cancel_1token_1create(JNIEnv* env, jclass cls) { @@ -282,18 +368,14 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_cancel_1token_1destroy(JNIEnv* /* Callback interfaces: the producer holds a global reference to the Kotlin implementation as `ctx`; trampolines call the matching static dispatch - method on JniBridge from whatever thread the producer uses. */ -static jmethodID Jni_describe = NULL; - -static jint Jni_load_callbacks(JNIEnv* env) { - Jni_describe = (*env)->GetStaticMethodID(env, Jni_bridge, "describe", "(Ljava/lang/Throwable;)[B"); - return Jni_describe != NULL ? JNI_OK : JNI_ERR; -} + shim on JniBridge from whatever thread the producer uses. The shim never + lets an exception escape: it reports a failure itself through `out_err` + (with the two exports below) and returns a zero value. */ /* Enters a trampoline: the JNIEnv with a local frame pushed, or NULL after reporting the failure through `out_err`. */ -static JNIEnv* Jni_callback_begin(edge_cases_error* out_err) { - JNIEnv* env = Jni_env(); +static JNIEnv* Jni_callback_begin(edge_cases_error* out_err, int* detach) { + JNIEnv* env = Jni_env(detach); if (env == NULL) { edge_cases_error_set(out_err, -4, "could not attach the calling thread to the JVM"); return NULL; @@ -301,46 +383,61 @@ static JNIEnv* Jni_callback_begin(edge_cases_error* out_err) { if ((*env)->PushLocalFrame(env, 16) != 0) { (*env)->ExceptionClear(env); edge_cases_error_set(out_err, -4, "JNI local frame exhausted"); + Jni_env_done(*detach); return NULL; } return env; } -/* Leaves a trampoline. When the implementation threw, reports its - `toString()` through `out_err` with code -4 and returns 1; nothing ever - unwinds through the producer's frames. */ -static int Jni_callback_end(JNIEnv* env, edge_cases_error* out_err) { - int failed = 0; +/* Leaves a trampoline. An exception still pending means the dispatch itself + failed (the shim catches everything the implementation throws), which + reaches the producer as code -4; nothing ever unwinds through its frames. */ +static void Jni_callback_end(JNIEnv* env, edge_cases_error* out_err, int detach) { if ((*env)->ExceptionCheck(env)) { - failed = 1; - jthrowable ex = (*env)->ExceptionOccurred(env); (*env)->ExceptionClear(env); - jbyteArray text = (jbyteArray)(*env)->CallStaticObjectMethod(env, Jni_bridge, Jni_describe, ex); - char* message = NULL; - if ((*env)->ExceptionCheck(env)) { - (*env)->ExceptionClear(env); - } else if (text != NULL) { - jsize len = (*env)->GetArrayLength(env, text); - message = (char*)malloc((size_t)len + 1u); - if (message != NULL) { - (*env)->GetByteArrayRegion(env, text, 0, len, (jbyte*)message); - message[len] = '\0'; - } - } - edge_cases_error_set(out_err, -4, message != NULL ? message : "callback implementation threw"); - free(message); + edge_cases_error_set(out_err, -4, "the JVM failed to dispatch the callback"); } (*env)->PopLocalFrame(env, NULL); - return failed; + Jni_env_done(detach); } /* The vtable `free` entry: drops the producer's reference to the Kotlin - implementation. */ + implementation, from whatever thread the producer releases it on. */ static void Jni_release_callback(void* ctx) { - JNIEnv* env = Jni_env(); - if (env != NULL && ctx != NULL) { - (*env)->DeleteGlobalRef(env, (jobject)ctx); + int detach = 0; + JNIEnv* env = Jni_env(&detach); + if (env != NULL) { + if (ctx != NULL) { + (*env)->DeleteGlobalRef(env, (jobject)ctx); + } + Jni_env_done(detach); + } +} + +/* `JniBridge.error_set`: reports a failed callback through the trampoline's + `out_err` (the message is cut at its first NUL). */ +JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_error_1set(JNIEnv* env, jclass cls, jlong err, jint code, jbyteArray message) { + (void)cls; + Jni_bytes m = Jni_borrow_bytes(env, message); + char* text = (char*)malloc(m.len + 1u); + if (text != NULL) { + if (m.len > 0) { + memcpy(text, m.ptr, m.len); + } + text[m.len] = '\0'; } + Jni_release_bytes(env, &m); + edge_cases_error_set((edge_cases_error*)(intptr_t)err, (int32_t)code, text != NULL ? text : "callback failed"); + free(text); +} + +/* `JniBridge.error_set_payload`: attaches a typed error's fields (a value + buffer) to the failure `error_set` reported. */ +JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_error_1set_1payload(JNIEnv* env, jclass cls, jlong err, jbyteArray payload) { + (void)cls; + Jni_bytes p = Jni_borrow_bytes(env, payload); + edge_cases_error_set_payload((edge_cases_error*)(intptr_t)err, p.ptr, p.len); + Jni_release_bytes(env, &p); } static jmethodID Jni_mid_edge_cases_BatchListener_on_batch = NULL; @@ -350,120 +447,170 @@ static jmethodID Jni_mid_edge_cases_BatchListener_should_continue = NULL; static jmethodID Jni_mid_edge_cases_BatchListener_pick_mode = NULL; static void Jni_edge_cases_BatchListener_on_batch(void* ctx, const uint8_t* p_items_ptr, size_t p_items_len, const uint8_t* p_raw_ptr, size_t p_raw_len, const uint8_t* p_first_ptr, size_t p_first_len, edge_cases_edge_cases_Mode p_mode, edge_cases_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return; } - (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_on_batch, (jobject)ctx, Jni_new_bytes(env, p_items_ptr, p_items_len), Jni_new_bytes(env, p_raw_ptr, p_raw_len), Jni_new_bytes(env, p_first_ptr, p_first_len), (jint)p_mode); - Jni_callback_end(env, out_err); + (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_on_batch, (jobject)ctx, (jlong)(intptr_t)out_err, Jni_new_bytes(env, p_items_ptr, p_items_len), Jni_new_bytes(env, p_raw_ptr, p_raw_len), Jni_new_bytes(env, p_first_ptr, p_first_len), (jint)p_mode); + Jni_callback_end(env, out_err, detach); } static void Jni_edge_cases_BatchListener_on_widget(void* ctx, edge_cases_edge_cases_Widget* p_widget, edge_cases_edge_cases_Widget* p_fallback, const uint8_t* p_children_ptr, size_t p_children_len, edge_cases_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { + edge_cases_edge_cases_Widget_destroy(p_widget); + edge_cases_edge_cases_Widget_destroy(p_fallback); return; } - (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_on_widget, (jobject)ctx, (jlong)(intptr_t)p_widget, (jlong)(intptr_t)p_fallback, Jni_new_bytes(env, p_children_ptr, p_children_len)); - Jni_callback_end(env, out_err); + (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_on_widget, (jobject)ctx, (jlong)(intptr_t)out_err, (jlong)(intptr_t)p_widget, (jlong)(intptr_t)p_fallback, Jni_new_bytes(env, p_children_ptr, p_children_len)); + Jni_callback_end(env, out_err, detach); } static void Jni_edge_cases_BatchListener_on_tick(void* ctx, edge_cases_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return; } - (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_on_tick, (jobject)ctx); - Jni_callback_end(env, out_err); + (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_on_tick, (jobject)ctx, (jlong)(intptr_t)out_err); + Jni_callback_end(env, out_err, detach); } static bool Jni_edge_cases_BatchListener_should_continue(void* ctx, int32_t p_class, edge_cases_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return (bool)0; } - jboolean rv = (*env)->CallStaticBooleanMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_should_continue, (jobject)ctx, (jint)p_class); - if (Jni_callback_end(env, out_err)) { - return (bool)0; - } + jboolean rv = (*env)->CallStaticBooleanMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_should_continue, (jobject)ctx, (jlong)(intptr_t)out_err, (jint)p_class); + Jni_callback_end(env, out_err, detach); return rv == JNI_TRUE; } static edge_cases_edge_cases_Mode Jni_edge_cases_BatchListener_pick_mode(void* ctx, edge_cases_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return (edge_cases_edge_cases_Mode)0; } - jint rv = (*env)->CallStaticIntMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_pick_mode, (jobject)ctx); - if (Jni_callback_end(env, out_err)) { - return (edge_cases_edge_cases_Mode)0; - } + jint rv = (*env)->CallStaticIntMethod(env, Jni_bridge, Jni_mid_edge_cases_BatchListener_pick_mode, (jobject)ctx, (jlong)(intptr_t)out_err); + Jni_callback_end(env, out_err, detach); return (edge_cases_edge_cases_Mode)rv; } -static const edge_cases_edge_cases_BatchListener_vtable Jni_edge_cases_BatchListener_vtable = {Jni_edge_cases_BatchListener_on_batch, Jni_edge_cases_BatchListener_on_widget, Jni_edge_cases_BatchListener_on_tick, Jni_edge_cases_BatchListener_should_continue, Jni_edge_cases_BatchListener_pick_mode, Jni_release_callback}; +static const edge_cases_edge_cases_BatchListener_vtable Jni_edge_cases_BatchListener_vtable = {sizeof(edge_cases_edge_cases_BatchListener_vtable), 0, Jni_release_callback, Jni_edge_cases_BatchListener_on_batch, Jni_edge_cases_BatchListener_on_widget, Jni_edge_cases_BatchListener_on_tick, Jni_edge_cases_BatchListener_should_continue, Jni_edge_cases_BatchListener_pick_mode}; static jmethodID Jni_mid_edge_cases_Keyword_delete = NULL; static jmethodID Jni_mid_edge_cases_Keyword_import = NULL; static void Jni_edge_cases_Keyword_delete(void* ctx, int32_t p_self, edge_cases_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return; } - (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_Keyword_delete, (jobject)ctx, (jint)p_self); - Jni_callback_end(env, out_err); + (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_edge_cases_Keyword_delete, (jobject)ctx, (jlong)(intptr_t)out_err, (jint)p_self); + Jni_callback_end(env, out_err, detach); } static int32_t Jni_edge_cases_Keyword_import(void* ctx, edge_cases_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return (int32_t)0; } - jint rv = (*env)->CallStaticIntMethod(env, Jni_bridge, Jni_mid_edge_cases_Keyword_import, (jobject)ctx); - if (Jni_callback_end(env, out_err)) { - return (int32_t)0; - } + jint rv = (*env)->CallStaticIntMethod(env, Jni_bridge, Jni_mid_edge_cases_Keyword_import, (jobject)ctx, (jlong)(intptr_t)out_err); + Jni_callback_end(env, out_err, detach); return (int32_t)rv; } -static const edge_cases_edge_cases_Keyword_vtable Jni_edge_cases_Keyword_vtable = {Jni_edge_cases_Keyword_delete, Jni_edge_cases_Keyword_import, Jni_release_callback}; +static const edge_cases_edge_cases_Keyword_vtable Jni_edge_cases_Keyword_vtable = {sizeof(edge_cases_edge_cases_Keyword_vtable), 0, Jni_release_callback, Jni_edge_cases_Keyword_delete, Jni_edge_cases_Keyword_import}; + +static const Jni_contract_entry Jni_contract_edge_cases[] = { + {UINT64_C(0x056a86511785ba2b), UINT64_C(0x5cba3f16d57c6cdb), "edge_cases.make_widget"}, + {UINT64_C(0x07bfa9ecb2b720d2), UINT64_C(0x3813b3f57c64bf35), "edge_cases.EdgeErrors"}, + {UINT64_C(0x0aaa11867e37f991), UINT64_C(0xf090dd52146ebdf0), "edge_cases.Widget.refresh"}, + {UINT64_C(0x0f9701493898ae4c), UINT64_C(0x6944f47a275c96e8), "edge_cases.Inner"}, + {UINT64_C(0x1226b1adac179f7d), UINT64_C(0x0e95a00ab0bef637), "edge_cases.async_keywords"}, + {UINT64_C(0x16d36bef1541136c), UINT64_C(0x5ea7021ec324298f), "edge_cases.find_widget"}, + {UINT64_C(0x1b7324c55e980feb), UINT64_C(0xabf87fe372358f42), "edge_cases.Widget.old_name"}, + {UINT64_C(0x1cbd731467c2bab8), UINT64_C(0x4915b9989c2c9074), "edge_cases.Vault"}, + {UINT64_C(0x2c2a6de67d894b0a), UINT64_C(0x50a4496b1e7191ca), "edge_cases.Vault.open"}, + {UINT64_C(0x322b4cfcd2d6861c), UINT64_C(0xca7527a379ae8465), "edge_cases.Keywords"}, + {UINT64_C(0x3f8e23a5f32906f4), UINT64_C(0x8d4c908d796f130d), "edge_cases.Leaf"}, + {UINT64_C(0x42ab8a685457b97c), UINT64_C(0xe5ce30500cadb32d), "edge_cases.Vault.handle"}, + {UINT64_C(0x4a68540e2836d3d0), UINT64_C(0xa20e44c71e420ed2), "edge_cases.mode_stream"}, + {UINT64_C(0x4e030580ac0b111d), UINT64_C(0xb68b7a18d910e175), "edge_cases.legacy_value"}, + {UINT64_C(0x4f5e4e5291864ef3), UINT64_C(0x55a1a88b21a08f8b), "edge_cases.Legacy"}, + {UINT64_C(0x4f84b06171b4213d), UINT64_C(0xbaedcfdebb6a2820), "edge_cases.scalar_stream"}, + {UINT64_C(0x5067a5ff2d9ef910), UINT64_C(0xbfc79b18fc4714fc), "edge_cases.Widget"}, + {UINT64_C(0x59aff159112fbf5b), UINT64_C(0x18e4def57a8cfa49), "edge_cases.optional_params"}, + {UINT64_C(0x5daf65b98fa77c48), UINT64_C(0xc817a8cb202e59c2), "edge_cases.keywords_roundtrip"}, + {UINT64_C(0x5de8a5c9726c2c5a), UINT64_C(0x7c50b5b65f7a896d), "edge_cases.register_batch"}, + {UINT64_C(0x618a620309994c3c), UINT64_C(0x826ad0b1d208d723), "edge_cases.async_bytes"}, + {UINT64_C(0x648285cdba0a7b8e), UINT64_C(0xcf6e80e783a4d8be), "edge_cases.WidgetHolder"}, + {UINT64_C(0x649c78079824b10d), UINT64_C(0x5b12dd26e0634b69), "edge_cases.async_widget"}, + {UINT64_C(0x7da7a7357d122a37), UINT64_C(0xf544e025ffaa3d20), "edge_cases.async_void"}, + {UINT64_C(0x7e4015bc3a472e2d), UINT64_C(0xfe22291d90b82e4a), "edge_cases.widget_stream"}, + {UINT64_C(0x805276e5d81fdba7), UINT64_C(0xd9b64e360bd03b2b), "edge_cases.Outer"}, + {UINT64_C(0x80e1183326542df4), UINT64_C(0xe8a57e12cc6ed276), "edge_cases.select"}, + {UINT64_C(0x923ce5e9bcd769a1), UINT64_C(0x0b973f5dc07dda8b), "edge_cases.Vault.to_string"}, + {UINT64_C(0x98ca8c01f5239404), UINT64_C(0xfe11c93eb6b303ef), "edge_cases.Widget.rename"}, + {UINT64_C(0xa13e22e98ea278d4), UINT64_C(0xf1fae215d7387800), "edge_cases.widgets_everywhere"}, + {UINT64_C(0xa84a8c9bf2a881e1), UINT64_C(0xbe1dc840bf221996), "edge_cases.async_optional"}, + {UINT64_C(0xad958c478fbf205d), UINT64_C(0xe386b0716d24f19b), "edge_cases.Retired"}, + {UINT64_C(0xb2fc3964ef56ab39), UINT64_C(0x73c325c8bc351236), "edge_cases.async_list"}, + {UINT64_C(0xb3403af521cf0006), UINT64_C(0x6abc30d48206a3ba), "edge_cases.Widget.close"}, + {UINT64_C(0xb794304a22a406b8), UINT64_C(0xf6fb98711c516105), "edge_cases.async_struct"}, + {UINT64_C(0xba3936106a49c47d), UINT64_C(0xb7025f2fadd6288e), "edge_cases.Vault.peek"}, + {UINT64_C(0xc155b0c3cfb164f4), UINT64_C(0xc777ae57d92ebc47), "edge_cases.noop"}, + {UINT64_C(0xcdc18607777019dd), UINT64_C(0xb81a31e5c20aa7a8), "edge_cases.inspect_widget"}, + {UINT64_C(0xd164a82aeb64b8cc), UINT64_C(0xded02c68005c4737), "edge_cases.nested_optional"}, + {UINT64_C(0xd978ccad2468038f), UINT64_C(0x30c39c65d6235153), "edge_cases.Mode"}, + {UINT64_C(0xdbe9d947431c9124), UINT64_C(0x39537fb848fdf980), "edge_cases.BatchListener"}, + {UINT64_C(0xeae8b7862c08bb07), UINT64_C(0xfe12eeaa830452ea), "edge_cases.Keyword"}, + {UINT64_C(0xf1e155a05ae262b8), UINT64_C(0xb575eb818a423d42), "edge_cases.async_enum"}, + {UINT64_C(0xf22e210bdbce7343), UINT64_C(0x2a5e0f801bc8c768), "edge_cases.Widget.dispose"}, + {UINT64_C(0xf3c7cf675563ee6c), UINT64_C(0x0ecffeef399facc8), "edge_cases.Vault.probe"}, + {UINT64_C(0xfd34bc67acac379e), UINT64_C(0x576caa14f3c0901a), "edge_cases.Widget.fetch"}, + {UINT64_C(0xff2e9d2bff941786), UINT64_C(0x7e96764ef1f5fbaa), "edge_cases.string_stream"}, +}; static jint Jni_load(JNIEnv* env) { (void)env; - if (edge_cases_edge_cases_checksum() != UINT64_C(0x1567f441073bd4aa)) { - return Jni_load_error(env, "edge_cases: module 'edge_cases' does not match these bindings (its contract checksum differs); regenerate the bindings or rebuild the library"); - } - if (Jni_load_async(env) != JNI_OK) { + if (Jni_check_contract(env, edge_cases_edge_cases_contract, Jni_contract_edge_cases, sizeof Jni_contract_edge_cases / sizeof Jni_contract_edge_cases[0]) != JNI_OK) { return JNI_ERR; } - if (Jni_load_callbacks(env) != JNI_OK) { + if (Jni_load_async(env) != JNI_OK) { return JNI_ERR; } - Jni_mid_edge_cases_BatchListener_on_batch = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_on_batch", "(Ledge_cases/BatchListener;[B[B[BI)V"); + Jni_mid_edge_cases_BatchListener_on_batch = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_on_batch", "(Ledge_cases/BatchListener;J[B[B[BI)V"); if (Jni_mid_edge_cases_BatchListener_on_batch == NULL) { return JNI_ERR; } - Jni_mid_edge_cases_BatchListener_on_widget = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_on_widget", "(Ledge_cases/BatchListener;JJ[B)V"); + Jni_mid_edge_cases_BatchListener_on_widget = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_on_widget", "(Ledge_cases/BatchListener;JJJ[B)V"); if (Jni_mid_edge_cases_BatchListener_on_widget == NULL) { return JNI_ERR; } - Jni_mid_edge_cases_BatchListener_on_tick = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_on_tick", "(Ledge_cases/BatchListener;)V"); + Jni_mid_edge_cases_BatchListener_on_tick = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_on_tick", "(Ledge_cases/BatchListener;J)V"); if (Jni_mid_edge_cases_BatchListener_on_tick == NULL) { return JNI_ERR; } - Jni_mid_edge_cases_BatchListener_should_continue = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_should_continue", "(Ledge_cases/BatchListener;I)Z"); + Jni_mid_edge_cases_BatchListener_should_continue = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_should_continue", "(Ledge_cases/BatchListener;JI)Z"); if (Jni_mid_edge_cases_BatchListener_should_continue == NULL) { return JNI_ERR; } - Jni_mid_edge_cases_BatchListener_pick_mode = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_pick_mode", "(Ledge_cases/BatchListener;)I"); + Jni_mid_edge_cases_BatchListener_pick_mode = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_BatchListener_pick_mode", "(Ledge_cases/BatchListener;J)I"); if (Jni_mid_edge_cases_BatchListener_pick_mode == NULL) { return JNI_ERR; } - Jni_mid_edge_cases_Keyword_delete = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_Keyword_delete", "(Ledge_cases/Keyword;I)V"); + Jni_mid_edge_cases_Keyword_delete = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_Keyword_delete", "(Ledge_cases/Keyword;JI)V"); if (Jni_mid_edge_cases_Keyword_delete == NULL) { return JNI_ERR; } - Jni_mid_edge_cases_Keyword_import = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_Keyword_import", "(Ledge_cases/Keyword;)I"); + Jni_mid_edge_cases_Keyword_import = (*env)->GetStaticMethodID(env, Jni_bridge, "edge_cases_Keyword_import", "(Ledge_cases/Keyword;J)I"); if (Jni_mid_edge_cases_Keyword_import == NULL) { return JNI_ERR; } @@ -624,12 +771,13 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1WidgetStreamItera } static void Jni_done_edge_cases_async_widget(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_long, (jlong)(intptr_t)result); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1widget(JNIEnv* env, jclass cls, jobject completion) { @@ -776,12 +924,13 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1ModeStreamIterato } static void Jni_done_edge_cases_async_void(void* context, edge_cases_error* err) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_unit); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1void(JNIEnv* env, jclass cls, jobject completion) { @@ -791,27 +940,29 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1void(JNIEn } static void Jni_done_edge_cases_async_bytes(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } -JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1bytes(JNIEnv* env, jclass cls, jlong p_size, jobject completion) { +JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1bytes(JNIEnv* env, jclass cls, jint p_size, jobject completion) { void* context = (void*)(*env)->NewGlobalRef(env, completion); (void)cls; edge_cases_edge_cases_async_bytes((uint32_t)p_size, Jni_done_edge_cases_async_bytes, context); } static void Jni_done_edge_cases_async_struct(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1struct(JNIEnv* env, jclass cls, jobject completion) { @@ -821,12 +972,13 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1struct(JNI } static void Jni_done_edge_cases_async_list(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1list(JNIEnv* env, jclass cls, jlong cancel_token, jobject completion) { @@ -836,12 +988,13 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1list(JNIEn } static void Jni_done_edge_cases_async_optional(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1optional(JNIEnv* env, jclass cls, jobject completion) { @@ -851,12 +1004,13 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1optional(J } static void Jni_done_edge_cases_async_enum(void* context, edge_cases_error* err, edge_cases_edge_cases_Mode result) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_int, (jint)result); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1enum(JNIEnv* env, jclass cls, jobject completion) { @@ -866,12 +1020,13 @@ JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1enum(JNIEn } static void Jni_done_edge_cases_async_keywords(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1async_1keywords(JNIEnv* env, jclass cls, jbyteArray p_class, jint p_import, jobject completion) { @@ -935,12 +1090,13 @@ JNIEXPORT jbyteArray JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Widget_1ren } static void Jni_done_edge_cases_Widget_fetch(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Widget_1fetch(JNIEnv* env, jclass cls, jlong self, jlong p_id, jobject completion) { @@ -961,6 +1117,27 @@ JNIEXPORT jbyteArray JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Widget_1old return Jni_take_bytes(env, rv, out_len); } +JNIEXPORT jboolean JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Widget_1close(JNIEnv* env, jclass cls, jlong self) { + edge_cases_error err = {0, NULL, NULL, 0}; + (void)cls; + bool rv = edge_cases_edge_cases_Widget_close((const edge_cases_edge_cases_Widget*)(intptr_t)self, &err); + if (err.code != 0) { + Jni_throw(env, &err, 0); + return 0; + } + return rv ? JNI_TRUE : JNI_FALSE; +} + +JNIEXPORT void JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Widget_1dispose(JNIEnv* env, jclass cls, jlong self, jboolean p_force) { + edge_cases_error err = {0, NULL, NULL, 0}; + (void)cls; + edge_cases_edge_cases_Widget_dispose((const edge_cases_edge_cases_Widget*)(intptr_t)self, p_force == JNI_TRUE, &err); + if (err.code != 0) { + Jni_throw(env, &err, 0); + return; + } +} + JNIEXPORT jlong JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Vault_1open(JNIEnv* env, jclass cls, jbyteArray p_path) { edge_cases_error err = {0, NULL, NULL, 0}; (void)cls; @@ -985,6 +1162,17 @@ JNIEXPORT jlong JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Vault_1peek(JNIE return (jlong)(intptr_t)rv; } +JNIEXPORT jlong JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Vault_1handle(JNIEnv* env, jclass cls, jlong self) { + edge_cases_error err = {0, NULL, NULL, 0}; + (void)cls; + int64_t rv = edge_cases_edge_cases_Vault_handle((const edge_cases_edge_cases_Vault*)(intptr_t)self, &err); + if (err.code != 0) { + Jni_throw(env, &err, 0); + return 0; + } + return (jlong)rv; +} + JNIEXPORT jboolean JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Vault_1probe(JNIEnv* env, jclass cls) { edge_cases_error err = {0, NULL, NULL, 0}; (void)cls; @@ -996,5 +1184,17 @@ JNIEXPORT jboolean JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Vault_1probe( return rv ? JNI_TRUE : JNI_FALSE; } +JNIEXPORT jbyteArray JNICALL Java_edge_1cases_JniBridge_edge_1cases_1Vault_1to_1string(JNIEnv* env, jclass cls) { + edge_cases_error err = {0, NULL, NULL, 0}; + (void)cls; + size_t out_len = 0; + const uint8_t* rv = edge_cases_edge_cases_Vault_to_string(&out_len, &err); + if (err.code != 0) { + Jni_throw(env, &err, 0); + return NULL; + } + return Jni_take_bytes(env, rv, out_len); +} + // END edge_cases_jni.c diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Async_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Async_kt.snap deleted file mode 100644 index 3ca21041..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Async_kt.snap +++ /dev/null @@ -1,124 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package edge_cases - -import kotlin.coroutines.cancellation.CancellationException -import kotlin.coroutines.resume -import kotlin.coroutines.resumeWithException -import kotlinx.coroutines.CancellableContinuation -import kotlinx.coroutines.suspendCancellableCoroutine - -/** - * Bridges one native async call to a suspended coroutine. The JNI shim pins - * this object with a global reference and calls exactly one `on*` method, - * from whichever producer thread completes the call. - */ -internal class NativeCompletion( - private val cont: CancellableContinuation, - private val domain: Int, - private val convert: (Any?) -> T, - cancellable: Boolean, -) { - // The native cancel token, or 0 when the call isn't cancellable or has - // completed. Guarded by `this` so a late cancel never touches a freed - // token; a completion that lands inside `cancel` (on the same thread) - // defers the release until the cancel returns. - private var token: Long = if (cancellable) JniBridge.cancel_token_create() else 0L - private var cancelling = false - private var released = false - - val cancelToken: Long - get() = synchronized(this) { token } - - fun cancel() { - synchronized(this) { - if (token == 0L) return - cancelling = true - try { - JniBridge.cancel_token_cancel(token) - } finally { - cancelling = false - } - if (released) destroyToken() - } - } - - fun releaseToken() { - synchronized(this) { - if (cancelling) released = true else destroyToken() - } - } - - private fun destroyToken() { - if (token != 0L) JniBridge.cancel_token_destroy(token) - token = 0L - } - - fun onUnit() = succeed(Unit) - - fun onBoolean(value: Boolean) = succeed(value) - - fun onByte(value: Byte) = succeed(value) - - fun onShort(value: Short) = succeed(value) - - fun onInt(value: Int) = succeed(value) - - fun onLong(value: Long) = succeed(value) - - fun onFloat(value: Float) = succeed(value) - - fun onDouble(value: Double) = succeed(value) - - fun onBytes(value: ByteArray) = succeed(value) - - fun onError(code: Int, message: ByteArray, payload: ByteArray?) { - releaseToken() - val error = if (code == -5) { - CancellationException(decodeUtf8(message)) - } else { - JniBridge.error(domain, code, message, payload) - } - cont.resumeWithException(error) - } - - private fun succeed(raw: Any?) { - releaseToken() - val result = try { - convert(raw) - } catch (t: Throwable) { - cont.resumeWithException(t) - return - } - cont.resume(result) - } -} - -/** - * Starts a native async call and suspends until it completes. [launch] gets - * the cancel token (0 unless [cancellable]) and the completion to hand to the - * shim. Cancelling the coroutine cancels the native call; a cancelled call - * (code -5) resumes with [CancellationException]. - */ -internal suspend fun awaitNative( - cancellable: Boolean, - domain: Int, - convert: (Any?) -> T, - launch: (Long, NativeCompletion) -> Unit, -): T = suspendCancellableCoroutine { cont -> - val completion = NativeCompletion(cont, domain, convert, cancellable) - cont.invokeOnCancellation { completion.cancel() } - try { - launch(completion.cancelToken, completion) - } catch (t: Throwable) { - completion.releaseToken() - throw t - } -} - -// END Async.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Buffers_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Buffers_kt.snap deleted file mode 100644 index 2fc4f91c..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Buffers_kt.snap +++ /dev/null @@ -1,255 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package edge_cases - -/** - * Growable little-endian writer for the value-buffer wire format that - * records, rich enums, optionals, lists, and maps cross the ABI in. - */ -internal class BufferWriter { - private var buf = ByteArray(64) - private var len = 0 - private var minted: ArrayList Unit>>? = null - - private fun reserve(extra: Int) { - if (len + extra <= buf.size) return - var cap = buf.size * 2 - while (cap < len + extra) cap *= 2 - buf = buf.copyOf(cap) - } - - fun toByteArray(): ByteArray = buf.copyOf(len) - - fun writeBool(v: Boolean) { - reserve(1) - buf[len++] = if (v) 1 else 0 - } - - fun writeI8(v: Byte) { - reserve(1) - buf[len++] = v - } - - fun writeU8(v: Byte) = writeI8(v) - - fun writeI16(v: Short) { - reserve(2) - val b = v.toInt() - buf[len++] = b.toByte() - buf[len++] = (b shr 8).toByte() - } - - fun writeU16(v: Short) = writeI16(v) - - fun writeI32(v: Int) { - reserve(4) - buf[len++] = v.toByte() - buf[len++] = (v shr 8).toByte() - buf[len++] = (v shr 16).toByte() - buf[len++] = (v shr 24).toByte() - } - - fun writeU32(v: Long) = writeI32(v.toInt()) - - fun writeI64(v: Long) { - writeI32(v.toInt()) - writeI32((v ushr 32).toInt()) - } - - fun writeU64(v: Long) = writeI64(v) - - fun writeF32(v: Float) = writeI32(v.toRawBits()) - - fun writeF64(v: Double) = writeI64(v.toRawBits()) - - fun writeString(v: String) = writeBytes(encodeUtf8(v)) - - fun writeBytes(v: ByteArray) { - writeI32(v.size) - reserve(v.size) - v.copyInto(buf, len) - len += v.size - } - - /** - * An object token: one new strong reference the reader adopts, which - * [abandon] releases with [release] if the encoding fails. - */ - fun writeObject(token: Long, release: (Long) -> Unit) { - val list = minted ?: ArrayList Unit>>().also { minted = it } - list.add(token to release) - writeI64(token) - } - - /** Releases every object token written so far, for an encoding that failed midway. */ - fun abandon() { - minted?.forEach { (token, release) -> release(token) } - minted = null - } - - inline fun writeOptional(v: T?, write: (T) -> Unit) { - if (v == null) { - writeBool(false) - } else { - writeBool(true) - write(v) - } - } - - inline fun writeList(v: List, write: (T) -> Unit) { - writeI32(v.size) - for (e in v) write(e) - } - - inline fun writeMap(v: Map, writeKey: (K) -> Unit, writeValue: (V) -> Unit) { - writeI32(v.size) - for ((k, e) in v) { - writeKey(k) - writeValue(e) - } - } -} - -/** - * Validating little-endian reader for the value-buffer wire format. A - * truncated buffer, an invalid flag byte, a zero object token, an oversized - * length, or trailing bytes raise [FfiException] code -3. - */ -internal class BufferReader(private val buf: ByteArray) { - private var pos = 0 - - private fun malformed(detail: String): Nothing = - throw FfiException(-3, "malformed value buffer: $detail") - - private fun take(n: Int): Int { - if (n > buf.size - pos) malformed("truncated buffer") - val at = pos - pos += n - return at - } - - fun readBool(): Boolean = when (buf[take(1)].toInt()) { - 0 -> false - 1 -> true - else -> malformed("invalid bool byte") - } - - fun readI8(): Byte = buf[take(1)] - - fun readU8(): Byte = readI8() - - fun readI16(): Short { - val at = take(2) - return ((buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)).toShort() - } - - fun readU16(): Short = readI16() - - fun readI32(): Int { - val at = take(4) - return (buf[at].toInt() and 0xFF) or - ((buf[at + 1].toInt() and 0xFF) shl 8) or - ((buf[at + 2].toInt() and 0xFF) shl 16) or - ((buf[at + 3].toInt() and 0xFF) shl 24) - } - - fun readU32(): Long = readI32().toLong() and 0xFFFFFFFFL - - fun readI64(): Long { - val lo = readI32().toLong() and 0xFFFFFFFFL - val hi = readI32().toLong() - return lo or (hi shl 32) - } - - fun readU64(): Long = readI64() - - fun readF32(): Float = Float.fromBits(readI32()) - - fun readF64(): Double = Double.fromBits(readI64()) - - private fun readLen(): Int { - val n = readI32() - if (n < 0 || n > buf.size - pos) malformed("length prefix exceeds remaining bytes") - return n - } - - fun readString(): String { - val n = readLen() - val at = take(n) - return try { - Charsets.UTF_8.newDecoder().decode(java.nio.ByteBuffer.wrap(buf, at, n)).toString() - } catch (e: java.nio.charset.CharacterCodingException) { - malformed("string is not valid UTF-8") - } - } - - fun readBytes(): ByteArray { - val n = readLen() - val at = take(n) - return buf.copyOfRange(at, at + n) - } - - /** An object token: one strong reference the caller adopts. */ - fun readObject(): Long { - val token = readI64() - if (token == 0L) malformed("null object token") - return token - } - - inline fun readOptional(read: () -> T): T? = if (readBool()) read() else null - - inline fun readList(read: () -> T): List { - val n = readCount() - val out = ArrayList(minOf(n, 4096)) - repeat(n) { out.add(read()) } - return out - } - - inline fun readMap(readKey: () -> K, readValue: () -> V): Map { - val n = readCount() - val out = LinkedHashMap(minOf(n, 4096)) - repeat(n) { - val k = readKey() - out[k] = readValue() - } - return out - } - - /** An element count; elements may encode to zero bytes, so only the sign is checked. */ - fun readCount(): Int { - val n = readI32() - if (n < 0) malformed("negative element count") - return n - } - - fun expectEnd() { - if (pos != buf.size) malformed("trailing bytes after value") - } -} - -/** Encodes one value into a fresh value buffer. */ -internal inline fun encodeBuffer(write: (BufferWriter) -> Unit): ByteArray { - val w = BufferWriter() - try { - write(w) - } catch (t: Throwable) { - w.abandon() - throw t - } - return w.toByteArray() -} - -/** Decodes one value from [bytes], rejecting malformed or trailing bytes. */ -internal inline fun decodeBuffer(bytes: ByteArray, read: (BufferReader) -> T): T { - val r = BufferReader(bytes) - val v = read(r) - r.expectEnd() - return v -} - -// END Buffers.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Codecs_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Codecs_kt.snap new file mode 100644 index 00000000..cdeb529b --- /dev/null +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Codecs_kt.snap @@ -0,0 +1,104 @@ +--- +source: crates/weaveffi-cli/tests/snapshots.rs +--- +// Generated by WeaveFFI [VERSION] +// DO NOT EDIT. Your changes will be overwritten. +// To regenerate: weaveffi generate + +@file:Suppress("DEPRECATION") + +package edge_cases + +internal fun packListOfI32(_w: BufferWriter, _v: List) = _w.writeList(_v) { _w.writeI32(it) } + +internal fun unpackListOfI32(_r: BufferReader): List = _r.readList { _r.readI32() } + +internal fun packListOfLeaf(_w: BufferWriter, _v: List) = _w.writeList(_v) { packLeaf(_w, it) } + +internal fun unpackListOfLeaf(_r: BufferReader): List = _r.readList { unpackLeaf(_r) } + +internal fun packListOfListOfI32(_w: BufferWriter, _v: List>) = _w.writeList(_v) { packListOfI32(_w, it) } + +internal fun unpackListOfListOfI32(_r: BufferReader): List> = _r.readList { unpackListOfI32(_r) } + +internal fun packListOfMode(_w: BufferWriter, _v: List) = _w.writeList(_v) { _w.writeI32(it.value) } + +internal fun unpackListOfMode(_r: BufferReader): List = _r.readList { Mode.fromValue(_r.readI32()) } + +internal fun packListOfOptionalOfI64(_w: BufferWriter, _v: List) = _w.writeList(_v) { packOptionalOfI64(_w, it) } + +internal fun unpackListOfOptionalOfI64(_r: BufferReader): List = _r.readList { unpackOptionalOfI64(_r) } + +internal fun packListOfOptionalOfOuter(_w: BufferWriter, _v: List) = _w.writeList(_v) { packOptionalOfOuter(_w, it) } + +internal fun unpackListOfOptionalOfOuter(_r: BufferReader): List = _r.readList { unpackOptionalOfOuter(_r) } + +internal fun packListOfOptionalOfWidget(_w: BufferWriter, _v: List) = _w.writeList(_v) { packOptionalOfWidget(_w, it) } + +internal fun unpackListOfOptionalOfWidget(_r: BufferReader): List = _r.readList { unpackOptionalOfWidget(_r) } + +internal fun packListOfOuter(_w: BufferWriter, _v: List) = _w.writeList(_v) { packOuter(_w, it) } + +internal fun unpackListOfOuter(_r: BufferReader): List = _r.readList { unpackOuter(_r) } + +internal fun packListOfString(_w: BufferWriter, _v: List) = _w.writeList(_v) { _w.writeString(it) } + +internal fun unpackListOfString(_r: BufferReader): List = _r.readList { _r.readString() } + +internal fun packListOfWidget(_w: BufferWriter, _v: List) = _w.writeList(_v) { _w.writeObject(it.cloneHandle(), JniBridge::edge_cases_Widget_destroy) } + +internal fun unpackListOfWidget(_r: BufferReader): List = _r.readList { Widget.fromHandle(_r.readObject()) } + +internal fun packMapOfI64ToString(_w: BufferWriter, _v: Map) = _w.writeMap(_v, { _w.writeI64(it) }, { _w.writeString(it) }) + +internal fun unpackMapOfI64ToString(_r: BufferReader): Map = _r.readMap({ _r.readI64() }, { _r.readString() }) + +internal fun packMapOfStringToInner(_w: BufferWriter, _v: Map) = _w.writeMap(_v, { _w.writeString(it) }, { packInner(_w, it) }) + +internal fun unpackMapOfStringToInner(_r: BufferReader): Map = _r.readMap({ _r.readString() }, { unpackInner(_r) }) + +internal fun packMapOfStringToWidget(_w: BufferWriter, _v: Map) = _w.writeMap(_v, { _w.writeString(it) }, { _w.writeObject(it.cloneHandle(), JniBridge::edge_cases_Widget_destroy) }) + +internal fun unpackMapOfStringToWidget(_r: BufferReader): Map = _r.readMap({ _r.readString() }, { Widget.fromHandle(_r.readObject()) }) + +internal fun packOptionalOfBytes(_w: BufferWriter, _v: ByteArray?) = _w.writeOptional(_v) { _w.writeBytes(it) } + +internal fun unpackOptionalOfBytes(_r: BufferReader): ByteArray? = _r.readOptional { _r.readBytes() } + +internal fun packOptionalOfI32(_w: BufferWriter, _v: Int?) = _w.writeOptional(_v) { _w.writeI32(it) } + +internal fun unpackOptionalOfI32(_r: BufferReader): Int? = _r.readOptional { _r.readI32() } + +internal fun packOptionalOfI64(_w: BufferWriter, _v: Long?) = _w.writeOptional(_v) { _w.writeI64(it) } + +internal fun unpackOptionalOfI64(_r: BufferReader): Long? = _r.readOptional { _r.readI64() } + +internal fun packOptionalOfInner(_w: BufferWriter, _v: Inner?) = _w.writeOptional(_v) { packInner(_w, it) } + +internal fun unpackOptionalOfInner(_r: BufferReader): Inner? = _r.readOptional { unpackInner(_r) } + +internal fun packOptionalOfListOfOptionalOfOuter(_w: BufferWriter, _v: List?) = _w.writeOptional(_v) { packListOfOptionalOfOuter(_w, it) } + +internal fun unpackOptionalOfListOfOptionalOfOuter(_r: BufferReader): List? = _r.readOptional { unpackListOfOptionalOfOuter(_r) } + +internal fun packOptionalOfListOfString(_w: BufferWriter, _v: List?) = _w.writeOptional(_v) { packListOfString(_w, it) } + +internal fun unpackOptionalOfListOfString(_r: BufferReader): List? = _r.readOptional { unpackListOfString(_r) } + +internal fun packOptionalOfMode(_w: BufferWriter, _v: Mode?) = _w.writeOptional(_v) { _w.writeI32(it.value) } + +internal fun unpackOptionalOfMode(_r: BufferReader): Mode? = _r.readOptional { Mode.fromValue(_r.readI32()) } + +internal fun packOptionalOfOuter(_w: BufferWriter, _v: Outer?) = _w.writeOptional(_v) { packOuter(_w, it) } + +internal fun unpackOptionalOfOuter(_r: BufferReader): Outer? = _r.readOptional { unpackOuter(_r) } + +internal fun packOptionalOfString(_w: BufferWriter, _v: String?) = _w.writeOptional(_v) { _w.writeString(it) } + +internal fun unpackOptionalOfString(_r: BufferReader): String? = _r.readOptional { _r.readString() } + +internal fun packOptionalOfWidget(_w: BufferWriter, _v: Widget?) = _w.writeOptional(_v) { _w.writeObject(it.cloneHandle(), JniBridge::edge_cases_Widget_destroy) } + +internal fun unpackOptionalOfWidget(_r: BufferReader): Widget? = _r.readOptional { Widget.fromHandle(_r.readObject()) } + +// END Codecs.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_EdgeCases_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_EdgeCases_kt.snap index 2d1dc03c..e684975f 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_EdgeCases_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_EdgeCases_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate @file:Suppress("DEPRECATION") @@ -32,7 +32,7 @@ enum class Legacy(val value: Int) { companion object { /** The variant whose ABI value is [value]. */ fun fromValue(value: Int): Legacy = entries.firstOrNull { it.value == value } - ?: throw FfiException(-3, "unknown Legacy value $value") + ?: throw NativeBugException(-3, "unknown Legacy value $value") } } @@ -43,7 +43,7 @@ enum class Mode(val value: Int) { companion object { /** The variant whose ABI value is [value]. */ fun fromValue(value: Int): Mode = entries.firstOrNull { it.value == value } - ?: throw FfiException(-3, "unknown Mode value $value") + ?: throw NativeBugException(-3, "unknown Mode value $value") } } @@ -62,40 +62,40 @@ data class Inner(val leaf: Leaf, val leaves: List) internal fun packInner(_w: BufferWriter, _v: Inner) { packLeaf(_w, _v.leaf) - _w.writeList(_v.leaves) { _v0 -> packLeaf(_w, _v0) } + packListOfLeaf(_w, _v.leaves) } internal fun unpackInner(_r: BufferReader): Inner = Inner( unpackLeaf(_r), - _r.readList { unpackLeaf(_r) }, + unpackListOfLeaf(_r), ) -data class Outer(val inner: Inner, val maybe_inner: Inner?, val by_name: Map, val modes: List, val matrix: List>, val sparse: List, val blob: ByteArray?, val note: String?, val by_id: Map, val _hidden: Boolean, val v2: Byte) +data class Outer(val inner: Inner, val maybe_inner: Inner?, val by_name: Map, val modes: List, val matrix: List>, val sparse: List, val blob: ByteArray?, val note: String?, val by_id: Map, val _hidden: Boolean, val v2: UByte) internal fun packOuter(_w: BufferWriter, _v: Outer) { packInner(_w, _v.inner) - _w.writeOptional(_v.maybe_inner) { _v0 -> packInner(_w, _v0) } - _w.writeMap(_v.by_name, { _k0 -> _w.writeString(_k0) }, { _v0 -> packInner(_w, _v0) }) - _w.writeList(_v.modes) { _v0 -> _w.writeI32(_v0.value) } - _w.writeList(_v.matrix) { _v0 -> _w.writeList(_v0) { _v1 -> _w.writeI32(_v1) } } - _w.writeList(_v.sparse) { _v0 -> _w.writeOptional(_v0) { _v1 -> _w.writeI64(_v1) } } - _w.writeOptional(_v.blob) { _v0 -> _w.writeBytes(_v0) } - _w.writeOptional(_v.note) { _v0 -> _w.writeString(_v0) } - _w.writeMap(_v.by_id, { _k0 -> _w.writeI64(_k0) }, { _v0 -> _w.writeString(_v0) }) + packOptionalOfInner(_w, _v.maybe_inner) + packMapOfStringToInner(_w, _v.by_name) + packListOfMode(_w, _v.modes) + packListOfListOfI32(_w, _v.matrix) + packListOfOptionalOfI64(_w, _v.sparse) + packOptionalOfBytes(_w, _v.blob) + packOptionalOfString(_w, _v.note) + packMapOfI64ToString(_w, _v.by_id) _w.writeBool(_v._hidden) _w.writeU8(_v.v2) } internal fun unpackOuter(_r: BufferReader): Outer = Outer( unpackInner(_r), - _r.readOptional { unpackInner(_r) }, - _r.readMap({ _r.readString() }, { unpackInner(_r) }), - _r.readList { Mode.fromValue(_r.readI32()) }, - _r.readList { _r.readList { _r.readI32() } }, - _r.readList { _r.readOptional { _r.readI64() } }, - _r.readOptional { _r.readBytes() }, - _r.readOptional { _r.readString() }, - _r.readMap({ _r.readI64() }, { _r.readString() }), + unpackOptionalOfInner(_r), + unpackMapOfStringToInner(_r), + unpackListOfMode(_r), + unpackListOfListOfI32(_r), + unpackListOfOptionalOfI64(_r), + unpackOptionalOfBytes(_r), + unpackOptionalOfString(_r), + unpackMapOfI64ToString(_r), _r.readBool(), _r.readU8(), ) @@ -244,16 +244,16 @@ data class WidgetHolder(val primary: Widget, val fallback: Widget?, val extras: internal fun packWidgetHolder(_w: BufferWriter, _v: WidgetHolder) { _w.writeObject(_v.primary.cloneHandle(), JniBridge::edge_cases_Widget_destroy) - _w.writeOptional(_v.fallback) { _v0 -> _w.writeObject(_v0.cloneHandle(), JniBridge::edge_cases_Widget_destroy) } - _w.writeList(_v.extras) { _v0 -> _w.writeObject(_v0.cloneHandle(), JniBridge::edge_cases_Widget_destroy) } - _w.writeMap(_v.by_name, { _k0 -> _w.writeString(_k0) }, { _v0 -> _w.writeObject(_v0.cloneHandle(), JniBridge::edge_cases_Widget_destroy) }) + packOptionalOfWidget(_w, _v.fallback) + packListOfWidget(_w, _v.extras) + packMapOfStringToWidget(_w, _v.by_name) } internal fun unpackWidgetHolder(_r: BufferReader): WidgetHolder = WidgetHolder( Widget.fromHandle(_r.readObject()), - _r.readOptional { Widget.fromHandle(_r.readObject()) }, - _r.readList { Widget.fromHandle(_r.readObject()) }, - _r.readMap({ _r.readString() }, { Widget.fromHandle(_r.readObject()) }), + unpackOptionalOfWidget(_r), + unpackListOfWidget(_r), + unpackMapOfStringToWidget(_r), ) /** Callback interface carrying composite payloads and objects */ @@ -295,11 +295,11 @@ class Widget private constructor(address: Long) : AutoCloseable { } fun rename(class_: String, import: String?): String = handle.borrow { _self -> - decodeUtf8(JniBridge.edge_cases_Widget_rename(_self, encodeUtf8(class_), encodeBuffer { _w -> _w.writeOptional(import) { _v0 -> _w.writeString(_v0) } })) + decodeUtf8(JniBridge.edge_cases_Widget_rename(_self, encodeUtf8(class_), encodeBuffer { _w -> packOptionalOfString(_w, import) })) } /** Async method with a struct result */ - suspend fun fetch(id: Long): Outer = awaitNative(false, 1, { decodeBuffer(it as ByteArray) { _r -> unpackOuter(_r) } }) { _, _done -> + suspend fun fetch(id: Long): Outer = awaitNative(false, 1, { _raw -> decodeBuffer(_raw as ByteArray) { _r -> unpackOuter(_r) } }) { _, _done -> handle.borrow { _self -> JniBridge.edge_cases_Widget_fetch(_self, id, _done) } @@ -310,6 +310,18 @@ class Widget private constructor(address: Long) : AutoCloseable { decodeUtf8(JniBridge.edge_cases_Widget_old_name(_self)) } + /** Spelled like the release method of several wrappers */ + fun close_(): Boolean = handle.borrow { _self -> + JniBridge.edge_cases_Widget_close(_self) + } + + /** Spelled like the release method of the Dart and .NET wrappers */ + fun dispose(force: Boolean) { + handle.borrow { _self -> + JniBridge.edge_cases_Widget_dispose(_self, force) + } + } + /** Releases this wrapper's native reference; safe to call more than once. */ override fun close() = handle.close() @@ -334,6 +346,11 @@ class Vault private constructor(address: Long) : AutoCloseable { Widget.fromHandleOrNull(JniBridge.edge_cases_Vault_peek(_self)) } + /** Spelled like the handle accessor of several wrappers */ + fun handle_(): Long = handle.borrow { _self -> + JniBridge.edge_cases_Vault_handle(_self) + } + /** Releases this wrapper's native reference; safe to call more than once. */ override fun close() = handle.close() @@ -349,6 +366,9 @@ class Vault private constructor(address: Long) : AutoCloseable { /** Throwing static */ fun probe(): Boolean = JniBridge.edge_cases_Vault_probe() + + /** Static spelled like an inherited object member */ + fun toString_(): String = decodeUtf8(JniBridge.edge_cases_Vault_to_string()) } } @@ -359,12 +379,12 @@ object EdgeCases { } /** Function named after a keyword whose parameters are all keywords */ - fun select(class_: Int, from: String, func: List, range: Int?, type: Boolean): Int = JniBridge.edge_cases_select(class_, encodeUtf8(from), encodeBuffer { _w -> _w.writeList(func) { _v0 -> _w.writeString(_v0) } }, encodeBuffer { _w -> _w.writeOptional(range) { _v0 -> _w.writeI32(_v0) } }, type) + fun select(class_: Int, from: String, func: List, range: Int?, type: Boolean): Int = JniBridge.edge_cases_select(class_, encodeUtf8(from), encodeBuffer { _w -> packListOfString(_w, func) }, encodeBuffer { _w -> packOptionalOfI32(_w, range) }, type) /** Optional list of optional structs, the deepest legal nesting */ - fun nestedOptional(items: List?): List? = decodeBuffer(JniBridge.edge_cases_nested_optional(encodeBuffer { _w -> _w.writeOptional(items) { _v0 -> _w.writeList(_v0) { _v1 -> _w.writeOptional(_v1) { _v2 -> packOuter(_w, _v2) } } } })) { _r -> _r.readOptional { _r.readList { _r.readOptional { unpackOuter(_r) } } } } + fun nestedOptional(items: List?): List? = decodeBuffer(JniBridge.edge_cases_nested_optional(encodeBuffer { _w -> packOptionalOfListOfOptionalOfOuter(_w, items) })) { _r -> unpackOptionalOfListOfOptionalOfOuter(_r) } - fun optionalParams(text: String?, count: Int?, outer: Outer?, tags: List?, blob: ByteArray?, mode: Mode?): Mode? = decodeBuffer(JniBridge.edge_cases_optional_params(encodeBuffer { _w -> _w.writeOptional(text) { _v0 -> _w.writeString(_v0) } }, encodeBuffer { _w -> _w.writeOptional(count) { _v0 -> _w.writeI32(_v0) } }, encodeBuffer { _w -> _w.writeOptional(outer) { _v0 -> packOuter(_w, _v0) } }, encodeBuffer { _w -> _w.writeOptional(tags) { _v0 -> _w.writeList(_v0) { _v1 -> _w.writeString(_v1) } } }, encodeBuffer { _w -> _w.writeOptional(blob) { _v0 -> _w.writeBytes(_v0) } }, encodeBuffer { _w -> _w.writeOptional(mode) { _v0 -> _w.writeI32(_v0.value) } })) { _r -> _r.readOptional { Mode.fromValue(_r.readI32()) } } + fun optionalParams(text: String?, count: Int?, outer: Outer?, tags: List?, blob: ByteArray?, mode: Mode?): Mode? = decodeBuffer(JniBridge.edge_cases_optional_params(encodeBuffer { _w -> packOptionalOfString(_w, text) }, encodeBuffer { _w -> packOptionalOfI32(_w, count) }, encodeBuffer { _w -> packOptionalOfOuter(_w, outer) }, encodeBuffer { _w -> packOptionalOfListOfString(_w, tags) }, encodeBuffer { _w -> packOptionalOfBytes(_w, blob) }, encodeBuffer { _w -> packOptionalOfMode(_w, mode) })) { _r -> unpackOptionalOfMode(_r) } /** Two callback interface parameters on one function */ fun registerBatch(listener: BatchListener, keyword: Keyword) { @@ -372,7 +392,7 @@ object EdgeCases { } /** Objects in every buffered position */ - fun widgetsEverywhere(list: List, byName: Map, sparse: List, holder: WidgetHolder): WidgetHolder = decodeBuffer(JniBridge.edge_cases_widgets_everywhere(encodeBuffer { _w -> _w.writeList(list) { _v0 -> _w.writeObject(_v0.cloneHandle(), JniBridge::edge_cases_Widget_destroy) } }, encodeBuffer { _w -> _w.writeMap(byName, { _k0 -> _w.writeString(_k0) }, { _v0 -> _w.writeObject(_v0.cloneHandle(), JniBridge::edge_cases_Widget_destroy) }) }, encodeBuffer { _w -> _w.writeList(sparse) { _v0 -> _w.writeOptional(_v0) { _v1 -> _w.writeObject(_v1.cloneHandle(), JniBridge::edge_cases_Widget_destroy) } } }, encodeBuffer { _w -> packWidgetHolder(_w, holder) })) { _r -> unpackWidgetHolder(_r) } + fun widgetsEverywhere(list: List, byName: Map, sparse: List, holder: WidgetHolder): WidgetHolder = decodeBuffer(JniBridge.edge_cases_widgets_everywhere(encodeBuffer { _w -> packListOfWidget(_w, list) }, encodeBuffer { _w -> packMapOfStringToWidget(_w, byName) }, encodeBuffer { _w -> packListOfOptionalOfWidget(_w, sparse) }, encodeBuffer { _w -> packWidgetHolder(_w, holder) })) { _r -> unpackWidgetHolder(_r) } /** Iterator of objects */ fun widgetStream(): NativeIterator = NativeIterator(JniBridge.edge_cases_widget_stream(), JniBridge::edge_cases_WidgetStreamIterator_destroy) { _it -> @@ -381,7 +401,7 @@ object EdgeCases { } /** Async returning an object */ - suspend fun asyncWidget(): Widget = awaitNative(false, 1, { Widget.fromHandle(it as Long) }) { _, _done -> + suspend fun asyncWidget(): Widget = awaitNative(false, 1, { _raw -> Widget.fromHandle(_raw as Long) }) { _, _done -> JniBridge.edge_cases_async_widget(_done) } @@ -423,27 +443,27 @@ object EdgeCases { } } - suspend fun asyncBytes(size: Long): ByteArray = awaitNative(false, 0, { it as ByteArray }) { _, _done -> - JniBridge.edge_cases_async_bytes(size, _done) + suspend fun asyncBytes(size: UInt): ByteArray = awaitNative(false, 0, { _raw -> _raw as ByteArray }) { _, _done -> + JniBridge.edge_cases_async_bytes(size.toInt(), _done) } - suspend fun asyncStruct(): Outer = awaitNative(false, 1, { decodeBuffer(it as ByteArray) { _r -> unpackOuter(_r) } }) { _, _done -> + suspend fun asyncStruct(): Outer = awaitNative(false, 1, { _raw -> decodeBuffer(_raw as ByteArray) { _r -> unpackOuter(_r) } }) { _, _done -> JniBridge.edge_cases_async_struct(_done) } - suspend fun asyncList(): List = awaitNative(true, 1, { decodeBuffer(it as ByteArray) { _r -> _r.readList { unpackOuter(_r) } } }) { _token, _done -> + suspend fun asyncList(): List = awaitNative(true, 1, { _raw -> decodeBuffer(_raw as ByteArray) { _r -> unpackListOfOuter(_r) } }) { _token, _done -> JniBridge.edge_cases_async_list(_token, _done) } - suspend fun asyncOptional(): Long? = awaitNative(false, 0, { decodeBuffer(it as ByteArray) { _r -> _r.readOptional { _r.readI64() } } }) { _, _done -> + suspend fun asyncOptional(): Long? = awaitNative(false, 0, { _raw -> decodeBuffer(_raw as ByteArray) { _r -> unpackOptionalOfI64(_r) } }) { _, _done -> JniBridge.edge_cases_async_optional(_done) } - suspend fun asyncEnum(): Mode = awaitNative(false, 0, { Mode.fromValue(it as Int) }) { _, _done -> + suspend fun asyncEnum(): Mode = awaitNative(false, 0, { _raw -> Mode.fromValue(_raw as Int) }) { _, _done -> JniBridge.edge_cases_async_enum(_done) } - suspend fun asyncKeywords(class_: String, import: Int): String = awaitNative(false, 0, { decodeUtf8(it as ByteArray) }) { _, _done -> + suspend fun asyncKeywords(class_: String, import: Int): String = awaitNative(false, 0, { _raw -> decodeUtf8(_raw as ByteArray) }) { _, _done -> JniBridge.edge_cases_async_keywords(encodeUtf8(class_), import, _done) } diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_JniBridge_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_JniBridge_kt.snap index 6ff7f386..316d0774 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_JniBridge_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_JniBridge_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate @file:Suppress("DEPRECATION") @@ -19,20 +19,35 @@ internal object JniBridge { NativeLibrary.ensureLoaded() } - /** The exception for a native failure; domain 0 is the generic one. */ + /** + * The exception for a native failure: through error domain [domain] for a + * throwing call, or a [NativeBugException] (domain 0) for one that can't fail. + */ @JvmStatic - @Suppress("UNUSED_PARAMETER") - fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): FfiException { + fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): Throwable { val text = decodeUtf8(message) return when (domain) { 1 -> EdgeErrorsException.fromCode(code, text, payload) - else -> FfiException(code, text) + else -> NativeBugException(code, text) } } - /** The message a failed callback reports to the producer (code -4). */ - @JvmStatic - fun describe(error: Throwable): ByteArray = encodeUtf8(error.toString()) + /** + * Reports a failed callback through its vtable entry's `out_err` ([err]): + * [typed], a domain error the method is declared to throw, with its code and + * payload fields, and anything else as code -4 with its message. + */ + private fun fail(err: Long, error: Throwable, typed: FfiException?) { + if (typed != null) { + error_set(err, typed.code, encodeUtf8(typed.message ?: "")) + typed.encodePayload()?.let { error_set_payload(err, it) } + } else { + error_set(err, -4, encodeUtf8(error.message ?: error.toString())) + } + } + + @JvmStatic external fun error_set(err: Long, code: Int, message: ByteArray) + @JvmStatic external fun error_set_payload(err: Long, payload: ByteArray) @JvmStatic external fun debug_live(kind: Int): Long @JvmStatic external fun cancel_token_create(): Long @@ -65,7 +80,7 @@ internal object JniBridge { @JvmStatic external fun edge_cases_ModeStreamIterator_next(_iter: Long, _out: IntArray): Boolean @JvmStatic external fun edge_cases_ModeStreamIterator_destroy(_iter: Long) @JvmStatic external fun edge_cases_async_void(_done: NativeCompletion<*>) - @JvmStatic external fun edge_cases_async_bytes(size: Long, _done: NativeCompletion<*>) + @JvmStatic external fun edge_cases_async_bytes(size: Int, _done: NativeCompletion<*>) @JvmStatic external fun edge_cases_async_struct(_done: NativeCompletion<*>) @JvmStatic external fun edge_cases_async_list(_token: Long, _done: NativeCompletion<*>) @JvmStatic external fun edge_cases_async_optional(_done: NativeCompletion<*>) @@ -77,30 +92,75 @@ internal object JniBridge { @JvmStatic external fun edge_cases_Widget_rename(_self: Long, class_: ByteArray, import: ByteArray): ByteArray @JvmStatic external fun edge_cases_Widget_fetch(_self: Long, id: Long, _done: NativeCompletion<*>) @JvmStatic external fun edge_cases_Widget_old_name(_self: Long): ByteArray + @JvmStatic external fun edge_cases_Widget_close(_self: Long): Boolean + @JvmStatic external fun edge_cases_Widget_dispose(_self: Long, force: Boolean) @JvmStatic external fun edge_cases_Vault_open(path: ByteArray): Long @JvmStatic external fun edge_cases_Vault_peek(_self: Long): Long + @JvmStatic external fun edge_cases_Vault_handle(_self: Long): Long @JvmStatic external fun edge_cases_Vault_probe(): Boolean + @JvmStatic external fun edge_cases_Vault_to_string(): ByteArray @JvmStatic - fun edge_cases_BatchListener_on_batch(_impl: BatchListener, items: ByteArray, raw: ByteArray, first: ByteArray, mode: Int) { _impl.onBatch(decodeBuffer(items) { _r -> _r.readList { unpackOuter(_r) } }, raw, decodeBuffer(first) { _r -> _r.readOptional { unpackOuter(_r) } }, Mode.fromValue(mode)) } + fun edge_cases_BatchListener_on_batch(_impl: BatchListener, _err: Long, items: ByteArray, raw: ByteArray, first: ByteArray, mode: Int) { + try { + _impl.onBatch(decodeBuffer(items) { _r -> unpackListOfOuter(_r) }, raw, decodeBuffer(first) { _r -> unpackOptionalOfOuter(_r) }, Mode.fromValue(mode)) + } catch (_e: Throwable) { + fail(_err, _e, null) + } + } @JvmStatic - fun edge_cases_BatchListener_on_widget(_impl: BatchListener, widget: Long, fallback: Long, children: ByteArray) { _impl.onWidget(Widget.fromHandle(widget), Widget.fromHandleOrNull(fallback), decodeBuffer(children) { _r -> _r.readList { Widget.fromHandle(_r.readObject()) } }) } + fun edge_cases_BatchListener_on_widget(_impl: BatchListener, _err: Long, widget: Long, fallback: Long, children: ByteArray) { + val _widget = Widget.fromHandle(widget) + val _fallback = Widget.fromHandleOrNull(fallback) + try { + _impl.onWidget(_widget, _fallback, decodeBuffer(children) { _r -> unpackListOfWidget(_r) }) + } catch (_e: Throwable) { + fail(_err, _e, null) + } + } @JvmStatic - fun edge_cases_BatchListener_on_tick(_impl: BatchListener) { _impl.onTick() } + fun edge_cases_BatchListener_on_tick(_impl: BatchListener, _err: Long) { + try { + _impl.onTick() + } catch (_e: Throwable) { + fail(_err, _e, null) + } + } @JvmStatic - fun edge_cases_BatchListener_should_continue(_impl: BatchListener, class_: Int): Boolean = _impl.shouldContinue(class_) + fun edge_cases_BatchListener_should_continue(_impl: BatchListener, _err: Long, class_: Int): Boolean = try { + _impl.shouldContinue(class_) + } catch (_e: Throwable) { + fail(_err, _e, null) + false + } @JvmStatic - fun edge_cases_BatchListener_pick_mode(_impl: BatchListener): Int = _impl.pickMode().value + fun edge_cases_BatchListener_pick_mode(_impl: BatchListener, _err: Long): Int = try { + _impl.pickMode().value + } catch (_e: Throwable) { + fail(_err, _e, null) + 0 + } @JvmStatic - fun edge_cases_Keyword_delete(_impl: Keyword, self: Int) { _impl.delete(self) } + fun edge_cases_Keyword_delete(_impl: Keyword, _err: Long, self: Int) { + try { + _impl.delete(self) + } catch (_e: Throwable) { + fail(_err, _e, null) + } + } @JvmStatic - fun edge_cases_Keyword_import(_impl: Keyword): Int = _impl.import() + fun edge_cases_Keyword_import(_impl: Keyword, _err: Long): Int = try { + _impl.import() + } catch (_e: Throwable) { + fail(_err, _e, null) + 0 + } } // END JniBridge.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Runtime_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Runtime_kt.snap deleted file mode 100644 index 788c6b10..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_edge_cases__src_main_kotlin_edge_cases_Runtime_kt.snap +++ /dev/null @@ -1,212 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package edge_cases - -import java.lang.ref.PhantomReference -import java.lang.ref.ReferenceQueue -import java.util.Collections -import java.util.concurrent.ConcurrentHashMap -import java.util.concurrent.atomic.AtomicLong - -/** - * A failure reported by the native library: a producer panic (code -2), a - * marshalling failure (-3), a callback implementation that threw (-4), or any - * code these bindings don't know. Typed error domains subclass it. - */ -open class FfiException(val code: Int, message: String) : Exception(message) - -/** - * A lazy sequence streamed from the native library, one element per step. - * The native iterator is released when the sequence is exhausted, when - * [close] is called, or once this object becomes unreachable, whichever - * comes first. - */ -class NativeIterator internal constructor( - address: Long, - release: (Long) -> Unit, - private val step: (Long) -> Any?, -) : Iterator, AutoCloseable { - private val handle = NativeCleaner.register(this, NativeHandle(address, release)) - private var lookahead: Any? = EMPTY - - override fun hasNext(): Boolean { - if (lookahead !== EMPTY) return true - if (handle.isClosed) return false - val item = handle.borrow(step) - if (item === DONE) { - close() - return false - } - lookahead = item - return true - } - - override fun next(): T { - if (!hasNext()) throw NoSuchElementException() - val item = lookahead - lookahead = EMPTY - @Suppress("UNCHECKED_CAST") - return item as T - } - - /** Releases the native iterator; safe to call more than once. */ - override fun close() = handle.close() - - internal companion object { - /** What a step function returns once the native sequence is exhausted. */ - val DONE = Any() - private val EMPTY = Any() - } -} - -/** Loads the producer library and then the JNI shim, exactly once. */ -internal object NativeLibrary { - private const val LIBRARY = "edge_cases" - private const val JNI_LIBRARY = "edge_cases_jni" - - init { - val explicit = System.getenv("EDGE_CASES_LIBRARY") - if (!explicit.isNullOrEmpty()) { - System.load(explicit) - } else if (!loadBundled(LIBRARY)) { - // The shim's own link dependency usually finds the producer when - // it isn't on java.library.path, so a miss here is only fatal if - // loading the shim fails too. - try { - System.loadLibrary(LIBRARY) - } catch (ignored: UnsatisfiedLinkError) { - } - } - if (!loadBundled(JNI_LIBRARY)) System.loadLibrary(JNI_LIBRARY) - } - - fun ensureLoaded() {} - - /** - * Loads `base` from the `natives/-/` classpath resources a - * packaged desktop build bundles; false when it isn't bundled. - */ - private fun loadBundled(base: String): Boolean { - val os = System.getProperty("os.name").lowercase() - val arch = System.getProperty("os.arch").lowercase() - val osId = when { - os.contains("mac") || os.contains("darwin") -> "darwin" - os.contains("win") -> "windows" - else -> "linux" - } - val archId = if (arch == "aarch64" || arch == "arm64") "arm64" else "x64" - val file = when (osId) { - "darwin" -> "lib$base.dylib" - "windows" -> "$base.dll" - else -> "lib$base.so" - } - val stream = NativeLibrary::class.java.getResourceAsStream("/natives/$osId-$archId/$file") ?: return false - val dir = java.nio.file.Files.createTempDirectory("$base-natives") - val target = dir.resolve(file) - stream.use { java.nio.file.Files.copy(it, target) } - target.toFile().deleteOnExit() - dir.toFile().deleteOnExit() - System.load(target.toAbsolutePath().toString()) - return true - } -} - -/** - * One strong reference to a native object, released exactly once: by - * [close], or by [NativeCleaner] once the owning wrapper is unreachable. - * Every call borrows the address for its duration, and a close that races an - * in-flight call defers the release until the last borrow ends, so the object - * is never freed while native code is using it. - */ -internal class NativeHandle(val address: Long, private val release: (Long) -> Unit) { - // Bit 0 is the closed flag; the remaining bits count in-flight borrows. - private val state = AtomicLong(0L) - - val isClosed: Boolean get() = state.get() and 1L != 0L - - fun acquire(): Long { - while (true) { - val s = state.get() - check(s and 1L == 0L) { "native object used after close()" } - if (state.compareAndSet(s, s + 2L)) return address - } - } - - fun unacquire() { - if (state.addAndGet(-2L) == 1L) release(address) - } - - fun close() { - while (true) { - val s = state.get() - if (s and 1L != 0L) return - if (state.compareAndSet(s, s or 1L)) { - if (s == 0L) release(address) - return - } - } - } -} - -/** Runs [block] with the native address borrowed for its duration. */ -internal inline fun NativeHandle.borrow(block: (Long) -> R): R { - val address = acquire() - try { - return block(address) - } finally { - unacquire() - } -} - -/** [borrow] for an optional object: a missing object lends address 0. */ -internal inline fun NativeHandle?.borrowOrNull(block: (Long) -> R): R = - if (this == null) block(0L) else borrow(block) - -/** - * Releases the native reference of every wrapper that becomes unreachable - * without being closed. A daemon thread drains a phantom-reference queue, so - * this works on every Android API level and every JVM. - */ -internal object NativeCleaner { - private val queue = ReferenceQueue() - private val pending: MutableSet = Collections.newSetFromMap(ConcurrentHashMap()) - - private class Entry(owner: Any, val handle: NativeHandle, queue: ReferenceQueue) : - PhantomReference(owner, queue) - - init { - val thread = Thread({ drain() }, "edge_cases-cleaner") - thread.isDaemon = true - thread.start() - } - - /** Releases [handle] once [owner] is unreachable; returns [handle]. */ - fun register(owner: Any, handle: NativeHandle): NativeHandle { - pending.add(Entry(owner, handle, queue)) - return handle - } - - private fun drain() { - while (true) { - val entry = queue.remove() as Entry - pending.remove(entry) - try { - entry.handle.close() - } catch (_: Throwable) { - } - } - } -} - -/** UTF-8 bytes of [value], for a string crossing into native code. */ -internal fun encodeUtf8(value: String): ByteArray = value.toByteArray(Charsets.UTF_8) - -/** The string a native UTF-8 byte sequence spells. */ -internal fun decodeUtf8(bytes: ByteArray): String = String(bytes, Charsets.UTF_8) - -// END Runtime.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__build_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__build_gradle_kts.snap index 4e443da1..e533c56b 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__build_gradle_kts.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__build_gradle_kts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate plugins { id("com.android.library") version "8.7.3" @@ -14,6 +14,10 @@ plugins { group = "kitchen_sink" version = "0.1.0" +// A packaged build ships a prebuilt libkitchen_sink_jni.so next to the producer +// library in src/main/jniLibs//; without one, CMake builds it. +val prebuiltJni = fileTree("src/main/jniLibs") { include("**/libkitchen_sink_jni.so") }.files.isNotEmpty() + android { namespace = "kitchen_sink" compileSdk = 35 @@ -23,9 +27,11 @@ android { } // Builds libkitchen_sink_jni.so per ABI, linked against the producer library in // src/main/jniLibs// (see src/main/cpp/CMakeLists.txt). - externalNativeBuild { - cmake { - path = file("src/main/cpp/CMakeLists.txt") + if (!prebuiltJni) { + externalNativeBuild { + cmake { + path = file("src/main/cpp/CMakeLists.txt") + } } } compileOptions { diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__consumer_rules_pro.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__consumer_rules_pro.snap index 38aee25a..62642aac 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__consumer_rules_pro.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__consumer_rules_pro.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate # The JNI shim looks classes and methods up by name. -keep class kitchen_sink.** { *; } diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__settings_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__settings_gradle_kts.snap index 30bdfbbf..176f7c5a 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__settings_gradle_kts.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__settings_gradle_kts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate pluginManagement { repositories { diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_CMakeLists_txt.snap index 4ed25966..1f99f6c6 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_CMakeLists_txt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_CMakeLists_txt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate cmake_minimum_required(VERSION 3.18) project(kitchen_sink_jni C) diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_h.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_h.snap deleted file mode 100644 index 14faeffb..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_h.snap +++ /dev/null @@ -1,219 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef KITCHEN_SINK_H -#define KITCHEN_SINK_H - -#include -#include -#include - -#ifndef KITCHEN_SINK_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef KITCHEN_SINK_BUILD -# define KITCHEN_SINK_API __declspec(dllexport) -# else -# define KITCHEN_SINK_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define KITCHEN_SINK_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define KITCHEN_SINK_API __attribute__((visibility("default"))) -# else -# define KITCHEN_SINK_API -# endif -#endif - -#ifndef KITCHEN_SINK_DEPRECATED -# if defined(_MSC_VER) -# define KITCHEN_SINK_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define KITCHEN_SINK_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define KITCHEN_SINK_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports kitchen_sink_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define KITCHEN_SINK_ABI_VERSION 3u -KITCHEN_SINK_API uint32_t kitchen_sink_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by kitchen_sink_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct kitchen_sink_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} kitchen_sink_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -KITCHEN_SINK_API void kitchen_sink_error_set(kitchen_sink_error* err, int32_t code, const char* message); -KITCHEN_SINK_API void kitchen_sink_error_clear(kitchen_sink_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - kitchen_sink_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_error_free(kitchen_sink_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -KITCHEN_SINK_API void kitchen_sink_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct kitchen_sink_cancel_token kitchen_sink_cancel_token; -KITCHEN_SINK_API kitchen_sink_cancel_token* kitchen_sink_cancel_token_create(void); -KITCHEN_SINK_API void kitchen_sink_cancel_token_cancel(kitchen_sink_cancel_token* token); -KITCHEN_SINK_API bool kitchen_sink_cancel_token_is_cancelled(const kitchen_sink_cancel_token* token); -KITCHEN_SINK_API void kitchen_sink_cancel_token_destroy(kitchen_sink_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -KITCHEN_SINK_API uint8_t* kitchen_sink_alloc(uint32_t size); -KITCHEN_SINK_API void kitchen_sink_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -KITCHEN_SINK_API uint64_t kitchen_sink_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define KITCHEN_SINK_SHARED_CHECKSUM 0x42c4ce2c8d0af052ull -KITCHEN_SINK_API uint64_t kitchen_sink_shared_checksum(void); -#define KITCHEN_SINK_KITCHEN_CHECKSUM 0xd83da75b24b545ffull -KITCHEN_SINK_API uint64_t kitchen_sink_kitchen_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with kitchen_sink_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * kitchen_sink_buffer.h declares a C struct and codec for each. - */ - -/** Task priority levels */ -typedef enum { - /** Low priority */ - kitchen_sink_kitchen_Priority_Low = 0, - /** Normal priority */ - kitchen_sink_kitchen_Priority_Normal = 1, - /** High priority */ - kitchen_sink_kitchen_Priority_High = 2 -} kitchen_sink_kitchen_Priority; -/** Error codes reported by throwing functions in the `kitchen` module tree. */ -typedef enum { kitchen_sink_kitchen_KitchenErrors_NotFound = 1, kitchen_sink_kitchen_KitchenErrors_InvalidInput = 2 } kitchen_sink_kitchen_KitchenErrors; -typedef struct kitchen_sink_kitchen_Gadget kitchen_sink_kitchen_Gadget; -typedef struct kitchen_sink_kitchen_StreamGadgetsIterator kitchen_sink_kitchen_StreamGadgetsIterator; -typedef struct kitchen_sink_kitchen_StreamItemsIterator kitchen_sink_kitchen_StreamItemsIterator; -/** - * Consumer-implemented observer of ready items - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct kitchen_sink_kitchen_ReadyListener_vtable { - /** Fires when an item is ready */ - void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); - /** Receives the item itself and says whether to keep listening */ - bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); -} kitchen_sink_kitchen_ReadyListener_vtable; -typedef void (*kitchen_sink_kitchen_do_async_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*kitchen_sink_kitchen_do_cancellable_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: shared -/** Trivial cross-module helper */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_shared_ping(size_t* out_len, kitchen_sink_error* out_err); - -// Module: kitchen -/** A pocket-sized interface exercising the object surface */ -/** Create a gadget with the given id */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_new(int64_t id, kitchen_sink_error* out_err); -/** The gadget subsystem version string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_version(size_t* out_len, kitchen_sink_error* out_err); -/** Render the gadget as a human-readable string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_describe(const kitchen_sink_kitchen_Gadget* self, size_t* out_len, kitchen_sink_error* out_err); -/** Poke the gadget a number of times, failing on a negative count */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_Gadget_poke(const kitchen_sink_kitchen_Gadget* self, int32_t times, kitchen_sink_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_clone(const kitchen_sink_kitchen_Gadget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_kitchen_Gadget_destroy(kitchen_sink_kitchen_Gadget* self); - -/** Identity for bool */ -KITCHEN_SINK_API bool kitchen_sink_kitchen_bool_id(bool x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_i32_id(int32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API uint32_t kitchen_sink_kitchen_u32_id(uint32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int64_t kitchen_sink_kitchen_i64_id(int64_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API double kitchen_sink_kitchen_f64_id(double x, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_string(const uint8_t* s_ptr, size_t s_len, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_bytes(const uint8_t* b_ptr, size_t b_len, size_t* out_len, kitchen_sink_error* out_err); -/** Register a listener, returning how many are attached */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); -/** Optional object in both directions */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_maybe_gadget(const kitchen_sink_kitchen_Gadget* fallback, kitchen_sink_error* out_err); -/** A list of objects */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_gadgets(size_t* out_len, kitchen_sink_error* out_err); -/** An iterator over objects */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamGadgetsIterator* kitchen_sink_kitchen_stream_gadgets(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamGadgetsIterator_next(kitchen_sink_kitchen_StreamGadgetsIterator* iter, kitchen_sink_kitchen_Gadget** out_item, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamGadgetsIterator_destroy(kitchen_sink_kitchen_StreamGadgetsIterator* iter); -/** Optional struct return */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_maybe_item(int64_t id, size_t* out_len, kitchen_sink_error* out_err); -/** List of structs */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_list_items(size_t* out_len, kitchen_sink_error* out_err); -/** Map return type */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_get_attrs(size_t* out_len, kitchen_sink_error* out_err); -/** Iterator return type */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamItemsIterator* kitchen_sink_kitchen_stream_items(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamItemsIterator_next(kitchen_sink_kitchen_StreamItemsIterator* iter, const uint8_t** out_item, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamItemsIterator_destroy(kitchen_sink_kitchen_StreamItemsIterator* iter); -/** Returns the shared Token type from another module */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_cross_module_token(size_t* out_len, kitchen_sink_error* out_err); -/** Async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_async(const uint8_t* input_ptr, size_t input_len, kitchen_sink_kitchen_do_async_callback callback, void* context); -/** Cancellable async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_cancellable(const uint8_t* input_ptr, size_t input_len, kitchen_sink_cancel_token* cancel_token, kitchen_sink_kitchen_do_cancellable_callback callback, void* context); -/** Legacy operation kept for compatibility */ -KITCHEN_SINK_DEPRECATED("Use new_op instead") -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_legacy_op(kitchen_sink_error* out_err); -/** Replacement for legacy_op */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_new_op(kitchen_sink_error* out_err); - -// Module: kitchen_nested -/** Trivial nested-module function */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_nested_hello(size_t* out_len, kitchen_sink_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // KITCHEN_SINK_H - -// END kitchen_sink.h diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_jni_c.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_jni_c.snap index feb61be5..8e24d0b9 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_jni_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_cpp_kitchen_sink_jni_c.snap @@ -1,14 +1,15 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #include #include #include #include +#include #include #include @@ -72,8 +73,9 @@ static inline jbyteArray Jni_take_bytes(JNIEnv* env, const uint8_t* ptr, size_t return out; } -/* Throws the Kotlin exception for `err` (mapped through domain `domain`, - 0 for the generic one) and clears `err`. */ +/* Throws the Kotlin exception for `err` (mapped through error domain + `domain`, or 0 for a call that can't fail, which raises the trap) and + clears `err`. */ static inline void Jni_throw(JNIEnv* env, kitchen_sink_error* err, jint domain) { const char* message = err->message != NULL ? err->message : ""; jbyteArray text = Jni_new_bytes(env, (const uint8_t*)message, strlen(message)); @@ -100,6 +102,50 @@ static jint Jni_load_error(JNIEnv* env, const char* message) { return JNI_ERR; } +/* One contract entry these bindings were generated with: a declaration's + id and signature hash, plus its dotted path for the load error. */ +typedef struct { + uint64_t id; + uint64_t hash; + const char* path; +} Jni_contract_entry; + +/* Checks that the library's contract table (from `contract`, sorted by id) + has every expected entry with an equal hash; entries only the library + has are fine. Fails loading with an error naming the first declaration + that's missing or changed. */ +static inline jint Jni_check_contract(JNIEnv* env, const kitchen_sink_contract_entry* (*contract)(size_t*), const Jni_contract_entry* expected, size_t count) { + size_t len = 0; + const kitchen_sink_contract_entry* table = contract(&len); + if (table == NULL) { + len = 0; + } + for (size_t i = 0; i < count; i++) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) { + lo = mid + 1; + } else { + hi = mid; + } + } + const char* problem = NULL; + if (lo == len || table[lo].id != expected[i].id) { + problem = "is missing from the library"; + } else if (table[lo].hash != expected[i].hash) { + problem = "changed since these bindings were generated"; + } + if (problem != NULL) { + char message[512]; + snprintf(message, sizeof message, "%s: %s %s", "kitchen_sink", expected[i].path, problem); + return Jni_load_error(env, message); + } + } + return JNI_OK; +} + /* Generated below: the contract checks and the method IDs the shim caches. */ static jint Jni_load(JNIEnv* env); @@ -110,15 +156,15 @@ JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { if ((*vm)->GetEnv(vm, (void**)&env, JNI_VERSION_1_6) != JNI_OK) { return JNI_ERR; } - if (kitchen_sink_abi_version() != 3u) { - return Jni_load_error(env, "kitchen_sink: the native library implements a different C ABI revision than these bindings (expected 3)"); + if (kitchen_sink_abi_version() != 4u) { + return Jni_load_error(env, "kitchen_sink: the native library implements a different C ABI revision than these bindings (expected 4)"); } jclass bridge = (*env)->FindClass(env, "kitchen_sink/JniBridge"); if (bridge == NULL) { return JNI_ERR; } Jni_bridge = (jclass)(*env)->NewGlobalRef(env, bridge); - Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Lkitchen_sink/FfiException;"); + Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ljava/lang/Throwable;"); if (Jni_error == NULL) { return JNI_ERR; } @@ -136,38 +182,68 @@ JNIEXPORT jlong JNICALL Java_kitchen_1sink_JniBridge_debug_1live(JNIEnv* env, jc /* Producer threads (async completions, callback calls, vtable `free`) are usually not JVM threads. The first time one calls in it attaches as a - daemon and keeps its JNIEnv; a thread-local destructor detaches it when - the thread exits, so a busy producer thread pays for one attach, not one - per call. */ + daemon, and a thread-local destructor (a pthread key, or a fiber-local + slot on Windows) detaches it when the thread exits, so a busy producer + thread pays for one attach, not one per call. If the destructor can't be + registered, the thread detaches again as soon as the call ends. */ #if defined(_WIN32) -static void Jni_remember_attach(void) {} +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#include + +static DWORD Jni_env_slot = FLS_OUT_OF_INDEXES; +static INIT_ONCE Jni_env_once = INIT_ONCE_STATIC_INIT; + +static VOID WINAPI Jni_thread_exit(PVOID value) { + if (value != NULL && Jni_vm != NULL) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + +static BOOL CALLBACK Jni_make_env_slot(PINIT_ONCE once, PVOID param, PVOID* context) { + (void)once; + (void)param; + (void)context; + Jni_env_slot = FlsAlloc(Jni_thread_exit); + return TRUE; +} + +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { + InitOnceExecuteOnce(&Jni_env_once, Jni_make_env_slot, NULL, NULL); + return Jni_env_slot != FLS_OUT_OF_INDEXES && FlsSetValue(Jni_env_slot, (PVOID)Jni_vm); +} #else #include static pthread_key_t Jni_env_key; +static int Jni_env_key_ok = 0; static pthread_once_t Jni_env_once = PTHREAD_ONCE_INIT; -static void Jni_detach(void* env) { - (void)env; - if (Jni_vm != NULL) { +static void Jni_thread_exit(void* value) { + if (value != NULL && Jni_vm != NULL) { (*Jni_vm)->DetachCurrentThread(Jni_vm); } } static void Jni_make_env_key(void) { - pthread_key_create(&Jni_env_key, Jni_detach); + Jni_env_key_ok = pthread_key_create(&Jni_env_key, Jni_thread_exit) == 0; } -static void Jni_remember_attach(void) { +/* Arranges for the calling thread to detach when it exits; 0 if it can't. */ +static int Jni_remember_attach(void) { pthread_once(&Jni_env_once, Jni_make_env_key); - pthread_setspecific(Jni_env_key, (void*)Jni_vm); + return Jni_env_key_ok && pthread_setspecific(Jni_env_key, (void*)Jni_vm) == 0; } #endif /* The calling thread's JNIEnv, attaching the thread if needed; NULL when - the JVM is unavailable. */ -static JNIEnv* Jni_env(void) { + the JVM is unavailable. `*detach` is set when the caller must hand it + back with Jni_env_done because the thread can't detach itself at exit. */ +static JNIEnv* Jni_env(int* detach) { JNIEnv* env = NULL; + *detach = 0; if (Jni_vm == NULL) { return NULL; } @@ -179,10 +255,17 @@ static JNIEnv* Jni_env(void) { if ((*Jni_vm)->AttachCurrentThreadAsDaemon(Jni_vm, (void*)&env, NULL) != JNI_OK) { return NULL; } - Jni_remember_attach(); + *detach = !Jni_remember_attach(); return env; } +/* Ends a use of Jni_env's JNIEnv, detaching the thread if Jni_env said so. */ +static void Jni_env_done(int detach) { + if (detach) { + (*Jni_vm)->DetachCurrentThread(Jni_vm); + } +} + /* Async completions deliver to the pinned Kotlin `NativeCompletion`. */ static jmethodID Jni_on_unit = NULL; static jmethodID Jni_on_boolean = NULL; @@ -219,8 +302,8 @@ static jint Jni_load_async(JNIEnv* env) { /* Starts a completion on the producer thread: returns the JNIEnv with a local frame pushed, or NULL when the call failed (the failure, or the loss of the JVM, has then been handled and `context` released). */ -static JNIEnv* Jni_complete_begin(void* context, kitchen_sink_error* err) { - JNIEnv* env = Jni_env(); +static JNIEnv* Jni_complete_begin(void* context, kitchen_sink_error* err, int* detach) { + JNIEnv* env = Jni_env(detach); if (env == NULL) { kitchen_sink_error_free(err); return NULL; @@ -228,6 +311,7 @@ static JNIEnv* Jni_complete_begin(void* context, kitchen_sink_error* err) { if ((*env)->PushLocalFrame(env, 8) != 0) { (*env)->ExceptionClear(env); kitchen_sink_error_free(err); + Jni_env_done(*detach); return NULL; } if (err == NULL) { @@ -248,18 +332,20 @@ static JNIEnv* Jni_complete_begin(void* context, kitchen_sink_error* err) { } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(*detach); return NULL; } /* Finishes a successful completion: reports anything the Kotlin side threw (it has no caller to propagate to) and releases `context`. */ -static void Jni_complete_end(JNIEnv* env, void* context) { +static void Jni_complete_end(JNIEnv* env, void* context, int detach) { if ((*env)->ExceptionCheck(env)) { (*env)->ExceptionDescribe(env); (*env)->ExceptionClear(env); } (*env)->PopLocalFrame(env, NULL); (*env)->DeleteGlobalRef(env, (jobject)context); + Jni_env_done(detach); } JNIEXPORT jlong JNICALL Java_kitchen_1sink_JniBridge_cancel_1token_1create(JNIEnv* env, jclass cls) { @@ -282,18 +368,14 @@ JNIEXPORT void JNICALL Java_kitchen_1sink_JniBridge_cancel_1token_1destroy(JNIEn /* Callback interfaces: the producer holds a global reference to the Kotlin implementation as `ctx`; trampolines call the matching static dispatch - method on JniBridge from whatever thread the producer uses. */ -static jmethodID Jni_describe = NULL; - -static jint Jni_load_callbacks(JNIEnv* env) { - Jni_describe = (*env)->GetStaticMethodID(env, Jni_bridge, "describe", "(Ljava/lang/Throwable;)[B"); - return Jni_describe != NULL ? JNI_OK : JNI_ERR; -} + shim on JniBridge from whatever thread the producer uses. The shim never + lets an exception escape: it reports a failure itself through `out_err` + (with the two exports below) and returns a zero value. */ /* Enters a trampoline: the JNIEnv with a local frame pushed, or NULL after reporting the failure through `out_err`. */ -static JNIEnv* Jni_callback_begin(kitchen_sink_error* out_err) { - JNIEnv* env = Jni_env(); +static JNIEnv* Jni_callback_begin(kitchen_sink_error* out_err, int* detach) { + JNIEnv* env = Jni_env(detach); if (env == NULL) { kitchen_sink_error_set(out_err, -4, "could not attach the calling thread to the JVM"); return NULL; @@ -301,96 +383,216 @@ static JNIEnv* Jni_callback_begin(kitchen_sink_error* out_err) { if ((*env)->PushLocalFrame(env, 16) != 0) { (*env)->ExceptionClear(env); kitchen_sink_error_set(out_err, -4, "JNI local frame exhausted"); + Jni_env_done(*detach); return NULL; } return env; } -/* Leaves a trampoline. When the implementation threw, reports its - `toString()` through `out_err` with code -4 and returns 1; nothing ever - unwinds through the producer's frames. */ -static int Jni_callback_end(JNIEnv* env, kitchen_sink_error* out_err) { - int failed = 0; +/* Leaves a trampoline. An exception still pending means the dispatch itself + failed (the shim catches everything the implementation throws), which + reaches the producer as code -4; nothing ever unwinds through its frames. */ +static void Jni_callback_end(JNIEnv* env, kitchen_sink_error* out_err, int detach) { if ((*env)->ExceptionCheck(env)) { - failed = 1; - jthrowable ex = (*env)->ExceptionOccurred(env); (*env)->ExceptionClear(env); - jbyteArray text = (jbyteArray)(*env)->CallStaticObjectMethod(env, Jni_bridge, Jni_describe, ex); - char* message = NULL; - if ((*env)->ExceptionCheck(env)) { - (*env)->ExceptionClear(env); - } else if (text != NULL) { - jsize len = (*env)->GetArrayLength(env, text); - message = (char*)malloc((size_t)len + 1u); - if (message != NULL) { - (*env)->GetByteArrayRegion(env, text, 0, len, (jbyte*)message); - message[len] = '\0'; - } - } - kitchen_sink_error_set(out_err, -4, message != NULL ? message : "callback implementation threw"); - free(message); + kitchen_sink_error_set(out_err, -4, "the JVM failed to dispatch the callback"); } (*env)->PopLocalFrame(env, NULL); - return failed; + Jni_env_done(detach); } /* The vtable `free` entry: drops the producer's reference to the Kotlin - implementation. */ + implementation, from whatever thread the producer releases it on. */ static void Jni_release_callback(void* ctx) { - JNIEnv* env = Jni_env(); - if (env != NULL && ctx != NULL) { - (*env)->DeleteGlobalRef(env, (jobject)ctx); + int detach = 0; + JNIEnv* env = Jni_env(&detach); + if (env != NULL) { + if (ctx != NULL) { + (*env)->DeleteGlobalRef(env, (jobject)ctx); + } + Jni_env_done(detach); + } +} + +/* `JniBridge.error_set`: reports a failed callback through the trampoline's + `out_err` (the message is cut at its first NUL). */ +JNIEXPORT void JNICALL Java_kitchen_1sink_JniBridge_error_1set(JNIEnv* env, jclass cls, jlong err, jint code, jbyteArray message) { + (void)cls; + Jni_bytes m = Jni_borrow_bytes(env, message); + char* text = (char*)malloc(m.len + 1u); + if (text != NULL) { + if (m.len > 0) { + memcpy(text, m.ptr, m.len); + } + text[m.len] = '\0'; + } + Jni_release_bytes(env, &m); + kitchen_sink_error_set((kitchen_sink_error*)(intptr_t)err, (int32_t)code, text != NULL ? text : "callback failed"); + free(text); +} + +/* `JniBridge.error_set_payload`: attaches a typed error's fields (a value + buffer) to the failure `error_set` reported. */ +JNIEXPORT void JNICALL Java_kitchen_1sink_JniBridge_error_1set_1payload(JNIEnv* env, jclass cls, jlong err, jbyteArray payload) { + (void)cls; + Jni_bytes p = Jni_borrow_bytes(env, payload); + kitchen_sink_error_set_payload((kitchen_sink_error*)(intptr_t)err, p.ptr, p.len); + Jni_release_bytes(env, &p); +} + +/* Hands a string, bytes, or buffer return to the producer: a run allocated + with kitchen_sink_alloc in the out slots, which the producer adopts. A NULL + `value` (the shim reported a failure) leaves the slots empty. */ +static void Jni_callback_return_bytes(JNIEnv* env, jbyteArray value, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err) { + if (value == NULL || (*env)->ExceptionCheck(env)) { + return; + } + jsize len = (*env)->GetArrayLength(env, value); + uint8_t* run = kitchen_sink_alloc((size_t)len); + if (len > 0 && run == NULL) { + kitchen_sink_error_set(out_err, -4, "out of memory"); + return; + } + if (len > 0) { + (*env)->GetByteArrayRegion(env, value, 0, len, (jbyte*)run); } + *out_ptr = run; + *out_len = (size_t)len; } static jmethodID Jni_mid_kitchen_ReadyListener_on_ready = NULL; static jmethodID Jni_mid_kitchen_ReadyListener_on_item = NULL; +static jmethodID Jni_mid_kitchen_ReadyListener_label = NULL; +static jmethodID Jni_mid_kitchen_ReadyListener_latest = NULL; +static jmethodID Jni_mid_kitchen_ReadyListener_favorite = NULL; static void Jni_kitchen_ReadyListener_on_ready(void* ctx, int32_t p_code, const uint8_t* p_msg_ptr, size_t p_msg_len, kitchen_sink_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { return; } - (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_kitchen_ReadyListener_on_ready, (jobject)ctx, (jint)p_code, Jni_new_bytes(env, p_msg_ptr, p_msg_len)); - Jni_callback_end(env, out_err); + (*env)->CallStaticVoidMethod(env, Jni_bridge, Jni_mid_kitchen_ReadyListener_on_ready, (jobject)ctx, (jlong)(intptr_t)out_err, (jint)p_code, Jni_new_bytes(env, p_msg_ptr, p_msg_len)); + Jni_callback_end(env, out_err, detach); } static bool Jni_kitchen_ReadyListener_on_item(void* ctx, const uint8_t* p_item_ptr, size_t p_item_len, kitchen_sink_kitchen_Gadget* p_gadget, kitchen_sink_error* out_err) { - JNIEnv* env = Jni_callback_begin(out_err); + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); if (env == NULL) { + kitchen_sink_kitchen_Gadget_destroy(p_gadget); return (bool)0; } - jboolean rv = (*env)->CallStaticBooleanMethod(env, Jni_bridge, Jni_mid_kitchen_ReadyListener_on_item, (jobject)ctx, Jni_new_bytes(env, p_item_ptr, p_item_len), (jlong)(intptr_t)p_gadget); - if (Jni_callback_end(env, out_err)) { - return (bool)0; - } + jboolean rv = (*env)->CallStaticBooleanMethod(env, Jni_bridge, Jni_mid_kitchen_ReadyListener_on_item, (jobject)ctx, (jlong)(intptr_t)out_err, Jni_new_bytes(env, p_item_ptr, p_item_len), (jlong)(intptr_t)p_gadget); + Jni_callback_end(env, out_err, detach); return rv == JNI_TRUE; } -static const kitchen_sink_kitchen_ReadyListener_vtable Jni_kitchen_ReadyListener_vtable = {Jni_kitchen_ReadyListener_on_ready, Jni_kitchen_ReadyListener_on_item, Jni_release_callback}; +static void Jni_kitchen_ReadyListener_label(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err) { + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); + if (env == NULL) { + return; + } + jbyteArray rv = (jbyteArray)(*env)->CallStaticObjectMethod(env, Jni_bridge, Jni_mid_kitchen_ReadyListener_label, (jobject)ctx, (jlong)(intptr_t)out_err); + Jni_callback_return_bytes(env, rv, out_ptr, out_len, out_err); + Jni_callback_end(env, out_err, detach); +} + +static void Jni_kitchen_ReadyListener_latest(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err) { + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); + if (env == NULL) { + return; + } + jbyteArray rv = (jbyteArray)(*env)->CallStaticObjectMethod(env, Jni_bridge, Jni_mid_kitchen_ReadyListener_latest, (jobject)ctx, (jlong)(intptr_t)out_err); + Jni_callback_return_bytes(env, rv, out_ptr, out_len, out_err); + Jni_callback_end(env, out_err, detach); +} + +static kitchen_sink_kitchen_Gadget* Jni_kitchen_ReadyListener_favorite(void* ctx, kitchen_sink_error* out_err) { + int detach = 0; + JNIEnv* env = Jni_callback_begin(out_err, &detach); + if (env == NULL) { + return (kitchen_sink_kitchen_Gadget*)0; + } + jlong rv = (*env)->CallStaticLongMethod(env, Jni_bridge, Jni_mid_kitchen_ReadyListener_favorite, (jobject)ctx, (jlong)(intptr_t)out_err); + Jni_callback_end(env, out_err, detach); + return (kitchen_sink_kitchen_Gadget*)(intptr_t)rv; +} + +static const kitchen_sink_kitchen_ReadyListener_vtable Jni_kitchen_ReadyListener_vtable = {sizeof(kitchen_sink_kitchen_ReadyListener_vtable), 0, Jni_release_callback, Jni_kitchen_ReadyListener_on_ready, Jni_kitchen_ReadyListener_on_item, Jni_kitchen_ReadyListener_label, Jni_kitchen_ReadyListener_latest, Jni_kitchen_ReadyListener_favorite}; + +static const Jni_contract_entry Jni_contract_shared[] = { + {UINT64_C(0x709e234caa962446), UINT64_C(0x3894f2ca452e9f98), "shared.ping"}, + {UINT64_C(0xcfb306fa5801ee5b), UINT64_C(0x1cbd9e10983cac2c), "shared.Token"}, +}; + +static const Jni_contract_entry Jni_contract_kitchen[] = { + {UINT64_C(0x01d4f09bbddc986f), UINT64_C(0x29eb88fecc66544d), "kitchen.Gadget.new"}, + {UINT64_C(0x10472d8fe748c2ee), UINT64_C(0xb265832b6a759fc4), "kitchen.maybe_gadget"}, + {UINT64_C(0x11368eec07a74078), UINT64_C(0xfa049f78b77e146a), "kitchen.gadgets"}, + {UINT64_C(0x137f08a29cde6021), UINT64_C(0xda8de7aeb2dbb8a3), "kitchen.stream_gadgets"}, + {UINT64_C(0x2111b54980826dd7), UINT64_C(0x452783f3e6ca6e1d), "kitchen.do_cancellable"}, + {UINT64_C(0x29dddbc60bf8d7db), UINT64_C(0x452a3e573d57f6db), "kitchen.Priority"}, + {UINT64_C(0x2b227003b74a0d16), UINT64_C(0x12332957470e9782), "kitchen.maybe_subscribe"}, + {UINT64_C(0x2dd66cb0064f0494), UINT64_C(0x60197363fefd92dd), "kitchen.Item"}, + {UINT64_C(0x33c265bbf079b0e9), UINT64_C(0xc3dcd0927e6c1deb), "kitchen.u32_id"}, + {UINT64_C(0x33e6381d01660dfe), UINT64_C(0x077c63c31fd7f90a), "kitchen.KitchenErrors"}, + {UINT64_C(0x3f5fc99d73504b76), UINT64_C(0x2df88d710f1ea10e), "kitchen.Gadget.poke"}, + {UINT64_C(0x4c5764cf109528d1), UINT64_C(0x2a65529af4338a58), "kitchen.maybe_item"}, + {UINT64_C(0x4de17dfb93fd494d), UINT64_C(0xba0ad0dd52ed74af), "kitchen.i32_id"}, + {UINT64_C(0x504e95b7f7e65570), UINT64_C(0x01b3e8e0c9a10e23), "kitchen.stream_items"}, + {UINT64_C(0x68cdb9a4bd3e3dd5), UINT64_C(0xc678b9164348d497), "kitchen.bool_id"}, + {UINT64_C(0x69e7cd703d16eb85), UINT64_C(0xf3b28dd22ae9e689), "kitchen.do_async"}, + {UINT64_C(0x70865079ce41be82), UINT64_C(0xbe3b69464672af46), "kitchen.Gadget.describe"}, + {UINT64_C(0x71c5a1116cb7c49a), UINT64_C(0xcba1837108a89b84), "kitchen.echo_bytes"}, + {UINT64_C(0x735ee1e0049ee91a), UINT64_C(0xca74320bc53eddab), "kitchen.echo_string"}, + {UINT64_C(0x76d7fc58f4e41d84), UINT64_C(0x7373a0e47d30e690), "kitchen.i64_id"}, + {UINT64_C(0x79abfe1aa960fd6a), UINT64_C(0xbca7a9e5eb26f3a6), "kitchen.nested.hello"}, + {UINT64_C(0x7d3dd7d5cf6749e3), UINT64_C(0xa192f5a7298adb39), "kitchen.new_op"}, + {UINT64_C(0x89a5e664bc25c67b), UINT64_C(0x5764a95e81ecfd31), "kitchen.f64_id"}, + {UINT64_C(0x8cfdbdcecac5f7f1), UINT64_C(0x07e272f09c41fffc), "kitchen.Gadget.version"}, + {UINT64_C(0x974b9f3e823c1738), UINT64_C(0xb81b91e975695f27), "kitchen.cross_module_token"}, + {UINT64_C(0xaca0655fac15d230), UINT64_C(0x11c4d2b28abe07bb), "kitchen.list_items"}, + {UINT64_C(0xc883b4afae20543e), UINT64_C(0xc40560e5411ff392), "kitchen.legacy_op"}, + {UINT64_C(0xf2f7753e6e1dffc2), UINT64_C(0xc9047a95bdf49e93), "kitchen.get_attrs"}, + {UINT64_C(0xfae25d3391922ead), UINT64_C(0x1c7f95739f3210b8), "kitchen.subscribe"}, + {UINT64_C(0xfd0059b05cb2fcfb), UINT64_C(0x248bf8404a5823b4), "kitchen.Gadget"}, + {UINT64_C(0xfed9c551fb9fd2d6), UINT64_C(0x59aec980ef3f3291), "kitchen.ReadyListener"}, +}; static jint Jni_load(JNIEnv* env) { (void)env; - if (kitchen_sink_shared_checksum() != UINT64_C(0x42c4ce2c8d0af052)) { - return Jni_load_error(env, "kitchen_sink: module 'shared' does not match these bindings (its contract checksum differs); regenerate the bindings or rebuild the library"); - } - if (kitchen_sink_kitchen_checksum() != UINT64_C(0xd83da75b24b545ff)) { - return Jni_load_error(env, "kitchen_sink: module 'kitchen' does not match these bindings (its contract checksum differs); regenerate the bindings or rebuild the library"); + if (Jni_check_contract(env, kitchen_sink_shared_contract, Jni_contract_shared, sizeof Jni_contract_shared / sizeof Jni_contract_shared[0]) != JNI_OK) { + return JNI_ERR; } - if (Jni_load_async(env) != JNI_OK) { + if (Jni_check_contract(env, kitchen_sink_kitchen_contract, Jni_contract_kitchen, sizeof Jni_contract_kitchen / sizeof Jni_contract_kitchen[0]) != JNI_OK) { return JNI_ERR; } - if (Jni_load_callbacks(env) != JNI_OK) { + if (Jni_load_async(env) != JNI_OK) { return JNI_ERR; } - Jni_mid_kitchen_ReadyListener_on_ready = (*env)->GetStaticMethodID(env, Jni_bridge, "kitchen_ReadyListener_on_ready", "(Lkitchen_sink/ReadyListener;I[B)V"); + Jni_mid_kitchen_ReadyListener_on_ready = (*env)->GetStaticMethodID(env, Jni_bridge, "kitchen_ReadyListener_on_ready", "(Lkitchen_sink/ReadyListener;JI[B)V"); if (Jni_mid_kitchen_ReadyListener_on_ready == NULL) { return JNI_ERR; } - Jni_mid_kitchen_ReadyListener_on_item = (*env)->GetStaticMethodID(env, Jni_bridge, "kitchen_ReadyListener_on_item", "(Lkitchen_sink/ReadyListener;[BJ)Z"); + Jni_mid_kitchen_ReadyListener_on_item = (*env)->GetStaticMethodID(env, Jni_bridge, "kitchen_ReadyListener_on_item", "(Lkitchen_sink/ReadyListener;J[BJ)Z"); if (Jni_mid_kitchen_ReadyListener_on_item == NULL) { return JNI_ERR; } + Jni_mid_kitchen_ReadyListener_label = (*env)->GetStaticMethodID(env, Jni_bridge, "kitchen_ReadyListener_label", "(Lkitchen_sink/ReadyListener;J)[B"); + if (Jni_mid_kitchen_ReadyListener_label == NULL) { + return JNI_ERR; + } + Jni_mid_kitchen_ReadyListener_latest = (*env)->GetStaticMethodID(env, Jni_bridge, "kitchen_ReadyListener_latest", "(Lkitchen_sink/ReadyListener;J)[B"); + if (Jni_mid_kitchen_ReadyListener_latest == NULL) { + return JNI_ERR; + } + Jni_mid_kitchen_ReadyListener_favorite = (*env)->GetStaticMethodID(env, Jni_bridge, "kitchen_ReadyListener_favorite", "(Lkitchen_sink/ReadyListener;J)J"); + if (Jni_mid_kitchen_ReadyListener_favorite == NULL) { + return JNI_ERR; + } return JNI_OK; } @@ -440,7 +642,7 @@ JNIEXPORT jint JNICALL Java_kitchen_1sink_JniBridge_kitchen_1i32_1id(JNIEnv* env return (jint)rv; } -JNIEXPORT jlong JNICALL Java_kitchen_1sink_JniBridge_kitchen_1u32_1id(JNIEnv* env, jclass cls, jlong p_x) { +JNIEXPORT jint JNICALL Java_kitchen_1sink_JniBridge_kitchen_1u32_1id(JNIEnv* env, jclass cls, jint p_x) { kitchen_sink_error err = {0, NULL, NULL, 0}; (void)cls; uint32_t rv = kitchen_sink_kitchen_u32_id((uint32_t)p_x, &err); @@ -448,7 +650,7 @@ JNIEXPORT jlong JNICALL Java_kitchen_1sink_JniBridge_kitchen_1u32_1id(JNIEnv* en Jni_throw(env, &err, 0); return 0; } - return (jlong)rv; + return (jint)rv; } JNIEXPORT jlong JNICALL Java_kitchen_1sink_JniBridge_kitchen_1i64_1id(JNIEnv* env, jclass cls, jlong p_x) { @@ -512,6 +714,17 @@ JNIEXPORT jint JNICALL Java_kitchen_1sink_JniBridge_kitchen_1subscribe(JNIEnv* e return (jint)rv; } +JNIEXPORT jboolean JNICALL Java_kitchen_1sink_JniBridge_kitchen_1maybe_1subscribe(JNIEnv* env, jclass cls, jobject p_listener) { + kitchen_sink_error err = {0, NULL, NULL, 0}; + (void)cls; + bool rv = kitchen_sink_kitchen_maybe_subscribe(p_listener != NULL ? (void*)(*env)->NewGlobalRef(env, p_listener) : NULL, p_listener != NULL ? &Jni_kitchen_ReadyListener_vtable : NULL, &err); + if (err.code != 0) { + Jni_throw(env, &err, 0); + return 0; + } + return rv ? JNI_TRUE : JNI_FALSE; +} + JNIEXPORT jlong JNICALL Java_kitchen_1sink_JniBridge_kitchen_1maybe_1gadget(JNIEnv* env, jclass cls, jlong p_fallback) { kitchen_sink_error err = {0, NULL, NULL, 0}; (void)cls; @@ -651,12 +864,13 @@ JNIEXPORT jbyteArray JNICALL Java_kitchen_1sink_JniBridge_kitchen_1cross_1module } static void Jni_done_kitchen_do_async(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_kitchen_1sink_JniBridge_kitchen_1do_1async(JNIEnv* env, jclass cls, jbyteArray p_input, jobject completion) { @@ -668,12 +882,13 @@ JNIEXPORT void JNICALL Java_kitchen_1sink_JniBridge_kitchen_1do_1async(JNIEnv* e } static void Jni_done_kitchen_do_cancellable(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len) { - JNIEnv* env = Jni_complete_begin(context, err); + int detach = 0; + JNIEnv* env = Jni_complete_begin(context, err, &detach); if (env == NULL) { return; } (*env)->CallVoidMethod(env, (jobject)context, Jni_on_bytes, Jni_take_bytes(env, result_ptr, result_len)); - Jni_complete_end(env, context); + Jni_complete_end(env, context, detach); } JNIEXPORT void JNICALL Java_kitchen_1sink_JniBridge_kitchen_1do_1cancellable(JNIEnv* env, jclass cls, jbyteArray p_input, jlong cancel_token, jobject completion) { diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Async_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Async_kt.snap index 1b5bacb0..9ef1e9d2 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Async_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Async_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Buffers_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Buffers_kt.snap index f712dd29..722dd74b 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Buffers_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Buffers_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink @@ -35,7 +35,7 @@ internal class BufferWriter { buf[len++] = v } - fun writeU8(v: Byte) = writeI8(v) + fun writeU8(v: UByte) = writeI8(v.toByte()) fun writeI16(v: Short) { reserve(2) @@ -44,7 +44,7 @@ internal class BufferWriter { buf[len++] = (b shr 8).toByte() } - fun writeU16(v: Short) = writeI16(v) + fun writeU16(v: UShort) = writeI16(v.toShort()) fun writeI32(v: Int) { reserve(4) @@ -54,14 +54,14 @@ internal class BufferWriter { buf[len++] = (v shr 24).toByte() } - fun writeU32(v: Long) = writeI32(v.toInt()) + fun writeU32(v: UInt) = writeI32(v.toInt()) fun writeI64(v: Long) { writeI32(v.toInt()) writeI32((v ushr 32).toInt()) } - fun writeU64(v: Long) = writeI64(v) + fun writeU64(v: ULong) = writeI64(v.toLong()) fun writeF32(v: Float) = writeI32(v.toRawBits()) @@ -118,13 +118,13 @@ internal class BufferWriter { /** * Validating little-endian reader for the value-buffer wire format. A * truncated buffer, an invalid flag byte, a zero object token, an oversized - * length, or trailing bytes raise [FfiException] code -3. + * length, or trailing bytes raise [NativeBugException] code -3. */ internal class BufferReader(private val buf: ByteArray) { private var pos = 0 private fun malformed(detail: String): Nothing = - throw FfiException(-3, "malformed value buffer: $detail") + throw NativeBugException(-3, "malformed value buffer: $detail") private fun take(n: Int): Int { if (n > buf.size - pos) malformed("truncated buffer") @@ -141,14 +141,14 @@ internal class BufferReader(private val buf: ByteArray) { fun readI8(): Byte = buf[take(1)] - fun readU8(): Byte = readI8() + fun readU8(): UByte = readI8().toUByte() fun readI16(): Short { val at = take(2) return ((buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)).toShort() } - fun readU16(): Short = readI16() + fun readU16(): UShort = readI16().toUShort() fun readI32(): Int { val at = take(4) @@ -158,7 +158,7 @@ internal class BufferReader(private val buf: ByteArray) { ((buf[at + 3].toInt() and 0xFF) shl 24) } - fun readU32(): Long = readI32().toLong() and 0xFFFFFFFFL + fun readU32(): UInt = readI32().toUInt() fun readI64(): Long { val lo = readI32().toLong() and 0xFFFFFFFFL @@ -166,7 +166,7 @@ internal class BufferReader(private val buf: ByteArray) { return lo or (hi shl 32) } - fun readU64(): Long = readI64() + fun readU64(): ULong = readI64().toULong() fun readF32(): Float = Float.fromBits(readI32()) diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Codecs_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Codecs_kt.snap new file mode 100644 index 00000000..b9aec5a9 --- /dev/null +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Codecs_kt.snap @@ -0,0 +1,42 @@ +--- +source: crates/weaveffi-cli/tests/snapshots.rs +--- +// Generated by WeaveFFI [VERSION] +// DO NOT EDIT. Your changes will be overwritten. +// To regenerate: weaveffi generate + +package kitchen_sink + +internal fun packListOfGadget(_w: BufferWriter, _v: List) = _w.writeList(_v) { _w.writeObject(it.cloneHandle(), JniBridge::kitchen_Gadget_destroy) } + +internal fun unpackListOfGadget(_r: BufferReader): List = _r.readList { Gadget.fromHandle(_r.readObject()) } + +internal fun packListOfItem(_w: BufferWriter, _v: List) = _w.writeList(_v) { packItem(_w, it) } + +internal fun unpackListOfItem(_r: BufferReader): List = _r.readList { unpackItem(_r) } + +internal fun packListOfString(_w: BufferWriter, _v: List) = _w.writeList(_v) { _w.writeString(it) } + +internal fun unpackListOfString(_r: BufferReader): List = _r.readList { _r.readString() } + +internal fun packMapOfStringToI32(_w: BufferWriter, _v: Map) = _w.writeMap(_v, { _w.writeString(it) }, { _w.writeI32(it) }) + +internal fun unpackMapOfStringToI32(_r: BufferReader): Map = _r.readMap({ _r.readString() }, { _r.readI32() }) + +internal fun packMapOfStringToString(_w: BufferWriter, _v: Map) = _w.writeMap(_v, { _w.writeString(it) }, { _w.writeString(it) }) + +internal fun unpackMapOfStringToString(_r: BufferReader): Map = _r.readMap({ _r.readString() }, { _r.readString() }) + +internal fun packOptionalOfGadget(_w: BufferWriter, _v: Gadget?) = _w.writeOptional(_v) { _w.writeObject(it.cloneHandle(), JniBridge::kitchen_Gadget_destroy) } + +internal fun unpackOptionalOfGadget(_r: BufferReader): Gadget? = _r.readOptional { Gadget.fromHandle(_r.readObject()) } + +internal fun packOptionalOfI64(_w: BufferWriter, _v: Long?) = _w.writeOptional(_v) { _w.writeI64(it) } + +internal fun unpackOptionalOfI64(_r: BufferReader): Long? = _r.readOptional { _r.readI64() } + +internal fun packOptionalOfItem(_w: BufferWriter, _v: Item?) = _w.writeOptional(_v) { packItem(_w, it) } + +internal fun unpackOptionalOfItem(_r: BufferReader): Item? = _r.readOptional { unpackItem(_r) } + +// END Codecs.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_JniBridge_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_JniBridge_kt.snap index c7b0826d..904daec0 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_JniBridge_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_JniBridge_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink @@ -17,20 +17,35 @@ internal object JniBridge { NativeLibrary.ensureLoaded() } - /** The exception for a native failure; domain 0 is the generic one. */ + /** + * The exception for a native failure: through error domain [domain] for a + * throwing call, or a [NativeBugException] (domain 0) for one that can't fail. + */ @JvmStatic - @Suppress("UNUSED_PARAMETER") - fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): FfiException { + fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): Throwable { val text = decodeUtf8(message) return when (domain) { 1 -> KitchenErrorsException.fromCode(code, text, payload) - else -> FfiException(code, text) + else -> NativeBugException(code, text) } } - /** The message a failed callback reports to the producer (code -4). */ - @JvmStatic - fun describe(error: Throwable): ByteArray = encodeUtf8(error.toString()) + /** + * Reports a failed callback through its vtable entry's `out_err` ([err]): + * [typed], a domain error the method is declared to throw, with its code and + * payload fields, and anything else as code -4 with its message. + */ + private fun fail(err: Long, error: Throwable, typed: FfiException?) { + if (typed != null) { + error_set(err, typed.code, encodeUtf8(typed.message ?: "")) + typed.encodePayload()?.let { error_set_payload(err, it) } + } else { + error_set(err, -4, encodeUtf8(error.message ?: error.toString())) + } + } + + @JvmStatic external fun error_set(err: Long, code: Int, message: ByteArray) + @JvmStatic external fun error_set_payload(err: Long, payload: ByteArray) @JvmStatic external fun debug_live(kind: Int): Long @JvmStatic external fun cancel_token_create(): Long @@ -41,12 +56,13 @@ internal object JniBridge { @JvmStatic external fun kitchen_Gadget_destroy(_self: Long) @JvmStatic external fun kitchen_bool_id(x: Boolean): Boolean @JvmStatic external fun kitchen_i32_id(x: Int): Int - @JvmStatic external fun kitchen_u32_id(x: Long): Long + @JvmStatic external fun kitchen_u32_id(x: Int): Int @JvmStatic external fun kitchen_i64_id(x: Long): Long @JvmStatic external fun kitchen_f64_id(x: Double): Double @JvmStatic external fun kitchen_echo_string(s: ByteArray): ByteArray @JvmStatic external fun kitchen_echo_bytes(b: ByteArray): ByteArray @JvmStatic external fun kitchen_subscribe(listener: ReadyListener): Int + @JvmStatic external fun kitchen_maybe_subscribe(listener: ReadyListener?): Boolean @JvmStatic external fun kitchen_maybe_gadget(fallback: Long): Long @JvmStatic external fun kitchen_gadgets(): ByteArray @JvmStatic external fun kitchen_stream_gadgets(): Long @@ -70,10 +86,48 @@ internal object JniBridge { @JvmStatic external fun kitchen_nested_hello(): ByteArray @JvmStatic - fun kitchen_ReadyListener_on_ready(_impl: ReadyListener, code: Int, msg: ByteArray) { _impl.onReady(code, decodeUtf8(msg)) } + fun kitchen_ReadyListener_on_ready(_impl: ReadyListener, _err: Long, code: Int, msg: ByteArray) { + try { + _impl.onReady(code, decodeUtf8(msg)) + } catch (_e: Throwable) { + fail(_err, _e, null) + } + } @JvmStatic - fun kitchen_ReadyListener_on_item(_impl: ReadyListener, item: ByteArray, gadget: Long): Boolean = _impl.onItem(decodeBuffer(item) { _r -> unpackItem(_r) }, Gadget.fromHandle(gadget)) + fun kitchen_ReadyListener_on_item(_impl: ReadyListener, _err: Long, item: ByteArray, gadget: Long): Boolean { + val _gadget = Gadget.fromHandle(gadget) + return try { + _impl.onItem(decodeBuffer(item) { _r -> unpackItem(_r) }, _gadget) + } catch (_e: Throwable) { + fail(_err, _e, null) + false + } + } + + @JvmStatic + fun kitchen_ReadyListener_label(_impl: ReadyListener, _err: Long): ByteArray? = try { + encodeUtf8(_impl.label()) + } catch (_e: Throwable) { + fail(_err, _e, _e as? KitchenErrorsException) + null + } + + @JvmStatic + fun kitchen_ReadyListener_latest(_impl: ReadyListener, _err: Long): ByteArray? = try { + encodeBuffer { _w -> packOptionalOfItem(_w, _impl.latest()) } + } catch (_e: Throwable) { + fail(_err, _e, null) + null + } + + @JvmStatic + fun kitchen_ReadyListener_favorite(_impl: ReadyListener, _err: Long): Long = try { + _impl.favorite().cloneHandle() + } catch (_e: Throwable) { + fail(_err, _e, null) + 0L + } } // END JniBridge.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Kitchen_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Kitchen_kt.snap index 68db0534..ea1d387c 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Kitchen_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Kitchen_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink @@ -34,7 +34,7 @@ enum class Priority(val value: Int) { companion object { /** The variant whose ABI value is [value]. */ fun fromValue(value: Int): Priority = entries.firstOrNull { it.value == value } - ?: throw FfiException(-3, "unknown Priority value $value") + ?: throw NativeBugException(-3, "unknown Priority value $value") } } @@ -50,7 +50,7 @@ data class Item( val enabled: Boolean, /** Scaling ratio */ val ratio: Double, - val nick: Long, + val nick: UInt, val payload: ByteArray, val tags: List, val attrs: Map, @@ -70,12 +70,12 @@ internal fun packItem(_w: BufferWriter, _v: Item) { _w.writeF64(_v.ratio) _w.writeU32(_v.nick) _w.writeBytes(_v.payload) - _w.writeList(_v.tags) { _v0 -> _w.writeString(_v0) } - _w.writeMap(_v.attrs, { _k0 -> _w.writeString(_k0) }, { _v0 -> _w.writeString(_v0) }) - _w.writeOptional(_v.parent) { _v0 -> _w.writeI64(_v0) } + packListOfString(_w, _v.tags) + packMapOfStringToString(_w, _v.attrs) + packOptionalOfI64(_w, _v.parent) packToken(_w, _v.token) _w.writeI32(_v.priority.value) - _w.writeOptional(_v.gadget) { _v0 -> _w.writeObject(_v0.cloneHandle(), JniBridge::kitchen_Gadget_destroy) } + packOptionalOfGadget(_w, _v.gadget) } internal fun unpackItem(_r: BufferReader): Item = Item( @@ -86,12 +86,12 @@ internal fun unpackItem(_r: BufferReader): Item = Item( _r.readF64(), _r.readU32(), _r.readBytes(), - _r.readList { _r.readString() }, - _r.readMap({ _r.readString() }, { _r.readString() }), - _r.readOptional { _r.readI64() }, + unpackListOfString(_r), + unpackMapOfStringToString(_r), + unpackOptionalOfI64(_r), unpackToken(_r), Priority.fromValue(_r.readI32()), - _r.readOptional { Gadget.fromHandle(_r.readObject()) }, + unpackOptionalOfGadget(_r), ) /** Consumer-implemented observer of ready items */ @@ -101,6 +101,19 @@ interface ReadyListener { /** Receives the item itself and says whether to keep listening */ fun onItem(item: Item, gadget: Gadget): Boolean + + /** + * A display label, failing with a kitchen error + * + * Throw a [KitchenErrorsException] to report a typed error to the library. + */ + fun label(): String + + /** The most recent item the listener kept, if any */ + fun latest(): Item? + + /** The listener's favorite gadget */ + fun favorite(): Gadget } /** A pocket-sized interface exercising the object surface */ @@ -144,7 +157,7 @@ object Kitchen { fun i32Id(x: Int): Int = JniBridge.kitchen_i32_id(x) - fun u32Id(x: Long): Long = JniBridge.kitchen_u32_id(x) + fun u32Id(x: UInt): UInt = JniBridge.kitchen_u32_id(x.toInt()).toUInt() fun i64Id(x: Long): Long = JniBridge.kitchen_i64_id(x) @@ -157,13 +170,16 @@ object Kitchen { /** Register a listener, returning how many are attached */ fun subscribe(listener: ReadyListener): Int = JniBridge.kitchen_subscribe(listener) + /** Register a listener if one is given */ + fun maybeSubscribe(listener: ReadyListener?): Boolean = JniBridge.kitchen_maybe_subscribe(listener) + /** Optional object in both directions */ fun maybeGadget(fallback: Gadget?): Gadget? = fallback?.handle.borrowOrNull { _h0 -> Gadget.fromHandleOrNull(JniBridge.kitchen_maybe_gadget(_h0)) } /** A list of objects */ - fun gadgets(): List = decodeBuffer(JniBridge.kitchen_gadgets()) { _r -> _r.readList { Gadget.fromHandle(_r.readObject()) } } + fun gadgets(): List = decodeBuffer(JniBridge.kitchen_gadgets()) { _r -> unpackListOfGadget(_r) } /** An iterator over objects */ fun streamGadgets(): NativeIterator = NativeIterator(JniBridge.kitchen_stream_gadgets(), JniBridge::kitchen_StreamGadgetsIterator_destroy) { _it -> @@ -172,13 +188,13 @@ object Kitchen { } /** Optional struct return */ - fun maybeItem(id: Long): Item? = decodeBuffer(JniBridge.kitchen_maybe_item(id)) { _r -> _r.readOptional { unpackItem(_r) } } + fun maybeItem(id: Long): Item? = decodeBuffer(JniBridge.kitchen_maybe_item(id)) { _r -> unpackOptionalOfItem(_r) } /** List of structs */ - fun listItems(): List = decodeBuffer(JniBridge.kitchen_list_items()) { _r -> _r.readList { unpackItem(_r) } } + fun listItems(): List = decodeBuffer(JniBridge.kitchen_list_items()) { _r -> unpackListOfItem(_r) } /** Map return type */ - fun getAttrs(): Map = decodeBuffer(JniBridge.kitchen_get_attrs()) { _r -> _r.readMap({ _r.readString() }, { _r.readI32() }) } + fun getAttrs(): Map = decodeBuffer(JniBridge.kitchen_get_attrs()) { _r -> unpackMapOfStringToI32(_r) } /** Iterator return type */ fun streamItems(): NativeIterator = NativeIterator(JniBridge.kitchen_stream_items(), JniBridge::kitchen_StreamItemsIterator_destroy) { _it -> @@ -190,12 +206,12 @@ object Kitchen { fun crossModuleToken(): Token = decodeBuffer(JniBridge.kitchen_cross_module_token()) { _r -> unpackToken(_r) } /** Async operation */ - suspend fun doAsync(input: String): String = awaitNative(false, 0, { decodeUtf8(it as ByteArray) }) { _, _done -> + suspend fun doAsync(input: String): String = awaitNative(false, 0, { _raw -> decodeUtf8(_raw as ByteArray) }) { _, _done -> JniBridge.kitchen_do_async(encodeUtf8(input), _done) } /** Cancellable async operation */ - suspend fun doCancellable(input: String): String = awaitNative(true, 0, { decodeUtf8(it as ByteArray) }) { _token, _done -> + suspend fun doCancellable(input: String): String = awaitNative(true, 0, { _raw -> decodeUtf8(_raw as ByteArray) }) { _token, _done -> JniBridge.kitchen_do_cancellable(encodeUtf8(input), _token, _done) } diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Runtime_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Runtime_kt.snap index 37419793..61ac3018 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Runtime_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Runtime_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink @@ -14,11 +14,28 @@ import java.util.concurrent.ConcurrentHashMap import java.util.concurrent.atomic.AtomicLong /** - * A failure reported by the native library: a producer panic (code -2), a - * marshalling failure (-3), a callback implementation that threw (-4), or any - * code these bindings don't know. Typed error domains subclass it. + * A failure a throwing call reports. Each error domain subclasses it with + * one class per declared (positive) code; a runtime failure keeps this class + * with its negative [code]: -1 a generic producer error, -2 a producer + * panic, -3 a marshalling failure, or -4 a callback implementation that + * failed. */ -open class FfiException(val code: Int, message: String) : Exception(message) +open class FfiException(val code: Int, message: String) : Exception(message) { + /** + * The fields of a typed error as the value buffer a callback reports + * with its code, or `null` when the code declares none. + */ + internal open fun encodePayload(): ByteArray? = null +} + +/** + * A call that can't fail failed anyway: a bug in the native library rather + * than an error to handle. [code] is the runtime code (-2 a producer panic, + * -3 a marshalling failure, -4 a callback failure the call couldn't report, + * and so on); the message names it and carries the producer's message. + */ +class NativeBugException(val code: Int, message: String) : + IllegalStateException("native call failed with code $code: $message") /** * A lazy sequence streamed from the native library, one element per step. diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Shared_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Shared_kt.snap index 252e2fbf..9a474ddf 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Shared_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_kitchen_sink__src_main_kotlin_kitchen_sink_Shared_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package kitchen_sink diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__build_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__build_gradle_kts.snap deleted file mode 100644 index fd4f8cd8..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__build_gradle_kts.snap +++ /dev/null @@ -1,59 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -plugins { - id("com.android.library") version "8.7.3" - id("org.jetbrains.kotlin.android") version "2.0.21" - `maven-publish` -} - -group = "nested_modules" -version = "0.1.0" - -android { - namespace = "nested_modules" - compileSdk = 35 - defaultConfig { - minSdk = 21 - consumerProguardFiles("consumer-rules.pro") - } - // Builds libnested_modules_jni.so per ABI, linked against the producer library in - // src/main/jniLibs// (see src/main/cpp/CMakeLists.txt). - externalNativeBuild { - cmake { - path = file("src/main/cpp/CMakeLists.txt") - } - } - compileOptions { - sourceCompatibility = JavaVersion.VERSION_1_8 - targetCompatibility = JavaVersion.VERSION_1_8 - } - publishing { - singleVariant("release") { - withSourcesJar() - } - } -} - -kotlin { - compilerOptions { - jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_1_8) - } -} - -publishing { - publications { - register("release") { - artifactId = "nested_modules" - afterEvaluate { - from(components["release"]) - } - } - } -} - -// END build.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__consumer_rules_pro.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__consumer_rules_pro.snap deleted file mode 100644 index ec1ab254..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__consumer_rules_pro.snap +++ /dev/null @@ -1,11 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# The JNI shim looks classes and methods up by name. --keep class nested_modules.** { *; } - -# END consumer-rules.pro diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__settings_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__settings_gradle_kts.snap deleted file mode 100644 index a5caa7f2..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__settings_gradle_kts.snap +++ /dev/null @@ -1,25 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -pluginManagement { - repositories { - google() - mavenCentral() - gradlePluginPortal() - } -} - -dependencyResolutionManagement { - repositories { - google() - mavenCentral() - } -} - -rootProject.name = "nested_modules" - -// END settings.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_CMakeLists_txt.snap deleted file mode 100644 index ad4cf329..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_CMakeLists_txt.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.18) -project(nested_modules_jni C) - -# The directory holding the producer library (libnested_modules). Android builds -# default to the per-ABI copy under src/main/jniLibs/. -set(NESTED_MODULES_LIBRARY_DIR "" CACHE PATH "Directory holding the nested_modules producer library") -if(NOT NESTED_MODULES_LIBRARY_DIR AND ANDROID) - set(NESTED_MODULES_LIBRARY_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../jniLibs/${ANDROID_ABI}") -endif() -if(NESTED_MODULES_LIBRARY_DIR) - find_library(NESTED_MODULES_PRODUCER NAMES nested_modules PATHS "${NESTED_MODULES_LIBRARY_DIR}" NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH REQUIRED) -else() - set(CMAKE_FIND_FRAMEWORK NEVER) - find_library(NESTED_MODULES_PRODUCER NAMES nested_modules REQUIRED) -endif() - -add_library(nested_modules_jni SHARED nested_modules_jni.c) -target_include_directories(nested_modules_jni PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}") -target_link_libraries(nested_modules_jni PRIVATE "${NESTED_MODULES_PRODUCER}") -if(NOT ANDROID) - find_package(JNI REQUIRED) - find_package(Threads REQUIRED) - target_include_directories(nested_modules_jni PRIVATE ${JNI_INCLUDE_DIRS}) - target_link_libraries(nested_modules_jni PRIVATE Threads::Threads) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_h.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_h.snap deleted file mode 100644 index 27c97f2b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_h.snap +++ /dev/null @@ -1,144 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef NESTED_MODULES_H -#define NESTED_MODULES_H - -#include -#include -#include - -#ifndef NESTED_MODULES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef NESTED_MODULES_BUILD -# define NESTED_MODULES_API __declspec(dllexport) -# else -# define NESTED_MODULES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define NESTED_MODULES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define NESTED_MODULES_API __attribute__((visibility("default"))) -# else -# define NESTED_MODULES_API -# endif -#endif - -#ifndef NESTED_MODULES_DEPRECATED -# if defined(_MSC_VER) -# define NESTED_MODULES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define NESTED_MODULES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define NESTED_MODULES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports nested_modules_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define NESTED_MODULES_ABI_VERSION 3u -NESTED_MODULES_API uint32_t nested_modules_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by nested_modules_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct nested_modules_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} nested_modules_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -NESTED_MODULES_API void nested_modules_error_set(nested_modules_error* err, int32_t code, const char* message); -NESTED_MODULES_API void nested_modules_error_clear(nested_modules_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - nested_modules_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -NESTED_MODULES_API void nested_modules_error_free(nested_modules_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -NESTED_MODULES_API void nested_modules_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct nested_modules_cancel_token nested_modules_cancel_token; -NESTED_MODULES_API nested_modules_cancel_token* nested_modules_cancel_token_create(void); -NESTED_MODULES_API void nested_modules_cancel_token_cancel(nested_modules_cancel_token* token); -NESTED_MODULES_API bool nested_modules_cancel_token_is_cancelled(const nested_modules_cancel_token* token); -NESTED_MODULES_API void nested_modules_cancel_token_destroy(nested_modules_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -NESTED_MODULES_API uint8_t* nested_modules_alloc(uint32_t size); -NESTED_MODULES_API void nested_modules_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -NESTED_MODULES_API uint64_t nested_modules_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define NESTED_MODULES_GRAPHICS_CHECKSUM 0x44d31b5b10fdb90dull -NESTED_MODULES_API uint64_t nested_modules_graphics_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with nested_modules_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * nested_modules_buffer.h declares a C struct and codec for each. - */ - -/** A measurement unit shared with nested submodules. */ -typedef enum { nested_modules_graphics_Unit_pixels = 0, nested_modules_graphics_Unit_points = 1 } nested_modules_graphics_Unit; -/** Error codes reported by throwing functions in the `graphics` module tree. */ -typedef enum { nested_modules_graphics_GraphicsError_OutOfBounds = 1 } nested_modules_graphics_GraphicsError; - -// Module: graphics -/** Construct a Canvas. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_new_canvas(int32_t width, size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes -/** Construct a Shape. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_make_shape(int32_t sides, size_t* out_len, nested_modules_error* out_err); -/** Nested function returning a struct owned by an ancestor module. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_describe(size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes_geometry -/** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_geometry_compute(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, nested_modules_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // NESTED_MODULES_H - -// END nested_modules.h diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_jni_c.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_jni_c.snap index ce7b8576..93dc9bd9 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_jni_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_cpp_nested_modules_jni_c.snap @@ -1,14 +1,15 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #include #include #include #include +#include #include #include @@ -72,8 +73,9 @@ static inline jbyteArray Jni_take_bytes(JNIEnv* env, const uint8_t* ptr, size_t return out; } -/* Throws the Kotlin exception for `err` (mapped through domain `domain`, - 0 for the generic one) and clears `err`. */ +/* Throws the Kotlin exception for `err` (mapped through error domain + `domain`, or 0 for a call that can't fail, which raises the trap) and + clears `err`. */ static inline void Jni_throw(JNIEnv* env, nested_modules_error* err, jint domain) { const char* message = err->message != NULL ? err->message : ""; jbyteArray text = Jni_new_bytes(env, (const uint8_t*)message, strlen(message)); @@ -100,6 +102,50 @@ static jint Jni_load_error(JNIEnv* env, const char* message) { return JNI_ERR; } +/* One contract entry these bindings were generated with: a declaration's + id and signature hash, plus its dotted path for the load error. */ +typedef struct { + uint64_t id; + uint64_t hash; + const char* path; +} Jni_contract_entry; + +/* Checks that the library's contract table (from `contract`, sorted by id) + has every expected entry with an equal hash; entries only the library + has are fine. Fails loading with an error naming the first declaration + that's missing or changed. */ +static inline jint Jni_check_contract(JNIEnv* env, const nested_modules_contract_entry* (*contract)(size_t*), const Jni_contract_entry* expected, size_t count) { + size_t len = 0; + const nested_modules_contract_entry* table = contract(&len); + if (table == NULL) { + len = 0; + } + for (size_t i = 0; i < count; i++) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) { + lo = mid + 1; + } else { + hi = mid; + } + } + const char* problem = NULL; + if (lo == len || table[lo].id != expected[i].id) { + problem = "is missing from the library"; + } else if (table[lo].hash != expected[i].hash) { + problem = "changed since these bindings were generated"; + } + if (problem != NULL) { + char message[512]; + snprintf(message, sizeof message, "%s: %s %s", "nested_modules", expected[i].path, problem); + return Jni_load_error(env, message); + } + } + return JNI_OK; +} + /* Generated below: the contract checks and the method IDs the shim caches. */ static jint Jni_load(JNIEnv* env); @@ -110,15 +156,15 @@ JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { if ((*vm)->GetEnv(vm, (void**)&env, JNI_VERSION_1_6) != JNI_OK) { return JNI_ERR; } - if (nested_modules_abi_version() != 3u) { - return Jni_load_error(env, "nested_modules: the native library implements a different C ABI revision than these bindings (expected 3)"); + if (nested_modules_abi_version() != 4u) { + return Jni_load_error(env, "nested_modules: the native library implements a different C ABI revision than these bindings (expected 4)"); } jclass bridge = (*env)->FindClass(env, "nested_modules/JniBridge"); if (bridge == NULL) { return JNI_ERR; } Jni_bridge = (jclass)(*env)->NewGlobalRef(env, bridge); - Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Lnested_modules/FfiException;"); + Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ljava/lang/Throwable;"); if (Jni_error == NULL) { return JNI_ERR; } @@ -134,10 +180,22 @@ JNIEXPORT jlong JNICALL Java_nested_1modules_JniBridge_debug_1live(JNIEnv* env, return (jlong)nested_modules_debug_live((int32_t)kind); } +static const Jni_contract_entry Jni_contract_graphics[] = { + {UINT64_C(0x2c29eef9327a6bd4), UINT64_C(0x12983d5271651389), "graphics.Unit"}, + {UINT64_C(0x304088bbc28f0015), UINT64_C(0x97e6e809913be6b6), "graphics.new_canvas"}, + {UINT64_C(0x49f8837532df9ac6), UINT64_C(0x32e5b27e81ee0957), "graphics.Canvas"}, + {UINT64_C(0x55717fa027336b47), UINT64_C(0xb2bd3a6bdcce48c8), "graphics.shapes.describe"}, + {UINT64_C(0x7626b91b8f1f53cd), UINT64_C(0x867bbf7cf0f41c74), "graphics.shapes.geometry.compute"}, + {UINT64_C(0xb3efcb50d9a61ab7), UINT64_C(0x3cb0207dfad2a149), "graphics.GraphicsError"}, + {UINT64_C(0xc62b16a9591b744b), UINT64_C(0x7895610c8487eb1b), "graphics.shapes.Shape"}, + {UINT64_C(0xe13001f4983ccd35), UINT64_C(0x16e7de070b4a4cfc), "graphics.shapes.geometry.Detail"}, + {UINT64_C(0xf3f658bcfc5dba62), UINT64_C(0x2665ccb47356e746), "graphics.shapes.make_shape"}, +}; + static jint Jni_load(JNIEnv* env) { (void)env; - if (nested_modules_graphics_checksum() != UINT64_C(0x44d31b5b10fdb90d)) { - return Jni_load_error(env, "nested_modules: module 'graphics' does not match these bindings (its contract checksum differs); regenerate the bindings or rebuild the library"); + if (Jni_check_contract(env, nested_modules_graphics_contract, Jni_contract_graphics, sizeof Jni_contract_graphics / sizeof Jni_contract_graphics[0]) != JNI_OK) { + return JNI_ERR; } return JNI_OK; } diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Buffers_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Buffers_kt.snap deleted file mode 100644 index 3a1a17e4..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Buffers_kt.snap +++ /dev/null @@ -1,255 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package nested_modules - -/** - * Growable little-endian writer for the value-buffer wire format that - * records, rich enums, optionals, lists, and maps cross the ABI in. - */ -internal class BufferWriter { - private var buf = ByteArray(64) - private var len = 0 - private var minted: ArrayList Unit>>? = null - - private fun reserve(extra: Int) { - if (len + extra <= buf.size) return - var cap = buf.size * 2 - while (cap < len + extra) cap *= 2 - buf = buf.copyOf(cap) - } - - fun toByteArray(): ByteArray = buf.copyOf(len) - - fun writeBool(v: Boolean) { - reserve(1) - buf[len++] = if (v) 1 else 0 - } - - fun writeI8(v: Byte) { - reserve(1) - buf[len++] = v - } - - fun writeU8(v: Byte) = writeI8(v) - - fun writeI16(v: Short) { - reserve(2) - val b = v.toInt() - buf[len++] = b.toByte() - buf[len++] = (b shr 8).toByte() - } - - fun writeU16(v: Short) = writeI16(v) - - fun writeI32(v: Int) { - reserve(4) - buf[len++] = v.toByte() - buf[len++] = (v shr 8).toByte() - buf[len++] = (v shr 16).toByte() - buf[len++] = (v shr 24).toByte() - } - - fun writeU32(v: Long) = writeI32(v.toInt()) - - fun writeI64(v: Long) { - writeI32(v.toInt()) - writeI32((v ushr 32).toInt()) - } - - fun writeU64(v: Long) = writeI64(v) - - fun writeF32(v: Float) = writeI32(v.toRawBits()) - - fun writeF64(v: Double) = writeI64(v.toRawBits()) - - fun writeString(v: String) = writeBytes(encodeUtf8(v)) - - fun writeBytes(v: ByteArray) { - writeI32(v.size) - reserve(v.size) - v.copyInto(buf, len) - len += v.size - } - - /** - * An object token: one new strong reference the reader adopts, which - * [abandon] releases with [release] if the encoding fails. - */ - fun writeObject(token: Long, release: (Long) -> Unit) { - val list = minted ?: ArrayList Unit>>().also { minted = it } - list.add(token to release) - writeI64(token) - } - - /** Releases every object token written so far, for an encoding that failed midway. */ - fun abandon() { - minted?.forEach { (token, release) -> release(token) } - minted = null - } - - inline fun writeOptional(v: T?, write: (T) -> Unit) { - if (v == null) { - writeBool(false) - } else { - writeBool(true) - write(v) - } - } - - inline fun writeList(v: List, write: (T) -> Unit) { - writeI32(v.size) - for (e in v) write(e) - } - - inline fun writeMap(v: Map, writeKey: (K) -> Unit, writeValue: (V) -> Unit) { - writeI32(v.size) - for ((k, e) in v) { - writeKey(k) - writeValue(e) - } - } -} - -/** - * Validating little-endian reader for the value-buffer wire format. A - * truncated buffer, an invalid flag byte, a zero object token, an oversized - * length, or trailing bytes raise [FfiException] code -3. - */ -internal class BufferReader(private val buf: ByteArray) { - private var pos = 0 - - private fun malformed(detail: String): Nothing = - throw FfiException(-3, "malformed value buffer: $detail") - - private fun take(n: Int): Int { - if (n > buf.size - pos) malformed("truncated buffer") - val at = pos - pos += n - return at - } - - fun readBool(): Boolean = when (buf[take(1)].toInt()) { - 0 -> false - 1 -> true - else -> malformed("invalid bool byte") - } - - fun readI8(): Byte = buf[take(1)] - - fun readU8(): Byte = readI8() - - fun readI16(): Short { - val at = take(2) - return ((buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)).toShort() - } - - fun readU16(): Short = readI16() - - fun readI32(): Int { - val at = take(4) - return (buf[at].toInt() and 0xFF) or - ((buf[at + 1].toInt() and 0xFF) shl 8) or - ((buf[at + 2].toInt() and 0xFF) shl 16) or - ((buf[at + 3].toInt() and 0xFF) shl 24) - } - - fun readU32(): Long = readI32().toLong() and 0xFFFFFFFFL - - fun readI64(): Long { - val lo = readI32().toLong() and 0xFFFFFFFFL - val hi = readI32().toLong() - return lo or (hi shl 32) - } - - fun readU64(): Long = readI64() - - fun readF32(): Float = Float.fromBits(readI32()) - - fun readF64(): Double = Double.fromBits(readI64()) - - private fun readLen(): Int { - val n = readI32() - if (n < 0 || n > buf.size - pos) malformed("length prefix exceeds remaining bytes") - return n - } - - fun readString(): String { - val n = readLen() - val at = take(n) - return try { - Charsets.UTF_8.newDecoder().decode(java.nio.ByteBuffer.wrap(buf, at, n)).toString() - } catch (e: java.nio.charset.CharacterCodingException) { - malformed("string is not valid UTF-8") - } - } - - fun readBytes(): ByteArray { - val n = readLen() - val at = take(n) - return buf.copyOfRange(at, at + n) - } - - /** An object token: one strong reference the caller adopts. */ - fun readObject(): Long { - val token = readI64() - if (token == 0L) malformed("null object token") - return token - } - - inline fun readOptional(read: () -> T): T? = if (readBool()) read() else null - - inline fun readList(read: () -> T): List { - val n = readCount() - val out = ArrayList(minOf(n, 4096)) - repeat(n) { out.add(read()) } - return out - } - - inline fun readMap(readKey: () -> K, readValue: () -> V): Map { - val n = readCount() - val out = LinkedHashMap(minOf(n, 4096)) - repeat(n) { - val k = readKey() - out[k] = readValue() - } - return out - } - - /** An element count; elements may encode to zero bytes, so only the sign is checked. */ - fun readCount(): Int { - val n = readI32() - if (n < 0) malformed("negative element count") - return n - } - - fun expectEnd() { - if (pos != buf.size) malformed("trailing bytes after value") - } -} - -/** Encodes one value into a fresh value buffer. */ -internal inline fun encodeBuffer(write: (BufferWriter) -> Unit): ByteArray { - val w = BufferWriter() - try { - write(w) - } catch (t: Throwable) { - w.abandon() - throw t - } - return w.toByteArray() -} - -/** Decodes one value from [bytes], rejecting malformed or trailing bytes. */ -internal inline fun decodeBuffer(bytes: ByteArray, read: (BufferReader) -> T): T { - val r = BufferReader(bytes) - val v = read(r) - r.expectEnd() - return v -} - -// END Buffers.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Graphics_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Graphics_kt.snap index fdd70d6d..61593497 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Graphics_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Graphics_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package nested_modules @@ -28,7 +28,7 @@ enum class Unit_(val value: Int) { companion object { /** The variant whose ABI value is [value]. */ fun fromValue(value: Int): Unit_ = entries.firstOrNull { it.value == value } - ?: throw FfiException(-3, "unknown Unit_ value $value") + ?: throw NativeBugException(-3, "unknown Unit_ value $value") } } diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_JniBridge_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_JniBridge_kt.snap index ab685c90..28f93ae8 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_JniBridge_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_JniBridge_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package nested_modules @@ -17,14 +17,16 @@ internal object JniBridge { NativeLibrary.ensureLoaded() } - /** The exception for a native failure; domain 0 is the generic one. */ + /** + * The exception for a native failure: through error domain [domain] for a + * throwing call, or a [NativeBugException] (domain 0) for one that can't fail. + */ @JvmStatic - @Suppress("UNUSED_PARAMETER") - fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): FfiException { + fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): Throwable { val text = decodeUtf8(message) return when (domain) { 1 -> GraphicsException.fromCode(code, text, payload) - else -> FfiException(code, text) + else -> NativeBugException(code, text) } } diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Runtime_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Runtime_kt.snap deleted file mode 100644 index aa07de7d..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_nested_modules__src_main_kotlin_nested_modules_Runtime_kt.snap +++ /dev/null @@ -1,212 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package nested_modules - -import java.lang.ref.PhantomReference -import java.lang.ref.ReferenceQueue -import java.util.Collections -import java.util.concurrent.ConcurrentHashMap -import java.util.concurrent.atomic.AtomicLong - -/** - * A failure reported by the native library: a producer panic (code -2), a - * marshalling failure (-3), a callback implementation that threw (-4), or any - * code these bindings don't know. Typed error domains subclass it. - */ -open class FfiException(val code: Int, message: String) : Exception(message) - -/** - * A lazy sequence streamed from the native library, one element per step. - * The native iterator is released when the sequence is exhausted, when - * [close] is called, or once this object becomes unreachable, whichever - * comes first. - */ -class NativeIterator internal constructor( - address: Long, - release: (Long) -> Unit, - private val step: (Long) -> Any?, -) : Iterator, AutoCloseable { - private val handle = NativeCleaner.register(this, NativeHandle(address, release)) - private var lookahead: Any? = EMPTY - - override fun hasNext(): Boolean { - if (lookahead !== EMPTY) return true - if (handle.isClosed) return false - val item = handle.borrow(step) - if (item === DONE) { - close() - return false - } - lookahead = item - return true - } - - override fun next(): T { - if (!hasNext()) throw NoSuchElementException() - val item = lookahead - lookahead = EMPTY - @Suppress("UNCHECKED_CAST") - return item as T - } - - /** Releases the native iterator; safe to call more than once. */ - override fun close() = handle.close() - - internal companion object { - /** What a step function returns once the native sequence is exhausted. */ - val DONE = Any() - private val EMPTY = Any() - } -} - -/** Loads the producer library and then the JNI shim, exactly once. */ -internal object NativeLibrary { - private const val LIBRARY = "nested_modules" - private const val JNI_LIBRARY = "nested_modules_jni" - - init { - val explicit = System.getenv("NESTED_MODULES_LIBRARY") - if (!explicit.isNullOrEmpty()) { - System.load(explicit) - } else if (!loadBundled(LIBRARY)) { - // The shim's own link dependency usually finds the producer when - // it isn't on java.library.path, so a miss here is only fatal if - // loading the shim fails too. - try { - System.loadLibrary(LIBRARY) - } catch (ignored: UnsatisfiedLinkError) { - } - } - if (!loadBundled(JNI_LIBRARY)) System.loadLibrary(JNI_LIBRARY) - } - - fun ensureLoaded() {} - - /** - * Loads `base` from the `natives/-/` classpath resources a - * packaged desktop build bundles; false when it isn't bundled. - */ - private fun loadBundled(base: String): Boolean { - val os = System.getProperty("os.name").lowercase() - val arch = System.getProperty("os.arch").lowercase() - val osId = when { - os.contains("mac") || os.contains("darwin") -> "darwin" - os.contains("win") -> "windows" - else -> "linux" - } - val archId = if (arch == "aarch64" || arch == "arm64") "arm64" else "x64" - val file = when (osId) { - "darwin" -> "lib$base.dylib" - "windows" -> "$base.dll" - else -> "lib$base.so" - } - val stream = NativeLibrary::class.java.getResourceAsStream("/natives/$osId-$archId/$file") ?: return false - val dir = java.nio.file.Files.createTempDirectory("$base-natives") - val target = dir.resolve(file) - stream.use { java.nio.file.Files.copy(it, target) } - target.toFile().deleteOnExit() - dir.toFile().deleteOnExit() - System.load(target.toAbsolutePath().toString()) - return true - } -} - -/** - * One strong reference to a native object, released exactly once: by - * [close], or by [NativeCleaner] once the owning wrapper is unreachable. - * Every call borrows the address for its duration, and a close that races an - * in-flight call defers the release until the last borrow ends, so the object - * is never freed while native code is using it. - */ -internal class NativeHandle(val address: Long, private val release: (Long) -> Unit) { - // Bit 0 is the closed flag; the remaining bits count in-flight borrows. - private val state = AtomicLong(0L) - - val isClosed: Boolean get() = state.get() and 1L != 0L - - fun acquire(): Long { - while (true) { - val s = state.get() - check(s and 1L == 0L) { "native object used after close()" } - if (state.compareAndSet(s, s + 2L)) return address - } - } - - fun unacquire() { - if (state.addAndGet(-2L) == 1L) release(address) - } - - fun close() { - while (true) { - val s = state.get() - if (s and 1L != 0L) return - if (state.compareAndSet(s, s or 1L)) { - if (s == 0L) release(address) - return - } - } - } -} - -/** Runs [block] with the native address borrowed for its duration. */ -internal inline fun NativeHandle.borrow(block: (Long) -> R): R { - val address = acquire() - try { - return block(address) - } finally { - unacquire() - } -} - -/** [borrow] for an optional object: a missing object lends address 0. */ -internal inline fun NativeHandle?.borrowOrNull(block: (Long) -> R): R = - if (this == null) block(0L) else borrow(block) - -/** - * Releases the native reference of every wrapper that becomes unreachable - * without being closed. A daemon thread drains a phantom-reference queue, so - * this works on every Android API level and every JVM. - */ -internal object NativeCleaner { - private val queue = ReferenceQueue() - private val pending: MutableSet = Collections.newSetFromMap(ConcurrentHashMap()) - - private class Entry(owner: Any, val handle: NativeHandle, queue: ReferenceQueue) : - PhantomReference(owner, queue) - - init { - val thread = Thread({ drain() }, "nested_modules-cleaner") - thread.isDaemon = true - thread.start() - } - - /** Releases [handle] once [owner] is unreachable; returns [handle]. */ - fun register(owner: Any, handle: NativeHandle): NativeHandle { - pending.add(Entry(owner, handle, queue)) - return handle - } - - private fun drain() { - while (true) { - val entry = queue.remove() as Entry - pending.remove(entry) - try { - entry.handle.close() - } catch (_: Throwable) { - } - } - } -} - -/** UTF-8 bytes of [value], for a string crossing into native code. */ -internal fun encodeUtf8(value: String): ByteArray = value.toByteArray(Charsets.UTF_8) - -/** The string a native UTF-8 byte sequence spells. */ -internal fun decodeUtf8(bytes: ByteArray): String = String(bytes, Charsets.UTF_8) - -// END Runtime.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__build_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__build_gradle_kts.snap deleted file mode 100644 index 339be7e1..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__build_gradle_kts.snap +++ /dev/null @@ -1,59 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -plugins { - id("com.android.library") version "8.7.3" - id("org.jetbrains.kotlin.android") version "2.0.21" - `maven-publish` -} - -group = "shapes" -version = "0.1.0" - -android { - namespace = "shapes" - compileSdk = 35 - defaultConfig { - minSdk = 21 - consumerProguardFiles("consumer-rules.pro") - } - // Builds libshapes_jni.so per ABI, linked against the producer library in - // src/main/jniLibs// (see src/main/cpp/CMakeLists.txt). - externalNativeBuild { - cmake { - path = file("src/main/cpp/CMakeLists.txt") - } - } - compileOptions { - sourceCompatibility = JavaVersion.VERSION_1_8 - targetCompatibility = JavaVersion.VERSION_1_8 - } - publishing { - singleVariant("release") { - withSourcesJar() - } - } -} - -kotlin { - compilerOptions { - jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_1_8) - } -} - -publishing { - publications { - register("release") { - artifactId = "shapes" - afterEvaluate { - from(components["release"]) - } - } - } -} - -// END build.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__consumer_rules_pro.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__consumer_rules_pro.snap deleted file mode 100644 index 049667cd..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__consumer_rules_pro.snap +++ /dev/null @@ -1,11 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# The JNI shim looks classes and methods up by name. --keep class shapes.** { *; } - -# END consumer-rules.pro diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__settings_gradle_kts.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__settings_gradle_kts.snap deleted file mode 100644 index 78bc6bdf..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__settings_gradle_kts.snap +++ /dev/null @@ -1,25 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -pluginManagement { - repositories { - google() - mavenCentral() - gradlePluginPortal() - } -} - -dependencyResolutionManagement { - repositories { - google() - mavenCentral() - } -} - -rootProject.name = "shapes" - -// END settings.gradle.kts diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_CMakeLists_txt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_CMakeLists_txt.snap deleted file mode 100644 index 8621c5ca..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_CMakeLists_txt.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -cmake_minimum_required(VERSION 3.18) -project(shapes_jni C) - -# The directory holding the producer library (libshapes). Android builds -# default to the per-ABI copy under src/main/jniLibs/. -set(SHAPES_LIBRARY_DIR "" CACHE PATH "Directory holding the shapes producer library") -if(NOT SHAPES_LIBRARY_DIR AND ANDROID) - set(SHAPES_LIBRARY_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../jniLibs/${ANDROID_ABI}") -endif() -if(SHAPES_LIBRARY_DIR) - find_library(SHAPES_PRODUCER NAMES shapes PATHS "${SHAPES_LIBRARY_DIR}" NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH REQUIRED) -else() - set(CMAKE_FIND_FRAMEWORK NEVER) - find_library(SHAPES_PRODUCER NAMES shapes REQUIRED) -endif() - -add_library(shapes_jni SHARED shapes_jni.c) -target_include_directories(shapes_jni PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}") -target_link_libraries(shapes_jni PRIVATE "${SHAPES_PRODUCER}") -if(NOT ANDROID) - find_package(JNI REQUIRED) - find_package(Threads REQUIRED) - target_include_directories(shapes_jni PRIVATE ${JNI_INCLUDE_DIRS}) - target_link_libraries(shapes_jni PRIVATE Threads::Threads) -endif() - -# END CMakeLists.txt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_h.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_h.snap deleted file mode 100644 index df20c3f5..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_h.snap +++ /dev/null @@ -1,151 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef SHAPES_H -#define SHAPES_H - -#include -#include -#include - -#ifndef SHAPES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef SHAPES_BUILD -# define SHAPES_API __declspec(dllexport) -# else -# define SHAPES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define SHAPES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define SHAPES_API __attribute__((visibility("default"))) -# else -# define SHAPES_API -# endif -#endif - -#ifndef SHAPES_DEPRECATED -# if defined(_MSC_VER) -# define SHAPES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define SHAPES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define SHAPES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports shapes_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define SHAPES_ABI_VERSION 3u -SHAPES_API uint32_t shapes_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by shapes_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct shapes_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} shapes_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -SHAPES_API void shapes_error_set(shapes_error* err, int32_t code, const char* message); -SHAPES_API void shapes_error_clear(shapes_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - shapes_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -SHAPES_API void shapes_error_free(shapes_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -SHAPES_API void shapes_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct shapes_cancel_token shapes_cancel_token; -SHAPES_API shapes_cancel_token* shapes_cancel_token_create(void); -SHAPES_API void shapes_cancel_token_cancel(shapes_cancel_token* token); -SHAPES_API bool shapes_cancel_token_is_cancelled(const shapes_cancel_token* token); -SHAPES_API void shapes_cancel_token_destroy(shapes_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -SHAPES_API uint8_t* shapes_alloc(uint32_t size); -SHAPES_API void shapes_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -SHAPES_API uint64_t shapes_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define SHAPES_SHAPES_CHECKSUM 0x06acf433ffa2a0f0ull -SHAPES_API uint64_t shapes_shapes_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with shapes_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * shapes_buffer.h declares a C struct and codec for each. - */ - -/** An algebraic shape (sum type with associated data) */ -typedef enum { - /** The empty shape (a unit variant) */ - shapes_shapes_Shape_Empty = 0, - /** A circle with a radius */ - shapes_shapes_Shape_Circle = 1, - /** An axis-aligned rectangle */ - shapes_shapes_Shape_Rectangle = 2, - /** An integer-bounded shape (exercises narrow + wide ints) */ - shapes_shapes_Shape_Bounded = 3, - /** A labeled shape with a small count */ - shapes_shapes_Shape_Labeled = 4 -} shapes_shapes_Shape_Tag; -/** A plain C-style enum (no payloads) */ -typedef enum { shapes_shapes_Channel_Red = 0, shapes_shapes_Channel_Green = 1, shapes_shapes_Channel_Blue = 2 } shapes_shapes_Channel; - -// Module: shapes -/** Render a shape to a string */ -SHAPES_API const uint8_t* shapes_shapes_describe(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, shapes_error* out_err); -/** Scale a shape by a factor, returning a new shape */ -SHAPES_API const uint8_t* shapes_shapes_scale(const uint8_t* shape_ptr, size_t shape_len, double factor, size_t* out_len, shapes_error* out_err); -/** Sum a list of bytes into a wide integer (numerics smoke) */ -SHAPES_API uint64_t shapes_shapes_sum_bytes(const uint8_t* values_ptr, size_t values_len, shapes_error* out_err); -/** Build a Numerics struct from every numeric primitive */ -SHAPES_API const uint8_t* shapes_shapes_make_numerics(int8_t a, int16_t b, int32_t c, int64_t d, uint8_t e, uint16_t f, uint32_t g, uint64_t h, float i, double j, size_t* out_len, shapes_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // SHAPES_H - -// END shapes.h diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_jni_c.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_jni_c.snap index 21717afb..2e5e44ac 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_jni_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_cpp_shapes_jni_c.snap @@ -1,14 +1,15 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #include #include #include #include +#include #include #include @@ -72,8 +73,9 @@ static inline jbyteArray Jni_take_bytes(JNIEnv* env, const uint8_t* ptr, size_t return out; } -/* Throws the Kotlin exception for `err` (mapped through domain `domain`, - 0 for the generic one) and clears `err`. */ +/* Throws the Kotlin exception for `err` (mapped through error domain + `domain`, or 0 for a call that can't fail, which raises the trap) and + clears `err`. */ static inline void Jni_throw(JNIEnv* env, shapes_error* err, jint domain) { const char* message = err->message != NULL ? err->message : ""; jbyteArray text = Jni_new_bytes(env, (const uint8_t*)message, strlen(message)); @@ -100,6 +102,50 @@ static jint Jni_load_error(JNIEnv* env, const char* message) { return JNI_ERR; } +/* One contract entry these bindings were generated with: a declaration's + id and signature hash, plus its dotted path for the load error. */ +typedef struct { + uint64_t id; + uint64_t hash; + const char* path; +} Jni_contract_entry; + +/* Checks that the library's contract table (from `contract`, sorted by id) + has every expected entry with an equal hash; entries only the library + has are fine. Fails loading with an error naming the first declaration + that's missing or changed. */ +static inline jint Jni_check_contract(JNIEnv* env, const shapes_contract_entry* (*contract)(size_t*), const Jni_contract_entry* expected, size_t count) { + size_t len = 0; + const shapes_contract_entry* table = contract(&len); + if (table == NULL) { + len = 0; + } + for (size_t i = 0; i < count; i++) { + size_t lo = 0; + size_t hi = len; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (table[mid].id < expected[i].id) { + lo = mid + 1; + } else { + hi = mid; + } + } + const char* problem = NULL; + if (lo == len || table[lo].id != expected[i].id) { + problem = "is missing from the library"; + } else if (table[lo].hash != expected[i].hash) { + problem = "changed since these bindings were generated"; + } + if (problem != NULL) { + char message[512]; + snprintf(message, sizeof message, "%s: %s %s", "shapes", expected[i].path, problem); + return Jni_load_error(env, message); + } + } + return JNI_OK; +} + /* Generated below: the contract checks and the method IDs the shim caches. */ static jint Jni_load(JNIEnv* env); @@ -110,15 +156,15 @@ JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { if ((*vm)->GetEnv(vm, (void**)&env, JNI_VERSION_1_6) != JNI_OK) { return JNI_ERR; } - if (shapes_abi_version() != 3u) { - return Jni_load_error(env, "shapes: the native library implements a different C ABI revision than these bindings (expected 3)"); + if (shapes_abi_version() != 4u) { + return Jni_load_error(env, "shapes: the native library implements a different C ABI revision than these bindings (expected 4)"); } jclass bridge = (*env)->FindClass(env, "shapes/JniBridge"); if (bridge == NULL) { return JNI_ERR; } Jni_bridge = (jclass)(*env)->NewGlobalRef(env, bridge); - Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Lshapes/FfiException;"); + Jni_error = (*env)->GetStaticMethodID(env, Jni_bridge, "error", "(II[B[B)Ljava/lang/Throwable;"); if (Jni_error == NULL) { return JNI_ERR; } @@ -134,10 +180,20 @@ JNIEXPORT jlong JNICALL Java_shapes_JniBridge_debug_1live(JNIEnv* env, jclass cl return (jlong)shapes_debug_live((int32_t)kind); } +static const Jni_contract_entry Jni_contract_shapes[] = { + {UINT64_C(0x0d7b3eec49243e7a), UINT64_C(0x5f232311808e1b1f), "shapes.Channel"}, + {UINT64_C(0x183fa8198100860e), UINT64_C(0xe40c6993cb31d359), "shapes.describe"}, + {UINT64_C(0x2d1395eb837b848e), UINT64_C(0x42202a0c12f3a16e), "shapes.sum_bytes"}, + {UINT64_C(0x7f9a5b87ba09b86b), UINT64_C(0xc5b32fb0efbbaa17), "shapes.scale"}, + {UINT64_C(0xc8edd779240fda17), UINT64_C(0x68e1be78fddb20e4), "shapes.Numerics"}, + {UINT64_C(0xd3d3881b2fd18a62), UINT64_C(0xf24d0eb76a99d918), "shapes.make_numerics"}, + {UINT64_C(0xf010844afff5ac3c), UINT64_C(0x50de28ae9b41ff15), "shapes.Shape"}, +}; + static jint Jni_load(JNIEnv* env) { (void)env; - if (shapes_shapes_checksum() != UINT64_C(0x06acf433ffa2a0f0)) { - return Jni_load_error(env, "shapes: module 'shapes' does not match these bindings (its contract checksum differs); regenerate the bindings or rebuild the library"); + if (Jni_check_contract(env, shapes_shapes_contract, Jni_contract_shapes, sizeof Jni_contract_shapes / sizeof Jni_contract_shapes[0]) != JNI_OK) { + return JNI_ERR; } return JNI_OK; } @@ -183,7 +239,7 @@ JNIEXPORT jlong JNICALL Java_shapes_JniBridge_shapes_1sum_1bytes(JNIEnv* env, jc return (jlong)rv; } -JNIEXPORT jbyteArray JNICALL Java_shapes_JniBridge_shapes_1make_1numerics(JNIEnv* env, jclass cls, jbyte p_a, jshort p_b, jint p_c, jlong p_d, jbyte p_e, jshort p_f, jlong p_g, jlong p_h, jfloat p_i, jdouble p_j) { +JNIEXPORT jbyteArray JNICALL Java_shapes_JniBridge_shapes_1make_1numerics(JNIEnv* env, jclass cls, jbyte p_a, jshort p_b, jint p_c, jlong p_d, jbyte p_e, jshort p_f, jint p_g, jlong p_h, jfloat p_i, jdouble p_j) { shapes_error err = {0, NULL, NULL, 0}; (void)cls; size_t out_len = 0; diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Buffers_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Buffers_kt.snap deleted file mode 100644 index 92661754..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Buffers_kt.snap +++ /dev/null @@ -1,255 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package shapes - -/** - * Growable little-endian writer for the value-buffer wire format that - * records, rich enums, optionals, lists, and maps cross the ABI in. - */ -internal class BufferWriter { - private var buf = ByteArray(64) - private var len = 0 - private var minted: ArrayList Unit>>? = null - - private fun reserve(extra: Int) { - if (len + extra <= buf.size) return - var cap = buf.size * 2 - while (cap < len + extra) cap *= 2 - buf = buf.copyOf(cap) - } - - fun toByteArray(): ByteArray = buf.copyOf(len) - - fun writeBool(v: Boolean) { - reserve(1) - buf[len++] = if (v) 1 else 0 - } - - fun writeI8(v: Byte) { - reserve(1) - buf[len++] = v - } - - fun writeU8(v: Byte) = writeI8(v) - - fun writeI16(v: Short) { - reserve(2) - val b = v.toInt() - buf[len++] = b.toByte() - buf[len++] = (b shr 8).toByte() - } - - fun writeU16(v: Short) = writeI16(v) - - fun writeI32(v: Int) { - reserve(4) - buf[len++] = v.toByte() - buf[len++] = (v shr 8).toByte() - buf[len++] = (v shr 16).toByte() - buf[len++] = (v shr 24).toByte() - } - - fun writeU32(v: Long) = writeI32(v.toInt()) - - fun writeI64(v: Long) { - writeI32(v.toInt()) - writeI32((v ushr 32).toInt()) - } - - fun writeU64(v: Long) = writeI64(v) - - fun writeF32(v: Float) = writeI32(v.toRawBits()) - - fun writeF64(v: Double) = writeI64(v.toRawBits()) - - fun writeString(v: String) = writeBytes(encodeUtf8(v)) - - fun writeBytes(v: ByteArray) { - writeI32(v.size) - reserve(v.size) - v.copyInto(buf, len) - len += v.size - } - - /** - * An object token: one new strong reference the reader adopts, which - * [abandon] releases with [release] if the encoding fails. - */ - fun writeObject(token: Long, release: (Long) -> Unit) { - val list = minted ?: ArrayList Unit>>().also { minted = it } - list.add(token to release) - writeI64(token) - } - - /** Releases every object token written so far, for an encoding that failed midway. */ - fun abandon() { - minted?.forEach { (token, release) -> release(token) } - minted = null - } - - inline fun writeOptional(v: T?, write: (T) -> Unit) { - if (v == null) { - writeBool(false) - } else { - writeBool(true) - write(v) - } - } - - inline fun writeList(v: List, write: (T) -> Unit) { - writeI32(v.size) - for (e in v) write(e) - } - - inline fun writeMap(v: Map, writeKey: (K) -> Unit, writeValue: (V) -> Unit) { - writeI32(v.size) - for ((k, e) in v) { - writeKey(k) - writeValue(e) - } - } -} - -/** - * Validating little-endian reader for the value-buffer wire format. A - * truncated buffer, an invalid flag byte, a zero object token, an oversized - * length, or trailing bytes raise [FfiException] code -3. - */ -internal class BufferReader(private val buf: ByteArray) { - private var pos = 0 - - private fun malformed(detail: String): Nothing = - throw FfiException(-3, "malformed value buffer: $detail") - - private fun take(n: Int): Int { - if (n > buf.size - pos) malformed("truncated buffer") - val at = pos - pos += n - return at - } - - fun readBool(): Boolean = when (buf[take(1)].toInt()) { - 0 -> false - 1 -> true - else -> malformed("invalid bool byte") - } - - fun readI8(): Byte = buf[take(1)] - - fun readU8(): Byte = readI8() - - fun readI16(): Short { - val at = take(2) - return ((buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)).toShort() - } - - fun readU16(): Short = readI16() - - fun readI32(): Int { - val at = take(4) - return (buf[at].toInt() and 0xFF) or - ((buf[at + 1].toInt() and 0xFF) shl 8) or - ((buf[at + 2].toInt() and 0xFF) shl 16) or - ((buf[at + 3].toInt() and 0xFF) shl 24) - } - - fun readU32(): Long = readI32().toLong() and 0xFFFFFFFFL - - fun readI64(): Long { - val lo = readI32().toLong() and 0xFFFFFFFFL - val hi = readI32().toLong() - return lo or (hi shl 32) - } - - fun readU64(): Long = readI64() - - fun readF32(): Float = Float.fromBits(readI32()) - - fun readF64(): Double = Double.fromBits(readI64()) - - private fun readLen(): Int { - val n = readI32() - if (n < 0 || n > buf.size - pos) malformed("length prefix exceeds remaining bytes") - return n - } - - fun readString(): String { - val n = readLen() - val at = take(n) - return try { - Charsets.UTF_8.newDecoder().decode(java.nio.ByteBuffer.wrap(buf, at, n)).toString() - } catch (e: java.nio.charset.CharacterCodingException) { - malformed("string is not valid UTF-8") - } - } - - fun readBytes(): ByteArray { - val n = readLen() - val at = take(n) - return buf.copyOfRange(at, at + n) - } - - /** An object token: one strong reference the caller adopts. */ - fun readObject(): Long { - val token = readI64() - if (token == 0L) malformed("null object token") - return token - } - - inline fun readOptional(read: () -> T): T? = if (readBool()) read() else null - - inline fun readList(read: () -> T): List { - val n = readCount() - val out = ArrayList(minOf(n, 4096)) - repeat(n) { out.add(read()) } - return out - } - - inline fun readMap(readKey: () -> K, readValue: () -> V): Map { - val n = readCount() - val out = LinkedHashMap(minOf(n, 4096)) - repeat(n) { - val k = readKey() - out[k] = readValue() - } - return out - } - - /** An element count; elements may encode to zero bytes, so only the sign is checked. */ - fun readCount(): Int { - val n = readI32() - if (n < 0) malformed("negative element count") - return n - } - - fun expectEnd() { - if (pos != buf.size) malformed("trailing bytes after value") - } -} - -/** Encodes one value into a fresh value buffer. */ -internal inline fun encodeBuffer(write: (BufferWriter) -> Unit): ByteArray { - val w = BufferWriter() - try { - write(w) - } catch (t: Throwable) { - w.abandon() - throw t - } - return w.toByteArray() -} - -/** Decodes one value from [bytes], rejecting malformed or trailing bytes. */ -internal inline fun decodeBuffer(bytes: ByteArray, read: (BufferReader) -> T): T { - val r = BufferReader(bytes) - val v = read(r) - r.expectEnd() - return v -} - -// END Buffers.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_JniBridge_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_JniBridge_kt.snap index 9ddcaa17..29806791 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_JniBridge_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_JniBridge_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package shapes @@ -17,13 +17,15 @@ internal object JniBridge { NativeLibrary.ensureLoaded() } - /** The exception for a native failure; domain 0 is the generic one. */ + /** + * The exception for a native failure: through error domain [domain] for a + * throwing call, or a [NativeBugException] (domain 0) for one that can't fail. + */ @JvmStatic - @Suppress("UNUSED_PARAMETER") - fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): FfiException { + fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): Throwable { val text = decodeUtf8(message) return when (domain) { - else -> FfiException(code, text) + else -> NativeBugException(code, text) } } @@ -31,7 +33,7 @@ internal object JniBridge { @JvmStatic external fun shapes_describe(shape: ByteArray): ByteArray @JvmStatic external fun shapes_scale(shape: ByteArray, factor: Double): ByteArray @JvmStatic external fun shapes_sum_bytes(values: ByteArray): Long - @JvmStatic external fun shapes_make_numerics(a: Byte, b: Short, c: Int, d: Long, e: Byte, f: Short, g: Long, h: Long, i: Float, j: Double): ByteArray + @JvmStatic external fun shapes_make_numerics(a: Byte, b: Short, c: Int, d: Long, e: Byte, f: Short, g: Int, h: Long, i: Float, j: Double): ByteArray } // END JniBridge.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Runtime_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Runtime_kt.snap deleted file mode 100644 index e2942e6d..00000000 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Runtime_kt.snap +++ /dev/null @@ -1,212 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -package shapes - -import java.lang.ref.PhantomReference -import java.lang.ref.ReferenceQueue -import java.util.Collections -import java.util.concurrent.ConcurrentHashMap -import java.util.concurrent.atomic.AtomicLong - -/** - * A failure reported by the native library: a producer panic (code -2), a - * marshalling failure (-3), a callback implementation that threw (-4), or any - * code these bindings don't know. Typed error domains subclass it. - */ -open class FfiException(val code: Int, message: String) : Exception(message) - -/** - * A lazy sequence streamed from the native library, one element per step. - * The native iterator is released when the sequence is exhausted, when - * [close] is called, or once this object becomes unreachable, whichever - * comes first. - */ -class NativeIterator internal constructor( - address: Long, - release: (Long) -> Unit, - private val step: (Long) -> Any?, -) : Iterator, AutoCloseable { - private val handle = NativeCleaner.register(this, NativeHandle(address, release)) - private var lookahead: Any? = EMPTY - - override fun hasNext(): Boolean { - if (lookahead !== EMPTY) return true - if (handle.isClosed) return false - val item = handle.borrow(step) - if (item === DONE) { - close() - return false - } - lookahead = item - return true - } - - override fun next(): T { - if (!hasNext()) throw NoSuchElementException() - val item = lookahead - lookahead = EMPTY - @Suppress("UNCHECKED_CAST") - return item as T - } - - /** Releases the native iterator; safe to call more than once. */ - override fun close() = handle.close() - - internal companion object { - /** What a step function returns once the native sequence is exhausted. */ - val DONE = Any() - private val EMPTY = Any() - } -} - -/** Loads the producer library and then the JNI shim, exactly once. */ -internal object NativeLibrary { - private const val LIBRARY = "shapes" - private const val JNI_LIBRARY = "shapes_jni" - - init { - val explicit = System.getenv("SHAPES_LIBRARY") - if (!explicit.isNullOrEmpty()) { - System.load(explicit) - } else if (!loadBundled(LIBRARY)) { - // The shim's own link dependency usually finds the producer when - // it isn't on java.library.path, so a miss here is only fatal if - // loading the shim fails too. - try { - System.loadLibrary(LIBRARY) - } catch (ignored: UnsatisfiedLinkError) { - } - } - if (!loadBundled(JNI_LIBRARY)) System.loadLibrary(JNI_LIBRARY) - } - - fun ensureLoaded() {} - - /** - * Loads `base` from the `natives/-/` classpath resources a - * packaged desktop build bundles; false when it isn't bundled. - */ - private fun loadBundled(base: String): Boolean { - val os = System.getProperty("os.name").lowercase() - val arch = System.getProperty("os.arch").lowercase() - val osId = when { - os.contains("mac") || os.contains("darwin") -> "darwin" - os.contains("win") -> "windows" - else -> "linux" - } - val archId = if (arch == "aarch64" || arch == "arm64") "arm64" else "x64" - val file = when (osId) { - "darwin" -> "lib$base.dylib" - "windows" -> "$base.dll" - else -> "lib$base.so" - } - val stream = NativeLibrary::class.java.getResourceAsStream("/natives/$osId-$archId/$file") ?: return false - val dir = java.nio.file.Files.createTempDirectory("$base-natives") - val target = dir.resolve(file) - stream.use { java.nio.file.Files.copy(it, target) } - target.toFile().deleteOnExit() - dir.toFile().deleteOnExit() - System.load(target.toAbsolutePath().toString()) - return true - } -} - -/** - * One strong reference to a native object, released exactly once: by - * [close], or by [NativeCleaner] once the owning wrapper is unreachable. - * Every call borrows the address for its duration, and a close that races an - * in-flight call defers the release until the last borrow ends, so the object - * is never freed while native code is using it. - */ -internal class NativeHandle(val address: Long, private val release: (Long) -> Unit) { - // Bit 0 is the closed flag; the remaining bits count in-flight borrows. - private val state = AtomicLong(0L) - - val isClosed: Boolean get() = state.get() and 1L != 0L - - fun acquire(): Long { - while (true) { - val s = state.get() - check(s and 1L == 0L) { "native object used after close()" } - if (state.compareAndSet(s, s + 2L)) return address - } - } - - fun unacquire() { - if (state.addAndGet(-2L) == 1L) release(address) - } - - fun close() { - while (true) { - val s = state.get() - if (s and 1L != 0L) return - if (state.compareAndSet(s, s or 1L)) { - if (s == 0L) release(address) - return - } - } - } -} - -/** Runs [block] with the native address borrowed for its duration. */ -internal inline fun NativeHandle.borrow(block: (Long) -> R): R { - val address = acquire() - try { - return block(address) - } finally { - unacquire() - } -} - -/** [borrow] for an optional object: a missing object lends address 0. */ -internal inline fun NativeHandle?.borrowOrNull(block: (Long) -> R): R = - if (this == null) block(0L) else borrow(block) - -/** - * Releases the native reference of every wrapper that becomes unreachable - * without being closed. A daemon thread drains a phantom-reference queue, so - * this works on every Android API level and every JVM. - */ -internal object NativeCleaner { - private val queue = ReferenceQueue() - private val pending: MutableSet = Collections.newSetFromMap(ConcurrentHashMap()) - - private class Entry(owner: Any, val handle: NativeHandle, queue: ReferenceQueue) : - PhantomReference(owner, queue) - - init { - val thread = Thread({ drain() }, "shapes-cleaner") - thread.isDaemon = true - thread.start() - } - - /** Releases [handle] once [owner] is unreachable; returns [handle]. */ - fun register(owner: Any, handle: NativeHandle): NativeHandle { - pending.add(Entry(owner, handle, queue)) - return handle - } - - private fun drain() { - while (true) { - val entry = queue.remove() as Entry - pending.remove(entry) - try { - entry.handle.close() - } catch (_: Throwable) { - } - } - } -} - -/** UTF-8 bytes of [value], for a string crossing into native code. */ -internal fun encodeUtf8(value: String): ByteArray = value.toByteArray(Charsets.UTF_8) - -/** The string a native UTF-8 byte sequence spells. */ -internal fun decodeUtf8(bytes: ByteArray): String = String(bytes, Charsets.UTF_8) - -// END Runtime.kt diff --git a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Shapes_kt.snap b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Shapes_kt.snap index 0e03c6b8..5ab27c24 100644 --- a/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Shapes_kt.snap +++ b/crates/weaveffi-cli/tests/snapshots/kotlin_shapes__src_main_kotlin_shapes_Shapes_kt.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate package shapes @@ -19,9 +19,9 @@ sealed class Shape { /** An axis-aligned rectangle */ data class Rectangle(val width: Float, val height: Float) : Shape() /** An integer-bounded shape (exercises narrow + wide ints) */ - data class Bounded(val min: Short, val max: Short, val weight: Long) : Shape() + data class Bounded(val min: Short, val max: Short, val weight: ULong) : Shape() /** A labeled shape with a small count */ - data class Labeled(val label: String, val count: Byte) : Shape() + data class Labeled(val label: String, val count: UByte) : Shape() } internal fun packShape(_w: BufferWriter, _v: Shape) { @@ -56,7 +56,7 @@ internal fun unpackShape(_r: BufferReader): Shape = when (val _tag = _r.readI32( 2 -> Shape.Rectangle(_r.readF32(), _r.readF32()) 3 -> Shape.Bounded(_r.readI16(), _r.readI16(), _r.readU64()) 4 -> Shape.Labeled(_r.readString(), _r.readU8()) - else -> throw FfiException(-3, "malformed value buffer: unknown Shape tag $_tag") + else -> throw NativeBugException(-3, "malformed value buffer: unknown Shape tag $_tag") } /** A plain C-style enum (no payloads) */ @@ -68,12 +68,12 @@ enum class Channel(val value: Int) { companion object { /** The variant whose ABI value is [value]. */ fun fromValue(value: Int): Channel = entries.firstOrNull { it.value == value } - ?: throw FfiException(-3, "unknown Channel value $value") + ?: throw NativeBugException(-3, "unknown Channel value $value") } } /** One field per supported numeric primitive */ -data class Numerics(val a: Byte, val b: Short, val c: Int, val d: Long, val e: Byte, val f: Short, val g: Long, val h: Long, val i: Float, val j: Double) +data class Numerics(val a: Byte, val b: Short, val c: Int, val d: Long, val e: UByte, val f: UShort, val g: UInt, val h: ULong, val i: Float, val j: Double) internal fun packNumerics(_w: BufferWriter, _v: Numerics) { _w.writeI8(_v.a) @@ -110,10 +110,10 @@ object Shapes { fun scale(shape: Shape, factor: Double): Shape = decodeBuffer(JniBridge.shapes_scale(encodeBuffer { _w -> packShape(_w, shape) }, factor)) { _r -> unpackShape(_r) } /** Sum a list of bytes into a wide integer (numerics smoke) */ - fun sumBytes(values: ByteArray): Long = JniBridge.shapes_sum_bytes(values) + fun sumBytes(values: ByteArray): ULong = JniBridge.shapes_sum_bytes(values).toULong() /** Build a Numerics struct from every numeric primitive */ - fun makeNumerics(a: Byte, b: Short, c: Int, d: Long, e: Byte, f: Short, g: Long, h: Long, i: Float, j: Double): Numerics = decodeBuffer(JniBridge.shapes_make_numerics(a, b, c, d, e, f, g, h, i, j)) { _r -> unpackNumerics(_r) } + fun makeNumerics(a: Byte, b: Short, c: Int, d: Long, e: UByte, f: UShort, g: UInt, h: ULong, i: Float, j: Double): Numerics = decodeBuffer(JniBridge.shapes_make_numerics(a, b, c, d, e.toByte(), f.toShort(), g.toInt(), h.toLong(), i, j)) { _r -> unpackNumerics(_r) } } // END Shapes.kt diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__README_md.snap deleted file mode 100644 index ae0e76ff..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__README_md.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# docs_everywhere - -Native bindings for docs_everywhere - -Node.js bindings for the `docs_everywhere` native library: an ES module over a small N-API addon (`docs_everywhere_node.c`) that `npm install` compiles with node-gyp. The addon links `docs_everywhere`; set `DOCS_EVERYWHERE_LIBRARY` to the library file (or its directory) when installing, or pass `--docs_everywhere-library=` to npm. - -```sh -DOCS_EVERYWHERE_LIBRARY=/path/to/libdocs_everywhere.so npm install docs_everywhere -``` - -Each top-level module of the API is a namespace export: - -```js -import { docs } from 'docs_everywhere'; -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__binding_gyp.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__binding_gyp.snap deleted file mode 100644 index 318fe42a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__binding_gyp.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -{ - "variables": { - "native_dir%": " { const path = require('path'); const p = process.env.DOCS_EVERYWHERE_LIBRARY || process.env.npm_config_docs_everywhere_library; if (!p) return path.resolve('.'); return require('fs').statSync(p).isFile() ? path.dirname(path.resolve(p)) : path.resolve(p); })()\")" - }, - "targets": [ - { - "target_name": "docs_everywhere_node", - "sources": [ - "docs_everywhere_node.c" - ], - "include_dirs": [ - "." - ], - "library_dirs": [ - "<(native_dir)" - ], - "libraries": [ - "-ldocs_everywhere" - ], - "conditions": [ - ["OS=='mac'", { "xcode_settings": { "OTHER_LDFLAGS": ["-Wl,-rpath,@loader_path", "-Wl,-rpath,<(native_dir)"] } }], - ["OS=='linux'", { "ldflags": ["-Wl,-rpath,'$$ORIGIN'", "-Wl,-rpath,<(native_dir)"] }] - ] - } - ] -} - -# END binding.gyp diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_h.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_h.snap deleted file mode 100644 index 31ab85e2..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_h.snap +++ /dev/null @@ -1,196 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef DOCS_EVERYWHERE_H -#define DOCS_EVERYWHERE_H - -#include -#include -#include - -#ifndef DOCS_EVERYWHERE_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef DOCS_EVERYWHERE_BUILD -# define DOCS_EVERYWHERE_API __declspec(dllexport) -# else -# define DOCS_EVERYWHERE_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define DOCS_EVERYWHERE_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define DOCS_EVERYWHERE_API __attribute__((visibility("default"))) -# else -# define DOCS_EVERYWHERE_API -# endif -#endif - -#ifndef DOCS_EVERYWHERE_DEPRECATED -# if defined(_MSC_VER) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define DOCS_EVERYWHERE_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports docs_everywhere_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define DOCS_EVERYWHERE_ABI_VERSION 3u -DOCS_EVERYWHERE_API uint32_t docs_everywhere_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by docs_everywhere_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct docs_everywhere_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} docs_everywhere_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_set(docs_everywhere_error* err, int32_t code, const char* message); -DOCS_EVERYWHERE_API void docs_everywhere_error_clear(docs_everywhere_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - docs_everywhere_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_free(docs_everywhere_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -DOCS_EVERYWHERE_API void docs_everywhere_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct docs_everywhere_cancel_token docs_everywhere_cancel_token; -DOCS_EVERYWHERE_API docs_everywhere_cancel_token* docs_everywhere_cancel_token_create(void); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_cancel(docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API bool docs_everywhere_cancel_token_is_cancelled(const docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_destroy(docs_everywhere_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -DOCS_EVERYWHERE_API uint8_t* docs_everywhere_alloc(uint32_t size); -DOCS_EVERYWHERE_API void docs_everywhere_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -DOCS_EVERYWHERE_API uint64_t docs_everywhere_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define DOCS_EVERYWHERE_DOCS_CHECKSUM 0x7950eec1e9cfc819ull -DOCS_EVERYWHERE_API uint64_t docs_everywhere_docs_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with docs_everywhere_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * docs_everywhere_buffer.h declares a C struct and codec for each. - */ - -/** - * Lifecycle status of a document. - * - * Used to drive UI badges and filters. - */ -typedef enum { - /** Document is being edited */ - docs_everywhere_docs_Status_Draft = 0, - /** Document is publicly visible */ - docs_everywhere_docs_Status_Published = 1, - /** Document is hidden from listings */ - docs_everywhere_docs_Status_Archived = 2 -} docs_everywhere_docs_Status; -/** Error codes reported by throwing functions in the `docs` module tree. */ -typedef enum { - /** Returned when no document matches the requested id */ - docs_everywhere_docs_DocsErrors_not_found = 1, - /** Raised when attempting to move into a forbidden state */ - docs_everywhere_docs_DocsErrors_invalid_status = 2 -} docs_everywhere_docs_DocsErrors; -typedef struct docs_everywhere_docs_Notebook docs_everywhere_docs_Notebook; -/** - * Consumer-implemented observer of document-saved notifications. - * - * Implementations must be safe to call from any thread. - * - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct docs_everywhere_docs_SavedListener_vtable { - /** Fires after a document has been persisted successfully */ - void (*on_saved)(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err); - void (*free)(void* ctx); -} docs_everywhere_docs_SavedListener_vtable; -typedef void (*docs_everywhere_docs_publish_async_callback)(void* context, docs_everywhere_error* err, bool result); - -// Module: docs -/** - * An ordered collection of documents. - * - * Notebooks own their documents and release them when destroyed. - */ -/** Create an empty notebook */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_new(docs_everywhere_error* out_err); -/** The largest number of documents one notebook will hold */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_max_documents(docs_everywhere_error* out_err); -/** Add a document, returning its position in the notebook */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_insert(const docs_everywhere_docs_Notebook* self, const uint8_t* document_ptr, size_t document_len, docs_everywhere_error* out_err); -/** Look up a document by identifier, failing when none matches */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_Notebook_find(const docs_everywhere_docs_Notebook* self, int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** The number of documents currently held */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_size(const docs_everywhere_docs_Notebook* self, docs_everywhere_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_clone(const docs_everywhere_docs_Notebook* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_Notebook_destroy(docs_everywhere_docs_Notebook* self); - -/** - * Create a brand new document. - * - * The document starts in `Draft` status; callers must publish - * it explicitly to make it visible. - */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_create_document(const uint8_t* title_ptr, size_t title_len, const uint8_t* tags_ptr, size_t tags_len, size_t* out_len, docs_everywhere_error* out_err); -/** Look up a document by identifier */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_get_document(int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** Publish a document in the background */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_publish_async(int64_t id, docs_everywhere_docs_publish_async_callback callback, void* context); -/** Register a listener for saved notifications */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_watch(void* listener_ctx, const docs_everywhere_docs_SavedListener_vtable* listener_vtable, docs_everywhere_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // DOCS_EVERYWHERE_H - -// END docs_everywhere.h diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap index ae260e29..e9463554 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef NAPI_VERSION #define NAPI_VERSION 8 @@ -12,6 +12,7 @@ source: crates/weaveffi-cli/tests/snapshots.rs #include #include #include +#include #include #include @@ -26,8 +27,9 @@ source: crates/weaveffi-cli/tests/snapshots.rs * raw convention the shared JavaScript layer documents: direct values as * numbers, bigints, and booleans; strings as strings; bytes and value * buffers as Uint8Arrays; objects, iterators, and cancel tokens as bigint - * handles (null when absent); callback interfaces as adapter objects. A - * failure throws an instance of the runtime's `$Fault` class. + * handles (null when absent); callback interfaces as adapter objects (null + * for an absent optional one). A failure throws an instance of the + * runtime's `$Fault` class. */ typedef docs_everywhere_error js_error; @@ -35,23 +37,106 @@ typedef docs_everywhere_error js_error; /* Helpers an API may not need. */ #if defined(__GNUC__) || defined(__clang__) #define JS_HELPER static inline __attribute__((unused)) +#define JS_TLS __thread #else #define JS_HELPER static inline +#define JS_TLS __declspec(thread) #endif +/* --------------------------------------------------------------------------- + * JavaScript threads. One `js_thread` per thread that runs JavaScript and + * loaded the addon (the main thread and each worker). It counts the + * synchronous calls into the library in progress on that thread, which a + * callback made on another thread consults while it waits for the + * JavaScript thread (see js_cb_hop). Each environment and each callback + * registration holds a reference. + * ------------------------------------------------------------------------- */ + +typedef struct { + uv_mutex_t mu; + uv_cond_t cv; /* broadcast when a call begins or ends, or a hop finishes */ + unsigned depth; /* synchronous calls into the library in progress */ + uint64_t calls; /* outermost synchronous calls begun so far */ + unsigned waiters; /* producer threads waiting in js_cb_hop */ + unsigned refs; +} js_thread; + +/* The state of the JavaScript thread this is, or NULL on any other thread. */ +static JS_TLS js_thread* js_current = NULL; + +static js_thread* js_thread_new(void) { + js_thread* t = (js_thread*)calloc(1, sizeof *t); + uv_mutex_init(&t->mu); + uv_cond_init(&t->cv); + t->refs = 1; + return t; +} + +static void js_thread_retain(js_thread* t) { + uv_mutex_lock(&t->mu); + t->refs++; + uv_mutex_unlock(&t->mu); +} + +static void js_thread_release(js_thread* t) { + uv_mutex_lock(&t->mu); + bool last = --t->refs == 0; + uv_mutex_unlock(&t->mu); + if (!last) return; + uv_cond_destroy(&t->cv); + uv_mutex_destroy(&t->mu); + free(t); +} + +/* Bracket every synchronous call into the library. */ +JS_HELPER void js_sync_begin(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + if (t->depth++ == 0) t->calls++; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +JS_HELPER void js_sync_end(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + t->depth--; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + /* Per-environment state (one per main thread or worker): the `$Fault` - * constructor `index.js` registers through `$setup`. */ + * constructor `index.js` registers through `$setup`, and the thread. */ typedef struct { napi_ref fault; + js_thread* thread; } js_env; static void js_env_free(napi_env env, void* data, void* hint) { (void)hint; js_env* e = (js_env*)data; if (e->fault != NULL) napi_delete_reference(env, e->fault); + if (js_current == e->thread) js_current = NULL; + js_thread_release(e->thread); free(e); } +/* Set up the environment of the thread loading the addon. */ +static void js_env_init(napi_env env) { + js_env* e = (js_env*)calloc(1, sizeof *e); + e->thread = js_thread_new(); + js_current = e->thread; + napi_set_instance_data(env, e, js_env_free, NULL); +} + +static js_env* js_env_of(napi_env env) { + js_env* e = NULL; + napi_get_instance_data(env, (void**)&e); + return e; +} + JS_HELPER napi_value js_undefined(napi_env env) { napi_value v; napi_get_undefined(env, &v); @@ -62,13 +147,22 @@ static napi_value js_setup(napi_env env, napi_callback_info info) { size_t argc = 1; napi_value argv[1]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); + js_env* e = js_env_of(env); if (e->fault != NULL) napi_delete_reference(env, e->fault); napi_create_reference(env, argv[0], 1, &e->fault); return js_undefined(env); } +/* The `$Fault` constructor, or NULL before `$setup`. */ +static napi_value js_fault_class(napi_env env) { + js_env* e = js_env_of(env); + napi_value ctor = NULL; + if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok) { + return NULL; + } + return ctor; +} + /* --------------------------------------------------------------------------- * Value conversions * ------------------------------------------------------------------------- */ @@ -149,6 +243,21 @@ JS_HELPER napi_value js_take_bytes(napi_env env, const uint8_t* ptr, size_t len) return out; } +/* A module's contract table as a BigUint64Array of `id, hash` pairs. */ +JS_HELPER napi_value js_new_contract(napi_env env, const docs_everywhere_contract_entry* table, + size_t len) { + napi_value ab, out; + void* data = NULL; + napi_create_arraybuffer(env, len * 2 * sizeof(uint64_t), &data, &ab); + uint64_t* words = (uint64_t*)data; + for (size_t i = 0; i < len; i++) { + words[2 * i] = table[i].id; + words[2 * i + 1] = table[i].hash; + } + napi_create_typedarray(env, napi_biguint64_array, len * 2, ab, 0, &out); + return out; +} + JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, const uint8_t* payload, size_t payload_len) { napi_value argv[3], ctor, out = NULL; @@ -159,10 +268,8 @@ JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, } else { napi_get_null(env, &argv[2]); } - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); - if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok || - ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { + ctor = js_fault_class(env); + if (ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { napi_create_error(env, NULL, argv[1], &out); napi_set_named_property(env, out, "code", argv[0]); } @@ -428,30 +535,42 @@ JS_HELPER void js_async_done(js_async* a, js_error* err) { * producer: a reference to the adapter object, the JavaScript thread it * belongs to, and a thread-safe function that runs calls made from any other * thread on that thread while the caller waits. + * + * Waiting is what makes a call from another thread safe, and also what could + * deadlock it: if the JavaScript thread is itself inside a synchronous call + * into the library that waits for the calling thread, neither can proceed. + * A waiting call therefore watches the JavaScript thread. While that thread + * stays inside one synchronous call for JS_CB_DEADLOCK_MS without having + * started the callback, the wait is abandoned and the callback fails with + * code -4 instead of hanging (a shorter synchronous call, or one that + * returns to the event loop, just delays the callback). * ------------------------------------------------------------------------- */ +#define JS_CB_DEADLOCK_MS 1000 + typedef struct { napi_env env; napi_ref ref; napi_threadsafe_function tsfn; uv_thread_t js_thread; + js_thread* thread; } js_cb; -/* Every callback frame starts with this header so the dispatcher can reach - * `out_err` without knowing the method. */ -typedef struct { - js_error* out_err; -} js_cb_frame; +typedef enum { JS_REQ_QUEUED, JS_REQ_RUNNING, JS_REQ_DONE, JS_REQ_ABANDONED } js_req_state; /* A call queued from another thread: `method` is the vtable index, or -1 to - * release the registration. */ + * release the registration; `frame` holds the method's slots. The request is + * heap-allocated and holds its own reference to the thread state, so a + * caller that abandons it can return (and the producer can then release the + * registration) while it is still queued: the dispatcher frees it without + * touching the frame or the registration. */ typedef struct { js_cb* cb; + js_thread* thread; int method; void* frame; - uv_mutex_t mu; - uv_cond_t cv; - bool done; + js_error* out_err; + js_req_state state; } js_cb_req; JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { @@ -460,10 +579,12 @@ JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { } JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* name, - napi_threadsafe_function_call_js dispatch) { + napi_threadsafe_function_call_js dispatch) { js_cb* cb = (js_cb*)calloc(1, sizeof *cb); cb->env = env; cb->js_thread = uv_thread_self(); + cb->thread = js_env_of(env)->thread; + js_thread_retain(cb->thread); napi_create_reference(env, adapter, 1, &cb->ref); napi_value resource; napi_create_string_utf8(env, name, NAPI_AUTO_LENGTH, &resource); @@ -477,14 +598,20 @@ JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* na JS_HELPER void js_cb_release(napi_env env, js_cb* cb) { if (env != NULL) napi_delete_reference(env, cb->ref); napi_release_threadsafe_function(cb->tsfn, napi_tsfn_release); + js_thread_release(cb->thread); free(cb); } -/* A callback-interface argument: the adapter object `index.js` built. */ +/* A callback-interface argument: the adapter object `index.js` built, or + * null for an absent optional one. */ JS_HELPER bool js_arg_cb(napi_env env, napi_value v, const char* name, - napi_threadsafe_function_call_js dispatch, js_cb** out) { + napi_threadsafe_function_call_js dispatch, bool nullable, js_cb** out) { napi_valuetype t; napi_typeof(env, v, &t); + if (nullable && (t == napi_null || t == napi_undefined)) { + *out = NULL; + return true; + } if (t != napi_object) return js_type_error(env, "expected a callback interface implementation"); *out = js_cb_register(env, v, name, dispatch); return true; @@ -505,62 +632,188 @@ JS_HELPER void js_cb_free(void* ctx) { if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) free(req); } -/* Run `req` on the JavaScript thread and wait for it. */ -JS_HELPER void js_cb_hop(js_cb_req* req) { - uv_mutex_init(&req->mu); - uv_cond_init(&req->cv); - req->done = false; - if (napi_call_threadsafe_function(req->cb->tsfn, req, napi_tsfn_blocking) == napi_ok) { - uv_mutex_lock(&req->mu); - while (!req->done) uv_cond_wait(&req->cv, &req->mu); - uv_mutex_unlock(&req->mu); - } else { - docs_everywhere_error_set(((js_cb_frame*)req->frame)->out_err, -4, - "the callback implementation is no longer reachable"); +/* Run method `method` of `cb` on its JavaScript thread and wait for it, + * failing it with code -4 rather than deadlocking (see above). `what` names + * the method in that failure. */ +JS_HELPER void js_cb_hop(js_cb* cb, int method, void* frame, js_error* out_err, + const char* what) { + js_thread* t = cb->thread; + js_cb_req* req = (js_cb_req*)calloc(1, sizeof *req); + req->cb = cb; + req->thread = t; + js_thread_retain(t); + req->method = method; + req->frame = frame; + req->out_err = out_err; + req->state = JS_REQ_QUEUED; + uv_mutex_lock(&t->mu); + t->waiters++; + uv_mutex_unlock(&t->mu); + if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) { + uv_mutex_lock(&t->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); + docs_everywhere_error_set(out_err, -4, "the callback implementation is no longer reachable"); + return; + } + const uint64_t limit = (uint64_t)JS_CB_DEADLOCK_MS * 1000000u; + bool blocked = false; + uint64_t call = 0, since = 0; + uv_mutex_lock(&t->mu); + while (req->state != JS_REQ_DONE) { + if (req->state == JS_REQ_QUEUED && t->depth > 0) { + uint64_t now = uv_hrtime(); + if (!blocked || call != t->calls) { + blocked = true; + call = t->calls; + since = now; + } + if (now - since >= limit) { + req->state = JS_REQ_ABANDONED; /* the dispatcher frees it */ + t->waiters--; + uv_mutex_unlock(&t->mu); + char msg[512]; + snprintf(msg, sizeof msg, + "%s was called on another thread while the JavaScript thread was blocked in a " + "synchronous call into the library for %d ms; the call likely waits for this " + "callback, so waiting longer would deadlock (call it asynchronously, or call " + "back on the calling thread)", + what, JS_CB_DEADLOCK_MS); + docs_everywhere_error_set(out_err, -4, msg); + return; + } + uv_cond_timedwait(&t->cv, &t->mu, limit - (now - since)); + } else { + blocked = false; + uv_cond_wait(&t->cv, &t->mu); + } + } + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); +} + +/* The dispatcher's side of a hop. js_cb_take claims a queued request (false + * when its caller gave up, in which case the request is freed); + * js_cb_finish reports it done. */ +JS_HELPER bool js_cb_take(js_cb_req* req) { + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + bool abandoned = req->state == JS_REQ_ABANDONED; + if (!abandoned) req->state = JS_REQ_RUNNING; + uv_mutex_unlock(&t->mu); + if (abandoned) { + free(req); + js_thread_release(t); } - uv_cond_destroy(&req->cv); - uv_mutex_destroy(&req->mu); + return !abandoned; } JS_HELPER void js_cb_finish(js_cb_req* req) { - uv_mutex_lock(&req->mu); - req->done = true; - uv_cond_signal(&req->cv); - uv_mutex_unlock(&req->mu); + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + req->state = JS_REQ_DONE; + uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); } -/* Report the pending JavaScript exception (or `fallback`) through `out_err` - * as a foreign failure (code -4); nothing unwinds through the C frame. */ +/* A JavaScript value's text as a heap-allocated C string (NULL on failure). */ +JS_HELPER char* js_cstring(napi_env env, napi_value v) { + napi_value str; + size_t len = 0; + if (napi_coerce_to_string(env, v, &str) != napi_ok || + napi_get_value_string_utf8(env, str, NULL, 0, &len) != napi_ok) { + return NULL; + } + char* out = (char*)malloc(len + 1); + napi_get_value_string_utf8(env, str, out, len + 1, NULL); + return out; +} + +/* Report the pending JavaScript exception (or `fallback`) through + * `out_err`; nothing unwinds through the C frame. A `$Fault` (a domain error + * a throwing method raised) reports its code, message, and payload; any + * other exception is a foreign failure, code -4, with its message. */ JS_HELPER void js_cb_report(napi_env env, js_error* out_err, const char* fallback) { - char msg[512]; - msg[0] = 0; + int32_t code = -4; + char* msg = NULL; + const uint8_t* payload = NULL; + size_t payload_len = 0; bool pending = false; napi_is_exception_pending(env, &pending); if (pending) { - napi_value exc, text, str; + napi_value exc, text; napi_get_and_clear_last_exception(env, &exc); + napi_value fault = js_fault_class(env); + bool is_fault = false; + if (fault != NULL) napi_instanceof(env, exc, fault, &is_fault); napi_valuetype t; napi_typeof(env, exc, &t); text = exc; if (t == napi_object) napi_get_named_property(env, exc, "message", &text); - if (napi_coerce_to_string(env, text, &str) == napi_ok) { - napi_get_value_string_utf8(env, str, msg, sizeof msg, NULL); + msg = js_cstring(env, text); + if (is_fault) { + napi_value v; + napi_get_named_property(env, exc, "code", &v); + napi_get_value_int32(env, v, &code); + napi_get_named_property(env, exc, "payload", &v); + bool typed = false; + napi_is_typedarray(env, v, &typed); + if (typed) { + napi_typedarray_type type; + void* data = NULL; + napi_get_typedarray_info(env, v, &type, &payload_len, &data, NULL, NULL); + payload = (const uint8_t*)data; + } } napi_is_exception_pending(env, &pending); if (pending) napi_get_and_clear_last_exception(env, &exc); } - docs_everywhere_error_set(out_err, -4, msg[0] != 0 ? msg : fallback); + docs_everywhere_error_set(out_err, code, msg != NULL && msg[0] != 0 ? msg : fallback); + if (payload != NULL) docs_everywhere_error_set_payload(out_err, payload, payload_len); + free(msg); } /* Look up and call `method` on the adapter; false when it threw. */ JS_HELPER bool js_cb_call(napi_env env, js_cb* cb, const char* method, size_t argc, - const napi_value* argv, napi_value* result) { + const napi_value* argv, napi_value* result) { napi_value adapter, fn; napi_get_reference_value(env, cb->ref, &adapter); napi_get_named_property(env, adapter, method, &fn); return napi_call_function(env, adapter, fn, argc, argv, result) == napi_ok; } +/* Hand `len` bytes to the producer through a callback's out slots, as a run + * from {p}_alloc that the producer adopts. */ +JS_HELPER void js_give(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = docs_everywhere_alloc(len); + if (len > 0) memcpy(run, data, len); + *out_ptr = run; + *out_len = len; +} + +/* A callback's string return: false (with a pending exception) when the + * value isn't a string. */ +JS_HELPER bool js_ret_str(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + js_str s = JS_STR_INIT; + if (!js_arg_str(env, v, &s)) return false; + js_give(s.ptr, s.len, out_ptr, out_len); + js_str_free(&s); + return true; +} + +/* A callback's bytes or buffer return: a Uint8Array. */ +JS_HELPER bool js_ret_bytes(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + const uint8_t* ptr = NULL; + size_t len = 0; + if (!js_arg_bytes(env, v, &ptr, &len)) return false; + js_give(ptr, len, out_ptr, out_len); + return true; +} + /* --------------------------------------------------------------------------- * The runtime symbols every library exports. * ------------------------------------------------------------------------- */ @@ -609,13 +862,15 @@ static napi_value js_cancel_token_destroy(napi_env env, napi_callback_info info) {"docs_everywhere_cancel_token_destroy", NULL, js_cancel_token_destroy, NULL, NULL, NULL, \ napi_default, NULL} -static napi_value nx_docs_everywhere_docs_checksum(napi_env env, napi_callback_info info) { +static napi_value nx_docs_everywhere_docs_contract(napi_env env, napi_callback_info info) { (void)info; - return js_new_u64(env, docs_everywhere_docs_checksum()); + size_t len = 0; + const docs_everywhere_contract_entry* table = docs_everywhere_docs_contract(&len); + return js_new_contract(env, table, len); } typedef struct { - js_cb_frame hdr; + js_error* out_err; int64_t p0; int32_t p1; } frame_docs_everywhere_docs_SavedListener_on_saved; @@ -628,7 +883,7 @@ static void invoke_docs_everywhere_docs_SavedListener_on_saved(napi_env env, js_ argv[1] = js_new_i32(env, (int32_t)f->p1); napi_value result; if (!js_cb_call(env, cb, "on_saved", 2, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "SavedListener.on_saved failed"); + js_cb_report(env, f->out_err, "SavedListener.onSaved failed"); } napi_close_handle_scope(env, scope); } @@ -636,19 +891,14 @@ static void invoke_docs_everywhere_docs_SavedListener_on_saved(napi_env env, js_ static void tramp_docs_everywhere_docs_SavedListener_on_saved(void* ctx, int64_t p0, int32_t p1, docs_everywhere_error* out_err) { frame_docs_everywhere_docs_SavedListener_on_saved f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; f.p0 = p0; f.p1 = p1; js_cb* cb = (js_cb*)ctx; if (js_cb_on_js_thread(cb)) { invoke_docs_everywhere_docs_SavedListener_on_saved(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 0; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 0, &f, out_err, "SavedListener.onSaved"); } } @@ -661,8 +911,9 @@ static void dispatch_docs_everywhere_docs_SavedListener(napi_env env, napi_value free(req); return; } + if (!js_cb_take(req)) return; if (env == NULL) { - docs_everywhere_error_set(((js_cb_frame*)req->frame)->out_err, -4, "the JavaScript environment is shutting down"); + docs_everywhere_error_set(req->out_err, -4, "the JavaScript environment is shutting down"); } else { switch (req->method) { case 0: invoke_docs_everywhere_docs_SavedListener_on_saved(env, req->cb, (frame_docs_everywhere_docs_SavedListener_on_saved*)req->frame); break; @@ -672,20 +923,25 @@ static void dispatch_docs_everywhere_docs_SavedListener(napi_env env, napi_value js_cb_finish(req); } -static const docs_everywhere_docs_SavedListener_vtable vtable_docs_everywhere_docs_SavedListener = {tramp_docs_everywhere_docs_SavedListener_on_saved, js_cb_free}; +static const docs_everywhere_docs_SavedListener_vtable vtable_docs_everywhere_docs_SavedListener = {sizeof(docs_everywhere_docs_SavedListener_vtable), 0, js_cb_free, tramp_docs_everywhere_docs_SavedListener_on_saved}; static napi_value nx_docs_everywhere_docs_Notebook_clone(napi_env env, napi_callback_info info) { JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; - return js_new_handle(env, docs_everywhere_docs_Notebook_clone((const docs_everywhere_docs_Notebook*)h)); + js_sync_begin(); + docs_everywhere_docs_Notebook* r = docs_everywhere_docs_Notebook_clone((const docs_everywhere_docs_Notebook*)h); + js_sync_end(); + return js_new_handle(env, r); } static napi_value nx_docs_everywhere_docs_Notebook_destroy(napi_env env, napi_callback_info info) { JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); docs_everywhere_docs_Notebook_destroy((docs_everywhere_docs_Notebook*)h); + js_sync_end(); return js_undefined(env); } @@ -700,7 +956,9 @@ static napi_value nx_docs_everywhere_docs_create_document(napi_env env, napi_cal { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = docs_everywhere_docs_create_document(JS_STR_PTR(a0), a0.len, a1, a1_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -721,7 +979,9 @@ static napi_value nx_docs_everywhere_docs_get_document(napi_env env, napi_callba { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = docs_everywhere_docs_get_document(a0, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -746,7 +1006,9 @@ static napi_value nx_docs_everywhere_docs_publish_async(napi_env env, napi_callb if (!js_arg_i64(env, argv[0], &a0)) goto done; { js_async* a = js_async_begin(env, JS_R_BOOL, "docs_everywhere_docs_publish_async", &ret); + js_sync_begin(); docs_everywhere_docs_publish_async(a0, done_docs_everywhere_docs_publish_async, a); + js_sync_end(); } goto done; done: @@ -757,10 +1019,12 @@ static napi_value nx_docs_everywhere_docs_watch(napi_env env, napi_callback_info JS_ARGS(1); napi_value ret = NULL; js_cb* a0 = NULL; - if (!js_arg_cb(env, argv[0], "docs_everywhere_docs_SavedListener", dispatch_docs_everywhere_docs_SavedListener, &a0)) goto done; + if (!js_arg_cb(env, argv[0], "docs_everywhere_docs_SavedListener", dispatch_docs_everywhere_docs_SavedListener, false, &a0)) goto done; { js_error err = {0}; + js_sync_begin(); docs_everywhere_docs_watch((void*)a0, &vtable_docs_everywhere_docs_SavedListener, &err); + js_sync_end(); a0 = NULL; if (err.code != 0) { js_throw(env, &err); @@ -779,7 +1043,9 @@ static napi_value nx_docs_everywhere_docs_Notebook_new(napi_env env, napi_callba napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); docs_everywhere_docs_Notebook* r = docs_everywhere_docs_Notebook_new(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -801,7 +1067,9 @@ static napi_value nx_docs_everywhere_docs_Notebook_insert(napi_env env, napi_cal if (!js_arg_bytes(env, argv[1], &a0, &a0_len)) goto done; { js_error err = {0}; + js_sync_begin(); int32_t r = docs_everywhere_docs_Notebook_insert((const docs_everywhere_docs_Notebook*)self_h, a0, a0_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -823,7 +1091,9 @@ static napi_value nx_docs_everywhere_docs_Notebook_find(napi_env env, napi_callb { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = docs_everywhere_docs_Notebook_find((const docs_everywhere_docs_Notebook*)self_h, a0, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -842,7 +1112,9 @@ static napi_value nx_docs_everywhere_docs_Notebook_size(napi_env env, napi_callb if (!js_arg_handle(env, argv[0], &self_h, false)) goto done; { js_error err = {0}; + js_sync_begin(); int32_t r = docs_everywhere_docs_Notebook_size((const docs_everywhere_docs_Notebook*)self_h, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -859,7 +1131,9 @@ static napi_value nx_docs_everywhere_docs_Notebook_max_documents(napi_env env, n napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); int32_t r = docs_everywhere_docs_Notebook_max_documents(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -872,11 +1146,10 @@ done: } NAPI_MODULE_INIT() { - js_env* e = (js_env*)calloc(1, sizeof *e); - napi_set_instance_data(env, e, js_env_free, NULL); + js_env_init(env); napi_property_descriptor props[] = { JS_RUNTIME_EXPORTS, - {"docs_everywhere_docs_checksum", NULL, nx_docs_everywhere_docs_checksum, NULL, NULL, NULL, napi_default, NULL}, + {"docs_everywhere_docs_contract", NULL, nx_docs_everywhere_docs_contract, NULL, NULL, NULL, napi_default, NULL}, {"docs_everywhere_docs_Notebook_clone", NULL, nx_docs_everywhere_docs_Notebook_clone, NULL, NULL, NULL, napi_default, NULL}, {"docs_everywhere_docs_Notebook_destroy", NULL, nx_docs_everywhere_docs_Notebook_destroy, NULL, NULL, NULL, napi_default, NULL}, {"docs_everywhere_docs_create_document", NULL, nx_docs_everywhere_docs_create_document, NULL, NULL, NULL, napi_default, NULL}, diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_d_ts.snap index 0be2d718..9a6a8519 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -24,18 +24,14 @@ export declare class CancelledError extends DocsEverywhereError { /** The root error class, under a name no namespace member can shadow. */ declare const $Error: typeof DocsEverywhereError; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** - * Create a brand new document. - * - * The document starts in `Draft` status; callers must publish - * it explicitly to make it visible. - */ +/** Document storage: create, publish, and observe documents. */ export declare namespace docs { /** The errors of the `docs` error domain. */ export class DocsErrorsError extends $Error { diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_js.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_js.snap index 755a173d..2fd23bcd 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -32,22 +35,30 @@ export { DocsEverywhereError, CancelledError } from './runtime.js'; import { createRequire } from 'node:module'; -// The native entry points: the N-API addon `npm install` builds from -// docs_everywhere_node.c (with node-gyp), or a prebuilt copy next to this file. +// The native entry points: the N-API addon built from docs_everywhere_node.c. A prebuilt +// copy comes first (from the `docs_everywhere--` package `npm install` +// picks for this platform, or under prebuilds/-/), then the one +// node-gyp compiles at install time when none matches. function $loadAddon(name) { const require = createRequire(import.meta.url); - const tried = []; - for (const dir of ['./build/Release/', './build/Debug/', './']) { + const platform = `${process.platform}-${process.arch}`; + const candidates = [ + `docs_everywhere-${platform}/${name}`, + `./prebuilds/${platform}/${name}`, + `./build/Release/${name}`, + `./build/Debug/${name}`, + ]; + for (const candidate of candidates) { try { - return require(dir + name); + return require(candidate); } catch (e) { if (e.code !== 'MODULE_NOT_FOUND') throw e; - tried.push(dir + name); } } throw new Error( - `the native addon ${name} was not found (tried ${tried.join(', ')}); ` + - 'build it with `npm install` or `node-gyp rebuild` in this package', + `the native addon ${name} was not found (tried ${candidates.join(', ')}); ` + + 'install the package for this platform, or build the addon with `npm install` ' + + 'or `node-gyp rebuild` in this package', ); } @@ -57,12 +68,34 @@ $raw.$setup($Fault); // Object tokens in value buffers are already bigint handles here. const $token = (t) => t; -$verify($raw, 'docs_everywhere', 'docs_everywhere', 3, [['docs', 0x7950eec1e9cfc819n]]); +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['docs', [ + [0x00b2187fd75634ecn, 0x2263c6f787bf6081n, 'docs.publish_async'], + [0x12fc242b9dc6444cn, 0x96405f484a642b31n, 'docs.Notebook.insert'], + [0x2213822ed05cd5bdn, 0x26c5833392aa63cen, 'docs.Notebook.new'], + [0x2bd762bcdaa207ffn, 0x1b0ebab2d04c49b0n, 'docs.SavedListener'], + [0x2e086c45e9d617den, 0x4d43bf32358c01ffn, 'docs.Notebook.find'], + [0x4817dc98197ad960n, 0xf8017b5a7d58b2c7n, 'docs.Notebook.size'], + [0x614de47452d5a55cn, 0xd9aae1472494ffefn, 'docs.get_document'], + [0x67c5cb9fc119bcd5n, 0x08d5b25a1a22a7fan, 'docs.Document'], + [0x8d484bf2d78c9b70n, 0xe9093cf96912194cn, 'docs.Notebook.max_documents'], + [0xaea61d6451691c67n, 0xe4704128b3524b3dn, 'docs.watch'], + [0xd68d6ad1540cdec5n, 0x01afeb40b0a8bfe1n, 'docs.Notebook'], + [0xdb0601444a4760ccn, 0xe56c69629b444797n, 'docs.create_document'], + [0xe12d6f4838347b3en, 0xa9bab86646d84f51n, 'docs.Status'], + [0xee2f8d25b3313b50n, 0x9691232f35439a2bn, 'docs.DocsErrors'], + ]], +]; +$verify($raw, 'docs_everywhere', 'docs_everywhere', 4, $contract); /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.docs_everywhere_debug_live(kind); @@ -81,9 +114,13 @@ const docs$InvalidStatusError = class InvalidStatusError extends docs$DocsErrors super(2, message); } }; -const $codes$docs$DocsErrors = new Map([[1, docs$NotFoundError], [2, docs$InvalidStatusError]]); +const $codes$docs$DocsErrors = new Map([ + [1, docs$NotFoundError], + [2, docs$InvalidStatusError], +]); +const $payloads$docs$DocsErrors = new Map(); function $from$docs$DocsErrors(e) { - return $domain(e, $codes$docs$DocsErrors); + return $domain(e, $codes$docs$DocsErrors, $payloads$docs$DocsErrors); } const docs$Status = Object.freeze({ @@ -99,16 +136,30 @@ function $w$docs$Document(w, v) { w.writeI64(v.id); w.writeString(v.title); w.writeI32(v.status); - w.writeList(v.tags, $W.String); + $w_list_string(w, v.tags); } function $r$docs$Document(r) { return { id: r.readI64(), title: r.readString(), status: r.readI32(), - tags: r.readList($R.String), + tags: $r_list_string(r), }; } +/** `[string]` */ +function $w_list_string(w, v) { + w.writeList(v, $W.String); +} +function $r_list_string(r) { + return r.readList($R.String); +} +/** `Document?` */ +function $w_opt_Document(w, v) { + w.writeOpt(v, $w$docs$Document); +} +function $r_opt_Document(r) { + return r.readOpt($r$docs$Document); +} function $adapt$docs$SavedListener(impl) { $impl(impl, 'SavedListener'); @@ -175,7 +226,7 @@ const docs$Notebook = class Notebook extends $Object { const docs$createDocument = function createDocument(title, tags) { try { - return $decode($raw.docs_everywhere_docs_create_document(title, $encode(tags, (w, v) => w.writeList(v, $W.String))), $r$docs$Document); + return $decode($raw.docs_everywhere_docs_create_document(title, $encode(tags, $w_list_string)), $r$docs$Document); } catch ($e) { throw $fault($e); } @@ -183,7 +234,7 @@ const docs$createDocument = function createDocument(title, tags) { const docs$getDocument = function getDocument(id) { try { - return $decode($raw.docs_everywhere_docs_get_document(id), (r) => r.readOpt($r$docs$Document)); + return $decode($raw.docs_everywhere_docs_get_document(id), $r_opt_Document); } catch ($e) { throw $fault($e); } diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__package_json.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__package_json.snap deleted file mode 100644 index e88b4546..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__package_json.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "docs_everywhere", - "version": "0.1.0", - "description": "Native bindings for docs_everywhere", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "binding.gyp", - "docs_everywhere_node.c", - "docs_everywhere.h" - ], - "gypfile": true, - "scripts": { - "install": "node-gyp rebuild" - }, - "engines": { - "node": ">=18" - } -} diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__runtime_js.snap deleted file mode 100644 index 75fa389b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class DocsEverywhereError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends DocsEverywhereError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new DocsEverywhereError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new DocsEverywhereError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new DocsEverywhereError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new DocsEverywhereError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__README_md.snap deleted file mode 100644 index 288dfe80..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__README_md.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# edge_cases - -Native bindings for edge_cases - -Node.js bindings for the `edge_cases` native library: an ES module over a small N-API addon (`edge_cases_node.c`) that `npm install` compiles with node-gyp. The addon links `edge_cases`; set `EDGE_CASES_LIBRARY` to the library file (or its directory) when installing, or pass `--edge_cases-library=` to npm. - -```sh -EDGE_CASES_LIBRARY=/path/to/libedge_cases.so npm install edge_cases -``` - -Each top-level module of the API is a namespace export: - -```js -import { edge_cases } from 'edge_cases'; -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__binding_gyp.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__binding_gyp.snap deleted file mode 100644 index ae606f1a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__binding_gyp.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -{ - "variables": { - "native_dir%": " { const path = require('path'); const p = process.env.EDGE_CASES_LIBRARY || process.env.npm_config_edge_cases_library; if (!p) return path.resolve('.'); return require('fs').statSync(p).isFile() ? path.dirname(path.resolve(p)) : path.resolve(p); })()\")" - }, - "targets": [ - { - "target_name": "edge_cases_node", - "sources": [ - "edge_cases_node.c" - ], - "include_dirs": [ - "." - ], - "library_dirs": [ - "<(native_dir)" - ], - "libraries": [ - "-ledge_cases" - ], - "conditions": [ - ["OS=='mac'", { "xcode_settings": { "OTHER_LDFLAGS": ["-Wl,-rpath,@loader_path", "-Wl,-rpath,<(native_dir)"] } }], - ["OS=='linux'", { "ldflags": ["-Wl,-rpath,'$$ORIGIN'", "-Wl,-rpath,<(native_dir)"] }] - ] - } - ] -} - -# END binding.gyp diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_h.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_h.snap deleted file mode 100644 index 94864086..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_h.snap +++ /dev/null @@ -1,245 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef EDGE_CASES_H -#define EDGE_CASES_H - -#include -#include -#include - -#ifndef EDGE_CASES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef EDGE_CASES_BUILD -# define EDGE_CASES_API __declspec(dllexport) -# else -# define EDGE_CASES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define EDGE_CASES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define EDGE_CASES_API __attribute__((visibility("default"))) -# else -# define EDGE_CASES_API -# endif -#endif - -#ifndef EDGE_CASES_DEPRECATED -# if defined(_MSC_VER) -# define EDGE_CASES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define EDGE_CASES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define EDGE_CASES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports edge_cases_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define EDGE_CASES_ABI_VERSION 3u -EDGE_CASES_API uint32_t edge_cases_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by edge_cases_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct edge_cases_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} edge_cases_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -EDGE_CASES_API void edge_cases_error_set(edge_cases_error* err, int32_t code, const char* message); -EDGE_CASES_API void edge_cases_error_clear(edge_cases_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - edge_cases_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -EDGE_CASES_API void edge_cases_error_free(edge_cases_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -EDGE_CASES_API void edge_cases_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct edge_cases_cancel_token edge_cases_cancel_token; -EDGE_CASES_API edge_cases_cancel_token* edge_cases_cancel_token_create(void); -EDGE_CASES_API void edge_cases_cancel_token_cancel(edge_cases_cancel_token* token); -EDGE_CASES_API bool edge_cases_cancel_token_is_cancelled(const edge_cases_cancel_token* token); -EDGE_CASES_API void edge_cases_cancel_token_destroy(edge_cases_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -EDGE_CASES_API uint8_t* edge_cases_alloc(uint32_t size); -EDGE_CASES_API void edge_cases_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -EDGE_CASES_API uint64_t edge_cases_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define EDGE_CASES_EDGE_CASES_CHECKSUM 0x1567f441073bd4aaull -EDGE_CASES_API uint64_t edge_cases_edge_cases_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with edge_cases_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * edge_cases_buffer.h declares a C struct and codec for each. - */ - -/** Deprecated enum with non-contiguous values and keyword-like variants */ -typedef enum { edge_cases_edge_cases_Legacy_None = 0, edge_cases_edge_cases_Legacy_Default = 5, edge_cases_edge_cases_Legacy_Class = 10 } edge_cases_edge_cases_Legacy; -typedef enum { edge_cases_edge_cases_Mode_Fast = 1, edge_cases_edge_cases_Mode_Safe = 2 } edge_cases_edge_cases_Mode; -/** Error codes reported by throwing functions in the `edge_cases` module tree. */ -typedef enum { edge_cases_edge_cases_EdgeErrors_Busy = 7 } edge_cases_edge_cases_EdgeErrors; -typedef struct edge_cases_edge_cases_Widget edge_cases_edge_cases_Widget; -typedef struct edge_cases_edge_cases_Vault edge_cases_edge_cases_Vault; -typedef struct edge_cases_edge_cases_WidgetStreamIterator edge_cases_edge_cases_WidgetStreamIterator; -typedef struct edge_cases_edge_cases_StringStreamIterator edge_cases_edge_cases_StringStreamIterator; -typedef struct edge_cases_edge_cases_ScalarStreamIterator edge_cases_edge_cases_ScalarStreamIterator; -typedef struct edge_cases_edge_cases_ModeStreamIterator edge_cases_edge_cases_ModeStreamIterator; -/** - * Callback interface carrying composite payloads and objects - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_BatchListener_vtable { - /** Composite payloads cross as value buffers */ - void (*on_batch)(void* ctx, const uint8_t* items_ptr, size_t items_len, const uint8_t* raw_ptr, size_t raw_len, const uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err); - /** Objects cross as strong references the consumer must release */ - void (*on_widget)(void* ctx, edge_cases_edge_cases_Widget* widget, edge_cases_edge_cases_Widget* fallback, const uint8_t* children_ptr, size_t children_len, edge_cases_error* out_err); - void (*on_tick)(void* ctx, edge_cases_error* out_err); - /** Scalar return travels back through the vtable */ - bool (*should_continue)(void* ctx, int32_t class_, edge_cases_error* out_err); - /** Enum return travels back as its integer discriminant */ - edge_cases_edge_cases_Mode (*pick_mode)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_BatchListener_vtable; -/** - * Callback interface whose name and methods are keywords - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_Keyword_vtable { - void (*delete_)(void* ctx, int32_t self, edge_cases_error* out_err); - int32_t (*import)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_Keyword_vtable; -typedef void (*edge_cases_edge_cases_async_widget_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result); -typedef void (*edge_cases_edge_cases_async_void_callback)(void* context, edge_cases_error* err); -typedef void (*edge_cases_edge_cases_async_bytes_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_struct_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_list_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_optional_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_enum_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Mode result); -typedef void (*edge_cases_edge_cases_async_keywords_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_Widget_fetch_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: edge_cases -/** Interface with no constructor; instances come from make_widget */ -/** Void method that can fail */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_refresh(const edge_cases_edge_cases_Widget* self, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_rename(const edge_cases_edge_cases_Widget* self, const uint8_t* class_ptr, size_t class_len, const uint8_t* import_ptr, size_t import_len, size_t* out_len, edge_cases_error* out_err); -/** Async method with a struct result */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_fetch(const edge_cases_edge_cases_Widget* self, int64_t id, edge_cases_edge_cases_Widget_fetch_callback callback, void* context); -EDGE_CASES_DEPRECATED("Use rename instead") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_old_name(const edge_cases_edge_cases_Widget* self, size_t* out_len, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Widget_clone(const edge_cases_edge_cases_Widget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_destroy(edge_cases_edge_cases_Widget* self); - -/** Deprecated interface with a throwing constructor */ -/** Fails when the vault is busy */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_open(const uint8_t* path_ptr, size_t path_len, edge_cases_error* out_err); -/** Throwing static */ -EDGE_CASES_API bool edge_cases_edge_cases_Vault_probe(edge_cases_error* out_err); -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Vault_peek(const edge_cases_edge_cases_Vault* self, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_clone(const edge_cases_edge_cases_Vault* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Vault_destroy(edge_cases_edge_cases_Vault* self); - -/** No parameters and no result */ -EDGE_CASES_API void edge_cases_edge_cases_noop(edge_cases_error* out_err); -/** Function named after a keyword whose parameters are all keywords */ -EDGE_CASES_API int32_t edge_cases_edge_cases_select(int32_t class_, const uint8_t* from_ptr, size_t from_len, const uint8_t* func_ptr, size_t func_len, const uint8_t* range_ptr, size_t range_len, bool type_, edge_cases_error* out_err); -/** Optional list of optional structs, the deepest legal nesting */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_nested_optional(const uint8_t* items_ptr, size_t items_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_optional_params(const uint8_t* text_ptr, size_t text_len, const uint8_t* count_ptr, size_t count_len, const uint8_t* outer_ptr, size_t outer_len, const uint8_t* tags_ptr, size_t tags_len, const uint8_t* blob_ptr, size_t blob_len, const uint8_t* mode_ptr, size_t mode_len, size_t* out_len, edge_cases_error* out_err); -/** Two callback interface parameters on one function */ -EDGE_CASES_API void edge_cases_edge_cases_register_batch(void* listener_ctx, const edge_cases_edge_cases_BatchListener_vtable* listener_vtable, void* keyword_ctx, const edge_cases_edge_cases_Keyword_vtable* keyword_vtable, edge_cases_error* out_err); -/** Objects in every buffered position */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_widgets_everywhere(const uint8_t* list_ptr, size_t list_len, const uint8_t* by_name_ptr, size_t by_name_len, const uint8_t* sparse_ptr, size_t sparse_len, const uint8_t* holder_ptr, size_t holder_len, size_t* out_len, edge_cases_error* out_err); -/** Iterator of objects */ -EDGE_CASES_API edge_cases_edge_cases_WidgetStreamIterator* edge_cases_edge_cases_widget_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_WidgetStreamIterator_next(edge_cases_edge_cases_WidgetStreamIterator* iter, edge_cases_edge_cases_Widget** out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_WidgetStreamIterator_destroy(edge_cases_edge_cases_WidgetStreamIterator* iter); -/** Async returning an object */ -EDGE_CASES_API void edge_cases_edge_cases_async_widget(edge_cases_edge_cases_async_widget_callback callback, void* context); -/** Factory for an interface with no constructor */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_make_widget(const uint8_t* label_ptr, size_t label_len, edge_cases_error* out_err); -/** Optional interface return */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_find_widget(int64_t id, edge_cases_error* out_err); -/** Interface parameter, including an optional one */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_inspect_widget(const edge_cases_edge_cases_Widget* widget, const edge_cases_edge_cases_Widget* fallback, size_t* out_len, edge_cases_error* out_err); -/** Iterator of strings */ -EDGE_CASES_API edge_cases_edge_cases_StringStreamIterator* edge_cases_edge_cases_string_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_StringStreamIterator_next(edge_cases_edge_cases_StringStreamIterator* iter, const uint8_t** out_item, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_StringStreamIterator_destroy(edge_cases_edge_cases_StringStreamIterator* iter); -/** Iterator of scalars */ -EDGE_CASES_API edge_cases_edge_cases_ScalarStreamIterator* edge_cases_edge_cases_scalar_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ScalarStreamIterator_next(edge_cases_edge_cases_ScalarStreamIterator* iter, int32_t* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ScalarStreamIterator_destroy(edge_cases_edge_cases_ScalarStreamIterator* iter); -/** Iterator of enums */ -EDGE_CASES_API edge_cases_edge_cases_ModeStreamIterator* edge_cases_edge_cases_mode_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ModeStreamIterator_next(edge_cases_edge_cases_ModeStreamIterator* iter, edge_cases_edge_cases_Mode* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ModeStreamIterator_destroy(edge_cases_edge_cases_ModeStreamIterator* iter); -/** Async with no result */ -EDGE_CASES_API void edge_cases_edge_cases_async_void(edge_cases_edge_cases_async_void_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_bytes(uint32_t size, edge_cases_edge_cases_async_bytes_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_struct(edge_cases_edge_cases_async_struct_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_list(edge_cases_cancel_token* cancel_token, edge_cases_edge_cases_async_list_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_optional(edge_cases_edge_cases_async_optional_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_enum(edge_cases_edge_cases_async_enum_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_keywords(const uint8_t* class_ptr, size_t class_len, int32_t import, edge_cases_edge_cases_async_keywords_callback callback, void* context); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_keywords_roundtrip(const uint8_t* value_ptr, size_t value_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_DEPRECATED("Use Mode-based APIs") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_legacy_value(edge_cases_edge_cases_Legacy value, size_t* out_len, edge_cases_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // EDGE_CASES_H - -// END edge_cases.h diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap index 62982286..b0346180 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef NAPI_VERSION #define NAPI_VERSION 8 @@ -12,6 +12,7 @@ source: crates/weaveffi-cli/tests/snapshots.rs #include #include #include +#include #include #include @@ -26,8 +27,9 @@ source: crates/weaveffi-cli/tests/snapshots.rs * raw convention the shared JavaScript layer documents: direct values as * numbers, bigints, and booleans; strings as strings; bytes and value * buffers as Uint8Arrays; objects, iterators, and cancel tokens as bigint - * handles (null when absent); callback interfaces as adapter objects. A - * failure throws an instance of the runtime's `$Fault` class. + * handles (null when absent); callback interfaces as adapter objects (null + * for an absent optional one). A failure throws an instance of the + * runtime's `$Fault` class. */ typedef edge_cases_error js_error; @@ -35,23 +37,106 @@ typedef edge_cases_error js_error; /* Helpers an API may not need. */ #if defined(__GNUC__) || defined(__clang__) #define JS_HELPER static inline __attribute__((unused)) +#define JS_TLS __thread #else #define JS_HELPER static inline +#define JS_TLS __declspec(thread) #endif +/* --------------------------------------------------------------------------- + * JavaScript threads. One `js_thread` per thread that runs JavaScript and + * loaded the addon (the main thread and each worker). It counts the + * synchronous calls into the library in progress on that thread, which a + * callback made on another thread consults while it waits for the + * JavaScript thread (see js_cb_hop). Each environment and each callback + * registration holds a reference. + * ------------------------------------------------------------------------- */ + +typedef struct { + uv_mutex_t mu; + uv_cond_t cv; /* broadcast when a call begins or ends, or a hop finishes */ + unsigned depth; /* synchronous calls into the library in progress */ + uint64_t calls; /* outermost synchronous calls begun so far */ + unsigned waiters; /* producer threads waiting in js_cb_hop */ + unsigned refs; +} js_thread; + +/* The state of the JavaScript thread this is, or NULL on any other thread. */ +static JS_TLS js_thread* js_current = NULL; + +static js_thread* js_thread_new(void) { + js_thread* t = (js_thread*)calloc(1, sizeof *t); + uv_mutex_init(&t->mu); + uv_cond_init(&t->cv); + t->refs = 1; + return t; +} + +static void js_thread_retain(js_thread* t) { + uv_mutex_lock(&t->mu); + t->refs++; + uv_mutex_unlock(&t->mu); +} + +static void js_thread_release(js_thread* t) { + uv_mutex_lock(&t->mu); + bool last = --t->refs == 0; + uv_mutex_unlock(&t->mu); + if (!last) return; + uv_cond_destroy(&t->cv); + uv_mutex_destroy(&t->mu); + free(t); +} + +/* Bracket every synchronous call into the library. */ +JS_HELPER void js_sync_begin(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + if (t->depth++ == 0) t->calls++; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +JS_HELPER void js_sync_end(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + t->depth--; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + /* Per-environment state (one per main thread or worker): the `$Fault` - * constructor `index.js` registers through `$setup`. */ + * constructor `index.js` registers through `$setup`, and the thread. */ typedef struct { napi_ref fault; + js_thread* thread; } js_env; static void js_env_free(napi_env env, void* data, void* hint) { (void)hint; js_env* e = (js_env*)data; if (e->fault != NULL) napi_delete_reference(env, e->fault); + if (js_current == e->thread) js_current = NULL; + js_thread_release(e->thread); free(e); } +/* Set up the environment of the thread loading the addon. */ +static void js_env_init(napi_env env) { + js_env* e = (js_env*)calloc(1, sizeof *e); + e->thread = js_thread_new(); + js_current = e->thread; + napi_set_instance_data(env, e, js_env_free, NULL); +} + +static js_env* js_env_of(napi_env env) { + js_env* e = NULL; + napi_get_instance_data(env, (void**)&e); + return e; +} + JS_HELPER napi_value js_undefined(napi_env env) { napi_value v; napi_get_undefined(env, &v); @@ -62,13 +147,22 @@ static napi_value js_setup(napi_env env, napi_callback_info info) { size_t argc = 1; napi_value argv[1]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); + js_env* e = js_env_of(env); if (e->fault != NULL) napi_delete_reference(env, e->fault); napi_create_reference(env, argv[0], 1, &e->fault); return js_undefined(env); } +/* The `$Fault` constructor, or NULL before `$setup`. */ +static napi_value js_fault_class(napi_env env) { + js_env* e = js_env_of(env); + napi_value ctor = NULL; + if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok) { + return NULL; + } + return ctor; +} + /* --------------------------------------------------------------------------- * Value conversions * ------------------------------------------------------------------------- */ @@ -149,6 +243,21 @@ JS_HELPER napi_value js_take_bytes(napi_env env, const uint8_t* ptr, size_t len) return out; } +/* A module's contract table as a BigUint64Array of `id, hash` pairs. */ +JS_HELPER napi_value js_new_contract(napi_env env, const edge_cases_contract_entry* table, + size_t len) { + napi_value ab, out; + void* data = NULL; + napi_create_arraybuffer(env, len * 2 * sizeof(uint64_t), &data, &ab); + uint64_t* words = (uint64_t*)data; + for (size_t i = 0; i < len; i++) { + words[2 * i] = table[i].id; + words[2 * i + 1] = table[i].hash; + } + napi_create_typedarray(env, napi_biguint64_array, len * 2, ab, 0, &out); + return out; +} + JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, const uint8_t* payload, size_t payload_len) { napi_value argv[3], ctor, out = NULL; @@ -159,10 +268,8 @@ JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, } else { napi_get_null(env, &argv[2]); } - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); - if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok || - ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { + ctor = js_fault_class(env); + if (ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { napi_create_error(env, NULL, argv[1], &out); napi_set_named_property(env, out, "code", argv[0]); } @@ -428,30 +535,42 @@ JS_HELPER void js_async_done(js_async* a, js_error* err) { * producer: a reference to the adapter object, the JavaScript thread it * belongs to, and a thread-safe function that runs calls made from any other * thread on that thread while the caller waits. + * + * Waiting is what makes a call from another thread safe, and also what could + * deadlock it: if the JavaScript thread is itself inside a synchronous call + * into the library that waits for the calling thread, neither can proceed. + * A waiting call therefore watches the JavaScript thread. While that thread + * stays inside one synchronous call for JS_CB_DEADLOCK_MS without having + * started the callback, the wait is abandoned and the callback fails with + * code -4 instead of hanging (a shorter synchronous call, or one that + * returns to the event loop, just delays the callback). * ------------------------------------------------------------------------- */ +#define JS_CB_DEADLOCK_MS 1000 + typedef struct { napi_env env; napi_ref ref; napi_threadsafe_function tsfn; uv_thread_t js_thread; + js_thread* thread; } js_cb; -/* Every callback frame starts with this header so the dispatcher can reach - * `out_err` without knowing the method. */ -typedef struct { - js_error* out_err; -} js_cb_frame; +typedef enum { JS_REQ_QUEUED, JS_REQ_RUNNING, JS_REQ_DONE, JS_REQ_ABANDONED } js_req_state; /* A call queued from another thread: `method` is the vtable index, or -1 to - * release the registration. */ + * release the registration; `frame` holds the method's slots. The request is + * heap-allocated and holds its own reference to the thread state, so a + * caller that abandons it can return (and the producer can then release the + * registration) while it is still queued: the dispatcher frees it without + * touching the frame or the registration. */ typedef struct { js_cb* cb; + js_thread* thread; int method; void* frame; - uv_mutex_t mu; - uv_cond_t cv; - bool done; + js_error* out_err; + js_req_state state; } js_cb_req; JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { @@ -460,10 +579,12 @@ JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { } JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* name, - napi_threadsafe_function_call_js dispatch) { + napi_threadsafe_function_call_js dispatch) { js_cb* cb = (js_cb*)calloc(1, sizeof *cb); cb->env = env; cb->js_thread = uv_thread_self(); + cb->thread = js_env_of(env)->thread; + js_thread_retain(cb->thread); napi_create_reference(env, adapter, 1, &cb->ref); napi_value resource; napi_create_string_utf8(env, name, NAPI_AUTO_LENGTH, &resource); @@ -477,14 +598,20 @@ JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* na JS_HELPER void js_cb_release(napi_env env, js_cb* cb) { if (env != NULL) napi_delete_reference(env, cb->ref); napi_release_threadsafe_function(cb->tsfn, napi_tsfn_release); + js_thread_release(cb->thread); free(cb); } -/* A callback-interface argument: the adapter object `index.js` built. */ +/* A callback-interface argument: the adapter object `index.js` built, or + * null for an absent optional one. */ JS_HELPER bool js_arg_cb(napi_env env, napi_value v, const char* name, - napi_threadsafe_function_call_js dispatch, js_cb** out) { + napi_threadsafe_function_call_js dispatch, bool nullable, js_cb** out) { napi_valuetype t; napi_typeof(env, v, &t); + if (nullable && (t == napi_null || t == napi_undefined)) { + *out = NULL; + return true; + } if (t != napi_object) return js_type_error(env, "expected a callback interface implementation"); *out = js_cb_register(env, v, name, dispatch); return true; @@ -505,62 +632,188 @@ JS_HELPER void js_cb_free(void* ctx) { if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) free(req); } -/* Run `req` on the JavaScript thread and wait for it. */ -JS_HELPER void js_cb_hop(js_cb_req* req) { - uv_mutex_init(&req->mu); - uv_cond_init(&req->cv); - req->done = false; - if (napi_call_threadsafe_function(req->cb->tsfn, req, napi_tsfn_blocking) == napi_ok) { - uv_mutex_lock(&req->mu); - while (!req->done) uv_cond_wait(&req->cv, &req->mu); - uv_mutex_unlock(&req->mu); - } else { - edge_cases_error_set(((js_cb_frame*)req->frame)->out_err, -4, - "the callback implementation is no longer reachable"); +/* Run method `method` of `cb` on its JavaScript thread and wait for it, + * failing it with code -4 rather than deadlocking (see above). `what` names + * the method in that failure. */ +JS_HELPER void js_cb_hop(js_cb* cb, int method, void* frame, js_error* out_err, + const char* what) { + js_thread* t = cb->thread; + js_cb_req* req = (js_cb_req*)calloc(1, sizeof *req); + req->cb = cb; + req->thread = t; + js_thread_retain(t); + req->method = method; + req->frame = frame; + req->out_err = out_err; + req->state = JS_REQ_QUEUED; + uv_mutex_lock(&t->mu); + t->waiters++; + uv_mutex_unlock(&t->mu); + if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) { + uv_mutex_lock(&t->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); + edge_cases_error_set(out_err, -4, "the callback implementation is no longer reachable"); + return; } - uv_cond_destroy(&req->cv); - uv_mutex_destroy(&req->mu); + const uint64_t limit = (uint64_t)JS_CB_DEADLOCK_MS * 1000000u; + bool blocked = false; + uint64_t call = 0, since = 0; + uv_mutex_lock(&t->mu); + while (req->state != JS_REQ_DONE) { + if (req->state == JS_REQ_QUEUED && t->depth > 0) { + uint64_t now = uv_hrtime(); + if (!blocked || call != t->calls) { + blocked = true; + call = t->calls; + since = now; + } + if (now - since >= limit) { + req->state = JS_REQ_ABANDONED; /* the dispatcher frees it */ + t->waiters--; + uv_mutex_unlock(&t->mu); + char msg[512]; + snprintf(msg, sizeof msg, + "%s was called on another thread while the JavaScript thread was blocked in a " + "synchronous call into the library for %d ms; the call likely waits for this " + "callback, so waiting longer would deadlock (call it asynchronously, or call " + "back on the calling thread)", + what, JS_CB_DEADLOCK_MS); + edge_cases_error_set(out_err, -4, msg); + return; + } + uv_cond_timedwait(&t->cv, &t->mu, limit - (now - since)); + } else { + blocked = false; + uv_cond_wait(&t->cv, &t->mu); + } + } + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); +} + +/* The dispatcher's side of a hop. js_cb_take claims a queued request (false + * when its caller gave up, in which case the request is freed); + * js_cb_finish reports it done. */ +JS_HELPER bool js_cb_take(js_cb_req* req) { + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + bool abandoned = req->state == JS_REQ_ABANDONED; + if (!abandoned) req->state = JS_REQ_RUNNING; + uv_mutex_unlock(&t->mu); + if (abandoned) { + free(req); + js_thread_release(t); + } + return !abandoned; } JS_HELPER void js_cb_finish(js_cb_req* req) { - uv_mutex_lock(&req->mu); - req->done = true; - uv_cond_signal(&req->cv); - uv_mutex_unlock(&req->mu); + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + req->state = JS_REQ_DONE; + uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +/* A JavaScript value's text as a heap-allocated C string (NULL on failure). */ +JS_HELPER char* js_cstring(napi_env env, napi_value v) { + napi_value str; + size_t len = 0; + if (napi_coerce_to_string(env, v, &str) != napi_ok || + napi_get_value_string_utf8(env, str, NULL, 0, &len) != napi_ok) { + return NULL; + } + char* out = (char*)malloc(len + 1); + napi_get_value_string_utf8(env, str, out, len + 1, NULL); + return out; } -/* Report the pending JavaScript exception (or `fallback`) through `out_err` - * as a foreign failure (code -4); nothing unwinds through the C frame. */ +/* Report the pending JavaScript exception (or `fallback`) through + * `out_err`; nothing unwinds through the C frame. A `$Fault` (a domain error + * a throwing method raised) reports its code, message, and payload; any + * other exception is a foreign failure, code -4, with its message. */ JS_HELPER void js_cb_report(napi_env env, js_error* out_err, const char* fallback) { - char msg[512]; - msg[0] = 0; + int32_t code = -4; + char* msg = NULL; + const uint8_t* payload = NULL; + size_t payload_len = 0; bool pending = false; napi_is_exception_pending(env, &pending); if (pending) { - napi_value exc, text, str; + napi_value exc, text; napi_get_and_clear_last_exception(env, &exc); + napi_value fault = js_fault_class(env); + bool is_fault = false; + if (fault != NULL) napi_instanceof(env, exc, fault, &is_fault); napi_valuetype t; napi_typeof(env, exc, &t); text = exc; if (t == napi_object) napi_get_named_property(env, exc, "message", &text); - if (napi_coerce_to_string(env, text, &str) == napi_ok) { - napi_get_value_string_utf8(env, str, msg, sizeof msg, NULL); + msg = js_cstring(env, text); + if (is_fault) { + napi_value v; + napi_get_named_property(env, exc, "code", &v); + napi_get_value_int32(env, v, &code); + napi_get_named_property(env, exc, "payload", &v); + bool typed = false; + napi_is_typedarray(env, v, &typed); + if (typed) { + napi_typedarray_type type; + void* data = NULL; + napi_get_typedarray_info(env, v, &type, &payload_len, &data, NULL, NULL); + payload = (const uint8_t*)data; + } } napi_is_exception_pending(env, &pending); if (pending) napi_get_and_clear_last_exception(env, &exc); } - edge_cases_error_set(out_err, -4, msg[0] != 0 ? msg : fallback); + edge_cases_error_set(out_err, code, msg != NULL && msg[0] != 0 ? msg : fallback); + if (payload != NULL) edge_cases_error_set_payload(out_err, payload, payload_len); + free(msg); } /* Look up and call `method` on the adapter; false when it threw. */ JS_HELPER bool js_cb_call(napi_env env, js_cb* cb, const char* method, size_t argc, - const napi_value* argv, napi_value* result) { + const napi_value* argv, napi_value* result) { napi_value adapter, fn; napi_get_reference_value(env, cb->ref, &adapter); napi_get_named_property(env, adapter, method, &fn); return napi_call_function(env, adapter, fn, argc, argv, result) == napi_ok; } +/* Hand `len` bytes to the producer through a callback's out slots, as a run + * from {p}_alloc that the producer adopts. */ +JS_HELPER void js_give(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = edge_cases_alloc(len); + if (len > 0) memcpy(run, data, len); + *out_ptr = run; + *out_len = len; +} + +/* A callback's string return: false (with a pending exception) when the + * value isn't a string. */ +JS_HELPER bool js_ret_str(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + js_str s = JS_STR_INIT; + if (!js_arg_str(env, v, &s)) return false; + js_give(s.ptr, s.len, out_ptr, out_len); + js_str_free(&s); + return true; +} + +/* A callback's bytes or buffer return: a Uint8Array. */ +JS_HELPER bool js_ret_bytes(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + const uint8_t* ptr = NULL; + size_t len = 0; + if (!js_arg_bytes(env, v, &ptr, &len)) return false; + js_give(ptr, len, out_ptr, out_len); + return true; +} + /* --------------------------------------------------------------------------- * The runtime symbols every library exports. * ------------------------------------------------------------------------- */ @@ -609,13 +862,15 @@ static napi_value js_cancel_token_destroy(napi_env env, napi_callback_info info) {"edge_cases_cancel_token_destroy", NULL, js_cancel_token_destroy, NULL, NULL, NULL, \ napi_default, NULL} -static napi_value nx_edge_cases_edge_cases_checksum(napi_env env, napi_callback_info info) { +static napi_value nx_edge_cases_edge_cases_contract(napi_env env, napi_callback_info info) { (void)info; - return js_new_u64(env, edge_cases_edge_cases_checksum()); + size_t len = 0; + const edge_cases_contract_entry* table = edge_cases_edge_cases_contract(&len); + return js_new_contract(env, table, len); } typedef struct { - js_cb_frame hdr; + js_error* out_err; const uint8_t* p0; size_t p1; const uint8_t* p2; @@ -635,7 +890,7 @@ static void invoke_edge_cases_edge_cases_BatchListener_on_batch(napi_env env, js argv[3] = js_new_i32(env, (int32_t)f->p6); napi_value result; if (!js_cb_call(env, cb, "on_batch", 4, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "BatchListener.on_batch failed"); + js_cb_report(env, f->out_err, "BatchListener.onBatch failed"); } napi_close_handle_scope(env, scope); } @@ -643,7 +898,7 @@ static void invoke_edge_cases_edge_cases_BatchListener_on_batch(napi_env env, js static void tramp_edge_cases_edge_cases_BatchListener_on_batch(void* ctx, const uint8_t* p0, size_t p1, const uint8_t* p2, size_t p3, const uint8_t* p4, size_t p5, edge_cases_edge_cases_Mode p6, edge_cases_error* out_err) { frame_edge_cases_edge_cases_BatchListener_on_batch f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; f.p0 = p0; f.p1 = p1; f.p2 = p2; @@ -655,17 +910,12 @@ static void tramp_edge_cases_edge_cases_BatchListener_on_batch(void* ctx, const if (js_cb_on_js_thread(cb)) { invoke_edge_cases_edge_cases_BatchListener_on_batch(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 0; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 0, &f, out_err, "BatchListener.onBatch"); } } typedef struct { - js_cb_frame hdr; + js_error* out_err; edge_cases_edge_cases_Widget* p0; edge_cases_edge_cases_Widget* p1; const uint8_t* p2; @@ -681,7 +931,7 @@ static void invoke_edge_cases_edge_cases_BatchListener_on_widget(napi_env env, j argv[2] = js_new_bytes(env, f->p2, f->p3); napi_value result; if (!js_cb_call(env, cb, "on_widget", 3, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "BatchListener.on_widget failed"); + js_cb_report(env, f->out_err, "BatchListener.onWidget failed"); } napi_close_handle_scope(env, scope); } @@ -689,7 +939,7 @@ static void invoke_edge_cases_edge_cases_BatchListener_on_widget(napi_env env, j static void tramp_edge_cases_edge_cases_BatchListener_on_widget(void* ctx, edge_cases_edge_cases_Widget* p0, edge_cases_edge_cases_Widget* p1, const uint8_t* p2, size_t p3, edge_cases_error* out_err) { frame_edge_cases_edge_cases_BatchListener_on_widget f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; f.p0 = p0; f.p1 = p1; f.p2 = p2; @@ -698,17 +948,12 @@ static void tramp_edge_cases_edge_cases_BatchListener_on_widget(void* ctx, edge_ if (js_cb_on_js_thread(cb)) { invoke_edge_cases_edge_cases_BatchListener_on_widget(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 1; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 1, &f, out_err, "BatchListener.onWidget"); } } typedef struct { - js_cb_frame hdr; + js_error* out_err; } frame_edge_cases_edge_cases_BatchListener_on_tick; static void invoke_edge_cases_edge_cases_BatchListener_on_tick(napi_env env, js_cb* cb, frame_edge_cases_edge_cases_BatchListener_on_tick* f) { @@ -717,7 +962,7 @@ static void invoke_edge_cases_edge_cases_BatchListener_on_tick(napi_env env, js_ napi_value argv[1]; napi_value result; if (!js_cb_call(env, cb, "on_tick", 0, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "BatchListener.on_tick failed"); + js_cb_report(env, f->out_err, "BatchListener.onTick failed"); } napi_close_handle_scope(env, scope); } @@ -725,22 +970,17 @@ static void invoke_edge_cases_edge_cases_BatchListener_on_tick(napi_env env, js_ static void tramp_edge_cases_edge_cases_BatchListener_on_tick(void* ctx, edge_cases_error* out_err) { frame_edge_cases_edge_cases_BatchListener_on_tick f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; js_cb* cb = (js_cb*)ctx; if (js_cb_on_js_thread(cb)) { invoke_edge_cases_edge_cases_BatchListener_on_tick(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 2; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 2, &f, out_err, "BatchListener.onTick"); } } typedef struct { - js_cb_frame hdr; + js_error* out_err; int32_t p0; bool result; } frame_edge_cases_edge_cases_BatchListener_should_continue; @@ -752,13 +992,13 @@ static void invoke_edge_cases_edge_cases_BatchListener_should_continue(napi_env argv[0] = js_new_i32(env, (int32_t)f->p0); napi_value result; if (!js_cb_call(env, cb, "should_continue", 1, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "BatchListener.should_continue failed"); + js_cb_report(env, f->out_err, "BatchListener.shouldContinue failed"); } else { bool v = 0; if (js_arg_bool(env, result, &v)) { f->result = (bool)v; } else { - js_cb_report(env, f->hdr.out_err, "BatchListener.should_continue returned a value of the wrong type"); + js_cb_report(env, f->out_err, "BatchListener.shouldContinue returned a value of the wrong type"); } } napi_close_handle_scope(env, scope); @@ -767,24 +1007,19 @@ static void invoke_edge_cases_edge_cases_BatchListener_should_continue(napi_env static bool tramp_edge_cases_edge_cases_BatchListener_should_continue(void* ctx, int32_t p0, edge_cases_error* out_err) { frame_edge_cases_edge_cases_BatchListener_should_continue f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; f.p0 = p0; js_cb* cb = (js_cb*)ctx; if (js_cb_on_js_thread(cb)) { invoke_edge_cases_edge_cases_BatchListener_should_continue(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 3; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 3, &f, out_err, "BatchListener.shouldContinue"); } return f.result; } typedef struct { - js_cb_frame hdr; + js_error* out_err; edge_cases_edge_cases_Mode result; } frame_edge_cases_edge_cases_BatchListener_pick_mode; @@ -794,13 +1029,13 @@ static void invoke_edge_cases_edge_cases_BatchListener_pick_mode(napi_env env, j napi_value argv[1]; napi_value result; if (!js_cb_call(env, cb, "pick_mode", 0, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "BatchListener.pick_mode failed"); + js_cb_report(env, f->out_err, "BatchListener.pickMode failed"); } else { int32_t v = 0; if (js_arg_i32(env, result, &v)) { f->result = (edge_cases_edge_cases_Mode)v; } else { - js_cb_report(env, f->hdr.out_err, "BatchListener.pick_mode returned a value of the wrong type"); + js_cb_report(env, f->out_err, "BatchListener.pickMode returned a value of the wrong type"); } } napi_close_handle_scope(env, scope); @@ -809,17 +1044,12 @@ static void invoke_edge_cases_edge_cases_BatchListener_pick_mode(napi_env env, j static edge_cases_edge_cases_Mode tramp_edge_cases_edge_cases_BatchListener_pick_mode(void* ctx, edge_cases_error* out_err) { frame_edge_cases_edge_cases_BatchListener_pick_mode f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; js_cb* cb = (js_cb*)ctx; if (js_cb_on_js_thread(cb)) { invoke_edge_cases_edge_cases_BatchListener_pick_mode(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 4; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 4, &f, out_err, "BatchListener.pickMode"); } return f.result; } @@ -833,8 +1063,9 @@ static void dispatch_edge_cases_edge_cases_BatchListener(napi_env env, napi_valu free(req); return; } + if (!js_cb_take(req)) return; if (env == NULL) { - edge_cases_error_set(((js_cb_frame*)req->frame)->out_err, -4, "the JavaScript environment is shutting down"); + edge_cases_error_set(req->out_err, -4, "the JavaScript environment is shutting down"); } else { switch (req->method) { case 0: invoke_edge_cases_edge_cases_BatchListener_on_batch(env, req->cb, (frame_edge_cases_edge_cases_BatchListener_on_batch*)req->frame); break; @@ -848,10 +1079,10 @@ static void dispatch_edge_cases_edge_cases_BatchListener(napi_env env, napi_valu js_cb_finish(req); } -static const edge_cases_edge_cases_BatchListener_vtable vtable_edge_cases_edge_cases_BatchListener = {tramp_edge_cases_edge_cases_BatchListener_on_batch, tramp_edge_cases_edge_cases_BatchListener_on_widget, tramp_edge_cases_edge_cases_BatchListener_on_tick, tramp_edge_cases_edge_cases_BatchListener_should_continue, tramp_edge_cases_edge_cases_BatchListener_pick_mode, js_cb_free}; +static const edge_cases_edge_cases_BatchListener_vtable vtable_edge_cases_edge_cases_BatchListener = {sizeof(edge_cases_edge_cases_BatchListener_vtable), 0, js_cb_free, tramp_edge_cases_edge_cases_BatchListener_on_batch, tramp_edge_cases_edge_cases_BatchListener_on_widget, tramp_edge_cases_edge_cases_BatchListener_on_tick, tramp_edge_cases_edge_cases_BatchListener_should_continue, tramp_edge_cases_edge_cases_BatchListener_pick_mode}; typedef struct { - js_cb_frame hdr; + js_error* out_err; int32_t p0; } frame_edge_cases_edge_cases_Keyword_delete; @@ -862,7 +1093,7 @@ static void invoke_edge_cases_edge_cases_Keyword_delete(napi_env env, js_cb* cb, argv[0] = js_new_i32(env, (int32_t)f->p0); napi_value result; if (!js_cb_call(env, cb, "delete", 1, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "Keyword.delete failed"); + js_cb_report(env, f->out_err, "Keyword.delete failed"); } napi_close_handle_scope(env, scope); } @@ -870,23 +1101,18 @@ static void invoke_edge_cases_edge_cases_Keyword_delete(napi_env env, js_cb* cb, static void tramp_edge_cases_edge_cases_Keyword_delete(void* ctx, int32_t p0, edge_cases_error* out_err) { frame_edge_cases_edge_cases_Keyword_delete f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; f.p0 = p0; js_cb* cb = (js_cb*)ctx; if (js_cb_on_js_thread(cb)) { invoke_edge_cases_edge_cases_Keyword_delete(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 0; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 0, &f, out_err, "Keyword.delete"); } } typedef struct { - js_cb_frame hdr; + js_error* out_err; int32_t result; } frame_edge_cases_edge_cases_Keyword_import; @@ -896,13 +1122,13 @@ static void invoke_edge_cases_edge_cases_Keyword_import(napi_env env, js_cb* cb, napi_value argv[1]; napi_value result; if (!js_cb_call(env, cb, "import", 0, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "Keyword.import failed"); + js_cb_report(env, f->out_err, "Keyword.import failed"); } else { int32_t v = 0; if (js_arg_i32(env, result, &v)) { f->result = (int32_t)v; } else { - js_cb_report(env, f->hdr.out_err, "Keyword.import returned a value of the wrong type"); + js_cb_report(env, f->out_err, "Keyword.import returned a value of the wrong type"); } } napi_close_handle_scope(env, scope); @@ -911,17 +1137,12 @@ static void invoke_edge_cases_edge_cases_Keyword_import(napi_env env, js_cb* cb, static int32_t tramp_edge_cases_edge_cases_Keyword_import(void* ctx, edge_cases_error* out_err) { frame_edge_cases_edge_cases_Keyword_import f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; js_cb* cb = (js_cb*)ctx; if (js_cb_on_js_thread(cb)) { invoke_edge_cases_edge_cases_Keyword_import(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 1; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 1, &f, out_err, "Keyword.import"); } return f.result; } @@ -935,8 +1156,9 @@ static void dispatch_edge_cases_edge_cases_Keyword(napi_env env, napi_value fn, free(req); return; } + if (!js_cb_take(req)) return; if (env == NULL) { - edge_cases_error_set(((js_cb_frame*)req->frame)->out_err, -4, "the JavaScript environment is shutting down"); + edge_cases_error_set(req->out_err, -4, "the JavaScript environment is shutting down"); } else { switch (req->method) { case 0: invoke_edge_cases_edge_cases_Keyword_delete(env, req->cb, (frame_edge_cases_edge_cases_Keyword_delete*)req->frame); break; @@ -947,20 +1169,25 @@ static void dispatch_edge_cases_edge_cases_Keyword(napi_env env, napi_value fn, js_cb_finish(req); } -static const edge_cases_edge_cases_Keyword_vtable vtable_edge_cases_edge_cases_Keyword = {tramp_edge_cases_edge_cases_Keyword_delete, tramp_edge_cases_edge_cases_Keyword_import, js_cb_free}; +static const edge_cases_edge_cases_Keyword_vtable vtable_edge_cases_edge_cases_Keyword = {sizeof(edge_cases_edge_cases_Keyword_vtable), 0, js_cb_free, tramp_edge_cases_edge_cases_Keyword_delete, tramp_edge_cases_edge_cases_Keyword_import}; static napi_value nx_edge_cases_edge_cases_Widget_clone(napi_env env, napi_callback_info info) { JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; - return js_new_handle(env, edge_cases_edge_cases_Widget_clone((const edge_cases_edge_cases_Widget*)h)); + js_sync_begin(); + edge_cases_edge_cases_Widget* r = edge_cases_edge_cases_Widget_clone((const edge_cases_edge_cases_Widget*)h); + js_sync_end(); + return js_new_handle(env, r); } static napi_value nx_edge_cases_edge_cases_Widget_destroy(napi_env env, napi_callback_info info) { JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); edge_cases_edge_cases_Widget_destroy((edge_cases_edge_cases_Widget*)h); + js_sync_end(); return js_undefined(env); } @@ -968,14 +1195,19 @@ static napi_value nx_edge_cases_edge_cases_Vault_clone(napi_env env, napi_callba JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; - return js_new_handle(env, edge_cases_edge_cases_Vault_clone((const edge_cases_edge_cases_Vault*)h)); + js_sync_begin(); + edge_cases_edge_cases_Vault* r = edge_cases_edge_cases_Vault_clone((const edge_cases_edge_cases_Vault*)h); + js_sync_end(); + return js_new_handle(env, r); } static napi_value nx_edge_cases_edge_cases_Vault_destroy(napi_env env, napi_callback_info info) { JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); edge_cases_edge_cases_Vault_destroy((edge_cases_edge_cases_Vault*)h); + js_sync_end(); return js_undefined(env); } @@ -984,7 +1216,9 @@ static napi_value nx_edge_cases_edge_cases_noop(napi_env env, napi_callback_info napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_noop(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1013,7 +1247,9 @@ static napi_value nx_edge_cases_edge_cases_select(napi_env env, napi_callback_in if (!js_arg_bool(env, argv[4], &a4)) goto done; { js_error err = {0}; + js_sync_begin(); int32_t r = edge_cases_edge_cases_select(a0, JS_STR_PTR(a1), a1.len, a2, a2_len, a3, a3_len, a4, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1035,7 +1271,9 @@ static napi_value nx_edge_cases_edge_cases_nested_optional(napi_env env, napi_ca { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_nested_optional(a0, a0_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1071,7 +1309,9 @@ static napi_value nx_edge_cases_edge_cases_optional_params(napi_env env, napi_ca { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_optional_params(a0, a0_len, a1, a1_len, a2, a2_len, a3, a3_len, a4, a4_len, a5, a5_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1088,11 +1328,13 @@ static napi_value nx_edge_cases_edge_cases_register_batch(napi_env env, napi_cal napi_value ret = NULL; js_cb* a0 = NULL; js_cb* a1 = NULL; - if (!js_arg_cb(env, argv[0], "edge_cases_edge_cases_BatchListener", dispatch_edge_cases_edge_cases_BatchListener, &a0)) goto done; - if (!js_arg_cb(env, argv[1], "edge_cases_edge_cases_Keyword", dispatch_edge_cases_edge_cases_Keyword, &a1)) goto done; + if (!js_arg_cb(env, argv[0], "edge_cases_edge_cases_BatchListener", dispatch_edge_cases_edge_cases_BatchListener, false, &a0)) goto done; + if (!js_arg_cb(env, argv[1], "edge_cases_edge_cases_Keyword", dispatch_edge_cases_edge_cases_Keyword, false, &a1)) goto done; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_register_batch((void*)a0, &vtable_edge_cases_edge_cases_BatchListener, (void*)a1, &vtable_edge_cases_edge_cases_Keyword, &err); + js_sync_end(); a0 = NULL; a1 = NULL; if (err.code != 0) { @@ -1126,7 +1368,9 @@ static napi_value nx_edge_cases_edge_cases_widgets_everywhere(napi_env env, napi { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_widgets_everywhere(a0, a0_len, a1, a1_len, a2, a2_len, a3, a3_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1143,7 +1387,9 @@ static napi_value nx_edge_cases_edge_cases_widget_stream(napi_env env, napi_call napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_WidgetStreamIterator* r = edge_cases_edge_cases_widget_stream(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1163,7 +1409,9 @@ static napi_value nx_edge_cases_edge_cases_WidgetStreamIterator_next(napi_env en memset(&item, 0, sizeof item); size_t item_len = 0; js_error err = {0}; + js_sync_begin(); int32_t has = edge_cases_edge_cases_WidgetStreamIterator_next((edge_cases_edge_cases_WidgetStreamIterator*)h, &item, &err); + js_sync_end(); (void)item_len; if (err.code != 0) return js_throw(env, &err); if (has == 0) return js_undefined(env); @@ -1174,7 +1422,9 @@ static napi_value nx_edge_cases_edge_cases_WidgetStreamIterator_destroy(napi_env JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); edge_cases_edge_cases_WidgetStreamIterator_destroy((edge_cases_edge_cases_WidgetStreamIterator*)h); + js_sync_end(); return js_undefined(env); } @@ -1189,7 +1439,9 @@ static napi_value nx_edge_cases_edge_cases_async_widget(napi_env env, napi_callb napi_value ret = NULL; { js_async* a = js_async_begin(env, JS_R_HANDLE, "edge_cases_edge_cases_async_widget", &ret); + js_sync_begin(); edge_cases_edge_cases_async_widget(done_edge_cases_edge_cases_async_widget, a); + js_sync_end(); } goto done; done: @@ -1203,7 +1455,9 @@ static napi_value nx_edge_cases_edge_cases_make_widget(napi_env env, napi_callba if (!js_arg_str(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_Widget* r = edge_cases_edge_cases_make_widget(JS_STR_PTR(a0), a0.len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1223,7 +1477,9 @@ static napi_value nx_edge_cases_edge_cases_find_widget(napi_env env, napi_callba if (!js_arg_i64(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_Widget* r = edge_cases_edge_cases_find_widget(a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1245,7 +1501,9 @@ static napi_value nx_edge_cases_edge_cases_inspect_widget(napi_env env, napi_cal { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_inspect_widget((const edge_cases_edge_cases_Widget*)a0, (const edge_cases_edge_cases_Widget*)a1, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1262,7 +1520,9 @@ static napi_value nx_edge_cases_edge_cases_string_stream(napi_env env, napi_call napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_StringStreamIterator* r = edge_cases_edge_cases_string_stream(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1282,7 +1542,9 @@ static napi_value nx_edge_cases_edge_cases_StringStreamIterator_next(napi_env en memset(&item, 0, sizeof item); size_t item_len = 0; js_error err = {0}; + js_sync_begin(); int32_t has = edge_cases_edge_cases_StringStreamIterator_next((edge_cases_edge_cases_StringStreamIterator*)h, &item, &item_len, &err); + js_sync_end(); if (err.code != 0) return js_throw(env, &err); if (has == 0) return js_undefined(env); return js_take_str(env, item, item_len); @@ -1292,7 +1554,9 @@ static napi_value nx_edge_cases_edge_cases_StringStreamIterator_destroy(napi_env JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); edge_cases_edge_cases_StringStreamIterator_destroy((edge_cases_edge_cases_StringStreamIterator*)h); + js_sync_end(); return js_undefined(env); } @@ -1301,7 +1565,9 @@ static napi_value nx_edge_cases_edge_cases_scalar_stream(napi_env env, napi_call napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_ScalarStreamIterator* r = edge_cases_edge_cases_scalar_stream(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1321,7 +1587,9 @@ static napi_value nx_edge_cases_edge_cases_ScalarStreamIterator_next(napi_env en memset(&item, 0, sizeof item); size_t item_len = 0; js_error err = {0}; + js_sync_begin(); int32_t has = edge_cases_edge_cases_ScalarStreamIterator_next((edge_cases_edge_cases_ScalarStreamIterator*)h, &item, &err); + js_sync_end(); (void)item_len; if (err.code != 0) return js_throw(env, &err); if (has == 0) return js_undefined(env); @@ -1332,7 +1600,9 @@ static napi_value nx_edge_cases_edge_cases_ScalarStreamIterator_destroy(napi_env JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); edge_cases_edge_cases_ScalarStreamIterator_destroy((edge_cases_edge_cases_ScalarStreamIterator*)h); + js_sync_end(); return js_undefined(env); } @@ -1341,7 +1611,9 @@ static napi_value nx_edge_cases_edge_cases_mode_stream(napi_env env, napi_callba napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_ModeStreamIterator* r = edge_cases_edge_cases_mode_stream(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1361,7 +1633,9 @@ static napi_value nx_edge_cases_edge_cases_ModeStreamIterator_next(napi_env env, memset(&item, 0, sizeof item); size_t item_len = 0; js_error err = {0}; + js_sync_begin(); int32_t has = edge_cases_edge_cases_ModeStreamIterator_next((edge_cases_edge_cases_ModeStreamIterator*)h, &item, &err); + js_sync_end(); (void)item_len; if (err.code != 0) return js_throw(env, &err); if (has == 0) return js_undefined(env); @@ -1372,7 +1646,9 @@ static napi_value nx_edge_cases_edge_cases_ModeStreamIterator_destroy(napi_env e JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); edge_cases_edge_cases_ModeStreamIterator_destroy((edge_cases_edge_cases_ModeStreamIterator*)h); + js_sync_end(); return js_undefined(env); } @@ -1386,7 +1662,9 @@ static napi_value nx_edge_cases_edge_cases_async_void(napi_env env, napi_callbac napi_value ret = NULL; { js_async* a = js_async_begin(env, JS_R_VOID, "edge_cases_edge_cases_async_void", &ret); + js_sync_begin(); edge_cases_edge_cases_async_void(done_edge_cases_edge_cases_async_void, a); + js_sync_end(); } goto done; done: @@ -1407,7 +1685,9 @@ static napi_value nx_edge_cases_edge_cases_async_bytes(napi_env env, napi_callba if (!js_arg_u32(env, argv[0], &a0)) goto done; { js_async* a = js_async_begin(env, JS_R_BYTES, "edge_cases_edge_cases_async_bytes", &ret); + js_sync_begin(); edge_cases_edge_cases_async_bytes(a0, done_edge_cases_edge_cases_async_bytes, a); + js_sync_end(); } goto done; done: @@ -1426,7 +1706,9 @@ static napi_value nx_edge_cases_edge_cases_async_struct(napi_env env, napi_callb napi_value ret = NULL; { js_async* a = js_async_begin(env, JS_R_BYTES, "edge_cases_edge_cases_async_struct", &ret); + js_sync_begin(); edge_cases_edge_cases_async_struct(done_edge_cases_edge_cases_async_struct, a); + js_sync_end(); } goto done; done: @@ -1447,7 +1729,9 @@ static napi_value nx_edge_cases_edge_cases_async_list(napi_env env, napi_callbac if (!js_arg_handle(env, argv[0], &token, true)) goto done; { js_async* a = js_async_begin(env, JS_R_BYTES, "edge_cases_edge_cases_async_list", &ret); + js_sync_begin(); edge_cases_edge_cases_async_list((edge_cases_cancel_token*)token, done_edge_cases_edge_cases_async_list, a); + js_sync_end(); } goto done; done: @@ -1466,7 +1750,9 @@ static napi_value nx_edge_cases_edge_cases_async_optional(napi_env env, napi_cal napi_value ret = NULL; { js_async* a = js_async_begin(env, JS_R_BYTES, "edge_cases_edge_cases_async_optional", &ret); + js_sync_begin(); edge_cases_edge_cases_async_optional(done_edge_cases_edge_cases_async_optional, a); + js_sync_end(); } goto done; done: @@ -1484,7 +1770,9 @@ static napi_value nx_edge_cases_edge_cases_async_enum(napi_env env, napi_callbac napi_value ret = NULL; { js_async* a = js_async_begin(env, JS_R_I32, "edge_cases_edge_cases_async_enum", &ret); + js_sync_begin(); edge_cases_edge_cases_async_enum(done_edge_cases_edge_cases_async_enum, a); + js_sync_end(); } goto done; done: @@ -1507,7 +1795,9 @@ static napi_value nx_edge_cases_edge_cases_async_keywords(napi_env env, napi_cal if (!js_arg_i32(env, argv[1], &a1)) goto done; { js_async* a = js_async_begin(env, JS_R_STR, "edge_cases_edge_cases_async_keywords", &ret); + js_sync_begin(); edge_cases_edge_cases_async_keywords(JS_STR_PTR(a0), a0.len, a1, done_edge_cases_edge_cases_async_keywords, a); + js_sync_end(); } goto done; done: @@ -1524,7 +1814,9 @@ static napi_value nx_edge_cases_edge_cases_keywords_roundtrip(napi_env env, napi { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_keywords_roundtrip(a0, a0_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1544,7 +1836,9 @@ static napi_value nx_edge_cases_edge_cases_legacy_value(napi_env env, napi_callb { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_legacy_value((edge_cases_edge_cases_Legacy)a0, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1563,7 +1857,9 @@ static napi_value nx_edge_cases_edge_cases_Widget_refresh(napi_env env, napi_cal if (!js_arg_handle(env, argv[0], &self_h, false)) goto done; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_Widget_refresh((const edge_cases_edge_cases_Widget*)self_h, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1588,7 +1884,9 @@ static napi_value nx_edge_cases_edge_cases_Widget_rename(napi_env env, napi_call { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_Widget_rename((const edge_cases_edge_cases_Widget*)self_h, JS_STR_PTR(a0), a0.len, a1, a1_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1617,7 +1915,9 @@ static napi_value nx_edge_cases_edge_cases_Widget_fetch(napi_env env, napi_callb if (!js_arg_i64(env, argv[1], &a0)) goto done; { js_async* a = js_async_begin(env, JS_R_BYTES, "edge_cases_edge_cases_Widget_fetch", &ret); + js_sync_begin(); edge_cases_edge_cases_Widget_fetch((const edge_cases_edge_cases_Widget*)self_h, a0, done_edge_cases_edge_cases_Widget_fetch, a); + js_sync_end(); } goto done; done: @@ -1632,7 +1932,9 @@ static napi_value nx_edge_cases_edge_cases_Widget_old_name(napi_env env, napi_ca { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = edge_cases_edge_cases_Widget_old_name((const edge_cases_edge_cases_Widget*)self_h, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1644,6 +1946,50 @@ done: return ret; } +static napi_value nx_edge_cases_edge_cases_Widget_close(napi_env env, napi_callback_info info) { + JS_ARGS(1); + napi_value ret = NULL; + void* self_h = NULL; + if (!js_arg_handle(env, argv[0], &self_h, false)) goto done; + { + js_error err = {0}; + js_sync_begin(); + bool r = edge_cases_edge_cases_Widget_close((const edge_cases_edge_cases_Widget*)self_h, &err); + js_sync_end(); + if (err.code != 0) { + js_throw(env, &err); + goto done; + } + ret = js_new_bool(env, r); + } + goto done; +done: + return ret; +} + +static napi_value nx_edge_cases_edge_cases_Widget_dispose(napi_env env, napi_callback_info info) { + JS_ARGS(2); + napi_value ret = NULL; + void* self_h = NULL; + bool a0 = 0; + if (!js_arg_handle(env, argv[0], &self_h, false)) goto done; + if (!js_arg_bool(env, argv[1], &a0)) goto done; + { + js_error err = {0}; + js_sync_begin(); + edge_cases_edge_cases_Widget_dispose((const edge_cases_edge_cases_Widget*)self_h, a0, &err); + js_sync_end(); + if (err.code != 0) { + js_throw(env, &err); + goto done; + } + ret = js_undefined(env); + } + goto done; +done: + return ret; +} + static napi_value nx_edge_cases_edge_cases_Vault_open(napi_env env, napi_callback_info info) { JS_ARGS(1); napi_value ret = NULL; @@ -1651,7 +1997,9 @@ static napi_value nx_edge_cases_edge_cases_Vault_open(napi_env env, napi_callbac if (!js_arg_str(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_Vault* r = edge_cases_edge_cases_Vault_open(JS_STR_PTR(a0), a0.len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1671,7 +2019,9 @@ static napi_value nx_edge_cases_edge_cases_Vault_peek(napi_env env, napi_callbac if (!js_arg_handle(env, argv[0], &self_h, false)) goto done; { js_error err = {0}; + js_sync_begin(); edge_cases_edge_cases_Widget* r = edge_cases_edge_cases_Vault_peek((const edge_cases_edge_cases_Vault*)self_h, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1683,12 +2033,35 @@ done: return ret; } +static napi_value nx_edge_cases_edge_cases_Vault_handle(napi_env env, napi_callback_info info) { + JS_ARGS(1); + napi_value ret = NULL; + void* self_h = NULL; + if (!js_arg_handle(env, argv[0], &self_h, false)) goto done; + { + js_error err = {0}; + js_sync_begin(); + int64_t r = edge_cases_edge_cases_Vault_handle((const edge_cases_edge_cases_Vault*)self_h, &err); + js_sync_end(); + if (err.code != 0) { + js_throw(env, &err); + goto done; + } + ret = js_new_i64(env, r); + } + goto done; +done: + return ret; +} + static napi_value nx_edge_cases_edge_cases_Vault_probe(napi_env env, napi_callback_info info) { JS_ARGS(0); napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); bool r = edge_cases_edge_cases_Vault_probe(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1700,12 +2073,31 @@ done: return ret; } +static napi_value nx_edge_cases_edge_cases_Vault_to_string(napi_env env, napi_callback_info info) { + JS_ARGS(0); + napi_value ret = NULL; + { + js_error err = {0}; + size_t out_len = 0; + js_sync_begin(); + const uint8_t* r = edge_cases_edge_cases_Vault_to_string(&out_len, &err); + js_sync_end(); + if (err.code != 0) { + js_throw(env, &err); + goto done; + } + ret = js_take_str(env, r, out_len); + } + goto done; +done: + return ret; +} + NAPI_MODULE_INIT() { - js_env* e = (js_env*)calloc(1, sizeof *e); - napi_set_instance_data(env, e, js_env_free, NULL); + js_env_init(env); napi_property_descriptor props[] = { JS_RUNTIME_EXPORTS, - {"edge_cases_edge_cases_checksum", NULL, nx_edge_cases_edge_cases_checksum, NULL, NULL, NULL, napi_default, NULL}, + {"edge_cases_edge_cases_contract", NULL, nx_edge_cases_edge_cases_contract, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Widget_clone", NULL, nx_edge_cases_edge_cases_Widget_clone, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Widget_destroy", NULL, nx_edge_cases_edge_cases_Widget_destroy, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Vault_clone", NULL, nx_edge_cases_edge_cases_Vault_clone, NULL, NULL, NULL, napi_default, NULL}, @@ -1745,9 +2137,13 @@ NAPI_MODULE_INIT() { {"edge_cases_edge_cases_Widget_rename", NULL, nx_edge_cases_edge_cases_Widget_rename, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Widget_fetch", NULL, nx_edge_cases_edge_cases_Widget_fetch, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Widget_old_name", NULL, nx_edge_cases_edge_cases_Widget_old_name, NULL, NULL, NULL, napi_default, NULL}, + {"edge_cases_edge_cases_Widget_close", NULL, nx_edge_cases_edge_cases_Widget_close, NULL, NULL, NULL, napi_default, NULL}, + {"edge_cases_edge_cases_Widget_dispose", NULL, nx_edge_cases_edge_cases_Widget_dispose, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Vault_open", NULL, nx_edge_cases_edge_cases_Vault_open, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Vault_peek", NULL, nx_edge_cases_edge_cases_Vault_peek, NULL, NULL, NULL, napi_default, NULL}, + {"edge_cases_edge_cases_Vault_handle", NULL, nx_edge_cases_edge_cases_Vault_handle, NULL, NULL, NULL, napi_default, NULL}, {"edge_cases_edge_cases_Vault_probe", NULL, nx_edge_cases_edge_cases_Vault_probe, NULL, NULL, NULL, napi_default, NULL}, + {"edge_cases_edge_cases_Vault_to_string", NULL, nx_edge_cases_edge_cases_Vault_to_string, NULL, NULL, NULL, napi_default, NULL}, }; napi_define_properties(env, exports, sizeof props / sizeof props[0], props); return exports; diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_d_ts.snap index f7a0d645..a7c2071e 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -24,13 +24,13 @@ export declare class CancelledError extends EdgeCasesError { /** The root error class, under a name no namespace member can shadow. */ declare const $Error: typeof EdgeCasesError; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** No parameters and no result */ export declare namespace edge_cases { /** The errors of the `edge_cases` error domain. */ export class EdgeErrorsError extends $Error { @@ -184,6 +184,10 @@ export declare namespace edge_cases { fetch(id: bigint): Promise; /** @deprecated Use rename instead */ oldName(): string; + /** Spelled like the release method of several wrappers */ + close_(): boolean; + /** Spelled like the release method of the Dart and .NET wrappers */ + dispose(force: boolean): void; /** * Release this wrapper's reference to the native object. Safe to call * more than once; a wrapper that is never closed is released when it is @@ -205,11 +209,15 @@ export declare namespace edge_cases { */ static open(path: string): edge_cases.Vault; peek(): edge_cases.Widget | null; + /** Spelled like the handle accessor of several wrappers */ + handle(): bigint; /** * Throwing static * @throws {edge_cases.EdgeErrorsError} */ static probe(): boolean; + /** Static spelled like an inherited object member */ + static toString(): string; /** * Release this wrapper's reference to the native object. Safe to call * more than once; a wrapper that is never closed is released when it is diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_js.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_js.snap index a3eb9a21..fe279862 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -32,22 +35,30 @@ export { EdgeCasesError, CancelledError } from './runtime.js'; import { createRequire } from 'node:module'; -// The native entry points: the N-API addon `npm install` builds from -// edge_cases_node.c (with node-gyp), or a prebuilt copy next to this file. +// The native entry points: the N-API addon built from edge_cases_node.c. A prebuilt +// copy comes first (from the `edge_cases--` package `npm install` +// picks for this platform, or under prebuilds/-/), then the one +// node-gyp compiles at install time when none matches. function $loadAddon(name) { const require = createRequire(import.meta.url); - const tried = []; - for (const dir of ['./build/Release/', './build/Debug/', './']) { + const platform = `${process.platform}-${process.arch}`; + const candidates = [ + `edge_cases-${platform}/${name}`, + `./prebuilds/${platform}/${name}`, + `./build/Release/${name}`, + `./build/Debug/${name}`, + ]; + for (const candidate of candidates) { try { - return require(dir + name); + return require(candidate); } catch (e) { if (e.code !== 'MODULE_NOT_FOUND') throw e; - tried.push(dir + name); } } throw new Error( - `the native addon ${name} was not found (tried ${tried.join(', ')}); ` + - 'build it with `npm install` or `node-gyp rebuild` in this package', + `the native addon ${name} was not found (tried ${candidates.join(', ')}); ` + + 'install the package for this platform, or build the addon with `npm install` ' + + 'or `node-gyp rebuild` in this package', ); } @@ -57,7 +68,61 @@ $raw.$setup($Fault); // Object tokens in value buffers are already bigint handles here. const $token = (t) => t; -$verify($raw, 'edge_cases', 'edge_cases', 3, [['edge_cases', 0x1567f441073bd4aan]]); +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['edge_cases', [ + [0x056a86511785ba2bn, 0x5cba3f16d57c6cdbn, 'edge_cases.make_widget'], + [0x07bfa9ecb2b720d2n, 0x3813b3f57c64bf35n, 'edge_cases.EdgeErrors'], + [0x0aaa11867e37f991n, 0xf090dd52146ebdf0n, 'edge_cases.Widget.refresh'], + [0x0f9701493898ae4cn, 0x6944f47a275c96e8n, 'edge_cases.Inner'], + [0x1226b1adac179f7dn, 0x0e95a00ab0bef637n, 'edge_cases.async_keywords'], + [0x16d36bef1541136cn, 0x5ea7021ec324298fn, 'edge_cases.find_widget'], + [0x1b7324c55e980febn, 0xabf87fe372358f42n, 'edge_cases.Widget.old_name'], + [0x1cbd731467c2bab8n, 0x4915b9989c2c9074n, 'edge_cases.Vault'], + [0x2c2a6de67d894b0an, 0x50a4496b1e7191can, 'edge_cases.Vault.open'], + [0x322b4cfcd2d6861cn, 0xca7527a379ae8465n, 'edge_cases.Keywords'], + [0x3f8e23a5f32906f4n, 0x8d4c908d796f130dn, 'edge_cases.Leaf'], + [0x42ab8a685457b97cn, 0xe5ce30500cadb32dn, 'edge_cases.Vault.handle'], + [0x4a68540e2836d3d0n, 0xa20e44c71e420ed2n, 'edge_cases.mode_stream'], + [0x4e030580ac0b111dn, 0xb68b7a18d910e175n, 'edge_cases.legacy_value'], + [0x4f5e4e5291864ef3n, 0x55a1a88b21a08f8bn, 'edge_cases.Legacy'], + [0x4f84b06171b4213dn, 0xbaedcfdebb6a2820n, 'edge_cases.scalar_stream'], + [0x5067a5ff2d9ef910n, 0xbfc79b18fc4714fcn, 'edge_cases.Widget'], + [0x59aff159112fbf5bn, 0x18e4def57a8cfa49n, 'edge_cases.optional_params'], + [0x5daf65b98fa77c48n, 0xc817a8cb202e59c2n, 'edge_cases.keywords_roundtrip'], + [0x5de8a5c9726c2c5an, 0x7c50b5b65f7a896dn, 'edge_cases.register_batch'], + [0x618a620309994c3cn, 0x826ad0b1d208d723n, 'edge_cases.async_bytes'], + [0x648285cdba0a7b8en, 0xcf6e80e783a4d8ben, 'edge_cases.WidgetHolder'], + [0x649c78079824b10dn, 0x5b12dd26e0634b69n, 'edge_cases.async_widget'], + [0x7da7a7357d122a37n, 0xf544e025ffaa3d20n, 'edge_cases.async_void'], + [0x7e4015bc3a472e2dn, 0xfe22291d90b82e4an, 'edge_cases.widget_stream'], + [0x805276e5d81fdba7n, 0xd9b64e360bd03b2bn, 'edge_cases.Outer'], + [0x80e1183326542df4n, 0xe8a57e12cc6ed276n, 'edge_cases.select'], + [0x923ce5e9bcd769a1n, 0x0b973f5dc07dda8bn, 'edge_cases.Vault.to_string'], + [0x98ca8c01f5239404n, 0xfe11c93eb6b303efn, 'edge_cases.Widget.rename'], + [0xa13e22e98ea278d4n, 0xf1fae215d7387800n, 'edge_cases.widgets_everywhere'], + [0xa84a8c9bf2a881e1n, 0xbe1dc840bf221996n, 'edge_cases.async_optional'], + [0xad958c478fbf205dn, 0xe386b0716d24f19bn, 'edge_cases.Retired'], + [0xb2fc3964ef56ab39n, 0x73c325c8bc351236n, 'edge_cases.async_list'], + [0xb3403af521cf0006n, 0x6abc30d48206a3ban, 'edge_cases.Widget.close'], + [0xb794304a22a406b8n, 0xf6fb98711c516105n, 'edge_cases.async_struct'], + [0xba3936106a49c47dn, 0xb7025f2fadd6288en, 'edge_cases.Vault.peek'], + [0xc155b0c3cfb164f4n, 0xc777ae57d92ebc47n, 'edge_cases.noop'], + [0xcdc18607777019ddn, 0xb81a31e5c20aa7a8n, 'edge_cases.inspect_widget'], + [0xd164a82aeb64b8ccn, 0xded02c68005c4737n, 'edge_cases.nested_optional'], + [0xd978ccad2468038fn, 0x30c39c65d6235153n, 'edge_cases.Mode'], + [0xdbe9d947431c9124n, 0x39537fb848fdf980n, 'edge_cases.BatchListener'], + [0xeae8b7862c08bb07n, 0xfe12eeaa830452ean, 'edge_cases.Keyword'], + [0xf1e155a05ae262b8n, 0xb575eb818a423d42n, 'edge_cases.async_enum'], + [0xf22e210bdbce7343n, 0x2a5e0f801bc8c768n, 'edge_cases.Widget.dispose'], + [0xf3c7cf675563ee6cn, 0x0ecffeef399facc8n, 'edge_cases.Vault.probe'], + [0xfd34bc67acac379en, 0x576caa14f3c0901an, 'edge_cases.Widget.fetch'], + [0xff2e9d2bff941786n, 0x7e96764ef1f5fbaan, 'edge_cases.string_stream'], + ]], +]; +$verify($raw, 'edge_cases', 'edge_cases', 4, $contract); // The native cancel tokens behind `AbortSignal`s. const $tokens = { @@ -67,9 +132,10 @@ const $tokens = { }; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.edge_cases_debug_live(kind); @@ -82,9 +148,12 @@ const edge_cases$BusyError = class BusyError extends edge_cases$EdgeErrorsError super(7, message); } }; -const $codes$edge_cases$EdgeErrors = new Map([[7, edge_cases$BusyError]]); +const $codes$edge_cases$EdgeErrors = new Map([ + [7, edge_cases$BusyError], +]); +const $payloads$edge_cases$EdgeErrors = new Map(); function $from$edge_cases$EdgeErrors(e) { - return $domain(e, $codes$edge_cases$EdgeErrors); + return $domain(e, $codes$edge_cases$EdgeErrors, $payloads$edge_cases$EdgeErrors); } const edge_cases$Legacy = Object.freeze({ @@ -113,38 +182,38 @@ function $r$edge_cases$Leaf(r) { } function $w$edge_cases$Inner(w, v) { $w$edge_cases$Leaf(w, v.leaf); - w.writeList(v.leaves, $w$edge_cases$Leaf); + $w_list_Leaf(w, v.leaves); } function $r$edge_cases$Inner(r) { return { leaf: $r$edge_cases$Leaf(r), - leaves: r.readList($r$edge_cases$Leaf), + leaves: $r_list_Leaf(r), }; } function $w$edge_cases$Outer(w, v) { $w$edge_cases$Inner(w, v.inner); - w.writeOpt(v.maybe_inner, $w$edge_cases$Inner); - w.writeMap(v.by_name, $W.String, $w$edge_cases$Inner); - w.writeList(v.modes, $W.I32); - w.writeList(v.matrix, (w, v) => w.writeList(v, $W.I32)); - w.writeList(v.sparse, (w, v) => w.writeOpt(v, $W.I64)); - w.writeOpt(v.blob, $W.Bytes); - w.writeOpt(v.note, $W.String); - w.writeMap(v.by_id, (w, k) => w.writeI64(BigInt(k)), $W.String); + $w_opt_Inner(w, v.maybe_inner); + $w_map_string_Inner(w, v.by_name); + $w_list_Mode(w, v.modes); + $w_list_list_i32(w, v.matrix); + $w_list_opt_i64(w, v.sparse); + $w_opt_bytes(w, v.blob); + $w_opt_string(w, v.note); + $w_map_i64_string(w, v.by_id); w.writeBool(v._hidden); w.writeU8(v.v2); } function $r$edge_cases$Outer(r) { return { inner: $r$edge_cases$Inner(r), - maybe_inner: r.readOpt($r$edge_cases$Inner), - by_name: r.readMap($R.String, $r$edge_cases$Inner), - modes: r.readList($R.I32), - matrix: r.readList((r) => r.readList($R.I32)), - sparse: r.readList((r) => r.readOpt($R.I64)), - blob: r.readOpt($R.Bytes), - note: r.readOpt($R.String), - by_id: r.readMap($R.I64, $R.String), + maybe_inner: $r_opt_Inner(r), + by_name: $r_map_string_Inner(r), + modes: $r_list_Mode(r), + matrix: $r_list_list_i32(r), + sparse: $r_list_opt_i64(r), + blob: $r_opt_bytes(r), + note: $r_opt_string(r), + by_id: $r_map_i64_string(r), _hidden: r.readBool(), v2: r.readU8(), }; @@ -282,28 +351,195 @@ function $r$edge_cases$Retired(r) { }; } function $w$edge_cases$WidgetHolder(w, v) { - w.writeU64(BigInt($clone(v.primary, edge_cases$Widget))); - w.writeOpt(v.fallback, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))); - w.writeList(v.extras, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))); - w.writeMap(v.by_name, $W.String, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))); + $w$edge_cases$Widget(w, v.primary); + $w_opt_Widget(w, v.fallback); + $w_list_Widget(w, v.extras); + $w_map_string_Widget(w, v.by_name); } function $r$edge_cases$WidgetHolder(r) { return { - primary: $adopt(edge_cases$Widget, $token(r.readU64())), - fallback: r.readOpt((r) => $adopt(edge_cases$Widget, $token(r.readU64()))), - extras: r.readList((r) => $adopt(edge_cases$Widget, $token(r.readU64()))), - by_name: r.readMap($R.String, (r) => $adopt(edge_cases$Widget, $token(r.readU64()))), + primary: $r$edge_cases$Widget(r), + fallback: $r_opt_Widget(r), + extras: $r_list_Widget(r), + by_name: $r_map_string_Widget(r), }; } +function $w$edge_cases$Widget(w, v) { + w.writeU64(BigInt($clone(v, edge_cases$Widget))); +} +function $r$edge_cases$Widget(r) { + return $adopt(edge_cases$Widget, $token(r.readU64())); +} +/** `[Leaf]` */ +function $w_list_Leaf(w, v) { + w.writeList(v, $w$edge_cases$Leaf); +} +function $r_list_Leaf(r) { + return r.readList($r$edge_cases$Leaf); +} +/** `[Mode]` */ +function $w_list_Mode(w, v) { + w.writeList(v, $W.I32); +} +function $r_list_Mode(r) { + return r.readList($R.I32); +} +/** `[Outer]` */ +function $w_list_Outer(w, v) { + w.writeList(v, $w$edge_cases$Outer); +} +function $r_list_Outer(r) { + return r.readList($r$edge_cases$Outer); +} +/** `[Widget]` */ +function $w_list_Widget(w, v) { + w.writeList(v, $w$edge_cases$Widget); +} +function $r_list_Widget(r) { + return r.readList($r$edge_cases$Widget); +} +/** `[i32]` */ +function $w_list_i32(w, v) { + w.writeList(v, $W.I32); +} +function $r_list_i32(r) { + return r.readList($R.I32); +} +/** `[[i32]]` */ +function $w_list_list_i32(w, v) { + w.writeList(v, $w_list_i32); +} +function $r_list_list_i32(r) { + return r.readList($r_list_i32); +} +/** `[Outer?]` */ +function $w_list_opt_Outer(w, v) { + w.writeList(v, $w_opt_Outer); +} +function $r_list_opt_Outer(r) { + return r.readList($r_opt_Outer); +} +/** `[Widget?]` */ +function $w_list_opt_Widget(w, v) { + w.writeList(v, $w_opt_Widget); +} +function $r_list_opt_Widget(r) { + return r.readList($r_opt_Widget); +} +/** `[i64?]` */ +function $w_list_opt_i64(w, v) { + w.writeList(v, $w_opt_i64); +} +function $r_list_opt_i64(r) { + return r.readList($r_opt_i64); +} +/** `[string]` */ +function $w_list_string(w, v) { + w.writeList(v, $W.String); +} +function $r_list_string(r) { + return r.readList($R.String); +} +/** `{i64:string}` */ +function $w_map_i64_string(w, v) { + w.writeMap(v, $WK.I64, $W.String); +} +function $r_map_i64_string(r) { + return r.readMap($R.I64, $R.String); +} +/** `{string:Inner}` */ +function $w_map_string_Inner(w, v) { + w.writeMap(v, $WK.String, $w$edge_cases$Inner); +} +function $r_map_string_Inner(r) { + return r.readMap($R.String, $r$edge_cases$Inner); +} +/** `{string:Widget}` */ +function $w_map_string_Widget(w, v) { + w.writeMap(v, $WK.String, $w$edge_cases$Widget); +} +function $r_map_string_Widget(r) { + return r.readMap($R.String, $r$edge_cases$Widget); +} +/** `Inner?` */ +function $w_opt_Inner(w, v) { + w.writeOpt(v, $w$edge_cases$Inner); +} +function $r_opt_Inner(r) { + return r.readOpt($r$edge_cases$Inner); +} +/** `Mode?` */ +function $w_opt_Mode(w, v) { + w.writeOpt(v, $W.I32); +} +function $r_opt_Mode(r) { + return r.readOpt($R.I32); +} +/** `Outer?` */ +function $w_opt_Outer(w, v) { + w.writeOpt(v, $w$edge_cases$Outer); +} +function $r_opt_Outer(r) { + return r.readOpt($r$edge_cases$Outer); +} +/** `Widget?` */ +function $w_opt_Widget(w, v) { + w.writeOpt(v, $w$edge_cases$Widget); +} +function $r_opt_Widget(r) { + return r.readOpt($r$edge_cases$Widget); +} +/** `bytes?` */ +function $w_opt_bytes(w, v) { + w.writeOpt(v, $W.Bytes); +} +function $r_opt_bytes(r) { + return r.readOpt($R.Bytes); +} +/** `i32?` */ +function $w_opt_i32(w, v) { + w.writeOpt(v, $W.I32); +} +function $r_opt_i32(r) { + return r.readOpt($R.I32); +} +/** `i64?` */ +function $w_opt_i64(w, v) { + w.writeOpt(v, $W.I64); +} +function $r_opt_i64(r) { + return r.readOpt($R.I64); +} +/** `[Outer?]?` */ +function $w_opt_list_opt_Outer(w, v) { + w.writeOpt(v, $w_list_opt_Outer); +} +function $r_opt_list_opt_Outer(r) { + return r.readOpt($r_list_opt_Outer); +} +/** `[string]?` */ +function $w_opt_list_string(w, v) { + w.writeOpt(v, $w_list_string); +} +function $r_opt_list_string(r) { + return r.readOpt($r_list_string); +} +/** `string?` */ +function $w_opt_string(w, v) { + w.writeOpt(v, $W.String); +} +function $r_opt_string(r) { + return r.readOpt($R.String); +} function $adapt$edge_cases$BatchListener(impl) { $impl(impl, 'BatchListener'); return { on_batch($a0, $a1, $a2, $a3) { - impl.onBatch($decode($a0, (r) => r.readList($r$edge_cases$Outer)), $a1, $decode($a2, (r) => r.readOpt($r$edge_cases$Outer)), $a3); + impl.onBatch($decode($a0, $r_list_Outer), $a1, $decode($a2, $r_opt_Outer), $a3); }, on_widget($a0, $a1, $a2) { - impl.onWidget($adopt(edge_cases$Widget, $a0), $adoptOpt(edge_cases$Widget, $a1), $decode($a2, (r) => r.readList((r) => $adopt(edge_cases$Widget, $token(r.readU64()))))); + impl.onWidget($adopt(edge_cases$Widget, $a0), $adoptOpt(edge_cases$Widget, $a1), $decode($a2, $r_list_Widget)); }, on_tick() { impl.onTick(); @@ -353,7 +589,7 @@ const edge_cases$Widget = class Widget extends $Object { rename(class_, import_) { const $self = $lend(this, edge_cases$Widget); try { - return $raw.edge_cases_edge_cases_Widget_rename($self, class_, $encode(import_, (w, v) => w.writeOpt(v, $W.String))); + return $raw.edge_cases_edge_cases_Widget_rename($self, class_, $encode(import_, $w_opt_string)); } catch ($e) { throw $fault($e); } finally { @@ -380,6 +616,26 @@ const edge_cases$Widget = class Widget extends $Object { $unlend(this); } } + close_() { + const $self = $lend(this, edge_cases$Widget); + try { + return $raw.edge_cases_edge_cases_Widget_close($self); + } catch ($e) { + throw $fault($e); + } finally { + $unlend(this); + } + } + dispose(force) { + const $self = $lend(this, edge_cases$Widget); + try { + $raw.edge_cases_edge_cases_Widget_dispose($self, force); + } catch ($e) { + throw $fault($e); + } finally { + $unlend(this); + } + } }; const edge_cases$Vault = class Vault extends $Object { @@ -410,6 +666,16 @@ const edge_cases$Vault = class Vault extends $Object { $unlend(this); } } + handle() { + const $self = $lend(this, edge_cases$Vault); + try { + return $raw.edge_cases_edge_cases_Vault_handle($self); + } catch ($e) { + throw $fault($e); + } finally { + $unlend(this); + } + } static probe() { try { return $raw.edge_cases_edge_cases_Vault_probe(); @@ -417,6 +683,13 @@ const edge_cases$Vault = class Vault extends $Object { throw $from$edge_cases$EdgeErrors($e); } } + static toString() { + try { + return $raw.edge_cases_edge_cases_Vault_to_string(); + } catch ($e) { + throw $fault($e); + } + } }; const edge_cases$noop = function noop() { @@ -429,7 +702,7 @@ const edge_cases$noop = function noop() { const edge_cases$select = function select(class_, from, func, range, type) { try { - return $raw.edge_cases_edge_cases_select(class_, from, $encode(func, (w, v) => w.writeList(v, $W.String)), $encode(range, (w, v) => w.writeOpt(v, $W.I32)), type); + return $raw.edge_cases_edge_cases_select(class_, from, $encode(func, $w_list_string), $encode(range, $w_opt_i32), type); } catch ($e) { throw $fault($e); } @@ -437,7 +710,7 @@ const edge_cases$select = function select(class_, from, func, range, type) { const edge_cases$nestedOptional = function nestedOptional(items) { try { - return $decode($raw.edge_cases_edge_cases_nested_optional($encode(items, (w, v) => w.writeOpt(v, (w, v) => w.writeList(v, (w, v) => w.writeOpt(v, $w$edge_cases$Outer))))), (r) => r.readOpt((r) => r.readList((r) => r.readOpt($r$edge_cases$Outer)))); + return $decode($raw.edge_cases_edge_cases_nested_optional($encode(items, $w_opt_list_opt_Outer)), $r_opt_list_opt_Outer); } catch ($e) { throw $fault($e); } @@ -445,7 +718,7 @@ const edge_cases$nestedOptional = function nestedOptional(items) { const edge_cases$optionalParams = function optionalParams(text, count, outer, tags, blob, mode) { try { - return $decode($raw.edge_cases_edge_cases_optional_params($encode(text, (w, v) => w.writeOpt(v, $W.String)), $encode(count, (w, v) => w.writeOpt(v, $W.I32)), $encode(outer, (w, v) => w.writeOpt(v, $w$edge_cases$Outer)), $encode(tags, (w, v) => w.writeOpt(v, (w, v) => w.writeList(v, $W.String))), $encode(blob, (w, v) => w.writeOpt(v, $W.Bytes)), $encode(mode, (w, v) => w.writeOpt(v, $W.I32))), (r) => r.readOpt($R.I32)); + return $decode($raw.edge_cases_edge_cases_optional_params($encode(text, $w_opt_string), $encode(count, $w_opt_i32), $encode(outer, $w_opt_Outer), $encode(tags, $w_opt_list_string), $encode(blob, $w_opt_bytes), $encode(mode, $w_opt_Mode)), $r_opt_Mode); } catch ($e) { throw $fault($e); } @@ -461,7 +734,7 @@ const edge_cases$registerBatch = function registerBatch(listener, keyword) { const edge_cases$widgetsEverywhere = function widgetsEverywhere(list, byName, sparse, holder) { try { - return $decode($raw.edge_cases_edge_cases_widgets_everywhere($encode(list, (w, v) => w.writeList(v, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget))))), $encode(byName, (w, v) => w.writeMap(v, $W.String, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget))))), $encode(sparse, (w, v) => w.writeList(v, (w, v) => w.writeOpt(v, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))))), $encode(holder, $w$edge_cases$WidgetHolder)), $r$edge_cases$WidgetHolder); + return $decode($raw.edge_cases_edge_cases_widgets_everywhere($encode(list, $w_list_Widget), $encode(byName, $w_map_string_Widget), $encode(sparse, $w_list_opt_Widget), $encode(holder, $w$edge_cases$WidgetHolder)), $r$edge_cases$WidgetHolder); } catch ($e) { throw $fault($e); } @@ -589,7 +862,7 @@ const edge_cases$asyncStruct = async function asyncStruct() { const edge_cases$asyncList = async function asyncList($options) { try { - return $decode(await $cancellable($options?.signal, $tokens, ($t) => $raw.edge_cases_edge_cases_async_list($t)), (r) => r.readList($r$edge_cases$Outer)); + return $decode(await $cancellable($options?.signal, $tokens, ($t) => $raw.edge_cases_edge_cases_async_list($t)), $r_list_Outer); } catch ($e) { throw $from$edge_cases$EdgeErrors($e); } @@ -597,7 +870,7 @@ const edge_cases$asyncList = async function asyncList($options) { const edge_cases$asyncOptional = async function asyncOptional() { try { - return $decode(await $raw.edge_cases_edge_cases_async_optional(), (r) => r.readOpt($R.I64)); + return $decode(await $raw.edge_cases_edge_cases_async_optional(), $r_opt_i64); } catch ($e) { throw $fault($e); } diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__package_json.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__package_json.snap deleted file mode 100644 index a8924019..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__package_json.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "edge_cases", - "version": "0.1.0", - "description": "Native bindings for edge_cases", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "binding.gyp", - "edge_cases_node.c", - "edge_cases.h" - ], - "gypfile": true, - "scripts": { - "install": "node-gyp rebuild" - }, - "engines": { - "node": ">=18" - } -} diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__runtime_js.snap deleted file mode 100644 index d7fbec01..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class EdgeCasesError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends EdgeCasesError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new EdgeCasesError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new EdgeCasesError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new EdgeCasesError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new EdgeCasesError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__README_md.snap index 4aff8ddc..f44cd897 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__README_md.snap @@ -1,15 +1,15 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # kitchen_sink Native bindings for kitchen_sink -Node.js bindings for the `kitchen_sink` native library: an ES module over a small N-API addon (`kitchen_sink_node.c`) that `npm install` compiles with node-gyp. The addon links `kitchen_sink`; set `KITCHEN_SINK_LIBRARY` to the library file (or its directory) when installing, or pass `--kitchen_sink-library=` to npm. +Node.js bindings for the `kitchen_sink` native library: an ES module over a small N-API addon (`kitchen_sink_node.c`). `npm install` uses a prebuilt addon when one matches the platform (from a per-platform package, or under `prebuilds/-/`) and otherwise compiles it with node-gyp. A compiled addon links `kitchen_sink`; set `KITCHEN_SINK_LIBRARY` to the library file (or its directory) when installing, or pass `--kitchen_sink-library=` to npm. ```sh KITCHEN_SINK_LIBRARY=/path/to/libkitchen_sink.so npm install kitchen_sink diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__binding_gyp.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__binding_gyp.snap index 78061921..0aacaa7e 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__binding_gyp.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__binding_gyp.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate { "variables": { diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_d_ts.snap index edc3eb0e..94b7afa1 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -24,13 +24,13 @@ export declare class CancelledError extends KitchenSinkError { /** The root error class, under a name no namespace member can shadow. */ declare const $Error: typeof KitchenSinkError; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** Trivial cross-module helper */ export declare namespace shared { /** A reusable token referenced across modules */ export interface Token { @@ -41,7 +41,6 @@ export declare namespace shared { export function ping(): string; } -/** Identity for bool */ export declare namespace kitchen { /** The errors of the `kitchen` error domain. */ export class KitchenErrorsError extends $Error { @@ -95,6 +94,15 @@ export declare namespace kitchen { onReady(code: number, msg: string): void; /** Receives the item itself and says whether to keep listening */ onItem(item: kitchen.Item, gadget: kitchen.Gadget): boolean; + /** + * A display label, failing with a kitchen error + * @throws {kitchen.KitchenErrorsError} to report that error, with its fields, to the library + */ + label(): string; + /** The most recent item the listener kept, if any */ + latest(): kitchen.Item | null; + /** The listener's favorite gadget */ + favorite(): kitchen.Gadget; } /** A pocket-sized interface exercising the object surface */ export class Gadget { @@ -128,6 +136,8 @@ export declare namespace kitchen { export function echoBytes(b: Uint8Array): Uint8Array; /** Register a listener, returning how many are attached */ export function subscribe(listener: kitchen.ReadyListener): number; + /** Register a listener if one is given */ + export function maybeSubscribe(listener: kitchen.ReadyListener | null): boolean; /** Optional object in both directions */ export function maybeGadget(fallback: kitchen.Gadget | null): kitchen.Gadget | null; /** A list of objects */ @@ -158,7 +168,6 @@ export declare namespace kitchen { export function legacyOp(): number; /** Replacement for legacy_op */ export function newOp(): number; - /** Trivial nested-module function */ export namespace nested { /** Trivial nested-module function */ export function hello(): string; diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_js.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_js.snap index 78e08876..1fbbf700 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -32,22 +35,30 @@ export { KitchenSinkError, CancelledError } from './runtime.js'; import { createRequire } from 'node:module'; -// The native entry points: the N-API addon `npm install` builds from -// kitchen_sink_node.c (with node-gyp), or a prebuilt copy next to this file. +// The native entry points: the N-API addon built from kitchen_sink_node.c. A prebuilt +// copy comes first (from the `kitchen_sink--` package `npm install` +// picks for this platform, or under prebuilds/-/), then the one +// node-gyp compiles at install time when none matches. function $loadAddon(name) { const require = createRequire(import.meta.url); - const tried = []; - for (const dir of ['./build/Release/', './build/Debug/', './']) { + const platform = `${process.platform}-${process.arch}`; + const candidates = [ + `kitchen_sink-${platform}/${name}`, + `./prebuilds/${platform}/${name}`, + `./build/Release/${name}`, + `./build/Debug/${name}`, + ]; + for (const candidate of candidates) { try { - return require(dir + name); + return require(candidate); } catch (e) { if (e.code !== 'MODULE_NOT_FOUND') throw e; - tried.push(dir + name); } } throw new Error( - `the native addon ${name} was not found (tried ${tried.join(', ')}); ` + - 'build it with `npm install` or `node-gyp rebuild` in this package', + `the native addon ${name} was not found (tried ${candidates.join(', ')}); ` + + 'install the package for this platform, or build the addon with `npm install` ' + + 'or `node-gyp rebuild` in this package', ); } @@ -57,7 +68,49 @@ $raw.$setup($Fault); // Object tokens in value buffers are already bigint handles here. const $token = (t) => t; -$verify($raw, 'kitchen_sink', 'kitchen_sink', 3, [['shared', 0x42c4ce2c8d0af052n], ['kitchen', 0xd83da75b24b545ffn]]); +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['shared', [ + [0x709e234caa962446n, 0x3894f2ca452e9f98n, 'shared.ping'], + [0xcfb306fa5801ee5bn, 0x1cbd9e10983cac2cn, 'shared.Token'], + ]], + ['kitchen', [ + [0x01d4f09bbddc986fn, 0x29eb88fecc66544dn, 'kitchen.Gadget.new'], + [0x10472d8fe748c2een, 0xb265832b6a759fc4n, 'kitchen.maybe_gadget'], + [0x11368eec07a74078n, 0xfa049f78b77e146an, 'kitchen.gadgets'], + [0x137f08a29cde6021n, 0xda8de7aeb2dbb8a3n, 'kitchen.stream_gadgets'], + [0x2111b54980826dd7n, 0x452783f3e6ca6e1dn, 'kitchen.do_cancellable'], + [0x29dddbc60bf8d7dbn, 0x452a3e573d57f6dbn, 'kitchen.Priority'], + [0x2b227003b74a0d16n, 0x12332957470e9782n, 'kitchen.maybe_subscribe'], + [0x2dd66cb0064f0494n, 0x60197363fefd92ddn, 'kitchen.Item'], + [0x33c265bbf079b0e9n, 0xc3dcd0927e6c1debn, 'kitchen.u32_id'], + [0x33e6381d01660dfen, 0x077c63c31fd7f90an, 'kitchen.KitchenErrors'], + [0x3f5fc99d73504b76n, 0x2df88d710f1ea10en, 'kitchen.Gadget.poke'], + [0x4c5764cf109528d1n, 0x2a65529af4338a58n, 'kitchen.maybe_item'], + [0x4de17dfb93fd494dn, 0xba0ad0dd52ed74afn, 'kitchen.i32_id'], + [0x504e95b7f7e65570n, 0x01b3e8e0c9a10e23n, 'kitchen.stream_items'], + [0x68cdb9a4bd3e3dd5n, 0xc678b9164348d497n, 'kitchen.bool_id'], + [0x69e7cd703d16eb85n, 0xf3b28dd22ae9e689n, 'kitchen.do_async'], + [0x70865079ce41be82n, 0xbe3b69464672af46n, 'kitchen.Gadget.describe'], + [0x71c5a1116cb7c49an, 0xcba1837108a89b84n, 'kitchen.echo_bytes'], + [0x735ee1e0049ee91an, 0xca74320bc53eddabn, 'kitchen.echo_string'], + [0x76d7fc58f4e41d84n, 0x7373a0e47d30e690n, 'kitchen.i64_id'], + [0x79abfe1aa960fd6an, 0xbca7a9e5eb26f3a6n, 'kitchen.nested.hello'], + [0x7d3dd7d5cf6749e3n, 0xa192f5a7298adb39n, 'kitchen.new_op'], + [0x89a5e664bc25c67bn, 0x5764a95e81ecfd31n, 'kitchen.f64_id'], + [0x8cfdbdcecac5f7f1n, 0x07e272f09c41fffcn, 'kitchen.Gadget.version'], + [0x974b9f3e823c1738n, 0xb81b91e975695f27n, 'kitchen.cross_module_token'], + [0xaca0655fac15d230n, 0x11c4d2b28abe07bbn, 'kitchen.list_items'], + [0xc883b4afae20543en, 0xc40560e5411ff392n, 'kitchen.legacy_op'], + [0xf2f7753e6e1dffc2n, 0xc9047a95bdf49e93n, 'kitchen.get_attrs'], + [0xfae25d3391922eadn, 0x1c7f95739f3210b8n, 'kitchen.subscribe'], + [0xfd0059b05cb2fcfbn, 0x248bf8404a5823b4n, 'kitchen.Gadget'], + [0xfed9c551fb9fd2d6n, 0x59aec980ef3f3291n, 'kitchen.ReadyListener'], + ]], +]; +$verify($raw, 'kitchen_sink', 'kitchen_sink', 4, $contract); // The native cancel tokens behind `AbortSignal`s. const $tokens = { @@ -67,9 +120,10 @@ const $tokens = { }; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.kitchen_sink_debug_live(kind); @@ -88,9 +142,17 @@ const kitchen$InvalidInputError = class InvalidInputError extends kitchen$Kitche super(2, message); } }; -const $codes$kitchen$KitchenErrors = new Map([[1, kitchen$NotFoundError], [2, kitchen$InvalidInputError]]); +const $codes$kitchen$KitchenErrors = new Map([ + [1, kitchen$NotFoundError], + [2, kitchen$InvalidInputError], +]); +const $payloads$kitchen$KitchenErrors = new Map(); function $from$kitchen$KitchenErrors(e) { - return $domain(e, $codes$kitchen$KitchenErrors); + return $domain(e, $codes$kitchen$KitchenErrors, $payloads$kitchen$KitchenErrors); +} +const $fields$kitchen$KitchenErrors = new Map(); +function $raise$kitchen$KitchenErrors(e) { + return $raise(e, kitchen$KitchenErrorsError, $codes$kitchen$KitchenErrors, $fields$kitchen$KitchenErrors); } const kitchen$Priority = Object.freeze({ @@ -120,12 +182,12 @@ function $w$kitchen$Item(w, v) { w.writeF64(v.ratio); w.writeU32(v.nick); w.writeBytes(v.payload); - w.writeList(v.tags, $W.String); - w.writeMap(v.attrs, $W.String, $W.String); - w.writeOpt(v.parent, $W.I64); + $w_list_string(w, v.tags); + $w_map_string_string(w, v.attrs); + $w_opt_i64(w, v.parent); $w$shared$Token(w, v.token); w.writeI32(v.priority); - w.writeOpt(v.gadget, (w, v) => w.writeU64(BigInt($clone(v, kitchen$Gadget)))); + $w_opt_Gadget(w, v.gadget); } function $r$kitchen$Item(r) { return { @@ -136,14 +198,76 @@ function $r$kitchen$Item(r) { ratio: r.readF64(), nick: r.readU32(), payload: r.readBytes(), - tags: r.readList($R.String), - attrs: r.readMap($R.String, $R.String), - parent: r.readOpt($R.I64), + tags: $r_list_string(r), + attrs: $r_map_string_string(r), + parent: $r_opt_i64(r), token: $r$shared$Token(r), priority: r.readI32(), - gadget: r.readOpt((r) => $adopt(kitchen$Gadget, $token(r.readU64()))), + gadget: $r_opt_Gadget(r), }; } +function $w$kitchen$Gadget(w, v) { + w.writeU64(BigInt($clone(v, kitchen$Gadget))); +} +function $r$kitchen$Gadget(r) { + return $adopt(kitchen$Gadget, $token(r.readU64())); +} +/** `[Gadget]` */ +function $w_list_Gadget(w, v) { + w.writeList(v, $w$kitchen$Gadget); +} +function $r_list_Gadget(r) { + return r.readList($r$kitchen$Gadget); +} +/** `[Item]` */ +function $w_list_Item(w, v) { + w.writeList(v, $w$kitchen$Item); +} +function $r_list_Item(r) { + return r.readList($r$kitchen$Item); +} +/** `[string]` */ +function $w_list_string(w, v) { + w.writeList(v, $W.String); +} +function $r_list_string(r) { + return r.readList($R.String); +} +/** `{string:i32}` */ +function $w_map_string_i32(w, v) { + w.writeMap(v, $WK.String, $W.I32); +} +function $r_map_string_i32(r) { + return r.readMap($R.String, $R.I32); +} +/** `{string:string}` */ +function $w_map_string_string(w, v) { + w.writeMap(v, $WK.String, $W.String); +} +function $r_map_string_string(r) { + return r.readMap($R.String, $R.String); +} +/** `Gadget?` */ +function $w_opt_Gadget(w, v) { + w.writeOpt(v, $w$kitchen$Gadget); +} +function $r_opt_Gadget(r) { + return r.readOpt($r$kitchen$Gadget); +} +/** `Item?` */ +function $w_opt_Item(w, v) { + w.writeOpt(v, $w$kitchen$Item); +} +function $r_opt_Item(r) { + return r.readOpt($r$kitchen$Item); +} +/** `i64?` */ +function $w_opt_i64(w, v) { + w.writeOpt(v, $W.I64); +} +function $r_opt_i64(r) { + return r.readOpt($R.I64); +} const shared$ping = function ping() { try { @@ -162,6 +286,19 @@ function $adapt$kitchen$ReadyListener(impl) { on_item($a0, $a1) { return $ret.Bool(impl.onItem($decode($a0, $r$kitchen$Item), $adopt(kitchen$Gadget, $a1)), 'ReadyListener.onItem'); }, + label() { + try { + return $ret.String(impl.label(), 'ReadyListener.label'); + } catch ($e) { + throw $raise$kitchen$KitchenErrors($e); + } + }, + latest() { + return $encode(impl.latest(), $w_opt_Item); + }, + favorite() { + return $clone(impl.favorite(), kitchen$Gadget); + }, }; } @@ -273,6 +410,14 @@ const kitchen$subscribe = function subscribe(listener) { } }; +const kitchen$maybeSubscribe = function maybeSubscribe(listener) { + try { + return $raw.kitchen_sink_kitchen_maybe_subscribe(listener == null ? null : $adapt$kitchen$ReadyListener(listener)); + } catch ($e) { + throw $fault($e); + } +}; + const kitchen$maybeGadget = function maybeGadget(fallback) { const $o0 = $lendOpt(fallback, kitchen$Gadget); try { @@ -286,7 +431,7 @@ const kitchen$maybeGadget = function maybeGadget(fallback) { const kitchen$gadgets = function gadgets() { try { - return $decode($raw.kitchen_sink_kitchen_gadgets(), (r) => r.readList((r) => $adopt(kitchen$Gadget, $token(r.readU64())))); + return $decode($raw.kitchen_sink_kitchen_gadgets(), $r_list_Gadget); } catch ($e) { throw $fault($e); } @@ -308,7 +453,7 @@ const kitchen$streamGadgets = function streamGadgets() { const kitchen$maybeItem = function maybeItem(id) { try { - return $decode($raw.kitchen_sink_kitchen_maybe_item(id), (r) => r.readOpt($r$kitchen$Item)); + return $decode($raw.kitchen_sink_kitchen_maybe_item(id), $r_opt_Item); } catch ($e) { throw $from$kitchen$KitchenErrors($e); } @@ -316,7 +461,7 @@ const kitchen$maybeItem = function maybeItem(id) { const kitchen$listItems = function listItems() { try { - return $decode($raw.kitchen_sink_kitchen_list_items(), (r) => r.readList($r$kitchen$Item)); + return $decode($raw.kitchen_sink_kitchen_list_items(), $r_list_Item); } catch ($e) { throw $fault($e); } @@ -324,7 +469,7 @@ const kitchen$listItems = function listItems() { const kitchen$getAttrs = function getAttrs() { try { - return $decode($raw.kitchen_sink_kitchen_get_attrs(), (r) => r.readMap($R.String, $R.I32)); + return $decode($raw.kitchen_sink_kitchen_get_attrs(), $r_map_string_i32); } catch ($e) { throw $fault($e); } @@ -410,6 +555,7 @@ export const kitchen = Object.freeze({ echoString: kitchen$echoString, echoBytes: kitchen$echoBytes, subscribe: kitchen$subscribe, + maybeSubscribe: kitchen$maybeSubscribe, maybeGadget: kitchen$maybeGadget, gadgets: kitchen$gadgets, streamGadgets: kitchen$streamGadgets, diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_h.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_h.snap deleted file mode 100644 index 14faeffb..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_h.snap +++ /dev/null @@ -1,219 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef KITCHEN_SINK_H -#define KITCHEN_SINK_H - -#include -#include -#include - -#ifndef KITCHEN_SINK_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef KITCHEN_SINK_BUILD -# define KITCHEN_SINK_API __declspec(dllexport) -# else -# define KITCHEN_SINK_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define KITCHEN_SINK_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define KITCHEN_SINK_API __attribute__((visibility("default"))) -# else -# define KITCHEN_SINK_API -# endif -#endif - -#ifndef KITCHEN_SINK_DEPRECATED -# if defined(_MSC_VER) -# define KITCHEN_SINK_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define KITCHEN_SINK_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define KITCHEN_SINK_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports kitchen_sink_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define KITCHEN_SINK_ABI_VERSION 3u -KITCHEN_SINK_API uint32_t kitchen_sink_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by kitchen_sink_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct kitchen_sink_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} kitchen_sink_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -KITCHEN_SINK_API void kitchen_sink_error_set(kitchen_sink_error* err, int32_t code, const char* message); -KITCHEN_SINK_API void kitchen_sink_error_clear(kitchen_sink_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - kitchen_sink_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_error_free(kitchen_sink_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -KITCHEN_SINK_API void kitchen_sink_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct kitchen_sink_cancel_token kitchen_sink_cancel_token; -KITCHEN_SINK_API kitchen_sink_cancel_token* kitchen_sink_cancel_token_create(void); -KITCHEN_SINK_API void kitchen_sink_cancel_token_cancel(kitchen_sink_cancel_token* token); -KITCHEN_SINK_API bool kitchen_sink_cancel_token_is_cancelled(const kitchen_sink_cancel_token* token); -KITCHEN_SINK_API void kitchen_sink_cancel_token_destroy(kitchen_sink_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -KITCHEN_SINK_API uint8_t* kitchen_sink_alloc(uint32_t size); -KITCHEN_SINK_API void kitchen_sink_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -KITCHEN_SINK_API uint64_t kitchen_sink_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define KITCHEN_SINK_SHARED_CHECKSUM 0x42c4ce2c8d0af052ull -KITCHEN_SINK_API uint64_t kitchen_sink_shared_checksum(void); -#define KITCHEN_SINK_KITCHEN_CHECKSUM 0xd83da75b24b545ffull -KITCHEN_SINK_API uint64_t kitchen_sink_kitchen_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with kitchen_sink_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * kitchen_sink_buffer.h declares a C struct and codec for each. - */ - -/** Task priority levels */ -typedef enum { - /** Low priority */ - kitchen_sink_kitchen_Priority_Low = 0, - /** Normal priority */ - kitchen_sink_kitchen_Priority_Normal = 1, - /** High priority */ - kitchen_sink_kitchen_Priority_High = 2 -} kitchen_sink_kitchen_Priority; -/** Error codes reported by throwing functions in the `kitchen` module tree. */ -typedef enum { kitchen_sink_kitchen_KitchenErrors_NotFound = 1, kitchen_sink_kitchen_KitchenErrors_InvalidInput = 2 } kitchen_sink_kitchen_KitchenErrors; -typedef struct kitchen_sink_kitchen_Gadget kitchen_sink_kitchen_Gadget; -typedef struct kitchen_sink_kitchen_StreamGadgetsIterator kitchen_sink_kitchen_StreamGadgetsIterator; -typedef struct kitchen_sink_kitchen_StreamItemsIterator kitchen_sink_kitchen_StreamItemsIterator; -/** - * Consumer-implemented observer of ready items - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct kitchen_sink_kitchen_ReadyListener_vtable { - /** Fires when an item is ready */ - void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); - /** Receives the item itself and says whether to keep listening */ - bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); -} kitchen_sink_kitchen_ReadyListener_vtable; -typedef void (*kitchen_sink_kitchen_do_async_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*kitchen_sink_kitchen_do_cancellable_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: shared -/** Trivial cross-module helper */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_shared_ping(size_t* out_len, kitchen_sink_error* out_err); - -// Module: kitchen -/** A pocket-sized interface exercising the object surface */ -/** Create a gadget with the given id */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_new(int64_t id, kitchen_sink_error* out_err); -/** The gadget subsystem version string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_version(size_t* out_len, kitchen_sink_error* out_err); -/** Render the gadget as a human-readable string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_describe(const kitchen_sink_kitchen_Gadget* self, size_t* out_len, kitchen_sink_error* out_err); -/** Poke the gadget a number of times, failing on a negative count */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_Gadget_poke(const kitchen_sink_kitchen_Gadget* self, int32_t times, kitchen_sink_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_clone(const kitchen_sink_kitchen_Gadget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_kitchen_Gadget_destroy(kitchen_sink_kitchen_Gadget* self); - -/** Identity for bool */ -KITCHEN_SINK_API bool kitchen_sink_kitchen_bool_id(bool x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_i32_id(int32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API uint32_t kitchen_sink_kitchen_u32_id(uint32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int64_t kitchen_sink_kitchen_i64_id(int64_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API double kitchen_sink_kitchen_f64_id(double x, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_string(const uint8_t* s_ptr, size_t s_len, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_bytes(const uint8_t* b_ptr, size_t b_len, size_t* out_len, kitchen_sink_error* out_err); -/** Register a listener, returning how many are attached */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); -/** Optional object in both directions */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_maybe_gadget(const kitchen_sink_kitchen_Gadget* fallback, kitchen_sink_error* out_err); -/** A list of objects */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_gadgets(size_t* out_len, kitchen_sink_error* out_err); -/** An iterator over objects */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamGadgetsIterator* kitchen_sink_kitchen_stream_gadgets(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamGadgetsIterator_next(kitchen_sink_kitchen_StreamGadgetsIterator* iter, kitchen_sink_kitchen_Gadget** out_item, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamGadgetsIterator_destroy(kitchen_sink_kitchen_StreamGadgetsIterator* iter); -/** Optional struct return */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_maybe_item(int64_t id, size_t* out_len, kitchen_sink_error* out_err); -/** List of structs */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_list_items(size_t* out_len, kitchen_sink_error* out_err); -/** Map return type */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_get_attrs(size_t* out_len, kitchen_sink_error* out_err); -/** Iterator return type */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamItemsIterator* kitchen_sink_kitchen_stream_items(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamItemsIterator_next(kitchen_sink_kitchen_StreamItemsIterator* iter, const uint8_t** out_item, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamItemsIterator_destroy(kitchen_sink_kitchen_StreamItemsIterator* iter); -/** Returns the shared Token type from another module */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_cross_module_token(size_t* out_len, kitchen_sink_error* out_err); -/** Async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_async(const uint8_t* input_ptr, size_t input_len, kitchen_sink_kitchen_do_async_callback callback, void* context); -/** Cancellable async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_cancellable(const uint8_t* input_ptr, size_t input_len, kitchen_sink_cancel_token* cancel_token, kitchen_sink_kitchen_do_cancellable_callback callback, void* context); -/** Legacy operation kept for compatibility */ -KITCHEN_SINK_DEPRECATED("Use new_op instead") -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_legacy_op(kitchen_sink_error* out_err); -/** Replacement for legacy_op */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_new_op(kitchen_sink_error* out_err); - -// Module: kitchen_nested -/** Trivial nested-module function */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_nested_hello(size_t* out_len, kitchen_sink_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // KITCHEN_SINK_H - -// END kitchen_sink.h diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap index 1fe6f418..80c9012c 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef NAPI_VERSION #define NAPI_VERSION 8 @@ -12,6 +12,7 @@ source: crates/weaveffi-cli/tests/snapshots.rs #include #include #include +#include #include #include @@ -26,8 +27,9 @@ source: crates/weaveffi-cli/tests/snapshots.rs * raw convention the shared JavaScript layer documents: direct values as * numbers, bigints, and booleans; strings as strings; bytes and value * buffers as Uint8Arrays; objects, iterators, and cancel tokens as bigint - * handles (null when absent); callback interfaces as adapter objects. A - * failure throws an instance of the runtime's `$Fault` class. + * handles (null when absent); callback interfaces as adapter objects (null + * for an absent optional one). A failure throws an instance of the + * runtime's `$Fault` class. */ typedef kitchen_sink_error js_error; @@ -35,23 +37,106 @@ typedef kitchen_sink_error js_error; /* Helpers an API may not need. */ #if defined(__GNUC__) || defined(__clang__) #define JS_HELPER static inline __attribute__((unused)) +#define JS_TLS __thread #else #define JS_HELPER static inline +#define JS_TLS __declspec(thread) #endif +/* --------------------------------------------------------------------------- + * JavaScript threads. One `js_thread` per thread that runs JavaScript and + * loaded the addon (the main thread and each worker). It counts the + * synchronous calls into the library in progress on that thread, which a + * callback made on another thread consults while it waits for the + * JavaScript thread (see js_cb_hop). Each environment and each callback + * registration holds a reference. + * ------------------------------------------------------------------------- */ + +typedef struct { + uv_mutex_t mu; + uv_cond_t cv; /* broadcast when a call begins or ends, or a hop finishes */ + unsigned depth; /* synchronous calls into the library in progress */ + uint64_t calls; /* outermost synchronous calls begun so far */ + unsigned waiters; /* producer threads waiting in js_cb_hop */ + unsigned refs; +} js_thread; + +/* The state of the JavaScript thread this is, or NULL on any other thread. */ +static JS_TLS js_thread* js_current = NULL; + +static js_thread* js_thread_new(void) { + js_thread* t = (js_thread*)calloc(1, sizeof *t); + uv_mutex_init(&t->mu); + uv_cond_init(&t->cv); + t->refs = 1; + return t; +} + +static void js_thread_retain(js_thread* t) { + uv_mutex_lock(&t->mu); + t->refs++; + uv_mutex_unlock(&t->mu); +} + +static void js_thread_release(js_thread* t) { + uv_mutex_lock(&t->mu); + bool last = --t->refs == 0; + uv_mutex_unlock(&t->mu); + if (!last) return; + uv_cond_destroy(&t->cv); + uv_mutex_destroy(&t->mu); + free(t); +} + +/* Bracket every synchronous call into the library. */ +JS_HELPER void js_sync_begin(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + if (t->depth++ == 0) t->calls++; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +JS_HELPER void js_sync_end(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + t->depth--; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + /* Per-environment state (one per main thread or worker): the `$Fault` - * constructor `index.js` registers through `$setup`. */ + * constructor `index.js` registers through `$setup`, and the thread. */ typedef struct { napi_ref fault; + js_thread* thread; } js_env; static void js_env_free(napi_env env, void* data, void* hint) { (void)hint; js_env* e = (js_env*)data; if (e->fault != NULL) napi_delete_reference(env, e->fault); + if (js_current == e->thread) js_current = NULL; + js_thread_release(e->thread); free(e); } +/* Set up the environment of the thread loading the addon. */ +static void js_env_init(napi_env env) { + js_env* e = (js_env*)calloc(1, sizeof *e); + e->thread = js_thread_new(); + js_current = e->thread; + napi_set_instance_data(env, e, js_env_free, NULL); +} + +static js_env* js_env_of(napi_env env) { + js_env* e = NULL; + napi_get_instance_data(env, (void**)&e); + return e; +} + JS_HELPER napi_value js_undefined(napi_env env) { napi_value v; napi_get_undefined(env, &v); @@ -62,13 +147,22 @@ static napi_value js_setup(napi_env env, napi_callback_info info) { size_t argc = 1; napi_value argv[1]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); + js_env* e = js_env_of(env); if (e->fault != NULL) napi_delete_reference(env, e->fault); napi_create_reference(env, argv[0], 1, &e->fault); return js_undefined(env); } +/* The `$Fault` constructor, or NULL before `$setup`. */ +static napi_value js_fault_class(napi_env env) { + js_env* e = js_env_of(env); + napi_value ctor = NULL; + if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok) { + return NULL; + } + return ctor; +} + /* --------------------------------------------------------------------------- * Value conversions * ------------------------------------------------------------------------- */ @@ -149,6 +243,21 @@ JS_HELPER napi_value js_take_bytes(napi_env env, const uint8_t* ptr, size_t len) return out; } +/* A module's contract table as a BigUint64Array of `id, hash` pairs. */ +JS_HELPER napi_value js_new_contract(napi_env env, const kitchen_sink_contract_entry* table, + size_t len) { + napi_value ab, out; + void* data = NULL; + napi_create_arraybuffer(env, len * 2 * sizeof(uint64_t), &data, &ab); + uint64_t* words = (uint64_t*)data; + for (size_t i = 0; i < len; i++) { + words[2 * i] = table[i].id; + words[2 * i + 1] = table[i].hash; + } + napi_create_typedarray(env, napi_biguint64_array, len * 2, ab, 0, &out); + return out; +} + JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, const uint8_t* payload, size_t payload_len) { napi_value argv[3], ctor, out = NULL; @@ -159,10 +268,8 @@ JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, } else { napi_get_null(env, &argv[2]); } - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); - if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok || - ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { + ctor = js_fault_class(env); + if (ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { napi_create_error(env, NULL, argv[1], &out); napi_set_named_property(env, out, "code", argv[0]); } @@ -428,30 +535,42 @@ JS_HELPER void js_async_done(js_async* a, js_error* err) { * producer: a reference to the adapter object, the JavaScript thread it * belongs to, and a thread-safe function that runs calls made from any other * thread on that thread while the caller waits. + * + * Waiting is what makes a call from another thread safe, and also what could + * deadlock it: if the JavaScript thread is itself inside a synchronous call + * into the library that waits for the calling thread, neither can proceed. + * A waiting call therefore watches the JavaScript thread. While that thread + * stays inside one synchronous call for JS_CB_DEADLOCK_MS without having + * started the callback, the wait is abandoned and the callback fails with + * code -4 instead of hanging (a shorter synchronous call, or one that + * returns to the event loop, just delays the callback). * ------------------------------------------------------------------------- */ +#define JS_CB_DEADLOCK_MS 1000 + typedef struct { napi_env env; napi_ref ref; napi_threadsafe_function tsfn; uv_thread_t js_thread; + js_thread* thread; } js_cb; -/* Every callback frame starts with this header so the dispatcher can reach - * `out_err` without knowing the method. */ -typedef struct { - js_error* out_err; -} js_cb_frame; +typedef enum { JS_REQ_QUEUED, JS_REQ_RUNNING, JS_REQ_DONE, JS_REQ_ABANDONED } js_req_state; /* A call queued from another thread: `method` is the vtable index, or -1 to - * release the registration. */ + * release the registration; `frame` holds the method's slots. The request is + * heap-allocated and holds its own reference to the thread state, so a + * caller that abandons it can return (and the producer can then release the + * registration) while it is still queued: the dispatcher frees it without + * touching the frame or the registration. */ typedef struct { js_cb* cb; + js_thread* thread; int method; void* frame; - uv_mutex_t mu; - uv_cond_t cv; - bool done; + js_error* out_err; + js_req_state state; } js_cb_req; JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { @@ -460,10 +579,12 @@ JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { } JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* name, - napi_threadsafe_function_call_js dispatch) { + napi_threadsafe_function_call_js dispatch) { js_cb* cb = (js_cb*)calloc(1, sizeof *cb); cb->env = env; cb->js_thread = uv_thread_self(); + cb->thread = js_env_of(env)->thread; + js_thread_retain(cb->thread); napi_create_reference(env, adapter, 1, &cb->ref); napi_value resource; napi_create_string_utf8(env, name, NAPI_AUTO_LENGTH, &resource); @@ -477,14 +598,20 @@ JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* na JS_HELPER void js_cb_release(napi_env env, js_cb* cb) { if (env != NULL) napi_delete_reference(env, cb->ref); napi_release_threadsafe_function(cb->tsfn, napi_tsfn_release); + js_thread_release(cb->thread); free(cb); } -/* A callback-interface argument: the adapter object `index.js` built. */ +/* A callback-interface argument: the adapter object `index.js` built, or + * null for an absent optional one. */ JS_HELPER bool js_arg_cb(napi_env env, napi_value v, const char* name, - napi_threadsafe_function_call_js dispatch, js_cb** out) { + napi_threadsafe_function_call_js dispatch, bool nullable, js_cb** out) { napi_valuetype t; napi_typeof(env, v, &t); + if (nullable && (t == napi_null || t == napi_undefined)) { + *out = NULL; + return true; + } if (t != napi_object) return js_type_error(env, "expected a callback interface implementation"); *out = js_cb_register(env, v, name, dispatch); return true; @@ -505,62 +632,188 @@ JS_HELPER void js_cb_free(void* ctx) { if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) free(req); } -/* Run `req` on the JavaScript thread and wait for it. */ -JS_HELPER void js_cb_hop(js_cb_req* req) { - uv_mutex_init(&req->mu); - uv_cond_init(&req->cv); - req->done = false; - if (napi_call_threadsafe_function(req->cb->tsfn, req, napi_tsfn_blocking) == napi_ok) { - uv_mutex_lock(&req->mu); - while (!req->done) uv_cond_wait(&req->cv, &req->mu); - uv_mutex_unlock(&req->mu); - } else { - kitchen_sink_error_set(((js_cb_frame*)req->frame)->out_err, -4, - "the callback implementation is no longer reachable"); +/* Run method `method` of `cb` on its JavaScript thread and wait for it, + * failing it with code -4 rather than deadlocking (see above). `what` names + * the method in that failure. */ +JS_HELPER void js_cb_hop(js_cb* cb, int method, void* frame, js_error* out_err, + const char* what) { + js_thread* t = cb->thread; + js_cb_req* req = (js_cb_req*)calloc(1, sizeof *req); + req->cb = cb; + req->thread = t; + js_thread_retain(t); + req->method = method; + req->frame = frame; + req->out_err = out_err; + req->state = JS_REQ_QUEUED; + uv_mutex_lock(&t->mu); + t->waiters++; + uv_mutex_unlock(&t->mu); + if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) { + uv_mutex_lock(&t->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); + kitchen_sink_error_set(out_err, -4, "the callback implementation is no longer reachable"); + return; + } + const uint64_t limit = (uint64_t)JS_CB_DEADLOCK_MS * 1000000u; + bool blocked = false; + uint64_t call = 0, since = 0; + uv_mutex_lock(&t->mu); + while (req->state != JS_REQ_DONE) { + if (req->state == JS_REQ_QUEUED && t->depth > 0) { + uint64_t now = uv_hrtime(); + if (!blocked || call != t->calls) { + blocked = true; + call = t->calls; + since = now; + } + if (now - since >= limit) { + req->state = JS_REQ_ABANDONED; /* the dispatcher frees it */ + t->waiters--; + uv_mutex_unlock(&t->mu); + char msg[512]; + snprintf(msg, sizeof msg, + "%s was called on another thread while the JavaScript thread was blocked in a " + "synchronous call into the library for %d ms; the call likely waits for this " + "callback, so waiting longer would deadlock (call it asynchronously, or call " + "back on the calling thread)", + what, JS_CB_DEADLOCK_MS); + kitchen_sink_error_set(out_err, -4, msg); + return; + } + uv_cond_timedwait(&t->cv, &t->mu, limit - (now - since)); + } else { + blocked = false; + uv_cond_wait(&t->cv, &t->mu); + } + } + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); +} + +/* The dispatcher's side of a hop. js_cb_take claims a queued request (false + * when its caller gave up, in which case the request is freed); + * js_cb_finish reports it done. */ +JS_HELPER bool js_cb_take(js_cb_req* req) { + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + bool abandoned = req->state == JS_REQ_ABANDONED; + if (!abandoned) req->state = JS_REQ_RUNNING; + uv_mutex_unlock(&t->mu); + if (abandoned) { + free(req); + js_thread_release(t); } - uv_cond_destroy(&req->cv); - uv_mutex_destroy(&req->mu); + return !abandoned; } JS_HELPER void js_cb_finish(js_cb_req* req) { - uv_mutex_lock(&req->mu); - req->done = true; - uv_cond_signal(&req->cv); - uv_mutex_unlock(&req->mu); + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + req->state = JS_REQ_DONE; + uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); } -/* Report the pending JavaScript exception (or `fallback`) through `out_err` - * as a foreign failure (code -4); nothing unwinds through the C frame. */ +/* A JavaScript value's text as a heap-allocated C string (NULL on failure). */ +JS_HELPER char* js_cstring(napi_env env, napi_value v) { + napi_value str; + size_t len = 0; + if (napi_coerce_to_string(env, v, &str) != napi_ok || + napi_get_value_string_utf8(env, str, NULL, 0, &len) != napi_ok) { + return NULL; + } + char* out = (char*)malloc(len + 1); + napi_get_value_string_utf8(env, str, out, len + 1, NULL); + return out; +} + +/* Report the pending JavaScript exception (or `fallback`) through + * `out_err`; nothing unwinds through the C frame. A `$Fault` (a domain error + * a throwing method raised) reports its code, message, and payload; any + * other exception is a foreign failure, code -4, with its message. */ JS_HELPER void js_cb_report(napi_env env, js_error* out_err, const char* fallback) { - char msg[512]; - msg[0] = 0; + int32_t code = -4; + char* msg = NULL; + const uint8_t* payload = NULL; + size_t payload_len = 0; bool pending = false; napi_is_exception_pending(env, &pending); if (pending) { - napi_value exc, text, str; + napi_value exc, text; napi_get_and_clear_last_exception(env, &exc); + napi_value fault = js_fault_class(env); + bool is_fault = false; + if (fault != NULL) napi_instanceof(env, exc, fault, &is_fault); napi_valuetype t; napi_typeof(env, exc, &t); text = exc; if (t == napi_object) napi_get_named_property(env, exc, "message", &text); - if (napi_coerce_to_string(env, text, &str) == napi_ok) { - napi_get_value_string_utf8(env, str, msg, sizeof msg, NULL); + msg = js_cstring(env, text); + if (is_fault) { + napi_value v; + napi_get_named_property(env, exc, "code", &v); + napi_get_value_int32(env, v, &code); + napi_get_named_property(env, exc, "payload", &v); + bool typed = false; + napi_is_typedarray(env, v, &typed); + if (typed) { + napi_typedarray_type type; + void* data = NULL; + napi_get_typedarray_info(env, v, &type, &payload_len, &data, NULL, NULL); + payload = (const uint8_t*)data; + } } napi_is_exception_pending(env, &pending); if (pending) napi_get_and_clear_last_exception(env, &exc); } - kitchen_sink_error_set(out_err, -4, msg[0] != 0 ? msg : fallback); + kitchen_sink_error_set(out_err, code, msg != NULL && msg[0] != 0 ? msg : fallback); + if (payload != NULL) kitchen_sink_error_set_payload(out_err, payload, payload_len); + free(msg); } /* Look up and call `method` on the adapter; false when it threw. */ JS_HELPER bool js_cb_call(napi_env env, js_cb* cb, const char* method, size_t argc, - const napi_value* argv, napi_value* result) { + const napi_value* argv, napi_value* result) { napi_value adapter, fn; napi_get_reference_value(env, cb->ref, &adapter); napi_get_named_property(env, adapter, method, &fn); return napi_call_function(env, adapter, fn, argc, argv, result) == napi_ok; } +/* Hand `len` bytes to the producer through a callback's out slots, as a run + * from {p}_alloc that the producer adopts. */ +JS_HELPER void js_give(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = kitchen_sink_alloc(len); + if (len > 0) memcpy(run, data, len); + *out_ptr = run; + *out_len = len; +} + +/* A callback's string return: false (with a pending exception) when the + * value isn't a string. */ +JS_HELPER bool js_ret_str(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + js_str s = JS_STR_INIT; + if (!js_arg_str(env, v, &s)) return false; + js_give(s.ptr, s.len, out_ptr, out_len); + js_str_free(&s); + return true; +} + +/* A callback's bytes or buffer return: a Uint8Array. */ +JS_HELPER bool js_ret_bytes(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + const uint8_t* ptr = NULL; + size_t len = 0; + if (!js_arg_bytes(env, v, &ptr, &len)) return false; + js_give(ptr, len, out_ptr, out_len); + return true; +} + /* --------------------------------------------------------------------------- * The runtime symbols every library exports. * ------------------------------------------------------------------------- */ @@ -609,18 +862,22 @@ static napi_value js_cancel_token_destroy(napi_env env, napi_callback_info info) {"kitchen_sink_cancel_token_destroy", NULL, js_cancel_token_destroy, NULL, NULL, NULL, \ napi_default, NULL} -static napi_value nx_kitchen_sink_shared_checksum(napi_env env, napi_callback_info info) { +static napi_value nx_kitchen_sink_shared_contract(napi_env env, napi_callback_info info) { (void)info; - return js_new_u64(env, kitchen_sink_shared_checksum()); + size_t len = 0; + const kitchen_sink_contract_entry* table = kitchen_sink_shared_contract(&len); + return js_new_contract(env, table, len); } -static napi_value nx_kitchen_sink_kitchen_checksum(napi_env env, napi_callback_info info) { +static napi_value nx_kitchen_sink_kitchen_contract(napi_env env, napi_callback_info info) { (void)info; - return js_new_u64(env, kitchen_sink_kitchen_checksum()); + size_t len = 0; + const kitchen_sink_contract_entry* table = kitchen_sink_kitchen_contract(&len); + return js_new_contract(env, table, len); } typedef struct { - js_cb_frame hdr; + js_error* out_err; int32_t p0; const uint8_t* p1; size_t p2; @@ -634,7 +891,7 @@ static void invoke_kitchen_sink_kitchen_ReadyListener_on_ready(napi_env env, js_ argv[1] = js_new_str(env, f->p1, f->p2); napi_value result; if (!js_cb_call(env, cb, "on_ready", 2, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "ReadyListener.on_ready failed"); + js_cb_report(env, f->out_err, "ReadyListener.onReady failed"); } napi_close_handle_scope(env, scope); } @@ -642,7 +899,7 @@ static void invoke_kitchen_sink_kitchen_ReadyListener_on_ready(napi_env env, js_ static void tramp_kitchen_sink_kitchen_ReadyListener_on_ready(void* ctx, int32_t p0, const uint8_t* p1, size_t p2, kitchen_sink_error* out_err) { frame_kitchen_sink_kitchen_ReadyListener_on_ready f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; f.p0 = p0; f.p1 = p1; f.p2 = p2; @@ -650,17 +907,12 @@ static void tramp_kitchen_sink_kitchen_ReadyListener_on_ready(void* ctx, int32_t if (js_cb_on_js_thread(cb)) { invoke_kitchen_sink_kitchen_ReadyListener_on_ready(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 0; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 0, &f, out_err, "ReadyListener.onReady"); } } typedef struct { - js_cb_frame hdr; + js_error* out_err; const uint8_t* p0; size_t p1; kitchen_sink_kitchen_Gadget* p2; @@ -675,13 +927,13 @@ static void invoke_kitchen_sink_kitchen_ReadyListener_on_item(napi_env env, js_c argv[1] = js_new_handle(env, f->p2); napi_value result; if (!js_cb_call(env, cb, "on_item", 2, argv, &result)) { - js_cb_report(env, f->hdr.out_err, "ReadyListener.on_item failed"); + js_cb_report(env, f->out_err, "ReadyListener.onItem failed"); } else { bool v = 0; if (js_arg_bool(env, result, &v)) { f->result = (bool)v; } else { - js_cb_report(env, f->hdr.out_err, "ReadyListener.on_item returned a value of the wrong type"); + js_cb_report(env, f->out_err, "ReadyListener.onItem returned a value of the wrong type"); } } napi_close_handle_scope(env, scope); @@ -690,7 +942,7 @@ static void invoke_kitchen_sink_kitchen_ReadyListener_on_item(napi_env env, js_c static bool tramp_kitchen_sink_kitchen_ReadyListener_on_item(void* ctx, const uint8_t* p0, size_t p1, kitchen_sink_kitchen_Gadget* p2, kitchen_sink_error* out_err) { frame_kitchen_sink_kitchen_ReadyListener_on_item f; memset(&f, 0, sizeof f); - f.hdr.out_err = out_err; + f.out_err = out_err; f.p0 = p0; f.p1 = p1; f.p2 = p2; @@ -698,12 +950,113 @@ static bool tramp_kitchen_sink_kitchen_ReadyListener_on_item(void* ctx, const ui if (js_cb_on_js_thread(cb)) { invoke_kitchen_sink_kitchen_ReadyListener_on_item(cb->env, cb, &f); } else { - js_cb_req req; - memset(&req, 0, sizeof req); - req.cb = cb; - req.method = 1; - req.frame = &f; - js_cb_hop(&req); + js_cb_hop(cb, 1, &f, out_err, "ReadyListener.onItem"); + } + return f.result; +} + +typedef struct { + js_error* out_err; + uint8_t** out_ptr; + size_t* out_len; +} frame_kitchen_sink_kitchen_ReadyListener_label; + +static void invoke_kitchen_sink_kitchen_ReadyListener_label(napi_env env, js_cb* cb, frame_kitchen_sink_kitchen_ReadyListener_label* f) { + napi_handle_scope scope; + napi_open_handle_scope(env, &scope); + napi_value argv[1]; + napi_value result; + if (!js_cb_call(env, cb, "label", 0, argv, &result)) { + js_cb_report(env, f->out_err, "ReadyListener.label failed"); + } else { + if (!js_ret_str(env, result, f->out_ptr, f->out_len)) { + js_cb_report(env, f->out_err, "ReadyListener.label returned a value of the wrong type"); + } + } + napi_close_handle_scope(env, scope); +} + +static void tramp_kitchen_sink_kitchen_ReadyListener_label(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err) { + frame_kitchen_sink_kitchen_ReadyListener_label f; + memset(&f, 0, sizeof f); + f.out_err = out_err; + f.out_ptr = out_ptr; + f.out_len = out_len; + js_cb* cb = (js_cb*)ctx; + if (js_cb_on_js_thread(cb)) { + invoke_kitchen_sink_kitchen_ReadyListener_label(cb->env, cb, &f); + } else { + js_cb_hop(cb, 2, &f, out_err, "ReadyListener.label"); + } +} + +typedef struct { + js_error* out_err; + uint8_t** out_ptr; + size_t* out_len; +} frame_kitchen_sink_kitchen_ReadyListener_latest; + +static void invoke_kitchen_sink_kitchen_ReadyListener_latest(napi_env env, js_cb* cb, frame_kitchen_sink_kitchen_ReadyListener_latest* f) { + napi_handle_scope scope; + napi_open_handle_scope(env, &scope); + napi_value argv[1]; + napi_value result; + if (!js_cb_call(env, cb, "latest", 0, argv, &result)) { + js_cb_report(env, f->out_err, "ReadyListener.latest failed"); + } else { + if (!js_ret_bytes(env, result, f->out_ptr, f->out_len)) { + js_cb_report(env, f->out_err, "ReadyListener.latest returned a value of the wrong type"); + } + } + napi_close_handle_scope(env, scope); +} + +static void tramp_kitchen_sink_kitchen_ReadyListener_latest(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err) { + frame_kitchen_sink_kitchen_ReadyListener_latest f; + memset(&f, 0, sizeof f); + f.out_err = out_err; + f.out_ptr = out_ptr; + f.out_len = out_len; + js_cb* cb = (js_cb*)ctx; + if (js_cb_on_js_thread(cb)) { + invoke_kitchen_sink_kitchen_ReadyListener_latest(cb->env, cb, &f); + } else { + js_cb_hop(cb, 3, &f, out_err, "ReadyListener.latest"); + } +} + +typedef struct { + js_error* out_err; + kitchen_sink_kitchen_Gadget* result; +} frame_kitchen_sink_kitchen_ReadyListener_favorite; + +static void invoke_kitchen_sink_kitchen_ReadyListener_favorite(napi_env env, js_cb* cb, frame_kitchen_sink_kitchen_ReadyListener_favorite* f) { + napi_handle_scope scope; + napi_open_handle_scope(env, &scope); + napi_value argv[1]; + napi_value result; + if (!js_cb_call(env, cb, "favorite", 0, argv, &result)) { + js_cb_report(env, f->out_err, "ReadyListener.favorite failed"); + } else { + void* h = NULL; + if (js_arg_handle(env, result, &h, false)) { + f->result = (kitchen_sink_kitchen_Gadget*)h; + } else { + js_cb_report(env, f->out_err, "ReadyListener.favorite returned a value of the wrong type"); + } + } + napi_close_handle_scope(env, scope); +} + +static kitchen_sink_kitchen_Gadget* tramp_kitchen_sink_kitchen_ReadyListener_favorite(void* ctx, kitchen_sink_error* out_err) { + frame_kitchen_sink_kitchen_ReadyListener_favorite f; + memset(&f, 0, sizeof f); + f.out_err = out_err; + js_cb* cb = (js_cb*)ctx; + if (js_cb_on_js_thread(cb)) { + invoke_kitchen_sink_kitchen_ReadyListener_favorite(cb->env, cb, &f); + } else { + js_cb_hop(cb, 4, &f, out_err, "ReadyListener.favorite"); } return f.result; } @@ -717,19 +1070,23 @@ static void dispatch_kitchen_sink_kitchen_ReadyListener(napi_env env, napi_value free(req); return; } + if (!js_cb_take(req)) return; if (env == NULL) { - kitchen_sink_error_set(((js_cb_frame*)req->frame)->out_err, -4, "the JavaScript environment is shutting down"); + kitchen_sink_error_set(req->out_err, -4, "the JavaScript environment is shutting down"); } else { switch (req->method) { case 0: invoke_kitchen_sink_kitchen_ReadyListener_on_ready(env, req->cb, (frame_kitchen_sink_kitchen_ReadyListener_on_ready*)req->frame); break; case 1: invoke_kitchen_sink_kitchen_ReadyListener_on_item(env, req->cb, (frame_kitchen_sink_kitchen_ReadyListener_on_item*)req->frame); break; + case 2: invoke_kitchen_sink_kitchen_ReadyListener_label(env, req->cb, (frame_kitchen_sink_kitchen_ReadyListener_label*)req->frame); break; + case 3: invoke_kitchen_sink_kitchen_ReadyListener_latest(env, req->cb, (frame_kitchen_sink_kitchen_ReadyListener_latest*)req->frame); break; + case 4: invoke_kitchen_sink_kitchen_ReadyListener_favorite(env, req->cb, (frame_kitchen_sink_kitchen_ReadyListener_favorite*)req->frame); break; default: break; } } js_cb_finish(req); } -static const kitchen_sink_kitchen_ReadyListener_vtable vtable_kitchen_sink_kitchen_ReadyListener = {tramp_kitchen_sink_kitchen_ReadyListener_on_ready, tramp_kitchen_sink_kitchen_ReadyListener_on_item, js_cb_free}; +static const kitchen_sink_kitchen_ReadyListener_vtable vtable_kitchen_sink_kitchen_ReadyListener = {sizeof(kitchen_sink_kitchen_ReadyListener_vtable), 0, js_cb_free, tramp_kitchen_sink_kitchen_ReadyListener_on_ready, tramp_kitchen_sink_kitchen_ReadyListener_on_item, tramp_kitchen_sink_kitchen_ReadyListener_label, tramp_kitchen_sink_kitchen_ReadyListener_latest, tramp_kitchen_sink_kitchen_ReadyListener_favorite}; static napi_value nx_kitchen_sink_shared_ping(napi_env env, napi_callback_info info) { JS_ARGS(0); @@ -737,7 +1094,9 @@ static napi_value nx_kitchen_sink_shared_ping(napi_env env, napi_callback_info i { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_shared_ping(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -753,14 +1112,19 @@ static napi_value nx_kitchen_sink_kitchen_Gadget_clone(napi_env env, napi_callba JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; - return js_new_handle(env, kitchen_sink_kitchen_Gadget_clone((const kitchen_sink_kitchen_Gadget*)h)); + js_sync_begin(); + kitchen_sink_kitchen_Gadget* r = kitchen_sink_kitchen_Gadget_clone((const kitchen_sink_kitchen_Gadget*)h); + js_sync_end(); + return js_new_handle(env, r); } static napi_value nx_kitchen_sink_kitchen_Gadget_destroy(napi_env env, napi_callback_info info) { JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); kitchen_sink_kitchen_Gadget_destroy((kitchen_sink_kitchen_Gadget*)h); + js_sync_end(); return js_undefined(env); } @@ -771,7 +1135,9 @@ static napi_value nx_kitchen_sink_kitchen_bool_id(napi_env env, napi_callback_in if (!js_arg_bool(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); bool r = kitchen_sink_kitchen_bool_id(a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -790,7 +1156,9 @@ static napi_value nx_kitchen_sink_kitchen_i32_id(napi_env env, napi_callback_inf if (!js_arg_i32(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); int32_t r = kitchen_sink_kitchen_i32_id(a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -809,7 +1177,9 @@ static napi_value nx_kitchen_sink_kitchen_u32_id(napi_env env, napi_callback_inf if (!js_arg_u32(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); uint32_t r = kitchen_sink_kitchen_u32_id(a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -828,7 +1198,9 @@ static napi_value nx_kitchen_sink_kitchen_i64_id(napi_env env, napi_callback_inf if (!js_arg_i64(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); int64_t r = kitchen_sink_kitchen_i64_id(a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -847,7 +1219,9 @@ static napi_value nx_kitchen_sink_kitchen_f64_id(napi_env env, napi_callback_inf if (!js_arg_f64(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); double r = kitchen_sink_kitchen_f64_id(a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -867,7 +1241,9 @@ static napi_value nx_kitchen_sink_kitchen_echo_string(napi_env env, napi_callbac { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_echo_string(JS_STR_PTR(a0), a0.len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -889,7 +1265,9 @@ static napi_value nx_kitchen_sink_kitchen_echo_bytes(napi_env env, napi_callback { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_echo_bytes(a0, a0_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -905,10 +1283,12 @@ static napi_value nx_kitchen_sink_kitchen_subscribe(napi_env env, napi_callback_ JS_ARGS(1); napi_value ret = NULL; js_cb* a0 = NULL; - if (!js_arg_cb(env, argv[0], "kitchen_sink_kitchen_ReadyListener", dispatch_kitchen_sink_kitchen_ReadyListener, &a0)) goto done; + if (!js_arg_cb(env, argv[0], "kitchen_sink_kitchen_ReadyListener", dispatch_kitchen_sink_kitchen_ReadyListener, false, &a0)) goto done; { js_error err = {0}; + js_sync_begin(); int32_t r = kitchen_sink_kitchen_subscribe((void*)a0, &vtable_kitchen_sink_kitchen_ReadyListener, &err); + js_sync_end(); a0 = NULL; if (err.code != 0) { js_throw(env, &err); @@ -922,6 +1302,29 @@ done: return ret; } +static napi_value nx_kitchen_sink_kitchen_maybe_subscribe(napi_env env, napi_callback_info info) { + JS_ARGS(1); + napi_value ret = NULL; + js_cb* a0 = NULL; + if (!js_arg_cb(env, argv[0], "kitchen_sink_kitchen_ReadyListener", dispatch_kitchen_sink_kitchen_ReadyListener, true, &a0)) goto done; + { + js_error err = {0}; + js_sync_begin(); + bool r = kitchen_sink_kitchen_maybe_subscribe((void*)a0, a0 != NULL ? &vtable_kitchen_sink_kitchen_ReadyListener : NULL, &err); + js_sync_end(); + a0 = NULL; + if (err.code != 0) { + js_throw(env, &err); + goto done; + } + ret = js_new_bool(env, r); + } + goto done; +done: + if (a0 != NULL) js_cb_release(env, a0); + return ret; +} + static napi_value nx_kitchen_sink_kitchen_maybe_gadget(napi_env env, napi_callback_info info) { JS_ARGS(1); napi_value ret = NULL; @@ -929,7 +1332,9 @@ static napi_value nx_kitchen_sink_kitchen_maybe_gadget(napi_env env, napi_callba if (!js_arg_handle(env, argv[0], &a0, true)) goto done; { js_error err = {0}; + js_sync_begin(); kitchen_sink_kitchen_Gadget* r = kitchen_sink_kitchen_maybe_gadget((const kitchen_sink_kitchen_Gadget*)a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -947,7 +1352,9 @@ static napi_value nx_kitchen_sink_kitchen_gadgets(napi_env env, napi_callback_in { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_gadgets(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -964,7 +1371,9 @@ static napi_value nx_kitchen_sink_kitchen_stream_gadgets(napi_env env, napi_call napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); kitchen_sink_kitchen_StreamGadgetsIterator* r = kitchen_sink_kitchen_stream_gadgets(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -984,7 +1393,9 @@ static napi_value nx_kitchen_sink_kitchen_StreamGadgetsIterator_next(napi_env en memset(&item, 0, sizeof item); size_t item_len = 0; js_error err = {0}; + js_sync_begin(); int32_t has = kitchen_sink_kitchen_StreamGadgetsIterator_next((kitchen_sink_kitchen_StreamGadgetsIterator*)h, &item, &err); + js_sync_end(); (void)item_len; if (err.code != 0) return js_throw(env, &err); if (has == 0) return js_undefined(env); @@ -995,7 +1406,9 @@ static napi_value nx_kitchen_sink_kitchen_StreamGadgetsIterator_destroy(napi_env JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); kitchen_sink_kitchen_StreamGadgetsIterator_destroy((kitchen_sink_kitchen_StreamGadgetsIterator*)h); + js_sync_end(); return js_undefined(env); } @@ -1007,7 +1420,9 @@ static napi_value nx_kitchen_sink_kitchen_maybe_item(napi_env env, napi_callback { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_maybe_item(a0, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1025,7 +1440,9 @@ static napi_value nx_kitchen_sink_kitchen_list_items(napi_env env, napi_callback { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_list_items(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1043,7 +1460,9 @@ static napi_value nx_kitchen_sink_kitchen_get_attrs(napi_env env, napi_callback_ { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_get_attrs(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1060,7 +1479,9 @@ static napi_value nx_kitchen_sink_kitchen_stream_items(napi_env env, napi_callba napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); kitchen_sink_kitchen_StreamItemsIterator* r = kitchen_sink_kitchen_stream_items(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1080,7 +1501,9 @@ static napi_value nx_kitchen_sink_kitchen_StreamItemsIterator_next(napi_env env, memset(&item, 0, sizeof item); size_t item_len = 0; js_error err = {0}; + js_sync_begin(); int32_t has = kitchen_sink_kitchen_StreamItemsIterator_next((kitchen_sink_kitchen_StreamItemsIterator*)h, &item, &item_len, &err); + js_sync_end(); if (err.code != 0) return js_throw(env, &err); if (has == 0) return js_undefined(env); return js_take_bytes(env, item, item_len); @@ -1090,7 +1513,9 @@ static napi_value nx_kitchen_sink_kitchen_StreamItemsIterator_destroy(napi_env e JS_ARGS(1); void* h = NULL; if (!js_arg_handle(env, argv[0], &h, false)) return NULL; + js_sync_begin(); kitchen_sink_kitchen_StreamItemsIterator_destroy((kitchen_sink_kitchen_StreamItemsIterator*)h); + js_sync_end(); return js_undefined(env); } @@ -1100,7 +1525,9 @@ static napi_value nx_kitchen_sink_kitchen_cross_module_token(napi_env env, napi_ { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_cross_module_token(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1126,7 +1553,9 @@ static napi_value nx_kitchen_sink_kitchen_do_async(napi_env env, napi_callback_i if (!js_arg_str(env, argv[0], &a0)) goto done; { js_async* a = js_async_begin(env, JS_R_STR, "kitchen_sink_kitchen_do_async", &ret); + js_sync_begin(); kitchen_sink_kitchen_do_async(JS_STR_PTR(a0), a0.len, done_kitchen_sink_kitchen_do_async, a); + js_sync_end(); } goto done; done: @@ -1150,7 +1579,9 @@ static napi_value nx_kitchen_sink_kitchen_do_cancellable(napi_env env, napi_call if (!js_arg_handle(env, argv[1], &token, true)) goto done; { js_async* a = js_async_begin(env, JS_R_STR, "kitchen_sink_kitchen_do_cancellable", &ret); + js_sync_begin(); kitchen_sink_kitchen_do_cancellable(JS_STR_PTR(a0), a0.len, (kitchen_sink_cancel_token*)token, done_kitchen_sink_kitchen_do_cancellable, a); + js_sync_end(); } goto done; done: @@ -1163,7 +1594,9 @@ static napi_value nx_kitchen_sink_kitchen_legacy_op(napi_env env, napi_callback_ napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); int32_t r = kitchen_sink_kitchen_legacy_op(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1180,7 +1613,9 @@ static napi_value nx_kitchen_sink_kitchen_new_op(napi_env env, napi_callback_inf napi_value ret = NULL; { js_error err = {0}; + js_sync_begin(); int32_t r = kitchen_sink_kitchen_new_op(&err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1199,7 +1634,9 @@ static napi_value nx_kitchen_sink_kitchen_Gadget_new(napi_env env, napi_callback if (!js_arg_i64(env, argv[0], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); kitchen_sink_kitchen_Gadget* r = kitchen_sink_kitchen_Gadget_new(a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1219,7 +1656,9 @@ static napi_value nx_kitchen_sink_kitchen_Gadget_describe(napi_env env, napi_cal { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_Gadget_describe((const kitchen_sink_kitchen_Gadget*)self_h, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1240,7 +1679,9 @@ static napi_value nx_kitchen_sink_kitchen_Gadget_poke(napi_env env, napi_callbac if (!js_arg_i32(env, argv[1], &a0)) goto done; { js_error err = {0}; + js_sync_begin(); int32_t r = kitchen_sink_kitchen_Gadget_poke((const kitchen_sink_kitchen_Gadget*)self_h, a0, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1258,7 +1699,9 @@ static napi_value nx_kitchen_sink_kitchen_Gadget_version(napi_env env, napi_call { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_Gadget_version(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1276,7 +1719,9 @@ static napi_value nx_kitchen_sink_kitchen_nested_hello(napi_env env, napi_callba { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = kitchen_sink_kitchen_nested_hello(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -1289,12 +1734,11 @@ done: } NAPI_MODULE_INIT() { - js_env* e = (js_env*)calloc(1, sizeof *e); - napi_set_instance_data(env, e, js_env_free, NULL); + js_env_init(env); napi_property_descriptor props[] = { JS_RUNTIME_EXPORTS, - {"kitchen_sink_shared_checksum", NULL, nx_kitchen_sink_shared_checksum, NULL, NULL, NULL, napi_default, NULL}, - {"kitchen_sink_kitchen_checksum", NULL, nx_kitchen_sink_kitchen_checksum, NULL, NULL, NULL, napi_default, NULL}, + {"kitchen_sink_shared_contract", NULL, nx_kitchen_sink_shared_contract, NULL, NULL, NULL, napi_default, NULL}, + {"kitchen_sink_kitchen_contract", NULL, nx_kitchen_sink_kitchen_contract, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_shared_ping", NULL, nx_kitchen_sink_shared_ping, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_kitchen_Gadget_clone", NULL, nx_kitchen_sink_kitchen_Gadget_clone, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_kitchen_Gadget_destroy", NULL, nx_kitchen_sink_kitchen_Gadget_destroy, NULL, NULL, NULL, napi_default, NULL}, @@ -1306,6 +1750,7 @@ NAPI_MODULE_INIT() { {"kitchen_sink_kitchen_echo_string", NULL, nx_kitchen_sink_kitchen_echo_string, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_kitchen_echo_bytes", NULL, nx_kitchen_sink_kitchen_echo_bytes, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_kitchen_subscribe", NULL, nx_kitchen_sink_kitchen_subscribe, NULL, NULL, NULL, napi_default, NULL}, + {"kitchen_sink_kitchen_maybe_subscribe", NULL, nx_kitchen_sink_kitchen_maybe_subscribe, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_kitchen_maybe_gadget", NULL, nx_kitchen_sink_kitchen_maybe_gadget, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_kitchen_gadgets", NULL, nx_kitchen_sink_kitchen_gadgets, NULL, NULL, NULL, napi_default, NULL}, {"kitchen_sink_kitchen_stream_gadgets", NULL, nx_kitchen_sink_kitchen_stream_gadgets, NULL, NULL, NULL, napi_default, NULL}, diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__package_json.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__package_json.snap index ed61743e..daa999c6 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__package_json.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__package_json.snap @@ -2,9 +2,9 @@ source: crates/weaveffi-cli/tests/snapshots.rs --- { - "//": "Generated by WeaveFFI [VERSION] from api.yml", + "//": "Generated by WeaveFFI [VERSION] "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", + "//regenerate": "To regenerate: weaveffi generate", "name": "kitchen_sink", "version": "0.1.0", "description": "Native bindings for kitchen_sink", @@ -27,7 +27,7 @@ source: crates/weaveffi-cli/tests/snapshots.rs ], "gypfile": true, "scripts": { - "install": "node-gyp rebuild" + "install": "node -e \"const p = process.platform + '-' + process.arch; try { require.resolve('kitchen_sink-' + p + '/kitchen_sink_node.node'); process.exit(0); } catch {} process.exit(require('fs').existsSync('prebuilds/' + p + '/kitchen_sink_node.node') ? 0 : 1)\" || node-gyp rebuild" }, "engines": { "node": ">=18" diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__runtime_js.snap index e11da31e..02cd16c0 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__runtime_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__runtime_js.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // The runtime shared by every binding in this package: the error classes, // the value-buffer codec, the object and iterator wrappers, cancellation, and @@ -66,25 +66,38 @@ export function $fault(e) { /** * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. + * class, and `payloads` maps each code that carries fields to the reader + * that decodes them (the class's constructor then takes the fields first). + * Codes outside the domain fall back to `$fault`. */ export function $domain(e, codes, payloads) { if (e instanceof $Fault) { const cls = codes.get(e.code); if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; + const message = e.message || undefined; + const read = payloads.get(e.code); + if (read === undefined) return new cls(message); + return new cls(e.payload === null ? {} : $decode(e.payload, read), message); } } return $fault(e); } +/** + * The fault a callback implementation reports for exception `e` from a + * method that throws the error domain `domain`: a declared code of the + * domain travels with its fields (`fields` maps each code that has any to + * the writer that encodes them); anything else is rethrown unchanged and + * reported as a code -4 failure. + */ +export function $raise(e, domain, codes, fields) { + if (e instanceof domain && codes.has(e.code)) { + const write = fields.get(e.code); + return new $Fault(e.code, e.message, write === undefined ? null : $encode(e, write)); + } + return e; +} + function $malformed(what) { return new KitchenSinkError(-3, 'malformed value buffer: ' + what); } @@ -313,6 +326,27 @@ export const $W = Object.freeze({ Bytes: (w, v) => w.writeBytes(v), }); +/** + * Map key writers. Plain-object keys are always strings, so numeric, + * `bigint`, and `bool` keys convert first (a `Map`'s typed keys pass + * through the same conversions unchanged). C-style enum keys use `I32`. + */ +export const $WK = Object.freeze({ + Bool: (w, k) => w.writeBool(k === true || k === 'true'), + I8: (w, k) => w.writeI8(Number(k)), + I16: (w, k) => w.writeI16(Number(k)), + I32: (w, k) => w.writeI32(Number(k)), + I64: (w, k) => w.writeI64(BigInt(k)), + U8: (w, k) => w.writeU8(Number(k)), + U16: (w, k) => w.writeU16(Number(k)), + U32: (w, k) => w.writeU32(Number(k)), + U64: (w, k) => w.writeU64(BigInt(k)), + F32: (w, k) => w.writeF32(Number(k)), + F64: (w, k) => w.writeF64(Number(k)), + String: (w, k) => w.writeString(k), + Bytes: (w, k) => w.writeBytes(k), +}); + /** Element readers for the fixed-width primitives, strings, and bytes. */ export const $R = Object.freeze({ Bool: (r) => r.readBool(), @@ -418,6 +452,11 @@ export function $clone(o, cls) { } } +/** `$clone`, with `null` for an absent object. */ +export function $cloneOpt(o, cls) { + return o === null || o === undefined ? null : $clone(o, cls); +} + const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); /** @@ -526,8 +565,9 @@ function $retBigint(v, where) { } /** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. + * Checks for a callback method's return value, by wire primitive (plus + * `String` and `Bytes`). A value of the wrong type throws, which reports + * the call as a code -4 failure. */ export const $ret = Object.freeze({ Bool(v, where) { @@ -544,14 +584,25 @@ export const $ret = Object.freeze({ F64: $num, I64: $retBigint, U64: $retBigint, + String(v, where) { + if (typeof v !== 'string') throw new TypeError(`${where} must return a string`); + return v; + }, + Bytes(v, where) { + if (!(v instanceof Uint8Array)) throw new TypeError(`${where} must return a Uint8Array`); + return v; + }, }); /** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. + * The load-time checks: the native library must implement ABI revision + * `abi`, and, for every top-level module, its contract table must hold each + * entry these bindings were generated with. `contract` lists + * `[module, [[id, hash, path], ...]]` per module; the native side returns + * a module's table as a `BigUint64Array` of `id, hash` pairs. Entries the + * library has and the bindings don't are fine. */ -export function $verify(raw, library, prefix, abi, checksums) { +export function $verify(raw, library, prefix, abi, contract) { const version = raw[`${prefix}_abi_version`]; if (typeof version !== 'function') { throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); @@ -562,15 +613,22 @@ export function $verify(raw, library, prefix, abi, checksums) { `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, ); } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); + for (const [module, expected] of contract) { + const table = raw[`${prefix}_${module}_contract`]; + if (typeof table !== 'function') { + throw new Error(`${library}: the native library does not export ${prefix}_${module}_contract`); + } + const pairs = table(); + const hashes = new Map(); + for (let i = 0; i + 1 < pairs.length; i += 2) hashes.set(pairs[i], pairs[i + 1]); + for (const [id, hash, path] of expected) { + const got = hashes.get(id); + if (got === undefined) { + throw new Error(`${library}: ${path} is missing from the library`); + } + if (got !== hash) { + throw new Error(`${library}: ${path} changed since these bindings were generated`); + } } } } diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__README_md.snap deleted file mode 100644 index 152f77c4..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__README_md.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# nested_modules - -Native bindings for nested_modules - -Node.js bindings for the `nested_modules` native library: an ES module over a small N-API addon (`nested_modules_node.c`) that `npm install` compiles with node-gyp. The addon links `nested_modules`; set `NESTED_MODULES_LIBRARY` to the library file (or its directory) when installing, or pass `--nested_modules-library=` to npm. - -```sh -NESTED_MODULES_LIBRARY=/path/to/libnested_modules.so npm install nested_modules -``` - -Each top-level module of the API is a namespace export: - -```js -import { graphics } from 'nested_modules'; -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__binding_gyp.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__binding_gyp.snap deleted file mode 100644 index f9dd2600..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__binding_gyp.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -{ - "variables": { - "native_dir%": " { const path = require('path'); const p = process.env.NESTED_MODULES_LIBRARY || process.env.npm_config_nested_modules_library; if (!p) return path.resolve('.'); return require('fs').statSync(p).isFile() ? path.dirname(path.resolve(p)) : path.resolve(p); })()\")" - }, - "targets": [ - { - "target_name": "nested_modules_node", - "sources": [ - "nested_modules_node.c" - ], - "include_dirs": [ - "." - ], - "library_dirs": [ - "<(native_dir)" - ], - "libraries": [ - "-lnested_modules" - ], - "conditions": [ - ["OS=='mac'", { "xcode_settings": { "OTHER_LDFLAGS": ["-Wl,-rpath,@loader_path", "-Wl,-rpath,<(native_dir)"] } }], - ["OS=='linux'", { "ldflags": ["-Wl,-rpath,'$$ORIGIN'", "-Wl,-rpath,<(native_dir)"] }] - ] - } - ] -} - -# END binding.gyp diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_d_ts.snap index 9df384d2..f957aa40 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -24,13 +24,13 @@ export declare class CancelledError extends NestedModulesError { /** The root error class, under a name no namespace member can shadow. */ declare const $Error: typeof NestedModulesError; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** Construct a Canvas. */ export declare namespace graphics { /** The errors of the `graphics` error domain. */ export class GraphicsError extends $Error { @@ -54,7 +54,6 @@ export declare namespace graphics { } /** Construct a Canvas. */ export function newCanvas(width: number): graphics.Canvas; - /** Construct a Shape. */ export namespace shapes { /** A struct one level deep that references both an ancestor enum and a deeper struct. */ export interface Shape { @@ -68,7 +67,6 @@ export declare namespace graphics { export function makeShape(sides: number): graphics.shapes.Shape; /** Nested function returning a struct owned by an ancestor module. */ export function describe(): graphics.Canvas; - /** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ export namespace geometry { /** A struct two levels deep. */ export interface Detail { diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_js.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_js.snap index b9a47871..4d89074c 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -32,22 +35,30 @@ export { NestedModulesError, CancelledError } from './runtime.js'; import { createRequire } from 'node:module'; -// The native entry points: the N-API addon `npm install` builds from -// nested_modules_node.c (with node-gyp), or a prebuilt copy next to this file. +// The native entry points: the N-API addon built from nested_modules_node.c. A prebuilt +// copy comes first (from the `nested_modules--` package `npm install` +// picks for this platform, or under prebuilds/-/), then the one +// node-gyp compiles at install time when none matches. function $loadAddon(name) { const require = createRequire(import.meta.url); - const tried = []; - for (const dir of ['./build/Release/', './build/Debug/', './']) { + const platform = `${process.platform}-${process.arch}`; + const candidates = [ + `nested_modules-${platform}/${name}`, + `./prebuilds/${platform}/${name}`, + `./build/Release/${name}`, + `./build/Debug/${name}`, + ]; + for (const candidate of candidates) { try { - return require(dir + name); + return require(candidate); } catch (e) { if (e.code !== 'MODULE_NOT_FOUND') throw e; - tried.push(dir + name); } } throw new Error( - `the native addon ${name} was not found (tried ${tried.join(', ')}); ` + - 'build it with `npm install` or `node-gyp rebuild` in this package', + `the native addon ${name} was not found (tried ${candidates.join(', ')}); ` + + 'install the package for this platform, or build the addon with `npm install` ' + + 'or `node-gyp rebuild` in this package', ); } @@ -57,12 +68,29 @@ $raw.$setup($Fault); // Object tokens in value buffers are already bigint handles here. const $token = (t) => t; -$verify($raw, 'nested_modules', 'nested_modules', 3, [['graphics', 0x44d31b5b10fdb90dn]]); +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['graphics', [ + [0x2c29eef9327a6bd4n, 0x12983d5271651389n, 'graphics.Unit'], + [0x304088bbc28f0015n, 0x97e6e809913be6b6n, 'graphics.new_canvas'], + [0x49f8837532df9ac6n, 0x32e5b27e81ee0957n, 'graphics.Canvas'], + [0x55717fa027336b47n, 0xb2bd3a6bdcce48c8n, 'graphics.shapes.describe'], + [0x7626b91b8f1f53cdn, 0x867bbf7cf0f41c74n, 'graphics.shapes.geometry.compute'], + [0xb3efcb50d9a61ab7n, 0x3cb0207dfad2a149n, 'graphics.GraphicsError'], + [0xc62b16a9591b744bn, 0x7895610c8487eb1bn, 'graphics.shapes.Shape'], + [0xe13001f4983ccd35n, 0x16e7de070b4a4cfcn, 'graphics.shapes.geometry.Detail'], + [0xf3f658bcfc5dba62n, 0x2665ccb47356e746n, 'graphics.shapes.make_shape'], + ]], +]; +$verify($raw, 'nested_modules', 'nested_modules', 4, $contract); /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.nested_modules_debug_live(kind); @@ -75,9 +103,12 @@ const graphics$OutOfBoundsError = class OutOfBoundsError extends graphics$Graphi super(1, message); } }; -const $codes$graphics$GraphicsError = new Map([[1, graphics$OutOfBoundsError]]); +const $codes$graphics$GraphicsError = new Map([ + [1, graphics$OutOfBoundsError], +]); +const $payloads$graphics$GraphicsError = new Map(); function $from$graphics$GraphicsError(e) { - return $domain(e, $codes$graphics$GraphicsError); + return $domain(e, $codes$graphics$GraphicsError, $payloads$graphics$GraphicsError); } const graphics$Unit = Object.freeze({ diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_h.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_h.snap deleted file mode 100644 index 27c97f2b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_h.snap +++ /dev/null @@ -1,144 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef NESTED_MODULES_H -#define NESTED_MODULES_H - -#include -#include -#include - -#ifndef NESTED_MODULES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef NESTED_MODULES_BUILD -# define NESTED_MODULES_API __declspec(dllexport) -# else -# define NESTED_MODULES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define NESTED_MODULES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define NESTED_MODULES_API __attribute__((visibility("default"))) -# else -# define NESTED_MODULES_API -# endif -#endif - -#ifndef NESTED_MODULES_DEPRECATED -# if defined(_MSC_VER) -# define NESTED_MODULES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define NESTED_MODULES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define NESTED_MODULES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports nested_modules_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define NESTED_MODULES_ABI_VERSION 3u -NESTED_MODULES_API uint32_t nested_modules_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by nested_modules_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct nested_modules_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} nested_modules_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -NESTED_MODULES_API void nested_modules_error_set(nested_modules_error* err, int32_t code, const char* message); -NESTED_MODULES_API void nested_modules_error_clear(nested_modules_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - nested_modules_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -NESTED_MODULES_API void nested_modules_error_free(nested_modules_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -NESTED_MODULES_API void nested_modules_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct nested_modules_cancel_token nested_modules_cancel_token; -NESTED_MODULES_API nested_modules_cancel_token* nested_modules_cancel_token_create(void); -NESTED_MODULES_API void nested_modules_cancel_token_cancel(nested_modules_cancel_token* token); -NESTED_MODULES_API bool nested_modules_cancel_token_is_cancelled(const nested_modules_cancel_token* token); -NESTED_MODULES_API void nested_modules_cancel_token_destroy(nested_modules_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -NESTED_MODULES_API uint8_t* nested_modules_alloc(uint32_t size); -NESTED_MODULES_API void nested_modules_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -NESTED_MODULES_API uint64_t nested_modules_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define NESTED_MODULES_GRAPHICS_CHECKSUM 0x44d31b5b10fdb90dull -NESTED_MODULES_API uint64_t nested_modules_graphics_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with nested_modules_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * nested_modules_buffer.h declares a C struct and codec for each. - */ - -/** A measurement unit shared with nested submodules. */ -typedef enum { nested_modules_graphics_Unit_pixels = 0, nested_modules_graphics_Unit_points = 1 } nested_modules_graphics_Unit; -/** Error codes reported by throwing functions in the `graphics` module tree. */ -typedef enum { nested_modules_graphics_GraphicsError_OutOfBounds = 1 } nested_modules_graphics_GraphicsError; - -// Module: graphics -/** Construct a Canvas. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_new_canvas(int32_t width, size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes -/** Construct a Shape. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_make_shape(int32_t sides, size_t* out_len, nested_modules_error* out_err); -/** Nested function returning a struct owned by an ancestor module. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_describe(size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes_geometry -/** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_geometry_compute(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, nested_modules_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // NESTED_MODULES_H - -// END nested_modules.h diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap index 789f9ee7..df1c3444 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef NAPI_VERSION #define NAPI_VERSION 8 @@ -12,6 +12,7 @@ source: crates/weaveffi-cli/tests/snapshots.rs #include #include #include +#include #include #include @@ -26,8 +27,9 @@ source: crates/weaveffi-cli/tests/snapshots.rs * raw convention the shared JavaScript layer documents: direct values as * numbers, bigints, and booleans; strings as strings; bytes and value * buffers as Uint8Arrays; objects, iterators, and cancel tokens as bigint - * handles (null when absent); callback interfaces as adapter objects. A - * failure throws an instance of the runtime's `$Fault` class. + * handles (null when absent); callback interfaces as adapter objects (null + * for an absent optional one). A failure throws an instance of the + * runtime's `$Fault` class. */ typedef nested_modules_error js_error; @@ -35,23 +37,106 @@ typedef nested_modules_error js_error; /* Helpers an API may not need. */ #if defined(__GNUC__) || defined(__clang__) #define JS_HELPER static inline __attribute__((unused)) +#define JS_TLS __thread #else #define JS_HELPER static inline +#define JS_TLS __declspec(thread) #endif +/* --------------------------------------------------------------------------- + * JavaScript threads. One `js_thread` per thread that runs JavaScript and + * loaded the addon (the main thread and each worker). It counts the + * synchronous calls into the library in progress on that thread, which a + * callback made on another thread consults while it waits for the + * JavaScript thread (see js_cb_hop). Each environment and each callback + * registration holds a reference. + * ------------------------------------------------------------------------- */ + +typedef struct { + uv_mutex_t mu; + uv_cond_t cv; /* broadcast when a call begins or ends, or a hop finishes */ + unsigned depth; /* synchronous calls into the library in progress */ + uint64_t calls; /* outermost synchronous calls begun so far */ + unsigned waiters; /* producer threads waiting in js_cb_hop */ + unsigned refs; +} js_thread; + +/* The state of the JavaScript thread this is, or NULL on any other thread. */ +static JS_TLS js_thread* js_current = NULL; + +static js_thread* js_thread_new(void) { + js_thread* t = (js_thread*)calloc(1, sizeof *t); + uv_mutex_init(&t->mu); + uv_cond_init(&t->cv); + t->refs = 1; + return t; +} + +static void js_thread_retain(js_thread* t) { + uv_mutex_lock(&t->mu); + t->refs++; + uv_mutex_unlock(&t->mu); +} + +static void js_thread_release(js_thread* t) { + uv_mutex_lock(&t->mu); + bool last = --t->refs == 0; + uv_mutex_unlock(&t->mu); + if (!last) return; + uv_cond_destroy(&t->cv); + uv_mutex_destroy(&t->mu); + free(t); +} + +/* Bracket every synchronous call into the library. */ +JS_HELPER void js_sync_begin(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + if (t->depth++ == 0) t->calls++; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +JS_HELPER void js_sync_end(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + t->depth--; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + /* Per-environment state (one per main thread or worker): the `$Fault` - * constructor `index.js` registers through `$setup`. */ + * constructor `index.js` registers through `$setup`, and the thread. */ typedef struct { napi_ref fault; + js_thread* thread; } js_env; static void js_env_free(napi_env env, void* data, void* hint) { (void)hint; js_env* e = (js_env*)data; if (e->fault != NULL) napi_delete_reference(env, e->fault); + if (js_current == e->thread) js_current = NULL; + js_thread_release(e->thread); free(e); } +/* Set up the environment of the thread loading the addon. */ +static void js_env_init(napi_env env) { + js_env* e = (js_env*)calloc(1, sizeof *e); + e->thread = js_thread_new(); + js_current = e->thread; + napi_set_instance_data(env, e, js_env_free, NULL); +} + +static js_env* js_env_of(napi_env env) { + js_env* e = NULL; + napi_get_instance_data(env, (void**)&e); + return e; +} + JS_HELPER napi_value js_undefined(napi_env env) { napi_value v; napi_get_undefined(env, &v); @@ -62,13 +147,22 @@ static napi_value js_setup(napi_env env, napi_callback_info info) { size_t argc = 1; napi_value argv[1]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); + js_env* e = js_env_of(env); if (e->fault != NULL) napi_delete_reference(env, e->fault); napi_create_reference(env, argv[0], 1, &e->fault); return js_undefined(env); } +/* The `$Fault` constructor, or NULL before `$setup`. */ +static napi_value js_fault_class(napi_env env) { + js_env* e = js_env_of(env); + napi_value ctor = NULL; + if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok) { + return NULL; + } + return ctor; +} + /* --------------------------------------------------------------------------- * Value conversions * ------------------------------------------------------------------------- */ @@ -149,6 +243,21 @@ JS_HELPER napi_value js_take_bytes(napi_env env, const uint8_t* ptr, size_t len) return out; } +/* A module's contract table as a BigUint64Array of `id, hash` pairs. */ +JS_HELPER napi_value js_new_contract(napi_env env, const nested_modules_contract_entry* table, + size_t len) { + napi_value ab, out; + void* data = NULL; + napi_create_arraybuffer(env, len * 2 * sizeof(uint64_t), &data, &ab); + uint64_t* words = (uint64_t*)data; + for (size_t i = 0; i < len; i++) { + words[2 * i] = table[i].id; + words[2 * i + 1] = table[i].hash; + } + napi_create_typedarray(env, napi_biguint64_array, len * 2, ab, 0, &out); + return out; +} + JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, const uint8_t* payload, size_t payload_len) { napi_value argv[3], ctor, out = NULL; @@ -159,10 +268,8 @@ JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, } else { napi_get_null(env, &argv[2]); } - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); - if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok || - ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { + ctor = js_fault_class(env); + if (ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { napi_create_error(env, NULL, argv[1], &out); napi_set_named_property(env, out, "code", argv[0]); } @@ -428,30 +535,42 @@ JS_HELPER void js_async_done(js_async* a, js_error* err) { * producer: a reference to the adapter object, the JavaScript thread it * belongs to, and a thread-safe function that runs calls made from any other * thread on that thread while the caller waits. + * + * Waiting is what makes a call from another thread safe, and also what could + * deadlock it: if the JavaScript thread is itself inside a synchronous call + * into the library that waits for the calling thread, neither can proceed. + * A waiting call therefore watches the JavaScript thread. While that thread + * stays inside one synchronous call for JS_CB_DEADLOCK_MS without having + * started the callback, the wait is abandoned and the callback fails with + * code -4 instead of hanging (a shorter synchronous call, or one that + * returns to the event loop, just delays the callback). * ------------------------------------------------------------------------- */ +#define JS_CB_DEADLOCK_MS 1000 + typedef struct { napi_env env; napi_ref ref; napi_threadsafe_function tsfn; uv_thread_t js_thread; + js_thread* thread; } js_cb; -/* Every callback frame starts with this header so the dispatcher can reach - * `out_err` without knowing the method. */ -typedef struct { - js_error* out_err; -} js_cb_frame; +typedef enum { JS_REQ_QUEUED, JS_REQ_RUNNING, JS_REQ_DONE, JS_REQ_ABANDONED } js_req_state; /* A call queued from another thread: `method` is the vtable index, or -1 to - * release the registration. */ + * release the registration; `frame` holds the method's slots. The request is + * heap-allocated and holds its own reference to the thread state, so a + * caller that abandons it can return (and the producer can then release the + * registration) while it is still queued: the dispatcher frees it without + * touching the frame or the registration. */ typedef struct { js_cb* cb; + js_thread* thread; int method; void* frame; - uv_mutex_t mu; - uv_cond_t cv; - bool done; + js_error* out_err; + js_req_state state; } js_cb_req; JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { @@ -460,10 +579,12 @@ JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { } JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* name, - napi_threadsafe_function_call_js dispatch) { + napi_threadsafe_function_call_js dispatch) { js_cb* cb = (js_cb*)calloc(1, sizeof *cb); cb->env = env; cb->js_thread = uv_thread_self(); + cb->thread = js_env_of(env)->thread; + js_thread_retain(cb->thread); napi_create_reference(env, adapter, 1, &cb->ref); napi_value resource; napi_create_string_utf8(env, name, NAPI_AUTO_LENGTH, &resource); @@ -477,14 +598,20 @@ JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* na JS_HELPER void js_cb_release(napi_env env, js_cb* cb) { if (env != NULL) napi_delete_reference(env, cb->ref); napi_release_threadsafe_function(cb->tsfn, napi_tsfn_release); + js_thread_release(cb->thread); free(cb); } -/* A callback-interface argument: the adapter object `index.js` built. */ +/* A callback-interface argument: the adapter object `index.js` built, or + * null for an absent optional one. */ JS_HELPER bool js_arg_cb(napi_env env, napi_value v, const char* name, - napi_threadsafe_function_call_js dispatch, js_cb** out) { + napi_threadsafe_function_call_js dispatch, bool nullable, js_cb** out) { napi_valuetype t; napi_typeof(env, v, &t); + if (nullable && (t == napi_null || t == napi_undefined)) { + *out = NULL; + return true; + } if (t != napi_object) return js_type_error(env, "expected a callback interface implementation"); *out = js_cb_register(env, v, name, dispatch); return true; @@ -505,62 +632,188 @@ JS_HELPER void js_cb_free(void* ctx) { if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) free(req); } -/* Run `req` on the JavaScript thread and wait for it. */ -JS_HELPER void js_cb_hop(js_cb_req* req) { - uv_mutex_init(&req->mu); - uv_cond_init(&req->cv); - req->done = false; - if (napi_call_threadsafe_function(req->cb->tsfn, req, napi_tsfn_blocking) == napi_ok) { - uv_mutex_lock(&req->mu); - while (!req->done) uv_cond_wait(&req->cv, &req->mu); - uv_mutex_unlock(&req->mu); - } else { - nested_modules_error_set(((js_cb_frame*)req->frame)->out_err, -4, - "the callback implementation is no longer reachable"); +/* Run method `method` of `cb` on its JavaScript thread and wait for it, + * failing it with code -4 rather than deadlocking (see above). `what` names + * the method in that failure. */ +JS_HELPER void js_cb_hop(js_cb* cb, int method, void* frame, js_error* out_err, + const char* what) { + js_thread* t = cb->thread; + js_cb_req* req = (js_cb_req*)calloc(1, sizeof *req); + req->cb = cb; + req->thread = t; + js_thread_retain(t); + req->method = method; + req->frame = frame; + req->out_err = out_err; + req->state = JS_REQ_QUEUED; + uv_mutex_lock(&t->mu); + t->waiters++; + uv_mutex_unlock(&t->mu); + if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) { + uv_mutex_lock(&t->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); + nested_modules_error_set(out_err, -4, "the callback implementation is no longer reachable"); + return; + } + const uint64_t limit = (uint64_t)JS_CB_DEADLOCK_MS * 1000000u; + bool blocked = false; + uint64_t call = 0, since = 0; + uv_mutex_lock(&t->mu); + while (req->state != JS_REQ_DONE) { + if (req->state == JS_REQ_QUEUED && t->depth > 0) { + uint64_t now = uv_hrtime(); + if (!blocked || call != t->calls) { + blocked = true; + call = t->calls; + since = now; + } + if (now - since >= limit) { + req->state = JS_REQ_ABANDONED; /* the dispatcher frees it */ + t->waiters--; + uv_mutex_unlock(&t->mu); + char msg[512]; + snprintf(msg, sizeof msg, + "%s was called on another thread while the JavaScript thread was blocked in a " + "synchronous call into the library for %d ms; the call likely waits for this " + "callback, so waiting longer would deadlock (call it asynchronously, or call " + "back on the calling thread)", + what, JS_CB_DEADLOCK_MS); + nested_modules_error_set(out_err, -4, msg); + return; + } + uv_cond_timedwait(&t->cv, &t->mu, limit - (now - since)); + } else { + blocked = false; + uv_cond_wait(&t->cv, &t->mu); + } } - uv_cond_destroy(&req->cv); - uv_mutex_destroy(&req->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); +} + +/* The dispatcher's side of a hop. js_cb_take claims a queued request (false + * when its caller gave up, in which case the request is freed); + * js_cb_finish reports it done. */ +JS_HELPER bool js_cb_take(js_cb_req* req) { + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + bool abandoned = req->state == JS_REQ_ABANDONED; + if (!abandoned) req->state = JS_REQ_RUNNING; + uv_mutex_unlock(&t->mu); + if (abandoned) { + free(req); + js_thread_release(t); + } + return !abandoned; } JS_HELPER void js_cb_finish(js_cb_req* req) { - uv_mutex_lock(&req->mu); - req->done = true; - uv_cond_signal(&req->cv); - uv_mutex_unlock(&req->mu); + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + req->state = JS_REQ_DONE; + uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +/* A JavaScript value's text as a heap-allocated C string (NULL on failure). */ +JS_HELPER char* js_cstring(napi_env env, napi_value v) { + napi_value str; + size_t len = 0; + if (napi_coerce_to_string(env, v, &str) != napi_ok || + napi_get_value_string_utf8(env, str, NULL, 0, &len) != napi_ok) { + return NULL; + } + char* out = (char*)malloc(len + 1); + napi_get_value_string_utf8(env, str, out, len + 1, NULL); + return out; } -/* Report the pending JavaScript exception (or `fallback`) through `out_err` - * as a foreign failure (code -4); nothing unwinds through the C frame. */ +/* Report the pending JavaScript exception (or `fallback`) through + * `out_err`; nothing unwinds through the C frame. A `$Fault` (a domain error + * a throwing method raised) reports its code, message, and payload; any + * other exception is a foreign failure, code -4, with its message. */ JS_HELPER void js_cb_report(napi_env env, js_error* out_err, const char* fallback) { - char msg[512]; - msg[0] = 0; + int32_t code = -4; + char* msg = NULL; + const uint8_t* payload = NULL; + size_t payload_len = 0; bool pending = false; napi_is_exception_pending(env, &pending); if (pending) { - napi_value exc, text, str; + napi_value exc, text; napi_get_and_clear_last_exception(env, &exc); + napi_value fault = js_fault_class(env); + bool is_fault = false; + if (fault != NULL) napi_instanceof(env, exc, fault, &is_fault); napi_valuetype t; napi_typeof(env, exc, &t); text = exc; if (t == napi_object) napi_get_named_property(env, exc, "message", &text); - if (napi_coerce_to_string(env, text, &str) == napi_ok) { - napi_get_value_string_utf8(env, str, msg, sizeof msg, NULL); + msg = js_cstring(env, text); + if (is_fault) { + napi_value v; + napi_get_named_property(env, exc, "code", &v); + napi_get_value_int32(env, v, &code); + napi_get_named_property(env, exc, "payload", &v); + bool typed = false; + napi_is_typedarray(env, v, &typed); + if (typed) { + napi_typedarray_type type; + void* data = NULL; + napi_get_typedarray_info(env, v, &type, &payload_len, &data, NULL, NULL); + payload = (const uint8_t*)data; + } } napi_is_exception_pending(env, &pending); if (pending) napi_get_and_clear_last_exception(env, &exc); } - nested_modules_error_set(out_err, -4, msg[0] != 0 ? msg : fallback); + nested_modules_error_set(out_err, code, msg != NULL && msg[0] != 0 ? msg : fallback); + if (payload != NULL) nested_modules_error_set_payload(out_err, payload, payload_len); + free(msg); } /* Look up and call `method` on the adapter; false when it threw. */ JS_HELPER bool js_cb_call(napi_env env, js_cb* cb, const char* method, size_t argc, - const napi_value* argv, napi_value* result) { + const napi_value* argv, napi_value* result) { napi_value adapter, fn; napi_get_reference_value(env, cb->ref, &adapter); napi_get_named_property(env, adapter, method, &fn); return napi_call_function(env, adapter, fn, argc, argv, result) == napi_ok; } +/* Hand `len` bytes to the producer through a callback's out slots, as a run + * from {p}_alloc that the producer adopts. */ +JS_HELPER void js_give(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = nested_modules_alloc(len); + if (len > 0) memcpy(run, data, len); + *out_ptr = run; + *out_len = len; +} + +/* A callback's string return: false (with a pending exception) when the + * value isn't a string. */ +JS_HELPER bool js_ret_str(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + js_str s = JS_STR_INIT; + if (!js_arg_str(env, v, &s)) return false; + js_give(s.ptr, s.len, out_ptr, out_len); + js_str_free(&s); + return true; +} + +/* A callback's bytes or buffer return: a Uint8Array. */ +JS_HELPER bool js_ret_bytes(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + const uint8_t* ptr = NULL; + size_t len = 0; + if (!js_arg_bytes(env, v, &ptr, &len)) return false; + js_give(ptr, len, out_ptr, out_len); + return true; +} + /* --------------------------------------------------------------------------- * The runtime symbols every library exports. * ------------------------------------------------------------------------- */ @@ -609,9 +862,11 @@ static napi_value js_cancel_token_destroy(napi_env env, napi_callback_info info) {"nested_modules_cancel_token_destroy", NULL, js_cancel_token_destroy, NULL, NULL, NULL, \ napi_default, NULL} -static napi_value nx_nested_modules_graphics_checksum(napi_env env, napi_callback_info info) { +static napi_value nx_nested_modules_graphics_contract(napi_env env, napi_callback_info info) { (void)info; - return js_new_u64(env, nested_modules_graphics_checksum()); + size_t len = 0; + const nested_modules_contract_entry* table = nested_modules_graphics_contract(&len); + return js_new_contract(env, table, len); } static napi_value nx_nested_modules_graphics_new_canvas(napi_env env, napi_callback_info info) { @@ -622,7 +877,9 @@ static napi_value nx_nested_modules_graphics_new_canvas(napi_env env, napi_callb { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = nested_modules_graphics_new_canvas(a0, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -642,7 +899,9 @@ static napi_value nx_nested_modules_graphics_shapes_make_shape(napi_env env, nap { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = nested_modules_graphics_shapes_make_shape(a0, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -660,7 +919,9 @@ static napi_value nx_nested_modules_graphics_shapes_describe(napi_env env, napi_ { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = nested_modules_graphics_shapes_describe(&out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -681,7 +942,9 @@ static napi_value nx_nested_modules_graphics_shapes_geometry_compute(napi_env en { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = nested_modules_graphics_shapes_geometry_compute(a0, a0_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -694,11 +957,10 @@ done: } NAPI_MODULE_INIT() { - js_env* e = (js_env*)calloc(1, sizeof *e); - napi_set_instance_data(env, e, js_env_free, NULL); + js_env_init(env); napi_property_descriptor props[] = { JS_RUNTIME_EXPORTS, - {"nested_modules_graphics_checksum", NULL, nx_nested_modules_graphics_checksum, NULL, NULL, NULL, napi_default, NULL}, + {"nested_modules_graphics_contract", NULL, nx_nested_modules_graphics_contract, NULL, NULL, NULL, napi_default, NULL}, {"nested_modules_graphics_new_canvas", NULL, nx_nested_modules_graphics_new_canvas, NULL, NULL, NULL, napi_default, NULL}, {"nested_modules_graphics_shapes_make_shape", NULL, nx_nested_modules_graphics_shapes_make_shape, NULL, NULL, NULL, napi_default, NULL}, {"nested_modules_graphics_shapes_describe", NULL, nx_nested_modules_graphics_shapes_describe, NULL, NULL, NULL, napi_default, NULL}, diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__package_json.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__package_json.snap deleted file mode 100644 index 9ed96de0..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__package_json.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "nested_modules", - "version": "0.1.0", - "description": "Native bindings for nested_modules", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "binding.gyp", - "nested_modules_node.c", - "nested_modules.h" - ], - "gypfile": true, - "scripts": { - "install": "node-gyp rebuild" - }, - "engines": { - "node": ">=18" - } -} diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__runtime_js.snap deleted file mode 100644 index 97a25292..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class NestedModulesError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends NestedModulesError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new NestedModulesError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new NestedModulesError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new NestedModulesError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new NestedModulesError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__README_md.snap deleted file mode 100644 index 1519f91c..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__README_md.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# shapes - -Native bindings for shapes - -Node.js bindings for the `shapes` native library: an ES module over a small N-API addon (`shapes_node.c`) that `npm install` compiles with node-gyp. The addon links `shapes`; set `SHAPES_LIBRARY` to the library file (or its directory) when installing, or pass `--shapes-library=` to npm. - -```sh -SHAPES_LIBRARY=/path/to/libshapes.so npm install shapes -``` - -Each top-level module of the API is a namespace export: - -```js -import { shapes } from 'shapes'; -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__binding_gyp.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__binding_gyp.snap deleted file mode 100644 index 2bfd672c..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__binding_gyp.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -{ - "variables": { - "native_dir%": " { const path = require('path'); const p = process.env.SHAPES_LIBRARY || process.env.npm_config_shapes_library; if (!p) return path.resolve('.'); return require('fs').statSync(p).isFile() ? path.dirname(path.resolve(p)) : path.resolve(p); })()\")" - }, - "targets": [ - { - "target_name": "shapes_node", - "sources": [ - "shapes_node.c" - ], - "include_dirs": [ - "." - ], - "library_dirs": [ - "<(native_dir)" - ], - "libraries": [ - "-lshapes" - ], - "conditions": [ - ["OS=='mac'", { "xcode_settings": { "OTHER_LDFLAGS": ["-Wl,-rpath,@loader_path", "-Wl,-rpath,<(native_dir)"] } }], - ["OS=='linux'", { "ldflags": ["-Wl,-rpath,'$$ORIGIN'", "-Wl,-rpath,<(native_dir)"] }] - ] - } - ] -} - -# END binding.gyp diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__index_d_ts.snap index 504ddfa2..d6ab315f 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_shapes__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -24,8 +24,9 @@ export declare class CancelledError extends ShapesError { /** The root error class, under a name no namespace member can shadow. */ declare const $Error: typeof ShapesError; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__index_js.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__index_js.snap index 60553893..cad4f052 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_shapes__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -32,22 +35,30 @@ export { ShapesError, CancelledError } from './runtime.js'; import { createRequire } from 'node:module'; -// The native entry points: the N-API addon `npm install` builds from -// shapes_node.c (with node-gyp), or a prebuilt copy next to this file. +// The native entry points: the N-API addon built from shapes_node.c. A prebuilt +// copy comes first (from the `shapes--` package `npm install` +// picks for this platform, or under prebuilds/-/), then the one +// node-gyp compiles at install time when none matches. function $loadAddon(name) { const require = createRequire(import.meta.url); - const tried = []; - for (const dir of ['./build/Release/', './build/Debug/', './']) { + const platform = `${process.platform}-${process.arch}`; + const candidates = [ + `shapes-${platform}/${name}`, + `./prebuilds/${platform}/${name}`, + `./build/Release/${name}`, + `./build/Debug/${name}`, + ]; + for (const candidate of candidates) { try { - return require(dir + name); + return require(candidate); } catch (e) { if (e.code !== 'MODULE_NOT_FOUND') throw e; - tried.push(dir + name); } } throw new Error( - `the native addon ${name} was not found (tried ${tried.join(', ')}); ` + - 'build it with `npm install` or `node-gyp rebuild` in this package', + `the native addon ${name} was not found (tried ${candidates.join(', ')}); ` + + 'install the package for this platform, or build the addon with `npm install` ' + + 'or `node-gyp rebuild` in this package', ); } @@ -57,12 +68,27 @@ $raw.$setup($Fault); // Object tokens in value buffers are already bigint handles here. const $token = (t) => t; -$verify($raw, 'shapes', 'shapes', 3, [['shapes', 0x06acf433ffa2a0f0n]]); +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['shapes', [ + [0x0d7b3eec49243e7an, 0x5f232311808e1b1fn, 'shapes.Channel'], + [0x183fa8198100860en, 0xe40c6993cb31d359n, 'shapes.describe'], + [0x2d1395eb837b848en, 0x42202a0c12f3a16en, 'shapes.sum_bytes'], + [0x7f9a5b87ba09b86bn, 0xc5b32fb0efbbaa17n, 'shapes.scale'], + [0xc8edd779240fda17n, 0x68e1be78fddb20e4n, 'shapes.Numerics'], + [0xd3d3881b2fd18a62n, 0xf24d0eb76a99d918n, 'shapes.make_numerics'], + [0xf010844afff5ac3cn, 0x50de28ae9b41ff15n, 'shapes.Shape'], + ]], +]; +$verify($raw, 'shapes', 'shapes', 4, $contract); /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.shapes_debug_live(kind); diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__package_json.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__package_json.snap deleted file mode 100644 index 6487043e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__package_json.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "shapes", - "version": "0.1.0", - "description": "Native bindings for shapes", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "binding.gyp", - "shapes_node.c", - "shapes.h" - ], - "gypfile": true, - "scripts": { - "install": "node-gyp rebuild" - }, - "engines": { - "node": ">=18" - } -} diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__runtime_js.snap deleted file mode 100644 index 323cda6a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class ShapesError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends ShapesError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new ShapesError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new ShapesError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new ShapesError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new ShapesError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_h.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_h.snap deleted file mode 100644 index df20c3f5..00000000 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_h.snap +++ /dev/null @@ -1,151 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef SHAPES_H -#define SHAPES_H - -#include -#include -#include - -#ifndef SHAPES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef SHAPES_BUILD -# define SHAPES_API __declspec(dllexport) -# else -# define SHAPES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define SHAPES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define SHAPES_API __attribute__((visibility("default"))) -# else -# define SHAPES_API -# endif -#endif - -#ifndef SHAPES_DEPRECATED -# if defined(_MSC_VER) -# define SHAPES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define SHAPES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define SHAPES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports shapes_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define SHAPES_ABI_VERSION 3u -SHAPES_API uint32_t shapes_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by shapes_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct shapes_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} shapes_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -SHAPES_API void shapes_error_set(shapes_error* err, int32_t code, const char* message); -SHAPES_API void shapes_error_clear(shapes_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - shapes_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -SHAPES_API void shapes_error_free(shapes_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -SHAPES_API void shapes_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct shapes_cancel_token shapes_cancel_token; -SHAPES_API shapes_cancel_token* shapes_cancel_token_create(void); -SHAPES_API void shapes_cancel_token_cancel(shapes_cancel_token* token); -SHAPES_API bool shapes_cancel_token_is_cancelled(const shapes_cancel_token* token); -SHAPES_API void shapes_cancel_token_destroy(shapes_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -SHAPES_API uint8_t* shapes_alloc(uint32_t size); -SHAPES_API void shapes_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -SHAPES_API uint64_t shapes_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define SHAPES_SHAPES_CHECKSUM 0x06acf433ffa2a0f0ull -SHAPES_API uint64_t shapes_shapes_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with shapes_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * shapes_buffer.h declares a C struct and codec for each. - */ - -/** An algebraic shape (sum type with associated data) */ -typedef enum { - /** The empty shape (a unit variant) */ - shapes_shapes_Shape_Empty = 0, - /** A circle with a radius */ - shapes_shapes_Shape_Circle = 1, - /** An axis-aligned rectangle */ - shapes_shapes_Shape_Rectangle = 2, - /** An integer-bounded shape (exercises narrow + wide ints) */ - shapes_shapes_Shape_Bounded = 3, - /** A labeled shape with a small count */ - shapes_shapes_Shape_Labeled = 4 -} shapes_shapes_Shape_Tag; -/** A plain C-style enum (no payloads) */ -typedef enum { shapes_shapes_Channel_Red = 0, shapes_shapes_Channel_Green = 1, shapes_shapes_Channel_Blue = 2 } shapes_shapes_Channel; - -// Module: shapes -/** Render a shape to a string */ -SHAPES_API const uint8_t* shapes_shapes_describe(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, shapes_error* out_err); -/** Scale a shape by a factor, returning a new shape */ -SHAPES_API const uint8_t* shapes_shapes_scale(const uint8_t* shape_ptr, size_t shape_len, double factor, size_t* out_len, shapes_error* out_err); -/** Sum a list of bytes into a wide integer (numerics smoke) */ -SHAPES_API uint64_t shapes_shapes_sum_bytes(const uint8_t* values_ptr, size_t values_len, shapes_error* out_err); -/** Build a Numerics struct from every numeric primitive */ -SHAPES_API const uint8_t* shapes_shapes_make_numerics(int8_t a, int16_t b, int32_t c, int64_t d, uint8_t e, uint16_t f, uint32_t g, uint64_t h, float i, double j, size_t* out_len, shapes_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // SHAPES_H - -// END shapes.h diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap index 06f456da..11c4480f 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate #ifndef NAPI_VERSION #define NAPI_VERSION 8 @@ -12,6 +12,7 @@ source: crates/weaveffi-cli/tests/snapshots.rs #include #include #include +#include #include #include @@ -26,8 +27,9 @@ source: crates/weaveffi-cli/tests/snapshots.rs * raw convention the shared JavaScript layer documents: direct values as * numbers, bigints, and booleans; strings as strings; bytes and value * buffers as Uint8Arrays; objects, iterators, and cancel tokens as bigint - * handles (null when absent); callback interfaces as adapter objects. A - * failure throws an instance of the runtime's `$Fault` class. + * handles (null when absent); callback interfaces as adapter objects (null + * for an absent optional one). A failure throws an instance of the + * runtime's `$Fault` class. */ typedef shapes_error js_error; @@ -35,23 +37,106 @@ typedef shapes_error js_error; /* Helpers an API may not need. */ #if defined(__GNUC__) || defined(__clang__) #define JS_HELPER static inline __attribute__((unused)) +#define JS_TLS __thread #else #define JS_HELPER static inline +#define JS_TLS __declspec(thread) #endif +/* --------------------------------------------------------------------------- + * JavaScript threads. One `js_thread` per thread that runs JavaScript and + * loaded the addon (the main thread and each worker). It counts the + * synchronous calls into the library in progress on that thread, which a + * callback made on another thread consults while it waits for the + * JavaScript thread (see js_cb_hop). Each environment and each callback + * registration holds a reference. + * ------------------------------------------------------------------------- */ + +typedef struct { + uv_mutex_t mu; + uv_cond_t cv; /* broadcast when a call begins or ends, or a hop finishes */ + unsigned depth; /* synchronous calls into the library in progress */ + uint64_t calls; /* outermost synchronous calls begun so far */ + unsigned waiters; /* producer threads waiting in js_cb_hop */ + unsigned refs; +} js_thread; + +/* The state of the JavaScript thread this is, or NULL on any other thread. */ +static JS_TLS js_thread* js_current = NULL; + +static js_thread* js_thread_new(void) { + js_thread* t = (js_thread*)calloc(1, sizeof *t); + uv_mutex_init(&t->mu); + uv_cond_init(&t->cv); + t->refs = 1; + return t; +} + +static void js_thread_retain(js_thread* t) { + uv_mutex_lock(&t->mu); + t->refs++; + uv_mutex_unlock(&t->mu); +} + +static void js_thread_release(js_thread* t) { + uv_mutex_lock(&t->mu); + bool last = --t->refs == 0; + uv_mutex_unlock(&t->mu); + if (!last) return; + uv_cond_destroy(&t->cv); + uv_mutex_destroy(&t->mu); + free(t); +} + +/* Bracket every synchronous call into the library. */ +JS_HELPER void js_sync_begin(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + if (t->depth++ == 0) t->calls++; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +JS_HELPER void js_sync_end(void) { + js_thread* t = js_current; + if (t == NULL) return; + uv_mutex_lock(&t->mu); + t->depth--; + if (t->waiters > 0) uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + /* Per-environment state (one per main thread or worker): the `$Fault` - * constructor `index.js` registers through `$setup`. */ + * constructor `index.js` registers through `$setup`, and the thread. */ typedef struct { napi_ref fault; + js_thread* thread; } js_env; static void js_env_free(napi_env env, void* data, void* hint) { (void)hint; js_env* e = (js_env*)data; if (e->fault != NULL) napi_delete_reference(env, e->fault); + if (js_current == e->thread) js_current = NULL; + js_thread_release(e->thread); free(e); } +/* Set up the environment of the thread loading the addon. */ +static void js_env_init(napi_env env) { + js_env* e = (js_env*)calloc(1, sizeof *e); + e->thread = js_thread_new(); + js_current = e->thread; + napi_set_instance_data(env, e, js_env_free, NULL); +} + +static js_env* js_env_of(napi_env env) { + js_env* e = NULL; + napi_get_instance_data(env, (void**)&e); + return e; +} + JS_HELPER napi_value js_undefined(napi_env env) { napi_value v; napi_get_undefined(env, &v); @@ -62,13 +147,22 @@ static napi_value js_setup(napi_env env, napi_callback_info info) { size_t argc = 1; napi_value argv[1]; napi_get_cb_info(env, info, &argc, argv, NULL, NULL); - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); + js_env* e = js_env_of(env); if (e->fault != NULL) napi_delete_reference(env, e->fault); napi_create_reference(env, argv[0], 1, &e->fault); return js_undefined(env); } +/* The `$Fault` constructor, or NULL before `$setup`. */ +static napi_value js_fault_class(napi_env env) { + js_env* e = js_env_of(env); + napi_value ctor = NULL; + if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok) { + return NULL; + } + return ctor; +} + /* --------------------------------------------------------------------------- * Value conversions * ------------------------------------------------------------------------- */ @@ -149,6 +243,21 @@ JS_HELPER napi_value js_take_bytes(napi_env env, const uint8_t* ptr, size_t len) return out; } +/* A module's contract table as a BigUint64Array of `id, hash` pairs. */ +JS_HELPER napi_value js_new_contract(napi_env env, const shapes_contract_entry* table, + size_t len) { + napi_value ab, out; + void* data = NULL; + napi_create_arraybuffer(env, len * 2 * sizeof(uint64_t), &data, &ab); + uint64_t* words = (uint64_t*)data; + for (size_t i = 0; i < len; i++) { + words[2 * i] = table[i].id; + words[2 * i + 1] = table[i].hash; + } + napi_create_typedarray(env, napi_biguint64_array, len * 2, ab, 0, &out); + return out; +} + JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, const uint8_t* payload, size_t payload_len) { napi_value argv[3], ctor, out = NULL; @@ -159,10 +268,8 @@ JS_HELPER napi_value js_fault(napi_env env, int32_t code, const char* message, } else { napi_get_null(env, &argv[2]); } - js_env* e = NULL; - napi_get_instance_data(env, (void**)&e); - if (e == NULL || e->fault == NULL || napi_get_reference_value(env, e->fault, &ctor) != napi_ok || - ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { + ctor = js_fault_class(env); + if (ctor == NULL || napi_new_instance(env, ctor, 3, argv, &out) != napi_ok) { napi_create_error(env, NULL, argv[1], &out); napi_set_named_property(env, out, "code", argv[0]); } @@ -428,30 +535,42 @@ JS_HELPER void js_async_done(js_async* a, js_error* err) { * producer: a reference to the adapter object, the JavaScript thread it * belongs to, and a thread-safe function that runs calls made from any other * thread on that thread while the caller waits. + * + * Waiting is what makes a call from another thread safe, and also what could + * deadlock it: if the JavaScript thread is itself inside a synchronous call + * into the library that waits for the calling thread, neither can proceed. + * A waiting call therefore watches the JavaScript thread. While that thread + * stays inside one synchronous call for JS_CB_DEADLOCK_MS without having + * started the callback, the wait is abandoned and the callback fails with + * code -4 instead of hanging (a shorter synchronous call, or one that + * returns to the event loop, just delays the callback). * ------------------------------------------------------------------------- */ +#define JS_CB_DEADLOCK_MS 1000 + typedef struct { napi_env env; napi_ref ref; napi_threadsafe_function tsfn; uv_thread_t js_thread; + js_thread* thread; } js_cb; -/* Every callback frame starts with this header so the dispatcher can reach - * `out_err` without knowing the method. */ -typedef struct { - js_error* out_err; -} js_cb_frame; +typedef enum { JS_REQ_QUEUED, JS_REQ_RUNNING, JS_REQ_DONE, JS_REQ_ABANDONED } js_req_state; /* A call queued from another thread: `method` is the vtable index, or -1 to - * release the registration. */ + * release the registration; `frame` holds the method's slots. The request is + * heap-allocated and holds its own reference to the thread state, so a + * caller that abandons it can return (and the producer can then release the + * registration) while it is still queued: the dispatcher frees it without + * touching the frame or the registration. */ typedef struct { js_cb* cb; + js_thread* thread; int method; void* frame; - uv_mutex_t mu; - uv_cond_t cv; - bool done; + js_error* out_err; + js_req_state state; } js_cb_req; JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { @@ -460,10 +579,12 @@ JS_HELPER bool js_cb_on_js_thread(js_cb* cb) { } JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* name, - napi_threadsafe_function_call_js dispatch) { + napi_threadsafe_function_call_js dispatch) { js_cb* cb = (js_cb*)calloc(1, sizeof *cb); cb->env = env; cb->js_thread = uv_thread_self(); + cb->thread = js_env_of(env)->thread; + js_thread_retain(cb->thread); napi_create_reference(env, adapter, 1, &cb->ref); napi_value resource; napi_create_string_utf8(env, name, NAPI_AUTO_LENGTH, &resource); @@ -477,14 +598,20 @@ JS_HELPER js_cb* js_cb_register(napi_env env, napi_value adapter, const char* na JS_HELPER void js_cb_release(napi_env env, js_cb* cb) { if (env != NULL) napi_delete_reference(env, cb->ref); napi_release_threadsafe_function(cb->tsfn, napi_tsfn_release); + js_thread_release(cb->thread); free(cb); } -/* A callback-interface argument: the adapter object `index.js` built. */ +/* A callback-interface argument: the adapter object `index.js` built, or + * null for an absent optional one. */ JS_HELPER bool js_arg_cb(napi_env env, napi_value v, const char* name, - napi_threadsafe_function_call_js dispatch, js_cb** out) { + napi_threadsafe_function_call_js dispatch, bool nullable, js_cb** out) { napi_valuetype t; napi_typeof(env, v, &t); + if (nullable && (t == napi_null || t == napi_undefined)) { + *out = NULL; + return true; + } if (t != napi_object) return js_type_error(env, "expected a callback interface implementation"); *out = js_cb_register(env, v, name, dispatch); return true; @@ -505,62 +632,188 @@ JS_HELPER void js_cb_free(void* ctx) { if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) free(req); } -/* Run `req` on the JavaScript thread and wait for it. */ -JS_HELPER void js_cb_hop(js_cb_req* req) { - uv_mutex_init(&req->mu); - uv_cond_init(&req->cv); - req->done = false; - if (napi_call_threadsafe_function(req->cb->tsfn, req, napi_tsfn_blocking) == napi_ok) { - uv_mutex_lock(&req->mu); - while (!req->done) uv_cond_wait(&req->cv, &req->mu); - uv_mutex_unlock(&req->mu); - } else { - shapes_error_set(((js_cb_frame*)req->frame)->out_err, -4, - "the callback implementation is no longer reachable"); +/* Run method `method` of `cb` on its JavaScript thread and wait for it, + * failing it with code -4 rather than deadlocking (see above). `what` names + * the method in that failure. */ +JS_HELPER void js_cb_hop(js_cb* cb, int method, void* frame, js_error* out_err, + const char* what) { + js_thread* t = cb->thread; + js_cb_req* req = (js_cb_req*)calloc(1, sizeof *req); + req->cb = cb; + req->thread = t; + js_thread_retain(t); + req->method = method; + req->frame = frame; + req->out_err = out_err; + req->state = JS_REQ_QUEUED; + uv_mutex_lock(&t->mu); + t->waiters++; + uv_mutex_unlock(&t->mu); + if (napi_call_threadsafe_function(cb->tsfn, req, napi_tsfn_nonblocking) != napi_ok) { + uv_mutex_lock(&t->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); + shapes_error_set(out_err, -4, "the callback implementation is no longer reachable"); + return; + } + const uint64_t limit = (uint64_t)JS_CB_DEADLOCK_MS * 1000000u; + bool blocked = false; + uint64_t call = 0, since = 0; + uv_mutex_lock(&t->mu); + while (req->state != JS_REQ_DONE) { + if (req->state == JS_REQ_QUEUED && t->depth > 0) { + uint64_t now = uv_hrtime(); + if (!blocked || call != t->calls) { + blocked = true; + call = t->calls; + since = now; + } + if (now - since >= limit) { + req->state = JS_REQ_ABANDONED; /* the dispatcher frees it */ + t->waiters--; + uv_mutex_unlock(&t->mu); + char msg[512]; + snprintf(msg, sizeof msg, + "%s was called on another thread while the JavaScript thread was blocked in a " + "synchronous call into the library for %d ms; the call likely waits for this " + "callback, so waiting longer would deadlock (call it asynchronously, or call " + "back on the calling thread)", + what, JS_CB_DEADLOCK_MS); + shapes_error_set(out_err, -4, msg); + return; + } + uv_cond_timedwait(&t->cv, &t->mu, limit - (now - since)); + } else { + blocked = false; + uv_cond_wait(&t->cv, &t->mu); + } } - uv_cond_destroy(&req->cv); - uv_mutex_destroy(&req->mu); + t->waiters--; + uv_mutex_unlock(&t->mu); + free(req); + js_thread_release(t); +} + +/* The dispatcher's side of a hop. js_cb_take claims a queued request (false + * when its caller gave up, in which case the request is freed); + * js_cb_finish reports it done. */ +JS_HELPER bool js_cb_take(js_cb_req* req) { + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + bool abandoned = req->state == JS_REQ_ABANDONED; + if (!abandoned) req->state = JS_REQ_RUNNING; + uv_mutex_unlock(&t->mu); + if (abandoned) { + free(req); + js_thread_release(t); + } + return !abandoned; } JS_HELPER void js_cb_finish(js_cb_req* req) { - uv_mutex_lock(&req->mu); - req->done = true; - uv_cond_signal(&req->cv); - uv_mutex_unlock(&req->mu); + js_thread* t = req->thread; + uv_mutex_lock(&t->mu); + req->state = JS_REQ_DONE; + uv_cond_broadcast(&t->cv); + uv_mutex_unlock(&t->mu); +} + +/* A JavaScript value's text as a heap-allocated C string (NULL on failure). */ +JS_HELPER char* js_cstring(napi_env env, napi_value v) { + napi_value str; + size_t len = 0; + if (napi_coerce_to_string(env, v, &str) != napi_ok || + napi_get_value_string_utf8(env, str, NULL, 0, &len) != napi_ok) { + return NULL; + } + char* out = (char*)malloc(len + 1); + napi_get_value_string_utf8(env, str, out, len + 1, NULL); + return out; } -/* Report the pending JavaScript exception (or `fallback`) through `out_err` - * as a foreign failure (code -4); nothing unwinds through the C frame. */ +/* Report the pending JavaScript exception (or `fallback`) through + * `out_err`; nothing unwinds through the C frame. A `$Fault` (a domain error + * a throwing method raised) reports its code, message, and payload; any + * other exception is a foreign failure, code -4, with its message. */ JS_HELPER void js_cb_report(napi_env env, js_error* out_err, const char* fallback) { - char msg[512]; - msg[0] = 0; + int32_t code = -4; + char* msg = NULL; + const uint8_t* payload = NULL; + size_t payload_len = 0; bool pending = false; napi_is_exception_pending(env, &pending); if (pending) { - napi_value exc, text, str; + napi_value exc, text; napi_get_and_clear_last_exception(env, &exc); + napi_value fault = js_fault_class(env); + bool is_fault = false; + if (fault != NULL) napi_instanceof(env, exc, fault, &is_fault); napi_valuetype t; napi_typeof(env, exc, &t); text = exc; if (t == napi_object) napi_get_named_property(env, exc, "message", &text); - if (napi_coerce_to_string(env, text, &str) == napi_ok) { - napi_get_value_string_utf8(env, str, msg, sizeof msg, NULL); + msg = js_cstring(env, text); + if (is_fault) { + napi_value v; + napi_get_named_property(env, exc, "code", &v); + napi_get_value_int32(env, v, &code); + napi_get_named_property(env, exc, "payload", &v); + bool typed = false; + napi_is_typedarray(env, v, &typed); + if (typed) { + napi_typedarray_type type; + void* data = NULL; + napi_get_typedarray_info(env, v, &type, &payload_len, &data, NULL, NULL); + payload = (const uint8_t*)data; + } } napi_is_exception_pending(env, &pending); if (pending) napi_get_and_clear_last_exception(env, &exc); } - shapes_error_set(out_err, -4, msg[0] != 0 ? msg : fallback); + shapes_error_set(out_err, code, msg != NULL && msg[0] != 0 ? msg : fallback); + if (payload != NULL) shapes_error_set_payload(out_err, payload, payload_len); + free(msg); } /* Look up and call `method` on the adapter; false when it threw. */ JS_HELPER bool js_cb_call(napi_env env, js_cb* cb, const char* method, size_t argc, - const napi_value* argv, napi_value* result) { + const napi_value* argv, napi_value* result) { napi_value adapter, fn; napi_get_reference_value(env, cb->ref, &adapter); napi_get_named_property(env, adapter, method, &fn); return napi_call_function(env, adapter, fn, argc, argv, result) == napi_ok; } +/* Hand `len` bytes to the producer through a callback's out slots, as a run + * from {p}_alloc that the producer adopts. */ +JS_HELPER void js_give(const void* data, size_t len, uint8_t** out_ptr, size_t* out_len) { + uint8_t* run = shapes_alloc(len); + if (len > 0) memcpy(run, data, len); + *out_ptr = run; + *out_len = len; +} + +/* A callback's string return: false (with a pending exception) when the + * value isn't a string. */ +JS_HELPER bool js_ret_str(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + js_str s = JS_STR_INIT; + if (!js_arg_str(env, v, &s)) return false; + js_give(s.ptr, s.len, out_ptr, out_len); + js_str_free(&s); + return true; +} + +/* A callback's bytes or buffer return: a Uint8Array. */ +JS_HELPER bool js_ret_bytes(napi_env env, napi_value v, uint8_t** out_ptr, size_t* out_len) { + const uint8_t* ptr = NULL; + size_t len = 0; + if (!js_arg_bytes(env, v, &ptr, &len)) return false; + js_give(ptr, len, out_ptr, out_len); + return true; +} + /* --------------------------------------------------------------------------- * The runtime symbols every library exports. * ------------------------------------------------------------------------- */ @@ -609,9 +862,11 @@ static napi_value js_cancel_token_destroy(napi_env env, napi_callback_info info) {"shapes_cancel_token_destroy", NULL, js_cancel_token_destroy, NULL, NULL, NULL, \ napi_default, NULL} -static napi_value nx_shapes_shapes_checksum(napi_env env, napi_callback_info info) { +static napi_value nx_shapes_shapes_contract(napi_env env, napi_callback_info info) { (void)info; - return js_new_u64(env, shapes_shapes_checksum()); + size_t len = 0; + const shapes_contract_entry* table = shapes_shapes_contract(&len); + return js_new_contract(env, table, len); } static napi_value nx_shapes_shapes_describe(napi_env env, napi_callback_info info) { @@ -623,7 +878,9 @@ static napi_value nx_shapes_shapes_describe(napi_env env, napi_callback_info inf { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = shapes_shapes_describe(a0, a0_len, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -646,7 +903,9 @@ static napi_value nx_shapes_shapes_scale(napi_env env, napi_callback_info info) { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = shapes_shapes_scale(a0, a0_len, a1, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -666,7 +925,9 @@ static napi_value nx_shapes_shapes_sum_bytes(napi_env env, napi_callback_info in if (!js_arg_bytes(env, argv[0], &a0, &a0_len)) goto done; { js_error err = {0}; + js_sync_begin(); uint64_t r = shapes_shapes_sum_bytes(a0, a0_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -704,7 +965,9 @@ static napi_value nx_shapes_shapes_make_numerics(napi_env env, napi_callback_inf { js_error err = {0}; size_t out_len = 0; + js_sync_begin(); const uint8_t* r = shapes_shapes_make_numerics(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, &out_len, &err); + js_sync_end(); if (err.code != 0) { js_throw(env, &err); goto done; @@ -717,11 +980,10 @@ done: } NAPI_MODULE_INIT() { - js_env* e = (js_env*)calloc(1, sizeof *e); - napi_set_instance_data(env, e, js_env_free, NULL); + js_env_init(env); napi_property_descriptor props[] = { JS_RUNTIME_EXPORTS, - {"shapes_shapes_checksum", NULL, nx_shapes_shapes_checksum, NULL, NULL, NULL, napi_default, NULL}, + {"shapes_shapes_contract", NULL, nx_shapes_shapes_contract, NULL, NULL, NULL, napi_default, NULL}, {"shapes_shapes_describe", NULL, nx_shapes_shapes_describe, NULL, NULL, NULL, napi_default, NULL}, {"shapes_shapes_scale", NULL, nx_shapes_shapes_scale, NULL, NULL, NULL, napi_default, NULL}, {"shapes_shapes_sum_bytes", NULL, nx_shapes_shapes_sum_bytes, NULL, NULL, NULL, napi_default, NULL}, diff --git a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__README_md.snap deleted file mode 100644 index 89f0942d..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__README_md.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# docs_everywhere (Python) - -Python bindings for the `docs_everywhere` native library, built on `ctypes`. - -## Requirements - -- Python 3.9 or later -- The native library (`libdocs_everywhere.so`, `libdocs_everywhere.dylib`, or `docs_everywhere.dll`) - -## Install - -```bash -pip install . -``` - -## Loading the native library - -The bindings load the library from `DOCS_EVERYWHERE_LIBRARY` when it is set (a full path), -then from inside the installed package, then from the system loader's search -path. - -## Usage - -```python -import docs_everywhere -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere___init___py.snap b/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere___init___py.snap deleted file mode 100644 index ede48601..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere___init___py.snap +++ /dev/null @@ -1,10 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -from .docs_everywhere import * # noqa: F401,F403 - -# END __init__.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_py.snap b/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_py.snap index 5a8ae4bc..6d85d09b 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_py.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_py.snap @@ -1,11 +1,13 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate """ctypes bindings for the docs_everywhere native library.""" +from __future__ import annotations + import abc import asyncio import builtins @@ -17,18 +19,19 @@ import threading import warnings from dataclasses import dataclass from enum import IntEnum -from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type +from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type, TypeVar class Error(Exception): - """An error reported by the native library. - - `code` is positive for one of the API's declared error codes (raised as - the declaring module's typed subclasses) and negative for a runtime trap: - a generic failure (-1), a producer panic (-2), a marshalling failure - (-3), or an exception raised by a callback-interface implementation - (-4). A cancelled async call (-5) raises `asyncio.CancelledError` - instead. + """An error a fallible call reports. + + `code` is positive for one of the API's declared error codes, raised as + the declaring module's typed subclasses, and negative for a runtime + failure of a call that declares errors: a generic failure (-1), a + producer panic (-2), a marshalling failure (-3), or an exception raised + by a callback-interface implementation (-4). A cancelled async call (-5) + raises `asyncio.CancelledError` instead. A call that declares no errors + never raises this; its failures are bugs and raise `InternalError`. """ GENERIC_ERROR_CODE = -1 @@ -42,6 +45,24 @@ class Error(Exception): self.code = code self.message = message + def _payload(self) -> bytes: + """The code's fields as a value buffer (a code with fields overrides + this).""" + return b"" + + +class InternalError(RuntimeError): + """A failure of a call that declares no errors: a producer panic (-2), a + marshalling failure (-3), a callback-interface failure the producer let + through (-4), or another runtime failure (-1). These are bugs, so they + are unchecked; `code` and `message` say what went wrong. + """ + + def __init__(self, code: int, message: str) -> None: + super().__init__(f"({code}) {message}") + self.code = code + self.message = message + class _ErrorStruct(ctypes.Structure): """The C `docs_everywhere_error` out-parameter every call reports through.""" @@ -91,18 +112,40 @@ def _bind(name: str, restype: Any, *argtypes: Any) -> Any: return fn -# The C ABI revision these bindings were generated against, and the contract -# checksum of every top-level module: (module, checksum symbol, value). -_ABI_VERSION = 3 -_CHECKSUMS: List[Tuple[str, str, int]] = [ - ("docs", "docs_everywhere_docs_checksum", 0x7950eec1e9cfc819), -] +class _ContractEntry(ctypes.Structure): + """The C `docs_everywhere_contract_entry`: one declaration's id and hash.""" + + _fields_ = [("id", ctypes.c_uint64), ("hash", ctypes.c_uint64)] + + +# The C ABI revision these bindings were generated against and, per +# top-level module's contract symbol, the declarations they rely on: +# (id, hash, dotted path). +_ABI_VERSION = 4 +_CONTRACTS: Dict[str, List[Tuple[int, int, str]]] = { + "docs_everywhere_docs_contract": [ + (0x00b2187fd75634ec, 0x2263c6f787bf6081, "docs.publish_async"), + (0x12fc242b9dc6444c, 0x96405f484a642b31, "docs.Notebook.insert"), + (0x2213822ed05cd5bd, 0x26c5833392aa63ce, "docs.Notebook.new"), + (0x2bd762bcdaa207ff, 0x1b0ebab2d04c49b0, "docs.SavedListener"), + (0x2e086c45e9d617de, 0x4d43bf32358c01ff, "docs.Notebook.find"), + (0x4817dc98197ad960, 0xf8017b5a7d58b2c7, "docs.Notebook.size"), + (0x614de47452d5a55c, 0xd9aae1472494ffef, "docs.get_document"), + (0x67c5cb9fc119bcd5, 0x08d5b25a1a22a7fa, "docs.Document"), + (0x8d484bf2d78c9b70, 0xe9093cf96912194c, "docs.Notebook.max_documents"), + (0xaea61d6451691c67, 0xe4704128b3524b3d, "docs.watch"), + (0xd68d6ad1540cdec5, 0x01afeb40b0a8bfe1, "docs.Notebook"), + (0xdb0601444a4760cc, 0xe56c69629b444797, "docs.create_document"), + (0xe12d6f4838347b3e, 0xa9bab86646d84f51, "docs.Status"), + (0xee2f8d25b3313b50, 0x9691232f35439a2b, "docs.DocsErrors"), + ], +} def _check_contract() -> None: - """Refuse to load a library built for a different ABI revision or a - different version of any module's API, before any other symbol is - bound.""" + """Refuse to load a library built for another ABI revision, or one that + lacks or changed a declaration these bindings use, before any other + symbol is bound.""" try: abi_version = _bind("docs_everywhere_abi_version", ctypes.c_uint32) except AttributeError: @@ -116,33 +159,40 @@ def _check_contract() -> None: f"C ABI mismatch: these bindings expect revision {_ABI_VERSION}, " f"but {_lib._name} reports revision {found}" ) - for module, symbol, expected in _CHECKSUMS: + for symbol, expected in _CONTRACTS.items(): try: - checksum = _bind(symbol, ctypes.c_uint64) - except AttributeError: - raise ImportError( - f"module {module!r}: {_lib._name} does not export {symbol}" - ) from None - found = checksum() - if found != expected: - raise ImportError( - f"module {module!r}: the native library's contract checksum " - f"{found:#018x} does not match these bindings ({expected:#018x}); " - "regenerate the bindings from the library's current API" + contract = _bind( + symbol, ctypes.POINTER(_ContractEntry), ctypes.POINTER(ctypes.c_size_t) ) + except AttributeError: + raise ImportError(f"{_lib._name} does not export {symbol}") from None + length = ctypes.c_size_t() + table = contract(ctypes.byref(length)) + hashes = {table[i].id: table[i].hash for i in range(length.value)} if table else {} + for entry_id, entry_hash, path in expected: + actual = hashes.get(entry_id) + if actual is None: + raise ImportError(f"{path} is missing from the library ({_lib._name})") + if actual != entry_hash: + raise ImportError( + f"{path} changed since these bindings were generated ({_lib._name})" + ) _check_contract() _error_clear = _bind("docs_everywhere_error_clear", None, ctypes.POINTER(_ErrorStruct)) _free_bytes = _bind("docs_everywhere_free_bytes", None, ctypes.c_void_p, ctypes.c_size_t) +_debug_live_fn = _bind("docs_everywhere_debug_live", ctypes.c_uint64, ctypes.c_int32) def _debug_live(kind: int) -> int: - """The producer's live-allocation counter `kind` (0 objects, 1 callbacks, - 2 iterators, 3 cancel tokens, 4 returned allocations). Always 0 unless - the producer was built with its leak counters enabled.""" - return _bind("docs_everywhere_debug_live", ctypes.c_uint64, ctypes.c_int32)(kind) + """The producer's live-resource counter `kind` (0 objects, 1 callbacks, + 2 iterators, 3 cancel tokens, 4 byte runs; -1 is 1 when the producer + counts at all). Always 0 unless the producer was built with its leak + counters enabled.""" + live: int = _debug_live_fn(kind) + return live def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: @@ -155,17 +205,9 @@ def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: return code, message, payload -def _error_from(code: int, message: str, payload: bytes = b"") -> Error: - """The exception for a code no declared error domain claims.""" - return Error(code, message) - - -def _check_error(err: _ErrorStruct) -> None: - """Raise for a non-zero out-err slot of a call that declares no errors: - only runtime traps (a panic, a marshalling failure, a failed callback) - land here.""" - if err.code: - raise _error_from(*_read_error(err)) +def _trap_from(code: int, message: str, payload: bytes = b"") -> InternalError: + """The exception for a failed call that declares no errors.""" + return InternalError(code, message) def _take_bytes(ptr: Optional[int], length: int) -> bytes: @@ -189,6 +231,9 @@ def _peek_bytes(ptr: Optional[int], length: int) -> bytes: return ctypes.string_at(ptr, length) if ptr and length else b"" +_H = TypeVar("_H", bound="_Handle") + + class _Handle: """Owns one native reference and releases it exactly once. @@ -202,12 +247,13 @@ class _Handle: def _init_handle(self, ptr: int) -> None: self._ptr: Optional[int] = ptr + self._key = ptr self._calls = 0 self._closing = False self._lock = threading.Lock() @classmethod - def _adopt(cls, ptr: int) -> Any: + def _adopt(cls: Type[_H], ptr: int) -> _H: """Wrap one strong reference the producer handed over.""" obj = cls.__new__(cls) obj._init_handle(ptr) @@ -217,7 +263,7 @@ class _Handle: with self._lock: ptr = self._ptr if ptr is None or self._closing: - raise Error(-1, f"{type(self).__name__} used after close()") + raise ValueError(f"{type(self).__name__} is closed") self._calls += 1 return ptr @@ -231,8 +277,8 @@ class _Handle: type(self)._destroy(ptr) def close(self) -> None: - """Release this wrapper's native reference. Idempotent; the wrapper - is unusable afterwards.""" + """Release the native reference. Idempotent; the wrapper is unusable + afterwards.""" lock = getattr(self, "_lock", None) if lock is None: return @@ -246,7 +292,7 @@ class _Handle: if ptr is not None: type(self)._destroy(ptr) - def __enter__(self) -> Any: + def __enter__(self: _H) -> _H: return self def __exit__(self, *exc: object) -> None: @@ -261,27 +307,36 @@ class _Handle: class _Object(_Handle): - """Base of every interface wrapper: a reference-counted native object.""" + """Base of every interface wrapper: a reference-counted native object. + Two wrappers are equal when they hold the same native object.""" # The native clone function, set per subclass. _clone: Callable[[int], int] def _clone_ref(self) -> int: """A new strong reference to the same object, for a receiver that - adopts it (an object token inside a value buffer).""" + adopts it (an object token, or a callback method's return).""" ptr = self._acquire() try: return type(self)._clone(ptr) finally: self._release() + def __eq__(self, other: object) -> bool: + if not isinstance(other, _Object): + return NotImplemented + return self is other or (self._ptr is not None and self._ptr == other._ptr) + + def __hash__(self) -> int: + return hash(self._key) + class _Iterator(_Handle): """Base of every lazy iterator over a producer stream: one native `next` call per step; the handle is released on exhaustion, `close()`, or garbage collection.""" - def __iter__(self) -> Any: + def __iter__(self: _H) -> _H: return self def _step(self) -> int: @@ -296,23 +351,23 @@ class _Iterator(_Handle): def _required(ptr: Optional[int]) -> int: """A non-null object pointer the producer returned.""" if not ptr: - raise Error(-1, "the producer returned a null object") + raise InternalError(-3, "the producer returned a null object") return ptr -def _lend(obj: Any, cls: "Type[_Object]") -> int: +def _lend(obj: Any, cls: Type[_Object]) -> int: """Lend `obj`'s native pointer to one call; pair with `obj._release()`.""" if not isinstance(obj, cls): raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") return obj._acquire() -def _lend_opt(obj: Any, cls: "Type[_Object]") -> Optional[int]: +def _lend_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: """`_lend` for an optional object: None lends a null pointer.""" return None if obj is None else _lend(obj, cls) -def _release_opt(obj: Any) -> None: +def _release_opt(obj: Optional[_Object]) -> None: if obj is not None: obj._release() @@ -328,9 +383,12 @@ _U64 = struct.Struct(" Error: - return Error(-3, f"malformed value buffer: {what}") +def _malformed(what: str) -> InternalError: + return InternalError(-3, f"malformed value buffer: {what}") class _Writer: @@ -413,7 +471,7 @@ class _Writer: self._buf += _U32.pack(len(values)) self._buf += struct.pack(f"<{len(values)}{code}", *values) - def write_object(self, obj: Any, cls: "Type[_Object]") -> None: + def write_object(self, obj: Any, cls: Type[_Object]) -> None: """Reserve an object token: one strong reference to `obj` the reader adopts, cloned in `finish()`. A closed wrapper is rejected now.""" if not isinstance(obj, cls): @@ -426,7 +484,8 @@ class _Writer: class _Reader: """Decodes values from the value-buffer wire format, rejecting truncated - buffers, invalid flag bytes, invalid UTF-8, and trailing data.""" + buffers, invalid flag bytes, invalid UTF-8, undeclared enum values, and + trailing data.""" __slots__ = ("_data", "_pos") @@ -499,6 +558,13 @@ class _Reader: except UnicodeDecodeError: raise _malformed("string is not valid UTF-8") from None + def read_enum(self, cls: Type[_E]) -> _E: + value = self.read_i32() + try: + return cls(value) + except ValueError: + raise _malformed(f"undeclared {cls.__name__} value {value}") from None + def read_numbers(self, code: str, size: int) -> List[Any]: """A list of one fixed-width numeric primitive, unpacked in one call.""" n = self.read_count() @@ -517,7 +583,7 @@ class _Reader: raise _malformed("trailing bytes") -def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: +def _decode(data: bytes, read: Callable[[_Reader], _T]) -> _T: """Decode exactly one value from `data`.""" r = _Reader(data) value = read(r) @@ -531,14 +597,14 @@ def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: # The trampolines live at module scope, so nothing a pending call needs can # be garbage collected before the producer fires the completion, even when # the awaiting coroutine is gone. -_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, "asyncio.Future[Any]"]] = {} +_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, asyncio.Future[Any]]] = {} _pending_lock = threading.Lock() _pending_next = 0 _error_free = _bind("docs_everywhere_error_free", None, ctypes.POINTER(_ErrorStruct)) -def _async_begin() -> Tuple[int, "asyncio.Future[Any]"]: +def _async_begin() -> Tuple[int, asyncio.Future[Any]]: """Register a new pending call on the running event loop.""" global _pending_next loop = asyncio.get_running_loop() @@ -583,7 +649,7 @@ def _async_settle(key: Optional[int], exc: Optional[BaseException], value: Any) pass -def _async_resolve(future: "asyncio.Future[Any]", exc: Optional[BaseException], value: Any) -> None: +def _async_resolve(future: asyncio.Future[Any], exc: Optional[BaseException], value: Any) -> None: if future.done(): return if isinstance(exc, asyncio.CancelledError): @@ -599,14 +665,23 @@ def _async_resolve(future: "asyncio.Future[Any]", exc: Optional[BaseException], # a call and removed when the producer calls the vtable's `free(ctx)`, so the # implementation lives exactly as long as the producer may call it and the # producer never holds a raw reference to a Python object. ctypes acquires -# the GIL on entry to a trampoline, so the producer may call from any thread. +# the GIL on entry to a trampoline, so the producer may call (and free) from +# any thread. _callbacks: Dict[int, Any] = {} _callbacks_lock = threading.Lock() _callbacks_next = 0 +_alloc = _bind("docs_everywhere_alloc", ctypes.c_void_p, ctypes.c_size_t) _error_set = _bind( "docs_everywhere_error_set", None, ctypes.POINTER(_ErrorStruct), ctypes.c_int32, ctypes.c_char_p ) +_error_set_payload = _bind( + "docs_everywhere_error_set_payload", + None, + ctypes.POINTER(_ErrorStruct), + ctypes.c_char_p, + ctypes.c_size_t, +) def _callback_register(impl: Any, cls: type) -> int: @@ -621,6 +696,11 @@ def _callback_register(impl: Any, cls: type) -> int: return ctx +def _callback_register_opt(impl: Any, cls: type) -> Optional[int]: + """`_callback_register` for an optional callback: None passes none.""" + return None if impl is None else _callback_register(impl, cls) + + def _callback_get(ctx: Optional[int]) -> Any: return _callbacks[ctx or 0] @@ -631,10 +711,53 @@ def _callback_free(ctx: Optional[int]) -> None: _callbacks.pop(ctx or 0, None) -def _callback_fail(out_err: Any, exc: BaseException) -> None: - """Report an implementation's exception to the producer, which aborts - its call with the foreign error code. Nothing unwinds through C.""" - message = str(exc) or type(exc).__name__ +# Every vtable's `free` entry: one function object for the process lifetime. +_CallbackFree = ctypes.CFUNCTYPE(None, ctypes.c_void_p) +_callback_free_fn = _CallbackFree(_callback_free) + + +def _callback_return_bytes(out_ptr: Any, out_len: Any, data: bytes) -> None: + """Hand a string, bytes, or value-buffer return to the producer: a run + allocated with docs_everywhere_alloc, which the producer adopts and frees.""" + n = len(data) + ptr = _alloc(n) if n else None + if n: + if not ptr: + raise MemoryError(f"docs_everywhere_alloc({n}) failed") + ctypes.memmove(ptr, data, n) + out_ptr[0] = ptr + out_len[0] = n + + +def _callback_return_object(obj: Any, cls: Type[_Object]) -> int: + """Hand an object return to the producer: one fresh strong reference it + adopts.""" + if not isinstance(obj, cls): + raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") + return obj._clone_ref() + + +def _callback_return_object_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: + """`_callback_return_object` for an optional object: None returns null.""" + return None if obj is None else _callback_return_object(obj, cls) + + +def _callback_fail(out_err: Any, exc: BaseException, domain: Optional[Type[Error]] = None) -> None: + """Report an implementation's exception to the producer. A method that + declares errors passes its module's error domain, and an exception of + that domain carrying a declared code travels as that code with its + fields; every other exception reaches the producer as the foreign error + code (-4). Nothing unwinds through C.""" + if domain is not None and isinstance(exc, domain) and exc.code > 0: + _error_set(out_err, exc.code, exc.message.encode("utf-8", "replace")) + payload = exc._payload() + if payload: + _error_set_payload(out_err, payload, len(payload)) + return + if isinstance(exc, (Error, InternalError)): + message = exc.message + else: + message = str(exc) or type(exc).__name__ _error_set(out_err, Error.FOREIGN_ERROR_CODE, message.encode("utf-8", "replace")) @@ -644,8 +767,8 @@ def _callback_fail(out_err: Any, exc: BaseException) -> None: class DocsErrorsError(Error): """Base exception for the `docs` module's error domain.""" - NotFound: "Type[NotFound]" - InvalidStatus: "Type[InvalidStatus]" + NotFound: Type[NotFound] + InvalidStatus: Type[InvalidStatus] class NotFound(DocsErrorsError): @@ -670,33 +793,23 @@ DocsErrorsError.NotFound = NotFound DocsErrorsError.InvalidStatus = InvalidStatus -_DOCS_ERRORS_ERROR_CODES: Dict[int, Any] = { - 1: NotFound, - 2: InvalidStatus, -} - - def _docs_errors_error_from(code: int, message: str, payload: bytes = b"") -> Error: - """Build the DocsErrorsError subclass matching `code`, or the plain - Error for codes outside the domain (panics, marshalling).""" - cls = _DOCS_ERRORS_ERROR_CODES.get(code) - if cls is None: - return Error(code, message) - exc: Any = cls(message) if message else cls() - return exc - - -def _check_docs_errors_error(err: _ErrorStruct) -> None: - if err.code: - raise _docs_errors_error_from(*_read_error(err)) + """The DocsErrorsError subclass for `code` with its payload fields, or the + plain Error for a runtime code (a panic, a marshalling failure). + """ + if code == 1: + return NotFound(message) + if code == 2: + return InvalidStatus(message) + return Error(code, message) class Status(IntEnum): - """ - Lifecycle status of a document. + """Lifecycle status of a document. Used to drive UI badges and filters. """ + # Document is being edited Draft = 0 # Document is publicly visible @@ -707,8 +820,7 @@ class Status(IntEnum): @dataclass class Document: - """ - Represents a single document tracked by the system. + """Represents a single document tracked by the system. Documents are persisted to disk and exposed via the public API. """ @@ -718,40 +830,36 @@ class Document: # Human-readable title shown in lists title: str # Current lifecycle status - status: "Status" + status: Status # Free-form tags for grouping tags: List[str] -def _write_Document(_w: _Writer, value: "Document") -> None: +def _write_Document(_w: _Writer, value: Document) -> None: _w.write_i64(value.id) _w.write_string(value.title) _w.write_i32(value.status) - _w.write_count(len(value.tags)) - for _e0 in value.tags: - _w.write_string(_e0) + _write_list_string(_w, value.tags) -def _read_Document(_r: _Reader) -> "Document": +def _read_Document(_r: _Reader) -> Document: return Document( id=_r.read_i64(), title=_r.read_string(), - status=Status(_r.read_i32()), - tags=[_r.read_string() for _i0 in range(_r.read_count())], + status=_r.read_enum(Status), + tags=_read_list_string(_r), ) class SavedListener(abc.ABC): - """ - Consumer-implemented observer of document-saved notifications. + """Consumer-implemented observer of document-saved notifications. Implementations must be safe to call from any thread. - Consumer-implemented callback interface. Subclass it, implement every - abstract method, and pass an instance wherever the API takes a `SavedListener`; - the producer may call the methods from any thread until it releases the - instance. An exception raised by a method is reported to the producer, - which fails the call in progress with Error.FOREIGN_ERROR_CODE (-4). + Subclass it and implement every method to pass an instance where the + API takes a `SavedListener`. The producer may call the methods from any + thread until it releases the instance. An exception a method raises + reaches the producer as Error.FOREIGN_ERROR_CODE (-4). """ @abc.abstractmethod @@ -760,25 +868,28 @@ class SavedListener(abc.ABC): Parameters ---------- - doc_id : Identifier of the saved document - revision : Monotonically increasing revision + doc_id : int + Identifier of the saved document + revision : int + Monotonically increasing revision """ _SavedListener_on_saved_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.c_int64, ctypes.c_int32, ctypes.POINTER(_ErrorStruct)) -_SavedListener_free_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p) class _SavedListenerVtable(ctypes.Structure): """The C vtable `docs_everywhere_docs_SavedListener_vtable`.""" _fields_ = [ + ("size", ctypes.c_uint32), + ("flags", ctypes.c_uint32), + ("free", _CallbackFree), ("on_saved", _SavedListener_on_saved_t), - ("free", _SavedListener_free_t), ] -def _SavedListener_on_saved(ctx, doc_id, revision, out_err) -> None: +def _SavedListener_on_saved(ctx: Optional[int], doc_id: int, revision: int, out_err: Any) -> None: try: _callback_get(ctx).on_saved(doc_id, revision) except BaseException as exc: @@ -786,9 +897,12 @@ def _SavedListener_on_saved(ctx, doc_id, revision, out_err) -> None: _SavedListener_vtable = _SavedListenerVtable( + ctypes.sizeof(_SavedListenerVtable), + 0, + _callback_free_fn, _SavedListener_on_saved_t(_SavedListener_on_saved), - _SavedListener_free_t(_callback_free), ) +_SavedListener_vtable_ptr = ctypes.addressof(_SavedListener_vtable) _c_docs_Notebook_new = _bind("docs_everywhere_docs_Notebook_new", ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) @@ -801,8 +915,7 @@ _c_docs_Notebook_destroy = _bind("docs_everywhere_docs_Notebook_destroy", None, class Notebook(_Object): - """ - An ordered collection of documents. + """An ordered collection of documents. Notebooks own their documents and release them when destroyed. """ @@ -814,15 +927,17 @@ class Notebook(_Object): """Create an empty notebook""" _err = _ErrorStruct() _ret = _c_docs_Notebook_new(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) self._init_handle(_required(_ret)) - def insert(self, document: "Document") -> int: + def insert(self, document: Document) -> int: """Add a document, returning its position in the notebook Parameters ---------- - document : The document to add + document : Document + The document to add """ _document_w = _Writer() _write_Document(_document_w, document) @@ -830,18 +945,20 @@ class Notebook(_Object): _self_p = self._acquire() try: _document_b = _document_w.finish() - _ret = _c_docs_Notebook_insert(_self_p, _document_b, len(_document_b), ctypes.byref(_err)) + _ret: int = _c_docs_Notebook_insert(_self_p, _document_b, len(_document_b), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret - def find(self, id: int) -> "Document": + def find(self, id: int) -> Document: """Look up a document by identifier, failing when none matches Parameters ---------- - id : Identifier of the document to find + id : int + Identifier of the document to find Raises ------ @@ -855,7 +972,8 @@ class Notebook(_Object): _ret = _c_docs_Notebook_find(_self_p, id, ctypes.byref(_out_len), ctypes.byref(_err)) finally: self._release() - _check_docs_errors_error(_err) + if _err.code: + raise _docs_errors_error_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Document) def size(self) -> int: @@ -863,76 +981,80 @@ class Notebook(_Object): _err = _ErrorStruct() _self_p = self._acquire() try: - _ret = _c_docs_Notebook_size(_self_p, ctypes.byref(_err)) + _ret: int = _c_docs_Notebook_size(_self_p, ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @staticmethod def max_documents() -> int: """The largest number of documents one notebook will hold""" _err = _ErrorStruct() - _ret = _c_docs_Notebook_max_documents(ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_docs_Notebook_max_documents(ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret _c_docs_create_document = _bind("docs_everywhere_docs_create_document", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def create_document(title: str, tags: List[str]) -> "Document": - """ - Create a brand new document. +def create_document(title: str, tags: List[str]) -> Document: + """Create a brand new document. The document starts in `Draft` status; callers must publish it explicitly to make it visible. Parameters ---------- - title : Human-readable title for the new document - tags : Initial set of tags (may be empty) + title : str + Human-readable title for the new document + tags : List[str] + Initial set of tags (may be empty) """ _title_b = title.encode("utf-8") _tags_w = _Writer() - _tags_w.write_count(len(tags)) - for _e0 in tags: - _tags_w.write_string(_e0) + _write_list_string(_tags_w, tags) _err = _ErrorStruct() _out_len = ctypes.c_size_t() _tags_b = _tags_w.finish() _ret = _c_docs_create_document(_title_b, len(_title_b), _tags_b, len(_tags_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Document) _c_docs_get_document = _bind("docs_everywhere_docs_get_document", ctypes.c_void_p, ctypes.c_int64, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def get_document(id: int) -> Optional["Document"]: +def get_document(id: int) -> Optional[Document]: """Look up a document by identifier Parameters ---------- - id : Identifier returned from create_document + id : int + Identifier returned from create_document """ _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_docs_get_document(id, ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) - return _decode(_take_bytes(_ret, _out_len.value), lambda _r: (_read_Document(_r) if _r.read_flag() else None)) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _decode(_take_bytes(_ret, _out_len.value), _read_opt_Document) _c_docs_publish_async_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_bool) _c_docs_publish_async = _bind("docs_everywhere_docs_publish_async", None, ctypes.c_int64, _c_docs_publish_async_callback, ctypes.c_void_p) -def _docs_publish_async_complete(context, err, result) -> None: +def _docs_publish_async_complete(context: Optional[int], err: Any, result: bool) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: _async_settle(context, None, result) except BaseException as exc: @@ -947,7 +1069,8 @@ async def publish_async(id: int) -> bool: Parameters ---------- - id : Identifier of the document to publish + id : int + Identifier of the document to publish """ _call, _future = _async_begin() try: @@ -955,22 +1078,65 @@ async def publish_async(id: int) -> bool: except BaseException: _async_abandon(_call) raise - return await _future + _result: bool = await _future + return _result _c_docs_watch = _bind("docs_everywhere_docs_watch", None, ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) -def watch(listener: "SavedListener") -> None: +def watch(listener: SavedListener) -> None: """Register a listener for saved notifications Parameters ---------- - listener : Receiver of saved notifications + listener : SavedListener + Receiver of saved notifications """ _err = _ErrorStruct() _listener_ctx = _callback_register(listener, SavedListener) - _c_docs_watch(_listener_ctx, ctypes.addressof(_SavedListener_vtable), ctypes.byref(_err)) - _check_error(_err) + _c_docs_watch(_listener_ctx, _SavedListener_vtable_ptr, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) + + +# === Value-buffer codecs of composite types === + + +def _write_list_string(_w: _Writer, value: List[str]) -> None: + _w.write_count(len(value)) + for item in value: + _w.write_string(item) + + +def _read_list_string(_r: _Reader) -> List[str]: + return [_r.read_string() for _ in range(_r.read_count())] + + +def _write_opt_Document(_w: _Writer, value: Optional[Document]) -> None: + _w.write_flag(value is not None) + if value is not None: + _write_Document(_w, value) + + +def _read_opt_Document(_r: _Reader) -> Optional[Document]: + return _read_Document(_r) if _r.read_flag() else None + + +__all__ = [ + "Error", + "InternalError", + "DocsErrorsError", + "NotFound", + "InvalidStatus", + "Status", + "Document", + "SavedListener", + "Notebook", + "create_document", + "get_document", + "publish_async", + "watch", +] # END docs_everywhere.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_pyi.snap b/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_pyi.snap deleted file mode 100644 index 19038d31..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_docs_everywhere_pyi.snap +++ /dev/null @@ -1,100 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -import builtins -from abc import ABC, abstractmethod -from enum import IntEnum -from typing import Dict, Iterator, List, Optional, Type - -class Error(Exception): - GENERIC_ERROR_CODE: int - PANIC_ERROR_CODE: int - MARSHAL_ERROR_CODE: int - FOREIGN_ERROR_CODE: int - CANCELLED_ERROR_CODE: int - code: int - message: str - def __init__(self, code: int, message: str) -> None: ... - -class DocsErrorsError(Error): - NotFound: Type["NotFound"] - InvalidStatus: Type["InvalidStatus"] - def __init__(self, code: int, message: str) -> None: ... - -# Returned when no document matches the requested id -class NotFound(DocsErrorsError): - CODE: int - def __init__(self, message: str = ...) -> None: ... - -# Raised when attempting to move into a forbidden state -class InvalidStatus(DocsErrorsError): - CODE: int - def __init__(self, message: str = ...) -> None: ... - -# Lifecycle status of a document. -# -# Used to drive UI badges and filters. -class Status(IntEnum): - # Document is being edited - Draft = 0 - # Document is publicly visible - Published = 1 - # Document is hidden from listings - Archived = 2 - -# Represents a single document tracked by the system. -# -# Documents are persisted to disk and exposed via the public API. -class Document: - # Stable opaque identifier - id: int - # Human-readable title shown in lists - title: str - # Current lifecycle status - status: "Status" - # Free-form tags for grouping - tags: List[str] - def __init__(self, id: int, title: str, status: "Status", tags: List[str]) -> None: ... - -# Consumer-implemented observer of document-saved notifications. -# -# Implementations must be safe to call from any thread. -class SavedListener(ABC): - # Fires after a document has been persisted successfully - @abstractmethod - def on_saved(self, doc_id: int, revision: int) -> None: ... - -# An ordered collection of documents. -# -# Notebooks own their documents and release them when destroyed. -class Notebook: - def __init__(self) -> None: ... - def close(self) -> None: ... - def __enter__(self) -> "Notebook": ... - def __exit__(self, *exc: object) -> None: ... - def insert(self, document: "Document") -> int: ... - def find(self, id: int) -> "Document": ... - def size(self) -> int: ... - @staticmethod - def max_documents() -> int: ... - -# Create a brand new document. -# -# The document starts in `Draft` status; callers must publish -# it explicitly to make it visible. -def create_document(title: str, tags: List[str]) -> "Document": ... - -# Look up a document by identifier -def get_document(id: int) -> Optional["Document"]: ... - -# Publish a document in the background -async def publish_async(id: int) -> bool: ... - -# Register a listener for saved notifications -def watch(listener: "SavedListener") -> None: ... - -# END docs_everywhere.pyi diff --git a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_py_typed.snap b/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_py_typed.snap deleted file mode 100644 index f8cd271e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__docs_everywhere_py_typed.snap +++ /dev/null @@ -1,6 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o diff --git a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__pyproject_toml.snap b/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__pyproject_toml.snap deleted file mode 100644 index 40c0948b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_docs_everywhere__pyproject_toml.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -[build-system] -requires = ["setuptools>=61.0"] -build-backend = "setuptools.build_meta" - -[project] -name = "docs_everywhere" -version = "0.1.0" -description = "Native bindings for docs_everywhere" -requires-python = ">=3.9" - -[tool.setuptools] -packages = ["docs_everywhere"] - -[tool.setuptools.package-data] -"docs_everywhere" = ["py.typed", "*.pyi", "*.so", "*.dylib", "*.dll"] - -# END pyproject.toml diff --git a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/python_edge_cases__README_md.snap deleted file mode 100644 index a2344bbc..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__README_md.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# edge_cases (Python) - -Python bindings for the `edge_cases` native library, built on `ctypes`. - -## Requirements - -- Python 3.9 or later -- The native library (`libedge_cases.so`, `libedge_cases.dylib`, or `edge_cases.dll`) - -## Install - -```bash -pip install . -``` - -## Loading the native library - -The bindings load the library from `EDGE_CASES_LIBRARY` when it is set (a full path), -then from inside the installed package, then from the system loader's search -path. - -## Usage - -```python -import edge_cases -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases___init___py.snap b/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases___init___py.snap deleted file mode 100644 index 3b6fde75..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases___init___py.snap +++ /dev/null @@ -1,10 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -from .edge_cases import * # noqa: F401,F403 - -# END __init__.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_py.snap b/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_py.snap index e7a587f3..a25d8179 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_py.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_py.snap @@ -1,11 +1,13 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate """ctypes bindings for the edge_cases native library.""" +from __future__ import annotations + import abc import asyncio import builtins @@ -17,18 +19,19 @@ import threading import warnings from dataclasses import dataclass from enum import IntEnum -from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type +from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type, TypeVar class Error(Exception): - """An error reported by the native library. - - `code` is positive for one of the API's declared error codes (raised as - the declaring module's typed subclasses) and negative for a runtime trap: - a generic failure (-1), a producer panic (-2), a marshalling failure - (-3), or an exception raised by a callback-interface implementation - (-4). A cancelled async call (-5) raises `asyncio.CancelledError` - instead. + """An error a fallible call reports. + + `code` is positive for one of the API's declared error codes, raised as + the declaring module's typed subclasses, and negative for a runtime + failure of a call that declares errors: a generic failure (-1), a + producer panic (-2), a marshalling failure (-3), or an exception raised + by a callback-interface implementation (-4). A cancelled async call (-5) + raises `asyncio.CancelledError` instead. A call that declares no errors + never raises this; its failures are bugs and raise `InternalError`. """ GENERIC_ERROR_CODE = -1 @@ -42,6 +45,24 @@ class Error(Exception): self.code = code self.message = message + def _payload(self) -> bytes: + """The code's fields as a value buffer (a code with fields overrides + this).""" + return b"" + + +class InternalError(RuntimeError): + """A failure of a call that declares no errors: a producer panic (-2), a + marshalling failure (-3), a callback-interface failure the producer let + through (-4), or another runtime failure (-1). These are bugs, so they + are unchecked; `code` and `message` say what went wrong. + """ + + def __init__(self, code: int, message: str) -> None: + super().__init__(f"({code}) {message}") + self.code = code + self.message = message + class _ErrorStruct(ctypes.Structure): """The C `edge_cases_error` out-parameter every call reports through.""" @@ -91,18 +112,73 @@ def _bind(name: str, restype: Any, *argtypes: Any) -> Any: return fn -# The C ABI revision these bindings were generated against, and the contract -# checksum of every top-level module: (module, checksum symbol, value). -_ABI_VERSION = 3 -_CHECKSUMS: List[Tuple[str, str, int]] = [ - ("edge_cases", "edge_cases_edge_cases_checksum", 0x1567f441073bd4aa), -] +class _ContractEntry(ctypes.Structure): + """The C `edge_cases_contract_entry`: one declaration's id and hash.""" + + _fields_ = [("id", ctypes.c_uint64), ("hash", ctypes.c_uint64)] + + +# The C ABI revision these bindings were generated against and, per +# top-level module's contract symbol, the declarations they rely on: +# (id, hash, dotted path). +_ABI_VERSION = 4 +_CONTRACTS: Dict[str, List[Tuple[int, int, str]]] = { + "edge_cases_edge_cases_contract": [ + (0x056a86511785ba2b, 0x5cba3f16d57c6cdb, "edge_cases.make_widget"), + (0x07bfa9ecb2b720d2, 0x3813b3f57c64bf35, "edge_cases.EdgeErrors"), + (0x0aaa11867e37f991, 0xf090dd52146ebdf0, "edge_cases.Widget.refresh"), + (0x0f9701493898ae4c, 0x6944f47a275c96e8, "edge_cases.Inner"), + (0x1226b1adac179f7d, 0x0e95a00ab0bef637, "edge_cases.async_keywords"), + (0x16d36bef1541136c, 0x5ea7021ec324298f, "edge_cases.find_widget"), + (0x1b7324c55e980feb, 0xabf87fe372358f42, "edge_cases.Widget.old_name"), + (0x1cbd731467c2bab8, 0x4915b9989c2c9074, "edge_cases.Vault"), + (0x2c2a6de67d894b0a, 0x50a4496b1e7191ca, "edge_cases.Vault.open"), + (0x322b4cfcd2d6861c, 0xca7527a379ae8465, "edge_cases.Keywords"), + (0x3f8e23a5f32906f4, 0x8d4c908d796f130d, "edge_cases.Leaf"), + (0x42ab8a685457b97c, 0xe5ce30500cadb32d, "edge_cases.Vault.handle"), + (0x4a68540e2836d3d0, 0xa20e44c71e420ed2, "edge_cases.mode_stream"), + (0x4e030580ac0b111d, 0xb68b7a18d910e175, "edge_cases.legacy_value"), + (0x4f5e4e5291864ef3, 0x55a1a88b21a08f8b, "edge_cases.Legacy"), + (0x4f84b06171b4213d, 0xbaedcfdebb6a2820, "edge_cases.scalar_stream"), + (0x5067a5ff2d9ef910, 0xbfc79b18fc4714fc, "edge_cases.Widget"), + (0x59aff159112fbf5b, 0x18e4def57a8cfa49, "edge_cases.optional_params"), + (0x5daf65b98fa77c48, 0xc817a8cb202e59c2, "edge_cases.keywords_roundtrip"), + (0x5de8a5c9726c2c5a, 0x7c50b5b65f7a896d, "edge_cases.register_batch"), + (0x618a620309994c3c, 0x826ad0b1d208d723, "edge_cases.async_bytes"), + (0x648285cdba0a7b8e, 0xcf6e80e783a4d8be, "edge_cases.WidgetHolder"), + (0x649c78079824b10d, 0x5b12dd26e0634b69, "edge_cases.async_widget"), + (0x7da7a7357d122a37, 0xf544e025ffaa3d20, "edge_cases.async_void"), + (0x7e4015bc3a472e2d, 0xfe22291d90b82e4a, "edge_cases.widget_stream"), + (0x805276e5d81fdba7, 0xd9b64e360bd03b2b, "edge_cases.Outer"), + (0x80e1183326542df4, 0xe8a57e12cc6ed276, "edge_cases.select"), + (0x923ce5e9bcd769a1, 0x0b973f5dc07dda8b, "edge_cases.Vault.to_string"), + (0x98ca8c01f5239404, 0xfe11c93eb6b303ef, "edge_cases.Widget.rename"), + (0xa13e22e98ea278d4, 0xf1fae215d7387800, "edge_cases.widgets_everywhere"), + (0xa84a8c9bf2a881e1, 0xbe1dc840bf221996, "edge_cases.async_optional"), + (0xad958c478fbf205d, 0xe386b0716d24f19b, "edge_cases.Retired"), + (0xb2fc3964ef56ab39, 0x73c325c8bc351236, "edge_cases.async_list"), + (0xb3403af521cf0006, 0x6abc30d48206a3ba, "edge_cases.Widget.close"), + (0xb794304a22a406b8, 0xf6fb98711c516105, "edge_cases.async_struct"), + (0xba3936106a49c47d, 0xb7025f2fadd6288e, "edge_cases.Vault.peek"), + (0xc155b0c3cfb164f4, 0xc777ae57d92ebc47, "edge_cases.noop"), + (0xcdc18607777019dd, 0xb81a31e5c20aa7a8, "edge_cases.inspect_widget"), + (0xd164a82aeb64b8cc, 0xded02c68005c4737, "edge_cases.nested_optional"), + (0xd978ccad2468038f, 0x30c39c65d6235153, "edge_cases.Mode"), + (0xdbe9d947431c9124, 0x39537fb848fdf980, "edge_cases.BatchListener"), + (0xeae8b7862c08bb07, 0xfe12eeaa830452ea, "edge_cases.Keyword"), + (0xf1e155a05ae262b8, 0xb575eb818a423d42, "edge_cases.async_enum"), + (0xf22e210bdbce7343, 0x2a5e0f801bc8c768, "edge_cases.Widget.dispose"), + (0xf3c7cf675563ee6c, 0x0ecffeef399facc8, "edge_cases.Vault.probe"), + (0xfd34bc67acac379e, 0x576caa14f3c0901a, "edge_cases.Widget.fetch"), + (0xff2e9d2bff941786, 0x7e96764ef1f5fbaa, "edge_cases.string_stream"), + ], +} def _check_contract() -> None: - """Refuse to load a library built for a different ABI revision or a - different version of any module's API, before any other symbol is - bound.""" + """Refuse to load a library built for another ABI revision, or one that + lacks or changed a declaration these bindings use, before any other + symbol is bound.""" try: abi_version = _bind("edge_cases_abi_version", ctypes.c_uint32) except AttributeError: @@ -116,33 +192,40 @@ def _check_contract() -> None: f"C ABI mismatch: these bindings expect revision {_ABI_VERSION}, " f"but {_lib._name} reports revision {found}" ) - for module, symbol, expected in _CHECKSUMS: + for symbol, expected in _CONTRACTS.items(): try: - checksum = _bind(symbol, ctypes.c_uint64) - except AttributeError: - raise ImportError( - f"module {module!r}: {_lib._name} does not export {symbol}" - ) from None - found = checksum() - if found != expected: - raise ImportError( - f"module {module!r}: the native library's contract checksum " - f"{found:#018x} does not match these bindings ({expected:#018x}); " - "regenerate the bindings from the library's current API" + contract = _bind( + symbol, ctypes.POINTER(_ContractEntry), ctypes.POINTER(ctypes.c_size_t) ) + except AttributeError: + raise ImportError(f"{_lib._name} does not export {symbol}") from None + length = ctypes.c_size_t() + table = contract(ctypes.byref(length)) + hashes = {table[i].id: table[i].hash for i in range(length.value)} if table else {} + for entry_id, entry_hash, path in expected: + actual = hashes.get(entry_id) + if actual is None: + raise ImportError(f"{path} is missing from the library ({_lib._name})") + if actual != entry_hash: + raise ImportError( + f"{path} changed since these bindings were generated ({_lib._name})" + ) _check_contract() _error_clear = _bind("edge_cases_error_clear", None, ctypes.POINTER(_ErrorStruct)) _free_bytes = _bind("edge_cases_free_bytes", None, ctypes.c_void_p, ctypes.c_size_t) +_debug_live_fn = _bind("edge_cases_debug_live", ctypes.c_uint64, ctypes.c_int32) def _debug_live(kind: int) -> int: - """The producer's live-allocation counter `kind` (0 objects, 1 callbacks, - 2 iterators, 3 cancel tokens, 4 returned allocations). Always 0 unless - the producer was built with its leak counters enabled.""" - return _bind("edge_cases_debug_live", ctypes.c_uint64, ctypes.c_int32)(kind) + """The producer's live-resource counter `kind` (0 objects, 1 callbacks, + 2 iterators, 3 cancel tokens, 4 byte runs; -1 is 1 when the producer + counts at all). Always 0 unless the producer was built with its leak + counters enabled.""" + live: int = _debug_live_fn(kind) + return live def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: @@ -155,17 +238,9 @@ def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: return code, message, payload -def _error_from(code: int, message: str, payload: bytes = b"") -> Error: - """The exception for a code no declared error domain claims.""" - return Error(code, message) - - -def _check_error(err: _ErrorStruct) -> None: - """Raise for a non-zero out-err slot of a call that declares no errors: - only runtime traps (a panic, a marshalling failure, a failed callback) - land here.""" - if err.code: - raise _error_from(*_read_error(err)) +def _trap_from(code: int, message: str, payload: bytes = b"") -> InternalError: + """The exception for a failed call that declares no errors.""" + return InternalError(code, message) def _take_bytes(ptr: Optional[int], length: int) -> bytes: @@ -189,6 +264,9 @@ def _peek_bytes(ptr: Optional[int], length: int) -> bytes: return ctypes.string_at(ptr, length) if ptr and length else b"" +_H = TypeVar("_H", bound="_Handle") + + class _Handle: """Owns one native reference and releases it exactly once. @@ -202,12 +280,13 @@ class _Handle: def _init_handle(self, ptr: int) -> None: self._ptr: Optional[int] = ptr + self._key = ptr self._calls = 0 self._closing = False self._lock = threading.Lock() @classmethod - def _adopt(cls, ptr: int) -> Any: + def _adopt(cls: Type[_H], ptr: int) -> _H: """Wrap one strong reference the producer handed over.""" obj = cls.__new__(cls) obj._init_handle(ptr) @@ -217,7 +296,7 @@ class _Handle: with self._lock: ptr = self._ptr if ptr is None or self._closing: - raise Error(-1, f"{type(self).__name__} used after close()") + raise ValueError(f"{type(self).__name__} is closed") self._calls += 1 return ptr @@ -231,8 +310,8 @@ class _Handle: type(self)._destroy(ptr) def close(self) -> None: - """Release this wrapper's native reference. Idempotent; the wrapper - is unusable afterwards.""" + """Release the native reference. Idempotent; the wrapper is unusable + afterwards.""" lock = getattr(self, "_lock", None) if lock is None: return @@ -246,7 +325,7 @@ class _Handle: if ptr is not None: type(self)._destroy(ptr) - def __enter__(self) -> Any: + def __enter__(self: _H) -> _H: return self def __exit__(self, *exc: object) -> None: @@ -261,27 +340,36 @@ class _Handle: class _Object(_Handle): - """Base of every interface wrapper: a reference-counted native object.""" + """Base of every interface wrapper: a reference-counted native object. + Two wrappers are equal when they hold the same native object.""" # The native clone function, set per subclass. _clone: Callable[[int], int] def _clone_ref(self) -> int: """A new strong reference to the same object, for a receiver that - adopts it (an object token inside a value buffer).""" + adopts it (an object token, or a callback method's return).""" ptr = self._acquire() try: return type(self)._clone(ptr) finally: self._release() + def __eq__(self, other: object) -> bool: + if not isinstance(other, _Object): + return NotImplemented + return self is other or (self._ptr is not None and self._ptr == other._ptr) + + def __hash__(self) -> int: + return hash(self._key) + class _Iterator(_Handle): """Base of every lazy iterator over a producer stream: one native `next` call per step; the handle is released on exhaustion, `close()`, or garbage collection.""" - def __iter__(self) -> Any: + def __iter__(self: _H) -> _H: return self def _step(self) -> int: @@ -296,23 +384,23 @@ class _Iterator(_Handle): def _required(ptr: Optional[int]) -> int: """A non-null object pointer the producer returned.""" if not ptr: - raise Error(-1, "the producer returned a null object") + raise InternalError(-3, "the producer returned a null object") return ptr -def _lend(obj: Any, cls: "Type[_Object]") -> int: +def _lend(obj: Any, cls: Type[_Object]) -> int: """Lend `obj`'s native pointer to one call; pair with `obj._release()`.""" if not isinstance(obj, cls): raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") return obj._acquire() -def _lend_opt(obj: Any, cls: "Type[_Object]") -> Optional[int]: +def _lend_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: """`_lend` for an optional object: None lends a null pointer.""" return None if obj is None else _lend(obj, cls) -def _release_opt(obj: Any) -> None: +def _release_opt(obj: Optional[_Object]) -> None: if obj is not None: obj._release() @@ -328,9 +416,12 @@ _U64 = struct.Struct(" Error: - return Error(-3, f"malformed value buffer: {what}") + +def _malformed(what: str) -> InternalError: + return InternalError(-3, f"malformed value buffer: {what}") class _Writer: @@ -413,7 +504,7 @@ class _Writer: self._buf += _U32.pack(len(values)) self._buf += struct.pack(f"<{len(values)}{code}", *values) - def write_object(self, obj: Any, cls: "Type[_Object]") -> None: + def write_object(self, obj: Any, cls: Type[_Object]) -> None: """Reserve an object token: one strong reference to `obj` the reader adopts, cloned in `finish()`. A closed wrapper is rejected now.""" if not isinstance(obj, cls): @@ -426,7 +517,8 @@ class _Writer: class _Reader: """Decodes values from the value-buffer wire format, rejecting truncated - buffers, invalid flag bytes, invalid UTF-8, and trailing data.""" + buffers, invalid flag bytes, invalid UTF-8, undeclared enum values, and + trailing data.""" __slots__ = ("_data", "_pos") @@ -499,6 +591,13 @@ class _Reader: except UnicodeDecodeError: raise _malformed("string is not valid UTF-8") from None + def read_enum(self, cls: Type[_E]) -> _E: + value = self.read_i32() + try: + return cls(value) + except ValueError: + raise _malformed(f"undeclared {cls.__name__} value {value}") from None + def read_numbers(self, code: str, size: int) -> List[Any]: """A list of one fixed-width numeric primitive, unpacked in one call.""" n = self.read_count() @@ -517,7 +616,7 @@ class _Reader: raise _malformed("trailing bytes") -def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: +def _decode(data: bytes, read: Callable[[_Reader], _T]) -> _T: """Decode exactly one value from `data`.""" r = _Reader(data) value = read(r) @@ -531,14 +630,14 @@ def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: # The trampolines live at module scope, so nothing a pending call needs can # be garbage collected before the producer fires the completion, even when # the awaiting coroutine is gone. -_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, "asyncio.Future[Any]"]] = {} +_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, asyncio.Future[Any]]] = {} _pending_lock = threading.Lock() _pending_next = 0 _error_free = _bind("edge_cases_error_free", None, ctypes.POINTER(_ErrorStruct)) -def _async_begin() -> Tuple[int, "asyncio.Future[Any]"]: +def _async_begin() -> Tuple[int, asyncio.Future[Any]]: """Register a new pending call on the running event loop.""" global _pending_next loop = asyncio.get_running_loop() @@ -583,7 +682,7 @@ def _async_settle(key: Optional[int], exc: Optional[BaseException], value: Any) pass -def _async_resolve(future: "asyncio.Future[Any]", exc: Optional[BaseException], value: Any) -> None: +def _async_resolve(future: asyncio.Future[Any], exc: Optional[BaseException], value: Any) -> None: if future.done(): return if isinstance(exc, asyncio.CancelledError): @@ -603,7 +702,7 @@ _cancel_token_cancel = _bind("edge_cases_cancel_token_cancel", None, ctypes.c_vo _cancel_token_destroy = _bind("edge_cases_cancel_token_destroy", None, ctypes.c_void_p) -async def _async_wait_cancellable(future: "asyncio.Future[Any]", token: int) -> Any: +async def _async_wait_cancellable(future: asyncio.Future[Any], token: int) -> Any: """Await a cancellable call. Cancelling the awaiting task cancels the native call and waits for its (cancelled) completion before raising, so nothing outlives the call.""" @@ -625,14 +724,23 @@ async def _async_wait_cancellable(future: "asyncio.Future[Any]", token: int) -> # a call and removed when the producer calls the vtable's `free(ctx)`, so the # implementation lives exactly as long as the producer may call it and the # producer never holds a raw reference to a Python object. ctypes acquires -# the GIL on entry to a trampoline, so the producer may call from any thread. +# the GIL on entry to a trampoline, so the producer may call (and free) from +# any thread. _callbacks: Dict[int, Any] = {} _callbacks_lock = threading.Lock() _callbacks_next = 0 +_alloc = _bind("edge_cases_alloc", ctypes.c_void_p, ctypes.c_size_t) _error_set = _bind( "edge_cases_error_set", None, ctypes.POINTER(_ErrorStruct), ctypes.c_int32, ctypes.c_char_p ) +_error_set_payload = _bind( + "edge_cases_error_set_payload", + None, + ctypes.POINTER(_ErrorStruct), + ctypes.c_char_p, + ctypes.c_size_t, +) def _callback_register(impl: Any, cls: type) -> int: @@ -647,6 +755,11 @@ def _callback_register(impl: Any, cls: type) -> int: return ctx +def _callback_register_opt(impl: Any, cls: type) -> Optional[int]: + """`_callback_register` for an optional callback: None passes none.""" + return None if impl is None else _callback_register(impl, cls) + + def _callback_get(ctx: Optional[int]) -> Any: return _callbacks[ctx or 0] @@ -657,10 +770,53 @@ def _callback_free(ctx: Optional[int]) -> None: _callbacks.pop(ctx or 0, None) -def _callback_fail(out_err: Any, exc: BaseException) -> None: - """Report an implementation's exception to the producer, which aborts - its call with the foreign error code. Nothing unwinds through C.""" - message = str(exc) or type(exc).__name__ +# Every vtable's `free` entry: one function object for the process lifetime. +_CallbackFree = ctypes.CFUNCTYPE(None, ctypes.c_void_p) +_callback_free_fn = _CallbackFree(_callback_free) + + +def _callback_return_bytes(out_ptr: Any, out_len: Any, data: bytes) -> None: + """Hand a string, bytes, or value-buffer return to the producer: a run + allocated with edge_cases_alloc, which the producer adopts and frees.""" + n = len(data) + ptr = _alloc(n) if n else None + if n: + if not ptr: + raise MemoryError(f"edge_cases_alloc({n}) failed") + ctypes.memmove(ptr, data, n) + out_ptr[0] = ptr + out_len[0] = n + + +def _callback_return_object(obj: Any, cls: Type[_Object]) -> int: + """Hand an object return to the producer: one fresh strong reference it + adopts.""" + if not isinstance(obj, cls): + raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") + return obj._clone_ref() + + +def _callback_return_object_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: + """`_callback_return_object` for an optional object: None returns null.""" + return None if obj is None else _callback_return_object(obj, cls) + + +def _callback_fail(out_err: Any, exc: BaseException, domain: Optional[Type[Error]] = None) -> None: + """Report an implementation's exception to the producer. A method that + declares errors passes its module's error domain, and an exception of + that domain carrying a declared code travels as that code with its + fields; every other exception reaches the producer as the foreign error + code (-4). Nothing unwinds through C.""" + if domain is not None and isinstance(exc, domain) and exc.code > 0: + _error_set(out_err, exc.code, exc.message.encode("utf-8", "replace")) + payload = exc._payload() + if payload: + _error_set_payload(out_err, payload, len(payload)) + return + if isinstance(exc, (Error, InternalError)): + message = exc.message + else: + message = str(exc) or type(exc).__name__ _error_set(out_err, Error.FOREIGN_ERROR_CODE, message.encode("utf-8", "replace")) @@ -670,7 +826,7 @@ def _callback_fail(out_err: Any, exc: BaseException) -> None: class EdgeErrorsError(Error): """Base exception for the `edge_cases` module's error domain.""" - Busy: "Type[Busy]" + Busy: Type[Busy] class Busy(EdgeErrorsError): @@ -685,28 +841,21 @@ class Busy(EdgeErrorsError): EdgeErrorsError.Busy = Busy -_EDGE_ERRORS_ERROR_CODES: Dict[int, Any] = { - 7: Busy, -} - - def _edge_errors_error_from(code: int, message: str, payload: bytes = b"") -> Error: - """Build the EdgeErrorsError subclass matching `code`, or the plain - Error for codes outside the domain (panics, marshalling).""" - cls = _EDGE_ERRORS_ERROR_CODES.get(code) - if cls is None: - return Error(code, message) - exc: Any = cls(message) if message else cls() - return exc + """The EdgeErrorsError subclass for `code` with its payload fields, or the + plain Error for a runtime code (a panic, a marshalling failure). + """ + if code == 7: + return Busy(message) + return Error(code, message) -def _check_edge_errors_error(err: _ErrorStruct) -> None: - if err.code: - raise _edge_errors_error_from(*_read_error(err)) +class Legacy(IntEnum): + """Deprecated enum with non-contiguous values and keyword-like variants + Deprecated: Use Mode instead + """ -class Legacy(IntEnum): - """Deprecated enum with non-contiguous values and keyword-like variants""" None_ = 0 Default = 5 Class = 10 @@ -722,11 +871,11 @@ class Leaf: weight: float -def _write_Leaf(_w: _Writer, value: "Leaf") -> None: +def _write_Leaf(_w: _Writer, value: Leaf) -> None: _w.write_f32(value.weight) -def _read_Leaf(_r: _Reader) -> "Leaf": +def _read_Leaf(_r: _Reader) -> Leaf: return Leaf( weight=_r.read_f32(), ) @@ -736,30 +885,28 @@ def _read_Leaf(_r: _Reader) -> "Leaf": class Inner: """Middle layer of a three-deep struct nesting""" - leaf: "Leaf" - leaves: List["Leaf"] + leaf: Leaf + leaves: List[Leaf] -def _write_Inner(_w: _Writer, value: "Inner") -> None: +def _write_Inner(_w: _Writer, value: Inner) -> None: _write_Leaf(_w, value.leaf) - _w.write_count(len(value.leaves)) - for _e0 in value.leaves: - _write_Leaf(_w, _e0) + _write_list_Leaf(_w, value.leaves) -def _read_Inner(_r: _Reader) -> "Inner": +def _read_Inner(_r: _Reader) -> Inner: return Inner( leaf=_read_Leaf(_r), - leaves=[_read_Leaf(_r) for _i0 in range(_r.read_count())], + leaves=_read_list_Leaf(_r), ) @dataclass class Outer: - inner: "Inner" - maybe_inner: Optional["Inner"] - by_name: Dict[str, "Inner"] - modes: List["Mode"] + inner: Inner + maybe_inner: Optional[Inner] + by_name: Dict[str, Inner] + modes: List[Mode] matrix: List[List[int]] sparse: List[Optional[int]] blob: Optional[bytes] @@ -769,59 +916,31 @@ class Outer: v2: int -def _write_Outer(_w: _Writer, value: "Outer") -> None: +def _write_Outer(_w: _Writer, value: Outer) -> None: _write_Inner(_w, value.inner) - if value.maybe_inner is None: - _w.write_flag(False) - else: - _w.write_flag(True) - _write_Inner(_w, value.maybe_inner) - _w.write_count(len(value.by_name)) - for _k0, _v0 in value.by_name.items(): - _w.write_string(_k0) - _write_Inner(_w, _v0) - _w.write_count(len(value.modes)) - for _e1 in value.modes: - _w.write_i32(_e1) - _w.write_count(len(value.matrix)) - for _e2 in value.matrix: - _w.write_numbers("i", _e2) - _w.write_count(len(value.sparse)) - for _e3 in value.sparse: - if _e3 is None: - _w.write_flag(False) - else: - _w.write_flag(True) - _w.write_i64(_e3) - if value.blob is None: - _w.write_flag(False) - else: - _w.write_flag(True) - _w.write_bytes(value.blob) - if value.note is None: - _w.write_flag(False) - else: - _w.write_flag(True) - _w.write_string(value.note) - _w.write_count(len(value.by_id)) - for _k4, _v4 in value.by_id.items(): - _w.write_i64(_k4) - _w.write_string(_v4) + _write_opt_Inner(_w, value.maybe_inner) + _write_map_string_Inner(_w, value.by_name) + _write_list_Mode(_w, value.modes) + _write_list_list_i32(_w, value.matrix) + _write_list_opt_i64(_w, value.sparse) + _write_opt_bytes(_w, value.blob) + _write_opt_string(_w, value.note) + _write_map_i64_string(_w, value.by_id) _w.write_bool(value._hidden) _w.write_u8(value.v2) -def _read_Outer(_r: _Reader) -> "Outer": +def _read_Outer(_r: _Reader) -> Outer: return Outer( inner=_read_Inner(_r), - maybe_inner=(_read_Inner(_r) if _r.read_flag() else None), - by_name=dict((_r.read_string(), _read_Inner(_r)) for _i0 in range(_r.read_count())), - modes=[Mode(_r.read_i32()) for _i0 in range(_r.read_count())], - matrix=[_r.read_numbers("i", 4) for _i0 in range(_r.read_count())], - sparse=[(_r.read_i64() if _r.read_flag() else None) for _i0 in range(_r.read_count())], - blob=(_r.read_bytes() if _r.read_flag() else None), - note=(_r.read_string() if _r.read_flag() else None), - by_id=dict((_r.read_i64(), _r.read_string()) for _i0 in range(_r.read_count())), + maybe_inner=_read_opt_Inner(_r), + by_name=_read_map_string_Inner(_r), + modes=_read_list_Mode(_r), + matrix=_read_list_list_i32(_r), + sparse=_read_list_opt_i64(_r), + blob=_read_opt_bytes(_r), + note=_read_opt_string(_r), + by_id=_read_map_i64_string(_r), _hidden=_r.read_bool(), v2=_r.read_u8(), ) @@ -892,7 +1011,7 @@ class Keywords: super: builtins.int -def _write_Keywords(_w: _Writer, value: "Keywords") -> None: +def _write_Keywords(_w: _Writer, value: Keywords) -> None: _w.write_i32(value.class_) _w.write_i32(value.def_) _w.write_i32(value.import_) @@ -954,7 +1073,7 @@ def _write_Keywords(_w: _Writer, value: "Keywords") -> None: _w.write_i32(value.super) -def _read_Keywords(_r: _Reader) -> "Keywords": +def _read_Keywords(_r: _Reader) -> Keywords: return Keywords( class_=_r.read_i32(), def_=_r.read_i32(), @@ -1020,16 +1139,19 @@ def _read_Keywords(_r: _Reader) -> "Keywords": @dataclass class Retired: - """Deprecated struct""" + """Deprecated struct + + Deprecated: Use Outer instead + """ id: int -def _write_Retired(_w: _Writer, value: "Retired") -> None: +def _write_Retired(_w: _Writer, value: Retired) -> None: _w.write_i64(value.id) -def _read_Retired(_r: _Reader) -> "Retired": +def _read_Retired(_r: _Reader) -> Retired: return Retired( id=_r.read_i64(), ) @@ -1039,54 +1161,43 @@ def _read_Retired(_r: _Reader) -> "Retired": class WidgetHolder: """Record carrying objects in every buffered position""" - primary: "Widget" - fallback: Optional["Widget"] - extras: List["Widget"] - by_name: Dict[str, "Widget"] + primary: Widget + fallback: Optional[Widget] + extras: List[Widget] + by_name: Dict[str, Widget] -def _write_WidgetHolder(_w: _Writer, value: "WidgetHolder") -> None: +def _write_WidgetHolder(_w: _Writer, value: WidgetHolder) -> None: _w.write_object(value.primary, Widget) - if value.fallback is None: - _w.write_flag(False) - else: - _w.write_flag(True) - _w.write_object(value.fallback, Widget) - _w.write_count(len(value.extras)) - for _e0 in value.extras: - _w.write_object(_e0, Widget) - _w.write_count(len(value.by_name)) - for _k1, _v1 in value.by_name.items(): - _w.write_string(_k1) - _w.write_object(_v1, Widget) - - -def _read_WidgetHolder(_r: _Reader) -> "WidgetHolder": + _write_opt_Widget(_w, value.fallback) + _write_list_Widget(_w, value.extras) + _write_map_string_Widget(_w, value.by_name) + + +def _read_WidgetHolder(_r: _Reader) -> WidgetHolder: return WidgetHolder( primary=Widget._adopt(_r.read_object()), - fallback=(Widget._adopt(_r.read_object()) if _r.read_flag() else None), - extras=[Widget._adopt(_r.read_object()) for _i0 in range(_r.read_count())], - by_name=dict((_r.read_string(), Widget._adopt(_r.read_object())) for _i0 in range(_r.read_count())), + fallback=_read_opt_Widget(_r), + extras=_read_list_Widget(_r), + by_name=_read_map_string_Widget(_r), ) class BatchListener(abc.ABC): - """ - Callback interface carrying composite payloads and objects + """Callback interface carrying composite payloads and objects - Consumer-implemented callback interface. Subclass it, implement every - abstract method, and pass an instance wherever the API takes a `BatchListener`; - the producer may call the methods from any thread until it releases the - instance. An exception raised by a method is reported to the producer, - which fails the call in progress with Error.FOREIGN_ERROR_CODE (-4). + Subclass it and implement every method to pass an instance where the + API takes a `BatchListener`. The producer may call the methods from any + thread until it releases the instance. An exception a method raises + reaches the producer as Error.FOREIGN_ERROR_CODE (-4). """ @abc.abstractmethod - def on_batch(self, items: List["Outer"], raw: bytes, first: Optional["Outer"], mode: "Mode") -> None: + def on_batch(self, items: List[Outer], raw: bytes, first: Optional[Outer], mode: Mode) -> None: """Composite payloads cross as value buffers""" @abc.abstractmethod - def on_widget(self, widget: "Widget", fallback: Optional["Widget"], children: List["Widget"]) -> None: + def on_widget(self, widget: Widget, fallback: Optional[Widget], children: List[Widget]) -> None: """Objects cross as strong references the consumer must release""" @abc.abstractmethod @@ -1098,7 +1209,7 @@ class BatchListener(abc.ABC): """Scalar return travels back through the vtable""" @abc.abstractmethod - def pick_mode(self) -> "Mode": + def pick_mode(self) -> Mode: """Enum return travels back as its integer discriminant""" @@ -1107,44 +1218,48 @@ _BatchListener_on_widget_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.c_vo _BatchListener_on_tick_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) _BatchListener_should_continue_t = ctypes.CFUNCTYPE(ctypes.c_bool, ctypes.c_void_p, ctypes.c_int32, ctypes.POINTER(_ErrorStruct)) _BatchListener_pick_mode_t = ctypes.CFUNCTYPE(ctypes.c_int32, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) -_BatchListener_free_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p) class _BatchListenerVtable(ctypes.Structure): """The C vtable `edge_cases_edge_cases_BatchListener_vtable`.""" _fields_ = [ + ("size", ctypes.c_uint32), + ("flags", ctypes.c_uint32), + ("free", _CallbackFree), ("on_batch", _BatchListener_on_batch_t), ("on_widget", _BatchListener_on_widget_t), ("on_tick", _BatchListener_on_tick_t), ("should_continue", _BatchListener_should_continue_t), ("pick_mode", _BatchListener_pick_mode_t), - ("free", _BatchListener_free_t), ] -def _BatchListener_on_batch(ctx, items_ptr, items_len, raw_ptr, raw_len, first_ptr, first_len, mode, out_err) -> None: +def _BatchListener_on_batch(ctx: Optional[int], items_ptr: Optional[int], items_len: int, raw_ptr: Optional[int], raw_len: int, first_ptr: Optional[int], first_len: int, mode: int, out_err: Any) -> None: try: - _callback_get(ctx).on_batch(_decode(_peek_bytes(items_ptr, items_len), lambda _r: [_read_Outer(_r) for _i0 in range(_r.read_count())]), _peek_bytes(raw_ptr, raw_len), _decode(_peek_bytes(first_ptr, first_len), lambda _r: (_read_Outer(_r) if _r.read_flag() else None)), Mode(mode)) + _callback_get(ctx).on_batch(_decode(_peek_bytes(items_ptr, items_len), _read_list_Outer), _peek_bytes(raw_ptr, raw_len), _decode(_peek_bytes(first_ptr, first_len), _read_opt_Outer), Mode(mode)) except BaseException as exc: _callback_fail(out_err, exc) -def _BatchListener_on_widget(ctx, widget, fallback, children_ptr, children_len, out_err) -> None: +def _BatchListener_on_widget(ctx: Optional[int], widget: Optional[int], fallback: Optional[int], children_ptr: Optional[int], children_len: int, out_err: Any) -> None: try: - _callback_get(ctx).on_widget(Widget._adopt(_required(widget)), (Widget._adopt(fallback) if fallback else None), _decode(_peek_bytes(children_ptr, children_len), lambda _r: [Widget._adopt(_r.read_object()) for _i0 in range(_r.read_count())])) + _widget = Widget._adopt(_required(widget)) + _fallback = Widget._adopt(fallback) if fallback else None + _children = _decode(_peek_bytes(children_ptr, children_len), _read_list_Widget) + _callback_get(ctx).on_widget(_widget, _fallback, _children) except BaseException as exc: _callback_fail(out_err, exc) -def _BatchListener_on_tick(ctx, out_err) -> None: +def _BatchListener_on_tick(ctx: Optional[int], out_err: Any) -> None: try: _callback_get(ctx).on_tick() except BaseException as exc: _callback_fail(out_err, exc) -def _BatchListener_should_continue(ctx, class_, out_err) -> bool: +def _BatchListener_should_continue(ctx: Optional[int], class_: int, out_err: Any) -> bool: try: return bool(_callback_get(ctx).should_continue(class_)) except BaseException as exc: @@ -1152,7 +1267,7 @@ def _BatchListener_should_continue(ctx, class_, out_err) -> bool: return False -def _BatchListener_pick_mode(ctx, out_err) -> int: +def _BatchListener_pick_mode(ctx: Optional[int], out_err: Any) -> int: try: return int(_callback_get(ctx).pick_mode()) except BaseException as exc: @@ -1161,24 +1276,25 @@ def _BatchListener_pick_mode(ctx, out_err) -> int: _BatchListener_vtable = _BatchListenerVtable( + ctypes.sizeof(_BatchListenerVtable), + 0, + _callback_free_fn, _BatchListener_on_batch_t(_BatchListener_on_batch), _BatchListener_on_widget_t(_BatchListener_on_widget), _BatchListener_on_tick_t(_BatchListener_on_tick), _BatchListener_should_continue_t(_BatchListener_should_continue), _BatchListener_pick_mode_t(_BatchListener_pick_mode), - _BatchListener_free_t(_callback_free), ) +_BatchListener_vtable_ptr = ctypes.addressof(_BatchListener_vtable) class Keyword(abc.ABC): - """ - Callback interface whose name and methods are keywords + """Callback interface whose name and methods are keywords - Consumer-implemented callback interface. Subclass it, implement every - abstract method, and pass an instance wherever the API takes a `Keyword`; - the producer may call the methods from any thread until it releases the - instance. An exception raised by a method is reported to the producer, - which fails the call in progress with Error.FOREIGN_ERROR_CODE (-4). + Subclass it and implement every method to pass an instance where the + API takes a `Keyword`. The producer may call the methods from any + thread until it releases the instance. An exception a method raises + reaches the producer as Error.FOREIGN_ERROR_CODE (-4). """ @abc.abstractmethod @@ -1192,27 +1308,28 @@ class Keyword(abc.ABC): _Keyword_delete_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.c_int32, ctypes.POINTER(_ErrorStruct)) _Keyword_import_t = ctypes.CFUNCTYPE(ctypes.c_int32, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) -_Keyword_free_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p) class _KeywordVtable(ctypes.Structure): """The C vtable `edge_cases_edge_cases_Keyword_vtable`.""" _fields_ = [ + ("size", ctypes.c_uint32), + ("flags", ctypes.c_uint32), + ("free", _CallbackFree), ("delete", _Keyword_delete_t), ("import", _Keyword_import_t), - ("free", _Keyword_free_t), ] -def _Keyword_delete(ctx, self_, out_err) -> None: +def _Keyword_delete(ctx: Optional[int], self_: int, out_err: Any) -> None: try: _callback_get(ctx).delete(self_) except BaseException as exc: _callback_fail(out_err, exc) -def _Keyword_import(ctx, out_err) -> int: +def _Keyword_import(ctx: Optional[int], out_err: Any) -> int: try: return int(_callback_get(ctx).import_()) except BaseException as exc: @@ -1221,10 +1338,13 @@ def _Keyword_import(ctx, out_err) -> int: _Keyword_vtable = _KeywordVtable( + ctypes.sizeof(_KeywordVtable), + 0, + _callback_free_fn, _Keyword_delete_t(_Keyword_delete), _Keyword_import_t(_Keyword_import), - _Keyword_free_t(_callback_free), ) +_Keyword_vtable_ptr = ctypes.addressof(_Keyword_vtable) _c_edge_cases_Widget_refresh = _bind("edge_cases_edge_cases_Widget_refresh", None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) @@ -1235,7 +1355,7 @@ _c_edge_cases_Widget_fetch_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ct _c_edge_cases_Widget_fetch = _bind("edge_cases_edge_cases_Widget_fetch", None, ctypes.c_void_p, ctypes.c_int64, _c_edge_cases_Widget_fetch_callback, ctypes.c_void_p) -def _edge_cases_Widget_fetch_complete(context, err, result_ptr, result_len) -> None: +def _edge_cases_Widget_fetch_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: @@ -1251,6 +1371,8 @@ _edge_cases_Widget_fetch_completion = _c_edge_cases_Widget_fetch_callback(_edge_ _c_edge_cases_Widget_old_name = _bind("edge_cases_edge_cases_Widget_old_name", ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) +_c_edge_cases_Widget_close = _bind("edge_cases_edge_cases_Widget_close", ctypes.c_bool, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) +_c_edge_cases_Widget_dispose = _bind("edge_cases_edge_cases_Widget_dispose", None, ctypes.c_void_p, ctypes.c_bool, ctypes.POINTER(_ErrorStruct)) _c_edge_cases_Widget_clone = _bind("edge_cases_edge_cases_Widget_clone", ctypes.c_void_p, ctypes.c_void_p) _c_edge_cases_Widget_destroy = _bind("edge_cases_edge_cases_Widget_destroy", None, ctypes.c_void_p) @@ -1278,16 +1400,13 @@ class Widget(_Object): _c_edge_cases_Widget_refresh(_self_p, ctypes.byref(_err)) finally: self._release() - _check_edge_errors_error(_err) + if _err.code: + raise _edge_errors_error_from(*_read_error(_err)) def rename(self, class_: str, import_: Optional[str]) -> str: _class__b = class_.encode("utf-8") _import__w = _Writer() - if import_ is None: - _import__w.write_flag(False) - else: - _import__w.write_flag(True) - _import__w.write_string(import_) + _write_opt_string(_import__w, import_) _err = _ErrorStruct() _out_len = ctypes.c_size_t() _self_p = self._acquire() @@ -1296,10 +1415,11 @@ class Widget(_Object): _ret = _c_edge_cases_Widget_rename(_self_p, _class__b, len(_class__b), _import__b, len(_import__b), ctypes.byref(_out_len), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) - async def fetch(self, id: int) -> "Outer": + async def fetch(self, id: int) -> Outer: """Async method with a struct result Raises @@ -1317,9 +1437,11 @@ class Widget(_Object): except BaseException: _async_abandon(_call) raise - return await _future + _result: Outer = await _future + return _result def old_name(self) -> str: + """Deprecated: Use rename instead""" warnings.warn("Use rename instead", DeprecationWarning, stacklevel=2) _err = _ErrorStruct() _out_len = ctypes.c_size_t() @@ -1328,19 +1450,48 @@ class Widget(_Object): _ret = _c_edge_cases_Widget_old_name(_self_p, ctypes.byref(_out_len), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) + def close_(self) -> bool: + """Spelled like the release method of several wrappers""" + _err = _ErrorStruct() + _self_p = self._acquire() + try: + _ret: bool = _c_edge_cases_Widget_close(_self_p, ctypes.byref(_err)) + finally: + self._release() + if _err.code: + raise _trap_from(*_read_error(_err)) + return _ret + + def dispose(self, force: bool) -> None: + """Spelled like the release method of the Dart and .NET wrappers""" + _err = _ErrorStruct() + _self_p = self._acquire() + try: + _c_edge_cases_Widget_dispose(_self_p, force, ctypes.byref(_err)) + finally: + self._release() + if _err.code: + raise _trap_from(*_read_error(_err)) + _c_edge_cases_Vault_open = _bind("edge_cases_edge_cases_Vault_open", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(_ErrorStruct)) _c_edge_cases_Vault_peek = _bind("edge_cases_edge_cases_Vault_peek", ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) +_c_edge_cases_Vault_handle = _bind("edge_cases_edge_cases_Vault_handle", ctypes.c_int64, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) _c_edge_cases_Vault_probe = _bind("edge_cases_edge_cases_Vault_probe", ctypes.c_bool, ctypes.POINTER(_ErrorStruct)) +_c_edge_cases_Vault_to_string = _bind("edge_cases_edge_cases_Vault_to_string", ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) _c_edge_cases_Vault_clone = _bind("edge_cases_edge_cases_Vault_clone", ctypes.c_void_p, ctypes.c_void_p) _c_edge_cases_Vault_destroy = _bind("edge_cases_edge_cases_Vault_destroy", None, ctypes.c_void_p) class Vault(_Object): - """Deprecated interface with a throwing constructor""" + """Deprecated interface with a throwing constructor + + Deprecated: Use Widget instead + """ _destroy = staticmethod(_c_edge_cases_Vault_destroy) _clone = staticmethod(_c_edge_cases_Vault_clone) @@ -1349,7 +1500,7 @@ class Vault(_Object): raise TypeError("Vault cannot be instantiated directly") @classmethod - def open(cls, path: str) -> "Vault": + def open(cls, path: str) -> Vault: """Fails when the vault is busy Raises @@ -1360,18 +1511,32 @@ class Vault(_Object): _path_b = path.encode("utf-8") _err = _ErrorStruct() _ret = _c_edge_cases_Vault_open(_path_b, len(_path_b), ctypes.byref(_err)) - _check_edge_errors_error(_err) + if _err.code: + raise _edge_errors_error_from(*_read_error(_err)) return cls._adopt(_required(_ret)) - def peek(self) -> Optional["Widget"]: + def peek(self) -> Optional[Widget]: _err = _ErrorStruct() _self_p = self._acquire() try: _ret = _c_edge_cases_Vault_peek(_self_p, ctypes.byref(_err)) finally: self._release() - _check_error(_err) - return (Widget._adopt(_ret) if _ret else None) + if _err.code: + raise _trap_from(*_read_error(_err)) + return Widget._adopt(_ret) if _ret else None + + def handle(self) -> int: + """Spelled like the handle accessor of several wrappers""" + _err = _ErrorStruct() + _self_p = self._acquire() + try: + _ret: int = _c_edge_cases_Vault_handle(_self_p, ctypes.byref(_err)) + finally: + self._release() + if _err.code: + raise _trap_from(*_read_error(_err)) + return _ret @staticmethod def probe() -> bool: @@ -1383,10 +1548,21 @@ class Vault(_Object): If the call reports one of the domain's error codes. """ _err = _ErrorStruct() - _ret = _c_edge_cases_Vault_probe(ctypes.byref(_err)) - _check_edge_errors_error(_err) + _ret: bool = _c_edge_cases_Vault_probe(ctypes.byref(_err)) + if _err.code: + raise _edge_errors_error_from(*_read_error(_err)) return _ret + @staticmethod + def to_string() -> str: + """Static spelled like an inherited object member""" + _err = _ErrorStruct() + _out_len = ctypes.c_size_t() + _ret = _c_edge_cases_Vault_to_string(ctypes.byref(_out_len), ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _take_str(_ret, _out_len.value) + _c_edge_cases_noop = _bind("edge_cases_edge_cases_noop", None, ctypes.POINTER(_ErrorStruct)) @@ -1395,7 +1571,8 @@ def noop() -> None: """No parameters and no result""" _err = _ErrorStruct() _c_edge_cases_noop(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) _c_edge_cases_select = _bind("edge_cases_edge_cases_select", ctypes.c_int32, ctypes.c_int32, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_bool, ctypes.POINTER(_ErrorStruct)) @@ -1405,90 +1582,50 @@ def select(class_: int, from_: str, func: List[str], range_: Optional[int], type """Function named after a keyword whose parameters are all keywords""" _from__b = from_.encode("utf-8") _func_w = _Writer() - _func_w.write_count(len(func)) - for _e0 in func: - _func_w.write_string(_e0) + _write_list_string(_func_w, func) _range__w = _Writer() - if range_ is None: - _range__w.write_flag(False) - else: - _range__w.write_flag(True) - _range__w.write_i32(range_) + _write_opt_i32(_range__w, range_) _err = _ErrorStruct() _func_b = _func_w.finish() _range__b = _range__w.finish() - _ret = _c_edge_cases_select(class_, _from__b, len(_from__b), _func_b, len(_func_b), _range__b, len(_range__b), type, ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_edge_cases_select(class_, _from__b, len(_from__b), _func_b, len(_func_b), _range__b, len(_range__b), type, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret _c_edge_cases_nested_optional = _bind("edge_cases_edge_cases_nested_optional", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def nested_optional(items: Optional[List[Optional["Outer"]]]) -> Optional[List[Optional["Outer"]]]: +def nested_optional(items: Optional[List[Optional[Outer]]]) -> Optional[List[Optional[Outer]]]: """Optional list of optional structs, the deepest legal nesting""" _items_w = _Writer() - if items is None: - _items_w.write_flag(False) - else: - _items_w.write_flag(True) - _items_w.write_count(len(items)) - for _e0 in items: - if _e0 is None: - _items_w.write_flag(False) - else: - _items_w.write_flag(True) - _write_Outer(_items_w, _e0) + _write_opt_list_opt_Outer(_items_w, items) _err = _ErrorStruct() _out_len = ctypes.c_size_t() _items_b = _items_w.finish() _ret = _c_edge_cases_nested_optional(_items_b, len(_items_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) - return _decode(_take_bytes(_ret, _out_len.value), lambda _r: ([(_read_Outer(_r) if _r.read_flag() else None) for _i0 in range(_r.read_count())] if _r.read_flag() else None)) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _decode(_take_bytes(_ret, _out_len.value), _read_opt_list_opt_Outer) _c_edge_cases_optional_params = _bind("edge_cases_edge_cases_optional_params", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def optional_params(text: Optional[str], count: Optional[int], outer: Optional["Outer"], tags: Optional[List[str]], blob: Optional[bytes], mode: Optional["Mode"]) -> Optional["Mode"]: +def optional_params(text: Optional[str], count: Optional[int], outer: Optional[Outer], tags: Optional[List[str]], blob: Optional[bytes], mode: Optional[Mode]) -> Optional[Mode]: _text_w = _Writer() - if text is None: - _text_w.write_flag(False) - else: - _text_w.write_flag(True) - _text_w.write_string(text) + _write_opt_string(_text_w, text) _count_w = _Writer() - if count is None: - _count_w.write_flag(False) - else: - _count_w.write_flag(True) - _count_w.write_i32(count) + _write_opt_i32(_count_w, count) _outer_w = _Writer() - if outer is None: - _outer_w.write_flag(False) - else: - _outer_w.write_flag(True) - _write_Outer(_outer_w, outer) + _write_opt_Outer(_outer_w, outer) _tags_w = _Writer() - if tags is None: - _tags_w.write_flag(False) - else: - _tags_w.write_flag(True) - _tags_w.write_count(len(tags)) - for _e0 in tags: - _tags_w.write_string(_e0) + _write_opt_list_string(_tags_w, tags) _blob_w = _Writer() - if blob is None: - _blob_w.write_flag(False) - else: - _blob_w.write_flag(True) - _blob_w.write_bytes(blob) + _write_opt_bytes(_blob_w, blob) _mode_w = _Writer() - if mode is None: - _mode_w.write_flag(False) - else: - _mode_w.write_flag(True) - _mode_w.write_i32(mode) + _write_opt_Mode(_mode_w, mode) _err = _ErrorStruct() _out_len = ctypes.c_size_t() _text_b = _text_w.finish() @@ -1498,44 +1635,35 @@ def optional_params(text: Optional[str], count: Optional[int], outer: Optional[" _blob_b = _blob_w.finish() _mode_b = _mode_w.finish() _ret = _c_edge_cases_optional_params(_text_b, len(_text_b), _count_b, len(_count_b), _outer_b, len(_outer_b), _tags_b, len(_tags_b), _blob_b, len(_blob_b), _mode_b, len(_mode_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) - return _decode(_take_bytes(_ret, _out_len.value), lambda _r: (Mode(_r.read_i32()) if _r.read_flag() else None)) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _decode(_take_bytes(_ret, _out_len.value), _read_opt_Mode) _c_edge_cases_register_batch = _bind("edge_cases_edge_cases_register_batch", None, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) -def register_batch(listener: "BatchListener", keyword: "Keyword") -> None: +def register_batch(listener: BatchListener, keyword: Keyword) -> None: """Two callback interface parameters on one function""" _err = _ErrorStruct() _listener_ctx = _callback_register(listener, BatchListener) _keyword_ctx = _callback_register(keyword, Keyword) - _c_edge_cases_register_batch(_listener_ctx, ctypes.addressof(_BatchListener_vtable), _keyword_ctx, ctypes.addressof(_Keyword_vtable), ctypes.byref(_err)) - _check_error(_err) + _c_edge_cases_register_batch(_listener_ctx, _BatchListener_vtable_ptr, _keyword_ctx, _Keyword_vtable_ptr, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) _c_edge_cases_widgets_everywhere = _bind("edge_cases_edge_cases_widgets_everywhere", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def widgets_everywhere(list: List["Widget"], by_name: Dict[str, "Widget"], sparse: List[Optional["Widget"]], holder: "WidgetHolder") -> "WidgetHolder": +def widgets_everywhere(list: List[Widget], by_name: Dict[str, Widget], sparse: List[Optional[Widget]], holder: WidgetHolder) -> WidgetHolder: """Objects in every buffered position""" _list_w = _Writer() - _list_w.write_count(len(list)) - for _e0 in list: - _list_w.write_object(_e0, Widget) + _write_list_Widget(_list_w, list) _by_name_w = _Writer() - _by_name_w.write_count(len(by_name)) - for _k1, _v1 in by_name.items(): - _by_name_w.write_string(_k1) - _by_name_w.write_object(_v1, Widget) + _write_map_string_Widget(_by_name_w, by_name) _sparse_w = _Writer() - _sparse_w.write_count(len(sparse)) - for _e2 in sparse: - if _e2 is None: - _sparse_w.write_flag(False) - else: - _sparse_w.write_flag(True) - _sparse_w.write_object(_e2, Widget) + _write_list_opt_Widget(_sparse_w, sparse) _holder_w = _Writer() _write_WidgetHolder(_holder_w, holder) _err = _ErrorStruct() @@ -1545,7 +1673,8 @@ def widgets_everywhere(list: List["Widget"], by_name: Dict[str, "Widget"], spars _sparse_b = _sparse_w.finish() _holder_b = _holder_w.finish() _ret = _c_edge_cases_widgets_everywhere(_list_b, len(_list_b), _by_name_b, len(_by_name_b), _sparse_b, len(_sparse_b), _holder_b, len(_holder_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_WidgetHolder) @@ -1560,7 +1689,7 @@ class _edge_cases_WidgetStreamIterator(_Iterator): _destroy = staticmethod(_c_edge_cases_WidgetStreamIterator_destroy) - def __next__(self) -> "Widget": + def __next__(self) -> Widget: _item = ctypes.c_void_p() _err = _ErrorStruct() _p = self._step() @@ -1568,16 +1697,16 @@ class _edge_cases_WidgetStreamIterator(_Iterator): _more = _c_edge_cases_WidgetStreamIterator_next(_p, ctypes.byref(_item), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) if not _more: self.close() raise StopIteration return Widget._adopt(_required(_item.value)) -def widget_stream() -> Iterator["Widget"]: - """ - Iterator of objects +def widget_stream() -> Iterator[Widget]: + """Iterator of objects Returns a lazy iterator: each step pulls one element from the producer. Exhaust or close() the iterator to release its native @@ -1585,7 +1714,8 @@ def widget_stream() -> Iterator["Widget"]: """ _err = _ErrorStruct() _ret = _c_edge_cases_widget_stream(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _edge_cases_WidgetStreamIterator._adopt(_ret) @@ -1593,7 +1723,7 @@ _c_edge_cases_async_widget_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ct _c_edge_cases_async_widget = _bind("edge_cases_edge_cases_async_widget", None, _c_edge_cases_async_widget_callback, ctypes.c_void_p) -def _edge_cases_async_widget_complete(context, err, result) -> None: +def _edge_cases_async_widget_complete(context: Optional[int], err: Any, result: Optional[int]) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: @@ -1608,7 +1738,7 @@ def _edge_cases_async_widget_complete(context, err, result) -> None: _edge_cases_async_widget_completion = _c_edge_cases_async_widget_callback(_edge_cases_async_widget_complete) -async def async_widget() -> "Widget": +async def async_widget() -> Widget: """Async returning an object Raises @@ -1622,25 +1752,27 @@ async def async_widget() -> "Widget": except BaseException: _async_abandon(_call) raise - return await _future + _result: Widget = await _future + return _result _c_edge_cases_make_widget = _bind("edge_cases_edge_cases_make_widget", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(_ErrorStruct)) -def make_widget(label: str) -> "Widget": +def make_widget(label: str) -> Widget: """Factory for an interface with no constructor""" _label_b = label.encode("utf-8") _err = _ErrorStruct() _ret = _c_edge_cases_make_widget(_label_b, len(_label_b), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return Widget._adopt(_required(_ret)) _c_edge_cases_find_widget = _bind("edge_cases_edge_cases_find_widget", ctypes.c_void_p, ctypes.c_int64, ctypes.POINTER(_ErrorStruct)) -def find_widget(id: int) -> Optional["Widget"]: +def find_widget(id: int) -> Optional[Widget]: """Optional interface return Raises @@ -1650,14 +1782,15 @@ def find_widget(id: int) -> Optional["Widget"]: """ _err = _ErrorStruct() _ret = _c_edge_cases_find_widget(id, ctypes.byref(_err)) - _check_edge_errors_error(_err) - return (Widget._adopt(_ret) if _ret else None) + if _err.code: + raise _edge_errors_error_from(*_read_error(_err)) + return Widget._adopt(_ret) if _ret else None _c_edge_cases_inspect_widget = _bind("edge_cases_edge_cases_inspect_widget", ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def inspect_widget(widget: "Widget", fallback: Optional["Widget"]) -> str: +def inspect_widget(widget: Widget, fallback: Optional[Widget]) -> str: """Interface parameter, including an optional one""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() @@ -1670,7 +1803,8 @@ def inspect_widget(widget: "Widget", fallback: Optional["Widget"]) -> str: _release_opt(fallback) finally: widget._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) @@ -1694,7 +1828,8 @@ class _edge_cases_StringStreamIterator(_Iterator): _more = _c_edge_cases_StringStreamIterator_next(_p, ctypes.byref(_item), ctypes.byref(_len), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) if not _more: self.close() raise StopIteration @@ -1702,8 +1837,7 @@ class _edge_cases_StringStreamIterator(_Iterator): def string_stream() -> Iterator[str]: - """ - Iterator of strings + """Iterator of strings Returns a lazy iterator: each step pulls one element from the producer. Exhaust or close() the iterator to release its native @@ -1711,7 +1845,8 @@ def string_stream() -> Iterator[str]: """ _err = _ErrorStruct() _ret = _c_edge_cases_string_stream(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _edge_cases_StringStreamIterator._adopt(_ret) @@ -1734,7 +1869,8 @@ class _edge_cases_ScalarStreamIterator(_Iterator): _more = _c_edge_cases_ScalarStreamIterator_next(_p, ctypes.byref(_item), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) if not _more: self.close() raise StopIteration @@ -1742,8 +1878,7 @@ class _edge_cases_ScalarStreamIterator(_Iterator): def scalar_stream() -> Iterator[int]: - """ - Iterator of scalars + """Iterator of scalars Returns a lazy iterator: each step pulls one element from the producer. Exhaust or close() the iterator to release its native @@ -1751,7 +1886,8 @@ def scalar_stream() -> Iterator[int]: """ _err = _ErrorStruct() _ret = _c_edge_cases_scalar_stream(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _edge_cases_ScalarStreamIterator._adopt(_ret) @@ -1766,7 +1902,7 @@ class _edge_cases_ModeStreamIterator(_Iterator): _destroy = staticmethod(_c_edge_cases_ModeStreamIterator_destroy) - def __next__(self) -> "Mode": + def __next__(self) -> Mode: _item = ctypes.c_int32() _err = _ErrorStruct() _p = self._step() @@ -1774,16 +1910,16 @@ class _edge_cases_ModeStreamIterator(_Iterator): _more = _c_edge_cases_ModeStreamIterator_next(_p, ctypes.byref(_item), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) if not _more: self.close() raise StopIteration return Mode(_item.value) -def mode_stream() -> Iterator["Mode"]: - """ - Iterator of enums +def mode_stream() -> Iterator[Mode]: + """Iterator of enums Returns a lazy iterator: each step pulls one element from the producer. Exhaust or close() the iterator to release its native @@ -1791,7 +1927,8 @@ def mode_stream() -> Iterator["Mode"]: """ _err = _ErrorStruct() _ret = _c_edge_cases_mode_stream(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _edge_cases_ModeStreamIterator._adopt(_ret) @@ -1799,12 +1936,12 @@ _c_edge_cases_async_void_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctyp _c_edge_cases_async_void = _bind("edge_cases_edge_cases_async_void", None, _c_edge_cases_async_void_callback, ctypes.c_void_p) -def _edge_cases_async_void_complete(context, err) -> None: +def _edge_cases_async_void_complete(context: Optional[int], err: Any) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: _async_settle(context, None, None) except BaseException as exc: @@ -1822,19 +1959,19 @@ async def async_void() -> None: except BaseException: _async_abandon(_call) raise - return await _future + await _future _c_edge_cases_async_bytes_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_void_p, ctypes.c_size_t) _c_edge_cases_async_bytes = _bind("edge_cases_edge_cases_async_bytes", None, ctypes.c_uint32, _c_edge_cases_async_bytes_callback, ctypes.c_void_p) -def _edge_cases_async_bytes_complete(context, err, result_ptr, result_len) -> None: +def _edge_cases_async_bytes_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: _async_settle(context, None, _take_bytes(result_ptr, result_len)) except BaseException as exc: @@ -1851,14 +1988,15 @@ async def async_bytes(size: int) -> bytes: except BaseException: _async_abandon(_call) raise - return await _future + _result: bytes = await _future + return _result _c_edge_cases_async_struct_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_void_p, ctypes.c_size_t) _c_edge_cases_async_struct = _bind("edge_cases_edge_cases_async_struct", None, _c_edge_cases_async_struct_callback, ctypes.c_void_p) -def _edge_cases_async_struct_complete(context, err, result_ptr, result_len) -> None: +def _edge_cases_async_struct_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: @@ -1873,7 +2011,7 @@ def _edge_cases_async_struct_complete(context, err, result_ptr, result_len) -> N _edge_cases_async_struct_completion = _c_edge_cases_async_struct_callback(_edge_cases_async_struct_complete) -async def async_struct() -> "Outer": +async def async_struct() -> Outer: """ Raises ------ @@ -1886,21 +2024,22 @@ async def async_struct() -> "Outer": except BaseException: _async_abandon(_call) raise - return await _future + _result: Outer = await _future + return _result _c_edge_cases_async_list_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_void_p, ctypes.c_size_t) _c_edge_cases_async_list = _bind("edge_cases_edge_cases_async_list", None, ctypes.c_void_p, _c_edge_cases_async_list_callback, ctypes.c_void_p) -def _edge_cases_async_list_complete(context, err, result_ptr, result_len) -> None: +def _edge_cases_async_list_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: _async_settle(context, _async_error(err, _edge_errors_error_from), None) else: - _async_settle(context, None, _decode(_take_bytes(result_ptr, result_len), lambda _r: [_read_Outer(_r) for _i0 in range(_r.read_count())])) + _async_settle(context, None, _decode(_take_bytes(result_ptr, result_len), _read_list_Outer)) except BaseException as exc: _async_settle(context, exc, None) @@ -1908,7 +2047,7 @@ def _edge_cases_async_list_complete(context, err, result_ptr, result_len) -> Non _edge_cases_async_list_completion = _c_edge_cases_async_list_callback(_edge_cases_async_list_complete) -async def async_list() -> List["Outer"]: +async def async_list() -> List[Outer]: """ Raises ------ @@ -1923,21 +2062,22 @@ async def async_list() -> List["Outer"]: _async_abandon(_call) _cancel_token_destroy(_token) raise - return await _async_wait_cancellable(_future, _token) + _result: List[Outer] = await _async_wait_cancellable(_future, _token) + return _result _c_edge_cases_async_optional_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_void_p, ctypes.c_size_t) _c_edge_cases_async_optional = _bind("edge_cases_edge_cases_async_optional", None, _c_edge_cases_async_optional_callback, ctypes.c_void_p) -def _edge_cases_async_optional_complete(context, err, result_ptr, result_len) -> None: +def _edge_cases_async_optional_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: - _async_settle(context, None, _decode(_take_bytes(result_ptr, result_len), lambda _r: (_r.read_i64() if _r.read_flag() else None))) + _async_settle(context, None, _decode(_take_bytes(result_ptr, result_len), _read_opt_i64)) except BaseException as exc: _async_settle(context, exc, None) @@ -1952,19 +2092,20 @@ async def async_optional() -> Optional[int]: except BaseException: _async_abandon(_call) raise - return await _future + _result: Optional[int] = await _future + return _result _c_edge_cases_async_enum_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_int32) _c_edge_cases_async_enum = _bind("edge_cases_edge_cases_async_enum", None, _c_edge_cases_async_enum_callback, ctypes.c_void_p) -def _edge_cases_async_enum_complete(context, err, result) -> None: +def _edge_cases_async_enum_complete(context: Optional[int], err: Any, result: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: _async_settle(context, None, Mode(result)) except BaseException as exc: @@ -1974,26 +2115,27 @@ def _edge_cases_async_enum_complete(context, err, result) -> None: _edge_cases_async_enum_completion = _c_edge_cases_async_enum_callback(_edge_cases_async_enum_complete) -async def async_enum() -> "Mode": +async def async_enum() -> Mode: _call, _future = _async_begin() try: _c_edge_cases_async_enum(_edge_cases_async_enum_completion, _call) except BaseException: _async_abandon(_call) raise - return await _future + _result: Mode = await _future + return _result _c_edge_cases_async_keywords_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_void_p, ctypes.c_size_t) _c_edge_cases_async_keywords = _bind("edge_cases_edge_cases_async_keywords", None, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_int32, _c_edge_cases_async_keywords_callback, ctypes.c_void_p) -def _edge_cases_async_keywords_complete(context, err, result_ptr, result_len) -> None: +def _edge_cases_async_keywords_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: _async_settle(context, None, _take_str(result_ptr, result_len)) except BaseException as exc: @@ -2011,32 +2153,325 @@ async def async_keywords(class_: str, import_: int) -> str: except BaseException: _async_abandon(_call) raise - return await _future + _result: str = await _future + return _result _c_edge_cases_keywords_roundtrip = _bind("edge_cases_edge_cases_keywords_roundtrip", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def keywords_roundtrip(value: "Keywords") -> "Keywords": +def keywords_roundtrip(value: Keywords) -> Keywords: _value_w = _Writer() _write_Keywords(_value_w, value) _err = _ErrorStruct() _out_len = ctypes.c_size_t() _value_b = _value_w.finish() _ret = _c_edge_cases_keywords_roundtrip(_value_b, len(_value_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Keywords) _c_edge_cases_legacy_value = _bind("edge_cases_edge_cases_legacy_value", ctypes.c_void_p, ctypes.c_int32, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def legacy_value(value: "Legacy") -> "Retired": +def legacy_value(value: Legacy) -> Retired: + """Deprecated: Use Mode-based APIs""" warnings.warn("Use Mode-based APIs", DeprecationWarning, stacklevel=2) _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_edge_cases_legacy_value(value, ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Retired) + +# === Value-buffer codecs of composite types === + + +def _write_list_Leaf(_w: _Writer, value: List[Leaf]) -> None: + _w.write_count(len(value)) + for item in value: + _write_Leaf(_w, item) + + +def _read_list_Leaf(_r: _Reader) -> List[Leaf]: + return [_read_Leaf(_r) for _ in range(_r.read_count())] + + +def _write_opt_Inner(_w: _Writer, value: Optional[Inner]) -> None: + _w.write_flag(value is not None) + if value is not None: + _write_Inner(_w, value) + + +def _read_opt_Inner(_r: _Reader) -> Optional[Inner]: + return _read_Inner(_r) if _r.read_flag() else None + + +def _write_map_string_Inner(_w: _Writer, value: Dict[str, Inner]) -> None: + _w.write_count(len(value)) + for key, item in value.items(): + _w.write_string(key) + _write_Inner(_w, item) + + +def _read_map_string_Inner(_r: _Reader) -> Dict[str, Inner]: + n = _r.read_count() + value = {_r.read_string(): _read_Inner(_r) for _ in range(n)} + if len(value) != n: + raise _malformed("repeated map key") + return value + + +def _write_list_Mode(_w: _Writer, value: List[Mode]) -> None: + _w.write_count(len(value)) + for item in value: + _w.write_i32(item) + + +def _read_list_Mode(_r: _Reader) -> List[Mode]: + return [_r.read_enum(Mode) for _ in range(_r.read_count())] + + +def _write_list_i32(_w: _Writer, value: List[int]) -> None: + _w.write_numbers("i", value) + + +def _read_list_i32(_r: _Reader) -> List[int]: + return _r.read_numbers("i", 4) + + +def _write_list_list_i32(_w: _Writer, value: List[List[int]]) -> None: + _w.write_count(len(value)) + for item in value: + _write_list_i32(_w, item) + + +def _read_list_list_i32(_r: _Reader) -> List[List[int]]: + return [_read_list_i32(_r) for _ in range(_r.read_count())] + + +def _write_opt_i64(_w: _Writer, value: Optional[int]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_i64(value) + + +def _read_opt_i64(_r: _Reader) -> Optional[int]: + return _r.read_i64() if _r.read_flag() else None + + +def _write_list_opt_i64(_w: _Writer, value: List[Optional[int]]) -> None: + _w.write_count(len(value)) + for item in value: + _write_opt_i64(_w, item) + + +def _read_list_opt_i64(_r: _Reader) -> List[Optional[int]]: + return [_read_opt_i64(_r) for _ in range(_r.read_count())] + + +def _write_opt_bytes(_w: _Writer, value: Optional[bytes]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_bytes(value) + + +def _read_opt_bytes(_r: _Reader) -> Optional[bytes]: + return _r.read_bytes() if _r.read_flag() else None + + +def _write_opt_string(_w: _Writer, value: Optional[str]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_string(value) + + +def _read_opt_string(_r: _Reader) -> Optional[str]: + return _r.read_string() if _r.read_flag() else None + + +def _write_map_i64_string(_w: _Writer, value: Dict[int, str]) -> None: + _w.write_count(len(value)) + for key, item in value.items(): + _w.write_i64(key) + _w.write_string(item) + + +def _read_map_i64_string(_r: _Reader) -> Dict[int, str]: + n = _r.read_count() + value = {_r.read_i64(): _r.read_string() for _ in range(n)} + if len(value) != n: + raise _malformed("repeated map key") + return value + + +def _write_opt_Widget(_w: _Writer, value: Optional[Widget]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_object(value, Widget) + + +def _read_opt_Widget(_r: _Reader) -> Optional[Widget]: + return Widget._adopt(_r.read_object()) if _r.read_flag() else None + + +def _write_list_Widget(_w: _Writer, value: List[Widget]) -> None: + _w.write_count(len(value)) + for item in value: + _w.write_object(item, Widget) + + +def _read_list_Widget(_r: _Reader) -> List[Widget]: + return [Widget._adopt(_r.read_object()) for _ in range(_r.read_count())] + + +def _write_map_string_Widget(_w: _Writer, value: Dict[str, Widget]) -> None: + _w.write_count(len(value)) + for key, item in value.items(): + _w.write_string(key) + _w.write_object(item, Widget) + + +def _read_map_string_Widget(_r: _Reader) -> Dict[str, Widget]: + n = _r.read_count() + value = {_r.read_string(): Widget._adopt(_r.read_object()) for _ in range(n)} + if len(value) != n: + raise _malformed("repeated map key") + return value + + +def _write_list_string(_w: _Writer, value: List[str]) -> None: + _w.write_count(len(value)) + for item in value: + _w.write_string(item) + + +def _read_list_string(_r: _Reader) -> List[str]: + return [_r.read_string() for _ in range(_r.read_count())] + + +def _write_opt_i32(_w: _Writer, value: Optional[int]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_i32(value) + + +def _read_opt_i32(_r: _Reader) -> Optional[int]: + return _r.read_i32() if _r.read_flag() else None + + +def _write_opt_Outer(_w: _Writer, value: Optional[Outer]) -> None: + _w.write_flag(value is not None) + if value is not None: + _write_Outer(_w, value) + + +def _read_opt_Outer(_r: _Reader) -> Optional[Outer]: + return _read_Outer(_r) if _r.read_flag() else None + + +def _write_list_opt_Outer(_w: _Writer, value: List[Optional[Outer]]) -> None: + _w.write_count(len(value)) + for item in value: + _write_opt_Outer(_w, item) + + +def _read_list_opt_Outer(_r: _Reader) -> List[Optional[Outer]]: + return [_read_opt_Outer(_r) for _ in range(_r.read_count())] + + +def _write_opt_list_opt_Outer(_w: _Writer, value: Optional[List[Optional[Outer]]]) -> None: + _w.write_flag(value is not None) + if value is not None: + _write_list_opt_Outer(_w, value) + + +def _read_opt_list_opt_Outer(_r: _Reader) -> Optional[List[Optional[Outer]]]: + return _read_list_opt_Outer(_r) if _r.read_flag() else None + + +def _write_opt_list_string(_w: _Writer, value: Optional[List[str]]) -> None: + _w.write_flag(value is not None) + if value is not None: + _write_list_string(_w, value) + + +def _read_opt_list_string(_r: _Reader) -> Optional[List[str]]: + return _read_list_string(_r) if _r.read_flag() else None + + +def _write_opt_Mode(_w: _Writer, value: Optional[Mode]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_i32(value) + + +def _read_opt_Mode(_r: _Reader) -> Optional[Mode]: + return _r.read_enum(Mode) if _r.read_flag() else None + + +def _write_list_opt_Widget(_w: _Writer, value: List[Optional[Widget]]) -> None: + _w.write_count(len(value)) + for item in value: + _write_opt_Widget(_w, item) + + +def _read_list_opt_Widget(_r: _Reader) -> List[Optional[Widget]]: + return [_read_opt_Widget(_r) for _ in range(_r.read_count())] + + +def _write_list_Outer(_w: _Writer, value: List[Outer]) -> None: + _w.write_count(len(value)) + for item in value: + _write_Outer(_w, item) + + +def _read_list_Outer(_r: _Reader) -> List[Outer]: + return [_read_Outer(_r) for _ in range(_r.read_count())] + + +__all__ = [ + "Error", + "InternalError", + "EdgeErrorsError", + "Busy", + "Legacy", + "Mode", + "Leaf", + "Inner", + "Outer", + "Keywords", + "Retired", + "WidgetHolder", + "BatchListener", + "Keyword", + "Widget", + "Vault", + "noop", + "select", + "nested_optional", + "optional_params", + "register_batch", + "widgets_everywhere", + "widget_stream", + "async_widget", + "make_widget", + "find_widget", + "inspect_widget", + "string_stream", + "scalar_stream", + "mode_stream", + "async_void", + "async_bytes", + "async_struct", + "async_list", + "async_optional", + "async_enum", + "async_keywords", + "keywords_roundtrip", + "legacy_value", +] + # END edge_cases.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_pyi.snap b/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_pyi.snap deleted file mode 100644 index 5677e104..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_edge_cases_pyi.snap +++ /dev/null @@ -1,246 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -import builtins -from abc import ABC, abstractmethod -from enum import IntEnum -from typing import Dict, Iterator, List, Optional, Type - -class Error(Exception): - GENERIC_ERROR_CODE: int - PANIC_ERROR_CODE: int - MARSHAL_ERROR_CODE: int - FOREIGN_ERROR_CODE: int - CANCELLED_ERROR_CODE: int - code: int - message: str - def __init__(self, code: int, message: str) -> None: ... - -class EdgeErrorsError(Error): - Busy: Type["Busy"] - def __init__(self, code: int, message: str) -> None: ... - -class Busy(EdgeErrorsError): - CODE: int - def __init__(self, message: str = ...) -> None: ... - -# Deprecated enum with non-contiguous values and keyword-like variants -class Legacy(IntEnum): - None_ = 0 - Default = 5 - Class = 10 - -class Mode(IntEnum): - Fast = 1 - Safe = 2 - -class Leaf: - weight: float - def __init__(self, weight: float) -> None: ... - -# Middle layer of a three-deep struct nesting -class Inner: - leaf: "Leaf" - leaves: List["Leaf"] - def __init__(self, leaf: "Leaf", leaves: List["Leaf"]) -> None: ... - -class Outer: - inner: "Inner" - maybe_inner: Optional["Inner"] - by_name: Dict[str, "Inner"] - modes: List["Mode"] - matrix: List[List[int]] - sparse: List[Optional[int]] - blob: Optional[bytes] - note: Optional[str] - by_id: Dict[int, str] - _hidden: bool - v2: int - def __init__(self, inner: "Inner", maybe_inner: Optional["Inner"], by_name: Dict[str, "Inner"], modes: List["Mode"], matrix: List[List[int]], sparse: List[Optional[int]], blob: Optional[bytes], note: Optional[str], by_id: Dict[int, str], _hidden: bool, v2: int) -> None: ... - -# Every field is a reserved word somewhere -class Keywords: - class_: builtins.int - def_: builtins.int - import_: builtins.int - from_: builtins.int - func: builtins.int - range: builtins.int - register: builtins.int - volatile: builtins.int - new: builtins.int - default: builtins.int - end: builtins.int - in_: builtins.int - is_: builtins.int - int: builtins.int - var: builtins.int - val: builtins.int - self: builtins.int - this: builtins.int - object: builtins.int - string: builtins.int - case: builtins.int - select: builtins.int - switch: builtins.int - void: builtins.int - union: builtins.int - namespace: builtins.int - operator: builtins.int - template: builtins.int - package: builtins.int - protocol: builtins.int - guard: builtins.int - defer: builtins.int - inout: builtins.int - internal: builtins.int - when: builtins.int - fun: builtins.int - data: builtins.int - late: builtins.int - part: builtins.int - out: builtins.int - ref: builtins.int - event: builtins.int - base: builtins.int - lambda_: builtins.int - yield_: builtins.int - nil: builtins.int - null: builtins.int - begin: builtins.int - rescue: builtins.int - module: builtins.int - chan: builtins.int - go: builtins.int - let: builtins.int - export: builtins.int - delete: builtins.int - typedef: builtins.int - sizeof: builtins.int - static: builtins.int - super: builtins.int - def __init__(__dataclass_self__, class_: builtins.int, def_: builtins.int, import_: builtins.int, from_: builtins.int, func: builtins.int, range: builtins.int, register: builtins.int, volatile: builtins.int, new: builtins.int, default: builtins.int, end: builtins.int, in_: builtins.int, is_: builtins.int, int: builtins.int, var: builtins.int, val: builtins.int, self: builtins.int, this: builtins.int, object: builtins.int, string: builtins.int, case: builtins.int, select: builtins.int, switch: builtins.int, void: builtins.int, union: builtins.int, namespace: builtins.int, operator: builtins.int, template: builtins.int, package: builtins.int, protocol: builtins.int, guard: builtins.int, defer: builtins.int, inout: builtins.int, internal: builtins.int, when: builtins.int, fun: builtins.int, data: builtins.int, late: builtins.int, part: builtins.int, out: builtins.int, ref: builtins.int, event: builtins.int, base: builtins.int, lambda_: builtins.int, yield_: builtins.int, nil: builtins.int, null: builtins.int, begin: builtins.int, rescue: builtins.int, module: builtins.int, chan: builtins.int, go: builtins.int, let: builtins.int, export: builtins.int, delete: builtins.int, typedef: builtins.int, sizeof: builtins.int, static: builtins.int, super: builtins.int) -> None: ... - -# Deprecated struct -class Retired: - id: int - def __init__(self, id: int) -> None: ... - -# Record carrying objects in every buffered position -class WidgetHolder: - primary: "Widget" - fallback: Optional["Widget"] - extras: List["Widget"] - by_name: Dict[str, "Widget"] - def __init__(self, primary: "Widget", fallback: Optional["Widget"], extras: List["Widget"], by_name: Dict[str, "Widget"]) -> None: ... - -# Callback interface carrying composite payloads and objects -class BatchListener(ABC): - # Composite payloads cross as value buffers - @abstractmethod - def on_batch(self, items: List["Outer"], raw: bytes, first: Optional["Outer"], mode: "Mode") -> None: ... - # Objects cross as strong references the consumer must release - @abstractmethod - def on_widget(self, widget: "Widget", fallback: Optional["Widget"], children: List["Widget"]) -> None: ... - @abstractmethod - def on_tick(self) -> None: ... - # Scalar return travels back through the vtable - @abstractmethod - def should_continue(self, class_: int) -> bool: ... - # Enum return travels back as its integer discriminant - @abstractmethod - def pick_mode(self) -> "Mode": ... - -# Callback interface whose name and methods are keywords -class Keyword(ABC): - @abstractmethod - def delete(self, self_: int) -> None: ... - @abstractmethod - def import_(self) -> int: ... - -# Interface with no constructor; instances come from make_widget -class Widget: - def close(self) -> None: ... - def __enter__(self) -> "Widget": ... - def __exit__(self, *exc: object) -> None: ... - def refresh(self) -> None: ... - def rename(self, class_: str, import_: Optional[str]) -> str: ... - async def fetch(self, id: int) -> "Outer": ... - def old_name(self) -> str: ... - -# Deprecated interface with a throwing constructor -class Vault: - def close(self) -> None: ... - def __enter__(self) -> "Vault": ... - def __exit__(self, *exc: object) -> None: ... - @classmethod - def open(cls, path: str) -> "Vault": ... - def peek(self) -> Optional["Widget"]: ... - @staticmethod - def probe() -> bool: ... - -# No parameters and no result -def noop() -> None: ... - -# Function named after a keyword whose parameters are all keywords -def select(class_: int, from_: str, func: List[str], range_: Optional[int], type: bool) -> int: ... - -# Optional list of optional structs, the deepest legal nesting -def nested_optional(items: Optional[List[Optional["Outer"]]]) -> Optional[List[Optional["Outer"]]]: ... - -def optional_params(text: Optional[str], count: Optional[int], outer: Optional["Outer"], tags: Optional[List[str]], blob: Optional[bytes], mode: Optional["Mode"]) -> Optional["Mode"]: ... - -# Two callback interface parameters on one function -def register_batch(listener: "BatchListener", keyword: "Keyword") -> None: ... - -# Objects in every buffered position -def widgets_everywhere(list: List["Widget"], by_name: Dict[str, "Widget"], sparse: List[Optional["Widget"]], holder: "WidgetHolder") -> "WidgetHolder": ... - -# Iterator of objects -def widget_stream() -> Iterator["Widget"]: ... - -# Async returning an object -async def async_widget() -> "Widget": ... - -# Factory for an interface with no constructor -def make_widget(label: str) -> "Widget": ... - -# Optional interface return -def find_widget(id: int) -> Optional["Widget"]: ... - -# Interface parameter, including an optional one -def inspect_widget(widget: "Widget", fallback: Optional["Widget"]) -> str: ... - -# Iterator of strings -def string_stream() -> Iterator[str]: ... - -# Iterator of scalars -def scalar_stream() -> Iterator[int]: ... - -# Iterator of enums -def mode_stream() -> Iterator["Mode"]: ... - -# Async with no result -async def async_void() -> None: ... - -async def async_bytes(size: int) -> bytes: ... - -async def async_struct() -> "Outer": ... - -async def async_list() -> List["Outer"]: ... - -async def async_optional() -> Optional[int]: ... - -async def async_enum() -> "Mode": ... - -async def async_keywords(class_: str, import_: int) -> str: ... - -def keywords_roundtrip(value: "Keywords") -> "Keywords": ... - -def legacy_value(value: "Legacy") -> "Retired": ... - -# END edge_cases.pyi diff --git a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_py_typed.snap b/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_py_typed.snap deleted file mode 100644 index f8cd271e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__edge_cases_py_typed.snap +++ /dev/null @@ -1,6 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o diff --git a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__pyproject_toml.snap b/crates/weaveffi-cli/tests/snapshots/python_edge_cases__pyproject_toml.snap deleted file mode 100644 index 6db35d80..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_edge_cases__pyproject_toml.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -[build-system] -requires = ["setuptools>=61.0"] -build-backend = "setuptools.build_meta" - -[project] -name = "edge_cases" -version = "0.1.0" -description = "Native bindings for edge_cases" -requires-python = ">=3.9" - -[tool.setuptools] -packages = ["edge_cases"] - -[tool.setuptools.package-data] -"edge_cases" = ["py.typed", "*.pyi", "*.so", "*.dylib", "*.dll"] - -# END pyproject.toml diff --git a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__README_md.snap index ca5362cf..eabc5553 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__README_md.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # kitchen_sink (Python) diff --git a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink___init___py.snap b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink___init___py.snap index 14ddd591..e399eb9d 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink___init___py.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink___init___py.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate from .kitchen_sink import * # noqa: F401,F403 diff --git a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_py.snap b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_py.snap index 56c40319..2c24145b 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_py.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_py.snap @@ -1,11 +1,13 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate """ctypes bindings for the kitchen_sink native library.""" +from __future__ import annotations + import abc import asyncio import builtins @@ -17,18 +19,19 @@ import threading import warnings from dataclasses import dataclass from enum import IntEnum -from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type +from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type, TypeVar class Error(Exception): - """An error reported by the native library. - - `code` is positive for one of the API's declared error codes (raised as - the declaring module's typed subclasses) and negative for a runtime trap: - a generic failure (-1), a producer panic (-2), a marshalling failure - (-3), or an exception raised by a callback-interface implementation - (-4). A cancelled async call (-5) raises `asyncio.CancelledError` - instead. + """An error a fallible call reports. + + `code` is positive for one of the API's declared error codes, raised as + the declaring module's typed subclasses, and negative for a runtime + failure of a call that declares errors: a generic failure (-1), a + producer panic (-2), a marshalling failure (-3), or an exception raised + by a callback-interface implementation (-4). A cancelled async call (-5) + raises `asyncio.CancelledError` instead. A call that declares no errors + never raises this; its failures are bugs and raise `InternalError`. """ GENERIC_ERROR_CODE = -1 @@ -42,6 +45,24 @@ class Error(Exception): self.code = code self.message = message + def _payload(self) -> bytes: + """The code's fields as a value buffer (a code with fields overrides + this).""" + return b"" + + +class InternalError(RuntimeError): + """A failure of a call that declares no errors: a producer panic (-2), a + marshalling failure (-3), a callback-interface failure the producer let + through (-4), or another runtime failure (-1). These are bugs, so they + are unchecked; `code` and `message` say what went wrong. + """ + + def __init__(self, code: int, message: str) -> None: + super().__init__(f"({code}) {message}") + self.code = code + self.message = message + class _ErrorStruct(ctypes.Structure): """The C `kitchen_sink_error` out-parameter every call reports through.""" @@ -91,19 +112,61 @@ def _bind(name: str, restype: Any, *argtypes: Any) -> Any: return fn -# The C ABI revision these bindings were generated against, and the contract -# checksum of every top-level module: (module, checksum symbol, value). -_ABI_VERSION = 3 -_CHECKSUMS: List[Tuple[str, str, int]] = [ - ("shared", "kitchen_sink_shared_checksum", 0x42c4ce2c8d0af052), - ("kitchen", "kitchen_sink_kitchen_checksum", 0xd83da75b24b545ff), -] +class _ContractEntry(ctypes.Structure): + """The C `kitchen_sink_contract_entry`: one declaration's id and hash.""" + + _fields_ = [("id", ctypes.c_uint64), ("hash", ctypes.c_uint64)] + + +# The C ABI revision these bindings were generated against and, per +# top-level module's contract symbol, the declarations they rely on: +# (id, hash, dotted path). +_ABI_VERSION = 4 +_CONTRACTS: Dict[str, List[Tuple[int, int, str]]] = { + "kitchen_sink_shared_contract": [ + (0x709e234caa962446, 0x3894f2ca452e9f98, "shared.ping"), + (0xcfb306fa5801ee5b, 0x1cbd9e10983cac2c, "shared.Token"), + ], + "kitchen_sink_kitchen_contract": [ + (0x01d4f09bbddc986f, 0x29eb88fecc66544d, "kitchen.Gadget.new"), + (0x10472d8fe748c2ee, 0xb265832b6a759fc4, "kitchen.maybe_gadget"), + (0x11368eec07a74078, 0xfa049f78b77e146a, "kitchen.gadgets"), + (0x137f08a29cde6021, 0xda8de7aeb2dbb8a3, "kitchen.stream_gadgets"), + (0x2111b54980826dd7, 0x452783f3e6ca6e1d, "kitchen.do_cancellable"), + (0x29dddbc60bf8d7db, 0x452a3e573d57f6db, "kitchen.Priority"), + (0x2b227003b74a0d16, 0x12332957470e9782, "kitchen.maybe_subscribe"), + (0x2dd66cb0064f0494, 0x60197363fefd92dd, "kitchen.Item"), + (0x33c265bbf079b0e9, 0xc3dcd0927e6c1deb, "kitchen.u32_id"), + (0x33e6381d01660dfe, 0x077c63c31fd7f90a, "kitchen.KitchenErrors"), + (0x3f5fc99d73504b76, 0x2df88d710f1ea10e, "kitchen.Gadget.poke"), + (0x4c5764cf109528d1, 0x2a65529af4338a58, "kitchen.maybe_item"), + (0x4de17dfb93fd494d, 0xba0ad0dd52ed74af, "kitchen.i32_id"), + (0x504e95b7f7e65570, 0x01b3e8e0c9a10e23, "kitchen.stream_items"), + (0x68cdb9a4bd3e3dd5, 0xc678b9164348d497, "kitchen.bool_id"), + (0x69e7cd703d16eb85, 0xf3b28dd22ae9e689, "kitchen.do_async"), + (0x70865079ce41be82, 0xbe3b69464672af46, "kitchen.Gadget.describe"), + (0x71c5a1116cb7c49a, 0xcba1837108a89b84, "kitchen.echo_bytes"), + (0x735ee1e0049ee91a, 0xca74320bc53eddab, "kitchen.echo_string"), + (0x76d7fc58f4e41d84, 0x7373a0e47d30e690, "kitchen.i64_id"), + (0x79abfe1aa960fd6a, 0xbca7a9e5eb26f3a6, "kitchen.nested.hello"), + (0x7d3dd7d5cf6749e3, 0xa192f5a7298adb39, "kitchen.new_op"), + (0x89a5e664bc25c67b, 0x5764a95e81ecfd31, "kitchen.f64_id"), + (0x8cfdbdcecac5f7f1, 0x07e272f09c41fffc, "kitchen.Gadget.version"), + (0x974b9f3e823c1738, 0xb81b91e975695f27, "kitchen.cross_module_token"), + (0xaca0655fac15d230, 0x11c4d2b28abe07bb, "kitchen.list_items"), + (0xc883b4afae20543e, 0xc40560e5411ff392, "kitchen.legacy_op"), + (0xf2f7753e6e1dffc2, 0xc9047a95bdf49e93, "kitchen.get_attrs"), + (0xfae25d3391922ead, 0x1c7f95739f3210b8, "kitchen.subscribe"), + (0xfd0059b05cb2fcfb, 0x248bf8404a5823b4, "kitchen.Gadget"), + (0xfed9c551fb9fd2d6, 0x59aec980ef3f3291, "kitchen.ReadyListener"), + ], +} def _check_contract() -> None: - """Refuse to load a library built for a different ABI revision or a - different version of any module's API, before any other symbol is - bound.""" + """Refuse to load a library built for another ABI revision, or one that + lacks or changed a declaration these bindings use, before any other + symbol is bound.""" try: abi_version = _bind("kitchen_sink_abi_version", ctypes.c_uint32) except AttributeError: @@ -117,33 +180,40 @@ def _check_contract() -> None: f"C ABI mismatch: these bindings expect revision {_ABI_VERSION}, " f"but {_lib._name} reports revision {found}" ) - for module, symbol, expected in _CHECKSUMS: + for symbol, expected in _CONTRACTS.items(): try: - checksum = _bind(symbol, ctypes.c_uint64) - except AttributeError: - raise ImportError( - f"module {module!r}: {_lib._name} does not export {symbol}" - ) from None - found = checksum() - if found != expected: - raise ImportError( - f"module {module!r}: the native library's contract checksum " - f"{found:#018x} does not match these bindings ({expected:#018x}); " - "regenerate the bindings from the library's current API" + contract = _bind( + symbol, ctypes.POINTER(_ContractEntry), ctypes.POINTER(ctypes.c_size_t) ) + except AttributeError: + raise ImportError(f"{_lib._name} does not export {symbol}") from None + length = ctypes.c_size_t() + table = contract(ctypes.byref(length)) + hashes = {table[i].id: table[i].hash for i in range(length.value)} if table else {} + for entry_id, entry_hash, path in expected: + actual = hashes.get(entry_id) + if actual is None: + raise ImportError(f"{path} is missing from the library ({_lib._name})") + if actual != entry_hash: + raise ImportError( + f"{path} changed since these bindings were generated ({_lib._name})" + ) _check_contract() _error_clear = _bind("kitchen_sink_error_clear", None, ctypes.POINTER(_ErrorStruct)) _free_bytes = _bind("kitchen_sink_free_bytes", None, ctypes.c_void_p, ctypes.c_size_t) +_debug_live_fn = _bind("kitchen_sink_debug_live", ctypes.c_uint64, ctypes.c_int32) def _debug_live(kind: int) -> int: - """The producer's live-allocation counter `kind` (0 objects, 1 callbacks, - 2 iterators, 3 cancel tokens, 4 returned allocations). Always 0 unless - the producer was built with its leak counters enabled.""" - return _bind("kitchen_sink_debug_live", ctypes.c_uint64, ctypes.c_int32)(kind) + """The producer's live-resource counter `kind` (0 objects, 1 callbacks, + 2 iterators, 3 cancel tokens, 4 byte runs; -1 is 1 when the producer + counts at all). Always 0 unless the producer was built with its leak + counters enabled.""" + live: int = _debug_live_fn(kind) + return live def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: @@ -156,17 +226,9 @@ def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: return code, message, payload -def _error_from(code: int, message: str, payload: bytes = b"") -> Error: - """The exception for a code no declared error domain claims.""" - return Error(code, message) - - -def _check_error(err: _ErrorStruct) -> None: - """Raise for a non-zero out-err slot of a call that declares no errors: - only runtime traps (a panic, a marshalling failure, a failed callback) - land here.""" - if err.code: - raise _error_from(*_read_error(err)) +def _trap_from(code: int, message: str, payload: bytes = b"") -> InternalError: + """The exception for a failed call that declares no errors.""" + return InternalError(code, message) def _take_bytes(ptr: Optional[int], length: int) -> bytes: @@ -190,6 +252,9 @@ def _peek_bytes(ptr: Optional[int], length: int) -> bytes: return ctypes.string_at(ptr, length) if ptr and length else b"" +_H = TypeVar("_H", bound="_Handle") + + class _Handle: """Owns one native reference and releases it exactly once. @@ -203,12 +268,13 @@ class _Handle: def _init_handle(self, ptr: int) -> None: self._ptr: Optional[int] = ptr + self._key = ptr self._calls = 0 self._closing = False self._lock = threading.Lock() @classmethod - def _adopt(cls, ptr: int) -> Any: + def _adopt(cls: Type[_H], ptr: int) -> _H: """Wrap one strong reference the producer handed over.""" obj = cls.__new__(cls) obj._init_handle(ptr) @@ -218,7 +284,7 @@ class _Handle: with self._lock: ptr = self._ptr if ptr is None or self._closing: - raise Error(-1, f"{type(self).__name__} used after close()") + raise ValueError(f"{type(self).__name__} is closed") self._calls += 1 return ptr @@ -232,8 +298,8 @@ class _Handle: type(self)._destroy(ptr) def close(self) -> None: - """Release this wrapper's native reference. Idempotent; the wrapper - is unusable afterwards.""" + """Release the native reference. Idempotent; the wrapper is unusable + afterwards.""" lock = getattr(self, "_lock", None) if lock is None: return @@ -247,7 +313,7 @@ class _Handle: if ptr is not None: type(self)._destroy(ptr) - def __enter__(self) -> Any: + def __enter__(self: _H) -> _H: return self def __exit__(self, *exc: object) -> None: @@ -262,27 +328,36 @@ class _Handle: class _Object(_Handle): - """Base of every interface wrapper: a reference-counted native object.""" + """Base of every interface wrapper: a reference-counted native object. + Two wrappers are equal when they hold the same native object.""" # The native clone function, set per subclass. _clone: Callable[[int], int] def _clone_ref(self) -> int: """A new strong reference to the same object, for a receiver that - adopts it (an object token inside a value buffer).""" + adopts it (an object token, or a callback method's return).""" ptr = self._acquire() try: return type(self)._clone(ptr) finally: self._release() + def __eq__(self, other: object) -> bool: + if not isinstance(other, _Object): + return NotImplemented + return self is other or (self._ptr is not None and self._ptr == other._ptr) + + def __hash__(self) -> int: + return hash(self._key) + class _Iterator(_Handle): """Base of every lazy iterator over a producer stream: one native `next` call per step; the handle is released on exhaustion, `close()`, or garbage collection.""" - def __iter__(self) -> Any: + def __iter__(self: _H) -> _H: return self def _step(self) -> int: @@ -297,23 +372,23 @@ class _Iterator(_Handle): def _required(ptr: Optional[int]) -> int: """A non-null object pointer the producer returned.""" if not ptr: - raise Error(-1, "the producer returned a null object") + raise InternalError(-3, "the producer returned a null object") return ptr -def _lend(obj: Any, cls: "Type[_Object]") -> int: +def _lend(obj: Any, cls: Type[_Object]) -> int: """Lend `obj`'s native pointer to one call; pair with `obj._release()`.""" if not isinstance(obj, cls): raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") return obj._acquire() -def _lend_opt(obj: Any, cls: "Type[_Object]") -> Optional[int]: +def _lend_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: """`_lend` for an optional object: None lends a null pointer.""" return None if obj is None else _lend(obj, cls) -def _release_opt(obj: Any) -> None: +def _release_opt(obj: Optional[_Object]) -> None: if obj is not None: obj._release() @@ -329,9 +404,12 @@ _U64 = struct.Struct(" Error: - return Error(-3, f"malformed value buffer: {what}") + +def _malformed(what: str) -> InternalError: + return InternalError(-3, f"malformed value buffer: {what}") class _Writer: @@ -414,7 +492,7 @@ class _Writer: self._buf += _U32.pack(len(values)) self._buf += struct.pack(f"<{len(values)}{code}", *values) - def write_object(self, obj: Any, cls: "Type[_Object]") -> None: + def write_object(self, obj: Any, cls: Type[_Object]) -> None: """Reserve an object token: one strong reference to `obj` the reader adopts, cloned in `finish()`. A closed wrapper is rejected now.""" if not isinstance(obj, cls): @@ -427,7 +505,8 @@ class _Writer: class _Reader: """Decodes values from the value-buffer wire format, rejecting truncated - buffers, invalid flag bytes, invalid UTF-8, and trailing data.""" + buffers, invalid flag bytes, invalid UTF-8, undeclared enum values, and + trailing data.""" __slots__ = ("_data", "_pos") @@ -500,6 +579,13 @@ class _Reader: except UnicodeDecodeError: raise _malformed("string is not valid UTF-8") from None + def read_enum(self, cls: Type[_E]) -> _E: + value = self.read_i32() + try: + return cls(value) + except ValueError: + raise _malformed(f"undeclared {cls.__name__} value {value}") from None + def read_numbers(self, code: str, size: int) -> List[Any]: """A list of one fixed-width numeric primitive, unpacked in one call.""" n = self.read_count() @@ -518,7 +604,7 @@ class _Reader: raise _malformed("trailing bytes") -def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: +def _decode(data: bytes, read: Callable[[_Reader], _T]) -> _T: """Decode exactly one value from `data`.""" r = _Reader(data) value = read(r) @@ -532,14 +618,14 @@ def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: # The trampolines live at module scope, so nothing a pending call needs can # be garbage collected before the producer fires the completion, even when # the awaiting coroutine is gone. -_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, "asyncio.Future[Any]"]] = {} +_pending: Dict[int, Tuple[asyncio.AbstractEventLoop, asyncio.Future[Any]]] = {} _pending_lock = threading.Lock() _pending_next = 0 _error_free = _bind("kitchen_sink_error_free", None, ctypes.POINTER(_ErrorStruct)) -def _async_begin() -> Tuple[int, "asyncio.Future[Any]"]: +def _async_begin() -> Tuple[int, asyncio.Future[Any]]: """Register a new pending call on the running event loop.""" global _pending_next loop = asyncio.get_running_loop() @@ -584,7 +670,7 @@ def _async_settle(key: Optional[int], exc: Optional[BaseException], value: Any) pass -def _async_resolve(future: "asyncio.Future[Any]", exc: Optional[BaseException], value: Any) -> None: +def _async_resolve(future: asyncio.Future[Any], exc: Optional[BaseException], value: Any) -> None: if future.done(): return if isinstance(exc, asyncio.CancelledError): @@ -604,7 +690,7 @@ _cancel_token_cancel = _bind("kitchen_sink_cancel_token_cancel", None, ctypes.c_ _cancel_token_destroy = _bind("kitchen_sink_cancel_token_destroy", None, ctypes.c_void_p) -async def _async_wait_cancellable(future: "asyncio.Future[Any]", token: int) -> Any: +async def _async_wait_cancellable(future: asyncio.Future[Any], token: int) -> Any: """Await a cancellable call. Cancelling the awaiting task cancels the native call and waits for its (cancelled) completion before raising, so nothing outlives the call.""" @@ -626,14 +712,23 @@ async def _async_wait_cancellable(future: "asyncio.Future[Any]", token: int) -> # a call and removed when the producer calls the vtable's `free(ctx)`, so the # implementation lives exactly as long as the producer may call it and the # producer never holds a raw reference to a Python object. ctypes acquires -# the GIL on entry to a trampoline, so the producer may call from any thread. +# the GIL on entry to a trampoline, so the producer may call (and free) from +# any thread. _callbacks: Dict[int, Any] = {} _callbacks_lock = threading.Lock() _callbacks_next = 0 +_alloc = _bind("kitchen_sink_alloc", ctypes.c_void_p, ctypes.c_size_t) _error_set = _bind( "kitchen_sink_error_set", None, ctypes.POINTER(_ErrorStruct), ctypes.c_int32, ctypes.c_char_p ) +_error_set_payload = _bind( + "kitchen_sink_error_set_payload", + None, + ctypes.POINTER(_ErrorStruct), + ctypes.c_char_p, + ctypes.c_size_t, +) def _callback_register(impl: Any, cls: type) -> int: @@ -648,6 +743,11 @@ def _callback_register(impl: Any, cls: type) -> int: return ctx +def _callback_register_opt(impl: Any, cls: type) -> Optional[int]: + """`_callback_register` for an optional callback: None passes none.""" + return None if impl is None else _callback_register(impl, cls) + + def _callback_get(ctx: Optional[int]) -> Any: return _callbacks[ctx or 0] @@ -658,10 +758,53 @@ def _callback_free(ctx: Optional[int]) -> None: _callbacks.pop(ctx or 0, None) -def _callback_fail(out_err: Any, exc: BaseException) -> None: - """Report an implementation's exception to the producer, which aborts - its call with the foreign error code. Nothing unwinds through C.""" - message = str(exc) or type(exc).__name__ +# Every vtable's `free` entry: one function object for the process lifetime. +_CallbackFree = ctypes.CFUNCTYPE(None, ctypes.c_void_p) +_callback_free_fn = _CallbackFree(_callback_free) + + +def _callback_return_bytes(out_ptr: Any, out_len: Any, data: bytes) -> None: + """Hand a string, bytes, or value-buffer return to the producer: a run + allocated with kitchen_sink_alloc, which the producer adopts and frees.""" + n = len(data) + ptr = _alloc(n) if n else None + if n: + if not ptr: + raise MemoryError(f"kitchen_sink_alloc({n}) failed") + ctypes.memmove(ptr, data, n) + out_ptr[0] = ptr + out_len[0] = n + + +def _callback_return_object(obj: Any, cls: Type[_Object]) -> int: + """Hand an object return to the producer: one fresh strong reference it + adopts.""" + if not isinstance(obj, cls): + raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") + return obj._clone_ref() + + +def _callback_return_object_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: + """`_callback_return_object` for an optional object: None returns null.""" + return None if obj is None else _callback_return_object(obj, cls) + + +def _callback_fail(out_err: Any, exc: BaseException, domain: Optional[Type[Error]] = None) -> None: + """Report an implementation's exception to the producer. A method that + declares errors passes its module's error domain, and an exception of + that domain carrying a declared code travels as that code with its + fields; every other exception reaches the producer as the foreign error + code (-4). Nothing unwinds through C.""" + if domain is not None and isinstance(exc, domain) and exc.code > 0: + _error_set(out_err, exc.code, exc.message.encode("utf-8", "replace")) + payload = exc._payload() + if payload: + _error_set_payload(out_err, payload, len(payload)) + return + if isinstance(exc, (Error, InternalError)): + message = exc.message + else: + message = str(exc) or type(exc).__name__ _error_set(out_err, Error.FOREIGN_ERROR_CODE, message.encode("utf-8", "replace")) @@ -676,12 +819,12 @@ class Token: label: str -def _write_Token(_w: _Writer, value: "Token") -> None: +def _write_Token(_w: _Writer, value: Token) -> None: _w.write_i64(value.id) _w.write_string(value.label) -def _read_Token(_r: _Reader) -> "Token": +def _read_Token(_r: _Reader) -> Token: return Token( id=_r.read_i64(), label=_r.read_string(), @@ -696,7 +839,8 @@ def ping() -> str: _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_shared_ping(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) @@ -706,8 +850,8 @@ def ping() -> str: class KitchenErrorsError(Error): """Base exception for the `kitchen` module's error domain.""" - NotFound: "Type[NotFound]" - InvalidInput: "Type[InvalidInput]" + NotFound: Type[NotFound] + InvalidInput: Type[InvalidInput] class NotFound(KitchenErrorsError): @@ -732,29 +876,20 @@ KitchenErrorsError.NotFound = NotFound KitchenErrorsError.InvalidInput = InvalidInput -_KITCHEN_ERRORS_ERROR_CODES: Dict[int, Any] = { - 1: NotFound, - 2: InvalidInput, -} - - def _kitchen_errors_error_from(code: int, message: str, payload: bytes = b"") -> Error: - """Build the KitchenErrorsError subclass matching `code`, or the plain - Error for codes outside the domain (panics, marshalling).""" - cls = _KITCHEN_ERRORS_ERROR_CODES.get(code) - if cls is None: - return Error(code, message) - exc: Any = cls(message) if message else cls() - return exc - - -def _check_kitchen_errors_error(err: _ErrorStruct) -> None: - if err.code: - raise _kitchen_errors_error_from(*_read_error(err)) + """The KitchenErrorsError subclass for `code` with its payload fields, or the + plain Error for a runtime code (a panic, a marshalling failure). + """ + if code == 1: + return NotFound(message) + if code == 2: + return InvalidInput(message) + return Error(code, message) class Priority(IntEnum): """Task priority levels""" + # Low priority Low = 0 # Normal priority @@ -783,13 +918,13 @@ class Item: attrs: Dict[str, str] parent: Optional[int] # Cross-module struct reference - token: "Token" - priority: "Priority" + token: Token + priority: Priority # An optional object inside a record - gadget: Optional["Gadget"] + gadget: Optional[Gadget] -def _write_Item(_w: _Writer, value: "Item") -> None: +def _write_Item(_w: _Writer, value: Item) -> None: _w.write_i64(value.id) _w.write_string(value.name) _w.write_i32(value.count) @@ -797,28 +932,15 @@ def _write_Item(_w: _Writer, value: "Item") -> None: _w.write_f64(value.ratio) _w.write_u32(value.nick) _w.write_bytes(value.payload) - _w.write_count(len(value.tags)) - for _e0 in value.tags: - _w.write_string(_e0) - _w.write_count(len(value.attrs)) - for _k1, _v1 in value.attrs.items(): - _w.write_string(_k1) - _w.write_string(_v1) - if value.parent is None: - _w.write_flag(False) - else: - _w.write_flag(True) - _w.write_i64(value.parent) + _write_list_string(_w, value.tags) + _write_map_string_string(_w, value.attrs) + _write_opt_i64(_w, value.parent) _write_Token(_w, value.token) _w.write_i32(value.priority) - if value.gadget is None: - _w.write_flag(False) - else: - _w.write_flag(True) - _w.write_object(value.gadget, Gadget) + _write_opt_Gadget(_w, value.gadget) -def _read_Item(_r: _Reader) -> "Item": +def _read_Item(_r: _Reader) -> Item: return Item( id=_r.read_i64(), name=_r.read_string(), @@ -827,24 +949,23 @@ def _read_Item(_r: _Reader) -> "Item": ratio=_r.read_f64(), nick=_r.read_u32(), payload=_r.read_bytes(), - tags=[_r.read_string() for _i0 in range(_r.read_count())], - attrs=dict((_r.read_string(), _r.read_string()) for _i0 in range(_r.read_count())), - parent=(_r.read_i64() if _r.read_flag() else None), + tags=_read_list_string(_r), + attrs=_read_map_string_string(_r), + parent=_read_opt_i64(_r), token=_read_Token(_r), - priority=Priority(_r.read_i32()), - gadget=(Gadget._adopt(_r.read_object()) if _r.read_flag() else None), + priority=_r.read_enum(Priority), + gadget=_read_opt_Gadget(_r), ) class ReadyListener(abc.ABC): - """ - Consumer-implemented observer of ready items + """Consumer-implemented observer of ready items - Consumer-implemented callback interface. Subclass it, implement every - abstract method, and pass an instance wherever the API takes a `ReadyListener`; - the producer may call the methods from any thread until it releases the - instance. An exception raised by a method is reported to the producer, - which fails the call in progress with Error.FOREIGN_ERROR_CODE (-4). + Subclass it and implement every method to pass an instance where the + API takes a `ReadyListener`. The producer may call the methods from any + thread until it releases the instance. A method that declares errors may + raise a `KitchenErrorsError` code, which reaches the producer with its fields; + any other exception reaches it as Error.FOREIGN_ERROR_CODE (-4). """ @abc.abstractmethod @@ -852,45 +973,104 @@ class ReadyListener(abc.ABC): """Fires when an item is ready""" @abc.abstractmethod - def on_item(self, item: "Item", gadget: "Gadget") -> bool: + def on_item(self, item: Item, gadget: Gadget) -> bool: """Receives the item itself and says whether to keep listening""" + @abc.abstractmethod + def label(self) -> str: + """A display label, failing with a kitchen error + + Raises + ------ + KitchenErrorsError + To report one of the domain's error codes to the producer. + """ + + @abc.abstractmethod + def latest(self) -> Optional[Item]: + """The most recent item the listener kept, if any""" + + @abc.abstractmethod + def favorite(self) -> Gadget: + """The listener's favorite gadget""" + _ReadyListener_on_ready_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.c_int32, ctypes.c_void_p, ctypes.c_size_t, ctypes.POINTER(_ErrorStruct)) _ReadyListener_on_item_t = ctypes.CFUNCTYPE(ctypes.c_bool, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_size_t, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) -_ReadyListener_free_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p) +_ReadyListener_label_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(ctypes.c_void_p), ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) +_ReadyListener_latest_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(ctypes.c_void_p), ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) +_ReadyListener_favorite_t = ctypes.CFUNCTYPE(ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) class _ReadyListenerVtable(ctypes.Structure): """The C vtable `kitchen_sink_kitchen_ReadyListener_vtable`.""" _fields_ = [ + ("size", ctypes.c_uint32), + ("flags", ctypes.c_uint32), + ("free", _CallbackFree), ("on_ready", _ReadyListener_on_ready_t), ("on_item", _ReadyListener_on_item_t), - ("free", _ReadyListener_free_t), + ("label", _ReadyListener_label_t), + ("latest", _ReadyListener_latest_t), + ("favorite", _ReadyListener_favorite_t), ] -def _ReadyListener_on_ready(ctx, code, msg_ptr, msg_len, out_err) -> None: +def _ReadyListener_on_ready(ctx: Optional[int], code: int, msg_ptr: Optional[int], msg_len: int, out_err: Any) -> None: try: _callback_get(ctx).on_ready(code, _peek_bytes(msg_ptr, msg_len).decode("utf-8")) except BaseException as exc: _callback_fail(out_err, exc) -def _ReadyListener_on_item(ctx, item_ptr, item_len, gadget, out_err) -> bool: +def _ReadyListener_on_item(ctx: Optional[int], item_ptr: Optional[int], item_len: int, gadget: Optional[int], out_err: Any) -> bool: try: - return bool(_callback_get(ctx).on_item(_decode(_peek_bytes(item_ptr, item_len), _read_Item), Gadget._adopt(_required(gadget)))) + _gadget = Gadget._adopt(_required(gadget)) + return bool(_callback_get(ctx).on_item(_decode(_peek_bytes(item_ptr, item_len), _read_Item), _gadget)) except BaseException as exc: _callback_fail(out_err, exc) return False +def _ReadyListener_label(ctx: Optional[int], out_ptr: Any, out_len: Any, out_err: Any) -> None: + try: + _ret = _callback_get(ctx).label() + _callback_return_bytes(out_ptr, out_len, _ret.encode("utf-8")) + except BaseException as exc: + _callback_fail(out_err, exc, KitchenErrorsError) + + +def _ReadyListener_latest(ctx: Optional[int], out_ptr: Any, out_len: Any, out_err: Any) -> None: + try: + _ret = _callback_get(ctx).latest() + _w = _Writer() + _write_opt_Item(_w, _ret) + _callback_return_bytes(out_ptr, out_len, _w.finish()) + except BaseException as exc: + _callback_fail(out_err, exc) + + +def _ReadyListener_favorite(ctx: Optional[int], out_err: Any) -> Optional[int]: + try: + _ret = _callback_get(ctx).favorite() + return _callback_return_object(_ret, Gadget) + except BaseException as exc: + _callback_fail(out_err, exc) + return None + + _ReadyListener_vtable = _ReadyListenerVtable( + ctypes.sizeof(_ReadyListenerVtable), + 0, + _callback_free_fn, _ReadyListener_on_ready_t(_ReadyListener_on_ready), _ReadyListener_on_item_t(_ReadyListener_on_item), - _ReadyListener_free_t(_callback_free), + _ReadyListener_label_t(_ReadyListener_label), + _ReadyListener_latest_t(_ReadyListener_latest), + _ReadyListener_favorite_t(_ReadyListener_favorite), ) +_ReadyListener_vtable_ptr = ctypes.addressof(_ReadyListener_vtable) _c_kitchen_Gadget_new = _bind("kitchen_sink_kitchen_Gadget_new", ctypes.c_void_p, ctypes.c_int64, ctypes.POINTER(_ErrorStruct)) @@ -911,7 +1091,8 @@ class Gadget(_Object): """Create a gadget with the given id""" _err = _ErrorStruct() _ret = _c_kitchen_Gadget_new(id, ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) self._init_handle(_required(_ret)) def describe(self) -> str: @@ -923,7 +1104,8 @@ class Gadget(_Object): _ret = _c_kitchen_Gadget_describe(_self_p, ctypes.byref(_out_len), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) def poke(self, times: int) -> int: @@ -937,10 +1119,11 @@ class Gadget(_Object): _err = _ErrorStruct() _self_p = self._acquire() try: - _ret = _c_kitchen_Gadget_poke(_self_p, times, ctypes.byref(_err)) + _ret: int = _c_kitchen_Gadget_poke(_self_p, times, ctypes.byref(_err)) finally: self._release() - _check_kitchen_errors_error(_err) + if _err.code: + raise _kitchen_errors_error_from(*_read_error(_err)) return _ret @staticmethod @@ -949,7 +1132,8 @@ class Gadget(_Object): _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_Gadget_version(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) @@ -959,8 +1143,9 @@ _c_kitchen_bool_id = _bind("kitchen_sink_kitchen_bool_id", ctypes.c_bool, ctypes def bool_id(x: bool) -> bool: """Identity for bool""" _err = _ErrorStruct() - _ret = _c_kitchen_bool_id(x, ctypes.byref(_err)) - _check_error(_err) + _ret: bool = _c_kitchen_bool_id(x, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @@ -969,8 +1154,9 @@ _c_kitchen_i32_id = _bind("kitchen_sink_kitchen_i32_id", ctypes.c_int32, ctypes. def i32_id(x: int) -> int: _err = _ErrorStruct() - _ret = _c_kitchen_i32_id(x, ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_kitchen_i32_id(x, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @@ -979,8 +1165,9 @@ _c_kitchen_u32_id = _bind("kitchen_sink_kitchen_u32_id", ctypes.c_uint32, ctypes def u32_id(x: int) -> int: _err = _ErrorStruct() - _ret = _c_kitchen_u32_id(x, ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_kitchen_u32_id(x, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @@ -989,8 +1176,9 @@ _c_kitchen_i64_id = _bind("kitchen_sink_kitchen_i64_id", ctypes.c_int64, ctypes. def i64_id(x: int) -> int: _err = _ErrorStruct() - _ret = _c_kitchen_i64_id(x, ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_kitchen_i64_id(x, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @@ -999,8 +1187,9 @@ _c_kitchen_f64_id = _bind("kitchen_sink_kitchen_f64_id", ctypes.c_double, ctypes def f64_id(x: float) -> float: _err = _ErrorStruct() - _ret = _c_kitchen_f64_id(x, ctypes.byref(_err)) - _check_error(_err) + _ret: float = _c_kitchen_f64_id(x, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @@ -1012,7 +1201,8 @@ def echo_string(s: str) -> str: _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_echo_string(_s_b, len(_s_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) @@ -1024,26 +1214,41 @@ def echo_bytes(b: bytes) -> bytes: _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_echo_bytes(_b_b, len(_b_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_bytes(_ret, _out_len.value) _c_kitchen_subscribe = _bind("kitchen_sink_kitchen_subscribe", ctypes.c_int32, ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) -def subscribe(listener: "ReadyListener") -> int: +def subscribe(listener: ReadyListener) -> int: """Register a listener, returning how many are attached""" _err = _ErrorStruct() _listener_ctx = _callback_register(listener, ReadyListener) - _ret = _c_kitchen_subscribe(_listener_ctx, ctypes.addressof(_ReadyListener_vtable), ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_kitchen_subscribe(_listener_ctx, _ReadyListener_vtable_ptr, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _ret + + +_c_kitchen_maybe_subscribe = _bind("kitchen_sink_kitchen_maybe_subscribe", ctypes.c_bool, ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) + + +def maybe_subscribe(listener: Optional[ReadyListener]) -> bool: + """Register a listener if one is given""" + _err = _ErrorStruct() + _listener_ctx = _callback_register_opt(listener, ReadyListener) + _ret: bool = _c_kitchen_maybe_subscribe(_listener_ctx, _ReadyListener_vtable_ptr if listener is not None else None, ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret _c_kitchen_maybe_gadget = _bind("kitchen_sink_kitchen_maybe_gadget", ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) -def maybe_gadget(fallback: Optional["Gadget"]) -> Optional["Gadget"]: +def maybe_gadget(fallback: Optional[Gadget]) -> Optional[Gadget]: """Optional object in both directions""" _err = _ErrorStruct() _fallback_p = _lend_opt(fallback, Gadget) @@ -1051,20 +1256,22 @@ def maybe_gadget(fallback: Optional["Gadget"]) -> Optional["Gadget"]: _ret = _c_kitchen_maybe_gadget(_fallback_p, ctypes.byref(_err)) finally: _release_opt(fallback) - _check_error(_err) - return (Gadget._adopt(_ret) if _ret else None) + if _err.code: + raise _trap_from(*_read_error(_err)) + return Gadget._adopt(_ret) if _ret else None _c_kitchen_gadgets = _bind("kitchen_sink_kitchen_gadgets", ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def gadgets() -> List["Gadget"]: +def gadgets() -> List[Gadget]: """A list of objects""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_gadgets(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) - return _decode(_take_bytes(_ret, _out_len.value), lambda _r: [Gadget._adopt(_r.read_object()) for _i0 in range(_r.read_count())]) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _decode(_take_bytes(_ret, _out_len.value), _read_list_Gadget) _c_kitchen_stream_gadgets = _bind("kitchen_sink_kitchen_stream_gadgets", ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) @@ -1078,7 +1285,7 @@ class _kitchen_StreamGadgetsIterator(_Iterator): _destroy = staticmethod(_c_kitchen_StreamGadgetsIterator_destroy) - def __next__(self) -> "Gadget": + def __next__(self) -> Gadget: _item = ctypes.c_void_p() _err = _ErrorStruct() _p = self._step() @@ -1086,16 +1293,16 @@ class _kitchen_StreamGadgetsIterator(_Iterator): _more = _c_kitchen_StreamGadgetsIterator_next(_p, ctypes.byref(_item), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) if not _more: self.close() raise StopIteration return Gadget._adopt(_required(_item.value)) -def stream_gadgets() -> Iterator["Gadget"]: - """ - An iterator over objects +def stream_gadgets() -> Iterator[Gadget]: + """An iterator over objects Returns a lazy iterator: each step pulls one element from the producer. Exhaust or close() the iterator to release its native @@ -1103,14 +1310,15 @@ def stream_gadgets() -> Iterator["Gadget"]: """ _err = _ErrorStruct() _ret = _c_kitchen_stream_gadgets(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _kitchen_StreamGadgetsIterator._adopt(_ret) _c_kitchen_maybe_item = _bind("kitchen_sink_kitchen_maybe_item", ctypes.c_void_p, ctypes.c_int64, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def maybe_item(id: int) -> Optional["Item"]: +def maybe_item(id: int) -> Optional[Item]: """Optional struct return Raises @@ -1121,20 +1329,22 @@ def maybe_item(id: int) -> Optional["Item"]: _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_maybe_item(id, ctypes.byref(_out_len), ctypes.byref(_err)) - _check_kitchen_errors_error(_err) - return _decode(_take_bytes(_ret, _out_len.value), lambda _r: (_read_Item(_r) if _r.read_flag() else None)) + if _err.code: + raise _kitchen_errors_error_from(*_read_error(_err)) + return _decode(_take_bytes(_ret, _out_len.value), _read_opt_Item) _c_kitchen_list_items = _bind("kitchen_sink_kitchen_list_items", ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def list_items() -> List["Item"]: +def list_items() -> List[Item]: """List of structs""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_list_items(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) - return _decode(_take_bytes(_ret, _out_len.value), lambda _r: [_read_Item(_r) for _i0 in range(_r.read_count())]) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _decode(_take_bytes(_ret, _out_len.value), _read_list_Item) _c_kitchen_get_attrs = _bind("kitchen_sink_kitchen_get_attrs", ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) @@ -1145,8 +1355,9 @@ def get_attrs() -> Dict[str, int]: _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_get_attrs(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) - return _decode(_take_bytes(_ret, _out_len.value), lambda _r: dict((_r.read_string(), _r.read_i32()) for _i0 in range(_r.read_count()))) + if _err.code: + raise _trap_from(*_read_error(_err)) + return _decode(_take_bytes(_ret, _out_len.value), _read_map_string_i32) _c_kitchen_stream_items = _bind("kitchen_sink_kitchen_stream_items", ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) @@ -1160,7 +1371,7 @@ class _kitchen_StreamItemsIterator(_Iterator): _destroy = staticmethod(_c_kitchen_StreamItemsIterator_destroy) - def __next__(self) -> "Item": + def __next__(self) -> Item: _item = ctypes.c_void_p() _len = ctypes.c_size_t() _err = _ErrorStruct() @@ -1169,16 +1380,16 @@ class _kitchen_StreamItemsIterator(_Iterator): _more = _c_kitchen_StreamItemsIterator_next(_p, ctypes.byref(_item), ctypes.byref(_len), ctypes.byref(_err)) finally: self._release() - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) if not _more: self.close() raise StopIteration return _decode(_take_bytes(_item.value, _len.value), _read_Item) -def stream_items() -> Iterator["Item"]: - """ - Iterator return type +def stream_items() -> Iterator[Item]: + """Iterator return type Returns a lazy iterator: each step pulls one element from the producer. Exhaust or close() the iterator to release its native @@ -1186,19 +1397,21 @@ def stream_items() -> Iterator["Item"]: """ _err = _ErrorStruct() _ret = _c_kitchen_stream_items(ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _kitchen_StreamItemsIterator._adopt(_ret) _c_kitchen_cross_module_token = _bind("kitchen_sink_kitchen_cross_module_token", ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def cross_module_token() -> "Token": +def cross_module_token() -> Token: """Returns the shared Token type from another module""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_cross_module_token(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Token) @@ -1206,12 +1419,12 @@ _c_kitchen_do_async_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.PO _c_kitchen_do_async = _bind("kitchen_sink_kitchen_do_async", None, ctypes.c_char_p, ctypes.c_size_t, _c_kitchen_do_async_callback, ctypes.c_void_p) -def _kitchen_do_async_complete(context, err, result_ptr, result_len) -> None: +def _kitchen_do_async_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: _async_settle(context, None, _take_str(result_ptr, result_len)) except BaseException as exc: @@ -1230,19 +1443,20 @@ async def do_async(input: str) -> str: except BaseException: _async_abandon(_call) raise - return await _future + _result: str = await _future + return _result _c_kitchen_do_cancellable_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct), ctypes.c_void_p, ctypes.c_size_t) _c_kitchen_do_cancellable = _bind("kitchen_sink_kitchen_do_cancellable", None, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_void_p, _c_kitchen_do_cancellable_callback, ctypes.c_void_p) -def _kitchen_do_cancellable_complete(context, err, result_ptr, result_len) -> None: +def _kitchen_do_cancellable_complete(context: Optional[int], err: Any, result_ptr: Optional[int], result_len: int) -> None: # Runs once, on a producer thread: take ownership of the result # here, then hand it to the awaiting event loop. try: if err: - _async_settle(context, _async_error(err, _error_from), None) + _async_settle(context, _async_error(err, _trap_from), None) else: _async_settle(context, None, _take_str(result_ptr, result_len)) except BaseException as exc: @@ -1263,18 +1477,23 @@ async def do_cancellable(input: str) -> str: _async_abandon(_call) _cancel_token_destroy(_token) raise - return await _async_wait_cancellable(_future, _token) + _result: str = await _async_wait_cancellable(_future, _token) + return _result _c_kitchen_legacy_op = _bind("kitchen_sink_kitchen_legacy_op", ctypes.c_int32, ctypes.POINTER(_ErrorStruct)) def legacy_op() -> int: - """Legacy operation kept for compatibility""" + """Legacy operation kept for compatibility + + Deprecated: Use new_op instead + """ warnings.warn("Use new_op instead", DeprecationWarning, stacklevel=2) _err = _ErrorStruct() - _ret = _c_kitchen_legacy_op(ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_kitchen_legacy_op(ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @@ -1284,8 +1503,9 @@ _c_kitchen_new_op = _bind("kitchen_sink_kitchen_new_op", ctypes.c_int32, ctypes. def new_op() -> int: """Replacement for legacy_op""" _err = _ErrorStruct() - _ret = _c_kitchen_new_op(ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_kitchen_new_op(ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret @@ -1300,7 +1520,138 @@ def hello() -> str: _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_kitchen_nested_hello(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) + +# === Value-buffer codecs of composite types === + + +def _write_list_string(_w: _Writer, value: List[str]) -> None: + _w.write_count(len(value)) + for item in value: + _w.write_string(item) + + +def _read_list_string(_r: _Reader) -> List[str]: + return [_r.read_string() for _ in range(_r.read_count())] + + +def _write_map_string_string(_w: _Writer, value: Dict[str, str]) -> None: + _w.write_count(len(value)) + for key, item in value.items(): + _w.write_string(key) + _w.write_string(item) + + +def _read_map_string_string(_r: _Reader) -> Dict[str, str]: + n = _r.read_count() + value = {_r.read_string(): _r.read_string() for _ in range(n)} + if len(value) != n: + raise _malformed("repeated map key") + return value + + +def _write_opt_i64(_w: _Writer, value: Optional[int]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_i64(value) + + +def _read_opt_i64(_r: _Reader) -> Optional[int]: + return _r.read_i64() if _r.read_flag() else None + + +def _write_opt_Gadget(_w: _Writer, value: Optional[Gadget]) -> None: + _w.write_flag(value is not None) + if value is not None: + _w.write_object(value, Gadget) + + +def _read_opt_Gadget(_r: _Reader) -> Optional[Gadget]: + return Gadget._adopt(_r.read_object()) if _r.read_flag() else None + + +def _write_list_Gadget(_w: _Writer, value: List[Gadget]) -> None: + _w.write_count(len(value)) + for item in value: + _w.write_object(item, Gadget) + + +def _read_list_Gadget(_r: _Reader) -> List[Gadget]: + return [Gadget._adopt(_r.read_object()) for _ in range(_r.read_count())] + + +def _write_opt_Item(_w: _Writer, value: Optional[Item]) -> None: + _w.write_flag(value is not None) + if value is not None: + _write_Item(_w, value) + + +def _read_opt_Item(_r: _Reader) -> Optional[Item]: + return _read_Item(_r) if _r.read_flag() else None + + +def _write_list_Item(_w: _Writer, value: List[Item]) -> None: + _w.write_count(len(value)) + for item in value: + _write_Item(_w, item) + + +def _read_list_Item(_r: _Reader) -> List[Item]: + return [_read_Item(_r) for _ in range(_r.read_count())] + + +def _write_map_string_i32(_w: _Writer, value: Dict[str, int]) -> None: + _w.write_count(len(value)) + for key, item in value.items(): + _w.write_string(key) + _w.write_i32(item) + + +def _read_map_string_i32(_r: _Reader) -> Dict[str, int]: + n = _r.read_count() + value = {_r.read_string(): _r.read_i32() for _ in range(n)} + if len(value) != n: + raise _malformed("repeated map key") + return value + + +__all__ = [ + "Error", + "InternalError", + "Token", + "ping", + "KitchenErrorsError", + "NotFound", + "InvalidInput", + "Priority", + "Item", + "ReadyListener", + "Gadget", + "bool_id", + "i32_id", + "u32_id", + "i64_id", + "f64_id", + "echo_string", + "echo_bytes", + "subscribe", + "maybe_subscribe", + "maybe_gadget", + "gadgets", + "stream_gadgets", + "maybe_item", + "list_items", + "get_attrs", + "stream_items", + "cross_module_token", + "do_async", + "do_cancellable", + "legacy_op", + "new_op", + "hello", +] + # END kitchen_sink.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_pyi.snap b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_pyi.snap deleted file mode 100644 index 5c24161c..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_kitchen_sink_pyi.snap +++ /dev/null @@ -1,155 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -import builtins -from abc import ABC, abstractmethod -from enum import IntEnum -from typing import Dict, Iterator, List, Optional, Type - -class Error(Exception): - GENERIC_ERROR_CODE: int - PANIC_ERROR_CODE: int - MARSHAL_ERROR_CODE: int - FOREIGN_ERROR_CODE: int - CANCELLED_ERROR_CODE: int - code: int - message: str - def __init__(self, code: int, message: str) -> None: ... - -# A reusable token referenced across modules -class Token: - id: int - label: str - def __init__(self, id: int, label: str) -> None: ... - -# Trivial cross-module helper -def ping() -> str: ... - -class KitchenErrorsError(Error): - NotFound: Type["NotFound"] - InvalidInput: Type["InvalidInput"] - def __init__(self, code: int, message: str) -> None: ... - -class NotFound(KitchenErrorsError): - CODE: int - def __init__(self, message: str = ...) -> None: ... - -class InvalidInput(KitchenErrorsError): - CODE: int - def __init__(self, message: str = ...) -> None: ... - -# Task priority levels -class Priority(IntEnum): - # Low priority - Low = 0 - # Normal priority - Normal = 1 - # High priority - High = 2 - -# Kitchen sink struct exercising every field feature -class Item: - # Stable identifier - id: int - # Display name - name: str - # Initial count - count: int - # Whether the item is enabled - enabled: bool - # Scaling ratio - ratio: float - nick: int - payload: bytes - tags: List[str] - attrs: Dict[str, str] - parent: Optional[int] - # Cross-module struct reference - token: "Token" - priority: "Priority" - # An optional object inside a record - gadget: Optional["Gadget"] - def __init__(self, id: int, name: str, count: int, enabled: bool, ratio: float, nick: int, payload: bytes, tags: List[str], attrs: Dict[str, str], parent: Optional[int], token: "Token", priority: "Priority", gadget: Optional["Gadget"]) -> None: ... - -# Consumer-implemented observer of ready items -class ReadyListener(ABC): - # Fires when an item is ready - @abstractmethod - def on_ready(self, code: int, msg: str) -> None: ... - # Receives the item itself and says whether to keep listening - @abstractmethod - def on_item(self, item: "Item", gadget: "Gadget") -> bool: ... - -# A pocket-sized interface exercising the object surface -class Gadget: - def __init__(self, id: int) -> None: ... - def close(self) -> None: ... - def __enter__(self) -> "Gadget": ... - def __exit__(self, *exc: object) -> None: ... - def describe(self) -> str: ... - def poke(self, times: int) -> int: ... - @staticmethod - def version() -> str: ... - -# Identity for bool -def bool_id(x: bool) -> bool: ... - -def i32_id(x: int) -> int: ... - -def u32_id(x: int) -> int: ... - -def i64_id(x: int) -> int: ... - -def f64_id(x: float) -> float: ... - -def echo_string(s: str) -> str: ... - -def echo_bytes(b: bytes) -> bytes: ... - -# Register a listener, returning how many are attached -def subscribe(listener: "ReadyListener") -> int: ... - -# Optional object in both directions -def maybe_gadget(fallback: Optional["Gadget"]) -> Optional["Gadget"]: ... - -# A list of objects -def gadgets() -> List["Gadget"]: ... - -# An iterator over objects -def stream_gadgets() -> Iterator["Gadget"]: ... - -# Optional struct return -def maybe_item(id: int) -> Optional["Item"]: ... - -# List of structs -def list_items() -> List["Item"]: ... - -# Map return type -def get_attrs() -> Dict[str, int]: ... - -# Iterator return type -def stream_items() -> Iterator["Item"]: ... - -# Returns the shared Token type from another module -def cross_module_token() -> "Token": ... - -# Async operation -async def do_async(input: str) -> str: ... - -# Cancellable async operation -async def do_cancellable(input: str) -> str: ... - -# Legacy operation kept for compatibility -def legacy_op() -> int: ... - -# Replacement for legacy_op -def new_op() -> int: ... - -# Trivial nested-module function -def hello() -> str: ... - -# END kitchen_sink.pyi diff --git a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_py_typed.snap b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_py_typed.snap index f8cd271e..6ac1df61 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_py_typed.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__kitchen_sink_py_typed.snap @@ -1,6 +1,6 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate diff --git a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__pyproject_toml.snap b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__pyproject_toml.snap index 2a35fd2b..6cc575f6 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__pyproject_toml.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_kitchen_sink__pyproject_toml.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate [build-system] requires = ["setuptools>=61.0"] @@ -19,6 +19,6 @@ requires-python = ">=3.9" packages = ["kitchen_sink"] [tool.setuptools.package-data] -"kitchen_sink" = ["py.typed", "*.pyi", "*.so", "*.dylib", "*.dll"] +"kitchen_sink" = ["py.typed", "*.so", "*.dylib", "*.dll"] # END pyproject.toml diff --git a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/python_nested_modules__README_md.snap deleted file mode 100644 index 04250551..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__README_md.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# nested_modules (Python) - -Python bindings for the `nested_modules` native library, built on `ctypes`. - -## Requirements - -- Python 3.9 or later -- The native library (`libnested_modules.so`, `libnested_modules.dylib`, or `nested_modules.dll`) - -## Install - -```bash -pip install . -``` - -## Loading the native library - -The bindings load the library from `NESTED_MODULES_LIBRARY` when it is set (a full path), -then from inside the installed package, then from the system loader's search -path. - -## Usage - -```python -import nested_modules -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules___init___py.snap b/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules___init___py.snap deleted file mode 100644 index 03409d55..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules___init___py.snap +++ /dev/null @@ -1,10 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -from .nested_modules import * # noqa: F401,F403 - -# END __init__.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_py.snap b/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_py.snap index 9c61443a..6232b8e9 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_py.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_py.snap @@ -1,11 +1,13 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate """ctypes bindings for the nested_modules native library.""" +from __future__ import annotations + import builtins import ctypes import os @@ -15,18 +17,19 @@ import threading import warnings from dataclasses import dataclass from enum import IntEnum -from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type +from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type, TypeVar class Error(Exception): - """An error reported by the native library. - - `code` is positive for one of the API's declared error codes (raised as - the declaring module's typed subclasses) and negative for a runtime trap: - a generic failure (-1), a producer panic (-2), a marshalling failure - (-3), or an exception raised by a callback-interface implementation - (-4). A cancelled async call (-5) raises `asyncio.CancelledError` - instead. + """An error a fallible call reports. + + `code` is positive for one of the API's declared error codes, raised as + the declaring module's typed subclasses, and negative for a runtime + failure of a call that declares errors: a generic failure (-1), a + producer panic (-2), a marshalling failure (-3), or an exception raised + by a callback-interface implementation (-4). A cancelled async call (-5) + raises `asyncio.CancelledError` instead. A call that declares no errors + never raises this; its failures are bugs and raise `InternalError`. """ GENERIC_ERROR_CODE = -1 @@ -40,6 +43,24 @@ class Error(Exception): self.code = code self.message = message + def _payload(self) -> bytes: + """The code's fields as a value buffer (a code with fields overrides + this).""" + return b"" + + +class InternalError(RuntimeError): + """A failure of a call that declares no errors: a producer panic (-2), a + marshalling failure (-3), a callback-interface failure the producer let + through (-4), or another runtime failure (-1). These are bugs, so they + are unchecked; `code` and `message` say what went wrong. + """ + + def __init__(self, code: int, message: str) -> None: + super().__init__(f"({code}) {message}") + self.code = code + self.message = message + class _ErrorStruct(ctypes.Structure): """The C `nested_modules_error` out-parameter every call reports through.""" @@ -89,18 +110,35 @@ def _bind(name: str, restype: Any, *argtypes: Any) -> Any: return fn -# The C ABI revision these bindings were generated against, and the contract -# checksum of every top-level module: (module, checksum symbol, value). -_ABI_VERSION = 3 -_CHECKSUMS: List[Tuple[str, str, int]] = [ - ("graphics", "nested_modules_graphics_checksum", 0x44d31b5b10fdb90d), -] +class _ContractEntry(ctypes.Structure): + """The C `nested_modules_contract_entry`: one declaration's id and hash.""" + + _fields_ = [("id", ctypes.c_uint64), ("hash", ctypes.c_uint64)] + + +# The C ABI revision these bindings were generated against and, per +# top-level module's contract symbol, the declarations they rely on: +# (id, hash, dotted path). +_ABI_VERSION = 4 +_CONTRACTS: Dict[str, List[Tuple[int, int, str]]] = { + "nested_modules_graphics_contract": [ + (0x2c29eef9327a6bd4, 0x12983d5271651389, "graphics.Unit"), + (0x304088bbc28f0015, 0x97e6e809913be6b6, "graphics.new_canvas"), + (0x49f8837532df9ac6, 0x32e5b27e81ee0957, "graphics.Canvas"), + (0x55717fa027336b47, 0xb2bd3a6bdcce48c8, "graphics.shapes.describe"), + (0x7626b91b8f1f53cd, 0x867bbf7cf0f41c74, "graphics.shapes.geometry.compute"), + (0xb3efcb50d9a61ab7, 0x3cb0207dfad2a149, "graphics.GraphicsError"), + (0xc62b16a9591b744b, 0x7895610c8487eb1b, "graphics.shapes.Shape"), + (0xe13001f4983ccd35, 0x16e7de070b4a4cfc, "graphics.shapes.geometry.Detail"), + (0xf3f658bcfc5dba62, 0x2665ccb47356e746, "graphics.shapes.make_shape"), + ], +} def _check_contract() -> None: - """Refuse to load a library built for a different ABI revision or a - different version of any module's API, before any other symbol is - bound.""" + """Refuse to load a library built for another ABI revision, or one that + lacks or changed a declaration these bindings use, before any other + symbol is bound.""" try: abi_version = _bind("nested_modules_abi_version", ctypes.c_uint32) except AttributeError: @@ -114,33 +152,40 @@ def _check_contract() -> None: f"C ABI mismatch: these bindings expect revision {_ABI_VERSION}, " f"but {_lib._name} reports revision {found}" ) - for module, symbol, expected in _CHECKSUMS: + for symbol, expected in _CONTRACTS.items(): try: - checksum = _bind(symbol, ctypes.c_uint64) - except AttributeError: - raise ImportError( - f"module {module!r}: {_lib._name} does not export {symbol}" - ) from None - found = checksum() - if found != expected: - raise ImportError( - f"module {module!r}: the native library's contract checksum " - f"{found:#018x} does not match these bindings ({expected:#018x}); " - "regenerate the bindings from the library's current API" + contract = _bind( + symbol, ctypes.POINTER(_ContractEntry), ctypes.POINTER(ctypes.c_size_t) ) + except AttributeError: + raise ImportError(f"{_lib._name} does not export {symbol}") from None + length = ctypes.c_size_t() + table = contract(ctypes.byref(length)) + hashes = {table[i].id: table[i].hash for i in range(length.value)} if table else {} + for entry_id, entry_hash, path in expected: + actual = hashes.get(entry_id) + if actual is None: + raise ImportError(f"{path} is missing from the library ({_lib._name})") + if actual != entry_hash: + raise ImportError( + f"{path} changed since these bindings were generated ({_lib._name})" + ) _check_contract() _error_clear = _bind("nested_modules_error_clear", None, ctypes.POINTER(_ErrorStruct)) _free_bytes = _bind("nested_modules_free_bytes", None, ctypes.c_void_p, ctypes.c_size_t) +_debug_live_fn = _bind("nested_modules_debug_live", ctypes.c_uint64, ctypes.c_int32) def _debug_live(kind: int) -> int: - """The producer's live-allocation counter `kind` (0 objects, 1 callbacks, - 2 iterators, 3 cancel tokens, 4 returned allocations). Always 0 unless - the producer was built with its leak counters enabled.""" - return _bind("nested_modules_debug_live", ctypes.c_uint64, ctypes.c_int32)(kind) + """The producer's live-resource counter `kind` (0 objects, 1 callbacks, + 2 iterators, 3 cancel tokens, 4 byte runs; -1 is 1 when the producer + counts at all). Always 0 unless the producer was built with its leak + counters enabled.""" + live: int = _debug_live_fn(kind) + return live def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: @@ -153,17 +198,9 @@ def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: return code, message, payload -def _error_from(code: int, message: str, payload: bytes = b"") -> Error: - """The exception for a code no declared error domain claims.""" - return Error(code, message) - - -def _check_error(err: _ErrorStruct) -> None: - """Raise for a non-zero out-err slot of a call that declares no errors: - only runtime traps (a panic, a marshalling failure, a failed callback) - land here.""" - if err.code: - raise _error_from(*_read_error(err)) +def _trap_from(code: int, message: str, payload: bytes = b"") -> InternalError: + """The exception for a failed call that declares no errors.""" + return InternalError(code, message) def _take_bytes(ptr: Optional[int], length: int) -> bytes: @@ -187,6 +224,9 @@ def _peek_bytes(ptr: Optional[int], length: int) -> bytes: return ctypes.string_at(ptr, length) if ptr and length else b"" +_H = TypeVar("_H", bound="_Handle") + + class _Handle: """Owns one native reference and releases it exactly once. @@ -200,12 +240,13 @@ class _Handle: def _init_handle(self, ptr: int) -> None: self._ptr: Optional[int] = ptr + self._key = ptr self._calls = 0 self._closing = False self._lock = threading.Lock() @classmethod - def _adopt(cls, ptr: int) -> Any: + def _adopt(cls: Type[_H], ptr: int) -> _H: """Wrap one strong reference the producer handed over.""" obj = cls.__new__(cls) obj._init_handle(ptr) @@ -215,7 +256,7 @@ class _Handle: with self._lock: ptr = self._ptr if ptr is None or self._closing: - raise Error(-1, f"{type(self).__name__} used after close()") + raise ValueError(f"{type(self).__name__} is closed") self._calls += 1 return ptr @@ -229,8 +270,8 @@ class _Handle: type(self)._destroy(ptr) def close(self) -> None: - """Release this wrapper's native reference. Idempotent; the wrapper - is unusable afterwards.""" + """Release the native reference. Idempotent; the wrapper is unusable + afterwards.""" lock = getattr(self, "_lock", None) if lock is None: return @@ -244,7 +285,7 @@ class _Handle: if ptr is not None: type(self)._destroy(ptr) - def __enter__(self) -> Any: + def __enter__(self: _H) -> _H: return self def __exit__(self, *exc: object) -> None: @@ -259,27 +300,36 @@ class _Handle: class _Object(_Handle): - """Base of every interface wrapper: a reference-counted native object.""" + """Base of every interface wrapper: a reference-counted native object. + Two wrappers are equal when they hold the same native object.""" # The native clone function, set per subclass. _clone: Callable[[int], int] def _clone_ref(self) -> int: """A new strong reference to the same object, for a receiver that - adopts it (an object token inside a value buffer).""" + adopts it (an object token, or a callback method's return).""" ptr = self._acquire() try: return type(self)._clone(ptr) finally: self._release() + def __eq__(self, other: object) -> bool: + if not isinstance(other, _Object): + return NotImplemented + return self is other or (self._ptr is not None and self._ptr == other._ptr) + + def __hash__(self) -> int: + return hash(self._key) + class _Iterator(_Handle): """Base of every lazy iterator over a producer stream: one native `next` call per step; the handle is released on exhaustion, `close()`, or garbage collection.""" - def __iter__(self) -> Any: + def __iter__(self: _H) -> _H: return self def _step(self) -> int: @@ -294,23 +344,23 @@ class _Iterator(_Handle): def _required(ptr: Optional[int]) -> int: """A non-null object pointer the producer returned.""" if not ptr: - raise Error(-1, "the producer returned a null object") + raise InternalError(-3, "the producer returned a null object") return ptr -def _lend(obj: Any, cls: "Type[_Object]") -> int: +def _lend(obj: Any, cls: Type[_Object]) -> int: """Lend `obj`'s native pointer to one call; pair with `obj._release()`.""" if not isinstance(obj, cls): raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") return obj._acquire() -def _lend_opt(obj: Any, cls: "Type[_Object]") -> Optional[int]: +def _lend_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: """`_lend` for an optional object: None lends a null pointer.""" return None if obj is None else _lend(obj, cls) -def _release_opt(obj: Any) -> None: +def _release_opt(obj: Optional[_Object]) -> None: if obj is not None: obj._release() @@ -326,9 +376,12 @@ _U64 = struct.Struct(" Error: - return Error(-3, f"malformed value buffer: {what}") +def _malformed(what: str) -> InternalError: + return InternalError(-3, f"malformed value buffer: {what}") class _Writer: @@ -411,7 +464,7 @@ class _Writer: self._buf += _U32.pack(len(values)) self._buf += struct.pack(f"<{len(values)}{code}", *values) - def write_object(self, obj: Any, cls: "Type[_Object]") -> None: + def write_object(self, obj: Any, cls: Type[_Object]) -> None: """Reserve an object token: one strong reference to `obj` the reader adopts, cloned in `finish()`. A closed wrapper is rejected now.""" if not isinstance(obj, cls): @@ -424,7 +477,8 @@ class _Writer: class _Reader: """Decodes values from the value-buffer wire format, rejecting truncated - buffers, invalid flag bytes, invalid UTF-8, and trailing data.""" + buffers, invalid flag bytes, invalid UTF-8, undeclared enum values, and + trailing data.""" __slots__ = ("_data", "_pos") @@ -497,6 +551,13 @@ class _Reader: except UnicodeDecodeError: raise _malformed("string is not valid UTF-8") from None + def read_enum(self, cls: Type[_E]) -> _E: + value = self.read_i32() + try: + return cls(value) + except ValueError: + raise _malformed(f"undeclared {cls.__name__} value {value}") from None + def read_numbers(self, code: str, size: int) -> List[Any]: """A list of one fixed-width numeric primitive, unpacked in one call.""" n = self.read_count() @@ -515,7 +576,7 @@ class _Reader: raise _malformed("trailing bytes") -def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: +def _decode(data: bytes, read: Callable[[_Reader], _T]) -> _T: """Decode exactly one value from `data`.""" r = _Reader(data) value = read(r) @@ -529,7 +590,7 @@ def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: class GraphicsError(Error): """Base exception for the `graphics` module's error domain.""" - OutOfBounds: "Type[OutOfBounds]" + OutOfBounds: Type[OutOfBounds] class OutOfBounds(GraphicsError): @@ -544,28 +605,18 @@ class OutOfBounds(GraphicsError): GraphicsError.OutOfBounds = OutOfBounds -_GRAPHICS_ERROR_CODES: Dict[int, Any] = { - 1: OutOfBounds, -} - - def _graphics_error_from(code: int, message: str, payload: bytes = b"") -> Error: - """Build the GraphicsError subclass matching `code`, or the plain - Error for codes outside the domain (panics, marshalling).""" - cls = _GRAPHICS_ERROR_CODES.get(code) - if cls is None: - return Error(code, message) - exc: Any = cls(message) if message else cls() - return exc - - -def _check_graphics_error(err: _ErrorStruct) -> None: - if err.code: - raise _graphics_error_from(*_read_error(err)) + """The GraphicsError subclass for `code` with its payload fields, or the + plain Error for a runtime code (a panic, a marshalling failure). + """ + if code == 1: + return OutOfBounds(message) + return Error(code, message) class Unit(IntEnum): """A measurement unit shared with nested submodules.""" + pixels = 0 points = 1 @@ -576,15 +627,15 @@ class Canvas: width: int # Reference down into a nested module - root: "Shape" + root: Shape -def _write_Canvas(_w: _Writer, value: "Canvas") -> None: +def _write_Canvas(_w: _Writer, value: Canvas) -> None: _w.write_i32(value.width) _write_Shape(_w, value.root) -def _read_Canvas(_r: _Reader) -> "Canvas": +def _read_Canvas(_r: _Reader) -> Canvas: return Canvas( width=_r.read_i32(), root=_read_Shape(_r), @@ -594,12 +645,13 @@ def _read_Canvas(_r: _Reader) -> "Canvas": _c_graphics_new_canvas = _bind("nested_modules_graphics_new_canvas", ctypes.c_void_p, ctypes.c_int32, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def new_canvas(width: int) -> "Canvas": +def new_canvas(width: int) -> Canvas: """Construct a Canvas.""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_graphics_new_canvas(width, ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Canvas) @@ -612,21 +664,21 @@ class Shape: sides: int # Reference up to the parent module's enum - unit: "Unit" + unit: Unit # Reference down into a deeper module - detail: "Detail" + detail: Detail -def _write_Shape(_w: _Writer, value: "Shape") -> None: +def _write_Shape(_w: _Writer, value: Shape) -> None: _w.write_i32(value.sides) _w.write_i32(value.unit) _write_Detail(_w, value.detail) -def _read_Shape(_r: _Reader) -> "Shape": +def _read_Shape(_r: _Reader) -> Shape: return Shape( sides=_r.read_i32(), - unit=Unit(_r.read_i32()), + unit=_r.read_enum(Unit), detail=_read_Detail(_r), ) @@ -634,24 +686,26 @@ def _read_Shape(_r: _Reader) -> "Shape": _c_graphics_shapes_make_shape = _bind("nested_modules_graphics_shapes_make_shape", ctypes.c_void_p, ctypes.c_int32, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def make_shape(sides: int) -> "Shape": +def make_shape(sides: int) -> Shape: """Construct a Shape.""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_graphics_shapes_make_shape(sides, ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Shape) _c_graphics_shapes_describe = _bind("nested_modules_graphics_shapes_describe", ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def describe() -> "Canvas": +def describe() -> Canvas: """Nested function returning a struct owned by an ancestor module.""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_graphics_shapes_describe(ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Canvas) @@ -665,11 +719,11 @@ class Detail: area: float -def _write_Detail(_w: _Writer, value: "Detail") -> None: +def _write_Detail(_w: _Writer, value: Detail) -> None: _w.write_f64(value.area) -def _read_Detail(_r: _Reader) -> "Detail": +def _read_Detail(_r: _Reader) -> Detail: return Detail( area=_r.read_f64(), ) @@ -678,7 +732,7 @@ def _read_Detail(_r: _Reader) -> "Detail": _c_graphics_shapes_geometry_compute = _bind("nested_modules_graphics_shapes_geometry_compute", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def compute(shape: "Shape") -> "Detail": +def compute(shape: Shape) -> Detail: """Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. Raises @@ -692,7 +746,24 @@ def compute(shape: "Shape") -> "Detail": _out_len = ctypes.c_size_t() _shape_b = _shape_w.finish() _ret = _c_graphics_shapes_geometry_compute(_shape_b, len(_shape_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_graphics_error(_err) + if _err.code: + raise _graphics_error_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Detail) + +__all__ = [ + "Error", + "InternalError", + "GraphicsError", + "OutOfBounds", + "Unit", + "Canvas", + "new_canvas", + "Shape", + "make_shape", + "describe", + "Detail", + "compute", +] + # END nested_modules.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_pyi.snap b/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_pyi.snap deleted file mode 100644 index 88754fc2..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_nested_modules_pyi.snap +++ /dev/null @@ -1,69 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -import builtins -from abc import ABC, abstractmethod -from enum import IntEnum -from typing import Dict, Iterator, List, Optional, Type - -class Error(Exception): - GENERIC_ERROR_CODE: int - PANIC_ERROR_CODE: int - MARSHAL_ERROR_CODE: int - FOREIGN_ERROR_CODE: int - CANCELLED_ERROR_CODE: int - code: int - message: str - def __init__(self, code: int, message: str) -> None: ... - -class GraphicsError(Error): - OutOfBounds: Type["OutOfBounds"] - def __init__(self, code: int, message: str) -> None: ... - -class OutOfBounds(GraphicsError): - CODE: int - def __init__(self, message: str = ...) -> None: ... - -# A measurement unit shared with nested submodules. -class Unit(IntEnum): - pixels = 0 - points = 1 - -# Top-level struct that references a struct defined one level deeper. -class Canvas: - width: int - # Reference down into a nested module - root: "Shape" - def __init__(self, width: int, root: "Shape") -> None: ... - -# Construct a Canvas. -def new_canvas(width: int) -> "Canvas": ... - -# A struct one level deep that references both an ancestor enum and a deeper struct. -class Shape: - sides: int - # Reference up to the parent module's enum - unit: "Unit" - # Reference down into a deeper module - detail: "Detail" - def __init__(self, sides: int, unit: "Unit", detail: "Detail") -> None: ... - -# Construct a Shape. -def make_shape(sides: int) -> "Shape": ... - -# Nested function returning a struct owned by an ancestor module. -def describe() -> "Canvas": ... - -# A struct two levels deep. -class Detail: - area: float - def __init__(self, area: float) -> None: ... - -# Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. -def compute(shape: "Shape") -> "Detail": ... - -# END nested_modules.pyi diff --git a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_py_typed.snap b/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_py_typed.snap deleted file mode 100644 index f8cd271e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__nested_modules_py_typed.snap +++ /dev/null @@ -1,6 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o diff --git a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__pyproject_toml.snap b/crates/weaveffi-cli/tests/snapshots/python_nested_modules__pyproject_toml.snap deleted file mode 100644 index ab68d0b3..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_nested_modules__pyproject_toml.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -[build-system] -requires = ["setuptools>=61.0"] -build-backend = "setuptools.build_meta" - -[project] -name = "nested_modules" -version = "0.1.0" -description = "Native bindings for nested_modules" -requires-python = ">=3.9" - -[tool.setuptools] -packages = ["nested_modules"] - -[tool.setuptools.package-data] -"nested_modules" = ["py.typed", "*.pyi", "*.so", "*.dylib", "*.dll"] - -# END pyproject.toml diff --git a/crates/weaveffi-cli/tests/snapshots/python_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/python_shapes__README_md.snap deleted file mode 100644 index 7c7e328e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_shapes__README_md.snap +++ /dev/null @@ -1,35 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# shapes (Python) - -Python bindings for the `shapes` native library, built on `ctypes`. - -## Requirements - -- Python 3.9 or later -- The native library (`libshapes.so`, `libshapes.dylib`, or `shapes.dll`) - -## Install - -```bash -pip install . -``` - -## Loading the native library - -The bindings load the library from `SHAPES_LIBRARY` when it is set (a full path), -then from inside the installed package, then from the system loader's search -path. - -## Usage - -```python -import shapes -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/python_shapes__pyproject_toml.snap b/crates/weaveffi-cli/tests/snapshots/python_shapes__pyproject_toml.snap deleted file mode 100644 index 5efd79d6..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_shapes__pyproject_toml.snap +++ /dev/null @@ -1,24 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -[build-system] -requires = ["setuptools>=61.0"] -build-backend = "setuptools.build_meta" - -[project] -name = "shapes" -version = "0.1.0" -description = "Native bindings for shapes" -requires-python = ">=3.9" - -[tool.setuptools] -packages = ["shapes"] - -[tool.setuptools.package-data] -"shapes" = ["py.typed", "*.pyi", "*.so", "*.dylib", "*.dll"] - -# END pyproject.toml diff --git a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes___init___py.snap b/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes___init___py.snap deleted file mode 100644 index 0029ff1c..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes___init___py.snap +++ /dev/null @@ -1,10 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -from .shapes import * # noqa: F401,F403 - -# END __init__.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_py_typed.snap b/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_py_typed.snap deleted file mode 100644 index f8cd271e..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_py_typed.snap +++ /dev/null @@ -1,6 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o diff --git a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_py.snap b/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_py.snap index 72a15b5f..6c94bd07 100644 --- a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_py.snap +++ b/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_py.snap @@ -1,11 +1,13 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate """ctypes bindings for the shapes native library.""" +from __future__ import annotations + import builtins import ctypes import os @@ -15,18 +17,19 @@ import threading import warnings from dataclasses import dataclass from enum import IntEnum -from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type +from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple, Type, TypeVar class Error(Exception): - """An error reported by the native library. - - `code` is positive for one of the API's declared error codes (raised as - the declaring module's typed subclasses) and negative for a runtime trap: - a generic failure (-1), a producer panic (-2), a marshalling failure - (-3), or an exception raised by a callback-interface implementation - (-4). A cancelled async call (-5) raises `asyncio.CancelledError` - instead. + """An error a fallible call reports. + + `code` is positive for one of the API's declared error codes, raised as + the declaring module's typed subclasses, and negative for a runtime + failure of a call that declares errors: a generic failure (-1), a + producer panic (-2), a marshalling failure (-3), or an exception raised + by a callback-interface implementation (-4). A cancelled async call (-5) + raises `asyncio.CancelledError` instead. A call that declares no errors + never raises this; its failures are bugs and raise `InternalError`. """ GENERIC_ERROR_CODE = -1 @@ -40,6 +43,24 @@ class Error(Exception): self.code = code self.message = message + def _payload(self) -> bytes: + """The code's fields as a value buffer (a code with fields overrides + this).""" + return b"" + + +class InternalError(RuntimeError): + """A failure of a call that declares no errors: a producer panic (-2), a + marshalling failure (-3), a callback-interface failure the producer let + through (-4), or another runtime failure (-1). These are bugs, so they + are unchecked; `code` and `message` say what went wrong. + """ + + def __init__(self, code: int, message: str) -> None: + super().__init__(f"({code}) {message}") + self.code = code + self.message = message + class _ErrorStruct(ctypes.Structure): """The C `shapes_error` out-parameter every call reports through.""" @@ -89,18 +110,33 @@ def _bind(name: str, restype: Any, *argtypes: Any) -> Any: return fn -# The C ABI revision these bindings were generated against, and the contract -# checksum of every top-level module: (module, checksum symbol, value). -_ABI_VERSION = 3 -_CHECKSUMS: List[Tuple[str, str, int]] = [ - ("shapes", "shapes_shapes_checksum", 0x06acf433ffa2a0f0), -] +class _ContractEntry(ctypes.Structure): + """The C `shapes_contract_entry`: one declaration's id and hash.""" + + _fields_ = [("id", ctypes.c_uint64), ("hash", ctypes.c_uint64)] + + +# The C ABI revision these bindings were generated against and, per +# top-level module's contract symbol, the declarations they rely on: +# (id, hash, dotted path). +_ABI_VERSION = 4 +_CONTRACTS: Dict[str, List[Tuple[int, int, str]]] = { + "shapes_shapes_contract": [ + (0x0d7b3eec49243e7a, 0x5f232311808e1b1f, "shapes.Channel"), + (0x183fa8198100860e, 0xe40c6993cb31d359, "shapes.describe"), + (0x2d1395eb837b848e, 0x42202a0c12f3a16e, "shapes.sum_bytes"), + (0x7f9a5b87ba09b86b, 0xc5b32fb0efbbaa17, "shapes.scale"), + (0xc8edd779240fda17, 0x68e1be78fddb20e4, "shapes.Numerics"), + (0xd3d3881b2fd18a62, 0xf24d0eb76a99d918, "shapes.make_numerics"), + (0xf010844afff5ac3c, 0x50de28ae9b41ff15, "shapes.Shape"), + ], +} def _check_contract() -> None: - """Refuse to load a library built for a different ABI revision or a - different version of any module's API, before any other symbol is - bound.""" + """Refuse to load a library built for another ABI revision, or one that + lacks or changed a declaration these bindings use, before any other + symbol is bound.""" try: abi_version = _bind("shapes_abi_version", ctypes.c_uint32) except AttributeError: @@ -114,33 +150,40 @@ def _check_contract() -> None: f"C ABI mismatch: these bindings expect revision {_ABI_VERSION}, " f"but {_lib._name} reports revision {found}" ) - for module, symbol, expected in _CHECKSUMS: + for symbol, expected in _CONTRACTS.items(): try: - checksum = _bind(symbol, ctypes.c_uint64) - except AttributeError: - raise ImportError( - f"module {module!r}: {_lib._name} does not export {symbol}" - ) from None - found = checksum() - if found != expected: - raise ImportError( - f"module {module!r}: the native library's contract checksum " - f"{found:#018x} does not match these bindings ({expected:#018x}); " - "regenerate the bindings from the library's current API" + contract = _bind( + symbol, ctypes.POINTER(_ContractEntry), ctypes.POINTER(ctypes.c_size_t) ) + except AttributeError: + raise ImportError(f"{_lib._name} does not export {symbol}") from None + length = ctypes.c_size_t() + table = contract(ctypes.byref(length)) + hashes = {table[i].id: table[i].hash for i in range(length.value)} if table else {} + for entry_id, entry_hash, path in expected: + actual = hashes.get(entry_id) + if actual is None: + raise ImportError(f"{path} is missing from the library ({_lib._name})") + if actual != entry_hash: + raise ImportError( + f"{path} changed since these bindings were generated ({_lib._name})" + ) _check_contract() _error_clear = _bind("shapes_error_clear", None, ctypes.POINTER(_ErrorStruct)) _free_bytes = _bind("shapes_free_bytes", None, ctypes.c_void_p, ctypes.c_size_t) +_debug_live_fn = _bind("shapes_debug_live", ctypes.c_uint64, ctypes.c_int32) def _debug_live(kind: int) -> int: - """The producer's live-allocation counter `kind` (0 objects, 1 callbacks, - 2 iterators, 3 cancel tokens, 4 returned allocations). Always 0 unless - the producer was built with its leak counters enabled.""" - return _bind("shapes_debug_live", ctypes.c_uint64, ctypes.c_int32)(kind) + """The producer's live-resource counter `kind` (0 objects, 1 callbacks, + 2 iterators, 3 cancel tokens, 4 byte runs; -1 is 1 when the producer + counts at all). Always 0 unless the producer was built with its leak + counters enabled.""" + live: int = _debug_live_fn(kind) + return live def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: @@ -153,17 +196,9 @@ def _read_error(err: _ErrorStruct) -> Tuple[int, str, bytes]: return code, message, payload -def _error_from(code: int, message: str, payload: bytes = b"") -> Error: - """The exception for a code no declared error domain claims.""" - return Error(code, message) - - -def _check_error(err: _ErrorStruct) -> None: - """Raise for a non-zero out-err slot of a call that declares no errors: - only runtime traps (a panic, a marshalling failure, a failed callback) - land here.""" - if err.code: - raise _error_from(*_read_error(err)) +def _trap_from(code: int, message: str, payload: bytes = b"") -> InternalError: + """The exception for a failed call that declares no errors.""" + return InternalError(code, message) def _take_bytes(ptr: Optional[int], length: int) -> bytes: @@ -187,6 +222,9 @@ def _peek_bytes(ptr: Optional[int], length: int) -> bytes: return ctypes.string_at(ptr, length) if ptr and length else b"" +_H = TypeVar("_H", bound="_Handle") + + class _Handle: """Owns one native reference and releases it exactly once. @@ -200,12 +238,13 @@ class _Handle: def _init_handle(self, ptr: int) -> None: self._ptr: Optional[int] = ptr + self._key = ptr self._calls = 0 self._closing = False self._lock = threading.Lock() @classmethod - def _adopt(cls, ptr: int) -> Any: + def _adopt(cls: Type[_H], ptr: int) -> _H: """Wrap one strong reference the producer handed over.""" obj = cls.__new__(cls) obj._init_handle(ptr) @@ -215,7 +254,7 @@ class _Handle: with self._lock: ptr = self._ptr if ptr is None or self._closing: - raise Error(-1, f"{type(self).__name__} used after close()") + raise ValueError(f"{type(self).__name__} is closed") self._calls += 1 return ptr @@ -229,8 +268,8 @@ class _Handle: type(self)._destroy(ptr) def close(self) -> None: - """Release this wrapper's native reference. Idempotent; the wrapper - is unusable afterwards.""" + """Release the native reference. Idempotent; the wrapper is unusable + afterwards.""" lock = getattr(self, "_lock", None) if lock is None: return @@ -244,7 +283,7 @@ class _Handle: if ptr is not None: type(self)._destroy(ptr) - def __enter__(self) -> Any: + def __enter__(self: _H) -> _H: return self def __exit__(self, *exc: object) -> None: @@ -259,27 +298,36 @@ class _Handle: class _Object(_Handle): - """Base of every interface wrapper: a reference-counted native object.""" + """Base of every interface wrapper: a reference-counted native object. + Two wrappers are equal when they hold the same native object.""" # The native clone function, set per subclass. _clone: Callable[[int], int] def _clone_ref(self) -> int: """A new strong reference to the same object, for a receiver that - adopts it (an object token inside a value buffer).""" + adopts it (an object token, or a callback method's return).""" ptr = self._acquire() try: return type(self)._clone(ptr) finally: self._release() + def __eq__(self, other: object) -> bool: + if not isinstance(other, _Object): + return NotImplemented + return self is other or (self._ptr is not None and self._ptr == other._ptr) + + def __hash__(self) -> int: + return hash(self._key) + class _Iterator(_Handle): """Base of every lazy iterator over a producer stream: one native `next` call per step; the handle is released on exhaustion, `close()`, or garbage collection.""" - def __iter__(self) -> Any: + def __iter__(self: _H) -> _H: return self def _step(self) -> int: @@ -294,23 +342,23 @@ class _Iterator(_Handle): def _required(ptr: Optional[int]) -> int: """A non-null object pointer the producer returned.""" if not ptr: - raise Error(-1, "the producer returned a null object") + raise InternalError(-3, "the producer returned a null object") return ptr -def _lend(obj: Any, cls: "Type[_Object]") -> int: +def _lend(obj: Any, cls: Type[_Object]) -> int: """Lend `obj`'s native pointer to one call; pair with `obj._release()`.""" if not isinstance(obj, cls): raise TypeError(f"expected {cls.__name__}, got {type(obj).__name__}") return obj._acquire() -def _lend_opt(obj: Any, cls: "Type[_Object]") -> Optional[int]: +def _lend_opt(obj: Any, cls: Type[_Object]) -> Optional[int]: """`_lend` for an optional object: None lends a null pointer.""" return None if obj is None else _lend(obj, cls) -def _release_opt(obj: Any) -> None: +def _release_opt(obj: Optional[_Object]) -> None: if obj is not None: obj._release() @@ -326,9 +374,12 @@ _U64 = struct.Struct(" Error: - return Error(-3, f"malformed value buffer: {what}") + +def _malformed(what: str) -> InternalError: + return InternalError(-3, f"malformed value buffer: {what}") class _Writer: @@ -411,7 +462,7 @@ class _Writer: self._buf += _U32.pack(len(values)) self._buf += struct.pack(f"<{len(values)}{code}", *values) - def write_object(self, obj: Any, cls: "Type[_Object]") -> None: + def write_object(self, obj: Any, cls: Type[_Object]) -> None: """Reserve an object token: one strong reference to `obj` the reader adopts, cloned in `finish()`. A closed wrapper is rejected now.""" if not isinstance(obj, cls): @@ -424,7 +475,8 @@ class _Writer: class _Reader: """Decodes values from the value-buffer wire format, rejecting truncated - buffers, invalid flag bytes, invalid UTF-8, and trailing data.""" + buffers, invalid flag bytes, invalid UTF-8, undeclared enum values, and + trailing data.""" __slots__ = ("_data", "_pos") @@ -497,6 +549,13 @@ class _Reader: except UnicodeDecodeError: raise _malformed("string is not valid UTF-8") from None + def read_enum(self, cls: Type[_E]) -> _E: + value = self.read_i32() + try: + return cls(value) + except ValueError: + raise _malformed(f"undeclared {cls.__name__} value {value}") from None + def read_numbers(self, code: str, size: int) -> List[Any]: """A list of one fixed-width numeric primitive, unpacked in one call.""" n = self.read_count() @@ -515,7 +574,7 @@ class _Reader: raise _malformed("trailing bytes") -def _decode(data: bytes, read: Callable[[_Reader], Any]) -> Any: +def _decode(data: bytes, read: Callable[[_Reader], _T]) -> _T: """Decode exactly one value from `data`.""" r = _Reader(data) value = read(r) @@ -541,15 +600,15 @@ class Shape: # A labeled shape with a small count Labeled = 4 - Empty: "Type[ShapeEmpty]" - Circle: "Type[ShapeCircle]" - Rectangle: "Type[ShapeRectangle]" - Bounded: "Type[ShapeBounded]" - Labeled: "Type[ShapeLabeled]" - TAG: "Shape.Tag" + Empty: Type[ShapeEmpty] + Circle: Type[ShapeCircle] + Rectangle: Type[ShapeRectangle] + Bounded: Type[ShapeBounded] + Labeled: Type[ShapeLabeled] + TAG: Shape.Tag @property - def tag(self) -> "Shape.Tag": + def tag(self) -> Shape.Tag: """The discriminant of this value's active variant.""" return type(self).TAG @@ -609,7 +668,7 @@ Shape.Bounded = ShapeBounded Shape.Labeled = ShapeLabeled -def _write_Shape(_w: _Writer, value: "Shape") -> None: +def _write_Shape(_w: _Writer, value: Shape) -> None: if isinstance(value, ShapeEmpty): _w.write_i32(0) return @@ -636,35 +695,36 @@ def _write_Shape(_w: _Writer, value: "Shape") -> None: raise TypeError(f"expected a Shape variant, got {type(value).__name__}") -def _read_Shape(_r: _Reader) -> "Shape": - _tag = _r.read_i32() - if _tag == 0: +def _read_Shape(_r: _Reader) -> Shape: + tag = _r.read_i32() + if tag == 0: return ShapeEmpty() - if _tag == 1: + if tag == 1: return ShapeCircle( radius=_r.read_f64(), ) - if _tag == 2: + if tag == 2: return ShapeRectangle( width=_r.read_f32(), height=_r.read_f32(), ) - if _tag == 3: + if tag == 3: return ShapeBounded( min=_r.read_i16(), max=_r.read_i16(), weight=_r.read_u64(), ) - if _tag == 4: + if tag == 4: return ShapeLabeled( label=_r.read_string(), count=_r.read_u8(), ) - raise _malformed(f"unknown Shape tag {_tag}") + raise _malformed(f"unknown Shape tag {tag}") class Channel(IntEnum): """A plain C-style enum (no payloads)""" + Red = 0 Green = 1 Blue = 2 @@ -686,7 +746,7 @@ class Numerics: j: float -def _write_Numerics(_w: _Writer, value: "Numerics") -> None: +def _write_Numerics(_w: _Writer, value: Numerics) -> None: _w.write_i8(value.a) _w.write_i16(value.b) _w.write_i32(value.c) @@ -699,7 +759,7 @@ def _write_Numerics(_w: _Writer, value: "Numerics") -> None: _w.write_f64(value.j) -def _read_Numerics(_r: _Reader) -> "Numerics": +def _read_Numerics(_r: _Reader) -> Numerics: return Numerics( a=_r.read_i8(), b=_r.read_i16(), @@ -717,7 +777,7 @@ def _read_Numerics(_r: _Reader) -> "Numerics": _c_shapes_describe = _bind("shapes_shapes_describe", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def describe(shape: "Shape") -> str: +def describe(shape: Shape) -> str: """Render a shape to a string""" _shape_w = _Writer() _write_Shape(_shape_w, shape) @@ -725,14 +785,15 @@ def describe(shape: "Shape") -> str: _out_len = ctypes.c_size_t() _shape_b = _shape_w.finish() _ret = _c_shapes_describe(_shape_b, len(_shape_b), ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _take_str(_ret, _out_len.value) _c_shapes_scale = _bind("shapes_shapes_scale", ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t, ctypes.c_double, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def scale(shape: "Shape", factor: float) -> "Shape": +def scale(shape: Shape, factor: float) -> Shape: """Scale a shape by a factor, returning a new shape""" _shape_w = _Writer() _write_Shape(_shape_w, shape) @@ -740,7 +801,8 @@ def scale(shape: "Shape", factor: float) -> "Shape": _out_len = ctypes.c_size_t() _shape_b = _shape_w.finish() _ret = _c_shapes_scale(_shape_b, len(_shape_b), factor, ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Shape) @@ -751,20 +813,40 @@ def sum_bytes(values: bytes) -> int: """Sum a list of bytes into a wide integer (numerics smoke)""" _values_b = bytes(values) _err = _ErrorStruct() - _ret = _c_shapes_sum_bytes(_values_b, len(_values_b), ctypes.byref(_err)) - _check_error(_err) + _ret: int = _c_shapes_sum_bytes(_values_b, len(_values_b), ctypes.byref(_err)) + if _err.code: + raise _trap_from(*_read_error(_err)) return _ret _c_shapes_make_numerics = _bind("shapes_shapes_make_numerics", ctypes.c_void_p, ctypes.c_int8, ctypes.c_int16, ctypes.c_int32, ctypes.c_int64, ctypes.c_uint8, ctypes.c_uint16, ctypes.c_uint32, ctypes.c_uint64, ctypes.c_float, ctypes.c_double, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) -def make_numerics(a: int, b: int, c: int, d: int, e: int, f: int, g: int, h: int, i: float, j: float) -> "Numerics": +def make_numerics(a: int, b: int, c: int, d: int, e: int, f: int, g: int, h: int, i: float, j: float) -> Numerics: """Build a Numerics struct from every numeric primitive""" _err = _ErrorStruct() _out_len = ctypes.c_size_t() _ret = _c_shapes_make_numerics(a, b, c, d, e, f, g, h, i, j, ctypes.byref(_out_len), ctypes.byref(_err)) - _check_error(_err) + if _err.code: + raise _trap_from(*_read_error(_err)) return _decode(_take_bytes(_ret, _out_len.value), _read_Numerics) + +__all__ = [ + "Error", + "InternalError", + "Shape", + "ShapeEmpty", + "ShapeCircle", + "ShapeRectangle", + "ShapeBounded", + "ShapeLabeled", + "Channel", + "Numerics", + "describe", + "scale", + "sum_bytes", + "make_numerics", +] + # END shapes.py diff --git a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_pyi.snap b/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_pyi.snap deleted file mode 100644 index 50f08b51..00000000 --- a/crates/weaveffi-cli/tests/snapshots/python_shapes__shapes_shapes_pyi.snap +++ /dev/null @@ -1,110 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -import builtins -from abc import ABC, abstractmethod -from enum import IntEnum -from typing import Dict, Iterator, List, Optional, Type - -class Error(Exception): - GENERIC_ERROR_CODE: int - PANIC_ERROR_CODE: int - MARSHAL_ERROR_CODE: int - FOREIGN_ERROR_CODE: int - CANCELLED_ERROR_CODE: int - code: int - message: str - def __init__(self, code: int, message: str) -> None: ... - -# An algebraic shape (sum type with associated data) -class Shape: - class Tag(IntEnum): - # The empty shape (a unit variant) - Empty = 0 - # A circle with a radius - Circle = 1 - # An axis-aligned rectangle - Rectangle = 2 - # An integer-bounded shape (exercises narrow + wide ints) - Bounded = 3 - # A labeled shape with a small count - Labeled = 4 - Empty: Type["ShapeEmpty"] - Circle: Type["ShapeCircle"] - Rectangle: Type["ShapeRectangle"] - Bounded: Type["ShapeBounded"] - Labeled: Type["ShapeLabeled"] - @property - def tag(self) -> "Shape.Tag": ... - -# The empty shape (a unit variant) -class ShapeEmpty(Shape): - TAG: "Shape.Tag" - def __init__(self) -> None: ... - -# A circle with a radius -class ShapeCircle(Shape): - TAG: "Shape.Tag" - # Radius in points - radius: float - def __init__(self, radius: float) -> None: ... - -# An axis-aligned rectangle -class ShapeRectangle(Shape): - TAG: "Shape.Tag" - width: float - height: float - def __init__(self, width: float, height: float) -> None: ... - -# An integer-bounded shape (exercises narrow + wide ints) -class ShapeBounded(Shape): - TAG: "Shape.Tag" - min: int - max: int - weight: int - def __init__(self, min: int, max: int, weight: int) -> None: ... - -# A labeled shape with a small count -class ShapeLabeled(Shape): - TAG: "Shape.Tag" - label: str - count: int - def __init__(self, label: str, count: int) -> None: ... - -# A plain C-style enum (no payloads) -class Channel(IntEnum): - Red = 0 - Green = 1 - Blue = 2 - -# One field per supported numeric primitive -class Numerics: - a: int - b: int - c: int - d: int - e: int - f: int - g: int - h: int - i: float - j: float - def __init__(self, a: int, b: int, c: int, d: int, e: int, f: int, g: int, h: int, i: float, j: float) -> None: ... - -# Render a shape to a string -def describe(shape: "Shape") -> str: ... - -# Scale a shape by a factor, returning a new shape -def scale(shape: "Shape", factor: float) -> "Shape": ... - -# Sum a list of bytes into a wide integer (numerics smoke) -def sum_bytes(values: bytes) -> int: ... - -# Build a Numerics struct from every numeric primitive -def make_numerics(a: int, b: int, c: int, d: int, e: int, f: int, g: int, h: int, i: float, j: float) -> "Numerics": ... - -# END shapes.pyi diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__README_md.snap deleted file mode 100644 index 33cb6c5b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__README_md.snap +++ /dev/null @@ -1,33 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# docs_everywhere (Ruby) - -Ruby bindings for `docs_everywhere` using the [ffi](https://github.com/ffi/ffi) gem. - -## Prerequisites - -- Ruby 2.7 or newer -- The native library (`libdocs_everywhere.so`, `libdocs_everywhere.dylib`, or `docs_everywhere.dll`) on the library - search path, or its path in the `DOCS_EVERYWHERE_LIBRARY` environment variable. - -## Install - -```bash -gem build docs_everywhere.gemspec -gem install docs_everywhere-0.1.0.gem -``` - -## Usage - -```ruby -require 'docs_everywhere' - -DocsEverywhere::ABI_VERSION -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__docs_everywhere_gemspec.snap b/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__docs_everywhere_gemspec.snap deleted file mode 100644 index efa508fe..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__docs_everywhere_gemspec.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -Gem::Specification.new do |s| - s.name = 'docs_everywhere' - s.version = '0.1.0' - s.summary = 'Native bindings for docs_everywhere' - s.authors = ['docs_everywhere'] - s.files = Dir['lib/**/*.rb'] - s.require_paths = ['lib'] - s.required_ruby_version = '>= 2.7' - - s.add_dependency 'ffi', '~> 1.15' -end - -# END docs_everywhere.gemspec diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_rb.snap index 8699cc91..0aebff09 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_rb.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_rb.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate # frozen_string_literal: true @@ -11,10 +11,26 @@ require_relative 'docs_everywhere/runtime' # Ruby bindings for docs_everywhere. module DocsEverywhere - # Contract checksums of the top-level modules these bindings were - # generated from; a library built from a different API fails to load. + # The declarations these bindings were generated with, per top-level + # module, as [id, hash, path]. Loading fails, naming the path, unless + # the library's contract table holds each one with an equal hash. _wv_check_contract!( - 'docs' => [:docs_everywhere_docs_checksum, 0x7950eec1e9cfc819], + docs_everywhere_docs_contract: [ + [0x00b2187fd75634ec, 0x2263c6f787bf6081, 'docs.publish_async'], + [0x12fc242b9dc6444c, 0x96405f484a642b31, 'docs.Notebook.insert'], + [0x2213822ed05cd5bd, 0x26c5833392aa63ce, 'docs.Notebook.new'], + [0x2bd762bcdaa207ff, 0x1b0ebab2d04c49b0, 'docs.SavedListener'], + [0x2e086c45e9d617de, 0x4d43bf32358c01ff, 'docs.Notebook.find'], + [0x4817dc98197ad960, 0xf8017b5a7d58b2c7, 'docs.Notebook.size'], + [0x614de47452d5a55c, 0xd9aae1472494ffef, 'docs.get_document'], + [0x67c5cb9fc119bcd5, 0x08d5b25a1a22a7fa, 'docs.Document'], + [0x8d484bf2d78c9b70, 0xe9093cf96912194c, 'docs.Notebook.max_documents'], + [0xaea61d6451691c67, 0xe4704128b3524b3d, 'docs.watch'], + [0xd68d6ad1540cdec5, 0x01afeb40b0a8bfe1, 'docs.Notebook'], + [0xdb0601444a4760cc, 0xe56c69629b444797, 'docs.create_document'], + [0xe12d6f4838347b3e, 0xa9bab86646d84f51, 'docs.Status'], + [0xee2f8d25b3313b50, 0x9691232f35439a2b, 'docs.DocsErrors'], + ], ) # === Module: docs === @@ -25,8 +41,8 @@ module DocsEverywhere class NotFound < DocsErrorsError CODE = 1 - def initialize(message = 'Document not found') - super(1, message) + def initialize(message = nil) + super(CODE, message || 'Document not found') end end @@ -34,31 +50,36 @@ module DocsEverywhere class InvalidStatus < DocsErrorsError CODE = 2 - def initialize(message = 'Invalid status transition') - super(2, message) + def initialize(message = nil) + super(CODE, message || 'Invalid status transition') end end end - # Maps each ABI code of the DocsErrorsError domain to its error class. - DOCS_ERRORS_ERROR_CODES = { - 1 => DocsErrorsError::NotFound, - 2 => DocsErrorsError::InvalidStatus, - }.freeze - - # Builds the DocsErrorsError subclass matching `code`, decoding any payload - # fields the code declares, or a generic Error for codes outside - # the domain (panics, marshalling, callback failures). - def self.docs_errors_error_from(code, message, payload = nil) - cls = DOCS_ERRORS_ERROR_CODES[code] - return _wv_error(code, message) if cls.nil? - message.empty? ? cls.new : cls.new(message) + # @api private + # The DocsErrorsError for a domain `code`, with its payload fields decoded; + # the root Error (or Cancelled) for any other code. + def self._wv_docs_errors_error_from(code, message, payload = nil) + message = nil if message.empty? + r = WvBufferReader.new(payload) + error = + case code + when 1 + DocsErrorsError::NotFound.new(message) + when 2 + DocsErrorsError::InvalidStatus.new(message) + else + return _wv_error(code, message.to_s) + end + r.expect_end! + error end - # Raises the typed DocsErrorsError for a non-zero error slot. - def self.check_docs_errors_error!(err) + # @api private + # Raises the DocsErrorsError for a non-zero error slot. + def self._wv_check_docs_errors_error!(err) taken = _wv_take_error(err) - raise docs_errors_error_from(*taken) if taken + raise _wv_docs_errors_error_from(*taken) unless taken.nil? end # Lifecycle status of a document. @@ -98,12 +119,11 @@ module DocsEverywhere # Structural equality over every field. def ==(other) - return false unless other.is_a?(Document) - return false unless id == other.id - return false unless title == other.title - return false unless status == other.status - return false unless tags == other.tags - true + other.is_a?(Document) && + id == other.id && + title == other.title && + status == other.status && + tags == other.tags end end @@ -113,27 +133,22 @@ module DocsEverywhere w.write_i64(v.id) w.write_string(v.title) w.write_i32(v.status) - w.write_len(v.tags.length) - v.tags.each do |_wv_e0| - w.write_string(_wv_e0) - end + _wv_write_list_string(w, v.tags) end # @api private # Unpacks a Document from the value-buffer wire format. def self._wv_read_document(r) - _wv_id = r.read_i64 - _wv_title = r.read_string - _wv_status = r.read_i32 - _wv_tags = Array.new(r.read_len) do - _wv_e0 = r.read_string - _wv_e0 - end - Document.new(id: _wv_id, title: _wv_title, status: _wv_status, tags: _wv_tags) + Document.new( + id: r.read_i64, + title: r.read_string, + status: r.read_i32, + tags: _wv_read_list_string(r), + ) end attach_function :docs_everywhere_docs_Notebook_clone, [:pointer], :pointer - attach_function :docs_everywhere_docs_Notebook_destroy, [:pointer], :void, blocking: true + attach_function :docs_everywhere_docs_Notebook_destroy, [:pointer], :void attach_function :docs_everywhere_docs_Notebook_new, [:pointer], :pointer, blocking: true attach_function :docs_everywhere_docs_Notebook_insert, [:pointer, :pointer, :size_t, :pointer], :int32, blocking: true attach_function :docs_everywhere_docs_Notebook_find, [:pointer, :int64, :pointer, :pointer], :pointer, blocking: true @@ -142,6 +157,7 @@ module DocsEverywhere attach_function :docs_everywhere_docs_create_document, [:pointer, :size_t, :pointer, :size_t, :pointer, :pointer], :pointer, blocking: true attach_function :docs_everywhere_docs_get_document, [:int64, :pointer, :pointer], :pointer, blocking: true attach_function :docs_everywhere_docs_publish_async, [:int64, :pointer, :pointer], :void, blocking: true + attach_function :docs_everywhere_docs_watch, [:pointer, :pointer, :pointer], :void, blocking: true # @api private # Completion trampoline for publish_async. @@ -150,7 +166,7 @@ module DocsEverywhere begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else value = result queue << value @@ -160,60 +176,57 @@ module DocsEverywhere end end - attach_function :docs_everywhere_docs_watch, [:pointer, :pointer, :pointer], :void, blocking: true - # Consumer-implemented observer of document-saved notifications. # # Implementations must be safe to call from any thread. # # Consumer-implemented callback interface. Any object responding to the # methods below is accepted wherever a SavedListener is expected; include - # this module to inherit NotImplementedError defaults. The producer + # this module to inherit NotImplementedError defaults. The library # may call the methods from any thread until it releases the # implementation. module SavedListener # Fires after a document has been persisted successfully - # @param doc_id [Object] Identifier of the saved document - # @param revision [Object] Monotonically increasing revision + # @param doc_id [Integer] Identifier of the saved document + # @param revision [Integer] Monotonically increasing revision def on_saved(doc_id, revision) raise NotImplementedError, "#{self.class}#on_saved is not implemented" end end # @api private - # The C vtable layout the producer calls a SavedListener through. + # The C vtable layout the library calls a SavedListener through. class WvSavedListenerVtable < FFI::Struct - layout :on_saved, :pointer, - :free, :pointer + layout :wv_size, :uint32, + :wv_flags, :uint32, + :wv_free, :pointer, + :on_saved, :pointer end # @api private - # Trampoline for SavedListener#on_saved. - WV_SAVED_LISTENER_ON_SAVED = FFI::Function.new(:void, [:pointer, :int64, :int32, :pointer]) do |ctx, doc_id, revision, out_err| - begin - doc_id_v = doc_id - revision_v = revision - impl = _wv_cb_lookup(ctx) - impl.on_saved(doc_id_v, revision_v) - nil - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - nil - end + # Releases a SavedListener implementation when the library drops its last + # reference. May run on any library thread. + WV_SAVED_LISTENER_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| + _wv_cb_free(ctx) end # @api private - # Releases a SavedListener implementation when the producer drops its last - # reference. - WV_SAVED_LISTENER_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| - _wv_cb_free(ctx) + # Trampoline for SavedListener#on_saved. + WV_SAVED_LISTENER_ON_SAVED = FFI::Function.new(:void, [:pointer, :int64, :int32, :pointer]) do |wv_ctx, doc_id, revision, wv_err| + _wv_cb_lookup(wv_ctx).on_saved(doc_id, revision) + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil end # The one process-wide SavedListener vtable every registration hands the - # producer; its entries live for the process lifetime. + # library; its entries live for the process lifetime. WV_SAVED_LISTENER_VTABLE = WvSavedListenerVtable.new + WV_SAVED_LISTENER_VTABLE[:wv_size] = WvSavedListenerVtable.size + WV_SAVED_LISTENER_VTABLE[:wv_flags] = 0 + WV_SAVED_LISTENER_VTABLE[:wv_free] = WV_SAVED_LISTENER_FREE WV_SAVED_LISTENER_VTABLE[:on_saved] = WV_SAVED_LISTENER_ON_SAVED - WV_SAVED_LISTENER_VTABLE[:free] = WV_SAVED_LISTENER_FREE # @api private # Owns one strong reference to a Notebook; releases it exactly once. @@ -238,55 +251,53 @@ module DocsEverywhere def initialize err = ErrorStruct.new result = DocsEverywhere.docs_everywhere_docs_Notebook_new(err) - DocsEverywhere.check_error!(err) - raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if result.null? - _wv_init(result) + DocsEverywhere._wv_check!(err) + _wv_init(DocsEverywhere._wv_nonnull(result)) end # Add a document, returning its position in the notebook - # @param document [Object] The document to add + # @param document [Document] The document to add + # @return [Integer] def insert(document) - document_w = WvBufferWriter.new - DocsEverywhere._wv_write_document(document_w, document) + document_s = DocsEverywhere._wv_encode { |w| DocsEverywhere._wv_write_document(w, document) } DocsEverywhere._wv_pin(self) do |_wv_self| err = ErrorStruct.new - document_s = DocsEverywhere._wv_seal(document_w).first result = DocsEverywhere.docs_everywhere_docs_Notebook_insert(_wv_self, document_s, document_s.bytesize, err) - DocsEverywhere.check_error!(err) + DocsEverywhere._wv_check!(err) result end end # Look up a document by identifier, failing when none matches - # @param id [Object] Identifier of the document to find + # @param id [Integer] Identifier of the document to find + # @return [Document] def find(id) DocsEverywhere._wv_pin(self) do |_wv_self| err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = DocsEverywhere.docs_everywhere_docs_Notebook_find(_wv_self, id, out_len, err) - DocsEverywhere.check_docs_errors_error!(err) - DocsEverywhere._wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = DocsEverywhere._wv_read_document(_wv_r) - _wv_value - end + DocsEverywhere._wv_check_docs_errors_error!(err) + DocsEverywhere._wv_decode(result, out_len.read(:size_t)) { |r| DocsEverywhere._wv_read_document(r) } end end # The number of documents currently held + # @return [Integer] def size DocsEverywhere._wv_pin(self) do |_wv_self| err = ErrorStruct.new result = DocsEverywhere.docs_everywhere_docs_Notebook_size(_wv_self, err) - DocsEverywhere.check_error!(err) + DocsEverywhere._wv_check!(err) result end end # The largest number of documents one notebook will hold + # @return [Integer] def self.max_documents err = ErrorStruct.new result = DocsEverywhere.docs_everywhere_docs_Notebook_max_documents(err) - DocsEverywhere.check_error!(err) + DocsEverywhere._wv_check!(err) result end end @@ -295,46 +306,34 @@ module DocsEverywhere # # The document starts in `Draft` status; callers must publish # it explicitly to make it visible. - # @param title [Object] Human-readable title for the new document - # @param tags [Object] Initial set of tags (may be empty) + # @param title [String] Human-readable title for the new document + # @param tags [Array] Initial set of tags (may be empty) + # @return [Document] def self.create_document(title, tags) title_s = _wv_str(title) - tags_w = WvBufferWriter.new - tags_w.write_len(tags.length) - tags.each do |_wv_e0| - tags_w.write_string(_wv_e0) - end + tags_s = _wv_encode { |w| _wv_write_list_string(w, tags) } err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - tags_s = _wv_seal(tags_w).first result = docs_everywhere_docs_create_document(title_s, title_s.bytesize, tags_s, tags_s.bytesize, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_document(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_document(r) } end # Look up a document by identifier - # @param id [Object] Identifier returned from create_document + # @param id [Integer] Identifier returned from create_document + # @return [Document, nil] def self.get_document(id) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = docs_everywhere_docs_get_document(id, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - if _wv_r.read_flag - _wv_value = _wv_read_document(_wv_r) - else - _wv_value = nil - end - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_opt_document(r) } end # Publish a document in the background - # @param id [Object] Identifier of the document to publish # Blocks until the call completes on a producer thread. + # @param id [Integer] Identifier of the document to publish + # @return [Boolean] def self.publish_async(id) ctx, queue = _wv_async_begin docs_everywhere_docs_publish_async(id, DOCS_EVERYWHERE_DOCS_PUBLISH_ASYNC_CALLBACK, ctx) @@ -342,12 +341,46 @@ module DocsEverywhere end # Register a listener for saved notifications - # @param listener [Object] Receiver of saved notifications + # @param listener [SavedListener] Receiver of saved notifications def self.watch(listener) + _wv_present!(listener, 'listener') err = ErrorStruct.new - listener_ctx = _wv_cb_register(listener) - docs_everywhere_docs_watch(listener_ctx, WV_SAVED_LISTENER_VTABLE.to_ptr, err) - check_error!(err) + listener_ctx, listener_vtable = _wv_cb_register(listener, WV_SAVED_LISTENER_VTABLE) + docs_everywhere_docs_watch(listener_ctx, listener_vtable, err) + _wv_check!(err) + nil + end + + # === Value-buffer codecs of the composite types === + + # @api private + # Packs `[string]` into the value-buffer wire format. + def self._wv_write_list_string(w, v) + w.write_len(v.length) + v.each { |e| w.write_string(e) } + end + + # @api private + # Unpacks `[string]` from the value-buffer wire format. + def self._wv_read_list_string(r) + Array.new(r.read_len) { r.read_string } + end + + # @api private + # Packs `Document?` into the value-buffer wire format. + def self._wv_write_opt_document(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + _wv_write_document(w, v) + end + end + + # @api private + # Unpacks `Document?` from the value-buffer wire format. + def self._wv_read_opt_document(r) + r.read_flag ? _wv_read_document(r) : nil end end diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_runtime_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_runtime_rb.snap deleted file mode 100644 index 59ccee30..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_docs_everywhere__lib_docs_everywhere_runtime_rb.snap +++ /dev/null @@ -1,705 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# frozen_string_literal: true - -require 'ffi' -require 'monitor' - -# The fixed runtime every generated DocsEverywhere binding builds on: the native -# library loader, the load-time ABI check, the error classes, the -# value-buffer reader and writer, cancel tokens, the handle table behind -# callback interfaces and async completions, and the object wrapper base. -module DocsEverywhere - extend FFI::Library - - # An explicit path in DOCS_EVERYWHERE_LIBRARY wins. Otherwise a library bundled - # under lib/native/ (a platform gem) is preferred over the system search - # path. - _wv_override = ENV['DOCS_EVERYWHERE_LIBRARY'] - if _wv_override && !_wv_override.empty? - ffi_lib _wv_override - else - _wv_name = - case FFI::Platform::OS - when /darwin/ then 'libdocs_everywhere.dylib' - when /mswin|mingw|windows/ then 'docs_everywhere.dll' - else 'libdocs_everywhere.so' - end - _wv_bundled = File.expand_path(File.join('..', 'native', _wv_name), __dir__) - ffi_lib(File.exist?(_wv_bundled) ? _wv_bundled : _wv_name) - end - - # The C ABI revision these bindings speak. Checked before anything else is - # attached, so a mismatched library fails at require time instead of - # misreading an error struct or a value buffer later. - ABI_VERSION = 3 - begin - attach_function :docs_everywhere_abi_version, [], :uint32 - rescue FFI::NotFoundError - raise LoadError, "the loaded docs_everywhere library does not export docs_everywhere_abi_version " \ - "(these bindings expect ABI revision #{ABI_VERSION})" - end - _wv_abi = docs_everywhere_abi_version - unless _wv_abi == ABI_VERSION - raise LoadError, "docs_everywhere ABI mismatch: these bindings expect revision #{ABI_VERSION} " \ - "but the loaded library reports revision #{_wv_abi}" - end - - # The `docs_everywhere_error` out-parameter every call reports through. - class ErrorStruct < FFI::Struct - layout :code, :int32, - :message, :pointer, - :payload_ptr, :pointer, - :payload_len, :size_t - end - - # Codes reserved by the runtime. Positive codes belong to a module's error - # domain; these negative codes are raised as a plain Error (or Cancelled) - # regardless of whether the function declares errors. - GENERIC_ERROR_CODE = -1 - PANIC_ERROR_CODE = -2 - MARSHAL_ERROR_CODE = -3 - # A callback-interface implementation raised; the message carries the - # exception text. - FOREIGN_ERROR_CODE = -4 - # The call was cancelled through its CancelToken. - CANCELLED_ERROR_CODE = -5 - - # The base class of every error the library raises. - class Error < StandardError - # The ABI error code: positive for a domain error, negative for a runtime - # failure. - attr_reader :code - - def initialize(code, message) - @code = code - super(message) - end - end - - # Raised when a cancellable call completes because its CancelToken fired. - class Cancelled < Error - def initialize(message = 'cancelled') - super(CANCELLED_ERROR_CODE, message) - end - end - - attach_function :docs_everywhere_error_set, [:pointer, :int32, :string], :void - attach_function :docs_everywhere_error_clear, [:pointer], :void - attach_function :docs_everywhere_error_free, [:pointer], :void - attach_function :docs_everywhere_free_bytes, [:pointer, :size_t], :void - attach_function :docs_everywhere_cancel_token_create, [], :pointer - attach_function :docs_everywhere_cancel_token_cancel, [:pointer], :void - attach_function :docs_everywhere_cancel_token_is_cancelled, [:pointer], :bool - attach_function :docs_everywhere_cancel_token_destroy, [:pointer], :void - - # @api private - # The error for a code outside any declared domain. - def self._wv_error(code, message) - return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE - - Error.new(code, message) - end - - # @api private - # Copies the code, message, and payload out of a reported error and clears - # it. Returns nil when the struct reports success. - def self._wv_take_error(err) - code = err[:code] - return nil if code.zero? - - msg_ptr = err[:message] - message = msg_ptr.null? ? '' : msg_ptr.read_string.force_encoding(Encoding::UTF_8) - payload_ptr = err[:payload_ptr] - payload = payload_ptr.null? ? nil : payload_ptr.read_string(err[:payload_len]) - docs_everywhere_error_clear(err.to_ptr) - [code, message, payload] - end - - # @api private - # Like _wv_take_error for the heap-boxed error an async completion - # receives; the box is released with docs_everywhere_error_free. - def self._wv_take_boxed_error(err_ptr) - return nil if err_ptr.null? - - taken = _wv_take_error(ErrorStruct.new(err_ptr)) - docs_everywhere_error_free(err_ptr) - taken - end - - # @api private - # Raises the generic error for a call that declares no error domain. - def self.check_error!(err) - taken = _wv_take_error(err) - return if taken.nil? - - raise _wv_error(taken[0], taken[1]) - end - - # @api private - # The UTF-8 bytes of a String argument, as a private binary copy the - # producer borrows for the call. - def self._wv_str(value) - value.to_s.encode(Encoding::UTF_8).b - end - - # @api private - # A private binary copy of a bytes argument. - def self._wv_bytes(value) - value.to_s.b - end - - # @api private - # Copies a borrowed (ptr, len) run without releasing it. - def self._wv_borrow_bytes(ptr, len) - ptr.null? || len.zero? ? ''.b : ptr.read_string(len) - end - - # @api private - # Copies a returned (ptr, len) allocation, then releases it. - def self._wv_take_bytes(ptr, len) - return ''.b if ptr.null? - - data = len.zero? ? ''.b : ptr.read_string(len) - docs_everywhere_free_bytes(ptr, len) - data - end - - # @api private - # Copies a returned UTF-8 string, then releases it. - def self._wv_take_string(ptr, len) - _wv_take_bytes(ptr, len).force_encoding(Encoding::UTF_8) - end - - # A cancellation signal for a cancellable async call. Pass one as the - # `cancel:` keyword, then call #cancel from any thread; the call raises - # Cancelled unless it already completed. A token may be shared by several - # calls and stays cancelled once cancelled. - class CancelToken - # @api private - # Owns the consumer's reference to the native token. - class Ptr < FFI::AutoPointer - def self.release(ptr) - DocsEverywhere.docs_everywhere_cancel_token_destroy(ptr) - end - end - - def initialize - @ptr = Ptr.new(DocsEverywhere.docs_everywhere_cancel_token_create) - end - - # Requests cancellation. Idempotent. - def cancel - DocsEverywhere.docs_everywhere_cancel_token_cancel(@ptr) unless @ptr.nil? - nil - end - - # Whether #cancel has been called. - def cancelled? - !@ptr.nil? && DocsEverywhere.docs_everywhere_cancel_token_is_cancelled(@ptr) - end - - # Releases the native token now rather than at GC time. Calls already - # launched with it keep their own reference. - def close - return if @ptr.nil? - - @ptr.free - @ptr = nil - end - - # @api private - def _wv_ptr - raise Error.new(GENERIC_ERROR_CODE, 'CancelToken used after close') if @ptr.nil? - - @ptr - end - end - - # @api private - # Appends values in the value-buffer wire format: little-endian, packed, - # no alignment. - class WvBufferWriter - def initialize - @buf = +''.b - @objects = [] - end - - # Reserves an object token. The strong reference it carries is minted - # by #seal, just before the buffer is handed to the producer, so an - # encoding abandoned on the way never strands a reference. - def write_object(object) - raise TypeError, "expected a wrapper object, got #{object.class}" unless object.is_a?(WvObject) - - @objects << [@buf.bytesize, object] - write_u64(0) - end - - # Mints the reference each reserved object token carries, recording it - # in `minted`, and returns the finished encoding. - def seal(minted) - @objects.each do |offset, object| - ptr = object._wv_clone_ptr - minted << [object.class, ptr] - @buf[offset, 8] = [ptr.address].pack('Q<') - end - @buf - end - - # Every append is a binary String, so the buffer stays binary and byte - # offsets index it directly. - def write_bool(value) - @buf << [value ? 1 : 0].pack('C') - end - - def write_flag(value) - write_bool(value) - end - - def write_i8(value) - @buf << [value].pack('c') - end - - def write_u8(value) - @buf << [value].pack('C') - end - - def write_i16(value) - @buf << [value].pack('s<') - end - - def write_u16(value) - @buf << [value].pack('S<') - end - - def write_i32(value) - @buf << [value].pack('l<') - end - - def write_u32(value) - @buf << [value].pack('L<') - end - - def write_len(value) - write_u32(value) - end - - def write_i64(value) - @buf << [value].pack('q<') - end - - def write_u64(value) - @buf << [value].pack('Q<') - end - - def write_f32(value) - @buf << [value].pack('e') - end - - def write_f64(value) - @buf << [value].pack('E') - end - - def write_string(value) - write_bytes(value.to_s.encode(Encoding::UTF_8)) - end - - def write_bytes(value) - bytes = value.to_s.b - write_u32(bytes.bytesize) - @buf << bytes - end - end - - # @api private - # Reads values in the value-buffer wire format, raising Error on any - # malformed buffer. - class WvBufferReader - def initialize(data) - @data = data.to_s.b - @pos = 0 - end - - def take(count, what) - if @pos + count > @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, "malformed value buffer: truncated #{what}") - end - - bytes = @data.byteslice(@pos, count) - @pos += count - bytes - end - - def read_bool - byte = take(1, 'bool').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: bool byte out of range') if byte > 1 - - byte == 1 - end - - def read_flag - byte = take(1, 'option flag').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: option flag out of range') if byte > 1 - - byte == 1 - end - - def read_i8 - take(1, 'i8').unpack1('c') - end - - def read_u8 - take(1, 'u8').unpack1('C') - end - - def read_i16 - take(2, 'i16').unpack1('s<') - end - - def read_u16 - take(2, 'u16').unpack1('S<') - end - - def read_i32 - take(4, 'i32').unpack1('l<') - end - - def read_u32 - take(4, 'u32').unpack1('L<') - end - - # A u32 element count. Counts are not checked against the remaining - # bytes, since an element can encode to zero bytes; a short buffer still - # fails on the first truncated read. - def read_len - read_u32 - end - - def read_i64 - take(8, 'i64').unpack1('q<') - end - - def read_u64 - take(8, 'u64').unpack1('Q<') - end - - def read_f32 - take(4, 'f32').unpack1('e') - end - - def read_f64 - take(8, 'f64').unpack1('E') - end - - def read_string - text = read_bytes.force_encoding(Encoding::UTF_8) - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: string is not valid UTF-8') unless text.valid_encoding? - - text - end - - def read_bytes - take(read_u32, 'byte run') - end - - # An object token: a non-zero u64 carrying one strong reference the - # caller adopts into a wrapper. - def read_object_token - addr = read_u64 - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: null object token') if addr.zero? - - FFI::Pointer.new(addr) - end - - def expect_end! - return if @pos == @data.bytesize - - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: trailing bytes after value') - end - end - - # @api private - # Seals the writers of one call's object-carrying buffers (see - # WvBufferWriter#seal) and returns their encodings. If any reference can't - # be minted (a closed wrapper), the ones already minted are released before - # the error propagates. The producer adopts every minted reference once the - # call is made. - def self._wv_seal(*writers) - minted = [] - writers.map { |writer| writer.seal(minted) } - rescue Exception # rubocop:disable Lint/RescueException - minted.each { |cls, ptr| cls::WV_PTR.release(ptr) } - raise - end - - # @api private - # Decodes a returned value buffer: copies and releases it, yields a reader - # to the block, and checks that the block consumed every byte. - def self._wv_decode(ptr, len) - reader = WvBufferReader.new(_wv_take_bytes(ptr, len)) - value = yield reader - reader.expect_end! - value - end - - # Ruby objects the producer refers to by an integer key: callback-interface - # implementations (the key is the vtable `ctx`) and pending async - # completions (the key is the completion `context`). The producer never - # sees a Ruby object address. A Monitor rather than a Mutex, so a - # finalizer that runs while this thread holds the lock can't deadlock it. - @wv_handles = {} - @wv_next_handle = 0 - @wv_handles_lock = Monitor.new - - # @api private - def self._wv_handle_put(value) - @wv_handles_lock.synchronize do - @wv_next_handle += 1 - @wv_handles[@wv_next_handle] = value - FFI::Pointer.new(@wv_next_handle) - end - end - - # @api private - def self._wv_handle_get(key) - @wv_handles_lock.synchronize { @wv_handles[key.address] } - end - - # @api private - def self._wv_handle_take(key) - @wv_handles_lock.synchronize { @wv_handles.delete(key.address) } - end - - # @api private - # Registers a callback-interface implementation and returns its `ctx`. - # Never raises, so nothing can strand the registration between this and - # the call that hands it to the producer (an unusable implementation fails - # when the producer calls it, with FOREIGN_ERROR_CODE). - def self._wv_cb_register(impl) - _wv_handle_put([:callback, impl]) - end - - # @api private - # Resolves a trampoline's `ctx` back to the registered implementation. - def self._wv_cb_lookup(ctx) - kind, impl = _wv_handle_get(ctx) - raise Error.new(FOREIGN_ERROR_CODE, 'callback context is not registered') unless kind == :callback - - impl - end - - # @api private - # Drops an implementation when the producer releases its last reference. - def self._wv_cb_free(ctx) - _wv_handle_take(ctx) - nil - end - - # @api private - # Reports a callback implementation's exception through `out_err`. The - # producer copies the message. - def self._wv_cb_fail(out_err, exception) - message = exception.message.to_s - message = exception.class.name if message.empty? - message = message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") - docs_everywhere_error_set(out_err, FOREIGN_ERROR_CODE, message) - nil - end - - # @api private - # Registers a pending async call; returns its completion context and the - # queue its completion is delivered on. - def self._wv_async_begin - queue = Queue.new - [_wv_handle_put([:async, queue]), queue] - end - - # @api private - # The queue of the pending call a completion belongs to. Each context - # completes exactly once, so its entry is removed here. - def self._wv_async_finish(ctx) - kind, queue = _wv_handle_take(ctx) - kind == :async ? queue : Queue.new - end - - # @api private - # Blocks until a pending call completes, raising its error. If the wait is - # interrupted (Thread#raise, Thread#kill, Timeout), the call's token is - # cancelled before the interruption propagates. - def self._wv_async_wait(queue, token = nil) - value = - begin - queue.pop - rescue Exception # rubocop:disable Lint/RescueException - token&.cancel - raise - end - raise value if value.is_a?(Exception) - - value - end - - # @api private - # The base of every interface wrapper. A wrapper owns one strong reference - # to its native object and releases it exactly once, from #close or (as a - # backstop) from the GC finalizer of its FFI::AutoPointer. Calls borrow the - # pointer through #_wv_call, so a #close that races an in-flight call on - # another thread only releases the reference once the call returns. - class WvObject - # @api private - # Adopts one strong reference the producer handed over. - def self._from_ptr(ptr) - obj = allocate - obj._wv_init(ptr) - obj - end - - # @api private - def _wv_init(ptr) - @wv_handle = self.class::WV_PTR.new(ptr) - @wv_lock = Mutex.new - @wv_calls = 0 - @wv_closing = false - end - - # The native object pointer (borrowed; valid until #close). - def handle - handle = @wv_handle - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if handle.nil? || @wv_closing - - handle - end - - # Whether #close has released this wrapper's reference. - def closed? - @wv_handle.nil? || @wv_closing - end - - # Releases this wrapper's reference now rather than at GC time. - # Idempotent. The object itself lives until its last reference anywhere - # (another wrapper, a record field, the producer) goes. - def close - handle = @wv_lock.synchronize do - next nil if @wv_handle.nil? || @wv_closing - - if @wv_calls.zero? - taken = @wv_handle - @wv_handle = nil - taken - else - @wv_closing = true - nil - end - end - handle&.free - nil - end - - # @api private - # Yields the native pointer, keeping the reference alive until the block - # returns. - def _wv_call - handle = @wv_lock.synchronize do - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if @wv_handle.nil? || @wv_closing - - @wv_calls += 1 - @wv_handle - end - begin - yield handle - ensure - _wv_leave - end - end - - # @api private - # Mints a new strong reference the caller owns (an object written into a - # value buffer carries its own reference). - def _wv_clone_ptr - _wv_call { |ptr| self.class._wv_clone(ptr) } - end - - # dup and clone produce an independent wrapper with its own reference to - # the same object. - def initialize_copy(other) - super - _wv_init(other._wv_clone_ptr) - end - - private - - def _wv_leave - release = @wv_lock.synchronize do - @wv_calls -= 1 - next nil unless @wv_closing && @wv_calls.zero? - - taken = @wv_handle - @wv_handle = nil - @wv_closing = false - taken - end - release&.free - end - end - - # @api private - # Yields the native pointers of `objects` (NULL for nil), each kept alive - # until the block returns. - def self._wv_pin(*objects, &block) - _wv_pin_from(objects, 0, [], &block) - end - - # @api private - def self._wv_pin_from(objects, index, pointers, &block) - return yield(*pointers) if index == objects.length - - object = objects[index] - if object.nil? - _wv_pin_from(objects, index + 1, pointers + [FFI::Pointer::NULL], &block) - else - object._wv_call { |ptr| _wv_pin_from(objects, index + 1, pointers + [ptr], &block) } - end - end - - # @api private - # Checks each top-level module's contract checksum (`module name => - # [checksum function, expected value]`) against the library, so bindings - # generated from a different API fail at require time. - def self._wv_check_contract!(checksums) - checksums.each do |mod, (symbol, expected)| - begin - attach_function symbol, [], :uint64 - rescue FFI::NotFoundError - raise LoadError, "the loaded docs_everywhere library has no module '#{mod}' (#{symbol} is missing)" - end - actual = public_send(symbol) - next if actual == expected - - raise LoadError, "docs_everywhere module '#{mod}' does not match these bindings: expected contract " \ - "checksum 0x#{expected.to_s(16)}, the library reports 0x#{actual.to_s(16)}; " \ - 'regenerate the bindings' - end - end - - # @api private - # Live native resources of one kind (0 objects, 1 callbacks, 2 iterators, - # 3 cancel tokens, 4 byte allocations). Only libraries built with the - # `leak-check` feature export the counter. - def self._wv_debug_live(kind) - unless respond_to?(:docs_everywhere_debug_live) - begin - attach_function :docs_everywhere_debug_live, [:int32], :uint64 - rescue FFI::NotFoundError - raise Error.new(GENERIC_ERROR_CODE, 'docs_everywhere was built without the leak-check feature') - end - end - docs_everywhere_debug_live(kind) - end -end - -# END runtime.rb diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__README_md.snap deleted file mode 100644 index 4f9ecfaa..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__README_md.snap +++ /dev/null @@ -1,33 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# edge_cases (Ruby) - -Ruby bindings for `edge_cases` using the [ffi](https://github.com/ffi/ffi) gem. - -## Prerequisites - -- Ruby 2.7 or newer -- The native library (`libedge_cases.so`, `libedge_cases.dylib`, or `edge_cases.dll`) on the library - search path, or its path in the `EDGE_CASES_LIBRARY` environment variable. - -## Install - -```bash -gem build edge_cases.gemspec -gem install edge_cases-0.1.0.gem -``` - -## Usage - -```ruby -require 'edge_cases' - -EdgeCases::ABI_VERSION -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__edge_cases_gemspec.snap b/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__edge_cases_gemspec.snap deleted file mode 100644 index 99d3cbc1..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__edge_cases_gemspec.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -Gem::Specification.new do |s| - s.name = 'edge_cases' - s.version = '0.1.0' - s.summary = 'Native bindings for edge_cases' - s.authors = ['edge_cases'] - s.files = Dir['lib/**/*.rb'] - s.require_paths = ['lib'] - s.required_ruby_version = '>= 2.7' - - s.add_dependency 'ffi', '~> 1.15' -end - -# END edge_cases.gemspec diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_rb.snap index 18d0f8fa..150ad23d 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_rb.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_rb.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate # frozen_string_literal: true @@ -11,10 +11,59 @@ require_relative 'edge_cases/runtime' # Ruby bindings for edge_cases. module EdgeCases - # Contract checksums of the top-level modules these bindings were - # generated from; a library built from a different API fails to load. + # The declarations these bindings were generated with, per top-level + # module, as [id, hash, path]. Loading fails, naming the path, unless + # the library's contract table holds each one with an equal hash. _wv_check_contract!( - 'edge_cases' => [:edge_cases_edge_cases_checksum, 0x1567f441073bd4aa], + edge_cases_edge_cases_contract: [ + [0x056a86511785ba2b, 0x5cba3f16d57c6cdb, 'edge_cases.make_widget'], + [0x07bfa9ecb2b720d2, 0x3813b3f57c64bf35, 'edge_cases.EdgeErrors'], + [0x0aaa11867e37f991, 0xf090dd52146ebdf0, 'edge_cases.Widget.refresh'], + [0x0f9701493898ae4c, 0x6944f47a275c96e8, 'edge_cases.Inner'], + [0x1226b1adac179f7d, 0x0e95a00ab0bef637, 'edge_cases.async_keywords'], + [0x16d36bef1541136c, 0x5ea7021ec324298f, 'edge_cases.find_widget'], + [0x1b7324c55e980feb, 0xabf87fe372358f42, 'edge_cases.Widget.old_name'], + [0x1cbd731467c2bab8, 0x4915b9989c2c9074, 'edge_cases.Vault'], + [0x2c2a6de67d894b0a, 0x50a4496b1e7191ca, 'edge_cases.Vault.open'], + [0x322b4cfcd2d6861c, 0xca7527a379ae8465, 'edge_cases.Keywords'], + [0x3f8e23a5f32906f4, 0x8d4c908d796f130d, 'edge_cases.Leaf'], + [0x42ab8a685457b97c, 0xe5ce30500cadb32d, 'edge_cases.Vault.handle'], + [0x4a68540e2836d3d0, 0xa20e44c71e420ed2, 'edge_cases.mode_stream'], + [0x4e030580ac0b111d, 0xb68b7a18d910e175, 'edge_cases.legacy_value'], + [0x4f5e4e5291864ef3, 0x55a1a88b21a08f8b, 'edge_cases.Legacy'], + [0x4f84b06171b4213d, 0xbaedcfdebb6a2820, 'edge_cases.scalar_stream'], + [0x5067a5ff2d9ef910, 0xbfc79b18fc4714fc, 'edge_cases.Widget'], + [0x59aff159112fbf5b, 0x18e4def57a8cfa49, 'edge_cases.optional_params'], + [0x5daf65b98fa77c48, 0xc817a8cb202e59c2, 'edge_cases.keywords_roundtrip'], + [0x5de8a5c9726c2c5a, 0x7c50b5b65f7a896d, 'edge_cases.register_batch'], + [0x618a620309994c3c, 0x826ad0b1d208d723, 'edge_cases.async_bytes'], + [0x648285cdba0a7b8e, 0xcf6e80e783a4d8be, 'edge_cases.WidgetHolder'], + [0x649c78079824b10d, 0x5b12dd26e0634b69, 'edge_cases.async_widget'], + [0x7da7a7357d122a37, 0xf544e025ffaa3d20, 'edge_cases.async_void'], + [0x7e4015bc3a472e2d, 0xfe22291d90b82e4a, 'edge_cases.widget_stream'], + [0x805276e5d81fdba7, 0xd9b64e360bd03b2b, 'edge_cases.Outer'], + [0x80e1183326542df4, 0xe8a57e12cc6ed276, 'edge_cases.select'], + [0x923ce5e9bcd769a1, 0x0b973f5dc07dda8b, 'edge_cases.Vault.to_string'], + [0x98ca8c01f5239404, 0xfe11c93eb6b303ef, 'edge_cases.Widget.rename'], + [0xa13e22e98ea278d4, 0xf1fae215d7387800, 'edge_cases.widgets_everywhere'], + [0xa84a8c9bf2a881e1, 0xbe1dc840bf221996, 'edge_cases.async_optional'], + [0xad958c478fbf205d, 0xe386b0716d24f19b, 'edge_cases.Retired'], + [0xb2fc3964ef56ab39, 0x73c325c8bc351236, 'edge_cases.async_list'], + [0xb3403af521cf0006, 0x6abc30d48206a3ba, 'edge_cases.Widget.close'], + [0xb794304a22a406b8, 0xf6fb98711c516105, 'edge_cases.async_struct'], + [0xba3936106a49c47d, 0xb7025f2fadd6288e, 'edge_cases.Vault.peek'], + [0xc155b0c3cfb164f4, 0xc777ae57d92ebc47, 'edge_cases.noop'], + [0xcdc18607777019dd, 0xb81a31e5c20aa7a8, 'edge_cases.inspect_widget'], + [0xd164a82aeb64b8cc, 0xded02c68005c4737, 'edge_cases.nested_optional'], + [0xd978ccad2468038f, 0x30c39c65d6235153, 'edge_cases.Mode'], + [0xdbe9d947431c9124, 0x39537fb848fdf980, 'edge_cases.BatchListener'], + [0xeae8b7862c08bb07, 0xfe12eeaa830452ea, 'edge_cases.Keyword'], + [0xf1e155a05ae262b8, 0xb575eb818a423d42, 'edge_cases.async_enum'], + [0xf22e210bdbce7343, 0x2a5e0f801bc8c768, 'edge_cases.Widget.dispose'], + [0xf3c7cf675563ee6c, 0x0ecffeef399facc8, 'edge_cases.Vault.probe'], + [0xfd34bc67acac379e, 0x576caa14f3c0901a, 'edge_cases.Widget.fetch'], + [0xff2e9d2bff941786, 0x7e96764ef1f5fbaa, 'edge_cases.string_stream'], + ], ) # === Module: edge_cases === @@ -25,33 +74,38 @@ module EdgeCases class Busy < EdgeErrorsError CODE = 7 - def initialize(message = 'Resource is busy') - super(7, message) + def initialize(message = nil) + super(CODE, message || 'Resource is busy') end end end - # Maps each ABI code of the EdgeErrorsError domain to its error class. - EDGE_ERRORS_ERROR_CODES = { - 7 => EdgeErrorsError::Busy, - }.freeze - - # Builds the EdgeErrorsError subclass matching `code`, decoding any payload - # fields the code declares, or a generic Error for codes outside - # the domain (panics, marshalling, callback failures). - def self.edge_errors_error_from(code, message, payload = nil) - cls = EDGE_ERRORS_ERROR_CODES[code] - return _wv_error(code, message) if cls.nil? - message.empty? ? cls.new : cls.new(message) + # @api private + # The EdgeErrorsError for a domain `code`, with its payload fields decoded; + # the root Error (or Cancelled) for any other code. + def self._wv_edge_errors_error_from(code, message, payload = nil) + message = nil if message.empty? + r = WvBufferReader.new(payload) + error = + case code + when 7 + EdgeErrorsError::Busy.new(message) + else + return _wv_error(code, message.to_s) + end + r.expect_end! + error end - # Raises the typed EdgeErrorsError for a non-zero error slot. - def self.check_edge_errors_error!(err) + # @api private + # Raises the EdgeErrorsError for a non-zero error slot. + def self._wv_check_edge_errors_error!(err) taken = _wv_take_error(err) - raise edge_errors_error_from(*taken) if taken + raise _wv_edge_errors_error_from(*taken) unless taken.nil? end # Deprecated enum with non-contiguous values and keyword-like variants + # @deprecated Use Mode instead module Legacy NONE = 0 DEFAULT = 5 @@ -72,9 +126,8 @@ module EdgeCases # Structural equality over every field. def ==(other) - return false unless other.is_a?(Leaf) - return false unless weight == other.weight - true + other.is_a?(Leaf) && + weight == other.weight end end @@ -91,10 +144,9 @@ module EdgeCases # Structural equality over every field. def ==(other) - return false unless other.is_a?(Inner) - return false unless leaf == other.leaf - return false unless leaves == other.leaves - true + other.is_a?(Inner) && + leaf == other.leaf && + leaves == other.leaves end end @@ -121,7 +173,19 @@ module EdgeCases attr_reader :v2 - def initialize(inner:, maybe_inner:, by_name:, modes:, matrix:, sparse:, blob:, note:, by_id:, _hidden:, v2:) + def initialize( + inner:, + maybe_inner:, + by_name:, + modes:, + matrix:, + sparse:, + blob:, + note:, + by_id:, + _hidden:, + v2: + ) @inner = inner @maybe_inner = maybe_inner @by_name = by_name @@ -137,19 +201,18 @@ module EdgeCases # Structural equality over every field. def ==(other) - return false unless other.is_a?(Outer) - return false unless inner == other.inner - return false unless maybe_inner == other.maybe_inner - return false unless by_name == other.by_name - return false unless modes == other.modes - return false unless matrix == other.matrix - return false unless sparse == other.sparse - return false unless blob == other.blob - return false unless note == other.note - return false unless by_id == other.by_id - return false unless _hidden == other._hidden - return false unless v2 == other.v2 - true + other.is_a?(Outer) && + inner == other.inner && + maybe_inner == other.maybe_inner && + by_name == other.by_name && + modes == other.modes && + matrix == other.matrix && + sparse == other.sparse && + blob == other.blob && + note == other.note && + by_id == other.by_id && + _hidden == other._hidden && + v2 == other.v2 end end @@ -273,7 +336,67 @@ module EdgeCases attr_reader :super_ - def initialize(class_:, def_:, import:, from:, func:, range:, register:, volatile:, new:, default:, end_:, in_:, is:, int:, var:, val:, self_:, this:, object:, string:, case_:, select:, switch:, void:, union:, namespace:, operator:, template:, package:, protocol:, guard:, defer:, inout:, internal:, when_:, fun:, data:, late:, part:, out:, ref:, event:, base:, lambda:, yield_:, nil_:, null:, begin_:, rescue_:, module_:, chan:, go:, let:, export:, delete:, typedef:, sizeof:, static:, super_:) + def initialize( + class_:, + def_:, + import:, + from:, + func:, + range:, + register:, + volatile:, + new:, + default:, + end_:, + in_:, + is:, + int:, + var:, + val:, + self_:, + this:, + object:, + string:, + case_:, + select:, + switch:, + void:, + union:, + namespace:, + operator:, + template:, + package:, + protocol:, + guard:, + defer:, + inout:, + internal:, + when_:, + fun:, + data:, + late:, + part:, + out:, + ref:, + event:, + base:, + lambda:, + yield_:, + nil_:, + null:, + begin_:, + rescue_:, + module_:, + chan:, + go:, + let:, + export:, + delete:, + typedef:, + sizeof:, + static:, + super_: + ) @class_ = class_ @def_ = def_ @import = import @@ -337,71 +460,71 @@ module EdgeCases # Structural equality over every field. def ==(other) - return false unless other.is_a?(Keywords) - return false unless class_ == other.class_ - return false unless def_ == other.def_ - return false unless import == other.import - return false unless from == other.from - return false unless func == other.func - return false unless range == other.range - return false unless register == other.register - return false unless volatile == other.volatile - return false unless new == other.new - return false unless default == other.default - return false unless end_ == other.end_ - return false unless in_ == other.in_ - return false unless is == other.is - return false unless int == other.int - return false unless var == other.var - return false unless val == other.val - return false unless self_ == other.self_ - return false unless this == other.this - return false unless object == other.object - return false unless string == other.string - return false unless case_ == other.case_ - return false unless select == other.select - return false unless switch == other.switch - return false unless void == other.void - return false unless union == other.union - return false unless namespace == other.namespace - return false unless operator == other.operator - return false unless template == other.template - return false unless package == other.package - return false unless protocol == other.protocol - return false unless guard == other.guard - return false unless defer == other.defer - return false unless inout == other.inout - return false unless internal == other.internal - return false unless when_ == other.when_ - return false unless fun == other.fun - return false unless data == other.data - return false unless late == other.late - return false unless part == other.part - return false unless out == other.out - return false unless ref == other.ref - return false unless event == other.event - return false unless base == other.base - return false unless lambda == other.lambda - return false unless yield_ == other.yield_ - return false unless nil_ == other.nil_ - return false unless null == other.null - return false unless begin_ == other.begin_ - return false unless rescue_ == other.rescue_ - return false unless module_ == other.module_ - return false unless chan == other.chan - return false unless go == other.go - return false unless let == other.let - return false unless export == other.export - return false unless delete == other.delete - return false unless typedef == other.typedef - return false unless sizeof == other.sizeof - return false unless static == other.static - return false unless super_ == other.super_ - true + other.is_a?(Keywords) && + class_ == other.class_ && + def_ == other.def_ && + import == other.import && + from == other.from && + func == other.func && + range == other.range && + register == other.register && + volatile == other.volatile && + new == other.new && + default == other.default && + end_ == other.end_ && + in_ == other.in_ && + is == other.is && + int == other.int && + var == other.var && + val == other.val && + self_ == other.self_ && + this == other.this && + object == other.object && + string == other.string && + case_ == other.case_ && + select == other.select && + switch == other.switch && + void == other.void && + union == other.union && + namespace == other.namespace && + operator == other.operator && + template == other.template && + package == other.package && + protocol == other.protocol && + guard == other.guard && + defer == other.defer && + inout == other.inout && + internal == other.internal && + when_ == other.when_ && + fun == other.fun && + data == other.data && + late == other.late && + part == other.part && + out == other.out && + ref == other.ref && + event == other.event && + base == other.base && + lambda == other.lambda && + yield_ == other.yield_ && + nil_ == other.nil_ && + null == other.null && + begin_ == other.begin_ && + rescue_ == other.rescue_ && + module_ == other.module_ && + chan == other.chan && + go == other.go && + let == other.let && + export == other.export && + delete == other.delete && + typedef == other.typedef && + sizeof == other.sizeof && + static == other.static && + super_ == other.super_ end end # Deprecated struct + # @deprecated Use Outer instead class Retired attr_reader :id @@ -411,9 +534,8 @@ module EdgeCases # Structural equality over every field. def ==(other) - return false unless other.is_a?(Retired) - return false unless id == other.id - true + other.is_a?(Retired) && + id == other.id end end @@ -436,12 +558,11 @@ module EdgeCases # Structural equality over every field. def ==(other) - return false unless other.is_a?(WidgetHolder) - return false unless primary == other.primary - return false unless fallback == other.fallback - return false unless extras == other.extras - return false unless by_name == other.by_name - true + other.is_a?(WidgetHolder) && + primary == other.primary && + fallback == other.fallback && + extras == other.extras && + by_name == other.by_name end end @@ -454,140 +575,54 @@ module EdgeCases # @api private # Unpacks a Leaf from the value-buffer wire format. def self._wv_read_leaf(r) - _wv_weight = r.read_f32 - Leaf.new(weight: _wv_weight) + Leaf.new(weight: r.read_f32) end # @api private - # Packs a Inner into the value-buffer wire format. + # Packs an Inner into the value-buffer wire format. def self._wv_write_inner(w, v) _wv_write_leaf(w, v.leaf) - w.write_len(v.leaves.length) - v.leaves.each do |_wv_e0| - _wv_write_leaf(w, _wv_e0) - end + _wv_write_list_leaf(w, v.leaves) end # @api private - # Unpacks a Inner from the value-buffer wire format. + # Unpacks an Inner from the value-buffer wire format. def self._wv_read_inner(r) - _wv_leaf = _wv_read_leaf(r) - _wv_leaves = Array.new(r.read_len) do - _wv_e0 = _wv_read_leaf(r) - _wv_e0 - end - Inner.new(leaf: _wv_leaf, leaves: _wv_leaves) + Inner.new(leaf: _wv_read_leaf(r), leaves: _wv_read_list_leaf(r)) end # @api private - # Packs a Outer into the value-buffer wire format. + # Packs an Outer into the value-buffer wire format. def self._wv_write_outer(w, v) _wv_write_inner(w, v.inner) - if v.maybe_inner.nil? - w.write_flag(false) - else - w.write_flag(true) - _wv_write_inner(w, v.maybe_inner) - end - w.write_len(v.by_name.length) - v.by_name.each do |_wv_k0, _wv_v0| - w.write_string(_wv_k0) - _wv_write_inner(w, _wv_v0) - end - w.write_len(v.modes.length) - v.modes.each do |_wv_e0| - w.write_i32(_wv_e0) - end - w.write_len(v.matrix.length) - v.matrix.each do |_wv_e0| - w.write_len(_wv_e0.length) - _wv_e0.each do |_wv_e1| - w.write_i32(_wv_e1) - end - end - w.write_len(v.sparse.length) - v.sparse.each do |_wv_e0| - if _wv_e0.nil? - w.write_flag(false) - else - w.write_flag(true) - w.write_i64(_wv_e0) - end - end - if v.blob.nil? - w.write_flag(false) - else - w.write_flag(true) - w.write_bytes(v.blob) - end - if v.note.nil? - w.write_flag(false) - else - w.write_flag(true) - w.write_string(v.note) - end - w.write_len(v.by_id.length) - v.by_id.each do |_wv_k0, _wv_v0| - w.write_i64(_wv_k0) - w.write_string(_wv_v0) - end + _wv_write_opt_inner(w, v.maybe_inner) + _wv_write_map_string_inner(w, v.by_name) + _wv_write_list_mode(w, v.modes) + _wv_write_list_list_i32(w, v.matrix) + _wv_write_list_opt_i64(w, v.sparse) + _wv_write_opt_bytes(w, v.blob) + _wv_write_opt_string(w, v.note) + _wv_write_map_i64_string(w, v.by_id) w.write_bool(v._hidden) w.write_u8(v.v2) end # @api private - # Unpacks a Outer from the value-buffer wire format. + # Unpacks an Outer from the value-buffer wire format. def self._wv_read_outer(r) - _wv_inner = _wv_read_inner(r) - if r.read_flag - _wv_maybe_inner = _wv_read_inner(r) - else - _wv_maybe_inner = nil - end - _wv_by_name = {} - r.read_len.times do - _wv_k0 = r.read_string - _wv_v0 = _wv_read_inner(r) - _wv_by_name[_wv_k0] = _wv_v0 - end - _wv_modes = Array.new(r.read_len) do - _wv_e0 = r.read_i32 - _wv_e0 - end - _wv_matrix = Array.new(r.read_len) do - _wv_e0 = Array.new(r.read_len) do - _wv_e1 = r.read_i32 - _wv_e1 - end - _wv_e0 - end - _wv_sparse = Array.new(r.read_len) do - if r.read_flag - _wv_e0 = r.read_i64 - else - _wv_e0 = nil - end - _wv_e0 - end - if r.read_flag - _wv_blob = r.read_bytes - else - _wv_blob = nil - end - if r.read_flag - _wv_note = r.read_string - else - _wv_note = nil - end - _wv_by_id = {} - r.read_len.times do - _wv_k0 = r.read_i64 - _wv_v0 = r.read_string - _wv_by_id[_wv_k0] = _wv_v0 - end - _wv__hidden = r.read_bool - _wv_v2 = r.read_u8 - Outer.new(inner: _wv_inner, maybe_inner: _wv_maybe_inner, by_name: _wv_by_name, modes: _wv_modes, matrix: _wv_matrix, sparse: _wv_sparse, blob: _wv_blob, note: _wv_note, by_id: _wv_by_id, _hidden: _wv__hidden, v2: _wv_v2) + Outer.new( + inner: _wv_read_inner(r), + maybe_inner: _wv_read_opt_inner(r), + by_name: _wv_read_map_string_inner(r), + modes: _wv_read_list_mode(r), + matrix: _wv_read_list_list_i32(r), + sparse: _wv_read_list_opt_i64(r), + blob: _wv_read_opt_bytes(r), + note: _wv_read_opt_string(r), + by_id: _wv_read_map_i64_string(r), + _hidden: r.read_bool, + v2: r.read_u8, + ) end # @api private @@ -657,66 +692,67 @@ module EdgeCases # @api private # Unpacks a Keywords from the value-buffer wire format. def self._wv_read_keywords(r) - _wv_class_ = r.read_i32 - _wv_def_ = r.read_i32 - _wv_import = r.read_i32 - _wv_from = r.read_i32 - _wv_func = r.read_i32 - _wv_range = r.read_i32 - _wv_register = r.read_i32 - _wv_volatile = r.read_i32 - _wv_new = r.read_i32 - _wv_default = r.read_i32 - _wv_end_ = r.read_i32 - _wv_in_ = r.read_i32 - _wv_is = r.read_i32 - _wv_int = r.read_i32 - _wv_var = r.read_i32 - _wv_val = r.read_i32 - _wv_self_ = r.read_i32 - _wv_this = r.read_i32 - _wv_object = r.read_i32 - _wv_string = r.read_i32 - _wv_case_ = r.read_i32 - _wv_select = r.read_i32 - _wv_switch = r.read_i32 - _wv_void = r.read_i32 - _wv_union = r.read_i32 - _wv_namespace = r.read_i32 - _wv_operator = r.read_i32 - _wv_template = r.read_i32 - _wv_package = r.read_i32 - _wv_protocol = r.read_i32 - _wv_guard = r.read_i32 - _wv_defer = r.read_i32 - _wv_inout = r.read_i32 - _wv_internal = r.read_i32 - _wv_when_ = r.read_i32 - _wv_fun = r.read_i32 - _wv_data = r.read_i32 - _wv_late = r.read_i32 - _wv_part = r.read_i32 - _wv_out = r.read_i32 - _wv_ref = r.read_i32 - _wv_event = r.read_i32 - _wv_base = r.read_i32 - _wv_lambda = r.read_i32 - _wv_yield_ = r.read_i32 - _wv_nil_ = r.read_i32 - _wv_null = r.read_i32 - _wv_begin_ = r.read_i32 - _wv_rescue_ = r.read_i32 - _wv_module_ = r.read_i32 - _wv_chan = r.read_i32 - _wv_go = r.read_i32 - _wv_let = r.read_i32 - _wv_export = r.read_i32 - _wv_delete = r.read_i32 - _wv_typedef = r.read_i32 - _wv_sizeof = r.read_i32 - _wv_static = r.read_i32 - _wv_super_ = r.read_i32 - Keywords.new(class_: _wv_class_, def_: _wv_def_, import: _wv_import, from: _wv_from, func: _wv_func, range: _wv_range, register: _wv_register, volatile: _wv_volatile, new: _wv_new, default: _wv_default, end_: _wv_end_, in_: _wv_in_, is: _wv_is, int: _wv_int, var: _wv_var, val: _wv_val, self_: _wv_self_, this: _wv_this, object: _wv_object, string: _wv_string, case_: _wv_case_, select: _wv_select, switch: _wv_switch, void: _wv_void, union: _wv_union, namespace: _wv_namespace, operator: _wv_operator, template: _wv_template, package: _wv_package, protocol: _wv_protocol, guard: _wv_guard, defer: _wv_defer, inout: _wv_inout, internal: _wv_internal, when_: _wv_when_, fun: _wv_fun, data: _wv_data, late: _wv_late, part: _wv_part, out: _wv_out, ref: _wv_ref, event: _wv_event, base: _wv_base, lambda: _wv_lambda, yield_: _wv_yield_, nil_: _wv_nil_, null: _wv_null, begin_: _wv_begin_, rescue_: _wv_rescue_, module_: _wv_module_, chan: _wv_chan, go: _wv_go, let: _wv_let, export: _wv_export, delete: _wv_delete, typedef: _wv_typedef, sizeof: _wv_sizeof, static: _wv_static, super_: _wv_super_) + Keywords.new( + class_: r.read_i32, + def_: r.read_i32, + import: r.read_i32, + from: r.read_i32, + func: r.read_i32, + range: r.read_i32, + register: r.read_i32, + volatile: r.read_i32, + new: r.read_i32, + default: r.read_i32, + end_: r.read_i32, + in_: r.read_i32, + is: r.read_i32, + int: r.read_i32, + var: r.read_i32, + val: r.read_i32, + self_: r.read_i32, + this: r.read_i32, + object: r.read_i32, + string: r.read_i32, + case_: r.read_i32, + select: r.read_i32, + switch: r.read_i32, + void: r.read_i32, + union: r.read_i32, + namespace: r.read_i32, + operator: r.read_i32, + template: r.read_i32, + package: r.read_i32, + protocol: r.read_i32, + guard: r.read_i32, + defer: r.read_i32, + inout: r.read_i32, + internal: r.read_i32, + when_: r.read_i32, + fun: r.read_i32, + data: r.read_i32, + late: r.read_i32, + part: r.read_i32, + out: r.read_i32, + ref: r.read_i32, + event: r.read_i32, + base: r.read_i32, + lambda: r.read_i32, + yield_: r.read_i32, + nil_: r.read_i32, + null: r.read_i32, + begin_: r.read_i32, + rescue_: r.read_i32, + module_: r.read_i32, + chan: r.read_i32, + go: r.read_i32, + let: r.read_i32, + export: r.read_i32, + delete: r.read_i32, + typedef: r.read_i32, + sizeof: r.read_i32, + static: r.read_i32, + super_: r.read_i32, + ) end # @api private @@ -728,58 +764,75 @@ module EdgeCases # @api private # Unpacks a Retired from the value-buffer wire format. def self._wv_read_retired(r) - _wv_id = r.read_i64 - Retired.new(id: _wv_id) + Retired.new(id: r.read_i64) end # @api private # Packs a WidgetHolder into the value-buffer wire format. def self._wv_write_widget_holder(w, v) - w.write_object(v.primary) - if v.fallback.nil? - w.write_flag(false) - else - w.write_flag(true) - w.write_object(v.fallback) - end - w.write_len(v.extras.length) - v.extras.each do |_wv_e0| - w.write_object(_wv_e0) - end - w.write_len(v.by_name.length) - v.by_name.each do |_wv_k0, _wv_v0| - w.write_string(_wv_k0) - w.write_object(_wv_v0) - end + w.write_object(v.primary, Widget) + _wv_write_opt_widget(w, v.fallback) + _wv_write_list_widget(w, v.extras) + _wv_write_map_string_widget(w, v.by_name) end # @api private # Unpacks a WidgetHolder from the value-buffer wire format. def self._wv_read_widget_holder(r) - _wv_primary = Widget._from_ptr(r.read_object_token) - if r.read_flag - _wv_fallback = Widget._from_ptr(r.read_object_token) - else - _wv_fallback = nil - end - _wv_extras = Array.new(r.read_len) do - _wv_e0 = Widget._from_ptr(r.read_object_token) - _wv_e0 - end - _wv_by_name = {} - r.read_len.times do - _wv_k0 = r.read_string - _wv_v0 = Widget._from_ptr(r.read_object_token) - _wv_by_name[_wv_k0] = _wv_v0 - end - WidgetHolder.new(primary: _wv_primary, fallback: _wv_fallback, extras: _wv_extras, by_name: _wv_by_name) + WidgetHolder.new( + primary: r.read_object(Widget), + fallback: _wv_read_opt_widget(r), + extras: _wv_read_list_widget(r), + by_name: _wv_read_map_string_widget(r), + ) end attach_function :edge_cases_edge_cases_Widget_clone, [:pointer], :pointer - attach_function :edge_cases_edge_cases_Widget_destroy, [:pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_Widget_destroy, [:pointer], :void attach_function :edge_cases_edge_cases_Widget_refresh, [:pointer, :pointer], :void, blocking: true attach_function :edge_cases_edge_cases_Widget_rename, [:pointer, :pointer, :size_t, :pointer, :size_t, :pointer, :pointer], :pointer, blocking: true attach_function :edge_cases_edge_cases_Widget_fetch, [:pointer, :int64, :pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_Widget_old_name, [:pointer, :pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_Widget_close, [:pointer, :pointer], :bool, blocking: true + attach_function :edge_cases_edge_cases_Widget_dispose, [:pointer, :bool, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_Vault_clone, [:pointer], :pointer + attach_function :edge_cases_edge_cases_Vault_destroy, [:pointer], :void + attach_function :edge_cases_edge_cases_Vault_open, [:pointer, :size_t, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_Vault_peek, [:pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_Vault_handle, [:pointer, :pointer], :int64, blocking: true + attach_function :edge_cases_edge_cases_Vault_probe, [:pointer], :bool, blocking: true + attach_function :edge_cases_edge_cases_Vault_to_string, [:pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_noop, [:pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_select, [:int32, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :bool, :pointer], :int32, blocking: true + attach_function :edge_cases_edge_cases_nested_optional, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_optional_params, [:pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_register_batch, [:pointer, :pointer, :pointer, :pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_widgets_everywhere, [:pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_widget_stream, [:pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_WidgetStreamIterator_next, [:pointer, :pointer, :pointer], :int32, blocking: true + attach_function :edge_cases_edge_cases_WidgetStreamIterator_destroy, [:pointer], :void + attach_function :edge_cases_edge_cases_async_widget, [:pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_make_widget, [:pointer, :size_t, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_find_widget, [:int64, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_inspect_widget, [:pointer, :pointer, :pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_string_stream, [:pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_StringStreamIterator_next, [:pointer, :pointer, :pointer, :pointer], :int32, blocking: true + attach_function :edge_cases_edge_cases_StringStreamIterator_destroy, [:pointer], :void + attach_function :edge_cases_edge_cases_scalar_stream, [:pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_ScalarStreamIterator_next, [:pointer, :pointer, :pointer], :int32, blocking: true + attach_function :edge_cases_edge_cases_ScalarStreamIterator_destroy, [:pointer], :void + attach_function :edge_cases_edge_cases_mode_stream, [:pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_ModeStreamIterator_next, [:pointer, :pointer, :pointer], :int32, blocking: true + attach_function :edge_cases_edge_cases_ModeStreamIterator_destroy, [:pointer], :void + attach_function :edge_cases_edge_cases_async_void, [:pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_async_bytes, [:uint32, :pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_async_struct, [:pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_async_list, [:pointer, :pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_async_optional, [:pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_async_enum, [:pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_async_keywords, [:pointer, :size_t, :int32, :pointer, :pointer], :void, blocking: true + attach_function :edge_cases_edge_cases_keywords_roundtrip, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true + attach_function :edge_cases_edge_cases_legacy_value, [:int32, :pointer, :pointer], :pointer, blocking: true # @api private # Completion trampoline for fetch. @@ -788,12 +841,9 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << edge_errors_error_from(*taken) + queue << _wv_edge_errors_error_from(*taken) else - value = _wv_decode(result_ptr, result_len) do |_wv_r| - _wv_value = _wv_read_outer(_wv_r) - _wv_value - end + value = _wv_decode(result_ptr, result_len) { |r| _wv_read_outer(r) } queue << value end rescue Exception => e # rubocop:disable Lint/RescueException @@ -801,23 +851,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_Widget_old_name, [:pointer, :pointer, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_Vault_clone, [:pointer], :pointer - attach_function :edge_cases_edge_cases_Vault_destroy, [:pointer], :void, blocking: true - attach_function :edge_cases_edge_cases_Vault_open, [:pointer, :size_t, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_Vault_peek, [:pointer, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_Vault_probe, [:pointer], :bool, blocking: true - attach_function :edge_cases_edge_cases_noop, [:pointer], :void, blocking: true - attach_function :edge_cases_edge_cases_select, [:int32, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :bool, :pointer], :int32, blocking: true - attach_function :edge_cases_edge_cases_nested_optional, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_optional_params, [:pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_register_batch, [:pointer, :pointer, :pointer, :pointer, :pointer], :void, blocking: true - attach_function :edge_cases_edge_cases_widgets_everywhere, [:pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :pointer, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_widget_stream, [:pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_WidgetStreamIterator_next, [:pointer, :pointer, :pointer], :int32, blocking: true - attach_function :edge_cases_edge_cases_WidgetStreamIterator_destroy, [:pointer], :void, blocking: true - attach_function :edge_cases_edge_cases_async_widget, [:pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_widget. EDGE_CASES_EDGE_CASES_ASYNC_WIDGET_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :pointer]) do |ctx, err, result| @@ -825,10 +858,9 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << edge_errors_error_from(*taken) + queue << _wv_edge_errors_error_from(*taken) else - raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if result.null? - value = Widget._from_ptr(result) + value = Widget._from_ptr(_wv_nonnull(result)) queue << value end rescue Exception => e # rubocop:disable Lint/RescueException @@ -836,20 +868,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_make_widget, [:pointer, :size_t, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_find_widget, [:int64, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_inspect_widget, [:pointer, :pointer, :pointer, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_string_stream, [:pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_StringStreamIterator_next, [:pointer, :pointer, :pointer, :pointer], :int32, blocking: true - attach_function :edge_cases_edge_cases_StringStreamIterator_destroy, [:pointer], :void, blocking: true - attach_function :edge_cases_edge_cases_scalar_stream, [:pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_ScalarStreamIterator_next, [:pointer, :pointer, :pointer], :int32, blocking: true - attach_function :edge_cases_edge_cases_ScalarStreamIterator_destroy, [:pointer], :void, blocking: true - attach_function :edge_cases_edge_cases_mode_stream, [:pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_ModeStreamIterator_next, [:pointer, :pointer, :pointer], :int32, blocking: true - attach_function :edge_cases_edge_cases_ModeStreamIterator_destroy, [:pointer], :void, blocking: true - attach_function :edge_cases_edge_cases_async_void, [:pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_void. EDGE_CASES_EDGE_CASES_ASYNC_VOID_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer]) do |ctx, err| @@ -857,7 +875,7 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else value = nil queue << value @@ -867,8 +885,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_async_bytes, [:uint32, :pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_bytes. EDGE_CASES_EDGE_CASES_ASYNC_BYTES_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :size_t]) do |ctx, err, result_ptr, result_len| @@ -876,7 +892,7 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else value = _wv_take_bytes(result_ptr, result_len) queue << value @@ -886,8 +902,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_async_struct, [:pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_struct. EDGE_CASES_EDGE_CASES_ASYNC_STRUCT_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :size_t]) do |ctx, err, result_ptr, result_len| @@ -895,12 +909,9 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << edge_errors_error_from(*taken) + queue << _wv_edge_errors_error_from(*taken) else - value = _wv_decode(result_ptr, result_len) do |_wv_r| - _wv_value = _wv_read_outer(_wv_r) - _wv_value - end + value = _wv_decode(result_ptr, result_len) { |r| _wv_read_outer(r) } queue << value end rescue Exception => e # rubocop:disable Lint/RescueException @@ -908,8 +919,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_async_list, [:pointer, :pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_list. EDGE_CASES_EDGE_CASES_ASYNC_LIST_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :size_t]) do |ctx, err, result_ptr, result_len| @@ -917,15 +926,9 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << edge_errors_error_from(*taken) + queue << _wv_edge_errors_error_from(*taken) else - value = _wv_decode(result_ptr, result_len) do |_wv_r| - _wv_value = Array.new(_wv_r.read_len) do - _wv_e0 = _wv_read_outer(_wv_r) - _wv_e0 - end - _wv_value - end + value = _wv_decode(result_ptr, result_len) { |r| _wv_read_list_outer(r) } queue << value end rescue Exception => e # rubocop:disable Lint/RescueException @@ -933,8 +936,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_async_optional, [:pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_optional. EDGE_CASES_EDGE_CASES_ASYNC_OPTIONAL_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :size_t]) do |ctx, err, result_ptr, result_len| @@ -942,16 +943,9 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else - value = _wv_decode(result_ptr, result_len) do |_wv_r| - if _wv_r.read_flag - _wv_value = _wv_r.read_i64 - else - _wv_value = nil - end - _wv_value - end + value = _wv_decode(result_ptr, result_len) { |r| _wv_read_opt_i64(r) } queue << value end rescue Exception => e # rubocop:disable Lint/RescueException @@ -959,8 +953,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_async_enum, [:pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_enum. EDGE_CASES_EDGE_CASES_ASYNC_ENUM_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :int32]) do |ctx, err, result| @@ -968,7 +960,7 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else value = result queue << value @@ -978,8 +970,6 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_async_keywords, [:pointer, :size_t, :int32, :pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for async_keywords. EDGE_CASES_EDGE_CASES_ASYNC_KEYWORDS_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :size_t]) do |ctx, err, result_ptr, result_len| @@ -987,7 +977,7 @@ module EdgeCases begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else value = _wv_take_string(result_ptr, result_len) queue << value @@ -997,14 +987,11 @@ module EdgeCases end end - attach_function :edge_cases_edge_cases_keywords_roundtrip, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true - attach_function :edge_cases_edge_cases_legacy_value, [:int32, :pointer, :pointer], :pointer, blocking: true - # Callback interface carrying composite payloads and objects # # Consumer-implemented callback interface. Any object responding to the # methods below is accepted wherever a BatchListener is expected; include - # this module to inherit NotImplementedError defaults. The producer + # this module to inherit NotImplementedError defaults. The library # may call the methods from any thread until it releases the # implementation. module BatchListener @@ -1023,138 +1010,109 @@ module EdgeCases end # Scalar return travels back through the vtable - # @return [Object] a bool + # @return [Boolean] def should_continue(class_) raise NotImplementedError, "#{self.class}#should_continue is not implemented" end # Enum return travels back as its integer discriminant - # @return [Object] a edge_cases.Mode + # @return [Integer] def pick_mode raise NotImplementedError, "#{self.class}#pick_mode is not implemented" end end # @api private - # The C vtable layout the producer calls a BatchListener through. + # The C vtable layout the library calls a BatchListener through. class WvBatchListenerVtable < FFI::Struct - layout :on_batch, :pointer, + layout :wv_size, :uint32, + :wv_flags, :uint32, + :wv_free, :pointer, + :on_batch, :pointer, :on_widget, :pointer, :on_tick, :pointer, :should_continue, :pointer, - :pick_mode, :pointer, - :free, :pointer + :pick_mode, :pointer + end + + # @api private + # Releases a BatchListener implementation when the library drops its last + # reference. May run on any library thread. + WV_BATCH_LISTENER_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| + _wv_cb_free(ctx) end # @api private # Trampoline for BatchListener#on_batch. - WV_BATCH_LISTENER_ON_BATCH = FFI::Function.new(:void, [:pointer, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :int32, :pointer]) do |ctx, items_ptr, items_len, raw_ptr, raw_len, first_ptr, first_len, mode, out_err| - begin - items_r = WvBufferReader.new(_wv_borrow_bytes(items_ptr, items_len)) - items_v = Array.new(items_r.read_len) do - _wv_e0 = _wv_read_outer(items_r) - _wv_e0 - end - items_r.expect_end! - raw_v = _wv_borrow_bytes(raw_ptr, raw_len) - first_r = WvBufferReader.new(_wv_borrow_bytes(first_ptr, first_len)) - if first_r.read_flag - first_v = _wv_read_outer(first_r) - else - first_v = nil - end - first_r.expect_end! - mode_v = mode - impl = _wv_cb_lookup(ctx) - impl.on_batch(items_v, raw_v, first_v, mode_v) - nil - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - nil - end + WV_BATCH_LISTENER_ON_BATCH = FFI::Function.new(:void, [:pointer, :pointer, :size_t, :pointer, :size_t, :pointer, :size_t, :int32, :pointer]) do |wv_ctx, items_ptr, items_len, raw_ptr, raw_len, first_ptr, first_len, mode, wv_err| + items = _wv_decode_borrowed(items_ptr, items_len) { |r| _wv_read_list_outer(r) } + raw = _wv_borrow_bytes(raw_ptr, raw_len) + first = _wv_decode_borrowed(first_ptr, first_len) { |r| _wv_read_opt_outer(r) } + _wv_cb_lookup(wv_ctx).on_batch(items, raw, first, mode) + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil end # @api private # Trampoline for BatchListener#on_widget. - WV_BATCH_LISTENER_ON_WIDGET = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :pointer, :size_t, :pointer]) do |ctx, widget, fallback, children_ptr, children_len, out_err| - begin - widget_v = Widget._from_ptr(widget) - fallback_v = fallback.null? ? nil : Widget._from_ptr(fallback) - children_r = WvBufferReader.new(_wv_borrow_bytes(children_ptr, children_len)) - children_v = Array.new(children_r.read_len) do - _wv_e0 = Widget._from_ptr(children_r.read_object_token) - _wv_e0 - end - children_r.expect_end! - impl = _wv_cb_lookup(ctx) - impl.on_widget(widget_v, fallback_v, children_v) - nil - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - nil - end + WV_BATCH_LISTENER_ON_WIDGET = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :pointer, :size_t, :pointer]) do |wv_ctx, widget_ptr, fallback_ptr, children_ptr, children_len, wv_err| + widget = Widget._from_ptr(widget_ptr) + fallback = fallback_ptr.null? ? nil : Widget._from_ptr(fallback_ptr) + children = _wv_decode_borrowed(children_ptr, children_len) { |r| _wv_read_list_widget(r) } + _wv_cb_lookup(wv_ctx).on_widget(widget, fallback, children) + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil end # @api private # Trampoline for BatchListener#on_tick. - WV_BATCH_LISTENER_ON_TICK = FFI::Function.new(:void, [:pointer, :pointer]) do |ctx, out_err| - begin - impl = _wv_cb_lookup(ctx) - impl.on_tick - nil - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - nil - end + WV_BATCH_LISTENER_ON_TICK = FFI::Function.new(:void, [:pointer, :pointer]) do |wv_ctx, wv_err| + _wv_cb_lookup(wv_ctx).on_tick + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil end # @api private # Trampoline for BatchListener#should_continue. - WV_BATCH_LISTENER_SHOULD_CONTINUE = FFI::Function.new(:bool, [:pointer, :int32, :pointer]) do |ctx, class_, out_err| - begin - class__v = class_ - impl = _wv_cb_lookup(ctx) - impl.should_continue(class__v) ? true : false - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - false - end + WV_BATCH_LISTENER_SHOULD_CONTINUE = FFI::Function.new(:bool, [:pointer, :int32, :pointer]) do |wv_ctx, class_, wv_err| + _wv_cb_lookup(wv_ctx).should_continue(class_) ? true : false + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + false end # @api private # Trampoline for BatchListener#pick_mode. - WV_BATCH_LISTENER_PICK_MODE = FFI::Function.new(:int32, [:pointer, :pointer]) do |ctx, out_err| - begin - impl = _wv_cb_lookup(ctx) - Integer(impl.pick_mode) - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - 0 - end - end - - # @api private - # Releases a BatchListener implementation when the producer drops its last - # reference. - WV_BATCH_LISTENER_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| - _wv_cb_free(ctx) + WV_BATCH_LISTENER_PICK_MODE = FFI::Function.new(:int32, [:pointer, :pointer]) do |wv_ctx, wv_err| + _wv_int(_wv_cb_lookup(wv_ctx).pick_mode, :i32) + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + 0 end # The one process-wide BatchListener vtable every registration hands the - # producer; its entries live for the process lifetime. + # library; its entries live for the process lifetime. WV_BATCH_LISTENER_VTABLE = WvBatchListenerVtable.new + WV_BATCH_LISTENER_VTABLE[:wv_size] = WvBatchListenerVtable.size + WV_BATCH_LISTENER_VTABLE[:wv_flags] = 0 + WV_BATCH_LISTENER_VTABLE[:wv_free] = WV_BATCH_LISTENER_FREE WV_BATCH_LISTENER_VTABLE[:on_batch] = WV_BATCH_LISTENER_ON_BATCH WV_BATCH_LISTENER_VTABLE[:on_widget] = WV_BATCH_LISTENER_ON_WIDGET WV_BATCH_LISTENER_VTABLE[:on_tick] = WV_BATCH_LISTENER_ON_TICK WV_BATCH_LISTENER_VTABLE[:should_continue] = WV_BATCH_LISTENER_SHOULD_CONTINUE WV_BATCH_LISTENER_VTABLE[:pick_mode] = WV_BATCH_LISTENER_PICK_MODE - WV_BATCH_LISTENER_VTABLE[:free] = WV_BATCH_LISTENER_FREE # Callback interface whose name and methods are keywords # # Consumer-implemented callback interface. Any object responding to the # methods below is accepted wherever a Keyword is expected; include - # this module to inherit NotImplementedError defaults. The producer + # this module to inherit NotImplementedError defaults. The library # may call the methods from any thread until it releases the # implementation. module Keyword @@ -1162,59 +1120,56 @@ module EdgeCases raise NotImplementedError, "#{self.class}#delete is not implemented" end - # @return [Object] a i32 + # @return [Integer] def import raise NotImplementedError, "#{self.class}#import is not implemented" end end # @api private - # The C vtable layout the producer calls a Keyword through. + # The C vtable layout the library calls a Keyword through. class WvKeywordVtable < FFI::Struct - layout :delete, :pointer, - :import, :pointer, - :free, :pointer + layout :wv_size, :uint32, + :wv_flags, :uint32, + :wv_free, :pointer, + :delete, :pointer, + :import, :pointer end # @api private - # Trampoline for Keyword#delete. - WV_KEYWORD_DELETE = FFI::Function.new(:void, [:pointer, :int32, :pointer]) do |ctx, self_, out_err| - begin - self__v = self_ - impl = _wv_cb_lookup(ctx) - impl.delete(self__v) - nil - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - nil - end + # Releases a Keyword implementation when the library drops its last + # reference. May run on any library thread. + WV_KEYWORD_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| + _wv_cb_free(ctx) end # @api private - # Trampoline for Keyword#import. - WV_KEYWORD_IMPORT = FFI::Function.new(:int32, [:pointer, :pointer]) do |ctx, out_err| - begin - impl = _wv_cb_lookup(ctx) - Integer(impl.import) - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - 0 - end + # Trampoline for Keyword#delete. + WV_KEYWORD_DELETE = FFI::Function.new(:void, [:pointer, :int32, :pointer]) do |wv_ctx, self_, wv_err| + _wv_cb_lookup(wv_ctx).delete(self_) + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil end # @api private - # Releases a Keyword implementation when the producer drops its last - # reference. - WV_KEYWORD_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| - _wv_cb_free(ctx) + # Trampoline for Keyword#import. + WV_KEYWORD_IMPORT = FFI::Function.new(:int32, [:pointer, :pointer]) do |wv_ctx, wv_err| + _wv_int(_wv_cb_lookup(wv_ctx).import, :i32) + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + 0 end # The one process-wide Keyword vtable every registration hands the - # producer; its entries live for the process lifetime. + # library; its entries live for the process lifetime. WV_KEYWORD_VTABLE = WvKeywordVtable.new + WV_KEYWORD_VTABLE[:wv_size] = WvKeywordVtable.size + WV_KEYWORD_VTABLE[:wv_flags] = 0 + WV_KEYWORD_VTABLE[:wv_free] = WV_KEYWORD_FREE WV_KEYWORD_VTABLE[:delete] = WV_KEYWORD_DELETE WV_KEYWORD_VTABLE[:import] = WV_KEYWORD_IMPORT - WV_KEYWORD_VTABLE[:free] = WV_KEYWORD_FREE # @api private # Owns one strong reference to a Widget; releases it exactly once. @@ -1239,51 +1194,69 @@ module EdgeCases EdgeCases._wv_pin(self) do |_wv_self| err = ErrorStruct.new EdgeCases.edge_cases_edge_cases_Widget_refresh(_wv_self, err) - EdgeCases.check_edge_errors_error!(err) + EdgeCases._wv_check_edge_errors_error!(err) + nil end end - def rename(class_, import) + # @return [String] + def rename(class_, import = nil) class__s = EdgeCases._wv_str(class_) - import_w = WvBufferWriter.new - if import.nil? - import_w.write_flag(false) - else - import_w.write_flag(true) - import_w.write_string(import) - end + import_s = EdgeCases._wv_encode { |w| EdgeCases._wv_write_opt_string(w, import) } EdgeCases._wv_pin(self) do |_wv_self| err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - import_s = EdgeCases._wv_seal(import_w).first result = EdgeCases.edge_cases_edge_cases_Widget_rename(_wv_self, class__s, class__s.bytesize, import_s, import_s.bytesize, out_len, err) - EdgeCases.check_error!(err) + EdgeCases._wv_check!(err) EdgeCases._wv_take_string(result, out_len.read(:size_t)) end end # Async method with a struct result # Blocks until the call completes on a producer thread. + # @return [Outer] def fetch(id) - queue = nil - EdgeCases._wv_pin(self) do |_wv_self| - ctx, queue = EdgeCases._wv_async_begin + queue = EdgeCases._wv_pin(self) do |_wv_self| + ctx, pending = EdgeCases._wv_async_begin EdgeCases.edge_cases_edge_cases_Widget_fetch(_wv_self, id, EDGE_CASES_EDGE_CASES_WIDGET_FETCH_CALLBACK, ctx) + pending end EdgeCases._wv_async_wait(queue) end + # @return [String] # @deprecated Use rename instead def old_name - warn '[DEPRECATED] Use rename instead' + warn('EdgeCases::Widget#old_name is deprecated: Use rename instead', uplevel: 1) EdgeCases._wv_pin(self) do |_wv_self| err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = EdgeCases.edge_cases_edge_cases_Widget_old_name(_wv_self, out_len, err) - EdgeCases.check_error!(err) + EdgeCases._wv_check!(err) EdgeCases._wv_take_string(result, out_len.read(:size_t)) end end + + # Spelled like the release method of several wrappers + # @return [Boolean] + def close_ + EdgeCases._wv_pin(self) do |_wv_self| + err = ErrorStruct.new + result = EdgeCases.edge_cases_edge_cases_Widget_close(_wv_self, err) + EdgeCases._wv_check!(err) + result + end + end + + # Spelled like the release method of the Dart and .NET wrappers + def dispose(force) + EdgeCases._wv_pin(self) do |_wv_self| + err = ErrorStruct.new + EdgeCases.edge_cases_edge_cases_Widget_dispose(_wv_self, (force ? true : false), err) + EdgeCases._wv_check!(err) + nil + end + end end # @api private @@ -1306,225 +1279,151 @@ module EdgeCases end # Fails when the vault is busy + # @return [Vault] def self.open(path) path_s = EdgeCases._wv_str(path) err = ErrorStruct.new result = EdgeCases.edge_cases_edge_cases_Vault_open(path_s, path_s.bytesize, err) - EdgeCases.check_edge_errors_error!(err) - raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if result.null? - _from_ptr(result) + EdgeCases._wv_check_edge_errors_error!(err) + _from_ptr(EdgeCases._wv_nonnull(result)) end + # @return [Widget, nil] def peek EdgeCases._wv_pin(self) do |_wv_self| err = ErrorStruct.new result = EdgeCases.edge_cases_edge_cases_Vault_peek(_wv_self, err) - EdgeCases.check_error!(err) + EdgeCases._wv_check!(err) result.null? ? nil : Widget._from_ptr(result) end end + # Spelled like the handle accessor of several wrappers + # @return [Integer] + def handle_ + EdgeCases._wv_pin(self) do |_wv_self| + err = ErrorStruct.new + result = EdgeCases.edge_cases_edge_cases_Vault_handle(_wv_self, err) + EdgeCases._wv_check!(err) + result + end + end + # Throwing static + # @return [Boolean] def self.probe err = ErrorStruct.new result = EdgeCases.edge_cases_edge_cases_Vault_probe(err) - EdgeCases.check_edge_errors_error!(err) + EdgeCases._wv_check_edge_errors_error!(err) result end + + # Static spelled like an inherited object member + # @return [String] + def self.to_string + err = ErrorStruct.new + out_len = FFI::MemoryPointer.new(:size_t) + result = EdgeCases.edge_cases_edge_cases_Vault_to_string(out_len, err) + EdgeCases._wv_check!(err) + EdgeCases._wv_take_string(result, out_len.read(:size_t)) + end end # No parameters and no result def self.noop err = ErrorStruct.new edge_cases_edge_cases_noop(err) - check_error!(err) + _wv_check!(err) + nil end # Function named after a keyword whose parameters are all keywords + # @return [Integer] def self.select(class_, from, func, range, type) from_s = _wv_str(from) - func_w = WvBufferWriter.new - func_w.write_len(func.length) - func.each do |_wv_e0| - func_w.write_string(_wv_e0) - end - range_w = WvBufferWriter.new - if range.nil? - range_w.write_flag(false) - else - range_w.write_flag(true) - range_w.write_i32(range) - end + func_s = _wv_encode { |w| _wv_write_list_string(w, func) } + range_s = _wv_encode { |w| _wv_write_opt_i32(w, range) } err = ErrorStruct.new - func_s, range_s = _wv_seal(func_w, range_w) result = edge_cases_edge_cases_select(class_, from_s, from_s.bytesize, func_s, func_s.bytesize, range_s, range_s.bytesize, (type ? true : false), err) - check_error!(err) + _wv_check!(err) result end # Optional list of optional structs, the deepest legal nesting - def self.nested_optional(items) - items_w = WvBufferWriter.new - if items.nil? - items_w.write_flag(false) - else - items_w.write_flag(true) - items_w.write_len(items.length) - items.each do |_wv_e0| - if _wv_e0.nil? - items_w.write_flag(false) - else - items_w.write_flag(true) - _wv_write_outer(items_w, _wv_e0) - end - end - end + # @return [Array, nil] + def self.nested_optional(items = nil) + items_s = _wv_encode { |w| _wv_write_opt_list_opt_outer(w, items) } err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - items_s = _wv_seal(items_w).first result = edge_cases_edge_cases_nested_optional(items_s, items_s.bytesize, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - if _wv_r.read_flag - _wv_value = Array.new(_wv_r.read_len) do - if _wv_r.read_flag - _wv_e0 = _wv_read_outer(_wv_r) - else - _wv_e0 = nil - end - _wv_e0 - end - else - _wv_value = nil - end - _wv_value - end - end - - def self.optional_params(text, count, outer, tags, blob, mode) - text_w = WvBufferWriter.new - if text.nil? - text_w.write_flag(false) - else - text_w.write_flag(true) - text_w.write_string(text) - end - count_w = WvBufferWriter.new - if count.nil? - count_w.write_flag(false) - else - count_w.write_flag(true) - count_w.write_i32(count) - end - outer_w = WvBufferWriter.new - if outer.nil? - outer_w.write_flag(false) - else - outer_w.write_flag(true) - _wv_write_outer(outer_w, outer) - end - tags_w = WvBufferWriter.new - if tags.nil? - tags_w.write_flag(false) - else - tags_w.write_flag(true) - tags_w.write_len(tags.length) - tags.each do |_wv_e0| - tags_w.write_string(_wv_e0) - end - end - blob_w = WvBufferWriter.new - if blob.nil? - blob_w.write_flag(false) - else - blob_w.write_flag(true) - blob_w.write_bytes(blob) - end - mode_w = WvBufferWriter.new - if mode.nil? - mode_w.write_flag(false) - else - mode_w.write_flag(true) - mode_w.write_i32(mode) - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_opt_list_opt_outer(r) } + end + + # @return [Integer, nil] + def self.optional_params(text = nil, count = nil, outer = nil, tags = nil, blob = nil, mode = nil) + text_s = _wv_encode { |w| _wv_write_opt_string(w, text) } + count_s = _wv_encode { |w| _wv_write_opt_i32(w, count) } + outer_s = _wv_encode { |w| _wv_write_opt_outer(w, outer) } + tags_s = _wv_encode { |w| _wv_write_opt_list_string(w, tags) } + blob_s = _wv_encode { |w| _wv_write_opt_bytes(w, blob) } + mode_s = _wv_encode { |w| _wv_write_opt_mode(w, mode) } err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - text_s, count_s, outer_s, tags_s, blob_s, mode_s = _wv_seal(text_w, count_w, outer_w, tags_w, blob_w, mode_w) result = edge_cases_edge_cases_optional_params(text_s, text_s.bytesize, count_s, count_s.bytesize, outer_s, outer_s.bytesize, tags_s, tags_s.bytesize, blob_s, blob_s.bytesize, mode_s, mode_s.bytesize, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - if _wv_r.read_flag - _wv_value = _wv_r.read_i32 - else - _wv_value = nil - end - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_opt_mode(r) } end # Two callback interface parameters on one function def self.register_batch(listener, keyword) + _wv_present!(listener, 'listener') + _wv_present!(keyword, 'keyword') err = ErrorStruct.new - listener_ctx = _wv_cb_register(listener) - keyword_ctx = _wv_cb_register(keyword) - edge_cases_edge_cases_register_batch(listener_ctx, WV_BATCH_LISTENER_VTABLE.to_ptr, keyword_ctx, WV_KEYWORD_VTABLE.to_ptr, err) - check_error!(err) + listener_ctx, listener_vtable = _wv_cb_register(listener, WV_BATCH_LISTENER_VTABLE) + keyword_ctx, keyword_vtable = _wv_cb_register(keyword, WV_KEYWORD_VTABLE) + edge_cases_edge_cases_register_batch(listener_ctx, listener_vtable, keyword_ctx, keyword_vtable, err) + _wv_check!(err) + nil end # Objects in every buffered position + # @return [WidgetHolder] def self.widgets_everywhere(list, by_name, sparse, holder) list_w = WvBufferWriter.new - list_w.write_len(list.length) - list.each do |_wv_e0| - list_w.write_object(_wv_e0) - end + _wv_write_list_widget(list_w, list) by_name_w = WvBufferWriter.new - by_name_w.write_len(by_name.length) - by_name.each do |_wv_k0, _wv_v0| - by_name_w.write_string(_wv_k0) - by_name_w.write_object(_wv_v0) - end + _wv_write_map_string_widget(by_name_w, by_name) sparse_w = WvBufferWriter.new - sparse_w.write_len(sparse.length) - sparse.each do |_wv_e0| - if _wv_e0.nil? - sparse_w.write_flag(false) - else - sparse_w.write_flag(true) - sparse_w.write_object(_wv_e0) - end - end + _wv_write_list_opt_widget(sparse_w, sparse) holder_w = WvBufferWriter.new _wv_write_widget_holder(holder_w, holder) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) list_s, by_name_s, sparse_s, holder_s = _wv_seal(list_w, by_name_w, sparse_w, holder_w) result = edge_cases_edge_cases_widgets_everywhere(list_s, list_s.bytesize, by_name_s, by_name_s.bytesize, sparse_s, sparse_s.bytesize, holder_s, holder_s.bytesize, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_widget_holder(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_widget_holder(r) } end # Iterator of objects # Returns a lazy Enumerator that pulls one element per step. The # producer iterator starts on the first pull and is released when # iteration finishes or is abandoned early. + # @return [Enumerator] def self.widget_stream Enumerator.new do |y| err = ErrorStruct.new iter = edge_cases_edge_cases_widget_stream(err) begin - check_error!(err) + _wv_check!(err) out_item = FFI::MemoryPointer.new(:pointer) loop do has_item = edge_cases_edge_cases_WidgetStreamIterator_next(iter, out_item, err) - check_error!(err) + _wv_check!(err) break if has_item.zero? item = out_item.read_pointer - raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if item.null? - value = Widget._from_ptr(item) + value = Widget._from_ptr(_wv_nonnull(item)) y << value end ensure @@ -1535,6 +1434,7 @@ module EdgeCases # Async returning an object # Blocks until the call completes on a producer thread. + # @return [Widget] def self.async_widget ctx, queue = _wv_async_begin edge_cases_edge_cases_async_widget(EDGE_CASES_EDGE_CASES_ASYNC_WIDGET_CALLBACK, ctx) @@ -1542,30 +1442,34 @@ module EdgeCases end # Factory for an interface with no constructor + # @return [Widget] def self.make_widget(label) label_s = _wv_str(label) err = ErrorStruct.new result = edge_cases_edge_cases_make_widget(label_s, label_s.bytesize, err) - check_error!(err) - raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if result.null? - Widget._from_ptr(result) + _wv_check!(err) + Widget._from_ptr(_wv_nonnull(result)) end # Optional interface return + # @return [Widget, nil] def self.find_widget(id) err = ErrorStruct.new result = edge_cases_edge_cases_find_widget(id, err) - check_edge_errors_error!(err) + _wv_check_edge_errors_error!(err) result.null? ? nil : Widget._from_ptr(result) end # Interface parameter, including an optional one - def self.inspect_widget(widget, fallback) + # @return [String] + def self.inspect_widget(widget, fallback = nil) + _wv_object!(widget, Widget, 'widget') + _wv_object!(fallback, Widget, 'fallback', true) _wv_pin(widget, fallback) do |_wv_widget, _wv_fallback| err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = edge_cases_edge_cases_inspect_widget(_wv_widget, _wv_fallback, out_len, err) - check_error!(err) + _wv_check!(err) _wv_take_string(result, out_len.read(:size_t)) end end @@ -1574,17 +1478,18 @@ module EdgeCases # Returns a lazy Enumerator that pulls one element per step. The # producer iterator starts on the first pull and is released when # iteration finishes or is abandoned early. + # @return [Enumerator] def self.string_stream Enumerator.new do |y| err = ErrorStruct.new iter = edge_cases_edge_cases_string_stream(err) begin - check_error!(err) + _wv_check!(err) out_item = FFI::MemoryPointer.new(:pointer) out_len = FFI::MemoryPointer.new(:size_t) loop do has_item = edge_cases_edge_cases_StringStreamIterator_next(iter, out_item, out_len, err) - check_error!(err) + _wv_check!(err) break if has_item.zero? item = out_item.read_pointer value = _wv_take_string(item, out_len.read(:size_t)) @@ -1600,16 +1505,17 @@ module EdgeCases # Returns a lazy Enumerator that pulls one element per step. The # producer iterator starts on the first pull and is released when # iteration finishes or is abandoned early. + # @return [Enumerator] def self.scalar_stream Enumerator.new do |y| err = ErrorStruct.new iter = edge_cases_edge_cases_scalar_stream(err) begin - check_error!(err) + _wv_check!(err) out_item = FFI::MemoryPointer.new(:int32) loop do has_item = edge_cases_edge_cases_ScalarStreamIterator_next(iter, out_item, err) - check_error!(err) + _wv_check!(err) break if has_item.zero? item = out_item.read(:int32) value = item @@ -1625,16 +1531,17 @@ module EdgeCases # Returns a lazy Enumerator that pulls one element per step. The # producer iterator starts on the first pull and is released when # iteration finishes or is abandoned early. + # @return [Enumerator] def self.mode_stream Enumerator.new do |y| err = ErrorStruct.new iter = edge_cases_edge_cases_mode_stream(err) begin - check_error!(err) + _wv_check!(err) out_item = FFI::MemoryPointer.new(:int32) loop do has_item = edge_cases_edge_cases_ModeStreamIterator_next(iter, out_item, err) - check_error!(err) + _wv_check!(err) break if has_item.zero? item = out_item.read(:int32) value = item @@ -1655,6 +1562,7 @@ module EdgeCases end # Blocks until the call completes on a producer thread. + # @return [String] def self.async_bytes(size) ctx, queue = _wv_async_begin edge_cases_edge_cases_async_bytes(size, EDGE_CASES_EDGE_CASES_ASYNC_BYTES_CALLBACK, ctx) @@ -1662,6 +1570,7 @@ module EdgeCases end # Blocks until the call completes on a producer thread. + # @return [Outer] def self.async_struct ctx, queue = _wv_async_begin edge_cases_edge_cases_async_struct(EDGE_CASES_EDGE_CASES_ASYNC_STRUCT_CALLBACK, ctx) @@ -1670,6 +1579,7 @@ module EdgeCases # Blocks until the call completes on a producer thread. # @param cancel [CancelToken, nil] cancels the call; it then raises Cancelled + # @return [Array] def self.async_list(cancel: nil) own_token = CancelToken.new if cancel.nil? token = cancel || own_token @@ -1681,6 +1591,7 @@ module EdgeCases end # Blocks until the call completes on a producer thread. + # @return [Integer, nil] def self.async_optional ctx, queue = _wv_async_begin edge_cases_edge_cases_async_optional(EDGE_CASES_EDGE_CASES_ASYNC_OPTIONAL_CALLBACK, ctx) @@ -1688,6 +1599,7 @@ module EdgeCases end # Blocks until the call completes on a producer thread. + # @return [Integer] def self.async_enum ctx, queue = _wv_async_begin edge_cases_edge_cases_async_enum(EDGE_CASES_EDGE_CASES_ASYNC_ENUM_CALLBACK, ctx) @@ -1695,6 +1607,7 @@ module EdgeCases end # Blocks until the call completes on a producer thread. + # @return [String] def self.async_keywords(class_, import) class__s = _wv_str(class_) ctx, queue = _wv_async_begin @@ -1702,32 +1615,376 @@ module EdgeCases _wv_async_wait(queue) end + # @return [Keywords] def self.keywords_roundtrip(value) - value_w = WvBufferWriter.new - _wv_write_keywords(value_w, value) + value_s = _wv_encode { |w| _wv_write_keywords(w, value) } err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - value_s = _wv_seal(value_w).first result = edge_cases_edge_cases_keywords_roundtrip(value_s, value_s.bytesize, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_keywords(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_keywords(r) } end + # @return [Retired] # @deprecated Use Mode-based APIs def self.legacy_value(value) - warn '[DEPRECATED] Use Mode-based APIs' + warn('EdgeCases.legacy_value is deprecated: Use Mode-based APIs', uplevel: 1) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = edge_cases_edge_cases_legacy_value(value, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_retired(_wv_r) - _wv_value + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_retired(r) } + end + + # === Value-buffer codecs of the composite types === + + # @api private + # Packs `[string]` into the value-buffer wire format. + def self._wv_write_list_string(w, v) + w.write_len(v.length) + v.each { |e| w.write_string(e) } + end + + # @api private + # Unpacks `[string]` from the value-buffer wire format. + def self._wv_read_list_string(r) + Array.new(r.read_len) { r.read_string } + end + + # @api private + # Packs `i32?` into the value-buffer wire format. + def self._wv_write_opt_i32(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_i32(v) + end + end + + # @api private + # Unpacks `i32?` from the value-buffer wire format. + def self._wv_read_opt_i32(r) + r.read_flag ? r.read_i32 : nil + end + + # @api private + # Packs `[Outer?]?` into the value-buffer wire format. + def self._wv_write_opt_list_opt_outer(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + _wv_write_list_opt_outer(w, v) + end + end + + # @api private + # Unpacks `[Outer?]?` from the value-buffer wire format. + def self._wv_read_opt_list_opt_outer(r) + r.read_flag ? _wv_read_list_opt_outer(r) : nil + end + + # @api private + # Packs `[Outer?]` into the value-buffer wire format. + def self._wv_write_list_opt_outer(w, v) + w.write_len(v.length) + v.each { |e| _wv_write_opt_outer(w, e) } + end + + # @api private + # Unpacks `[Outer?]` from the value-buffer wire format. + def self._wv_read_list_opt_outer(r) + Array.new(r.read_len) { _wv_read_opt_outer(r) } + end + + # @api private + # Packs `Outer?` into the value-buffer wire format. + def self._wv_write_opt_outer(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + _wv_write_outer(w, v) + end + end + + # @api private + # Unpacks `Outer?` from the value-buffer wire format. + def self._wv_read_opt_outer(r) + r.read_flag ? _wv_read_outer(r) : nil + end + + # @api private + # Packs `string?` into the value-buffer wire format. + def self._wv_write_opt_string(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_string(v) + end + end + + # @api private + # Unpacks `string?` from the value-buffer wire format. + def self._wv_read_opt_string(r) + r.read_flag ? r.read_string : nil + end + + # @api private + # Packs `[string]?` into the value-buffer wire format. + def self._wv_write_opt_list_string(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + _wv_write_list_string(w, v) + end + end + + # @api private + # Unpacks `[string]?` from the value-buffer wire format. + def self._wv_read_opt_list_string(r) + r.read_flag ? _wv_read_list_string(r) : nil + end + + # @api private + # Packs `bytes?` into the value-buffer wire format. + def self._wv_write_opt_bytes(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_bytes(v) + end + end + + # @api private + # Unpacks `bytes?` from the value-buffer wire format. + def self._wv_read_opt_bytes(r) + r.read_flag ? r.read_bytes : nil + end + + # @api private + # Packs `Mode?` into the value-buffer wire format. + def self._wv_write_opt_mode(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_i32(v) end end + + # @api private + # Unpacks `Mode?` from the value-buffer wire format. + def self._wv_read_opt_mode(r) + r.read_flag ? r.read_i32 : nil + end + + # @api private + # Packs `[Widget]` into the value-buffer wire format. + def self._wv_write_list_widget(w, v) + w.write_len(v.length) + v.each { |e| w.write_object(e, Widget) } + end + + # @api private + # Unpacks `[Widget]` from the value-buffer wire format. + def self._wv_read_list_widget(r) + Array.new(r.read_len) { r.read_object(Widget) } + end + + # @api private + # Packs `{string:Widget}` into the value-buffer wire format. + def self._wv_write_map_string_widget(w, v) + w.write_len(v.length) + v.each do |k, e| + w.write_string(k) + w.write_object(e, Widget) + end + end + + # @api private + # Unpacks `{string:Widget}` from the value-buffer wire format. + def self._wv_read_map_string_widget(r) + Array.new(r.read_len) { [r.read_string, r.read_object(Widget)] }.to_h + end + + # @api private + # Packs `[Widget?]` into the value-buffer wire format. + def self._wv_write_list_opt_widget(w, v) + w.write_len(v.length) + v.each { |e| _wv_write_opt_widget(w, e) } + end + + # @api private + # Unpacks `[Widget?]` from the value-buffer wire format. + def self._wv_read_list_opt_widget(r) + Array.new(r.read_len) { _wv_read_opt_widget(r) } + end + + # @api private + # Packs `Widget?` into the value-buffer wire format. + def self._wv_write_opt_widget(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_object(v, Widget) + end + end + + # @api private + # Unpacks `Widget?` from the value-buffer wire format. + def self._wv_read_opt_widget(r) + r.read_flag ? r.read_object(Widget) : nil + end + + # @api private + # Packs `[Outer]` into the value-buffer wire format. + def self._wv_write_list_outer(w, v) + w.write_len(v.length) + v.each { |e| _wv_write_outer(w, e) } + end + + # @api private + # Unpacks `[Outer]` from the value-buffer wire format. + def self._wv_read_list_outer(r) + Array.new(r.read_len) { _wv_read_outer(r) } + end + + # @api private + # Packs `i64?` into the value-buffer wire format. + def self._wv_write_opt_i64(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_i64(v) + end + end + + # @api private + # Unpacks `i64?` from the value-buffer wire format. + def self._wv_read_opt_i64(r) + r.read_flag ? r.read_i64 : nil + end + + # @api private + # Packs `[Leaf]` into the value-buffer wire format. + def self._wv_write_list_leaf(w, v) + w.write_len(v.length) + v.each { |e| _wv_write_leaf(w, e) } + end + + # @api private + # Unpacks `[Leaf]` from the value-buffer wire format. + def self._wv_read_list_leaf(r) + Array.new(r.read_len) { _wv_read_leaf(r) } + end + + # @api private + # Packs `Inner?` into the value-buffer wire format. + def self._wv_write_opt_inner(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + _wv_write_inner(w, v) + end + end + + # @api private + # Unpacks `Inner?` from the value-buffer wire format. + def self._wv_read_opt_inner(r) + r.read_flag ? _wv_read_inner(r) : nil + end + + # @api private + # Packs `{string:Inner}` into the value-buffer wire format. + def self._wv_write_map_string_inner(w, v) + w.write_len(v.length) + v.each do |k, e| + w.write_string(k) + _wv_write_inner(w, e) + end + end + + # @api private + # Unpacks `{string:Inner}` from the value-buffer wire format. + def self._wv_read_map_string_inner(r) + Array.new(r.read_len) { [r.read_string, _wv_read_inner(r)] }.to_h + end + + # @api private + # Packs `[Mode]` into the value-buffer wire format. + def self._wv_write_list_mode(w, v) + w.write_len(v.length) + v.each { |e| w.write_i32(e) } + end + + # @api private + # Unpacks `[Mode]` from the value-buffer wire format. + def self._wv_read_list_mode(r) + Array.new(r.read_len) { r.read_i32 } + end + + # @api private + # Packs `[[i32]]` into the value-buffer wire format. + def self._wv_write_list_list_i32(w, v) + w.write_len(v.length) + v.each { |e| _wv_write_list_i32(w, e) } + end + + # @api private + # Unpacks `[[i32]]` from the value-buffer wire format. + def self._wv_read_list_list_i32(r) + Array.new(r.read_len) { _wv_read_list_i32(r) } + end + + # @api private + # Packs `[i32]` into the value-buffer wire format. + def self._wv_write_list_i32(w, v) + w.write_len(v.length) + v.each { |e| w.write_i32(e) } + end + + # @api private + # Unpacks `[i32]` from the value-buffer wire format. + def self._wv_read_list_i32(r) + Array.new(r.read_len) { r.read_i32 } + end + + # @api private + # Packs `[i64?]` into the value-buffer wire format. + def self._wv_write_list_opt_i64(w, v) + w.write_len(v.length) + v.each { |e| _wv_write_opt_i64(w, e) } + end + + # @api private + # Unpacks `[i64?]` from the value-buffer wire format. + def self._wv_read_list_opt_i64(r) + Array.new(r.read_len) { _wv_read_opt_i64(r) } + end + + # @api private + # Packs `{i64:string}` into the value-buffer wire format. + def self._wv_write_map_i64_string(w, v) + w.write_len(v.length) + v.each do |k, e| + w.write_i64(k) + w.write_string(e) + end + end + + # @api private + # Unpacks `{i64:string}` from the value-buffer wire format. + def self._wv_read_map_i64_string(r) + Array.new(r.read_len) { [r.read_i64, r.read_string] }.to_h + end end # END edge_cases.rb diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_runtime_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_runtime_rb.snap deleted file mode 100644 index 2ea694c4..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_edge_cases__lib_edge_cases_runtime_rb.snap +++ /dev/null @@ -1,705 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# frozen_string_literal: true - -require 'ffi' -require 'monitor' - -# The fixed runtime every generated EdgeCases binding builds on: the native -# library loader, the load-time ABI check, the error classes, the -# value-buffer reader and writer, cancel tokens, the handle table behind -# callback interfaces and async completions, and the object wrapper base. -module EdgeCases - extend FFI::Library - - # An explicit path in EDGE_CASES_LIBRARY wins. Otherwise a library bundled - # under lib/native/ (a platform gem) is preferred over the system search - # path. - _wv_override = ENV['EDGE_CASES_LIBRARY'] - if _wv_override && !_wv_override.empty? - ffi_lib _wv_override - else - _wv_name = - case FFI::Platform::OS - when /darwin/ then 'libedge_cases.dylib' - when /mswin|mingw|windows/ then 'edge_cases.dll' - else 'libedge_cases.so' - end - _wv_bundled = File.expand_path(File.join('..', 'native', _wv_name), __dir__) - ffi_lib(File.exist?(_wv_bundled) ? _wv_bundled : _wv_name) - end - - # The C ABI revision these bindings speak. Checked before anything else is - # attached, so a mismatched library fails at require time instead of - # misreading an error struct or a value buffer later. - ABI_VERSION = 3 - begin - attach_function :edge_cases_abi_version, [], :uint32 - rescue FFI::NotFoundError - raise LoadError, "the loaded edge_cases library does not export edge_cases_abi_version " \ - "(these bindings expect ABI revision #{ABI_VERSION})" - end - _wv_abi = edge_cases_abi_version - unless _wv_abi == ABI_VERSION - raise LoadError, "edge_cases ABI mismatch: these bindings expect revision #{ABI_VERSION} " \ - "but the loaded library reports revision #{_wv_abi}" - end - - # The `edge_cases_error` out-parameter every call reports through. - class ErrorStruct < FFI::Struct - layout :code, :int32, - :message, :pointer, - :payload_ptr, :pointer, - :payload_len, :size_t - end - - # Codes reserved by the runtime. Positive codes belong to a module's error - # domain; these negative codes are raised as a plain Error (or Cancelled) - # regardless of whether the function declares errors. - GENERIC_ERROR_CODE = -1 - PANIC_ERROR_CODE = -2 - MARSHAL_ERROR_CODE = -3 - # A callback-interface implementation raised; the message carries the - # exception text. - FOREIGN_ERROR_CODE = -4 - # The call was cancelled through its CancelToken. - CANCELLED_ERROR_CODE = -5 - - # The base class of every error the library raises. - class Error < StandardError - # The ABI error code: positive for a domain error, negative for a runtime - # failure. - attr_reader :code - - def initialize(code, message) - @code = code - super(message) - end - end - - # Raised when a cancellable call completes because its CancelToken fired. - class Cancelled < Error - def initialize(message = 'cancelled') - super(CANCELLED_ERROR_CODE, message) - end - end - - attach_function :edge_cases_error_set, [:pointer, :int32, :string], :void - attach_function :edge_cases_error_clear, [:pointer], :void - attach_function :edge_cases_error_free, [:pointer], :void - attach_function :edge_cases_free_bytes, [:pointer, :size_t], :void - attach_function :edge_cases_cancel_token_create, [], :pointer - attach_function :edge_cases_cancel_token_cancel, [:pointer], :void - attach_function :edge_cases_cancel_token_is_cancelled, [:pointer], :bool - attach_function :edge_cases_cancel_token_destroy, [:pointer], :void - - # @api private - # The error for a code outside any declared domain. - def self._wv_error(code, message) - return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE - - Error.new(code, message) - end - - # @api private - # Copies the code, message, and payload out of a reported error and clears - # it. Returns nil when the struct reports success. - def self._wv_take_error(err) - code = err[:code] - return nil if code.zero? - - msg_ptr = err[:message] - message = msg_ptr.null? ? '' : msg_ptr.read_string.force_encoding(Encoding::UTF_8) - payload_ptr = err[:payload_ptr] - payload = payload_ptr.null? ? nil : payload_ptr.read_string(err[:payload_len]) - edge_cases_error_clear(err.to_ptr) - [code, message, payload] - end - - # @api private - # Like _wv_take_error for the heap-boxed error an async completion - # receives; the box is released with edge_cases_error_free. - def self._wv_take_boxed_error(err_ptr) - return nil if err_ptr.null? - - taken = _wv_take_error(ErrorStruct.new(err_ptr)) - edge_cases_error_free(err_ptr) - taken - end - - # @api private - # Raises the generic error for a call that declares no error domain. - def self.check_error!(err) - taken = _wv_take_error(err) - return if taken.nil? - - raise _wv_error(taken[0], taken[1]) - end - - # @api private - # The UTF-8 bytes of a String argument, as a private binary copy the - # producer borrows for the call. - def self._wv_str(value) - value.to_s.encode(Encoding::UTF_8).b - end - - # @api private - # A private binary copy of a bytes argument. - def self._wv_bytes(value) - value.to_s.b - end - - # @api private - # Copies a borrowed (ptr, len) run without releasing it. - def self._wv_borrow_bytes(ptr, len) - ptr.null? || len.zero? ? ''.b : ptr.read_string(len) - end - - # @api private - # Copies a returned (ptr, len) allocation, then releases it. - def self._wv_take_bytes(ptr, len) - return ''.b if ptr.null? - - data = len.zero? ? ''.b : ptr.read_string(len) - edge_cases_free_bytes(ptr, len) - data - end - - # @api private - # Copies a returned UTF-8 string, then releases it. - def self._wv_take_string(ptr, len) - _wv_take_bytes(ptr, len).force_encoding(Encoding::UTF_8) - end - - # A cancellation signal for a cancellable async call. Pass one as the - # `cancel:` keyword, then call #cancel from any thread; the call raises - # Cancelled unless it already completed. A token may be shared by several - # calls and stays cancelled once cancelled. - class CancelToken - # @api private - # Owns the consumer's reference to the native token. - class Ptr < FFI::AutoPointer - def self.release(ptr) - EdgeCases.edge_cases_cancel_token_destroy(ptr) - end - end - - def initialize - @ptr = Ptr.new(EdgeCases.edge_cases_cancel_token_create) - end - - # Requests cancellation. Idempotent. - def cancel - EdgeCases.edge_cases_cancel_token_cancel(@ptr) unless @ptr.nil? - nil - end - - # Whether #cancel has been called. - def cancelled? - !@ptr.nil? && EdgeCases.edge_cases_cancel_token_is_cancelled(@ptr) - end - - # Releases the native token now rather than at GC time. Calls already - # launched with it keep their own reference. - def close - return if @ptr.nil? - - @ptr.free - @ptr = nil - end - - # @api private - def _wv_ptr - raise Error.new(GENERIC_ERROR_CODE, 'CancelToken used after close') if @ptr.nil? - - @ptr - end - end - - # @api private - # Appends values in the value-buffer wire format: little-endian, packed, - # no alignment. - class WvBufferWriter - def initialize - @buf = +''.b - @objects = [] - end - - # Reserves an object token. The strong reference it carries is minted - # by #seal, just before the buffer is handed to the producer, so an - # encoding abandoned on the way never strands a reference. - def write_object(object) - raise TypeError, "expected a wrapper object, got #{object.class}" unless object.is_a?(WvObject) - - @objects << [@buf.bytesize, object] - write_u64(0) - end - - # Mints the reference each reserved object token carries, recording it - # in `minted`, and returns the finished encoding. - def seal(minted) - @objects.each do |offset, object| - ptr = object._wv_clone_ptr - minted << [object.class, ptr] - @buf[offset, 8] = [ptr.address].pack('Q<') - end - @buf - end - - # Every append is a binary String, so the buffer stays binary and byte - # offsets index it directly. - def write_bool(value) - @buf << [value ? 1 : 0].pack('C') - end - - def write_flag(value) - write_bool(value) - end - - def write_i8(value) - @buf << [value].pack('c') - end - - def write_u8(value) - @buf << [value].pack('C') - end - - def write_i16(value) - @buf << [value].pack('s<') - end - - def write_u16(value) - @buf << [value].pack('S<') - end - - def write_i32(value) - @buf << [value].pack('l<') - end - - def write_u32(value) - @buf << [value].pack('L<') - end - - def write_len(value) - write_u32(value) - end - - def write_i64(value) - @buf << [value].pack('q<') - end - - def write_u64(value) - @buf << [value].pack('Q<') - end - - def write_f32(value) - @buf << [value].pack('e') - end - - def write_f64(value) - @buf << [value].pack('E') - end - - def write_string(value) - write_bytes(value.to_s.encode(Encoding::UTF_8)) - end - - def write_bytes(value) - bytes = value.to_s.b - write_u32(bytes.bytesize) - @buf << bytes - end - end - - # @api private - # Reads values in the value-buffer wire format, raising Error on any - # malformed buffer. - class WvBufferReader - def initialize(data) - @data = data.to_s.b - @pos = 0 - end - - def take(count, what) - if @pos + count > @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, "malformed value buffer: truncated #{what}") - end - - bytes = @data.byteslice(@pos, count) - @pos += count - bytes - end - - def read_bool - byte = take(1, 'bool').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: bool byte out of range') if byte > 1 - - byte == 1 - end - - def read_flag - byte = take(1, 'option flag').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: option flag out of range') if byte > 1 - - byte == 1 - end - - def read_i8 - take(1, 'i8').unpack1('c') - end - - def read_u8 - take(1, 'u8').unpack1('C') - end - - def read_i16 - take(2, 'i16').unpack1('s<') - end - - def read_u16 - take(2, 'u16').unpack1('S<') - end - - def read_i32 - take(4, 'i32').unpack1('l<') - end - - def read_u32 - take(4, 'u32').unpack1('L<') - end - - # A u32 element count. Counts are not checked against the remaining - # bytes, since an element can encode to zero bytes; a short buffer still - # fails on the first truncated read. - def read_len - read_u32 - end - - def read_i64 - take(8, 'i64').unpack1('q<') - end - - def read_u64 - take(8, 'u64').unpack1('Q<') - end - - def read_f32 - take(4, 'f32').unpack1('e') - end - - def read_f64 - take(8, 'f64').unpack1('E') - end - - def read_string - text = read_bytes.force_encoding(Encoding::UTF_8) - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: string is not valid UTF-8') unless text.valid_encoding? - - text - end - - def read_bytes - take(read_u32, 'byte run') - end - - # An object token: a non-zero u64 carrying one strong reference the - # caller adopts into a wrapper. - def read_object_token - addr = read_u64 - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: null object token') if addr.zero? - - FFI::Pointer.new(addr) - end - - def expect_end! - return if @pos == @data.bytesize - - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: trailing bytes after value') - end - end - - # @api private - # Seals the writers of one call's object-carrying buffers (see - # WvBufferWriter#seal) and returns their encodings. If any reference can't - # be minted (a closed wrapper), the ones already minted are released before - # the error propagates. The producer adopts every minted reference once the - # call is made. - def self._wv_seal(*writers) - minted = [] - writers.map { |writer| writer.seal(minted) } - rescue Exception # rubocop:disable Lint/RescueException - minted.each { |cls, ptr| cls::WV_PTR.release(ptr) } - raise - end - - # @api private - # Decodes a returned value buffer: copies and releases it, yields a reader - # to the block, and checks that the block consumed every byte. - def self._wv_decode(ptr, len) - reader = WvBufferReader.new(_wv_take_bytes(ptr, len)) - value = yield reader - reader.expect_end! - value - end - - # Ruby objects the producer refers to by an integer key: callback-interface - # implementations (the key is the vtable `ctx`) and pending async - # completions (the key is the completion `context`). The producer never - # sees a Ruby object address. A Monitor rather than a Mutex, so a - # finalizer that runs while this thread holds the lock can't deadlock it. - @wv_handles = {} - @wv_next_handle = 0 - @wv_handles_lock = Monitor.new - - # @api private - def self._wv_handle_put(value) - @wv_handles_lock.synchronize do - @wv_next_handle += 1 - @wv_handles[@wv_next_handle] = value - FFI::Pointer.new(@wv_next_handle) - end - end - - # @api private - def self._wv_handle_get(key) - @wv_handles_lock.synchronize { @wv_handles[key.address] } - end - - # @api private - def self._wv_handle_take(key) - @wv_handles_lock.synchronize { @wv_handles.delete(key.address) } - end - - # @api private - # Registers a callback-interface implementation and returns its `ctx`. - # Never raises, so nothing can strand the registration between this and - # the call that hands it to the producer (an unusable implementation fails - # when the producer calls it, with FOREIGN_ERROR_CODE). - def self._wv_cb_register(impl) - _wv_handle_put([:callback, impl]) - end - - # @api private - # Resolves a trampoline's `ctx` back to the registered implementation. - def self._wv_cb_lookup(ctx) - kind, impl = _wv_handle_get(ctx) - raise Error.new(FOREIGN_ERROR_CODE, 'callback context is not registered') unless kind == :callback - - impl - end - - # @api private - # Drops an implementation when the producer releases its last reference. - def self._wv_cb_free(ctx) - _wv_handle_take(ctx) - nil - end - - # @api private - # Reports a callback implementation's exception through `out_err`. The - # producer copies the message. - def self._wv_cb_fail(out_err, exception) - message = exception.message.to_s - message = exception.class.name if message.empty? - message = message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") - edge_cases_error_set(out_err, FOREIGN_ERROR_CODE, message) - nil - end - - # @api private - # Registers a pending async call; returns its completion context and the - # queue its completion is delivered on. - def self._wv_async_begin - queue = Queue.new - [_wv_handle_put([:async, queue]), queue] - end - - # @api private - # The queue of the pending call a completion belongs to. Each context - # completes exactly once, so its entry is removed here. - def self._wv_async_finish(ctx) - kind, queue = _wv_handle_take(ctx) - kind == :async ? queue : Queue.new - end - - # @api private - # Blocks until a pending call completes, raising its error. If the wait is - # interrupted (Thread#raise, Thread#kill, Timeout), the call's token is - # cancelled before the interruption propagates. - def self._wv_async_wait(queue, token = nil) - value = - begin - queue.pop - rescue Exception # rubocop:disable Lint/RescueException - token&.cancel - raise - end - raise value if value.is_a?(Exception) - - value - end - - # @api private - # The base of every interface wrapper. A wrapper owns one strong reference - # to its native object and releases it exactly once, from #close or (as a - # backstop) from the GC finalizer of its FFI::AutoPointer. Calls borrow the - # pointer through #_wv_call, so a #close that races an in-flight call on - # another thread only releases the reference once the call returns. - class WvObject - # @api private - # Adopts one strong reference the producer handed over. - def self._from_ptr(ptr) - obj = allocate - obj._wv_init(ptr) - obj - end - - # @api private - def _wv_init(ptr) - @wv_handle = self.class::WV_PTR.new(ptr) - @wv_lock = Mutex.new - @wv_calls = 0 - @wv_closing = false - end - - # The native object pointer (borrowed; valid until #close). - def handle - handle = @wv_handle - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if handle.nil? || @wv_closing - - handle - end - - # Whether #close has released this wrapper's reference. - def closed? - @wv_handle.nil? || @wv_closing - end - - # Releases this wrapper's reference now rather than at GC time. - # Idempotent. The object itself lives until its last reference anywhere - # (another wrapper, a record field, the producer) goes. - def close - handle = @wv_lock.synchronize do - next nil if @wv_handle.nil? || @wv_closing - - if @wv_calls.zero? - taken = @wv_handle - @wv_handle = nil - taken - else - @wv_closing = true - nil - end - end - handle&.free - nil - end - - # @api private - # Yields the native pointer, keeping the reference alive until the block - # returns. - def _wv_call - handle = @wv_lock.synchronize do - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if @wv_handle.nil? || @wv_closing - - @wv_calls += 1 - @wv_handle - end - begin - yield handle - ensure - _wv_leave - end - end - - # @api private - # Mints a new strong reference the caller owns (an object written into a - # value buffer carries its own reference). - def _wv_clone_ptr - _wv_call { |ptr| self.class._wv_clone(ptr) } - end - - # dup and clone produce an independent wrapper with its own reference to - # the same object. - def initialize_copy(other) - super - _wv_init(other._wv_clone_ptr) - end - - private - - def _wv_leave - release = @wv_lock.synchronize do - @wv_calls -= 1 - next nil unless @wv_closing && @wv_calls.zero? - - taken = @wv_handle - @wv_handle = nil - @wv_closing = false - taken - end - release&.free - end - end - - # @api private - # Yields the native pointers of `objects` (NULL for nil), each kept alive - # until the block returns. - def self._wv_pin(*objects, &block) - _wv_pin_from(objects, 0, [], &block) - end - - # @api private - def self._wv_pin_from(objects, index, pointers, &block) - return yield(*pointers) if index == objects.length - - object = objects[index] - if object.nil? - _wv_pin_from(objects, index + 1, pointers + [FFI::Pointer::NULL], &block) - else - object._wv_call { |ptr| _wv_pin_from(objects, index + 1, pointers + [ptr], &block) } - end - end - - # @api private - # Checks each top-level module's contract checksum (`module name => - # [checksum function, expected value]`) against the library, so bindings - # generated from a different API fail at require time. - def self._wv_check_contract!(checksums) - checksums.each do |mod, (symbol, expected)| - begin - attach_function symbol, [], :uint64 - rescue FFI::NotFoundError - raise LoadError, "the loaded edge_cases library has no module '#{mod}' (#{symbol} is missing)" - end - actual = public_send(symbol) - next if actual == expected - - raise LoadError, "edge_cases module '#{mod}' does not match these bindings: expected contract " \ - "checksum 0x#{expected.to_s(16)}, the library reports 0x#{actual.to_s(16)}; " \ - 'regenerate the bindings' - end - end - - # @api private - # Live native resources of one kind (0 objects, 1 callbacks, 2 iterators, - # 3 cancel tokens, 4 byte allocations). Only libraries built with the - # `leak-check` feature export the counter. - def self._wv_debug_live(kind) - unless respond_to?(:edge_cases_debug_live) - begin - attach_function :edge_cases_debug_live, [:int32], :uint64 - rescue FFI::NotFoundError - raise Error.new(GENERIC_ERROR_CODE, 'edge_cases was built without the leak-check feature') - end - end - edge_cases_debug_live(kind) - end -end - -# END runtime.rb diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__README_md.snap index 6f0e6f67..a338208c 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__README_md.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # kitchen_sink (Ruby) diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__kitchen_sink_gemspec.snap b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__kitchen_sink_gemspec.snap index 7d8d8fcf..4fbc646f 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__kitchen_sink_gemspec.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__kitchen_sink_gemspec.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate Gem::Specification.new do |s| s.name = 'kitchen_sink' diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_rb.snap index d08bb7c9..0cdde38e 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_rb.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_rb.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate # frozen_string_literal: true @@ -11,11 +11,47 @@ require_relative 'kitchen_sink/runtime' # Ruby bindings for kitchen_sink. module KitchenSink - # Contract checksums of the top-level modules these bindings were - # generated from; a library built from a different API fails to load. + # The declarations these bindings were generated with, per top-level + # module, as [id, hash, path]. Loading fails, naming the path, unless + # the library's contract table holds each one with an equal hash. _wv_check_contract!( - 'shared' => [:kitchen_sink_shared_checksum, 0x42c4ce2c8d0af052], - 'kitchen' => [:kitchen_sink_kitchen_checksum, 0xd83da75b24b545ff], + kitchen_sink_shared_contract: [ + [0x709e234caa962446, 0x3894f2ca452e9f98, 'shared.ping'], + [0xcfb306fa5801ee5b, 0x1cbd9e10983cac2c, 'shared.Token'], + ], + kitchen_sink_kitchen_contract: [ + [0x01d4f09bbddc986f, 0x29eb88fecc66544d, 'kitchen.Gadget.new'], + [0x10472d8fe748c2ee, 0xb265832b6a759fc4, 'kitchen.maybe_gadget'], + [0x11368eec07a74078, 0xfa049f78b77e146a, 'kitchen.gadgets'], + [0x137f08a29cde6021, 0xda8de7aeb2dbb8a3, 'kitchen.stream_gadgets'], + [0x2111b54980826dd7, 0x452783f3e6ca6e1d, 'kitchen.do_cancellable'], + [0x29dddbc60bf8d7db, 0x452a3e573d57f6db, 'kitchen.Priority'], + [0x2b227003b74a0d16, 0x12332957470e9782, 'kitchen.maybe_subscribe'], + [0x2dd66cb0064f0494, 0x60197363fefd92dd, 'kitchen.Item'], + [0x33c265bbf079b0e9, 0xc3dcd0927e6c1deb, 'kitchen.u32_id'], + [0x33e6381d01660dfe, 0x077c63c31fd7f90a, 'kitchen.KitchenErrors'], + [0x3f5fc99d73504b76, 0x2df88d710f1ea10e, 'kitchen.Gadget.poke'], + [0x4c5764cf109528d1, 0x2a65529af4338a58, 'kitchen.maybe_item'], + [0x4de17dfb93fd494d, 0xba0ad0dd52ed74af, 'kitchen.i32_id'], + [0x504e95b7f7e65570, 0x01b3e8e0c9a10e23, 'kitchen.stream_items'], + [0x68cdb9a4bd3e3dd5, 0xc678b9164348d497, 'kitchen.bool_id'], + [0x69e7cd703d16eb85, 0xf3b28dd22ae9e689, 'kitchen.do_async'], + [0x70865079ce41be82, 0xbe3b69464672af46, 'kitchen.Gadget.describe'], + [0x71c5a1116cb7c49a, 0xcba1837108a89b84, 'kitchen.echo_bytes'], + [0x735ee1e0049ee91a, 0xca74320bc53eddab, 'kitchen.echo_string'], + [0x76d7fc58f4e41d84, 0x7373a0e47d30e690, 'kitchen.i64_id'], + [0x79abfe1aa960fd6a, 0xbca7a9e5eb26f3a6, 'kitchen.nested.hello'], + [0x7d3dd7d5cf6749e3, 0xa192f5a7298adb39, 'kitchen.new_op'], + [0x89a5e664bc25c67b, 0x5764a95e81ecfd31, 'kitchen.f64_id'], + [0x8cfdbdcecac5f7f1, 0x07e272f09c41fffc, 'kitchen.Gadget.version'], + [0x974b9f3e823c1738, 0xb81b91e975695f27, 'kitchen.cross_module_token'], + [0xaca0655fac15d230, 0x11c4d2b28abe07bb, 'kitchen.list_items'], + [0xc883b4afae20543e, 0xc40560e5411ff392, 'kitchen.legacy_op'], + [0xf2f7753e6e1dffc2, 0xc9047a95bdf49e93, 'kitchen.get_attrs'], + [0xfae25d3391922ead, 0x1c7f95739f3210b8, 'kitchen.subscribe'], + [0xfd0059b05cb2fcfb, 0x248bf8404a5823b4, 'kitchen.Gadget'], + [0xfed9c551fb9fd2d6, 0x59aec980ef3f3291, 'kitchen.ReadyListener'], + ], ) # === Module: shared === @@ -33,10 +69,9 @@ module KitchenSink # Structural equality over every field. def ==(other) - return false unless other.is_a?(Token) - return false unless id == other.id - return false unless label == other.label - true + other.is_a?(Token) && + id == other.id && + label == other.label end end @@ -50,19 +85,18 @@ module KitchenSink # @api private # Unpacks a Token from the value-buffer wire format. def self._wv_read_token(r) - _wv_id = r.read_i64 - _wv_label = r.read_string - Token.new(id: _wv_id, label: _wv_label) + Token.new(id: r.read_i64, label: r.read_string) end attach_function :kitchen_sink_shared_ping, [:pointer, :pointer], :pointer, blocking: true # Trivial cross-module helper + # @return [String] def self.ping err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_shared_ping(out_len, err) - check_error!(err) + _wv_check!(err) _wv_take_string(result, out_len.read(:size_t)) end @@ -74,8 +108,8 @@ module KitchenSink class NotFound < KitchenErrorsError CODE = 1 - def initialize(message = 'Item not found') - super(1, message) + def initialize(message = nil) + super(CODE, message || 'Item not found') end end @@ -83,31 +117,49 @@ module KitchenSink class InvalidInput < KitchenErrorsError CODE = 2 - def initialize(message = 'Invalid input') - super(2, message) + def initialize(message = nil) + super(CODE, message || 'Invalid input') end end end - # Maps each ABI code of the KitchenErrorsError domain to its error class. - KITCHEN_ERRORS_ERROR_CODES = { - 1 => KitchenErrorsError::NotFound, - 2 => KitchenErrorsError::InvalidInput, - }.freeze - - # Builds the KitchenErrorsError subclass matching `code`, decoding any payload - # fields the code declares, or a generic Error for codes outside - # the domain (panics, marshalling, callback failures). - def self.kitchen_errors_error_from(code, message, payload = nil) - cls = KITCHEN_ERRORS_ERROR_CODES[code] - return _wv_error(code, message) if cls.nil? - message.empty? ? cls.new : cls.new(message) + # @api private + # The KitchenErrorsError for a domain `code`, with its payload fields decoded; + # the root Error (or Cancelled) for any other code. + def self._wv_kitchen_errors_error_from(code, message, payload = nil) + message = nil if message.empty? + r = WvBufferReader.new(payload) + error = + case code + when 1 + KitchenErrorsError::NotFound.new(message) + when 2 + KitchenErrorsError::InvalidInput.new(message) + else + return _wv_error(code, message.to_s) + end + r.expect_end! + error end - # Raises the typed KitchenErrorsError for a non-zero error slot. - def self.check_kitchen_errors_error!(err) + # @api private + # Raises the KitchenErrorsError for a non-zero error slot. + def self._wv_check_kitchen_errors_error!(err) taken = _wv_take_error(err) - raise kitchen_errors_error_from(*taken) if taken + raise _wv_kitchen_errors_error_from(*taken) unless taken.nil? + end + + # @api private + # The code and value-buffer payload of a KitchenErrorsError a callback + # implementation raised, or nil for an error of no declared code. + def self._wv_kitchen_errors_error_payload(error) + w = WvBufferWriter.new + code = + case error + when KitchenErrorsError::NotFound then 1 + when KitchenErrorsError::InvalidInput then 2 + end + code && [code, w.bytes] end # Task priority levels @@ -155,7 +207,21 @@ module KitchenSink # An optional object inside a record attr_reader :gadget - def initialize(id:, name:, count:, enabled:, ratio:, nick:, payload:, tags:, attrs:, parent:, token:, priority:, gadget:) + def initialize( + id:, + name:, + count:, + enabled:, + ratio:, + nick:, + payload:, + tags:, + attrs:, + parent:, + token:, + priority:, + gadget: + ) @id = id @name = name @count = count @@ -173,26 +239,25 @@ module KitchenSink # Structural equality over every field. def ==(other) - return false unless other.is_a?(Item) - return false unless id == other.id - return false unless name == other.name - return false unless count == other.count - return false unless enabled == other.enabled - return false unless ratio == other.ratio - return false unless nick == other.nick - return false unless payload == other.payload - return false unless tags == other.tags - return false unless attrs == other.attrs - return false unless parent == other.parent - return false unless token == other.token - return false unless priority == other.priority - return false unless gadget == other.gadget - true + other.is_a?(Item) && + id == other.id && + name == other.name && + count == other.count && + enabled == other.enabled && + ratio == other.ratio && + nick == other.nick && + payload == other.payload && + tags == other.tags && + attrs == other.attrs && + parent == other.parent && + token == other.token && + priority == other.priority && + gadget == other.gadget end end # @api private - # Packs a Item into the value-buffer wire format. + # Packs an Item into the value-buffer wire format. def self._wv_write_item(w, v) w.write_i64(v.id) w.write_string(v.name) @@ -201,68 +266,36 @@ module KitchenSink w.write_f64(v.ratio) w.write_u32(v.nick) w.write_bytes(v.payload) - w.write_len(v.tags.length) - v.tags.each do |_wv_e0| - w.write_string(_wv_e0) - end - w.write_len(v.attrs.length) - v.attrs.each do |_wv_k0, _wv_v0| - w.write_string(_wv_k0) - w.write_string(_wv_v0) - end - if v.parent.nil? - w.write_flag(false) - else - w.write_flag(true) - w.write_i64(v.parent) - end + _wv_write_list_string(w, v.tags) + _wv_write_map_string_string(w, v.attrs) + _wv_write_opt_i64(w, v.parent) _wv_write_token(w, v.token) w.write_i32(v.priority) - if v.gadget.nil? - w.write_flag(false) - else - w.write_flag(true) - w.write_object(v.gadget) - end + _wv_write_opt_gadget(w, v.gadget) end # @api private - # Unpacks a Item from the value-buffer wire format. + # Unpacks an Item from the value-buffer wire format. def self._wv_read_item(r) - _wv_id = r.read_i64 - _wv_name = r.read_string - _wv_count = r.read_i32 - _wv_enabled = r.read_bool - _wv_ratio = r.read_f64 - _wv_nick = r.read_u32 - _wv_payload = r.read_bytes - _wv_tags = Array.new(r.read_len) do - _wv_e0 = r.read_string - _wv_e0 - end - _wv_attrs = {} - r.read_len.times do - _wv_k0 = r.read_string - _wv_v0 = r.read_string - _wv_attrs[_wv_k0] = _wv_v0 - end - if r.read_flag - _wv_parent = r.read_i64 - else - _wv_parent = nil - end - _wv_token = _wv_read_token(r) - _wv_priority = r.read_i32 - if r.read_flag - _wv_gadget = Gadget._from_ptr(r.read_object_token) - else - _wv_gadget = nil - end - Item.new(id: _wv_id, name: _wv_name, count: _wv_count, enabled: _wv_enabled, ratio: _wv_ratio, nick: _wv_nick, payload: _wv_payload, tags: _wv_tags, attrs: _wv_attrs, parent: _wv_parent, token: _wv_token, priority: _wv_priority, gadget: _wv_gadget) + Item.new( + id: r.read_i64, + name: r.read_string, + count: r.read_i32, + enabled: r.read_bool, + ratio: r.read_f64, + nick: r.read_u32, + payload: r.read_bytes, + tags: _wv_read_list_string(r), + attrs: _wv_read_map_string_string(r), + parent: _wv_read_opt_i64(r), + token: _wv_read_token(r), + priority: r.read_i32, + gadget: _wv_read_opt_gadget(r), + ) end attach_function :kitchen_sink_kitchen_Gadget_clone, [:pointer], :pointer - attach_function :kitchen_sink_kitchen_Gadget_destroy, [:pointer], :void, blocking: true + attach_function :kitchen_sink_kitchen_Gadget_destroy, [:pointer], :void attach_function :kitchen_sink_kitchen_Gadget_new, [:int64, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_Gadget_describe, [:pointer, :pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_Gadget_poke, [:pointer, :int32, :pointer], :int32, blocking: true @@ -275,19 +308,23 @@ module KitchenSink attach_function :kitchen_sink_kitchen_echo_string, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_echo_bytes, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_subscribe, [:pointer, :pointer, :pointer], :int32, blocking: true + attach_function :kitchen_sink_kitchen_maybe_subscribe, [:pointer, :pointer, :pointer], :bool, blocking: true attach_function :kitchen_sink_kitchen_maybe_gadget, [:pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_gadgets, [:pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_stream_gadgets, [:pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_StreamGadgetsIterator_next, [:pointer, :pointer, :pointer], :int32, blocking: true - attach_function :kitchen_sink_kitchen_StreamGadgetsIterator_destroy, [:pointer], :void, blocking: true + attach_function :kitchen_sink_kitchen_StreamGadgetsIterator_destroy, [:pointer], :void attach_function :kitchen_sink_kitchen_maybe_item, [:int64, :pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_list_items, [:pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_get_attrs, [:pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_stream_items, [:pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_StreamItemsIterator_next, [:pointer, :pointer, :pointer, :pointer], :int32, blocking: true - attach_function :kitchen_sink_kitchen_StreamItemsIterator_destroy, [:pointer], :void, blocking: true + attach_function :kitchen_sink_kitchen_StreamItemsIterator_destroy, [:pointer], :void attach_function :kitchen_sink_kitchen_cross_module_token, [:pointer, :pointer], :pointer, blocking: true attach_function :kitchen_sink_kitchen_do_async, [:pointer, :size_t, :pointer, :pointer], :void, blocking: true + attach_function :kitchen_sink_kitchen_do_cancellable, [:pointer, :size_t, :pointer, :pointer, :pointer], :void, blocking: true + attach_function :kitchen_sink_kitchen_legacy_op, [:pointer], :int32, blocking: true + attach_function :kitchen_sink_kitchen_new_op, [:pointer], :int32, blocking: true # @api private # Completion trampoline for do_async. @@ -296,7 +333,7 @@ module KitchenSink begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else value = _wv_take_string(result_ptr, result_len) queue << value @@ -306,8 +343,6 @@ module KitchenSink end end - attach_function :kitchen_sink_kitchen_do_cancellable, [:pointer, :size_t, :pointer, :pointer, :pointer], :void, blocking: true - # @api private # Completion trampoline for do_cancellable. KITCHEN_SINK_KITCHEN_DO_CANCELLABLE_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :size_t]) do |ctx, err, result_ptr, result_len| @@ -315,7 +350,7 @@ module KitchenSink begin taken = _wv_take_boxed_error(err) if taken - queue << _wv_error(taken[0], taken[1]) + queue << _wv_trap(*taken) else value = _wv_take_string(result_ptr, result_len) queue << value @@ -325,14 +360,11 @@ module KitchenSink end end - attach_function :kitchen_sink_kitchen_legacy_op, [:pointer], :int32, blocking: true - attach_function :kitchen_sink_kitchen_new_op, [:pointer], :int32, blocking: true - # Consumer-implemented observer of ready items # # Consumer-implemented callback interface. Any object responding to the # methods below is accepted wherever a ReadyListener is expected; include - # this module to inherit NotImplementedError defaults. The producer + # this module to inherit NotImplementedError defaults. The library # may call the methods from any thread until it releases the # implementation. module ReadyListener @@ -342,64 +374,117 @@ module KitchenSink end # Receives the item itself and says whether to keep listening - # @return [Object] a bool + # @return [Boolean] def on_item(item, gadget) raise NotImplementedError, "#{self.class}#on_item is not implemented" end + + # A display label, failing with a kitchen error + # @return [String] + # @raise [KitchenErrorsError] reported to the library with its code and fields + def label + raise NotImplementedError, "#{self.class}#label is not implemented" + end + + # The most recent item the listener kept, if any + # @return [Item, nil] + def latest + raise NotImplementedError, "#{self.class}#latest is not implemented" + end + + # The listener's favorite gadget + # @return [Gadget] + def favorite + raise NotImplementedError, "#{self.class}#favorite is not implemented" + end end # @api private - # The C vtable layout the producer calls a ReadyListener through. + # The C vtable layout the library calls a ReadyListener through. class WvReadyListenerVtable < FFI::Struct - layout :on_ready, :pointer, + layout :wv_size, :uint32, + :wv_flags, :uint32, + :wv_free, :pointer, + :on_ready, :pointer, :on_item, :pointer, - :free, :pointer + :label, :pointer, + :latest, :pointer, + :favorite, :pointer + end + + # @api private + # Releases a ReadyListener implementation when the library drops its last + # reference. May run on any library thread. + WV_READY_LISTENER_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| + _wv_cb_free(ctx) end # @api private # Trampoline for ReadyListener#on_ready. - WV_READY_LISTENER_ON_READY = FFI::Function.new(:void, [:pointer, :int32, :pointer, :size_t, :pointer]) do |ctx, code, msg_ptr, msg_len, out_err| - begin - code_v = code - msg_v = _wv_borrow_bytes(msg_ptr, msg_len).force_encoding(Encoding::UTF_8) - impl = _wv_cb_lookup(ctx) - impl.on_ready(code_v, msg_v) - nil - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - nil - end + WV_READY_LISTENER_ON_READY = FFI::Function.new(:void, [:pointer, :int32, :pointer, :size_t, :pointer]) do |wv_ctx, code, msg_ptr, msg_len, wv_err| + msg = _wv_borrow_string(msg_ptr, msg_len) + _wv_cb_lookup(wv_ctx).on_ready(code, msg) + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil end # @api private # Trampoline for ReadyListener#on_item. - WV_READY_LISTENER_ON_ITEM = FFI::Function.new(:bool, [:pointer, :pointer, :size_t, :pointer, :pointer]) do |ctx, item_ptr, item_len, gadget, out_err| - begin - gadget_v = Gadget._from_ptr(gadget) - item_r = WvBufferReader.new(_wv_borrow_bytes(item_ptr, item_len)) - item_v = _wv_read_item(item_r) - item_r.expect_end! - impl = _wv_cb_lookup(ctx) - impl.on_item(item_v, gadget_v) ? true : false - rescue Exception => e # rubocop:disable Lint/RescueException - _wv_cb_fail(out_err, e) - false - end + WV_READY_LISTENER_ON_ITEM = FFI::Function.new(:bool, [:pointer, :pointer, :size_t, :pointer, :pointer]) do |wv_ctx, item_ptr, item_len, gadget_ptr, wv_err| + gadget = Gadget._from_ptr(gadget_ptr) + item = _wv_decode_borrowed(item_ptr, item_len) { |r| _wv_read_item(r) } + _wv_cb_lookup(wv_ctx).on_item(item, gadget) ? true : false + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + false end # @api private - # Releases a ReadyListener implementation when the producer drops its last - # reference. - WV_READY_LISTENER_FREE = FFI::Function.new(:void, [:pointer]) do |ctx| - _wv_cb_free(ctx) + # Trampoline for ReadyListener#label. + WV_READY_LISTENER_LABEL = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :pointer]) do |wv_ctx, wv_out_ptr, wv_out_len, wv_err| + _wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_str(_wv_cb_lookup(wv_ctx).label)) + rescue KitchenErrorsError => e + _wv_cb_throw(wv_err, e) { _wv_kitchen_errors_error_payload(e) } + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil + end + + # @api private + # Trampoline for ReadyListener#latest. + WV_READY_LISTENER_LATEST = FFI::Function.new(:void, [:pointer, :pointer, :pointer, :pointer]) do |wv_ctx, wv_out_ptr, wv_out_len, wv_err| + result = _wv_cb_lookup(wv_ctx).latest + writer = WvBufferWriter.new + _wv_write_opt_item(writer, result) + _wv_cb_return_bytes(wv_out_ptr, wv_out_len, _wv_seal(writer).first) + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + nil + end + + # @api private + # Trampoline for ReadyListener#favorite. + WV_READY_LISTENER_FAVORITE = FFI::Function.new(:pointer, [:pointer, :pointer]) do |wv_ctx, wv_err| + _wv_cb_return_object(_wv_cb_lookup(wv_ctx).favorite, Gadget) + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(wv_err, e) + FFI::Pointer::NULL end # The one process-wide ReadyListener vtable every registration hands the - # producer; its entries live for the process lifetime. + # library; its entries live for the process lifetime. WV_READY_LISTENER_VTABLE = WvReadyListenerVtable.new + WV_READY_LISTENER_VTABLE[:wv_size] = WvReadyListenerVtable.size + WV_READY_LISTENER_VTABLE[:wv_flags] = 0 + WV_READY_LISTENER_VTABLE[:wv_free] = WV_READY_LISTENER_FREE WV_READY_LISTENER_VTABLE[:on_ready] = WV_READY_LISTENER_ON_READY WV_READY_LISTENER_VTABLE[:on_item] = WV_READY_LISTENER_ON_ITEM - WV_READY_LISTENER_VTABLE[:free] = WV_READY_LISTENER_FREE + WV_READY_LISTENER_VTABLE[:label] = WV_READY_LISTENER_LABEL + WV_READY_LISTENER_VTABLE[:latest] = WV_READY_LISTENER_LATEST + WV_READY_LISTENER_VTABLE[:favorite] = WV_READY_LISTENER_FAVORITE # @api private # Owns one strong reference to a Gadget; releases it exactly once. @@ -422,148 +507,166 @@ module KitchenSink def initialize(id) err = ErrorStruct.new result = KitchenSink.kitchen_sink_kitchen_Gadget_new(id, err) - KitchenSink.check_error!(err) - raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if result.null? - _wv_init(result) + KitchenSink._wv_check!(err) + _wv_init(KitchenSink._wv_nonnull(result)) end # Render the gadget as a human-readable string + # @return [String] def describe KitchenSink._wv_pin(self) do |_wv_self| err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = KitchenSink.kitchen_sink_kitchen_Gadget_describe(_wv_self, out_len, err) - KitchenSink.check_error!(err) + KitchenSink._wv_check!(err) KitchenSink._wv_take_string(result, out_len.read(:size_t)) end end # Poke the gadget a number of times, failing on a negative count + # @return [Integer] def poke(times) KitchenSink._wv_pin(self) do |_wv_self| err = ErrorStruct.new result = KitchenSink.kitchen_sink_kitchen_Gadget_poke(_wv_self, times, err) - KitchenSink.check_kitchen_errors_error!(err) + KitchenSink._wv_check_kitchen_errors_error!(err) result end end # The gadget subsystem version string + # @return [String] def self.version err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = KitchenSink.kitchen_sink_kitchen_Gadget_version(out_len, err) - KitchenSink.check_error!(err) + KitchenSink._wv_check!(err) KitchenSink._wv_take_string(result, out_len.read(:size_t)) end end # Identity for bool + # @return [Boolean] def self.bool_id(x) err = ErrorStruct.new result = kitchen_sink_kitchen_bool_id((x ? true : false), err) - check_error!(err) + _wv_check!(err) result end + # @return [Integer] def self.i32_id(x) err = ErrorStruct.new result = kitchen_sink_kitchen_i32_id(x, err) - check_error!(err) + _wv_check!(err) result end + # @return [Integer] def self.u32_id(x) err = ErrorStruct.new result = kitchen_sink_kitchen_u32_id(x, err) - check_error!(err) + _wv_check!(err) result end + # @return [Integer] def self.i64_id(x) err = ErrorStruct.new result = kitchen_sink_kitchen_i64_id(x, err) - check_error!(err) + _wv_check!(err) result end + # @return [Float] def self.f64_id(x) err = ErrorStruct.new result = kitchen_sink_kitchen_f64_id(x, err) - check_error!(err) + _wv_check!(err) result end + # @return [String] def self.echo_string(s) s_s = _wv_str(s) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_echo_string(s_s, s_s.bytesize, out_len, err) - check_error!(err) + _wv_check!(err) _wv_take_string(result, out_len.read(:size_t)) end + # @return [String] def self.echo_bytes(b) b_s = _wv_bytes(b) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_echo_bytes(b_s, b_s.bytesize, out_len, err) - check_error!(err) + _wv_check!(err) _wv_take_bytes(result, out_len.read(:size_t)) end # Register a listener, returning how many are attached + # @return [Integer] def self.subscribe(listener) + _wv_present!(listener, 'listener') err = ErrorStruct.new - listener_ctx = _wv_cb_register(listener) - result = kitchen_sink_kitchen_subscribe(listener_ctx, WV_READY_LISTENER_VTABLE.to_ptr, err) - check_error!(err) + listener_ctx, listener_vtable = _wv_cb_register(listener, WV_READY_LISTENER_VTABLE) + result = kitchen_sink_kitchen_subscribe(listener_ctx, listener_vtable, err) + _wv_check!(err) + result + end + + # Register a listener if one is given + # @return [Boolean] + def self.maybe_subscribe(listener = nil) + err = ErrorStruct.new + listener_ctx, listener_vtable = _wv_cb_register(listener, WV_READY_LISTENER_VTABLE) + result = kitchen_sink_kitchen_maybe_subscribe(listener_ctx, listener_vtable, err) + _wv_check!(err) result end # Optional object in both directions - def self.maybe_gadget(fallback) + # @return [Gadget, nil] + def self.maybe_gadget(fallback = nil) + _wv_object!(fallback, Gadget, 'fallback', true) _wv_pin(fallback) do |_wv_fallback| err = ErrorStruct.new result = kitchen_sink_kitchen_maybe_gadget(_wv_fallback, err) - check_error!(err) + _wv_check!(err) result.null? ? nil : Gadget._from_ptr(result) end end # A list of objects + # @return [Array] def self.gadgets err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_gadgets(out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = Array.new(_wv_r.read_len) do - _wv_e0 = Gadget._from_ptr(_wv_r.read_object_token) - _wv_e0 - end - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_list_gadget(r) } end # An iterator over objects # Returns a lazy Enumerator that pulls one element per step. The # producer iterator starts on the first pull and is released when # iteration finishes or is abandoned early. + # @return [Enumerator] def self.stream_gadgets Enumerator.new do |y| err = ErrorStruct.new iter = kitchen_sink_kitchen_stream_gadgets(err) begin - check_error!(err) + _wv_check!(err) out_item = FFI::MemoryPointer.new(:pointer) loop do has_item = kitchen_sink_kitchen_StreamGadgetsIterator_next(iter, out_item, err) - check_error!(err) + _wv_check!(err) break if has_item.zero? item = out_item.read_pointer - raise Error.new(GENERIC_ERROR_CODE, 'null object pointer') if item.null? - value = Gadget._from_ptr(item) + value = Gadget._from_ptr(_wv_nonnull(item)) y << value end ensure @@ -573,74 +676,54 @@ module KitchenSink end # Optional struct return + # @return [Item, nil] def self.maybe_item(id) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_maybe_item(id, out_len, err) - check_kitchen_errors_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - if _wv_r.read_flag - _wv_value = _wv_read_item(_wv_r) - else - _wv_value = nil - end - _wv_value - end + _wv_check_kitchen_errors_error!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_opt_item(r) } end # List of structs + # @return [Array] def self.list_items err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_list_items(out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = Array.new(_wv_r.read_len) do - _wv_e0 = _wv_read_item(_wv_r) - _wv_e0 - end - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_list_item(r) } end # Map return type + # @return [Hash{String => Integer}] def self.get_attrs err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_get_attrs(out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = {} - _wv_r.read_len.times do - _wv_k0 = _wv_r.read_string - _wv_v0 = _wv_r.read_i32 - _wv_value[_wv_k0] = _wv_v0 - end - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_map_string_i32(r) } end # Iterator return type # Returns a lazy Enumerator that pulls one element per step. The # producer iterator starts on the first pull and is released when # iteration finishes or is abandoned early. + # @return [Enumerator] def self.stream_items Enumerator.new do |y| err = ErrorStruct.new iter = kitchen_sink_kitchen_stream_items(err) begin - check_error!(err) + _wv_check!(err) out_item = FFI::MemoryPointer.new(:pointer) out_len = FFI::MemoryPointer.new(:size_t) loop do has_item = kitchen_sink_kitchen_StreamItemsIterator_next(iter, out_item, out_len, err) - check_error!(err) + _wv_check!(err) break if has_item.zero? item = out_item.read_pointer - value = _wv_decode(item, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_item(_wv_r) - _wv_value - end + value = _wv_decode(item, out_len.read(:size_t)) { |r| _wv_read_item(r) } y << value end ensure @@ -650,19 +733,18 @@ module KitchenSink end # Returns the shared Token type from another module + # @return [Token] def self.cross_module_token err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_cross_module_token(out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_token(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_token(r) } end # Async operation # Blocks until the call completes on a producer thread. + # @return [String] def self.do_async(input) input_s = _wv_str(input) ctx, queue = _wv_async_begin @@ -673,6 +755,7 @@ module KitchenSink # Cancellable async operation # Blocks until the call completes on a producer thread. # @param cancel [CancelToken, nil] cancels the call; it then raises Cancelled + # @return [String] def self.do_cancellable(input, cancel: nil) input_s = _wv_str(input) own_token = CancelToken.new if cancel.nil? @@ -685,20 +768,22 @@ module KitchenSink end # Legacy operation kept for compatibility + # @return [Integer] # @deprecated Use new_op instead def self.legacy_op - warn '[DEPRECATED] Use new_op instead' + warn('KitchenSink.legacy_op is deprecated: Use new_op instead', uplevel: 1) err = ErrorStruct.new result = kitchen_sink_kitchen_legacy_op(err) - check_error!(err) + _wv_check!(err) result end # Replacement for legacy_op + # @return [Integer] def self.new_op err = ErrorStruct.new result = kitchen_sink_kitchen_new_op(err) - check_error!(err) + _wv_check!(err) result end @@ -707,13 +792,138 @@ module KitchenSink attach_function :kitchen_sink_kitchen_nested_hello, [:pointer, :pointer], :pointer, blocking: true # Trivial nested-module function + # @return [String] def self.hello err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = kitchen_sink_kitchen_nested_hello(out_len, err) - check_error!(err) + _wv_check!(err) _wv_take_string(result, out_len.read(:size_t)) end + + # === Value-buffer codecs of the composite types === + + # @api private + # Packs `[Gadget]` into the value-buffer wire format. + def self._wv_write_list_gadget(w, v) + w.write_len(v.length) + v.each { |e| w.write_object(e, Gadget) } + end + + # @api private + # Unpacks `[Gadget]` from the value-buffer wire format. + def self._wv_read_list_gadget(r) + Array.new(r.read_len) { r.read_object(Gadget) } + end + + # @api private + # Packs `Item?` into the value-buffer wire format. + def self._wv_write_opt_item(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + _wv_write_item(w, v) + end + end + + # @api private + # Unpacks `Item?` from the value-buffer wire format. + def self._wv_read_opt_item(r) + r.read_flag ? _wv_read_item(r) : nil + end + + # @api private + # Packs `[Item]` into the value-buffer wire format. + def self._wv_write_list_item(w, v) + w.write_len(v.length) + v.each { |e| _wv_write_item(w, e) } + end + + # @api private + # Unpacks `[Item]` from the value-buffer wire format. + def self._wv_read_list_item(r) + Array.new(r.read_len) { _wv_read_item(r) } + end + + # @api private + # Packs `{string:i32}` into the value-buffer wire format. + def self._wv_write_map_string_i32(w, v) + w.write_len(v.length) + v.each do |k, e| + w.write_string(k) + w.write_i32(e) + end + end + + # @api private + # Unpacks `{string:i32}` from the value-buffer wire format. + def self._wv_read_map_string_i32(r) + Array.new(r.read_len) { [r.read_string, r.read_i32] }.to_h + end + + # @api private + # Packs `[string]` into the value-buffer wire format. + def self._wv_write_list_string(w, v) + w.write_len(v.length) + v.each { |e| w.write_string(e) } + end + + # @api private + # Unpacks `[string]` from the value-buffer wire format. + def self._wv_read_list_string(r) + Array.new(r.read_len) { r.read_string } + end + + # @api private + # Packs `{string:string}` into the value-buffer wire format. + def self._wv_write_map_string_string(w, v) + w.write_len(v.length) + v.each do |k, e| + w.write_string(k) + w.write_string(e) + end + end + + # @api private + # Unpacks `{string:string}` from the value-buffer wire format. + def self._wv_read_map_string_string(r) + Array.new(r.read_len) { [r.read_string, r.read_string] }.to_h + end + + # @api private + # Packs `i64?` into the value-buffer wire format. + def self._wv_write_opt_i64(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_i64(v) + end + end + + # @api private + # Unpacks `i64?` from the value-buffer wire format. + def self._wv_read_opt_i64(r) + r.read_flag ? r.read_i64 : nil + end + + # @api private + # Packs `Gadget?` into the value-buffer wire format. + def self._wv_write_opt_gadget(w, v) + if v.nil? + w.write_flag(false) + else + w.write_flag(true) + w.write_object(v, Gadget) + end + end + + # @api private + # Unpacks `Gadget?` from the value-buffer wire format. + def self._wv_read_opt_gadget(r) + r.read_flag ? r.read_object(Gadget) : nil + end end # END kitchen_sink.rb diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_runtime_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_runtime_rb.snap index 834b8f57..e1a525a7 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_runtime_rb.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_kitchen_sink__lib_kitchen_sink_runtime_rb.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate # frozen_string_literal: true @@ -11,8 +11,8 @@ require 'ffi' require 'monitor' # The fixed runtime every generated KitchenSink binding builds on: the native -# library loader, the load-time ABI check, the error classes, the -# value-buffer reader and writer, cancel tokens, the handle table behind +# library loader, the load-time ABI and contract checks, the error classes, +# the value-buffer reader and writer, cancel tokens, the handle table behind # callback interfaces and async completions, and the object wrapper base. module KitchenSink extend FFI::Library @@ -37,7 +37,7 @@ module KitchenSink # The C ABI revision these bindings speak. Checked before anything else is # attached, so a mismatched library fails at require time instead of # misreading an error struct or a value buffer later. - ABI_VERSION = 3 + ABI_VERSION = 4 begin attach_function :kitchen_sink_abi_version, [], :uint32 rescue FFI::NotFoundError @@ -59,8 +59,7 @@ module KitchenSink end # Codes reserved by the runtime. Positive codes belong to a module's error - # domain; these negative codes are raised as a plain Error (or Cancelled) - # regardless of whether the function declares errors. + # domain; these negative codes report runtime failures. GENERIC_ERROR_CODE = -1 PANIC_ERROR_CODE = -2 MARSHAL_ERROR_CODE = -3 @@ -82,6 +81,16 @@ module KitchenSink end end + # Raised when a call that declares no errors fails anyway. That's a bug in + # the library (a panic, an argument it couldn't accept, a malformed value + # it returned, a callback failure it let through), never an outcome to + # handle; the message names the code and the library's message. + class NativeBugError < Error + def initialize(code, message) + super(code, "native call failed with code #{code}: #{message}") + end + end + # Raised when a cancellable call completes because its CancelToken fired. class Cancelled < Error def initialize(message = 'cancelled') @@ -90,8 +99,10 @@ module KitchenSink end attach_function :kitchen_sink_error_set, [:pointer, :int32, :string], :void + attach_function :kitchen_sink_error_set_payload, [:pointer, :pointer, :size_t], :void attach_function :kitchen_sink_error_clear, [:pointer], :void attach_function :kitchen_sink_error_free, [:pointer], :void + attach_function :kitchen_sink_alloc, [:size_t], :pointer attach_function :kitchen_sink_free_bytes, [:pointer, :size_t], :void attach_function :kitchen_sink_cancel_token_create, [], :pointer attach_function :kitchen_sink_cancel_token_cancel, [:pointer], :void @@ -99,13 +110,65 @@ module KitchenSink attach_function :kitchen_sink_cancel_token_destroy, [:pointer], :void # @api private - # The error for a code outside any declared domain. - def self._wv_error(code, message) + # One entry of a module's contract table. + class ContractEntry < FFI::Struct + layout :id, :uint64, + :hash, :uint64 + end + + # @api private + # Checks, for every top-level module, that the library's contract table + # (`kitchen_sink_{module}_contract`) holds each declaration these bindings + # were generated with, as `[id, hash, path]`, with an equal signature + # hash. Declarations the library adds are fine. + def self._wv_check_contract!(tables) + tables.each do |symbol, expected| + begin + attach_function symbol, [:pointer], :pointer unless respond_to?(symbol) + rescue FFI::NotFoundError + raise LoadError, "the loaded kitchen_sink library does not export #{symbol}" + end + out_len = FFI::MemoryPointer.new(:size_t) + table = public_send(symbol, out_len) + count = table.null? ? 0 : out_len.read(:size_t) + actual = Array.new(count) do |i| + entry = ContractEntry.new(table + (i * ContractEntry.size)) + [entry[:id], entry[:hash]] + end.to_h + expected.each do |id, hash, path| + found = actual[id] + raise LoadError, "kitchen_sink: #{path} is missing from the library" if found.nil? + next if found == hash + + raise LoadError, "kitchen_sink: #{path} changed since these bindings were generated" + end + end + end + + # @api private + # The error a throwing call raises for a code outside its domain: + # Cancelled for -5, otherwise the root Error. + def self._wv_error(code, message, _payload = nil) return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE Error.new(code, message) end + # @api private + # The error a call that declares no errors raises when it fails anyway: + # Cancelled for -5, otherwise NativeBugError. + def self._wv_trap(code, message, _payload = nil) + return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE + + NativeBugError.new(code, message) + end + + # @api private + # The error for a malformed value the library handed over. + def self._wv_malformed(what) + NativeBugError.new(MARSHAL_ERROR_CODE, "malformed value buffer: #{what}") + end + # @api private # Copies the code, message, and payload out of a reported error and clears # it. Returns nil when the struct reports success. @@ -133,25 +196,71 @@ module KitchenSink end # @api private - # Raises the generic error for a call that declares no error domain. - def self.check_error!(err) + # Raises NativeBugError for a failed call that declares no errors. + def self._wv_check!(err) taken = _wv_take_error(err) - return if taken.nil? - - raise _wv_error(taken[0], taken[1]) + raise _wv_trap(*taken) unless taken.nil? end # @api private # The UTF-8 bytes of a String argument, as a private binary copy the - # producer borrows for the call. + # library borrows for the call. def self._wv_str(value) - value.to_s.encode(Encoding::UTF_8).b + str = String.try_convert(value) + raise TypeError, "expected a String, got #{value.class}" if str.nil? + + utf8 = str.encode(Encoding::UTF_8) + raise ArgumentError, 'string is not valid UTF-8' unless utf8.valid_encoding? + + utf8.b end # @api private # A private binary copy of a bytes argument. def self._wv_bytes(value) - value.to_s.b + str = String.try_convert(value) + raise TypeError, "expected a String of bytes, got #{value.class}" if str.nil? + + str.b + end + + # The inclusive range of each fixed-width integer type. + INT_RANGES = { + i8: -0x80..0x7f, + u8: 0..0xff, + i16: -0x8000..0x7fff, + u16: 0..0xffff, + i32: -0x8000_0000..0x7fff_ffff, + u32: 0..0xffff_ffff, + i64: -0x8000_0000_0000_0000..0x7fff_ffff_ffff_ffff, + u64: 0..0xffff_ffff_ffff_ffff + }.freeze + + # @api private + # `value`, checked to be an Integer in the range of `kind` (`:i32`, ...): + # out-of-range values raise RangeError rather than wrap. + def self._wv_int(value, kind) + raise TypeError, "expected an Integer for #{kind}, got #{value.class}" unless value.is_a?(Integer) + raise RangeError, "#{value} is out of range for #{kind}" unless INT_RANGES[kind].cover?(value) + + value + end + + # @api private + # Raises TypeError unless `value` is a `cls` wrapper (or nil, when + # `optional`), so an object of the wrong interface never reaches the + # library. + def self._wv_object!(value, cls, name, optional = false) + return if value.is_a?(cls) || (optional && value.nil?) + + expected = optional ? "a #{cls.name} or nil" : "a #{cls.name}" + raise TypeError, "#{name} must be #{expected}, got #{value.class}" + end + + # @api private + # Raises TypeError for a nil callback implementation where one is required. + def self._wv_present!(value, name) + raise TypeError, "#{name} must not be nil" if value.nil? end # @api private @@ -160,6 +269,12 @@ module KitchenSink ptr.null? || len.zero? ? ''.b : ptr.read_string(len) end + # @api private + # Copies a borrowed (ptr, len) UTF-8 run without releasing it. + def self._wv_borrow_string(ptr, len) + _wv_borrow_bytes(ptr, len).force_encoding(Encoding::UTF_8) + end + # @api private # Copies a returned (ptr, len) allocation, then releases it. def self._wv_take_bytes(ptr, len) @@ -223,21 +338,23 @@ module KitchenSink # @api private # Appends values in the value-buffer wire format: little-endian, packed, - # no alignment. + # no alignment. Integers out of their type's range raise RangeError rather + # than wrap. class WvBufferWriter def initialize @buf = +''.b @objects = [] end - # Reserves an object token. The strong reference it carries is minted - # by #seal, just before the buffer is handed to the producer, so an - # encoding abandoned on the way never strands a reference. - def write_object(object) - raise TypeError, "expected a wrapper object, got #{object.class}" unless object.is_a?(WvObject) + # Reserves an object token for a `cls` wrapper. The strong reference it + # carries is minted by #seal, just before the buffer is handed to the + # library, so an encoding abandoned on the way never strands a + # reference. + def write_object(object, cls) + raise TypeError, "expected a #{cls.name}, got #{object.class}" unless object.is_a?(cls) @objects << [@buf.bytesize, object] - write_u64(0) + @buf << [0].pack('Q<') end # Mints the reference each reserved object token carries, recording it @@ -251,10 +368,15 @@ module KitchenSink @buf end - # Every append is a binary String, so the buffer stays binary and byte - # offsets index it directly. + # The finished encoding of a buffer that carries no objects. + def bytes + raise TypeError, 'this encoding cannot carry objects' unless @objects.empty? + + @buf + end + def write_bool(value) - @buf << [value ? 1 : 0].pack('C') + @buf << (value ? "\x01" : "\x00").b end def write_flag(value) @@ -262,27 +384,27 @@ module KitchenSink end def write_i8(value) - @buf << [value].pack('c') + @buf << [int(value, :i8)].pack('c') end def write_u8(value) - @buf << [value].pack('C') + @buf << [int(value, :u8)].pack('C') end def write_i16(value) - @buf << [value].pack('s<') + @buf << [int(value, :i16)].pack('s<') end def write_u16(value) - @buf << [value].pack('S<') + @buf << [int(value, :u16)].pack('S<') end def write_i32(value) - @buf << [value].pack('l<') + @buf << [int(value, :i32)].pack('l<') end def write_u32(value) - @buf << [value].pack('L<') + @buf << [int(value, :u32)].pack('L<') end def write_len(value) @@ -290,11 +412,11 @@ module KitchenSink end def write_i64(value) - @buf << [value].pack('q<') + @buf << [int(value, :i64)].pack('q<') end def write_u64(value) - @buf << [value].pack('Q<') + @buf << [int(value, :u64)].pack('Q<') end def write_f32(value) @@ -306,19 +428,25 @@ module KitchenSink end def write_string(value) - write_bytes(value.to_s.encode(Encoding::UTF_8)) + write_bytes(KitchenSink._wv_str(value)) end def write_bytes(value) - bytes = value.to_s.b + bytes = KitchenSink._wv_bytes(value) write_u32(bytes.bytesize) @buf << bytes end + + private + + def int(value, kind) + KitchenSink._wv_int(value, kind) + end end # @api private - # Reads values in the value-buffer wire format, raising Error on any - # malformed buffer. + # Reads values in the value-buffer wire format, raising NativeBugError on + # any malformed buffer. class WvBufferReader def initialize(data) @data = data.to_s.b @@ -326,9 +454,7 @@ module KitchenSink end def take(count, what) - if @pos + count > @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, "malformed value buffer: truncated #{what}") - end + raise KitchenSink._wv_malformed("truncated #{what}") if @pos + count > @data.bytesize bytes = @data.byteslice(@pos, count) @pos += count @@ -337,14 +463,14 @@ module KitchenSink def read_bool byte = take(1, 'bool').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: bool byte out of range') if byte > 1 + raise KitchenSink._wv_malformed('bool byte out of range') if byte > 1 byte == 1 end def read_flag byte = take(1, 'option flag').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: option flag out of range') if byte > 1 + raise KitchenSink._wv_malformed('option flag out of range') if byte > 1 byte == 1 end @@ -398,7 +524,7 @@ module KitchenSink def read_string text = read_bytes.force_encoding(Encoding::UTF_8) - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: string is not valid UTF-8') unless text.valid_encoding? + raise KitchenSink._wv_malformed('string is not valid UTF-8') unless text.valid_encoding? text end @@ -407,19 +533,19 @@ module KitchenSink take(read_u32, 'byte run') end - # An object token: a non-zero u64 carrying one strong reference the - # caller adopts into a wrapper. - def read_object_token + # An object token: a non-zero u64 carrying one strong reference, adopted + # into a new `cls` wrapper. + def read_object(cls) addr = read_u64 - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: null object token') if addr.zero? + raise KitchenSink._wv_malformed('null object token') if addr.zero? - FFI::Pointer.new(addr) + cls._from_ptr(FFI::Pointer.new(addr)) end def expect_end! return if @pos == @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: trailing bytes after value') + raise KitchenSink._wv_malformed('trailing bytes after value') end end @@ -427,7 +553,7 @@ module KitchenSink # Seals the writers of one call's object-carrying buffers (see # WvBufferWriter#seal) and returns their encodings. If any reference can't # be minted (a closed wrapper), the ones already minted are released before - # the error propagates. The producer adopts every minted reference once the + # the error propagates. The library adopts every minted reference once the # call is made. def self._wv_seal(*writers) minted = [] @@ -437,6 +563,23 @@ module KitchenSink raise end + # @api private + # Encodes an argument that carries no objects: yields a writer and + # returns the finished encoding. + def self._wv_encode + writer = WvBufferWriter.new + yield writer + writer.bytes + end + + # @api private + # A returned object pointer that must not be NULL. + def self._wv_nonnull(ptr) + raise NativeBugError.new(MARSHAL_ERROR_CODE, 'null object pointer') if ptr.null? + + ptr + end + # @api private # Decodes a returned value buffer: copies and releases it, yields a reader # to the block, and checks that the block consumed every byte. @@ -447,9 +590,19 @@ module KitchenSink value end - # Ruby objects the producer refers to by an integer key: callback-interface + # @api private + # Decodes a borrowed value buffer (a callback argument) without releasing + # it. + def self._wv_decode_borrowed(ptr, len) + reader = WvBufferReader.new(_wv_borrow_bytes(ptr, len)) + value = yield reader + reader.expect_end! + value + end + + # Ruby objects the library refers to by an integer key: callback-interface # implementations (the key is the vtable `ctx`) and pending async - # completions (the key is the completion `context`). The producer never + # completions (the key is the completion `context`). The library never # sees a Ruby object address. A Monitor rather than a Mutex, so a # finalizer that runs while this thread holds the lock can't deadlock it. @wv_handles = {} @@ -476,12 +629,15 @@ module KitchenSink end # @api private - # Registers a callback-interface implementation and returns its `ctx`. - # Never raises, so nothing can strand the registration between this and - # the call that hands it to the producer (an unusable implementation fails - # when the producer calls it, with FOREIGN_ERROR_CODE). - def self._wv_cb_register(impl) - _wv_handle_put([:callback, impl]) + # Registers a callback-interface implementation and returns its `ctx` and + # `vtable` arguments: two NULLs for a nil (optional) implementation. Never + # raises, so nothing can strand the registration between this and the + # call that hands it to the library (an unusable implementation fails + # when the library calls it, with FOREIGN_ERROR_CODE). + def self._wv_cb_register(impl, vtable) + return [nil, nil] if impl.nil? + + [_wv_handle_put([:callback, impl]), vtable.to_ptr] end # @api private @@ -494,23 +650,82 @@ module KitchenSink end # @api private - # Drops an implementation when the producer releases its last reference. + # Drops an implementation when the library releases its last reference. def self._wv_cb_free(ctx) _wv_handle_take(ctx) nil end # @api private - # Reports a callback implementation's exception through `out_err`. The - # producer copies the message. - def self._wv_cb_fail(out_err, exception) + # The message a callback failure reports: the exception's message (its + # class name when that's empty), as UTF-8 without NULs. + def self._wv_cb_message(exception) message = exception.message.to_s - message = exception.class.name if message.empty? - message = message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") - kitchen_sink_error_set(out_err, FOREIGN_ERROR_CODE, message) + message = exception.class.name.to_s if message.empty? + message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") + end + + # @api private + # Reports a callback implementation's exception through `out_err` as + # FOREIGN_ERROR_CODE. The library copies the message. + def self._wv_cb_fail(out_err, exception) + kitchen_sink_error_set(out_err, FOREIGN_ERROR_CODE, _wv_cb_message(exception)) nil end + # @api private + # Reports a domain error raised by a callback method declared to throw: + # the block returns its `[code, payload]`, or nil when it isn't one of the + # domain's codes, which then fails as FOREIGN_ERROR_CODE like any other + # exception. + def self._wv_cb_throw(out_err, exception) + code, payload = yield + return _wv_cb_fail(out_err, exception) if code.nil? + + kitchen_sink_error_set(out_err, code, _wv_cb_message(exception)) + kitchen_sink_error_set_payload(out_err, payload, payload.bytesize) unless payload.empty? + nil + rescue Exception => e # rubocop:disable Lint/RescueException + _wv_cb_fail(out_err, e) + end + + # @api private + # A callback method's floating-point return, checked to be numeric. + def self._wv_float(value) + raise TypeError, "expected a Float, got #{value.class}" unless value.is_a?(Numeric) + + value.to_f + end + + # @api private + # Hands a callback method's string, bytes, or buffer return to the + # library: a run allocated with kitchen_sink_alloc, written to the out slots. + # The library adopts and frees it. + def self._wv_cb_return_bytes(out_ptr, out_len, data) + len = data.bytesize + run = FFI::Pointer::NULL + unless len.zero? + run = kitchen_sink_alloc(len) + raise NoMemoryError, 'kitchen_sink_alloc failed' if run.null? + + run.put_bytes(0, data) + end + out_ptr.write_pointer(run) + out_len.write(:size_t, len) + nil + end + + # @api private + # A callback method's object return: a fresh strong reference the library + # adopts, or NULL for nil (which the library rejects where the return + # isn't optional). + def self._wv_cb_return_object(value, cls) + return FFI::Pointer::NULL if value.nil? + raise TypeError, "expected a #{cls.name}, got #{value.class}" unless value.is_a?(cls) + + value._wv_clone_ptr + end + # @api private # Registers a pending async call; returns its completion context and the # queue its completion is delivered on. @@ -552,7 +767,7 @@ module KitchenSink # another thread only releases the reference once the call returns. class WvObject # @api private - # Adopts one strong reference the producer handed over. + # Adopts one strong reference the library handed over. def self._from_ptr(ptr) obj = allocate obj._wv_init(ptr) @@ -582,7 +797,7 @@ module KitchenSink # Releases this wrapper's reference now rather than at GC time. # Idempotent. The object itself lives until its last reference anywhere - # (another wrapper, a record field, the producer) goes. + # (another wrapper, a record field, the library) goes. def close handle = @wv_lock.synchronize do next nil if @wv_handle.nil? || @wv_closing @@ -619,7 +834,7 @@ module KitchenSink # @api private # Mints a new strong reference the caller owns (an object written into a - # value buffer carries its own reference). + # value buffer or returned from a callback carries its own reference). def _wv_clone_ptr _wv_call { |ptr| self.class._wv_clone(ptr) } end @@ -666,36 +881,16 @@ module KitchenSink end end - # @api private - # Checks each top-level module's contract checksum (`module name => - # [checksum function, expected value]`) against the library, so bindings - # generated from a different API fail at require time. - def self._wv_check_contract!(checksums) - checksums.each do |mod, (symbol, expected)| - begin - attach_function symbol, [], :uint64 - rescue FFI::NotFoundError - raise LoadError, "the loaded kitchen_sink library has no module '#{mod}' (#{symbol} is missing)" - end - actual = public_send(symbol) - next if actual == expected - - raise LoadError, "kitchen_sink module '#{mod}' does not match these bindings: expected contract " \ - "checksum 0x#{expected.to_s(16)}, the library reports 0x#{actual.to_s(16)}; " \ - 'regenerate the bindings' - end - end - # @api private # Live native resources of one kind (0 objects, 1 callbacks, 2 iterators, - # 3 cancel tokens, 4 byte allocations). Only libraries built with the - # `leak-check` feature export the counter. + # 3 cancel tokens, 4 byte runs; -1 reports whether the library counts at + # all). Only libraries built with the `leak-check` feature count. def self._wv_debug_live(kind) unless respond_to?(:kitchen_sink_debug_live) begin attach_function :kitchen_sink_debug_live, [:int32], :uint64 rescue FFI::NotFoundError - raise Error.new(GENERIC_ERROR_CODE, 'kitchen_sink was built without the leak-check feature') + raise Error.new(GENERIC_ERROR_CODE, 'kitchen_sink does not export kitchen_sink_debug_live') end end kitchen_sink_debug_live(kind) diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__README_md.snap deleted file mode 100644 index 35016398..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__README_md.snap +++ /dev/null @@ -1,33 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# nested_modules (Ruby) - -Ruby bindings for `nested_modules` using the [ffi](https://github.com/ffi/ffi) gem. - -## Prerequisites - -- Ruby 2.7 or newer -- The native library (`libnested_modules.so`, `libnested_modules.dylib`, or `nested_modules.dll`) on the library - search path, or its path in the `NESTED_MODULES_LIBRARY` environment variable. - -## Install - -```bash -gem build nested_modules.gemspec -gem install nested_modules-0.1.0.gem -``` - -## Usage - -```ruby -require 'nested_modules' - -NestedModules::ABI_VERSION -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_rb.snap index 077d5ac6..7d74d436 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_rb.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_rb.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate # frozen_string_literal: true @@ -11,10 +11,21 @@ require_relative 'nested_modules/runtime' # Ruby bindings for nested_modules. module NestedModules - # Contract checksums of the top-level modules these bindings were - # generated from; a library built from a different API fails to load. + # The declarations these bindings were generated with, per top-level + # module, as [id, hash, path]. Loading fails, naming the path, unless + # the library's contract table holds each one with an equal hash. _wv_check_contract!( - 'graphics' => [:nested_modules_graphics_checksum, 0x44d31b5b10fdb90d], + nested_modules_graphics_contract: [ + [0x2c29eef9327a6bd4, 0x12983d5271651389, 'graphics.Unit'], + [0x304088bbc28f0015, 0x97e6e809913be6b6, 'graphics.new_canvas'], + [0x49f8837532df9ac6, 0x32e5b27e81ee0957, 'graphics.Canvas'], + [0x55717fa027336b47, 0xb2bd3a6bdcce48c8, 'graphics.shapes.describe'], + [0x7626b91b8f1f53cd, 0x867bbf7cf0f41c74, 'graphics.shapes.geometry.compute'], + [0xb3efcb50d9a61ab7, 0x3cb0207dfad2a149, 'graphics.GraphicsError'], + [0xc62b16a9591b744b, 0x7895610c8487eb1b, 'graphics.shapes.Shape'], + [0xe13001f4983ccd35, 0x16e7de070b4a4cfc, 'graphics.shapes.geometry.Detail'], + [0xf3f658bcfc5dba62, 0x2665ccb47356e746, 'graphics.shapes.make_shape'], + ], ) # === Module: graphics === @@ -25,30 +36,34 @@ module NestedModules class OutOfBounds < GraphicsError CODE = 1 - def initialize(message = 'coordinate out of bounds') - super(1, message) + def initialize(message = nil) + super(CODE, message || 'coordinate out of bounds') end end end - # Maps each ABI code of the GraphicsError domain to its error class. - GRAPHICS_ERROR_CODES = { - 1 => GraphicsError::OutOfBounds, - }.freeze - - # Builds the GraphicsError subclass matching `code`, decoding any payload - # fields the code declares, or a generic Error for codes outside - # the domain (panics, marshalling, callback failures). - def self.graphics_error_from(code, message, payload = nil) - cls = GRAPHICS_ERROR_CODES[code] - return _wv_error(code, message) if cls.nil? - message.empty? ? cls.new : cls.new(message) + # @api private + # The GraphicsError for a domain `code`, with its payload fields decoded; + # the root Error (or Cancelled) for any other code. + def self._wv_graphics_error_from(code, message, payload = nil) + message = nil if message.empty? + r = WvBufferReader.new(payload) + error = + case code + when 1 + GraphicsError::OutOfBounds.new(message) + else + return _wv_error(code, message.to_s) + end + r.expect_end! + error end - # Raises the typed GraphicsError for a non-zero error slot. - def self.check_graphics_error!(err) + # @api private + # Raises the GraphicsError for a non-zero error slot. + def self._wv_check_graphics_error!(err) taken = _wv_take_error(err) - raise graphics_error_from(*taken) if taken + raise _wv_graphics_error_from(*taken) unless taken.nil? end # A measurement unit shared with nested submodules. @@ -71,10 +86,9 @@ module NestedModules # Structural equality over every field. def ==(other) - return false unless other.is_a?(Canvas) - return false unless width == other.width - return false unless root == other.root - true + other.is_a?(Canvas) && + width == other.width && + root == other.root end end @@ -88,23 +102,19 @@ module NestedModules # @api private # Unpacks a Canvas from the value-buffer wire format. def self._wv_read_canvas(r) - _wv_width = r.read_i32 - _wv_root = _wv_read_shape(r) - Canvas.new(width: _wv_width, root: _wv_root) + Canvas.new(width: r.read_i32, root: _wv_read_shape(r)) end - attach_function :nested_modules_graphics_new_canvas, [:int32, :pointer, :pointer], :pointer + attach_function :nested_modules_graphics_new_canvas, [:int32, :pointer, :pointer], :pointer, blocking: true # Construct a Canvas. + # @return [Canvas] def self.new_canvas(width) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = nested_modules_graphics_new_canvas(width, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_canvas(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_canvas(r) } end # === Module: graphics.shapes === @@ -127,11 +137,10 @@ module NestedModules # Structural equality over every field. def ==(other) - return false unless other.is_a?(Shape) - return false unless sides == other.sides - return false unless unit == other.unit - return false unless detail == other.detail - true + other.is_a?(Shape) && + sides == other.sides && + unit == other.unit && + detail == other.detail end end @@ -146,37 +155,30 @@ module NestedModules # @api private # Unpacks a Shape from the value-buffer wire format. def self._wv_read_shape(r) - _wv_sides = r.read_i32 - _wv_unit = r.read_i32 - _wv_detail = _wv_read_detail(r) - Shape.new(sides: _wv_sides, unit: _wv_unit, detail: _wv_detail) + Shape.new(sides: r.read_i32, unit: r.read_i32, detail: _wv_read_detail(r)) end - attach_function :nested_modules_graphics_shapes_make_shape, [:int32, :pointer, :pointer], :pointer - attach_function :nested_modules_graphics_shapes_describe, [:pointer, :pointer], :pointer + attach_function :nested_modules_graphics_shapes_make_shape, [:int32, :pointer, :pointer], :pointer, blocking: true + attach_function :nested_modules_graphics_shapes_describe, [:pointer, :pointer], :pointer, blocking: true # Construct a Shape. + # @return [Shape] def self.make_shape(sides) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = nested_modules_graphics_shapes_make_shape(sides, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_shape(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_shape(r) } end # Nested function returning a struct owned by an ancestor module. + # @return [Canvas] def self.describe err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = nested_modules_graphics_shapes_describe(out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_canvas(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_canvas(r) } end # === Module: graphics.shapes.geometry === @@ -191,9 +193,8 @@ module NestedModules # Structural equality over every field. def ==(other) - return false unless other.is_a?(Detail) - return false unless area == other.area - true + other.is_a?(Detail) && + area == other.area end end @@ -206,25 +207,20 @@ module NestedModules # @api private # Unpacks a Detail from the value-buffer wire format. def self._wv_read_detail(r) - _wv_area = r.read_f64 - Detail.new(area: _wv_area) + Detail.new(area: r.read_f64) end - attach_function :nested_modules_graphics_shapes_geometry_compute, [:pointer, :size_t, :pointer, :pointer], :pointer + attach_function :nested_modules_graphics_shapes_geometry_compute, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true # Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. + # @return [Detail] def self.compute(shape) - shape_w = WvBufferWriter.new - _wv_write_shape(shape_w, shape) + shape_s = _wv_encode { |w| _wv_write_shape(w, shape) } err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - shape_s = _wv_seal(shape_w).first result = nested_modules_graphics_shapes_geometry_compute(shape_s, shape_s.bytesize, out_len, err) - check_graphics_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_detail(_wv_r) - _wv_value - end + _wv_check_graphics_error!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_detail(r) } end end diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_runtime_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_runtime_rb.snap deleted file mode 100644 index 4e9941a8..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__lib_nested_modules_runtime_rb.snap +++ /dev/null @@ -1,705 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# frozen_string_literal: true - -require 'ffi' -require 'monitor' - -# The fixed runtime every generated NestedModules binding builds on: the native -# library loader, the load-time ABI check, the error classes, the -# value-buffer reader and writer, cancel tokens, the handle table behind -# callback interfaces and async completions, and the object wrapper base. -module NestedModules - extend FFI::Library - - # An explicit path in NESTED_MODULES_LIBRARY wins. Otherwise a library bundled - # under lib/native/ (a platform gem) is preferred over the system search - # path. - _wv_override = ENV['NESTED_MODULES_LIBRARY'] - if _wv_override && !_wv_override.empty? - ffi_lib _wv_override - else - _wv_name = - case FFI::Platform::OS - when /darwin/ then 'libnested_modules.dylib' - when /mswin|mingw|windows/ then 'nested_modules.dll' - else 'libnested_modules.so' - end - _wv_bundled = File.expand_path(File.join('..', 'native', _wv_name), __dir__) - ffi_lib(File.exist?(_wv_bundled) ? _wv_bundled : _wv_name) - end - - # The C ABI revision these bindings speak. Checked before anything else is - # attached, so a mismatched library fails at require time instead of - # misreading an error struct or a value buffer later. - ABI_VERSION = 3 - begin - attach_function :nested_modules_abi_version, [], :uint32 - rescue FFI::NotFoundError - raise LoadError, "the loaded nested_modules library does not export nested_modules_abi_version " \ - "(these bindings expect ABI revision #{ABI_VERSION})" - end - _wv_abi = nested_modules_abi_version - unless _wv_abi == ABI_VERSION - raise LoadError, "nested_modules ABI mismatch: these bindings expect revision #{ABI_VERSION} " \ - "but the loaded library reports revision #{_wv_abi}" - end - - # The `nested_modules_error` out-parameter every call reports through. - class ErrorStruct < FFI::Struct - layout :code, :int32, - :message, :pointer, - :payload_ptr, :pointer, - :payload_len, :size_t - end - - # Codes reserved by the runtime. Positive codes belong to a module's error - # domain; these negative codes are raised as a plain Error (or Cancelled) - # regardless of whether the function declares errors. - GENERIC_ERROR_CODE = -1 - PANIC_ERROR_CODE = -2 - MARSHAL_ERROR_CODE = -3 - # A callback-interface implementation raised; the message carries the - # exception text. - FOREIGN_ERROR_CODE = -4 - # The call was cancelled through its CancelToken. - CANCELLED_ERROR_CODE = -5 - - # The base class of every error the library raises. - class Error < StandardError - # The ABI error code: positive for a domain error, negative for a runtime - # failure. - attr_reader :code - - def initialize(code, message) - @code = code - super(message) - end - end - - # Raised when a cancellable call completes because its CancelToken fired. - class Cancelled < Error - def initialize(message = 'cancelled') - super(CANCELLED_ERROR_CODE, message) - end - end - - attach_function :nested_modules_error_set, [:pointer, :int32, :string], :void - attach_function :nested_modules_error_clear, [:pointer], :void - attach_function :nested_modules_error_free, [:pointer], :void - attach_function :nested_modules_free_bytes, [:pointer, :size_t], :void - attach_function :nested_modules_cancel_token_create, [], :pointer - attach_function :nested_modules_cancel_token_cancel, [:pointer], :void - attach_function :nested_modules_cancel_token_is_cancelled, [:pointer], :bool - attach_function :nested_modules_cancel_token_destroy, [:pointer], :void - - # @api private - # The error for a code outside any declared domain. - def self._wv_error(code, message) - return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE - - Error.new(code, message) - end - - # @api private - # Copies the code, message, and payload out of a reported error and clears - # it. Returns nil when the struct reports success. - def self._wv_take_error(err) - code = err[:code] - return nil if code.zero? - - msg_ptr = err[:message] - message = msg_ptr.null? ? '' : msg_ptr.read_string.force_encoding(Encoding::UTF_8) - payload_ptr = err[:payload_ptr] - payload = payload_ptr.null? ? nil : payload_ptr.read_string(err[:payload_len]) - nested_modules_error_clear(err.to_ptr) - [code, message, payload] - end - - # @api private - # Like _wv_take_error for the heap-boxed error an async completion - # receives; the box is released with nested_modules_error_free. - def self._wv_take_boxed_error(err_ptr) - return nil if err_ptr.null? - - taken = _wv_take_error(ErrorStruct.new(err_ptr)) - nested_modules_error_free(err_ptr) - taken - end - - # @api private - # Raises the generic error for a call that declares no error domain. - def self.check_error!(err) - taken = _wv_take_error(err) - return if taken.nil? - - raise _wv_error(taken[0], taken[1]) - end - - # @api private - # The UTF-8 bytes of a String argument, as a private binary copy the - # producer borrows for the call. - def self._wv_str(value) - value.to_s.encode(Encoding::UTF_8).b - end - - # @api private - # A private binary copy of a bytes argument. - def self._wv_bytes(value) - value.to_s.b - end - - # @api private - # Copies a borrowed (ptr, len) run without releasing it. - def self._wv_borrow_bytes(ptr, len) - ptr.null? || len.zero? ? ''.b : ptr.read_string(len) - end - - # @api private - # Copies a returned (ptr, len) allocation, then releases it. - def self._wv_take_bytes(ptr, len) - return ''.b if ptr.null? - - data = len.zero? ? ''.b : ptr.read_string(len) - nested_modules_free_bytes(ptr, len) - data - end - - # @api private - # Copies a returned UTF-8 string, then releases it. - def self._wv_take_string(ptr, len) - _wv_take_bytes(ptr, len).force_encoding(Encoding::UTF_8) - end - - # A cancellation signal for a cancellable async call. Pass one as the - # `cancel:` keyword, then call #cancel from any thread; the call raises - # Cancelled unless it already completed. A token may be shared by several - # calls and stays cancelled once cancelled. - class CancelToken - # @api private - # Owns the consumer's reference to the native token. - class Ptr < FFI::AutoPointer - def self.release(ptr) - NestedModules.nested_modules_cancel_token_destroy(ptr) - end - end - - def initialize - @ptr = Ptr.new(NestedModules.nested_modules_cancel_token_create) - end - - # Requests cancellation. Idempotent. - def cancel - NestedModules.nested_modules_cancel_token_cancel(@ptr) unless @ptr.nil? - nil - end - - # Whether #cancel has been called. - def cancelled? - !@ptr.nil? && NestedModules.nested_modules_cancel_token_is_cancelled(@ptr) - end - - # Releases the native token now rather than at GC time. Calls already - # launched with it keep their own reference. - def close - return if @ptr.nil? - - @ptr.free - @ptr = nil - end - - # @api private - def _wv_ptr - raise Error.new(GENERIC_ERROR_CODE, 'CancelToken used after close') if @ptr.nil? - - @ptr - end - end - - # @api private - # Appends values in the value-buffer wire format: little-endian, packed, - # no alignment. - class WvBufferWriter - def initialize - @buf = +''.b - @objects = [] - end - - # Reserves an object token. The strong reference it carries is minted - # by #seal, just before the buffer is handed to the producer, so an - # encoding abandoned on the way never strands a reference. - def write_object(object) - raise TypeError, "expected a wrapper object, got #{object.class}" unless object.is_a?(WvObject) - - @objects << [@buf.bytesize, object] - write_u64(0) - end - - # Mints the reference each reserved object token carries, recording it - # in `minted`, and returns the finished encoding. - def seal(minted) - @objects.each do |offset, object| - ptr = object._wv_clone_ptr - minted << [object.class, ptr] - @buf[offset, 8] = [ptr.address].pack('Q<') - end - @buf - end - - # Every append is a binary String, so the buffer stays binary and byte - # offsets index it directly. - def write_bool(value) - @buf << [value ? 1 : 0].pack('C') - end - - def write_flag(value) - write_bool(value) - end - - def write_i8(value) - @buf << [value].pack('c') - end - - def write_u8(value) - @buf << [value].pack('C') - end - - def write_i16(value) - @buf << [value].pack('s<') - end - - def write_u16(value) - @buf << [value].pack('S<') - end - - def write_i32(value) - @buf << [value].pack('l<') - end - - def write_u32(value) - @buf << [value].pack('L<') - end - - def write_len(value) - write_u32(value) - end - - def write_i64(value) - @buf << [value].pack('q<') - end - - def write_u64(value) - @buf << [value].pack('Q<') - end - - def write_f32(value) - @buf << [value].pack('e') - end - - def write_f64(value) - @buf << [value].pack('E') - end - - def write_string(value) - write_bytes(value.to_s.encode(Encoding::UTF_8)) - end - - def write_bytes(value) - bytes = value.to_s.b - write_u32(bytes.bytesize) - @buf << bytes - end - end - - # @api private - # Reads values in the value-buffer wire format, raising Error on any - # malformed buffer. - class WvBufferReader - def initialize(data) - @data = data.to_s.b - @pos = 0 - end - - def take(count, what) - if @pos + count > @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, "malformed value buffer: truncated #{what}") - end - - bytes = @data.byteslice(@pos, count) - @pos += count - bytes - end - - def read_bool - byte = take(1, 'bool').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: bool byte out of range') if byte > 1 - - byte == 1 - end - - def read_flag - byte = take(1, 'option flag').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: option flag out of range') if byte > 1 - - byte == 1 - end - - def read_i8 - take(1, 'i8').unpack1('c') - end - - def read_u8 - take(1, 'u8').unpack1('C') - end - - def read_i16 - take(2, 'i16').unpack1('s<') - end - - def read_u16 - take(2, 'u16').unpack1('S<') - end - - def read_i32 - take(4, 'i32').unpack1('l<') - end - - def read_u32 - take(4, 'u32').unpack1('L<') - end - - # A u32 element count. Counts are not checked against the remaining - # bytes, since an element can encode to zero bytes; a short buffer still - # fails on the first truncated read. - def read_len - read_u32 - end - - def read_i64 - take(8, 'i64').unpack1('q<') - end - - def read_u64 - take(8, 'u64').unpack1('Q<') - end - - def read_f32 - take(4, 'f32').unpack1('e') - end - - def read_f64 - take(8, 'f64').unpack1('E') - end - - def read_string - text = read_bytes.force_encoding(Encoding::UTF_8) - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: string is not valid UTF-8') unless text.valid_encoding? - - text - end - - def read_bytes - take(read_u32, 'byte run') - end - - # An object token: a non-zero u64 carrying one strong reference the - # caller adopts into a wrapper. - def read_object_token - addr = read_u64 - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: null object token') if addr.zero? - - FFI::Pointer.new(addr) - end - - def expect_end! - return if @pos == @data.bytesize - - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: trailing bytes after value') - end - end - - # @api private - # Seals the writers of one call's object-carrying buffers (see - # WvBufferWriter#seal) and returns their encodings. If any reference can't - # be minted (a closed wrapper), the ones already minted are released before - # the error propagates. The producer adopts every minted reference once the - # call is made. - def self._wv_seal(*writers) - minted = [] - writers.map { |writer| writer.seal(minted) } - rescue Exception # rubocop:disable Lint/RescueException - minted.each { |cls, ptr| cls::WV_PTR.release(ptr) } - raise - end - - # @api private - # Decodes a returned value buffer: copies and releases it, yields a reader - # to the block, and checks that the block consumed every byte. - def self._wv_decode(ptr, len) - reader = WvBufferReader.new(_wv_take_bytes(ptr, len)) - value = yield reader - reader.expect_end! - value - end - - # Ruby objects the producer refers to by an integer key: callback-interface - # implementations (the key is the vtable `ctx`) and pending async - # completions (the key is the completion `context`). The producer never - # sees a Ruby object address. A Monitor rather than a Mutex, so a - # finalizer that runs while this thread holds the lock can't deadlock it. - @wv_handles = {} - @wv_next_handle = 0 - @wv_handles_lock = Monitor.new - - # @api private - def self._wv_handle_put(value) - @wv_handles_lock.synchronize do - @wv_next_handle += 1 - @wv_handles[@wv_next_handle] = value - FFI::Pointer.new(@wv_next_handle) - end - end - - # @api private - def self._wv_handle_get(key) - @wv_handles_lock.synchronize { @wv_handles[key.address] } - end - - # @api private - def self._wv_handle_take(key) - @wv_handles_lock.synchronize { @wv_handles.delete(key.address) } - end - - # @api private - # Registers a callback-interface implementation and returns its `ctx`. - # Never raises, so nothing can strand the registration between this and - # the call that hands it to the producer (an unusable implementation fails - # when the producer calls it, with FOREIGN_ERROR_CODE). - def self._wv_cb_register(impl) - _wv_handle_put([:callback, impl]) - end - - # @api private - # Resolves a trampoline's `ctx` back to the registered implementation. - def self._wv_cb_lookup(ctx) - kind, impl = _wv_handle_get(ctx) - raise Error.new(FOREIGN_ERROR_CODE, 'callback context is not registered') unless kind == :callback - - impl - end - - # @api private - # Drops an implementation when the producer releases its last reference. - def self._wv_cb_free(ctx) - _wv_handle_take(ctx) - nil - end - - # @api private - # Reports a callback implementation's exception through `out_err`. The - # producer copies the message. - def self._wv_cb_fail(out_err, exception) - message = exception.message.to_s - message = exception.class.name if message.empty? - message = message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") - nested_modules_error_set(out_err, FOREIGN_ERROR_CODE, message) - nil - end - - # @api private - # Registers a pending async call; returns its completion context and the - # queue its completion is delivered on. - def self._wv_async_begin - queue = Queue.new - [_wv_handle_put([:async, queue]), queue] - end - - # @api private - # The queue of the pending call a completion belongs to. Each context - # completes exactly once, so its entry is removed here. - def self._wv_async_finish(ctx) - kind, queue = _wv_handle_take(ctx) - kind == :async ? queue : Queue.new - end - - # @api private - # Blocks until a pending call completes, raising its error. If the wait is - # interrupted (Thread#raise, Thread#kill, Timeout), the call's token is - # cancelled before the interruption propagates. - def self._wv_async_wait(queue, token = nil) - value = - begin - queue.pop - rescue Exception # rubocop:disable Lint/RescueException - token&.cancel - raise - end - raise value if value.is_a?(Exception) - - value - end - - # @api private - # The base of every interface wrapper. A wrapper owns one strong reference - # to its native object and releases it exactly once, from #close or (as a - # backstop) from the GC finalizer of its FFI::AutoPointer. Calls borrow the - # pointer through #_wv_call, so a #close that races an in-flight call on - # another thread only releases the reference once the call returns. - class WvObject - # @api private - # Adopts one strong reference the producer handed over. - def self._from_ptr(ptr) - obj = allocate - obj._wv_init(ptr) - obj - end - - # @api private - def _wv_init(ptr) - @wv_handle = self.class::WV_PTR.new(ptr) - @wv_lock = Mutex.new - @wv_calls = 0 - @wv_closing = false - end - - # The native object pointer (borrowed; valid until #close). - def handle - handle = @wv_handle - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if handle.nil? || @wv_closing - - handle - end - - # Whether #close has released this wrapper's reference. - def closed? - @wv_handle.nil? || @wv_closing - end - - # Releases this wrapper's reference now rather than at GC time. - # Idempotent. The object itself lives until its last reference anywhere - # (another wrapper, a record field, the producer) goes. - def close - handle = @wv_lock.synchronize do - next nil if @wv_handle.nil? || @wv_closing - - if @wv_calls.zero? - taken = @wv_handle - @wv_handle = nil - taken - else - @wv_closing = true - nil - end - end - handle&.free - nil - end - - # @api private - # Yields the native pointer, keeping the reference alive until the block - # returns. - def _wv_call - handle = @wv_lock.synchronize do - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if @wv_handle.nil? || @wv_closing - - @wv_calls += 1 - @wv_handle - end - begin - yield handle - ensure - _wv_leave - end - end - - # @api private - # Mints a new strong reference the caller owns (an object written into a - # value buffer carries its own reference). - def _wv_clone_ptr - _wv_call { |ptr| self.class._wv_clone(ptr) } - end - - # dup and clone produce an independent wrapper with its own reference to - # the same object. - def initialize_copy(other) - super - _wv_init(other._wv_clone_ptr) - end - - private - - def _wv_leave - release = @wv_lock.synchronize do - @wv_calls -= 1 - next nil unless @wv_closing && @wv_calls.zero? - - taken = @wv_handle - @wv_handle = nil - @wv_closing = false - taken - end - release&.free - end - end - - # @api private - # Yields the native pointers of `objects` (NULL for nil), each kept alive - # until the block returns. - def self._wv_pin(*objects, &block) - _wv_pin_from(objects, 0, [], &block) - end - - # @api private - def self._wv_pin_from(objects, index, pointers, &block) - return yield(*pointers) if index == objects.length - - object = objects[index] - if object.nil? - _wv_pin_from(objects, index + 1, pointers + [FFI::Pointer::NULL], &block) - else - object._wv_call { |ptr| _wv_pin_from(objects, index + 1, pointers + [ptr], &block) } - end - end - - # @api private - # Checks each top-level module's contract checksum (`module name => - # [checksum function, expected value]`) against the library, so bindings - # generated from a different API fail at require time. - def self._wv_check_contract!(checksums) - checksums.each do |mod, (symbol, expected)| - begin - attach_function symbol, [], :uint64 - rescue FFI::NotFoundError - raise LoadError, "the loaded nested_modules library has no module '#{mod}' (#{symbol} is missing)" - end - actual = public_send(symbol) - next if actual == expected - - raise LoadError, "nested_modules module '#{mod}' does not match these bindings: expected contract " \ - "checksum 0x#{expected.to_s(16)}, the library reports 0x#{actual.to_s(16)}; " \ - 'regenerate the bindings' - end - end - - # @api private - # Live native resources of one kind (0 objects, 1 callbacks, 2 iterators, - # 3 cancel tokens, 4 byte allocations). Only libraries built with the - # `leak-check` feature export the counter. - def self._wv_debug_live(kind) - unless respond_to?(:nested_modules_debug_live) - begin - attach_function :nested_modules_debug_live, [:int32], :uint64 - rescue FFI::NotFoundError - raise Error.new(GENERIC_ERROR_CODE, 'nested_modules was built without the leak-check feature') - end - end - nested_modules_debug_live(kind) - end -end - -# END runtime.rb diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__nested_modules_gemspec.snap b/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__nested_modules_gemspec.snap deleted file mode 100644 index 323d0099..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_nested_modules__nested_modules_gemspec.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -Gem::Specification.new do |s| - s.name = 'nested_modules' - s.version = '0.1.0' - s.summary = 'Native bindings for nested_modules' - s.authors = ['nested_modules'] - s.files = Dir['lib/**/*.rb'] - s.require_paths = ['lib'] - s.required_ruby_version = '>= 2.7' - - s.add_dependency 'ffi', '~> 1.15' -end - -# END nested_modules.gemspec diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/ruby_shapes__README_md.snap deleted file mode 100644 index 439163f5..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__README_md.snap +++ /dev/null @@ -1,33 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# shapes (Ruby) - -Ruby bindings for `shapes` using the [ffi](https://github.com/ffi/ffi) gem. - -## Prerequisites - -- Ruby 2.7 or newer -- The native library (`libshapes.so`, `libshapes.dylib`, or `shapes.dll`) on the library - search path, or its path in the `SHAPES_LIBRARY` environment variable. - -## Install - -```bash -gem build shapes.gemspec -gem install shapes-0.1.0.gem -``` - -## Usage - -```ruby -require 'shapes' - -Shapes::ABI_VERSION -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_rb.snap index 38d3a63a..05df05ba 100644 --- a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_rb.snap +++ b/crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_rb.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -# Generated by WeaveFFI [VERSION] from api.yml +# Generated by WeaveFFI [VERSION] # DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o +# To regenerate: weaveffi generate # frozen_string_literal: true @@ -11,10 +11,19 @@ require_relative 'shapes/runtime' # Ruby bindings for shapes. module Shapes - # Contract checksums of the top-level modules these bindings were - # generated from; a library built from a different API fails to load. + # The declarations these bindings were generated with, per top-level + # module, as [id, hash, path]. Loading fails, naming the path, unless + # the library's contract table holds each one with an equal hash. _wv_check_contract!( - 'shapes' => [:shapes_shapes_checksum, 0x06acf433ffa2a0f0], + shapes_shapes_contract: [ + [0x0d7b3eec49243e7a, 0x5f232311808e1b1f, 'shapes.Channel'], + [0x183fa8198100860e, 0xe40c6993cb31d359, 'shapes.describe'], + [0x2d1395eb837b848e, 0x42202a0c12f3a16e, 'shapes.sum_bytes'], + [0x7f9a5b87ba09b86b, 0xc5b32fb0efbbaa17, 'shapes.scale'], + [0xc8edd779240fda17, 0x68e1be78fddb20e4, 'shapes.Numerics'], + [0xd3d3881b2fd18a62, 0xf24d0eb76a99d918, 'shapes.make_numerics'], + [0xf010844afff5ac3c, 0x50de28ae9b41ff15, 'shapes.Shape'], + ], ) # === Module: shapes === @@ -30,10 +39,9 @@ module Shapes class Empty < Shape TAG = 0 - # Structural equality over the variant and its fields. + # Structural equality over every field. def ==(other) - return false unless other.is_a?(Empty) - true + other.is_a?(Empty) end end @@ -48,11 +56,10 @@ module Shapes @radius = radius end - # Structural equality over the variant and its fields. + # Structural equality over every field. def ==(other) - return false unless other.is_a?(Circle) - return false unless radius == other.radius - true + other.is_a?(Circle) && + radius == other.radius end end @@ -69,12 +76,11 @@ module Shapes @height = height end - # Structural equality over the variant and its fields. + # Structural equality over every field. def ==(other) - return false unless other.is_a?(Rectangle) - return false unless width == other.width - return false unless height == other.height - true + other.is_a?(Rectangle) && + width == other.width && + height == other.height end end @@ -94,13 +100,12 @@ module Shapes @weight = weight end - # Structural equality over the variant and its fields. + # Structural equality over every field. def ==(other) - return false unless other.is_a?(Bounded) - return false unless min == other.min - return false unless max == other.max - return false unless weight == other.weight - true + other.is_a?(Bounded) && + min == other.min && + max == other.max && + weight == other.weight end end @@ -117,12 +122,11 @@ module Shapes @count = count end - # Structural equality over the variant and its fields. + # Structural equality over every field. def ==(other) - return false unless other.is_a?(Labeled) - return false unless label == other.label - return false unless count == other.count - true + other.is_a?(Labeled) && + label == other.label && + count == other.count end end end @@ -171,18 +175,17 @@ module Shapes # Structural equality over every field. def ==(other) - return false unless other.is_a?(Numerics) - return false unless a == other.a - return false unless b == other.b - return false unless c == other.c - return false unless d == other.d - return false unless e == other.e - return false unless f == other.f - return false unless g == other.g - return false unless h == other.h - return false unless i == other.i - return false unless j == other.j - true + other.is_a?(Numerics) && + a == other.a && + b == other.b && + c == other.c && + d == other.d && + e == other.e && + f == other.f && + g == other.g && + h == other.h && + i == other.i && + j == other.j end end @@ -204,17 +207,18 @@ module Shapes # @api private # Unpacks a Numerics from the value-buffer wire format. def self._wv_read_numerics(r) - _wv_a = r.read_i8 - _wv_b = r.read_i16 - _wv_c = r.read_i32 - _wv_d = r.read_i64 - _wv_e = r.read_u8 - _wv_f = r.read_u16 - _wv_g = r.read_u32 - _wv_h = r.read_u64 - _wv_i = r.read_f32 - _wv_j = r.read_f64 - Numerics.new(a: _wv_a, b: _wv_b, c: _wv_c, d: _wv_d, e: _wv_e, f: _wv_f, g: _wv_g, h: _wv_h, i: _wv_i, j: _wv_j) + Numerics.new( + a: r.read_i8, + b: r.read_i16, + c: r.read_i32, + d: r.read_i64, + e: r.read_u8, + f: r.read_u16, + g: r.read_u32, + h: r.read_u64, + i: r.read_f32, + j: r.read_f64, + ) end # @api private @@ -240,7 +244,7 @@ module Shapes w.write_string(v.label) w.write_u8(v.count) else - raise Error.new(MARSHAL_ERROR_CODE, 'unknown Shape variant') + raise TypeError, "expected a Shape, got #{v.class}" end end @@ -252,77 +256,63 @@ module Shapes when 0 Shape::Empty.new when 1 - _wv_radius = r.read_f64 - Shape::Circle.new(radius: _wv_radius) + Shape::Circle.new(radius: r.read_f64) when 2 - _wv_width = r.read_f32 - _wv_height = r.read_f32 - Shape::Rectangle.new(width: _wv_width, height: _wv_height) + Shape::Rectangle.new(width: r.read_f32, height: r.read_f32) when 3 - _wv_min = r.read_i16 - _wv_max = r.read_i16 - _wv_weight = r.read_u64 - Shape::Bounded.new(min: _wv_min, max: _wv_max, weight: _wv_weight) + Shape::Bounded.new(min: r.read_i16, max: r.read_i16, weight: r.read_u64) when 4 - _wv_label = r.read_string - _wv_count = r.read_u8 - Shape::Labeled.new(label: _wv_label, count: _wv_count) + Shape::Labeled.new(label: r.read_string, count: r.read_u8) else - raise Error.new(MARSHAL_ERROR_CODE, "malformed value buffer: unknown Shape tag #{tag}") + raise _wv_malformed("unknown Shape tag #{tag}") end end - attach_function :shapes_shapes_describe, [:pointer, :size_t, :pointer, :pointer], :pointer - attach_function :shapes_shapes_scale, [:pointer, :size_t, :double, :pointer, :pointer], :pointer - attach_function :shapes_shapes_sum_bytes, [:pointer, :size_t, :pointer], :uint64 - attach_function :shapes_shapes_make_numerics, [:int8, :int16, :int32, :int64, :uint8, :uint16, :uint32, :uint64, :float, :double, :pointer, :pointer], :pointer + attach_function :shapes_shapes_describe, [:pointer, :size_t, :pointer, :pointer], :pointer, blocking: true + attach_function :shapes_shapes_scale, [:pointer, :size_t, :double, :pointer, :pointer], :pointer, blocking: true + attach_function :shapes_shapes_sum_bytes, [:pointer, :size_t, :pointer], :uint64, blocking: true + attach_function :shapes_shapes_make_numerics, [:int8, :int16, :int32, :int64, :uint8, :uint16, :uint32, :uint64, :float, :double, :pointer, :pointer], :pointer, blocking: true # Render a shape to a string + # @return [String] def self.describe(shape) - shape_w = WvBufferWriter.new - _wv_write_shape(shape_w, shape) + shape_s = _wv_encode { |w| _wv_write_shape(w, shape) } err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - shape_s = _wv_seal(shape_w).first result = shapes_shapes_describe(shape_s, shape_s.bytesize, out_len, err) - check_error!(err) + _wv_check!(err) _wv_take_string(result, out_len.read(:size_t)) end # Scale a shape by a factor, returning a new shape + # @return [Shape] def self.scale(shape, factor) - shape_w = WvBufferWriter.new - _wv_write_shape(shape_w, shape) + shape_s = _wv_encode { |w| _wv_write_shape(w, shape) } err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) - shape_s = _wv_seal(shape_w).first result = shapes_shapes_scale(shape_s, shape_s.bytesize, factor, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_shape(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_shape(r) } end # Sum a list of bytes into a wide integer (numerics smoke) + # @return [Integer] def self.sum_bytes(values) values_s = _wv_bytes(values) err = ErrorStruct.new result = shapes_shapes_sum_bytes(values_s, values_s.bytesize, err) - check_error!(err) + _wv_check!(err) result end # Build a Numerics struct from every numeric primitive + # @return [Numerics] def self.make_numerics(a, b, c, d, e, f, g, h, i, j) err = ErrorStruct.new out_len = FFI::MemoryPointer.new(:size_t) result = shapes_shapes_make_numerics(a, b, c, d, e, f, g, h, i, j, out_len, err) - check_error!(err) - _wv_decode(result, out_len.read(:size_t)) do |_wv_r| - _wv_value = _wv_read_numerics(_wv_r) - _wv_value - end + _wv_check!(err) + _wv_decode(result, out_len.read(:size_t)) { |r| _wv_read_numerics(r) } end end diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_runtime_rb.snap b/crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_runtime_rb.snap deleted file mode 100644 index 3a154ba9..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__lib_shapes_runtime_rb.snap +++ /dev/null @@ -1,705 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -# frozen_string_literal: true - -require 'ffi' -require 'monitor' - -# The fixed runtime every generated Shapes binding builds on: the native -# library loader, the load-time ABI check, the error classes, the -# value-buffer reader and writer, cancel tokens, the handle table behind -# callback interfaces and async completions, and the object wrapper base. -module Shapes - extend FFI::Library - - # An explicit path in SHAPES_LIBRARY wins. Otherwise a library bundled - # under lib/native/ (a platform gem) is preferred over the system search - # path. - _wv_override = ENV['SHAPES_LIBRARY'] - if _wv_override && !_wv_override.empty? - ffi_lib _wv_override - else - _wv_name = - case FFI::Platform::OS - when /darwin/ then 'libshapes.dylib' - when /mswin|mingw|windows/ then 'shapes.dll' - else 'libshapes.so' - end - _wv_bundled = File.expand_path(File.join('..', 'native', _wv_name), __dir__) - ffi_lib(File.exist?(_wv_bundled) ? _wv_bundled : _wv_name) - end - - # The C ABI revision these bindings speak. Checked before anything else is - # attached, so a mismatched library fails at require time instead of - # misreading an error struct or a value buffer later. - ABI_VERSION = 3 - begin - attach_function :shapes_abi_version, [], :uint32 - rescue FFI::NotFoundError - raise LoadError, "the loaded shapes library does not export shapes_abi_version " \ - "(these bindings expect ABI revision #{ABI_VERSION})" - end - _wv_abi = shapes_abi_version - unless _wv_abi == ABI_VERSION - raise LoadError, "shapes ABI mismatch: these bindings expect revision #{ABI_VERSION} " \ - "but the loaded library reports revision #{_wv_abi}" - end - - # The `shapes_error` out-parameter every call reports through. - class ErrorStruct < FFI::Struct - layout :code, :int32, - :message, :pointer, - :payload_ptr, :pointer, - :payload_len, :size_t - end - - # Codes reserved by the runtime. Positive codes belong to a module's error - # domain; these negative codes are raised as a plain Error (or Cancelled) - # regardless of whether the function declares errors. - GENERIC_ERROR_CODE = -1 - PANIC_ERROR_CODE = -2 - MARSHAL_ERROR_CODE = -3 - # A callback-interface implementation raised; the message carries the - # exception text. - FOREIGN_ERROR_CODE = -4 - # The call was cancelled through its CancelToken. - CANCELLED_ERROR_CODE = -5 - - # The base class of every error the library raises. - class Error < StandardError - # The ABI error code: positive for a domain error, negative for a runtime - # failure. - attr_reader :code - - def initialize(code, message) - @code = code - super(message) - end - end - - # Raised when a cancellable call completes because its CancelToken fired. - class Cancelled < Error - def initialize(message = 'cancelled') - super(CANCELLED_ERROR_CODE, message) - end - end - - attach_function :shapes_error_set, [:pointer, :int32, :string], :void - attach_function :shapes_error_clear, [:pointer], :void - attach_function :shapes_error_free, [:pointer], :void - attach_function :shapes_free_bytes, [:pointer, :size_t], :void - attach_function :shapes_cancel_token_create, [], :pointer - attach_function :shapes_cancel_token_cancel, [:pointer], :void - attach_function :shapes_cancel_token_is_cancelled, [:pointer], :bool - attach_function :shapes_cancel_token_destroy, [:pointer], :void - - # @api private - # The error for a code outside any declared domain. - def self._wv_error(code, message) - return Cancelled.new(message.empty? ? 'cancelled' : message) if code == CANCELLED_ERROR_CODE - - Error.new(code, message) - end - - # @api private - # Copies the code, message, and payload out of a reported error and clears - # it. Returns nil when the struct reports success. - def self._wv_take_error(err) - code = err[:code] - return nil if code.zero? - - msg_ptr = err[:message] - message = msg_ptr.null? ? '' : msg_ptr.read_string.force_encoding(Encoding::UTF_8) - payload_ptr = err[:payload_ptr] - payload = payload_ptr.null? ? nil : payload_ptr.read_string(err[:payload_len]) - shapes_error_clear(err.to_ptr) - [code, message, payload] - end - - # @api private - # Like _wv_take_error for the heap-boxed error an async completion - # receives; the box is released with shapes_error_free. - def self._wv_take_boxed_error(err_ptr) - return nil if err_ptr.null? - - taken = _wv_take_error(ErrorStruct.new(err_ptr)) - shapes_error_free(err_ptr) - taken - end - - # @api private - # Raises the generic error for a call that declares no error domain. - def self.check_error!(err) - taken = _wv_take_error(err) - return if taken.nil? - - raise _wv_error(taken[0], taken[1]) - end - - # @api private - # The UTF-8 bytes of a String argument, as a private binary copy the - # producer borrows for the call. - def self._wv_str(value) - value.to_s.encode(Encoding::UTF_8).b - end - - # @api private - # A private binary copy of a bytes argument. - def self._wv_bytes(value) - value.to_s.b - end - - # @api private - # Copies a borrowed (ptr, len) run without releasing it. - def self._wv_borrow_bytes(ptr, len) - ptr.null? || len.zero? ? ''.b : ptr.read_string(len) - end - - # @api private - # Copies a returned (ptr, len) allocation, then releases it. - def self._wv_take_bytes(ptr, len) - return ''.b if ptr.null? - - data = len.zero? ? ''.b : ptr.read_string(len) - shapes_free_bytes(ptr, len) - data - end - - # @api private - # Copies a returned UTF-8 string, then releases it. - def self._wv_take_string(ptr, len) - _wv_take_bytes(ptr, len).force_encoding(Encoding::UTF_8) - end - - # A cancellation signal for a cancellable async call. Pass one as the - # `cancel:` keyword, then call #cancel from any thread; the call raises - # Cancelled unless it already completed. A token may be shared by several - # calls and stays cancelled once cancelled. - class CancelToken - # @api private - # Owns the consumer's reference to the native token. - class Ptr < FFI::AutoPointer - def self.release(ptr) - Shapes.shapes_cancel_token_destroy(ptr) - end - end - - def initialize - @ptr = Ptr.new(Shapes.shapes_cancel_token_create) - end - - # Requests cancellation. Idempotent. - def cancel - Shapes.shapes_cancel_token_cancel(@ptr) unless @ptr.nil? - nil - end - - # Whether #cancel has been called. - def cancelled? - !@ptr.nil? && Shapes.shapes_cancel_token_is_cancelled(@ptr) - end - - # Releases the native token now rather than at GC time. Calls already - # launched with it keep their own reference. - def close - return if @ptr.nil? - - @ptr.free - @ptr = nil - end - - # @api private - def _wv_ptr - raise Error.new(GENERIC_ERROR_CODE, 'CancelToken used after close') if @ptr.nil? - - @ptr - end - end - - # @api private - # Appends values in the value-buffer wire format: little-endian, packed, - # no alignment. - class WvBufferWriter - def initialize - @buf = +''.b - @objects = [] - end - - # Reserves an object token. The strong reference it carries is minted - # by #seal, just before the buffer is handed to the producer, so an - # encoding abandoned on the way never strands a reference. - def write_object(object) - raise TypeError, "expected a wrapper object, got #{object.class}" unless object.is_a?(WvObject) - - @objects << [@buf.bytesize, object] - write_u64(0) - end - - # Mints the reference each reserved object token carries, recording it - # in `minted`, and returns the finished encoding. - def seal(minted) - @objects.each do |offset, object| - ptr = object._wv_clone_ptr - minted << [object.class, ptr] - @buf[offset, 8] = [ptr.address].pack('Q<') - end - @buf - end - - # Every append is a binary String, so the buffer stays binary and byte - # offsets index it directly. - def write_bool(value) - @buf << [value ? 1 : 0].pack('C') - end - - def write_flag(value) - write_bool(value) - end - - def write_i8(value) - @buf << [value].pack('c') - end - - def write_u8(value) - @buf << [value].pack('C') - end - - def write_i16(value) - @buf << [value].pack('s<') - end - - def write_u16(value) - @buf << [value].pack('S<') - end - - def write_i32(value) - @buf << [value].pack('l<') - end - - def write_u32(value) - @buf << [value].pack('L<') - end - - def write_len(value) - write_u32(value) - end - - def write_i64(value) - @buf << [value].pack('q<') - end - - def write_u64(value) - @buf << [value].pack('Q<') - end - - def write_f32(value) - @buf << [value].pack('e') - end - - def write_f64(value) - @buf << [value].pack('E') - end - - def write_string(value) - write_bytes(value.to_s.encode(Encoding::UTF_8)) - end - - def write_bytes(value) - bytes = value.to_s.b - write_u32(bytes.bytesize) - @buf << bytes - end - end - - # @api private - # Reads values in the value-buffer wire format, raising Error on any - # malformed buffer. - class WvBufferReader - def initialize(data) - @data = data.to_s.b - @pos = 0 - end - - def take(count, what) - if @pos + count > @data.bytesize - raise Error.new(MARSHAL_ERROR_CODE, "malformed value buffer: truncated #{what}") - end - - bytes = @data.byteslice(@pos, count) - @pos += count - bytes - end - - def read_bool - byte = take(1, 'bool').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: bool byte out of range') if byte > 1 - - byte == 1 - end - - def read_flag - byte = take(1, 'option flag').unpack1('C') - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: option flag out of range') if byte > 1 - - byte == 1 - end - - def read_i8 - take(1, 'i8').unpack1('c') - end - - def read_u8 - take(1, 'u8').unpack1('C') - end - - def read_i16 - take(2, 'i16').unpack1('s<') - end - - def read_u16 - take(2, 'u16').unpack1('S<') - end - - def read_i32 - take(4, 'i32').unpack1('l<') - end - - def read_u32 - take(4, 'u32').unpack1('L<') - end - - # A u32 element count. Counts are not checked against the remaining - # bytes, since an element can encode to zero bytes; a short buffer still - # fails on the first truncated read. - def read_len - read_u32 - end - - def read_i64 - take(8, 'i64').unpack1('q<') - end - - def read_u64 - take(8, 'u64').unpack1('Q<') - end - - def read_f32 - take(4, 'f32').unpack1('e') - end - - def read_f64 - take(8, 'f64').unpack1('E') - end - - def read_string - text = read_bytes.force_encoding(Encoding::UTF_8) - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: string is not valid UTF-8') unless text.valid_encoding? - - text - end - - def read_bytes - take(read_u32, 'byte run') - end - - # An object token: a non-zero u64 carrying one strong reference the - # caller adopts into a wrapper. - def read_object_token - addr = read_u64 - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: null object token') if addr.zero? - - FFI::Pointer.new(addr) - end - - def expect_end! - return if @pos == @data.bytesize - - raise Error.new(MARSHAL_ERROR_CODE, 'malformed value buffer: trailing bytes after value') - end - end - - # @api private - # Seals the writers of one call's object-carrying buffers (see - # WvBufferWriter#seal) and returns their encodings. If any reference can't - # be minted (a closed wrapper), the ones already minted are released before - # the error propagates. The producer adopts every minted reference once the - # call is made. - def self._wv_seal(*writers) - minted = [] - writers.map { |writer| writer.seal(minted) } - rescue Exception # rubocop:disable Lint/RescueException - minted.each { |cls, ptr| cls::WV_PTR.release(ptr) } - raise - end - - # @api private - # Decodes a returned value buffer: copies and releases it, yields a reader - # to the block, and checks that the block consumed every byte. - def self._wv_decode(ptr, len) - reader = WvBufferReader.new(_wv_take_bytes(ptr, len)) - value = yield reader - reader.expect_end! - value - end - - # Ruby objects the producer refers to by an integer key: callback-interface - # implementations (the key is the vtable `ctx`) and pending async - # completions (the key is the completion `context`). The producer never - # sees a Ruby object address. A Monitor rather than a Mutex, so a - # finalizer that runs while this thread holds the lock can't deadlock it. - @wv_handles = {} - @wv_next_handle = 0 - @wv_handles_lock = Monitor.new - - # @api private - def self._wv_handle_put(value) - @wv_handles_lock.synchronize do - @wv_next_handle += 1 - @wv_handles[@wv_next_handle] = value - FFI::Pointer.new(@wv_next_handle) - end - end - - # @api private - def self._wv_handle_get(key) - @wv_handles_lock.synchronize { @wv_handles[key.address] } - end - - # @api private - def self._wv_handle_take(key) - @wv_handles_lock.synchronize { @wv_handles.delete(key.address) } - end - - # @api private - # Registers a callback-interface implementation and returns its `ctx`. - # Never raises, so nothing can strand the registration between this and - # the call that hands it to the producer (an unusable implementation fails - # when the producer calls it, with FOREIGN_ERROR_CODE). - def self._wv_cb_register(impl) - _wv_handle_put([:callback, impl]) - end - - # @api private - # Resolves a trampoline's `ctx` back to the registered implementation. - def self._wv_cb_lookup(ctx) - kind, impl = _wv_handle_get(ctx) - raise Error.new(FOREIGN_ERROR_CODE, 'callback context is not registered') unless kind == :callback - - impl - end - - # @api private - # Drops an implementation when the producer releases its last reference. - def self._wv_cb_free(ctx) - _wv_handle_take(ctx) - nil - end - - # @api private - # Reports a callback implementation's exception through `out_err`. The - # producer copies the message. - def self._wv_cb_fail(out_err, exception) - message = exception.message.to_s - message = exception.class.name if message.empty? - message = message.encode(Encoding::UTF_8, invalid: :replace, undef: :replace).delete("\0") - shapes_error_set(out_err, FOREIGN_ERROR_CODE, message) - nil - end - - # @api private - # Registers a pending async call; returns its completion context and the - # queue its completion is delivered on. - def self._wv_async_begin - queue = Queue.new - [_wv_handle_put([:async, queue]), queue] - end - - # @api private - # The queue of the pending call a completion belongs to. Each context - # completes exactly once, so its entry is removed here. - def self._wv_async_finish(ctx) - kind, queue = _wv_handle_take(ctx) - kind == :async ? queue : Queue.new - end - - # @api private - # Blocks until a pending call completes, raising its error. If the wait is - # interrupted (Thread#raise, Thread#kill, Timeout), the call's token is - # cancelled before the interruption propagates. - def self._wv_async_wait(queue, token = nil) - value = - begin - queue.pop - rescue Exception # rubocop:disable Lint/RescueException - token&.cancel - raise - end - raise value if value.is_a?(Exception) - - value - end - - # @api private - # The base of every interface wrapper. A wrapper owns one strong reference - # to its native object and releases it exactly once, from #close or (as a - # backstop) from the GC finalizer of its FFI::AutoPointer. Calls borrow the - # pointer through #_wv_call, so a #close that races an in-flight call on - # another thread only releases the reference once the call returns. - class WvObject - # @api private - # Adopts one strong reference the producer handed over. - def self._from_ptr(ptr) - obj = allocate - obj._wv_init(ptr) - obj - end - - # @api private - def _wv_init(ptr) - @wv_handle = self.class::WV_PTR.new(ptr) - @wv_lock = Mutex.new - @wv_calls = 0 - @wv_closing = false - end - - # The native object pointer (borrowed; valid until #close). - def handle - handle = @wv_handle - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if handle.nil? || @wv_closing - - handle - end - - # Whether #close has released this wrapper's reference. - def closed? - @wv_handle.nil? || @wv_closing - end - - # Releases this wrapper's reference now rather than at GC time. - # Idempotent. The object itself lives until its last reference anywhere - # (another wrapper, a record field, the producer) goes. - def close - handle = @wv_lock.synchronize do - next nil if @wv_handle.nil? || @wv_closing - - if @wv_calls.zero? - taken = @wv_handle - @wv_handle = nil - taken - else - @wv_closing = true - nil - end - end - handle&.free - nil - end - - # @api private - # Yields the native pointer, keeping the reference alive until the block - # returns. - def _wv_call - handle = @wv_lock.synchronize do - raise Error.new(GENERIC_ERROR_CODE, "#{self.class.name} used after close") if @wv_handle.nil? || @wv_closing - - @wv_calls += 1 - @wv_handle - end - begin - yield handle - ensure - _wv_leave - end - end - - # @api private - # Mints a new strong reference the caller owns (an object written into a - # value buffer carries its own reference). - def _wv_clone_ptr - _wv_call { |ptr| self.class._wv_clone(ptr) } - end - - # dup and clone produce an independent wrapper with its own reference to - # the same object. - def initialize_copy(other) - super - _wv_init(other._wv_clone_ptr) - end - - private - - def _wv_leave - release = @wv_lock.synchronize do - @wv_calls -= 1 - next nil unless @wv_closing && @wv_calls.zero? - - taken = @wv_handle - @wv_handle = nil - @wv_closing = false - taken - end - release&.free - end - end - - # @api private - # Yields the native pointers of `objects` (NULL for nil), each kept alive - # until the block returns. - def self._wv_pin(*objects, &block) - _wv_pin_from(objects, 0, [], &block) - end - - # @api private - def self._wv_pin_from(objects, index, pointers, &block) - return yield(*pointers) if index == objects.length - - object = objects[index] - if object.nil? - _wv_pin_from(objects, index + 1, pointers + [FFI::Pointer::NULL], &block) - else - object._wv_call { |ptr| _wv_pin_from(objects, index + 1, pointers + [ptr], &block) } - end - end - - # @api private - # Checks each top-level module's contract checksum (`module name => - # [checksum function, expected value]`) against the library, so bindings - # generated from a different API fail at require time. - def self._wv_check_contract!(checksums) - checksums.each do |mod, (symbol, expected)| - begin - attach_function symbol, [], :uint64 - rescue FFI::NotFoundError - raise LoadError, "the loaded shapes library has no module '#{mod}' (#{symbol} is missing)" - end - actual = public_send(symbol) - next if actual == expected - - raise LoadError, "shapes module '#{mod}' does not match these bindings: expected contract " \ - "checksum 0x#{expected.to_s(16)}, the library reports 0x#{actual.to_s(16)}; " \ - 'regenerate the bindings' - end - end - - # @api private - # Live native resources of one kind (0 objects, 1 callbacks, 2 iterators, - # 3 cancel tokens, 4 byte allocations). Only libraries built with the - # `leak-check` feature export the counter. - def self._wv_debug_live(kind) - unless respond_to?(:shapes_debug_live) - begin - attach_function :shapes_debug_live, [:int32], :uint64 - rescue FFI::NotFoundError - raise Error.new(GENERIC_ERROR_CODE, 'shapes was built without the leak-check feature') - end - end - shapes_debug_live(kind) - end -end - -# END runtime.rb diff --git a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__shapes_gemspec.snap b/crates/weaveffi-cli/tests/snapshots/ruby_shapes__shapes_gemspec.snap deleted file mode 100644 index e7740347..00000000 --- a/crates/weaveffi-cli/tests/snapshots/ruby_shapes__shapes_gemspec.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -# Generated by WeaveFFI [VERSION] from api.yml -# DO NOT EDIT. Your changes will be overwritten. -# To regenerate: weaveffi generate api.yml -o - -Gem::Specification.new do |s| - s.name = 'shapes' - s.version = '0.1.0' - s.summary = 'Native bindings for shapes' - s.authors = ['shapes'] - s.files = Dir['lib/**/*.rb'] - s.require_paths = ['lib'] - s.required_ruby_version = '>= 2.7' - - s.add_dependency 'ffi', '~> 1.15' -end - -# END shapes.gemspec diff --git a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Package_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Package_swift.snap deleted file mode 100644 index 683f7ccc..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Package_swift.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// swift-tools-version:5.9 -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -import Foundation -import PackageDescription - -// The C module `CDocsEverywhere` comes from one of two places. Next to this -// manifest, a prebuilt `CDocsEverywhere.xcframework` (assembled from the native -// libraries, see the README) wins. Otherwise `Sources/CDocsEverywhere` declares -// the generated header as a system library that links `docs_everywhere` from the -// linker search path, e.g. `swift build -Xlinker -L/path/to/lib`. -let xcframework = "CDocsEverywhere.xcframework" -let cModule: Target = FileManager.default.fileExists(atPath: "\(Context.packageDirectory)/\(xcframework)") - ? .binaryTarget(name: "CDocsEverywhere", path: xcframework) - : .systemLibrary(name: "CDocsEverywhere") - -let package = Package( - name: "DocsEverywhere", - platforms: [.macOS(.v10_15), .iOS(.v13), .tvOS(.v13), .watchOS(.v6)], - products: [ - .library(name: "DocsEverywhere", targets: ["DocsEverywhere"]), - ], - targets: [ - cModule, - .target(name: "DocsEverywhere", dependencies: ["CDocsEverywhere"]), - ] -) - -// END Package.swift diff --git a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_docs_everywhere_h.snap b/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_docs_everywhere_h.snap deleted file mode 100644 index 31ab85e2..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_docs_everywhere_h.snap +++ /dev/null @@ -1,196 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef DOCS_EVERYWHERE_H -#define DOCS_EVERYWHERE_H - -#include -#include -#include - -#ifndef DOCS_EVERYWHERE_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef DOCS_EVERYWHERE_BUILD -# define DOCS_EVERYWHERE_API __declspec(dllexport) -# else -# define DOCS_EVERYWHERE_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define DOCS_EVERYWHERE_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define DOCS_EVERYWHERE_API __attribute__((visibility("default"))) -# else -# define DOCS_EVERYWHERE_API -# endif -#endif - -#ifndef DOCS_EVERYWHERE_DEPRECATED -# if defined(_MSC_VER) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define DOCS_EVERYWHERE_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define DOCS_EVERYWHERE_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports docs_everywhere_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define DOCS_EVERYWHERE_ABI_VERSION 3u -DOCS_EVERYWHERE_API uint32_t docs_everywhere_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by docs_everywhere_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct docs_everywhere_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} docs_everywhere_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_set(docs_everywhere_error* err, int32_t code, const char* message); -DOCS_EVERYWHERE_API void docs_everywhere_error_clear(docs_everywhere_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - docs_everywhere_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_error_free(docs_everywhere_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -DOCS_EVERYWHERE_API void docs_everywhere_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct docs_everywhere_cancel_token docs_everywhere_cancel_token; -DOCS_EVERYWHERE_API docs_everywhere_cancel_token* docs_everywhere_cancel_token_create(void); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_cancel(docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API bool docs_everywhere_cancel_token_is_cancelled(const docs_everywhere_cancel_token* token); -DOCS_EVERYWHERE_API void docs_everywhere_cancel_token_destroy(docs_everywhere_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -DOCS_EVERYWHERE_API uint8_t* docs_everywhere_alloc(uint32_t size); -DOCS_EVERYWHERE_API void docs_everywhere_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -DOCS_EVERYWHERE_API uint64_t docs_everywhere_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define DOCS_EVERYWHERE_DOCS_CHECKSUM 0x7950eec1e9cfc819ull -DOCS_EVERYWHERE_API uint64_t docs_everywhere_docs_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with docs_everywhere_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * docs_everywhere_buffer.h declares a C struct and codec for each. - */ - -/** - * Lifecycle status of a document. - * - * Used to drive UI badges and filters. - */ -typedef enum { - /** Document is being edited */ - docs_everywhere_docs_Status_Draft = 0, - /** Document is publicly visible */ - docs_everywhere_docs_Status_Published = 1, - /** Document is hidden from listings */ - docs_everywhere_docs_Status_Archived = 2 -} docs_everywhere_docs_Status; -/** Error codes reported by throwing functions in the `docs` module tree. */ -typedef enum { - /** Returned when no document matches the requested id */ - docs_everywhere_docs_DocsErrors_not_found = 1, - /** Raised when attempting to move into a forbidden state */ - docs_everywhere_docs_DocsErrors_invalid_status = 2 -} docs_everywhere_docs_DocsErrors; -typedef struct docs_everywhere_docs_Notebook docs_everywhere_docs_Notebook; -/** - * Consumer-implemented observer of document-saved notifications. - * - * Implementations must be safe to call from any thread. - * - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct docs_everywhere_docs_SavedListener_vtable { - /** Fires after a document has been persisted successfully */ - void (*on_saved)(void* ctx, int64_t doc_id, int32_t revision, docs_everywhere_error* out_err); - void (*free)(void* ctx); -} docs_everywhere_docs_SavedListener_vtable; -typedef void (*docs_everywhere_docs_publish_async_callback)(void* context, docs_everywhere_error* err, bool result); - -// Module: docs -/** - * An ordered collection of documents. - * - * Notebooks own their documents and release them when destroyed. - */ -/** Create an empty notebook */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_new(docs_everywhere_error* out_err); -/** The largest number of documents one notebook will hold */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_max_documents(docs_everywhere_error* out_err); -/** Add a document, returning its position in the notebook */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_insert(const docs_everywhere_docs_Notebook* self, const uint8_t* document_ptr, size_t document_len, docs_everywhere_error* out_err); -/** Look up a document by identifier, failing when none matches */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_Notebook_find(const docs_everywhere_docs_Notebook* self, int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** The number of documents currently held */ -DOCS_EVERYWHERE_API int32_t docs_everywhere_docs_Notebook_size(const docs_everywhere_docs_Notebook* self, docs_everywhere_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -DOCS_EVERYWHERE_API docs_everywhere_docs_Notebook* docs_everywhere_docs_Notebook_clone(const docs_everywhere_docs_Notebook* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_Notebook_destroy(docs_everywhere_docs_Notebook* self); - -/** - * Create a brand new document. - * - * The document starts in `Draft` status; callers must publish - * it explicitly to make it visible. - */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_create_document(const uint8_t* title_ptr, size_t title_len, const uint8_t* tags_ptr, size_t tags_len, size_t* out_len, docs_everywhere_error* out_err); -/** Look up a document by identifier */ -DOCS_EVERYWHERE_API const uint8_t* docs_everywhere_docs_get_document(int64_t id, size_t* out_len, docs_everywhere_error* out_err); -/** Publish a document in the background */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_publish_async(int64_t id, docs_everywhere_docs_publish_async_callback callback, void* context); -/** Register a listener for saved notifications */ -DOCS_EVERYWHERE_API void docs_everywhere_docs_watch(void* listener_ctx, const docs_everywhere_docs_SavedListener_vtable* listener_vtable, docs_everywhere_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // DOCS_EVERYWHERE_H - -// END docs_everywhere.h diff --git a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_module_modulemap.snap b/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_module_modulemap.snap deleted file mode 100644 index 9cbbef82..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_CDocsEverywhere_module_modulemap.snap +++ /dev/null @@ -1,14 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module CDocsEverywhere [system] { - header "docs_everywhere.h" - link "docs_everywhere" - export * -} - -// END module.modulemap diff --git a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_DocsEverywhere_DocsEverywhere_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_DocsEverywhere_DocsEverywhere_swift.snap index 15326810..e36f0457 100644 --- a/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_DocsEverywhere_DocsEverywhere_swift.snap +++ b/crates/weaveffi-cli/tests/snapshots/swift_docs_everywhere__Sources_DocsEverywhere_DocsEverywhere_swift.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import CDocsEverywhere import Foundation @@ -11,7 +11,7 @@ import Foundation // MARK: - Runtime support /// A failure the native library reports outside any declared error domain: -/// an unknown error code or a runtime trap. Throwing wrappers raise it; +/// an unknown error code or a runtime failure. Throwing wrappers raise it; /// non-throwing wrappers stop the process with `fatalError` instead, since /// Swift has no unchecked errors. public struct DocsEverywhereRuntimeError: Error, LocalizedError, Hashable, Sendable { @@ -42,29 +42,43 @@ public struct DocsEverywhereRuntimeError: Error, LocalizedError, Hashable, Senda typealias WvError = docs_everywhere_error /// The C ABI revision these bindings were generated against. -let wvAbiVersion: UInt32 = 3 +let wvAbiVersion: UInt32 = 4 -/// The contract check, run once before the first native call: the library -/// must implement this ABI revision and every top-level module must carry -/// the checksum these bindings were generated from. -let wvContract: Void = { +/// Runs the load-time checks if they haven't run yet. +@inline(__always) +func wvLoad() { + _ = wvContract +} + +/// Stops the process unless the library implements the ABI revision these +/// bindings were generated against. +func wvCheckAbiVersion() { let abi = docs_everywhere_abi_version() guard abi == wvAbiVersion else { fatalError("DocsEverywhere: the native library 'docs_everywhere' implements C ABI revision \(abi), but these bindings need revision \(wvAbiVersion)") } - wvCheckModule("docs", docs_everywhere_docs_checksum(), 0x7950eec1e9cfc819) -}() - -/// Runs the contract check if it hasn't run yet. -@inline(__always) -func wvLoad() { - _ = wvContract } -/// Stops the process when a module's contract checksum doesn't match. -func wvCheckModule(_ module: StaticString, _ actual: UInt64, _ expected: UInt64) { - guard actual == expected else { - fatalError("DocsEverywhere: module '\(module)' of the native library 'docs_everywhere' doesn't match these bindings (checksum 0x\(String(actual, radix: 16)), expected 0x\(String(expected, radix: 16))); regenerate the bindings or rebuild the library") +/// Stops the process unless every declaration these bindings were generated +/// with is in a top-level module's contract table with an equal signature +/// hash. Declarations the library adds are fine. +func wvCheckContract( + _ table: (UnsafeMutablePointer?) -> UnsafePointer?, + _ expected: [(id: UInt64, hash: UInt64, path: StaticString)] +) { + var len = 0 + let entries = UnsafeBufferPointer(start: table(&len), count: len) + var hashes: [UInt64: UInt64] = [:] + for entry in entries { + hashes[entry.id] = entry.hash + } + for (id, hash, path) in expected { + guard let actual = hashes[id] else { + fatalError("DocsEverywhere: \(path) is missing from the library 'docs_everywhere'; regenerate the bindings or rebuild the library") + } + guard actual == hash else { + fatalError("DocsEverywhere: \(path) changed since these bindings were generated; regenerate the bindings or rebuild the library 'docs_everywhere'") + } } } @@ -160,10 +174,12 @@ func wvWithBytes(_ d: Data, _ body: (UnsafePointer?, Int) throws -> R) try d.withUnsafeBytes { try body($0.baseAddress?.assumingMemoryBound(to: UInt8.self), $0.count) } } -/// Lends an encoded value buffer for one call. +/// Encodes `value` as a value buffer and lends it for one call. @inline(__always) -func wvWithBuffer(_ w: WvWriter, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { - try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } +func wvWithEncoded(_ value: T, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { + var w = WvWriter() + w.write(value) + return try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } } /// Copies a returned string, then releases the library's allocation. @@ -181,11 +197,11 @@ func wvTakeBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a returned value buffer in place, then releases it. -func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { defer { if let ptr = ptr { docs_everywhere_free_bytes(UnsafeMutablePointer(mutating: ptr), len) } } - return wvBorrowBuffer(ptr, len, decode) + return wvBorrowBuffer(ptr, len, as: type) } /// Copies a borrowed string. @@ -201,9 +217,9 @@ func wvBorrowBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a borrowed value buffer in place. -func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { var r = WvReader(UnsafeRawPointer(ptr), len) - let value = decode(&r) + let value: T = r.read() r.finish() return value } @@ -262,6 +278,12 @@ struct WvWriter { /// Writes an object token. `p` must be a strong reference the buffer now /// owns (a freshly cloned pointer), never one a wrapper still holds. mutating func writeObject(_ p: OpaquePointer) { append(UInt64(UInt(bitPattern: p))) } + + /// Writes any value that crosses inside a value buffer. + @inline(__always) + mutating func write(_ value: T) { + value.wvWrite(&self) + } } /// Deserializes values from the value-buffer wire format directly from the @@ -355,11 +377,154 @@ struct WvReader { return p } + /// Reads any value that crosses inside a value buffer, typed by context. + @inline(__always) + mutating func read() -> T { + T.wvRead(&self) + } + func finish() { if remaining != 0 { wvDecodeFailure("trailing bytes after value") } } } +// MARK: Value-buffer codecs + +/// A type that crosses the C ABI inside a value buffer. Every primitive, +/// optional, list, and map conforms here; each record, enum, and interface +/// conforms where it's declared. +protocol WvCodable { + /// Reads one value, adopting the reference of any object token. + static func wvRead(_ r: inout WvReader) -> Self + /// Writes this value, writing a fresh reference for any object. + func wvWrite(_ w: inout WvWriter) +} + +/// A C-style enum, which crosses as its `i32` discriminant. +protocol WvCEnum: WvCodable, RawRepresentable where RawValue == Int32 {} + +extension WvCEnum { + static func wvRead(_ r: inout WvReader) -> Self { wvEnumCase(Self.self, r.readI32()) } + func wvWrite(_ w: inout WvWriter) { w.writeI32(rawValue) } +} + +extension Bool: WvCodable { + static func wvRead(_ r: inout WvReader) -> Bool { r.readBool() } + func wvWrite(_ w: inout WvWriter) { w.writeBool(self) } +} + +extension Int8: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int8 { r.readI8() } + func wvWrite(_ w: inout WvWriter) { w.writeI8(self) } +} + +extension UInt8: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt8 { r.readU8() } + func wvWrite(_ w: inout WvWriter) { w.writeU8(self) } +} + +extension Int16: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int16 { r.readI16() } + func wvWrite(_ w: inout WvWriter) { w.writeI16(self) } +} + +extension UInt16: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt16 { r.readU16() } + func wvWrite(_ w: inout WvWriter) { w.writeU16(self) } +} + +extension Int32: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int32 { r.readI32() } + func wvWrite(_ w: inout WvWriter) { w.writeI32(self) } +} + +extension UInt32: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt32 { r.readU32() } + func wvWrite(_ w: inout WvWriter) { w.writeU32(self) } +} + +extension Int64: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int64 { r.readI64() } + func wvWrite(_ w: inout WvWriter) { w.writeI64(self) } +} + +extension UInt64: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt64 { r.readU64() } + func wvWrite(_ w: inout WvWriter) { w.writeU64(self) } +} + +extension Float: WvCodable { + static func wvRead(_ r: inout WvReader) -> Float { r.readF32() } + func wvWrite(_ w: inout WvWriter) { w.writeF32(self) } +} + +extension Double: WvCodable { + static func wvRead(_ r: inout WvReader) -> Double { r.readF64() } + func wvWrite(_ w: inout WvWriter) { w.writeF64(self) } +} + +extension String: WvCodable { + static func wvRead(_ r: inout WvReader) -> String { r.readString() } + func wvWrite(_ w: inout WvWriter) { w.writeString(self) } +} + +extension Data: WvCodable { + static func wvRead(_ r: inout WvReader) -> Data { r.readBytes() } + func wvWrite(_ w: inout WvWriter) { w.writeBytes(self) } +} + +extension Optional: WvCodable where Wrapped: WvCodable { + static func wvRead(_ r: inout WvReader) -> Wrapped? { + r.readOptionFlag() ? .some(r.read()) : nil + } + + func wvWrite(_ w: inout WvWriter) { + w.writeOptionFlag(self != nil) + if let value = self { w.write(value) } + } +} + +extension Array: WvCodable where Element: WvCodable { + static func wvRead(_ r: inout WvReader) -> [Element] { + let n = r.readCount() + var items: [Element] = [] + items.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0.. [Key: Value] { + let n = r.readCount() + var map: [Key: Value] = [:] + map.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0..: @unchecked Sendable { } } -/// Reports a thrown Swift error to the library as a callback failure -/// (code -4); the library aborts its current call with that code and message. -func wvForeignError(_ outErr: UnsafeMutablePointer?, _ error: Error) { - let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) - message.withCString { docs_everywhere_error_set(outErr, DocsEverywhereRuntimeError.foreignCode, $0) } +/// Holds one callback implementation behind the `ctx` the library keeps. +final class WvBox: @unchecked Sendable { + let value: T + + init(_ value: T) { + self.value = value + } +} + +/// Retains `impl` for the library: the `ctx` of a callback parameter, which +/// the vtable's `free` entry releases with ``wvRelease(_:as:)``. +func wvRetain(_ impl: T) -> UnsafeMutableRawPointer { + Unmanaged.passRetained(WvBox(impl)).toOpaque() +} + +/// Releases the implementation behind `ctx`, from the vtable's `free` entry. +func wvRelease(_ ctx: UnsafeMutableRawPointer?, as type: T.Type) { + Unmanaged>.fromOpaque(ctx!).release() +} + +/// Runs one callback method on the implementation behind `ctx`. A thrown +/// error never unwinds through the library: `domain` reports it when it's +/// the method's declared error, anything else is reported as a callback +/// failure (`-4`), and the method returns `fallback`. +func wvInvoke( + _ ctx: UnsafeMutableRawPointer?, + _ outErr: UnsafeMutablePointer?, + as type: T.Type, + fallback: R, + domain: (Error, UnsafeMutablePointer?) -> Bool = { _, _ in false }, + _ body: (T) throws -> R +) -> R { + let impl = Unmanaged>.fromOpaque(ctx!).takeUnretainedValue().value + do { + return try body(impl) + } catch { + if !domain(error, outErr) { + let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) + message.withCString { docs_everywhere_error_set(outErr, DocsEverywhereRuntimeError.foreignCode, $0) } + } + return fallback + } +} + +/// Reports a declared domain error from a callback method: its code, its +/// message, and its fields as the payload. +func wvSetError(_ outErr: UnsafeMutablePointer?, _ code: Int32, _ message: String, _ payload: WvWriter? = nil) { + message.withCString { docs_everywhere_error_set(outErr, code, $0) } + if let payload = payload { + payload.bytes.withUnsafeBufferPointer { docs_everywhere_error_set_payload(outErr, $0.baseAddress, $0.count) } + } +} + +/// Hands `bytes` to the library as a callback method's return: a run +/// allocated with `docs_everywhere_alloc` that the library adopts. +func wvHandOver(_ bytes: UnsafeRawBufferPointer, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + let run = docs_everywhere_alloc(bytes.count) + if let run = run, let base = bytes.baseAddress { + run.initialize(from: base.assumingMemoryBound(to: UInt8.self), count: bytes.count) + } + outPtr?.pointee = run + outLen?.pointee = run == nil ? 0 : bytes.count +} + +/// Returns a string from a callback method through its out slots. +func wvReturnString(_ value: String, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var value = value + value.withUTF8 { wvHandOver(UnsafeRawBufferPointer($0), outPtr, outLen) } } +/// Returns bytes from a callback method through its out slots. +func wvReturnBytes(_ value: Data, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + value.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +/// Returns a value buffer from a callback method through its out slots. +func wvReturnBuffer(_ value: T, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var w = WvWriter() + w.write(value) + w.bytes.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +// MARK: - Load-time checks + +/// The load-time checks, run once before the first native call: the library +/// must implement C ABI revision 4 and carry every declaration these +/// bindings were generated with, unchanged. +let wvContract: Void = { + wvCheckAbiVersion() + wvCheckContract(docs_everywhere_docs_contract, [ + (0x00b2187fd75634ec, 0x2263c6f787bf6081, "docs.publish_async"), + (0x12fc242b9dc6444c, 0x96405f484a642b31, "docs.Notebook.insert"), + (0x2213822ed05cd5bd, 0x26c5833392aa63ce, "docs.Notebook.new"), + (0x2bd762bcdaa207ff, 0x1b0ebab2d04c49b0, "docs.SavedListener"), + (0x2e086c45e9d617de, 0x4d43bf32358c01ff, "docs.Notebook.find"), + (0x4817dc98197ad960, 0xf8017b5a7d58b2c7, "docs.Notebook.size"), + (0x614de47452d5a55c, 0xd9aae1472494ffef, "docs.get_document"), + (0x67c5cb9fc119bcd5, 0x08d5b25a1a22a7fa, "docs.Document"), + (0x8d484bf2d78c9b70, 0xe9093cf96912194c, "docs.Notebook.max_documents"), + (0xaea61d6451691c67, 0xe4704128b3524b3d, "docs.watch"), + (0xd68d6ad1540cdec5, 0x01afeb40b0a8bfe1, "docs.Notebook"), + (0xdb0601444a4760cc, 0xe56c69629b444797, "docs.create_document"), + (0xe12d6f4838347b3e, 0xa9bab86646d84f51, "docs.Status"), + (0xee2f8d25b3313b50, 0x9691232f35439a2b, "docs.DocsErrors"), + ]) +}() + // MARK: - API /// Typed errors reported by the `docs` module. @@ -435,7 +700,7 @@ public enum DocsErrorsError: Error, LocalizedError, Sendable { } } -func mapDocs(_ err: WvError) -> Error { +func wvMapDocs(_ err: WvError) -> Error { let message = wvErrorMessage(err) switch err.code { case 1: return DocsErrorsError.notFound(message: message.isEmpty ? "Document not found" : message) @@ -444,9 +709,9 @@ func mapDocs(_ err: WvError) -> Error { } } -func checkDocs(_ err: inout WvError) throws { +func wvCheckDocs(_ err: inout WvError) throws { guard err.code != 0 else { return } - let error = mapDocs(err) + let error = wvMapDocs(err) docs_everywhere_error_clear(&err) throw error } @@ -463,6 +728,8 @@ public enum Status: Int32, Sendable { case archived = 2 } +extension Status: WvCEnum {} + /// Represents a single document tracked by the system. /// /// Documents are persisted to disk and exposed via the public API. @@ -485,28 +752,17 @@ public struct Document: Hashable, Sendable { } } -func wvWriteDocument(_ value: Document, into w: inout WvWriter) { - w.writeI64(value.id) - w.writeString(value.title) - w.writeI32(value.status.rawValue) - w.writeLen(value.tags.count) - for v0 in value.tags { - w.writeString(v0) +extension Document: WvCodable { + static func wvRead(_ r: inout WvReader) -> Document { + Document(id: r.read(), title: r.read(), status: r.read(), tags: r.read()) } -} -func wvReadDocument(_ r: inout WvReader) -> Document { - let v1 = r.readI64() - let v2 = r.readString() - let v3 = wvEnumCase(Status.self, r.readI32()) - let n5 = r.readCount() - var v4: [String] = [] - v4.reserveCapacity(min(n5, r.remaining)) - for _ in 0.. Document { /// Implementations must be safe to call from any thread. /// /// Implement this protocol with a class and pass an instance where the API -/// expects it; the library keeps it alive for as long as it holds the -/// callback and may call any method from any thread. A thrown error aborts -/// the library's call and surfaces to the original caller as a foreign error -/// (code -4). +/// expects it. The library keeps the instance until it no longer needs it and +/// may call any method from any thread. A method that throws fails the +/// library's call in progress: a method documented to throw a module error +/// reports that error, with its code and fields, and any other error reaches +/// the library as a callback failure (code -4). public protocol SavedListener: AnyObject, Sendable { /// Fires after a document has been persisted successfully + /// /// - Parameter docId: Identifier of the saved document /// - Parameter revision: Monotonically increasing revision func onSaved(docId: Int64, revision: Int32) throws } -/// Retains one `SavedListener` implementation across the C boundary; the library's -/// `free` entry releases it. -final class WvSavedListenerBox: Sendable { - let impl: any SavedListener - init(_ impl: any SavedListener) { self.impl = impl } -} - -/// The process-wide `docs_everywhere_docs_SavedListener_vtable` the library calls `SavedListener` -/// implementations through. Every entry recovers the box from `ctx`, copies -/// or adopts its arguments, and reports a thrown error through `out_err` -/// instead of unwinding. +/// The process-wide vtable the library calls `SavedListener` implementations +/// through (`docs_everywhere_docs_SavedListener_vtable`). Every entry recovers the implementation +/// from `ctx`, copies or adopts its arguments, hands its result back, and +/// reports a thrown error through `out_err` instead of unwinding. enum WvSavedListenerVtable { - static let shared = WvVtable(docs_everywhere_docs_SavedListener_vtable( + static let shared = WvVtable(docs_everywhere_docs_SavedListener_vtable( + size: UInt32(MemoryLayout.stride), + flags: 0, + free: { ctx in + wvRelease(ctx, as: (any SavedListener).self) + }, on_saved: { ctx, doc_id, revision, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - try wvBox.impl.onSaved(docId: doc_id, revision: revision) - } catch { - wvForeignError(out_err, error) + wvInvoke(ctx, out_err, as: (any SavedListener).self, fallback: ()) { wvImpl in + try wvImpl.onSaved(docId: doc_id, revision: revision) } - }, - free: { ctx in - Unmanaged.fromOpaque(ctx!).release() } )) } @@ -567,7 +817,7 @@ public final class Notebook: @unchecked Sendable { } /// Returns a new strong reference to the same object, for a position that - /// takes ownership of it (an object token inside a value buffer). + /// takes ownership of it. func clonePtr() -> OpaquePointer { wvNonNull(docs_everywhere_docs_Notebook_clone(ptr)) } @@ -582,12 +832,11 @@ public final class Notebook: @unchecked Sendable { } /// Add a document, returning its position in the notebook + /// /// - Parameter document: The document to add public func insert(document: Document) -> Int32 { var err = WvError() - var documentWriter = WvWriter() - wvWriteDocument(document, into: &documentWriter) - let rv: Int32 = wvWithBuffer(documentWriter) { document_ptr, document_len in + let rv: Int32 = wvWithEncoded(document) { document_ptr, document_len in docs_everywhere_docs_Notebook_insert(ptr, document_ptr, document_len, &err) } wvTrap(&err) @@ -595,13 +844,15 @@ public final class Notebook: @unchecked Sendable { } /// Look up a document by identifier, failing when none matches + /// /// - Parameter id: Identifier of the document to find + /// - Throws: ``DocsErrorsError`` for a declared failure, ``DocsEverywhereRuntimeError`` otherwise. public func find(id: Int64) throws -> Document { var err = WvError() var outLen = 0 let rv = docs_everywhere_docs_Notebook_find(ptr, id, &outLen, &err) - try checkDocs(&err) - return wvTakeBuffer(rv, outLen, wvReadDocument) + try wvCheckDocs(&err) + return wvTakeBuffer(rv, outLen, as: Document.self) } /// The number of documents currently held @@ -622,32 +873,35 @@ public final class Notebook: @unchecked Sendable { } } +extension Notebook: WvCodable { + static func wvRead(_ r: inout WvReader) -> Notebook { Notebook(ptr: r.readObject()) } + func wvWrite(_ w: inout WvWriter) { w.writeObject(clonePtr()) } +} + +/// Document storage: create, publish, and observe documents. public enum Docs { /// Create a brand new document. /// /// The document starts in `Draft` status; callers must publish /// it explicitly to make it visible. + /// /// - Parameter title: Human-readable title for the new document /// - Parameter tags: Initial set of tags (may be empty) public static func createDocument(title: String, tags: [String]) -> Document { wvLoad() var err = WvError() - var tagsWriter = WvWriter() - tagsWriter.writeLen(tags.count) - for v0 in tags { - tagsWriter.writeString(v0) - } var outLen = 0 let rv: UnsafePointer? = wvWithUTF8(title) { title_ptr, title_len in - wvWithBuffer(tagsWriter) { tags_ptr, tags_len in + wvWithEncoded(tags) { tags_ptr, tags_len in docs_everywhere_docs_create_document(title_ptr, title_len, tags_ptr, tags_len, &outLen, &err) } } wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadDocument) + return wvTakeBuffer(rv, outLen, as: Document.self) } /// Look up a document by identifier + /// /// - Parameter id: Identifier returned from create_document public static func getDocument(id: Int64) -> Document? { wvLoad() @@ -655,17 +909,11 @@ public enum Docs { var outLen = 0 let rv = docs_everywhere_docs_get_document(id, &outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen) { (r: inout WvReader) -> Document? in - var v0: Document? = nil - if r.readOptionFlag() { - let v1 = wvReadDocument(&r) - v0 = v1 - } - return v0 - } + return wvTakeBuffer(rv, outLen, as: Document?.self) } /// Publish a document in the background + /// /// - Parameter id: Identifier of the document to publish public static func publishAsync(id: Int64) async -> Bool { wvLoad() @@ -682,11 +930,12 @@ public enum Docs { } /// Register a listener for saved notifications + /// /// - Parameter listener: Receiver of saved notifications - public static func watch(listener: SavedListener) { + public static func watch(listener: any SavedListener) { wvLoad() var err = WvError() - let listener_ctx = Unmanaged.passRetained(WvSavedListenerBox(listener)).toOpaque() + let listener_ctx = wvRetain(listener) docs_everywhere_docs_watch(listener_ctx, WvSavedListenerVtable.shared.pointer, &err) wvTrap(&err) } diff --git a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Package_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Package_swift.snap deleted file mode 100644 index abd466d1..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Package_swift.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// swift-tools-version:5.9 -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -import Foundation -import PackageDescription - -// The C module `CEdgeCases` comes from one of two places. Next to this -// manifest, a prebuilt `CEdgeCases.xcframework` (assembled from the native -// libraries, see the README) wins. Otherwise `Sources/CEdgeCases` declares -// the generated header as a system library that links `edge_cases` from the -// linker search path, e.g. `swift build -Xlinker -L/path/to/lib`. -let xcframework = "CEdgeCases.xcframework" -let cModule: Target = FileManager.default.fileExists(atPath: "\(Context.packageDirectory)/\(xcframework)") - ? .binaryTarget(name: "CEdgeCases", path: xcframework) - : .systemLibrary(name: "CEdgeCases") - -let package = Package( - name: "EdgeCases", - platforms: [.macOS(.v10_15), .iOS(.v13), .tvOS(.v13), .watchOS(.v6)], - products: [ - .library(name: "EdgeCases", targets: ["EdgeCases"]), - ], - targets: [ - cModule, - .target(name: "EdgeCases", dependencies: ["CEdgeCases"]), - ] -) - -// END Package.swift diff --git a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_edge_cases_h.snap b/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_edge_cases_h.snap deleted file mode 100644 index 94864086..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_edge_cases_h.snap +++ /dev/null @@ -1,245 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef EDGE_CASES_H -#define EDGE_CASES_H - -#include -#include -#include - -#ifndef EDGE_CASES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef EDGE_CASES_BUILD -# define EDGE_CASES_API __declspec(dllexport) -# else -# define EDGE_CASES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define EDGE_CASES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define EDGE_CASES_API __attribute__((visibility("default"))) -# else -# define EDGE_CASES_API -# endif -#endif - -#ifndef EDGE_CASES_DEPRECATED -# if defined(_MSC_VER) -# define EDGE_CASES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define EDGE_CASES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define EDGE_CASES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports edge_cases_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define EDGE_CASES_ABI_VERSION 3u -EDGE_CASES_API uint32_t edge_cases_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by edge_cases_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct edge_cases_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} edge_cases_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -EDGE_CASES_API void edge_cases_error_set(edge_cases_error* err, int32_t code, const char* message); -EDGE_CASES_API void edge_cases_error_clear(edge_cases_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - edge_cases_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -EDGE_CASES_API void edge_cases_error_free(edge_cases_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -EDGE_CASES_API void edge_cases_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct edge_cases_cancel_token edge_cases_cancel_token; -EDGE_CASES_API edge_cases_cancel_token* edge_cases_cancel_token_create(void); -EDGE_CASES_API void edge_cases_cancel_token_cancel(edge_cases_cancel_token* token); -EDGE_CASES_API bool edge_cases_cancel_token_is_cancelled(const edge_cases_cancel_token* token); -EDGE_CASES_API void edge_cases_cancel_token_destroy(edge_cases_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -EDGE_CASES_API uint8_t* edge_cases_alloc(uint32_t size); -EDGE_CASES_API void edge_cases_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -EDGE_CASES_API uint64_t edge_cases_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define EDGE_CASES_EDGE_CASES_CHECKSUM 0x1567f441073bd4aaull -EDGE_CASES_API uint64_t edge_cases_edge_cases_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with edge_cases_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * edge_cases_buffer.h declares a C struct and codec for each. - */ - -/** Deprecated enum with non-contiguous values and keyword-like variants */ -typedef enum { edge_cases_edge_cases_Legacy_None = 0, edge_cases_edge_cases_Legacy_Default = 5, edge_cases_edge_cases_Legacy_Class = 10 } edge_cases_edge_cases_Legacy; -typedef enum { edge_cases_edge_cases_Mode_Fast = 1, edge_cases_edge_cases_Mode_Safe = 2 } edge_cases_edge_cases_Mode; -/** Error codes reported by throwing functions in the `edge_cases` module tree. */ -typedef enum { edge_cases_edge_cases_EdgeErrors_Busy = 7 } edge_cases_edge_cases_EdgeErrors; -typedef struct edge_cases_edge_cases_Widget edge_cases_edge_cases_Widget; -typedef struct edge_cases_edge_cases_Vault edge_cases_edge_cases_Vault; -typedef struct edge_cases_edge_cases_WidgetStreamIterator edge_cases_edge_cases_WidgetStreamIterator; -typedef struct edge_cases_edge_cases_StringStreamIterator edge_cases_edge_cases_StringStreamIterator; -typedef struct edge_cases_edge_cases_ScalarStreamIterator edge_cases_edge_cases_ScalarStreamIterator; -typedef struct edge_cases_edge_cases_ModeStreamIterator edge_cases_edge_cases_ModeStreamIterator; -/** - * Callback interface carrying composite payloads and objects - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_BatchListener_vtable { - /** Composite payloads cross as value buffers */ - void (*on_batch)(void* ctx, const uint8_t* items_ptr, size_t items_len, const uint8_t* raw_ptr, size_t raw_len, const uint8_t* first_ptr, size_t first_len, edge_cases_edge_cases_Mode mode, edge_cases_error* out_err); - /** Objects cross as strong references the consumer must release */ - void (*on_widget)(void* ctx, edge_cases_edge_cases_Widget* widget, edge_cases_edge_cases_Widget* fallback, const uint8_t* children_ptr, size_t children_len, edge_cases_error* out_err); - void (*on_tick)(void* ctx, edge_cases_error* out_err); - /** Scalar return travels back through the vtable */ - bool (*should_continue)(void* ctx, int32_t class_, edge_cases_error* out_err); - /** Enum return travels back as its integer discriminant */ - edge_cases_edge_cases_Mode (*pick_mode)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_BatchListener_vtable; -/** - * Callback interface whose name and methods are keywords - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct edge_cases_edge_cases_Keyword_vtable { - void (*delete_)(void* ctx, int32_t self, edge_cases_error* out_err); - int32_t (*import)(void* ctx, edge_cases_error* out_err); - void (*free)(void* ctx); -} edge_cases_edge_cases_Keyword_vtable; -typedef void (*edge_cases_edge_cases_async_widget_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Widget* result); -typedef void (*edge_cases_edge_cases_async_void_callback)(void* context, edge_cases_error* err); -typedef void (*edge_cases_edge_cases_async_bytes_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_struct_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_list_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_optional_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_async_enum_callback)(void* context, edge_cases_error* err, edge_cases_edge_cases_Mode result); -typedef void (*edge_cases_edge_cases_async_keywords_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*edge_cases_edge_cases_Widget_fetch_callback)(void* context, edge_cases_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: edge_cases -/** Interface with no constructor; instances come from make_widget */ -/** Void method that can fail */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_refresh(const edge_cases_edge_cases_Widget* self, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_rename(const edge_cases_edge_cases_Widget* self, const uint8_t* class_ptr, size_t class_len, const uint8_t* import_ptr, size_t import_len, size_t* out_len, edge_cases_error* out_err); -/** Async method with a struct result */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_fetch(const edge_cases_edge_cases_Widget* self, int64_t id, edge_cases_edge_cases_Widget_fetch_callback callback, void* context); -EDGE_CASES_DEPRECATED("Use rename instead") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_Widget_old_name(const edge_cases_edge_cases_Widget* self, size_t* out_len, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Widget_clone(const edge_cases_edge_cases_Widget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Widget_destroy(edge_cases_edge_cases_Widget* self); - -/** Deprecated interface with a throwing constructor */ -/** Fails when the vault is busy */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_open(const uint8_t* path_ptr, size_t path_len, edge_cases_error* out_err); -/** Throwing static */ -EDGE_CASES_API bool edge_cases_edge_cases_Vault_probe(edge_cases_error* out_err); -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_Vault_peek(const edge_cases_edge_cases_Vault* self, edge_cases_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -EDGE_CASES_API edge_cases_edge_cases_Vault* edge_cases_edge_cases_Vault_clone(const edge_cases_edge_cases_Vault* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -EDGE_CASES_API void edge_cases_edge_cases_Vault_destroy(edge_cases_edge_cases_Vault* self); - -/** No parameters and no result */ -EDGE_CASES_API void edge_cases_edge_cases_noop(edge_cases_error* out_err); -/** Function named after a keyword whose parameters are all keywords */ -EDGE_CASES_API int32_t edge_cases_edge_cases_select(int32_t class_, const uint8_t* from_ptr, size_t from_len, const uint8_t* func_ptr, size_t func_len, const uint8_t* range_ptr, size_t range_len, bool type_, edge_cases_error* out_err); -/** Optional list of optional structs, the deepest legal nesting */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_nested_optional(const uint8_t* items_ptr, size_t items_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_optional_params(const uint8_t* text_ptr, size_t text_len, const uint8_t* count_ptr, size_t count_len, const uint8_t* outer_ptr, size_t outer_len, const uint8_t* tags_ptr, size_t tags_len, const uint8_t* blob_ptr, size_t blob_len, const uint8_t* mode_ptr, size_t mode_len, size_t* out_len, edge_cases_error* out_err); -/** Two callback interface parameters on one function */ -EDGE_CASES_API void edge_cases_edge_cases_register_batch(void* listener_ctx, const edge_cases_edge_cases_BatchListener_vtable* listener_vtable, void* keyword_ctx, const edge_cases_edge_cases_Keyword_vtable* keyword_vtable, edge_cases_error* out_err); -/** Objects in every buffered position */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_widgets_everywhere(const uint8_t* list_ptr, size_t list_len, const uint8_t* by_name_ptr, size_t by_name_len, const uint8_t* sparse_ptr, size_t sparse_len, const uint8_t* holder_ptr, size_t holder_len, size_t* out_len, edge_cases_error* out_err); -/** Iterator of objects */ -EDGE_CASES_API edge_cases_edge_cases_WidgetStreamIterator* edge_cases_edge_cases_widget_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_WidgetStreamIterator_next(edge_cases_edge_cases_WidgetStreamIterator* iter, edge_cases_edge_cases_Widget** out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_WidgetStreamIterator_destroy(edge_cases_edge_cases_WidgetStreamIterator* iter); -/** Async returning an object */ -EDGE_CASES_API void edge_cases_edge_cases_async_widget(edge_cases_edge_cases_async_widget_callback callback, void* context); -/** Factory for an interface with no constructor */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_make_widget(const uint8_t* label_ptr, size_t label_len, edge_cases_error* out_err); -/** Optional interface return */ -EDGE_CASES_API edge_cases_edge_cases_Widget* edge_cases_edge_cases_find_widget(int64_t id, edge_cases_error* out_err); -/** Interface parameter, including an optional one */ -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_inspect_widget(const edge_cases_edge_cases_Widget* widget, const edge_cases_edge_cases_Widget* fallback, size_t* out_len, edge_cases_error* out_err); -/** Iterator of strings */ -EDGE_CASES_API edge_cases_edge_cases_StringStreamIterator* edge_cases_edge_cases_string_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_StringStreamIterator_next(edge_cases_edge_cases_StringStreamIterator* iter, const uint8_t** out_item, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_StringStreamIterator_destroy(edge_cases_edge_cases_StringStreamIterator* iter); -/** Iterator of scalars */ -EDGE_CASES_API edge_cases_edge_cases_ScalarStreamIterator* edge_cases_edge_cases_scalar_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ScalarStreamIterator_next(edge_cases_edge_cases_ScalarStreamIterator* iter, int32_t* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ScalarStreamIterator_destroy(edge_cases_edge_cases_ScalarStreamIterator* iter); -/** Iterator of enums */ -EDGE_CASES_API edge_cases_edge_cases_ModeStreamIterator* edge_cases_edge_cases_mode_stream(edge_cases_error* out_err); -EDGE_CASES_API int32_t edge_cases_edge_cases_ModeStreamIterator_next(edge_cases_edge_cases_ModeStreamIterator* iter, edge_cases_edge_cases_Mode* out_item, edge_cases_error* out_err); -EDGE_CASES_API void edge_cases_edge_cases_ModeStreamIterator_destroy(edge_cases_edge_cases_ModeStreamIterator* iter); -/** Async with no result */ -EDGE_CASES_API void edge_cases_edge_cases_async_void(edge_cases_edge_cases_async_void_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_bytes(uint32_t size, edge_cases_edge_cases_async_bytes_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_struct(edge_cases_edge_cases_async_struct_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_list(edge_cases_cancel_token* cancel_token, edge_cases_edge_cases_async_list_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_optional(edge_cases_edge_cases_async_optional_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_enum(edge_cases_edge_cases_async_enum_callback callback, void* context); -EDGE_CASES_API void edge_cases_edge_cases_async_keywords(const uint8_t* class_ptr, size_t class_len, int32_t import, edge_cases_edge_cases_async_keywords_callback callback, void* context); -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_keywords_roundtrip(const uint8_t* value_ptr, size_t value_len, size_t* out_len, edge_cases_error* out_err); -EDGE_CASES_DEPRECATED("Use Mode-based APIs") -EDGE_CASES_API const uint8_t* edge_cases_edge_cases_legacy_value(edge_cases_edge_cases_Legacy value, size_t* out_len, edge_cases_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // EDGE_CASES_H - -// END edge_cases.h diff --git a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_module_modulemap.snap b/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_module_modulemap.snap deleted file mode 100644 index d8186d40..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_CEdgeCases_module_modulemap.snap +++ /dev/null @@ -1,14 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module CEdgeCases [system] { - header "edge_cases.h" - link "edge_cases" - export * -} - -// END module.modulemap diff --git a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_EdgeCases_EdgeCases_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_EdgeCases_EdgeCases_swift.snap index eac2dbe9..d7dbab81 100644 --- a/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_EdgeCases_EdgeCases_swift.snap +++ b/crates/weaveffi-cli/tests/snapshots/swift_edge_cases__Sources_EdgeCases_EdgeCases_swift.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import CEdgeCases import Foundation @@ -11,7 +11,7 @@ import Foundation // MARK: - Runtime support /// A failure the native library reports outside any declared error domain: -/// an unknown error code or a runtime trap. Throwing wrappers raise it; +/// an unknown error code or a runtime failure. Throwing wrappers raise it; /// non-throwing wrappers stop the process with `fatalError` instead, since /// Swift has no unchecked errors. public struct EdgeCasesRuntimeError: Error, LocalizedError, Hashable, Sendable { @@ -42,29 +42,43 @@ public struct EdgeCasesRuntimeError: Error, LocalizedError, Hashable, Sendable { typealias WvError = edge_cases_error /// The C ABI revision these bindings were generated against. -let wvAbiVersion: UInt32 = 3 +let wvAbiVersion: UInt32 = 4 -/// The contract check, run once before the first native call: the library -/// must implement this ABI revision and every top-level module must carry -/// the checksum these bindings were generated from. -let wvContract: Void = { +/// Runs the load-time checks if they haven't run yet. +@inline(__always) +func wvLoad() { + _ = wvContract +} + +/// Stops the process unless the library implements the ABI revision these +/// bindings were generated against. +func wvCheckAbiVersion() { let abi = edge_cases_abi_version() guard abi == wvAbiVersion else { fatalError("EdgeCases: the native library 'edge_cases' implements C ABI revision \(abi), but these bindings need revision \(wvAbiVersion)") } - wvCheckModule("edge_cases", edge_cases_edge_cases_checksum(), 0x1567f441073bd4aa) -}() - -/// Runs the contract check if it hasn't run yet. -@inline(__always) -func wvLoad() { - _ = wvContract } -/// Stops the process when a module's contract checksum doesn't match. -func wvCheckModule(_ module: StaticString, _ actual: UInt64, _ expected: UInt64) { - guard actual == expected else { - fatalError("EdgeCases: module '\(module)' of the native library 'edge_cases' doesn't match these bindings (checksum 0x\(String(actual, radix: 16)), expected 0x\(String(expected, radix: 16))); regenerate the bindings or rebuild the library") +/// Stops the process unless every declaration these bindings were generated +/// with is in a top-level module's contract table with an equal signature +/// hash. Declarations the library adds are fine. +func wvCheckContract( + _ table: (UnsafeMutablePointer?) -> UnsafePointer?, + _ expected: [(id: UInt64, hash: UInt64, path: StaticString)] +) { + var len = 0 + let entries = UnsafeBufferPointer(start: table(&len), count: len) + var hashes: [UInt64: UInt64] = [:] + for entry in entries { + hashes[entry.id] = entry.hash + } + for (id, hash, path) in expected { + guard let actual = hashes[id] else { + fatalError("EdgeCases: \(path) is missing from the library 'edge_cases'; regenerate the bindings or rebuild the library") + } + guard actual == hash else { + fatalError("EdgeCases: \(path) changed since these bindings were generated; regenerate the bindings or rebuild the library 'edge_cases'") + } } } @@ -160,10 +174,12 @@ func wvWithBytes(_ d: Data, _ body: (UnsafePointer?, Int) throws -> R) try d.withUnsafeBytes { try body($0.baseAddress?.assumingMemoryBound(to: UInt8.self), $0.count) } } -/// Lends an encoded value buffer for one call. +/// Encodes `value` as a value buffer and lends it for one call. @inline(__always) -func wvWithBuffer(_ w: WvWriter, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { - try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } +func wvWithEncoded(_ value: T, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { + var w = WvWriter() + w.write(value) + return try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } } /// Copies a returned string, then releases the library's allocation. @@ -181,11 +197,11 @@ func wvTakeBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a returned value buffer in place, then releases it. -func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { defer { if let ptr = ptr { edge_cases_free_bytes(UnsafeMutablePointer(mutating: ptr), len) } } - return wvBorrowBuffer(ptr, len, decode) + return wvBorrowBuffer(ptr, len, as: type) } /// Copies a borrowed string. @@ -201,9 +217,9 @@ func wvBorrowBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a borrowed value buffer in place. -func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { var r = WvReader(UnsafeRawPointer(ptr), len) - let value = decode(&r) + let value: T = r.read() r.finish() return value } @@ -262,6 +278,12 @@ struct WvWriter { /// Writes an object token. `p` must be a strong reference the buffer now /// owns (a freshly cloned pointer), never one a wrapper still holds. mutating func writeObject(_ p: OpaquePointer) { append(UInt64(UInt(bitPattern: p))) } + + /// Writes any value that crosses inside a value buffer. + @inline(__always) + mutating func write(_ value: T) { + value.wvWrite(&self) + } } /// Deserializes values from the value-buffer wire format directly from the @@ -355,11 +377,154 @@ struct WvReader { return p } + /// Reads any value that crosses inside a value buffer, typed by context. + @inline(__always) + mutating func read() -> T { + T.wvRead(&self) + } + func finish() { if remaining != 0 { wvDecodeFailure("trailing bytes after value") } } } +// MARK: Value-buffer codecs + +/// A type that crosses the C ABI inside a value buffer. Every primitive, +/// optional, list, and map conforms here; each record, enum, and interface +/// conforms where it's declared. +protocol WvCodable { + /// Reads one value, adopting the reference of any object token. + static func wvRead(_ r: inout WvReader) -> Self + /// Writes this value, writing a fresh reference for any object. + func wvWrite(_ w: inout WvWriter) +} + +/// A C-style enum, which crosses as its `i32` discriminant. +protocol WvCEnum: WvCodable, RawRepresentable where RawValue == Int32 {} + +extension WvCEnum { + static func wvRead(_ r: inout WvReader) -> Self { wvEnumCase(Self.self, r.readI32()) } + func wvWrite(_ w: inout WvWriter) { w.writeI32(rawValue) } +} + +extension Bool: WvCodable { + static func wvRead(_ r: inout WvReader) -> Bool { r.readBool() } + func wvWrite(_ w: inout WvWriter) { w.writeBool(self) } +} + +extension Int8: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int8 { r.readI8() } + func wvWrite(_ w: inout WvWriter) { w.writeI8(self) } +} + +extension UInt8: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt8 { r.readU8() } + func wvWrite(_ w: inout WvWriter) { w.writeU8(self) } +} + +extension Int16: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int16 { r.readI16() } + func wvWrite(_ w: inout WvWriter) { w.writeI16(self) } +} + +extension UInt16: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt16 { r.readU16() } + func wvWrite(_ w: inout WvWriter) { w.writeU16(self) } +} + +extension Int32: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int32 { r.readI32() } + func wvWrite(_ w: inout WvWriter) { w.writeI32(self) } +} + +extension UInt32: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt32 { r.readU32() } + func wvWrite(_ w: inout WvWriter) { w.writeU32(self) } +} + +extension Int64: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int64 { r.readI64() } + func wvWrite(_ w: inout WvWriter) { w.writeI64(self) } +} + +extension UInt64: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt64 { r.readU64() } + func wvWrite(_ w: inout WvWriter) { w.writeU64(self) } +} + +extension Float: WvCodable { + static func wvRead(_ r: inout WvReader) -> Float { r.readF32() } + func wvWrite(_ w: inout WvWriter) { w.writeF32(self) } +} + +extension Double: WvCodable { + static func wvRead(_ r: inout WvReader) -> Double { r.readF64() } + func wvWrite(_ w: inout WvWriter) { w.writeF64(self) } +} + +extension String: WvCodable { + static func wvRead(_ r: inout WvReader) -> String { r.readString() } + func wvWrite(_ w: inout WvWriter) { w.writeString(self) } +} + +extension Data: WvCodable { + static func wvRead(_ r: inout WvReader) -> Data { r.readBytes() } + func wvWrite(_ w: inout WvWriter) { w.writeBytes(self) } +} + +extension Optional: WvCodable where Wrapped: WvCodable { + static func wvRead(_ r: inout WvReader) -> Wrapped? { + r.readOptionFlag() ? .some(r.read()) : nil + } + + func wvWrite(_ w: inout WvWriter) { + w.writeOptionFlag(self != nil) + if let value = self { w.write(value) } + } +} + +extension Array: WvCodable where Element: WvCodable { + static func wvRead(_ r: inout WvReader) -> [Element] { + let n = r.readCount() + var items: [Element] = [] + items.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0.. [Key: Value] { + let n = r.readCount() + var map: [Key: Value] = [:] + map.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0..: @unchecked Sendable { } } -/// Reports a thrown Swift error to the library as a callback failure -/// (code -4); the library aborts its current call with that code and message. -func wvForeignError(_ outErr: UnsafeMutablePointer?, _ error: Error) { - let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) - message.withCString { edge_cases_error_set(outErr, EdgeCasesRuntimeError.foreignCode, $0) } +/// Holds one callback implementation behind the `ctx` the library keeps. +final class WvBox: @unchecked Sendable { + let value: T + + init(_ value: T) { + self.value = value + } +} + +/// Retains `impl` for the library: the `ctx` of a callback parameter, which +/// the vtable's `free` entry releases with ``wvRelease(_:as:)``. +func wvRetain(_ impl: T) -> UnsafeMutableRawPointer { + Unmanaged.passRetained(WvBox(impl)).toOpaque() +} + +/// Releases the implementation behind `ctx`, from the vtable's `free` entry. +func wvRelease(_ ctx: UnsafeMutableRawPointer?, as type: T.Type) { + Unmanaged>.fromOpaque(ctx!).release() +} + +/// Runs one callback method on the implementation behind `ctx`. A thrown +/// error never unwinds through the library: `domain` reports it when it's +/// the method's declared error, anything else is reported as a callback +/// failure (`-4`), and the method returns `fallback`. +func wvInvoke( + _ ctx: UnsafeMutableRawPointer?, + _ outErr: UnsafeMutablePointer?, + as type: T.Type, + fallback: R, + domain: (Error, UnsafeMutablePointer?) -> Bool = { _, _ in false }, + _ body: (T) throws -> R +) -> R { + let impl = Unmanaged>.fromOpaque(ctx!).takeUnretainedValue().value + do { + return try body(impl) + } catch { + if !domain(error, outErr) { + let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) + message.withCString { edge_cases_error_set(outErr, EdgeCasesRuntimeError.foreignCode, $0) } + } + return fallback + } +} + +/// Reports a declared domain error from a callback method: its code, its +/// message, and its fields as the payload. +func wvSetError(_ outErr: UnsafeMutablePointer?, _ code: Int32, _ message: String, _ payload: WvWriter? = nil) { + message.withCString { edge_cases_error_set(outErr, code, $0) } + if let payload = payload { + payload.bytes.withUnsafeBufferPointer { edge_cases_error_set_payload(outErr, $0.baseAddress, $0.count) } + } +} + +/// Hands `bytes` to the library as a callback method's return: a run +/// allocated with `edge_cases_alloc` that the library adopts. +func wvHandOver(_ bytes: UnsafeRawBufferPointer, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + let run = edge_cases_alloc(bytes.count) + if let run = run, let base = bytes.baseAddress { + run.initialize(from: base.assumingMemoryBound(to: UInt8.self), count: bytes.count) + } + outPtr?.pointee = run + outLen?.pointee = run == nil ? 0 : bytes.count +} + +/// Returns a string from a callback method through its out slots. +func wvReturnString(_ value: String, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var value = value + value.withUTF8 { wvHandOver(UnsafeRawBufferPointer($0), outPtr, outLen) } +} + +/// Returns bytes from a callback method through its out slots. +func wvReturnBytes(_ value: Data, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + value.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } } +/// Returns a value buffer from a callback method through its out slots. +func wvReturnBuffer(_ value: T, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var w = WvWriter() + w.write(value) + w.bytes.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +// MARK: - Load-time checks + +/// The load-time checks, run once before the first native call: the library +/// must implement C ABI revision 4 and carry every declaration these +/// bindings were generated with, unchanged. +let wvContract: Void = { + wvCheckAbiVersion() + wvCheckContract(edge_cases_edge_cases_contract, [ + (0x056a86511785ba2b, 0x5cba3f16d57c6cdb, "edge_cases.make_widget"), + (0x07bfa9ecb2b720d2, 0x3813b3f57c64bf35, "edge_cases.EdgeErrors"), + (0x0aaa11867e37f991, 0xf090dd52146ebdf0, "edge_cases.Widget.refresh"), + (0x0f9701493898ae4c, 0x6944f47a275c96e8, "edge_cases.Inner"), + (0x1226b1adac179f7d, 0x0e95a00ab0bef637, "edge_cases.async_keywords"), + (0x16d36bef1541136c, 0x5ea7021ec324298f, "edge_cases.find_widget"), + (0x1b7324c55e980feb, 0xabf87fe372358f42, "edge_cases.Widget.old_name"), + (0x1cbd731467c2bab8, 0x4915b9989c2c9074, "edge_cases.Vault"), + (0x2c2a6de67d894b0a, 0x50a4496b1e7191ca, "edge_cases.Vault.open"), + (0x322b4cfcd2d6861c, 0xca7527a379ae8465, "edge_cases.Keywords"), + (0x3f8e23a5f32906f4, 0x8d4c908d796f130d, "edge_cases.Leaf"), + (0x42ab8a685457b97c, 0xe5ce30500cadb32d, "edge_cases.Vault.handle"), + (0x4a68540e2836d3d0, 0xa20e44c71e420ed2, "edge_cases.mode_stream"), + (0x4e030580ac0b111d, 0xb68b7a18d910e175, "edge_cases.legacy_value"), + (0x4f5e4e5291864ef3, 0x55a1a88b21a08f8b, "edge_cases.Legacy"), + (0x4f84b06171b4213d, 0xbaedcfdebb6a2820, "edge_cases.scalar_stream"), + (0x5067a5ff2d9ef910, 0xbfc79b18fc4714fc, "edge_cases.Widget"), + (0x59aff159112fbf5b, 0x18e4def57a8cfa49, "edge_cases.optional_params"), + (0x5daf65b98fa77c48, 0xc817a8cb202e59c2, "edge_cases.keywords_roundtrip"), + (0x5de8a5c9726c2c5a, 0x7c50b5b65f7a896d, "edge_cases.register_batch"), + (0x618a620309994c3c, 0x826ad0b1d208d723, "edge_cases.async_bytes"), + (0x648285cdba0a7b8e, 0xcf6e80e783a4d8be, "edge_cases.WidgetHolder"), + (0x649c78079824b10d, 0x5b12dd26e0634b69, "edge_cases.async_widget"), + (0x7da7a7357d122a37, 0xf544e025ffaa3d20, "edge_cases.async_void"), + (0x7e4015bc3a472e2d, 0xfe22291d90b82e4a, "edge_cases.widget_stream"), + (0x805276e5d81fdba7, 0xd9b64e360bd03b2b, "edge_cases.Outer"), + (0x80e1183326542df4, 0xe8a57e12cc6ed276, "edge_cases.select"), + (0x923ce5e9bcd769a1, 0x0b973f5dc07dda8b, "edge_cases.Vault.to_string"), + (0x98ca8c01f5239404, 0xfe11c93eb6b303ef, "edge_cases.Widget.rename"), + (0xa13e22e98ea278d4, 0xf1fae215d7387800, "edge_cases.widgets_everywhere"), + (0xa84a8c9bf2a881e1, 0xbe1dc840bf221996, "edge_cases.async_optional"), + (0xad958c478fbf205d, 0xe386b0716d24f19b, "edge_cases.Retired"), + (0xb2fc3964ef56ab39, 0x73c325c8bc351236, "edge_cases.async_list"), + (0xb3403af521cf0006, 0x6abc30d48206a3ba, "edge_cases.Widget.close"), + (0xb794304a22a406b8, 0xf6fb98711c516105, "edge_cases.async_struct"), + (0xba3936106a49c47d, 0xb7025f2fadd6288e, "edge_cases.Vault.peek"), + (0xc155b0c3cfb164f4, 0xc777ae57d92ebc47, "edge_cases.noop"), + (0xcdc18607777019dd, 0xb81a31e5c20aa7a8, "edge_cases.inspect_widget"), + (0xd164a82aeb64b8cc, 0xded02c68005c4737, "edge_cases.nested_optional"), + (0xd978ccad2468038f, 0x30c39c65d6235153, "edge_cases.Mode"), + (0xdbe9d947431c9124, 0x39537fb848fdf980, "edge_cases.BatchListener"), + (0xeae8b7862c08bb07, 0xfe12eeaa830452ea, "edge_cases.Keyword"), + (0xf1e155a05ae262b8, 0xb575eb818a423d42, "edge_cases.async_enum"), + (0xf22e210bdbce7343, 0x2a5e0f801bc8c768, "edge_cases.Widget.dispose"), + (0xf3c7cf675563ee6c, 0x0ecffeef399facc8, "edge_cases.Vault.probe"), + (0xfd34bc67acac379e, 0x576caa14f3c0901a, "edge_cases.Widget.fetch"), + (0xff2e9d2bff941786, 0x7e96764ef1f5fbaa, "edge_cases.string_stream"), + ]) +}() + // MARK: - API /// Typed errors reported by the `edge_cases` module. @@ -430,7 +728,7 @@ public enum EdgeErrorsError: Error, LocalizedError, Sendable { } } -func mapEdgeCases(_ err: WvError) -> Error { +func wvMapEdgeCases(_ err: WvError) -> Error { let message = wvErrorMessage(err) switch err.code { case 7: return EdgeErrorsError.busy(message: message.isEmpty ? "Resource is busy" : message) @@ -438,9 +736,9 @@ func mapEdgeCases(_ err: WvError) -> Error { } } -func checkEdgeCases(_ err: inout WvError) throws { +func wvCheckEdgeCases(_ err: inout WvError) throws { guard err.code != 0 else { return } - let error = mapEdgeCases(err) + let error = wvMapEdgeCases(err) edge_cases_error_clear(&err) throw error } @@ -453,11 +751,16 @@ public enum Legacy: Int32, Sendable { case class_ = 10 } +@available(*, deprecated, message: "Use Mode instead") +extension Legacy: WvCEnum {} + public enum Mode: Int32, Sendable { case fast = 1 case safe = 2 } +extension Mode: WvCEnum {} + public struct Leaf: Hashable, Sendable { public var weight: Float @@ -467,13 +770,14 @@ public struct Leaf: Hashable, Sendable { } } -func wvWriteLeaf(_ value: Leaf, into w: inout WvWriter) { - w.writeF32(value.weight) -} +extension Leaf: WvCodable { + static func wvRead(_ r: inout WvReader) -> Leaf { + Leaf(weight: r.read()) + } -func wvReadLeaf(_ r: inout WvReader) -> Leaf { - let v0 = r.readF32() - return Leaf(weight: v0) + func wvWrite(_ w: inout WvWriter) { + w.write(self.weight) + } } /// Middle layer of a three-deep struct nesting @@ -488,24 +792,15 @@ public struct Inner: Hashable, Sendable { } } -func wvWriteInner(_ value: Inner, into w: inout WvWriter) { - wvWriteLeaf(value.leaf, into: &w) - w.writeLen(value.leaves.count) - for v0 in value.leaves { - wvWriteLeaf(v0, into: &w) +extension Inner: WvCodable { + static func wvRead(_ r: inout WvReader) -> Inner { + Inner(leaf: r.read(), leaves: r.read()) } -} -func wvReadInner(_ r: inout WvReader) -> Inner { - let v1 = wvReadLeaf(&r) - let n3 = r.readCount() - var v2: [Leaf] = [] - v2.reserveCapacity(min(n3, r.remaining)) - for _ in 0.. Outer { - let v12 = wvReadInner(&r) - var v13: Inner? = nil - if r.readOptionFlag() { - let v14 = wvReadInner(&r) - v13 = v14 - } - let n16 = r.readCount() - var v15: [String: Inner] = [:] - v15.reserveCapacity(min(n16, r.remaining)) - for _ in 0.. Outer { + Outer(inner: r.read(), maybeInner: r.read(), byName: r.read(), modes: r.read(), matrix: r.read(), sparse: r.read(), blob: r.read(), note: r.read(), byId: r.read(), hidden: r.read(), v2: r.read()) } - var v31: Data? = nil - if r.readOptionFlag() { - let v32 = r.readBytes() - v31 = v32 - } - var v33: String? = nil - if r.readOptionFlag() { - let v34 = r.readString() - v33 = v34 - } - let n36 = r.readCount() - var v35: [Int64: String] = [:] - v35.reserveCapacity(min(n36, r.remaining)) - for _ in 0.. Keywords { - let v0 = r.readI32() - let v1 = r.readI32() - let v2 = r.readI32() - let v3 = r.readI32() - let v4 = r.readI32() - let v5 = r.readI32() - let v6 = r.readI32() - let v7 = r.readI32() - let v8 = r.readI32() - let v9 = r.readI32() - let v10 = r.readI32() - let v11 = r.readI32() - let v12 = r.readI32() - let v13 = r.readI32() - let v14 = r.readI32() - let v15 = r.readI32() - let v16 = r.readI32() - let v17 = r.readI32() - let v18 = r.readI32() - let v19 = r.readI32() - let v20 = r.readI32() - let v21 = r.readI32() - let v22 = r.readI32() - let v23 = r.readI32() - let v24 = r.readI32() - let v25 = r.readI32() - let v26 = r.readI32() - let v27 = r.readI32() - let v28 = r.readI32() - let v29 = r.readI32() - let v30 = r.readI32() - let v31 = r.readI32() - let v32 = r.readI32() - let v33 = r.readI32() - let v34 = r.readI32() - let v35 = r.readI32() - let v36 = r.readI32() - let v37 = r.readI32() - let v38 = r.readI32() - let v39 = r.readI32() - let v40 = r.readI32() - let v41 = r.readI32() - let v42 = r.readI32() - let v43 = r.readI32() - let v44 = r.readI32() - let v45 = r.readI32() - let v46 = r.readI32() - let v47 = r.readI32() - let v48 = r.readI32() - let v49 = r.readI32() - let v50 = r.readI32() - let v51 = r.readI32() - let v52 = r.readI32() - let v53 = r.readI32() - let v54 = r.readI32() - let v55 = r.readI32() - let v56 = r.readI32() - let v57 = r.readI32() - let v58 = r.readI32() - return Keywords(class_: v0, def: v1, import_: v2, from: v3, func_: v4, range: v5, register: v6, volatile: v7, new: v8, default_: v9, end: v10, in_: v11, is_: v12, int: v13, var_: v14, val: v15, self_: v16, this: v17, object: v18, string: v19, case_: v20, select: v21, switch_: v22, void: v23, union: v24, namespace: v25, operator_: v26, template: v27, package: v28, protocol_: v29, guard_: v30, defer_: v31, inout_: v32, internal_: v33, when: v34, fun: v35, data: v36, late: v37, part: v38, out: v39, ref: v40, event: v41, base: v42, lambda: v43, yield: v44, nil_: v45, null: v46, begin: v47, rescue: v48, module: v49, chan: v50, go: v51, let_: v52, export: v53, delete: v54, typedef: v55, sizeof: v56, static_: v57, super_: v58) +extension Keywords: WvCodable { + static func wvRead(_ r: inout WvReader) -> Keywords { + Keywords(class_: r.read(), def: r.read(), import_: r.read(), from: r.read(), func_: r.read(), range: r.read(), register: r.read(), volatile: r.read(), new: r.read(), default_: r.read(), end: r.read(), in_: r.read(), is_: r.read(), int: r.read(), var_: r.read(), val: r.read(), self_: r.read(), this: r.read(), object: r.read(), string: r.read(), case_: r.read(), select: r.read(), switch_: r.read(), void: r.read(), union: r.read(), namespace: r.read(), operator_: r.read(), template: r.read(), package: r.read(), protocol_: r.read(), guard_: r.read(), defer_: r.read(), inout_: r.read(), internal_: r.read(), when: r.read(), fun: r.read(), data: r.read(), late: r.read(), part: r.read(), out: r.read(), ref: r.read(), event: r.read(), base: r.read(), lambda: r.read(), yield: r.read(), nil_: r.read(), null: r.read(), begin: r.read(), rescue: r.read(), module: r.read(), chan: r.read(), go: r.read(), let_: r.read(), export: r.read(), delete: r.read(), typedef: r.read(), sizeof: r.read(), static_: r.read(), super_: r.read()) + } + + func wvWrite(_ w: inout WvWriter) { + w.write(self.class_) + w.write(self.def) + w.write(self.import_) + w.write(self.from) + w.write(self.func_) + w.write(self.range) + w.write(self.register) + w.write(self.volatile) + w.write(self.new) + w.write(self.default_) + w.write(self.end) + w.write(self.in_) + w.write(self.is_) + w.write(self.int) + w.write(self.var_) + w.write(self.val) + w.write(self.self_) + w.write(self.this) + w.write(self.object) + w.write(self.string) + w.write(self.case_) + w.write(self.select) + w.write(self.switch_) + w.write(self.void) + w.write(self.union) + w.write(self.namespace) + w.write(self.operator_) + w.write(self.template) + w.write(self.package) + w.write(self.protocol_) + w.write(self.guard_) + w.write(self.defer_) + w.write(self.inout_) + w.write(self.internal_) + w.write(self.when) + w.write(self.fun) + w.write(self.data) + w.write(self.late) + w.write(self.part) + w.write(self.out) + w.write(self.ref) + w.write(self.event) + w.write(self.base) + w.write(self.lambda) + w.write(self.yield) + w.write(self.nil_) + w.write(self.null) + w.write(self.begin) + w.write(self.rescue) + w.write(self.module) + w.write(self.chan) + w.write(self.go) + w.write(self.let_) + w.write(self.export) + w.write(self.delete) + w.write(self.typedef) + w.write(self.sizeof) + w.write(self.static_) + w.write(self.super_) + } } /// Deprecated struct @@ -923,14 +1058,14 @@ public struct Retired: Hashable, Sendable { } @available(*, deprecated, message: "Use Outer instead") -func wvWriteRetired(_ value: Retired, into w: inout WvWriter) { - w.writeI64(value.id) -} +extension Retired: WvCodable { + static func wvRead(_ r: inout WvReader) -> Retired { + Retired(id: r.read()) + } -@available(*, deprecated, message: "Use Outer instead") -func wvReadRetired(_ r: inout WvReader) -> Retired { - let v0 = r.readI64() - return Retired(id: v0) + func wvWrite(_ w: inout WvWriter) { + w.write(self.id) + } } /// Record carrying objects in every buffered position @@ -949,152 +1084,78 @@ public struct WidgetHolder: Sendable { } } -func wvWriteWidgetHolder(_ value: WidgetHolder, into w: inout WvWriter) { - w.writeObject(value.primary.clonePtr()) - if let v0 = value.fallback { - w.writeOptionFlag(true) - w.writeObject(v0.clonePtr()) - } else { - w.writeOptionFlag(false) - } - w.writeLen(value.extras.count) - for v1 in value.extras { - w.writeObject(v1.clonePtr()) +extension WidgetHolder: WvCodable { + static func wvRead(_ r: inout WvReader) -> WidgetHolder { + WidgetHolder(primary: r.read(), fallback: r.read(), extras: r.read(), byName: r.read()) } - w.writeLen(value.byName.count) - for (v2, v3) in value.byName { - w.writeString(v2) - w.writeObject(v3.clonePtr()) - } -} -func wvReadWidgetHolder(_ r: inout WvReader) -> WidgetHolder { - let v4 = Widget(ptr: r.readObject()) - var v5: Widget? = nil - if r.readOptionFlag() { - let v6 = Widget(ptr: r.readObject()) - v5 = v6 - } - let n8 = r.readCount() - var v7: [Widget] = [] - v7.reserveCapacity(min(n8, r.remaining)) - for _ in 0.. Bool + /// Enum return travels back as its integer discriminant func pickMode() throws -> Mode } -/// Retains one `BatchListener` implementation across the C boundary; the library's -/// `free` entry releases it. -final class WvBatchListenerBox: Sendable { - let impl: any BatchListener - init(_ impl: any BatchListener) { self.impl = impl } -} - -/// The process-wide `edge_cases_edge_cases_BatchListener_vtable` the library calls `BatchListener` -/// implementations through. Every entry recovers the box from `ctx`, copies -/// or adopts its arguments, and reports a thrown error through `out_err` -/// instead of unwinding. +/// The process-wide vtable the library calls `BatchListener` implementations +/// through (`edge_cases_edge_cases_BatchListener_vtable`). Every entry recovers the implementation +/// from `ctx`, copies or adopts its arguments, hands its result back, and +/// reports a thrown error through `out_err` instead of unwinding. enum WvBatchListenerVtable { - static let shared = WvVtable(edge_cases_edge_cases_BatchListener_vtable( + static let shared = WvVtable(edge_cases_edge_cases_BatchListener_vtable( + size: UInt32(MemoryLayout.stride), + flags: 0, + free: { ctx in + wvRelease(ctx, as: (any BatchListener).self) + }, on_batch: { ctx, items_ptr, items_len, raw_ptr, raw_len, first_ptr, first_len, mode, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - try wvBox.impl.onBatch(items: wvBorrowBuffer(items_ptr, items_len) { (r: inout WvReader) -> [Outer] in - let n1 = r.readCount() - var v0: [Outer] = [] - v0.reserveCapacity(min(n1, r.remaining)) - for _ in 0.. Outer? in - var v0: Outer? = nil - if r.readOptionFlag() { - let v1 = wvReadOuter(&r) - v0 = v1 - } - return v0 - }, mode: wvEnumCase(Mode.self, numericCast(mode.rawValue))) - } catch { - wvForeignError(out_err, error) + wvInvoke(ctx, out_err, as: (any BatchListener).self, fallback: ()) { wvImpl in + try wvImpl.onBatch(items: wvBorrowBuffer(items_ptr, items_len, as: [Outer].self), raw: wvBorrowBytes(raw_ptr, raw_len), first: wvBorrowBuffer(first_ptr, first_len, as: Outer?.self), mode: wvEnumCase(Mode.self, numericCast(mode.rawValue))) } }, on_widget: { ctx, widget, fallback, children_ptr, children_len, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - try wvBox.impl.onWidget(widget: Widget(ptr: wvNonNull(widget)), fallback: fallback.map { Widget(ptr: $0) }, children: wvBorrowBuffer(children_ptr, children_len) { (r: inout WvReader) -> [Widget] in - let n1 = r.readCount() - var v0: [Widget] = [] - v0.reserveCapacity(min(n1, r.remaining)) - for _ in 0...fromOpaque(ctx!).takeUnretainedValue() - do { - try wvBox.impl.onTick() - } catch { - wvForeignError(out_err, error) + wvInvoke(ctx, out_err, as: (any BatchListener).self, fallback: ()) { wvImpl in + try wvImpl.onTick() } }, should_continue: { ctx, class_, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - return try wvBox.impl.shouldContinue(class_: class_) - } catch { - wvForeignError(out_err, error) - return false + wvInvoke(ctx, out_err, as: (any BatchListener).self, fallback: false) { wvImpl in + try wvImpl.shouldContinue(class_: class_) } }, pick_mode: { ctx, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - let value = try wvBox.impl.pickMode() - return edge_cases_edge_cases_Mode(rawValue: numericCast(value.rawValue)) - } catch { - wvForeignError(out_err, error) - return edge_cases_edge_cases_Mode(rawValue: 0) + wvInvoke(ctx, out_err, as: (any BatchListener).self, fallback: edge_cases_edge_cases_Mode(rawValue: 0)) { wvImpl in + try edge_cases_edge_cases_Mode(rawValue: numericCast(wvImpl.pickMode().rawValue)) } - }, - free: { ctx in - Unmanaged.fromOpaque(ctx!).release() } )) } @@ -1102,47 +1163,37 @@ enum WvBatchListenerVtable { /// Callback interface whose name and methods are keywords /// /// Implement this protocol with a class and pass an instance where the API -/// expects it; the library keeps it alive for as long as it holds the -/// callback and may call any method from any thread. A thrown error aborts -/// the library's call and surfaces to the original caller as a foreign error -/// (code -4). +/// expects it. The library keeps the instance until it no longer needs it and +/// may call any method from any thread. A method that throws fails the +/// library's call in progress: a method documented to throw a module error +/// reports that error, with its code and fields, and any other error reaches +/// the library as a callback failure (code -4). public protocol Keyword: AnyObject, Sendable { func delete(self_: Int32) throws - func import_() throws -> Int32 -} -/// Retains one `Keyword` implementation across the C boundary; the library's -/// `free` entry releases it. -final class WvKeywordBox: Sendable { - let impl: any Keyword - init(_ impl: any Keyword) { self.impl = impl } + func import_() throws -> Int32 } -/// The process-wide `edge_cases_edge_cases_Keyword_vtable` the library calls `Keyword` -/// implementations through. Every entry recovers the box from `ctx`, copies -/// or adopts its arguments, and reports a thrown error through `out_err` -/// instead of unwinding. +/// The process-wide vtable the library calls `Keyword` implementations +/// through (`edge_cases_edge_cases_Keyword_vtable`). Every entry recovers the implementation +/// from `ctx`, copies or adopts its arguments, hands its result back, and +/// reports a thrown error through `out_err` instead of unwinding. enum WvKeywordVtable { - static let shared = WvVtable(edge_cases_edge_cases_Keyword_vtable( + static let shared = WvVtable(edge_cases_edge_cases_Keyword_vtable( + size: UInt32(MemoryLayout.stride), + flags: 0, + free: { ctx in + wvRelease(ctx, as: (any Keyword).self) + }, delete_: { ctx, self_, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - try wvBox.impl.delete(self_: self_) - } catch { - wvForeignError(out_err, error) + wvInvoke(ctx, out_err, as: (any Keyword).self, fallback: ()) { wvImpl in + try wvImpl.delete(self_: self_) } }, import: { ctx, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - return try wvBox.impl.import_() - } catch { - wvForeignError(out_err, error) - return 0 + wvInvoke(ctx, out_err, as: (any Keyword).self, fallback: 0) { wvImpl in + try wvImpl.import_() } - }, - free: { ctx in - Unmanaged.fromOpaque(ctx!).release() } )) } @@ -1160,30 +1211,25 @@ public final class Widget: @unchecked Sendable { } /// Returns a new strong reference to the same object, for a position that - /// takes ownership of it (an object token inside a value buffer). + /// takes ownership of it. func clonePtr() -> OpaquePointer { wvNonNull(edge_cases_edge_cases_Widget_clone(ptr)) } /// Void method that can fail + /// + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise. public func refresh() throws { var err = WvError() edge_cases_edge_cases_Widget_refresh(ptr, &err) - try checkEdgeCases(&err) + try wvCheckEdgeCases(&err) } public func rename(class_: String, import_: String?) -> String { var err = WvError() - var import_Writer = WvWriter() - if let v0 = import_ { - import_Writer.writeOptionFlag(true) - import_Writer.writeString(v0) - } else { - import_Writer.writeOptionFlag(false) - } var outLen = 0 let rv: UnsafePointer? = wvWithUTF8(class_) { class__ptr, class__len in - wvWithBuffer(import_Writer) { import__ptr, import__len in + wvWithEncoded(import_) { import__ptr, import__len in edge_cases_edge_cases_Widget_rename(ptr, class__ptr, class__len, import__ptr, import__len, &outLen, &err) } } @@ -1192,16 +1238,18 @@ public final class Widget: @unchecked Sendable { } /// Async method with a struct result + /// + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise. public func fetch(id: Int64) async throws -> Outer { return try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in let context = Unmanaged.passRetained(WvContinuation(continuation)).toOpaque() edge_cases_edge_cases_Widget_fetch(self.ptr, id, { context, err, resultPtr, resultLen in let cont = Unmanaged>.fromOpaque(context!).takeRetainedValue().value if let err = err { - cont.resume(throwing: wvTakeError(err, mapEdgeCases)) + cont.resume(throwing: wvTakeError(err, wvMapEdgeCases)) return } - cont.resume(returning: wvTakeBuffer(resultPtr, resultLen, wvReadOuter)) + cont.resume(returning: wvTakeBuffer(resultPtr, resultLen, as: Outer.self)) }, context) } } @@ -1214,6 +1262,26 @@ public final class Widget: @unchecked Sendable { wvTrap(&err) return wvTakeString(rv, outLen) } + + /// Spelled like the release method of several wrappers + public func close() -> Bool { + var err = WvError() + let rv = edge_cases_edge_cases_Widget_close(ptr, &err) + wvTrap(&err) + return rv + } + + /// Spelled like the release method of the Dart and .NET wrappers + public func dispose(force: Bool) { + var err = WvError() + edge_cases_edge_cases_Widget_dispose(ptr, force, &err) + wvTrap(&err) + } +} + +extension Widget: WvCodable { + static func wvRead(_ r: inout WvReader) -> Widget { Widget(ptr: r.readObject()) } + func wvWrite(_ w: inout WvWriter) { w.writeObject(clonePtr()) } } /// Deprecated interface with a throwing constructor @@ -1230,19 +1298,21 @@ public final class Vault: @unchecked Sendable { } /// Returns a new strong reference to the same object, for a position that - /// takes ownership of it (an object token inside a value buffer). + /// takes ownership of it. func clonePtr() -> OpaquePointer { wvNonNull(edge_cases_edge_cases_Vault_clone(ptr)) } /// Fails when the vault is busy + /// + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise. public static func open(path: String) throws -> Vault { wvLoad() var err = WvError() let rv: OpaquePointer? = wvWithUTF8(path) { path_ptr, path_len in edge_cases_edge_cases_Vault_open(path_ptr, path_len, &err) } - try checkEdgeCases(&err) + try wvCheckEdgeCases(&err) return Vault(ptr: wvNonNull(rv)) } @@ -1253,14 +1323,40 @@ public final class Vault: @unchecked Sendable { return rv.map { Widget(ptr: $0) } } + /// Spelled like the handle accessor of several wrappers + public func handle() -> Int64 { + var err = WvError() + let rv = edge_cases_edge_cases_Vault_handle(ptr, &err) + wvTrap(&err) + return rv + } + /// Throwing static + /// + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise. public static func probe() throws -> Bool { wvLoad() var err = WvError() let rv = edge_cases_edge_cases_Vault_probe(&err) - try checkEdgeCases(&err) + try wvCheckEdgeCases(&err) return rv } + + /// Static spelled like an inherited object member + public static func toString() -> String { + wvLoad() + var err = WvError() + var outLen = 0 + let rv = edge_cases_edge_cases_Vault_to_string(&outLen, &err) + wvTrap(&err) + return wvTakeString(rv, outLen) + } +} + +@available(*, deprecated, message: "Use Widget instead") +extension Vault: WvCodable { + static func wvRead(_ r: inout WvReader) -> Vault { Vault(ptr: r.readObject()) } + func wvWrite(_ w: inout WvWriter) { w.writeObject(clonePtr()) } } /// A lazy sequence over the `Widget` elements streamed by `edge_cases_edge_cases_widget_stream`. @@ -1425,21 +1521,9 @@ public enum EdgeCases { public static func select(class_: Int32, from: String, func_: [String], range: Int32?, type: Bool) -> Int32 { wvLoad() var err = WvError() - var func_Writer = WvWriter() - func_Writer.writeLen(func_.count) - for v0 in func_ { - func_Writer.writeString(v0) - } - var rangeWriter = WvWriter() - if let v1 = range { - rangeWriter.writeOptionFlag(true) - rangeWriter.writeI32(v1) - } else { - rangeWriter.writeOptionFlag(false) - } let rv: Int32 = wvWithUTF8(from) { from_ptr, from_len in - wvWithBuffer(func_Writer) { func__ptr, func__len in - wvWithBuffer(rangeWriter) { range_ptr, range_len in + wvWithEncoded(func_) { func__ptr, func__len in + wvWithEncoded(range) { range_ptr, range_len in edge_cases_edge_cases_select(class_, from_ptr, from_len, func__ptr, func__len, range_ptr, range_len, type, &err) } } @@ -1452,101 +1536,24 @@ public enum EdgeCases { public static func nestedOptional(items: [Outer?]?) -> [Outer?]? { wvLoad() var err = WvError() - var itemsWriter = WvWriter() - if let v0 = items { - itemsWriter.writeOptionFlag(true) - itemsWriter.writeLen(v0.count) - for v1 in v0 { - if let v2 = v1 { - itemsWriter.writeOptionFlag(true) - wvWriteOuter(v2, into: &itemsWriter) - } else { - itemsWriter.writeOptionFlag(false) - } - } - } else { - itemsWriter.writeOptionFlag(false) - } var outLen = 0 - let rv: UnsafePointer? = wvWithBuffer(itemsWriter) { items_ptr, items_len in + let rv: UnsafePointer? = wvWithEncoded(items) { items_ptr, items_len in edge_cases_edge_cases_nested_optional(items_ptr, items_len, &outLen, &err) } wvTrap(&err) - return wvTakeBuffer(rv, outLen) { (r: inout WvReader) -> [Outer?]? in - var v0: [Outer?]? = nil - if r.readOptionFlag() { - let n2 = r.readCount() - var v1: [Outer?] = [] - v1.reserveCapacity(min(n2, r.remaining)) - for _ in 0.. Mode? { wvLoad() var err = WvError() - var textWriter = WvWriter() - if let v0 = text { - textWriter.writeOptionFlag(true) - textWriter.writeString(v0) - } else { - textWriter.writeOptionFlag(false) - } - var countWriter = WvWriter() - if let v1 = count { - countWriter.writeOptionFlag(true) - countWriter.writeI32(v1) - } else { - countWriter.writeOptionFlag(false) - } - var outerWriter = WvWriter() - if let v2 = outer { - outerWriter.writeOptionFlag(true) - wvWriteOuter(v2, into: &outerWriter) - } else { - outerWriter.writeOptionFlag(false) - } - var tagsWriter = WvWriter() - if let v3 = tags { - tagsWriter.writeOptionFlag(true) - tagsWriter.writeLen(v3.count) - for v4 in v3 { - tagsWriter.writeString(v4) - } - } else { - tagsWriter.writeOptionFlag(false) - } - var blobWriter = WvWriter() - if let v5 = blob { - blobWriter.writeOptionFlag(true) - blobWriter.writeBytes(v5) - } else { - blobWriter.writeOptionFlag(false) - } - var modeWriter = WvWriter() - if let v6 = mode { - modeWriter.writeOptionFlag(true) - modeWriter.writeI32(v6.rawValue) - } else { - modeWriter.writeOptionFlag(false) - } var outLen = 0 - let rv: UnsafePointer? = wvWithBuffer(textWriter) { text_ptr, text_len in - wvWithBuffer(countWriter) { count_ptr, count_len in - wvWithBuffer(outerWriter) { outer_ptr, outer_len in - wvWithBuffer(tagsWriter) { tags_ptr, tags_len in - wvWithBuffer(blobWriter) { blob_ptr, blob_len in - wvWithBuffer(modeWriter) { mode_ptr, mode_len in + let rv: UnsafePointer? = wvWithEncoded(text) { text_ptr, text_len in + wvWithEncoded(count) { count_ptr, count_len in + wvWithEncoded(outer) { outer_ptr, outer_len in + wvWithEncoded(tags) { tags_ptr, tags_len in + wvWithEncoded(blob) { blob_ptr, blob_len in + wvWithEncoded(mode) { mode_ptr, mode_len in edge_cases_edge_cases_optional_params(text_ptr, text_len, count_ptr, count_len, outer_ptr, outer_len, tags_ptr, tags_len, blob_ptr, blob_len, mode_ptr, mode_len, &outLen, &err) } } @@ -1555,22 +1562,15 @@ public enum EdgeCases { } } wvTrap(&err) - return wvTakeBuffer(rv, outLen) { (r: inout WvReader) -> Mode? in - var v0: Mode? = nil - if r.readOptionFlag() { - let v1 = wvEnumCase(Mode.self, r.readI32()) - v0 = v1 - } - return v0 - } + return wvTakeBuffer(rv, outLen, as: Mode?.self) } /// Two callback interface parameters on one function - public static func registerBatch(listener: BatchListener, keyword: Keyword) { + public static func registerBatch(listener: any BatchListener, keyword: any Keyword) { wvLoad() var err = WvError() - let listener_ctx = Unmanaged.passRetained(WvBatchListenerBox(listener)).toOpaque() - let keyword_ctx = Unmanaged.passRetained(WvKeywordBox(keyword)).toOpaque() + let listener_ctx = wvRetain(listener) + let keyword_ctx = wvRetain(keyword) edge_cases_edge_cases_register_batch(listener_ctx, WvBatchListenerVtable.shared.pointer, keyword_ctx, WvKeywordVtable.shared.pointer, &err) wvTrap(&err) } @@ -1579,47 +1579,23 @@ public enum EdgeCases { public static func widgetsEverywhere(list: [Widget], byName: [String: Widget], sparse: [Widget?], holder: WidgetHolder) -> WidgetHolder { wvLoad() var err = WvError() - var listWriter = WvWriter() - listWriter.writeLen(list.count) - for v0 in list { - listWriter.writeObject(v0.clonePtr()) - } - var byNameWriter = WvWriter() - byNameWriter.writeLen(byName.count) - for (v1, v2) in byName { - byNameWriter.writeString(v1) - byNameWriter.writeObject(v2.clonePtr()) - } - var sparseWriter = WvWriter() - sparseWriter.writeLen(sparse.count) - for v3 in sparse { - if let v4 = v3 { - sparseWriter.writeOptionFlag(true) - sparseWriter.writeObject(v4.clonePtr()) - } else { - sparseWriter.writeOptionFlag(false) - } - } - var holderWriter = WvWriter() - wvWriteWidgetHolder(holder, into: &holderWriter) var outLen = 0 - let rv: UnsafePointer? = wvWithBuffer(listWriter) { list_ptr, list_len in - wvWithBuffer(byNameWriter) { byName_ptr, byName_len in - wvWithBuffer(sparseWriter) { sparse_ptr, sparse_len in - wvWithBuffer(holderWriter) { holder_ptr, holder_len in + let rv: UnsafePointer? = wvWithEncoded(list) { list_ptr, list_len in + wvWithEncoded(byName) { byName_ptr, byName_len in + wvWithEncoded(sparse) { sparse_ptr, sparse_len in + wvWithEncoded(holder) { holder_ptr, holder_len in edge_cases_edge_cases_widgets_everywhere(list_ptr, list_len, byName_ptr, byName_len, sparse_ptr, sparse_len, holder_ptr, holder_len, &outLen, &err) } } } } wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadWidgetHolder) + return wvTakeBuffer(rv, outLen, as: WidgetHolder.self) } /// Iterator of objects - /// - Returns: A lazy sequence that pulls one element per step from the - /// library; the underlying iterator is destroyed when the sequence is - /// exhausted or deinitialized. + /// + /// - Returns: A lazy sequence that pulls one element per step from the library. public static func widgetStream() -> EdgeCasesWidgetStreamIterator { wvLoad() var err = WvError() @@ -1629,6 +1605,8 @@ public enum EdgeCases { } /// Async returning an object + /// + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise. public static func asyncWidget() async throws -> Widget { wvLoad() return try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in @@ -1636,7 +1614,7 @@ public enum EdgeCases { edge_cases_edge_cases_async_widget({ context, err, result in let cont = Unmanaged>.fromOpaque(context!).takeRetainedValue().value if let err = err { - cont.resume(throwing: wvTakeError(err, mapEdgeCases)) + cont.resume(throwing: wvTakeError(err, wvMapEdgeCases)) return } cont.resume(returning: Widget(ptr: wvNonNull(result))) @@ -1656,11 +1634,13 @@ public enum EdgeCases { } /// Optional interface return + /// + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise. public static func findWidget(id: Int64) throws -> Widget? { wvLoad() var err = WvError() let rv = edge_cases_edge_cases_find_widget(id, &err) - try checkEdgeCases(&err) + try wvCheckEdgeCases(&err) return rv.map { Widget(ptr: $0) } } @@ -1675,9 +1655,8 @@ public enum EdgeCases { } /// Iterator of strings - /// - Returns: A lazy sequence that pulls one element per step from the - /// library; the underlying iterator is destroyed when the sequence is - /// exhausted or deinitialized. + /// + /// - Returns: A lazy sequence that pulls one element per step from the library. public static func stringStream() -> EdgeCasesStringStreamIterator { wvLoad() var err = WvError() @@ -1687,9 +1666,8 @@ public enum EdgeCases { } /// Iterator of scalars - /// - Returns: A lazy sequence that pulls one element per step from the - /// library; the underlying iterator is destroyed when the sequence is - /// exhausted or deinitialized. + /// + /// - Returns: A lazy sequence that pulls one element per step from the library. public static func scalarStream() -> EdgeCasesScalarStreamIterator { wvLoad() var err = WvError() @@ -1699,9 +1677,8 @@ public enum EdgeCases { } /// Iterator of enums - /// - Returns: A lazy sequence that pulls one element per step from the - /// library; the underlying iterator is destroyed when the sequence is - /// exhausted or deinitialized. + /// + /// - Returns: A lazy sequence that pulls one element per step from the library. public static func modeStream() -> EdgeCasesModeStreamIterator { wvLoad() var err = WvError() @@ -1739,6 +1716,7 @@ public enum EdgeCases { } } + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise. public static func asyncStruct() async throws -> Outer { wvLoad() return try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in @@ -1746,16 +1724,15 @@ public enum EdgeCases { edge_cases_edge_cases_async_struct({ context, err, resultPtr, resultLen in let cont = Unmanaged>.fromOpaque(context!).takeRetainedValue().value if let err = err { - cont.resume(throwing: wvTakeError(err, mapEdgeCases)) + cont.resume(throwing: wvTakeError(err, wvMapEdgeCases)) return } - cont.resume(returning: wvTakeBuffer(resultPtr, resultLen, wvReadOuter)) + cont.resume(returning: wvTakeBuffer(resultPtr, resultLen, as: Outer.self)) }, context) } } - /// - Throws: `CancellationError` when the calling task is cancelled, which - /// also cancels the native call. + /// - Throws: ``EdgeErrorsError`` for a declared failure, ``EdgeCasesRuntimeError`` otherwise, and `CancellationError` when the calling task is cancelled, which cancels the native call. public static func asyncList() async throws -> [Outer] { wvLoad() let token = WvCancelToken() @@ -1765,19 +1742,10 @@ public enum EdgeCases { edge_cases_edge_cases_async_list(token.raw, { context, err, resultPtr, resultLen in let cont = Unmanaged>.fromOpaque(context!).takeRetainedValue().value if let err = err { - cont.resume(throwing: wvTakeError(err, mapEdgeCases)) + cont.resume(throwing: wvTakeError(err, wvMapEdgeCases)) return } - cont.resume(returning: wvTakeBuffer(resultPtr, resultLen) { (r: inout WvReader) -> [Outer] in - let n1 = r.readCount() - var v0: [Outer] = [] - v0.reserveCapacity(min(n1, r.remaining)) - for _ in 0.. Int64? in - var v0: Int64? = nil - if r.readOptionFlag() { - let v1 = r.readI64() - v0 = v1 - } - return v0 - }) + cont.resume(returning: wvTakeBuffer(resultPtr, resultLen, as: Int64?.self)) }, context) } } @@ -1839,14 +1800,12 @@ public enum EdgeCases { public static func keywordsRoundtrip(value: Keywords) -> Keywords { wvLoad() var err = WvError() - var valueWriter = WvWriter() - wvWriteKeywords(value, into: &valueWriter) var outLen = 0 - let rv: UnsafePointer? = wvWithBuffer(valueWriter) { value_ptr, value_len in + let rv: UnsafePointer? = wvWithEncoded(value) { value_ptr, value_len in edge_cases_edge_cases_keywords_roundtrip(value_ptr, value_len, &outLen, &err) } wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadKeywords) + return wvTakeBuffer(rv, outLen, as: Keywords.self) } @available(*, deprecated, message: "Use Mode-based APIs") @@ -1856,7 +1815,7 @@ public enum EdgeCases { var outLen = 0 let rv = edge_cases_edge_cases_legacy_value(edge_cases_edge_cases_Legacy(rawValue: numericCast(value.rawValue)), &outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadRetired) + return wvTakeBuffer(rv, outLen, as: Retired.self) } } diff --git a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Package_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Package_swift.snap index c40434d4..01417bcc 100644 --- a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Package_swift.snap +++ b/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Package_swift.snap @@ -2,18 +2,18 @@ source: crates/weaveffi-cli/tests/snapshots.rs --- // swift-tools-version:5.9 -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import Foundation import PackageDescription // The C module `CKitchenSink` comes from one of two places. Next to this -// manifest, a prebuilt `CKitchenSink.xcframework` (assembled from the native -// libraries, see the README) wins. Otherwise `Sources/CKitchenSink` declares -// the generated header as a system library that links `kitchen_sink` from the -// linker search path, e.g. `swift build -Xlinker -L/path/to/lib`. +// manifest, a prebuilt `CKitchenSink.xcframework` (from `weaveffi package`) wins. +// Otherwise `Sources/CKitchenSink` declares the generated header as a system +// library that links `kitchen_sink` from the linker search path, e.g. +// `swift build -Xlinker -L/path/to/lib`. let xcframework = "CKitchenSink.xcframework" let cModule: Target = FileManager.default.fileExists(atPath: "\(Context.packageDirectory)/\(xcframework)") ? .binaryTarget(name: "CKitchenSink", path: xcframework) @@ -21,7 +21,7 @@ let cModule: Target = FileManager.default.fileExists(atPath: "\(Context.packageD let package = Package( name: "KitchenSink", - platforms: [.macOS(.v10_15), .iOS(.v13), .tvOS(.v13), .watchOS(.v6)], + platforms: [.macOS("11.0"), .iOS("13.0")], products: [ .library(name: "KitchenSink", targets: ["KitchenSink"]), ], diff --git a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_kitchen_sink_h.snap b/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_kitchen_sink_h.snap deleted file mode 100644 index 14faeffb..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_kitchen_sink_h.snap +++ /dev/null @@ -1,219 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef KITCHEN_SINK_H -#define KITCHEN_SINK_H - -#include -#include -#include - -#ifndef KITCHEN_SINK_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef KITCHEN_SINK_BUILD -# define KITCHEN_SINK_API __declspec(dllexport) -# else -# define KITCHEN_SINK_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define KITCHEN_SINK_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define KITCHEN_SINK_API __attribute__((visibility("default"))) -# else -# define KITCHEN_SINK_API -# endif -#endif - -#ifndef KITCHEN_SINK_DEPRECATED -# if defined(_MSC_VER) -# define KITCHEN_SINK_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define KITCHEN_SINK_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define KITCHEN_SINK_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports kitchen_sink_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define KITCHEN_SINK_ABI_VERSION 3u -KITCHEN_SINK_API uint32_t kitchen_sink_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by kitchen_sink_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct kitchen_sink_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} kitchen_sink_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -KITCHEN_SINK_API void kitchen_sink_error_set(kitchen_sink_error* err, int32_t code, const char* message); -KITCHEN_SINK_API void kitchen_sink_error_clear(kitchen_sink_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - kitchen_sink_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_error_free(kitchen_sink_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -KITCHEN_SINK_API void kitchen_sink_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct kitchen_sink_cancel_token kitchen_sink_cancel_token; -KITCHEN_SINK_API kitchen_sink_cancel_token* kitchen_sink_cancel_token_create(void); -KITCHEN_SINK_API void kitchen_sink_cancel_token_cancel(kitchen_sink_cancel_token* token); -KITCHEN_SINK_API bool kitchen_sink_cancel_token_is_cancelled(const kitchen_sink_cancel_token* token); -KITCHEN_SINK_API void kitchen_sink_cancel_token_destroy(kitchen_sink_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -KITCHEN_SINK_API uint8_t* kitchen_sink_alloc(uint32_t size); -KITCHEN_SINK_API void kitchen_sink_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -KITCHEN_SINK_API uint64_t kitchen_sink_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define KITCHEN_SINK_SHARED_CHECKSUM 0x42c4ce2c8d0af052ull -KITCHEN_SINK_API uint64_t kitchen_sink_shared_checksum(void); -#define KITCHEN_SINK_KITCHEN_CHECKSUM 0xd83da75b24b545ffull -KITCHEN_SINK_API uint64_t kitchen_sink_kitchen_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with kitchen_sink_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * kitchen_sink_buffer.h declares a C struct and codec for each. - */ - -/** Task priority levels */ -typedef enum { - /** Low priority */ - kitchen_sink_kitchen_Priority_Low = 0, - /** Normal priority */ - kitchen_sink_kitchen_Priority_Normal = 1, - /** High priority */ - kitchen_sink_kitchen_Priority_High = 2 -} kitchen_sink_kitchen_Priority; -/** Error codes reported by throwing functions in the `kitchen` module tree. */ -typedef enum { kitchen_sink_kitchen_KitchenErrors_NotFound = 1, kitchen_sink_kitchen_KitchenErrors_InvalidInput = 2 } kitchen_sink_kitchen_KitchenErrors; -typedef struct kitchen_sink_kitchen_Gadget kitchen_sink_kitchen_Gadget; -typedef struct kitchen_sink_kitchen_StreamGadgetsIterator kitchen_sink_kitchen_StreamGadgetsIterator; -typedef struct kitchen_sink_kitchen_StreamItemsIterator kitchen_sink_kitchen_StreamItemsIterator; -/** - * Consumer-implemented observer of ready items - * - * Consumer-implemented callback interface. The consumer passes a context pointer plus a pointer to a static instance of this vtable; the producer may call any entry from any thread until it calls `free(ctx)` exactly once. - */ -typedef struct kitchen_sink_kitchen_ReadyListener_vtable { - /** Fires when an item is ready */ - void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); - /** Receives the item itself and says whether to keep listening */ - bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); -} kitchen_sink_kitchen_ReadyListener_vtable; -typedef void (*kitchen_sink_kitchen_do_async_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); -typedef void (*kitchen_sink_kitchen_do_cancellable_callback)(void* context, kitchen_sink_error* err, const uint8_t* result_ptr, size_t result_len); - -// Module: shared -/** Trivial cross-module helper */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_shared_ping(size_t* out_len, kitchen_sink_error* out_err); - -// Module: kitchen -/** A pocket-sized interface exercising the object surface */ -/** Create a gadget with the given id */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_new(int64_t id, kitchen_sink_error* out_err); -/** The gadget subsystem version string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_version(size_t* out_len, kitchen_sink_error* out_err); -/** Render the gadget as a human-readable string */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_Gadget_describe(const kitchen_sink_kitchen_Gadget* self, size_t* out_len, kitchen_sink_error* out_err); -/** Poke the gadget a number of times, failing on a negative count */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_Gadget_poke(const kitchen_sink_kitchen_Gadget* self, int32_t times, kitchen_sink_error* out_err); -/** Returns a new strong reference to the same object (the pointer value is unchanged). Null is a no-op returning null. */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_Gadget_clone(const kitchen_sink_kitchen_Gadget* self); -/** Releases one strong reference; the object is dropped when the last reference is released. Null is a no-op. */ -KITCHEN_SINK_API void kitchen_sink_kitchen_Gadget_destroy(kitchen_sink_kitchen_Gadget* self); - -/** Identity for bool */ -KITCHEN_SINK_API bool kitchen_sink_kitchen_bool_id(bool x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_i32_id(int32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API uint32_t kitchen_sink_kitchen_u32_id(uint32_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API int64_t kitchen_sink_kitchen_i64_id(int64_t x, kitchen_sink_error* out_err); -KITCHEN_SINK_API double kitchen_sink_kitchen_f64_id(double x, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_string(const uint8_t* s_ptr, size_t s_len, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_echo_bytes(const uint8_t* b_ptr, size_t b_len, size_t* out_len, kitchen_sink_error* out_err); -/** Register a listener, returning how many are attached */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); -/** Optional object in both directions */ -KITCHEN_SINK_API kitchen_sink_kitchen_Gadget* kitchen_sink_kitchen_maybe_gadget(const kitchen_sink_kitchen_Gadget* fallback, kitchen_sink_error* out_err); -/** A list of objects */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_gadgets(size_t* out_len, kitchen_sink_error* out_err); -/** An iterator over objects */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamGadgetsIterator* kitchen_sink_kitchen_stream_gadgets(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamGadgetsIterator_next(kitchen_sink_kitchen_StreamGadgetsIterator* iter, kitchen_sink_kitchen_Gadget** out_item, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamGadgetsIterator_destroy(kitchen_sink_kitchen_StreamGadgetsIterator* iter); -/** Optional struct return */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_maybe_item(int64_t id, size_t* out_len, kitchen_sink_error* out_err); -/** List of structs */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_list_items(size_t* out_len, kitchen_sink_error* out_err); -/** Map return type */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_get_attrs(size_t* out_len, kitchen_sink_error* out_err); -/** Iterator return type */ -KITCHEN_SINK_API kitchen_sink_kitchen_StreamItemsIterator* kitchen_sink_kitchen_stream_items(kitchen_sink_error* out_err); -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_StreamItemsIterator_next(kitchen_sink_kitchen_StreamItemsIterator* iter, const uint8_t** out_item, size_t* out_len, kitchen_sink_error* out_err); -KITCHEN_SINK_API void kitchen_sink_kitchen_StreamItemsIterator_destroy(kitchen_sink_kitchen_StreamItemsIterator* iter); -/** Returns the shared Token type from another module */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_cross_module_token(size_t* out_len, kitchen_sink_error* out_err); -/** Async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_async(const uint8_t* input_ptr, size_t input_len, kitchen_sink_kitchen_do_async_callback callback, void* context); -/** Cancellable async operation */ -KITCHEN_SINK_API void kitchen_sink_kitchen_do_cancellable(const uint8_t* input_ptr, size_t input_len, kitchen_sink_cancel_token* cancel_token, kitchen_sink_kitchen_do_cancellable_callback callback, void* context); -/** Legacy operation kept for compatibility */ -KITCHEN_SINK_DEPRECATED("Use new_op instead") -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_legacy_op(kitchen_sink_error* out_err); -/** Replacement for legacy_op */ -KITCHEN_SINK_API int32_t kitchen_sink_kitchen_new_op(kitchen_sink_error* out_err); - -// Module: kitchen_nested -/** Trivial nested-module function */ -KITCHEN_SINK_API const uint8_t* kitchen_sink_kitchen_nested_hello(size_t* out_len, kitchen_sink_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // KITCHEN_SINK_H - -// END kitchen_sink.h diff --git a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_module_modulemap.snap b/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_module_modulemap.snap index 1aa9b7c9..1eaaf986 100644 --- a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_module_modulemap.snap +++ b/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_CKitchenSink_module_modulemap.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate module CKitchenSink [system] { header "kitchen_sink.h" diff --git a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_KitchenSink_KitchenSink_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_KitchenSink_KitchenSink_swift.snap index 99157aaf..85e77d8c 100644 --- a/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_KitchenSink_KitchenSink_swift.snap +++ b/crates/weaveffi-cli/tests/snapshots/swift_kitchen_sink__Sources_KitchenSink_KitchenSink_swift.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import CKitchenSink import Foundation @@ -11,7 +11,7 @@ import Foundation // MARK: - Runtime support /// A failure the native library reports outside any declared error domain: -/// an unknown error code or a runtime trap. Throwing wrappers raise it; +/// an unknown error code or a runtime failure. Throwing wrappers raise it; /// non-throwing wrappers stop the process with `fatalError` instead, since /// Swift has no unchecked errors. public struct KitchenSinkRuntimeError: Error, LocalizedError, Hashable, Sendable { @@ -42,30 +42,43 @@ public struct KitchenSinkRuntimeError: Error, LocalizedError, Hashable, Sendable typealias WvError = kitchen_sink_error /// The C ABI revision these bindings were generated against. -let wvAbiVersion: UInt32 = 3 +let wvAbiVersion: UInt32 = 4 -/// The contract check, run once before the first native call: the library -/// must implement this ABI revision and every top-level module must carry -/// the checksum these bindings were generated from. -let wvContract: Void = { +/// Runs the load-time checks if they haven't run yet. +@inline(__always) +func wvLoad() { + _ = wvContract +} + +/// Stops the process unless the library implements the ABI revision these +/// bindings were generated against. +func wvCheckAbiVersion() { let abi = kitchen_sink_abi_version() guard abi == wvAbiVersion else { fatalError("KitchenSink: the native library 'kitchen_sink' implements C ABI revision \(abi), but these bindings need revision \(wvAbiVersion)") } - wvCheckModule("shared", kitchen_sink_shared_checksum(), 0x42c4ce2c8d0af052) - wvCheckModule("kitchen", kitchen_sink_kitchen_checksum(), 0xd83da75b24b545ff) -}() - -/// Runs the contract check if it hasn't run yet. -@inline(__always) -func wvLoad() { - _ = wvContract } -/// Stops the process when a module's contract checksum doesn't match. -func wvCheckModule(_ module: StaticString, _ actual: UInt64, _ expected: UInt64) { - guard actual == expected else { - fatalError("KitchenSink: module '\(module)' of the native library 'kitchen_sink' doesn't match these bindings (checksum 0x\(String(actual, radix: 16)), expected 0x\(String(expected, radix: 16))); regenerate the bindings or rebuild the library") +/// Stops the process unless every declaration these bindings were generated +/// with is in a top-level module's contract table with an equal signature +/// hash. Declarations the library adds are fine. +func wvCheckContract( + _ table: (UnsafeMutablePointer?) -> UnsafePointer?, + _ expected: [(id: UInt64, hash: UInt64, path: StaticString)] +) { + var len = 0 + let entries = UnsafeBufferPointer(start: table(&len), count: len) + var hashes: [UInt64: UInt64] = [:] + for entry in entries { + hashes[entry.id] = entry.hash + } + for (id, hash, path) in expected { + guard let actual = hashes[id] else { + fatalError("KitchenSink: \(path) is missing from the library 'kitchen_sink'; regenerate the bindings or rebuild the library") + } + guard actual == hash else { + fatalError("KitchenSink: \(path) changed since these bindings were generated; regenerate the bindings or rebuild the library 'kitchen_sink'") + } } } @@ -161,10 +174,12 @@ func wvWithBytes(_ d: Data, _ body: (UnsafePointer?, Int) throws -> R) try d.withUnsafeBytes { try body($0.baseAddress?.assumingMemoryBound(to: UInt8.self), $0.count) } } -/// Lends an encoded value buffer for one call. +/// Encodes `value` as a value buffer and lends it for one call. @inline(__always) -func wvWithBuffer(_ w: WvWriter, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { - try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } +func wvWithEncoded(_ value: T, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { + var w = WvWriter() + w.write(value) + return try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } } /// Copies a returned string, then releases the library's allocation. @@ -182,11 +197,11 @@ func wvTakeBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a returned value buffer in place, then releases it. -func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { defer { if let ptr = ptr { kitchen_sink_free_bytes(UnsafeMutablePointer(mutating: ptr), len) } } - return wvBorrowBuffer(ptr, len, decode) + return wvBorrowBuffer(ptr, len, as: type) } /// Copies a borrowed string. @@ -202,9 +217,9 @@ func wvBorrowBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a borrowed value buffer in place. -func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { var r = WvReader(UnsafeRawPointer(ptr), len) - let value = decode(&r) + let value: T = r.read() r.finish() return value } @@ -263,6 +278,12 @@ struct WvWriter { /// Writes an object token. `p` must be a strong reference the buffer now /// owns (a freshly cloned pointer), never one a wrapper still holds. mutating func writeObject(_ p: OpaquePointer) { append(UInt64(UInt(bitPattern: p))) } + + /// Writes any value that crosses inside a value buffer. + @inline(__always) + mutating func write(_ value: T) { + value.wvWrite(&self) + } } /// Deserializes values from the value-buffer wire format directly from the @@ -356,11 +377,154 @@ struct WvReader { return p } + /// Reads any value that crosses inside a value buffer, typed by context. + @inline(__always) + mutating func read() -> T { + T.wvRead(&self) + } + func finish() { if remaining != 0 { wvDecodeFailure("trailing bytes after value") } } } +// MARK: Value-buffer codecs + +/// A type that crosses the C ABI inside a value buffer. Every primitive, +/// optional, list, and map conforms here; each record, enum, and interface +/// conforms where it's declared. +protocol WvCodable { + /// Reads one value, adopting the reference of any object token. + static func wvRead(_ r: inout WvReader) -> Self + /// Writes this value, writing a fresh reference for any object. + func wvWrite(_ w: inout WvWriter) +} + +/// A C-style enum, which crosses as its `i32` discriminant. +protocol WvCEnum: WvCodable, RawRepresentable where RawValue == Int32 {} + +extension WvCEnum { + static func wvRead(_ r: inout WvReader) -> Self { wvEnumCase(Self.self, r.readI32()) } + func wvWrite(_ w: inout WvWriter) { w.writeI32(rawValue) } +} + +extension Bool: WvCodable { + static func wvRead(_ r: inout WvReader) -> Bool { r.readBool() } + func wvWrite(_ w: inout WvWriter) { w.writeBool(self) } +} + +extension Int8: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int8 { r.readI8() } + func wvWrite(_ w: inout WvWriter) { w.writeI8(self) } +} + +extension UInt8: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt8 { r.readU8() } + func wvWrite(_ w: inout WvWriter) { w.writeU8(self) } +} + +extension Int16: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int16 { r.readI16() } + func wvWrite(_ w: inout WvWriter) { w.writeI16(self) } +} + +extension UInt16: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt16 { r.readU16() } + func wvWrite(_ w: inout WvWriter) { w.writeU16(self) } +} + +extension Int32: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int32 { r.readI32() } + func wvWrite(_ w: inout WvWriter) { w.writeI32(self) } +} + +extension UInt32: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt32 { r.readU32() } + func wvWrite(_ w: inout WvWriter) { w.writeU32(self) } +} + +extension Int64: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int64 { r.readI64() } + func wvWrite(_ w: inout WvWriter) { w.writeI64(self) } +} + +extension UInt64: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt64 { r.readU64() } + func wvWrite(_ w: inout WvWriter) { w.writeU64(self) } +} + +extension Float: WvCodable { + static func wvRead(_ r: inout WvReader) -> Float { r.readF32() } + func wvWrite(_ w: inout WvWriter) { w.writeF32(self) } +} + +extension Double: WvCodable { + static func wvRead(_ r: inout WvReader) -> Double { r.readF64() } + func wvWrite(_ w: inout WvWriter) { w.writeF64(self) } +} + +extension String: WvCodable { + static func wvRead(_ r: inout WvReader) -> String { r.readString() } + func wvWrite(_ w: inout WvWriter) { w.writeString(self) } +} + +extension Data: WvCodable { + static func wvRead(_ r: inout WvReader) -> Data { r.readBytes() } + func wvWrite(_ w: inout WvWriter) { w.writeBytes(self) } +} + +extension Optional: WvCodable where Wrapped: WvCodable { + static func wvRead(_ r: inout WvReader) -> Wrapped? { + r.readOptionFlag() ? .some(r.read()) : nil + } + + func wvWrite(_ w: inout WvWriter) { + w.writeOptionFlag(self != nil) + if let value = self { w.write(value) } + } +} + +extension Array: WvCodable where Element: WvCodable { + static func wvRead(_ r: inout WvReader) -> [Element] { + let n = r.readCount() + var items: [Element] = [] + items.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0.. [Key: Value] { + let n = r.readCount() + var map: [Key: Value] = [:] + map.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0..: @unchecked Sendable { } } -/// Reports a thrown Swift error to the library as a callback failure -/// (code -4); the library aborts its current call with that code and message. -func wvForeignError(_ outErr: UnsafeMutablePointer?, _ error: Error) { - let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) - message.withCString { kitchen_sink_error_set(outErr, KitchenSinkRuntimeError.foreignCode, $0) } +/// Holds one callback implementation behind the `ctx` the library keeps. +final class WvBox: @unchecked Sendable { + let value: T + + init(_ value: T) { + self.value = value + } +} + +/// Retains `impl` for the library: the `ctx` of a callback parameter, which +/// the vtable's `free` entry releases with ``wvRelease(_:as:)``. +func wvRetain(_ impl: T) -> UnsafeMutableRawPointer { + Unmanaged.passRetained(WvBox(impl)).toOpaque() } +/// Releases the implementation behind `ctx`, from the vtable's `free` entry. +func wvRelease(_ ctx: UnsafeMutableRawPointer?, as type: T.Type) { + Unmanaged>.fromOpaque(ctx!).release() +} + +/// Runs one callback method on the implementation behind `ctx`. A thrown +/// error never unwinds through the library: `domain` reports it when it's +/// the method's declared error, anything else is reported as a callback +/// failure (`-4`), and the method returns `fallback`. +func wvInvoke( + _ ctx: UnsafeMutableRawPointer?, + _ outErr: UnsafeMutablePointer?, + as type: T.Type, + fallback: R, + domain: (Error, UnsafeMutablePointer?) -> Bool = { _, _ in false }, + _ body: (T) throws -> R +) -> R { + let impl = Unmanaged>.fromOpaque(ctx!).takeUnretainedValue().value + do { + return try body(impl) + } catch { + if !domain(error, outErr) { + let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) + message.withCString { kitchen_sink_error_set(outErr, KitchenSinkRuntimeError.foreignCode, $0) } + } + return fallback + } +} + +/// Reports a declared domain error from a callback method: its code, its +/// message, and its fields as the payload. +func wvSetError(_ outErr: UnsafeMutablePointer?, _ code: Int32, _ message: String, _ payload: WvWriter? = nil) { + message.withCString { kitchen_sink_error_set(outErr, code, $0) } + if let payload = payload { + payload.bytes.withUnsafeBufferPointer { kitchen_sink_error_set_payload(outErr, $0.baseAddress, $0.count) } + } +} + +/// Hands `bytes` to the library as a callback method's return: a run +/// allocated with `kitchen_sink_alloc` that the library adopts. +func wvHandOver(_ bytes: UnsafeRawBufferPointer, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + let run = kitchen_sink_alloc(bytes.count) + if let run = run, let base = bytes.baseAddress { + run.initialize(from: base.assumingMemoryBound(to: UInt8.self), count: bytes.count) + } + outPtr?.pointee = run + outLen?.pointee = run == nil ? 0 : bytes.count +} + +/// Returns a string from a callback method through its out slots. +func wvReturnString(_ value: String, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var value = value + value.withUTF8 { wvHandOver(UnsafeRawBufferPointer($0), outPtr, outLen) } +} + +/// Returns bytes from a callback method through its out slots. +func wvReturnBytes(_ value: Data, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + value.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +/// Returns a value buffer from a callback method through its out slots. +func wvReturnBuffer(_ value: T, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var w = WvWriter() + w.write(value) + w.bytes.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +// MARK: - Load-time checks + +/// The load-time checks, run once before the first native call: the library +/// must implement C ABI revision 4 and carry every declaration these +/// bindings were generated with, unchanged. +let wvContract: Void = { + wvCheckAbiVersion() + wvCheckContract(kitchen_sink_shared_contract, [ + (0x709e234caa962446, 0x3894f2ca452e9f98, "shared.ping"), + (0xcfb306fa5801ee5b, 0x1cbd9e10983cac2c, "shared.Token"), + ]) + wvCheckContract(kitchen_sink_kitchen_contract, [ + (0x01d4f09bbddc986f, 0x29eb88fecc66544d, "kitchen.Gadget.new"), + (0x10472d8fe748c2ee, 0xb265832b6a759fc4, "kitchen.maybe_gadget"), + (0x11368eec07a74078, 0xfa049f78b77e146a, "kitchen.gadgets"), + (0x137f08a29cde6021, 0xda8de7aeb2dbb8a3, "kitchen.stream_gadgets"), + (0x2111b54980826dd7, 0x452783f3e6ca6e1d, "kitchen.do_cancellable"), + (0x29dddbc60bf8d7db, 0x452a3e573d57f6db, "kitchen.Priority"), + (0x2b227003b74a0d16, 0x12332957470e9782, "kitchen.maybe_subscribe"), + (0x2dd66cb0064f0494, 0x60197363fefd92dd, "kitchen.Item"), + (0x33c265bbf079b0e9, 0xc3dcd0927e6c1deb, "kitchen.u32_id"), + (0x33e6381d01660dfe, 0x077c63c31fd7f90a, "kitchen.KitchenErrors"), + (0x3f5fc99d73504b76, 0x2df88d710f1ea10e, "kitchen.Gadget.poke"), + (0x4c5764cf109528d1, 0x2a65529af4338a58, "kitchen.maybe_item"), + (0x4de17dfb93fd494d, 0xba0ad0dd52ed74af, "kitchen.i32_id"), + (0x504e95b7f7e65570, 0x01b3e8e0c9a10e23, "kitchen.stream_items"), + (0x68cdb9a4bd3e3dd5, 0xc678b9164348d497, "kitchen.bool_id"), + (0x69e7cd703d16eb85, 0xf3b28dd22ae9e689, "kitchen.do_async"), + (0x70865079ce41be82, 0xbe3b69464672af46, "kitchen.Gadget.describe"), + (0x71c5a1116cb7c49a, 0xcba1837108a89b84, "kitchen.echo_bytes"), + (0x735ee1e0049ee91a, 0xca74320bc53eddab, "kitchen.echo_string"), + (0x76d7fc58f4e41d84, 0x7373a0e47d30e690, "kitchen.i64_id"), + (0x79abfe1aa960fd6a, 0xbca7a9e5eb26f3a6, "kitchen.nested.hello"), + (0x7d3dd7d5cf6749e3, 0xa192f5a7298adb39, "kitchen.new_op"), + (0x89a5e664bc25c67b, 0x5764a95e81ecfd31, "kitchen.f64_id"), + (0x8cfdbdcecac5f7f1, 0x07e272f09c41fffc, "kitchen.Gadget.version"), + (0x974b9f3e823c1738, 0xb81b91e975695f27, "kitchen.cross_module_token"), + (0xaca0655fac15d230, 0x11c4d2b28abe07bb, "kitchen.list_items"), + (0xc883b4afae20543e, 0xc40560e5411ff392, "kitchen.legacy_op"), + (0xf2f7753e6e1dffc2, 0xc9047a95bdf49e93, "kitchen.get_attrs"), + (0xfae25d3391922ead, 0x1c7f95739f3210b8, "kitchen.subscribe"), + (0xfd0059b05cb2fcfb, 0x248bf8404a5823b4, "kitchen.Gadget"), + (0xfed9c551fb9fd2d6, 0x59aec980ef3f3291, "kitchen.ReadyListener"), + ]) +}() + // MARK: - API /// A reusable token referenced across modules @@ -424,15 +709,15 @@ public struct Token: Hashable, Sendable { } } -func wvWriteToken(_ value: Token, into w: inout WvWriter) { - w.writeI64(value.id) - w.writeString(value.label) -} +extension Token: WvCodable { + static func wvRead(_ r: inout WvReader) -> Token { + Token(id: r.read(), label: r.read()) + } -func wvReadToken(_ r: inout WvReader) -> Token { - let v0 = r.readI64() - let v1 = r.readString() - return Token(id: v0, label: v1) + func wvWrite(_ w: inout WvWriter) { + w.write(self.id) + w.write(self.label) + } } /// Typed errors reported by the `kitchen` module. @@ -457,7 +742,7 @@ public enum KitchenErrorsError: Error, LocalizedError, Sendable { } } -func mapKitchen(_ err: WvError) -> Error { +func wvMapKitchen(_ err: WvError) -> Error { let message = wvErrorMessage(err) switch err.code { case 1: return KitchenErrorsError.notFound(message: message.isEmpty ? "Item not found" : message) @@ -466,13 +751,27 @@ func mapKitchen(_ err: WvError) -> Error { } } -func checkKitchen(_ err: inout WvError) throws { +func wvCheckKitchen(_ err: inout WvError) throws { guard err.code != 0 else { return } - let error = mapKitchen(err) + let error = wvMapKitchen(err) kitchen_sink_error_clear(&err) throw error } +/// Reports a ``KitchenErrorsError`` thrown by a callback method declared to throw it: its +/// code, its message, and its fields as the payload. Any other error is left +/// to the caller, which reports a callback failure. +func wvReportKitchen(_ error: Error, _ outErr: UnsafeMutablePointer?) -> Bool { + guard let error = error as? KitchenErrorsError else { return false } + switch error { + case let .notFound(message): + wvSetError(outErr, 1, message) + case let .invalidInput(message): + wvSetError(outErr, 2, message) + } + return true +} + /// Task priority levels public enum Priority: Int32, Sendable { /// Low priority @@ -483,6 +782,8 @@ public enum Priority: Int32, Sendable { case high = 2 } +extension Priority: WvCEnum {} + /// Kitchen sink struct exercising every field feature public struct Item: Sendable { /// Stable identifier @@ -524,123 +825,90 @@ public struct Item: Sendable { } } -func wvWriteItem(_ value: Item, into w: inout WvWriter) { - w.writeI64(value.id) - w.writeString(value.name) - w.writeI32(value.count) - w.writeBool(value.enabled) - w.writeF64(value.ratio) - w.writeU32(value.nick) - w.writeBytes(value.payload) - w.writeLen(value.tags.count) - for v0 in value.tags { - w.writeString(v0) - } - w.writeLen(value.attrs.count) - for (v1, v2) in value.attrs { - w.writeString(v1) - w.writeString(v2) - } - if let v3 = value.parent { - w.writeOptionFlag(true) - w.writeI64(v3) - } else { - w.writeOptionFlag(false) - } - wvWriteToken(value.token, into: &w) - w.writeI32(value.priority.rawValue) - if let v4 = value.gadget { - w.writeOptionFlag(true) - w.writeObject(v4.clonePtr()) - } else { - w.writeOptionFlag(false) - } -} - -func wvReadItem(_ r: inout WvReader) -> Item { - let v5 = r.readI64() - let v6 = r.readString() - let v7 = r.readI32() - let v8 = r.readBool() - let v9 = r.readF64() - let v10 = r.readU32() - let v11 = r.readBytes() - let n13 = r.readCount() - var v12: [String] = [] - v12.reserveCapacity(min(n13, r.remaining)) - for _ in 0.. Item { + Item(id: r.read(), name: r.read(), count: r.read(), enabled: r.read(), ratio: r.read(), nick: r.read(), payload: r.read(), tags: r.read(), attrs: r.read(), parent: r.read(), token: r.read(), priority: r.read(), gadget: r.read()) + } + + func wvWrite(_ w: inout WvWriter) { + w.write(self.id) + w.write(self.name) + w.write(self.count) + w.write(self.enabled) + w.write(self.ratio) + w.write(self.nick) + w.write(self.payload) + w.write(self.tags) + w.write(self.attrs) + w.write(self.parent) + w.write(self.token) + w.write(self.priority) + w.write(self.gadget) + } } /// Consumer-implemented observer of ready items /// /// Implement this protocol with a class and pass an instance where the API -/// expects it; the library keeps it alive for as long as it holds the -/// callback and may call any method from any thread. A thrown error aborts -/// the library's call and surfaces to the original caller as a foreign error -/// (code -4). +/// expects it. The library keeps the instance until it no longer needs it and +/// may call any method from any thread. A method that throws fails the +/// library's call in progress: a method documented to throw a module error +/// reports that error, with its code and fields, and any other error reaches +/// the library as a callback failure (code -4). public protocol ReadyListener: AnyObject, Sendable { /// Fires when an item is ready func onReady(code: Int32, msg: String) throws + /// Receives the item itself and says whether to keep listening func onItem(item: Item, gadget: Gadget) throws -> Bool -} -/// Retains one `ReadyListener` implementation across the C boundary; the library's -/// `free` entry releases it. -final class WvReadyListenerBox: Sendable { - let impl: any ReadyListener - init(_ impl: any ReadyListener) { self.impl = impl } + /// A display label, failing with a kitchen error + /// + /// - Throws: ``KitchenErrorsError`` to report a declared failure with its fields, or any other error to fail the call. + func label() throws -> String + + /// The most recent item the listener kept, if any + func latest() throws -> Item? + + /// The listener's favorite gadget + func favorite() throws -> Gadget } -/// The process-wide `kitchen_sink_kitchen_ReadyListener_vtable` the library calls `ReadyListener` -/// implementations through. Every entry recovers the box from `ctx`, copies -/// or adopts its arguments, and reports a thrown error through `out_err` -/// instead of unwinding. +/// The process-wide vtable the library calls `ReadyListener` implementations +/// through (`kitchen_sink_kitchen_ReadyListener_vtable`). Every entry recovers the implementation +/// from `ctx`, copies or adopts its arguments, hands its result back, and +/// reports a thrown error through `out_err` instead of unwinding. enum WvReadyListenerVtable { - static let shared = WvVtable(kitchen_sink_kitchen_ReadyListener_vtable( + static let shared = WvVtable(kitchen_sink_kitchen_ReadyListener_vtable( + size: UInt32(MemoryLayout.stride), + flags: 0, + free: { ctx in + wvRelease(ctx, as: (any ReadyListener).self) + }, on_ready: { ctx, code, msg_ptr, msg_len, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - try wvBox.impl.onReady(code: code, msg: wvBorrowString(msg_ptr, msg_len)) - } catch { - wvForeignError(out_err, error) + wvInvoke(ctx, out_err, as: (any ReadyListener).self, fallback: ()) { wvImpl in + try wvImpl.onReady(code: code, msg: wvBorrowString(msg_ptr, msg_len)) } }, on_item: { ctx, item_ptr, item_len, gadget, out_err in - let wvBox = Unmanaged.fromOpaque(ctx!).takeUnretainedValue() - do { - return try wvBox.impl.onItem(item: wvBorrowBuffer(item_ptr, item_len, wvReadItem), gadget: Gadget(ptr: wvNonNull(gadget))) - } catch { - wvForeignError(out_err, error) - return false + wvInvoke(ctx, out_err, as: (any ReadyListener).self, fallback: false) { wvImpl in + try wvImpl.onItem(item: wvBorrowBuffer(item_ptr, item_len, as: Item.self), gadget: Gadget(ptr: wvNonNull(gadget))) } }, - free: { ctx in - Unmanaged.fromOpaque(ctx!).release() + label: { ctx, out_ptr, out_len, out_err in + wvInvoke(ctx, out_err, as: (any ReadyListener).self, fallback: (), domain: wvReportKitchen) { wvImpl in + try wvReturnString(wvImpl.label(), out_ptr, out_len) + } + }, + latest: { ctx, out_ptr, out_len, out_err in + wvInvoke(ctx, out_err, as: (any ReadyListener).self, fallback: ()) { wvImpl in + try wvReturnBuffer(wvImpl.latest(), out_ptr, out_len) + } + }, + favorite: { ctx, out_err in + wvInvoke(ctx, out_err, as: (any ReadyListener).self, fallback: nil) { wvImpl in + try wvImpl.favorite().clonePtr() + } } )) } @@ -658,7 +926,7 @@ public final class Gadget: @unchecked Sendable { } /// Returns a new strong reference to the same object, for a position that - /// takes ownership of it (an object token inside a value buffer). + /// takes ownership of it. func clonePtr() -> OpaquePointer { wvNonNull(kitchen_sink_kitchen_Gadget_clone(ptr)) } @@ -682,10 +950,12 @@ public final class Gadget: @unchecked Sendable { } /// Poke the gadget a number of times, failing on a negative count + /// + /// - Throws: ``KitchenErrorsError`` for a declared failure, ``KitchenSinkRuntimeError`` otherwise. public func poke(times: Int32) throws -> Int32 { var err = WvError() let rv = kitchen_sink_kitchen_Gadget_poke(ptr, times, &err) - try checkKitchen(&err) + try wvCheckKitchen(&err) return rv } @@ -700,6 +970,11 @@ public final class Gadget: @unchecked Sendable { } } +extension Gadget: WvCodable { + static func wvRead(_ r: inout WvReader) -> Gadget { Gadget(ptr: r.readObject()) } + func wvWrite(_ w: inout WvWriter) { w.writeObject(clonePtr()) } +} + /// A lazy sequence over the `Gadget` elements streamed by `kitchen_sink_kitchen_stream_gadgets`. /// /// Each `next()` call pulls exactly one element from the library. The @@ -771,7 +1046,7 @@ public final class KitchenStreamItemsIterator: Sequence, IteratorProtocol { destroyHandle() return nil } - return wvTakeBuffer(item, itemLen, wvReadItem) + return wvTakeBuffer(item, itemLen, as: Item.self) } } @@ -852,15 +1127,25 @@ public enum Kitchen { } /// Register a listener, returning how many are attached - public static func subscribe(listener: ReadyListener) -> Int32 { + public static func subscribe(listener: any ReadyListener) -> Int32 { wvLoad() var err = WvError() - let listener_ctx = Unmanaged.passRetained(WvReadyListenerBox(listener)).toOpaque() + let listener_ctx = wvRetain(listener) let rv = kitchen_sink_kitchen_subscribe(listener_ctx, WvReadyListenerVtable.shared.pointer, &err) wvTrap(&err) return rv } + /// Register a listener if one is given + public static func maybeSubscribe(listener: (any ReadyListener)?) -> Bool { + wvLoad() + var err = WvError() + let listener_ctx = listener.map { wvRetain($0) } + let rv = kitchen_sink_kitchen_maybe_subscribe(listener_ctx, listener == nil ? nil : WvReadyListenerVtable.shared.pointer, &err) + wvTrap(&err) + return rv + } + /// Optional object in both directions public static func maybeGadget(fallback: Gadget?) -> Gadget? { wvLoad() @@ -877,22 +1162,12 @@ public enum Kitchen { var outLen = 0 let rv = kitchen_sink_kitchen_gadgets(&outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen) { (r: inout WvReader) -> [Gadget] in - let n1 = r.readCount() - var v0: [Gadget] = [] - v0.reserveCapacity(min(n1, r.remaining)) - for _ in 0.. KitchenStreamGadgetsIterator { wvLoad() var err = WvError() @@ -902,20 +1177,15 @@ public enum Kitchen { } /// Optional struct return + /// + /// - Throws: ``KitchenErrorsError`` for a declared failure, ``KitchenSinkRuntimeError`` otherwise. public static func maybeItem(id: Int64) throws -> Item? { wvLoad() var err = WvError() var outLen = 0 let rv = kitchen_sink_kitchen_maybe_item(id, &outLen, &err) - try checkKitchen(&err) - return wvTakeBuffer(rv, outLen) { (r: inout WvReader) -> Item? in - var v0: Item? = nil - if r.readOptionFlag() { - let v1 = wvReadItem(&r) - v0 = v1 - } - return v0 - } + try wvCheckKitchen(&err) + return wvTakeBuffer(rv, outLen, as: Item?.self) } /// List of structs @@ -925,16 +1195,7 @@ public enum Kitchen { var outLen = 0 let rv = kitchen_sink_kitchen_list_items(&outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen) { (r: inout WvReader) -> [Item] in - let n1 = r.readCount() - var v0: [Item] = [] - v0.reserveCapacity(min(n1, r.remaining)) - for _ in 0.. [String: Int32] in - let n1 = r.readCount() - var v0: [String: Int32] = [:] - v0.reserveCapacity(min(n1, r.remaining)) - for _ in 0.. KitchenStreamItemsIterator { wvLoad() var err = WvError() @@ -976,7 +1226,7 @@ public enum Kitchen { var outLen = 0 let rv = kitchen_sink_kitchen_cross_module_token(&outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadToken) + return wvTakeBuffer(rv, outLen, as: Token.self) } /// Async operation @@ -997,8 +1247,8 @@ public enum Kitchen { } /// Cancellable async operation - /// - Throws: `CancellationError` when the calling task is cancelled, which - /// also cancels the native call. + /// + /// - Throws: `CancellationError` when the calling task is cancelled, which cancels the native call. public static func doCancellable(input: String) async throws -> String { wvLoad() let token = WvCancelToken() diff --git a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Package_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Package_swift.snap deleted file mode 100644 index 49a75d55..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Package_swift.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// swift-tools-version:5.9 -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -import Foundation -import PackageDescription - -// The C module `CNestedModules` comes from one of two places. Next to this -// manifest, a prebuilt `CNestedModules.xcframework` (assembled from the native -// libraries, see the README) wins. Otherwise `Sources/CNestedModules` declares -// the generated header as a system library that links `nested_modules` from the -// linker search path, e.g. `swift build -Xlinker -L/path/to/lib`. -let xcframework = "CNestedModules.xcframework" -let cModule: Target = FileManager.default.fileExists(atPath: "\(Context.packageDirectory)/\(xcframework)") - ? .binaryTarget(name: "CNestedModules", path: xcframework) - : .systemLibrary(name: "CNestedModules") - -let package = Package( - name: "NestedModules", - platforms: [.macOS(.v10_15), .iOS(.v13), .tvOS(.v13), .watchOS(.v6)], - products: [ - .library(name: "NestedModules", targets: ["NestedModules"]), - ], - targets: [ - cModule, - .target(name: "NestedModules", dependencies: ["CNestedModules"]), - ] -) - -// END Package.swift diff --git a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_module_modulemap.snap b/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_module_modulemap.snap deleted file mode 100644 index 4bd72477..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_module_modulemap.snap +++ /dev/null @@ -1,14 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module CNestedModules [system] { - header "nested_modules.h" - link "nested_modules" - export * -} - -// END module.modulemap diff --git a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_nested_modules_h.snap b/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_nested_modules_h.snap deleted file mode 100644 index 27c97f2b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_CNestedModules_nested_modules_h.snap +++ /dev/null @@ -1,144 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef NESTED_MODULES_H -#define NESTED_MODULES_H - -#include -#include -#include - -#ifndef NESTED_MODULES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef NESTED_MODULES_BUILD -# define NESTED_MODULES_API __declspec(dllexport) -# else -# define NESTED_MODULES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define NESTED_MODULES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define NESTED_MODULES_API __attribute__((visibility("default"))) -# else -# define NESTED_MODULES_API -# endif -#endif - -#ifndef NESTED_MODULES_DEPRECATED -# if defined(_MSC_VER) -# define NESTED_MODULES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define NESTED_MODULES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define NESTED_MODULES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports nested_modules_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define NESTED_MODULES_ABI_VERSION 3u -NESTED_MODULES_API uint32_t nested_modules_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by nested_modules_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct nested_modules_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} nested_modules_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -NESTED_MODULES_API void nested_modules_error_set(nested_modules_error* err, int32_t code, const char* message); -NESTED_MODULES_API void nested_modules_error_clear(nested_modules_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - nested_modules_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -NESTED_MODULES_API void nested_modules_error_free(nested_modules_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -NESTED_MODULES_API void nested_modules_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct nested_modules_cancel_token nested_modules_cancel_token; -NESTED_MODULES_API nested_modules_cancel_token* nested_modules_cancel_token_create(void); -NESTED_MODULES_API void nested_modules_cancel_token_cancel(nested_modules_cancel_token* token); -NESTED_MODULES_API bool nested_modules_cancel_token_is_cancelled(const nested_modules_cancel_token* token); -NESTED_MODULES_API void nested_modules_cancel_token_destroy(nested_modules_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -NESTED_MODULES_API uint8_t* nested_modules_alloc(uint32_t size); -NESTED_MODULES_API void nested_modules_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -NESTED_MODULES_API uint64_t nested_modules_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define NESTED_MODULES_GRAPHICS_CHECKSUM 0x44d31b5b10fdb90dull -NESTED_MODULES_API uint64_t nested_modules_graphics_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with nested_modules_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * nested_modules_buffer.h declares a C struct and codec for each. - */ - -/** A measurement unit shared with nested submodules. */ -typedef enum { nested_modules_graphics_Unit_pixels = 0, nested_modules_graphics_Unit_points = 1 } nested_modules_graphics_Unit; -/** Error codes reported by throwing functions in the `graphics` module tree. */ -typedef enum { nested_modules_graphics_GraphicsError_OutOfBounds = 1 } nested_modules_graphics_GraphicsError; - -// Module: graphics -/** Construct a Canvas. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_new_canvas(int32_t width, size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes -/** Construct a Shape. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_make_shape(int32_t sides, size_t* out_len, nested_modules_error* out_err); -/** Nested function returning a struct owned by an ancestor module. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_describe(size_t* out_len, nested_modules_error* out_err); - -// Module: graphics_shapes_geometry -/** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ -NESTED_MODULES_API const uint8_t* nested_modules_graphics_shapes_geometry_compute(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, nested_modules_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // NESTED_MODULES_H - -// END nested_modules.h diff --git a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_NestedModules_NestedModules_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_NestedModules_NestedModules_swift.snap index 71d61a75..b9935f75 100644 --- a/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_NestedModules_NestedModules_swift.snap +++ b/crates/weaveffi-cli/tests/snapshots/swift_nested_modules__Sources_NestedModules_NestedModules_swift.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import CNestedModules import Foundation @@ -11,7 +11,7 @@ import Foundation // MARK: - Runtime support /// A failure the native library reports outside any declared error domain: -/// an unknown error code or a runtime trap. Throwing wrappers raise it; +/// an unknown error code or a runtime failure. Throwing wrappers raise it; /// non-throwing wrappers stop the process with `fatalError` instead, since /// Swift has no unchecked errors. public struct NestedModulesRuntimeError: Error, LocalizedError, Hashable, Sendable { @@ -42,29 +42,43 @@ public struct NestedModulesRuntimeError: Error, LocalizedError, Hashable, Sendab typealias WvError = nested_modules_error /// The C ABI revision these bindings were generated against. -let wvAbiVersion: UInt32 = 3 +let wvAbiVersion: UInt32 = 4 -/// The contract check, run once before the first native call: the library -/// must implement this ABI revision and every top-level module must carry -/// the checksum these bindings were generated from. -let wvContract: Void = { +/// Runs the load-time checks if they haven't run yet. +@inline(__always) +func wvLoad() { + _ = wvContract +} + +/// Stops the process unless the library implements the ABI revision these +/// bindings were generated against. +func wvCheckAbiVersion() { let abi = nested_modules_abi_version() guard abi == wvAbiVersion else { fatalError("NestedModules: the native library 'nested_modules' implements C ABI revision \(abi), but these bindings need revision \(wvAbiVersion)") } - wvCheckModule("graphics", nested_modules_graphics_checksum(), 0x44d31b5b10fdb90d) -}() - -/// Runs the contract check if it hasn't run yet. -@inline(__always) -func wvLoad() { - _ = wvContract } -/// Stops the process when a module's contract checksum doesn't match. -func wvCheckModule(_ module: StaticString, _ actual: UInt64, _ expected: UInt64) { - guard actual == expected else { - fatalError("NestedModules: module '\(module)' of the native library 'nested_modules' doesn't match these bindings (checksum 0x\(String(actual, radix: 16)), expected 0x\(String(expected, radix: 16))); regenerate the bindings or rebuild the library") +/// Stops the process unless every declaration these bindings were generated +/// with is in a top-level module's contract table with an equal signature +/// hash. Declarations the library adds are fine. +func wvCheckContract( + _ table: (UnsafeMutablePointer?) -> UnsafePointer?, + _ expected: [(id: UInt64, hash: UInt64, path: StaticString)] +) { + var len = 0 + let entries = UnsafeBufferPointer(start: table(&len), count: len) + var hashes: [UInt64: UInt64] = [:] + for entry in entries { + hashes[entry.id] = entry.hash + } + for (id, hash, path) in expected { + guard let actual = hashes[id] else { + fatalError("NestedModules: \(path) is missing from the library 'nested_modules'; regenerate the bindings or rebuild the library") + } + guard actual == hash else { + fatalError("NestedModules: \(path) changed since these bindings were generated; regenerate the bindings or rebuild the library 'nested_modules'") + } } } @@ -160,10 +174,12 @@ func wvWithBytes(_ d: Data, _ body: (UnsafePointer?, Int) throws -> R) try d.withUnsafeBytes { try body($0.baseAddress?.assumingMemoryBound(to: UInt8.self), $0.count) } } -/// Lends an encoded value buffer for one call. +/// Encodes `value` as a value buffer and lends it for one call. @inline(__always) -func wvWithBuffer(_ w: WvWriter, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { - try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } +func wvWithEncoded(_ value: T, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { + var w = WvWriter() + w.write(value) + return try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } } /// Copies a returned string, then releases the library's allocation. @@ -181,11 +197,11 @@ func wvTakeBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a returned value buffer in place, then releases it. -func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { defer { if let ptr = ptr { nested_modules_free_bytes(UnsafeMutablePointer(mutating: ptr), len) } } - return wvBorrowBuffer(ptr, len, decode) + return wvBorrowBuffer(ptr, len, as: type) } /// Copies a borrowed string. @@ -201,9 +217,9 @@ func wvBorrowBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a borrowed value buffer in place. -func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { var r = WvReader(UnsafeRawPointer(ptr), len) - let value = decode(&r) + let value: T = r.read() r.finish() return value } @@ -262,6 +278,12 @@ struct WvWriter { /// Writes an object token. `p` must be a strong reference the buffer now /// owns (a freshly cloned pointer), never one a wrapper still holds. mutating func writeObject(_ p: OpaquePointer) { append(UInt64(UInt(bitPattern: p))) } + + /// Writes any value that crosses inside a value buffer. + @inline(__always) + mutating func write(_ value: T) { + value.wvWrite(&self) + } } /// Deserializes values from the value-buffer wire format directly from the @@ -355,11 +377,154 @@ struct WvReader { return p } + /// Reads any value that crosses inside a value buffer, typed by context. + @inline(__always) + mutating func read() -> T { + T.wvRead(&self) + } + func finish() { if remaining != 0 { wvDecodeFailure("trailing bytes after value") } } } +// MARK: Value-buffer codecs + +/// A type that crosses the C ABI inside a value buffer. Every primitive, +/// optional, list, and map conforms here; each record, enum, and interface +/// conforms where it's declared. +protocol WvCodable { + /// Reads one value, adopting the reference of any object token. + static func wvRead(_ r: inout WvReader) -> Self + /// Writes this value, writing a fresh reference for any object. + func wvWrite(_ w: inout WvWriter) +} + +/// A C-style enum, which crosses as its `i32` discriminant. +protocol WvCEnum: WvCodable, RawRepresentable where RawValue == Int32 {} + +extension WvCEnum { + static func wvRead(_ r: inout WvReader) -> Self { wvEnumCase(Self.self, r.readI32()) } + func wvWrite(_ w: inout WvWriter) { w.writeI32(rawValue) } +} + +extension Bool: WvCodable { + static func wvRead(_ r: inout WvReader) -> Bool { r.readBool() } + func wvWrite(_ w: inout WvWriter) { w.writeBool(self) } +} + +extension Int8: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int8 { r.readI8() } + func wvWrite(_ w: inout WvWriter) { w.writeI8(self) } +} + +extension UInt8: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt8 { r.readU8() } + func wvWrite(_ w: inout WvWriter) { w.writeU8(self) } +} + +extension Int16: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int16 { r.readI16() } + func wvWrite(_ w: inout WvWriter) { w.writeI16(self) } +} + +extension UInt16: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt16 { r.readU16() } + func wvWrite(_ w: inout WvWriter) { w.writeU16(self) } +} + +extension Int32: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int32 { r.readI32() } + func wvWrite(_ w: inout WvWriter) { w.writeI32(self) } +} + +extension UInt32: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt32 { r.readU32() } + func wvWrite(_ w: inout WvWriter) { w.writeU32(self) } +} + +extension Int64: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int64 { r.readI64() } + func wvWrite(_ w: inout WvWriter) { w.writeI64(self) } +} + +extension UInt64: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt64 { r.readU64() } + func wvWrite(_ w: inout WvWriter) { w.writeU64(self) } +} + +extension Float: WvCodable { + static func wvRead(_ r: inout WvReader) -> Float { r.readF32() } + func wvWrite(_ w: inout WvWriter) { w.writeF32(self) } +} + +extension Double: WvCodable { + static func wvRead(_ r: inout WvReader) -> Double { r.readF64() } + func wvWrite(_ w: inout WvWriter) { w.writeF64(self) } +} + +extension String: WvCodable { + static func wvRead(_ r: inout WvReader) -> String { r.readString() } + func wvWrite(_ w: inout WvWriter) { w.writeString(self) } +} + +extension Data: WvCodable { + static func wvRead(_ r: inout WvReader) -> Data { r.readBytes() } + func wvWrite(_ w: inout WvWriter) { w.writeBytes(self) } +} + +extension Optional: WvCodable where Wrapped: WvCodable { + static func wvRead(_ r: inout WvReader) -> Wrapped? { + r.readOptionFlag() ? .some(r.read()) : nil + } + + func wvWrite(_ w: inout WvWriter) { + w.writeOptionFlag(self != nil) + if let value = self { w.write(value) } + } +} + +extension Array: WvCodable where Element: WvCodable { + static func wvRead(_ r: inout WvReader) -> [Element] { + let n = r.readCount() + var items: [Element] = [] + items.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0.. [Key: Value] { + let n = r.readCount() + var map: [Key: Value] = [:] + map.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0..: @unchecked Sendable { } } -/// Reports a thrown Swift error to the library as a callback failure -/// (code -4); the library aborts its current call with that code and message. -func wvForeignError(_ outErr: UnsafeMutablePointer?, _ error: Error) { - let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) - message.withCString { nested_modules_error_set(outErr, NestedModulesRuntimeError.foreignCode, $0) } +/// Holds one callback implementation behind the `ctx` the library keeps. +final class WvBox: @unchecked Sendable { + let value: T + + init(_ value: T) { + self.value = value + } +} + +/// Retains `impl` for the library: the `ctx` of a callback parameter, which +/// the vtable's `free` entry releases with ``wvRelease(_:as:)``. +func wvRetain(_ impl: T) -> UnsafeMutableRawPointer { + Unmanaged.passRetained(WvBox(impl)).toOpaque() +} + +/// Releases the implementation behind `ctx`, from the vtable's `free` entry. +func wvRelease(_ ctx: UnsafeMutableRawPointer?, as type: T.Type) { + Unmanaged>.fromOpaque(ctx!).release() +} + +/// Runs one callback method on the implementation behind `ctx`. A thrown +/// error never unwinds through the library: `domain` reports it when it's +/// the method's declared error, anything else is reported as a callback +/// failure (`-4`), and the method returns `fallback`. +func wvInvoke( + _ ctx: UnsafeMutableRawPointer?, + _ outErr: UnsafeMutablePointer?, + as type: T.Type, + fallback: R, + domain: (Error, UnsafeMutablePointer?) -> Bool = { _, _ in false }, + _ body: (T) throws -> R +) -> R { + let impl = Unmanaged>.fromOpaque(ctx!).takeUnretainedValue().value + do { + return try body(impl) + } catch { + if !domain(error, outErr) { + let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) + message.withCString { nested_modules_error_set(outErr, NestedModulesRuntimeError.foreignCode, $0) } + } + return fallback + } +} + +/// Reports a declared domain error from a callback method: its code, its +/// message, and its fields as the payload. +func wvSetError(_ outErr: UnsafeMutablePointer?, _ code: Int32, _ message: String, _ payload: WvWriter? = nil) { + message.withCString { nested_modules_error_set(outErr, code, $0) } + if let payload = payload { + payload.bytes.withUnsafeBufferPointer { nested_modules_error_set_payload(outErr, $0.baseAddress, $0.count) } + } } +/// Hands `bytes` to the library as a callback method's return: a run +/// allocated with `nested_modules_alloc` that the library adopts. +func wvHandOver(_ bytes: UnsafeRawBufferPointer, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + let run = nested_modules_alloc(bytes.count) + if let run = run, let base = bytes.baseAddress { + run.initialize(from: base.assumingMemoryBound(to: UInt8.self), count: bytes.count) + } + outPtr?.pointee = run + outLen?.pointee = run == nil ? 0 : bytes.count +} + +/// Returns a string from a callback method through its out slots. +func wvReturnString(_ value: String, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var value = value + value.withUTF8 { wvHandOver(UnsafeRawBufferPointer($0), outPtr, outLen) } +} + +/// Returns bytes from a callback method through its out slots. +func wvReturnBytes(_ value: Data, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + value.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +/// Returns a value buffer from a callback method through its out slots. +func wvReturnBuffer(_ value: T, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var w = WvWriter() + w.write(value) + w.bytes.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +// MARK: - Load-time checks + +/// The load-time checks, run once before the first native call: the library +/// must implement C ABI revision 4 and carry every declaration these +/// bindings were generated with, unchanged. +let wvContract: Void = { + wvCheckAbiVersion() + wvCheckContract(nested_modules_graphics_contract, [ + (0x2c29eef9327a6bd4, 0x12983d5271651389, "graphics.Unit"), + (0x304088bbc28f0015, 0x97e6e809913be6b6, "graphics.new_canvas"), + (0x49f8837532df9ac6, 0x32e5b27e81ee0957, "graphics.Canvas"), + (0x55717fa027336b47, 0xb2bd3a6bdcce48c8, "graphics.shapes.describe"), + (0x7626b91b8f1f53cd, 0x867bbf7cf0f41c74, "graphics.shapes.geometry.compute"), + (0xb3efcb50d9a61ab7, 0x3cb0207dfad2a149, "graphics.GraphicsError"), + (0xc62b16a9591b744b, 0x7895610c8487eb1b, "graphics.shapes.Shape"), + (0xe13001f4983ccd35, 0x16e7de070b4a4cfc, "graphics.shapes.geometry.Detail"), + (0xf3f658bcfc5dba62, 0x2665ccb47356e746, "graphics.shapes.make_shape"), + ]) +}() + // MARK: - API /// Typed errors reported by the `graphics` module. @@ -430,7 +690,7 @@ public enum GraphicsError: Error, LocalizedError, Sendable { } } -func mapGraphics(_ err: WvError) -> Error { +func wvMapGraphics(_ err: WvError) -> Error { let message = wvErrorMessage(err) switch err.code { case 1: return GraphicsError.outOfBounds(message: message.isEmpty ? "coordinate out of bounds" : message) @@ -438,9 +698,9 @@ func mapGraphics(_ err: WvError) -> Error { } } -func checkGraphics(_ err: inout WvError) throws { +func wvCheckGraphics(_ err: inout WvError) throws { guard err.code != 0 else { return } - let error = mapGraphics(err) + let error = wvMapGraphics(err) nested_modules_error_clear(&err) throw error } @@ -451,6 +711,8 @@ public enum Unit: Int32, Sendable { case points = 1 } +extension Unit: WvCEnum {} + /// Top-level struct that references a struct defined one level deeper. public struct Canvas: Hashable, Sendable { public var width: Int32 @@ -464,15 +726,15 @@ public struct Canvas: Hashable, Sendable { } } -func wvWriteCanvas(_ value: Canvas, into w: inout WvWriter) { - w.writeI32(value.width) - wvWriteShape(value.root, into: &w) -} +extension Canvas: WvCodable { + static func wvRead(_ r: inout WvReader) -> Canvas { + Canvas(width: r.read(), root: r.read()) + } -func wvReadCanvas(_ r: inout WvReader) -> Canvas { - let v0 = r.readI32() - let v1 = wvReadShape(&r) - return Canvas(width: v0, root: v1) + func wvWrite(_ w: inout WvWriter) { + w.write(self.width) + w.write(self.root) + } } /// A struct one level deep that references both an ancestor enum and a deeper struct. @@ -491,17 +753,16 @@ public struct Shape: Hashable, Sendable { } } -func wvWriteShape(_ value: Shape, into w: inout WvWriter) { - w.writeI32(value.sides) - w.writeI32(value.unit.rawValue) - wvWriteDetail(value.detail, into: &w) -} +extension Shape: WvCodable { + static func wvRead(_ r: inout WvReader) -> Shape { + Shape(sides: r.read(), unit: r.read(), detail: r.read()) + } -func wvReadShape(_ r: inout WvReader) -> Shape { - let v0 = r.readI32() - let v1 = wvEnumCase(Unit.self, r.readI32()) - let v2 = wvReadDetail(&r) - return Shape(sides: v0, unit: v1, detail: v2) + func wvWrite(_ w: inout WvWriter) { + w.write(self.sides) + w.write(self.unit) + w.write(self.detail) + } } /// A struct two levels deep. @@ -514,13 +775,14 @@ public struct Detail: Hashable, Sendable { } } -func wvWriteDetail(_ value: Detail, into w: inout WvWriter) { - w.writeF64(value.area) -} +extension Detail: WvCodable { + static func wvRead(_ r: inout WvReader) -> Detail { + Detail(area: r.read()) + } -func wvReadDetail(_ r: inout WvReader) -> Detail { - let v0 = r.readF64() - return Detail(area: v0) + func wvWrite(_ w: inout WvWriter) { + w.write(self.area) + } } public enum Graphics { @@ -531,7 +793,7 @@ public enum Graphics { var outLen = 0 let rv = nested_modules_graphics_new_canvas(width, &outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadCanvas) + return wvTakeBuffer(rv, outLen, as: Canvas.self) } public enum Shapes { @@ -542,7 +804,7 @@ public enum Graphics { var outLen = 0 let rv = nested_modules_graphics_shapes_make_shape(sides, &outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadShape) + return wvTakeBuffer(rv, outLen, as: Shape.self) } /// Nested function returning a struct owned by an ancestor module. @@ -552,22 +814,22 @@ public enum Graphics { var outLen = 0 let rv = nested_modules_graphics_shapes_describe(&outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadCanvas) + return wvTakeBuffer(rv, outLen, as: Canvas.self) } public enum Geometry { /// Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. + /// + /// - Throws: ``GraphicsError`` for a declared failure, ``NestedModulesRuntimeError`` otherwise. public static func compute(shape: Shape) throws -> Detail { wvLoad() var err = WvError() - var shapeWriter = WvWriter() - wvWriteShape(shape, into: &shapeWriter) var outLen = 0 - let rv: UnsafePointer? = wvWithBuffer(shapeWriter) { shape_ptr, shape_len in + let rv: UnsafePointer? = wvWithEncoded(shape) { shape_ptr, shape_len in nested_modules_graphics_shapes_geometry_compute(shape_ptr, shape_len, &outLen, &err) } - try checkGraphics(&err) - return wvTakeBuffer(rv, outLen, wvReadDetail) + try wvCheckGraphics(&err) + return wvTakeBuffer(rv, outLen, as: Detail.self) } } } diff --git a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Package_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_shapes__Package_swift.snap deleted file mode 100644 index db67e1ba..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Package_swift.snap +++ /dev/null @@ -1,34 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// swift-tools-version:5.9 -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -import Foundation -import PackageDescription - -// The C module `CShapes` comes from one of two places. Next to this -// manifest, a prebuilt `CShapes.xcframework` (assembled from the native -// libraries, see the README) wins. Otherwise `Sources/CShapes` declares -// the generated header as a system library that links `shapes` from the -// linker search path, e.g. `swift build -Xlinker -L/path/to/lib`. -let xcframework = "CShapes.xcframework" -let cModule: Target = FileManager.default.fileExists(atPath: "\(Context.packageDirectory)/\(xcframework)") - ? .binaryTarget(name: "CShapes", path: xcframework) - : .systemLibrary(name: "CShapes") - -let package = Package( - name: "Shapes", - platforms: [.macOS(.v10_15), .iOS(.v13), .tvOS(.v13), .watchOS(.v6)], - products: [ - .library(name: "Shapes", targets: ["Shapes"]), - ], - targets: [ - cModule, - .target(name: "Shapes", dependencies: ["CShapes"]), - ] -) - -// END Package.swift diff --git a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_module_modulemap.snap b/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_module_modulemap.snap deleted file mode 100644 index 5ce537e7..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_module_modulemap.snap +++ /dev/null @@ -1,14 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -module CShapes [system] { - header "shapes.h" - link "shapes" - export * -} - -// END module.modulemap diff --git a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_shapes_h.snap b/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_shapes_h.snap deleted file mode 100644 index df20c3f5..00000000 --- a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_CShapes_shapes_h.snap +++ /dev/null @@ -1,151 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -#ifndef SHAPES_H -#define SHAPES_H - -#include -#include -#include - -#ifndef SHAPES_API -# if defined(_WIN32) || defined(__CYGWIN__) -# ifdef SHAPES_BUILD -# define SHAPES_API __declspec(dllexport) -# else -# define SHAPES_API __declspec(dllimport) -# endif -# elif defined(__EMSCRIPTEN__) -# define SHAPES_API __attribute__((used, visibility("default"))) -# elif defined(__GNUC__) && (__GNUC__ >= 4) -# define SHAPES_API __attribute__((visibility("default"))) -# else -# define SHAPES_API -# endif -#endif - -#ifndef SHAPES_DEPRECATED -# if defined(_MSC_VER) -# define SHAPES_DEPRECATED(msg) __declspec(deprecated(msg)) -# elif defined(__GNUC__) || defined(__clang__) -# define SHAPES_DEPRECATED(msg) __attribute__((deprecated(msg))) -# else -# define SHAPES_DEPRECATED(msg) -# endif -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -/* The C ABI revision this header was generated against. The producer - exports shapes_abi_version() so a consumer can refuse to load a - library built for a different revision. */ -#define SHAPES_ABI_VERSION 3u -SHAPES_API uint32_t shapes_abi_version(void); - -/* Error slot written by every fallible call. `message` is NUL-terminated - UTF-8. `payload_ptr`/`payload_len` hold the matched error code's fields - serialized in the value buffer format (null when the code declares no - fields). Both are released by shapes_error_clear. Positive codes are - the API's declared error codes; negative codes are runtime traps: - -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - interface implementation failed, -5 cancelled. */ -typedef struct shapes_error { - int32_t code; - const char* message; - const uint8_t* payload_ptr; - size_t payload_len; -} shapes_error; - -/* Fill `err` with `code` and a producer-owned copy of `message`. Callback - interface implementations call this to report a failure (code -4) - without allocating with a foreign allocator. */ -SHAPES_API void shapes_error_set(shapes_error* err, int32_t code, const char* message); -SHAPES_API void shapes_error_clear(shapes_error* err); - -/* Async completions receive a heap-boxed error the consumer owns; - shapes_error_free releases the message, the payload, and the box. - NULL is a no-op. */ -SHAPES_API void shapes_error_free(shapes_error* err); - -/* Release a string, bytes, or value buffer the producer returned. */ -SHAPES_API void shapes_free_bytes(uint8_t* ptr, size_t len); - -/* Cancellation for `cancellable` async functions. A token starts with - one reference owned by the consumer; the producer takes its own for - each call it is passed to, so the consumer may cancel and destroy its - reference at any time. A cancelled call completes with code -5. */ -typedef struct shapes_cancel_token shapes_cancel_token; -SHAPES_API shapes_cancel_token* shapes_cancel_token_create(void); -SHAPES_API void shapes_cancel_token_cancel(shapes_cancel_token* token); -SHAPES_API bool shapes_cancel_token_is_cancelled(const shapes_cancel_token* token); -SHAPES_API void shapes_cancel_token_destroy(shapes_cancel_token* token); - -/* Linear-memory allocator the Wasm glue stages arguments through. - Exported by wasm32 producers only. */ -SHAPES_API uint8_t* shapes_alloc(uint32_t size); -SHAPES_API void shapes_dealloc(uint8_t* ptr, uint32_t size); - -/* Live-allocation counters for leak checks in tests: 0 objects, - 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned allocations. - Always 0 unless the producer enables the `leak-check` feature. */ -SHAPES_API uint64_t shapes_debug_live(int32_t kind); - -/* Contract checksums, one per top-level module. A consumer compares - each function's result with the constant it was generated against - before making any other call. */ -#define SHAPES_SHAPES_CHECKSUM 0x06acf433ffa2a0f0ull -SHAPES_API uint64_t shapes_shapes_checksum(void); - -/* - * Strings, bytes, and value buffers cross as (ptr, len) pairs. A - * parameter named "v" expands to a view borrowed for the call: - * const uint8_t* v_ptr, size_t v_len - * Strings are UTF-8 without a NUL terminator; NULL with length 0 is - * the empty value. A returned string, bytes, or buffer is - * `const uint8_t*` with a trailing `size_t* out_len`, owned by the - * caller and released with shapes_free_bytes. - * Records, rich enums, lists, maps, and optionals are serialized - * in the little-endian value buffer format; when generated, - * shapes_buffer.h declares a C struct and codec for each. - */ - -/** An algebraic shape (sum type with associated data) */ -typedef enum { - /** The empty shape (a unit variant) */ - shapes_shapes_Shape_Empty = 0, - /** A circle with a radius */ - shapes_shapes_Shape_Circle = 1, - /** An axis-aligned rectangle */ - shapes_shapes_Shape_Rectangle = 2, - /** An integer-bounded shape (exercises narrow + wide ints) */ - shapes_shapes_Shape_Bounded = 3, - /** A labeled shape with a small count */ - shapes_shapes_Shape_Labeled = 4 -} shapes_shapes_Shape_Tag; -/** A plain C-style enum (no payloads) */ -typedef enum { shapes_shapes_Channel_Red = 0, shapes_shapes_Channel_Green = 1, shapes_shapes_Channel_Blue = 2 } shapes_shapes_Channel; - -// Module: shapes -/** Render a shape to a string */ -SHAPES_API const uint8_t* shapes_shapes_describe(const uint8_t* shape_ptr, size_t shape_len, size_t* out_len, shapes_error* out_err); -/** Scale a shape by a factor, returning a new shape */ -SHAPES_API const uint8_t* shapes_shapes_scale(const uint8_t* shape_ptr, size_t shape_len, double factor, size_t* out_len, shapes_error* out_err); -/** Sum a list of bytes into a wide integer (numerics smoke) */ -SHAPES_API uint64_t shapes_shapes_sum_bytes(const uint8_t* values_ptr, size_t values_len, shapes_error* out_err); -/** Build a Numerics struct from every numeric primitive */ -SHAPES_API const uint8_t* shapes_shapes_make_numerics(int8_t a, int16_t b, int32_t c, int64_t d, uint8_t e, uint16_t f, uint32_t g, uint64_t h, float i, double j, size_t* out_len, shapes_error* out_err); - - -#ifdef __cplusplus -} -#endif - -#endif // SHAPES_H - -// END shapes.h diff --git a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_Shapes_Shapes_swift.snap b/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_Shapes_Shapes_swift.snap index 3d983c30..95182c14 100644 --- a/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_Shapes_Shapes_swift.snap +++ b/crates/weaveffi-cli/tests/snapshots/swift_shapes__Sources_Shapes_Shapes_swift.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import CShapes import Foundation @@ -11,7 +11,7 @@ import Foundation // MARK: - Runtime support /// A failure the native library reports outside any declared error domain: -/// an unknown error code or a runtime trap. Throwing wrappers raise it; +/// an unknown error code or a runtime failure. Throwing wrappers raise it; /// non-throwing wrappers stop the process with `fatalError` instead, since /// Swift has no unchecked errors. public struct ShapesRuntimeError: Error, LocalizedError, Hashable, Sendable { @@ -42,29 +42,43 @@ public struct ShapesRuntimeError: Error, LocalizedError, Hashable, Sendable { typealias WvError = shapes_error /// The C ABI revision these bindings were generated against. -let wvAbiVersion: UInt32 = 3 +let wvAbiVersion: UInt32 = 4 -/// The contract check, run once before the first native call: the library -/// must implement this ABI revision and every top-level module must carry -/// the checksum these bindings were generated from. -let wvContract: Void = { +/// Runs the load-time checks if they haven't run yet. +@inline(__always) +func wvLoad() { + _ = wvContract +} + +/// Stops the process unless the library implements the ABI revision these +/// bindings were generated against. +func wvCheckAbiVersion() { let abi = shapes_abi_version() guard abi == wvAbiVersion else { fatalError("Shapes: the native library 'shapes' implements C ABI revision \(abi), but these bindings need revision \(wvAbiVersion)") } - wvCheckModule("shapes", shapes_shapes_checksum(), 0x06acf433ffa2a0f0) -}() - -/// Runs the contract check if it hasn't run yet. -@inline(__always) -func wvLoad() { - _ = wvContract } -/// Stops the process when a module's contract checksum doesn't match. -func wvCheckModule(_ module: StaticString, _ actual: UInt64, _ expected: UInt64) { - guard actual == expected else { - fatalError("Shapes: module '\(module)' of the native library 'shapes' doesn't match these bindings (checksum 0x\(String(actual, radix: 16)), expected 0x\(String(expected, radix: 16))); regenerate the bindings or rebuild the library") +/// Stops the process unless every declaration these bindings were generated +/// with is in a top-level module's contract table with an equal signature +/// hash. Declarations the library adds are fine. +func wvCheckContract( + _ table: (UnsafeMutablePointer?) -> UnsafePointer?, + _ expected: [(id: UInt64, hash: UInt64, path: StaticString)] +) { + var len = 0 + let entries = UnsafeBufferPointer(start: table(&len), count: len) + var hashes: [UInt64: UInt64] = [:] + for entry in entries { + hashes[entry.id] = entry.hash + } + for (id, hash, path) in expected { + guard let actual = hashes[id] else { + fatalError("Shapes: \(path) is missing from the library 'shapes'; regenerate the bindings or rebuild the library") + } + guard actual == hash else { + fatalError("Shapes: \(path) changed since these bindings were generated; regenerate the bindings or rebuild the library 'shapes'") + } } } @@ -160,10 +174,12 @@ func wvWithBytes(_ d: Data, _ body: (UnsafePointer?, Int) throws -> R) try d.withUnsafeBytes { try body($0.baseAddress?.assumingMemoryBound(to: UInt8.self), $0.count) } } -/// Lends an encoded value buffer for one call. +/// Encodes `value` as a value buffer and lends it for one call. @inline(__always) -func wvWithBuffer(_ w: WvWriter, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { - try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } +func wvWithEncoded(_ value: T, _ body: (UnsafePointer?, Int) throws -> R) rethrows -> R { + var w = WvWriter() + w.write(value) + return try w.bytes.withUnsafeBufferPointer { try body($0.baseAddress, $0.count) } } /// Copies a returned string, then releases the library's allocation. @@ -181,11 +197,11 @@ func wvTakeBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a returned value buffer in place, then releases it. -func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvTakeBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { defer { if let ptr = ptr { shapes_free_bytes(UnsafeMutablePointer(mutating: ptr), len) } } - return wvBorrowBuffer(ptr, len, decode) + return wvBorrowBuffer(ptr, len, as: type) } /// Copies a borrowed string. @@ -201,9 +217,9 @@ func wvBorrowBytes(_ ptr: UnsafePointer?, _ len: Int) -> Data { } /// Decodes a borrowed value buffer in place. -func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, _ decode: (inout WvReader) -> T) -> T { +func wvBorrowBuffer(_ ptr: UnsafePointer?, _ len: Int, as type: T.Type) -> T { var r = WvReader(UnsafeRawPointer(ptr), len) - let value = decode(&r) + let value: T = r.read() r.finish() return value } @@ -262,6 +278,12 @@ struct WvWriter { /// Writes an object token. `p` must be a strong reference the buffer now /// owns (a freshly cloned pointer), never one a wrapper still holds. mutating func writeObject(_ p: OpaquePointer) { append(UInt64(UInt(bitPattern: p))) } + + /// Writes any value that crosses inside a value buffer. + @inline(__always) + mutating func write(_ value: T) { + value.wvWrite(&self) + } } /// Deserializes values from the value-buffer wire format directly from the @@ -355,11 +377,154 @@ struct WvReader { return p } + /// Reads any value that crosses inside a value buffer, typed by context. + @inline(__always) + mutating func read() -> T { + T.wvRead(&self) + } + func finish() { if remaining != 0 { wvDecodeFailure("trailing bytes after value") } } } +// MARK: Value-buffer codecs + +/// A type that crosses the C ABI inside a value buffer. Every primitive, +/// optional, list, and map conforms here; each record, enum, and interface +/// conforms where it's declared. +protocol WvCodable { + /// Reads one value, adopting the reference of any object token. + static func wvRead(_ r: inout WvReader) -> Self + /// Writes this value, writing a fresh reference for any object. + func wvWrite(_ w: inout WvWriter) +} + +/// A C-style enum, which crosses as its `i32` discriminant. +protocol WvCEnum: WvCodable, RawRepresentable where RawValue == Int32 {} + +extension WvCEnum { + static func wvRead(_ r: inout WvReader) -> Self { wvEnumCase(Self.self, r.readI32()) } + func wvWrite(_ w: inout WvWriter) { w.writeI32(rawValue) } +} + +extension Bool: WvCodable { + static func wvRead(_ r: inout WvReader) -> Bool { r.readBool() } + func wvWrite(_ w: inout WvWriter) { w.writeBool(self) } +} + +extension Int8: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int8 { r.readI8() } + func wvWrite(_ w: inout WvWriter) { w.writeI8(self) } +} + +extension UInt8: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt8 { r.readU8() } + func wvWrite(_ w: inout WvWriter) { w.writeU8(self) } +} + +extension Int16: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int16 { r.readI16() } + func wvWrite(_ w: inout WvWriter) { w.writeI16(self) } +} + +extension UInt16: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt16 { r.readU16() } + func wvWrite(_ w: inout WvWriter) { w.writeU16(self) } +} + +extension Int32: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int32 { r.readI32() } + func wvWrite(_ w: inout WvWriter) { w.writeI32(self) } +} + +extension UInt32: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt32 { r.readU32() } + func wvWrite(_ w: inout WvWriter) { w.writeU32(self) } +} + +extension Int64: WvCodable { + static func wvRead(_ r: inout WvReader) -> Int64 { r.readI64() } + func wvWrite(_ w: inout WvWriter) { w.writeI64(self) } +} + +extension UInt64: WvCodable { + static func wvRead(_ r: inout WvReader) -> UInt64 { r.readU64() } + func wvWrite(_ w: inout WvWriter) { w.writeU64(self) } +} + +extension Float: WvCodable { + static func wvRead(_ r: inout WvReader) -> Float { r.readF32() } + func wvWrite(_ w: inout WvWriter) { w.writeF32(self) } +} + +extension Double: WvCodable { + static func wvRead(_ r: inout WvReader) -> Double { r.readF64() } + func wvWrite(_ w: inout WvWriter) { w.writeF64(self) } +} + +extension String: WvCodable { + static func wvRead(_ r: inout WvReader) -> String { r.readString() } + func wvWrite(_ w: inout WvWriter) { w.writeString(self) } +} + +extension Data: WvCodable { + static func wvRead(_ r: inout WvReader) -> Data { r.readBytes() } + func wvWrite(_ w: inout WvWriter) { w.writeBytes(self) } +} + +extension Optional: WvCodable where Wrapped: WvCodable { + static func wvRead(_ r: inout WvReader) -> Wrapped? { + r.readOptionFlag() ? .some(r.read()) : nil + } + + func wvWrite(_ w: inout WvWriter) { + w.writeOptionFlag(self != nil) + if let value = self { w.write(value) } + } +} + +extension Array: WvCodable where Element: WvCodable { + static func wvRead(_ r: inout WvReader) -> [Element] { + let n = r.readCount() + var items: [Element] = [] + items.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0.. [Key: Value] { + let n = r.readCount() + var map: [Key: Value] = [:] + map.reserveCapacity(Swift.min(n, r.remaining)) + for _ in 0..: @unchecked Sendable { } } -/// Reports a thrown Swift error to the library as a callback failure -/// (code -4); the library aborts its current call with that code and message. -func wvForeignError(_ outErr: UnsafeMutablePointer?, _ error: Error) { - let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) - message.withCString { shapes_error_set(outErr, ShapesRuntimeError.foreignCode, $0) } +/// Holds one callback implementation behind the `ctx` the library keeps. +final class WvBox: @unchecked Sendable { + let value: T + + init(_ value: T) { + self.value = value + } } +/// Retains `impl` for the library: the `ctx` of a callback parameter, which +/// the vtable's `free` entry releases with ``wvRelease(_:as:)``. +func wvRetain(_ impl: T) -> UnsafeMutableRawPointer { + Unmanaged.passRetained(WvBox(impl)).toOpaque() +} + +/// Releases the implementation behind `ctx`, from the vtable's `free` entry. +func wvRelease(_ ctx: UnsafeMutableRawPointer?, as type: T.Type) { + Unmanaged>.fromOpaque(ctx!).release() +} + +/// Runs one callback method on the implementation behind `ctx`. A thrown +/// error never unwinds through the library: `domain` reports it when it's +/// the method's declared error, anything else is reported as a callback +/// failure (`-4`), and the method returns `fallback`. +func wvInvoke( + _ ctx: UnsafeMutableRawPointer?, + _ outErr: UnsafeMutablePointer?, + as type: T.Type, + fallback: R, + domain: (Error, UnsafeMutablePointer?) -> Bool = { _, _ in false }, + _ body: (T) throws -> R +) -> R { + let impl = Unmanaged>.fromOpaque(ctx!).takeUnretainedValue().value + do { + return try body(impl) + } catch { + if !domain(error, outErr) { + let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) + message.withCString { shapes_error_set(outErr, ShapesRuntimeError.foreignCode, $0) } + } + return fallback + } +} + +/// Reports a declared domain error from a callback method: its code, its +/// message, and its fields as the payload. +func wvSetError(_ outErr: UnsafeMutablePointer?, _ code: Int32, _ message: String, _ payload: WvWriter? = nil) { + message.withCString { shapes_error_set(outErr, code, $0) } + if let payload = payload { + payload.bytes.withUnsafeBufferPointer { shapes_error_set_payload(outErr, $0.baseAddress, $0.count) } + } +} + +/// Hands `bytes` to the library as a callback method's return: a run +/// allocated with `shapes_alloc` that the library adopts. +func wvHandOver(_ bytes: UnsafeRawBufferPointer, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + let run = shapes_alloc(bytes.count) + if let run = run, let base = bytes.baseAddress { + run.initialize(from: base.assumingMemoryBound(to: UInt8.self), count: bytes.count) + } + outPtr?.pointee = run + outLen?.pointee = run == nil ? 0 : bytes.count +} + +/// Returns a string from a callback method through its out slots. +func wvReturnString(_ value: String, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var value = value + value.withUTF8 { wvHandOver(UnsafeRawBufferPointer($0), outPtr, outLen) } +} + +/// Returns bytes from a callback method through its out slots. +func wvReturnBytes(_ value: Data, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + value.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +/// Returns a value buffer from a callback method through its out slots. +func wvReturnBuffer(_ value: T, _ outPtr: UnsafeMutablePointer?>?, _ outLen: UnsafeMutablePointer?) { + var w = WvWriter() + w.write(value) + w.bytes.withUnsafeBytes { wvHandOver($0, outPtr, outLen) } +} + +// MARK: - Load-time checks + +/// The load-time checks, run once before the first native call: the library +/// must implement C ABI revision 4 and carry every declaration these +/// bindings were generated with, unchanged. +let wvContract: Void = { + wvCheckAbiVersion() + wvCheckContract(shapes_shapes_contract, [ + (0x0d7b3eec49243e7a, 0x5f232311808e1b1f, "shapes.Channel"), + (0x183fa8198100860e, 0xe40c6993cb31d359, "shapes.describe"), + (0x2d1395eb837b848e, 0x42202a0c12f3a16e, "shapes.sum_bytes"), + (0x7f9a5b87ba09b86b, 0xc5b32fb0efbbaa17, "shapes.scale"), + (0xc8edd779240fda17, 0x68e1be78fddb20e4, "shapes.Numerics"), + (0xd3d3881b2fd18a62, 0xf24d0eb76a99d918, "shapes.make_numerics"), + (0xf010844afff5ac3c, 0x50de28ae9b41ff15, "shapes.Shape"), + ]) +}() + // MARK: - API /// An algebraic shape (sum type with associated data) @@ -425,52 +683,40 @@ public enum Shape: Hashable, Sendable { case labeled(label: String, count: UInt8) } -func wvWriteShape(_ value: Shape, into w: inout WvWriter) { - switch value { - case .empty: - w.writeI32(0) - case let .circle(v0): - w.writeI32(1) - w.writeF64(v0) - case let .rectangle(v1, v2): - w.writeI32(2) - w.writeF32(v1) - w.writeF32(v2) - case let .bounded(v3, v4, v5): - w.writeI32(3) - w.writeI16(v3) - w.writeI16(v4) - w.writeU64(v5) - case let .labeled(v6, v7): - w.writeI32(4) - w.writeString(v6) - w.writeU8(v7) - } -} - -func wvReadShape(_ r: inout WvReader) -> Shape { - let tag = r.readI32() - switch tag { - case 0: - return .empty - case 1: - let v8 = r.readF64() - return .circle(radius: v8) - case 2: - let v9 = r.readF32() - let v10 = r.readF32() - return .rectangle(width: v9, height: v10) - case 3: - let v11 = r.readI16() - let v12 = r.readI16() - let v13 = r.readU64() - return .bounded(min: v11, max: v12, weight: v13) - case 4: - let v14 = r.readString() - let v15 = r.readU8() - return .labeled(label: v14, count: v15) - default: - wvDecodeFailure("unknown Shape tag \(tag)") +extension Shape: WvCodable { + static func wvRead(_ r: inout WvReader) -> Shape { + let tag = r.readI32() + switch tag { + case 0: return .empty + case 1: return .circle(radius: r.read()) + case 2: return .rectangle(width: r.read(), height: r.read()) + case 3: return .bounded(min: r.read(), max: r.read(), weight: r.read()) + case 4: return .labeled(label: r.read(), count: r.read()) + default: wvDecodeFailure("unknown Shape tag \(tag)") + } + } + + func wvWrite(_ w: inout WvWriter) { + switch self { + case .empty: + w.writeI32(0) + case let .circle(v0): + w.writeI32(1) + w.write(v0) + case let .rectangle(v0, v1): + w.writeI32(2) + w.write(v0) + w.write(v1) + case let .bounded(v0, v1, v2): + w.writeI32(3) + w.write(v0) + w.write(v1) + w.write(v2) + case let .labeled(v0, v1): + w.writeI32(4) + w.write(v0) + w.write(v1) + } } } @@ -481,6 +727,8 @@ public enum Channel: Int32, Sendable { case blue = 2 } +extension Channel: WvCEnum {} + /// One field per supported numeric primitive public struct Numerics: Hashable, Sendable { public var a: Int8 @@ -509,42 +757,33 @@ public struct Numerics: Hashable, Sendable { } } -func wvWriteNumerics(_ value: Numerics, into w: inout WvWriter) { - w.writeI8(value.a) - w.writeI16(value.b) - w.writeI32(value.c) - w.writeI64(value.d) - w.writeU8(value.e) - w.writeU16(value.f) - w.writeU32(value.g) - w.writeU64(value.h) - w.writeF32(value.i) - w.writeF64(value.j) -} +extension Numerics: WvCodable { + static func wvRead(_ r: inout WvReader) -> Numerics { + Numerics(a: r.read(), b: r.read(), c: r.read(), d: r.read(), e: r.read(), f: r.read(), g: r.read(), h: r.read(), i: r.read(), j: r.read()) + } -func wvReadNumerics(_ r: inout WvReader) -> Numerics { - let v0 = r.readI8() - let v1 = r.readI16() - let v2 = r.readI32() - let v3 = r.readI64() - let v4 = r.readU8() - let v5 = r.readU16() - let v6 = r.readU32() - let v7 = r.readU64() - let v8 = r.readF32() - let v9 = r.readF64() - return Numerics(a: v0, b: v1, c: v2, d: v3, e: v4, f: v5, g: v6, h: v7, i: v8, j: v9) + func wvWrite(_ w: inout WvWriter) { + w.write(self.a) + w.write(self.b) + w.write(self.c) + w.write(self.d) + w.write(self.e) + w.write(self.f) + w.write(self.g) + w.write(self.h) + w.write(self.i) + w.write(self.j) + } } +/// Rich (algebraic) enums and the expanded numeric primitives. public enum Shapes { /// Render a shape to a string public static func describe(shape: Shape) -> String { wvLoad() var err = WvError() - var shapeWriter = WvWriter() - wvWriteShape(shape, into: &shapeWriter) var outLen = 0 - let rv: UnsafePointer? = wvWithBuffer(shapeWriter) { shape_ptr, shape_len in + let rv: UnsafePointer? = wvWithEncoded(shape) { shape_ptr, shape_len in shapes_shapes_describe(shape_ptr, shape_len, &outLen, &err) } wvTrap(&err) @@ -555,14 +794,12 @@ public enum Shapes { public static func scale(shape: Shape, factor: Double) -> Shape { wvLoad() var err = WvError() - var shapeWriter = WvWriter() - wvWriteShape(shape, into: &shapeWriter) var outLen = 0 - let rv: UnsafePointer? = wvWithBuffer(shapeWriter) { shape_ptr, shape_len in + let rv: UnsafePointer? = wvWithEncoded(shape) { shape_ptr, shape_len in shapes_shapes_scale(shape_ptr, shape_len, factor, &outLen, &err) } wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadShape) + return wvTakeBuffer(rv, outLen, as: Shape.self) } /// Sum a list of bytes into a wide integer (numerics smoke) @@ -583,7 +820,7 @@ public enum Shapes { var outLen = 0 let rv = shapes_shapes_make_numerics(a, b, c, d, e, f, g, h, i, j, &outLen, &err) wvTrap(&err) - return wvTakeBuffer(rv, outLen, wvReadNumerics) + return wvTakeBuffer(rv, outLen, as: Numerics.self) } } diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__README_md.snap b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__README_md.snap deleted file mode 100644 index 999e96c8..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__README_md.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# docs_everywhere - -Native bindings for docs_everywhere - -WebAssembly bindings for a `wasm32-unknown-unknown` build of the `docs_everywhere` library, linked with `--export-table --growable-table` (for example `RUSTFLAGS="-C link-arg=--export-table -C link-arg=--growable-table"`). `init()` loads `docs_everywhere.wasm` from next to this package, or from the path in `DOCS_EVERYWHERE_LIBRARY` on Node.js; pass a URL, bytes, or a compiled module to load it from elsewhere: - -```js -import { init, docs } from 'docs_everywhere'; - -await init(); -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_d_ts.snap index b7cb6012..5be9440f 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -31,18 +31,14 @@ export declare function init( source?: string | URL | BufferSource | WebAssembly.Module | Response, ): Promise; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** - * Create a brand new document. - * - * The document starts in `Draft` status; callers must publish - * it explicitly to make it visible. - */ +/** Document storage: create, publish, and observe documents. */ export declare namespace docs { /** The errors of the `docs` error domain. */ export class DocsErrorsError extends $Error { diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_js.snap index 04b73368..252ff588 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -36,6 +39,28 @@ let $raw = $unloaded('docs_everywhere'); const $token = Number; let $loading = null; +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['docs', [ + [0x00b2187fd75634ecn, 0x2263c6f787bf6081n, 'docs.publish_async'], + [0x12fc242b9dc6444cn, 0x96405f484a642b31n, 'docs.Notebook.insert'], + [0x2213822ed05cd5bdn, 0x26c5833392aa63cen, 'docs.Notebook.new'], + [0x2bd762bcdaa207ffn, 0x1b0ebab2d04c49b0n, 'docs.SavedListener'], + [0x2e086c45e9d617den, 0x4d43bf32358c01ffn, 'docs.Notebook.find'], + [0x4817dc98197ad960n, 0xf8017b5a7d58b2c7n, 'docs.Notebook.size'], + [0x614de47452d5a55cn, 0xd9aae1472494ffefn, 'docs.get_document'], + [0x67c5cb9fc119bcd5n, 0x08d5b25a1a22a7fan, 'docs.Document'], + [0x8d484bf2d78c9b70n, 0xe9093cf96912194cn, 'docs.Notebook.max_documents'], + [0xaea61d6451691c67n, 0xe4704128b3524b3dn, 'docs.watch'], + [0xd68d6ad1540cdec5n, 0x01afeb40b0a8bfe1n, 'docs.Notebook'], + [0xdb0601444a4760ccn, 0xe56c69629b444797n, 'docs.create_document'], + [0xe12d6f4838347b3en, 0xa9bab86646d84f51n, 'docs.Status'], + [0xee2f8d25b3313b50n, 0x9691232f35439a2bn, 'docs.DocsErrors'], + ]], +]; + /** * Load the WebAssembly module (by default `docs_everywhere.wasm` next to this file, or * the file named by `DOCS_EVERYWHERE_LIBRARY` on Node.js) and check that it matches @@ -47,7 +72,7 @@ export function init(source) { .then((m) => { $raw = $bind(m); try { - $verify($raw, 'docs_everywhere', 'docs_everywhere', 3, [['docs', 0x7950eec1e9cfc819n]]); + $verify($raw, 'docs_everywhere', 'docs_everywhere', 4, $contract); } catch (e) { $raw = $unloaded('docs_everywhere'); throw e; @@ -77,11 +102,11 @@ function $bind(m) { const x = m.x; return { docs_everywhere_abi_version: () => x.docs_everywhere_abi_version() >>> 0, - docs_everywhere_debug_live: (kind) => BigInt.asUintN(64, x.docs_everywhere_debug_live(m.num(kind))), + docs_everywhere_debug_live: (kind) => m.live(x.docs_everywhere_debug_live(m.num(kind)), kind), docs_everywhere_cancel_token_create: () => x.docs_everywhere_cancel_token_create(), docs_everywhere_cancel_token_cancel: (t) => x.docs_everywhere_cancel_token_cancel(m.handle(t)), docs_everywhere_cancel_token_destroy: (t) => x.docs_everywhere_cancel_token_destroy(m.handle(t)), - docs_everywhere_docs_checksum: () => BigInt.asUintN(64, x.docs_everywhere_docs_checksum()), + docs_everywhere_docs_contract: () => m.contract(x.docs_everywhere_docs_contract), docs_everywhere_docs_Notebook_clone: (h) => x.docs_everywhere_docs_Notebook_clone(m.handle(h)), docs_everywhere_docs_Notebook_destroy: (h) => x.docs_everywhere_docs_Notebook_destroy(m.handle(h)), docs_everywhere_docs_create_document: (a0, a1) => { @@ -144,9 +169,10 @@ function $bind(m) { } /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.docs_everywhere_debug_live(kind); @@ -165,9 +191,13 @@ const docs$InvalidStatusError = class InvalidStatusError extends docs$DocsErrors super(2, message); } }; -const $codes$docs$DocsErrors = new Map([[1, docs$NotFoundError], [2, docs$InvalidStatusError]]); +const $codes$docs$DocsErrors = new Map([ + [1, docs$NotFoundError], + [2, docs$InvalidStatusError], +]); +const $payloads$docs$DocsErrors = new Map(); function $from$docs$DocsErrors(e) { - return $domain(e, $codes$docs$DocsErrors); + return $domain(e, $codes$docs$DocsErrors, $payloads$docs$DocsErrors); } const docs$Status = Object.freeze({ @@ -183,16 +213,30 @@ function $w$docs$Document(w, v) { w.writeI64(v.id); w.writeString(v.title); w.writeI32(v.status); - w.writeList(v.tags, $W.String); + $w_list_string(w, v.tags); } function $r$docs$Document(r) { return { id: r.readI64(), title: r.readString(), status: r.readI32(), - tags: r.readList($R.String), + tags: $r_list_string(r), }; } +/** `[string]` */ +function $w_list_string(w, v) { + w.writeList(v, $W.String); +} +function $r_list_string(r) { + return r.readList($R.String); +} +/** `Document?` */ +function $w_opt_Document(w, v) { + w.writeOpt(v, $w$docs$Document); +} +function $r_opt_Document(r) { + return r.readOpt($r$docs$Document); +} function $adapt$docs$SavedListener(impl) { $impl(impl, 'SavedListener'); @@ -259,7 +303,7 @@ const docs$Notebook = class Notebook extends $Object { const docs$createDocument = function createDocument(title, tags) { try { - return $decode($raw.docs_everywhere_docs_create_document(title, $encode(tags, (w, v) => w.writeList(v, $W.String))), $r$docs$Document); + return $decode($raw.docs_everywhere_docs_create_document(title, $encode(tags, $w_list_string)), $r$docs$Document); } catch ($e) { throw $fault($e); } @@ -267,7 +311,7 @@ const docs$createDocument = function createDocument(title, tags) { const docs$getDocument = function getDocument(id) { try { - return $decode($raw.docs_everywhere_docs_get_document(id), (r) => r.readOpt($r$docs$Document)); + return $decode($raw.docs_everywhere_docs_get_document(id), $r_opt_Document); } catch ($e) { throw $fault($e); } diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__linear_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__linear_js.snap deleted file mode 100644 index 608de11a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__linear_js.snap +++ /dev/null @@ -1,403 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The WebAssembly transport: loading the module, staging arguments in its -// linear memory, reading results and errors back, and installing JavaScript -// functions in its function table (async completions and callback -// interface vtables). The generated `index.js` builds one native entry point -// per C symbol on top of this. - -import { $Fault } from './runtime.js'; - -const $enc = new TextEncoder(); -const $dec = new TextDecoder(); - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v, bits, what) { - let b; - if (typeof v === 'bigint') { - b = v; - } else if (typeof v === 'number' && Number.isInteger(v)) { - b = BigInt(v); - } else { - throw $expected('a bigint', v); - } - if (bits(64, b) !== b) throw new RangeError(`bigint does not fit in ${what} 64-bit integer`); - return b; -} - -/** - * One loaded module's linear memory and runtime symbols. `x` holds the - * module's exports by C symbol name; `memory()` returns the current - * `ArrayBuffer`; `fnptr(params, results, sig, fn)` installs `fn` in the - * function table and returns its index. - */ -export class $Linear { - constructor(x, prefix, memory, fnptr) { - this.x = x; - this.memory = memory; - this.fnptr = fnptr; - this.allocFn = x[`${prefix}_alloc`]; - this.deallocFn = x[`${prefix}_dealloc`]; - this.freeBytesFn = x[`${prefix}_free_bytes`]; - this.errorClearFn = x[`${prefix}_error_clear`]; - this.errorFreeFn = x[`${prefix}_error_free`]; - this.errorSetFn = x[`${prefix}_error_set`]; - if (typeof this.allocFn !== 'function' || typeof this.deallocFn !== 'function') { - throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_dealloc`); - } - // One scratch block reused by every call: the error slot (16 bytes), - // the out_len slot (4), and the iterator out_item slot (8). The producer - // only writes them as a call returns, so a call made from a callback in - // the middle of another call can reuse them safely. - const scratch = this.allocFn(32); - this.bytes().fill(0, scratch, scratch + 32); - this.err = scratch; - this.len = scratch + 16; - this.item = scratch + 24; - this.callbacks = new Map(); - this.nextCallback = 1; - this.pending = new Map(); - this.nextPending = 1; - this.completions = new Map(); - this.vtables = new Map(); - } - - bytes() { - const buffer = this.memory(); - if (buffer !== this.buffer) { - this.buffer = buffer; - this.u8 = new Uint8Array(buffer); - this.dv = new DataView(buffer); - } - return this.u8; - } - - view() { - this.bytes(); - return this.dv; - } - - // Argument checks and conversions for by-value slots. - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - i64(v) { - return $bigint(v, BigInt.asIntN, 'a signed'); - } - - u64(v) { - return $bigint(v, BigInt.asUintN, 'an unsigned'); - } - - bool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - return v ? 1 : 0; - } - - handle(v) { - if (typeof v !== 'number' || v === 0) throw $expected('an object handle', v); - return v; - } - - handleOpt(v) { - return v === null || v === undefined ? 0 : this.handle(v); - } - - // Staging: copy a string or bytes into linear memory for one call. A - // staged value is [ptr, len, size]; release it with unstage(). - str(s) { - if (typeof s !== 'string') throw $expected('a string', s); - if (s.length === 0) return [0, 0, 0]; - const size = s.length * 3; - const ptr = this.allocFn(size); - const { written } = $enc.encodeInto(s, this.bytes().subarray(ptr, ptr + size)); - return [ptr, written, size]; - } - - data(b) { - if (!(b instanceof Uint8Array)) throw $expected('a Uint8Array', b); - if (b.length === 0) return [0, 0, 0]; - const ptr = this.allocFn(b.length); - this.bytes().set(b, ptr); - return [ptr, b.length, b.length]; - } - - unstage(staged) { - if (staged !== null && staged[2] !== 0) this.deallocFn(staged[0], staged[2]); - } - - // Results. - outLen() { - return this.view().getUint32(this.len, true); - } - - readStr(ptr, len) { - return len === 0 ? '' : $dec.decode(this.bytes().subarray(ptr, ptr + len)); - } - - readData(ptr, len) { - return len === 0 ? new Uint8Array(0) : this.bytes().slice(ptr, ptr + len); - } - - takeStr(ptr, len) { - const s = this.readStr(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return s; - } - - takeData(ptr, len) { - const b = this.readData(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return b; - } - - cstr(ptr) { - if (ptr === 0) return ''; - const u8 = this.bytes(); - let end = ptr; - while (u8[end] !== 0) end++; - return $dec.decode(u8.subarray(ptr, end)); - } - - fault(errPtr) { - const dv = this.view(); - const code = dv.getInt32(errPtr, true); - if (code === 0) return null; - const payloadPtr = dv.getUint32(errPtr + 8, true); - const payloadLen = dv.getUint32(errPtr + 12, true); - return new $Fault( - code, - this.cstr(dv.getUint32(errPtr + 4, true)), - payloadPtr === 0 ? null : this.bytes().slice(payloadPtr, payloadPtr + payloadLen), - ); - } - - /** Throw the error the last call wrote to the scratch error slot. */ - check() { - if (this.view().getInt32(this.err, true) === 0) return; - const fault = this.fault(this.err); - this.errorClearFn(this.err); - throw fault; - } - - // Callback interfaces: implementations live in a table keyed by the - // integer passed as `ctx`, until the producer calls the vtable's `free`. - register(adapter) { - const id = this.nextCallback++; - this.callbacks.set(id, adapter); - return id; - } - - adapter(id) { - const adapter = this.callbacks.get(id); - if (adapter === undefined) throw new Error('callback implementation was already released'); - return adapter; - } - - unregister(id) { - this.callbacks.delete(id); - } - - /** Report a callback implementation's exception through `errPtr` (code -4). */ - foreign(errPtr, e) { - const message = e instanceof Error ? e.message || e.name : String(e); - const bytes = $enc.encode(message); - const ptr = this.allocFn(bytes.length + 1); - const u8 = this.bytes(); - u8.set(bytes, ptr); - u8[ptr + bytes.length] = 0; - this.errorSetFn(errPtr, -4, ptr); - this.deallocFn(ptr, bytes.length + 1); - } - - /** - * The static vtable of one callback interface: one table entry per - * method (`methods()` returns each as `[params, results, sig, fn]`), then - * `free`. Built on first use. - */ - vtable(name, methods) { - let ptr = this.vtables.get(name); - if (ptr !== undefined) return ptr; - const entries = [...methods(), [['i32'], [], 'vi', (ctx) => this.unregister(ctx)]]; - ptr = this.allocFn(entries.length * 4); - const dv = this.view(); - entries.forEach(([params, results, sig, fn], i) => { - dv.setUint32(ptr + i * 4, this.fnptr(params, results, sig, fn), true); - }); - this.vtables.set(name, ptr); - return ptr; - } - - /** - * Launch an async call. `call(callback, context)` invokes the launcher; - * the completion (one table entry per signature, shared by every call) - * settles the returned promise, converting the result slots with - * `convert`. On this single-threaded target the completion normally - * fires before the launcher returns. - */ - launch(params, results, sig, call, convert) { - let callback = this.completions.get(sig); - if (callback === undefined) { - const complete = (ctx, errPtr, ...result) => this.complete(ctx, errPtr, result); - callback = this.fnptr(params, results, sig, complete); - this.completions.set(sig, callback); - } - const id = this.nextPending++; - return new Promise((resolve, reject) => { - this.pending.set(id, { resolve, reject, convert }); - try { - call(callback, id); - } catch (e) { - this.pending.delete(id); - reject(e); - } - }); - } - - complete(id, errPtr, result) { - const pending = this.pending.get(id); - if (pending === undefined) return; - this.pending.delete(id); - try { - const fault = errPtr === 0 ? null : this.fault(errPtr); - if (errPtr !== 0) this.errorFreeFn(errPtr); - if (fault !== null) { - pending.reject(fault); - } else { - pending.resolve(pending.convert(...result)); - } - } catch (e) { - pending.reject(e); - } - } -} - -/** A stand-in for the native entry points until `init()` has finished. */ -export function $unloaded(library) { - return new Proxy( - {}, - { - get() { - throw new Error(`${library}: call init() before using the bindings`); - }, - }, - ); -} - -const $valtype = { i32: 0x7f, i64: 0x7e, f32: 0x7d, f64: 0x7c }; - -function $leb(n) { - const out = []; - do { - let b = n & 0x7f; - n >>>= 7; - if (n !== 0) b |= 0x80; - out.push(b); - } while (n !== 0); - return out; -} - -function $section(id, body) { - return [id, ...$leb(body.length), ...body]; -} - -/** - * Turn a JavaScript function into a WebAssembly function with the given - * signature: `WebAssembly.Function` where the engine has it, otherwise a - * tiny module that imports `fn` and re-exports it. - */ -function $wasmFunction(params, results, fn) { - if (typeof WebAssembly.Function === 'function') { - return new WebAssembly.Function({ parameters: params, results }, fn); - } - const type = [0x60, ...$leb(params.length), ...params.map((t) => $valtype[t])]; - type.push(...$leb(results.length), ...results.map((t) => $valtype[t])); - const bytes = new Uint8Array([ - 0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, - ...$section(1, [1, ...type]), - ...$section(2, [1, 1, 0x65, 1, 0x66, 0x00, 0x00]), - ...$section(7, [1, 1, 0x66, 0x00, 0x00]), - ]); - const instance = new WebAssembly.Instance(new WebAssembly.Module(bytes), { e: { f: fn } }); - return instance.exports.f; -} - -async function $bytesOf(source, envVar, fallback) { - if (source === undefined || source === null) { - const env = typeof process === 'object' && process.env ? process.env[envVar] : undefined; - if (env) { - const { readFile } = await import('node:fs/promises'); - return readFile(env); - } - source = fallback; - } - if (source instanceof WebAssembly.Module || source instanceof ArrayBuffer || ArrayBuffer.isView(source)) { - return source; - } - if (typeof Response === 'function' && source instanceof Response) { - return source.arrayBuffer(); - } - const url = source instanceof URL ? source : new URL(String(source), fallback); - if (url.protocol === 'file:') { - const { readFile } = await import('node:fs/promises'); - return readFile(url); - } - const response = await fetch(url); - if (!response.ok) throw new Error(`failed to fetch ${url}: ${response.status}`); - return response.arrayBuffer(); -} - -/** - * Instantiate a `wasm32-unknown-unknown` module from `source` (a URL, a - * path, bytes, a `Response`, or a compiled module). Without a source it - * reads the file named by the `envVar` environment variable (on Node.js), - * else `fallback`. The module must export its memory and a growable - * function table (`__indirect_function_table`). - */ -export async function $loadWasm(source, prefix, envVar, fallback) { - const bytes = await $bytesOf(source, envVar, fallback); - const module = bytes instanceof WebAssembly.Module ? bytes : await WebAssembly.compile(bytes); - const { exports } = await WebAssembly.instantiate(module, {}); - const table = exports.__indirect_function_table; - const fnptr = (params, results, sig, fn) => { - if (!(table instanceof WebAssembly.Table)) { - throw new Error('the WebAssembly module does not export __indirect_function_table (link with --export-table --growable-table)'); - } - const index = table.grow(1); - table.set(index, $wasmFunction(params, results, fn)); - return index; - }; - return new $Linear(exports, prefix, () => exports.memory.buffer, fnptr); -} - -/** - * Adopt an initialized Emscripten module (or the promise its `MODULARIZE` - * factory returns). Its exports carry a leading underscore, its memory is - * read through `HEAPU8`, and table entries are installed with - * `addFunction`, so it must be linked with `-sALLOW_TABLE_GROWTH`, - * `-sWASM_BIGINT`, and `-sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8`. - */ -export async function $loadEmscripten(module, prefix, symbols) { - const m = await module; - const x = {}; - for (const symbol of symbols) x[symbol] = m[`_${symbol}`]; - const fnptr = (params, results, sig, fn) => { - if (typeof m.addFunction !== 'function') { - throw new Error('the Emscripten module does not export addFunction (link with -sALLOW_TABLE_GROWTH and -sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8)'); - } - return m.addFunction(fn, sig); - }; - return new $Linear(x, prefix, () => m.HEAPU8.buffer, fnptr); -} - -// END linear.js diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__package_json.snap b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__package_json.snap deleted file mode 100644 index 90367f12..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__package_json.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "docs_everywhere", - "version": "0.1.0", - "description": "Native bindings for docs_everywhere", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "linear.js" - ] -} diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__runtime_js.snap deleted file mode 100644 index 75fa389b..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_docs_everywhere__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class DocsEverywhereError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends DocsEverywhereError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new DocsEverywhereError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new DocsEverywhereError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new DocsEverywhereError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new DocsEverywhereError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__README_md.snap b/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__README_md.snap deleted file mode 100644 index 7b081a38..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__README_md.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# edge_cases - -Native bindings for edge_cases - -WebAssembly bindings for a `wasm32-unknown-unknown` build of the `edge_cases` library, linked with `--export-table --growable-table` (for example `RUSTFLAGS="-C link-arg=--export-table -C link-arg=--growable-table"`). `init()` loads `edge_cases.wasm` from next to this package, or from the path in `EDGE_CASES_LIBRARY` on Node.js; pass a URL, bytes, or a compiled module to load it from elsewhere: - -```js -import { init, edge_cases } from 'edge_cases'; - -await init(); -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_d_ts.snap index 0a65aa11..ce2fd5ab 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -31,13 +31,13 @@ export declare function init( source?: string | URL | BufferSource | WebAssembly.Module | Response, ): Promise; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** No parameters and no result */ export declare namespace edge_cases { /** The errors of the `edge_cases` error domain. */ export class EdgeErrorsError extends $Error { @@ -191,6 +191,10 @@ export declare namespace edge_cases { fetch(id: bigint): Promise; /** @deprecated Use rename instead */ oldName(): string; + /** Spelled like the release method of several wrappers */ + close_(): boolean; + /** Spelled like the release method of the Dart and .NET wrappers */ + dispose(force: boolean): void; /** * Release this wrapper's reference to the native object. Safe to call * more than once; a wrapper that is never closed is released when it is @@ -212,11 +216,15 @@ export declare namespace edge_cases { */ static open(path: string): edge_cases.Vault; peek(): edge_cases.Widget | null; + /** Spelled like the handle accessor of several wrappers */ + handle(): bigint; /** * Throwing static * @throws {edge_cases.EdgeErrorsError} */ static probe(): boolean; + /** Static spelled like an inherited object member */ + static toString(): string; /** * Release this wrapper's reference to the native object. Safe to call * more than once; a wrapper that is never closed is released when it is diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_js.snap index 641e87b9..5d5b6db3 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -36,6 +39,61 @@ let $raw = $unloaded('edge_cases'); const $token = Number; let $loading = null; +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['edge_cases', [ + [0x056a86511785ba2bn, 0x5cba3f16d57c6cdbn, 'edge_cases.make_widget'], + [0x07bfa9ecb2b720d2n, 0x3813b3f57c64bf35n, 'edge_cases.EdgeErrors'], + [0x0aaa11867e37f991n, 0xf090dd52146ebdf0n, 'edge_cases.Widget.refresh'], + [0x0f9701493898ae4cn, 0x6944f47a275c96e8n, 'edge_cases.Inner'], + [0x1226b1adac179f7dn, 0x0e95a00ab0bef637n, 'edge_cases.async_keywords'], + [0x16d36bef1541136cn, 0x5ea7021ec324298fn, 'edge_cases.find_widget'], + [0x1b7324c55e980febn, 0xabf87fe372358f42n, 'edge_cases.Widget.old_name'], + [0x1cbd731467c2bab8n, 0x4915b9989c2c9074n, 'edge_cases.Vault'], + [0x2c2a6de67d894b0an, 0x50a4496b1e7191can, 'edge_cases.Vault.open'], + [0x322b4cfcd2d6861cn, 0xca7527a379ae8465n, 'edge_cases.Keywords'], + [0x3f8e23a5f32906f4n, 0x8d4c908d796f130dn, 'edge_cases.Leaf'], + [0x42ab8a685457b97cn, 0xe5ce30500cadb32dn, 'edge_cases.Vault.handle'], + [0x4a68540e2836d3d0n, 0xa20e44c71e420ed2n, 'edge_cases.mode_stream'], + [0x4e030580ac0b111dn, 0xb68b7a18d910e175n, 'edge_cases.legacy_value'], + [0x4f5e4e5291864ef3n, 0x55a1a88b21a08f8bn, 'edge_cases.Legacy'], + [0x4f84b06171b4213dn, 0xbaedcfdebb6a2820n, 'edge_cases.scalar_stream'], + [0x5067a5ff2d9ef910n, 0xbfc79b18fc4714fcn, 'edge_cases.Widget'], + [0x59aff159112fbf5bn, 0x18e4def57a8cfa49n, 'edge_cases.optional_params'], + [0x5daf65b98fa77c48n, 0xc817a8cb202e59c2n, 'edge_cases.keywords_roundtrip'], + [0x5de8a5c9726c2c5an, 0x7c50b5b65f7a896dn, 'edge_cases.register_batch'], + [0x618a620309994c3cn, 0x826ad0b1d208d723n, 'edge_cases.async_bytes'], + [0x648285cdba0a7b8en, 0xcf6e80e783a4d8ben, 'edge_cases.WidgetHolder'], + [0x649c78079824b10dn, 0x5b12dd26e0634b69n, 'edge_cases.async_widget'], + [0x7da7a7357d122a37n, 0xf544e025ffaa3d20n, 'edge_cases.async_void'], + [0x7e4015bc3a472e2dn, 0xfe22291d90b82e4an, 'edge_cases.widget_stream'], + [0x805276e5d81fdba7n, 0xd9b64e360bd03b2bn, 'edge_cases.Outer'], + [0x80e1183326542df4n, 0xe8a57e12cc6ed276n, 'edge_cases.select'], + [0x923ce5e9bcd769a1n, 0x0b973f5dc07dda8bn, 'edge_cases.Vault.to_string'], + [0x98ca8c01f5239404n, 0xfe11c93eb6b303efn, 'edge_cases.Widget.rename'], + [0xa13e22e98ea278d4n, 0xf1fae215d7387800n, 'edge_cases.widgets_everywhere'], + [0xa84a8c9bf2a881e1n, 0xbe1dc840bf221996n, 'edge_cases.async_optional'], + [0xad958c478fbf205dn, 0xe386b0716d24f19bn, 'edge_cases.Retired'], + [0xb2fc3964ef56ab39n, 0x73c325c8bc351236n, 'edge_cases.async_list'], + [0xb3403af521cf0006n, 0x6abc30d48206a3ban, 'edge_cases.Widget.close'], + [0xb794304a22a406b8n, 0xf6fb98711c516105n, 'edge_cases.async_struct'], + [0xba3936106a49c47dn, 0xb7025f2fadd6288en, 'edge_cases.Vault.peek'], + [0xc155b0c3cfb164f4n, 0xc777ae57d92ebc47n, 'edge_cases.noop'], + [0xcdc18607777019ddn, 0xb81a31e5c20aa7a8n, 'edge_cases.inspect_widget'], + [0xd164a82aeb64b8ccn, 0xded02c68005c4737n, 'edge_cases.nested_optional'], + [0xd978ccad2468038fn, 0x30c39c65d6235153n, 'edge_cases.Mode'], + [0xdbe9d947431c9124n, 0x39537fb848fdf980n, 'edge_cases.BatchListener'], + [0xeae8b7862c08bb07n, 0xfe12eeaa830452ean, 'edge_cases.Keyword'], + [0xf1e155a05ae262b8n, 0xb575eb818a423d42n, 'edge_cases.async_enum'], + [0xf22e210bdbce7343n, 0x2a5e0f801bc8c768n, 'edge_cases.Widget.dispose'], + [0xf3c7cf675563ee6cn, 0x0ecffeef399facc8n, 'edge_cases.Vault.probe'], + [0xfd34bc67acac379en, 0x576caa14f3c0901an, 'edge_cases.Widget.fetch'], + [0xff2e9d2bff941786n, 0x7e96764ef1f5fbaan, 'edge_cases.string_stream'], + ]], +]; + /** * Load the WebAssembly module (by default `edge_cases.wasm` next to this file, or * the file named by `EDGE_CASES_LIBRARY` on Node.js) and check that it matches @@ -47,7 +105,7 @@ export function init(source) { .then((m) => { $raw = $bind(m); try { - $verify($raw, 'edge_cases', 'edge_cases', 3, [['edge_cases', 0x1567f441073bd4aan]]); + $verify($raw, 'edge_cases', 'edge_cases', 4, $contract); } catch (e) { $raw = $unloaded('edge_cases'); throw e; @@ -127,11 +185,11 @@ function $bind(m) { const x = m.x; return { edge_cases_abi_version: () => x.edge_cases_abi_version() >>> 0, - edge_cases_debug_live: (kind) => BigInt.asUintN(64, x.edge_cases_debug_live(m.num(kind))), + edge_cases_debug_live: (kind) => m.live(x.edge_cases_debug_live(m.num(kind)), kind), edge_cases_cancel_token_create: () => x.edge_cases_cancel_token_create(), edge_cases_cancel_token_cancel: (t) => x.edge_cases_cancel_token_cancel(m.handle(t)), edge_cases_cancel_token_destroy: (t) => x.edge_cases_cancel_token_destroy(m.handle(t)), - edge_cases_edge_cases_checksum: () => BigInt.asUintN(64, x.edge_cases_edge_cases_checksum()), + edge_cases_edge_cases_contract: () => m.contract(x.edge_cases_edge_cases_contract), edge_cases_edge_cases_Widget_clone: (h) => x.edge_cases_edge_cases_Widget_clone(m.handle(h)), edge_cases_edge_cases_Widget_destroy: (h) => x.edge_cases_edge_cases_Widget_destroy(m.handle(h)), edge_cases_edge_cases_Vault_clone: (h) => x.edge_cases_edge_cases_Vault_clone(m.handle(h)), @@ -350,6 +408,15 @@ function $bind(m) { m.check(); return m.takeStr(r, m.outLen()); }, + edge_cases_edge_cases_Widget_close: (self) => { + const r = x.edge_cases_edge_cases_Widget_close(m.handle(self), m.err); + m.check(); + return r !== 0; + }, + edge_cases_edge_cases_Widget_dispose: (self, a0) => { + x.edge_cases_edge_cases_Widget_dispose(m.handle(self), m.bool(a0), m.err); + m.check(); + }, edge_cases_edge_cases_Vault_open: (a0) => { let s0 = null; try { @@ -366,11 +433,21 @@ function $bind(m) { m.check(); return r === 0 ? null : r; }, + edge_cases_edge_cases_Vault_handle: (self) => { + const r = x.edge_cases_edge_cases_Vault_handle(m.handle(self), m.err); + m.check(); + return r; + }, edge_cases_edge_cases_Vault_probe: () => { const r = x.edge_cases_edge_cases_Vault_probe(m.err); m.check(); return r !== 0; }, + edge_cases_edge_cases_Vault_to_string: () => { + const r = x.edge_cases_edge_cases_Vault_to_string(m.len, m.err); + m.check(); + return m.takeStr(r, m.outLen()); + }, }; } @@ -382,9 +459,10 @@ const $tokens = { }; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.edge_cases_debug_live(kind); @@ -397,9 +475,12 @@ const edge_cases$BusyError = class BusyError extends edge_cases$EdgeErrorsError super(7, message); } }; -const $codes$edge_cases$EdgeErrors = new Map([[7, edge_cases$BusyError]]); +const $codes$edge_cases$EdgeErrors = new Map([ + [7, edge_cases$BusyError], +]); +const $payloads$edge_cases$EdgeErrors = new Map(); function $from$edge_cases$EdgeErrors(e) { - return $domain(e, $codes$edge_cases$EdgeErrors); + return $domain(e, $codes$edge_cases$EdgeErrors, $payloads$edge_cases$EdgeErrors); } const edge_cases$Legacy = Object.freeze({ @@ -428,38 +509,38 @@ function $r$edge_cases$Leaf(r) { } function $w$edge_cases$Inner(w, v) { $w$edge_cases$Leaf(w, v.leaf); - w.writeList(v.leaves, $w$edge_cases$Leaf); + $w_list_Leaf(w, v.leaves); } function $r$edge_cases$Inner(r) { return { leaf: $r$edge_cases$Leaf(r), - leaves: r.readList($r$edge_cases$Leaf), + leaves: $r_list_Leaf(r), }; } function $w$edge_cases$Outer(w, v) { $w$edge_cases$Inner(w, v.inner); - w.writeOpt(v.maybe_inner, $w$edge_cases$Inner); - w.writeMap(v.by_name, $W.String, $w$edge_cases$Inner); - w.writeList(v.modes, $W.I32); - w.writeList(v.matrix, (w, v) => w.writeList(v, $W.I32)); - w.writeList(v.sparse, (w, v) => w.writeOpt(v, $W.I64)); - w.writeOpt(v.blob, $W.Bytes); - w.writeOpt(v.note, $W.String); - w.writeMap(v.by_id, (w, k) => w.writeI64(BigInt(k)), $W.String); + $w_opt_Inner(w, v.maybe_inner); + $w_map_string_Inner(w, v.by_name); + $w_list_Mode(w, v.modes); + $w_list_list_i32(w, v.matrix); + $w_list_opt_i64(w, v.sparse); + $w_opt_bytes(w, v.blob); + $w_opt_string(w, v.note); + $w_map_i64_string(w, v.by_id); w.writeBool(v._hidden); w.writeU8(v.v2); } function $r$edge_cases$Outer(r) { return { inner: $r$edge_cases$Inner(r), - maybe_inner: r.readOpt($r$edge_cases$Inner), - by_name: r.readMap($R.String, $r$edge_cases$Inner), - modes: r.readList($R.I32), - matrix: r.readList((r) => r.readList($R.I32)), - sparse: r.readList((r) => r.readOpt($R.I64)), - blob: r.readOpt($R.Bytes), - note: r.readOpt($R.String), - by_id: r.readMap($R.I64, $R.String), + maybe_inner: $r_opt_Inner(r), + by_name: $r_map_string_Inner(r), + modes: $r_list_Mode(r), + matrix: $r_list_list_i32(r), + sparse: $r_list_opt_i64(r), + blob: $r_opt_bytes(r), + note: $r_opt_string(r), + by_id: $r_map_i64_string(r), _hidden: r.readBool(), v2: r.readU8(), }; @@ -597,28 +678,195 @@ function $r$edge_cases$Retired(r) { }; } function $w$edge_cases$WidgetHolder(w, v) { - w.writeU64(BigInt($clone(v.primary, edge_cases$Widget))); - w.writeOpt(v.fallback, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))); - w.writeList(v.extras, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))); - w.writeMap(v.by_name, $W.String, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))); + $w$edge_cases$Widget(w, v.primary); + $w_opt_Widget(w, v.fallback); + $w_list_Widget(w, v.extras); + $w_map_string_Widget(w, v.by_name); } function $r$edge_cases$WidgetHolder(r) { return { - primary: $adopt(edge_cases$Widget, $token(r.readU64())), - fallback: r.readOpt((r) => $adopt(edge_cases$Widget, $token(r.readU64()))), - extras: r.readList((r) => $adopt(edge_cases$Widget, $token(r.readU64()))), - by_name: r.readMap($R.String, (r) => $adopt(edge_cases$Widget, $token(r.readU64()))), + primary: $r$edge_cases$Widget(r), + fallback: $r_opt_Widget(r), + extras: $r_list_Widget(r), + by_name: $r_map_string_Widget(r), }; } +function $w$edge_cases$Widget(w, v) { + w.writeU64(BigInt($clone(v, edge_cases$Widget))); +} +function $r$edge_cases$Widget(r) { + return $adopt(edge_cases$Widget, $token(r.readU64())); +} +/** `[Leaf]` */ +function $w_list_Leaf(w, v) { + w.writeList(v, $w$edge_cases$Leaf); +} +function $r_list_Leaf(r) { + return r.readList($r$edge_cases$Leaf); +} +/** `[Mode]` */ +function $w_list_Mode(w, v) { + w.writeList(v, $W.I32); +} +function $r_list_Mode(r) { + return r.readList($R.I32); +} +/** `[Outer]` */ +function $w_list_Outer(w, v) { + w.writeList(v, $w$edge_cases$Outer); +} +function $r_list_Outer(r) { + return r.readList($r$edge_cases$Outer); +} +/** `[Widget]` */ +function $w_list_Widget(w, v) { + w.writeList(v, $w$edge_cases$Widget); +} +function $r_list_Widget(r) { + return r.readList($r$edge_cases$Widget); +} +/** `[i32]` */ +function $w_list_i32(w, v) { + w.writeList(v, $W.I32); +} +function $r_list_i32(r) { + return r.readList($R.I32); +} +/** `[[i32]]` */ +function $w_list_list_i32(w, v) { + w.writeList(v, $w_list_i32); +} +function $r_list_list_i32(r) { + return r.readList($r_list_i32); +} +/** `[Outer?]` */ +function $w_list_opt_Outer(w, v) { + w.writeList(v, $w_opt_Outer); +} +function $r_list_opt_Outer(r) { + return r.readList($r_opt_Outer); +} +/** `[Widget?]` */ +function $w_list_opt_Widget(w, v) { + w.writeList(v, $w_opt_Widget); +} +function $r_list_opt_Widget(r) { + return r.readList($r_opt_Widget); +} +/** `[i64?]` */ +function $w_list_opt_i64(w, v) { + w.writeList(v, $w_opt_i64); +} +function $r_list_opt_i64(r) { + return r.readList($r_opt_i64); +} +/** `[string]` */ +function $w_list_string(w, v) { + w.writeList(v, $W.String); +} +function $r_list_string(r) { + return r.readList($R.String); +} +/** `{i64:string}` */ +function $w_map_i64_string(w, v) { + w.writeMap(v, $WK.I64, $W.String); +} +function $r_map_i64_string(r) { + return r.readMap($R.I64, $R.String); +} +/** `{string:Inner}` */ +function $w_map_string_Inner(w, v) { + w.writeMap(v, $WK.String, $w$edge_cases$Inner); +} +function $r_map_string_Inner(r) { + return r.readMap($R.String, $r$edge_cases$Inner); +} +/** `{string:Widget}` */ +function $w_map_string_Widget(w, v) { + w.writeMap(v, $WK.String, $w$edge_cases$Widget); +} +function $r_map_string_Widget(r) { + return r.readMap($R.String, $r$edge_cases$Widget); +} +/** `Inner?` */ +function $w_opt_Inner(w, v) { + w.writeOpt(v, $w$edge_cases$Inner); +} +function $r_opt_Inner(r) { + return r.readOpt($r$edge_cases$Inner); +} +/** `Mode?` */ +function $w_opt_Mode(w, v) { + w.writeOpt(v, $W.I32); +} +function $r_opt_Mode(r) { + return r.readOpt($R.I32); +} +/** `Outer?` */ +function $w_opt_Outer(w, v) { + w.writeOpt(v, $w$edge_cases$Outer); +} +function $r_opt_Outer(r) { + return r.readOpt($r$edge_cases$Outer); +} +/** `Widget?` */ +function $w_opt_Widget(w, v) { + w.writeOpt(v, $w$edge_cases$Widget); +} +function $r_opt_Widget(r) { + return r.readOpt($r$edge_cases$Widget); +} +/** `bytes?` */ +function $w_opt_bytes(w, v) { + w.writeOpt(v, $W.Bytes); +} +function $r_opt_bytes(r) { + return r.readOpt($R.Bytes); +} +/** `i32?` */ +function $w_opt_i32(w, v) { + w.writeOpt(v, $W.I32); +} +function $r_opt_i32(r) { + return r.readOpt($R.I32); +} +/** `i64?` */ +function $w_opt_i64(w, v) { + w.writeOpt(v, $W.I64); +} +function $r_opt_i64(r) { + return r.readOpt($R.I64); +} +/** `[Outer?]?` */ +function $w_opt_list_opt_Outer(w, v) { + w.writeOpt(v, $w_list_opt_Outer); +} +function $r_opt_list_opt_Outer(r) { + return r.readOpt($r_list_opt_Outer); +} +/** `[string]?` */ +function $w_opt_list_string(w, v) { + w.writeOpt(v, $w_list_string); +} +function $r_opt_list_string(r) { + return r.readOpt($r_list_string); +} +/** `string?` */ +function $w_opt_string(w, v) { + w.writeOpt(v, $W.String); +} +function $r_opt_string(r) { + return r.readOpt($R.String); +} function $adapt$edge_cases$BatchListener(impl) { $impl(impl, 'BatchListener'); return { on_batch($a0, $a1, $a2, $a3) { - impl.onBatch($decode($a0, (r) => r.readList($r$edge_cases$Outer)), $a1, $decode($a2, (r) => r.readOpt($r$edge_cases$Outer)), $a3); + impl.onBatch($decode($a0, $r_list_Outer), $a1, $decode($a2, $r_opt_Outer), $a3); }, on_widget($a0, $a1, $a2) { - impl.onWidget($adopt(edge_cases$Widget, $a0), $adoptOpt(edge_cases$Widget, $a1), $decode($a2, (r) => r.readList((r) => $adopt(edge_cases$Widget, $token(r.readU64()))))); + impl.onWidget($adopt(edge_cases$Widget, $a0), $adoptOpt(edge_cases$Widget, $a1), $decode($a2, $r_list_Widget)); }, on_tick() { impl.onTick(); @@ -668,7 +916,7 @@ const edge_cases$Widget = class Widget extends $Object { rename(class_, import_) { const $self = $lend(this, edge_cases$Widget); try { - return $raw.edge_cases_edge_cases_Widget_rename($self, class_, $encode(import_, (w, v) => w.writeOpt(v, $W.String))); + return $raw.edge_cases_edge_cases_Widget_rename($self, class_, $encode(import_, $w_opt_string)); } catch ($e) { throw $fault($e); } finally { @@ -695,6 +943,26 @@ const edge_cases$Widget = class Widget extends $Object { $unlend(this); } } + close_() { + const $self = $lend(this, edge_cases$Widget); + try { + return $raw.edge_cases_edge_cases_Widget_close($self); + } catch ($e) { + throw $fault($e); + } finally { + $unlend(this); + } + } + dispose(force) { + const $self = $lend(this, edge_cases$Widget); + try { + $raw.edge_cases_edge_cases_Widget_dispose($self, force); + } catch ($e) { + throw $fault($e); + } finally { + $unlend(this); + } + } }; const edge_cases$Vault = class Vault extends $Object { @@ -725,6 +993,16 @@ const edge_cases$Vault = class Vault extends $Object { $unlend(this); } } + handle() { + const $self = $lend(this, edge_cases$Vault); + try { + return $raw.edge_cases_edge_cases_Vault_handle($self); + } catch ($e) { + throw $fault($e); + } finally { + $unlend(this); + } + } static probe() { try { return $raw.edge_cases_edge_cases_Vault_probe(); @@ -732,6 +1010,13 @@ const edge_cases$Vault = class Vault extends $Object { throw $from$edge_cases$EdgeErrors($e); } } + static toString() { + try { + return $raw.edge_cases_edge_cases_Vault_to_string(); + } catch ($e) { + throw $fault($e); + } + } }; const edge_cases$noop = function noop() { @@ -744,7 +1029,7 @@ const edge_cases$noop = function noop() { const edge_cases$select = function select(class_, from, func, range, type) { try { - return $raw.edge_cases_edge_cases_select(class_, from, $encode(func, (w, v) => w.writeList(v, $W.String)), $encode(range, (w, v) => w.writeOpt(v, $W.I32)), type); + return $raw.edge_cases_edge_cases_select(class_, from, $encode(func, $w_list_string), $encode(range, $w_opt_i32), type); } catch ($e) { throw $fault($e); } @@ -752,7 +1037,7 @@ const edge_cases$select = function select(class_, from, func, range, type) { const edge_cases$nestedOptional = function nestedOptional(items) { try { - return $decode($raw.edge_cases_edge_cases_nested_optional($encode(items, (w, v) => w.writeOpt(v, (w, v) => w.writeList(v, (w, v) => w.writeOpt(v, $w$edge_cases$Outer))))), (r) => r.readOpt((r) => r.readList((r) => r.readOpt($r$edge_cases$Outer)))); + return $decode($raw.edge_cases_edge_cases_nested_optional($encode(items, $w_opt_list_opt_Outer)), $r_opt_list_opt_Outer); } catch ($e) { throw $fault($e); } @@ -760,7 +1045,7 @@ const edge_cases$nestedOptional = function nestedOptional(items) { const edge_cases$optionalParams = function optionalParams(text, count, outer, tags, blob, mode) { try { - return $decode($raw.edge_cases_edge_cases_optional_params($encode(text, (w, v) => w.writeOpt(v, $W.String)), $encode(count, (w, v) => w.writeOpt(v, $W.I32)), $encode(outer, (w, v) => w.writeOpt(v, $w$edge_cases$Outer)), $encode(tags, (w, v) => w.writeOpt(v, (w, v) => w.writeList(v, $W.String))), $encode(blob, (w, v) => w.writeOpt(v, $W.Bytes)), $encode(mode, (w, v) => w.writeOpt(v, $W.I32))), (r) => r.readOpt($R.I32)); + return $decode($raw.edge_cases_edge_cases_optional_params($encode(text, $w_opt_string), $encode(count, $w_opt_i32), $encode(outer, $w_opt_Outer), $encode(tags, $w_opt_list_string), $encode(blob, $w_opt_bytes), $encode(mode, $w_opt_Mode)), $r_opt_Mode); } catch ($e) { throw $fault($e); } @@ -776,7 +1061,7 @@ const edge_cases$registerBatch = function registerBatch(listener, keyword) { const edge_cases$widgetsEverywhere = function widgetsEverywhere(list, byName, sparse, holder) { try { - return $decode($raw.edge_cases_edge_cases_widgets_everywhere($encode(list, (w, v) => w.writeList(v, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget))))), $encode(byName, (w, v) => w.writeMap(v, $W.String, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget))))), $encode(sparse, (w, v) => w.writeList(v, (w, v) => w.writeOpt(v, (w, v) => w.writeU64(BigInt($clone(v, edge_cases$Widget)))))), $encode(holder, $w$edge_cases$WidgetHolder)), $r$edge_cases$WidgetHolder); + return $decode($raw.edge_cases_edge_cases_widgets_everywhere($encode(list, $w_list_Widget), $encode(byName, $w_map_string_Widget), $encode(sparse, $w_list_opt_Widget), $encode(holder, $w$edge_cases$WidgetHolder)), $r$edge_cases$WidgetHolder); } catch ($e) { throw $fault($e); } @@ -904,7 +1189,7 @@ const edge_cases$asyncStruct = async function asyncStruct() { const edge_cases$asyncList = async function asyncList($options) { try { - return $decode(await $cancellable($options?.signal, $tokens, ($t) => $raw.edge_cases_edge_cases_async_list($t)), (r) => r.readList($r$edge_cases$Outer)); + return $decode(await $cancellable($options?.signal, $tokens, ($t) => $raw.edge_cases_edge_cases_async_list($t)), $r_list_Outer); } catch ($e) { throw $from$edge_cases$EdgeErrors($e); } @@ -912,7 +1197,7 @@ const edge_cases$asyncList = async function asyncList($options) { const edge_cases$asyncOptional = async function asyncOptional() { try { - return $decode(await $raw.edge_cases_edge_cases_async_optional(), (r) => r.readOpt($R.I64)); + return $decode(await $raw.edge_cases_edge_cases_async_optional(), $r_opt_i64); } catch ($e) { throw $fault($e); } diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__package_json.snap b/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__package_json.snap deleted file mode 100644 index 903ddfb1..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__package_json.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "edge_cases", - "version": "0.1.0", - "description": "Native bindings for edge_cases", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "linear.js" - ] -} diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__runtime_js.snap deleted file mode 100644 index d7fbec01..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_edge_cases__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class EdgeCasesError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends EdgeCasesError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new EdgeCasesError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new EdgeCasesError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new EdgeCasesError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new EdgeCasesError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__README_md.snap b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__README_md.snap index 72a2d3b8..84377620 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__README_md.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__README_md.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- - + - + # kitchen_sink diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_d_ts.snap index be649b18..f2659778 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -31,13 +31,13 @@ export declare function init( source?: string | URL | BufferSource | WebAssembly.Module | Response, ): Promise; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** Trivial cross-module helper */ export declare namespace shared { /** A reusable token referenced across modules */ export interface Token { @@ -48,7 +48,6 @@ export declare namespace shared { export function ping(): string; } -/** Identity for bool */ export declare namespace kitchen { /** The errors of the `kitchen` error domain. */ export class KitchenErrorsError extends $Error { @@ -102,6 +101,15 @@ export declare namespace kitchen { onReady(code: number, msg: string): void; /** Receives the item itself and says whether to keep listening */ onItem(item: kitchen.Item, gadget: kitchen.Gadget): boolean; + /** + * A display label, failing with a kitchen error + * @throws {kitchen.KitchenErrorsError} to report that error, with its fields, to the library + */ + label(): string; + /** The most recent item the listener kept, if any */ + latest(): kitchen.Item | null; + /** The listener's favorite gadget */ + favorite(): kitchen.Gadget; } /** A pocket-sized interface exercising the object surface */ export class Gadget { @@ -135,6 +143,8 @@ export declare namespace kitchen { export function echoBytes(b: Uint8Array): Uint8Array; /** Register a listener, returning how many are attached */ export function subscribe(listener: kitchen.ReadyListener): number; + /** Register a listener if one is given */ + export function maybeSubscribe(listener: kitchen.ReadyListener | null): boolean; /** Optional object in both directions */ export function maybeGadget(fallback: kitchen.Gadget | null): kitchen.Gadget | null; /** A list of objects */ @@ -165,7 +175,6 @@ export declare namespace kitchen { export function legacyOp(): number; /** Replacement for legacy_op */ export function newOp(): number; - /** Trivial nested-module function */ export namespace nested { /** Trivial nested-module function */ export function hello(): string; diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_js.snap index 0a1347ca..2623bd36 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -36,6 +39,49 @@ let $raw = $unloaded('kitchen_sink'); const $token = Number; let $loading = null; +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['shared', [ + [0x709e234caa962446n, 0x3894f2ca452e9f98n, 'shared.ping'], + [0xcfb306fa5801ee5bn, 0x1cbd9e10983cac2cn, 'shared.Token'], + ]], + ['kitchen', [ + [0x01d4f09bbddc986fn, 0x29eb88fecc66544dn, 'kitchen.Gadget.new'], + [0x10472d8fe748c2een, 0xb265832b6a759fc4n, 'kitchen.maybe_gadget'], + [0x11368eec07a74078n, 0xfa049f78b77e146an, 'kitchen.gadgets'], + [0x137f08a29cde6021n, 0xda8de7aeb2dbb8a3n, 'kitchen.stream_gadgets'], + [0x2111b54980826dd7n, 0x452783f3e6ca6e1dn, 'kitchen.do_cancellable'], + [0x29dddbc60bf8d7dbn, 0x452a3e573d57f6dbn, 'kitchen.Priority'], + [0x2b227003b74a0d16n, 0x12332957470e9782n, 'kitchen.maybe_subscribe'], + [0x2dd66cb0064f0494n, 0x60197363fefd92ddn, 'kitchen.Item'], + [0x33c265bbf079b0e9n, 0xc3dcd0927e6c1debn, 'kitchen.u32_id'], + [0x33e6381d01660dfen, 0x077c63c31fd7f90an, 'kitchen.KitchenErrors'], + [0x3f5fc99d73504b76n, 0x2df88d710f1ea10en, 'kitchen.Gadget.poke'], + [0x4c5764cf109528d1n, 0x2a65529af4338a58n, 'kitchen.maybe_item'], + [0x4de17dfb93fd494dn, 0xba0ad0dd52ed74afn, 'kitchen.i32_id'], + [0x504e95b7f7e65570n, 0x01b3e8e0c9a10e23n, 'kitchen.stream_items'], + [0x68cdb9a4bd3e3dd5n, 0xc678b9164348d497n, 'kitchen.bool_id'], + [0x69e7cd703d16eb85n, 0xf3b28dd22ae9e689n, 'kitchen.do_async'], + [0x70865079ce41be82n, 0xbe3b69464672af46n, 'kitchen.Gadget.describe'], + [0x71c5a1116cb7c49an, 0xcba1837108a89b84n, 'kitchen.echo_bytes'], + [0x735ee1e0049ee91an, 0xca74320bc53eddabn, 'kitchen.echo_string'], + [0x76d7fc58f4e41d84n, 0x7373a0e47d30e690n, 'kitchen.i64_id'], + [0x79abfe1aa960fd6an, 0xbca7a9e5eb26f3a6n, 'kitchen.nested.hello'], + [0x7d3dd7d5cf6749e3n, 0xa192f5a7298adb39n, 'kitchen.new_op'], + [0x89a5e664bc25c67bn, 0x5764a95e81ecfd31n, 'kitchen.f64_id'], + [0x8cfdbdcecac5f7f1n, 0x07e272f09c41fffcn, 'kitchen.Gadget.version'], + [0x974b9f3e823c1738n, 0xb81b91e975695f27n, 'kitchen.cross_module_token'], + [0xaca0655fac15d230n, 0x11c4d2b28abe07bbn, 'kitchen.list_items'], + [0xc883b4afae20543en, 0xc40560e5411ff392n, 'kitchen.legacy_op'], + [0xf2f7753e6e1dffc2n, 0xc9047a95bdf49e93n, 'kitchen.get_attrs'], + [0xfae25d3391922eadn, 0x1c7f95739f3210b8n, 'kitchen.subscribe'], + [0xfd0059b05cb2fcfbn, 0x248bf8404a5823b4n, 'kitchen.Gadget'], + [0xfed9c551fb9fd2d6n, 0x59aec980ef3f3291n, 'kitchen.ReadyListener'], + ]], +]; + /** * Load the WebAssembly module (by default `kitchen_sink.wasm` next to this file, or * the file named by `KITCHEN_SINK_LIBRARY` on Node.js) and check that it matches @@ -47,7 +93,7 @@ export function init(source) { .then((m) => { $raw = $bind(m); try { - $verify($raw, 'kitchen_sink', 'kitchen_sink', 3, [['shared', 0x42c4ce2c8d0af052n], ['kitchen', 0xd83da75b24b545ffn]]); + $verify($raw, 'kitchen_sink', 'kitchen_sink', 4, $contract); } catch (e) { $raw = $unloaded('kitchen_sink'); throw e; @@ -77,6 +123,28 @@ function $vt_kitchen_sink_kitchen_ReadyListener(m) { return 0; } }], + [['i32', 'i32', 'i32', 'i32'], [], 'viiii', (ctx, out, outLen, err) => { + try { + m.giveStr(out, outLen, m.adapter(ctx).label()); + } catch (e) { + m.foreign(err, e); + } + }], + [['i32', 'i32', 'i32', 'i32'], [], 'viiii', (ctx, out, outLen, err) => { + try { + m.give(out, outLen, m.adapter(ctx).latest()); + } catch (e) { + m.foreign(err, e); + } + }], + [['i32', 'i32'], ['i32'], 'iii', (ctx, err) => { + try { + return m.handleOpt(m.adapter(ctx).favorite()); + } catch (e) { + m.foreign(err, e); + return 0; + } + }], ]; } @@ -85,12 +153,12 @@ function $bind(m) { const x = m.x; return { kitchen_sink_abi_version: () => x.kitchen_sink_abi_version() >>> 0, - kitchen_sink_debug_live: (kind) => BigInt.asUintN(64, x.kitchen_sink_debug_live(m.num(kind))), + kitchen_sink_debug_live: (kind) => m.live(x.kitchen_sink_debug_live(m.num(kind)), kind), kitchen_sink_cancel_token_create: () => x.kitchen_sink_cancel_token_create(), kitchen_sink_cancel_token_cancel: (t) => x.kitchen_sink_cancel_token_cancel(m.handle(t)), kitchen_sink_cancel_token_destroy: (t) => x.kitchen_sink_cancel_token_destroy(m.handle(t)), - kitchen_sink_shared_checksum: () => BigInt.asUintN(64, x.kitchen_sink_shared_checksum()), - kitchen_sink_kitchen_checksum: () => BigInt.asUintN(64, x.kitchen_sink_kitchen_checksum()), + kitchen_sink_shared_contract: () => m.contract(x.kitchen_sink_shared_contract), + kitchen_sink_kitchen_contract: () => m.contract(x.kitchen_sink_kitchen_contract), kitchen_sink_shared_ping: () => { const r = x.kitchen_sink_shared_ping(m.len, m.err); m.check(); @@ -150,6 +218,11 @@ function $bind(m) { m.check(); return r; }, + kitchen_sink_kitchen_maybe_subscribe: (a0) => { + const r = x.kitchen_sink_kitchen_maybe_subscribe(m.register(a0), m.vtableOpt(a0, 'kitchen_sink_kitchen_ReadyListener', () => $vt_kitchen_sink_kitchen_ReadyListener(m)), m.err); + m.check(); + return r !== 0; + }, kitchen_sink_kitchen_maybe_gadget: (a0) => { const r = x.kitchen_sink_kitchen_maybe_gadget(m.handleOpt(a0), m.err); m.check(); @@ -270,9 +343,10 @@ const $tokens = { }; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.kitchen_sink_debug_live(kind); @@ -291,9 +365,17 @@ const kitchen$InvalidInputError = class InvalidInputError extends kitchen$Kitche super(2, message); } }; -const $codes$kitchen$KitchenErrors = new Map([[1, kitchen$NotFoundError], [2, kitchen$InvalidInputError]]); +const $codes$kitchen$KitchenErrors = new Map([ + [1, kitchen$NotFoundError], + [2, kitchen$InvalidInputError], +]); +const $payloads$kitchen$KitchenErrors = new Map(); function $from$kitchen$KitchenErrors(e) { - return $domain(e, $codes$kitchen$KitchenErrors); + return $domain(e, $codes$kitchen$KitchenErrors, $payloads$kitchen$KitchenErrors); +} +const $fields$kitchen$KitchenErrors = new Map(); +function $raise$kitchen$KitchenErrors(e) { + return $raise(e, kitchen$KitchenErrorsError, $codes$kitchen$KitchenErrors, $fields$kitchen$KitchenErrors); } const kitchen$Priority = Object.freeze({ @@ -323,12 +405,12 @@ function $w$kitchen$Item(w, v) { w.writeF64(v.ratio); w.writeU32(v.nick); w.writeBytes(v.payload); - w.writeList(v.tags, $W.String); - w.writeMap(v.attrs, $W.String, $W.String); - w.writeOpt(v.parent, $W.I64); + $w_list_string(w, v.tags); + $w_map_string_string(w, v.attrs); + $w_opt_i64(w, v.parent); $w$shared$Token(w, v.token); w.writeI32(v.priority); - w.writeOpt(v.gadget, (w, v) => w.writeU64(BigInt($clone(v, kitchen$Gadget)))); + $w_opt_Gadget(w, v.gadget); } function $r$kitchen$Item(r) { return { @@ -339,14 +421,76 @@ function $r$kitchen$Item(r) { ratio: r.readF64(), nick: r.readU32(), payload: r.readBytes(), - tags: r.readList($R.String), - attrs: r.readMap($R.String, $R.String), - parent: r.readOpt($R.I64), + tags: $r_list_string(r), + attrs: $r_map_string_string(r), + parent: $r_opt_i64(r), token: $r$shared$Token(r), priority: r.readI32(), - gadget: r.readOpt((r) => $adopt(kitchen$Gadget, $token(r.readU64()))), + gadget: $r_opt_Gadget(r), }; } +function $w$kitchen$Gadget(w, v) { + w.writeU64(BigInt($clone(v, kitchen$Gadget))); +} +function $r$kitchen$Gadget(r) { + return $adopt(kitchen$Gadget, $token(r.readU64())); +} +/** `[Gadget]` */ +function $w_list_Gadget(w, v) { + w.writeList(v, $w$kitchen$Gadget); +} +function $r_list_Gadget(r) { + return r.readList($r$kitchen$Gadget); +} +/** `[Item]` */ +function $w_list_Item(w, v) { + w.writeList(v, $w$kitchen$Item); +} +function $r_list_Item(r) { + return r.readList($r$kitchen$Item); +} +/** `[string]` */ +function $w_list_string(w, v) { + w.writeList(v, $W.String); +} +function $r_list_string(r) { + return r.readList($R.String); +} +/** `{string:i32}` */ +function $w_map_string_i32(w, v) { + w.writeMap(v, $WK.String, $W.I32); +} +function $r_map_string_i32(r) { + return r.readMap($R.String, $R.I32); +} +/** `{string:string}` */ +function $w_map_string_string(w, v) { + w.writeMap(v, $WK.String, $W.String); +} +function $r_map_string_string(r) { + return r.readMap($R.String, $R.String); +} +/** `Gadget?` */ +function $w_opt_Gadget(w, v) { + w.writeOpt(v, $w$kitchen$Gadget); +} +function $r_opt_Gadget(r) { + return r.readOpt($r$kitchen$Gadget); +} +/** `Item?` */ +function $w_opt_Item(w, v) { + w.writeOpt(v, $w$kitchen$Item); +} +function $r_opt_Item(r) { + return r.readOpt($r$kitchen$Item); +} +/** `i64?` */ +function $w_opt_i64(w, v) { + w.writeOpt(v, $W.I64); +} +function $r_opt_i64(r) { + return r.readOpt($R.I64); +} const shared$ping = function ping() { try { @@ -365,6 +509,19 @@ function $adapt$kitchen$ReadyListener(impl) { on_item($a0, $a1) { return $ret.Bool(impl.onItem($decode($a0, $r$kitchen$Item), $adopt(kitchen$Gadget, $a1)), 'ReadyListener.onItem'); }, + label() { + try { + return $ret.String(impl.label(), 'ReadyListener.label'); + } catch ($e) { + throw $raise$kitchen$KitchenErrors($e); + } + }, + latest() { + return $encode(impl.latest(), $w_opt_Item); + }, + favorite() { + return $clone(impl.favorite(), kitchen$Gadget); + }, }; } @@ -476,6 +633,14 @@ const kitchen$subscribe = function subscribe(listener) { } }; +const kitchen$maybeSubscribe = function maybeSubscribe(listener) { + try { + return $raw.kitchen_sink_kitchen_maybe_subscribe(listener == null ? null : $adapt$kitchen$ReadyListener(listener)); + } catch ($e) { + throw $fault($e); + } +}; + const kitchen$maybeGadget = function maybeGadget(fallback) { const $o0 = $lendOpt(fallback, kitchen$Gadget); try { @@ -489,7 +654,7 @@ const kitchen$maybeGadget = function maybeGadget(fallback) { const kitchen$gadgets = function gadgets() { try { - return $decode($raw.kitchen_sink_kitchen_gadgets(), (r) => r.readList((r) => $adopt(kitchen$Gadget, $token(r.readU64())))); + return $decode($raw.kitchen_sink_kitchen_gadgets(), $r_list_Gadget); } catch ($e) { throw $fault($e); } @@ -511,7 +676,7 @@ const kitchen$streamGadgets = function streamGadgets() { const kitchen$maybeItem = function maybeItem(id) { try { - return $decode($raw.kitchen_sink_kitchen_maybe_item(id), (r) => r.readOpt($r$kitchen$Item)); + return $decode($raw.kitchen_sink_kitchen_maybe_item(id), $r_opt_Item); } catch ($e) { throw $from$kitchen$KitchenErrors($e); } @@ -519,7 +684,7 @@ const kitchen$maybeItem = function maybeItem(id) { const kitchen$listItems = function listItems() { try { - return $decode($raw.kitchen_sink_kitchen_list_items(), (r) => r.readList($r$kitchen$Item)); + return $decode($raw.kitchen_sink_kitchen_list_items(), $r_list_Item); } catch ($e) { throw $fault($e); } @@ -527,7 +692,7 @@ const kitchen$listItems = function listItems() { const kitchen$getAttrs = function getAttrs() { try { - return $decode($raw.kitchen_sink_kitchen_get_attrs(), (r) => r.readMap($R.String, $R.I32)); + return $decode($raw.kitchen_sink_kitchen_get_attrs(), $r_map_string_i32); } catch ($e) { throw $fault($e); } @@ -613,6 +778,7 @@ export const kitchen = Object.freeze({ echoString: kitchen$echoString, echoBytes: kitchen$echoBytes, subscribe: kitchen$subscribe, + maybeSubscribe: kitchen$maybeSubscribe, maybeGadget: kitchen$maybeGadget, gadgets: kitchen$gadgets, streamGadgets: kitchen$streamGadgets, diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__linear_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__linear_js.snap index 608de11a..3359c8df 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__linear_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__linear_js.snap @@ -1,15 +1,21 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // The WebAssembly transport: loading the module, staging arguments in its // linear memory, reading results and errors back, and installing JavaScript // functions in its function table (async completions and callback // interface vtables). The generated `index.js` builds one native entry point // per C symbol on top of this. +// +// Byte runs from `{prefix}_alloc` have alignment 1, so every multi-byte +// read and write goes through a `DataView`, and the slots the producer +// writes through typed pointers (the error struct, `out_len`, iterator +// items, vtables) live in 8-byte-aligned blocks carved from the transport's +// own arenas. import { $Fault } from './runtime.js'; @@ -33,6 +39,9 @@ function $bigint(v, bits, what) { return b; } +/** The size of each arena the transport carves its permanent blocks from. */ +const $ARENA = 4096; + /** * One loaded module's linear memory and runtime symbols. `x` holds the * module's exports by C symbol name; `memory()` returns the current @@ -45,20 +54,25 @@ export class $Linear { this.memory = memory; this.fnptr = fnptr; this.allocFn = x[`${prefix}_alloc`]; - this.deallocFn = x[`${prefix}_dealloc`]; this.freeBytesFn = x[`${prefix}_free_bytes`]; this.errorClearFn = x[`${prefix}_error_clear`]; this.errorFreeFn = x[`${prefix}_error_free`]; this.errorSetFn = x[`${prefix}_error_set`]; - if (typeof this.allocFn !== 'function' || typeof this.deallocFn !== 'function') { - throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_dealloc`); + this.errorSetPayloadFn = x[`${prefix}_error_set_payload`]; + if (typeof this.allocFn !== 'function' || typeof this.freeBytesFn !== 'function') { + throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_free_bytes`); } + // Permanent blocks (the scratch slots and the vtables) come from + // arenas allocated with {prefix}_alloc and never freed; `arenas` counts + // them so the leak counter of byte runs can leave them out. + this.arenas = 0; + this.arenaNext = 0; + this.arenaEnd = 0; // One scratch block reused by every call: the error slot (16 bytes), // the out_len slot (4), and the iterator out_item slot (8). The producer // only writes them as a call returns, so a call made from a callback in // the middle of another call can reuse them safely. - const scratch = this.allocFn(32); - this.bytes().fill(0, scratch, scratch + 32); + const scratch = this.reserve(32); this.err = scratch; this.len = scratch + 16; this.item = scratch + 24; @@ -85,6 +99,38 @@ export class $Linear { return this.dv; } + /** A run of `size` bytes from {prefix}_alloc (`0` for an empty one). */ + alloc(size) { + if (size === 0) return 0; + const ptr = this.allocFn(size) >>> 0; + if (ptr === 0) throw new RangeError(`the WebAssembly module could not allocate ${size} bytes`); + return ptr; + } + + /** A permanent, zero-filled, 8-byte-aligned block of `size` bytes. */ + reserve(size) { + const need = Math.ceil(size / 8) * 8; + if (this.arenaNext + need > this.arenaEnd) { + const length = Math.max($ARENA, need + 8); + const base = this.alloc(length); + this.arenas++; + this.arenaNext = Math.ceil(base / 8) * 8; + this.arenaEnd = base + length; + } + const at = this.arenaNext; + this.arenaNext += need; + return at; + } + + /** + * A `{prefix}_debug_live(kind)` result as a bigint, leaving out the + * transport's own arenas from the count of byte runs (kind 4). + */ + live(raw, kind) { + const n = BigInt.asUintN(64, raw); + return kind === 4 && n > 0n ? n - BigInt(this.arenas) : n; + } + // Argument checks and conversions for by-value slots. num(v) { if (typeof v !== 'number') throw $expected('a number', v); @@ -113,13 +159,13 @@ export class $Linear { return v === null || v === undefined ? 0 : this.handle(v); } - // Staging: copy a string or bytes into linear memory for one call. A - // staged value is [ptr, len, size]; release it with unstage(). + // Staging: copy a string or bytes into a run for one call. A staged value + // is [ptr, len, size]; release it with unstage(). str(s) { if (typeof s !== 'string') throw $expected('a string', s); if (s.length === 0) return [0, 0, 0]; const size = s.length * 3; - const ptr = this.allocFn(size); + const ptr = this.alloc(size); const { written } = $enc.encodeInto(s, this.bytes().subarray(ptr, ptr + size)); return [ptr, written, size]; } @@ -127,13 +173,13 @@ export class $Linear { data(b) { if (!(b instanceof Uint8Array)) throw $expected('a Uint8Array', b); if (b.length === 0) return [0, 0, 0]; - const ptr = this.allocFn(b.length); + const ptr = this.alloc(b.length); this.bytes().set(b, ptr); return [ptr, b.length, b.length]; } unstage(staged) { - if (staged !== null && staged[2] !== 0) this.deallocFn(staged[0], staged[2]); + if (staged !== null && staged[2] !== 0) this.freeBytesFn(staged[0], staged[2]); } // Results. @@ -142,10 +188,12 @@ export class $Linear { } readStr(ptr, len) { + ptr >>>= 0; return len === 0 ? '' : $dec.decode(this.bytes().subarray(ptr, ptr + len)); } readData(ptr, len) { + ptr >>>= 0; return len === 0 ? new Uint8Array(0) : this.bytes().slice(ptr, ptr + len); } @@ -162,6 +210,7 @@ export class $Linear { } cstr(ptr) { + ptr >>>= 0; if (ptr === 0) return ''; const u8 = this.bytes(); let end = ptr; @@ -178,7 +227,7 @@ export class $Linear { return new $Fault( code, this.cstr(dv.getUint32(errPtr + 4, true)), - payloadPtr === 0 ? null : this.bytes().slice(payloadPtr, payloadPtr + payloadLen), + payloadPtr === 0 ? null : this.readData(payloadPtr, payloadLen), ); } @@ -190,9 +239,21 @@ export class $Linear { throw fault; } + /** A module's contract table as a `BigUint64Array` of `id, hash` pairs. */ + contract(table) { + const ptr = table(this.len) >>> 0; + const n = this.outLen() * 2; + const dv = this.view(); + const out = new BigUint64Array(n); + for (let i = 0; i < n; i++) out[i] = dv.getBigUint64(ptr + i * 8, true); + return out; + } + // Callback interfaces: implementations live in a table keyed by the // integer passed as `ctx`, until the producer calls the vtable's `free`. + // An absent optional implementation is ctx 0 with a null vtable. register(adapter) { + if (adapter === null) return 0; const id = this.nextCallback++; this.callbacks.set(id, adapter); return id; @@ -208,36 +269,91 @@ export class $Linear { this.callbacks.delete(id); } - /** Report a callback implementation's exception through `errPtr` (code -4). */ + /** Stage `text` as a NUL-terminated C string (release it with unstage()). */ + cstrArg(text) { + const bytes = $enc.encode(text); + const ptr = this.alloc(bytes.length + 1); + this.bytes().set(bytes, ptr); + return [ptr, bytes.length, bytes.length + 1]; + } + + /** + * Report a callback implementation's exception through `errPtr`: a + * `$Fault` (a domain error a throwing method raised) with its code, + * message, and payload, anything else as code -4 with its message. + */ foreign(errPtr, e) { - const message = e instanceof Error ? e.message || e.name : String(e); - const bytes = $enc.encode(message); - const ptr = this.allocFn(bytes.length + 1); - const u8 = this.bytes(); - u8.set(bytes, ptr); - u8[ptr + bytes.length] = 0; - this.errorSetFn(errPtr, -4, ptr); - this.deallocFn(ptr, bytes.length + 1); + let code = -4; + let message; + let payload = null; + if (e instanceof $Fault) { + code = e.code; + message = e.message; + payload = e.payload; + } else { + message = e instanceof Error ? e.message || e.name : String(e); + } + const text = this.cstrArg(message); + try { + this.errorSetFn(errPtr, code, text[0]); + } finally { + this.unstage(text); + } + if (payload !== null && payload.length > 0) { + const staged = this.data(payload); + try { + this.errorSetPayloadFn(errPtr, staged[0], staged[1]); + } finally { + this.unstage(staged); + } + } } /** - * The static vtable of one callback interface: one table entry per - * method (`methods()` returns each as `[params, results, sig, fn]`), then - * `free`. Built on first use. + * Hand `bytes` (a `Uint8Array`) to the producer through a callback's + * `out_ptr`/`out_len` slots, as a run from {prefix}_alloc it adopts. + */ + give(outPtr, outLen, bytes) { + if (!(bytes instanceof Uint8Array)) throw $expected('a Uint8Array', bytes); + const run = this.alloc(bytes.length); + if (bytes.length > 0) this.bytes().set(bytes, run); + const dv = this.view(); + dv.setUint32(outPtr >>> 0, run, true); + dv.setUint32(outLen >>> 0, bytes.length, true); + } + + /** `give` for a string, as UTF-8. */ + giveStr(outPtr, outLen, s) { + if (typeof s !== 'string') throw $expected('a string', s); + this.give(outPtr, outLen, $enc.encode(s)); + } + + /** + * The static vtable of one callback interface, built on first use: the + * header (`size`, `flags`, then the `free` entry), then one table entry + * per method (`methods()` returns each as `[params, results, sig, fn]`). */ vtable(name, methods) { let ptr = this.vtables.get(name); if (ptr !== undefined) return ptr; - const entries = [...methods(), [['i32'], [], 'vi', (ctx) => this.unregister(ctx)]]; - ptr = this.allocFn(entries.length * 4); + const entries = [[['i32'], [], 'vi', (ctx) => this.unregister(ctx)], ...methods()]; + const size = 8 + entries.length * 4; + ptr = this.reserve(size); const dv = this.view(); + dv.setUint32(ptr, size, true); + dv.setUint32(ptr + 4, 0, true); entries.forEach(([params, results, sig, fn], i) => { - dv.setUint32(ptr + i * 4, this.fnptr(params, results, sig, fn), true); + dv.setUint32(ptr + 8 + i * 4, this.fnptr(params, results, sig, fn), true); }); this.vtables.set(name, ptr); return ptr; } + /** `vtable`, or a null vtable for an absent optional implementation. */ + vtableOpt(adapter, name, methods) { + return adapter === null ? 0 : this.vtable(name, methods); + } + /** * Launch an async call. `call(callback, context)` invokes the launcher; * the completion (one table entry per signature, shared by every call) @@ -269,7 +385,7 @@ export class $Linear { if (pending === undefined) return; this.pending.delete(id); try { - const fault = errPtr === 0 ? null : this.fault(errPtr); + const fault = errPtr === 0 ? null : this.fault(errPtr >>> 0); if (errPtr !== 0) this.errorFreeFn(errPtr); if (fault !== null) { pending.reject(fault); diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__package_json.snap b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__package_json.snap index f2e1b29d..e56dd31c 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__package_json.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__package_json.snap @@ -2,9 +2,9 @@ source: crates/weaveffi-cli/tests/snapshots.rs --- { - "//": "Generated by WeaveFFI [VERSION] from api.yml", + "//": "Generated by WeaveFFI [VERSION] "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", + "//regenerate": "To regenerate: weaveffi generate", "name": "kitchen_sink", "version": "0.1.0", "description": "Native bindings for kitchen_sink", diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__runtime_js.snap index e11da31e..02cd16c0 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__runtime_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_kitchen_sink__runtime_js.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate // The runtime shared by every binding in this package: the error classes, // the value-buffer codec, the object and iterator wrappers, cancellation, and @@ -66,25 +66,38 @@ export function $fault(e) { /** * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. + * class, and `payloads` maps each code that carries fields to the reader + * that decodes them (the class's constructor then takes the fields first). + * Codes outside the domain fall back to `$fault`. */ export function $domain(e, codes, payloads) { if (e instanceof $Fault) { const cls = codes.get(e.code); if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; + const message = e.message || undefined; + const read = payloads.get(e.code); + if (read === undefined) return new cls(message); + return new cls(e.payload === null ? {} : $decode(e.payload, read), message); } } return $fault(e); } +/** + * The fault a callback implementation reports for exception `e` from a + * method that throws the error domain `domain`: a declared code of the + * domain travels with its fields (`fields` maps each code that has any to + * the writer that encodes them); anything else is rethrown unchanged and + * reported as a code -4 failure. + */ +export function $raise(e, domain, codes, fields) { + if (e instanceof domain && codes.has(e.code)) { + const write = fields.get(e.code); + return new $Fault(e.code, e.message, write === undefined ? null : $encode(e, write)); + } + return e; +} + function $malformed(what) { return new KitchenSinkError(-3, 'malformed value buffer: ' + what); } @@ -313,6 +326,27 @@ export const $W = Object.freeze({ Bytes: (w, v) => w.writeBytes(v), }); +/** + * Map key writers. Plain-object keys are always strings, so numeric, + * `bigint`, and `bool` keys convert first (a `Map`'s typed keys pass + * through the same conversions unchanged). C-style enum keys use `I32`. + */ +export const $WK = Object.freeze({ + Bool: (w, k) => w.writeBool(k === true || k === 'true'), + I8: (w, k) => w.writeI8(Number(k)), + I16: (w, k) => w.writeI16(Number(k)), + I32: (w, k) => w.writeI32(Number(k)), + I64: (w, k) => w.writeI64(BigInt(k)), + U8: (w, k) => w.writeU8(Number(k)), + U16: (w, k) => w.writeU16(Number(k)), + U32: (w, k) => w.writeU32(Number(k)), + U64: (w, k) => w.writeU64(BigInt(k)), + F32: (w, k) => w.writeF32(Number(k)), + F64: (w, k) => w.writeF64(Number(k)), + String: (w, k) => w.writeString(k), + Bytes: (w, k) => w.writeBytes(k), +}); + /** Element readers for the fixed-width primitives, strings, and bytes. */ export const $R = Object.freeze({ Bool: (r) => r.readBool(), @@ -418,6 +452,11 @@ export function $clone(o, cls) { } } +/** `$clone`, with `null` for an absent object. */ +export function $cloneOpt(o, cls) { + return o === null || o === undefined ? null : $clone(o, cls); +} + const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); /** @@ -526,8 +565,9 @@ function $retBigint(v, where) { } /** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. + * Checks for a callback method's return value, by wire primitive (plus + * `String` and `Bytes`). A value of the wrong type throws, which reports + * the call as a code -4 failure. */ export const $ret = Object.freeze({ Bool(v, where) { @@ -544,14 +584,25 @@ export const $ret = Object.freeze({ F64: $num, I64: $retBigint, U64: $retBigint, + String(v, where) { + if (typeof v !== 'string') throw new TypeError(`${where} must return a string`); + return v; + }, + Bytes(v, where) { + if (!(v instanceof Uint8Array)) throw new TypeError(`${where} must return a Uint8Array`); + return v; + }, }); /** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. + * The load-time checks: the native library must implement ABI revision + * `abi`, and, for every top-level module, its contract table must hold each + * entry these bindings were generated with. `contract` lists + * `[module, [[id, hash, path], ...]]` per module; the native side returns + * a module's table as a `BigUint64Array` of `id, hash` pairs. Entries the + * library has and the bindings don't are fine. */ -export function $verify(raw, library, prefix, abi, checksums) { +export function $verify(raw, library, prefix, abi, contract) { const version = raw[`${prefix}_abi_version`]; if (typeof version !== 'function') { throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); @@ -562,15 +613,22 @@ export function $verify(raw, library, prefix, abi, checksums) { `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, ); } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); + for (const [module, expected] of contract) { + const table = raw[`${prefix}_${module}_contract`]; + if (typeof table !== 'function') { + throw new Error(`${library}: the native library does not export ${prefix}_${module}_contract`); + } + const pairs = table(); + const hashes = new Map(); + for (let i = 0; i + 1 < pairs.length; i += 2) hashes.set(pairs[i], pairs[i + 1]); + for (const [id, hash, path] of expected) { + const got = hashes.get(id); + if (got === undefined) { + throw new Error(`${library}: ${path} is missing from the library`); + } + if (got !== hash) { + throw new Error(`${library}: ${path} changed since these bindings were generated`); + } } } } diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__README_md.snap b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__README_md.snap deleted file mode 100644 index 4b5e06a1..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__README_md.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# nested_modules - -Native bindings for nested_modules - -WebAssembly bindings for a `wasm32-unknown-unknown` build of the `nested_modules` library, linked with `--export-table --growable-table` (for example `RUSTFLAGS="-C link-arg=--export-table -C link-arg=--growable-table"`). `init()` loads `nested_modules.wasm` from next to this package, or from the path in `NESTED_MODULES_LIBRARY` on Node.js; pass a URL, bytes, or a compiled module to load it from elsewhere: - -```js -import { init, graphics } from 'nested_modules'; - -await init(); -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_d_ts.snap index 093fc136..bdae45d9 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -31,13 +31,13 @@ export declare function init( source?: string | URL | BufferSource | WebAssembly.Module | Response, ): Promise; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; -/** Construct a Canvas. */ export declare namespace graphics { /** The errors of the `graphics` error domain. */ export class GraphicsError extends $Error { @@ -61,7 +61,6 @@ export declare namespace graphics { } /** Construct a Canvas. */ export function newCanvas(width: number): graphics.Canvas; - /** Construct a Shape. */ export namespace shapes { /** A struct one level deep that references both an ancestor enum and a deeper struct. */ export interface Shape { @@ -75,7 +74,6 @@ export declare namespace graphics { export function makeShape(sides: number): graphics.shapes.Shape; /** Nested function returning a struct owned by an ancestor module. */ export function describe(): graphics.Canvas; - /** Deeper function taking an ancestor-module struct. Throws through the root module's inherited GraphicsError domain. */ export namespace geometry { /** A struct two levels deep. */ export interface Detail { diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_js.snap index 466a0d96..f1b39276 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -36,6 +39,23 @@ let $raw = $unloaded('nested_modules'); const $token = Number; let $loading = null; +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['graphics', [ + [0x2c29eef9327a6bd4n, 0x12983d5271651389n, 'graphics.Unit'], + [0x304088bbc28f0015n, 0x97e6e809913be6b6n, 'graphics.new_canvas'], + [0x49f8837532df9ac6n, 0x32e5b27e81ee0957n, 'graphics.Canvas'], + [0x55717fa027336b47n, 0xb2bd3a6bdcce48c8n, 'graphics.shapes.describe'], + [0x7626b91b8f1f53cdn, 0x867bbf7cf0f41c74n, 'graphics.shapes.geometry.compute'], + [0xb3efcb50d9a61ab7n, 0x3cb0207dfad2a149n, 'graphics.GraphicsError'], + [0xc62b16a9591b744bn, 0x7895610c8487eb1bn, 'graphics.shapes.Shape'], + [0xe13001f4983ccd35n, 0x16e7de070b4a4cfcn, 'graphics.shapes.geometry.Detail'], + [0xf3f658bcfc5dba62n, 0x2665ccb47356e746n, 'graphics.shapes.make_shape'], + ]], +]; + /** * Load the WebAssembly module (by default `nested_modules.wasm` next to this file, or * the file named by `NESTED_MODULES_LIBRARY` on Node.js) and check that it matches @@ -47,7 +67,7 @@ export function init(source) { .then((m) => { $raw = $bind(m); try { - $verify($raw, 'nested_modules', 'nested_modules', 3, [['graphics', 0x44d31b5b10fdb90dn]]); + $verify($raw, 'nested_modules', 'nested_modules', 4, $contract); } catch (e) { $raw = $unloaded('nested_modules'); throw e; @@ -65,11 +85,11 @@ function $bind(m) { const x = m.x; return { nested_modules_abi_version: () => x.nested_modules_abi_version() >>> 0, - nested_modules_debug_live: (kind) => BigInt.asUintN(64, x.nested_modules_debug_live(m.num(kind))), + nested_modules_debug_live: (kind) => m.live(x.nested_modules_debug_live(m.num(kind)), kind), nested_modules_cancel_token_create: () => x.nested_modules_cancel_token_create(), nested_modules_cancel_token_cancel: (t) => x.nested_modules_cancel_token_cancel(m.handle(t)), nested_modules_cancel_token_destroy: (t) => x.nested_modules_cancel_token_destroy(m.handle(t)), - nested_modules_graphics_checksum: () => BigInt.asUintN(64, x.nested_modules_graphics_checksum()), + nested_modules_graphics_contract: () => m.contract(x.nested_modules_graphics_contract), nested_modules_graphics_new_canvas: (a0) => { const r = x.nested_modules_graphics_new_canvas(m.num(a0), m.len, m.err); m.check(); @@ -100,9 +120,10 @@ function $bind(m) { } /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.nested_modules_debug_live(kind); @@ -115,9 +136,12 @@ const graphics$OutOfBoundsError = class OutOfBoundsError extends graphics$Graphi super(1, message); } }; -const $codes$graphics$GraphicsError = new Map([[1, graphics$OutOfBoundsError]]); +const $codes$graphics$GraphicsError = new Map([ + [1, graphics$OutOfBoundsError], +]); +const $payloads$graphics$GraphicsError = new Map(); function $from$graphics$GraphicsError(e) { - return $domain(e, $codes$graphics$GraphicsError); + return $domain(e, $codes$graphics$GraphicsError, $payloads$graphics$GraphicsError); } const graphics$Unit = Object.freeze({ diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__linear_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__linear_js.snap deleted file mode 100644 index 608de11a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__linear_js.snap +++ /dev/null @@ -1,403 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The WebAssembly transport: loading the module, staging arguments in its -// linear memory, reading results and errors back, and installing JavaScript -// functions in its function table (async completions and callback -// interface vtables). The generated `index.js` builds one native entry point -// per C symbol on top of this. - -import { $Fault } from './runtime.js'; - -const $enc = new TextEncoder(); -const $dec = new TextDecoder(); - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v, bits, what) { - let b; - if (typeof v === 'bigint') { - b = v; - } else if (typeof v === 'number' && Number.isInteger(v)) { - b = BigInt(v); - } else { - throw $expected('a bigint', v); - } - if (bits(64, b) !== b) throw new RangeError(`bigint does not fit in ${what} 64-bit integer`); - return b; -} - -/** - * One loaded module's linear memory and runtime symbols. `x` holds the - * module's exports by C symbol name; `memory()` returns the current - * `ArrayBuffer`; `fnptr(params, results, sig, fn)` installs `fn` in the - * function table and returns its index. - */ -export class $Linear { - constructor(x, prefix, memory, fnptr) { - this.x = x; - this.memory = memory; - this.fnptr = fnptr; - this.allocFn = x[`${prefix}_alloc`]; - this.deallocFn = x[`${prefix}_dealloc`]; - this.freeBytesFn = x[`${prefix}_free_bytes`]; - this.errorClearFn = x[`${prefix}_error_clear`]; - this.errorFreeFn = x[`${prefix}_error_free`]; - this.errorSetFn = x[`${prefix}_error_set`]; - if (typeof this.allocFn !== 'function' || typeof this.deallocFn !== 'function') { - throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_dealloc`); - } - // One scratch block reused by every call: the error slot (16 bytes), - // the out_len slot (4), and the iterator out_item slot (8). The producer - // only writes them as a call returns, so a call made from a callback in - // the middle of another call can reuse them safely. - const scratch = this.allocFn(32); - this.bytes().fill(0, scratch, scratch + 32); - this.err = scratch; - this.len = scratch + 16; - this.item = scratch + 24; - this.callbacks = new Map(); - this.nextCallback = 1; - this.pending = new Map(); - this.nextPending = 1; - this.completions = new Map(); - this.vtables = new Map(); - } - - bytes() { - const buffer = this.memory(); - if (buffer !== this.buffer) { - this.buffer = buffer; - this.u8 = new Uint8Array(buffer); - this.dv = new DataView(buffer); - } - return this.u8; - } - - view() { - this.bytes(); - return this.dv; - } - - // Argument checks and conversions for by-value slots. - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - i64(v) { - return $bigint(v, BigInt.asIntN, 'a signed'); - } - - u64(v) { - return $bigint(v, BigInt.asUintN, 'an unsigned'); - } - - bool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - return v ? 1 : 0; - } - - handle(v) { - if (typeof v !== 'number' || v === 0) throw $expected('an object handle', v); - return v; - } - - handleOpt(v) { - return v === null || v === undefined ? 0 : this.handle(v); - } - - // Staging: copy a string or bytes into linear memory for one call. A - // staged value is [ptr, len, size]; release it with unstage(). - str(s) { - if (typeof s !== 'string') throw $expected('a string', s); - if (s.length === 0) return [0, 0, 0]; - const size = s.length * 3; - const ptr = this.allocFn(size); - const { written } = $enc.encodeInto(s, this.bytes().subarray(ptr, ptr + size)); - return [ptr, written, size]; - } - - data(b) { - if (!(b instanceof Uint8Array)) throw $expected('a Uint8Array', b); - if (b.length === 0) return [0, 0, 0]; - const ptr = this.allocFn(b.length); - this.bytes().set(b, ptr); - return [ptr, b.length, b.length]; - } - - unstage(staged) { - if (staged !== null && staged[2] !== 0) this.deallocFn(staged[0], staged[2]); - } - - // Results. - outLen() { - return this.view().getUint32(this.len, true); - } - - readStr(ptr, len) { - return len === 0 ? '' : $dec.decode(this.bytes().subarray(ptr, ptr + len)); - } - - readData(ptr, len) { - return len === 0 ? new Uint8Array(0) : this.bytes().slice(ptr, ptr + len); - } - - takeStr(ptr, len) { - const s = this.readStr(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return s; - } - - takeData(ptr, len) { - const b = this.readData(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return b; - } - - cstr(ptr) { - if (ptr === 0) return ''; - const u8 = this.bytes(); - let end = ptr; - while (u8[end] !== 0) end++; - return $dec.decode(u8.subarray(ptr, end)); - } - - fault(errPtr) { - const dv = this.view(); - const code = dv.getInt32(errPtr, true); - if (code === 0) return null; - const payloadPtr = dv.getUint32(errPtr + 8, true); - const payloadLen = dv.getUint32(errPtr + 12, true); - return new $Fault( - code, - this.cstr(dv.getUint32(errPtr + 4, true)), - payloadPtr === 0 ? null : this.bytes().slice(payloadPtr, payloadPtr + payloadLen), - ); - } - - /** Throw the error the last call wrote to the scratch error slot. */ - check() { - if (this.view().getInt32(this.err, true) === 0) return; - const fault = this.fault(this.err); - this.errorClearFn(this.err); - throw fault; - } - - // Callback interfaces: implementations live in a table keyed by the - // integer passed as `ctx`, until the producer calls the vtable's `free`. - register(adapter) { - const id = this.nextCallback++; - this.callbacks.set(id, adapter); - return id; - } - - adapter(id) { - const adapter = this.callbacks.get(id); - if (adapter === undefined) throw new Error('callback implementation was already released'); - return adapter; - } - - unregister(id) { - this.callbacks.delete(id); - } - - /** Report a callback implementation's exception through `errPtr` (code -4). */ - foreign(errPtr, e) { - const message = e instanceof Error ? e.message || e.name : String(e); - const bytes = $enc.encode(message); - const ptr = this.allocFn(bytes.length + 1); - const u8 = this.bytes(); - u8.set(bytes, ptr); - u8[ptr + bytes.length] = 0; - this.errorSetFn(errPtr, -4, ptr); - this.deallocFn(ptr, bytes.length + 1); - } - - /** - * The static vtable of one callback interface: one table entry per - * method (`methods()` returns each as `[params, results, sig, fn]`), then - * `free`. Built on first use. - */ - vtable(name, methods) { - let ptr = this.vtables.get(name); - if (ptr !== undefined) return ptr; - const entries = [...methods(), [['i32'], [], 'vi', (ctx) => this.unregister(ctx)]]; - ptr = this.allocFn(entries.length * 4); - const dv = this.view(); - entries.forEach(([params, results, sig, fn], i) => { - dv.setUint32(ptr + i * 4, this.fnptr(params, results, sig, fn), true); - }); - this.vtables.set(name, ptr); - return ptr; - } - - /** - * Launch an async call. `call(callback, context)` invokes the launcher; - * the completion (one table entry per signature, shared by every call) - * settles the returned promise, converting the result slots with - * `convert`. On this single-threaded target the completion normally - * fires before the launcher returns. - */ - launch(params, results, sig, call, convert) { - let callback = this.completions.get(sig); - if (callback === undefined) { - const complete = (ctx, errPtr, ...result) => this.complete(ctx, errPtr, result); - callback = this.fnptr(params, results, sig, complete); - this.completions.set(sig, callback); - } - const id = this.nextPending++; - return new Promise((resolve, reject) => { - this.pending.set(id, { resolve, reject, convert }); - try { - call(callback, id); - } catch (e) { - this.pending.delete(id); - reject(e); - } - }); - } - - complete(id, errPtr, result) { - const pending = this.pending.get(id); - if (pending === undefined) return; - this.pending.delete(id); - try { - const fault = errPtr === 0 ? null : this.fault(errPtr); - if (errPtr !== 0) this.errorFreeFn(errPtr); - if (fault !== null) { - pending.reject(fault); - } else { - pending.resolve(pending.convert(...result)); - } - } catch (e) { - pending.reject(e); - } - } -} - -/** A stand-in for the native entry points until `init()` has finished. */ -export function $unloaded(library) { - return new Proxy( - {}, - { - get() { - throw new Error(`${library}: call init() before using the bindings`); - }, - }, - ); -} - -const $valtype = { i32: 0x7f, i64: 0x7e, f32: 0x7d, f64: 0x7c }; - -function $leb(n) { - const out = []; - do { - let b = n & 0x7f; - n >>>= 7; - if (n !== 0) b |= 0x80; - out.push(b); - } while (n !== 0); - return out; -} - -function $section(id, body) { - return [id, ...$leb(body.length), ...body]; -} - -/** - * Turn a JavaScript function into a WebAssembly function with the given - * signature: `WebAssembly.Function` where the engine has it, otherwise a - * tiny module that imports `fn` and re-exports it. - */ -function $wasmFunction(params, results, fn) { - if (typeof WebAssembly.Function === 'function') { - return new WebAssembly.Function({ parameters: params, results }, fn); - } - const type = [0x60, ...$leb(params.length), ...params.map((t) => $valtype[t])]; - type.push(...$leb(results.length), ...results.map((t) => $valtype[t])); - const bytes = new Uint8Array([ - 0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, - ...$section(1, [1, ...type]), - ...$section(2, [1, 1, 0x65, 1, 0x66, 0x00, 0x00]), - ...$section(7, [1, 1, 0x66, 0x00, 0x00]), - ]); - const instance = new WebAssembly.Instance(new WebAssembly.Module(bytes), { e: { f: fn } }); - return instance.exports.f; -} - -async function $bytesOf(source, envVar, fallback) { - if (source === undefined || source === null) { - const env = typeof process === 'object' && process.env ? process.env[envVar] : undefined; - if (env) { - const { readFile } = await import('node:fs/promises'); - return readFile(env); - } - source = fallback; - } - if (source instanceof WebAssembly.Module || source instanceof ArrayBuffer || ArrayBuffer.isView(source)) { - return source; - } - if (typeof Response === 'function' && source instanceof Response) { - return source.arrayBuffer(); - } - const url = source instanceof URL ? source : new URL(String(source), fallback); - if (url.protocol === 'file:') { - const { readFile } = await import('node:fs/promises'); - return readFile(url); - } - const response = await fetch(url); - if (!response.ok) throw new Error(`failed to fetch ${url}: ${response.status}`); - return response.arrayBuffer(); -} - -/** - * Instantiate a `wasm32-unknown-unknown` module from `source` (a URL, a - * path, bytes, a `Response`, or a compiled module). Without a source it - * reads the file named by the `envVar` environment variable (on Node.js), - * else `fallback`. The module must export its memory and a growable - * function table (`__indirect_function_table`). - */ -export async function $loadWasm(source, prefix, envVar, fallback) { - const bytes = await $bytesOf(source, envVar, fallback); - const module = bytes instanceof WebAssembly.Module ? bytes : await WebAssembly.compile(bytes); - const { exports } = await WebAssembly.instantiate(module, {}); - const table = exports.__indirect_function_table; - const fnptr = (params, results, sig, fn) => { - if (!(table instanceof WebAssembly.Table)) { - throw new Error('the WebAssembly module does not export __indirect_function_table (link with --export-table --growable-table)'); - } - const index = table.grow(1); - table.set(index, $wasmFunction(params, results, fn)); - return index; - }; - return new $Linear(exports, prefix, () => exports.memory.buffer, fnptr); -} - -/** - * Adopt an initialized Emscripten module (or the promise its `MODULARIZE` - * factory returns). Its exports carry a leading underscore, its memory is - * read through `HEAPU8`, and table entries are installed with - * `addFunction`, so it must be linked with `-sALLOW_TABLE_GROWTH`, - * `-sWASM_BIGINT`, and `-sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8`. - */ -export async function $loadEmscripten(module, prefix, symbols) { - const m = await module; - const x = {}; - for (const symbol of symbols) x[symbol] = m[`_${symbol}`]; - const fnptr = (params, results, sig, fn) => { - if (typeof m.addFunction !== 'function') { - throw new Error('the Emscripten module does not export addFunction (link with -sALLOW_TABLE_GROWTH and -sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8)'); - } - return m.addFunction(fn, sig); - }; - return new $Linear(x, prefix, () => m.HEAPU8.buffer, fnptr); -} - -// END linear.js diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__package_json.snap b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__package_json.snap deleted file mode 100644 index f5f96d6c..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__package_json.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "nested_modules", - "version": "0.1.0", - "description": "Native bindings for nested_modules", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "linear.js" - ] -} diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__runtime_js.snap deleted file mode 100644 index 97a25292..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_nested_modules__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class NestedModulesError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends NestedModulesError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new NestedModulesError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new NestedModulesError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new NestedModulesError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new NestedModulesError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__README_md.snap b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__README_md.snap deleted file mode 100644 index 6550cedf..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__README_md.snap +++ /dev/null @@ -1,20 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- - - - - -# shapes - -Native bindings for shapes - -WebAssembly bindings for a `wasm32-unknown-unknown` build of the `shapes` library, linked with `--export-table --growable-table` (for example `RUSTFLAGS="-C link-arg=--export-table -C link-arg=--growable-table"`). `init()` loads `shapes.wasm` from next to this package, or from the path in `SHAPES_LIBRARY` on Node.js; pass a URL, bytes, or a compiled module to load it from elsewhere: - -```js -import { init, shapes } from 'shapes'; - -await init(); -``` - - diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_d_ts.snap b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_d_ts.snap index cb80c226..d74e8799 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_d_ts.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_d_ts.snap @@ -1,9 +1,9 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate /// @@ -31,8 +31,9 @@ export declare function init( source?: string | URL | BufferSource | WebAssembly.Module | Response, ): Promise; /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all. * @internal */ export declare function __debugLive(kind: number): bigint; diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_js.snap index 269496e7..baaf6eff 100644 --- a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_js.snap +++ b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__index_js.snap @@ -1,15 +1,17 @@ --- source: crates/weaveffi-cli/tests/snapshots.rs --- -// Generated by WeaveFFI [VERSION] from api.yml +// Generated by WeaveFFI [VERSION] // DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o +// To regenerate: weaveffi generate import { $Fault, $fault, $domain, + $raise, $W, + $WK, $R, $encode, $decode, @@ -21,6 +23,7 @@ import { $lendOpt, $unlend, $clone, + $cloneOpt, $Iterator, $cancellable, $impl, @@ -36,6 +39,21 @@ let $raw = $unloaded('shapes'); const $token = Number; let $loading = null; +// The declarations these bindings were generated with, per top-level +// module, as [id, hash, path]: the library's contract tables must hold +// each one (see $verify). +const $contract = [ + ['shapes', [ + [0x0d7b3eec49243e7an, 0x5f232311808e1b1fn, 'shapes.Channel'], + [0x183fa8198100860en, 0xe40c6993cb31d359n, 'shapes.describe'], + [0x2d1395eb837b848en, 0x42202a0c12f3a16en, 'shapes.sum_bytes'], + [0x7f9a5b87ba09b86bn, 0xc5b32fb0efbbaa17n, 'shapes.scale'], + [0xc8edd779240fda17n, 0x68e1be78fddb20e4n, 'shapes.Numerics'], + [0xd3d3881b2fd18a62n, 0xf24d0eb76a99d918n, 'shapes.make_numerics'], + [0xf010844afff5ac3cn, 0x50de28ae9b41ff15n, 'shapes.Shape'], + ]], +]; + /** * Load the WebAssembly module (by default `shapes.wasm` next to this file, or * the file named by `SHAPES_LIBRARY` on Node.js) and check that it matches @@ -47,7 +65,7 @@ export function init(source) { .then((m) => { $raw = $bind(m); try { - $verify($raw, 'shapes', 'shapes', 3, [['shapes', 0x06acf433ffa2a0f0n]]); + $verify($raw, 'shapes', 'shapes', 4, $contract); } catch (e) { $raw = $unloaded('shapes'); throw e; @@ -65,11 +83,11 @@ function $bind(m) { const x = m.x; return { shapes_abi_version: () => x.shapes_abi_version() >>> 0, - shapes_debug_live: (kind) => BigInt.asUintN(64, x.shapes_debug_live(m.num(kind))), + shapes_debug_live: (kind) => m.live(x.shapes_debug_live(m.num(kind)), kind), shapes_cancel_token_create: () => x.shapes_cancel_token_create(), shapes_cancel_token_cancel: (t) => x.shapes_cancel_token_cancel(m.handle(t)), shapes_cancel_token_destroy: (t) => x.shapes_cancel_token_destroy(m.handle(t)), - shapes_shapes_checksum: () => BigInt.asUintN(64, x.shapes_shapes_checksum()), + shapes_shapes_contract: () => m.contract(x.shapes_shapes_contract), shapes_shapes_describe: (a0) => { let s0 = null; try { @@ -112,9 +130,10 @@ function $bind(m) { } /** - * The native library's live-allocation counters (0 objects, 1 callbacks, - * 2 iterators, 3 cancel tokens, 4 returned buffers), for leak tests. Always - * zero unless the library enables its `leak-check` feature. + * The native library's live-resource counters (0 objects, 1 callbacks, + * 2 iterators, 3 cancel tokens, 4 byte runs), for leak tests. Kind -1 is + * `1n` when the library counts at all (its `leak-check` feature), else + * every kind is `0n`. */ export function __debugLive(kind) { return $raw.shapes_debug_live(kind); diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__linear_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__linear_js.snap deleted file mode 100644 index 608de11a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__linear_js.snap +++ /dev/null @@ -1,403 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The WebAssembly transport: loading the module, staging arguments in its -// linear memory, reading results and errors back, and installing JavaScript -// functions in its function table (async completions and callback -// interface vtables). The generated `index.js` builds one native entry point -// per C symbol on top of this. - -import { $Fault } from './runtime.js'; - -const $enc = new TextEncoder(); -const $dec = new TextDecoder(); - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v, bits, what) { - let b; - if (typeof v === 'bigint') { - b = v; - } else if (typeof v === 'number' && Number.isInteger(v)) { - b = BigInt(v); - } else { - throw $expected('a bigint', v); - } - if (bits(64, b) !== b) throw new RangeError(`bigint does not fit in ${what} 64-bit integer`); - return b; -} - -/** - * One loaded module's linear memory and runtime symbols. `x` holds the - * module's exports by C symbol name; `memory()` returns the current - * `ArrayBuffer`; `fnptr(params, results, sig, fn)` installs `fn` in the - * function table and returns its index. - */ -export class $Linear { - constructor(x, prefix, memory, fnptr) { - this.x = x; - this.memory = memory; - this.fnptr = fnptr; - this.allocFn = x[`${prefix}_alloc`]; - this.deallocFn = x[`${prefix}_dealloc`]; - this.freeBytesFn = x[`${prefix}_free_bytes`]; - this.errorClearFn = x[`${prefix}_error_clear`]; - this.errorFreeFn = x[`${prefix}_error_free`]; - this.errorSetFn = x[`${prefix}_error_set`]; - if (typeof this.allocFn !== 'function' || typeof this.deallocFn !== 'function') { - throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_dealloc`); - } - // One scratch block reused by every call: the error slot (16 bytes), - // the out_len slot (4), and the iterator out_item slot (8). The producer - // only writes them as a call returns, so a call made from a callback in - // the middle of another call can reuse them safely. - const scratch = this.allocFn(32); - this.bytes().fill(0, scratch, scratch + 32); - this.err = scratch; - this.len = scratch + 16; - this.item = scratch + 24; - this.callbacks = new Map(); - this.nextCallback = 1; - this.pending = new Map(); - this.nextPending = 1; - this.completions = new Map(); - this.vtables = new Map(); - } - - bytes() { - const buffer = this.memory(); - if (buffer !== this.buffer) { - this.buffer = buffer; - this.u8 = new Uint8Array(buffer); - this.dv = new DataView(buffer); - } - return this.u8; - } - - view() { - this.bytes(); - return this.dv; - } - - // Argument checks and conversions for by-value slots. - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - i64(v) { - return $bigint(v, BigInt.asIntN, 'a signed'); - } - - u64(v) { - return $bigint(v, BigInt.asUintN, 'an unsigned'); - } - - bool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - return v ? 1 : 0; - } - - handle(v) { - if (typeof v !== 'number' || v === 0) throw $expected('an object handle', v); - return v; - } - - handleOpt(v) { - return v === null || v === undefined ? 0 : this.handle(v); - } - - // Staging: copy a string or bytes into linear memory for one call. A - // staged value is [ptr, len, size]; release it with unstage(). - str(s) { - if (typeof s !== 'string') throw $expected('a string', s); - if (s.length === 0) return [0, 0, 0]; - const size = s.length * 3; - const ptr = this.allocFn(size); - const { written } = $enc.encodeInto(s, this.bytes().subarray(ptr, ptr + size)); - return [ptr, written, size]; - } - - data(b) { - if (!(b instanceof Uint8Array)) throw $expected('a Uint8Array', b); - if (b.length === 0) return [0, 0, 0]; - const ptr = this.allocFn(b.length); - this.bytes().set(b, ptr); - return [ptr, b.length, b.length]; - } - - unstage(staged) { - if (staged !== null && staged[2] !== 0) this.deallocFn(staged[0], staged[2]); - } - - // Results. - outLen() { - return this.view().getUint32(this.len, true); - } - - readStr(ptr, len) { - return len === 0 ? '' : $dec.decode(this.bytes().subarray(ptr, ptr + len)); - } - - readData(ptr, len) { - return len === 0 ? new Uint8Array(0) : this.bytes().slice(ptr, ptr + len); - } - - takeStr(ptr, len) { - const s = this.readStr(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return s; - } - - takeData(ptr, len) { - const b = this.readData(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return b; - } - - cstr(ptr) { - if (ptr === 0) return ''; - const u8 = this.bytes(); - let end = ptr; - while (u8[end] !== 0) end++; - return $dec.decode(u8.subarray(ptr, end)); - } - - fault(errPtr) { - const dv = this.view(); - const code = dv.getInt32(errPtr, true); - if (code === 0) return null; - const payloadPtr = dv.getUint32(errPtr + 8, true); - const payloadLen = dv.getUint32(errPtr + 12, true); - return new $Fault( - code, - this.cstr(dv.getUint32(errPtr + 4, true)), - payloadPtr === 0 ? null : this.bytes().slice(payloadPtr, payloadPtr + payloadLen), - ); - } - - /** Throw the error the last call wrote to the scratch error slot. */ - check() { - if (this.view().getInt32(this.err, true) === 0) return; - const fault = this.fault(this.err); - this.errorClearFn(this.err); - throw fault; - } - - // Callback interfaces: implementations live in a table keyed by the - // integer passed as `ctx`, until the producer calls the vtable's `free`. - register(adapter) { - const id = this.nextCallback++; - this.callbacks.set(id, adapter); - return id; - } - - adapter(id) { - const adapter = this.callbacks.get(id); - if (adapter === undefined) throw new Error('callback implementation was already released'); - return adapter; - } - - unregister(id) { - this.callbacks.delete(id); - } - - /** Report a callback implementation's exception through `errPtr` (code -4). */ - foreign(errPtr, e) { - const message = e instanceof Error ? e.message || e.name : String(e); - const bytes = $enc.encode(message); - const ptr = this.allocFn(bytes.length + 1); - const u8 = this.bytes(); - u8.set(bytes, ptr); - u8[ptr + bytes.length] = 0; - this.errorSetFn(errPtr, -4, ptr); - this.deallocFn(ptr, bytes.length + 1); - } - - /** - * The static vtable of one callback interface: one table entry per - * method (`methods()` returns each as `[params, results, sig, fn]`), then - * `free`. Built on first use. - */ - vtable(name, methods) { - let ptr = this.vtables.get(name); - if (ptr !== undefined) return ptr; - const entries = [...methods(), [['i32'], [], 'vi', (ctx) => this.unregister(ctx)]]; - ptr = this.allocFn(entries.length * 4); - const dv = this.view(); - entries.forEach(([params, results, sig, fn], i) => { - dv.setUint32(ptr + i * 4, this.fnptr(params, results, sig, fn), true); - }); - this.vtables.set(name, ptr); - return ptr; - } - - /** - * Launch an async call. `call(callback, context)` invokes the launcher; - * the completion (one table entry per signature, shared by every call) - * settles the returned promise, converting the result slots with - * `convert`. On this single-threaded target the completion normally - * fires before the launcher returns. - */ - launch(params, results, sig, call, convert) { - let callback = this.completions.get(sig); - if (callback === undefined) { - const complete = (ctx, errPtr, ...result) => this.complete(ctx, errPtr, result); - callback = this.fnptr(params, results, sig, complete); - this.completions.set(sig, callback); - } - const id = this.nextPending++; - return new Promise((resolve, reject) => { - this.pending.set(id, { resolve, reject, convert }); - try { - call(callback, id); - } catch (e) { - this.pending.delete(id); - reject(e); - } - }); - } - - complete(id, errPtr, result) { - const pending = this.pending.get(id); - if (pending === undefined) return; - this.pending.delete(id); - try { - const fault = errPtr === 0 ? null : this.fault(errPtr); - if (errPtr !== 0) this.errorFreeFn(errPtr); - if (fault !== null) { - pending.reject(fault); - } else { - pending.resolve(pending.convert(...result)); - } - } catch (e) { - pending.reject(e); - } - } -} - -/** A stand-in for the native entry points until `init()` has finished. */ -export function $unloaded(library) { - return new Proxy( - {}, - { - get() { - throw new Error(`${library}: call init() before using the bindings`); - }, - }, - ); -} - -const $valtype = { i32: 0x7f, i64: 0x7e, f32: 0x7d, f64: 0x7c }; - -function $leb(n) { - const out = []; - do { - let b = n & 0x7f; - n >>>= 7; - if (n !== 0) b |= 0x80; - out.push(b); - } while (n !== 0); - return out; -} - -function $section(id, body) { - return [id, ...$leb(body.length), ...body]; -} - -/** - * Turn a JavaScript function into a WebAssembly function with the given - * signature: `WebAssembly.Function` where the engine has it, otherwise a - * tiny module that imports `fn` and re-exports it. - */ -function $wasmFunction(params, results, fn) { - if (typeof WebAssembly.Function === 'function') { - return new WebAssembly.Function({ parameters: params, results }, fn); - } - const type = [0x60, ...$leb(params.length), ...params.map((t) => $valtype[t])]; - type.push(...$leb(results.length), ...results.map((t) => $valtype[t])); - const bytes = new Uint8Array([ - 0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, - ...$section(1, [1, ...type]), - ...$section(2, [1, 1, 0x65, 1, 0x66, 0x00, 0x00]), - ...$section(7, [1, 1, 0x66, 0x00, 0x00]), - ]); - const instance = new WebAssembly.Instance(new WebAssembly.Module(bytes), { e: { f: fn } }); - return instance.exports.f; -} - -async function $bytesOf(source, envVar, fallback) { - if (source === undefined || source === null) { - const env = typeof process === 'object' && process.env ? process.env[envVar] : undefined; - if (env) { - const { readFile } = await import('node:fs/promises'); - return readFile(env); - } - source = fallback; - } - if (source instanceof WebAssembly.Module || source instanceof ArrayBuffer || ArrayBuffer.isView(source)) { - return source; - } - if (typeof Response === 'function' && source instanceof Response) { - return source.arrayBuffer(); - } - const url = source instanceof URL ? source : new URL(String(source), fallback); - if (url.protocol === 'file:') { - const { readFile } = await import('node:fs/promises'); - return readFile(url); - } - const response = await fetch(url); - if (!response.ok) throw new Error(`failed to fetch ${url}: ${response.status}`); - return response.arrayBuffer(); -} - -/** - * Instantiate a `wasm32-unknown-unknown` module from `source` (a URL, a - * path, bytes, a `Response`, or a compiled module). Without a source it - * reads the file named by the `envVar` environment variable (on Node.js), - * else `fallback`. The module must export its memory and a growable - * function table (`__indirect_function_table`). - */ -export async function $loadWasm(source, prefix, envVar, fallback) { - const bytes = await $bytesOf(source, envVar, fallback); - const module = bytes instanceof WebAssembly.Module ? bytes : await WebAssembly.compile(bytes); - const { exports } = await WebAssembly.instantiate(module, {}); - const table = exports.__indirect_function_table; - const fnptr = (params, results, sig, fn) => { - if (!(table instanceof WebAssembly.Table)) { - throw new Error('the WebAssembly module does not export __indirect_function_table (link with --export-table --growable-table)'); - } - const index = table.grow(1); - table.set(index, $wasmFunction(params, results, fn)); - return index; - }; - return new $Linear(exports, prefix, () => exports.memory.buffer, fnptr); -} - -/** - * Adopt an initialized Emscripten module (or the promise its `MODULARIZE` - * factory returns). Its exports carry a leading underscore, its memory is - * read through `HEAPU8`, and table entries are installed with - * `addFunction`, so it must be linked with `-sALLOW_TABLE_GROWTH`, - * `-sWASM_BIGINT`, and `-sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8`. - */ -export async function $loadEmscripten(module, prefix, symbols) { - const m = await module; - const x = {}; - for (const symbol of symbols) x[symbol] = m[`_${symbol}`]; - const fnptr = (params, results, sig, fn) => { - if (typeof m.addFunction !== 'function') { - throw new Error('the Emscripten module does not export addFunction (link with -sALLOW_TABLE_GROWTH and -sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8)'); - } - return m.addFunction(fn, sig); - }; - return new $Linear(x, prefix, () => m.HEAPU8.buffer, fnptr); -} - -// END linear.js diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__package_json.snap b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__package_json.snap deleted file mode 100644 index e1c5847a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__package_json.snap +++ /dev/null @@ -1,26 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -{ - "//": "Generated by WeaveFFI [VERSION] from api.yml", - "//warning": "DO NOT EDIT. Your changes will be overwritten.", - "//regenerate": "To regenerate: weaveffi generate api.yml -o ", - "name": "shapes", - "version": "0.1.0", - "description": "Native bindings for shapes", - "type": "module", - "main": "index.js", - "types": "index.d.ts", - "exports": { - ".": { - "types": "./index.d.ts", - "default": "./index.js" - } - }, - "files": [ - "index.js", - "index.d.ts", - "runtime.js", - "linear.js" - ] -} diff --git a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__runtime_js.snap b/crates/weaveffi-cli/tests/snapshots/wasm_shapes__runtime_js.snap deleted file mode 100644 index 323cda6a..00000000 --- a/crates/weaveffi-cli/tests/snapshots/wasm_shapes__runtime_js.snap +++ /dev/null @@ -1,578 +0,0 @@ ---- -source: crates/weaveffi-cli/tests/snapshots.rs ---- -// Generated by WeaveFFI [VERSION] from api.yml -// DO NOT EDIT. Your changes will be overwritten. -// To regenerate: weaveffi generate api.yml -o - -// The runtime shared by every binding in this package: the error classes, -// the value-buffer codec, the object and iterator wrappers, cancellation, and -// the load-time contract check. The generated `index.js` imports it; the -// native transport (an N-API addon or a WebAssembly instance) never sees it, -// except for `$Fault`, the shape every native failure arrives in. - -/** - * The root of every error these bindings throw. `code` is the ABI error - * code: positive for a declared domain error, negative for a runtime trap - * (-1 generic, -2 producer panic, -3 marshalling failure, -4 a callback - * implementation failed, -5 cancelled). - */ -export class ShapesError extends Error { - constructor(code, message) { - super(message === undefined ? '' : message); - this.name = new.target.name; - this.code = code; - } -} - -/** The error a cancelled async call rejects with (code -5). */ -export class CancelledError extends ShapesError { - constructor(message = 'cancelled') { - super(-5, message); - } -} - -/** - * A failure as the native transport reports it, before the generated - * wrappers map it onto an error class. `payload` holds the matched error - * code's fields as a value buffer, or `null`. - */ -export class $Fault { - constructor(code, message, payload) { - this.code = code; - this.message = message; - this.payload = payload === undefined ? null : payload; - } -} - -/** - * Map an exception raised by a native call onto the public error classes: - * a cancelled call becomes `CancelledError`, any other fault the root error, - * and a WebAssembly trap (a producer panic on a `panic = "abort"` build) a - * code -2 error. Anything else (a `TypeError` from argument checking, say) - * passes through unchanged. - */ -export function $fault(e) { - if (e instanceof $Fault) { - return e.code === -5 - ? new CancelledError(e.message || undefined) - : new ShapesError(e.code, e.message); - } - if (typeof WebAssembly === 'object' && e instanceof WebAssembly.RuntimeError) { - return new ShapesError(-2, 'the native library trapped: ' + e.message); - } - return e; -} - -/** - * Map a fault onto an error domain: `codes` maps each declared code to its - * class, and `payloads` (optional) maps codes that carry fields to the - * reader that decodes them onto the error. Codes outside the domain fall - * back to `$fault`. - */ -export function $domain(e, codes, payloads) { - if (e instanceof $Fault) { - const cls = codes.get(e.code); - if (cls !== undefined) { - const err = new cls(e.message || undefined); - const read = payloads === undefined ? undefined : payloads.get(e.code); - if (read !== undefined && e.payload !== null) { - Object.assign(err, $decode(e.payload, read)); - } - return err; - } - } - return $fault(e); -} - -function $malformed(what) { - return new ShapesError(-3, 'malformed value buffer: ' + what); -} - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw $expected('a bigint', v); -} - -const $utf8 = new TextEncoder(); -const $utf8Strict = new TextDecoder('utf-8', { fatal: true }); - -/** - * Writes the value-buffer wire format: little-endian, packed, no alignment. - * Strings and bytes are a u32 length then the bytes, optionals a presence - * byte then the value, lists a u32 count then the elements, and maps a u32 - * count then alternating keys and values. - */ -export class $Writer { - constructor() { - this.buf = new Uint8Array(64); - this.view = new DataView(this.buf.buffer); - this.len = 0; - } - - reserve(n) { - if (this.len + n <= this.buf.length) return; - let cap = this.buf.length * 2; - while (cap < this.len + n) cap *= 2; - const grown = new Uint8Array(cap); - grown.set(this.buf.subarray(0, this.len)); - this.buf = grown; - this.view = new DataView(grown.buffer); - } - - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - big(v, bits) { - const b = $bigint(v); - if (bits(64, b) !== b) throw new RangeError(`${b} does not fit in a 64-bit integer`); - return b; - } - - writeBool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - this.reserve(1); - this.buf[this.len++] = v ? 1 : 0; - } - - writeI8(v) { this.reserve(1); this.view.setInt8(this.len, this.num(v)); this.len += 1; } - writeU8(v) { this.reserve(1); this.view.setUint8(this.len, this.num(v)); this.len += 1; } - writeI16(v) { this.reserve(2); this.view.setInt16(this.len, this.num(v), true); this.len += 2; } - writeU16(v) { this.reserve(2); this.view.setUint16(this.len, this.num(v), true); this.len += 2; } - writeI32(v) { this.reserve(4); this.view.setInt32(this.len, this.num(v), true); this.len += 4; } - writeU32(v) { this.reserve(4); this.view.setUint32(this.len, this.num(v), true); this.len += 4; } - writeI64(v) { this.reserve(8); this.view.setBigInt64(this.len, this.big(v, BigInt.asIntN), true); this.len += 8; } - writeU64(v) { this.reserve(8); this.view.setBigUint64(this.len, this.big(v, BigInt.asUintN), true); this.len += 8; } - writeF32(v) { this.reserve(4); this.view.setFloat32(this.len, this.num(v), true); this.len += 4; } - writeF64(v) { this.reserve(8); this.view.setFloat64(this.len, this.num(v), true); this.len += 8; } - - writeString(v) { - if (typeof v !== 'string') throw $expected('a string', v); - // UTF-8 never needs more than three bytes per UTF-16 code unit. - this.reserve(4 + v.length * 3); - const { written } = $utf8.encodeInto(v, this.buf.subarray(this.len + 4)); - this.view.setUint32(this.len, written, true); - this.len += 4 + written; - } - - writeBytes(v) { - if (!(v instanceof Uint8Array)) throw $expected('a Uint8Array', v); - this.reserve(4 + v.length); - this.view.setUint32(this.len, v.length, true); - this.buf.set(v, this.len + 4); - this.len += 4 + v.length; - } - - writeOpt(v, write) { - if (v === null || v === undefined) { - this.writeBool(false); - } else { - this.writeBool(true); - write(this, v); - } - } - - writeList(v, write) { - if (!Array.isArray(v)) throw $expected('an array', v); - this.writeU32(v.length); - for (const e of v) write(this, e); - } - - writeMap(v, writeKey, writeValue) { - if (v === null || typeof v !== 'object') throw $expected('an object or a Map', v); - const entries = v instanceof Map ? [...v.entries()] : Object.entries(v); - this.writeU32(entries.length); - for (const [k, e] of entries) { - writeKey(this, k); - writeValue(this, e); - } - } - - finish() { - return this.buf.subarray(0, this.len); - } -} - -/** - * Reads the value-buffer wire format. Every read is bounds-checked; a - * truncated buffer, an invalid bool or presence byte, invalid UTF-8, or - * trailing bytes throw a code -3 error. - */ -export class $Reader { - constructor(bytes) { - this.buf = bytes; - this.view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength); - this.pos = 0; - } - - take(n) { - if (n > this.buf.length - this.pos) throw $malformed('truncated'); - const at = this.pos; - this.pos += n; - return at; - } - - readBool() { - const b = this.buf[this.take(1)]; - if (b > 1) throw $malformed('invalid bool byte ' + b); - return b === 1; - } - - readI8() { return this.view.getInt8(this.take(1)); } - readU8() { return this.view.getUint8(this.take(1)); } - readI16() { return this.view.getInt16(this.take(2), true); } - readU16() { return this.view.getUint16(this.take(2), true); } - readI32() { return this.view.getInt32(this.take(4), true); } - readU32() { return this.view.getUint32(this.take(4), true); } - readI64() { return this.view.getBigInt64(this.take(8), true); } - readU64() { return this.view.getBigUint64(this.take(8), true); } - readF32() { return this.view.getFloat32(this.take(4), true); } - readF64() { return this.view.getFloat64(this.take(8), true); } - - readString() { - const n = this.readU32(); - const at = this.take(n); - try { - return $utf8Strict.decode(this.buf.subarray(at, at + n)); - } catch { - throw $malformed('invalid UTF-8'); - } - } - - readBytes() { - const n = this.readU32(); - const at = this.take(n); - return this.buf.slice(at, at + n); - } - - readOpt(read) { - return this.readBool() ? read(this) : null; - } - - // A count may exceed the remaining bytes when elements encode to zero - // bytes (a record without fields), so it is not checked up front; a - // truncated element still fails on its first read. - readList(read) { - const n = this.readU32(); - const out = []; - for (let i = 0; i < n; i++) out.push(read(this)); - return out; - } - - readMap(readKey, readValue) { - const n = this.readU32(); - const out = {}; - for (let i = 0; i < n; i++) { - const k = readKey(this); - out[k] = readValue(this); - } - return out; - } - - end() { - if (this.pos !== this.buf.length) throw $malformed('trailing bytes'); - } -} - -/** Encode `v` with `write` into a fresh value buffer. */ -export function $encode(v, write) { - const w = new $Writer(); - write(w, v); - return w.finish(); -} - -/** Decode a whole value buffer with `read`, rejecting trailing bytes. */ -export function $decode(bytes, read) { - const r = new $Reader(bytes); - const v = read(r); - r.end(); - return v; -} - -/** Element writers for the fixed-width primitives, strings, and bytes. */ -export const $W = Object.freeze({ - Bool: (w, v) => w.writeBool(v), - I8: (w, v) => w.writeI8(v), - I16: (w, v) => w.writeI16(v), - I32: (w, v) => w.writeI32(v), - I64: (w, v) => w.writeI64(v), - U8: (w, v) => w.writeU8(v), - U16: (w, v) => w.writeU16(v), - U32: (w, v) => w.writeU32(v), - U64: (w, v) => w.writeU64(v), - F32: (w, v) => w.writeF32(v), - F64: (w, v) => w.writeF64(v), - String: (w, v) => w.writeString(v), - Bytes: (w, v) => w.writeBytes(v), -}); - -/** Element readers for the fixed-width primitives, strings, and bytes. */ -export const $R = Object.freeze({ - Bool: (r) => r.readBool(), - I8: (r) => r.readI8(), - I16: (r) => r.readI16(), - I32: (r) => r.readI32(), - I64: (r) => r.readI64(), - U8: (r) => r.readU8(), - U16: (r) => r.readU16(), - U32: (r) => r.readU32(), - U64: (r) => r.readU64(), - F32: (r) => r.readF32(), - F64: (r) => r.readF64(), - String: (r) => r.readString(), - Bytes: (r) => r.readBytes(), -}); - -/** The key `using` declarations call, or a stand-in where it is missing. */ -export const $dispose = typeof Symbol.dispose === 'symbol' ? Symbol.dispose : Symbol.for('Symbol.dispose'); - -// Releases the native reference of a wrapper collected without close(). -// The held value never references the wrapper, so it can't keep it alive. -const $objects = new FinalizationRegistry((held) => held.destroy(held.handle)); - -function $release(o) { - const handle = o.$h; - o.$h = null; - o.constructor.$destroy(handle); -} - -/** - * The base of every interface class. An instance owns one strong reference - * to a native object. `close()` (or a `using` declaration) releases it; - * a wrapper that is garbage collected unclosed is released by a - * `FinalizationRegistry`. A `close()` that races an in-flight call (from a - * callback, or while an async method is pending) is deferred until that - * call returns, so the native object is never freed mid-call. - */ -export class $Object { - close() { - if (this.$closed !== false) return; - this.$closed = true; - $objects.unregister(this); - if (this.$calls === 0) $release(this); - } - - [$dispose]() { - this.close(); - } -} - -/** Bind a native handle (one strong reference) to wrapper `o`. */ -export function $own(o, handle) { - o.$h = handle; - o.$calls = 0; - o.$closed = false; - $objects.register(o, { handle, destroy: o.constructor.$destroy }, o); - return o; -} - -/** Wrap a handle the native side handed over in a new `cls` instance. */ -export function $adopt(cls, handle) { - return $own(Object.create(cls.prototype), handle); -} - -/** `$adopt`, with `null` for an absent object. */ -export function $adoptOpt(cls, handle) { - return handle === null ? null : $adopt(cls, handle); -} - -/** - * Lend wrapper `o` (an instance of `cls`) to one native call and return its - * handle. Every `$lend` is paired with an `$unlend` when the call returns. - */ -export function $lend(o, cls) { - if (!(o instanceof cls)) throw $expected(`a ${cls.name}`, o); - if (o.$closed !== false) throw new ShapesError(-3, `${cls.name} used after close()`); - o.$calls++; - return o.$h; -} - -/** `$lend`, with `null` for an absent object. */ -export function $lendOpt(o, cls) { - return o === null || o === undefined ? null : $lend(o, cls); -} - -/** End one loan of `o`, completing a `close()` deferred by the loan. */ -export function $unlend(o) { - if (o === null || o === undefined) return; - if (--o.$calls === 0 && o.$closed) $release(o); -} - -/** - * A second strong reference to `o`'s native object, for an object token - * written into a value buffer (the native side adopts it). - */ -export function $clone(o, cls) { - const handle = $lend(o, cls); - try { - return cls.$clone(handle); - } finally { - $unlend(o); - } -} - -const $iterators = new FinalizationRegistry((held) => held.destroy(held.handle)); - -/** - * A lazy iterator over a native `iter`: each `next()` makes exactly one - * native call. The native iterator is destroyed exactly once: when it is - * exhausted or fails, by `return()` (which `for...of` calls on `break`), by - * `close()` or a `using` declaration, or when the iterator is collected. - * `spec` supplies `next`, `destroy`, `map` (the error mapping), and - * `convert` (or `null` when elements need no conversion). - */ -export class $Iterator { - constructor(handle, spec) { - this.$h = handle; - this.$spec = spec; - $iterators.register(this, { handle, destroy: spec.destroy }, this); - } - - next() { - if (this.$h === null) return { done: true, value: undefined }; - const spec = this.$spec; - let v; - try { - v = spec.next(this.$h); - } catch (e) { - this.close(); - throw spec.map(e); - } - if (v === undefined) { - this.close(); - return { done: true, value: undefined }; - } - return { done: false, value: spec.convert === null ? v : spec.convert(v) }; - } - - return(value) { - this.close(); - return { done: true, value }; - } - - close() { - const handle = this.$h; - if (handle === null) return; - this.$h = null; - $iterators.unregister(this); - this.$spec.destroy(handle); - } - - [Symbol.iterator]() { - return this; - } - - [$dispose]() { - this.close(); - } -} - -/** - * Run a cancellable launch. Without a signal, `launch(null)` runs as is. - * With one, a native cancel token is created and passed to `launch`, - * aborting the signal cancels it (an already-aborted signal cancels it - * before the launch, so the call completes with `CancelledError` at once), - * and the token is destroyed once the call settles. `tokens` supplies the - * native `create`, `cancel`, and `destroy`. - */ -export function $cancellable(signal, tokens, launch) { - if (signal === undefined || signal === null) return launch(null); - if (typeof signal.addEventListener !== 'function') throw $expected('an AbortSignal', signal); - const token = tokens.create(); - const abort = () => tokens.cancel(token); - const done = () => { - signal.removeEventListener('abort', abort); - tokens.destroy(token); - }; - if (signal.aborted) { - abort(); - } else { - signal.addEventListener('abort', abort, { once: true }); - } - let pending; - try { - pending = launch(token); - } catch (e) { - done(); - throw e; - } - return pending.finally(done); -} - -/** Check a callback interface implementation. */ -export function $impl(impl, what) { - if (impl === null || (typeof impl !== 'object' && typeof impl !== 'function')) { - throw $expected(`a ${what} implementation`, impl); - } - return impl; -} - -function $num(v, where) { - if (typeof v !== 'number') throw new TypeError(`${where} must return a number`); - return v; -} - -function $retBigint(v, where) { - if (typeof v === 'bigint') return v; - if (typeof v === 'number' && Number.isInteger(v)) return BigInt(v); - throw new TypeError(`${where} must return a bigint`); -} - -/** - * Checks for a callback method's return value, by wire primitive. A value - * of the wrong type throws, which reports the call as a code -4 failure. - */ -export const $ret = Object.freeze({ - Bool(v, where) { - if (typeof v !== 'boolean') throw new TypeError(`${where} must return a boolean`); - return v; - }, - I8: $num, - I16: $num, - I32: $num, - U8: $num, - U16: $num, - U32: $num, - F32: $num, - F64: $num, - I64: $retBigint, - U64: $retBigint, -}); - -/** - * The load-time contract check: the native library must implement ABI - * revision `abi`, and every top-level module's contract checksum must - * match the one these bindings were generated from. - */ -export function $verify(raw, library, prefix, abi, checksums) { - const version = raw[`${prefix}_abi_version`]; - if (typeof version !== 'function') { - throw new Error(`${library}: the native library does not export ${prefix}_abi_version`); - } - const found = Number(version()); - if (found !== abi) { - throw new Error( - `${library}: the native library implements ABI revision ${found}, but these bindings require revision ${abi}`, - ); - } - for (const [module, expected] of checksums) { - const checksum = raw[`${prefix}_${module}_checksum`]; - const got = typeof checksum === 'function' ? BigInt.asUintN(64, BigInt(checksum())) : null; - if (got !== expected) { - const what = got === null ? 'no contract checksum' : `contract checksum 0x${got.toString(16)}`; - throw new Error( - `${library}: module '${module}' of the native library does not match these bindings ` + - `(${what}, expected 0x${expected.toString(16)}); regenerate the bindings for this build`, - ); - } - } -} - -// END runtime.js diff --git a/crates/weaveffi-fuzz/Cargo.toml b/crates/weaveffi-fuzz/Cargo.toml index 431cf77d..4903b894 100644 --- a/crates/weaveffi-fuzz/Cargo.toml +++ b/crates/weaveffi-fuzz/Cargo.toml @@ -12,6 +12,7 @@ fuzzing = ["dep:libfuzzer-sys"] [dependencies] libfuzzer-sys = { version = "0.4", optional = true } +weaveffi = { workspace = true } weaveffi-model = { workspace = true, features = ["idl"] } [lib] @@ -53,6 +54,13 @@ test = false doc = false required-features = ["fuzzing"] +[[bin]] +name = "fuzz_value_buffer" +path = "fuzz_targets/value_buffer.rs" +test = false +doc = false +required-features = ["fuzzing"] + [lints.rust] unsafe_code = "deny" # `cargo-fuzz` sets `--cfg fuzzing` automatically when it builds a target. diff --git a/crates/weaveffi-fuzz/README.md b/crates/weaveffi-fuzz/README.md index ae2a6200..4be6bc36 100644 --- a/crates/weaveffi-fuzz/README.md +++ b/crates/weaveffi-fuzz/README.md @@ -1,6 +1,7 @@ # weaveffi-fuzz -Fuzz harnesses for WeaveFFI's parser and validator. Powered by +Fuzz harnesses for WeaveFFI's parsers, its validator, and the runtime's +value-buffer decoders. Powered by [cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz) and `libFuzzer`. This crate is not published to crates.io; it exists only to drive fuzzing @@ -14,11 +15,16 @@ locally and in CI. | `fuzz_parse_json` | `weaveffi_model::parse::parse_api_str(..., "json")` | | `fuzz_parse_toml` | `weaveffi_model::parse::parse_api_str(..., "toml")` | | `fuzz_parse_type_ref` | `weaveffi_model::ir::parse_type_ref` | -| `fuzz_validate` | parses YAML, then runs `weaveffi_model::validate::validate_api` on success | +| `fuzz_validate` | parses YAML, then runs `weaveffi_model::validate::validate` on success | +| `fuzz_value_buffer` | `weaveffi::abi::decode_value` over several value-buffer shapes; a successful decode must re-encode to the same bytes | Seed inputs for each target live in `fuzz/seeds//` and are committed -to git. `fuzz/corpus/`, `fuzz/artifacts/`, and `fuzz/coverage/` are generated -at runtime and gitignored. +to git. `seed_corpus.py ` copies them into a corpus directory +and adds inputs derived from the snapshot fixtures in +`crates/weaveffi-cli/tests/fixtures/` (as YAML, JSON, and TOML documents, +and every type string they use); it needs `pip install pyyaml tomli-w`. The +nightly CI run starts from that corpus. `corpus/`, `artifacts/`, and +`coverage/` are generated at runtime and gitignored. ## Prerequisites @@ -28,25 +34,27 @@ at runtime and gitignored. ## Run a target locally -From the workspace root: +From the workspace root, seed the target's corpus (libFuzzer adds new +inputs to it, so never fuzz the committed `fuzz/seeds/` directory itself), +then run it: ```bash +python3 crates/weaveffi-fuzz/seed_corpus.py fuzz_parse_yaml \ + crates/weaveffi-fuzz/corpus/fuzz_parse_yaml cargo +nightly fuzz run \ --fuzz-dir crates/weaveffi-fuzz \ --features fuzzing \ fuzz_parse_yaml \ - crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_yaml \ -- -max_total_time=60 -seed=1 ``` -Swap `fuzz_parse_yaml` for any of the targets above (and the matching seeds -directory) to fuzz a different entry point. Drop `-max_total_time=60` to fuzz -forever. +Swap `fuzz_parse_yaml` for any of the targets above to fuzz a different +entry point. Drop `-max_total_time=60` to fuzz forever. ## Triage a crash When a target finds an input that panics or aborts, libFuzzer writes the -offending bytes to `crates/weaveffi-fuzz/fuzz/artifacts//crash-`. +offending bytes to `crates/weaveffi-fuzz/artifacts//crash-`. To pretty-print the input as the fuzz target sees it: @@ -55,7 +63,7 @@ cargo +nightly fuzz fmt \ --fuzz-dir crates/weaveffi-fuzz \ --features fuzzing \ fuzz_parse_yaml \ - crates/weaveffi-fuzz/fuzz/artifacts/fuzz_parse_yaml/crash- + crates/weaveffi-fuzz/artifacts/fuzz_parse_yaml/crash- ``` To minimize the input to the smallest reproducer: @@ -65,12 +73,12 @@ cargo +nightly fuzz tmin \ --fuzz-dir crates/weaveffi-fuzz \ --features fuzzing \ fuzz_parse_yaml \ - crates/weaveffi-fuzz/fuzz/artifacts/fuzz_parse_yaml/crash- + crates/weaveffi-fuzz/artifacts/fuzz_parse_yaml/crash- ``` Then turn the minimized reproducer into a regression test in -`weaveffi-model` (the parsers and the validator both live there) before -fixing the bug. +`weaveffi-model` (for the parsers and the validator) or `weaveffi` (for the +value-buffer decoders) before fixing the bug. ## Why is this gated behind a feature? diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_json/minimal.json b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_json/minimal.json index 8091b8c1..9638515c 100644 --- a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_json/minimal.json +++ b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_json/minimal.json @@ -1,5 +1,5 @@ { - "version": "0.10.0", + "version": "0.11.0", "modules": [ { "name": "m", diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_toml/minimal.toml b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_toml/minimal.toml index 3a82d15a..ee785de0 100644 --- a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_toml/minimal.toml +++ b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_toml/minimal.toml @@ -1,4 +1,4 @@ -version = "0.10.0" +version = "0.11.0" [[modules]] name = "m" diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_yaml/minimal.yml b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_yaml/minimal.yml index 767ed103..04bd8781 100644 --- a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_yaml/minimal.yml +++ b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_parse_yaml/minimal.yml @@ -1,4 +1,4 @@ -version: "0.10.0" +version: "0.11.0" modules: - name: m functions: diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_validate/minimal.yml b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_validate/minimal.yml index df536087..58a9d9d6 100644 --- a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_validate/minimal.yml +++ b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_validate/minimal.yml @@ -1,4 +1,4 @@ -version: "0.10.0" +version: "0.11.0" modules: - name: m functions: diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/f64_list.bin b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/f64_list.bin new file mode 100644 index 0000000000000000000000000000000000000000..d9b5fe8d94dc291b60763f35e9fcbf31ed6fff6e GIT binary patch literal 21 QcmZQ!Vt|1k_E1&>01Ws6z5oCK literal 0 HcmV?d00001 diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/nested_bytes_list.bin b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/nested_bytes_list.bin new file mode 100644 index 0000000000000000000000000000000000000000..04642131d27c359e20a5a1dc4acf4ecfcfd39b6c GIT binary patch literal 17 TcmZQ)VqjokWCRjm1``7S0Mq~x literal 0 HcmV?d00001 diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/optional_string_list.bin b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/optional_string_list.bin new file mode 100644 index 0000000000000000000000000000000000000000..e620a0a386a1ede07662c8d33228a70c2b044370 GIT binary patch literal 12 RcmZQ$VqjokWCRk43;+Q+0BisN literal 0 HcmV?d00001 diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/string.bin b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/string.bin new file mode 100644 index 0000000000000000000000000000000000000000..b3487806f8f772ee0fe3c734e2100ff7bd168521 GIT binary patch literal 10 RcmZQ%Wnf^)NX^N~2LJ|W0viAT literal 0 HcmV?d00001 diff --git a/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/string_map.bin b/crates/weaveffi-fuzz/fuzz/seeds/fuzz_value_buffer/string_map.bin new file mode 100644 index 0000000000000000000000000000000000000000..4f936c4c57cd7baaf9f18af48afc5bcb0ba33589 GIT binary patch literal 20 Vcmd;JWME(b;cO-lgB?gP000H70Du4h literal 0 HcmV?d00001 diff --git a/crates/weaveffi-fuzz/fuzz_targets/fuzz_validate.rs b/crates/weaveffi-fuzz/fuzz_targets/fuzz_validate.rs index 3e1b50d4..b64687fa 100644 --- a/crates/weaveffi-fuzz/fuzz_targets/fuzz_validate.rs +++ b/crates/weaveffi-fuzz/fuzz_targets/fuzz_validate.rs @@ -6,7 +6,9 @@ use libfuzzer_sys::fuzz_target; #[cfg(fuzzing)] use weaveffi_model::parse::parse_api_str; #[cfg(fuzzing)] -use weaveffi_model::validate::validate_api; +use weaveffi_model::pkg::Identity; +#[cfg(fuzzing)] +use weaveffi_model::validate::validate; #[cfg(fuzzing)] fuzz_target!(|data: &[u8]| { @@ -16,7 +18,7 @@ fuzz_target!(|data: &[u8]| { let Ok(api) = parse_api_str(s, "yaml") else { return; }; - let _ = validate_api(api, None); + let _ = validate(&api, &Identity::default(), None); }); #[cfg(not(fuzzing))] diff --git a/crates/weaveffi-fuzz/fuzz_targets/value_buffer.rs b/crates/weaveffi-fuzz/fuzz_targets/value_buffer.rs new file mode 100644 index 00000000..d42ae9c5 --- /dev/null +++ b/crates/weaveffi-fuzz/fuzz_targets/value_buffer.rs @@ -0,0 +1,67 @@ +//! Fuzzes the runtime's value-buffer decoders (`weaveffi::abi::decode_value`) +//! with arbitrary bytes. The first byte picks a shape; the rest is the +//! buffer. A decode must either fail cleanly or produce a value that encodes +//! back to the same bytes (maps, which collapse duplicate keys and don't keep +//! wire order, must instead survive a second round trip unchanged). +//! +//! Interface tokens (`Arc`) are left out on purpose: decoding one adopts +//! the object it names, so arbitrary bytes there are undefined behavior by +//! contract, not a decoder bug. +#![cfg_attr(fuzzing, no_main)] +#![allow(unsafe_code)] + +#[cfg(fuzzing)] +use std::collections::{BTreeMap, HashMap}; + +#[cfg(fuzzing)] +use libfuzzer_sys::fuzz_target; +#[cfg(fuzzing)] +use weaveffi::abi::{encode_value, BufferDecodeError, BufferValue}; + +/// Decode a buffer of one of the fuzzed types, none of which carries an +/// object token, so no input can make the decode unsound. +fn decode_value(data: &[u8]) -> Result { + // SAFETY: the fuzzed types contain no interfaces, so no token is adopted. + unsafe { weaveffi::abi::decode_value(data) } +} + +/// Decodes `data` as `T`; on success, re-encodes it and checks the bytes +/// match. +#[cfg(fuzzing)] +fn exact(data: &[u8]) { + if let Ok(value) = decode_value::(data) { + assert_eq!(encode_value(&value), data, "re-encoding changed the bytes"); + } +} + +/// Decodes `data` as `T`; on success, checks that the value survives an +/// encode and decode unchanged. +#[cfg(fuzzing)] +fn stable(data: &[u8]) { + if let Ok(value) = decode_value::(data) { + let again: T = decode_value(&encode_value(&value)).expect("re-encoded value decodes"); + assert_eq!(again, value); + } +} + +#[cfg(fuzzing)] +fuzz_target!(|data: &[u8]| { + let Some((&shape, buf)) = data.split_first() else { + return; + }; + match shape % 10 { + 0 => exact::(buf), + 1 => exact::(buf), + 2 => exact::>(buf), + 3 => exact::>(buf), + 4 => exact::>(buf), + 5 => exact::>(buf), + 6 => exact::>>(buf), + 7 => exact::>>>>(buf), + 8 => stable::>>>(buf), + _ => stable::>>(buf), + } +}); + +#[cfg(not(fuzzing))] +fn main() {} diff --git a/crates/weaveffi-fuzz/seed_corpus.py b/crates/weaveffi-fuzz/seed_corpus.py new file mode 100644 index 00000000..7f2e3abc --- /dev/null +++ b/crates/weaveffi-fuzz/seed_corpus.py @@ -0,0 +1,90 @@ +#!/usr/bin/env python3 +"""Seed a fuzz target's corpus from its committed seeds and the snapshot fixtures. + +Usage: seed_corpus.py + +Copies crates/weaveffi-fuzz/fuzz/seeds// into , then adds +inputs derived from crates/weaveffi-cli/tests/fixtures/*.yml, which between +them cover the whole IDL surface: + +- fuzz_parse_yaml, fuzz_validate: each fixture as is +- fuzz_parse_json: each fixture converted to JSON +- fuzz_parse_toml: each fixture converted to TOML (needs `tomli-w`) +- fuzz_parse_type_ref: every distinct `type` and `return` string +- fuzz_value_buffer: the committed seeds only + +Needs PyYAML (`pip install pyyaml tomli-w`). +""" + +import json +import pathlib +import shutil +import sys + +import yaml + +ROOT = pathlib.Path(__file__).resolve().parents[2] +SEEDS = ROOT / "crates/weaveffi-fuzz/fuzz/seeds" +FIXTURES = sorted((ROOT / "crates/weaveffi-cli/tests/fixtures").glob("*.yml")) + + +def type_strings(node, out): + """Collect every string under a `type` or `return` key.""" + if isinstance(node, dict): + for key, value in node.items(): + if key in ("type", "return") and isinstance(value, str): + out.add(value) + type_strings(value, out) + elif isinstance(node, list): + for item in node: + type_strings(item, out) + + +class FixtureLoader(yaml.SafeLoader): + """Reads a plain `null` as the string "null", as the IDL parser does where + it expects a string (`edge_cases` has a parameter named `null`). The + fixtures use no real nulls, which TOML couldn't express anyway.""" + + +FixtureLoader.yaml_implicit_resolvers = { + first: [(tag, regex) for tag, regex in resolvers if tag != "tag:yaml.org,2002:null"] + for first, resolvers in yaml.SafeLoader.yaml_implicit_resolvers.items() +} + + +def load(fixture): + return yaml.load(fixture.read_text(), Loader=FixtureLoader) + + +def main(): + target, corpus = sys.argv[1], pathlib.Path(sys.argv[2]) + corpus.mkdir(parents=True, exist_ok=True) + for seed in (SEEDS / target).iterdir(): + shutil.copy(seed, corpus / seed.name) + + if target in ("fuzz_parse_yaml", "fuzz_validate"): + for fixture in FIXTURES: + shutil.copy(fixture, corpus / fixture.name) + elif target == "fuzz_parse_json": + for fixture in FIXTURES: + text = json.dumps(load(fixture), indent=2) + (corpus / f"{fixture.stem}.json").write_text(text) + elif target == "fuzz_parse_toml": + import tomli_w + + for fixture in FIXTURES: + (corpus / f"{fixture.stem}.toml").write_text(tomli_w.dumps(load(fixture))) + elif target == "fuzz_parse_type_ref": + types = set() + for fixture in FIXTURES: + type_strings(load(fixture), types) + for i, ty in enumerate(sorted(types)): + (corpus / f"fixture_type_{i:03}.txt").write_text(ty) + elif target != "fuzz_value_buffer": + sys.exit(f"unknown fuzz target: {target}") + + print(f"{target}: {len(list(corpus.iterdir()))} corpus inputs") + + +if __name__ == "__main__": + main() diff --git a/crates/weaveffi-fuzz/tests/seeds.rs b/crates/weaveffi-fuzz/tests/seeds.rs index 6c4ad984..76b11bbc 100644 --- a/crates/weaveffi-fuzz/tests/seeds.rs +++ b/crates/weaveffi-fuzz/tests/seeds.rs @@ -4,16 +4,28 @@ //! 1. every committed seed is well-formed for its target's parser, and //! 2. each fuzz target's underlying call pattern (the body of the //! `fuzz_target!` macro invocation) keeps compiling against the upstream -//! APIs in `weaveffi-model`. +//! APIs in `weaveffi-model` and `weaveffi::abi`. //! //! If a parser or validator signature changes, this file is what should //! break first, long before a nightly fuzz run. +#![allow(unsafe_code)] + +use std::collections::{BTreeMap, HashMap}; use std::path::PathBuf; +use weaveffi::abi::{encode_value, BufferDecodeError, BufferValue}; + +/// Decode a buffer of one of the fuzzed types, none of which carries an +/// object token, so no input can make the decode unsound. +fn decode_value(data: &[u8]) -> Result { + // SAFETY: the fuzzed types contain no interfaces, so no token is adopted. + unsafe { weaveffi::abi::decode_value(data) } +} use weaveffi_model::ir::parse_type_ref; use weaveffi_model::parse::parse_api_str; -use weaveffi_model::validate::validate_api; +use weaveffi_model::pkg::Identity; +use weaveffi_model::validate::validate; fn seed(target: &str, name: &str) -> String { let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")) @@ -56,7 +68,7 @@ fn parse_type_ref_seed_is_well_formed() { fn validate_seed_passes_validation() { let s = seed("fuzz_validate", "minimal.yml"); let api = parse_api_str(&s, "yaml").expect("seed must parse as YAML"); - validate_api(api, None).expect("seed must pass validation"); + validate(&api, &Identity::default(), None).expect("seed must pass validation"); } /// Mirrors the body of `fuzz_target!` in `parse_yaml.rs`: arbitrary bytes must @@ -102,3 +114,84 @@ fn validate_target_skips_unparseable_input() { let bad = "not: [valid"; assert!(parse_api_str(bad, "yaml").is_err()); } + +fn seed_bytes(target: &str, name: &str) -> Vec { + let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("fuzz") + .join("seeds") + .join(target) + .join(name); + std::fs::read(&path).unwrap_or_else(|e| panic!("failed to read seed {}: {e}", path.display())) +} + +/// Mirrors `exact` in `value_buffer.rs`: a successful decode re-encodes to +/// the same bytes. Returns whether the decode succeeded. +fn exact(data: &[u8]) -> bool { + let Ok(value) = decode_value::(data) else { + return false; + }; + assert_eq!(encode_value(&value), data, "re-encoding changed the bytes"); + true +} + +/// Mirrors `stable` in `value_buffer.rs`: a successful decode survives an +/// encode and decode unchanged. Returns whether the decode succeeded. +fn stable(data: &[u8]) -> bool { + let Ok(value) = decode_value::(data) else { + return false; + }; + let again: T = decode_value(&encode_value(&value)).expect("re-encoded value decodes"); + assert_eq!(again, value); + true +} + +/// Mirrors the body of `fuzz_target!` in `value_buffer.rs`. +fn value_buffer_target(data: &[u8]) -> bool { + let Some((&shape, buf)) = data.split_first() else { + return false; + }; + match shape % 10 { + 0 => exact::(buf), + 1 => exact::(buf), + 2 => exact::>(buf), + 3 => exact::>(buf), + 4 => exact::>(buf), + 5 => exact::>(buf), + 6 => exact::>>(buf), + 7 => exact::>>>>(buf), + 8 => stable::>>>(buf), + _ => stable::>>(buf), + } +} + +#[test] +fn value_buffer_seeds_decode() { + for name in [ + "string.bin", + "f64_list.bin", + "optional_string_list.bin", + "nested_bytes_list.bin", + "string_map.bin", + ] { + let data = seed_bytes("fuzz_value_buffer", name); + assert!(value_buffer_target(&data), "seed {name} must decode"); + } +} + +/// Malformed buffers for several shapes: an empty input, a bool byte out of +/// range, invalid UTF-8, a truncated list, a huge fixed-width count, and a +/// huge map count. Each must fail to decode without panicking. +#[test] +fn value_buffer_target_rejects_garbage() { + let garbage: [&[u8]; 6] = [ + b"", + b"\x00\x02", + b"\x01\x02\x00\x00\x00\xff\xfe", + b"\x03\x02\x00\x00\x00\x01", + b"\x04\xff\xff\xff\xff", + b"\x09\xff\xff\xff\x7f\x00", + ]; + for data in garbage { + assert!(!value_buffer_target(data), "{data:?} must not decode"); + } +} diff --git a/crates/weaveffi-gen/Cargo.toml b/crates/weaveffi-gen/Cargo.toml deleted file mode 100644 index 2579d4b8..00000000 --- a/crates/weaveffi-gen/Cargo.toml +++ /dev/null @@ -1,33 +0,0 @@ -[package] -name = "weaveffi-gen" -version.workspace = true -edition.workspace = true -rust-version.workspace = true -license.workspace = true -authors.workspace = true -repository.workspace = true -description = "Code-generation orchestrator and the eleven language generators for WeaveFFI" -keywords = ["ffi", "codegen", "bindings", "code-generation", "generator"] -categories = ["development-tools::ffi"] - -[dependencies] -weaveffi-model = { workspace = true, features = ["idl"] } -anyhow = { workspace = true } -thiserror = { workspace = true } -camino = { workspace = true } -serde = { workspace = true } -serde_json = { workspace = true } -sha2 = { workspace = true } -rayon = { workspace = true } -heck = { workspace = true } - -[dev-dependencies] -tempfile = "3" -criterion = { workspace = true } - -[[bench]] -name = "codegen_bench" -harness = false - -[lints] -workspace = true diff --git a/crates/weaveffi-gen/src/backend/mod.rs b/crates/weaveffi-gen/src/backend/mod.rs deleted file mode 100644 index 7527f9bf..00000000 --- a/crates/weaveffi-gen/src/backend/mod.rs +++ /dev/null @@ -1,379 +0,0 @@ -//! The language-backend framework. -//! -//! Every idiomatic WeaveFFI generator does the same three things: it walks the -//! [`BindingModel`] in a fixed order (errors, enums, structs, callback -//! interfaces, interfaces, functions), dispatches each function on its -//! [`CallShape`], and writes a primary source file plus a handful of package -//! manifests. -//! -//! [`LanguageBackend`] is the one trait a target implements. It captures the -//! common structure as hooks; [`ConfiguredBackend`](crate::codegen::ConfiguredBackend) -//! builds the model and exposes the backend to the orchestrator through the -//! object-safe [`Target`](crate::codegen::Target) trait. A backend owns *only* language-specific rendering: type mapping, -//! marshalling, and the exact text of each declaration. The traversal order, -//! the call-shape dispatch, the model construction, and the erasure all live -//! here, once. -//! -//! [`BindingModel`]: weaveffi_model::model::BindingModel -//! [`CallShape`]: weaveffi_model::model::CallShape - -use camino::{Utf8Path, Utf8PathBuf}; -use serde::Serialize; - -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; -use weaveffi_model::model::{ - BindingModel, CallbackInterfaceBinding, EnumBinding, ErrorBinding, FnBinding, InterfaceBinding, - ModuleBinding, StructBinding, -}; -use weaveffi_model::resolved::ResolvedApi; - -/// A single generated file: its full path (under the output directory) and the -/// rendered contents. Backends return these from [`LanguageBackend::files`]; -/// the driver creates parent directories and writes them. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct OutputFile { - /// Full path to write, under (or anchored at) the output directory. - pub path: Utf8PathBuf, - /// The rendered file contents. - pub contents: String, -} - -impl OutputFile { - /// Pair a destination path with its rendered contents. - /// - /// The path is normalized to `/` separators, which every platform's file - /// APIs accept, so listings, cache records, and tests see the same path - /// on Windows as elsewhere. - pub fn new(path: impl Into, contents: impl Into) -> Self { - let path: Utf8PathBuf = path.into(); - let path = if path.as_str().contains('\\') { - Utf8PathBuf::from(path.as_str().replace('\\', "/")) - } else { - path - }; - Self { - path, - contents: contents.into(), - } - } -} - -/// An idiomatic language backend over the shared [`BindingModel`]. -/// -/// The required methods are [`name`](Self::name), -/// [`capabilities`](Self::capabilities), and [`files`](Self::files), which -/// assembles the complete output set; wrap the type in -/// [`ConfiguredBackend`](crate::codegen::ConfiguredBackend) to hand it to the -/// orchestrator. That alone gives every backend the shared driver and the -/// [`OutputFile`] model (rendering is pure; the orchestrator does the I/O). -/// -/// Backends whose primary file is a straightforward per-module walk (Python, -/// Ruby, Go, Dart, .NET) override the per-entity hooks (`render_enum`, -/// `render_struct`, `render_callback_interface`, `render_interface`, -/// `render_function`) and call the provided -/// [`emit_members`](Self::emit_members) from inside their module scoping; that -/// is what removes the hand-rolled walk + call-shape dispatch each generator -/// used to carry. Backends whose output is not one linear pass leave the hooks -/// at their no-op defaults and build their layout directly in -/// [`files`](Self::files): C and C++ order declarations by dependency, Swift -/// splits types from the namespaced module body, and Kotlin, Node, and Wasm -/// each render parallel files (Kotlin + JNI C, addon C + JS, JS + `.d.ts`) -/// that would each need their own walk anyway. -/// -/// Each hook renders into a `String` (matching how generators accumulate -/// output) and is responsible for emitting its own doc comments; doc-comment -/// shape varies too much between targets (docstrings, `///`, KDoc, ``) -/// to centralise here, but every backend shares -/// [`emit_doc`](crate::codegen::common::emit_doc) for the line/block flavours. -pub trait LanguageBackend: Send + Sync { - /// Per-target, fully-typed configuration. Must round-trip through - /// `serde_json` so the orchestrator can hash it as part of the cache key. - type Config: Serialize + Default + Clone + Send + Sync; - - /// Stable short name (`"swift"`, `"python"`, ...): the `--target` token - /// and the cache file basename. - fn name(&self) -> &'static str; - - /// The gated IDL features this backend implements (async functions, - /// callback interfaces, iterators) under `config`. Required: declaring - /// capabilities explicitly is what lets the orchestrator fail loudly - /// instead of a backend silently skipping a feature it never implemented. - /// The config is passed because some backends have modes with different - /// ceilings (the Wasm backend's Emscripten mode, for example). - fn capabilities(&self, config: &Self::Config) -> TargetCapabilities; - - /// Whether the user explicitly opted in to generating this target even - /// though the API uses features the target does not support (via an - /// `allow_unsupported = true` flag in the target's config). When `true` - /// the orchestrator downgrades the capability failure to a loud warning - /// and the backend must emit an explicit unsupported surface (throwing - /// stubs, documentation) rather than silently omitting the feature. - /// Backends with partial capabilities override this to read their - /// `allow_unsupported` config flag; full-capability backends keep the - /// `false` default. - fn allows_unsupported(&self, config: &Self::Config) -> bool { - let _ = config; - false - } - - /// Render one enum (its declaration and any helpers), including doc - /// comments. Override when using [`emit_members`](Self::emit_members). - fn render_enum(&self, out: &mut String, e: &EnumBinding, config: &Self::Config) { - let _ = (out, e, config); - } - - /// Render one struct: the wrapper type, its getters, lifecycle, and the - /// optional builder. `module` is the owning module (for symbol paths). - /// Override when using [`emit_members`](Self::emit_members). - fn render_struct( - &self, - out: &mut String, - module: &ModuleBinding, - s: &StructBinding, - config: &Self::Config, - ) { - let _ = (out, module, s, config); - } - - /// Render the typed error surface for a module that *declares* an error - /// domain: the target's error enum/class hierarchy mapping each - /// [`ErrorBinding`] code to a case the consumer can match on. Override when - /// using [`emit_members`](Self::emit_members); inheriting modules reference - /// the ancestor's type, so this hook only fires where - /// [`ModuleBinding::declares_error`] is true. - fn render_error( - &self, - out: &mut String, - module: &ModuleBinding, - e: &ErrorBinding, - config: &Self::Config, - ) { - let _ = (out, module, e, config); - } - - /// Render one callback interface: the protocol, abstract class, or - /// interface type the consumer implements, plus the vtable trampolines - /// that adapt an implementation to the C ABI. Override when using - /// [`emit_members`](Self::emit_members). - fn render_callback_interface( - &self, - out: &mut String, - module: &ModuleBinding, - cb: &CallbackInterfaceBinding, - config: &Self::Config, - ) { - let _ = (out, module, cb, config); - } - - /// Render one interface: the wrapper class with its constructors, methods, - /// statics, and reference-count wiring. Override when using - /// [`emit_members`](Self::emit_members). - fn render_interface( - &self, - out: &mut String, - module: &ModuleBinding, - i: &InterfaceBinding, - config: &Self::Config, - ) { - let _ = (out, module, i, config); - } - - /// Render one function. Implementations match on `f.shape` (sync / async / - /// iterator) and emit the idiomatic wrapper plus its doc comment. Override - /// when using [`emit_members`](Self::emit_members). - fn render_function( - &self, - out: &mut String, - module: &ModuleBinding, - f: &FnBinding, - config: &Self::Config, - ) { - let _ = (out, module, f, config); - } - - /// Emit every member of `module` in canonical order (error domain, enums, - /// structs, callback interfaces, interfaces, functions). Callback - /// interfaces precede interfaces because interface members take them as - /// parameters. Backends call this from within their own module scoping; - /// overriding the per-entity hooks is what guarantees a single-pass - /// backend cannot silently skip an entity kind. - fn emit_members(&self, out: &mut String, module: &ModuleBinding, config: &Self::Config) { - if let Some(e) = module.error.as_ref().filter(|e| e.declared_here) { - self.render_error(out, module, e, config); - } - for e in &module.enums { - self.render_enum(out, e, config); - } - for s in &module.structs { - self.render_struct(out, module, s, config); - } - for cb in &module.callback_interfaces { - self.render_callback_interface(out, module, cb, config); - } - for i in &module.interfaces { - self.render_interface(out, module, i, config); - } - for f in &module.functions { - self.render_function(out, module, f, config); - } - } - - /// Assemble the complete output set. The driver has already built `model` - /// (via [`BindingModel::build`], with the API identity's prefix) and passes - /// the resolved `api` too, for the rare file (e.g. a `.pyi` stub) that - /// needs the full resolved tree. Most backends render a primary source - /// file by composing [`emit_members`](Self::emit_members) over - /// `model.modules`, then append package manifests (`package.json`, - /// `pyproject.toml`, `go.mod`, …) as additional [`OutputFile`]s. - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec; - - /// Assemble a distributable package that bundles a prebuilt native library - /// for each platform in `ctx.binaries`, returning `None` when this target - /// does not support packaging yet. - /// - /// This is the `weaveffi package` analogue of [`files`](Self::files): it - /// returns [`PackagedFile`]s (rendered manifests, loaders, and binding - /// source as [`FileContent::Text`](crate::package::FileContent::Text), plus - /// the bundled libraries as - /// [`FileContent::Copy`](crate::package::FileContent::Copy)) anchored under - /// `out_dir`, and the [`write_package`](crate::package::write_package) - /// driver does the I/O. Override this to emit the ecosystem's idiomatic - /// per-platform layout (npm `optionalDependencies`, a NuGet `runtimes/` - /// tree, platform-tagged Python wheels, …). The default returns `None`. - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let _ = (api, model, ctx, out_dir, config); - None - } -} - -#[cfg(test)] -mod tests { - use super::*; - use weaveffi_model::ir::{Api, Function, Module, Param, TypeRef}; - use weaveffi_model::model::CallShape; - - #[derive(Default, Clone, serde::Serialize)] - struct FakeConfig; - - /// A trivial backend that records the canonical traversal order so we can - /// assert the driver walks and dispatches correctly. - struct FakeBackend; - - impl LanguageBackend for FakeBackend { - type Config = FakeConfig; - - fn name(&self) -> &'static str { - "fake" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn render_function( - &self, - out: &mut String, - _m: &ModuleBinding, - f: &FnBinding, - _c: &Self::Config, - ) { - let shape = match &f.shape { - CallShape::Sync(_) => "sync", - CallShape::Async(_) => "async", - CallShape::Iterator(_) => "iter", - }; - out.push_str(&format!("fn {} [{}] {}\n", f.name, shape, f.c_base)); - } - - fn files( - &self, - _api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let mut out = String::new(); - for m in &model.modules { - out.push_str(&format!("module {}\n", m.path)); - self.emit_members(&mut out, m, config); - } - vec![OutputFile::new(out_dir.join("fake/out.txt"), out)] - } - } - - fn func(name: &str, returns: Option, is_async: bool) -> Function { - Function { - name: name.into(), - params: vec![Param { - name: "x".into(), - ty: TypeRef::I32, - doc: None, - }], - returns, - doc: None, - throws: false, - r#async: is_async, - cancellable: false, - deprecated: None, - } - } - - fn api() -> ResolvedApi { - ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![Module { - name: "math".into(), - doc: None, - functions: vec![ - func("add", Some(TypeRef::I32), false), - func("fetch", Some(TypeRef::StringUtf8), true), - func( - "scan", - Some(TypeRef::Iterator(Box::new(TypeRef::I32))), - false, - ), - ], - interfaces: vec![], - structs: vec![], - enums: vec![], - callback_interfaces: vec![], - errors: None, - modules: vec![], - }], - }) - } - - #[test] - fn driver_walks_dispatches_and_honors_the_identity_prefix() { - use crate::codegen::{ConfiguredBackend, Target}; - let out_dir = Utf8Path::new("out"); - let render = |n: &str| { - let api = api().with_identity(weaveffi_model::pkg::Identity::named(n)); - let files = ConfiguredBackend::new(FakeBackend, FakeConfig).render(&api, out_dir); - assert_eq!(files.len(), 1); - assert_eq!(files[0].path, out_dir.join("fake/out.txt")); - files[0].contents.clone() - }; - assert_eq!( - render("weaveffi"), - "module math\nfn add [sync] weaveffi_math_add\nfn fetch [async] weaveffi_math_fetch\nfn scan [iter] weaveffi_math_scan\n" - ); - let body = render("acme"); - assert!(body.contains("acme_math_add"), "{body}"); - assert!(!body.contains("weaveffi_math_add")); - } -} diff --git a/crates/weaveffi-gen/src/cache.rs b/crates/weaveffi-gen/src/cache.rs deleted file mode 100644 index d53165b8..00000000 --- a/crates/weaveffi-gen/src/cache.rs +++ /dev/null @@ -1,213 +0,0 @@ -//! Per-target generation records for skip-if-unchanged builds and stale-file -//! cleanup. -//! -//! After generating a target, the orchestrator writes -//! `{out_dir}/.weaveffi-cache/{target}.json`: a hash of every input that -//! affects the target's output (the canonical IR, the library identity, the -//! target name, its serialized config, and the CLI version) plus the path and -//! content hash of every file it wrote. A later run skips the target only -//! when the input hash matches *and* every recorded file is still on disk -//! unmodified, so deleting or hand-editing a generated file regenerates it. -//! Files recorded by the previous run that the current run no longer -//! produces are removed, so a renamed type never leaves a stale file behind, -//! while files the user added (a `node_modules/`, a build directory) are -//! never touched. - -use std::collections::BTreeMap; - -use anyhow::{Context, Result}; -use camino::{Utf8Path, Utf8PathBuf}; -use serde::{Deserialize, Serialize}; -use sha2::{Digest, Sha256}; -use weaveffi_model::ir::Api; -use weaveffi_model::resolved::ResolvedApi; - -const CACHE_DIR: &str = ".weaveffi-cache"; - -/// Version string folded into every input hash, so upgrading the CLI -/// regenerates everything. -pub const CLI_VERSION: &str = env!("CARGO_PKG_VERSION"); - -/// What one generation of one target produced. -#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] -pub struct Record { - /// [`hash_generator_inputs`] for the run that produced the files. - pub inputs: String, - /// Every file written, relative to the output directory (with `/` - /// separators), mapped to the SHA-256 of its contents. - pub files: BTreeMap, -} - -impl Record { - /// Whether every recorded file still exists under `out_dir` with the - /// recorded contents. - #[must_use] - pub fn files_intact(&self, out_dir: &Utf8Path) -> bool { - self.files.iter().all(|(rel, hash)| { - std::fs::read(out_dir.join(rel).as_std_path()) - .is_ok_and(|bytes| &hash_bytes(&bytes) == hash) - }) - } -} - -/// The SHA-256 hex digest of `bytes`. -#[must_use] -pub fn hash_bytes(bytes: &[u8]) -> String { - format!("{:x}", Sha256::digest(bytes)) -} - -/// Serialize `api` to canonical JSON (keys in lexicographic order, so two -/// runs over the same IR always agree). -fn canonical_json(api: &Api) -> String { - let value = serde_json::to_value(api).expect("Api serialization should not fail"); - serde_json::to_string(&value).expect("Value serialization should not fail") -} - -/// The SHA-256 hex digest of the API's canonical JSON. -/// -/// # Panics -/// -/// Panics if the API cannot be serialized to JSON, which does not happen for -/// well-formed inputs. -#[must_use] -pub fn hash_api(api: &Api) -> String { - hash_bytes(canonical_json(api).as_bytes()) -} - -/// The SHA-256 hex digest of every input that affects a single target's -/// output: the canonical IR, the library identity, the target's name, the -/// target's serialized config, and the CLI version. -/// -/// # Panics -/// -/// Panics if the API or identity cannot be serialized to JSON, which does not -/// happen for well-formed inputs. -#[must_use] -pub fn hash_generator_inputs(api: &ResolvedApi, target: &str, config_bytes: &[u8]) -> String { - let identity = - serde_json::to_string(api.identity()).expect("identity serialization should not fail"); - let mut hasher = Sha256::new(); - for part in [ - b"v3".as_slice(), - CLI_VERSION.as_bytes(), - target.as_bytes(), - canonical_json(api.api()).as_bytes(), - identity.as_bytes(), - config_bytes, - ] { - hasher.update(part); - hasher.update(b"\0"); - } - format!("{:x}", hasher.finalize()) -} - -fn record_path(out_dir: &Utf8Path, target: &str) -> Utf8PathBuf { - out_dir.join(CACHE_DIR).join(format!("{target}.json")) -} - -/// Read the record of the last generation of `target` under `out_dir`, if -/// any (a missing or unreadable record means "never generated"). -#[must_use] -pub fn read_record(out_dir: &Utf8Path, target: &str) -> Option { - let text = std::fs::read_to_string(record_path(out_dir, target)).ok()?; - serde_json::from_str(&text).ok() -} - -/// Persist the record of a generation of `target`. -/// -/// # Errors -/// -/// Returns an error when the cache directory cannot be created or the record -/// cannot be written. -/// -/// # Panics -/// -/// Panics if the record cannot be serialized, which does not happen. -pub fn write_record(out_dir: &Utf8Path, target: &str, record: &Record) -> Result<()> { - let path = record_path(out_dir, target); - let dir = out_dir.join(CACHE_DIR); - std::fs::create_dir_all(dir.as_std_path()) - .with_context(|| format!("failed to create cache directory: {dir}"))?; - let json = serde_json::to_string_pretty(record).expect("record serializes"); - std::fs::write(path.as_std_path(), json) - .with_context(|| format!("failed to write cache file: {path}")) -} - -/// Forget every target's record, so the next run regenerates everything. -/// -/// # Errors -/// -/// Returns an error if the cache directory exists but cannot be removed. -pub fn invalidate_all(out_dir: &Utf8Path) -> Result<()> { - let cache_dir = out_dir.join(CACHE_DIR); - if cache_dir.exists() { - std::fs::remove_dir_all(cache_dir.as_std_path()) - .with_context(|| format!("failed to remove cache directory: {cache_dir}"))?; - } - Ok(()) -} - -#[cfg(test)] -mod tests { - use super::*; - use weaveffi_model::ir::{Module, CURRENT_SCHEMA_VERSION}; - - fn api(name: &str) -> Api { - Api { - version: CURRENT_SCHEMA_VERSION.into(), - modules: vec![Module { - name: name.into(), - doc: None, - functions: vec![], - interfaces: vec![], - callback_interfaces: vec![], - structs: vec![], - enums: vec![], - errors: None, - modules: vec![], - }], - } - } - - #[test] - fn input_hash_covers_api_target_config_and_identity() { - let r = ResolvedApi::assume_valid(api("math")); - let base = hash_generator_inputs(&r, "swift", b"{}"); - assert_eq!(base, hash_generator_inputs(&r, "swift", b"{}")); - assert_ne!(base, hash_generator_inputs(&r, "c", b"{}")); - assert_ne!(base, hash_generator_inputs(&r, "swift", b"{\"x\":1}")); - assert_ne!( - base, - hash_generator_inputs(&ResolvedApi::assume_valid(api("other")), "swift", b"{}") - ); - let renamed = r - .clone() - .with_identity(weaveffi_model::pkg::Identity::named("kv")); - assert_ne!(base, hash_generator_inputs(&renamed, "swift", b"{}")); - assert_eq!(hash_api(&api("math")), hash_api(&api("math"))); - } - - #[test] - fn records_round_trip_and_detect_tampering() { - let dir = tempfile::tempdir().unwrap(); - let out = Utf8Path::from_path(dir.path()).unwrap(); - assert_eq!(read_record(out, "swift"), None); - std::fs::create_dir_all(out.join("swift")).unwrap(); - std::fs::write(out.join("swift/a.swift"), "a").unwrap(); - let record = Record { - inputs: "abc".into(), - files: [("swift/a.swift".to_string(), hash_bytes(b"a"))].into(), - }; - write_record(out, "swift", &record).unwrap(); - let back = read_record(out, "swift").unwrap(); - assert_eq!(back, record); - assert!(back.files_intact(out)); - std::fs::write(out.join("swift/a.swift"), "edited").unwrap(); - assert!(!back.files_intact(out)); - std::fs::remove_file(out.join("swift/a.swift")).unwrap(); - assert!(!back.files_intact(out)); - invalidate_all(out).unwrap(); - assert_eq!(read_record(out, "swift"), None); - invalidate_all(out).unwrap(); - } -} diff --git a/crates/weaveffi-gen/src/capabilities.rs b/crates/weaveffi-gen/src/capabilities.rs deleted file mode 100644 index bd1a23fc..00000000 --- a/crates/weaveffi-gen/src/capabilities.rs +++ /dev/null @@ -1,333 +0,0 @@ -//! Per-target **feature capability declarations** and the loud-failure -//! contract that replaces silent feature skipping. -//! -//! A backend that does not implement an IDL feature must not simply omit it -//! from its output with nothing telling the user. Every generator declares a -//! [`TargetCapabilities`] and the orchestrator refuses to run a generator -//! against an API that uses a feature the target does not support, listing -//! each offending declaration by path. -//! -//! A backend that gains a feature flips the corresponding flag and the gate -//! opens; a backend that loses one (or a new feature lands in the IR before -//! every backend implements it) fails generation with an actionable error -//! instead of producing incomplete bindings. - -use std::collections::BTreeMap; -use std::fmt; - -use weaveffi_model::ir::{Api, Module, TypeRef}; - -/// An IDL feature whose support varies (or could vary) per target. -/// -/// Core types (scalars, strings, bytes, structs, enums, interfaces, -/// optionals, lists, maps) are mandatory for every backend and are not gated. -#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] -pub enum Feature { - /// `async: true` functions (callback-completed launchers). - AsyncFunctions, - /// Callback interfaces (`callback_interfaces:`): consumer-implemented - /// vtables the producer calls into. - CallbackInterfaces, - /// `iter` returns (opaque iterator handle + `next`/`destroy`). - Iterators, -} - -impl Feature { - /// Every gated feature, for exhaustive iteration in checks and tests. - pub const ALL: [Feature; 3] = [ - Feature::AsyncFunctions, - Feature::CallbackInterfaces, - Feature::Iterators, - ]; - - /// The IDL-facing name used in error messages. - pub fn idl_name(&self) -> &'static str { - match self { - Feature::AsyncFunctions => "async functions", - Feature::CallbackInterfaces => "callback interfaces", - Feature::Iterators => "iterator returns (iter)", - } - } -} - -impl fmt::Display for Feature { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - f.write_str(self.idl_name()) - } -} - -/// The feature set a backend implements. Declared by every backend via -/// [`LanguageBackend::capabilities`](crate::backend::LanguageBackend::capabilities). -/// -/// There is intentionally no `Default` impl: a backend must state what it -/// supports explicitly so a new gated feature cannot be claimed by omission. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct TargetCapabilities { - /// Whether the target generates `async: true` functions. - pub async_functions: bool, - /// Whether the target generates callback-interface implementations. - pub callback_interfaces: bool, - /// Whether the target generates `iter` returns. - pub iterators: bool, -} - -impl TargetCapabilities { - /// Full support for every gated feature. Every shipped WeaveFFI backend - /// declares this; partial sets exist for backends under development. - pub const fn full() -> Self { - Self { - async_functions: true, - callback_interfaces: true, - iterators: true, - } - } - - /// Whether this set includes `feature`. - pub const fn supports(&self, feature: Feature) -> bool { - match feature { - Feature::AsyncFunctions => self.async_functions, - Feature::CallbackInterfaces => self.callback_interfaces, - Feature::Iterators => self.iterators, - } - } -} - -/// Every gated feature `api` uses, mapped to the locations (dotted IDL paths) -/// that use it. Deterministic ordering for stable error output. -pub fn used_features(api: &Api) -> BTreeMap> { - let mut used: BTreeMap> = BTreeMap::new(); - for module in &api.modules { - collect_module(module, "", &mut used); - } - used -} - -fn collect_module(module: &Module, parent: &str, used: &mut BTreeMap>) { - let path = if parent.is_empty() { - module.name.clone() - } else { - format!("{parent}.{}", module.name) - }; - for cb in &module.callback_interfaces { - used.entry(Feature::CallbackInterfaces) - .or_default() - .push(format!("{path}.{}", cb.name)); - } - let members = module.interfaces.iter().flat_map(|i| { - i.constructors - .iter() - .chain(&i.methods) - .chain(&i.statics) - .map(move |f| (format!("{}.{}", i.name, f.name), f)) - }); - for (name, f) in module - .functions - .iter() - .map(|f| (f.name.clone(), f)) - .chain(members) - { - let loc = format!("{path}.{name}"); - if f.r#async { - used.entry(Feature::AsyncFunctions) - .or_default() - .push(loc.clone()); - } - if matches!(f.returns, Some(TypeRef::Iterator(_))) { - used.entry(Feature::Iterators).or_default().push(loc); - } - } - for child in &module.modules { - collect_module(child, &path, used); - } -} - -/// A target was asked to generate bindings for an API that uses features it -/// does not support. Carries every violation so the user sees the complete -/// picture in one failure. -#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] -pub struct UnsupportedFeatures { - /// The `--target` token of the failing generator. - pub target: String, - /// Each unsupported feature with the IDL paths that use it. - pub violations: Vec<(Feature, Vec)>, -} - -impl fmt::Display for UnsupportedFeatures { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - writeln!( - f, - "target '{}' does not support every feature this IDL uses:", - self.target - )?; - for (feature, locations) in &self.violations { - writeln!(f, " - {feature} (used by: {})", locations.join(", "))?; - } - write!( - f, - "remove the unsupported declarations, drop '{}' from --target, or set \ - `allow_unsupported = true` under `[generators.{}]` in weaveffi.toml to generate \ - the supported surface anyway (unsupported entry points become explicit throwing \ - stubs)", - self.target, self.target - ) - } -} - -/// Check `api` against one target's declared capabilities. `Ok(())` when the -/// target supports every feature the API uses. -/// -/// # Errors -/// -/// Returns [`UnsupportedFeatures`] when `api` uses one or more gated features -/// that `caps` does not declare support for. The error carries every offending -/// feature paired with the IDL paths that use it, so the caller can report all -/// violations at once. -pub fn check( - api: &Api, - target: &str, - caps: &TargetCapabilities, -) -> Result<(), UnsupportedFeatures> { - let violations: Vec<(Feature, Vec)> = used_features(api) - .into_iter() - .filter(|(feature, _)| !caps.supports(*feature)) - .collect(); - if violations.is_empty() { - Ok(()) - } else { - Err(UnsupportedFeatures { - target: target.to_string(), - violations, - }) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use weaveffi_model::ir::{CallbackInterfaceDef, Function, Param}; - - fn func(name: &str, is_async: bool, returns: Option) -> Function { - Function { - name: name.into(), - params: vec![Param { - name: "x".into(), - ty: TypeRef::I32, - doc: None, - }], - returns, - doc: None, - throws: false, - r#async: is_async, - cancellable: false, - deprecated: None, - } - } - - fn module(name: &str) -> Module { - Module { - name: name.into(), - doc: None, - functions: vec![], - interfaces: vec![], - callback_interfaces: vec![], - structs: vec![], - enums: vec![], - errors: None, - modules: vec![], - } - } - - fn api(modules: Vec) -> Api { - Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules, - } - } - - fn events_api() -> Api { - api(vec![Module { - callback_interfaces: vec![CallbackInterfaceDef { - name: "MessageListener".into(), - doc: None, - deprecated: None, - methods: vec![func("on_message", false, None)], - }], - functions: vec![ - func("send", false, None), - func("fetch", true, Some(TypeRef::StringUtf8)), - func( - "all", - false, - Some(TypeRef::Iterator(Box::new(TypeRef::StringUtf8))), - ), - ], - ..module("events") - }]) - } - - #[test] - fn full_capabilities_pass_everything() { - assert!(check(&events_api(), "c", &TargetCapabilities::full()).is_ok()); - } - - #[test] - fn plain_api_uses_no_gated_features() { - let plain = api(vec![Module { - functions: vec![func("add", false, Some(TypeRef::I32))], - ..module("math") - }]); - assert!(used_features(&plain).is_empty()); - } - - #[test] - fn used_features_collects_locations() { - let used = used_features(&events_api()); - assert_eq!( - used[&Feature::CallbackInterfaces], - vec!["events.MessageListener".to_string()] - ); - assert_eq!( - used[&Feature::AsyncFunctions], - vec!["events.fetch".to_string()] - ); - assert_eq!(used[&Feature::Iterators], vec!["events.all".to_string()]); - } - - #[test] - fn nested_modules_use_dotted_paths() { - let nested = api(vec![Module { - modules: vec![Module { - functions: vec![func("fetch", true, None)], - ..module("inner") - }], - ..module("outer") - }]); - let used = used_features(&nested); - assert_eq!( - used[&Feature::AsyncFunctions], - vec!["outer.inner.fetch".to_string()] - ); - } - - #[test] - fn missing_capability_is_reported_with_locations() { - let caps = TargetCapabilities { - async_functions: false, - callback_interfaces: false, - ..TargetCapabilities::full() - }; - let err = check(&events_api(), "go", &caps).unwrap_err(); - assert_eq!(err.target, "go"); - assert_eq!(err.violations.len(), 2); - let msg = err.to_string(); - assert!(msg.contains("target 'go' does not support"), "{msg}"); - assert!( - msg.contains("async functions (used by: events.fetch)"), - "{msg}" - ); - assert!( - msg.contains("callback interfaces (used by: events.MessageListener)"), - "{msg}" - ); - } -} diff --git a/crates/weaveffi-gen/src/codegen/mod.rs b/crates/weaveffi-gen/src/codegen/mod.rs deleted file mode 100644 index a3326fb5..00000000 --- a/crates/weaveffi-gen/src/codegen/mod.rs +++ /dev/null @@ -1,562 +0,0 @@ -//! Target erasure and orchestration. -//! -//! Each language target implements [`LanguageBackend`] with its own typed -//! `Config`. The orchestrator works on the object-safe [`Target`] trait, which -//! erases the concrete config; [`ConfiguredBackend`] is the adapter that pairs -//! a backend with a concrete config value and is what the CLI and tests pass -//! into [`Orchestrator::with_target`]. -//! -//! Rendering is pure: a target returns its files in memory, and the -//! [`Orchestrator`] does every write. That is what lets it skip unchanged -//! targets, rewrite only files whose contents changed, remove files a -//! previous run wrote that the current run no longer produces (see -//! [`crate::cache`]), and lets `weaveffi diff` compare without touching disk. - -use std::collections::BTreeMap; - -use anyhow::{bail, Context, Result}; -use camino::Utf8Path; -use rayon::prelude::*; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::cache::{self, Record}; -use crate::capabilities::{self, TargetCapabilities}; -use crate::package::{PackageContext, PackagedFile}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; - -pub mod common; -pub mod writer; - -pub use writer::CodeWriter; - -/// Object-safe view of a [`LanguageBackend`] paired with a concrete config. -/// -/// The orchestrator stores targets as `&dyn Target` so it can hold a -/// heterogeneous set whose `Config` types differ. [`ConfiguredBackend`] is -/// the canonical adapter. -pub trait Target: Send + Sync { - /// The target's stable short name. Mirrors [`LanguageBackend::name`]. - fn name(&self) -> &'static str; - /// The gated features the target implements. Mirrors - /// [`LanguageBackend::capabilities`]. - fn capabilities(&self) -> TargetCapabilities; - /// See [`LanguageBackend::allows_unsupported`], evaluated against the - /// bound config. - fn allows_unsupported(&self) -> bool; - /// Render every file the target produces for `api`, with paths under - /// `out_dir`. Pure: nothing is written. - fn render(&self, api: &ResolvedApi, out_dir: &Utf8Path) -> Vec; - /// Assemble the distributable package for this target, using the bound - /// config. Returns `None` when the target does not support packaging. - fn package( - &self, - api: &ResolvedApi, - ctx: &PackageContext, - out_dir: &Utf8Path, - ) -> Option>; - /// Canonical-JSON encoding of the bound config, fed into the cache - /// hash so a config-only change invalidates the record. - fn config_hash_input(&self) -> Vec; -} - -/// Binds a [`LanguageBackend`] to a concrete config value so it can be erased -/// to `&dyn Target`. -/// -/// ```ignore -/// let swift = ConfiguredBackend::new(SwiftGenerator, SwiftConfig::default()); -/// orchestrator.with_target(&swift); -/// ``` -pub struct ConfiguredBackend { - inner: B, - config: B::Config, -} - -impl ConfiguredBackend { - /// Pair a backend with the concrete config it should run under. - pub fn new(inner: B, config: B::Config) -> Self { - Self { inner, config } - } -} - -impl Target for ConfiguredBackend { - fn name(&self) -> &'static str { - self.inner.name() - } - - fn capabilities(&self) -> TargetCapabilities { - self.inner.capabilities(&self.config) - } - - fn allows_unsupported(&self) -> bool { - self.inner.allows_unsupported(&self.config) - } - - fn render(&self, api: &ResolvedApi, out_dir: &Utf8Path) -> Vec { - let model = BindingModel::build(api); - self.inner.files(api, &model, out_dir, &self.config) - } - - fn package( - &self, - api: &ResolvedApi, - ctx: &PackageContext, - out_dir: &Utf8Path, - ) -> Option> { - let model = BindingModel::build(api); - self.inner.package(api, &model, ctx, out_dir, &self.config) - } - - fn config_hash_input(&self) -> Vec { - let value = - serde_json::to_value(&self.config).expect("backend config should serialize to JSON"); - serde_json::to_vec(&value).expect("JSON Value should serialize") - } -} - -/// The path of `file` relative to `out_dir`, with `/` separators on every -/// platform, so listings and cache records are OS-independent. -/// -/// # Panics -/// -/// Panics if a backend emitted a path outside `out_dir`, which is a backend -/// bug. -#[must_use] -pub fn relative_path(out_dir: &Utf8Path, file: &Utf8Path) -> String { - let rel = file - .strip_prefix(out_dir) - .unwrap_or_else(|_| panic!("generated file {file} is outside {out_dir}")); - rel.components() - .map(|c| c.as_str()) - .collect::>() - .join("/") -} - -/// Points in a generation run where the caller may act (the CLI runs the -/// project's `pre_generate` and `post_generate` commands here). -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum Hook { - /// At least one target is out of date; nothing has been written yet. - BeforeWrite, - /// Every out-of-date target has been written. - AfterWrite, -} - -/// What a generation run did. -#[derive(Debug, Default, Clone, PartialEq, Eq)] -pub struct GenerateReport { - /// Targets that were regenerated. - pub generated: Vec<&'static str>, - /// Targets whose inputs and files were unchanged, so they were skipped. - pub up_to_date: Vec<&'static str>, - /// Files written because they were new or their contents changed. - pub written: usize, - /// Files a regenerated target produced with unchanged contents. - pub unchanged: usize, - /// Files removed because the previous run wrote them and this run did - /// not (paths relative to the output directory). - pub removed: Vec, - /// Advisory messages, such as a target generating with - /// `allow_unsupported` set. - pub warnings: Vec, -} - -/// Runs a set of targets against one API and output directory. -#[derive(Default)] -pub struct Orchestrator<'a> { - targets: Vec<&'a dyn Target>, -} - -impl<'a> Orchestrator<'a> { - /// An orchestrator with no targets. - pub fn new() -> Self { - Self::default() - } - - /// Add a target. - #[must_use] - pub fn with_target(mut self, target: &'a dyn Target) -> Self { - self.targets.push(target); - self - } - - /// Check every target's capabilities against the features `api` uses. - /// Returns the warnings for targets that proceed under - /// `allow_unsupported`. - /// - /// # Errors - /// - /// Returns an error listing every target that cannot deliver a feature - /// the API uses and did not opt in to generating anyway. - pub fn check_capabilities(&self, api: &ResolvedApi) -> Result> { - let mut violations = Vec::new(); - let mut warnings = Vec::new(); - for t in &self.targets { - let Err(err) = capabilities::check(api.api(), t.name(), &t.capabilities()) else { - continue; - }; - if t.allows_unsupported() { - let mut msg = format!( - "target '{}' does not support every feature this API uses; generating \ - anyway because allow_unsupported is set:", - t.name() - ); - for (feature, locations) in &err.violations { - msg.push_str(&format!( - "\n - {feature} (used by: {})", - locations.join(", ") - )); - } - warnings.push(msg); - } else { - violations.push(err.to_string()); - } - } - if !violations.is_empty() { - bail!("{}", violations.join("\n")); - } - Ok(warnings) - } - - /// Generate every out-of-date target under `out_dir`. - /// - /// A target is up to date when its last record's input hash matches and - /// every file it recorded is still on disk unmodified; `force` treats - /// every target as out of date. Out-of-date targets render in parallel; - /// only files whose contents changed are written, and files the previous - /// run of a target wrote that this run no longer produces are removed. - /// `hook` runs before the first write and after the last, and only when - /// something is out of date. - /// - /// # Errors - /// - /// Returns an error when a capability check fails, a hook fails, or a - /// file cannot be written or removed. - pub fn run( - &self, - api: &ResolvedApi, - out_dir: &Utf8Path, - force: bool, - hook: &mut dyn FnMut(Hook) -> Result<()>, - ) -> Result { - let mut report = GenerateReport { - warnings: self.check_capabilities(api)?, - ..GenerateReport::default() - }; - if force { - cache::invalidate_all(out_dir)?; - } - let mut pending: Vec<(&'a dyn Target, String, Option)> = Vec::new(); - for &t in &self.targets { - let inputs = cache::hash_generator_inputs(api, t.name(), &t.config_hash_input()); - let previous = cache::read_record(out_dir, t.name()); - let fresh = previous - .as_ref() - .is_some_and(|r| r.inputs == inputs && r.files_intact(out_dir)); - if fresh { - report.up_to_date.push(t.name()); - } else { - pending.push((t, inputs, previous)); - } - } - if pending.is_empty() { - return Ok(report); - } - hook(Hook::BeforeWrite)?; - let rendered: Vec> = pending - .par_iter() - .map(|(t, _, _)| t.render(api, out_dir)) - .collect(); - for ((t, inputs, previous), files) in pending.iter().zip(rendered) { - let mut record = Record { - inputs: inputs.clone(), - files: BTreeMap::new(), - }; - for file in files { - let rel = relative_path(out_dir, &file.path); - let hash = cache::hash_bytes(file.contents.as_bytes()); - let current = std::fs::read(file.path.as_std_path()).ok(); - if current.as_deref() == Some(file.contents.as_bytes()) { - report.unchanged += 1; - } else { - if let Some(parent) = file.path.parent() { - std::fs::create_dir_all(parent.as_std_path()) - .with_context(|| format!("failed to create {parent}"))?; - } - std::fs::write(file.path.as_std_path(), &file.contents) - .with_context(|| format!("failed to write {}", file.path))?; - report.written += 1; - } - record.files.insert(rel, hash); - } - for stale in previous - .iter() - .flat_map(|p| p.files.keys()) - .filter(|rel| !record.files.contains_key(*rel)) - { - let path = out_dir.join(stale); - if path.exists() { - std::fs::remove_file(path.as_std_path()) - .with_context(|| format!("failed to remove stale file {path}"))?; - report.removed.push(stale.clone()); - } - } - cache::write_record(out_dir, t.name(), &record)?; - report.generated.push(t.name()); - } - hook(Hook::AfterWrite)?; - Ok(report) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use std::sync::atomic::{AtomicUsize, Ordering}; - use std::sync::{Arc, Mutex}; - use weaveffi_model::ir::{Api, CallbackInterfaceDef, Function, Module}; - - #[derive(Default, Clone, serde::Serialize)] - struct TestConfig { - knob: Option, - allow_unsupported: bool, - } - - struct Counting { - name: &'static str, - calls: Arc, - caps: TargetCapabilities, - files: Arc>>, - } - - impl LanguageBackend for Counting { - type Config = TestConfig; - - fn name(&self) -> &'static str { - self.name - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - self.caps - } - - fn allows_unsupported(&self, config: &Self::Config) -> bool { - config.allow_unsupported - } - - fn files( - &self, - _api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - _config: &Self::Config, - ) -> Vec { - self.calls.fetch_add(1, Ordering::SeqCst); - self.files - .lock() - .unwrap() - .iter() - .map(|f| { - OutputFile::new( - out_dir.join(self.name).join(f), - format!("{} {}", model.prefix, model.modules[0].name), - ) - }) - .collect() - } - } - - fn module(name: &str) -> Module { - Module { - name: name.into(), - doc: None, - functions: vec![], - interfaces: vec![], - structs: vec![], - enums: vec![], - callback_interfaces: vec![], - errors: None, - modules: vec![], - } - } - - fn api_named(name: &str) -> ResolvedApi { - ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![module(name)], - }) - } - - fn listener_api() -> ResolvedApi { - let mut m = module("math"); - m.callback_interfaces = vec![CallbackInterfaceDef { - name: "OnChange".into(), - doc: None, - deprecated: None, - methods: vec![Function { - name: "changed".into(), - params: vec![], - returns: None, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - }], - }]; - ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![m], - }) - } - - fn backend( - name: &'static str, - calls: &Arc, - callbacks: bool, - config: TestConfig, - ) -> ConfiguredBackend { - ConfiguredBackend::new( - Counting { - name, - calls: Arc::clone(calls), - caps: TargetCapabilities { - callback_interfaces: callbacks, - ..TargetCapabilities::full() - }, - files: Arc::new(Mutex::new(vec!["out.txt"])), - }, - config, - ) - } - - fn no_hook(_: Hook) -> Result<()> { - Ok(()) - } - - #[test] - fn capability_gate_blocks_unless_opted_in() { - let dir = tempfile::tempdir().unwrap(); - let out = Utf8Path::from_path(dir.path()).unwrap(); - let calls = Arc::new(AtomicUsize::new(0)); - let strict = backend("strict", &calls, false, TestConfig::default()); - let opted = backend( - "opted", - &calls, - false, - TestConfig { - knob: None, - allow_unsupported: true, - }, - ); - let err = Orchestrator::new() - .with_target(&strict) - .with_target(&opted) - .run(&listener_api(), out, false, &mut no_hook) - .unwrap_err() - .to_string(); - assert!(err.contains("target 'strict' does not support"), "{err}"); - assert!(!err.contains("target 'opted'"), "{err}"); - assert_eq!(calls.load(Ordering::SeqCst), 0); - let report = Orchestrator::new() - .with_target(&opted) - .run(&listener_api(), out, false, &mut no_hook) - .unwrap(); - assert_eq!(report.generated, ["opted"]); - assert_eq!(report.warnings.len(), 1); - } - - #[test] - fn records_skip_unchanged_targets_and_detect_tampering() { - let dir = tempfile::tempdir().unwrap(); - let out = Utf8Path::from_path(dir.path()).unwrap(); - let calls = Arc::new(AtomicUsize::new(0)); - let c = backend("c", &calls, true, TestConfig::default()); - let orch = Orchestrator::new().with_target(&c); - let api = api_named("math"); - - let first = orch.run(&api, out, false, &mut no_hook).unwrap(); - assert_eq!((first.generated.len(), first.written), (1, 1)); - let second = orch.run(&api, out, false, &mut no_hook).unwrap(); - assert_eq!(second.up_to_date, ["c"]); - assert_eq!(calls.load(Ordering::SeqCst), 1); - - std::fs::write(out.join("c/out.txt"), "edited").unwrap(); - let third = orch.run(&api, out, false, &mut no_hook).unwrap(); - assert_eq!((third.generated.len(), third.written), (1, 1)); - - let forced = orch.run(&api, out, true, &mut no_hook).unwrap(); - assert_eq!((forced.written, forced.unchanged), (0, 1)); - - let reconfigured = backend( - "c", - &calls, - true, - TestConfig { - knob: Some("changed".into()), - allow_unsupported: false, - }, - ); - let report = Orchestrator::new() - .with_target(&reconfigured) - .run(&api, out, false, &mut no_hook) - .unwrap(); - assert_eq!(report.generated, ["c"]); - } - - #[test] - fn stale_files_are_removed_but_user_files_are_kept() { - let dir = tempfile::tempdir().unwrap(); - let out = Utf8Path::from_path(dir.path()).unwrap(); - let calls = Arc::new(AtomicUsize::new(0)); - let c = backend("c", &calls, true, TestConfig::default()); - let files = Arc::clone(&c.inner.files); - *files.lock().unwrap() = vec!["a.txt", "b.txt"]; - let orch = Orchestrator::new().with_target(&c); - orch.run(&api_named("math"), out, false, &mut no_hook) - .unwrap(); - std::fs::write(out.join("c/user.txt"), "mine").unwrap(); - - *files.lock().unwrap() = vec!["a.txt"]; - let report = orch - .run(&api_named("math2"), out, false, &mut no_hook) - .unwrap(); - assert_eq!(report.removed, ["c/b.txt"]); - assert!(out.join("c/a.txt").exists()); - assert!(!out.join("c/b.txt").exists()); - assert!(out.join("c/user.txt").exists()); - } - - #[test] - fn hooks_run_only_when_something_is_out_of_date() { - let dir = tempfile::tempdir().unwrap(); - let out = Utf8Path::from_path(dir.path()).unwrap(); - let calls = Arc::new(AtomicUsize::new(0)); - let c = backend("c", &calls, true, TestConfig::default()); - let orch = Orchestrator::new().with_target(&c); - let mut seen = Vec::new(); - orch.run(&api_named("math"), out, false, &mut |h| { - seen.push(h); - Ok(()) - }) - .unwrap(); - orch.run(&api_named("math"), out, false, &mut |h| { - seen.push(h); - Ok(()) - }) - .unwrap(); - assert_eq!(seen, [Hook::BeforeWrite, Hook::AfterWrite]); - let err = orch - .run(&api_named("other"), out, false, &mut |_| { - bail!("hook failed") - }) - .unwrap_err(); - assert!(err.to_string().contains("hook failed")); - } - - #[test] - fn relative_paths_use_forward_slashes() { - let out = Utf8Path::new("gen"); - assert_eq!(relative_path(out, &out.join("c").join("x.h")), "c/x.h"); - } -} diff --git a/crates/weaveffi-gen/src/lib.rs b/crates/weaveffi-gen/src/lib.rs deleted file mode 100644 index f296b451..00000000 --- a/crates/weaveffi-gen/src/lib.rs +++ /dev/null @@ -1,23 +0,0 @@ -//! Code generation for WeaveFFI: the `LanguageBackend` trait, the -//! code-generation orchestrator, the C ABI header renderer, and the eleven -//! language generators under [`targets`]. -//! -//! Every generator renders from the resolved API and binding model in -//! [`weaveffi_model`], so symbol names and parameter lowering are computed -//! once and shared. -#![deny(missing_docs)] -#![warn(clippy::missing_errors_doc)] -#![warn(clippy::missing_panics_doc)] -#![warn(clippy::doc_markdown)] - -pub mod backend; -pub mod cabi; -pub mod cache; -pub mod capabilities; -pub mod codegen; -pub mod lang; -pub mod manifest; -pub mod package; -pub mod platform; -pub mod targets; -pub mod utils; diff --git a/crates/weaveffi-gen/src/package.rs b/crates/weaveffi-gen/src/package.rs deleted file mode 100644 index 75db6827..00000000 --- a/crates/weaveffi-gen/src/package.rs +++ /dev/null @@ -1,219 +0,0 @@ -//! The packaging layer: the data a backend needs to assemble a distributable -//! package, and the driver that materializes one to disk. -//! -//! `weaveffi generate` emits binding *source* that a consumer must compile or -//! point at a native library themselves. `weaveffi package` produces the next -//! artifact up: a ready-to-publish package for an ecosystem (an npm tarball -//! tree, a NuGet-ready project, a Python wheel tree, …) with a prebuilt native -//! library bundled for each [`Platform`](crate::platform::Platform) so -//! `npm install` / `pip install` / `dotnet add package` "just works" with no -//! local toolchain. -//! -//! A backend opts in by overriding -//! [`LanguageBackend::package`](crate::backend::LanguageBackend::package), -//! returning the full set of [`PackagedFile`]s that make up the package. The -//! [`write_package`] driver then writes the rendered text and copies the -//! bundled binaries into place. Rendering stays pure (it returns values, it -//! does no I/O), so package layouts are snapshot-testable exactly like -//! generated source. - -use anyhow::{Context, Result}; -use camino::{Utf8Path, Utf8PathBuf}; - -use crate::platform::BinarySet; - -/// The contents of one [`PackagedFile`]: either rendered text or a native -/// binary to copy in from elsewhere on disk. -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum FileContent { - /// Rendered text (a manifest, loader, README, or binding source file) - /// written verbatim. - Text(String), - /// A native library copied byte-for-byte from this source path. Used for - /// the prebuilt shared libraries a package bundles; keeping them out of - /// [`Text`](Self::Text) means package rendering never has to hold a - /// multi-megabyte binary in memory as a `String`. - Copy(Utf8PathBuf), -} - -/// One file in a packaged artifact: where to write it and what it contains. -/// -/// `path` is the full destination path (anchored under the package output -/// directory), mirroring [`OutputFile`](crate::backend::OutputFile) so backends -/// build paths with the same `out_dir.join(...)` idiom they already use in -/// [`files`](crate::backend::LanguageBackend::files). -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct PackagedFile { - /// Full destination path, anchored under the package output directory, with - /// `/` separators on every host (see [`text`](Self::text)). - pub path: Utf8PathBuf, - /// What to materialize at `path`. - pub content: FileContent, -} - -impl PackagedFile { - /// A file whose contents are rendered text. - /// - /// `path` separators are normalized to `/` on every host, so package - /// layouts stay identical across platforms. - pub fn text(path: impl Into, contents: impl Into) -> Self { - Self { - path: normalize_separators(path.into()), - content: FileContent::Text(contents.into()), - } - } - - /// A file copied from a prebuilt native library at `source`. - /// - /// The destination `path` is normalized to `/` separators; `source` is left - /// as-is, since it is a host path read directly off disk. - pub fn copy(path: impl Into, source: impl Into) -> Self { - Self { - path: normalize_separators(path.into()), - content: FileContent::Copy(source.into()), - } - } - - /// True when this entry copies in a native binary rather than writing text. - pub fn is_binary(&self) -> bool { - matches!(self.content, FileContent::Copy(_)) - } -} - -/// Normalize a package destination path to use `/` separators on every host. -/// -/// Backends build paths with `out_dir.join(...)`, which uses the host separator -/// (`\` on Windows). A package layout is logically `/`-separated, though: it is -/// published and consumed identically on every OS, and `std::fs` accepts `/` on -/// Windows so [`write_package`] still writes correctly. Doing this once here -/// keeps [`PackagedFile::path`] stable across platforms (so package layouts are -/// snapshot-testable like generated source) and mirrors how -/// [`output_files`](crate::backend::output_files) presents generated paths. -fn normalize_separators(path: Utf8PathBuf) -> Utf8PathBuf { - if cfg!(windows) { - Utf8PathBuf::from(path.into_string().replace('\\', "/")) - } else { - path - } -} - -/// Everything a backend's [`package`](crate::backend::LanguageBackend::package) -/// hook needs beyond the [`Api`](weaveffi_model::ir::Api) and -/// [`BindingModel`](weaveffi_model::model::BindingModel) it already receives. -/// -/// The prebuilt libraries to bundle live in [`binaries`](Self::binaries); -/// `input_basename` is the IDL file stem, used (as in `generate`) as the -/// fallback package name when the IDL omits a `package:` block. -#[derive(Debug, Clone, Copy)] -pub struct PackageContext<'a> { - /// The prebuilt native libraries to bundle, one per platform, plus the - /// logical library base name every loader and bundled filename derives - /// from. - pub binaries: &'a BinarySet, - /// The IDL file stem, used as the fallback package name. `None` when the - /// package identity comes entirely from the `package:` block or a config - /// override. - pub input_basename: Option<&'a str>, -} - -/// Write a rendered package to disk: create parent directories, write every -/// [`FileContent::Text`] verbatim, and copy every [`FileContent::Copy`] native -/// binary into place. -/// -/// # Errors -/// -/// Returns an error if a parent directory cannot be created, a text file cannot -/// be written, or a bundled binary's source path cannot be read or copied. -pub fn write_package(files: &[PackagedFile]) -> Result<()> { - for file in files { - if let Some(parent) = file.path.parent() { - std::fs::create_dir_all(parent.as_std_path()) - .with_context(|| format!("failed to create directory {parent}"))?; - } - match &file.content { - FileContent::Text(contents) => { - std::fs::write(file.path.as_std_path(), contents) - .with_context(|| format!("failed to write {}", file.path))?; - } - FileContent::Copy(source) => copy_binary(source, &file.path)?, - } - } - Ok(()) -} - -fn copy_binary(source: &Utf8Path, dest: &Utf8Path) -> Result<()> { - std::fs::copy(source.as_std_path(), dest.as_std_path()) - .with_context(|| format!("failed to copy native library {source} -> {dest}"))?; - Ok(()) -} - -/// Count the text files and bundled binaries in a rendered package, for the -/// CLI's end-of-run summary. -/// -/// Returns `(text_files, bundled_binaries)`. -pub fn summarize(files: &[PackagedFile]) -> (usize, usize) { - let binaries = files.iter().filter(|f| f.is_binary()).count(); - (files.len() - binaries, binaries) -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn write_package_writes_text_and_copies_binaries() { - let dir = tempfile::tempdir().unwrap(); - let root = Utf8Path::from_path(dir.path()).unwrap(); - - // A source binary to copy. - let src = root.join("src-lib.bin"); - std::fs::write(src.as_std_path(), b"\x00native\x01").unwrap(); - - let files = vec![ - PackagedFile::text(root.join("pkg/manifest.json"), "{\"name\":\"x\"}"), - PackagedFile::copy(root.join("pkg/native/lib.bin"), src.clone()), - ]; - write_package(&files).unwrap(); - - assert_eq!( - std::fs::read_to_string(root.join("pkg/manifest.json")).unwrap(), - "{\"name\":\"x\"}" - ); - assert_eq!( - std::fs::read(root.join("pkg/native/lib.bin")).unwrap(), - b"\x00native\x01" - ); - assert_eq!(summarize(&files), (1, 1)); - } - - #[test] - fn destination_paths_are_forward_slashed() { - // Built with `join` (host separator on Windows) but stored - // `/`-normalized, so a package layout matches on every OS. On Windows - // this exercises the `\` -> `/` rewrite; elsewhere it confirms the - // happy path is left intact. - let text = PackagedFile::text(Utf8Path::new("out").join("dotnet").join("x.cs"), "x"); - assert_eq!(text.path.as_str(), "out/dotnet/x.cs"); - - let copied = PackagedFile::copy( - Utf8Path::new("out") - .join("runtimes") - .join("osx-arm64") - .join("native"), - "/src/libcalculator.dylib", - ); - assert_eq!(copied.path.as_str(), "out/runtimes/osx-arm64/native"); - } - - #[test] - fn missing_binary_source_is_an_error() { - let dir = tempfile::tempdir().unwrap(); - let root = Utf8Path::from_path(dir.path()).unwrap(); - let files = vec![PackagedFile::copy( - root.join("pkg/native/lib.bin"), - root.join("does-not-exist.bin"), - )]; - let err = write_package(&files).unwrap_err(); - assert!(err.to_string().contains("failed to copy native library")); - } -} diff --git a/crates/weaveffi-gen/src/targets/c/mod.rs b/crates/weaveffi-gen/src/targets/c/mod.rs deleted file mode 100644 index 5ef78eaf..00000000 --- a/crates/weaveffi-gen/src/targets/c/mod.rs +++ /dev/null @@ -1,155 +0,0 @@ -//! C header generator for WeaveFFI. -//! -//! Emits `{library}.h`, describing the stable C ABI surface of a -//! [`ResolvedApi`]. This is the canonical backend: the header it emits *is* the -//! C ABI every other language binds to, and a C or C++ producer implementing -//! an IDL by hand implements exactly the functions it declares. -//! -//! When the API passes any value buffers and [`CConfig::buffer_helpers`] is -//! on (the default), it also emits `{library}_buffer.h`: C structs for the -//! API's records, rich enums, lists, and maps, with `static inline` codecs -//! that encode and decode them (see [`render_buffer_header`]). -//! -//! Like every WeaveFFI backend it renders from the shared -//! [`weaveffi_model::model::BindingModel`], so symbol names and parameter -//! lowering are computed once and shared, never re-derived here. - -mod buffer; -mod header; -mod package; -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; -use camino::Utf8Path; -use serde::{Deserialize, Serialize}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; - -pub use buffer::{buffer_header_name, render_buffer_header}; -pub use header::{render_c_header, render_c_header_from_model}; - -use package::{render_packaged_cmake, render_packaged_readme}; - -/// Per-target configuration for [`CGenerator`]. -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct CConfig { - /// Basename of the IDL the CLI was invoked with (e.g. `kvstore.yml`). - /// Embedded in the prelude header of every generated file. Populated - /// by the CLI; not user-configurable via the `[c]` config section. - #[serde(skip)] - pub input_basename: Option, - /// Emit `{library}_buffer.h`, the value-buffer helper header, next to - /// `{library}.h` when the API passes any value buffers. On by default. - pub buffer_helpers: bool, -} - -impl Default for CConfig { - fn default() -> Self { - Self { - input_basename: None, - buffer_helpers: true, - } - } -} - -impl CConfig { - /// The IDL basename for generated-file preludes. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } -} - -/// The header file name for an API: `{library}.h`. -#[must_use] -pub fn header_name(api: &ResolvedApi) -> String { - format!("{}.h", api.identity().library) -} - -/// The helper header's name and contents, when the configuration asks for it -/// and the API has value buffers to describe. -fn buffer_file( - api: &ResolvedApi, - model: &BindingModel, - config: &CConfig, -) -> Option<(String, String)> { - if !config.buffer_helpers { - return None; - } - let name = buffer_header_name(&api.identity().library); - render_buffer_header(model, config.input_basename(), &header_name(api), &name) - .map(|contents| (name, contents)) -} - -/// The C header backend. -pub struct CGenerator; - -impl LanguageBackend for CGenerator { - type Config = CConfig; - - fn name(&self) -> &'static str { - "c" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let header = header_name(api); - let mut files = vec![OutputFile::new( - out_dir.join("c").join(&header), - render_c_header_from_model(model, config.input_basename(), &header), - )]; - if let Some((name, contents)) = buffer_file(api, model, config) { - files.push(OutputFile::new(out_dir.join("c").join(name), contents)); - } - files - } - - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let input_basename = config.input_basename(); - let dir = out_dir.join("c"); - let header_name = header_name(api); - let lib = &ctx.binaries.lib_name; - - let mut files = vec![ - PackagedFile::text( - dir.join("include").join(&header_name), - render_c_header_from_model(model, input_basename, &header_name), - ), - PackagedFile::text( - dir.join("CMakeLists.txt"), - render_packaged_cmake(lib, input_basename), - ), - PackagedFile::text( - dir.join("README.md"), - render_packaged_readme(lib, &header_name, ctx, input_basename), - ), - ]; - if let Some((name, contents)) = buffer_file(api, model, config) { - files.push(PackagedFile::text(dir.join("include").join(name), contents)); - } - for nb in &ctx.binaries.binaries { - let dest = dir - .join("lib") - .join(nb.platform.id()) - .join(ctx.binaries.bundled_filename(nb.platform)); - files.push(PackagedFile::copy(dest, nb.source.clone())); - } - Some(files) - } -} diff --git a/crates/weaveffi-gen/src/targets/cpp/callbacks.rs b/crates/weaveffi-gen/src/targets/cpp/callbacks.rs deleted file mode 100644 index 9a145a8d..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/callbacks.rs +++ /dev/null @@ -1,282 +0,0 @@ -//! Callback-interface rendering: the abstract class the consumer implements, -//! the `extern "C"`-compatible trampolines that adapt an implementation to -//! the C vtable, and the process-wide static vtable, per -//! `weaveffi_model::plan::CallbackProtocol`. - -use crate::codegen::common::DocCommentStyle; -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding, Ty}; -use weaveffi_model::plan::ArgPass; - -use crate::targets::cpp::codec::emit_read_decl; -use crate::targets::cpp::types::{ - cpp_cb_param_decl, cpp_fn_name, cpp_ident, cpp_type, render_param_decls, slot_name, - trampoline_struct, vtable_accessor, -}; - -/// The foreign-error code a trampoline reports when the implementation throws. -const FOREIGN_ERROR_CODE: i32 = -4; - -/// Render the abstract class a consumer subclasses to implement a callback -/// interface: a virtual destructor and one pure virtual method per IDL -/// method, with idiomatic C++ signatures (strings as `std::string_view`, -/// buffered values by const reference, object arguments by value as the -/// adopted wrapper). -pub(crate) fn render_callback_class(out: &mut String, cb: &CallbackInterfaceBinding) { - let name = &cb.name; - let usage = format!( - "Implement this interface and pass a `std::shared_ptr<{name}>` to any \ - function that accepts one. The producer may call the methods from any \ - thread until it releases its last reference to the implementation. An \ - exception thrown by a method is reported to the producer as a foreign \ - error (code -4) and aborts the call that triggered it." - ); - let doc = match &cb.doc { - Some(d) => format!("{d}\n\n{usage}"), - None => usage, - }; - let mut w = CodeWriter::four_space(); - w.doc(&Some(doc), DocCommentStyle::Javadoc); - w.line(format!("class {name} {{")); - w.line("public:"); - w.scope(|w| { - w.line(format!("virtual ~{name}() = default;")); - w.blank(); - for m in &cb.methods { - w.doc(&m.doc, DocCommentStyle::Javadoc); - if let Some(msg) = &m.deprecated { - let escaped = msg.replace('"', "\\\""); - w.line(format!("[[deprecated(\"{escaped}\")]]")); - } - let ret = m.ret.as_ref().map_or("void".to_string(), cpp_type); - let params: Vec = m - .params - .iter() - .map(|p| cpp_cb_param_decl(&p.ty, &cpp_ident(&p.name))) - .collect(); - w.line(format!( - "virtual {ret} {}({}) = 0;", - cpp_fn_name(&m.name), - params.join(", ") - )); - } - }); - w.line("};"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render, inside `detail`, the trampolines and the static vtable for one -/// callback interface. -/// -/// `ctx` is a heap-allocated `std::shared_ptr` box created when the -/// implementation is passed to the producer; the trailing `free` entry -/// deletes it. Each method trampoline receives its arguments per the -/// callback protocol (strings viewed in place, bytes and buffers borrowed and -/// copied or decoded; objects adopted), calls the virtual method, and on any exception -/// reports `{prefix}_error_set(out_err, -4, message)` and returns a default -/// value, so nothing ever unwinds through the C frame. -pub(crate) fn render_callback_trampolines( - out: &mut String, - cb: &CallbackInterfaceBinding, - prefix: &str, -) { - let name = &cb.name; - let strukt = trampoline_struct(name); - let mut w = CodeWriter::four_space(); - w.line("namespace detail {"); - w.blank(); - w.line(format!( - "/** Vtable trampolines adapting a `{name}` implementation to `{}`. */", - cb.vtable_tag - )); - w.line(format!("struct {strukt} {{")); - w.scope(|w| { - for m in &cb.methods { - render_trampoline(w, cb, m, prefix); - } - w.line( - "/** Releases the implementation box once the producer drops its last reference. */", - ); - w.line("static void free_ctx(void* ctx) {"); - w.scope(|w| { - w.line(format!( - "delete static_cast*>(ctx);" - )); - }); - w.line("}"); - }); - w.line("};"); - w.blank(); - - w.line(format!( - "/** The process-wide vtable every `{name}` implementation is passed with. */" - )); - w.line(format!( - "inline const {}& {}() {{", - cb.vtable_tag, - vtable_accessor(name) - )); - w.scope(|w| { - w.line(format!("static const {} vtable = {{", cb.vtable_tag)); - w.scope(|w| { - for m in &cb.methods { - w.line(format!("&{strukt}::{},", cpp_ident(&m.name))); - } - w.line(format!("&{strukt}::free_ctx,")); - }); - w.line("};"); - w.line("return vtable;"); - }); - w.line("}"); - w.blank(); - w.line("} // namespace detail"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Emit one method trampoline: a static function with the vtable entry's -/// exact C signature. -fn render_trampoline( - w: &mut CodeWriter, - cb: &CallbackInterfaceBinding, - m: &CallbackMethodBinding, - prefix: &str, -) { - let iface = &cb.name; - let method = cpp_fn_name(&m.name); - let c_ret = m.abi_ret.render_c(prefix); - let params = render_param_decls(&m.abi_params, prefix).join(", "); - - w.line(format!( - "static {c_ret} {}({params}) {{", - cpp_ident(&m.name) - )); - w.scope(|w| { - // Object arguments transfer one strong reference each. Adopting them - // cannot throw, so it happens before anything that can, which - // guarantees every reference is released even when a later argument - // fails to decode. - for p in &m.params { - if let ArgPass::Object { slot, nullable } = p.arg_pass() { - let class = local_type_name( - p.ty - .interface_name() - .expect("object arguments name an interface"), - ); - let slot = slot_name(slot); - let var = arg_var(p); - if nullable { - w.line(format!("std::optional<{class}> {var};")); - w.line(format!("if ({slot}) {var}.emplace(adopt, {slot});")); - } else { - w.line(format!("{class} {var}(adopt, {slot});")); - } - } - } - w.line("try {"); - w.scope(|w| { - w.line(format!( - "{iface}& impl = **static_cast*>(ctx);" - )); - let args: Vec = m - .params - .iter() - .map(|p| emit_trampoline_arg(w, p)) - .collect(); - let call = format!("impl.{method}({})", args.join(", ")); - // Validation restricts callback returns to the direct family, so - // the value is written straight into the C return. - match &m.ret { - None => { - w.line(format!("{call};")); - w.line("return;"); - } - Some(Ty::Enum(_)) => { - w.line(format!( - "return static_cast<{c_ret}>(static_cast({call}));" - )); - } - Some(_) => { - w.line(format!("return {call};")); - } - } - }); - w.line("} catch (const std::exception& e) {"); - w.scope(|w| { - w.line(format!( - "{prefix}_error_set(out_err, {FOREIGN_ERROR_CODE}, e.what());" - )); - }); - w.line("} catch (...) {"); - w.scope(|w| { - w.line(format!( - "{prefix}_error_set(out_err, {FOREIGN_ERROR_CODE}, \"{iface}::{method} threw a non-standard exception\");" - )); - }); - w.line("}"); - if m.ret.is_some() { - w.line(format!("return {c_ret}{{}};")); - } - }); - w.line("}"); - w.blank(); -} - -/// The local variable holding one decoded or adopted trampoline argument. -fn arg_var(p: &ParamBinding) -> String { - format!("{}_val", p.name) -} - -/// Emit any decode statements for one trampoline argument and return the -/// expression handed to the virtual method. Strings, bytes, and buffered -/// values are borrowed for the call: a string is viewed in place, bytes and -/// buffers are copied or decoded into owned C++ values, and nothing is freed. -/// Objects were adopted before the `try` block and are moved into the call. -fn emit_trampoline_arg(w: &mut CodeWriter, p: &ParamBinding) -> String { - let var = arg_var(p); - match p.arg_pass() { - ArgPass::Buffer { ptr, len } => { - let rdr = format!("{}_r", p.name); - w.line(format!( - "detail::BufferReader {rdr}({}, {});", - slot_name(ptr), - slot_name(len) - )); - emit_read_decl(w, &p.ty, &var, &rdr); - w.line(format!("{rdr}.expect_end();")); - var - } - ArgPass::String { ptr, len } => { - w.line(format!( - "std::string_view {var} = detail::borrow_string({}, {});", - slot_name(ptr), - slot_name(len) - )); - var - } - ArgPass::Bytes { ptr, len } => { - w.line(format!( - "std::vector {var} = detail::borrow_bytes({}, {});", - slot_name(ptr), - slot_name(len) - )); - var - } - ArgPass::Object { .. } => format!("std::move({var})"), - ArgPass::Direct { slot } => { - let n = slot_name(slot); - match &p.ty { - Ty::Enum(e) => format!( - "static_cast<{}>(static_cast({n}))", - local_type_name(e) - ), - _ => n, - } - } - ArgPass::Callback { .. } => { - unreachable!("callback interfaces are never callback-method parameters") - } - } -} diff --git a/crates/weaveffi-gen/src/targets/cpp/calls.rs b/crates/weaveffi-gen/src/targets/cpp/calls.rs deleted file mode 100644 index 8897a718..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/calls.rs +++ /dev/null @@ -1,882 +0,0 @@ -//! Callable rendering: free functions and interface members in the sync, -//! iterator, and async call shapes, each marshalled per the shared passing -//! plans ([`ArgPass`], [`RetPass`], `IteratorProtocol`, `AsyncProtocol`). -//! -//! Interface members are rendered twice: a declaration inside the class body -//! (so the class is complete before any record that holds it by value) and an -//! out-of-line `inline` definition after every value type and codec exists. -//! Free functions are defined inline inside their module namespace, which is -//! rendered last. - -use crate::codegen::common::DocCommentStyle; -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use heck::ToUpperCamelCase; -use weaveffi_model::abi::AbiParam; -use weaveffi_model::model::Ty; -use weaveffi_model::model::{ - iterator_item_ctype, AbiFn, AsyncBinding, CallShape, FnBinding, IteratorBinding, ModuleBinding, - ParamBinding, -}; -use weaveffi_model::plan::{ret_pass, ArgPass, ErrorStrategy, RetPass}; - -use crate::targets::cpp::codec::{emit_read_decl, emit_write_value}; -use crate::targets::cpp::types::{ - cpp_fn_name, cpp_ident, cpp_namespace_path, cpp_param_decl, cpp_type, render_param_decls, - slot_name, vtable_accessor, -}; - -// ── Error routing ── - -/// The `detail::check*` helper a wrapper calls after the C call returns, -/// selected by the callable's [`ErrorStrategy`]: the per-domain variant -/// (throwing the typed exception) for [`ErrorStrategy::Throws`] in a module -/// with an error domain in scope, the generic trap (`Error`) otherwise. -pub(crate) fn check_helper(f: &FnBinding, module: &ModuleBinding) -> String { - match (&module.error, f.error_strategy()) { - (Some(eb), ErrorStrategy::Throws) => format!("detail::check_{}", eb.owner_path), - _ => "detail::check".to_string(), - } -} - -/// The full `detail::make*_error(...)` call expression an async trampoline -/// uses to convert a callback error into the `std::exception_ptr` set on the -/// promise. The typed domain helper also receives the borrowed payload slots; -/// the generic helper takes only the code and message. -fn make_error_call(f: &FnBinding, module: &ModuleBinding) -> String { - match (&module.error, f.error_strategy()) { - (Some(eb), ErrorStrategy::Throws) => format!( - "detail::make_{}_error(err->code, msg, err->payload_ptr, err->payload_len)", - eb.owner_path - ), - _ => "detail::make_error(err->code, msg)".to_string(), - } -} - -// ── Parameter and return marshalling ── - -/// The local wrapper class name behind an object-passed type: the interface -/// itself, or the interface inside `Interface?`. -fn object_class(ty: &Ty) -> &str { - local_type_name( - ty.interface_name() - .expect("object passing only applies to interfaces"), - ) -} - -/// Emit the setup statements for one C++ parameter and return the C argument -/// expressions its ABI slots receive, dispatching on the parameter's -/// [`ArgPass`] plan. -/// -/// * A string is passed as its UTF-8 pointer and length (no terminator, so -/// interior NULs survive); bytes likewise. -/// * A buffered parameter is encoded into a local `detail::BufferWriter` and -/// passed as `(data(), size())`; the caller owns the encoding for the -/// duration of the call. -/// * An object parameter is borrowed: the wrapper's pointer is passed and the -/// wrapper keeps its own reference. `Interface?` passes null for none. -/// * A callback interface moves the caller's `std::shared_ptr` into a heap -/// box that becomes `ctx`; the producer releases it through the vtable's -/// `free`. The vtable is the process-wide static for the interface. The -/// box is held by a `std::unique_ptr` until the C call's argument list -/// releases it, so a throw while marshalling a later parameter frees it. -fn emit_param_setup(w: &mut CodeWriter, p: &ParamBinding, prefix: &str) -> Vec { - let name = cpp_ident(&p.name); - match p.arg_pass() { - ArgPass::Buffer { .. } => { - let buf = format!("{name}_buf"); - w.line(format!("detail::BufferWriter {buf};")); - emit_write_value(w, &p.ty, &name, &buf, 0); - vec![format!("{buf}.data()"), format!("{buf}.size()")] - } - ArgPass::String { .. } => { - vec![format!("detail::utf8({name})"), format!("{name}.size()")] - } - ArgPass::Bytes { .. } => { - vec![format!("{name}.data()"), format!("{name}.size()")] - } - ArgPass::Object { nullable, .. } => { - if nullable { - vec![format!("{name}.has_value() ? {name}->handle() : nullptr")] - } else { - vec![format!("{name}.handle()")] - } - } - ArgPass::Callback { .. } => { - let iface = local_type_name( - p.ty.callback_interface_name() - .expect("callback passing only applies to callback interfaces"), - ); - w.line(format!( - "if (!{name}) throw std::invalid_argument(\"{name}: null callback interface\");" - )); - w.line(format!( - "auto {name}_ctx = std::make_unique>(std::move({name}));" - )); - vec![ - format!("static_cast({name}_ctx.release())"), - format!("&detail::{}()", vtable_accessor(iface)), - ] - } - ArgPass::Direct { slot } => match &p.ty { - Ty::Enum(_) => vec![format!( - "static_cast<{}>(static_cast({name}))", - slot.ty.render_c(prefix) - )], - _ => vec![name], - }, - } -} - -/// Marshal a sync callable's C result (already error-checked) into the C++ -/// return value at the writer's current depth, dispatching on the return's -/// [`RetPass`] plan. Strings, bytes, and buffered returns are copied or -/// decoded and then released with `{prefix}_free_bytes` (by a scope guard -/// for buffers, so the release happens even when decoding throws). An object -/// return is adopted into the RAII wrapper, which owes the eventual -/// `_destroy`. -fn emit_sync_return(w: &mut CodeWriter, ty: &Ty, prefix: &str) { - match ret_pass(Some(ty), prefix) { - RetPass::Buffer => { - w.line("detail::BufferGuard result_guard(result, out_len);"); - w.line("detail::BufferReader result_r(result, out_len);"); - emit_read_decl(w, ty, "ret", "result_r"); - w.line("result_r.expect_end();"); - w.line("return ret;"); - } - RetPass::String => { - w.line("return detail::take_string(result, out_len);"); - } - RetPass::Bytes => { - w.line("return detail::take_bytes(result, out_len);"); - } - RetPass::Object { nullable, .. } => { - if nullable { - w.line("if (!result) return std::nullopt;"); - } - w.line(format!("return {}(adopt, result);", object_class(ty))); - } - RetPass::Direct => match ty { - Ty::Enum(n) => { - w.line(format!( - "return static_cast<{}>(result);", - local_type_name(n) - )); - } - _ => { - w.line("return result;"); - } - }, - RetPass::Void => unreachable!("void returns emit no result marshalling"), - } -} - -// ── Callable kinds ── - -/// How a rendered callable is declared in the C++ surface. -#[derive(Clone, Copy)] -pub(crate) enum FnKind<'a> { - /// A namespace-scope free function (`inline` linkage). - Free, - /// An instance method on an interface class: passes the wrapped pointer as - /// the leading C argument and is declared `const` (the ABI receiver is a - /// const pointer). - Method { - /// The wrapper class name. - class: &'a str, - }, - /// A static member function: interface statics and the factory form of - /// constructors not named `new`. - Static { - /// The wrapper class name. - class: &'a str, - }, - /// The canonical constructor (an interface constructor named `new`): - /// rendered as a real C++ constructor adopting the returned pointer. - Ctor { - /// The wrapper class name. - class: &'a str, - }, -} - -impl<'a> FnKind<'a> { - /// The expression passed as the leading `self` C argument, when present. - fn self_arg(self) -> Option { - match self { - FnKind::Method { .. } => Some("handle_".to_string()), - _ => None, - } - } - - /// The owning class, for member kinds. - fn class(self) -> Option<&'a str> { - match self { - FnKind::Free => None, - FnKind::Method { class } | FnKind::Static { class } | FnKind::Ctor { class } => { - Some(class) - } - } - } -} - -/// The name of the lazy range class an iterator-returning callable yields: -/// `{PascalName}Iterator` for a free function (defined beside it in the module -/// namespace) and `{Class}{PascalName}Iterator` for an interface member -/// (defined at namespace scope after every value type). -pub(crate) fn iterator_class_name(f: &FnBinding, kind: FnKind<'_>) -> String { - let pascal = f.name.to_upper_camel_case(); - match kind.class() { - Some(class) => format!("{class}{pascal}Iterator"), - None => format!("{pascal}Iterator"), - } -} - -/// The C++ return type of a callable in its call shape: the mapped type (or -/// `void`) for sync, `std::future` for async, and the range class for an -/// iterator. `None` for a canonical constructor, which has no return type. -fn return_type(f: &FnBinding, kind: FnKind<'_>) -> Option { - if matches!(kind, FnKind::Ctor { .. }) { - return None; - } - let value = || f.ret.as_ref().map_or("void".to_string(), cpp_type); - Some(match &f.shape { - CallShape::Sync(_) => value(), - CallShape::Async(_) => format!("std::future<{}>", value()), - CallShape::Iterator(_) => iterator_class_name(f, kind), - }) -} - -/// The C++ parameter list of a callable. A cancellable async call takes a -/// trailing `const CancelToken&`; `with_defaults` adds its -/// `= CancelToken::none()` default, which belongs on the first declaration -/// only. -fn param_decls(f: &FnBinding, with_defaults: bool) -> String { - let mut decls: Vec = f - .params - .iter() - .map(|p| cpp_param_decl(&p.ty, &cpp_ident(&p.name))) - .collect(); - if f.is_async && f.cancellable { - let default = if with_defaults { - " = CancelToken::none()" - } else { - "" - }; - decls.push(format!("const CancelToken& cancel_token{default}")); - } - decls.join(", ") -} - -/// Emit the doc comment and any `[[deprecated]]` attribute for a callable. -/// An iterator-returning callable's doc also names the range class it yields. -fn emit_callable_attrs(w: &mut CodeWriter, f: &FnBinding, kind: FnKind<'_>) { - let doc = if let CallShape::Iterator(_) = &f.shape { - let class_name = iterator_class_name(f, kind); - let return_doc = format!( - "@return A lazy `{class_name}` range that streams one element per iteration \ - step and releases the producer iterator when exhausted or destroyed." - ); - Some(match &f.doc { - Some(d) => format!("{d}\n\n{return_doc}"), - None => return_doc, - }) - } else { - f.doc.clone() - }; - w.doc(&doc, DocCommentStyle::Javadoc); - if let Some(msg) = &f.deprecated { - let escaped = msg.replace('"', "\\\""); - w.line(format!("[[deprecated(\"{escaped}\")]]")); - } -} - -/// Render the in-class declaration of an interface member (one level of -/// indentation), with its doc comment and deprecation marker. -pub(crate) fn render_member_decl( - out: &mut String, - f: &FnBinding, - cpp_name: &str, - kind: FnKind<'_>, -) { - let mut w = CodeWriter::four_space().with_depth(1); - emit_callable_attrs(&mut w, f, kind); - let params = param_decls(f, true); - match (kind, return_type(f, kind)) { - // Explicit, so no argument list converts into a new producer object - // behind the caller's back (`std::optional t(5)`). - (FnKind::Ctor { class }, _) => w.line(format!("explicit {class}({params});")), - (FnKind::Method { .. }, Some(ret)) => w.line(format!("{ret} {cpp_name}({params}) const;")), - (FnKind::Static { .. }, Some(ret)) => w.line(format!("static {ret} {cpp_name}({params});")), - _ => unreachable!("free functions are defined inline, never declared"), - }; - w.blank(); - out.push_str(&w.finish()); -} - -/// Render the definition of a callable: an `inline` free function inside its -/// module namespace (with doc comment and deprecation marker), or the -/// out-of-line `inline` definition of an interface member declared earlier -/// by [`render_member_decl`]. -/// -/// Wrappers are deliberately never marked `noexcept`: a callable with -/// `throws == false` still surfaces producer panics as the generic `Error`. -/// -/// Every free function, constructor, and static member begins with -/// `check_library()`, which verifies the linked library once. Instance -/// methods skip it: their receiver could only come from a checked call. -pub(crate) fn render_definition( - out: &mut String, - f: &FnBinding, - cpp_name: &str, - kind: FnKind<'_>, - module: &ModuleBinding, - prefix: &str, -) { - let mut w = CodeWriter::four_space(); - if matches!(kind, FnKind::Free) { - emit_callable_attrs(&mut w, f, kind); - } - let params = param_decls(f, matches!(kind, FnKind::Free)); - let ret = return_type(f, kind); - let header = match (kind, ret) { - (FnKind::Ctor { class }, _) => { - // The canonical constructor adopts the pointer the C constructor - // returns; `handle_` starts null so a throw from the error check - // leaves nothing for the destructor to release. - format!("inline {class}::{class}({params}) : handle_(nullptr) {{") - } - (FnKind::Method { class }, Some(ret)) => { - format!("inline {ret} {class}::{cpp_name}({params}) const {{") - } - (FnKind::Static { class }, Some(ret)) => { - format!("inline {ret} {class}::{cpp_name}({params}) {{") - } - (FnKind::Free, Some(ret)) => format!("inline {ret} {cpp_name}({params}) {{"), - _ => unreachable!("every non-constructor callable has a return type"), - }; - w.line(header); - w.scope(|w| { - if kind.self_arg().is_none() { - w.line("check_library();"); - } - match &f.shape { - CallShape::Sync(abi) => emit_sync_body(w, f, abi, kind, module, prefix), - CallShape::Iterator(it) => emit_iterator_launch_body(w, f, it, kind, module, prefix), - CallShape::Async(a) => emit_async_body(w, f, a, kind, module, prefix), - } - }); - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Emit a synchronous callable's body: marshal the parameters (packing -/// buffered values into local buffers), call the C symbol, run the -/// throws-split error check, and marshal the return value (decoding buffered -/// returns then releasing the producer buffer, adopting object returns). For -/// a canonical constructor the "return" adopts the pointer instead. -fn emit_sync_body( - w: &mut CodeWriter, - f: &FnBinding, - abi: &AbiFn, - kind: FnKind<'_>, - module: &ModuleBinding, - prefix: &str, -) { - let check = check_helper(f, module); - let mut c_args = Vec::new(); - if let Some(self_arg) = kind.self_arg() { - c_args.push(self_arg); - } - for p in &f.params { - c_args.extend(emit_param_setup(w, p, prefix)); - } - - // A string, bytes, or buffered return carries a trailing `size_t* out_len`. - let has_out_len = matches!( - ret_pass(f.ret.as_ref(), prefix), - RetPass::Buffer | RetPass::String | RetPass::Bytes - ); - if has_out_len { - w.line("size_t out_len = 0;"); - c_args.push("&out_len".into()); - } - c_args.push("&err".into()); - let args_str = c_args.join(", "); - - w.line(format!("{prefix}_error err{{}};")); - if f.ret.is_none() { - w.line(format!("{}({args_str});", abi.symbol)); - } else { - w.line(format!("auto result = {}({args_str});", abi.symbol)); - } - w.line(format!("{check}(err);")); - - if matches!(kind, FnKind::Ctor { .. }) { - w.line("handle_ = result;"); - } else if let Some(ret) = &f.ret { - emit_sync_return(w, ret, prefix); - } -} - -// ── Iterators ── - -/// Render the lazy range class an iterator-returning callable yields. -/// -/// The C ABI yields an opaque iterator pointer plus `_next`/`_destroy`. The -/// range class is move-only, owns the pointer, and pulls exactly one element -/// per consumer step, honoring the `iter` streaming contract -/// (`weaveffi_model::plan::IteratorProtocol`): -/// -/// * `begin()`/`end()` expose a single-pass input iterator with a sentinel -/// end, so `for (auto&& item : fn())` streams in constant memory. -/// * Each pulled element is received per the plan's `elem` (strings, bytes, -/// and buffered values copied or decoded then released with -/// `{prefix}_free_bytes`; objects adopted into their RAII wrapper). -/// * `destroy` runs exactly once: eagerly on exhaustion or a `next` error, -/// from the destructor otherwise. The pointer is nulled on every path. -/// * Launch and per-`next` errors follow the callable's [`ErrorStrategy`] -/// (the typed domain exception for `Throws`, the generic `Error` trap -/// otherwise). -pub(crate) fn render_iterator_range( - out: &mut String, - f: &FnBinding, - it: &IteratorBinding, - cpp_name: &str, - kind: FnKind<'_>, - module: &ModuleBinding, - prefix: &str, -) { - let elem_cpp = cpp_type(&it.elem); - let class_name = iterator_class_name(f, kind); - let iter_tag = &it.iter_tag; - let destroy = &it.destroy_symbol; - let check = check_helper(f, module); - let owner = match kind.class() { - Some(class) => format!("{class}::{cpp_name}()"), - None => format!("{cpp_name}()"), - }; - - let mut w = CodeWriter::four_space(); - w.doc( - &Some(format!( - "A lazy, move-only range over the `{elem_cpp}` elements produced by \ - `{owner}`.\n\nEach iteration step pulls exactly one element from the \ - producer, so results stream in constant memory. The range owns the \ - producer-side iterator and releases it exactly once: eagerly when the \ - range is exhausted, or from the destructor when iteration is abandoned \ - early." - )), - DocCommentStyle::Javadoc, - ); - w.line(format!("class {class_name} {{")); - w.scope(|w| { - w.line(format!("{iter_tag}* handle_;")); - w.blank(); - }); - w.line("public:"); - w.scope(|w| { - w.line("/** Adopts ownership of a raw producer iterator. */"); - w.line(format!( - "explicit {class_name}(adopt_t, {iter_tag}* h) noexcept : handle_(h) {{}}" - )); - w.blank(); - - w.line(format!("~{class_name}() {{")); - w.scope(|w| { - w.line(format!("if (handle_) {destroy}(handle_);")); - }); - w.line("}"); - w.blank(); - - w.line(format!("{class_name}(const {class_name}&) = delete;")); - w.line(format!( - "{class_name}& operator=(const {class_name}&) = delete;" - )); - w.blank(); - - w.line(format!( - "{class_name}({class_name}&& other) noexcept : handle_(other.handle_) {{" - )); - w.scope(|w| { - w.line("other.handle_ = nullptr;"); - }); - w.line("}"); - w.blank(); - - w.line(format!( - "{class_name}& operator=({class_name}&& other) noexcept {{" - )); - w.scope(|w| { - w.line("if (this != &other) {"); - w.scope(|w| { - w.line(format!("if (handle_) {destroy}(handle_);")); - w.line("handle_ = other.handle_;"); - w.line("other.handle_ = nullptr;"); - }); - w.line("}"); - w.line("return *this;"); - }); - w.line("}"); - w.blank(); - - render_iterator_next_method(w, f, it, module, prefix, &check); - - w.line("/** Sentinel type marking the end of the range. */"); - w.line("struct sentinel {};"); - w.blank(); - - w.line( - "/** Single-pass input iterator; each increment pulls one element from the producer. */", - ); - w.line("class iterator {"); - w.scope(|w| { - w.line(format!("{class_name}* range_;")); - w.line(format!("std::optional<{elem_cpp}> current_;")); - w.blank(); - }); - w.line("public:"); - w.scope(|w| { - w.line("using iterator_category = std::input_iterator_tag;"); - w.line(format!("using value_type = {elem_cpp};")); - w.line("using difference_type = std::ptrdiff_t;"); - w.line(format!("using pointer = {elem_cpp}*;")); - w.line(format!("using reference = {elem_cpp}&;")); - w.blank(); - w.line("/** Binds to `range` and pulls the first element. */"); - w.line(format!( - "explicit iterator({class_name}* range) : range_(range), current_(range->next()) {{}}" - )); - w.line("reference operator*() { return *current_; }"); - w.line("pointer operator->() { return &*current_; }"); - w.line("iterator& operator++() { current_ = range_->next(); return *this; }"); - w.line("void operator++(int) { current_ = range_->next(); }"); - w.line("bool operator==(sentinel) const { return !current_.has_value(); }"); - w.line("bool operator!=(sentinel) const { return current_.has_value(); }"); - }); - w.line("};"); - w.blank(); - - w.line("/** Begins iteration by pulling the first element. */"); - w.line("iterator begin() { return iterator(this); }"); - w.blank(); - w.line("/** The past-the-end sentinel. */"); - w.line("sentinel end() const { return sentinel{}; }"); - }); - w.line("};"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Emit an iterator-returning callable's body: marshal the parameters, call -/// the launcher, check `out_err`, and wrap the returned pointer in the range -/// class. -fn emit_iterator_launch_body( - w: &mut CodeWriter, - f: &FnBinding, - it: &IteratorBinding, - kind: FnKind<'_>, - module: &ModuleBinding, - prefix: &str, -) { - let class_name = iterator_class_name(f, kind); - let check = check_helper(f, module); - let mut c_args = Vec::new(); - if let Some(self_arg) = kind.self_arg() { - c_args.push(self_arg); - } - for p in &f.params { - c_args.extend(emit_param_setup(w, p, prefix)); - } - c_args.push("&err".into()); - w.line(format!("{prefix}_error err{{}};")); - w.line(format!( - "{}* iter = {}({});", - it.iter_tag, - it.launch.symbol, - c_args.join(", ") - )); - w.line(format!("{check}(err);")); - w.line(format!("return {class_name}(adopt, iter);")); -} - -/// Emit the range class's `next()` member: one producer `next` call that -/// yields the received element (or `std::nullopt` on exhaustion), releasing -/// the pulled slot per the plan's `elem` and destroying the iterator exactly -/// once on exhaustion or error. -fn render_iterator_next_method( - w: &mut CodeWriter, - f: &FnBinding, - it: &IteratorBinding, - module: &ModuleBinding, - prefix: &str, - check: &str, -) { - let elem_cpp = cpp_type(&it.elem); - let destroy = &it.destroy_symbol; - let item_ty = iterator_item_ctype(&it.elem, &module.path).render_c(prefix); - let elem_pass = ret_pass(Some(&it.elem), prefix); - let strategy_doc = match f.error_strategy() { - ErrorStrategy::Throws => "throws the module's typed exception", - ErrorStrategy::Trap => "throws the generic Error", - }; - - w.doc( - &Some(format!( - "Pulls the next element from the producer, or `std::nullopt` once \ - exhausted (which releases the producer iterator eagerly). A producer \ - error {strategy_doc} after releasing the iterator." - )), - DocCommentStyle::Javadoc, - ); - w.line(format!("std::optional<{elem_cpp}> next() {{")); - w.scope(|w| { - w.line("if (!handle_) return std::nullopt;"); - w.line(format!("{prefix}_error err{{}};")); - w.line(format!("{item_ty} item{{}};")); - let mut next_args = vec!["handle_".to_string(), "&item".to_string()]; - if matches!( - elem_pass, - RetPass::Buffer | RetPass::String | RetPass::Bytes - ) { - w.line("size_t item_len = 0;"); - next_args.push("&item_len".to_string()); - } - next_args.push("&err".to_string()); - w.line(format!( - "int32_t has_item = {}({});", - it.next.symbol, - next_args.join(", ") - )); - w.line("if (err.code != 0) {"); - w.scope(|w| { - w.line(format!("{destroy}(handle_);")); - w.line("handle_ = nullptr;"); - w.line(format!("{check}(err);")); - }); - w.line("}"); - w.line("if (has_item == 0) {"); - w.scope(|w| { - w.line(format!("{destroy}(handle_);")); - w.line("handle_ = nullptr;"); - w.line("return std::nullopt;"); - }); - w.line("}"); - match elem_pass { - RetPass::Buffer => { - // A buffered element is producer-allocated: decode into an - // owned value, then release with free_bytes via the scope - // guard. - w.line("detail::BufferGuard item_guard(item, item_len);"); - w.line("detail::BufferReader item_r(item, item_len);"); - emit_read_decl(w, &it.elem, "value", "item_r"); - w.line("item_r.expect_end();"); - w.line("return value;"); - } - RetPass::Bytes => { - w.line("return detail::take_bytes(item, item_len);"); - } - RetPass::String => { - w.line("return detail::take_string(item, item_len);"); - } - // An object element transfers one strong reference: adopt it. A - // nullable element yields an engaged outer optional holding an - // empty inner one for null, which is distinct from exhaustion. - RetPass::Object { nullable, .. } => { - let class = object_class(&it.elem); - if nullable { - w.line(format!("{elem_cpp} value;")); - w.line("if (item) value.emplace(adopt, item);"); - w.line("return value;"); - } else { - w.line(format!("return {class}(adopt, item);")); - } - } - RetPass::Direct => match &it.elem { - Ty::Enum(n) => { - let n = local_type_name(n); - w.line(format!("return static_cast<{n}>(item);")); - } - _ => { - w.line("return item;"); - } - }, - RetPass::Void => unreachable!("iterator elements are never void"), - } - }); - w.line("}"); - w.blank(); -} - -// ── Async ── - -/// Emit an asynchronous callable's body as a `std::future` wrapper. -/// -/// The parameters are marshalled first (so a throw there leaks nothing), -/// then a heap-allocated promise is threaded through the C `context` pointer -/// and adopted back into a `std::unique_ptr` by the completion callback, -/// which settles it exactly once. A callback error settles the promise with -/// the typed domain exception (payload fields decoded) when the callable -/// throws, `Cancelled` for code -5, or the generic `Error` otherwise. The -/// callback owns everything it receives: the boxed error and any string or -/// buffer result are released after copying, and an object result is -/// adopted. A cancellable call passes its `CancelToken`'s native token (null -/// for `CancelToken::none()`); the producer takes its own reference. -fn emit_async_body( - w: &mut CodeWriter, - f: &FnBinding, - a: &AsyncBinding, - kind: FnKind<'_>, - module: &ModuleBinding, - prefix: &str, -) { - let cpp_ret = f.ret.as_ref().map_or("void".to_string(), cpp_type); - let promise = format!("std::promise<{cpp_ret}>"); - let cb_params = render_param_decls(&a.callback_params, prefix).join(", "); - let make_error = make_error_call(f, module); - - let mut c_args = Vec::new(); - if let Some(self_arg) = kind.self_arg() { - c_args.push(self_arg); - } - for p in &f.params { - c_args.extend(emit_param_setup(w, p, prefix)); - } - if f.cancellable { - c_args.push("cancel_token.handle()".to_string()); - } - w.line(format!("auto promise = std::make_unique<{promise}>();")); - w.line("auto future = promise->get_future();"); - - if c_args.is_empty() { - w.line(format!("{}([]({cb_params}) {{", a.launch.symbol)); - } else { - w.line(format!( - "{}({}, []({cb_params}) {{", - a.launch.symbol, - c_args.join(", ") - )); - } - w.scope(|w| { - w.line(format!( - "std::unique_ptr<{promise}> p(static_cast<{promise}*>(context));" - )); - // The callback runs on a producer thread inside a C frame, so a - // decode failure (of the result or of an error payload) must settle - // the promise, never unwind. - w.line("try {"); - w.scope(|w| { - w.line("if (err != nullptr && err->code != 0) {"); - w.scope(|w| { - w.line("std::string msg = detail::error_message(*err);"); - w.line(format!("p->set_exception({make_error});")); - }); - w.line("} else {"); - w.scope(|w| { - if let Some(ret) = &f.ret { - emit_async_set_value(w, ret, &a.callback_params[2..], prefix); - } else { - w.line("p->set_value();"); - } - }); - w.line("}"); - }); - w.line("} catch (...) {"); - w.scope(|w| { - w.line("p->set_exception(std::current_exception());"); - }); - w.line("}"); - w.line(format!("{prefix}_error_free(err);")); - }); - w.line("}, static_cast(promise.release()));"); - w.line("return future;"); -} - -/// Settle an async promise from the callback's result slots (`slots`, the -/// callback parameters after `context` and `err`) at the writer's current -/// depth, dispatching on the result's [`RetPass`] plan. -/// -/// Per the async completion contract (`weaveffi_model::plan::AsyncProtocol`), -/// result buffers handed to the callback (strings, bytes, and buffered -/// values) are *owned by the consumer*: the wrapper copies or decodes them, -/// then releases the producer allocation with `{prefix}_free_bytes`. An -/// owned interface result is instead adopted into the RAII wrapper. -fn emit_async_set_value(w: &mut CodeWriter, ty: &Ty, slots: &[AbiParam], prefix: &str) { - let slot = |i: usize| slot_name(&slots[i]); - match ret_pass(Some(ty), prefix) { - RetPass::Buffer => { - let (ptr, len) = (slot(0), slot(1)); - w.line(format!("detail::BufferGuard result_guard({ptr}, {len});")); - w.line(format!("detail::BufferReader result_r({ptr}, {len});")); - emit_read_decl(w, ty, "value", "result_r"); - w.line("result_r.expect_end();"); - w.line("p->set_value(std::move(value));"); - } - RetPass::String => { - w.line(format!( - "p->set_value(detail::take_string({}, {}));", - slot(0), - slot(1) - )); - } - RetPass::Bytes => { - w.line(format!( - "p->set_value(detail::take_bytes({}, {}));", - slot(0), - slot(1) - )); - } - RetPass::Object { nullable, .. } => { - let class = object_class(ty); - let result = slot(0); - if nullable { - w.line(format!("if ({result} == nullptr) {{")); - w.scope(|w| { - w.line("p->set_value(std::nullopt);"); - }); - w.line("} else {"); - w.scope(|w| { - w.line(format!("p->set_value({class}(adopt, {result}));")); - }); - w.line("}"); - } else { - w.line(format!("p->set_value({class}(adopt, {result}));")); - } - } - RetPass::Direct => match ty { - Ty::Enum(n) => { - w.line(format!( - "p->set_value(static_cast<{}>({}));", - local_type_name(n), - slot(0) - )); - } - _ => { - w.line(format!("p->set_value({});", slot(0))); - } - }, - RetPass::Void => unreachable!("void results settle the promise with no value"), - } -} - -// ── Namespace: per-module function namespaces ── - -/// Emit one module's nested namespace holding its free functions with bare -/// snake_case names (`namespace kv::stats { ... }`). An iterator-returning -/// function is preceded by its range class. Modules with no functions emit -/// nothing; their types live at the namespace root. -pub(crate) fn render_cpp_module_ns(out: &mut String, module: &ModuleBinding, prefix: &str) { - if module.functions.is_empty() { - return; - } - let ns = cpp_namespace_path(module); - out.push_str(&format!("namespace {ns} {{\n\n")); - for f in &module.functions { - let name = cpp_fn_name(&f.name); - if let CallShape::Iterator(it) = &f.shape { - render_iterator_range(out, f, it, &name, FnKind::Free, module, prefix); - } - render_definition(out, f, &name, FnKind::Free, module, prefix); - } - out.push_str(&format!("}} // namespace {ns}\n\n")); -} diff --git a/crates/weaveffi-gen/src/targets/cpp/codec.rs b/crates/weaveffi-gen/src/targets/cpp/codec.rs deleted file mode 100644 index be374e75..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/codec.rs +++ /dev/null @@ -1,302 +0,0 @@ -//! Value-buffer codec emission: the statements and expressions that move C++ -//! values through the value-buffer wire format, plus the per-type pack and unpack -//! routines. -//! -//! Every encode/decode decision dispatches on [`Ty::wire`], the shared wire -//! classification, so this backend cannot drift from the wire format's -//! canonical shapes: primitives, enums as `i32`, records and rich enums as -//! one user-codec shape, and interfaces as `u64` object tokens carrying one -//! strong reference. - -use crate::codegen::common::DocCommentStyle; -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use weaveffi_model::model::Ty; -use weaveffi_model::model::WireType; -use weaveffi_model::model::{EnumBinding, StructBinding}; - -use crate::targets::cpp::types::{cpp_ident, cpp_type}; - -/// Emit statements appending `expr` (a C++ lvalue of IDL type `ty`) to the -/// buffer writer variable `wtr`, in wire order. `depth` disambiguates nested -/// loop variable names. -/// -/// An object token is written from a *fresh* strong reference: the wrapper's -/// `clone_handle()` calls the interface's `_clone` symbol, so the reader -/// adopts a reference of its own and the wrapper keeps the one it holds. -pub(crate) fn emit_write_value(w: &mut CodeWriter, ty: &Ty, expr: &str, wtr: &str, depth: usize) { - let leaf = |w: &mut CodeWriter, method: &str| { - w.line(format!("{wtr}.{method}({expr});")); - }; - match ty.wire() { - WireType::Prim(p) => leaf(w, &format!("write_{}", p.snake())), - WireType::Enum(_) => { - w.line(format!("{wtr}.write_i32(static_cast({expr}));")); - } - WireType::Object(_) => { - w.line(format!( - "{wtr}.write_u64(static_cast(reinterpret_cast({expr}.clone_handle())));" - )); - } - WireType::User(n) => { - w.line(format!( - "detail::write_{}({wtr}, {expr});", - local_type_name(n) - )); - } - WireType::Optional(inner) => { - w.line(format!("{wtr}.write_option_flag({expr}.has_value());")); - w.line(format!("if ({expr}.has_value()) {{")); - w.scope(|w| emit_write_value(w, inner, &format!("(*{expr})"), wtr, depth)); - w.line("}"); - } - WireType::List(inner) => { - w.line(format!("{wtr}.write_len({expr}.size());")); - w.line(format!("for (const auto& item{depth} : {expr}) {{")); - w.scope(|w| emit_write_value(w, inner, &format!("item{depth}"), wtr, depth + 1)); - w.line("}"); - } - WireType::Map(k, v) => { - w.line(format!("{wtr}.write_len({expr}.size());")); - w.line(format!("for (const auto& kv{depth} : {expr}) {{")); - w.scope(|w| { - emit_write_value(w, k, &format!("kv{depth}.first"), wtr, depth + 1); - emit_write_value(w, v, &format!("kv{depth}.second"), wtr, depth + 1); - }); - w.line("}"); - } - } -} - -/// The single expression decoding one leaf value from the reader variable -/// `rdr`, or `None` when `ty` is a composite that needs statements. -/// -/// An object token decodes into a new wrapper that adopts the strong -/// reference the token carries; the wrapper's destructor releases it. -fn read_leaf_expr(ty: &Ty, rdr: &str) -> Option { - Some(match ty.wire() { - WireType::Prim(p) => format!("{rdr}.read_{}()", p.snake()), - WireType::Enum(n) => format!("static_cast<{}>({rdr}.read_i32())", local_type_name(n)), - WireType::Object(n) => { - let class = local_type_name(n); - format!( - "{class}(adopt, reinterpret_cast<{class}::raw_type*>(static_cast({rdr}.read_u64())))" - ) - } - WireType::User(n) => format!("detail::read_{}({rdr})", local_type_name(n)), - WireType::Optional(_) | WireType::List(_) | WireType::Map(_, _) => return None, - }) -} - -/// Emit statements decoding one value of IDL type `ty` from the reader -/// variable `rdr` into the existing (default-initialized) lvalue `target`. -/// `tmp` seeds unique names for any temporaries the composite cases need. -pub(crate) fn emit_read_into(w: &mut CodeWriter, ty: &Ty, target: &str, tmp: &str, rdr: &str) { - if let Some(expr) = read_leaf_expr(ty, rdr) { - w.line(format!("{target} = {expr};")); - return; - } - match ty.wire() { - WireType::Optional(inner) => { - w.line(format!("if ({rdr}.read_option_flag()) {{")); - w.scope(|w| { - let var = format!("{tmp}_v"); - emit_read_decl(w, inner, &var, rdr); - w.line(format!("{target} = std::move({var});")); - }); - w.line("}"); - } - WireType::List(inner) => { - w.line("{"); - w.scope(|w| { - w.line(format!("size_t {tmp}_n = {rdr}.read_count();")); - w.line(format!("{target}.reserve({rdr}.reserve_hint({tmp}_n));")); - w.line(format!( - "for (size_t {tmp}_i = 0; {tmp}_i < {tmp}_n; ++{tmp}_i) {{" - )); - w.scope(|w| { - let var = format!("{tmp}_item"); - emit_read_decl(w, inner, &var, rdr); - w.line(format!("{target}.push_back(std::move({var}));")); - }); - w.line("}"); - }); - w.line("}"); - } - WireType::Map(k, v) => { - w.line("{"); - w.scope(|w| { - w.line(format!("size_t {tmp}_n = {rdr}.read_count();")); - w.line(format!( - "for (size_t {tmp}_i = 0; {tmp}_i < {tmp}_n; ++{tmp}_i) {{" - )); - w.scope(|w| { - let key = format!("{tmp}_key"); - let val = format!("{tmp}_val"); - emit_read_decl(w, k, &key, rdr); - emit_read_decl(w, v, &val, rdr); - w.line(format!( - "{target}.emplace(std::move({key}), std::move({val}));" - )); - }); - w.line("}"); - }); - w.line("}"); - } - _ => unreachable!("leaf handled above"), - } -} - -/// Emit statements declaring a fresh variable `var` and decoding one value of -/// IDL type `ty` into it from the reader variable `rdr`. Leaf types decode in -/// a single declaration; composites (optionals, lists, and maps, which are -/// always default constructible) declare then fill. Records are never -/// default constructed here: a record holding an interface has no default -/// constructor, so record values are only ever produced by their `read_` -/// routine. -pub(crate) fn emit_read_decl(w: &mut CodeWriter, ty: &Ty, var: &str, rdr: &str) { - let cpp = cpp_type(ty); - if let Some(expr) = read_leaf_expr(ty, rdr) { - w.line(format!("{cpp} {var} = {expr};")); - } else { - w.line(format!("{cpp} {var}{{}};")); - emit_read_into(w, ty, var, var, rdr); - } -} - -/// Emit the pack and unpack routines for one record (inside `detail`). -/// -/// The unpack routine decodes every field into a local and then -/// aggregate-initializes the record in one expression, so a record that holds -/// an interface (and therefore has no default constructor) decodes like any -/// other. If a later field fails to decode, the locals already holding -/// adopted object references are destroyed on unwind, releasing them. -pub(crate) fn render_record_codec(out: &mut String, s: &StructBinding) { - let name = &s.name; - let mut w = CodeWriter::four_space(); - w.doc( - &Some(format!( - "Encodes a `{name}` in the value-buffer wire format." - )), - DocCommentStyle::Javadoc, - ); - w.line(format!( - "inline void write_{name}(BufferWriter& w, const {name}& v) {{" - )); - w.scope(|w| { - if s.fields.is_empty() { - w.line("(void)w;"); - w.line("(void)v;"); - } - for f in &s.fields { - emit_write_value(w, &f.ty, &format!("v.{}", cpp_ident(&f.name)), "w", 0); - } - }); - w.line("}"); - w.blank(); - - w.doc( - &Some(format!( - "Decodes a `{name}` from the value-buffer wire format." - )), - DocCommentStyle::Javadoc, - ); - w.line(format!("inline {name} read_{name}(BufferReader& r) {{")); - w.scope(|w| { - if s.fields.is_empty() { - w.line("(void)r;"); - } - let mut inits = Vec::new(); - for f in &s.fields { - let var = format!("f_{}", f.name); - emit_read_decl(w, &f.ty, &var, "r"); - inits.push(format!("std::move({var})")); - } - w.line(format!("return {name}{{{}}};", inits.join(", "))); - }); - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Emit the pack and unpack routines for one rich enum (inside `detail`): -/// an `i32` tag followed by the active variant's fields in wire order. Like -/// records, variant payloads are aggregate-initialized from decoded locals. -pub(crate) fn render_rich_enum_codec(out: &mut String, e: &EnumBinding) { - let name = &e.name; - let mut w = CodeWriter::four_space(); - w.doc( - &Some(format!( - "Encodes a `{name}` in the value-buffer wire format." - )), - DocCommentStyle::Javadoc, - ); - w.line(format!( - "inline void write_{name}(BufferWriter& w, const {name}& v) {{" - )); - w.scope(|w| { - w.line("switch (v.value.index()) {"); - for (i, variant) in e.variants.iter().enumerate() { - let vn = cpp_ident(&variant.name); - w.line(format!("case {i}: {{")); - w.scope(|w| { - w.line(format!("w.write_i32({});", variant.value)); - if !variant.fields.is_empty() { - w.line(format!("const {name}::{vn}& p = std::get<{i}>(v.value);")); - for f in &variant.fields { - emit_write_value(w, &f.ty, &format!("p.{}", cpp_ident(&f.name)), "w", 0); - } - } - w.line("break;"); - }); - w.line("}"); - } - w.line("}"); - }); - w.line("}"); - w.blank(); - - w.doc( - &Some(format!( - "Decodes a `{name}` from the value-buffer wire format." - )), - DocCommentStyle::Javadoc, - ); - w.line(format!("inline {name} read_{name}(BufferReader& r) {{")); - w.scope(|w| { - w.line("int32_t tag = r.read_i32();"); - w.line("switch (tag) {"); - for variant in &e.variants { - let vn = cpp_ident(&variant.name); - w.line(format!("case {}: {{", variant.value)); - w.scope(|w| { - if variant.fields.is_empty() { - w.line(format!("return {name}{{{name}::{vn}{{}}}};")); - } else { - let mut inits = Vec::new(); - for f in &variant.fields { - let var = format!("f_{}", f.name); - emit_read_decl(w, &f.ty, &var, "r"); - inits.push(format!("std::move({var})")); - } - w.line(format!( - "return {name}{{{name}::{vn}{{{}}}}};", - inits.join(", ") - )); - } - }); - w.line("}"); - } - w.line("default:"); - w.scope(|w| { - w.line("break;"); - }); - w.line("}"); - w.line(format!( - "throw Error(-3, \"malformed value buffer: unknown {name} tag\");" - )); - }); - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} diff --git a/crates/weaveffi-gen/src/targets/cpp/entities.rs b/crates/weaveffi-gen/src/targets/cpp/entities.rs deleted file mode 100644 index 8dc4ae41..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/entities.rs +++ /dev/null @@ -1,545 +0,0 @@ -//! Declared-entity rendering: plain enums, value types (records and rich -//! enums), typed error domains, and interface RAII classes, plus the -//! dependency ordering that keeps by-value members complete before use. - -use std::collections::HashMap; - -use crate::codegen::common::DocCommentStyle; -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use weaveffi_model::model::Ty; -use weaveffi_model::model::{ - CallShape, EnumBinding, ErrorBinding, FnBinding, InterfaceBinding, ModuleBinding, StructBinding, -}; - -use crate::targets::cpp::calls::{ - iterator_class_name, render_definition, render_iterator_range, render_member_decl, FnKind, -}; -use crate::targets::cpp::codec::emit_read_decl; -use crate::targets::cpp::types::{cpp_error_class, cpp_fn_name, cpp_ident, cpp_type}; - -// ── Namespace: enums ── - -/// Emit one module's plain C-style enums as `enum class {Name} : int32_t`. -/// Rich (algebraic) enums are value types, emitted as variant structs -/// alongside records; this renderer skips them. -pub(crate) fn render_cpp_enums(out: &mut String, module: &ModuleBinding) { - let mut w = CodeWriter::four_space(); - for e in &module.enums { - if e.is_rich() { - continue; - } - w.doc(&e.doc, DocCommentStyle::Javadoc); - w.block(format!("enum class {} : int32_t {{", e.name), "};", |w| { - for (i, v) in e.variants.iter().enumerate() { - w.doc(&v.doc, DocCommentStyle::Javadoc); - let comma = if i + 1 < e.variants.len() { "," } else { "" }; - w.line(format!("{} = {}{}", v.name, v.value, comma)); - } - }); - w.blank(); - } - out.push_str(&w.finish()); -} - -// ── Value types: records and rich enums ── - -/// A value type emitted as a plain C++ struct: a record or a rich (algebraic) -/// enum. Both cross the ABI serialized in value buffers, may nest one -/// another, and are ordered together so a by-value member's type is complete -/// before its holder. -pub(crate) enum ValueDef<'a> { - /// A record: a plain struct with typed members. - Record(&'a StructBinding), - /// A rich enum: a `std::variant`-backed sum type. - Rich(&'a EnumBinding), -} - -impl ValueDef<'_> { - /// The value type's local C++ name. - pub(crate) fn name(&self) -> &str { - match self { - ValueDef::Record(s) => &s.name, - ValueDef::Rich(e) => &e.name, - } - } - - /// Local names of other value types this one holds by value. - pub(crate) fn deps(&self) -> Vec { - let mut deps = Vec::new(); - match self { - ValueDef::Record(s) => { - for f in &s.fields { - collect_value_deps(&f.ty, &mut deps); - } - } - ValueDef::Rich(e) => { - for v in &e.variants { - for f in &v.fields { - collect_value_deps(&f.ty, &mut deps); - } - } - } - } - deps - } -} - -/// Collect the local names of value types (records and rich enums) reachable -/// from `ty`, recursing through optional/list/map wrappers. Interfaces are -/// not collected: every interface class is complete before any value type. -fn collect_value_deps(ty: &Ty, deps: &mut Vec) { - match ty { - Ty::Record(n) | Ty::RichEnum(n) => deps.push(local_type_name(n).to_string()), - Ty::Optional(inner) | Ty::List(inner) => collect_value_deps(inner, deps), - Ty::Map(k, v) => { - collect_value_deps(k, deps); - collect_value_deps(v, deps); - } - _ => {} - } -} - -/// Order entries so that anything an entry depends on is emitted before it. -/// Pure DFS post-order; original walk order is the stable tiebreaker, and -/// the first definition wins when two modules share a local name (the -/// flattened C++ type namespace can't hold duplicates anyway). -pub(crate) fn topo_order(names: &[String], deps: &[Vec]) -> Vec { - fn visit( - i: usize, - deps: &[Vec], - name_to_idx: &HashMap<&str, usize>, - state: &mut [u8], - order: &mut Vec, - ) { - // 0 = unvisited, 1 = on stack (skip to break any cycle), 2 = emitted. - if state[i] != 0 { - return; - } - state[i] = 1; - for d in &deps[i] { - if let Some(&j) = name_to_idx.get(d.as_str()) { - if j != i { - visit(j, deps, name_to_idx, state, order); - } - } - } - state[i] = 2; - order.push(i); - } - - let mut name_to_idx: HashMap<&str, usize> = HashMap::new(); - for (i, n) in names.iter().enumerate() { - name_to_idx.entry(n.as_str()).or_insert(i); - } - let mut state = vec![0u8; names.len()]; - let mut order = Vec::with_capacity(names.len()); - for i in 0..names.len() { - visit(i, deps, &name_to_idx, &mut state, &mut order); - } - order -} - -/// Render a record as a plain C++ value struct: typed members in declaration -/// (and wire) order. An interface-typed member is the RAII wrapper held by -/// value, so copying the record clones the reference and destroying it -/// releases one. -pub(crate) fn render_cpp_record(out: &mut String, s: &StructBinding) { - let mut w = CodeWriter::four_space(); - w.doc(&s.doc, DocCommentStyle::Javadoc); - w.line(format!("struct {} {{", s.name)); - w.scope(|w| { - for f in &s.fields { - w.doc(&f.doc, DocCommentStyle::Javadoc); - w.line(format!("{} {};", cpp_type(&f.ty), cpp_ident(&f.name))); - } - }); - w.line("};"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render a rich (algebraic) enum as a `std::variant`-backed sum type: one -/// payload struct per variant, a `value` member holding the active payload, -/// a nested `Tag` enum mirroring the wire discriminants, and a `tag()` reader. -/// Construct one as `Shape{Shape::Circle{2.0}}`. -pub(crate) fn render_cpp_rich_enum(out: &mut String, e: &EnumBinding) { - let name = &e.name; - let mut w = CodeWriter::four_space(); - w.doc(&e.doc, DocCommentStyle::Javadoc); - w.line(format!("struct {name} {{")); - w.scope(|w| { - w.line(format!( - "/** Discriminant identifying the active variant of `{name}`. */" - )); - w.block("enum class Tag : int32_t {", "};", |w| { - for (i, v) in e.variants.iter().enumerate() { - let comma = if i + 1 < e.variants.len() { "," } else { "" }; - w.line(format!("{} = {}{}", cpp_ident(&v.name), v.value, comma)); - } - }); - w.blank(); - - for v in &e.variants { - w.doc(&v.doc, DocCommentStyle::Javadoc); - w.line(format!("struct {} {{", cpp_ident(&v.name))); - w.scope(|w| { - for f in &v.fields { - w.doc(&f.doc, DocCommentStyle::Javadoc); - w.line(format!("{} {};", cpp_type(&f.ty), cpp_ident(&f.name))); - } - }); - w.line("};"); - w.blank(); - } - - let alts: Vec = e.variants.iter().map(|v| cpp_ident(&v.name)).collect(); - w.line("/** The active variant's payload. */"); - w.line(format!("std::variant<{}> value;", alts.join(", "))); - w.blank(); - w.line("/** The tag of the active variant. */"); - w.line("Tag tag() const {"); - w.scope(|w| { - w.line("static constexpr Tag tags[] = {"); - w.scope(|w| { - for (i, v) in e.variants.iter().enumerate() { - let comma = if i + 1 < e.variants.len() { "," } else { "" }; - w.line(format!("Tag::{}{}", cpp_ident(&v.name), comma)); - } - }); - w.line("};"); - w.line("return tags[value.index()];"); - }); - w.line("}"); - }); - w.line("};"); - w.blank(); - out.push_str(&w.finish()); -} - -// ── Typed error domains ── - -/// Emit one module's typed error domain: a domain exception derived from -/// `Error`, one subclass per declared code (with typed members for -/// any payload fields the code declares), and the per-domain -/// `detail::make_{path}_error`/`detail::check_{path}` helpers that throwing -/// wrappers use to map a nonzero `out_err` to the typed exception, decoding -/// the payload buffer along the way. -/// -/// Domain codes are validated positive-only, and the runtime reserves every -/// negative code (generic error, producer panic, marshalling failure, foreign -/// callback failure, cancellation), so the mapping helper routes negative -/// codes to `detail::make_error` (the generic `Error`, or `Cancelled` for -/// -5) before consulting the domain's cases; unknown positive codes fall back -/// to the domain exception itself. -pub(crate) fn render_domain_error(out: &mut String, eb: &ErrorBinding, prefix: &str) { - let domain = &eb.type_name; - let path = &eb.owner_path; - - let mut w = CodeWriter::four_space(); - w.line(format!( - "/** Typed errors reported by the `{}` module's throwing functions. */", - eb.owner_path - )); - w.line(format!("class {domain} : public Error {{")); - w.line("public:"); - w.scope(|w| { - w.line(format!( - "{domain}(int32_t code, const std::string& msg) : Error(code, msg) {{}}" - )); - }); - w.line("};"); - w.blank(); - - for code in &eb.codes { - let class = cpp_error_class(&code.name); - let doc = code.doc.clone().unwrap_or_else(|| code.message.clone()); - w.doc(&Some(doc), DocCommentStyle::Javadoc); - w.line(format!("class {class} : public {domain} {{")); - w.line("public:"); - w.scope(|w| { - for fld in &code.fields { - w.doc(&fld.doc, DocCommentStyle::Javadoc); - w.line(format!("{} {};", cpp_type(&fld.ty), cpp_ident(&fld.name))); - } - if code.fields.is_empty() { - w.line(format!( - "explicit {class}(const std::string& msg) : {domain}({}, msg) {{}}", - code.value - )); - } else { - w.blank(); - let mut params = vec!["const std::string& msg".to_string()]; - let mut inits = vec![format!("{domain}({}, msg)", code.value)]; - for fld in &code.fields { - let name = cpp_ident(&fld.name); - params.push(format!("{} {name}", cpp_type(&fld.ty))); - inits.push(format!("{name}(std::move({name}))")); - } - w.line(format!( - "{class}({}) : {} {{}}", - params.join(", "), - inits.join(", ") - )); - } - }); - w.line("};"); - w.blank(); - } - - w.line("namespace detail {"); - w.blank(); - w.line(format!( - "/** Map a `{path}` error code and payload to its typed exception (the runtime exception for negative codes, {domain} for unknown codes). */" - )); - w.line(format!( - "inline std::exception_ptr make_{path}_error(int32_t code, const std::string& msg, const uint8_t* payload_ptr, size_t payload_len) {{" - )); - w.scope(|w| { - // The negative range is reserved for the runtime (-1 generic error, - // -2 producer panic, -3 marshalling failure, -4 foreign callback - // failure, -5 cancelled); those never map to a typed domain error, on - // the throwing path or anywhere else. - w.line("if (code < 0) return make_error(code, msg);"); - if eb.codes.iter().all(|c| c.fields.is_empty()) { - w.line("(void)payload_ptr;"); - w.line("(void)payload_len;"); - } - w.line("switch (code) {"); - for code in &eb.codes { - let class = cpp_error_class(&code.name); - if code.fields.is_empty() { - w.line(format!( - "case {}: return std::make_exception_ptr({class}(msg));", - code.value - )); - } else { - w.line(format!("case {}: {{", code.value)); - w.scope(|w| { - w.line("BufferReader payload_r(payload_ptr, payload_len);"); - let mut args = vec!["msg".to_string()]; - for fld in &code.fields { - let var = format!("f_{}", fld.name); - emit_read_decl(w, &fld.ty, &var, "payload_r"); - args.push(format!("std::move({var})")); - } - w.line("payload_r.expect_end();"); - w.line(format!( - "return std::make_exception_ptr({class}({}));", - args.join(", ") - )); - }); - w.line("}"); - } - } - w.line(format!( - "default: return std::make_exception_ptr({domain}(code, msg));" - )); - w.line("}"); - }); - w.line("}"); - w.blank(); - w.line(format!( - "/** Throw the typed `{path}` exception if `err` carries a nonzero code. */" - )); - w.line(format!("inline void check_{path}({prefix}_error& err) {{")); - w.scope(|w| { - w.line("if (err.code == 0) return;"); - w.line("std::string msg = error_message(err);"); - // The payload buffer is owned by the error and released by - // error_clear, so the exception (which decodes it) is built first. - w.line(format!( - "std::exception_ptr ex = make_{path}_error(err.code, msg, err.payload_ptr, err.payload_len);" - )); - w.line(format!("{prefix}_error_clear(&err);")); - w.line("std::rethrow_exception(ex);"); - }); - w.line("}"); - w.blank(); - w.line("} // namespace detail"); - w.blank(); - out.push_str(&w.finish()); -} - -// ── Namespace: interfaces ── - -/// Emit the reference-counting RAII skeleton of an interface class: the -/// `raw_type` alias of the C tag, the tagged constructor adopting one -/// strong reference, a destructor releasing one reference, copy operations -/// that take a new reference through the `_clone` symbol (so C++ copy -/// semantics equal a reference-count clone), move operations that transfer -/// the pointer, and the `handle()`/`clone_handle()` readers the marshalling -/// code uses to borrow or to mint a second reference. -fn emit_raii_skeleton(w: &mut CodeWriter, i: &InterfaceBinding) { - let name = &i.name; - let tag = &i.c_tag; - let clone = &i.clone_symbol; - let destroy = &i.destroy_symbol; - - w.line("/** The C type this class wraps. */"); - w.line(format!("using raw_type = {tag};")); - w.blank(); - - w.line(format!( - "/** Adopts one strong reference to a producer object: `{name}(adopt, raw)`. */" - )); - w.line(format!( - "explicit {name}(adopt_t, {tag}* h) noexcept : handle_(h) {{}}" - )); - w.blank(); - - w.line("/** Releases this wrapper's reference; the object is dropped with its last one. */"); - w.line(format!("~{name}() {{")); - w.scope(|w| { - w.line(format!("if (handle_) {destroy}(handle_);")); - }); - w.line("}"); - w.blank(); - - w.line("/** Copies share the object: the copy takes a new strong reference. */"); - w.line(format!( - "{name}(const {name}& other) : handle_({clone}(other.handle_)) {{}}" - )); - w.blank(); - - w.line(format!("{name}& operator=(const {name}& other) {{")); - w.scope(|w| { - w.line("if (this != &other) {"); - w.scope(|w| { - w.line(format!("{tag}* h = {clone}(other.handle_);")); - w.line(format!("if (handle_) {destroy}(handle_);")); - w.line("handle_ = h;"); - }); - w.line("}"); - w.line("return *this;"); - }); - w.line("}"); - w.blank(); - - w.line(format!( - "{name}({name}&& other) noexcept : handle_(other.handle_) {{" - )); - w.scope(|w| { - w.line("other.handle_ = nullptr;"); - }); - w.line("}"); - w.blank(); - - w.line(format!("{name}& operator=({name}&& other) noexcept {{")); - w.scope(|w| { - w.line("if (this != &other) {"); - w.scope(|w| { - w.line(format!("if (handle_) {destroy}(handle_);")); - w.line("handle_ = other.handle_;"); - w.line("other.handle_ = nullptr;"); - }); - w.line("}"); - w.line("return *this;"); - }); - w.line("}"); - w.blank(); - - w.line("/** The wrapped pointer, borrowed: this wrapper keeps its reference. */"); - w.line(format!("const {tag}* handle() const {{ return handle_; }}")); - w.blank(); - - w.line( - "/** A new strong reference the caller owns (for example to write into a value buffer). */", - ); - w.line(format!( - "{tag}* clone_handle() const {{ return {clone}(handle_); }}" - )); - w.blank(); -} - -/// The C++ name and declaration kind of one interface member. The constructor -/// named `new` becomes the canonical C++ constructor; every other constructor -/// becomes a static factory named after it. -fn member_kinds<'a>(i: &'a InterfaceBinding) -> Vec<(&'a FnBinding, String, FnKind<'a>)> { - let class = i.name.as_str(); - let mut members = Vec::new(); - for c in &i.constructors { - if c.name == "new" && matches!(c.shape, CallShape::Sync(_)) { - members.push((c, class.to_string(), FnKind::Ctor { class })); - } else { - members.push((c, cpp_fn_name(&c.name), FnKind::Static { class })); - } - } - for m in &i.methods { - members.push((m, cpp_fn_name(&m.name), FnKind::Method { class })); - } - for s in &i.statics { - members.push((s, cpp_fn_name(&s.name), FnKind::Static { class })); - } - members -} - -/// Emit the forward declarations an interface needs before any class body: -/// the wrapper class itself and the range class of every iterator-returning -/// member, so member declarations can name them as return types. -pub(crate) fn render_cpp_interface_forward_decls(out: &mut String, i: &InterfaceBinding) { - out.push_str(&format!("class {};\n", i.name)); - for (f, _, kind) in member_kinds(i) { - if matches!(f.shape, CallShape::Iterator(_)) { - out.push_str(&format!("class {};\n", iterator_class_name(f, kind))); - } - } -} - -/// Render an interface's class definition: the reference-counting RAII -/// skeleton plus the *declarations* of its constructors, methods, and -/// statics. Member bodies are emitted later by -/// [`render_cpp_interface_members`], once every value type and codec they -/// marshal through is complete; the class itself must be complete first so -/// records can hold it by value. -pub(crate) fn render_cpp_interface_class(out: &mut String, i: &InterfaceBinding) { - let name = &i.name; - let mut w = CodeWriter::four_space(); - w.doc(&i.doc, DocCommentStyle::Javadoc); - w.line(format!("class {name} {{")); - w.scope(|w| { - w.line(format!("{}* handle_;", i.c_tag)); - w.blank(); - }); - w.line("public:"); - w.scope(|w| emit_raii_skeleton(w, i)); - let mut members = String::new(); - for (f, cpp_name, kind) in member_kinds(i) { - render_member_decl(&mut members, f, &cpp_name, kind); - } - w.raw(members); - w.line("};"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render the range classes of an interface's iterator-returning members at -/// namespace scope. They need every element type complete, so they follow -/// the value types and precede the member definitions that construct them. -pub(crate) fn render_cpp_interface_iterators( - out: &mut String, - i: &InterfaceBinding, - module: &ModuleBinding, - prefix: &str, -) { - for (f, cpp_name, kind) in member_kinds(i) { - if let CallShape::Iterator(it) = &f.shape { - render_iterator_range(out, f, it, &cpp_name, kind, module, prefix); - } - } -} - -/// Render the out-of-line `inline` definitions of an interface's members. -/// Methods pass the wrapped pointer as the leading C argument; constructors -/// adopt the returned reference; sync, async, and iterator shapes reuse the -/// free-function marshalling paths. -pub(crate) fn render_cpp_interface_members( - out: &mut String, - i: &InterfaceBinding, - module: &ModuleBinding, - prefix: &str, -) { - for (f, cpp_name, kind) in member_kinds(i) { - render_definition(out, f, &cpp_name, kind, module, prefix); - } -} diff --git a/crates/weaveffi-gen/src/targets/cpp/mod.rs b/crates/weaveffi-gen/src/targets/cpp/mod.rs deleted file mode 100644 index 56ad94da..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/mod.rs +++ /dev/null @@ -1,465 +0,0 @@ -//! C++ wrapper generator. -//! -//! Produces an idiomatic header-only C++17 library over the C ABI: the C -//! header `{library}.h` (the same file the [`c`](super::c) target emits), a -//! wrapper header `{library}.hpp` that includes it, a `CMakeLists.txt` -//! exporting an INTERFACE target, and a README. Everything lives in -//! `namespace {prefix}` by default. Implements [`LanguageBackend`]; the -//! shared driver bridges it into the generator pipeline. -//! -//! The generated surface follows C ABI revision 3: -//! -//! * Strings cross as a UTF-8 pointer and length: parameters are -//! `std::string_view`, returns are `std::string` copied out of the -//! producer's allocation and released with `{prefix}_free_bytes`. -//! * Records are plain C++ value structs with typed members; rich (algebraic) -//! enums are `std::variant`-backed sum types with one payload struct per -//! variant. Neither has any C symbols: values cross the ABI serialized in -//! the value-buffer format as one `(const uint8_t*, size_t)` pair, through -//! a small private reader and writer in `detail` plus one generated pack -//! and unpack routine per type. -//! * Interfaces are reference-counted objects owned by the producer. Each -//! becomes an RAII class holding one strong reference: the destructor calls -//! the `_destroy` symbol exactly once, copying calls `_clone`, and moving -//! transfers the pointer. Adopting a raw pointer takes the `adopt` tag, so -//! no integer or pointer converts into an object by accident. A top-level -//! object parameter is borrowed for the call; a returned object, an async -//! result, and an iterator element are adopted. `Interface?` maps to -//! `std::optional`. An interface inside a value buffer crosses as -//! a `u64` token minted with `_clone` on write and adopted on read. -//! * Callback interfaces become abstract classes. The consumer passes a -//! `std::shared_ptr`; the wrapper boxes it on the heap as `ctx` and -//! hands the producer the process-wide static vtable, whose `free` deletes -//! the box. Trampolines report any exception through `{prefix}_error_set` -//! with the foreign-error code -4 instead of unwinding through the C frame. -//! * Free functions live in a nested namespace per IDL module -//! (`kvstore::kv::stats::get_stats`). -//! * An `iter` callable returns a move-only lazy range class that pulls -//! one element per iteration step and releases the producer iterator from -//! its destructor (or eagerly on exhaustion). -//! * Async callables return `std::future`. A cancellable one takes a -//! trailing `const CancelToken&` (an RAII wrapper over the native token); -//! a cancelled call settles the future with `Cancelled`. -//! * Each declaring module's error domain becomes an exception type derived -//! from `Error`, with one subclass per code. Negative runtime codes surface -//! as the generic `Error` (or `Cancelled` for -5), never a typed domain -//! exception. -//! * `check_library()` verifies the producer's ABI revision and every -//! top-level module's contract checksum once, and throws `LoadError` on a -//! mismatch. Every free function, constructor, and static member calls it. - -mod callbacks; -mod calls; -mod codec; -mod entities; -mod package; -mod runtime; -mod types; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; -use crate::targets::c::{header_name as c_header_name, render_c_header_from_model}; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use camino::Utf8Path; -use serde::{Deserialize, Serialize}; -use weaveffi_model::model::{BindingModel, EnumBinding, ModuleBinding}; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; - -use crate::targets::cpp::callbacks::{render_callback_class, render_callback_trampolines}; -use crate::targets::cpp::calls::render_cpp_module_ns; -use crate::targets::cpp::entities::{ - render_cpp_enums, render_cpp_interface_class, render_cpp_interface_forward_decls, - render_cpp_interface_iterators, render_cpp_interface_members, render_cpp_record, - render_cpp_rich_enum, render_domain_error, topo_order, ValueDef, -}; -use crate::targets::cpp::package::{ - render_cmake, render_packaged_cmake, render_packaged_readme, render_readme, CmakeNames, -}; -use crate::targets::cpp::runtime::{render_buffer_runtime, render_prelude_runtime}; - -/// Per-target configuration for [`CppGenerator`]. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct CppConfig { - /// C++ namespace holding every generated declaration (default: the - /// package's C prefix). - pub namespace: Option, - /// Filename of the wrapper header (default `{library}.hpp`). - pub header_name: Option, - /// C++ standard advertised in the generated `CMakeLists.txt` (default - /// `"17"`). - pub standard: Option, - /// Basename of the IDL the CLI was invoked with. - #[serde(skip)] - pub input_basename: Option, -} - -impl CppConfig { - /// Returns the configured C++ namespace, falling back to the identity's - /// C prefix. - pub fn namespace(&self, identity: &Identity) -> String { - self.namespace - .clone() - .unwrap_or_else(|| identity.prefix.clone()) - } - - /// Returns the wrapper header's filename, falling back to - /// `{library}.hpp`. - pub fn header_name(&self, identity: &Identity) -> String { - self.header_name - .clone() - .unwrap_or_else(|| format!("{}.hpp", identity.library)) - } - - /// Returns the C++ standard advertised in the generated `CMakeLists.txt`, - /// falling back to `"17"`. - pub fn standard(&self) -> &str { - self.standard.as_deref().unwrap_or("17") - } - - /// Returns the input IDL basename embedded in generated file headers, - /// falling back to `"api.yml"`. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } -} - -/// C++ backend: emits a wrapper header (`{library}.hpp` by default), a copy -/// of the C header it includes, a `CMakeLists.txt`, and a README. -pub struct CppGenerator; - -/// The names every rendered file agrees on, resolved once per run. -struct Names { - namespace: String, - header: String, - c_header: String, -} - -impl Names { - fn new(api: &ResolvedApi, config: &CppConfig) -> Self { - Self { - namespace: config.namespace(api.identity()), - header: config.header_name(api.identity()), - c_header: c_header_name(api), - } - } - - fn cmake<'a>(&'a self, identity: &'a Identity, config: &'a CppConfig) -> CmakeNames<'a> { - CmakeNames { - identity, - namespace: &self.namespace, - header: &self.header, - standard: config.standard(), - input_basename: config.input_basename(), - } - } -} - -impl LanguageBackend for CppGenerator { - type Config = CppConfig; - - fn name(&self) -> &'static str { - "cpp" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let dir = out_dir.join("cpp"); - let identity = api.identity(); - let names = Names::new(api, config); - let input_basename = config.input_basename(); - let cmake = names.cmake(identity, config); - vec![ - OutputFile::new( - dir.join(&names.c_header), - render_c_header_from_model(model, input_basename, &names.c_header), - ), - OutputFile::new( - dir.join(&names.header), - render_cpp_header( - model, - identity, - &names.namespace, - &names.c_header, - input_basename, - &names.header, - ), - ), - OutputFile::new(dir.join("CMakeLists.txt"), render_cmake(&cmake)), - OutputFile::new(dir.join("README.md"), render_readme(&cmake)), - ] - } - - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let dir = out_dir.join("cpp"); - let identity = api.identity(); - let names = Names::new(api, config); - let input_basename = config.input_basename(); - let cmake = names.cmake(identity, config); - let lib = &ctx.binaries.lib_name; - - let include = dir.join("include"); - let mut files = vec![ - PackagedFile::text( - include.join(&names.c_header), - render_c_header_from_model(model, input_basename, &names.c_header), - ), - PackagedFile::text( - include.join(&names.header), - render_cpp_header( - model, - identity, - &names.namespace, - &names.c_header, - input_basename, - &names.header, - ), - ), - PackagedFile::text( - dir.join("CMakeLists.txt"), - render_packaged_cmake(&cmake, lib), - ), - PackagedFile::text(dir.join("README.md"), render_packaged_readme(&cmake, ctx)), - ]; - // The packaged CMake selects among the desktop platforms only, so - // Android and Wasm binaries (which belong to other ecosystems' - // packages) are not bundled here. - for nb in ctx - .binaries - .binaries - .iter() - .filter(|nb| nb.platform.is_desktop()) - { - let dest = dir - .join("lib") - .join(nb.platform.id()) - .join(ctx.binaries.bundled_filename(nb.platform)); - files.push(PackagedFile::copy(dest, nb.source.clone())); - } - Some(files) - } -} - -/// Render the complete C++ wrapper header from the driver-built binding -/// model. `c_header` is the C header file it includes. -/// -/// Layout inside `namespace {namespace}`, in an order that keeps every type -/// complete before it is held by value or marshalled: -/// -/// 1. the runtime: error types, the adopt tag, `CancelToken`, string helpers, -/// `check_library()`, and the private value-buffer codec (when any -/// buffered value crosses the ABI); -/// 2. plain enums; -/// 3. forward declarations of every value type, interface class, -/// callback-interface class, and member iterator range class; -/// 4. interface class definitions (the reference-counting skeleton plus -/// member *declarations*), so records can hold objects by value; -/// 5. value types (record structs and rich-enum variants) in dependency -/// order with their pack/unpack routines; -/// 6. typed exception domains; -/// 7. callback-interface abstract classes and their trampolines and static -/// vtables; -/// 8. the range classes of iterator-returning interface members; -/// 9. the out-of-line definitions of every interface member; and -/// 10. one nested namespace per module holding its free functions. -pub(crate) fn render_cpp_header( - model: &BindingModel, - identity: &Identity, - namespace: &str, - c_header: &str, - input_basename: &str, - filename: &str, -) -> String { - let prefix = model.prefix.as_str(); - let needs_buffers = model.has_buffers(); - let has_rich_enums = model - .modules - .iter() - .any(|m| m.enums.iter().any(EnumBinding::is_rich)); - let mut out = String::new(); - - out.push_str(&render_prelude(CommentStyle::DoubleSlash, input_basename)); - out.push_str("#pragma once\n\n"); - out.push_str(&format!("#include \"{c_header}\"\n\n")); - let mut includes = vec![ - "cstddef", - "cstdint", - "exception", - "memory", - "new", - "optional", - "stdexcept", - "string", - "string_view", - "unordered_map", - "utility", - "vector", - ]; - if has_rich_enums { - includes.push("variant"); - } - if needs_buffers { - // The buffer runtime needs memcpy for float bits. - includes.push("cstring"); - } - if model.has_async() { - includes.push("future"); - } - if model.has_iterators() { - // The lazy range classes need std::input_iterator_tag. - includes.push("iterator"); - } - includes.sort_unstable(); - for inc in includes { - out.push_str(&format!("#include <{inc}>\n")); - } - out.push('\n'); - - out.push_str(&format!("namespace {namespace} {{\n\n")); - out.push_str(&render_prelude_runtime(model, identity)); - if needs_buffers { - out.push_str(&render_buffer_runtime(prefix)); - } - - // Enums first: they reference no other types and are used by value. - for module in &model.modules { - render_cpp_enums(&mut out, module); - } - - // Forward declarations let interface member declarations name any value - // type, interface, callback interface, or range class as a parameter or - // return type before its definition. - let forward_start = out.len(); - for module in &model.modules { - for s in &module.structs { - out.push_str(&format!("struct {};\n", s.name)); - } - for e in module.enums.iter().filter(|e| e.is_rich()) { - out.push_str(&format!("struct {};\n", e.name)); - } - for i in &module.interfaces { - render_cpp_interface_forward_decls(&mut out, i); - } - for cb in &module.callback_interfaces { - out.push_str(&format!("class {};\n", cb.name)); - } - } - if out.len() > forward_start { - out.push('\n'); - } - - // Interface classes: the RAII skeleton and member declarations only. A - // record may hold an object by value, so the class must be complete - // before the value types; the member bodies, which marshal value types, - // follow those. - for module in &model.modules { - for i in &module.interfaces { - render_cpp_interface_class(&mut out, i); - } - } - - // Value types (records and rich enums) in dependency order: a member of - // record type is held by value, which requires the member's type to be - // complete, so nested types are emitted first. The pack/unpack routines - // follow in the same order so a codec can call the codecs of the types it - // nests. - let value_entries: Vec = model - .modules - .iter() - .flat_map(|m: &ModuleBinding| { - let records = m.structs.iter().map(ValueDef::Record); - let rich = m.enums.iter().filter(|e| e.is_rich()).map(ValueDef::Rich); - records.chain(rich) - }) - .collect(); - let value_order = topo_order( - &value_entries - .iter() - .map(|v| v.name().to_string()) - .collect::>(), - &value_entries.iter().map(ValueDef::deps).collect::>(), - ); - for &idx in &value_order { - match &value_entries[idx] { - ValueDef::Record(s) => render_cpp_record(&mut out, s), - ValueDef::Rich(e) => render_cpp_rich_enum(&mut out, e), - } - } - if !value_entries.is_empty() { - out.push_str("namespace detail {\n\n"); - for &idx in &value_order { - match &value_entries[idx] { - ValueDef::Record(s) => codec::render_record_codec(&mut out, s), - ValueDef::Rich(e) => codec::render_rich_enum_codec(&mut out, e), - } - } - out.push_str("} // namespace detail\n\n"); - } - - // Typed error domains come after the value types: a code's payload fields - // may hold records, and the domain's decode helper calls their codecs. - for m in &model.modules { - if m.declares_error() { - let eb = m.error.as_ref().expect("declares_error implies Some"); - render_domain_error(&mut out, eb, prefix); - } - } - - // Callback interfaces: the abstract class, then the trampolines (which - // decode record arguments and adopt object arguments) and the static - // vtable in `detail`. - for m in &model.modules { - for cb in &m.callback_interfaces { - render_callback_class(&mut out, cb); - render_callback_trampolines(&mut out, cb, prefix); - } - } - - // Range classes of iterator-returning members need their element types - // complete and are constructed by the member definitions that follow. - for m in &model.modules { - for i in &m.interfaces { - render_cpp_interface_iterators(&mut out, i, m, prefix); - } - } - - // Interface member definitions: every type they accept, return, or - // marshal through is now complete. - for m in &model.modules { - for i in &m.interfaces { - render_cpp_interface_members(&mut out, i, m, prefix); - } - } - - // Module namespaces last: every type is defined, so a function may accept - // or return any of them by value. Functions get bare snake_case names - // inside `namespace {module path}`. - for module in &model.modules { - render_cpp_module_ns(&mut out, module, prefix); - } - out.push_str(&format!("}} // namespace {namespace}\n\n")); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, filename)); - - out -} - -#[cfg(test)] -mod tests; diff --git a/crates/weaveffi-gen/src/targets/cpp/runtime.rs b/crates/weaveffi-gen/src/targets/cpp/runtime.rs deleted file mode 100644 index 5967ac96..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/runtime.rs +++ /dev/null @@ -1,46 +0,0 @@ -//! The fixed runtime of the generated header: the error types, the adopt -//! tag, the cancel token, the string helpers, the library check, and the -//! value-buffer reader and writer. -//! -//! The code lives in real C++ sources under `runtime/`, spliced into the -//! header with `{{PLACEHOLDER}}` substitution. - -use weaveffi_model::model::{checksum_symbol, BindingModel}; -use weaveffi_model::pkg::Identity; - -/// The error types, adopt tag, cancel token, string helpers, and -/// `check_library()`. -const PRELUDE: &str = include_str!("runtime/prelude.hpp"); - -/// The value-buffer reader, writer, and release guard. -const BUFFER: &str = include_str!("runtime/buffer.hpp"); - -/// Render the runtime prelude for `model`, the binding model of the library -/// `identity` describes. `check_library()` compares the producer's ABI revision and -/// every top-level module's contract checksum against the values this header -/// was generated with. -pub(crate) fn render_prelude_runtime(model: &BindingModel, identity: &Identity) -> String { - let prefix = &model.prefix; - let library = &identity.library; - let mut checks = String::new(); - for root in model.roots() { - let Some(sum) = root.checksum else { continue }; - let symbol = checksum_symbol(prefix, &root.path); - checks.push_str(&format!( - " if ({symbol}() != {sum:#018x}ull) {{\n \ - return std::string(\"{library}: module '{name}' does not match the linked library \ - (contract checksum mismatch); regenerate the bindings for this build\");\n }}\n", - name = root.dot_path, - )); - } - PRELUDE - .replace("{{PREFIX}}", prefix) - .replace("{{MACRO}}", &identity.macro_prefix()) - .replace("{{LIBRARY}}", library) - .replace("{{CHECKSUMS}}", &checks) -} - -/// Render the value-buffer runtime. -pub(crate) fn render_buffer_runtime(prefix: &str) -> String { - BUFFER.replace("{{PREFIX}}", prefix) -} diff --git a/crates/weaveffi-gen/src/targets/cpp/tests.rs b/crates/weaveffi-gen/src/targets/cpp/tests.rs deleted file mode 100644 index add27499..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/tests.rs +++ /dev/null @@ -1,755 +0,0 @@ -//! Unit tests: render a small API that exercises every C ABI revision 3 -//! shape and assert the header carries the key pieces of each contract. - -use camino::Utf8Path; -use weaveffi_model::ir::{ - Api, CallbackInterfaceDef, EnumDef, EnumVariant, ErrorCode, ErrorDomain, Function, - InterfaceDef, Module, Param, StructDef, StructField, TypeRef, -}; -use weaveffi_model::model::{BindingModel, CallShape}; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; -use weaveffi_model::validate::validate_api; - -use crate::backend::LanguageBackend; -use crate::targets::cpp::{render_cpp_header, CppConfig, CppGenerator}; - -fn param(name: &str, ty: TypeRef) -> Param { - Param { - name: name.into(), - ty, - doc: None, - } -} - -fn func(name: &str, params: Vec, returns: Option) -> Function { - Function { - name: name.into(), - params, - returns, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - } -} - -fn field(name: &str, ty: TypeRef) -> StructField { - StructField { - name: name.into(), - ty, - doc: None, - } -} - -fn named(name: &str) -> TypeRef { - TypeRef::Named(name.into()) -} - -fn optional(ty: TypeRef) -> TypeRef { - TypeRef::Optional(Box::new(ty)) -} - -/// One module covering: an interface with a constructor, methods, a static, -/// an iterator of objects, and an async object result; a function taking and -/// returning `Interface?`; a record with `Interface`, `[Interface]`, and -/// `Interface?` fields; a callback interface with string, i32, record, -/// object, nullable object, and enum parameters and `bool`, void, and enum -/// returns; and a function taking that callback interface. -fn fixture() -> Api { - let store = InterfaceDef { - name: "Store".into(), - doc: Some("A reference-counted key-value store.".into()), - deprecated: None, - constructors: vec![func("new", vec![param("path", TypeRef::StringUtf8)], None)], - methods: vec![ - func( - "get", - vec![param("key", TypeRef::StringUtf8)], - Some(optional(TypeRef::StringUtf8)), - ), - func("sibling", vec![], Some(optional(named("Store")))), - func( - "scan", - vec![], - Some(TypeRef::Iterator(Box::new(named("Store")))), - ), - Function { - throws: true, - ..func("save", vec![param("contact", named("Contact"))], None) - }, - Function { - r#async: true, - ..func("fetch", vec![], Some(named("Store"))) - }, - ], - statics: vec![func("open_default", vec![], Some(named("Store")))], - }; - let listener = CallbackInterfaceDef { - name: "Listener".into(), - doc: Some("Receives store events.".into()), - deprecated: None, - methods: vec![ - func( - "on_message", - vec![ - param("text", TypeRef::StringUtf8), - param("weight", TypeRef::I32), - param("contact", named("Contact")), - param("store", named("Store")), - ], - Some(TypeRef::Bool), - ), - func( - "on_reset", - vec![param("alt", optional(named("Store")))], - None, - ), - func( - "pick", - vec![param("color", named("Color"))], - Some(named("Color")), - ), - ], - }; - Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![Module { - name: "kv".into(), - doc: None, - functions: vec![ - func( - "lookup", - vec![param("store", optional(named("Store")))], - Some(optional(named("Store"))), - ), - func( - "subscribe", - vec![param("listener", named("Listener"))], - None, - ), - func( - "all_stores", - vec![], - Some(TypeRef::Iterator(Box::new(named("Store")))), - ), - func( - "first", - vec![param("stores", TypeRef::List(Box::new(named("Store"))))], - Some(named("Store")), - ), - ], - interfaces: vec![store], - callback_interfaces: vec![listener], - structs: vec![StructDef { - name: "Contact".into(), - doc: None, - deprecated: None, - fields: vec![ - field("name", TypeRef::StringUtf8), - field("store", named("Store")), - field("mirrors", TypeRef::List(Box::new(named("Store")))), - field("primary", optional(named("Store"))), - ], - }], - enums: vec![EnumDef { - name: "Color".into(), - doc: None, - deprecated: None, - variants: vec![ - EnumVariant { - name: "Red".into(), - value: 0, - doc: None, - fields: vec![], - }, - EnumVariant { - name: "Blue".into(), - value: 1, - doc: None, - fields: vec![], - }, - ], - }], - errors: Some(ErrorDomain { - name: "KvError".into(), - codes: vec![ErrorCode { - name: "NOT_FOUND".into(), - code: 1, - message: "not found".into(), - doc: None, - fields: vec![], - }], - }), - modules: vec![], - }], - } -} - -/// Render the wrapper header for `api` with the default names (namespace and -/// files named after the identity, `api` for an API built in memory). -fn render_api(api: &ResolvedApi) -> (BindingModel, String) { - let model = BindingModel::build(api); - let id = api.identity(); - let header = render_cpp_header( - &model, - id, - &id.prefix, - &format!("{}.h", id.library), - "kv.yml", - &format!("{}.hpp", id.library), - ); - (model, header) -} - -fn render() -> (BindingModel, String) { - render_api(&validate_api(fixture(), None).expect("fixture validates")) -} - -fn assert_contains(header: &str, needle: &str) { - assert!( - header.contains(needle), - "expected header to contain {needle:?}\n---\n{header}" - ); -} - -fn assert_before(header: &str, first: &str, second: &str) { - let a = header - .find(first) - .unwrap_or_else(|| panic!("missing {first:?}")); - let b = header - .find(second) - .unwrap_or_else(|| panic!("missing {second:?}")); - assert!(a < b, "{first:?} must precede {second:?}"); -} - -#[test] -fn interface_wrapper_is_reference_counted_raii() { - let (_, h) = render(); - assert_contains(&h, "class Store {"); - assert_contains(&h, "api_kv_Store* handle_;"); - assert_contains(&h, "using raw_type = api_kv_Store;"); - assert_contains( - &h, - "explicit Store(adopt_t, api_kv_Store* h) noexcept : handle_(h) {}", - ); - assert_contains(&h, "if (handle_) api_kv_Store_destroy(handle_);"); - assert_contains( - &h, - "Store(const Store& other) : handle_(api_kv_Store_clone(other.handle_)) {}", - ); - assert_contains( - &h, - "Store(Store&& other) noexcept : handle_(other.handle_) {", - ); - assert_contains(&h, "const api_kv_Store* handle() const { return handle_; }"); - assert_contains( - &h, - "api_kv_Store* clone_handle() const { return api_kv_Store_clone(handle_); }", - ); - // Constructor adopts the returned reference; methods borrow `handle_`. - assert_contains(&h, " explicit Store(std::string_view path);"); - assert_contains( - &h, - "inline Store::Store(std::string_view path) : handle_(nullptr) {\n check_library();", - ); - assert_contains( - &h, - "auto result = api_kv_Store_new(detail::utf8(path), path.size(), &err);", - ); - assert_contains(&h, "handle_ = result;"); - assert_contains( - &h, - "inline std::optional Store::get(std::string_view key) const {\n size_t out_len = 0;", - ); - assert_contains( - &h, - "api_kv_Store_get(handle_, detail::utf8(key), key.size(), &out_len, &err);", - ); - assert_contains(&h, "static Store open_default();"); - assert_contains( - &h, - "inline Store Store::open_default() {\n check_library();", - ); - // The throwing method routes through the typed domain check. - assert_contains( - &h, - "inline void Store::save(const Contact& contact) const {", - ); - assert_contains(&h, "detail::check_kv(err);"); - // Async object results are adopted. - assert_contains(&h, "inline std::future Store::fetch() const {"); - assert_contains(&h, "p->set_value(Store(adopt, result));"); -} - -#[test] -fn nullable_objects_map_to_optional() { - let (_, h) = render(); - assert_contains( - &h, - "inline std::optional lookup(const std::optional& store) {\n check_library();", - ); - assert_contains( - &h, - "api_kv_lookup(store.has_value() ? store->handle() : nullptr, &err);", - ); - assert_contains(&h, "if (!result) return std::nullopt;"); - assert_contains(&h, "return Store(adopt, result);"); - assert_contains(&h, "std::optional sibling() const;"); -} - -#[test] -fn records_hold_objects_as_cloned_tokens() { - let (_, h) = render(); - assert_contains(&h, "struct Contact {"); - assert_contains(&h, " Store store;"); - assert_contains(&h, " std::vector mirrors;"); - assert_contains(&h, " std::optional primary;"); - // The wrapper class is complete before the record that holds it, and - // the member bodies come after the record's codec. - assert_before(&h, "class Store {", "struct Contact {"); - assert_before(&h, "inline Contact read_Contact(", "inline Store::Store("); - // Writing mints a fresh reference; reading adopts the token. - assert_contains( - &h, - "w.write_u64(static_cast(reinterpret_cast(v.store.clone_handle())));", - ); - assert_contains( - &h, - "w.write_u64(static_cast(reinterpret_cast(item0.clone_handle())));", - ); - assert_contains( - &h, - "Store f_store = Store(adopt, reinterpret_cast(static_cast(r.read_u64())));", - ); - assert_contains( - &h, - "return Contact{std::move(f_name), std::move(f_store), std::move(f_mirrors), std::move(f_primary)};", - ); - // A list of objects as a parameter is encoded the same way. - assert_contains(&h, "inline Store first(const std::vector& stores) {"); - assert_contains(&h, "stores_buf.write_len(stores.size());"); -} - -#[test] -fn iterators_over_objects_adopt_each_element() { - let (model, h) = render(); - let scan = &model.modules[0].interfaces[0].methods[2]; - let CallShape::Iterator(it) = &scan.shape else { - panic!("scan is an iterator"); - }; - assert_contains(&h, "class StoreScanIterator;"); - assert_contains(&h, "class StoreScanIterator {"); - assert_contains(&h, &format!("{}* handle_;", it.iter_tag)); - assert_contains(&h, &format!("if (handle_) {}(handle_);", it.destroy_symbol)); - assert_contains(&h, "std::optional next() {"); - assert_contains(&h, "return Store(adopt, item);"); - assert_contains(&h, "StoreScanIterator scan() const;"); - assert_contains(&h, "inline StoreScanIterator Store::scan() const {"); - assert_contains(&h, "class AllStoresIterator {"); - assert_contains(&h, "inline AllStoresIterator all_stores() {"); - assert_before( - &h, - "class StoreScanIterator {", - "inline StoreScanIterator Store::scan()", - ); -} - -#[test] -fn callback_interface_renders_abstract_class_vtable_and_trampolines() { - let (_, h) = render(); - assert_contains(&h, "class Listener {"); - assert_contains(&h, "virtual ~Listener() = default;"); - assert_contains( - &h, - "virtual bool on_message(std::string_view text, int32_t weight, const Contact& contact, Store store) = 0;", - ); - assert_contains(&h, "virtual void on_reset(std::optional alt) = 0;"); - assert_contains(&h, "virtual Color pick(Color color) = 0;"); - - // Trampolines carry the exact vtable entry signatures. - assert_contains(&h, "struct Listener_trampolines {"); - assert_contains( - &h, - "static bool on_message(void* ctx, const uint8_t* text_ptr, size_t text_len, int32_t weight, const uint8_t* contact_ptr, size_t contact_len, api_kv_Store* store, api_error* out_err) {", - ); - assert_contains( - &h, - "static void on_reset(void* ctx, api_kv_Store* alt, api_error* out_err) {", - ); - assert_contains( - &h, - "static api_kv_Color pick(void* ctx, api_kv_Color color, api_error* out_err) {", - ); - // Objects are adopted before anything that can throw; buffers and - // strings are decoded (borrowed, never freed). - assert_contains(&h, "Store store_val(adopt, store);"); - assert_before( - &h, - "Store store_val(adopt, store);", - "Listener& impl = **static_cast*>(ctx);", - ); - assert_contains(&h, "std::optional alt_val;"); - assert_contains(&h, "if (alt) alt_val.emplace(adopt, alt);"); - assert_contains( - &h, - "std::string_view text_val = detail::borrow_string(text_ptr, text_len);", - ); - assert_contains( - &h, - "detail::BufferReader contact_r(contact_ptr, contact_len);", - ); - assert_contains(&h, "Contact contact_val = detail::read_Contact(contact_r);"); - assert_contains( - &h, - "return impl.on_message(text_val, weight, contact_val, std::move(store_val));", - ); - assert_contains(&h, "impl.on_reset(std::move(alt_val));"); - assert_contains( - &h, - "return static_cast(static_cast(impl.pick(static_cast(static_cast(color)))));", - ); - // Failure path: foreign error code -4, default return, no unwinding. - assert_contains(&h, "} catch (const std::exception& e) {"); - assert_contains(&h, "api_error_set(out_err, -4, e.what());"); - assert_contains(&h, "return bool{};"); - assert_contains(&h, "return api_kv_Color{};"); - assert!(!h.contains("api_free_bytes(const_cast(contact_ptr)")); - - // Exactly one process-wide vtable, methods in declaration order then free. - assert_eq!( - h.matches("static const api_kv_Listener_vtable vtable = {") - .count(), - 1 - ); - assert_contains( - &h, - "inline const api_kv_Listener_vtable& Listener_vtable() {", - ); - let vtable_start = h - .find("static const api_kv_Listener_vtable vtable = {") - .unwrap(); - let vtable = &h[vtable_start..]; - let order: Vec = [ - "&Listener_trampolines::on_message,", - "&Listener_trampolines::on_reset,", - "&Listener_trampolines::pick,", - "&Listener_trampolines::free_ctx,", - ] - .iter() - .map(|entry| { - vtable - .find(entry) - .unwrap_or_else(|| panic!("missing {entry}")) - }) - .collect(); - assert!(order.windows(2).all(|w| w[0] < w[1]), "{order:?}"); - assert_contains(&h, "delete static_cast*>(ctx);"); -} - -#[test] -fn passing_a_callback_interface_boxes_the_shared_ptr() { - let (_, h) = render(); - assert_contains( - &h, - "inline void subscribe(std::shared_ptr listener) {", - ); - assert_contains( - &h, - "if (!listener) throw std::invalid_argument(\"listener: null callback interface\");", - ); - assert_contains( - &h, - "auto listener_ctx = std::make_unique>(std::move(listener));", - ); - assert_contains( - &h, - "api_kv_subscribe(static_cast(listener_ctx.release()), &detail::Listener_vtable(), &err);", - ); -} - -#[test] -fn runtime_surface_matches_abi_revision_3() { - let (model, h) = render(); - // The C ABI comes from the shipped C header, never re-declared. - assert_contains(&h, "#include \"api.h\""); - assert!( - !h.contains("extern \"C\""), - "C declarations are not inlined" - ); - assert_contains(&h, "namespace api {"); - // The library check covers the ABI revision and every root checksum. - let sum = model.modules[0].checksum.expect("roots carry a checksum"); - assert_contains(&h, "inline void check_library() {"); - assert_contains(&h, "if (abi != API_ABI_VERSION) {"); - assert_contains(&h, &format!("if (api_kv_checksum() != {sum:#018x}ull) {{")); - assert_contains(&h, "module 'kv' does not match the linked library"); - assert_contains(&h, "if (!failure.empty()) throw LoadError(failure);"); - // Errors: the base, cancellation, and load failures. - assert_contains(&h, "class Error : public std::runtime_error {"); - assert_contains(&h, "class Cancelled : public Error {"); - assert_contains(&h, "class LoadError : public Error {"); - assert_contains( - &h, - "if (code == -5) return std::make_exception_ptr(Cancelled(message));", - ); - // Negative codes, including -4 and -5, route to the runtime exceptions. - assert_contains(&h, "if (code < 0) return make_error(code, msg);"); - // Returned strings are copied by length and released with free_bytes. - assert_contains( - &h, - "~Release() { api_free_bytes(const_cast(ptr), len); }", - ); - assert_contains( - &h, - "return std::string(reinterpret_cast(ptr), len);", - ); - assert_contains(&h, "class CancelToken {"); - assert_contains(&h, "CancelToken() : handle_(api_cancel_token_create()) {"); - for legacy in [ - "free_string", - "c_str()", - "WeaveFFI", - "weaveffi", - "check_abi_version", - "arena", - "handle_t", - "std::function", - ] { - let body = h.split_once("#pragma once").expect("prelude").1; - assert!(!body.contains(legacy), "legacy token {legacy:?} in header"); - } -} - -#[test] -fn identity_names_every_file_and_config_overrides_win() { - let api = validate_api(fixture(), None) - .expect("fixture validates") - .with_identity(Identity::named("kv-store")); - let model = BindingModel::build(&api); - let out = Utf8Path::new("/out"); - let files = CppGenerator.files(&api, &model, out, &CppConfig::default()); - let paths: Vec = files.iter().map(|f| f.path.to_string()).collect(); - assert_eq!( - paths, - [ - "/out/cpp/kv_store.h", - "/out/cpp/kv_store.hpp", - "/out/cpp/CMakeLists.txt", - "/out/cpp/README.md", - ] - ); - let hpp = &files[1].contents; - assert_contains(hpp, "#include \"kv_store.h\""); - assert_contains(hpp, "namespace kv_store {"); - assert_contains(hpp, "kv_store: C ABI revision mismatch"); - let cmake = &files[2].contents; - assert_contains(cmake, "add_library(kv_store_cpp INTERFACE)"); - assert_contains(cmake, "add_library(kv_store::cpp ALIAS kv_store_cpp)"); - assert_contains( - cmake, - "set(KV_STORE_LIBRARY \"$ENV{KV_STORE_LIBRARY}\" CACHE FILEPATH", - ); - assert_contains( - cmake, - "target_link_libraries(kv_store_cpp INTERFACE kv_store)", - ); - - let config = CppConfig { - namespace: Some("acme::kv".into()), - header_name: Some("acme_kv.hpp".into()), - ..CppConfig::default() - }; - let files = CppGenerator.files(&api, &model, out, &config); - assert_eq!(files[1].path, "/out/cpp/acme_kv.hpp"); - assert_contains(&files[1].contents, "namespace acme::kv {"); - assert_contains(&files[1].contents, "#include \"kv_store.h\""); -} - -#[test] -fn constructors_and_adoption_are_explicit() { - let (_, h) = render(); - // No implicit conversion from arguments or raw pointers into an object. - assert!(!h.contains("\n Store(std::string_view path);")); - assert!(!h.contains("Store(api_kv_Store* h)")); - assert_contains(&h, "inline constexpr adopt_t adopt{};"); -} - -#[test] -fn extended_shapes_render() { - let api = Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![Module { - name: "ext".into(), - doc: None, - functions: vec![ - func( - "maybe_stores", - vec![], - Some(TypeRef::Iterator(Box::new(optional(named("Store"))))), - ), - func( - "roundtrip", - vec![ - param("blob", TypeRef::Bytes), - param( - "m", - TypeRef::Map(Box::new(TypeRef::StringUtf8), Box::new(named("Store"))), - ), - param("shape", named("Shape")), - ], - Some(named("Shape")), - ), - Function { - r#async: true, - cancellable: true, - throws: true, - ..func("slow", vec![param("n", TypeRef::I64)], Some(TypeRef::Bytes)) - }, - Function { - r#async: true, - ..func("fire", vec![], None) - }, - Function { - throws: true, - ..func("fail", vec![], Some(TypeRef::F64)) - }, - func( - "watch", - vec![param("w", named("Watcher"))], - Some(TypeRef::Bool), - ), - ], - interfaces: vec![InterfaceDef { - name: "Store".into(), - doc: None, - deprecated: None, - constructors: vec![], - methods: vec![func("size", vec![], Some(TypeRef::U64))], - statics: vec![], - }], - callback_interfaces: vec![CallbackInterfaceDef { - name: "Watcher".into(), - doc: None, - deprecated: None, - methods: vec![ - func( - "on_data", - vec![ - param("blob", TypeRef::Bytes), - param("names", TypeRef::List(Box::new(TypeRef::StringUtf8))), - param("label", optional(TypeRef::StringUtf8)), - param("shape", named("Shape")), - param("ratio", TypeRef::F64), - ], - Some(TypeRef::I64), - ), - func( - "on_flag", - vec![param("flag", TypeRef::Bool)], - Some(TypeRef::Bool), - ), - ], - }], - structs: vec![], - enums: vec![EnumDef { - name: "Shape".into(), - doc: None, - deprecated: None, - variants: vec![ - EnumVariant { - name: "Dot".into(), - value: 0, - doc: None, - fields: vec![], - }, - EnumVariant { - name: "Boxed".into(), - value: 1, - doc: None, - fields: vec![ - field("store", named("Store")), - field("tags", TypeRef::List(Box::new(TypeRef::StringUtf8))), - ], - }, - ], - }], - errors: Some(ErrorDomain { - name: "ExtError".into(), - codes: vec![ErrorCode { - name: "TOO_BIG".into(), - code: 1, - message: "too big".into(), - doc: None, - fields: vec![ - field("limit", TypeRef::I64), - field("culprit", optional(named("Store"))), - ], - }], - }), - modules: vec![], - }], - }; - let (_, h) = render_api(&validate_api(api, None).expect("fixture validates")); - - // Nullable iterator elements, object payloads in rich enums and error - // fields, maps of objects, and a cancellable async bytes result. - assert_contains(&h, "std::optional> next() {"); - assert_contains(&h, "if (item) value.emplace(adopt, item);"); - assert_contains( - &h, - "return Shape{Shape::Boxed{std::move(f_store), std::move(f_tags)}};", - ); - assert_contains( - &h, - "w.write_u64(static_cast(reinterpret_cast(p.store.clone_handle())));", - ); - assert_contains( - &h, - "TooBigError(const std::string& msg, int64_t limit, std::optional culprit)", - ); - assert_contains( - &h, - "m_buf.write_u64(static_cast(reinterpret_cast(kv0.second.clone_handle())));", - ); - assert_contains( - &h, - "inline std::future> slow(int64_t n, const CancelToken& cancel_token = CancelToken::none()) {", - ); - assert_contains( - &h, - "api_ext_slow(n, cancel_token.handle(), [](void* context, api_error* err, const uint8_t* result_ptr, size_t result_len) {", - ); - assert_contains( - &h, - "std::unique_ptr>> p(static_cast>*>(context));", - ); - assert_contains( - &h, - "p->set_value(detail::take_bytes(result_ptr, result_len));", - ); - assert_contains(&h, "}, static_cast(promise.release()));"); - assert_contains( - &h, - "p->set_exception(detail::make_ext_error(err->code, msg, err->payload_ptr, err->payload_len));", - ); - // Callback parameters in the buffered family decode into owned values. - assert_contains( - &h, - "virtual int64_t on_data(const std::vector& blob, const std::vector& names, const std::optional& label, const Shape& shape, double ratio) = 0;", - ); - assert_contains( - &h, - "std::vector blob_val = detail::borrow_bytes(blob_ptr, blob_len);", - ); - assert_contains(&h, "Shape shape_val = detail::read_Shape(shape_r);"); - assert_contains(&h, "return int64_t{};"); -} - -#[test] -fn rendering_is_deterministic() { - let (_, a) = render(); - let (_, b) = render(); - assert_eq!(a, b); -} diff --git a/crates/weaveffi-gen/src/targets/cpp/types.rs b/crates/weaveffi-gen/src/targets/cpp/types.rs deleted file mode 100644 index 27d89c34..00000000 --- a/crates/weaveffi-gen/src/targets/cpp/types.rs +++ /dev/null @@ -1,170 +0,0 @@ -//! C++ spellings: identifier escaping, name casing, namespace paths, and the -//! IR-to-C++ type mapping. - -use crate::cabi::c_param_name; -use crate::lang::{self, CPP_KEYWORDS}; -use crate::utils::local_type_name; -use heck::ToSnakeCase; -use weaveffi_model::abi::AbiParam; -use weaveffi_model::errors; -use weaveffi_model::model::ModuleBinding; -use weaveffi_model::model::Ty; - -/// Idiomatic C++ exception class name for an error code: PascalCase with a -/// single `Error` suffix (`KEY_NOT_FOUND` becomes `KeyNotFoundError`), instead -/// of the raw SCREAMING_SNAKE `KEY_NOT_FOUNDError` spelling. -pub(crate) fn cpp_error_class(name: &str) -> String { - errors::type_name(name, "Error") -} - -/// C++ reserved words the shared [`CPP_KEYWORDS`] table doesn't carry: the -/// alternative operator tokens spelled with `_eq`, the extended character -/// types, the two remaining cast keywords, and `thread_local`. Sorted for -/// binary search; kept disjoint from the shared table so each name is listed -/// exactly once. -pub(crate) const CPP_EXTRA_KEYWORDS: &[&str] = &[ - "and_eq", - "char16_t", - "char32_t", - "char8_t", - "const_cast", - "not_eq", - "or_eq", - "reinterpret_cast", - "thread_local", - "wchar_t", - "xor_eq", -]; - -/// Escape an identifier that collides with a C++ keyword by appending an -/// underscore (`delete` becomes `delete_`); other names pass through. -/// Combines the shared [`CPP_KEYWORDS`] table with [`CPP_EXTRA_KEYWORDS`]. -pub(crate) fn cpp_ident(name: &str) -> String { - if lang::is_reserved(name, CPP_EXTRA_KEYWORDS) { - return format!("{name}_"); - } - lang::escape_ident(name, CPP_KEYWORDS) -} - -/// The C++ spelling of a callable name: snake_case (via `heck`) with C++ -/// keyword collisions escaped. -pub(crate) fn cpp_fn_name(name: &str) -> String { - cpp_ident(&name.to_snake_case()) -} - -/// The nested C++ namespace path for a module: each IDL segment converted to -/// snake case and keyword-escaped, joined with `::` (`kv.stats` becomes -/// `kv::stats`). -pub(crate) fn cpp_namespace_path(module: &ModuleBinding) -> String { - module - .segments - .iter() - .map(|s| cpp_ident(&s.to_snake_case())) - .collect::>() - .join("::") -} - -/// The C++ spelling of one ABI slot name: the IDL-chosen identifier with the -/// same keyword escape the shared C declarations apply, so the trampoline and -/// lambda parameter lists compile against the `extern "C"` prototypes. -pub(crate) fn slot_name(p: &AbiParam) -> String { - c_param_name(&p.name) -} - -/// Renders ABI parameter slots to C declarations (` `), the form -/// used inside async completion lambdas and callback-interface trampolines. -pub(crate) fn render_param_decls(params: &[AbiParam], prefix: &str) -> Vec { - params - .iter() - .map(|p| format!("{} {}", p.ty.render_c(prefix), slot_name(p))) - .collect() -} - -/// The `detail` accessor returning the process-wide static vtable for a -/// callback interface, named by its local C++ class name. -pub(crate) fn vtable_accessor(local_name: &str) -> String { - format!("{local_name}_vtable") -} - -/// The `detail` struct holding a callback interface's trampolines, named by -/// its local C++ class name. -pub(crate) fn trampoline_struct(local_name: &str) -> String { - format!("{local_name}_trampolines") -} - -/// The idiomatic C++ spelling of an IR type. Interfaces map to their RAII -/// wrapper class, `Interface?` to `std::optional` of it, and callback -/// interfaces to a `std::shared_ptr` of the abstract class the consumer -/// implements. -pub(crate) fn cpp_type(ty: &Ty) -> String { - match ty { - Ty::I8 => "int8_t".into(), - Ty::I16 => "int16_t".into(), - Ty::I32 => "int32_t".into(), - Ty::U8 => "uint8_t".into(), - Ty::U16 => "uint16_t".into(), - Ty::U32 => "uint32_t".into(), - Ty::I64 => "int64_t".into(), - Ty::U64 => "uint64_t".into(), - Ty::F32 => "float".into(), - Ty::F64 => "double".into(), - Ty::Bool => "bool".into(), - Ty::StringUtf8 => "std::string".into(), - Ty::Bytes => "std::vector".into(), - // Records and rich (algebraic) enums are plain value types; both are - // named by their bare local C++ type. - Ty::Record(n) | Ty::RichEnum(n) => local_type_name(n).to_string(), - // A cross-module type (e.g. `graphics.Unit`) is emitted as the bare - // local C++ type `Unit`; never the dot-qualified IR name (invalid C++). - Ty::Enum(n) => local_type_name(n).to_string(), - Ty::Interface(n) => local_type_name(n).to_string(), - Ty::CallbackInterface(n) => format!("std::shared_ptr<{}>", local_type_name(n)), - Ty::Optional(inner) => format!("std::optional<{}>", cpp_type(inner)), - Ty::List(inner) => format!("std::vector<{}>", cpp_type(inner)), - Ty::Map(k, v) => { - format!("std::unordered_map<{}, {}>", cpp_type(k), cpp_type(v)) - } - // An `iter` return renders as a per-function lazy range class, not - // through this generic mapping. - Ty::Iterator(_) => unreachable!("iterator returns render as range classes"), - } -} - -/// One C++ parameter declaration (` `) for a wrapper signature. -/// Strings borrow as `std::string_view` (the ABI passes a pointer and a -/// length, so interior NULs survive); heavier types borrow by const -/// reference; scalars and enums pass by value. A callback interface is taken -/// as a `std::shared_ptr` by value, which the wrapper moves into the heap box -/// it hands the producer as `ctx`. -pub(crate) fn cpp_param_decl(ty: &Ty, name: &str) -> String { - match ty { - Ty::StringUtf8 => format!("std::string_view {name}"), - Ty::Bytes => format!("const std::vector& {name}"), - // Records and rich enums borrow: the wrapper encodes them into a local - // buffer, so the value stays with the caller. Interfaces borrow their - // pointer for the call. - Ty::Record(n) | Ty::RichEnum(n) | Ty::Interface(n) => { - format!("const {}& {name}", local_type_name(n)) - } - Ty::Optional(_) | Ty::List(_) | Ty::Map(_, _) => { - format!("const {}& {name}", cpp_type(ty)) - } - _ => format!("{} {name}", cpp_type(ty)), - } -} - -/// One C++ parameter declaration for a callback-interface method the consumer -/// implements. Borrowed strings arrive as a `std::string_view` valid for the -/// call; bytes and buffered values arrive by const reference (the trampoline -/// owns the decoded copy for the call); object -/// arguments transfer one strong reference, so they arrive by value as the -/// RAII wrapper (or `std::optional` of it) the implementation now owns. -pub(crate) fn cpp_cb_param_decl(ty: &Ty, name: &str) -> String { - match ty { - Ty::Interface(n) => format!("{} {name}", local_type_name(n)), - Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => { - format!("{} {name}", cpp_type(ty)) - } - _ => cpp_param_decl(ty, name), - } -} diff --git a/crates/weaveffi-gen/src/targets/dart/codec.rs b/crates/weaveffi-gen/src/targets/dart/codec.rs deleted file mode 100644 index 68ca6490..00000000 --- a/crates/weaveffi-gen/src/targets/dart/codec.rs +++ /dev/null @@ -1,113 +0,0 @@ -//! Value-buffer codec emitters: the inline read expressions and write -//! statements for any wire shape, naming the per-record and per-rich-enum -//! `_pack{Name}`/`_unpack{Name}` helpers. -//! -//! Every dispatch goes through [`Ty::wire`], and primitives map onto the -//! runtime's `read{Prim}`/`write{Prim}` methods by -//! [`Prim::pascal`](weaveffi_model::model::Prim::pascal), so this -//! module never re-derives the wire format. -//! -//! An object token carries one strong reference. Writing one calls the -//! wrapper's `_cloneRef()` so the encoding owns a fresh reference and the -//! wrapper keeps its own; reading one adopts the pointer into a new wrapper. - -use crate::codegen::CodeWriter; -use weaveffi_model::model::{Ty, WireType}; - -use crate::targets::dart::types::{dart_class, dart_type}; - -/// The `_pack{Name}` helper of a record or rich enum. -pub(crate) fn pack_fn(name: &str) -> String { - format!("_pack{}", dart_class(name)) -} - -/// The `_unpack{Name}` helper of a record or rich enum. -pub(crate) fn unpack_fn(name: &str) -> String { - format!("_unpack{}", dart_class(name)) -} - -/// Mint a fresh `_t{n}` temporary name. -pub(crate) fn fresh(tmp: &mut usize) -> String { - let n = *tmp; - *tmp += 1; - format!("_t{n}") -} - -/// The Dart expression decoding one value of `ty` from the reader `r`. Read -/// expressions evaluate strictly left to right, so composing them preserves -/// the wire order. Collections grow as they decode, so a corrupt count can't -/// force a huge allocation up front. -pub(crate) fn read_expr(r: &str, ty: &Ty) -> String { - match ty.wire() { - WireType::Prim(p) => format!("{r}.read{}()", p.pascal()), - // A token is one adopted strong reference. - WireType::Object(n) => format!( - "{}._(Pointer.fromAddress({r}.readU64()))", - dart_class(n) - ), - WireType::Enum(n) => format!("{}.fromValue({r}.readI32())", dart_class(n)), - WireType::User(n) => format!("{}({r})", unpack_fn(n)), - WireType::Optional(inner) => { - format!("({r}.readFlag() ? {} : null)", read_expr(r, inner)) - } - WireType::List(inner) => format!( - "<{}>[for (var i = {r}.readLength(); i > 0; i--) {}]", - dart_type(inner), - read_expr(r, inner) - ), - WireType::Map(k, v) => format!( - "<{}, {}>{{for (var i = {r}.readLength(); i > 0; i--) {}: {}}}", - dart_type(k), - dart_type(v), - read_expr(r, k), - read_expr(r, v) - ), - } -} - -/// Emit the statements encoding `expr` (a value of `ty`) into the writer -/// `wr`. Optionals, lists, and maps recurse through fresh `_t{n}` -/// temporaries. -pub(crate) fn write_stmts(w: &mut CodeWriter, wr: &str, expr: &str, ty: &Ty, tmp: &mut usize) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("{wr}.write{}({expr});", p.pascal())); - } - WireType::Object(_) => { - w.line(format!("{wr}.writeU64({expr}._cloneRef().address);")); - } - WireType::Enum(_) => { - w.line(format!("{wr}.writeI32({expr}.value);")); - } - WireType::User(n) => { - w.line(format!("{}({wr}, {expr});", pack_fn(n))); - } - WireType::Optional(inner) => { - let t = fresh(tmp); - w.line(format!("final {t} = {expr};")); - w.line(format!("{wr}.writeFlag({t} != null);")); - w.block(format!("if ({t} != null) {{"), "}", |w| { - write_stmts(w, wr, &t, inner, tmp); - }); - } - WireType::List(inner) => { - let t = fresh(tmp); - let e = fresh(tmp); - w.line(format!("final {t} = {expr};")); - w.line(format!("{wr}.writeLength({t}.length);")); - w.block(format!("for (final {e} in {t}) {{"), "}", |w| { - write_stmts(w, wr, &e, inner, tmp); - }); - } - WireType::Map(k, v) => { - let t = fresh(tmp); - let e = fresh(tmp); - w.line(format!("final {t} = {expr};")); - w.line(format!("{wr}.writeLength({t}.length);")); - w.block(format!("for (final {e} in {t}.entries) {{"), "}", |w| { - write_stmts(w, wr, &format!("{e}.key"), k, tmp); - write_stmts(w, wr, &format!("{e}.value"), v, tmp); - }); - } - } -} diff --git a/crates/weaveffi-gen/src/targets/dart/entities.rs b/crates/weaveffi-gen/src/targets/dart/entities.rs deleted file mode 100644 index 79b8abbe..00000000 --- a/crates/weaveffi-gen/src/targets/dart/entities.rs +++ /dev/null @@ -1,451 +0,0 @@ -//! Entity rendering: error domains, C-style enums, rich enums, records, and -//! interface wrapper classes, plus their value-buffer codec helpers. - -use crate::codegen::CodeWriter; -use heck::ToUpperCamelCase; -use weaveffi_model::model::{ - EnumBinding, ErrorBinding, FieldBinding, InterfaceBinding, ModuleBinding, StructBinding, Ty, -}; - -use crate::targets::dart::calls::{ - err_ctx, finalizer, lookup, mapper_fn, render_callable, DartDecl, -}; -use crate::targets::dart::codec::{fresh, pack_fn, read_expr, unpack_fn, write_stmts}; -use crate::targets::dart::docs::{write_deprecated, write_doc}; -use crate::targets::dart::types::{dart_class, dart_ident, dart_str_literal, dart_type, ffi_var}; - -/// The Dart exception class of an error domain or code: the PascalCase name -/// with a trailing `Error` swapped for `Exception` (`KvError` becomes -/// `KvException`, a code `IoError` becomes `IoException`, and a bare -/// `Error` becomes `ErrorException`). -pub(crate) fn dart_exception_name(raw: &str) -> String { - let pascal = raw.to_upper_camel_case(); - let stem = pascal - .strip_suffix("Error") - .filter(|s| !s.is_empty()) - .unwrap_or(&pascal); - if stem.ends_with("Exception") { - dart_class(stem) - } else { - dart_class(&format!("{stem}Exception")) - } -} - -/// The indefinite article for a class name (`an Entry`, `a Contact`). -fn article(class: &str) -> &'static str { - if class.starts_with(['A', 'E', 'I', 'O', 'U']) { - "an" - } else { - "a" - } -} - -/// Render the final fields of a value class, with their docs. -fn render_fields(w: &mut CodeWriter, fields: &[FieldBinding]) { - for f in fields { - write_doc(w, &f.doc); - w.line(format!( - "final {} {};", - dart_type(&f.ty), - dart_ident(&f.name) - )); - } -} - -/// Render one module's declared error domain: the domain exception (a -/// `NativeException` subclass), one subclass per code carrying its code, -/// default message, and decoded payload fields, and the `_map{Domain}` -/// mapper that throwing wrappers route codes through. Only declared codes -/// gain a `case`; every other code, including the negative runtime range, -/// maps onto the runtime exceptions. -pub(crate) fn render_error(out: &mut String, module: &ModuleBinding, eb: &ErrorBinding) { - let exc = dart_exception_name(&eb.type_name); - let mut w = CodeWriter::two_space(); - w.blank(); - w.line(format!( - "/// The `{}` error domain of module `{}`.", - eb.name, module.dot_path - )); - w.block( - format!("class {exc} extends NativeException {{"), - "}", - |w| { - w.line("/// Creates a domain error carrying [code] and [message]."); - w.line(format!("{exc}(super.code, super.message);")); - }, - ); - - for c in &eb.codes { - let class = dart_exception_name(&c.name); - let message = dart_str_literal(&c.message); - w.blank(); - write_doc(&mut w, &c.doc.clone().or_else(|| Some(c.message.clone()))); - w.block(format!("class {class} extends {exc} {{"), "}", |w| { - if c.fields.is_empty() { - w.line(format!( - "{class}([String message = '{message}']) : super({}, message);", - c.value - )); - } else { - let params: Vec = c - .fields - .iter() - .map(|f| format!("this.{}", dart_ident(&f.name))) - .collect(); - w.line(format!( - "{class}({}, [String message = '{message}'])", - params.join(", ") - )); - w.line(format!(" : super({}, message);", c.value)); - w.blank(); - render_fields(w, &c.fields); - } - }); - } - - w.blank(); - w.block( - format!( - "NativeException {}(int code, String message, Uint8List payload) {{", - mapper_fn(&exc) - ), - "}", - |w| { - w.block("switch (code) {", "}", |w| { - for c in &eb.codes { - let class = dart_exception_name(&c.name); - w.line(format!("case {}:", c.value)); - w.scope(|w| { - if c.fields.is_empty() { - w.line(format!("return {class}(message);")); - } else { - w.line("final r = _BufferReader(payload);"); - let args: Vec = - c.fields.iter().map(|f| read_expr("r", &f.ty)).collect(); - w.line(format!( - "final error = {class}({}, message);", - args.join(", ") - )); - w.line("r.expectEnd();"); - w.line("return error;"); - } - }); - } - w.line("default:"); - w.scope(|w| { - w.line("return _runtimeError(code, message, payload);"); - }); - }); - }, - ); - out.push_str(&w.finish()); -} - -/// Render one interface as a wrapper class over the runtime's -/// `_NativeObject`: the canonical `new` constructor is an unnamed factory, -/// other constructors are named factories, methods borrow the wrapper's -/// pointer, and statics are `static`. Member bindings stay at file scope. -pub(crate) fn render_interface( - out: &mut String, - module: &ModuleBinding, - i: &InterfaceBinding, - leaf: bool, -) { - let class = dart_class(&i.name); - let destroy = ffi_var(&i.destroy_symbol); - let clone = ffi_var(&i.clone_symbol); - out.push_str(&lookup( - &i.clone_symbol, - "Pointer Function(Pointer)", - "Pointer Function(Pointer)", - leaf, - )); - out.push_str(&lookup( - &i.destroy_symbol, - "Void Function(Pointer)", - "void Function(Pointer)", - leaf, - )); - out.push_str(&finalizer(&i.destroy_symbol)); - - let exc = module - .error - .as_ref() - .map(|e| dart_exception_name(&e.type_name)); - let mut members = String::new(); - for c in &i.constructors { - let kind = DartDecl::Factory { - class_name: &class, - named: c.name != "new", - }; - let name = dart_ident(&c.name); - render_callable( - out, - &mut members, - c, - &kind, - &name, - err_ctx(c, exc.as_deref()), - leaf, - ); - } - for m in &i.methods { - let name = dart_ident(&m.name); - let err = err_ctx(m, exc.as_deref()); - render_callable(out, &mut members, m, &DartDecl::Method, &name, err, leaf); - } - for s in &i.statics { - let name = dart_ident(&s.name); - let err = err_ctx(s, exc.as_deref()); - render_callable(out, &mut members, s, &DartDecl::Static, &name, err, leaf); - } - - let mut w = CodeWriter::two_space(); - w.blank(); - write_doc(&mut w, &i.doc); - if i.doc.is_some() { - w.line("///"); - } - w.line("/// A reference-counted native object. Each instance holds one strong"); - w.line("/// reference, released by [dispose] or, when an undisposed instance is"); - w.line("/// collected, by a finalizer."); - write_deprecated(&mut w, &i.deprecated); - w.block( - format!("final class {class} extends _NativeObject {{"), - "}", - |w| { - w.line(format!("{class}._(super._ptr);")); - w.blank(); - w.line("@override"); - w.line(format!( - "NativeFinalizer get _finalizer => {destroy}Finalizer;" - )); - w.blank(); - w.line("@override"); - w.line(format!( - "void _destroy(Pointer ptr) => {destroy}(ptr);" - )); - w.blank(); - w.line("@override"); - w.line(format!( - "Pointer _clone(Pointer ptr) => {clone}(ptr);" - )); - w.block_raw(&members); - }, - ); - out.push_str(&w.finish()); -} - -/// Enum members `enum` declarations already define; a variant spelled like -/// one gains a trailing `_`. -const ENUM_MEMBERS: &[&str] = &["fromValue", "index", "name", "value", "values"]; - -/// Render one enum: a C-style enum becomes an enhanced Dart `enum` carrying -/// its discriminant; a rich enum becomes a sealed class hierarchy. -pub(crate) fn render_enum(out: &mut String, e: &EnumBinding) { - if e.is_rich() { - render_rich_enum(out, e); - return; - } - let name = dart_class(&e.name); - let mut w = CodeWriter::two_space(); - w.blank(); - write_doc(&mut w, &e.doc); - write_deprecated(&mut w, &e.deprecated); - w.block(format!("enum {name} {{"), "}", |w| { - for (i, v) in e.variants.iter().enumerate() { - let mut variant = dart_ident(&v.name); - if ENUM_MEMBERS.contains(&variant.as_str()) { - variant.push('_'); - } - write_doc(w, &v.doc); - let end = if i + 1 == e.variants.len() { ';' } else { ',' }; - w.line(format!("{variant}({}){end}", v.value)); - } - w.blank(); - w.line(format!("const {name}(this.value);")); - w.blank(); - w.line("/// The C discriminant."); - w.line("final int value;"); - w.blank(); - w.line("/// The variant whose discriminant is [value]."); - w.line(format!("static {name} fromValue(int value) => values.firstWhere(")); - w.line(" (e) => e.value == value,"); - w.line(format!( - " orElse: () => throw ArgumentError.value(value, 'value', 'not a {name} discriminant'));" - )); - }); - out.push_str(&w.finish()); -} - -/// Render one record as a plain Dart value class (final fields, one named -/// constructor argument per field, optional fields not required) plus its -/// `_pack{Name}`/`_unpack{Name}` helpers. -pub(crate) fn render_struct(out: &mut String, s: &StructBinding) { - let class = dart_class(&s.name); - let mut w = CodeWriter::two_space(); - w.blank(); - write_doc(&mut w, &s.doc); - write_deprecated(&mut w, &s.deprecated); - w.block(format!("class {class} {{"), "}", |w| { - if s.fields.is_empty() { - w.line(format!("/// Creates {} [{class}].", article(&class))); - w.line(format!("const {class}();")); - return; - } - let params: Vec = s - .fields - .iter() - .map(|f| { - let n = dart_ident(&f.name); - if matches!(f.ty, Ty::Optional(_)) { - format!("this.{n}") - } else { - format!("required this.{n}") - } - }) - .collect(); - w.line(format!("/// Creates {} [{class}].", article(&class))); - w.line(format!("{class}({{{}}});", params.join(", "))); - w.blank(); - render_fields(w, &s.fields); - }); - - w.blank(); - w.block( - format!("void {}(_BufferWriter w, {class} v) {{", pack_fn(&s.name)), - "}", - |w| { - let mut tmp = 0usize; - for f in &s.fields { - let expr = format!("v.{}", dart_ident(&f.name)); - write_stmts(w, "w", &expr, &f.ty, &mut tmp); - } - }, - ); - - // Named arguments evaluate in source order, which is the wire order. - w.blank(); - if s.fields.is_empty() { - w.line(format!( - "{class} {}(_BufferReader r) => const {class}();", - unpack_fn(&s.name) - )); - } else { - w.line(format!( - "{class} {}(_BufferReader r) => {class}(", - unpack_fn(&s.name) - )); - w.scope(|w| { - for f in &s.fields { - w.line(format!( - "{}: {},", - dart_ident(&f.name), - read_expr("r", &f.ty) - )); - } - }); - w.line(");"); - } - out.push_str(&w.finish()); -} - -/// Render one rich enum as a sealed class hierarchy: a sealed base plus one -/// subclass per variant carrying its fields (positional constructor), and -/// `_pack{Name}`/`_unpack{Name}` helpers encoding the `i32` tag followed by -/// the active variant's fields. -fn render_rich_enum(out: &mut String, e: &EnumBinding) { - let base = dart_class(&e.name); - let variant_class = |name: &str| dart_class(&format!("{}{}", e.name, name)); - let mut w = CodeWriter::two_space(); - w.blank(); - write_doc(&mut w, &e.doc); - write_deprecated(&mut w, &e.deprecated); - w.block(format!("sealed class {base} {{"), "}", |w| { - w.line(format!("const {base}();")); - }); - - for v in &e.variants { - let cls = variant_class(&v.name); - w.blank(); - write_doc(&mut w, &v.doc); - if v.fields.is_empty() { - w.block(format!("final class {cls} extends {base} {{"), "}", |w| { - w.line(format!("/// Creates {} [{cls}].", article(&cls))); - w.line(format!("const {cls}();")); - }); - continue; - } - w.block(format!("final class {cls} extends {base} {{"), "}", |w| { - let params: Vec = v - .fields - .iter() - .map(|f| format!("this.{}", dart_ident(&f.name))) - .collect(); - w.line(format!("/// Creates {} [{cls}].", article(&cls))); - w.line(format!("{cls}({});", params.join(", "))); - w.blank(); - render_fields(w, &v.fields); - }); - } - - // Pack: the tag, then the active variant's fields. The sealed base makes - // the switch exhaustive. One temp counter spans every case so names stay - // unique across the switch. - w.blank(); - w.block( - format!("void {}(_BufferWriter w, {base} v) {{", pack_fn(&e.name)), - "}", - |w| { - w.block("switch (v) {", "}", |w| { - let mut tmp = 0usize; - for v in &e.variants { - let cls = variant_class(&v.name); - if v.fields.is_empty() { - w.line(format!("case {cls}():")); - w.scope(|w| { - w.line(format!("w.writeI32({});", v.value)); - }); - continue; - } - let b = fresh(&mut tmp); - w.line(format!("case final {cls} {b}:")); - w.scope(|w| { - w.line(format!("w.writeI32({});", v.value)); - for f in &v.fields { - let expr = format!("{b}.{}", dart_ident(&f.name)); - write_stmts(w, "w", &expr, &f.ty, &mut tmp); - } - }); - } - }); - }, - ); - - // Unpack: positional arguments evaluate left to right, in wire order. - w.blank(); - w.block( - format!("{base} {}(_BufferReader r) {{", unpack_fn(&e.name)), - "}", - |w| { - w.line("final tag = r.readI32();"); - w.block("switch (tag) {", "}", |w| { - for v in &e.variants { - let cls = variant_class(&v.name); - w.line(format!("case {}:", v.value)); - w.scope(|w| { - let args: Vec = - v.fields.iter().map(|f| read_expr("r", &f.ty)).collect(); - let ctor = if v.fields.is_empty() { "const " } else { "" }; - w.line(format!("return {ctor}{cls}({});", args.join(", "))); - }); - } - w.line("default:"); - w.scope(|w| { - w.line(format!("_bufferError('unknown {base} tag $tag');")); - }); - }); - }, - ); - out.push_str(&w.finish()); -} diff --git a/crates/weaveffi-gen/src/targets/dart/mod.rs b/crates/weaveffi-gen/src/targets/dart/mod.rs deleted file mode 100644 index 401d9a93..00000000 --- a/crates/weaveffi-gen/src/targets/dart/mod.rs +++ /dev/null @@ -1,231 +0,0 @@ -//! Dart (`dart:ffi`) binding generator. -//! -//! Emits a standalone Dart package (`pubspec.yaml`, `README.md`, and -//! `lib/{package}.dart`) whose bindings call the C ABI through `dart:ffi`. -//! The package is named after the identity's C prefix and loads the -//! identity's library, honoring the `{PREFIX}_LIBRARY` override. On first use -//! the library's ABI revision and every top-level module's contract checksum -//! are verified. -//! -//! Records and rich enums are value types: plain Dart classes (a sealed -//! hierarchy for a rich enum) that cross the ABI serialized in the value -//! buffer format. Interfaces are wrapper classes owning one strong reference, -//! released by `dispose()` or a `NativeFinalizer`, and guarded so a -//! `dispose()` during an in-flight call defers the release until the call -//! returns. Async functions return a `Future` completed by a -//! `NativeCallable.listener`; cancellable ones take a `CancelToken`. -//! Callback interfaces are abstract classes: value-returning methods are -//! isolate-local trampolines (callable only on the isolate's thread), and -//! void methods are isolate-group-bound forwarders that may run on any thread -//! and deliver the call on the isolate's event loop. - -mod callbacks; -mod calls; -mod codec; -mod docs; -mod entities; -mod package; -mod runtime; -mod types; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use camino::Utf8Path; -use serde::{Deserialize, Serialize}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; - -use crate::targets::dart::callbacks::render_callback_interface; -use crate::targets::dart::calls::render_function; -use crate::targets::dart::entities::{render_enum, render_error, render_interface, render_struct}; -use crate::targets::dart::package::{render_packaged_readme, render_pubspec, render_readme}; -use crate::targets::dart::runtime::{bundles_platform, render_runtime, LoaderCandidates}; - -/// Per-target configuration for [`DartGenerator`]. -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct DartConfig { - /// Dart package name: the pubspec `name` and the library file - /// `lib/{package_name}.dart`. Defaults to the identity's C prefix. - pub package_name: Option, - /// When `true` (the default), module-level functions drop their module - /// path, so a `contacts` module exports `createContact` rather than - /// `contactsCreateContact`. - pub strip_module_prefix: bool, - /// Basename of the IDL the CLI was invoked with, for file preludes. - /// Populated by the CLI. - #[serde(skip)] - pub input_basename: Option, -} - -impl Default for DartConfig { - fn default() -> Self { - Self { - package_name: None, - strip_module_prefix: true, - input_basename: None, - } - } -} - -impl DartConfig { - /// The Dart package name for `api`: the configured name, else the - /// identity's C prefix. - pub fn package_name(&self, api: &ResolvedApi) -> String { - self.package_name - .clone() - .unwrap_or_else(|| api.identity().prefix.clone()) - } - - /// The IDL basename for generated-file preludes. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } -} - -/// The Dart backend. -pub struct DartGenerator; - -impl DartGenerator { - /// Render `lib/{package}.dart`: imports, the runtime sections the API - /// uses, then every module's declarations. - fn render_library( - &self, - api: &ResolvedApi, - model: &BindingModel, - config: &DartConfig, - loader: &LoaderCandidates, - file_name: &str, - ) -> String { - let mut out = render_prelude(CommentStyle::DoubleSlash, config.input_basename()); - out.push_str( - "// ignore_for_file: camel_case_types, non_constant_identifier_names, unused_element\n\n", - ); - let callbacks = model.has_callback_interfaces(); - if model.has_async() || callbacks { - out.push_str("import 'dart:async';\n"); - } - out.push_str("import 'dart:convert';\n"); - out.push_str("import 'dart:ffi';\n"); - out.push_str("import 'dart:io' show Platform;\n"); - if callbacks { - out.push_str("import 'dart:isolate';\n"); - } - out.push_str("import 'dart:typed_data';\n\n"); - out.push_str("import 'package:ffi/ffi.dart';\n"); - - render_runtime(&mut out, api.identity(), model, loader); - // Synchronous calls are leaf calls only when no callback interface - // exists, so no call can re-enter Dart. That is a property of the - // whole API, so the walk is done here rather than through the - // per-entity hooks (in `emit_members` order). - let leaf = !callbacks; - for module in &model.modules { - if let Some(e) = module.error.as_ref().filter(|e| e.declared_here) { - render_error(&mut out, module, e); - } - for e in &module.enums { - render_enum(&mut out, e); - } - for s in &module.structs { - render_struct(&mut out, s); - } - for cb in &module.callback_interfaces { - render_callback_interface(&mut out, cb); - } - for i in &module.interfaces { - render_interface(&mut out, module, i, leaf); - } - for f in &module.functions { - render_function(&mut out, module, f, config.strip_module_prefix, leaf); - } - } - out.push('\n'); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, file_name)); - out - } -} - -impl LanguageBackend for DartGenerator { - type Config = DartConfig; - - fn name(&self) -> &'static str { - "dart" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let package = config.package_name(api); - let dir = out_dir.join("dart"); - let file = format!("{package}.dart"); - let loader = LoaderCandidates::system(api.identity()); - let input = config.input_basename(); - vec![ - OutputFile::new( - dir.join("lib").join(&file), - self.render_library(api, model, config, &loader, &file), - ), - OutputFile::new( - dir.join("pubspec.yaml"), - render_pubspec(api.identity(), &package, input), - ), - OutputFile::new( - dir.join("README.md"), - render_readme(api.identity(), &package, input), - ), - ] - } - - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let package = config.package_name(api); - let dir = out_dir.join("dart"); - let file = format!("{package}.dart"); - let loader = LoaderCandidates::bundled(&ctx.binaries.lib_name); - let input = config.input_basename(); - let mut files = vec![ - PackagedFile::text( - dir.join("lib").join(&file), - self.render_library(api, model, config, &loader, &file), - ), - PackagedFile::text( - dir.join("pubspec.yaml"), - render_pubspec(api.identity(), &package, input), - ), - PackagedFile::text( - dir.join("README.md"), - render_packaged_readme(api.identity(), ctx, input), - ), - ]; - for nb in ctx - .binaries - .binaries - .iter() - .filter(|nb| bundles_platform(nb.platform)) - { - let dest = dir - .join("native") - .join(nb.platform.id()) - .join(ctx.binaries.bundled_filename(nb.platform)); - files.push(PackagedFile::copy(dest, nb.source.clone())); - } - Some(files) - } -} diff --git a/crates/weaveffi-gen/src/targets/dart/runtime.rs b/crates/weaveffi-gen/src/targets/dart/runtime.rs deleted file mode 100644 index d4deb6f4..00000000 --- a/crates/weaveffi-gen/src/targets/dart/runtime.rs +++ /dev/null @@ -1,133 +0,0 @@ -//! The fixed Dart runtime every generated library ships, kept as real Dart -//! source under `runtime/` and spliced in with `{{PLACEHOLDER}}` -//! substitution: the loader and load-time contract check, the error and -//! memory helpers, the object base class, the value-buffer codec, async and -//! cancellation glue, the callback-interface machinery, and the iterator -//! anchor. Sections an API doesn't use are left out. - -use crate::platform::{Os, Platform}; -use weaveffi_model::model::{checksum_symbol, BindingModel, ABI_VERSION}; -use weaveffi_model::pkg::Identity; - -use crate::targets::dart::types::dart_str_literal; - -const LOADER: &str = include_str!("runtime/loader.dart"); -const CORE: &str = include_str!("runtime/core.dart"); -const OBJECT: &str = include_str!("runtime/object.dart"); -const CODEC: &str = include_str!("runtime/codec.dart"); -const ASYNC: &str = include_str!("runtime/async.dart"); -const CANCEL: &str = include_str!("runtime/cancel.dart"); -const CALLBACKS: &str = include_str!("runtime/callbacks.dart"); -const ITERATOR: &str = include_str!("runtime/iterator.dart"); - -/// The library file names the loader tries, per OS family, in order. -pub(crate) struct LoaderCandidates { - /// macOS candidates. - pub(crate) macos: Vec, - /// Windows candidates. - pub(crate) windows: Vec, - /// Linux, Android, and every other POSIX candidate. - pub(crate) linux: Vec, -} - -impl LoaderCandidates { - /// The platform file names of `library`, found on the system search - /// path. - pub(crate) fn system(identity: &Identity) -> Self { - let (macos, linux, windows) = identity.library_files(); - Self { - macos: vec![macos], - windows: vec![windows], - linux: vec![linux], - } - } - - /// The bundled `native//` copies of `lib` (relative to the - /// working directory) for every desktop platform of each OS family, then - /// the system file name. - pub(crate) fn bundled(lib: &str) -> Self { - let family = |os: Os| { - let mut names: Vec = Platform::DESKTOP - .iter() - .filter(|p| p.os() == os) - .map(|p| format!("native/{}/{}", p.id(), p.lib_filename(lib))) - .collect(); - if let Some(p) = Platform::DESKTOP.iter().find(|p| p.os() == os) { - names.push(p.lib_filename(lib)); - } - names - }; - Self { - macos: family(Os::MacOs), - windows: family(Os::Windows), - linux: family(Os::Linux), - } - } -} - -/// Whether the packaged library bundles a prebuilt binary for `platform`: -/// the loader resolves `native//` for the desktop matrix only -/// (Android libraries ship through the app's own packaging, and a `.wasm` -/// module can't be opened with `DynamicLibrary`). -pub(crate) fn bundles_platform(platform: Platform) -> bool { - platform.is_desktop() -} - -/// Substitute the placeholders every runtime file may use. -fn splice(source: &str, identity: &Identity) -> String { - source - .replace("{{PREFIX}}", &identity.prefix) - .replace("{{LIBRARY_ENV}}", &identity.library_env_var()) - .replace("{{LIBRARY}}", &dart_str_literal(&identity.library)) - .replace("{{NAME}}", &dart_str_literal(&identity.name)) -} - -/// Render the runtime sections `model` needs, with `loader` choosing where -/// the library is looked for. -pub(crate) fn render_runtime( - out: &mut String, - identity: &Identity, - model: &BindingModel, - loader: &LoaderCandidates, -) { - let quote = |names: &[String]| { - names - .iter() - .map(|n| format!("'{}'", dart_str_literal(n))) - .collect::>() - .join(", ") - }; - let contracts: String = model - .roots() - .map(|m| { - format!( - " ('{}', '{}', 0x{:016x}),\n", - m.name, - checksum_symbol(&model.prefix, &m.name), - m.checksum.expect("top-level modules carry a checksum") - ) - }) - .collect(); - let loader_src = LOADER - .replace("{{MACOS_CANDIDATES}}", "e(&loader.macos)) - .replace("{{WINDOWS_CANDIDATES}}", "e(&loader.windows)) - .replace("{{LINUX_CANDIDATES}}", "e(&loader.linux)) - .replace("{{ABI_VERSION}}", &ABI_VERSION.to_string()) - .replace("{{CONTRACTS}}", &contracts); - - let cancellable = model.callables().any(|(_, f)| f.cancellable); - let sections = [ - (true, loader_src.as_str()), - (true, CORE), - (model.has_interfaces(), OBJECT), - (model.has_buffers(), CODEC), - (model.has_async(), ASYNC), - (cancellable, CANCEL), - (model.has_callback_interfaces(), CALLBACKS), - (model.has_iterators(), ITERATOR), - ]; - for (_, source) in sections.iter().filter(|(used, _)| *used) { - out.push('\n'); - out.push_str(&splice(source, identity)); - } -} diff --git a/crates/weaveffi-gen/src/targets/dart/runtime/loader.dart b/crates/weaveffi-gen/src/targets/dart/runtime/loader.dart deleted file mode 100644 index 1099a61c..00000000 --- a/crates/weaveffi-gen/src/targets/dart/runtime/loader.dart +++ /dev/null @@ -1,74 +0,0 @@ -// ── Library loading ── -// The native library opens on first use and is verified before any other -// lookup: its C ABI revision and every module's contract checksum must match -// the values these bindings were generated against. - -/// The native library every binding in this file calls. -final DynamicLibrary _lib = _verifyLibrary(_openLibrary()); - -DynamicLibrary _openLibrary() { - // An explicit path wins, so a caller can point at any build artifact. - final override = Platform.environment['{{LIBRARY_ENV}}']; - if (override != null && override.isNotEmpty) { - return DynamicLibrary.open(override); - } - // iOS links native code into the app binary. - if (Platform.isIOS) return DynamicLibrary.process(); - final candidates = Platform.isMacOS - ? const [{{MACOS_CANDIDATES}}] - : Platform.isWindows - ? const [{{WINDOWS_CANDIDATES}}] - : const [{{LINUX_CANDIDATES}}]; - ArgumentError? failure; - for (final candidate in candidates) { - try { - return DynamicLibrary.open(candidate); - } on ArgumentError catch (e) { - failure ??= e; - } - } - // A macOS app may link the library into its executable instead. - if (Platform.isMacOS) return DynamicLibrary.process(); - throw failure!; -} - -/// The C ABI revision these bindings were generated against. -const int _abiVersion = {{ABI_VERSION}}; - -/// Each top-level module's name, checksum symbol, and expected checksum. -const List<(String, String, int)> _contracts = <(String, String, int)>[ -{{CONTRACTS}}]; - -DynamicLibrary _verifyLibrary(DynamicLibrary lib) { - final int Function() abiVersion; - try { - abiVersion = lib.lookupFunction( - '{{PREFIX}}_abi_version'); - } on ArgumentError { - throw StateError( - "the native library '{{LIBRARY}}' isn't loaded; set {{LIBRARY_ENV}} to " - 'its path'); - } - final found = abiVersion(); - if (found != _abiVersion) { - throw StateError('{{NAME}}: these bindings expect C ABI revision ' - '$_abiVersion, but the loaded library reports revision $found'); - } - for (final (module, symbol, expected) in _contracts) { - final int actual; - try { - actual = lib.lookupFunction(symbol)(); - } on ArgumentError { - throw StateError( - "{{NAME}}: the loaded library doesn't export module '$module'"); - } - if (actual != expected) { - throw StateError("{{NAME}}: module '$module' doesn't match the loaded " - 'library (contract checksum ${_hex(actual)}, expected ' - '${_hex(expected)}); regenerate the bindings'); - } - } - return lib; -} - -String _hex(int v) => '0x${v.toUnsigned(64).toRadixString(16).padLeft(16, '0')}'; diff --git a/crates/weaveffi-gen/src/targets/dotnet/callbacks.rs b/crates/weaveffi-gen/src/targets/dotnet/callbacks.rs deleted file mode 100644 index b6f072f2..00000000 --- a/crates/weaveffi-gen/src/targets/dotnet/callbacks.rs +++ /dev/null @@ -1,193 +0,0 @@ -//! Callback-interface renderers: the consumer-facing C# `interface` plus the -//! process-wide vtable and its `[UnmanagedCallersOnly]` trampolines, -//! following the shared `CallbackProtocol`. -//! -//! * One static vtable per callback interface, allocated once in native -//! memory and never freed, so its address is stable for the process. -//! * `ctx` is a `GCHandle` to the implementation, released by the vtable's -//! `free` entry, so the implementation lives exactly as long as the -//! producer holds it. -//! * Trampoline arguments are received like returns, except that strings, -//! bytes, and buffers are borrowed (copied, never freed); objects are -//! adopted into wrappers the implementation owns. -//! * A thrown exception never unwinds into native code: the trampoline -//! reports it through `out_err` with the foreign error code and returns a -//! default value. Trampolines run on whatever thread the producer calls -//! from. - -use crate::codegen::CodeWriter; -use heck::{ToLowerCamelCase, ToUpperCamelCase}; -use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ModuleBinding}; - -use crate::targets::dotnet::calls::{ - direct_to_slot, receive, write_obsolete, Own, UNMANAGED_CALLERS_ONLY, -}; -use crate::targets::dotnet::docs::{write_doc, write_fn_doc}; -use crate::targets::dotnet::types::{ - callback_interface_cs, cs_ctype, cs_type, fn_pointer_type, safe_cs_name, vtable_class_cs, Cx, -}; - -/// Render one callback interface: the `public interface I{Name}` the -/// consumer implements, then the internal class hosting its vtable. -pub(crate) fn render_callback_interface( - w: &mut CodeWriter, - module: &ModuleBinding, - cb: &CallbackInterfaceBinding, - cx: Cx<'_>, -) { - render_consumer_interface(w, cb, cx.base); - render_vtable_class(w, cx, module, cb); -} - -fn method_cs(m: &CallbackMethodBinding) -> String { - m.name.to_upper_camel_case() -} - -/// The consumer-facing interface: one method per callback method. -fn render_consumer_interface(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, base: &str) { - let iface = callback_interface_cs(&cb.name); - write_doc(w, &cb.doc); - w.line("/// The native library may call any method from any thread until"); - w.line("/// it releases its last reference to the implementation. Object arguments"); - w.line("/// belong to the implementation. An exception thrown by a method reaches"); - w.line(format!( - "/// the native caller as ." - )); - write_obsolete(w, &cb.deprecated); - w.line(format!("public interface {iface}")); - w.block("{", "}", |w| { - for m in &cb.methods { - let mut params = m.params.clone(); - for p in &mut params { - p.name = p.name.to_lower_camel_case(); - } - write_fn_doc(w, &m.doc, ¶ms); - write_obsolete(w, &m.deprecated); - let ret = m.ret.as_ref().map(cs_type).unwrap_or_else(|| "void".into()); - let sig: Vec = params - .iter() - .map(|p| format!("{} {}", cs_type(&p.ty), safe_cs_name(&p.name))) - .collect(); - w.line(format!("{ret} {}({});", method_cs(m), sig.join(", "))); - } - }); - w.blank(); -} - -/// The internal class owning the vtable (function pointers in declaration -/// order, then `free`) and the trampolines it points at. -fn render_vtable_class( - w: &mut CodeWriter, - cx: Cx<'_>, - module: &ModuleBinding, - cb: &CallbackInterfaceBinding, -) { - let iface = callback_interface_cs(&cb.name); - let class = vtable_class_cs(&module.path, &cb.name); - w.line(format!( - "/// The process-wide {} and the trampolines behind it,", - cb.vtable_tag - )); - w.line(format!( - "/// adapting an to the C ABI." - )); - w.line(format!("internal static unsafe class {class}")); - w.block("{", "}", |w| { - w.line("[StructLayout(LayoutKind.Sequential)]"); - w.line("private struct Layout"); - w.block("{", "}", |w| { - for m in &cb.methods { - w.line(format!( - "public {} {};", - fn_pointer_type(&m.abi_params, &m.abi_ret), - safe_cs_name(&m.name) - )); - } - w.line("public delegate* unmanaged[Cdecl] free;"); - }); - w.blank(); - w.line("/// The vtable's address, valid for the process lifetime."); - w.line("internal static readonly IntPtr Pointer = Allocate();"); - w.blank(); - w.line("private static IntPtr Allocate()"); - w.block("{", "}", |w| { - w.line("var vtable = (Layout*)NativeMemory.Alloc((nuint)sizeof(Layout));"); - for m in &cb.methods { - w.line(format!( - "vtable->{} = &{}Trampoline;", - safe_cs_name(&m.name), - method_cs(m) - )); - } - w.line("vtable->free = &FreeTrampoline;"); - w.line("return (IntPtr)vtable;"); - }); - w.blank(); - for m in &cb.methods { - render_trampoline(w, cx, &iface, m); - } - w.line(UNMANAGED_CALLERS_ONLY); - w.line("private static void FreeTrampoline(IntPtr ctx)"); - w.block("{", "}", |w| { - w.line("Ffi.Unregister(ctx);"); - }); - }); - w.blank(); -} - -/// One trampoline: receive the slots, call the implementation, and return -/// the direct result; any exception is reported through `out_err`. -fn render_trampoline(w: &mut CodeWriter, cx: Cx<'_>, iface: &str, m: &CallbackMethodBinding) { - let sig: Vec = m - .abi_params - .iter() - .map(|s| format!("{} {}", cs_ctype(&s.ty), safe_cs_name(&s.name))) - .collect(); - let ret = cs_ctype(&m.abi_ret); - w.line(UNMANAGED_CALLERS_ONLY); - w.line(format!( - "private static {ret} {}Trampoline({})", - method_cs(m), - sig.join(", ") - )); - w.block("{", "}", |w| { - w.line("try"); - w.block("{", "}", |w| { - let mut args = Vec::new(); - for (idx, p) in m.params.iter().enumerate() { - let ptr = safe_cs_name(&p.abi[0].name); - let len = p - .abi - .get(1) - .map(|s| safe_cs_name(&s.name)) - .unwrap_or_default(); - args.push(receive( - w, - cx, - &p.ty, - &ptr, - &len, - Own::Borrow, - &format!("arg{idx}"), - )); - } - let call = format!( - "Ffi.Target<{iface}>(ctx).{}({})", - method_cs(m), - args.join(", ") - ); - match &m.ret { - None => w.line(format!("{call};")), - Some(ty) => w.line(format!("return {};", direct_to_slot(ty, &call))), - }; - }); - w.line("catch (Exception e)"); - w.block("{", "}", |w| { - w.line("Ffi.SetForeignError(out_err, e);"); - if m.ret.is_some() { - w.line("return default;"); - } - }); - }); - w.blank(); -} diff --git a/crates/weaveffi-gen/src/targets/dotnet/codec.rs b/crates/weaveffi-gen/src/targets/dotnet/codec.rs deleted file mode 100644 index dce630e3..00000000 --- a/crates/weaveffi-gen/src/targets/dotnet/codec.rs +++ /dev/null @@ -1,146 +0,0 @@ -//! Value-buffer codec emitters: the statements encoding a C# value into an -//! `FfiBufferWriter` and decoding one back from an `FfiBufferReader`. -//! -//! Every dispatch goes through [`Ty::wire`], so the wire shapes are never -//! re-derived here. Primitives map onto `Write{Pascal}`/`Read{Pascal}` by -//! their [`Prim`](weaveffi_model::model::Prim) name. An object token carries -//! one strong reference: writing one clones the wrapper's handle through the -//! interface's `_clone` symbol, and reading one adopts the pointer into a new -//! wrapper. - -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use weaveffi_model::model::{Ty, WireType}; - -use crate::targets::dotnet::types::{cs_type, is_cs_value_type, Cx}; - -/// Emit statements serializing `expr` (a C# expression of the surface type -/// mapped from `ty`) into the writer named `writer`. `depth` keeps nested -/// loop locals unique. -pub(crate) fn emit_write(w: &mut CodeWriter, ty: &Ty, expr: &str, writer: &str, depth: usize) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("{writer}.Write{}({expr});", p.pascal())); - } - WireType::Enum(_) => { - w.line(format!("{writer}.WriteI32((int){expr});")); - } - // The token must carry its own strong reference, so clone the - // wrapper's handle rather than writing the pointer it still owns. - WireType::Object(_) => { - w.line(format!("{writer}.WriteObject({expr}.CloneHandle());")); - } - WireType::User(_) => { - w.line(format!("{expr}.WriteTo({writer});")); - } - WireType::Optional(inner) => { - let value = if is_cs_value_type(inner) { - format!("{expr}.Value") - } else { - format!("{expr}!") - }; - w.line(format!("{writer}.WriteBool({expr} != null);")); - w.line(format!("if ({expr} != null)")); - w.block("{", "}", |w| { - emit_write(w, inner, &value, writer, depth); - }); - } - WireType::List(inner) => { - let item = format!("item{depth}"); - w.line(format!("{writer}.WriteLen({expr}.Length);")); - w.line(format!("foreach (var {item} in {expr})")); - w.block("{", "}", |w| { - emit_write(w, inner, &item, writer, depth + 1); - }); - } - WireType::Map(k, v) => { - let entry = format!("entry{depth}"); - w.line(format!("{writer}.WriteLen({expr}.Count);")); - w.line(format!("foreach (var {entry} in {expr})")); - w.block("{", "}", |w| { - emit_write(w, k, &format!("{entry}.Key"), writer, depth + 1); - emit_write(w, v, &format!("{entry}.Value"), writer, depth + 1); - }); - } - } -} - -/// Emit statements declaring a local named `var` and decoding a value of `ty` -/// into it from the reader named `reader`, the inverse of [`emit_write`]. -pub(crate) fn emit_read( - w: &mut CodeWriter, - cx: Cx<'_>, - ty: &Ty, - var: &str, - reader: &str, - depth: usize, -) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("var {var} = {reader}.Read{}();", p.pascal())); - } - WireType::Enum(name) => { - let cn = local_type_name(name); - w.line(format!("var {var} = ({cn}){reader}.ReadI32();")); - } - // Adopt the token's strong reference into a fresh wrapper. - WireType::Object(name) => { - let cn = cx.ty(name); - w.line(format!("var {var} = {cn}.Adopt({reader}.ReadObject());")); - } - WireType::User(name) => { - let cn = cx.ty(name); - w.line(format!("var {var} = {cn}.ReadFrom({reader});")); - } - WireType::Optional(inner) => { - w.line(format!("{} {var} = null;", cs_type(ty))); - w.line(format!("if ({reader}.ReadBool())")); - w.block("{", "}", |w| { - emit_read(w, cx, inner, &format!("{var}Value"), reader, depth); - w.line(format!("{var} = {var}Value;")); - }); - } - // Elements can encode to zero bytes, so a count isn't bounded by the - // remaining input; the list grows from a capped capacity instead of - // preallocating whatever the count claims. - WireType::List(inner) => { - let i = format!("i{depth}"); - w.line(format!("var {var}Count = {reader}.ReadLen();")); - w.line(format!( - "var {var}List = new List<{}>({reader}.Capacity({var}Count));", - cs_type(inner) - )); - w.line(format!("for (var {i} = 0; {i} < {var}Count; {i}++)")); - w.block("{", "}", |w| { - emit_read(w, cx, inner, &format!("{var}Item"), reader, depth + 1); - w.line(format!("{var}List.Add({var}Item);")); - }); - w.line(format!("var {var} = {var}List.ToArray();")); - } - WireType::Map(k, v) => { - let i = format!("i{depth}"); - w.line(format!("var {var}Count = {reader}.ReadLen();")); - w.line(format!( - "var {var} = new Dictionary<{}, {}>({reader}.Capacity({var}Count));", - cs_type(k), - cs_type(v) - )); - w.line(format!("for (var {i} = 0; {i} < {var}Count; {i}++)")); - w.block("{", "}", |w| { - emit_read(w, cx, k, &format!("{var}Key"), reader, depth + 1); - emit_read(w, cx, v, &format!("{var}Val"), reader, depth + 1); - w.line(format!("{var}[{var}Key] = {var}Val;")); - }); - } - } -} - -/// Emit statements decoding a whole value buffer held by the reader -/// expression `reader_expr` into a local named `var`, then checking that the -/// buffer was fully consumed. -pub(crate) fn emit_decode(w: &mut CodeWriter, cx: Cx<'_>, ty: &Ty, var: &str, reader_expr: &str) { - let reader = format!("{var}Reader"); - w.line(format!("var {reader} = {reader_expr};")); - emit_read(w, cx, ty, var, &reader, 0); - w.line(format!("{reader}.ExpectEnd();")); -} diff --git a/crates/weaveffi-gen/src/targets/dotnet/runtime.rs b/crates/weaveffi-gen/src/targets/dotnet/runtime.rs deleted file mode 100644 index 79eebdd1..00000000 --- a/crates/weaveffi-gen/src/targets/dotnet/runtime.rs +++ /dev/null @@ -1,156 +0,0 @@ -//! The fixed runtime (`runtime/Runtime.cs`, spliced with the library's -//! names) and the per-domain typed exceptions. - -use crate::codegen::CodeWriter; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use heck::ToUpperCamelCase; -use weaveffi_model::errors; -use weaveffi_model::model::{BindingModel, ErrorBinding, ABI_VERSION}; - -use crate::targets::dotnet::codec::emit_read; -use crate::targets::dotnet::docs::write_doc; -use crate::targets::dotnet::types::{cs_str, xml_text, Cx}; - -/// The runtime source shared by every generated library. -const RUNTIME_CS: &str = include_str!("runtime/Runtime.cs"); - -/// The names a generated library splices into the fixed runtime. -pub(crate) struct RuntimeNames<'a> { - /// The C# namespace. - pub namespace: &'a str, - /// The base exception class (see [`base_exception_name`]). - pub exception: &'a str, - /// The C symbol prefix. - pub prefix: &'a str, - /// The native library base name P/Invoke loads. - pub library: &'a str, - /// The `{PREFIX}_LIBRARY` override variable. - pub library_env: &'a str, -} - -/// Render `Runtime.cs`: the fixed runtime with every placeholder replaced. -pub(crate) fn render_runtime( - names: &RuntimeNames<'_>, - input_basename: &str, - filename: &str, -) -> String { - let body = RUNTIME_CS - .replace("{{NAMESPACE}}", names.namespace) - .replace("{{EXCEPTION}}", names.exception) - .replace("{{PREFIX}}", names.prefix) - .replace("{{LIBRARY}}", names.library) - .replace("{{LIBRARY_ENV}}", names.library_env) - .replace("{{ABI_VERSION}}", &ABI_VERSION.to_string()); - format!( - "{}{body}\n{}", - render_prelude(CommentStyle::DoubleSlash, input_basename), - render_trailer(CommentStyle::DoubleSlash, filename) - ) -} - -/// The base exception every failure derives from: `NativeException`, unless -/// the API declares a type of that name, in which case the namespace -/// qualifies it (`KvstoreNativeException`). -pub(crate) fn base_exception_name(model: &BindingModel, namespace: &str) -> String { - const NAME: &str = "NativeException"; - let taken = model.modules.iter().any(|m| { - m.structs.iter().any(|s| s.name == NAME) - || m.enums.iter().any(|e| e.name == NAME) - || m.interfaces.iter().any(|i| i.name == NAME) - || m.error - .as_ref() - .is_some_and(|e| dotnet_exception_name(e) == NAME) - }); - if taken { - format!("{namespace}{NAME}") - } else { - NAME.to_string() - } -} - -/// The C# exception class for one error domain: the domain stem with one -/// `Exception` suffix, so `KvError` becomes `KvException`. -pub(crate) fn dotnet_exception_name(eb: &ErrorBinding) -> String { - errors::exception_type_name(&eb.type_name) -} - -/// One typed exception class per declared error domain, deriving from the -/// base exception. Each code is a `public const int`, and `FromError` maps a -/// raw error to the typed exception, decoding any payload fields into -/// `Data` (keyed by the IDL field name). Undeclared codes (the negative -/// runtime codes) fall back to the base exception's mapping. -pub(crate) fn render_domain_exception(w: &mut CodeWriter, eb: &ErrorBinding, cx: Cx<'_>) { - let exc = dotnet_exception_name(eb); - let base = cx.base; - w.line(format!( - "/// Typed exception for the {} error domain (module {}).", - eb.type_name, - eb.owner_path.replace('_', ".") - )); - w.line(format!("public class {exc} : {base}")); - w.block("{", "}", |w| { - for c in &eb.codes { - if c.doc.is_some() { - write_doc(w, &c.doc); - } else { - w.line(format!("/// {}", xml_text(&c.message))); - } - w.line(format!( - "public const int {} = {};", - errors::pascal(&c.name), - c.value - )); - w.blank(); - } - w.line("/// Creates an exception carrying a domain error code."); - w.line(format!( - "public {exc}(int code, string message) : base(code, message)" - )); - w.line("{"); - w.line("}"); - w.blank(); - w.line( - "internal static new Exception FromError(int code, string message, byte[]? payload)", - ); - w.block("{", "}", |w| { - w.line("switch (code)"); - w.block("{", "}", |w| { - for c in &eb.codes { - let default = cs_str(&c.message); - let ctor = format!( - "new {exc}(code, string.IsNullOrEmpty(message) ? \"{default}\" : message)" - ); - w.line(format!("case {}:", errors::pascal(&c.name))); - if c.fields.is_empty() { - w.indent(); - w.line(format!("return {ctor};")); - w.dedent(); - continue; - } - w.block("{", "}", |w| { - w.line(format!("var e = {ctor};")); - w.line("if (payload != null)"); - w.block("{", "}", |w| { - w.line("var reader = new FfiBufferReader(payload);"); - for f in &c.fields { - let var = format!("f{}", f.name.to_upper_camel_case()); - emit_read(w, cx, &f.ty, &var, "reader", 0); - w.line(format!("e.Data[\"{}\"] = {var};", cs_str(&f.name))); - } - w.line("reader.ExpectEnd();"); - }); - w.line("return e;"); - }); - } - w.line("default:"); - w.indent(); - w.line(format!( - "return {}.FromError(code, message, payload);", - cx.ty(base) - )); - w.dedent(); - }); - }); - }); - w.blank(); -} diff --git a/crates/weaveffi-gen/src/targets/dotnet/tests.rs b/crates/weaveffi-gen/src/targets/dotnet/tests.rs deleted file mode 100644 index 2b666f8e..00000000 --- a/crates/weaveffi-gen/src/targets/dotnet/tests.rs +++ /dev/null @@ -1,506 +0,0 @@ -//! Rendering tests over a small hand-built API exercising every ABI -//! revision 3 shape: objects, nullable objects, objects in records and -//! lists, iterators of objects, a callback interface, async and cancellable -//! functions, and bytes parameters. - -use crate::backend::LanguageBackend; -use crate::package::PackageContext; -use crate::platform::{BinarySet, Platform}; -use camino::Utf8Path; -use weaveffi_model::ir::{ - Api, CallbackInterfaceDef, Function, InterfaceDef, Module, Param, StructDef, StructField, - TypeRef, -}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; -use weaveffi_model::validate::validate_api; - -use crate::targets::dotnet::{DotnetConfig, DotnetGenerator}; - -fn param(name: &str, ty: TypeRef) -> Param { - Param { - name: name.into(), - ty, - doc: None, - } -} - -fn func(name: &str, params: Vec, returns: Option) -> Function { - Function { - name: name.into(), - params, - returns, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - } -} - -fn field(name: &str, ty: TypeRef) -> StructField { - StructField { - name: name.into(), - ty, - doc: None, - } -} - -fn named(name: &str) -> TypeRef { - TypeRef::Named(name.into()) -} - -/// A `bus` module with a `Ticker` interface, a `Bundle` record holding -/// objects, a `Subscriber` callback interface, and free functions covering -/// nullable objects, an iterator of objects, async and cancellable calls, a -/// callback-interface parameter, and bytes. -fn fixture() -> ResolvedApi { - let ticker = InterfaceDef { - name: "Ticker".into(), - doc: Some("A counter the producer owns.".into()), - deprecated: None, - constructors: vec![func("new", vec![param("start", TypeRef::I32)], None)], - methods: vec![func("value", vec![], Some(TypeRef::I32))], - statics: vec![], - }; - let bundle = StructDef { - name: "Bundle".into(), - doc: None, - deprecated: None, - fields: vec![ - field("primary", named("Ticker")), - field("others", TypeRef::List(Box::new(named("Ticker")))), - field("label", TypeRef::StringUtf8), - ], - }; - let subscriber = CallbackInterfaceDef { - name: "Subscriber".into(), - doc: Some("Receives bus events.".into()), - deprecated: None, - methods: vec![ - func( - "on_message", - vec![ - param("text", TypeRef::StringUtf8), - param("weight", TypeRef::I32), - param("bundle", named("Bundle")), - param("ticker", named("Ticker")), - ], - Some(TypeRef::Bool), - ), - func("on_close", vec![param("code", TypeRef::I64)], None), - ], - }; - let module = Module { - name: "bus".into(), - doc: None, - functions: vec![ - func( - "maybe_ticker", - vec![param("input", TypeRef::Optional(Box::new(named("Ticker"))))], - Some(TypeRef::Optional(Box::new(named("Ticker")))), - ), - func( - "tickers", - vec![], - Some(TypeRef::Iterator(Box::new(named("Ticker")))), - ), - func( - "subscribe", - vec![param("listener", named("Subscriber"))], - None, - ), - Function { - r#async: true, - ..func("spawn", vec![], Some(named("Ticker"))) - }, - Function { - r#async: true, - cancellable: true, - ..func("wait", vec![param("ms", TypeRef::I64)], Some(TypeRef::I64)) - }, - func( - "roundtrip_bundle", - vec![param("b", named("Bundle"))], - Some(named("Bundle")), - ), - // A method named like the type it returns. - func("bundle", vec![], Some(named("Bundle"))), - func( - "digest", - vec![ - param("data", TypeRef::Bytes), - param("label", TypeRef::StringUtf8), - ], - Some(TypeRef::StringUtf8), - ), - ], - interfaces: vec![ticker], - callback_interfaces: vec![subscriber], - structs: vec![bundle], - enums: vec![], - errors: None, - modules: vec![], - }; - validate_api( - Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![module], - }, - None, - ) - .expect("fixture validates") - .with_identity(Identity::named("bus-lib")) -} - -/// Every generated file's text, keyed by file name. -fn render_files(config: &DotnetConfig) -> Vec<(String, String)> { - let api = fixture(); - let model = BindingModel::build(&api); - DotnetGenerator - .files(&api, &model, Utf8Path::new("out"), config) - .into_iter() - .map(|f| (f.path.file_name().unwrap().to_string(), f.contents)) - .collect() -} - -fn file(files: &[(String, String)], name: &str) -> String { - files - .iter() - .find(|(n, _)| n == name) - .unwrap_or_else(|| { - panic!( - "no {name} in {:?}", - files.iter().map(|f| &f.0).collect::>() - ) - }) - .1 - .clone() -} - -fn render() -> String { - file(&render_files(&DotnetConfig::default()), "BusLib.cs") -} - -fn assert_has(out: &str, needle: &str) { - assert!(out.contains(needle), "missing `{needle}` in:\n{out}"); -} - -#[test] -fn names_come_from_the_identity() { - let files = render_files(&DotnetConfig::default()); - let names: Vec<&str> = files.iter().map(|f| f.0.as_str()).collect(); - assert_eq!( - names, - ["BusLib.cs", "Runtime.cs", "BusLib.csproj", "README.md"] - ); - let runtime = file(&files, "Runtime.cs"); - assert_has(&runtime, "namespace BusLib;"); - assert_has(&runtime, "internal const string LibName = \"bus_lib\";"); - assert_has( - &runtime, - "internal const string LibraryEnvVar = \"BUS_LIB_LIBRARY\";", - ); - assert_has(&runtime, "EntryPoint = \"bus_lib_abi_version\""); - assert_has(&runtime, "internal const uint AbiVersion = 3;"); - assert_has(&runtime, "public class NativeException : Exception"); - assert_has(&runtime, "public const int CancelledErrorCode = -5;"); - assert!( - !runtime.contains("{{"), - "unreplaced placeholder:\n{runtime}" - ); - let csproj = file(&files, "BusLib.csproj"); - assert_has(&csproj, "BusLib"); - assert_has(&csproj, "BusLib"); - for (name, text) in &files { - let body: String = text - .lines() - .filter(|l| !l.contains("Generated by WeaveFFI") && !l.contains("weaveffi generate")) - .collect(); - assert!( - !body.to_lowercase().contains("weaveffi"), - "{name} is branded:\n{text}" - ); - } -} - -#[test] -fn configured_namespace_wins() { - let config = DotnetConfig { - namespace: Some("Acme.Bus".into()), - ..DotnetConfig::default() - }; - let files = render_files(&config); - assert_has(&file(&files, "Acme.Bus.cs"), "namespace Acme.Bus;"); - assert_has( - &file(&files, "Acme.Bus.csproj"), - "Acme.Bus", - ); -} - -#[test] -fn load_time_check_covers_every_root_checksum() { - let out = render(); - assert_has(&out, "static partial void VerifyChecksums()"); - assert_has(&out, "VerifyChecksum(\"bus\", &bus_lib_bus_checksum, 0x"); - assert_has( - &out, - "internal static partial ulong bus_lib_bus_checksum();", - ); -} - -#[test] -fn interfaces_wrap_a_safe_handle() { - let out = render(); - assert_has( - &out, - "public sealed unsafe class Ticker : IDisposable, IEquatable", - ); - assert_has(&out, "internal sealed class NativeHandle : SafeHandle"); - assert_has(&out, "NativeMethods.bus_lib_bus_Ticker_destroy(handle);"); - assert!( - !out.contains("~Ticker()"), - "the SafeHandle is the finalizer" - ); - // Every import borrows through the handle, never a raw pointer. - assert_has( - &out, - "internal static partial int bus_lib_bus_Ticker_value(Ticker.NativeHandle self, FfiError* out_err);", - ); - assert_has( - &out, - "internal static partial IntPtr bus_lib_bus_Ticker_clone(Ticker.NativeHandle self);", - ); - assert_has( - &out, - "var ffiResult = NativeMethods.bus_lib_bus_Ticker_value(Handle, &ffiErr);", - ); - assert_has(&out, "public Ticker(int start)"); - assert_has(&out, "Handle = new NativeHandle(ffiResult);"); - assert_has( - &out, - "return NativeMethods.bus_lib_bus_Ticker_clone(Handle);", - ); -} - -#[test] -fn nullable_objects_pass_the_null_handle() { - let out = render(); - assert_has(&out, "public static Ticker? MaybeTicker(Ticker? input)"); - assert_has( - &out, - "NativeMethods.bus_lib_bus_maybe_ticker(input?.Handle ?? global::BusLib.Ticker.NativeHandle.Null, &ffiErr);", - ); - assert_has( - &out, - "return ffiResult == IntPtr.Zero ? null : global::BusLib.Ticker.Adopt(ffiResult);", - ); -} - -#[test] -fn objects_in_records_use_cloned_tokens_and_adopt_on_read() { - let out = render(); - assert_has(&out, "public sealed class Bundle"); - assert_has(&out, "writer.WriteObject(Primary.CloneHandle());"); - assert_has(&out, "writer.WriteObject(item0.CloneHandle());"); - assert_has( - &out, - "var fPrimary = global::BusLib.Ticker.Adopt(reader.ReadObject());", - ); - assert_has( - &out, - "var fOthersItem = global::BusLib.Ticker.Adopt(reader.ReadObject());", - ); -} - -#[test] -fn type_references_survive_a_member_of_the_same_name() { - let out = render(); - // Inside `Bus`, the method `Bundle()` would shadow the type `Bundle` in - // an expression, so static accesses go through the global namespace. - assert_has(&out, "public static Bundle Bundle()"); - assert_has( - &out, - "var ffiDecoded = global::BusLib.Bundle.ReadFrom(ffiDecodedReader);", - ); -} - -#[test] -fn strings_and_bytes_cross_as_pointer_and_length() { - let out = render(); - assert_has( - &out, - "internal static partial byte* bus_lib_bus_digest(byte* data_ptr, nuint data_len, byte* label_ptr, nuint label_len, nuint* out_len, FfiError* out_err);", - ); - assert_has( - &out, - "public static string Digest(ReadOnlySpan data, string label)", - ); - assert_has(&out, "var labelBytes = Ffi.Utf8(label);"); - assert_has(&out, "fixed (byte* dataPtr = data, labelPtr = labelBytes)"); - assert_has(&out, "return Ffi.TakeString(ffiResult, ffiOutLen);"); - // The byte[] overload forwards to the span one. - assert_has( - &out, - "public static string Digest(byte[] data, string label) => Digest((ReadOnlySpan)data, label);", - ); -} - -#[test] -fn iterators_stream_through_a_safe_handle() { - let out = render(); - assert_has(&out, "public static IEnumerable Tickers()"); - assert_has( - &out, - "return new FfiSequence(new FfiIteratorHandle(ffiResult, &NativeMethods.bus_lib_bus_TickersIterator_destroy), &NextTickers);", - ); - assert_has( - &out, - "private static bool NextTickers(FfiIteratorHandle iter, out Ticker item)", - ); - assert_has( - &out, - "internal static partial int bus_lib_bus_TickersIterator_next(FfiIteratorHandle iter, IntPtr* out_item, FfiError* out_err);", - ); - assert_has(&out, "item = global::BusLib.Ticker.Adopt(ffiItem);"); -} - -#[test] -fn async_calls_complete_through_an_unmanaged_callback() { - let out = render(); - assert_has(&out, "public static Task Spawn()"); - assert_has(&out, "var ffiCall = new FfiCall();"); - assert_has( - &out, - "NativeMethods.bus_lib_bus_spawn(&CompleteSpawn, ffiCall.Context);", - ); - assert_has( - &out, - "internal static partial void bus_lib_bus_spawn(delegate* unmanaged[Cdecl] callback, IntPtr context);", - ); - assert_has( - &out, - "private static void CompleteSpawn(IntPtr context, FfiError* err, IntPtr result)", - ); - assert_has( - &out, - "ffiCall.SetResult(global::BusLib.Ticker.Adopt(result));", - ); - assert_has(&out, "ffiCall.Abandon();"); -} - -#[test] -fn cancellable_calls_take_a_cancellation_token() { - let out = render(); - assert_has( - &out, - "public static Task Wait(long ms, CancellationToken cancellationToken = default)", - ); - assert_has(&out, "return Task.FromCanceled(cancellationToken);"); - assert_has(&out, "var ffiCall = new FfiCall(cancellationToken);"); - assert_has( - &out, - "NativeMethods.bus_lib_bus_wait(ms, ffiCall.CancelToken(), &CompleteWait, ffiCall.Context);", - ); - let runtime = file(&render_files(&DotnetConfig::default()), "Runtime.cs"); - assert_has(&runtime, "_tcs.TrySetCanceled(_cancellation);"); - assert_has(&runtime, "NativeMethods.CancelTokenCancel(_token);"); -} - -#[test] -fn callback_interfaces_render_a_static_vtable_of_trampolines() { - let out = render(); - assert_has(&out, "public interface ISubscriber"); - assert_has( - &out, - "bool OnMessage(string text, int weight, Bundle bundle, Ticker ticker);", - ); - assert_has( - &out, - "internal static unsafe class FfiVtable_bus_Subscriber", - ); - assert_has( - &out, - "public delegate* unmanaged[Cdecl] on_message;", - ); - assert_has(&out, "vtable->on_message = &OnMessageTrampoline;"); - assert_has(&out, "vtable->free = &FreeTrampoline;"); - assert_has( - &out, - "private static byte OnMessageTrampoline(IntPtr ctx, byte* text_ptr, nuint text_len, int weight, byte* bundle_ptr, nuint bundle_len, IntPtr ticker, FfiError* out_err)", - ); - assert_has( - &out, - "return (byte)(Ffi.Target(ctx).OnMessage(Ffi.ReadString(text_ptr, text_len), weight, arg2, global::BusLib.Ticker.Adopt(ticker)) ? 1 : 0);", - ); - assert_has(&out, "Ffi.SetForeignError(out_err, e);"); - // Arguments are borrowed: nothing is freed inside a trampoline. - let start = out.find("OnMessageTrampoline(IntPtr").unwrap(); - let end = out[start..].find("FreeTrampoline(IntPtr").unwrap() + start; - assert!(!out[start..end].contains("Take"), "{}", &out[start..end]); -} - -#[test] -fn passing_a_callback_interface_registers_it_after_encoding() { - let out = render(); - assert_has(&out, "public static void Subscribe(ISubscriber listener)"); - assert_has(&out, "var listenerCtx = Ffi.Register(listener);"); - assert_has( - &out, - "NativeMethods.bus_lib_bus_subscribe(listenerCtx, FfiVtable_bus_Subscriber.Pointer, &ffiErr);", - ); - // A call that never reaches the producer releases the registration. - assert_has(&out, "Ffi.Unregister(listenerCtx);"); -} - -#[test] -fn output_is_deterministic() { - assert_eq!( - render_files(&DotnetConfig::default()), - render_files(&DotnetConfig::default()) - ); -} - -#[test] -fn package_bundles_desktop_runtimes_only() { - let api = fixture(); - let model = BindingModel::build(&api); - let mut binaries = BinarySet::new("bus_lib"); - for p in Platform::ALL { - binaries.insert(p, format!("/tmp/{}/lib", p.id())); - } - let ctx = PackageContext { - binaries: &binaries, - input_basename: Some("bus"), - }; - let files = DotnetGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &DotnetConfig::default(), - ) - .expect("dotnet packages"); - let paths: Vec = files.iter().map(|f| f.path.to_string()).collect(); - let native: Vec<&String> = paths.iter().filter(|p| p.contains("/runtimes/")).collect(); - assert_eq!(native.len(), Platform::DESKTOP.len(), "{paths:?}"); - for rid in [ - "osx-arm64", - "osx-x64", - "linux-x64", - "linux-arm64", - "win-x64", - ] { - assert!( - paths - .iter() - .any(|p| p.contains(&format!("/runtimes/{rid}/native/"))), - "{paths:?}" - ); - } - assert!(!paths - .iter() - .any(|p| p.contains("android") || p.contains("wasm"))); -} diff --git a/crates/weaveffi-gen/src/targets/go/calls.rs b/crates/weaveffi-gen/src/targets/go/calls.rs deleted file mode 100644 index 1771bd52..00000000 --- a/crates/weaveffi-gen/src/targets/go/calls.rs +++ /dev/null @@ -1,967 +0,0 @@ -//! Call rendering: sync, async, and iterator wrappers, callback-interface -//! types and trampolines, and the argument/return marshalling they share. -//! -//! Marshalling dispatch follows the shared plan layer: each parameter's -//! passing contract comes from [`ParamBinding::arg_pass`], each result's -//! receiving contract from [`plan::ret_pass`], so this module only spells -//! those contracts in Go rather than re-deriving them from `Ty`. - -use crate::codegen::CodeWriter; -use crate::lang; -use heck::ToUpperCamelCase; -use weaveffi_model::abi::{AbiParam, CType}; -use weaveffi_model::model::Ty; -use weaveffi_model::model::{ - AsyncBinding, BindingModel, CallShape, CallbackInterfaceBinding, CallbackMethodBinding, - FnBinding, IteratorBinding, ParamBinding, -}; -use weaveffi_model::plan::{self, ArgPass, ErrorStrategy, RetPass}; - -use crate::targets::go::codec::{emit_buffer_read, emit_buffer_write}; -use crate::targets::go::docs::{emit_doc, emit_fn_doc}; -use crate::targets::go::types::{ - cgo_type, from_c_direct, go_adopt_expr, go_param_ident, go_type, go_zero, strip_const, - to_c_direct, vtable_accessor, vtable_var, -}; - -/// Package names the generated bindings use inside function bodies. A slot -/// or parameter spelled like one of them is escaped so it can't shadow the -/// package. -const SHADOWED_PACKAGES: &[&str] = &["cgo", "context", "fmt", "iter", "runtime", "unsafe"]; - -// ── Errors ── - -/// How a wrapper body reports a non-zero error slot. -/// -/// A callable with `throws == true` returns `(T, error)` and maps codes -/// through the declaring module's typed helper (`wvMapKv`), falling back to -/// the generic `Error` when no domain is in scope. A callable with -/// `throws == false` has a plain signature and panics via `wvTrap` instead, -/// since a reported error can only be a producer panic, an -/// argument-marshalling failure, or a callback implementation that panicked. -#[derive(Clone, Copy)] -pub(crate) struct ErrCtx<'a> { - /// `true` when the wrapper returns `(T, error)` and surfaces typed errors. - pub(crate) throws: bool, - /// PascalCase stem of the domain in effect (`Kv` names `wvMapKv`); `None` - /// falls back to the generic error. - stem: Option<&'a str>, -} - -impl<'a> ErrCtx<'a> { - /// Build the wrapper error context for `f` from the shared plan's - /// [`ErrorStrategy`]. - pub(crate) fn of(f: &FnBinding, stem: Option<&'a str>) -> Self { - Self { - throws: matches!(f.error_strategy(), ErrorStrategy::Throws), - stem, - } - } - - /// The Go expression converting the `wvFailure` value `failure` into an - /// `error`. - fn map_call(&self, failure: &str) -> String { - match self.stem { - Some(stem) => format!("wvMap{stem}({failure})"), - None => format!("{failure}.err()"), - } - } - - /// Emit the statement(s) checking the error slot named `slot`. A throwing - /// wrapper returns `zero` (when the function has a result) plus the - /// mapped error; a plain wrapper traps. - fn emit_check(&self, w: &mut CodeWriter, slot: &str, zero: Option<&str>) { - if self.throws { - let map = self.map_call(&format!("wvTakeError(&{slot})")); - w.block(format!("if {slot}.code != 0 {{"), "}", |w| { - match zero { - Some(z) => w.line(format!("return {z}, {map}")), - None => w.line(format!("return {map}")), - }; - }); - } else { - w.line(format!("wvTrap(&{slot})")); - } - } - - /// The Go return-type suffix (including the leading space) of a sync - /// wrapper returning `ret`. - fn ret_sig(&self, ret: Option<&Ty>) -> String { - match (ret, self.throws) { - (Some(r), true) => format!(" ({}, error)", go_type(r)), - (Some(r), false) => format!(" {}", go_type(r)), - (None, true) => " error".into(), - (None, false) => String::new(), - } - } - - /// The suffix of every successful `return`: `, nil` when the wrapper also - /// returns an error. - fn ok_tail(&self) -> &'static str { - if self.throws { - ", nil" - } else { - "" - } - } -} - -// ── Trampolines and the cgo preamble ── - -/// The Go spelling of one C ABI slot name inside an exported trampoline's -/// formal list: a slot named after a Go keyword or a package the body uses -/// gains a trailing underscore. The preamble `extern` uses the same spelling -/// so the two prototypes agree. -fn go_slot_ident(name: &str) -> String { - let ident = lang::escape_ident(name, lang::GO_KEYWORDS); - if SHADOWED_PACKAGES.contains(&ident.as_str()) { - format!("{ident}_") - } else { - ident - } -} - -/// The Go spelling of a wrapper parameter, escaped against keywords, the -/// packages wrapper bodies use, and (in an async wrapper) the leading `ctx`. -fn param_ident(p: &ParamBinding, is_async: bool) -> String { - let ident = go_param_ident(&p.name, is_async); - if SHADOWED_PACKAGES.contains(&ident.as_str()) { - format!("{ident}_") - } else { - ident - } -} - -/// The C name of the exported Go trampoline for an async completion typedef. -fn trampoline_name(c_type_name: &str) -> String { - format!("goWv_{c_type_name}") -} - -/// The C name of the exported Go trampoline behind one callback-interface -/// vtable entry (`goWv_{c_tag}_{method}`); `free` names the trailing entry. -fn cb_trampoline_name(c_tag: &str, method: &str) -> String { - format!("goWv_{c_tag}_{method}") -} - -/// The preamble `extern` declaration for one exported trampoline. Pointer -/// types are rendered const-free to match the prototypes cgo writes into -/// `_cgo_export.h` from the Go signature. -fn extern_decl(name: &str, ret: &CType, params: &[AbiParam], prefix: &str) -> String { - let args: Vec = params - .iter() - .map(|p| { - format!( - "{} {}", - strip_const(&p.ty).render_c(prefix), - go_slot_ident(&p.name) - ) - }) - .collect(); - format!( - "extern {} {name}({});", - ret.render_c(prefix), - args.join(", ") - ) -} - -/// The preamble definition of the one process-wide static vtable for `cb`: -/// one trampoline per method in declaration order, then `free`. An entry -/// whose C signature carries `const` pointers is cast back to the vtable's -/// exact field type, since the exported Go function is declared const-free. -fn vtable_def(cb: &CallbackInterfaceBinding, prefix: &str) -> String { - let mut s = format!( - "static const {} {} = {{\n", - cb.vtable_tag, - vtable_var(&cb.vtable_tag) - ); - for m in &cb.methods { - let tramp = cb_trampoline_name(&cb.c_tag, &m.name); - let needs_cast = m.abi_params.iter().any(|p| strip_const(&p.ty) != p.ty); - if needs_cast { - let types: Vec = m.abi_params.iter().map(|p| p.ty.render_c(prefix)).collect(); - s.push_str(&format!( - " ({} (*)({})){tramp},\n", - m.abi_ret.render_c(prefix), - types.join(", ") - )); - } else { - s.push_str(&format!(" {tramp},\n")); - } - } - s.push_str(&format!( - " {},\n}};\n", - cb_trampoline_name(&cb.c_tag, "free") - )); - // cgo reaches a C variable through `//go:cgo_import_static`, which needs - // external linkage, so a `static const` table can't be named as - // `&C.wvVtable_...` from Go. A static accessor function is callable - // through the ordinary cgo stub in the same translation unit. - s.push_str(&format!( - "__attribute__((unused)) static const {}* {}(void) {{ return &{}; }}", - cb.vtable_tag, - vtable_accessor(&cb.vtable_tag), - vtable_var(&cb.vtable_tag) - )); - s -} - -/// Every declaration the cgo preamble needs beyond the header include: for -/// each callback interface, the `extern` prototypes of its method and `free` -/// trampolines followed by its static vtable and accessor; then one `extern` -/// per async completion callback, including async interface members. -/// -/// A file that uses `//export` may only put declarations in its preamble -/// (the preamble is compiled into two C translation units); the vtable is a -/// `static const` so each unit gets a private copy and no symbol is -/// duplicated. Go takes the address of the copy in its own unit through the -/// static accessor, so the producer always sees one vtable whose entries -/// live for the process. -pub(crate) fn collect_preamble_decls(model: &BindingModel) -> Vec { - let prefix = model.prefix.as_str(); - let mut decls = Vec::new(); - for m in &model.modules { - for cb in &m.callback_interfaces { - for meth in &cb.methods { - decls.push(extern_decl( - &cb_trampoline_name(&cb.c_tag, &meth.name), - &meth.abi_ret, - &meth.abi_params, - prefix, - )); - } - decls.push(extern_decl( - &cb_trampoline_name(&cb.c_tag, "free"), - &CType::Void, - &[AbiParam::new("ctx", CType::ptr(CType::Void))], - prefix, - )); - decls.push(vtable_def(cb, prefix)); - } - for f in m.callables() { - if let CallShape::Async(ab) = &f.shape { - decls.push(extern_decl( - &trampoline_name(&ab.callback_type), - &CType::Void, - &ab.callback_params, - prefix, - )); - } - } - } - decls -} - -// ── Callback interfaces ── - -/// The Go method signature of one callback-interface method, as it appears -/// in the consumer-implemented interface type: `OnMessage(text string, -/// weight int32) int64`. -fn cb_method_sig(m: &CallbackMethodBinding) -> String { - let params: Vec = m - .params - .iter() - .map(|p| format!("{} {}", param_ident(p, false), go_type(&p.ty))) - .collect(); - let ret = match &m.ret { - Some(ty) => format!(" {}", go_type(ty)), - None => String::new(), - }; - format!( - "{}({}){ret}", - m.name.to_upper_camel_case(), - params.join(", ") - ) -} - -/// Emit statements converting one callback-method parameter's C slots into -/// a Go value bound to `arg{idx}`, returning that local's name. -/// -/// Strings, bytes, and buffers arriving in a trampoline are borrowed for the -/// dispatch: they're copied or decoded and nothing is freed. An object -/// argument transfers one strong reference, which is adopted into a wrapper -/// (a null `Interface?` adopts to nil). -fn emit_cb_param_arg(w: &mut CodeWriter, idx: usize, p: &ParamBinding) -> String { - let arg = format!("arg{idx}"); - match p.arg_pass() { - ArgPass::Buffer { ptr, len } => { - let r = format!("rArg{idx}"); - w.line(format!( - "{r} := &wvReader{{buf: wvBorrowBytes({}, {})}}", - go_slot_ident(&ptr.name), - go_slot_ident(&len.name) - )); - w.line(format!("var {arg} {}", go_type(&p.ty))); - emit_buffer_read(w, &r, &arg, &p.ty, &format!("Arg{idx}"), 0); - w.line(format!("{r}.expectEnd()")); - } - ArgPass::String { ptr, len } => { - w.line(format!( - "{arg} := wvBorrowString({}, {})", - go_slot_ident(&ptr.name), - go_slot_ident(&len.name) - )); - } - ArgPass::Bytes { ptr, len } => { - w.line(format!( - "{arg} := wvBorrowBytes({}, {})", - go_slot_ident(&ptr.name), - go_slot_ident(&len.name) - )); - } - ArgPass::Object { slot, .. } => { - w.line(format!( - "{arg} := {}", - go_adopt_expr(&p.ty, &go_slot_ident(&slot.name)) - )); - } - ArgPass::Direct { slot } => { - w.line(format!( - "{arg} := {}", - from_c_direct(&go_slot_ident(&slot.name), &p.ty) - )); - } - ArgPass::Callback { .. } => { - unreachable!("validation rejects callback interfaces as callback-method parameters") - } - } - arg -} - -/// Emit the exported trampoline behind one vtable entry. It recovers the -/// implementation from the `cgo.Handle` passed as `ctx`, converts the -/// borrowed arguments, calls the Go method, and writes a direct-family -/// result into the C return. A panic in the implementation is recovered and -/// reported through `{prefix}_error_set(out_err, -4, message)`, and the zero -/// value is returned; nothing ever unwinds through the C frame. -fn render_cb_trampoline( - w: &mut CodeWriter, - prefix: &str, - cb: &CallbackInterfaceBinding, - m: &CallbackMethodBinding, - iface_name: &str, -) { - let tramp = cb_trampoline_name(&cb.c_tag, &m.name); - let formals: Vec = m - .abi_params - .iter() - .map(|s| format!("{} {}", go_slot_ident(&s.name), cgo_type(&s.ty, prefix))) - .collect(); - let ctx = go_slot_ident(&m.abi_params[0].name); - let out_err = go_slot_ident(&m.abi_params[m.abi_params.len() - 1].name); - let ret_sig = match &m.ret { - Some(_) => format!(" (ret {})", cgo_type(&m.abi_ret, prefix)), - None => String::new(), - }; - w.line(format!("//export {tramp}")); - w.block( - format!("func {tramp}({}){ret_sig} {{", formals.join(", ")), - "}", - |w| { - w.block("defer func() {", "}()", |w| { - w.block("if r := recover(); r != nil {", "}", |w| { - w.line(format!("wvForeignError({out_err}, r)")); - }); - }); - w.line(format!( - "impl := cgo.Handle(uintptr({ctx})).Value().({iface_name})" - )); - let args: Vec = m - .params - .iter() - .enumerate() - .map(|(idx, p)| emit_cb_param_arg(w, idx, p)) - .collect(); - let call = format!("impl.{}({})", m.name.to_upper_camel_case(), args.join(", ")); - if m.ret.is_some() { - w.line(format!("ret = {}", to_c_direct(&call, &m.abi_ret, prefix))); - w.line("return"); - } else { - w.line(call); - } - }, - ); - w.blank(); -} - -/// Render one callback interface: the Go `interface` type the consumer -/// implements (one method per IDL method, PascalCase, direct-family or void -/// returns), the exported trampoline behind each vtable entry, and the -/// `free` trampoline that deletes the `cgo.Handle` once the producer drops -/// its last reference to the callback. -/// -/// Passing an implementation to a producer function stores it in a -/// `cgo.Handle` (the `void* ctx` slot) and passes the address of the -/// interface's static vtable from the cgo preamble (see -/// [`collect_preamble_decls`]). The producer may invoke any trampoline from -/// any thread; cgo attaches the calling thread to the Go runtime. -pub(crate) fn render_callback_interface( - out: &mut String, - prefix: &str, - cb: &CallbackInterfaceBinding, -) { - let name = cb.name.to_upper_camel_case(); - let mut w = CodeWriter::tabs(); - let mut d = String::new(); - emit_doc(&mut d, &cb.doc, "", Some(&name)); - if d.is_empty() { - w.line(format!( - "// {name} is a callback interface: implement it in Go and pass the value to" - )); - w.line("// native functions that accept it."); - } else { - w.raw(d); - w.line("//"); - w.line("// Implement this interface in Go and pass the value to native functions"); - w.line("// that accept it."); - } - w.line("//"); - w.line("// The native library may call any method from any thread until it releases"); - w.line("// the implementation. A panic in a method is reported to the native caller"); - w.line("// as a foreign error (code -4) instead of crashing the process."); - if let Some(msg) = &cb.deprecated { - w.line("//"); - w.line(format!("// Deprecated: {msg}")); - } - w.block(format!("type {name} interface {{"), "}", |w| { - for m in &cb.methods { - let mut md = String::new(); - emit_fn_doc( - &mut md, - &m.doc, - &m.params, - "\t", - &m.name.to_upper_camel_case(), - ); - let has_doc = !md.is_empty(); - w.raw(md); - emit_deprecated(w, has_doc, &m.deprecated); - w.line(cb_method_sig(m)); - } - }); - w.blank(); - - for m in &cb.methods { - render_cb_trampoline(&mut w, prefix, cb, m, &name); - } - - let free = cb_trampoline_name(&cb.c_tag, "free"); - w.line(format!("//export {free}")); - w.block(format!("func {free}(ctx unsafe.Pointer) {{"), "}", |w| { - w.line("cgo.Handle(uintptr(ctx)).Delete()"); - }); - w.blank(); - out.push_str(&w.finish()); -} - -// ── Shared wrapper pieces ── - -/// The wrapper signature line: `func (s *Store) Name(params) ret {` or -/// `func Name(params) ret {`. -fn header(receiver: Option<&str>, go_name: &str, params: &[String], ret_sig: &str) -> String { - match receiver { - Some(ty) => format!( - "func (s *{ty}) {go_name}({}){ret_sig} {{", - params.join(", ") - ), - None => format!("func {go_name}({}){ret_sig} {{", params.join(", ")), - } -} - -/// The Go formal parameter list of a wrapper. -fn go_params(f: &FnBinding) -> Vec { - let mut params: Vec = Vec::new(); - if f.is_async { - params.push("ctx context.Context".into()); - } - params.extend( - f.params - .iter() - .map(|p| format!("{} {}", param_ident(p, f.is_async), go_type(&p.ty))), - ); - params -} - -/// Emit the receiver borrow every method opens with: `s.native()` panics -/// when the wrapper was already closed, and the deferred release keeps the -/// native object alive (even across a concurrent `Close`) until the call -/// returns. -fn emit_receiver(w: &mut CodeWriter, args: &mut Vec) { - w.line("cSelf := s.native()"); - w.line("defer s.ref.release()"); - args.push("cSelf".into()); -} - -/// Emit the staging statements and C argument expressions for one Go -/// parameter, dispatching on the shared [`ArgPass`] contract. -/// -/// Strings and bytes pass a borrowed view of the Go memory (cgo pins it for -/// the call); a buffered parameter is packed into a `wvWriter` first. An -/// object parameter borrows the wrapper's pointer for the call, exactly like -/// a receiver; a nil `Interface?` passes NULL. A callback interface stores -/// the implementation in a `cgo.Handle` passed as `ctx` alongside the -/// address of the interface's static vtable. -fn emit_param( - w: &mut CodeWriter, - args: &mut Vec, - p: &ParamBinding, - prefix: &str, - is_async: bool, -) { - let name = param_ident(p, is_async); - let pascal = name.to_upper_camel_case(); - match p.arg_pass() { - ArgPass::Buffer { .. } => { - let wv = format!("w{pascal}"); - w.line(format!("{wv} := &wvWriter{{}}")); - emit_buffer_write(w, &wv, &name, &p.ty, &pascal, 0); - w.line(format!("c{pascal}Ptr, c{pascal}Len := wvBytes({wv}.buf)")); - args.push(format!("c{pascal}Ptr")); - args.push(format!("c{pascal}Len")); - } - ArgPass::String { .. } => { - w.line(format!("c{pascal}Ptr, c{pascal}Len := wvStr({name})")); - args.push(format!("c{pascal}Ptr")); - args.push(format!("c{pascal}Len")); - } - ArgPass::Bytes { .. } => { - w.line(format!("c{pascal}Ptr, c{pascal}Len := wvBytes({name})")); - args.push(format!("c{pascal}Ptr")); - args.push(format!("c{pascal}Len")); - } - ArgPass::Object { slot, nullable } => { - let cv = format!("c{pascal}"); - if nullable { - w.line(format!( - "var {cv} {}", - cgo_type(&strip_const(&slot.ty), prefix) - )); - w.block(format!("if {name} != nil {{"), "}", |w| { - w.line(format!("{cv} = {name}.native()")); - w.line(format!("defer {name}.ref.release()")); - }); - } else { - w.line(format!("{cv} := {name}.native()")); - w.line(format!("defer {name}.ref.release()")); - } - args.push(cv); - } - ArgPass::Callback { vtable, .. } => { - let CType::Ptr { pointee, .. } = &vtable.ty else { - unreachable!("a callback vtable slot is a pointer") - }; - let hv = format!("h{pascal}"); - w.line(format!("{hv} := cgo.NewHandle({name})")); - args.push(format!("C.wvHandlePtr(C.uintptr_t({hv}))")); - args.push(format!( - "C.{}()", - vtable_accessor(&pointee.render_c(prefix)) - )); - } - ArgPass::Direct { slot } => args.push(to_c_direct(&name, &slot.ty, prefix)), - } -} - -/// Emit the statements receiving one producer-owned value per its -/// [`RetPass`] contract, binding it to the Go local `dst` (declared here). -/// `ptr` and `len` name the C slots holding it (`len` only for strings, -/// bytes, and buffers). -/// -/// Strings and bytes are copied and released with `{prefix}_free_bytes`; a -/// buffer is copied, released, decoded, and checked for trailing bytes; an -/// object transfers one strong reference that the wrapper adopts (a null -/// `Interface?` adopts to nil). -fn emit_receive(w: &mut CodeWriter, ty: &Ty, pass: &RetPass, dst: &str, ptr: &str, len: &str) { - match receive_expr(ty, pass, ptr, len) { - Some(expr) => { - w.line(format!("{dst} := {expr}")); - } - None => { - let r = format!("r{}", dst.to_upper_camel_case()); - w.line(format!( - "{r} := &wvReader{{buf: wvTakeBytes({ptr}, {len})}}" - )); - w.line(format!("var {dst} {}", go_type(ty))); - emit_buffer_read(w, &r, dst, ty, &dst.to_upper_camel_case(), 0); - w.line(format!("{r}.expectEnd()")); - } - } -} - -/// The single Go expression receiving a value per `pass`, or `None` for a -/// buffer, which needs decoding statements (see [`emit_receive`]). -fn receive_expr(ty: &Ty, pass: &RetPass, ptr: &str, len: &str) -> Option { - match pass { - RetPass::Void => unreachable!("a present value is never void"), - RetPass::Direct => Some(from_c_direct(ptr, ty)), - RetPass::String => Some(format!("wvTakeString({ptr}, {len})")), - RetPass::Bytes => Some(format!("wvTakeBytes({ptr}, {len})")), - RetPass::Object { .. } => Some(go_adopt_expr(ty, ptr)), - RetPass::Buffer => None, - } -} - -/// Emit the statements returning a received value: `return `, -/// or the decoding statements followed by `return ret` for a buffer. -fn emit_return(w: &mut CodeWriter, ty: &Ty, pass: &RetPass, ptr: &str, len: &str, tail: &str) { - match receive_expr(ty, pass, ptr, len) { - Some(expr) => { - w.line(format!("return {expr}{tail}")); - } - None => { - emit_receive(w, ty, pass, "ret", ptr, len); - w.line(format!("return ret{tail}")); - } - } -} - -/// Emit a `Deprecated:` paragraph after a doc comment that may be empty. -fn emit_deprecated(w: &mut CodeWriter, has_doc: bool, deprecated: &Option) { - if let Some(msg) = deprecated { - if has_doc { - w.line("//"); - } - w.line(format!("// Deprecated: {msg}")); - } -} - -/// `true` when a value received per `pass` arrives as a `(ptr, len)` pair. -fn has_len(pass: &RetPass) -> bool { - matches!(pass, RetPass::String | RetPass::Bytes | RetPass::Buffer) -} - -// ── Async ── - -/// An async callable: a Go wrapper taking a leading `context.Context` that -/// launches the C call with a completion trampoline and blocks on a buffered -/// channel, plus the exported trampoline itself. The channel travels to the -/// producer as a `cgo.Handle` in the `context` slot; `wvComplete` resolves -/// and deletes it, so the completion is delivered exactly once. -/// -/// Every async wrapper returns an `error` (the context's, at least). A -/// cancellable function creates a native cancel token, cancels it when `ctx` -/// is done, and returns `ctx.Err()` for the cancelled (`-5`) completion; any -/// other function returns `ctx.Err()` as soon as `ctx` is done and abandons -/// the result. A producer error is mapped through the domain when the -/// function throws, and panics with the generic `Error` otherwise. -pub(crate) fn render_async_function( - out: &mut String, - prefix: &str, - f: &FnBinding, - ab: &AsyncBinding, - go_name: &str, - receiver: Option<&str>, - err: ErrCtx, -) { - let proto = ab.protocol(f, prefix); - let tramp = trampoline_name(&ab.callback_type); - let val_ty = f - .ret - .as_ref() - .map_or_else(|| "struct{}".to_string(), go_type); - - let mut w = CodeWriter::tabs(); - - // The exported completion trampoline. - let formals: Vec = ab - .callback_params - .iter() - .map(|s| format!("{} {}", go_slot_ident(&s.name), cgo_type(&s.ty, prefix))) - .collect(); - let handle = go_slot_ident(&ab.callback_params[0].name); - let err_slot = go_slot_ident(&ab.callback_params[1].name); - let results: Vec = ab.callback_params[2..] - .iter() - .map(|s| go_slot_ident(&s.name)) - .collect(); - w.line(format!("//export {tramp}")); - w.block( - format!("func {tramp}({}) {{", formals.join(", ")), - "}", - |w| { - w.block( - format!("wvComplete({handle}, {err_slot}, func() {val_ty} {{"), - "})", - |w| match &f.ret { - None => { - w.line("return struct{}{}"); - } - Some(ty) => { - let len = results.get(1).map_or("", String::as_str); - emit_return(w, ty, &proto.result, &results[0], len, ""); - } - }, - ); - }, - ); - w.blank(); - - // The blocking wrapper. - let ret_sig = match &f.ret { - Some(r) => format!(" ({}, error)", go_type(r)), - None => " error".into(), - }; - let zero_ret = |z: &str| match &f.ret { - Some(_) => format!("return {z}, "), - None => "return ".to_string(), - }; - let zero = f.ret.as_ref().map(go_zero).unwrap_or_default(); - let mut doc = String::new(); - emit_fn_doc(&mut doc, &f.doc, &f.params, "", go_name); - if doc.is_empty() { - doc = format!("// {go_name} calls the native async function.\n"); - } - w.raw(doc); - w.line("//"); - if f.cancellable { - w.line("// It blocks until the call completes. Cancelling ctx cancels the native"); - w.line("// call, which then returns ctx.Err()."); - } else { - w.line("// It blocks until the call completes or ctx is done; in the latter case it"); - w.line("// returns ctx.Err() at once and the native result is discarded."); - } - emit_deprecated(&mut w, true, &f.deprecated); - - w.block( - header(receiver, go_name, &go_params(f), &ret_sig), - "}", - |w| { - w.block("if err := ctx.Err(); err != nil {", "}", |w| { - w.line(format!("{}err", zero_ret(&zero))); - }); - let mut args: Vec = Vec::new(); - if receiver.is_some() { - emit_receiver(w, &mut args); - } - for p in &f.params { - emit_param(w, &mut args, p, prefix, true); - } - let token = if proto.cancellable { - w.line(format!("wvToken := C.{prefix}_cancel_token_create()")); - w.line(format!("defer C.{prefix}_cancel_token_destroy(wvToken)")); - args.push("wvToken".into()); - "wvToken" - } else { - "nil" - }; - w.line(format!("wvDone := make(chan wvOutcome[{val_ty}], 1)")); - w.line("wvHandle := cgo.NewHandle(wvDone)"); - args.push(format!("C.{}(unsafe.Pointer(C.{tramp}))", ab.callback_type)); - args.push("C.wvHandlePtr(C.uintptr_t(wvHandle))".into()); - w.line(format!("C.{}({})", ab.launch.symbol, args.join(", "))); - let res = if f.ret.is_some() { "res" } else { "_" }; - w.line(format!("{res}, fail, err := wvAwait(ctx, wvDone, {token})")); - w.block("if err != nil {", "}", |w| { - w.line(format!("{}err", zero_ret(&zero))); - }); - w.block("if fail != nil {", "}", |w| { - if err.throws { - w.line(format!("{}{}", zero_ret(&zero), err.map_call("*fail"))); - } else { - w.line("panic(fail.err())"); - } - }); - match &f.ret { - Some(_) => w.line("return res, nil"), - None => w.line("return nil"), - }; - }, - ); - w.blank(); - out.push_str(&w.finish()); -} - -// ── Functions ── - -/// A sync or iterator callable: the Go wrapper marshalling parameters in, -/// invoking the C symbol, checking the error slot per `err` (typed -/// `(T, error)` when throwing, `wvTrap` panic when plain), and converting the -/// result out. An iterator-returning callable renders through -/// [`render_iterator_fn`] as a lazy sequence instead. With `receiver` set, -/// the wrapper is a method on that wrapper type passing its native pointer -/// as the leading C argument. -pub(crate) fn render_function( - out: &mut String, - prefix: &str, - f: &FnBinding, - go_name: &str, - receiver: Option<&str>, - err: ErrCtx, -) { - if let CallShape::Iterator(ib) = &f.shape { - render_iterator_fn(out, prefix, f, ib, go_name, receiver, err); - return; - } - - let mut w = CodeWriter::tabs(); - let mut doc = String::new(); - emit_fn_doc(&mut doc, &f.doc, &f.params, "", go_name); - let has_doc = !doc.is_empty(); - w.raw(doc); - emit_deprecated(&mut w, has_doc, &f.deprecated); - - let ret_sig = err.ret_sig(f.ret.as_ref()); - let pass = plan::ret_pass(f.ret.as_ref(), prefix); - w.block( - header(receiver, go_name, &go_params(f), &ret_sig), - "}", - |w| { - let mut args: Vec = Vec::new(); - if receiver.is_some() { - emit_receiver(w, &mut args); - } - for p in &f.params { - emit_param(w, &mut args, p, prefix, false); - } - if has_len(&pass) { - w.line("var cRetLen C.size_t"); - args.push("&cRetLen".into()); - } - w.line(format!("var cErr C.{prefix}_error")); - args.push("&cErr".into()); - let call = format!("C.{}({})", f.c_base, args.join(", ")); - match &f.ret { - Some(_) => w.line(format!("cRet := {call}")), - None => w.line(call), - }; - err.emit_check(w, "cErr", f.ret.as_ref().map(go_zero).as_deref()); - match &f.ret { - Some(ty) => { - emit_return(w, ty, &pass, "cRet", "cRetLen", err.ok_tail()); - } - None if err.throws => { - w.line("return nil"); - } - None => {} - } - }, - ); - w.blank(); - out.push_str(&w.finish()); -} - -/// An `iter`-returning callable, rendered per the shared -/// [`weaveffi_model::plan::IteratorProtocol`] pull contract as Go's standard -/// lazy iteration idiom (the `iter` package): -/// -/// - A non-throwing function returns `iter.Seq[T]`. A launch or per-`next` -/// error can only be a producer bug, so it panics via `wvTrap`. -/// - A throwing function returns `iter.Seq2[T, error]`. A launch or -/// per-`next` domain error is yielded as the final `(zero, err)` pair and -/// iteration stops. -/// -/// The producer iterator is launched lazily inside the returned closure, one -/// C `next` call runs per consumer step, and the destroy runs exactly once -/// through a `defer`, whether the sequence is exhausted, stops on an error, -/// or is abandoned by an early `break`. A receiver stays borrowed until then. -fn render_iterator_fn( - out: &mut String, - prefix: &str, - f: &FnBinding, - ib: &IteratorBinding, - go_name: &str, - receiver: Option<&str>, - err: ErrCtx, -) { - let proto = ib.protocol(f, prefix); - let throws = matches!(proto.error, ErrorStrategy::Throws); - let elem = &ib.elem; - let elem_go = go_type(elem); - let CType::Ptr { pointee, .. } = &ib.next.params[1].ty else { - unreachable!("an iterator's out_item slot is always a pointer") - }; - let item_ty = cgo_type(&strip_const(pointee), prefix); - let with_len = has_len(&proto.elem); - let zero = go_zero(elem); - - let (seq_ty, yield_ty) = if throws { - ( - format!("iter.Seq2[{elem_go}, error]"), - format!("func({elem_go}, error) bool"), - ) - } else { - ( - format!("iter.Seq[{elem_go}]"), - format!("func({elem_go}) bool"), - ) - }; - - let mut w = CodeWriter::tabs(); - let mut doc = String::new(); - emit_fn_doc(&mut doc, &f.doc, &f.params, "", go_name); - if doc.is_empty() { - doc = format!("// {go_name} returns a lazy sequence over the native iterator.\n"); - } - w.raw(doc); - w.line("//"); - w.line("// The sequence launches the native iterator on each range and calls next once"); - w.line("// per element; the iterator is destroyed when the range ends, early or not."); - if throws { - w.line("// A failure is yielded as a final (zero value, error) pair."); - } - emit_deprecated(&mut w, true, &f.deprecated); - - // Statements surfacing a non-zero error slot: yield the mapped domain - // error and stop when throwing, trap when plain. - let emit_err_check = |w: &mut CodeWriter, slot: &str| { - if throws { - let map = err.map_call(&format!("wvTakeError(&{slot})")); - w.block(format!("if {slot}.code != 0 {{"), "}", |w| { - w.line(format!("yield({zero}, {map})")); - w.line("return"); - }); - } else { - w.line(format!("wvTrap(&{slot})")); - } - }; - - w.block( - header(receiver, go_name, &go_params(f), &format!(" {seq_ty}")), - "}", - |w| { - w.block(format!("return func(yield {yield_ty}) {{"), "}", |w| { - let mut args: Vec = Vec::new(); - if receiver.is_some() { - emit_receiver(w, &mut args); - } - for p in &f.params { - emit_param(w, &mut args, p, prefix, false); - } - w.line(format!("var cErr C.{prefix}_error")); - args.push("&cErr".into()); - w.line(format!( - "cIter := C.{}({})", - ib.launch.symbol, - args.join(", ") - )); - emit_err_check(w, "cErr"); - w.line(format!("defer C.{}(cIter)", ib.destroy_symbol)); - w.block("for {", "}", |w| { - w.line(format!("var cItem {item_ty}")); - let next_args = if with_len { - w.line("var cItemLen C.size_t"); - "cIter, &cItem, &cItemLen, &cIterErr" - } else { - "cIter, &cItem, &cIterErr" - }; - w.line(format!("var cIterErr C.{prefix}_error")); - w.line(format!("more := C.{}({next_args}) != 0", ib.next.symbol)); - emit_err_check(w, "cIterErr"); - w.block("if !more {", "}", |w| { - w.line("return"); - }); - emit_receive(w, elem, &proto.elem, "item", "cItem", "cItemLen"); - let yield_call = if throws { - "if !yield(item, nil) {" - } else { - "if !yield(item) {" - }; - w.block(yield_call, "}", |w| { - w.line("return"); - }); - }); - }); - }, - ); - w.blank(); - out.push_str(&w.finish()); -} diff --git a/crates/weaveffi-gen/src/targets/go/codec.rs b/crates/weaveffi-gen/src/targets/go/codec.rs deleted file mode 100644 index 7d6592e0..00000000 --- a/crates/weaveffi-gen/src/targets/go/codec.rs +++ /dev/null @@ -1,153 +0,0 @@ -//! Value-buffer codec emitters: the Go statements serializing and decoding -//! one value in the wire format, against the `wvWriter`/`wvReader` pair in -//! `runtime/codec.go`. -//! -//! Both emitters dispatch on the shared [`Ty::wire`] classification, so the -//! non-obvious folds (objects as `u64` tokens carrying one strong reference, -//! records and rich enums through one user codec) are decided centrally -//! rather than re-derived from `Ty` here. Every primitive maps onto the -//! runtime method named after it (`writeI32`, `readString`). - -use crate::codegen::CodeWriter; -use weaveffi_model::model::{Ty, WireType}; - -use crate::targets::go::types::{go_local, go_type, optional_derefs, token_fn, untoken_fn}; - -/// Emit statements appending `expr` (a Go value of type `ty`) to the -/// `wvWriter` named `writer`. `site` and `depth` uniquify the loop locals -/// generated for nested lists and maps. -/// -/// An object is written as a token carrying a *fresh* strong reference: the -/// per-interface `wvToken{Name}` helper calls the interface's `_clone` symbol -/// and the wrapper keeps its own reference. -pub(crate) fn emit_buffer_write( - w: &mut CodeWriter, - writer: &str, - expr: &str, - ty: &Ty, - site: &str, - depth: usize, -) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("{writer}.write{}({expr})", p.pascal())); - } - WireType::Object(n) => { - w.line(format!("{writer}.writeU64({}({expr}))", token_fn(n))); - } - WireType::Enum(_) => { - w.line(format!("{writer}.writeI32(int32({expr}))")); - } - WireType::User(n) => { - w.line(format!("wvPack{}({writer}, {expr})", go_local(n))); - } - WireType::Optional(inner) => { - w.line(format!("if {expr} == nil {{")); - w.indent(); - w.line(format!("{writer}.writeOptionFlag(false)")); - w.dedent(); - w.line("} else {"); - w.indent(); - w.line(format!("{writer}.writeOptionFlag(true)")); - let inner_expr = if optional_derefs(inner) { - format!("*{expr}") - } else { - expr.to_string() - }; - emit_buffer_write(w, writer, &inner_expr, inner, site, depth + 1); - w.dedent(); - w.line("}"); - } - WireType::List(inner) => { - let e = format!("e{site}{depth}"); - w.line(format!("{writer}.writeLen(len({expr}))")); - w.block(format!("for _, {e} := range {expr} {{"), "}", |w| { - emit_buffer_write(w, writer, &e, inner, site, depth + 1); - }); - } - WireType::Map(k, v) => { - let kv = format!("k{site}{depth}"); - let vv = format!("v{site}{depth}"); - w.line(format!("{writer}.writeLen(len({expr}))")); - w.block(format!("for {kv}, {vv} := range {expr} {{"), "}", |w| { - emit_buffer_write(w, writer, &kv, k, site, depth + 1); - emit_buffer_write(w, writer, &vv, v, site, depth + 1); - }); - } - } -} - -/// Emit statements decoding one value of type `ty` from the `wvReader` named -/// `reader` and assigning it into the pre-declared destination `dst`. -/// `site` and `depth` uniquify the locals generated for nested containers. -/// -/// An object token is adopted into a new wrapper by the per-interface -/// `wvUntoken{Name}` helper; the wrapper's `Close` (or finalizer) releases -/// the reference the token carried. Collections cap their preallocation by -/// the bytes left in the buffer, since a zero-sized element encoding makes -/// any count valid. -pub(crate) fn emit_buffer_read( - w: &mut CodeWriter, - reader: &str, - dst: &str, - ty: &Ty, - site: &str, - depth: usize, -) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("{dst} = {reader}.read{}()", p.pascal())); - } - WireType::Object(n) => { - w.line(format!("{dst} = {}({reader}.readU64())", untoken_fn(n))); - } - WireType::Enum(n) => { - w.line(format!("{dst} = {}({reader}.readI32())", go_local(n))); - } - WireType::User(n) => { - w.line(format!("{dst} = wvUnpack{}({reader})", go_local(n))); - } - WireType::Optional(inner) => { - let o = format!("o{site}{depth}"); - w.block(format!("if {reader}.readOptionFlag() {{"), "}", |w| { - w.line(format!("var {o} {}", go_type(inner))); - emit_buffer_read(w, reader, &o, inner, site, depth + 1); - if optional_derefs(inner) { - w.line(format!("{dst} = &{o}")); - } else { - w.line(format!("{dst} = {o}")); - } - }); - } - WireType::List(inner) => { - let n = format!("n{site}{depth}"); - let e = format!("e{site}{depth}"); - let gt = go_type(inner); - w.line(format!("{n} := {reader}.readLen()")); - w.line(format!("{dst} = make([]{gt}, 0, {reader}.capHint({n}))")); - w.block(format!("for range {n} {{"), "}", |w| { - w.line(format!("var {e} {gt}")); - emit_buffer_read(w, reader, &e, inner, site, depth + 1); - w.line(format!("{dst} = append({dst}, {e})")); - }); - } - WireType::Map(k, v) => { - let n = format!("n{site}{depth}"); - let kv = format!("k{site}{depth}"); - let vv = format!("v{site}{depth}"); - let gk = go_type(k); - let gv = go_type(v); - w.line(format!("{n} := {reader}.readLen()")); - w.line(format!( - "{dst} = make(map[{gk}]{gv}, {reader}.capHint({n}))" - )); - w.block(format!("for range {n} {{"), "}", |w| { - w.line(format!("var {kv} {gk}")); - emit_buffer_read(w, reader, &kv, k, site, depth + 1); - w.line(format!("var {vv} {gv}")); - emit_buffer_read(w, reader, &vv, v, site, depth + 1); - w.line(format!("{dst}[{kv}] = {vv}")); - }); - } - } -} diff --git a/crates/weaveffi-gen/src/targets/go/docs.rs b/crates/weaveffi-gen/src/targets/go/docs.rs deleted file mode 100644 index 00fb96cc..00000000 --- a/crates/weaveffi-gen/src/targets/go/docs.rs +++ /dev/null @@ -1,104 +0,0 @@ -//! Godoc comment emission: the symbol-prefixed doc convention and the -//! parameter continuation lines. - -use crate::codegen::common::{emit_doc as common_emit_doc, DocCommentStyle}; -use weaveffi_model::model::ParamBinding; - -/// Emits a Go `// ...` doc comment at `indent`. If `symbol` is provided, the -/// first non-empty line is prefixed with the symbol name to follow Go's doc -/// convention. Subsequent lines are emitted verbatim with `// `. -/// -/// Without a symbol, this delegates to the shared -/// [`crate::codegen::common::emit_doc`] helper using -/// [`DocCommentStyle::DoubleSlash`]. The symbol-prefix flavour stays -/// generator-local because godoc's first-line convention is unique to Go. -pub(crate) fn emit_doc(out: &mut String, doc: &Option, indent: &str, symbol: Option<&str>) { - let Some(symbol) = symbol else { - common_emit_doc(out, doc, indent, DocCommentStyle::DoubleSlash); - return; - }; - let Some(doc) = doc else { - return; - }; - let doc = doc.trim(); - if doc.is_empty() { - return; - } - let mut lines = doc.lines(); - if let Some(first) = lines.next() { - out.push_str(indent); - let lower = first - .chars() - .next() - .map(|c| c.is_lowercase()) - .unwrap_or(false); - if lower { - out.push_str(&format!("// {symbol} {}\n", first)); - } else { - out.push_str(&format!("// {symbol}: {}\n", first)); - } - } - for line in lines { - out.push_str(indent); - if line.is_empty() { - out.push_str("//\n"); - } else { - out.push_str("// "); - out.push_str(line); - out.push('\n'); - } - } -} - -/// Emits a Go function doc comment with continuation lines for any documented -/// parameters. Skips entirely when there is nothing to emit. -pub(crate) fn emit_fn_doc( - out: &mut String, - doc: &Option, - params: &[ParamBinding], - indent: &str, - symbol: &str, -) { - let trimmed_doc = doc.as_ref().map(|d| d.trim()).filter(|d| !d.is_empty()); - let documented_params: Vec<&ParamBinding> = params - .iter() - .filter(|p| { - p.doc - .as_ref() - .map(|d| !d.trim().is_empty()) - .unwrap_or(false) - }) - .collect(); - if trimmed_doc.is_none() && documented_params.is_empty() { - return; - } - if let Some(d) = trimmed_doc { - emit_doc(out, &Some(d.to_string()), indent, Some(symbol)); - } else { - out.push_str(indent); - out.push_str(&format!("// {symbol} ...\n")); - } - if !documented_params.is_empty() { - out.push_str(indent); - out.push_str("//\n"); - out.push_str(indent); - out.push_str("// Parameters:\n"); - for p in documented_params { - let pdoc = p.doc.as_ref().unwrap().trim(); - let mut lines = pdoc.lines(); - let first = lines.next().unwrap_or(""); - out.push_str(indent); - out.push_str(&format!("// - {}: {}\n", p.name, first)); - for line in lines { - out.push_str(indent); - if line.is_empty() { - out.push_str("//\n"); - } else { - out.push_str("// "); - out.push_str(line); - out.push('\n'); - } - } - } - } -} diff --git a/crates/weaveffi-gen/src/targets/go/entities.rs b/crates/weaveffi-gen/src/targets/go/entities.rs deleted file mode 100644 index 5ab9133e..00000000 --- a/crates/weaveffi-gen/src/targets/go/entities.rs +++ /dev/null @@ -1,578 +0,0 @@ -//! Entity rendering: plain and rich enums, records, typed error domains, and -//! interface (object) wrapper types. - -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use heck::ToUpperCamelCase; -use weaveffi_model::model::{ - CallShape, EnumBinding, ErrorBinding, InterfaceBinding, ModuleBinding, StructBinding, -}; - -use crate::targets::go::calls::{render_async_function, render_function, ErrCtx}; -use crate::targets::go::codec::{emit_buffer_read, emit_buffer_write}; -use crate::targets::go::docs::emit_doc; -use crate::targets::go::types::{adopt_fn, go_str, go_type, token_fn, untoken_fn}; - -/// One `name rest` line inside a struct or const block, with its optional -/// doc comment, for [`emit_aligned`]. -struct AlignedEntry { - doc: Option, - name: String, - rest: String, -} - -/// Emit `entries` as the body of a struct or const block the way gofmt lays -/// it out: gofmt aligns the second column of consecutive lines and starts a -/// fresh column after a comment line, so each run of entries following a -/// documented one (or the start) is padded to the widest name in that run. -fn emit_aligned(w: &mut CodeWriter, entries: &[AlignedEntry]) { - let docs: Vec = entries - .iter() - .map(|e| { - let mut d = String::new(); - emit_doc(&mut d, &e.doc, "\t", Some(&e.name)); - d - }) - .collect(); - let mut i = 0; - while i < entries.len() { - let mut j = i + 1; - while j < entries.len() && docs[j].is_empty() { - j += 1; - } - let width = entries[i..j] - .iter() - .map(|e| e.name.len()) - .max() - .unwrap_or(0); - for (e, d) in entries[i..j].iter().zip(&docs[i..j]) { - w.raw(d.clone()); - w.line(format!("{: Option { - module - .error - .as_ref() - .map(|e| e.owner_path.to_upper_camel_case()) -} - -/// Render one declaring module's typed error surface: a -/// `type {TypeName} struct` implementing `error` (so `errors.As` selects on -/// the domain), exported `int32` code constants in the plain-enum const style -/// (`{TypeName}{CodePascal}`), one payload struct per code that declares -/// fields, and the `wvMap{Stem}` helper converting a non-zero slot's -/// `(code, message, payload)` into the typed error (default message when the -/// slot carried none, decoded payload attached when the code declares fields, -/// generic `Error` fallback for unknown codes). -/// -/// Domain codes are validated positive-only, so the runtime's reserved -/// negative codes (generic error, producer panic, marshalling failure) can -/// never match a `case` arm here: they fall through to the generic -/// `Error` fallback rather than a typed domain case. -pub(crate) fn render_error(out: &mut String, module: &ModuleBinding, eb: &ErrorBinding) { - let stem = eb.owner_path.to_upper_camel_case(); - let ty = &eb.type_name; - let dotted = module.segments.join("."); - let has_payloads = eb.codes.iter().any(|c| !c.fields.is_empty()); - - let mut w = CodeWriter::tabs(); - w.line(format!( - "// {ty} is a typed error reported by the `{dotted}` module." - )); - w.block(format!("type {ty} struct {{"), "}", |w| { - w.line(format!( - "// Code is the numeric ABI error code (one of the {ty} constants)." - )); - w.line("Code int32"); - w.line("// Message is the human-readable error message."); - w.line("Message string"); - if has_payloads { - w.line("// Payload holds the matched code's structured fields when that code"); - w.line("// declares any (a pointer to the per-code payload struct), else nil."); - w.line("Payload any"); - } - }); - w.blank(); - w.block(format!("func (e *{ty}) Error() string {{"), "}", |w| { - w.line(format!( - "return fmt.Sprintf(\"{dotted}: %s (code %d)\", e.Message, e.Code)" - )); - }); - w.blank(); - - w.line(format!("// {ty} codes.")); - w.block("const (", ")", |w| { - let entries: Vec = eb - .codes - .iter() - .map(|c| AlignedEntry { - doc: Some(c.doc.clone().unwrap_or_else(|| c.message.clone())), - name: format!("{ty}{}", c.name.to_upper_camel_case()), - rest: format!("int32 = {}", c.value), - }) - .collect(); - emit_aligned(w, &entries); - }); - w.blank(); - - // One payload struct per code that declares structured fields. - for c in &eb.codes { - if c.fields.is_empty() { - continue; - } - let cname = format!("{ty}{}", c.name.to_upper_camel_case()); - let pname = format!("{cname}Payload"); - w.line(format!( - "// {pname} carries the structured fields of {cname}." - )); - w.block(format!("type {pname} struct {{"), "}", |w| { - let entries: Vec = c - .fields - .iter() - .map(|f| AlignedEntry { - doc: f.doc.clone(), - name: f.name.to_upper_camel_case(), - rest: go_type(&f.ty), - }) - .collect(); - emit_aligned(w, &entries); - }); - w.blank(); - } - - w.line(format!( - "// wvMap{stem} converts a non-zero code from the `{dotted}` domain into a" - )); - w.line(format!( - "// *{ty}, falling back to the generic *Error for unknown codes." - )); - w.block( - format!("func wvMap{stem}(f wvFailure) error {{"), - "}", - |w| { - w.line("switch f.code {"); - for c in &eb.codes { - let cname = format!("{ty}{}", c.name.to_upper_camel_case()); - w.line(format!("case {cname}:")); - w.indent(); - w.block("if f.message == \"\" {", "}", |w| { - w.line(format!("f.message = {}", go_str(&c.message))); - }); - if c.fields.is_empty() { - w.line(format!("return &{ty}{{Code: f.code, Message: f.message}}")); - } else { - let pname = format!("{cname}Payload"); - w.line(format!("e := &{ty}{{Code: f.code, Message: f.message}}")); - w.block("if f.payload != nil {", "}", |w| { - w.line("r := &wvReader{buf: f.payload}"); - w.line(format!("p := &{pname}{{}}")); - for f in &c.fields { - let fname = f.name.to_upper_camel_case(); - emit_buffer_read(w, "r", &format!("p.{fname}"), &f.ty, &fname, 0); - } - w.line("r.expectEnd()"); - w.line("e.Payload = p"); - }); - w.line("return e"); - } - w.dedent(); - } - w.line("default:"); - w.indent(); - w.line("return f.err()"); - w.dedent(); - w.line("}"); - }, - ); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render one plain C-style enum as an `int32` newtype with exported -/// constants. Rich (algebraic) enums are value sum types rendered by -/// [`render_rich_enum`]; each renderer skips the other kind. -pub(crate) fn render_enum(out: &mut String, e: &EnumBinding) { - if e.is_rich() { - return; - } - let name = e.name.to_upper_camel_case(); - let mut w = CodeWriter::tabs(); - let mut d = String::new(); - emit_doc(&mut d, &e.doc, "", Some(&name)); - w.raw(d); - w.line(format!("type {name} int32")); - w.blank(); - w.block("const (", ")", |w| { - let entries: Vec = e - .variants - .iter() - .map(|v| AlignedEntry { - doc: v.doc.clone(), - name: format!("{name}{}", v.name.to_upper_camel_case()), - rest: format!("{name} = {}", v.value), - }) - .collect(); - emit_aligned(w, &entries); - }); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render a rich (algebraic) enum as an idiomatic Go sum type: a sealed -/// interface (`type Shape interface { isShape() }`) with one struct per -/// variant (`ShapeCircle`, holding that variant's fields as exported struct -/// fields), plus the pack/unpack pair serializing the `i32` tag followed by -/// the active variant's fields in wire order. Rich enums have no C symbols; -/// values only cross the ABI inside value buffers. -/// -/// A plain C-style enum is skipped here (it is handled by [`render_enum`]). -pub(crate) fn render_rich_enum(out: &mut String, package: &str, e: &EnumBinding) { - if !e.is_rich() { - return; - } - let name = e.name.to_upper_camel_case(); - - let mut w = CodeWriter::tabs(); - if e.doc.is_some() { - let mut d = String::new(); - emit_doc(&mut d, &e.doc, "", Some(&name)); - w.raw(d); - w.line("//"); - } - w.line(format!( - "// {name} is a sealed sum type: exactly one of its variant structs is the" - )); - w.line("// value at a time."); - w.block(format!("type {name} interface {{"), "}", |w| { - w.line(format!("is{name}()")); - }); - w.blank(); - - for v in &e.variants { - let vn = format!("{name}{}", v.name.to_upper_camel_case()); - let mut vd = String::new(); - emit_doc(&mut vd, &v.doc, "", Some(&vn)); - if vd.is_empty() { - w.line(format!("// {vn} is the `{}` variant of {name}.", v.name)); - } else { - w.raw(vd); - } - if v.fields.is_empty() { - w.line(format!("type {vn} struct{{}}")); - } else { - w.block(format!("type {vn} struct {{"), "}", |w| { - let entries: Vec = v - .fields - .iter() - .map(|f| AlignedEntry { - doc: f.doc.clone(), - name: f.name.to_upper_camel_case(), - rest: go_type(&f.ty), - }) - .collect(); - emit_aligned(w, &entries); - }); - } - w.blank(); - w.line(format!("func ({vn}) is{name}() {{}}")); - w.blank(); - } - - w.line(format!( - "// wvPack{name} appends v to w in the value-buffer wire format." - )); - w.block( - format!("func wvPack{name}(w *wvWriter, v {name}) {{"), - "}", - |w| { - w.line("switch x := v.(type) {"); - for v in &e.variants { - let vn = format!("{name}{}", v.name.to_upper_camel_case()); - w.line(format!("case {vn}:")); - w.indent(); - w.line(format!("w.writeI32({})", v.value)); - for f in &v.fields { - let fname = f.name.to_upper_camel_case(); - let site = format!("{}{fname}", v.name.to_upper_camel_case()); - emit_buffer_write(w, "w", &format!("x.{fname}"), &f.ty, &site, 0); - } - w.dedent(); - } - w.line("default:"); - w.indent(); - w.line(format!( - "panic(\"{package}: {name} value is not one of its variants\")" - )); - w.dedent(); - w.line("}"); - }, - ); - w.blank(); - - w.line(format!("// wvUnpack{name} decodes one {name} from r.")); - w.block( - format!("func wvUnpack{name}(r *wvReader) {name} {{"), - "}", - |w| { - w.line("switch r.readI32() {"); - for v in &e.variants { - let vn = format!("{name}{}", v.name.to_upper_camel_case()); - w.line(format!("case {}:", v.value)); - w.indent(); - if v.fields.is_empty() { - w.line(format!("return {vn}{{}}")); - } else { - w.line(format!("var x {vn}")); - for f in &v.fields { - let fname = f.name.to_upper_camel_case(); - let site = format!("{}{fname}", v.name.to_upper_camel_case()); - emit_buffer_read(w, "r", &format!("x.{fname}"), &f.ty, &site, 0); - } - w.line("return x"); - } - w.dedent(); - } - w.line("default:"); - w.indent(); - w.line(format!( - "panic(\"{package}: malformed value buffer: {name} tag out of range\")" - )); - w.dedent(); - w.line("}"); - }, - ); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render one record as a plain Go value struct with exported, typed fields, -/// plus its pack/unpack pair serializing the fields in declaration (wire) -/// order. Records have no C symbols: no create, no destroy, no getters, no -/// builders; instances only cross the ABI inside value buffers. An interface -/// field holds a wrapper pointer and crosses as an object token (see -/// [`crate::targets::go::codec`]). -pub(crate) fn render_struct(out: &mut String, s: &StructBinding) { - let name = s.name.to_upper_camel_case(); - - let mut w = CodeWriter::tabs(); - let mut d = String::new(); - emit_doc(&mut d, &s.doc, "", Some(&name)); - w.raw(d); - w.block(format!("type {name} struct {{"), "}", |w| { - let entries: Vec = s - .fields - .iter() - .map(|f| AlignedEntry { - doc: f.doc.clone(), - name: f.name.to_upper_camel_case(), - rest: go_type(&f.ty), - }) - .collect(); - emit_aligned(w, &entries); - }); - w.blank(); - - w.line(format!( - "// wvPack{name} appends v to w in the value-buffer wire format." - )); - w.block( - format!("func wvPack{name}(w *wvWriter, v {name}) {{"), - "}", - |w| { - for f in &s.fields { - let fname = f.name.to_upper_camel_case(); - emit_buffer_write(w, "w", &format!("v.{fname}"), &f.ty, &fname, 0); - } - }, - ); - w.blank(); - - w.line(format!("// wvUnpack{name} decodes one {name} from r.")); - w.block( - format!("func wvUnpack{name}(r *wvReader) {name} {{"), - "}", - |w| { - w.line(format!("var v {name}")); - for f in &s.fields { - let fname = f.name.to_upper_camel_case(); - emit_buffer_read(w, "r", &format!("v.{fname}"), &f.ty, &fname, 0); - } - w.line("return v"); - }, - ); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render one interface as a reference-counted object wrapper: a struct -/// whose `wvRef` owns one strong reference, released by an explicit `Close` -/// (idempotent and safe to race with in-flight calls) or, as a backstop, by -/// a finalizer, so `destroy` runs exactly once per wrapper. -/// -/// Private helpers accompany the type: `native` borrows the pointer for one -/// call (panicking after `Close`); `wvAdopt{Name}` wraps an owned pointer -/// coming back from the producer (a return, an async result, an iterator -/// element, a callback-method argument) and returns nil for a null pointer, -/// so `Interface?` needs no separate path; `wvToken{Name}` clones the -/// wrapper's reference into a value-buffer object token; and -/// `wvUntoken{Name}` adopts the reference a token carries. -/// -/// Constructors become package-level factory functions named -/// `{PascalCtor}{Type}` (`new` gives `NewStore`, `open` gives `OpenStore`); -/// methods are methods on the wrapper passing its pointer as the leading C -/// argument; statics are package-level functions namespaced by the type -/// (`StoreDefaultCapacity`). Members reuse the free-function marshalling -/// paths, including the sync/async/iterator shapes and the throws split. -pub(crate) fn render_interface( - out: &mut String, - prefix: &str, - package: &str, - iface: &InterfaceBinding, - stem: Option<&str>, -) { - let name = local_type_name(&iface.name).to_upper_camel_case(); - let c_tag = &iface.c_tag; - let adopt = adopt_fn(&iface.name); - let token = token_fn(&iface.name); - let untoken = untoken_fn(&iface.name); - let destroy = format!("wvDestroy{name}"); - - let mut w = CodeWriter::tabs(); - let mut d = String::new(); - emit_doc(&mut d, &iface.doc, "", Some(&name)); - if d.is_empty() { - w.line(format!( - "// {name} is a reference-counted object owned by the native library." - )); - } else { - w.raw(d); - w.line("//"); - } - w.line("// Each wrapper holds one strong reference; Close releases it, and a"); - w.line("// finalizer releases it if the wrapper is garbage collected first."); - if let Some(msg) = &iface.deprecated { - w.line("//"); - w.line(format!("// Deprecated: {msg}")); - } - w.block(format!("type {name} struct {{"), "}", |w| { - w.line("ref wvRef"); - }); - w.blank(); - - w.block(format!("func {destroy}(ptr unsafe.Pointer) {{"), "}", |w| { - w.line(format!("C.{}((*C.{c_tag})(ptr))", iface.destroy_symbol)); - }); - w.blank(); - - w.line(format!( - "// {adopt} adopts one owned strong reference into a new wrapper. A null" - )); - w.line("// pointer adopts to nil."); - w.block( - format!("func {adopt}(ptr *C.{c_tag}) *{name} {{"), - "}", - |w| { - w.block("if ptr == nil {", "}", |w| { - w.line("return nil"); - }); - w.line(format!("s := &{name}{{}}")); - w.line(format!("s.ref.init(unsafe.Pointer(ptr), {destroy})")); - w.line(format!("runtime.SetFinalizer(s, (*{name}).Close)")); - w.line("return s"); - }, - ); - w.blank(); - - w.line("// native borrows the wrapper's pointer for one call; pair it with a"); - w.line("// deferred s.ref.release()."); - w.block( - format!("func (s *{name}) native() *C.{c_tag} {{"), - "}", - |w| { - w.block("if s == nil {", "}", |w| { - w.line(format!("panic(\"{package}: nil *{name}\")")); - }); - w.line(format!("return (*C.{c_tag})(s.ref.acquire(\"{name}\"))")); - }, - ); - w.blank(); - - w.line(format!( - "// {token} clones o's reference into a value-buffer object token. The" - )); - w.line("// wrapper keeps its own reference; the token carries the new one."); - w.block(format!("func {token}(o *{name}) uint64 {{"), "}", |w| { - w.line("ptr := o.native()"); - w.line("defer o.ref.release()"); - w.line(format!( - "return uint64(uintptr(unsafe.Pointer(C.{}(ptr))))", - iface.clone_symbol - )); - }); - w.blank(); - - w.line(format!( - "// {untoken} adopts the strong reference carried by a value-buffer object" - )); - w.line("// token."); - w.block( - format!("func {untoken}(token uint64) *{name} {{"), - "}", - |w| { - w.block("if token == 0 {", "}", |w| { - w.line(format!( - "panic(\"{package}: malformed value buffer: null object token\")" - )); - }); - w.line(format!( - "return {adopt}((*C.{c_tag})(C.wvHandlePtr(C.uintptr_t(token))))" - )); - }, - ); - w.blank(); - - w.line("// Close releases the wrapper's strong reference and always returns nil."); - w.line("// It's idempotent and safe to call from any goroutine, even while a call"); - w.line("// on the wrapper is in flight: the reference is then released when that"); - w.line("// call returns. The object itself is dropped once its last reference, from"); - w.line("// any wrapper, record, or the native side, is gone."); - w.block(format!("func (s *{name}) Close() error {{"), "}", |w| { - w.line("runtime.SetFinalizer(s, nil)"); - w.line("s.ref.close()"); - w.line("return nil"); - }); - w.blank(); - out.push_str(&w.finish()); - - for c in &iface.constructors { - let go_name = format!("{}{name}", c.name.to_upper_camel_case()); - render_function(out, prefix, c, &go_name, None, ErrCtx::of(c, stem)); - } - - for f in &iface.methods { - let go_name = f.name.to_upper_camel_case(); - let err = ErrCtx::of(f, stem); - if let CallShape::Async(ab) = &f.shape { - render_async_function(out, prefix, f, ab, &go_name, Some(&name), err); - } else { - render_function(out, prefix, f, &go_name, Some(&name), err); - } - } - - for f in &iface.statics { - let go_name = format!("{name}{}", f.name.to_upper_camel_case()); - let err = ErrCtx::of(f, stem); - if let CallShape::Async(ab) = &f.shape { - render_async_function(out, prefix, f, ab, &go_name, None, err); - } else { - render_function(out, prefix, f, &go_name, None, err); - } - } -} diff --git a/crates/weaveffi-gen/src/targets/go/mod.rs b/crates/weaveffi-gen/src/targets/go/mod.rs deleted file mode 100644 index 0a503500..00000000 --- a/crates/weaveffi-gen/src/targets/go/mod.rs +++ /dev/null @@ -1,884 +0,0 @@ -//! Go (cgo) binding generator for WeaveFFI. -//! -//! Emits a self-contained Go module under `go/`: `go.mod`, a copy of the C -//! header, `bindings.go` (the API), `runtime.go` (error, string, object, and -//! async plumbing), and, when the API has buffered values, `codec.go` (the -//! value-buffer writer and reader). The module path defaults to the package -//! name and the Go package name is the C prefix; the bindings link -//! `-l{library}`. Implements [`LanguageBackend`]; the shared driver bridges -//! it into the generator pipeline. -//! -//! Records, rich enums, optionals, lists, and maps are value types that -//! cross the C ABI serialized in the value-buffer format (one -//! `const uint8_t*` + `size_t` pair), with one pack and one unpack function -//! per record and rich enum. Strings and bytes cross as borrowed -//! `(ptr, len)` views of Go memory and come back as producer allocations the -//! bindings copy and release with `{prefix}_free_bytes`. -//! -//! Interfaces are reference-counted objects: each Go wrapper holds one strong -//! reference released by `Close` or, as a backstop, by a finalizer, and an -//! object inside a value buffer crosses as a cloned-reference token. -//! Callback interfaces are Go `interface` types the consumer implements; an -//! implementation crosses as a `cgo.Handle` plus the address of one static -//! vtable per interface, filled with exported Go trampolines that recover -//! panics into the producer's error slot. Async functions take a -//! `context.Context`, and a cancellable one cancels its native token when -//! the context is done. - -mod calls; -mod codec; -mod docs; -mod entities; -mod package; -mod runtime; -mod types; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::lang; -use crate::package::{PackageContext, PackagedFile}; -use crate::targets::c::render_c_header_from_model; -use crate::utils::{render_prelude, render_trailer, wrapper_name, CommentStyle}; -use camino::Utf8Path; -use heck::ToUpperCamelCase; -use serde::{Deserialize, Serialize}; -use std::collections::{BTreeMap, BTreeSet}; -use weaveffi_model::model::{ - checksum_symbol, BindingModel, CallShape, CallbackInterfaceBinding, EnumBinding, ErrorBinding, - FnBinding, InterfaceBinding, ModuleBinding, StructBinding, -}; -use weaveffi_model::resolved::ResolvedApi; - -use crate::targets::go::calls::{ - collect_preamble_decls, render_async_function, render_callback_interface, render_function, - ErrCtx, -}; -use crate::targets::go::entities::{ - domain_stem, render_enum, render_error, render_interface, render_rich_enum, render_struct, -}; -use crate::targets::go::package::{package_files, render_go_mod, render_readme}; -use crate::targets::go::runtime::{render_codec, render_runtime, RuntimeNames}; - -/// Per-target configuration for [`GoGenerator`]. -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct GoConfig { - /// Go module path written to `go.mod`. Defaults to the package name. - pub module_path: Option, - /// When `true` (the default), strip the IR module path from emitted - /// package-level function names, so module `kv`'s `delete` surfaces as - /// `Delete` rather than `KvDelete`. Set to `false` to restore the - /// module-prefixed spelling. Interface members are namespaced by their - /// wrapper type and never carry the module prefix. - pub strip_module_prefix: bool, - /// Basename of the IDL the CLI was invoked with. - #[serde(skip)] - pub input_basename: Option, - /// The identity-derived names the entity hooks render with, filled in by - /// [`render_bindings`] for the duration of one render. - #[serde(skip)] - render: RenderNames, -} - -/// What the [`LanguageBackend`] entity hooks need beyond the user's -/// configuration, carried on a per-render copy of [`GoConfig`]. -#[derive(Debug, Clone, Default)] -struct RenderNames { - /// The C symbol prefix. - prefix: String, - /// The Go package name. - package: String, - /// Base C symbols of the free functions that keep their module prefix - /// even when `strip_module_prefix` is on (see [`colliding_functions`]). - prefixed: BTreeSet, -} - -impl Default for GoConfig { - fn default() -> Self { - Self { - module_path: None, - strip_module_prefix: true, - input_basename: None, - render: RenderNames::default(), - } - } -} - -impl GoConfig { - /// The input IDL basename embedded in generated file headers, falling - /// back to `"api.yml"`. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } -} - -/// Every identity-derived name the generated module uses. -pub(crate) struct Names { - /// The Go module path (`go.mod`): the configured `module_path`, else the - /// package name. - pub(crate) module_path: String, - /// The Go package name: the C prefix, escaped if it's a Go keyword. - pub(crate) package: String, - /// The native library base name the bindings link (`-l{library}`). - pub(crate) library: String, - /// The bundled C header's file name, `{library}.h`. - pub(crate) header: String, -} - -impl Names { - pub(crate) fn new(api: &ResolvedApi, config: &GoConfig) -> Self { - let id = api.identity(); - Self { - module_path: config - .module_path - .as_deref() - .map(str::trim) - .filter(|p| !p.is_empty()) - .unwrap_or(&id.name) - .to_string(), - package: lang::escape_ident(&id.prefix, lang::GO_KEYWORDS), - library: id.library.clone(), - header: crate::targets::c::header_name(api), - } - } -} - -/// Go backend: emits a cgo module binding the C ABI exposed by the -/// underlying cdylib. -pub struct GoGenerator; - -impl LanguageBackend for GoGenerator { - type Config = GoConfig; - - fn name(&self) -> &'static str { - "go" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn render_enum(&self, out: &mut String, e: &EnumBinding, config: &Self::Config) { - // A plain C-style enum becomes an `int32` + constants; a rich - // (algebraic) enum becomes a sealed sum type. Each renderer skips - // the other kind. - render_enum(out, e); - render_rich_enum(out, &config.render.package, e); - } - - fn render_struct( - &self, - out: &mut String, - _module: &ModuleBinding, - s: &StructBinding, - _config: &Self::Config, - ) { - render_struct(out, s); - } - - fn render_error( - &self, - out: &mut String, - module: &ModuleBinding, - e: &ErrorBinding, - _config: &Self::Config, - ) { - // Emitted once, in the declaring module; inheriting submodules - // reference the ancestor's type through `wvMap{Stem}`. - render_error(out, module, e); - } - - fn render_callback_interface( - &self, - out: &mut String, - _module: &ModuleBinding, - cb: &CallbackInterfaceBinding, - config: &Self::Config, - ) { - render_callback_interface(out, &config.render.prefix, cb); - } - - fn render_interface( - &self, - out: &mut String, - module: &ModuleBinding, - i: &InterfaceBinding, - config: &Self::Config, - ) { - let stem = domain_stem(module); - let r = &config.render; - render_interface(out, &r.prefix, &r.package, i, stem.as_deref()); - } - - fn render_function( - &self, - out: &mut String, - module: &ModuleBinding, - f: &FnBinding, - config: &Self::Config, - ) { - let strip = config.strip_module_prefix && !config.render.prefixed.contains(&f.c_base); - let go_name = wrapper_name(&module.path, &f.name, strip).to_upper_camel_case(); - let stem = domain_stem(module); - let err = ErrCtx::of(f, stem.as_deref()); - let prefix = &config.render.prefix; - if let CallShape::Async(ab) = &f.shape { - render_async_function(out, prefix, f, ab, &go_name, None, err); - } else { - render_function(out, prefix, f, &go_name, None, err); - } - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let dir = out_dir.join("go"); - render_files(api, model, config) - .into_iter() - .map(|(name, contents)| OutputFile::new(dir.join(name), contents)) - .collect() - } - - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - Some(package_files(api, model, ctx, out_dir, config)) - } -} - -/// Every file of the generated module, as `(file name, contents)` pairs. -pub(crate) fn render_files( - api: &ResolvedApi, - model: &BindingModel, - config: &GoConfig, -) -> Vec<(String, String)> { - let input_basename = config.input_basename(); - let names = Names::new(api, config); - let runtime_names = RuntimeNames { - package: &names.package, - prefix: &model.prefix, - header: &names.header, - }; - let mut files = vec![ - ( - "go.mod".to_string(), - render_go_mod(&names.module_path, input_basename), - ), - ( - "README.md".to_string(), - render_readme(&names, input_basename), - ), - ( - names.header.clone(), - render_c_header_from_model(model, input_basename, &names.header), - ), - ( - "bindings.go".to_string(), - render_bindings(model, config, &names), - ), - ( - "runtime.go".to_string(), - render_runtime(&runtime_names, input_basename), - ), - ]; - if model.has_buffers() { - files.push(( - "codec.go".to_string(), - render_codec(&runtime_names, input_basename), - )); - } - files -} - -// ── Name collisions ── - -/// The free functions whose module-stripped Go name would clash in the one -/// flat package the module tree renders into: with a declared type, -/// constant, or interface factory or static, with the runtime's exported -/// names, or with another module's function of the same name. Those keep -/// their module prefix (`directory.card` beside a `Card` record surfaces as -/// `DirectoryCard`). -fn colliding_functions(model: &BindingModel) -> BTreeSet { - let mut taken: BTreeSet = ["Error", "DebugLive"].map(String::from).into(); - for m in &model.modules { - if let Some(e) = m.error.as_ref().filter(|e| e.declared_here) { - taken.insert(e.type_name.clone()); - for c in &e.codes { - let code = format!("{}{}", e.type_name, c.name.to_upper_camel_case()); - taken.insert(format!("{code}Payload")); - taken.insert(code); - } - } - for e in &m.enums { - let name = e.name.to_upper_camel_case(); - for v in &e.variants { - taken.insert(format!("{name}{}", v.name.to_upper_camel_case())); - } - taken.insert(name); - } - let types = m - .structs - .iter() - .map(|s| &s.name) - .chain(m.callback_interfaces.iter().map(|c| &c.name)); - taken.extend(types.map(|n| n.to_upper_camel_case())); - for i in &m.interfaces { - let name = i.name.to_upper_camel_case(); - for c in &i.constructors { - taken.insert(format!("{}{name}", c.name.to_upper_camel_case())); - } - for f in &i.statics { - taken.insert(format!("{name}{}", f.name.to_upper_camel_case())); - } - taken.insert(name); - } - } - let mut seen: BTreeMap = BTreeMap::new(); - for (_, f) in model.functions() { - *seen.entry(f.name.to_upper_camel_case()).or_default() += 1; - } - model - .functions() - .filter(|(_, f)| { - let name = f.name.to_upper_camel_case(); - taken.contains(&name) || seen[&name] > 1 - }) - .map(|(_, f)| f.c_base.clone()) - .collect() -} - -// ── Import scanning ── - -/// The standard-library packages `bindings.go` imports, computed by one pass -/// over the lowered model. Everything else lives in `runtime.go` and -/// `codec.go`, whose imports are fixed. -#[derive(Default, Clone, Copy)] -struct Imports { - /// `context` (async wrappers take a `context.Context`). - context: bool, - /// `fmt` (typed error domains format their message). - fmt: bool, - /// `iter` (lazy sequences returned by `iter` functions). - iter: bool, - /// `runtime` (`SetFinalizer` on object wrappers). - runtime: bool, - /// `runtime/cgo` (`cgo.Handle` for callback-interface implementations - /// and async completion channels). - cgo: bool, - /// `unsafe` (object pointers, callback contexts, completion trampolines). - unsafe_ptr: bool, - /// The `wvHandlePtr` preamble helper that widens a `cgo.Handle` or an - /// object token into the `void*` the ABI carries. Doing the integer to - /// pointer conversion in C keeps `go vet` from flagging the generated - /// file for a "possible misuse of unsafe.Pointer". - handle_ptr: bool, -} - -fn scan_imports(model: &BindingModel) -> Imports { - let has_async = model.has_async(); - let has_interfaces = model.has_interfaces(); - let has_callbacks = model.has_callback_interfaces(); - Imports { - context: has_async, - fmt: model.modules.iter().any(ModuleBinding::declares_error), - iter: model.has_iterators(), - runtime: has_interfaces, - cgo: has_callbacks || has_async, - unsafe_ptr: has_interfaces || has_callbacks || has_async, - handle_ptr: has_interfaces || has_callbacks || has_async, - } -} - -// ── Top-level rendering ── - -/// Render `bindings.go`: the cgo preamble, imports, the load-time ABI and -/// checksum checks, and every module's entities and wrappers in the canonical -/// member order (error domain, enums, structs, callback interfaces, -/// interfaces, functions). -pub(crate) fn render_bindings(model: &BindingModel, config: &GoConfig, names: &Names) -> String { - let prefix = model.prefix.as_str(); - let imports = scan_imports(model); - let mut out = render_prelude(CommentStyle::DoubleSlash, config.input_basename()); - - out.push_str(&format!("package {}\n\n", names.package)); - out.push_str("/*\n"); - out.push_str(&format!("#cgo LDFLAGS: -l{}\n", names.library)); - if model.callables().any(|(_, f)| f.deprecated.is_some()) { - // The bindings call deprecated functions on the user's behalf; only - // the user's own calls should warn, through Go's `Deprecated:` docs. - out.push_str("#cgo CFLAGS: -Wno-deprecated-declarations\n"); - } - out.push_str(&format!("#include \"{}\"\n", names.header)); - if imports.handle_ptr { - // Each preamble helper is compiled into every cgo translation unit, - // some of which never call it. - out.push_str( - "__attribute__((unused)) static void* wvHandlePtr(uintptr_t h) { return (void*)h; }\n", - ); - } - // Forward declarations for the //export trampolines below (mirroring the - // const-free prototypes cgo emits into _cgo_export.h) and the static - // vtable of each callback interface. - for decl in collect_preamble_decls(model) { - out.push_str(&decl); - out.push('\n'); - } - out.push_str("*/\n"); - out.push_str("import \"C\"\n"); - - let packages = [ - (imports.context, "context"), - (imports.fmt, "fmt"), - (imports.iter, "iter"), - (imports.runtime, "runtime"), - (imports.cgo, "runtime/cgo"), - (imports.unsafe_ptr, "unsafe"), - ]; - let used: Vec<&str> = packages - .iter() - .filter(|(on, _)| *on) - .map(|(_, p)| *p) - .collect(); - if !used.is_empty() { - out.push_str("\nimport (\n"); - for p in used { - out.push_str(&format!("\t\"{p}\"\n")); - } - out.push_str(")\n"); - } - out.push('\n'); - - // Load-time checks: the ABI revision, then each top-level module's - // contract checksum. - out.push_str("func init() {\n\twvCheckABI()\n"); - for root in model.roots() { - let checksum = root.checksum.expect("top-level modules carry a checksum"); - out.push_str(&format!( - "\twvCheckModule(\"{}\", {checksum:#018x}, uint64(C.{}()))\n", - root.name, - checksum_symbol(prefix, &root.name) - )); - } - out.push_str("}\n\n"); - - let mut config = config.clone(); - config.render = RenderNames { - prefix: prefix.to_string(), - package: names.package.clone(), - prefixed: colliding_functions(model), - }; - for m in &model.modules { - GoGenerator.emit_members(&mut out, m, &config); - } - - // Exactly one blank line before the trailer keeps the file gofmt-clean. - out.truncate(out.trim_end().len()); - out.push_str("\n\n"); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, "bindings.go")); - out -} - -#[cfg(test)] -mod tests { - use super::*; - use weaveffi_model::ir::{ - Api, CallbackInterfaceDef, Function, InterfaceDef, Module, Param, StructDef, StructField, - TypeRef, - }; - use weaveffi_model::pkg::Identity; - - fn param(name: &str, ty: TypeRef) -> Param { - Param { - name: name.into(), - ty, - doc: None, - } - } - - fn func(name: &str, params: Vec, returns: Option) -> Function { - Function { - name: name.into(), - params, - returns, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - } - } - - fn field(name: &str, ty: TypeRef) -> StructField { - StructField { - name: name.into(), - ty, - doc: None, - } - } - - fn named(n: &str) -> TypeRef { - TypeRef::Named(n.into()) - } - - fn optional(t: TypeRef) -> TypeRef { - TypeRef::Optional(Box::new(t)) - } - - /// One module exercising every shape: an interface with a constructor - /// and a method, `Interface?` in and out, a record with `Interface` and - /// `[Interface]` fields, an iterator of interfaces, a callback interface - /// taking a string, an i32, a record, and an object (one method returning - /// `bool`, one returning void), a function taking that callback - /// interface, and plain and cancellable async functions. - fn fixture() -> ResolvedApi { - let module = Module { - name: "bus".into(), - doc: None, - functions: vec![ - func( - "pick", - vec![param("preferred", optional(named("Ticker")))], - Some(optional(named("Ticker"))), - ), - func( - "all_tickers", - vec![], - Some(TypeRef::Iterator(Box::new(named("Ticker")))), - ), - func( - "subscribe", - vec![param("subscriber", named("Subscriber"))], - None, - ), - func( - "echo", - vec![param("text", TypeRef::StringUtf8)], - Some(TypeRef::StringUtf8), - ), - Function { - r#async: true, - ..func( - "fetch", - vec![param("id", TypeRef::I32)], - Some(named("Ticker")), - ) - }, - Function { - r#async: true, - cancellable: true, - ..func("wait", vec![param("ctx", TypeRef::I64)], None) - }, - ], - interfaces: vec![InterfaceDef { - name: "Ticker".into(), - doc: Some("A ticking counter.".into()), - deprecated: None, - constructors: vec![func("new", vec![param("start", TypeRef::I64)], None)], - methods: vec![func("value", vec![], Some(TypeRef::I64))], - statics: vec![], - }], - callback_interfaces: vec![CallbackInterfaceDef { - name: "Subscriber".into(), - doc: Some("Receives bus events.".into()), - deprecated: None, - methods: vec![ - func( - "on_message", - vec![ - param("text", TypeRef::StringUtf8), - param("weight", TypeRef::I32), - param("envelope", named("Envelope")), - ], - None, - ), - func( - "on_ticker", - vec![param("ticker", named("Ticker"))], - Some(TypeRef::Bool), - ), - ], - }], - structs: vec![StructDef { - name: "Envelope".into(), - doc: None, - deprecated: None, - fields: vec![ - field("topic", TypeRef::StringUtf8), - field("primary", named("Ticker")), - field("others", TypeRef::List(Box::new(named("Ticker")))), - ], - }], - enums: vec![], - errors: None, - modules: vec![], - }; - ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![module], - }) - .with_identity(Identity::named("acme-bus")) - } - - fn render_all(config: &GoConfig) -> Vec<(String, String)> { - let api = fixture(); - let model = BindingModel::build(&api); - render_files(&api, &model, config) - } - - fn render(config: &GoConfig) -> String { - render_all(config) - .into_iter() - .find(|(n, _)| n == "bindings.go") - .map(|(_, c)| c) - .expect("bindings.go") - } - - fn assert_has(src: &str, needle: &str) { - assert!(src.contains(needle), "missing `{needle}` in:\n{src}"); - } - - #[test] - fn identity_drives_every_name() { - let files = render_all(&GoConfig::default()); - let names: Vec<&str> = files.iter().map(|(n, _)| n.as_str()).collect(); - assert_eq!( - names, - [ - "go.mod", - "README.md", - "acme_bus.h", - "bindings.go", - "runtime.go", - "codec.go" - ] - ); - let get = |n: &str| &files.iter().find(|(f, _)| f == n).unwrap().1; - assert_has(get("go.mod"), "module acme-bus\n"); - let src = get("bindings.go"); - assert_has(src, "package acme_bus\n"); - assert_has(src, "#cgo LDFLAGS: -lacme_bus\n#include \"acme_bus.h\"\n"); - assert_has(src, "\twvCheckABI()\n\twvCheckModule(\"bus\", 0x"); - assert_has(src, "uint64(C.acme_bus_bus_checksum()))"); - let rt = get("runtime.go"); - assert_has(rt, "package acme_bus\n"); - assert_has(rt, "C.acme_bus_free_bytes(ptr, n)"); - assert_has(rt, "const wvABIVersion uint32 = 3\n"); - assert!(!rt.contains("{{"), "unsubstituted placeholder"); - for (name, contents) in &files { - if name.ends_with(".go") || name == "go.mod" { - let body: String = contents.lines().skip(3).collect::>().join("\n"); - assert!( - !body.to_lowercase().contains("weaveffi"), - "{name} is branded:\n{body}" - ); - } - } - - let custom = render_all(&GoConfig { - module_path: Some("example.com/bus".into()), - ..GoConfig::default() - }); - assert_has(&custom[0].1, "module example.com/bus\n"); - } - - #[test] - fn preamble_declares_trampolines_and_one_static_vtable() { - let src = render(&GoConfig::default()); - assert_has( - &src, - "extern void goWv_acme_bus_bus_Subscriber_on_message(void* ctx, uint8_t* text_ptr, size_t text_len, int32_t weight, uint8_t* envelope_ptr, size_t envelope_len, acme_bus_error* out_err);", - ); - assert_has( - &src, - "extern bool goWv_acme_bus_bus_Subscriber_on_ticker(void* ctx, acme_bus_bus_Ticker* ticker, acme_bus_error* out_err);", - ); - assert_has( - &src, - "static const acme_bus_bus_Subscriber_vtable wvVtable_acme_bus_bus_Subscriber_vtable = {", - ); - assert_has( - &src, - "(void (*)(void*, const uint8_t*, size_t, int32_t, const uint8_t*, size_t, acme_bus_error*))goWv_acme_bus_bus_Subscriber_on_message,", - ); - assert_has(&src, " goWv_acme_bus_bus_Subscriber_on_ticker,"); - assert_has(&src, " goWv_acme_bus_bus_Subscriber_free,\n};"); - assert_has( - &src, - "extern void goWv_acme_bus_bus_fetch_callback(void* context_, acme_bus_error* err, acme_bus_bus_Ticker* result);", - ); - } - - #[test] - fn strings_cross_as_borrowed_views_and_owned_runs() { - let src = render(&GoConfig::default()); - assert_has(&src, "cTextPtr, cTextLen := wvStr(text)"); - assert_has(&src, "var cRetLen C.size_t"); - assert_has( - &src, - "cRet := C.acme_bus_bus_echo(cTextPtr, cTextLen, &cRetLen, &cErr)", - ); - assert_has(&src, "return wvTakeString(cRet, cRetLen)"); - assert!(!src.contains("CString"), "no NUL-terminated strings"); - assert_has(&src, "arg0 := wvBorrowString(text_ptr, text_len)"); - } - - #[test] - fn callback_interface_renders_go_interface_and_trampolines() { - let src = render(&GoConfig::default()); - assert_has(&src, "type Subscriber interface {"); - assert_has( - &src, - "\tOnMessage(text string, weight int32, envelope Envelope)\n", - ); - assert_has(&src, "\tOnTicker(ticker *Ticker) bool\n"); - assert_has(&src, "wvForeignError(out_err, r)"); - assert_has(&src, "arg1 := int32(weight)"); - assert_has( - &src, - "rArg2 := &wvReader{buf: wvBorrowBytes(envelope_ptr, envelope_len)}", - ); - assert_has(&src, "arg0 := wvAdoptTicker(ticker)"); - assert_has(&src, "ret = C._Bool(impl.OnTicker(arg0))"); - assert_has(&src, "hSubscriber := cgo.NewHandle(subscriber)"); - assert_has( - &src, - "C.acme_bus_bus_subscribe(C.wvHandlePtr(C.uintptr_t(hSubscriber)), C.wvVtablePtr_acme_bus_bus_Subscriber_vtable(), &cErr)", - ); - } - - #[test] - fn interface_wrapper_guards_its_reference() { - let src = render(&GoConfig::default()); - assert_has(&src, "type Ticker struct {\n\tref wvRef\n}"); - assert_has(&src, "s.ref.init(unsafe.Pointer(ptr), wvDestroyTicker)"); - assert_has( - &src, - "func (s *Ticker) Close() error {\n\truntime.SetFinalizer(s, nil)\n\ts.ref.close()\n\treturn nil\n}", - ); - assert_has( - &src, - "func (s *Ticker) Value() int64 {\n\tcSelf := s.native()\n\tdefer s.ref.release()\n", - ); - assert_has(&src, "panic(\"acme_bus: nil *Ticker\")"); - assert_has( - &src, - "var cPreferred *C.acme_bus_bus_Ticker\n\tif preferred != nil {\n\t\tcPreferred = preferred.native()\n\t\tdefer preferred.ref.release()\n\t}", - ); - assert_has(&src, "v.Primary = wvUntokenTicker(r.readU64())"); - } - - #[test] - fn async_functions_take_a_context_and_cancel() { - let src = render(&GoConfig::default()); - assert_has( - &src, - "func Fetch(ctx context.Context, id int32) (*Ticker, error) {", - ); - assert_has(&src, "res, fail, err := wvAwait(ctx, wvDone, nil)"); - assert_has( - &src, - "wvComplete(context_, err, func() *Ticker {\n\t\treturn wvAdoptTicker(result)", - ); - assert_has(&src, "func Wait(ctx context.Context, ctx_ int64) error {"); - assert_has(&src, "wvToken := C.acme_bus_cancel_token_create()"); - assert_has(&src, "defer C.acme_bus_cancel_token_destroy(wvToken)"); - assert_has( - &src, - "C.acme_bus_bus_wait(C.int64_t(ctx_), wvToken, C.acme_bus_bus_wait_callback(unsafe.Pointer(C.goWv_acme_bus_bus_wait_callback)), C.wvHandlePtr(C.uintptr_t(wvHandle)))", - ); - assert_has(&src, "_, fail, err := wvAwait(ctx, wvDone, wvToken)"); - } - - #[test] - fn iterators_are_lazy_sequences() { - let src = render(&GoConfig::default()); - assert_has(&src, "func AllTickers() iter.Seq[*Ticker] {"); - assert_has(&src, "var cItem *C.acme_bus_bus_Ticker"); - assert_has( - &src, - "more := C.acme_bus_bus_AllTickersIterator_next(cIter, &cItem, &cIterErr) != 0", - ); - assert_has(&src, "item := wvAdoptTicker(cItem)"); - assert_has( - &src, - "defer C.acme_bus_bus_AllTickersIterator_destroy(cIter)", - ); - } - - #[test] - fn undocumented_enum_members_are_gofmt_aligned() { - use weaveffi_model::ir::{EnumDef, EnumVariant}; - let variant = |name: &str, value: i32, fields: Vec| EnumVariant { - name: name.into(), - value, - doc: None, - fields, - }; - let module = Module { - name: "paint".into(), - doc: None, - functions: vec![], - interfaces: vec![], - callback_interfaces: vec![], - structs: vec![], - enums: vec![ - EnumDef { - name: "Channel".into(), - doc: None, - deprecated: None, - variants: vec![ - variant("red", 0, vec![]), - variant("green", 1, vec![]), - variant("blue", 2, vec![]), - ], - }, - EnumDef { - name: "Shape".into(), - doc: None, - deprecated: None, - variants: vec![variant( - "rectangle", - 0, - vec![field("width", TypeRef::F32), field("height", TypeRef::F32)], - )], - }, - ], - errors: None, - modules: vec![], - }; - let api = ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![module], - }); - let config = GoConfig::default(); - let model = BindingModel::build(&api); - let src = render_bindings(&model, &config, &Names::new(&api, &config)); - // gofmt pads a run of undocumented const and field names to one - // column; emitting it pre-aligned keeps the file gofmt-clean. - assert_has( - &src, - "const (\n\tChannelRed Channel = 0\n\tChannelGreen Channel = 1\n\tChannelBlue Channel = 2\n)", - ); - assert_has( - &src, - "type ShapeRectangle struct {\n\tWidth float32\n\tHeight float32\n}", - ); - } - - #[test] - fn output_is_deterministic() { - assert_eq!( - render_all(&GoConfig::default()), - render_all(&GoConfig::default()) - ); - } -} diff --git a/crates/weaveffi-gen/src/targets/go/runtime/runtime.go b/crates/weaveffi-gen/src/targets/go/runtime/runtime.go deleted file mode 100644 index 7795f40c..00000000 --- a/crates/weaveffi-gen/src/targets/go/runtime/runtime.go +++ /dev/null @@ -1,296 +0,0 @@ -package {{PACKAGE}} - -/* -#include -#include "{{HEADER}}" -*/ -import "C" - -import ( - "context" - "fmt" - "runtime/cgo" - "sync/atomic" - "unsafe" -) - -// wvABIVersion is the C ABI revision these bindings were generated against. -const wvABIVersion uint32 = {{ABI_VERSION}} - -// The runtime error codes this package produces or interprets itself. -const ( - wvCodeMarshal int32 = -3 - wvCodeForeign int32 = -4 - wvCodeCancelled int32 = -5 -) - -// Error reports a failure that no typed error domain claims: an unknown -// code, a generic failure (code -1), a producer panic (-2), a marshalling -// failure (-3), or a Go callback-interface implementation that panicked -// (-4). The code is preserved so callers can tell them apart. -type Error struct { - // Code is the numeric ABI error code. - Code int32 - // Message is the human-readable error message. - Message string -} - -func (e *Error) Error() string { - return fmt.Sprintf("{{PACKAGE}}: %s (code %d)", e.Message, e.Code) -} - -// DebugLive reports how many resources of one kind the native library holds: -// 0 objects, 1 callbacks, 2 iterators, 3 cancel tokens, 4 returned -// allocations. It's always 0 unless the library was built with leak checks -// enabled; tests use it to assert that nothing leaked. -func DebugLive(kind int32) uint64 { - return uint64(C.{{PREFIX}}_debug_live(C.int32_t(kind))) -} - -// wvCheckABI panics when the linked library implements a different C ABI -// revision than these bindings were generated for. -func wvCheckABI() { - if found := uint32(C.{{PREFIX}}_abi_version()); found != wvABIVersion { - panic(fmt.Sprintf("{{PACKAGE}}: ABI mismatch: these bindings expect revision %d but the linked library reports revision %d", wvABIVersion, found)) - } -} - -// wvCheckModule panics when the linked library's contract checksum for a -// top-level module differs from the one these bindings were generated -// against, which means the bindings are stale. -func wvCheckModule(module string, want, found uint64) { - if found != want { - panic(fmt.Sprintf("{{PACKAGE}}: module %q doesn't match the linked library (bindings checksum %#016x, library checksum %#016x); regenerate the bindings", module, want, found)) - } -} - -// ── Errors ── - -// wvFailure is a non-zero error slot copied into Go memory. -type wvFailure struct { - code int32 - message string - payload []byte -} - -// err converts the failure into the generic *Error. -func (f wvFailure) err() error { - return &Error{Code: f.code, Message: f.message} -} - -func wvCopyFailure(cErr *C.{{PREFIX}}_error) wvFailure { - f := wvFailure{code: int32(cErr.code)} - if cErr.message != nil { - f.message = C.GoString(cErr.message) - } - if cErr.payload_ptr != nil { - f.payload = wvBorrowBytes(cErr.payload_ptr, cErr.payload_len) - } - return f -} - -// wvTakeError copies a non-zero error slot and clears it. -func wvTakeError(cErr *C.{{PREFIX}}_error) wvFailure { - f := wvCopyFailure(cErr) - C.{{PREFIX}}_error_clear(cErr) - return f -} - -// wvTakeBoxedError copies the heap-boxed error an async completion receives -// and frees the box. -func wvTakeBoxedError(cErr *C.{{PREFIX}}_error) wvFailure { - f := wvCopyFailure(cErr) - C.{{PREFIX}}_error_free(cErr) - return f -} - -// wvTrap panics with the generic *Error when the slot reports a failure. -// Wrappers of functions that declare no error domain check their slot with -// it: a failure there can only be a producer bug, a marshalling failure, or -// a callback-interface implementation that panicked. -func wvTrap(cErr *C.{{PREFIX}}_error) { - if cErr.code != 0 { - panic(wvTakeError(cErr).err()) - } -} - -// wvForeignError reports a panic recovered inside a callback-interface -// trampoline through the producer's error slot (code -4). The producer -// copies the message, so it's only borrowed for the call. -func wvForeignError(outErr *C.{{PREFIX}}_error, recovered any) { - msg := C.CString(fmt.Sprint(recovered)) - defer C.free(unsafe.Pointer(msg)) - C.{{PREFIX}}_error_set(outErr, C.int32_t(wvCodeForeign), msg) -} - -// ── Strings and bytes ── - -// wvStr returns a borrowed (pointer, length) view of s's UTF-8 bytes for the -// duration of one call. Strings may contain NUL bytes. -func wvStr(s string) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.StringData(s))), C.size_t(len(s)) -} - -// wvBytes returns a borrowed (pointer, length) view of b for the duration of -// one call. -func wvBytes(b []byte) (*C.uint8_t, C.size_t) { - return (*C.uint8_t)(unsafe.Pointer(unsafe.SliceData(b))), C.size_t(len(b)) -} - -// wvBorrowString copies a borrowed UTF-8 run into a Go string. -func wvBorrowString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - return string(unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)) -} - -// wvTakeString copies a returned UTF-8 run into a Go string and releases the -// producer's allocation. -func wvTakeString(ptr *C.uint8_t, n C.size_t) string { - if ptr == nil { - return "" - } - s := wvBorrowString(ptr, n) - C.{{PREFIX}}_free_bytes(ptr, n) - return s -} - -// wvBorrowBytes copies a borrowed byte run into Go memory. A null pointer -// copies to nil. -func wvBorrowBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - return append([]byte{}, unsafe.Slice((*byte)(unsafe.Pointer(ptr)), n)...) -} - -// wvTakeBytes copies a returned byte run (raw bytes or a value buffer) into -// Go memory and releases the producer's allocation. -func wvTakeBytes(ptr *C.uint8_t, n C.size_t) []byte { - if ptr == nil { - return nil - } - b := wvBorrowBytes(ptr, n) - C.{{PREFIX}}_free_bytes(ptr, n) - return b -} - -// ── Objects ── - -// wvRef owns one strong reference to a native object on behalf of a wrapper. -// Calls borrow the pointer between acquire and release. Close may race them: -// the reference is destroyed exactly once, by close when no call is in -// flight, or else by the last in-flight call to finish. -type wvRef struct { - ptr unsafe.Pointer - destroy func(unsafe.Pointer) - // state counts in-flight calls in its upper bits; the low bit is set - // once close has run. - state atomic.Uint64 -} - -func (r *wvRef) init(ptr unsafe.Pointer, destroy func(unsafe.Pointer)) { - r.ptr = ptr - r.destroy = destroy -} - -// acquire borrows the pointer for one call, panicking when the wrapper was -// already closed. -func (r *wvRef) acquire(typeName string) unsafe.Pointer { - for { - s := r.state.Load() - if s&1 != 0 { - panic("{{PACKAGE}}: " + typeName + " used after Close") - } - if r.state.CompareAndSwap(s, s+2) { - return r.ptr - } - } -} - -// release ends one call's borrow. -func (r *wvRef) release() { - if r.state.Add(^uint64(1)) == 1 { - r.destroy(r.ptr) - } -} - -// close releases the wrapper's reference once no call is in flight. Calling -// it again is a no-op. -func (r *wvRef) close() { - for { - s := r.state.Load() - if s&1 != 0 { - return - } - if r.state.CompareAndSwap(s, s|1) { - if s == 0 { - r.destroy(r.ptr) - } - return - } - } -} - -// ── Async ── - -// wvOutcome is one async completion: a converted result, or the failure the -// producer reported. -type wvOutcome[T any] struct { - val T - fail *wvFailure -} - -// wvComplete delivers one async completion to the waiting wrapper. It runs on -// a producer thread inside an exported trampoline, so it never panics: a -// result that fails to decode is delivered as a marshalling failure. -func wvComplete[T any](handle unsafe.Pointer, cErr *C.{{PREFIX}}_error, convert func() T) { - h := cgo.Handle(uintptr(handle)) - done := h.Value().(chan wvOutcome[T]) - h.Delete() - done <- wvSettle(cErr, convert) -} - -func wvSettle[T any](cErr *C.{{PREFIX}}_error, convert func() T) (o wvOutcome[T]) { - if cErr != nil { - if f := wvTakeBoxedError(cErr); f.code != 0 { - o.fail = &f - return o - } - } - defer func() { - if r := recover(); r != nil { - o = wvOutcome[T]{fail: &wvFailure{code: wvCodeMarshal, message: fmt.Sprint(r)}} - } - }() - o.val = convert() - return o -} - -// wvAwait blocks until an async call completes or ctx is done. With a cancel -// token, ctx's cancellation cancels the native call and waits for its -// completion, which the producer delivers promptly; without one, the call is -// abandoned and ctx.Err() returned at once. A cancelled completion (code -5) -// returns ctx.Err(), or context.Canceled when ctx itself isn't done. -func wvAwait[T any](ctx context.Context, done chan wvOutcome[T], token *C.{{PREFIX}}_cancel_token) (T, *wvFailure, error) { - var o wvOutcome[T] - select { - case o = <-done: - case <-ctx.Done(): - if token == nil { - var zero T - return zero, nil, ctx.Err() - } - C.{{PREFIX}}_cancel_token_cancel(token) - o = <-done - } - if o.fail != nil && o.fail.code == wvCodeCancelled { - var zero T - if err := ctx.Err(); err != nil { - return zero, nil, err - } - return zero, nil, context.Canceled - } - return o.val, o.fail, nil -} diff --git a/crates/weaveffi-gen/src/targets/go/types.rs b/crates/weaveffi-gen/src/targets/go/types.rs deleted file mode 100644 index f7480c03..00000000 --- a/crates/weaveffi-gen/src/targets/go/types.rs +++ /dev/null @@ -1,213 +0,0 @@ -//! Go type mapping and naming: how IR types, zero values, scalar -//! conversions, and user-chosen identifiers are spelled in the generated -//! package. - -use crate::lang; -use crate::utils::local_type_name; -use heck::{ToLowerCamelCase, ToUpperCamelCase}; -use weaveffi_model::abi::{CType, ConstPos}; -use weaveffi_model::model::Ty; - -/// The local Go type name (PascalCase) of a user-defined type reference, -/// stripping its module path. -pub(crate) fn go_local(n: &str) -> String { - local_type_name(n).to_upper_camel_case() -} - -/// The Go spelling of a user-chosen parameter name: lowerCamelCase, with a -/// trailing `_` appended when the conversion lands on a Go keyword (a param -/// named `type` surfaces as `type_`). In an async wrapper, a parameter named -/// `ctx` is escaped the same way so it can't shadow the leading -/// `context.Context`. -/// -/// Only parameter positions need escaping: every other user-chosen name is -/// emitted in PascalCase, and Go keywords are all lowercase. -pub(crate) fn go_param_ident(name: &str, is_async: bool) -> String { - let ident = lang::escape_ident(&name.to_lower_camel_case(), lang::GO_KEYWORDS); - if is_async && ident == "ctx" { - "ctx_".into() - } else { - ident - } -} - -/// The Go type spelling of an IR type reference. -pub(crate) fn go_type(ty: &Ty) -> String { - match ty { - Ty::I8 => "int8".into(), - Ty::I16 => "int16".into(), - Ty::I32 => "int32".into(), - Ty::U8 => "uint8".into(), - Ty::U16 => "uint16".into(), - Ty::U32 => "uint32".into(), - Ty::U64 => "uint64".into(), - Ty::I64 => "int64".into(), - Ty::F32 => "float32".into(), - Ty::F64 => "float64".into(), - Ty::Bool => "bool".into(), - Ty::StringUtf8 => "string".into(), - Ty::Bytes => "[]byte".into(), - // Records are plain value structs; rich enums are sealed interfaces - // (nil-able), so neither takes a pointer at the type site. - Ty::Record(n) | Ty::RichEnum(n) | Ty::Enum(n) | Ty::CallbackInterface(n) => go_local(n), - // An object wrapper is always handled through a pointer so one Go - // value owns the strong reference. - Ty::Interface(n) => format!("*{}", go_local(n)), - Ty::Optional(inner) => { - if optional_derefs(inner) { - format!("*{}", go_type(inner)) - } else { - // Already nil-able in Go (interface, slice, map, byte slice, - // object wrapper pointer): nil is the none marker. - go_type(inner) - } - } - Ty::List(inner) => format!("[]{}", go_type(inner)), - // The bare (non-throwing) sequence type; a throwing iterator wrapper - // spells `iter.Seq2[T, error]` at its signature site instead. - Ty::Iterator(inner) => format!("iter.Seq[{}]", go_type(inner)), - Ty::Map(k, v) => format!("map[{}]{}", go_type(k), go_type(v)), - } -} - -/// `true` when `T?` surfaces as `*T` in Go (the value must be dereferenced -/// when present). Types that are already nil-able (rich enums, slices, maps, -/// byte slices, object wrapper pointers) use nil directly as the none marker -/// instead. -pub(crate) fn optional_derefs(inner: &Ty) -> bool { - !matches!( - inner, - Ty::RichEnum(_) | Ty::List(_) | Ty::Map(_, _) | Ty::Bytes | Ty::Interface(_) - ) -} - -/// The Go zero-value expression of a type, returned on an error path. -pub(crate) fn go_zero(ty: &Ty) -> String { - match ty { - Ty::I8 - | Ty::I16 - | Ty::I32 - | Ty::I64 - | Ty::U8 - | Ty::U16 - | Ty::U32 - | Ty::U64 - | Ty::F32 - | Ty::F64 - | Ty::Enum(_) => "0".into(), - Ty::Bool => "false".into(), - Ty::StringUtf8 => "\"\"".into(), - // A record is a value struct: its zero is the empty literal. - Ty::Record(n) => format!("{}{{}}", go_local(n)), - _ => "nil".into(), - } -} - -/// The Go expression converting the Go value `expr` into the by-value C slot -/// of type `slot` (a scalar, `bool`, or C-style enum). -pub(crate) fn to_c_direct(expr: &str, slot: &CType, prefix: &str) -> String { - format!("{}({expr})", cgo_type(slot, prefix)) -} - -/// The Go expression converting a by-value C slot `expr` back into the Go -/// value of type `ty`. -pub(crate) fn from_c_direct(expr: &str, ty: &Ty) -> String { - format!("{}({expr})", go_type(ty)) -} - -/// The name of the per-interface adopt helper (`wvAdoptStore`) that wraps one -/// owned strong reference in a new wrapper (or returns nil for a null -/// pointer). -pub(crate) fn adopt_fn(n: &str) -> String { - format!("wvAdopt{}", go_local(n)) -} - -/// The name of the per-interface token writer (`wvTokenStore`) that clones a -/// wrapper's reference into a value-buffer object token. -pub(crate) fn token_fn(n: &str) -> String { - format!("wvToken{}", go_local(n)) -} - -/// The name of the per-interface token reader (`wvUntokenStore`) that adopts -/// the reference carried by a value-buffer object token. -pub(crate) fn untoken_fn(n: &str) -> String { - format!("wvUntoken{}", go_local(n)) -} - -/// The Go expression adopting the owned object pointer `ptr_expr` into a -/// wrapper for the interface named by `ty` (a bare or optional interface). -/// A null pointer adopts to nil, so the same expression serves `Interface` -/// and `Interface?`. -pub(crate) fn go_adopt_expr(ty: &Ty, ptr_expr: &str) -> String { - let n = ty - .interface_name() - .expect("only interfaces and optional interfaces adopt C pointers"); - format!("{}({ptr_expr})", adopt_fn(n)) -} - -/// The C identifier of the process-wide static vtable emitted in the cgo -/// preamble for the vtable struct `vtable_tag`. -pub(crate) fn vtable_var(vtable_tag: &str) -> String { - format!("wvVtable_{vtable_tag}") -} - -/// The C identifier of the static preamble function returning the address -/// of [`vtable_var`]'s table; Go can't take a `static` variable's address -/// through cgo directly, so wrappers call this instead. -pub(crate) fn vtable_accessor(vtable_tag: &str) -> String { - format!("wvVtablePtr_{vtable_tag}") -} - -/// Quote `s` as a Go string literal, escaping backslashes, quotes, and -/// newlines. -pub(crate) fn go_str(s: &str) -> String { - let mut out = String::with_capacity(s.len() + 2); - out.push('"'); - for ch in s.chars() { - match ch { - '\\' => out.push_str("\\\\"), - '"' => out.push_str("\\\""), - '\n' => out.push_str("\\n"), - '\t' => out.push_str("\\t"), - _ => out.push(ch), - } - } - out.push('"'); - out -} - -/// The cgo spelling of one C ABI type: `C.int32_t`, `*C.uint8_t`, -/// `*C.kv_kv_Store`, and `unsafe.Pointer` for `void*`. -pub(crate) fn cgo_type(ct: &CType, prefix: &str) -> String { - match ct { - CType::Int8 => "C.int8_t".into(), - CType::Int16 => "C.int16_t".into(), - CType::Int32 => "C.int32_t".into(), - CType::Uint8 => "C.uint8_t".into(), - CType::Uint16 => "C.uint16_t".into(), - CType::Uint32 => "C.uint32_t".into(), - CType::Int64 => "C.int64_t".into(), - CType::Uint64 => "C.uint64_t".into(), - CType::Float => "C.float".into(), - CType::Double => "C.double".into(), - CType::Bool => "C._Bool".into(), - CType::Size => "C.size_t".into(), - CType::Char => "C.char".into(), - CType::Ptr { pointee, .. } if **pointee == CType::Void => "unsafe.Pointer".into(), - CType::Ptr { pointee, .. } => format!("*{}", cgo_type(pointee, prefix)), - CType::Void => unreachable!("void only appears behind a pointer"), - named => format!("C.{}", named.render_c(prefix)), - } -} - -/// `ct` with every `const` qualifier dropped, matching the const-free -/// prototypes cgo writes into `_cgo_export.h` for exported Go functions. -pub(crate) fn strip_const(ct: &CType) -> CType { - match ct { - CType::Ptr { pointee, .. } => CType::Ptr { - konst: ConstPos::None, - pointee: Box::new(strip_const(pointee)), - }, - other => other.clone(), - } -} diff --git a/crates/weaveffi-gen/src/targets/js/codec.rs b/crates/weaveffi-gen/src/targets/js/codec.rs deleted file mode 100644 index f4e35ac9..00000000 --- a/crates/weaveffi-gen/src/targets/js/codec.rs +++ /dev/null @@ -1,200 +0,0 @@ -//! Value-buffer codec emitters. -//! -//! Records, rich enums, optionals, lists, maps, and error payloads cross the -//! ABI as value buffers. The runtime's `$Writer` and `$Reader` implement the -//! wire primitives (`writeI32`, `readString`, ...) and the optional, list, -//! and map combinators; this module emits the expressions that compose them -//! for a given type, plus one writer and one reader function per record and -//! rich enum (`$w$kv$Entry`, `$r$kv$Entry`). Every dispatch goes through -//! [`Ty::wire`], and primitives collapse onto one arm through -//! [`Prim::pascal`]. -//! -//! Object tokens carry a strong reference: writing one clones the wrapper's -//! native object (`$clone`), and reading one adopts the reference into a new -//! wrapper. The transport defines `$token`, which turns the `u64` token into -//! its handle representation. - -use weaveffi_model::model::{BindingModel, Prim, Ty, WireType}; - -use crate::codegen::CodeWriter; -use crate::targets::js::names::{helper, js_string, type_decl}; - -/// A function expression `(w, v) => ...` writing one value of `ty`. -pub(crate) fn writer_fn(ty: &Ty) -> String { - match ty.wire() { - WireType::Prim(p) => format!("$W.{}", p.pascal()), - WireType::Enum(_) => "$W.I32".into(), - WireType::User(n) => helper("w", n), - _ => format!("(w, v) => {}", write_expr(ty, "v")), - } -} - -/// A function expression `(r) => value` reading one value of `ty`. -pub(crate) fn reader_fn(ty: &Ty) -> String { - match ty.wire() { - WireType::Prim(p) => format!("$R.{}", p.pascal()), - WireType::Enum(_) => "$R.I32".into(), - WireType::User(n) => helper("r", n), - _ => format!("(r) => {}", read_expr(ty)), - } -} - -/// The expression writing `val` of type `ty` to the writer `w`. -pub(crate) fn write_expr(ty: &Ty, val: &str) -> String { - match ty.wire() { - WireType::Prim(p) => format!("w.write{}({val})", p.pascal()), - WireType::Enum(_) => format!("w.writeI32({val})"), - WireType::Object(n) => format!("w.writeU64(BigInt($clone({val}, {})))", type_decl(n)), - WireType::User(n) => format!("{}(w, {val})", helper("w", n)), - WireType::Optional(inner) => format!("w.writeOpt({val}, {})", writer_fn(inner)), - WireType::List(inner) => format!("w.writeList({val}, {})", writer_fn(inner)), - WireType::Map(k, v) => format!("w.writeMap({val}, {}, {})", key_writer_fn(k), writer_fn(v)), - } -} - -/// The expression reading one value of type `ty` from the reader `r`. -pub(crate) fn read_expr(ty: &Ty) -> String { - match ty.wire() { - WireType::Prim(p) => format!("r.read{}()", p.pascal()), - WireType::Enum(_) => "r.readI32()".into(), - WireType::Object(n) => format!("$adopt({}, $token(r.readU64()))", type_decl(n)), - WireType::User(n) => format!("{}(r)", helper("r", n)), - WireType::Optional(inner) => format!("r.readOpt({})", reader_fn(inner)), - WireType::List(inner) => format!("r.readList({})", reader_fn(inner)), - WireType::Map(k, v) => format!("r.readMap({}, {})", reader_fn(k), reader_fn(v)), - } -} - -/// A writer for one map key. Plain-object keys are always strings, so -/// numeric, `bigint`, `bool`, and enum keys convert first (a `Map`'s typed -/// keys pass through the same conversions unchanged). -fn key_writer_fn(ty: &Ty) -> String { - match ty.wire() { - WireType::Prim(Prim::String) => "$W.String".into(), - WireType::Prim(Prim::Bool) => "(w, k) => w.writeBool(k === true || k === 'true')".into(), - WireType::Prim(p @ (Prim::I64 | Prim::U64)) => { - format!("(w, k) => w.write{}(BigInt(k))", p.pascal()) - } - WireType::Prim(p) => format!("(w, k) => w.write{}(Number(k))", p.pascal()), - WireType::Enum(_) => "(w, k) => w.writeI32(Number(k))".into(), - other => unreachable!("validation admits no {other:?} map keys"), - } -} - -/// Emit the writer and reader of every record and rich enum in the model. -pub(crate) fn emit_codecs(w: &mut CodeWriter, model: &BindingModel) { - for m in &model.modules { - for s in &m.structs { - let dotted = format!("{}.{}", m.dot_path, s.name); - w.block( - format!("function {}(w, v) {{", helper("w", &dotted)), - "}", - |w| { - for f in &s.fields { - w.line(format!("{};", write_expr(&f.ty, &format!("v.{}", f.name)))); - } - }, - ); - w.block( - format!("function {}(r) {{", helper("r", &dotted)), - "}", - |w| { - if s.fields.is_empty() { - w.line("return {};"); - } else { - w.block("return {", "};", |w| { - for f in &s.fields { - w.line(format!("{}: {},", f.name, read_expr(&f.ty))); - } - }); - } - }, - ); - } - for e in m.enums.iter().filter(|e| e.rich) { - let dotted = format!("{}.{}", m.dot_path, e.name); - let what = js_string(&format!("unknown {} tag: ", e.name)); - w.block( - format!("function {}(w, v) {{", helper("w", &dotted)), - "}", - |w| { - w.block("switch (v.tag) {", "}", |w| { - for v in &e.variants { - w.line(format!("case {}:", js_string(&v.name))); - w.scope(|w| { - w.line(format!("w.writeI32({});", v.value)); - for f in &v.fields { - w.line(format!( - "{};", - write_expr(&f.ty, &format!("v.{}", f.name)) - )); - } - w.line("return;"); - }); - } - w.line("default:"); - w.line(format!(" throw new TypeError({what} + (v && v.tag));")); - }); - }, - ); - w.block( - format!("function {}(r) {{", helper("r", &dotted)), - "}", - |w| { - w.line("const tag = r.readI32();"); - w.block("switch (tag) {", "}", |w| { - for v in &e.variants { - let fields: String = v - .fields - .iter() - .map(|f| format!(", {}: {}", f.name, read_expr(&f.ty))) - .collect(); - w.line(format!( - "case {}: return {{ tag: {}{fields} }};", - v.value, - js_string(&v.name) - )); - } - w.line("default:"); - w.line(format!(" throw new $Error(-3, {what} + tag);")); - }); - }, - ); - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn primitives_collapse_onto_pascal_names() { - assert_eq!(write_expr(&Ty::I64, "v.n"), "w.writeI64(v.n)"); - assert_eq!(read_expr(&Ty::StringUtf8), "r.readString()"); - assert_eq!(writer_fn(&Ty::Bytes), "$W.Bytes"); - assert_eq!(reader_fn(&Ty::Enum("kv.Kind".into())), "$R.I32"); - } - - #[test] - fn composites_use_the_runtime_combinators() { - let ty = Ty::Optional(Box::new(Ty::List(Box::new(Ty::Record("kv.Entry".into()))))); - assert_eq!( - write_expr(&ty, "v.x"), - "w.writeOpt(v.x, (w, v) => w.writeList(v, $w$kv$Entry))" - ); - assert_eq!(read_expr(&ty), "r.readOpt((r) => r.readList($r$kv$Entry))"); - let map = Ty::Map( - Box::new(Ty::U64), - Box::new(Ty::Interface("kv.Store".into())), - ); - assert_eq!( - write_expr(&map, "v.m"), - "w.writeMap(v.m, (w, k) => w.writeU64(BigInt(k)), (w, v) => w.writeU64(BigInt($clone(v, kv$Store))))" - ); - assert_eq!( - read_expr(&map), - "r.readMap($R.U64, (r) => $adopt(kv$Store, $token(r.readU64())))" - ); - } -} diff --git a/crates/weaveffi-gen/src/targets/kotlin/bridge.rs b/crates/weaveffi-gen/src/targets/kotlin/bridge.rs deleted file mode 100644 index 1b22fcf5..00000000 --- a/crates/weaveffi-gen/src/targets/kotlin/bridge.rs +++ /dev/null @@ -1,206 +0,0 @@ -//! `JniBridge.kt`: the internal object declaring one `external fun` per C -//! symbol (named after the symbol, so two modules' functions never -//! collide), the error factory the shim throws through, and the static -//! dispatch shims callback trampolines call. - -use crate::codegen::CodeWriter; -use weaveffi_model::model::{ - BindingModel, CallShape, CallbackInterfaceBinding, CallbackMethodBinding, FnBinding, -}; - -use crate::targets::kotlin::calls::{item_slot, lift, lower}; -use crate::targets::kotlin::names::{kt_member, kt_param, Names}; - -/// The `JniBridge` member name of a callback method's dispatch shim. -pub(crate) fn shim_name( - n: &Names, - cb: &CallbackInterfaceBinding, - m: &CallbackMethodBinding, -) -> String { - n.native(&format!("{}_{}", cb.c_tag, m.name)) -} - -/// The JVM descriptor of a callback method's dispatch shim. -pub(crate) fn shim_descriptor( - n: &Names, - cb: &CallbackInterfaceBinding, - m: &CallbackMethodBinding, -) -> String { - let mut sig = format!("(L{}/{};", n.package_path, n.ty(&cb.name)); - for p in &m.params { - sig.push_str(&n.jni_descriptor(Some(&p.ty))); - } - sig.push(')'); - sig.push_str(&n.jni_descriptor(m.ret.as_ref())); - sig -} - -/// The `external fun` declarations backing callable `f`. -fn callable_natives(w: &mut CodeWriter, n: &Names, f: &FnBinding) { - let mut params: Vec = Vec::new(); - if f.has_self { - params.push("_self: Long".to_string()); - } - params.extend( - f.params - .iter() - .map(|p| format!("{}: {}", kt_param(&p.name), n.jni_type(&p.ty))), - ); - match &f.shape { - CallShape::Sync(abi) => { - let ret = f - .ret - .as_ref() - .map(|t| format!(": {}", n.jni_type(t))) - .unwrap_or_default(); - w.line(format!( - "@JvmStatic external fun {}({}){ret}", - n.native(&abi.symbol), - params.join(", ") - )); - } - CallShape::Async(ab) => { - if f.cancellable { - params.push("_token: Long".to_string()); - } - params.push("_done: NativeCompletion<*>".to_string()); - w.line(format!( - "@JvmStatic external fun {}({})", - n.native(&ab.launch.symbol), - params.join(", ") - )); - } - CallShape::Iterator(it) => { - w.line(format!( - "@JvmStatic external fun {}({}): Long", - n.native(&it.launch.symbol), - params.join(", ") - )); - match item_slot(&it.elem) { - None => w.line(format!( - "@JvmStatic external fun {}(_iter: Long): ByteArray?", - n.native(&it.next.symbol) - )), - Some(slot) => w.line(format!( - "@JvmStatic external fun {}(_iter: Long, _out: {}): Boolean", - n.native(&it.next.symbol), - slot.kotlin_array - )), - }; - w.line(format!( - "@JvmStatic external fun {}(_iter: Long)", - n.native(&it.destroy_symbol) - )); - } - } -} - -/// The static shim a trampoline calls for one callback method: lifts each -/// argument from its JNI form, calls the implementation, and lowers the -/// result. -fn callback_shim( - w: &mut CodeWriter, - n: &Names, - cb: &CallbackInterfaceBinding, - m: &CallbackMethodBinding, -) { - let mut params = vec![format!("_impl: {}", n.ty(&cb.name))]; - params.extend( - m.params - .iter() - .map(|p| format!("{}: {}", kt_param(&p.name), n.jni_type(&p.ty))), - ); - let args: Vec = m - .params - .iter() - .map(|p| lift(n, &p.ty, &kt_param(&p.name))) - .collect(); - let call = format!("_impl.{}({})", kt_member(&m.name), args.join(", ")); - let name = shim_name(n, cb, m); - w.line("@JvmStatic"); - match &m.ret { - None => w.line(format!("fun {name}({}) {{ {call} }}", params.join(", "))), - Some(t) => w.line(format!( - "fun {name}({}): {} = {}", - params.join(", "), - n.jni_type(t), - lower(n, t, &call) - )), - }; -} - -/// Render the body of `JniBridge.kt` (after the package line). -pub(crate) fn render_bridge(n: &Names, model: &BindingModel) -> String { - let mut w = CodeWriter::four_space(); - w.blank(); - w.line("/**"); - w.line(format!( - " * The JNI entry points of `lib{}_jni`: one native per C symbol, named after", - n.library - )); - w.line(" * it, plus the hooks the shim calls back into. Internal plumbing for the"); - w.line(" * public wrappers."); - w.line(" */"); - w.line("internal object JniBridge {"); - w.scope(|w| { - w.line("init {"); - w.line(" NativeLibrary.ensureLoaded()"); - w.line("}"); - w.blank(); - w.line("/** The exception for a native failure; domain 0 is the generic one. */"); - w.line("@JvmStatic"); - w.line("@Suppress(\"UNUSED_PARAMETER\")"); - w.line("fun error(domain: Int, code: Int, message: ByteArray, payload: ByteArray?): FfiException {"); - w.scope(|w| { - w.line("val text = decodeUtf8(message)"); - w.line("return when (domain) {"); - w.scope(|w| { - for (i, eb) in n.domains(model) { - w.line(format!( - "{i} -> {}.fromCode(code, text, payload)", - n.exception(eb) - )); - } - w.line("else -> FfiException(code, text)"); - }); - w.line("}"); - }); - w.line("}"); - if model.has_callback_interfaces() { - w.blank(); - w.line("/** The message a failed callback reports to the producer (code -4). */"); - w.line("@JvmStatic"); - w.line("fun describe(error: Throwable): ByteArray = encodeUtf8(error.toString())"); - } - w.blank(); - w.line("@JvmStatic external fun debug_live(kind: Int): Long"); - if model.has_async() { - w.line("@JvmStatic external fun cancel_token_create(): Long"); - w.line("@JvmStatic external fun cancel_token_cancel(token: Long)"); - w.line("@JvmStatic external fun cancel_token_destroy(token: Long)"); - } - for m in &model.modules { - for i in &m.interfaces { - w.line(format!( - "@JvmStatic external fun {}(_self: Long): Long", - n.native(&i.clone_symbol) - )); - w.line(format!( - "@JvmStatic external fun {}(_self: Long)", - n.native(&i.destroy_symbol) - )); - } - for f in m.callables() { - callable_natives(w, n, f); - } - } - for (_, cb) in model.callback_interfaces() { - for m in &cb.methods { - w.blank(); - callback_shim(w, n, cb, m); - } - } - }); - w.line("}"); - w.finish() -} diff --git a/crates/weaveffi-gen/src/targets/kotlin/codec.rs b/crates/weaveffi-gen/src/targets/kotlin/codec.rs deleted file mode 100644 index 4ceeca63..00000000 --- a/crates/weaveffi-gen/src/targets/kotlin/codec.rs +++ /dev/null @@ -1,88 +0,0 @@ -//! Value-buffer codec expressions: the Kotlin statement writing and the -//! expression reading every wire shape, dispatched on [`Ty::wire`]. The -//! runtime's `BufferWriter`/`BufferReader` name one method per [`Prim`] -//! (`writeI32`, `readString`), so primitives need a single arm. -//! -//! [`Prim`]: weaveffi_model::model::Prim - -use weaveffi_model::model::{Ty, WireType}; -use weaveffi_model::plan::destroy_symbol; - -use crate::targets::kotlin::names::Names; - -/// The Kotlin statement writing `expr` (the public Kotlin value of `t`) into -/// the writer `w`. Nested lambdas name their parameter `_v{depth}` (and -/// `_k{depth}` for map keys), which no user identifier can spell. -/// -/// An object is written as a token carrying a new strong reference -/// (`cloneHandle()`), so the encoding never hands over the reference the -/// wrapper still owns; the writer releases the token again if the encoding -/// fails before the buffer reaches the producer. -pub(crate) fn write_expr(n: &Names, t: &Ty, w: &str, expr: &str, depth: usize) -> String { - match t.wire() { - WireType::Prim(p) => format!("{w}.write{}({expr})", p.pascal()), - WireType::Enum(_) => format!("{w}.writeI32({expr}.value)"), - WireType::Object(name) => format!( - "{w}.writeObject({expr}.cloneHandle(), JniBridge::{})", - n.native(&destroy_symbol(name, &n.prefix)) - ), - WireType::User(name) => format!("pack{}({w}, {expr})", n.ty(name)), - WireType::Optional(inner) => { - let v = format!("_v{depth}"); - format!( - "{w}.writeOptional({expr}) {{ {v} -> {} }}", - write_expr(n, inner, w, &v, depth + 1) - ) - } - WireType::List(inner) => { - let v = format!("_v{depth}"); - format!( - "{w}.writeList({expr}) {{ {v} -> {} }}", - write_expr(n, inner, w, &v, depth + 1) - ) - } - WireType::Map(k, v) => { - let kv = format!("_k{depth}"); - let vv = format!("_v{depth}"); - format!( - "{w}.writeMap({expr}, {{ {kv} -> {} }}, {{ {vv} -> {} }})", - write_expr(n, k, w, &kv, depth + 1), - write_expr(n, v, w, &vv, depth + 1) - ) - } - } -} - -/// The Kotlin expression reading a value of `t` from the reader `r`; the -/// inverse of [`write_expr`]. An object token is adopted into a new wrapper, -/// which owes the reference's release. -pub(crate) fn read_expr(n: &Names, t: &Ty, r: &str) -> String { - match t.wire() { - WireType::Prim(p) => format!("{r}.read{}()", p.pascal()), - WireType::Enum(name) => format!("{}.fromValue({r}.readI32())", n.ty(name)), - WireType::Object(name) => format!("{}.fromHandle({r}.readObject())", n.ty(name)), - WireType::User(name) => format!("unpack{}({r})", n.ty(name)), - WireType::Optional(inner) => format!("{r}.readOptional {{ {} }}", read_expr(n, inner, r)), - WireType::List(inner) => format!("{r}.readList {{ {} }}", read_expr(n, inner, r)), - WireType::Map(k, v) => format!( - "{r}.readMap({{ {} }}, {{ {} }})", - read_expr(n, k, r), - read_expr(n, v, r) - ), - } -} - -/// The Kotlin expression encoding the public value `expr` of `t` into a -/// fresh value buffer. -pub(crate) fn encode_expr(n: &Names, t: &Ty, expr: &str) -> String { - format!( - "encodeBuffer {{ _w -> {} }}", - write_expr(n, t, "_w", expr, 0) - ) -} - -/// The Kotlin expression decoding the value buffer `expr` into the public -/// value of `t`. -pub(crate) fn decode_expr(n: &Names, t: &Ty, expr: &str) -> String { - format!("decodeBuffer({expr}) {{ _r -> {} }}", read_expr(n, t, "_r")) -} diff --git a/crates/weaveffi-gen/src/targets/kotlin/docs.rs b/crates/weaveffi-gen/src/targets/kotlin/docs.rs deleted file mode 100644 index 19ed7159..00000000 --- a/crates/weaveffi-gen/src/targets/kotlin/docs.rs +++ /dev/null @@ -1,98 +0,0 @@ -//! KDoc comment emission. - -use crate::codegen::common::{emit_doc as common_emit_doc, DocCommentStyle}; -use crate::codegen::CodeWriter; -use weaveffi_model::model::ParamBinding; - -use crate::targets::kotlin::names::kt_param; - -/// Emits a Kotlin KDoc comment at `indent`. Single-line docs collapse to -/// `/** text */`; multi-line docs expand to a block with ` * ` prefixed lines. -pub(crate) fn emit_doc(out: &mut String, doc: &Option, indent: &str) { - common_emit_doc(out, doc, indent, DocCommentStyle::Javadoc); -} - -/// Emits a KDoc block for a function: function doc plus `@param name desc` -/// lines for each documented parameter; without parameter docs this is -/// [`emit_doc`]. -pub(crate) fn emit_fn_doc( - out: &mut String, - doc: &Option, - params: &[ParamBinding], - indent: &str, -) { - let has_param_docs = params.iter().any(|p| p.doc.is_some()); - if !has_param_docs { - emit_doc(out, doc, indent); - return; - } - let trimmed_doc = doc.as_ref().map(|d| d.trim()).filter(|d| !d.is_empty()); - out.push_str(indent); - out.push_str("/**\n"); - if let Some(d) = trimmed_doc { - for line in d.lines() { - out.push_str(indent); - if line.is_empty() { - out.push_str(" *\n"); - } else { - out.push_str(" * "); - out.push_str(line); - out.push('\n'); - } - } - } - for p in params { - if let Some(pdoc) = &p.doc { - let pdoc = pdoc.trim(); - if pdoc.is_empty() { - continue; - } - let mut lines = pdoc.lines(); - if let Some(first) = lines.next() { - out.push_str(indent); - out.push_str(&format!(" * @param {} {}\n", p.name, first)); - } - for line in lines { - out.push_str(indent); - if line.is_empty() { - out.push_str(" *\n"); - } else { - out.push_str(" * "); - out.push_str(line); - out.push('\n'); - } - } - } - } - out.push_str(indent); - out.push_str(" */\n"); -} - -/// Emit [`emit_doc`] at the writer's current depth by rendering into a scratch -/// buffer and splicing it verbatim, so a [`CodeWriter`]-based renderer can -/// interleave KDoc comments without re-implementing their formatting. -pub(crate) fn writer_doc(w: &mut CodeWriter, doc: &Option) { - let mut tmp = String::new(); - emit_doc(&mut tmp, doc, &w.indent_str()); - w.raw(tmp); -} - -/// Emit [`emit_fn_doc`] at the writer's current depth (KDoc plus `@param` -/// tags), splicing the pre-indented block verbatim like [`writer_doc`]. -pub(crate) fn writer_fn_doc(w: &mut CodeWriter, doc: &Option, params: &[ParamBinding]) { - let mut tmp = String::new(); - emit_fn_doc(&mut tmp, doc, params, &w.indent_str()); - w.raw(tmp); -} - -/// Clone `params` with their Kotlin spellings so KDoc `@param` tags match -/// the emitted parameter names. -pub(crate) fn camel_params(params: &[ParamBinding]) -> Vec { - params - .iter() - .map(|p| ParamBinding { - name: kt_param(&p.name), - ..p.clone() - }) - .collect() -} diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_callbacks.c b/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_callbacks.c deleted file mode 100644 index b834fe86..00000000 --- a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_callbacks.c +++ /dev/null @@ -1,63 +0,0 @@ - -/* Callback interfaces: the producer holds a global reference to the Kotlin - implementation as `ctx`; trampolines call the matching static dispatch - method on JniBridge from whatever thread the producer uses. */ -static jmethodID Jni_describe = NULL; - -static jint Jni_load_callbacks(JNIEnv* env) { - Jni_describe = (*env)->GetStaticMethodID(env, Jni_bridge, "describe", "(Ljava/lang/Throwable;)[B"); - return Jni_describe != NULL ? JNI_OK : JNI_ERR; -} - -/* Enters a trampoline: the JNIEnv with a local frame pushed, or NULL after - reporting the failure through `out_err`. */ -static JNIEnv* Jni_callback_begin({{PREFIX}}_error* out_err) { - JNIEnv* env = Jni_env(); - if (env == NULL) { - {{PREFIX}}_error_set(out_err, -4, "could not attach the calling thread to the JVM"); - return NULL; - } - if ((*env)->PushLocalFrame(env, 16) != 0) { - (*env)->ExceptionClear(env); - {{PREFIX}}_error_set(out_err, -4, "JNI local frame exhausted"); - return NULL; - } - return env; -} - -/* Leaves a trampoline. When the implementation threw, reports its - `toString()` through `out_err` with code -4 and returns 1; nothing ever - unwinds through the producer's frames. */ -static int Jni_callback_end(JNIEnv* env, {{PREFIX}}_error* out_err) { - int failed = 0; - if ((*env)->ExceptionCheck(env)) { - failed = 1; - jthrowable ex = (*env)->ExceptionOccurred(env); - (*env)->ExceptionClear(env); - jbyteArray text = (jbyteArray)(*env)->CallStaticObjectMethod(env, Jni_bridge, Jni_describe, ex); - char* message = NULL; - if ((*env)->ExceptionCheck(env)) { - (*env)->ExceptionClear(env); - } else if (text != NULL) { - jsize len = (*env)->GetArrayLength(env, text); - message = (char*)malloc((size_t)len + 1u); - if (message != NULL) { - (*env)->GetByteArrayRegion(env, text, 0, len, (jbyte*)message); - message[len] = '\0'; - } - } - {{PREFIX}}_error_set(out_err, -4, message != NULL ? message : "callback implementation threw"); - free(message); - } - (*env)->PopLocalFrame(env, NULL); - return failed; -} - -/* The vtable `free` entry: drops the producer's reference to the Kotlin - implementation. */ -static void Jni_release_callback(void* ctx) { - JNIEnv* env = Jni_env(); - if (env != NULL && ctx != NULL) { - (*env)->DeleteGlobalRef(env, (jobject)ctx); - } -} diff --git a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_threads.c b/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_threads.c deleted file mode 100644 index 84cd06c0..00000000 --- a/crates/weaveffi-gen/src/targets/kotlin/runtime/jni_threads.c +++ /dev/null @@ -1,49 +0,0 @@ - -/* Producer threads (async completions, callback calls, vtable `free`) are - usually not JVM threads. The first time one calls in it attaches as a - daemon and keeps its JNIEnv; a thread-local destructor detaches it when - the thread exits, so a busy producer thread pays for one attach, not one - per call. */ -#if defined(_WIN32) -static void Jni_remember_attach(void) {} -#else -#include - -static pthread_key_t Jni_env_key; -static pthread_once_t Jni_env_once = PTHREAD_ONCE_INIT; - -static void Jni_detach(void* env) { - (void)env; - if (Jni_vm != NULL) { - (*Jni_vm)->DetachCurrentThread(Jni_vm); - } -} - -static void Jni_make_env_key(void) { - pthread_key_create(&Jni_env_key, Jni_detach); -} - -static void Jni_remember_attach(void) { - pthread_once(&Jni_env_once, Jni_make_env_key); - pthread_setspecific(Jni_env_key, (void*)Jni_vm); -} -#endif - -/* The calling thread's JNIEnv, attaching the thread if needed; NULL when - the JVM is unavailable. */ -static JNIEnv* Jni_env(void) { - JNIEnv* env = NULL; - if (Jni_vm == NULL) { - return NULL; - } - if ((*Jni_vm)->GetEnv(Jni_vm, (void**)&env, JNI_VERSION_1_6) == JNI_OK) { - return env; - } - /* `void*` converts to the `JNIEnv**` Android declares and the `void**` - the JDK declares. */ - if ((*Jni_vm)->AttachCurrentThreadAsDaemon(Jni_vm, (void*)&env, NULL) != JNI_OK) { - return NULL; - } - Jni_remember_attach(); - return env; -} diff --git a/crates/weaveffi-gen/src/targets/kotlin/tests.rs b/crates/weaveffi-gen/src/targets/kotlin/tests.rs deleted file mode 100644 index 258a9244..00000000 --- a/crates/weaveffi-gen/src/targets/kotlin/tests.rs +++ /dev/null @@ -1,384 +0,0 @@ -//! Unit tests rendering a small IR through the generator: identity-driven -//! names and layout, configuration overrides, name escaping, the ABI 3 JNI -//! surface (ptr+len strings, load-time checks, cancellation), object -//! lifetime, callbacks, iterators, and packaging. - -use camino::{Utf8Path, Utf8PathBuf}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; - -use crate::backend::LanguageBackend; -use crate::package::{FileContent, PackageContext}; -use crate::platform::{BinarySet, NativeBinary, Platform}; -use crate::targets::kotlin::{KotlinConfig, KotlinFlavor, KotlinGenerator}; - -const FIXTURE: &str = r#" -version: "0.10.0" -modules: - - name: bus - enums: - - name: Level - variants: - - { name: Low, value: 0 } - - { name: High, value: 1 } - structs: - - name: Event - fields: - - { name: id, type: i64 } - - { name: source, type: Store } - callback_interfaces: - - name: Subscriber - methods: - - name: on_event - params: - - { name: event, type: Event } - - { name: note, type: string } - return: Level - errors: - name: BusError - codes: - - { name: Closed, code: 1, message: "Bus is closed" } - interfaces: - - name: Store - constructors: - - name: new - params: - - { name: name, type: string } - methods: - - name: get - params: - - { name: key, type: string } - return: "string?" - throws: true - - name: larger - params: - - { name: other, type: "Store?" } - return: "Store?" - - name: keys - params: [] - return: "iter" - - name: counts - params: [] - return: "iter" - - name: subscribe - params: - - { name: sub, type: Subscriber } - - name: drain - params: [] - return: i64 - async: true - cancellable: true - throws: true - functions: - - name: get - params: - - { name: id, type: i64 } - return: Event - modules: - - name: stats - structs: - - name: Stats - fields: - - { name: total, type: i64 } - functions: - - name: get - params: - - { name: store, type: Store } - return: Stats - - name: units - enums: - - name: Unit - variants: - - { name: Px, value: 0 } - functions: - - name: echo - params: - - { name: unit, type: Unit } - return: Unit -"#; - -fn api() -> ResolvedApi { - let api = weaveffi_model::parse::parse_api_str(FIXTURE, "yaml").expect("fixture parses"); - weaveffi_model::validate::validate_api(api, None) - .expect("fixture validates") - .with_identity(Identity::named("event-bus")) -} - -/// Render with `config` and return `(path, contents)` pairs, paths with `/`. -fn render(config: &KotlinConfig) -> Vec<(String, String)> { - let api = api(); - let model = BindingModel::build(&api); - KotlinGenerator - .files(&api, &model, Utf8Path::new("out"), config) - .into_iter() - .map(|f| (f.path.as_str().replace('\\', "/"), f.contents)) - .collect() -} - -fn file<'a>(files: &'a [(String, String)], suffix: &str) -> &'a str { - files - .iter() - .find(|(p, _)| p.ends_with(suffix)) - .map(|(_, c)| c.as_str()) - .unwrap_or_else(|| { - let paths: Vec<&String> = files.iter().map(|f| &f.0).collect(); - panic!("{suffix} missing from {paths:?}") - }) -} - -#[test] -fn layout_follows_the_identity() { - let files = render(&KotlinConfig::default()); - let paths: Vec<&str> = files.iter().map(|(p, _)| p.as_str()).collect(); - assert_eq!( - paths, - [ - "out/kotlin/settings.gradle.kts", - "out/kotlin/build.gradle.kts", - "out/kotlin/consumer-rules.pro", - "out/kotlin/src/main/cpp/CMakeLists.txt", - "out/kotlin/src/main/cpp/event_bus.h", - "out/kotlin/src/main/cpp/event_bus_jni.c", - "out/kotlin/src/main/kotlin/event_bus/Runtime.kt", - "out/kotlin/src/main/kotlin/event_bus/Buffers.kt", - "out/kotlin/src/main/kotlin/event_bus/Async.kt", - "out/kotlin/src/main/kotlin/event_bus/JniBridge.kt", - "out/kotlin/src/main/kotlin/event_bus/Bus.kt", - "out/kotlin/src/main/kotlin/event_bus/Units.kt", - ] - ); - let runtime = file(&files, "Runtime.kt"); - assert!(runtime.contains("package event_bus\n")); - assert!(runtime.contains("private const val LIBRARY = \"event_bus\"")); - assert!(runtime.contains("private const val JNI_LIBRARY = \"event_bus_jni\"")); - assert!(runtime.contains("System.getenv(\"EVENT_BUS_LIBRARY\")")); - let settings = file(&files, "settings.gradle.kts"); - assert!(settings.contains("rootProject.name = \"event-bus\"")); - assert!(settings.contains("google()")); - let gradle = file(&files, "build.gradle.kts"); - assert!(gradle.contains("id(\"com.android.library\")")); - assert!(gradle.contains("namespace = \"event_bus\"")); - assert!(gradle.contains("minSdk = 21")); - assert!(gradle.contains("`maven-publish`")); - assert!(gradle.contains("kotlinx-coroutines-core")); - let cmake = file(&files, "CMakeLists.txt"); - assert!(cmake.contains("add_library(event_bus_jni SHARED event_bus_jni.c)")); - assert!(cmake.contains("find_library(EVENT_BUS_PRODUCER NAMES event_bus")); - assert!(cmake.contains("../jniLibs/${ANDROID_ABI}")); - // The bundled header is the C target's, verbatim. - for (path, contents) in files.iter().filter(|(p, _)| !p.ends_with(".h")) { - let body: String = contents.lines().skip(3).collect::>().join("\n"); - assert!( - !body.contains("WeaveFFI") && !body.contains("weaveffi"), - "{path} carries WeaveFFI branding" - ); - } -} - -#[test] -fn configuration_overrides_package_and_flavor() { - let config = KotlinConfig { - package: Some("com.example.bus".into()), - flavor: KotlinFlavor::Jvm, - ..KotlinConfig::default() - }; - let files = render(&config); - assert!(files - .iter() - .all(|(p, _)| !p.ends_with("consumer-rules.pro"))); - assert!(files - .iter() - .any(|(p, _)| p == "out/kotlin/src/main/kotlin/com/example/bus/JniBridge.kt")); - let gradle = file(&files, "build.gradle.kts"); - assert!(gradle.contains("kotlin(\"jvm\")")); - assert!(gradle.contains("group = \"com.example.bus\"")); - assert!(!gradle.contains("com.android.library")); - let c = file(&files, "event_bus_jni.c"); - assert!(c.contains("Java_com_example_bus_JniBridge_bus_1Store_1get(")); - assert!(c.contains("FindClass(env, \"com/example/bus/JniBridge\")")); -} - -#[test] -fn modules_are_objects_and_shadowing_names_are_escaped() { - let files = render(&KotlinConfig::default()); - let bus = file(&files, "Bus.kt"); - // Two modules each declare `get`; the module objects keep them apart, - // and the `stats` object steps aside for the `Stats` record. - assert!(bus.contains("object Bus {")); - assert!(bus.contains(" fun get(id: Long): Event = ")); - assert!(bus.contains(" object StatsModule {")); - assert!(bus.contains("fun get(store: Store): Stats = store.handle.borrow { _h0 ->")); - // A user type named like a Kotlin builtin gains a trailing underscore. - let units = file(&files, "Units.kt"); - assert!(units.contains("enum class Unit_(val value: Int) {")); - assert!(units.contains( - "fun echo(unit: Unit_): Unit_ = Unit_.fromValue(JniBridge.units_echo(unit.value))" - )); - // JNI natives are named after their (unique) C symbols. - let bridge = file(&files, "JniBridge.kt"); - assert!(bridge.contains("@JvmStatic external fun bus_get(id: Long): ByteArray")); - assert!(bridge.contains("@JvmStatic external fun bus_stats_get(store: Long): ByteArray")); -} - -#[test] -fn strings_cross_as_bytes_with_lengths() { - let files = render(&KotlinConfig::default()); - let bus = file(&files, "Bus.kt"); - assert!(bus.contains( - "fun get(key: String): String? = handle.borrow { _self ->\n decodeBuffer(JniBridge.bus_Store_get(_self, encodeUtf8(key))) { _r -> _r.readOptional { _r.readString() } }" - )); - let c = file(&files, "event_bus_jni.c"); - assert!(c.contains("Jni_bytes p_key_b = Jni_borrow_bytes(env, p_key);")); - assert!(c.contains( - "event_bus_bus_Store_get((const event_bus_bus_Store*)(intptr_t)self, p_key_b.ptr, p_key_b.len, &out_len, &err);" - )); - assert!(c.contains("return Jni_take_bytes(env, rv, out_len);")); - assert!(c.contains("event_bus_free_bytes((uint8_t*)ptr, len);")); - assert!(!c.contains("free_string")); - assert!(!c.contains("GetStringUTFChars")); -} - -#[test] -fn load_checks_the_abi_revision_and_every_root_checksum() { - let api = api(); - let model = BindingModel::build(&api); - let files = render(&KotlinConfig::default()); - let c = file(&files, "event_bus_jni.c"); - assert!(c.contains("if (event_bus_abi_version() != 3u) {")); - for root in model.roots() { - let sum = root.checksum.expect("roots carry a checksum"); - assert!(c.contains(&format!( - "if (event_bus_{}_checksum() != UINT64_C(0x{sum:016x})) {{", - root.name - ))); - assert!(c.contains(&format!( - "module '{}' does not match these bindings", - root.name - ))); - } -} - -#[test] -fn objects_are_borrowed_for_every_call() { - let files = render(&KotlinConfig::default()); - let bus = file(&files, "Bus.kt"); - assert!(bus.contains("class Store private constructor(address: Long) : AutoCloseable {")); - assert!(bus.contains( - "internal val handle: NativeHandle = NativeCleaner.register(this, NativeHandle(address, JniBridge::bus_Store_destroy))" - )); - assert!(bus.contains( - "fun larger(other: Store?): Store? = handle.borrow { _self ->\n other?.handle.borrowOrNull { _h0 ->\n Store.fromHandleOrNull(JniBridge.bus_Store_larger(_self, _h0))" - )); - // Objects inside buffers are cloned on the way in and released again if - // the encoding fails. - assert!(bus.contains("_w.writeObject(_v.source.cloneHandle(), JniBridge::bus_Store_destroy)")); - let runtime = file(&files, "Runtime.kt"); - assert!(runtime.contains("PhantomReference")); - assert!(!runtime.contains("java.lang.ref.Cleaner")); -} - -#[test] -fn cancellable_async_wires_the_token() { - let files = render(&KotlinConfig::default()); - let bus = file(&files, "Bus.kt"); - assert!(bus.contains( - "suspend fun drain(): Long = awaitNative(true, 1, { it as Long }) { _token, _done ->" - )); - assert!(bus.contains("JniBridge.bus_Store_drain(_self, _token, _done)")); - let runtime = file(&files, "Async.kt"); - assert!(runtime.contains("cont.invokeOnCancellation { completion.cancel() }")); - assert!(runtime.contains("if (code == -5)")); - let c = file(&files, "event_bus_jni.c"); - assert!(c.contains( - "(event_bus_cancel_token*)(intptr_t)cancel_token, Jni_done_bus_Store_drain, context);" - )); - assert!(c.contains("CallVoidMethod(env, (jobject)context, Jni_on_long, (jlong)result);")); -} - -#[test] -fn callbacks_dispatch_through_cached_static_shims() { - let files = render(&KotlinConfig::default()); - let bridge = file(&files, "JniBridge.kt"); - assert!(bridge.contains( - "fun bus_Subscriber_on_event(_impl: Subscriber, event: ByteArray, note: ByteArray): Int = _impl.onEvent(decodeBuffer(event) { _r -> unpackEvent(_r) }, decodeUtf8(note)).value" - )); - let c = file(&files, "event_bus_jni.c"); - assert!(c.contains( - "GetStaticMethodID(env, Jni_bridge, \"bus_Subscriber_on_event\", \"(Levent_bus/Subscriber;[B[B)I\");" - )); - assert!(c.contains("pthread_key_create(&Jni_env_key, Jni_detach);")); - assert!(c.contains("AttachCurrentThreadAsDaemon")); - assert!(c.contains("event_bus_error_set(out_err, -4,")); - assert!(c.contains( - "static const event_bus_bus_Subscriber_vtable Jni_bus_Subscriber_vtable = {Jni_bus_Subscriber_on_event, Jni_release_callback};" - )); -} - -#[test] -fn iterators_stream_through_native_iterator() { - let files = render(&KotlinConfig::default()); - let bus = file(&files, "Bus.kt"); - assert!(bus.contains( - "fun keys(): NativeIterator = handle.borrow { _self ->\n NativeIterator(JniBridge.bus_Store_keys(_self), JniBridge::bus_Store_KeysIterator_destroy) { _it ->" - )); - assert!(bus.contains( - "if (JniBridge.bus_Store_CountsIterator_next(_it, _slot)) _slot[0] else NativeIterator.DONE" - )); - let c = file(&files, "event_bus_jni.c"); - assert!(c.contains("(*env)->SetIntArrayRegion(env, out, 0, 1, &value);")); -} - -#[test] -fn package_bundles_prebuilt_binaries() { - let api = api(); - let model = BindingModel::build(&api); - let mut binaries = BinarySet::new("event_bus"); - binaries.binaries.push(NativeBinary { - platform: Platform::AndroidArm64, - source: Utf8PathBuf::from("/prebuilt/android-arm64/libevent_bus.so"), - }); - binaries.binaries.push(NativeBinary { - platform: Platform::MacosArm64, - source: Utf8PathBuf::from("/prebuilt/darwin-arm64/libevent_bus.dylib"), - }); - binaries.binaries.push(NativeBinary { - platform: Platform::Wasm32, - source: Utf8PathBuf::from("/prebuilt/wasm32/event_bus.wasm"), - }); - let ctx = PackageContext { - binaries: &binaries, - input_basename: Some("bus.yml"), - }; - let files = KotlinGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &KotlinConfig::default(), - ) - .expect("kotlin supports packaging"); - let paths: Vec = files - .iter() - .map(|f| f.path.as_str().replace('\\', "/")) - .collect(); - for expected in [ - "out/kotlin/README.md", - "out/kotlin/src/main/jniLibs/arm64-v8a/libevent_bus.so", - "out/kotlin/src/main/resources/natives/darwin-arm64/libevent_bus.dylib", - ] { - assert!(paths.iter().any(|p| p == expected), "{expected} missing"); - } - assert!(!paths.iter().any(|p| p.contains("wasm"))); - let readme = files - .iter() - .find(|f| f.path.as_str().ends_with("README.md")) - .map(|f| match &f.content { - FileContent::Text(t) => t.clone(), - FileContent::Copy(_) => panic!("README is text"), - }) - .expect("README"); - assert!(readme.contains("- `src/main/jniLibs/arm64-v8a/libevent_bus.so`")); - assert!(readme.contains("-DEVENT_BUS_LIBRARY_DIR=")); -} diff --git a/crates/weaveffi-gen/src/targets/node/mod.rs b/crates/weaveffi-gen/src/targets/node/mod.rs deleted file mode 100644 index f6993255..00000000 --- a/crates/weaveffi-gen/src/targets/node/mod.rs +++ /dev/null @@ -1,243 +0,0 @@ -//! Node.js binding generator. -//! -//! Emits an npm package that builds with `npm install`: the ES module -//! (`index.js`, `index.d.ts`, `runtime.js`) shared with the WebAssembly -//! target through the shared JavaScript layer, and its transport, a small N-API -//! addon (`{library}_node.c`, compiled by node-gyp from `binding.gyp` -//! against the bundled copy of the C header) that calls the library's C ABI. -//! Implements [`LanguageBackend`]; the shared driver bridges it into the -//! generator pipeline. - -mod addon; -mod package; - -use camino::Utf8Path; -use serde::{Deserialize, Serialize}; -use weaveffi_model::model::{BindingModel, CallbackInterfaceBinding, ModuleBinding}; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::codegen::CodeWriter; -use crate::package::{PackageContext, PackagedFile}; -use crate::targets::c::render_c_header_from_model; -use crate::targets::js; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; - -use self::addon::render_addon_c; -use self::package::{ - node_platform_tokens, render_binding_gyp, render_package_json, render_platform_package_json, - render_readme, NodeNames, -}; - -/// The fixed transport section of `index.js`: loading the addon. -const LOADER_JS: &str = include_str!("runtime/loader.js"); - -/// Per-target configuration for [`NodeGenerator`]. -#[derive(Debug, Clone, Default, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct NodeConfig { - /// npm package name (default: the package identity's name). - pub package_name: Option, - /// Basename of the IDL the CLI was invoked with. - #[serde(skip)] - pub input_basename: Option, -} - -impl NodeConfig { - /// The input IDL basename embedded in generated file headers. - fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } - - fn names<'a>(&self, identity: &'a Identity) -> NodeNames<'a> { - NodeNames { - identity, - package: self - .package_name - .clone() - .unwrap_or_else(|| identity.name.clone()), - addon: format!("{}_node", identity.library), - header: format!("{}.h", identity.library), - } - } -} - -/// Node.js backend: an ES module over an N-API addon. -pub struct NodeGenerator; - -/// The generated `index.js`. -fn render_index(model: &BindingModel, names: &NodeNames<'_>, input_basename: &str) -> String { - let mut out = render_prelude(CommentStyle::DoubleSlash, input_basename); - out.push_str(&js::render_imports(names.identity)); - out.push('\n'); - out.push_str(&LOADER_JS.replace("{{ADDON}}", &names.addon)); - out.push('\n'); - out.push_str(&js::render_verify(model, names.identity)); - out.push('\n'); - out.push_str(&js::render_api(model)); - out.push_str(&render_trailer(CommentStyle::DoubleSlash, "index.js")); - out -} - -/// Every file of the generated package, as `(name, contents)` pairs. -fn render_files( - model: &BindingModel, - names: &NodeNames<'_>, - platforms: &[String], - default_dir: &str, - input_basename: &str, -) -> Vec<(String, String)> { - let modules: Vec = model.roots().map(js::names::module_name).collect(); - vec![ - ( - "index.js".into(), - render_index(model, names, input_basename), - ), - ( - "index.d.ts".into(), - js::render_dts(model, names.identity, input_basename, ""), - ), - ( - "runtime.js".into(), - js::render_runtime(names.identity, input_basename), - ), - ( - format!("{}.c", names.addon), - render_addon_c( - model, - &names.header, - input_basename, - &format!("{}.c", names.addon), - ), - ), - ( - names.header.clone(), - render_c_header_from_model(model, input_basename, &names.header), - ), - ( - "binding.gyp".into(), - render_binding_gyp(names, default_dir, input_basename), - ), - ( - "package.json".into(), - render_package_json(names, platforms, input_basename), - ), - ( - "README.md".into(), - render_readme(names, &modules, platforms, input_basename), - ), - ] -} - -impl LanguageBackend for NodeGenerator { - type Config = NodeConfig; - - fn name(&self) -> &'static str { - "node" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - /// The TypeScript `interface` a consumer implements for one callback - /// interface, as `index.d.ts` declares it. - fn render_callback_interface( - &self, - out: &mut String, - _module: &ModuleBinding, - cb: &CallbackInterfaceBinding, - _config: &Self::Config, - ) { - let mut w = CodeWriter::two_space(); - js::render_callback_interface_dts(&mut w, cb); - out.push_str(&w.finish()); - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let dir = out_dir.join("node"); - let names = config.names(api.identity()); - render_files( - model, - &names, - &[], - "path.resolve('.')", - config.input_basename(), - ) - .into_iter() - .map(|(name, contents)| OutputFile::new(dir.join(name), contents)) - .collect() - } - - /// The packaged layout follows the esbuild convention: the main package - /// lists one `{package}-{os}-{cpu}` package per platform in - /// `optionalDependencies`, each gated by npm `os`/`cpu` and carrying the - /// prebuilt library, and the addon links the installed one. - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let dir = out_dir.join("node"); - let names = config.names(api.identity()); - let platforms: Vec<(crate::platform::Platform, (&str, &str), String)> = ctx - .binaries - .platforms() - .filter_map(|p| { - let tokens = node_platform_tokens(p)?; - Some(( - p, - tokens, - format!("{}-{}-{}", names.package, tokens.0, tokens.1), - )) - }) - .collect(); - let platform_names: Vec = platforms.iter().map(|(_, _, n)| n.clone()).collect(); - let default_dir = format!( - "path.dirname(require.resolve('{}-' + process.platform + '-' + process.arch + '/package.json'))", - names.package - ); - let mut files: Vec = render_files( - model, - &names, - &platform_names, - &default_dir, - config.input_basename(), - ) - .into_iter() - .map(|(name, contents)| PackagedFile::text(dir.join(name), contents)) - .collect(); - for (platform, tokens, name) in &platforms { - let pkg_dir = dir.join("npm").join(name); - files.push(PackagedFile::text( - pkg_dir.join("package.json"), - render_platform_package_json( - name, - &names.identity.library, - &names.identity.version, - *tokens, - ), - )); - let binary = ctx.binaries.get(*platform).expect("platform has a binary"); - files.push(PackagedFile::copy( - pkg_dir.join(platform.lib_filename(&names.identity.library)), - binary.source.clone(), - )); - } - Some(files) - } -} - -#[cfg(test)] -mod tests; diff --git a/crates/weaveffi-gen/src/targets/node/runtime/loader.js b/crates/weaveffi-gen/src/targets/node/runtime/loader.js deleted file mode 100644 index c322bf6f..00000000 --- a/crates/weaveffi-gen/src/targets/node/runtime/loader.js +++ /dev/null @@ -1,26 +0,0 @@ -import { createRequire } from 'node:module'; - -// The native entry points: the N-API addon `npm install` builds from -// {{ADDON}}.c (with node-gyp), or a prebuilt copy next to this file. -function $loadAddon(name) { - const require = createRequire(import.meta.url); - const tried = []; - for (const dir of ['./build/Release/', './build/Debug/', './']) { - try { - return require(dir + name); - } catch (e) { - if (e.code !== 'MODULE_NOT_FOUND') throw e; - tried.push(dir + name); - } - } - throw new Error( - `the native addon ${name} was not found (tried ${tried.join(', ')}); ` + - 'build it with `npm install` or `node-gyp rebuild` in this package', - ); -} - -const $raw = $loadAddon('{{ADDON}}.node'); -$raw.$setup($Fault); - -// Object tokens in value buffers are already bigint handles here. -const $token = (t) => t; diff --git a/crates/weaveffi-gen/src/targets/node/tests.rs b/crates/weaveffi-gen/src/targets/node/tests.rs deleted file mode 100644 index a1bd3fcd..00000000 --- a/crates/weaveffi-gen/src/targets/node/tests.rs +++ /dev/null @@ -1,180 +0,0 @@ -use super::*; -use crate::platform::{BinarySet, Platform}; -use crate::targets::js::test_api; - -fn files(name: &str, config: &NodeConfig) -> Vec { - let api = test_api(name); - let model = BindingModel::build(&api); - NodeGenerator.files(&api, &model, Utf8Path::new("out"), config) -} - -fn file(files: &[OutputFile], name: &str) -> String { - files - .iter() - .find(|f| f.path.file_name() == Some(name)) - .unwrap_or_else(|| panic!("no {name}")) - .contents - .clone() -} - -fn has(src: &str, needle: &str) { - assert!(src.contains(needle), "missing `{needle}` in:\n{src}"); -} - -#[test] -fn the_package_is_named_after_the_identity() { - let out = files("acme-kv", &NodeConfig::default()); - let names: Vec<&str> = out.iter().filter_map(|f| f.path.file_name()).collect(); - assert_eq!( - names, - [ - "index.js", - "index.d.ts", - "runtime.js", - "acme_kv_node.c", - "acme_kv.h", - "binding.gyp", - "package.json", - "README.md" - ] - ); - has(&file(&out, "package.json"), "\"name\": \"acme-kv\","); - has( - &file(&out, "binding.gyp"), - "\"target_name\": \"acme_kv_node\",", - ); - has(&file(&out, "binding.gyp"), "\"-lacme_kv\""); - has( - &file(&out, "binding.gyp"), - "process.env.ACME_KV_LIBRARY || process.env.npm_config_acme_kv_library", - ); - has(&file(&out, "binding.gyp"), "-Wl,-rpath,@loader_path"); - has( - &file(&out, "index.js"), - "const $raw = $loadAddon('acme_kv_node.node');", - ); - for f in &out { - if f.path.file_name() != Some("package.json") { - assert!(!f.contents.contains("weaveffi_"), "{} is branded", f.path); - } - } - - let renamed = files( - "acme-kv", - &NodeConfig { - package_name: Some("@acme/kv".into()), - ..NodeConfig::default() - }, - ); - has(&file(&renamed, "package.json"), "\"name\": \"@acme/kv\","); -} - -#[test] -fn index_checks_the_contract_at_load() { - let api = test_api("kv"); - let model = BindingModel::build(&api); - let checksum = model.modules[0].checksum.unwrap(); - let js = file(&files("kv", &NodeConfig::default()), "index.js"); - has(&js, "$raw.$setup($Fault);"); - has( - &js, - &format!("$verify($raw, 'kv', 'kv', 3, [['kv', 0x{checksum:016x}n]]);"), - ); - has( - &js, - "export { KvError, CancelledError } from './runtime.js';", - ); -} - -#[test] -fn addon_exports_every_symbol_with_the_raw_convention() { - let c = file(&files("kv", &NodeConfig::default()), "kv_node.c"); - assert!(!c.contains("{{"), "unsubstituted placeholder"); - has(&c, "#include \"kv.h\""); - has(&c, "typedef kv_error js_error;"); - for symbol in [ - "kv_kv_checksum", - "kv_kv_Store_new", - "kv_kv_Store_clone", - "kv_kv_Store_destroy", - "kv_kv_Store_ScanIterator_next", - "kv_kv_Store_ScanIterator_destroy", - "kv_kv_Store_wait", - "kv_kv_stats_count", - ] { - has( - &c, - &format!("{{\"{symbol}\", NULL, nx_{symbol}, NULL, NULL, NULL, napi_default, NULL}},"), - ); - } - // Strings cross as (ptr, len); a returned string is freed after copying. - has( - &c, - "kv_kv_Store_get((const kv_kv_Store*)self_h, JS_STR_PTR(a0), a0.len, &out_len, &err);", - ); - has(&c, "ret = js_take_bytes(env, r, out_len);"); - // A cancellable launcher takes the token handle after its inputs. - has( - &c, - "if (!js_arg_handle(env, argv[2], &token, true)) goto done;", - ); - has(&c, "kv_kv_Store_wait((const kv_kv_Store*)self_h, a0, (kv_cancel_token*)token, done_kv_kv_Store_wait, a);"); - has( - &c, - "static void done_kv_kv_Store_wait(void* context, kv_error* err, int64_t result) {", - ); - has( - &c, - "js_async* a = js_async_begin(env, JS_R_HANDLE, \"kv_kv_Store_fetch\", &ret);", - ); - // Every declaration precedes the first read that can fail. - has( - &c, - " js_cb* a0 = NULL;\n int32_t a1 = 0;\n if (!js_arg_cb(", - ); -} - -#[test] -fn callback_vtables_hop_to_the_javascript_thread() { - let c = file(&files("kv", &NodeConfig::default()), "kv_node.c"); - has(&c, "static bool tramp_kv_kv_Listener_on_message(void* ctx, const uint8_t* p0, size_t p1, uint64_t p2, kv_error* out_err) {"); - has(&c, " if (js_cb_on_js_thread(cb)) {"); - has(&c, " js_cb_hop(&req);"); - has(&c, "argv[0] = js_new_str(env, f->p0, f->p1);"); - has(&c, "argv[1] = js_new_u64(env, f->p2);"); - has(&c, "argv[2] = js_new_handle(env, f->p3);"); - has(&c, "static const kv_kv_Listener_vtable vtable_kv_kv_Listener = {tramp_kv_kv_Listener_on_message, tramp_kv_kv_Listener_on_bundle, js_cb_free};"); -} - -#[test] -fn package_ships_one_npm_package_per_node_platform() { - let api = test_api("kv"); - let model = BindingModel::build(&api); - let mut binaries = BinarySet::new("kv"); - binaries.insert(Platform::MacosArm64, "lib/libkv.dylib"); - binaries.insert(Platform::Wasm32, "lib/kv.wasm"); - let ctx = PackageContext { - binaries: &binaries, - input_basename: None, - }; - let files = NodeGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &NodeConfig::default(), - ) - .expect("node packages"); - let paths: Vec = files.iter().map(|f| f.path.to_string()).collect(); - assert!( - paths.contains(&"out/node/npm/kv-darwin-arm64/package.json".into()), - "{paths:?}" - ); - assert!( - paths.contains(&"out/node/npm/kv-darwin-arm64/libkv.dylib".into()), - "{paths:?}" - ); - assert!(!paths.iter().any(|p| p.contains("wasm")), "{paths:?}"); - assert_eq!(node_platform_tokens(Platform::AndroidArm64), None); -} diff --git a/crates/weaveffi-gen/src/targets/python/codec.rs b/crates/weaveffi-gen/src/targets/python/codec.rs deleted file mode 100644 index a529e5ec..00000000 --- a/crates/weaveffi-gen/src/targets/python/codec.rs +++ /dev/null @@ -1,272 +0,0 @@ -//! Value-buffer codec emitters: the inline read expressions and write -//! statements for any wire shape, plus the per-record and per-rich-enum -//! codec functions (`_write_X`, `_read_X`). -//! -//! Every dispatch here goes through [`Ty::wire`], and primitives dispatch -//! on [`Prim::snake`] (`write_i32`, `read_string`), so this module never -//! re-derives the wire folds. - -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use weaveffi_model::model::Ty; -use weaveffi_model::model::{EnumBinding, StructBinding}; -use weaveffi_model::model::{Prim, WireType}; - -use crate::targets::python::types::{py_field, py_variant}; - -/// `_write_{Name}`, the statement-level writer for a record or rich enum. -/// `name` may be a qualified IR reference; the emitted function uses the -/// bare local class name. -pub(crate) fn py_write_fn_name(name: &str) -> String { - format!("_write_{}", local_type_name(name)) -} - -/// `_read_{Name}`, the reader consuming one encoded value from a `_Reader`. -pub(crate) fn py_read_fn_name(name: &str) -> String { - format!("_read_{}", local_type_name(name)) -} - -/// The `struct` format character and byte width of a fixed-width numeric -/// primitive, for the packed list fast path. -fn numeric_format(p: Prim) -> Option<(char, usize)> { - Some(match p { - Prim::I8 => ('b', 1), - Prim::U8 => ('B', 1), - Prim::I16 => ('h', 2), - Prim::U16 => ('H', 2), - Prim::I32 => ('i', 4), - Prim::U32 => ('I', 4), - Prim::I64 => ('q', 8), - Prim::U64 => ('Q', 8), - Prim::F32 => ('f', 4), - Prim::F64 => ('d', 8), - Prim::Bool | Prim::String | Prim::Bytes => return None, - }) -} - -/// The packed fast path for a list of one numeric primitive, if `inner` is -/// one. -fn numeric_list(inner: &Ty) -> Option<(char, usize)> { - match inner.wire() { - WireType::Prim(p) => numeric_format(p), - _ => None, - } -} - -/// The Python expression reading one `ty` value from the reader `_r`, -/// following the value-buffer wire format. `depth` uniquifies comprehension -/// loop variables when composites nest. -/// -/// Expressions (rather than statements) compose: Python evaluates a -/// comprehension's `range(_r.read_count())` before its body and a -/// conditional expression's test before its arms, which matches the wire -/// order exactly. -pub(crate) fn py_read_expr(ty: &Ty, depth: usize) -> String { - match ty.wire() { - WireType::Prim(p) => format!("_r.read_{}()", p.snake()), - // An object token carries one strong reference: adopt it into a new - // wrapper. A decode that fails later in the buffer drops the - // wrapper, and its finalizer releases the adopted reference. - WireType::Object(name) => { - format!("{}._adopt(_r.read_object())", local_type_name(name)) - } - WireType::Enum(name) => format!("{}(_r.read_i32())", local_type_name(name)), - WireType::User(name) => format!("{}(_r)", py_read_fn_name(name)), - WireType::Optional(inner) => format!( - "({} if _r.read_flag() else None)", - py_read_expr(inner, depth) - ), - WireType::List(inner) => match numeric_list(inner) { - Some((code, size)) => format!("_r.read_numbers(\"{code}\", {size})"), - None => format!( - "[{} for _i{depth} in range(_r.read_count())]", - py_read_expr(inner, depth + 1) - ), - }, - WireType::Map(k, v) => format!( - "dict(({}, {}) for _i{depth} in range(_r.read_count()))", - py_read_expr(k, depth + 1), - py_read_expr(v, depth + 1) - ), - } -} - -/// Append the statements writing `expr` (one `ty` value) into the `_Writer` -/// named `writer`, following the value-buffer wire format. `next` numbers -/// loop variables, so every loop in one function body gets its own. -pub(crate) fn py_write_stmts( - w: &mut CodeWriter, - writer: &str, - expr: &str, - ty: &Ty, - next: &mut usize, -) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("{writer}.write_{}({expr})", p.snake())); - } - // The token written must be a reference the reader can own: the - // writer clones the wrapper's pointer when the buffer is finished, - // never writing the pointer the wrapper still holds. - WireType::Object(name) => { - w.line(format!( - "{writer}.write_object({expr}, {})", - local_type_name(name) - )); - } - // IntEnum members are ints, so the discriminant packs directly. - WireType::Enum(_) => { - w.line(format!("{writer}.write_i32({expr})")); - } - WireType::User(name) => { - w.line(format!("{}({writer}, {expr})", py_write_fn_name(name))); - } - WireType::Optional(inner) => { - w.line(format!("if {expr} is None:")); - w.scope(|w| { - w.line(format!("{writer}.write_flag(False)")); - }); - w.line("else:"); - w.scope(|w| { - w.line(format!("{writer}.write_flag(True)")); - py_write_stmts(w, writer, expr, inner, next); - }); - } - WireType::List(inner) => match numeric_list(inner) { - Some((code, _)) => { - w.line(format!("{writer}.write_numbers(\"{code}\", {expr})")); - } - None => { - let i = *next; - *next += 1; - w.line(format!("{writer}.write_count(len({expr}))")); - w.line(format!("for _e{i} in {expr}:")); - w.scope(|w| { - py_write_stmts(w, writer, &format!("_e{i}"), inner, next); - }); - } - }, - WireType::Map(k, v) => { - let i = *next; - *next += 1; - w.line(format!("{writer}.write_count(len({expr}))")); - w.line(format!("for _k{i}, _v{i} in {expr}.items():")); - w.scope(|w| { - py_write_stmts(w, writer, &format!("_k{i}"), k, next); - py_write_stmts(w, writer, &format!("_v{i}"), v, next); - }); - } - } -} - -/// The expression decoding a value-buffer `bytes` expression `data` into -/// one `ty` value, rejecting trailing data. -pub(crate) fn py_decode_expr(data: &str, ty: &Ty) -> String { - match ty.wire() { - WireType::User(name) => format!("_decode({data}, {})", py_read_fn_name(name)), - _ => format!("_decode({data}, lambda _r: {})", py_read_expr(ty, 0)), - } -} - -/// Append a record's buffer codec functions: the statement writer and the -/// reader. -pub(crate) fn render_record_codecs(w: &mut CodeWriter, s: &StructBinding) { - let name = &s.name; - w.blank().blank(); - w.line(format!( - "def {}(_w: _Writer, value: \"{name}\") -> None:", - py_write_fn_name(name) - )); - w.scope(|w| { - if s.fields.is_empty() { - w.line("pass"); - } - let mut next = 0; - for f in &s.fields { - let expr = format!("value.{}", py_field(&f.name)); - py_write_stmts(w, "_w", &expr, &f.ty, &mut next); - } - }); - - w.blank().blank(); - w.line(format!( - "def {}(_r: _Reader) -> \"{name}\":", - py_read_fn_name(name) - )); - w.scope(|w| { - if s.fields.is_empty() { - w.line(format!("return {name}()")); - return; - } - // Keyword arguments evaluate left to right, matching the wire order - // of the record's fields. - w.line(format!("return {name}(")); - w.scope(|w| { - for f in &s.fields { - w.line(format!("{}={},", py_field(&f.name), py_read_expr(&f.ty, 0))); - } - }); - w.line(")"); - }); -} - -/// Append a rich enum's buffer codec functions: the statement writer and -/// the reader. The wire shape is an `i32` tag followed by the active -/// variant's fields. -pub(crate) fn render_rich_enum_codecs(w: &mut CodeWriter, e: &EnumBinding) { - let name = &e.name; - w.blank().blank(); - w.line(format!( - "def {}(_w: _Writer, value: \"{name}\") -> None:", - py_write_fn_name(name) - )); - w.scope(|w| { - let mut next = 0; - for v in &e.variants { - let class = format!("{name}{}", py_variant(&v.name)); - w.line(format!("if isinstance(value, {class}):")); - w.scope(|w| { - w.line(format!("_w.write_i32({})", v.value)); - for f in &v.fields { - let expr = format!("value.{}", py_field(&f.name)); - py_write_stmts(w, "_w", &expr, &f.ty, &mut next); - } - w.line("return"); - }); - } - w.line(format!( - "raise TypeError(f\"expected a {name} variant, got {{type(value).__name__}}\")" - )); - }); - - w.blank().blank(); - w.line(format!( - "def {}(_r: _Reader) -> \"{name}\":", - py_read_fn_name(name) - )); - w.scope(|w| { - w.line("_tag = _r.read_i32()"); - for v in &e.variants { - let class = format!("{name}{}", py_variant(&v.name)); - w.line(format!("if _tag == {}:", v.value)); - w.scope(|w| { - if v.fields.is_empty() { - w.line(format!("return {class}()")); - return; - } - // Keyword arguments evaluate left to right, matching the - // wire order of the variant's fields. - w.line(format!("return {class}(")); - w.scope(|w| { - for f in &v.fields { - w.line(format!("{}={},", py_field(&f.name), py_read_expr(&f.ty, 0))); - } - }); - w.line(")"); - }); - } - w.line(format!( - "raise _malformed(f\"unknown {name} tag {{_tag}}\")" - )); - }); -} diff --git a/crates/weaveffi-gen/src/targets/python/docs.rs b/crates/weaveffi-gen/src/targets/python/docs.rs deleted file mode 100644 index aac36ed9..00000000 --- a/crates/weaveffi-gen/src/targets/python/docs.rs +++ /dev/null @@ -1,140 +0,0 @@ -//! Doc emission for the generated Python: `#` line comments, triple-quoted -//! docstrings, and the NumPy-style function docstring with `Parameters` and -//! `Raises` sections. - -use crate::codegen::common::{emit_doc as common_emit_doc, DocCommentStyle}; -use weaveffi_model::model::ParamBinding; - -use crate::targets::python::types::py_name; - -/// Emits a Python `# ...` line comment at `indent`. Used above C ABI binding -/// declarations (`attach_function`-style binds) where docstrings can't live. -pub(crate) fn emit_doc(out: &mut String, doc: &Option, indent: &str) { - common_emit_doc(out, doc, indent, DocCommentStyle::Hash); -} - -/// Emits a Python triple-quoted `"""..."""` docstring as the first statement -/// of a class or function body, at the given `indent`. -pub(crate) fn emit_docstring(out: &mut String, doc: &Option, indent: &str) { - let Some(doc) = doc else { - return; - }; - let doc = doc.trim(); - if doc.is_empty() { - return; - } - if doc.contains('\n') { - out.push_str(indent); - out.push_str("\"\"\"\n"); - for line in doc.lines() { - if line.is_empty() { - out.push('\n'); - } else { - out.push_str(indent); - out.push_str(line); - out.push('\n'); - } - } - out.push_str(indent); - out.push_str("\"\"\"\n"); - } else { - out.push_str(indent); - out.push_str("\"\"\""); - out.push_str(doc); - out.push_str("\"\"\"\n"); - } -} - -/// Emits a NumPy/Google-style docstring with a `Parameters` section listing -/// each parameter that has a `doc:` value, and a `Raises` section naming the -/// domain error type when `raises` is set (throwing callables only). Skips -/// entirely when there is nothing to document. -pub(crate) fn emit_fn_docstring( - out: &mut String, - doc: &Option, - params: &[ParamBinding], - indent: &str, - raises: Option<&str>, -) { - let trimmed_doc = doc.as_ref().map(|d| d.trim()).filter(|d| !d.is_empty()); - let documented_params: Vec<&ParamBinding> = params - .iter() - .filter(|p| { - p.doc - .as_ref() - .map(|d| !d.trim().is_empty()) - .unwrap_or(false) - }) - .collect(); - if documented_params.is_empty() && raises.is_none() { - // Nothing beyond the summary: an ordinary docstring. - emit_docstring(out, &trimmed_doc.map(str::to_string), indent); - return; - } - out.push_str(indent); - out.push_str("\"\"\""); - let mut has_content = false; - if let Some(d) = trimmed_doc { - if d.contains('\n') { - out.push('\n'); - for line in d.lines() { - if line.is_empty() { - out.push('\n'); - } else { - out.push_str(indent); - out.push_str(line); - out.push('\n'); - } - } - } else { - out.push_str(d); - out.push('\n'); - } - has_content = true; - } else { - out.push('\n'); - } - if !documented_params.is_empty() { - if has_content { - out.push('\n'); - } - out.push_str(indent); - out.push_str("Parameters\n"); - out.push_str(indent); - out.push_str("----------\n"); - for p in documented_params { - let pdoc = p.doc.as_ref().unwrap().trim(); - let mut lines = pdoc.lines(); - let first = lines.next().unwrap_or(""); - out.push_str(indent); - out.push_str(&format!("{} : {}\n", py_name(&p.name), first)); - for line in lines { - out.push_str(indent); - if line.is_empty() { - out.push('\n'); - } else { - out.push_str(" "); - out.push_str(line); - out.push('\n'); - } - } - } - has_content = true; - } - if let Some(domain) = raises { - if has_content { - out.push('\n'); - } - out.push_str(indent); - out.push_str("Raises\n"); - out.push_str(indent); - out.push_str("------\n"); - out.push_str(indent); - out.push_str(domain); - out.push('\n'); - out.push_str(indent); - out.push_str(" If the call reports one of the domain's error codes.\n"); - } - out.push_str(indent); - out.push_str("\"\"\"\n"); -} diff --git a/crates/weaveffi-gen/src/targets/python/entities.rs b/crates/weaveffi-gen/src/targets/python/entities.rs deleted file mode 100644 index 6851f950..00000000 --- a/crates/weaveffi-gen/src/targets/python/entities.rs +++ /dev/null @@ -1,460 +0,0 @@ -//! Entity rendering: error domains, plain and rich enums, record -//! dataclasses, and interface wrapper classes. - -use crate::codegen::CodeWriter; -use heck::{ToShoutySnakeCase, ToSnakeCase}; -use weaveffi_model::model::{ - BindingModel, CallShape, EnumBinding, ErrorBinding, ErrorCodeBinding, FieldBinding, FnBinding, - InterfaceBinding, ModuleBinding, StructBinding, -}; - -use crate::targets::python::calls::{render_bindings, render_callable, FnScope}; -use crate::targets::python::codec::{py_read_expr, render_record_codecs, render_rich_enum_codecs}; -use crate::targets::python::docs::emit_docstring; -use crate::targets::python::types::{ - py_binding_name, py_field, py_field_hint, py_str_literal, py_variant, -}; -use crate::targets::python::Gen; - -// ── Errors ── - -/// The root exception class every error the bindings raise derives from: -/// `Error` (so consumers write `except kvstore.Error`), unless the API -/// declares a type or error code with that name, in which case -/// `{PascalName}Error`, then `{PascalName}BaseError`. -pub(crate) fn root_error_name(model: &BindingModel, pascal_name: &str) -> String { - let taken = |name: &str| { - model.modules.iter().any(|m| { - m.enums.iter().any(|e| e.name == name) - || m.structs.iter().any(|s| s.name == name) - || m.interfaces.iter().any(|i| i.name == name) - || m.callback_interfaces.iter().any(|c| c.name == name) - || m.error.as_ref().is_some_and(|e| { - e.type_name == name - || e.codes.iter().any(|c| py_code_class_name(&c.name) == name) - }) - }) - }; - [ - "Error".to_string(), - format!("{pascal_name}Error"), - format!("{pascal_name}BaseError"), - ] - .into_iter() - .find(|n| !taken(n)) - .unwrap_or_else(|| format!("{pascal_name}RootError")) -} - -/// The snake_case stem shared by a domain's private helper names. -fn py_error_stem(eb: &ErrorBinding) -> String { - eb.type_name.to_snake_case() -} - -/// `_{stem}_from`: builds the domain exception matching an ABI code. -pub(crate) fn py_factory_name(eb: &ErrorBinding) -> String { - format!("_{}_from", py_error_stem(eb)) -} - -/// `_check_{stem}`: raises the domain exception for a non-zero out-err slot. -fn py_domain_checker_name(eb: &ErrorBinding) -> String { - format!("_check_{}", py_error_stem(eb)) -} - -/// The error-check call a callable's out-err slot goes through: the module -/// domain's typed checker when the callable throws, the generic -/// `_check_error` (the root exception: panics, marshalling failures, and -/// failed callbacks only) otherwise. -pub(crate) fn py_checker_name(f: &FnBinding, error: Option<&ErrorBinding>) -> String { - match error { - Some(eb) if f.throws => py_domain_checker_name(eb), - _ => "_check_error".to_string(), - } -} - -/// The Python class name for one error code: plain PascalCase with no forced -/// suffix (`KeyNotFound`, not `KeyNotFoundError`). Each class is also -/// attached to its domain class (`KvError.KeyNotFound`), which stays -/// unambiguous even if two domains declare codes with the same name. -pub(crate) fn py_code_class_name(name: &str) -> String { - weaveffi_model::errors::pascal(name) -} - -/// `_{stem}_payload_{code}`: decodes one code's payload fields onto an -/// exception instance. -fn py_payload_decoder_name(eb: &ErrorBinding, code: &ErrorCodeBinding) -> String { - format!( - "_{}_payload_{}", - py_error_stem(eb), - code.name.to_snake_case() - ) -} - -/// Render one module's declared error domain: a base exception named after -/// the domain (subclassing the root exception), one exception subclass per -/// code carrying its stable `CODE` and default message, the code-to-class -/// table, per-code payload decoders, and the factory/checker helpers -/// throwing wrappers route their out-err slots through. Each code class is -/// also attached to the domain class, so consumers can catch -/// `KvError.KeyNotFound`. -pub(crate) fn render_error( - out: &mut String, - g: &Gen<'_>, - module: &ModuleBinding, - eb: &ErrorBinding, -) { - let domain = &eb.type_name; - let root = g.root_error; - let factory = py_factory_name(eb); - let checker = py_domain_checker_name(eb); - let table = format!("_{}_CODES", eb.type_name.to_shouty_snake_case()); - let payloads = format!("_{}_PAYLOADS", eb.type_name.to_shouty_snake_case()); - let has_payloads = eb.codes.iter().any(|c| !c.fields.is_empty()); - - let mut w = CodeWriter::four_space(); - w.blank().blank(); - w.line(format!("class {domain}({root}):")); - w.scope(|w| { - w.line(format!( - "\"\"\"Base exception for the `{}` module's error domain.\"\"\"", - module.dot_path - )); - // The per-code classes, attached below once they exist. - w.blank(); - for c in &eb.codes { - let class = py_code_class_name(&c.name); - w.line(format!("{class}: \"Type[{class}]\"")); - } - }); - - for c in &eb.codes { - let class = py_code_class_name(&c.name); - let message = py_str_literal(&c.message); - w.blank().blank(); - w.line(format!("class {class}({domain}):")); - w.indent(); - let mut doc = String::new(); - emit_docstring(&mut doc, &c.doc, &w.indent_str()); - if doc.is_empty() { - emit_docstring(&mut doc, &Some(c.message.clone()), &w.indent_str()); - } - w.raw(doc); - w.blank(); - w.line(format!("CODE = {}", c.value)); - w.blank(); - w.line(format!( - "def __init__(self, message: str = \"{message}\") -> None:" - )); - w.scope(|w| { - w.line(format!("super().__init__({}, message)", c.value)); - }); - w.dedent(); - } - - // Scoped aliases: `except KvError.KeyNotFound` stays unambiguous even if - // another domain declares a code with the same name. - w.blank().blank(); - for c in &eb.codes { - let class = py_code_class_name(&c.name); - w.line(format!("{domain}.{class} = {class}")); - } - - w.blank().blank(); - w.line(format!("{table}: Dict[int, Any] = {{")); - w.scope(|w| { - for c in &eb.codes { - let class = py_code_class_name(&c.name); - w.line(format!("{}: {class},", c.value)); - } - }); - w.line("}"); - - // Payload decoders: one per code that declares structured fields. Each - // reads the code's fields (in declaration order) from the payload buffer - // and attaches them as attributes on the exception instance. - if has_payloads { - for c in eb.codes.iter().filter(|c| !c.fields.is_empty()) { - let decoder = py_payload_decoder_name(eb, c); - let class = py_code_class_name(&c.name); - w.blank().blank(); - w.line(format!("def {decoder}(_exc: {root}, _r: _Reader) -> None:")); - w.scope(|w| { - w.line(format!( - "\"\"\"Decode the {class} payload fields onto `_exc`.\"\"\"" - )); - for f in &c.fields { - w.line(format!( - "_exc.{} = {}", - py_field(&f.name), - py_read_expr(&f.ty, 0) - )); - } - }); - } - w.blank().blank(); - w.line(format!( - "{payloads}: Dict[int, Callable[[{root}, _Reader], None]] = {{" - )); - w.scope(|w| { - for c in eb.codes.iter().filter(|c| !c.fields.is_empty()) { - w.line(format!("{}: {},", c.value, py_payload_decoder_name(eb, c))); - } - }); - w.line("}"); - } - - w.blank().blank(); - w.line(format!( - "def {factory}(code: int, message: str, payload: bytes = b\"\") -> {root}:" - )); - w.scope(|w| { - w.line(format!( - "\"\"\"Build the {domain} subclass matching `code`, or the plain" - )); - w.line(format!( - "{root} for codes outside the domain (panics, marshalling).\"\"\"" - )); - w.line(format!("cls = {table}.get(code)")); - w.line("if cls is None:"); - w.scope(|w| { - w.line(format!("return {root}(code, message)")); - }); - w.line("exc: Any = cls(message) if message else cls()"); - if has_payloads { - w.line(format!("decoder = {payloads}.get(code)")); - w.line("if decoder is not None and payload:"); - w.scope(|w| { - w.line("r = _Reader(payload)"); - w.line("decoder(exc, r)"); - w.line("r.expect_end()"); - }); - } - w.line("return exc"); - }); - - w.blank().blank(); - w.line(format!("def {checker}(err: _ErrorStruct) -> None:")); - w.scope(|w| { - w.line("if err.code:"); - w.scope(|w| { - w.line(format!("raise {factory}(*_read_error(err))")); - }); - }); - - out.push_str(&w.finish()); -} - -// ── Enums ── - -/// Render one enum: a plain `IntEnum` for a C-style enum, or the dataclass -/// sum-type hierarchy for a rich (algebraic) enum. -pub(crate) fn render_enum(out: &mut String, e: &EnumBinding) { - if e.is_rich() { - render_rich_enum(out, e); - return; - } - let mut w = CodeWriter::four_space(); - w.blank().blank(); - w.line(format!("class {}(IntEnum):", e.name)); - w.indent(); - let mut doc = String::new(); - emit_docstring(&mut doc, &e.doc, " "); - w.raw(doc); - for v in &e.variants { - emit_comment(&mut w, &v.doc); - w.line(format!("{} = {}", py_variant(&v.name), v.value)); - } - out.push_str(&w.finish()); -} - -/// Emit a doc string as `#` comment lines at the writer's indent. -fn emit_comment(w: &mut CodeWriter, doc: &Option) { - if let Some(d) = doc { - for line in d.trim().lines() { - w.line(format!("# {line}").trim_end()); - } - } -} - -/// Render a rich (algebraic) enum as an idiomatic Python sum type: a base -/// class holding the nested `Tag` discriminant enum and a `tag` property, -/// one module-level `@dataclass` subclass per variant carrying its fields, -/// scoped aliases (`Shape.Circle` is `ShapeCircle`), and the buffer codec -/// functions implementing the wire shape `i32 tag + active variant's -/// fields`. Consumers construct variants directly and discriminate with -/// `isinstance` (or the `tag` property). -fn render_rich_enum(out: &mut String, e: &EnumBinding) { - let name = &e.name; - let mut w = CodeWriter::four_space(); - w.blank().blank(); - w.line(format!("class {name}:")); - w.indent(); - let mut doc = String::new(); - emit_docstring(&mut doc, &e.doc, &w.indent_str()); - if !doc.is_empty() { - w.raw(doc); - w.blank(); - } - w.line("class Tag(IntEnum):"); - w.scope(|w| { - for v in &e.variants { - emit_comment(w, &v.doc); - w.line(format!("{} = {}", py_variant(&v.name), v.value)); - } - }); - w.blank(); - // The variant classes, attached below once they exist. - for v in &e.variants { - w.line(format!("{0}: \"Type[{name}{0}]\"", py_variant(&v.name))); - } - w.line(format!("TAG: \"{name}.Tag\"")); - w.blank(); - w.line("@property"); - w.line(format!("def tag(self) -> \"{name}.Tag\":")); - w.scope(|w| { - w.line("\"\"\"The discriminant of this value's active variant.\"\"\""); - w.line("return type(self).TAG"); - }); - w.dedent(); - - for v in &e.variants { - let class = format!("{name}{}", py_variant(&v.name)); - w.blank().blank(); - w.line("@dataclass"); - w.line(format!("class {class}({name}):")); - w.indent(); - let mut doc = String::new(); - emit_docstring(&mut doc, &v.doc, &w.indent_str()); - if !doc.is_empty() { - w.raw(doc); - w.blank(); - } - w.line(format!("TAG = {name}.Tag.{}", py_variant(&v.name))); - if !v.fields.is_empty() { - w.blank(); - render_dataclass_fields(&mut w, &v.fields); - } - w.dedent(); - } - - // Scoped aliases (`Shape.Circle`), assigned once every variant class - // exists. - w.blank().blank(); - for v in &e.variants { - w.line(format!("{name}.{0} = {name}{0}", py_variant(&v.name))); - } - - render_rich_enum_codecs(&mut w, e); - out.push_str(&w.finish()); -} - -// ── Records ── - -/// Render a record as a plain `@dataclass` value class plus its buffer codec -/// functions. Records have no C symbols: construction, equality, and repr -/// all come from the dataclass, and instances cross the ABI serialized in -/// value buffers. -pub(crate) fn render_struct(out: &mut String, s: &StructBinding) { - let mut w = CodeWriter::four_space(); - w.blank().blank(); - w.line("@dataclass"); - w.line(format!("class {}:", s.name)); - w.indent(); - let mut doc = String::new(); - emit_docstring(&mut doc, &s.doc, &w.indent_str()); - let has_doc = !doc.is_empty(); - w.raw(doc); - if s.fields.is_empty() { - if !has_doc { - w.line("pass"); - } - } else { - if has_doc { - w.blank(); - } - render_dataclass_fields(&mut w, &s.fields); - } - w.dedent(); - render_record_codecs(&mut w, s); - out.push_str(&w.finish()); -} - -/// Emit dataclass field lines (`name: hint`), with field docs as leading -/// comments. -fn render_dataclass_fields(w: &mut CodeWriter, fields: &[FieldBinding]) { - for f in fields { - emit_comment(w, &f.doc); - w.line(format!( - "{}: {}", - py_field(&f.name), - py_field_hint(fields, &f.ty) - )); - } -} - -// ── Interfaces ── - -/// Render one interface as a reference-counted object wrapper class -/// (subclassing the runtime's `_Object`), preceded by the module-level -/// bindings of its members, clone, and destroy. The class holds one strong -/// reference and releases it exactly once, from `close()`, the -/// context-manager exit, or the `__del__` backstop; calls lend the pointer so -/// a concurrent `close()` defers the release until they return. `_adopt` -/// wraps a pointer the producer handed over (a return, an async result, an -/// iterator element, or a buffer token); `_clone_ref` mints a second strong -/// reference when the object is written into a value buffer. A constructor -/// named `new` becomes `__init__`; every other constructor becomes a -/// `@classmethod` factory; methods lend `self` as the leading C argument; -/// statics are `@staticmethod`s. -pub(crate) fn render_interface( - out: &mut String, - g: &Gen<'_>, - module: &ModuleBinding, - i: &InterfaceBinding, -) { - let error = module.error.as_ref(); - let name = &i.name; - // Consecutive sync member bindings are packed into one block. - let mut packed = false; - for m in i.constructors.iter().chain(&i.methods).chain(&i.statics) { - render_bindings(out, g, m, error, &format!("{name}.{}", m.name), packed); - packed = matches!(m.shape, CallShape::Sync(_)); - } - let clone = py_binding_name(&i.clone_symbol, g.prefix); - let destroy = py_binding_name(&i.destroy_symbol, g.prefix); - if !packed { - out.push_str("\n\n"); - } - out.push_str(&format!( - "{clone} = _bind(\"{}\", ctypes.c_void_p, ctypes.c_void_p)\n\ - {destroy} = _bind(\"{}\", None, ctypes.c_void_p)\n", - i.clone_symbol, i.destroy_symbol - )); - - out.push_str(&format!("\n\nclass {name}(_Object):\n")); - emit_docstring(out, &i.doc, " "); - out.push_str(&format!( - "\n _destroy = staticmethod({destroy})\n _clone = staticmethod({clone})\n" - )); - - let is_new = |f: &&FnBinding| f.name == "new"; - match i.constructors.iter().find(is_new) { - Some(c) => render_callable(out, g, module, c, FnScope::Init), - None => { - // No canonical constructor: instances only come from factories - // and producer returns. - out.push_str("\n def __init__(self) -> None:"); - out.push_str(&format!( - "\n raise TypeError(\"{name} cannot be instantiated directly\")\n" - )); - } - } - for c in i.constructors.iter().filter(|c| c.name != "new") { - render_callable(out, g, module, c, FnScope::Factory); - } - for m in &i.methods { - render_callable(out, g, module, m, FnScope::Method); - } - for s in &i.statics { - render_callable(out, g, module, s, FnScope::Static); - } -} diff --git a/crates/weaveffi-gen/src/targets/python/mod.rs b/crates/weaveffi-gen/src/targets/python/mod.rs deleted file mode 100644 index 337bd837..00000000 --- a/crates/weaveffi-gen/src/targets/python/mod.rs +++ /dev/null @@ -1,313 +0,0 @@ -//! Python (`ctypes`) binding generator. -//! -//! Emits a pip-installable package of `ctypes` bindings and `.pyi` type -//! stubs over the C ABI (revision 3): distribution `{name}`, import package -//! `{prefix}`, implementation module `{prefix}/{prefix}.py`. Every C -//! prototype is bound once at import time, after the load-time ABI and -//! contract-checksum check. Records and rich enums are dataclasses crossing -//! the boundary as value buffers; interfaces are reference-counted wrapper -//! classes with `close()` and a `__del__` backstop; callback interfaces are -//! abstract base classes backed by one static vtable of `ctypes` -//! trampolines; async functions are coroutines whose cancellation cancels -//! the native call; `iter` returns are lazy Python iterators. - -mod calls; -mod codec; -mod docs; -mod entities; -mod package; -mod runtime; -mod stubs; -#[cfg(test)] -mod tests; -mod types; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use camino::Utf8Path; -use serde::{Deserialize, Serialize}; -use weaveffi_model::model::{BindingModel, ModuleBinding}; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; - -use crate::targets::python::calls::{ - render_bindings, render_callable, render_callback_interface, FnScope, -}; -use crate::targets::python::entities::{ - render_enum, render_error, render_interface, render_struct, root_error_name, -}; -use crate::targets::python::package::{ - render_packaged_readme, render_packaged_setup_py, render_py_typed, render_pyproject_toml, - render_readme, -}; -use crate::targets::python::runtime::{render_runtime, RuntimeNames}; -use crate::targets::python::stubs::render_pyi_module; - -/// Per-target configuration for [`PythonGenerator`]. -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct PythonConfig { - /// The distribution name published to PyPI (default: the package - /// identity's `name`). - pub package_name: Option, - /// The import package (default: the identity's C prefix, which is - /// always a valid Python identifier). Also names the package directory - /// and the implementation module inside it. - pub import_name: Option, - /// When `true` (the default), strip the IR module name prefix from - /// emitted Python function names, so a `contacts` module exports - /// `create_contact` rather than `contacts_create_contact`. Set to - /// `false` to restore module-prefixed names. - pub strip_module_prefix: bool, - /// Basename of the IDL the CLI was invoked with. - #[serde(skip)] - pub input_basename: Option, -} - -impl Default for PythonConfig { - fn default() -> Self { - Self { - package_name: None, - import_name: None, - strip_module_prefix: true, - input_basename: None, - } - } -} - -impl PythonConfig { - /// The input IDL basename embedded in generated file headers. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } -} - -/// The names one generation run uses, resolved from the identity and the -/// configuration. -struct Names { - /// The distribution name. - dist: String, - /// The import package and implementation module name. - import: String, -} - -impl Names { - fn new(id: &Identity, config: &PythonConfig) -> Self { - Self { - dist: config - .package_name - .clone() - .unwrap_or_else(|| id.name.clone()), - import: config - .import_name - .clone() - .unwrap_or_else(|| id.prefix.clone()), - } - } -} - -/// The shared context every rendering function receives. -pub(crate) struct Gen<'a> { - /// The C symbol prefix (stripped from binding names). - pub prefix: &'a str, - /// The root exception class name. - pub root_error: &'a str, - /// Whether free functions drop their module-path prefix. - pub strip_module_prefix: bool, -} - -/// Python backend: emits a pip-installable package of `ctypes` bindings and -/// `.pyi` type stubs over the C ABI exposed by the underlying cdylib. -pub struct PythonGenerator; - -impl PythonGenerator { - /// Render the implementation module: the runtime, then every module's - /// members in canonical order (error domain, enums, records, callback - /// interfaces, interfaces, functions). - fn render_py_source( - &self, - api: &ResolvedApi, - model: &BindingModel, - config: &PythonConfig, - file_name: &str, - ) -> String { - let id = api.identity(); - let root_error = root_error_name(model, &id.pascal_name()); - let g = Gen { - prefix: &model.prefix, - root_error: &root_error, - strip_module_prefix: config.strip_module_prefix, - }; - let mut out = render_prelude(CommentStyle::Hash, config.input_basename()); - render_runtime( - &mut out, - model, - &RuntimeNames { - identity: id, - error: &root_error, - }, - ); - for m in &model.modules { - out.push_str(&format!("\n\n# === Module: {} ===\n", m.dot_path)); - emit_module(&mut out, &g, m); - } - out.push('\n'); - out.push_str(&render_trailer(CommentStyle::Hash, file_name)); - out - } - - /// The `__init__.py` re-exporting the implementation module. - fn render_init(&self, import: &str, config: &PythonConfig) -> String { - let hash = CommentStyle::Hash; - format!( - "{}from .{import} import * # noqa: F401,F403\n\n{}", - render_prelude(hash, config.input_basename()), - render_trailer(hash, "__init__.py"), - ) - } - - /// The implementation module and its stub, keyed by file name. - fn package_sources( - &self, - api: &ResolvedApi, - model: &BindingModel, - config: &PythonConfig, - names: &Names, - ) -> [(String, String); 4] { - let py = format!("{}.py", names.import); - let pyi = format!("{}.pyi", names.import); - let root_error = root_error_name(model, &api.identity().pascal_name()); - [ - ( - "__init__.py".into(), - self.render_init(&names.import, config), - ), - (py.clone(), self.render_py_source(api, model, config, &py)), - ( - pyi.clone(), - render_pyi_module( - model, - &root_error, - config.strip_module_prefix, - config.input_basename(), - &pyi, - ), - ), - ("py.typed".into(), render_py_typed(config.input_basename())), - ] - } -} - -/// Emit every member of `m` in canonical order. Callback interfaces precede -/// interfaces because interface members take them as parameters. -fn emit_module(out: &mut String, g: &Gen<'_>, m: &ModuleBinding) { - if let Some(e) = m.error.as_ref().filter(|e| e.declared_here) { - render_error(out, g, m, e); - } - for e in &m.enums { - render_enum(out, e); - } - for s in &m.structs { - render_struct(out, s); - } - for cb in &m.callback_interfaces { - render_callback_interface(out, g, cb); - } - for i in &m.interfaces { - render_interface(out, g, m, i); - } - for f in &m.functions { - render_bindings(out, g, f, m.error.as_ref(), &f.name, false); - render_callable(out, g, m, f, FnScope::Free); - } -} - -impl LanguageBackend for PythonGenerator { - type Config = PythonConfig; - - fn name(&self) -> &'static str { - "python" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let names = Names::new(api.identity(), config); - let input_basename = config.input_basename(); - let dir = out_dir.join("python"); - let pkg_dir = dir.join(&names.import); - let mut files: Vec = self - .package_sources(api, model, config, &names) - .into_iter() - .map(|(name, contents)| OutputFile::new(pkg_dir.join(name), contents)) - .collect(); - files.push(OutputFile::new( - dir.join("pyproject.toml"), - render_pyproject_toml(api.identity(), &names.dist, &names.import, input_basename), - )); - files.push(OutputFile::new( - dir.join("README.md"), - render_readme(api.identity(), &names.dist, &names.import, input_basename), - )); - files - } - - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let id = api.identity(); - let names = Names::new(id, config); - let input_basename = config.input_basename(); - let sources = self.package_sources(api, model, config, &names); - let setup_py = - render_packaged_setup_py(&names.dist, &id.version, &names.import, input_basename); - let pyproject = render_pyproject_toml(id, &names.dist, &names.import, input_basename); - - let py_dir = out_dir.join("python"); - let mut files = Vec::new(); - // Wheels exist only for the platforms that have a wheel platform tag; - // a binary for any other platform (Android, wasm32) has no wheel to - // land in and is skipped. - for nb in &ctx.binaries.binaries { - let platform = nb.platform; - let Some(tag) = platform.python_platform_tag() else { - continue; - }; - let tree = py_dir.join(platform.id()); - let pkg_dir = tree.join(&names.import); - for (name, contents) in &sources { - files.push(PackagedFile::text(pkg_dir.join(name), contents.clone())); - } - // Bundled under the file name the loader looks for. - files.push(PackagedFile::copy( - pkg_dir.join(platform.lib_filename(&id.library)), - nb.source.clone(), - )); - files.push(PackagedFile::text( - tree.join("pyproject.toml"), - pyproject.clone(), - )); - files.push(PackagedFile::text(tree.join("setup.py"), setup_py.clone())); - files.push(PackagedFile::text( - tree.join("README.md"), - render_packaged_readme(&names.dist, &names.import, platform, tag, input_basename), - )); - } - Some(files) - } -} diff --git a/crates/weaveffi-gen/src/targets/python/runtime.rs b/crates/weaveffi-gen/src/targets/python/runtime.rs deleted file mode 100644 index 9d2f61ef..00000000 --- a/crates/weaveffi-gen/src/targets/python/runtime.rs +++ /dev/null @@ -1,77 +0,0 @@ -//! The fixed Python runtime every generated module starts with. -//! -//! The runtime is ordinary Python kept in `runtime/*.py` and spliced in with -//! `{{PLACEHOLDER}}` substitution: `core.py` (the root exception, library -//! loading, the load-time ABI and checksum check, the owned-result helpers, -//! the handle base classes, and the value-buffer codec), `aio.py` (the async -//! completion registry), `cancel.py` (cancel tokens), and `callbacks.py` -//! (the callback-interface handle table). The last three are emitted only -//! when the API uses the feature. - -use weaveffi_model::model::{checksum_symbol, BindingModel, ABI_VERSION}; -use weaveffi_model::pkg::Identity; - -const CORE: &str = include_str!("runtime/core.py"); -const AIO: &str = include_str!("runtime/aio.py"); -const CANCEL: &str = include_str!("runtime/cancel.py"); -const CALLBACKS: &str = include_str!("runtime/callbacks.py"); - -/// The identity-driven values the runtime templates are specialized with. -pub(crate) struct RuntimeNames<'a> { - /// The resolved identity of the bound library. - pub identity: &'a Identity, - /// The root exception class name (see [`super::entities::root_error_name`]). - pub error: &'a str, -} - -/// Substitute every `{{PLACEHOLDER}}` a runtime template uses. -fn fill(template: &str, model: &BindingModel, names: &RuntimeNames<'_>) -> String { - let id = names.identity; - let (darwin, linux, windows) = id.library_files(); - let checksums: String = model - .roots() - .map(|m| { - format!( - " (\"{}\", \"{}\", {:#018x}),\n", - m.name, - checksum_symbol(&model.prefix, &m.name), - m.checksum.expect("top-level modules carry a checksum"), - ) - }) - .collect(); - template - .replace("{{NAME}}", &id.name) - .replace("{{ERROR}}", names.error) - .replace("{{PREFIX}}", &model.prefix) - .replace("{{LIBRARY_ENV}}", &id.library_env_var()) - .replace("{{LIB_DARWIN}}", &darwin) - .replace("{{LIB_LINUX}}", &linux) - .replace("{{LIB_WINDOWS}}", &windows) - .replace("{{ABI_VERSION}}", &ABI_VERSION.to_string()) - .replace("{{CHECKSUMS}}\n", &checksums) -} - -/// Append the runtime: the core, then the feature parts the API needs. -pub(crate) fn render_runtime(out: &mut String, model: &BindingModel, names: &RuntimeNames<'_>) { - let callbacks = model.has_callback_interfaces(); - let is_async = model.has_async(); - let cancellable = model.callables().any(|(_, f)| f.cancellable); - let mut imports = String::new(); - if callbacks { - imports.push_str("import abc\n"); - } - if is_async { - imports.push_str("import asyncio\n"); - } - out.push_str(&fill(CORE, model, names).replace("{{IMPORTS}}", &imports)); - for (wanted, template) in [ - (is_async, AIO), - (cancellable, CANCEL), - (callbacks, CALLBACKS), - ] { - if wanted { - out.push_str("\n\n"); - out.push_str(&fill(template, model, names)); - } - } -} diff --git a/crates/weaveffi-gen/src/targets/python/runtime/callbacks.py b/crates/weaveffi-gen/src/targets/python/runtime/callbacks.py deleted file mode 100644 index c67ce0b2..00000000 --- a/crates/weaveffi-gen/src/targets/python/runtime/callbacks.py +++ /dev/null @@ -1,42 +0,0 @@ -# Callback-interface implementations the producer holds, keyed by the integer -# it receives as `ctx`. An entry is added when an implementation is passed to -# a call and removed when the producer calls the vtable's `free(ctx)`, so the -# implementation lives exactly as long as the producer may call it and the -# producer never holds a raw reference to a Python object. ctypes acquires -# the GIL on entry to a trampoline, so the producer may call from any thread. -_callbacks: Dict[int, Any] = {} -_callbacks_lock = threading.Lock() -_callbacks_next = 0 - -_error_set = _bind( - "{{PREFIX}}_error_set", None, ctypes.POINTER(_ErrorStruct), ctypes.c_int32, ctypes.c_char_p -) - - -def _callback_register(impl: Any, cls: type) -> int: - """Register an implementation for one call; returns its `ctx` key.""" - global _callbacks_next - if not isinstance(impl, cls): - raise TypeError(f"expected {cls.__name__}, got {type(impl).__name__}") - with _callbacks_lock: - _callbacks_next += 1 - ctx = _callbacks_next - _callbacks[ctx] = impl - return ctx - - -def _callback_get(ctx: Optional[int]) -> Any: - return _callbacks[ctx or 0] - - -def _callback_free(ctx: Optional[int]) -> None: - # The producer's last reference is gone; it never passes `ctx` again. - with _callbacks_lock: - _callbacks.pop(ctx or 0, None) - - -def _callback_fail(out_err: Any, exc: BaseException) -> None: - """Report an implementation's exception to the producer, which aborts - its call with the foreign error code. Nothing unwinds through C.""" - message = str(exc) or type(exc).__name__ - _error_set(out_err, {{ERROR}}.FOREIGN_ERROR_CODE, message.encode("utf-8", "replace")) diff --git a/crates/weaveffi-gen/src/targets/python/stubs.rs b/crates/weaveffi-gen/src/targets/python/stubs.rs deleted file mode 100644 index 2776dccd..00000000 --- a/crates/weaveffi-gen/src/targets/python/stubs.rs +++ /dev/null @@ -1,298 +0,0 @@ -//! `.pyi` type-stub rendering: a typed mirror of the public surface the -//! generated implementation module exposes. - -use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use weaveffi_model::model::{ - BindingModel, CallbackInterfaceBinding, EnumBinding, ErrorBinding, FieldBinding, FnBinding, - InterfaceBinding, StructBinding, -}; - -use crate::targets::python::docs::emit_doc; -use crate::targets::python::entities::py_code_class_name; -use crate::targets::python::types::{ - py_field, py_field_hint, py_member_name, py_name, py_type_hint, py_variant, py_wrapper_fn_name, -}; - -/// Render the full `.pyi` stub (`file_name`) for the model. -pub(crate) fn render_pyi_module( - model: &BindingModel, - root_error: &str, - strip_module_prefix: bool, - input_basename: &str, - file_name: &str, -) -> String { - let mut out = render_prelude(CommentStyle::Hash, input_basename); - out.push_str( - "import builtins\nfrom abc import ABC, abstractmethod\nfrom enum import IntEnum\n\ - from typing import Dict, Iterator, List, Optional, Type\n", - ); - out.push_str(&format!("\nclass {root_error}(Exception):\n")); - out.push_str(" GENERIC_ERROR_CODE: int\n"); - out.push_str(" PANIC_ERROR_CODE: int\n"); - out.push_str(" MARSHAL_ERROR_CODE: int\n"); - out.push_str(" FOREIGN_ERROR_CODE: int\n"); - out.push_str(" CANCELLED_ERROR_CODE: int\n"); - out.push_str(" code: int\n"); - out.push_str(" message: str\n"); - out.push_str(" def __init__(self, code: int, message: str) -> None: ...\n"); - for m in &model.modules { - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { - render_pyi_error(&mut out, eb, root_error); - } - for e in &m.enums { - if e.is_rich() { - render_pyi_rich_enum(&mut out, e); - } else { - render_pyi_enum(&mut out, e); - } - } - for s in &m.structs { - render_pyi_struct(&mut out, s); - } - for cb in &m.callback_interfaces { - render_pyi_callback_interface(&mut out, cb); - } - for i in &m.interfaces { - render_pyi_interface(&mut out, i); - } - for f in &m.functions { - render_pyi_function(&mut out, &m.path, f, strip_module_prefix); - } - } - out.push('\n'); - out.push_str(&render_trailer(CommentStyle::Hash, file_name)); - out -} - -/// `.pyi` stub for one module's error domain: the domain base class (with -/// its scoped per-code aliases) plus a per-code subclass carrying its stable -/// `CODE` and any structured payload fields, mirroring -/// [`crate::targets::python::entities::render_error`]. -fn render_pyi_error(out: &mut String, eb: &ErrorBinding, root_error: &str) { - let domain = &eb.type_name; - out.push('\n'); - out.push_str(&format!("class {domain}({root_error}):\n")); - for c in &eb.codes { - let class = py_code_class_name(&c.name); - out.push_str(&format!(" {class}: Type[\"{class}\"]\n")); - } - out.push_str(" def __init__(self, code: int, message: str) -> None: ...\n"); - for c in &eb.codes { - let class = py_code_class_name(&c.name); - out.push('\n'); - emit_doc(out, &c.doc, ""); - out.push_str(&format!("class {class}({domain}):\n")); - out.push_str(" CODE: int\n"); - render_field_lines(out, &c.fields); - out.push_str(" def __init__(self, message: str = ...) -> None: ...\n"); - } -} - -/// `.pyi` stub for one callback interface: the abstract base class with one -/// abstract method per IDL method, mirroring -/// [`crate::targets::python::calls::render_callback_interface`]. -fn render_pyi_callback_interface(out: &mut String, cb: &CallbackInterfaceBinding) { - out.push('\n'); - emit_doc(out, &cb.doc, ""); - out.push_str(&format!("class {}(ABC):\n", cb.name)); - for m in &cb.methods { - let mut params = vec!["self".to_string()]; - params.extend( - m.params - .iter() - .map(|p| format!("{}: {}", py_name(&p.name), py_type_hint(&p.ty))), - ); - let ret = m - .ret - .as_ref() - .map(py_type_hint) - .unwrap_or_else(|| "None".into()); - emit_doc(out, &m.doc, " "); - out.push_str(&format!( - " @abstractmethod\n def {}({}) -> {}: ...\n", - py_member_name(&m.name), - params.join(", "), - ret - )); - } -} - -/// `.pyi` stub for one interface wrapper class: `__init__` for the canonical -/// `new` constructor, a classmethod per remaining constructor, the -/// reference-releasing `close()` and context-manager pair, then methods and -/// statics, mirroring [`crate::targets::python::entities::render_interface`]. -fn render_pyi_interface(out: &mut String, i: &InterfaceBinding) { - out.push('\n'); - emit_doc(out, &i.doc, ""); - out.push_str(&format!("class {}:\n", i.name)); - let member_sig = |f: &FnBinding, receiver: Option<&str>| -> String { - let mut params: Vec = receiver.iter().map(|r| r.to_string()).collect(); - params.extend( - f.params - .iter() - .map(|p| format!("{}: {}", py_name(&p.name), py_type_hint(&p.ty))), - ); - params.join(", ") - }; - let async_kw = |f: &FnBinding| if f.is_async { "async " } else { "" }; - if let Some(c) = i.constructors.iter().find(|c| c.name == "new") { - out.push_str(&format!( - " def __init__({}) -> None: ...\n", - member_sig(c, Some("self")) - )); - } - out.push_str(" def close(self) -> None: ...\n"); - out.push_str(&format!(" def __enter__(self) -> \"{}\": ...\n", i.name)); - out.push_str(" def __exit__(self, *exc: object) -> None: ...\n"); - for c in i.constructors.iter().filter(|c| c.name != "new") { - out.push_str(&format!( - " @classmethod\n def {}({}) -> \"{}\": ...\n", - py_member_name(&c.name), - member_sig(c, Some("cls")), - i.name - )); - } - for m in &i.methods { - let ret = m - .ret - .as_ref() - .map(py_type_hint) - .unwrap_or_else(|| "None".into()); - out.push_str(&format!( - " {}def {}({}) -> {}: ...\n", - async_kw(m), - py_member_name(&m.name), - member_sig(m, Some("self")), - ret - )); - } - for s in &i.statics { - let ret = s - .ret - .as_ref() - .map(py_type_hint) - .unwrap_or_else(|| "None".into()); - out.push_str(&format!( - " @staticmethod\n {}def {}({}) -> {}: ...\n", - async_kw(s), - py_member_name(&s.name), - member_sig(s, None), - ret - )); - } -} - -/// `.pyi` stub for a plain C-style enum: an `IntEnum` with typed members. -fn render_pyi_enum(out: &mut String, e: &EnumBinding) { - out.push('\n'); - emit_doc(out, &e.doc, ""); - out.push_str(&format!("class {}(IntEnum):\n", e.name)); - for v in &e.variants { - emit_doc(out, &v.doc, " "); - out.push_str(&format!(" {} = {}\n", py_variant(&v.name), v.value)); - } -} - -/// `.pyi` stub for a rich (algebraic) enum: the base class with its nested -/// `Tag` `IntEnum`, scoped variant aliases, and `tag` reader, plus one -/// dataclass-shaped variant subclass with its fields and constructor, -/// mirroring the rich-enum rendering in [`crate::targets::python::entities`]. -fn render_pyi_rich_enum(out: &mut String, e: &EnumBinding) { - let name = &e.name; - out.push('\n'); - emit_doc(out, &e.doc, ""); - out.push_str(&format!("class {name}:\n")); - out.push_str(" class Tag(IntEnum):\n"); - for v in &e.variants { - emit_doc(out, &v.doc, " "); - out.push_str(&format!(" {} = {}\n", py_variant(&v.name), v.value)); - } - for v in &e.variants { - out.push_str(&format!( - " {0}: Type[\"{name}{0}\"]\n", - py_variant(&v.name) - )); - } - out.push_str(&format!( - " @property\n def tag(self) -> \"{name}.Tag\": ...\n" - )); - for v in &e.variants { - let class = format!("{name}{}", py_variant(&v.name)); - out.push('\n'); - emit_doc(out, &v.doc, ""); - out.push_str(&format!("class {class}({name}):\n")); - out.push_str(&format!(" TAG: \"{name}.Tag\"\n")); - render_field_lines(out, &v.fields); - out.push_str(&dataclass_init(&v.fields)); - } -} - -/// The `__init__` a dataclass generates for `fields`. Its receiver is named -/// like the one `dataclasses` itself uses when a field is called `self`. -fn dataclass_init(fields: &[FieldBinding]) -> String { - let receiver = if fields.iter().any(|f| f.name == "self") { - "__dataclass_self__" - } else { - "self" - }; - let params: Vec = std::iter::once(receiver.to_string()) - .chain( - fields - .iter() - .map(|f| format!("{}: {}", py_field(&f.name), py_field_hint(fields, &f.ty))), - ) - .collect(); - format!(" def __init__({}) -> None: ...\n", params.join(", ")) -} - -/// Emit one `name: hint` attribute line per field, with field docs. -fn render_field_lines(out: &mut String, fields: &[FieldBinding]) { - for f in fields { - emit_doc(out, &f.doc, " "); - out.push_str(&format!( - " {}: {}\n", - py_field(&f.name), - py_field_hint(fields, &f.ty) - )); - } -} - -/// `.pyi` stub for a record: a dataclass-shaped value class with typed field -/// attributes and the generated constructor, mirroring -/// [`crate::targets::python::entities::render_struct`]. -fn render_pyi_struct(out: &mut String, s: &StructBinding) { - out.push('\n'); - emit_doc(out, &s.doc, ""); - out.push_str(&format!("class {}:\n", s.name)); - render_field_lines(out, &s.fields); - out.push_str(&dataclass_init(&s.fields)); -} - -/// `.pyi` stub for one module-level free function. -fn render_pyi_function( - out: &mut String, - module_name: &str, - f: &FnBinding, - strip_module_prefix: bool, -) { - let func_name = py_wrapper_fn_name(module_name, &f.name, strip_module_prefix); - let params: Vec = f - .params - .iter() - .map(|p| format!("{}: {}", py_name(&p.name), py_type_hint(&p.ty))) - .collect(); - let ret = f - .ret - .as_ref() - .map(py_type_hint) - .unwrap_or_else(|| "None".into()); - let async_kw = if f.is_async { "async " } else { "" }; - out.push('\n'); - emit_doc(out, &f.doc, ""); - out.push_str(&format!( - "{async_kw}def {}({}) -> {}: ...\n", - func_name, - params.join(", "), - ret - )); -} diff --git a/crates/weaveffi-gen/src/targets/python/tests.rs b/crates/weaveffi-gen/src/targets/python/tests.rs deleted file mode 100644 index 098ec200..00000000 --- a/crates/weaveffi-gen/src/targets/python/tests.rs +++ /dev/null @@ -1,461 +0,0 @@ -//! Unit tests: render a small API that exercises every ABI revision 3 shape -//! (prefixed runtime, pointer + length strings, reference-counted objects, -//! nullable objects, objects inside buffers, iterators, async and -//! cancellable functions, and callback interfaces) and assert the generated -//! Python carries the pieces each contract clause requires. - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::package::PackageContext; -use crate::platform::{BinarySet, Platform}; -use camino::Utf8Path; -use weaveffi_model::ir::{ - Api, CallbackInterfaceDef, Function, InterfaceDef, Module, Param, StructDef, StructField, - TypeRef, CURRENT_SCHEMA_VERSION, -}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; -use weaveffi_model::validate::validate_api; - -use crate::targets::python::{PythonConfig, PythonGenerator}; - -fn param(name: &str, ty: TypeRef) -> Param { - Param { - name: name.into(), - ty, - doc: None, - } -} - -fn func(name: &str, params: Vec, returns: Option) -> Function { - Function { - name: name.into(), - params, - returns, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - } -} - -fn field(name: &str, ty: TypeRef) -> StructField { - StructField { - name: name.into(), - ty, - doc: None, - } -} - -fn named(name: &str) -> TypeRef { - TypeRef::Named(name.into()) -} - -fn module(name: &str) -> Module { - Module { - name: name.into(), - doc: None, - functions: vec![], - interfaces: vec![], - callback_interfaces: vec![], - structs: vec![], - enums: vec![], - errors: None, - modules: vec![], - } -} - -fn resolve(modules: Vec) -> ResolvedApi { - let api = Api { - version: CURRENT_SCHEMA_VERSION.into(), - modules, - }; - validate_api(api, None) - .unwrap_or_else(|d| panic!("fixture must validate: {d:?}")) - .with_identity(Identity::named("kv-lib")) -} - -/// An interface with a constructor and methods; a function taking and -/// returning `Interface?`; a record with object fields; an iterator over -/// objects; a callback interface; an async function and a cancellable one. -fn kv_api() -> ResolvedApi { - let kv = Module { - structs: vec![StructDef { - name: "Entry".into(), - doc: None, - deprecated: None, - fields: vec![ - field("key", TypeRef::StringUtf8), - field("store", named("Store")), - field("mirrors", TypeRef::List(Box::new(named("Store")))), - field("weights", TypeRef::List(Box::new(TypeRef::I32))), - ], - }], - interfaces: vec![InterfaceDef { - name: "Store".into(), - doc: Some("A key-value store.".into()), - deprecated: None, - constructors: vec![func("new", vec![param("path", TypeRef::StringUtf8)], None)], - methods: vec![ - func( - "get", - vec![param("key", TypeRef::StringUtf8)], - Some(TypeRef::StringUtf8), - ), - func( - "has", - vec![param("key", TypeRef::StringUtf8)], - Some(TypeRef::Bool), - ), - ], - statics: vec![], - }], - callback_interfaces: vec![CallbackInterfaceDef { - name: "Watcher".into(), - doc: Some("Observes store changes.".into()), - deprecated: None, - methods: vec![ - func( - "on_change", - vec![ - param("text", TypeRef::StringUtf8), - param("entry", named("Entry")), - param("store", named("Store")), - ], - None, - ), - func( - "should_stop", - vec![param("count", TypeRef::I32)], - Some(TypeRef::Bool), - ), - ], - }], - functions: vec![ - func( - "pick", - vec![param("store", TypeRef::Optional(Box::new(named("Store"))))], - Some(TypeRef::Optional(Box::new(named("Store")))), - ), - func("watch", vec![param("watcher", named("Watcher"))], None), - func( - "all_stores", - vec![], - Some(TypeRef::Iterator(Box::new(named("Store")))), - ), - Function { - r#async: true, - ..func( - "fetch", - vec![param("id", TypeRef::I64)], - Some(named("Store")), - ) - }, - Function { - r#async: true, - cancellable: true, - ..func( - "sleep", - vec![param("ms", TypeRef::I64)], - Some(TypeRef::StringUtf8), - ) - }, - ], - ..module("kv") - }; - resolve(vec![kv]) -} - -fn render_files(api: &ResolvedApi, config: &PythonConfig) -> Vec { - let model = BindingModel::build(api); - PythonGenerator.files(api, &model, Utf8Path::new("out"), config) -} - -/// Render the fixture and return `(module source, stub)`. -fn render(api: &ResolvedApi) -> (String, String) { - let files = render_files(api, &PythonConfig::default()); - let find = |suffix: &str| { - files - .iter() - .find(|f| f.path.as_str().ends_with(suffix)) - .unwrap_or_else(|| panic!("missing {suffix}")) - .contents - .clone() - }; - (find("kv_lib.py"), find("kv_lib.pyi")) -} - -fn assert_has(hay: &str, needle: &str) { - assert!(hay.contains(needle), "missing `{needle}` in:\n{hay}"); -} - -#[test] -fn layout_follows_the_identity() { - let files = render_files(&kv_api(), &PythonConfig::default()); - let paths: Vec<&str> = files.iter().map(|f| f.path.as_str()).collect(); - assert_eq!( - paths, - [ - "out/python/kv_lib/__init__.py", - "out/python/kv_lib/kv_lib.py", - "out/python/kv_lib/kv_lib.pyi", - "out/python/kv_lib/py.typed", - "out/python/pyproject.toml", - "out/python/README.md", - ] - ); - assert_has(&files[0].contents, "from .kv_lib import *"); - let pyproject = &files[4].contents; - assert_has(pyproject, "name = \"kv-lib\""); - assert_has(pyproject, "requires-python = \">=3.9\""); - assert_has(pyproject, "packages = [\"kv_lib\"]"); - - let config = PythonConfig { - package_name: Some("kv-python".into()), - import_name: Some("kvpy".into()), - ..PythonConfig::default() - }; - let files = render_files(&kv_api(), &config); - assert!(files.iter().any(|f| f.path == "out/python/kvpy/kvpy.py")); - let pyproject = files.iter().find(|f| f.path.ends_with("pyproject.toml")); - assert_has(&pyproject.unwrap().contents, "name = \"kv-python\""); -} - -#[test] -fn runtime_is_prefixed_and_checks_the_contract() { - let (py, pyi) = render(&kv_api()); - assert_has(&py, "_ABI_VERSION = 3\n"); - assert_has(&py, " (\"kv\", \"kv_lib_kv_checksum\", 0x"); - assert_has(&py, "_bind(\"kv_lib_abi_version\", ctypes.c_uint32)"); - assert_has(&py, "os.environ.get(\"KV_LIB_LIBRARY\")"); - assert_has(&py, "name = \"libkv_lib.dylib\""); - assert_has( - &py, - "_bind(\"kv_lib_free_bytes\", None, ctypes.c_void_p, ctypes.c_size_t)", - ); - assert_has(&py, "CANCELLED_ERROR_CODE = -5"); - assert_has(&py, "class Error(Exception):"); - assert_has(&pyi, "class Error(Exception):"); - assert!(!py.contains("free_string")); - // Nothing but the generated-file header names WeaveFFI. - for text in [&py, &pyi] { - for line in text.lines().filter(|l| !l.starts_with('#')) { - assert!( - !line.to_ascii_lowercase().contains("weaveffi"), - "branded line: {line}" - ); - } - } -} - -#[test] -fn prototypes_are_bound_once_with_c_bool_and_ptr_len_strings() { - let (py, _) = render(&kv_api()); - assert_has( - &py, - "_c_kv_Store_has = _bind(\"kv_lib_kv_Store_has\", ctypes.c_bool, ctypes.c_void_p, \ - ctypes.c_char_p, ctypes.c_size_t, ctypes.POINTER(_ErrorStruct))", - ); - assert_eq!(py.matches("_c_kv_Store_get = _bind(").count(), 1); - assert!(!py.contains(".argtypes = ["), "no per-call binding"); - // A string crosses as UTF-8 bytes plus a length and comes back owned. - assert_has(&py, "_key_b = key.encode(\"utf-8\")"); - assert_has(&py, "_key_b, len(_key_b)"); - assert_has(&py, "return _take_str(_ret, _out_len.value)"); -} - -#[test] -fn objects_are_lent_for_calls_and_adopted_from_returns() { - let (py, pyi) = render(&kv_api()); - assert_has(&py, "class Store(_Object):"); - assert_has(&py, " _destroy = staticmethod(_c_kv_Store_destroy)\n"); - assert_has(&py, " _clone = staticmethod(_c_kv_Store_clone)\n"); - assert_has(&py, "def __init__(self, path: str) -> None:"); - assert_has(&py, "self._init_handle(_required(_ret))"); - // `self` is lent for the call and released after it, even on error. - assert_has( - &py, - "_self_p = self._acquire()\n try:\n _ret = _c_kv_Store_get(_self_p,", - ); - assert_has(&py, " finally:\n self._release()\n"); - // Nullable objects map None to NULL both ways. - assert_has(&py, "_store_p = _lend_opt(store, Store)"); - assert_has(&py, "_release_opt(store)"); - assert_has(&py, "return (Store._adopt(_ret) if _ret else None)"); - // Objects inside buffers are cloned when the buffer is finished and - // adopted on read; numeric lists take the packed fast path. - assert_has(&py, "_w.write_object(value.store, Store)"); - assert_has(&py, "store=Store._adopt(_r.read_object()),"); - assert_has(&py, "_w.write_numbers(\"i\", value.weights)"); - assert_has(&py, "weights=_r.read_numbers(\"i\", 4),"); - assert_has(&pyi, " def close(self) -> None: ...\n"); - assert_has(&pyi, " def __enter__(self) -> \"Store\": ...\n"); -} - -#[test] -fn iterators_pull_lazily_through_prebound_functions() { - let (py, pyi) = render(&kv_api()); - assert_has(&py, "class _kv_AllStoresIterator(_Iterator):"); - assert_has( - &py, - " _destroy = staticmethod(_c_kv_AllStoresIterator_destroy)", - ); - assert_has( - &py, - "_more = _c_kv_AllStoresIterator_next(_p, ctypes.byref(_item), ctypes.byref(_err))", - ); - assert_has(&py, "return Store._adopt(_required(_item.value))"); - assert_has(&py, "return _kv_AllStoresIterator._adopt(_ret)"); - assert_has(&pyi, "def all_stores() -> Iterator[\"Store\"]: ..."); -} - -#[test] -fn async_calls_complete_through_static_trampolines() { - let (py, pyi) = render(&kv_api()); - assert_has(&py, "import asyncio\n"); - assert_has( - &py, - "_c_kv_fetch_callback = ctypes.CFUNCTYPE(None, ctypes.c_void_p, \ - ctypes.POINTER(_ErrorStruct), ctypes.c_void_p)", - ); - assert_has(&py, "def _kv_fetch_complete(context, err, result) -> None:"); - assert_has( - &py, - "_async_settle(context, None, Store._adopt(_required(result)))", - ); - assert_has( - &py, - "_kv_fetch_completion = _c_kv_fetch_callback(_kv_fetch_complete)", - ); - assert_has(&py, "_c_kv_fetch(id, _kv_fetch_completion, _call)"); - assert_has(&py, "return await _future"); - // A cancellable call owns a token for its duration. - assert_has(&py, "_token = _cancel_token_create()"); - assert_has(&py, "_c_kv_sleep(ms, _token, _kv_sleep_completion, _call)"); - assert_has(&py, "return await _async_wait_cancellable(_future, _token)"); - assert_has( - &py, - "_async_settle(context, None, _take_str(result_ptr, result_len))", - ); - assert_has(&pyi, "async def sleep(ms: int) -> str: ..."); -} - -#[test] -fn callback_interfaces_render_abc_vtable_and_trampolines() { - let (py, pyi) = render(&kv_api()); - assert_has(&py, "import abc\n"); - assert_has(&py, "class Watcher(abc.ABC):"); - assert_has( - &py, - "_Watcher_on_change_t = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.c_void_p, \ - ctypes.c_size_t, ctypes.c_void_p, ctypes.c_size_t, ctypes.c_void_p, \ - ctypes.POINTER(_ErrorStruct))", - ); - assert_has( - &py, - "_Watcher_should_stop_t = ctypes.CFUNCTYPE(ctypes.c_bool, ctypes.c_void_p, \ - ctypes.c_int32, ctypes.POINTER(_ErrorStruct))", - ); - assert_has(&py, "class _WatcherVtable(ctypes.Structure):"); - assert_has(&py, "The C vtable `kv_lib_kv_Watcher_vtable`."); - assert_has( - &py, - "_callback_get(ctx).on_change(_peek_bytes(text_ptr, text_len).decode(\"utf-8\"), \ - _decode(_peek_bytes(entry_ptr, entry_len), _read_Entry), \ - Store._adopt(_required(store)))", - ); - assert_has(&py, "return bool(_callback_get(ctx).should_stop(count))"); - assert_has(&py, "_callback_fail(out_err, exc)\n return False"); - assert_has(&py, "_Watcher_free_t(_callback_free),"); - assert_eq!(py.matches("_Watcher_vtable = _WatcherVtable(").count(), 1); - assert_has(&py, "_watcher_ctx = _callback_register(watcher, Watcher)"); - assert_has( - &py, - "_c_kv_watch(_watcher_ctx, ctypes.addressof(_Watcher_vtable), ctypes.byref(_err))", - ); - assert_has(&pyi, "class Watcher(ABC):"); -} - -#[test] -fn feature_runtime_is_emitted_only_when_used() { - let plain = Module { - functions: vec![func( - "add", - vec![param("a", TypeRef::I32), param("b", TypeRef::I32)], - Some(TypeRef::I32), - )], - ..module("math") - }; - let (py, _) = render(&resolve(vec![plain])); - assert!(!py.contains("import abc")); - assert!(!py.contains("import asyncio")); - assert!(!py.contains("_callbacks")); - assert!(!py.contains("_cancel_token")); - assert_has(&py, "def add(a: int, b: int) -> int:"); -} - -#[test] -fn root_error_avoids_user_type_names() { - let m = Module { - structs: vec![StructDef { - name: "Error".into(), - doc: None, - deprecated: None, - fields: vec![field("code", TypeRef::I32)], - }], - ..module("kv") - }; - let (py, pyi) = render(&resolve(vec![m])); - assert_has(&py, "class KvLibError(Exception):"); - assert_has(&pyi, "class KvLibError(Exception):"); -} - -#[test] -fn generated_source_is_deterministic() { - let api = kv_api(); - assert_eq!(render(&api), render(&api)); -} - -#[test] -fn package_bundles_the_identity_library_per_wheel_platform() { - let api = kv_api(); - let model = BindingModel::build(&api); - let mut binaries = BinarySet::new("kv_lib"); - for p in Platform::ALL { - binaries.insert(p, format!("/tmp/{}/lib", p.id())); - } - let ctx = PackageContext { - binaries: &binaries, - input_basename: Some("kv.yml"), - }; - let files = PythonGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &PythonConfig::default(), - ) - .expect("python packages"); - let trees: std::collections::BTreeSet<&str> = files - .iter() - .filter_map(|f| f.path.as_str().strip_prefix("out/python/")) - .filter_map(|rest| rest.split('/').next()) - .collect(); - let expected: std::collections::BTreeSet<&str> = - Platform::DESKTOP.iter().map(|p| p.id()).collect(); - assert_eq!(trees, expected); - assert!(files.iter().any(|f| f - .path - .as_str() - .ends_with("macos-arm64/kv_lib/libkv_lib.dylib") - || f.path.as_str().ends_with("/kv_lib/libkv_lib.dylib"))); - for p in [ - Platform::AndroidArm64, - Platform::AndroidX64, - Platform::Wasm32, - ] { - assert!(!files.iter().any(|f| f.path.as_str().contains(p.id()))); - } -} diff --git a/crates/weaveffi-gen/src/targets/ruby/callbacks.rs b/crates/weaveffi-gen/src/targets/ruby/callbacks.rs deleted file mode 100644 index 3ffd8d02..00000000 --- a/crates/weaveffi-gen/src/targets/ruby/callbacks.rs +++ /dev/null @@ -1,326 +0,0 @@ -//! Callback-interface rendering: the consumer-facing Ruby module (duck-typed -//! method set with `NotImplementedError` defaults), the `FFI::Struct` vtable -//! layout, one trampoline `FFI::Function` per method plus the trailing -//! `free`, and the single process-wide vtable instance every registration -//! passes to the producer. -//! -//! The trampolines follow [`CallbackProtocol`](weaveffi_model::plan::CallbackProtocol): -//! `ctx` is resolved through the module's implementation registry, borrowed -//! string, bytes, and buffer arguments are copied or decoded before the -//! implementation runs, object arguments are adopted into wrappers, and any -//! exception is reported through `{prefix}_error_set(out_err, -4, message)` -//! instead of unwinding through the C frame. - -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use heck::ToShoutySnakeCase; -use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, Ty}; -use weaveffi_model::plan::{ArgPass, RetPass}; - -use crate::targets::ruby::docs::{emit_doc, emit_param_docs}; -use crate::targets::ruby::types::{rb_abi_types, rb_direct_default, rb_ffi_type, rb_param_name}; - -/// The `FFI::Struct` subclass laying out one callback interface's vtable: -/// `Listener` (or `events.Listener`) becomes `WvListenerVtable`. -fn rb_vtable_class(name: &str) -> String { - format!("Wv{}Vtable", local_type_name(name)) -} - -/// The constant holding the one process-wide vtable instance for a callback -/// interface: `Listener` becomes `WV_LISTENER_VTABLE`. -pub(crate) fn rb_vtable_const(name: &str) -> String { - format!("WV_{}_VTABLE", local_type_name(name).to_shouty_snake_case()) -} - -/// The constant pinning one method's trampoline `FFI::Function`: -/// `Listener.on_message` becomes `WV_LISTENER_ON_MESSAGE`. -fn rb_trampoline_const(cb: &str, method: &str) -> String { - format!( - "WV_{}_{}", - local_type_name(cb).to_shouty_snake_case(), - method.to_shouty_snake_case() - ) -} - -/// The constant pinning the trailing `free` trampoline. -fn rb_free_const(cb: &str) -> String { - format!("WV_{}_FREE", local_type_name(cb).to_shouty_snake_case()) -} - -/// The trampoline's Ruby formal parameter names for one method: `ctx`, one -/// formal per parameter slot (a `_ptr`/`_len` pair for strings, bytes, and -/// buffered values), then `out_err`. The Ruby spellings mirror the C slot -/// names so the list lines up with [`CallbackMethodBinding::abi_params`] -/// position for position. -fn rb_tramp_formals(m: &CallbackMethodBinding) -> Vec { - let mut formals = vec!["ctx".to_string()]; - for p in &m.params { - let n = rb_param_name(&p.name); - match p.arg_pass() { - ArgPass::String { .. } | ArgPass::Bytes { .. } | ArgPass::Buffer { .. } => { - formals.push(format!("{n}_ptr")); - formals.push(format!("{n}_len")); - } - _ => formals.push(n), - } - } - formals.push("out_err".to_string()); - formals -} - -/// Render one callback interface: the consumer-facing module, the vtable -/// layout, the trampolines, and the static vtable instance. `prefix` is the -/// C symbol prefix the model was built with (needed to resolve each -/// parameter's receiving plan). -pub(crate) fn render_callback_interface( - out: &mut String, - cb: &CallbackInterfaceBinding, - prefix: &str, -) { - let protocol = cb.protocol(prefix); - let local = local_type_name(&cb.name); - let vtable_class = rb_vtable_class(&cb.name); - let vtable_const = rb_vtable_const(&cb.name); - - let mut w = CodeWriter::two_space().with_depth(1); - - // The consumer-facing module: documentation of the required methods, and - // NotImplementedError defaults for consumers who include it. - w.blank(); - let mut d = String::new(); - emit_doc(&mut d, &cb.doc, " "); - w.raw(d); - if cb.doc.is_some() { - w.line("#"); - } - w.line("# Consumer-implemented callback interface. Any object responding to the"); - w.line(format!( - "# methods below is accepted wherever a {local} is expected; include" - )); - w.line("# this module to inherit NotImplementedError defaults. The producer"); - w.line("# may call the methods from any thread until it releases the"); - w.line("# implementation."); - if let Some(msg) = &cb.deprecated { - w.line(format!("# @deprecated {msg}")); - } - w.block(format!("module {local}"), "end", |w| { - for (idx, m) in cb.methods.iter().enumerate() { - if idx > 0 { - w.blank(); - } - let mut md = String::new(); - emit_doc(&mut md, &m.doc, " "); - w.raw(md); - if let Some(msg) = &m.deprecated { - w.line(format!("# @deprecated {msg}")); - } - emit_param_docs(w, &m.params); - if let Some(ret) = &m.ret { - w.line(format!("# @return [Object] a {ret}")); - } - let formals: Vec = m.params.iter().map(|p| rb_param_name(&p.name)).collect(); - let open = if formals.is_empty() { - format!("def {}", m.name) - } else { - format!("def {}({})", m.name, formals.join(", ")) - }; - w.block(open, "end", |w| { - w.line(format!( - "raise NotImplementedError, \"#{{self.class}}#{} is not implemented\"", - m.name - )); - }); - } - }); - - // The vtable layout: one pointer per method in declaration order, then - // the trailing free. - w.blank(); - w.line("# @api private"); - w.line(format!( - "# The C vtable layout the producer calls a {local} through." - )); - w.block(format!("class {vtable_class} < FFI::Struct"), "end", |w| { - let mut fields: Vec = cb - .methods - .iter() - .map(|m| format!(":{}, :pointer", m.name)) - .collect(); - fields.push(":free, :pointer".to_string()); - w.line(format!("layout {}", fields.join(",\n "))); - }); - - // One trampoline per method. - for (m, args) in cb.methods.iter().zip(&protocol.method_args) { - render_trampoline(&mut w, cb, m, args); - } - - // The trailing free: drop the registry entry; the producer never touches - // ctx again after this fires. - w.blank(); - w.line("# @api private"); - w.line(format!( - "# Releases a {local} implementation when the producer drops its last" - )); - w.line("# reference."); - w.block( - format!( - "{} = FFI::Function.new(:void, [:pointer]) do |ctx|", - rb_free_const(&cb.name) - ), - "end", - |w| { - w.line("_wv_cb_free(ctx)"); - }, - ); - - // The single static vtable instance, filled with the pinned trampolines. - w.blank(); - w.line(format!( - "# The one process-wide {local} vtable every registration hands the" - )); - w.line("# producer; its entries live for the process lifetime."); - w.line(format!("{vtable_const} = {vtable_class}.new")); - for m in &cb.methods { - w.line(format!( - "{vtable_const}[:{}] = {}", - m.name, - rb_trampoline_const(&cb.name, &m.name) - )); - } - w.line(format!( - "{vtable_const}[:free] = {}", - rb_free_const(&cb.name) - )); - - out.push_str(&w.finish()); -} - -/// Render one method's trampoline: an `FFI::Function` matching the vtable -/// entry's C signature that receives each argument per its [`RetPass`] plan, -/// resolves `ctx`, invokes the implementation, and converts the direct -/// return. Object arguments are adopted first, so their references are owned -/// by a wrapper even when a later step raises. Any exception is reported -/// through `_wv_cb_fail` (which calls `{prefix}_error_set` with `-4`) and the -/// method's default value is returned instead. -fn render_trampoline( - w: &mut CodeWriter, - cb: &CallbackInterfaceBinding, - m: &CallbackMethodBinding, - args: &[RetPass], -) { - let local = local_type_name(&cb.name); - let types = rb_abi_types(&m.abi_params); - let formals = rb_tramp_formals(m); - let default = rb_direct_default(m.ret.as_ref()); - - w.blank(); - w.line("# @api private"); - w.line(format!("# Trampoline for {local}#{}.", m.name)); - w.block( - format!( - "{} = FFI::Function.new({}, [{}]) do |{}|", - rb_trampoline_const(&cb.name, &m.name), - rb_ffi_type(&m.abi_ret), - types.join(", "), - formals.join(", ") - ), - "end", - |w| { - w.line("begin"); - w.scope(|w| { - let params: Vec<_> = m.params.iter().zip(args).collect(); - let is_object = |pass: &RetPass| matches!(pass, RetPass::Object { .. }); - for (p, pass) in params.iter().filter(|(_, pass)| is_object(pass)) { - render_tramp_arg(w, &p.name, &p.ty, pass); - } - for (p, pass) in params.iter().filter(|(_, pass)| !is_object(pass)) { - render_tramp_arg(w, &p.name, &p.ty, pass); - } - w.line("impl = _wv_cb_lookup(ctx)"); - let call_args: Vec = m - .params - .iter() - .map(|p| format!("{}_v", rb_param_name(&p.name))) - .collect(); - let call = if call_args.is_empty() { - format!("impl.{}", m.name) - } else { - format!("impl.{}({})", m.name, call_args.join(", ")) - }; - match &m.ret { - None => { - w.line(call); - w.line("nil"); - } - // Coerce inside the rescue so a wrong-typed return - // surfaces as a foreign error rather than escaping - // through ffi's own conversion. - Some(Ty::Bool) => { - w.line(format!("{call} ? true : false")); - } - Some(Ty::F32 | Ty::F64) => { - w.line(format!("Float({call})")); - } - Some(_) => { - w.line(format!("Integer({call})")); - } - } - }); - // Rescue Exception, not StandardError: NotImplementedError is a - // ScriptError, and nothing may unwind through the C frame. - w.line("rescue Exception => e # rubocop:disable Lint/RescueException"); - w.scope(|w| { - w.line("_wv_cb_fail(out_err, e)"); - w.line(default); - }); - w.line("end"); - }, - ); -} - -/// Emit the statements receiving one trampoline argument into the local -/// `{name}_v`, per its [`RetPass`] plan. Strings, bytes, and buffers are -/// borrowed `(ptr, len)` pairs copied or decoded before the implementation -/// runs; objects transfer one strong reference that is adopted into a -/// wrapper (`nil` for a null nullable slot); direct values pass through. -fn render_tramp_arg(w: &mut CodeWriter, name: &str, ty: &Ty, pass: &RetPass) { - let n = rb_param_name(name); - match pass { - RetPass::Void => unreachable!("callback parameters always have a type"), - RetPass::Direct => { - w.line(format!("{n}_v = {n}")); - } - RetPass::String => { - w.line(format!( - "{n}_v = _wv_borrow_bytes({n}_ptr, {n}_len).force_encoding(Encoding::UTF_8)" - )); - } - RetPass::Bytes => { - w.line(format!("{n}_v = _wv_borrow_bytes({n}_ptr, {n}_len)")); - } - RetPass::Buffer => { - w.line(format!( - "{n}_r = WvBufferReader.new(_wv_borrow_bytes({n}_ptr, {n}_len))" - )); - crate::targets::ruby::codec::render_wv_read( - w, - &format!("{n}_r"), - &format!("{n}_v"), - ty, - 0, - "", - ); - w.line(format!("{n}_r.expect_end!")); - } - RetPass::Object { nullable, .. } => { - let iface = ty.interface_name().expect("object plan names an interface"); - let class = local_type_name(iface); - if *nullable { - w.line(format!("{n}_v = {n}.null? ? nil : {class}._from_ptr({n})")); - } else { - w.line(format!("{n}_v = {class}._from_ptr({n})")); - } - } - } -} diff --git a/crates/weaveffi-gen/src/targets/ruby/codec.rs b/crates/weaveffi-gen/src/targets/ruby/codec.rs deleted file mode 100644 index c8de152d..00000000 --- a/crates/weaveffi-gen/src/targets/ruby/codec.rs +++ /dev/null @@ -1,287 +0,0 @@ -//! Value-buffer codec emitters: the statement-level pack and unpack -//! renderers for any wire shape, plus the per-record and per-rich-enum -//! codec pairs (`_wv_write_{stem}`, `_wv_read_{stem}`). -//! -//! Every dispatch here goes through [`Ty::wire`], so this module never -//! re-derives the wire folds (records and rich enums as one user-codec -//! shape, C-style enums as `i32`, interfaces as `u64` object tokens). -//! -//! An object token carries one strong reference. Writing one reserves the -//! token (`write_object`); the runtime's `_wv_seal` mints the references -//! (the interface's `_clone` symbol) just before the call, so the wrapper -//! keeps its own reference and an abandoned encoding strands none. Reading -//! one adopts the pointer into a fresh wrapper through `_from_ptr`, whose -//! finalizer owes the `_destroy`. - -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use heck::ToSnakeCase; -use weaveffi_model::model::Ty; -use weaveffi_model::model::{EnumBinding, StructBinding}; -use weaveffi_model::model::{Prim, WireType}; - -use crate::targets::ruby::types::rb_field_name; - -/// The snake_case stem naming a record's or rich enum's generated pack and -/// unpack helpers: `Contact` (or `other.Contact`) becomes `contact`, naming -/// `_wv_write_contact` and `_wv_read_contact`. -pub(crate) fn wv_stem(name: &str) -> String { - local_type_name(name).to_snake_case() -} - -/// The `WvBufferWriter`/`WvBufferReader` method stem (`i32`, `string`, ...) -/// for one scalar wire shape, or `None` for the composite shapes that need -/// statement-level rendering. C-style enums travel as their `i32` -/// discriminant. -fn wv_scalar(shape: &WireType) -> Option<&'static str> { - match shape { - WireType::Prim(p) => Some(p.snake()), - WireType::Enum(_) => Some(Prim::I32.snake()), - _ => None, - } -} - -/// Emit the Ruby statements appending `expr` (a value of IR type `ty`) to -/// the buffer writer named `wvar`, following the value-buffer wire format. -/// `q` is the dotted receiver (`"Kvstore."` or `""`) qualifying -/// module-singleton codec calls inside class bodies. -pub(crate) fn render_wv_write( - w: &mut CodeWriter, - wvar: &str, - expr: &str, - ty: &Ty, - depth: usize, - q: &str, -) { - let shape = ty.wire(); - if let Some(m) = wv_scalar(&shape) { - w.line(format!("{wvar}.write_{m}({expr})")); - return; - } - match shape { - WireType::Optional(inner) => { - w.line(format!("if {expr}.nil?")); - w.scope(|w| { - w.line(format!("{wvar}.write_flag(false)")); - }); - w.line("else"); - w.scope(|w| { - w.line(format!("{wvar}.write_flag(true)")); - render_wv_write(w, wvar, expr, inner, depth, q); - }); - w.line("end"); - } - WireType::List(elem) => { - let e = format!("_wv_e{depth}"); - w.line(format!("{wvar}.write_len({expr}.length)")); - w.block(format!("{expr}.each do |{e}|"), "end", |w| { - render_wv_write(w, wvar, &e, elem, depth + 1, q); - }); - } - WireType::Map(k, v) => { - let kn = format!("_wv_k{depth}"); - let vn = format!("_wv_v{depth}"); - w.line(format!("{wvar}.write_len({expr}.length)")); - w.block(format!("{expr}.each do |{kn}, {vn}|"), "end", |w| { - render_wv_write(w, wvar, &kn, k, depth + 1, q); - render_wv_write(w, wvar, &vn, v, depth + 1, q); - }); - } - WireType::User(n) => { - w.line(format!("{q}_wv_write_{}({wvar}, {expr})", wv_stem(n))); - } - // Reserved here; `_wv_seal` mints the token's reference. - WireType::Object(_) => { - w.line(format!("{wvar}.write_object({expr})")); - } - _ => unreachable!("scalar handled above"), - } -} - -/// Emit the Ruby statements decoding one `ty` value from the buffer reader -/// named `rvar` into the local `var`. `q` is the dotted receiver qualifying -/// module-singleton codec calls inside class bodies. -pub(crate) fn render_wv_read( - w: &mut CodeWriter, - rvar: &str, - var: &str, - ty: &Ty, - depth: usize, - q: &str, -) { - let shape = ty.wire(); - if let Some(m) = wv_scalar(&shape) { - w.line(format!("{var} = {rvar}.read_{m}")); - return; - } - match shape { - WireType::Optional(inner) => { - w.line(format!("if {rvar}.read_flag")); - w.scope(|w| { - render_wv_read(w, rvar, var, inner, depth, q); - }); - w.line("else"); - w.scope(|w| { - w.line(format!("{var} = nil")); - }); - w.line("end"); - } - WireType::List(elem) => { - let e = format!("_wv_e{depth}"); - w.block( - format!("{var} = Array.new({rvar}.read_len) do"), - "end", - |w| { - render_wv_read(w, rvar, &e, elem, depth + 1, q); - w.line(e.clone()); - }, - ); - } - WireType::Map(k, v) => { - let kn = format!("_wv_k{depth}"); - let vn = format!("_wv_v{depth}"); - w.line(format!("{var} = {{}}")); - w.block(format!("{rvar}.read_len.times do"), "end", |w| { - render_wv_read(w, rvar, &kn, k, depth + 1, q); - render_wv_read(w, rvar, &vn, v, depth + 1, q); - w.line(format!("{var}[{kn}] = {vn}")); - }); - } - WireType::User(n) => { - w.line(format!("{var} = {q}_wv_read_{}({rvar})", wv_stem(n))); - } - // The reader adopts the token's strong reference into a new wrapper - // whose finalizer owes the `_destroy`; a zero token is malformed. - WireType::Object(n) => { - w.line(format!( - "{var} = {}._from_ptr({rvar}.read_object_token)", - local_type_name(n) - )); - } - _ => unreachable!("scalar handled above"), - } -} - -/// Render the private pack/unpack pair for one record: module singleton -/// methods `_wv_write_{stem}(w, v)` and `_wv_read_{stem}(r)` serializing the -/// fields in declaration (wire) order. -pub(crate) fn render_struct_codec(out: &mut String, s: &StructBinding) { - let stem = wv_stem(&s.name); - let mut w = CodeWriter::two_space().with_depth(1); - w.blank(); - w.line("# @api private"); - w.line(format!( - "# Packs a {} into the value-buffer wire format.", - s.name - )); - w.block(format!("def self._wv_write_{stem}(w, v)"), "end", |w| { - for f in &s.fields { - let field = rb_field_name(&f.name); - render_wv_write(w, "w", &format!("v.{field}"), &f.ty, 0, ""); - } - }); - w.blank(); - w.line("# @api private"); - w.line(format!( - "# Unpacks a {} from the value-buffer wire format.", - s.name - )); - w.block(format!("def self._wv_read_{stem}(r)"), "end", |w| { - for f in &s.fields { - let field = rb_field_name(&f.name); - render_wv_read(w, "r", &format!("_wv_{field}"), &f.ty, 0, ""); - } - let kwargs = s - .fields - .iter() - .map(|f| { - let field = rb_field_name(&f.name); - format!("{field}: _wv_{field}") - }) - .collect::>() - .join(", "); - if kwargs.is_empty() { - w.line(format!("{}.new", s.name)); - } else { - w.line(format!("{}.new({kwargs})", s.name)); - } - }); - out.push_str(&w.finish()); -} - -/// Render the private pack/unpack pair for one rich enum: `_wv_write_{stem}` -/// dispatches on the variant class and writes the `i32` tag followed by the -/// variant's fields; `_wv_read_{stem}` switches on the decoded tag. -pub(crate) fn render_rich_enum_codec(out: &mut String, e: &EnumBinding) { - let stem = wv_stem(&e.name); - let mut w = CodeWriter::two_space().with_depth(1); - w.blank(); - w.line("# @api private"); - w.line(format!( - "# Packs a {} into the value-buffer wire format.", - e.name - )); - w.block(format!("def self._wv_write_{stem}(w, v)"), "end", |w| { - w.line("case v"); - for v in &e.variants { - w.line(format!("when {}::{}", e.name, v.name)); - w.scope(|w| { - w.line(format!("w.write_i32({})", v.value)); - for f in &v.fields { - let field = rb_field_name(&f.name); - render_wv_write(w, "w", &format!("v.{field}"), &f.ty, 0, ""); - } - }); - } - w.line("else"); - w.scope(|w| { - w.line(format!( - "raise Error.new(MARSHAL_ERROR_CODE, 'unknown {} variant')", - e.name - )); - }); - w.line("end"); - }); - w.blank(); - w.line("# @api private"); - w.line(format!( - "# Unpacks a {} from the value-buffer wire format.", - e.name - )); - w.block(format!("def self._wv_read_{stem}(r)"), "end", |w| { - w.line("tag = r.read_i32"); - w.line("case tag"); - for v in &e.variants { - w.line(format!("when {}", v.value)); - w.scope(|w| { - for f in &v.fields { - let field = rb_field_name(&f.name); - render_wv_read(w, "r", &format!("_wv_{field}"), &f.ty, 0, ""); - } - let kwargs = v - .fields - .iter() - .map(|f| { - let field = rb_field_name(&f.name); - format!("{field}: _wv_{field}") - }) - .collect::>() - .join(", "); - if kwargs.is_empty() { - w.line(format!("{}::{}.new", e.name, v.name)); - } else { - w.line(format!("{}::{}.new({kwargs})", e.name, v.name)); - } - }); - } - w.line("else"); - w.scope(|w| { - w.line(format!( - "raise Error.new(MARSHAL_ERROR_CODE, \"malformed value buffer: unknown {} tag #{{tag}}\")", - e.name - )); - }); - w.line("end"); - }); - out.push_str(&w.finish()); -} diff --git a/crates/weaveffi-gen/src/targets/ruby/docs.rs b/crates/weaveffi-gen/src/targets/ruby/docs.rs deleted file mode 100644 index 95933529..00000000 --- a/crates/weaveffi-gen/src/targets/ruby/docs.rs +++ /dev/null @@ -1,41 +0,0 @@ -//! Doc-comment emission: the Ruby `# ...` spelling of IDL doc strings and -//! `@param` tags. - -use crate::codegen::common::{emit_doc as common_emit_doc, DocCommentStyle}; -use crate::codegen::CodeWriter; -use weaveffi_model::model::ParamBinding; - -/// Emits a Ruby `# ...` doc comment at `indent`. Each input line is prefixed -/// with `# `; blank lines become `#`. -pub(crate) fn emit_doc(out: &mut String, doc: &Option, indent: &str) { - common_emit_doc(out, doc, indent, DocCommentStyle::Hash); -} - -/// Emit one `# @param` tag per documented parameter into the writer, naming -/// each parameter by its emitted Ruby spelling (`rb_param_name`). -/// Continuation lines indent under the tag; blank doc lines become `#`. -pub(crate) fn emit_param_docs(w: &mut CodeWriter, params: &[ParamBinding]) { - for p in params { - if let Some(pdoc) = &p.doc { - let trimmed = pdoc.trim(); - if trimmed.is_empty() { - continue; - } - let mut lines = trimmed.lines(); - if let Some(first) = lines.next() { - w.line(format!( - "# @param {} [Object] {}", - crate::targets::ruby::types::rb_param_name(&p.name), - first - )); - } - for line in lines { - if line.is_empty() { - w.line("#"); - } else { - w.line(format!("# {}", line)); - } - } - } - } -} diff --git a/crates/weaveffi-gen/src/targets/ruby/entities.rs b/crates/weaveffi-gen/src/targets/ruby/entities.rs deleted file mode 100644 index cf261afd..00000000 --- a/crates/weaveffi-gen/src/targets/ruby/entities.rs +++ /dev/null @@ -1,443 +0,0 @@ -//! Entity renderers: plain enums, records, rich enums, interfaces, and the -//! typed error surface of a module's declared error domain. - -use crate::codegen::common::DocCommentStyle; -use crate::codegen::CodeWriter; -use heck::{ToShoutySnakeCase, ToSnakeCase}; -use weaveffi_model::model::{ - EnumBinding, ErrorBinding, FnBinding, InterfaceBinding, ModuleBinding, StructBinding, -}; -use weaveffi_model::plan::ErrorStrategy; - -use crate::targets::ruby::calls::{render_attach_function, render_callable, RbScope}; -use crate::targets::ruby::codec::render_wv_read; -use crate::targets::ruby::docs::emit_doc; -use crate::targets::ruby::types::{rb_field_name, rb_str_literal}; - -/// The snake_case stem of a domain's generated helpers: `KvError` becomes -/// `kv_error`, naming `kv_error_from` and `check_kv_error!`. Domain type -/// names are globally unique (validated), so the helpers can't collide. -fn rb_error_stem(eb: &ErrorBinding) -> String { - eb.type_name.to_snake_case() -} - -/// `{stem}_from`: builds the domain error matching an ABI code. -pub(crate) fn rb_error_factory_name(eb: &ErrorBinding) -> String { - format!("{}_from", rb_error_stem(eb)) -} - -/// `check_{stem}!`: raises the typed domain error for a non-zero out-err slot. -pub(crate) fn rb_error_checker_name(eb: &ErrorBinding) -> String { - format!("check_{}!", rb_error_stem(eb)) -} - -/// The error-check call a callable's out-err slot goes through, per the -/// function's [`ErrorStrategy`]: the module domain's typed checker for -/// [`ErrorStrategy::Throws`], the generic `check_error!` (plain `Error`; -/// producer panics and marshalling failures only) for -/// [`ErrorStrategy::Trap`]. -pub(crate) fn rb_checker_name(f: &FnBinding, error: Option<&ErrorBinding>) -> String { - match (f.error_strategy(), error) { - (ErrorStrategy::Throws, Some(eb)) => rb_error_checker_name(eb), - _ => "check_error!".to_string(), - } -} - -/// Render one module's declared error domain: a domain class subclassing the -/// generic `Error`, one nested subclass per code carrying its stable `CODE` -/// constant, default message, and any declared payload fields as attributes, -/// the code-to-class table, and the factory/checker helpers throwing wrappers -/// route their out-err slots through. Nesting the code classes keeps -/// `KvError::KeyNotFound` spellable and unambiguous even across domains. -/// -/// Domain codes are validated positive-only; the negative range is reserved -/// for the runtime (`-1` generic, `-2` producer panic, `-3` marshalling -/// failure, `-4` a callback-interface implementation that raised, `-5` -/// cancelled). The factory therefore maps only declared codes onto typed -/// classes and lets everything else fall through to `_wv_error` (the -/// generic `Error`, or `Cancelled`). -pub(crate) fn render_error(out: &mut String, module: &ModuleBinding, eb: &ErrorBinding) { - let domain = &eb.type_name; - let factory = rb_error_factory_name(eb); - let checker = rb_error_checker_name(eb); - let table = format!("{}_CODES", eb.type_name.to_shouty_snake_case()); - - let mut w = CodeWriter::two_space().with_depth(1); - w.blank(); - w.line(format!( - "# Base error for the `{}` module's error domain.", - module.path - )); - w.line(format!("class {domain} < Error")); - w.scope(|w| { - for (idx, c) in eb.codes.iter().enumerate() { - if idx > 0 { - w.blank(); - } - let class = weaveffi_model::errors::pascal(&c.name); - let doc = c.doc.clone().unwrap_or_else(|| c.message.clone()); - let mut d = String::new(); - emit_doc(&mut d, &Some(doc), " "); - w.raw(d); - w.block(format!("class {class} < {domain}"), "end", |w| { - w.line(format!("CODE = {}", c.value)); - if !c.fields.is_empty() { - w.blank(); - for f in &c.fields { - let mut fd = String::new(); - emit_doc(&mut fd, &f.doc, " "); - w.raw(fd); - w.line(format!("attr_reader :{}", rb_field_name(&f.name))); - } - } - w.blank(); - let kw: String = c - .fields - .iter() - .map(|f| format!(", {}: nil", rb_field_name(&f.name))) - .collect(); - w.block( - format!( - "def initialize(message = '{}'{kw})", - rb_str_literal(&c.message) - ), - "end", - |w| { - for f in &c.fields { - let field = rb_field_name(&f.name); - w.line(format!("@{field} = {field}")); - } - w.line(format!("super({}, message)", c.value)); - }, - ); - }); - } - }); - w.line("end"); - - w.blank(); - w.line(format!( - "# Maps each ABI code of the {domain} domain to its error class." - )); - w.line(format!("{table} = {{")); - w.scope(|w| { - for c in &eb.codes { - w.line(format!( - "{} => {domain}::{},", - c.value, - weaveffi_model::errors::pascal(&c.name) - )); - } - }); - w.line("}.freeze"); - - w.blank(); - w.line(format!( - "# Builds the {domain} subclass matching `code`, decoding any payload" - )); - w.line("# fields the code declares, or a generic Error for codes outside"); - w.line("# the domain (panics, marshalling, callback failures)."); - w.block( - format!("def self.{factory}(code, message, payload = nil)"), - "end", - |w| { - if eb.codes.iter().any(|c| !c.fields.is_empty()) { - w.line("case code"); - for c in eb.codes.iter().filter(|c| !c.fields.is_empty()) { - let class = weaveffi_model::errors::pascal(&c.name); - w.line(format!("when {}", c.value)); - w.scope(|w| { - w.line("r = WvBufferReader.new(payload || ''.b)"); - for f in &c.fields { - let field = rb_field_name(&f.name); - render_wv_read(w, "r", &format!("_wv_{field}"), &f.ty, 0, ""); - } - w.line("r.expect_end!"); - let kwargs = c - .fields - .iter() - .map(|f| { - let field = rb_field_name(&f.name); - format!("{field}: _wv_{field}") - }) - .collect::>() - .join(", "); - w.line(format!( - "return {domain}::{class}.new({kwargs}) if message.empty?" - )); - w.line(format!("return {domain}::{class}.new(message, {kwargs})")); - }); - } - w.line("end"); - } - w.line(format!("cls = {table}[code]")); - w.line("return _wv_error(code, message) if cls.nil?"); - w.line("message.empty? ? cls.new : cls.new(message)"); - }, - ); - - w.blank(); - w.line(format!( - "# Raises the typed {domain} for a non-zero error slot." - )); - w.block(format!("def self.{checker}(err)"), "end", |w| { - w.line("taken = _wv_take_error(err)"); - w.line(format!("raise {factory}(*taken) if taken")); - }); - out.push_str(&w.finish()); -} - -/// Render one plain C-style enum as a module of integer constants, one -/// `SHOUTY_SNAKE` constant per variant. -pub(crate) fn render_enum(out: &mut String, e: &EnumBinding) { - let mut w = CodeWriter::two_space().with_depth(1); - w.blank(); - let mut d = String::new(); - emit_doc(&mut d, &e.doc, " "); - w.raw(d); - w.line(format!("module {}", e.name)); - w.scope(|w| { - for v in &e.variants { - let mut vd = String::new(); - emit_doc(&mut vd, &v.doc, " "); - w.raw(vd); - w.line(format!("{} = {}", v.name.to_shouty_snake_case(), v.value)); - } - }); - w.line("end"); - out.push_str(&w.finish()); -} - -/// Render one record as a plain Ruby value class: one documented -/// `attr_reader` per field, a keyword-argument `initialize`, and structural -/// `==`. Records are value types: they own no C symbols, no destroy, and no -/// builders; they cross the ABI packed into value buffers by the module's -/// `_wv_write_*`/`_wv_read_*` codec helpers. -pub(crate) fn render_struct_class(out: &mut String, s: &StructBinding) { - let mut w = CodeWriter::two_space().with_depth(1); - w.blank(); - let mut d = String::new(); - emit_doc(&mut d, &s.doc, " "); - w.raw(d); - w.line(format!("class {}", s.name)); - w.scope(|w| { - for (idx, f) in s.fields.iter().enumerate() { - if idx > 0 { - w.blank(); - } - let mut fd = String::new(); - emit_doc(&mut fd, &f.doc, " "); - w.raw(fd); - w.line(format!("attr_reader :{}", rb_field_name(&f.name))); - } - w.blank(); - let kw = s - .fields - .iter() - .map(|f| format!("{}:", rb_field_name(&f.name))) - .collect::>() - .join(", "); - let open = if kw.is_empty() { - "def initialize".to_string() - } else { - format!("def initialize({kw})") - }; - w.block(open, "end", |w| { - for f in &s.fields { - let field = rb_field_name(&f.name); - w.line(format!("@{field} = {field}")); - } - }); - w.blank(); - w.line("# Structural equality over every field."); - w.block("def ==(other)", "end", |w| { - w.line(format!("return false unless other.is_a?({})", s.name)); - for f in &s.fields { - let field = rb_field_name(&f.name); - w.line(format!("return false unless {field} == other.{field}")); - } - w.line("true"); - }); - }); - w.line("end"); - out.push_str(&w.finish()); -} - -/// Render one rich (algebraic) enum as an idiomatic tagged class hierarchy: -/// a base class exposing `tag`, plus one nested value-class subclass per -/// variant carrying that variant's fields (documented `attr_reader`s, a -/// keyword-argument `initialize`, structural `==`). Rich enums own no C -/// symbols; they cross the ABI packed into value buffers as an `i32` tag -/// followed by the active variant's fields in declaration order. -pub(crate) fn render_rich_enum_class(out: &mut String, e: &EnumBinding) { - let mut w = CodeWriter::two_space().with_depth(1); - w.blank(); - let mut d = String::new(); - emit_doc(&mut d, &e.doc, " "); - w.raw(d); - w.line(format!("class {}", e.name)); - w.scope(|w| { - w.line("# The active variant's integer tag."); - w.block("def tag", "end", |w| { - w.line("self.class::TAG"); - }); - for v in &e.variants { - w.blank(); - let mut vd = String::new(); - emit_doc(&mut vd, &v.doc, " "); - w.raw(vd); - w.line(format!("class {} < {}", v.name, e.name)); - w.scope(|w| { - w.line(format!("TAG = {}", v.value)); - if !v.fields.is_empty() { - for f in &v.fields { - w.blank(); - let mut fd = String::new(); - emit_doc(&mut fd, &f.doc, " "); - w.raw(fd); - w.line(format!("attr_reader :{}", rb_field_name(&f.name))); - } - w.blank(); - let kw = v - .fields - .iter() - .map(|f| format!("{}:", rb_field_name(&f.name))) - .collect::>() - .join(", "); - w.block(format!("def initialize({kw})"), "end", |w| { - for f in &v.fields { - let field = rb_field_name(&f.name); - w.line(format!("@{field} = {field}")); - } - }); - } - w.blank(); - w.line("# Structural equality over the variant and its fields."); - w.block("def ==(other)", "end", |w| { - w.line(format!("return false unless other.is_a?({})", v.name)); - for f in &v.fields { - let field = rb_field_name(&f.name); - w.line(format!("return false unless {field} == other.{field}")); - } - w.line("true"); - }); - }); - w.line("end"); - } - }); - w.line("end"); - out.push_str(&w.finish()); -} - -/// Declare the FFI bindings for one interface: the clone and destroy -/// lifecycle symbols plus every constructor, method, and static through the -/// shared attach path. Methods carry their implicit leading `self` pointer -/// slot in the precomputed ABI signatures, so no special casing is needed -/// here. `blocking` is the API-wide GVL policy (see -/// [`render_attach_function`]); destroying an object can release callback -/// implementations it holds, so it follows the same policy. -pub(crate) fn render_interface_ffi( - out: &mut String, - module: &ModuleBinding, - i: &InterfaceBinding, - blocking: bool, -) { - let mut w = CodeWriter::two_space().with_depth(1); - w.line(format!( - "attach_function :{}, [:pointer], :pointer", - i.clone_symbol - )); - w.line(format!( - "attach_function :{}, [:pointer], :void{}", - i.destroy_symbol, - if blocking { ", blocking: true" } else { "" } - )); - out.push_str(&w.finish()); - for f in i - .constructors - .iter() - .chain(i.methods.iter()) - .chain(i.statics.iter()) - { - render_attach_function(out, module, f, blocking); - } -} - -/// Render one interface as a reference-counted wrapper class on the -/// runtime's `WvObject` base (which supplies `handle`, `close`, `closed?`, -/// `dup`/`clone`, and the call pinning). A `{Name}Ptr < FFI::AutoPointer` -/// subclass owns exactly one strong reference and releases it through the -/// interface's `_destroy` symbol, either from `close` or, as a backstop, -/// from the GC finalizer. A constructor named `new` becomes `initialize`; -/// every other constructor becomes a class-method factory; methods borrow -/// the wrapper's pointer as the leading C argument; statics are class -/// methods. `_from_ptr` adopts a reference the producer handed over (a -/// return, an async result, an iterator element, a buffer token) without -/// re-running `initialize`. -pub(crate) fn render_interface_class( - out: &mut String, - module: &ModuleBinding, - i: &InterfaceBinding, - rb_module_name: &str, -) { - let ptr_class = format!("{}Ptr", i.name); - let mut w = CodeWriter::two_space().with_depth(1); - w.blank(); - w.line("# @api private"); - w.line(format!( - "# Owns one strong reference to a {}; releases it exactly once.", - i.name - )); - w.block( - format!("class {ptr_class} < FFI::AutoPointer"), - "end", - |w| { - w.block("def self.release(ptr)", "end", |w| { - w.line(format!("{rb_module_name}.{}(ptr)", i.destroy_symbol)); - }); - }, - ); - w.blank(); - w.doc(&i.doc, DocCommentStyle::Hash); - if let Some(msg) = &i.deprecated { - w.line(format!("# @deprecated {msg}")); - } - w.line(format!("class {} < WvObject", i.name)); - w.scope(|w| { - w.line(format!("WV_PTR = {ptr_class}")); - if !i.constructors.iter().any(|c| c.name == "new") { - w.line("private_class_method :new"); - } - w.blank(); - w.line("# @api private"); - w.block("def self._wv_clone(ptr)", "end", |w| { - w.line(format!("{rb_module_name}.{}(ptr)", i.clone_symbol)); - }); - }); - out.push_str(&w.finish()); - - // Members render at class depth through the shared callable paths, so - // sync, async, and iterator members reuse the free-function marshalling. - let scope_of = |f: &FnBinding, ctor: bool| -> RbScope { - let module_name = rb_module_name; - if ctor && f.name == "new" { - RbScope::Init { module_name } - } else if ctor { - RbScope::Factory { module_name } - } else if f.has_self { - RbScope::Method { module_name } - } else { - RbScope::Static { module_name } - } - }; - for c in &i.constructors { - render_callable(out, module, c, &scope_of(c, true)); - } - for f in i.methods.iter().chain(i.statics.iter()) { - render_callable(out, module, f, &scope_of(f, false)); - } - - let mut close = CodeWriter::two_space().with_depth(1); - close.line("end"); - out.push_str(&close.finish()); -} diff --git a/crates/weaveffi-gen/src/targets/ruby/mod.rs b/crates/weaveffi-gen/src/targets/ruby/mod.rs deleted file mode 100644 index 703b7000..00000000 --- a/crates/weaveffi-gen/src/targets/ruby/mod.rs +++ /dev/null @@ -1,748 +0,0 @@ -//! Ruby (ffi gem) binding generator. -//! -//! Emits a gem that binds the C ABI (revision 3) with the `ffi` gem: -//! `lib/{prefix}.rb` (the generated bindings, required as `{prefix}`), -//! `lib/{prefix}/runtime.rb` (the fixed runtime from `runtime/runtime.rb`), -//! a `{name}.gemspec`, and a README. Everything lives in one Ruby module, -//! `PascalCase(name)` unless configured. Interfaces become reference-counted -//! wrapper classes (`close` plus a GC finalizer backstop, `dup`/`clone` for a -//! second reference), records and rich enums become value classes packed -//! into value buffers, async functions block on a queue fed by a -//! module-level completion trampoline (cancellable ones take a `cancel:` -//! token), iterators are lazy `Enumerator`s, and callback interfaces are -//! duck-typed modules backed by one static vtable of pinned `FFI::Function` -//! trampolines per interface. - -mod callbacks; -mod calls; -mod codec; -mod docs; -mod entities; -mod package; -mod runtime; -mod types; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use camino::Utf8Path; -use serde::{Deserialize, Serialize}; -use weaveffi_model::model::{checksum_symbol, BindingModel}; -use weaveffi_model::resolved::ResolvedApi; - -use crate::targets::ruby::calls::{render_attach_function, render_callable, RbScope}; -use crate::targets::ruby::entities::{ - render_enum, render_error, render_interface_class, render_interface_ffi, - render_rich_enum_class, render_struct_class, -}; -use crate::targets::ruby::package::{ - render_gemspec, render_packaged_gemspec, render_packaged_readme, render_readme, GemNames, -}; -use crate::targets::ruby::runtime::render_runtime; - -/// Per-target configuration for [`RubyGenerator`]. -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct RubyConfig { - /// Top-level Ruby module name (default: the package name in PascalCase). - pub module_name: Option, - /// Gem name written into the gemspec (default: the package name). - pub gem_name: Option, - /// When `true` (the default), strip the IR module name prefix from - /// emitted Ruby method names, so a `contacts` module exports - /// `create_contact` rather than `contacts_create_contact`. Set to - /// `false` to restore module-prefixed names. - pub strip_module_prefix: bool, - /// Basename of the IDL the CLI was invoked with. - #[serde(skip)] - pub input_basename: Option, -} - -impl Default for RubyConfig { - fn default() -> Self { - Self { - module_name: None, - gem_name: None, - strip_module_prefix: true, - input_basename: None, - } - } -} - -impl RubyConfig { - /// Returns the input IDL basename embedded in generated file headers, - /// falling back to `"api.yml"`. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } - - /// The gem, module, and require names for `api`: the identity's, unless - /// overridden here. - fn names<'a>(&self, api: &'a ResolvedApi) -> GemNames<'a> { - let identity = api.identity(); - let configured = |v: &Option| { - v.as_deref() - .map(str::trim) - .filter(|s| !s.is_empty()) - .map(str::to_string) - }; - GemNames { - identity, - gem: configured(&self.gem_name).unwrap_or_else(|| identity.name.clone()), - module: configured(&self.module_name).unwrap_or_else(|| identity.pascal_name()), - } - } -} - -/// Ruby backend: emits an `ffi`-gem package binding the C ABI exposed by the -/// underlying cdylib. -pub struct RubyGenerator; - -impl LanguageBackend for RubyGenerator { - type Config = RubyConfig; - - fn name(&self) -> &'static str { - "ruby" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - let names = config.names(api); - let input_basename = config.input_basename(); - let dir = out_dir.join("ruby"); - let lib_dir = dir.join("lib"); - let lib_file = format!("{}.rb", names.require()); - vec![ - OutputFile::new( - lib_dir.join(&lib_file), - render_bindings(model, &names, config, &lib_file), - ), - OutputFile::new( - lib_dir.join(names.require()).join("runtime.rb"), - render_runtime(names.identity, &names.module, input_basename, "runtime.rb"), - ), - OutputFile::new( - dir.join(names.gemspec_file()), - render_gemspec(&names, input_basename), - ), - OutputFile::new(dir.join("README.md"), render_readme(&names, input_basename)), - ] - } - - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let names = config.names(api); - let input_basename = config.input_basename(); - let lib_file = format!("{}.rb", names.require()); - let bindings = render_bindings(model, &names, config, &lib_file); - let runtime = render_runtime(names.identity, &names.module, input_basename, "runtime.rb"); - let readme = render_packaged_readme(&names, input_basename); - - let ruby_dir = out_dir.join("ruby"); - let mut files = Vec::new(); - for nb in &ctx.binaries.binaries { - let platform = nb.platform; - // RubyGems has no platform string for Android or wasm32 builds, - // so those binaries have no gem to land in. - let Some(ruby_platform) = platform.ruby_platform() else { - continue; - }; - let gem_dir = ruby_dir.join(platform.id()); - let lib_dir = gem_dir.join("lib"); - files.push(PackagedFile::text( - lib_dir.join(&lib_file), - bindings.clone(), - )); - files.push(PackagedFile::text( - lib_dir.join(names.require()).join("runtime.rb"), - runtime.clone(), - )); - // The runtime loader looks for the bundled library under - // `lib/native/` by its platform file name. - files.push(PackagedFile::copy( - lib_dir - .join("native") - .join(platform.lib_filename(&names.identity.library)), - nb.source.clone(), - )); - files.push(PackagedFile::text( - gem_dir.join(names.gemspec_file()), - render_packaged_gemspec(&names, ruby_platform, input_basename), - )); - files.push(PackagedFile::text( - gem_dir.join("README.md"), - readme.clone(), - )); - } - Some(files) - } -} - -/// Render `lib/{prefix}.rb`: the runtime require, the contract check, then -/// each module's typed error surface, entities, codecs, FFI attachments, -/// callback interfaces, interface classes, and free functions. -fn render_bindings( - model: &BindingModel, - names: &GemNames, - config: &RubyConfig, - lib_file: &str, -) -> String { - let module_name = &names.module; - // Any call can reach a stored callback implementation, so an API with - // callback interfaces releases the GVL on every call. - let blocking = model.has_callback_interfaces(); - let mut out = render_prelude(CommentStyle::Hash, config.input_basename()); - out.push_str("# frozen_string_literal: true\n\n"); - out.push_str(&format!( - "require_relative '{}/runtime'\n\n", - names.require() - )); - out.push_str(&format!("# Ruby bindings for {}.\n", names.identity.name)); - out.push_str(&format!("module {module_name}\n")); - out.push_str(" # Contract checksums of the top-level modules these bindings were\n"); - out.push_str(" # generated from; a library built from a different API fails to load.\n"); - out.push_str(" _wv_check_contract!(\n"); - for root in model.roots() { - let checksum = root.checksum.expect("top-level modules carry a checksum"); - out.push_str(&format!( - " '{}' => [:{}, 0x{checksum:016x}],\n", - root.name, - checksum_symbol(&model.prefix, &root.name) - )); - } - out.push_str(" )\n"); - for m in &model.modules { - out.push_str(&format!("\n # === Module: {} ===\n", m.dot_path)); - // The typed error surface comes first so the domain class exists - // before any wrapper references its checker. - if let Some(eb) = m.error.as_ref().filter(|e| e.declared_here) { - render_error(&mut out, m, eb); - } - for e in &m.enums { - // A plain C-style enum is a module of integer constants; a rich - // (algebraic) enum is a tagged value-class hierarchy packed into - // value buffers by the codec helpers below. - if e.is_rich() { - render_rich_enum_class(&mut out, e); - } else { - render_enum(&mut out, e); - } - } - for s in &m.structs { - render_struct_class(&mut out, s); - } - // Value-buffer codecs: one pack/unpack pair per record and rich enum. - for s in &m.structs { - codec::render_struct_codec(&mut out, s); - } - for e in m.enums.iter().filter(|e| e.is_rich()) { - codec::render_rich_enum_codec(&mut out, e); - } - let attaches = !m.interfaces.is_empty() || !m.functions.is_empty(); - if attaches { - out.push('\n'); - } - for i in &m.interfaces { - render_interface_ffi(&mut out, m, i, blocking); - } - for f in &m.functions { - render_attach_function(&mut out, m, f, blocking); - } - // Callback interfaces precede interfaces and functions because their - // static vtables are what interface members and free functions pass. - for cb in &m.callback_interfaces { - callbacks::render_callback_interface(&mut out, cb, &model.prefix); - } - for i in &m.interfaces { - render_interface_class(&mut out, m, i, module_name); - } - for f in &m.functions { - let scope = RbScope::Free { - module_path: &m.path, - strip_module_prefix: config.strip_module_prefix, - }; - render_callable(&mut out, m, f, &scope); - } - } - out.push_str("end\n\n"); - out.push_str(&render_trailer(CommentStyle::Hash, lib_file)); - out -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::platform::{BinarySet, Platform}; - use weaveffi_model::ir::{ - Api, CallbackInterfaceDef, Function, InterfaceDef, Module, Param, StructDef, StructField, - TypeRef, - }; - use weaveffi_model::pkg::Identity; - - fn param(name: &str, ty: TypeRef) -> Param { - Param { - name: name.into(), - ty, - doc: None, - } - } - - fn func(name: &str, params: Vec, returns: Option) -> Function { - Function { - name: name.into(), - params, - returns, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - } - } - - fn field(name: &str, ty: TypeRef) -> StructField { - StructField { - name: name.into(), - ty, - doc: None, - } - } - - fn named(name: &str) -> TypeRef { - TypeRef::Named(name.into()) - } - - /// One module exercising every object shape plus strings, an iterator, - /// a callback interface, and a cancellable async method, under the - /// identity `kv-store` (prefix `kv_store`, module `KvStore`). - fn fixture() -> ResolvedApi { - let mut compact = func("compact", vec![], Some(TypeRef::I64)); - compact.r#async = true; - compact.cancellable = true; - let kv = Module { - name: "kv".into(), - doc: None, - functions: vec![ - func( - "maybe_store", - vec![param("store", TypeRef::Optional(Box::new(named("Store"))))], - Some(TypeRef::Optional(Box::new(named("Store")))), - ), - func( - "scan", - vec![], - Some(TypeRef::Iterator(Box::new(named("Store")))), - ), - func( - "subscribe", - vec![param("listener", named("Listener"))], - None, - ), - func( - "describe", - vec![param("bundle", named("Bundle"))], - Some(named("Bundle")), - ), - ], - interfaces: vec![InterfaceDef { - name: "Store".into(), - doc: Some("A key-value store.".into()), - deprecated: None, - constructors: vec![func("new", vec![param("path", TypeRef::StringUtf8)], None)], - methods: vec![ - func( - "get", - vec![param("key", TypeRef::StringUtf8)], - Some(TypeRef::StringUtf8), - ), - compact, - ], - statics: vec![], - }], - callback_interfaces: vec![CallbackInterfaceDef { - name: "Listener".into(), - doc: Some("Receives store events.".into()), - deprecated: None, - methods: vec![ - func( - "on_message", - vec![ - param("text", TypeRef::StringUtf8), - param("weight", TypeRef::I32), - ], - Some(TypeRef::Bool), - ), - func( - "on_bundle", - vec![ - param("bundle", named("Bundle")), - param("store", named("Store")), - param("alt", TypeRef::Optional(Box::new(named("Store")))), - ], - None, - ), - ], - }], - structs: vec![StructDef { - name: "Bundle".into(), - doc: None, - deprecated: None, - fields: vec![ - field("primary", named("Store")), - field("extras", TypeRef::List(Box::new(named("Store")))), - ], - }], - enums: vec![], - errors: None, - modules: vec![], - }; - ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![kv], - }) - .with_identity(Identity::named("kv-store")) - } - - fn generate(config: &RubyConfig) -> Vec { - let api = fixture(); - let model = BindingModel::build(&api); - RubyGenerator.files(&api, &model, Utf8Path::new("out"), config) - } - - fn file<'a>(files: &'a [OutputFile], path: &str) -> &'a str { - &files - .iter() - .find(|f| f.path.as_str() == path) - .unwrap_or_else(|| { - panic!( - "no {path} in {:?}", - files.iter().map(|f| &f.path).collect::>() - ) - }) - .contents - } - - fn bindings() -> String { - file( - &generate(&RubyConfig::default()), - "out/ruby/lib/kv_store.rb", - ) - .to_string() - } - - fn assert_has(src: &str, needle: &str) { - assert!(src.contains(needle), "missing `{needle}` in:\n{src}"); - } - - /// Nothing but the generated-file banner mentions WeaveFFI. - fn assert_unbranded(src: &str) { - let banner = |l: &str| { - l.contains("Generated by WeaveFFI") || l.contains("To regenerate: weaveffi generate") - }; - for line in src.lines().filter(|l| !banner(l)) { - assert!( - !line.to_ascii_lowercase().contains("weaveffi"), - "branded line: {line}" - ); - } - } - - #[test] - fn layout_follows_the_identity() { - let files = generate(&RubyConfig::default()); - let paths: Vec<&str> = files.iter().map(|f| f.path.as_str()).collect(); - assert_eq!( - paths, - [ - "out/ruby/lib/kv_store.rb", - "out/ruby/lib/kv_store/runtime.rb", - "out/ruby/kv-store.gemspec", - "out/ruby/README.md", - ] - ); - let gemspec = file(&files, "out/ruby/kv-store.gemspec"); - assert_has(gemspec, "s.name = 'kv-store'"); - assert_has(gemspec, "s.authors = ['kv-store']"); - let src = file(&files, "out/ruby/lib/kv_store.rb"); - assert_has(src, "require_relative 'kv_store/runtime'"); - assert_has(src, "module KvStore\n"); - for f in &files { - assert_unbranded(&f.contents); - } - } - - #[test] - fn config_overrides_gem_and_module_names() { - let config = RubyConfig { - gem_name: Some("acme-kv".into()), - module_name: Some("Acme".into()), - ..RubyConfig::default() - }; - let files = generate(&config); - assert_has( - file(&files, "out/ruby/acme-kv.gemspec"), - "s.name = 'acme-kv'", - ); - // The require path stays the identity prefix. - let src = file(&files, "out/ruby/lib/kv_store.rb"); - assert_has(src, "module Acme\n"); - assert_has(src, "Acme.kv_store_kv_Store_destroy(ptr)"); - assert_has( - file(&files, "out/ruby/lib/kv_store/runtime.rb"), - "module Acme\n", - ); - } - - #[test] - fn runtime_loads_the_identity_library_and_checks_abi_3() { - let files = generate(&RubyConfig::default()); - let rt = file(&files, "out/ruby/lib/kv_store/runtime.rb"); - assert_has(rt, "ENV['KV_STORE_LIBRARY']"); - assert_has(rt, "'libkv_store.dylib'"); - assert_has(rt, "'libkv_store.so'"); - assert_has(rt, "'kv_store.dll'"); - assert_has(rt, &format!("ABI_VERSION = {}", crate::cabi::ABI_VERSION)); - assert_has(rt, "attach_function :kv_store_abi_version, [], :uint32"); - assert_has( - rt, - "attach_function :kv_store_free_bytes, [:pointer, :size_t], :void", - ); - assert_has( - rt, - "attach_function :kv_store_cancel_token_create, [], :pointer", - ); - assert_has(rt, "CANCELLED_ERROR_CODE = -5"); - assert_has(rt, "class Cancelled < Error"); - assert!(!rt.contains("free_string"), "{rt}"); - assert!(!rt.contains("{{"), "unsubstituted placeholder in:\n{rt}"); - } - - #[test] - fn bindings_check_every_root_checksum_at_load() { - let api = fixture(); - let model = BindingModel::build(&api); - let checksum = model.roots().next().unwrap().checksum.unwrap(); - assert_has( - &bindings(), - &format!("'kv' => [:kv_store_kv_checksum, 0x{checksum:016x}],"), - ); - } - - #[test] - fn interface_wrapper_owns_one_reference() { - let src = bindings(); - assert_has( - &src, - "attach_function :kv_store_kv_Store_clone, [:pointer], :pointer", - ); - assert_has( - &src, - "attach_function :kv_store_kv_Store_destroy, [:pointer], :void, blocking: true", - ); - assert_has(&src, "class StorePtr < FFI::AutoPointer"); - assert_has(&src, "class Store < WvObject\n WV_PTR = StorePtr\n"); - assert_has(&src, "KvStore.kv_store_kv_Store_clone(ptr)"); - // The constructor adopts the returned reference; methods pin `self`. - assert_has(&src, "def initialize(path)"); - assert_has(&src, "_wv_init(result)"); - assert_has(&src, "KvStore._wv_pin(self) do |_wv_self|"); - } - - #[test] - fn strings_cross_as_pointer_and_length() { - let src = bindings(); - assert_has(&src, "key_s = KvStore._wv_str(key)"); - assert_has( - &src, - "result = KvStore.kv_store_kv_Store_get(_wv_self, key_s, key_s.bytesize, out_len, err)", - ); - assert_has( - &src, - "KvStore._wv_take_string(result, out_len.read(:size_t))", - ); - // Callback string arguments are borrowed (ptr, len) runs. - assert_has(&src, "do |ctx, text_ptr, text_len, weight, out_err|"); - assert_has( - &src, - "text_v = _wv_borrow_bytes(text_ptr, text_len).force_encoding(Encoding::UTF_8)", - ); - } - - #[test] - fn nullable_objects_map_to_nil_in_both_directions() { - let src = bindings(); - assert_has(&src, "def self.maybe_store(store)"); - assert_has(&src, "_wv_pin(store) do |_wv_store|"); - assert_has(&src, "result.null? ? nil : Store._from_ptr(result)"); - } - - #[test] - fn records_reserve_object_tokens_and_seal_them_before_the_call() { - let src = bindings(); - assert_has(&src, "w.write_object(v.primary)"); - assert_has(&src, "bundle_s = _wv_seal(bundle_w).first"); - assert_has(&src, "_wv_primary = Store._from_ptr(r.read_object_token)"); - } - - #[test] - fn iterator_is_a_lazy_enumerator_adopting_objects() { - let src = bindings(); - assert_has(&src, "Enumerator.new do |y|"); - assert_has( - &src, - "has_item = kv_store_kv_ScanIterator_next(iter, out_item, err)", - ); - assert_has(&src, "value = Store._from_ptr(item)"); - assert_has( - &src, - "kv_store_kv_ScanIterator_destroy(iter) unless iter.null?", - ); - } - - #[test] - fn cancellable_async_takes_a_cancel_token() { - let src = bindings(); - assert_has(&src, "def compact(cancel: nil)"); - assert_has(&src, "own_token = CancelToken.new if cancel.nil?"); - assert_has( - &src, - "KvStore.kv_store_kv_Store_compact(_wv_self, token._wv_ptr, KV_STORE_KV_STORE_COMPACT_CALLBACK, ctx)", - ); - assert_has(&src, "KvStore._wv_async_wait(queue, token)"); - assert_has(&src, "own_token&.close"); - assert_has( - &src, - "attach_function :kv_store_kv_Store_compact, [:pointer, :pointer, :pointer, :pointer], :void, blocking: true", - ); - assert_has( - &src, - "KV_STORE_KV_STORE_COMPACT_CALLBACK = FFI::Function.new(:void, [:pointer, :pointer, :int64]) do |ctx, err, result|", - ); - assert_has( - &src, - "rescue Exception => e # rubocop:disable Lint/RescueException\n queue << e", - ); - } - - #[test] - fn callback_interface_renders_module_vtable_and_trampolines() { - let src = bindings(); - assert_has(&src, "module Listener\n"); - assert_has(&src, "raise NotImplementedError"); - assert_has(&src, "class WvListenerVtable < FFI::Struct"); - assert_has( - &src, - "layout :on_message, :pointer,\n :on_bundle, :pointer,\n :free, :pointer", - ); - assert_has(&src, "WV_LISTENER_ON_MESSAGE = FFI::Function.new(:bool,"); - assert_has(&src, "impl.on_message(text_v, weight_v) ? true : false"); - // Objects are adopted before anything that can raise. - let bundle = src.find("WV_LISTENER_ON_BUNDLE").unwrap(); - let tail = &src[bundle..]; - assert!( - tail.find("store_v = Store._from_ptr(store)").unwrap() - < tail.find("bundle_r = ").unwrap() - ); - assert_has(&src, "alt_v = alt.null? ? nil : Store._from_ptr(alt)"); - assert_has(&src, "WV_LISTENER_VTABLE[:free] = WV_LISTENER_FREE"); - assert_eq!(src.matches("= WvListenerVtable.new").count(), 1); - // Passing an implementation registers it right before the call. - assert_has(&src, "listener_ctx = _wv_cb_register(listener)"); - assert_has( - &src, - "kv_store_kv_subscribe(listener_ctx, WV_LISTENER_VTABLE.to_ptr, err)", - ); - } - - #[test] - fn calls_keep_the_gvl_without_callback_interfaces() { - let api = ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![Module { - name: "math".into(), - doc: None, - functions: vec![func( - "add", - vec![param("a", TypeRef::I32), param("b", TypeRef::Bool)], - Some(TypeRef::I32), - )], - interfaces: vec![], - callback_interfaces: vec![], - structs: vec![], - enums: vec![], - errors: None, - modules: vec![], - }], - }) - .with_identity(Identity::named("calc")); - let model = BindingModel::build(&api); - let files = RubyGenerator.files(&api, &model, Utf8Path::new("out"), &RubyConfig::default()); - let src = file(&files, "out/ruby/lib/calc.rb").to_string(); - assert_has( - &src, - "attach_function :calc_math_add, [:int32, :bool, :pointer], :int32\n", - ); - assert_has(&src, "calc_math_add(a, (b ? true : false), err)"); - } - - #[test] - fn rendering_is_deterministic() { - assert_eq!(bindings(), bindings()); - } - - #[test] - fn package_skips_platforms_without_a_gem_string() { - let api = fixture(); - let model = BindingModel::build(&api); - let mut binaries = BinarySet::new("kv_store"); - binaries.insert(Platform::MacosArm64, "/tmp/darwin-arm64/libkv_store.dylib"); - binaries.insert(Platform::AndroidArm64, "/tmp/android-arm64/libkv_store.so"); - binaries.insert(Platform::Wasm32, "/tmp/wasm32/kv_store.wasm"); - let ctx = PackageContext { - binaries: &binaries, - input_basename: Some("kv.yml"), - }; - let files = RubyGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &RubyConfig::default(), - ) - .expect("ruby supports packaging"); - let paths: Vec<&str> = files.iter().map(|f| f.path.as_str()).collect(); - assert!( - paths - .iter() - .all(|p| p.starts_with("out/ruby/darwin-arm64/")), - "{paths:?}" - ); - assert!(paths.contains(&"out/ruby/darwin-arm64/lib/native/libkv_store.dylib")); - assert!(paths.contains(&"out/ruby/darwin-arm64/lib/kv_store/runtime.rb")); - let gemspec = files - .iter() - .find(|f| f.path.as_str().ends_with("kv-store.gemspec")) - .unwrap(); - let crate::package::FileContent::Text(text) = &gemspec.content else { - panic!("gemspec is text"); - }; - assert_has(text, "s.platform = 'arm64-darwin'"); - } -} diff --git a/crates/weaveffi-gen/src/targets/ruby/types.rs b/crates/weaveffi-gen/src/targets/ruby/types.rs deleted file mode 100644 index 10636283..00000000 --- a/crates/weaveffi-gen/src/targets/ruby/types.rs +++ /dev/null @@ -1,90 +0,0 @@ -//! Ruby type mapping and naming: the FFI-gem vocabulary for ABI slots, the -//! identifier policy for user-chosen IDL names, and string escaping for Ruby -//! literals. - -use crate::lang::{escape_ident, RUBY_KEYWORDS}; -use heck::ToSnakeCase; -use weaveffi_model::abi::{AbiParam, CType}; -use weaveffi_model::model::Ty; - -/// The Ruby spelling of a user-chosen parameter name: snake_case via heck, -/// then keyword-escaped (a reserved name like `end` gains a trailing `_`). -/// IDL names are usually already snake, so the case conversion is a safety -/// net for camelCase inputs. -pub(crate) fn rb_param_name(name: &str) -> String { - escape_ident(&name.to_snake_case(), RUBY_KEYWORDS) -} - -/// The Ruby spelling of a user-chosen field name: verbatim unless it's a -/// Ruby reserved word, in which case it gains a trailing `_` so keyword -/// arguments and locals derived from it stay parseable. -pub(crate) fn rb_field_name(name: &str) -> String { - escape_ident(name, RUBY_KEYWORDS) -} - -/// Maps a shared ABI [`CType`] onto its Ruby FFI type symbol. Every pointer -/// (strings and buffers cross as `const uint8_t*`, objects and out-slots as -/// typed pointers) is `:pointer`; `bool` is the 1-byte C `bool`. -pub(crate) fn rb_ffi_type(ty: &CType) -> &'static str { - match ty { - CType::Bool => ":bool", - CType::Int8 => ":int8", - CType::Int16 => ":int16", - CType::Int32 | CType::Enum { .. } => ":int32", - CType::Uint8 => ":uint8", - CType::Uint16 => ":uint16", - CType::Uint32 => ":uint32", - CType::Int64 => ":int64", - CType::Uint64 => ":uint64", - CType::Float => ":float", - CType::Double => ":double", - CType::Size => ":size_t", - CType::Void => ":void", - _ => ":pointer", - } -} - -/// Map lowered ABI slots onto Ruby FFI type symbols. -pub(crate) fn rb_abi_types(params: &[AbiParam]) -> Vec { - params - .iter() - .map(|p| rb_ffi_type(&p.ty).to_string()) - .collect() -} - -/// The `FFI::MemoryPointer` type of a direct iterator element out-slot, read -/// back with `Pointer#read`. -pub(crate) fn rb_direct_type(ty: &Ty) -> &'static str { - match ty { - Ty::Bool => ":bool", - Ty::I8 => ":int8", - Ty::I16 => ":int16", - Ty::I32 | Ty::Enum(_) => ":int32", - Ty::U8 => ":uint8", - Ty::U16 => ":uint16", - Ty::U32 => ":uint32", - Ty::I64 => ":int64", - Ty::U64 => ":uint64", - Ty::F32 => ":float", - Ty::F64 => ":double", - other => unreachable!("{other} is not a direct type"), - } -} - -/// The Ruby literal a callback-interface trampoline returns to the producer -/// after its implementation raised: the zero value of the method's direct -/// return type, or `nil` for a void method. -pub(crate) fn rb_direct_default(ty: Option<&Ty>) -> &'static str { - match ty { - None => "nil", - Some(Ty::Bool) => "false", - Some(Ty::F32 | Ty::F64) => "0.0", - Some(_) => "0", - } -} - -/// Escape a string for embedding in a single-quoted Ruby literal (the two -/// characters with meaning there: backslash and the quote itself). -pub(crate) fn rb_str_literal(s: &str) -> String { - s.replace('\\', "\\\\").replace('\'', "\\'") -} diff --git a/crates/weaveffi-gen/src/targets/swift/callbacks.rs b/crates/weaveffi-gen/src/targets/swift/callbacks.rs deleted file mode 100644 index 4bd5729d..00000000 --- a/crates/weaveffi-gen/src/targets/swift/callbacks.rs +++ /dev/null @@ -1,276 +0,0 @@ -//! Callback-interface rendering: the Swift `protocol` a consumer implements, -//! the box class that retains one implementation across the C boundary, and -//! the process-wide vtable of `@convention(c)` trampolines the library calls -//! through. -//! -//! Every clause of [`CallbackProtocol`] is rendered here: one static vtable -//! per interface, a pointer-keyed context (the retained box), arguments -//! received per [`RetPass`] (borrowed strings, bytes, and buffers are copied -//! or decoded; object arguments are adopted), and a `do`/`catch` around every -//! implementation call so a thrown Swift error is reported through the -//! runtime's `error_set` with the foreign error code instead of unwinding -//! through the C frame. - -use crate::cabi::c_param_name; -use crate::codegen::common::DocCommentStyle; -use crate::codegen::CodeWriter; -use crate::lang; -use crate::utils::local_type_name; -use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding, Ty}; -use weaveffi_model::plan::{CallbackProtocol, RetPass}; - -use crate::targets::swift::codec::decode_closure; -use crate::targets::swift::docs::emit_fn_doc; -use crate::targets::swift::types::{ - callback_box_name, callback_vtable_name, deprecated_attr, swift_ident, SwiftCtx, -}; - -/// The Swift spelling of one trampoline closure formal: the ABI slot name, -/// keyword-escaped (a param named `in` yields a slot named `in`, which can't -/// bind as a closure formal unescaped). -fn slot_ident(name: &str) -> String { - lang::escape_ident(name, lang::SWIFT_KEYWORDS) -} - -/// Clone a callback method with its parameter names camel-cased and -/// keyword-escaped, so the protocol requirement's argument labels and the -/// trampoline's call site agree. The ABI slot names are left untouched. -fn camel_method(m: &CallbackMethodBinding) -> CallbackMethodBinding { - let mut m = m.clone(); - for p in &mut m.params { - p.name = swift_ident(&p.name); - } - m -} - -/// The expression handing one trampoline argument to the implementation, -/// received per its [`RetPass`]: borrowed strings, bytes, and buffers are -/// copied or decoded before the implementation runs (the library owns them -/// only for the dispatch); object arguments carry one strong reference the -/// new wrapper adopts. -fn receive_arg(w: &CodeWriter, p: &ParamBinding, rp: &RetPass, ctx: &SwiftCtx) -> String { - let n0 = slot_ident(&p.abi[0].name); - let n1 = || slot_ident(&p.abi[1].name); - match rp { - RetPass::Direct => match &p.ty { - Ty::Enum(name) => format!( - "wvEnumCase({}.self, numericCast({n0}.rawValue))", - ctx.ty_name(local_type_name(name)) - ), - _ => n0, - }, - RetPass::String => format!("wvBorrowString({n0}, {})", n1()), - RetPass::Bytes => format!("wvBorrowBytes({n0}, {})", n1()), - RetPass::Buffer => { - let decode = decode_closure(w, &p.ty, ctx); - if decode.starts_with('{') { - format!("wvBorrowBuffer({n0}, {}) {decode}", n1()) - } else { - format!("wvBorrowBuffer({n0}, {}, {decode})", n1()) - } - } - RetPass::Object { nullable, .. } => { - let wrapper = ctx.ty_name(local_type_name( - p.ty.interface_name() - .expect("object argument names an interface"), - )); - if *nullable { - format!("{n0}.map {{ {wrapper}(ptr: $0) }}") - } else { - format!("{wrapper}(ptr: wvNonNull({n0}))") - } - } - RetPass::Void => unreachable!("a parameter is never void"), - } -} - -/// Render the `public protocol` the consumer conforms to: one `throws` -/// requirement per method with idiomatic labels and types. Implementations -/// are classes (`AnyObject`) the library may call from any thread -/// (`Sendable`). -fn render_protocol(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, ctx: &SwiftCtx) { - let name = ctx.ty_name(local_type_name(&cb.name)); - if cb.doc.is_some() { - w.doc(&cb.doc, DocCommentStyle::TripleSlash); - w.line("///"); - } - w.line("/// Implement this protocol with a class and pass an instance where the API"); - w.line("/// expects it; the library keeps it alive for as long as it holds the"); - w.line("/// callback and may call any method from any thread. A thrown error aborts"); - w.line("/// the library's call and surfaces to the original caller as a foreign error"); - w.line("/// (code -4)."); - if let Some(msg) = &cb.deprecated { - w.line(deprecated_attr(msg)); - } - w.line(format!("public protocol {name}: AnyObject, Sendable {{")); - w.indent(); - for m in &cb.methods { - let m = camel_method(m); - { - let mut tmp = String::new(); - emit_fn_doc(&mut tmp, &m.doc, &m.params, &w.indent_str()); - w.raw(tmp); - } - if let Some(msg) = &m.deprecated { - w.line(deprecated_attr(msg)); - } - let params = m - .params - .iter() - .map(|p| format!("{}: {}", p.name, ctx.swift_type(&p.ty))) - .collect::>() - .join(", "); - let ret = m - .ret - .as_ref() - .map(|t| format!(" -> {}", ctx.swift_type(t))) - .unwrap_or_default(); - w.line(format!( - "func {}({params}) throws{ret}", - swift_ident(&m.name) - )); - } - w.dedent(); - w.line("}"); - w.blank(); -} - -/// Render the internal box that pins one implementation behind the `void* -/// ctx` the library holds: `Unmanaged.passRetained` when passing, one -/// `release` from the vtable's `free`. -fn render_box(w: &mut CodeWriter, cb: &CallbackInterfaceBinding, ctx: &SwiftCtx) { - let proto = ctx.ty_name(local_type_name(&cb.name)); - let box_name = callback_box_name(&cb.name); - w.line(format!( - "/// Retains one `{proto}` implementation across the C boundary; the library's" - )); - w.line("/// `free` entry releases it."); - w.line(format!("final class {box_name}: Sendable {{")); - w.scope(|w| { - w.line(format!("let impl: any {proto}")); - w.line(format!("init(_ impl: any {proto}) {{ self.impl = impl }}")); - }); - w.line("}"); - w.blank(); -} - -/// Render one trampoline closure literal (the vtable entry for `m`) as a -/// labeled argument of the vtable struct initializer. -fn render_trampoline( - w: &mut CodeWriter, - m: &CallbackMethodBinding, - args: &[RetPass], - box_name: &str, - ctx: &SwiftCtx, -) { - let camel = camel_method(m); - let formals = m - .abi_params - .iter() - .map(|s| slot_ident(&s.name)) - .collect::>() - .join(", "); - // The vtable field is spelled exactly as the C header declares it, so a - // method whose name is a C keyword carries the header's escape. - w.line(format!("{}: {{ {formals} in", c_param_name(&m.name))); - w.indent(); - w.line(format!( - "let wvBox = Unmanaged<{box_name}>.fromOpaque(ctx!).takeUnretainedValue()" - )); - w.line("do {"); - w.indent(); - let call_args = camel - .params - .iter() - .zip(args) - .map(|(p, rp)| format!("{}: {}", p.name, receive_arg(w, p, rp, ctx))) - .collect::>() - .join(", "); - let call = format!("try wvBox.impl.{}({call_args})", swift_ident(&m.name)); - let default = match &m.ret { - None => None, - Some(Ty::Enum(name)) => { - let c_type = ctx.c_enum_type(name); - w.line(format!("let value = {call}")); - w.line(format!( - "return {c_type}(rawValue: numericCast(value.rawValue))" - )); - Some(format!("{c_type}(rawValue: 0)")) - } - Some(t) => { - w.line(format!("return {call}")); - Some(if matches!(t, Ty::Bool) { "false" } else { "0" }.to_string()) - } - }; - if default.is_none() { - w.line(call); - } - w.dedent(); - w.line("} catch {"); - w.indent(); - w.line("wvForeignError(out_err, error)"); - if let Some(default) = default { - w.line(format!("return {default}")); - } - w.dedent(); - w.line("}"); - w.dedent(); - w.line("},"); -} - -/// Render the process-wide vtable namespace: `shared` holds the C struct -/// filled with capture-free trampolines (implicitly `@convention(c)`) at a -/// stable address the library can hold for the process lifetime. -fn render_vtable( - w: &mut CodeWriter, - cb: &CallbackInterfaceBinding, - protocol: &CallbackProtocol<'_>, - ctx: &SwiftCtx, -) { - let proto = ctx.ty_name(local_type_name(&cb.name)); - let box_name = callback_box_name(&cb.name); - let vtable_name = callback_vtable_name(&cb.name); - let vtable_tag = &cb.vtable_tag; - w.line(format!( - "/// The process-wide `{vtable_tag}` the library calls `{proto}`" - )); - w.line("/// implementations through. Every entry recovers the box from `ctx`, copies"); - w.line("/// or adopts its arguments, and reports a thrown error through `out_err`"); - w.line("/// instead of unwinding."); - w.line(format!("enum {vtable_name} {{")); - w.indent(); - w.line(format!( - "static let shared = WvVtable<{vtable_tag}>({vtable_tag}(" - )); - w.indent(); - for (m, args) in cb.methods.iter().zip(&protocol.method_args) { - render_trampoline(w, m, args, &box_name, ctx); - } - w.line("free: { ctx in"); - w.scope(|w| { - w.line(format!("Unmanaged<{box_name}>.fromOpaque(ctx!).release()")); - }); - w.line("}"); - w.dedent(); - w.line("))"); - w.dedent(); - w.line("}"); - w.blank(); -} - -/// Render everything one callback interface contributes at file scope: the -/// public protocol, the retaining box, and the vtable namespace whose -/// `shared.pointer` every call site passing this interface hands to the -/// library. -pub(crate) fn render_swift_callback_interface( - out: &mut String, - cb: &CallbackInterfaceBinding, - ctx: &SwiftCtx, -) { - let protocol = cb.protocol(ctx.c_prefix); - let mut w = CodeWriter::four_space(); - render_protocol(&mut w, cb, ctx); - render_box(&mut w, cb, ctx); - render_vtable(&mut w, cb, &protocol, ctx); - out.push_str(&w.finish()); -} diff --git a/crates/weaveffi-gen/src/targets/swift/codec.rs b/crates/weaveffi-gen/src/targets/swift/codec.rs deleted file mode 100644 index d7c18129..00000000 --- a/crates/weaveffi-gen/src/targets/swift/codec.rs +++ /dev/null @@ -1,190 +0,0 @@ -//! Value-buffer codec emitters: the Swift statements serializing and -//! decoding one value in the wire format. -//! -//! Both emitters dispatch on the shared [`Ty::wire`] classification, so the -//! non-obvious folds (interfaces as `u64` object tokens carrying one strong -//! reference, records and rich enums through one user codec) are decided -//! centrally rather than re-derived from `Ty` here. Primitives map onto the -//! runtime's `write{Prim}`/`read{Prim}` pairs by name. - -use crate::codegen::CodeWriter; -use crate::utils::local_type_name; -use weaveffi_model::model::{Ty, WireType}; - -use crate::targets::swift::types::SwiftCtx; - -/// A fresh generated-variable name (`v0`, `n1`, ...) unique within one -/// rendering scope. -pub(crate) fn fresh(counter: &mut usize, prefix: &str) -> String { - let id = *counter; - *counter += 1; - format!("{prefix}{id}") -} - -/// Emit statements serializing `expr` (of type `ty`) into the `WvWriter` -/// variable named `writer`, recursing through optionals, lists, and maps and -/// delegating records and rich enums to their generated `wvWrite*` codecs. -pub(crate) fn write_value_stmts( - w: &mut CodeWriter, - ty: &Ty, - expr: &str, - writer: &str, - counter: &mut usize, -) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("{writer}.write{}({expr})", p.pascal())); - } - // An object token carries one strong reference, so the wrapper - // writes a freshly cloned pointer and keeps its own. - WireType::Object(_) => { - w.line(format!("{writer}.writeObject({expr}.clonePtr())")); - } - // C-style enums cross as their `i32` discriminant. - WireType::Enum(_) => { - w.line(format!("{writer}.writeI32({expr}.rawValue)")); - } - WireType::User(name) => { - w.line(format!( - "wvWrite{}({expr}, into: &{writer})", - local_type_name(name) - )); - } - WireType::Optional(inner) => { - let v = fresh(counter, "v"); - w.line(format!("if let {v} = {expr} {{")); - w.indent(); - w.line(format!("{writer}.writeOptionFlag(true)")); - write_value_stmts(w, inner, &v, writer, counter); - w.dedent(); - w.line("} else {"); - w.indent(); - w.line(format!("{writer}.writeOptionFlag(false)")); - w.dedent(); - w.line("}"); - } - WireType::List(inner) => { - let v = fresh(counter, "v"); - w.line(format!("{writer}.writeLen({expr}.count)")); - w.line(format!("for {v} in {expr} {{")); - w.indent(); - write_value_stmts(w, inner, &v, writer, counter); - w.dedent(); - w.line("}"); - } - WireType::Map(k, val) => { - let kv = fresh(counter, "v"); - let vv = fresh(counter, "v"); - w.line(format!("{writer}.writeLen({expr}.count)")); - w.line(format!("for ({kv}, {vv}) in {expr} {{")); - w.indent(); - write_value_stmts(w, k, &kv, writer, counter); - write_value_stmts(w, val, &vv, writer, counter); - w.dedent(); - w.line("}"); - } - } -} - -/// Emit statements deserializing one value of type `ty` from the `WvReader` -/// variable named `reader`, binding the result to `out`. -pub(crate) fn read_value_stmts( - w: &mut CodeWriter, - ty: &Ty, - out: &str, - reader: &str, - ctx: &SwiftCtx, - counter: &mut usize, -) { - match ty.wire() { - WireType::Prim(p) => { - w.line(format!("let {out} = {reader}.read{}()", p.pascal())); - } - // The token's reference is adopted by a new wrapper, whose deinit - // owes the `_destroy`. - WireType::Object(name) => { - w.line(format!( - "let {out} = {}(ptr: {reader}.readObject())", - ctx.ty_name(local_type_name(name)) - )); - } - WireType::Enum(name) => { - w.line(format!( - "let {out} = wvEnumCase({}.self, {reader}.readI32())", - ctx.ty_name(local_type_name(name)) - )); - } - WireType::User(name) => { - w.line(format!( - "let {out} = wvRead{}(&{reader})", - local_type_name(name) - )); - } - WireType::Optional(inner) => { - let t = ctx.swift_type(inner); - w.line(format!("var {out}: {t}? = nil")); - w.line(format!("if {reader}.readOptionFlag() {{")); - w.indent(); - let v = fresh(counter, "v"); - read_value_stmts(w, inner, &v, reader, ctx, counter); - w.line(format!("{out} = {v}")); - w.dedent(); - w.line("}"); - } - WireType::List(inner) => { - let t = ctx.swift_type(inner); - let cnt = fresh(counter, "n"); - w.line(format!("let {cnt} = {reader}.readCount()")); - w.line(format!("var {out}: [{t}] = []")); - w.line(format!( - "{out}.reserveCapacity(min({cnt}, {reader}.remaining))" - )); - w.line(format!("for _ in 0..<{cnt} {{")); - w.indent(); - let v = fresh(counter, "v"); - read_value_stmts(w, inner, &v, reader, ctx, counter); - w.line(format!("{out}.append({v})")); - w.dedent(); - w.line("}"); - } - WireType::Map(k, val) => { - let kt = ctx.swift_type(k); - let vt = ctx.swift_type(val); - let cnt = fresh(counter, "n"); - w.line(format!("let {cnt} = {reader}.readCount()")); - w.line(format!("var {out}: [{kt}: {vt}] = [:]")); - w.line(format!( - "{out}.reserveCapacity(min({cnt}, {reader}.remaining))" - )); - w.line(format!("for _ in 0..<{cnt} {{")); - w.indent(); - let kv = fresh(counter, "v"); - let vv = fresh(counter, "v"); - read_value_stmts(w, k, &kv, reader, ctx, counter); - read_value_stmts(w, val, &vv, reader, ctx, counter); - w.line(format!("{out}[{kv}] = {vv}")); - w.dedent(); - w.line("}"); - } - } -} - -/// A closure literal decoding one value of type `ty` from a reader named `r`, -/// for the runtime's `wvTakeBuffer`/`wvBorrowBuffer`, rendered at `w`'s -/// depth. A record or rich enum decodes through its codec directly. -pub(crate) fn decode_closure(w: &CodeWriter, ty: &Ty, ctx: &SwiftCtx) -> String { - if let WireType::User(name) = ty.wire() { - return format!("wvRead{}", local_type_name(name)); - } - let mut inner = CodeWriter::four_space().with_depth(w.depth() + 1); - let mut counter = 0usize; - let v = fresh(&mut counter, "v"); - read_value_stmts(&mut inner, ty, &v, "r", ctx, &mut counter); - inner.line(format!("return {v}")); - format!( - "{{ (r: inout WvReader) -> {} in\n{}{}}}", - ctx.swift_type(ty), - inner.finish(), - w.indent_str() - ) -} diff --git a/crates/weaveffi-gen/src/targets/swift/docs.rs b/crates/weaveffi-gen/src/targets/swift/docs.rs deleted file mode 100644 index 7adc7ceb..00000000 --- a/crates/weaveffi-gen/src/targets/swift/docs.rs +++ /dev/null @@ -1,51 +0,0 @@ -//! Swift doc-comment emission: item docs and per-parameter `- Parameter` -//! lines. - -use crate::codegen::common::{emit_doc as common_emit_doc, DocCommentStyle}; -use weaveffi_model::model::ParamBinding; - -/// Emits a `///`-prefixed Swift doc comment at `indent`. Each line of the -/// (possibly multi-line) doc gets its own `///` prefix. -pub(crate) fn emit_doc(out: &mut String, doc: &Option, indent: &str) { - common_emit_doc(out, doc, indent, DocCommentStyle::TripleSlash); -} - -/// Emits Swift doc comments for a function: the function's own doc followed by -/// `/// - Parameter name: ...` lines for each documented parameter. Callers -/// pass params whose names are already camel-cased and keyword-escaped (see -/// `calls::camel_params`), so the doc labels match the emitted signature. -pub(crate) fn emit_fn_doc( - out: &mut String, - doc: &Option, - params: &[ParamBinding], - indent: &str, -) { - let has_param_docs = params.iter().any(|p| p.doc.is_some()); - if doc.is_none() && !has_param_docs { - return; - } - emit_doc(out, doc, indent); - for p in params { - if let Some(pdoc) = &p.doc { - let pdoc = pdoc.trim(); - if pdoc.is_empty() { - continue; - } - let mut lines = pdoc.lines(); - if let Some(first) = lines.next() { - out.push_str(indent); - out.push_str(&format!("/// - Parameter {}: {}\n", p.name, first)); - } - for line in lines { - out.push_str(indent); - if line.is_empty() { - out.push_str("///\n"); - } else { - out.push_str("/// "); - out.push_str(line); - out.push('\n'); - } - } - } - } -} diff --git a/crates/weaveffi-gen/src/targets/swift/entities.rs b/crates/weaveffi-gen/src/targets/swift/entities.rs deleted file mode 100644 index bbc62cf9..00000000 --- a/crates/weaveffi-gen/src/targets/swift/entities.rs +++ /dev/null @@ -1,551 +0,0 @@ -//! Entity rendering: enums, rich enums, records, interfaces, typed error -//! domains, and the recursive module walk emitting them (callback interfaces -//! render through [`crate::targets::swift::callbacks`]). - -use crate::codegen::common::DocCommentStyle; -use crate::codegen::CodeWriter; -use crate::utils::wrapper_name; -use heck::ToUpperCamelCase; -use weaveffi_model::model::{ - BindingModel, CallShape, EnumBinding, ErrorBinding, FieldBinding, InterfaceBinding, - ModuleBinding, StructBinding, -}; - -use crate::targets::swift::callbacks::render_swift_callback_interface; -use crate::targets::swift::calls::{ - camel_params, render_callable, render_swift_iterator_class, ErrCtx, Receiver, -}; -use crate::targets::swift::codec::{fresh, read_value_stmts, write_value_stmts}; -use crate::targets::swift::types::{deprecated_attr, swift_ident, swift_str, SwiftCtx}; - -/// The PascalCase helper stem of the domain in effect for `module`, naming the -/// per-domain `check{Stem}`/`map{Stem}` helpers (derived from the *declaring* -/// module's path, so inheriting submodules reference the ancestor's helper). -fn domain_stem(module: &ModuleBinding) -> Option { - module - .error - .as_ref() - .map(|e| e.owner_path.to_upper_camel_case()) -} - -/// The conformance list of a value type: `Hashable` (which implies -/// `Equatable`) when every field is, and always `Sendable`. -fn value_conformances<'t>( - ctx: &SwiftCtx, - fields: impl IntoIterator, -) -> &'static str { - if fields.into_iter().all(|f| ctx.is_hashable(&f.ty)) { - "Hashable, Sendable" - } else { - "Sendable" - } -} - -/// Render a C-style enum as an `Int32` raw-value Swift enum, one case per -/// variant. Wrappers convert to and from the imported C type at the call. -fn render_swift_enum(out: &mut String, e: &EnumBinding) { - let mut w = CodeWriter::four_space(); - w.doc(&e.doc, DocCommentStyle::TripleSlash); - if let Some(msg) = &e.deprecated { - w.line(deprecated_attr(msg)); - } - w.line(format!("public enum {}: Int32, Sendable {{", e.name)); - w.scope(|w| { - for v in &e.variants { - w.doc(&v.doc, DocCommentStyle::TripleSlash); - w.line(format!("case {} = {}", swift_ident(&v.name), v.value)); - } - }); - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render a rich (algebraic) enum as a native Swift enum with associated -/// values: one case per variant, with labeled associated values matching the -/// variant's field names, plus its `wvWrite`/`wvRead` codec pair. The writer -/// switches on the case, writes the `i32` tag, then the active variant's -/// fields in order; the reader inverts it and stops on an unknown tag. -fn render_swift_rich_enum(out: &mut String, e: &EnumBinding, ctx: &SwiftCtx) { - let name = &e.name; - let ty_name = ctx.ty_name(name); - let fields = e.variants.iter().flat_map(|v| v.fields.iter()); - let mut w = CodeWriter::four_space(); - w.doc(&e.doc, DocCommentStyle::TripleSlash); - if let Some(msg) = &e.deprecated { - w.line(deprecated_attr(msg)); - } - w.line(format!( - "public enum {name}: {} {{", - value_conformances(ctx, fields) - )); - w.scope(|w| { - for v in &e.variants { - w.doc(&v.doc, DocCommentStyle::TripleSlash); - let case_name = swift_ident(&v.name); - if v.fields.is_empty() { - w.line(format!("case {case_name}")); - } else { - let assoc = v - .fields - .iter() - .map(|f| format!("{}: {}", swift_ident(&f.name), ctx.swift_type(&f.ty))) - .collect::>() - .join(", "); - w.line(format!("case {case_name}({assoc})")); - } - } - }); - w.line("}"); - w.blank(); - - let mut counter = 0usize; - // The codecs of a deprecated type are deprecated too, so they can name it - // without a warning. - let codec_attr = e.deprecated.as_deref().map(deprecated_attr); - if let Some(attr) = &codec_attr { - w.line(attr); - } - w.line(format!( - "func wvWrite{name}(_ value: {ty_name}, into w: inout WvWriter) {{" - )); - w.indent(); - w.line("switch value {"); - for v in &e.variants { - let case_name = swift_ident(&v.name); - if v.fields.is_empty() { - w.line(format!("case .{case_name}:")); - w.scope(|w| { - w.line(format!("w.writeI32({})", v.value)); - }); - } else { - let binds: Vec = v.fields.iter().map(|_| fresh(&mut counter, "v")).collect(); - w.line(format!("case let .{case_name}({}):", binds.join(", "))); - w.indent(); - w.line(format!("w.writeI32({})", v.value)); - for (f, bind) in v.fields.iter().zip(&binds) { - write_value_stmts(&mut w, &f.ty, bind, "w", &mut counter); - } - w.dedent(); - } - } - w.line("}"); - w.dedent(); - w.line("}"); - w.blank(); - - if let Some(attr) = &codec_attr { - w.line(attr); - } - w.line(format!( - "func wvRead{name}(_ r: inout WvReader) -> {ty_name} {{" - )); - w.indent(); - w.line("let tag = r.readI32()"); - w.line("switch tag {"); - for v in &e.variants { - let case_name = swift_ident(&v.name); - w.line(format!("case {}:", v.value)); - w.indent(); - if v.fields.is_empty() { - w.line(format!("return .{case_name}")); - } else { - let mut labeled = Vec::new(); - for f in &v.fields { - let var = fresh(&mut counter, "v"); - read_value_stmts(&mut w, &f.ty, &var, "r", ctx, &mut counter); - labeled.push(format!("{}: {var}", swift_ident(&f.name))); - } - w.line(format!("return .{case_name}({})", labeled.join(", "))); - } - w.dedent(); - } - w.line("default:"); - w.scope(|w| { - w.line(format!("wvDecodeFailure(\"unknown {name} tag \\(tag)\")")); - }); - w.line("}"); - w.dedent(); - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render a record as a plain Swift struct with one `public var` per field -/// and an explicit public memberwise initializer (the synthesized one is -/// internal), plus its `wvWrite`/`wvRead` codec pair (fields in declaration -/// order). -fn render_swift_struct(out: &mut String, s: &StructBinding, ctx: &SwiftCtx) { - let name = &s.name; - let ty_name = ctx.ty_name(name); - let mut w = CodeWriter::four_space(); - w.doc(&s.doc, DocCommentStyle::TripleSlash); - if let Some(msg) = &s.deprecated { - w.line(deprecated_attr(msg)); - } - w.line(format!( - "public struct {name}: {} {{", - value_conformances(ctx, &s.fields) - )); - w.indent(); - for f in &s.fields { - w.doc(&f.doc, DocCommentStyle::TripleSlash); - w.line(format!( - "public var {}: {}", - swift_ident(&f.name), - ctx.swift_type(&f.ty) - )); - } - if !s.fields.is_empty() { - w.blank(); - } - let params = s - .fields - .iter() - .map(|f| format!("{}: {}", swift_ident(&f.name), ctx.swift_type(&f.ty))) - .collect::>() - .join(", "); - w.line("/// Creates a value from its fields."); - w.line(format!("public init({params}) {{")); - w.scope(|w| { - for f in &s.fields { - let prop = swift_ident(&f.name); - w.line(format!("self.{prop} = {prop}")); - } - }); - w.line("}"); - w.dedent(); - w.line("}"); - w.blank(); - - let mut counter = 0usize; - let value = if s.fields.is_empty() { "_" } else { "value" }; - let writer = if s.fields.is_empty() { "_" } else { "w" }; - let codec_attr = s.deprecated.as_deref().map(deprecated_attr); - if let Some(attr) = &codec_attr { - w.line(attr); - } - w.line(format!( - "func wvWrite{name}(_ {value}: {ty_name}, into {writer}: inout WvWriter) {{" - )); - w.indent(); - for f in &s.fields { - let expr = format!("value.{}", swift_ident(&f.name)); - write_value_stmts(&mut w, &f.ty, &expr, "w", &mut counter); - } - w.dedent(); - w.line("}"); - w.blank(); - - let reader = if s.fields.is_empty() { "_" } else { "r" }; - if let Some(attr) = &codec_attr { - w.line(attr); - } - w.line(format!( - "func wvRead{name}(_ {reader}: inout WvReader) -> {ty_name} {{" - )); - w.indent(); - let mut labeled = Vec::new(); - for f in &s.fields { - let var = fresh(&mut counter, "v"); - read_value_stmts(&mut w, &f.ty, &var, "r", ctx, &mut counter); - labeled.push(format!("{}: {var}", swift_ident(&f.name))); - } - w.line(format!("return {ty_name}({})", labeled.join(", "))); - w.dedent(); - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render one declaring module's typed error surface: a `public enum -/// {TypeName}: Error` whose lowerCamel cases carry the runtime message plus, -/// for codes that declare payload fields, one labeled associated value per -/// field. Also emits the file-scope `map{Stem}` and `check{Stem}` helpers -/// that convert a filled error slot into it, decoding the payload buffer for -/// codes with fields. -/// -/// Only declared codes get typed cases. Domain codes are validated positive, -/// so the mapper's `default` arm is what every reserved negative runtime -/// code falls through to: the runtime error, or `CancellationError` for `-5`. -fn render_swift_error(out: &mut String, module: &ModuleBinding, eb: &ErrorBinding, ctx: &SwiftCtx) { - let stem = eb.owner_path.to_upper_camel_case(); - let ty = &eb.type_name; - - let case_decl = |fields: &[FieldBinding]| -> String { - let mut parts = vec!["message: String".to_string()]; - for f in fields { - parts.push(format!( - "{}: {}", - swift_ident(&f.name), - ctx.swift_type(&f.ty) - )); - } - parts.join(", ") - }; - - let mut w = CodeWriter::four_space(); - w.doc( - &Some(format!( - "Typed errors reported by the `{}` module.", - module.segments.join(".") - )), - DocCommentStyle::TripleSlash, - ); - w.line(format!( - "public enum {ty}: Error, LocalizedError, Sendable {{" - )); - w.indent(); - for c in &eb.codes { - w.doc(&c.doc, DocCommentStyle::TripleSlash); - w.line(format!( - "case {}({})", - swift_ident(&c.name), - case_decl(&c.fields) - )); - } - w.blank(); - w.line("/// The message the library reported, or the code's documented default."); - w.line("public var errorDescription: String? {"); - w.scope(|w| { - w.line("switch self {"); - for c in &eb.codes { - // Bind only the message; wildcard the payload fields. - let mut binds = vec!["message".to_string()]; - binds.extend(c.fields.iter().map(|_| "_".to_string())); - w.line(format!( - "case let .{}({}): return message", - swift_ident(&c.name), - binds.join(", ") - )); - } - w.line("}"); - }); - w.line("}"); - w.blank(); - w.line("/// The numeric ABI code carried by this error."); - w.line("public var errorCode: Int32 {"); - w.scope(|w| { - w.line("switch self {"); - for c in &eb.codes { - w.line(format!( - "case .{}: return {}", - swift_ident(&c.name), - c.value - )); - } - w.line("}"); - }); - w.line("}"); - w.dedent(); - w.line("}"); - w.blank(); - - // `map{Stem}`: code -> typed case (default message when the slot carried - // none), decoding the payload buffer for codes that declare fields. - w.line(format!("func map{stem}(_ err: WvError) -> Error {{")); - w.indent(); - w.line("let message = wvErrorMessage(err)"); - w.line("switch err.code {"); - for c in &eb.codes { - let case_name = swift_ident(&c.name); - let message_arg = format!( - "message: message.isEmpty ? \"{}\" : message", - swift_str(&c.message) - ); - if c.fields.is_empty() { - w.line(format!( - "case {}: return {ty}.{case_name}({message_arg})", - c.value - )); - } else { - w.line(format!("case {}:", c.value)); - w.indent(); - w.line("var r = WvReader(err.payload_ptr, err.payload_len)"); - let mut counter = 0usize; - let mut args = vec![message_arg]; - for f in &c.fields { - let var = fresh(&mut counter, "v"); - read_value_stmts(&mut w, &f.ty, &var, "r", ctx, &mut counter); - args.push(format!("{}: {var}", swift_ident(&f.name))); - } - w.line("r.finish()"); - w.line(format!("return {ty}.{case_name}({})", args.join(", "))); - w.dedent(); - } - } - w.line("default: return wvRuntimeError(err)"); - w.line("}"); - w.dedent(); - w.line("}"); - w.blank(); - - w.line(format!("func check{stem}(_ err: inout WvError) throws {{")); - w.indent(); - w.line("guard err.code != 0 else { return }"); - w.line(format!("let error = map{stem}(err)")); - w.line(format!("{}_error_clear(&err)", ctx.c_prefix)); - w.line("throw error"); - w.dedent(); - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Render one interface as a `public final class` owning one strong -/// reference to its object: a stored `ptr`, an internal reference-adopting -/// `init(ptr:)`, a `deinit` that releases the reference exactly once through -/// the destroy symbol, and an internal `clonePtr()` that mints a second -/// reference for positions that take ownership (an object token inside a -/// value buffer). The native object is thread-safe, so the class is -/// `@unchecked Sendable`. -/// -/// The constructor named `new` surfaces as `public init`; every other -/// constructor becomes a `public static func` factory. Methods are instance -/// funcs passing `ptr` as the leading C argument; statics are plain `public -/// static func`s. -fn render_swift_interface( - out: &mut String, - module: &ModuleBinding, - iface: &InterfaceBinding, - ctx: &SwiftCtx, -) { - let stem = domain_stem(module); - let class_name = &iface.name; - - let mut w = CodeWriter::four_space(); - w.doc(&iface.doc, DocCommentStyle::TripleSlash); - if let Some(msg) = &iface.deprecated { - w.line(deprecated_attr(msg)); - } - w.line(format!( - "public final class {class_name}: @unchecked Sendable {{" - )); - w.indent(); - w.line("let ptr: OpaquePointer"); - w.blank(); - w.line("init(ptr: OpaquePointer) {"); - w.scope(|w| { - w.line("self.ptr = ptr"); - }); - w.line("}"); - w.blank(); - w.line("deinit {"); - w.scope(|w| { - w.line(format!("{}(ptr)", iface.destroy_symbol)); - }); - w.line("}"); - w.blank(); - w.line("/// Returns a new strong reference to the same object, for a position that"); - w.line("/// takes ownership of it (an object token inside a value buffer)."); - w.line("func clonePtr() -> OpaquePointer {"); - w.scope(|w| { - w.line(format!("wvNonNull({}(ptr))", iface.clone_symbol)); - }); - w.line("}"); - w.dedent(); - - let mut members = String::new(); - let mut render = |f: &weaveffi_model::model::FnBinding, receiver: Receiver| { - let f = camel_params(f); - let err = ErrCtx::for_fn(&f, stem.as_deref()); - members.push('\n'); - let receiver = if receiver == Receiver::Static && f.name == "new" { - Receiver::Init - } else { - receiver - }; - render_callable( - &mut members, - &f, - &swift_ident(&f.name), - receiver, - err, - ctx, - 1, - ); - }; - for c in &iface.constructors { - render(c, Receiver::Static); - } - for m in &iface.methods { - render(m, Receiver::Instance); - } - for s in &iface.statics { - render(s, Receiver::Static); - } - w.raw(members); - - w.line("}"); - w.blank(); - out.push_str(&w.finish()); -} - -/// Emit every file-scope type one module contributes: the typed error -/// surface, enums and records with their codecs, callback-interface -/// protocols with their vtables, interface classes, and the sequence classes -/// backing `iter` callables. -pub(crate) fn render_swift_module_types(out: &mut String, mb: &ModuleBinding, ctx: &SwiftCtx) { - if let Some(eb) = mb.error.as_ref().filter(|e| e.declared_here) { - render_swift_error(out, mb, eb, ctx); - } - for e in &mb.enums { - if e.is_rich() { - render_swift_rich_enum(out, e, ctx); - } else { - render_swift_enum(out, e); - } - } - for s in &mb.structs { - render_swift_struct(out, s, ctx); - } - for cb in &mb.callback_interfaces { - render_swift_callback_interface(out, cb, ctx); - } - for i in &mb.interfaces { - render_swift_interface(out, mb, i, ctx); - } - let stem = domain_stem(mb); - for f in mb.callables() { - if let CallShape::Iterator(it) = &f.shape { - render_swift_iterator_class(out, f, it, ErrCtx::for_fn(f, stem.as_deref()), ctx); - } - } -} - -/// Emit one module's namespace `enum` at `depth`: its function wrappers, -/// then one nested namespace per submodule. -pub(crate) fn render_swift_namespace( - out: &mut String, - model: &BindingModel, - mb: &ModuleBinding, - depth: usize, - strip_module_prefix: bool, - ctx: &SwiftCtx, -) { - let indent = " ".repeat(depth); - let stem = domain_stem(mb); - out.push_str(&format!( - "{indent}public enum {} {{\n", - mb.name.to_upper_camel_case() - )); - let mut first = true; - for f in &mb.functions { - if !first { - out.push('\n'); - } - first = false; - let f = camel_params(f); - let swift_name = swift_ident(&wrapper_name(&mb.path, &f.name, strip_module_prefix)); - let err = ErrCtx::for_fn(&f, stem.as_deref()); - render_callable(out, &f, &swift_name, Receiver::Static, err, ctx, depth + 1); - } - for sub in model.children(mb) { - if !first { - out.push('\n'); - } - first = false; - render_swift_namespace(out, model, sub, depth + 1, strip_module_prefix, ctx); - } - out.push_str(&format!("{indent}}}\n")); -} diff --git a/crates/weaveffi-gen/src/targets/swift/mod.rs b/crates/weaveffi-gen/src/targets/swift/mod.rs deleted file mode 100644 index 77244c9f..00000000 --- a/crates/weaveffi-gen/src/targets/swift/mod.rs +++ /dev/null @@ -1,564 +0,0 @@ -//! Swift binding generator. -//! -//! Emits a standalone SwiftPM package: the Swift wrapper module over the C -//! ABI, plus a `C{Module}` system-library target holding a copy of the C -//! header and a module map that links the native library. Implements -//! [`LanguageBackend`]; the shared driver bridges it into the generator -//! pipeline. -//! -//! Records, rich enums, optionals, lists, and maps cross the C ABI as value -//! buffers. The wrapper ships a small private writer/reader pair -//! (`WvWriter`/`WvReader`, decoding in place from the library's memory) plus -//! one pack and one unpack routine per record and rich enum, so records -//! surface as plain Swift structs and rich enums as native Swift enums with -//! associated values. Objects surface as `final class` wrappers owning one -//! strong reference each, callback interfaces as class-bound `Sendable` -//! protocols whose implementations cross the boundary through a process-wide -//! vtable of `@convention(c)` trampolines, and async functions as `async` -//! functions over checked continuations, with task cancellation wired to the -//! native cancel token. -//! -//! The fixed Swift sources (runtime, manifest, module map, README) live -//! under `runtime/` and are spliced with the package's names. - -mod callbacks; -mod calls; -mod codec; -mod docs; -mod entities; -mod package; -mod runtime; -mod types; - -use crate::backend::{LanguageBackend, OutputFile}; -use crate::capabilities::TargetCapabilities; -use crate::package::{PackageContext, PackagedFile}; -use crate::targets::c::{header_name, render_c_header_from_model}; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; -use camino::{Utf8Path, Utf8PathBuf}; -use serde::{Deserialize, Serialize}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::resolved::ResolvedApi; - -use crate::targets::swift::entities::{render_swift_module_types, render_swift_namespace}; -use crate::targets::swift::package::{ - render_modulemap, render_package_swift, render_packaged_readme, -}; -use crate::targets::swift::runtime::{render_runtime, Names}; -use crate::targets::swift::types::SwiftCtx; - -/// Per-target configuration for [`SwiftGenerator`]. -#[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(default, deny_unknown_fields)] -pub struct SwiftConfig { - /// SwiftPM package, product, and module name. Defaults to the package - /// name in PascalCase (`kvstore` becomes `Kvstore`); the C module is - /// always this name with a `C` prefix. - pub module_name: Option, - /// When `true` (the default), strip the IR module path from emitted - /// function names, so `enum Kv` exposes `openStore` rather than - /// `kvOpenStore`. Set to `false` to restore the module-prefixed spelling. - pub strip_module_prefix: bool, - /// Basename of the IDL the CLI was invoked with. - /// Populated by the CLI; not user-configurable via `[swift]`. - #[serde(skip)] - pub input_basename: Option, -} - -impl Default for SwiftConfig { - fn default() -> Self { - Self { - module_name: None, - strip_module_prefix: true, - input_basename: None, - } - } -} - -impl SwiftConfig { - /// The input IDL basename embedded in generated file headers, falling - /// back to `"api.yml"`. - pub fn input_basename(&self) -> &str { - self.input_basename.as_deref().unwrap_or("api.yml") - } -} - -/// The resolved names of one generated package. -struct Layout { - /// The Swift module (and package and product) name. - module: String, - /// The C module name, `C{module}`. - c_module: String, - /// The native library base name. - library: String, - /// The C header file name. - header: String, -} - -impl Layout { - fn new(api: &ResolvedApi, config: &SwiftConfig) -> Self { - let identity = api.identity(); - let module = config - .module_name - .clone() - .unwrap_or_else(|| identity.pascal_name()); - Self { - c_module: format!("C{module}"), - module, - library: identity.library.clone(), - header: header_name(api), - } - } - - fn names(&self) -> Names<'_> { - Names { - module: &self.module, - c_module: &self.c_module, - library: &self.library, - header: &self.header, - } - } - - /// The error type for codes outside any declared domain. - fn runtime_error(&self) -> String { - format!("{}RuntimeError", self.module) - } - - /// The `(path, contents)` of every source file of the package rooted at - /// `dir`. - fn sources( - &self, - model: &BindingModel, - dir: &Utf8Path, - config: &SwiftConfig, - ) -> Vec<(Utf8PathBuf, String)> { - let input_basename = config.input_basename(); - let c_dir = dir.join("Sources").join(&self.c_module); - let swift_file = format!("{}.swift", self.module); - vec![ - ( - dir.join("Package.swift"), - render_package_swift(&self.names(), input_basename), - ), - ( - c_dir.join("module.modulemap"), - render_modulemap(&self.names(), input_basename), - ), - ( - c_dir.join(&self.header), - render_c_header_from_model(model, input_basename, &self.header), - ), - ( - dir.join("Sources").join(&self.module).join(&swift_file), - render_swift_wrapper(self, model, config, &swift_file), - ), - ] - } -} - -/// Swift backend: emits a standalone SwiftPM package wrapping the C ABI. -pub struct SwiftGenerator; - -impl LanguageBackend for SwiftGenerator { - type Config = SwiftConfig; - - fn name(&self) -> &'static str { - "swift" - } - - fn capabilities(&self, _config: &Self::Config) -> TargetCapabilities { - TargetCapabilities::full() - } - - fn files( - &self, - api: &ResolvedApi, - model: &BindingModel, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Vec { - Layout::new(api, config) - .sources(model, &out_dir.join("swift"), config) - .into_iter() - .map(|(path, contents)| OutputFile::new(path, contents)) - .collect() - } - - fn package( - &self, - api: &ResolvedApi, - model: &BindingModel, - ctx: &PackageContext, - out_dir: &Utf8Path, - config: &Self::Config, - ) -> Option> { - let layout = Layout::new(api, config); - let dir = out_dir.join("swift"); - let mut files: Vec = layout - .sources(model, &dir, config) - .into_iter() - .map(|(path, contents)| PackagedFile::text(path, contents)) - .collect(); - files.push(PackagedFile::text( - dir.join("README.md"), - render_packaged_readme(&layout.names(), ctx, config.input_basename()), - )); - // Desktop slices only: they link through the system-library target - // or fuse into the XCFramework the README describes. - for nb in ctx - .binaries - .binaries - .iter() - .filter(|nb| nb.platform.is_desktop()) - { - let dest = dir - .join("lib") - .join(nb.platform.id()) - .join(ctx.binaries.bundled_filename(nb.platform)); - files.push(PackagedFile::copy(dest, nb.source.clone())); - } - Some(files) - } -} - -/// Render the complete Swift wrapper file: prelude and imports, the private -/// runtime, every module's file-scope types, and one namespace `enum` per -/// top-level module. -fn render_swift_wrapper( - layout: &Layout, - model: &BindingModel, - config: &SwiftConfig, - filename: &str, -) -> String { - let runtime_error = layout.runtime_error(); - let ctx = SwiftCtx::new(model, &layout.module); - let mut out = String::with_capacity(16 * 1024); - out.push_str(&render_prelude( - CommentStyle::DoubleSlash, - config.input_basename(), - )); - out.push_str(&format!( - "import {}\nimport Foundation\n\n", - layout.c_module - )); - out.push_str(&render_runtime(model, &layout.names(), &runtime_error)); - out.push_str("\n// MARK: - API\n\n"); - for mb in &model.modules { - render_swift_module_types(&mut out, mb, &ctx); - } - for root in model.roots() { - render_swift_namespace(&mut out, model, root, 0, config.strip_module_prefix, &ctx); - out.push('\n'); - } - out.push_str(&render_trailer(CommentStyle::DoubleSlash, filename)); - out -} - -#[cfg(test)] -mod tests { - use super::*; - use weaveffi_model::ir::{ - Api, CallbackInterfaceDef, Function, InterfaceDef, Module, Param, StructDef, StructField, - TypeRef, - }; - use weaveffi_model::pkg::Identity; - - fn param(name: &str, ty: TypeRef) -> Param { - Param { - name: name.into(), - ty, - doc: None, - } - } - - fn func(name: &str, params: Vec, returns: Option) -> Function { - Function { - name: name.into(), - params, - returns, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - } - } - - fn named(name: &str) -> TypeRef { - TypeRef::Named(name.into()) - } - - fn opt(ty: TypeRef) -> TypeRef { - TypeRef::Optional(Box::new(ty)) - } - - /// One module exercising every object and callback shape the ABI - /// contract distinguishes, plus a cancellable async function. - fn fixture() -> ResolvedApi { - let mut wait = func("wait", vec![param("ms", TypeRef::I64)], Some(TypeRef::I64)); - wait.r#async = true; - wait.cancellable = true; - ResolvedApi::assume_valid(Api { - version: weaveffi_model::ir::CURRENT_SCHEMA_VERSION.into(), - modules: vec![Module { - name: "shop".into(), - doc: None, - functions: vec![ - func( - "find_cart", - vec![param("current", opt(named("shop.Cart")))], - Some(opt(named("shop.Cart"))), - ), - func( - "all_carts", - vec![], - Some(TypeRef::Iterator(Box::new(named("shop.Cart")))), - ), - func("watch", vec![param("watcher", named("shop.Watcher"))], None), - func( - "echo", - vec![param("text", TypeRef::StringUtf8)], - Some(TypeRef::StringUtf8), - ), - wait, - ], - interfaces: vec![InterfaceDef { - name: "Cart".into(), - doc: None, - deprecated: None, - constructors: vec![func( - "new", - vec![param("owner", TypeRef::StringUtf8)], - None, - )], - methods: vec![func( - "add", - vec![param("sku", TypeRef::StringUtf8)], - Some(TypeRef::Bool), - )], - statics: vec![], - }], - callback_interfaces: vec![CallbackInterfaceDef { - name: "Watcher".into(), - doc: None, - deprecated: None, - methods: vec![ - func( - "on_event", - vec![ - param("name", TypeRef::StringUtf8), - param("count", TypeRef::I32), - param("order", named("shop.Order")), - param("cart", named("shop.Cart")), - ], - Some(TypeRef::Bool), - ), - func("on_done", vec![], None), - ], - }], - structs: vec![StructDef { - name: "Order".into(), - doc: None, - deprecated: None, - fields: vec![ - StructField { - name: "cart".into(), - ty: named("shop.Cart"), - doc: None, - }, - StructField { - name: "history".into(), - ty: TypeRef::List(Box::new(named("shop.Cart"))), - doc: None, - }, - ], - }], - enums: vec![], - errors: None, - modules: vec![], - }], - }) - .with_identity(Identity::named("shop_kit")) - } - - fn render() -> String { - let api = fixture(); - let model = BindingModel::build(&api); - let config = SwiftConfig::default(); - let layout = Layout::new(&api, &config); - render_swift_wrapper(&layout, &model, &config, "ShopKit.swift") - } - - #[track_caller] - fn assert_has(out: &str, needle: &str) { - assert!(out.contains(needle), "missing {needle:?} in:\n{out}"); - } - - #[test] - fn names_come_from_the_identity() { - let api = fixture(); - let model = BindingModel::build(&api); - let config = SwiftConfig::default(); - let files = Layout::new(&api, &config).sources(&model, Utf8Path::new("out"), &config); - let paths: Vec = files - .iter() - .map(|(p, _)| p.as_str().replace('\\', "/")) - .collect(); - assert_eq!( - paths, - [ - "out/Package.swift", - "out/Sources/CShopKit/module.modulemap", - "out/Sources/CShopKit/shop_kit.h", - "out/Sources/ShopKit/ShopKit.swift", - ] - ); - assert_has(&files[0].1, "name: \"ShopKit\""); - assert_has(&files[0].1, ".systemLibrary(name: \"CShopKit\")"); - assert_has( - &files[0].1, - ".binaryTarget(name: \"CShopKit\", path: xcframework)", - ); - assert_has(&files[1].1, "header \"shop_kit.h\""); - assert_has(&files[1].1, "link \"shop_kit\""); - let all = files.iter().map(|(_, c)| c.as_str()).collect::(); - assert!(!all.contains("weaveffi_"), "{all}"); - assert!(!all.contains("WeaveFFIError"), "{all}"); - } - - #[test] - fn module_name_override_wins() { - let api = fixture(); - let config = SwiftConfig { - module_name: Some("Shop".into()), - ..SwiftConfig::default() - }; - let layout = Layout::new(&api, &config); - assert_eq!(layout.module, "Shop"); - assert_eq!(layout.c_module, "CShop"); - assert_eq!(layout.library, "shop_kit"); - } - - #[test] - fn contract_is_checked_before_the_first_call() { - let out = render(); - assert_has(&out, "let abi = shop_kit_abi_version()"); - assert_has(&out, "wvCheckModule(\"shop\", shop_kit_shop_checksum(), 0x"); - assert_has(&out, "public struct ShopKitRuntimeError: Error"); - assert_has( - &out, - "public static func echo(text: String) -> String {\n wvLoad()", - ); - } - - #[test] - fn strings_cross_as_pointer_and_length() { - let out = render(); - assert_has(&out, "wvWithUTF8(text) { text_ptr, text_len in"); - assert_has( - &out, - "shop_kit_shop_echo(text_ptr, text_len, &outLen, &err)", - ); - assert_has(&out, "return wvTakeString(rv, outLen)"); - } - - #[test] - fn interface_class_owns_one_reference() { - let out = render(); - assert_has(&out, "public final class Cart: @unchecked Sendable {"); - assert_has(&out, "let ptr: OpaquePointer"); - assert_has(&out, "shop_kit_shop_Cart_destroy(ptr)"); - assert_has(&out, "wvNonNull(shop_kit_shop_Cart_clone(ptr))"); - assert_eq!(out.matches("shop_kit_shop_Cart_destroy(").count(), 1); - assert_has(&out, "public init(owner: String) {"); - assert_has(&out, "self.ptr = wvNonNull(rv)"); - assert_has(&out, "public func add(sku: String) -> Bool {"); - assert_has(&out, "shop_kit_shop_Cart_add(ptr, sku_ptr, sku_len, &err)"); - } - - #[test] - fn nullable_objects_map_to_optional_wrappers() { - let out = render(); - assert_has( - &out, - "public static func findCart(current: Cart?) -> Cart? {", - ); - assert_has(&out, "shop_kit_shop_find_cart(current?.ptr, &err)"); - assert_has(&out, "return rv.map { Cart(ptr: $0) }"); - } - - #[test] - fn records_carry_object_tokens() { - let out = render(); - assert_has(&out, "public struct Order: Sendable {"); - assert_has(&out, "w.writeObject(value.cart.clonePtr())"); - assert_has(&out, "Cart(ptr: r.readObject())"); - } - - #[test] - fn object_iterator_adopts_each_element() { - let out = render(); - assert_has( - &out, - "public final class ShopAllCartsIterator: Sequence, IteratorProtocol {", - ); - assert_has(&out, "var item: OpaquePointer? = nil"); - assert_has( - &out, - "shop_kit_shop_AllCartsIterator_next(handle, &item, &err)", - ); - assert_has(&out, "shop_kit_shop_AllCartsIterator_destroy(handle)"); - assert_has(&out, "return Cart(ptr: wvNonNull(item))"); - } - - #[test] - fn callback_interface_renders_protocol_box_and_vtable() { - let out = render(); - assert_has(&out, "public protocol Watcher: AnyObject, Sendable {"); - assert_has( - &out, - "func onEvent(name: String, count: Int32, order: Order, cart: Cart) throws -> Bool", - ); - assert_has(&out, "final class WvWatcherBox: Sendable {"); - assert_has( - &out, - "static let shared = WvVtable(shop_kit_shop_Watcher_vtable(", - ); - assert_has( - &out, - "on_event: { ctx, name_ptr, name_len, count, order_ptr, order_len, cart, out_err in", - ); - assert_has(&out, "name: wvBorrowString(name_ptr, name_len)"); - assert_has( - &out, - "order: wvBorrowBuffer(order_ptr, order_len, wvReadOrder)", - ); - assert_has(&out, "cart: Cart(ptr: wvNonNull(cart))"); - assert_has(&out, "wvForeignError(out_err, error)"); - assert_has(&out, "Unmanaged.fromOpaque(ctx!).release()"); - assert_has( - &out, - "shop_kit_shop_watch(watcher_ctx, WvWatcherVtable.shared.pointer, &err)", - ); - } - - #[test] - fn cancellable_async_cancels_the_native_token() { - let out = render(); - assert_has( - &out, - "public static func wait(ms: Int64) async throws -> Int64 {", - ); - assert_has(&out, "let token = WvCancelToken()"); - assert_has(&out, "return try await withTaskCancellationHandler {"); - assert_has( - &out, - "shop_kit_shop_wait(ms, token.raw, { context, err, result in", - ); - assert_has( - &out, - "cont.resume(throwing: wvTakeError(err, wvCancelledOrTrap))", - ); - assert_has(&out, "} onCancel: {\n token.cancel()"); - } -} diff --git a/crates/weaveffi-gen/src/targets/swift/package.rs b/crates/weaveffi-gen/src/targets/swift/package.rs deleted file mode 100644 index 802c5d3e..00000000 --- a/crates/weaveffi-gen/src/targets/swift/package.rs +++ /dev/null @@ -1,59 +0,0 @@ -//! SwiftPM packaging surfaces: `Package.swift`, the system-library module -//! map, and the packaged-artifact README, framed with the generated-file -//! prelude and trailer. - -use crate::package::PackageContext; -use crate::utils::{render_prelude, render_trailer, CommentStyle}; - -use crate::targets::swift::runtime::{render_module_map, render_package, render_readme, Names}; - -/// The SwiftPM tools version the manifest declares (Swift 5 language mode). -const TOOLS_VERSION: &str = "5.9"; - -/// Render `Package.swift`. The manifest picks the C module's source at -/// resolution time: a prebuilt `C{Module}.xcframework` next to it when one -/// exists, otherwise the bundled system-library target. -pub(crate) fn render_package_swift(names: &Names, input_basename: &str) -> String { - // `swift-tools-version` must be the very first line of the manifest, so - // the generated-file prelude follows it. - format!( - "// swift-tools-version:{TOOLS_VERSION}\n{}{}\n{}", - render_prelude(CommentStyle::DoubleSlash, input_basename), - render_package(names), - render_trailer(CommentStyle::DoubleSlash, "Package.swift"), - ) -} - -/// Render the module map of the `C{Module}` system-library target, which -/// sits next to its copy of the C header. -pub(crate) fn render_modulemap(names: &Names, input_basename: &str) -> String { - format!( - "{}{}\n{}", - render_prelude(CommentStyle::DoubleSlash, input_basename), - render_module_map(names), - render_trailer(CommentStyle::DoubleSlash, "module.modulemap"), - ) -} - -/// README for a packaged Swift artifact: linking the bundled libraries and -/// assembling the optional `XCFramework`, the one step that needs Apple -/// tooling. -pub(crate) fn render_packaged_readme( - names: &Names, - ctx: &PackageContext, - input_basename: &str, -) -> String { - let platforms = ctx - .binaries - .platforms() - .filter(|p| p.is_desktop()) - .map(|p| format!("- `lib/{}/`", p.id())) - .collect::>() - .join("\n"); - format!( - "{}{}\n{}", - render_prelude(CommentStyle::Xml, input_basename), - render_readme(names, &platforms), - render_trailer(CommentStyle::Xml, "README.md"), - ) -} diff --git a/crates/weaveffi-gen/src/targets/swift/runtime.rs b/crates/weaveffi-gen/src/targets/swift/runtime.rs deleted file mode 100644 index 785be7dc..00000000 --- a/crates/weaveffi-gen/src/targets/swift/runtime.rs +++ /dev/null @@ -1,108 +0,0 @@ -//! The fixed Swift sources the generator splices names into: the private -//! runtime (codec, error helpers, contract check, async and callback -//! support), the `Package.swift` manifest, the C module map, and the -//! packaged README. Each lives as a real file under `runtime/` with -//! `{{PLACEHOLDER}}` markers. - -use std::fmt::Write; - -use weaveffi_model::model::{checksum_symbol, BindingModel, ABI_VERSION}; - -const RUNTIME: &str = include_str!("runtime/Runtime.swift"); -const PACKAGE: &str = include_str!("runtime/Package.swift"); -const MODULE_MAP: &str = include_str!("runtime/module.modulemap"); -const README: &str = include_str!("runtime/README.md"); - -/// Replace every `{{KEY}}` in `template` with its value. -/// -/// # Panics -/// -/// Panics (in debug builds) when a placeholder is left unfilled, which is a -/// bug in the caller's substitution list. -fn fill(template: &str, vars: &[(&str, &str)]) -> String { - let mut out = template.to_string(); - for (key, value) in vars { - out = out.replace(&format!("{{{{{key}}}}}"), value); - } - debug_assert!(!out.contains("{{"), "unfilled placeholder in:\n{out}"); - out -} - -/// The names the fixed sources are spliced with. -pub(crate) struct Names<'a> { - /// The Swift module (and package and product) name. - pub(crate) module: &'a str, - /// The C module name, `C{module}`. - pub(crate) c_module: &'a str, - /// The native library base name. - pub(crate) library: &'a str, - /// The C header file name. - pub(crate) header: &'a str, -} - -/// The private runtime the wrapper file starts with, including the contract -/// check against every top-level module's checksum. -pub(crate) fn render_runtime(model: &BindingModel, names: &Names, runtime_error: &str) -> String { - let mut checks = String::new(); - for root in model.roots() { - let checksum = root - .checksum - .expect("every top-level module carries a checksum"); - let _ = writeln!( - checks, - " wvCheckModule(\"{}\", {}(), 0x{checksum:016x})", - root.name, - checksum_symbol(&model.prefix, &root.name) - ); - } - let checks = checks.trim_end_matches('\n'); - fill( - RUNTIME, - &[ - ("RUNTIME_ERROR", runtime_error), - ("PREFIX", &model.prefix), - ("MODULE", names.module), - ("LIBRARY", names.library), - ("ABI_VERSION", &ABI_VERSION.to_string()), - ("CONTRACT_CHECKS", checks), - ], - ) -} - -/// The `Package.swift` body (after the tools-version line and prelude). -pub(crate) fn render_package(names: &Names) -> String { - fill( - PACKAGE, - &[ - ("MODULE", names.module), - ("C_MODULE", names.c_module), - ("LIBRARY", names.library), - ], - ) -} - -/// The system-library module map declaring the bundled header. -pub(crate) fn render_module_map(names: &Names) -> String { - fill( - MODULE_MAP, - &[ - ("C_MODULE", names.c_module), - ("HEADER", names.header), - ("LIBRARY", names.library), - ], - ) -} - -/// The packaged README, listing the bundled `platforms`. -pub(crate) fn render_readme(names: &Names, platforms: &str) -> String { - fill( - README, - &[ - ("MODULE", names.module), - ("C_MODULE", names.c_module), - ("LIBRARY", names.library), - ("HEADER", names.header), - ("PLATFORMS", platforms), - ], - ) -} diff --git a/crates/weaveffi-gen/src/targets/swift/runtime/README.md b/crates/weaveffi-gen/src/targets/swift/runtime/README.md deleted file mode 100644 index 23300db3..00000000 --- a/crates/weaveffi-gen/src/targets/swift/runtime/README.md +++ /dev/null @@ -1,25 +0,0 @@ -# {{MODULE}} (Swift) - -A SwiftPM package (product and module `{{MODULE}}`) over the native library -`{{LIBRARY}}`, whose C declarations are the module `{{C_MODULE}}`. - -## Linking the bundled library - -The prebuilt libraries are bundled under `lib//`. Without further -setup the package links `{{LIBRARY}}` from the linker search path: - -```bash -swift build -Xlinker -L"$PWD/lib/darwin-arm64" -``` - -## iOS and the XCFramework - -When `{{C_MODULE}}.xcframework` sits next to `Package.swift`, the manifest uses -it as a binary target instead, so apps (including iOS apps) link the library -with no flags. `weaveffi package --target swift` assembles it on macOS -whenever the requested platforms include an iOS slice, for example -`--platforms ios-arm64,ios-sim-arm64,darwin-arm64`. - -## Bundled platforms - -{{PLATFORMS}} diff --git a/crates/weaveffi-gen/src/targets/wasm/runtime/linear.js b/crates/weaveffi-gen/src/targets/wasm/runtime/linear.js deleted file mode 100644 index 21a432d8..00000000 --- a/crates/weaveffi-gen/src/targets/wasm/runtime/linear.js +++ /dev/null @@ -1,394 +0,0 @@ -// The WebAssembly transport: loading the module, staging arguments in its -// linear memory, reading results and errors back, and installing JavaScript -// functions in its function table (async completions and callback -// interface vtables). The generated `index.js` builds one native entry point -// per C symbol on top of this. - -import { $Fault } from './runtime.js'; - -const $enc = new TextEncoder(); -const $dec = new TextDecoder(); - -function $expected(what, v) { - return new TypeError(`expected ${what}, got ${v === null ? 'null' : typeof v}`); -} - -function $bigint(v, bits, what) { - let b; - if (typeof v === 'bigint') { - b = v; - } else if (typeof v === 'number' && Number.isInteger(v)) { - b = BigInt(v); - } else { - throw $expected('a bigint', v); - } - if (bits(64, b) !== b) throw new RangeError(`bigint does not fit in ${what} 64-bit integer`); - return b; -} - -/** - * One loaded module's linear memory and runtime symbols. `x` holds the - * module's exports by C symbol name; `memory()` returns the current - * `ArrayBuffer`; `fnptr(params, results, sig, fn)` installs `fn` in the - * function table and returns its index. - */ -export class $Linear { - constructor(x, prefix, memory, fnptr) { - this.x = x; - this.memory = memory; - this.fnptr = fnptr; - this.allocFn = x[`${prefix}_alloc`]; - this.deallocFn = x[`${prefix}_dealloc`]; - this.freeBytesFn = x[`${prefix}_free_bytes`]; - this.errorClearFn = x[`${prefix}_error_clear`]; - this.errorFreeFn = x[`${prefix}_error_free`]; - this.errorSetFn = x[`${prefix}_error_set`]; - if (typeof this.allocFn !== 'function' || typeof this.deallocFn !== 'function') { - throw new Error(`the WebAssembly module does not export ${prefix}_alloc and ${prefix}_dealloc`); - } - // One scratch block reused by every call: the error slot (16 bytes), - // the out_len slot (4), and the iterator out_item slot (8). The producer - // only writes them as a call returns, so a call made from a callback in - // the middle of another call can reuse them safely. - const scratch = this.allocFn(32); - this.bytes().fill(0, scratch, scratch + 32); - this.err = scratch; - this.len = scratch + 16; - this.item = scratch + 24; - this.callbacks = new Map(); - this.nextCallback = 1; - this.pending = new Map(); - this.nextPending = 1; - this.completions = new Map(); - this.vtables = new Map(); - } - - bytes() { - const buffer = this.memory(); - if (buffer !== this.buffer) { - this.buffer = buffer; - this.u8 = new Uint8Array(buffer); - this.dv = new DataView(buffer); - } - return this.u8; - } - - view() { - this.bytes(); - return this.dv; - } - - // Argument checks and conversions for by-value slots. - num(v) { - if (typeof v !== 'number') throw $expected('a number', v); - return v; - } - - i64(v) { - return $bigint(v, BigInt.asIntN, 'a signed'); - } - - u64(v) { - return $bigint(v, BigInt.asUintN, 'an unsigned'); - } - - bool(v) { - if (typeof v !== 'boolean') throw $expected('a boolean', v); - return v ? 1 : 0; - } - - handle(v) { - if (typeof v !== 'number' || v === 0) throw $expected('an object handle', v); - return v; - } - - handleOpt(v) { - return v === null || v === undefined ? 0 : this.handle(v); - } - - // Staging: copy a string or bytes into linear memory for one call. A - // staged value is [ptr, len, size]; release it with unstage(). - str(s) { - if (typeof s !== 'string') throw $expected('a string', s); - if (s.length === 0) return [0, 0, 0]; - const size = s.length * 3; - const ptr = this.allocFn(size); - const { written } = $enc.encodeInto(s, this.bytes().subarray(ptr, ptr + size)); - return [ptr, written, size]; - } - - data(b) { - if (!(b instanceof Uint8Array)) throw $expected('a Uint8Array', b); - if (b.length === 0) return [0, 0, 0]; - const ptr = this.allocFn(b.length); - this.bytes().set(b, ptr); - return [ptr, b.length, b.length]; - } - - unstage(staged) { - if (staged !== null && staged[2] !== 0) this.deallocFn(staged[0], staged[2]); - } - - // Results. - outLen() { - return this.view().getUint32(this.len, true); - } - - readStr(ptr, len) { - return len === 0 ? '' : $dec.decode(this.bytes().subarray(ptr, ptr + len)); - } - - readData(ptr, len) { - return len === 0 ? new Uint8Array(0) : this.bytes().slice(ptr, ptr + len); - } - - takeStr(ptr, len) { - const s = this.readStr(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return s; - } - - takeData(ptr, len) { - const b = this.readData(ptr, len); - if (ptr !== 0) this.freeBytesFn(ptr, len); - return b; - } - - cstr(ptr) { - if (ptr === 0) return ''; - const u8 = this.bytes(); - let end = ptr; - while (u8[end] !== 0) end++; - return $dec.decode(u8.subarray(ptr, end)); - } - - fault(errPtr) { - const dv = this.view(); - const code = dv.getInt32(errPtr, true); - if (code === 0) return null; - const payloadPtr = dv.getUint32(errPtr + 8, true); - const payloadLen = dv.getUint32(errPtr + 12, true); - return new $Fault( - code, - this.cstr(dv.getUint32(errPtr + 4, true)), - payloadPtr === 0 ? null : this.bytes().slice(payloadPtr, payloadPtr + payloadLen), - ); - } - - /** Throw the error the last call wrote to the scratch error slot. */ - check() { - if (this.view().getInt32(this.err, true) === 0) return; - const fault = this.fault(this.err); - this.errorClearFn(this.err); - throw fault; - } - - // Callback interfaces: implementations live in a table keyed by the - // integer passed as `ctx`, until the producer calls the vtable's `free`. - register(adapter) { - const id = this.nextCallback++; - this.callbacks.set(id, adapter); - return id; - } - - adapter(id) { - const adapter = this.callbacks.get(id); - if (adapter === undefined) throw new Error('callback implementation was already released'); - return adapter; - } - - unregister(id) { - this.callbacks.delete(id); - } - - /** Report a callback implementation's exception through `errPtr` (code -4). */ - foreign(errPtr, e) { - const message = e instanceof Error ? e.message || e.name : String(e); - const bytes = $enc.encode(message); - const ptr = this.allocFn(bytes.length + 1); - const u8 = this.bytes(); - u8.set(bytes, ptr); - u8[ptr + bytes.length] = 0; - this.errorSetFn(errPtr, -4, ptr); - this.deallocFn(ptr, bytes.length + 1); - } - - /** - * The static vtable of one callback interface: one table entry per - * method (`methods()` returns each as `[params, results, sig, fn]`), then - * `free`. Built on first use. - */ - vtable(name, methods) { - let ptr = this.vtables.get(name); - if (ptr !== undefined) return ptr; - const entries = [...methods(), [['i32'], [], 'vi', (ctx) => this.unregister(ctx)]]; - ptr = this.allocFn(entries.length * 4); - const dv = this.view(); - entries.forEach(([params, results, sig, fn], i) => { - dv.setUint32(ptr + i * 4, this.fnptr(params, results, sig, fn), true); - }); - this.vtables.set(name, ptr); - return ptr; - } - - /** - * Launch an async call. `call(callback, context)` invokes the launcher; - * the completion (one table entry per signature, shared by every call) - * settles the returned promise, converting the result slots with - * `convert`. On this single-threaded target the completion normally - * fires before the launcher returns. - */ - launch(params, results, sig, call, convert) { - let callback = this.completions.get(sig); - if (callback === undefined) { - const complete = (ctx, errPtr, ...result) => this.complete(ctx, errPtr, result); - callback = this.fnptr(params, results, sig, complete); - this.completions.set(sig, callback); - } - const id = this.nextPending++; - return new Promise((resolve, reject) => { - this.pending.set(id, { resolve, reject, convert }); - try { - call(callback, id); - } catch (e) { - this.pending.delete(id); - reject(e); - } - }); - } - - complete(id, errPtr, result) { - const pending = this.pending.get(id); - if (pending === undefined) return; - this.pending.delete(id); - try { - const fault = errPtr === 0 ? null : this.fault(errPtr); - if (errPtr !== 0) this.errorFreeFn(errPtr); - if (fault !== null) { - pending.reject(fault); - } else { - pending.resolve(pending.convert(...result)); - } - } catch (e) { - pending.reject(e); - } - } -} - -/** A stand-in for the native entry points until `init()` has finished. */ -export function $unloaded(library) { - return new Proxy( - {}, - { - get() { - throw new Error(`${library}: call init() before using the bindings`); - }, - }, - ); -} - -const $valtype = { i32: 0x7f, i64: 0x7e, f32: 0x7d, f64: 0x7c }; - -function $leb(n) { - const out = []; - do { - let b = n & 0x7f; - n >>>= 7; - if (n !== 0) b |= 0x80; - out.push(b); - } while (n !== 0); - return out; -} - -function $section(id, body) { - return [id, ...$leb(body.length), ...body]; -} - -/** - * Turn a JavaScript function into a WebAssembly function with the given - * signature: `WebAssembly.Function` where the engine has it, otherwise a - * tiny module that imports `fn` and re-exports it. - */ -function $wasmFunction(params, results, fn) { - if (typeof WebAssembly.Function === 'function') { - return new WebAssembly.Function({ parameters: params, results }, fn); - } - const type = [0x60, ...$leb(params.length), ...params.map((t) => $valtype[t])]; - type.push(...$leb(results.length), ...results.map((t) => $valtype[t])); - const bytes = new Uint8Array([ - 0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, - ...$section(1, [1, ...type]), - ...$section(2, [1, 1, 0x65, 1, 0x66, 0x00, 0x00]), - ...$section(7, [1, 1, 0x66, 0x00, 0x00]), - ]); - const instance = new WebAssembly.Instance(new WebAssembly.Module(bytes), { e: { f: fn } }); - return instance.exports.f; -} - -async function $bytesOf(source, envVar, fallback) { - if (source === undefined || source === null) { - const env = typeof process === 'object' && process.env ? process.env[envVar] : undefined; - if (env) { - const { readFile } = await import('node:fs/promises'); - return readFile(env); - } - source = fallback; - } - if (source instanceof WebAssembly.Module || source instanceof ArrayBuffer || ArrayBuffer.isView(source)) { - return source; - } - if (typeof Response === 'function' && source instanceof Response) { - return source.arrayBuffer(); - } - const url = source instanceof URL ? source : new URL(String(source), fallback); - if (url.protocol === 'file:') { - const { readFile } = await import('node:fs/promises'); - return readFile(url); - } - const response = await fetch(url); - if (!response.ok) throw new Error(`failed to fetch ${url}: ${response.status}`); - return response.arrayBuffer(); -} - -/** - * Instantiate a `wasm32-unknown-unknown` module from `source` (a URL, a - * path, bytes, a `Response`, or a compiled module). Without a source it - * reads the file named by the `envVar` environment variable (on Node.js), - * else `fallback`. The module must export its memory and a growable - * function table (`__indirect_function_table`). - */ -export async function $loadWasm(source, prefix, envVar, fallback) { - const bytes = await $bytesOf(source, envVar, fallback); - const module = bytes instanceof WebAssembly.Module ? bytes : await WebAssembly.compile(bytes); - const { exports } = await WebAssembly.instantiate(module, {}); - const table = exports.__indirect_function_table; - const fnptr = (params, results, sig, fn) => { - if (!(table instanceof WebAssembly.Table)) { - throw new Error('the WebAssembly module does not export __indirect_function_table (link with --export-table --growable-table)'); - } - const index = table.grow(1); - table.set(index, $wasmFunction(params, results, fn)); - return index; - }; - return new $Linear(exports, prefix, () => exports.memory.buffer, fnptr); -} - -/** - * Adopt an initialized Emscripten module (or the promise its `MODULARIZE` - * factory returns). Its exports carry a leading underscore, its memory is - * read through `HEAPU8`, and table entries are installed with - * `addFunction`, so it must be linked with `-sALLOW_TABLE_GROWTH`, - * `-sWASM_BIGINT`, and `-sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8`. - */ -export async function $loadEmscripten(module, prefix, symbols) { - const m = await module; - const x = {}; - for (const symbol of symbols) x[symbol] = m[`_${symbol}`]; - const fnptr = (params, results, sig, fn) => { - if (typeof m.addFunction !== 'function') { - throw new Error('the Emscripten module does not export addFunction (link with -sALLOW_TABLE_GROWTH and -sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8)'); - } - return m.addFunction(fn, sig); - }; - return new $Linear(x, prefix, () => m.HEAPU8.buffer, fnptr); -} diff --git a/crates/weaveffi-gen/src/utils.rs b/crates/weaveffi-gen/src/utils.rs deleted file mode 100644 index cc04bf60..00000000 --- a/crates/weaveffi-gen/src/utils.rs +++ /dev/null @@ -1,213 +0,0 @@ -//! Shared text helpers for the generators: the standard generated-file -//! prelude/trailer and cross-module type-name resolution. -//! -//! These are the small string-building routines every backend reaches for -//! when stamping the "generated by WeaveFFI" banner onto a file or turning a -//! qualified `module.Type` reference into its flattened C symbol. Centralizing -//! them keeps the banner and the dotted-path flattening identical across all -//! of the language generators. - -/// Comment syntax used to emit the standard prelude/trailer in generated files. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum CommentStyle { - /// `// ...` line comments (C, C++, Swift, Kotlin, JS/TS, C#, Dart, Go). - DoubleSlash, - /// `# ...` line comments (Python, Ruby, YAML, TOML, CMake, GYP, gradle.properties). - Hash, - /// `` block comments (XML, HTML, Markdown). - Xml, -} - -impl CommentStyle { - fn open(self) -> &'static str { - match self { - Self::DoubleSlash => "// ", - Self::Hash => "# ", - Self::Xml => "", - _ => "", - } - } -} - -/// Renders the standard `Generated by WeaveFFI {VERSION} from {input}` prelude -/// followed by the `DO NOT EDIT` warning and a regenerate command. Trailing -/// blank line included so generators can append their content directly. -pub fn render_prelude(style: CommentStyle, input_basename: &str) -> String { - let version = env!("CARGO_PKG_VERSION"); - let o = style.open(); - let c = style.close(); - format!( - "{o}Generated by WeaveFFI {version} from {input_basename}{c}\n\ - {o}DO NOT EDIT. Your changes will be overwritten.{c}\n\ - {o}To regenerate: weaveffi generate {input_basename} -o {c}\n\n" - ) -} - -/// Renders the closing `END {filename}` marker. Caller is responsible for any -/// preceding newline; the returned string ends with `\n`. -pub fn render_trailer(style: CommentStyle, filename: &str) -> String { - let o = style.open(); - let c = style.close(); - format!("{o}END {filename}{c}\n") -} - -/// Renders the JSON-friendly prelude as `"//"` key/value pairs (recognised by -/// npm). Each line is two-space-indented and comma-terminated so it can be -/// embedded at the top of any JSON object literal that opens with `{` on the -/// previous line. -pub fn render_json_prelude(input_basename: &str) -> String { - let version = env!("CARGO_PKG_VERSION"); - format!( - " \"//\": \"Generated by WeaveFFI {version} from {input_basename}\",\n \ - \"//warning\": \"DO NOT EDIT. Your changes will be overwritten.\",\n \ - \"//regenerate\": \"To regenerate: weaveffi generate {input_basename} -o \",\n" - ) -} - -/// Build the wrapper function name exposed to the foreign language. -/// -/// When `strip_module_prefix` is `true`, returns just `func`. -/// When `false`, returns `{module}_{func}`. -pub fn wrapper_name(module: &str, func: &str, strip_module_prefix: bool) -> String { - if strip_module_prefix { - func.to_string() - } else { - format!("{module}_{func}") - } -} - -/// Extract the local type name from a potentially qualified `module.TypeName`. -/// -/// Uses `rsplit_once` so that *multi-level* module paths keep working: only the -/// final dotted segment is the type name. -/// -/// `"other.Contact"` → `"Contact"`, `"a.b.Widget"` → `"Widget"`, -/// `"Contact"` → `"Contact"`. -pub fn local_type_name(name: &str) -> &str { - name.rsplit_once('.').map_or(name, |(_, local)| local) -} - -/// Build the C ABI struct name, resolving cross-module qualified references. -/// -/// Qualified references use dot-separated module paths; the C ABI flattens -/// those to underscore-joined symbol prefixes, so `rsplit_once` peels off the -/// type name and the remaining dotted path becomes underscores. -/// -/// `"other.Contact"` with any current module → `"{prefix}_other_Contact"`. -/// `"a.b.Widget"` with any current module → `"{prefix}_a_b_Widget"`. -/// `"Contact"` with current module `"math"` → `"{prefix}_math_Contact"`. -pub fn c_abi_struct_name(name: &str, current_module: &str, prefix: &str) -> String { - if let Some((module_path, type_name)) = name.rsplit_once('.') { - let module_path = module_path.replace('.', "_"); - format!("{prefix}_{module_path}_{type_name}") - } else { - format!("{prefix}_{current_module}_{name}") - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn local_type_name_unqualified() { - assert_eq!(local_type_name("Contact"), "Contact"); - } - - #[test] - fn local_type_name_qualified() { - assert_eq!(local_type_name("other.Contact"), "Contact"); - } - - #[test] - fn local_type_name_multi_level() { - assert_eq!(local_type_name("a.b.Widget"), "Widget"); - } - - #[test] - fn c_abi_struct_name_unqualified() { - assert_eq!( - c_abi_struct_name("Contact", "math", "weaveffi"), - "weaveffi_math_Contact" - ); - } - - #[test] - fn c_abi_struct_name_qualified() { - assert_eq!( - c_abi_struct_name("types.Name", "ops", "weaveffi"), - "weaveffi_types_Name" - ); - } - - #[test] - fn c_abi_struct_name_multi_level_flattens_path() { - assert_eq!( - c_abi_struct_name("a.b.Widget", "ops", "weaveffi"), - "weaveffi_a_b_Widget" - ); - } - - #[test] - fn prelude_double_slash_carries_required_phrases() { - let p = render_prelude(CommentStyle::DoubleSlash, "calc.yml"); - assert!(p.starts_with("// Generated by WeaveFFI ")); - assert!(p.contains(" from calc.yml\n")); - assert!(p.contains("// DO NOT EDIT")); - assert!(p.contains("// To regenerate: weaveffi generate calc.yml -o ")); - assert!(p.ends_with("\n\n")); - } - - #[test] - fn prelude_hash_uses_hash_marker() { - let p = render_prelude(CommentStyle::Hash, "calc.yml"); - assert!(p.starts_with("# Generated by WeaveFFI ")); - assert!(p.contains("# DO NOT EDIT")); - assert!(p.contains("# To regenerate: weaveffi generate calc.yml -o ")); - } - - #[test] - fn prelude_xml_wraps_lines_in_brackets() { - let p = render_prelude(CommentStyle::Xml, "calc.yml"); - assert!(p.starts_with("")); - assert!(p.contains("")); - } - - #[test] - fn trailer_has_correct_marker_per_style() { - assert_eq!( - render_trailer(CommentStyle::DoubleSlash, "lib.rs"), - "// END lib.rs\n" - ); - assert_eq!( - render_trailer(CommentStyle::Hash, "build.toml"), - "# END build.toml\n" - ); - assert_eq!( - render_trailer(CommentStyle::Xml, "package.csproj"), - "\n" - ); - } - - #[test] - fn json_prelude_contains_required_phrases_in_first_lines() { - let p = render_json_prelude("calc.yml"); - let lines: Vec<&str> = p.lines().collect(); - assert!(lines[0].contains("Generated by WeaveFFI")); - assert!(lines[0].contains("from calc.yml")); - assert!(lines[1].contains("DO NOT EDIT")); - assert!(lines[2].contains("To regenerate")); - for line in &lines { - assert!(line.starts_with(" ")); - assert!(line.ends_with(',')); - } - } -} diff --git a/crates/weaveffi-gen/tests/dart_render.rs b/crates/weaveffi-gen/tests/dart_render.rs deleted file mode 100644 index 9cf19f17..00000000 --- a/crates/weaveffi-gen/tests/dart_render.rs +++ /dev/null @@ -1,457 +0,0 @@ -//! Dart rendering checks: identity-driven names and loading, ABI 3 string -//! and runtime symbols, the load-time contract check, object lifetimes, -//! nullable objects, object tokens in value buffers, iterators, async -//! cancellation, callback-interface vtables (isolate-local value methods and -//! forwarded void methods), and packaging. - -use camino::Utf8Path; -use weaveffi_gen::backend::LanguageBackend; -use weaveffi_gen::package::{FileContent, PackageContext}; -use weaveffi_gen::platform::{BinarySet, Platform}; -use weaveffi_gen::targets::dart::{DartConfig, DartGenerator}; -use weaveffi_model::ir::{ - Api, CallbackInterfaceDef, EnumDef, EnumVariant, Function, InterfaceDef, Module, Param, - StructDef, StructField, TypeRef, CURRENT_SCHEMA_VERSION, -}; -use weaveffi_model::model::BindingModel; -use weaveffi_model::pkg::Identity; -use weaveffi_model::resolved::ResolvedApi; -use weaveffi_model::validate::validate_api; - -fn param(name: &str, ty: TypeRef) -> Param { - Param { - name: name.into(), - ty, - doc: None, - } -} - -fn func(name: &str, params: Vec, returns: Option) -> Function { - Function { - name: name.into(), - params, - returns, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - } -} - -fn field(name: &str, ty: TypeRef) -> StructField { - StructField { - name: name.into(), - ty, - doc: None, - } -} - -fn named(name: &str) -> TypeRef { - TypeRef::Named(name.into()) -} - -/// One module exercising every shape: an interface, `Interface?` both ways, -/// a record holding objects, an iterator of objects, a string echo, a -/// cancellable async call, and a callback interface with value-returning -/// and void methods. -fn fixture() -> Api { - Api { - version: CURRENT_SCHEMA_VERSION.into(), - modules: vec![Module { - name: "bus".into(), - doc: None, - functions: vec![ - func( - "lookup", - vec![param( - "fallback", - TypeRef::Optional(Box::new(named("Ticker"))), - )], - Some(TypeRef::Optional(Box::new(named("Ticker")))), - ), - func( - "tickers", - vec![], - Some(TypeRef::Iterator(Box::new(named("Ticker")))), - ), - func( - "echo", - vec![param("text", TypeRef::StringUtf8)], - Some(TypeRef::StringUtf8), - ), - func( - "subscribe", - vec![param("listener", named("Subscriber"))], - None, - ), - func( - "describe", - vec![param("env", named("Envelope"))], - Some(named("Envelope")), - ), - Function { - r#async: true, - cancellable: true, - ..func( - "fetch", - vec![param("id", TypeRef::I64)], - Some(TypeRef::Optional(Box::new(named("Ticker")))), - ) - }, - ], - interfaces: vec![InterfaceDef { - name: "Ticker".into(), - doc: Some("A counter.".into()), - deprecated: None, - constructors: vec![func("new", vec![param("start", TypeRef::I64)], None)], - methods: vec![func("value", vec![], Some(TypeRef::I64))], - statics: vec![], - }], - callback_interfaces: vec![CallbackInterfaceDef { - name: "Subscriber".into(), - doc: Some("Receives bus events.".into()), - deprecated: None, - methods: vec![ - func( - "on_message", - vec![ - param("text", TypeRef::StringUtf8), - param("weight", TypeRef::I32), - param("envelope", named("Envelope")), - ], - Some(TypeRef::Bool), - ), - func( - "on_ticker", - vec![ - param("label", TypeRef::StringUtf8), - param("ticker", named("Ticker")), - param("alt", TypeRef::Optional(Box::new(named("Ticker")))), - ], - None, - ), - func( - "classify", - vec![param("weight", TypeRef::I32)], - Some(named("Priority")), - ), - ], - }], - structs: vec![StructDef { - name: "Envelope".into(), - doc: None, - deprecated: None, - fields: vec![ - field("topic", TypeRef::StringUtf8), - field("primary", named("Ticker")), - field("others", TypeRef::List(Box::new(named("Ticker")))), - ], - }], - enums: vec![EnumDef { - name: "Priority".into(), - doc: None, - deprecated: None, - variants: vec![ - EnumVariant { - name: "Low".into(), - value: 0, - doc: None, - fields: vec![], - }, - EnumVariant { - name: "High".into(), - value: 1, - doc: None, - fields: vec![], - }, - ], - }], - errors: None, - modules: vec![], - }], - } -} - -fn api() -> ResolvedApi { - validate_api(fixture(), None) - .expect("fixture validates") - .with_identity(Identity::named("bus-kit")) -} - -fn render() -> String { - let api = api(); - let model = BindingModel::build(&api); - let files = DartGenerator.files(&api, &model, Utf8Path::new("out"), &DartConfig::default()); - let paths: Vec<&str> = files.iter().map(|f| f.path.as_str()).collect(); - assert_eq!( - paths, - [ - "out/dart/lib/bus_kit.dart", - "out/dart/pubspec.yaml", - "out/dart/README.md" - ] - ); - files[0].contents.clone() -} - -fn assert_has(src: &str, needle: &str) { - assert!(src.contains(needle), "missing `{needle}` in:\n{src}"); -} - -#[test] -fn identity_names_the_package_and_library() { - let api = api(); - let model = BindingModel::build(&api); - let files = DartGenerator.files(&api, &model, Utf8Path::new("out"), &DartConfig::default()); - assert_has(&files[1].contents, "name: bus_kit\n"); - assert_has(&files[2].contents, "import 'package:bus_kit/bus_kit.dart';"); - - let config = DartConfig { - package_name: Some("custom".into()), - ..DartConfig::default() - }; - let files = DartGenerator.files(&api, &model, Utf8Path::new("out"), &config); - assert_eq!(files[0].path.as_str(), "out/dart/lib/custom.dart"); - assert_has(&files[1].contents, "name: custom\n"); - - let src = render(); - assert_has(&src, "Platform.environment['BUS_KIT_LIBRARY']"); - assert_has(&src, "const ['libbus_kit.dylib']"); - assert_has(&src, "const ['bus_kit.dll']"); - assert_has(&src, "if (Platform.isIOS) return DynamicLibrary.process();"); - assert!( - !src.to_lowercase().contains("weaveffi_"), - "branded symbol in output" - ); -} - -#[test] -fn library_load_checks_abi_and_checksums() { - let api = api(); - let model = BindingModel::build(&api); - let checksum = model.modules[0].checksum.expect("root checksum"); - let src = render(); - assert_has(&src, "const int _abiVersion = 3;"); - assert_has(&src, "'bus_kit_abi_version'"); - assert_has( - &src, - &format!(" ('bus', 'bus_kit_bus_checksum', 0x{checksum:016x}),"), - ); - assert_has(&src, "regenerate the bindings"); -} - -#[test] -fn strings_cross_as_pointer_and_length() { - let src = render(); - assert_has( - &src, - "Pointer Function(Pointer, Size, Pointer, Pointer<_Error>)", - ); - assert_has(&src, "final _textBytes = utf8.encode(text);"); - assert_has( - &src, - "_busKitBusEcho(_stage(_arena, _textBytes), _textBytes.length, _outLen, _err)", - ); - assert_has(&src, "return _takeString(_result, _outLen.value);"); - assert_has(&src, "'bus_kit_free_bytes'"); - assert!(!src.contains("free_string"), "ABI 2 free_string in output"); -} - -#[test] -fn objects_are_guarded_reference_counted_wrappers() { - let src = render(); - assert_has(&src, "final class Ticker extends _NativeObject {"); - assert_has(&src, "('bus_kit_bus_Ticker_clone')"); - assert_has(&src, "('bus_kit_bus_Ticker_destroy')"); - assert_has( - &src, - "NativeFinalizer get _finalizer => _busKitBusTickerDestroyFinalizer;", - ); - // A dispose during an in-flight call defers the release. - assert_has(&src, "if (_calls == 0) _release();"); - assert_has(&src, "if (--_calls == 0 && _disposed) _release();"); - assert_has(&src, "factory Ticker(int start) {"); - assert_has(&src, "return Ticker._(_result);"); - assert_has(&src, "final _self = _enter();"); - assert_has(&src, "_busKitBusTickerValue(_self, _err)"); -} - -#[test] -fn nullable_objects_are_borrowed_and_adopted() { - let src = render(); - assert_has(&src, "Ticker? lookup(Ticker? fallback) {"); - assert_has( - &src, - "final _fallbackPtr = fallback == null ? nullptr : _borrow(_arena, fallback);", - ); - assert_has( - &src, - "return _result == nullptr ? null : Ticker._(_result);", - ); -} - -#[test] -fn objects_inside_records_are_cloned_tokens() { - let src = render(); - assert_has(&src, "w.writeU64(v.primary._cloneRef().address);"); - assert_has(&src, "w.writeU64(_t1._cloneRef().address);"); - assert_has( - &src, - "primary: Ticker._(Pointer.fromAddress(r.readU64())),", - ); - assert_has( - &src, - "others: [for (var i = r.readLength(); i > 0; i--) Ticker._(Pointer.fromAddress(r.readU64()))],", - ); -} - -#[test] -fn iterators_pull_lazily_and_destroy_once() { - let src = render(); - assert_has(&src, "Iterable tickers() sync* {"); - assert_has( - &src, - "_busKitBusTickersIteratorNext(_iter, _outItem.cast(), _err)", - ); - assert_has( - &src, - "yield Ticker._(_outItem.cast>().value);", - ); - assert_has( - &src, - "_busKitBusTickersIteratorDestroyFinalizer.attach(_anchor, _iter, detach: _anchor);", - ); - assert_has(&src, "_busKitBusTickersIteratorDestroy(_iter);"); -} - -#[test] -fn cancellable_async_binds_a_native_token() { - let src = render(); - assert_has( - &src, - "Future fetch(int id, {CancelToken? cancelToken}) {", - ); - assert_has(&src, "final _cancel = _NativeCancel.bind(cancelToken);"); - assert_has(&src, "NativeCallable<_BusKitBusFetchCallback>.listener(("); - assert_has( - &src, - "_busKitBusFetch(id, _cancel?.pointer ?? nullptr, _callback.nativeFunction, nullptr);", - ); - assert_has(&src, "_cancel?.release();"); - assert_has(&src, "'bus_kit_cancel_token_create'"); - assert_has(&src, "class CancelledException extends NativeException {"); - assert_has(&src, "static const int cancelledCode = -5;"); -} - -#[test] -fn callback_interfaces_split_value_and_void_methods() { - let src = render(); - assert_has(&src, "abstract class Subscriber {"); - assert_has( - &src, - "bool onMessage(String text, int weight, Envelope envelope);", - ); - assert_has( - &src, - "void onTicker(String label, Ticker ticker, Ticker? alt);", - ); - // A value-returning method is an isolate-local trampoline that reports - // a thrown exception as the foreign code. - assert_has( - &src, - "..onMessage = _pin(NativeCallable<_BusKitBusSubscriberOnMessage>.isolateLocal(_subscriberOnMessage, exceptionalReturn: false))", - ); - assert_has( - &src, - "return (_callbackTarget(ctx) as Subscriber).onMessage(_readString(textPtr, textLen), weight, _decode(_copyBytes(envelopePtr, envelopeLen), _unpackEnvelope));", - ); - assert_has(&src, "_foreignError(outErr, e);"); - assert_has( - &src, - "return (_callbackTarget(ctx) as Subscriber).classify(weight).value;", - ); - // A void method is forwarded from any thread and dispatched on the - // event loop, adopting objects before decoding the rest. - assert_has( - &src, - "..onTicker = _pin(NativeCallable<_BusKitBusSubscriberOnTicker>.isolateGroupBound(_subscriberOnTicker))", - ); - assert_has( - &src, - " _CallbackMessage(ctx, 1, 3)\n ..bytes(labelPtr, labelLen)\n ..pointer(ticker)\n ..pointer(alt)\n ..send();", - ); - assert_has( - &src, - "final _a3Address = message[3]! as int;\n final _a3 = Ticker._(Pointer.fromAddress(_a3Address));", - ); - assert_has(&src, "final _a2 = utf8.decode(message[2]! as Uint8List);"); - assert_has(&src, "target.onTicker(_a2, _a3, _a4);"); - assert_has(&src, "..free = _callbackFree;"); - // Passing an implementation registers it last. - assert_has( - &src, - "final _listenerCtx = _registerCallback(listener, _subscriberDispatch);", - ); - assert_has( - &src, - "_busKitBusSubscribe(_listenerCtx, _subscriberVtable.cast(), _err);", - ); - // No call may be a leaf call when a callback can re-enter Dart. - assert!(!src.contains("isLeaf: true);\n\nfinal _busKitBus")); - assert_has(&src, "'bus_kit_error_set'"); -} - -#[test] -fn package_bundles_only_desktop_binaries() { - let api = api(); - let model = BindingModel::build(&api); - let mut binaries = BinarySet::new("bus_kit"); - for p in Platform::ALL { - binaries.insert(p, format!("/prebuilt/{}/lib", p.id())); - } - let ctx = PackageContext { - binaries: &binaries, - input_basename: Some("bus.yml"), - }; - let files = DartGenerator - .package( - &api, - &model, - &ctx, - Utf8Path::new("out"), - &DartConfig::default(), - ) - .expect("dart packages"); - let bundled: Vec<&str> = files - .iter() - .filter(|f| f.is_binary()) - .map(|f| f.path.as_str()) - .collect(); - assert_eq!(bundled.len(), Platform::DESKTOP.len()); - for p in [ - Platform::AndroidArm64, - Platform::AndroidX64, - Platform::Wasm32, - ] { - assert!( - !bundled.iter().any(|b| b.contains(p.id())), - "{} bundled", - p.id() - ); - } - let src = files - .iter() - .find(|f| f.path.as_str().ends_with("lib/bus_kit.dart")) - .and_then(|f| match &f.content { - FileContent::Text(s) => Some(s.as_str()), - FileContent::Copy(_) => None, - }) - .expect("packaged source"); - assert_has( - src, - "'native/darwin-arm64/libbus_kit.dylib', 'native/darwin-x64/libbus_kit.dylib', 'libbus_kit.dylib'", - ); - assert_has(src, "'native/windows-x64/bus_kit.dll', 'bus_kit.dll'"); - assert_has(src, "Platform.environment['BUS_KIT_LIBRARY']"); -} diff --git a/crates/weaveffi-macros/src/codegen/async_fns.rs b/crates/weaveffi-macros/src/codegen/async_fns.rs index 6d8c5584..bbbb8aba 100644 --- a/crates/weaveffi-macros/src/codegen/async_fns.rs +++ b/crates/weaveffi-macros/src/codegen/async_fns.rs @@ -7,233 +7,23 @@ //! the consumer*. A non-null `err` is heap-boxed and released with //! `{prefix}_error_free`; a string, bytes, or buffered result is a //! `(result_ptr, result_len)` run released with `{prefix}_free_bytes`; an -//! object result transfers one strong reference. This ownership transfer -//! (unlike the borrow-and-copy contract of synchronous `out_err` slots) is -//! what lets consumers defer decoding past the callback's return, which -//! runtimes such as Dart's `NativeCallable.listener` require. +//! object result transfers one strong reference. //! -//! The launcher lifts its inputs on the caller's thread and hands the rest to -//! [`weaveffi_abi::run_async`], which owns the exactly-once guarantee: it -//! catches panics, races a `cancellable` call against its token (dropping the -//! producer's future and completing with the cancelled code when the token -//! fires), and completes with the cancelled code from a drop guard if the -//! executor drops the task early. +//! The launcher hands three closures to [`weaveffi::abi::launch_async`]: one +//! that lifts the inputs into owned values on the caller's thread (under +//! `catch_unwind`) and builds the producer's future, one that lowers the +//! result into the callback's slots, and one that invokes the callback. The +//! runtime owns the exactly-once guarantee: it catches panics, races a +//! `cancellable` call against its token, and completes from a drop guard if +//! the executor drops the task early. use proc_macro2::TokenStream; use quote::quote; -use weaveffi_model::model::{AsyncBinding, FnBinding, ParamBinding, Ty}; +use weaveffi_model::model::{AsyncBinding, FnBinding}; -use super::helpers::{ - ctype_to_rust, fn_slots, ident, is_copy, rust_type_ident, sentinel, thunk_attrs, CallTarget, - UserSig, -}; +use super::helpers::{ctype_to_rust, fn_slots, ident, thunk_attrs, CallTarget, UserSig}; +use super::lift::{lift_param, lower_async_result, Lifts}; use super::sync::throws; -use super::unsupported; - -/// The "fire the callback with a marshalling error and leave the launcher" -/// tail for an input rejected on the caller's thread. -fn reject_pre(msg: &str, sentinels: &[TokenStream]) -> TokenStream { - quote! {{ - callback( - context, - ::weaveffi::abi::boxed_error(::weaveffi::abi::FfiError::new( - ::weaveffi::abi::MARSHAL_ERROR_CODE, - #msg, - )) - #(, #sentinels)* - ); - return; - }} -} - -/// The `return Err(..)` tail for an input rejected inside the future. -fn reject_in_future(msg: &str) -> TokenStream { - quote! { - return ::std::result::Result::Err(::weaveffi::abi::FfiError::new( - ::weaveffi::abi::MARSHAL_ERROR_CODE, - #msg, - )) - } -} - -/// Lift one async-launcher input into an *owned* Rust value, as `(pre_spawn, -/// in_future, arg)`. -/// -/// Async inputs must own their data before the future is spawned: the -/// consumer may free or reuse the argument buffers as soon as the launcher -/// returns. So strings and bytes are copied and objects retained (a new -/// strong reference) on the caller's thread; a borrowed spelling (`&str`, -/// `&T`) is then satisfied by lending the owned value. There is no `out_err` -/// slot on a launcher, so an invalid input fires the completion callback with -/// a marshalling error, which keeps the "exactly once" promise. -fn lift_async_input( - pb: &ParamBinding, - user: &UserSig<'_>, - sentinels: &[TokenStream], -) -> syn::Result<(TokenStream, TokenStream, TokenStream)> { - let name = ident(&pb.name); - let none = TokenStream::new(); - let arg = if user.param_is_ref(&pb.name) { - quote!(&#name) - } else { - quote!(#name) - }; - let invalid = reject_pre(&format!("{} is null or invalid", pb.name), sentinels); - let ptr = ident(&format!("{}_ptr", pb.name)); - let len = ident(&format!("{}_len", pb.name)); - - // A buffered parameter is copied on the caller's thread and decoded - // inside the future (a malformed buffer is delivered through the - // callback's `err` with the marshalling code). - if pb.ty.is_buffered() { - let decode_fail = reject_in_future(&format!("{}: malformed value buffer", pb.name)); - return Ok(( - quote! { - let #name = match ::weaveffi::abi::lift_bytes(#ptr, #len) { - ::std::option::Option::Some(__wv_v) => __wv_v, - ::std::option::Option::None => #invalid - }; - }, - quote! { - let #name = match ::weaveffi::abi::decode_value(&#name) { - ::std::result::Result::Ok(__wv_v) => __wv_v, - ::std::result::Result::Err(_) => { #decode_fail } - }; - }, - arg, - )); - } - Ok(match &pb.ty { - Ty::Enum(enum_name) => { - let et = user - .param_object(&pb.name) - .unwrap_or_else(|| quote::ToTokens::into_token_stream(rust_type_ident(enum_name))); - ( - quote! { - let #name = match <#et>::__weaveffi_from_i32(#name) { - ::std::option::Option::Some(__wv_v) => __wv_v, - ::std::option::Option::None => #invalid - }; - }, - none, - arg, - ) - } - ty if is_copy(ty) => (none.clone(), none, arg), - Ty::StringUtf8 | Ty::Bytes => { - let lift = if matches!(pb.ty, Ty::StringUtf8) { - quote!(lift_string) - } else { - quote!(lift_bytes) - }; - ( - quote! { - let #name = match ::weaveffi::abi::#lift(#ptr, #len) { - ::std::option::Option::Some(__wv_v) => __wv_v, - ::std::option::Option::None => #invalid - }; - }, - none, - arg, - ) - } - // An object is always retained across the spawn (the consumer may - // release its own reference the moment the launcher returns). A `&T` - // spelling is satisfied by lending the `Arc`, which derefs to `&T`. - Ty::Interface(_) => { - let arg = if user.param_wants_arc(&pb.name) { - arg - } else { - quote!(&#name) - }; - ( - quote! { - let #name = match ::weaveffi::abi::object_arc(#name) { - ::std::option::Option::Some(__wv_o) => __wv_o, - ::std::option::Option::None => #invalid - }; - }, - none, - arg, - ) - } - Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => { - let arg = if user.param_wants_arc(&pb.name) { - arg - } else { - quote!(#name.as_deref()) - }; - ( - quote!(let #name = ::weaveffi::abi::object_arc(#name);), - none, - arg, - ) - } - Ty::CallbackInterface(_) => { - let dyn_ty = user.param_callback(&pb.name)?; - let ctx = ident(&format!("{}_ctx", pb.name)); - let vtable = ident(&format!("{}_vtable", pb.name)); - let null_fail = reject_pre(&format!("{}: null callback vtable", pb.name), sentinels); - ( - quote! { - let #name = match ::weaveffi::abi::lift_callback::<#dyn_ty>(#ctx, #vtable) { - ::std::option::Option::Some(__wv_cb) => __wv_cb, - ::std::option::Option::None => #null_fail - }; - }, - none, - arg, - ) - } - _ => { - return Err(unsupported( - user.param_span(&pb.name), - &pb.name, - "async parameter type", - )) - } - }) -} - -/// The expression lowering the future's output `__wv_val` into the -/// completion callback's *result* arguments (the slots after `context` and -/// `err`), as a tuple. Every result transfers ownership to the consumer. -fn async_result_slots( - ty: &Ty, - object: Option<&TokenStream>, - user: &UserSig<'_>, -) -> syn::Result<(TokenStream, Vec)> { - let pair = |bytes: TokenStream| { - ( - quote!(let (__wv_r0, __wv_r1) = ::weaveffi::abi::bytes_into_raw(#bytes);), - vec![quote!(__wv_r0), quote!(__wv_r1)], - ) - }; - if ty.is_buffered() { - return Ok(pair(quote!(::weaveffi::abi::encode_value(&__wv_val)))); - } - let typed = |call: TokenStream| match object { - Some(obj) => quote!(let __wv_r0: *mut #obj = #call;), - None => quote!(let __wv_r0 = #call;), - }; - let one = vec![quote!(__wv_r0)]; - Ok(match ty { - Ty::Enum(_) => (quote!(let __wv_r0 = __wv_val.__weaveffi_to_i32();), one), - t if is_copy(t) => (quote!(let __wv_r0 = __wv_val;), one), - Ty::StringUtf8 => pair(quote! { - ::std::string::String::into_bytes(::std::convert::Into::into(__wv_val)) - }), - Ty::Bytes => pair(quote!(::std::convert::Into::<::std::vec::Vec>::into( - __wv_val - ))), - Ty::Interface(_) => (typed(quote!(::weaveffi::abi::lower_object(__wv_val))), one), - Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => ( - typed(quote!(::weaveffi::abi::lower_object_opt(__wv_val))), - one, - ), - _ => return Err(unsupported(user.ret_span(), "async return", "result type")), - }) -} /// Generate the completion-callback typedef and the launcher for an /// `async fn`. @@ -245,6 +35,7 @@ pub(crate) fn gen_async_function( prefix: &str, ) -> syn::Result { let is_object_ret = f.ret.as_ref().is_some_and(|t| t.interface_name().is_some()); + let object = user.ret_object(); let cb_ty = ident(&a.callback_type); let cb_slots: Vec = a .callback_params @@ -252,7 +43,7 @@ pub(crate) fn gen_async_function( .map(|p| { // The result slot of an object return is spelled with the // producer's pointee so `super::T` stays in scope. - match (p.name.as_str(), user.ret_object()) { + match (p.name.as_str(), &object) { ("result", Some(obj)) if is_object_ret => quote!(*mut #obj), _ => ctype_to_rust(&p.ty, prefix), } @@ -266,97 +57,53 @@ pub(crate) fn gen_async_function( let launch_sym = ident(&a.launch.symbol); let launch_params = fn_slots(&a.launch.params, &f.params, user, prefix)?; - let sentinels: Vec = a - .callback_params - .iter() - .skip(2) - .filter_map(|p| sentinel(&p.ty)) - .collect(); - - // Lift each input into three parts: a pre-spawn statement on the - // caller's thread (owning borrowed data, retaining objects), an - // in-future statement that finishes the lift (decoding a value buffer), - // and the argument forwarded to the producer. - let mut pre_spawn = TokenStream::new(); - let mut in_future = TokenStream::new(); - let mut call_args: Vec = Vec::new(); - - // An async method retains its receiver for the life of the call: the - // consumer may release its own reference the moment the launcher returns. - if let CallTarget::Method(ty) = target { - let null_self = reject_pre("self is null", &sentinels); - pre_spawn.extend(quote! { - let __wv_obj = match ::weaveffi::abi::object_arc::<#ty>(__wv_self) { - ::std::option::Option::Some(__wv_o) => __wv_o, - ::std::option::Option::None => #null_self - }; - }); - } + // Every input is owned before the future is spawned: the consumer may + // free or reuse its arguments (and release its object references) the + // moment the launcher returns. + let mut lifts = Lifts::default(); + lifts.receiver(target.self_lift(true)); for pb in &f.params { - let (pre, inc, arg) = lift_async_input(pb, user, &sentinels)?; - pre_spawn.extend(pre); - in_future.extend(inc); - call_args.push(arg); + lift_param(&mut lifts, pb, user, true)?; } // A cancellable function takes its own reference on the token (so the // consumer may destroy it at any time) and hands one handle to the // producer, as its final parameter, and one to the runtime, which drops // the future and completes with the cancelled code when it fires. - let cancel = if f.cancellable { - pre_spawn.extend(quote! { - let __wv_cancel = ::weaveffi::abi::CancelToken::from_raw(cancel_token); - let __wv_cancel_arg = ::std::clone::Clone::clone(&__wv_cancel); - }); - call_args.push(quote!(__wv_cancel_arg)); - quote!(::std::option::Option::Some(__wv_cancel)) + let (cancel_pre, cancel) = if f.cancellable { + lifts.args.push(quote!(__wv_cancel_arg)); + ( + quote! { + let __wv_cancel = ::weaveffi::abi::CancelToken::from_raw(cancel_token); + let __wv_cancel_arg = ::std::clone::Clone::clone(&__wv_cancel); + }, + quote!(::std::option::Option::Some(__wv_cancel)), + ) } else { - quote!(::std::option::Option::None) + (TokenStream::new(), quote!(::std::option::Option::None)) }; - - let call = target.call(&f.name, &call_args); + let lifted = lifts.finish(); + let call = target.call(&f.name, &lifts.args); let output = if throws(f) { quote! { - __wv_out.map_err(|__wv_e| ::weaveffi::abi::FfiError::from_report(&__wv_e)) + #call.await.map_err(|__wv_e| ::weaveffi::abi::FfiError::from_report(&__wv_e)) } } else { - quote!(::std::result::Result::Ok(__wv_out)) + quote!(::std::result::Result::Ok(#call.await)) }; - let object = user.ret_object(); - // Lowering can panic (an oversized buffer, say) and must not unwind past - // the exactly-once promise, so it's caught and reported like a producer - // panic. A `void` result has nothing to lower. - let success_arm = match &f.ret { - Some(ty) => { - let (lower, slots) = async_result_slots(ty, object.as_ref(), user)?; - quote! { - ::std::result::Result::Ok(__wv_val) => { - match ::std::panic::catch_unwind(::std::panic::AssertUnwindSafe( - move || { - #lower - (#(#slots,)*) - }, - )) { - ::std::result::Result::Ok((#(#slots,)*)) => { - callback(__wv_ctx, ::std::ptr::null_mut() #(, #slots)*); - return; - } - ::std::result::Result::Err(__wv_panic) => { - ::weaveffi::abi::FfiError::from_panic(&*__wv_panic) - } - } - } - } + let lower = match &f.ret { + Some(_) => { + let lowered = lower_async_result(f.ret.as_ref(), object.as_ref()); + quote!(move |__wv_val| #lowered) } - None => quote! { - ::std::result::Result::Ok(()) => { - callback(__wv_ctx, ::std::ptr::null_mut()); - return; - } - }, + None => quote!(move |()| ()), }; + let slot_count = a.callback_params.len() - 2; + let slots: Vec = (0..slot_count) + .map(|i| ident(&format!("__wv_r{i}"))) + .collect(); let attrs = thunk_attrs(); Ok(quote! { @@ -364,31 +111,18 @@ pub(crate) fn gen_async_function( #attrs pub unsafe extern "C" fn #launch_sym(#(#launch_params),*) { - #[allow(unused_unsafe)] - unsafe { - #pre_spawn - let __wv_ctx = context as usize; - ::weaveffi::abi::run_async( - async move { - #in_future - let __wv_out = #call.await; - #output - }, - #cancel, - move |__wv_res: ::std::result::Result<_, ::weaveffi::abi::FfiError>| { - let __wv_ctx = __wv_ctx as *mut ::std::ffi::c_void; - let __wv_err = match __wv_res { - #success_arm - ::std::result::Result::Err(__wv_e) => __wv_e, - }; - callback( - __wv_ctx, - ::weaveffi::abi::boxed_error(__wv_err) - #(, #sentinels)* - ); - }, - ); - } + let __wv_ctx = context as usize; + ::weaveffi::abi::launch_async( + move || unsafe { + #lifted + #cancel_pre + ::std::result::Result::Ok((async move { #output }, #cancel)) + }, + #lower, + move |__wv_err, (#(#slots,)*)| { + callback(__wv_ctx as *mut ::std::ffi::c_void, __wv_err #(, #slots)*) + }, + ); } }) } diff --git a/crates/weaveffi-macros/src/codegen/callbacks.rs b/crates/weaveffi-macros/src/codegen/callbacks.rs index e00b18e1..19b9745b 100644 --- a/crates/weaveffi-macros/src/codegen/callbacks.rs +++ b/crates/weaveffi-macros/src/codegen/callbacks.rs @@ -7,24 +7,24 @@ //! [`weaveffi_model::plan::CallbackProtocol`]: each method lowers its //! arguments (strings, bytes, and buffers are borrowed for the call as //! `(ptr, len)`; objects transfer one strong reference the consumer adopts), -//! calls the entry with `(ctx, args…, &mut err)`, and then checks `err`. A -//! method declared to return `Result` returns the -//! consumer's failure as an `Err`; one returning a plain `T` raises it -//! through `weaveffi::abi::raise_foreign_error` (an unwind on unwinding -//! builds, a deferred report to the enclosing thunk on `panic = "abort"` -//! builds). +//! calls the entry with `(ctx, args…, [out_ptr, out_len,] &mut err)`, adopts +//! whatever the consumer returned, and then checks `err`. Every method +//! returns `Result`, so a consumer failure is an +//! `Err` and nothing unwinds. use proc_macro2::TokenStream; -use quote::{quote, ToTokens}; +use quote::quote; use syn::spanned::Spanned as _; -use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding, Ty}; +use weaveffi_model::model::{CallbackInterfaceBinding, CallbackMethodBinding, ParamBinding}; +use weaveffi_model::ty::{Family, Ty}; -use super::helpers::{ident, is_copy, ret_arrow, rust_type_ident, slot_type_for, UserSig}; +use super::helpers::{ident, ret_arrow_for, slot_type_for, UserSig}; use super::unsupported; /// Build the lowering for one callback-method argument as `(preamble, C /// arguments)`. The Rust parameter is whatever the producer's trait -/// declares; everything is borrowed for the duration of the vtable call. +/// declares; everything but an object is borrowed for the duration of the +/// vtable call. fn lower_callback_arg( pb: &ParamBinding, user: &UserSig<'_>, @@ -34,68 +34,59 @@ fn lower_callback_arg( let none = TokenStream::new(); // Borrow the value whether it arrived owned or borrowed. let borrowed = if is_ref { quote!(#n) } else { quote!(&#n) }; - let tmp = ident(&format!("__wv_cb_{}", pb.name)); - - // A buffered payload is borrowed for the call: encode it into a local - // value buffer, hand the consumer its `(ptr, len)` view, and let the - // encoding drop afterward. - if pb.ty.is_buffered() { - return Ok(( - quote!(let #tmp = ::weaveffi::abi::encode_value(#borrowed);), - vec![quote!(#tmp.as_ptr()), quote!(#tmp.len())], - )); - } - Ok(match &pb.ty { - Ty::Enum(_) => (none, vec![quote!(#n.__weaveffi_to_i32())]), - ty if is_copy(ty) => { - let v = if is_ref { quote!(*#n) } else { quote!(#n) }; - (none, vec![v]) + let ptr = ident(&format!("__wv_{}_ptr", pb.name)); + let len = ident(&format!("__wv_{}_len", pb.name)); + let pair = |slots: TokenStream| { + ( + quote!(let (#ptr, #len) = #slots;), + vec![quote!(#ptr), quote!(#len)], + ) + }; + Ok(match pb.ty.family() { + Family::Direct => match &pb.ty { + Ty::Enum(_) => ( + none, + vec![quote!(::weaveffi::abi::CEnum::to_i32(#borrowed))], + ), + _ => (none, vec![if is_ref { quote!(*#n) } else { quote!(#n) }]), + }, + Family::String => pair(quote!(::weaveffi::abi::str_slots(&#n))), + Family::Bytes => pair(quote!(::weaveffi::abi::byte_slots(&#n))), + // A buffered argument is encoded into a local value buffer the + // consumer borrows for the call. + Family::Buffer => { + let tmp = ident(&format!("__wv_{}_buf", pb.name)); + let value = if user.param_is_slice(&pb.name) { + quote!(&<[_]>::to_vec(#n)) + } else { + borrowed + }; + ( + quote!(let #tmp = ::weaveffi::abi::encode_value(#value);), + vec![quote!(#tmp.as_ptr()), quote!(#tmp.len())], + ) } - Ty::StringUtf8 => ( - quote!(let #tmp: &str = ::std::convert::AsRef::::as_ref(#borrowed);), - vec![quote!(#tmp.as_ptr()), quote!(#tmp.len())], - ), - Ty::Bytes => ( - quote!(let #tmp: &[u8] = ::std::convert::AsRef::<[u8]>::as_ref(#borrowed);), - vec![quote!(#tmp.as_ptr()), quote!(#tmp.len())], - ), // An object transfers one strong reference the consumer adopts (and // eventually `_destroy`s). The producer must hold it as an `Arc`, // since only an `Arc` allocation can carry a reference count. - Ty::Interface(_) => { + Family::Object { nullable } => { if !user.param_wants_arc(&pb.name) { return Err(unsupported( user.param_span(&pb.name), &pb.name, "callback-interface object parameter that is not an `Arc` (the consumer \ - adopts a reference, so spell it `Arc`)", - )); - } - let obj = user.param_object(&pb.name); - ( - none, - vec![ - quote!(::weaveffi::abi::lower_object::<#obj>(::std::sync::Arc::clone(#borrowed))), - ], - ) - } - Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => { - if !user.param_wants_arc(&pb.name) { - return Err(unsupported( - user.param_span(&pb.name), - &pb.name, - "callback-interface object parameter that is not an `Option>`", + adopts a reference, so spell it `Arc` or `Option>`)", )); } let obj = user.param_object(&pb.name); - ( - none, - vec![ - quote!(::weaveffi::abi::lower_object_opt::<#obj>(::std::clone::Clone::clone(#borrowed))), - ], - ) + let arg = if nullable { + quote!(::weaveffi::abi::lower_object_opt::<#obj>(::std::clone::Clone::clone(#borrowed))) + } else { + quote!(::weaveffi::abi::lower_object::<#obj>(::std::sync::Arc::clone(#borrowed))) + }; + (none, vec![arg]) } - _ => { + Family::Callback { .. } | Family::Iterator => { return Err(unsupported( user.param_span(&pb.name), &pb.name, @@ -105,52 +96,55 @@ fn lower_callback_arg( }) } -/// The lift of a successful vtable return `__wv_ret` into the method's value -/// type, as `(lift, fallback)`. `lift` evaluates to -/// `Result` (an out-of-range enum discriminant is a -/// marshalling failure); `fallback` is the value a plain-`T` method returns -/// after raising a failure on a `panic = "abort"` build. Callback returns are -/// direct-family only. -fn lift_callback_ret( - ret: Option<&Ty>, - user: &UserSig<'_>, -) -> syn::Result<(TokenStream, TokenStream)> { - Ok(match ret { - None => (quote!(::std::result::Result::Ok(())), TokenStream::new()), - Some(Ty::Enum(name)) => { - let et = user - .ret_object() - .unwrap_or_else(|| rust_type_ident(name).into_token_stream()); - ( - quote! { - <#et>::__weaveffi_from_i32(__wv_ret).ok_or_else(|| { - ::weaveffi::abi::ForeignError { - code: ::weaveffi::abi::MARSHAL_ERROR_CODE, - message: ::std::string::String::from( - "callback interface returned an invalid enum discriminant", - ), - } - }) - }, - quote!(<#et>::__weaveffi_placeholder()), - ) +/// The statements adopting the vtable entry's return (`__wv_ret`, or the +/// `out_ptr`/`out_len` run) as `__wv_value: Result`. +fn adopt_callback_ret(ret: Option<&Ty>, user: &UserSig<'_>) -> TokenStream { + let Some(ty) = ret else { + return quote!(let __wv_value = ::std::result::Result::Ok(__wv_ret);); + }; + let value = match ty.family() { + Family::Direct => match ty { + Ty::Enum(_) => { + let et = user.ret_object().unwrap_or_else(|| quote!(_)); + quote!(::weaveffi::abi::callback_ret_enum::<#et>(__wv_ret)) + } + _ => quote!(::std::result::Result::Ok(__wv_ret)), + }, + Family::Object { nullable: false } => { + quote!(unsafe { ::weaveffi::abi::callback_ret_object(__wv_ret) }) } - Some(ty) if is_copy(ty) => ( - quote!(::std::result::Result::Ok(__wv_ret)), - quote!(__wv_ret), - ), - Some(_) => { - return Err(unsupported( - user.ret_span(), - "callback return", - "non-direct return type", - )) + Family::Object { nullable: true } => { + quote!(::std::result::Result::Ok(unsafe { + ::weaveffi::abi::callback_ret_object_opt(__wv_ret) + })) } - }) + Family::String => { + quote!(unsafe { ::weaveffi::abi::callback_ret_string(__wv_out_ptr, __wv_out_len) }) + } + Family::Bytes => { + quote!(unsafe { ::weaveffi::abi::callback_ret_bytes(__wv_out_ptr, __wv_out_len) }) + } + Family::Buffer => { + let ty = user.ret_value_type().unwrap_or_else(|| quote!(_)); + quote!(unsafe { + ::weaveffi::abi::callback_ret_buffer::<#ty>(__wv_out_ptr, __wv_out_len) + }) + } + Family::Callback { .. } | Family::Iterator => { + unreachable!("validation never admits {ty} as a callback return") + } + }; + quote!(let __wv_value = #value;) } -/// Emit one method of the foreign wrapper's trait impl. -fn gen_foreign_method(m: &CallbackMethodBinding, sig: &syn::Signature) -> syn::Result { +/// Emit one method of the foreign wrapper's trait impl. `domain` is the path +/// of the error domain in scope, which a `throws` method's failures are +/// checked against. +fn gen_foreign_method( + m: &CallbackMethodBinding, + sig: &syn::Signature, + domain: Option<&TokenStream>, +) -> syn::Result { let user = UserSig::new(sig, None); let field = ident(&m.name); let mut pre = TokenStream::new(); @@ -160,25 +154,34 @@ fn gen_foreign_method(m: &CallbackMethodBinding, sig: &syn::Signature) -> syn::R pre.extend(p); c_args.extend(args); } - let (lift, fallback) = lift_callback_ret(m.ret.as_ref(), &user)?; - let result = quote! { - ::weaveffi::abi::foreign_status(&__wv_err).and_then(|()| #lift) - }; - let tail = if user.returns_result() { - result - } else { - quote! { - match #result { - ::std::result::Result::Ok(__wv_v) => __wv_v, - ::std::result::Result::Err(__wv_e) => { - ::weaveffi::abi::raise_foreign_error(__wv_e); - #fallback - } - } + let run = m + .ret + .as_ref() + .is_some_and(|t| matches!(t.family(), Family::String | Family::Bytes | Family::Buffer)); + if run { + pre.extend(quote! { + let mut __wv_out_ptr: *mut u8 = ::std::ptr::null_mut(); + let mut __wv_out_len: usize = 0; + }); + c_args.push(quote!(&mut __wv_out_ptr)); + c_args.push(quote!(&mut __wv_out_len)); + } + let adopt = adopt_callback_ret(m.ret.as_ref(), &user); + let status = match (m.throws, domain) { + (true, Some(domain)) => { + quote!(::weaveffi::abi::callback_status_in::<#domain>(&__wv_err)?;) + } + (true, None) => { + return Err(syn::Error::new( + sig.span(), + "weaveffi: a #[weaveffi::throws] callback method needs a #[weaveffi::error] \ + domain in this module or a parent module", + )) } + (false, _) => quote!(::weaveffi::abi::callback_status(&__wv_err)?;), }; Ok(quote! { - #[allow(unsafe_code, unused_variables, clippy::let_unit_value, clippy::unit_arg)] + #[allow(unsafe_code, unused_variables, unused_mut, clippy::let_unit_value, clippy::unit_arg)] #sig { #pre let mut __wv_err = ::weaveffi::abi::FfiError::default(); @@ -187,18 +190,23 @@ fn gen_foreign_method(m: &CallbackMethodBinding, sig: &syn::Signature) -> syn::R let __wv_ret = unsafe { (self.0.vtable().#field)(self.0.ctx(), #(#c_args,)* &mut __wv_err) }; - #tail + // The consumer's return is adopted (and released on failure) + // before its status is checked. + #adopt + #status + __wv_value } }) } /// Emit everything a callback interface needs on the producer side. /// -/// * `{vtable_tag}`: the `#[repr(C)]` vtable struct, one `unsafe extern "C"` -/// function pointer per method in declaration order plus the trailing -/// `free`, exactly as the generated header declares it. +/// * `{vtable_tag}`: the `#[repr(C)]` vtable struct, the runtime's +/// `VtableHeader` (`size`, `flags`, `free`) followed by one +/// `unsafe extern "C"` function pointer per method in declaration order, +/// exactly as the generated header declares it. /// * `__WeaveffiForeign_{Trait}`: a newtype over -/// [`ForeignCallback`](weaveffi_abi::ForeignCallback) implementing the +/// [`ForeignCallback`](weaveffi::abi::ForeignCallback) implementing the /// producer's trait by forwarding each call through the vtable. /// * `impl CallbackInterface for dyn Trait`, which is how a thunk elsewhere /// in the crate (possibly a nested module) names the vtable type and lifts @@ -206,10 +214,11 @@ fn gen_foreign_method(m: &CallbackMethodBinding, sig: &syn::Signature) -> syn::R pub(crate) fn gen_callback_interface( cb: &CallbackInterfaceBinding, item: &syn::ItemTrait, + domain: Option<&TokenStream>, prefix: &str, ) -> syn::Result { let vt = ident(&cb.vtable_tag); - let trait_ident = ident(&cb.name); + let trait_ident = &item.ident; let foreign = ident(&format!("__WeaveffiForeign_{}", cb.name)); let mut fields: Vec = Vec::new(); @@ -238,9 +247,9 @@ pub(crate) fn gen_callback_interface( .iter() .map(|p| slot_type_for(p, &m.params, &user, prefix)) .collect::>>()?; - let arrow = ret_arrow(&m.abi_ret, prefix); + let arrow = ret_arrow_for(&m.abi_ret, m.ret.as_ref(), &user, prefix); fields.push(quote!(pub #field: unsafe extern "C" fn(#(#slots),*) #arrow,)); - methods.push(gen_foreign_method(m, sig)?); + methods.push(gen_foreign_method(m, sig, domain)?); } let vtable_doc = format!( @@ -253,15 +262,13 @@ pub(crate) fn gen_callback_interface( #[repr(C)] #[allow(non_camel_case_types, non_snake_case)] pub struct #vt { + pub header: ::weaveffi::abi::VtableHeader, #(#fields)* - pub free: unsafe extern "C" fn(ctx: *mut ::std::ffi::c_void), } - impl ::weaveffi::abi::Vtable for #vt { - fn free(&self) -> unsafe extern "C" fn(*mut ::std::ffi::c_void) { - self.free - } - } + // SAFETY: `#[repr(C)]` with the header as its first field. + #[allow(unsafe_code)] + unsafe impl ::weaveffi::abi::Vtable for #vt {} #[doc(hidden)] #[allow(non_camel_case_types)] diff --git a/crates/weaveffi-macros/src/codegen/contract.rs b/crates/weaveffi-macros/src/codegen/contract.rs new file mode 100644 index 00000000..96f4ad18 --- /dev/null +++ b/crates/weaveffi-macros/src/codegen/contract.rs @@ -0,0 +1,63 @@ +//! The `{prefix}_{module}_contract` export of a top-level module. +//! +//! The entries come from [`weaveffi_model::contract::entries`] over the +//! validated model, exactly as the CLI computes them for the bindings. Each +//! entry carries the `#[cfg]` of its declaration and of every module and +//! interface around it; the table is compacted in constant context to the +//! entries whose `cfg!` holds, so it lists exactly what this build exports. + +use proc_macro2::TokenStream; +use quote::quote; +use weaveffi_model::model::{contract_symbol, Model, ModuleBinding}; +use weaveffi_model::rust::SourceMap; + +use super::helpers::ident; + +/// The `cfg` predicates (the `p` of `#[cfg(p)]`) that apply to the +/// declaration at `path`: those on it and on everything enclosing it below +/// the root module (whose own `#[cfg]` removes the contract function too). +pub(crate) fn cfg_predicates(source: &SourceMap, path: &[String]) -> Vec { + (2..=path.len()) + .flat_map(|n| source.cfg(&path[..n])) + .filter_map(|attr| match &attr.meta { + syn::Meta::List(list) => Some(list.tokens.clone()), + _ => None, + }) + .collect() +} + +/// Generate the contract function of the top-level module `root`. +pub(crate) fn gen_contract( + model: &Model, + root: &ModuleBinding, + source: &SourceMap, + prefix: &str, +) -> TokenStream { + let sym = ident(&contract_symbol(prefix, &root.name)); + let entries: Vec = weaveffi_model::contract::entries(model, root) + .into_iter() + .map(|e| { + let path: Vec = e.path.split('.').map(str::to_string).collect(); + let preds = cfg_predicates(source, &path); + let (id, hash) = (e.id, e.hash); + quote! { + (::std::cfg!(all(#(#preds),*)), ::weaveffi::abi::ContractEntry::new(#id, #hash)) + } + }) + .collect(); + let n = entries.len(); + quote! { + /// The contract table of this module tree, which generated bindings + /// check against the entries they were generated with. + #[doc(hidden)] + #[unsafe(no_mangle)] + #[allow(unsafe_code, clippy::missing_safety_doc)] + pub unsafe extern "C" fn #sym(out_len: *mut usize) -> *const ::weaveffi::abi::ContractEntry { + const __WV_ALL: [(bool, ::weaveffi::abi::ContractEntry); #n] = [#(#entries),*]; + const __WV_LEN: usize = ::weaveffi::abi::contract_len(&__WV_ALL); + static __WV_TABLE: [::weaveffi::abi::ContractEntry; __WV_LEN] = + ::weaveffi::abi::contract_compact(__WV_ALL); + unsafe { ::weaveffi::abi::contract_table(&__WV_TABLE, out_len) } + } + } +} diff --git a/crates/weaveffi-macros/src/codegen/diagnostics.rs b/crates/weaveffi-macros/src/codegen/diagnostics.rs new file mode 100644 index 00000000..1cc0d69f --- /dev/null +++ b/crates/weaveffi-macros/src/codegen/diagnostics.rs @@ -0,0 +1,144 @@ +//! Turn validation errors into compile errors on the offending Rust item. +//! +//! The macro validates its module tree with the same rules the CLI applies +//! to an IDL. Each error comes with its declaration path (module segments, +//! then declaration, member, and parameter or field names), and the +//! extraction's [`SourceMap`] records the span of every such path, so the +//! error lands on the item, member, or written type at fault. + +use proc_macro2::Span; +use weaveffi_model::rust::SourceMap; +use weaveffi_model::validate::{Found, ValidationError}; + +/// What to look for under an error's declaration path. +struct Target<'a> { + /// A member named by the error, appended to the path. + member: Option<&'a str>, + /// The error is about a written type: prefer the type's span. + typed: bool, + /// The error is about a return type. + returned: bool, + /// The error is about a duplicate: point at the last declaration. + last: bool, +} + +fn target(error: &ValidationError) -> Target<'_> { + use ValidationError as E; + let mut t = Target { + member: None, + typed: false, + returned: false, + last: false, + }; + match error { + E::DuplicateParamName { param: name, .. } + | E::DuplicateStructField { field: name, .. } + | E::DuplicateEnumVariant { variant: name, .. } + | E::DuplicateEnumVariantField { field: name, .. } + | E::DuplicateInterfaceMember { name, .. } + | E::DuplicateCallbackMethod { name, .. } + | E::DuplicateTypeName { name, .. } + | E::DuplicateFunctionName { name, .. } + | E::DuplicateErrorCodeName { name, .. } => { + t.member = Some(name); + t.last = true; + } + E::InvalidIdentifier { name, .. } + | E::ReservedKeyword { name } + | E::EmptyStruct { name, .. } + | E::EmptyEnum { name, .. } + | E::EmptyInterface { name, .. } + | E::EmptyCallbackInterface { name, .. } + | E::NameCollisionWithErrorDomain { name, .. } + | E::InvalidErrorCode { name, .. } => t.member = Some(name), + E::DuplicateEnumValue { enum_name, .. } => t.member = Some(enum_name), + E::ConstructorHasReturn { constructor, .. } | E::AsyncConstructor { constructor, .. } => { + t.member = Some(constructor); + } + E::InvalidCallbackMethod { method, reason, .. } => { + t.member = Some(method); + t.returned = reason.contains("return"); + } + E::AsyncIteratorReturn { .. } => t.returned = true, + E::UnknownTypeRef { .. } + | E::QualifiedTypeRef { .. } + | E::UnsupportedPrimitive { .. } + | E::InvalidMapKey { .. } + | E::InterfaceInInvalidPosition { .. } + | E::CallbackInterfaceInInvalidPosition { .. } + | E::IteratorInInvalidPosition { .. } => t.typed = true, + _ => {} + } + t +} + +/// The span `error`, reported under the declaration path `scope`, belongs +/// to: the most specific recorded path, falling back to enclosing ones. +fn locate(error: &ValidationError, scope: &[String], source: &SourceMap) -> Option { + let t = target(error); + let mut base = scope.to_vec(); + if let Some(member) = t.member { + let with = [scope, &[member.to_string()]].concat(); + if !source.spans(&with).is_empty() { + base = with; + } + } + let mut candidates = Vec::new(); + if t.typed { + candidates.push([&base[..], &[SourceMap::TYPE.to_string()]].concat()); + } + if t.typed || t.returned { + candidates.push([&base[..], &[SourceMap::RETURN.to_string()]].concat()); + } + for n in (1..=base.len()).rev() { + candidates.push(base[..n].to_vec()); + } + candidates.iter().find_map(|path| { + let spans = source.spans(path); + if t.last { + spans.last().copied() + } else { + spans.first().copied() + } + }) +} + +/// The fix for the errors a Rust producer most often meets, phrased in Rust +/// terms (the IDL help text speaks of `errors:` blocks and IDL spellings). +fn hint(error: &ValidationError) -> Option<&'static str> { + use ValidationError as E; + Some(match error { + E::UnsupportedPrimitive { .. } => { + "use `u64` or `i64` for sizes and counts (their width is the same on every \ + platform), or `String` for a `char`" + } + E::ThrowsWithoutErrorDomain { .. } => { + "declare a #[weaveffi::error] enum in this module or a parent module" + } + _ => return None, + }) +} + +/// One compile error per validation error, each at its item's span (or at +/// `fallback`, the module's name, when nothing more specific is known). +pub(crate) fn validation_errors( + found: Vec, + source: &SourceMap, + fallback: Span, +) -> syn::Error { + let mut errors = found.into_iter().map(|(error, scope)| { + let span = locate(&error, &scope, source).unwrap_or(fallback); + let message = match hint(&error) { + Some(hint) => format!("weaveffi: {error}; {hint}"), + None => format!("weaveffi: {error}"), + }; + syn::Error::new(span, message) + }); + let mut first = errors + .next() + .unwrap_or_else(|| syn::Error::new(fallback, "weaveffi: invalid module")); + for e in errors { + first.combine(e); + } + first +} diff --git a/crates/weaveffi-macros/src/codegen/enums.rs b/crates/weaveffi-macros/src/codegen/enums.rs index 223d001c..04a9d333 100644 --- a/crates/weaveffi-macros/src/codegen/enums.rs +++ b/crates/weaveffi-macros/src/codegen/enums.rs @@ -1,5 +1,6 @@ -//! Codegen for enums: the private `i32 -> enum` conversion plus the -//! [`weaveffi_abi::BufferValue`] implementation. +//! Codegen for enums: the [`weaveffi::abi::CEnum`] implementation of a +//! C-style enum, and the [`weaveffi::abi::BufferValue`] implementation of +//! every enum. //! //! A C-style enum crosses the ABI by value as an `i32`, and inside a value //! buffer as the same four discriminant bytes. A rich (algebraic) enum is a @@ -11,100 +12,63 @@ use proc_macro2::TokenStream; use quote::quote; use weaveffi_model::model::EnumBinding; -use super::helpers::{ident, rust_type_ident}; -use super::marshal::{field_read_expr, field_write_stmt_ref}; +use super::helpers::ident; -/// Generate the surface for one enum: `__weaveffi_from_i32` / -/// `__weaveffi_to_i32` plus a `BufferValue` impl for a C-style enum, or the -/// tag-and-fields `BufferValue` impl for a rich (algebraic) enum. -/// -/// `__weaveffi_to_i32` takes `&self` so a thunk can lower an enum it holds by -/// value or by reference without requiring `Copy` or `Clone`. -pub(crate) fn gen_enum(e: &EnumBinding, item: &syn::ItemEnum) -> syn::Result { +/// Generate the surface for one enum: `CEnum` plus a `BufferValue` impl for +/// a C-style enum, or the tag-and-fields `BufferValue` impl for a rich +/// (algebraic) enum. +pub(crate) fn gen_enum(e: &EnumBinding) -> TokenStream { if e.is_rich() { - return gen_rich_enum(e, item); + return gen_rich_enum(e); } - let ty = rust_type_ident(&e.name); - let arms = e.variants.iter().map(|v| { + let ty = ident(&e.name); + let from_arms = e.variants.iter().map(|v| { let value = v.value; let vident = ident(&v.name); quote!(#value => ::std::option::Option::Some(Self::#vident),) }); - let write_arms = e.variants.iter().map(|v| { + let to_arms = e.variants.iter().map(|v| { let value = v.value; let vident = ident(&v.name); quote!(Self::#vident => #value,) }); - let first = e.variants.first().map(|v| ident(&v.name)).ok_or_else(|| { - syn::Error::new_spanned( - &item.ident, - format!("weaveffi: enum `{}` must have at least one variant", e.name), - ) - })?; - Ok(quote! { - #[allow(dead_code)] - impl #ty { - #[doc(hidden)] - pub fn __weaveffi_from_i32(__v: i32) -> ::std::option::Option { - match __v { - #(#arms)* + quote! { + impl ::weaveffi::abi::CEnum for #ty { + fn from_i32(value: i32) -> ::std::option::Option { + match value { + #(#from_arms)* _ => ::std::option::Option::None, } } - - /// The value a callback-interface method yields after its - /// consumer implementation has already been reported as failed - /// (see `weaveffi::abi::raise_foreign_error`); never observed by - /// a caller because the thunk discards the producer's result. - #[doc(hidden)] - pub fn __weaveffi_placeholder() -> Self { - Self::#first - } - - #[doc(hidden)] - pub fn __weaveffi_to_i32(&self) -> i32 { + fn to_i32(&self) -> i32 { match self { - #(#write_arms)* + #(#to_arms)* } } } + #[allow(unsafe_code, unused_unsafe)] impl ::weaveffi::abi::BufferValue for #ty { fn encoded_len(&self) -> usize { 4 } fn write_value(&self, __wv_w: &mut ::weaveffi::abi::BufferWriter) { - __wv_w.write_i32(self.__weaveffi_to_i32()); + ::weaveffi::abi::write_enum(self, __wv_w); } - fn read_value( + unsafe fn read_value( __wv_r: &mut ::weaveffi::abi::BufferReader<'_>, ) -> ::std::result::Result { - let __wv_v = __wv_r.read_i32()?; - Self::__weaveffi_from_i32(__wv_v).ok_or(::weaveffi::abi::BufferDecodeError { - context: "enum discriminant out of range", - }) + ::weaveffi::abi::read_enum(__wv_r) } } - }) + } } /// Generate the `BufferValue` impl for a rich (algebraic) enum: the write /// side emits the active variant's tag then its fields in declaration order; /// the read side dispatches on the tag and reconstructs the variant. -fn gen_rich_enum(e: &EnumBinding, item: &syn::ItemEnum) -> syn::Result { - let ty = rust_type_ident(&e.name); - - // Reject the variant shapes the codegen can't construct: only unit and - // named-field (struct) variants are supported. - for v in &item.variants { - if matches!(v.fields, syn::Fields::Unnamed(_)) { - return Err(syn::Error::new_spanned( - v, - "weaveffi: tuple-style rich-enum variants are not supported; use named fields", - )); - } - } - +fn gen_rich_enum(e: &EnumBinding) -> TokenStream { + let ty = ident(&e.name); let len_arms = e.variants.iter().map(|v| { let vident = ident(&v.name); if v.fields.is_empty() { @@ -118,49 +82,39 @@ fn gen_rich_enum(e: &EnumBinding, item: &syn::ItemEnum) -> syn::Result { __wv_w.write_i32(#value); }) + let bindings: Vec = v.fields.iter().map(|f| ident(&f.name)).collect(); + let pattern = if bindings.is_empty() { + quote!(Self::#vident) } else { - let bindings: Vec = v.fields.iter().map(|f| ident(&f.name)).collect(); - let writes: Vec = v - .fields - .iter() - .zip(&bindings) - .map(|(f, b)| field_write_stmt_ref(f, b)) - .collect(); - quote! { - Self::#vident { #(#bindings),* } => { - __wv_w.write_i32(#value); - #(#writes)* - } + quote!(Self::#vident { #(#bindings),* }) + }; + quote! { + #pattern => { + __wv_w.write_i32(#value); + #(::weaveffi::abi::BufferValue::write_value(#bindings, __wv_w);)* } } }); - let read_arms = e.variants.iter().map(|v| { let value = v.value; let vident = ident(&v.name); if v.fields.is_empty() { quote!(#value => Self::#vident,) } else { - let inits: Vec = v - .fields - .iter() - .map(|f| { - let fname = ident(&f.name); - let read = field_read_expr(f); - quote!(#fname: #read?) - }) - .collect(); - quote!(#value => Self::#vident { #(#inits),* },) + let names: Vec = v.fields.iter().map(|f| ident(&f.name)).collect(); + quote! { + #value => Self::#vident { + #(#names: ::weaveffi::abi::BufferValue::read_value(__wv_r)?),* + }, + } } }); - Ok(quote! { + quote! { + #[allow(unsafe_code, unused_unsafe)] impl ::weaveffi::abi::BufferValue for #ty { fn encoded_len(&self) -> usize { match self { @@ -172,21 +126,26 @@ fn gen_rich_enum(e: &EnumBinding, item: &syn::ItemEnum) -> syn::Result, ) -> ::std::result::Result { let __wv_tag = __wv_r.read_i32()?; - ::std::result::Result::Ok(match __wv_tag { - #(#read_arms)* - _ => { - return ::std::result::Result::Err(::weaveffi::abi::BufferDecodeError { - context: "rich enum tag out of range", - }) + // SAFETY: forwarded from the caller. + ::std::result::Result::Ok(unsafe { + match __wv_tag { + #(#read_arms)* + _ => { + return ::std::result::Result::Err( + ::weaveffi::abi::BufferDecodeError { + context: "rich enum tag out of range", + }, + ) + } } }) } } impl ::weaveffi::abi::ByValue for #ty {} - }) + } } diff --git a/crates/weaveffi-macros/src/codegen/helpers.rs b/crates/weaveffi-macros/src/codegen/helpers.rs index f10a53ea..0f5f5e8b 100644 --- a/crates/weaveffi-macros/src/codegen/helpers.rs +++ b/crates/weaveffi-macros/src/codegen/helpers.rs @@ -1,13 +1,13 @@ //! Shared rendering helpers: identifiers, C-type spelling, ABI slot lists, -//! sentinels, call targets, the producer-signature reader, and the -//! panic-catching thunk wrapper. +//! call targets, the producer-signature reader, and `#[cfg]` wrapping. use proc_macro2::{Span, TokenStream}; use quote::{quote, ToTokens}; use syn::spanned::Spanned as _; use syn::Ident; use weaveffi_model::abi::{AbiParam, CType, ConstPos}; -use weaveffi_model::model::{ParamBinding, Ty}; +use weaveffi_model::model::ParamBinding; +use weaveffi_model::ty::Ty; /// Make a call-site identifier from a string. pub(crate) fn ident(name: &str) -> Ident { @@ -81,80 +81,36 @@ pub(crate) fn ret_arrow(ret: &CType, prefix: &str) -> TokenStream { } } -/// The statement that leaves a thunk early: `return sentinel;`, or a bare -/// `return;` for a `void` symbol (`sentinel` is `None`). -pub(crate) fn early_return(sentinel: Option<&TokenStream>) -> TokenStream { - match sentinel { - Some(s) => quote!(return #s;), - None => quote!(return;), - } -} - -/// The zero/null value a fallible symbol returns on the error path, or -/// `None` for a `void` symbol. -pub(crate) fn sentinel(ret: &CType) -> Option { - Some(match ret { - CType::Void => return None, - CType::Ptr { - konst: ConstPos::None, - .. - } => quote!(::std::ptr::null_mut()), - CType::Ptr { .. } => quote!(::std::ptr::null()), - CType::Bool => quote!(false), - CType::Float | CType::Double => quote!(0.0), - _ => quote!(0), - }) -} - -/// The statement every thunk runs right after the producer's code returns: -/// if a consumer callback failure was deferred during the call (a -/// `panic = "abort"` build), report it through `out_err` and return -/// `sentinel`, dropping the producer's result unlowered. -pub(crate) fn deferred_failure_check(sentinel: Option<&TokenStream>) -> TokenStream { - let ret = early_return(sentinel); - quote! { - if let ::std::option::Option::Some(__wv_f) = ::weaveffi::abi::take_foreign_error() { - unsafe { ::weaveffi::abi::error_set(out_err, __wv_f.code, &__wv_f.message) }; - #ret - } +/// The C return type a thunk's body evaluates to (`()` for `void`). +pub(crate) fn ret_type(ret: &CType, prefix: &str) -> TokenStream { + if matches!(ret, CType::Void) { + quote!(()) + } else { + ctype_to_rust(ret, prefix) } } -/// The `error_set(out_err, MARSHAL_ERROR_CODE, msg); return sentinel;` tail a -/// synchronous thunk uses to reject an invalid input. -pub(crate) fn reject(msg: &str, sentinel: Option<&TokenStream>) -> TokenStream { - let ret = early_return(sentinel); - quote! { - unsafe { - ::weaveffi::abi::error_set(out_err, ::weaveffi::abi::MARSHAL_ERROR_CODE, #msg) - }; - #ret +/// Put the `#[cfg]` attributes of the declaration generated items belong +/// to on each of them, so they exist exactly when it does. Items without a +/// `#[cfg]` pass through unchanged. +pub(crate) fn cfg_wrap(cfgs: &[&syn::Attribute], items: TokenStream) -> TokenStream { + if cfgs.is_empty() || items.is_empty() { + return items; + } + match syn::parse2::(items.clone()) { + Ok(file) => file + .items + .iter() + .map(|item| quote!(#(#cfgs)* #item)) + .collect(), + // Unreachable for well-formed output; let the compiler report it. + Err(_) => items, } } -/// True when this type crosses the ABI by value without owning heap data. -pub(crate) fn is_copy(ty: &Ty) -> bool { - matches!( - ty, - Ty::I8 - | Ty::I16 - | Ty::I32 - | Ty::I64 - | Ty::U8 - | Ty::U16 - | Ty::U32 - | Ty::U64 - | Ty::F32 - | Ty::F64 - | Ty::Bool - | Ty::Enum(_) - ) -} - -/// The bare Rust type name of a struct/enum reference (dropping any qualifying -/// module path the resolver added). -pub(crate) fn rust_type_ident(name: &str) -> Ident { - ident(name.rsplit('.').next().unwrap_or(name)) +/// A string literal naming a parameter, for marshalling error messages. +pub(crate) fn name_lit(name: &str) -> syn::LitStr { + syn::LitStr::new(name, Span::call_site()) } // ── the producer's own signature ───────────────────────────────────────── @@ -162,7 +118,7 @@ pub(crate) fn rust_type_ident(name: &str) -> Ident { /// A view of the producer's written signature, used wherever the thunk must /// spell a type the way the producer did. /// -/// The binding model knows every type *semantically* (`Ty::Interface("Store")`) +/// The model knows every type *semantically* (`Ty::Interface("Store")`) /// but not how the producer spelled it (`&Store`, `Arc`, /// `Option>`). Thunks are emitted inside the producer's /// module, so reusing the written path keeps parent-module types in scope, and @@ -243,10 +199,49 @@ impl<'a> UserSig<'a> { } } - /// Whether the method returns `Result` (a callback - /// method that receives the consumer's failure as a value). - pub(crate) fn returns_result(&self) -> bool { - weaveffi_model::rust::output_is_result(&self.sig.output) + /// The span of the parameter's written type (or the function name), the + /// place a type error about it should point. + pub(crate) fn param_type_span(&self, name: &str) -> Span { + self.param_type(name) + .map_or_else(|| self.sig.ident.span(), syn::spanned::Spanned::span) + } + + /// The producer's spelling of a by-value parameter type with any `&` + /// removed (`&Contact` is `Contact`, `&[Item]` is `Vec`), for a + /// lift that decodes an owned value. + pub(crate) fn param_owned(&self, name: &str) -> Option { + let ty = self.param_type(name)?; + Some(match ty { + syn::Type::Reference(r) => match r.elem.as_ref() { + syn::Type::Slice(slice) => { + let elem = self.spell(&slice.elem); + quote!(::std::vec::Vec<#elem>) + } + other => self.spell(other), + }, + other => self.spell(other), + }) + } + + /// Whether the parameter is written as a slice reference (`&[T]`). + pub(crate) fn param_is_slice(&self, name: &str) -> bool { + matches!( + self.param_type(name), + Some(syn::Type::Reference(r)) if matches!(r.elem.as_ref(), syn::Type::Slice(_)) + ) + } + + /// The producer's return type with `Result` peeled, spelled for a thunk. + pub(crate) fn ret_value_type(&self) -> Option { + self.ret_syn().map(|t| self.spell(t)) + } + + /// The span of the written return type (or the function name). + pub(crate) fn ret_type_span(&self) -> Span { + match &self.sig.output { + syn::ReturnType::Type(_, ty) => syn::spanned::Spanned::span(ty), + syn::ReturnType::Default => self.sig.ident.span(), + } } /// Whether the parameter's type (under any `&` and `Option`) is an @@ -522,6 +517,22 @@ pub(crate) fn ret_arrow_for( ret_arrow(ret, prefix) } +/// The C return type a thunk's body evaluates to (`()` for `void`), +/// spelling an object return with the producer's own type. +pub(crate) fn ret_type_for( + ret: &CType, + ret_ty: Option<&Ty>, + user: &UserSig<'_>, + prefix: &str, +) -> TokenStream { + if ret_ty.is_some_and(|t| t.interface_name().is_some()) { + if let Some(obj) = user.ret_object() { + return quote!(*mut #obj); + } + } + ret_type(ret, prefix) +} + /// The attributes every exported thunk carries. Thunks are `unsafe extern /// "C"` (they trust the caller's pointers), hidden from the producer's docs, /// and exempt from the lints generated code can't satisfy. @@ -575,70 +586,18 @@ impl CallTarget { } } - /// The receiver-lift preamble for a synchronous method: null-check, report - /// through `out_err`, and bind `__wv_obj` as a borrow (`&self`) or a - /// retained reference (`self: Arc`). Empty for free functions and - /// statics. - pub(crate) fn self_preamble( - &self, - sentinel: Option<&TokenStream>, - as_arc: bool, - ) -> TokenStream { + /// The receiver lift for a method (`None` for free functions and + /// statics): a borrow for `&self`, or a retained reference for + /// `self: Arc` and for every async method (`retain`), whose + /// future outlives the call. + pub(crate) fn self_lift(&self, retain: bool) -> Option { let CallTarget::Method(ty) = self else { - return TokenStream::new(); + return None; }; - let fail = reject("self is null", sentinel); - let lift = if as_arc { - quote!(unsafe { ::weaveffi::abi::object_arc::<#ty>(__wv_self) }) + Some(if retain { + quote!(::weaveffi::abi::lift_self_arc::<#ty>(__wv_self)) } else { - quote!(unsafe { ::weaveffi::abi::object_ref::<#ty>(__wv_self) }) - }; - quote! { - let __wv_obj = match #lift { - ::std::option::Option::Some(__wv_o) => __wv_o, - ::std::option::Option::None => { #fail } - }; - } - } -} - -/// Wrap a thunk body in `catch_unwind` so a producer panic is reported through -/// `out_err` (with the reserved panic code, or the foreign code when the -/// payload is a consumer callback's failure) instead of unwinding across the C -/// boundary. On a non-throwing function this is the only way `out_err` can -/// report failure, which consumers interpret per -/// [`weaveffi_model::plan::ErrorStrategy::Trap`]. `sentinel` is the value the -/// thunk returns on the panic path (`None` for a `void` thunk). -/// -/// The body runs inside a [`ThunkScope`](weaveffi_abi::ThunkScope), so on a -/// `panic = "abort"` build a consumer failure deferred during the call is -/// recorded for this thunk (and checked by [`deferred_failure_check`] in the -/// body) rather than for some later, unrelated call. -pub(crate) fn wrap_unwind(body: TokenStream, sentinel: Option<&TokenStream>) -> TokenStream { - let tail = match sentinel { - Some(s) => quote!(#s), - None => TokenStream::new(), - }; - quote! { - let __wv_scope = ::weaveffi::abi::ThunkScope::enter(); - let __wv_out = match ::std::panic::catch_unwind(::std::panic::AssertUnwindSafe(move || { - #[allow(unused_unsafe)] - unsafe { - #body - } - })) { - ::std::result::Result::Ok(__wv_v) => __wv_v, - ::std::result::Result::Err(__wv_panic) => { - unsafe { - ::weaveffi::abi::error_store( - out_err, - ::weaveffi::abi::FfiError::from_panic(&*__wv_panic), - ) - }; - #tail - } - }; - ::std::mem::drop(__wv_scope); - __wv_out + quote!(::weaveffi::abi::lift_self::<#ty>(__wv_self)) + }) } } diff --git a/crates/weaveffi-macros/src/codegen/interfaces.rs b/crates/weaveffi-macros/src/codegen/interfaces.rs index 3da1ce28..27dcc712 100644 --- a/crates/weaveffi-macros/src/codegen/interfaces.rs +++ b/crates/weaveffi-macros/src/codegen/interfaces.rs @@ -4,7 +4,7 @@ //! Interface members (constructors, methods, statics) share the callable //! dispatch in [`super::sync`]; only the object lifecycle is //! interface-specific. An interface object is an `Arc` handed out as a raw -//! pointer (see `weaveffi_abi::object`), so `_clone` bumps the count and +//! pointer (see `weaveffi::abi::object`), so `_clone` bumps the count and //! `_destroy` releases one reference. use proc_macro2::TokenStream; diff --git a/crates/weaveffi-macros/src/codegen/iterators.rs b/crates/weaveffi-macros/src/codegen/iterators.rs index f9f7577c..edda0a7c 100644 --- a/crates/weaveffi-macros/src/codegen/iterators.rs +++ b/crates/weaveffi-macros/src/codegen/iterators.rs @@ -5,27 +5,24 @@ //! [`weaveffi_model::plan::IteratorProtocol`]: each `_next` call yields exactly //! one element the consumer then owns (and releases per the protocol's //! `elem` pass), and `_destroy` releases the handle exactly once. Errors from -//! the launcher and from each `_next` follow the owning function's -//! [`ErrorStrategy`](weaveffi_model::plan::ErrorStrategy). +//! the launcher follow the owning function's +//! [`ErrorStrategy`](weaveffi_model::plan::ErrorStrategy); `_next` fails only +//! on a null handle or a concurrent or re-entrant `_next`. use proc_macro2::TokenStream; use quote::quote; use weaveffi_model::model::{FnBinding, IteratorBinding}; -use super::helpers::{ - deferred_failure_check, fn_slots, ident, reject, slot_tokens, thunk_attrs, wrap_unwind, - CallTarget, UserSig, -}; -use super::marshal::{lift_param, lower_value}; -use super::sync::throws; +use super::helpers::{fn_slots, ident, slot_tokens, thunk_attrs, CallTarget, UserSig}; +use super::lift::lower_ret; +use super::sync::{checked_call, sync_lifts}; use super::unsupported; /// Generate the launcher / `_next` / `_destroy` trio for a function returning /// `iter`. The producer returns a `weaveffi::Iter` (optionally wrapped in /// `Result`); the launcher boxes it behind a -/// [`weaveffi_abi::IterHandle`], `_next` pulls one element under the -/// handle's lock and lowers it through `out_item`, and `_destroy` drops the -/// handle. +/// [`weaveffi::abi::IterHandle`], `_next` pulls one element and lowers it +/// through `out_item`, and `_destroy` drops the handle. pub(crate) fn gen_iterator_function( f: &FnBinding, it: &IteratorBinding, @@ -55,51 +52,18 @@ pub(crate) fn gen_iterator_function( // ── launcher: lift inputs, run the producer's fn, box the iterator ── let launch_sym = ident(&it.launch.symbol); let launch_params = fn_slots(&it.launch.params, &f.params, user, prefix)?; - let null = quote!(::std::ptr::null_mut()); - let launch_sentinel = Some(&null); - - let self_pre = target.self_preamble(launch_sentinel, user.receiver_is_arc()); - let mut preamble = TokenStream::new(); - let mut call_args: Vec = Vec::new(); - for pb in &f.params { - let (pre, arg) = lift_param(pb, user, launch_sentinel)?; - preamble.extend(pre); - call_args.push(arg); - } - let call = target.call(&f.name, &call_args); - let deferred = deferred_failure_check(launch_sentinel); - let bind_iter = if throws(f) { - quote! { - let __wv_iter = match __wv_out { - ::std::result::Result::Ok(__wv_v) => __wv_v, - ::std::result::Result::Err(__wv_err) => { - ::weaveffi::abi::error_store( - out_err, - ::weaveffi::abi::FfiError::from_report(&__wv_err), - ); - return ::std::ptr::null_mut(); - } - }; - } - } else { - quote!(let __wv_iter = __wv_out;) - }; - let launch_body = wrap_unwind( - quote! { - #self_pre - #preamble - let __wv_out = #call; - #deferred - #bind_iter - ::weaveffi::abi::error_clear(out_err); - ::weaveffi::abi::iter_into_raw(__wv_iter) - }, - launch_sentinel, - ); + let lifts = sync_lifts(f, user, target)?; + let lifted = lifts.finish(); + let call = checked_call(f, target.call(&f.name, &lifts.args), user); let launch = quote! { #attrs pub unsafe extern "C" fn #launch_sym(#(#launch_params),*) -> *mut #handle { - #launch_body + unsafe { + ::weaveffi::abi::call_sync(out_err, move || unsafe { + #lifted + ::std::result::Result::Ok(::weaveffi::abi::iter_into_raw(#call)) + }) + } } }; @@ -116,36 +80,27 @@ pub(crate) fn gen_iterator_function( _ => slot_tokens(p, prefix), }) .collect(); - let item_lowered = lower_value(&it.elem, quote!(__wv_item), elem_object.as_ref(), user)?; - let zero = quote!(0); - let next_sentinel = Some(&zero); - let null_fail = reject("iterator or out_item is null", next_sentinel); - let deferred = deferred_failure_check(next_sentinel); - let next_body = wrap_unwind( - quote! { - if out_item.is_null() { - #null_fail - } - let __wv_pulled = match ::weaveffi::abi::iter_next(iter) { - ::std::option::Option::Some(__wv_p) => __wv_p, - ::std::option::Option::None => { #null_fail } - }; - #deferred - ::weaveffi::abi::error_clear(out_err); - match __wv_pulled { - ::std::option::Option::Some(__wv_item) => { - *out_item = #item_lowered; - 1 - } - ::std::option::Option::None => 0, - } - }, - next_sentinel, - ); + let item = lower_ret(&it.elem, "e!(__wv_item), elem_object.as_ref()); let next = quote! { #attrs pub unsafe extern "C" fn #next_sym(iter: *const #handle, #(#rest_params),*) -> i32 { - #next_body + unsafe { + ::weaveffi::abi::call_sync(out_err, move || unsafe { + if out_item.is_null() { + return ::std::result::Result::Err(::weaveffi::abi::FfiError::new( + ::weaveffi::abi::MARSHAL_ERROR_CODE, + "out_item is null", + )); + } + ::std::result::Result::Ok(match ::weaveffi::abi::iter_next(iter)? { + ::std::option::Option::Some(__wv_item) => { + *out_item = #item; + 1 + } + ::std::option::Option::None => 0, + }) + }) + } } }; diff --git a/crates/weaveffi-macros/src/codegen/lift.rs b/crates/weaveffi-macros/src/codegen/lift.rs new file mode 100644 index 00000000..358ca948 --- /dev/null +++ b/crates/weaveffi-macros/src/codegen/lift.rs @@ -0,0 +1,227 @@ +//! Parameter lifting and return lowering: one runtime call per parameter +//! and per return, chosen by the value's family. +//! +//! Every lift evaluates to a `Result` bound to the parameter's +//! name. A thunk lifts **all** of its parameters first and only then unwraps +//! them in order ([`Lifts::finish`]), so when one fails the others, already +//! adopted (a callback context, a buffer's object tokens), are dropped and +//! released rather than leaked. The runtime functions behind each lift live +//! in `weaveffi::abi::marshal`; this module only picks one. + +use proc_macro2::TokenStream; +use quote::{quote, quote_spanned}; +use weaveffi_model::model::ParamBinding; +use weaveffi_model::ty::{Family, Prim, Ty}; + +use super::helpers::{ident, name_lit, UserSig}; +use super::unsupported; + +/// The lifted parameters of one thunk. +#[derive(Default)] +pub(crate) struct Lifts { + /// `let x = ;` for each parameter that needs lifting. + lifts: TokenStream, + /// `let x = x?;` in parameter order. + unwraps: TokenStream, + /// The call arguments, in parameter order. + pub(crate) args: Vec, +} + +impl Lifts { + /// Lift the receiver first (when there is one), so a null `self` is + /// reported only after every other input was adopted. + pub(crate) fn receiver(&mut self, lift: Option) { + if let Some(lift) = lift { + self.lifts.extend(quote!(let __wv_obj = #lift;)); + self.unwraps.extend(quote!(let __wv_obj = __wv_obj?;)); + } + } + + fn push(&mut self, name: &syn::Ident, lift: Option, arg: TokenStream) { + if let Some(lift) = lift { + self.lifts.extend(quote!(let #name = #lift;)); + self.unwraps.extend(quote!(let #name = #name?;)); + } + self.args.push(arg); + } + + /// Every lift followed by every unwrap. + pub(crate) fn finish(&self) -> TokenStream { + let (lifts, unwraps) = (&self.lifts, &self.unwraps); + quote!(#lifts #unwraps) + } +} + +/// Lift one parameter into `lifts`. With `owned`, the value must outlive the +/// call (an async launcher's future): strings and bytes are copied and +/// objects retained even when the producer borrows them, and the argument +/// lends the owned value. +pub(crate) fn lift_param( + lifts: &mut Lifts, + pb: &ParamBinding, + user: &UserSig<'_>, + owned: bool, +) -> syn::Result<()> { + let name = ident(&pb.name); + let lit = name_lit(&pb.name); + let span = user.param_type_span(&pb.name); + let is_ref = user.param_is_ref(&pb.name); + let lent = if is_ref { + quote!(&#name) + } else { + quote!(#name) + }; + let ptr = ident(&format!("{}_ptr", pb.name)); + let len = ident(&format!("{}_len", pb.name)); + + match pb.ty.family() { + Family::Direct => match &pb.ty { + Ty::Enum(_) => { + let et = user.param_object(&pb.name).unwrap_or_else(|| quote!(_)); + let lift = quote_spanned!(span=> ::weaveffi::abi::lift_enum::<#et>(#name, #lit)); + lifts.push(&name, Some(lift), lent); + } + _ => lifts.push(&name, None, lent), + }, + Family::String | Family::Bytes => { + let (elem, borrow, copy) = if matches!(pb.ty, Ty::Prim(Prim::String)) { + ("str", quote!(lift_str_param), quote!(lift_string_param)) + } else { + ("u8", quote!(lift_slice_param), quote!(lift_bytes_param)) + }; + // `&str` and `&[u8]` borrow the caller's bytes for the call + // without copying; any other spelling gets an owned copy, lent + // when written as a reference. + let borrowed = !owned && user.param_is_borrowed(&pb.name, elem); + let f = if borrowed { borrow } else { copy }; + let lift = quote_spanned!(span=> ::weaveffi::abi::#f(#ptr, #len, #lit)); + let arg = if borrowed { quote!(#name) } else { lent }; + lifts.push(&name, Some(lift), arg); + } + Family::Buffer => { + let ty = user.param_owned(&pb.name).unwrap_or_else(|| quote!(_)); + let lift = + quote_spanned!(span=> ::weaveffi::abi::lift_buffer_param::<#ty>(#ptr, #len, #lit)); + let arg = if user.param_is_slice(&pb.name) { + quote!(&#name[..]) + } else { + lent + }; + lifts.push(&name, Some(lift), arg); + } + Family::Object { nullable } => { + let obj = user.param_object(&pb.name).ok_or_else(|| { + unsupported( + user.param_span(&pb.name), + &pb.name, + "interface parameter spelling", + ) + })?; + let wants_arc = user.param_wants_arc(&pb.name); + if !nullable && !wants_arc && !is_ref { + return Err(unsupported( + user.param_span(&pb.name), + &pb.name, + "by-value interface parameter (accept `&T` to borrow the object for the \ + call, or `Arc` to retain it)", + )); + } + let retain = wants_arc || owned; + let f = match (nullable, retain) { + (false, false) => quote!(lift_object_param::<#obj>(#name, #lit)), + (false, true) => quote!(lift_object_arc_param::<#obj>(#name, #lit)), + (true, false) => quote!(lift_object_opt_param::<#obj>(#name)), + (true, true) => quote!(lift_object_arc_opt_param::<#obj>(#name)), + }; + let lift = quote_spanned!(span=> ::weaveffi::abi::#f); + // A borrowed lift already *is* the `&T` (or `Option<&T>`) the + // producer takes; a retained one lends its `Arc` when the + // producer borrows. + let arg = match (nullable, wants_arc, retain) { + (_, true, _) => lent, + (false, false, false) | (true, false, false) => quote!(#name), + (false, false, true) => quote!(&#name), + (true, false, true) => quote!(#name.as_deref()), + }; + lifts.push(&name, Some(lift), arg); + } + Family::Callback { nullable } => { + let dyn_ty = user.param_callback(&pb.name)?; + let ctx = ident(&format!("{}_ctx", pb.name)); + let vtable = ident(&format!("{}_vtable", pb.name)); + let f = if nullable { + quote!(lift_callback_opt_param) + } else { + quote!(lift_callback_param) + }; + let lift = quote_spanned!(span=> ::weaveffi::abi::#f::<#dyn_ty>(#ctx, #vtable, #lit)); + lifts.push(&name, Some(lift), lent); + } + Family::Iterator => { + return Err(unsupported( + user.param_span(&pb.name), + &pb.name, + "parameter type", + )) + } + } + Ok(()) +} + +/// The expression lowering the owned value `value` of type `ty` into a sync +/// return (or an iterator element): the C return value, with a string, +/// bytes, or buffer written as a producer-allocated run whose length goes to +/// `out_len`. `object` is the producer's spelling of an object return's +/// pointee, which pins the `Arc` a `Self` return converts into. +pub(crate) fn lower_ret(ty: &Ty, value: &TokenStream, object: Option<&TokenStream>) -> TokenStream { + let typed = |call: TokenStream| match object { + Some(obj) => quote!({ let __wv_p: *mut #obj = #call; __wv_p }), + None => call, + }; + match ty.family() { + Family::Direct => match ty { + Ty::Enum(_) => quote!(::weaveffi::abi::CEnum::to_i32(&#value)), + _ => value.clone(), + }, + Family::String => quote!(::weaveffi::abi::lower_string_ret(#value, out_len)), + Family::Bytes => quote!(::weaveffi::abi::lower_bytes_ret(#value, out_len)), + Family::Buffer => quote!(::weaveffi::abi::lower_buffer_ret(&#value, out_len)), + Family::Object { nullable: false } => typed(quote!(::weaveffi::abi::lower_object(#value))), + Family::Object { nullable: true } => { + typed(quote!(::weaveffi::abi::lower_object_opt(#value))) + } + Family::Callback { .. } | Family::Iterator => { + unreachable!("validation never admits {ty} as a value return") + } + } +} + +/// The closure body lowering an async result `__wv_val` into the completion +/// callback's result slots, as a tuple (`()` for a `void` function). +pub(crate) fn lower_async_result(ty: Option<&Ty>, object: Option<&TokenStream>) -> TokenStream { + let Some(ty) = ty else { + return quote!(()); + }; + let typed = |call: TokenStream| match object { + Some(obj) => quote!({ let __wv_p: *mut #obj = #call; (__wv_p,) }), + None => quote!((#call,)), + }; + match ty.family() { + Family::Direct => match ty { + Ty::Enum(_) => quote!((::weaveffi::abi::CEnum::to_i32(&__wv_val),)), + _ => quote!((__wv_val,)), + }, + Family::String => quote!(::weaveffi::abi::string_run(__wv_val)), + Family::Bytes => quote!(::weaveffi::abi::bytes_run(__wv_val)), + Family::Buffer => quote!(::weaveffi::abi::buffer_run(&__wv_val)), + Family::Object { nullable: false } => { + typed(quote!(::weaveffi::abi::lower_object(__wv_val))) + } + Family::Object { nullable: true } => { + typed(quote!(::weaveffi::abi::lower_object_opt(__wv_val))) + } + Family::Callback { .. } | Family::Iterator => { + unreachable!("validation never admits {ty} as an async result") + } + } +} diff --git a/crates/weaveffi-macros/src/codegen/marshal.rs b/crates/weaveffi-macros/src/codegen/marshal.rs deleted file mode 100644 index 981ecfb9..00000000 --- a/crates/weaveffi-macros/src/codegen/marshal.rs +++ /dev/null @@ -1,305 +0,0 @@ -//! Parameter lifting and return-value lowering: the marshalling that turns -//! ABI slots into the Rust values a producer function takes, and its results -//! back into C representations. -//! -//! Buffered types (records, rich enums, optionals, lists, maps) arrive as a -//! borrowed `(ptr, len)` value-buffer pair and are decoded through -//! [`weaveffi_abi::decode_value`]; buffered returns are encoded with -//! [`weaveffi_abi::encode_value`] and handed to the consumer as a -//! producer-allocated buffer it frees with `{prefix}_free_bytes`. Interface -//! objects are reference counted: a parameter is borrowed for the call -//! ([`weaveffi_abi::object_ref`]) or retained ([`weaveffi_abi::object_arc`]) -//! depending on whether the producer wrote `&T` or `Arc`, and a return -//! hands the consumer one strong reference ([`weaveffi_abi::lower_object`]). -//! A callback interface arrives as `(ctx, vtable)` and is lifted into the -//! `Arc` the producer takes. The remaining ownership rules here are -//! the producer half of the contract stated by -//! [`weaveffi_model::plan::return_free`]. - -use proc_macro2::TokenStream; -use quote::{quote, ToTokens}; -use weaveffi_model::abi::CType; -use weaveffi_model::model::{FieldBinding, ParamBinding, Ty}; - -use super::helpers::{ - deferred_failure_check, early_return, ident, is_copy, reject, rust_type_ident, sentinel, - UserSig, -}; -use super::unsupported; - -/// The call argument for a lifted value bound to `name`: lent when the -/// producer wrote `&T`, moved otherwise. Deref coercion turns `&String` into -/// `&str`, `&Vec` into `&[u8]`, and `&Arc` into `&T` at the call. -fn arg_for(name: &syn::Ident, is_ref: bool) -> TokenStream { - if is_ref { - quote!(&#name) - } else { - quote!(#name) - } -} - -/// Generate the lift preamble and the call-argument expression for one -/// parameter of a synchronous thunk (or an iterator launcher). The preamble -/// runs inside the thunk's `unsafe` body. -pub(crate) fn lift_param( - pb: &ParamBinding, - user: &UserSig<'_>, - sentinel: Option<&TokenStream>, -) -> syn::Result<(TokenStream, TokenStream)> { - let name = ident(&pb.name); - let is_ref = user.param_is_ref(&pb.name); - let arg = arg_for(&name, is_ref); - let fail = reject(&format!("{} is null or invalid", pb.name), sentinel); - let ptr = ident(&format!("{}_ptr", pb.name)); - let len = ident(&format!("{}_len", pb.name)); - - // A buffered parameter is one `(const uint8_t*, size_t)` pair holding the - // value serialized in the WeaveFFI buffer format. Decode it into the - // owned Rust value the producer's signature names; the concrete type - // (including the map flavor `HashMap`/`BTreeMap`) is inferred from the - // call site. A malformed buffer is a producer/consumer contract - // violation, reported with the reserved marshalling code so it can't - // shadow a domain's typed codes. - if pb.ty.is_buffered() { - let decode_fail = reject(&format!("{}: malformed value buffer", pb.name), sentinel); - let pre = quote! { - let #name = match ::weaveffi::abi::lift_byte_slice(#ptr, #len) { - ::std::option::Option::Some(__wv_buf) => match ::weaveffi::abi::decode_value(__wv_buf) { - ::std::result::Result::Ok(__wv_v) => __wv_v, - ::std::result::Result::Err(_) => { #decode_fail } - }, - ::std::option::Option::None => { #fail } - }; - }; - return Ok((pre, arg)); - } - - Ok(match &pb.ty { - Ty::Enum(enum_name) => { - // Prefer the producer's path (it may be `super::Kind`). - let et = user - .param_object(&pb.name) - .unwrap_or_else(|| rust_type_ident(enum_name).into_token_stream()); - let pre = quote! { - let #name = match <#et>::__weaveffi_from_i32(#name) { - ::std::option::Option::Some(__wv_v) => __wv_v, - ::std::option::Option::None => { #fail } - }; - }; - (pre, arg) - } - ty if is_copy(ty) => (TokenStream::new(), arg), - // `&str` and `&[u8]` borrow the caller's bytes for the call without - // copying; any other spelling (`String`, `&String`, `Vec`) gets - // an owned copy, lent when written as a reference. - Ty::StringUtf8 | Ty::Bytes => { - let (elem, borrow, copy) = if matches!(pb.ty, Ty::StringUtf8) { - ("str", quote!(lift_str), quote!(lift_string)) - } else { - ("u8", quote!(lift_byte_slice), quote!(lift_bytes)) - }; - let borrowed = user.param_is_borrowed(&pb.name, elem); - let lift = if borrowed { borrow } else { copy }; - let pre = quote! { - let #name = match ::weaveffi::abi::#lift(#ptr, #len) { - ::std::option::Option::Some(__wv_v) => __wv_v, - ::std::option::Option::None => { #fail } - }; - }; - (pre, if borrowed { quote!(#name) } else { arg }) - } - // An object parameter is borrowed for the call. The producer's own - // spelling decides how it is lifted: `&T` borrows through the pointer - // without touching the count; `Arc` takes a new strong reference - // so the producer may keep the object. - Ty::Interface(_) => { - let wants_arc = user.param_wants_arc(&pb.name); - if !wants_arc && !is_ref { - return Err(unsupported( - user.param_span(&pb.name), - &pb.name, - "by-value interface parameter (accept `&T` to borrow the object for the \ - call, or `Arc` to retain it)", - )); - } - let lift = if wants_arc { - quote!(::weaveffi::abi::object_arc(#name)) - } else { - quote!(::weaveffi::abi::object_ref(#name)) - }; - let pre = quote! { - let #name = match #lift { - ::std::option::Option::Some(__wv_o) => __wv_o, - ::std::option::Option::None => { #fail } - }; - }; - // A borrowed lift already *is* the `&T` the producer takes. - let arg = if wants_arc { arg } else { quote!(#name) }; - (pre, arg) - } - // `Interface?` is the one optional that isn't buffered: a nullable - // pointer, lifted to `Option<&T>` or `Option>`. - Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => { - let lift = if user.param_wants_arc(&pb.name) { - quote!(::weaveffi::abi::object_arc(#name)) - } else { - quote!(::weaveffi::abi::object_ref(#name)) - }; - (quote!(let #name = #lift;), arg) - } - // A callback interface is `(ctx, vtable)`; a null vtable is a contract - // violation. The `dyn Trait` comes from the producer's `Arc` - // spelling and resolves the vtable type through `CallbackInterface`. - Ty::CallbackInterface(_) => { - let dyn_ty = user.param_callback(&pb.name)?; - let ctx = ident(&format!("{}_ctx", pb.name)); - let vtable = ident(&format!("{}_vtable", pb.name)); - let fail = reject(&format!("{}: null callback vtable", pb.name), sentinel); - let pre = quote! { - let #name = match ::weaveffi::abi::lift_callback::<#dyn_ty>(#ctx, #vtable) { - ::std::option::Option::Some(__wv_cb) => __wv_cb, - ::std::option::Option::None => { #fail } - }; - }; - (pre, arg) - } - _ => { - return Err(unsupported( - user.param_span(&pb.name), - &pb.name, - "parameter type", - )) - } - }) -} - -/// Lower an owned Rust `value` of IR type `ty` into its C return expression -/// (evaluated inside the thunk's `unsafe` body). `out_len` names the trailing -/// length slot for the `(ptr, len)` shapes; `object` is the producer's -/// spelling of the pointee for an object return (see -/// [`UserSig::ret_object`]), which pins the `Arc` the value converts into. -/// -/// Every heap-owning lowering here creates the consumer obligation stated by -/// [`weaveffi_model::plan::return_free`]: strings, bytes, and value buffers -/// are released with `{prefix}_free_bytes`, object references with the -/// type's `_destroy` symbol. -pub(crate) fn lower_value( - ty: &Ty, - value: TokenStream, - object: Option<&TokenStream>, - user: &UserSig<'_>, -) -> syn::Result { - // A buffered return is encoded into a producer-allocated value buffer and - // returned exactly like a bytes return: base pointer plus `*out_len`. - if ty.is_buffered() { - return Ok(quote! { - ::weaveffi::abi::lower_bytes(::weaveffi::abi::encode_value(&(#value)), out_len) - }); - } - // `let __wv_p: *mut T = lower_object(v)` pins `T` so a `Self`/`Arc` - // return converts into the right `Arc` without a turbofish. - let typed = |call: TokenStream| match object { - Some(obj) => quote!({ let __wv_p: *mut #obj = #call; __wv_p }), - None => call, - }; - Ok(match ty { - Ty::Enum(_) => quote!((#value).__weaveffi_to_i32()), - t if is_copy(t) => value, - // `Into` accepts both an owned value and a borrowed `&str`/`&[u8]`. - Ty::StringUtf8 => quote! { - ::weaveffi::abi::lower_string(::std::convert::Into::into(#value), out_len) - }, - Ty::Bytes => quote! { - ::weaveffi::abi::lower_bytes(::std::convert::Into::into(#value), out_len) - }, - // A returned object hands the consumer one strong reference, which it - // releases with the type's `_destroy` symbol. The producer may return - // `Self`/`T` or `Arc`/`Arc`. - Ty::Interface(_) => typed(quote!(::weaveffi::abi::lower_object(#value))), - Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => { - typed(quote!(::weaveffi::abi::lower_object_opt(#value))) - } - _ => return Err(unsupported(user.ret_span(), "return", "return type")), - }) -} - -/// Assemble the call, error handling, and return lowering for a callable -/// whose `call` expression invokes the producer's code. -/// -/// `is_throws` selects the `Result`-matching body; it comes from the plan's -/// [`ErrorStrategy`](weaveffi_model::plan::ErrorStrategy) (`Throws` routes the -/// producer's `Err` through `out_err` as a typed domain error, carrying the -/// matched code's serialized payload fields; `Trap` leaves `out_err` to the -/// panic path only). -pub(crate) fn build_call_body( - ret_ty: Option<&Ty>, - ret_ctype: &CType, - is_throws: bool, - call: TokenStream, - user: &UserSig<'_>, -) -> syn::Result { - let sentinel = sentinel(ret_ctype); - let object = user.ret_object(); - let lowered = match ret_ty { - Some(ty) => lower_value(ty, quote!(__wv_ret), object.as_ref(), user)?, - None => TokenStream::new(), - }; - let deferred = deferred_failure_check(sentinel.as_ref()); - let err_return = early_return(sentinel.as_ref()); - let bind = if is_throws { - quote! { - let __wv_ret = match __wv_out { - ::std::result::Result::Ok(__wv_v) => __wv_v, - ::std::result::Result::Err(__wv_err) => { - ::weaveffi::abi::error_store( - out_err, - ::weaveffi::abi::FfiError::from_report(&__wv_err), - ); - #err_return - } - }; - } - } else { - quote!(let __wv_ret = __wv_out;) - }; - let tail = if ret_ty.is_some() { - quote!(#lowered) - } else { - quote!(let () = __wv_ret;) - }; - Ok(quote! { - let __wv_out = #call; - #deferred - #bind - ::weaveffi::abi::error_clear(out_err); - #tail - }) -} - -/// The statement that appends one field's encoding to the writer `__wv_w`, -/// where `access` is a place expression for the field (e.g. `self.name`). -/// -/// Every buffer-legal type routes through the [`weaveffi_abi::BufferValue`] -/// trait, which the expansion implements for records, rich enums, and C-style -/// enums, and the runtime blanket-implements for primitives, `String`, -/// collections, `Option`, and `Arc` (an object token), so arbitrary -/// nesting composes. -pub(crate) fn field_write_stmt(_field: &FieldBinding, access: TokenStream) -> TokenStream { - quote!(::weaveffi::abi::BufferValue::write_value(&#access, __wv_w);) -} - -/// The expression that decodes one field's value from the reader `__wv_r`, -/// evaluating to `Result` (callers apply `?`). -/// The concrete field type is inferred from the surrounding struct or enum -/// constructor, so the producer's own type (including the map flavor) is -/// what decoding targets. -pub(crate) fn field_read_expr(_field: &FieldBinding) -> TokenStream { - quote!(::weaveffi::abi::BufferValue::read_value(__wv_r)) -} - -/// The statement that writes one *borrowed* field binding (`&T`, as produced -/// by a match on `&self`) to the writer `__wv_w`. Mirrors -/// [`field_write_stmt`] with reference access. -pub(crate) fn field_write_stmt_ref(_field: &FieldBinding, binding: &syn::Ident) -> TokenStream { - quote!(::weaveffi::abi::BufferValue::write_value(#binding, __wv_w);) -} diff --git a/crates/weaveffi-macros/src/codegen/meta.rs b/crates/weaveffi-macros/src/codegen/meta.rs new file mode 100644 index 00000000..7dcb2c3a --- /dev/null +++ b/crates/weaveffi-macros/src/codegen/meta.rs @@ -0,0 +1,67 @@ +//! The library metadata of one module: an exported static per declaration +//! holding its [`Frame`](weaveffi_model::meta::Frame), from which +//! `weaveffi generate` reads the API out of the built library. +//! +//! Each static is emitted inside the module it describes, so the module's +//! own `#[cfg]` (and its ancestors') applies to it, and carries the +//! declaration's `#[cfg]` (for an interface member, the interface's and its +//! `impl` block's). A declaration a build compiles out is therefore absent +//! from the library's metadata as well as from its symbols. On `wasm32`, +//! which has no symbol table to read data from, the statics go to the +//! `weaveffi_meta` custom section instead. + +use proc_macro2::{Span, TokenStream}; +use quote::quote; +use weaveffi_model::ir::Module; +use weaveffi_model::meta::{module_frames, SECTION}; + +use super::helpers::{cfg_wrap, ident}; + +/// The metadata statics of the module at `segments` in `tree` (the tree's +/// root is `segments[0]`). `cfg` returns the `#[cfg]` attributes of a +/// declaration in that module, given the names that locate it. +pub(crate) fn gen_metadata<'a>( + tree: &Module, + segments: &[String], + prefix: &str, + cfg: impl Fn(&[&str]) -> Vec<&'a syn::Attribute>, +) -> TokenStream { + let Some((module, index)) = find(tree, segments) else { + return TokenStream::new(); + }; + let parent = &segments[..segments.len() - 1]; + module_frames(module, parent, index, prefix) + .into_iter() + .map(|frame| { + let bytes = frame.encode(); + let len = bytes.len(); + let lit = syn::LitByteStr::new(&bytes, Span::call_site()); + let name = ident(&frame.symbol()); + let item = quote! { + #[doc(hidden)] + #[allow(unsafe_code)] + #[unsafe(no_mangle)] + #[cfg_attr(target_family = "wasm", unsafe(link_section = #SECTION))] + #[used] + pub static #name: [u8; #len] = *#lit; + }; + cfg_wrap(&cfg(&frame.local_path()), item) + }) + .collect() +} + +/// The module at `segments` and its index among its parent's submodules. +fn find<'m>(tree: &'m Module, segments: &[String]) -> Option<(&'m Module, u32)> { + let (first, rest) = segments.split_first()?; + if *first != tree.name { + return None; + } + let mut module = tree; + let mut index = 0; + for name in rest { + let i = module.modules.iter().position(|m| m.name == *name)?; + module = &module.modules[i]; + index = u32::try_from(i).ok()?; + } + Some((module, index)) +} diff --git a/crates/weaveffi-macros/src/codegen/mod.rs b/crates/weaveffi-macros/src/codegen/mod.rs index 90112717..ac92f465 100644 --- a/crates/weaveffi-macros/src/codegen/mod.rs +++ b/crates/weaveffi-macros/src/codegen/mod.rs @@ -1,47 +1,56 @@ -//! Lower a `#[weaveffi::module]` to its IR and emit the C ABI thunks. +//! Lower a `#[weaveffi::module]` to its IR, validate it, and emit the C ABI +//! thunks. //! -//! The flow mirrors the rest of WeaveFFI: parse the annotated module tree to -//! the IR via [`weaveffi_model::rust`], resolve it, build the canonical -//! [`BindingModel`], then render each lowered symbol. Signatures come straight -//! from the model (so they match the generated header by construction); only -//! the body marshalling (lift each ABI slot into a Rust value, call the -//! producer's safe function, lower the result) is new here, and every -//! `unsafe` operation in it bottoms out in a `weaveffi-abi` helper. +//! The flow mirrors the rest of WeaveFFI: extract the annotated module tree +//! to the IR via [`weaveffi_model::rust`], validate it with the shared +//! validator (tolerating names declared in another tree, which must be +//! records or rich enums) to build the canonical [`Model`], then render +//! each lowered symbol. A validation error becomes a compile error on the +//! offending item ([`diagnostics`]). Signatures come straight from the model +//! (so they match the generated header by construction); the bodies call +//! the runtime's per-family lifting and lowering functions (`weaveffi::abi`), +//! so every `unsafe` operation has one audited home. //! //! The emission is split by surface: [`sync`] for synchronous callables, //! [`async_fns`] for `async fn` launchers, [`iterators`] for `iter` trios, //! [`records`] and [`enums`] for the generated `BufferValue` serialization //! impls of value types, [`interfaces`] for the object reference-count //! symbols, [`callbacks`] for callback-interface vtables and foreign -//! wrappers, and [`foreign`] for the compile-time checks on types declared in -//! another module tree. [`helpers`] and [`marshal`] hold the shared slot -//! rendering and lift/lower machinery. +//! wrappers, [`contract`] for the module's contract table, [`meta`] for the +//! library metadata the CLI reads the API from, and [`foreign`] for the +//! compile-time checks on types declared in another module tree. +//! [`helpers`] and [`lift`] hold the shared slot rendering and the +//! lift/lower dispatch. An item's `#[cfg]` wraps everything generated for it. mod async_fns; mod callbacks; +mod contract; +mod diagnostics; mod enums; mod foreign; mod helpers; mod interfaces; mod iterators; -mod marshal; +mod lift; +mod meta; mod records; mod sync; use std::collections::{BTreeSet, HashMap}; use proc_macro2::{Span, TokenStream}; -use quote::{quote, quote_spanned, ToTokens}; -use weaveffi_model::ir::{Api, CURRENT_SCHEMA_VERSION}; -use weaveffi_model::model::{BindingModel, ErrorBinding, ModuleBinding}; -use weaveffi_model::plan::ErrorStrategy; +use quote::{quote, quote_spanned}; +use weaveffi_model::ir::{Api, TypeRef, CURRENT_SCHEMA_VERSION}; +use weaveffi_model::model::{ErrorBinding, Model, ModuleBinding}; +use weaveffi_model::rust::SourceMap; +use weaveffi_model::validate::{validate_scoped, Options}; pub(crate) use self::helpers::ident; -use self::helpers::CallTarget; +use self::helpers::{cfg_wrap, CallTarget}; use self::sync::gen_function; /// The C symbol prefix of the crate being expanded: its crate name, which is -/// also what the CLI derives for a `.rs` input, so the macro and +/// also the prefix the CLI gives the crate's identity, so the macro and /// `weaveffi generate` emit identical symbols. pub(crate) fn prefix() -> syn::Result { std::env::var("CARGO_CRATE_NAME").map_err(|_| { @@ -56,52 +65,40 @@ pub(crate) fn prefix() -> syn::Result { /// Expand a top-level `#[weaveffi::module]` into the original module plus /// generated thunks. pub fn expand_module(item_mod: &syn::ItemMod) -> syn::Result { - if item_mod.content.is_none() { - return Err(syn::Error::new_spanned( - item_mod, - "#[weaveffi::module] requires an inline module body (`mod foo { ... }`)", - )); - } - - // 1. Lower the whole tree to IR through the shared bridge (it recurses - // into nested `#[weaveffi::module]` submodules). The contract checksum - // is computed from this IR, exactly as the CLI computes it from the - // same source. - let module_ir = weaveffi_model::rust::module_from_item_mod(item_mod)?; - let checksum = weaveffi_model::checksum::module_checksum(&module_ir); + // 1. Extract the whole tree (it recurses into nested + // `#[weaveffi::module]` submodules), recording where each declaration + // came from and its `#[cfg]`. + let (module_ir, source) = weaveffi_model::rust::extract_module(item_mod)?; let prefix = prefix()?; - // A module tree is expanded in isolation, so a type declared in a - // *sibling* tree (`orders` using `products::Product`) can't be - // resolved here. `assume_valid` lowers such names as value buffers and - // `unresolved` lists them; `foreign` asserts each one really is a - // record or rich enum, so the thunk agrees with the CLI's header. - // Whole-API rule checks stay with the CLI's `validate`. - let api = weaveffi_model::ResolvedApi::assume_valid(Api { + // 2. Validate and build the model. A module tree is expanded in + // isolation, so a type declared in a *sibling* tree (`orders` using + // `products::Product`) can't be resolved here; `foreign_names` + // accepts such names as records or rich enums, `unresolved` (below) + // lists them, and `foreign` asserts each one really is one. + let api = Api { version: CURRENT_SCHEMA_VERSION.to_string(), modules: vec![module_ir], - }) - .with_identity(weaveffi_model::pkg::Identity::named(&prefix)); - let unresolved: BTreeSet = api.unresolved().into_iter().collect(); - check_callback_returns(item_mod, &api)?; - - // 2. Build the lowered model. Nested modules are flattened into one - // binding each, keyed here by their path segments so the recursive - // rebuild can match each `mod` to its binding. - // The lowering asserts invariants the CLI's validation guarantees; the - // checks above cover the ones a producer can reach, and this turns any - // other violation into a compile error instead of a macro panic. - let model = std::panic::catch_unwind(|| BindingModel::build(&api)).map_err(|payload| { - let why = payload - .downcast_ref::() - .map(String::as_str) - .or_else(|| payload.downcast_ref::<&str>().copied()) - .unwrap_or("invalid module"); - syn::Error::new_spanned( - &item_mod.ident, - format!("weaveffi: this module can't be lowered to the C ABI: {why}"), - ) - })?; + }; + let identity = weaveffi_model::pkg::Identity::named(&prefix); + let model = validate_scoped( + &api, + &identity, + Options { + foreign_names: true, + }, + ) + .map_err(|found| diagnostics::validation_errors(found, &source, item_mod.ident.span()))?; + let mut unresolved = BTreeSet::new(); + api.for_each_type_ref(&mut |ty| { + ty.walk(&mut |t| { + if let TypeRef::Named(name) = t { + if model.types.get(name).is_none() { + unresolved.insert(name.clone()); + } + } + }); + }); let by_path: HashMap, &ModuleBinding> = model .modules .iter() @@ -112,80 +109,30 @@ pub fn expand_module(item_mod: &syn::ItemMod) -> syn::Result { // and stripping inner `#[weaveffi::module]` markers so nested modules // expand here (with the right symbol path) instead of standalone. let ctx = Expansion { + model: &model, + tree: &api.modules[0], prefix: &prefix, by_path: &by_path, unresolved: &unresolved, + source: &source, }; - let checksum_sym = ident(&format!("{prefix}_{}_checksum", item_mod.ident)); - let checksum_fn = quote! { - /// The contract checksum of this module tree, which generated - /// bindings compare with the value they were generated against. - #[doc(hidden)] - #[unsafe(no_mangle)] - #[allow(unsafe_code)] - pub extern "C" fn #checksum_sym() -> u64 { - #checksum - } - }; - ctx.rebuild_module(item_mod, &[], checksum_fn) -} - -/// Reject callback-interface method returns the ABI can't carry (the CLI's -/// validation reports the same rule for IDL input): a method returns nothing -/// or a direct value (a number, `bool`, or C-style enum), optionally wrapped -/// as `Result`. -fn check_callback_returns( - item_mod: &syn::ItemMod, - api: &weaveffi_model::ResolvedApi, -) -> syn::Result<()> { - use weaveffi_model::model::Family; - use weaveffi_model::rust::{ - has_marker, output_is_result, return_type_from_syn, returns_foreign_result, - }; - let Some((_, items)) = &item_mod.content else { - return Ok(()); - }; - for item in items { - match item { - syn::Item::Trait(t) if has_marker(&t.attrs, "callback_interface") => { - for ti in &t.items { - let syn::TraitItem::Fn(f) = ti else { - continue; - }; - let syn::ReturnType::Type(_, ret) = &f.sig.output else { - continue; - }; - if output_is_result(&f.sig.output) && !returns_foreign_result(&f.sig.output) { - return Err(syn::Error::new_spanned( - ret, - "weaveffi: a callback interface method can return \ - `Result` (to receive the consumer's \ - failure as a value) but no other `Result`", - )); - } - let direct = return_type_from_syn(&f.sig.output)? - .is_none_or(|r| matches!(api.resolve(&r).family(), Family::Direct)); - if !direct { - return Err(syn::Error::new_spanned( - ret, - "weaveffi: a callback interface method can only return a number, \ - `bool`, or C-style enum (or nothing)", - )); - } - } - } - syn::Item::Mod(m) if has_marker(&m.attrs, "module") => check_callback_returns(m, api)?, - _ => {} - } - } - Ok(()) + let root = model + .roots() + .next() + .ok_or_else(|| syn::Error::new_spanned(&item_mod.ident, "internal error: no root"))?; + let contract_fn = contract::gen_contract(&model, root, &source, &prefix); + ctx.rebuild_module(item_mod, &[], contract_fn) } /// What every module in one expansion shares. struct Expansion<'a> { + model: &'a Model, + /// The extracted tree, which the library metadata describes. + tree: &'a weaveffi_model::ir::Module, prefix: &'a str, by_path: &'a HashMap, &'a ModuleBinding>, unresolved: &'a BTreeSet, + source: &'a SourceMap, } impl Expansion<'_> { @@ -213,17 +160,21 @@ impl Expansion<'_> { ) })?; - let mut generated = render_symbols(mb, items, &item_mod.ident, self.prefix)?; + let mut generated = self.render_symbols(mb, items, &item_mod.ident)?; generated.extend(foreign::by_value_assertions(items, self.unresolved)); + generated.extend(meta::gen_metadata( + self.tree, + &segments, + self.prefix, + |names| self.cfg(mb, names), + )); // Pass items through verbatim, except nested `#[weaveffi::module]`s, // which expand inline (recursively) with their marker stripped. let mut body = TokenStream::new(); for item in items { if let syn::Item::Mod(child) = item { - if weaveffi_model::rust::has_marker(&child.attrs, "module") - && child.content.is_some() - { + if weaveffi_model::rust::has_marker(&child.attrs, "module") { body.extend(self.rebuild_module(child, &segments, TokenStream::new())?); continue; } @@ -248,124 +199,136 @@ impl Expansion<'_> { } }) } -} -/// Render every C ABI symbol for one lowered module binding. -/// -/// `items` are the syn items directly in that module's body; they are indexed -/// so body marshalling can read reference-ness and `Result` returns from the -/// producer's signatures, and record and enum codegen can read the real -/// field and variant types. `mod_ident` anchors module-level diagnostics. -fn render_symbols( - mb: &ModuleBinding, - items: &[syn::Item], - mod_ident: &syn::Ident, - prefix: &str, -) -> syn::Result { - let mut fns: HashMap = HashMap::new(); - let mut enums: HashMap = HashMap::new(); - let mut traits: HashMap = HashMap::new(); - // Interface member signatures, keyed by `(type name, fn name)` across all - // inherent `impl` blocks of the type. - let mut member_sigs: HashMap<(String, String), &syn::Signature> = HashMap::new(); - for item in items { - match item { - syn::Item::Fn(f) => { - fns.insert(f.sig.ident.to_string(), f); - } - syn::Item::Enum(e) => { - enums.insert(e.ident.to_string(), e); - } - syn::Item::Trait(t) => { - traits.insert(t.ident.to_string(), t); - } - syn::Item::Impl(i) if i.trait_.is_none() => { - let Some(ty_name) = impl_type_name(i) else { - continue; - }; - for impl_item in &i.items { - if let syn::ImplItem::Fn(f) = impl_item { - member_sigs.insert((ty_name.clone(), f.sig.ident.to_string()), &f.sig); + /// The `#[cfg]` attributes on the declaration `names` in module `mb`. + fn cfg(&self, mb: &ModuleBinding, names: &[&str]) -> Vec<&syn::Attribute> { + let mut path = mb.segments.clone(); + let mut out = Vec::new(); + for name in names { + path.push((*name).to_string()); + out.extend(self.source.cfg(&path)); + } + out + } + + /// The path, relative to `mb`, of the error domain in scope there. + fn domain_path(&self, mb: &ModuleBinding) -> Option { + let domain = self.model.error_domain(mb)?; + let owner = self + .model + .modules + .iter() + .find(|m| m.errors.as_ref().is_some_and(|e| e.name == domain.name))?; + let ups = mb.segments.len() - owner.segments.len(); + let supers = std::iter::repeat_n(quote!(super::), ups); + let name = ident(&domain.name); + Some(quote!(#(#supers)* #name)) + } + + /// Render every C ABI symbol for one lowered module binding. + /// + /// `items` are the syn items directly in that module's body; they are + /// indexed so body marshalling can read reference-ness and `Result` + /// returns from the producer's signatures. `mod_ident` anchors + /// module-level diagnostics. + fn render_symbols( + &self, + mb: &ModuleBinding, + items: &[syn::Item], + mod_ident: &syn::Ident, + ) -> syn::Result { + let prefix = self.prefix; + let mut fns: HashMap = HashMap::new(); + let mut enums: HashMap = HashMap::new(); + let mut traits: HashMap = HashMap::new(); + // Interface member signatures, keyed by `(type name, fn name)` across + // all inherent `impl` blocks of the type. + let mut member_sigs: HashMap<(String, String), &syn::Signature> = HashMap::new(); + for item in items { + match item { + syn::Item::Fn(f) => { + fns.insert(f.sig.ident.to_string(), f); + } + syn::Item::Enum(e) => { + enums.insert(e.ident.to_string(), e); + } + syn::Item::Trait(t) => { + traits.insert(t.ident.to_string(), t); + } + syn::Item::Impl(i) if i.trait_.is_none() => { + let Some(ty_name) = impl_type_name(i) else { + continue; + }; + for impl_item in &i.items { + if let syn::ImplItem::Fn(f) = impl_item { + member_sigs.insert((ty_name.clone(), f.sig.ident.to_string()), &f.sig); + } } } + _ => {} } - _ => {} } - } - let missing = |what: &str, name: &str| { - syn::Error::new_spanned( - mod_ident, - format!("internal error: no source for exported {what} `{name}`"), - ) - }; + let missing = |what: &str, name: &str| { + syn::Error::new_spanned( + mod_ident, + format!("internal error: no source for exported {what} `{name}`"), + ) + }; - // Mirror the CLI's validation: a callable whose error strategy is - // `Throws` reports typed domain errors through `out_err`, so an error - // domain must be in scope for its codes to have a C spelling. - if mb.error.is_none() { - if let Some(t) = mb - .callables() - .find(|f| f.error_strategy() == ErrorStrategy::Throws) - { - let at = fns.get(&t.name).map_or_else( - || mod_ident.to_token_stream(), - |f| f.sig.output.to_token_stream(), - ); - return Err(syn::Error::new_spanned( - at, - format!( - "weaveffi: `{}` returns a Result but no error domain is in scope; declare \ - a #[weaveffi::error] enum in this module (or a parent module)", - t.name - ), + let mut generated = TokenStream::new(); + if let Some(eb) = &mb.errors { + let item = enums + .get(&eb.name) + .ok_or_else(|| missing("error domain", &eb.name))?; + generated.extend(cfg_wrap( + &self.cfg(mb, &[&eb.name]), + gen_error_domain(eb, item), )); } - } - - let mut generated = TokenStream::new(); - if let Some(eb) = mb.error.as_ref().filter(|e| e.declared_here) { - let item = enums - .get(&eb.name) - .ok_or_else(|| missing("error domain", &eb.name))?; - generated.extend(gen_error_report(eb, item)); - } - for e in &mb.enums { - let item = enums.get(&e.name).ok_or_else(|| missing("enum", &e.name))?; - generated.extend(enums::gen_enum(e, item)?); - } - for s in &mb.structs { - generated.extend(records::gen_record(s)); - } - for c in &mb.callback_interfaces { - let item = traits - .get(&c.name) - .ok_or_else(|| missing("callback interface", &c.name))?; - generated.extend(callbacks::gen_callback_interface(c, item, prefix)?); - } - for i in &mb.interfaces { - let ty = ident(&i.name); - for (members, target) in [ - (&i.constructors, CallTarget::Static(ty.clone())), - (&i.statics, CallTarget::Static(ty.clone())), - (&i.methods, CallTarget::Method(ty.clone())), - ] { - for m in members { - let sig = member_sigs - .get(&(i.name.clone(), m.name.clone())) - .ok_or_else(|| missing("interface member", &m.name))?; - generated.extend(gen_function(m, sig, &target, prefix)?); + for e in &mb.enums { + generated.extend(cfg_wrap(&self.cfg(mb, &[&e.name]), enums::gen_enum(e))); + } + for s in &mb.structs { + generated.extend(cfg_wrap(&self.cfg(mb, &[&s.name]), records::gen_record(s))); + } + let domain = self.domain_path(mb); + for c in &mb.callback_interfaces { + let item = traits + .get(&c.name) + .ok_or_else(|| missing("callback interface", &c.name))?; + let code = callbacks::gen_callback_interface(c, item, domain.as_ref(), prefix)?; + generated.extend(cfg_wrap(&self.cfg(mb, &[&c.name]), code)); + } + for i in &mb.interfaces { + let ty = ident(&i.name); + for (members, target) in [ + (&i.constructors, CallTarget::Static(ty.clone())), + (&i.statics, CallTarget::Static(ty.clone())), + (&i.methods, CallTarget::Method(ty.clone())), + ] { + for m in members { + let sig = member_sigs + .get(&(i.name.clone(), m.name.clone())) + .ok_or_else(|| missing("interface member", &m.name))?; + let code = gen_function(m, sig, &target, prefix)?; + generated.extend(cfg_wrap(&self.cfg(mb, &[&i.name, &m.name]), code)); + } } + generated.extend(cfg_wrap( + &self.cfg(mb, &[&i.name]), + interfaces::gen_interface_lifecycle(i), + )); } - generated.extend(interfaces::gen_interface_lifecycle(i)); - } - for f in &mb.functions { - let sfn = fns - .get(&f.name) - .ok_or_else(|| missing("function", &f.name))?; - generated.extend(gen_function(f, &sfn.sig, &CallTarget::Free, prefix)?); + for f in &mb.functions { + let sfn = fns + .get(&f.name) + .ok_or_else(|| missing("function", &f.name))?; + let code = gen_function(f, &sfn.sig, &CallTarget::Free, prefix)?; + generated.extend(cfg_wrap(&self.cfg(mb, &[&f.name]), code)); + } + Ok(generated) } - Ok(generated) } /// The bare type name an inherent `impl` block targets, when it is a plain @@ -377,13 +340,15 @@ fn impl_type_name(item_impl: &syn::ItemImpl) -> Option { p.path.segments.last().map(|s| s.ident.to_string()) } -/// Generate the [`ErrorReport`](weaveffi_abi::ErrorReport) implementation for -/// a module's `#[weaveffi::error]` enum: each variant maps to its declared -/// code, the message is the enum's `Display` output, and a payload variant's -/// fields are serialized into the error's value-buffer payload. This is what -/// routes `Err(Domain::Case)` from a throwing producer function to the -/// matching C error constant. -fn gen_error_report(eb: &ErrorBinding, item: &syn::ItemEnum) -> TokenStream { +/// Generate the [`ErrorReport`](weaveffi::abi::ErrorReport) and +/// [`ErrorDomain`](weaveffi::abi::ErrorDomain) implementations for a +/// module's `#[weaveffi::error]` enum: each variant maps to its declared +/// code, the message is the enum's `Display` output, and a payload +/// variant's fields are serialized into (and decoded from) the error's +/// value-buffer payload. This is what routes `Err(Domain::Case)` from a +/// throwing producer function to the matching C error constant, and a +/// callback's domain error back to `Domain::Case`. +fn gen_error_domain(eb: &ErrorBinding, item: &syn::ItemEnum) -> TokenStream { let ty = ident(&eb.name); // A payload-carrying variant matches with `{ .. }`; a unit variant by name. let pattern = |c: &weaveffi_model::model::ErrorCodeBinding| { @@ -429,6 +394,20 @@ fn gen_error_report(eb: &ErrorBinding, item: &syn::ItemEnum) -> TokenStream { } } }; + let read_arms = eb.codes.iter().map(|c| { + let v = ident(&c.name); + let value = c.value; + if c.fields.is_empty() { + quote!(#value => Self::#v,) + } else { + let names: Vec = c.fields.iter().map(|f| ident(&f.name)).collect(); + quote! { + #value => Self::#v { + #(#names: ::weaveffi::abi::BufferValue::read_value(__wv_r)?),* + }, + } + } + }); // Spanned at the enum so a missing `Display` impl is one error that // points at it and names the requirement. let user_ty = &item.ident; @@ -454,6 +433,25 @@ fn gen_error_report(eb: &ErrorBinding, item: &syn::ItemEnum) -> TokenStream { } #payload_fn } + + #[allow(unsafe_code, unused_unsafe)] + impl ::weaveffi::abi::ErrorDomain for #ty { + unsafe fn read_code( + code: i32, + __wv_r: &mut ::weaveffi::abi::BufferReader<'_>, + ) -> ::std::result::Result< + ::std::option::Option, + ::weaveffi::abi::BufferDecodeError, + > { + // SAFETY: forwarded from the caller. + ::std::result::Result::Ok(::std::option::Option::Some(unsafe { + match code { + #(#read_arms)* + _ => return ::std::result::Result::Ok(::std::option::Option::None), + } + })) + } + } } } diff --git a/crates/weaveffi-macros/src/codegen/records.rs b/crates/weaveffi-macros/src/codegen/records.rs index af98a00e..f871d1c3 100644 --- a/crates/weaveffi-macros/src/codegen/records.rs +++ b/crates/weaveffi-macros/src/codegen/records.rs @@ -1,4 +1,4 @@ -//! Codegen for records: the generated [`weaveffi_abi::BufferValue`] +//! Codegen for records: the generated [`weaveffi::abi::BufferValue`] //! implementation that serializes the producer's struct field by field in //! declaration (wire) order. //! @@ -11,35 +11,28 @@ use proc_macro2::TokenStream; use quote::quote; use weaveffi_model::model::StructBinding; -use super::helpers::{ident, rust_type_ident}; -use super::marshal::{field_read_expr, field_write_stmt}; +use super::helpers::ident; /// Generate the `BufferValue` and `ByValue` implementations for one record. pub(crate) fn gen_record(s: &StructBinding) -> TokenStream { - let rust_ty = rust_type_ident(&s.name); + let rust_ty = ident(&s.name); let names: Vec = s.fields.iter().map(|f| ident(&f.name)).collect(); - let writes = s - .fields - .iter() - .zip(&names) - .map(|(f, n)| field_write_stmt(f, quote!(self.#n))); - let reads = s.fields.iter().zip(&names).map(|(f, n)| { - let read = field_read_expr(f); - quote!(#n: #read?) - }); - quote! { + #[allow(unsafe_code, unused_unsafe)] impl ::weaveffi::abi::BufferValue for #rust_ty { fn encoded_len(&self) -> usize { 0 #(+ ::weaveffi::abi::BufferValue::encoded_len(&self.#names))* } fn write_value(&self, __wv_w: &mut ::weaveffi::abi::BufferWriter) { - #(#writes)* + #(::weaveffi::abi::BufferValue::write_value(&self.#names, __wv_w);)* } - fn read_value( + unsafe fn read_value( __wv_r: &mut ::weaveffi::abi::BufferReader<'_>, ) -> ::std::result::Result { - ::std::result::Result::Ok(Self { #(#reads),* }) + // SAFETY: forwarded from the caller. + ::std::result::Result::Ok(unsafe { + Self { #(#names: ::weaveffi::abi::BufferValue::read_value(__wv_r)?),* } + }) } } diff --git a/crates/weaveffi-macros/src/codegen/sync.rs b/crates/weaveffi-macros/src/codegen/sync.rs index 3b3542bf..1e0c88fa 100644 --- a/crates/weaveffi-macros/src/codegen/sync.rs +++ b/crates/weaveffi-macros/src/codegen/sync.rs @@ -2,25 +2,53 @@ //! every callable (free function or interface member) goes through. use proc_macro2::TokenStream; -use quote::quote; +use quote::{quote, quote_spanned}; use weaveffi_model::model::{AbiFn, CallShape, FnBinding}; use weaveffi_model::plan::ErrorStrategy; use super::async_fns::gen_async_function; use super::helpers::{ - fn_slots, ident, ret_arrow_for, sentinel, thunk_attrs, wrap_unwind, CallTarget, UserSig, + fn_slots, ident, ret_arrow_for, ret_type_for, thunk_attrs, CallTarget, UserSig, }; use super::iterators::gen_iterator_function; -use super::marshal::{build_call_body, lift_param}; +use super::lift::{lift_param, lower_ret, Lifts}; /// Whether this callable routes typed domain errors through `out_err`, per /// the plan's error contract ([`weaveffi_model::plan::ErrorStrategy`]). The -/// producer declared it by returning a `Result`, so the thunk matches on the -/// call's `Ok`/`Err` instead of binding the value directly. +/// producer declared it by returning a `Result`, so the thunk propagates the +/// call's `Err` as the domain error. pub(crate) fn throws(f: &FnBinding) -> bool { f.error_strategy() == ErrorStrategy::Throws } +/// The producer call, with a throwing callable's `Err` turned into its +/// `FfiError` and propagated with `?`. +pub(crate) fn checked_call(f: &FnBinding, call: TokenStream, user: &UserSig<'_>) -> TokenStream { + if throws(f) { + let span = user.ret_type_span(); + quote_spanned! {span=> + #call.map_err(|__wv_e| ::weaveffi::abi::FfiError::from_report(&__wv_e))? + } + } else { + call + } +} + +/// Lift every parameter (and the receiver) of `f` for a call that borrows +/// them for its duration. +pub(crate) fn sync_lifts( + f: &FnBinding, + user: &UserSig<'_>, + target: &CallTarget, +) -> syn::Result { + let mut lifts = Lifts::default(); + lifts.receiver(target.self_lift(user.receiver_is_arc())); + for pb in &f.params { + lift_param(&mut lifts, pb, user, false)?; + } + Ok(lifts) +} + /// Dispatch one callable (free function or interface member) to the codegen /// for its call shape. pub(crate) fn gen_function( @@ -37,7 +65,9 @@ pub(crate) fn gen_function( } } -/// Generate the `extern "C"` thunk for one synchronous callable. +/// Generate the `extern "C"` thunk for one synchronous callable: lift every +/// input, call the producer, and lower the result, all inside +/// `weaveffi::abi::call_sync`, which catches panics and fills `out_err`. fn gen_sync_function( f: &FnBinding, abi: &AbiFn, @@ -48,27 +78,25 @@ fn gen_sync_function( let sym = ident(&abi.symbol); let params = fn_slots(&abi.params, &f.params, user, prefix)?; let arrow = ret_arrow_for(&abi.ret, f.ret.as_ref(), user, prefix); - let sentinel = sentinel(&abi.ret); - - // Lift each parameter, collecting the preambles and the call arguments. - let self_pre = target.self_preamble(sentinel.as_ref(), user.receiver_is_arc()); - let mut preamble = TokenStream::new(); - let mut call_args: Vec = Vec::new(); - for pb in &f.params { - let (pre, arg) = lift_param(pb, user, sentinel.as_ref())?; - preamble.extend(pre); - call_args.push(arg); - } - - let call = target.call(&f.name, &call_args); - let body = build_call_body(f.ret.as_ref(), &abi.ret, throws(f), call, user)?; - let wrapped = wrap_unwind(quote! { #self_pre #preamble #body }, sentinel.as_ref()); + let ret_ty = ret_type_for(&abi.ret, f.ret.as_ref(), user, prefix); + let lifts = sync_lifts(f, user, target)?; + let lifted = lifts.finish(); + let call = checked_call(f, target.call(&f.name, &lifts.args), user); + let lowered = match &f.ret { + Some(ty) => lower_ret(ty, "e!(__wv_ret), user.ret_object().as_ref()), + None => quote!(__wv_ret), + }; let attrs = thunk_attrs(); - Ok(quote! { #attrs pub unsafe extern "C" fn #sym(#(#params),*) #arrow { - #wrapped + unsafe { + ::weaveffi::abi::call_sync::<#ret_ty>(out_err, move || unsafe { + #lifted + let __wv_ret = #call; + ::std::result::Result::Ok(#lowered) + }) + } } }) } diff --git a/crates/weaveffi-macros/src/lib.rs b/crates/weaveffi-macros/src/lib.rs index c8b4fb27..8a1ee6ba 100644 --- a/crates/weaveffi-macros/src/lib.rs +++ b/crates/weaveffi-macros/src/lib.rs @@ -4,9 +4,9 @@ //! tags the items it wants to export, and calls `weaveffi::export_runtime!()` //! once. The module macro lowers the module tree to the WeaveFFI IR (through //! [`weaveffi_model::rust`]), builds the canonical -//! [`BindingModel`](weaveffi_model::model::BindingModel), and emits the +//! [`Model`](weaveffi_model::model::Model), and emits the //! `extern "C"` thunks every generated language binding calls. All of the -//! `unsafe` marshalling lives in the `weaveffi-abi` runtime, so the producer +//! `unsafe` marshalling lives in the `weaveffi::abi` runtime, so the producer //! writes only safe Rust. //! //! ```ignore @@ -22,10 +22,10 @@ //! weaveffi::export_runtime!(); //! ``` //! -//! Every C symbol starts with the crate's name (`CARGO_CRATE_NAME`), which -//! is also the prefix `weaveffi generate` derives for a `.rs` input, so the -//! generated bindings and the producer can't drift: they're two views of one -//! parse. +//! Every C symbol starts with the crate's name (`CARGO_CRATE_NAME`). The +//! macro also embeds the module's API in the library as metadata, which +//! `weaveffi generate` reads back out of the built library, so the generated +//! bindings describe exactly what the library was compiled with. //! //! # Attributes //! @@ -38,7 +38,9 @@ //! * [`macro@error`] declares the module's error domain from an enum with //! explicit discriminants; a variant's named fields become the code's //! structured payload. -//! * [`macro@callback_interface`] declares a trait the consumer implements; +//! * [`macro@callback_interface`] declares a trait the consumer implements, +//! whose methods return `Result`; +//! [`macro@throws`] lets one of them report the module's domain errors. //! [`macro@cancellable`] marks an async function as cancellable. //! * [`export_runtime!`] emits the runtime symbols (memory, errors, cancel //! tokens, ABI version) once per library. @@ -55,11 +57,13 @@ mod runtime; /// Mark an inline `mod` as an exported WeaveFFI namespace. /// -/// The macro re-emits the module and appends the generated C ABI thunks for -/// every tagged item it contains, recursing into nested `#[weaveffi::module]` -/// submodules (whose symbols carry the joined module path). A top-level -/// module also exports `{prefix}_{module}_checksum()`, the contract checksum -/// generated bindings verify when they load the library. +/// The macro validates the module tree with the same rules the CLI applies +/// to an IDL, re-emits the module, and appends the generated C ABI thunks +/// for every tagged item it contains, recursing into nested +/// `#[weaveffi::module]` submodules (whose symbols carry the joined module +/// path). A `#[cfg]` on an exported item applies to its thunks too. A +/// top-level module also exports `{prefix}_{module}_contract()`, the +/// contract table generated bindings verify when they load the library. #[proc_macro_attribute] pub fn module(_attr: TokenStream, item: TokenStream) -> TokenStream { let item_mod = syn::parse_macro_input!(item as syn::ItemMod); @@ -70,9 +74,9 @@ pub fn module(_attr: TokenStream, item: TokenStream) -> TokenStream { /// Emit the runtime symbols every WeaveFFI library exports, prefixed with the /// crate's name: `{prefix}_abi_version`, `{prefix}_error_set`, -/// `{prefix}_error_clear`, `{prefix}_error_free`, `{prefix}_free_bytes`, the -/// four `{prefix}_cancel_token_*` functions, `{prefix}_debug_live`, and on -/// `wasm32` `{prefix}_alloc` and `{prefix}_dealloc`. +/// `{prefix}_error_set_payload`, `{prefix}_error_clear`, +/// `{prefix}_error_free`, `{prefix}_alloc`, `{prefix}_free_bytes`, the four +/// `{prefix}_cancel_token_*` functions, and `{prefix}_debug_live`. /// /// Invoke it exactly once, at the crate root of the `cdylib`. It takes no /// arguments. @@ -125,11 +129,18 @@ marker_attr! { } marker_attr! { /// Declare a callback interface: a trait whose `&self` methods the - /// consumer implements. Producers accept one as `Arc`. A - /// method may return `Result` to receive the - /// consumer's failure as a value. + /// consumer implements. Producers accept one as `Arc` (or + /// `Option>`). Every method returns + /// `Result`, which carries the consumer's + /// failure as a value. callback_interface } +marker_attr! { + /// Mark a callback-interface method as able to report the error domain + /// in scope: the consumer may fail with one of its codes, which the + /// producer decodes with `weaveffi::ForeignError::domain`. + throws +} marker_attr! { /// Mark an `async fn` as cancellable: it takes a `weaveffi::CancelToken` /// as its final parameter, and cancelling the token completes the call diff --git a/crates/weaveffi-macros/src/runtime.rs b/crates/weaveffi-macros/src/runtime.rs index 617467ba..2653565b 100644 --- a/crates/weaveffi-macros/src/runtime.rs +++ b/crates/weaveffi-macros/src/runtime.rs @@ -1,7 +1,7 @@ //! Expansion of `weaveffi::export_runtime!()`: the fixed runtime surface //! every library exports under its own prefix. //! -//! The bodies live in `weaveffi-abi`; only the `extern "C"` thunks are +//! The bodies live in `weaveffi::abi`; only the `extern "C"` thunks are //! emitted here, in the producer's crate, because `#[no_mangle]` symbols in //! a transitive `rlib` aren't guaranteed to be exported from a `cdylib`. //! @@ -27,6 +27,7 @@ pub(crate) fn expand(input: TokenStream) -> syn::Result { let sym = |name: &str| ident(&format!("{p}_{name}")); let abi_version = sym("abi_version"); let error_set = sym("error_set"); + let error_set_payload = sym("error_set_payload"); let error_clear = sym("error_clear"); let error_free = sym("error_free"); let free_bytes = sym("free_bytes"); @@ -36,7 +37,6 @@ pub(crate) fn expand(input: TokenStream) -> syn::Result { let token_destroy = sym("cancel_token_destroy"); let debug_live = sym("debug_live"); let alloc = sym("alloc"); - let dealloc = sym("dealloc"); Ok(quote! { // Consumers compare this against the revision they were generated // for before touching any other symbol, so it must stay a plain @@ -59,6 +59,17 @@ pub(crate) fn expand(input: TokenStream) -> syn::Result { unsafe { ::weaveffi::abi::error_set_c(err, code, message) } } + #[doc(hidden)] + #[unsafe(no_mangle)] + #[allow(unsafe_code, unused_unsafe, clippy::missing_safety_doc)] + pub unsafe extern "C" fn #error_set_payload( + err: *mut ::weaveffi::abi::FfiError, + ptr: *const u8, + len: usize, + ) { + unsafe { ::weaveffi::abi::error_set_payload_c(err, ptr, len) } + } + #[doc(hidden)] #[unsafe(no_mangle)] #[allow(unsafe_code, unused_unsafe, clippy::missing_safety_doc)] @@ -115,22 +126,14 @@ pub(crate) fn expand(input: TokenStream) -> syn::Result { ::weaveffi::abi::debug_live(kind) } - // Wasm has no host allocator, so the generated JS glue stages - // input buffers and return slots through these. - #[cfg(target_arch = "wasm32")] + // Consumers allocate the runs they hand to the producer (a + // callback's string, bytes, or buffer return) and, on wasm32, + // stage arguments through this. #[doc(hidden)] #[unsafe(no_mangle)] #[allow(unsafe_code)] - pub extern "C" fn #alloc(size: u32) -> *mut u8 { - ::weaveffi::abi::wasm_alloc(size as usize) - } - - #[cfg(target_arch = "wasm32")] - #[doc(hidden)] - #[unsafe(no_mangle)] - #[allow(unsafe_code, unused_unsafe, clippy::missing_safety_doc)] - pub unsafe extern "C" fn #dealloc(ptr: *mut u8, size: u32) { - unsafe { ::weaveffi::abi::wasm_dealloc(ptr, size as usize) } + pub extern "C" fn #alloc(len: usize) -> *mut u8 { + ::weaveffi::abi::alloc(len) } }) } diff --git a/crates/weaveffi-macros/tests/ui/fail_callback_plain_return.rs b/crates/weaveffi-macros/tests/ui/fail_callback_plain_return.rs new file mode 100644 index 00000000..57d41afb --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_callback_plain_return.rs @@ -0,0 +1,13 @@ +//! A callback method returns `Result`: the +//! consumer's implementation can fail, and a plain `T` has nowhere to put +//! the failure. + +#[weaveffi::module] +mod bad { + #[weaveffi::callback_interface] + pub trait Namer: Send + Sync { + fn name(&self) -> String; + } +} + +fn main() {} diff --git a/crates/weaveffi-macros/tests/ui/fail_callback_plain_return.stderr b/crates/weaveffi-macros/tests/ui/fail_callback_plain_return.stderr new file mode 100644 index 00000000..335a0ff9 --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_callback_plain_return.stderr @@ -0,0 +1,5 @@ +error: weaveffi: a callback interface method must return `Result` (or `Result<(), weaveffi::ForeignError>` for no value), because the consumer's implementation can fail and the `Err` is how its failure reaches you + --> tests/ui/fail_callback_plain_return.rs:9:27 + | +9 | fn name(&self) -> String; + | ^^^^^^ diff --git a/crates/weaveffi-macros/tests/ui/fail_callback_string_return.rs b/crates/weaveffi-macros/tests/ui/fail_callback_string_return.rs deleted file mode 100644 index e745ffef..00000000 --- a/crates/weaveffi-macros/tests/ui/fail_callback_string_return.rs +++ /dev/null @@ -1,9 +0,0 @@ -#[weaveffi::module] -mod bad { - #[weaveffi::callback_interface] - pub trait Namer: Send + Sync { - fn name(&self) -> String; - } -} - -fn main() {} diff --git a/crates/weaveffi-macros/tests/ui/fail_callback_string_return.stderr b/crates/weaveffi-macros/tests/ui/fail_callback_string_return.stderr deleted file mode 100644 index f32ba146..00000000 --- a/crates/weaveffi-macros/tests/ui/fail_callback_string_return.stderr +++ /dev/null @@ -1,5 +0,0 @@ -error: weaveffi: a callback interface method can only return a number, `bool`, or C-style enum (or nothing) - --> tests/ui/fail_callback_string_return.rs:5:27 - | -5 | fn name(&self) -> String; - | ^^^^^^ diff --git a/crates/weaveffi-macros/tests/ui/fail_iter_param.stderr b/crates/weaveffi-macros/tests/ui/fail_iter_param.stderr index b8336163..2bcd606f 100644 --- a/crates/weaveffi-macros/tests/ui/fail_iter_param.stderr +++ b/crates/weaveffi-macros/tests/ui/fail_iter_param.stderr @@ -1,4 +1,4 @@ -error: weaveffi: `Iter` is a pull handle the consumer drives and is only valid as a function's outermost return type, not as a parameter +error: weaveffi: iterator type is only valid as a function return type, found in param 'items' of function 'bad::drain' --> tests/ui/fail_iter_param.rs:4:25 | 4 | pub fn drain(items: weaveffi::Iter) -> i64 { diff --git a/crates/weaveffi-macros/tests/ui/fail_member_cfg.rs b/crates/weaveffi-macros/tests/ui/fail_member_cfg.rs new file mode 100644 index 00000000..1aa678dd --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_member_cfg.rs @@ -0,0 +1,14 @@ +//! A `#[cfg]` on a member (a field, variant, or method) can't be followed by +//! the generated bindings; it belongs on the whole item. + +#[weaveffi::module] +mod bad { + #[weaveffi::record] + pub struct Point { + pub x: i32, + #[cfg(unix)] + pub y: i32, + } +} + +fn main() {} diff --git a/crates/weaveffi-macros/tests/ui/fail_member_cfg.stderr b/crates/weaveffi-macros/tests/ui/fail_member_cfg.stderr new file mode 100644 index 00000000..a4ebdb73 --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_member_cfg.stderr @@ -0,0 +1,5 @@ +error: weaveffi: `#[cfg]` on a record field isn't supported, because the generated bindings can't follow it; put the `#[cfg]` on the whole type + --> tests/ui/fail_member_cfg.rs:9:9 + | +9 | #[cfg(unix)] + | ^ diff --git a/crates/weaveffi-macros/tests/ui/fail_nested_iter_return.stderr b/crates/weaveffi-macros/tests/ui/fail_nested_iter_return.stderr index 88faf542..52d45fc2 100644 --- a/crates/weaveffi-macros/tests/ui/fail_nested_iter_return.stderr +++ b/crates/weaveffi-macros/tests/ui/fail_nested_iter_return.stderr @@ -1,4 +1,4 @@ -error: weaveffi: `Iter` is only valid as the outermost return type; it can't be nested in an `Option`, `Vec`, map, or another `Iter` +error: weaveffi: iterator type is only valid as a function return type, found in nested inside the return type of bad::maybe_stream --> tests/ui/fail_nested_iter_return.rs:4:38 | 4 | pub fn maybe_stream(on: bool) -> Option> { diff --git a/crates/weaveffi-macros/tests/ui/fail_out_of_line_module.rs b/crates/weaveffi-macros/tests/ui/fail_out_of_line_module.rs new file mode 100644 index 00000000..8e017ac1 --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_out_of_line_module.rs @@ -0,0 +1,10 @@ +//! The macro reads its module tree from the token stream, so a submodule in +//! another file is invisible to it. + +#[weaveffi::module] +mod outer { + #[weaveffi::module] + mod inner; +} + +fn main() {} diff --git a/crates/weaveffi-macros/tests/ui/fail_out_of_line_module.stderr b/crates/weaveffi-macros/tests/ui/fail_out_of_line_module.stderr new file mode 100644 index 00000000..0ede1844 --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_out_of_line_module.stderr @@ -0,0 +1,5 @@ +error: weaveffi: a #[weaveffi::module] must have an inline body (`mod name { ... }`); the macro can't read a module from another file + --> tests/ui/fail_out_of_line_module.rs:6:5 + | +6 | #[weaveffi::module] + | ^ diff --git a/crates/weaveffi-macros/tests/ui/fail_sibling_c_enum.rs b/crates/weaveffi-macros/tests/ui/fail_out_of_tree_c_enum.rs similarity index 100% rename from crates/weaveffi-macros/tests/ui/fail_sibling_c_enum.rs rename to crates/weaveffi-macros/tests/ui/fail_out_of_tree_c_enum.rs diff --git a/crates/weaveffi-macros/tests/ui/fail_sibling_c_enum.stderr b/crates/weaveffi-macros/tests/ui/fail_out_of_tree_c_enum.stderr similarity index 72% rename from crates/weaveffi-macros/tests/ui/fail_sibling_c_enum.stderr rename to crates/weaveffi-macros/tests/ui/fail_out_of_tree_c_enum.stderr index 232375b6..d050a701 100644 --- a/crates/weaveffi-macros/tests/ui/fail_sibling_c_enum.stderr +++ b/crates/weaveffi-macros/tests/ui/fail_out_of_tree_c_enum.stderr @@ -1,18 +1,18 @@ -error[E0277]: `Color` is declared in a different `#[weaveffi::module]` tree and isn't a record or rich enum - --> tests/ui/fail_sibling_c_enum.rs:20:23 +error[E0277]: `Color` is not a WeaveFFI record or rich enum; declare C-style enums and interfaces in the module tree that uses them + --> tests/ui/fail_out_of_tree_c_enum.rs:20:23 | 20 | pub fn mix(color: Color) -> i32 { | ^^^^^ declared outside this module tree | help: the trait `ByValue` is not implemented for `Color` - --> tests/ui/fail_sibling_c_enum.rs:9:5 + --> tests/ui/fail_out_of_tree_c_enum.rs:9:5 | 9 | pub enum Color { | ^^^^^^^^^^^^^^ = note: a `#[weaveffi::module]` only sees the declarations inside its own tree, so it can pass a type from another tree only as a value buffer (a record or rich enum) = note: nest the modules under one `#[weaveffi::module]` root (as inner `mod`s) so the macro can see the declaration note: required by a bound in `__weaveffi_assert_by_value` - --> tests/ui/fail_sibling_c_enum.rs:20:23 + --> tests/ui/fail_out_of_tree_c_enum.rs:20:23 | 20 | pub fn mix(color: Color) -> i32 { | ^^^^^ required by this bound in `__weaveffi_assert_by_value` diff --git a/crates/weaveffi-macros/tests/ui/fail_result_without_error_domain.stderr b/crates/weaveffi-macros/tests/ui/fail_result_without_error_domain.stderr index bef87159..767ae24e 100644 --- a/crates/weaveffi-macros/tests/ui/fail_result_without_error_domain.stderr +++ b/crates/weaveffi-macros/tests/ui/fail_result_without_error_domain.stderr @@ -1,5 +1,5 @@ -error: weaveffi: `risky` returns a Result but no error domain is in scope; declare a #[weaveffi::error] enum in this module (or a parent module) - --> tests/ui/fail_result_without_error_domain.rs:8:20 +error: weaveffi: function 'bad::risky' declares throws but no error domain is in scope; declare a #[weaveffi::error] enum in this module or a parent module + --> tests/ui/fail_result_without_error_domain.rs:8:12 | 8 | pub fn risky() -> Result { - | ^^^^^^^^^^^^^^^^^^^^ + | ^^^^^ diff --git a/crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.rs b/crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.rs new file mode 100644 index 00000000..266075f8 --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.rs @@ -0,0 +1,11 @@ +//! `usize` varies with the platform, so it has no stable C ABI type. + +#[weaveffi::module] +mod bad { + #[weaveffi::export] + pub fn count(n: usize) -> i64 { + n as i64 + } +} + +fn main() {} diff --git a/crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.stderr b/crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.stderr new file mode 100644 index 00000000..e682bedf --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/fail_unsupported_primitive.stderr @@ -0,0 +1,5 @@ +error: weaveffi: unsupported primitive type 'usize'; use `u64` or `i64` for sizes and counts (their width is the same on every platform), or `String` for a `char` + --> tests/ui/fail_unsupported_primitive.rs:6:21 + | +6 | pub fn count(n: usize) -> i64 { + | ^^^^^ diff --git a/crates/weaveffi-macros/tests/ui/pass_aliases_and_cfg.rs b/crates/weaveffi-macros/tests/ui/pass_aliases_and_cfg.rs new file mode 100644 index 00000000..c671a7df --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/pass_aliases_and_cfg.rs @@ -0,0 +1,53 @@ +//! Type aliases in the tree are resolved by substitution, and an item's +//! `#[cfg]` applies to everything generated for it. + +#[weaveffi::module] +mod tree { + pub type Id = u64; + pub type Ids = Vec; + + #[weaveffi::export] + pub fn first(ids: Ids) -> Option { + ids.first().copied() + } + + #[cfg(any())] + #[weaveffi::export] + pub fn never(id: Id) -> Id { + id + } + + #[cfg(any())] + #[weaveffi::record] + pub struct Hidden { + pub id: Id, + } + + #[weaveffi::interface] + pub struct Store; + + #[cfg(all())] + impl Store { + pub fn new() -> Self { + Store + } + } + + #[cfg(any())] + impl Store { + pub fn missing(&self) -> Hidden { + Hidden { id: 0 } + } + } + + #[cfg(any())] + #[weaveffi::module] + pub mod gone { + #[weaveffi::export] + pub fn vanished() {} + } +} + +fn main() { + let _ = tree::Store::new(); +} diff --git a/crates/weaveffi-macros/tests/ui/pass_async_factory.rs b/crates/weaveffi-macros/tests/ui/pass_async_factory.rs new file mode 100644 index 00000000..c0fa95ed --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/pass_async_factory.rs @@ -0,0 +1,58 @@ +//! An `async fn` associated function returning the interface is an async +//! static factory (constructors are synchronous), in every return spelling. +#![deny(unsafe_code)] + +#[weaveffi::module] +mod store { + use std::sync::Arc; + + #[weaveffi::error] + #[repr(i32)] + pub enum StoreError { + /// Empty path + InvalidPath = 1, + } + + impl std::fmt::Display for StoreError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("invalid path") + } + } + + #[weaveffi::interface] + pub struct Store { + path: String, + } + + impl Store { + pub fn new(path: String) -> Self { + Self { path } + } + + pub async fn open_async(path: String) -> Result, StoreError> { + if path.is_empty() { + Err(StoreError::InvalidPath) + } else { + Ok(Arc::new(Self { path })) + } + } + + #[weaveffi::cancellable] + pub async fn open_later(path: String, cancel: weaveffi::CancelToken) -> Self { + let _ = cancel; + Self { path } + } + + pub async fn reopen(path: String) -> Store { + Self { path } + } + + pub fn path(&self) -> String { + self.path.clone() + } + } +} + +fn main() { + let _ = store::Store::new("x".into()).path(); +} diff --git a/crates/weaveffi-macros/tests/ui/pass_full_surface.rs b/crates/weaveffi-macros/tests/ui/pass_full_surface.rs index e91b9c4a..2977daaf 100644 --- a/crates/weaveffi-macros/tests/ui/pass_full_surface.rs +++ b/crates/weaveffi-macros/tests/ui/pass_full_surface.rs @@ -44,9 +44,9 @@ mod surface { #[weaveffi::callback_interface] pub trait Observer: Send + Sync { - fn on_item(&self, item: &Item, widget: Arc); - fn should_continue(&self, n: i32) -> bool; - fn pick(&self) -> Mode; + fn on_item(&self, item: &Item, widget: Arc) -> Result<(), weaveffi::ForeignError>; + fn should_continue(&self, n: i32) -> Result; + fn pick(&self) -> Result; fn label(&self, text: &str, data: &[u8]) -> Result; } @@ -85,17 +85,20 @@ mod surface { #[weaveffi::export] pub fn watch(observer: Arc, widget: Arc) -> bool { - observer.on_item( + let Ok(mode) = observer.pick() else { + return false; + }; + let _ = observer.on_item( &Item { id: 1, name: "x".into(), tags: vec![], - mode: observer.pick(), + mode, owner: Some(widget.clone()), }, widget, ); - observer.should_continue(1) + observer.should_continue(1).unwrap_or(false) } #[weaveffi::export] diff --git a/crates/weaveffi-macros/tests/ui/pass_rich_callbacks.rs b/crates/weaveffi-macros/tests/ui/pass_rich_callbacks.rs new file mode 100644 index 00000000..cc43734c --- /dev/null +++ b/crates/weaveffi-macros/tests/ui/pass_rich_callbacks.rs @@ -0,0 +1,68 @@ +//! Callback methods return any family, may throw the domain in scope, and a +//! callback parameter may be optional. + +use std::sync::Arc; + +#[weaveffi::module] +mod rich { + use std::sync::Arc; + + use weaveffi::ForeignError; + + #[weaveffi::error] + #[repr(i32)] + #[derive(Debug)] + pub enum LookupError { + Missing { key: String } = 1, + } + + impl std::fmt::Display for LookupError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("missing") + } + } + + #[weaveffi::record] + pub struct Card { + pub name: String, + } + + #[weaveffi::interface] + pub struct Token; + + impl Token { + pub fn new() -> Self { + Token + } + } + + #[weaveffi::callback_interface] + pub trait Source: Send + Sync { + fn name(&self) -> Result; + fn blob(&self) -> Result, ForeignError>; + fn card(&self) -> Result; + fn token(&self) -> Result, ForeignError>; + fn maybe(&self) -> Result>, ForeignError>; + #[weaveffi::throws] + fn lookup(&self, key: &str) -> Result; + } + + #[weaveffi::export] + pub fn read(source: Option>) -> Result { + match source { + Some(s) => s.name(), + None => Ok(String::new()), + } + } + + #[weaveffi::export] + pub fn missing_key(source: Arc) -> Option { + match source.lookup("k").err()?.domain::()? { + LookupError::Missing { key } => Some(key), + } + } +} + +fn main() { + let _ = Arc::new(rich::Token::new()); +} diff --git a/crates/weaveffi-model/Cargo.toml b/crates/weaveffi-model/Cargo.toml index b8647408..896be84d 100644 --- a/crates/weaveffi-model/Cargo.toml +++ b/crates/weaveffi-model/Cargo.toml @@ -6,7 +6,7 @@ rust-version.workspace = true license.workspace = true authors.workspace = true repository.workspace = true -description = "IR, parsers, Rust extractor, validation, and the resolved binding model for WeaveFFI API definitions" +description = "IR, parsers, Rust extractor, validation, and the validated model for WeaveFFI API definitions" keywords = ["ffi", "codegen", "bindings", "ir", "parser"] categories = ["development-tools::ffi"] @@ -14,22 +14,22 @@ categories = ["development-tools::ffi"] default = ["idl"] # The IDL text formats (YAML, JSON, TOML), JSON Schema derivation, and fancy # miette diagnostics. The proc-macros build without it. -idl = ["dep:serde_yaml", "dep:serde_json", "dep:toml", "dep:schemars", "dep:miette"] +idl = ["dep:serde_yaml", "dep:toml", "dep:schemars", "dep:miette"] [dependencies] serde = { workspace = true } thiserror = { workspace = true } heck = { workspace = true } syn = { workspace = true } +proc-macro2 = "1.0" serde_yaml = { workspace = true, optional = true } -serde_json = { workspace = true, optional = true } +serde_json = { workspace = true } toml = { workspace = true, optional = true } schemars = { workspace = true, optional = true } miette = { workspace = true, optional = true } [dev-dependencies] serde_yaml = { workspace = true } -serde_json = { workspace = true } toml = { workspace = true } [lints] diff --git a/crates/weaveffi-model/src/abi/ctype.rs b/crates/weaveffi-model/src/abi/ctype.rs index 7ae5c9df..f1273512 100644 --- a/crates/weaveffi-model/src/abi/ctype.rs +++ b/crates/weaveffi-model/src/abi/ctype.rs @@ -142,7 +142,7 @@ impl CType { CType::Size => "usize".to_string(), CType::Char => "c_char".to_string(), CType::Void => "std::ffi::c_void".to_string(), - // The runtime types come from `weaveffi-abi` and keep their fixed + // The runtime types come from `weaveffi::abi` and keep their fixed // names regardless of the configured business-symbol prefix. CType::CancelToken => "::weaveffi::abi::FfiCancelToken".to_string(), CType::Error => "::weaveffi::abi::FfiError".to_string(), diff --git a/crates/weaveffi-model/src/abi/lower.rs b/crates/weaveffi-model/src/abi/lower.rs index a3e30153..3b024e18 100644 --- a/crates/weaveffi-model/src/abi/lower.rs +++ b/crates/weaveffi-model/src/abi/lower.rs @@ -16,9 +16,16 @@ //! * [`Family::Object`] is an interface pointer, borrowed as a parameter and //! one strong reference as a return; a nullable one is `Interface?`. //! * [`Family::Callback`] is a `void* ctx` plus `const {tag}_vtable*` pair, -//! only ever a parameter. +//! only ever a parameter; a nullable one (`Cb?`) passes a null vtable for +//! none. +//! +//! A callback-interface method's return lowers differently from a producer +//! return ([`lower_callback_return`]): the consumer produces the value, so a +//! string, bytes, or buffer return is a run the consumer allocates with +//! `{prefix}_alloc` and hands back through `uint8_t** out_ptr` and +//! `size_t* out_len`. -use crate::model::{Family, Ty}; +use crate::ty::{Family, Prim, Ty, TypeIndex}; use super::ctype::{CType, ConstPos}; @@ -52,57 +59,59 @@ pub struct AbiReturn { pub out_params: Vec, } -/// Split a (possibly qualified) type reference into its C module-path segment -/// and bare type name. +/// The underscore-joined C path of the module declaring the user type +/// `name`. /// -/// Qualified references use dot-separated module paths (`a.b.Name`); the C ABI -/// flattens those to underscore-joined symbol prefixes (`a_b`). An unqualified -/// reference belongs to `current_module`. Using `rsplit_once` (rather than -/// `split_once`) is what makes *multi-level* nesting work: only the final -/// segment is the type name, everything before it is the module path. +/// # Panics /// -/// * `a.b.Name`, current `x` -> (`a_b`, `Name`) -/// * `shared.Status`, current `orders` -> (`shared`, `Status`) -/// * `Name`, current `a_b` -> (`a_b`, `Name`) -pub fn split_qualified(name: &str, current_module: &str) -> (String, String) { - match name.rsplit_once('.') { - Some((module_path, type_name)) => (module_path.replace('.', "_"), type_name.to_string()), - None => (current_module.to_string(), name.to_string()), - } +/// Panics when `name` isn't declared. Only records and rich enums may be +/// undeclared (the proc-macro assumes a name from a sibling module tree is +/// one of those), and they cross as value buffers, which never name a C +/// type. +fn owner_path(name: &str, types: &TypeIndex) -> String { + types + .module_path(name) + .unwrap_or_else(|| panic!("type '{name}' is not declared")) + .to_string() } -/// Resolve an interface reference (possibly `module.Name`) to its opaque C -/// tag type. -pub fn struct_tag(name: &str, current_module: &str) -> CType { - let (module, name) = split_qualified(name, current_module); - CType::StructTag { module, name } +/// The opaque C tag type of the interface `name`. +fn struct_tag(name: &str, types: &TypeIndex) -> CType { + CType::StructTag { + module: owner_path(name, types), + name: name.to_string(), + } } -/// Resolve a callback interface reference (possibly `module.Name`) to its -/// vtable struct type. -pub fn vtable_tag(name: &str, current_module: &str) -> CType { - let (module, name) = split_qualified(name, current_module); - CType::VtableTag { module, name } +/// The vtable struct type of the callback interface `name`. +fn vtable_tag(name: &str, types: &TypeIndex) -> CType { + CType::VtableTag { + module: owner_path(name, types), + name: name.to_string(), + } } /// The by-value C type of a [`Family::Direct`] type. -fn direct_ctype(ty: &Ty, module: &str) -> CType { +fn direct_ctype(ty: &Ty, types: &TypeIndex) -> CType { match ty { - Ty::I8 => CType::Int8, - Ty::I16 => CType::Int16, - Ty::I32 => CType::Int32, - Ty::I64 => CType::Int64, - Ty::U8 => CType::Uint8, - Ty::U16 => CType::Uint16, - Ty::U32 => CType::Uint32, - Ty::U64 => CType::Uint64, - Ty::F32 => CType::Float, - Ty::F64 => CType::Double, - Ty::Bool => CType::Bool, - Ty::Enum(e) => { - let (module, name) = split_qualified(e, module); - CType::Enum { module, name } - } + Ty::Prim(p) => match p { + Prim::I8 => CType::Int8, + Prim::I16 => CType::Int16, + Prim::I32 => CType::Int32, + Prim::I64 => CType::Int64, + Prim::U8 => CType::Uint8, + Prim::U16 => CType::Uint16, + Prim::U32 => CType::Uint32, + Prim::U64 => CType::Uint64, + Prim::F32 => CType::Float, + Prim::F64 => CType::Double, + Prim::Bool => CType::Bool, + Prim::String | Prim::Bytes => unreachable!("{p} is not a direct type"), + }, + Ty::Enum(e) => CType::Enum { + module: owner_path(e, types), + name: e.clone(), + }, other => unreachable!("{other} is not a direct type"), } } @@ -121,9 +130,9 @@ fn ptr_len_slots(name: &str) -> Vec { /// # Panics /// /// Panics on an iterator type, which validation never admits as a parameter. -pub fn lower_param(name: &str, ty: &Ty, module: &str) -> Vec { +pub fn lower_param(name: &str, ty: &Ty, types: &TypeIndex) -> Vec { match ty.family() { - Family::Direct => vec![AbiParam::new(name, direct_ctype(ty, module))], + Family::Direct => vec![AbiParam::new(name, direct_ctype(ty, types))], Family::String | Family::Bytes | Family::Buffer => ptr_len_slots(name), // An interface parameter borrows the object for the call: the callee // reads through the const pointer and clones if it wants to retain @@ -136,13 +145,14 @@ pub fn lower_param(name: &str, ty: &Ty, module: &str) -> Vec { name, CType::Ptr { konst: ConstPos::West, - pointee: Box::new(struct_tag(iface, module)), + pointee: Box::new(struct_tag(iface, types)), }, )] } // A callback interface is an opaque consumer context plus the - // consumer's static vtable for the interface. - Family::Callback => { + // consumer's static vtable for the interface. A nullable one passes a + // null vtable for none. + Family::Callback { .. } => { let cb = ty .callback_interface_name() .expect("callback family names a callback interface"); @@ -150,7 +160,7 @@ pub fn lower_param(name: &str, ty: &Ty, module: &str) -> Vec { AbiParam::new(format!("{name}_ctx"), CType::ptr(CType::Void)), AbiParam::new( format!("{name}_vtable"), - CType::const_ptr(vtable_tag(cb, module)), + CType::const_ptr(vtable_tag(cb, types)), ), ] } @@ -165,13 +175,13 @@ pub fn lower_param(name: &str, ty: &Ty, module: &str) -> Vec { /// Panics on an iterator type, whose launcher is lowered by the function /// lowering in [`crate::model`] rather than as a plain value return, and on a /// callback interface, which validation never admits as a return. -pub fn lower_return(ty: &Ty, module: &str) -> AbiReturn { +pub fn lower_return(ty: &Ty, types: &TypeIndex) -> AbiReturn { let no_out = |ret| AbiReturn { ret, out_params: vec![], }; match ty.family() { - Family::Direct => no_out(direct_ctype(ty, module)), + Family::Direct => no_out(direct_ctype(ty, types)), // String, bytes, and buffered returns are producer-allocated byte // runs: the caller copies or decodes them and then calls // `{prefix}_free_bytes(ptr, len)`. @@ -185,34 +195,64 @@ pub fn lower_return(ty: &Ty, module: &str) -> AbiReturn { let iface = ty .interface_name() .expect("object family names an interface"); - no_out(CType::ptr(struct_tag(iface, module))) + no_out(CType::ptr(struct_tag(iface, types))) } - Family::Callback => unreachable!("callback interfaces are never returned"), + Family::Callback { .. } => unreachable!("callback interfaces are never returned"), Family::Iterator => { unreachable!("iterator return handled specially by the function lowering") } } } +/// Lower the return type of a callback-interface method to its C return type +/// plus trailing out-parameters (placed before the method's `out_err`). +/// +/// A direct value is returned by value and an object as one strong +/// reference (`{tag}*`) the producer adopts. A string, bytes, or buffer +/// return is `void` with `uint8_t** out_ptr` and `size_t* out_len` slots: +/// the consumer allocates the run with `{prefix}_alloc`, writes it there, +/// and the producer adopts and frees it. +/// +/// # Panics +/// +/// Panics on an iterator or a callback interface, which validation never +/// admits as a callback method return. +pub fn lower_callback_return(ty: &Ty, types: &TypeIndex) -> AbiReturn { + match ty.family() { + Family::String | Family::Bytes | Family::Buffer => AbiReturn { + ret: CType::Void, + out_params: vec![ + AbiParam::new("out_ptr", CType::ptr(CType::ptr(CType::Uint8))), + AbiParam::new("out_len", CType::ptr(CType::Size)), + ], + }, + Family::Callback { .. } | Family::Iterator => { + unreachable!("{ty} is never a callback method return") + } + _ => lower_return(ty, types), + } +} + /// The trailing result fields appended to an async callback after the /// `(context, err)` prefix. /// /// String, bytes, and buffered results are passed as an owned `ptr` + `len` /// pair (released by the consumer with `{prefix}_free_bytes`); everything /// else reuses its return slot type by value. -pub fn callback_result_params(ty: &Ty, module: &str) -> Vec { +pub fn callback_result_params(ty: &Ty, types: &TypeIndex) -> Vec { match ty.family() { Family::String | Family::Bytes | Family::Buffer => vec![ AbiParam::new("result_ptr", CType::const_ptr(CType::Uint8)), AbiParam::new("result_len", CType::Size), ], - _ => vec![AbiParam::new("result", lower_return(ty, module).ret)], + _ => vec![AbiParam::new("result", lower_return(ty, types).ret)], } } #[cfg(test)] mod tests { use super::*; + use crate::ty::{TypeDecl, TypeKind}; fn render(params: &[AbiParam]) -> Vec { params @@ -221,50 +261,68 @@ mod tests { .collect() } + /// `Store` in `kv`, `Status` in `shared`, `Listener` in `events`. + fn types() -> TypeIndex { + let mut types = TypeIndex::default(); + for (module, name, kind) in [ + ("kv", "Store", TypeKind::Interface), + ("shared", "Status", TypeKind::Enum), + ("events", "Listener", TypeKind::CallbackInterface), + ] { + let module = types.push_module(module.into()); + types.declare( + name, + TypeDecl { + kind, + module, + index: 0, + }, + ); + } + types + } + + const I32: Ty = Ty::Prim(Prim::I32); + #[test] fn params_lower_by_family() { - assert_eq!(render(&lower_param("x", &Ty::I32, "m")), ["int32_t x"]); + let t = types(); + assert_eq!(render(&lower_param("x", &I32, &t)), ["int32_t x"]); assert_eq!( - render(&lower_param("s", &Ty::StringUtf8, "m")), + render(&lower_param("s", &Ty::Prim(Prim::String), &t)), ["const uint8_t* s_ptr", "size_t s_len"] ); assert_eq!( - render(&lower_param("data", &Ty::Bytes, "m")), + render(&lower_param("data", &Ty::Prim(Prim::Bytes), &t)), ["const uint8_t* data_ptr", "size_t data_len"] ); assert_eq!( - render(&lower_param("xs", &Ty::List(Box::new(Ty::I32)), "m")), + render(&lower_param("xs", &Ty::List(Box::new(I32)), &t)), ["const uint8_t* xs_ptr", "size_t xs_len"] ); + // An undeclared record (a proc-macro's sibling-tree type) is fine: + // value buffers never name a C type. assert_eq!( - render(&lower_param( - "c", - &Ty::Record("other.Contact".into()), - "ops" - )), + render(&lower_param("c", &Ty::Record("Contact".into()), &t)), ["const uint8_t* c_ptr", "size_t c_len"] ); assert_eq!( render(&lower_param( "s", &Ty::Optional(Box::new(Ty::Interface("Store".into()))), - "kv" + &t )), ["const weaveffi_kv_Store* s"] ); assert_eq!( - render(&lower_param( - "s", - &Ty::Enum("shared.Status".into()), - "orders" - )), + render(&lower_param("s", &Ty::Enum("Status".into()), &t)), ["weaveffi_shared_Status s"] ); assert_eq!( render(&lower_param( "listener", - &Ty::CallbackInterface("events.Listener".into()), - "kv" + &Ty::CallbackInterface("Listener".into()), + &t )), [ "void* listener_ctx", @@ -275,7 +333,8 @@ mod tests { #[test] fn returns_lower_by_family() { - let r = lower_return(&Ty::Bytes, "m"); + let t = types(); + let r = lower_return(&Ty::Prim(Prim::Bytes), &t); assert_eq!(r.ret.render_c("weaveffi"), "const uint8_t*"); assert_eq!(render(&r.out_params), ["size_t* out_len"]); for ty in [ @@ -283,52 +342,67 @@ mod tests { Ty::RichEnum("Shape".into()), Ty::List(Box::new(Ty::Record("Contact".into()))), Ty::List(Box::new(Ty::Interface("Store".into()))), - Ty::Map(Box::new(Ty::StringUtf8), Box::new(Ty::I32)), - Ty::Optional(Box::new(Ty::I64)), + Ty::Map(Box::new(Ty::Prim(Prim::String)), Box::new(I32)), + Ty::Optional(Box::new(Ty::Prim(Prim::I64))), ] { - let r = lower_return(&ty, "m"); + let r = lower_return(&ty, &t); assert_eq!(r.ret.render_c("weaveffi"), "const uint8_t*", "{ty}"); assert_eq!(render(&r.out_params), ["size_t* out_len"], "{ty}"); } - let r = lower_return(&Ty::Optional(Box::new(Ty::Interface("Store".into()))), "kv"); + let r = lower_return(&Ty::Optional(Box::new(Ty::Interface("Store".into()))), &t); assert_eq!(r.ret.render_c("weaveffi"), "weaveffi_kv_Store*"); assert!(r.out_params.is_empty()); assert_eq!( - lower_return(&Ty::Enum("shared.Status".into()), "orders") + lower_return(&Ty::Enum("Status".into()), &t) .ret .render_c("weaveffi"), "weaveffi_shared_Status" ); } + #[test] + fn callback_method_returns_use_consumer_allocated_runs() { + let t = types(); + let r = lower_callback_return(&Ty::Prim(Prim::String), &t); + assert_eq!(r.ret, CType::Void); + assert_eq!( + render(&r.out_params), + ["uint8_t** out_ptr", "size_t* out_len"] + ); + let r = lower_callback_return(&Ty::Interface("Store".into()), &t); + assert_eq!(r.ret.render_c("weaveffi"), "weaveffi_kv_Store*"); + assert!(r.out_params.is_empty()); + assert_eq!(lower_callback_return(&I32, &t).ret, CType::Int32); + assert_eq!( + render(&lower_param( + "l", + &Ty::Optional(Box::new(Ty::CallbackInterface("Listener".into()))), + &t + )), + [ + "void* l_ctx", + "const weaveffi_events_Listener_vtable* l_vtable" + ] + ); + } + #[test] fn callback_results() { + let t = types(); assert_eq!( render(&callback_result_params( - &Ty::List(Box::new(Ty::StringUtf8)), - "m" + &Ty::List(Box::new(Ty::Prim(Prim::String))), + &t )), ["const uint8_t* result_ptr", "size_t result_len"] ); assert_eq!( - render(&callback_result_params(&Ty::Bytes, "m")), + render(&callback_result_params(&Ty::Prim(Prim::Bytes), &t)), ["const uint8_t* result_ptr", "size_t result_len"] ); assert_eq!( - render(&callback_result_params(&Ty::I32, "m")), + render(&callback_result_params(&I32, &t)), ["int32_t result"] ); } - - #[test] - fn split_qualified_handles_levels() { - assert_eq!( - split_qualified("Name", "current"), - ("current".to_string(), "Name".to_string()) - ); - assert_eq!( - split_qualified("a.b.c.Name", "x"), - ("a_b_c".to_string(), "Name".to_string()) - ); - } } diff --git a/crates/weaveffi-model/src/abi/mod.rs b/crates/weaveffi-model/src/abi/mod.rs index 3af9d0dc..7e5aa658 100644 --- a/crates/weaveffi-model/src/abi/mod.rs +++ b/crates/weaveffi-model/src/abi/mod.rs @@ -1,38 +1,24 @@ -//! The canonical WeaveFFI C ABI model. +//! The canonical WeaveFFI C ABI lowering. //! -//! This module is the single source of truth for *how a resolved API lowers -//! onto the stable C ABI*: which symbols exist, the exact ordered parameter -//! list of each, and how every [`Ty`] crosses the boundary (by value, as a +//! This module is the single source of truth for *how a resolved type lowers +//! onto the stable C ABI*: the ordered C slots of every parameter and return +//! and how every [`Ty`](crate::ty::Ty) crosses the boundary (by value, as a //! pointer, as `ptr`+`len`, as a serialized value buffer, with a trailing -//! `out_err`, and so on). +//! `out_err`, and so on). The [`Model`](crate::model::Model) assembles these +//! slots into each symbol's full signature once. //! -//! Every language generator and the producer macro share [`lower_param`], -//! [`lower_return`], [`callback_result_params`], and the signature assembly -//! helpers below, and map the resulting [`CType`] onto their own FFI -//! vocabulary. The C rendering ([`CType::render_c`]) is the canonical one. +//! Every language generator and the producer macro read the resulting +//! [`CType`]s from the model and map them onto their own FFI vocabulary. The +//! C rendering ([`CType::render_c`]) is the canonical one. pub mod ctype; pub mod lower; pub use ctype::{CType, ConstPos}; pub use lower::{ - callback_result_params, lower_param, lower_return, split_qualified, struct_tag, vtable_tag, - AbiParam, AbiReturn, + callback_result_params, lower_callback_return, lower_param, lower_return, AbiParam, AbiReturn, }; -use crate::model::{Family, ParamBinding, Ty}; - -/// A fully-assembled C ABI signature: the ordered parameter slots and the -/// C return type. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct AbiSig { - /// Ordered C parameter slots, including any trailing out-parameters and the - /// final `out_err`. - pub params: Vec, - /// The C return type. - pub ret: CType, -} - /// The trailing `out_err` parameter every fallible WeaveFFI symbol carries. pub fn error_out_param() -> AbiParam { AbiParam::new("out_err", CType::ptr(CType::Error)) @@ -49,164 +35,7 @@ pub fn ctx_param() -> AbiParam { AbiParam::new("ctx", CType::ptr(CType::Void)) } -/// Assemble the signature of one callback-interface method as it appears in -/// the vtable: `ctx`, then every parameter's slots, then `out_err`. The return -/// is the method's direct C type or `void`; validation restricts callback -/// method returns to the direct family, so no out-parameters ever appear. -/// -/// Object parameters differ from a plain call: the producer transfers one -/// strong reference the consumer adopts, so the slot is a mutable `{tag}*` -/// rather than the borrowed `const {tag}*` of a top-level parameter. -pub fn callback_method_signature( - params: &[ParamBinding], - returns: Option<&Ty>, - module: &str, -) -> AbiSig { - let mut out = vec![ctx_param()]; - for p in params { - if matches!(p.ty.family(), Family::Object { .. }) { - let [slot] = p.abi.as_slice() else { - unreachable!("object parameter '{}' has one slot", p.name); - }; - let CType::Ptr { pointee, .. } = &slot.ty else { - unreachable!("object slot '{}' is a pointer", p.name); - }; - out.push(AbiParam::new(&slot.name, CType::ptr((**pointee).clone()))); - } else { - out.extend(p.abi.iter().cloned()); - } - } - let ret = match returns { - Some(ty) => { - let r = lower_return(ty, module); - debug_assert!( - r.out_params.is_empty(), - "callback method returns are direct-family only" - ); - r.ret - } - None => CType::Void, - }; - out.push(error_out_param()); - AbiSig { params: out, ret } -} - /// The optional `{prefix}_cancel_token*` parameter of a cancellable async call. pub fn cancel_token_param() -> AbiParam { AbiParam::new("cancel_token", CType::ptr(CType::CancelToken)) } - -/// Assemble the full C signature of a *synchronous* function: every input -/// parameter's slots, then the return type's out-parameters, then `out_err`. -pub fn sync_signature(params: &[ParamBinding], returns: Option<&Ty>, module: &str) -> AbiSig { - let mut out: Vec = params.iter().flat_map(|p| p.abi.iter().cloned()).collect(); - let ret = match returns { - Some(ty) => { - let r = lower_return(ty, module); - out.extend(r.out_params); - r.ret - } - None => CType::Void, - }; - out.push(error_out_param()); - AbiSig { params: out, ret } -} - -/// Assemble the parameters of the async completion callback typedef: -/// `(void* context, {prefix}_error* err, )`. -pub fn async_callback_params(returns: Option<&Ty>, module: &str) -> Vec { - let mut params = vec![ - context_param(), - AbiParam::new("err", CType::ptr(CType::Error)), - ]; - if let Some(ret) = returns { - params.extend(callback_result_params(ret, module)); - } - params -} - -/// Assemble the input parameters of an async launcher function (excluding the -/// trailing `callback` and `context`, which are appended by the caller because -/// the callback's C type name is generator-derived). `cancellable` inserts the -/// cancel-token slot in the canonical position. -pub fn async_input_params(params: &[ParamBinding], cancellable: bool) -> Vec { - let mut out: Vec = params.iter().flat_map(|p| p.abi.iter().cloned()).collect(); - if cancellable { - out.push(cancel_token_param()); - } - out -} - -#[cfg(test)] -mod tests { - use super::*; - - fn param(name: &str, ty: Ty) -> ParamBinding { - ParamBinding::new(name, ty, None, "math") - } - - fn render(params: &[AbiParam]) -> Vec { - params - .iter() - .map(|p| format!("{} {}", p.ty.render_c("weaveffi"), p.name)) - .collect() - } - - #[test] - fn sync_signature_appends_out_err_last() { - let sig = sync_signature( - &[param("a", Ty::I32), param("b", Ty::I32)], - Some(&Ty::I32), - "math", - ); - assert_eq!(sig.ret, CType::Int32); - assert_eq!( - render(&sig.params), - ["int32_t a", "int32_t b", "weaveffi_error* out_err"] - ); - let sig = sync_signature(&[], Some(&Ty::Bytes), "m"); - assert_eq!(sig.ret.render_c("weaveffi"), "const uint8_t*"); - assert_eq!( - render(&sig.params), - ["size_t* out_len", "weaveffi_error* out_err"] - ); - let sig = sync_signature(&[param("x", Ty::I32)], None, "m"); - assert_eq!(sig.ret, CType::Void); - } - - #[test] - fn async_shapes() { - let params = async_callback_params(Some(&Ty::I32), "m"); - assert_eq!( - render(¶ms), - ["void* context", "weaveffi_error* err", "int32_t result"] - ); - let inputs = async_input_params(&[param("id", Ty::I64)], true); - assert_eq!( - render(&inputs), - ["int64_t id", "weaveffi_cancel_token* cancel_token"] - ); - } - - #[test] - fn callback_method_signature_wraps_ctx_and_out_err() { - let sig = - callback_method_signature(&[param("text", Ty::StringUtf8)], Some(&Ty::Bool), "events"); - assert_eq!(sig.ret, CType::Bool); - assert_eq!( - render(&sig.params), - [ - "void* ctx", - "const uint8_t* text_ptr", - "size_t text_len", - "weaveffi_error* out_err" - ] - ); - let sig = callback_method_signature(&[], None, "events"); - assert_eq!(sig.ret, CType::Void); - assert_eq!( - render(&sig.params), - ["void* ctx", "weaveffi_error* out_err"] - ); - } -} diff --git a/crates/weaveffi-model/src/checksum.rs b/crates/weaveffi-model/src/checksum.rs deleted file mode 100644 index ac06c4c2..00000000 --- a/crates/weaveffi-model/src/checksum.rs +++ /dev/null @@ -1,215 +0,0 @@ -//! Contract checksums: a stable fingerprint of a top-level module's ABI. -//! -//! A producer exports `uint64_t {prefix}_{module}_checksum(void)` for every -//! top-level module, and every generated consumer compares it with the value -//! it was generated against when it loads the library. Both sides compute the -//! value with [`module_checksum`] over the same IR (the `#[weaveffi::module]` -//! macro on the module it expands, the CLI on the parsed IDL or the extracted -//! Rust source), so bindings that are stale relative to the library they load -//! fail loudly instead of misreading memory. -//! -//! The hash covers everything that shapes the C ABI or the value-buffer wire -//! format: every declaration name, every type reference, parameter and field -//! order, function flags (`throws`, `async`, `cancellable`), enum -//! discriminants, and error-code values. It deliberately skips documentation, -//! deprecation notes, and error-code messages, so editing prose never breaks a -//! deployed binding. - -use crate::ir::{ - CallbackInterfaceDef, EnumDef, ErrorDomain, Function, InterfaceDef, Module, StructDef, - StructField, -}; - -/// FNV-1a 64-bit hasher; tiny, dependency-free, and stable across platforms -/// and Rust versions. -struct Fnv(u64); - -impl Fnv { - const OFFSET: u64 = 0xcbf2_9ce4_8422_2325; - const PRIME: u64 = 0x0000_0100_0000_01b3; - - fn bytes(&mut self, bytes: &[u8]) { - for b in bytes { - self.0 ^= u64::from(*b); - self.0 = self.0.wrapping_mul(Self::PRIME); - } - } - - /// Hash a tagged string. The tag and a terminator keep adjacent fields - /// from running together (`"ab" + "c"` never equals `"a" + "bc"`). - fn str(&mut self, tag: u8, s: &str) { - self.bytes(&[tag]); - self.bytes(s.as_bytes()); - self.bytes(&[0xff]); - } - - fn int(&mut self, tag: u8, v: i64) { - self.bytes(&[tag]); - self.bytes(&v.to_le_bytes()); - } - - fn flag(&mut self, tag: u8, v: bool) { - self.bytes(&[tag, u8::from(v)]); - } -} - -/// The contract checksum of one top-level module (including its nested -/// modules). -#[must_use] -pub fn module_checksum(module: &Module) -> u64 { - let mut h = Fnv(Fnv::OFFSET); - hash_module(&mut h, module); - h.0 -} - -fn hash_module(h: &mut Fnv, m: &Module) { - h.str(b'M', &m.name); - for f in &m.functions { - hash_function(h, b'F', f); - } - for i in &m.interfaces { - hash_interface(h, i); - } - for c in &m.callback_interfaces { - hash_callback(h, c); - } - for s in &m.structs { - hash_struct(h, s); - } - for e in &m.enums { - hash_enum(h, e); - } - if let Some(d) = &m.errors { - hash_errors(h, d); - } - for child in &m.modules { - hash_module(h, child); - } - h.bytes(b"}"); -} - -fn hash_function(h: &mut Fnv, tag: u8, f: &Function) { - h.str(tag, &f.name); - for p in &f.params { - h.str(b'p', &p.name); - h.str(b't', &p.ty.to_string()); - } - if let Some(r) = &f.returns { - h.str(b'r', &r.to_string()); - } - h.flag(b'T', f.throws); - h.flag(b'A', f.r#async); - h.flag(b'C', f.cancellable); -} - -fn hash_interface(h: &mut Fnv, i: &InterfaceDef) { - h.str(b'I', &i.name); - for c in &i.constructors { - hash_function(h, b'c', c); - } - for m in &i.methods { - hash_function(h, b'm', m); - } - for s in &i.statics { - hash_function(h, b's', s); - } -} - -fn hash_callback(h: &mut Fnv, c: &CallbackInterfaceDef) { - h.str(b'K', &c.name); - for m in &c.methods { - hash_function(h, b'k', m); - } -} - -fn hash_fields(h: &mut Fnv, fields: &[StructField]) { - for f in fields { - h.str(b'f', &f.name); - h.str(b't', &f.ty.to_string()); - } -} - -fn hash_struct(h: &mut Fnv, s: &StructDef) { - h.str(b'S', &s.name); - hash_fields(h, &s.fields); -} - -fn hash_enum(h: &mut Fnv, e: &EnumDef) { - h.str(b'E', &e.name); - for v in &e.variants { - h.str(b'v', &v.name); - h.int(b'd', i64::from(v.value)); - hash_fields(h, &v.fields); - } -} - -fn hash_errors(h: &mut Fnv, d: &ErrorDomain) { - h.str(b'D', &d.name); - for c in &d.codes { - h.str(b'e', &c.name); - h.int(b'n', i64::from(c.code)); - hash_fields(h, &c.fields); - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::ir::{Param, TypeRef}; - - fn module() -> Module { - Module { - name: "kv".into(), - doc: Some("docs".into()), - functions: vec![Function { - name: "get".into(), - params: vec![Param { - name: "key".into(), - ty: TypeRef::StringUtf8, - doc: None, - }], - returns: Some(TypeRef::Optional(Box::new(TypeRef::Bytes))), - doc: Some("Fetch a value.".into()), - throws: true, - r#async: false, - cancellable: false, - deprecated: None, - }], - interfaces: vec![], - callback_interfaces: vec![], - structs: vec![], - enums: vec![], - errors: None, - modules: vec![], - } - } - - #[test] - fn docs_do_not_affect_the_checksum() { - let a = module(); - let mut b = module(); - b.doc = None; - b.functions[0].doc = Some("Changed prose.".into()); - b.functions[0].deprecated = Some("old".into()); - assert_eq!(module_checksum(&a), module_checksum(&b)); - } - - #[test] - fn abi_changes_do() { - let a = module(); - let mut b = module(); - b.functions[0].throws = false; - assert_ne!(module_checksum(&a), module_checksum(&b)); - let mut c = module(); - c.functions[0].params[0].ty = TypeRef::Bytes; - assert_ne!(module_checksum(&a), module_checksum(&c)); - } - - #[test] - fn value_is_stable() { - // Pin the algorithm: changing it invalidates every deployed binding, - // so it must be a deliberate, documented ABI change. - assert_eq!(module_checksum(&module()), module_checksum(&module())); - assert_eq!(format!("{:016x}", Fnv(Fnv::OFFSET).0), "cbf29ce484222325"); - } -} diff --git a/crates/weaveffi-model/src/contract.rs b/crates/weaveffi-model/src/contract.rs new file mode 100644 index 00000000..b27ec6b8 --- /dev/null +++ b/crates/weaveffi-model/src/contract.rs @@ -0,0 +1,347 @@ +//! Contract tables: a per-declaration fingerprint of each top-level module's +//! ABI. +//! +//! A producer exports, for every top-level module `m`, +//! `const {prefix}_contract_entry* {prefix}_{m}_contract(size_t* out_len)`: +//! a static table with one [`ContractEntry`] per declaration in `m` and its +//! submodules, sorted ascending by id. Every generated consumer checks, when +//! it loads the library, that each entry it was generated with is present in +//! the producer's table with an equal hash. Extra producer entries are fine, +//! so adding a declaration never breaks a deployed binding, while removing +//! or changing one is reported by its dotted path. +//! +//! [`entries`] is the one function that computes a table. The +//! `#[weaveffi::module]` macro, the CLI, and the C header generator all call +//! it on the validated [`Model`], so the two sides agree by construction. +//! +//! * The **id** is [`fnv1a64`] of the declaration's dotted path +//! (`kv.Store.put`, `kv.KvError`, `kv.stats.Summary`). +//! * The **hash** is [`fnv1a64`] of a canonical signature string computed +//! from the resolved model, never from source spelling: type aliases are +//! already substituted, `[u8]` is already `bytes`, and every type is its +//! resolved IDL spelling (see [`Ty`]'s `Display`). Docs, deprecation text, +//! and error messages are excluded, and so is the order of sibling +//! declarations, because each entry is independent. +//! +//! The canonical strings are: +//! +//! | Declaration | Signature | +//! |---|---| +//! | function or interface member | `{kind} {name}({param}: {type}, ...) -> {type or void}`, then ` throws`, ` async`, ` cancellable` when set; `kind` is `function`, `constructor`, `method`, or `static` | +//! | record | `record {name} {{field}: {type}, ...}` | +//! | enum | `enum {name} {{Variant} = {value}, ...}`, a rich variant followed by ` {{field}: {type}, ...}` | +//! | callback interface | `callback {name} {{method}({param}: {type}, ...) -> {type or void}; ...}`, a method followed by ` throws` when set | +//! | error domain | `errors {name} {{Code} = {value}, ...}`, a code with fields followed by ` {{field}: {type}, ...}` | +//! | interface | `interface {name}` (its members have their own entries) | + +use std::fmt::Write; + +use crate::model::{FieldBinding, FnBinding, Model, ModuleBinding, ParamBinding}; +use crate::ty::Ty; + +/// One declaration's entry in a module's contract table. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ContractEntry { + /// The declaration's dotted path (`kv.Store.put`), which consumers name + /// in their load errors. + pub path: String, + /// [`fnv1a64`] of [`path`](Self::path). + pub id: u64, + /// [`fnv1a64`] of the declaration's canonical signature string. + pub hash: u64, +} + +/// The 64-bit FNV-1a hash of `data`: tiny, dependency-free, and stable +/// across platforms and Rust versions. +#[must_use] +pub const fn fnv1a64(data: &[u8]) -> u64 { + let mut hash: u64 = 0xcbf2_9ce4_8422_2325; + let mut i = 0; + while i < data.len() { + hash ^= data[i] as u64; + hash = hash.wrapping_mul(0x0000_0100_0000_01b3); + i += 1; + } + hash +} + +/// The contract table of the top-level module `root` (which must be one of +/// `model`'s [roots](Model::roots)): one entry per declaration in `root` and +/// its submodules, sorted ascending by id. +#[must_use] +pub fn entries(model: &Model, root: &ModuleBinding) -> Vec { + let mut out = Vec::new(); + for m in model + .modules + .iter() + .filter(|m| m.segments.first() == root.segments.first()) + { + module_entries(m, &mut out); + } + out.sort_by_key(|e| e.id); + out +} + +fn entry(path: String, signature: &str) -> ContractEntry { + ContractEntry { + id: fnv1a64(path.as_bytes()), + hash: fnv1a64(signature.as_bytes()), + path, + } +} + +fn module_entries(m: &ModuleBinding, out: &mut Vec) { + let dot = &m.dot_path; + for f in &m.functions { + out.push(entry( + format!("{dot}.{}", f.name), + &callable_signature("function", f), + )); + } + for i in &m.interfaces { + let owner = format!("{dot}.{}", i.name); + out.push(entry(owner.clone(), &format!("interface {}", i.name))); + for (kind, members) in [ + ("constructor", &i.constructors), + ("method", &i.methods), + ("static", &i.statics), + ] { + for f in members { + out.push(entry( + format!("{owner}.{}", f.name), + &callable_signature(kind, f), + )); + } + } + } + for s in &m.structs { + out.push(entry( + format!("{dot}.{}", s.name), + &format!("record {} {{{}}}", s.name, fields(&s.fields)), + )); + } + for e in &m.enums { + let variants: Vec = e + .variants + .iter() + .map(|v| case(&v.name, v.value, &v.fields)) + .collect(); + out.push(entry( + format!("{dot}.{}", e.name), + &format!("enum {} {{{}}}", e.name, variants.join(", ")), + )); + } + for c in &m.callback_interfaces { + let methods: Vec = c + .methods + .iter() + .map(|cm| { + let mut sig = format!( + "{}({}) -> {}", + cm.name, + params(&cm.params), + ret(cm.ret.as_ref()) + ); + if cm.throws { + sig.push_str(" throws"); + } + sig + }) + .collect(); + out.push(entry( + format!("{dot}.{}", c.name), + &format!("callback {} {{{}}}", c.name, methods.join("; ")), + )); + } + if let Some(d) = &m.errors { + let codes: Vec = d + .codes + .iter() + .map(|c| case(&c.name, c.value, &c.fields)) + .collect(); + out.push(entry( + format!("{dot}.{}", d.name), + &format!("errors {} {{{}}}", d.name, codes.join(", ")), + )); + } +} + +fn params(params: &[ParamBinding]) -> String { + let all: Vec = params + .iter() + .map(|p| format!("{}: {}", p.name, p.ty)) + .collect(); + all.join(", ") +} + +fn fields(fields: &[FieldBinding]) -> String { + let all: Vec = fields + .iter() + .map(|f| format!("{}: {}", f.name, f.ty)) + .collect(); + all.join(", ") +} + +fn ret(ty: Option<&Ty>) -> String { + ty.map_or_else(|| "void".to_string(), ToString::to_string) +} + +/// `Name = value`, followed by ` {fields}` when the case carries any. +fn case(name: &str, value: i32, case_fields: &[FieldBinding]) -> String { + let mut out = format!("{name} = {value}"); + if !case_fields.is_empty() { + let _ = write!(out, " {{{}}}", fields(case_fields)); + } + out +} + +fn callable_signature(kind: &str, f: &FnBinding) -> String { + let mut sig = format!( + "{kind} {}({}) -> {}", + f.name, + params(&f.params), + ret(f.ret.as_ref()) + ); + if f.throws { + sig.push_str(" throws"); + } + if f.is_async() { + sig.push_str(" async"); + } + if f.cancellable { + sig.push_str(" cancellable"); + } + sig +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::ir::Api; + use crate::pkg::Identity; + + fn model(yaml: &str) -> Model { + let api: Api = serde_yaml::from_str(yaml).unwrap(); + crate::validate::validate(&api, &Identity::named("kv"), None).unwrap() + } + + const KV: &str = r#" +version: "0.11.0" +modules: + - name: kv + doc: Key-value storage. + errors: + name: KvError + codes: + - { name: NotFound, code: 1, message: missing, fields: [{ name: key, type: string }] } + structs: + - name: Entry + fields: [{ name: key, type: string }, { name: value, type: bytes }] + interfaces: + - name: Store + constructors: [{ name: open, params: [{ name: path, type: string }], throws: true }] + methods: + - { name: get, params: [{ name: key, type: string }], return: "bytes?", throws: true } + callback_interfaces: + - name: Listener + methods: + - { name: on_put, params: [{ name: entry, type: Entry }], return: string, throws: true } + functions: + - { name: version, params: [], return: string, doc: The version. } + modules: + - name: stats + enums: + - name: Kind + variants: [{ name: Hot, value: 0 }, { name: Cold, value: 1 }] +"#; + + fn paths(model: &Model) -> Vec { + let root = model.roots().next().unwrap(); + entries(model, root).into_iter().map(|e| e.path).collect() + } + + #[test] + fn one_entry_per_declaration_sorted_by_id() { + let model = model(KV); + let root = model.roots().next().unwrap(); + let table = entries(&model, root); + let mut got = paths(&model); + got.sort(); + assert_eq!( + got, + [ + "kv.Entry", + "kv.KvError", + "kv.Listener", + "kv.Store", + "kv.Store.get", + "kv.Store.open", + "kv.stats.Kind", + "kv.version" + ] + ); + assert!(table.windows(2).all(|w| w[0].id < w[1].id)); + let get = table.iter().find(|e| e.path == "kv.Store.get").unwrap(); + assert_eq!(get.id, fnv1a64(b"kv.Store.get")); + assert_eq!( + get.hash, + fnv1a64(b"method get(key: string) -> bytes? throws") + ); + let listener = table.iter().find(|e| e.path == "kv.Listener").unwrap(); + assert_eq!( + listener.hash, + fnv1a64(b"callback Listener {on_put(entry: Entry) -> string throws}") + ); + } + + fn hash_of(model: &Model, path: &str) -> u64 { + let root = model.roots().next().unwrap(); + entries(model, root) + .into_iter() + .find(|e| e.path == path) + .unwrap() + .hash + } + + #[test] + fn prose_and_sibling_order_are_excluded() { + let a = model(KV); + let edited = KV + .replace("doc: The version.", "doc: Changed prose.") + .replace("message: missing", "message: gone"); + let b = model(&edited); + assert_eq!( + entries(&a, a.roots().next().unwrap()), + entries(&b, b.roots().next().unwrap()) + ); + // Swapping two variants changes the enum (its own order is part of + // the signature) but not its neighbors. + let swapped = KV.replace( + "[{ name: Hot, value: 0 }, { name: Cold, value: 1 }]", + "[{ name: Cold, value: 1 }, { name: Hot, value: 0 }]", + ); + let c = model(&swapped); + assert_ne!(hash_of(&a, "kv.stats.Kind"), hash_of(&c, "kv.stats.Kind")); + assert_eq!(hash_of(&a, "kv.Entry"), hash_of(&c, "kv.Entry")); + } + + #[test] + fn abi_changes_change_only_their_entry() { + let a = model(KV); + let b = model(&KV.replace( + "return: \"bytes?\", throws: true", + "return: bytes, throws: true", + )); + assert_ne!(hash_of(&a, "kv.Store.get"), hash_of(&b, "kv.Store.get")); + assert_eq!(hash_of(&a, "kv.Store.open"), hash_of(&b, "kv.Store.open")); + assert_eq!(hash_of(&a, "kv.Store"), hash_of(&b, "kv.Store")); + } + + #[test] + fn fnv_is_pinned() { + // Changing the algorithm invalidates every deployed binding, so it + // must be a deliberate, documented ABI change. + assert_eq!(fnv1a64(b""), 0xcbf2_9ce4_8422_2325); + assert_eq!(fnv1a64(b"a"), 0xaf63_dc4c_8601_ec8c); + } +} diff --git a/crates/weaveffi-model/src/errors.rs b/crates/weaveffi-model/src/errors.rs index 33c5c77a..022c16e1 100644 --- a/crates/weaveffi-model/src/errors.rs +++ b/crates/weaveffi-model/src/errors.rs @@ -1,7 +1,7 @@ //! The shared error **naming policy**. //! //! The error *model* (which domains exist, which codes they carry, which -//! module owns them) lives in the binding model as +//! module owns them) lives in the model as //! [`ErrorBinding`](crate::model::ErrorBinding). This module holds only the //! idiomatic naming rules every backend applies to those names, centralized so //! no target drifts into `KEY_NOT_FOUNDError` (raw SCREAMING_SNAKE with a diff --git a/crates/weaveffi-model/src/ir.rs b/crates/weaveffi-model/src/ir.rs index 952981c8..f3fc9aaf 100644 --- a/crates/weaveffi-model/src/ir.rs +++ b/crates/weaveffi-model/src/ir.rs @@ -9,12 +9,14 @@ //! `Contact?`, and so on) rather than as a tagged object. //! //! The document model is deliberately *unresolved*: every user-defined type -//! reference is a [`TypeRef::Named`] carrying the name exactly as written. +//! reference is a [`TypeRef::Named`] carrying the bare name exactly as +//! written. Type names are global across the API, so a name alone identifies +//! a declaration; there are no module-qualified (`a.b.T`) references. //! Whether that name is a record, an enum, an interface, or a callback //! interface is decided by the [validator](crate::validate), which lowers the -//! document into the resolved binding model generators consume. Keeping the -//! two representations distinct means an IDL document always round-trips -//! losslessly through this module. +//! document into the [`Model`](crate::model::Model) generators consume. +//! Keeping the two representations distinct means an IDL document always +//! round-trips losslessly through this module. //! //! Package identity and per-generator options are not part of the IDL: they //! describe how bindings are *shipped*, not what the API *is*, and live in the @@ -24,6 +26,8 @@ use schemars::JsonSchema; use serde::{Deserialize, Serialize}; +use crate::ty::Prim; + /// The current IR schema version that the parser, validator, and every /// generator expect. /// @@ -35,7 +39,7 @@ use serde::{Deserialize, Serialize}; /// /// See [`docs/src/stability.md`](https://github.com/weavefoundry/weaveffi/blob/main/docs/src/stability.md) /// for the full schema policy and the surfaces covered by SemVer. -pub const CURRENT_SCHEMA_VERSION: &str = "0.10.0"; +pub const CURRENT_SCHEMA_VERSION: &str = "0.11.0"; /// Every IR schema version the current tools accept. /// @@ -65,7 +69,7 @@ fn is_false(b: &bool) -> bool { schemars(description = "Top-level WeaveFFI API definition.") )] pub struct Api { - /// IR schema version this document targets (for example `0.10.0`). + /// IR schema version this document targets (for example `0.11.0`). /// Validation rejects any value not listed in [`SUPPORTED_VERSIONS`]. pub version: String, /// Top-level modules that make up the API surface. Each is an independent @@ -73,6 +77,55 @@ pub struct Api { pub modules: Vec, } +impl Api { + /// Call `f` on every type reference written anywhere in the document: + /// parameters and returns of every function, interface member, and + /// callback method, and the fields of every record, enum variant, and + /// error code, in declaration order (outermost references only; use + /// [`TypeRef::walk`] to reach nested ones). + pub fn for_each_type_ref<'a>(&'a self, f: &mut dyn FnMut(&'a TypeRef)) { + fn functions<'a>(fs: &'a [Function], f: &mut dyn FnMut(&'a TypeRef)) { + for func in fs { + for p in &func.params { + f(&p.ty); + } + if let Some(r) = &func.returns { + f(r); + } + } + } + fn fields<'a>(fields: &'a [StructField], f: &mut dyn FnMut(&'a TypeRef)) { + for field in fields { + f(&field.ty); + } + } + fn modules<'a>(ms: &'a [Module], f: &mut dyn FnMut(&'a TypeRef)) { + for m in ms { + functions(&m.functions, f); + for i in &m.interfaces { + functions(&i.constructors, f); + functions(&i.methods, f); + functions(&i.statics, f); + } + for c in &m.callback_interfaces { + functions(&c.methods, f); + } + for s in &m.structs { + fields(&s.fields, f); + } + for v in m.enums.iter().flat_map(|e| &e.variants) { + fields(&v.fields, f); + } + for c in m.errors.iter().flat_map(|d| &d.codes) { + fields(&c.fields, f); + } + modules(&m.modules, f); + } + } + modules(&self.modules, f); + } +} + /// A module: a named namespace grouping related functions, types, callback /// interfaces, and an error domain. /// @@ -231,10 +284,14 @@ pub struct InterfaceDef { /// thread. At the C ABI it lowers to a context pointer plus a vtable of /// function pointers; see the C ABI contract. /// -/// Methods are synchronous, can't declare `throws`, `async`, or `cancellable`, -/// and return either nothing or a value in the direct family (integers, -/// floats, `bool`, or a C-style enum). Parameters may use any type other than -/// another callback interface or an iterator. +/// Methods are synchronous (never `async` or `cancellable`) and return +/// nothing or any type except an iterator or a callback interface; a string, +/// bytes, or buffer return is a run the consumer allocates and the producer +/// adopts. A method may declare `throws` when an error domain is in scope, +/// letting the consumer report that domain's codes. Parameters may use any +/// type other than a callback interface or an iterator. A callback interface +/// may itself be passed, bare or optional (`Listener?`), as a parameter of a +/// function or interface member. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[cfg_attr(feature = "idl", derive(JsonSchema))] #[serde(deny_unknown_fields)] @@ -290,36 +347,13 @@ pub struct Param { /// C ABI can't represent uniformly. #[derive(Debug, Clone, PartialEq, Eq, Hash)] pub enum TypeRef { - /// Signed 8-bit integer (`i8`). - I8, - /// Signed 16-bit integer (`i16`). - I16, - /// Signed 32-bit integer (`i32`). - I32, - /// Signed 64-bit integer (`i64`). - I64, - /// Unsigned 8-bit integer (`u8`). - U8, - /// Unsigned 16-bit integer (`u16`). - U16, - /// Unsigned 32-bit integer (`u32`). - U32, - /// Unsigned 64-bit integer (`u64`). - U64, - /// 32-bit IEEE 754 floating-point number (`f32`). - F32, - /// 64-bit IEEE 754 floating-point number (`f64`). - F64, - /// Boolean (`bool`). - Bool, - /// UTF-8 string (`string`). Borrowed when passed in, owned when returned. - StringUtf8, - /// Byte buffer (`bytes`). Borrowed when passed in, owned when returned. - Bytes, - /// A reference to a user-defined type (record, enum, interface, or callback - /// interface) by its bare or dot-qualified name (`Contact`, - /// `shared.Status`), exactly as parsed. Resolution happens in the - /// validator, not here. + /// A built-in primitive (`i32`, `bool`, `string`, `bytes`, and so on). + /// Strings and bytes are borrowed when passed in and owned when returned. + Prim(Prim), + /// A reference to a user-defined type (record, enum, interface, or + /// callback interface) by its bare name (`Contact`), exactly as parsed. + /// Resolution happens in the validator, which rejects a dotted + /// (module-qualified) name: type names are global. Named(String), /// Optional value (`T?`): either the inner type or nothing. Optional(Box), @@ -357,7 +391,7 @@ pub fn parse_type_ref(s: &str) -> Result { // what Rust producers declare (`Vec`), so canonicalizing here // keeps the top-level ABI slots consistent between an IDL-driven // consumer and a macro-driven producer. - TypeRef::U8 => TypeRef::Bytes, + TypeRef::Prim(Prim::U8) => TypeRef::Prim(Prim::Bytes), t => TypeRef::List(Box::new(t)), }); } @@ -378,22 +412,7 @@ pub fn parse_type_ref(s: &str) -> Result { { return parse_type_ref(inner).map(|t| TypeRef::Iterator(Box::new(t))); } - match s { - "i8" => Ok(TypeRef::I8), - "i16" => Ok(TypeRef::I16), - "i32" => Ok(TypeRef::I32), - "i64" => Ok(TypeRef::I64), - "u8" => Ok(TypeRef::U8), - "u16" => Ok(TypeRef::U16), - "u32" => Ok(TypeRef::U32), - "u64" => Ok(TypeRef::U64), - "f32" => Ok(TypeRef::F32), - "f64" => Ok(TypeRef::F64), - "bool" => Ok(TypeRef::Bool), - "string" => Ok(TypeRef::StringUtf8), - "bytes" => Ok(TypeRef::Bytes), - name => Ok(TypeRef::Named(name.to_string())), - } + Ok(Prim::from_name(s).map_or_else(|| TypeRef::Named(s.to_string()), TypeRef::Prim)) } /// The byte offset of the top-level `:` separating a map's key from its value, @@ -440,19 +459,7 @@ impl TypeRef { fn type_ref_to_string(ty: &TypeRef) -> String { match ty { - TypeRef::I8 => "i8".to_string(), - TypeRef::I16 => "i16".to_string(), - TypeRef::I32 => "i32".to_string(), - TypeRef::I64 => "i64".to_string(), - TypeRef::U8 => "u8".to_string(), - TypeRef::U16 => "u16".to_string(), - TypeRef::U32 => "u32".to_string(), - TypeRef::U64 => "u64".to_string(), - TypeRef::F32 => "f32".to_string(), - TypeRef::F64 => "f64".to_string(), - TypeRef::Bool => "bool".to_string(), - TypeRef::StringUtf8 => "string".to_string(), - TypeRef::Bytes => "bytes".to_string(), + TypeRef::Prim(p) => p.snake().to_string(), TypeRef::Named(name) => name.clone(), TypeRef::Optional(inner) => format!("{}?", type_ref_to_string(inner)), TypeRef::List(inner) => format!("[{}]", type_ref_to_string(inner)), @@ -488,7 +495,7 @@ impl<'de> Deserialize<'de> for TypeRef { /// Manual `JsonSchema` impl because `TypeRef` (de)serializes as a string with /// custom syntax: primitive names (`i32`, `string`, ...), `iter<{T}>`, -/// `[{T}]`, `{ {K}: {V} }`, `{name}?`, or any user-defined type name. +/// `[{T}]`, `{ {K}: {V} }`, `{name}?`, or a user-defined type's bare name. #[cfg(feature = "idl")] impl JsonSchema for TypeRef { fn schema_name() -> String { @@ -510,8 +517,9 @@ impl JsonSchema for TypeRef { "Reference to a type. Encoded as a string with custom syntax: \ primitives (`i8`, `i16`, `i32`, `i64`, `u8`, `u16`, `u32`, `u64`, \ `f32`, `f64`, `bool`, `string`, `bytes`), iterators (`iter<{T}>`), \ - lists (`[{T}]`), maps (`{{K:V}}`), optionals (`{T}?`), or any \ - user-defined struct, enum, interface, or callback interface name." + lists (`[{T}]`), maps (`{{K:V}}`), optionals (`{T}?`), or the bare \ + (unqualified) name of a user-defined struct, enum, interface, or callback \ + interface." .to_string(), ); schema.into() @@ -672,19 +680,19 @@ mod tests { #[test] fn primitives_round_trip() { let cases: &[(&str, TypeRef)] = &[ - ("i8", TypeRef::I8), - ("i16", TypeRef::I16), - ("i32", TypeRef::I32), - ("i64", TypeRef::I64), - ("u8", TypeRef::U8), - ("u16", TypeRef::U16), - ("u32", TypeRef::U32), - ("u64", TypeRef::U64), - ("f32", TypeRef::F32), - ("f64", TypeRef::F64), - ("bool", TypeRef::Bool), - ("string", TypeRef::StringUtf8), - ("bytes", TypeRef::Bytes), + ("i8", TypeRef::Prim(Prim::I8)), + ("i16", TypeRef::Prim(Prim::I16)), + ("i32", TypeRef::Prim(Prim::I32)), + ("i64", TypeRef::Prim(Prim::I64)), + ("u8", TypeRef::Prim(Prim::U8)), + ("u16", TypeRef::Prim(Prim::U16)), + ("u32", TypeRef::Prim(Prim::U32)), + ("u64", TypeRef::Prim(Prim::U64)), + ("f32", TypeRef::Prim(Prim::F32)), + ("f64", TypeRef::Prim(Prim::F64)), + ("bool", TypeRef::Prim(Prim::Bool)), + ("string", TypeRef::Prim(Prim::String)), + ("bytes", TypeRef::Prim(Prim::Bytes)), ]; for (s, expected) in cases { assert_eq!(&rt(s), expected); @@ -694,15 +702,23 @@ mod tests { #[test] fn composites_round_trip() { assert_eq!(rt("Contact"), TypeRef::Named("Contact".into())); + // A dotted name parses as written; validation rejects it with a + // dedicated diagnostic (type names are global). assert_eq!(rt("shared.Status"), TypeRef::Named("shared.Status".into())); assert_eq!( rt("Contact?"), TypeRef::Optional(Box::new(TypeRef::Named("Contact".into()))) ); - assert_eq!(rt("[string]"), TypeRef::List(Box::new(TypeRef::StringUtf8))); + assert_eq!( + rt("[string]"), + TypeRef::List(Box::new(TypeRef::Prim(Prim::String))) + ); assert_eq!( rt("{string:i32}"), - TypeRef::Map(Box::new(TypeRef::StringUtf8), Box::new(TypeRef::I32)) + TypeRef::Map( + Box::new(TypeRef::Prim(Prim::String)), + Box::new(TypeRef::Prim(Prim::I32)) + ) ); assert_eq!( rt("iter"), @@ -711,9 +727,9 @@ mod tests { assert_eq!( rt("[{string:[i32?]}]"), TypeRef::List(Box::new(TypeRef::Map( - Box::new(TypeRef::StringUtf8), + Box::new(TypeRef::Prim(Prim::String)), Box::new(TypeRef::List(Box::new(TypeRef::Optional(Box::new( - TypeRef::I32 + TypeRef::Prim(Prim::I32) ))))) ))) ); @@ -721,10 +737,10 @@ mod tests { #[test] fn u8_list_canonicalizes_to_bytes() { - assert_eq!(parse_type_ref("[u8]").unwrap(), TypeRef::Bytes); + assert_eq!(parse_type_ref("[u8]").unwrap(), TypeRef::Prim(Prim::Bytes)); assert_eq!( parse_type_ref("[[u8]]").unwrap(), - TypeRef::List(Box::new(TypeRef::Bytes)) + TypeRef::List(Box::new(TypeRef::Prim(Prim::Bytes))) ); } @@ -734,10 +750,10 @@ mod tests { parse_type_ref("{{string:i32}:bool}").unwrap(), TypeRef::Map( Box::new(TypeRef::Map( - Box::new(TypeRef::StringUtf8), - Box::new(TypeRef::I32) + Box::new(TypeRef::Prim(Prim::String)), + Box::new(TypeRef::Prim(Prim::I32)) )), - Box::new(TypeRef::Bool) + Box::new(TypeRef::Prim(Prim::Bool)) ) ); } @@ -771,7 +787,7 @@ mod tests { #[test] fn document_round_trips_through_yaml_and_json() { let yaml = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: contacts doc: Address book diff --git a/crates/weaveffi-model/src/lib.rs b/crates/weaveffi-model/src/lib.rs index ef85b398..b753c952 100644 --- a/crates/weaveffi-model/src/lib.rs +++ b/crates/weaveffi-model/src/lib.rs @@ -4,22 +4,27 @@ //! * [`ir`] holds the in-memory IR types that an IDL document or annotated //! Rust source lowers to, and [`parse`] reads the IDL text formats (YAML, //! JSON, and TOML). -//! * [`rust`] extracts the IR from annotated Rust source; the proc-macros and -//! the CLI share it. -//! * [`validate`] checks a document and produces the [`ResolvedApi`] every -//! consumer works from. -//! * [`model`], [`abi`], [`plan`], [`errors`], and [`pkg`] build the canonical -//! binding model, the C ABI lowering, the marshalling plan, the error -//! mapping, and the package identity. -//! * [`checksum`] fingerprints each top-level module's contract so a stale -//! binding refuses to load. +//! * [`rust`] extracts the IR from annotated Rust source for the +//! proc-macros. +//! * [`validate`] checks a document against the library's +//! [`Identity`](pkg::Identity) and builds the one [`Model`](model::Model) +//! every generator consumes. +//! * [`ty`], [`model`], [`abi`], [`plan`], [`errors`], and [`pkg`] hold the +//! resolved types and type index, the model with every C symbol and ABI +//! signature, the C ABI lowering, the marshalling plan, the error naming +//! policy, and the package identity. +//! * [`contract`] computes each top-level module's contract table, one +//! fingerprint per declaration, so a stale binding refuses to load and +//! names what changed. +//! * [`meta`] is the library metadata: the frames the macro embeds in a +//! producer's library and the CLI reads back into an [`Api`](ir::Api). //! //! # Features //! //! * `idl` (default): the IDL text formats, JSON Schema derivation for the IR //! types, and fancy miette diagnostics. Without it the crate still provides -//! the IR types, Rust extraction, validation, and the binding model, which -//! is all the proc-macros need. +//! the IR types, Rust extraction, validation, and the model, which is all +//! the proc-macros need. #![deny(missing_docs)] #![warn(clippy::missing_errors_doc)] #![warn(clippy::missing_panics_doc)] @@ -29,16 +34,15 @@ #![cfg_attr(not(feature = "idl"), allow(rustdoc::broken_intra_doc_links))] pub mod abi; -pub mod checksum; +pub mod contract; pub mod errors; pub mod ir; +pub mod meta; pub mod model; #[cfg(feature = "idl")] pub mod parse; pub mod pkg; pub mod plan; -pub mod resolved; pub mod rust; +pub mod ty; pub mod validate; - -pub use resolved::ResolvedApi; diff --git a/crates/weaveffi-model/src/meta.rs b/crates/weaveffi-model/src/meta.rs new file mode 100644 index 00000000..3cf945b9 --- /dev/null +++ b/crates/weaveffi-model/src/meta.rs @@ -0,0 +1,744 @@ +//! Library metadata: the API a Rust producer embeds in its compiled library. +//! +//! The `#[weaveffi::module]` macro describes every declaration it exports in +//! a [`Frame`] and embeds the frame's [encoding](Frame::encode) in the +//! producer's library as an exported static named [`Frame::symbol`] (on +//! `wasm32`, in the [`SECTION`] custom section instead). The CLI reads the +//! frames back out of the built library, [decodes](decode) them, keeps those +//! with the crate's own prefix, and [assembles](assemble) the [`Api`] it +//! generates from. Because the frames are compiled with the code they +//! describe, a declaration removed by `#[cfg]` is absent from both, and the +//! generated bindings describe exactly what the library exports. +//! +//! # Encoding +//! +//! A frame is a little-endian `u32` byte length followed by that many bytes +//! of compact JSON (the [`Frame`] serialized with serde). Frames are +//! self-delimiting, so a section holding several frames back to back +//! [decodes](decode) into all of them. +//! +//! One frame describes one declaration: a module (its name and doc), its +//! error domain, a free function, an interface (without its members), one +//! interface member, a record, an enum, or a callback interface. Each frame +//! records the path of the module it belongs to and its position among its +//! siblings of the same kind, so [`assemble`] restores declaration order. + +use std::collections::BTreeMap; + +use serde::{Deserialize, Serialize}; + +use crate::contract::fnv1a64; +use crate::ir::{ + Api, CallbackInterfaceDef, EnumDef, ErrorDomain, Function, InterfaceDef, Module, StructDef, + CURRENT_SCHEMA_VERSION, +}; + +/// The `wasm32` custom section the frames are placed in. +pub const SECTION: &str = "weaveffi_meta"; + +/// The infix of every metadata symbol: `{PREFIX}_META_{HASH16}`. +const SYMBOL_INFIX: &str = "_META_"; + +/// One declaration of a producer's API, as embedded in its library. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Frame { + /// The schema version the producer was built with + /// ([`CURRENT_SCHEMA_VERSION`] when built by this version). + pub schema: String, + /// The C symbol prefix of the crate that declared the item (its library + /// name), which tells a crate's own frames from a dependency's. + pub prefix: String, + /// The path of the module the item belongs to; for a module frame, its + /// parent's path (empty for a top-level module). + pub module: Vec, + /// The item's position among its siblings of the same kind (functions + /// among functions, submodules among submodules, and so on), in + /// declaration order. + pub index: u32, + /// The declaration. + pub item: Item, +} + +/// The declaration a [`Frame`] describes. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "kind", content = "def", rename_all = "snake_case")] +pub enum Item { + /// A module's own name and doc; its declarations have frames of their + /// own. + Module(ModuleHeader), + /// A module's error domain. + Error(ErrorDomain), + /// A free function. + Function(Function), + /// An interface, with no members: each has a [`Member`](Self::Member) + /// frame. + Interface(InterfaceDef), + /// One constructor, method, or static of an interface. + Member(Member), + /// A record. + Record(StructDef), + /// An enum. + Enum(EnumDef), + /// A callback interface, with its methods. + Callback(CallbackInterfaceDef), +} + +/// The part of a [`Module`] that isn't a declaration of its own. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ModuleHeader { + /// The module name. + pub name: String, + /// The module's documentation. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub doc: Option, +} + +/// One member of an interface. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct Member { + /// The name of the interface the member belongs to, declared in the same + /// module. + pub interface: String, + /// Which list of the interface the member is in. + pub role: Role, + /// The member's signature. + pub function: Function, +} + +/// Which list of an [`InterfaceDef`] a [`Member`] belongs to. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum Role { + /// [`InterfaceDef::constructors`]. + Constructor, + /// [`InterfaceDef::methods`]. + Method, + /// [`InterfaceDef::statics`]. + Static, +} + +/// A failure to decode or assemble metadata frames. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum MetaError { + /// The bytes end inside a frame. + #[error("metadata frame at byte {offset} is truncated")] + Truncated { + /// Where the frame starts. + offset: usize, + }, + /// A frame's JSON doesn't describe a frame. + #[error("metadata frame at byte {offset} is malformed: {message}")] + Malformed { + /// Where the frame starts. + offset: usize, + /// The JSON error. + message: String, + }, + /// A frame was written for another schema version. + #[error( + "`{path}` was built with WeaveFFI schema {found}, but this CLI reads schema {expected}; \ + build the producer with the same WeaveFFI version as the CLI" + )] + Schema { + /// The declaration's dotted path. + path: String, + /// The frame's schema version. + found: String, + /// The schema version this crate reads. + expected: String, + }, + /// Two different frames describe the same declaration. + #[error("the library describes `{path}` twice, differently")] + Conflict { + /// The declaration's dotted path. + path: String, + }, + /// A frame belongs to a module or interface no frame describes. + #[error("`{path}` belongs to `{parent}`, which the library doesn't describe")] + Orphan { + /// The declaration's dotted path. + path: String, + /// The missing module or interface. + parent: String, + }, +} + +impl Frame { + /// The declaration's own name. + #[must_use] + pub fn name(&self) -> &str { + match &self.item { + Item::Module(m) => &m.name, + Item::Error(e) => &e.name, + Item::Function(f) => &f.name, + Item::Interface(i) => &i.name, + Item::Member(m) => &m.function.name, + Item::Record(r) => &r.name, + Item::Enum(e) => &e.name, + Item::Callback(c) => &c.name, + } + } + + /// The names that locate the declaration inside its module: `[name]`, + /// `[interface, name]` for a member, and nothing for a module frame. + #[must_use] + pub fn local_path(&self) -> Vec<&str> { + match &self.item { + Item::Module(_) => Vec::new(), + Item::Member(m) => vec![&m.interface, &m.function.name], + _ => vec![self.name()], + } + } + + /// The declaration's dotted path (`kv.Store.put`, `kv.stats`). + #[must_use] + pub fn path(&self) -> String { + let mut parts: Vec<&str> = self.module.iter().map(String::as_str).collect(); + match &self.item { + Item::Module(m) => parts.push(&m.name), + _ => parts.extend(self.local_path()), + } + parts.join(".") + } + + /// The name of the exported static that holds the frame: + /// `{PREFIX}_META_{HASH16}`, where `HASH16` is 16 uppercase hex digits + /// of [`fnv1a64`] over the [dotted path](Self::path) (followed by a `.` + /// for a module, so a module and a same-named declaration beside it + /// differ). + #[must_use] + pub fn symbol(&self) -> String { + let mut key = self.path(); + if matches!(self.item, Item::Module(_)) { + key.push('.'); + } + format!( + "{}{SYMBOL_INFIX}{:016X}", + self.prefix.to_ascii_uppercase(), + fnv1a64(key.as_bytes()) + ) + } + + /// The frame's bytes: a little-endian `u32` length, then compact JSON. + /// + /// # Panics + /// + /// Panics if the JSON is longer than `u32::MAX` bytes. + #[must_use] + pub fn encode(&self) -> Vec { + let json = serde_json::to_vec(self).expect("a frame always serializes"); + let len = u32::try_from(json.len()).expect("a frame fits in u32::MAX bytes"); + let mut out = Vec::with_capacity(4 + json.len()); + out.extend_from_slice(&len.to_le_bytes()); + out.extend_from_slice(&json); + out + } +} + +/// Whether `symbol` names a metadata static: it ends in `_META_` and 16 +/// uppercase hex digits. A leading underscore (Mach-O's symbol prefix) is +/// fine; the frame's own prefix decides whose it is. +#[must_use] +pub fn is_symbol(symbol: &str) -> bool { + let Some((head, hash)) = symbol.rsplit_once(SYMBOL_INFIX) else { + return false; + }; + !head.is_empty() + && hash.len() == 16 + && hash + .bytes() + .all(|b| b.is_ascii_digit() || (b'A'..=b'F').contains(&b)) +} + +/// The byte length of the frame starting at `bytes`, length prefix +/// included, read from its prefix alone. +/// +/// # Errors +/// +/// Returns [`MetaError::Truncated`] when `bytes` is shorter than the length +/// prefix. +pub fn frame_len(bytes: &[u8]) -> Result { + let prefix: [u8; 4] = bytes + .get(..4) + .and_then(|b| b.try_into().ok()) + .ok_or(MetaError::Truncated { offset: 0 })?; + Ok(4 + u32::from_le_bytes(prefix) as usize) +} + +/// Decode every frame in `bytes`, which holds zero or more frames back to +/// back (one static's contents, or a whole `wasm32` custom section). +/// +/// # Errors +/// +/// Returns an error when a frame is truncated or its JSON isn't a frame. +pub fn decode(bytes: &[u8]) -> Result, MetaError> { + let mut frames = Vec::new(); + let mut offset = 0; + while offset < bytes.len() { + let rest = &bytes[offset..]; + let len = frame_len(rest).map_err(|_| MetaError::Truncated { offset })?; + let json = rest.get(4..len).ok_or(MetaError::Truncated { offset })?; + let frame = serde_json::from_slice(json).map_err(|e| MetaError::Malformed { + offset, + message: e.to_string(), + })?; + frames.push(frame); + offset += len; + } + Ok(frames) +} + +/// The frames describing `module` itself and its own declarations (not its +/// submodules'): what the macro embeds for one module of a tree. `parent` +/// is the path of the module's parent and `index` its position among its +/// parent's submodules. +#[must_use] +pub fn module_frames(module: &Module, parent: &[String], index: u32, prefix: &str) -> Vec { + let mut path = parent.to_vec(); + path.push(module.name.clone()); + let frame = |module: &[String], index: usize, item: Item| Frame { + schema: CURRENT_SCHEMA_VERSION.to_string(), + prefix: prefix.to_string(), + module: module.to_vec(), + index: u32::try_from(index).unwrap_or(u32::MAX), + item, + }; + let mut out = vec![frame( + parent, + index as usize, + Item::Module(ModuleHeader { + name: module.name.clone(), + doc: module.doc.clone(), + }), + )]; + if let Some(errors) = &module.errors { + out.push(frame(&path, 0, Item::Error(errors.clone()))); + } + for (i, f) in module.functions.iter().enumerate() { + out.push(frame(&path, i, Item::Function(f.clone()))); + } + for (i, iface) in module.interfaces.iter().enumerate() { + out.push(frame( + &path, + i, + Item::Interface(InterfaceDef { + constructors: Vec::new(), + methods: Vec::new(), + statics: Vec::new(), + ..iface.clone() + }), + )); + for (role, members) in [ + (Role::Constructor, &iface.constructors), + (Role::Method, &iface.methods), + (Role::Static, &iface.statics), + ] { + for (j, function) in members.iter().enumerate() { + out.push(frame( + &path, + j, + Item::Member(Member { + interface: iface.name.clone(), + role, + function: function.clone(), + }), + )); + } + } + } + for (i, r) in module.structs.iter().enumerate() { + out.push(frame(&path, i, Item::Record(r.clone()))); + } + for (i, e) in module.enums.iter().enumerate() { + out.push(frame(&path, i, Item::Enum(e.clone()))); + } + for (i, c) in module.callback_interfaces.iter().enumerate() { + out.push(frame(&path, i, Item::Callback(c.clone()))); + } + out +} + +/// Every frame of `api` (each module's [`module_frames`], recursively), as +/// though one crate with `prefix` declared it. +#[must_use] +pub fn frames(api: &Api, prefix: &str) -> Vec { + fn walk(modules: &[Module], parent: &[String], prefix: &str, out: &mut Vec) { + for (i, m) in modules.iter().enumerate() { + out.extend(module_frames( + m, + parent, + u32::try_from(i).unwrap_or(u32::MAX), + prefix, + )); + let mut path = parent.to_vec(); + path.push(m.name.clone()); + walk(&m.modules, &path, prefix, out); + } + } + let mut out = Vec::new(); + walk(&api.modules, &[], prefix, &mut out); + out +} + +/// One module while assembling: its header and its declarations, each with +/// its sibling index. +#[derive(Default)] +struct Node { + header: Option<(u32, ModuleHeader)>, + errors: Option, + functions: Vec<(u32, Function)>, + interfaces: Vec<(u32, InterfaceDef)>, + members: Vec<(u32, Member)>, + structs: Vec<(u32, StructDef)>, + enums: Vec<(u32, EnumDef)>, + callbacks: Vec<(u32, CallbackInterfaceDef)>, +} + +/// Assemble the [`Api`] that the frames with `prefix` describe, ignoring +/// every other frame (a dependency's). Declarations keep the order the +/// frames record; top-level modules, which separate macro invocations +/// declare, are sorted by name. A frame repeated verbatim (a library that +/// links an object twice) counts once. +/// +/// # Errors +/// +/// Returns an error when a frame was built for another schema version, two +/// different frames describe the same declaration, or a frame belongs to a +/// module or interface no frame describes. +pub fn assemble(frames: &[Frame], prefix: &str) -> Result { + let mut seen: BTreeMap<(String, &'static str), &Frame> = BTreeMap::new(); + let mut nodes: BTreeMap, Node> = BTreeMap::new(); + for frame in frames.iter().filter(|f| f.prefix == prefix) { + let path = frame.path(); + if frame.schema != CURRENT_SCHEMA_VERSION { + return Err(MetaError::Schema { + path, + found: frame.schema.clone(), + expected: CURRENT_SCHEMA_VERSION.to_string(), + }); + } + let kind = match &frame.item { + Item::Module(_) => "module", + Item::Member(_) => "member", + _ => "declaration", + }; + match seen.get(&(path.clone(), kind)) { + Some(prev) if *prev == frame => continue, + Some(_) => return Err(MetaError::Conflict { path }), + None => { + seen.insert((path, kind), frame); + } + } + let i = frame.index; + match &frame.item { + Item::Module(h) => { + let mut full = frame.module.clone(); + full.push(h.name.clone()); + nodes.entry(full).or_default().header = Some((i, h.clone())); + } + Item::Error(e) => { + nodes.entry(frame.module.clone()).or_default().errors = Some(e.clone()); + } + Item::Function(f) => nodes + .entry(frame.module.clone()) + .or_default() + .functions + .push((i, f.clone())), + Item::Interface(d) => nodes + .entry(frame.module.clone()) + .or_default() + .interfaces + .push((i, d.clone())), + Item::Member(m) => nodes + .entry(frame.module.clone()) + .or_default() + .members + .push((i, m.clone())), + Item::Record(r) => nodes + .entry(frame.module.clone()) + .or_default() + .structs + .push((i, r.clone())), + Item::Enum(e) => nodes + .entry(frame.module.clone()) + .or_default() + .enums + .push((i, e.clone())), + Item::Callback(c) => nodes + .entry(frame.module.clone()) + .or_default() + .callbacks + .push((i, c.clone())), + } + } + + // Every declaration's module must have a header, and every module's + // parent too. + for (path, node) in &nodes { + if node.header.is_none() { + let (child, parent) = orphan_of(path, node); + return Err(MetaError::Orphan { + path: child, + parent, + }); + } + if path.len() > 1 && !nodes.contains_key(&path[..path.len() - 1]) { + return Err(MetaError::Orphan { + path: path.join("."), + parent: path[..path.len() - 1].join("."), + }); + } + } + + let mut modules = build(&mut nodes, &[])?; + // Top-level modules come from separate macro invocations, whose + // relative order the frames can't record. + modules.sort_by(|a, b| a.name.cmp(&b.name)); + Ok(Api { + version: CURRENT_SCHEMA_VERSION.to_string(), + modules, + }) +} + +/// The first declaration of a header-less module, for the orphan error. +fn orphan_of(path: &[String], node: &Node) -> (String, String) { + let module = path.join("."); + let name = node + .functions + .first() + .map(|(_, f)| f.name.clone()) + .or_else(|| node.interfaces.first().map(|(_, d)| d.name.clone())) + .or_else(|| node.members.first().map(|(_, m)| m.interface.clone())) + .or_else(|| node.structs.first().map(|(_, d)| d.name.clone())) + .or_else(|| node.enums.first().map(|(_, d)| d.name.clone())) + .or_else(|| node.callbacks.first().map(|(_, d)| d.name.clone())) + .or_else(|| node.errors.as_ref().map(|e| e.name.clone())) + .unwrap_or_default(); + (format!("{module}.{name}"), module) +} + +/// Sort `(index, item)` pairs by index (then name, for a stable order when +/// a producer's frames disagree) and drop the indices. +fn ordered(mut items: Vec<(u32, T)>, name: impl Fn(&T) -> &str) -> Vec { + items.sort_by(|(i, a), (j, b)| i.cmp(j).then_with(|| name(a).cmp(name(b)))); + items.into_iter().map(|(_, t)| t).collect() +} + +/// Build the children of the module at `parent`, recursively. +fn build( + nodes: &mut BTreeMap, Node>, + parent: &[String], +) -> Result, MetaError> { + let children: Vec> = nodes + .keys() + .filter(|p| p.len() == parent.len() + 1 && p.starts_with(parent)) + .cloned() + .collect(); + let mut modules = Vec::new(); + for path in children { + let node = nodes.remove(&path).unwrap_or_default(); + let Some((index, header)) = node.header else { + continue; + }; + let mut interfaces = ordered(node.interfaces, |d| &d.name); + for member in ordered(node.members, |m| &m.function.name) { + let Some(iface) = interfaces.iter_mut().find(|d| d.name == member.interface) else { + return Err(MetaError::Orphan { + path: format!( + "{}.{}.{}", + path.join("."), + member.interface, + member.function.name + ), + parent: format!("{}.{}", path.join("."), member.interface), + }); + }; + let list = match member.role { + Role::Constructor => &mut iface.constructors, + Role::Method => &mut iface.methods, + Role::Static => &mut iface.statics, + }; + list.push(member.function); + } + let mut module = Module { + name: header.name, + doc: header.doc, + functions: ordered(node.functions, |f| &f.name), + interfaces, + callback_interfaces: ordered(node.callbacks, |d| &d.name), + structs: ordered(node.structs, |d| &d.name), + enums: ordered(node.enums, |d| &d.name), + errors: node.errors, + modules: Vec::new(), + }; + module.modules = build(nodes, &path)?; + modules.push((index, module)); + } + Ok(ordered(modules, |m| &m.name)) +} + +#[cfg(all(test, feature = "idl"))] +mod tests { + use super::*; + use crate::parse::parse_api_str; + + const API: &str = r#" +version: "0.11.0" +modules: + - name: kv + doc: The store. + errors: + name: KvError + codes: + - { name: Missing, code: 1, message: missing, fields: [{ name: key, type: string }] } + functions: + - { name: open_store, params: [{ name: path, type: string }], return: Store, throws: true } + - { name: version, return: u32 } + interfaces: + - name: Store + constructors: + - { name: new } + - { name: open, params: [{ name: path, type: string }], throws: true } + methods: + - { name: put, params: [{ name: key, type: string }] } + - { name: count, return: u32 } + statics: + - { name: default_capacity, return: u32 } + callback_interfaces: + - name: Listener + methods: + - { name: on_change, params: [{ name: key, type: string }] } + structs: + - { name: Entry, fields: [{ name: key, type: string }, { name: tags, type: "[string]" }] } + enums: + - { name: Kind, variants: [{ name: A, value: 0 }, { name: B, value: 1 }] } + modules: + - name: stats + functions: + - { name: summarize, params: [{ name: store, type: Store }], return: u32 } + - name: admin + functions: + - { name: wipe, params: [{ name: store, type: Store }] } + - name: report + functions: + - { name: render, params: [{ name: entries, type: "[Entry]" }], return: "[string]" } +"#; + + fn api() -> Api { + parse_api_str(API, "yaml").unwrap() + } + + fn section(frames: &[Frame]) -> Vec { + frames.iter().flat_map(Frame::encode).collect() + } + + #[test] + fn frames_round_trip_to_the_same_api() { + let api = api(); + let mut frames = frames(&api, "kvstore"); + // Link order is arbitrary; declaration order comes from the indices. + frames.reverse(); + let decoded = decode(§ion(&frames)).unwrap(); + assert_eq!(decoded, frames); + assert_eq!(assemble(&decoded, "kvstore").unwrap(), api); + } + + #[test] + fn foreign_and_repeated_frames() { + let api = api(); + let mut all = frames(&api, "kvstore"); + all.extend(frames(&api, "kvstore")); + all.extend(frames(&api, "other")); + assert_eq!(assemble(&all, "kvstore").unwrap(), api); + assert!(assemble(&all, "nobody").unwrap().modules.is_empty()); + } + + #[test] + fn symbols_paths_and_names() { + let all = frames(&api(), "kvstore"); + let put = all + .iter() + .find(|f| matches!(&f.item, Item::Member(m) if m.function.name == "put")) + .unwrap(); + assert_eq!(put.path(), "kv.Store.put"); + assert_eq!(put.local_path(), ["Store", "put"]); + let symbol = put.symbol(); + assert_eq!( + symbol, + format!("KVSTORE_META_{:016X}", fnv1a64(b"kv.Store.put")) + ); + assert!(is_symbol(&symbol)); + assert!(is_symbol(&format!("_{symbol}"))); + assert!(!is_symbol("KVSTORE_META_0123")); + assert!(!is_symbol("KVSTORE_META_0123456789abcdef")); + assert!(!is_symbol("_META_0123456789ABCDEF")); + let stats = all + .iter() + .find(|f| matches!(&f.item, Item::Module(m) if m.name == "stats")) + .unwrap(); + assert_eq!(stats.path(), "kv.stats"); + assert_ne!( + stats.symbol(), + format!("KVSTORE_META_{:016X}", fnv1a64(b"kv.stats")) + ); + let mut symbols: Vec = all.iter().map(Frame::symbol).collect(); + symbols.sort(); + symbols.dedup(); + assert_eq!(symbols.len(), all.len(), "every symbol is unique"); + } + + #[test] + fn malformed_and_inconsistent_frames_are_errors() { + let all = frames(&api(), "kvstore"); + let bytes = section(&all[..2]); + assert!(matches!( + decode(&bytes[..bytes.len() - 1]), + Err(MetaError::Truncated { .. }) + )); + assert!(matches!( + decode(&[2, 0, 0, 0, b'{', b'}']), + Err(MetaError::Malformed { offset: 0, .. }) + )); + + let mut stale = all.clone(); + stale[3].schema = "0.10.0".into(); + let err = assemble(&stale, "kvstore").unwrap_err(); + assert!(err.to_string().contains("schema 0.10.0"), "{err}"); + + let mut conflict = all.clone(); + let mut changed = all[2].clone(); + changed.index += 7; + conflict.push(changed); + assert!(matches!( + assemble(&conflict, "kvstore"), + Err(MetaError::Conflict { .. }) + )); + + let headless: Vec = all + .iter() + .filter(|f| !matches!(&f.item, Item::Module(m) if m.name == "stats")) + .cloned() + .collect(); + let err = assemble(&headless, "kvstore").unwrap_err(); + assert_eq!( + err, + MetaError::Orphan { + path: "kv.stats.summarize".into(), + parent: "kv.stats".into() + } + ); + + let memberless: Vec = all + .iter() + .filter(|f| !matches!(&f.item, Item::Interface(_))) + .cloned() + .collect(); + assert!(matches!( + assemble(&memberless, "kvstore"), + Err(MetaError::Orphan { .. }) + )); + } +} diff --git a/crates/weaveffi-model/src/model/build.rs b/crates/weaveffi-model/src/model/build.rs new file mode 100644 index 00000000..b7297a77 --- /dev/null +++ b/crates/weaveffi-model/src/model/build.rs @@ -0,0 +1,506 @@ +//! Building the [`Model`]: index every type declaration, resolve each written +//! [`TypeRef`] to a [`Ty`], and lower every declaration to its C symbols and +//! ABI signatures in one walk. + +use heck::ToUpperCamelCase; + +use super::{ + AbiFn, AsyncBinding, CallShape, CallbackInterfaceBinding, CallbackMethodBinding, EnumBinding, + EnumVariantBinding, ErrorBinding, ErrorCodeBinding, FieldBinding, FnBinding, InterfaceBinding, + IteratorBinding, Model, ModuleBinding, ParamBinding, StructBinding, +}; +use crate::abi::{ + callback_result_params, cancel_token_param, context_param, ctx_param, error_out_param, + lower_callback_return, lower_param, lower_return, AbiParam, CType, ConstPos, +}; +use crate::ir::{ + Api, CallbackInterfaceDef, EnumDef, ErrorDomain, Function, InterfaceDef, Module, Param, + StructDef, StructField, TypeRef, +}; +use crate::pkg::Identity; +use crate::ty::{Family, Ty, TypeDecl, TypeIndex, TypeKind}; + +/// Index every type declaration in `api`, numbering modules in the +/// depth-first pre-order [`Model::modules`] uses. The first declaration of a +/// name wins; validation reports any later one. +pub(crate) fn index(api: &Api) -> TypeIndex { + fn walk(modules: &[Module], parent: &[&str], types: &mut TypeIndex) { + for m in modules { + let mut segments = parent.to_vec(); + segments.push(&m.name); + let module = types.push_module(segments.join("_")); + let mut declare = |name: &str, kind: TypeKind, index: usize| { + types.declare( + name, + TypeDecl { + kind, + module, + index, + }, + ); + }; + for (i, s) in m.structs.iter().enumerate() { + declare(&s.name, TypeKind::Record, i); + } + for (i, e) in m.enums.iter().enumerate() { + let kind = if e.is_rich() { + TypeKind::RichEnum + } else { + TypeKind::Enum + }; + declare(&e.name, kind, i); + } + for (i, iface) in m.interfaces.iter().enumerate() { + declare(&iface.name, TypeKind::Interface, i); + } + for (i, c) in m.callback_interfaces.iter().enumerate() { + declare(&c.name, TypeKind::CallbackInterface, i); + } + walk(&m.modules, &segments, types); + } + } + let mut types = TypeIndex::default(); + walk(&api.modules, &[], &mut types); + types +} + +/// Map a written type reference to its resolved type. A name no declaration +/// provides resolves to a [`Ty::Record`] (see [`Model::assume_valid`]). +pub(crate) fn resolve(types: &TypeIndex, ty: &TypeRef) -> Ty { + match ty { + TypeRef::Prim(p) => Ty::Prim(*p), + TypeRef::Named(name) => { + let name = name.clone(); + match types.kind(&name) { + Some(TypeKind::Enum) => Ty::Enum(name), + Some(TypeKind::RichEnum) => Ty::RichEnum(name), + Some(TypeKind::Interface) => Ty::Interface(name), + Some(TypeKind::CallbackInterface) => Ty::CallbackInterface(name), + Some(TypeKind::Record) | None => Ty::Record(name), + } + } + TypeRef::Optional(inner) => Ty::Optional(Box::new(resolve(types, inner))), + TypeRef::List(inner) => Ty::List(Box::new(resolve(types, inner))), + TypeRef::Map(k, v) => Ty::Map(Box::new(resolve(types, k)), Box::new(resolve(types, v))), + TypeRef::Iterator(inner) => Ty::Iterator(Box::new(resolve(types, inner))), + } +} + +/// Build the model of `api` under `identity`, indexing its declarations +/// first. +pub(crate) fn build(api: &Api, identity: Identity) -> Model { + let types = index(api); + build_indexed(api, identity, types) +} + +/// Build the model of `api` from an index already built by [`index`]. +pub(crate) fn build_indexed(api: &Api, identity: Identity, types: TypeIndex) -> Model { + let mut modules = Vec::new(); + let lowerer = Lowerer { + types: &types, + prefix: &identity.prefix, + }; + for m in &api.modules { + lowerer.module(m, &[], &mut modules); + } + Model { + version: api.version.clone(), + modules, + identity, + types, + } +} + +/// The per-build lowering context: the type index and the symbol prefix. +struct Lowerer<'a> { + types: &'a TypeIndex, + prefix: &'a str, +} + +/// A fully assembled C signature: the ordered parameter slots and the C +/// return type. +struct AbiSig { + params: Vec, + ret: CType, +} + +impl Lowerer<'_> { + fn ty(&self, ty: &TypeRef) -> Ty { + resolve(self.types, ty) + } + + /// Recursively lower `module` and its descendants into the flat `out` + /// list, pre-order (parent before children) so symbol declarations + /// precede uses and positions match the [`TypeIndex`]. + fn module(&self, module: &Module, parent: &[String], out: &mut Vec) { + let mut segments = parent.to_vec(); + segments.push(module.name.clone()); + let path = segments.join("_"); + let prefix = self.prefix; + + let functions = module + .functions + .iter() + .map(|f| self.callable(f, &format!("{prefix}_{path}_{}", f.name), None)) + .collect(); + out.push(ModuleBinding { + name: module.name.clone(), + dot_path: segments.join("."), + doc: module.doc.clone(), + errors: module.errors.as_ref().map(|d| self.error_domain(d, &path)), + enums: module + .enums + .iter() + .map(|e| self.enum_def(e, &path)) + .collect(), + structs: module.structs.iter().map(|s| self.struct_def(s)).collect(), + interfaces: module + .interfaces + .iter() + .map(|i| self.interface(i, &path)) + .collect(), + callback_interfaces: module + .callback_interfaces + .iter() + .map(|c| self.callback_interface(c, &path)) + .collect(), + functions, + segments: segments.clone(), + path, + }); + + for child in &module.modules { + self.module(child, &segments, out); + } + } + + fn error_domain(&self, domain: &ErrorDomain, path: &str) -> ErrorBinding { + let c_tag = format!("{}_{path}_{}", self.prefix, domain.name); + ErrorBinding { + name: domain.name.clone(), + type_name: crate::errors::type_name(&domain.name, "Error"), + owner_path: path.to_string(), + codes: domain + .codes + .iter() + .map(|c| ErrorCodeBinding { + name: c.name.clone(), + value: c.code, + message: c.message.clone(), + doc: c.doc.clone(), + c_const: format!("{c_tag}_{}", c.name), + fields: self.fields(&c.fields), + }) + .collect(), + c_tag, + } + } + + fn fields(&self, fields: &[StructField]) -> Vec { + fields + .iter() + .map(|f| FieldBinding { + name: f.name.clone(), + doc: f.doc.clone(), + ty: self.ty(&f.ty), + }) + .collect() + } + + fn struct_def(&self, s: &StructDef) -> StructBinding { + StructBinding { + name: s.name.clone(), + doc: s.doc.clone(), + deprecated: s.deprecated.clone(), + fields: self.fields(&s.fields), + } + } + + fn enum_def(&self, e: &EnumDef, path: &str) -> EnumBinding { + let c_tag = format!("{}_{path}_{}", self.prefix, e.name); + let variants = e + .variants + .iter() + .map(|v| EnumVariantBinding { + name: v.name.clone(), + value: v.value, + doc: v.doc.clone(), + c_const: format!("{c_tag}_{}", v.name), + fields: self.fields(&v.fields), + }) + .collect(); + EnumBinding { + name: e.name.clone(), + doc: e.doc.clone(), + deprecated: e.deprecated.clone(), + c_tag, + variants, + rich: e.is_rich(), + } + } + + fn params(&self, params: &[Param]) -> Vec { + params + .iter() + .map(|p| { + let ty = self.ty(&p.ty); + ParamBinding { + name: p.name.clone(), + abi: lower_param(&p.name, &ty, self.types), + ty, + doc: p.doc.clone(), + } + }) + .collect() + } + + /// Lower a return type to its C return plus out-parameters, or `void`. + fn ret(&self, ret: Option<&Ty>) -> (CType, Vec) { + match ret { + Some(ty) => { + let r = lower_return(ty, self.types); + (r.ret, r.out_params) + } + None => (CType::Void, vec![]), + } + } + + /// The signature of one callback-interface method as it appears in the + /// vtable: `ctx`, then every parameter's slots, then the return's out + /// slots, then `out_err` (see + /// [`lower_callback_return`](crate::abi::lower_callback_return)). + /// + /// Object parameters differ from a plain call: the producer transfers one + /// strong reference the consumer adopts, so the slot is a mutable `{tag}*` + /// rather than the borrowed `const {tag}*` of a top-level parameter. + fn callback_method_signature(&self, params: &[ParamBinding], ret: Option<&Ty>) -> AbiSig { + let mut out = vec![ctx_param()]; + for p in params { + if matches!(p.ty.family(), Family::Object { .. }) { + let [slot] = p.abi.as_slice() else { + unreachable!("object parameter '{}' has one slot", p.name); + }; + let CType::Ptr { pointee, .. } = &slot.ty else { + unreachable!("object slot '{}' is a pointer", p.name); + }; + out.push(AbiParam::new(&slot.name, CType::ptr((**pointee).clone()))); + } else { + out.extend(p.abi.iter().cloned()); + } + } + let ret = match ret { + Some(ty) => { + let r = lower_callback_return(ty, self.types); + out.extend(r.out_params); + r.ret + } + None => CType::Void, + }; + out.push(error_out_param()); + AbiSig { params: out, ret } + } + + /// The full C signature of a *synchronous* call: every input parameter's + /// slots, then the return type's out-parameters, then `out_err`. + fn sync_signature(&self, params: &[ParamBinding], ret: Option<&Ty>) -> AbiSig { + let mut out: Vec = params.iter().flat_map(|p| p.abi.iter().cloned()).collect(); + let (ret, out_params) = self.ret(ret); + out.extend(out_params); + out.push(error_out_param()); + AbiSig { params: out, ret } + } + + fn callback_interface(&self, c: &CallbackInterfaceDef, path: &str) -> CallbackInterfaceBinding { + let c_tag = format!("{}_{path}_{}", self.prefix, c.name); + let methods = c + .methods + .iter() + .map(|m| { + let params = self.params(&m.params); + let ret = m.returns.as_ref().map(|r| self.ty(r)); + let sig = self.callback_method_signature(¶ms, ret.as_ref()); + CallbackMethodBinding { + name: m.name.clone(), + doc: m.doc.clone(), + deprecated: m.deprecated.clone(), + throws: m.throws, + params, + ret, + abi_params: sig.params, + abi_ret: sig.ret, + } + }) + .collect(); + CallbackInterfaceBinding { + name: c.name.clone(), + doc: c.doc.clone(), + deprecated: c.deprecated.clone(), + vtable_tag: format!("{c_tag}_vtable"), + c_tag, + methods, + } + } + + /// Lower an interface: constructors become statics returning the + /// interface, methods gain the implicit `self` slot, and all member + /// symbols hang off the interface's `c_tag`. + fn interface(&self, iface: &InterfaceDef, path: &str) -> InterfaceBinding { + let c_tag = format!("{}_{path}_{}", self.prefix, iface.name); + let self_slot = AbiParam::new( + "self", + CType::Ptr { + konst: ConstPos::West, + pointee: Box::new(CType::StructTag { + module: path.to_string(), + name: iface.name.clone(), + }), + }, + ); + let member = |name: &str| format!("{c_tag}_{name}"); + let constructors = iface + .constructors + .iter() + .map(|c| { + // A constructor yields a new strong reference to the + // interface, exactly like a static returning it. + let mut f = c.clone(); + f.returns = Some(TypeRef::Named(iface.name.clone())); + self.callable(&f, &member(&c.name), None) + }) + .collect(); + let methods = iface + .methods + .iter() + .map(|m| self.callable(m, &member(&m.name), Some(self_slot.clone()))) + .collect(); + let statics = iface + .statics + .iter() + .map(|s| self.callable(s, &member(&s.name), None)) + .collect(); + InterfaceBinding { + name: iface.name.clone(), + doc: iface.doc.clone(), + deprecated: iface.deprecated.clone(), + clone_symbol: format!("{c_tag}_clone"), + destroy_symbol: format!("{c_tag}_destroy"), + c_tag, + constructors, + methods, + statics, + } + } + + /// Lower one callable (free function or interface member) whose full base + /// C symbol is `c_base`. When `self_slot` is given (an instance method), + /// it is prepended to every ABI signature but never appears in the + /// retained [`ParamBinding`] list. + fn callable(&self, f: &Function, c_base: &str, self_slot: Option) -> FnBinding { + let prefix = self.prefix; + let params = self.params(&f.params); + let ret = f.returns.as_ref().map(|r| self.ty(r)); + // The prefix-stripped spelling used for `CType::Named` cores (which + // render as `{prefix}_{core}`), e.g. `kv_Store_scan` from + // `weaveffi_kv_Store_scan`. + let core_base = c_base + .strip_prefix(&format!("{prefix}_")) + .expect("c_base always starts with the symbol prefix") + .to_string(); + let with_self = |mut params: Vec| { + if let Some(s) = &self_slot { + params.insert(0, s.clone()); + } + params + }; + let inputs = + || -> Vec { params.iter().flat_map(|p| p.abi.iter().cloned()).collect() }; + + let shape = if let Some(elem) = ret.as_ref().and_then(Ty::iterator_elem) { + let pascal = f.name.to_upper_camel_case(); + // `{owner}_{Pascal}Iterator`, where owner is the module path for a + // free function or `{module path}_{Interface}` for a method. + let owner = &core_base[..core_base.len() - f.name.len() - 1]; + let iter_core = format!("{owner}_{pascal}Iterator"); + let iter_tag = format!("{prefix}_{iter_core}"); + + let mut launch_params = inputs(); + launch_params.push(error_out_param()); + let launch = AbiFn { + symbol: c_base.to_string(), + params: with_self(launch_params), + ret: CType::ptr(CType::Named(iter_core.clone())), + }; + + let item = lower_return(elem, self.types); + let mut next_params = vec![ + AbiParam::new("iter", CType::ptr(CType::Named(iter_core.clone()))), + AbiParam::new("out_item", CType::ptr(item.ret)), + ]; + next_params.extend(item.out_params); + next_params.push(error_out_param()); + let next = AbiFn { + symbol: format!("{iter_tag}_next"), + params: next_params, + ret: CType::Int32, + }; + + CallShape::Iterator(IteratorBinding { + elem: elem.clone(), + iter_tag: iter_tag.clone(), + launch, + next, + destroy_symbol: format!("{iter_tag}_destroy"), + }) + } else if f.r#async { + // Launcher: the input slots, the cancel token when cancellable, + // then the completion callback and its context. + let mut launch_params = inputs(); + if f.cancellable { + launch_params.push(cancel_token_param()); + } + launch_params.push(AbiParam::new( + "callback", + CType::Named(format!("{core_base}_callback")), + )); + launch_params.push(context_param()); + // Completion callback: `(void* context, {prefix}_error* err, + // )`. + let mut callback_params = vec![ + context_param(), + AbiParam::new("err", CType::ptr(CType::Error)), + ]; + if let Some(ret) = &ret { + callback_params.extend(callback_result_params(ret, self.types)); + } + CallShape::Async(AsyncBinding { + launch: AbiFn { + symbol: c_base.to_string(), + params: with_self(launch_params), + ret: CType::Void, + }, + callback_type: format!("{c_base}_callback"), + callback_params, + }) + } else { + let sig = self.sync_signature(¶ms, ret.as_ref()); + CallShape::Sync(AbiFn { + symbol: c_base.to_string(), + params: with_self(sig.params), + ret: sig.ret, + }) + }; + + FnBinding { + name: f.name.clone(), + doc: f.doc.clone(), + deprecated: f.deprecated.clone(), + cancellable: f.cancellable, + throws: f.throws, + has_self: self_slot.is_some(), + params, + ret, + c_base: c_base.to_string(), + shape, + } + } +} diff --git a/crates/weaveffi-model/src/model/mod.rs b/crates/weaveffi-model/src/model/mod.rs index e3faf644..e1a33e7b 100644 --- a/crates/weaveffi-model/src/model/mod.rs +++ b/crates/weaveffi-model/src/model/mod.rs @@ -1,40 +1,38 @@ -//! The **binding model**: a normalized, fully-lowered view of a -//! [`ResolvedApi`] that every language backend consumes. +//! The **model**: the one validated, fully lowered view of an API that every +//! language backend consumes. //! -//! [`BindingModel::build`] walks the IR exactly once and produces a flat list -//! of [`ModuleBinding`]s in which: +//! [`validate`](crate::validate::validate) checks an [`Api`] and builds the +//! [`Model`] exactly once per run. The model owns everything a generator +//! reads: //! -//! * every **type** is a resolved [`Ty`] (its kind and owning module known, -//! no unresolved names), so a backend dispatches on [`Ty::family`] and -//! [`Ty::wire`] instead of re-deriving what a name means; -//! * every emitted **C symbol name** is precomputed once, so all backends -//! agree by construction and a non-default prefix is honored everywhere; and -//! * every function, interface member, and callback-interface method is -//! paired with its lowered [`AbiFn`] signature (built from [`crate::abi`]), -//! so no backend re-derives parameter arity, ordering, or `out_*`/`out_err` -//! placement. +//! * the library's [`Identity`] (C symbol prefix, library name, package +//! metadata) and the schema version; +//! * a flat list of [`ModuleBinding`]s in which every **type** is a resolved +//! [`Ty`] (no unresolved names), so a backend dispatches on [`Ty::family`] +//! and [`Ty::wire`] instead of re-deriving what a name means; +//! * every emitted **C symbol name**, precomputed once, so all backends agree +//! by construction; +//! * every function, interface member, and callback-interface method paired +//! with its lowered [`AbiFn`] signature (built from [`crate::abi`]), so no +//! backend re-derives parameter arity, ordering, or `out_*`/`out_err` +//! placement; and +//! * the [`TypeIndex`], which maps each (global) type name to its +//! declaration and owning module, behind typed lookups such as +//! [`Model::interface`] and [`Model::owner`]. //! //! A backend reads the *idiomatic* shape from the retained [`Ty`]s //! (`param.ty`, `field.ty`, ...) and the *native* shape from the [`AbiFn`]s, //! then writes only the marshalling that bridges the two (see -//! [`crate::plan`]) in its own idioms. The hard, drift-prone facts live here; -//! only language syntax lives in the backends. +//! [`crate::plan`]) in its own idioms. Generators never see the [`Api`]. -mod ty; +mod build; -pub use ty::{Family, Prim, Ty, WireType}; +pub(crate) use build::{build_indexed, index}; -use crate::ir::{ - CallbackInterfaceDef, EnumDef, ErrorDomain, Function, InterfaceDef, Module, StructDef, - StructField, TypeRef, -}; -use heck::ToUpperCamelCase; - -use crate::abi::{ - async_callback_params, async_input_params, callback_method_signature, context_param, - error_out_param, lower_param, lower_return, sync_signature, AbiParam, CType, ConstPos, -}; -use crate::resolved::ResolvedApi; +use crate::abi::{AbiParam, CType}; +use crate::ir::{Api, TypeRef}; +use crate::pkg::Identity; +use crate::ty::{Ty, TypeDecl, TypeIndex, TypeKind}; /// The runtime symbols every producer exports (and every C header declares), /// without the `{prefix}_` that begins each one. See the C ABI contract for @@ -47,6 +45,7 @@ pub const RUNTIME_SYMBOLS: &[&str] = &[ "abi_version", "error", "error_set", + "error_set_payload", "error_clear", "error_free", "free_bytes", @@ -56,8 +55,8 @@ pub const RUNTIME_SYMBOLS: &[&str] = &[ "cancel_token_is_cancelled", "cancel_token_destroy", "alloc", - "dealloc", "debug_live", + "contract_entry", ]; /// Identifier families the generated C value-buffer helper header @@ -68,15 +67,27 @@ pub const RESERVED_SYMBOL_FAMILIES: &[&str] = &[ "str_", "bytes_", "reader_", "writer_", "list_", "map_", "opt_", ]; +/// The codec functions the C value-buffer helper header declares for each +/// record, rich enum, and error payload struct `T`: `T_write`, `T_read`, +/// `T_decode`, and `T_free`. +pub const VALUE_CODECS: &[&str] = &["write", "read", "decode", "free"]; + /// The C ABI revision this model lowers to. Producers export it from /// `{prefix}_abi_version()` and every generated consumer checks it at load. -pub const ABI_VERSION: u32 = 3; +pub const ABI_VERSION: u32 = 4; + +/// The symbol of a top-level module's contract table function, +/// `{prefix}_{module}_contract` (see [`crate::contract`]). +#[must_use] +pub fn contract_symbol(prefix: &str, module: &str) -> String { + format!("{prefix}_{module}_contract") +} -/// The symbol of a top-level module's contract checksum function, -/// `{prefix}_{module}_checksum`. +/// The C header's `static inline` checker of a top-level module's contract, +/// `{prefix}_{module}_contract_check`. #[must_use] -pub fn checksum_symbol(prefix: &str, module: &str) -> String { - format!("{prefix}_{module}_checksum") +pub fn contract_check_symbol(prefix: &str, module: &str) -> String { + format!("{prefix}_{module}_contract_check") } /// A single lowered C symbol: its name, ordered ABI parameter slots, and C @@ -132,6 +143,22 @@ pub struct IteratorBinding { pub destroy_symbol: String, } +impl IteratorBinding { + /// The C type of one element: the pointee of `next`'s `T* out_item` + /// slot. + /// + /// # Panics + /// + /// Panics if the model built `next` without its `out_item` pointer slot, + /// which would be a bug in the model construction. + pub fn item_ctype(&self) -> &CType { + match &self.next.params[1].ty { + CType::Ptr { pointee, .. } => pointee, + other => panic!("iterator `out_item` slot is {other:?}, not a pointer"), + } + } +} + /// One IR parameter, retained with its lowered ABI slots. #[derive(Debug, Clone, PartialEq, Eq)] pub struct ParamBinding { @@ -145,16 +172,6 @@ pub struct ParamBinding { pub abi: Vec, } -impl ParamBinding { - /// Lower one parameter declared in the module whose underscore-joined C - /// path is `module`. - pub fn new(name: impl Into, ty: Ty, doc: Option, module: &str) -> Self { - let name = name.into(); - let abi = lower_param(&name, &ty, module); - Self { name, ty, doc, abi } - } -} - /// A function, fully lowered. /// /// Free functions and interface members share this shape. For an instance @@ -172,11 +189,10 @@ pub struct FnBinding { pub deprecated: Option, /// Whether an async function accepts a trailing `cancel_token` slot. pub cancellable: bool, - /// Whether the function is `async` (lowered as a callback-completed launcher). - pub is_async: bool, /// Whether the function reports typed domain errors. A throwing function - /// surfaces as `throws`/`raises` in idiomatic wrappers using the module's - /// [`ErrorBinding`]; a non-throwing function has a plain signature, and a + /// surfaces as `throws`/`raises` in idiomatic wrappers using the error + /// domain in scope ([`Model::error_domain`]); a non-throwing function + /// has a plain signature, and a /// reported error (only ever a producer panic) surfaces as the target's /// unrecoverable-error idiom instead. pub throws: bool, @@ -198,6 +214,14 @@ pub struct FnBinding { pub shape: CallShape, } +impl FnBinding { + /// `true` when the function is `async` (lowered as a callback-completed + /// launcher). + pub fn is_async(&self) -> bool { + matches!(self.shape, CallShape::Async(_)) + } +} + /// A field of a record, a rich-enum variant, or an error code's payload. /// /// Records and rich enums are value types: they declare no C symbols of their @@ -313,8 +337,9 @@ pub struct InterfaceBinding { /// /// The consumer implements this; the producer calls it through the vtable. /// [`abi_params`](Self::abi_params) is the full C slot list of the vtable -/// entry (`void* ctx`, the parameter slots, `{prefix}_error* out_err`) and -/// [`abi_ret`](Self::abi_ret) its C return type. +/// entry (`void* ctx`, the parameter slots, the return's out slots, then +/// `{prefix}_error* out_err`) and [`abi_ret`](Self::abi_ret) its C return +/// type. #[derive(Debug, Clone, PartialEq, Eq)] pub struct CallbackMethodBinding { /// The method name as written in the IDL; also the vtable field name. @@ -323,12 +348,18 @@ pub struct CallbackMethodBinding { pub doc: Option, /// Deprecation message when the method is marked deprecated, else `None`. pub deprecated: Option, + /// Whether the method may report a positive code of the error domain in + /// scope ([`Model::error_domain`]) through its `out_err`. + pub throws: bool, /// Input parameters with their lowered slots. pub params: Vec, - /// The resolved return type (`None` = void); always in the direct family. + /// The resolved return type (`None` = void). Any family but iterators + /// and callback interfaces; see + /// [`lower_callback_return`](crate::abi::lower_callback_return). pub ret: Option, /// The vtable entry's C parameter slots: `ctx`, then every parameter's - /// slots, then `out_err`. + /// slots, then the return's out slots (`out_ptr` and `out_len` for a + /// string, bytes, or buffer return), then `out_err`. pub abi_params: Vec, /// The vtable entry's C return type. pub abi_ret: CType, @@ -336,9 +367,11 @@ pub struct CallbackMethodBinding { /// A callback interface, fully lowered. /// -/// The C ABI sees a vtable struct ([`vtable_tag`](Self::vtable_tag)) with one -/// function-pointer field per method, in declaration order, followed by a -/// trailing `void (*free)(void* ctx)`. A parameter of this type lowers to two +/// The C ABI sees a vtable struct ([`vtable_tag`](Self::vtable_tag)) that +/// starts with a fixed header, `uint32_t size` (the vtable's size as the +/// consumer compiled it), `uint32_t flags` (reserved, `0`), and +/// `void (*free)(void* ctx)`, followed by one function-pointer field per +/// method in declaration order. A parameter of this type lowers to two /// slots, `void* {name}_ctx` and `const {vtable_tag}* {name}_vtable`. #[derive(Debug, Clone, PartialEq, Eq)] pub struct CallbackInterfaceBinding { @@ -377,13 +410,20 @@ pub struct ErrorCodeBinding { pub fields: Vec, } -/// The error domain in effect for a module: its own `errors:` block, or the -/// nearest ancestor's when the module declares none. +impl ErrorCodeBinding { + /// `{c_const}_payload`, the struct the C value-buffer header declares + /// for the code's [`fields`](Self::fields) (only when there are any). + pub fn payload_tag(&self) -> String { + format!("{}_payload", self.c_const) + } +} + +/// An error domain, lowered on the module that declares it. /// -/// Every throwing function in the module reports codes from this domain. -/// Backends emit one error type per *declaring* module -/// ([`declared_here`](Self::declared_here) is `true`) and reference the -/// ancestor's type from inheriting submodules. +/// Every throwing function reports codes from the domain in scope for its +/// module: the module's own, or the nearest ancestor's +/// ([`Model::error_domain`]). Backends emit one error type per declaring +/// module and reference it from inheriting submodules. #[derive(Debug, Clone, PartialEq, Eq)] pub struct ErrorBinding { /// The domain name as written in the IDL (e.g. `KvError`). @@ -392,11 +432,8 @@ pub struct ErrorBinding { /// backends that brand exceptions swap the suffix via /// [`crate::errors::type_name`]. pub type_name: String, - /// Underscore-joined path of the module that *declares* the domain. + /// Underscore-joined path of the module that declares the domain. pub owner_path: String, - /// `true` when this module declares the domain itself; `false` when it - /// inherits the domain from an ancestor module. - pub declared_here: bool, /// `{prefix}_{owner_path}_{name}`, the C tag naming the domain's code /// constants. pub c_tag: String, @@ -409,24 +446,18 @@ pub struct ErrorBinding { pub struct ModuleBinding { /// The module name (its final path segment). pub name: String, - /// The contract checksum ([`crate::checksum::module_checksum`]) of a - /// top-level module; `None` for nested modules, which are covered by - /// their root's checksum. - pub checksum: Option, /// Path segments from the root (e.g. `["outer", "inner"]`). pub segments: Vec, /// Underscore-joined path used as the C symbol segment (e.g. `outer_inner`). pub path: String, - /// Dot-joined path used to qualify cross-module type names - /// (e.g. `outer.inner`). + /// Dot-joined path (e.g. `outer.inner`), the declaration path diagnostics + /// and generated comments quote. pub dot_path: String, - /// The module's doc comment when the IDL records one, else the doc of the - /// first documented function in the module. + /// The module's doc comment, if the IDL records one. pub doc: Option, - /// The error domain in effect for this module's throwing functions: - /// its own domain, the nearest ancestor's, or `None` when no domain is in - /// scope (in which case validation has rejected any `throws` here). - pub error: Option, + /// The error domain this module declares, if any. A module without one + /// inherits the nearest ancestor's ([`Model::error_domain`]). + pub errors: Option, /// Enums declared in this module, fully lowered. pub enums: Vec, /// Structs declared in this module, fully lowered. @@ -440,27 +471,6 @@ pub struct ModuleBinding { } impl ModuleBinding { - /// Find a callback interface declared in this module by name. - pub fn callback_interface(&self, name: &str) -> Option<&CallbackInterfaceBinding> { - self.callback_interfaces.iter().find(|c| c.name == name) - } - - /// True when this module declares no API surface at all. - pub fn is_empty(&self) -> bool { - self.enums.is_empty() - && self.structs.is_empty() - && self.interfaces.is_empty() - && self.callback_interfaces.is_empty() - && self.functions.is_empty() - && !self.declares_error() - } - - /// True when this module declares its own error domain (as opposed to - /// inheriting one from an ancestor). - pub fn declares_error(&self) -> bool { - self.error.as_ref().is_some_and(|e| e.declared_here) - } - /// Every callable in this module: free functions, then each interface's /// constructors, methods, and statics. pub fn callables(&self) -> impl Iterator { @@ -476,7 +486,7 @@ impl ModuleBinding { /// `true` when any callable in this module is `async`. pub fn has_async(&self) -> bool { - self.callables().any(|f| f.is_async) + self.callables().any(FnBinding::is_async) } /// `true` when any callable in this module returns an iterator. @@ -486,33 +496,46 @@ impl ModuleBinding { } } -/// The whole API, normalized and lowered for code generation. +/// The whole API, validated and lowered for code generation. +/// +/// Built once per run by [`validate`](crate::validate::validate); every +/// generator renders from it alone. #[derive(Debug, Clone, PartialEq, Eq)] -pub struct BindingModel { - /// The C symbol prefix every emitted name is built from. - pub prefix: String, - /// The IR schema version of the source `Api`. +pub struct Model { + /// The library's identity: its name, C symbol prefix, native library + /// name, and package metadata. + pub identity: Identity, + /// The IR schema version of the source document. pub version: String, /// Modules in depth-first pre-order, each carrying its joined symbol path. pub modules: Vec, + /// Every user type name, mapped to its declaration. + pub types: TypeIndex, } -impl BindingModel { - /// Build the model from a [`ResolvedApi`]. Every C symbol name starts - /// with the API identity's prefix, which is the same for the producer and - /// every consumer of one library. - pub fn build(api: &ResolvedApi) -> Self { - let prefix = api.identity().prefix.as_str(); - let mut modules = Vec::new(); - let lowerer = Lowerer { api, prefix }; - for m in &api.modules { - lowerer.module(m, &[], None, &mut modules); - } - Self { - prefix: prefix.to_string(), - version: api.version.clone(), - modules, - } +impl Model { + /// Build the model without validating the document. + /// + /// Type names that resolve to no declaration become [`Ty::Record`] + /// references named exactly as written. Tests use it for hand-built + /// trees; everything else goes through + /// [`validate`](crate::validate::validate). + #[doc(hidden)] + #[must_use] + pub fn assume_valid(api: &Api, identity: Identity) -> Self { + build::build(api, identity) + } + + /// Resolve a written type reference against this model's declarations. + /// A name no declaration provides resolves to a [`Ty::Record`] (see + /// [`assume_valid`](Self::assume_valid)). + pub fn resolve(&self, ty: &TypeRef) -> Ty { + build::resolve(&self.types, ty) + } + + /// The C symbol prefix every emitted name starts with. + pub fn prefix(&self) -> &str { + &self.identity.prefix } /// The top-level modules (those with a single path segment), in order. @@ -558,18 +581,75 @@ impl BindingModel { .flat_map(|m| m.callback_interfaces.iter().map(move |c| (m, c))) } - /// Find an interface anywhere in the API by its (possibly dot-qualified) - /// resolved name, as carried by [`Ty::Interface`]. A bare name is looked - /// up in `current` (the referencing module's dot path) first. - pub fn interface(&self, name: &str, current: &str) -> Option<&InterfaceBinding> { - let (dot_path, bare) = match name.rsplit_once('.') { - Some((p, n)) => (p, n), - None => (current, name), - }; - self.modules - .iter() - .find(|m| m.dot_path == dot_path) - .and_then(|m| m.interfaces.iter().find(|i| i.name == bare)) + /// The declaration of the user type `name`. + /// + /// # Panics + /// + /// Panics when `name` isn't declared, which validation rules out for + /// every name a [`Ty`] carries. + fn decl(&self, name: &str, kinds: &[TypeKind]) -> &TypeDecl { + match self.types.get(name) { + Some(d) if kinds.contains(&d.kind) => d, + Some(d) => panic!("type '{name}' is a {:?}, not one of {kinds:?}", d.kind), + None => panic!("type '{name}' is not declared"), + } + } + + /// The module declaring the user type `name`. + /// + /// # Panics + /// + /// Panics when `name` isn't declared, which validation rules out for + /// every name a [`Ty`] carries. + pub fn owner(&self, name: &str) -> &ModuleBinding { + let decl = self.types.get(name); + let decl = decl.unwrap_or_else(|| panic!("type '{name}' is not declared")); + &self.modules[decl.module] + } + + /// The interface named `name`, as carried by [`Ty::Interface`]. + /// + /// # Panics + /// + /// Panics when no interface has that name. + pub fn interface(&self, name: &str) -> &InterfaceBinding { + let d = self.decl(name, &[TypeKind::Interface]); + &self.modules[d.module].interfaces[d.index] + } + + /// The callback interface named `name`, as carried by + /// [`Ty::CallbackInterface`]. + /// + /// # Panics + /// + /// Panics when no callback interface has that name. + pub fn callback_interface(&self, name: &str) -> &CallbackInterfaceBinding { + let d = self.decl(name, &[TypeKind::CallbackInterface]); + &self.modules[d.module].callback_interfaces[d.index] + } + + /// The enum (C-style or rich) named `name`, as carried by [`Ty::Enum`] + /// and [`Ty::RichEnum`]. + /// + /// # Panics + /// + /// Panics when no enum has that name. + pub fn enumeration(&self, name: &str) -> &EnumBinding { + let d = self.decl(name, &[TypeKind::Enum, TypeKind::RichEnum]); + &self.modules[d.module].enums[d.index] + } + + /// The error domain in scope for `module`'s throwing functions: its own, + /// else the nearest ancestor's, else `None` (in which case validation + /// has rejected any `throws` there). + pub fn error_domain(&self, module: &ModuleBinding) -> Option<&ErrorBinding> { + (1..=module.segments.len()).rev().find_map(|n| { + let scope = &module.segments[..n]; + self.modules + .iter() + .find(|m| m.segments == scope) + .and_then(|m| m.errors.as_ref()) + }) } /// Every C identifier the library's ABI declares, paired with a @@ -578,21 +658,31 @@ impl BindingModel { /// an identifier. /// /// Covers the runtime surface ([`RUNTIME_SYMBOLS`]), each top-level - /// module's checksum function, every callable's symbols (sync entry - /// point, async launcher and completion type, iterator launcher, type, - /// `_next`, and `_destroy`), interface types with their `_clone` and - /// `_destroy`, C-style enum types and constants, error-domain types and - /// code constants, and callback-interface vtable types. + /// module's contract table function and the C header's checker for it, + /// every callable's symbols (sync entry point, async launcher and + /// completion type, iterator launcher, type, `_next`, and `_destroy`), + /// interface types with their `_clone` and `_destroy`, enum types and + /// constants (and a rich enum's tag type), error-domain types and code + /// constants, callback-interface vtable types, and what the C + /// value-buffer helper header declares per user type: the struct of + /// every record, rich enum, and error code with fields + /// ([`ErrorCodeBinding::payload_tag`]), each with its + /// [`VALUE_CODECS`]. (The header's list, map, and optional shapes fall + /// in [`RESERVED_SYMBOL_FAMILIES`].) pub fn c_symbols(&self) -> Vec<(String, String)> { - let p = &self.prefix; + let p = self.prefix(); let mut out: Vec<(String, String)> = RUNTIME_SYMBOLS .iter() .map(|s| (format!("{p}_{s}"), format!("the runtime symbol '{p}_{s}'"))) .collect(); for m in self.roots() { out.push(( - checksum_symbol(p, &m.name), - format!("the checksum of module '{}'", m.name), + contract_symbol(p, &m.name), + format!("the contract table of module '{}'", m.name), + )); + out.push(( + contract_check_symbol(p, &m.name), + format!("the contract check of module '{}'", m.name), )); } let callable = |out: &mut Vec<(String, String)>, f: &FnBinding, owner: &str| { @@ -620,15 +710,26 @@ impl BindingModel { } } }; + let codecs = |out: &mut Vec<(String, String)>, tag: &str, what: &str| { + for codec in VALUE_CODECS { + out.push(( + format!("{tag}_{codec}"), + format!("the value-buffer codec '{tag}_{codec}' of {what}"), + )); + } + }; for m in &self.modules { let dot = &m.dot_path; - if let Some(e) = m.error.as_ref().filter(|e| e.declared_here) { + if let Some(e) = &m.errors { out.push((e.c_tag.clone(), format!("error domain '{dot}.{}'", e.name))); for c in &e.codes { - out.push(( - c.c_const.clone(), - format!("error code '{dot}.{}.{}'", e.name, c.name), - )); + let what = format!("error code '{dot}.{}.{}'", e.name, c.name); + out.push((c.c_const.clone(), what.clone())); + if !c.fields.is_empty() { + let payload = c.payload_tag(); + codecs(&mut out, &payload, &format!("the payload of {what}")); + out.push((payload, format!("the payload struct of {what}"))); + } } } for e in &m.enums { @@ -638,6 +739,7 @@ impl BindingModel { format!("{}_Tag", e.c_tag), format!("the tag type of enum '{dot}.{}'", e.name), )); + codecs(&mut out, &e.c_tag, &format!("enum '{dot}.{}'", e.name)); } for v in &e.variants { out.push(( @@ -647,10 +749,10 @@ impl BindingModel { } } for s in &m.structs { - out.push(( - format!("{p}_{}_{}", m.path, s.name), - format!("record '{dot}.{}'", s.name), - )); + let tag = format!("{p}_{}_{}", m.path, s.name); + let what = format!("record '{dot}.{}'", s.name); + codecs(&mut out, &tag, &what); + out.push((tag, what)); } for c in &m.callback_interfaces { out.push(( @@ -702,454 +804,32 @@ impl BindingModel { /// `true` when any type anywhere in the API crosses the ABI as a value /// buffer (records, rich enums, optionals, lists, maps, error payloads). pub fn has_buffers(&self) -> bool { + let buffered = |ty: &Ty| ty.any(&Ty::is_buffered); + let signature = |params: &[ParamBinding], ret: &Option| { + params.iter().any(|p| buffered(&p.ty)) || ret.as_ref().is_some_and(buffered) + }; self.modules.iter().any(|m| { !m.structs.is_empty() || m.enums.iter().any(|e| e.rich) - || m.error + || m.errors .as_ref() .is_some_and(|e| e.codes.iter().any(|c| !c.fields.is_empty())) - }) || self.any_type(&|t| t.is_buffered()) - } - - /// `true` when any interface appears inside a buffered position (a record - /// field, a collection element, an optional payload, a rich-enum field), - /// so codecs need object-token support. - pub fn has_buffered_objects(&self) -> bool { - self.any_top_type(&|ty, position| match position { - // Fields are always serialized, so any object inside one is a - // buffered object. - TypePosition::Field => ty.contains_object(), - // A top-level `Store` or `Store?` is a plain pointer slot; only a - // top-level *buffered* type that nests an object counts. - TypePosition::Boundary => ty.is_buffered() && ty.contains_object(), - }) - } - - /// Visit every boundary-crossing type in the API (callable and - /// callback-method parameters and returns, iterator elements, record, - /// variant, and error payload fields), recursing into composites, and - /// return whether any satisfies `pred`. Backends use this to decide which - /// runtime helpers a given API actually needs. - pub fn any_type(&self, pred: &dyn Fn(&Ty) -> bool) -> bool { - self.any_top_type(&|ty, _| ty.any(pred)) - } - - /// Visit every *top-level* type in the API (no recursion into composites) - /// together with the kind of position it occupies, and return whether any - /// satisfies `pred`. - fn any_top_type(&self, pred: &dyn Fn(&Ty, TypePosition) -> bool) -> bool { - let boundary = |ty: &Ty| pred(ty, TypePosition::Boundary); - let field = |ty: &Ty| pred(ty, TypePosition::Field); - self.modules.iter().any(|m| { - m.callables().any(|f| { - f.params.iter().any(|p| boundary(&p.ty)) || f.ret.as_ref().is_some_and(boundary) - }) || m.callback_interfaces.iter().any(|c| { - c.methods.iter().any(|f| { - f.params.iter().any(|p| boundary(&p.ty)) || f.ret.as_ref().is_some_and(boundary) - }) - }) || m - .structs - .iter() - .any(|s| s.fields.iter().any(|f| field(&f.ty))) - || m.enums.iter().any(|e| { - e.variants - .iter() - .any(|v| v.fields.iter().any(|f| field(&f.ty))) - }) - || m.error.as_ref().is_some_and(|e| { - e.declared_here - && e.codes - .iter() - .any(|c| c.fields.iter().any(|f| field(&f.ty))) - }) + || m.callables().any(|f| signature(&f.params, &f.ret)) + || m.callback_interfaces + .iter() + .any(|c| c.methods.iter().any(|f| signature(&f.params, &f.ret))) }) } } -/// Where a top-level type sits, for [`BindingModel::any_top_type`]. -#[derive(Clone, Copy)] -enum TypePosition { - /// A callable or callback-method parameter or return. - Boundary, - /// A record, rich-enum variant, or error payload field (always serialized). - Field, -} - -/// The per-build lowering context: the resolver and the symbol prefix. -struct Lowerer<'a> { - api: &'a ResolvedApi, - prefix: &'a str, -} - -/// The per-module lowering context: where we are in the tree. -struct Scope<'a> { - /// Underscore-joined C path (`outer_inner`). - path: &'a str, -} - -impl Lowerer<'_> { - fn ty(&self, ty: &TypeRef) -> Ty { - self.api.resolve(ty) - } - - /// Recursively lower `module` and its descendants into the flat `out` - /// list, pre-order (parent before children) so symbol declarations precede - /// uses. `inherited_error` is the nearest ancestor's error domain, - /// threaded down so every module knows which domain its throwing - /// functions report. - fn module( - &self, - module: &Module, - parent: &[String], - inherited_error: Option<&ErrorBinding>, - out: &mut Vec, - ) { - let mut segments = parent.to_vec(); - segments.push(module.name.clone()); - let path = segments.join("_"); - let dot_path = segments.join("."); - let scope = Scope { path: &path }; - let prefix = self.prefix; - - let error = match &module.errors { - Some(domain) => Some(self.error_domain(domain, &scope)), - None => inherited_error.cloned().map(|mut e| { - e.declared_here = false; - e - }), - }; - - let enums = module - .enums - .iter() - .map(|e| self.enum_def(e, &scope)) - .collect(); - let structs = module.structs.iter().map(|s| self.struct_def(s)).collect(); - let interfaces = module - .interfaces - .iter() - .map(|i| self.interface(i, &scope)) - .collect(); - let callback_interfaces = module - .callback_interfaces - .iter() - .map(|c| self.callback_interface(c, &scope)) - .collect(); - let functions = module - .functions - .iter() - .map(|f| { - let c_base = format!("{prefix}_{path}_{}", f.name); - self.callable(f, &scope, &c_base, None) - }) - .collect(); - - let doc = module - .doc - .clone() - .or_else(|| module.functions.iter().find_map(|f| f.doc.clone())); - - out.push(ModuleBinding { - name: module.name.clone(), - checksum: parent - .is_empty() - .then(|| crate::checksum::module_checksum(module)), - segments: segments.clone(), - path: path.clone(), - dot_path: dot_path.clone(), - doc, - error: error.clone(), - enums, - structs, - interfaces, - callback_interfaces, - functions, - }); - - for child in &module.modules { - self.module(child, &segments, error.as_ref(), out); - } - } - - fn error_domain(&self, domain: &ErrorDomain, scope: &Scope<'_>) -> ErrorBinding { - let c_tag = format!("{}_{}_{}", self.prefix, scope.path, domain.name); - ErrorBinding { - name: domain.name.clone(), - type_name: crate::errors::type_name(&domain.name, "Error"), - owner_path: scope.path.to_string(), - declared_here: true, - c_tag: c_tag.clone(), - codes: domain - .codes - .iter() - .map(|c| ErrorCodeBinding { - name: c.name.clone(), - value: c.code, - message: c.message.clone(), - doc: c.doc.clone(), - c_const: format!("{c_tag}_{}", c.name), - fields: self.fields(&c.fields), - }) - .collect(), - } - } - - fn fields(&self, fields: &[StructField]) -> Vec { - fields - .iter() - .map(|f| FieldBinding { - name: f.name.clone(), - doc: f.doc.clone(), - ty: self.ty(&f.ty), - }) - .collect() - } - - fn struct_def(&self, s: &StructDef) -> StructBinding { - StructBinding { - name: s.name.clone(), - doc: s.doc.clone(), - deprecated: s.deprecated.clone(), - fields: self.fields(&s.fields), - } - } - - fn enum_def(&self, e: &EnumDef, scope: &Scope<'_>) -> EnumBinding { - let c_tag = format!("{}_{}_{}", self.prefix, scope.path, e.name); - let variants = e - .variants - .iter() - .map(|v| EnumVariantBinding { - name: v.name.clone(), - value: v.value, - doc: v.doc.clone(), - c_const: format!("{c_tag}_{}", v.name), - fields: self.fields(&v.fields), - }) - .collect(); - EnumBinding { - name: e.name.clone(), - doc: e.doc.clone(), - deprecated: e.deprecated.clone(), - c_tag, - variants, - rich: e.is_rich(), - } - } - - fn params(&self, params: &[crate::ir::Param], scope: &Scope<'_>) -> Vec { - params - .iter() - .map(|p| ParamBinding::new(&p.name, self.ty(&p.ty), p.doc.clone(), scope.path)) - .collect() - } - - fn callback_interface( - &self, - c: &CallbackInterfaceDef, - scope: &Scope<'_>, - ) -> CallbackInterfaceBinding { - let c_tag = format!("{}_{}_{}", self.prefix, scope.path, c.name); - let methods = c - .methods - .iter() - .map(|m| { - let params = self.params(&m.params, scope); - let ret = m.returns.as_ref().map(|r| self.ty(r)); - let sig = callback_method_signature(¶ms, ret.as_ref(), scope.path); - CallbackMethodBinding { - name: m.name.clone(), - doc: m.doc.clone(), - deprecated: m.deprecated.clone(), - params, - ret, - abi_params: sig.params, - abi_ret: sig.ret, - } - }) - .collect(); - CallbackInterfaceBinding { - name: c.name.clone(), - doc: c.doc.clone(), - deprecated: c.deprecated.clone(), - vtable_tag: format!("{c_tag}_vtable"), - c_tag, - methods, - } - } - - /// Lower an interface: constructors become statics returning the - /// interface, methods gain the implicit `self` slot, and all member - /// symbols hang off the interface's `c_tag`. - fn interface(&self, iface: &InterfaceDef, scope: &Scope<'_>) -> InterfaceBinding { - let c_tag = format!("{}_{}_{}", self.prefix, scope.path, iface.name); - let self_slot = AbiParam::new( - "self", - CType::Ptr { - konst: ConstPos::West, - pointee: Box::new(CType::StructTag { - module: scope.path.to_string(), - name: iface.name.clone(), - }), - }, - ); - let member = |name: &str| format!("{c_tag}_{name}"); - let constructors = iface - .constructors - .iter() - .map(|c| { - // A constructor yields a new strong reference to the - // interface, exactly like a static returning it. - let mut f = c.clone(); - f.returns = Some(TypeRef::Named(iface.name.clone())); - self.callable(&f, scope, &member(&c.name), None) - }) - .collect(); - let methods = iface - .methods - .iter() - .map(|m| self.callable(m, scope, &member(&m.name), Some(self_slot.clone()))) - .collect(); - let statics = iface - .statics - .iter() - .map(|s| self.callable(s, scope, &member(&s.name), None)) - .collect(); - InterfaceBinding { - name: iface.name.clone(), - doc: iface.doc.clone(), - deprecated: iface.deprecated.clone(), - c_tag: c_tag.clone(), - constructors, - methods, - statics, - clone_symbol: format!("{c_tag}_clone"), - destroy_symbol: format!("{c_tag}_destroy"), - } - } - - /// Lower one callable (free function or interface member) whose full base - /// C symbol is `c_base`. When `self_slot` is given (an instance method), - /// it is prepended to every ABI signature but never appears in the - /// retained [`ParamBinding`] list. - fn callable( - &self, - f: &Function, - scope: &Scope<'_>, - c_base: &str, - self_slot: Option, - ) -> FnBinding { - let prefix = self.prefix; - let path = scope.path; - let params = self.params(&f.params, scope); - let ret = f.returns.as_ref().map(|r| self.ty(r)); - // The prefix-stripped spelling used for `CType::Named` cores (which - // render as `{prefix}_{core}`), e.g. `kv_Store_scan` from - // `weaveffi_kv_Store_scan`. - let core_base = c_base - .strip_prefix(&format!("{prefix}_")) - .expect("c_base always starts with the symbol prefix") - .to_string(); - let with_self = |mut params: Vec| { - if let Some(s) = &self_slot { - params.insert(0, s.clone()); - } - params - }; - - let shape = if let Some(elem) = ret.as_ref().and_then(Ty::iterator_elem) { - let pascal = f.name.to_upper_camel_case(); - // `{owner}_{Pascal}Iterator`, where owner is the module path for a - // free function or `{module path}_{Interface}` for a method. - let owner = &core_base[..core_base.len() - f.name.len() - 1]; - let iter_core = format!("{owner}_{pascal}Iterator"); - let iter_tag = format!("{prefix}_{iter_core}"); - - let mut launch_params: Vec = - params.iter().flat_map(|p| p.abi.iter().cloned()).collect(); - launch_params.push(error_out_param()); - let launch = AbiFn { - symbol: c_base.to_string(), - params: with_self(launch_params), - ret: CType::ptr(CType::Named(iter_core.clone())), - }; - - let item = lower_return(elem, path); - let mut next_params = vec![ - AbiParam::new("iter", CType::ptr(CType::Named(iter_core.clone()))), - AbiParam::new("out_item", CType::ptr(item.ret)), - ]; - next_params.extend(item.out_params); - next_params.push(error_out_param()); - let next = AbiFn { - symbol: format!("{iter_tag}_next"), - params: next_params, - ret: CType::Int32, - }; - - CallShape::Iterator(IteratorBinding { - elem: elem.clone(), - iter_tag: iter_tag.clone(), - launch, - next, - destroy_symbol: format!("{iter_tag}_destroy"), - }) - } else if f.r#async { - let callback_type = format!("{c_base}_callback"); - let mut launch_params = async_input_params(¶ms, f.cancellable); - launch_params.push(AbiParam::new( - "callback", - CType::Named(format!("{core_base}_callback")), - )); - launch_params.push(context_param()); - let launch = AbiFn { - symbol: c_base.to_string(), - params: with_self(launch_params), - ret: CType::Void, - }; - CallShape::Async(AsyncBinding { - launch, - callback_type, - callback_params: async_callback_params(ret.as_ref(), path), - }) - } else { - let sig = sync_signature(¶ms, ret.as_ref(), path); - CallShape::Sync(AbiFn { - symbol: c_base.to_string(), - params: with_self(sig.params), - ret: sig.ret, - }) - }; - - FnBinding { - name: f.name.clone(), - doc: f.doc.clone(), - deprecated: f.deprecated.clone(), - cancellable: f.cancellable, - is_async: f.r#async, - throws: f.throws, - has_self: self_slot.is_some(), - params, - ret, - c_base: c_base.to_string(), - shape, - } - } -} - -/// The element C type of an iterator's `out_item` slot (the pointee of -/// `T* out_item`). Exposed for backends that materialize iterator results. -pub fn iterator_item_ctype(elem: &Ty, module: &str) -> CType { - lower_return(elem, module).ret -} - #[cfg(test)] mod tests { use super::*; use crate::ir::{ - Api, CallbackInterfaceDef, EnumDef, EnumVariant, Function, InterfaceDef, Module, Param, - StructDef, StructField, + CallbackInterfaceDef, EnumDef, EnumVariant, ErrorCode, ErrorDomain, Function, InterfaceDef, + Module, Param, StructDef, StructField, TypeRef, }; + use crate::ty::Prim; fn param(name: &str, ty: TypeRef) -> Param { Param { @@ -1186,11 +866,12 @@ mod tests { } } - fn api(modules: Vec) -> ResolvedApi { - ResolvedApi::assume_valid(Api { + fn build(modules: Vec, prefix: &str) -> Model { + let api = Api { version: crate::ir::CURRENT_SCHEMA_VERSION.into(), modules, - }) + }; + Model::assume_valid(&api, Identity::named(prefix)) } fn rendered(abi: &AbiFn) -> Vec { @@ -1205,14 +886,15 @@ mod tests { let m = Module { functions: vec![func( "add", - vec![param("a", TypeRef::I32), param("b", TypeRef::I32)], - Some(TypeRef::I32), + vec![ + param("a", TypeRef::Prim(Prim::I32)), + param("b", TypeRef::Prim(Prim::I32)), + ], + Some(TypeRef::Prim(Prim::I32)), )], ..module("math") }; - let model = BindingModel::build( - &api(vec![m]).with_identity(crate::pkg::Identity::named("weaveffi")), - ); + let model = build(vec![m], "weaveffi"); let f = &model.modules[0].functions[0]; assert_eq!(f.c_base, "weaveffi_math_add"); let CallShape::Sync(abi) = &f.shape else { @@ -1225,10 +907,8 @@ mod tests { ["int32_t a", "int32_t b", "weaveffi_error* out_err"] ); - let model = BindingModel::build( - &api(vec![module("net")]).with_identity(crate::pkg::Identity::named("acme")), - ); - assert_eq!(model.prefix, "acme"); + let model = build(vec![module("net")], "acme"); + assert_eq!(model.prefix(), "acme"); } #[test] @@ -1239,15 +919,13 @@ mod tests { r#async: true, ..func( "fetch", - vec![param("id", TypeRef::I64)], - Some(TypeRef::StringUtf8), + vec![param("id", TypeRef::Prim(Prim::I64))], + Some(TypeRef::Prim(Prim::String)), ) }], ..module("net") }; - let model = BindingModel::build( - &api(vec![m]).with_identity(crate::pkg::Identity::named("weaveffi")), - ); + let model = build(vec![m], "weaveffi"); let CallShape::Async(a) = &model.modules[0].functions[0].shape else { panic!("expected async") }; @@ -1267,13 +945,11 @@ mod tests { functions: vec![func( "get_messages", vec![], - Some(TypeRef::Iterator(Box::new(TypeRef::StringUtf8))), + Some(TypeRef::Iterator(Box::new(TypeRef::Prim(Prim::String)))), )], ..module("events") }; - let model = BindingModel::build( - &api(vec![m]).with_identity(crate::pkg::Identity::named("weaveffi")), - ); + let model = build(vec![m], "weaveffi"); let CallShape::Iterator(it) = &model.modules[0].functions[0].shape else { panic!("expected iterator") }; @@ -1284,9 +960,10 @@ mod tests { it.destroy_symbol, "weaveffi_events_GetMessagesIterator_destroy" ); - assert_eq!(it.elem, Ty::StringUtf8); + assert_eq!(it.elem, Ty::Prim(Prim::String)); assert_eq!(it.next.ret, CType::Int32); assert_eq!(it.next.params[1].ty.render_c("weaveffi"), "const uint8_t**"); + assert_eq!(it.item_ctype().render_c("weaveffi"), "const uint8_t*"); assert!(model.has_iterators()); } @@ -1300,7 +977,7 @@ mod tests { fields: vec![ StructField { name: "name".into(), - ty: TypeRef::StringUtf8, + ty: TypeRef::Prim(Prim::String), doc: None, }, StructField { @@ -1348,16 +1025,14 @@ mod tests { )], ..module("ops") }; - let model = BindingModel::build( - &api(vec![shared, other]).with_identity(crate::pkg::Identity::named("weaveffi")), - ); + let model = build(vec![shared, other], "weaveffi"); let contacts = &model.modules[0]; let s = &contacts.structs[0]; assert_eq!(s.deprecated.as_deref(), Some("use Person")); - assert_eq!(s.fields[1].ty, Ty::Enum("contacts.Status".into())); + assert_eq!(s.fields[1].ty, Ty::Enum("Status".into())); assert_eq!( s.fields[2].ty, - Ty::Optional(Box::new(Ty::Interface("contacts.Store".into()))) + Ty::Optional(Box::new(Ty::Interface("Store".into()))) ); assert_eq!(contacts.enums[0].c_tag, "weaveffi_contacts_Status"); assert_eq!( @@ -1371,11 +1046,11 @@ mod tests { assert_eq!(iface.destroy_symbol, "weaveffi_contacts_Store_destroy"); assert_eq!( iface.constructors[0].ret, - Some(Ty::Interface("contacts.Store".into())) + Some(Ty::Interface("Store".into())) ); let save = &iface.methods[0]; assert!(save.has_self); - assert_eq!(save.params[0].ty, Ty::Record("contacts.Contact".into())); + assert_eq!(save.params[0].ty, Ty::Record("Contact".into())); let CallShape::Sync(abi) = &save.shape else { panic!("expected sync") }; @@ -1393,8 +1068,8 @@ mod tests { let ops = &model.modules[1]; let f = &ops.functions[0]; - assert_eq!(f.params[0].ty, Ty::Interface("contacts.Store".into())); - assert_eq!(f.ret, Some(Ty::Enum("contacts.Status".into()))); + assert_eq!(f.params[0].ty, Ty::Interface("Store".into())); + assert_eq!(f.ret, Some(Ty::Enum("Status".into()))); let CallShape::Sync(abi) = &f.shape else { panic!("expected sync") }; @@ -1407,15 +1082,12 @@ mod tests { ); assert_eq!(abi.ret.render_c("weaveffi"), "weaveffi_contacts_Status"); assert!(model.has_buffers()); - assert!(model.has_buffered_objects()); assert!(model.has_interfaces()); + assert_eq!(model.interface("Store").c_tag, "weaveffi_contacts_Store"); + assert_eq!(model.owner("Status").path, "contacts"); assert_eq!( - model.interface("contacts.Store", "ops").map(|i| &i.c_tag), - Some(&"weaveffi_contacts_Store".to_string()) - ); - assert_eq!( - model.interface("Store", "contacts").map(|i| &i.c_tag), - Some(&"weaveffi_contacts_Store".to_string()) + model.enumeration("Status").c_tag, + "weaveffi_contacts_Status" ); } @@ -1427,8 +1099,12 @@ mod tests { doc: Some("Receives messages.".into()), deprecated: None, methods: vec![ - func("on_message", vec![param("text", TypeRef::StringUtf8)], None), - func("should_stop", vec![], Some(TypeRef::Bool)), + func( + "on_message", + vec![param("text", TypeRef::Prim(Prim::String))], + None, + ), + func("should_stop", vec![], Some(TypeRef::Prim(Prim::Bool))), ], }], functions: vec![func( @@ -1438,9 +1114,7 @@ mod tests { )], ..module("events") }; - let model = BindingModel::build( - &api(vec![m]).with_identity(crate::pkg::Identity::named("weaveffi")), - ); + let model = build(vec![m], "weaveffi"); let mb = &model.modules[0]; let cb = &mb.callback_interfaces[0]; assert_eq!(cb.c_tag, "weaveffi_events_Listener"); @@ -1462,14 +1136,11 @@ mod tests { ); assert_eq!(on_message.abi_ret, CType::Void); assert_eq!(cb.methods[1].abi_ret, CType::Bool); - assert!(mb.callback_interface("Listener").is_some()); + assert_eq!(model.callback_interface("Listener").c_tag, cb.c_tag); assert!(model.has_callback_interfaces()); let f = &mb.functions[0]; - assert_eq!( - f.params[0].ty, - Ty::CallbackInterface("events.Listener".into()) - ); + assert_eq!(f.params[0].ty, Ty::CallbackInterface("Listener".into())); let CallShape::Sync(abi) = &f.shape else { panic!("expected sync") }; @@ -1482,7 +1153,6 @@ mod tests { ] ); assert!(!model.has_buffers()); - assert!(!model.has_buffered_objects()); } #[test] @@ -1500,9 +1170,7 @@ mod tests { modules: vec![inner], ..module("outer") }; - let model = BindingModel::build( - &api(vec![outer]).with_identity(crate::pkg::Identity::named("weaveffi")), - ); + let model = build(vec![outer], "weaveffi"); let paths: Vec<&str> = model.modules.iter().map(|m| m.path.as_str()).collect(); assert_eq!(paths, ["outer", "outer_inner"]); assert_eq!(model.modules[1].dot_path, "outer.inner"); @@ -1511,6 +1179,36 @@ mod tests { "weaveffi_outer_inner_leaf_fn" ); assert_eq!(model.modules[0].doc.as_deref(), Some("Outer module.")); - assert_eq!(model.modules[1].doc.as_deref(), Some("Leaf.")); + assert_eq!(model.modules[1].doc, None); + } + + #[test] + fn error_domains_are_declared_once_and_inherited_by_lookup() { + let domain = ErrorDomain { + name: "OuterError".into(), + codes: vec![ErrorCode { + name: "Bad".into(), + code: 1, + message: "bad".into(), + doc: None, + fields: vec![], + }], + }; + let outer = Module { + errors: Some(domain), + modules: vec![module("inner")], + ..module("outer") + }; + let model = build(vec![outer, module("other")], "weaveffi"); + let [outer, inner, other] = &model.modules[..] else { + panic!("three modules"); + }; + assert!(outer.errors.is_some() && inner.errors.is_none()); + assert_eq!( + model.error_domain(inner).map(|e| e.c_tag.as_str()), + Some("weaveffi_outer_OuterError") + ); + assert_eq!(model.error_domain(outer), outer.errors.as_ref()); + assert!(model.error_domain(other).is_none()); } } diff --git a/crates/weaveffi-model/src/parse.rs b/crates/weaveffi-model/src/parse.rs index 465aceaf..3c1c7b86 100644 --- a/crates/weaveffi-model/src/parse.rs +++ b/crates/weaveffi-model/src/parse.rs @@ -171,6 +171,7 @@ pub fn parse_api_str(s: &str, format: &str) -> Result { mod tests { use super::*; use crate::ir::{Function, Module, Param, TypeRef, CURRENT_SCHEMA_VERSION}; + use crate::ty::Prim; fn expected_api() -> Api { Api { @@ -183,16 +184,16 @@ mod tests { params: vec![ Param { name: "a".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, Param { name: "b".to_string(), - ty: TypeRef::I32, + ty: TypeRef::Prim(Prim::I32), doc: None, }, ], - returns: Some(TypeRef::I32), + returns: Some(TypeRef::Prim(Prim::I32)), doc: Some("Adds two numbers".to_string()), throws: false, r#async: false, @@ -212,7 +213,7 @@ mod tests { #[test] fn every_format_parses_the_same_document() { let yaml = r#" -version: "0.10.0" +version: "0.11.0" modules: - name: math functions: @@ -226,7 +227,7 @@ modules: doc: "Adds two numbers" "#; let json = r#"{ - "version": "0.10.0", + "version": "0.11.0", "modules": [{ "name": "math", "functions": [{ @@ -241,7 +242,7 @@ modules: }] }"#; let toml_str = r#" -version = "0.10.0" +version = "0.11.0" [[modules]] name = "math" @@ -285,7 +286,7 @@ type = "i32" #[test] fn parse_errors_carry_spans() { - let yaml = "version: \"0.10.0\"\nmodules:\n - name: [oops\n"; + let yaml = "version: \"0.11.0\"\nmodules:\n - name: [oops\n"; match parse_api_str(yaml, "yaml").unwrap_err() { ParseError::Yaml { line, span, .. } => { assert!(line > 0); @@ -305,7 +306,7 @@ type = "i32" #[test] fn unknown_type_syntax_is_a_parse_error() { - let yaml = "version: \"0.10.0\"\nmodules:\n - name: m\n functions:\n - name: f\n params: [{ name: x, type: \"{string}\" }]\n"; + let yaml = "version: \"0.11.0\"\nmodules:\n - name: m\n functions:\n - name: f\n params: [{ name: x, type: \"{string}\" }]\n"; let err = parse_api_str(yaml, "yaml").unwrap_err(); assert!(err.to_string().contains("map type missing"), "{err}"); } diff --git a/crates/weaveffi-model/src/pkg.rs b/crates/weaveffi-model/src/pkg.rs index 54a7c8c4..bce2264c 100644 --- a/crates/weaveffi-model/src/pkg.rs +++ b/crates/weaveffi-model/src/pkg.rs @@ -2,13 +2,12 @@ //! //! The `[package]` table of a project's `weaveffi.toml` ([`Package`]) declares //! what the library is called and how it is published. The CLI resolves it -//! once into an [`Identity`] and attaches it to the [`ResolvedApi`], so the C -//! symbol prefix, the native library name, and every ecosystem's package, -//! module, and namespace name derive from the same values. Nothing a -//! generator emits is named after WeaveFFI itself, so any number of -//! WeaveFFI-built libraries can coexist in one process and one application. -//! -//! [`ResolvedApi`]: crate::ResolvedApi +//! once into an [`Identity`], and validation builds it into the +//! [`Model`](crate::model::Model), so the C symbol prefix, the native library +//! name, and every ecosystem's package, module, and namespace name derive +//! from the same values. Nothing a generator emits is named after WeaveFFI +//! itself, so any number of WeaveFFI-built libraries can coexist in one +//! process and one application. use serde::{Deserialize, Serialize}; @@ -43,8 +42,8 @@ pub struct Package { /// Source repository URL. pub repository: Option, /// C symbol prefix for an IDL-defined library. Defaults to the snake-case - /// package name. A Rust producer's prefix is always its crate name, so - /// setting this for a `.rs` input is an error. + /// package name. A Rust producer's prefix is always its crate's library + /// name, so setting this for a Rust producer is an error. pub c_prefix: Option, /// Base name of the native library (`lib{library}.so`, `{library}.dll`) /// for an IDL-defined library. Defaults to the snake-case package name. diff --git a/crates/weaveffi-model/src/plan.rs b/crates/weaveffi-model/src/plan.rs index f21b9c19..c1c3971e 100644 --- a/crates/weaveffi-model/src/plan.rs +++ b/crates/weaveffi-model/src/plan.rs @@ -8,13 +8,11 @@ //! * **Passing** ([`ArgPass`], [`RetPass`]): how each argument crosses into //! its ABI slots (by value, pinned string/bytes, serialized value buffer, //! borrowed object pointer, or callback context plus vtable) and what the -//! wrapper does with the result (use, copy, decode, or adopt) before any -//! owed release. +//! wrapper does with the result (use, copy, decode, or adopt) and which +//! release it owes (`{prefix}_free_bytes` or the interface's `_destroy`). //! * **Errors** ([`ErrorStrategy`]): when a call reports through `out_err`, //! is that a typed domain error the caller can catch, or a producer bug the //! wrapper must trap on? -//! * **Ownership** ([`RetPass::frees_bytes`]): after copying a returned value -//! into a native one, does the wrapper owe `{prefix}_free_bytes`? //! * **Iterators** ([`IteratorProtocol`]): the pull contract of `iter`, //! including the requirement that wrappers stay **lazy** (one producer //! `next` per consumer step, never a hidden drain into a list). @@ -24,15 +22,17 @@ //! * **Callback interfaces** ([`CallbackProtocol`]): the contract for the //! consumer-implemented vtable the producer calls back into. //! -//! Every classification here derives from [`Ty::family`], so a backend that -//! renders these plans in its own syntax cannot drift from the others on -//! semantics; only the spelling differs. +//! Every classification here derives from [`Ty::family`] and every symbol +//! comes from the [`Model`](crate::model::Model), so a backend that renders +//! these plans in its own syntax cannot drift from the others on semantics; +//! only the spelling differs. -use crate::abi::split_qualified; use crate::abi::AbiParam; use crate::model::{ - AsyncBinding, CallbackInterfaceBinding, Family, FnBinding, IteratorBinding, ParamBinding, Ty, + AsyncBinding, CallbackInterfaceBinding, CallbackMethodBinding, FnBinding, IteratorBinding, + ParamBinding, }; +use crate::ty::{Family, Ty}; /// How a callable's `out_err` slot is interpreted by idiomatic wrappers. /// @@ -112,7 +112,8 @@ pub enum ArgPass<'a> { /// ([`Ty::wire`]), passes the encoding as a borrowed `(ptr, len)` pair, /// and releases its own encoding after the call returns. Any object /// token written into the encoding must be a freshly cloned reference - /// (see [`clone_symbol`]). + /// (the interface's + /// [`clone_symbol`](crate::model::InterfaceBinding::clone_symbol)). Buffer { /// The `const uint8_t*` data slot. ptr: &'a AbiParam, @@ -131,12 +132,15 @@ pub enum ArgPass<'a> { /// A callback interface: the wrapper registers its native implementation /// in a handle table, passes the table key as `ctx` and the interface's /// static vtable as `vtable`, and removes the entry when the producer - /// calls the vtable's `free` (see [`CallbackProtocol`]). + /// calls the vtable's `free` (see [`CallbackProtocol`]). When + /// `nullable`, the IDL type is `Cb?` and a null `vtable` means none. Callback { /// The `void*` context slot. ctx: &'a AbiParam, /// The `const {vtable}*` slot. vtable: &'a AbiParam, + /// `true` for `Cb?`: a null vtable is a legal "none" argument. + nullable: bool, }, } @@ -183,9 +187,13 @@ impl ParamBinding { slot: single(), nullable, }, - Family::Callback => { + Family::Callback { nullable } => { let (ctx, vtable) = pair(); - ArgPass::Callback { ctx, vtable } + ArgPass::Callback { + ctx, + vtable, + nullable, + } } Family::Iterator => unreachable!("iterators are never parameters"), } @@ -215,81 +223,45 @@ pub enum RetPass { /// release with `{prefix}_free_bytes`. Buffer, /// One strong object reference the wrapper adopts into its disposal - /// idiom. When `nullable`, the IDL type is `Interface?` and a null return - /// means none. + /// idiom, eventually released with the interface's `_destroy` symbol + /// ([`InterfaceBinding::destroy_symbol`], from + /// [`Model::interface`](crate::model::Model::interface)). When + /// `nullable`, the IDL type is `Interface?` and a null return means + /// none. + /// + /// [`InterfaceBinding::destroy_symbol`]: crate::model::InterfaceBinding::destroy_symbol Object { - /// The `{prefix}_{module}_{Name}_destroy` symbol the adopted - /// reference eventually owes. - destroy_symbol: String, - /// The `{prefix}_{module}_{Name}_clone` symbol the wrapper calls when - /// it needs a second reference (for example to write the object into - /// a value buffer). - clone_symbol: String, /// `true` for `Interface?`: a null return is a legal "none" result. nullable: bool, }, } impl RetPass { - /// Whether the wrapper owes `{prefix}_free_bytes(ptr, len)` after copying - /// or decoding a *returned* value: true for strings, bytes, and value - /// buffers. Adopted objects owe their `destroy` symbol instead (see - /// [`RetPass::Object`]). Callback-method *arguments* reuse this plan but - /// are borrowed for the call, so a trampoline never frees them. - #[must_use] - pub fn frees_bytes(&self) -> bool { - matches!(self, RetPass::String | RetPass::Bytes | RetPass::Buffer) - } -} - -/// The receiving contract for a value of type `ty` produced by a callable -/// declared inside `module` under `prefix`. `None` (a void return) is -/// [`RetPass::Void`]. -/// -/// # Panics -/// -/// Panics on an iterator return, whose contract is [`IteratorProtocol`], not -/// a value-passing plan (backends dispatch on -/// [`CallShape`](crate::model::CallShape) before consulting this), and on a -/// callback interface, which validation never admits as a return. -pub fn ret_pass(ty: Option<&Ty>, prefix: &str) -> RetPass { - let Some(ty) = ty else { - return RetPass::Void; - }; - match ty.family() { - Family::Direct => RetPass::Direct, - Family::String => RetPass::String, - Family::Bytes => RetPass::Bytes, - Family::Buffer => RetPass::Buffer, - Family::Object { nullable } => { - let iface = ty - .interface_name() - .expect("object family names an interface"); - RetPass::Object { - destroy_symbol: destroy_symbol(iface, prefix), - clone_symbol: clone_symbol(iface, prefix), - nullable, - } + /// The receiving contract for a value of type `ty` produced by a + /// callable. `None` (a void return) is [`RetPass::Void`]. + /// + /// # Panics + /// + /// Panics on an iterator return, whose contract is [`IteratorProtocol`], + /// not a value-passing plan (backends dispatch on + /// [`CallShape`](crate::model::CallShape) before consulting this), and on + /// a callback interface, which validation never admits as a return. + pub fn of(ty: Option<&Ty>) -> RetPass { + let Some(ty) = ty else { + return RetPass::Void; + }; + match ty.family() { + Family::Direct => RetPass::Direct, + Family::String => RetPass::String, + Family::Bytes => RetPass::Bytes, + Family::Buffer => RetPass::Buffer, + Family::Object { nullable } => RetPass::Object { nullable }, + Family::Callback { .. } => panic!("callback interfaces are never returned"), + Family::Iterator => panic!("iterator returns follow IteratorProtocol, not a RetPass"), } - Family::Callback => panic!("callback interfaces are never returned"), - Family::Iterator => panic!("iterator returns follow IteratorProtocol, not a RetPass"), } } -/// The `{prefix}_{module}_{Name}_destroy` symbol for a (possibly -/// dot-qualified) interface name referenced from `current_module`. -pub fn destroy_symbol(name: &str, prefix: &str) -> String { - let (module, name) = split_qualified(name, ""); - format!("{prefix}_{module}_{name}_destroy") -} - -/// The `{prefix}_{module}_{Name}_clone` symbol for a (possibly -/// dot-qualified) interface name referenced from `current_module`. -pub fn clone_symbol(name: &str, prefix: &str) -> String { - let (module, name) = split_qualified(name, ""); - format!("{prefix}_{module}_{name}_clone") -} - /// The `iter` pull contract every backend renders. /// /// The producer returns an opaque iterator handle; the consumer then calls @@ -325,13 +297,11 @@ pub struct IteratorProtocol<'a> { } impl IteratorBinding { - /// Build the full pull contract for this iterator, resolving the element - /// plan against the declaring `module` and `prefix`. - pub fn protocol<'a>(&'a self, f: &FnBinding, prefix: &str) -> IteratorProtocol<'a> { - let elem = ret_pass(Some(&self.elem), prefix); + /// Build the full pull contract for this iterator of `f`. + pub fn protocol<'a>(&'a self, f: &FnBinding) -> IteratorProtocol<'a> { IteratorProtocol { binding: self, - elem, + elem: RetPass::of(Some(&self.elem)), error: f.error_strategy(), } } @@ -373,20 +343,20 @@ pub struct AsyncProtocol<'a> { /// `callback`/`context`. pub cancellable: bool, /// How the callback's result slots are received, including the release - /// owed ([`RetPass::frees_bytes`]) or the destroy symbol an adopted object owes. + /// owed (`{prefix}_free_bytes` or the destroy symbol an adopted object + /// owes). pub result: RetPass, /// How the callback's `err` slot is interpreted. pub error: ErrorStrategy, } impl AsyncBinding { - /// Build the full completion contract for this async function, resolving - /// the result plan against the declaring `module` and `prefix`. - pub fn protocol<'a>(&'a self, f: &FnBinding, prefix: &str) -> AsyncProtocol<'a> { + /// Build the full completion contract for this async function `f`. + pub fn protocol<'a>(&'a self, f: &FnBinding) -> AsyncProtocol<'a> { AsyncProtocol { binding: self, cancellable: f.cancellable, - result: ret_pass(f.ret.as_ref(), prefix), + result: RetPass::of(f.ret.as_ref()), error: f.error_strategy(), } } @@ -395,26 +365,40 @@ impl AsyncBinding { /// The callback-interface contract every backend renders. /// /// A callback interface is the consumer's side of the boundary: the consumer -/// supplies an implementation, the producer calls it. The contract has four +/// supplies an implementation, the producer calls it. The contract has five /// clauses: /// /// 1. **One static vtable per interface.** The wrapper emits exactly one -/// process-wide vtable value for the interface whose entries are -/// trampolines from the C signature ([`CallbackMethodBinding::abi_params`]) -/// into the native implementation, plus a trailing `free` entry. +/// process-wide vtable value for the interface. It starts with the fixed +/// header, `size` (`sizeof` the vtable as the wrapper compiled it), +/// `flags` (`0`), and `free`, followed by one trampoline per method from +/// the C signature ([`CallbackMethodBinding::abi_params`]) into the native +/// implementation. The producer rejects a vtable smaller than the one it +/// was built with, so a stale binding fails with `-3` instead of calling +/// through a missing slot. /// 2. **Context is a handle-table key.** The wrapper stores the native /// implementation in a table keyed by an integer or pointer it passes as /// `ctx`, so the implementation stays alive as long as the producer holds -/// the callback and garbage collectors never see a raw pointer. +/// the callback and garbage collectors never see a raw pointer. `free` +/// may run on any producer thread. /// 3. **Arguments are received like returns.** Strings, bytes, and buffers /// arriving in a trampoline are borrowed for the call: the wrapper copies /// or decodes them before returning and frees nothing. Object arguments -/// transfer one strong reference the wrapper adopts. Method returns are -/// direct values written straight into the C return. -/// 4. **Foreign failures trap.** When the native implementation raises, the -/// trampoline calls `{prefix}_error_set(out_err, -4, message)` and returns -/// a default value; it must never let an exception unwind through the C -/// frame. The producer then aborts its call with `FOREIGN_ERROR_CODE`. +/// transfer one strong reference the wrapper adopts. +/// 4. **Returns transfer to the producer** ([`method_returns`](Self::method_returns)). +/// A direct value is the C return. An object is returned as one strong +/// reference (a fresh `_clone` of the wrapper's handle). A string, bytes, +/// or buffer is written to the trailing `out_ptr`/`out_len` slots as a run +/// the wrapper allocates with `{prefix}_alloc`; the producer adopts it. +/// 5. **Failures go through `out_err`.** When the native implementation +/// raises, the trampoline calls `{prefix}_error_set(out_err, code, +/// message)` and returns a zero value; it must never let an exception +/// unwind through the C frame. A method whose +/// [`method_errors`](Self::method_errors) entry is +/// [`ErrorStrategy::Throws`] reports a declared domain error with its +/// positive code and its payload fields (`{prefix}_error_set_payload`); +/// every other failure uses `-4`, and the producer treats any code it +/// can't attribute to the method's domain as `-4`. /// /// Trampolines may be invoked from any producer thread; the wrapper is /// responsible for whatever thread affinity its runtime demands (a GIL @@ -428,24 +412,47 @@ pub struct CallbackProtocol<'a> { /// How each method's parameters are received inside a trampoline, in /// method order then parameter order. pub method_args: Vec>, + /// How each method's return value crosses back to the producer, in + /// method order. [`RetPass::String`], [`RetPass::Bytes`], and + /// [`RetPass::Buffer`] are written to the `out_ptr`/`out_len` slots as a + /// `{prefix}_alloc` run. + pub method_returns: Vec, + /// How each method may report failure, in method order. + pub method_errors: Vec, +} + +impl CallbackMethodBinding { + /// The error strategy of this method: [`ErrorStrategy::Throws`] when it + /// declares `throws: true`, otherwise [`ErrorStrategy::Trap`] (any + /// failure reaches the producer as `-4`). + pub fn error_strategy(&self) -> ErrorStrategy { + if self.throws { + ErrorStrategy::Throws + } else { + ErrorStrategy::Trap + } + } } impl CallbackInterfaceBinding { - /// Build the full contract for this callback interface, resolving each - /// parameter's receiving plan against the declaring `module` and - /// `prefix`. - pub fn protocol<'a>(&'a self, prefix: &str) -> CallbackProtocol<'a> { + /// Build the full contract for this callback interface. + pub fn protocol(&self) -> CallbackProtocol<'_> { CallbackProtocol { binding: self, method_args: self .methods .iter() - .map(|m| { - m.params - .iter() - .map(|p| ret_pass(Some(&p.ty), prefix)) - .collect() - }) + .map(|m| m.params.iter().map(|p| RetPass::of(Some(&p.ty))).collect()) + .collect(), + method_returns: self + .methods + .iter() + .map(|m| RetPass::of(m.ret.as_ref())) + .collect(), + method_errors: self + .methods + .iter() + .map(CallbackMethodBinding::error_strategy) .collect(), } } @@ -454,87 +461,113 @@ impl CallbackInterfaceBinding { #[cfg(test)] mod tests { use super::*; + use crate::ir::{Api, Function, Param, TypeRef}; + use crate::model::Model; + use crate::pkg::Identity; + use crate::ty::Prim; + + fn param(name: &str, ty: &str) -> Param { + Param { + name: name.into(), + ty: crate::ir::parse_type_ref(ty).unwrap(), + doc: None, + } + } + + /// A model with interface `Store` and callback interface `Listener` in + /// `kv`, a record `Contact`, and one function `f` taking `params`. + fn model(params: Vec) -> Model { + let yaml = r#" +version: "0.11.0" +modules: + - name: kv + structs: [{ name: Contact, fields: [{ name: id, type: i64 }] }] + interfaces: [{ name: Store, methods: [{ name: get }] }] + callback_interfaces: [{ name: Listener, methods: [{ name: on }] }] +"#; + let mut api: Api = serde_yaml::from_str(yaml).unwrap(); + api.modules[0].functions.push(Function { + name: "f".into(), + params, + returns: Some(TypeRef::Prim(Prim::I32)), + doc: None, + throws: false, + r#async: false, + cancellable: false, + deprecated: None, + }); + Model::assume_valid(&api, Identity::named("weaveffi")) + } #[test] fn arg_pass_classifies_every_family() { - let pb = |name: &str, ty: Ty| ParamBinding::new(name, ty, None, "m"); - assert!(matches!( - pb("x", Ty::I32).arg_pass(), - ArgPass::Direct { slot } if slot.name == "x" - )); + let m = model(vec![ + param("x", "i32"), + param("s", "string"), + param("data", "bytes"), + param("c", "Contact"), + param("o", "i32?"), + param("store", "Store"), + param("maybe", "Store?"), + param("l", "Listener"), + param("ml", "Listener?"), + ]); + let p = &m.modules[0].functions[0].params; + assert!(matches!(p[0].arg_pass(), ArgPass::Direct { slot } if slot.name == "x")); + assert!(matches!(p[1].arg_pass(), ArgPass::String { ptr, .. } if ptr.name == "s_ptr")); assert!(matches!( - pb("s", Ty::StringUtf8).arg_pass(), - ArgPass::String { ptr, .. } if ptr.name == "s_ptr" - )); - assert!(matches!( - pb("data", Ty::Bytes).arg_pass(), + p[2].arg_pass(), ArgPass::Bytes { ptr, len } if ptr.name == "data_ptr" && len.name == "data_len" )); assert!(matches!( - pb("c", Ty::Record("Contact".into())).arg_pass(), + p[3].arg_pass(), ArgPass::Buffer { ptr, len } if ptr.name == "c_ptr" && len.name == "c_len" )); + assert!(matches!(p[4].arg_pass(), ArgPass::Buffer { .. })); assert!(matches!( - pb("o", Ty::Optional(Box::new(Ty::I32))).arg_pass(), - ArgPass::Buffer { .. } - )); - assert!(matches!( - pb("store", Ty::Interface("Store".into())).arg_pass(), + p[5].arg_pass(), ArgPass::Object { nullable: false, .. } )); assert!(matches!( - pb( - "store", - Ty::Optional(Box::new(Ty::Interface("Store".into()))) - ) - .arg_pass(), + p[6].arg_pass(), ArgPass::Object { nullable: true, .. } )); assert!(matches!( - pb("l", Ty::CallbackInterface("Listener".into())).arg_pass(), - ArgPass::Callback { ctx, vtable } if ctx.name == "l_ctx" && vtable.name == "l_vtable" + p[7].arg_pass(), + ArgPass::Callback { ctx, vtable, nullable: false } + if ctx.name == "l_ctx" && vtable.name == "l_vtable" + )); + assert!(matches!( + p[8].arg_pass(), + ArgPass::Callback { nullable: true, .. } )); } #[test] fn ret_pass_distinguishes_copy_decode_and_adopt() { - assert_eq!(ret_pass(None, "weaveffi"), RetPass::Void); - assert_eq!(ret_pass(Some(&Ty::I64), "weaveffi"), RetPass::Direct); - assert_eq!(ret_pass(Some(&Ty::StringUtf8), "weaveffi"), RetPass::String); - assert_eq!(ret_pass(Some(&Ty::Bytes), "weaveffi"), RetPass::Bytes); + assert_eq!(RetPass::of(None), RetPass::Void); + assert_eq!(RetPass::of(Some(&Ty::Prim(Prim::I64))), RetPass::Direct); + assert_eq!(RetPass::of(Some(&Ty::Prim(Prim::String))), RetPass::String); + assert_eq!(RetPass::of(Some(&Ty::Prim(Prim::Bytes))), RetPass::Bytes); for ty in [ Ty::Record("Contact".into()), Ty::RichEnum("Shape".into()), - Ty::List(Box::new(Ty::StringUtf8)), + Ty::List(Box::new(Ty::Prim(Prim::String))), Ty::List(Box::new(Ty::Interface("Store".into()))), - Ty::Optional(Box::new(Ty::I64)), + Ty::Optional(Box::new(Ty::Prim(Prim::I64))), ] { - assert_eq!(ret_pass(Some(&ty), "weaveffi"), RetPass::Buffer, "{ty}"); + assert_eq!(RetPass::of(Some(&ty)), RetPass::Buffer, "{ty}"); } assert_eq!( - ret_pass(Some(&Ty::Interface("kv.Store".into())), "weaveffi"), - RetPass::Object { - destroy_symbol: "weaveffi_kv_Store_destroy".into(), - clone_symbol: "weaveffi_kv_Store_clone".into(), - nullable: false, - } + RetPass::of(Some(&Ty::Interface("Store".into()))), + RetPass::Object { nullable: false } ); assert_eq!( - ret_pass( - Some(&Ty::Optional(Box::new(Ty::Interface("kv.Store".into())))), - "weaveffi" - ), - RetPass::Object { - destroy_symbol: "weaveffi_kv_Store_destroy".into(), - clone_symbol: "weaveffi_kv_Store_clone".into(), - nullable: true, - } + RetPass::of(Some(&Ty::Optional(Box::new(Ty::Interface("Store".into()))))), + RetPass::Object { nullable: true } ); - assert!(RetPass::String.frees_bytes()); - assert!(RetPass::Buffer.frees_bytes()); - assert!(!RetPass::Direct.frees_bytes()); } } diff --git a/crates/weaveffi-model/src/resolved.rs b/crates/weaveffi-model/src/resolved.rs deleted file mode 100644 index c6aa4bef..00000000 --- a/crates/weaveffi-model/src/resolved.rs +++ /dev/null @@ -1,431 +0,0 @@ -//! The resolved IR: a validated [`Api`] paired with the type index that turns -//! its written [`TypeRef`]s into resolved [`Ty`]s. -//! -//! WeaveFFI keeps two distinct representations of an API: -//! -//! * the **IDL document**: the [`Api`] tree exactly as parsed from -//! YAML/JSON/TOML (or extracted from annotated Rust), in which every -//! user-type reference is a [`TypeRef::Named`] string; and -//! * the **binding model** ([`crate::model::BindingModel`]): the lowered view -//! generators consume, in which every type is a [`Ty`] whose kind (record, -//! enum, interface) and owning module are known. -//! -//! [`ResolvedApi`] is the bridge. The only checked way to obtain one is -//! [`validate_api`](crate::validate::validate_api), which proves every rule -//! holds; [`ResolvedApi::resolve`] then maps a written reference to its -//! resolved type against the index built from the document's declarations. -//! The document itself is never mutated, so an IDL always round-trips. - -use std::collections::BTreeMap; - -use crate::ir::{Api, Module, TypeRef}; - -use crate::model::Ty; -use crate::pkg::Identity; - -/// What kind of declaration a bare type name refers to. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum TypeKind { - /// A `structs:` entry. - Record, - /// An `enums:` entry with no payload-carrying variant. - Enum, - /// An `enums:` entry with at least one payload-carrying variant. - RichEnum, - /// An `interfaces:` entry. - Interface, - /// A `callback_interfaces:` entry. - CallbackInterface, -} - -/// Where a type is declared: the owning module's dot-joined path and its kind. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct TypeDecl { - /// Dot-joined path of the declaring module (e.g. `graphics.shapes`). - pub module_path: String, - /// The declaration kind. - pub kind: TypeKind, -} - -/// A validated API whose type references can be resolved. -/// -/// This is the input to [`BindingModel::build`](crate::model::BindingModel::build). -/// The checked way to obtain one is -/// [`validate_api`](crate::validate::validate_api). [`assume_valid`](Self::assume_valid) -/// exists for the `#[weaveffi::module]` proc-macro, which expands one module -/// tree in isolation, and for tests that build well-formed trees by hand. -/// -/// `ResolvedApi` dereferences to [`Api`] for read access; there is no mutable -/// access, so the index can never go stale. It also carries the library's -/// [`Identity`], which the CLI attaches with [`with_identity`](Self::with_identity) -/// and every generator reads names from. -#[derive(Debug, Clone, PartialEq)] -pub struct ResolvedApi { - api: Api, - types: BTreeMap, - identity: Identity, -} - -impl ResolvedApi { - /// Attach the library's resolved identity. - #[must_use] - pub fn with_identity(mut self, identity: Identity) -> Self { - self.identity = identity; - self - } - - /// The library's identity: its name, C symbol prefix, native library - /// name, and package metadata. - pub fn identity(&self) -> &Identity { - &self.identity - } - - /// Wrap a document without running validation. - /// - /// Type names that resolve to no declaration become [`Ty::Record`] - /// references named exactly as written; [`unresolved`](Self::unresolved) - /// lists them. The proc-macro relies on this to expand one module tree - /// while a sibling tree declares some of the types it mentions, and - /// asserts at compile time that each such type really crosses as a value - /// buffer. - #[doc(hidden)] - pub fn assume_valid(api: Api) -> Self { - let mut types = BTreeMap::new(); - for module in &api.modules { - index_module(module, "", &mut types); - } - Self { - api, - types, - identity: Identity::default(), - } - } - - /// The underlying document. - pub fn api(&self) -> &Api { - &self.api - } - - /// Look up the declaration a bare type name refers to. - pub fn declaration(&self, name: &str) -> Option<&TypeDecl> { - self.types.get(bare(name)) - } - - /// Every type name the document references that no declaration in it - /// provides, deduplicated and sorted. Always empty for an API that - /// passed validation. - pub fn unresolved(&self) -> Vec { - let mut out = std::collections::BTreeSet::new(); - let mut visit = |ty: &TypeRef| { - ty.walk(&mut |t| { - if let TypeRef::Named(n) = t { - if self.declaration(n).is_none() { - out.insert(n.clone()); - } - } - }); - }; - for_each_type_ref(&self.api.modules, &mut visit); - out.into_iter().collect() - } - - /// The absolute, dot-joined name of the type a reference denotes - /// (`kv.Store`, `shapes.geo.Point`). Unknown names come back as written. - pub fn qualified_name(&self, name: &str) -> String { - match self.declaration(name) { - Some(decl) => format!("{}.{}", decl.module_path, bare(name)), - None => name.to_string(), - } - } - - /// Map a written type reference to its resolved type. User types carry - /// their kind and absolute dot-joined name, so two references to one - /// declaration compare equal wherever they are written. - pub fn resolve(&self, ty: &TypeRef) -> Ty { - match ty { - TypeRef::I8 => Ty::I8, - TypeRef::I16 => Ty::I16, - TypeRef::I32 => Ty::I32, - TypeRef::I64 => Ty::I64, - TypeRef::U8 => Ty::U8, - TypeRef::U16 => Ty::U16, - TypeRef::U32 => Ty::U32, - TypeRef::U64 => Ty::U64, - TypeRef::F32 => Ty::F32, - TypeRef::F64 => Ty::F64, - TypeRef::Bool => Ty::Bool, - TypeRef::StringUtf8 => Ty::StringUtf8, - TypeRef::Bytes => Ty::Bytes, - TypeRef::Named(name) => { - let qualified = self.qualified_name(name); - match self.declaration(name).map(|d| d.kind) { - Some(TypeKind::Enum) => Ty::Enum(qualified), - Some(TypeKind::RichEnum) => Ty::RichEnum(qualified), - Some(TypeKind::Interface) => Ty::Interface(qualified), - Some(TypeKind::CallbackInterface) => Ty::CallbackInterface(qualified), - Some(TypeKind::Record) | None => Ty::Record(qualified), - } - } - TypeRef::Optional(inner) => Ty::Optional(Box::new(self.resolve(inner))), - TypeRef::List(inner) => Ty::List(Box::new(self.resolve(inner))), - TypeRef::Map(k, v) => Ty::Map(Box::new(self.resolve(k)), Box::new(self.resolve(v))), - TypeRef::Iterator(inner) => Ty::Iterator(Box::new(self.resolve(inner))), - } - } -} - -/// Call `f` on every type reference written anywhere in `modules`. -pub(crate) fn for_each_type_ref(modules: &[Module], f: &mut dyn FnMut(&TypeRef)) { - fn fns(fs: &[crate::ir::Function], f: &mut dyn FnMut(&TypeRef)) { - for func in fs { - for p in &func.params { - f(&p.ty); - } - if let Some(r) = &func.returns { - f(r); - } - } - } - for m in modules { - fns(&m.functions, f); - for i in &m.interfaces { - fns(&i.constructors, f); - fns(&i.methods, f); - fns(&i.statics, f); - } - for c in &m.callback_interfaces { - fns(&c.methods, f); - } - for s in &m.structs { - for field in &s.fields { - f(&field.ty); - } - } - for e in &m.enums { - for v in &e.variants { - for field in &v.fields { - f(&field.ty); - } - } - } - if let Some(d) = &m.errors { - for c in &d.codes { - for field in &c.fields { - f(&field.ty); - } - } - } - for_each_type_ref(&m.modules, f); - } -} - -fn bare(name: &str) -> &str { - name.rsplit('.').next().unwrap_or(name) -} - -fn index_module(module: &Module, parent: &str, out: &mut BTreeMap) { - let path = if parent.is_empty() { - module.name.clone() - } else { - format!("{parent}.{}", module.name) - }; - let mut add = |name: &str, kind: TypeKind| { - out.entry(name.to_string()).or_insert(TypeDecl { - module_path: path.clone(), - kind, - }); - }; - for s in &module.structs { - add(&s.name, TypeKind::Record); - } - for e in &module.enums { - let kind = if e.is_rich() { - TypeKind::RichEnum - } else { - TypeKind::Enum - }; - add(&e.name, kind); - } - for i in &module.interfaces { - add(&i.name, TypeKind::Interface); - } - for c in &module.callback_interfaces { - add(&c.name, TypeKind::CallbackInterface); - } - for child in &module.modules { - index_module(child, &path, out); - } -} - -impl std::ops::Deref for ResolvedApi { - type Target = Api; - - fn deref(&self) -> &Api { - &self.api - } -} - -impl AsRef for ResolvedApi { - fn as_ref(&self) -> &Api { - &self.api - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::ir::{ - CallbackInterfaceDef, EnumDef, EnumVariant, Function, InterfaceDef, StructDef, StructField, - CURRENT_SCHEMA_VERSION, - }; - - fn module(name: &str) -> Module { - Module { - name: name.into(), - doc: None, - functions: vec![], - interfaces: vec![], - callback_interfaces: vec![], - structs: vec![], - enums: vec![], - errors: None, - modules: vec![], - } - } - - fn api() -> ResolvedApi { - let shared = Module { - enums: vec![ - EnumDef { - name: "Status".into(), - doc: None, - deprecated: None, - variants: vec![EnumVariant { - name: "Ok".into(), - value: 0, - doc: None, - fields: vec![], - }], - }, - EnumDef { - name: "Shape".into(), - doc: None, - deprecated: None, - variants: vec![EnumVariant { - name: "Circle".into(), - value: 0, - doc: None, - fields: vec![StructField { - name: "r".into(), - ty: TypeRef::F64, - doc: None, - }], - }], - }, - ], - structs: vec![StructDef { - name: "Point".into(), - doc: None, - deprecated: None, - fields: vec![], - }], - modules: vec![Module { - interfaces: vec![InterfaceDef { - name: "Store".into(), - doc: None, - deprecated: None, - constructors: vec![], - methods: vec![], - statics: vec![], - }], - callback_interfaces: vec![CallbackInterfaceDef { - name: "Watcher".into(), - doc: None, - deprecated: None, - methods: vec![Function { - name: "on_change".into(), - params: vec![], - returns: None, - doc: None, - throws: false, - r#async: false, - cancellable: false, - deprecated: None, - }], - }], - ..module("inner") - }], - ..module("shared") - }; - ResolvedApi::assume_valid(Api { - version: CURRENT_SCHEMA_VERSION.into(), - modules: vec![shared, module("orders")], - }) - } - - #[test] - fn kinds_and_qualification() { - let api = api(); - assert_eq!( - api.resolve(&TypeRef::Named("Status".into())), - Ty::Enum("shared.Status".into()) - ); - assert_eq!( - api.resolve(&TypeRef::Named("Status".into())), - Ty::Enum("shared.Status".into()) - ); - assert_eq!( - api.resolve(&TypeRef::Named("Shape".into())), - Ty::RichEnum("shared.Shape".into()) - ); - assert_eq!( - api.resolve(&TypeRef::Named("Store".into())), - Ty::Interface("shared.inner.Store".into()) - ); - assert_eq!( - api.resolve(&TypeRef::Named("Store".into())), - Ty::Interface("shared.inner.Store".into()) - ); - // Already-qualified spellings normalize to the owner's path. - assert_eq!( - api.resolve(&TypeRef::Named("shared.Point".into())), - Ty::Record("shared.Point".into()) - ); - assert_eq!( - api.resolve(&TypeRef::Named("Watcher".into())), - Ty::CallbackInterface("shared.inner.Watcher".into()) - ); - assert_eq!( - api.declaration("Watcher").unwrap().kind, - TypeKind::CallbackInterface - ); - } - - #[test] - fn composites_resolve_recursively() { - let api = api(); - let ty = TypeRef::List(Box::new(TypeRef::Map( - Box::new(TypeRef::StringUtf8), - Box::new(TypeRef::Optional(Box::new(TypeRef::Named("Point".into())))), - ))); - assert_eq!( - api.resolve(&ty), - Ty::List(Box::new(Ty::Map( - Box::new(Ty::StringUtf8), - Box::new(Ty::Optional(Box::new(Ty::Record("shared.Point".into())))) - ))) - ); - } - - #[test] - fn unknown_names_fall_back_to_records() { - let api = api(); - assert_eq!( - api.resolve(&TypeRef::Named("Elsewhere".into())), - Ty::Record("Elsewhere".into()) - ); - assert!(api.unresolved().is_empty()); - assert!(api.declaration("Elsewhere").is_none()); - assert_eq!(api.declaration("Point").unwrap().kind, TypeKind::Record); - } -} diff --git a/crates/weaveffi-model/src/rust.rs b/crates/weaveffi-model/src/rust.rs index 15259eb3..b120f2bd 100644 --- a/crates/weaveffi-model/src/rust.rs +++ b/crates/weaveffi-model/src/rust.rs @@ -1,15 +1,18 @@ -//! Extract a WeaveFFI [`Api`] from annotated Rust source. +//! Extract the IR of a `#[weaveffi::module]` tree from its Rust syntax. //! -//! This module is the single Rust-to-IR bridge shared by two callers: +//! This is the `#[weaveffi::module]` proc-macro's reader: it lowers a module +//! tree to its IR ([`extract_module`]), which the macro validates, builds +//! the C ABI scaffolding from, and embeds in the library as metadata (see +//! [`meta`](crate::meta)). The CLI never reads Rust source; it reads that +//! metadata back out of the built library, so the API it generates bindings +//! for is, by construction, the one the library was compiled with. //! -//! * the `#[weaveffi::module]` proc-macro, which lowers a module to its IR to -//! build the C ABI scaffolding at compile time; and -//! * the `weaveffi` CLI, which reads the same annotated source to drive -//! `generate`/`extract`. -//! -//! Because both paths call into the *same* extraction, the IDL the CLI emits -//! and the symbols the macro produces cannot drift: they are two views of one -//! parse. Annotated Rust is therefore the single source of truth. +//! The extraction reports only what syntax alone rules out (a raw pointer, +//! a tuple variant, a `#[cfg]` on a member); every other rule is the +//! validator's. Alongside the IR, [`extract_module`] returns a [`SourceMap`] +//! recording where each declaration came from and the `#[cfg]` attributes on +//! it, so the macro can point a validation error at the offending item and +//! apply each item's `#[cfg]` to its generated code. //! //! # The annotation scheme //! @@ -31,7 +34,11 @@ //! * `#[weaveffi::error]` on an enum declares the module's error domain. //! * `#[weaveffi::callback_interface]` on a `trait` declares a callback //! interface: a method set the consumer implements. Producers accept one as -//! `Arc`. +//! `Arc` (or `Option>`). Every method returns +//! `Result`, which maps to a `T` return; a +//! method marked `#[weaveffi::throws]` throws. +//! * `pub type Name = T;` in the tree is an alias: every use of `Name` is +//! extracted as `T`. //! //! The type mapping mirrors the IDL: `String` (or `&str`) is a string, `Vec` //! (or `&[u8]`) is a byte buffer, every integer primitive (including `u64`) is @@ -41,12 +48,120 @@ //! names the callback interface `Trait`, and any other named path is a record, //! enum, or interface resolved later. +use std::collections::{BTreeMap, HashMap}; + use crate::ir::{ - Api, CallbackInterfaceDef, EnumDef, EnumVariant, ErrorCode, ErrorDomain, Function, - InterfaceDef, Module, Param, StructDef, StructField, TypeRef, CURRENT_SCHEMA_VERSION, + CallbackInterfaceDef, EnumDef, EnumVariant, ErrorCode, ErrorDomain, Function, InterfaceDef, + Module, Param, StructDef, StructField, TypeRef, }; +use crate::ty::Prim; +use proc_macro2::Span; use syn::spanned::Spanned; +/// Where each extracted declaration came from, keyed by its declaration +/// path: module segments from the tree's root, then the declaration name, +/// then a member name (an interface's constructor, method, or static; a +/// callback interface's method; an enum's variant; an error domain's code), +/// then a parameter or field name. These are the paths validation reports +/// its errors under. +/// +/// A parameter's or field's path extended with [`TYPE`](Self::TYPE) locates +/// its written type, and a callable's path extended with +/// [`RETURN`](Self::RETURN) its written return type. +#[derive(Clone, Default)] +pub struct SourceMap { + spans: BTreeMap, Vec>, + cfgs: BTreeMap, Vec>, +} + +impl std::fmt::Debug for SourceMap { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("SourceMap") + .field("paths", &self.spans.keys().collect::>()) + .finish_non_exhaustive() + } +} + +impl SourceMap { + /// The key segment that, appended to a parameter's or field's path, + /// names its written type. + pub const TYPE: &'static str = "#type"; + /// The key segment that, appended to a callable's path, names its + /// written return type. + pub const RETURN: &'static str = "#return"; + + fn record(&mut self, path: Vec, span: Span) { + self.spans.entry(path).or_default().push(span); + } + + fn record_cfg(&mut self, path: Vec, attrs: &[syn::Attribute]) { + let cfg: Vec = attrs.iter().filter(|a| is_cfg(a)).cloned().collect(); + if !cfg.is_empty() { + self.cfgs.entry(path).or_default().extend(cfg); + } + } + + /// Every span recorded for `path`, in source order (a duplicated name + /// has several). + #[must_use] + pub fn spans(&self, path: &[String]) -> &[Span] { + self.spans.get(path).map_or(&[], Vec::as_slice) + } + + /// The `#[cfg]` attributes written on the declaration at `path` itself + /// (for an interface member, on its `impl` block), not including its + /// enclosing modules' or interface's. + #[must_use] + pub fn cfg(&self, path: &[String]) -> &[syn::Attribute] { + self.cfgs.get(path).map_or(&[], Vec::as_slice) + } +} + +/// Whether `attr` is a `#[cfg(...)]`. +fn is_cfg(attr: &syn::Attribute) -> bool { + attr.path().is_ident("cfg") +} + +/// Reject a `#[cfg]` on a member of an exported item. +fn no_member_cfg(attrs: &[syn::Attribute], what: &str, instead: &str) -> syn::Result<()> { + match attrs.iter().find(|a| is_cfg(a)) { + Some(attr) => Err(syn::Error::new( + attr.span(), + format!( + "weaveffi: `#[cfg]` on {what} isn't supported, because the generated bindings \ + can't follow it; {instead}" + ), + )), + None => Ok(()), + } +} + +/// The type aliases declared in a module tree (`pub type Id = u64;`), by +/// name. Generic aliases are left alone. +type Aliases = HashMap; + +fn collect_aliases(item_mod: &syn::ItemMod, out: &mut Aliases) { + let Some((_, items)) = &item_mod.content else { + return; + }; + for item in items { + match item { + syn::Item::Type(t) if t.generics.params.is_empty() => { + out.insert(t.ident.to_string(), (*t.ty).clone()); + } + syn::Item::Mod(m) if has_marker(&m.attrs, "module") => collect_aliases(m, out), + _ => {} + } + } +} + +/// The extraction context: the tree's aliases and the source map being +/// built. +struct Extractor<'a> { + aliases: &'a Aliases, + map: SourceMap, +} + /// Match a WeaveFFI marker attribute by its final path segment. /// /// Accepts both the namespaced form (`#[weaveffi::record]`) and the bare form @@ -96,6 +211,10 @@ fn parse_deprecated(attrs: &[syn::Attribute]) -> (Option, Option (since, note) } +/// The doc comment of an item, one line per `///` line with the leading +/// space removed, and with rustdoc intra-doc links unwrapped to the inline +/// code they display (see [`unwrap_doc_links`]), since no binding resolves +/// Rust paths. fn extract_doc(attrs: &[syn::Attribute]) -> Option { let lines: Vec = attrs .iter() @@ -120,7 +239,56 @@ fn extract_doc(attrs: &[syn::Attribute]) -> Option { }) }) .collect(); - (!lines.is_empty()).then(|| lines.join("\n")) + if lines.is_empty() { + return None; + } + let mut fenced = false; + let lines: Vec = lines + .into_iter() + .map(|line| { + if line.trim_start().starts_with("```") { + fenced = !fenced; + line + } else if fenced { + line + } else { + unwrap_doc_links(&line) + } + }) + .collect(); + Some(lines.join("\n")) +} + +/// Unwrap every rustdoc intra-doc link written as inline code in brackets +/// (`` [`Store`] ``, `` [`KvError::KeyNotFound`] ``) to the inline code +/// alone. A bracketed span followed by `(`, `[`, or `:` is an ordinary +/// Markdown link or link definition, and stays as written. +fn unwrap_doc_links(line: &str) -> String { + let mut out = String::with_capacity(line.len()); + let mut rest = line; + while let Some(start) = rest.find("[`") { + let after = &rest[start + 2..]; + let Some(end) = after.find('`') else { + break; + }; + let code = &after[..end]; + let tail = &after[end + 1..]; + let link = tail.starts_with(']') + && !code.is_empty() + && !matches!(tail[1..].chars().next(), Some('(' | '[' | ':')); + if link { + out.push_str(&rest[..start]); + out.push('`'); + out.push_str(code); + out.push('`'); + rest = &tail[1..]; + } else { + out.push_str(&rest[..start + 2]); + rest = after; + } + } + out.push_str(rest); + out } fn is_ident(ty: &syn::Type, name: &str) -> bool { @@ -216,7 +384,8 @@ fn is_cancel_token(ty: &syn::Type) -> bool { type_path_ident(ty).as_deref() == Some("CancelToken") } -/// Map a Rust [`syn::Type`] onto the WeaveFFI [`TypeRef`] it represents. +/// Map a Rust [`syn::Type`] onto the WeaveFFI [`TypeRef`] it represents, +/// substituting the tree's `aliases`. /// /// This is the canonical mapping every caller shares. Notable conventions: /// @@ -236,21 +405,26 @@ fn is_cancel_token(ty: &syn::Type) -> bool { /// Returns a spanned error for type syntax WeaveFFI cannot express across the /// FFI boundary (raw pointers, `Box`, tuples, a generic with the /// wrong arity, and so on). -pub fn type_ref_from_syn(ty: &syn::Type) -> syn::Result { +fn type_ref_from_syn(ty: &syn::Type, aliases: &Aliases) -> syn::Result { + type_ref_at(ty, aliases, 0) +} + +fn type_ref_at(ty: &syn::Type, aliases: &Aliases, depth: usize) -> syn::Result { + let recurse = |t: &syn::Type| type_ref_at(t, aliases, depth); match ty { syn::Type::Reference(r) => { if let syn::Type::Path(p) = r.elem.as_ref() { if p.path.is_ident("str") { - return Ok(TypeRef::StringUtf8); + return Ok(TypeRef::Prim(Prim::String)); } } if let syn::Type::Slice(slice) = r.elem.as_ref() { if is_ident(&slice.elem, "u8") { - return Ok(TypeRef::Bytes); + return Ok(TypeRef::Prim(Prim::Bytes)); } - return Ok(TypeRef::List(Box::new(type_ref_from_syn(&slice.elem)?))); + return Ok(TypeRef::List(Box::new(recurse(&slice.elem)?))); } - type_ref_from_syn(&r.elem) + recurse(&r.elem) } syn::Type::Ptr(_) => Err(syn::Error::new( ty.span(), @@ -274,23 +448,19 @@ pub fn type_ref_from_syn(ty: &syn::Type) -> syn::Result { .last() .ok_or_else(|| syn::Error::new(ty.span(), "empty type path"))?; let ident = seg.ident.to_string(); + // A Rust scalar (`i32`, `bool`, ...) is the primitive of the same + // name; `string` and `bytes` are IDL spellings, not Rust types. + if let Some(p) = + Prim::from_name(&ident).filter(|p| !matches!(p, Prim::String | Prim::Bytes)) + { + return Ok(TypeRef::Prim(p)); + } match ident.as_str() { - "i8" => Ok(TypeRef::I8), - "i16" => Ok(TypeRef::I16), - "i32" => Ok(TypeRef::I32), - "i64" => Ok(TypeRef::I64), - "u8" => Ok(TypeRef::U8), - "u16" => Ok(TypeRef::U16), - "u32" => Ok(TypeRef::U32), - "f32" => Ok(TypeRef::F32), - "f64" => Ok(TypeRef::F64), - "bool" => Ok(TypeRef::Bool), - "String" => Ok(TypeRef::StringUtf8), - "u64" => Ok(TypeRef::U64), + "String" => Ok(TypeRef::Prim(Prim::String)), // `Arc` is a reference to the interface object `T`, and // `Arc` a consumer-implemented callback interface; // the IR names the pointee in both cases. - "Arc" => type_ref_from_syn(single_generic_arg(seg)?), + "Arc" => recurse(single_generic_arg(seg)?), "Box" | "Rc" => Err(syn::Error::new( ty.span(), format!( @@ -301,28 +471,36 @@ pub fn type_ref_from_syn(ty: &syn::Type) -> syn::Result { "Vec" => { let inner = single_generic_arg(seg)?; if is_ident(inner, "u8") { - return Ok(TypeRef::Bytes); + return Ok(TypeRef::Prim(Prim::Bytes)); } - Ok(TypeRef::List(Box::new(type_ref_from_syn(inner)?))) + Ok(TypeRef::List(Box::new(recurse(inner)?))) } "Option" => { let inner = single_generic_arg(seg)?; - Ok(TypeRef::Optional(Box::new(type_ref_from_syn(inner)?))) + Ok(TypeRef::Optional(Box::new(recurse(inner)?))) } // `weaveffi::Iter` is the producer spelling of an `iter` // return: a lazily-pulled stream rather than a materialized list. "Iter" => { let inner = single_generic_arg(seg)?; - Ok(TypeRef::Iterator(Box::new(type_ref_from_syn(inner)?))) + Ok(TypeRef::Iterator(Box::new(recurse(inner)?))) } "HashMap" | "BTreeMap" => { let (k, v) = two_generic_args(seg)?; - Ok(TypeRef::Map( - Box::new(type_ref_from_syn(k)?), - Box::new(type_ref_from_syn(v)?), - )) + Ok(TypeRef::Map(Box::new(recurse(k)?), Box::new(recurse(v)?))) } - other => Ok(TypeRef::Named(other.to_string())), + other => match aliases.get(other) { + Some(target) if matches!(seg.arguments, syn::PathArguments::None) => { + if depth > 16 { + return Err(syn::Error::new( + ty.span(), + format!("weaveffi: type alias `{other}` refers to itself"), + )); + } + type_ref_at(target, aliases, depth + 1) + } + _ => Ok(TypeRef::Named(other.to_string())), + }, } } _ => Err(syn::Error::new(ty.span(), "unsupported type syntax")), @@ -361,26 +539,6 @@ fn is_unit(ty: &syn::Type) -> bool { matches!(ty, syn::Type::Tuple(t) if t.elems.is_empty()) } -/// Map a function's return type to its IDL return [`TypeRef`], peeling -/// `Result` and treating `()` (and `Result<(), E>`) as no return. -/// -/// # Errors -/// -/// Propagates any error from [`type_ref_from_syn`] on the (peeled) return type. -pub fn return_type_from_syn(output: &syn::ReturnType) -> syn::Result> { - match output { - syn::ReturnType::Default => Ok(None), - syn::ReturnType::Type(_, ty) => { - let inner = peel_result(ty); - if is_unit(inner) { - Ok(None) - } else { - Ok(Some(type_ref_from_syn(inner)?)) - } - } - } -} - fn parse_discriminant(expr: &syn::Expr) -> syn::Result { match expr { syn::Expr::Lit(lit) => { @@ -402,129 +560,37 @@ fn parse_discriminant(expr: &syn::Expr) -> syn::Result { } } -fn extract_params( - inputs: &syn::punctuated::Punctuated, -) -> syn::Result> { - inputs - .iter() - .filter_map(|arg| match arg { - syn::FnArg::Typed(pt) => Some(pt), - syn::FnArg::Receiver(_) => None, - }) - // The cancellation token is part of the async calling convention, not a - // logical parameter, so it never appears in the IDL. - .filter(|pt| !is_cancel_token(&pt.ty)) - .map(|pt| { - let param_name = match pt.pat.as_ref() { - syn::Pat::Ident(id) => id.ident.to_string(), - _ => return Err(syn::Error::new(pt.span(), "unsupported parameter pattern")), - }; - if matches!(pt.ty.as_ref(), syn::Type::Reference(r) if r.mutability.is_some()) { - return Err(syn::Error::new( - pt.ty.span(), - "weaveffi: `&mut` parameters cannot cross the FFI boundary; take the value \ - by `&T` or by value and return the updated result", - )); - } - let ty = type_ref_from_syn(&pt.ty)?; - // The proc macro path skips `validate_api` (see the macro crate), - // so the one type shape lowering can't represent as a parameter - // must be caught here with a span instead of panicking later. - let mut has_iterator = false; - ty.walk(&mut |t| has_iterator |= matches!(t, TypeRef::Iterator(_))); - if has_iterator { - return Err(syn::Error::new( - pt.ty.span(), - "weaveffi: `Iter` is a pull handle the consumer drives and is only \ - valid as a function's outermost return type, not as a parameter", - )); - } - Ok(Param { - name: param_name, - ty, - doc: extract_doc(&pt.attrs), - }) - }) - .collect() -} - -/// Map one `fn` signature (free function, interface member, or callback -/// method) to the IR [`Function`]. `returns` is supplied by the caller because -/// constructors and `Self` returns need interface-aware handling. -fn function_from_sig( - sig: &syn::Signature, - attrs: &[syn::Attribute], - returns: Option, -) -> syn::Result { - if let Some(ret) = &returns { - let nested_iterator = |t: &TypeRef| { - let mut found = false; - t.walk(&mut |inner| found |= matches!(inner, TypeRef::Iterator(_))); - found - }; - let invalid = match ret { - TypeRef::Iterator(elem) => nested_iterator(elem), - other => nested_iterator(other), - }; - if invalid { - let span = match &sig.output { - syn::ReturnType::Type(_, ty) => ty.span(), - other => other.span(), - }; - return Err(syn::Error::new( - span, - "weaveffi: `Iter` is only valid as the outermost return type; it can't be \ - nested in an `Option`, `Vec`, map, or another `Iter`", - )); - } - } - Ok(Function { - name: sig.ident.to_string(), - params: extract_params(&sig.inputs)?, - returns, - doc: extract_doc(attrs), - throws: output_is_result(&sig.output), - r#async: sig.asyncness.is_some(), - cancellable: has_marker(attrs, "cancellable"), - deprecated: parse_deprecated(attrs).1, - }) -} - -fn extract_function(item: &syn::ItemFn) -> syn::Result { - let returns = return_type_from_syn(&item.sig.output)?; - function_from_sig(&item.sig, &item.attrs, returns) +/// `path` extended with `names`. +fn at(path: &[String], names: &[&str]) -> Vec { + let mut out = path.to_vec(); + out.extend(names.iter().map(|n| (*n).to_string())); + out } -/// Whether the (peeled) return type names the interface itself (`Self`, -/// `Arc`, or the interface's own name), which classifies an associated -/// function as a constructor. -fn returns_self(output: &syn::ReturnType, iface: &str) -> bool { +/// Whether a return type is `Result`, the return type every +/// callback-interface method must have. +pub fn returns_foreign_result(output: &syn::ReturnType) -> bool { let syn::ReturnType::Type(_, ty) = output else { return false; }; - match type_path_ident(peel_arc(peel_result(ty))) { - Some(name) => name == "Self" || name == iface, - None => false, + let syn::Type::Path(p) = ty.as_ref() else { + return false; + }; + let Some(seg) = p.path.segments.last() else { + return false; + }; + if seg.ident != "Result" { + return false; } -} - -/// Extract one interface member from an `impl` block function. -/// -/// The receiver decides the member kind at the call site (see -/// [`members_from_impl`]); this helper maps the signature. A `Self` (or -/// interface-named) return on a constructor is dropped: the IR leaves a -/// constructor's `return` empty because the instance is implicit. -fn extract_member(item: &syn::ImplItemFn, iface: &str, is_ctor: bool) -> syn::Result { - let returns = if is_ctor { - None - } else { - match return_type_from_syn(&item.sig.output)? { - // `fn hand(&self) -> Arc` style returns name the interface. - Some(TypeRef::Named(name)) if name == "Self" => Some(TypeRef::Named(iface.to_string())), - other => other, - } + let syn::PathArguments::AngleBracketed(args) = &seg.arguments else { + return false; }; - function_from_sig(&item.sig, &item.attrs, returns) + match args.args.iter().nth(1) { + Some(syn::GenericArgument::Type(err)) => { + type_path_ident(err).as_deref() == Some("ForeignError") + } + _ => false, + } } /// Whether a method receiver is one WeaveFFI can lift: `&self` (a borrow for @@ -543,205 +609,335 @@ pub fn receiver_is_supported(recv: &syn::Receiver) -> bool { recv.colon_token.is_some() && type_path_ident(&recv.ty).as_deref() == Some("Arc") } -/// Classify and extract every `pub fn` of an interface's `impl` block into -/// the interface's constructors, methods, and statics. -/// -/// * a function with a `&self` or `self: Arc` receiver is a **method**; -/// * an associated function returning `Self` or `Arc` (or the interface -/// type, optionally inside `Result`) is a **constructor**; -/// * any other associated function is a **static**. -/// -/// Non-`pub` items are private helpers and stay unexported. -fn members_from_impl(item_impl: &syn::ItemImpl, iface: &mut InterfaceDef) -> syn::Result<()> { - for impl_item in &item_impl.items { - let syn::ImplItem::Fn(f) = impl_item else { - continue; - }; - if !matches!(f.vis, syn::Visibility::Public(_)) { - continue; - } - match f.sig.receiver() { - Some(recv) => { - if !receiver_is_supported(recv) { - return Err(syn::Error::new( - recv.span(), - "weaveffi: interface methods must take `&self` or `self: Arc`; \ - use interior mutability (Mutex, RwLock, atomics) for mutable state, \ - because the object is shared across the FFI boundary", - )); +/// Whether the (peeled) return type names the interface itself (`Self`, +/// `Arc`, or the interface's own name), which classifies a +/// synchronous associated function as a constructor. +fn returns_self(output: &syn::ReturnType, iface: &str) -> bool { + let syn::ReturnType::Type(_, ty) = output else { + return false; + }; + match type_path_ident(peel_arc(peel_result(ty))) { + Some(name) => name == "Self" || name == iface, + None => false, + } +} + +impl Extractor<'_> { + fn ty(&self, ty: &syn::Type) -> syn::Result { + type_ref_from_syn(ty, self.aliases) + } + + /// Map a function's return type to its IDL return [`TypeRef`], peeling + /// `Result` and treating `()` (and `Result<(), E>`) as no return. + fn return_type(&self, output: &syn::ReturnType) -> syn::Result> { + match output { + syn::ReturnType::Default => Ok(None), + syn::ReturnType::Type(_, ty) => { + let inner = peel_result(ty); + if is_unit(inner) { + Ok(None) + } else { + Ok(Some(self.ty(inner)?)) } - iface.methods.push(extract_member(f, &iface.name, false)?); } - None if returns_self(&f.sig.output, &iface.name) => { - iface - .constructors - .push(extract_member(f, &iface.name, true)?); + } + } + + fn params( + &mut self, + path: &[String], + inputs: &syn::punctuated::Punctuated, + ) -> syn::Result> { + let mut out = Vec::new(); + for pt in inputs.iter().filter_map(|arg| match arg { + syn::FnArg::Typed(pt) => Some(pt), + syn::FnArg::Receiver(_) => None, + }) { + // The cancellation token is part of the async calling + // convention, not a logical parameter, so it never appears in + // the IDL. + if is_cancel_token(&pt.ty) { + continue; } - None => { - iface.statics.push(extract_member(f, &iface.name, false)?); + let name = match pt.pat.as_ref() { + syn::Pat::Ident(id) => id.ident.to_string(), + _ => return Err(syn::Error::new(pt.span(), "unsupported parameter pattern")), + }; + if matches!(pt.ty.as_ref(), syn::Type::Reference(r) if r.mutability.is_some()) { + return Err(syn::Error::new( + pt.ty.span(), + "weaveffi: `&mut` parameters cannot cross the FFI boundary; take the value \ + by `&T` or by value and return the updated result", + )); } + self.map.record(at(path, &[&name]), pt.pat.span()); + self.map + .record(at(path, &[&name, SourceMap::TYPE]), pt.ty.span()); + out.push(Param { + ty: self.ty(&pt.ty)?, + name, + doc: extract_doc(&pt.attrs), + }); } + Ok(out) } - Ok(()) -} -/// Whether a return type is `Result`: a callback method's -/// way of receiving the consumer's failure as a value. The `Result` is a -/// producer-side convention (the IDL method still returns `T` and doesn't -/// throw), so it's peeled without marking the method `throws`. -pub fn returns_foreign_result(output: &syn::ReturnType) -> bool { - let syn::ReturnType::Type(_, ty) = output else { - return false; - }; - let syn::Type::Path(p) = ty.as_ref() else { - return false; - }; - let Some(seg) = p.path.segments.last() else { - return false; - }; - if seg.ident != "Result" { - return false; + /// Map one `fn` signature (free function, interface member, or callback + /// method) declared at `path` to the IR [`Function`]. `returns` is + /// supplied by the caller because constructors, `Self` returns, and + /// callback methods need their own handling. + fn function( + &mut self, + path: &[String], + sig: &syn::Signature, + attrs: &[syn::Attribute], + returns: Option, + ) -> syn::Result { + let ret_span = match &sig.output { + syn::ReturnType::Type(_, ty) => ty.span(), + syn::ReturnType::Default => sig.ident.span(), + }; + self.map.record(at(path, &[SourceMap::RETURN]), ret_span); + Ok(Function { + name: sig.ident.to_string(), + params: self.params(path, &sig.inputs)?, + returns, + doc: extract_doc(attrs), + throws: output_is_result(&sig.output), + r#async: sig.asyncness.is_some(), + cancellable: has_marker(attrs, "cancellable"), + deprecated: parse_deprecated(attrs).1, + }) } - let syn::PathArguments::AngleBracketed(args) = &seg.arguments else { - return false; - }; - match args.args.iter().nth(1) { - Some(syn::GenericArgument::Type(err)) => { - type_path_ident(err).as_deref() == Some("ForeignError") + + /// Extract one interface member from an `impl` block function. A `Self` + /// (or interface-named) return on a constructor is dropped: the IR + /// leaves a constructor's `return` empty because the instance is + /// implicit. + fn member( + &mut self, + path: &[String], + item: &syn::ImplItemFn, + iface: &str, + is_ctor: bool, + ) -> syn::Result { + let returns = if is_ctor { + None + } else { + match self.return_type(&item.sig.output)? { + // `fn hand(&self) -> Arc` style returns name the + // interface. + Some(TypeRef::Named(name)) if name == "Self" => { + Some(TypeRef::Named(iface.to_string())) + } + other => other, + } + }; + let member = at(path, &[&item.sig.ident.to_string()]); + self.map.record(member.clone(), item.sig.ident.span()); + self.function(&member, &item.sig, &item.attrs, returns) + } + + /// Classify and extract every `pub fn` of an interface's `impl` block + /// into the interface's constructors, methods, and statics: + /// + /// * a function with a `&self` or `self: Arc` receiver is a + /// **method**; + /// * a synchronous associated function returning `Self` or `Arc` + /// (or the interface type, optionally inside `Result`) is a + /// **constructor**; + /// * any other associated function is a **static**, including an + /// `async fn` returning the interface (constructors are synchronous, + /// so that's an async factory). + /// + /// Non-`pub` items are private helpers and stay unexported. The block's + /// `#[cfg]` is recorded on each member it declares. + fn members( + &mut self, + path: &[String], + item_impl: &syn::ItemImpl, + iface: &mut InterfaceDef, + ) -> syn::Result<()> { + for impl_item in &item_impl.items { + let syn::ImplItem::Fn(f) = impl_item else { + continue; + }; + if !matches!(f.vis, syn::Visibility::Public(_)) { + continue; + } + no_member_cfg( + &f.attrs, + "an interface member", + "move the member into its own `impl` block and put the `#[cfg]` on that block", + )?; + let member = at(path, &[&f.sig.ident.to_string()]); + self.map.record_cfg(member, &item_impl.attrs); + match f.sig.receiver() { + Some(recv) => { + if !receiver_is_supported(recv) { + return Err(syn::Error::new( + recv.span(), + "weaveffi: interface methods must take `&self` or `self: Arc`; \ + use interior mutability (Mutex, RwLock, atomics) for mutable state, \ + because the object is shared across the FFI boundary", + )); + } + let m = self.member(path, f, &iface.name, false)?; + iface.methods.push(m); + } + None if f.sig.asyncness.is_none() && returns_self(&f.sig.output, &iface.name) => { + let m = self.member(path, f, &iface.name, true)?; + iface.constructors.push(m); + } + None => { + let m = self.member(path, f, &iface.name, false)?; + iface.statics.push(m); + } + } } - _ => false, + Ok(()) } -} -/// Extract a `#[weaveffi::callback_interface]` trait into a -/// [`CallbackInterfaceDef`]. -/// -/// Every trait method is a callback method the consumer implements. Methods -/// must take `&self`. A method may return `Result` -/// to receive the consumer's failure as a value; that maps to a plain `T` -/// return. The validator rejects `async`, other `Result` returns, and -/// non-direct return types, so those are mapped faithfully here and reported -/// with the IDL-level diagnostics. -fn extract_callback_interface(item: &syn::ItemTrait) -> syn::Result { - let mut methods = Vec::new(); - for trait_item in &item.items { - let syn::TraitItem::Fn(f) = trait_item else { - continue; - }; - match f.sig.receiver() { - Some(recv) if recv.reference.is_some() && recv.mutability.is_none() => {} - Some(recv) => { - return Err(syn::Error::new( - recv.span(), - "weaveffi: callback interface methods must take `&self`", - )); + /// Extract a `#[weaveffi::callback_interface]` trait into a + /// [`CallbackInterfaceDef`]. + /// + /// Every trait method is a callback method the consumer implements. A + /// method takes `&self` and returns `Result`, + /// which maps to a `T` return; `#[weaveffi::throws]` marks it throwing. + fn callback_interface( + &mut self, + path: &[String], + item: &syn::ItemTrait, + ) -> syn::Result { + let mut methods = Vec::new(); + for trait_item in &item.items { + let syn::TraitItem::Fn(f) = trait_item else { + continue; + }; + no_member_cfg( + &f.attrs, + "a callback interface method", + "put the `#[cfg]` on the whole trait", + )?; + match f.sig.receiver() { + Some(recv) if recv.reference.is_some() && recv.mutability.is_none() => {} + Some(recv) => { + return Err(syn::Error::new( + recv.span(), + "weaveffi: callback interface methods must take `&self`", + )); + } + None => { + return Err(syn::Error::new( + f.sig.span(), + "weaveffi: callback interface methods must take `&self` (associated \ + functions can't be implemented by the consumer)", + )); + } } - None => { + if !returns_foreign_result(&f.sig.output) { + let span = match &f.sig.output { + syn::ReturnType::Type(_, ty) => ty.span(), + syn::ReturnType::Default => f.sig.ident.span(), + }; return Err(syn::Error::new( - f.sig.span(), - "weaveffi: callback interface methods must take `&self` (associated \ - functions can't be implemented by the consumer)", + span, + "weaveffi: a callback interface method must return \ + `Result` (or `Result<(), weaveffi::ForeignError>` \ + for no value), because the consumer's implementation can fail and the \ + `Err` is how its failure reaches you", )); } + let returns = self.return_type(&f.sig.output)?; + let method_path = at(path, &[&f.sig.ident.to_string()]); + self.map.record(method_path.clone(), f.sig.ident.span()); + let mut method = self.function(&method_path, &f.sig, &f.attrs, returns)?; + method.throws = has_marker(&f.attrs, "throws"); + methods.push(method); } - let returns = return_type_from_syn(&f.sig.output)?; - let mut method = function_from_sig(&f.sig, &f.attrs, returns)?; - if returns_foreign_result(&f.sig.output) { - method.throws = false; - } - methods.push(method); + Ok(CallbackInterfaceDef { + name: item.ident.to_string(), + doc: extract_doc(&item.attrs), + deprecated: parse_deprecated(&item.attrs).1, + methods, + }) } - Ok(CallbackInterfaceDef { - name: item.ident.to_string(), - doc: extract_doc(&item.attrs), - deprecated: parse_deprecated(&item.attrs).1, - methods, - }) -} -fn extract_struct(item: &syn::ItemStruct) -> syn::Result { - let name = item.ident.to_string(); - let fields = match &item.fields { - syn::Fields::Named(named) => named - .named - .iter() - .map(|f| { - let field_name = f - .ident - .as_ref() - .ok_or_else(|| syn::Error::new(f.span(), "unnamed field in record"))? - .to_string(); - Ok(StructField { - name: field_name, - ty: type_ref_from_syn(&f.ty)?, - doc: extract_doc(&f.attrs), - }) - }) - .collect::>()?, - _ => { - return Err(syn::Error::new( - item.span(), - "only named fields are supported for #[weaveffi::record]", - )) - } - }; - Ok(StructDef { - name, - doc: extract_doc(&item.attrs), - deprecated: parse_deprecated(&item.attrs).1, - fields, - }) -} - -/// Extract the named fields of one rich (algebraic) enum variant into the IR's -/// [`StructField`] list, the same shape a record's fields take. -fn extract_variant_fields(fields: &syn::FieldsNamed) -> syn::Result> { - fields - .named - .iter() - .map(|f| { - let field_name = f + /// The named fields of a record, a rich enum variant, or an error code, + /// declared under `path`. + fn fields( + &mut self, + path: &[String], + fields: &syn::FieldsNamed, + what: &str, + ) -> syn::Result> { + let mut out = Vec::new(); + for f in &fields.named { + no_member_cfg(&f.attrs, what, "put the `#[cfg]` on the whole type")?; + let name = f .ident .as_ref() - .ok_or_else(|| syn::Error::new(f.span(), "unnamed field in enum variant"))? + .ok_or_else(|| syn::Error::new(f.span(), "unnamed field"))? .to_string(); - Ok(StructField { - name: field_name, - ty: type_ref_from_syn(&f.ty)?, + self.map.record(at(path, &[&name]), f.span()); + self.map + .record(at(path, &[&name, SourceMap::TYPE]), f.ty.span()); + out.push(StructField { + ty: self.ty(&f.ty)?, + name, doc: extract_doc(&f.attrs), - }) + }); + } + Ok(out) + } + + fn record(&mut self, path: &[String], item: &syn::ItemStruct) -> syn::Result { + let syn::Fields::Named(named) = &item.fields else { + return Err(syn::Error::new( + item.span(), + "only named fields are supported for #[weaveffi::record]", + )); + }; + Ok(StructDef { + name: item.ident.to_string(), + doc: extract_doc(&item.attrs), + deprecated: parse_deprecated(&item.attrs).1, + fields: self.fields(path, named, "a record field")?, }) - .collect() -} + } -fn extract_enum(item: &syn::ItemEnum) -> syn::Result { - let name = item.ident.to_string(); + fn enumeration(&mut self, path: &[String], item: &syn::ItemEnum) -> syn::Result { + let name = item.ident.to_string(); - // A *rich* (algebraic) enum has at least one variant carrying data. Rust - // forbids explicit discriminants on such an enum, so its tags are the - // declaration-order positions (0, 1, 2, …) - exactly what the IDL records. - // A *C-style* enum (every variant fieldless) keeps the stricter contract: - // it must be `#[repr(i32)]` with an explicit discriminant on each variant. - let is_rich = item - .variants - .iter() - .any(|v| !matches!(v.fields, syn::Fields::Unit)); + // A *rich* (algebraic) enum has at least one variant carrying data. + // Rust forbids explicit discriminants on such an enum, so its tags + // are the declaration-order positions (0, 1, 2, ...), exactly what + // the IDL records. A *C-style* enum (every variant fieldless) keeps + // the stricter contract: it must be `#[repr(i32)]` with an explicit + // discriminant on each variant. + let is_rich = item + .variants + .iter() + .any(|v| !matches!(v.fields, syn::Fields::Unit)); - if !is_rich && !has_repr_i32(&item.attrs) { - return Err(syn::Error::new( - item.span(), - format!( - "enum `{}` must have #[repr(i32)] to be a #[weaveffi::enumeration]", - item.ident - ), - )); - } + if !is_rich && !has_repr_i32(&item.attrs) { + return Err(syn::Error::new( + item.span(), + format!( + "enum `{}` must have #[repr(i32)] to be a #[weaveffi::enumeration]", + item.ident + ), + )); + } - let mut next_value: i32 = 0; - let variants = item - .variants - .iter() - .map(|v| { + let mut next_value: i32 = 0; + let mut variants = Vec::new(); + for v in &item.variants { + no_member_cfg( + &v.attrs, + "an enum variant", + "put the `#[cfg]` on the whole enum", + )?; let value = match v.discriminant.as_ref() { Some((_, expr)) => parse_discriminant(expr)?, None if is_rich => next_value, @@ -756,58 +952,64 @@ fn extract_enum(item: &syn::ItemEnum) -> syn::Result { } }; next_value = value.wrapping_add(1); - + let variant = v.ident.to_string(); + let vpath = at(path, &[&variant]); + self.map.record(vpath.clone(), v.ident.span()); let fields = match &v.fields { syn::Fields::Unit => vec![], - syn::Fields::Named(named) => extract_variant_fields(named)?, + syn::Fields::Named(named) => self.fields(&vpath, named, "an enum variant field")?, syn::Fields::Unnamed(_) => { return Err(syn::Error::new( v.span(), format!( - "enum `{name}` variant `{}`: tuple-style variants are not supported; \ - use named fields", - v.ident + "enum `{name}` variant `{variant}`: tuple-style variants are not \ + supported; use named fields" ), )) } }; - - Ok(EnumVariant { - name: v.ident.to_string(), + variants.push(EnumVariant { + name: variant, value, doc: extract_doc(&v.attrs), fields, - }) + }); + } + Ok(EnumDef { + name, + doc: extract_doc(&item.attrs), + deprecated: parse_deprecated(&item.attrs).1, + variants, }) - .collect::>()?; - Ok(EnumDef { - name, - doc: extract_doc(&item.attrs), - deprecated: parse_deprecated(&item.attrs).1, - variants, - }) -} + } -/// Extract a `#[weaveffi::error]` enum into the module's [`ErrorDomain`]. -/// -/// Every variant needs an explicit integer discriminant: the code's stable -/// ABI value. A variant may be a unit variant or carry named fields, which -/// become the code's structured payload (serialized in the value-buffer -/// format alongside the `(code, message)` pair). Note that Rust requires a -/// primitive representation such as `#[repr(i32)]` on an enum that mixes -/// explicit discriminants with data-carrying variants. A variant's doc -/// comment becomes the code's default message (falling back to the variant -/// name). The enum's name is the domain name, and the macro generates a -/// matching `ErrorReport` implementation so `Err(KvError::...)` reports its -/// declared code and payload. -fn extract_error_domain(item: &syn::ItemEnum) -> syn::Result { - let codes = item - .variants - .iter() - .map(|v| { + /// Extract a `#[weaveffi::error]` enum into the module's + /// [`ErrorDomain`]. + /// + /// Every variant needs an explicit integer discriminant: the code's + /// stable ABI value. A variant may be a unit variant or carry named + /// fields, which become the code's structured payload (serialized in the + /// value-buffer format alongside the `(code, message)` pair). Note that + /// Rust requires a primitive representation such as `#[repr(i32)]` on an + /// enum that mixes explicit discriminants with data-carrying variants. A + /// variant's doc comment becomes the code's default message (falling + /// back to the variant name). The enum's name is the domain name, and + /// the macro generates matching `ErrorReport` and `ErrorDomain` + /// implementations. + fn error_domain(&mut self, path: &[String], item: &syn::ItemEnum) -> syn::Result { + let mut codes = Vec::new(); + for v in &item.variants { + no_member_cfg( + &v.attrs, + "an error code", + "put the `#[cfg]` on the whole error enum", + )?; + let code = v.ident.to_string(); + let cpath = at(path, &[&code]); + self.map.record(cpath.clone(), v.ident.span()); let fields = match &v.fields { syn::Fields::Unit => vec![], - syn::Fields::Named(named) => extract_variant_fields(named)?, + syn::Fields::Named(named) => self.fields(&cpath, named, "an error code field")?, syn::Fields::Unnamed(_) => { return Err(syn::Error::new( v.span(), @@ -820,9 +1022,8 @@ fn extract_error_domain(item: &syn::ItemEnum) -> syn::Result { return Err(syn::Error::new( v.span(), format!( - "weaveffi: error variant `{}` must have an explicit discriminant \ - (its stable ABI error code)", - v.ident + "weaveffi: error variant `{code}` must have an explicit discriminant \ + (its stable ABI error code)" ), )); }; @@ -830,56 +1031,72 @@ fn extract_error_domain(item: &syn::ItemEnum) -> syn::Result { let message = doc .as_deref() .and_then(|d| d.lines().next()) - .unwrap_or(&v.ident.to_string()) + .unwrap_or(&code) .to_string(); - Ok(ErrorCode { - name: v.ident.to_string(), + codes.push(ErrorCode { code: parse_discriminant(expr)?, + name: code, message, doc, fields, - }) + }); + } + Ok(ErrorDomain { + name: item.ident.to_string(), + codes, }) - .collect::>>()?; - Ok(ErrorDomain { - name: item.ident.to_string(), - codes, - }) -} + } -/// Extract a single [`Module`] from a `#[weaveffi::module]`-annotated `mod`. -/// -/// Only items carrying a WeaveFFI marker are exported; everything else (private -/// helpers, `use` items, free functions without `#[weaveffi::export]`) is -/// ignored, so a module can freely mix exported surface and implementation. -/// -/// # Errors -/// -/// Returns a spanned error when an annotated item cannot be mapped to the IR -/// (an unsupported type, an enum without `#[repr(i32)]`, a callback interface -/// method without `&self`, and so on). -pub fn module_from_item_mod(item_mod: &syn::ItemMod) -> syn::Result { - let name = item_mod.ident.to_string(); - let mut functions = Vec::new(); - let mut interfaces: Vec = Vec::new(); - let mut structs = Vec::new(); - let mut enums = Vec::new(); - let mut callback_interfaces = Vec::new(); - let mut modules = Vec::new(); - let mut errors: Option = None; - - if let Some((_, items)) = &item_mod.content { + /// Extract one `#[weaveffi::module]`, whose parent path is `parent`. + fn module(&mut self, parent: &[String], item_mod: &syn::ItemMod) -> syn::Result { + let name = item_mod.ident.to_string(); + let path = at(parent, &[&name]); + self.map.record(path.clone(), item_mod.ident.span()); + self.map.record_cfg(path.clone(), &item_mod.attrs); + let mut module = Module { + name, + doc: extract_doc(&item_mod.attrs), + functions: Vec::new(), + interfaces: Vec::new(), + callback_interfaces: Vec::new(), + structs: Vec::new(), + enums: Vec::new(), + errors: None, + modules: Vec::new(), + }; + let Some((_, items)) = &item_mod.content else { + return Err(syn::Error::new( + item_mod.span(), + "weaveffi: a #[weaveffi::module] must have an inline body (`mod name { ... }`); \ + the macro can't read a module from another file", + )); + }; + // Records a declaration's span and `#[cfg]`. + let declare = |map: &mut SourceMap, decl: &str, span: Span, attrs: &[syn::Attribute]| { + let decl = at(&path, &[decl]); + map.record(decl.clone(), span); + map.record_cfg(decl, attrs); + }; for item in items { match item { syn::Item::Trait(t) if has_marker(&t.attrs, "callback_interface") => { - callback_interfaces.push(extract_callback_interface(t)?); + let decl = t.ident.to_string(); + declare(&mut self.map, &decl, t.ident.span(), &t.attrs); + let cb = self.callback_interface(&at(&path, &[&decl]), t)?; + module.callback_interfaces.push(cb); } syn::Item::Fn(f) if has_marker(&f.attrs, "export") => { - functions.push(extract_function(f)?); + let decl = f.sig.ident.to_string(); + declare(&mut self.map, &decl, f.sig.ident.span(), &f.attrs); + let returns = self.return_type(&f.sig.output)?; + let func = self.function(&at(&path, &[&decl]), &f.sig, &f.attrs, returns)?; + module.functions.push(func); } syn::Item::Struct(s) if has_marker(&s.attrs, "interface") => { - interfaces.push(InterfaceDef { - name: s.ident.to_string(), + let decl = s.ident.to_string(); + declare(&mut self.map, &decl, s.ident.span(), &s.attrs); + module.interfaces.push(InterfaceDef { + name: decl, doc: extract_doc(&s.attrs), deprecated: parse_deprecated(&s.attrs).1, constructors: vec![], @@ -888,23 +1105,32 @@ pub fn module_from_item_mod(item_mod: &syn::ItemMod) -> syn::Result { }); } syn::Item::Struct(s) if has_marker(&s.attrs, "record") => { - structs.push(extract_struct(s)?); + let decl = s.ident.to_string(); + declare(&mut self.map, &decl, s.ident.span(), &s.attrs); + let record = self.record(&at(&path, &[&decl]), s)?; + module.structs.push(record); } syn::Item::Enum(e) if has_marker(&e.attrs, "error") => { - if errors.is_some() { + if module.errors.is_some() { return Err(syn::Error::new( e.span(), "weaveffi: a module may declare at most one #[weaveffi::error] \ domain", )); } - errors = Some(extract_error_domain(e)?); + let decl = e.ident.to_string(); + declare(&mut self.map, &decl, e.ident.span(), &e.attrs); + module.errors = Some(self.error_domain(&at(&path, &[&decl]), e)?); } syn::Item::Enum(e) if has_marker(&e.attrs, "enumeration") => { - enums.push(extract_enum(e)?); + let decl = e.ident.to_string(); + declare(&mut self.map, &decl, e.ident.span(), &e.attrs); + let def = self.enumeration(&at(&path, &[&decl]), e)?; + module.enums.push(def); } - syn::Item::Mod(m) if has_marker(&m.attrs, "module") && m.content.is_some() => { - modules.push(module_from_item_mod(m)?); + syn::Item::Mod(m) if has_marker(&m.attrs, "module") => { + let child = self.module(&path, m)?; + module.modules.push(child); } _ => {} } @@ -923,70 +1149,74 @@ pub fn module_from_item_mod(item_mod: &syn::ItemMod) -> syn::Result { let Some(self_name) = type_path_ident(&item_impl.self_ty) else { continue; }; - if let Some(iface) = interfaces.iter_mut().find(|i| i.name == self_name) { - members_from_impl(item_impl, iface)?; + if let Some(i) = module.interfaces.iter().position(|i| i.name == self_name) { + let mut iface = std::mem::replace( + &mut module.interfaces[i], + InterfaceDef { + name: String::new(), + doc: None, + deprecated: None, + constructors: vec![], + methods: vec![], + statics: vec![], + }, + ); + let result = self.members(&at(&path, &[&self_name]), item_impl, &mut iface); + module.interfaces[i] = iface; + result?; } } + Ok(module) } - - Ok(Module { - name, - doc: extract_doc(&item_mod.attrs), - functions, - interfaces, - structs, - enums, - callback_interfaces, - errors, - modules, - }) } -/// Extract a full [`Api`] from a parsed Rust [`syn::File`], collecting every -/// top-level `#[weaveffi::module]`. -/// -/// # Errors +/// Extract the module tree rooted at a `#[weaveffi::module]`-annotated +/// `mod`, together with its [`SourceMap`]. /// -/// Propagates any error from [`module_from_item_mod`]. -pub fn api_from_file(file: &syn::File) -> syn::Result { - let mut modules = Vec::new(); - for item in &file.items { - if let syn::Item::Mod(item_mod) = item { - if has_marker(&item_mod.attrs, "module") && item_mod.content.is_some() { - modules.push(module_from_item_mod(item_mod)?); - } - } - } - Ok(Api { - version: CURRENT_SCHEMA_VERSION.to_string(), - modules, - }) -} - -/// Parse Rust source text and extract its [`Api`]. +/// Only items carrying a WeaveFFI marker are exported; everything else +/// (private helpers, `use` items, free functions without +/// `#[weaveffi::export]`) is ignored, so a module can freely mix exported +/// surface and implementation. Type aliases declared anywhere in the tree are +/// substituted. /// /// # Errors /// -/// Returns a (line/column aware) error when the source does not parse as Rust, -/// or when an annotated item cannot be mapped to the IR. -pub fn api_from_src(src: &str) -> syn::Result { - let file = syn::parse_file(src)?; - api_from_file(&file) -} - -/// A convenience wrapper around [`api_from_src`] that renders any error to a -/// plain string, for non-macro callers (e.g. the CLI). -/// -/// # Errors -/// -/// Returns the formatted error message when extraction fails. -pub fn api_from_src_stringly(src: &str) -> Result { - api_from_src(src).map_err(|e| e.to_string()) +/// Returns a spanned error when an annotated item cannot be mapped to the IR +/// (an unsupported type, an enum without `#[repr(i32)]`, a callback interface +/// method that doesn't return `Result`, a +/// `#[cfg]` on a member, an out-of-line submodule, and so on). +pub fn extract_module(item_mod: &syn::ItemMod) -> syn::Result<(Module, SourceMap)> { + let mut aliases = Aliases::new(); + collect_aliases(item_mod, &mut aliases); + let mut extractor = Extractor { + aliases: &aliases, + map: SourceMap::default(), + }; + let module = extractor.module(&[], item_mod)?; + Ok((module, extractor.map)) } #[cfg(test)] mod tests { use super::*; + use crate::ir::{Api, CURRENT_SCHEMA_VERSION}; + + /// Extract every top-level `#[weaveffi::module]` in `src`. + fn api_from_src(src: &str) -> syn::Result { + let file = syn::parse_file(src)?; + let mut modules = Vec::new(); + for item in &file.items { + if let syn::Item::Mod(item_mod) = item { + if has_marker(&item_mod.attrs, "module") { + modules.push(extract_module(item_mod)?.0); + } + } + } + Ok(Api { + version: CURRENT_SCHEMA_VERSION.to_string(), + modules, + }) + } fn one_module(src: &str) -> Module { let api = api_from_src(src).unwrap(); @@ -1009,8 +1239,8 @@ mod tests { assert_eq!(m.functions.len(), 1); let f = &m.functions[0]; assert_eq!(f.name, "add"); - assert_eq!(f.params[0].ty, TypeRef::I32); - assert_eq!(f.returns, Some(TypeRef::I32)); + assert_eq!(f.params[0].ty, TypeRef::Prim(Prim::I32)); + assert_eq!(f.returns, Some(TypeRef::Prim(Prim::I32))); assert!(!f.r#async); } @@ -1043,7 +1273,7 @@ mod tests { m.functions[0].returns, Some(TypeRef::Named("Contact".into())) ); - assert_eq!(m.functions[0].params[0].ty, TypeRef::U64); + assert_eq!(m.functions[0].params[0].ty, TypeRef::Prim(Prim::U64)); } #[test] @@ -1099,7 +1329,7 @@ mod tests { let s = &m.structs[0]; assert_eq!( s.fields[1].ty, - TypeRef::Optional(Box::new(TypeRef::StringUtf8)) + TypeRef::Optional(Box::new(TypeRef::Prim(Prim::String))) ); assert_eq!(s.fields[2].ty, TypeRef::Named("ContactType".into())); } @@ -1133,11 +1363,11 @@ mod tests { "#, ); let open = &m.functions[0]; - assert_eq!(open.params[0].ty, TypeRef::StringUtf8); - assert_eq!(open.params[1].ty, TypeRef::Bytes); + assert_eq!(open.params[0].ty, TypeRef::Prim(Prim::String)); + assert_eq!(open.params[1].ty, TypeRef::Prim(Prim::Bytes)); assert_eq!( open.params[2].ty, - TypeRef::List(Box::new(TypeRef::StringUtf8)) + TypeRef::List(Box::new(TypeRef::Prim(Prim::String))) ); assert_eq!(open.returns, Some(TypeRef::Named("Store".into()))); let peek = &m.functions[1]; @@ -1233,6 +1463,74 @@ mod tests { assert_eq!(s.statics.len(), 1); } + #[test] + fn doc_links_unwrap_to_inline_code() { + assert_eq!( + unwrap_doc_links("Fails with [`KvError::KeyNotFound`] for [`Store`]."), + "Fails with `KvError::KeyNotFound` for `Store`." + ); + assert_eq!( + unwrap_doc_links("See [`docs`](https://x.dev) and [`a`][b]; [`c`]: d"), + "See [`docs`](https://x.dev) and [`a`][b]; [`c`]: d" + ); + assert_eq!( + unwrap_doc_links("A `[string]`, [`]`, [x]."), + "A `[string]`, [`]`, [x]." + ); + assert_eq!(unwrap_doc_links("open [`"), "open [`"); + + let m = one_module( + r#" + /// Uses [`Store`]. + #[weaveffi::module] + mod m { + /// Returns [`Store::size`] items. + /// + /// ``` + /// let v = [`x`]; + /// ``` + #[weaveffi::export] + pub fn count() -> i32 { 0 } + } + "#, + ); + assert_eq!( + m.functions[0].doc.as_deref(), + Some("Returns `Store::size` items.\n\n```\nlet v = [`x`];\n```") + ); + } + + #[test] + fn async_associated_functions_returning_the_interface_are_statics() { + let m = one_module( + r#" + #[weaveffi::module] + mod m { + #[weaveffi::interface] + pub struct Store; + impl Store { + pub async fn open_async(path: String) -> Result, StoreError> { todo!() } + pub async fn fresh() -> Self { Store } + pub async fn named() -> Store { Store } + } + } + "#, + ); + let s = &m.interfaces[0]; + assert!(s.constructors.is_empty()); + assert_eq!(s.statics.len(), 3); + for f in &s.statics { + assert!(f.r#async, "{} is async", f.name); + assert_eq!( + f.returns, + Some(TypeRef::Named("Store".into())), + "{}", + f.name + ); + } + assert!(s.statics[0].throws); + } + #[test] fn mut_self_receiver_is_rejected() { let err = api_from_src( @@ -1260,13 +1558,15 @@ mod tests { /// Receives messages. #[weaveffi::callback_interface] pub trait Listener: Send + Sync { - fn on_message(&self, text: String, meta: Option); - fn level(&self) -> i32; + fn on_message(&self, text: String, meta: Option) -> Result<(), ForeignError>; + fn level(&self) -> Result; + #[weaveffi::throws] + fn label(&self) -> Result; } #[weaveffi::export] pub fn subscribe(listener: Arc) {} #[weaveffi::export] - pub fn subscribe_bounded(listener: Arc) {} + pub fn subscribe_bounded(listener: Option>) {} } "#, ); @@ -1274,40 +1574,124 @@ mod tests { let cb = &m.callback_interfaces[0]; assert_eq!(cb.name, "Listener"); assert_eq!(cb.doc.as_deref(), Some("Receives messages.")); - assert_eq!(cb.methods.len(), 2); - assert_eq!(cb.methods[0].params.len(), 2); - assert_eq!(cb.methods[1].returns, Some(TypeRef::I32)); + let methods = &cb.methods; + assert_eq!(methods[0].params.len(), 2); + assert_eq!(methods[0].returns, None); + assert!(!methods[0].throws); + assert_eq!(methods[1].returns, Some(TypeRef::Prim(Prim::I32))); + assert!(!methods[1].throws); + assert_eq!(methods[2].returns, Some(TypeRef::Prim(Prim::String))); + assert!(methods[2].throws); assert_eq!( m.functions[0].params[0].ty, TypeRef::Named("Listener".into()) ); assert_eq!( m.functions[1].params[0].ty, - TypeRef::Named("Listener".into()) + TypeRef::Optional(Box::new(TypeRef::Named("Listener".into()))) ); } #[test] - fn callback_methods_may_return_foreign_results() { + fn callback_methods_must_return_foreign_results() { + for ret in ["", "-> i32", "-> Result"] { + let src = format!( + "#[weaveffi::module] mod m {{ #[weaveffi::callback_interface] \ + pub trait L {{ fn f(&self) {ret}; }} }}" + ); + let err = api_from_src(&src).unwrap_err(); + assert!(err.to_string().contains("ForeignError"), "{ret}: {err}"); + } + } + + #[test] + fn aliases_are_substituted() { let m = one_module( r#" #[weaveffi::module] mod m { - #[weaveffi::callback_interface] - pub trait Listener: Send + Sync { - fn level(&self) -> Result; - fn ping(&self) -> Result<(), ForeignError>; - fn strict(&self) -> Result; + pub type Id = u64; + pub type Ids = Vec; + #[weaveffi::export] + pub fn get(id: Id, all: Ids) -> Option { None } + #[weaveffi::module] + mod inner { + #[weaveffi::export] + pub fn deep(id: super::Id) {} } } "#, ); - let methods = &m.callback_interfaces[0].methods; - assert_eq!(methods[0].returns, Some(TypeRef::I32)); - assert!(!methods[0].throws); - assert_eq!(methods[1].returns, None); - assert!(!methods[1].throws); - assert!(methods[2].throws, "other error types still throw"); + let f = &m.functions[0]; + assert_eq!(f.params[0].ty, TypeRef::Prim(Prim::U64)); + assert_eq!( + f.params[1].ty, + TypeRef::List(Box::new(TypeRef::Prim(Prim::U64))) + ); + assert_eq!( + f.returns, + Some(TypeRef::Optional(Box::new(TypeRef::Prim(Prim::U64)))) + ); + assert_eq!( + m.modules[0].functions[0].params[0].ty, + TypeRef::Prim(Prim::U64) + ); + } + + #[test] + fn member_cfgs_and_out_of_line_modules_are_rejected() { + for (src, needle) in [ + ( + "#[weaveffi::module] mod m { #[weaveffi::record] pub struct R { #[cfg(unix)] pub x: i32 } }", + "record field", + ), + ( + "#[weaveffi::module] mod m { #[weaveffi::interface] pub struct S; impl S { #[cfg(unix)] pub fn f(&self) {} } }", + "interface member", + ), + ( + "#[weaveffi::module] mod m { #[weaveffi::enumeration] #[repr(i32)] pub enum E { #[cfg(unix)] A = 0 } }", + "enum variant", + ), + ( + "#[weaveffi::module] mod m { #[weaveffi::module] mod inner; }", + "inline body", + ), + ] { + let err = api_from_src(src).unwrap_err().to_string(); + assert!(err.contains(needle), "{src}: {err}"); + } + } + + #[test] + fn the_source_map_records_spans_and_item_cfgs() { + let file: syn::File = syn::parse_str( + r#" + #[weaveffi::module] + mod m { + #[cfg(unix)] + #[weaveffi::export] + pub fn f(x: usize) {} + #[weaveffi::interface] + pub struct S; + #[cfg(feature = "s")] + impl S { + pub fn get(&self) -> i32 { 0 } + } + } + "#, + ) + .unwrap(); + let syn::Item::Mod(item) = &file.items[0] else { + panic!("a module") + }; + let (_, map) = extract_module(item).unwrap(); + let path = |p: &[&str]| p.iter().map(ToString::to_string).collect::>(); + assert_eq!(map.spans(&path(&["m", "f"])).len(), 1); + assert_eq!(map.spans(&path(&["m", "f", "x", SourceMap::TYPE])).len(), 1); + assert_eq!(map.cfg(&path(&["m", "f"])).len(), 1); + assert_eq!(map.cfg(&path(&["m", "S", "get"])).len(), 1); + assert!(map.cfg(&path(&["m", "S"])).is_empty()); } #[test] diff --git a/crates/weaveffi-model/src/model/ty.rs b/crates/weaveffi-model/src/ty.rs similarity index 61% rename from crates/weaveffi-model/src/model/ty.rs rename to crates/weaveffi-model/src/ty.rs index 6a4a7d29..375415a9 100644 --- a/crates/weaveffi-model/src/model/ty.rs +++ b/crates/weaveffi-model/src/ty.rs @@ -1,13 +1,15 @@ -//! The **resolved type** every backend consumes, and the single taxonomy that -//! classifies it for the C ABI and the value-buffer wire format. +//! The **resolved type** every backend consumes, the single taxonomy that +//! classifies it for the C ABI and the value-buffer wire format, and the +//! [`TypeIndex`] that maps each user type name to its declaration. //! -//! The IDL's [`TypeRef`](crate::ir::TypeRef) is the type *as written*: -//! a user-defined type is a bare `Named` string because the parser cannot -//! know whether it names a record, an enum, an interface, or a callback +//! The IDL's [`TypeRef`](crate::ir::TypeRef) is the type *as written*: a +//! user-defined type is a bare `Named` string because the parser can't know +//! whether it names a record, an enum, an interface, or a callback //! interface. [`Ty`] is the type *as resolved* by validation: every user -//! reference carries its kind, cross-module references are qualified with the -//! owner's dot-joined module path, and there is no "unresolved" variant for a -//! backend to trip over. +//! reference carries its kind, and there is no "unresolved" variant for a +//! backend to trip over. Type names are global (validation rejects two +//! declarations with one name), so a [`Ty`] carries the bare name and the +//! [`TypeIndex`] answers where it's declared. //! //! Three questions about a type are answered once here: //! @@ -21,132 +23,49 @@ //! * [`Ty::contains_user_type`] and [`Ty::contains_object`]: whether encoding //! the type needs a user-defined codec function or object-token support. +use std::collections::BTreeMap; use std::fmt; -/// A fully resolved type: the shape generators render and the ABI lowers. -#[derive(Debug, Clone, PartialEq, Eq, Hash)] -pub enum Ty { - /// Signed 8-bit integer (`i8`). - I8, - /// Signed 16-bit integer (`i16`). - I16, - /// Signed 32-bit integer (`i32`). - I32, - /// Signed 64-bit integer (`i64`). - I64, - /// Unsigned 8-bit integer (`u8`). - U8, - /// Unsigned 16-bit integer (`u16`). - U16, - /// Unsigned 32-bit integer (`u32`). - U32, - /// Unsigned 64-bit integer (`u64`). - U64, - /// 32-bit IEEE 754 floating-point number (`f32`). - F32, - /// 64-bit IEEE 754 floating-point number (`f64`). - F64, - /// Boolean (`bool`). - Bool, - /// UTF-8 string (`string`). Borrowed as a parameter, owned as a return. - StringUtf8, - /// Byte buffer (`bytes`). Borrowed as a parameter, owned as a return. - Bytes, - /// A user record (struct): a plain value type. Crosses the C ABI by value - /// as a serialized buffer (`ptr` + `len`), borrowed for a call as a - /// parameter and owned (freed with `{prefix}_free_bytes`) as a return. - /// The name is dot-qualified when the record lives in another module. - Record(String), - /// An algebraic (rich) enum: a sum type with at least one payload-carrying - /// variant. A value type that crosses the C ABI as a serialized buffer - /// (an `i32` tag followed by the active variant's fields), exactly like a - /// [`Record`](Self::Record). - RichEnum(String), - /// A C-style integer enum (no variant payloads). Lowers by value. - Enum(String), - /// A user interface: a reference-counted object. As a parameter the - /// object is borrowed for the call; as a return (or inside a buffer) the - /// receiver adopts one strong reference it must eventually release. - Interface(String), - /// A callback interface: a method set the consumer implements. Only valid - /// as a top-level parameter, where it lowers to a context pointer plus a - /// vtable pointer. - CallbackInterface(String), - /// Optional value (`T?`): either the inner type or nothing. - Optional(Box), - /// Homogeneous list (`[T]`) of the inner element type. - List(Box), - /// Map (`{K:V}`) from a key type to a value type. - Map(Box, Box), - /// Lazy sequence (`iter`) of the inner type, lowered to a next/destroy - /// iterator object rather than a materialized collection. - Iterator(Box), -} - -/// How a value of some [`Ty`] crosses a **call boundary**: the one -/// classification the ABI lowering, the marshalling plan, and every backend's -/// argument and return handling agree on. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum Family { - /// One C slot passed by value: scalars, bools, and C-style enums. - Direct, - /// A `(ptr, len)` pair of UTF-8 bytes, never NUL-terminated. Borrowed for - /// a call as a parameter; producer-owned and released with - /// `{prefix}_free_bytes` as a return. - String, - /// A `(ptr, len)` raw byte pair. Borrowed as a parameter; producer-owned - /// and released with `{prefix}_free_bytes` as a return. - Bytes, - /// A `(ptr, len)` serialized value buffer (record, rich enum, optional, - /// list, or map). Borrowed as a parameter; producer-owned and released - /// with `{prefix}_free_bytes` after decoding as a return. - Buffer, - /// An object pointer to an interface. Borrowed as a parameter; one - /// strong reference (released with its `_destroy` symbol) as a return. - Object { - /// `true` for `Interface?`: null is a legal "none" value. - nullable: bool, - }, - /// A callback interface: a `void* ctx` plus `const {tag}_vtable*` pair. - /// Only ever a parameter. - Callback, - /// An `iter` return: an opaque iterator handle with its own - /// `next`/`destroy` protocol. Never a parameter. - Iterator, -} - -/// A fixed-width wire primitive inside a value buffer. +/// A built-in primitive type: the fixed vocabulary the IDL's +/// [`TypeRef`](crate::ir::TypeRef) and the resolved [`Ty`] share. /// -/// Every backend spells the read/write routine for one of these the same way -/// modulo casing (`read_i32`, `readI32`, `ReadI32`), so the shared name -/// helpers let a codec emitter dispatch on the whole set with one arm. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +/// Adding a primitive means adding a variant here; every classification +/// ([`Ty::family`], [`Ty::wire`], the C ABI lowering) dispatches on it. +/// Inside a value buffer each primitive is one fixed-width or +/// length-prefixed run (see the variant docs), and every backend spells the +/// read/write routine for one the same way modulo casing (`read_i32`, +/// `readI32`, `ReadI32`), so [`snake`](Self::snake) and +/// [`pascal`](Self::pascal) let a codec emitter dispatch on the whole set +/// with one arm. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] pub enum Prim { - /// One byte, `0` or `1`. + /// Boolean (`bool`): one byte, `0` or `1`. Bool, - /// One signed byte. + /// Signed 8-bit integer (`i8`): one signed byte. I8, - /// Two bytes, little-endian, signed. + /// Signed 16-bit integer (`i16`): two bytes, little-endian. I16, - /// Four bytes, little-endian, signed. + /// Signed 32-bit integer (`i32`): four bytes, little-endian. I32, - /// Eight bytes, little-endian, signed. + /// Signed 64-bit integer (`i64`): eight bytes, little-endian. I64, - /// One unsigned byte. + /// Unsigned 8-bit integer (`u8`): one unsigned byte. U8, - /// Two bytes, little-endian, unsigned. + /// Unsigned 16-bit integer (`u16`): two bytes, little-endian. U16, - /// Four bytes, little-endian, unsigned. + /// Unsigned 32-bit integer (`u32`): four bytes, little-endian. U32, - /// Eight bytes, little-endian, unsigned. + /// Unsigned 64-bit integer (`u64`): eight bytes, little-endian. U64, - /// Four bytes, IEEE 754 bits. + /// 32-bit IEEE 754 floating-point number (`f32`): four bytes. F32, - /// Eight bytes, IEEE 754 bits. + /// 64-bit IEEE 754 floating-point number (`f64`): eight bytes. F64, - /// A `u32` byte length followed by UTF-8 bytes, no NUL terminator. + /// UTF-8 string (`string`): a `u32` byte length followed by UTF-8 bytes, + /// no NUL terminator. Borrowed as a parameter, owned as a return. String, - /// A `u32` length followed by raw bytes. + /// Byte buffer (`bytes`): a `u32` length followed by raw bytes. Borrowed + /// as a parameter, owned as a return. Bytes, } @@ -168,8 +87,16 @@ impl Prim { Prim::Bytes, ]; - /// The lower-case spelling (`bool`, `i32`, `string`, `bytes`), the stem - /// of snake-case routine names such as `read_i32`. + /// The primitive an IDL spelling names (`i32`, `string`, `bytes`), or + /// `None` when `name` isn't a primitive. + #[must_use] + pub fn from_name(name: &str) -> Option { + Prim::ALL.into_iter().find(|p| p.snake() == name) + } + + /// The lower-case IDL spelling (`bool`, `i32`, `string`, `bytes`), also + /// the stem of snake-case routine names such as `read_i32`. + #[must_use] pub fn snake(self) -> &'static str { match self { Prim::Bool => "bool", @@ -190,6 +117,7 @@ impl Prim { /// The capitalized spelling (`Bool`, `I32`, `String`, `Bytes`), the stem /// of camel-case routine names such as `readI32`. + #[must_use] pub fn pascal(self) -> &'static str { match self { Prim::Bool => "Bool", @@ -210,9 +138,95 @@ impl Prim { /// `true` for the integer and float primitives (everything but `bool`, /// `string`, and `bytes`). + #[must_use] pub fn is_numeric(self) -> bool { !matches!(self, Prim::Bool | Prim::String | Prim::Bytes) } + + /// `true` for the integer primitives. + #[must_use] + pub fn is_integer(self) -> bool { + self.is_numeric() && !matches!(self, Prim::F32 | Prim::F64) + } +} + +impl fmt::Display for Prim { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.snake()) + } +} + +/// A fully resolved type: the shape generators render and the ABI lowers. +/// +/// User types carry their bare declared name; [`TypeIndex`] (on the +/// [`Model`](crate::model::Model)) maps the name to its declaration. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub enum Ty { + /// A built-in primitive: a scalar, `bool`, `string`, or `bytes`. + Prim(Prim), + /// A user record (struct): a plain value type. Crosses the C ABI by value + /// as a serialized buffer (`ptr` + `len`), borrowed for a call as a + /// parameter and owned (freed with `{prefix}_free_bytes`) as a return. + Record(String), + /// An algebraic (rich) enum: a sum type with at least one payload-carrying + /// variant. A value type that crosses the C ABI as a serialized buffer + /// (an `i32` tag followed by the active variant's fields), exactly like a + /// [`Record`](Self::Record). + RichEnum(String), + /// A C-style integer enum (no variant payloads). Lowers by value. + Enum(String), + /// A user interface: a reference-counted object. As a parameter the + /// object is borrowed for the call; as a return (or inside a buffer) the + /// receiver adopts one strong reference it must eventually release. + Interface(String), + /// A callback interface: a method set the consumer implements. Only valid + /// as a top-level parameter (bare, or optional as `Cb?`), where it lowers + /// to a context pointer plus a vtable pointer. + CallbackInterface(String), + /// Optional value (`T?`): either the inner type or nothing. + Optional(Box), + /// Homogeneous list (`[T]`) of the inner element type. + List(Box), + /// Map (`{K:V}`) from a key type to a value type. + Map(Box, Box), + /// Lazy sequence (`iter`) of the inner type, lowered to a next/destroy + /// iterator object rather than a materialized collection. + Iterator(Box), +} + +/// How a value of some [`Ty`] crosses a **call boundary**: the one +/// classification the ABI lowering, the marshalling plan, and every backend's +/// argument and return handling agree on. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Family { + /// One C slot passed by value: scalars, bools, and C-style enums. + Direct, + /// A `(ptr, len)` pair of UTF-8 bytes, never NUL-terminated. Borrowed for + /// a call as a parameter; producer-owned and released with + /// `{prefix}_free_bytes` as a return. + String, + /// A `(ptr, len)` raw byte pair. Borrowed as a parameter; producer-owned + /// and released with `{prefix}_free_bytes` as a return. + Bytes, + /// A `(ptr, len)` serialized value buffer (record, rich enum, optional, + /// list, or map). Borrowed as a parameter; producer-owned and released + /// with `{prefix}_free_bytes` after decoding as a return. + Buffer, + /// An object pointer to an interface. Borrowed as a parameter; one + /// strong reference (released with its `_destroy` symbol) as a return. + Object { + /// `true` for `Interface?`: null is a legal "none" value. + nullable: bool, + }, + /// A callback interface: a `void* ctx` plus `const {tag}_vtable*` pair. + /// Only ever a top-level parameter. + Callback { + /// `true` for `Cb?`: a null vtable pointer is a legal "none" value. + nullable: bool, + }, + /// An `iter` return: an opaque iterator handle with its own + /// `next`/`destroy` protocol. Never a parameter. + Iterator, } /// The closed set of shapes a value inside a value buffer can take. @@ -227,18 +241,16 @@ pub enum WireType<'a> { Prim(Prim), /// An object token: eight bytes, little-endian, unsigned, holding the /// object pointer and carrying one strong reference. Carries the - /// (possibly dot-qualified) interface name so backends can wrap the - /// adopted pointer in the right class and, when writing, call the right - /// `_clone` symbol. + /// interface name so backends can wrap the adopted pointer in the right + /// class and, when writing, call the right `_clone` symbol. Object(&'a str), - /// A C-style enum: an `i32` discriminant. Carries the (possibly - /// dot-qualified) enum name so backends can wrap the integer in their - /// typed enum. + /// A C-style enum: an `i32` discriminant. Carries the enum name so + /// backends can wrap the integer in their typed enum. Enum(&'a str), /// A record or rich enum: the named type's own codec applies (fields in /// declaration order for a record; an `i32` tag plus the active /// variant's fields for a rich enum). Backends emit one codec function - /// per user type and delegate here by name (possibly dot-qualified). + /// per user type and delegate here by name. User(&'a str), /// A one-byte presence flag (`0` absent, `1` present) followed by the /// inner value when present. @@ -253,8 +265,9 @@ impl Ty { /// How this type crosses a call boundary. Total over every `Ty`. pub fn family(&self) -> Family { match self { - Ty::StringUtf8 => Family::String, - Ty::Bytes => Family::Bytes, + Ty::Prim(Prim::String) => Family::String, + Ty::Prim(Prim::Bytes) => Family::Bytes, + Ty::Prim(_) | Ty::Enum(_) => Family::Direct, Ty::Record(_) | Ty::RichEnum(_) | Ty::List(_) | Ty::Map(_, _) => Family::Buffer, Ty::Interface(_) => Family::Object { nullable: false }, // The one optional that is not buffered: `Interface?` stays a @@ -263,21 +276,14 @@ impl Ty { Ty::Optional(inner) if matches!(inner.as_ref(), Ty::Interface(_)) => { Family::Object { nullable: true } } + // Likewise `Cb?` stays a `(ctx, vtable)` pair with a null vtable + // meaning none; callback interfaces never enter a value buffer. + Ty::Optional(inner) if matches!(inner.as_ref(), Ty::CallbackInterface(_)) => { + Family::Callback { nullable: true } + } Ty::Optional(_) => Family::Buffer, - Ty::CallbackInterface(_) => Family::Callback, + Ty::CallbackInterface(_) => Family::Callback { nullable: false }, Ty::Iterator(_) => Family::Iterator, - Ty::I8 - | Ty::I16 - | Ty::I32 - | Ty::I64 - | Ty::U8 - | Ty::U16 - | Ty::U32 - | Ty::U64 - | Ty::F32 - | Ty::F64 - | Ty::Bool - | Ty::Enum(_) => Family::Direct, } } @@ -310,10 +316,12 @@ impl Ty { } } - /// The callback interface name, or `None` for every other type. + /// The callback interface name inside a bare or optional callback + /// interface type, or `None` for every other type. pub fn callback_interface_name(&self) -> Option<&str> { match self { Ty::CallbackInterface(n) => Some(n), + Ty::Optional(inner) => inner.callback_interface_name(), _ => None, } } @@ -331,19 +339,7 @@ impl Ty { /// which can legally appear inside a value buffer. pub fn wire(&self) -> WireType<'_> { match self { - Ty::Bool => WireType::Prim(Prim::Bool), - Ty::I8 => WireType::Prim(Prim::I8), - Ty::I16 => WireType::Prim(Prim::I16), - Ty::I32 => WireType::Prim(Prim::I32), - Ty::I64 => WireType::Prim(Prim::I64), - Ty::U8 => WireType::Prim(Prim::U8), - Ty::U16 => WireType::Prim(Prim::U16), - Ty::U32 => WireType::Prim(Prim::U32), - Ty::U64 => WireType::Prim(Prim::U64), - Ty::F32 => WireType::Prim(Prim::F32), - Ty::F64 => WireType::Prim(Prim::F64), - Ty::StringUtf8 => WireType::Prim(Prim::String), - Ty::Bytes => WireType::Prim(Prim::Bytes), + Ty::Prim(p) => WireType::Prim(*p), Ty::Interface(n) => WireType::Object(n), Ty::Enum(name) => WireType::Enum(name), Ty::Record(name) | Ty::RichEnum(name) => WireType::User(name), @@ -397,19 +393,7 @@ impl fmt::Display for Ty { /// doc comments quote. fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { - Ty::I8 => f.write_str("i8"), - Ty::I16 => f.write_str("i16"), - Ty::I32 => f.write_str("i32"), - Ty::I64 => f.write_str("i64"), - Ty::U8 => f.write_str("u8"), - Ty::U16 => f.write_str("u16"), - Ty::U32 => f.write_str("u32"), - Ty::U64 => f.write_str("u64"), - Ty::F32 => f.write_str("f32"), - Ty::F64 => f.write_str("f64"), - Ty::Bool => f.write_str("bool"), - Ty::StringUtf8 => f.write_str("string"), - Ty::Bytes => f.write_str("bytes"), + Ty::Prim(p) => f.write_str(p.snake()), Ty::Record(n) | Ty::RichEnum(n) | Ty::Enum(n) @@ -423,24 +407,100 @@ impl fmt::Display for Ty { } } +/// What kind of declaration a user type name refers to. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum TypeKind { + /// A `structs:` entry. + Record, + /// An `enums:` entry with no payload-carrying variant. + Enum, + /// An `enums:` entry with at least one payload-carrying variant. + RichEnum, + /// An `interfaces:` entry. + Interface, + /// A `callback_interfaces:` entry. + CallbackInterface, +} + +/// Where a user type is declared. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct TypeDecl { + /// The declaration kind. + pub kind: TypeKind, + /// Position of the declaring module in + /// [`Model::modules`](crate::model::Model::modules). + pub module: usize, + /// Position of the declaration in its module's list for its kind + /// (`structs`, `enums`, `interfaces`, or `callback_interfaces`). + pub index: usize, +} + +/// Every user type name in an API, mapped to its declaration. +/// +/// Type names are unique across the whole API, so a bare name identifies +/// one declaration. The [`Model`](crate::model::Model) owns the index and +/// exposes typed lookups (`interface`, `record`, `owner`, and so on) on top +/// of it. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct TypeIndex { + decls: BTreeMap, + /// Underscore-joined C path of every module, by position. + module_paths: Vec, +} + +impl TypeIndex { + /// Register a module (in [`Model::modules`](crate::model::Model::modules) + /// order) by its underscore-joined C path, returning its position. + pub(crate) fn push_module(&mut self, path: String) -> usize { + self.module_paths.push(path); + self.module_paths.len() - 1 + } + + /// Record a declaration. The first declaration of a name wins; + /// validation reports any later one. + pub(crate) fn declare(&mut self, name: &str, decl: TypeDecl) { + self.decls.entry(name.to_string()).or_insert(decl); + } + + /// The declaration `name` refers to, or `None` for an undeclared name. + pub fn get(&self, name: &str) -> Option<&TypeDecl> { + self.decls.get(name) + } + + /// The kind of the declaration `name` refers to. + pub fn kind(&self, name: &str) -> Option { + self.get(name).map(|d| d.kind) + } + + /// The underscore-joined C path of the module declaring `name` (the + /// `outer_inner` in `{prefix}_outer_inner_Name`). + pub fn module_path(&self, name: &str) -> Option<&str> { + self.get(name).map(|d| self.module_paths[d.module].as_str()) + } +} + #[cfg(test)] mod tests { use super::*; + const I32: Ty = Ty::Prim(Prim::I32); + const STRING: Ty = Ty::Prim(Prim::String); + #[test] fn families_are_total_and_agree_with_the_abi_contract() { - assert_eq!(Ty::I32.family(), Family::Direct); + assert_eq!(I32.family(), Family::Direct); + assert_eq!(Ty::Prim(Prim::Bool).family(), Family::Direct); assert_eq!(Ty::Enum("Color".into()).family(), Family::Direct); - assert_eq!(Ty::StringUtf8.family(), Family::String); - assert_eq!(Ty::Bytes.family(), Family::Bytes); + assert_eq!(STRING.family(), Family::String); + assert_eq!(Ty::Prim(Prim::Bytes).family(), Family::Bytes); for ty in [ Ty::Record("C".into()), Ty::RichEnum("S".into()), - Ty::List(Box::new(Ty::I32)), + Ty::List(Box::new(I32)), Ty::List(Box::new(Ty::Interface("Store".into()))), - Ty::Map(Box::new(Ty::StringUtf8), Box::new(Ty::I32)), - Ty::Optional(Box::new(Ty::I32)), - Ty::Optional(Box::new(Ty::StringUtf8)), + Ty::Map(Box::new(STRING), Box::new(I32)), + Ty::Optional(Box::new(I32)), + Ty::Optional(Box::new(STRING)), Ty::Optional(Box::new(Ty::Record("C".into()))), ] { assert_eq!(ty.family(), Family::Buffer, "{ty}"); @@ -456,23 +516,28 @@ mod tests { ); assert_eq!( Ty::CallbackInterface("Listener".into()).family(), - Family::Callback + Family::Callback { nullable: false } + ); + assert_eq!( + Ty::Optional(Box::new(Ty::CallbackInterface("Listener".into()))).family(), + Family::Callback { nullable: true } ); - assert_eq!(Ty::Iterator(Box::new(Ty::I32)).family(), Family::Iterator); + assert_eq!(Ty::Iterator(Box::new(I32)).family(), Family::Iterator); } #[test] fn wire_shapes() { - assert_eq!(Ty::StringUtf8.wire(), WireType::Prim(Prim::String)); - assert_eq!(Ty::Bytes.wire(), WireType::Prim(Prim::Bytes)); + for p in Prim::ALL { + assert_eq!(Ty::Prim(p).wire(), WireType::Prim(p)); + } assert_eq!( - Ty::Interface("kv.Store".into()).wire(), - WireType::Object("kv.Store") + Ty::Interface("Store".into()).wire(), + WireType::Object("Store") ); assert_eq!(Ty::RichEnum("Shape".into()).wire(), WireType::User("Shape")); assert_eq!(Ty::Enum("Color".into()).wire(), WireType::Enum("Color")); - let list = Ty::List(Box::new(Ty::I32)); - assert_eq!(list.wire(), WireType::List(&Ty::I32)); + let list = Ty::List(Box::new(I32)); + assert_eq!(list.wire(), WireType::List(&I32)); } #[test] @@ -485,7 +550,7 @@ mod tests { fn containment_recurses() { assert!(Ty::Record("C".into()).contains_user_type()); assert!(Ty::Map( - Box::new(Ty::StringUtf8), + Box::new(STRING), Box::new(Ty::Optional(Box::new(Ty::RichEnum("S".into())))) ) .contains_user_type()); @@ -497,10 +562,10 @@ mod tests { #[test] fn display_is_the_idl_spelling() { let ty = Ty::List(Box::new(Ty::Map( - Box::new(Ty::StringUtf8), - Box::new(Ty::Optional(Box::new(Ty::Record("a.Contact".into())))), + Box::new(STRING), + Box::new(Ty::Optional(Box::new(Ty::Record("Contact".into())))), ))); - assert_eq!(ty.to_string(), "[{string:a.Contact?}]"); + assert_eq!(ty.to_string(), "[{string:Contact?}]"); assert_eq!( Ty::Iterator(Box::new(Ty::Interface("Store".into()))).to_string(), "iter" @@ -508,4 +573,13 @@ mod tests { assert_eq!(Prim::String.snake(), "string"); assert_eq!(Prim::I32.pascal(), "I32"); } + + #[test] + fn primitive_names_round_trip() { + for p in Prim::ALL { + assert_eq!(Prim::from_name(p.snake()), Some(p)); + } + assert_eq!(Prim::from_name("usize"), None); + assert!(Prim::U8.is_integer() && !Prim::F64.is_integer() && !Prim::Bool.is_numeric()); + } } diff --git a/crates/weaveffi-model/src/validate.rs b/crates/weaveffi-model/src/validate.rs index 4b9b04f1..68e53b09 100644 --- a/crates/weaveffi-model/src/validate.rs +++ b/crates/weaveffi-model/src/validate.rs @@ -1,18 +1,19 @@ //! IDL validation. This module owns the [`ValidationError`] catalog and the -//! [`validate_api`] entry point; the work is split across submodules: -//! `rules` (per-module checks), `diagnostic` (miette span attachment), and -//! `warnings` (advisory lints). +//! [`validate`] entry point, which checks an [`Api`] against the library's +//! [`Identity`] and builds the one [`Model`] every generator consumes. The +//! work is split across submodules: `rules` (name, type, and shape checks), +//! `diagnostic` (miette span attachment), and `warnings` (advisory lints). //! //! Validation collects *every* rule violation before failing, so a document //! with several problems reports them all in one run rather than one per //! invocation. use crate::ir::{Api, SUPPORTED_VERSIONS}; +use crate::model::{Model, RESERVED_SYMBOL_FAMILIES, RUNTIME_SYMBOLS}; +use crate::pkg::Identity; #[cfg(feature = "idl")] use miette::Diagnostic; -use std::collections::BTreeSet; - -use crate::resolved::ResolvedApi; +use std::collections::BTreeMap; mod diagnostic; mod rules; @@ -25,13 +26,32 @@ pub use warnings::{collect_warnings, ValidationWarning}; /// Every way an [`Api`] can fail validation. /// -/// `validate_api` collects every variant it encounters. Each variant carries +/// [`validate`] collects every variant it encounters. Each variant carries /// the names needed to render an actionable diagnostic, and the /// `#[error]`/`#[diagnostic]` attributes supply the message and help text -/// shown to the user. -#[derive(Debug, thiserror::Error)] +/// shown to the user. Module fields hold the module's dotted path +/// (`outer.inner`). The variant serializes as an object whose `code` is the +/// variant name, with the variant's fields alongside (the shape of +/// `weaveffi validate --format json` failures). +#[derive(Debug, thiserror::Error, serde::Serialize)] #[cfg_attr(feature = "idl", derive(Diagnostic))] +#[serde(tag = "code")] pub enum ValidationError { + /// The document declares a schema version this build doesn't support. + #[error("unsupported schema version '{version}'; supported versions: {supported}")] + #[cfg_attr( + feature = "idl", + diagnostic(help( + "set the version field to the current schema version and update the \ + document to match the current schema (see docs/src/reference/idl.md)" + )) + )] + UnsupportedSchemaVersion { + /// Version requested by the document. + version: String, + /// Comma-separated list of versions this build accepts. + supported: String, + }, /// A module is missing its required `name` field. #[error("module has no name")] #[cfg_attr( @@ -39,125 +59,133 @@ pub enum ValidationError { diagnostic(help("every module must have a non-empty 'name' field")) )] NoModuleName, - /// Two modules share the same name. - #[error("duplicate module name: {0}")] + /// Two sibling modules share a name. + #[error("duplicate module name: {module}")] #[cfg_attr( feature = "idl", diagnostic(help( - "module names must be unique within an API definition; rename or merge the duplicate" - )) - )] - DuplicateModuleName(String), - /// A module name is not a valid identifier; the second field explains why. - #[error("invalid module name '{0}': {1}")] - #[cfg_attr( - feature = "idl", - diagnostic(help( - "choose a valid identifier (a-z, A-Z, 0-9, _) that is not a reserved word" + "module names must be unique among siblings; rename or merge the duplicate" )) )] - InvalidModuleName(String, &'static str), - /// Two functions in the same module share a name. - #[error("duplicate function name in module '{module}': {function}")] - #[cfg_attr( - feature = "idl", - diagnostic(help("function names must be unique within a module; rename the duplicate")) - )] - DuplicateFunctionName { - /// Module that contains the colliding functions. + DuplicateModuleName { + /// Dotted path of the duplicated module. module: String, - /// Duplicated function name. - function: String, }, - /// Two parameters of one function share a name. - #[error("duplicate param name in function '{function}' of module '{module}': {param}")] + /// A module name is not a valid identifier. + #[error("invalid module name '{name}': {reason}")] #[cfg_attr( feature = "idl", diagnostic(help( - "parameter names must be unique within a function; rename the duplicate" + "choose a valid identifier (a-z, A-Z, 0-9, _) that is not a reserved word" )) )] - DuplicateParamName { - /// Module that contains the function. - module: String, - /// Function that contains the colliding parameters. - function: String, - /// Duplicated parameter name. - param: String, + InvalidModuleName { + /// The rejected module name. + name: String, + /// Why the name was rejected. + reason: &'static str, }, /// A name matches a reserved keyword in one of the target languages. - #[error("reserved keyword used: {0}")] + #[error("reserved keyword used: {name}")] #[cfg_attr( feature = "idl", diagnostic(help("choose a different name that is not a language reserved word")) )] - ReservedKeyword(String), - /// An identifier is malformed; the second field explains why. - #[error("invalid identifier '{0}': {1}")] + ReservedKeyword { + /// The reserved name. + name: String, + }, + /// An identifier is malformed. + #[error("invalid identifier '{name}': {reason}")] #[cfg_attr(feature = "idl", diagnostic(help("identifiers must start with a letter or underscore and contain only alphanumeric or underscore characters")))] - InvalidIdentifier(String, &'static str), - /// An error domain in the named module is missing its `name` field. - #[error("error domain missing name in module '{0}'")] - #[cfg_attr( - feature = "idl", - diagnostic(help("add a non-empty 'name' field to the error domain")) - )] - ErrorDomainMissingName(String), - /// Two error codes in the same module share a name. - #[error("duplicate error code name in module '{module}': {name}")] + InvalidIdentifier { + /// The rejected identifier. + name: String, + /// Why the identifier was rejected. + reason: &'static str, + }, + /// Two types (records, enums, interfaces, callback interfaces, or error + /// domains) share a name. + /// + /// Type names are global: generators emit flat per-language type names + /// and a reference names a type by its bare name, so two types called + /// `Config` would collide in generated code and make references + /// ambiguous. + #[error("duplicate type name '{name}' (declared in '{first}' and '{second}')")] #[cfg_attr( feature = "idl", - diagnostic(help("error code names must be unique within a module; rename the duplicate")) + diagnostic(help( + "struct, enum, interface, callback interface, and error domain names must be unique \ + across the whole API; rename one of the declarations" + )) )] - DuplicateErrorName { - /// Module that contains the error domain. - module: String, - /// Duplicated error code name. + DuplicateTypeName { + /// The colliding type name. name: String, + /// Module path of the first declaration. + first: String, + /// Module path of the second declaration. + second: String, }, - /// Two error codes in the same module share a numeric value. - #[error("duplicate error numeric code in module '{module}': {code}")] + /// Two free functions share a name. + /// + /// Function names are global: several targets flatten every module's + /// functions into one namespace, so `a.open` and `b.open` would collide + /// there. + #[error("duplicate function name '{name}' (declared in '{first}' and '{second}')")] #[cfg_attr( feature = "idl", diagnostic(help( - "numeric error codes must be unique within a module; assign a different value" + "free function names must be unique across the whole API, because targets \ + flatten module functions into one namespace; rename one of them (for example \ + 'open_store')" )) )] - DuplicateErrorCode { - /// Module that contains the error domain. - module: String, - /// Conflicting numeric error code. - code: i32, + DuplicateFunctionName { + /// The colliding function name. + name: String, + /// Module path of the first declaration. + first: String, + /// Module path of the second declaration. + second: String, }, - /// An error code uses a reserved value: `0` means success and the - /// negative range belongs to the runtime's trap codes. - #[error("invalid error code in module '{module}' for '{name}': must be a positive integer")] + /// A free function has the same name as an error domain. + #[error("function '{function}.{name}' has the same name as error domain '{domain}.{name}'")] #[cfg_attr( feature = "idl", diagnostic(help( - "0 means success and negative codes are reserved for the runtime (-1 generic, \ - -2 panic, -3 marshalling failure, -4 foreign callback error, -5 cancelled); use a positive integer" + "function and error domain names share one flat namespace in several targets; \ + rename one of them" )) )] - InvalidErrorCode { - /// Module that contains the error domain. - module: String, - /// Error code name with the invalid value. + NameCollisionWithErrorDomain { + /// The shared name. name: String, + /// Module path of the function. + function: String, + /// Module path of the error domain. + domain: String, }, - /// A function name collides with an error domain name in the same module. - #[error("function name collides with error domain name in module '{module}': {name}")] + /// Two error codes share a name. + /// + /// Code names are global: backends with flat namespaces (Python, Node, + /// Go) derive one error class or constant per code, so `NotFound` in two + /// domains would collide in generated code. + #[error("duplicate error code name '{name}' (declared in '{first}' and '{second}')")] #[cfg_attr( feature = "idl", diagnostic(help( - "function and error domain names share a namespace; rename one to avoid the collision" + "error code names must be unique across the whole API; qualify one of them \ + (for example 'OrderNotFound')" )) )] - NameCollisionWithErrorDomain { - /// Module where the collision occurs. - module: String, - /// Name shared by the function and the error domain. + DuplicateErrorCodeName { + /// The colliding code name. name: String, + /// Domain of the first declaration, as `module.Domain`. + first: String, + /// Domain of the second declaration, as `module.Domain`. + second: String, }, /// Two declarations lower to the same C identifier. /// @@ -182,6 +210,22 @@ pub enum ValidationError { /// The declaration that collided with it. second: String, }, + /// Two parameters of one function share a name. + #[error("duplicate param name in function '{function}' of module '{module}': {param}")] + #[cfg_attr( + feature = "idl", + diagnostic(help( + "parameter names must be unique within a function; rename the duplicate" + )) + )] + DuplicateParamName { + /// Module that contains the function. + module: String, + /// Function that contains the colliding parameters. + function: String, + /// Duplicated parameter name. + param: String, + }, /// A synchronous callable declares `cancellable: true`. #[error("function '{module}::{function}' is cancellable but not async")] #[cfg_attr( @@ -197,13 +241,15 @@ pub enum ValidationError { /// The offending function. function: String, }, - /// A function declares `throws: true` but no error domain is in scope. + /// A function or callback-interface method declares `throws: true` but + /// no error domain is in scope. #[error("function '{module}::{function}' declares throws but no error domain is in scope")] #[cfg_attr( feature = "idl", diagnostic(help( - "a throwing function reports codes from its module's error domain; declare an \ - `errors:` block on this module (or an ancestor module), or remove `throws: true`" + "a throwing function (or callback-interface method) reports codes from its \ + module's error domain; declare an `errors:` block on this module (or an ancestor \ + module), or remove `throws: true`" )) )] ThrowsWithoutErrorDomain { @@ -212,74 +258,60 @@ pub enum ValidationError { /// Function marked `throws` with no domain in scope. function: String, }, - /// Two types (structs, enums, interfaces, or callback interfaces) share a - /// bare name. - /// - /// Type names must be unique across the whole API: generators emit flat - /// per-language type names, and unqualified cross-module references - /// resolve by bare name, so two types called `Config` would collide in - /// generated code and make references ambiguous. - #[error("duplicate type name '{name}' (declared in '{first}' and '{second}')")] + /// An async function tries to return an iterator, which has no async ABI. + #[error("async function '{module}::{function}' cannot return an iterator")] #[cfg_attr( feature = "idl", diagnostic(help( - "struct, enum, interface, callback interface, and error domain names must be unique \ - across the whole API; rename one of the declarations" + "the callback-completed async ABI has no streaming protocol; return a list ([T]) \ + from the async function, or make the function synchronous and return iter" )) )] - DuplicateTypeName { - /// The colliding bare type name. - name: String, - /// Module path of the first declaration. - first: String, - /// Module path of the second declaration. - second: String, + AsyncIteratorReturn { + /// Module that contains the function. + module: String, + /// Async function with the iterator return. + function: String, }, - /// Two error domains declare a code with the same name. - /// - /// Code names must be unique across every domain in the API: backends - /// with flat namespaces (Python, Node, Go) derive one error class or - /// constant per code, so `NotFound` in two domains would collide in - /// generated code. - #[error("duplicate error code name '{name}' (declared in '{first}' and '{second}')")] + /// An error domain in the named module is missing its `name` field. + #[error("error domain missing name in module '{module}'")] #[cfg_attr( feature = "idl", - diagnostic(help( - "error code names must be unique across the whole API; qualify one of them \ - (e.g. 'OrderNotFound')" - )) + diagnostic(help("add a non-empty 'name' field to the error domain")) )] - DuplicateErrorCodeName { - /// The colliding code name. - name: String, - /// Domain of the first declaration, as `module.Domain`. - first: String, - /// Domain of the second declaration, as `module.Domain`. - second: String, + ErrorDomainMissingName { + /// Module that declares the error domain. + module: String, }, - /// Two structs in the same module share a name. - #[error("duplicate struct name in module '{module}': {name}")] + /// Two error codes in one domain share a numeric value. + #[error("duplicate error numeric code in module '{module}': {value}")] #[cfg_attr( feature = "idl", - diagnostic(help("struct names must be unique within a module; rename the duplicate")) + diagnostic(help( + "numeric error codes must be unique within a domain; assign a different value" + )) )] - DuplicateStructName { - /// Module that contains the structs. + DuplicateErrorCode { + /// Module that declares the error domain. module: String, - /// Duplicated struct name. - name: String, + /// Conflicting numeric error code. + value: i32, }, - /// Two fields of one struct share a name. - #[error("duplicate field name in struct '{struct_name}': {field}")] + /// An error code uses a reserved value: `0` means success and the + /// negative range belongs to the runtime's trap codes. + #[error("invalid error code in module '{module}' for '{name}': must be a positive integer")] #[cfg_attr( feature = "idl", - diagnostic(help("field names must be unique within a struct; rename the duplicate")) + diagnostic(help( + "0 means success and negative codes are reserved for the runtime (-1 generic, \ + -2 panic, -3 marshalling failure, -4 foreign callback error, -5 cancelled); use a positive integer" + )) )] - DuplicateStructField { - /// Struct that contains the colliding fields. - struct_name: String, - /// Duplicated field name. - field: String, + InvalidErrorCode { + /// Module that declares the error domain. + module: String, + /// Error code name with the invalid value. + name: String, }, /// A struct declares no fields. #[error("empty struct in module '{module}': {name}")] @@ -295,17 +327,17 @@ pub enum ValidationError { /// Name of the empty struct. name: String, }, - /// Two enums in the same module share a name. - #[error("duplicate enum name in module '{module}': {name}")] + /// Two fields of one struct share a name. + #[error("duplicate field name in struct '{struct_name}': {field}")] #[cfg_attr( feature = "idl", - diagnostic(help("enum names must be unique within a module; rename the duplicate")) + diagnostic(help("field names must be unique within a struct; rename the duplicate")) )] - DuplicateEnumName { - /// Module that contains the enums. - module: String, - /// Duplicated enum name. - name: String, + DuplicateStructField { + /// Struct that contains the colliding fields. + struct_name: String, + /// Duplicated field name. + field: String, }, /// An enum declares no variants. #[error("empty enum in module '{module}': {name}")] @@ -363,16 +395,19 @@ pub enum ValidationError { /// Conflicting numeric discriminant. value: i32, }, - /// Two interfaces in the same module share a name. - #[error("duplicate interface name in module '{module}': {name}")] + /// An interface declares no members at all. + #[error("empty interface in module '{module}': {name}")] #[cfg_attr( feature = "idl", - diagnostic(help("interface names must be unique within a module; rename the duplicate")) + diagnostic(help( + "interfaces must declare at least one constructor, method, or static; \ + add a member or remove the interface" + )) )] - DuplicateInterfaceName { - /// Module that contains the interfaces. + EmptyInterface { + /// Module that contains the interface. module: String, - /// Duplicated interface name. + /// Name of the empty interface. name: String, }, /// Two members (constructors, methods, or statics) of one interface share @@ -391,21 +426,6 @@ pub enum ValidationError { /// Duplicated member name. name: String, }, - /// An interface declares no members at all. - #[error("empty interface in module '{module}': {name}")] - #[cfg_attr( - feature = "idl", - diagnostic(help( - "interfaces must declare at least one constructor, method, or static; \ - add a member or remove the interface" - )) - )] - EmptyInterface { - /// Module that contains the interface. - module: String, - /// Name of the empty interface. - name: String, - }, /// An interface constructor declares an explicit return type. #[error("constructor '{constructor}' of interface '{interface}' declares a return type")] #[cfg_attr( @@ -436,63 +456,6 @@ pub enum ValidationError { /// The offending constructor. constructor: String, }, - /// An interface reference appears as a map key. - #[error("interface type '{name}' is not valid in {location}")] - #[cfg_attr( - feature = "idl", - diagnostic(help( - "interface objects may appear anywhere except as map keys; key a map by a scalar, \ - string, or C-style enum instead" - )) - )] - InterfaceInInvalidPosition { - /// The referenced interface name. - name: String, - /// Position where the interface reference appeared. - location: String, - }, - /// A type reference names a struct, enum, interface, or callback interface - /// that doesn't exist. - #[error("unknown type reference: {name}")] - #[cfg_attr( - feature = "idl", - diagnostic(help( - "define a struct, enum, interface, or callback interface with this name, or check \ - for typos" - )) - )] - UnknownTypeRef { - /// Unresolved type name. - name: String, - }, - /// A map uses a key type the C ABI can't represent. - #[error("invalid map key type: {key_type}; only integers, bools, strings, and C-style enums are allowed as map keys")] - #[cfg_attr(feature = "idl", diagnostic(help("map keys must be integers (i8 through u64), bool, string, or a C-style enum; structs, rich enums, interfaces, optionals, lists, and maps cannot be keys")))] - InvalidMapKey { - /// Rejected key type, rendered as it appears in the IDL. - key_type: String, - }, - /// An iterator type appears somewhere other than a function return. - #[error("iterator type is only valid as a function return type, found in {location}")] - #[cfg_attr(feature = "idl", diagnostic(help("iterator types can only be used as function return types, not as parameters or struct fields")))] - IteratorInInvalidPosition { - /// Position where the iterator type appeared. - location: String, - }, - /// Two callback interfaces in the same module share a name. - #[error("duplicate callback interface name in module '{module}': {name}")] - #[cfg_attr( - feature = "idl", - diagnostic(help( - "callback interface names must be unique within a module; rename the duplicate" - )) - )] - DuplicateCallbackInterfaceName { - /// Module that contains the callback interfaces. - module: String, - /// Duplicated callback interface name. - name: String, - }, /// A callback interface declares no methods. #[error("empty callback interface in module '{module}': {name}")] #[cfg_attr( @@ -528,9 +491,9 @@ pub enum ValidationError { #[cfg_attr( feature = "idl", diagnostic(help( - "callback-interface methods are synchronous, never throw, and return either nothing \ - or a direct value (a scalar, bool, or C-style enum): the consumer implements them, \ - so no producer-owned allocation or error payload can flow back" + "callback-interface methods are synchronous calls the consumer implements: they \ + can't be async or cancellable, and they return nothing or any value except an \ + iterator or a callback interface" )) )] InvalidCallbackMethod { @@ -541,52 +504,92 @@ pub enum ValidationError { /// Why the method was rejected. reason: &'static str, }, - /// A callback-interface type appears in a position other than a callable - /// parameter. - #[error("callback interface '{name}' is not valid in {location}")] + /// A type reference names a struct, enum, interface, or callback interface + /// that doesn't exist. + #[error("unknown type reference: {name}")] #[cfg_attr( feature = "idl", diagnostic(help( - "a callback interface is a consumer-implemented vtable: it may be passed as a \ - parameter of a function, constructor, static, or method, but it can't be returned, \ - nested inside another type, or passed to another callback-interface method" + "define a struct, enum, interface, or callback interface with this name, or check \ + for typos" )) )] - CallbackInterfaceInInvalidPosition { - /// The referenced callback interface name. + UnknownTypeRef { + /// Unresolved type name. name: String, - /// Position where the reference appeared. - location: String, }, - /// An async function tries to return an iterator, which has no async ABI. - #[error("async function '{module}::{function}' cannot return an iterator")] + /// A type reference is module-qualified (`a.b.T`). + #[error("qualified type reference '{name}': type names are global, so refer to a type by its bare name")] #[cfg_attr( feature = "idl", diagnostic(help( - "the callback-completed async ABI has no streaming protocol; return a list ([T]) \ - from the async function, or make the function synchronous and return iter" + "every type name is unique across the API, so a reference is just the name: write \ + 'Contact', not 'contacts.Contact'" )) )] - AsyncIteratorReturn { - /// Module that contains the function. - module: String, - /// Async function with the iterator return. - function: String, + QualifiedTypeRef { + /// The dotted name as written. + name: String, }, - /// The document declares a schema version this build doesn't support. - #[error("unsupported schema version '{version}'; supported versions: {supported}")] + /// A type reference names a primitive WeaveFFI doesn't support. + #[error("unsupported primitive type '{name}'")] #[cfg_attr( feature = "idl", diagnostic(help( - "set the version field to the current schema version and update the \ - document to match the current schema (see docs/src/reference/idl.md)" + "`usize` and `isize` vary with the platform and `u128`/`i128` have no portable C \ + type: use `u64` or `i64`; for `char`, use `string`" )) )] - UnsupportedSchemaVersion { - /// Version requested by the document. - version: String, - /// Comma-separated list of versions this build accepts. - supported: String, + UnsupportedPrimitive { + /// The primitive's name as written. + name: String, + }, + /// A map uses a key type the C ABI can't represent. + #[error("invalid map key type: {key_type}; only integers, bools, strings, and C-style enums are allowed as map keys")] + #[cfg_attr(feature = "idl", diagnostic(help("map keys must be integers (i8 through u64), bool, string, or a C-style enum; structs, rich enums, interfaces, optionals, lists, and maps cannot be keys")))] + InvalidMapKey { + /// Rejected key type, rendered as it appears in the IDL. + key_type: String, + }, + /// An interface reference appears as a map key. + #[error("interface type '{name}' is not valid in {location}")] + #[cfg_attr( + feature = "idl", + diagnostic(help( + "interface objects may appear anywhere except as map keys; key a map by a scalar, \ + string, or C-style enum instead" + )) + )] + InterfaceInInvalidPosition { + /// The referenced interface name. + name: String, + /// Position where the interface reference appeared. + location: String, + }, + /// A callback-interface type appears in a position other than a callable + /// parameter. + #[error("callback interface '{name}' is not valid in {location}")] + #[cfg_attr( + feature = "idl", + diagnostic(help( + "a callback interface is a consumer-implemented vtable: it may be passed (bare, or \ + optional as `Cb?`) as a parameter of a function, constructor, static, or method, but \ + it can't be returned, nested inside another type, or passed to another \ + callback-interface method" + )) + )] + CallbackInterfaceInInvalidPosition { + /// The referenced callback interface name. + name: String, + /// Position where the reference appeared. + location: String, + }, + /// An iterator type appears somewhere other than a function return. + #[error("iterator type is only valid as a function return type, found in {location}")] + #[cfg_attr(feature = "idl", diagnostic(help("iterator types can only be used as function return types, not as parameters or struct fields")))] + IteratorInInvalidPosition { + /// Position where the iterator type appeared. + location: String, }, } @@ -650,94 +653,125 @@ impl Diagnostic for ValidationDiagnostics { } } -/// Validate an [`Api`] document, reporting **every** rule violation found. -/// The optional `source` is `(filename, contents)` of the IDL file and is -/// used to attach spans to the returned diagnostics. Pass `None` when the -/// API is constructed in memory (tests, programmatic builds) and there is no -/// on-disk source. +/// Validate an [`Api`] document against the library's `identity`, reporting +/// **every** rule violation found, and build its [`Model`]. /// -/// This is the one checked way to turn a parsed document into the -/// [`ResolvedApi`] every generator consumes. The document is never rewritten; -/// the resolved view indexes its declarations so that -/// [`ResolvedApi::resolve`] can turn each written reference into a -/// [`Ty`](crate::model::Ty) whose kind and owning module are known. +/// This is the one checked way to obtain the [`Model`] every generator +/// consumes, and it builds the model exactly once. The C symbol table is +/// checked with the real `identity`, in the same pass. The optional `source` +/// is `(filename, contents)` of the IDL file and is used to attach spans to +/// the returned diagnostics, each located within its enclosing declaration. +/// Pass `None` when the API is constructed in memory and there is no on-disk +/// source. /// /// # Errors /// /// Returns [`ValidationDiagnostics`] carrying one [`ValidationDiagnostic`] /// per violation: an unsupported schema version, a duplicate or invalid name, -/// an unknown or misplaced type, an empty struct or enum, a `throws` without -/// an error domain, or any other rule violation in the catalog above. -pub fn validate_api( - api: Api, +/// an unknown, qualified, or misplaced type, an empty struct or enum, a +/// `throws` without an error domain, a C symbol collision, or any other rule +/// violation in the catalog above. +pub fn validate( + api: &Api, + identity: &Identity, source: Option<(&str, &str)>, -) -> Result { - let errors = validate_api_inner(&api); - if errors.is_empty() { - return Ok(ResolvedApi::assume_valid(api)); - } - Err(ValidationDiagnostics { +) -> Result { + validate_scoped(api, identity, Options::default()).map_err(|errors| ValidationDiagnostics { diagnostics: errors .into_iter() - .map(|e| ValidationDiagnostic::new(e, source)) + .map(|(error, scope)| ValidationDiagnostic::new(error, &scope, source)) .collect(), }) } -fn validate_api_inner(api: &Api) -> Vec { - let mut errors = Vec::new(); +/// Options for [`validate_scoped`]. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct Options { + /// Accept a type name that no declaration in the document provides, + /// assuming it's a record or rich enum declared elsewhere, and resolve + /// it to [`Ty::Record`](crate::ty::Ty::Record). The `#[weaveffi::module]` + /// macro sets this: it validates one module tree at a time, and asserts + /// at compile time that each such name really crosses as a value buffer. + pub foreign_names: bool, +} + +/// A rule violation paired with the declaration path that encloses it +/// (module segments, then the declaration, member, and parameter or field +/// names). [`validate`] uses the path to locate the offending IDL text; the +/// `#[weaveffi::module]` macro uses it to find the offending Rust item. +pub type Found = (ValidationError, Vec); + +/// Validate `api` against `identity` exactly like [`validate`], with +/// `options`, returning each violation with the declaration path that +/// encloses it instead of a located diagnostic. +/// +/// # Errors +/// +/// Returns every rule violation found, each paired with its declaration +/// path. +pub fn validate_scoped( + api: &Api, + identity: &Identity, + options: Options, +) -> Result> { if !SUPPORTED_VERSIONS.contains(&api.version.as_str()) { // A wrong-schema document is checked no further: the rules below // assume the current schema's shape. - return vec![ValidationError::UnsupportedSchemaVersion { - version: api.version.clone(), - supported: SUPPORTED_VERSIONS.join(", "), - }]; + return Err(vec![( + ValidationError::UnsupportedSchemaVersion { + version: api.version.clone(), + supported: SUPPORTED_VERSIONS.join(", "), + }, + vec![], + )]); } - let types = rules::TypeIndex::build(&api.modules); - let mut module_names = BTreeSet::new(); - for m in &api.modules { - if !module_names.insert(m.name.clone()) { - errors.push(ValidationError::DuplicateModuleName(m.name.clone())); - } - rules::validate_module(m, &types, false, &mut errors); + let types = crate::model::index(api); + let mut found = Vec::new(); + rules::check(api, &types, options, &mut found); + if !found.is_empty() { + // The model's lowering assumes every rule holds, so the symbol + // table is only built for an otherwise valid document. + return Err(found); } - rules::check_global_type_names(&api.modules, &mut errors); - rules::check_global_error_code_names(&api.modules, &mut errors); - if errors.is_empty() { - check_symbol_collisions(api, &mut errors); + let model = crate::model::build_indexed(api, identity.clone(), types); + check_symbol_collisions(&model, &mut found); + if found.is_empty() { + Ok(model) + } else { + Err(found) } - errors } -/// Build the binding model (any prefix works: collisions are invariant under -/// a shared prefix) and reject any C identifier claimed twice. -fn check_symbol_collisions(api: &Api, errors: &mut Vec) { - let resolved = ResolvedApi::assume_valid(api.clone()); - let model = crate::model::BindingModel::build(&resolved); - let mut seen: std::collections::BTreeMap = std::collections::BTreeMap::new(); - let runtime = crate::model::RUNTIME_SYMBOLS.len(); +/// Reject any C identifier the model claims twice, or that falls in a +/// family the generated C value-buffer helpers reserve. +fn check_symbol_collisions(model: &Model, found: &mut Vec) { + let prefix = model.prefix(); + let mut seen: BTreeMap = BTreeMap::new(); for (i, (symbol, origin)) in model.c_symbols().into_iter().enumerate() { - let family = symbol - .strip_prefix(&format!("{}_", model.prefix)) - .and_then(|rest| { - crate::model::RESERVED_SYMBOL_FAMILIES - .iter() - .find(|f| rest.starts_with(*f)) - }); - if let (Some(family), true) = (family, i >= runtime) { - errors.push(ValidationError::SymbolCollision { - symbol: symbol.clone(), - first: format!("the C value-buffer helpers ('{}_{family}*')", model.prefix), - second: origin.clone(), - }); + let family = symbol.strip_prefix(&format!("{prefix}_")).and_then(|rest| { + RESERVED_SYMBOL_FAMILIES + .iter() + .find(|f| rest.starts_with(*f)) + }); + if let (Some(family), true) = (family, i >= RUNTIME_SYMBOLS.len()) { + found.push(( + ValidationError::SymbolCollision { + symbol: symbol.clone(), + first: format!("the C value-buffer helpers ('{prefix}_{family}*')"), + second: origin.clone(), + }, + vec![], + )); } match seen.get(&symbol) { - Some(first) => errors.push(ValidationError::SymbolCollision { - symbol, - first: first.clone(), - second: origin, - }), + Some(first) => found.push(( + ValidationError::SymbolCollision { + symbol, + first: first.clone(), + second: origin, + }, + vec![], + )), None => { seen.insert(symbol, origin); } diff --git a/crates/weaveffi-model/src/validate/diagnostic.rs b/crates/weaveffi-model/src/validate/diagnostic.rs index aff280d1..1f1d0f80 100644 --- a/crates/weaveffi-model/src/validate/diagnostic.rs +++ b/crates/weaveffi-model/src/validate/diagnostic.rs @@ -1,6 +1,13 @@ //! [`ValidationDiagnostic`]: a [`ValidationError`] plus an optional source //! snippet and best-effort span for fancy miette rendering. //! +//! The span is located within the error's enclosing declaration: the +//! validator records the declaration path (module segments, then declaration +//! and member names), and the search walks that path through the source, +//! finding each name as the value of a `name` key, before looking for the +//! offending text. So a duplicate parameter `x` underlines the second `x` +//! of the right function, not the first `x` anywhere in the file. +//! //! The source snippet, the span, and the miette `Diagnostic` impl are only //! available with the `idl` feature; without it the wrapper carries the //! error alone. @@ -77,18 +84,23 @@ impl Diagnostic for ValidationDiagnostic { } impl ValidationDiagnostic { - /// Build a [`ValidationDiagnostic`] from a [`ValidationError`] and an + /// Wrap `error`, located within the declaration path `scope`, with an /// optional `(filename, contents)` source. When a source is provided the - /// constructor performs a best-effort search for the offending identifier - /// (e.g. a duplicate module name or unknown type reference) and attaches - /// a [`SourceSpan`] for fancy rendering. If no span can be computed the - /// label is omitted and miette still produces a nicer message + help - /// section than plain `Display`. + /// constructor searches for the offending text within the enclosing + /// declaration and attaches a [`SourceSpan`] for fancy rendering. If no + /// span can be computed the label is omitted and miette still produces + /// a nicer message + help section than plain `Display`. #[cfg(feature = "idl")] - pub fn new(error: ValidationError, source: Option<(&str, &str)>) -> Self { + pub(crate) fn new( + error: ValidationError, + scope: &[String], + source: Option<(&str, &str)>, + ) -> Self { let (src, span) = match source { Some((filename, contents)) => { - let span = find_offending_span(&error, contents); + let span = error + .needle() + .and_then(|needle| locate(contents, scope, needle)); (Some(NamedSource::new(filename, contents.to_string())), span) } None => (None, None), @@ -96,60 +108,194 @@ impl ValidationDiagnostic { Self { error, src, span } } - /// Build a [`ValidationDiagnostic`] from a [`ValidationError`]. Without - /// the `idl` feature there's no miette rendering, so the optional - /// `(filename, contents)` source is accepted for signature parity and + /// Wrap `error`. Without the `idl` feature there's no miette rendering, + /// so the scope and source are accepted for signature parity and /// ignored. #[cfg(not(feature = "idl"))] - pub fn new(error: ValidationError, _source: Option<(&str, &str)>) -> Self { + pub(crate) fn new( + error: ValidationError, + _scope: &[String], + _source: Option<(&str, &str)>, + ) -> Self { Self { error } } } +/// What to look for in the source, after walking the enclosing declaration +/// path. +#[cfg(feature = "idl")] +#[derive(Clone, Copy)] +enum Needle<'a> { + /// A declared name: the value of a `name` key. + Decl(&'a str), + /// The second declaration of a name (the duplicate). + Second(&'a str), + /// Any other text, such as a type reference, as a whole token. + Text(&'a str), + /// The innermost declaration of the scope itself. + Scope, + /// The second declaration of the scope's innermost name (a duplicate + /// sibling). + SecondScope, +} + #[cfg(feature = "idl")] -fn find_offending_span(err: &ValidationError, src: &str) -> Option { - let needle: &str = match err { - ValidationError::DuplicateModuleName(n) => Some(n.as_str()), - ValidationError::InvalidModuleName(n, _) => Some(n.as_str()), - ValidationError::DuplicateFunctionName { function, .. } => Some(function.as_str()), - ValidationError::DuplicateParamName { param, .. } => Some(param.as_str()), - ValidationError::ReservedKeyword(n) => Some(n.as_str()), - ValidationError::InvalidIdentifier(n, _) => Some(n.as_str()), - ValidationError::DuplicateErrorName { name, .. } => Some(name.as_str()), - ValidationError::InvalidErrorCode { name, .. } => Some(name.as_str()), - ValidationError::NameCollisionWithErrorDomain { name, .. } => Some(name.as_str()), - ValidationError::DuplicateStructName { name, .. } => Some(name.as_str()), - ValidationError::DuplicateStructField { field, .. } => Some(field.as_str()), - ValidationError::EmptyStruct { name, .. } => Some(name.as_str()), - ValidationError::DuplicateEnumName { name, .. } => Some(name.as_str()), - ValidationError::EmptyEnum { name, .. } => Some(name.as_str()), - ValidationError::DuplicateEnumVariant { variant, .. } => Some(variant.as_str()), - ValidationError::UnknownTypeRef { name } => Some(name.as_str()), - ValidationError::DuplicateCallbackInterfaceName { name, .. } => Some(name.as_str()), - ValidationError::EmptyCallbackInterface { name, .. } => Some(name.as_str()), - ValidationError::DuplicateCallbackMethod { name, .. } => Some(name.as_str()), - ValidationError::InvalidCallbackMethod { method, .. } => Some(method.as_str()), - ValidationError::CallbackInterfaceInInvalidPosition { name, .. } => Some(name.as_str()), - ValidationError::UnsupportedSchemaVersion { version, .. } => Some(version.as_str()), - ValidationError::AsyncIteratorReturn { function, .. } => Some(function.as_str()), - ValidationError::DuplicateInterfaceName { name, .. } => Some(name.as_str()), - ValidationError::DuplicateInterfaceMember { name, .. } => Some(name.as_str()), - ValidationError::EmptyInterface { name, .. } => Some(name.as_str()), - ValidationError::ConstructorHasReturn { constructor, .. } => Some(constructor.as_str()), - ValidationError::AsyncConstructor { constructor, .. } => Some(constructor.as_str()), - ValidationError::InterfaceInInvalidPosition { name, .. } => Some(name.as_str()), - ValidationError::DuplicateTypeName { name, .. } => Some(name.as_str()), - ValidationError::DuplicateErrorCodeName { name, .. } => Some(name.as_str()), - ValidationError::ThrowsWithoutErrorDomain { function, .. } => Some(function.as_str()), - ValidationError::CancellableNotAsync { function, .. } => { - Some(function.rsplit('.').next().unwrap_or(function)) +impl ValidationError { + /// The text a diagnostic underlines, or `None` when no one spot in the + /// source is to blame. + fn needle(&self) -> Option> { + use Needle::{Decl, Scope, Second, SecondScope, Text}; + /// A same-module duplicate is the second declaration in scope; a + /// cross-module one is the first declaration in the second module. + fn duplicate<'a>(name: &'a str, first: &str, second: &str) -> Needle<'a> { + if first == second { + Second(name) + } else { + Decl(name) + } } - _ => None, - }?; - let quoted = format!("\"{needle}\""); - if let Some(pos) = src.find("ed) { - return Some(SourceSpan::new(pos.into(), quoted.len())); + Some(match self { + Self::UnsupportedSchemaVersion { version, .. } => Text(version), + Self::DuplicateModuleName { .. } => SecondScope, + Self::InvalidModuleName { name, .. } + | Self::ReservedKeyword { name } + | Self::InvalidIdentifier { name, .. } + | Self::NameCollisionWithErrorDomain { name, .. } + | Self::InvalidErrorCode { name, .. } + | Self::EmptyStruct { name, .. } + | Self::EmptyEnum { name, .. } + | Self::EmptyInterface { name, .. } + | Self::EmptyCallbackInterface { name, .. } => Decl(name), + Self::DuplicateTypeName { + name, + first, + second, + } + | Self::DuplicateFunctionName { + name, + first, + second, + } + | Self::DuplicateErrorCodeName { + name, + first, + second, + } => duplicate(name, first, second), + Self::DuplicateParamName { param: name, .. } + | Self::DuplicateStructField { field: name, .. } + | Self::DuplicateEnumVariant { variant: name, .. } + | Self::DuplicateEnumVariantField { field: name, .. } + | Self::DuplicateInterfaceMember { name, .. } + | Self::DuplicateCallbackMethod { name, .. } => Second(name), + Self::CancellableNotAsync { .. } + | Self::ThrowsWithoutErrorDomain { .. } + | Self::AsyncIteratorReturn { .. } => Scope, + Self::DuplicateEnumValue { enum_name, .. } => Decl(enum_name), + Self::ConstructorHasReturn { constructor, .. } + | Self::AsyncConstructor { constructor, .. } => Decl(constructor), + Self::InvalidCallbackMethod { method, .. } => Decl(method), + Self::UnknownTypeRef { name } + | Self::QualifiedTypeRef { name } + | Self::UnsupportedPrimitive { name } + | Self::InterfaceInInvalidPosition { name, .. } + | Self::CallbackInterfaceInInvalidPosition { name, .. } => Text(name), + Self::InvalidMapKey { key_type } => Text(key_type), + Self::IteratorInInvalidPosition { .. } => Text("iter"), + Self::NoModuleName + | Self::SymbolCollision { .. } + | Self::ErrorDomainMissingName { .. } + | Self::DuplicateErrorCode { .. } => return None, + }) + } +} + +/// Locate `needle` in `src` within the declaration path `scope`, falling +/// back to its first occurrence anywhere when the scoped search fails. +#[cfg(feature = "idl")] +fn locate(src: &str, scope: &[String], needle: Needle<'_>) -> Option { + // A needle naming the scope itself searches for the innermost name + // within the scope around it. + let (scope, needle) = match (needle, scope.split_last()) { + (Needle::Scope, Some((last, outer))) => (outer, Needle::Decl(last)), + (Needle::SecondScope, Some((last, outer))) => (outer, Needle::Second(last)), + (Needle::Scope | Needle::SecondScope, None) => return None, + _ => (scope, needle), + }; + let mut from = 0; + for anchor in scope { + if let Some(at) = find_decl(src, from, anchor) { + from = at + anchor.len(); + } + } + let (text, scoped) = match needle { + Needle::Decl(name) => (name, find_decl(src, from, name)), + Needle::Second(name) => ( + name, + find_decl(src, from, name) + .and_then(|first| find_decl(src, first + name.len(), name)) + .or_else(|| find_decl(src, from, name)), + ), + Needle::Text(text) => (text, find_token(src, from, text)), + Needle::Scope | Needle::SecondScope => unreachable!("resolved above"), + }; + let at = scoped.or_else(|| find_token(src, 0, text))?; + Some(span(src, at, text.len())) +} + +#[cfg(feature = "idl")] +fn is_ident_char(c: char) -> bool { + c.is_ascii_alphanumeric() || c == '_' +} + +/// The byte offset of the first whole-token occurrence of `needle` at or +/// after `from`: one not glued to identifier characters on either side. +#[cfg(feature = "idl")] +fn find_token(src: &str, from: usize, needle: &str) -> Option { + if needle.is_empty() { + return None; + } + src.get(from..)? + .match_indices(needle) + .map(|(i, _)| from + i) + .find(|&at| { + let before = src[..at].chars().next_back(); + let after = src[at + needle.len()..].chars().next(); + !before.is_some_and(is_ident_char) && !after.is_some_and(is_ident_char) + }) +} + +/// The byte offset of the first occurrence of `name` at or after `from` that +/// is the value of a `name` key, in YAML (`name: x`), JSON +/// (`"name": "x"`), or TOML (`name = "x"`) syntax. +#[cfg(feature = "idl")] +fn find_decl(src: &str, from: usize, name: &str) -> Option { + let mut at = from; + loop { + let hit = find_token(src, at, name)?; + let before = src[..hit].trim_end_matches(['"', '\'']).trim_end(); + if let Some(key) = before.strip_suffix([':', '=']) { + let key = key.trim_end().trim_end_matches(['"', '\'']); + if key + .strip_suffix("name") + .is_some_and(|rest| !rest.chars().next_back().is_some_and(is_ident_char)) + { + return Some(hit); + } + } + at = hit + name.len(); + } +} + +/// The span of `len` bytes at `at`, widened to its surrounding quotes when +/// the text is a quoted scalar. +#[cfg(feature = "idl")] +fn span(src: &str, at: usize, len: usize) -> SourceSpan { + let quote = src[..at] + .chars() + .next_back() + .filter(|c| matches!(c, '"' | '\'')); + match quote { + Some(q) if src[at + len..].starts_with(q) => SourceSpan::new((at - 1).into(), len + 2), + _ => SourceSpan::new(at.into(), len), } - src.find(needle) - .map(|pos| SourceSpan::new(pos.into(), needle.len())) } diff --git a/crates/weaveffi-model/src/validate/rules.rs b/crates/weaveffi-model/src/validate/rules.rs index 2fbffd74..87c83719 100644 --- a/crates/weaveffi-model/src/validate/rules.rs +++ b/crates/weaveffi-model/src/validate/rules.rs @@ -1,15 +1,19 @@ -//! The validation rules: per-module name/uniqueness checks, type-reference -//! existence, ABI-representability of element shapes, callback-interface -//! shape rules, interface positions, and error-domain consistency. +//! The validation rules: global name uniqueness, per-module identifier and +//! shape checks, type-reference resolution, ABI-representability of element +//! shapes, callback-interface rules, interface positions, and error-domain +//! consistency. //! -//! Every rule pushes into a shared `Vec` sink instead of -//! returning early, so one validation pass reports every violation in the -//! document. +//! Every rule pushes into a shared sink instead of returning early, so one +//! validation pass reports every violation in the document. Each violation +//! carries the declaration path that encloses it (module segments, then +//! declaration and member names), which the diagnostic uses to locate the +//! offending text. -use super::ValidationError; +use super::{Found, Options, ValidationError}; use crate::ir::{ - CallbackInterfaceDef, ErrorDomain, Function, InterfaceDef, Module, Param, StructField, TypeRef, + Api, CallbackInterfaceDef, ErrorDomain, Function, InterfaceDef, Module, StructField, TypeRef, }; +use crate::ty::{Prim, TypeIndex, TypeKind}; use std::collections::{BTreeMap, BTreeSet}; const RESERVED: &[&str] = &[ @@ -17,6 +21,14 @@ const RESERVED: &[&str] = &[ "continue", "fn", "struct", "enum", "mod", "use", ]; +/// Primitive spellings other languages have that WeaveFFI deliberately +/// doesn't support; a reference to one gets a dedicated diagnostic instead +/// of "unknown type". +const UNSUPPORTED_PRIMITIVES: &[&str] = &["usize", "isize", "u128", "i128", "char"]; + +const IDENTIFIER_RULE: &str = + "must start with a letter or underscore and contain only alphanumeric characters or underscores"; + fn is_valid_identifier(s: &str) -> bool { let mut chars = s.chars(); match chars.next() { @@ -26,168 +38,59 @@ fn is_valid_identifier(s: &str) -> bool { } } -fn check_identifier(name: &str, errors: &mut Vec) -> bool { - if !is_valid_identifier(name) { - errors.push(ValidationError::InvalidIdentifier( - name.to_string(), - "must start with a letter or underscore and contain only alphanumeric characters or underscores", - )); - return false; - } - if RESERVED.contains(&name) { - errors.push(ValidationError::ReservedKeyword(name.to_string())); - return false; - } - true +/// `scope` extended with `names`. +fn within(scope: &[String], names: &[&str]) -> Vec { + let mut out = scope.to_vec(); + out.extend(names.iter().map(|n| (*n).to_string())); + out } -/// The kinds of declarations a bare name can refer to, gathered from the -/// whole module forest before any rule runs. Validation operates on the -/// document as written (every user reference is a [`TypeRef::Named`]), so -/// positional rules consult this index to learn what a name is. -#[derive(Default)] -pub(super) struct TypeIndex { - /// Bare name of every struct, enum, interface, and callback interface - /// anywhere in the API, mapped to its declaring module's dot path. - pub all: BTreeMap, - /// Bare names of every interface anywhere in the API. - pub interfaces: BTreeSet, - /// Bare names of every callback interface anywhere in the API. - pub callback_interfaces: BTreeSet, - /// Bare names of every C-style (payload-free) enum anywhere in the API. - pub plain_enums: BTreeSet, +/// Run every rule over `api`, whose declarations `types` indexes. +pub(super) fn check(api: &Api, types: &TypeIndex, options: Options, found: &mut Vec) { + let mut cx = Cx { + types, + options, + found, + }; + let mut names = BTreeSet::new(); + for m in &api.modules { + if !names.insert(m.name.as_str()) { + cx.push( + &within(&[], &[&m.name]), + ValidationError::DuplicateModuleName { + module: m.name.clone(), + }, + ); + } + cx.module(m, &[], false); + } + check_global_names(&api.modules, found); } -impl TypeIndex { - pub(super) fn build(modules: &[Module]) -> Self { - let mut index = Self::default(); - index.walk(modules, ""); - index - } - - fn walk(&mut self, modules: &[Module], parent: &str) { - for m in modules { - let path = if parent.is_empty() { - m.name.clone() - } else { - format!("{parent}.{}", m.name) - }; - let mut declare = |name: &str| { - self.all - .entry(name.to_string()) - .or_insert_with(|| path.clone()); - }; - for s in &m.structs { - declare(&s.name); - } - for e in &m.enums { - declare(&e.name); - if !e.is_rich() { - self.plain_enums.insert(e.name.clone()); - } - } - for i in &m.interfaces { - declare(&i.name); - self.interfaces.insert(i.name.clone()); - } - for cb in &m.callback_interfaces { - declare(&cb.name); - self.callback_interfaces.insert(cb.name.clone()); - } - self.walk(&m.modules, &path); - } - } - - /// Is `name` (bare or dot-qualified) an interface? - fn is_interface(&self, name: &str) -> bool { - self.interfaces.contains(bare(name)) - } - - /// Is `name` (bare or dot-qualified) a callback interface? - fn is_callback_interface(&self, name: &str) -> bool { - self.callback_interfaces.contains(bare(name)) - } - - /// Does a declaration with this name exist? A bare name may refer to a - /// declaration in any module; a dot-qualified name must spell the - /// declaring module's path exactly. - fn exists(&self, name: &str) -> bool { - match (self.all.get(bare(name)), name.rsplit_once('.')) { - (None, _) => false, - (Some(_), None) => true, - (Some(path), Some((qualifier, _))) => path == qualifier, - } - } - - /// May `ty` be a map key? Every target must be able to use the key in - /// its native dictionary idiom with exact equality, so only integers, - /// bools, strings, and C-style enums qualify; floats (NaN and signed zero - /// break key equality in several languages), composites, optionals, - /// bytes, and objects are rejected. - fn is_map_key(&self, ty: &TypeRef) -> bool { - match ty { - TypeRef::I8 - | TypeRef::I16 - | TypeRef::I32 - | TypeRef::U8 - | TypeRef::U16 - | TypeRef::U32 - | TypeRef::I64 - | TypeRef::U64 - | TypeRef::Bool - | TypeRef::StringUtf8 => true, - TypeRef::Named(name) => self.plain_enums.contains(bare(name)), - _ => false, - } - } - - /// May `ty` be returned from a callback-interface method? Only `void` - /// and the direct family: scalars, bool, and C-style enums. - fn is_direct(&self, ty: &TypeRef) -> bool { - match ty { - TypeRef::I8 - | TypeRef::I16 - | TypeRef::I32 - | TypeRef::U8 - | TypeRef::U16 - | TypeRef::U32 - | TypeRef::I64 - | TypeRef::U64 - | TypeRef::F32 - | TypeRef::F64 - | TypeRef::Bool => true, - TypeRef::Named(name) => self.plain_enums.contains(bare(name)), - _ => false, - } +/// Enforce API-wide name uniqueness: every type name (records, enums, +/// interfaces, callback interfaces, and error domains), every free-function +/// name, and every error-code name is global. A free function also may not +/// share its name with an error domain. +fn check_global_names(modules: &[Module], found: &mut Vec) { + #[derive(Default)] + struct Seen<'a> { + types: BTreeMap<&'a str, String>, + functions: BTreeMap<&'a str, String>, + codes: BTreeMap<&'a str, String>, + domains: BTreeMap<&'a str, String>, + /// Every free function, as `(name, module path, module segments)`. + all_functions: Vec<(&'a str, String, Vec)>, } -} - -/// The final segment of a possibly dot-qualified name. -fn bare(name: &str) -> &str { - name.rsplit('.').next().unwrap_or(name) -} - -/// Enforce global bare-name uniqueness for the type namespace: structs, -/// enums, interfaces, callback interfaces, and error domains across every -/// module (including nested submodules). -/// -/// Generators emit flat per-language type names, and unqualified cross-module -/// references resolve by bare name, so two types sharing a name would collide -/// in generated code and make references ambiguous. -pub(super) fn check_global_type_names(modules: &[Module], errors: &mut Vec) { fn walk<'a>( modules: &'a [Module], - prefix: &str, - seen: &mut BTreeMap<&'a str, String>, - errors: &mut Vec, + parent: &[String], + seen: &mut Seen<'a>, + found: &mut Vec, ) { for m in modules { - let path = if prefix.is_empty() { - m.name.clone() - } else { - format!("{prefix}.{}", m.name) - }; - let names = m + let segments = within(parent, &[&m.name]); + let path = segments.join("."); + let types = m .structs .iter() .map(|s| s.name.as_str()) @@ -195,609 +98,711 @@ pub(super) fn check_global_type_names(modules: &[Module], errors: &mut Vec found.push(( + ValidationError::DuplicateTypeName { + name: name.to_string(), + first: first.clone(), + second: path.clone(), + }, + segments.clone(), + )), + None => { + seen.types.insert(name, path.clone()); + } } } - walk(&m.modules, &path, seen, errors); - } - } - let mut seen = BTreeMap::new(); - walk(modules, "", &mut seen, errors); -} - -/// Enforce API-global uniqueness of error *code* names across domains. -/// -/// Backends with flat namespaces derive one error class or constant per -/// code, so `NotFound` declared in two different domains would collide in -/// generated code even though each domain is internally consistent. -pub(super) fn check_global_error_code_names(modules: &[Module], errors: &mut Vec) { - fn walk<'a>( - modules: &'a [Module], - prefix: &str, - seen: &mut BTreeMap<&'a str, String>, - errors: &mut Vec, - ) { - for m in modules { - let path = if prefix.is_empty() { - m.name.clone() - } else { - format!("{prefix}.{}", m.name) - }; + for f in &m.functions { + match seen.functions.get(f.name.as_str()) { + Some(first) => found.push(( + ValidationError::DuplicateFunctionName { + name: f.name.clone(), + first: first.clone(), + second: path.clone(), + }, + segments.clone(), + )), + None => { + seen.functions.insert(&f.name, path.clone()); + } + } + seen.all_functions + .push((&f.name, path.clone(), segments.clone())); + } if let Some(domain) = &m.errors { + seen.domains.entry(&domain.name).or_insert(path.clone()); let owner = format!("{path}.{}", domain.name); for code in &domain.codes { - if let Some(first) = seen.get(code.name.as_str()) { - errors.push(ValidationError::DuplicateErrorCodeName { - name: code.name.clone(), - first: first.clone(), - second: owner.clone(), - }); - } else { - seen.insert(&code.name, owner.clone()); + match seen.codes.get(code.name.as_str()) { + Some(first) => found.push(( + ValidationError::DuplicateErrorCodeName { + name: code.name.clone(), + first: first.clone(), + second: owner.clone(), + }, + within(&segments, &[&domain.name]), + )), + None => { + seen.codes.insert(&code.name, owner.clone()); + } } } } - walk(&m.modules, &path, seen, errors); + walk(&m.modules, &segments, seen, found); } } - let mut seen = BTreeMap::new(); - walk(modules, "", &mut seen, errors); -} - -pub(super) fn validate_module( - module: &Module, - types: &TypeIndex, - ancestor_has_domain: bool, - errors: &mut Vec, -) { - if module.name.trim().is_empty() { - errors.push(ValidationError::NoModuleName); - return; - } - if !is_valid_identifier(&module.name) { - errors.push(ValidationError::InvalidModuleName( - module.name.clone(), - "must start with a letter or underscore and contain only alphanumeric characters or underscores", - )); - } else if RESERVED.contains(&module.name.as_str()) { - errors.push(ValidationError::InvalidModuleName( - module.name.clone(), - "reserved word", - )); - } - - let has_domain = ancestor_has_domain || module.errors.is_some(); - - let mut function_names = BTreeSet::new(); - for f in &module.functions { - if !function_names.insert(f.name.clone()) { - errors.push(ValidationError::DuplicateFunctionName { - module: module.name.clone(), - function: f.name.clone(), - }); + let mut seen = Seen::default(); + walk(modules, &[], &mut seen, found); + for (name, function, segments) in &seen.all_functions { + if let Some(domain) = seen.domains.get(name) { + found.push(( + ValidationError::NameCollisionWithErrorDomain { + name: (*name).to_string(), + function: function.clone(), + domain: domain.clone(), + }, + segments.clone(), + )); } - validate_function(&module.name, &f.name, f, has_domain, errors); } +} - let mut struct_names = BTreeSet::new(); - for s in &module.structs { - check_identifier(&s.name, errors); - if !struct_names.insert(s.name.clone()) { - errors.push(ValidationError::DuplicateStructName { - module: module.name.clone(), - name: s.name.clone(), - }); - } - if s.fields.is_empty() { - errors.push(ValidationError::EmptyStruct { - module: module.name.clone(), - name: s.name.clone(), - }); - } - let mut field_names = BTreeSet::new(); - for f in &s.fields { - check_identifier(&f.name, errors); - if !field_names.insert(f.name.clone()) { - errors.push(ValidationError::DuplicateStructField { - struct_name: s.name.clone(), - field: f.name.clone(), - }); +/// The rule context: the type index, the options, and the violation sink. +struct Cx<'a> { + types: &'a TypeIndex, + options: Options, + found: &'a mut Vec, +} + +impl Cx<'_> { + fn push(&mut self, scope: &[String], error: ValidationError) { + self.found.push((error, scope.to_vec())); + } + + /// Check that `name` is a valid, non-reserved identifier. + fn identifier(&mut self, scope: &[String], name: &str) { + if !is_valid_identifier(name) { + self.push( + scope, + ValidationError::InvalidIdentifier { + name: name.to_string(), + reason: IDENTIFIER_RULE, + }, + ); + } else if RESERVED.contains(&name) { + self.push( + scope, + ValidationError::ReservedKeyword { + name: name.to_string(), + }, + ); + } + } + + fn kind(&self, name: &str) -> Option { + self.types.kind(name) + } + + fn module(&mut self, module: &Module, parent: &[String], ancestor_has_domain: bool) { + if module.name.trim().is_empty() { + self.push(parent, ValidationError::NoModuleName); + return; + } + if !is_valid_identifier(&module.name) { + self.push( + parent, + ValidationError::InvalidModuleName { + name: module.name.clone(), + reason: IDENTIFIER_RULE, + }, + ); + } else if RESERVED.contains(&module.name.as_str()) { + self.push( + parent, + ValidationError::InvalidModuleName { + name: module.name.clone(), + reason: "reserved word", + }, + ); + } + let scope = within(parent, &[&module.name]); + let path = scope.join("."); + let has_domain = ancestor_has_domain || module.errors.is_some(); + + for f in &module.functions { + self.function(&scope, &path, &f.name, f, has_domain); + self.callable_types(&within(&scope, &[&f.name]), &path, &f.name, f); + } + + for s in &module.structs { + self.identifier(&scope, &s.name); + if s.fields.is_empty() { + self.push( + &scope, + ValidationError::EmptyStruct { + module: path.clone(), + name: s.name.clone(), + }, + ); + } + let decl = within(&scope, &[&s.name]); + let mut names = BTreeSet::new(); + for f in &s.fields { + self.identifier(&decl, &f.name); + if !names.insert(&f.name) { + self.push( + &decl, + ValidationError::DuplicateStructField { + struct_name: s.name.clone(), + field: f.name.clone(), + }, + ); + } + let location = || format!("field '{}' of struct '{}'", f.name, s.name); + self.buffered_field(&decl, f, &location); } } - } - let mut enum_names = BTreeSet::new(); - for e in &module.enums { - check_identifier(&e.name, errors); - if !enum_names.insert(e.name.clone()) { - errors.push(ValidationError::DuplicateEnumName { - module: module.name.clone(), - name: e.name.clone(), - }); - } - if e.variants.is_empty() { - errors.push(ValidationError::EmptyEnum { - module: module.name.clone(), - name: e.name.clone(), - }); - } - let mut variant_names = BTreeSet::new(); - let mut variant_values = BTreeMap::new(); - for v in &e.variants { - check_identifier(&v.name, errors); - if !variant_names.insert(v.name.clone()) { - errors.push(ValidationError::DuplicateEnumVariant { - enum_name: e.name.clone(), - variant: v.name.clone(), - }); - } - if variant_values.insert(v.value, v.name.clone()).is_some() { - errors.push(ValidationError::DuplicateEnumValue { - enum_name: e.name.clone(), - value: v.value, - }); - } - let mut variant_field_names = BTreeSet::new(); - for f in &v.fields { - check_identifier(&f.name, errors); - if !variant_field_names.insert(f.name.clone()) { - errors.push(ValidationError::DuplicateEnumVariantField { - enum_name: e.name.clone(), - variant: v.name.clone(), - field: f.name.clone(), - }); + for e in &module.enums { + self.identifier(&scope, &e.name); + if e.variants.is_empty() { + self.push( + &scope, + ValidationError::EmptyEnum { + module: path.clone(), + name: e.name.clone(), + }, + ); + } + let decl = within(&scope, &[&e.name]); + let mut names = BTreeSet::new(); + let mut values = BTreeSet::new(); + for v in &e.variants { + self.identifier(&decl, &v.name); + if !names.insert(&v.name) { + self.push( + &decl, + ValidationError::DuplicateEnumVariant { + enum_name: e.name.clone(), + variant: v.name.clone(), + }, + ); + } + if !values.insert(v.value) { + self.push( + &scope, + ValidationError::DuplicateEnumValue { + enum_name: e.name.clone(), + value: v.value, + }, + ); + } + let variant = within(&decl, &[&v.name]); + let mut fields = BTreeSet::new(); + for f in &v.fields { + self.identifier(&variant, &f.name); + if !fields.insert(&f.name) { + self.push( + &variant, + ValidationError::DuplicateEnumVariantField { + enum_name: e.name.clone(), + variant: v.name.clone(), + field: f.name.clone(), + }, + ); + } + let location = + || format!("field '{}' of variant '{}::{}'", f.name, e.name, v.name); + self.buffered_field(&variant, f, &location); } } } - } - let mut local_interface_names = BTreeSet::new(); - for i in &module.interfaces { - check_identifier(&i.name, errors); - if !local_interface_names.insert(i.name.clone()) { - errors.push(ValidationError::DuplicateInterfaceName { - module: module.name.clone(), - name: i.name.clone(), - }); + for i in &module.interfaces { + self.identifier(&scope, &i.name); + self.interface(&scope, &path, i, has_domain); } - validate_interface(module, i, has_domain, errors); - } - let mut callback_names = BTreeSet::new(); - for cb in &module.callback_interfaces { - check_identifier(&cb.name, errors); - if !callback_names.insert(cb.name.clone()) { - errors.push(ValidationError::DuplicateCallbackInterfaceName { - module: module.name.clone(), - name: cb.name.clone(), - }); + for cb in &module.callback_interfaces { + self.identifier(&scope, &cb.name); + self.callback_interface(&scope, &path, cb, has_domain); } - validate_callback_interface(module, cb, types, errors); - } - // A field of a record, a rich-enum variant, or an error payload is - // serialized inside a value buffer, so it obeys the buffered positional - // rules: no iterators, no callback interfaces, no interface map keys, and - // every reference must resolve. - let mut check_buffered_field = |f: &StructField, location: &dyn Fn() -> String| { - if contains_iterator(&f.ty) { - errors.push(ValidationError::IteratorInInvalidPosition { - location: location(), - }); + if let Some(domain) = &module.errors { + self.error_domain(&scope, &path, domain); } - validate_type_ref(&f.ty, types, errors); - check_interface_positions(&f.ty, types, location, errors); - check_no_callback_interface(&f.ty, types, location, errors); - }; - for s in &module.structs { - for f in &s.fields { - let location = || format!("field '{}' of struct '{}'", f.name, s.name); - check_buffered_field(f, &location); + + let mut names = BTreeSet::new(); + for sub in &module.modules { + if !names.insert(sub.name.as_str()) { + self.push( + &within(&scope, &[&sub.name]), + ValidationError::DuplicateModuleName { + module: format!("{path}.{}", sub.name), + }, + ); + } + self.module(sub, &scope, has_domain); } } - for e in &module.enums { - for v in &e.variants { - for f in &v.fields { - let location = || format!("field '{}' of variant '{}::{}'", f.name, e.name, v.name); - check_buffered_field(f, &location); + + /// A field of a record, a rich-enum variant, or an error payload is + /// serialized inside a value buffer, so it obeys the buffered positional + /// rules: no iterators, no callback interfaces, no interface map keys, + /// and every reference must resolve. + fn buffered_field(&mut self, scope: &[String], f: &StructField, location: &dyn Fn() -> String) { + let scope = within(scope, &[&f.name]); + if contains_iterator(&f.ty) { + self.push( + &scope, + ValidationError::IteratorInInvalidPosition { + location: location(), + }, + ); + } + self.type_ref(&scope, &f.ty); + self.interface_positions(&scope, &f.ty, location); + self.no_callback_interface(&scope, &f.ty, location); + } + + /// Validate an interface's shape: unique member names across + /// constructors, methods, and statics; constructor restrictions; + /// per-member signature rules. C symbol collisions are checked API-wide + /// once the model is built. + fn interface(&mut self, scope: &[String], path: &str, iface: &InterfaceDef, has_domain: bool) { + if iface.constructors.is_empty() && iface.methods.is_empty() && iface.statics.is_empty() { + self.push( + scope, + ValidationError::EmptyInterface { + module: path.to_string(), + name: iface.name.clone(), + }, + ); + } + let decl = within(scope, &[&iface.name]); + let mut names = BTreeSet::new(); + for (f, constructor) in iface.constructors.iter().map(|c| (c, true)).chain( + iface + .methods + .iter() + .chain(&iface.statics) + .map(|m| (m, false)), + ) { + if !names.insert(&f.name) { + self.push( + &decl, + ValidationError::DuplicateInterfaceMember { + interface: iface.name.clone(), + name: f.name.clone(), + }, + ); + } + let display = format!("{}.{}", iface.name, f.name); + self.function(&decl, path, &display, f, has_domain); + self.callable_types(&within(&decl, &[&f.name]), path, &display, f); + if constructor && f.returns.is_some() { + self.push( + &decl, + ValidationError::ConstructorHasReturn { + interface: iface.name.clone(), + constructor: f.name.clone(), + }, + ); + } + if constructor && f.r#async { + self.push( + &decl, + ValidationError::AsyncConstructor { + interface: iface.name.clone(), + constructor: f.name.clone(), + }, + ); } } } - if let Some(domain) = &module.errors { - for c in &domain.codes { - for f in &c.fields { + + /// Validate a callback interface: at least one method, unique method + /// names, and per-method restrictions. A callback method is implemented + /// by the consumer, so it is synchronous, never takes a cancel token, + /// returns nothing or any value but an iterator or a callback interface, + /// takes no callback interface or iterator as a parameter, and may + /// declare `throws` only when an error domain is in scope. + fn callback_interface( + &mut self, + scope: &[String], + path: &str, + cb: &CallbackInterfaceDef, + has_domain: bool, + ) { + if cb.methods.is_empty() { + self.push( + scope, + ValidationError::EmptyCallbackInterface { + module: path.to_string(), + name: cb.name.clone(), + }, + ); + } + let decl = within(scope, &[&cb.name]); + let mut names = BTreeSet::new(); + for m in &cb.methods { + self.identifier(&decl, &m.name); + if !names.insert(&m.name) { + self.push( + &decl, + ValidationError::DuplicateCallbackMethod { + interface: cb.name.clone(), + name: m.name.clone(), + }, + ); + } + let reject = |cx: &mut Self, reason: &'static str| { + cx.push( + &decl, + ValidationError::InvalidCallbackMethod { + interface: cb.name.clone(), + method: m.name.clone(), + reason, + }, + ); + }; + if m.r#async { + reject(self, "cannot be async"); + } + if m.cancellable { + reject(self, "cannot be cancellable"); + } + let method = within(&decl, &[&m.name]); + if m.throws && !has_domain { + self.push( + &method, + ValidationError::ThrowsWithoutErrorDomain { + module: path.to_string(), + function: format!("{}.{}", cb.name, m.name), + }, + ); + } + if let Some(ret) = &m.returns { + let location = || format!("return type of {path}::{}.{}", cb.name, m.name); + if contains_iterator(ret) { + reject(self, "cannot return an iterator"); + } + self.type_ref(&method, ret); + self.interface_positions(&method, ret, &location); + self.no_callback_interface(&method, ret, &location); + } + let mut params = BTreeSet::new(); + for p in &m.params { + self.identifier(&method, &p.name); + if !params.insert(&p.name) { + self.push( + &method, + ValidationError::DuplicateParamName { + module: path.to_string(), + function: format!("{}.{}", cb.name, m.name), + param: p.name.clone(), + }, + ); + } + let param = within(&method, &[&p.name]); let location = || { format!( - "payload field '{}' of error code '{}::{}'", - f.name, domain.name, c.name + "param '{}' of callback interface method '{}.{}'", + p.name, cb.name, m.name ) }; - check_buffered_field(f, &location); + if contains_iterator(&p.ty) { + self.push( + ¶m, + ValidationError::IteratorInInvalidPosition { + location: location(), + }, + ); + } + self.type_ref(¶m, &p.ty); + self.interface_positions(¶m, &p.ty, &location); + self.no_callback_interface(¶m, &p.ty, &location); } } } - for f in &module.functions { - validate_callable_types(&module.name, &f.name, f, types, errors); - } - for i in &module.interfaces { - for f in i.constructors.iter().chain(&i.methods).chain(&i.statics) { - let display = format!("{}.{}", i.name, f.name); - validate_callable_types(&module.name, &display, f, types, errors); - } - } - - if let Some(domain) = &module.errors { - validate_error_domain(module, domain, &function_names, errors); - } - let mut sub_module_names = BTreeSet::new(); - for sub in &module.modules { - if !sub_module_names.insert(sub.name.clone()) { - errors.push(ValidationError::DuplicateModuleName(sub.name.clone())); - } - validate_module(sub, types, has_domain, errors); - } -} - -/// Validate an interface's shape: unique member names across constructors, -/// methods, and statics; constructor restrictions; per-member signature -/// rules. C symbol collisions are checked API-wide once the model is built. -fn validate_interface( - module: &Module, - iface: &InterfaceDef, - has_domain: bool, - errors: &mut Vec, -) { - if iface.constructors.is_empty() && iface.methods.is_empty() && iface.statics.is_empty() { - errors.push(ValidationError::EmptyInterface { - module: module.name.clone(), - name: iface.name.clone(), - }); - } - let mut member_names = BTreeSet::new(); - let mut check_member = |f: &Function, errors: &mut Vec| { - if !member_names.insert(f.name.clone()) { - errors.push(ValidationError::DuplicateInterfaceMember { - interface: iface.name.clone(), - name: f.name.clone(), - }); - } - let display = format!("{}.{}", iface.name, f.name); - validate_function(&module.name, &display, f, has_domain, errors); - }; - for c in &iface.constructors { - check_member(c, errors); - if c.returns.is_some() { - errors.push(ValidationError::ConstructorHasReturn { - interface: iface.name.clone(), - constructor: c.name.clone(), - }); - } - if c.r#async { - errors.push(ValidationError::AsyncConstructor { - interface: iface.name.clone(), - constructor: c.name.clone(), - }); + /// Name-level checks for one callable declared in `scope`: a valid + /// identifier, unique parameter names, and an error domain in scope when + /// the callable declares `throws`. + fn function( + &mut self, + scope: &[String], + path: &str, + display: &str, + f: &Function, + has_domain: bool, + ) { + self.identifier(scope, &f.name); + let decl = within(scope, &[&f.name]); + if f.cancellable && !f.r#async { + self.push( + &decl, + ValidationError::CancellableNotAsync { + module: path.to_string(), + function: display.to_string(), + }, + ); + } + if f.throws && !has_domain { + self.push( + &decl, + ValidationError::ThrowsWithoutErrorDomain { + module: path.to_string(), + function: display.to_string(), + }, + ); + } + let mut names = BTreeSet::new(); + for p in &f.params { + self.identifier(&decl, &p.name); + if !names.insert(&p.name) { + self.push( + &decl, + ValidationError::DuplicateParamName { + module: path.to_string(), + function: display.to_string(), + param: p.name.clone(), + }, + ); + } } } - for m in &iface.methods { - check_member(m, errors); - } - for s in &iface.statics { - check_member(s, errors); - } -} -/// Validate a callback interface: at least one method, unique method names, -/// and per-method restrictions. A callback method is implemented by the -/// consumer, so it is synchronous, never throws, never takes a cancel token, -/// returns nothing or a direct value, and takes no callback interface or -/// iterator as a parameter. -fn validate_callback_interface( - module: &Module, - cb: &CallbackInterfaceDef, - types: &TypeIndex, - errors: &mut Vec, -) { - if cb.methods.is_empty() { - errors.push(ValidationError::EmptyCallbackInterface { - module: module.name.clone(), - name: cb.name.clone(), - }); - } - let mut method_names = BTreeSet::new(); - for m in &cb.methods { - check_identifier(&m.name, errors); - if !method_names.insert(m.name.clone()) { - errors.push(ValidationError::DuplicateCallbackMethod { - interface: cb.name.clone(), - name: m.name.clone(), - }); - } - let reject = |reason: &'static str, errors: &mut Vec| { - errors.push(ValidationError::InvalidCallbackMethod { - interface: cb.name.clone(), - method: m.name.clone(), - reason, - }); - }; - if m.r#async { - reject("cannot be async", errors); - } - if m.throws { - reject("cannot declare throws", errors); - } - if m.cancellable { - reject("cannot be cancellable", errors); - } - if let Some(ret) = &m.returns { - validate_type_ref(ret, types, errors); - if !types.is_direct(ret) { - reject( - "must return nothing or a direct value (scalar, bool, or C-style enum)", - errors, + /// Type-level checks for one callable's parameters and return: iterator + /// positions, async-iterator exclusion, reference resolution, element + /// shapes, interface map keys, and callback-interface positions. `scope` + /// ends with the callable's own name. + fn callable_types(&mut self, scope: &[String], path: &str, display: &str, f: &Function) { + for p in &f.params { + let param = within(scope, &[&p.name]); + let location = || format!("param '{}' of function '{path}::{display}'", p.name); + if contains_iterator(&p.ty) { + self.push( + ¶m, + ValidationError::IteratorInInvalidPosition { + location: location(), + }, ); } - } - let mut param_names = BTreeSet::new(); - for p in &m.params { - validate_param(p, errors); - if !param_names.insert(p.name.clone()) { - errors.push(ValidationError::DuplicateParamName { - module: module.name.clone(), - function: format!("{}.{}", cb.name, m.name), - param: p.name.clone(), - }); - } - let location = || { - format!( - "param '{}' of callback interface method '{}.{}'", - p.name, cb.name, m.name - ) + self.type_ref(¶m, &p.ty); + self.interface_positions(¶m, &p.ty, &location); + // A bare or optional callback interface is the one legal + // callback position; anything nested (`[Listener]`, a record + // field) is not. + let top = match &p.ty { + TypeRef::Optional(inner) => inner.as_ref(), + other => other, }; - if contains_iterator(&p.ty) { - errors.push(ValidationError::IteratorInInvalidPosition { - location: location(), - }); + match top { + TypeRef::Named(name) if self.kind(name) == Some(TypeKind::CallbackInterface) => {} + _ => self.no_callback_interface(¶m, &p.ty, &location), } - validate_type_ref(&p.ty, types, errors); - check_interface_positions(&p.ty, types, location, errors); - check_no_callback_interface(&p.ty, types, location, errors); } - } -} - -/// Name-level checks for one callable: a valid identifier, unique parameter -/// names, and an error domain in scope when the callable declares `throws`. -fn validate_function( - module_name: &str, - display_name: &str, - f: &Function, - has_domain: bool, - errors: &mut Vec, -) { - check_identifier(&f.name, errors); - - if f.cancellable && !f.r#async { - errors.push(ValidationError::CancellableNotAsync { - module: module_name.to_string(), - function: display_name.to_string(), + if let Some(ret) = &f.returns { + let location = || format!("return type of {path}::{display}"); + // An async function completes through a one-shot callback; an + // iterator needs a pull-based handle. The two shapes cannot + // compose on the C ABI, so reject the combination up front + // instead of letting backends lower it inconsistently. + if f.r#async && contains_iterator(ret) { + self.push( + scope, + ValidationError::AsyncIteratorReturn { + module: path.to_string(), + function: display.to_string(), + }, + ); + } + // An iterator is a pull handle, valid only as the outermost + // return shape: `[iter]`, `iter?`, `iter>`, and + // iterators inside a map have no lowering. + let nested_iterator = match ret { + TypeRef::Iterator(elem) => contains_iterator(elem), + other => contains_iterator(other), + }; + if nested_iterator { + self.push( + scope, + ValidationError::IteratorInInvalidPosition { + location: format!("nested inside the {}", location()), + }, + ); + } + self.type_ref(scope, ret); + self.interface_positions(scope, ret, &location); + self.no_callback_interface(scope, ret, &location); + } + } + + /// Enforce the one place an interface reference may not appear: as a + /// map key. Objects are reference-counted tokens, so they compose with + /// every other buffered shape (fields, elements, map values, optionals), + /// but no target can hash an object by identity in a way that survives + /// the ABI. + fn interface_positions( + &mut self, + scope: &[String], + ty: &TypeRef, + location: &dyn Fn() -> String, + ) { + let mut keys = Vec::new(); + ty.walk(&mut |t| { + if let TypeRef::Map(k, _) = t { + if let TypeRef::Named(name) = &**k { + keys.push(name.clone()); + } + } }); + for name in keys { + if self.kind(&name) == Some(TypeKind::Interface) { + self.push( + scope, + ValidationError::InterfaceInInvalidPosition { + name, + location: format!("map key of {}", location()), + }, + ); + } + } } - if f.throws && !has_domain { - errors.push(ValidationError::ThrowsWithoutErrorDomain { - module: module_name.to_string(), - function: display_name.to_string(), + /// Reject any callback-interface reference reachable from `ty`. Callers + /// that permit a bare top-level callback interface unwrap it first. + fn no_callback_interface( + &mut self, + scope: &[String], + ty: &TypeRef, + location: &dyn Fn() -> String, + ) { + let mut names = Vec::new(); + ty.walk(&mut |t| { + if let TypeRef::Named(name) = t { + names.push(name.clone()); + } }); - } - - let mut param_names = BTreeSet::new(); - for p in &f.params { - validate_param(p, errors); - if !param_names.insert(p.name.clone()) { - errors.push(ValidationError::DuplicateParamName { - module: module_name.to_string(), - function: display_name.to_string(), - param: p.name.clone(), - }); + for name in names { + if self.kind(&name) == Some(TypeKind::CallbackInterface) { + self.push( + scope, + ValidationError::CallbackInterfaceInInvalidPosition { + name, + location: location(), + }, + ); + } } } -} -/// Type-level checks for one callable's parameters and return: iterator -/// positions, async-iterator exclusion, reference existence, element shapes, -/// interface map keys, and callback-interface positions. -fn validate_callable_types( - module_name: &str, - display_name: &str, - f: &Function, - types: &TypeIndex, - errors: &mut Vec, -) { - for p in &f.params { - let location = || { - format!( - "param '{}' of function '{module_name}::{display_name}'", - p.name - ) - }; - if contains_iterator(&p.ty) { - errors.push(ValidationError::IteratorInInvalidPosition { - location: location(), - }); - } - validate_type_ref(&p.ty, types, errors); - check_interface_positions(&p.ty, types, location, errors); - // A bare callback interface is the one legal callback position; - // anything nested (`Listener?`, `[Listener]`) is not. - match &p.ty { - TypeRef::Named(name) if types.is_callback_interface(name) => {} - other => check_no_callback_interface(other, types, location, errors), + /// Check that every name in `ty` resolves to a declaration, and that + /// every map key is a legal key type. + fn type_ref(&mut self, scope: &[String], ty: &TypeRef) { + match ty { + TypeRef::Named(name) => { + let error = if name.contains('.') { + ValidationError::QualifiedTypeRef { name: name.clone() } + } else if self.kind(name).is_some() { + return; + } else if UNSUPPORTED_PRIMITIVES.contains(&name.as_str()) { + ValidationError::UnsupportedPrimitive { name: name.clone() } + } else if self.options.foreign_names { + // A record or rich enum from another module tree. + return; + } else { + ValidationError::UnknownTypeRef { name: name.clone() } + }; + self.push(scope, error); + } + TypeRef::Optional(inner) | TypeRef::List(inner) | TypeRef::Iterator(inner) => { + self.type_ref(scope, inner); + } + TypeRef::Map(k, v) => { + if !self.is_map_key(k) { + self.push( + scope, + ValidationError::InvalidMapKey { + key_type: k.to_string(), + }, + ); + } + self.type_ref(scope, k); + self.type_ref(scope, v); + } + TypeRef::Prim(_) => {} } } - if let Some(ret) = &f.returns { - let location = || format!("return type of {module_name}::{display_name}"); - // An async function completes through a one-shot callback; an - // iterator needs a pull-based handle. The two shapes cannot - // compose on the C ABI, so reject the combination up front - // instead of letting backends lower it inconsistently. - if f.r#async && contains_iterator(ret) { - errors.push(ValidationError::AsyncIteratorReturn { - module: module_name.to_string(), - function: display_name.to_string(), - }); - } - // An iterator is a pull handle, valid only as the outermost return - // shape: `[iter]`, `iter?`, `iter>`, and iterators - // inside a map have no lowering. - let nested_iterator = match ret { - TypeRef::Iterator(elem) => contains_iterator(elem), - other => contains_iterator(other), - }; - if nested_iterator { - errors.push(ValidationError::IteratorInInvalidPosition { - location: format!("nested inside the {}", location()), - }); + + /// May `ty` be a map key? Every target must be able to use the key in + /// its native dictionary idiom with exact equality, so only integers, + /// bools, strings, and C-style enums qualify; floats (NaN and signed zero + /// break key equality in several languages), composites, optionals, + /// bytes, and objects are rejected. + fn is_map_key(&self, ty: &TypeRef) -> bool { + match ty { + TypeRef::Prim(p) => p.is_integer() || matches!(p, Prim::Bool | Prim::String), + TypeRef::Named(name) => self.kind(name) == Some(TypeKind::Enum), + _ => false, } - validate_type_ref(ret, types, errors); - check_interface_positions(ret, types, location, errors); - check_no_callback_interface(ret, types, location, errors); } -} -fn validate_param(p: &Param, errors: &mut Vec) { - check_identifier(&p.name, errors); -} - -/// Enforce the one place an interface reference may not appear: as a map -/// key. Objects are reference-counted tokens, so they compose with every -/// other buffered shape (fields, elements, map values, optionals), but no -/// target can hash an object by identity in a way that survives the ABI. -fn check_interface_positions( - ty: &TypeRef, - types: &TypeIndex, - location: impl Fn() -> String + Copy, - errors: &mut Vec, -) { - ty.walk(&mut |t| { - if let TypeRef::Map(k, _) = t { - if let TypeRef::Named(name) = &**k { - if types.is_interface(name) { - errors.push(ValidationError::InterfaceInInvalidPosition { - name: name.clone(), - location: format!("map key of {}", location()), - }); - } - } + fn error_domain(&mut self, scope: &[String], path: &str, domain: &ErrorDomain) { + if domain.name.trim().is_empty() { + self.push( + scope, + ValidationError::ErrorDomainMissingName { + module: path.to_string(), + }, + ); + return; } - }); -} - -/// Reject any callback-interface reference reachable from `ty`. Callers that -/// permit a bare top-level callback interface unwrap it before calling. -fn check_no_callback_interface( - ty: &TypeRef, - types: &TypeIndex, - location: impl Fn() -> String + Copy, - errors: &mut Vec, -) { - ty.walk(&mut |t| { - if let TypeRef::Named(name) = t { - if types.is_callback_interface(name) { - errors.push(ValidationError::CallbackInterfaceInInvalidPosition { - name: name.clone(), - location: location(), - }); + self.identifier(scope, &domain.name); + let decl = within(scope, &[&domain.name]); + let mut values = BTreeSet::new(); + for c in &domain.codes { + self.identifier(&decl, &c.name); + // 0 means success and the whole negative range is reserved for + // the runtime (-1 generic error, -2 panic, -3 marshalling + // failure, -4 foreign callback error, -5 cancelled, and room to + // grow), so domain codes must be positive. + if c.code <= 0 { + self.push( + &decl, + ValidationError::InvalidErrorCode { + module: path.to_string(), + name: c.name.clone(), + }, + ); + } + if !values.insert(c.code) { + self.push( + &decl, + ValidationError::DuplicateErrorCode { + module: path.to_string(), + value: c.code, + }, + ); + } + let code = within(&decl, &[&c.name]); + for f in &c.fields { + let location = || { + format!( + "payload field '{}' of error code '{}::{}'", + f.name, domain.name, c.name + ) + }; + self.buffered_field(&code, f, &location); } } - }); + } } fn contains_iterator(ty: &TypeRef) -> bool { let mut found = false; - ty.walk(&mut |t| { - if matches!(t, TypeRef::Iterator(_)) { - found = true; - } - }); + ty.walk(&mut |t| found |= matches!(t, TypeRef::Iterator(_))); found } - -fn validate_type_ref(ty: &TypeRef, types: &TypeIndex, errors: &mut Vec) { - match ty { - TypeRef::Named(name) => { - if !types.exists(name) { - errors.push(ValidationError::UnknownTypeRef { name: name.clone() }); - } - } - TypeRef::Optional(inner) | TypeRef::List(inner) | TypeRef::Iterator(inner) => { - validate_type_ref(inner, types, errors); - } - TypeRef::Map(k, v) => { - if !types.is_map_key(k) { - errors.push(ValidationError::InvalidMapKey { - key_type: k.to_string(), - }); - } - validate_type_ref(k, types, errors); - validate_type_ref(v, types, errors); - } - _ => {} - } -} - -fn validate_error_domain( - module: &Module, - domain: &ErrorDomain, - function_names: &BTreeSet, - errors: &mut Vec, -) { - if domain.name.trim().is_empty() { - errors.push(ValidationError::ErrorDomainMissingName(module.name.clone())); - return; - } - check_identifier(&domain.name, errors); - if function_names.contains(&domain.name) { - errors.push(ValidationError::NameCollisionWithErrorDomain { - module: module.name.clone(), - name: domain.name.clone(), - }); - } - - let mut by_name: BTreeSet = BTreeSet::new(); - let mut by_code: BTreeMap = BTreeMap::new(); - for c in &domain.codes { - check_identifier(&c.name, errors); - // 0 means success and the whole negative range is reserved for the - // runtime (-1 generic error, -2 panic, -3 marshalling failure, -4 - // foreign callback error, and room to grow), so domain codes must be - // positive. - if c.code <= 0 { - errors.push(ValidationError::InvalidErrorCode { - module: module.name.clone(), - name: c.name.clone(), - }); - } - if !by_name.insert(c.name.clone()) { - errors.push(ValidationError::DuplicateErrorName { - module: module.name.clone(), - name: c.name.clone(), - }); - } - if by_code.insert(c.code, c.name.clone()).is_some() { - errors.push(ValidationError::DuplicateErrorCode { - module: module.name.clone(), - code: c.code, - }); - } - } -} diff --git a/crates/weaveffi-model/src/validate/tests.rs b/crates/weaveffi-model/src/validate/tests.rs index 6f5ab4b2..af754190 100644 --- a/crates/weaveffi-model/src/validate/tests.rs +++ b/crates/weaveffi-model/src/validate/tests.rs @@ -10,10 +10,28 @@ use crate::ir::CURRENT_SCHEMA_VERSION; use crate::parse::parse_api_str; /// Parse a YAML body (without the `version:` line) and validate it. -fn check(body: &str) -> Result { +fn check(body: &str) -> Result { let doc = format!("version: \"{CURRENT_SCHEMA_VERSION}\"\n{body}"); let api = parse_api_str(&doc, "yaml").expect("fixture must parse"); - validate_api(api, None) + validate(&api, &Identity::named("api"), None) +} + +/// Validate a full YAML document with its source attached, returning the +/// underlined text of every diagnostic. +fn spans(doc: &str) -> Vec<(String, String)> { + let api = parse_api_str(doc, "yaml").expect("fixture must parse"); + let err = validate(&api, &Identity::named("api"), Some(("api.yml", doc))).unwrap_err(); + err.diagnostics + .iter() + .map(|d| { + let span = d.span.expect("span located"); + let line = doc[..span.offset()].lines().count(); + ( + variant(&d.error), + format!("{line}:{}", &doc[span.offset()..span.offset() + span.len()]), + ) + }) + .collect() } /// Variant discriminator names of every error the document produced. @@ -90,9 +108,14 @@ modules: params: [{ name: p, type: Point }, { name: raw, type: bytes }, { name: from, type: Store }] - { name: wants_more, params: [{ name: seen, type: u32 }], return: bool } - { name: status, params: [], return: Status } + - { name: label, params: [], return: string, throws: true } + - { name: nearest, params: [], return: "Point?" } + - { name: store, params: [], return: "Store?" } + - { name: payload, params: [], return: bytes } functions: - { name: fetch, params: [{ name: id, type: i64 }], return: "[Point]", async: true, cancellable: true } - { name: subscribe, params: [{ name: l, type: PointListener }, { name: s, type: Store }] } + - { name: maybe_subscribe, params: [{ name: l, type: "PointListener?" }] } modules: - name: inner functions: @@ -131,6 +154,18 @@ modules: functions: - { name: f, params: [] } - { name: f, params: [] } +"#, + ), + ( + "DuplicateFunctionName", + r#" +modules: + - name: a + functions: [{ name: open, params: [] }] + - name: b + modules: + - name: c + functions: [{ name: open, params: [] }] "#, ), ( @@ -159,7 +194,7 @@ modules: "#, ), ( - "DuplicateErrorName", + "DuplicateErrorCodeName", r#" modules: - name: m @@ -209,6 +244,16 @@ modules: - name: m functions: [{ name: E, params: [] }] errors: { name: E, codes: [{ name: A, code: 1, message: a }] } +"#, + ), + ( + "NameCollisionWithErrorDomain", + r#" +modules: + - name: m + errors: { name: E, codes: [{ name: A, code: 1, message: a }] } + - name: n + functions: [{ name: E, params: [] }] "#, ), ( @@ -267,14 +312,22 @@ modules: "#, ), ( - "UnknownTypeRef", + "QualifiedTypeRef", r#" modules: - name: m structs: [{ name: P, fields: [{ name: v, type: i32 }] }] - functions: [{ name: f, params: [{ name: p, type: nowhere.P }] }] + functions: [{ name: f, params: [{ name: p, type: m.P }] }] "#, ), + ( + "UnsupportedPrimitive", + "modules: [{ name: m, functions: [{ name: f, params: [{ name: n, type: usize }] }] }]", + ), + ( + "UnsupportedPrimitive", + "modules: [{ name: m, structs: [{ name: S, fields: [{ name: c, type: '[char]' }] }] }]", + ), ( "SymbolCollision", r#" @@ -342,7 +395,7 @@ modules: "#, ), ( - "DuplicateStructName", + "DuplicateTypeName", r#" modules: - name: m @@ -365,7 +418,7 @@ modules: "modules: [{ name: m, structs: [{ name: S, fields: [] }] }]", ), ( - "DuplicateEnumName", + "DuplicateTypeName", r#" modules: - name: m @@ -408,7 +461,7 @@ modules: "#, ), ( - "DuplicateInterfaceName", + "DuplicateTypeName", r#" modules: - name: m @@ -503,7 +556,7 @@ modules: "modules: [{ name: m, functions: [{ name: f, params: [], return: '{bytes:i32}' }] }]", ), ( - "DuplicateCallbackInterfaceName", + "DuplicateTypeName", r#" modules: - name: m @@ -530,7 +583,7 @@ modules: "modules: [{ name: m, callback_interfaces: [{ name: L, methods: [{ name: a, params: [], async: true }] }] }]", ), ( - "InvalidCallbackMethod", + "ThrowsWithoutErrorDomain", "modules: [{ name: m, callback_interfaces: [{ name: L, methods: [{ name: a, params: [], throws: true }] }] }]", ), ( @@ -539,15 +592,14 @@ modules: ), ( "InvalidCallbackMethod", - "modules: [{ name: m, callback_interfaces: [{ name: L, methods: [{ name: a, params: [], return: string }] }] }]", + "modules: [{ name: m, callback_interfaces: [{ name: L, methods: [{ name: a, params: [], return: 'iter' }] }] }]", ), ( - "InvalidCallbackMethod", + "CallbackInterfaceInInvalidPosition", r#" modules: - name: m - interfaces: [{ name: I, methods: [{ name: a, params: [] }] }] - callback_interfaces: [{ name: L, methods: [{ name: a, params: [], return: I }] }] + callback_interfaces: [{ name: L, methods: [{ name: a, params: [], return: "L?" }] }] "#, ), ( @@ -565,7 +617,7 @@ modules: modules: - name: m callback_interfaces: [{ name: L, methods: [{ name: a, params: [] }] }] - functions: [{ name: f, params: [{ name: l, type: "L?" }] }] + functions: [{ name: f, params: [{ name: l, type: "[L]" }] }] "#, ), ( @@ -639,7 +691,7 @@ fn unsupported_schema_version_short_circuits() { "yaml", ) .unwrap(); - let err = validate_api(api, None).unwrap_err(); + let err = validate(&api, &Identity::named("api"), None).unwrap_err(); assert_eq!(err.diagnostics.len(), 1); assert!(matches!( &err.first().error, @@ -684,21 +736,149 @@ fn diagnostics_render_every_message_and_related() { assert!(err.help().is_some()); } +#[test] +fn global_duplicates_are_reported_once() { + let got = errors( + r#" +modules: + - name: m + structs: + - { name: S, fields: [{ name: x, type: i32 }] } + - { name: S, fields: [{ name: x, type: i32 }] } + functions: + - { name: f, params: [] } + - { name: f, params: [] } + errors: + name: E + codes: + - { name: A, code: 1, message: a } + - { name: A, code: 2, message: a } +"#, + ); + assert_eq!( + got, + [ + "DuplicateTypeName", + "DuplicateFunctionName", + "DuplicateErrorCodeName" + ] + ); +} + +#[test] +fn symbol_collisions_use_the_real_prefix() { + // `acme_list_*` is reserved by the C buffer helpers only under the + // `acme` prefix the library actually uses. + let doc = format!( + "version: \"{CURRENT_SCHEMA_VERSION}\"\nmodules: [{{ name: list, functions: [{{ name: names }}] }}]" + ); + let api = parse_api_str(&doc, "yaml").unwrap(); + let err = validate(&api, &Identity::named("acme"), None).unwrap_err(); + assert!(err.to_string().contains("'acme_list_names'"), "{err}"); +} + +#[test] +fn buffer_header_structs_and_codecs_are_reserved_symbols() { + // The C buffer header declares a struct for every record, rich enum, + // and error code with fields, each with `_write`, `_read`, `_decode`, + // and `_free`; a function spelled like any of them would redefine it. + let cases = [ + ( + "errors: { name: E, codes: [{ name: Busy, code: 1, message: b, fields: [{ name: n, type: i32 }] }] }\n functions: [{ name: E_Busy_payload }]", + "'kv_m_E_Busy_payload'", + "the payload struct of error code 'm.E.Busy'", + ), + ( + "errors: { name: E, codes: [{ name: Busy, code: 1, message: b, fields: [{ name: n, type: i32 }] }] }\n functions: [{ name: E_Busy_payload_read }]", + "'kv_m_E_Busy_payload_read'", + "codec 'kv_m_E_Busy_payload_read'", + ), + ( + "structs: [{ name: S, fields: [{ name: x, type: i32 }] }]\n functions: [{ name: S_free }]", + "'kv_m_S_free'", + "of record 'm.S'", + ), + ( + "enums: [{ name: Shape, variants: [{ name: Dot, value: 0, fields: [{ name: r, type: f64 }] }] }]\n functions: [{ name: Shape_decode }]", + "'kv_m_Shape_decode'", + "of enum 'm.Shape'", + ), + ]; + for (decls, symbol, origin) in cases { + let doc = + format!("version: \"{CURRENT_SCHEMA_VERSION}\"\nmodules:\n - name: m\n {decls}\n"); + let api = parse_api_str(&doc, "yaml").unwrap(); + let err = validate(&api, &Identity::named("kv"), None).unwrap_err(); + let text = err.to_string(); + assert!(text.contains(symbol) && text.contains(origin), "{text}"); + } + // An error code without fields has no payload struct. + let doc = format!( + "version: \"{CURRENT_SCHEMA_VERSION}\"\nmodules:\n - name: m\n errors: {{ name: E, codes: [{{ name: Busy, code: 1, message: b }}] }}\n functions: [{{ name: E_Busy_payload }}]\n" + ); + let api = parse_api_str(&doc, "yaml").unwrap(); + assert!(validate(&api, &Identity::named("kv"), None).is_ok()); +} + #[test] fn source_spans_underline_the_offending_identifier() { - let src = "version: \"0.10.0\"\nmodules:\n - name: \"dup\"\n - name: \"dup\"\n"; - let api = parse_api_str(src, "yaml").unwrap(); - let err = validate_api(api, Some(("api.yml", src))).unwrap_err(); + let src = format!( + "version: \"{CURRENT_SCHEMA_VERSION}\"\nmodules:\n - name: \"dup\"\n - name: \"dup\"\n" + ); + let api = parse_api_str(&src, "yaml").unwrap(); + let err = validate(&api, &Identity::named("api"), Some(("api.yml", &src))).unwrap_err(); let d = err.first(); let span = d.span.expect("span located"); + assert_eq!(span.offset(), src.rfind("\"dup\"").unwrap()); assert_eq!(&src[span.offset()..span.offset() + span.len()], "\"dup\""); assert!(d.labels().is_some()); assert!(d.source_code().is_some()); } #[test] -fn resolved_api_qualifies_cross_module_types() { - let api = check( +fn spans_are_scoped_to_the_enclosing_declaration() { + let doc = format!( + r#"version: "{CURRENT_SCHEMA_VERSION}" +modules: + - name: a + structs: + - name: x + fields: [{{ name: Nope, type: i32 }}] + functions: + - name: f + params: [{{ name: x, type: i32 }}] + - name: g + params: [{{ name: x, type: i32 }}, {{ name: x, type: Nope }}] + - name: b + functions: + - name: h + params: [{{ name: s, type: a.x }}] + return: usize + - name: c + interfaces: + - name: Store + methods: + - {{ name: h }} + - {{ name: put, cancellable: true }} + - name: c +"# + ); + assert_eq!( + spans(&doc), + [ + ("DuplicateParamName".to_string(), "11:x".to_string()), + ("UnknownTypeRef".to_string(), "11:Nope".to_string()), + ("QualifiedTypeRef".to_string(), "15:a.x".to_string()), + ("UnsupportedPrimitive".to_string(), "16:usize".to_string()), + ("CancellableNotAsync".to_string(), "22:put".to_string()), + ("DuplicateModuleName".to_string(), "23:c".to_string()), + ] + ); +} + +#[test] +fn model_resolves_bare_names() { + let model = check( r#" modules: - name: shared @@ -714,35 +894,20 @@ modules: "#, ) .unwrap(); - use crate::ir::TypeRef; - use crate::model::Ty; - let f = &api.modules[1].functions[0]; - assert_eq!( - api.resolve(&f.params[0].ty), - Ty::Interface("shared.Store".into()) - ); - assert_eq!( - api.resolve(&f.params[1].ty), - Ty::Enum("shared.Status".into()) - ); - assert_eq!( - api.resolve(&f.params[2].ty), - Ty::CallbackInterface("shared.Watcher".into()) - ); - assert_eq!( - api.resolve(f.returns.as_ref().unwrap()), - Ty::List(Box::new(Ty::Record("shared.Point".into()))) - ); - assert_eq!( - api.resolve(&TypeRef::Named("Point".into())), - Ty::Record("shared.Point".into()) - ); + use crate::ty::Ty; + let f = &model.modules[1].functions[0]; + assert_eq!(f.params[0].ty, Ty::Interface("Store".into())); + assert_eq!(f.params[1].ty, Ty::Enum("Status".into())); + assert_eq!(f.params[2].ty, Ty::CallbackInterface("Watcher".into())); + assert_eq!(f.ret, Some(Ty::List(Box::new(Ty::Record("Point".into()))))); + assert_eq!(model.owner("Point").dot_path, "shared"); + assert_eq!(model.prefix(), "api"); + assert_eq!(model.version, CURRENT_SCHEMA_VERSION); } #[test] fn warnings_are_advisory() { - let api = check( - r#" + let body = r#" modules: - name: m enums: @@ -757,10 +922,10 @@ modules: - name: documented doc: Has a module doc. functions: [{ name: f, params: [] }] -"#, - ) - .unwrap(); - let warnings = collect_warnings(&api); +"#; + check(body).unwrap(); + let doc = format!("version: \"{CURRENT_SCHEMA_VERSION}\"\n{body}"); + let warnings = collect_warnings(&parse_api_str(&doc, "yaml").unwrap()); let kinds: Vec = warnings .iter() .map(|w| format!("{w:?}").split(' ').next().unwrap().to_string()) diff --git a/crates/weaveffi/Cargo.toml b/crates/weaveffi/Cargo.toml index 40c3ef6e..7ce16461 100644 --- a/crates/weaveffi/Cargo.toml +++ b/crates/weaveffi/Cargo.toml @@ -11,23 +11,26 @@ keywords = ["ffi", "abi", "bindings", "codegen", "proc-macro"] categories = ["development-tools::ffi"] [dependencies] -weaveffi-abi = { workspace = true } weaveffi-macros = { workspace = true } +tokio = { version = "1", default-features = false, features = ["rt-multi-thread"], optional = true } [features] -# Count live objects, callbacks, iterators, cancel tokens, and returned -# allocations for `{prefix}_debug_live`. Every sample enables it. -leak-check = ["weaveffi-abi/leak-check"] +# Count live objects, callbacks, iterators, cancel tokens, and returned byte +# allocations so `{prefix}_debug_live` can report leaks. Off by default +# because every counted operation pays for an atomic update. Every sample +# enables it. +leak-check = [] +# Run exported `async fn`s on Tokio: the current runtime when a launcher is +# called from inside one, otherwise a multi-thread runtime created on first +# use. `set_spawner` still overrides it. +tokio = ["dep:tokio"] -# Dev-dependencies are path-only (no version) so `cargo publish` strips them: -# a versioned dev-dependency must already be on crates.io when this crate is -# published, which would force `weaveffi-gen` to publish first. [dev-dependencies] -# Only for the ABI-revision lockstep test; the facade never links the -# generators at runtime. -weaveffi-gen = { path = "../weaveffi-gen" } -# The leak-counter test needs counting on. -weaveffi-abi = { path = "../weaveffi-abi", features = ["leak-check"] } +# The leak-counter test needs counting on. A path-only self dependency turns +# the feature on for this crate's tests, and `cargo publish` strips it. +weaveffi = { path = ".", features = ["leak-check"] } +proptest = "1.11.0" +tokio = { version = "1", features = ["rt-multi-thread", "time", "sync", "macros"] } [lints] workspace = true diff --git a/crates/weaveffi-abi/src/buffer.rs b/crates/weaveffi/src/abi/buffer.rs similarity index 81% rename from crates/weaveffi-abi/src/buffer.rs rename to crates/weaveffi/src/abi/buffer.rs index 9e05ebca..10d5f83a 100644 --- a/crates/weaveffi-abi/src/buffer.rs +++ b/crates/weaveffi/src/abi/buffer.rs @@ -39,10 +39,18 @@ //! no per-shape special cases. Iterators and callback interfaces never appear //! inside a buffer; validation rejects them in buffered positions. //! -//! An interface object is encoded as a token (see [`object`](crate::object)) +//! An interface object is encoded as a token (see [`object`](crate::abi::object)) //! that carries one strong reference: the writer clones the object before //! encoding and the reader adopts the reference. Because adopting is a -//! side effect, a buffer holding object tokens must be decoded exactly once. +//! side effect, a buffer holding object tokens must be decoded exactly once, +//! and only from a buffer whose tokens are genuine. That's why +//! [`BufferValue::read_value`] and [`decode_value`] are `unsafe`: no safe +//! function can turn an arbitrary `u64` into an object reference. A reader +//! built with [`BufferReader::token_free`] refuses every token instead, which +//! makes decoding through it sound for any input. +//! +//! A map's encoding never repeats a key; decoding one that does fails, so a +//! value can't silently disappear between the two sides. //! //! Encoded lengths and counts are `u32`, capping any single string, byte //! buffer, or collection at `u32::MAX` entries; [`BufferWriter`] panics past @@ -80,7 +88,7 @@ impl std::error::Error for BufferDecodeError {} /// The `#[weaveffi::module]` expansion writes record fields, enum payloads, /// collection elements, and error payloads through one of these, then hands /// the finished bytes across the ABI (via -/// [`lower_bytes`](crate::lower_bytes) for returns, or borrowed directly for +/// [`lower_bytes`](crate::abi::lower_bytes) for returns, or borrowed directly for /// callback arguments). #[derive(Debug, Default)] pub struct BufferWriter { @@ -217,13 +225,40 @@ impl BufferWriter { pub struct BufferReader<'a> { data: &'a [u8], pos: usize, + tokens: bool, } impl<'a> BufferReader<'a> { - /// Wrap an encoded buffer for reading. + /// Wrap an encoded buffer for reading. Object tokens in it are adopted + /// by [`BufferValue::read_value`], whose contract covers them. #[must_use] pub fn new(data: &'a [u8]) -> Self { - Self { data, pos: 0 } + Self { + data, + pos: 0, + tokens: true, + } + } + + /// Wrap an encoded buffer whose object tokens must not be adopted: + /// reading an interface value fails instead. Decoding through such a + /// reader is sound for any input, which is how the runtime reads a + /// callback's error payload (see + /// [`ForeignError::domain`](crate::abi::ForeignError::domain)). + #[must_use] + pub fn token_free(data: &'a [u8]) -> Self { + Self { + data, + pos: 0, + tokens: false, + } + } + + /// Whether this reader may adopt object tokens (it wasn't built with + /// [`token_free`](Self::token_free)). + #[must_use] + pub fn adopts_tokens(&self) -> bool { + self.tokens } /// The number of bytes not yet consumed. @@ -500,7 +535,16 @@ pub trait BufferValue: Sized { /// /// Returns an error when the buffer is exhausted or holds invalid data /// for this type. - fn read_value(r: &mut BufferReader<'_>) -> Result; + /// + /// # Safety + /// + /// Unless `r` was built with [`BufferReader::token_free`], every object + /// token the value contains must carry one strong reference to a live + /// object of the token's type that hasn't been adopted yet; reading + /// adopts it. Generated thunks satisfy this because the ABI requires + /// every token a consumer writes to come from a fresh `_clone`, and they + /// decode each buffer exactly once. + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result; } /// Marks a type that crosses the C ABI as a value buffer: a @@ -513,7 +557,7 @@ pub trait BufferValue: Sized { /// would cross differently in the generated header, so it must fail to /// compile instead of silently mismatching. #[diagnostic::on_unimplemented( - message = "`{Self}` is declared in a different `#[weaveffi::module]` tree and isn't a record or rich enum", + message = "`{Self}` is not a WeaveFFI record or rich enum; declare C-style enums and interfaces in the module tree that uses them", label = "declared outside this module tree", note = "a `#[weaveffi::module]` only sees the declarations inside its own tree, so it can pass a type from another tree only as a value buffer (a record or rich enum)", note = "nest the modules under one `#[weaveffi::module]` root (as inner `mod`s) so the macro can see the declaration" @@ -530,7 +574,7 @@ macro_rules! scalar_buffer_value { fn write_value(&self, w: &mut BufferWriter) { w.$write(*self); } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { r.$read() } } @@ -553,7 +597,7 @@ macro_rules! fixed_width_buffer_value { fn write_value(&self, w: &mut BufferWriter) { w.$write(*self); } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { r.$read() } } @@ -585,7 +629,7 @@ impl BufferValue for String { fn write_value(&self, w: &mut BufferWriter) { w.write_string(self); } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { r.read_string() } } @@ -594,20 +638,25 @@ impl BufferValue for String { /// reference. Writing clones the `Arc`; reading adopts the reference, so a /// buffer holding tokens must be decoded exactly once (the generated thunks /// guarantee this). A zero token is a contract violation and decodes as an -/// error. +/// error, and so does any token read through a +/// [`token_free`](BufferReader::token_free) reader. impl BufferValue for Arc { fn encoded_len(&self) -> usize { 8 } fn write_value(&self, w: &mut BufferWriter) { - w.write_u64(crate::object::object_to_token(self)); + w.write_u64(crate::abi::object::object_to_token(self)); } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { + if !r.tokens { + return Err(BufferDecodeError { + context: "object token in a buffer that may not carry objects", + }); + } let token = r.read_u64()?; - // SAFETY: by the ABI contract every non-zero token in a buffer was - // produced by `object_to_token` (or a consumer `_clone`) for this - // type and has not been adopted yet. - unsafe { crate::object::object_from_token(token) }.ok_or(BufferDecodeError { + // SAFETY: the caller guarantees every token in an adopting reader + // carries one unadopted reference to a live `T`. + unsafe { crate::abi::object::object_from_token(token) }.ok_or(BufferDecodeError { context: "null object token", }) } @@ -626,9 +675,10 @@ impl BufferValue for Option { None => w.write_option_flag(false), } } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { if r.read_option_flag()? { - Ok(Some(T::read_value(r)?)) + // SAFETY: forwarded from the caller. + Ok(Some(unsafe { T::read_value(r) }?)) } else { Ok(None) } @@ -654,7 +704,7 @@ impl BufferValue for Vec { } } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { let count = r.read_count()?; if let Some(proof) = T::FIXED_WIDTH { let size = count @@ -667,7 +717,8 @@ impl BufferValue for Vec { } let mut out = Vec::with_capacity(count.min(r.remaining())); for _ in 0..count { - out.push(T::read_value(r)?); + // SAFETY: forwarded from the caller. + out.push(unsafe { T::read_value(r) }?); } Ok(out) } @@ -699,6 +750,11 @@ fn fixed_width_from_bytes( out } +/// The error a map encoding that repeats a key decodes to. +const DUPLICATE_KEY: BufferDecodeError = BufferDecodeError { + context: "duplicate map key", +}; + impl BufferValue for std::collections::BTreeMap { fn encoded_len(&self) -> usize { 4 + self @@ -713,13 +769,15 @@ impl BufferValue for std::collections::BTr v.write_value(w); } } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { let len = r.read_count()?; let mut out = Self::new(); for _ in 0..len { - let k = K::read_value(r)?; - let v = V::read_value(r)?; - out.insert(k, v); + // SAFETY: forwarded from the caller. + let (k, v) = unsafe { (K::read_value(r)?, V::read_value(r)?) }; + if out.insert(k, v).is_some() { + return Err(DUPLICATE_KEY); + } } Ok(out) } @@ -741,13 +799,15 @@ impl BufferValue v.write_value(w); } } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(r: &mut BufferReader<'_>) -> Result { let len = r.read_count()?; let mut out = Self::with_capacity(len.min(r.remaining())); for _ in 0..len { - let k = K::read_value(r)?; - let v = V::read_value(r)?; - out.insert(k, v); + // SAFETY: forwarded from the caller. + let (k, v) = unsafe { (K::read_value(r)?, V::read_value(r)?) }; + if out.insert(k, v).is_some() { + return Err(DUPLICATE_KEY); + } } Ok(out) } @@ -769,9 +829,16 @@ pub fn encode_value(value: &T) -> Vec { /// /// Returns an error when the buffer is malformed for `T` or holds trailing /// bytes. -pub fn decode_value(data: &[u8]) -> Result { +/// +/// # Safety +/// +/// Same contract as [`BufferValue::read_value`]: every object token in +/// `data` must carry an unadopted reference to a live object of its type. +/// A buffer holding object tokens may be decoded only once. +pub unsafe fn decode_value(data: &[u8]) -> Result { let mut r = BufferReader::new(data); - let value = T::read_value(&mut r)?; + // SAFETY: forwarded from the caller. + let value = unsafe { T::read_value(&mut r) }?; r.expect_end()?; Ok(value) } @@ -781,9 +848,16 @@ mod tests { use super::*; use std::collections::BTreeMap; + /// Decode a buffer these tests encoded themselves. + fn decode(data: &[u8]) -> Result { + // SAFETY: every token in these buffers was written by `encode_value` + // and is decoded once. + unsafe { decode_value(data) } + } + fn roundtrip(value: T) { let bytes = encode_value(&value); - let back: T = decode_value(&bytes).unwrap(); + let back: T = decode(&bytes).unwrap(); assert_eq!(back, value); } @@ -836,14 +910,41 @@ mod tests { let obj = Arc::new(String::from("shared")); let bytes = encode_value(&vec![Some(Arc::clone(&obj)), None, Some(Arc::clone(&obj))]); assert_eq!(Arc::strong_count(&obj), 3); - let back: Vec>> = decode_value(&bytes).unwrap(); + let back: Vec>> = decode(&bytes).unwrap(); assert_eq!(Arc::strong_count(&obj), 3); assert!(Arc::ptr_eq(back[0].as_ref().unwrap(), &obj)); assert!(back[1].is_none()); drop(back); assert_eq!(Arc::strong_count(&obj), 1); let zero = [0u8; 8]; - assert!(decode_value::>(&zero).is_err()); + assert!(decode::>(&zero).is_err()); + } + + #[test] + fn duplicate_map_keys_are_rejected() { + let mut w = BufferWriter::new(); + w.write_len(2); + for v in [1i32, 2] { + w.write_string("k"); + w.write_i32(v); + } + let bytes = w.finish(); + let err = decode::>(&bytes).unwrap_err(); + assert_eq!(err.context, "duplicate map key"); + assert!(decode::>(&bytes).is_err()); + } + + #[test] + fn token_free_readers_refuse_objects() { + let obj = Arc::new(5u8); + let bytes = encode_value(&obj); + let mut r = BufferReader::token_free(&bytes); + assert!(!r.adopts_tokens()); + // SAFETY: a token-free reader never adopts. + assert!(unsafe { Arc::::read_value(&mut r) }.is_err()); + // The token still carries its reference; adopt it to balance. + let back: Arc = decode(&bytes).unwrap(); + assert!(Arc::ptr_eq(&back, &obj)); } #[test] @@ -856,7 +957,7 @@ mod tests { #[test] fn truncated_buffer_is_rejected() { let bytes = encode_value(&"hello".to_string()); - let err = decode_value::(&bytes[..bytes.len() - 1]).unwrap_err(); + let err = decode::(&bytes[..bytes.len() - 1]).unwrap_err(); assert!(err.to_string().contains("malformed")); } @@ -864,7 +965,7 @@ mod tests { fn trailing_bytes_are_rejected() { let mut bytes = encode_value(&7i32); bytes.push(0); - assert!(decode_value::(&bytes).is_err()); + assert!(decode::(&bytes).is_err()); } #[test] @@ -872,19 +973,19 @@ mod tests { // A length claiming more elements than bytes remain must fail fast // instead of attempting a huge allocation. let bytes = [0xFF, 0xFF, 0xFF, 0xFF]; - assert!(decode_value::>(&bytes).is_err()); + assert!(decode::>(&bytes).is_err()); } #[test] fn invalid_bool_and_flag_bytes_are_rejected() { - assert!(decode_value::(&[2]).is_err()); - assert!(decode_value::>(&[9]).is_err()); + assert!(decode::(&[2]).is_err()); + assert!(decode::>(&[9]).is_err()); } #[test] fn invalid_utf8_is_rejected() { let bytes = [2, 0, 0, 0, 0xFF, 0xFE]; - assert!(decode_value::(&bytes).is_err()); + assert!(decode::(&bytes).is_err()); } #[derive(Debug, PartialEq)] @@ -892,7 +993,7 @@ mod tests { impl BufferValue for NoFields { fn write_value(&self, _w: &mut BufferWriter) {} - fn read_value(_r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value(_r: &mut BufferReader<'_>) -> Result { Ok(NoFields) } } @@ -938,9 +1039,9 @@ mod tests { fn truncated_fixed_width_list_is_rejected() { let mut bytes = encode_value(&vec![7u32, 8]); bytes.pop(); - assert!(decode_value::>(&bytes).is_err()); + assert!(decode::>(&bytes).is_err()); let huge = [0xFF, 0xFF, 0xFF, 0xFF]; - assert!(decode_value::>(&huge).is_err()); - assert!(decode_value::>(&huge).is_err()); + assert!(decode::>(&huge).is_err()); + assert!(decode::>(&huge).is_err()); } } diff --git a/crates/weaveffi/src/abi/callback.rs b/crates/weaveffi/src/abi/callback.rs new file mode 100644 index 00000000..cf5fffc3 --- /dev/null +++ b/crates/weaveffi/src/abi/callback.rs @@ -0,0 +1,585 @@ +//! Consumer-implemented callback interfaces: the producer half of the ABI's +//! vtable contract. +//! +//! A callback-interface parameter arrives as two slots, a `void* ctx` the +//! consumer owns and a pointer to a process-wide static vtable. Every vtable +//! starts with the same header ([`VtableHeader`]: its `size`, reserved +//! `flags`, and the `free(ctx)` release hook), followed by one `extern "C"` +//! function pointer per method. The `#[weaveffi::module]` expansion wraps +//! the pair in a [`ForeignCallback`], implements the producer's trait on top +//! of it (each method lowers its arguments, calls the vtable entry, adopts +//! the return, then checks `out_err`), and hands the producer an +//! `Arc`. When the last `Arc` drops, [`ForeignCallback`] calls +//! `free(ctx)` exactly once. +//! +//! The producer rejects a vtable whose `size` is smaller than the vtable it +//! was built with, so a consumer generated from an older contract can't make +//! the producer call through a slot that doesn't exist. A larger `size` is +//! accepted, which lets a newer consumer pass methods this producer doesn't +//! know yet. +//! +//! A callback method returns `Result`. A failure the +//! consumer reports through `out_err` becomes the `Err`: a method that +//! `throws` keeps a declared code of its module's error domain (decode it +//! with [`ForeignError::domain`]), and every other failure has the code +//! [`FOREIGN_ERROR_CODE`]. + +use std::ffi::c_void; +use std::sync::Arc; + +use crate::abi::buffer::BufferReader; +use crate::abi::error::{ErrorDomain, FfiError, FOREIGN_ERROR_CODE, MARSHAL_ERROR_CODE}; + +/// The fixed header every callback-interface vtable starts with, in this +/// order and at these offsets (`{prefix}_..._vtable` in C). +#[repr(C)] +#[derive(Debug, Clone, Copy)] +pub struct VtableHeader { + /// `sizeof` the whole vtable as the consumer compiled it. + pub size: u32, + /// Reserved; consumers set it to `0`. + pub flags: u32, + /// The consumer's release hook, called exactly once (from any producer + /// thread) when the producer drops its last reference to the callback. + pub free: unsafe extern "C" fn(ctx: *mut c_void), +} + +/// Implemented by every generated `#[repr(C)]` vtable struct, whose first +/// field is its [`VtableHeader`]. +/// +/// # Safety +/// +/// The implementing type must be `#[repr(C)]` and start with a +/// [`VtableHeader`], so a pointer to it is also a valid pointer to the +/// header; [`ForeignCallback`] reads the header before trusting the rest. +pub unsafe trait Vtable: 'static {} + +/// Ties a callback interface's `dyn Trait` to its generated vtable and +/// foreign wrapper. +/// +/// The `#[weaveffi::module]` expansion implements this for `dyn Trait` of +/// every `#[weaveffi::callback_interface]`, which lets a thunk that only knows +/// the producer's written type (`Arc`) name the vtable slot type +/// (`::Vtable`) and lift the pair into a +/// shared trait object with [`lift_callback`]. +pub trait CallbackInterface { + /// The generated `#[repr(C)]` vtable struct for this interface. + type Vtable: Vtable; + + /// Wrap a lifted foreign callback as a shared trait object. + fn from_foreign(cb: ForeignCallback) -> Arc; +} + +/// Lift a callback-interface parameter's `(ctx, vtable)` slots into the +/// `Arc` the producer's function takes. `name` is the parameter's +/// name, for the error message. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] error for a null vtable or one smaller +/// than the producer's. +/// +/// # Safety +/// +/// Same contract as [`ForeignCallback::from_raw`]. +pub unsafe fn lift_callback( + ctx: *mut c_void, + vtable: *const C::Vtable, + name: &str, +) -> Result, FfiError> { + // SAFETY: forwarded from the caller. + match unsafe { lift_callback_opt::(ctx, vtable, name) }? { + Some(cb) => Ok(cb), + None => Err(FfiError::new( + MARSHAL_ERROR_CODE, + &format!("{name}: null callback vtable"), + )), + } +} + +/// Lift an optional callback-interface parameter (`Cb?`): a null vtable is +/// `None`. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] error for a vtable smaller than the +/// producer's. +/// +/// # Safety +/// +/// Same contract as [`ForeignCallback::from_raw`]. +pub unsafe fn lift_callback_opt( + ctx: *mut c_void, + vtable: *const C::Vtable, + name: &str, +) -> Result>, FfiError> { + // SAFETY: forwarded from the caller. + match unsafe { ForeignCallback::from_raw(ctx, vtable) } { + Ok(Some(cb)) => Ok(Some(C::from_foreign(cb))), + Ok(None) => Ok(None), + Err(VtableTooSmall { size, expected }) => Err(FfiError::new( + MARSHAL_ERROR_CODE, + &format!( + "{name}: the callback vtable is {size} bytes but this library expects at least \ + {expected}; regenerate the bindings" + ), + )), + } +} + +/// A vtable's `size` is smaller than the vtable the producer was built with. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct VtableTooSmall { + /// The `size` the consumer wrote. + pub size: u32, + /// The size of the producer's vtable struct. + pub expected: usize, +} + +/// A consumer-implemented callback interface held by the producer. +/// +/// Holds the consumer's `ctx` and a pointer to its static vtable `V`. The +/// generated `impl Trait for ForeignX` calls +/// [`vtable`](Self::vtable)`().method(ctx, ...)` per method; dropping the +/// last owner calls `free(ctx)`. +pub struct ForeignCallback { + ctx: *mut c_void, + vtable: *const V, +} + +// SAFETY: the ABI contract obliges the consumer to make every vtable entry +// (including `free`) callable from any thread, and `ctx` is only ever handed +// back to those entries. The producer never dereferences `ctx` itself. +unsafe impl Send for ForeignCallback {} +// SAFETY: see the `Send` impl; the vtable is immutable static consumer data. +unsafe impl Sync for ForeignCallback {} + +impl ForeignCallback { + /// Adopt a `(ctx, vtable)` pair lifted from a callback-interface + /// parameter's slots. Returns `Ok(None)` when the vtable pointer is null. + /// + /// # Errors + /// + /// Returns [`VtableTooSmall`] when the vtable's `size` is smaller than + /// `V`. The context is released through the header's `free` first (when + /// the header itself is complete), so a rejected callback doesn't leak. + /// + /// # Safety + /// + /// `vtable` must be null or point to a vtable whose header is + /// initialized and whose first `size` bytes stay valid and unmodified + /// for the life of the returned value (consumers use a process-wide + /// static). `ctx` must remain valid until the vtable's `free` entry is + /// called with it. + pub unsafe fn from_raw( + ctx: *mut c_void, + vtable: *const V, + ) -> Result, VtableTooSmall> { + if vtable.is_null() { + return Ok(None); + } + // SAFETY: `V: Vtable` starts with the header, which the caller + // guarantees is initialized. + let header = unsafe { &*vtable.cast::() }; + let expected = std::mem::size_of::(); + if (header.size as usize) < expected { + if header.size as usize >= std::mem::size_of::() { + // SAFETY: the header is complete, so `free` is the + // consumer's release hook for `ctx`; nothing else will + // release it. + unsafe { (header.free)(ctx) }; + } + return Err(VtableTooSmall { + size: header.size, + expected, + }); + } + crate::abi::leak::track(crate::abi::leak::CALLBACKS, 1); + Ok(Some(Self { ctx, vtable })) + } + + /// The consumer's context pointer, passed as the first argument of every + /// vtable entry. + #[must_use] + pub fn ctx(&self) -> *mut c_void { + self.ctx + } + + /// The consumer's vtable. + #[must_use] + pub fn vtable(&self) -> &V { + // SAFETY: `from_raw` rejected null and short vtables, and its + // contract keeps the vtable alive and immutable for the life of + // `self`. + unsafe { &*self.vtable } + } +} + +impl Drop for ForeignCallback { + fn drop(&mut self) { + // SAFETY: `V` starts with the header (see `Vtable`). + let free = unsafe { &*self.vtable.cast::() }.free; + crate::abi::leak::track(crate::abi::leak::CALLBACKS, -1); + // SAFETY: `ctx` is handed back to the consumer's own release hook + // exactly once, as the ABI contract requires. + unsafe { free(self.ctx) }; + } +} + +/// A failure a consumer's callback-interface implementation reported, or a +/// value it returned that the producer couldn't accept. +/// +/// Every callback trait method returns `Result`. `code` is +/// [`FOREIGN_ERROR_CODE`] for any consumer failure, except that a method +/// declared `throws` keeps a code of its module's error domain, whose fields +/// [`domain`](Self::domain) decodes. A malformed return value (a string that +/// isn't UTF-8, a null object, a bad enum value or buffer) has the code +/// [`MARSHAL_ERROR_CODE`]. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ForeignError { + /// The failure's code (see the type's docs). + pub code: i32, + /// The consumer's message, copied out of `out_err` before it was freed. + pub message: String, + /// The domain code's fields as a value buffer, or empty. + pub payload: Vec, +} + +impl ForeignError { + /// A failure with `code` and `message` and no payload. + #[must_use] + pub fn new(code: i32, message: impl Into) -> Self { + Self { + code, + message: message.into(), + payload: Vec::new(), + } + } + + /// Decode a domain error the consumer reported: `Some` when `code` is + /// one of `E`'s declared codes and the payload holds that code's fields. + /// + /// Object fields can't be decoded here (a callback's error payload may + /// not carry object references), so a code whose fields include an + /// interface yields `None`. + #[must_use] + pub fn domain(&self) -> Option { + if self.code <= 0 { + return None; + } + let mut r = BufferReader::token_free(&self.payload); + // SAFETY: a token-free reader never adopts an object token. + let decoded = unsafe { E::read_code(self.code, &mut r) }.ok().flatten()?; + r.expect_end().ok().map(|()| decoded) + } + + fn marshal(message: &str) -> Self { + Self::new(MARSHAL_ERROR_CODE, message) + } +} + +impl std::fmt::Display for ForeignError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.message) + } +} + +impl std::error::Error for ForeignError {} + +/// A `ForeignError` is reported with its own code, message, and payload, so +/// a producer function whose error type is `ForeignError` can propagate a +/// callback failure with `?`. +impl crate::abi::ErrorReport for ForeignError { + fn code(&self) -> i32 { + self.code + } + fn message(&self) -> String { + self.message.clone() + } + fn payload(&self) -> Vec { + self.payload.clone() + } +} + +fn failure(err: &FfiError, code: i32) -> ForeignError { + // SAFETY: the consumer fills `out_err` only through `{prefix}_error_set`, + // so a non-null message is a NUL-terminated string this runtime owns. + let message = unsafe { err.message_str() } + .unwrap_or("callback interface implementation failed") + .to_string(); + ForeignError { + code, + message, + payload: if code > 0 { + err.payload().to_vec() + } else { + Vec::new() + }, + } +} + +/// Read the `out_err` slot a vtable entry of a method that doesn't `throw` +/// wrote: `Ok(())` when the consumer succeeded, otherwise its failure with +/// the code [`FOREIGN_ERROR_CODE`] (whatever code it wrote). +/// +/// # Errors +/// +/// Returns the consumer's failure when `err.code` isn't `0`. +pub fn callback_status(err: &FfiError) -> Result<(), ForeignError> { + if err.code == 0 { + Ok(()) + } else { + Err(failure(err, FOREIGN_ERROR_CODE)) + } +} + +/// Read the `out_err` slot a vtable entry of a `throws` method wrote: a +/// positive code that `E` declares (with a payload that decodes) is kept, +/// and every other failure gets the code [`FOREIGN_ERROR_CODE`]. +/// +/// # Errors +/// +/// Returns the consumer's failure when `err.code` isn't `0`. +pub fn callback_status_in(err: &FfiError) -> Result<(), ForeignError> { + if err.code == 0 { + return Ok(()); + } + let kept = failure(err, err.code); + if kept.domain::().is_some() { + Err(kept) + } else { + Err(failure(err, FOREIGN_ERROR_CODE)) + } +} + +/// Lift a C-style enum a vtable entry returned. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] failure for a value `E` doesn't declare. +pub fn callback_ret_enum(value: i32) -> Result { + E::from_i32(value) + .ok_or_else(|| ForeignError::marshal("callback interface returned an invalid enum value")) +} + +/// Adopt the strong object reference a vtable entry returned. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] failure for a null pointer. +/// +/// # Safety +/// +/// `ptr` must be null or carry one strong reference to a live `T` (a fresh +/// `_clone`), which this adopts. +pub unsafe fn callback_ret_object(ptr: *mut T) -> Result, ForeignError> { + // SAFETY: forwarded from the caller. + unsafe { callback_ret_object_opt(ptr) } + .ok_or_else(|| ForeignError::marshal("callback interface returned a null object")) +} + +/// Adopt the optional object reference a vtable entry returned (`I?`). +/// +/// # Safety +/// +/// Same contract as [`callback_ret_object`]. +#[must_use] +pub unsafe fn callback_ret_object_opt(ptr: *mut T) -> Option> { + // SAFETY: an object pointer is its token widened; forwarded from the + // caller. + unsafe { crate::abi::object::object_from_token(ptr as usize as u64) } +} + +/// Adopt the byte run a vtable entry wrote to its `out_ptr`/`out_len` slots. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] failure for a null pointer with a +/// non-zero length. +/// +/// # Safety +/// +/// `ptr` must be null or a run of `len` bytes from `{prefix}_alloc`, which +/// this adopts and frees. +pub unsafe fn callback_ret_bytes(ptr: *mut u8, len: usize) -> Result, ForeignError> { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::convert::adopt_bytes(ptr, len) } + .ok_or_else(|| ForeignError::marshal("callback interface returned a null run")) +} + +/// Adopt a string a vtable entry wrote to its `out_ptr`/`out_len` slots. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] failure for a null pointer with a +/// non-zero length or bytes that aren't UTF-8. +/// +/// # Safety +/// +/// Same contract as [`callback_ret_bytes`]. +pub unsafe fn callback_ret_string(ptr: *mut u8, len: usize) -> Result { + // SAFETY: forwarded from the caller. + String::from_utf8(unsafe { callback_ret_bytes(ptr, len) }?) + .map_err(|_| ForeignError::marshal("callback interface returned a string that isn't UTF-8")) +} + +/// Adopt and decode a value buffer a vtable entry wrote to its +/// `out_ptr`/`out_len` slots. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] failure for a null pointer with a +/// non-zero length or a malformed buffer. +/// +/// # Safety +/// +/// Same contract as [`callback_ret_bytes`], and every object token in the +/// buffer must carry an unadopted reference (see +/// [`BufferValue::read_value`](crate::abi::BufferValue::read_value)). +pub unsafe fn callback_ret_buffer( + ptr: *mut u8, + len: usize, +) -> Result { + // SAFETY: forwarded from the caller. + let bytes = unsafe { callback_ret_bytes(ptr, len) }?; + // SAFETY: forwarded from the caller; the run is decoded once. + unsafe { crate::abi::decode_value(&bytes) } + .map_err(|e| ForeignError::marshal(&format!("callback interface returned {e}"))) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::atomic::{AtomicUsize, Ordering}; + + #[repr(C)] + struct TestVtable { + header: VtableHeader, + ping: unsafe extern "C" fn(*mut c_void, i32, *mut FfiError) -> i32, + } + + // SAFETY: `#[repr(C)]` and starts with the header. + unsafe impl Vtable for TestVtable {} + + unsafe extern "C" fn ping(_ctx: *mut c_void, x: i32, out_err: *mut FfiError) -> i32 { + if x < 0 { + unsafe { crate::abi::error_set(out_err, 7, "negative") }; + return 0; + } + x * 2 + } + + /// Each test owns its counter: the context points at it, so tests running + /// in parallel never observe each other's releases. + unsafe extern "C" fn free(ctx: *mut c_void) { + unsafe { &*ctx.cast::() }.fetch_add(1, Ordering::SeqCst); + } + + static VTABLE: TestVtable = TestVtable { + header: VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free, + }, + ping, + }; + + fn ctx(freed: &AtomicUsize) -> *mut c_void { + std::ptr::from_ref(freed).cast_mut().cast() + } + + fn call(cb: &ForeignCallback, x: i32) -> Result { + let mut err = FfiError::default(); + let out = unsafe { (cb.vtable().ping)(cb.ctx(), x, &mut err) }; + callback_status(&err).map(|()| out) + } + + #[test] + fn calls_through_and_frees_once() { + let freed = AtomicUsize::new(0); + let cb = Arc::new( + unsafe { ForeignCallback::from_raw(ctx(&freed), &VTABLE) } + .unwrap() + .unwrap(), + ); + assert_eq!(call(&cb, 21), Ok(42)); + let second = Arc::clone(&cb); + drop(cb); + assert_eq!(freed.load(Ordering::SeqCst), 0); + drop(second); + assert_eq!(freed.load(Ordering::SeqCst), 1); + } + + #[test] + fn every_failure_of_a_non_throwing_method_is_foreign() { + let freed = AtomicUsize::new(0); + let cb = unsafe { ForeignCallback::from_raw(ctx(&freed), &VTABLE) } + .unwrap() + .unwrap(); + let err = call(&cb, -1).unwrap_err(); + assert_eq!(err.code, FOREIGN_ERROR_CODE); + assert_eq!(err.message, "negative"); + let err = FfiError::new(MARSHAL_ERROR_CODE, "kept?"); + assert_eq!(callback_status(&err).unwrap_err().code, FOREIGN_ERROR_CODE); + } + + #[test] + fn short_vtables_are_rejected_and_released() { + let freed = AtomicUsize::new(0); + let short = TestVtable { + header: VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free, + }, + ping, + }; + let err = unsafe { ForeignCallback::from_raw(ctx(&freed), &short) } + .err() + .unwrap(); + assert_eq!(err.expected, std::mem::size_of::()); + assert_eq!(freed.load(Ordering::SeqCst), 1); + // A larger vtable (a newer consumer) is fine. + let long = TestVtable { + header: VtableHeader { + size: 1024, + flags: 0, + free, + }, + ping, + }; + let cb = unsafe { ForeignCallback::from_raw(ctx(&freed), &long) }.unwrap(); + drop(cb); + assert_eq!(freed.load(Ordering::SeqCst), 2); + } + + #[test] + fn null_vtable_is_none() { + assert!(matches!( + unsafe { + ForeignCallback::::from_raw(std::ptr::null_mut(), std::ptr::null()) + }, + Ok(None) + )); + } + + #[test] + fn returned_runs_are_adopted() { + let ptr = crate::abi::alloc(5); + unsafe { std::ptr::copy_nonoverlapping(b"hello".as_ptr(), ptr, 5) }; + assert_eq!(unsafe { callback_ret_string(ptr, 5) }.unwrap(), "hello"); + assert_eq!( + unsafe { callback_ret_bytes(std::ptr::null_mut(), 0) }, + Ok(vec![]) + ); + assert_eq!( + unsafe { callback_ret_bytes(std::ptr::null_mut(), 3) } + .unwrap_err() + .code, + MARSHAL_ERROR_CODE + ); + let bad = crate::abi::alloc(1); + unsafe { *bad = 0xff }; + assert!(unsafe { callback_ret_string(bad, 1) }.is_err()); + } +} diff --git a/crates/weaveffi-abi/src/cancel.rs b/crates/weaveffi/src/abi/cancel.rs similarity index 58% rename from crates/weaveffi-abi/src/cancel.rs rename to crates/weaveffi/src/abi/cancel.rs index 4e2f559e..2fcb776e 100644 --- a/crates/weaveffi-abi/src/cancel.rs +++ b/crates/weaveffi/src/abi/cancel.rs @@ -7,14 +7,98 @@ //! at any time afterwards, even before the call completes. Cancelling wakes //! every future registered on the token; the launcher's [`Cancellable`] //! wrapper then drops the producer's future and the call completes with -//! [`CANCELLED_ERROR_CODE`](crate::CANCELLED_ERROR_CODE). +//! [`CANCELLED_ERROR_CODE`](crate::abi::CANCELLED_ERROR_CODE). +//! +//! Each in-flight call registers once, when it's first polled, and keeps its +//! current waker in an [`AtomicWaker`] of its own, so a poll refreshes the +//! waker without taking a lock. +use std::cell::UnsafeCell; use std::future::Future; use std::pin::Pin; -use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; use std::sync::{Arc, Mutex, PoisonError}; use std::task::{Context, Poll, Waker}; +/// A waker slot that one task registers into and any thread wakes, without a +/// lock (the algorithm of `futures`' `AtomicWaker`). +#[derive(Default)] +pub struct AtomicWaker { + state: AtomicUsize, + waker: UnsafeCell>, +} + +const WAITING: usize = 0; +const REGISTERING: usize = 0b01; +const WAKING: usize = 0b10; + +// SAFETY: the `state` protocol gives exactly one thread at a time access to +// `waker` (the registering thread holds REGISTERING, the waking thread holds +// WAKING), and `Waker` itself is `Send + Sync`. +unsafe impl Send for AtomicWaker {} +// SAFETY: see the `Send` impl. +unsafe impl Sync for AtomicWaker {} + +impl std::fmt::Debug for AtomicWaker { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("AtomicWaker").finish_non_exhaustive() + } +} + +impl AtomicWaker { + /// Store `waker` as the one to wake, replacing any earlier one. A wake + /// that races the registration wakes `waker` immediately. + pub fn register(&self, waker: &Waker) { + match self + .state + .compare_exchange(WAITING, REGISTERING, Ordering::Acquire, Ordering::Acquire) + .unwrap_or_else(|actual| actual) + { + WAITING => { + // SAFETY: holding REGISTERING gives exclusive access. + let slot = unsafe { &mut *self.waker.get() }; + match slot { + Some(old) if old.will_wake(waker) => {} + _ => *slot = Some(waker.clone()), + } + if self + .state + .compare_exchange(REGISTERING, WAITING, Ordering::AcqRel, Ordering::Acquire) + .is_err() + { + // A wake arrived while registering (state is now + // REGISTERING | WAKING): deliver it here. + // SAFETY: the waker defers to us while REGISTERING is set. + let woken = unsafe { (*self.waker.get()).take() }; + self.state.swap(WAITING, Ordering::AcqRel); + if let Some(w) = woken { + w.wake(); + } + } + } + // A concurrent wake is in progress: make sure this task polls + // again. + WAKING => waker.wake_by_ref(), + // Registered concurrently from another poll of the same task; + // nothing to do. + _ => {} + } + } + + /// Wake the registered waker, if any. + pub fn wake(&self) { + if let WAITING = self.state.fetch_or(WAKING, Ordering::AcqRel) { + // SAFETY: holding WAKING (without REGISTERING) gives exclusive + // access. + let woken = unsafe { (*self.waker.get()).take() }; + self.state.fetch_and(!WAKING, Ordering::Release); + if let Some(w) = woken { + w.wake(); + } + } + } +} + /// The shared state behind a cancel token (`{prefix}_cancel_token` in C, an /// opaque type). /// @@ -24,15 +108,17 @@ use std::task::{Context, Poll, Waker}; #[derive(Debug, Default)] pub struct FfiCancelToken { cancelled: AtomicBool, - wakers: Mutex>, + /// One slot per in-flight call; touched once when a call starts and once + /// when it ends, never per poll. + registrations: Mutex>>, } impl FfiCancelToken { fn cancel(&self) { self.cancelled.store(true, Ordering::SeqCst); - let wakers = std::mem::take(&mut *self.lock_wakers()); - for (_, waker) in wakers { - waker.wake(); + let registrations = std::mem::take(&mut *self.lock()); + for r in registrations { + r.wake(); } } @@ -40,25 +126,18 @@ impl FfiCancelToken { self.cancelled.load(Ordering::SeqCst) } - fn lock_wakers(&self) -> std::sync::MutexGuard<'_, Vec<(u64, Waker)>> { - self.wakers.lock().unwrap_or_else(PoisonError::into_inner) + fn lock(&self) -> std::sync::MutexGuard<'_, Vec>> { + self.registrations + .lock() + .unwrap_or_else(PoisonError::into_inner) } - /// Store (or refresh) the waker registered under `id`. - fn register(&self, id: u64, waker: &Waker) { - let mut wakers = self.lock_wakers(); - match wakers.iter_mut().find(|(k, _)| *k == id) { - Some((_, w)) => { - if !w.will_wake(waker) { - w.clone_from(waker); - } - } - None => wakers.push((id, waker.clone())), - } + fn register(&self, slot: &Arc) { + self.lock().push(Arc::clone(slot)); } - fn deregister(&self, id: u64) { - self.lock_wakers().retain(|(k, _)| *k != id); + fn deregister(&self, slot: &Arc) { + self.lock().retain(|r| !Arc::ptr_eq(r, slot)); } } @@ -67,7 +146,7 @@ impl FfiCancelToken { /// [`cancel_token_destroy`]. #[must_use] pub fn cancel_token_create() -> *mut FfiCancelToken { - crate::leak::track(crate::leak::TOKENS, 1); + crate::abi::leak::track(crate::abi::leak::TOKENS, 1); Arc::into_raw(Arc::new(FfiCancelToken::default())).cast_mut() } @@ -110,7 +189,7 @@ pub unsafe fn cancel_token_destroy(token: *mut FfiCancelToken) { if token.is_null() { return; } - crate::leak::track(crate::leak::TOKENS, -1); + crate::abi::leak::track(crate::abi::leak::TOKENS, -1); // SAFETY: `token` came from `Arc::into_raw` and its reference is // released exactly once. unsafe { Arc::decrement_strong_count(token.cast_const()) }; @@ -152,7 +231,7 @@ impl CancelToken { Arc::increment_strong_count(raw); Arc::from_raw(raw) }; - crate::leak::track(crate::leak::TOKENS, 1); + crate::abi::leak::track(crate::abi::leak::TOKENS, 1); Self { state: Some(state) } } @@ -168,7 +247,7 @@ impl CancelToken { impl Drop for CancelToken { fn drop(&mut self) { if self.state.is_some() { - crate::leak::track(crate::leak::TOKENS, -1); + crate::abi::leak::track(crate::abi::leak::TOKENS, -1); } } } @@ -176,7 +255,7 @@ impl Drop for CancelToken { impl Clone for CancelToken { fn clone(&self) -> Self { if self.state.is_some() { - crate::leak::track(crate::leak::TOKENS, 1); + crate::abi::leak::track(crate::abi::leak::TOKENS, 1); } Self { state: self.state.clone(), @@ -184,8 +263,6 @@ impl Clone for CancelToken { } } -static NEXT_REGISTRATION: AtomicU64 = AtomicU64::new(1); - /// A future that resolves to `Some(output)` when `inner` completes, or to /// `None` as soon as the token is cancelled, dropping `inner` first. /// @@ -196,7 +273,7 @@ static NEXT_REGISTRATION: AtomicU64 = AtomicU64::new(1); pub struct Cancellable { inner: Option>>, token: CancelToken, - registration: u64, + slot: Option>, } impl Cancellable { @@ -205,7 +282,7 @@ impl Cancellable { Self { inner: Some(Box::pin(fut)), token, - registration: NEXT_REGISTRATION.fetch_add(1, Ordering::Relaxed), + slot: None, } } } @@ -216,10 +293,15 @@ impl Future for Cancellable { fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll { let this = &mut *self; if let Some(state) = this.token.state.as_deref() { - state.register(this.registration, cx.waker()); + let slot = this.slot.get_or_insert_with(|| { + let slot = Arc::new(AtomicWaker::default()); + state.register(&slot); + slot + }); + slot.register(cx.waker()); if state.is_cancelled() { this.inner = None; - state.deregister(this.registration); + state.deregister(slot); return Poll::Ready(None); } } @@ -231,8 +313,8 @@ impl Future for Cancellable { Poll::Pending => Poll::Pending, Poll::Ready(out) => { this.inner = None; - if let Some(state) = this.token.state.as_deref() { - state.deregister(this.registration); + if let (Some(state), Some(slot)) = (this.token.state.as_deref(), &this.slot) { + state.deregister(slot); } Poll::Ready(Some(out)) } @@ -242,8 +324,8 @@ impl Future for Cancellable { impl Drop for Cancellable { fn drop(&mut self) { - if let Some(state) = self.token.state.as_deref() { - state.deregister(self.registration); + if let (Some(state), Some(slot)) = (self.token.state.as_deref(), &self.slot) { + state.deregister(slot); } } } @@ -304,16 +386,58 @@ mod tests { unsafe { cancel_token_cancel(raw as *mut FfiCancelToken) }; unsafe { cancel_token_destroy(raw as *mut FfiCancelToken) }; }); - assert!(crate::block_on(fut).is_none()); + assert!(crate::abi::block_on(fut).is_none()); assert!(dropped.load(Ordering::SeqCst)); canceller.join().unwrap(); } + #[test] + fn atomic_waker_wakes_the_latest_registration() { + struct Count(AtomicUsize); + impl std::task::Wake for Count { + fn wake(self: Arc) { + self.0.fetch_add(1, Ordering::SeqCst); + } + } + let a = Arc::new(Count(AtomicUsize::new(0))); + let b = Arc::new(Count(AtomicUsize::new(0))); + let slot = AtomicWaker::default(); + slot.wake(); + slot.register(&Waker::from(Arc::clone(&a))); + slot.register(&Waker::from(Arc::clone(&b))); + slot.wake(); + slot.wake(); + assert_eq!(a.0.load(Ordering::SeqCst), 0); + assert_eq!(b.0.load(Ordering::SeqCst), 1); + } + + #[test] + fn one_token_cancels_every_call_it_was_passed_to() { + let token = cancel_token_create(); + let calls: Vec<_> = (0..3) + .map(|_| { + Cancellable::new(std::future::pending::<()>(), unsafe { + CancelToken::from_raw(token) + }) + }) + .collect(); + let raw = token as usize; + let canceller = std::thread::spawn(move || { + std::thread::sleep(std::time::Duration::from_millis(20)); + unsafe { cancel_token_cancel(raw as *mut FfiCancelToken) }; + }); + for call in calls { + assert!(crate::abi::block_on(call).is_none()); + } + canceller.join().unwrap(); + unsafe { cancel_token_destroy(token) }; + } + #[test] fn an_uncancelled_future_completes() { let handle = CancelToken::default(); assert_eq!( - crate::block_on(Cancellable::new(async { 3 }, handle)), + crate::abi::block_on(Cancellable::new(async { 3 }, handle)), Some(3) ); } diff --git a/crates/weaveffi/src/abi/contract.rs b/crates/weaveffi/src/abi/contract.rs new file mode 100644 index 00000000..073c611f --- /dev/null +++ b/crates/weaveffi/src/abi/contract.rs @@ -0,0 +1,118 @@ +//! Contract tables: the per-declaration fingerprints a consumer checks when +//! it loads the library. +//! +//! The `#[weaveffi::module]` expansion exports, for each top-level module +//! `m`, `const {prefix}_contract_entry* {prefix}_{m}_contract(size_t* +//! out_len)`, returning a static table of [`ContractEntry`] values sorted by +//! id: one per declaration (function, interface and each of its members, +//! record, enum, callback interface, and error domain) in `m` and its +//! submodules. The expansion computes every entry at compile time and keeps +//! only those whose `#[cfg]` holds, using [`contract_len`] and +//! [`contract_compact`] in constant context, so the table reflects exactly +//! what the build exports. + +/// One declaration's fingerprint (`{prefix}_contract_entry` in C). +#[repr(C)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ContractEntry { + /// FNV-1a 64 of the declaration's dotted path (`kv.Store.put`). + pub id: u64, + /// FNV-1a 64 of the declaration's canonical signature. + pub hash: u64, +} + +impl ContractEntry { + /// An entry with `id` and `hash`. + #[must_use] + pub const fn new(id: u64, hash: u64) -> Self { + Self { id, hash } + } +} + +/// How many of `entries` are enabled (their flag is `true`). +#[must_use] +pub const fn contract_len(entries: &[(bool, ContractEntry); N]) -> usize { + let mut count = 0; + let mut i = 0; + while i < N { + if entries[i].0 { + count += 1; + } + i += 1; + } + count +} + +/// The enabled entries of `entries`, in order. `M` must be +/// [`contract_len`] of `entries`. +/// +/// # Panics +/// +/// Panics (at compile time, in a constant) when `M` isn't the number of +/// enabled entries. +#[must_use] +pub const fn contract_compact( + entries: [(bool, ContractEntry); N], +) -> [ContractEntry; M] { + let mut out = [ContractEntry::new(0, 0); M]; + let mut i = 0; + let mut j = 0; + while i < N { + if entries[i].0 { + out[j] = entries[i].1; + j += 1; + } + i += 1; + } + assert!(j == M, "contract table length mismatch"); + out +} + +/// The body of `{prefix}_{module}_contract`: write the table's length to +/// `out_len` and return its first entry (null for an empty table). +/// +/// # Safety +/// +/// `out_len` must be null or point to a writable `usize`. +pub unsafe fn contract_table( + table: &'static [ContractEntry], + out_len: *mut usize, +) -> *const ContractEntry { + if !out_len.is_null() { + // SAFETY: the caller guarantees `out_len` is writable when non-null. + unsafe { *out_len = table.len() }; + } + if table.is_empty() { + std::ptr::null() + } else { + table.as_ptr() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const ALL: [(bool, ContractEntry); 3] = [ + (true, ContractEntry::new(1, 10)), + (false, ContractEntry::new(2, 20)), + (true, ContractEntry::new(3, 30)), + ]; + const LEN: usize = contract_len(&ALL); + static TABLE: [ContractEntry; LEN] = contract_compact(ALL); + + #[test] + fn disabled_entries_are_compacted_out() { + assert_eq!(LEN, 2); + assert_eq!( + TABLE, + [ContractEntry::new(1, 10), ContractEntry::new(3, 30)] + ); + let mut len = 0; + let ptr = unsafe { contract_table(&TABLE, &mut len) }; + assert_eq!(len, 2); + assert_eq!(unsafe { *ptr.add(1) }.hash, 30); + assert!(unsafe { contract_table(&[], &mut len) }.is_null()); + assert_eq!(len, 0); + } +} diff --git a/crates/weaveffi-abi/src/convert.rs b/crates/weaveffi/src/abi/convert.rs similarity index 72% rename from crates/weaveffi-abi/src/convert.rs rename to crates/weaveffi/src/abi/convert.rs index 3eb9d7f1..d7ea71ea 100644 --- a/crates/weaveffi-abi/src/convert.rs +++ b/crates/weaveffi/src/abi/convert.rs @@ -11,6 +11,11 @@ //! * **lower** functions hand an owned value to the consumer as a //! producer-allocated `(ptr, len)` run, which the consumer releases with //! `{prefix}_free_bytes` (see [`free_bytes`]). +//! * [`alloc`] is the body of `{prefix}_alloc`: a run the consumer fills and +//! either frees with `{prefix}_free_bytes` or hands to the producer (a +//! callback's string, bytes, or buffer return), which adopts it with +//! [`adopt_bytes`]. Every run is a `Box<[u8]>` allocation, so one release +//! function covers both directions. //! //! Strings, bytes, and serialized value buffers all share this one //! representation: a string is its UTF-8 bytes, never NUL-terminated, so an @@ -86,7 +91,7 @@ pub unsafe fn lift_string(ptr: *const u8, len: usize) -> Option { /// [`free_bytes`]. An empty vector yields `(null, 0)` and allocates nothing. /// /// When the vector's capacity equals its length (as it does for every -/// encoding [`encode_value`](crate::encode_value) produces) the conversion +/// encoding [`encode_value`](crate::abi::encode_value) produces) the conversion /// reuses the allocation as is; otherwise it's shrunk to fit first. #[must_use] pub fn bytes_into_raw(data: Vec) -> (*const u8, usize) { @@ -97,7 +102,7 @@ pub fn bytes_into_raw(data: Vec) -> (*const u8, usize) { let ptr = Box::into_raw(data.into_boxed_slice()) .cast::() .cast_const(); - crate::leak::track(crate::leak::ALLOCATIONS, 1); + crate::abi::leak::track(crate::abi::leak::ALLOCATIONS, 1); (ptr, len) } @@ -126,18 +131,54 @@ pub unsafe fn lower_string(s: String, out_len: *mut usize) -> *const u8 { unsafe { lower_bytes(s.into_bytes(), out_len) } } +/// The body of `{prefix}_alloc`: allocate a zero-filled run of `len` bytes +/// the consumer fills, released with [`free_bytes`] (the same `len`) or +/// handed to the producer, which adopts it. A `len` of `0` returns null, +/// which is the empty run. +#[must_use] +pub fn alloc(len: usize) -> *mut u8 { + if len == 0 { + return std::ptr::null_mut(); + } + crate::abi::leak::track(crate::abi::leak::ALLOCATIONS, 1); + Box::into_raw(vec![0u8; len].into_boxed_slice()).cast::() +} + +/// Adopt a run from [`alloc`] (or one this runtime returned) as an owned +/// vector. Null with a length of `0` is the empty run; null with any other +/// length yields `None`. +/// +/// # Safety +/// +/// `ptr` must be null or a run of exactly `len` bytes from [`alloc`] or +/// [`bytes_into_raw`] that nothing else releases or uses afterward. +#[must_use] +pub unsafe fn adopt_bytes(ptr: *mut u8, len: usize) -> Option> { + if ptr.is_null() { + return (len == 0).then(Vec::new); + } + if len == 0 { + return Some(Vec::new()); + } + crate::abi::leak::track(crate::abi::leak::ALLOCATIONS, -1); + // SAFETY: the caller guarantees `ptr` is a `Box<[u8]>` of `len` bytes + // transferred to us. + Some(unsafe { Box::from_raw(std::ptr::slice_from_raw_parts_mut(ptr, len)) }.into_vec()) +} + /// The body of `{prefix}_free_bytes`: release a run returned by this runtime -/// (a string, bytes, or a value buffer). A null `ptr` is a no-op. +/// (a string, bytes, or a value buffer) or allocated with [`alloc`]. A null +/// `ptr` or a `len` of `0` is a no-op. /// /// # Safety /// /// `ptr` must be null or a pointer this runtime returned together with -/// `len`, released exactly once. +/// `len` (or [`alloc`] returned for `len`), released exactly once. pub unsafe fn free_bytes(ptr: *mut u8, len: usize) { - if ptr.is_null() { + if ptr.is_null() || len == 0 { return; } - crate::leak::track(crate::leak::ALLOCATIONS, -1); + crate::abi::leak::track(crate::abi::leak::ALLOCATIONS, -1); // SAFETY: `bytes_into_raw` produced `ptr` from a `Box<[u8]>` of `len` // bytes, and the caller hands it back exactly once. drop(unsafe { Box::from_raw(std::ptr::slice_from_raw_parts_mut(ptr, len)) }); @@ -186,6 +227,22 @@ mod tests { unsafe { free_bytes(ptr.cast_mut(), len) }; } + #[test] + fn alloc_runs_free_and_adopt() { + assert!(alloc(0).is_null()); + let p = alloc(4); + assert_eq!(unsafe { std::slice::from_raw_parts(p, 4) }, [0; 4]); + unsafe { free_bytes(p, 4) }; + let p = alloc(2); + unsafe { *p = 9 }; + assert_eq!(unsafe { adopt_bytes(p, 2) }, Some(vec![9, 0])); + assert_eq!( + unsafe { adopt_bytes(std::ptr::null_mut(), 0) }, + Some(vec![]) + ); + assert_eq!(unsafe { adopt_bytes(std::ptr::null_mut(), 1) }, None); + } + #[test] fn invalid_utf8_is_rejected() { let bad = [0xFFu8, 0xFE]; diff --git a/crates/weaveffi-abi/src/error.rs b/crates/weaveffi/src/abi/error.rs similarity index 79% rename from crates/weaveffi-abi/src/error.rs rename to crates/weaveffi/src/abi/error.rs index 54236fdf..6bfe26a8 100644 --- a/crates/weaveffi-abi/src/error.rs +++ b/crates/weaveffi/src/abi/error.rs @@ -11,7 +11,7 @@ use std::ffi::{CStr, CString}; use std::os::raw::c_char; use std::ptr; -use crate::callback::ForeignError; +use crate::abi::buffer::{BufferDecodeError, BufferReader}; /// The reserved error code for an **untyped producer error**: the default /// [`ErrorReport::code`], used by `Result` and other error types @@ -41,10 +41,10 @@ pub const MARSHAL_ERROR_CODE: i32 = -3; /// implementation failed**. /// /// A consumer's method implementation that raises reports it through the -/// vtable entry's `out_err` slot with this code (via `{prefix}_error_set`); -/// the producer aborts the call it was making and the original caller -/// observes this code with the consumer's message. See -/// [`callback`](crate::callback). +/// vtable entry's `out_err` slot (via `{prefix}_error_set`). The producer +/// sees every failure with this code, except a declared domain code from a +/// method that `throws` (see +/// [`ForeignError`](crate::abi::ForeignError)). pub const FOREIGN_ERROR_CODE: i32 = -4; /// The reserved error code reporting that an async call was **cancelled**: @@ -58,7 +58,7 @@ pub const CANCELLED_ERROR_CODE: i32 = -5; /// `code == 0` means success and every pointer is null. Otherwise `message` /// is a NUL-terminated UTF-8 string and `payload_ptr`/`payload_len` /// optionally hold the matched error code's fields serialized in the -/// [`buffer`](crate::buffer) format. The producer allocates both, and +/// [`buffer`](crate::abi::buffer) format. The producer allocates both, and /// [`error_clear`] (the body of `{prefix}_error_clear`) frees them. /// /// The struct owns its allocations, so it must not be copied bitwise while @@ -115,14 +115,10 @@ impl FfiError { err } - /// Build the error a caught unwind payload describes: a [`ForeignError`] - /// payload keeps its own code and message, and anything else is a - /// producer panic reported with [`PANIC_ERROR_CODE`]. + /// Build the error a caught unwind payload describes: a producer panic, + /// reported with [`PANIC_ERROR_CODE`]. #[must_use] pub fn from_panic(payload: &(dyn std::any::Any + Send)) -> Self { - if let Some(f) = payload.downcast_ref::() { - return Self::new(f.code, &f.message); - } Self::new( PANIC_ERROR_CODE, &format!("producer panicked: {}", panic_message(payload)), @@ -163,7 +159,7 @@ impl FfiError { if !self.payload_ptr.is_null() { // SAFETY: every non-null payload was produced by // `bytes_into_raw` in `set_payload` with this length. - unsafe { crate::free_bytes(self.payload_ptr.cast_mut(), self.payload_len) }; + unsafe { crate::abi::free_bytes(self.payload_ptr.cast_mut(), self.payload_len) }; self.payload_ptr = ptr::null(); } self.payload_len = 0; @@ -186,10 +182,26 @@ impl FfiError { } fn set_payload(&mut self, payload: Vec) { - let (ptr, len) = crate::convert::bytes_into_raw(payload); + if !self.payload_ptr.is_null() { + // SAFETY: every non-null payload was produced by + // `bytes_into_raw` with this length, and is replaced right after. + unsafe { crate::abi::free_bytes(self.payload_ptr.cast_mut(), self.payload_len) }; + } + let (ptr, len) = crate::abi::convert::bytes_into_raw(payload); self.payload_ptr = ptr; self.payload_len = len; } + + /// The payload as a byte slice (empty when there is none). + #[must_use] + pub fn payload(&self) -> &[u8] { + if self.payload_ptr.is_null() { + return &[]; + } + // SAFETY: a non-null payload is an owned run of `payload_len` bytes + // (see `set_payload`), alive as long as `self`. + unsafe { std::slice::from_raw_parts(self.payload_ptr, self.payload_len) } + } } impl Drop for FfiError { @@ -249,6 +261,28 @@ pub unsafe fn error_set_c(out_err: *mut FfiError, code: i32, message: *const c_c unsafe { error_set(out_err, code, text) }; } +/// The body of `{prefix}_error_set_payload`: replace `out_err`'s payload +/// with a producer-owned copy of the `len` bytes at `ptr`. A null `out_err` +/// is a no-op; a null `ptr` (or a `len` of `0`) clears the payload. +/// +/// A callback-interface implementation of a `throws` method calls it after +/// `{prefix}_error_set` to attach a domain code's fields, encoded as a value +/// buffer, so the producer can decode the typed error. +/// +/// # Safety +/// +/// Same contract as [`error_set`] for `out_err`. `ptr` must be null or point +/// to `len` readable bytes for the duration of the call. +pub unsafe fn error_set_payload_c(out_err: *mut FfiError, ptr: *const u8, len: usize) { + // SAFETY: forwarded from the caller. + let Some(err) = (unsafe { out_err.as_mut() }) else { + return; + }; + // SAFETY: forwarded from the caller. + let bytes = unsafe { crate::abi::lift_byte_slice(ptr, len) }.unwrap_or(&[]); + err.set_payload(bytes.to_vec()); +} + /// The body of `{prefix}_error_clear`: free any message and payload and set /// the code to `0`. A null `err` is a no-op. /// @@ -297,7 +331,7 @@ pub fn boxed_error(err: FfiError) -> *mut FfiError { /// To implement the trait by hand: /// /// ``` -/// use weaveffi_abi::ErrorReport; +/// use weaveffi::abi::ErrorReport; /// /// enum KvError { /// KeyNotFound, @@ -336,6 +370,30 @@ pub trait ErrorReport { } } +/// An error type the `#[weaveffi::error]` expansion generates: a module's +/// error domain, decodable from a code and its payload. +/// +/// [`ForeignError::domain`](crate::abi::ForeignError::domain) uses it to turn +/// a domain error a consumer's callback reported into the typed error. +pub trait ErrorDomain: ErrorReport + Sized { + /// Decode the error `code` names, reading its payload fields (if any) + /// from `r`. Returns `Ok(None)` when `code` isn't one of the domain's + /// declared codes. + /// + /// # Errors + /// + /// Returns an error when the payload is malformed for the code. + /// + /// # Safety + /// + /// Same contract as + /// [`BufferValue::read_value`](crate::abi::BufferValue::read_value). + unsafe fn read_code( + code: i32, + r: &mut BufferReader<'_>, + ) -> Result, BufferDecodeError>; +} + impl ErrorReport for String { fn message(&self) -> String { self.clone() @@ -360,16 +418,14 @@ impl ErrorReport for Box { } } -/// Best-effort extraction of a panic payload's message (`&str`, `String`, -/// and [`ForeignError`] payloads; anything else yields a fixed placeholder). +/// Best-effort extraction of a panic payload's message (`&str` and `String` +/// payloads; anything else yields a fixed placeholder). #[must_use] pub fn panic_message(payload: &(dyn std::any::Any + Send)) -> String { if let Some(s) = payload.downcast_ref::<&str>() { (*s).to_string() } else if let Some(s) = payload.downcast_ref::() { s.clone() - } else if let Some(f) = payload.downcast_ref::() { - f.message.clone() } else { "producer panicked".to_string() } @@ -455,7 +511,7 @@ mod tests { fn payload(&self) -> Vec { match self { DomainError::NotFound => Vec::new(), - DomainError::Io(detail) => crate::encode_value(detail), + DomainError::Io(detail) => crate::abi::encode_value(detail), } } } @@ -469,8 +525,10 @@ mod tests { let err = FfiError::from_report(&DomainError::Io("disk".into())); assert_eq!(err.code, 1004); - let payload = unsafe { std::slice::from_raw_parts(err.payload_ptr, err.payload_len) }; - assert_eq!(crate::decode_value::(payload).unwrap(), "disk"); + assert_eq!( + unsafe { crate::abi::decode_value::(err.payload()) }.unwrap(), + "disk" + ); let mut slot = FfiError::default(); unsafe { error_store(&mut slot, err) }; @@ -486,15 +544,18 @@ mod tests { } #[test] - fn panic_payloads_map_to_panic_or_foreign_codes() { - let payload: Box = Box::new(ForeignError { - code: FOREIGN_ERROR_CODE, - message: "listener threw".into(), - }); - let err = FfiError::from_panic(&*payload); - assert_eq!(err.code, FOREIGN_ERROR_CODE); - assert_eq!(message(&err).as_deref(), Some("listener threw")); + fn set_payload_copies_and_replaces() { + let mut err = FfiError::new(3, "with fields"); + let bytes = [1u8, 2, 3]; + unsafe { error_set_payload_c(&mut err, bytes.as_ptr(), bytes.len()) }; + assert_eq!(err.payload(), &bytes); + unsafe { error_set_payload_c(&mut err, ptr::null(), 0) }; + assert!(err.payload().is_empty()); + unsafe { error_set_payload_c(ptr::null_mut(), bytes.as_ptr(), bytes.len()) }; + } + #[test] + fn panic_payloads_map_to_the_panic_code() { let payload: Box = Box::new("bug"); let err = FfiError::from_panic(&*payload); assert_eq!(err.code, PANIC_ERROR_CODE); diff --git a/crates/weaveffi-abi/src/iter.rs b/crates/weaveffi/src/abi/iter.rs similarity index 51% rename from crates/weaveffi-abi/src/iter.rs rename to crates/weaveffi/src/abi/iter.rs index 5442a18d..85d0af8b 100644 --- a/crates/weaveffi-abi/src/iter.rs +++ b/crates/weaveffi/src/abi/iter.rs @@ -3,6 +3,8 @@ use std::sync::{Mutex, PoisonError}; +use crate::abi::error::{FfiError, MARSHAL_ERROR_CODE}; + /// An owned, type-erased iterator returned by a producer function whose IDL /// return type is `iter`. /// @@ -48,39 +50,82 @@ impl std::fmt::Debug for Iter { /// The opaque iterator handle a launcher returns (`{prefix}_..._Iterator*` /// in C). /// -/// `_next` is internally synchronized: the iterator sits behind a mutex, so -/// concurrent calls on one handle are safe (though they race for elements). +/// The handle never holds its lock while the producer's `next` runs: `_next` +/// takes the iterator out, advances it, and puts it back. A `_next` that +/// arrives while another is advancing the same handle (from another thread, +/// or re-entrantly from inside the producer's `next`) fails with +/// [`MARSHAL_ERROR_CODE`] instead of blocking, so no misuse can deadlock. #[derive(Debug)] pub struct IterHandle { - inner: Mutex>, + inner: Mutex>>, } /// Box `iter` behind a new handle for the consumer, who releases it with /// `_destroy` (see [`iter_destroy`]). #[must_use] pub fn iter_into_raw(iter: Iter) -> *mut IterHandle { - crate::leak::track(crate::leak::ITERATORS, 1); + crate::abi::leak::track(crate::abi::leak::ITERATORS, 1); Box::into_raw(Box::new(IterHandle { - inner: Mutex::new(iter), + inner: Mutex::new(Some(iter)), })) } -/// Pull the next element from a handle: `Some(None)` when the iterator is -/// exhausted, `None` when `handle` is null. +/// Puts the iterator back into its handle when dropped, including when the +/// producer's `next` unwinds, so a panic doesn't leave the handle busy. +struct Advancing<'a, T> { + handle: &'a IterHandle, + iter: Option>, +} + +impl Drop for Advancing<'_, T> { + fn drop(&mut self) { + let mut slot = self + .handle + .inner + .lock() + .unwrap_or_else(PoisonError::into_inner); + *slot = self.iter.take(); + } +} + +/// Pull the next element from a handle: `Ok(None)` when the iterator is +/// exhausted. /// /// A producer panic from an earlier `next` doesn't wedge the handle: the -/// mutex's poison flag is ignored, and the iterator keeps whatever state the -/// panic left it in. +/// iterator is put back in whatever state the panic left it. +/// +/// # Errors +/// +/// Returns a [`MARSHAL_ERROR_CODE`] error when `handle` is null or another +/// `_next` is advancing it right now. /// /// # Safety /// /// `handle` must be null or a live pointer from [`iter_into_raw`] that isn't /// destroyed during the call. -pub unsafe fn iter_next(handle: *const IterHandle) -> Option> { +pub unsafe fn iter_next(handle: *const IterHandle) -> Result, FfiError> { // SAFETY: the caller guarantees a live handle or null. - let handle = unsafe { handle.as_ref() }?; - let mut iter = handle.inner.lock().unwrap_or_else(PoisonError::into_inner); - Some(iter.next()) + let Some(handle) = (unsafe { handle.as_ref() }) else { + return Err(FfiError::new(MARSHAL_ERROR_CODE, "iterator is null")); + }; + let taken = handle + .inner + .lock() + .unwrap_or_else(PoisonError::into_inner) + .take(); + let Some(iter) = taken else { + return Err(FfiError::new( + MARSHAL_ERROR_CODE, + "iterator is already being advanced (a concurrent or re-entrant _next)", + )); + }; + let mut advancing = Advancing { + handle, + iter: Some(iter), + }; + let item = advancing.iter.as_mut().and_then(Iterator::next); + drop(advancing); + Ok(item) } /// The body of every iterator's `_destroy` symbol: drop the iterator and its @@ -95,7 +140,7 @@ pub unsafe fn iter_destroy(handle: *mut IterHandle) { if handle.is_null() { return; } - crate::leak::track(crate::leak::ITERATORS, -1); + crate::abi::leak::track(crate::abi::leak::ITERATORS, -1); let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { // SAFETY: the caller hands back a pointer from `Box::into_raw` // exactly once. @@ -107,21 +152,25 @@ pub unsafe fn iter_destroy(handle: *mut IterHandle) { mod tests { use super::*; + fn next(h: *const IterHandle) -> Option { + unsafe { iter_next(h) }.unwrap() + } + #[test] fn pulls_until_exhausted() { let h = iter_into_raw(Iter::new(vec![1, 2])); + assert_eq!(next(h), Some(1)); + assert_eq!(next(h), Some(2)); + assert_eq!(next(h), None); unsafe { - assert_eq!(iter_next(h), Some(Some(1))); - assert_eq!(iter_next(h), Some(Some(2))); - assert_eq!(iter_next(h), Some(None)); - assert_eq!(iter_next::(std::ptr::null()), None); + assert!(iter_next::(std::ptr::null()).is_err()); iter_destroy(h); iter_destroy::(std::ptr::null_mut()); } } #[test] - fn concurrent_next_yields_each_element_once() { + fn concurrent_next_yields_each_element_once_or_reports_busy() { let h = iter_into_raw(Iter::new(0..10_000u32)); let addr = h as usize; let threads: Vec<_> = (0..8) @@ -129,8 +178,15 @@ mod tests { std::thread::spawn(move || { let h = addr as *const IterHandle; let mut got = Vec::new(); - while let Some(Some(x)) = unsafe { iter_next(h) } { - got.push(x); + loop { + match unsafe { iter_next(h) } { + Ok(Some(x)) => got.push(x), + Ok(None) => break, + Err(e) => { + assert_eq!(e.code, MARSHAL_ERROR_CODE); + std::thread::yield_now(); + } + } } got }) @@ -144,4 +200,22 @@ mod tests { assert_eq!(all, (0..10_000).collect::>()); unsafe { iter_destroy(h) }; } + + #[test] + fn a_panicking_next_puts_the_iterator_back() { + let mut n = 0; + let h = iter_into_raw(Iter::new(std::iter::from_fn(move || { + n += 1; + assert!(n != 2, "second element"); + Some(n) + }))); + assert_eq!(next(h), Some(1)); + let addr = h as usize; + assert!(std::panic::catch_unwind(move || { + let _ = unsafe { iter_next(addr as *const IterHandle) }; + }) + .is_err()); + assert_eq!(next(h), Some(3)); + unsafe { iter_destroy(h) }; + } } diff --git a/crates/weaveffi-abi/src/leak.rs b/crates/weaveffi/src/abi/leak.rs similarity index 84% rename from crates/weaveffi-abi/src/leak.rs rename to crates/weaveffi/src/abi/leak.rs index c23e39b2..06781edd 100644 --- a/crates/weaveffi-abi/src/leak.rs +++ b/crates/weaveffi/src/abi/leak.rs @@ -8,12 +8,16 @@ //! //! The symbol is exported whether or not the feature is on, so a consumer //! never has to know how the producer was built: without the feature every -//! counter reads `0`. Check [`ENABLED`] to tell "nothing is live" from -//! "nothing is counted". +//! counter reads `0`. Ask for kind [`COUNTING`] (`-1`) to tell "nothing is +//! live" from "nothing is counted". /// Whether this build counts live resources (the `leak-check` feature). pub const ENABLED: bool = cfg!(feature = "leak-check"); +/// The pseudo-kind `{prefix}_debug_live` answers with `1` when the producer +/// counts live resources ([`ENABLED`]) and `0` when it doesn't. +pub const COUNTING: i32 = -1; + /// Interface object references held by the consumer: one per pointer a /// thunk returns, per `_clone`, and per object token written into a value /// buffer; released by `_destroy` and by adopting a token. @@ -72,12 +76,16 @@ pub(crate) fn track(kind: i32, delta: i64) { /// The body of `{prefix}_debug_live`: how many resources of `kind` (one of /// [`OBJECTS`], [`CALLBACKS`], [`ITERATORS`], [`TOKENS`], or -/// [`ALLOCATIONS`]) are live right now. +/// [`ALLOCATIONS`]) are live right now, or, for [`COUNTING`], `1` when this +/// build counts at all. /// -/// Returns `0` for an unknown kind, and for every kind when the +/// Returns `0` for an unknown kind, and for every counter when the /// `leak-check` feature is off. #[must_use] pub fn debug_live(kind: i32) -> u64 { + if kind == COUNTING { + return u64::from(ENABLED); + } #[cfg(feature = "leak-check")] { counters::live(kind) diff --git a/crates/weaveffi/src/abi/marshal.rs b/crates/weaveffi/src/abi/marshal.rs new file mode 100644 index 00000000..1e826b5b --- /dev/null +++ b/crates/weaveffi/src/abi/marshal.rs @@ -0,0 +1,520 @@ +//! The glue every generated thunk calls: per-family parameter lifting, +//! return lowering, and the synchronous call driver. +//! +//! The `#[weaveffi::module]` expansion emits one short call per parameter +//! and per return into this module instead of inlining the marshalling, so +//! sync thunks, async launchers, iterator steps, and callback-interface +//! wrappers share one implementation of each family's rules: +//! +//! * `lift_*` functions turn a parameter's C slots into the Rust value the +//! producer's function takes, or a [`MARSHAL_ERROR_CODE`] error naming the +//! parameter. A thunk lifts **every** parameter before it looks at any +//! result, so a callback context or the object tokens in a buffer are +//! always adopted (and released again) even when another parameter fails. +//! * `lower_*` functions (and [`CEnum::to_i32`]) turn a returned value into +//! its C return and out slots. +//! * [`call_sync`] runs a synchronous thunk's body under `catch_unwind` and +//! reports its outcome through `out_err`. + +use std::ffi::c_void; +use std::sync::Arc; + +use crate::abi::callback::{lift_callback, lift_callback_opt, CallbackInterface}; +use crate::abi::error::{error_clear, error_store, FfiError, MARSHAL_ERROR_CODE}; +use crate::abi::BufferValue; + +/// A C-style enum: an `i32` discriminant at the C ABI and inside value +/// buffers. The `#[weaveffi::module]` expansion implements it for every +/// `#[repr(i32)]` `#[weaveffi::enumeration]`. +pub trait CEnum: Sized { + /// The variant whose discriminant is `value`, or `None` when no variant + /// has it. + fn from_i32(value: i32) -> Option; + + /// This variant's discriminant. + fn to_i32(&self) -> i32; +} + +/// Write a C-style enum into a value buffer (the body of its +/// `BufferValue::write_value`). +pub fn write_enum(value: &E, w: &mut crate::abi::BufferWriter) { + w.write_i32(value.to_i32()); +} + +/// Read a C-style enum from a value buffer (the body of its +/// `BufferValue::read_value`). +/// +/// # Errors +/// +/// Returns an error when the buffer is exhausted or the discriminant isn't +/// one of `E`'s. +pub fn read_enum( + r: &mut crate::abi::BufferReader<'_>, +) -> Result { + E::from_i32(r.read_i32()?).ok_or(crate::abi::BufferDecodeError { + context: "enum discriminant out of range", + }) +} + +/// The zero value a C return slot carries when a call fails: `0`, `false`, +/// `0.0`, or null. Implemented for every C return type a thunk can have, +/// `()` for `void`, and pairs and tuples of them (an async completion's +/// result slots). +pub trait Sentinel { + /// The zero value. + fn sentinel() -> Self; +} + +macro_rules! zero_sentinel { + ($($t:ty => $v:expr),* $(,)?) => { + $(impl Sentinel for $t { + fn sentinel() -> Self { + $v + } + })* + }; +} + +zero_sentinel! { + () => (), + bool => false, + i8 => 0, i16 => 0, i32 => 0, i64 => 0, + u8 => 0, u16 => 0, u32 => 0, u64 => 0, usize => 0, + f32 => 0.0, f64 => 0.0, +} + +impl Sentinel for *const T { + fn sentinel() -> Self { + std::ptr::null() + } +} + +impl Sentinel for *mut T { + fn sentinel() -> Self { + std::ptr::null_mut() + } +} + +impl Sentinel for (A,) { + fn sentinel() -> Self { + (A::sentinel(),) + } +} + +impl Sentinel for (A, B) { + fn sentinel() -> Self { + (A::sentinel(), B::sentinel()) + } +} + +/// Run a synchronous thunk's body and report its outcome: `Ok` clears +/// `out_err` and returns the value; `Err` stores the error and returns the +/// [`Sentinel`]; a panic is caught and reported with the panic code, so it +/// never unwinds into C. +/// +/// # Safety +/// +/// `out_err` must be null or point to a valid, writable error slot whose +/// pointers are null or owned by this runtime. +pub unsafe fn call_sync( + out_err: *mut FfiError, + body: impl FnOnce() -> Result, +) -> R { + let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(body)) + .unwrap_or_else(|payload| Err(FfiError::from_panic(&*payload))); + match outcome { + Ok(value) => { + // SAFETY: forwarded from the caller. + unsafe { error_clear(out_err) }; + value + } + Err(e) => { + // SAFETY: forwarded from the caller. + unsafe { error_store(out_err, e) }; + R::sentinel() + } + } +} + +fn invalid(name: &str) -> FfiError { + FfiError::new(MARSHAL_ERROR_CODE, &format!("{name} is null or invalid")) +} + +/// Lift a C-style enum parameter. +/// +/// # Errors +/// +/// Returns a marshalling error when `value` isn't one of `E`'s +/// discriminants. +pub fn lift_enum(value: i32, name: &str) -> Result { + E::from_i32(value).ok_or_else(|| { + FfiError::new( + MARSHAL_ERROR_CODE, + &format!("{name}: {value} is not a valid enum value"), + ) + }) +} + +/// Borrow a string parameter's UTF-8 `(ptr, len)` run as a `&str`. +/// +/// # Errors +/// +/// Returns a marshalling error for null with a non-zero length or bytes that +/// aren't UTF-8. +/// +/// # Safety +/// +/// Same contract as [`lift_byte_slice`](crate::abi::lift_byte_slice). +pub unsafe fn lift_str_param<'a>( + ptr: *const u8, + len: usize, + name: &str, +) -> Result<&'a str, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::lift_str(ptr, len) }.ok_or_else(|| invalid(name)) +} + +/// Copy a string parameter into an owned `String`. +/// +/// # Errors +/// +/// Same as [`lift_str_param`]. +/// +/// # Safety +/// +/// Same contract as [`lift_str_param`]. +pub unsafe fn lift_string_param( + ptr: *const u8, + len: usize, + name: &str, +) -> Result { + // SAFETY: forwarded from the caller. + unsafe { lift_str_param(ptr, len, name) }.map(str::to_owned) +} + +/// Borrow a bytes parameter's `(ptr, len)` run as a slice. +/// +/// # Errors +/// +/// Returns a marshalling error for null with a non-zero length. +/// +/// # Safety +/// +/// Same contract as [`lift_byte_slice`](crate::abi::lift_byte_slice). +pub unsafe fn lift_slice_param<'a>( + ptr: *const u8, + len: usize, + name: &str, +) -> Result<&'a [u8], FfiError> { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::lift_byte_slice(ptr, len) }.ok_or_else(|| invalid(name)) +} + +/// Copy a bytes parameter into an owned `Vec`. +/// +/// # Errors +/// +/// Same as [`lift_slice_param`]. +/// +/// # Safety +/// +/// Same contract as [`lift_slice_param`]. +pub unsafe fn lift_bytes_param( + ptr: *const u8, + len: usize, + name: &str, +) -> Result, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { lift_slice_param(ptr, len, name) }.map(<[u8]>::to_vec) +} + +/// Decode a value-buffer parameter (a record, rich enum, optional, list, or +/// map), adopting any object tokens it carries. +/// +/// # Errors +/// +/// Returns a marshalling error for null with a non-zero length or a +/// malformed buffer. +/// +/// # Safety +/// +/// Same contract as [`lift_slice_param`], plus that of +/// [`decode_value`](crate::abi::decode_value) for the buffer's object tokens. +pub unsafe fn lift_buffer_param( + ptr: *const u8, + len: usize, + name: &str, +) -> Result { + // SAFETY: forwarded from the caller. + let bytes = unsafe { lift_slice_param(ptr, len, name) }?; + // SAFETY: forwarded from the caller; the thunk decodes each buffer once. + unsafe { crate::abi::decode_value(bytes) } + .map_err(|e| FfiError::new(MARSHAL_ERROR_CODE, &format!("{name}: {e}"))) +} + +/// Borrow an object parameter for the call. +/// +/// # Errors +/// +/// Returns a marshalling error for null. +/// +/// # Safety +/// +/// Same contract as [`object_ref`](crate::abi::object_ref). +pub unsafe fn lift_object_param<'a, T>(ptr: *const T, name: &str) -> Result<&'a T, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::object_ref(ptr) }.ok_or_else(|| invalid(name)) +} + +/// Take a new strong reference to an object parameter, so the producer can +/// retain it. +/// +/// # Errors +/// +/// Returns a marshalling error for null. +/// +/// # Safety +/// +/// Same contract as [`object_arc`](crate::abi::object_arc). +pub unsafe fn lift_object_arc_param(ptr: *const T, name: &str) -> Result, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::object_arc(ptr) }.ok_or_else(|| invalid(name)) +} + +/// Borrow an optional object parameter (`I?`); null is `None`. +/// +/// # Errors +/// +/// Never fails; the `Result` keeps every lift the same shape. +/// +/// # Safety +/// +/// Same contract as [`object_ref`](crate::abi::object_ref). +pub unsafe fn lift_object_opt_param<'a, T>(ptr: *const T) -> Result, FfiError> { + // SAFETY: forwarded from the caller. + Ok(unsafe { crate::abi::object_ref(ptr) }) +} + +/// Retain an optional object parameter (`I?`); null is `None`. +/// +/// # Errors +/// +/// Never fails; the `Result` keeps every lift the same shape. +/// +/// # Safety +/// +/// Same contract as [`object_arc`](crate::abi::object_arc). +pub unsafe fn lift_object_arc_opt_param(ptr: *const T) -> Result>, FfiError> { + // SAFETY: forwarded from the caller. + Ok(unsafe { crate::abi::object_arc(ptr) }) +} + +/// Adopt a callback-interface parameter's `(ctx, vtable)` pair. +/// +/// # Errors +/// +/// Returns a marshalling error for a null or too-small vtable. +/// +/// # Safety +/// +/// Same contract as +/// [`ForeignCallback::from_raw`](crate::abi::ForeignCallback::from_raw). +pub unsafe fn lift_callback_param( + ctx: *mut c_void, + vtable: *const C::Vtable, + name: &str, +) -> Result, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { lift_callback::(ctx, vtable, name) } +} + +/// Adopt an optional callback-interface parameter (`Cb?`); a null vtable is +/// `None`. +/// +/// # Errors +/// +/// Returns a marshalling error for a too-small vtable. +/// +/// # Safety +/// +/// Same contract as [`lift_callback_param`]. +pub unsafe fn lift_callback_opt_param( + ctx: *mut c_void, + vtable: *const C::Vtable, + name: &str, +) -> Result>, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { lift_callback_opt::(ctx, vtable, name) } +} + +/// Lift a method's receiver (`self`), borrowed for the call. +/// +/// # Errors +/// +/// Returns a marshalling error for null. +/// +/// # Safety +/// +/// Same contract as [`lift_object_param`]. +pub unsafe fn lift_self<'a, T>(ptr: *const T) -> Result<&'a T, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { lift_object_param(ptr, "self") } +} + +/// Lift a method's receiver as a retained reference (`self: Arc`, and +/// every async method, whose future outlives the launcher). +/// +/// # Errors +/// +/// Returns a marshalling error for null. +/// +/// # Safety +/// +/// Same contract as [`lift_object_arc_param`]. +pub unsafe fn lift_self_arc(ptr: *const T) -> Result, FfiError> { + // SAFETY: forwarded from the caller. + unsafe { lift_object_arc_param(ptr, "self") } +} + +/// Lower a returned string as a producer-allocated run plus `*out_len`. +/// +/// # Safety +/// +/// `out_len` must be null or point to a writable `usize`. +pub unsafe fn lower_string_ret(value: impl Into, out_len: *mut usize) -> *const u8 { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::lower_string(value.into(), out_len) } +} + +/// Lower returned bytes as a producer-allocated run plus `*out_len`. +/// +/// # Safety +/// +/// Same contract as [`lower_string_ret`]. +pub unsafe fn lower_bytes_ret(value: impl Into>, out_len: *mut usize) -> *const u8 { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::lower_bytes(value.into(), out_len) } +} + +/// Encode a returned value buffer as a producer-allocated run plus +/// `*out_len`. +/// +/// # Safety +/// +/// Same contract as [`lower_string_ret`]. +pub unsafe fn lower_buffer_ret(value: &T, out_len: *mut usize) -> *const u8 { + // SAFETY: forwarded from the caller. + unsafe { crate::abi::lower_bytes(crate::abi::encode_value(value), out_len) } +} + +/// A string as an async completion's owned `(result_ptr, result_len)` run. +#[must_use] +pub fn string_run(value: impl Into) -> (*const u8, usize) { + crate::abi::bytes_into_raw(value.into().into_bytes()) +} + +/// Bytes as an async completion's owned `(result_ptr, result_len)` run. +#[must_use] +pub fn bytes_run(value: impl Into>) -> (*const u8, usize) { + crate::abi::bytes_into_raw(value.into()) +} + +/// A value buffer as an async completion's owned `(result_ptr, result_len)` +/// run. +#[must_use] +pub fn buffer_run(value: &T) -> (*const u8, usize) { + crate::abi::bytes_into_raw(crate::abi::encode_value(value)) +} + +/// Borrow a string-like argument as the `(ptr, len)` slots of a callback +/// method call. +pub fn str_slots + ?Sized>(value: &S) -> (*const u8, usize) { + let s = value.as_ref(); + (s.as_ptr(), s.len()) +} + +/// Borrow a bytes-like argument as the `(ptr, len)` slots of a callback +/// method call. +pub fn byte_slots + ?Sized>(value: &B) -> (*const u8, usize) { + let b = value.as_ref(); + (b.as_ptr(), b.len()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[derive(Debug, PartialEq)] + enum Color { + Red, + Blue, + } + + impl CEnum for Color { + fn from_i32(value: i32) -> Option { + match value { + 0 => Some(Self::Red), + 2 => Some(Self::Blue), + _ => None, + } + } + fn to_i32(&self) -> i32 { + match self { + Self::Red => 0, + Self::Blue => 2, + } + } + } + + #[test] + fn enums_lift_and_round_trip_through_buffers() { + assert_eq!(lift_enum::(2, "c").unwrap(), Color::Blue); + let err = lift_enum::(1, "c").unwrap_err(); + assert_eq!(err.code, MARSHAL_ERROR_CODE); + let mut w = crate::abi::BufferWriter::new(); + write_enum(&Color::Blue, &mut w); + let bytes = w.finish(); + let mut r = crate::abi::BufferReader::new(&bytes); + assert_eq!(read_enum::(&mut r).unwrap(), Color::Blue); + } + + #[test] + fn call_sync_reports_errors_and_panics() { + let mut err = FfiError::new(1, "stale"); + assert_eq!(unsafe { call_sync(&mut err, || Ok(5i32)) }, 5); + assert_eq!(err.code, 0); + let p: *const u8 = + unsafe { call_sync(&mut err, || Err(FfiError::new(MARSHAL_ERROR_CODE, "bad"))) }; + assert!(p.is_null()); + assert_eq!(err.code, MARSHAL_ERROR_CODE); + let v: f64 = unsafe { + call_sync(&mut err, || { + if true { + panic!("boom"); + } + Ok(1.0) + }) + }; + assert_eq!(v, 0.0); + assert_eq!(err.code, crate::abi::PANIC_ERROR_CODE); + } + + #[test] + fn param_lifts_name_the_parameter() { + let err = unsafe { lift_str_param(std::ptr::null(), 3, "title") }.unwrap_err(); + assert_eq!( + unsafe { err.message_str() }, + Some("title is null or invalid") + ); + let bad = [0xffu8]; + assert!(unsafe { lift_string_param(bad.as_ptr(), 1, "s") }.is_err()); + let err = unsafe { lift_buffer_param::>(bad.as_ptr(), 1, "xs") }.unwrap_err(); + assert!(unsafe { err.message_str() } + .unwrap() + .starts_with("xs: malformed")); + assert!(unsafe { lift_object_param::(std::ptr::null(), "o") }.is_err()); + assert!(unsafe { lift_object_opt_param::(std::ptr::null()) } + .unwrap() + .is_none()); + } +} diff --git a/crates/weaveffi/src/abi/mod.rs b/crates/weaveffi/src/abi/mod.rs new file mode 100644 index 00000000..4fe42192 --- /dev/null +++ b/crates/weaveffi/src/abi/mod.rs @@ -0,0 +1,84 @@ +//! The stable C ABI runtime: the error struct, memory helpers, +//! reference-counted objects, cancel tokens, callback-interface vtables, +//! iterators, the value-buffer codec, and the async executor hook. +//! +//! Producers rarely name these items. The `#[weaveffi::module]` expansion +//! and [`export_runtime!`](crate::export_runtime) call into them as +//! `::weaveffi::abi::*`, so every `unsafe` pointer operation a producer +//! performs has one audited home. The normative description of the contract +//! this module implements is `docs/src/reference/abi.md` in the WeaveFFI +//! repository. +#![allow(unsafe_code)] + +pub mod buffer; +pub mod callback; +pub mod cancel; +pub mod contract; +pub mod convert; +pub mod error; +pub mod iter; +pub mod leak; +pub mod marshal; +pub mod object; +pub mod spawn; + +pub use buffer::{ + decode_value, encode_value, BufferDecodeError, BufferReader, BufferValue, BufferWriter, + ByValue, FixedWidth, +}; +pub use callback::{ + callback_ret_buffer, callback_ret_bytes, callback_ret_enum, callback_ret_object, + callback_ret_object_opt, callback_ret_string, callback_status, callback_status_in, + lift_callback, lift_callback_opt, CallbackInterface, ForeignCallback, ForeignError, Vtable, + VtableHeader, VtableTooSmall, +}; +pub use cancel::{ + cancel_token_cancel, cancel_token_create, cancel_token_destroy, cancel_token_is_cancelled, + AtomicWaker, CancelToken, Cancellable, FfiCancelToken, +}; +pub use contract::{contract_compact, contract_len, contract_table, ContractEntry}; +pub use convert::{ + adopt_bytes, alloc, bytes_into_raw, free_bytes, lift_byte_slice, lift_bytes, lift_str, + lift_string, lower_bytes, lower_string, +}; +pub use error::{ + boxed_error, error_clear, error_free, error_set, error_set_c, error_set_payload_c, error_store, + panic_message, ErrorDomain, ErrorReport, FfiError, CANCELLED_ERROR_CODE, FOREIGN_ERROR_CODE, + GENERIC_ERROR_CODE, MARSHAL_ERROR_CODE, PANIC_ERROR_CODE, +}; +pub use iter::{iter_destroy, iter_into_raw, iter_next, Iter, IterHandle}; +pub use leak::debug_live; +pub use marshal::{ + buffer_run, byte_slots, bytes_run, call_sync, lift_buffer_param, lift_bytes_param, + lift_callback_opt_param, lift_callback_param, lift_enum, lift_object_arc_opt_param, + lift_object_arc_param, lift_object_opt_param, lift_object_param, lift_self, lift_self_arc, + lift_slice_param, lift_str_param, lift_string_param, lower_buffer_ret, lower_bytes_ret, + lower_string_ret, read_enum, str_slots, string_run, write_enum, CEnum, Sentinel, +}; +pub use object::{ + lower_object, lower_object_opt, object_arc, object_clone, object_destroy, object_from_token, + object_ref, object_to_token, +}; +pub use spawn::{ + block_on, default_pool_size, launch_async, run_async, set_spawner, spawn, BoxFuture, + CatchUnwind, SpawnError, Spawner, SpawnerAlreadySet, +}; + +/// The revision of the WeaveFFI C ABI this runtime implements. +/// +/// Every producer exports it as `{prefix}_abi_version()` (via +/// `weaveffi::export_runtime!()`), and every generated consumer compares it +/// with the revision it was generated against when it loads the library, +/// turning a silent memory-layout mismatch into a clear error. +/// +/// The number only changes when the runtime surface (the error layout, the +/// value-buffer encoding, the object or callback-interface conventions, or +/// the set and signatures of the runtime symbols) changes incompatibly. It's +/// independent of the crate version and of the IDL schema version. +/// +/// Revision 4 replaced the per-module checksums with per-declaration +/// contract tables, gave callback vtables a size-checked header and every +/// return family (with `throws` and optional callback parameters), exported +/// `{prefix}_alloc` on every target (removing `{prefix}_dealloc`), and added +/// `{prefix}_error_set_payload`. +pub const ABI_VERSION: u32 = 4; diff --git a/crates/weaveffi-abi/src/object.rs b/crates/weaveffi/src/abi/object.rs similarity index 96% rename from crates/weaveffi-abi/src/object.rs rename to crates/weaveffi/src/abi/object.rs index efd3ea3a..5ea85bd9 100644 --- a/crates/weaveffi-abi/src/object.rs +++ b/crates/weaveffi/src/abi/object.rs @@ -23,7 +23,7 @@ use std::sync::Arc; /// The consumer releases the reference with `{tag}_destroy`. #[must_use] pub fn lower_object(value: impl Into>) -> *mut T { - crate::leak::track(crate::leak::OBJECTS, 1); + crate::abi::leak::track(crate::abi::leak::OBJECTS, 1); Arc::into_raw(value.into()).cast_mut() } @@ -91,7 +91,7 @@ pub unsafe fn object_clone(ptr: *const T) -> *mut T { } // SAFETY: as documented on the function. unsafe { Arc::increment_strong_count(ptr) }; - crate::leak::track(crate::leak::OBJECTS, 1); + crate::abi::leak::track(crate::abi::leak::OBJECTS, 1); ptr.cast_mut() } @@ -108,7 +108,7 @@ pub unsafe fn object_destroy(ptr: *mut T) { if ptr.is_null() { return; } - crate::leak::track(crate::leak::OBJECTS, -1); + crate::abi::leak::track(crate::abi::leak::OBJECTS, -1); let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { // SAFETY: as documented on the function. unsafe { Arc::decrement_strong_count(ptr) }; @@ -120,7 +120,7 @@ pub unsafe fn object_destroy(ptr: *mut T) { /// reference; whoever decodes it adopts that reference. #[must_use] pub fn object_to_token(value: &Arc) -> u64 { - crate::leak::track(crate::leak::OBJECTS, 1); + crate::abi::leak::track(crate::abi::leak::OBJECTS, 1); Arc::into_raw(Arc::clone(value)) as usize as u64 } @@ -138,7 +138,7 @@ pub unsafe fn object_from_token(token: u64) -> Option> { if ptr.is_null() { return None; } - crate::leak::track(crate::leak::OBJECTS, -1); + crate::abi::leak::track(crate::abi::leak::OBJECTS, -1); // SAFETY: the caller guarantees the token carries one live strong // reference that has not been adopted yet. Some(unsafe { Arc::from_raw(ptr) }) diff --git a/crates/weaveffi/src/abi/spawn.rs b/crates/weaveffi/src/abi/spawn.rs new file mode 100644 index 00000000..bbb43943 --- /dev/null +++ b/crates/weaveffi/src/abi/spawn.rs @@ -0,0 +1,709 @@ +//! The pluggable executor async exports run on, and the launcher driver that +//! keeps the completion promise. +//! +//! An exported `async fn` lowers to a launcher that returns immediately and a +//! completion callback that fires when the future resolves. Something has to +//! drive the future in between: +//! +//! * By default, a fixed pool of worker threads (one per available core, at +//! least two) started lazily on the first launch. Workers share one run +//! queue, and a future's waker puts it back on the queue, so a launch +//! never costs a thread. The pool has no reactor, and its workers are few: +//! a future that blocks a worker for a long time (synchronous I/O, a long +//! computation) holds up every other call, so such work belongs on a +//! runtime of its own (the `tokio` feature, or [`set_spawner`]). +//! * With the `tokio` feature, a Tokio runtime: the current one when the +//! launcher is called from inside a runtime, otherwise a multi-thread +//! runtime the library creates on first use. +//! * On `wasm32`, which has no threads, the future is polled inline before +//! the launcher returns. A future still pending once nothing wakes it +//! completes with [`GENERIC_ERROR_CODE`] +//! ("async function suspended with no executor on wasm32") instead of +//! spinning. +//! +//! A producer installs its own [`Spawner`] once at startup with +//! [`set_spawner`], which overrides every default: +//! +//! ``` +//! # fn spawn_on_my_runtime(_f: weaveffi::abi::BoxFuture) {} +//! // e.g. `|fut| { tokio_handle.spawn(fut); }` +//! let _ = weaveffi::abi::set_spawner(|fut| spawn_on_my_runtime(fut)); +//! ``` +//! +//! Generated launchers call [`launch_async`], which runs the launcher's own +//! work (lifting the inputs) under `catch_unwind` and hands the producer's +//! future to [`run_async`]. Every future handed to a spawner is +//! `Send + 'static` and already wrapped so a panic inside it is caught and +//! reported through the completion callback; a spawner never observes an +//! unwinding future. A spawner may also drop a future without finishing it +//! (a runtime shutting down, say): the completion still fires, with +//! [`CANCELLED_ERROR_CODE`](crate::abi::CANCELLED_ERROR_CODE). A spawner that +//! panics, or a default executor that can't start, completes the call with +//! [`GENERIC_ERROR_CODE`]. + +use std::future::Future; +use std::pin::Pin; +use std::sync::{Arc, Mutex, OnceLock, PoisonError}; +use std::task::{Context, Poll, Wake, Waker}; + +use crate::abi::marshal::Sentinel; +use crate::abi::{CancelToken, Cancellable, FfiError, GENERIC_ERROR_CODE}; + +/// The type-erased future a [`Spawner`] receives. +pub type BoxFuture = Pin + Send + 'static>>; + +/// An executor hook that drives the futures produced by async exports. +/// +/// Implemented for any `Fn(BoxFuture) + Send + Sync + 'static`, so a closure +/// that forwards to a runtime handle is enough. +pub trait Spawner: Send + Sync + 'static { + /// Schedule `fut` to run to completion. Must not block the caller: the + /// launcher that invokes it is on the consumer's thread. + fn spawn(&self, fut: BoxFuture); +} + +impl Spawner for F +where + F: Fn(BoxFuture) + Send + Sync + 'static, +{ + fn spawn(&self, fut: BoxFuture) { + self(fut); + } +} + +static SPAWNER: OnceLock> = OnceLock::new(); + +/// The error returned when [`set_spawner`] is called a second time. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SpawnerAlreadySet; + +impl std::fmt::Display for SpawnerAlreadySet { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("a WeaveFFI spawner has already been installed") + } +} + +impl std::error::Error for SpawnerAlreadySet {} + +/// Install the process-wide spawner async exports run on. +/// +/// Call once, before the first async export is launched (an initialization +/// export or a library constructor is the natural place). Until it's called, +/// and forever after if it never is, the default executor is used (see the +/// [module docs](self)). +/// +/// # Errors +/// +/// Returns [`SpawnerAlreadySet`] if a spawner was installed already; the +/// first one wins. +pub fn set_spawner(spawner: impl Spawner) -> Result<(), SpawnerAlreadySet> { + SPAWNER + .set(Box::new(spawner)) + .map_err(|_| SpawnerAlreadySet) +} + +/// Why the executor didn't take a future. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum SpawnError { + /// The spawner panicked, or the default executor couldn't start. + Failed(String), + /// `wasm32` only: the future was still pending after nothing was left + /// to wake it. + Suspended, +} + +impl std::fmt::Display for SpawnError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Failed(why) => write!(f, "could not start the async call: {why}"), + Self::Suspended => f.write_str("async function suspended with no executor on wasm32"), + } + } +} + +impl std::error::Error for SpawnError {} + +/// Run `fut` on the installed spawner, or on the default executor. +/// +/// # Errors +/// +/// Returns [`SpawnError`] when the spawner panics (the future is dropped), +/// when the default executor can't start, or (on `wasm32`) when the future +/// is still pending once nothing wakes it. +pub fn spawn(fut: impl Future + Send + 'static) -> Result<(), SpawnError> { + let fut: BoxFuture = Box::pin(fut); + match SPAWNER.get() { + Some(spawner) => std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + spawner.spawn(fut); + })) + .map_err(|payload| { + SpawnError::Failed(format!( + "the spawner panicked: {}", + crate::abi::panic_message(&*payload) + )) + }), + None => default_spawn(fut), + } +} + +#[cfg(all(not(target_arch = "wasm32"), not(feature = "tokio")))] +fn default_spawn(fut: BoxFuture) -> Result<(), SpawnError> { + pool::spawn(fut) +} + +#[cfg(all(not(target_arch = "wasm32"), feature = "tokio"))] +fn default_spawn(fut: BoxFuture) -> Result<(), SpawnError> { + if let Ok(handle) = tokio::runtime::Handle::try_current() { + drop(handle.spawn(fut)); + return Ok(()); + } + static RUNTIME: OnceLock> = OnceLock::new(); + match RUNTIME.get_or_init(|| { + tokio::runtime::Builder::new_multi_thread() + .thread_name("weaveffi-async") + .enable_all() + .build() + .map_err(|e| e.to_string()) + }) { + Ok(rt) => { + drop(rt.spawn(fut)); + Ok(()) + } + Err(why) => Err(SpawnError::Failed(format!( + "could not start the Tokio runtime: {why}" + ))), + } +} + +#[cfg(target_arch = "wasm32")] +fn default_spawn(mut fut: BoxFuture) -> Result<(), SpawnError> { + use std::sync::atomic::{AtomicBool, Ordering}; + + struct Woken(AtomicBool); + impl Wake for Woken { + fn wake(self: Arc) { + self.0.store(true, Ordering::SeqCst); + } + fn wake_by_ref(self: &Arc) { + self.0.store(true, Ordering::SeqCst); + } + } + + let woken = Arc::new(Woken(AtomicBool::new(false))); + let waker = Waker::from(Arc::clone(&woken)); + let mut cx = Context::from_waker(&waker); + loop { + woken.0.store(false, Ordering::SeqCst); + if fut.as_mut().poll(&mut cx).is_ready() { + return Ok(()); + } + if !woken.0.load(Ordering::SeqCst) { + // Nothing can wake it on a thread-less target: dropping it fires + // the completion through `run_async`'s guard. + drop(fut); + return Err(SpawnError::Suspended); + } + } +} + +/// The default native executor: a fixed pool of worker threads sharing one +/// run queue, started on the first launch. +#[cfg(all(not(target_arch = "wasm32"), not(feature = "tokio")))] +mod pool { + use std::collections::VecDeque; + use std::sync::atomic::{AtomicBool, Ordering}; + use std::sync::{Arc, Condvar, Mutex, OnceLock, PoisonError}; + use std::task::{Context, Wake, Waker}; + + use super::{BoxFuture, SpawnError}; + + struct Pool { + queue: Mutex>>, + ready: Condvar, + } + + struct Task { + future: Mutex>, + /// Set while the task sits on the queue, so a burst of wakes queues + /// it once. + queued: AtomicBool, + pool: &'static Pool, + } + + impl Task { + fn schedule(self: Arc) { + if !self.queued.swap(true, Ordering::AcqRel) { + let pool = self.pool; + pool.queue + .lock() + .unwrap_or_else(PoisonError::into_inner) + .push_back(self); + pool.ready.notify_one(); + } + } + } + + impl Wake for Task { + fn wake(self: Arc) { + self.schedule(); + } + fn wake_by_ref(self: &Arc) { + Arc::clone(self).schedule(); + } + } + + impl Pool { + fn work(&'static self) { + loop { + let task = { + let mut queue = self.queue.lock().unwrap_or_else(PoisonError::into_inner); + loop { + match queue.pop_front() { + Some(task) => break task, + None => { + queue = self + .ready + .wait(queue) + .unwrap_or_else(PoisonError::into_inner); + } + } + } + }; + // Clear the flag before polling, so a wake during the poll + // queues the task again. + task.queued.store(false, Ordering::Release); + let mut slot = task.future.lock().unwrap_or_else(PoisonError::into_inner); + if let Some(fut) = slot.as_mut() { + let waker = Waker::from(Arc::clone(&task)); + // The launcher wraps every future in `CatchUnwind`, so + // polling never unwinds into the worker. + if fut + .as_mut() + .poll(&mut Context::from_waker(&waker)) + .is_ready() + { + *slot = None; + } + } + } + } + } + + /// The number of workers the pool starts. + pub(super) fn size() -> usize { + std::thread::available_parallelism() + .map_or(2, std::num::NonZero::get) + .max(2) + } + + fn pool() -> Result<&'static Pool, SpawnError> { + static POOL: OnceLock> = OnceLock::new(); + POOL.get_or_init(|| { + let pool: &'static Pool = Box::leak(Box::new(Pool { + queue: Mutex::new(VecDeque::new()), + ready: Condvar::new(), + })); + let started = (0..size()) + .filter(|i| { + std::thread::Builder::new() + .name(format!("weaveffi-async-{i}")) + .spawn(move || pool.work()) + .is_ok() + }) + .count(); + if started == 0 { + Err("no executor thread could be started".to_string()) + } else { + Ok(pool) + } + }) + .clone() + .map_err(SpawnError::Failed) + } + + pub(super) fn spawn(fut: BoxFuture) -> Result<(), SpawnError> { + let pool = pool()?; + Arc::new(Task { + future: Mutex::new(Some(fut)), + queued: AtomicBool::new(false), + pool, + }) + .schedule(); + Ok(()) + } +} + +/// The number of worker threads the default executor runs: one per +/// available core, at least two. (With the `tokio` feature or on `wasm32` +/// the default executor is something else, and this is `0`.) +#[must_use] +pub fn default_pool_size() -> usize { + #[cfg(all(not(target_arch = "wasm32"), not(feature = "tokio")))] + { + pool::size() + } + #[cfg(any(target_arch = "wasm32", feature = "tokio"))] + { + 0 + } +} + +/// Drive a future to completion on the current thread, blocking until it +/// resolves. +/// +/// This is a minimal, dependency-free executor for tests and examples. It +/// parks the thread between polls and wakes on `Waker::wake`, so a future +/// that yields (for example, one awaiting a channel woken from another +/// thread) makes progress without busy-spinning. +/// +/// # Examples +/// +/// ``` +/// let n = weaveffi::abi::block_on(async { 1 + 2 }); +/// assert_eq!(n, 3); +/// ``` +pub fn block_on(fut: F) -> F::Output { + struct ThreadWaker(std::thread::Thread); + impl Wake for ThreadWaker { + fn wake(self: Arc) { + self.0.unpark(); + } + fn wake_by_ref(self: &Arc) { + self.0.unpark(); + } + } + + let mut fut = std::pin::pin!(fut); + let waker = Waker::from(Arc::new(ThreadWaker(std::thread::current()))); + let mut cx = Context::from_waker(&waker); + loop { + match fut.as_mut().poll(&mut cx) { + Poll::Ready(out) => return out, + Poll::Pending => std::thread::park(), + } + } +} + +/// The completion an async call owes, shared between the launcher and the +/// spawned task so it fires exactly once whichever side finishes the call. +struct Completion { + state: Mutex>, +} + +struct CompletionState { + complete: Option, + /// The launcher is still inside the spawner. + launching: bool, + /// The task was dropped unfinished while launching. + dropped: bool, +} + +impl Completion { + fn lock(&self) -> std::sync::MutexGuard<'_, CompletionState> { + self.state.lock().unwrap_or_else(PoisonError::into_inner) + } +} + +/// The task's handle on its [`Completion`]: fires it with the result, or +/// (when dropped unfinished) with the cancelled code, unless the launcher is +/// still inside the spawner, in which case the launcher decides. +struct Guard)> { + shared: Arc>, + _result: std::marker::PhantomData, +} + +impl)> Guard { + fn complete(self, result: Result) { + let complete = self.shared.lock().complete.take(); + if let Some(c) = complete { + c(result); + } + } +} + +impl)> Drop for Guard { + fn drop(&mut self) { + let complete = { + let mut state = self.shared.lock(); + if state.launching { + state.dropped = true; + None + } else { + state.complete.take() + } + }; + if let Some(c) = complete { + c(Err(FfiError::cancelled())); + } + } +} + +/// Run an async export to completion and deliver its result exactly once. +/// +/// `fut` is the producer's work, already mapped to `Result` (a +/// domain error becomes its [`FfiError`]); `complete` lowers the outcome +/// into the C completion callback's slots and invokes it. The runtime calls +/// `complete` exactly once: +/// +/// * with `fut`'s result when it resolves; +/// * with a [`PANIC_ERROR_CODE`](crate::abi::PANIC_ERROR_CODE) error when +/// polling it panics; +/// * with [`CANCELLED_ERROR_CODE`](crate::abi::CANCELLED_ERROR_CODE) as soon +/// as `cancel` fires, after dropping `fut`, or when the executor drops the +/// task before it resolves; +/// * with [`GENERIC_ERROR_CODE`] when the executor can't take the task (see +/// [`spawn`]). +pub fn run_async(fut: Fut, cancel: Option, complete: C) +where + T: 'static, + Fut: Future> + Send + 'static, + C: FnOnce(Result) + Send + 'static, +{ + let shared = Arc::new(Completion { + state: Mutex::new(CompletionState { + complete: Some(complete), + launching: true, + dropped: false, + }), + }); + let guard = Guard { + shared: Arc::clone(&shared), + _result: std::marker::PhantomData, + }; + let task = async move { + let guard = guard; + let caught = CatchUnwind::new(fut); + let outcome = match cancel { + Some(token) => Cancellable::new(caught, token).await, + None => Some(caught.await), + }; + guard.complete(match outcome { + None => Err(FfiError::cancelled()), + Some(Ok(result)) => result, + Some(Err(payload)) => Err(FfiError::from_panic(&*payload)), + }); + }; + let spawned = spawn(task); + let (complete, error) = { + let mut state = shared.lock(); + state.launching = false; + match spawned { + Err(e) => ( + state.complete.take(), + FfiError::new(GENERIC_ERROR_CODE, &e.to_string()), + ), + Ok(()) if state.dropped => (state.complete.take(), FfiError::cancelled()), + Ok(()) => (None, FfiError::default()), + } + }; + if let Some(c) = complete { + c(Err(error)); + } +} + +/// The body of every generated async launcher. +/// +/// `prepare` runs on the caller's thread under `catch_unwind`: it lifts the +/// launcher's inputs into owned values and returns the producer's future +/// (with the call's cancel token, for a `cancellable` function), or the +/// first input that failed to lift. `lower` turns the producer's value into +/// the completion callback's result slots, and `deliver` invokes the +/// callback with a heap-boxed error (or null) and those slots; on any +/// failure the slots are [`Sentinel::sentinel`] values. `deliver` runs +/// exactly once, from any thread, possibly before this returns. +pub fn launch_async( + prepare: impl FnOnce() -> Result<(Fut, Option), FfiError>, + lower: impl FnOnce(T) -> S + Send + 'static, + deliver: impl FnOnce(*mut FfiError, S) + Send + 'static, +) where + T: 'static, + S: Sentinel, + Fut: Future> + Send + 'static, +{ + let finish = move |result: Result| { + let lowered = result.and_then(|value| { + // Lowering can panic (an oversized buffer, say) and must not + // unwind past the exactly-once promise. + std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| lower(value))) + .map_err(|payload| FfiError::from_panic(&*payload)) + }); + match lowered { + Ok(slots) => deliver(std::ptr::null_mut(), slots), + Err(e) => deliver(crate::abi::boxed_error(e), S::sentinel()), + } + }; + match std::panic::catch_unwind(std::panic::AssertUnwindSafe(prepare)) { + Ok(Ok((fut, cancel))) => run_async(fut, cancel, finish), + Ok(Err(e)) => finish(Err(e)), + Err(payload) => finish(Err(FfiError::from_panic(&*payload))), + } +} + +/// A future adapter that catches a panic from the inner future's `poll` and +/// resolves to `Err(payload)` instead of unwinding into the executor. +/// +/// This is how [`run_async`] keeps the "callback fires exactly once" promise +/// even when the producer's future panics: it awaits +/// `CatchUnwind::new(user_future)` and reports an `Err` through the +/// completion callback with the reserved panic code. +pub struct CatchUnwind { + inner: Option>>, +} + +impl CatchUnwind { + /// Wrap `fut`. + pub fn new(fut: F) -> Self { + Self { + inner: Some(Box::pin(fut)), + } + } +} + +impl Future for CatchUnwind { + type Output = Result>; + + fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll { + let this = &mut *self; + let Some(inner) = this.inner.as_mut() else { + // Polled after completion; a well-behaved executor never does + // this, and there is nothing left to drive. + return Poll::Pending; + }; + match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| inner.as_mut().poll(cx))) { + Ok(Poll::Pending) => Poll::Pending, + Ok(Poll::Ready(out)) => { + this.inner = None; + Poll::Ready(Ok(out)) + } + Err(payload) => { + this.inner = None; + Poll::Ready(Err(payload)) + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::mpsc; + + #[test] + fn default_spawner_runs_the_future() { + let (tx, rx) = mpsc::channel(); + spawn(async move { + tx.send(7).unwrap(); + }) + .unwrap(); + assert_eq!(rx.recv_timeout(std::time::Duration::from_secs(30)), Ok(7)); + } + + #[test] + fn catch_unwind_reports_panics_and_values() { + let ok = block_on(CatchUnwind::new(async { 5 })); + assert_eq!(ok.unwrap(), 5); + let err = block_on(CatchUnwind::new(async { + if true { + panic!("boom"); + } + 1 + })); + let payload = err.unwrap_err(); + assert_eq!(crate::abi::panic_message(&*payload), "boom"); + } + + fn run( + fut: impl Future> + Send + 'static, + cancel: Option, + ) -> mpsc::Receiver> { + let (tx, rx) = mpsc::channel(); + run_async(fut, cancel, move |r: Result| { + tx.send(r.map_err(|e| e.code)).unwrap(); + }); + rx + } + + const WAIT: std::time::Duration = std::time::Duration::from_secs(30); + + #[test] + fn run_async_delivers_values_errors_and_panics() { + assert_eq!(run(async { Ok(4) }, None).recv_timeout(WAIT), Ok(Ok(4))); + let rx = run(async { Err(FfiError::new(9, "domain")) }, None); + assert_eq!(rx.recv_timeout(WAIT), Ok(Err(9))); + let rx = run( + async { + if true { + panic!("bug"); + } + Ok(0) + }, + None, + ); + assert_eq!(rx.recv_timeout(WAIT), Ok(Err(crate::abi::PANIC_ERROR_CODE))); + } + + #[test] + fn run_async_completes_cancelled_when_the_token_fires() { + let raw = crate::abi::cancel_token_create(); + let token = unsafe { CancelToken::from_raw(raw) }; + let rx = run( + async { + std::future::pending::<()>().await; + Ok(1) + }, + Some(token), + ); + unsafe { crate::abi::cancel_token_cancel(raw) }; + unsafe { crate::abi::cancel_token_destroy(raw) }; + assert_eq!( + rx.recv_timeout(WAIT), + Ok(Err(crate::abi::CANCELLED_ERROR_CODE)) + ); + assert!(rx + .recv_timeout(std::time::Duration::from_millis(50)) + .is_err()); + } + + #[test] + fn a_dropped_guard_completes_cancelled_once() { + let (tx, rx) = mpsc::channel(); + let shared = Arc::new(Completion { + state: Mutex::new(CompletionState { + complete: Some(move |r: Result<(), FfiError>| { + tx.send(r.map_err(|e| e.code)).unwrap(); + }), + launching: false, + dropped: false, + }), + }); + drop(Guard { + shared, + _result: std::marker::PhantomData, + }); + assert_eq!(rx.recv(), Ok(Err(crate::abi::CANCELLED_ERROR_CODE))); + assert!(rx.recv().is_err()); + } + + type Prepared = Result< + ( + std::future::Ready>, + Option, + ), + FfiError, + >; + + #[test] + fn a_panicking_prepare_completes_with_the_panic_code() { + let (tx, rx) = mpsc::channel(); + launch_async( + || -> Prepared { panic!("lifting blew up") }, + |v: i32| v, + move |err, v| { + let code = unsafe { err.as_ref() }.map_or(0, |e| e.code); + unsafe { crate::abi::error_free(err) }; + tx.send((code, v)).unwrap(); + }, + ); + assert_eq!(rx.recv_timeout(WAIT), Ok((crate::abi::PANIC_ERROR_CODE, 0))); + } +} diff --git a/crates/weaveffi/src/lib.rs b/crates/weaveffi/src/lib.rs index 63375156..339327ca 100644 --- a/crates/weaveffi/src/lib.rs +++ b/crates/weaveffi/src/lib.rs @@ -42,9 +42,10 @@ //! weaveffi::export_runtime!(); //! ``` //! -//! The same annotated source is what `weaveffi generate path/to/lib.rs` reads to -//! emit the IDL, header, and bindings, so the producer and the bindings cannot -//! drift: they are two views of one parse. +//! The macro also embeds the API in the built library, which is what +//! `weaveffi generate` (run in the crate) reads to emit the header and +//! bindings, so the producer and the bindings can't drift: the bindings +//! describe exactly what the library was compiled with. //! //! # What you get //! @@ -58,10 +59,12 @@ //! as `&T` or `Arc`, return one as `Self`, `T`, or `Arc`. //! * [`macro@error`] - the module's error domain. //! * [`macro@callback_interface`] - a trait the consumer implements; accept -//! one as `Arc`. +//! one as `Arc` (or `Option>`). Its methods +//! return `Result`; [`macro@throws`] lets one report the +//! module's domain errors. //! * [`macro@cancellable`] - mark an `async fn` as accepting a cancel token. -//! * [`set_spawner`] - install the executor async exports run on (Tokio, for -//! example); the default drives each future on its own thread. +//! * [`set_spawner`] - install the executor async exports run on; the default +//! is a small pool of worker threads, or Tokio with the `tokio` feature. //! * [`export_runtime!`] - export the runtime symbols (memory, errors, cancel //! tokens, ABI version) under the crate's prefix, once per library. //! * [`abi`] - the C ABI runtime: the error struct, memory helpers, and the @@ -72,34 +75,43 @@ //! * `leak-check` counts live objects, callbacks, iterators, cancel tokens, //! and returned allocations, reported by `{prefix}_debug_live` so a test //! harness can assert a consumer released everything. Off by default. +//! * `tokio` runs exported `async fn`s on Tokio: the current runtime when a +//! launcher is called from inside one, otherwise a multi-thread runtime +//! created on first use. [`set_spawner`] still overrides it. Off by +//! default. #![deny(missing_docs)] +#![warn(clippy::missing_errors_doc)] +#![warn(clippy::missing_panics_doc)] +#![warn(clippy::missing_safety_doc)] +#![warn(clippy::doc_markdown)] -/// The stable C ABI runtime: error type, cancel tokens, memory management, and -/// the `lift_*`/`lower_*` marshalling converters the macro expansion calls. -/// -/// Re-exported from [`weaveffi_abi`] so producers depend on a single `weaveffi` -/// crate; the generated thunks reference these items as `::weaveffi::abi::*`. -pub use weaveffi_abi as abi; +pub mod abi; /// An owned, lazily-pulled iterator returned by a producer function whose IDL /// return type is `iter`. Construct one from any iterator with -/// [`Iter::new`](weaveffi_abi::Iter::new); the [`macro@module`] expansion turns +/// [`Iter::new`](abi::Iter::new); the [`macro@module`] expansion turns /// it into the opaque iterator handle the generated bindings consume. -pub use weaveffi_abi::Iter; +pub use abi::Iter; /// The producer's handle on a consumer's cancel token, accepted as the final /// parameter of a `#[weaveffi::cancellable]` `async fn`. When the consumer /// cancels, the runtime drops the function's future and completes the call /// with the cancelled code; poll -/// [`is_cancelled`](weaveffi_abi::CancelToken::is_cancelled) only for +/// [`is_cancelled`](abi::CancelToken::is_cancelled) only for /// cooperative cleanup (work on other threads, say). -pub use weaveffi_abi::CancelToken; +pub use abi::CancelToken; -/// A consumer's callback-interface implementation failed. A callback trait -/// method declared to return `Result` receives the failure -/// as an `Err` instead of unwinding. -pub use weaveffi_abi::ForeignError; +/// A consumer's callback-interface implementation failed. Every callback +/// trait method returns `Result`, so the failure arrives as +/// an `Err`; a method marked `#[weaveffi::throws]` can decode a declared +/// domain error from it with [`ForeignError::domain`](abi::ForeignError::domain). +pub use abi::ForeignError; + +/// A module's error domain: the trait the `#[weaveffi::error]` expansion +/// implements so [`ForeignError::domain`](abi::ForeignError::domain) can +/// decode a typed error a callback reported. +pub use abi::ErrorDomain; /// Maps a producer error onto the ABI's `(code, message)` pair. A fallible /// `#[weaveffi::export]` function reports `Err(e)` through its trailing @@ -107,22 +119,23 @@ pub use weaveffi_abi::ForeignError; /// generic code `-1` out of the box, while a `#[weaveffi::error]` enum (or a /// manual [`ErrorReport`] impl) surfaces the named codes of an IDL error /// domain, with its `Display` output as the message. -pub use weaveffi_abi::ErrorReport; +pub use abi::ErrorReport; /// Install the process-wide executor that exported `async fn`s run on. Call it /// once at startup (before the first async export is launched) to hand futures -/// to a runtime such as Tokio; until then, and if never called, each future is -/// driven to completion on its own thread. -pub use weaveffi_abi::set_spawner; +/// to a runtime of your choice; until then, and if never called, futures run +/// on the default executor (a small worker pool, or Tokio with the `tokio` +/// feature). +pub use abi::set_spawner; /// The executor hook [`set_spawner`] accepts: anything callable as /// `Fn(BoxFuture)` that is `Send + Sync + 'static`. -pub use weaveffi_abi::Spawner; +pub use abi::Spawner; /// The type-erased `Send + 'static` future a [`Spawner`] receives. -pub use weaveffi_abi::BoxFuture; +pub use abi::BoxFuture; pub use weaveffi_macros::{ callback_interface, cancellable, enumeration, error, export, export_runtime, interface, module, - record, + record, throws, }; diff --git a/crates/weaveffi/tests/abi_version.rs b/crates/weaveffi/tests/abi_version.rs index e11bac5c..93b5bf77 100644 --- a/crates/weaveffi/tests/abi_version.rs +++ b/crates/weaveffi/tests/abi_version.rs @@ -1,17 +1,10 @@ -//! The ABI revision is declared twice on purpose: once in `weaveffi-abi` -//! (which producers link) and once in `weaveffi-gen` (which generators -//! read). The runtime crate must not depend on the generator stack, so -//! this test is what keeps the two numbers equal. +//! `export_runtime!` exports the runtime's ABI revision under the crate's +//! prefix (this test crate is `abi_version`). #![allow(unsafe_code)] weaveffi::export_runtime!(); -#[test] -fn runtime_and_generator_abi_revisions_agree() { - assert_eq!(weaveffi::abi::ABI_VERSION, weaveffi_gen::cabi::ABI_VERSION); -} - #[test] fn exported_thunk_reports_the_runtime_revision() { assert_eq!(abi_version_abi_version(), weaveffi::abi::ABI_VERSION); diff --git a/crates/weaveffi-abi/tests/buffer_proptest.rs b/crates/weaveffi/tests/buffer_proptest.rs similarity index 90% rename from crates/weaveffi-abi/tests/buffer_proptest.rs rename to crates/weaveffi/tests/buffer_proptest.rs index 7021e02c..e69511c5 100644 --- a/crates/weaveffi-abi/tests/buffer_proptest.rs +++ b/crates/weaveffi/tests/buffer_proptest.rs @@ -12,10 +12,19 @@ //! A third family checks that arbitrary byte soup never panics the reader: //! it either decodes or returns `BufferDecodeError`. +#![allow(unsafe_code)] + use std::collections::BTreeMap; use proptest::prelude::*; -use weaveffi_abi::{decode_value, encode_value, BufferReader, BufferValue, BufferWriter}; +use weaveffi::abi::{encode_value, BufferDecodeError, BufferReader, BufferValue, BufferWriter}; + +/// Decode a buffer of a type without object tokens, which no input can make +/// unsound. +fn decode_value(data: &[u8]) -> Result { + // SAFETY: the types these tests decode carry no object tokens. + unsafe { weaveffi::abi::decode_value(data) } +} /// A hand-rolled record covering every scalar, the two byte-oriented shapes, /// and the three containers, nested two deep. Mirrors what @@ -73,7 +82,18 @@ impl BufferValue for Composite { self.deep.write_value(w); } - fn read_value(r: &mut BufferReader<'_>) -> Result { + unsafe fn read_value( + r: &mut BufferReader<'_>, + ) -> Result { + // SAFETY: none of these types carries an object token. + unsafe { Self::read_fields(r) } + } +} + +impl Composite { + unsafe fn read_fields( + r: &mut BufferReader<'_>, + ) -> Result { let flag = bool::read_value(r)?; let tiny = i8::read_value(r)?; let byte = u8::read_value(r)?; @@ -240,8 +260,9 @@ proptest! { let mut bytes = encode_value(&a); bytes.extend(encode_value(&b)); let mut r = BufferReader::new(&bytes); - let first = Composite::read_value(&mut r).expect("first"); - let second = Composite::read_value(&mut r).expect("second"); + // SAFETY: `Composite` carries no object tokens. + let first = unsafe { Composite::read_value(&mut r) }.expect("first"); + let second = unsafe { Composite::read_value(&mut r) }.expect("second"); r.expect_end().expect("exactly two values"); prop_assert!(same_composite(&a, &first)); prop_assert!(same_composite(&b, &second)); diff --git a/crates/weaveffi/tests/dropped_future.rs b/crates/weaveffi/tests/dropped_future.rs index b2edef65..ad811b06 100644 --- a/crates/weaveffi/tests/dropped_future.rs +++ b/crates/weaveffi/tests/dropped_future.rs @@ -20,7 +20,9 @@ pub mod work { } extern "C" fn done(ctx: *mut c_void, err: *mut FfiError, result: i32) { - let tx = unsafe { &*ctx.cast::>() }; + // Clone the sender before sending: the test may free the context as soon + // as the value arrives, which can be before `send` returns. + let tx = unsafe { &*ctx.cast::>() }.clone(); let code = if err.is_null() { 0 } else { @@ -39,7 +41,7 @@ fn a_dropped_future_completes_with_the_cancelled_code_once() { let ctx: *mut c_void = Box::into_raw(Box::new(tx)).cast(); unsafe { work::dropped_future_work_answer(done, ctx) }; assert_eq!( - rx.recv_timeout(Duration::from_secs(5)).unwrap(), + rx.recv_timeout(Duration::from_secs(30)).unwrap(), (abi::CANCELLED_ERROR_CODE, 0) ); assert!(rx.recv_timeout(Duration::from_millis(100)).is_err()); diff --git a/crates/weaveffi/tests/export_runtime.rs b/crates/weaveffi/tests/export_runtime.rs index 60f310bd..d6d77443 100644 --- a/crates/weaveffi/tests/export_runtime.rs +++ b/crates/weaveffi/tests/export_runtime.rs @@ -16,6 +16,8 @@ weaveffi::export_runtime!(); fn every_runtime_symbol_has_its_c_signature() { let _: extern "C" fn() -> u32 = export_runtime_abi_version; let _: unsafe extern "C" fn(*mut FfiError, i32, *const c_char) = export_runtime_error_set; + let _: unsafe extern "C" fn(*mut FfiError, *const u8, usize) = export_runtime_error_set_payload; + let _: extern "C" fn(usize) -> *mut u8 = export_runtime_alloc; let _: unsafe extern "C" fn(*mut FfiError) = export_runtime_error_clear; let _: unsafe extern "C" fn(*mut FfiError) = export_runtime_error_free; let _: unsafe extern "C" fn(*mut u8, usize) = export_runtime_free_bytes; @@ -25,7 +27,7 @@ fn every_runtime_symbol_has_its_c_signature() { export_runtime_cancel_token_is_cancelled; let _: unsafe extern "C" fn(*mut FfiCancelToken) = export_runtime_cancel_token_destroy; let _: extern "C" fn(i32) -> u64 = export_runtime_debug_live; - assert_eq!(export_runtime_abi_version(), 3); + assert_eq!(export_runtime_abi_version(), 4); } #[test] @@ -47,10 +49,31 @@ fn error_set_copies_and_clear_frees() { } #[test] -fn free_bytes_releases_returned_runs() { +fn error_set_payload_copies_the_run() { + let mut err = FfiError::default(); + let fields = abi::encode_value(&"key".to_string()); + unsafe { + export_runtime_error_set(&mut err, 1, ptr::null()); + export_runtime_error_set_payload(&mut err, fields.as_ptr(), fields.len()); + } + drop(fields); + assert_eq!( + unsafe { abi::decode_value::(err.payload()) }.unwrap(), + "key" + ); + unsafe { export_runtime_error_clear(&mut err) }; + assert!(err.payload_ptr.is_null()); +} + +#[test] +fn free_bytes_releases_returned_and_allocated_runs() { let (p, len) = abi::bytes_into_raw(vec![1, 2, 3]); + let staged = export_runtime_alloc(16); + assert!(!staged.is_null()); + assert!(export_runtime_alloc(0).is_null()); unsafe { export_runtime_free_bytes(p.cast_mut(), len); + export_runtime_free_bytes(staged, 16); export_runtime_free_bytes(ptr::null_mut(), 0); } } @@ -73,7 +96,9 @@ fn cancel_tokens_round_trip() { } #[test] -fn unknown_leak_kinds_read_zero() { - assert_eq!(export_runtime_debug_live(-1), 0); +fn debug_live_reports_whether_it_counts() { + // This crate's tests enable `leak-check`, so kind -1 reads 1. + assert_eq!(export_runtime_debug_live(-1), 1); + assert_eq!(export_runtime_debug_live(-2), 0); assert_eq!(export_runtime_debug_live(99), 0); } diff --git a/crates/weaveffi/tests/leak.rs b/crates/weaveffi/tests/leak.rs index 3b469a56..cf666719 100644 --- a/crates/weaveffi/tests/leak.rs +++ b/crates/weaveffi/tests/leak.rs @@ -41,7 +41,7 @@ pub mod pool { #[weaveffi::callback_interface] pub trait Watcher: Send + Sync { /// See a slot number. - fn seen(&self, n: i32); + fn seen(&self, n: i32) -> Result<(), weaveffi::ForeignError>; } /// Wrap a slot in a record (an object token inside a value buffer). @@ -65,7 +65,15 @@ pub mod pool { /// Call the watcher once. #[weaveffi::export] pub fn notify(watcher: Arc, n: i32) { - watcher.seen(n); + let _ = watcher.seen(n); + } + + /// Take a slot, a record holding one, and a watcher, all of which a + /// failing lift must still release. + #[weaveffi::export] + pub fn take_all(slot: &Slot, holder: Holder, watcher: Arc, label: &str) -> i32 { + let _ = (holder, watcher, label); + slot.n } /// Count up lazily. @@ -103,10 +111,19 @@ unsafe extern "C" fn free(ctx: *mut c_void) { drop(unsafe { Box::from_raw(ctx.cast::()) }); } -static VTABLE: pool::leak_pool_Watcher_vtable = pool::leak_pool_Watcher_vtable { seen, free }; +static VTABLE: pool::leak_pool_Watcher_vtable = pool::leak_pool_Watcher_vtable { + header: abi::VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free, + }, + seen, +}; extern "C" fn parked(ctx: *mut c_void, err: *mut FfiError) { - let tx = unsafe { &*ctx.cast::>() }; + // Clone the sender before sending: the test may free the context as soon + // as the value arrives, which can be before `send` returns. + let tx = unsafe { &*ctx.cast::>() }.clone(); let code = unsafe { (*err).code }; unsafe { abi::error_free(err) }; tx.send(code).unwrap(); @@ -145,6 +162,46 @@ fn every_counter_returns_to_zero() { let ctx = Box::into_raw(Box::new(0u8)).cast(); pool::leak_pool_notify(ctx, &VTABLE, 3, &mut err); + // Every input is lifted before the first failure is reported: an + // invalid string (or a null `slot`) still adopts and releases the + // record's token and the callback. + let slot = pool::leak_pool_Slot_new(2, &mut err); + let holder = abi::encode_value(&pool::Holder { + slot: abi::object_arc(slot).unwrap(), + }); + let bad = [0xffu8]; + let ctx = Box::into_raw(Box::new(0u8)).cast(); + let n = pool::leak_pool_take_all( + slot, + holder.as_ptr(), + holder.len(), + ctx, + &VTABLE, + bad.as_ptr(), + bad.len(), + &mut err, + ); + assert_eq!((n, err.code), (0, abi::MARSHAL_ERROR_CODE)); + abi::error_clear(&mut err); + let holder = abi::encode_value(&pool::Holder { + slot: abi::object_arc(slot).unwrap(), + }); + let ctx = Box::into_raw(Box::new(0u8)).cast(); + pool::leak_pool_take_all( + std::ptr::null(), + holder.as_ptr(), + holder.len(), + ctx, + &VTABLE, + bad.as_ptr(), + 0, + &mut err, + ); + assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); + abi::error_clear(&mut err); + pool::leak_pool_Slot_destroy(slot); + assert_eq!(live()[..2], [0, 0], "every adopted input was released"); + // Iterators: live until destroyed. let it = pool::leak_pool_count(2, &mut err); assert_eq!(live()[2], 1); @@ -159,15 +216,16 @@ fn every_counter_returns_to_zero() { leak_cancel_token_cancel(token); leak_cancel_token_destroy(token); assert_eq!( - rx.recv_timeout(Duration::from_secs(5)).unwrap(), + rx.recv_timeout(Duration::from_secs(30)).unwrap(), abi::CANCELLED_ERROR_CODE ); drop(Box::from_raw(ctx.cast::>())); } // The completion fires before the task's last locals drop, so give the - // executor thread a moment to finish unwinding its frame. - for _ in 0..100 { + // executor thread time to finish its poll (generously, so a loaded + // machine can't fail the test; a passing run waits a few milliseconds). + for _ in 0..1000 { if live() == [0; 5] { break; } diff --git a/crates/weaveffi/tests/runtime.rs b/crates/weaveffi/tests/runtime.rs index 974774a9..171e5f87 100644 --- a/crates/weaveffi/tests/runtime.rs +++ b/crates/weaveffi/tests/runtime.rs @@ -16,7 +16,10 @@ use std::time::Duration; use weaveffi::abi::{self, FfiError}; -const WAIT: Duration = Duration::from_secs(5); +/// How long a test waits for an async completion. Generous, so a loaded +/// machine can't turn a slow completion into a failure; a passing run never +/// waits this long. +const WAIT: Duration = Duration::from_secs(30); #[weaveffi::module] pub mod demo { @@ -190,35 +193,151 @@ pub mod dispatch { } } -/// Stands in for a `panic = "abort"` build, where a consumer callback failure -/// can't unwind: the producer records it with `defer_foreign_error` and keeps -/// running, and the thunk must report the recorded failure in place of the -/// producer's result. +/// Callback interfaces at full width: returns of every family adopted from +/// the consumer, a `throws` method decoded into the domain's typed error, +/// optional callback parameters, a type alias, and `#[cfg]`'d items. #[weaveffi::module] -pub mod deferred { - /// Record a foreign failure, then return normally with a value the - /// caller must never see. - #[weaveffi::export] - pub fn sync_then_fail(fail: bool) -> i32 { - if fail { - weaveffi::abi::defer_foreign_error(weaveffi::ForeignError { - code: weaveffi::abi::FOREIGN_ERROR_CODE, - message: "consumer said no".to_string(), - }); +pub mod rich { + use std::sync::Arc; + + use weaveffi::ForeignError; + + /// The lookup error domain; `Missing` carries the key. + #[weaveffi::error] + #[derive(Debug, PartialEq)] + #[repr(i32)] + pub enum LookupError { + /// no such key + Missing { + /// The key that wasn't found. + key: String, + } = 1, + /// the source is busy + Busy = 2, + } + + impl std::fmt::Display for LookupError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Missing { key } => write!(f, "missing {key}"), + Self::Busy => f.write_str("busy"), + } } - 77 } - /// The async twin of `sync_then_fail`. + /// A key, spelled through an alias the macro substitutes. + pub type Key = String; + + /// A record a callback returns. + #[weaveffi::record] + #[derive(Clone, Debug, PartialEq)] + pub struct Card { + /// The card's name. + pub name: String, + /// Its tags. + pub tags: Vec, + } + + /// An object a callback returns. + #[weaveffi::interface] + pub struct Token { + n: i64, + } + + impl Token { + /// Create a token. + pub fn new(n: i64) -> Self { + Self { n } + } + /// Read the number. + pub fn n(&self) -> i64 { + self.n + } + } + + // Compiled out: neither the member thunk nor its contract entry exist. + #[cfg(not(test))] + impl Token { + /// Never exported. + pub fn hidden(&self) -> i64 { + 0 + } + } + + /// A consumer-implemented source of values of every family. + #[weaveffi::callback_interface] + pub trait Source: Send + Sync { + /// A string, returned as a consumer-allocated run. + fn name(&self) -> Result; + /// Bytes, returned as a consumer-allocated run. + fn blob(&self) -> Result, ForeignError>; + /// A record, returned as a consumer-allocated value buffer. + fn card(&self, key: &str) -> Result; + /// An object, returned as one strong reference. + fn token(&self) -> Result, ForeignError>; + /// An optional object. + fn maybe_token(&self) -> Result>, ForeignError>; + /// Look up a key, failing with a `LookupError`. + #[weaveffi::throws] + fn lookup(&self, key: &str) -> Result; + } + + /// Call every value-returning method and describe the results. #[weaveffi::export] - pub async fn later_then_fail(fail: bool) -> i32 { - if fail { - weaveffi::abi::defer_foreign_error(weaveffi::ForeignError { - code: weaveffi::abi::FOREIGN_ERROR_CODE, - message: "consumer said no, later".to_string(), - }); + pub fn describe(source: Arc) -> Result { + let card = source.card("k")?; + let token = source.token()?; + Ok(format!( + "{} {:?} {}:{:?} {} {}", + source.name()?, + source.blob()?, + card.name, + card.tags, + token.n(), + source.maybe_token()?.is_some() + )) + } + + /// Look up `key` and describe the typed outcome. + #[weaveffi::export] + pub fn lookup_via(source: Arc, key: Key) -> String { + match source.lookup(&key) { + Ok(v) => format!("ok {v}"), + Err(e) => match e.domain::() { + Some(LookupError::Missing { key }) => format!("missing {key}"), + Some(LookupError::Busy) => "busy".to_string(), + None => format!("foreign {}: {}", e.code, e.message), + }, } - 88 + } + + /// Whether a source was passed (an optional callback parameter). + #[weaveffi::export] + pub fn has_source(source: Option>, label: &str) -> String { + format!("{label}:{}", source.is_some()) + } + + /// Present in every build of this test. + #[cfg(test)] + #[weaveffi::export] + pub fn always() -> i32 { + 1 + } + + /// Present in no build: no thunk, no contract entry. + #[cfg(not(test))] + #[weaveffi::export] + pub fn never() -> i32 { + 0 + } + + /// A submodule present in no build. + #[cfg(not(test))] + #[weaveffi::module] + pub mod gone { + /// Never exported. + #[weaveffi::export] + pub fn vanished() {} } } @@ -303,12 +422,53 @@ pub mod stream { pub fn squares(count: i32) -> weaveffi::Iter { weaveffi::Iter::new((0..count).map(|i| i * i)) } + + /// The handle `reentrant`'s iterator advances from inside its own + /// `next` (set by the test once the launcher returns it). + pub static REENTRANT_HANDLE: std::sync::atomic::AtomicUsize = + std::sync::atomic::AtomicUsize::new(0); + + /// Yield three elements, each the error code a nested `_next` on the + /// same handle reported from inside the producer's `next`. + #[weaveffi::export] + pub fn reentrant() -> weaveffi::Iter { + weaveffi::Iter::new((0..3).map(|_| { + let handle = REENTRANT_HANDLE.load(std::sync::atomic::Ordering::SeqCst); + let mut err = weaveffi::abi::FfiError::default(); + let mut item = 0; + unsafe { + runtime_stream_ReentrantIterator_next( + handle as *const weaveffi::abi::IterHandle, + &mut item, + &mut err, + ) + }; + err.code + })) + } } #[weaveffi::module] pub mod bus { use std::sync::{Arc, Mutex, PoisonError}; + use weaveffi::ForeignError; + + /// The bus's error domain. A subscriber failure propagates with its own + /// code; the domain lets the bus's calls throw. + #[weaveffi::error] + #[derive(Debug)] + pub enum BusError { + /// the bus is closed + Closed = 1, + } + + impl std::fmt::Display for BusError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("the bus is closed") + } + } + /// A message priority (a C-style enum crossing a callback boundary). #[weaveffi::enumeration] #[repr(i32)] @@ -334,14 +494,22 @@ pub mod bus { #[weaveffi::callback_interface] pub trait Subscriber: Send + Sync { /// Receive a message; returns the subscriber's running total. - fn on_message(&self, text: String, weight: i32, envelope: &Envelope) -> i64; + fn on_message( + &self, + text: String, + weight: i32, + envelope: &Envelope, + ) -> Result; /// Ask the subscriber how urgent it considers `weight`. - fn classify(&self, weight: i32) -> Priority; + fn classify(&self, weight: i32) -> Result; /// Inspect a shared object without retaining it. - fn on_ticker(&self, ticker: Arc, alt: Option>) -> bool; - /// Weigh a message; a consumer failure comes back as an `Err` - /// instead of unwinding. - fn weigh(&self, weight: i32) -> Result; + fn on_ticker( + &self, + ticker: Arc, + alt: Option>, + ) -> Result; + /// Weigh a message; a consumer failure comes back as an `Err`. + fn weigh(&self, weight: i32) -> Result; } /// A shared object handed to subscribers. @@ -383,12 +551,11 @@ pub mod bus { .push(subscriber); } - /// Publish to every subscriber, returning the sum of their totals. - /// - /// A subscriber failure unwinds through this frame like a panic, so - /// the subscriber list is snapshotted and the lock released before - /// any callback runs (the same discipline a panic-safe producer uses). - pub fn publish(&self, text: &str, weight: i32) -> i64 { + /// Publish to every subscriber, returning the sum of their totals. A + /// subscriber failure fails the call with the subscriber's code and + /// message. The list is snapshotted so no lock is held while a + /// callback runs. + pub fn publish(&self, text: &str, weight: i32) -> Result { let env = Envelope { seq: 1, topic: Some("t".to_string()), @@ -404,7 +571,7 @@ pub mod bus { } /// Publish asynchronously. - pub async fn publish_later(&self, text: String, weight: i32) -> i64 { + pub async fn publish_later(&self, text: String, weight: i32) -> Result { self.publish(&text, weight) } @@ -419,16 +586,19 @@ pub mod bus { /// Call the subscriber once directly without retaining it. #[weaveffi::export] - pub fn classify_once(subscriber: Arc, weight: i32) -> Priority { + pub fn classify_once( + subscriber: Arc, + weight: i32, + ) -> Result { subscriber.classify(weight) } /// Hand the subscriber a ticker object. #[weaveffi::export] - pub fn tick(subscriber: &Arc, value: i64) -> bool { + pub fn tick(subscriber: &Arc, value: i64) -> Result { let ticker = Arc::new(Ticker::new(value)); - subscriber.on_ticker(ticker.clone(), None) - && subscriber.on_ticker(ticker.clone(), Some(ticker)) + Ok(subscriber.on_ticker(ticker.clone(), None)? + && subscriber.on_ticker(ticker.clone(), Some(ticker))?) } /// Weigh through the subscriber and describe the outcome, handling a @@ -444,6 +614,13 @@ pub mod bus { #[weaveffi::module] pub mod tasks { + /// A tag for the thread running the call, to observe the executor. + #[weaveffi::export] + pub async fn thread_tag() -> String { + let t = std::thread::current(); + format!("{:?} {}", t.id(), t.name().unwrap_or("")) + } + /// The task module's error domain. #[weaveffi::error] #[derive(Debug)] @@ -516,7 +693,8 @@ fn decode_ret(ptr: *const u8, len: usize) -> T { "buffered return must not be null on success" ); let bytes = unsafe { std::slice::from_raw_parts(ptr, len) }; - let value = abi::decode_value(bytes).expect("well-formed value buffer"); + // SAFETY: a producer's return; its tokens are ours to adopt, once. + let value = unsafe { abi::decode_value(bytes) }.expect("well-formed value buffer"); unsafe { abi::free_bytes(ptr.cast_mut(), len) }; value } @@ -709,7 +887,7 @@ fn record_buffer_round_trip() { }; let bytes = abi::encode_value(&original); assert_eq!(bytes.len(), abi::BufferValue::encoded_len(&original)); - let back: demo::Point = abi::decode_value(&bytes).expect("round-trip"); + let back: demo::Point = unsafe { abi::decode_value(&bytes) }.expect("round-trip"); assert_eq!(back.x, 7); assert_eq!(back.label, "corner"); assert_eq!(back.nickname.as_deref(), Some("nw")); @@ -785,21 +963,84 @@ fn sibling_module_record_param_and_return() { assert_eq!(c2.label, "gadget"); } +/// FNV-1a 64, the contract table's hash. +fn fnv(data: &str) -> u64 { + data.bytes().fold(0xcbf2_9ce4_8422_2325, |h, b| { + (h ^ u64::from(b)).wrapping_mul(0x0000_0100_0000_01b3) + }) +} + +/// A top-level module's contract table, as a consumer reads it. +fn contract( + f: unsafe extern "C" fn(*mut usize) -> *const abi::ContractEntry, +) -> Vec { + let mut len = 0usize; + let ptr = unsafe { f(&mut len) }; + if len == 0 { + return Vec::new(); + } + unsafe { std::slice::from_raw_parts(ptr, len) }.to_vec() +} + #[test] -fn every_top_level_module_exports_a_checksum() { - let sums = [ - demo::runtime_demo_checksum(), - warehouse::runtime_warehouse_checksum(), - dispatch::runtime_dispatch_checksum(), - outer::runtime_outer_checksum(), - ]; - for (i, a) in sums.iter().enumerate() { - for b in &sums[i + 1..] { - assert_ne!(a, b, "different modules hash differently"); - } +fn contract_tables_hold_one_sorted_entry_per_declaration() { + let table = contract(demo::runtime_demo_contract); + assert!(table.windows(2).all(|w| w[0].id < w[1].id), "sorted by id"); + // demo: the error domain, Color, Point, and thirteen functions. + assert_eq!(table.len(), 16); + let add = table.iter().find(|e| e.id == fnv("demo.add")).unwrap(); + assert_eq!(add.hash, fnv("function add(a: i32, b: i32) -> i32")); + let point = table.iter().find(|e| e.id == fnv("demo.Point")).unwrap(); + assert_eq!( + point.hash, + fnv("record Point {x: i32, label: string, nickname: string?, color: Color}") + ); + let div = table + .iter() + .find(|e| e.id == fnv("demo.checked_div")) + .unwrap(); + assert_eq!( + div.hash, + fnv("function checked_div(a: i32, b: i32) -> i32 throws") + ); + + // Nested modules and interface members are in their root's table. + let outer = contract(outer::runtime_outer_contract); + for path in [ + "outer.Session", + "outer.Session.open", + "outer.inner.summarize", + ] { + assert!(outer.iter().any(|e| e.id == fnv(path)), "{path}"); } } +#[test] +fn contract_tables_follow_cfg() { + let table = contract(rich::runtime_rich_contract); + let has = |path: &str| table.iter().any(|e| e.id == fnv(path)); + assert!(has("rich.always")); + assert!(has("rich.Token.n")); + assert!(!has("rich.never"), "a cfg'd-out function has no entry"); + assert!( + !has("rich.Token.hidden"), + "nor a member of a cfg'd-out impl" + ); + assert!( + !has("rich.gone.vanished"), + "nor anything in a cfg'd-out module" + ); + let lookup = table + .iter() + .find(|e| e.id == fnv("rich.lookup_via")) + .unwrap(); + assert_eq!( + lookup.hash, + fnv("function lookup_via(source: Source, key: string) -> string"), + "the alias resolved to its target" + ); +} + #[test] fn map_param_and_return_are_buffered() { use std::collections::BTreeMap; @@ -836,7 +1077,7 @@ fn widget_optional_field_round_trips() { note: None, }, ] { - let back: build::Widget = abi::decode_value(&abi::encode_value(&w)).unwrap(); + let back: build::Widget = unsafe { abi::decode_value(&abi::encode_value(&w)) }.unwrap(); assert_eq!(back, w); } } @@ -858,11 +1099,11 @@ fn rich_enum_encodes_tag_then_fields() { }; let bytes = abi::encode_value(&labeled); assert_eq!(&bytes[..4], [2, 0, 0, 0]); - let back: geom::Shape = abi::decode_value(&bytes).unwrap(); + let back: geom::Shape = unsafe { abi::decode_value(&bytes) }.unwrap(); assert_eq!(back, labeled); // An out-of-range tag is a decode error, not a silent default. - assert!(abi::decode_value::(&[9, 0, 0, 0]).is_err()); + assert!(unsafe { abi::decode_value::(&[9, 0, 0, 0]) }.is_err()); } #[test] @@ -918,7 +1159,7 @@ fn iterator_scalar_elements() { } #[test] -fn concurrent_iterator_next_yields_each_element_once() { +fn concurrent_iterator_next_yields_each_element_once_or_reports_busy() { const N: i32 = 2000; let mut err = ok_err(); let iter = unsafe { stream::runtime_stream_squares(N, &mut err) }; @@ -934,6 +1175,11 @@ fn concurrent_iterator_next_yields_each_element_once() { let has = unsafe { stream::runtime_stream_SquaresIterator_next(iter, &mut item, &mut err) }; + // A `_next` racing another fails instead of blocking. + if err.code == abi::MARSHAL_ERROR_CODE { + std::thread::yield_now(); + continue; + } assert_eq!(err.code, 0); if has == 0 { break got; @@ -952,6 +1198,45 @@ fn concurrent_iterator_next_yields_each_element_once() { unsafe { stream::runtime_stream_SquaresIterator_destroy(iter) }; } +#[test] +fn a_reentrant_next_fails_instead_of_deadlocking() { + let mut err = ok_err(); + let iter = unsafe { stream::runtime_stream_reentrant(&mut err) }; + stream::REENTRANT_HANDLE.store(iter as usize, std::sync::atomic::Ordering::SeqCst); + let mut got = Vec::new(); + loop { + let mut item = 0i32; + if unsafe { stream::runtime_stream_ReentrantIterator_next(iter, &mut item, &mut err) } == 0 + { + break; + } + assert_eq!(err.code, 0); + got.push(item); + } + unsafe { stream::runtime_stream_ReentrantIterator_destroy(iter) }; + assert_eq!(got, vec![abi::MARSHAL_ERROR_CODE; 3]); +} + +#[test] +fn duplicate_map_keys_are_marshalling_errors() { + let mut err = ok_err(); + let mut w = abi::BufferWriter::new(); + w.write_len(2); + for v in [1i32, 2] { + w.write_string("same"); + w.write_i32(v); + } + let bytes = w.finish(); + let total = unsafe { maps::runtime_maps_total(bytes.as_ptr(), bytes.len(), &mut err) }; + assert_eq!(total, 0); + assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); + assert!( + message(&err).contains("duplicate map key"), + "{}", + message(&err) + ); +} + /// A consumer-side `Subscriber` implementation: the context is a heap-allocated /// `SubState`, the vtable is a process-wide static, exactly as a generated /// binding would do it. @@ -978,7 +1263,7 @@ mod consumer_subscriber { let state = unsafe { &*(ctx as *const SubState) }; let text = unsafe { abi::lift_str(text_ptr, text_len) }.unwrap(); let env: bus::Envelope = - abi::decode_value(unsafe { std::slice::from_raw_parts(envelope_ptr, envelope_len) }) + unsafe { abi::decode_value(std::slice::from_raw_parts(envelope_ptr, envelope_len)) } .unwrap(); *state.last_topic.lock().unwrap() = env.topic.clone(); if weight == state.fail_at { @@ -1032,13 +1317,32 @@ mod consumer_subscriber { } pub static VTABLE: bus::runtime_bus_Subscriber_vtable = bus::runtime_bus_Subscriber_vtable { + header: abi::VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free, + }, on_message, classify, on_ticker, weigh, - free, }; + /// A vtable whose `size` claims only the header: a consumer built from + /// an older contract. + pub static SHORT_VTABLE: bus::runtime_bus_Subscriber_vtable = + bus::runtime_bus_Subscriber_vtable { + header: abi::VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free, + }, + on_message, + classify, + on_ticker, + weigh, + }; + pub fn new_ctx(fail_at: i32, freed: &Arc) -> *mut c_void { Box::into_raw(Box::new(SubState { total: AtomicI64::new(0), @@ -1083,6 +1387,24 @@ fn callback_interface_sync_paths() { assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); } +#[test] +fn a_vtable_smaller_than_the_producers_is_rejected_and_released() { + use consumer_subscriber::{new_ctx, SHORT_VTABLE}; + use std::sync::atomic::{AtomicUsize, Ordering}; + let freed = Arc::new(AtomicUsize::new(0)); + let mut err = ok_err(); + let r = + unsafe { bus::runtime_bus_classify_once(new_ctx(-1, &freed), &SHORT_VTABLE, 9, &mut err) }; + assert_eq!(r, 0); + assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); + assert!( + message(&err).contains("regenerate the bindings"), + "{}", + message(&err) + ); + assert_eq!(freed.load(Ordering::SeqCst), 1, "the context was released"); +} + #[test] fn callback_result_methods_return_failures_as_values() { use consumer_subscriber::{new_ctx, VTABLE}; @@ -1158,7 +1480,9 @@ fn drop_ctx(ctx: *mut c_void) { } fn send(ctx: *mut c_void, err: *mut FfiError, value: T) { - let tx = unsafe { &*ctx.cast::>() }; + // Clone the sender before sending: the test may free the context as soon + // as the value arrives, which can be before `send` returns. + let tx = unsafe { &*ctx.cast::>() }.clone(); let (code, msg) = take_async_err(err); tx.send((code, msg, value)).unwrap(); } @@ -1219,7 +1543,9 @@ fn async_struct_result_completes_via_callback() { // The buffered result is owned by the consumer: decode it, then release // the producer allocation with `free_bytes`. extern "C" fn cb(ctx: *mut c_void, err: *mut FfiError, ptr: *const u8, len: usize) { - let tx = unsafe { &*ctx.cast::>() }; + // Clone the sender before sending: the test may free the context as soon + // as the value arrives, which can be before `send` returns. + let tx = unsafe { &*ctx.cast::>() }.clone(); let (code, msg) = take_async_err(err); let value = (!ptr.is_null()).then(|| { let r: tasks::TaskResult = decode_ret(ptr, len); @@ -1330,60 +1656,6 @@ fn cancel_then_destroy_races_the_launch_safely() { drop_ctx::(ctx); } -#[test] -fn deferred_foreign_error_replaces_a_sync_result() { - let mut err = ok_err(); - assert_eq!( - unsafe { deferred::runtime_deferred_sync_then_fail(false, &mut err) }, - 77 - ); - assert_eq!(err.code, 0); - - let r = unsafe { deferred::runtime_deferred_sync_then_fail(true, &mut err) }; - assert_eq!(r, 0, "the producer's value is discarded for the sentinel"); - assert_eq!(err.code, abi::FOREIGN_ERROR_CODE); - assert_eq!(message(&err), "consumer said no"); - - assert_eq!( - unsafe { deferred::runtime_deferred_sync_then_fail(false, &mut err) }, - 77 - ); - assert_eq!(err.code, 0, "a later call on the same thread is unaffected"); -} - -#[test] -fn deferring_outside_any_call_is_dropped() { - // No thunk is running on this thread, so the failure goes to stderr - // instead of leaking into the next call. - abi::defer_foreign_error(weaveffi::ForeignError { - code: abi::FOREIGN_ERROR_CODE, - message: "stray".into(), - }); - let mut err = ok_err(); - assert_eq!(unsafe { demo::runtime_demo_add(1, 2, &mut err) }, 3); - assert_eq!(err.code, 0); -} - -#[test] -fn deferred_foreign_error_fires_the_async_callback_once() { - let (tx, rx) = mpsc::channel(); - let ctx = new_ctx::(tx); - - unsafe { deferred::runtime_deferred_later_then_fail(false, on_i32, ctx) }; - assert_eq!(rx.recv_timeout(WAIT).unwrap(), (0, String::new(), 88)); - - unsafe { deferred::runtime_deferred_later_then_fail(true, on_i32, ctx) }; - let (code, message, result) = rx.recv_timeout(WAIT).unwrap(); - assert_eq!(code, abi::FOREIGN_ERROR_CODE); - assert_eq!(message, "consumer said no, later"); - assert_eq!(result, 0); - assert!( - rx.recv_timeout(Duration::from_millis(200)).is_err(), - "the completion callback fires exactly once" - ); - drop_ctx::(ctx); -} - /// Exercises nested-module codegen: the inner module's symbols carry the /// joined `outer_inner` path, and a nested function may reference an /// interface declared in its parent module via `super::`. @@ -2093,3 +2365,214 @@ fn error_payload_fields_cross_the_abi() { assert_eq!(err.code, 3002); assert!(err.payload_ptr.is_null()); } + +/// A consumer-side `Source`: returns of every family, allocated with the +/// exported `{prefix}_alloc`, and typed failures with a payload. +mod consumer_source { + use super::*; + use std::sync::atomic::{AtomicUsize, Ordering}; + + unsafe fn give(bytes: &[u8], out_ptr: *mut *mut u8, out_len: *mut usize) { + let run = super::runtime_alloc(bytes.len()); + unsafe { + if !bytes.is_empty() { + std::ptr::copy_nonoverlapping(bytes.as_ptr(), run, bytes.len()); + } + *out_ptr = run; + *out_len = bytes.len(); + } + } + + unsafe extern "C" fn name( + _ctx: *mut c_void, + out_ptr: *mut *mut u8, + out_len: *mut usize, + _err: *mut FfiError, + ) { + unsafe { give(b"src", out_ptr, out_len) }; + } + + unsafe extern "C" fn blob( + _ctx: *mut c_void, + out_ptr: *mut *mut u8, + out_len: *mut usize, + _err: *mut FfiError, + ) { + unsafe { give(&[1, 2], out_ptr, out_len) }; + } + + unsafe extern "C" fn card( + _ctx: *mut c_void, + key_ptr: *const u8, + key_len: usize, + out_ptr: *mut *mut u8, + out_len: *mut usize, + _err: *mut FfiError, + ) { + let key = unsafe { abi::lift_string(key_ptr, key_len) }.unwrap(); + let card = rich::Card { + name: format!("card-{key}"), + tags: vec!["t".into()], + }; + unsafe { give(&abi::encode_value(&card), out_ptr, out_len) }; + } + + unsafe extern "C" fn token(_ctx: *mut c_void, _err: *mut FfiError) -> *mut rich::Token { + let mut err = FfiError::default(); + unsafe { rich::runtime_rich_Token_new(5, &mut err) } + } + + unsafe extern "C" fn maybe_token(_ctx: *mut c_void, _err: *mut FfiError) -> *mut rich::Token { + std::ptr::null_mut() + } + + unsafe extern "C" fn lookup( + _ctx: *mut c_void, + key_ptr: *const u8, + key_len: usize, + out_err: *mut FfiError, + ) -> i64 { + let key = unsafe { abi::lift_string(key_ptr, key_len) }.unwrap(); + let fail = |code: i32, payload: Option<&str>| unsafe { + let msg = std::ffi::CString::new(format!("failed {key}")).unwrap(); + super::runtime_error_set(out_err, code, msg.as_ptr()); + if let Some(p) = payload { + let fields = abi::encode_value(&p.to_string()); + super::runtime_error_set_payload(out_err, fields.as_ptr(), fields.len()); + } + }; + match key.as_str() { + "missing" => fail(1, Some("missing")), + "busy" => fail(2, None), + "bogus" => fail(99, None), + "trap" => fail(-1, None), + _ => return 42, + } + 0 + } + + pub static FREED: AtomicUsize = AtomicUsize::new(0); + + unsafe extern "C" fn free(_ctx: *mut c_void) { + FREED.fetch_add(1, Ordering::SeqCst); + } + + pub static VTABLE: rich::runtime_rich_Source_vtable = rich::runtime_rich_Source_vtable { + header: abi::VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free, + }, + name, + blob, + card, + token, + maybe_token, + lookup, + }; +} + +#[test] +fn callback_returns_of_every_family_are_adopted() { + let mut err = ok_err(); + let mut len = 0usize; + let ptr = unsafe { + rich::runtime_rich_describe( + std::ptr::null_mut(), + &consumer_source::VTABLE, + &mut len, + &mut err, + ) + }; + assert_eq!(err.code, 0, "{}", message(&err)); + assert_eq!(take_string(ptr, len), r#"src [1, 2] card-k:["t"] 5 false"#); +} + +#[test] +fn throwing_callbacks_report_typed_domain_errors() { + let lookup = |key: &str| { + let mut err = ok_err(); + let mut len = 0usize; + let ptr = unsafe { + rich::runtime_rich_lookup_via( + std::ptr::null_mut(), + &consumer_source::VTABLE, + key.as_ptr(), + key.len(), + &mut len, + &mut err, + ) + }; + assert_eq!(err.code, 0); + take_string(ptr, len) + }; + assert_eq!(lookup("a"), "ok 42"); + assert_eq!( + lookup("missing"), + "missing missing", + "code 1 with its payload" + ); + assert_eq!(lookup("busy"), "busy"); + // An undeclared positive code, or any negative one, is a foreign failure. + assert_eq!(lookup("bogus"), "foreign -4: failed bogus"); + assert_eq!(lookup("trap"), "foreign -4: failed trap"); +} + +#[test] +fn optional_callback_parameters_accept_null() { + let label = "x"; + let mut err = ok_err(); + let mut len = 0usize; + let none = unsafe { + rich::runtime_rich_has_source( + std::ptr::null_mut(), + std::ptr::null(), + label.as_ptr(), + label.len(), + &mut len, + &mut err, + ) + }; + assert_eq!(take_string(none, len), "x:false"); + let some = unsafe { + rich::runtime_rich_has_source( + std::ptr::null_mut(), + &consumer_source::VTABLE, + label.as_ptr(), + label.len(), + &mut len, + &mut err, + ) + }; + assert_eq!(take_string(some, len), "x:true"); + assert_eq!(unsafe { rich::runtime_rich_always(&mut err) }, 1); +} + +#[test] +fn many_concurrent_launches_share_a_few_executor_threads() { + extern "C" fn on_tag(ctx: *mut c_void, err: *mut FfiError, ptr: *const u8, len: usize) { + on_string(ctx, err, ptr, len); + } + const CALLS: usize = 64; + let (tx, rx) = mpsc::channel(); + let ctx = new_ctx::(tx); + for _ in 0..CALLS { + unsafe { tasks::runtime_tasks_thread_tag(on_tag, ctx) }; + } + let mut threads = std::collections::BTreeSet::new(); + for _ in 0..CALLS { + let (code, _, tag) = rx.recv_timeout(WAIT).unwrap(); + assert_eq!(code, 0); + assert!(tag.contains("weaveffi-async"), "{tag}"); + threads.insert(tag); + } + drop_ctx::(ctx); + let workers = std::thread::available_parallelism() + .map_or(2, |n| n.get()) + .max(2); + assert!( + threads.len() <= workers, + "{} threads ran {CALLS} calls with {workers} workers", + threads.len() + ); +} diff --git a/crates/weaveffi/tests/spawn_failure.rs b/crates/weaveffi/tests/spawn_failure.rs new file mode 100644 index 00000000..58594edf --- /dev/null +++ b/crates/weaveffi/tests/spawn_failure.rs @@ -0,0 +1,43 @@ +//! A spawner that fails (panics) completes the call with the generic code +//! and a message, exactly once. This runs in its own test binary because it +//! installs a process-wide spawner. + +#![allow(unsafe_code)] + +use std::os::raw::c_void; +use std::sync::mpsc; +use std::time::Duration; + +use weaveffi::abi::{self, FfiError}; + +#[weaveffi::module] +pub mod work { + /// Would answer 42, if the executor took it. + #[weaveffi::export] + pub async fn answer() -> i32 { + 42 + } +} + +extern "C" fn done(ctx: *mut c_void, err: *mut FfiError, result: i32) { + // Clone the sender before sending: the test may free the context as soon + // as the value arrives, which can be before `send` returns. + let tx = unsafe { &*ctx.cast::>() }.clone(); + let (code, message) = unsafe { ((*err).code, (*err).message_str().unwrap_or("").to_string()) }; + unsafe { abi::error_free(err) }; + tx.send((code, message, result)).unwrap(); +} + +#[test] +fn a_failing_spawner_completes_with_the_generic_code_once() { + weaveffi::set_spawner(|_fut: weaveffi::BoxFuture| panic!("executor is gone")).unwrap(); + + let (tx, rx) = mpsc::channel::<(i32, String, i32)>(); + let ctx: *mut c_void = Box::into_raw(Box::new(tx)).cast(); + unsafe { work::spawn_failure_work_answer(done, ctx) }; + let (code, message, result) = rx.recv_timeout(Duration::from_secs(30)).unwrap(); + assert_eq!((code, result), (abi::GENERIC_ERROR_CODE, 0)); + assert!(message.contains("executor is gone"), "{message}"); + assert!(rx.recv_timeout(Duration::from_millis(100)).is_err()); + drop(unsafe { Box::from_raw(ctx.cast::>()) }); +} diff --git a/crates/weaveffi/tests/tokio.rs b/crates/weaveffi/tests/tokio.rs new file mode 100644 index 00000000..75a8fe87 --- /dev/null +++ b/crates/weaveffi/tests/tokio.rs @@ -0,0 +1,64 @@ +//! The `tokio` feature: exported `async fn`s run on Tokio, so a future may +//! use Tokio's timers and I/O. Outside a runtime the library starts its own; +//! inside one (a consumer calling from a Tokio thread) it uses the current +//! handle. Run with `cargo test -p weaveffi --features tokio`. + +#![cfg(feature = "tokio")] +#![allow(unsafe_code)] + +use std::os::raw::c_void; +use std::sync::mpsc; +use std::time::Duration; + +use weaveffi::abi::{self, FfiError}; + +#[weaveffi::module] +pub mod timers { + /// Sleep on a Tokio timer, then report the runtime thread's name. + #[weaveffi::export] + pub async fn nap(ms: u64) -> String { + tokio::time::sleep(std::time::Duration::from_millis(ms)).await; + std::thread::current().name().unwrap_or("").to_string() + } +} + +extern "C" fn done(ctx: *mut c_void, err: *mut FfiError, ptr: *const u8, len: usize) { + // Clone the sender before sending: the test may free the context as soon + // as the value arrives, which can be before `send` returns. + let tx = unsafe { &*ctx.cast::>() }.clone(); + let code = if err.is_null() { + 0 + } else { + let code = unsafe { (*err).code }; + unsafe { abi::error_free(err) }; + code + }; + let name = unsafe { abi::lift_string(ptr, len) }.unwrap_or_default(); + unsafe { abi::free_bytes(ptr.cast_mut(), len) }; + tx.send((code, name)).unwrap(); +} + +fn nap(ms: u64) -> (i32, String) { + let (tx, rx) = mpsc::channel::<(i32, String)>(); + let ctx: *mut c_void = Box::into_raw(Box::new(tx)).cast(); + unsafe { timers::tokio_timers_nap(ms, done, ctx) }; + let out = rx.recv_timeout(Duration::from_secs(30)).unwrap(); + drop(unsafe { Box::from_raw(ctx.cast::>()) }); + out +} + +#[test] +fn outside_a_runtime_the_library_starts_its_own() { + assert_eq!(nap(5), (0, "weaveffi-async".to_string())); +} + +#[test] +fn inside_a_runtime_the_current_handle_is_used() { + let rt = tokio::runtime::Builder::new_multi_thread() + .thread_name("consumer-runtime") + .enable_all() + .build() + .unwrap(); + let (code, name) = rt.block_on(async { tokio::task::spawn_blocking(|| nap(5)).await.unwrap() }); + assert_eq!((code, name.as_str()), (0, "consumer-runtime")); +} diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 52aaf55b..4745fe79 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -11,7 +11,7 @@ - [Async and Cancellation](guides/async.md) - [Project Configuration](guides/config.md) - [Packaging](guides/packaging.md) -- [Extracting an IDL from Rust](guides/extract.md) +- [Library Mode](guides/extract.md) # Reference diff --git a/docs/src/api/README.md b/docs/src/api/README.md index 8b91ae88..f30d2a7c 100644 --- a/docs/src/api/README.md +++ b/docs/src/api/README.md @@ -1,18 +1,16 @@ # Rust Crates -WeaveFFI is published as six crates that share one version: +WeaveFFI is published as four crates that share one version: | Crate | Use it when | |-------|-------------| -| [`weaveffi`](https://docs.rs/weaveffi) | You're writing a Rust producer. It's the only dependency a producer needs. | -| [`weaveffi-abi`](https://docs.rs/weaveffi-abi) | You need the runtime directly; producers reach it as `weaveffi::abi`. | +| [`weaveffi`](https://docs.rs/weaveffi) | You're writing a Rust producer. It's the only dependency a producer needs, and its `abi` module is the runtime. | | [`weaveffi-macros`](https://docs.rs/weaveffi-macros) | Never directly; `weaveffi` re-exports its macros. | -| [`weaveffi-model`](https://docs.rs/weaveffi-model) | You're building a tool on the IR: parsing, validating, or extracting an API, or reading the binding model. | -| [`weaveffi-gen`](https://docs.rs/weaveffi-gen) | You're driving or extending the generators from Rust. | -| [`weaveffi-cli`](https://crates.io/crates/weaveffi-cli) | You want the `weaveffi` command (`cargo install weaveffi-cli`). | +| [`weaveffi-model`](https://docs.rs/weaveffi-model) | You're building a tool on the IR: parsing, validating, or extracting an API, reading library metadata, or reading the validated `Model`. Its default `idl` feature adds the YAML, JSON, and TOML formats, the JSON Schema, and rich diagnostics. | +| [`weaveffi-cli`](https://docs.rs/weaveffi-cli) | You want the `weaveffi` command (`cargo install weaveffi-cli`), or you're driving or extending the generators from Rust (`project::Project` generates a project, from a `build.rs` too). | - [Rust API Map](rust.md) lists the items a producer uses from `weaveffi` - and `weaveffi-abi`, and which exist only for the macro expansion. + and its `abi` runtime, and which exist only for the macro expansion. - [Doc Comment Style](doc-style.md) is the convention for the doc comments themselves. diff --git a/docs/src/api/doc-style.md b/docs/src/api/doc-style.md index 365cf2af..4bde63cc 100644 --- a/docs/src/api/doc-style.md +++ b/docs/src/api/doc-style.md @@ -13,7 +13,7 @@ read consistently and the doc lints stay green in CI. - Document fallible and panicking behavior with `# Errors`, `# Panics`, and `# Safety` sections. These are the Rust analog of "what can go wrong," and the matching Clippy lints require them. -- Link other items with intra-doc links: `` [`BindingModel`] `` or +- Link other items with intra-doc links: `` [`Model`] `` or `` [`Api`](weaveffi_model::ir::Api) ``. - Wrap code-like identifiers in backticks. Product and tool names (WeaveFFI, SwiftPM, CMake) are allow-listed in `clippy.toml` instead. @@ -47,7 +47,7 @@ Google-style docstring. ```rust /// Generate bindings for every requested target and write them to `out_dir`. /// -/// Targets are rendered from a shared [`BindingModel`] so symbol names and +/// Targets are rendered from a shared [`Model`] so symbol names and /// parameter lowering are computed once and reused across languages. /// /// # Errors @@ -111,7 +111,7 @@ obligations. Clippy's `missing_safety_doc` enforces this. /// /// `arena` must be a valid pointer returned by `arena_create`. `ptr` and /// `dtor` must stay valid until `arena_destroy` is called. -pub fn arena_register(arena: *mut HandleArena, ptr: *mut c_void, dtor: Dtor) { +pub unsafe fn arena_register(arena: *mut HandleArena, ptr: *mut c_void, dtor: Dtor) { // ... } ``` @@ -125,10 +125,16 @@ flags undocumented `pub` fields and variants, not just the type itself. /// Error struct passed across the C ABI boundary. #[repr(C)] pub struct FfiError { - /// Status code. `0` means success; any non-zero value indicates failure. + /// Status code. `0` means success; a positive value is a domain error + /// code and a negative value is one of the reserved runtime codes. pub code: i32, /// Owned, NUL-terminated UTF-8 message, or null when `code` is `0`. pub message: *const c_char, + /// Owned value buffer holding the error code's payload fields, or null + /// when the code declares no fields. + pub payload_ptr: *const u8, + /// Byte length of `payload_ptr`; `0` when null. + pub payload_len: usize, } /// How a value crosses the ABI boundary. @@ -181,7 +187,7 @@ Link to other items so rustdoc can resolve and cross-reference them. This is the Rust analog of the docs site's autorefs: ```rust -/// Renders from the shared [`BindingModel`], never re-deriving lowering. +/// Renders from the shared [`Model`], never re-deriving lowering. /// /// See [`Api`](weaveffi_model::ir::Api) for the input model and /// [`LanguageBackend`](crate::backend::LanguageBackend) for the trait every @@ -195,7 +201,7 @@ Use the short `` [`Type`] `` form when the item is in scope, and the Clippy's `doc_markdown` lint flags identifiers that look like code but aren't wrapped in backticks. Wrap real identifiers, types, paths, and -file names in backticks (`` `BindingModel` ``, `` `weaveffi.yml` ``). +file names in backticks (`` `Model` ``, `` `weaveffi.toml` ``). Product names, tool names, and naming-convention terms (WeaveFFI, SwiftPM, CMake, NuGet, `snake_case`, `PascalCase`) read as prose, not code. Rather @@ -205,8 +211,11 @@ name. ## Enforcement -The doc lints are configured per library crate (in each crate's `lib.rs`) -and centrally in `clippy.toml`: +The doc lints are set in each library crate's `lib.rs`, and `clippy.toml` +holds the `doc_markdown` allow list. Every library crate denies +`missing_docs`; `weaveffi`, `weaveffi-model`, and `weaveffi-cli` also turn +on the Clippy doc lints below (`weaveffi-macros` exports only proc macros, +which `missing_docs` covers): | Lint | What it requires | | --- | --- | @@ -224,12 +233,12 @@ fail the build. Check your changes locally before pushing: cargo clippy --workspace --all-targets -- -D warnings # Build the API docs the way the rustdoc job does. -RUSTDOCFLAGS="-D rustdoc::all -D rustdoc::missing_crate_level_docs" \ +RUSTDOCFLAGS="-D warnings -D rustdoc::all -D rustdoc::missing_crate_level_docs" \ cargo doc --workspace --no-deps -# Or run both through the shared recipe. -just doc +# Or run formatting, clippy, and rustdoc together. +just check ``` -The generated API reference is published under -[`/api/rust/`](rust.md) when the docs site deploys. +The generated API reference is published under `/api/rust/` on the docs +site when it deploys; the [Rust API Map](rust.md) is the guided tour. diff --git a/docs/src/api/rust.md b/docs/src/api/rust.md index e588d38f..c2f775fe 100644 --- a/docs/src/api/rust.md +++ b/docs/src/api/rust.md @@ -1,56 +1,60 @@ # Rust API Map -A map of the public API a Rust producer sees: the `weaveffi` facade and the -`weaveffi-abi` runtime it re-exports as `weaveffi::abi`. `cargo doc` has the -full signatures and safety contracts; this page says which items you'll -reach for. +A map of the public API a Rust producer sees: the `weaveffi` facade and its +`weaveffi::abi` runtime. `cargo doc` has the full signatures and safety +contracts; this page says which items you'll reach for. ```toml [dependencies] -weaveffi = "0.23" +weaveffi = "0.24" ``` ## The `weaveffi` facade | Item | Purpose | |------|---------| -| `#[weaveffi::module]` | Marks an exported module and emits its thunks and checksum. | +| `#[weaveffi::module]` | Marks an exported module, validates its tree, and emits its thunks, contract table, and the library metadata the CLI reads the API from. | | `#[weaveffi::export]` | Exports a function (`async fn` is async; a `Result` return throws). | | `#[weaveffi::record]` | A by-value struct. | | `#[weaveffi::enumeration]` | A `#[repr(i32)]` C-style enum or a rich enum. | | `#[weaveffi::interface]` | A reference-counted object type, with its inherent `impl`. | -| `#[weaveffi::callback_interface]` | A trait the consumer implements; accepted as `Arc`. | +| `#[weaveffi::callback_interface]` | A trait the consumer implements, each method returning `Result`; accepted as `Arc` or `Option>`. | +| `#[weaveffi::throws]` | A callback method whose consumer may fail with the error domain in scope. | | `#[weaveffi::error]` | The module's error domain; requires `Display`. | | `#[weaveffi::cancellable]` | An async function taking a `CancelToken`. | | `weaveffi::export_runtime!()` | Exports the runtime symbols under the crate's prefix; call once. | | `Iter` | The return type for `iter`; build with `Iter::new(iterator)`. | | `CancelToken` | A cancellable function's last parameter; `is_cancelled()` for cooperative cleanup. | -| `ForeignError` | A consumer callback's failure (`code`, `message`), returned by `Result`-returning callback methods. | -| `ErrorReport` | Maps an error type to `(code, message, payload)`; implemented for `String`, `&str`, boxed errors, and every `#[weaveffi::error]` enum. | -| `set_spawner`, `Spawner`, `BoxFuture` | Install the executor async functions run on. | -| `abi` | The `weaveffi-abi` crate. | - -The `leak-check` feature enables the runtime's leak counters. The +| `ForeignError` | A consumer callback's failure (`code`, `message`, `payload`), the `Err` of every callback method; `domain::()` decodes a declared domain error. Return it from an exported function to propagate a failure with `?`. | +| `ErrorDomain` | Implemented by every `#[weaveffi::error]` enum, so `ForeignError::domain` can decode it. | +| `ErrorReport` | Maps an error type to `(code, message, payload)`; implemented for `String`, `&str`, boxed errors, `ForeignError`, and every `#[weaveffi::error]` enum. | +| `set_spawner`, `Spawner`, `BoxFuture` | Install the executor async functions run on, overriding the default worker pool (or Tokio). | +| `abi` | The C ABI runtime (below). | + +The `leak-check` feature enables the runtime's leak counters, and the +`tokio` feature runs async functions on Tokio by default. The [producer macro guide](../guides/producer-macro.md) covers every attribute. -## The `weaveffi-abi` runtime +## The `weaveffi::abi` runtime Items marked **(macro)** are called by generated thunks; read them to -understand the expansion, but you shouldn't need to call them. Everything is -re-exported at the crate root. +understand the expansion, but you shouldn't need to call them. Everything +but the `leak` constants is re-exported at the root of `weaveffi::abi`. | Module | Items | |--------|-------| -| root | `ABI_VERSION` (`3`). | -| `error` | `FfiError`, the `#[repr(C)]` error struct (the C `{prefix}_error`); the codes `GENERIC_ERROR_CODE` through `CANCELLED_ERROR_CODE` (`-1` to `-5`); `ErrorReport`; `panic_message`. **(macro)** `error_set`, `error_store`, `error_clear`, `error_free`, `boxed_error`. | -| `convert` | `(ptr, len)` conversions: `lift_str`, `lift_string`, `lift_byte_slice`, `lift_bytes` read borrowed parameters; `lower_string`, `lower_bytes`, `bytes_into_raw` hand out allocations; `free_bytes` releases them. Useful in tests that call thunks directly. | -| `object` | **(macro)** `lower_object`, `object_ref`, `object_arc`, `object_clone`, `object_destroy`, and the buffer token pair `object_to_token` / `object_from_token`. | -| `callback` | `ForeignError`; `raise_foreign_error`, which propagates a callback failure from producer code; `foreign_status`; `ThunkScope`. **(macro)** `ForeignCallback`, `Vtable`, `CallbackInterface`, `lift_callback`, `defer_foreign_error`, `take_foreign_error`. | -| `cancel` | `CancelToken`. **(macro)** `Cancellable`, `FfiCancelToken`, and the `cancel_token_*` bodies. | +| root | `ABI_VERSION` (`4`). | +| `error` | `FfiError`, the `#[repr(C)]` error struct (the C `{prefix}_error`); the codes `GENERIC_ERROR_CODE` through `CANCELLED_ERROR_CODE` (`-1` to `-5`); `ErrorReport`; `ErrorDomain`; `panic_message`. **(macro)** `error_set`, `error_set_c`, `error_set_payload_c`, `error_store`, `error_clear`, `error_free`, `boxed_error`. | +| `convert` | `(ptr, len)` conversions: `lift_str`, `lift_string`, `lift_byte_slice`, `lift_bytes` read borrowed parameters; `lower_string`, `lower_bytes`, `bytes_into_raw` hand out allocations; `alloc` allocates a run for the producer to adopt; `adopt_bytes` adopts one; `free_bytes` releases either kind. Useful in tests that call thunks directly. | +| `marshal` | `CEnum`, the trait every C-style enum implements (`from_i32`, `to_i32`). **(macro)** the per-family `lift_*_param` and `lower_*_ret` functions, `lift_self`, `lift_self_arc`, `lift_enum`, `read_enum`, `write_enum`, the `*_slots` and `*_run` helpers, `call_sync`, and `Sentinel`. | +| `object` | **(macro)** `lower_object`, `lower_object_opt`, `object_ref`, `object_arc`, `object_clone`, `object_destroy`, and the buffer token pair `object_to_token` / `object_from_token`. | +| `callback` | `ForeignError`; `VtableHeader`, the fixed start of every vtable. **(macro)** `ForeignCallback`, `Vtable`, `VtableTooSmall`, `CallbackInterface`, `lift_callback`, `lift_callback_opt`, `callback_status`, `callback_status_in`, and the `callback_ret_*` adopters. | +| `cancel` | `CancelToken`. **(macro)** `Cancellable`, `FfiCancelToken`, `AtomicWaker`, and the `cancel_token_*` bodies. | | `iter` | `Iter`. **(macro)** `IterHandle`, `iter_into_raw`, `iter_next`, `iter_destroy`. | -| `buffer` | The value-buffer codec: `BufferValue`, `BufferWriter`, `BufferReader`, `BufferDecodeError`, `encode_value`, `decode_value`, and the `ByValue` marker that records and rich enums implement. | -| `spawn` | `set_spawner`, `Spawner`, `SpawnerAlreadySet`, `BoxFuture`, and `block_on` (the default executor). **(macro)** `run_async`, `spawn`, `CatchUnwind`. | -| `leak` | `debug_live` and the `ENABLED` flag; counters are live only with `leak-check`. | +| `buffer` | The value-buffer codec: `BufferValue`, `BufferWriter`, `BufferReader`, `BufferDecodeError`, `encode_value`, `decode_value`, the `ByValue` marker that records and rich enums implement, and `FixedWidth`, the token behind single-copy lists of numbers. `decode_value` and `BufferValue::read_value` are `unsafe`, because decoding an object token adopts a reference; a `BufferReader::token_free` reader refuses tokens and is safe for any input. | +| `contract` | `ContractEntry`. **(macro)** `contract_table`, `contract_len`, `contract_compact`. | +| `spawn` | `set_spawner`, `Spawner`, `SpawnerAlreadySet`, `BoxFuture`, `SpawnError`, `default_pool_size`, and `block_on` (a blocking executor for tests). **(macro)** `launch_async`, `run_async`, `spawn`, `CatchUnwind`. | +| `leak` | `debug_live`, the `ENABLED` flag, and the kind constants `COUNTING` (`-1`), `OBJECTS`, `CALLBACKS`, `ITERATORS`, `TOKENS`, and `ALLOCATIONS` (`0` to `4`); counters are live only with `leak-check`. | The [C ABI contract](../reference/abi.md) is the specification these items implement, and [Async and Cancellation](../guides/async.md) covers the diff --git a/docs/src/architecture.md b/docs/src/architecture.md index 363b7281..9a350bc8 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -7,97 +7,105 @@ everything here implements. ## The pipeline -Every generating command (`generate`, `diff`, `package`) runs the same -stages: +Every generating command (`generate`, `diff`, `dev`, `package`) runs the +same stages: ```text -annotated Rust (.rs) IDL (.yml / .json / .toml) - weaveffi_model::rust weaveffi_model::parse +Rust producer crate IDL (.yml / .json / .toml) + cargo build, then the library's weaveffi_model::parse + metadata frames (weaveffi_cli::library, | + weaveffi_model::meta::assemble) | \ / +--------> Api (the IR) <------+ | v - validate weaveffi_model::validate: every rule, including the - C symbol table; reports all violations at once + validate weaveffi_model::validate::validate(&api, &identity): every + rule, including the C symbol table under the library's + real Identity (from weaveffi.toml / cargo metadata); reports + all violations at once and builds the Model exactly once | v - resolve ResolvedApi: the untouched document plus a type index; - every reference resolves to a Ty with an absolute path. - The CLI attaches the Identity from weaveffi.toml / Cargo.toml + Model the identity, the schema version, per-module bindings with + every C symbol and lowered ABI signature, and the type + index; the plan (ArgPass, RetPass, ErrorStrategy, and the + iterator, async, and callback protocols) is computed from it | v - model BindingModel::build: per-module bindings with every C - symbol and lowered ABI signature computed once + render Target::render(&Model): each generator returns its files + in memory (pure; no I/O) and never sees the Api | v - plan weaveffi_model::plan: ArgPass, RetPass, ErrorStrategy, - IteratorProtocol, AsyncProtocol, CallbackProtocol - | - v - render Target::render: each generator returns its files in - memory (pure; no I/O) - | - v - orchestrate Orchestrator: capability gate, cache records, writes - changed files, removes stale ones, runs hooks + orchestrate Orchestrator: writes changed files, removes stale ones, + keeps generation records ``` -`validate` stops after resolution; `diff` renders and compares without -writing; `package` renders package layouts instead of source trees. +`validate` stops once the model is built; `diff` renders and compares +without writing; `package` renders installable artifacts (wheels, npm +tarballs, gems, ...) around the per-platform libraries `build` compiled. The **producer macro** runs the front half of the same pipeline inside the producer's build. `#[weaveffi::module]` lowers its module tree with -`weaveffi_model::rust::module_from_item_mod` (the extractor the CLI uses for -`.rs` inputs), computes the module's checksum with -`weaveffi_model::checksum::module_checksum`, builds a `BindingModel` with an -identity named after `CARGO_CRATE_NAME`, and emits `extern "C"` thunks -instead of rendering a binding. Because both sides share the extractor, the -model, and the checksum function, the symbols and checksums a producer -exports are exactly those its generated bindings expect. +`weaveffi_model::rust::extract_module`, validates it into a `Model` with the +same validator the CLI uses +(`validate_scoped`, which accepts names declared in another tree as records +or rich enums and leaves the macro to assert that at compile time), with an +identity named after `CARGO_CRATE_NAME`, and reports each validation error +on the offending item through the extractor's `SourceMap`. It then computes +the module's contract table with `weaveffi_model::contract::entries` and +emits `extern "C"` thunks instead of rendering a binding, wrapping each +item's output in that item's `#[cfg]`. Finally it embeds the tree's IR in +the library: one `weaveffi_model::meta::Frame` per declaration, in an +exported `{PREFIX}_META_{HASH16}` static (the `weaveffi_meta` custom section +on `wasm32`) carrying the same `#[cfg]`. The CLI reads those frames back out +of the built library (see [Library Mode](guides/extract.md)), so it +generates from exactly the declarations the build compiled, with the same +validator, model, and contract function, and the symbols and contract +entries a producer exports are exactly those its generated bindings +expect. ## Crates ```text -weaveffi-cli ──► weaveffi-gen ──► weaveffi-model - ▲ -weaveffi ──┬──► weaveffi-macros ───────┘ (proc macro, build time only) - └──► weaveffi-abi (runtime linked into every producer) +weaveffi-cli ─────────────────► weaveffi-model + ▲ +weaveffi ──► weaveffi-macros ────────┘ (proc macro, build time only) -weaveffi-fuzz ──► weaveffi-model (unpublished) +weaveffi-fuzz ──► weaveffi-model (unpublished) ``` -`weaveffi-abi` depends on no other workspace crate, so a producer never -links the generators, and the generators never link the runtime. The ABI -revision is therefore declared twice, in `weaveffi_abi::ABI_VERSION` and in -the model (re-exported as `weaveffi_gen::cabi::ABI_VERSION`), and -`crates/weaveffi/tests/abi_version.rs` keeps them equal. +The runtime (`weaveffi::abi`) depends on no other workspace crate at run +time, so a producer never links the generators, and the generators never +link the runtime. The ABI revision is therefore declared twice, in +`weaveffi::abi::ABI_VERSION` and in the model (re-exported as +`weaveffi_cli::cabi::ABI_VERSION`), and +`crates/weaveffi-cli/tests/abi_version.rs` keeps them equal. | Crate | Owns | |-------|------| -| `weaveffi-model` | Everything between a definition and code generation: the IR (`ir`), IDL parsing (`parse`, behind the default `idl` feature), Rust extraction (`rust`), validation and diagnostics (`validate`), the resolved view (`resolved`), package identity (`pkg`), the binding model with the symbol table (`model`), C ABI lowering (`abi`), the marshalling plan (`plan`), error naming (`errors`), and contract checksums (`checksum`). The macro uses it without the `idl` feature. | -| `weaveffi-gen` | The backend framework and all eleven generators: `backend` (the `LanguageBackend` trait), `codegen` (the object-safe `Target`, `ConfiguredBackend`, the `Orchestrator`, and the `CodeWriter` toolkit), `cabi` (the shared C declaration renderer), `cache` (generation records), `capabilities` (the feature gate), `lang` (keyword tables and escaping), `manifest` (JSON and XML escaping for package manifests), `package` and `platform` (packaging), and `targets::{c, cpp, swift, kotlin, node, wasm, python, dotnet, dart, go, ruby}`. | -| `weaveffi-cli` | The `weaveffi` binary: argument parsing (`main.rs`), `weaveffi.toml` and the target registry (`config.rs`), one module per subcommand under `commands/`, identity resolution, and extraction (`extract.rs`). | -| `weaveffi-abi` | The runtime: the error struct and codes, `(ptr, len)` conversions, reference-counted objects, cancel tokens, callback vtables and foreign errors, iterators, the value-buffer codec, the async spawner and `run_async`, and leak counters. | -| `weaveffi-macros` | `#[weaveffi::module]`, the marker attributes, and `export_runtime!`. Emission is split by concern under `src/codegen/` (`sync`, `async_fns`, `iterators`, `records`, `enums`, `interfaces`, `callbacks`, `foreign`, `marshal`). | -| `weaveffi` | The producer facade: re-exports the macros, the few runtime types a producer names, and `weaveffi_abi` as `weaveffi::abi`. | -| `weaveffi-fuzz` | `cargo-fuzz` targets for the parsers, `parse_type_ref`, and the validator. | - -The workspace denies `unsafe_code`; `weaveffi-abi` opts in, and the thunks +| `weaveffi-model` | Everything between a definition and code generation: the IR (`ir`), IDL parsing (`parse`, behind the default `idl` feature), Rust extraction for the macro (`rust`), library metadata frames (`meta`), validation and diagnostics (`validate`), resolved types and the type index (`ty`), package identity (`pkg`), the model with the symbol table (`model`), C ABI lowering (`abi`), the marshalling plan (`plan`), error naming (`errors`), and contract tables (`contract`). The macro uses it without the `idl` feature. | +| `weaveffi-cli` | The library (`src/lib.rs`): the backend framework and all eleven generators: `backend` (the `LanguageBackend` trait), `codegen` (the object-safe `Target`, `ConfiguredBackend`, the `Orchestrator`, and the `CodeWriter` toolkit), `cabi` (the shared C declaration renderer), `cache` (generation records), `lang` (keyword tables and escaping), `manifest` (JSON and XML escaping for package manifests), `cargo` (resolving a producer crate with `cargo metadata`), `build` (cross-compiling a producer per platform and prebuilding the Node.js and JNI glue), `package` (artifacts and the wheel, npm, gem, tarball, and zip writers) and `platform` (the platform matrix), `config` (`weaveffi.toml` and the target registry), `project` (locating a project, identity resolution, and loading its API), `library` (reading a built library's metadata), `utils` (generated-file banners), and `targets::{c, cpp, swift, kotlin, node, wasm, python, dotnet, dart, go, ruby}`. The `weaveffi` binary (`src/main.rs`): argument parsing and one module per subcommand under `commands/`. | +| `weaveffi-macros` | `#[weaveffi::module]`, the marker attributes, and `export_runtime!`. Emission is split by concern under `src/codegen/` (`sync`, `async_fns`, `iterators`, `records`, `enums`, `interfaces`, `callbacks`, `contract`, `meta`, `foreign`, `diagnostics`, `helpers`, `lift`). | +| `weaveffi` | The producer facade: re-exports the macros and the few runtime types a producer names. Its `abi` module is the runtime: the error struct and codes, `(ptr, len)` conversions, reference-counted objects, cancel tokens, callback vtables and foreign errors, iterators, the value-buffer codec, the async spawner (a worker pool by default, Tokio with the `tokio` feature) and `run_async`, contract tables, and leak counters (the `leak-check` feature). | +| `weaveffi-fuzz` | `cargo-fuzz` targets for the parsers, `parse_type_ref`, the validator, and the value-buffer decoder. | + +The workspace denies `unsafe_code`; `weaveffi::abi` opts in, and the thunks the macro emits carry a scoped allowance, so a macro-based producer needs no `unsafe` of its own. ## Key invariants - **Generators consume the model only.** Symbol names, ABI signatures, type - families, and wire shapes come from `BindingModel`, `Ty::family()`, and - `Ty::wire()`; a generator never re-derives a symbol, splits a qualified - name, or reads the raw IR. Names come from the `Identity` on the - `ResolvedApi`. -- **Absolute types.** Every resolved `Ty` names a user type by its absolute - path (`kv.Store`), so equality works across modules; - `weaveffi_gen::utils::local_type_name` gives the display name. -- **One symbol table.** `BindingModel::c_symbols` lists every C identifier - with the declaration that owns it, and validation rejects duplicates. + families, and wire shapes come from `Model`, `Ty::family()`, and + `Ty::wire()`; a generator never re-derives a symbol or reads the raw IR. + Names come from the `Identity` on the `Model`. +- **Global names.** Type names, free-function names, and error-code names + are unique across the API, so a `Ty` carries the bare name (`Store`) and + equality works across modules. Where a generator needs the declaring + module (for a C type name or a namespace path), it asks the model's type + index (`Model::owner`, `Model::interface`, and so on); nothing splits a + dotted string. +- **One symbol table.** `Model::c_symbols` lists every C identifier with the + declaration that owns it, and validation rejects duplicates. - **Fixed runtime code is real source.** A target's codec, error base types, and loader live as source files under `targets//runtime/`, included with `include_str!` and filled in with `{{PLACEHOLDER}}` @@ -107,13 +115,17 @@ the macro emits carry a scoped allowance, so a macro-based producer needs no - **No branding.** Nothing generated is named after WeaveFFI except the generated-file header and runtime-version comments. -## The binding model and the plan +## The model and the plan -`BindingModel::build` walks a `ResolvedApi` once and produces one -`ModuleBinding` per module (nested modules flattened, each with its -segments, its underscore `path`, its dotted path, and, for a top-level -module, its checksum). Every function, interface member, and callback method -carries: +`validate` indexes every type declaration, checks every rule, and then walks +the document once to produce the `Model`: the identity, the schema version, +the type index (each name's kind and declaring module), and one +`ModuleBinding` per module (nested modules flattened, each with its segments, +its underscore `path`, its dotted path, its own error domain if it declares +one). A top-level module's contract table is computed from the model on +demand by `weaveffi_model::contract::entries`. A module without a domain +inherits the nearest ancestor's, found with `Model::error_domain`. Every +function, interface member, and callback method carries: - its resolved parameter and return `Ty`s, the idiomatic shape a generator renders; @@ -132,75 +144,72 @@ inside a value buffer (`Prim`, `Object`, `Enum`, `User`, `Optional`, `List`, `Map`), which codec emitters match on. `weaveffi_model::plan` states the calling contracts once, derived from the -family: how each argument is passed (`ArgPass`), what a wrapper does with a -result and which runtime release it owes (`RetPass`, `Free`), whether a -non-zero error code is a typed domain error or a trap (`ErrorStrategy`), and -the iterator, async, and callback protocols. A generator renders these in its -own syntax; it doesn't decide them. +family: how each argument is passed (`ParamBinding::arg_pass`), what a +wrapper does with a result (`RetPass::of`; an adopted object owes its +interface binding's `destroy_symbol`), whether a non-zero error code is a +typed domain error or a trap (`FnBinding::error_strategy`), and the +iterator, async, and callback protocols (`protocol()` on each binding). A +generator renders these in its own syntax; it doesn't decide them. ## Backends and the orchestrator -A generator implements `weaveffi_gen::backend::LanguageBackend`: +A generator implements `weaveffi_cli::backend::LanguageBackend`: ```rust,ignore pub trait LanguageBackend: Send + Sync { - type Config: Serialize + Default + Clone + Send + Sync; + type Config: Default + Clone + Send + Sync; fn name(&self) -> &'static str; - fn capabilities(&self, config: &Self::Config) -> TargetCapabilities; - fn files(&self, api: &ResolvedApi, model: &BindingModel, - out_dir: &Utf8Path, config: &Self::Config) -> Vec; - fn package(/* ... */) -> Option> { None } - // Optional per-entity hooks (render_error, render_enum, render_struct, - // render_callback_interface, render_interface, render_function) and - // emit_members, which walks a module in canonical order. + fn files(&self, model: &Model, out_dir: &Utf8Path, + config: &Self::Config) -> Vec; + fn package(/* ... */) -> Option> { None } } ``` -`ConfiguredBackend::new(backend, config)` builds the `BindingModel` and -erases the backend to `dyn Target`, whose `render` returns `OutputFile`s. -Single-pass backends override the per-entity hooks and call `emit_members`; -backends with several parallel files (C++, Swift, Kotlin, Node.js, Wasm) -build their layout in `files`. +`ConfiguredBackend::new(backend, config)` erases the backend to +`dyn Target`, whose `render(&Model, out_dir)` returns `OutputFile`s. +Each backend walks the model in the order its language needs and builds its +own file layout in `files`. The `Orchestrator` runs the selected targets: -1. **Capability gate.** Each target declares the features it implements for - its config; an API using anything else fails with every offending - declaration listed (or warns, for targets configured to emit throwing - stubs). -2. **Freshness.** For each target it hashes the canonical IR, the identity, - the target name, its serialized config, and the CLI version, and compares - with `{out}/.weaveffi-cache/{target}.json`. A target is fresh when the - hash matches and every recorded file is on disk unchanged. -3. **Render and write.** Stale targets render in parallel (rayon). Each file - is written only if its contents differ, and files recorded by the - previous run but no longer produced are deleted. The `pre_generate` and - `post_generate` hooks run around the writes, only when something is - stale. +1. **Render.** Every target renders in parallel (rayon), in memory. +2. **Compare.** Each file is compared with the output directory, and each + target's file list with its record from the previous run, + `{out}/.weaveffi-cache/{target}.json`. When nothing differs, nothing is + written. +3. **Write.** Each file is written only if its contents differ, files + recorded by the previous run but no longer produced are deleted, and the + record is updated. `weaveffi diff` uses `render` directly and compares in memory. ## The CLI -`config.rs` holds `ProjectConfig` (the `[project]`, `[package]`, `[global]`, -and `[generators.*]` tables, with discovery and path resolution) and the -`cli_targets!` registry: one line per target that expands to the typed -`[generators.]` field, the `--target` name, and the orchestrator -registration. `commands/mod.rs` has the shared front half every generating -command runs (locate the project, resolve the identity, load and validate, -attach the identity), and each subcommand has its own module: `init`, -`generate`, `validate`, `diff`, `package`. +The library's `config` module holds `ProjectConfig` (the `[project]`, +`[package]`, `[build]`, and `[generators.*]` tables, with discovery and path +resolution) and the `cli_targets!` registry: one line per target that +expands to the typed `[generators.]` field, the `--target` name, and the +orchestrator registration. Its `project` module is the front half every +command runs: `Project` locates the project and its input (an IDL, a crate, +or a library alone), builds a crate's library with `cargo` and reads its +frames with `library`, resolves the identity, and validates the API into +the `Model` once. A `build.rs` can use `Project` too. The binary's +`commands/` has one module per subcommand: `init`, `generate`, `dev`, +`validate`, `diff`, `extract`, `build`, `package`. ## Adding a generator -1. Add `crates/weaveffi-gen/src/targets//` following an existing +1. Add `crates/weaveffi-cli/src/targets//` following an existing target's layout: `mod.rs` (config, generator, `LanguageBackend` impl), `types.rs`, `codec.rs`, `calls.rs`, `entities.rs`, `package.rs`, a `runtime/` directory of fixed source, and `tests.rs`. -2. Implement `LanguageBackend`. Declare honest capabilities; take every name - from the model and the identity; load `{library}` and honor - `{PREFIX}_LIBRARY`; check the ABI revision and every module checksum at - load; surface cancellation idiomatically; keep objects alive across calls. +2. Implement `LanguageBackend`. Take every name from the model and the + identity; bind to `{library}` (and, if the language loads it at run time, + accept a full path in `{PREFIX}_LIBRARY`); check the ABI revision and + every top-level module's contract table at load; follow the + [trap policy](guides/errors-and-memory.md#the-trap-policy); surface + cancellation idiomatically; keep objects alive across calls. Implement + `package` if the ecosystem has an installable artifact. 3. Register it with one line in `cli_targets!` in `crates/weaveffi-cli/src/config.rs`. 4. Add it to `snapshot_tests!` in `crates/weaveffi-cli/tests/snapshots.rs` @@ -219,16 +228,16 @@ usually has to pass all of them. | Layer | Where | What it pins | |-------|-------|--------------| -| Unit tests | `#[cfg(test)]` modules; `tests.rs` in each target | Parsing, validation rules, `Ty` classification, lowering, naming helpers | +| Unit tests | `#[cfg(test)]` modules; `tests.rs` in each target | Parsing, validation rules, `Ty` classification, lowering, naming helpers, metadata frames (`meta`) | | Validation tests | `crates/weaveffi-model/src/validate/tests.rs` | Every `ValidationError` with its source span | -| Property tests | `crates/weaveffi-abi/tests/buffer_proptest.rs` | Codec laws: round trips, self-delimiting encodings, rejected trailing bytes | -| Runtime tests | `crates/weaveffi/tests/` | Exported runtime symbols, cancellation and dropped futures, foreign-error routing, leak counters, the ABI revision lockstep | +| Property tests | `crates/weaveffi/tests/buffer_proptest.rs` | Codec laws: round trips, self-delimiting encodings, rejected trailing bytes | +| Runtime tests | `crates/weaveffi/tests/` | Exported runtime symbols, cancellation and dropped futures, foreign-error routing, leak counters | | Macro tests | `crates/weaveffi-macros/tests/ui/` (`trybuild`) | `pass_*.rs` compile; `fail_*.rs` fail with the pinned `.stderr` | -| Snapshots | `crates/weaveffi-cli/tests/snapshots.rs` (`insta`) | Byte-exact output of every target for every fixture in `tests/fixtures/` | -| CLI tests | `crates/weaveffi-cli/tests/cli/` | Every subcommand's behavior and exit codes, extraction round trips, determinism, no stub markers in output | +| Snapshots | `crates/weaveffi-cli/tests/snapshots.rs` (`insta`) | Byte-exact output of every target: fixture-dependent files for every fixture in `tests/fixtures/`, fixed runtimes and manifests once, and copies of the C header asserted equal to the C target's | +| CLI tests | `crates/weaveffi-cli/tests/cli/`, `crates/weaveffi-cli/tests/abi_version.rs` | Every subcommand's behavior and exit codes, library mode (the `tests/fixtures/producer` crate extracts to its `expected.yml`, and generating from its library equals generating from that IDL), determinism, no stub markers in output, the ABI revision lockstep | | Fixture compile checks | `scripts/check-fixtures.sh ` | Every fixture's generated tree compiles or type-checks with the target's toolchain | | Conformance | `conformance/run.sh` | Real consumers in every language against every sample, with leak counters at zero | -| Fuzzing | `crates/weaveffi-fuzz` | Parsers and the validator never panic | +| Fuzzing | `crates/weaveffi-fuzz` | Parsers and the validator never panic; value-buffer decoders reject malformed bytes cleanly | The fixtures are `kitchen_sink` (every feature), `edge_cases` (reserved words, deep nesting, objects in every legal position), `nested_modules` diff --git a/docs/src/comparison.md b/docs/src/comparison.md index 55bd604b..f2c01fed 100644 --- a/docs/src/comparison.md +++ b/docs/src/comparison.md @@ -8,7 +8,7 @@ hand-written glue, the single-language bridges **swift-bridge**, **napi-rs**, and **wasm-bindgen**, and the C/C++-first generators **SWIG** and **autocxx**. > All comparisons reflect the public state of each project at the time of -> writing (WeaveFFI ABI revision 3, schema `0.10.0`). If something here is out +> writing (WeaveFFI ABI revision 4, schema `0.11.0`). If something here is out > of date, please open a PR. ## The short version @@ -20,14 +20,16 @@ package is standalone: consumers never install WeaveFFI. The object model is on par with the Rust-first tools: interface objects are reference counted (`Arc`), can be shared between wrappers and nested inside records, lists, maps, optionals, iterators, and async results, and callback interfaces let -the producer call consumer code through a vtable. Every symbol and package is -named after the library, never after WeaveFFI, so several WeaveFFI-built -libraries coexist in one process. +the producer call consumer code through a vtable whose methods can return any +value and raise typed errors. A Rust producer's API is read from the library +it compiled (library mode, as in UniFFI), and `weaveffi package` turns the +builds into installable artifacts for every ecosystem. Every symbol and +package is named after the library, never after WeaveFFI, so several +WeaveFFI-built libraries coexist in one process. The honest flip side is that WeaveFFI is pre-1.0 and its type system is -deliberately smaller than UniFFI's: no user-defined generics or trait objects -with generics, callback methods return only direct values (no strings, -buffers, or objects yet), no async callback methods, and no multi-file IDL +deliberately smaller than UniFFI's: no user-defined generics or trait +objects, no custom types, no async callback methods, and no multi-file IDL imports. The [roadmap](roadmap.md) lists what's planned. ## At a glance @@ -35,13 +37,13 @@ imports. The [roadmap](roadmap.md) lists what's planned. | | **WeaveFFI** | **UniFFI** | **Diplomat** | **cbindgen + glue** | **swift-bridge** | **napi-rs** | **wasm-bindgen** | **SWIG** | **autocxx** | |------------------------------------|:------------:|:----------:|:------------:|:-------------------:|:----------------:|:-----------:|:----------------:|:--------:|:-----------:| | Producer language | Rust, C, C++, Zig (anything with a C ABI) | Rust | Rust | Rust | Rust | Rust | Rust | C / C++ | C++ (consumed from Rust) | -| Input | YAML / JSON / TOML IDL or annotated Rust | UDL or proc-macros | annotated Rust bridge crate | Rust source | annotated Rust bridge module | annotated Rust | annotated Rust | C/C++ headers + `.i` file | C++ headers | -| Consumer languages | C, C++, Swift, Kotlin, Node.js, Wasm/JS, Python, .NET, Dart, Go, Ruby | Kotlin, Swift, Python, Ruby first-party; Go, C#, Dart, Kotlin Multiplatform, React Native as external bindgens | C, C++, JS/TS (Wasm), Dart, Kotlin (JNA), Python (nanobind) | C (you write the rest) | Swift | Node.js | JS/TS (Wasm) | many (Python, Java, C#, Ruby, Lua, Perl, PHP, R, ...) | Rust | +| Input | YAML / JSON / TOML IDL or annotated Rust (read from the built library) | UDL or proc-macros (library mode reads the built library) | annotated Rust bridge crate | Rust source | annotated Rust bridge module | annotated Rust | annotated Rust | C/C++ headers + `.i` file | C++ headers | +| Consumer languages | C, C++, Swift, Kotlin, Node.js, Wasm/JS, Python, .NET, Dart, Go, Ruby | Kotlin, Swift, Python, Ruby first-party; Go, C#, Dart, Kotlin Multiplatform, React Native (`uniffi-bindgen-react-native`) as external bindgens | C, C++, JS/TS (Wasm), Dart, Kotlin (JNA), Python (nanobind) | C (you write the rest) | Swift | Node.js | JS/TS (Wasm) | many (Python, Java, C#, Ruby, Lua, Perl, PHP, R, ...) | Rust | | Standalone C header | ✓ (the contract every target shares) | ✗ (private scaffolding ABI) | ✓ (C backend) | ✓ (its purpose) | ✓ (generated) | ✗ | ✗ | ✗ | n/a | | **Type system** | | | | | | | | | | | Records / enums / optionals / lists / maps | ✓ (value buffers) | ✓ | ✓ (structs, enums, `Option`, slices; no maps) | manual | ✓ (transparent structs and enums, `Option`, `Vec`) | ✓ (serde objects) | ✓ (`serde-wasm-bindgen` or classes) | ✓ | ✓ | | Objects with methods | ✓ reference counted, shareable, nestable | ✓ (`Arc`) | ✓ (opaques, borrowed or owned) | manual | ✓ (opaque types) | ✓ (`#[napi]` classes) | ✓ (classes) | ✓ (classes) | ✓ (C++ classes) | -| Callback interfaces | ✓ (vtable; sync, direct returns, failures as `Result`) | ✓ (foreign traits; any return, `throws`, async) | partial (Kotlin, C, C++; input-only) | manual fn pointers | partial (closures Rust to Swift) | ✓ (`ThreadsafeFunction`) | ✓ (closures) | partial (directors) | partial | +| Callback interfaces | ✓ (vtable; synchronous, any return, `throws`, optional, failures as `Result`) | ✓ (foreign traits; any return, `throws`, async) | partial (Kotlin, C, C++; input-only) | manual fn pointers | partial (closures Rust to Swift) | ✓ (`ThreadsafeFunction`) | ✓ (closures) | partial (directors) | partial | | Typed error domains | ✓ (per-module codes, opt-in `throws`, payload fields) | ✓ (error enums) | ✓ (`Result`) | manual | ✓ (`Result`) | ✓ (JS `Error`) | ✓ (`Result`) | ✗ | ✗ | | Async functions | ✓ (callback ABI, pluggable spawner, native cancellation on every target) | ✓ (poll-based, foreign executors) | ✗ | manual | ✓ (both directions) | ✓ (Tokio) | ✓ (`Promise`) | ✗ | ✗ | | Iterators | ✓ (`iter`, lazy on every target) | ✗ (materialize or use a trait) | partial (`DiplomatWrite`) | manual | ✗ | ✗ | ✓ (JS iterators) | partial | ✗ | @@ -49,9 +51,9 @@ imports. The [roadmap](roadmap.md) lists what's planned. | Multi-file / multi-crate definitions | ✗ (one document per API) | ✓ (external types across crates) | ✓ (one bridge crate, many modules) | n/a | ✗ | n/a | n/a | ✓ (`%include`) | ✓ | | **Workflow** | | | | | | | | | | | Standalone CLI | ✓ (`cargo install weaveffi-cli`) | `uniffi-bindgen` (build.rs or CLI) | `diplomat-tool` | ✓ | `swift-bridge-cli` | `napi` CLI (npm) | `wasm-bindgen-cli` | system package | cargo build | -| Publishable per-ecosystem packages | ✓ (`weaveffi package` for every target) | partial | partial | n/a | ✓ (SwiftPM) | ✓ (npm) | ✓ (npm via wasm-pack) | ✗ | n/a | +| Publishable per-ecosystem packages | ✓ (`weaveffi package`: wheels, npm tarballs, gems, an XCFramework SwiftPM package, a `.nupkg`; Gradle, pub, and Go as ready-to-publish directories) | via companion tools (`cargo-swift`, `maturin`, `uniffi-bindgen-react-native`) | partial | n/a | ✓ (SwiftPM) | ✓ (npm) | ✓ (npm via wasm-pack) | ✗ | n/a | | Schema-checked IDL with JSON Schema | ✓ | ✗ | n/a | n/a | n/a | n/a | n/a | ✗ | n/a | -| Load-time ABI check | ✓ (ABI revision plus per-module contract checksums) | ✓ (checksums) | ✗ | ✗ | ✗ | ✓ (N-API version) | ✗ | ✗ | n/a | +| Load-time ABI check | ✓ (ABI revision plus per-declaration contract tables) | ✓ (checksums) | ✗ | ✗ | ✗ | ✓ (N-API version) | ✗ | ✗ | n/a | | Generated-output drift check in CI | ✓ (`weaveffi diff --check`) | build-time | build-time | ✓ | build-time | build-time | build-time | ✗ | build-time | | Maturity | pre-1.0 | shipping in Firefox and Mozilla products since 2020 | shipping in ICU4X | 1.0+, widely deployed | 0.1.x, active | 2.x+, widely deployed | 0.2.x, ubiquitous | 30+ years | pre-1.0 | | License | MIT OR Apache-2.0 | MPL-2.0 | MIT OR Apache-2.0 | MPL-2.0 | MIT OR Apache-2.0 | MIT | MIT OR Apache-2.0 | GPL-3.0 (generated code exempt) | MIT OR Apache-2.0 | @@ -67,13 +69,16 @@ is ahead of WeaveFFI today. - **UniFFI has the richer type system and more production mileage.** It ships in Firefox, Firefox Sync, Glean, and Nimbus and has years of - battle-testing across iOS, Android, and desktop. Its foreign traits can - return any type (strings, records, objects), can `throw`, and can be - `async`; WeaveFFI's callback-interface methods are synchronous and return - only scalars, `bool`, or a C-style enum. UniFFI also supports external types - across crates, whereas a WeaveFFI API is one IDL document (or one annotated - Rust source tree). If your matrix is Kotlin, Swift, and Python and you want - maximum maturity, UniFFI is the safer pick. + battle-testing across iOS, Android, and desktop. Its foreign traits can be + `async`; WeaveFFI's callback-interface methods can return any type and + `throw`, but are synchronous. UniFFI also supports custom types and + external types across crates, whereas a WeaveFFI API is one IDL document + (or one Rust producer crate). Its ecosystem of companion tools is larger + too: `cargo-swift` builds the XCFramework and SwiftPM package, `maturin` + builds Python wheels, and `uniffi-bindgen-react-native` targets React + Native, where WeaveFFI's `weaveffi package` covers the first two itself + and has no React Native target. If your matrix is Kotlin, Swift, and + Python and you want maximum maturity, UniFFI is the safer pick. - **Diplomat has the more mature Kotlin and JS story for a Rust library.** It powers ICU4X, its bridge-crate model keeps everything in Rust, and its C++ backend is polished for slotting into existing C++ builds. Its opaques @@ -102,17 +107,23 @@ is ahead of WeaveFFI today. parse C++; you describe the surface you want to expose and implement the generated header. - **WeaveFFI's Wasm target is single-threaded.** The default - `wasm32-unknown-unknown` build has no threads, so async functions resolve - inline (which leaves little for an `AbortSignal` to cancel) and - callback-interface methods fire only while a call into the module is on the - stack; a producer that calls back from a spawned thread can't run there. wasm-bindgen's `wasm-bindgen-futures` integrates with the - JS event loop natively, and Emscripten-based toolchains can use + `wasm32-unknown-unknown` build has no threads, so async functions are + polled inline until they finish (a future that waits on something outside + the call fails with `-1`, and there's little for an `AbortSignal` to + cancel), and callback-interface methods fire only while a call into the + module is on the stack. wasm-bindgen's `wasm-bindgen-futures` integrates + with the JS event loop natively, and Emscripten-based toolchains can use `pthread`s. In WeaveFFI's Emscripten compatibility mode, async functions and callback interfaces are not available at all. +- **The single-language tools package better for their one ecosystem.** + `maturin` (with PyO3) and napi-rs's CLI build wheels and npm packages for + every platform from one CI matrix with years of edge cases handled; + `weaveffi package` is newer, and it doesn't prebuild the Node.js addon or + the JNI shim for Windows. - **No formal stability guarantee yet.** WeaveFFI is pre-1.0; schema - `0.10.0` and ABI revision 3 renamed every symbol and changed string - passing without compatibility shims (see the - [migration guide](stability.md#migrating-to-schema-010-and-abi-3)). + `0.11.0` and ABI revision 4 changed names, the load-time contract check, + and the callback vtable layout without compatibility shims (see the + [migration guide](stability.md#migrating-to-schema-011-and-abi-4)). UniFFI, cbindgen, napi-rs, wasm-bindgen, and SWIG offer stronger compatibility commitments today. @@ -137,29 +148,33 @@ WeaveFFI is the right pick when you want: 4. **Callback interfaces without hand-written trampolines.** Declare the methods once and each target gets a protocol, interface, or abstract class to implement; the producer receives an `Arc` it can retain and - call from any thread, and a consumer-side exception surfaces to the - original caller as a typed foreign error. + call from any thread. Methods can return strings, records, and objects, + and a consumer-side exception reaches the producer as a + `weaveffi::ForeignError` (carrying a typed domain error when the method + is `throws`) that it can propagate to the original caller with `?`. 5. **Standalone, publishable consumer packages.** Generated packages are - self-contained; `weaveffi package` bundles the native library per platform - (including `jniLibs//` for Kotlin and the `.wasm` for npm). There is no - "install WeaveFFI" step on the consumer side. + self-contained; `weaveffi build` cross-compiles the producer per platform + and `weaveffi package` bundles it into wheels, npm tarballs, gems, an + XCFramework SwiftPM package, a NuGet package, and Gradle, pub, Go, and + C/C++ distributions. There is no "install WeaveFFI" step on the consumer + side. 6. **Idiomatic per-target output, not a lowest-common-denominator API.** Async functions become `async/await` in Swift, `Promise`s in Node and Wasm, `suspend fun` in Kotlin, `async def` in Python, `Task` in C#, and `Future` in Dart, all from the same `async: true` flag; Rust producers can plug their own executor with `weaveffi::set_spawner`. Cancelling the awaiting task, coroutine, or promise cancels the native call. -7. **A CI-first CLI.** `validate`, `diff --check`, `extract`, `schema`, and - `package` are designed to drop into pipelines, every generator's output is +7. **A CI-first CLI.** `validate`, `diff --check`, `extract`, `schema`, + `build`, and `package` are designed to drop into pipelines, every generator's output is byte-for-byte deterministic, and generated consumers refuse to load a producer built for a different ABI revision or from a different API - definition (per-module contract checksums). + definition (per-declaration contract tables, so adding a declaration + never breaks a deployed binding). ## When to choose something else - **You only need Kotlin, Swift, and Python and want maximum stability, or - you need callback methods that return strings, records, or objects, or - async callbacks**: use UniFFI. + you need async callbacks**: use UniFFI. - **You only need a C header for a Rust crate**: use cbindgen. - **You ship to exactly one of Swift, Node.js, or the browser**: use swift-bridge, napi-rs, or wasm-bindgen respectively. @@ -175,8 +190,9 @@ WeaveFFI is the right pick when you want: WeaveFFI's IDL is intentionally close to UniFFI's UDL surface area (records, enums, interfaces, callback interfaces, error enums, `async`), which makes hand-porting straightforward in either direction. There is no automatic UDL -to WeaveFFI converter today, but `weaveffi extract` can read annotated Rust -source and produce a starting IDL, which is often the fastest path off any -Rust-only generator. See the [extract guide](guides/extract.md) for details, -and the [C ABI contract](reference/abi.md) if you're bringing a non-Rust -producer to the generated header. +to WeaveFFI converter today. Going the other way, `weaveffi extract` prints +the IDL a `#[weaveffi::module]` producer's built library embeds, which is the +starting point for reimplementing a Rust producer in another language. See +[Library Mode](guides/extract.md) for details, and the +[C ABI contract](reference/abi.md) if you're bringing a non-Rust producer to +the generated header. diff --git a/docs/src/faq.md b/docs/src/faq.md index b4826dc9..3c957243 100644 --- a/docs/src/faq.md +++ b/docs/src/faq.md @@ -8,9 +8,9 @@ maximum maturity, it's the safer pick. WeaveFFI exists for a different set of needs: eleven first-party targets from one definition, a producer that can be written in any language with a C ABI (the generated header is a public contract, not private scaffolding), a standalone CLI built for CI, and a -schema-checked YAML, JSON, or TOML IDL. UniFFI is still ahead on callback -methods that return rich types, async callback methods, and cross-crate type -imports. See the [comparison](comparison.md) for the full table. +schema-checked YAML, JSON, or TOML IDL. UniFFI is still ahead on async +callback methods, custom types, and cross-crate type imports. See the +[comparison](comparison.md) for the full table. ## Can I use it with a C++ codebase? @@ -38,7 +38,7 @@ borrowed `(ptr, len)` views, so the producer copies only what it keeps. Objects pass as one pointer. Records and collections are encoded into one value buffer per value, with single-copy fast paths for byte and numeric lists. Async calls add a completion callback and whatever executor drives -the future. The runtime itself is the small `weaveffi-abi` crate; the +the future. The runtime itself is the small `weaveffi::abi` module; the `leak-check` counters cost an atomic update per counted operation and are off by default. @@ -50,7 +50,7 @@ domain, negative codes are the runtime's (generic, panic, marshalling, callback failure, cancelled). A function marked `throws: true` (a `Result` in Rust) surfaces domain codes as typed errors in every language; a function without it traps on failure, since a failure there is a bug. See -[Errors and Memory](guides/errors-and-memory.md). +[Errors and Memory](guides/errors-and-memory.md#the-trap-policy). ## Can two WeaveFFI libraries live in one process? @@ -63,27 +63,41 @@ own copy of the small runtime. ## What happens if the bindings and the library drift apart? They refuse to load. Every generated consumer compares the library's ABI -revision and each top-level module's contract checksum with the values it -was generated against, and raises the language's load error naming the -module on a mismatch. The checksum covers names, types, order, and flags, but -not documentation, so editing doc comments never breaks a deployed binding. +revision with the one it was generated against, and checks that every +declaration it uses is in the library's contract table with the same +signature hash. On a mismatch it raises the language's load error naming the +declaration that's missing or changed. The hashes cover names, types, and +flags, but not documentation or declaration order, so editing doc comments +never breaks a deployed binding, and neither does adding a function, method, +or type. In CI, `weaveffi diff --check` catches stale committed bindings before they ship. +## Does the CLI parse my Rust source? + +No. `#[weaveffi::module]` embeds a description of the API in the library it +compiles, and `weaveffi generate` builds the crate and reads that +description back out of the library, so macros, type aliases, and `#[cfg]` +are already resolved and the bindings match the build exactly. `weaveffi +extract` prints it as an IDL. See [Library Mode](guides/extract.md). + ## Can I customize the generated code? Through `weaveffi.toml`: package metadata, per-target names (the Swift -module, the Kotlin package, the Go module path), target-specific options, and -`pre_generate`/`post_generate` hooks for formatters. See +module, the Kotlin package, the Go module path, each set with `name`), and +target-specific options. Run formatters as a separate step after +`weaveffi generate`. See [Project Configuration](guides/config.md). Changing the C ABI itself is a generator contribution; see the [architecture guide](architecture.md). ## Does it work with Flutter? The Dart target emits `dart:ffi` bindings with a `pubspec.yaml` usable from -Flutter and plain Dart on every platform that supports `dart:ffi`. You ship -the native library for each platform yourself; Flutter's native-assets build -isn't generated yet. For the web, use the WebAssembly target. See +Flutter and plain Dart on every platform that supports `dart:ffi`. +`weaveffi package --target dart` bundles the desktop libraries under +`native//`; for iOS and Android you add the library to the Flutter +app's native build yourself, since Flutter's native-assets build isn't +generated yet. For the web, use the WebAssembly target. See [Dart](generators/dart.md). ## Is it Windows-friendly? @@ -115,9 +129,11 @@ objects passed to the producer are borrowed. See ## Which executor runs my async functions, and how does cancellation work? -The one you install with `weaveffi::set_spawner`, such as a Tokio handle; -by default each future runs on its own thread. Consumers see their native -async idiom, and cancelling it (a Swift `Task`, a Kotlin coroutine, an +By default, a small pool of worker threads (one per core, at least two) +started on the first call. With the `weaveffi` crate's `tokio` feature it's +Tokio: the current runtime when the call is made from inside one, otherwise +one the library creates. `weaveffi::set_spawner` installs any other executor +and overrides both. Consumers see their native async idiom, and cancelling it (a Swift `Task`, a Kotlin coroutine, an `AbortSignal`, a `CancellationToken`, a Go `context`) cancels the native call: the runtime drops the future and completes with the cancelled code, which surfaces as the language's cancellation error. See diff --git a/docs/src/generators/README.md b/docs/src/generators/README.md index fa59cbf4..0eac540b 100644 --- a/docs/src/generators/README.md +++ b/docs/src/generators/README.md @@ -1,34 +1,38 @@ # Generators WeaveFFI ships eleven generators, one per target. Every generator implements -the whole IDL surface of schema 0.10.0 on ABI revision 3: records, C-style +the whole IDL surface of schema 0.11.0 on ABI revision 4: records, C-style and rich enums, reference-counted objects in every position, callback interfaces, optionals, lists, maps, typed error domains, nested modules, async functions with cancellation, and lazy iterators. Each one names its -output from the library's [identity](../reference/naming.md), loads -`{library}` (overridable with `{PREFIX}_LIBRARY`), checks the ABI revision -and every module checksum at load time, and produces a tree under +output from the library's [identity](../reference/naming.md), binds to the +native library `{library}`, checks the ABI revision and every top-level +module's contract table at load time, and produces a tree under `{out}/{target}/` that builds with the language's normal tooling. -A generator that can't support a feature in some mode (Wasm's Emscripten -mode, for example) declares it, and `weaveffi generate` fails with the -offending declarations listed instead of silently skipping them. +The targets that load the library at run time (Python, Ruby, .NET, Dart, +Kotlin on the JVM, and WebAssembly on Node.js) accept a full path to it in +the `{PREFIX}_LIBRARY` environment variable (`KVSTORE_LIBRARY`). The others +link it when they're built: C, Swift, and Go through the platform linker, C++ +through its CMake target (which reads `{PREFIX}_LIBRARY` at configure time), +and the Node.js addon when `npm install` compiles it. Each language page says +where its package looks. ## Capability matrix | Target | Package / module | Object release | Async | Cancellation | Callback interface | `iter` | Minimum toolchain | |--------|------------------|----------------|-------|--------------|--------------------|-----------|-------------------| | [C](c.md) | header `{library}.h` | `_destroy` | launcher plus callback | `{prefix}_cancel_token*` | `ctx` plus vtable struct | `_next`/`_destroy` | C11 | -| [C++](cpp.md) | header `{library}.hpp`, namespace `{prefix}` | RAII destructor; copies clone | `std::future` | RAII `CancelToken` | abstract class | input range | C++17 | -| [Swift](swift.md) | module `PascalCase(name)` | `deinit` | `async` / `async throws` | `Task` cancellation | protocol | `Sequence` | Swift 5.7 | -| [Kotlin](kotlin.md) | package `{prefix}` | `close()`, cleaner backstop | `suspend fun` | coroutine cancellation | interface | `Iterator` | Kotlin 1.9; Android API 21 or a JVM | +| [C++](cpp.md) | header `{library}.hpp`, namespace `{prefix}` | RAII destructor; copies clone | `std::future` | RAII `CancelToken` | abstract class | input range | C++17, CMake 3.14 | +| [Swift](swift.md) | module `PascalCase(name)` | `deinit` | `async` / `async throws` | `Task` cancellation | protocol | `Sequence` | Swift 5.9 | +| [Kotlin](kotlin.md) | package `{prefix}` | `close()`, cleaner backstop | `suspend fun` | coroutine cancellation | interface | `NativeIterator` (an `Iterator`) | Kotlin 2.0; Android API 21 or a JVM | | [Node.js](node.md) | npm `{name}` | `close()`, `Symbol.dispose`, finalizer | `Promise` | `AbortSignal` | object implementing an interface | `IterableIterator` | Node.js 18 | | [WebAssembly](wasm.md) | npm `{name}` | `close()`, `Symbol.dispose`, finalizer | `Promise` | `AbortSignal` | object implementing an interface | `IterableIterator` | Node.js 18 or a current browser | | [Python](python.md) | dist `{name}`, import `{prefix}` | `close()`, `with`, finalizer | awaitable | task cancellation | abstract base class | iterator | Python 3.9 | | [.NET](dotnet.md) | namespace `PascalCase(name)` | `Dispose()` (`SafeHandle`) | `Task` | `CancellationToken` | interface | `IEnumerable` | .NET 8 | -| [Dart](dart.md) | package `{prefix}` | `dispose()`, `NativeFinalizer` | `Future` | cancel-token object | abstract class | `Iterable` | Dart 3.1 | +| [Dart](dart.md) | package `{prefix}` | `dispose()`, `NativeFinalizer` | `Future` | cancel-token object | abstract class | `Iterable` | Dart 3.10 | | [Go](go.md) | module `{name}`, package `{prefix}` | `Close()`, finalizer | blocking call | `context.Context` | interface | `iter.Seq` / `iter.Seq2` | Go 1.23 | -| [Ruby](ruby.md) | gem `{name}`, module `PascalCase(name)` | `close`, `FFI::AutoPointer` | blocking call | `cancel:` keyword | duck-typed object | `Enumerator` | Ruby with the `ffi` gem 1.15 | +| [Ruby](ruby.md) | gem `{name}`, module `PascalCase(name)` | `close`, `FFI::AutoPointer` | blocking call | `cancel:` keyword | duck-typed object | `Enumerator` | Ruby 2.7 with the `ffi` gem 1.15 | The language pages are authoritative for exact type names, signatures, and toolchain versions. Notes that apply across targets: @@ -43,8 +47,9 @@ toolchain versions. Notes that apply across targets: - **Async.** A cancelled call raises the language's cancellation error. Go and Ruby block the calling goroutine or thread while the native work runs elsewhere. See [Async and Cancellation](../guides/async.md). -- **64-bit integers.** Node.js and WebAssembly use `bigint`; Kotlin and Dart - carry `u64` in a signed 64-bit type. `NaN`, infinities, and `-0` cross +- **64-bit integers.** Node.js and WebAssembly use `bigint`; Kotlin uses + `ULong` for `u64`; Dart carries `u64` in its signed 64-bit `int` (the bits + are preserved, so values above `2^63 - 1` read as negative). `NaN`, infinities, and `-0` cross unchanged everywhere. ## Fixture compile checks @@ -58,19 +63,33 @@ output with the target's own toolchain: `cc` and `c++` with `-Werror` for C, `clang++ -fsyntax-only` for C++, `swiftc -typecheck`, `kotlinc`, `tsc --strict` and `node --check`, `py_compile` plus a type checker on the stubs, `dotnet build -warnaserror`, `dart analyze --fatal-infos`, `go vet` and `go -build`, and `ruby -wc`. CI runs one job per target. +build`, and `ruby -wc`. CI runs one job per target. A missing tool is a +printed skip locally and a failure when `CI=true`. ## Conformance lanes -`conformance/run.sh` is the behavioral gate. It builds the six sample -producers with leak counting enabled, runs `weaveffi generate` on each -sample's `src/lib.rs`, and then runs `conformance//run.sh` for every -language. Each lane (`python-kvstore`, `go-events`, and so on) compiles and -runs one consumer against one sample under a timeout; the consumer binds -only through the generated package, asserts concrete results, and checks -that every `{prefix}_debug_live` counter is zero at exit after forcing its -garbage collector. The C lanes also implement a producer by hand -(`c-producer-exports`) and check each checksum against the header. +`conformance/run.sh` is the behavioral gate. It builds the three +[sample](../samples.md) producers with leak counting enabled, runs `weaveffi +generate` on each sample crate, reading the API from that same build +(`--library`), and then runs `conformance//run.sh` for every language. Each language declares +exactly three lanes, one per sample (`python-calculator`, `python-codec`, +`python-kvstore`, and so on); a lane compiles and runs that sample's consumer +under a timeout. The consumer binds only through the generated package, +asserts concrete results, and checks that every `{prefix}_debug_live` +counter is zero at exit after forcing its garbage collector: + +- **calculator**: the getting-started surface (a call, a typed error, a + string in and out). +- **codec**: the shared-vector loop. Fetch each of the producer's test + vectors, decode it, pass it back to `check_vector`, and echo each + primitive through the direct ABI families; then check a few vectors built + from literals. +- **kvstore**: every feature, including callbacks implemented in the + consumer's language, cancellation, and iterators. + +The wasm lanes reuse the Node.js consumers, since both targets generate the +same JavaScript API. The C lanes also implement a producer by hand +(`c-producer-exports`) and check its contract table against the header. ```bash bash conformance/run.sh # every lane @@ -80,5 +99,6 @@ ONLY=c,go-kvstore SKIP_GEN=1 bash conformance/run.sh `SKIP` excludes lanes, `SKIP_GEN=1` reuses previously generated bindings, and `LANE_TIMEOUT` sets the per-lane limit in seconds (default 300). A -missing toolchain fails its lanes; skip them explicitly. CI runs every +missing toolchain skips its language with a note locally and fails it when +`CI=true`. CI runs every language on Linux and macOS. diff --git a/docs/src/generators/c.md b/docs/src/generators/c.md index 8730c09e..ecea0546 100644 --- a/docs/src/generators/c.md +++ b/docs/src/generators/c.md @@ -1,7 +1,7 @@ # C The C target emits the canonical header for a library's -[C ABI](../reference/abi.md) (revision 3). Every other target binds to the +[C ABI](../reference/abi.md) (revision 4). Every other target binds to the symbols it declares, and a C or C++ producer implementing an IDL by hand implements exactly those symbols. Alongside it, the target emits a helper header with C structs and codecs for the API's value buffers, so a C @@ -27,9 +27,12 @@ turned off: buffer_helpers = false ``` -`weaveffi package` lays the same headers out under `include/`, adds the -prebuilt library for each platform under `lib//`, and writes a -`CMakeLists.txt` that exposes it as the imported target `kvstore::kvstore`. +`weaveffi package` writes `kvstore-{version}-c.tar.gz`: the same headers +under `include/`, the library of every platform built under +`lib//` (with the import library on Windows), a README, and a +`CMakeLists.txt` (CMake 3.15 or later) that exposes the host's library as +the imported target `kvstore::kvstore` (see +[Packaging](../guides/packaging.md)). ## Build and load @@ -37,7 +40,7 @@ Both headers are plain C11 that also compiles as C++. Point the compiler at the `c/` directory and link the producer's library: ```sh -cc -std=c11 -I generated/c app.c -L target/release -lkvstore -o app +cc -std=c11 -I bindings/c app.c -L target/release -lkvstore -o app ``` C links the library at build time, so the platform loader finds it at run @@ -47,18 +50,29 @@ the library dynamically; a C program that wants it calls `dlopen` itself. Every C identifier starts with the prefix, so any number of WeaveFFI libraries link into one program. Before its first call, a consumer should -check the contract it was generated against: +check the ABI revision and the contract it was generated against: ```c -if (kvstore_abi_version() != KVSTORE_ABI_VERSION || - kvstore_kv_checksum() != KVSTORE_KV_CHECKSUM) { +if (kvstore_abi_version() != KVSTORE_ABI_VERSION) { + fprintf(stderr, "kvstore: library does not match kvstore.h's ABI revision\n"); + exit(1); +} +if (kvstore_kv_contract_check() != 0) { fprintf(stderr, "kvstore: library does not match kvstore.h (module kv)\n"); exit(1); } ``` -There's one `{PREFIX}_{ROOT}_CHECKSUM` constant and one -`{prefix}_{root}_checksum()` function per top-level module. +Each top-level module has a contract table: the library exports +`{p}_{m}_contract(size_t* out_len)`, and the header defines the entries the +bindings were generated with as `{P}_{M}_CONTRACT` (each commented with its +declaration's dotted path), their count as `{P}_{M}_CONTRACT_LEN`, and a +`static inline uint64_t {p}_{m}_contract_check(void)`. The checker returns +`0` when every expected entry is in the library's table with an equal hash, +and otherwise the id of the first one that's missing or changed, which you +can look up in the macro to name the declaration. A library with extra +entries passes, so adding a declaration doesn't break existing consumers. +See [load-time checks](../reference/abi.md#load-time-checks). ## Type mapping @@ -74,7 +88,7 @@ There's one `{PREFIX}_{ROOT}_CHECKSUM` constant and one | `T?` | value buffer | value buffer | `T*` (`NULL` is absent) | | interface `I` | `const {p}_{path}_I*` (borrowed) | `{p}_{path}_I*` (owned) | `{p}_{path}_I*` | | `I?` | `const {p}_{path}_I*` (`NULL` is absent) | `{p}_{path}_I*` | `{p}_{path}_I*` | -| callback interface | `void* x_ctx, const {p}_{path}_{Cb}_vtable* x_vtable` | not allowed | not allowed | +| callback interface, `Cb?` | `void* x_ctx, const {p}_{path}_{Cb}_vtable* x_vtable` (`NULL` vtable is absent for `Cb?`) | not allowed | not allowed | | `iter` | not allowed | iterator handle | not applicable | Strings are UTF-8 without a NUL terminator and may contain NUL bytes. A @@ -163,8 +177,9 @@ Every synchronous function takes a trailing `{p}_error* out_err`. On failure `code` is nonzero and `message` is NUL-terminated UTF-8; release both with `{p}_error_clear`. Positive codes are the module's error domain (`kvstore_kv_KvError_KeyNotFound`); negative codes are runtime traps: `-1` -generic, `-2` panic, `-3` marshalling failure, `-4` a callback -implementation failed, and `-5` cancelled. An error code with fields carries +generic (or an async call that couldn't start), `-2` panic, `-3` +marshalling failure, `-4` a callback implementation failed, and `-5` +cancelled. An error code with fields carries them in `payload_ptr`/`payload_len`, which the helper decodes with `{code constant}_payload_decode`. @@ -193,23 +208,85 @@ with code `-5`, even if it was cancelled before launch. ## Callback interfaces A callback interface is a vtable the consumer fills in, passed with a -context pointer: +context pointer. The vtable starts with a header (`size`, `flags`, and the +`free` hook), followed by one entry per method in declaration order: ```c typedef struct kitchen_sink_kitchen_ReadyListener_vtable { + uint32_t size; + uint32_t flags; + void (*free)(void* ctx); /** Fires when an item is ready */ void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); /** Receives the item itself and says whether to keep listening */ bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); + /** A display label, failing with a kitchen error */ + void (*label)(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); + /** The most recent item the listener kept, if any */ + void (*latest)(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); + /** The listener's favorite gadget */ + kitchen_sink_kitchen_Gadget* (*favorite)(void* ctx, kitchen_sink_error* out_err); } kitchen_sink_kitchen_ReadyListener_vtable; ``` +Define one static vtable per interface, with `size` set to `sizeof` the +vtable and `flags` to `0`. The producer rejects a vtable smaller than its +own with `-3` (calling `free(ctx)` first), so a consumer built from an older +header can't make it call a missing entry: + +```c +static const kitchen_sink_kitchen_ReadyListener_vtable LISTENER = { + sizeof(kitchen_sink_kitchen_ReadyListener_vtable), + 0, + listener_free, + listener_on_ready, + listener_on_item, + listener_label, + listener_latest, + listener_favorite, +}; +``` + The producer may call any entry from any thread, so the context must be -thread-safe. String and buffer arguments are borrowed for the call; an -object argument carries one reference the method adopts. A method reports -failure with `{p}_error_set(out_err, -4, "message")`. The producer calls -`free(ctx)` exactly once, when it drops its last reference. +thread-safe, and it calls `free(ctx)` exactly once, from any thread, when it +drops its last reference. For an optional parameter (`ReadyListener?`), pass +a `NULL` vtable for none. + +String and buffer arguments are borrowed for the call; an object argument +carries one reference the method adopts. Returns go the other way: + +- A direct value is the C return. +- An object (`favorite`) is the C return, one strong reference the producer + adopts: return a fresh `_clone`, never `NULL` for a non-optional type. +- A string, bytes, or buffer (`label`, `latest`) is written to the trailing + `out_ptr` and `out_len` slots as a run allocated with `{p}_alloc`, which + the producer adopts and frees. Leave them `NULL` and `0` for an empty + value. + +A method reports failure with `{p}_error_set(out_err, code, "message")`, +which copies the message. A method declared `throws` (`label`) may use a +code of its module's error domain and attach the code's fields, encoded as a +value buffer, with `{p}_error_set_payload(out_err, ptr, len)`; any other +failure reaches the producer as `-4`: + +```c +static void listener_label(void* ctx, uint8_t** out_ptr, size_t* out_len, + kitchen_sink_error* out_err) { + const char* label = ((listener*)ctx)->label; + if (label == NULL) { + kitchen_sink_error_set(out_err, kitchen_sink_kitchen_KitchenErrors_NotFound, "no label"); + return; + } + size_t n = strlen(label); + uint8_t* run = kitchen_sink_alloc(n); + if (n > 0) memcpy(run, label, n); + *out_ptr = run; + *out_len = n; +} +``` + +The producer adopts whatever the out slots hold even when the method fails, +so a method that fails after allocating doesn't leak. ## Iterators @@ -228,25 +305,35 @@ String, bytes, and buffer items are owned (release each with ## Implementing an IDL in C A hand-written producer defines every function the header declares, -including the runtime and one checksum per top-level module. For the -calculator sample that's: +including the runtime and one contract function per top-level module, which +returns the header's table. For the calculator sample that's: ```c uint32_t calculator_abi_version(void) { return CALCULATOR_ABI_VERSION; } -uint64_t calculator_calculator_checksum(void) { return CALCULATOR_CALCULATOR_CHECKSUM; } + +const calculator_contract_entry* calculator_calculator_contract(size_t* out_len) { + static const calculator_contract_entry table[] = CALCULATOR_CALCULATOR_CONTRACT; + *out_len = CALCULATOR_CALCULATOR_CONTRACT_LEN; + return table; +} void calculator_error_set(calculator_error* err, int32_t code, const char* message); +void calculator_error_set_payload(calculator_error* err, const uint8_t* ptr, size_t len); void calculator_error_clear(calculator_error* err); void calculator_error_free(calculator_error* err); +uint8_t* calculator_alloc(size_t len); void calculator_free_bytes(uint8_t* ptr, size_t len); calculator_cancel_token* calculator_cancel_token_create(void); void calculator_cancel_token_cancel(calculator_cancel_token* token); bool calculator_cancel_token_is_cancelled(const calculator_cancel_token* token); void calculator_cancel_token_destroy(calculator_cancel_token* token); -uint64_t calculator_debug_live(int32_t kind); /* may return 0 */ +uint64_t calculator_debug_live(int32_t kind); /* may return 0 for every kind */ ``` -`{p}_alloc` and `{p}_dealloc` are needed only on wasm32. Every prototype +`{p}_alloc` returns a zero-filled run (`NULL` for `0`), and `{p}_free_bytes` +must release both the runs the producer returns and the runs `{p}_alloc` +hands out, so both use one allocator. `{p}_debug_live(-1)` returns `1` when +the producer counts live resources and `0` when it doesn't. Every prototype carries `{PREFIX}_API`, so the definitions stay exported under `-fvisibility=hidden`; on Windows, define `{PREFIX}_BUILD` while building the producer. `conformance/c/producer.c` is a complete example. @@ -257,6 +344,3 @@ the producer. `conformance/c/producer.c` is a complete example. loader, so `{PREFIX}_LIBRARY` isn't consulted. - The helper's decoders don't validate UTF-8 in decoded strings; the producer has already done so. -- Composite helper names are derived from the type (`{p}_list_kv_Entry`, - `{p}_map_string_i64`), so a module or function named like one of them - can collide with it. diff --git a/docs/src/generators/cpp.md b/docs/src/generators/cpp.md index 89de16bd..e5cdeb11 100644 --- a/docs/src/generators/cpp.md +++ b/docs/src/generators/cpp.md @@ -1,10 +1,11 @@ # C++ -The C++ target emits a header-only C++17 library over the C ABI. Records -become value structs, rich enums become `std::variant`-backed sum types, -interfaces become copyable RAII classes, callback interfaces become abstract -classes you subclass, error domains become exception hierarchies, async -functions return `std::future`, and `iter` returns a lazy range. +The C++ target emits a header-only C++17 library over the C ABI (revision +4). Records become value structs, rich enums become `std::variant`-backed +sum types, interfaces become copyable RAII classes, callback interfaces +become abstract classes you subclass, error domains become exception +hierarchies, async functions return `std::future`, and `iter` returns a +lazy range. ## What gets generated @@ -19,18 +20,19 @@ cpp/ For the kvstore sample that's `kvstore.h`, `kvstore.hpp`, `CMakeLists.txt`, and `README.md`. Everything in the wrapper lives in `namespace {prefix}` (`kvstore`), and each IDL module's free functions live in a nested namespace -named after the module path (`kvstore::kv::stats::get_stats`). Types sit at +named after the module path (`kvstore::kv::stats::summarize`). Types sit at the namespace root, since type names are unique across an API. -| `[cpp]` key | Default | Meaning | +| `[generators.cpp]` key | Default | Meaning | |---|---|---| -| `namespace` | the C prefix | Namespace holding every declaration | +| `name` | the C prefix | Namespace holding every declaration | | `header_name` | `{library}.hpp` | Wrapper header file name | | `standard` | `17` | C++ standard the CMake target requires | ## Build and link -Add the generated directory to a CMake build and link the INTERFACE target: +Add the generated directory to a CMake build (3.14 or later) and link the +INTERFACE target: ```cmake add_subdirectory(path/to/cpp) @@ -46,32 +48,46 @@ CMake, add `cpp/` to the include path and link `-lkvstore`. Both headers can be included in one translation unit, in either order. +`weaveffi package` writes `kvstore-{version}-cpp.tar.gz`: both headers under +`include/`, each desktop platform's library under `lib//`, a +README, and a `CMakeLists.txt` that links the host's library into +`kvstore::cpp`, so the packaged build ignores `KVSTORE_LIBRARY` (see +[Packaging](../guides/packaging.md)). + ### The library check `check_library()` verifies that the linked library implements C ABI revision -3 and the exact contract of every top-level module, and throws `LoadError` -naming the first mismatch: +4 and every declaration the header was generated with, and throws +`LoadError` naming the first mismatch. The header embeds each top-level +module's contract table as `(id, hash, path)` entries and compares them with +the table the library returns from `{prefix}_{module}_contract`: ```cpp inline void check_library() { - static const std::string failure = [] { - uint32_t abi = kitchen_sink_abi_version(); - if (abi != KITCHEN_SINK_ABI_VERSION) { - return "kitchen_sink: C ABI revision mismatch (header " + std::to_string(KITCHEN_SINK_ABI_VERSION) + + static const std::string failure = []() -> std::string { + uint32_t abi = kvstore_abi_version(); + if (abi != KVSTORE_ABI_VERSION) { + return "kvstore: C ABI revision mismatch (header " + std::to_string(KVSTORE_ABI_VERSION) + ", library " + std::to_string(abi) + ")"; } - if (kitchen_sink_shared_checksum() != 0x42c4ce2c8d0af052ull) { - return std::string("kitchen_sink: module 'shared' does not match the linked library (contract checksum mismatch); regenerate the bindings for this build"); + if (std::string why = detail::contract_mismatch(kvstore_kv_contract, detail::kv_contract); !why.empty()) { + return "kvstore: " + why; } ... ``` +A declaration the library lacks fails with `kvstore: kv.Store.put is +missing from the library`, and one whose signature differs with +`kvstore: kv.Store.put changed since these bindings were generated`. +Declarations the library has and the header doesn't are fine, so a library +that adds functions keeps working with older bindings. + A C++ binary links its library rather than loading it, and an exception thrown during static initialization can't be caught, so the check runs on first use instead: every free function, constructor, and static member calls -`check_library()` before its first native call. The result is cached, so the -cost after the first call is one guard-variable read. Call it at startup to -fail early. +`check_library()` before its first native call. The result is cached in a +function-local static, so the cost after the first call is one +guard-variable read. Call it at startup to fail early. ## Type mapping @@ -85,15 +101,17 @@ fail early. | rich enum | `struct E` with `std::variant<...> value` | `const E&` | | interface | RAII class | `const I&` (borrowed) | | `I?` | `std::optional` | `const std::optional&` | -| callback interface | abstract class | `std::shared_ptr` | +| callback interface, `Cb?` | abstract class | `std::shared_ptr` (empty for none when optional) | | `T?`, `[T]`, `{K: V}` | `std::optional`, `std::vector`, `std::unordered_map` | by const reference | | `iter` | lazy range class (returns only) | n/a | Strings cross the ABI as a UTF-8 pointer and a length, so interior NULs survive in both directions. A returned string is copied into a `std::string` -and the producer's allocation is released with `{prefix}_free_bytes`. -Records, rich enums, optionals, lists, and maps cross as one value buffer, -encoded and decoded by generated routines in `detail`. +and the producer's run is released with `{prefix}_free_bytes`. Records, rich +enums, optionals, lists, and maps cross as one value buffer. The header has +one writer and one reader per record, rich enum, and distinct composite type +(`detail::write_list_Entry`, `detail::read_map_string_Store`), and every call +site delegates to them. ## Objects and lifetime @@ -101,29 +119,32 @@ An interface class holds one strong reference to a reference-counted producer object: ```cpp -class Gadget { - kitchen_sink_kitchen_Gadget* handle_; +class Store { + kvstore_kv_Store* raw_; public: /** The C type this class wraps. */ - using raw_type = kitchen_sink_kitchen_Gadget; + using raw_type = kvstore_kv_Store; - /** Adopts one strong reference to a producer object: `Gadget(adopt, raw)`. */ - explicit Gadget(adopt_t, kitchen_sink_kitchen_Gadget* h) noexcept : handle_(h) {} + /** Adopts one strong reference to a producer object: `Store(adopt, raw)`. */ + explicit Store(adopt_t, kvstore_kv_Store* h) noexcept : raw_(h) {} ``` - The destructor releases the reference with `_destroy`; copying takes a new one with `_clone`, so copies share the object; moving transfers it. -- The constructor named `new` becomes an `explicit` C++ constructor; any - other constructor becomes a static factory. Nothing converts implicitly - into an object, and adopting a raw pointer requires the `adopt` tag. +- The constructor named `new` becomes a C++ constructor (`explicit` when it + takes arguments); any other constructor becomes a static factory + (`Store::open(path)`). Nothing converts implicitly into an object, and + adopting a raw pointer requires the `adopt` tag. - A parameter borrows the caller's reference for the call. A returned object, an async result, an iterator element, and an object argument handed to a callback are adopted. - An object inside a record, list, map, or optional crosses as a token minted with `_clone`, so a record can be copied and destroyed freely. -- `handle()` borrows the C pointer; `clone_handle()` returns a new reference - the caller owns. +- `handle()` borrows the C pointer (compare two wrappers' handles to test + identity); `clone_handle()` returns a new reference the caller owns. An + interface member named `handle`, `clone_handle`, or `raw_type` gains a + trailing `_` ([reserved member names](../reference/naming.md#identifiers-in-generated-code)). The C++ object model makes lifetimes the caller's responsibility: a wrapper must outlive the calls made through it, and destroying a wrapper on one @@ -132,21 +153,39 @@ object. A moved-from wrapper holds null. ## Errors -Every failure is an exception derived from `{namespace}::Error`, which -carries `code()`: +A call that declares errors (`throws`) throws an exception derived from +`{namespace}::Error`, which derives from `std::runtime_error` and carries +`code()`: - Each module that declares an error domain gets a domain class - (`KvError : Error`) and one subclass per code (`KeyNotFoundError`). A code - with payload fields exposes them as members. -- A throwing function maps a positive code to its typed exception. Negative - codes are runtime traps and always surface as the generic `Error`: -1 - generic, -2 producer panic, -3 marshalling failure, and -4 a callback - implementation that threw. -- -5 surfaces as `Cancelled`. -- `LoadError` reports a library that doesn't match the headers. - -Wrappers are never `noexcept`, since even a non-throwing function can -surface a trap. + (`KvError : Error`) and one subclass per code (`KeyNotFoundError`). Both + names end in `Error`, which is appended when the IDL name lacks it (the + `KitchenErrors` domain becomes `KitchenErrorsError`). A code with payload + fields exposes them as public members (`e.key`). +- A positive code throws its typed subclass. A negative runtime code + throws the root `Error` with that code: -1 generic, -2 producer panic, -3 + marshalling failure, and -4 a callback implementation that failed. +- -5 throws `Cancelled`, which also derives from `Error`. + +A call that declares no errors can still fail, but only through a producer +bug, so it follows the [trap policy](../guides/errors-and-memory.md#the-trap-policy): +it throws `InternalError`, a `std::runtime_error` that isn't an `Error`, +whose `what()` names the runtime code and the producer's message and whose +`code()` and `message()` return them. A value buffer the wrapper can't +decode is also an `InternalError` with code -3. + +`LoadError` (also a `std::runtime_error`) reports a library that doesn't +match the headers. Wrappers are never `noexcept`. + +```cpp +try { + store.get("missing"); +} catch (const kvstore::KeyNotFoundError& e) { + std::cerr << e.what() << " (" << e.key << ")\n"; +} catch (const kvstore::KvError& e) { + // Any other kv code. +} +``` ## Async and cancellation @@ -154,7 +193,7 @@ An async function returns `std::future`, settled from a producer thread. A `cancellable` function also takes a trailing `const CancelToken&`: ```cpp -inline std::future do_cancellable(std::string_view input, const CancelToken& cancel_token = CancelToken::none()) { +std::future compact(uint32_t pause_ms, const CancelToken& cancel_token = CancelToken::none()) const; ``` `CancelToken` is a move-only RAII wrapper over the native token. Pass one @@ -166,7 +205,7 @@ cancels. ```cpp kvstore::CancelToken token; -std::future pending = store.compact(token); +std::future pending = store.compact(60000, token); token.cancel(); try { pending.get(); @@ -184,30 +223,61 @@ A callback interface is an abstract class with one pure virtual method per IDL method: ```cpp -class ReadyListener { +class Policy { public: - virtual ~ReadyListener() = default; + virtual ~Policy() = default; - /** Fires when an item is ready */ - virtual void on_ready(int32_t code, std::string_view msg) = 0; + virtual Entry admit(const Entry& entry) = 0; + virtual Store route(std::string_view key, Store home) = 0; +}; ``` Pass an implementation as `std::shared_ptr`. The wrapper boxes the pointer -as the vtable context, and the producer's last release deletes the box, -which runs your destructor if nothing else holds the implementation. The -producer may call methods from any thread, so implementations must be -thread-safe. String arguments are views valid for the call. An exception -thrown by a method is reported to the producer with code -4 and its -`what()` message; it never unwinds through the C frame. Whether the -producer method returns `T` or `Result`, the caller sees -the generic `Error` with code -4. +as the vtable context and hands the producer a static vtable whose header +records its `size` and whose `free` deletes the box when the producer +releases the implementation, which runs your destructor if nothing else +holds it. For an optional callback parameter (`Loader?`), an empty +`std::shared_ptr` (or `nullptr`) passes none; a required one throws +`std::invalid_argument` when empty. The producer may call methods, and +release the implementation, from any thread, so implementations must be +thread-safe. + +Arguments follow the parameter rules seen from your side: a string is a +`std::string_view` valid for the call, bytes and buffered values arrive by +const reference to a decoded copy, and an object arrives by value as a +wrapper you own. + +A method can return any family. A direct value is returned as is; an object +(`Store`, `std::optional`) hands the producer a fresh reference; a +string, bytes, or buffered value is copied into a run allocated with +`{prefix}_alloc`, which the producer adopts. A required object return that's +null (a moved-from wrapper) is refused by the producer with -3, as is a +record the producer can't decode. + +A method declared `throws` may throw its module's domain exception: the +code, the message (`what()`), and the code's fields reach the producer, +which decodes them into its own error type. + +```cpp +Entry admit(const Entry& entry) override { + if (entry.key == "secret") throw kvstore::RejectedError("secrets are not stored", entry.key, "no secrets"); + return entry; +} +``` + +Any other exception, or a domain exception from a method that doesn't +declare errors, reaches the producer as a callback failure (-4) with the +exception's `what()`; nothing unwinds through the C frame. Whether the +original call then fails is the producer's decision: a Rust producer that +propagates the failure with `?` makes a throwing call throw the same +`RejectedError` (or the root `Error` with code -4). ## Iterators An `iter` function returns a move-only range class: ```cpp -for (const std::string& key : store.list_keys(std::nullopt)) { +for (const std::string& key : store.keys(std::nullopt)) { // One producer call per step. } ``` @@ -225,5 +295,3 @@ it's exhausted or destroyed, so abandoning a loop early leaks nothing. - Exceptions not derived from `std::exception` are reported to the producer with a generic message. - The library check runs on first use, not at load time (see above). -- The C ABI's callback methods can't return strings or buffered values, so - neither can the abstract classes. diff --git a/docs/src/generators/dart.md b/docs/src/generators/dart.md index 5be90018..9c3746a7 100644 --- a/docs/src/generators/dart.md +++ b/docs/src/generators/dart.md @@ -1,23 +1,38 @@ # Dart -The Dart target generates a standalone Dart package that calls the C ABI -through `dart:ffi`. It works in Dart command-line apps and in Flutter apps on -the desktop, Android, and iOS. +The Dart target generates a standalone Dart package that calls the +producer's C ABI (revision 4) through `dart:ffi`. It works in Dart +command-line apps and in Flutter apps on the desktop, Android, and iOS. +Records become value classes, rich enums sealed class hierarchies, +interfaces disposable wrapper classes, async functions `Future`s, and +callback interfaces abstract classes you implement. ## What gets generated ```text dart/ - pubspec.yaml package `{prefix}`, depends on `ffi` + pubspec.yaml package `{package}`, depends on `ffi` README.md - lib/{prefix}.dart the whole binding: runtime, types, and functions + lib/{package}.dart the whole binding: runtime, types, and functions ``` -The package name is the library's C prefix (`kvstore` for the kvstore -sample), so consumers write `import 'package:kvstore/kvstore.dart';`. Set -`package_name` under `[generators.dart]` to choose another name; the library -file follows it. With `strip_module_prefix = false`, module-level functions -keep their module path (`kvStatsGetStats` instead of `getStats`). +The package name defaults to the library's C prefix (`kvstore` for the +kvstore sample), so consumers write `import 'package:kvstore/kvstore.dart';`. +Module-level functions use their bare names (`openStore`, `renderReport`), +which global names keep unique. + +## Configuration + +```toml +[generators.dart] +name = "kvstore" # package name and library file (default: the C prefix) +sdk = ">=3.10.0 <4.0.0" # the pubspec's SDK constraint (this is the default) +``` + +The package needs Dart 3.10 or newer, the first release with +`NativeCallable.isolateGroupBound`, which delivers void callback methods from +any thread (see [Threading](#threading)), and the `ffi` package, which +supplies `Arena` and `calloc`. ## Install, build, and load @@ -27,7 +42,7 @@ dependency, and run `dart pub get`: ```yaml dependencies: kvstore: - path: generated/dart + path: bindings/dart ``` The bindings open the native library on first use: `libkvstore.dylib` on @@ -36,17 +51,29 @@ found through the platform's normal search. Set `KVSTORE_LIBRARY` (the identity's `{PREFIX}_LIBRARY` variable) to a full path to load a specific build. On iOS, and on macOS when the file isn't found, the bindings look the symbols up in the running executable, which is how a statically linked or -embedded library is found. `weaveffi package` bundles prebuilt desktop -libraries under `native//` and tries those first. - -Before any other call, the bindings check the library: `kvstore_abi_version()` -must report 3, and every top-level module's checksum function must match the -value the bindings were generated against. A mismatch throws a `StateError` -that names the module and asks you to regenerate the bindings. - -The package needs Dart 3.12 or newer and the `ffi` package. Flutter native -assets (build hooks that compile the library) aren't generated; ship the -library with your app as you would any other native dependency. +embedded library is found. + +`weaveffi package` writes the pub package `dart/{package}/` with each desktop +platform's library under `native//`. The packaged loader picks the +running platform's library (by `Abi.current()`) and looks for it in the +package itself, resolved with `Isolate.resolvePackageUriSync`, then under the +working directory (for a compiled app that copied `native/` next to itself), +and only then on the system search path. `{PREFIX}_LIBRARY` still wins over +all of them. See [Packaging](../guides/packaging.md). + +### Load-time checks + +Before any other call, the bindings check the library and throw a +`NativeLibraryError` (an `Error`) if it doesn't match: + +- `kvstore_abi_version()` must report revision 4. +- For every top-level module, the bindings embed the contract entries they + were generated with (one `(id, hash, path)` record per declaration), call + `kvstore_{module}_contract`, and require each entry to be present with an + equal hash. A missing entry fails with "kvstore: kv.Store.put is missing + from the library"; a different hash with "kvstore: kv.Store.put changed + since these bindings were generated". Declarations the library has and the bindings don't are + fine, so adding a function never breaks a deployed binding. ## Type mapping @@ -57,18 +84,27 @@ library with your app as you would any other native dependency. | `bool` | `bool` | C `bool` | | `string` | `String` | UTF-8 `(ptr, len)`, so interior NULs survive | | `bytes` | `List` (a `Uint8List` when returned) | `(ptr, len)` | -| Record | a class with final fields and named constructor arguments | value buffer | +| Record | a `final class` with final fields, named constructor arguments, and value equality | value buffer | | Plain enum | an enhanced `enum` with `value` and `fromValue` | `int32_t` | -| Rich enum | a `sealed` class with one subclass per variant | value buffer | +| Rich enum | a `sealed` class with one `final` subclass per variant | value buffer | | Interface | a `final class` wrapper | object pointer | -| `T?` | `T?` | value buffer (`Interface?` is a nullable pointer) | +| `T?` | `T?` | value buffer (`Interface?` is a nullable pointer, `Cb?` a null vtable) | | `[T]`, `{K:V}` | `List`, `Map` | value buffer | | `iter` | `Iterable` | native iterator | | Callback interface | an `abstract class` you implement | `ctx` plus vtable | -Wrapper functions and members use lowerCamelCase; types use UpperCamelCase. -A name that collides with a Dart keyword or a type the bindings use gains a -trailing `_` (`class_`, `String_`). +Records and rich-enum variants implement `==`, `hashCode`, and `toString`: +lists, maps, and byte arrays compare element by element, and interface +wrappers by identity. Wrapper functions and members use lowerCamelCase; +types use UpperCamelCase. A name that collides with a Dart keyword or a type +the bindings use gains a trailing `_` (`class_`, `String_`), as does an +interface member named like a member every wrapper declares or inherits +(`dispose`, `hashCode`, `toString`, `runtimeType`, `noSuchMethod`): a method +`dispose` is `dispose_()`. See [Naming](../reference/naming.md#identifiers-in-generated-code). + +Each distinct optional, list, or map type gets one private codec pair +(`_packListOfEntry`, `_unpackMapOfStringToString`), built on the runtime's +generic `writeList`/`readList` helpers, so no call site inlines a loop. ## Objects and lifetime @@ -78,11 +114,11 @@ collected. Disposing twice is harmless, and using a disposed wrapper throws a `StateError`. ```dart -final store = kv.Store.open('/tmp/data'); // factory constructor -store.put('key', [1, 2, 3], kv.EntryKind.persistent, null); -final copy = store.share(); // a second reference +final store = Store.open('/tmp/data'); // factory constructor +store.put('key', [1, 2, 3], EntryKind.persistent, null); +final copy = store.share(); // a second reference store.dispose(); -copy.count(); // still valid +copy.count(); // still valid copy.dispose(); ``` @@ -95,24 +131,33 @@ element) is encoded as a fresh reference, so the wrapper stays usable. ## Errors -Every failure is a `NativeException` with a `code` and a `message`. A -module's error domain becomes a subclass, with one subclass per code: +A module's error domain becomes a sealed exception class, extending +`NativeException`, with one final subclass per code. Payload fields are typed +properties, and the code's documented message is the default for the +optional trailing `message` argument: ```dart try { store.get('missing'); -} on kv.KeyNotFoundException catch (e) { - print('${e.code}: ${e.message}'); // 1001: key not found -} on kv.KvException { +} on KeyNotFoundException catch (e) { + print('${e.code}: ${e.message} (${e.key})'); // 1001: key not found: missing (missing) +} on KvException { // any other KvError code } ``` -Only functions declared `throws` raise domain exceptions. The runtime codes -are constants on `NativeException`: `genericCode` (-1), `panicCode` (-2), -`marshalCode` (-3), `foreignCode` (-4, a callback implementation threw), and -`cancelledCode` (-5, raised as `CancelledException`). A domain code with -fields carries them as typed properties on its exception. +Only functions declared `throws` raise exceptions. A positive code becomes +its domain subclass; a negative runtime code becomes a plain +`NativeException` with that code. The runtime codes are constants on +`NativeException`: `genericCode` (-1), `panicCode` (-2), `marshalCode` +(-3), `foreignCode` (-4, a callback implementation failed), and +`cancelledCode` (-5, raised as `CancelledException`). + +A failure of a function that isn't declared `throws` is a bug in the +producer, so it follows the [trap policy](../guides/errors-and-memory.md#the-trap-policy): +the binding throws a `NativeError`, an `Error` (not an `Exception`) carrying +the runtime `code` and the producer's `message`. Cancellation always +surfaces as `CancelledException`. ## Async and cancellation @@ -123,12 +168,12 @@ decoded on the event loop. A cancellable function takes an optional named `CancelToken`: ```dart -final token = kv.CancelToken(); -final pending = store.compact(cancelToken: token); +final token = CancelToken(); +final pending = store.compact(60000, cancelToken: token); token.cancel(); try { await pending; -} on kv.CancelledException { +} on CancelledException { // completed with code -5 without waiting for the work } ``` @@ -141,35 +186,71 @@ bindings create a native token for each call, cancel it when the ## Callbacks Implement the generated abstract class and pass an instance. The bindings -keep it in a table until the producer releases it, so you don't need to hold -another reference. +keep it in a table until the producer releases it through the vtable's +`free`, so you don't need to hold another reference. An optional callback +parameter (`Policy?`) takes `null` for none, which passes a null vtable. ```dart -class Logger extends kv.EvictionListener { +class AdmitAll implements Policy { @override - bool onEvict(kv.Entry entry, kv.EvictionReason reason) { - print('evicted ${entry.key}'); - return true; + Entry admit(Entry entry) { + if (entry.key.startsWith('secret')) { + throw RejectedException(entry.key, 'no secrets', 'secrets are not stored'); + } + return entry; } + + @override + Store route(String key, Store home) => home; } -store.setEvictionListener(Logger()); +store.setPolicy(AdmitAll()); ``` +A method that returns a value hands it to the producer in the shape the ABI +expects: a direct value by value; an object as a fresh strong reference the +producer adopts; and a string, bytes, or record as a run the bindings +allocate with `{prefix}_alloc` and the producer frees. Object arguments are +owned by the implementation; call `dispose()` on them when you're done (or +let the finalizer release them). + +Exceptions never unwind into the producer. In a method declared `throws`, +throwing the module's domain exception (here a `KvException` subclass) +reports its code and its fields, encoded as a value buffer, so the producer +sees exactly that error: `put` above fails with the same +`RejectedException`. Any other exception, or any exception from a method +that isn't declared `throws`, reports `foreignCode` (-4) with the +exception's text. + +### Threading + How a method runs depends on its return type: -- A method that returns a value is a `NativeCallable.isolateLocal` +- **A method that returns a value** is a `NativeCallable.isolateLocal` trampoline. It runs synchronously, so the producer must call it on the - isolate's thread while a call from Dart is in progress. If it throws, the - producer's call fails with `foreignCode` (-4) and the exception's text. -- A void method is a `NativeCallable.isolateGroupBound` forwarder. The + isolate's thread while a call from Dart is in progress, as a policy or a + filter consulted during the call is. +- **A void method** is a `NativeCallable.isolateGroupBound` forwarder. The producer may call it from any thread; the forwarder copies its arguments and returns at once, and the method runs later on the event loop, in the zone that passed the instance. An exception there is an uncaught error of - that zone. - -Object arguments are owned by the implementation; call `dispose()` on them -when you're done. + that zone, and a void method can't fail the producer's call even when it's + declared `throws`. The vtable's `free` is forwarded the same way, after + every call made before it. + +A value-returning method called from any other thread (a producer worker +thread, an async executor) aborts the process. That's a Dart VM rule, not a +choice of these bindings. Only an `isolateLocal` callable can run the +isolate's code synchronously, and the VM aborts before any Dart code runs +when one is called from a thread that isn't running the isolate. The entry +points that accept any thread can't take its place: an `isolateGroupBound` +callable runs outside the isolate, even on the isolate's own thread, so it +can neither run the method nor hand it to the isolate while the isolate is +blocked in the very call that invoked it, and a `listener` returns before +the method runs. So don't give a producer an implementation with +value-returning methods that it calls from its own threads. In the kvstore +sample, a `Listener`'s `accepts` is consulted by `compact` on a worker +thread, so no listener should be subscribed while a compaction runs. ## Iterators @@ -179,7 +260,7 @@ iterator is destroyed when iteration finishes or fails, or by a finalizer when an iteration is abandoned (`first`, `take`, a `break`) and collected. ```dart -for (final key in store.listKeys('user:')) { +for (final key in store.keys('user.')) { print(key); } ``` @@ -195,15 +276,15 @@ functions. ## Known limitations -- A value-returning callback method invoked from a thread other than the - isolate's aborts the process. That's a Dart VM rule for - `NativeCallable.isolateLocal`, and pure Dart can't detect the thread first: - an any-thread entry point can't re-enter the isolate to run the method. - Producers must call value-returning methods synchronously during a call - from Dart. +- A value-returning callback method called from a thread that isn't the + isolate's (a producer worker thread, an async executor) aborts the + process; void methods are safe from any thread (see + [Threading](#threading)). - Void callback methods are always delivered asynchronously, even when the producer calls them on the isolate's thread, so their exceptions can't fail the producer's call. -- `NativeCallable.isolateGroupBound` is marked experimental in the Dart SDK. +- `NativeCallable.isolateGroupBound`, which callback interfaces rely on, is + marked experimental in the Dart SDK ("may change in the future"), so a + future SDK release could change how it's enabled. - Library loading uses `DynamicLibrary`; Flutter's native-assets build hooks aren't generated. diff --git a/docs/src/generators/dotnet.md b/docs/src/generators/dotnet.md index f08b4a65..f134201b 100644 --- a/docs/src/generators/dotnet.md +++ b/docs/src/generators/dotnet.md @@ -1,9 +1,10 @@ # .NET The .NET target emits a standalone C# class library over the C ABI -(revision 3). Native calls go through source-generated +(revision 4). Native calls go through source-generated [`[LibraryImport]`](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke-source-generation) -declarations whose slots are all blittable, so the bindings need no runtime +declarations whose slots are all blittable, and callbacks go through +`[UnmanagedCallersOnly]` function pointers, so the bindings need no runtime marshalling and are trim- and AOT-friendly. The project targets `net8.0` and builds with any newer SDK. @@ -13,32 +14,32 @@ For a package named `kvstore`: ```text dotnet/ - Kvstore.cs generated API: types, wrappers, native imports + Kvstore.cs generated API: types, wrappers, codecs, native imports Runtime.cs fixed runtime: exceptions, loader, codec, async and iterator helpers Kvstore.csproj SDK-style project with the NuGet metadata README.md ``` The namespace, assembly name, and NuGet id are the package name in -PascalCase (`my-kv` becomes `MyKv`). Set `namespace` to override all three: +PascalCase (`my-kv` becomes `MyKv`). Set `name` to override all three: ```toml [generators.dotnet] -namespace = "Acme.Storage" -strip_module_prefix = true # the default; false keeps `kv_get`-style names +name = "Acme.Storage" ``` Each module becomes a static class named by its full path (`kv` is `Kv`, -`kv.stats` is `KvStats`); records, enums, interfaces, and exceptions sit -directly in the namespace. +`kv.stats` is `KvStats`); records, enums, interfaces, callback interfaces, +and exceptions sit directly in the namespace, since every type name is +unique across the API. ## Build and load Reference the generated project from yours, or pack it: ```bash -dotnet add reference generated/dotnet/Kvstore.csproj -dotnet pack generated/dotnet/Kvstore.csproj -c Release +dotnet add reference bindings/dotnet/Kvstore.csproj +dotnet pack bindings/dotnet/Kvstore.csproj -c Release ``` The bindings load the native library by its base name (`kvstore`), so .NET @@ -47,25 +48,43 @@ app and on the platform's library path. Set `KVSTORE_LIBRARY` (the `{PREFIX}_LIBRARY` variable) to a full path to load a specific file. The override is a `DllImportResolver` on the generated assembly. -`weaveffi package` additionally bundles prebuilt libraries under -`runtimes//native/`, where NuGet resolves them at restore time. +`weaveffi package` writes the project to `dotnet/Kvstore/` with each desktop +platform's library under `runtimes//native/`, where NuGet resolves it +at restore time, and runs `dotnet pack` on it to produce +`dotnet/Kvstore.{version}.nupkg` (so it needs the .NET SDK). See +[Packaging](../guides/packaging.md). + +## Load-time checks The first native call runs `NativeMethods`' static constructor, which -installs the resolver, checks `{prefix}_abi_version()` against 3, and -compares every top-level module's contract checksum with the value the -bindings were generated from: +installs the resolver, checks `{prefix}_abi_version()` against 4, and then +checks every top-level module's [contract table](../reference/abi.md#load-time-checks). +The bindings embed the `(id, hash, path)` entry of every declaration they +were generated with: ```csharp -static partial void VerifyChecksums() +static partial void VerifyContracts() { - VerifyChecksum("shared", &kitchen_sink_shared_checksum, 0x42c4ce2c8d0af052UL); - VerifyChecksum("kitchen", &kitchen_sink_kitchen_checksum, 0xd83da75b24b545ffUL); + VerifyContract("kvstore_kv_contract", &kvstore_kv_contract, new (ulong, ulong, string)[] + { + (0xbe2b91d3c1f42f80UL, 0xf47a03bed8a6a39aUL, "kv.Store.put"), + // ... one entry per declaration in `kv` and its submodules + }); + VerifyContract("kvstore_report_contract", &kvstore_report_contract, new (ulong, ulong, string)[] + { + // ... + }); } ``` -A mismatch throws `InvalidOperationException` naming the module (surfaced -as the `InnerException` of a `TypeInitializationException`), so stale -bindings fail before any call can misread memory. +Each entry must be in the library's table with an equal hash; entries the +library has and the bindings don't are fine, so a library that only adds +declarations keeps working with older bindings. A failure throws +`InvalidOperationException` naming the declaration (`kv.Store.put is +missing from the library kvstore` or `kv.Store.put changed since these +bindings were generated`), surfaced as the `InnerException` of a +`TypeInitializationException`, so stale bindings fail before any call can +misread memory. ## Type mapping @@ -78,7 +97,7 @@ bindings fail before any call can misread memory. | C-style enum | `enum` with the IDL discriminants | `int32_t` | | record | `sealed class` with get-only properties | value buffer | | rich enum | `abstract class` with one nested sealed class per variant | value buffer | -| `T?` | `T?` | value buffer (`Iface?` is a nullable pointer) | +| `T?` | `T?` | value buffer (`Iface?` is a nullable pointer, `Cb?` a null vtable) | | `[T]` | `T[]` | value buffer | | `{K: V}` | `Dictionary` | value buffer | | interface | `sealed class : IDisposable` | object pointer | @@ -87,6 +106,10 @@ bindings fail before any call can misread memory. Strings are encoded to UTF-8 and pinned for the call; returned strings, bytes, and buffers are copied and released with `{prefix}_free_bytes`. +Records and rich enums encode themselves (`WriteTo` and `ReadFrom`), and +each distinct optional, list, or map type gets one encode and decode pair in +the internal `FfiCodecs` class (`WriteListOfEntry`, +`ReadMapOfStringToStore`), which every call site and field shares. ## Objects and lifetime @@ -102,32 +125,71 @@ passing one as an argument, throws `ObjectDisposedException`. Returned objects, object fields of decoded records, and objects handed to a callback each arrive as a new wrapper that owns its reference. Two wrappers are equal (`Equals`, `GetHashCode`) when they reference the same -native object. Writing an -object into a value buffer clones its reference first, so the producer -can keep it. +native object. Writing an object into a value buffer clones its reference +first, so the producer can keep it. + +An interface member spelled like a member the wrapper declares or inherits +(`Dispose`, `Handle`, `Equals`, `ToString`, ...) or like the interface +itself gains a trailing `_`: an IDL method `dispose` is `Dispose_()` +([reserved member names](../reference/naming.md#identifiers-in-generated-code)). ## Errors -Every failure derives from `NativeException`, whose `Code` is the native -error code. Runtime codes are constants on it: `GenericErrorCode` (-1), -`PanicErrorCode` (-2), `MarshalErrorCode` (-3), and `ForeignErrorCode` -(-4). Each error domain becomes a typed subclass with one constant per -code; payload fields land in `Exception.Data` keyed by field name: +What a failed call throws depends on whether it's declared `throws` +([the trap policy](../guides/errors-and-memory.md#the-trap-policy)): + +- **A throwing call** throws its module's domain exception. Each error + domain becomes an abstract class deriving from `NativeException`, with one + nested sealed class per code whose fields are typed properties; catch the + nested class for one code or the domain class for any of them. A runtime + code (-1 generic, -2 panic, -3 marshalling failure, -4 a failed callback) + throws `NativeException` itself, whose `Code` property carries it and + whose constants name it (`GenericErrorCode`, `PanicErrorCode`, + `MarshalErrorCode`, `ForeignErrorCode`). +- **A call that isn't `throws`** can't fail by contract, so a failure is a + producer bug: it throws `NativeBugException`, an + `InvalidOperationException` whose `Code` is the runtime code and whose + message names the code and the producer's message. +- **Cancellation** (-5) always throws `OperationCanceledException`. + +For the `kvstore` sample's `KvError` domain: + +```csharp +public abstract class KvException : NativeException +{ + /// key not found + public sealed class KeyNotFound : KvException + { + /// The domain code, 1001. + public const int ErrorCode = 1001; + + /// The key that was looked up. + public string Key { get; } + + public KeyNotFound(string key, string? message = null) : base(ErrorCode, message ?? "key not found") + { + Key = key; + } + } + + // ... Expired, StoreFull, InvalidPath, Rejected +} +``` ```csharp -/// Poke the gadget a number of times, failing on a negative count -/// The call reported a KitchenErrorsError code. -public int Poke(int times) +try +{ + store.Get("nope"); +} +catch (KvException.KeyNotFound e) { - var ffiErr = default(FfiError); - var ffiResult = NativeMethods.kitchen_sink_kitchen_Gadget_poke(Handle, times, &ffiErr); - if (ffiErr.Code != 0) throw Ffi.TakeError(&ffiErr, global::KitchenSink.KitchenErrorsException.FromError); - return ffiResult; + Console.WriteLine($"{e.Code} {e.Key}: {e.Message}"); // 1001 nope: key not found: nope } ``` -The cancelled code (-5) surfaces as `OperationCanceledException`. If the -API declares a type named `NativeException`, the base class takes the +The message is the producer's (a Rust producer's `Display` output); a code +class constructed in C# defaults to the IDL message. If the API declares a +type named `NativeException` or `NativeBugException`, that class takes the namespace as a prefix (`KvstoreNativeException`). ## Async and cancellation @@ -166,34 +228,76 @@ public static Task DoCancellable(string input, CancellationToken cancell `CancelToken()` creates the native token and registers the `CancellationToken` to cancel it. When the producer completes with -5, the -task becomes canceled, so `await` throws `TaskCanceledException`. The -consumer's token reference is destroyed after completion, and a -registration racing the completion never touches a destroyed token. +task becomes canceled, so `await` throws `TaskCanceledException` (an +`OperationCanceledException`). The consumer's token reference is destroyed +after completion, and a registration racing the completion never touches a +destroyed token. ## Callbacks -A callback interface becomes a C# interface the consumer implements: +A callback interface becomes a C# interface the consumer implements. Its +methods take and return the same C# types as the rest of the API: ```csharp -public interface IReadyListener +public interface IPolicy +{ + /// Reported to the library with its code and fields. + Entry Admit(Entry entry); + + Store Route(string key, Store home); +} + +public interface ILoader { - /// Fires when an item is ready - void OnReady(int code, string msg); - /// Receives the item itself and says whether to keep listening - bool OnItem(Item item, Gadget gadget); + string Name(); + Store? Fallback(string key); + /// Reported to the library with its code and fields. + byte[] Load(string key); } ``` Passing an implementation pins it in a `GCHandle` (the `ctx`) and hands the -producer one process-wide vtable of `[UnmanagedCallersOnly]` trampolines; -the vtable's `free` entry releases the handle when the producer drops its -last reference. Trampolines run on whatever thread the producer calls -from. String, bytes, and buffer arguments are copied; object arguments are -adopted. An exception thrown by the implementation never unwinds into -native code: the trampoline reports it with `{prefix}_error_set(out_err, --4, message)`, and the call that triggered it throws `NativeException` with -`ForeignErrorCode` and the original message. That holds whether the -producer's trait method returns `T` or `Result`. +producer one process-wide vtable of `[UnmanagedCallersOnly]` trampolines, +which starts with the `{size, flags, free}` header (`size` is the vtable's +real size as C# lays it out). The `free` entry releases the handle when the +producer drops its last reference, possibly on another thread. An optional +callback parameter (`ILoader? loader`) passes a null vtable for `null`. + +Trampolines run on whatever thread the producer calls from. String, bytes, +and buffer arguments are copied, and object arguments are adopted: each is a +new wrapper the implementation owns. Return values go back to the producer +by family: + +- a scalar, `bool`, or C-style enum is the C return; +- an object is a fresh reference the producer adopts (`CloneHandle()`); a + `null` for a required object reaches the producer as null, which it + rejects as a marshalling failure (-3); +- a string, bytes, record, rich enum, optional, list, or map is encoded + into a run allocated with `{prefix}_alloc` and written to the method's + out slots, which the producer adopts and frees. + +An exception thrown by an implementation never unwinds into native code. +A method declared `throws` that throws its module's domain exception +reports the code, the message, and the code's fields (encoded as the error +payload with `{prefix}_error_set_payload`), so the producer sees exactly +that domain error: + +```csharp +public Entry Admit(Entry entry) +{ + if (entry.Key.StartsWith("secret")) + { + throw new KvException.Rejected(entry.Key, "no secrets", "secrets are not stored"); + } + return entry; +} +``` + +Any other exception, or one thrown from a method that isn't `throws`, is +reported as -4 with the exception's message. What the failure means for the +call in progress is the producer's decision. When a Rust producer +propagates it with `?`, the original (throwing) call throws that same domain +exception, or `NativeException` with `ForeignErrorCode` for a -4. ## Iterators @@ -206,11 +310,9 @@ throws `InvalidOperationException`. ## Known limitations -- Every module's types share one namespace, so two modules declaring the - same type name collide. -- A module class whose name equals the namespace (an `events` module in - the `Events` package) must be reached as `Events.Events` or through - `using static`. +- A module class whose name equals the namespace (a `calculator` module in + the `Calculator` package) must be reached as `Calculator.Calculator` or + through `using static`. - Only one `DllImportResolver` can exist per assembly. If you compile the generated sources into an assembly that already has one, the `{PREFIX}_LIBRARY` override is skipped and default probing applies. @@ -218,3 +320,5 @@ throws `InvalidOperationException`. - Object references cloned into a value buffer leak if encoding a later field throws before the call (for example a disposed wrapper in the same list). +- C# can name an undeclared enum value (`(Color)3`); the producer rejects + it as a marshalling failure (-3). diff --git a/docs/src/generators/go.md b/docs/src/generators/go.md index cb2366b4..433afff4 100644 --- a/docs/src/generators/go.md +++ b/docs/src/generators/go.md @@ -2,7 +2,8 @@ The Go target emits a self-contained cgo module that binds the library's C ABI. Records and rich enums are plain Go values, interfaces are -reference-counted wrappers with `Close`, async functions take a +reference-counted wrappers with `Close`, throwing calls return `error` +values you match with `errors.As`, async functions block on a `context.Context`, and `iter` returns standard-library `iter.Seq` sequences, so the module requires Go 1.23 or later. @@ -12,39 +13,56 @@ For a library whose identity is `kvstore`: ```text go/ - go.mod module path (the package name, or `module_path`) + go.mod module path (the package name, or `name`) README.md kvstore.h a copy of the C header bindings.go the API: types, wrappers, trampolines, load-time checks - runtime.go errors, string and object helpers, the async bridge - codec.go the value-buffer writer and reader (only when needed) + runtime.go load-time checks, errors, strings, objects, callbacks, async + codec.go the value-buffer writer and reader ``` Every name comes from the package identity: | Name | Default | Override | |------|---------|----------| -| Module path | the package `name` | `[generators.go] module_path` | +| Module path | the package `name` | `[generators.go] name` | | Package name | the C prefix | none | | Linked library | `-l{library}` | none | | Header | `{library}.h` | none | ```toml [generators.go] -module_path = "github.com/example/kvstore" -strip_module_prefix = true # the default +name = "github.com/example/kvstore" ``` -With `strip_module_prefix`, module `kv`'s `delete` surfaces as `Delete`. -All modules render into one Go package, so a function whose stripped name -would clash with a type, a constant, or another module's function keeps its -module prefix (`directory.card` next to a `Card` record becomes -`DirectoryCard`). +All modules render into one Go package, and Go spells every exported name +in PascalCase with Go's initialisms (`user_id` is `UserID`, `ttl_seconds` +is `TTLSeconds`): + +| Declaration | Go name | +|-------------|---------| +| function `kv.stats.summarize` | `Summarize` | +| interface `Store` | `*Store` | +| constructor `Store.open` | `OpenStore` (and `new` is `NewStore`) | +| method `Store.get_or_load` | `(*Store).GetOrLoad` | +| static `Store.default_capacity` | `StoreDefaultCapacity` | +| C-style enum `EntryKind`, variant `Volatile` | `EntryKind`, `EntryKindVolatile` | +| rich enum `Change`, variant `Put` | `Change`, `ChangePut` | +| error domain `KvError`, code `KeyNotFound` | `KvError`, `*KeyNotFoundError` | + +A free function whose bare name would clash with a type, an enum +constant, a constructor, a static, an error type, the runtime's `Error` +or `DebugLive`, or another free function's bare name is prefixed with its +module path: the kvstore sample's async `kv.open_store` sits beside the +`OpenStore` constructor, so it's `KvOpenStore` (and a function in module +`kv.stats` would get a `KvStats` prefix). Methods never clash this way, +since each lives on its own type. ## Build and load -The bindings link with `#cgo LDFLAGS: -l{library}` and include the bundled -header, so a consumer only supplies the library's directory: +The bindings include the bundled header and link with +`#cgo LDFLAGS: -l{library}`, so a consumer supplies only the library's +directory: ```sh export CGO_LDFLAGS="-L/path/to/lib" @@ -52,27 +70,71 @@ export LD_LIBRARY_PATH="/path/to/lib" # DYLD_LIBRARY_PATH on macOS go build ./... ``` -Depend on the module with `require` (plus `replace` for a local checkout) -and import it by its module path. The library is bound when the program is -linked and loaded by the platform's dynamic loader, so the -`{PREFIX}_LIBRARY` override other targets honor doesn't apply; use the -loader's search path or an rpath instead. +cgo links the library into the program when it's built, and the +platform's dynamic loader finds it when the program starts. The bindings +never open the library themselves, so the `{PREFIX}_LIBRARY` override the +other targets read at load time doesn't exist for Go; to choose the library +at run time, set the loader's search path (`LD_LIBRARY_PATH`, +`DYLD_LIBRARY_PATH`, or `PATH` on Windows). To record the directory in the +binary instead, add an rpath when building: -At package initialization, `init` checks the library's ABI revision and the -contract checksum of every top-level module, and panics with the stale -module's name on a mismatch: +```sh +export CGO_LDFLAGS="-L/path/to/lib -Wl,-rpath,/path/to/lib" +``` + +`weaveffi package` writes the module to `go/{package}/` with each desktop +platform's library under `lib//` and replaces the single link +line with one per GOOS and GOARCH, so cgo picks the right library with no +`CGO_LDFLAGS`: + +```go +/* +#cgo darwin,arm64 LDFLAGS: -L${SRCDIR}/lib/darwin-arm64 -Wl,-rpath,${SRCDIR}/lib/darwin-arm64 +#cgo linux,amd64 LDFLAGS: -L${SRCDIR}/lib/linux-x64 -Wl,-rpath,${SRCDIR}/lib/linux-x64 +#cgo windows,amd64 LDFLAGS: -L${SRCDIR}/lib/windows-x64 +#cgo LDFLAGS: -lkvstore +*/ +``` + +`${SRCDIR}` expands to the module's directory (in the module cache, for a +dependency), so a program built on a machine runs there without a loader +path. A binary you ship elsewhere still needs the library beside it or on +the loader's path: copy it and link with an rpath such as +`-Wl,-rpath,$ORIGIN` (`@executable_path` on macOS). Windows has no rpath; +put the DLL next to the executable or on `PATH`. Android, iOS, and +`wasm32` libraries aren't bundled. Publish the module by pushing the +directory to the module path's repository; see +[Packaging](../guides/packaging.md). + +## Load-time checks + +Importing the package runs `init`, which checks the library's ABI revision +and then, for every top-level module, the library's +[contract table](../reference/abi.md#load-time-checks) against the entries +the bindings were generated with: ```go func init() { wvCheckABI() - wvCheckModule("shared", 0x42c4ce2c8d0af052, uint64(C.kitchen_sink_shared_checksum())) - wvCheckModule("kitchen", 0xd83da75b24b545ff, uint64(C.kitchen_sink_kitchen_checksum())) + var n C.size_t + table := C.kvstore_kv_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0x0969575bfbb012d7, 0xebd38766e3532c4f, "kv.Store.fork"}, + // ...one entry per declaration + }) + table = C.kvstore_report_contract(&n) + wvCheckContract(table, n, []wvContractEntry{ + {0xa21a2e7274bf28c6, 0x342e0b83b12d73ec, "report.render_report"}, + {0xd504fae45f64ab45, 0xd430b8de9d574a4c, "report.ReportError"}, + }) } ``` -`weaveffi package` bundles a prebuilt library per platform under -`lib//` and adds `${SRCDIR}`-relative search and rpath lines per -GOOS and GOARCH, so no `CGO_LDFLAGS` is needed. +A mismatch panics with an error naming the declaration: +`kvstore: kv.Store.put is missing from the library` or +`kvstore: kv.Store.put changed since these bindings were generated`. +Declarations the library has and the bindings don't are fine, so a library +that only adds functions keeps working with older bindings. ## Type mapping @@ -86,137 +148,192 @@ GOOS and GOARCH, so no `CGO_LDFLAGS` is needed. | C-style enum | `type E int32` plus `E{Variant}` constants | `int32_t` | | rich enum | sealed `interface` plus one struct per variant | value buffer | | record | plain struct with exported fields | value buffer | -| `T?` | `*T` (nil-able types such as slices stay as they are) | value buffer | +| `T?` | `*T`; a slice, map, rich enum, `[]byte`, or wrapper stays as it is and nil is none | value buffer | | `[T]`, `{K: V}` | `[]T`, `map[K]V` | value buffer | | interface | `*Iface` | object pointer | -| `Iface?` | `*Iface` (nil is none) | object pointer or NULL | -| callback interface | a Go `interface` you implement | `cgo.Handle` plus a static vtable | +| `Iface?` | `*Iface` (nil is none) | object pointer or null | +| callback interface | a Go `interface` you implement | handle plus a static vtable | +| `Cb?` | the same interface (nil is none) | handle plus a vtable, or a null vtable | | `iter` | `iter.Seq[T]` or `iter.Seq2[T, error]` | iterator handle | Strings pass a view of Go memory with no copy and no NUL terminator, so -they may contain `\x00`. Lengths are never narrowed to `C.int`. - -## Objects and lifetime +they may contain `\x00`. A Go string can hold bytes that aren't UTF-8; the +library rejects them as a marshalling failure (`-3`). -Each wrapper owns one strong reference. `Close() error` releases it (so a -wrapper satisfies `io.Closer`), and a finalizer releases it if you forget: +Each type that crosses inside a value buffer has one codec pair, +`wvWrite{T}` and `wvRead{T}`, built from the pairs of its parts and shared +by every parameter, result, field, and callback argument of that type: ```go -func (s *Gadget) Close() error { - runtime.SetFinalizer(s, nil) - s.ref.close() - return nil +func wvWriteMapStringStore(w *wvWriter, v map[string]*Store) { + wvWriteMap(w, v, (*wvWriter).writeString, wvWriteStore) } ``` +## Objects and lifetime + +Each wrapper owns one strong reference. `Close() error` releases it (so a +wrapper satisfies `io.Closer`), and a finalizer releases it if you forget. Every call borrows the native pointer for its duration: ```go -func (s *Gadget) Poke(times int32) (int32, error) { +func (s *Store) Get(key string) (Entry, error) { cSelf := s.native() defer s.ref.release() - var cErr C.kitchen_sink_error - cRet := C.kitchen_sink_kitchen_Gadget_poke(cSelf, C.int32_t(times), &cErr) + cKeyPtr, cKeyLen := wvStr(key) + var cRetLen C.size_t + var cErr C.kvstore_error + cRet := C.kvstore_kv_Store_get(cSelf, cKeyPtr, cKeyLen, &cRetLen, &cErr) if cErr.code != 0 { - return 0, wvMapKitchen(wvTakeError(&cErr)) + return Entry{}, wvKvError(wvTakeError(&cErr)) } - return int32(cRet), nil + return wvDecode(cRet, cRetLen, wvReadEntry), nil } ``` `Close` is idempotent and goroutine-safe. When it races a call in flight (even one that closes the wrapper from inside a callback), the reference is released as soon as that call returns, never mid-call. Using a wrapper after -`Close` panics. An object inside a record or list crosses as a token that -carries a fresh reference, so the wrapper you passed stays valid. +`Close`, or passing a nil `*Iface` where an object is required, panics. An +object inside a record, list, or map crosses as a token that carries a +fresh reference, so the wrapper you passed stays valid. Two wrappers may +refer to the same object (a method that returns its receiver hands back a +new wrapper); each is closed on its own. + +Because every wrapper has `Close`, an IDL method named `close` is +`Close_` ([reserved member names](../reference/naming.md#identifiers-in-generated-code)). ## Errors -A function that declares `throws: true` returns `(T, error)`. Domain codes -map to the module's error type, which `errors.As` selects: +A function declared `throws` returns `(T, error)`. A positive code comes +back as its code's type, a pointer to a struct holding the code's payload +fields and a `Message`, and every code type of a domain implements the +domain's sealed interface: ```go _, err := store.Get("missing") -var kerr *kvstore.KvError -if errors.As(err, &kerr) && kerr.Code == kvstore.KvErrorKeyNotFound { - // ... +var missing *kvstore.KeyNotFoundError +if errors.As(err, &missing) { + fmt.Println(missing.Key, missing.Code()) // missing 1001 +} +var kvErr kvstore.KvError +if errors.As(err, &kvErr) { + // any code of the kv domain } ``` -Codes with payload fields attach a `{Type}{Code}Payload` struct as -`Payload`. Anything a domain doesn't claim is the package's generic -`*Error`, which keeps the code: `-1` generic, `-2` producer panic, `-3` -marshalling failure, and `-4` a callback implementation that panicked. A -function without `throws` has a plain signature and panics with `*Error` -instead, since only a bug can make it fail. +`Error()` returns the library's message, or the code's default message when +`Message` is empty. A runtime code (`-1` generic, `-2` a panic in the +library, `-3` a marshalling failure, `-4` a callback failure) or a code +outside the domain comes back as the package's `*Error`, which keeps the +code and message and never matches a domain interface. + +A function that isn't `throws` can only fail because of a bug, so its +wrapper has a plain signature and panics with an `*Error` instead (see +[the trap policy](../guides/errors-and-memory.md#the-trap-policy)). The +panic is an ordinary recoverable one, and its text names the code and the +library's message, as in `kvstore: (code -3)`. ## Async and cancellation Async functions take a `context.Context` first and always return an error. They block the calling goroutine on a channel that a completion trampoline -fills from the producer's thread. A `cancellable` function creates a -native cancel token, cancels it when the context is done, and returns -`ctx.Err()` for the cancelled completion (code `-5`): +fills from the library's thread; call them from a goroutine of your own for +concurrency. A `cancellable` function creates a native cancel token, +cancels it when the context is done, and returns `ctx.Err()` +(`context.Canceled` or `context.DeadlineExceeded`) for the cancelled +completion (code `-5`), or `context.Canceled` if the library cancelled the +call on its own: ```go ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() -n, err := store.Compact(ctx) +removed, err := store.Compact(ctx, 60000) if errors.Is(err, context.DeadlineExceeded) { // The native call was cancelled. } ``` Any other async function returns `ctx.Err()` as soon as the context is done -and discards the native result when it arrives. An already-cancelled -context never launches the call. +and discards the native result when it arrives. A context that's already +done never launches the call. A throwing async function returns its domain +errors like a sync one; any other panics on failure. ## Callbacks A callback interface is a Go interface. Passing an implementation stores it -in a `cgo.Handle` and hands the producer one static vtable of `//export` -trampolines; the producer's `free` entry deletes the handle. The producer -may call methods from any thread. A panic inside a method is recovered and -reported to the producer as code `-4` with the panic's text, whether the -producer's method returns a plain value or a `Result`, and the caller sees -it as `*Error`. +in a handle table and hands the library the address of one static vtable +per interface: the header's `size` and `flags`, the `free` entry that +deletes the handle once the library is done, and one `//export` trampoline +per method. The library may call methods from any thread, and `free` may run +on any thread too. ```go -type ReadyListener interface { - // OnReady: Fires when an item is ready - OnReady(code int32, msg string) - // OnItem: Receives the item itself and says whether to keep listening - OnItem(item Item, gadget *Gadget) bool +type Policy interface { + // Admit an entry about to be stored, returning it as it should be + // stored. ... + // + // Return a KvError to report that code, with its fields, to the native + // caller. + Admit(entry Entry) (Entry, error) + // Route returns the store `key` belongs in: ... + Route(key string, home *Store) *Store } ``` +- **Arguments.** Strings, bytes, and buffered values are copied or decoded + for the call. An object argument (`home` above) is a new wrapper that + owns its own reference: keep it, return it, or `Close` it (the finalizer + releases it otherwise). +- **Returns.** A method may return any type but an iterator or a callback + interface. A direct value returns by value; a string, `[]byte`, or + buffered value is copied into a run allocated with the library's + allocator; an object returns a fresh reference to the wrapper's object + (the wrapper stays yours). Returning nil for a required object, or a value + the library can't read (text that isn't UTF-8, an undeclared enum value), + fails the call in progress with `-3`. +- **Errors.** A method declared `throws` returns `(T, error)` or `error`. + Returning one of the domain's code types (for example + `&kvstore.RejectedError{Key: k, Reason: "no secrets", Message: "secrets + are not stored"}`) reports that code, its message, and its payload + fields, so the original caller receives exactly that error. Any other + error reports `-4` with its text. A method that panics, throwing or not, + is recovered and reported as `-4` with the panic's text; nothing unwinds + into the library. +- **Optional callbacks.** A `Cb?` parameter accepts nil, which passes a null + vtable. A required one panics on nil. + ## Iterators An `iter` return becomes a lazy sequence: each `range` launches the native iterator, pulls one element per step, and destroys the iterator when -the loop ends, including on an early `break`. A throwing function yields -its error as a final `(zero, err)` pair of an `iter.Seq2[T, error]`. +the loop ends, including on an early `break`. A throwing function returns +an `iter.Seq2[T, error]` that yields its error as a final `(zero, err)` +pair: ```go -for g := range kitchen_sink.StreamGadgets() { - fmt.Println(g.Describe()) - g.Close() +for key, err := range store.Keys(nil) { + if err != nil { + return err + } + fmt.Println(key) } ``` ## Leak checks -`DebugLive(kind)` reports the producer's live objects (0), callbacks (1), -iterators (2), cancel tokens (3), and returned allocations (4). It's always -0 unless the library was built with the `leak-check` feature. +`DebugLive(kind)` reports the library's live objects (0), callbacks (1), +iterators (2), cancel tokens (3), and byte runs (4), and `DebugLive(-1)` is +1 when the library counts at all. Every count is 0 unless the library was +built with the `leak-check` feature. Wrappers you don't close are released +by finalizers, so run `runtime.GC()` before reading the counts. ## Limitations -- All modules share one Go package, so two modules can't declare types - with the same name. -- The library is bound at link time, so there's no runtime library path - override. -- Callback methods can only return direct values (scalars, `bool`, and - C-style enums). +- All modules share one Go package, which the global naming rules make + safe; only a free function whose Go name collides gains its module path + (see [What's generated](#whats-generated)). +- The library is bound at link time, so there's no run-time override of + which library file loads beyond the platform loader's search path. - A non-cancellable async call abandoned by its context keeps running in - the producer until it finishes. + the library until it finishes. diff --git a/docs/src/generators/kotlin.md b/docs/src/generators/kotlin.md index 9acca90d..cf0c9ebc 100644 --- a/docs/src/generators/kotlin.md +++ b/docs/src/generators/kotlin.md @@ -1,7 +1,7 @@ # Kotlin The Kotlin target emits a Gradle library module: Kotlin sources over a JNI -shim that calls the producer's C ABI (revision 3). The default flavor is an +shim that calls the producer's C ABI (revision 4). The default flavor is an Android library whose shim builds through the NDK's CMake; the `jvm` flavor is a plain Kotlin/JVM library for desktop and server JVMs. Records become data classes, interfaces `AutoCloseable` wrappers, async functions `suspend` @@ -23,9 +23,10 @@ kotlin/ kitchen_sink_jni.c the JNI shim src/main/kotlin/kitchen_sink/ Kitchen.kt, Shared.kt one file per top-level module - JniBridge.kt internal: one native per C symbol - Runtime.kt FfiException, NativeIterator, loader, lifetime - Buffers.kt value-buffer codec (when the API needs it) + JniBridge.kt internal: one native per C symbol, callback dispatch + Runtime.kt FfiException, NativeBugException, NativeIterator, loader, lifetime + Buffers.kt value-buffer reader and writer (when the API needs them) + Codecs.kt one codec per composite type (when the API has any) Async.kt coroutine bridge (when the API has async) ``` @@ -38,44 +39,74 @@ branding. ```toml [generators.kotlin] -package = "com.example.kitchen" # default: the C prefix -flavor = "jvm" # "android" (default) or "jvm" -strip_module_prefix = true # default +name = "com.example.kitchen" # the Kotlin package; default: the C prefix +flavor = "jvm" # "android" (default) or "jvm" +min_sdk = 21 # Android minSdk (default 21) +compile_sdk = 35 # Android compileSdk (default 35) ``` ## Building and loading -The tree builds with Gradle as-is: `settings.gradle.kts` declares -`google()`, `mavenCentral()`, and the Gradle plugin portal, and -`./gradlew publishToMavenLocal` publishes the module. +The tree builds with Gradle as-is (no wrapper is generated), and +`gradle publishToMavenLocal` publishes the module. `settings.gradle.kts` +declares `mavenCentral()` and the Gradle plugin portal, plus `google()` for +the Android flavor. The Android flavor uses the Android Gradle plugin 8.7.3 +and Kotlin 2.0.21 and compiles to Java 8 bytecode; the JVM flavor uses +Kotlin 2.0.21. The shim's `CMakeLists.txt` needs CMake 3.18 or later, and an +API with async functions depends on `kotlinx-coroutines-core` 1.9.0. - **Android.** Put the producer library for each ABI in - `src/main/jniLibs//libkitchen_sink.so`. The NDK build links - `libkitchen_sink_jni.so` against it and the AAR packages both. `minSdk` is - 21. -- **JVM.** Build the shim with CMake, pointing it at the producer library, - and put the result on `java.library.path`: + `src/main/jniLibs//libkitchen_sink.so`. When + `libkitchen_sink_jni.so` sits next to it (prebuilt), the AAR packages both + as they are; otherwise the NDK's CMake build compiles the shim against the + producer. +- **JVM.** Put the producer and a prebuilt shim for each platform under + `src/main/resources/natives//` (the bindings extract them from + the classpath), or build the shim with CMake, pointing it at the producer + library, and put the result on `java.library.path`: ```bash cmake -S src/main/cpp -B build -DKITCHEN_SINK_LIBRARY_DIR=/path/to/lib/dir cmake --build build ``` -At first use the bindings load the producer (`System.loadLibrary("kitchen_sink")`) -and then the shim (`System.loadLibrary("kitchen_sink_jni")`). Set -`KITCHEN_SINK_LIBRARY` to an absolute path to load the producer from there -instead. `JNI_OnLoad` checks that the producer implements C ABI revision 3 -and that every top-level module's contract checksum matches the bindings; -otherwise loading fails with `UnsatisfiedLinkError` naming the module. +`weaveffi package` writes the Gradle project, plus a README, to +`kotlin/{name}/` with both layouts filled in: `weaveffi build` prebuilds the +shim for every Android ABI (with the NDK) and for each desktop platform the +host can compile for (with the JDK headers; not `windows-x64`), so neither +CMake nor the NDK is needed to build the AAR or JAR. Prebuilt Android shims +ship only when every Android ABI has one; otherwise Gradle builds them all +through the NDK's CMake. Building the AAR or JAR still takes Gradle +(`gradle assembleRelease`, `gradle jar`); see +[Packaging](../guides/packaging.md). + +At first use the bindings load the producer and then the shim, each from the +classpath's `natives//` when present, else with +`System.loadLibrary("kitchen_sink")` and +`System.loadLibrary("kitchen_sink_jni")`. Set `KITCHEN_SINK_LIBRARY` to an +absolute path to load the producer from there instead. + +`JNI_OnLoad` then makes the two [load-time checks](../reference/abi.md#load-time-checks). +It compares the producer's ABI revision with 4, and for every top-level +module it looks up each [contract](../reference/abi.md#load-time-checks) +entry the bindings were generated with in the producer's table +(`kitchen_sink_kitchen_contract`). If either check fails, loading fails with +an `UnsatisfiedLinkError` that names the problem: + +```text +kitchen_sink: kitchen.Gadget.describe is missing from the library +kitchen_sink: kitchen.Item changed since these bindings were generated +``` + +A producer that adds declarations still loads: only the entries the +bindings know about are checked. ## Type mapping | IDL type | Kotlin type | Crosses JNI as | |---|---|---| -| `i8`, `u8` | `Byte` | `jbyte` | -| `i16`, `u16` | `Short` | `jshort` | -| `i32` | `Int` | `jint` | -| `u32`, `i64`, `u64` | `Long` | `jlong` | +| `i8`, `i16`, `i32`, `i64` | `Byte`, `Short`, `Int`, `Long` | `jbyte`, `jshort`, `jint`, `jlong` | +| `u8`, `u16`, `u32`, `u64` | `UByte`, `UShort`, `UInt`, `ULong` | the signed type of the same width | | `f32`, `f64` | `Float`, `Double` | `jfloat`, `jdouble` | | `bool` | `Boolean` | `jboolean` | | `string` | `String` | UTF-8 `ByteArray` (pointer plus length) | @@ -85,12 +116,23 @@ otherwise loading fails with `UnsatisfiedLinkError` naming the module. | rich enum | `sealed class` | value buffer | | `T?`, `[T]`, `{K:V}` | `T?`, `List`, `Map` | value buffer | | interface, `Iface?` | wrapper class, nullable wrapper | address (`0` is none) | -| callback interface | Kotlin `interface` | the implementing object | +| callback interface, `Cb?` | Kotlin `interface`, nullable | the implementing object (`null` is none) | | `iter` | `NativeIterator` | iterator address | -Unsigned integers ride in the signed type of the same width (`u64` max is -`-1L`). Strings cross as UTF-8 bytes with an explicit length, so interior -NULs and supplementary characters round-trip exactly. +Unsigned integers are Kotlin's unsigned types everywhere: parameters, +returns, record and variant fields, error payload fields, collection +elements and map keys, iterator elements, async results, and callback +parameters and returns. A JNI native can't take an unsigned value class, so +each one crosses in the signed type of the same width, converted bit for bit +(`toUInt()`, `toInt()`), and nothing is truncated: `u64` max is +`ULong.MAX_VALUE`. Strings cross as UTF-8 bytes with an explicit length, so +interior NULs and supplementary characters round-trip exactly. + +Value buffers are encoded and decoded by generated functions: one `pack` +and `unpack` pair per record and rich enum (next to its declaration) and +one per distinct composite type in `Codecs.kt` (`packListOfEntry`, +`unpackMapOfStringToStore`, `packOptionalOfStoreInfo`), so every call site +calls a function instead of inlining a loop. ## Naming @@ -99,14 +141,19 @@ nested objects, so two modules can each declare a `get`. User types live at the top level of the package. Two rules keep the generated names valid: - A user type named like a Kotlin keyword, a type Kotlin or Java imports by - default (`Unit`, `String`, `Result`, ...), or a runtime declaration - (`FfiException`, `NativeIterator`, ...) gains a trailing underscore: - `Unit` becomes `Unit_`. -- A module object whose name equals a type's gains a `Module` suffix: a - `kv.stats` module beside a `Stats` record is `Kv.StatsModule`. - -Parameters and methods are lowerCamelCase; record fields keep their IDL -spelling. Keywords gain a trailing underscore (`class_`). + default (`Unit`, `String`, `UInt`, `Result`, ...), or a runtime + declaration (`FfiException`, `NativeIterator`, ...) gains a trailing + underscore: `Unit` becomes `Unit_`. +- A module object whose name equals a type's, or the name of a generated + file (`Runtime`, `Buffers`, `Codecs`, `Async`, `JniBridge`), gains a + `Module` suffix: a `kv.stats` module beside a `Stats` record is + `Kv.StatsModule`. + +Parameters and methods are lowerCamelCase; record fields, variant fields, +and error payload fields keep their IDL spelling (`expires_at`). Keywords +gain a trailing underscore (`class_`), as does a method named like a member +every wrapper already declares or inherits (`close`, `handle`, `toString`, +`wait`, ...; see [reserved member names](../reference/naming.md#identifiers-in-generated-code)). ## Objects and lifetime @@ -137,17 +184,31 @@ and the decoder adopts each one into a new wrapper. ## Errors -Every failure is an `FfiException` with a `code`. Each error domain becomes -a sealed subclass with one class per code: +A callable declared `throws` raises an `FfiException`. Each error domain +becomes a sealed subclass named with an `Exception` suffix (`KvError` +becomes `KvException`; `KitchenErrors` becomes `KitchenErrorsException`), +with one class per code whose payload fields are properties and whose +message defaults to the IDL's: ```kotlin sealed class KitchenErrorsException(code: Int, message: String) : FfiException(code, message) { class NotFound(message: String = "Item not found") : KitchenErrorsException(1, message) ``` -Runtime failures use the generic class: `-2` producer panic, `-3` -marshalling failure (including a malformed value buffer), and `-4` a -callback implementation that threw. +A runtime failure of a throwing call keeps the root `FfiException` class +with its negative `code`: `-1` a generic producer error, `-2` a producer +panic, `-3` a marshalling failure, and `-4` a callback implementation that +failed (the message is the callback's). + +A callable that isn't `throws` can't report an error, so any failure is a +bug in the library, and the binding raises the unchecked +`NativeBugException` (an `IllegalStateException`) instead, following the +[trap policy](../guides/errors-and-memory.md#the-trap-policy). Its `code` +and message name the runtime code and the producer's message, as in +`native call failed with code -2: {the panic message}`. + +A malformed value buffer or an undeclared enum value read from the producer +raises `NativeBugException` with code `-3` as well. ## Async and cancellation @@ -157,7 +218,7 @@ by its scope, or through `withTimeout`) cancels the token, so the producer drops the pending work: ```kotlin -suspend fun doCancellable(input: String): String = awaitNative(true, 0, { decodeUtf8(it as ByteArray) }) { _token, _done -> +suspend fun doCancellable(input: String): String = awaitNative(true, 0, { _raw -> decodeUtf8(_raw as ByteArray) }) { _token, _done -> JniBridge.kitchen_do_cancellable(encodeUtf8(input), _token, _done) } ``` @@ -170,13 +231,48 @@ API has async functions. ## Callbacks -A callback interface is a plain Kotlin `interface`. Passing an -implementation pins it with a JNI global reference until the producer -releases it. Producer threads may call it from anywhere: the shim attaches -a thread to the JVM once (as a daemon) and detaches it when the thread -exits. A method that throws reports `-4` with the exception's `toString()` -to the producer, which surfaces it to the original caller as an -`FfiException`; nothing unwinds through native frames. +A callback interface is a plain Kotlin `interface`, and an optional callback +parameter (`Cb?`) takes `null` for none, which passes a null vtable. Passing +an implementation pins it with a JNI global reference until the producer +calls the vtable's `free`. + +```kotlin +interface Policy { + /** + * Admit an entry about to be stored, returning it as it should be + * stored. ... + * + * Throw a [KvException] to report a typed error to the library. + */ + fun admit(entry: Entry): Entry + + fun route(key: String, home: Store): Store +} +``` + +- **Parameters.** Strings, bytes, and value buffers are decoded before the + method runs. An object parameter arrives as a new wrapper that owns the + reference the producer handed over; close it, or let the cleaner release + it. +- **Returns.** A method may return any type but an iterator or a callback + interface. A scalar or enum is returned by value. An object is returned as + a new strong reference (the wrapper keeps its own). A string, bytes, or + value buffer is encoded and copied into a run allocated with + `{prefix}_alloc`, which the producer adopts. +- **Failures.** Whatever a method throws is caught in the dispatch shim on + `JniBridge`; nothing unwinds through native frames. A method declared + `throws` may throw its module's domain exception (`KvException.Rejected`, + say), which reaches the producer with its code, message, and payload + fields, so a producer that propagates it hands the original caller the + same typed error. Anything else, including a domain exception from a + method that doesn't declare `throws`, reaches the producer as code `-4` + with the exception's message. + +Producer threads may call an implementation from anywhere. The shim attaches +a thread to the JVM the first time it calls in (as a daemon) and detaches it +when the thread exits, through a pthread key destructor or, on Windows, a +fiber-local storage callback; if neither can be registered, it detaches the +thread again as soon as the call returns. ## Iterators @@ -189,6 +285,8 @@ cleaner. - Gradle builds aren't exercised by the conformance suite, which compiles the same sources with `kotlinc` and the shim with CMake. -- On Windows the shim never detaches producer threads it attached. -- Interior NULs in an exception message are cut at the first NUL. -- Unsigned integers have no `UInt`/`ULong` surface. +- Interior NULs in an error message are cut at the first NUL, both in + messages the producer reports and in a failing callback's message. +- Kotlin's type system can't express a callback return the producer would + reject (a null object for a non-optional `I`, or a malformed record), so + those `-3` paths aren't reachable from Kotlin. diff --git a/docs/src/generators/node.md b/docs/src/generators/node.md index 6873efd6..249d516f 100644 --- a/docs/src/generators/node.md +++ b/docs/src/generators/node.md @@ -1,10 +1,10 @@ # Node.js The Node.js target generates an npm package: an ES module with TypeScript -declarations, and a small N-API addon that `npm install` compiles with -node-gyp and that calls the library's C ABI. The JavaScript API is shared -with the [WebAssembly target](wasm.md); only the transport underneath -differs, so code written against one runs against the other. +declarations, and a small N-API addon that calls the library's C ABI. The +JavaScript API is shared with the [WebAssembly target](wasm.md); only the +transport underneath differs, so code written against one runs against the +other. ## What gets generated @@ -12,26 +12,30 @@ For a library named `kvstore` (the snapshot fixtures use `kitchen_sink`): ```text node/ -├── package.json ES module manifest; `npm install` runs `node-gyp rebuild` +├── package.json ES module manifest; `npm install` compiles the addon only without a prebuilt one ├── binding.gyp builds the addon and links the library ├── kvstore_node.c the N-API addon ├── kvstore.h a copy of the C header the addon compiles against ├── index.js the API: one namespace export per top-level module ├── index.d.ts TypeScript declarations -├── runtime.js the shared runtime (codec, errors, wrappers) +├── runtime.js the shared runtime (codec, errors, wrappers, load-time checks) └── README.md ``` -The package name defaults to the library identity's name; set -`package_name` under `[generators.node]` in `weaveffi.toml` to override it. -The addon is named `{library}_node.node`. +The package name defaults to the package identity's `name`; set `name` under +`[generators.node]` in `weaveffi.toml` to override it, and `node_engine` to +change `engines.node` (default `>=18`). The addon is named +`{library}_node.node`. ## Install, build, and load -The generated directory is a complete package. Install it like any other, -pointing the build at the native library with the `{PREFIX}_LIBRARY` -environment variable (the library file or its directory) or the matching -npm option: +The generated directory is a complete package. Its loader uses a prebuilt +addon when one matches the platform (from an installed +`{package}-{os}-{cpu}` package, or under `prebuilds/-/`), and its +`install` script compiles the addon with node-gyp only when none does. To +compile it, install the package pointing the build at the native library +with the `{PREFIX}_LIBRARY` environment variable (the library file or its +directory) or the matching npm option: ```sh KVSTORE_LIBRARY=/opt/kvstore/lib/libkvstore.so npm install ./node @@ -52,15 +56,39 @@ store.put('greeting', new TextEncoder().encode('hi'), kv.EntryKind.Volatile, nul store.close(); ``` -Importing the package loads the addon and checks the contract before any -other call: the library must implement ABI revision 3, and each top-level -module's checksum must match the one the bindings were generated from. -Otherwise the import fails with an error naming the mismatched module. +`weaveffi package` writes npm tarballs in the esbuild-style layout: one +`{package}-{os}-{cpu}` package per desktop platform, gated by npm `os` and +`cpu` and carrying the library and the addon `weaveffi build` prebuilt for +it, plus the main package, which lists them in `optionalDependencies`. So +`npm install` needs no compiler on a platform whose addon `weaveffi build` +prebuilt; elsewhere, Windows included (where the addon is never prebuilt), +it compiles the addon against the installed platform package's library. +See [Packaging](../guides/packaging.md) for which addons are prebuilt. -`weaveffi package` produces the esbuild-style layout instead: the main -package lists one `{package}-{os}-{cpu}` package per platform in -`optionalDependencies`, each gated by npm `os` and `cpu` and carrying the -prebuilt library, and the addon links the one npm installed. +## Load-time checks + +Importing the package loads the addon and, before any other call, checks +that the library matches the bindings (see +[Load-time checks](../reference/abi.md#load-time-checks)). The library must +implement ABI revision 4, and every top-level module's contract table must +hold each declaration the bindings were generated with. `index.js` embeds +those entries: + +```js +const $contract = [ + ['kv', [ + [0x0969575bfbb012d7n, 0xebd38766e3532c4fn, 'kv.Store.fork'], + // ... + ]], +]; +$verify($raw, 'kvstore', 'kvstore', 4, $contract); +``` + +If a check fails, the import throws an `Error` naming the declaration: +`kvstore: kv.Store.put is missing from the library` or `kvstore: +kv.Store.put changed since these bindings were generated`. Declarations the +library has and the bindings don't are fine, so a library that only adds to +its API keeps working with older bindings. ## Modules and names @@ -77,10 +105,14 @@ export const kitchen = Object.freeze({ boolId: kitchen$boolId, ``` -Functions and methods are camelCase. A name that is a JavaScript reserved -word gains a trailing underscore (`delete` becomes `delete_`), as does an -instance method named `close` or `constructor` and a static named `name` or -`length`. Record fields keep their IDL spelling. +Functions, methods, and parameters are camelCase. A free function, +parameter, or module whose name is a JavaScript reserved word gains a +trailing underscore (a function `delete` becomes `delete_`). Methods keep +reserved words, which are legal property names (`store.delete(key)`), +except that an instance method named `close` or `constructor`, and a +static named `name`, `length`, `prototype`, or `caller`, gains one too +([reserved member names](../reference/naming.md#identifiers-in-generated-code)). +Record fields keep their IDL spelling. ## Type mapping @@ -93,18 +125,23 @@ instance method named `close` or `constructor` and a static named `name` or | `string` | `string` | UTF-8, passed as pointer and length, so interior NULs survive | | `bytes` | `Uint8Array` | | | record | `interface` | A plain object; fields keep their IDL names | -| C-style enum | `enum` | A frozen object with forward and reverse mappings | +| C-style enum | `enum` | A frozen object with forward and reverse mappings (negative values too) | | rich enum | tagged union | `{ tag: 'Circle', radius: 2.5 }` | | interface | `class` | See [Objects](#objects-and-lifetime) | -| callback interface | `interface` | Any object with the methods | +| callback interface | `interface` | Any object with the methods; see [Callback interfaces](#callback-interfaces) | | `T?` | `T \| null` | `undefined` is accepted as `null` | | `[T]` | `T[]` | | -| `{K: V}` | `Record` | Keys are strings at run time; a `Map` is accepted too | +| `{K: V}` | `Record` | `Record` for integer keys of 32 bits or fewer, `Partial>` for C-style enum keys; keys are property names (strings) at run time, and a `Map` is accepted as an argument | | `iter` | `IterableIterator` | Lazy | Arguments are type-checked: a value of the wrong type throws a `TypeError` before the native call. +Value buffers are encoded and decoded by one function per type: each record +and rich enum (`$w$kv$Entry`, `$r$kv$Entry`), each interface carried as an +object token, and each distinct optional, list, and map type in the API +(`$w_list_opt_Entry` writes `[Entry?]`). Call sites name these functions. + ## Objects and lifetime An interface is a class whose instances each own one strong reference to the @@ -130,9 +167,9 @@ to return. Using a closed wrapper throws a code -3 error. The synchronous constructor named `new` is the class constructor; other constructors are static factories, and a class without one can't be -constructed directly. Objects returned, awaited, iterated, or decoded from a -record are new wrappers the caller owns. Two wrappers may share one native -object. +constructed directly. Objects returned, awaited, iterated, decoded from a +record, or passed to a callback method are new wrappers the receiver owns. +Two wrappers may share one native object. ## Errors @@ -141,12 +178,18 @@ Every error extends the package's root class, `{Package}Error` (here - Each error domain is a class (`kitchen.KitchenErrorsError`), and each of its codes a subclass (`kitchen.NotFoundError`) with a static `CODE` and the - IDL message as its default message. Payload fields become properties. + IDL message as its default message. Payload fields become properties. A + code without fields is constructed as `new kv.InvalidPathError(message?)`; + a code with fields takes them first: + `new kv.RejectedError({ key, reason }, message?)`. - A cancelled call rejects with `CancelledError` (code -5). - Runtime failures throw the root class itself: -1 generic, -2 producer panic, -3 marshalling failure, -4 a callback implementation failed. -Only functions declared `throws` map codes onto their module's domain. +Only functions declared `throws` map codes onto their module's domain; a +failure of any other call is a producer bug, and the binding throws the +root class with the code and the producer's message (see +[the trap policy](../guides/errors-and-memory.md#the-trap-policy)). ## Async and cancellation @@ -165,15 +208,79 @@ starts. ## Callback interfaces Any object with the interface's methods implements it; the methods are -camelCase. Arguments arrive converted (records decoded, objects as wrappers -the implementation owns), and the return value is checked against the IDL -type. A method that throws, or returns a value of the wrong type, fails the -native call with code -4 and the exception's message; nothing unwinds -through native code. +camelCase. Arguments arrive converted: strings and bytes as `string` and +`Uint8Array`, records decoded, and objects as new wrappers the +implementation owns (close them, or let the garbage collector release +them). A parameter of an optional callback interface (`Cb?`) takes `null` +for none. + +A method's return value is checked and handed to the library in its family's +form: + +| IDL return | Return from JavaScript | What the library receives | +|------------|------------------------|---------------------------| +| direct (numbers, `bool`, C-style enums) | the value | the value | +| `string`, `bytes` | a `string`, a `Uint8Array` | a copy in a run the library adopts | +| record, rich enum, `T?`, `[T]`, `{K:V}` | the value | its encoding, in a run the library adopts | +| `I` | a wrapper (never `null`) | a new strong reference to its object | +| `I?` | a wrapper or `null` | a new strong reference, or null | + +The implementation keeps its own wrapper; the library gets a reference of +its own. A method that throws, or returns a value of the wrong type, fails +the native call with code -4 and the exception's message; nothing unwinds +through native code. A method declared `throws` may instead throw an error +of its module's domain, and the library receives that code, message, and +fields: + +```js +const policy = { + admit(entry) { + if (entry.key.startsWith('secret')) { + throw new kv.RejectedError({ key: entry.key, reason: 'no secrets' }, 'secrets are not stored'); + } + return { ...entry, tags: ['admitted'] }; + }, + route(key, home) { + return home; + }, +}; +store.setPolicy(policy); +``` -The producer may call back from any thread. A call made on the JavaScript -thread runs directly; one made on another thread runs on the JavaScript -thread while the producer thread waits. +### Threading + +The producer may call back from any thread, and every JavaScript call runs +on the JavaScript thread (the main thread or the worker that passed the +implementation): + +- A call made on the JavaScript thread (because the JavaScript thread is + inside a call into the library) runs directly, nested inside that call. +- A call made on any other thread is queued for the JavaScript thread and + runs from its event loop, while the producer thread waits for the result. + Nothing runs while the JavaScript thread is busy, so such calls are + delivered only once the current JavaScript task returns to the event loop. +- The library's release of an implementation (its vtable's `free`) may come + from any thread; the addon drops the implementation on the JavaScript + thread. + +The waiting in the second rule could deadlock: if the JavaScript thread is +inside a synchronous call into the library that itself waits for the thread +making the callback (joins it, or waits on a lock it holds), neither thread +can proceed. The addon detects this instead of hanging. It counts the +synchronous calls in progress on each JavaScript thread, and a callback from +another thread that has waited while the JavaScript thread stayed inside one +synchronous call for a second gives up: the callback fails with code -4 and +a message saying that it would deadlock, the producer sees that failure, and +the synchronous call can finish. A callback that waits through a shorter +synchronous call, or for the event loop, simply runs later. So: + +- Make calls that wait for callbacks from other threads `async`, or have + the producer call back on the calling thread. +- Avoid synchronous calls that block for a second or more while other + threads call back; such a callback fails with -4 rather than waiting + longer. +- Return to the event loop regularly while callbacks from other threads are + expected: a long-running JavaScript task delays them. ## Iterators @@ -184,8 +291,8 @@ iterator is garbage collected. ## Known limitations -- A synchronous call whose producer blocks on a callback from another thread - deadlocks, because the JavaScript thread is busy with the call. +- A callback from another thread fails with -4 if it waits for a second on + a synchronous call (see [Threading](#threading)). - A callback implementation that holds a wrapper of the object retaining it forms a cycle the garbage collector can't see; close the wrapper or release the callback explicitly. diff --git a/docs/src/generators/python.md b/docs/src/generators/python.md index 552d9129..f766a1a5 100644 --- a/docs/src/generators/python.md +++ b/docs/src/generators/python.md @@ -1,10 +1,13 @@ # Python The Python target produces a pip-installable package of pure-Python -`ctypes` bindings over the C ABI (revision 3), plus `.pyi` type stubs. There's -no compilation step, no native extension, and no third-party runtime -dependency; the package works on any Python 3.9+ interpreter that can load -the producer library. +`ctypes` bindings over the C ABI (revision 4). There's no compilation step, +no native extension, and no third-party runtime dependency; the package +works on any Python 3.9+ interpreter that can load the producer library. +The implementation module is fully annotated and the package ships +`py.typed`, so type checkers read the bindings themselves (there's no +separate stub to drift out of date) and the generated code passes +`mypy --strict`. Records and rich enums are dataclasses that cross the boundary as value buffers. Interfaces are reference-counted wrapper classes. Callback @@ -21,46 +24,67 @@ python/ ├── pyproject.toml # PEP 621 metadata, built with setuptools ├── README.md └── kvstore/ # the import package, named after the C prefix - ├── __init__.py # re-exports the public API + ├── __init__.py # re-exports the module's __all__ ├── kvstore.py # runtime, prototypes, and wrappers - ├── kvstore.pyi # type stub └── py.typed # PEP 561 marker ``` The distribution name is the package identity's `name` and the import -package is its C prefix, so a crate named `async-demo` installs as -`async-demo` and imports as `async_demo`. Both are configurable: +package is its C prefix, so a crate named `key-store` installs as +`key-store` and imports as `key_store`. Both are configurable: ```toml [generators.python] -package_name = "kvstore-python" # distribution name (default: package name) -import_name = "kvstore_py" # import package (default: C prefix) -strip_module_prefix = true # `get_stats`, not `kv_stats_get_stats` +name = "kvstore-python" # distribution name (default: package name) +import_name = "kvstore_py" # import package (default: C prefix) +requires_python = ">=3.10" # default ">=3.9" ``` +The module lists its public names in `__all__` (the error classes, every +type, and the free functions), so `from kvstore import *` and the package's +`__init__.py` export the API and none of the runtime's helpers. + ## Install and load ```bash -weaveffi generate src/lib.rs -o generated --target python -pip install generated/python +weaveffi generate -o bindings --target python # in the producer crate +pip install bindings/python ``` At import, the module loads the producer library from the first of: 1. the path in `{PREFIX}_LIBRARY` (for example `KVSTORE_LIBRARY`), 2. a library bundled inside the package directory (what `weaveffi package` - produces), and + produces, and where `weaveffi dev` copies the debug build), and 3. the system loader's search path, by platform file name (`libkvstore.dylib`, `libkvstore.so`, or `kvstore.dll`). -It then checks `kvstore_abi_version()` against revision 3 and every -top-level module's contract checksum (`kvstore_kv_checksum()`) against the -value the bindings were generated with. A mismatch raises `ImportError` -naming the module, so a stale binding never misreads a buffer at runtime. -Every C prototype is bound once at this point, in a module-level table: +It then makes the two [load-time checks](../reference/abi.md#load-time-checks) +before binding anything else. `kvstore_abi_version()` must report revision +4, and every top-level module's contract table must contain each +declaration the bindings were generated with, with an equal hash. The +expected entries are embedded in the module, one `(id, hash, path)` per +declaration: + +```python +_CONTRACTS: Dict[str, List[Tuple[int, int, str]]] = { + "kvstore_kv_contract": [ + (0x0969575bfbb012d7, 0xebd38766e3532c4f, "kv.Store.fork"), + ... + ], + "kvstore_report_contract": [...], +} +``` + +A failed check raises `ImportError` naming the declaration +(`kv.Store.put is missing from the library` or `kv.Store.put changed since +these bindings were generated`), so a stale binding never misreads a +buffer at runtime. Declarations the library has and the bindings don't are +fine, so a library that grew new functions still loads. Every C prototype +is then bound once, in a module-level table: ```python -_c_kitchen_Gadget_describe = _bind("kitchen_sink_kitchen_Gadget_describe", ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t), ctypes.POINTER(_ErrorStruct)) +_c_kv_Store_count = _bind("kvstore_kv_Store_count", ctypes.c_uint32, ctypes.c_void_p, ctypes.POINTER(_ErrorStruct)) ``` On macOS, the system `python3` strips `DYLD_LIBRARY_PATH`, so prefer @@ -83,106 +107,169 @@ On macOS, the system `python3` strips `DYLD_LIBRARY_PATH`, so prefer | interface | wrapper class | `c_void_p` | | interface `?` | `Optional[...]` | `c_void_p`, NULL for `None` | | callback interface | subclass of the generated ABC | context key plus vtable | +| callback interface `?` | `Optional[...]` | context key plus vtable, NULL for `None` | | `iter` | `Iterator[T]` | iterator handle | Strings and bytes pass without per-byte loops: a `str` is encoded once and -its bytes are lent to the call, so interior NUL characters survive. Returned -strings, bytes, and buffers are copied with `ctypes.string_at` and released -with `{prefix}_free_bytes`. Lists of a fixed-width number type are packed -and unpacked with one `struct` call. +its bytes are lent to the call, so interior NUL characters survive. +Returned strings, bytes, and buffers are copied with `ctypes.string_at` and +released with `{prefix}_free_bytes`. + +Every record and rich enum has one writer and one reader function +(`_write_Entry`, `_read_Entry`), and so does every distinct composite type +the API uses (`_write_list_string`, `_read_map_string_Store`, +`_read_opt_Entry`); a call site makes one call instead of inlining a loop. +Lists of a fixed-width number type are packed and unpacked with one +`struct` call. A decoded buffer that's truncated, has trailing bytes, an +invalid flag, invalid UTF-8, an undeclared enum value, or a repeated map +key raises `InternalError` with code `-3`. All modules share one Python namespace. Nested modules' declarations are -exported next to their parents', so names must be unique across modules -(set `strip_module_prefix = false` to prefix function names). +exported next to their parents' under their bare names (`summarize`, not +`kv_stats_summarize`), which global names keep unique. The module starts +with `from __future__ import annotations`, so annotations name types +declared later in the file without quotes. ## Objects and lifetime An interface wrapper holds one strong reference and releases it exactly once: from `close()`, the `with` statement, or the `__del__` backstop. -`close()` is idempotent, and a closed wrapper raises -`Error(-1, "... used after close()")`. Each call lends the pointer for its -duration, so `close()` from another thread while a call is in flight defers -the release until the call returns: +`close()` is idempotent, and using a closed wrapper raises `ValueError`. +Each call lends the pointer for its duration, so `close()` from another +thread while a call is in flight defers the release until the call +returns. Two wrappers compare equal (and hash alike) when they hold the +same native object, so `store.share() == store` and a record holding a +`Store` compares by identity. An interface member named `close` is +`close_`, so it never replaces the wrapper's own ([reserved member +names](../reference/naming.md#identifiers-in-generated-code)). + +A constructor named `new` becomes `__init__`; other constructors are +`@classmethod` factories: ```python - def describe(self) -> str: - """Render the gadget as a human-readable string""" + @classmethod + def open(cls, path: str) -> Store: + _path_b = path.encode("utf-8") _err = _ErrorStruct() - _out_len = ctypes.c_size_t() - _self_p = self._acquire() - try: - _ret = _c_kitchen_Gadget_describe(_self_p, ctypes.byref(_out_len), ctypes.byref(_err)) - finally: - self._release() - _check_error(_err) - return _take_str(_ret, _out_len.value) + _ret = _c_kv_Store_open(_path_b, len(_path_b), ctypes.byref(_err)) + if _err.code: + raise _kv_error_from(*_read_error(_err)) + return cls._adopt(_required(_ret)) ``` -A constructor named `new` becomes `__init__`; other constructors are -`@classmethod` factories. Objects returned by the producer, decoded from a -buffer, or passed to a callback are adopted into new wrappers. Writing an -object into a value buffer (a record field or a list element) mints a new -reference only after the whole argument list has encoded, so a failed -encoding leaks nothing. +Objects returned by the producer, decoded from a buffer, or passed to a +callback are adopted into new wrappers. Writing an object into a value +buffer (a record field or a list element) mints a new reference only after +the whole argument list has encoded, so a failed encoding leaks nothing. ## Errors -Every exception derives from the package's root `Error` (named -`{PascalName}Error` if the API declares its own `Error` type), which +A call that declares errors (`throws`) raises the error domain in scope for +its module. Every such exception derives from the package's root `Error` +(named `{PascalName}Error` if the API declares its own `Error` type), which carries `code` and `message`. A module's error domain becomes a subclass (`KvError`) with one subclass per code (`KeyNotFound`), each also reachable -as `KvError.KeyNotFound` and carrying its stable `CODE`. Payload fields -decode onto the exception as attributes. +as `KvError.KeyNotFound` and carrying its stable `CODE`. A code's payload +fields are constructor arguments and attributes: -Negative codes are runtime traps with constants on `Error`: +```python +try: + store.get("nope") +except kvstore.KvError.KeyNotFound as e: + print(e.code, e.key, e.message) # 1001 nope key not found: nope +``` + +A negative code on a throwing call (a panic, a marshalling failure, or a +failed callback) raises the root `Error` with that code; the constants are `GENERIC_ERROR_CODE` (-1), `PANIC_ERROR_CODE` (-2), `MARSHAL_ERROR_CODE` -(-3), and `FOREIGN_ERROR_CODE` (-4, a callback implementation raised). A -cancelled async call (-5) raises `asyncio.CancelledError` instead. +(-3), and `FOREIGN_ERROR_CODE` (-4). + +A call that declares no errors can only fail through a bug, so it follows +the [trap policy](../guides/errors-and-memory.md#the-trap-policy): it +raises `InternalError`, a `RuntimeError` subclass outside the `Error` +hierarchy whose message names the code and the producer's message +(`InternalError: (-3) ...`). A cancelled async call (-5) raises +`asyncio.CancelledError` instead. ## Async and cancellation An async function is an `async def` that must run on an `asyncio` event loop. The launcher receives a static completion trampoline and an integer -key; the trampoline takes ownership of the result on the producer's -thread and settles the future through `call_soon_threadsafe`. - -For a `cancellable` function, each call owns a native cancel token. -Cancelling the awaiting task (directly, through `asyncio.wait_for`, or by -cancelling a `gather`) cancels the token, waits for the producer's -cancelled completion, and raises `asyncio.CancelledError`: +key; the trampoline takes ownership of the result on the producer's thread +and settles the future through `call_soon_threadsafe`: ```python -async def do_cancellable(input: str) -> str: - """Cancellable async operation""" - _input_b = input.encode("utf-8") +async def open_store(path: str) -> Store: + _path_b = path.encode("utf-8") _call, _future = _async_begin() - _token = _cancel_token_create() try: - _c_kitchen_do_cancellable(_input_b, len(_input_b), _token, _kitchen_do_cancellable_completion, _call) + _c_kv_open_store(_path_b, len(_path_b), _kv_open_store_completion, _call) except BaseException: _async_abandon(_call) - _cancel_token_destroy(_token) raise - return await _async_wait_cancellable(_future, _token) + _result: Store = await _future + return _result ``` -Cancelling the task awaiting a function that isn't `cancellable` stops the -wait only; the native call runs to completion and its result is dropped. +For a `cancellable` function, each call owns a native cancel token. +Cancelling the awaiting task (directly, through `asyncio.wait_for`, or by +cancelling a `gather`) cancels the token, waits for the producer's +cancelled completion, and raises `asyncio.CancelledError`. Cancelling the +task awaiting a function that isn't `cancellable` stops the wait only; the +native call runs to completion and its result is dropped. ## Callbacks A callback interface is an `abc.ABC` with one abstract method per IDL method. Passing an instance registers it in a handle table under an integer key, which the producer receives as `ctx` along with the address of the -interface's one static vtable. The producer's `free(ctx)` removes the -entry, so an implementation lives exactly as long as the producer holds it. +interface's one static vtable; `None` for an optional callback parameter +passes a null vtable. The vtable starts with the revision 4 header (its +`size`, `flags`, and the shared `free` entry), and the producer's +`free(ctx)` removes the table entry, so an implementation lives exactly as +long as the producer holds it. Trampolines run on whichever thread the producer calls from; `ctypes` -acquires the GIL first. An exception raised by an implementation is -reported through `{prefix}_error_set(out_err, -4, message)` and never -unwinds through C; the producer's call then fails with -`FOREIGN_ERROR_CODE`. A producer method declared to return -`Result` receives the failure as an `Err` value. +acquires the GIL first. A trampoline adopts the object arguments first, +converts the rest, calls the method, and hands the return back: + +- a number, `bool`, or enum as the C return value; +- an object as a fresh strong reference the producer adopts (`None` is + allowed only for `I?`); +- a `str`, `bytes`, record, or other buffered value through the + `out_ptr`/`out_len` slots, in a run allocated with `{prefix}_alloc` that + the producer adopts and frees. + +```python +def _Loader_load(ctx: Optional[int], key_ptr: Optional[int], key_len: int, out_ptr: Any, out_len: Any, out_err: Any) -> None: + try: + _ret = _callback_get(ctx).load(_peek_bytes(key_ptr, key_len).decode("utf-8")) + _callback_return_bytes(out_ptr, out_len, bytes(_ret)) + except BaseException as exc: + _callback_fail(out_err, exc, KvError) +``` + +Nothing unwinds through C. A method that declares errors may raise its +module's domain codes, which reach the producer as that code with the +fields as the error payload (`{prefix}_error_set` and +`{prefix}_error_set_payload`): + +```python +class EmptyLoader(kvstore.Loader): + def name(self) -> str: + return "empty" + + def fallback(self, key: str) -> Optional[kvstore.Store]: + return None + + def load(self, key: str) -> bytes: + raise kvstore.KeyNotFound(key=key, message="not in the loader") +``` + +Any other exception, a return of the wrong type, or a domain error raised +by a method that doesn't declare errors reaches the producer as +`FOREIGN_ERROR_CODE` (-4) with the exception's message. A Rust producer +receives either as the `Err` of the method's `Result`. ## Iterators @@ -191,26 +278,34 @@ An `iter` return is a lazy iterator: each `next()` makes one native exhaustion, `close()`, or garbage collection. ```python -for key in store.list_keys(None): +for key in store.keys(None): print(key) ``` ## Packaging -`weaveffi package --target python` writes one wheel source tree per -desktop platform under `python//`, with the producer library -bundled inside the import package and a `setup.py` that forces a -platform-tagged wheel. Build each tree with `python -m build --wheel` and -tag it with the platform tag listed in its `README.md`. Android and -`wasm32` binaries have no wheel platform and are skipped. +`weaveffi package --target python` writes one wheel per desktop platform +built, `python/{name}-{version}-py3-none-{platform}.whl` (the distribution +name escaped for a wheel file name, so `Key-Store` becomes `key_store`), +with the producer library inside the import package. The platform tag +matches the build: the `[build]` macOS deployment target +(`macosx_11_0_arm64`), the newest glibc the library links against on Linux +(`manylinux_2_17_x86_64` or later), or `win_amd64`. The wheels install +with `pip` and upload with `twine`; Android, iOS, and `wasm32` builds have +no wheel platform and are skipped. See [Packaging](../guides/packaging.md). ## Known limitations - `ctypes` dispatch costs more per call than a compiled extension; hot - loops that cross the boundary per element will feel it. + loops that cross the boundary per element will feel it. The bindings + keep the per-call work small (prototypes and vtable addresses are + resolved at import, and the error check is inline), but each call still + allocates its error slot. +- `ctypes` truncates an out-of-range integer argument to the parameter's C + width instead of raising; inside a value buffer, `struct` raises. - All modules share one namespace (see "Type mapping"). - A long-running callback stalls the producer thread that called it, and a callback that waits for the thread blocked in the calling Python code deadlocks. -- The stubs type `iter` returns as `Iterator[T]`, so a type checker +- `iter` returns are annotated as `Iterator[T]`, so a type checker doesn't know about their `close()` method. diff --git a/docs/src/generators/ruby.md b/docs/src/generators/ruby.md index 44835608..d815857c 100644 --- a/docs/src/generators/ruby.md +++ b/docs/src/generators/ruby.md @@ -1,6 +1,6 @@ # Ruby -The Ruby target generates a pure-Ruby gem that binds the C ABI (revision 3) +The Ruby target generates a pure-Ruby gem that binds the C ABI (revision 4) through the [ffi](https://github.com/ffi/ffi) gem. There's no native extension to compile: the gem depends on `ffi ~> 1.15` and loads the producer's shared library at `require` time. Ruby 2.7 or newer is required. @@ -16,20 +16,20 @@ ruby/ lib/ kvstore.rb require 'kvstore': module Kvstore, the bindings kvstore/ - runtime.rb loader, ABI check, errors, codec, object base + runtime.rb loader, load-time checks, errors, codec, object base ``` Names come from the package identity: | Name | Default | Override (`[generators.ruby]`) | |------|---------|--------------------------------| -| Gem | the package name (`async-demo`) | `gem_name` | -| Ruby module | the package name in PascalCase (`AsyncDemo`) | `module_name` | -| `require` path | the C prefix (`async_demo`) | none | +| Gem | the package name (`key-store`) | `name` | +| Ruby module | the package name in PascalCase (`KeyStore`) | `module_name` | +| `require` path | the C prefix (`key_store`) | none | -`strip_module_prefix` (default `true`) controls whether a function in -module `contacts` is spelled `create_contact` or `contacts_create_contact`. -Every IDL module, nested ones included, lands in the one Ruby module. +Every IDL module, nested ones included, lands in the one Ruby module, and a +function keeps its bare name (`render_report` in module `report`), which +global names keep collision-free. `lib/{prefix}/runtime.rb` is fixed code shipped with the generator (the source lives in `targets/ruby/runtime/runtime.rb`); only the module name, @@ -38,9 +38,9 @@ the C prefix, and the library names are substituted into it. ## Install, build, and load ```bash -weaveffi generate samples/kvstore/src/lib.rs -o generated --target ruby +weaveffi generate samples/kvstore -o bindings --target ruby cargo build --release -p kvstore -cd generated/ruby +cd bindings/ruby gem build kvstore.gemspec gem install kvstore-1.0.0.gem ``` @@ -52,26 +52,48 @@ The loader opens the library in this order: 3. `libkvstore.dylib`, `libkvstore.so`, or `kvstore.dll` on the system search path. -At `require` time the bindings then check that the library reports ABI -revision 3 and that every top-level module's contract checksum matches -the one the bindings were generated from. A mismatch raises `LoadError` -naming the module: +```ruby +require 'kvstore' + +store = Kvstore::Store.open('/tmp/data') +store.put('greeting', 'hi'.b, Kvstore::EntryKind::VOLATILE) +store.close +``` + +`weaveffi package --target ruby` writes one precompiled platform gem per +desktop platform built, `ruby/{gem}-{version}-{platform}.gem` (`arm64-darwin`, +`x86_64-darwin`, `x86_64-linux`, `aarch64-linux`, `x64-mingw-ucrt`), with the +library at `lib/native/`. The gems install with `gem install` and publish +with `gem push`; Android, iOS, and `wasm32` builds are skipped. See +[Packaging](../guides/packaging.md). + +## Load-time checks + +`require` checks, before anything else is attached, that the library +implements C ABI revision 4, and then that every declaration the bindings +were generated with is in its top-level module's +[contract table](../reference/abi.md#load-time-checks) with the same +signature. The expected entries are embedded in the bindings: ```ruby -module KitchenSink - # Contract checksums of the top-level modules these bindings were - # generated from; a library built from a different API fails to load. +module Kvstore _wv_check_contract!( - 'shared' => [:kitchen_sink_shared_checksum, 0x42c4ce2c8d0af052], - 'kitchen' => [:kitchen_sink_kitchen_checksum, 0xd83da75b24b545ff], + kvstore_kv_contract: [ + [0x0969575bfbb012d7, 0xebd38766e3532c4f, 'kv.Store.fork'], + # ...one entry per declaration in `kv` and its submodules + ], + kvstore_report_contract: [ + [0xa21a2e7274bf28c6, 0x342e0b83b12d73ec, 'report.render_report'], + [0xd504fae45f64ab45, 0xd430b8de9d574a4c, 'report.ReportError'], + ], ) ``` -`weaveffi package --target ruby` writes one platform gem tree per desktop -binary under `ruby//`, with the library at `lib/native/` and -`s.platform` set (`arm64-darwin`, `x86_64-darwin`, `x86_64-linux`, -`aarch64-linux`, `x64-mingw-ucrt`). Android and `wasm32` binaries are -skipped. +A failed check raises `LoadError` naming the declaration +(`kvstore: kv.Store.put is missing from the library` or +`kvstore: kv.Store.put changed since these bindings were generated`). +Declarations the library has and the bindings don't are fine, so a library +that only adds to its API keeps working with older bindings. ## Type mapping @@ -90,14 +112,27 @@ skipped. | interface | wrapper class | `:pointer` | | `Interface?` | wrapper or `nil` | `:pointer` (NULL for `nil`) | | callback interface | any object with the methods | context + vtable | +| `Cb?` | such an object or `nil` | context + vtable (NULL for `nil`) | | `iter` | lazy `Enumerator` | iterator handle | +Trailing optional parameters (`T?`, `Interface?`, `Cb?`) default to `nil`, +so `store.keys` is `store.keys(nil)`. + Strings cross as a pointer plus a length, so interior NUL characters -survive. Returned strings are copied, tagged UTF-8, and released with -`{prefix}_free_bytes`. Value buffers use the -[value-buffer format](../reference/value-buffers.md), encoded and decoded -by the runtime's `WvBufferWriter` and `WvBufferReader` and one -`_wv_write_*`/`_wv_read_*` pair per record and rich enum. +survive. A string argument must be a `String` (or convert with `to_str`) +holding valid UTF-8; anything else raises `TypeError` or `ArgumentError` +before the call. Returned strings are copied, tagged UTF-8, and released +with `{prefix}_free_bytes`. Integers written into a value buffer are +range-checked (`RangeError` rather than silent wrapping), and an object +argument must be a wrapper of the declared interface (`TypeError` +otherwise). + +Value buffers use the [value-buffer format](../reference/value-buffers.md), +encoded and decoded by the runtime's `WvBufferWriter` and `WvBufferReader` +and by generated codec methods: one `_wv_write_*`/`_wv_read_*` pair per +record and rich enum, and one per distinct composite type the API uses +(`[Entry?]` is `_wv_write_list_opt_entry`, `{string:string}` is +`_wv_write_map_string_string`), so no call site inlines a loop. ## Objects and lifetime @@ -105,80 +140,89 @@ Each interface becomes a subclass of the runtime's `WvObject`, holding one strong reference in an `FFI::AutoPointer`: ```ruby - # A pocket-sized interface exercising the object surface - class Gadget < WvObject - WV_PTR = GadgetPtr - - # @api private - def self._wv_clone(ptr) - KitchenSink.kitchen_sink_kitchen_Gadget_clone(ptr) - end + # An embedded key-value store. + class Store < WvObject + WV_PTR = StorePtr - # Create a gadget with the given id - def initialize(id) + # Create an empty store with the path `memory`. + def initialize ``` - `close` releases the reference now and is idempotent; otherwise the GC finalizer releases it. The object itself lives until its last reference - anywhere (another wrapper, a record field, the producer) goes. + anywhere (another wrapper, a record field, the library) goes. - `dup` and `clone` make an independent wrapper with its own reference. - A constructor named `new` becomes `initialize`; other constructors are - class methods. An interface without one hides `new`. + class methods (`Store.open`). An interface without one hides `new`. - Every call pins its receiver and object arguments (`_wv_pin`), so a `close` from another thread or from a callback during the call takes effect only when the call returns. - An object inside a value buffer carries its own reference, minted just before the call; an object decoded from one is adopted into a new wrapper. -- Using a closed wrapper raises `Error` ("KitchenSink::Gadget used after close"). +- Using a closed wrapper raises `Error` ("Kvstore::Store used after close"). +- A method spelled like one the wrapper relies on (`close`, `handle`, + `initialize`, `class`, `dup`, `hash`, ...), or a static or factory named + `allocate` or `name`, gains a trailing `_`: an IDL method `close` is + `close_` ([reserved member names](../reference/naming.md#identifiers-in-generated-code)). ## Errors Every error derives from `{Module}::Error`, which carries `code`. A module with an error domain gets a domain class (`KvError < Error`) with one nested class per code (`KvError::KeyNotFound`, carrying `CODE` and any -payload fields as attributes). Functions that declare errors raise the -domain class; runtime failures raise the base classes: +payload fields as attributes): + +```ruby +begin + store.get('missing') +rescue Kvstore::KvError::KeyNotFound => e + e.code # => 1001 + e.key # => "missing" + e.message # => "key not found: missing" +end +``` -| Code | Constant | Raised as | -|------|----------|-----------| -| -1 | `GENERIC_ERROR_CODE` | `Error` | -| -2 | `PANIC_ERROR_CODE` | `Error` (the producer panicked) | -| -3 | `MARSHAL_ERROR_CODE` | `Error` (a malformed argument or buffer) | -| -4 | `FOREIGN_ERROR_CODE` | `Error` (a callback implementation raised) | -| -5 | `CANCELLED_ERROR_CODE` | `Cancelled` | +What a failure raises follows the +[trap policy](../guides/errors-and-memory.md#the-trap-policy): + +| Call | Domain code | Runtime code (`-1` to `-4`) | `-5` | +|------|-------------|-----------------------------|------| +| declared `throws` | the code's class (`KvError::Expired`) | `Error` with that code and the library's message | `Cancelled` | +| not `throws` | not possible | `NativeBugError` | `Cancelled` | + +`NativeBugError < Error` marks a bug in the library (a panic, an argument it +couldn't accept, a callback failure it let through a call that can't report +one), never an outcome to handle. Its message names the code and the +library's message, as in `native call failed with code -2: {the panic +message}`; a malformed value the library returned raises it with code `-3`. ## Async and cancellation -An async function blocks the calling thread until the producer's -completion fires, then returns the result or raises. The wait is a -`Queue#pop`, which releases the GVL, so other threads keep running; run -the call in a `Thread` for concurrency. A cancellable function takes a -`cancel:` keyword: +An async function blocks the calling thread until the library's completion +fires, then returns the result or raises. The wait is a `Queue#pop`, which +releases the GVL, so other threads keep running; run the call in a `Thread` +for concurrency. A cancellable function takes a `cancel:` keyword: ```ruby - # Cancellable async operation - # Blocks until the call completes on a producer thread. - # @param cancel [CancelToken, nil] cancels the call; it then raises Cancelled - def self.do_cancellable(input, cancel: nil) - input_s = _wv_str(input) - own_token = CancelToken.new if cancel.nil? - token = cancel || own_token - ctx, queue = _wv_async_begin - kitchen_sink_kitchen_do_cancellable(input_s, input_s.bytesize, token._wv_ptr, KITCHEN_SINK_KITCHEN_DO_CANCELLABLE_CALLBACK, ctx) - _wv_async_wait(queue, token) - ensure - own_token&.close - end +token = Kvstore::CancelToken.new +Thread.new { sleep 0.5; token.cancel } +begin + store.compact(60_000, cancel: token) +rescue Kvstore::Cancelled + # cancelled while the pause was running +ensure + token.close +end ``` `CancelToken#cancel` may be called from any thread; the call then raises -`Cancelled` unless it already completed. A token stays cancelled, can be -shared by several calls, and is released by `close` or the GC. Without a -token, interrupting the waiting thread (`Thread#raise`, `Timeout`) still -cancels the producer's work through a private token. Each async function -has one completion trampoline constant, so nothing the producer may still -call is ever garbage-collected. +`Cancelled` (code `-5`) unless it already completed. A token stays +cancelled, can be shared by several calls, and is released by `close` or +the GC. Without a token, interrupting the waiting thread (`Thread#raise`, +`Timeout`) still cancels the library's work through a private token. Each +async function has one completion trampoline constant, so nothing the +library may still call is ever garbage-collected. ## Callback interfaces @@ -187,31 +231,70 @@ A callback interface becomes a Ruby module whose methods raise including the module documents intent and supplies the defaults. ```ruby -class Recorder - include Events::Subscriber +class FileLoader + include Kvstore::Loader + + def name + 'files' + end - def route(topic) - Events::Delivery::ACCEPT + def fallback(_key) + nil + end + + def load(key) + path = File.join('/data', key) + raise Kvstore::KvError::KeyNotFound.new("no file for #{key}", key: key) unless File.exist?(path) + + File.binread(path) end end -``` -The implementation is stored in a handle table and the producer receives -its key, so Ruby object addresses never cross the boundary. The entry is -removed when the producer calls the vtable's `free`. The trampolines -copy borrowed strings and buffers, adopt object arguments, and report any -exception (including `NotImplementedError`) through -`{prefix}_error_set(out_err, -4, message)`; the caller sees -`Error` with code `-4` and the exception's message. +entry = store.get_or_load('settings', FileLoader.new) +``` -Producer threads may call back at any time: the ffi gem runs the -trampoline on a Ruby thread. In an API with callback interfaces every -call releases the GVL (`blocking: true`), so a producer thread can run a -callback while the call that triggered it is in flight. +Passing an implementation stores it in a handle table and hands the library +its key, so Ruby object addresses never cross the boundary; the entry is +removed when the library calls the vtable's `free`, which may happen on any +library thread. Each interface has one static vtable (an `FFI::Struct` whose +`{size, flags, free}` header is followed by one trampoline per method). A +required callback parameter rejects `nil` with `TypeError`; an optional one +(`loader = nil`) passes a null vtable. + +Arguments and returns follow the +[ABI](../reference/abi.md#callback-interfaces): + +- Strings, bytes, and value buffers arrive as copies; an object argument + arrives as a wrapper that adopted the reference the library passed, which + the implementation may keep. +- A returned number, boolean, or C-style enum crosses by value (checked + against its type's range); a returned object (`Store` or `Store?`) as a + fresh reference the library adopts (`nil` is null, which the library + rejects with `-3` where the return isn't optional); and a returned string, + bytes, or value buffer (a record, a rich enum, an optional, a list, or a + map) as a run allocated with `{prefix}_alloc`, which the library adopts. +- A method declared `throws` in the IDL may raise one of its module's + domain error classes: the library receives its code, its message, and its + fields as the payload, exactly as if the producer had raised it. Any other + exception (including `NotImplementedError`, and a domain error from a + method not declared `throws`) reaches the library as a callback failure + (`-4`) with the exception's message. Nothing unwinds through C. + +### Threads and the GVL + +Every call into the library except the trivial runtime helpers (`_clone`, +`_destroy`, an iterator's `_destroy`, `{prefix}_free_bytes`, `{prefix}_alloc`, +`{prefix}_debug_live`, the error and cancel-token functions, and the +load-time checks) releases the GVL +(`blocking: true`), so a long call doesn't stall other Ruby threads. That's +also what lets a callback arrive while the call that triggered it is in +flight: a callback on the calling thread reacquires the GVL there, and one +from a library thread (such as the notifications `compact` sends from its +worker) runs on the ffi gem's callback thread, which needs the GVL too. ## Iterators -An `iter` return is a lazy `Enumerator`. The producer iterator starts +An `iter` return is a lazy `Enumerator`. The library's iterator starts on the first pull, each step makes one `next` call, and the handle is destroyed exactly once, when iteration finishes, raises, or stops early (`first(2)`, `break`). @@ -223,8 +306,8 @@ destroyed exactly once, when iteration finishes, raises, or stops early - An `Enumerator` driven externally with `next` and then abandoned releases its iterator only when the GC collects it. - Callback interfaces are duck-typed: a missing method is detected only - when the producer calls it. + when the library calls it. - A callback that blocks waiting for the thread that made the triggering call deadlocks. -- Every IDL module shares one Ruby module, so two modules can't both - declare a function or type with the same Ruby name. +- A deprecated callable warns on every call (`warn ... uplevel: 1`); + silence it with `$VERBOSE = nil` like any Ruby warning. diff --git a/docs/src/generators/swift.md b/docs/src/generators/swift.md index f3e32cfe..ce9b193f 100644 --- a/docs/src/generators/swift.md +++ b/docs/src/generators/swift.md @@ -22,18 +22,48 @@ swift/ ``` The package, product, and Swift module are the package name in PascalCase -(`async-demo` becomes `AsyncDemo`); set `[generators.swift] module_name` to +(`key-store` becomes `KeyStore`); set `[generators.swift] name` to override it. The C module is always the Swift module's name with a `C` prefix. -Nothing else is configurable: the C symbol prefix and the native library name -come from the package identity. +The C symbol prefix and the native library name come from the package +identity. + +Types (records, enums, error domains, interfaces, callback protocols, and +iterator classes) sit at file scope, since type names are unique across an +API. Each IDL module's free functions become `static func`s of a caseless +namespace `enum` named after the module in PascalCase, a submodule's enum +nests inside its parent's, and the module's doc comment documents its enum: + +```swift +/// An embedded key-value store with listeners, policies, read-through +/// loaders, iteration, and async compaction. +public enum Kv { + public static func openStore(path: String) async throws -> Store { ... } + + /// Aggregate statistics, namespaced under `kv.stats`: ... + public enum Stats { + public static func summarize(store: Store, prefix: String?) throws -> Kvstore.Stats { ... } + } +} +``` + +Parameters, methods, record fields, and enum cases are lowerCamelCase +(`expiresAt`, `.keyNotFound`), and every argument is labeled +(`Kv.Stats.summarize(store: store, prefix: nil)`). + +| `[generators.swift]` key | Default | Meaning | +|---|---|---| +| `name` | `PascalCase(name)` | Package, product, and module name | +| `min_macos`, `min_ios` | `"11.0"`, `"13.0"` | The manifest's `platforms:` | +| `xcframework_url` | a placeholder | Where the packaged binary target downloads `C{Module}.xcframework.zip`; `{version}` and `{file}` are substituted | ## Build and link -The package builds as-is with SwiftPM. Depend on it by path or from a -repository, and tell the linker where `libkvstore` lives: +The package builds as-is with SwiftPM (its manifest declares +`swift-tools-version:5.9`). Depend on it by path or from a repository, and +tell the linker where `libkvstore` lives: ```swift -dependencies: [.package(path: "generated/swift")], +dependencies: [.package(path: "bindings/swift")], targets: [ .executableTarget(name: "App", dependencies: [.product(name: "Kvstore", package: "swift")]), ] @@ -50,14 +80,45 @@ doesn't apply. When a `CKvstore.xcframework` sits next to `Package.swift`, the manifest uses it as a binary target instead of the system library, so apps link without -flags. `weaveffi package` bundles the desktop libraries under `lib//` -and, when the platforms include an iOS slice, assembles the XCFramework itself -on macOS (see [Packaging](../guides/packaging.md#ios)). +flags. -Before the first call, the bindings check that the library implements C ABI -revision 3 and that every top-level module's contract checksum matches the -one they were generated from. A mismatch stops the process with a message -naming the module, since a stale binding would otherwise corrupt memory. +`weaveffi package` (on macOS) fuses the static libraries of every Apple +platform built (macOS, iOS, the iOS simulator) into +`CKvstore.xcframework.zip`, writes its SHA-256 checksum, and writes the +package `swift/Kvstore/`, whose manifest declares `CKvstore` as +`.binaryTarget(url:checksum:)` at `xcframework_url`. Upload the archive there +and publish the package; apps on any Apple platform then link with no flags +(see [Packaging](../guides/packaging.md#swift-and-the-xcframework)). + +## Load-time checks + +Before the first call (the first constructor, static, or free function), the +bindings check, once per process, that the library implements C ABI +revision 4 and that every declaration they were generated with is in its +top-level module's [contract table](../reference/abi.md#load-time-checks) +with the same signature. The expected entries are embedded in the wrapper: + +```swift +let wvContract: Void = { + wvCheckAbiVersion() + wvCheckContract(kvstore_kv_contract, [ + (0x0969575bfbb012d7, 0xebd38766e3532c4f, "kv.Store.fork"), + // ...one entry per declaration in `kv` and its submodules + ]) + wvCheckContract(kvstore_report_contract, [ + // ... + ]) +}() +``` + +Declarations the library adds are fine. A missing or changed one stops the +process with a message naming it, since a stale binding would otherwise +corrupt memory: + +```text +Kvstore: kv.Store.put is missing from the library 'kvstore'; regenerate the bindings or rebuild the library +Kvstore: kv.Store.put changed since these bindings were generated; regenerate the bindings or rebuild the library 'kvstore' +``` ## Type mapping @@ -74,13 +135,34 @@ naming the module, since a stale binding would otherwise corrupt memory. | `T?`, `[T]`, `{K: V}` | `T?`, `[T]`, `[K: V]` | Value buffer | | Interface | `final class`, `@unchecked Sendable` | Object pointer | | `Interface?` | optional wrapper | Nullable pointer | -| Callback interface | `protocol P: AnyObject, Sendable` | Context and vtable | +| Callback interface | `any P`, where `protocol P: AnyObject, Sendable` | Context and vtable | +| `Callback?` | `(any P)?` | Context and nullable vtable | | `iter` | generated `Sequence` class | Iterator handle | Strings cross as pointer and length in both directions, so interior NULs survive. Value buffers decode in place from the library's memory before it's released. Floats round-trip bit for bit, including NaN and `-0.0`. +Every type that crosses inside a value buffer conforms to the internal +`WvCodable` protocol: the primitives in the runtime, `Optional`, `Array`, and +`Dictionary` generically, and each record, enum, and interface where it's +declared. A composite type such as `[String: [Entry?]]` therefore has one +encoder and one decoder, the generic ones, rather than a loop at each use: + +```swift +extension Entry: WvCodable { + static func wvRead(_ r: inout WvReader) -> Entry { + Entry(key: r.read(), value: r.read(), kind: r.read(), version: r.read(), expiresAt: r.read(), tags: r.read(), metadata: r.read()) + } + + func wvWrite(_ w: inout WvWriter) { + w.write(self.key) + w.write(self.value) + // ... + } +} +``` + ## Objects and lifetime An interface becomes a `final class` holding one strong reference. `deinit` @@ -100,7 +182,7 @@ public final class Gadget: @unchecked Sendable { } /// Returns a new strong reference to the same object, for a position that - /// takes ownership of it (an object token inside a value buffer). + /// takes ownership of it. func clonePtr() -> OpaquePointer { wvNonNull(kitchen_sink_kitchen_Gadget_clone(ptr)) } @@ -126,42 +208,53 @@ public final class Gadget: @unchecked Sendable { ``` Swift keeps every argument and the receiver alive for the whole call, so a -wrapper can't be freed mid-call. Writing an object into a value buffer clones -its reference, and reading one adopts the reference into a new wrapper, so two -wrappers can share one native object. There's no `close()`: release happens -when the last Swift reference goes away. +wrapper can't be freed mid-call. Writing an object into a value buffer or +returning it from a callback method clones its reference, and reading one +adopts the reference into a new wrapper, so two wrappers can share one native +object. There's no `close()`: release happens when the last Swift reference +goes away. An interface member spelled like one the class declares (`ptr`, +`clonePtr`, `wvRead`, `wvWrite`) gains a trailing `_` ([reserved member +names](../reference/naming.md#identifiers-in-generated-code)). ## Errors -A module's error domain becomes an `enum` conforming to `Error` and -`LocalizedError`, one case per code, carrying the message (the code's -documented message when the library sends none) and any payload fields. -`errorCode` returns the numeric code: +A module's error domain becomes an `enum` conforming to `Error`, +`LocalizedError`, and `Sendable`, named after the domain with an `Error` +suffix unless it already has one (`KvError` stays `KvError`; `KitchenErrors` +becomes `KitchenErrorsError`). It has one case per code, carrying the message +(the code's documented message when the library sends none) and any payload +fields. `errorCode` returns the numeric code: ```swift -public enum KitchenErrorsError: Error, LocalizedError, Sendable { - case notFound(message: String) - case invalidInput(message: String) +public enum KvError: Error, LocalizedError, Sendable { + case keyNotFound(message: String, key: String) + case expired(message: String, key: String, expiredAt: Int64) + case invalidPath(message: String) + // ... } ``` -Throwing wrappers raise that enum for domain codes and -`{Module}RuntimeError` (with `errorCode` and `message`) for runtime failures: +A function declared `throws` raises that enum for domain codes and +`{SwiftModule}RuntimeError` (`KvstoreRuntimeError`, with `errorCode` and +`message`) for unknown codes and runtime failures: `-1` generic, `-2` panic, `-3` marshalling, and `-4` when a callback -implementation threw. Swift has no unchecked errors, so a function that -doesn't declare `throws` stops the process with `fatalError` when the library -reports a failure, with a message naming the function, code, and message (from -the `events` sample, where a subscriber threw): +implementation failed. Swift has no unchecked errors, so a function that +doesn't declare `throws` follows the [trap policy](../guides/errors-and-memory.md#the-trap-policy): +it stops the process with `fatalError` when the library reports a failure, +naming the function, the code, and the message: ```text -Events.publish(topic:text:tags:) failed with code -4: route rejected boom +Kvstore.count() failed with code -2: ``` ## Async and cancellation An async function becomes an `async` function over a checked continuation; the library's completion callback resumes it exactly once, from any thread. -A cancellable function also `throws` and runs inside +It's marked `throws` only when the IDL function declares errors or is +cancellable; otherwise a failure follows the trap policy +(`Kv.Stats.summarizeAll(stores:)` is `async -> Kvstore.Stats`). A +cancellable function runs inside `withTaskCancellationHandler`: it creates a native cancel token, passes it to the launch, and cancels it when the calling task is cancelled. A cancelled call (code `-5`) throws `CancellationError`, even when the task was cancelled @@ -196,24 +289,67 @@ reference for the call's duration. ## Callback interfaces -A callback interface becomes a protocol whose methods are all `throws`: +A callback interface becomes a class-bound protocol. Every requirement is +`throws`, so any implementation can fail the library's call in progress: ```swift -public protocol ReadyListener: AnyObject, Sendable { - /// Fires when an item is ready - func onReady(code: Int32, msg: String) throws - /// Receives the item itself and says whether to keep listening - func onItem(item: Item, gadget: Gadget) throws -> Bool +public protocol Loader: AnyObject, Sendable { + /// The loader's name, recorded in a loaded entry's metadata under + /// `source`. + func name() throws -> String + + /// A store that may already hold `key`, consulted first, if any. + func fallback(key: String) throws -> Store? + + /// The value for `key`. Fail with `KeyNotFound` naming + /// `key` when there's none. + /// + /// - Throws: ``KvError`` to report a declared failure with its fields, or any other error to fail the call. + func load(key: String) throws -> Data } ``` -Implement it with a class. The library may call it from any thread, which is -why the protocol requires `Sendable`; a class guarding its own state can -declare `@unchecked Sendable`. Passing an implementation retains it until the -library calls the vtable's `free`, and objects passed into a method are -adopted wrappers the implementation may keep. A thrown error is reported to -the library as code `-4` with the error's description; it never unwinds -through C. +Implement it with a class and pass an instance (`any Loader`) where the API +expects one. The library may call it from any thread, which is why the +protocol requires `Sendable`; a class guarding its own state can declare +`@unchecked Sendable`. Passing an implementation retains it until the library +calls the vtable's `free`, which may happen on any library thread, and +releases it then. An optional parameter (`loader: (any Loader)?`) passes a +null vtable for `nil`. + +Arguments and returns follow the [ABI](../reference/abi.md#callback-interfaces): + +- Strings, bytes, and value buffers arrive as copies; an object argument + arrives as a wrapper that adopted the reference the library passed, which + the implementation may keep. +- A returned number, `Bool`, or C-style enum crosses by value; a returned + object (`Store` or `Store?`) as a fresh reference the library adopts; and a + returned string, `Data`, or value buffer (a record, a rich enum, an + optional, a list, or a map) as a run allocated with `{prefix}_alloc`, which + the library adopts. +- A requirement declared `throws` in the IDL may throw the module's error + enum: the library receives its code, its message, and its fields as the + payload, exactly as if the producer had raised it. Any other error, and + any error from a requirement not declared `throws`, reaches the library as + a callback failure (`-4`) with the error's `localizedDescription`. Nothing + unwinds through C. + +```swift +final class FileLoader: Loader, @unchecked Sendable { + func name() throws -> String { "files" } + + func fallback(key: String) throws -> Store? { nil } + + func load(key: String) throws -> Data { + guard let data = FileManager.default.contents(atPath: "/data/\(key)") else { + throw KvError.keyNotFound(message: "no file for \(key)", key: key) + } + return data + } +} + +let entry = try store.getOrLoad(key: "settings", loader: FileLoader()) +``` ## Iterators @@ -228,12 +364,10 @@ property. - The library is linked at build time, so `{PREFIX}_LIBRARY` isn't honored and library paths come from the linker and loader. -- Contract mismatches, malformed value buffers, and failures of non-throwing - calls stop the process instead of throwing. -- Callback methods return `Void` or a direct value (a number, `Bool`, or - C-style enum); richer returns aren't part of the ABI yet. -- File-scope type names must be unique across modules, since every module's - types share the Swift module's namespace. A type that shares its name with a - module namespace is qualified with the Swift module name where needed. +- Load-time check failures, malformed value buffers, and failures of + non-throwing calls stop the process instead of throwing. +- A type that shares its name with a module's namespace `enum` (the `Stats` + record beside the `kv.stats` module's `Kv.Stats`) is qualified with the + Swift module name where the two would clash (`Kvstore.Stats`). - Interface wrappers aren't `Hashable`, so records that contain objects are only `Sendable`. diff --git a/docs/src/generators/wasm.md b/docs/src/generators/wasm.md index c5af40ba..343dcd61 100644 --- a/docs/src/generators/wasm.md +++ b/docs/src/generators/wasm.md @@ -21,9 +21,10 @@ wasm/ └── README.md ``` -The package name defaults to the library identity's name; set -`package_name` under `[generators.wasm]` to override it. `weaveffi package` -adds the prebuilt module as `{library}.wasm`. +The package name defaults to the package identity's `name`; set `name` under +`[generators.wasm]` to override it. `weaveffi package` writes the npm tarball +`wasm/{package}-{version}.tgz` with the module `weaveffi build` linked for +`wasm32` as `{library}.wasm`. ## Build the module @@ -36,12 +37,18 @@ RUSTFLAGS="-C link-arg=--export-table -C link-arg=--growable-table" \ cargo build --release --target wasm32-unknown-unknown ``` +`weaveffi build --platforms wasm32` passes those flags itself and writes the +module to `target/weaveffi/wasm32/{library}.wasm`. + ## Load `init()` loads the module and checks the contract before any other export -works: the module must implement ABI revision 3, and each top-level module's -checksum must match the bindings. Until the promise resolves, every other -call throws. +works: the module must implement ABI revision 4, and each top-level module's +contract table must hold every declaration the bindings were generated with +(see the [Node.js page](node.md#load-time-checks)). A failed check rejects +with an `Error` naming the declaration (`kvstore: kv.Store.put changed +since these bindings were generated`). Until the promise resolves, every +other call throws. ```js import { init, kv } from 'kvstore'; @@ -50,10 +57,12 @@ await init(); const store = kv.Store.open('/tmp/data'); ``` -Without an argument, `init()` loads `{library}.wasm` from next to `index.js` -(by `fetch` in a browser, from disk on Node.js), or, on Node.js, the file -named by the `{PREFIX}_LIBRARY` environment variable. It also takes a URL, a -path, bytes, a `Response`, or a compiled `WebAssembly.Module`: +Without an argument, `init()` reads the file named by the `{PREFIX}_LIBRARY` +environment variable (`KVSTORE_LIBRARY`) when it's set on Node.js, and +otherwise loads `{library}.wasm` from next to `index.js` (by `fetch` in a +browser, from disk on Node.js). It also takes a URL, a path (resolved +against `index.js`), bytes, a `Response`, or a compiled +`WebAssembly.Module`: ```ts export declare function init( @@ -84,8 +93,14 @@ returned strings and buffers out before releasing them: }, ``` -One error slot, one `out_len` slot, and one iterator item slot are allocated -at load and reused by every call. Object handles are linear-memory addresses. +Staged arguments are runs from `{prefix}_alloc`, released with +`{prefix}_free_bytes` once the call returns. Runs have alignment 1, so the +glue reads and writes multi-byte values through a `DataView`. One error +slot, one `out_len` slot, and one iterator item slot are reused by every +call; they and the callback vtables live in 8-byte-aligned blocks carved +from arenas the glue allocates once and never frees. `__debugLive(4)` +leaves those arenas out of its count of byte runs. Object handles are +linear-memory addresses. Async completions and callback-interface methods are JavaScript functions installed in the module's function table. The glue uses @@ -99,24 +114,36 @@ The mapping is the Node.js target's: 64-bit integers are `bigint`, `bytes` is `Uint8Array`, records are plain objects, rich enums are tagged unions, C-style enums are frozen objects, interfaces are classes with `close()` and `[Symbol.dispose]()`, optionals are `T | null`, lists are arrays, maps are -`Record`, and `iter` is a lazy `IterableIterator`. +`Record` objects (a `Map` is accepted as an argument), and `iter` is a +lazy `IterableIterator`. ## Async and cancellation Async functions return a `Promise`, and cancellable ones take an optional `{ signal }` last argument, exactly as on Node.js. The module is -single-threaded and the default executor runs a future to completion inside -its launcher, so an async call has usually completed by the time the -`Promise` is returned; only a signal aborted before the call can cancel it. -A cancelled call rejects with `CancelledError` (code -5). +single-threaded, and on `wasm32` the default executor polls a future inline +until it completes, inside its launcher, so an async call has completed by +the time the `Promise` is returned; only a signal aborted before the call +can cancel it. A future that's still pending once nothing can wake it (it +awaits something only another thread or a reactor would complete) fails +with code -1 instead (`async function suspended with no executor on +wasm32`). A cancelled call rejects with `CancelledError` (code -5). ## Callbacks and errors -A callback runs only while a call into the module is on the stack, on the -calling thread. An implementation that throws fails the call with code -4, -as on Node.js. Because `wasm32-unknown-unknown` aborts on a panic, a producer -panic traps the module; the trap surfaces as the package's root error with -code -2, and the module may be unusable afterward. +Callback interfaces work as on Node.js: returns of every family (a +string, bytes, or buffer return is copied into a run from `{prefix}_alloc` +that the producer adopts, and an object return is a new reference), domain +errors from methods declared `throws` with their fields, and `null` for an +optional callback (`Cb?`), which passes a null vtable. The vtable starts +with its `size`, `flags`, and `free` entries, like any consumer's. + +There are no threads, so a callback runs only while a call into the module +is on the stack, on the calling thread, and the Node.js addon's threading +rules don't arise. An implementation that throws fails the call with code +-4, as on Node.js. Because `wasm32-unknown-unknown` aborts on a panic, a +producer panic traps the module; the trap surfaces as the package's root +error with code -2, and the module may be unusable afterward. ## Emscripten @@ -126,8 +153,9 @@ returns), binds its underscore-prefixed exports, reads memory through `HEAPU8`, and installs table functions with `addFunction`. Link the module with `-sALLOW_TABLE_GROWTH`, `-sWASM_BIGINT`, and `-sEXPORTED_RUNTIME_METHODS=addFunction,HEAPU8`, and export the library's C -symbols, including `{prefix}_alloc` and `{prefix}_dealloc`. The packaged -layout then ships glue only. +symbols, including the runtime surface (`{prefix}_alloc`, +`{prefix}_free_bytes`, `{prefix}_error_set_payload`, and the rest) and each +`{prefix}_{module}_contract`. The packaged layout then ships glue only. ```ts export declare function init(module: object | Promise): Promise; @@ -137,7 +165,8 @@ export declare function init(module: object | Promise): Promise; - The module must export its function table and allow it to grow. - An async call can't be cancelled once launched, since it completes before - its launcher returns. + its launcher returns, and a future that waits on another thread or an + external event fails with code -1. - A producer panic traps rather than unwinding, which can leave the module's state inconsistent. - Emscripten mode isn't covered by the conformance suite. diff --git a/docs/src/getting-started.md b/docs/src/getting-started.md index ffa2ce35..fa714026 100644 --- a/docs/src/getting-started.md +++ b/docs/src/getting-started.md @@ -92,31 +92,37 @@ objects, callbacks, iterators, and async functions. ## Configure and generate Run `weaveffi init` in the crate. It writes a `weaveffi.toml` whose -`[project]` table points at `src/lib.rs`, then lists anything the crate still -needs (a `cdylib` crate type, the `weaveffi` dependency, the -`export_runtime!` call): +`[project]` table points at the crate itself, then lists anything the crate +still needs (a `cdylib` crate type, the `export_runtime!` call, a +`#[weaveffi::module]`): ```bash weaveffi init ``` +The file it writes generates all eleven targets; uncomment `targets` to pick +a subset: + ```toml # weaveffi.toml [project] -input = "src/lib.rs" +input = "." # this crate out = "bindings" -targets = ["c", "python"] # omit to generate all eleven +targets = ["c", "python"] ``` -Build the library and generate the bindings. With a `[project]` table, -`weaveffi generate` needs no arguments and works from any directory in the -project: +Generate the bindings. With a `[project]` table, `weaveffi generate` needs +no arguments and works from any directory in the project: ```bash -cargo build weaveffi generate ``` +`weaveffi generate` runs `cargo build`, then reads the API from the library +it built: the macro embeds a description of every exported declaration in +the library, so the bindings match the compiled code exactly, `#[cfg]` +included. `weaveffi extract` prints that description as an IDL. + The output has one directory per target. The C target writes `bindings/c/mathlib.h`; the Python target writes an installable package named `mathlib`. Run `weaveffi generate` again after any change: it rewrites @@ -125,16 +131,22 @@ that are no longer produced. ## Call it from Python -Install the generated package and point it at the library you built. Every -generated loader honors a `{PREFIX}_LIBRARY` environment variable, here -`MATHLIB_LIBRARY`; a packaged release bundles the library instead (see -[Packaging](guides/packaging.md)). +Run `weaveffi dev` instead of `weaveffi generate` while you iterate: it +builds the debug library, generates, and copies the library into the +generated Python package, which loads a bundled copy first. Then install the +package: ```bash +weaveffi dev pip install ./bindings/python -export MATHLIB_LIBRARY="$PWD/target/debug/libmathlib.dylib" # .so on Linux ``` +To load a library from somewhere else, set `MATHLIB_LIBRARY` to its full +path; the Python, Ruby, .NET, Dart, and Kotlin (JVM) packages all honor that +`{PREFIX}_LIBRARY` variable at run time, and `weaveffi dev` prints the value +for them. A packaged release bundles the library instead (see +[Packaging](guides/packaging.md)). + ```python import mathlib @@ -144,12 +156,12 @@ print(mathlib.greet("Python")) # Hello, Python! try: mathlib.div(1, 0) except mathlib.MathError as e: - print("caught:", e) # caught: division by zero + print(e.code, e.message) # 1 division by zero ``` -On import the package checks the library's ABI revision and the contract -checksum of the `math` module, so a library built from different source -fails to load with an error naming the module. The +On import the package checks the library's ABI revision and the `math` +module's contract table, so a library built from incompatible source fails +to load with an error naming the declaration that's missing or changed. The [Python page](generators/python.md) documents the generated surface. ## Call it from C @@ -165,7 +177,7 @@ returned strings are released with `mathlib_free_bytes`: int main(void) { if (mathlib_abi_version() != MATHLIB_ABI_VERSION || - mathlib_math_checksum() != MATHLIB_MATH_CHECKSUM) { + mathlib_math_contract_check() != 0) { fprintf(stderr, "mathlib.h does not match the loaded library\n"); return 1; } @@ -197,6 +209,25 @@ DYLD_LIBRARY_PATH=target/debug ./main # LD_LIBRARY_PATH on Linux [Errors and Memory](guides/errors-and-memory.md) states every ownership rule the generated bindings follow for you. +## Ship it + +```bash +weaveffi package --target python,node +``` + +`weaveffi package` builds the library in release mode for this machine +(that's `weaveffi build`), then writes installable artifacts to `dist/`: +here a wheel with the library inside, and npm tarballs carrying the library +and a prebuilt addon. Neither needs `MATHLIB_LIBRARY` or a compiler: + +```bash +pip install dist/python/mathlib-*.whl +npm install dist/node/*.tgz +``` + +[Packaging](guides/packaging.md) covers more platforms (iOS, Android, +Windows, `wasm32`), every target's artifacts, and publishing. + ## Implementing an IDL in C A producer doesn't have to be Rust. Start from an IDL, generate the C header, @@ -212,7 +243,7 @@ Edit `greeter.yml` to declare the API (the [IDL reference](reference/idl.md) has the full schema): ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: greeter functions: @@ -233,12 +264,19 @@ weaveffi generate --target c # writes bindings/c/greeter.h ```c /* greeter.c */ +#include #include #include #include "greeter.h" uint32_t greeter_abi_version(void) { return GREETER_ABI_VERSION; } -uint64_t greeter_greeter_checksum(void) { return GREETER_GREETER_CHECKSUM; } + +/* The contract table the header was generated with. */ +const greeter_contract_entry* greeter_greeter_contract(size_t* out_len) { + static const greeter_contract_entry table[] = GREETER_GREETER_CONTRACT; + *out_len = GREETER_GREETER_CONTRACT_LEN; + return table; +} int32_t greeter_greeter_add(int32_t a, int32_t b, greeter_error* out_err) { (void)out_err; /* written only on failure */ @@ -257,31 +295,72 @@ void greeter_error_clear(greeter_error* err) { memset(err, 0, sizeof *err); } +void greeter_error_set_payload(greeter_error* err, const uint8_t* ptr, size_t len) { + free((void*)err->payload_ptr); + err->payload_ptr = NULL; + err->payload_len = 0; + if (ptr && len) { + uint8_t* copy = malloc(len); + memcpy(copy, ptr, len); + err->payload_ptr = copy; + err->payload_len = len; + } +} + void greeter_error_free(greeter_error* err) { if (err) { greeter_error_clear(err); free(err); } } +/* One allocator for every byte run, returned or consumer-allocated. */ +uint8_t* greeter_alloc(size_t len) { return len ? calloc(len, 1) : NULL; } void greeter_free_bytes(uint8_t* ptr, size_t len) { (void)len; free(ptr); } -/* Also required: the four greeter_cancel_token_* functions and - greeter_debug_live (which may return 0). */ +/* Cancel tokens (used by async functions; this API has none, but the runtime + surface is the same for every library). */ +struct greeter_cancel_token { atomic_bool cancelled; }; +greeter_cancel_token* greeter_cancel_token_create(void) { + return calloc(1, sizeof(greeter_cancel_token)); +} +void greeter_cancel_token_cancel(greeter_cancel_token* token) { + if (token) atomic_store(&token->cancelled, true); +} +bool greeter_cancel_token_is_cancelled(const greeter_cancel_token* token) { + return token && atomic_load(&((greeter_cancel_token*)token)->cancelled); +} +void greeter_cancel_token_destroy(greeter_cancel_token* token) { free(token); } + +/* Leak counters: a producer that doesn't count returns 0 for every kind. */ +uint64_t greeter_debug_live(int32_t kind) { (void)kind; return 0; } ``` The library must export every runtime symbol the header declares, each -top-level module's checksum function, and the API itself; the +top-level module's contract function, and the API itself; the [C ABI contract](reference/abi.md#runtime-surface) lists them, and [`conformance/c/producer.c`](https://github.com/weavefoundry/weaveffi/blob/main/conformance/c/producer.c) is a complete hand-written producer to copy from. Build it as -`libgreeter.so` (`libgreeter.dylib`, `greeter.dll`), then run -`weaveffi generate` for the other targets; they load it exactly as they load -a Rust producer. +`libgreeter.so` (`libgreeter.dylib` on macOS, `greeter.dll` on Windows): + +```bash +cc -shared -fPIC -I bindings/c greeter.c -o libgreeter.so +``` + +Then run `weaveffi generate` for the other targets; they load it exactly as +they load a Rust producer. From Python, for example: + +```bash +weaveffi generate --target python +pip install ./bindings/python +GREETER_LIBRARY=$PWD/libgreeter.so python3 -c "import greeter; print(greeter.add(2, 3))" # 5 +``` ## Next steps -- [Project Configuration](guides/config.md): `[package]` metadata, per-target - options, and the generation cache. -- [Samples](samples.md): six complete producers, from `calculator` to the - kitchen-sink `kvstore`. +- [Project Configuration](guides/config.md): `[package]` metadata, `[build]` + settings, per-target options, and how regeneration cleans up stale files. +- [Packaging](guides/packaging.md): building every platform and publishing + the artifacts. +- [Samples](samples.md): three complete producers, from the minimal + `calculator` to the feature-complete `kvstore`. - [Generators](generators/README.md): what each language gets. - Gate CI on `weaveffi diff --check` so committed bindings can't drift; see [Stability and Versioning](stability.md#ci-workflow). diff --git a/docs/src/guides/async.md b/docs/src/guides/async.md index a92824b4..f4bfb1c2 100644 --- a/docs/src/guides/async.md +++ b/docs/src/guides/async.md @@ -11,19 +11,19 @@ coroutine, or promise cancels the native call. In Rust, write an `async fn` (a free function, method, or static). Add `#[weaveffi::cancellable]` and a final `weaveffi::CancelToken` parameter to -make it cancellable: +make it cancellable. This is the `kvstore` sample's `Store::compact`: ```rust #[weaveffi::interface] pub struct Store { /* ... */ } impl Store { - /// Reclaim space; returns the bytes reclaimed. + /// Remove every expired entry and complete with how many were removed, + /// after pausing for `pause_ms` on a thread that polls the token. #[weaveffi::cancellable] - pub async fn compact(&self, cancel: weaveffi::CancelToken) -> Result { - let _ = cancel; - /* ... */ - Ok(0) + pub async fn compact(&self, pause_ms: u32, cancel: weaveffi::CancelToken) -> u32 { + Pause::start(pause_ms, &cancel).await; + self.sweep() } } ``` @@ -32,13 +32,13 @@ In an IDL, set `async: true` and optionally `cancellable: true` on a function, method, or static: ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: kv errors: name: KvError codes: - - { name: IoError, code: 1004, message: "I/O failure" } + - { name: InvalidPath, code: 1004, message: "invalid path" } interfaces: - name: Store constructors: @@ -48,16 +48,20 @@ modules: throws: true methods: - name: compact - return: i64 - throws: true + params: + - { name: pause_ms, type: u32 } + return: u32 async: true cancellable: true ``` `async` composes with `throws`, with object and buffered results, and with -every parameter type. Constructors can't be async (use an async static that -returns the interface), async functions can't return iterators, and -`cancellable` without `async` is a validation error (`CancellableNotAsync`). +every parameter type. Constructors can't be async (`AsyncConstructor`): +declare an async static that returns the interface instead, which in Rust is +an `async fn` associated function returning `Self` (the macro reads only a +synchronous one as a constructor). Async functions can't return iterators +(`AsyncIteratorReturn`), and `cancellable` without `async` is a validation +error (`CancellableNotAsync`). ## The C ABI shape @@ -66,10 +70,11 @@ completion callback typedef named `{symbol}_callback`: ```c typedef void (*kvstore_kv_Store_compact_callback)( - void* context, kvstore_error* err, int64_t result); + void* context, kvstore_error* err, uint32_t result); void kvstore_kv_Store_compact( const kvstore_kv_Store* self, + uint32_t pause_ms, kvstore_cancel_token* cancel_token, /* cancellable functions only */ kvstore_kv_Store_compact_callback callback, void* context); @@ -86,8 +91,10 @@ callback. The runtime guarantees these for every launch: 1. **Exactly once.** The callback fires exactly once, on success, on a - domain error, on a marshalling failure (`-3`), on a panic (`-2`), on a - consumer callback failure (`-4`), and on cancellation (`-5`). + domain error, on a marshalling failure (`-3`), on a panic (`-2`, including + one while the launcher lifts its inputs), on a consumer callback failure + (`-4`), on cancellation (`-5`), and when the executor can't take the call + (`-1`). 2. **From any thread.** It runs on whatever thread completes the future: an executor thread, a Tokio worker, or (on `wasm32`) the caller's own stack. Each binding hops back to its own scheduler before touching consumer state. @@ -99,6 +106,9 @@ The runtime guarantees these for every launch: 4. **Dropped futures complete.** If the executor drops a future before it finishes (a runtime shutting down, say), the drop guard completes the call with `-5`. +5. **Spawn failures complete.** If the executor can't take the future (a + custom spawner panics, or the default executor can't start its threads), + the call completes with `-1` and a message saying why. A consumer of the raw C surface follows the same rules in its callback: read or decode everything, free it, and adopt object pointers. @@ -109,7 +119,7 @@ The runtime surface provides reference-counted cancel tokens: ```c kvstore_cancel_token* token = kvstore_cancel_token_create(); /* refcount 1 */ -kvstore_kv_Store_compact(store, token, on_done, ctx); +kvstore_kv_Store_compact(store, 100, token, on_done, ctx); kvstore_cancel_token_cancel(token); /* any thread, any time; idempotent */ kvstore_cancel_token_destroy(token); /* drops the consumer's reference */ ``` @@ -151,14 +161,35 @@ language page. ## Choosing an executor Something has to drive each future between launch and completion. The -runtime routes every future to the process-wide spawner: +runtime routes every future to the process-wide executor: + +- **Default.** A fixed pool of worker threads, one per available core and + at least two, started lazily on the first launch. The workers share one + run queue, and a future's waker puts it back on the queue, so a launch + never costs a thread and a future woken from any thread resumes on the + pool. The pool has no reactor, so a future that awaits Tokio's I/O or + timers never wakes, and because its workers are few, a future that blocks + one for a long time (synchronous I/O, a long computation) holds up every + other async call. Run such work on a runtime of its own: enable the + `tokio` feature or install a spawner. +- **The `tokio` feature.** With the `weaveffi` crate's `tokio` feature on, + the default executor is Tokio: the current runtime when the launcher is + called from inside one, otherwise a multi-thread runtime the library + creates on first use (its threads are named `weaveffi-async`). That + runtime enables every Tokio driver the build compiles in, so turn on the + Tokio features your futures need (`time`, `net`, and so on) in your own + dependency on `tokio`. Use `tokio::task::spawn_blocking` for synchronous + work inside a future. + + ```toml + [dependencies] + weaveffi = { version = "0.24", features = ["tokio"] } + tokio = { version = "1", features = ["time"] } + ``` -- **Default.** Each launch gets a fresh thread that drives the future with a - minimal executor. It needs no runtime and suits CPU-bound work and futures - woken from other threads, but it has no reactor: a future that awaits - Tokio's I/O or timers never wakes. - **Custom.** Call `weaveffi::set_spawner` once at startup, before the first - async launch. It accepts any `Fn(BoxFuture) + Send + Sync + 'static`: + async launch; it overrides both defaults. It accepts any + `Fn(BoxFuture) + Send + Sync + 'static`: ```rust let handle = tokio::runtime::Handle::current(); @@ -171,19 +202,25 @@ runtime routes every future to the process-wide spawner: The first call wins; later calls return `Err(SpawnerAlreadySet)`. The spawner must not block, because the launcher calls it on the consumer's thread. Every future it receives is already wrapped so a panic becomes a - `-2` completion, and a spawner that drops a future still produces a `-5` - completion. -- **`wasm32`.** There are no threads, so the default drives the future - inline before the launcher returns. Futures that finish after bounded work - are fine; one that waits for an external wake (a timer, I/O) hangs the - module. A call usually completes before an `AbortSignal` can fire, so on - this target cancellation mostly matters for a signal that's already - aborted when the call starts. + `-2` completion, a spawner that drops a future still produces a `-5` + completion, and a spawner that panics completes the call with `-1`. +- **`wasm32`.** There are no threads, so the default polls the future + inline before the launcher returns. A future that finishes after bounded + work is fine. A future that's still pending once nothing can wake it (it + awaits a timer or I/O) is dropped, and the call completes with `-1` and + the message "async function suspended with no executor on wasm32" rather + than hanging the module. A call usually completes before an `AbortSignal` + can fire, so on this target cancellation mostly matters for a signal + that's already aborted when the call starts. ## Pitfalls -- **Awaiting Tokio under the default spawner.** The future never wakes and - the callback never fires; install a Tokio spawner first. +- **Awaiting Tokio under the default executor.** The pool has no reactor, + so the future never wakes and the callback never fires; enable the `tokio` + feature or install a Tokio spawner. +- **Blocking a pool worker.** A future that blocks (synchronous I/O, a long + computation) stalls every other call queued behind it on the default + pool; move the work to a runtime of its own. - **Expecting the token to stop CPU-bound work.** Cancellation drops the future at a suspension point. Long synchronous loops should yield or check `is_cancelled()`. @@ -192,5 +229,5 @@ runtime routes every future to the process-wide spawner: - **Leaking a result buffer.** A raw C callback that only copies the result must still free it. - **Async functions with no return value.** Valid, but `weaveffi validate - --warn` flags them (`AsyncVoidFunction`), since they're usually a missing - return type. + --warn` flags an async free function that returns nothing + (`AsyncVoidFunction`), since it's usually a missing return type. diff --git a/docs/src/guides/config.md b/docs/src/guides/config.md index a7795e2a..c98223da 100644 --- a/docs/src/guides/config.md +++ b/docs/src/guides/config.md @@ -1,14 +1,15 @@ # Project Configuration A project's settings live in one `weaveffi.toml` at its root. The API -definition (annotated Rust or an IDL) describes only the API; this file says -where it is, what the library is called, and how each target is generated. +definition (a Rust producer crate or an IDL) describes only the API; this file says +where it is, what the library is called, how it's built and packaged, and how +each target is generated. Every table is optional, and every unknown table or key is an error, so a typo fails the run instead of being ignored. ```toml [project] -input = "src/lib.rs" # or an IDL such as "kvstore.yml" +input = "." # this crate, or an IDL such as "kvstore.yml" out = "bindings" targets = ["c", "swift", "kotlin", "python"] @@ -20,20 +21,21 @@ license = "MIT" authors = ["Example "] repository = "https://github.com/example/kvstore" -[global] -post_generate = "swiftformat bindings/swift" +[build] +platforms = ["darwin-arm64", "ios-arm64", "ios-sim-arm64", "android-arm64"] [generators.kotlin] -package = "com.example.kvstore" +name = "com.example.kvstore" ``` `weaveffi init` writes a starter file: in a Rust crate it points `input` at -`src/lib.rs`; elsewhere it also writes a starter IDL. +the crate (`"."`); elsewhere it also writes a starter IDL. ## Discovery -With an explicit input (`weaveffi generate api.yml`), the CLI uses the nearest -`weaveffi.toml` at or above the input's directory. With no input, it looks for +With an explicit input (`weaveffi generate api.yml`, `weaveffi generate +path/to/crate`), the CLI uses the nearest `weaveffi.toml` at or above the +input (for a crate, at or above its directory). With no input, it looks for `weaveffi.toml` in the current directory and its parents and reads `[project] input`, so `weaveffi generate` works from anywhere inside the project, like `cargo build`. `--config ` names a file explicitly. @@ -43,8 +45,8 @@ Without any file, every setting takes its default. | Key | Default | Meaning | |-----|---------|---------| -| `input` | none | The API definition: a `.rs` file or a `.yml`, `.yaml`, `.json`, or `.toml` IDL | -| `out` | `generated` | Output directory for `generate` and `diff` | +| `input` | none | The API definition: a Rust producer crate (its directory, usually `"."`, or its `Cargo.toml`), whose API is read from its built library (see [Library Mode](extract.md)), or a `.yml`, `.yaml`, `.json`, or `.toml` IDL | +| `out` | `bindings` | Output directory for `generate` and `diff` | | `targets` | all eleven | Targets to generate when `--target` isn't given | Paths are relative to the directory holding `weaveffi.toml`. Command-line @@ -64,91 +66,105 @@ these three. | `description`, `license`, `authors`, `homepage`, `repository` | Manifest metadata | | `c_prefix` | C symbol prefix; IDL inputs only | | `library` | Native library base name (`lib{library}.so`); IDL inputs only | +| `dist` | Where `weaveffi package` writes its artifacts (default `dist`); not part of the identity | The rules depend on the input: -| | Rust producer (`.rs`) | IDL | +| | Rust producer (a crate) | IDL | |-|-----------------------|-----| | `name` | `[package] name`, else the crate's Cargo package name | `[package] name`, else the input file stem | | `prefix` | the crate's library name (`[lib] name`, else the package name with `-` mapped to `_`) | `c_prefix`, else snake-case `name` | | `library` | same as `prefix` (the cdylib Cargo builds) | `library`, else snake-case `name` | -| metadata | `[package]`, falling back to `Cargo.toml` | `[package]` | +| metadata | `[package]`, falling back to `cargo metadata` (workspace-inherited `version.workspace = true` included) | `[package]` | A Rust producer's prefix is what the macro compiled into its symbols, so it -can't be configured: setting `c_prefix` or `library` for a `.rs` input is an -error. Renaming the published package with `name` is fine. +can't be configured: setting `c_prefix` or `library` for a Rust producer is +an error. Renaming the published package with `name` is fine. -## `[global]` +## `[build]` -| Key | Meaning | -|-----|---------| -| `strip_module_prefix` | Sets `strip_module_prefix` on every target that supports it, overriding their own tables | -| `pre_generate` | Shell command run once before `generate` writes anything | -| `post_generate` | Shell command run once after `generate` finishes writing | - -Targets that place every module's members in one flat namespace strip the -module name from free functions by default (`get_stats`, not -`kv_stats_get_stats`). Set `strip_module_prefix = false` when two modules -declare the same function name; targets that namespace each module -separately don't need it. Each language page says which applies. +How `weaveffi build` and `weaveffi package` compile a Rust producer (see +[Packaging](packaging.md)): -Hooks run through `sh -c` (`cmd /C` on Windows), only when at least one -target actually regenerates; an up-to-date run skips them. `weaveffi diff` -never runs them. Don't put untrusted input in a hook. +| Key | Default | Meaning | +|-----|---------|---------| +| `platforms` | the host | Platform ids to build when `--platforms` isn't given | +| `release` | `true` | Build with the release profile (`--debug` overrides it) | +| `macos_deployment_target` | `"11.0"` | `MACOSX_DEPLOYMENT_TARGET`, and the macOS version wheel tags carry | +| `ios_deployment_target` | `"13.0"` | `IPHONEOS_DEPLOYMENT_TARGET` | +| `android_api` | `21` | The minimum Android API level, which picks the NDK compiler | +| `manifest` | the project's crate | The producer's `Cargo.toml`, for an IDL project built from a Rust crate | ## `[generators.]` One table per target (`c`, `cpp`, `swift`, `kotlin`, `node`, `wasm`, `python`, `dotnet`, `dart`, `go`, `ruby`) holds that generator's options. -These keys override names derived from the identity: +Every target that names something after the package spells the override +`name`: + +| Table | `name` sets | Default | +|-------|-------------|---------| +| `[generators.cpp]` | the C++ namespace | `{prefix}` | +| `[generators.swift]` | the SwiftPM package, product, and module (the C module is `C{name}`) | `PascalCase(name)` | +| `[generators.kotlin]` | the Kotlin package | `{prefix}` | +| `[generators.node]` | the npm package | `{name}` | +| `[generators.wasm]` | the npm package | `{name}` | +| `[generators.python]` | the PyPI distribution | `{name}` | +| `[generators.dotnet]` | the namespace, assembly, and NuGet id | `PascalCase(name)` | +| `[generators.dart]` | the pub package | `{prefix}` | +| `[generators.go]` | the module path | `{name}` | +| `[generators.ruby]` | the gem | `{name}` | + +The C target has no package name. A few targets name a second thing: | Table | Key | Default | |-------|-----|---------| -| `[generators.cpp]` | `namespace`, `header_name` | `{prefix}`, `{library}.hpp` | -| `[generators.swift]` | `module_name` | `PascalCase(name)` | -| `[generators.kotlin]` | `package` | `{prefix}` | -| `[generators.node]` | `package_name` | `{name}` | -| `[generators.wasm]` | `package_name` | `{name}` | -| `[generators.python]` | `package_name`, `import_name` | `{name}`, `{prefix}` | -| `[generators.dotnet]` | `namespace` | `PascalCase(name)` | -| `[generators.dart]` | `package_name` | `{prefix}` | -| `[generators.go]` | `module_path` | `{name}` | -| `[generators.ruby]` | `gem_name`, `module_name` | `{name}`, `PascalCase(name)` | - -Every other option (Kotlin's Android or JVM flavor, Wasm's Emscripten mode, -and so on) is documented on the target's page under -[Generators](../generators/README.md). No target has its own C prefix: the -prefix belongs to the library. - -## Generation and caching +| `[generators.cpp]` | `header_name` | `{library}.hpp` | +| `[generators.python]` | `import_name` (the import package) | `{prefix}` | +| `[generators.ruby]` | `module_name` (the top-level Ruby module) | `PascalCase(name)` | + +The constants that go into package manifests are options too: + +| Table | Key | Default | +|-------|-----|---------| +| `[generators.swift]` | `min_macos`, `min_ios` (the `platforms:` of `Package.swift`) | `"11.0"`, `"13.0"` | +| `[generators.swift]` | `xcframework_url` (where the packaged binary target downloads from; `{version}` and `{file}` are substituted) | a placeholder | +| `[generators.kotlin]` | `min_sdk`, `compile_sdk` | `21`, `35` | +| `[generators.python]` | `requires_python` | `">=3.9"` | +| `[generators.node]` | `node_engine` (`engines.node`) | `">=18"` | +| `[generators.dart]` | `sdk` (the pubspec SDK constraint) | `">=3.10.0 <4.0.0"` | + +Keep `min_macos`, `min_ios`, and `min_sdk` at or above the `[build]` +deployment targets and API level, or the toolchains warn that the library is +newer than the package claims. The remaining options (C's +`buffer_helpers`, C++'s `standard`, Kotlin's `flavor`, and Wasm's +`emscripten`) are documented on the target's page +under [Generators](../generators/README.md). No target has its own C prefix: +the prefix belongs to the library. + +## Generation records Rendering is pure: each target renders its files in memory, and the -orchestrator does every write. After a target generates, the orchestrator -writes a record to `{out}/.weaveffi-cache/{target}.json` holding a hash of -every input that affects the output (the API, the identity, the target's -config, and the CLI version) and the path and hash of every file written. On -the next run: - -- A target is skipped when its input hash matches and every recorded file is - still on disk unmodified. Deleting or hand-editing a generated file - regenerates it. -- A regenerated target rewrites only files whose contents changed, so file - timestamps (and incremental builds) stay stable. -- Files the previous run wrote that the current run no longer produces (a - renamed type, a removed module) are deleted. Files you added under the - output directory (a `node_modules/`, a build directory) are never touched. - -`--force` ignores the records and regenerates every selected target. +orchestrator does every write. Every run renders every selected target, then: + +- Only files whose contents changed are rewritten, so file timestamps (and + incremental builds) stay stable. When nothing changed, nothing is written. +- The orchestrator records the files each target produced in + `{out}/.weaveffi-cache/{target}.json`. Files the previous run wrote that + the current run no longer produces (a renamed type, a removed module) are + deleted. Files you added under the output directory (a `node_modules/`, a + build directory) are never touched. + `--dry-run` validates and prints the files that would be written. ## CI `weaveffi diff --check` regenerates in memory and compares with the output -directory without writing anything or running hooks: +directory without writing anything: ```bash weaveffi diff --check # uses [project] -weaveffi diff src/lib.rs -o bindings --target c,swift --check +weaveffi diff path/to/crate -o bindings --target c,swift --check ``` It exits `0` when the directory is up to date, `2` when files would change, @@ -165,5 +181,8 @@ warnings, which never change the exit status. Commit the file next to the definition and let discovery find it. - **Name overrides don't rename the library.** A target's package or module override changes what consumers import, not which native library it loads; - that's always `{library}`, overridable at run time with the - `{PREFIX}_LIBRARY` environment variable. + that's always `{library}`. The targets that load the library at run time + (Python, Ruby, .NET, Dart, Kotlin on the JVM, and WebAssembly on Node.js) + also accept a full path in the `{PREFIX}_LIBRARY` environment variable; + C, C++, Swift, Go, and the Node.js addon link it when they're built (see + each target's page). diff --git a/docs/src/guides/errors-and-memory.md b/docs/src/guides/errors-and-memory.md index 43425d51..f2a1c2bb 100644 --- a/docs/src/guides/errors-and-memory.md +++ b/docs/src/guides/errors-and-memory.md @@ -19,6 +19,30 @@ normative text. Examples use the `kvstore` sample, whose prefix is `kvstore`. *Borrowed* means the callee may read the value only until the call returns and copies anything it keeps. Everything a producer hands back belongs to the receiver, who releases it exactly once with the function in the last column. +The one transfer in the other direction is a callback method's return value, +which the producer adopts (see [Callback interfaces](#callback-interfaces)). + +## Byte runs + +Every string, bytes, and value buffer the producer returns is a **byte run** +it allocated, and the producer owns the allocator. Two runtime functions +manage runs: + +```c +uint8_t* kvstore_alloc(size_t len); /* a zero-filled run, NULL for 0 */ +void kvstore_free_bytes(uint8_t* ptr, size_t len); /* returned runs and alloc runs */ +``` + +A consumer releases a returned run with `{prefix}_free_bytes(ptr, len)`, +passing the exact length it received. It allocates a run with +`{prefix}_alloc(len)` only when it hands bytes *to* the producer: a callback +method's string, bytes, or buffer return, which the producer adopts and +frees, or (on `wasm32`) an argument staged in the module's linear memory, +which the consumer frees itself with `{prefix}_free_bytes`. Null with length +`0` is the empty run everywhere: `{prefix}_alloc(0)` returns null, and +`{prefix}_free_bytes(ptr, 0)` is a no-op. Never free a run with the +language's own allocator, and never hand the producer memory that didn't come +from `{prefix}_alloc`. ## Strings and bytes @@ -35,11 +59,11 @@ passing the exact length it received. An empty result may come back as `NULL` with length `0`. ```c -/* The calculator sample's `echo(s: string) -> string`. */ +/* The calculator sample's `greet(name: string) -> string`. */ calculator_error err = {0}; size_t len = 0; const uint8_t* text = - calculator_calculator_echo((const uint8_t*)"hi", 2, &len, &err); + calculator_calculator_greet((const uint8_t*)"Ada", 3, &len, &err); if (err.code == 0) { printf("%.*s\n", (int)len, (const char*)text); calculator_free_bytes((uint8_t*)text, len); @@ -96,21 +120,39 @@ The [capability matrix](../generators/README.md) lists each target's idiom. A callback-interface parameter lowers to `void* {name}_ctx` plus `const {vtable}* {name}_vtable`. The consumer owns `ctx` (a generated binding uses a key into a table that keeps the implementation alive) and a static -vtable per interface. The producer may call any entry, any number of times, -from any thread, until it calls `free(ctx)` exactly once. +vtable per interface, whose header records its `size` and `free` hook. The +producer may call any entry, any number of times, from any thread, until it +calls `free(ctx)` exactly once; `free` may also run on any producer thread. +An optional callback parameter (`Cb?`) passes a null vtable for none. Arguments to a callback method follow the parameter rules above, seen from the consumer: strings, bytes, and buffers are borrowed for the duration of the call and must be copied or decoded before returning; objects transfer one reference the consumer adopts. +A callback method's return transfers to the producer: + +| Return | Slot | Ownership | +|--------|------|-----------| +| direct value | the C return | copied | +| object (`I`, `I?`) | the C return, `{prefix}_{path}_{I}*` | one strong reference (a fresh `_clone`) the producer adopts; `I` must not be null | +| `string`, `bytes`, value buffer | trailing `uint8_t** out_ptr, size_t* out_len` | a run the consumer allocated with `{prefix}_alloc`, which the producer adopts and frees | + +The producer adopts whatever the slots hold even when the method fails, so a +consumer that fails after allocating doesn't leak. A return the producer +can't accept (text that isn't UTF-8, a null `I`, an out-of-range enum value, +a malformed buffer) fails the method with `-3`. + ## Iterators An `iter` return is an opaque handle. `{Iter}_next` writes one element (returning `1`) or reports the end (returning `0`); each element is owned by the consumer under the return rules above. `{Iter}_destroy` is called exactly once, either on exhaustion or when the consumer abandons the iteration early. -The generated wrappers do both. +The generated wrappers do both. An iterator is advanced by one caller at a +time: a `_next` that arrives while another is in progress on the same +iterator (from another thread, or re-entrantly from a callback the producer's +`next` calls) fails with `-3` rather than blocking. ## Errors @@ -152,9 +194,9 @@ success), released with `{prefix}_error_free`, which also frees the box. |------|---------| | `0` | success | | `> 0` | a declared code of the module's [error domain](../reference/idl.md#error-domains) | -| `-1` | generic: an untyped producer error (`Result`, an error type without a domain code) | +| `-1` | generic: an untyped producer error (`Result`, an error type without a domain code), or an async call the executor couldn't start | | `-2` | the producer panicked; the message carries the panic text | -| `-3` | marshalling failure: a null or invalid argument, non-UTF-8 text, an out-of-range enum value, a malformed value buffer | +| `-3` | marshalling failure: a null or invalid argument, non-UTF-8 text, an out-of-range enum value, a malformed value buffer (including a map with a repeated key), a callback vtable smaller than the producer's, an iterator advanced concurrently | | `-4` | a consumer's callback-interface implementation failed; the message is the consumer's | | `-5` | cancelled: an async call's cancel token fired, or its executor dropped it (async completions only) | @@ -172,10 +214,9 @@ flag, and every target applies the same rule: producer's message, and the payload fields as properties. A negative code surfaces as the package's root error type with that code. - **No `throws`.** The function can't report a domain error, so any non-zero - code is a producer bug or a failed callback. The binding raises it through - the target's programming-error idiom (an unchecked exception, a Go panic, - a Swift `fatalError` that includes the code and message) and never dresses - it up as a domain error. + code is a bug, and the binding traps on it (see + [The trap policy](#the-trap-policy)). It never dresses the code up as a + domain error. - **`-5` on an async call** always surfaces as the language's own cancellation error (Swift's `CancellationError`, Kotlin's `CancellationException`, .NET's `OperationCanceledException`, Python's @@ -189,6 +230,31 @@ A Rust producer's domain error message is its `Display` output. An IDL `message:` (or a Rust variant's doc comment) is the documented default that generated docs and consumer-side constructors use. +### The trap policy + +A failure of a call that isn't `throws` is a producer bug (a panic, an +argument the producer couldn't lift, a callback failure it let through a +call that can't report one), and every binding treats it the same way: it +raises the language's unchecked error rather than a declared one, naming the +runtime code and the producer's message so the bug can be diagnosed from the +report. + +| Target | What a failed non-throwing call raises | +|--------|-----------------------------------------| +| C | nothing; the caller reads `err.code` itself | +| C++ | `{namespace}::InternalError` (a `std::runtime_error`) | +| Swift | `fatalError`, since Swift has no unchecked errors (as in UniFFI) | +| Kotlin | `NativeBugException` (an `IllegalStateException`) | +| Node.js, WebAssembly | the package's root error class, `{PascalCase(name)}Error` (`KvstoreError`), with the negative `code` | +| Python | `InternalError` (a `RuntimeError`) | +| .NET | `NativeBugException` (an `InvalidOperationException`) | +| Dart | `NativeError` (an `Error`, not an `Exception`) | +| Go | a `panic` whose value is the package's `*Error` | +| Ruby | `NativeBugError` (a subclass of the gem's root `Error < StandardError`) | + +Cancellation (`-5`) isn't a bug and always surfaces as the language's +cancellation error, as above. + ### Payloads An error code may declare fields. When the producer raises it, the fields are @@ -201,21 +267,48 @@ no fields leaves the payload null. `{prefix}_error_clear` and A consumer's callback implementation can fail in the consumer's language. The binding catches the exception in its trampoline and reports it through the -vtable entry's `out_err` by calling +vtable entry's `out_err` with the runtime surface: ```c void kvstore_error_set(kvstore_error* err, int32_t code, const char* message); +void kvstore_error_set_payload(kvstore_error* err, const uint8_t* ptr, size_t len); ``` -with code `-4` and a borrowed message, which the producer copies with its own -allocator. A hand-written C consumer must do the same and never store its own -allocation in `message`. A positive code written there is reported as `-4`, -so a consumer failure can't impersonate a domain error. The producer then -abandons the call it was making (or handles the failure, if its callback -method returns `Result`), and the original caller sees `-4` -with the consumer's message. Because `-4` isn't a domain code, a consumer -that wants to catch its own callback failures should call the producer -through a `throws: true` function. +Both copy their arguments with the producer's allocator; a consumer never +stores its own allocation in `message`. What the producer receives depends +on the method: + +- A method declared `throws` (it needs an error domain in scope for its + module) may report a positive code of that domain, attaching the code's + fields encoded as a value buffer with `{prefix}_error_set_payload`. The + fields may not include objects. The producer receives the code, message, + and payload as written. +- Every other failure, including a code the domain doesn't declare or a + positive code from a method without `throws`, reaches the producer as `-4` + with the consumer's message, so a consumer bug can't impersonate a domain + error. + +The producer decides what the failure means for the call in progress. Nothing +unwinds and nothing is deferred: in a Rust producer every callback trait +method returns `Result`, and the failure arrives +as an `Err`: + +```rust +pub struct ForeignError { + pub code: i32, // -4, a declared domain code, or -3 for a bad return + pub message: String, // the consumer's message + pub payload: Vec, // a domain code's fields, else empty +} +``` + +`ForeignError::domain::()` decodes a declared domain code and its payload +into the module's `#[weaveffi::error]` enum, returning `None` for anything +else. A producer function that returns `Result` (and is +therefore `throws`, so its module needs an error domain in scope) can +propagate the failure with `?`, and the original caller then sees the +consumer's code, message, and payload. A function that handles the failure +itself returns whatever it likes. See the +[producer macro guide](producer-macro.md#callback-interfaces) for examples. ### Panics @@ -224,6 +317,17 @@ A Rust producer's thunks catch panics and report them as `-2`. Destructors swallowed rather than unwinding into C. Async functions report panics through the completion callback. +## Leak checks + +`{prefix}_debug_live(kind)` reports how many resources are live, so a test +harness can assert that a consumer released everything: `0` objects, `1` +foreign callbacks, `2` iterators, `3` cancel tokens, and `4` byte runs +(returned runs and `{prefix}_alloc` runs not yet freed or adopted). Kind `-1` +returns `1` when the producer counts at all and `0` when it doesn't, so a +harness can tell "nothing is live" from "nothing is counted". A Rust producer +counts only with the `weaveffi` crate's `leak-check` feature; without it +every kind returns `0`. + ## Thread safety Every `#[weaveffi::interface]` type is `Send + Sync` (the macro asserts it), @@ -245,6 +349,9 @@ should document it. - **Not clearing the error.** The message and payload leak, and a stale non-zero `code` confuses the next check. Start every slot at `{0}`. - **`error_clear` on an async error.** It leaks the box; use `error_free`. -- **Holding a lock across a callback call.** With a plain-return callback - method, a consumer failure unwinds through the producer and poisons a - `std::sync::Mutex`; snapshot state, release the lock, then call. +- **Freeing a callback's returned run.** A string, bytes, or buffer a + callback method returns belongs to the producer once it's written to the + out slots; allocate it with `{prefix}_alloc` and don't free it. +- **Holding a lock across a callback call.** The consumer's implementation + may call back into the producer, which then waits on the lock you hold; + snapshot state, release the lock, then call. diff --git a/docs/src/guides/extract.md b/docs/src/guides/extract.md index fbc80fd9..5d2233b1 100644 --- a/docs/src/guides/extract.md +++ b/docs/src/guides/extract.md @@ -1,46 +1,172 @@ -# Extracting an IDL from Rust +# Library Mode -The CLI reads annotated Rust directly: `weaveffi generate src/lib.rs` lowers -the source to the IR in memory with the same extractor the -`#[weaveffi::module]` macro uses, so the bindings and the compiled symbols -come from one parse. `weaveffi extract` writes that IR out as an IDL -document, which is useful for reviewing the API surface, committing a -canonical IDL next to the source, or handing the API to a non-Rust -implementation. +The CLI never reads Rust source. A Rust producer's API is read from the +library the crate builds: `#[weaveffi::module]` embeds a description of every +declaration it exports in the compiled library, and `weaveffi generate`, +`diff`, `validate`, `extract`, `dev`, `build`, and `package` read it back +out. The bindings therefore describe exactly what the library was compiled +with, `#[cfg]` and type aliases resolved, macro expansion and all. + +A Rust project's input is the crate: its directory (`[project] input = "."`, +which `weaveffi init` writes) or its `Cargo.toml`. + +```bash +weaveffi generate # build the crate (debug), read it, generate +weaveffi generate --release # read a release build instead +weaveffi generate --library target/x86_64-unknown-linux-gnu/release/libkvstore.so +``` + +Without `--library`, the CLI resolves the crate with `cargo metadata` (its +name, version, library name, and target directory), runs +`cargo build --lib --message-format=json-render-diagnostics` (Cargo's errors +and warnings print as usual), and reads the crate's `cdylib`, or its +`staticlib` when it has no `cdylib`. `--library` skips the build and reads +the given file: a shared library, a static archive, or a `.wasm` module, for +any platform (an Android `.so` reads fine on a Mac). With no crate at all (no +input argument and no `[project] input`), `--library` reads the library on +its own: the prefix comes from its metadata and the library name from its +file name, and a file holding the metadata of several WeaveFFI crates is an +error that asks for the crate. + +## Extracting an IDL + +`weaveffi extract` prints the API the library embeds as an IDL document, +which is useful for reviewing the API surface, committing a canonical IDL +next to the crate, or handing the API to a non-Rust implementation. ```bash -weaveffi extract src/lib.rs # YAML to stdout -weaveffi extract src/lib.rs -o api.yml # YAML to a file -weaveffi extract src/lib.rs -f json -o api.json # JSON (or -f toml) +weaveffi extract # the project's crate, YAML to stdout +weaveffi extract path/to/crate -o api.yml +weaveffi extract -f json -o api.json # JSON (or -f toml) +weaveffi extract --library libkvstore.dylib ``` | Flag | Default | Meaning | |------|---------|---------| | `-o`, `--output` | stdout | Write to a file | | `-f`, `--format` | `yaml` | `yaml`, `json`, or `toml` | -| `--lenient` | off | Emit the IDL even if it doesn't validate | +| `--library` | build the crate | Read this library instead | +| `--release` | off | Build the crate in release mode | +| `--config` | discovered | The `weaveffi.toml` to use | -The extracted document is validated, and extraction fails with the -validator's diagnostics if the result wouldn't generate (a duplicate name, an -unknown type, a callback method returning a string). `--lenient` prints those -diagnostics as a warning and emits the IDL anyway, which helps when the -source references types declared somewhere the extractor can't see. Rust -syntax with no ABI representation (a raw pointer, a `Box`, a tuple) is a hard -error even with `--lenient`. +The document is validated before it's printed, so what `extract` prints +always generates. An IDL input is an error: there's nothing to extract. The +macro validates each module tree as it compiles; the CLI's validation adds +the rules that span trees (names unique across the whole crate, C symbols +that collide between trees). The extracted IDL carries no package identity. When you generate from it, the identity comes from `[package]` in `weaveffi.toml` (IDL rules), not from `Cargo.toml`, so set `name`, `c_prefix`, and `library` to match the crate if the generated bindings must load the Rust library. -## What the extractor reads +## How the API is embedded + +For every declaration (a module, its error domain, a function, an interface +and each of its members, a record, an enum, a callback interface) the macro +emits an exported static: + +```rust,ignore +#[unsafe(no_mangle)] +#[cfg_attr(target_family = "wasm", unsafe(link_section = "weaveffi_meta"))] +#[used] +pub static KVSTORE_META_3A5F0C2E9B1D4467: [u8; 141] = *b"\x89\x00\x00\x00{\"schema\":...}"; +``` + +The bytes are a frame: a little-endian `u32` length, then compact JSON that +records the declaration (in the IDL's own shape), the path of its module, +its position among its siblings, the crate's prefix, and the schema version. +The name is the crate's uppercase prefix, `_META_`, and 16 uppercase hex +digits of the FNV-1a 64 hash of the declaration's dotted path (followed by a +`.` for a module, so a module and a declaration of the same name differ), so +it never collides with another declaration or another crate. + +The CLI reads native libraries with the `object` crate, walking the symbol +tables (and a PE export table) for `_META_` symbols and reading each one's +bytes from the section that holds it. A `wasm32` module keeps its data out +of reach of any symbol table, so there the frames go to the `weaveffi_meta` +custom section, which the linker fills with every frame back to back. The +CLI keeps the frames carrying the crate's own prefix (a dependency that is +itself a WeaveFFI producer embeds frames too) and assembles them, in +declaration order, into the API. Separate top-level modules are separate +macro invocations, so they're ordered by name. + +A static carries its declaration's `#[cfg]`, and sits inside the module it +describes, so the module's `#[cfg]` applies too. A declaration a build +compiles out is missing from that build's metadata exactly as it's missing +from its symbols and its contract table: + +```rust +#[weaveffi::module] +pub mod shop { + #[cfg(feature = "extra")] + #[weaveffi::export] + pub fn discount() -> u32 { 10 } +} +``` -Only inline modules marked `#[weaveffi::module]` in the input file are read, -recursively. Inside them it reads the items carrying the markers listed in -[the producer macro guide](producer-macro.md#the-attributes), the `pub fn`s of -each interface's inherent `impl` block, doc comments, and -`#[deprecated(note = "...")]`. Markers match by their last path segment, so -`#[weaveffi::record]` and a bare `#[record]` are equivalent. +`weaveffi extract` lists `discount` only for a build with the `extra` +feature (`cargo build --features extra`, then `--library`). + +A library with no frames for the crate (the API isn't annotated, or the +crate builds no `cdylib` or `staticlib`) is an error naming what's missing: +`#[weaveffi::module]` on the API and one `weaveffi::export_runtime!()` call. + +## Iterating: `weaveffi dev` + +`weaveffi dev` builds the crate's debug library, generates, and makes the +generated packages find that library without any environment variables +where a package looks for a bundled copy (when the crate builds a shared +library; a static-only build gets the environment variable instead): + +- **Python**: the library is copied into the package directory, which the + loader checks first. +- **Node.js**: the library is copied next to `binding.gyp`, which links the + addon against it and records its directory as an rpath, so `npm install` + in the package builds an addon that loads it. +- **Ruby**: the library is copied to `lib/native/`, which the loader checks + first. + +For every other target, `dev` prints what to set: `{PREFIX}_LIBRARY=` +for the targets that load the library at run time (.NET, Dart, Kotlin), and +the library directory to link with and put on the loader path for the +targets that link at build time (C, C++, Swift, Go). WebAssembly needs a +`wasm32` build, which `dev` doesn't make (`weaveffi build --platforms +wasm32` does). Rerun `weaveffi dev` after changing the crate: copies don't +follow later builds. + +## Generating from a build script + +The CLI is also a library. `weaveffi_cli::project::Project` locates a +project the way the `weaveffi` command does and generates it, so a crate +that ships bindings can generate them from its `build.rs`: + +```rust,ignore +// build.rs +fn main() -> miette::Result<()> { + println!("cargo::rerun-if-changed=weaveffi.toml"); + println!("cargo::rerun-if-changed=api.yml"); + weaveffi_cli::project::Project::discover(env!("CARGO_MANIFEST_DIR"))?.generate()?; + Ok(()) +} +``` + +```toml +[build-dependencies] +weaveffi-cli = "0.24" +miette = "7" +``` + +`Project::discover` finds the nearest `weaveffi.toml` at or above the +directory, and `generate` writes the `[project] targets` into `[project] +out`, rewriting only changed files. + +A build script runs before its crate is compiled, so it can't read its own +crate's library: from a `build.rs`, the project's input must be an IDL, or +the API must come from a library built earlier, named with +`Project::library(path)`. Pointing a crate's own `build.rs` at the crate is +an error that says so. For a Rust producer, generate after the build +instead: `weaveffi generate` in CI or a `cargo xtask`. ## Type mapping @@ -55,30 +181,32 @@ each interface's inherent `impl` block, doc comments, and | `weaveffi::Iter` | `iter` | | `&T`, `Arc` (interface), `Arc` | `T` | | `Arc` (callback interface) | `Trait` | -| `weaveffi::CancelToken` | removed; sets `cancellable: true` | +| `weaveffi::CancelToken` | removed (`#[weaveffi::cancellable]` sets `cancellable: true`) | | `Result` | return `T`, sets `throws: true` | +| a type alias declared in the tree | its target | | any other name | that name, resolved by the validator | Compositions map recursively: `Option>` is `[i32]?` and -`Vec>` is `[Gadget]`. An `async fn` sets `async: true`. A -cross-module reference is emitted with its owning module's path -(`kv.Store`) once the API validates. +`Vec>` is `[Gadget]`. An `async fn` sets `async: true`, `()` and +`Result<(), E>` returns are no return, and a callback method's +`Result` is a `T` return that throws only with +`#[weaveffi::throws]`. A type is always referenced by its bare name +(`Store`, wherever it's declared), because type names are global; a path +such as `super::kv::Store` contributes only its last segment. ## Limits -The extractor parses syntax; it doesn't compile or expand anything. - -- **One file.** Every annotated module must be inline (`mod kv { ... }`) in - the input file. `mod kv;` declarations pointing at other files are skipped. -- **No macro expansion or generics.** Items produced by macros are invisible, - and only a `#[weaveffi::interface]` type's inherent `impl` block is read. -- **Error docs.** A `#[weaveffi::error]` variant's doc comment becomes the - code's `message:`, so an IDL code's separate `doc:` has no Rust spelling. +- **Error messages.** A `#[weaveffi::error]` variant's doc comment becomes + the code's `doc:`, and its first line the code's `message:` (the variant + name when it has no doc), so a Rust producer can't give a code a message + that differs from its doc's first line. - **Deprecation versions.** `#[deprecated(since = "...")]` keeps only the note; the IDL has no `since` field. -- **Parameter docs.** Rust accepts `///` on parameters, but formatters tend - to strip them; plan for `Param.doc` to be lossy. +- **Parameter docs.** Rust doesn't allow doc comments on function + parameters, so an extracted API has no `Param.doc`; describe parameters in + the function's own doc comment. -`crates/weaveffi-cli/tests/cli/extract_roundtrip.rs` checks that an annotated -form of the kitchen-sink fixture extracts to the same IR as the YAML, modulo -the error-doc gap above. +`crates/weaveffi-cli/tests/cli/extract.rs` builds the fixture producer in +`crates/weaveffi-cli/tests/fixtures/producer`, checks that `weaveffi +extract` prints exactly its `expected.yml`, and checks that generating from +the library and from that IDL gives identical output. diff --git a/docs/src/guides/packaging.md b/docs/src/guides/packaging.md index 07cd98c6..63b05c2f 100644 --- a/docs/src/guides/packaging.md +++ b/docs/src/guides/packaging.md @@ -1,113 +1,171 @@ # Packaging -`weaveffi generate` emits binding source. `weaveffi package` goes one step -further: it assembles publishable packages that bundle a prebuilt native -library for each platform, laid out the way each ecosystem resolves native -code, so `pip install`, `npm install`, `dotnet add package`, and friends work -without a local toolchain. +`weaveffi generate` emits binding source. Shipping it takes two more steps: +`weaveffi build` cross-compiles the Rust producer once per platform, and +`weaveffi package` turns those builds into the artifacts each ecosystem +installs: wheels, npm tarballs, gems, a SwiftPM package with its +`XCFramework`, a NuGet package, and so on. Consumers install them with +`pip`, `npm`, `gem`, SwiftPM, or `dotnet` and need no compiler. ```bash -weaveffi package --binaries prebuilt --target python,node,dotnet -o dist +weaveffi build --platforms darwin-arm64,ios-arm64,ios-sim-arm64 +weaveffi package --target python,node,swift ``` -Like `generate`, it reads `[project]` and `[package]` from `weaveffi.toml` -(or takes an explicit input), and every package it writes loads the library -named by the identity's `library` field. - -## Where the libraries come from - -Pass exactly one source: - -- `--binaries `: prebuilt libraries laid out as - `//`. This is the CI path: build each - platform on its own runner, collect the results, package once. -- `--build `: cross-compile a Cargo package as a `cdylib` for each - requested platform. Each platform needs its rustup target and a working - linker (the Android targets need the NDK's linker); a missing target fails - with the `rustup target add` command to run. +Both read `[project]`, `[package]`, and `[build]` from `weaveffi.toml` (or +take an explicit input), and every artifact loads the library named by the +identity's `library` field. + +## Build + +`weaveffi build` finds the producer crate with `cargo metadata` (the +project's crate, `[build] manifest`, or `--manifest-path`) and runs +`cargo rustc --lib` once per platform with the artifact kinds that platform +needs, so the crate's own `crate-type` doesn't matter. A Rust producer's API +is then read from the first library it built (see [Library +Mode](extract.md)), so nothing is compiled twice: + +| Platform | What `build` produces | +|----------|------------------------| +| macOS | the `cdylib`, with install name `@rpath/lib{library}.dylib`, plus the `staticlib` an `XCFramework` slice is made from | +| iOS | the `staticlib` | +| Linux, Android | the `cdylib`, with soname `lib{library}.so` (Android also 16 KB page-aligned) | +| Windows | the DLL and its import library | +| `wasm32` | the `.wasm` module, linked with `--export-table --growable-table` | + +It sets `MACOSX_DEPLOYMENT_TARGET` and `IPHONEOS_DEPLOYMENT_TARGET` from +`[build]`, and for Android it finds the NDK (`ANDROID_NDK_HOME`, +`ANDROID_NDK_ROOT`, or the newest `ndk/` of the Android SDK) and +points Cargo's linker and the `cc` crate at the NDK's clang for the +configured API level. Everything lands in the Cargo target directory: ```text -prebuilt/ - darwin-arm64/libkvstore.dylib - linux-x64/libkvstore.so - windows-x64/kvstore.dll - android-arm64/libkvstore.so - wasm32/kvstore.wasm +target/weaveffi/ + darwin-arm64/ libkvstore.dylib libkvstore.a kvstore_node.node libkvstore_jni.dylib + ios-arm64/ libkvstore.a + android-arm64/ libkvstore.so libkvstore_jni.so + wasm32/ kvstore.wasm ``` +The Node.js and Kotlin targets reach the C ABI through C glue of their own +(an N-API addon and a JNI shim). When those targets are selected (`--target`, +else `[project] targets`, else all), `build` compiles the glue next to the +library so packages ship it prebuilt: + +- **Node.js addon**, against the running Node.js's headers, for each + requested desktop platform the host can compile: both macOS architectures + on a macOS host, and the host's own architecture on a Linux host. Windows + isn't prebuilt (it links `node.lib`); the package compiles it at install + time. +- **JNI shim** for every Android ABI (with the NDK) and for the same desktop + platforms as the addon (with the JDK headers from `JAVA_HOME`); never for + `windows-x64`, where the generated CMake project builds it. + +Glue a host can't compile is skipped with a warning, and the package falls +back to compiling it. Every platform is checked before anything compiles, so +a missing rustup target (`rustup target add ...`), a missing NDK, or a +platform this host can't build (Apple platforms need macOS, Windows needs +Windows) fails at once with every problem listed. + +## Package + +`weaveffi package` runs `build` for the requested platforms, then writes +each target's artifacts to the dist directory (`-o`, else `[package] dist`, +else `dist`). With `--binaries `, it skips the build and reads a +directory laid out like `target/weaveffi/` instead; that's how CI hands +libraries built on separate runners to one packaging job, and how a +producer that isn't a Rust crate is packaged. A Rust producer's API is read +from the first library in the directory. + +| Target | Artifacts | Publish with | +|--------|-----------|--------------| +| `python` | one `py3-none-{platform}.whl` per desktop platform, the library inside the import package | `twine upload dist/python/*.whl` | +| `node` | `{name}-{os}-{cpu}-{version}.tgz` per desktop platform (library and prebuilt addon, gated by npm `os`/`cpu`) plus `{name}-{version}.tgz`, which lists them in `optionalDependencies` | `npm publish ` for each, platform packages first | +| `wasm` | `{name}-{version}.tgz` with the `.wasm` module | `npm publish ` | +| `ruby` | one `{gem}-{version}-{platform}.gem` per desktop platform | `gem push ` | +| `swift` | `C{Module}.xcframework.zip` (static libraries for every Apple platform built), its `.sha256`, and the SwiftPM package `swift/{Module}/` whose binary target points at the archive's URL and checksum | upload the archive, then commit and tag the package | +| `dotnet` | the project `dotnet/{Namespace}/` with `runtimes//native/` libraries, and the `.nupkg` that `dotnet pack` builds from it | `dotnet nuget push dist/dotnet/*.nupkg` | +| `kotlin` | the Gradle project `kotlin/{name}/` with prebuilt `jniLibs//` (library and shim) and desktop natives under `resources/natives//` | `gradle publish` | +| `dart` | the pub package `dart/{package}/` with desktop libraries under `native//` | `dart pub publish` | +| `go` | the module `go/{package}/` with libraries under `lib//` and cgo flags that link the right one | push the module to its repository | +| `c`, `cpp` | `{library}-{version}-c.tar.gz` and `-cpp.tar.gz`: headers, libraries under `lib//`, and a `CMakeLists.txt` exposing the host's library as an imported target | attach to a release | + +Wheel tags carry the build's macOS deployment target +(`macosx_11_0_arm64`) and, on Linux, the newest glibc the library actually +links against (`manylinux_2_17_x86_64` or later), so `pip` only installs a +wheel where the library loads. Each archive is deterministic (fixed +timestamps and order), so the same builds always produce the same bytes and +checksums. + +A target skips platforms its ecosystem has no slot for (a wheel has no +Android tag, NuGet no `wasm32` runtime), and a target with nothing to ship is +named in a note. + ## Platforms -`--platforms` takes a comma-separated list of platform ids. With `--build`, -**the default is the host platform only**, so a local run packages what you -just built; CI passes the full list. With `--binaries`, the default is every -platform directory present. +`--platforms` takes a comma-separated list of platform ids; without it, +`build` and `package` use `[build] platforms`, else the host. With +`--binaries`, the default is every platform directory present. | Platform id | Rust target | Shipped by | |-------------|-------------|------------| -| `darwin-arm64` | `aarch64-apple-darwin` | every desktop target | -| `darwin-x64` | `x86_64-apple-darwin` | every desktop target | +| `darwin-arm64` | `aarch64-apple-darwin` | every desktop target, and `swift` | +| `darwin-x64` | `x86_64-apple-darwin` | every desktop target, and `swift` | | `linux-x64` | `x86_64-unknown-linux-gnu` | every desktop target | | `linux-arm64` | `aarch64-unknown-linux-gnu` | every desktop target | | `windows-x64` | `x86_64-pc-windows-msvc` | every desktop target | -| `ios-arm64` | `aarch64-apple-ios` | `swift` (static library) | -| `ios-sim-arm64` | `aarch64-apple-ios-sim` | `swift` (static library) | -| `ios-sim-x64` | `x86_64-apple-ios` | `swift` (static library) | +| `ios-arm64` | `aarch64-apple-ios` | `swift` | +| `ios-sim-arm64` | `aarch64-apple-ios-sim` | `swift` | +| `ios-sim-x64` | `x86_64-apple-ios` | `swift` | | `android-arm64` | `aarch64-linux-android` | `kotlin` (and `c`) | | `android-x64` | `x86_64-linux-android` | `kotlin` (and `c`) | | `wasm32` | `wasm32-unknown-unknown` | `wasm` (and `c`) | -A requested platform with no library is skipped with a warning, and a target -skips platforms its ecosystem has no slot for (a wheel has no wasm tag, NuGet -has no Android runtime). The command prints one line per target it packaged -and a note naming any target that produced nothing. +## Swift and the XCFramework -## What each target produces +`package` fuses the Apple static libraries into `C{Module}.xcframework` +(iOS device, iOS simulator, and macOS slices, with a group's architectures +fused by `lipo`), zips it, and writes its SHA-256 checksum. The packaged +`Package.swift` declares the C module as +`.binaryTarget(url:checksum:)`, so apps link the library with no flags. +The URL comes from `[generators.swift] xcframework_url`; until you set it, +it's a placeholder under `example.invalid`: -| Target | Package layout | -|--------|----------------| -| `c`, `cpp` | Headers plus `lib//` libraries and a `CMakeLists.txt` that picks the host's library | -| `swift` | A SwiftPM package with the desktop libraries under `lib//`; when an iOS slice is included, `C{Module}.xcframework` (iOS device, iOS simulator, and macOS slices, fused with `lipo`) beside `Package.swift`, which then uses it as a binary target | -| `kotlin` | A Gradle module with Android libraries under `jniLibs//` and desktop libraries as classpath resources | -| `node` | A main npm package plus one `optionalDependencies` package per platform, each gated by `os` and `cpu` | -| `wasm` | An npm package with the ES module, its `.d.ts`, and the `.wasm` binary | -| `python` | One platform-wheel tree per platform with the library inside the import package | -| `dotnet` | A NuGet-ready project with libraries under `runtimes//native/` | -| `dart` | A pub package with desktop libraries under `native//` | -| `go` | A Go module with per-platform libraries and a cgo preamble that links the right one | -| `ruby` | One precompiled platform gem per platform | +```toml +[generators.swift] +xcframework_url = "https://github.com/acme/kvstore/releases/download/{version}/{file}" +``` -Each language page documents its layout in detail. Bundled libraries are -found before the system loader, and `{PREFIX}_LIBRARY` still overrides them. +To publish, upload the archive to that URL and commit `swift/{Module}/` to +the repository consumers depend on. For local use, unzip the archive next to +the packaged `Package.swift`; the manifest prefers a local +`C{Module}.xcframework` over the URL. This step runs `xcodebuild`, so it +needs macOS. -## Not automated yet +## What still needs the ecosystem's tools - **Publishing.** There's no `weaveffi publish`; run each ecosystem's own - publish command on the output. -- **Building wheels and gems.** The Python and Ruby trees are ready to build - (`python -m build --wheel`, `gem build`), but the CLI doesn't run those - tools or retag wheels. -- **The Node addon.** The N-API addon is compiled at install time - (`node-gyp`), so consumers need a C compiler; only the producer library is - prebuilt. -- **Flutter mobile.** Flutter's native-assets build isn't generated. - -## iOS - -iOS slices are static libraries, so a Rust producer that ships to iOS -declares `crate-type = ["cdylib", "staticlib"]`. The XCFramework's macOS -slice is the `lib{library}.a` Cargo writes next to the macOS dylib; macOS-only -packages keep linking the dylib. Assembly runs `lipo` and -`xcodebuild -create-xcframework`, so it needs macOS: - -```bash -rustup target add aarch64-apple-ios aarch64-apple-ios-sim -weaveffi package --target swift --build kvstore \ - --platforms ios-arm64,ios-sim-arm64,darwin-arm64 -o dist -``` + upload command on the artifacts. +- **NuGet.** `package` runs `dotnet pack`, so the .NET SDK must be + installed; without it the run fails after writing everything else. +- **Kotlin.** The output is a Gradle project, not an `.aar` or `.jar`: + building one needs the Kotlin compiler, so run `gradle assembleRelease` + (Android) or `gradle jar` (JVM) on it. With prebuilt shims present, Gradle + skips the CMake build and needs no NDK. +- **Node.js on Windows.** The addon isn't prebuilt for `windows-x64`, so + installing there compiles it with node-gyp. +- **Cross-architecture Linux.** Building `linux-arm64` on an x64 host needs a + cross linker (`CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER`), and its glue + isn't prebuilt; use an arm64 runner instead. +- **Dart.** The pub package's loader looks for `native//` inside + the package itself (resolved through its `package:` URI), then relative to + the working directory, so a compiled app that no longer has the package + sources copies those libraries next to it. Flutter's native-assets build + isn't generated. ## CI recipe -Build each platform natively, upload each library under its platform id, then +Build each platform on a runner that can, upload `target/weaveffi/`, then package once: ```yaml @@ -116,42 +174,42 @@ jobs: strategy: matrix: include: - - { platform: darwin-arm64, runner: macos-latest, target: aarch64-apple-darwin, lib: libkvstore.dylib } - - { platform: linux-x64, runner: ubuntu-latest, target: x86_64-unknown-linux-gnu, lib: libkvstore.so } - - { platform: linux-arm64, runner: ubuntu-24.04-arm, target: aarch64-unknown-linux-gnu, lib: libkvstore.so } - - { platform: windows-x64, runner: windows-latest, target: x86_64-pc-windows-msvc, lib: kvstore.dll } - - { platform: wasm32, runner: ubuntu-latest, target: wasm32-unknown-unknown, lib: kvstore.wasm } + - runner: macos-latest + platforms: darwin-arm64,darwin-x64,ios-arm64,ios-sim-arm64 + targets: aarch64-apple-darwin,x86_64-apple-darwin,aarch64-apple-ios,aarch64-apple-ios-sim + - runner: ubuntu-latest + platforms: linux-x64,android-arm64,android-x64,wasm32 + targets: aarch64-linux-android,x86_64-linux-android,wasm32-unknown-unknown + - { runner: ubuntu-24.04-arm, platforms: linux-arm64, targets: "" } + - { runner: windows-latest, platforms: windows-x64, targets: "" } runs-on: ${{ matrix.runner }} steps: - uses: actions/checkout@v4 - uses: dtolnay/rust-toolchain@stable with: - targets: ${{ matrix.target }} - - run: cargo build --release --target ${{ matrix.target }} - - run: | - mkdir -p "prebuilt/${{ matrix.platform }}" - cp "target/${{ matrix.target }}/release/${{ matrix.lib }}" "prebuilt/${{ matrix.platform }}/" - shell: bash + targets: ${{ matrix.targets }} + - uses: actions/setup-node@v4 # for the prebuilt Node.js addon + - run: cargo install weaveffi-cli + - run: weaveffi build --platforms ${{ matrix.platforms }} - uses: actions/upload-artifact@v4 with: - name: prebuilt-${{ matrix.platform }} - path: prebuilt/ + name: weaveffi-${{ matrix.runner }} + path: target/weaveffi/ package: needs: build - runs-on: ubuntu-latest + runs-on: macos-latest # the XCFramework needs xcodebuild steps: - uses: actions/checkout@v4 - uses: actions/download-artifact@v4 with: - pattern: prebuilt-* + pattern: weaveffi-* path: prebuilt merge-multiple: true - run: cargo install weaveffi-cli - - run: > - weaveffi package --binaries prebuilt -o dist - --platforms darwin-arm64,linux-x64,linux-arm64,windows-x64,wasm32 + - run: weaveffi package --binaries prebuilt ``` -A platform you can't build can be dropped from the matrix; every target -packages whatever subset is present. +The Linux runner finds the NDK through `ANDROID_NDK_HOME` (or the SDK +preinstalled on GitHub's images). A platform you can't build can be dropped +from the matrix; every target packages whatever subset is present. diff --git a/docs/src/guides/producer-macro.md b/docs/src/guides/producer-macro.md index 1ad5ecb4..ad5aa671 100644 --- a/docs/src/guides/producer-macro.md +++ b/docs/src/guides/producer-macro.md @@ -4,9 +4,10 @@ A Rust producer is an ordinary library crate. You annotate the modules you want to export with `#[weaveffi::module]`, tag the items inside, and call `weaveffi::export_runtime!()` once. The macro emits the `extern "C"` thunks, marshalling every argument through the audited `weaveffi::abi` runtime, so -the crate contains no `unsafe` glue. The CLI reads the same source -(`weaveffi generate src/lib.rs`) through the same extractor, so the compiled -symbols and the generated bindings can't drift. +the crate contains no `unsafe` glue. The macro also embeds a description of +every exported declaration in the library, and the CLI reads the API from +there (`weaveffi generate` in the crate builds it first), so the generated +bindings describe exactly what the library was compiled with. ## Setup @@ -15,24 +16,38 @@ symbols and the generated bindings can't drift. crate-type = ["cdylib"] # add "staticlib" for iOS, "rlib" for in-crate tests [dependencies] -weaveffi = "0.23" +weaveffi = "0.24" ``` **The prefix is the crate name.** Every C symbol starts with the crate's library name (`[lib] name`, else the package name with `-` mapped to `_`), which the macro reads from `CARGO_CRATE_NAME` at expansion time. The CLI -derives the same prefix from `Cargo.toml`, and it isn't configurable: setting -`[package] c_prefix` or `library` in `weaveffi.toml` for a `.rs` input is an -error. A crate named `kvstore` with a module `kv` exports -`kvstore_kv_Store_open`, `kvstore_error_clear`, `kvstore_kv_checksum`, and so +derives the same prefix from `cargo metadata`, and it isn't configurable: +setting `[package] c_prefix` or `library` in `weaveffi.toml` for a Rust +producer is an error. A crate named `kvstore` with a module `kv` exports +`kvstore_kv_Store_open`, `kvstore_error_clear`, `kvstore_kv_contract`, and so on, and builds `libkvstore.so`. **`export_runtime!()` exactly once,** at the crate root. It takes no arguments and exports the [runtime surface](../reference/abi.md#runtime-surface) -under the crate's prefix: `abi_version`, the error helpers, `free_bytes`, the -cancel-token functions, `debug_live`, and on `wasm32` `alloc` and `dealloc`. -Each top-level `#[weaveffi::module]` additionally exports its contract -checksum, `{prefix}_{module}_checksum()`. +under the crate's prefix: `abi_version`, the error helpers (`error_set`, +`error_set_payload`, `error_clear`, `error_free`), `alloc`, `free_bytes`, the +cancel-token functions, and `debug_live`. + +**Each top-level module exports its contract table,** +`{prefix}_{module}_contract()`: one `{id, hash}` entry per declaration in the +module tree, which generated bindings check at load time (see +[load-time checks](../reference/abi.md#load-time-checks)). The macro computes +the table with the same function the CLI uses for the bindings +(`weaveffi_model::contract::entries`), so a library and bindings generated +from it always agree, and adding a declaration never breaks bindings +generated before it. + +**Each declaration is described in the library,** as an exported static +named `{PREFIX}_META_{HASH}` holding a small JSON frame (on `wasm32`, in the +`weaveffi_meta` custom section). `weaveffi generate`, `diff`, `validate`, +`extract`, and every other command that reads a Rust producer's API read +these frames out of the built library; see [Library Mode](extract.md). ## The attributes @@ -43,7 +58,8 @@ checksum, `{prefix}_{module}_checksum()`. | `#[weaveffi::record]` | struct with named fields | A by-value record, serialized as a value buffer. | | `#[weaveffi::enumeration]` | `#[repr(i32)]` enum, or enum with named-field variants | A C-style enum (explicit `= N` on every variant) or a rich enum. | | `#[weaveffi::interface]` | struct plus its inherent `impl` | A reference-counted object type; the `impl`'s `pub fn`s become constructors, methods, and statics. | -| `#[weaveffi::callback_interface]` | `trait Name: Send + Sync` | Methods the consumer implements; accepted as `Arc`. | +| `#[weaveffi::callback_interface]` | `trait Name: Send + Sync` | Methods the consumer implements, each returning `Result`; accepted as `Arc` or `Option>`. | +| `#[weaveffi::throws]` | callback-interface method | The consumer may fail with a code of the error domain in scope. | | `#[weaveffi::error]` | enum with explicit discriminants | The module's error domain. Must implement `Display`. | | `#[weaveffi::cancellable]` | exported `async fn` or async method | Takes a `weaveffi::CancelToken` as its last parameter. | @@ -51,18 +67,24 @@ Only tagged items are exported; private helpers, `use` items, and state are left alone. Doc comments flow into the IDL and every binding, and `#[deprecated(note = "...")]` becomes the IDL's `deprecated:` text. +The macro validates the module tree with the same validator the CLI applies +to an IDL (duplicate names, C symbol collisions, a `Result` with no error +domain in scope, iterators out of place, and so on), and each error is a +compile error on the offending item, member, or type. + ## Types | Rust | IDL | Crosses the ABI as | |------|-----|--------------------| | `i8`..`i64`, `u8`..`u64`, `f32`, `f64`, `bool` | same | one value | +| `usize`, `isize`, `u128`, `i128`, `char` | none (rejected) | use `u64`, `i64`, or `String` | | `String`, `&str` | `string` | UTF-8 `(ptr, len)` | | `Vec`, `&[u8]` | `bytes` | `(ptr, len)` | | `#[weaveffi::record]` struct, rich enum | the type | value buffer | | `#[repr(i32)]` enum | the enum | `int32_t` | -| `Option`, `Vec`, `HashMap`, `BTreeMap` | `T?`, `[T]`, `{K:V}` | value buffer | -| `&T`, `Arc`, `Option>` (interface `T`) | `T`, `T?` | object pointer | -| `Arc` (callback interface) | `Trait` | `ctx` plus vtable (parameters only) | +| `Option`, `Vec` or `&[T]`, `HashMap`, `BTreeMap` | `T?`, `[T]`, `{K:V}` | value buffer | +| `&T`, `Arc`, `Option>` (interface `T`); `Self` or `T` as a return | `T`, `T?` | object pointer | +| `Arc`, `Option>` (callback interface) | `Trait`, `Trait?` | `ctx` plus vtable (parameters only) | | `weaveffi::Iter` | `iter` | iterator handle (returns only) | | `weaveffi::CancelToken` | none | the launcher's cancel-token slot | @@ -70,7 +92,7 @@ A reference (`&str`, `&[u8]`, `&Contact`, `&Store`) is a calling convention, not an IDL distinction: the thunk lends the lifted argument for the call. Objects compose with every buffered shape, so `Vec>`, `Option>`, an `Arc` record field, and `weaveffi::Iter>` all -work. [Extracting an IDL from Rust](extract.md#type-mapping) has the full +work. [Library Mode](extract.md#type-mapping) has the full mapping. ## Functions and errors @@ -124,9 +146,10 @@ pub enum QuotaError { A module declares at most one domain; it's in scope for that module and every module nested in it, and a `Result` with no domain in scope is a compile -error. `Result` compiles too (it reports the generic code `-1`), -and any type implementing `weaveffi::ErrorReport` can be an error type. A -panic in producer code is caught at the boundary and reported as code `-2`. +error, whatever its error type. With a domain in scope, `Result` +compiles too (it reports the generic code `-1`), and any type implementing +`weaveffi::ErrorReport` can be an error type. A panic in producer code is +caught at the boundary and reported as code `-2`. See [Errors and Memory](errors-and-memory.md). ## Records and enums @@ -160,9 +183,10 @@ pub enum Shape { ``` Records and rich enums get a generated `weaveffi::abi::BufferValue` -implementation; no per-type C symbols exist. Rich-enum variants use named -fields (tuple variants are rejected), and tags follow declaration order unless -the variant declares a discriminant. +implementation, and a C-style enum implements `weaveffi::abi::CEnum`; none +of them exports a C function. Rich-enum variants use named fields (tuple +variants are rejected), and tags follow declaration order unless the variant +declares a discriminant. ## Interfaces @@ -199,20 +223,25 @@ impl Shelf { /// Static: no receiver, returns something other than `Self`. pub fn capacity() -> i64 { 10_000 } + + /// Async static factory: an `async fn` returning `Self` is a static. + pub async fn load(name: String) -> Result, LibraryError> { /* ... */ } } ``` -- Constructors return `Self`, `Arc`, or the type by name, optionally in - a `Result`. They can't be `async`; use an async static that returns - `Arc`. +- Constructors are synchronous associated functions that return `Self`, + `Arc`, or the type by name, optionally in a `Result`. +- An `async fn` associated function that returns the type is an async + static factory, since constructors can't be async. Bindings call it like + any other async static (`await Shelf.load("x")`). - Methods take `&self` or `self: Arc`. `&mut self` and `self` by value are rejected. - An object parameter `&T` is borrowed for the call; `Arc` (or `Option>`) is a new strong reference the thunk took for you, so you - may store it. + may store it. A by-value `T` parameter is rejected. - An object you return transfers one strong reference to the consumer. -The macro also emits `{prefix}_{module}_{Type}_clone` and `_destroy`. Every +The macro also emits `{prefix}_{path}_{Type}_clone` and `_destroy`. Every binding wraps the reference in a class that releases it exactly once. ## Callback interfaces @@ -220,25 +249,34 @@ binding wraps the reference in a class that releases it exactly once. A callback interface is the inverse of an interface: the consumer implements it and the producer calls it, from any thread, for as long as it holds the `Arc`. When the last clone drops, the consumer's release hook runs exactly -once. +once (on whichever thread drops it). + +Every method returns `Result`, because the +consumer's implementation can fail and the `Err` is how its failure reaches +you; `T` maps to the IDL return. These are two of the `kvstore` sample's +callback interfaces: ```rust +use std::sync::Arc; +use weaveffi::ForeignError; + +/// Retained by `Store::subscribe` and told about every change. #[weaveffi::callback_interface] -pub trait Subscriber: Send + Sync { - /// A consumer failure comes back as an `Err`. - fn route(&self, topic: &str) -> Result; - /// A consumer failure aborts the enclosing call. - fn on_message(&self, message: &Message) -> i64; - /// The consumer adopts the reference it receives. - fn on_attached(&self, bus: Arc); +pub trait Listener: Send + Sync { + /// A direct return. + fn accepts(&self, key: &str) -> Result; + /// No return; the change (a rich enum) is lent for the call. + fn on_change(&self, change: &Change) -> Result<(), ForeignError>; } -#[weaveffi::export] -pub fn route_once(subscriber: Arc, topic: &str) -> Delivery { - subscriber.route(topic).unwrap_or_else(|e| { - weaveffi::abi::raise_foreign_error(e); - Delivery::Skip - }) +/// Consulted by every `put` while installed. +#[weaveffi::callback_interface] +pub trait Policy: Send + Sync { + /// A record return. The consumer may fail with a `KvError` code. + #[weaveffi::throws] + fn admit(&self, entry: &Entry) -> Result; + /// The consumer adopts `home` and returns a reference you adopt. + fn route(&self, key: &str, home: Arc) -> Result, ForeignError>; } ``` @@ -247,29 +285,91 @@ The rules: - The receiver is `&self`. Methods are synchronous. - Parameters may be any IDL type except a callback interface or an iterator. Strings, bytes, and buffered values are lent to the consumer for the call; - an object parameter transfers one strong reference to the consumer. -- The return is `()`, a scalar, `bool`, a C-style enum, or - `Result` of one of those. Strings, buffers, and - objects can't be returned yet (see the [roadmap](../roadmap.md)). -- A callback interface may appear only as a parameter, never in a return, - record, `Option`, or collection. - -**When the consumer fails.** The binding reports a consumer exception -through the vtable entry's error slot with code `-4`. A method declared with -`Result` receives it as `Err(ForeignError { code, message })` -and nothing unwinds; propagate it with `weaveffi::abi::raise_foreign_error` -or handle it. A method with a plain return can't hand the error back, so the -failure aborts the producer call: on an unwinding build it unwinds to the -enclosing thunk, which reports `-4` with the consumer's message, and on a -`panic = "abort"` build (notably `wasm32-unknown-unknown`) the method returns -its type's zero value, your code keeps running, and the thunk reports the -failure when it returns. Either way the original caller sees the consumer's -message. A failure on a thread with no WeaveFFI call active (a thread you -spawned) is written to stderr and dropped, never attached to a later call. - -Prefer the `Result` form. With plain returns, don't hold a `Mutex` guard -across a callback call: snapshot the state you need, release the lock, then -call out, as `samples/events` does. + an object parameter transfers one strong reference to the consumer, so + it's spelled `Arc` (or `Option>`), never `&T`. +- The return is `Result`, where `T` is `()` or any IDL type + except an iterator or a callback interface: a direct value, a `String`, a + `Vec`, a record, rich enum, `Option`, `Vec`, or map, or an object + (`Arc`, or `Option>` when the consumer may return none). A plain + `T` is a compile error. The consumer allocates a returned string, bytes, + or buffer with `{prefix}_alloc`, and the runtime adopts it for you. +- A callback interface may appear only as a top-level parameter of a + function or an interface member, either bare (`Arc`) or + optional (`Option>`, where `None` is a null vtable), never + in a return, a record, a collection, or another callback. + +**When the consumer fails,** the method returns +`Err(ForeignError { code, message, payload })`. Nothing unwinds and nothing +is deferred; the failure is a value you handle or propagate. `code` is `-4` +for any consumer failure, except that a method marked +`#[weaveffi::throws]` keeps a code of the error domain in scope, and it's +`-3` when the consumer returned a value the runtime couldn't accept (text +that isn't UTF-8, a null object, an undeclared enum value, a malformed +buffer). + +To propagate the failure, return an error type that reports a +`ForeignError`'s own code, message, and payload, and use `?`. That's +`ForeignError` itself (its `ErrorReport` implementation does exactly this), +or your own enum implementing `weaveffi::ErrorReport` that wraps either a +domain error or a `ForeignError`, as the `kvstore` sample's `StoreError` does +for `Store::put`. The function is `throws`, so its module needs an error +domain in scope. The original caller sees the consumer's code, message, and +payload, so a typed domain error from a `throws` method reaches it typed. + +**Typed callback errors.** A method marked `#[weaveffi::throws]` is `throws` +in the IDL, which needs an error domain in scope for its module. Its +consumer may fail with one of the domain's codes, fields included (but no +object fields), and `ForeignError::domain::()` decodes that code back +into your `#[weaveffi::error]` enum, returning `None` for any other failure: + +```rust +#[weaveffi::error] +#[derive(Debug)] +#[repr(i32)] +pub enum LookupError { + /// No such key. + Missing { key: String } = 1, + /// The source is busy. + Busy = 2, +} + +#[weaveffi::callback_interface] +pub trait Source: Send + Sync { + /// Look up a key, failing with a `LookupError`. + #[weaveffi::throws] + fn lookup(&self, key: &str) -> Result; +} + +/// Describe the outcome of a lookup without failing. +#[weaveffi::export] +pub fn lookup_via(source: Arc, key: String) -> String { + match source.lookup(&key) { + Ok(v) => format!("ok {v}"), + Err(e) => match e.domain::() { + Some(LookupError::Missing { key }) => format!("missing {key}"), + Some(LookupError::Busy) => "busy".to_string(), + None => format!("foreign {}: {}", e.code, e.message), + }, + } +} +``` + +(`LookupError` needs a `Display` impl, omitted here.) + +**Optional callbacks.** Accept `Option>` when the consumer may +pass none: + +```rust +#[weaveffi::export] +pub fn has_source(source: Option>) -> bool { + source.is_some() +} +``` + +Don't hold a `Mutex` guard across a callback call: the consumer's +implementation may call back into your library and wait on the same lock. +Snapshot the state you need, release the lock, then call out, as +`samples/kvstore` does. ## Iterators @@ -291,7 +391,7 @@ returns immediately and a completion callback that fires exactly once. #[weaveffi::cancellable] pub async fn wait(timeout_ms: i64, cancel: weaveffi::CancelToken) -> i64 { let _ = cancel; // the runtime drops this future when the token fires - Delay::new(timeout_ms as u64).await; + Delay::new(timeout_ms as u64).await; // any future the executor can drive timeout_ms } ``` @@ -299,78 +399,145 @@ pub async fn wait(timeout_ms: i64, cancel: weaveffi::CancelToken) -> i64 { When the consumer cancels, the runtime drops your future at its next suspension point and completes the call with code `-5`; you don't need to poll the token. Poll `cancel.is_cancelled()` only for cooperative cleanup, -such as work running on another thread. Futures run on the executor -installed with `weaveffi::set_spawner` (a thread per future by default), and -a future the executor drops without finishing also completes with `-5`. See -[Async and Cancellation](async.md). +such as work running on another thread. Futures run on a small pool of +worker threads by default, on Tokio with the `weaveffi` crate's `tokio` +feature, or on the executor installed with `weaveffi::set_spawner`; a future +the executor drops without finishing also completes with `-5`. See +[Async and Cancellation](async.md#choosing-an-executor). ## Modules and cross-module references Each `#[weaveffi::module]` at the top level of the crate is a root; nested `#[weaveffi::module]` modules are its submodules, and their symbols carry the -joined path (`contacts_contacts_groups_count_of_type`). Inside one root tree, +joined path (`kvstore_kv_stats_summarize`). Inside one root tree, any module may use any declaration in the tree: records, enums, interfaces, callback interfaces, and the error domains of its ancestors. +Names are global across every root: type, free-function, and error-code +names must each be unique in the whole crate's API (see +[Naming](../reference/naming.md#global-idl-names)), so two modules can't both +export a function `open`. `weaveffi generate` reports a clash with the +module paths of both declarations. + The macro expands each root on its own and can't see a sibling root's -declarations. A named type it can't resolve is assumed to be a value buffer, -and the macro asserts at compile time that the type implements -`weaveffi::abi::ByValue`. Records and rich enums do, so sharing a record -between roots works. A C-style enum, an interface, or a callback interface -from another root does not, and the build fails: +declarations, so it validates a tree assuming that a name declared outside +it is a record or rich enum (a value buffer), and asserts that at compile +time. Records and rich enums pass, so sharing a record between roots works. A +C-style enum, an interface, or a callback interface from another root +doesn't, and the build fails on the use: ```text -error[E0277]: `Color` is declared in a different `#[weaveffi::module]` tree and isn't a record or rich enum +error[E0277]: `Color` is not a WeaveFFI record or rich enum; declare C-style enums and interfaces in the module tree that uses them + = note: a `#[weaveffi::module]` only sees the declarations inside its own tree, so it can pass a type from another tree only as a value buffer (a record or rich enum) = note: nest the modules under one `#[weaveffi::module]` root (as inner `mod`s) so the macro can see the declaration ``` -The fix is in the message: put the modules under one root. The `contacts` -sample shows both shapes: a nested `contacts::groups` that uses the parent's -interface and enum, and a sibling `directory` root that shares only the -`Contact` record. +The fix is in the message: put the modules under one root. The `kvstore` +sample shows both shapes: a nested `kv::stats` that uses the parent's +interface and enum and reports the parent's error domain, and a sibling +`report` root that shares only the `Entry` record. + +## Conditional compilation and type aliases + +A `#[cfg]` on an exported item (a function, an `impl` block, a record, +enum, error domain, interface struct, callback trait, or nested +`#[weaveffi::module]`) applies to everything the macro generates for it: its +thunks and its contract table entries. An item the build compiles out has +no symbols, no entry, and no metadata, so the library never claims a +declaration it doesn't have. To make a single interface member conditional, move it into +its own `impl` block and put the `#[cfg]` on that block: + +```rust +#[weaveffi::interface] +pub struct Store { /* ... */ } + +impl Store { + pub fn new() -> Self { /* ... */ } +} + +#[cfg(feature = "admin")] +impl Store { + pub fn purge(&self) { /* ... */ } +} +``` + +Because `weaveffi generate` reads the API from the built library, the +bindings follow the build's `#[cfg]`: generate from a build without the +`admin` feature and `purge` isn't in the bindings. Loading bindings +generated from a build that had an item against a build that compiles it out +fails the contract check with the item's name ("is missing from the +library"); a build with extra items loads fine. + +A `#[cfg]` on a member (a record field, an enum variant, an error code, a +method inside an `impl` block, a callback trait method) is a compile error, +because the generated bindings can't follow it, and so is an out-of-line +submodule (`mod x;`), whose body the macro can't read. + +A non-generic type alias declared in the module tree is substituted wherever +it's used, so the IDL and the bindings see the target type: + +```rust +#[weaveffi::module] +pub mod tree { + pub type Id = u64; + pub type Ids = Vec; + + #[weaveffi::export] + pub fn first(ids: Ids) -> Option { + ids.first().copied() + } +} +``` ## Leak checks The `leak-check` cargo feature (on `weaveffi`) counts live objects, foreign callbacks, iterators, cancel tokens, and returned allocations. `{prefix}_debug_live(kind)` reports them (`0` objects, `1` callbacks, -`2` iterators, `3` tokens, `4` allocations); without the feature the symbol -still exists and returns `0`. Every sample enables it, and every conformance -consumer asserts all five counts are zero at exit. +`2` iterators, `3` tokens, `4` byte runs), and kind `-1` returns `1` to say +the library counts. Without the feature the symbol still exists and returns +`0` for every kind. Every sample enables it, and every conformance consumer +asserts all five counts are zero at exit. ```toml [dependencies] -weaveffi = { version = "0.23", features = ["leak-check"] } +weaveffi = { version = "0.24", features = ["leak-check"] } ``` ## What the macro rejects -Each rejection is a spanned compile error, pinned by a `trybuild` test in -`crates/weaveffi-macros/tests/ui/`: +Each rejection is a spanned compile error, most of them pinned by a +`trybuild` test in `crates/weaveffi-macros/tests/ui/`: | Source | Why | |--------|-----| | `*const T`, `*mut T` | Declare the pointee as an interface and pass `&T` or `Arc`. | +| `usize`, `isize`, `u128`, `i128`, `char` | Use `u64` or `i64` (the same width everywhere), or `String`. | +| interface parameter by value (`T`) | Take `&T` to borrow it or `Arc` to keep it. | | `Box`, `Rc`, `Box` | Objects and callbacks are shared: use `Arc`, `Arc`. | | `&mut T` parameter | Take `&T` or a value and return the result. | | interface method on `self` or `&mut self` | Use `&self` or `self: Arc` with interior mutability. | | interface that isn't `Send + Sync` | The compiler names the offending field. | -| callback method on `&mut self`, or returning a string, buffer, or object | Callback returns are direct values only. | +| callback method on `&mut self` | Take `&self`; the producer may call it from any thread. | +| callback method returning a plain `T` | Return `Result`. | | `Result` with no error domain in scope | Declare a `#[weaveffi::error]` enum in the module or an ancestor. | | `#[weaveffi::error]` enum without `Display` | `Display` supplies the runtime message. | | C-style enum without `#[repr(i32)]` | The discriminant crosses as `int32_t`. | | `Iter` parameter or nested `Iter` | Iterators are outermost returns only. | | non-value type from another root | See [cross-module references](#modules-and-cross-module-references). | +| `#[cfg]` on a field, variant, error code, method, or callback method | Put it on the whole item (an `impl` block for a method). | +| out-of-line submodule (`mod x;`) | The macro reads the tree from its tokens; write the module inline. | Also rejected: tuple-style rich-enum variants, error variants without an -explicit discriminant, two `#[weaveffi::error]` enums in one module, and an -`Arc` naming more than one trait. Whole-API rules (duplicate -names, [C symbol collisions](../reference/idl.md#validation)) are checked by -the CLI, not the macro, so run `weaveffi validate` or `weaveffi generate` in -CI. +explicit discriminant, two `#[weaveffi::error]` enums in one module, an +`Arc` naming more than one trait, and anything else the validator +rejects in the tree. The macro validates one root tree at a time, so the +rules that span roots (names unique across the whole crate's API, and +[C symbol collisions](../reference/idl.md#validation) between roots) are +checked by the CLI; run `weaveffi validate` or `weaveffi generate` in CI. ## See also - [Samples](../samples.md): `kvstore` uses every feature on this page. - [C ABI Contract](../reference/abi.md): what the thunks implement. -- [Rust API Map](../api/rust.md): the `weaveffi` and `weaveffi-abi` items. +- [Rust API Map](../api/rust.md): the `weaveffi` and `weaveffi::abi` items. diff --git a/docs/src/intro.md b/docs/src/intro.md index f094e84e..31c6a0c8 100644 --- a/docs/src/intro.md +++ b/docs/src/intro.md @@ -10,11 +10,12 @@ Go, and Ruby that all call the same C functions. ## The design in one page **One definition, two front doors.** A Rust producer annotates ordinary -modules; the `#[weaveffi::module]` macro emits the `extern "C"` thunks, and -the CLI reads the same source to emit the bindings. Both run one shared -extractor, so the library you compile and the bindings you ship are two views -of one parse. A non-Rust producer writes an [IDL](reference/idl.md) instead -and implements the C header WeaveFFI generates from it. +modules; the `#[weaveffi::module]` macro emits the `extern "C"` thunks and +embeds a description of the API in the compiled library, and the CLI reads +that description back out to emit the bindings, so the bindings you ship +describe exactly the library you compiled, `#[cfg]` included. A non-Rust +producer writes an [IDL](reference/idl.md) instead and implements the C +header WeaveFFI generates from it. **Identity drives every name.** A library has one resolved [identity](reference/naming.md): a package `name`, a C symbol `prefix`, and a @@ -23,15 +24,16 @@ native `library` base name. For a Rust producer all three come from the crate or NuGet package named `kvstore`). Nothing a generator emits is named after WeaveFFI, so any number of WeaveFFI-built libraries can live in one process. -**A small, explicit C ABI.** [Revision 3](reference/abi.md) has five value +**A small, explicit C ABI.** [Revision 4](reference/abi.md) has five value families: direct scalars, UTF-8 strings and bytes as `(ptr, len)` runs, serialized [value buffers](reference/value-buffers.md) for records, enums, optionals, lists, and maps, reference-counted objects, and consumer-implemented callback vtables. Every fallible call reports through a `{prefix}_error` struct. Async functions complete through a callback that fires exactly once, and cancellation is a first-class runtime code. Every -consumer checks the ABI revision and a per-module contract checksum when it -loads the library, so a stale binding fails loudly instead of misreading +consumer checks the ABI revision and each module's contract table (a hash of +every declaration's signature) when it loads the library, so a stale binding +fails loudly, naming the declaration that changed, instead of misreading memory. **Idiomatic generators.** Each target maps the model onto its own idioms: @@ -56,5 +58,5 @@ bundles prebuilt native libraries into publishable packages per ecosystem. boundary. - [Comparison](comparison.md) and [FAQ](faq.md): how WeaveFFI relates to UniFFI, Diplomat, cbindgen, and the single-language bridges. -- [Stability and Versioning](stability.md): what changed in schema 0.10 and - ABI 3, and how to migrate. +- [Stability and Versioning](stability.md): what the version numbers + promise, and how to migrate to schema 0.11 and ABI revision 4. diff --git a/docs/src/reference/abi.md b/docs/src/reference/abi.md index 1f6439b8..8970d50b 100644 --- a/docs/src/reference/abi.md +++ b/docs/src/reference/abi.md @@ -1,6 +1,6 @@ # C ABI Contract -This page is the normative description of the WeaveFFI C ABI, **revision 3**. +This page is the normative description of the WeaveFFI C ABI, **revision 4**. The `#[weaveffi::module]` macro implements it on the producer side, the C generator renders it as a header, and every other generator consumes it. Where another page disagrees with this one, this one wins. @@ -23,39 +23,87 @@ dynamically. ## Load-time checks -The revision is the constant `{P}_ABI_VERSION` in the header and the result -of `{p}_abi_version()`. It changes only when the runtime surface, the -value-buffer encoding, or the object, callback, async, or iterator -conventions change incompatibly; it's independent of the crate version and -the IDL schema version. - -Every top-level module `m` also has a **contract checksum**: the producer -exports `uint64_t {p}_{m}_checksum(void)`, and the header defines -`#define {P}_{M}_CHECKSUM 0x...ull` (`M` is the module name uppercased). The -value is a 64-bit FNV-1a hash over a canonical walk of the module and its -submodules: every declaration name, every type reference, parameter and -field order, declaration order, the `throws`, `async`, and `cancellable` -flags, enum values, and error-code values. Documentation, deprecation -messages, and error messages are excluded, so prose edits never break a -deployed binding. The macro and the CLI compute it with the same function -(`weaveffi_model::checksum::module_checksum`) over the same IR. - -A generated consumer checks `{p}_abi_version()` and every top-level -module's checksum once, when it loads the library, and raises the language's -load or import error naming the mismatched module. A C or C++ producer that -implements an IDL by hand returns the header's constant: +A consumer makes two checks once, when it loads the library, before any +other call, and refuses to load if either fails (with the language's load or +import error, or a trap where loading can't throw, as in Swift and Go). + +**The revision.** `{P}_ABI_VERSION` in the header and `{p}_abi_version()` in +the library must be equal. The revision changes only when the runtime +surface, the value-buffer encoding, or the object, callback, async, or +iterator conventions change incompatibly; it's independent of the crate +version and the IDL schema version. Revision 4 replaced revision 3's +per-module checksums with the per-declaration contract tables below, gave +callback vtables their size-checked header and every return family (with +`throws` and optional callback parameters), exported `{p}_alloc` on every +target (removing `{p}_dealloc`), and added `{p}_error_set_payload`. + +**The contract.** Every top-level module `m` has a contract table: one entry +per declaration in `m` and its submodules. ```c -uint64_t kv_kv_checksum(void) { return KV_KV_CHECKSUM; } +typedef struct {p}_contract_entry { uint64_t id; uint64_t hash; } {p}_contract_entry; +const {p}_contract_entry* {p}_{m}_contract(size_t* out_len); /* static, sorted by id */ ``` +A declaration is a function, an interface, each interface member +(constructor, method, or static), a record, an enum, a callback interface, +or an error domain. Its entry is: + +- **`id`**: the 64-bit FNV-1a hash of its dotted path, the module path then + the name (`kv.open_store`, `kv.Store`, `kv.Store.get`, `kv.KvError`, + `kv.stats.Stats`); +- **`hash`**: the 64-bit FNV-1a hash of its canonical signature string. + +The canonical signature is computed from the resolved model, never from +source spelling: type aliases are substituted, `[u8]` is `bytes`, and each +type is written in its IDL spelling (`i32`, `string`, `[Contact]`, +`{string:i64}`, `Store?`, `iter`). Documentation, deprecation text, +and error messages are excluded, and so is the order of sibling +declarations, because each entry stands alone. + +| Declaration | Canonical signature | +|---|---| +| function or interface member | `{kind} {name}({param}: {type}, ...) -> {type}`, where `kind` is `function`, `constructor`, `method`, or `static` and a missing return is `void`, followed by ` throws`, ` async`, and ` cancellable` in that order when set (a constructor's return is its interface) | +| record | `record {name} {{field}: {type}, ...}` | +| enum | `enum {name} {{Variant} = {value}, ...}`, a variant with fields followed by ` {{field}: {type}, ...}` | +| callback interface | `callback {name} {{method}({param}: {type}, ...) -> {type}; ...}`, a method that throws followed by ` throws` | +| error domain | `errors {name} {{Code} = {value}, ...}`, a code with fields followed by ` {{field}: {type}, ...}` | +| interface | `interface {name}` (each member has its own entry) | + +For example, the `kvstore` sample's `fn get(&self, key: String) -> +Result` on interface `Store` in module `kv` has the id +`fnv1a64("kv.Store.get")` (`0x3bf340d3781ddb6d`) and the hash +`fnv1a64("method get(key: string) -> Entry throws")` +(`0xd86b757fadf495cb`). FNV-1a 64 starts from `0xcbf29ce484222325` and, for +each byte, XORs the byte in and multiplies by `0x100000001b3`, wrapping. + +The header defines the entries the bindings were generated with, and a +checker: + +```c +#define {P}_{M}_CONTRACT_LEN n +#define {P}_{M}_CONTRACT { {0x...ull, 0x...ull}, /* kv.Store.get */ ... } +static inline uint64_t {p}_{m}_contract_check(void); /* first failing id, or 0 */ +``` + +A consumer passes when **every entry it was generated with** is present in +the producer's table with an equal hash. Entries the producer has and the +consumer doesn't are fine, so adding a declaration (a function, a method, a +record) never breaks a deployed binding. A missing entry fails with +"`kv.Store.get` is missing from the library"; a different hash fails with +"`kv.Store.get` changed since these bindings were generated". `{M}` is the module name +uppercased. The macro and the CLI compute the table with the same function +(`weaveffi_model::contract::entries`) over the same model, and a Rust +producer's table lists only the declarations its build compiles in (an +item's `#[cfg]` applies to its entry). + ## Runtime surface Every producer exports these symbols (`weaveffi::export_runtime!()` emits them for a Rust producer): ```c -#define {P}_ABI_VERSION 3u +#define {P}_ABI_VERSION 4u uint32_t {p}_abi_version(void); typedef struct {p}_error { @@ -64,11 +112,13 @@ typedef struct {p}_error { const uint8_t* payload_ptr; /* the code's fields as a value buffer, or NULL */ size_t payload_len; } {p}_error; -void {p}_error_set({p}_error* err, int32_t code, const char* message); /* copies message */ +void {p}_error_set({p}_error* err, int32_t code, const char* message); /* copies */ +void {p}_error_set_payload({p}_error* err, const uint8_t* ptr, size_t len); /* copies */ void {p}_error_clear({p}_error* err); /* frees message and payload; code = 0 */ void {p}_error_free({p}_error* err); /* clear, then free a heap-boxed error */ -void {p}_free_bytes(uint8_t* ptr, size_t len); /* strings, bytes, buffers */ +uint8_t* {p}_alloc(size_t len); /* a zero-filled run, NULL for 0 */ +void {p}_free_bytes(uint8_t* ptr, size_t len); /* returned runs and {p}_alloc runs */ typedef struct {p}_cancel_token {p}_cancel_token; {p}_cancel_token* {p}_cancel_token_create(void); /* refcount 1 */ @@ -76,43 +126,64 @@ void {p}_cancel_token_cancel({p}_cancel_token* token); /* idempotent */ bool {p}_cancel_token_is_cancelled(const {p}_cancel_token* token); void {p}_cancel_token_destroy({p}_cancel_token* token); /* drops one reference */ -uint64_t {p}_debug_live(int32_t kind); /* leak counters; 0 when not counted */ +uint64_t {p}_debug_live(int32_t kind); /* leak counters */ -/* wasm32 producers only */ -uint8_t* {p}_alloc(uint32_t size); -void {p}_dealloc(uint8_t* ptr, uint32_t size); +typedef struct {p}_contract_entry { uint64_t id; uint64_t hash; } {p}_contract_entry; ``` -Every function accepts null where a pointer is optional (`error_clear`, -`error_free`, `free_bytes`, the token functions, `_clone`, `_destroy`) and -treats it as a no-op; a null token reads as never cancelled. +Every function accepts null where a pointer is optional (`error_set`, +`error_set_payload`, `error_clear`, `error_free`, `free_bytes`, the token +functions, `_clone`, `_destroy`) and treats it as a no-op; a null token reads +as never cancelled. + +`{p}_alloc` and `{p}_free_bytes` manage **byte runs**. Every string, bytes, +or value buffer the producer returns is a run the consumer releases with +`{p}_free_bytes(ptr, len)`, passing the exact length it received. A consumer +allocates a run with `{p}_alloc(len)` when it hands bytes *to* the producer +(a callback method's string, bytes, or buffer return, which the producer +adopts) or, on `wasm32`, when it stages an argument in the module's linear +memory (released with `{p}_free_bytes`). `{p}_alloc(0)` returns null, and +null with length `0` is the empty run everywhere; `{p}_free_bytes(ptr, 0)` +is a no-op. `{p}_debug_live(kind)` reports live resources for leak checks: `0` objects, -`1` foreign callbacks, `2` iterators, `3` cancel tokens, `4` returned byte -allocations. A Rust producer counts them only with the `leak-check` feature; -otherwise, and in a producer that doesn't count, it returns `0`. +`1` foreign callbacks, `2` iterators, `3` cancel tokens, `4` byte runs +(returned and allocated ones not yet freed or adopted). Kind `-1` returns `1` +when the producer counts at all and `0` when it doesn't, so a harness can +tell "nothing is live" from "nothing is counted"; an unknown kind returns +`0`. A Rust producer counts only with the `leak-check` feature; without it +every kind but `-1` returns `0`. ## Errors -Every synchronous symbol except `_clone`, `_destroy`, and an iterator's -`_destroy` takes a trailing `{p}_error* out_err` after all inputs and -out-parameters. The caller owns the struct and zero-initializes it; the -producer writes it only on failure, and then returns a zero sentinel (`0`, -`false`, `NULL`) the caller must not use. The caller releases a failure's -message and payload with `{p}_error_clear`. +Every synchronous API symbol (a function or interface member, an iterator +launcher, and an iterator's `_next`) takes a trailing `{p}_error* out_err` +after all inputs and out-parameters; `_clone`, `_destroy`, an iterator's +`_destroy`, async launchers, and the runtime surface don't. The caller owns +the struct and zero-initializes it. On failure the producer fills it and +returns a zero sentinel (`0`, `false`, `NULL`) the caller must not use; on +success it clears it (releasing anything an earlier failure left there), so +one struct can serve many calls. The caller releases a failure's message and +payload with `{p}_error_clear`. | Code | Meaning | |------|---------| | `0` | success; every pointer is null | | `> 0` | a declared code of the callable's error domain; `payload_ptr`/`payload_len` hold the code's fields, if any | -| `-1` | generic: an untyped producer error | +| `-1` | generic: an untyped producer error, or an async call the executor couldn't start | | `-2` | the producer panicked | -| `-3` | marshalling failure: an argument couldn't be lifted (null with a non-zero length, invalid UTF-8, an out-of-range enum value, a malformed buffer) | +| `-3` | marshalling failure: an argument couldn't be lifted (null with a non-zero length, a null `self`, object, or vtable where one is required, invalid UTF-8, an out-of-range enum value, a malformed buffer, a map with a repeated key, a callback vtable smaller than the producer's, an iterator advanced concurrently) | | `-4` | a consumer's callback-interface implementation failed | | `-5` | cancelled (async completions only) | +A producer lifts **every** argument of a call before it reports the first +one that failed, so the inputs it adopts (a callback context, the object +tokens in a value buffer) are adopted and released even when another +argument, or a null `self`, fails. + A non-zero code on a callable without `throws` is a producer bug or a failed -callback; consumers trap on it rather than report a domain error. +callback; consumers trap on it rather than report a domain error (see +[Errors and memory](../guides/errors-and-memory.md)). ## Families and slots @@ -120,12 +191,12 @@ Every resolved type belongs to one family, which decides how it crosses: | Family | IDL types | Parameter slots | Return | |--------|-----------|-----------------|--------| -| Direct | integers, `f32`, `f64`, `bool`, C-style enums | one by value (`bool` is C `bool`; enums are `int32_t`) | by value | +| Direct | integers, `f32`, `f64`, `bool`, C-style enums | one by value (`bool` is C `bool`; an enum is its header type `{p}_{path}_{E}`, an `int`-sized value that crosses as `int32_t`) | by value | | String | `string` | `const uint8_t* {n}_ptr, size_t {n}_len`: UTF-8, not NUL-terminated, borrowed | `const uint8_t*` plus trailing `size_t* out_len` | | Bytes | `bytes` | same as String | same as String | -| Buffer | structs, rich enums, `T?` (except `Interface?`), `[T]`, `{K:V}` | `const uint8_t* {n}_ptr, size_t {n}_len` holding a [value buffer](value-buffers.md), borrowed | same as String | +| Buffer | structs, rich enums, `T?` (except `Interface?` and `Cb?`), `[T]`, `{K:V}` | `const uint8_t* {n}_ptr, size_t {n}_len` holding a [value buffer](value-buffers.md), borrowed | same as String | | Object | interfaces, `Interface?` | `const {p}_{path}_{I}* {n}`, borrowed; null means none | `{p}_{path}_{I}*`, one strong reference; null means none | -| Callback | callback interfaces | `void* {n}_ctx, const {p}_{path}_{Cb}_vtable* {n}_vtable` | not allowed | +| Callback | callback interfaces, `Cb?` | `void* {n}_ctx, const {p}_{path}_{Cb}_vtable* {n}_vtable`; for `Cb?` a null vtable means none | not allowed | | Iterator | `iter` | not allowed | `{Iter}*` (see [Iterators](#iterators)) | A parameter `ptr` may be null when its `len` is `0`. A String, Bytes, or @@ -136,7 +207,7 @@ with length `0` is the empty value. Out-parameters for a return precede ## Symbol names -All names are computed once by the binding model and recorded in one symbol +All names are computed once by the model and recorded in one symbol table; validation fails with `SymbolCollision` if two declarations would produce the same identifier. @@ -147,18 +218,30 @@ produce the same identifier. | async function `f` | launcher `{p}_{path}_f` (no suffix); completion type `{p}_{path}_f_callback` | | iterator returned by `f` | type `{p}_{path}_{Owner_}{PascalF}Iterator`, plus `{Iter}_next` and `{Iter}_destroy` | | C-style enum `E`, variant `V` | type `{p}_{path}_E`, constant `{p}_{path}_E_V` | -| error domain `D`, code `C` | type `{p}_{path}_D`, constant `{p}_{path}_D_C` | +| rich enum `E`, variant `V` | tag type `{p}_{path}_E_Tag`, constant `{p}_{path}_E_V`; struct `{p}_{path}_E` and its codecs in the buffer header | +| record `S` | struct `{p}_{path}_S` and its codecs in the buffer header | +| error domain `D`, code `C` | type `{p}_{path}_D`, constant `{p}_{path}_D_C`; for a code with fields, struct `{p}_{path}_D_C_payload` and its codecs in the buffer header | | callback interface `Cb` | vtable type `{p}_{path}_Cb_vtable` | -| top-level module `m` | `{p}_{m}_checksum`, `{P}_{M}_CHECKSUM` | -| runtime | the names in [Runtime surface](#runtime-surface) (reserved) | +| top-level module `m` | `{p}_{m}_contract`, `{p}_{m}_contract_check`, `{P}_{M}_CONTRACT`, `{P}_{M}_CONTRACT_LEN` | +| runtime | the names in [Runtime surface](#runtime-surface), plus the buffer header's `{p}_str`, `{p}_bytes`, `{p}_reader`, and `{p}_writer` (reserved) | `{Owner_}` is empty for a free function and the interface name plus `_` for a member, and the function name is converted to PascalCase: `stream_items` in `kitchen` is `kitchen_sink_kitchen_StreamItemsIterator`, and -`Store.list_keys` in `kv` is `kvstore_kv_Store_ListKeysIterator`. Records and -rich enums produce no C identifiers (they're buffers), but their names are -still reserved in the module's type namespace. IDL spellings are never -re-cased in C. +`Store.keys` in `kv` is `kvstore_kv_Store_KeysIterator`. Records and rich +enums export no functions (they cross as buffers), but the C generator's +value-buffer helper header (`{library}_buffer.h`) declares a struct for each, +and for each error code with fields, plus four codecs per struct `T` +(`T_write`, `T_read`, `T_decode`, and `T_free`), so those names are reserved +like the rest. That header also +reserves the identifier families `{p}_str_*`, `{p}_bytes_*`, `{p}_reader_*`, +`{p}_writer_*`, `{p}_list_*`, `{p}_map_*`, and `{p}_opt_*`: a declaration +whose symbol falls in one (anything in a top-level module named `list`, say) +is a `SymbolCollision`. IDL spellings are never re-cased in C. + +The header's macros are `{P}_ABI_VERSION`, the per-module contract macros, +`{P}_API` (symbol visibility; define `{P}_BUILD` when building the producer +on Windows), `{P}_DEPRECATED(msg)`, and the include guard `{P}_H`. ## Objects @@ -173,6 +256,7 @@ references and use them from any thread. | parameter | borrowed for the call; the producer takes its own reference if it retains the object | | return, async result, iterator element | one strong reference transfers to the consumer | | callback method parameter | one strong reference transfers to the consumer (the slot is `{p}_{path}_{I}*`, not `const`) | +| callback method return | one strong reference transfers to the producer | | object token inside a value buffer | carries one strong reference; the reader adopts it | A consumer encoding an object into a buffer writes a fresh reference from @@ -183,39 +267,60 @@ wrapper while a call is pending is safe. ```c typedef struct {p}_{path}_{Cb}_vtable { - (*{method})(void* ctx, , {p}_error* out_err); /* declaration order */ - void (*free)(void* ctx); /* always last */ + uint32_t size; /* sizeof the vtable, as the consumer compiled it */ + uint32_t flags; /* reserved; 0 */ + void (*free)(void* ctx); /* always first, at a fixed offset */ + /* one entry per method, in declaration order: */ + (*{method})(void* ctx, , , {p}_error* out_err); } {p}_{path}_{Cb}_vtable; ``` A callback interface has no exported symbols; it lowers to a vtable type. The consumer passes a `ctx` it owns and a pointer to a vtable that outlives -the producer's use of it (in practice, one static vtable per interface). The -producer may call any entry, any number of times, from any thread, until it -calls `free(ctx)` exactly once, after which it never touches `ctx` again. - -Method parameters are borrowed for the duration of the call (strings, bytes, +the producer's use of it (in practice, one static vtable per interface), +with `size` set to `sizeof` the vtable and `flags` to `0`. The producer +rejects a vtable whose `size` is smaller than the vtable it was built with, +failing the call with `-3` (after calling `free(ctx)`, when the header itself +is complete); a larger `size` is accepted, so a consumer generated from a +newer contract may carry methods this producer doesn't call. For an optional +callback parameter (`Cb?`) a null vtable pointer means none. + +The producer may call any entry, any number of times, from any thread, until +it calls `free(ctx)` exactly once, after which it never touches `ctx` again. +`free` may also run on any producer thread. + +**Parameters** are borrowed for the duration of the call (strings, bytes, and buffers as `ptr`/`len`), except objects, which transfer one strong -reference. Returns are `void` or Direct, so no consumer allocation crosses -back; methods are never async or throwing. - -A consumer implementation that fails reports it by calling -`{p}_error_set(out_err, -4, message)` with a borrowed message, which the -producer copies with its own allocator; it never writes `message` itself. The -producer treats any non-zero code as a failure and reports a positive one as -`-4`, so a consumer bug can't masquerade as a domain error. It abandons the -operation in progress, and the original caller observes `-4` with the -consumer's message (through `out_err` or the async completion). - -A Rust producer sees the failure according to the trait method's return -type. A method returning `Result` gets -`Err(ForeignError { code, message })` and nothing unwinds. A method returning -a plain `T` unwinds to the enclosing thunk on a `panic = "unwind"` build; on -a `panic = "abort"` build (notably `wasm32-unknown-unknown`) it returns the -type's zero value, the producer continues, and the thunk reports the recorded -failure in place of its result. Deferral applies only while a WeaveFFI thunk -is running on the current thread; a failure on any other thread is written to -stderr and dropped, never attached to a later call. +reference the consumer adopts. + +**Returns** of any family but iterators and callback interfaces: + +- A Direct value is the C return (``). +- An object (`I` or `I?`) is the C return, `{p}_{path}_{I}*`: one strong + reference (a fresh `_clone`) the producer adopts. `I?` may return null; + `I` must not. +- A string, bytes, or buffer return makes `` `void` and adds two + trailing slots before `out_err`: `uint8_t** out_ptr, size_t* out_len`. + The consumer allocates the run with `{p}_alloc`, writes it there, and the + producer adopts and frees it. Null with length `0` is the empty value. + +The producer adopts whatever the return and out slots hold whether or not +the method failed, so a consumer that fails after allocating doesn't leak. A +return it can't accept (a string that isn't UTF-8, a null `I`, an +out-of-range enum value, a malformed buffer) fails the method with `-3`. + +**Errors.** A consumer implementation that fails reports it with +`{p}_error_set(out_err, code, message)`, which copies a borrowed message with +the producer's allocator; it never writes `message` itself. A method declared +`throws` may report a positive code of the error domain in scope for its +module, attaching the code's fields encoded as a value buffer with +`{p}_error_set_payload` (they may not include objects). Any other non-zero +code, a code the domain doesn't declare or whose payload doesn't decode, or +a positive code on a method without `throws` reaches the producer as `-4`, +so a consumer bug can't masquerade as a domain error. The producer decides +what a failure means for the call in progress: a Rust producer's method +returns `Result`, and propagating the `Err` reports its +code, message, and payload to the original caller. ## Async functions @@ -236,14 +341,22 @@ The runtime guarantees: - The callback fires **exactly once**, from any thread, possibly before the launcher returns. -- If the executor drops the future before it completes, the completion fires - with `-5` from the drop guard. +- The launcher takes everything it needs before returning (it copies + strings, bytes, and buffers and retains objects and `self`), so the + consumer may release its arguments as soon as it returns. +- If the executor can't take the call (it failed to start, or a custom + executor panicked), the completion fires with `-1` and a message. +- If the executor drops the future before it completes, the completion + fires with `-5` from the drop guard. - For a `cancellable` function, the producer takes its own reference on the token at launch; the consumer may cancel and destroy its reference at any time afterward. Cancelling wakes the future; the runtime then drops it and completes with `-5` and the message `cancelled`, unless it already completed. Producer code may still poll the token for cooperative cleanup. +Which threads drive the futures is the producer's business; see +[Async](../guides/async.md) for the Rust runtime's executors. + ## Iterators ```c @@ -254,10 +367,11 @@ void {Iter}_destroy({Iter}* iter); Item out slots are `T* out_item` for a Direct or object element and `const uint8_t** out_item, size_t* out_len` for a String, Bytes, or Buffer -element. Each element follows the return rules of its family. `_next` is -internally synchronized, so concurrent calls are safe (though pointless). -The consumer calls `_destroy` exactly once, after exhaustion or to abandon -the iteration. +element. Each element follows the return rules of its family. `_next` never +blocks on another `_next`: a call that arrives while the same iterator is +being advanced (from another thread, or re-entrantly from inside the +producer's own `next`) fails with `-3`. The consumer calls `_destroy` exactly +once, after exhaustion or to abandon the iteration. ## Value buffers @@ -270,13 +384,22 @@ Error payloads use the same encoding. A producer implementing a generated header exports: 1. every function the header declares for the API; -2. one `{p}_{m}_checksum` per top-level module, returning the header's - `{P}_{M}_CHECKSUM`; -3. the runtime surface above (`alloc` and `dealloc` only for `wasm32`), with - `{p}_abi_version` returning `{P}_ABI_VERSION`. - -The producer owns every allocation it hands out, so `error_clear`, -`error_free`, and `free_bytes` must free with the same allocator that -produced the message, payload, or run. When building the library, define -`{P}_BUILD` so the header's `{P}_API` macro exports symbols on Windows. -`conformance/c/producer.c` is a complete example. +2. one `{p}_{m}_contract` per top-level module, returning a table with (at + least) the header's `{P}_{M}_CONTRACT` entries, sorted by id; +3. the runtime surface above, with `{p}_abi_version` returning + `{P}_ABI_VERSION`. + +```c +const calculator_contract_entry* calculator_calculator_contract(size_t* out_len) { + static const calculator_contract_entry table[] = CALCULATOR_CALCULATOR_CONTRACT; + if (out_len != NULL) *out_len = CALCULATOR_CALCULATOR_CONTRACT_LEN; + return table; +} +``` + +The producer owns every allocation it hands out and adopts every run a +consumer allocates, so `error_clear`, `error_free`, `free_bytes`, and the +adoption of a callback's returned run must all use the allocator `alloc` +uses. When building the library, define `{P}_BUILD` so the header's `{P}_API` +macro exports symbols on Windows. `conformance/c/producer.c` is a complete +example. diff --git a/docs/src/reference/idl.md b/docs/src/reference/idl.md index a2c732ac..8c7ac720 100644 --- a/docs/src/reference/idl.md +++ b/docs/src/reference/idl.md @@ -2,8 +2,9 @@ An IDL document describes an API: its modules, types, and functions. YAML, JSON, and TOML are all accepted; this page uses YAML. It documents schema -version `0.10.0`, the only version the current tools accept. A Rust producer -never writes one by hand (the CLI extracts it from the source), but the +version `0.11.0`, the only version the current tools accept. A Rust producer +never writes one by hand (the macro embeds the API in the built library, and +[`weaveffi extract`](../guides/extract.md) prints it as an IDL), but the schema is the same either way. Every YAML block on this page is a complete document that passes @@ -24,7 +25,7 @@ Editors using the YAML Language Server pick it up from a header comment: ## Document structure ```text -version: "0.10.0" +version: "0.11.0" modules: - name: my_module doc: "..." @@ -43,7 +44,7 @@ keys, so a misspelled field fails to parse instead of being ignored. | Field | Type | Required | Meaning | |-------|------|----------|---------| -| `version` | string | yes | Must be `"0.10.0"` | +| `version` | string | yes | Must be `"0.11.0"` | | `modules` | array of Module | yes | Top-level modules | ### Module @@ -90,12 +91,19 @@ A `TypeRef` is a string: | `bool` | Boolean | | `string` | UTF-8 text | | `bytes` | Byte string (`[u8]` is accepted and becomes `bytes`) | -| `Name`, `a.b.Name` | A struct, enum, interface, or callback interface | +| `Name` | A struct, enum, interface, or callback interface, by its bare name | | `T?` | Optional | | `[T]` | List | | `{K:V}` | Map | | `iter` | Lazy iterator (returns only) | +A user type is always written as its bare name: type names are global (see +[Modules and names](#modules-and-names)), so there's no module-qualified +spelling, and a dotted name such as `kv.Store` is rejected with +`QualifiedTypeRef`. `usize`, `isize`, `u128`, `i128`, and `char` aren't +primitives; a reference to one is reported as `UnsupportedPrimitive` (use +`u64` or `i64` for sizes and wide integers, and `string` for characters). + The parser reads outside in: `[Contact?]` is a list of optional contacts and `[Contact]?` an optional list. Composites nest to any depth. Quote anything YAML would otherwise interpret (`"string?"`, `"[i32]"`, `"{string:i32}"`). @@ -107,7 +115,7 @@ every target can use its native dictionary with exact equality. Floats, ```yaml # rejected: InvalidMapKey -version: "0.10.0" +version: "0.11.0" modules: - name: maps functions: @@ -124,7 +132,7 @@ and as the return of an async function (`AsyncIteratorReturn`): ```yaml # rejected: IteratorInInvalidPosition -version: "0.10.0" +version: "0.11.0" modules: - name: streaming functions: @@ -141,13 +149,13 @@ payload field. | Type | Param | Return | Buffered field | Inside `T?` `[T]` `{K:V}` | Map key | Callback param | Callback return | |------|-------|--------|----------------|---------------------------|---------|----------------|-----------------| | integers, floats, `bool` | yes | yes | yes | yes | integers, `bool` | yes | yes | -| `string` | yes | yes | yes | yes | yes | yes | no | -| `bytes` | yes | yes | yes | yes | no | yes | no | +| `string` | yes | yes | yes | yes | yes | yes | yes | +| `bytes` | yes | yes | yes | yes | no | yes | yes | | C-style enum | yes | yes | yes | yes | yes | yes | yes | -| struct, rich enum | yes | yes | yes | yes | no | yes | no | -| interface | yes | yes | yes | yes | no | yes | no | -| callback interface | yes | no | no | no | no | no | no | -| `T?`, `[T]`, `{K:V}` | yes | yes | yes | yes | no | yes | no | +| struct, rich enum | yes | yes | yes | yes | no | yes | yes | +| interface | yes | yes | yes | yes | no | yes | yes | +| callback interface | yes | no | no | `Cb?` as a top-level parameter only | no | no | no | +| `T?`, `[T]`, `{K:V}` | yes | yes | yes | yes | no | yes | yes | | `iter` | no | yes (sync) | no | no | no | no | no | How each family crosses the boundary is in the [C ABI contract](abi.md#families-and-slots). @@ -166,7 +174,7 @@ class, struct, or record type; no per-struct C functions exist. | `fields` | array of Field (required, non-empty) | Each with `name`, `type`, optional `doc` | ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: geometry structs: @@ -197,14 +205,13 @@ An enum is a fixed set of variants, each with an explicit `i32` `value`. | `variants` | array of Variant (required, non-empty) | Each with `name`, `value`, optional `doc` and `fields` | When no variant has `fields`, the enum is **C-style**: it crosses as an -`int32_t`, may be a map key, and may be returned from a callback method. -When any variant has `fields`, the enum is **rich** (a sum type): it crosses +`int32_t` and may be a map key. When any variant has `fields`, the enum is **rich** (a sum type): it crosses as a value buffer holding the variant's `value` as a tag followed by its fields, and bindings map it to a Swift enum with associated values, a sealed class hierarchy, a tagged union, and so on. Unit and data variants may mix. ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: shapes enums: @@ -258,13 +265,15 @@ static that returns the interface. An interface may appear in every position except a map key: parameters, returns, optionals, lists, map values, record and variant fields, error -payloads, iterator elements, async results, and callback parameters. `Store?` +payloads, iterator elements, async results, and callback parameters and +returns. A map keyed by an interface reports both `InvalidMapKey` and +`InterfaceInInvalidPosition`. `Store?` at the top level of a parameter or return is a nullable pointer. Every position transfers or borrows references by the rules in [Errors and Memory](../guides/errors-and-memory.md#objects). ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: kv errors: @@ -308,8 +317,8 @@ modules: ``` ```yaml -# rejected: InterfaceInInvalidPosition -version: "0.10.0" +# rejected: InvalidMapKey, InterfaceInInvalidPosition +version: "0.11.0" modules: - name: kv interfaces: @@ -347,22 +356,34 @@ for as long as it holds the implementation. | `doc`, `deprecated` | string | Documentation, deprecation message | | `methods` | array of Function (required, non-empty) | In vtable order | -Methods are restricted, each enforced as `InvalidCallbackMethod`: - -- synchronous: no `async`, no `cancellable`; -- no `throws`: a consumer failure is reported as the runtime code `-4`; -- return nothing or a direct value (an integer, float, `bool`, or C-style - enum), because no consumer allocation can flow back to the producer; -- parameters may be any type except a callback interface or an iterator; - an object parameter transfers one reference to the consumer. +Methods are restricted: + +- synchronous: no `async`, no `cancellable` (`InvalidCallbackMethod`); +- return nothing or any type except an iterator (`InvalidCallbackMethod`) + or a callback interface (`CallbackInterfaceInInvalidPosition`); a string, + bytes, or buffered return is a run the consumer allocates with + `{prefix}_alloc` and the producer adopts, and an object return transfers + one reference to the producer; +- parameters may be any type except an iterator + (`IteratorInInvalidPosition`) or a callback interface + (`CallbackInterfaceInInvalidPosition`); an object parameter transfers one + reference to the consumer. + +A method may set `throws: true` when an error domain is in scope for its +module (otherwise it's `ThrowsWithoutErrorDomain`). Its consumer may then +fail with one of the domain's codes, with the code's fields (which can't +include objects), and the producer receives the typed error. Any other +consumer failure, and every failure of a method without `throws`, reaches +the producer as the runtime code `-4`. A callback interface is valid only as a top-level parameter of a function, -constructor, method, or static. Anything else is +constructor, method, or static, passed bare (`Listener`) or optional +(`Listener?`, where none is a null vtable). Anything else is `CallbackInterfaceInInvalidPosition`: ```yaml # rejected: CallbackInterfaceInInvalidPosition -version: "0.10.0" +version: "0.11.0" modules: - name: events callback_interfaces: @@ -375,7 +396,7 @@ modules: ``` ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: events enums: @@ -398,6 +419,8 @@ modules: - name: on_message params: - { name: message, type: Message } + - name: name + return: string interfaces: - name: EventBus constructors: @@ -409,18 +432,27 @@ modules: return: i64 ``` -A callback interface lowers to a vtable type with one entry per method and a -trailing `free`; a parameter lowers to a `ctx` pointer plus the vtable. From -the `kitchen_sink` fixture: +A callback interface lowers to a vtable type that starts with a header (the +vtable's `size`, reserved `flags`, and the `free` release hook) followed by +one entry per method in declaration order; a parameter lowers to a `ctx` +pointer plus the vtable. From the `kitchen_sink` fixture, whose `label` +returns a `string` and throws, `latest` returns an `Item?`, and `favorite` +returns a `Gadget`: ```c typedef struct kitchen_sink_kitchen_ReadyListener_vtable { + uint32_t size; + uint32_t flags; + void (*free)(void* ctx); void (*on_ready)(void* ctx, int32_t code, const uint8_t* msg_ptr, size_t msg_len, kitchen_sink_error* out_err); bool (*on_item)(void* ctx, const uint8_t* item_ptr, size_t item_len, kitchen_sink_kitchen_Gadget* gadget, kitchen_sink_error* out_err); - void (*free)(void* ctx); + void (*label)(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); + void (*latest)(void* ctx, uint8_t** out_ptr, size_t* out_len, kitchen_sink_error* out_err); + kitchen_sink_kitchen_Gadget* (*favorite)(void* ctx, kitchen_sink_error* out_err); } kitchen_sink_kitchen_ReadyListener_vtable; int32_t kitchen_sink_kitchen_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); +bool kitchen_sink_kitchen_maybe_subscribe(void* listener_ctx, const kitchen_sink_kitchen_ReadyListener_vtable* listener_vtable, kitchen_sink_error* out_err); ``` ## Error domains @@ -435,13 +467,13 @@ names you declared. | `name` | string (required) | Domain type name (`KvError`) | | `codes` | array of Code (required) | The codes | -Each code has a `name` (PascalCase by convention), a positive `code`, a -default `message`, an optional `doc`, and optional `fields`: a structured -payload with the same shape as struct fields, delivered as properties of the -raised error. +Each code has a required `name` (PascalCase by convention), positive `code`, +and default `message`, plus an optional `doc` and optional `fields`: a +structured payload with the same shape as struct fields, delivered as +properties of the raised error. ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: kv errors: @@ -473,7 +505,7 @@ A domain is in scope for its module and every module nested in it, and ```yaml # rejected: ThrowsWithoutErrorDomain -version: "0.10.0" +version: "0.11.0" modules: - name: contacts functions: @@ -498,7 +530,7 @@ Constructors can't be async, and `cancellable` requires `async`: ```yaml # rejected: CancellableNotAsync -version: "0.10.0" +version: "0.11.0" modules: - name: net functions: @@ -509,29 +541,36 @@ modules: callback interface propagates its message to every binding's native deprecation marker. -## Modules and type references +## Modules and names Modules nest to any depth. A nested module shares the validation rules of a top-level one, and an error domain declared on a parent serves the subtree. -C symbols join the path with underscores: a function `get_record` in module -`app.data` is `{prefix}_app_data_get_record`. -Every resolved type is identified by its **absolute path**: the declaring -module's path plus the name (`kv.Store`, `app.data.Record`). A reference -spells it one of two ways: - -- **Bare** (`Store`): resolves through the API-wide type namespace. Struct, - enum, interface, callback interface, and error domain names are unique - across the whole API (`DuplicateTypeName`), so a bare name is never - ambiguous. -- **Qualified** (`kv.Store`): the qualifier must match the declaring module's - path exactly; `stats.Store` for a `Store` declared in `kv` is an - `UnknownTypeRef`. - -A reference to a name nothing declares is always an `UnknownTypeRef`. +**Names are global.** Modules group declarations and namespace their C +symbols, but they don't scope names: + +- Struct, enum, interface, callback interface, and error domain names are + unique across the whole API (`DuplicateTypeName`). +- Free-function names are unique across the whole API + (`DuplicateFunctionName`), because several targets flatten every module's + functions into one namespace. A free function also can't share its name + with an error domain (`NameCollisionWithErrorDomain`). +- Error code names are unique across every domain (`DuplicateErrorCodeName`). + +Members are scoped to their owner: parameters to their callable, fields to +their record or variant, variants to their enum, and constructors, methods, +and statics to their interface. Sibling modules need distinct names +(`DuplicateModuleName`), but modules in different subtrees may share one. + +Because every type name identifies one declaration, a reference is always the +bare name, wherever the type is declared. A reference to a name nothing +declares is an `UnknownTypeRef`. C symbols join the declaring module's path +with underscores: a function `get_record` in module `app.data` is +`{prefix}_app_data_get_record`, and a record `Record` declared there is +`{prefix}_app_data_Record`. ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: kv interfaces: @@ -549,10 +588,26 @@ modules: functions: - name: get_stats params: - - { name: store, type: kv.Store } + - { name: store, type: Store } return: Stats ``` +```yaml +# rejected: DuplicateFunctionName, QualifiedTypeRef +version: "0.11.0" +modules: + - name: kv + interfaces: + - name: Store + constructors: + - { name: open, params: [{ name: path, type: string }] } + functions: + - { name: stats, params: [{ name: store, type: kv.Store }], return: i64 } + - name: metrics + functions: + - { name: stats, return: i64 } +``` + ## Documentation comments `doc:` is accepted on modules, functions, parameters, structs, fields, enums, @@ -565,9 +620,10 @@ block strings (`|`) are preserved. An absent `doc:` emits nothing. `weaveffi validate` (and every generating command) runs every rule and reports every violation in one pass, each with a message, a suggestion, and a -source span when one can be located. `--format json` emits the same -diagnostics with a `code` field naming the variant; `--warn` adds advisory -warnings. +source span located within the enclosing declaration when one can be found. +`--format json` emits the same diagnostics, each with a `code` field naming +the variant alongside the variant's own fields (`name`, `module`, and so +on); `--warn` adds advisory warnings. The rules in brief: @@ -575,20 +631,25 @@ The rules in brief: `_`) and not one of `if`, `else`, `for`, `while`, `loop`, `match`, `type`, `return`, `async`, `await`, `break`, `continue`, `fn`, `struct`, `enum`, `mod`, `use`. Generators escape other languages' keywords themselves. -- Names are unique in their scope; type names and error code names are - unique across the whole API. +- Type names, free-function names, and error code names are unique across + the whole API; every other name is unique in its scope (see + [Modules and names](#modules-and-names)). - Structs, enums, interfaces, and callback interfaces aren't empty. -- Types resolve, and appear only in the positions in the table above. +- Types are bare names that resolve, and appear only in the positions in the + table above. - `throws` needs a domain in scope; `cancellable` needs `async`. -- Codes are positive and unique; a domain's name doesn't collide with a - function in the same module. +- Codes are positive and their values unique within a domain; no free + function shares a domain's name. - **No two declarations lower to the same C symbol.** Every symbol is the prefix plus the underscore-joined module path plus the declaration name, - so `m.x_y`, `m.x.y`, and `m_x.y` collide; so do a free function - `Store_get` and a `get` method on interface `Store`, a function named - `Store_clone` and `Store`'s implicit `_clone`, and a function named - `foo_callback` and the completion type of an async `foo`. The full symbol - table is in the [C ABI contract](abi.md#symbol-names). + so a function `x_y` in module `m` collides with a function `y` in module + `m.x` or `m_x`; so do a free function `Store_get` and a `get` method on + interface `Store`, a function named `Store_clone` and `Store`'s implicit + `_clone`, and a function named `foo_callback` and the completion type of + an async `foo`. Symbols that fall in a family the C value-buffer helpers + reserve (`{prefix}_list_*`, `{prefix}_map_*`, and so on, which covers + everything in a top-level module named `list`) collide too. The full + symbol table is in the [C ABI contract](abi.md#symbol-names). A document with an unsupported `version` reports only `UnsupportedSchemaVersion`. @@ -597,47 +658,44 @@ A document with an unsupported `version` reports only | Code | Reported when | |------|---------------| -| `UnsupportedSchemaVersion` | `version` isn't `0.10.0` | +| `UnsupportedSchemaVersion` | `version` isn't `0.11.0` | | `NoModuleName` | a module has no name | | `InvalidModuleName` | a module name isn't an identifier or is reserved | | `DuplicateModuleName` | two sibling modules share a name | | `InvalidIdentifier` | a name isn't a valid identifier | | `ReservedKeyword` | a name is a reserved word | -| `DuplicateFunctionName` | two functions in a module share a name | +| `DuplicateFunctionName` | two free functions anywhere in the API share a name | | `DuplicateParamName` | two parameters of a callable share a name | -| `NameCollisionWithErrorDomain` | a function and the module's error domain share a name | -| `DuplicateStructName` | two structs in a module share a name | +| `NameCollisionWithErrorDomain` | a free function and an error domain share a name | | `DuplicateStructField` | two fields of a struct share a name | | `EmptyStruct` | a struct has no fields | -| `DuplicateEnumName` | two enums in a module share a name | | `EmptyEnum` | an enum has no variants | | `DuplicateEnumVariant` | two variants of an enum share a name | | `DuplicateEnumValue` | two variants of an enum share a value | | `DuplicateEnumVariantField` | two fields of a rich-enum variant share a name | -| `DuplicateInterfaceName` | two interfaces in a module share a name | | `DuplicateInterfaceMember` | two members of an interface share a name | | `EmptyInterface` | an interface has no members | | `ConstructorHasReturn` | a constructor declares `return` | | `AsyncConstructor` | a constructor is `async` | | `InterfaceInInvalidPosition` | an interface is used as a map key | -| `DuplicateCallbackInterfaceName` | two callback interfaces in a module share a name | | `EmptyCallbackInterface` | a callback interface has no methods | | `DuplicateCallbackMethod` | two methods of a callback interface share a name | -| `InvalidCallbackMethod` | a callback method is async, cancellable, throwing, or returns a non-direct type | -| `CallbackInterfaceInInvalidPosition` | a callback interface appears other than as a top-level parameter | -| `DuplicateTypeName` | two types anywhere in the API share a bare name | -| `UnknownTypeRef` | a reference names no declaration, or its qualifier doesn't match the declaring module | +| `InvalidCallbackMethod` | a callback method is async or cancellable, or returns an iterator | +| `CallbackInterfaceInInvalidPosition` | a callback interface appears other than as a top-level parameter (bare or `Cb?`) | +| `DuplicateTypeName` | two types (including error domains) anywhere in the API share a name | +| `UnknownTypeRef` | a reference names no declaration | +| `QualifiedTypeRef` | a reference is a dotted, module-qualified name | +| `UnsupportedPrimitive` | a reference names `usize`, `isize`, `u128`, `i128`, or `char` | | `InvalidMapKey` | a map key isn't an integer, `bool`, `string`, or C-style enum | | `IteratorInInvalidPosition` | `iter` appears other than as an outermost return | | `AsyncIteratorReturn` | an async function returns `iter` | | `CancellableNotAsync` | a synchronous callable is `cancellable` | -| `ThrowsWithoutErrorDomain` | `throws: true` with no error domain in scope | +| `ThrowsWithoutErrorDomain` | `throws: true` on a callable or callback method with no error domain in scope | | `ErrorDomainMissingName` | an error domain has no name | -| `DuplicateErrorName` | two codes in a domain share a name | | `DuplicateErrorCode` | two codes in a domain share a value | -| `DuplicateErrorCodeName` | two domains declare a code with the same name | +| `DuplicateErrorCodeName` | two codes anywhere in the API share a name | | `InvalidErrorCode` | a code is zero or negative | -| `SymbolCollision` | two declarations lower to the same C identifier | +| `SymbolCollision` | two declarations lower to the same C identifier, or one falls in a reserved helper family | ### Warnings @@ -653,11 +711,11 @@ Warnings never fail validation. ## Complete example -A trimmed version of the `contacts` sample, combining an enum, a record, an -error domain, an interface, and a nested module that uses its parent's types: +An address book combining an enum, a record, an error domain, an interface, +and a nested module that uses its parent's types: ```yaml -version: "0.10.0" +version: "0.11.0" modules: - name: contacts doc: An in-memory address book. diff --git a/docs/src/reference/naming.md b/docs/src/reference/naming.md index 92a8c37b..c843cfe4 100644 --- a/docs/src/reference/naming.md +++ b/docs/src/reference/naming.md @@ -20,19 +20,28 @@ Rust producer, `Cargo.toml` (the full rules are in Two conversions turn an arbitrary name into identifiers: - **snake**: lowercase ASCII letters and digits, with every run of other - characters collapsed to one `_` and trailing `_` trimmed; a leading digit - gets a `lib_` prefix. `my-kv.store` becomes `my_kv_store`, `3d-engine` - becomes `lib_3d_engine`. + characters collapsed to one `_` and leading and trailing `_` dropped; a + leading digit gets a `lib_` prefix. `my-kv.store` becomes `my_kv_store`, + `3d-engine` becomes `lib_3d_engine`. - **PascalCase**: each run of letters and digits starts a word whose first letter is uppercased, and existing capitals are kept; a leading digit gets an `N` prefix. `my-kv.store` becomes `MyKvStore`. -`{PREFIX}` is the prefix uppercased. +`{PREFIX}` is the prefix uppercased. A `c_prefix` you set is itself passed +through snake. Setting `c_prefix` or `library` for a Rust producer is an +error, because the macro derives the prefix from the crate. A library read +on its own with `--library` (no crate or IDL) takes its prefix from its +embedded metadata, its `library` from its file name, and its `name` from +`[package] name`, else the prefix. ## Per-target defaults -| Target | Default names (each overridable in `[generators.]`) | -|--------|---------------------------------------------------------------| +The names a `[generators.]` key can override are listed in +[Project Configuration](../guides/config.md#generatorstarget); the rest +follow the identity. + +| Target | Default names | +|--------|---------------| | C | header `{library}.h`; include guard `{PREFIX}_H`; export macro `{PREFIX}_API` | | C++ | header `{library}.hpp`, which includes `{library}.h` (shipped alongside); namespace `{prefix}` | | Swift | SwiftPM package, product, and module `PascalCase(name)`; C module `C{PascalCase(name)}` | @@ -40,33 +49,49 @@ Two conversions turn an arbitrary name into identifiers: | Node.js | npm package `{name}`; addon `{library}_node.node`; one namespace export per module | | WebAssembly | npm package `{name}`; ES module; one namespace export per module | | Python | distribution `{name}`; import package `{prefix}` | -| .NET | namespace and assembly `PascalCase(name)` | +| .NET | namespace, assembly, and NuGet id `PascalCase(name)` | | Dart | package `{prefix}` | -| Go | module path `{name}` unless `module_path` is set; package `{prefix}` | +| Go | module path `{name}`; package `{prefix}` | | Ruby | gem `{name}`; module `PascalCase(name)`; require path `{prefix}` | For the `kvstore` sample (crate `kvstore`) that's `kvstore.h`, Swift module `Kvstore`, `import kvstore` in Python, and `require "kvstore"` in Ruby. -**Library loading.** Every target loads the native library by its +**Library loading.** Every target finds the native library by its `library` name using the platform's file naming (`lib{library}.dylib`, -`lib{library}.so`, `{library}.dll`). Every loader first honors the -environment variable `{PREFIX}_LIBRARY` (for example `KVSTORE_LIBRARY`), an -explicit path that wins over bundled and system copies. - -Nothing a generator emits is named after WeaveFFI; the only mentions are the -generated-file header comment and runtime-version comments. Two libraries -built with WeaveFFI therefore never collide in symbols, packages, types, or -environment variables. +`lib{library}.so`, `{library}.dll`). The environment variable +`{PREFIX}_LIBRARY` (for example `KVSTORE_LIBRARY`) names an explicit path +that wins over bundled and system copies where a target resolves the +library itself: at run time for Python, Ruby, .NET, Dart, Kotlin (the JVM +loads it before the JNI shim), and WebAssembly on Node.js (the `.wasm` +file); at install time for Node.js, whose fallback addon build links it; and +at CMake configure time for C++. C, Swift, and Go link the library at build +time and don't read it. + +Nothing a generator emits is named after WeaveFFI; the only mentions are +comments (the generated-file header and notes on regenerating or +packaging). Two libraries built with WeaveFFI therefore never collide in +symbols, packages, types, or environment variables. + +## Global IDL names + +Type names (records, enums, interfaces, callback interfaces, and error +domains), free-function names, and error-code names are **global**: each is +unique across the whole API, and validation rejects a second declaration +anywhere in the module tree. Modules group declarations and namespace their +C symbols, but a name never needs its module to be unambiguous, so a type +reference in the IDL is always the bare name (`Store`, never `kv.Store`). +Member names (parameters, fields, variants, and interface members) are +scoped to their owner. ## C identifiers -C identifiers are the prefix, the underscore-joined module path, and the IDL -name, verbatim: `kvstore_kv_Store_open`, `kvstore_kv_KvError_KeyNotFound`. +C identifiers are the prefix, the underscore-joined path of the declaring +module, and the IDL name, verbatim: `kvstore_kv_Store_open`, `kvstore_kv_KvError_KeyNotFound`. IDL names are never re-cased in C, except that an iterator type's function -part is PascalCase (`kvstore_kv_Store_ListKeysIterator`). The full table is -in the [C ABI contract](abi.md#symbol-names); validation rejects any API in -which two declarations produce the same identifier. +part is PascalCase (`kvstore_kv_Store_KeysIterator` for `Store.keys`). The +full table is in the [C ABI contract](abi.md#symbol-names); validation +rejects any API in which two declarations produce the same identifier. ## Identifiers in generated code @@ -89,7 +114,8 @@ keep their IDL names in every target, except where a language idiom adds a prefix or suffix (.NET prefixes interfaces it generates for callback interfaces with `I`, for example). **C-style enum variants** follow the target's enum idiom: `Volatile` in most targets, `.volatile` in Swift, -`EntryKindVolatile` in Go, `EntryKind::VOLATILE` in Ruby. +`volatile` in Dart, `EntryKindVolatile` in Go, `EntryKind::VOLATILE` in +Ruby. **Constructors.** A constructor named `new` becomes the language's constructor (`init`, `__init__`, `initialize`, a C++ or C# constructor); @@ -97,20 +123,23 @@ other constructors become static factories (`Store.open`). Go has no constructors, so they become package functions (`NewEventBus`, `OpenStore`). **Modules.** How module members are grouped depends on the target's notion -of a namespace. Kotlin emits one `object` per IDL module, and Node.js and -WebAssembly export one namespace object per module (nested modules nest), -so two modules can both declare `get`. Targets that place members in one -flat namespace drop the module name from free functions by default and -offer `strip_module_prefix = false` to keep it (`kv_stats_get_stats`); see -[Project Configuration](../guides/config.md#global). Each language page -documents its layout. +of a namespace. C++ emits a nested namespace per IDL module, Swift a +caseless `enum` per module (nested modules nest), Kotlin an `object` per +module, .NET a static class per module named after its path (`Kv`, +`KvStats`), and Node.js and WebAssembly a namespace object per module +(nested modules nest). Targets that place members in one flat namespace +(Python, Ruby, Dart, Go) use the bare function name, which global names keep +collision-free; Go falls back to the module-prefixed spelling only for a +free function whose name would clash with another generated name, such as a +constructor's (kvstore's async `open_store` becomes `KvOpenStore` because +`Store.open` is `OpenStore`). Each language page documents its layout. **Errors.** Each target has a root error type named after the package and one type (or case) per error domain and code, named from the IDL with at most one idiomatic suffix (`Error` or `Exception`). Write code names without a suffix (`KeyNotFound`, not `KeyNotFoundError`) and let the generator add -it. Code names are unique across the API because several targets flatten -them into one namespace. +it. Code names are global because several targets flatten them into one +namespace. **Reserved words.** A name that's a keyword in the target language gains a trailing `_` (`type` becomes `type_`), a rule that's stable under repetition. @@ -118,6 +147,32 @@ A user type whose name would shadow a standard type in the target (a Kotlin type named `Result`, say) is referenced in a way that avoids the clash; the language pages describe how. +**Reserved member names.** An interface member (a constructor that becomes +a factory, a method, or a static) whose target spelling matches a member the +object wrapper declares or inherits gains a trailing `_` as well, so the +wrapper's own release, handle, and identity members keep working. The rule +applies after case conversion and keyword escaping, to these names: + +| Target | Reserved member names | +|--------|-----------------------| +| C++ | `handle`, `clone_handle`, `raw_type` | +| Swift | `ptr`, `clonePtr`, `wvRead`, `wvWrite` | +| Kotlin | `close`, `handle`, `cloneHandle`, `fromHandle`, `fromHandleOrNull`, `invoke`, `equals`, `hashCode`, `toString`, `wait`, `notify`, `notifyAll` | +| Node.js, WebAssembly | methods: `close`, `constructor`; statics and factories: `name`, `length`, `prototype`, `caller` | +| Python | `close` | +| .NET | `Dispose`, `Handle`, `Adopt`, `CloneHandle`, `NativeHandle`, `Equals`, `GetHashCode`, `GetType`, `ToString`, `ReferenceEquals`, `MemberwiseClone`, `Finalize`, and the interface's own name | +| Dart | `dispose`, `hashCode`, `toString`, `runtimeType`, `noSuchMethod` | +| Go | methods: `Close` (factories and statics are package functions spelled with the type name, so they can't collide) | +| Ruby | methods: `close`, `handle`, `initialize`, `initialize_copy`, `class`, `clone`, `dup`, `freeze`, `hash`, `object_id`, `send`; statics and factories: `allocate`, `name` | + +A method `close` on `Store` is therefore `close_` in Kotlin, Node.js, +WebAssembly, Python, and Ruby and `Close_` in Go, and a method `dispose` is +`dispose_` in Dart and `Dispose_` in .NET; every other target keeps the +plain spelling. C needs no rule, because members are prefixed symbols +(`kvstore_kv_Store_close`), but a member named `clone` or `destroy` lowers to +the same symbol as the interface's own `_clone` or `_destroy`, so validation +rejects it (`SymbolCollision`). + ## Target names A target is named after what its consumer identifies with: the language when diff --git a/docs/src/reference/value-buffers.md b/docs/src/reference/value-buffers.md index 02bafaf1..b5708098 100644 --- a/docs/src/reference/value-buffers.md +++ b/docs/src/reference/value-buffers.md @@ -4,7 +4,7 @@ Records, rich enums, optionals, lists, maps, and error payloads cross the C ABI *by value*, serialized in one compact binary format. A buffered value occupies one `(const uint8_t*, size_t)` slot pair however deeply it nests. This page is the normative wire format every generator and the -`weaveffi-abi` runtime (`weaveffi_abi::buffer`) implement. +`weaveffi::abi` runtime (`weaveffi::abi::buffer`) implement. ## Which types are buffered @@ -12,7 +12,8 @@ This page is the normative wire format every generator and the - a rich enum (any variant has fields) - `[T]`, `{K:V}` - `T?`, except `Interface?`, which stays a nullable object pointer at the top - level of a parameter or return + level of a parameter or return, and `Cb?`, an optional callback parameter + whose null vtable means none Everything else keeps its own slot shape at the top level (see the [C ABI contract](abi.md#families-and-slots)): direct values by value, strings @@ -41,14 +42,15 @@ encoder refuses to write more rather than truncate. | `bytes` | `u32` length, then raw bytes | | `T?` | 1 flag byte (`0` absent, `1` present), then the value if present | | `[T]` | `u32` count, then each element | -| `{K:V}` | `u32` count, then alternating key and value | +| `{K:V}` | `u32` count, then alternating key and value; no key appears twice | | struct | each field, in declaration order | | rich enum | `i32` tag (the variant's `value`), then the variant's fields in order | | error payload | the code's fields, in declaration order | The format is compositional, so `{string:[T?]}`, records inside records, and -lists of rich enums need no special cases. `[u8]` is canonicalized to `bytes` -at parse time; the two encode identically. +lists of rich enums need no special cases. A map's encoding never repeats a +key, so no entry can silently disappear between the two sides. `[u8]` is +canonicalized to `bytes` at parse time; the two encode identically. Example: a `Point { x: f64, y: f64 }` with `x = 1.5`, `y = -2.0` is 16 bytes, `00 00 00 00 00 00 F8 3F 00 00 00 00 00 00 00 C0`; a `[string]` of @@ -63,6 +65,7 @@ A decoder rejects a buffer that: - holds invalid UTF-8 in a string; - declares a string or byte length larger than the bytes remaining; - holds an enum value or rich-enum tag that isn't declared; +- repeats a key in a map; - has bytes left over after the complete value. A collection count is not checked against the bytes remaining, because an @@ -92,6 +95,14 @@ exactly one strong reference, in either direction. A zero token is invalid in a non-optional position; `Interface?` inside a buffer uses the optional flag byte followed by the token. +A callback method's error payload can't carry object tokens: an error code +whose fields include an interface can't be reported from a callback, and a +Rust producer decodes a callback's payload with a reader that refuses every +token. That reader (`BufferReader::token_free`) is also why decoding is +otherwise `unsafe` in the Rust runtime: only a reader that refuses tokens can +safely decode arbitrary bytes, because no safe function can turn an arbitrary +`u64` into an object reference. + ## Slots and ownership | Position | Slots | Owner | diff --git a/docs/src/roadmap.md b/docs/src/roadmap.md index 047a9700..591f3a8c 100644 --- a/docs/src/roadmap.md +++ b/docs/src/roadmap.md @@ -1,27 +1,19 @@ # Roadmap -WeaveFFI is in active `0.x` development. Schema 0.10 and ABI revision 3 made -every name derive from the library's identity, moved strings to `(ptr, len)` -runs, added contract checksums, and made cancellation work idiomatically on -every target. This page lists what comes next. Items are **planned** (the -design is settled) or **exploring** (wanted, with open design questions). -Nothing carries a date; the +WeaveFFI is in active `0.x` development. Schema 0.11 and ABI revision 4 made +names global, replaced the per-module checksums with per-declaration contract +tables, gave callback methods rich returns, typed `throws`, optional callback +parameters, and a size-checked vtable header, made `{prefix}_alloc` the one +allocator contract on every target, moved the codec conformance lanes onto +shared test vectors, and taught the CLI to read a Rust producer's API from +its built library. This page lists what comes next. Items are **planned** +(the design is settled) or **exploring** (wanted, with open design +questions). Nothing carries a date; the [changelog](https://github.com/weavefoundry/weaveffi/blob/main/CHANGELOG.md) records what shipped. ## Callback interfaces -### Rich callback returns (planned) - -Callback methods return nothing or a direct value, because a consumer -allocation can't safely cross back to a producer with a different allocator. -The plan is an allocator contract for callback returns: the consumer writes -the result into producer-owned storage obtained through the runtime (as the -Wasm glue already does with `{prefix}_alloc`), or returns its own allocation -with a release function the producer calls after copying. With that, -strings, bytes, records, rich enums, optionals, lists, maps, and objects -become valid callback returns, and typed `throws` on callback methods follows. - ### Async callback methods (planned) A callback method that returns a future on the consumer side needs a @@ -29,23 +21,26 @@ completion flowing the other way and a cancellation story when the producer drops the future. The vtable shape is simple (a completion function and context per async method); the hard part is one producer working the same way whether the consumer runtime is an event loop, a thread pool, or the -single-threaded Wasm host. This follows the allocator contract above. - -### Vtable versioning (exploring) - -A vtable is a fixed struct, so adding a method to a callback interface is a -breaking change even when no consumer needs it. A size or version field at -the head of each vtable would let a newer producer detect an older consumer's -shorter table and fall back, which matters once 1.0 promises additive -changes are compatible. +single-threaded Wasm host. -### Callback thread affinity (exploring) +### Off-thread callbacks in Dart (exploring) Dart can't run a value-returning callback method synchronously on a thread -other than its isolate's, so a producer that calls one from a worker thread -aborts the process today (void methods are forwarded safely). A per-vtable -thread-affinity hint, or a runtime helper that lets a consumer refuse an -off-thread call with `-4`, would turn that abort into an error. +that isn't a Dart isolate thread, so a producer that calls one from its own +worker thread aborts the process today (void methods are forwarded to the +isolate safely). Pure `dart:ffi` can't route that return, so the fix is +either a small native shim that hops to the isolate and waits, or a +per-vtable thread-affinity hint that lets the producer refuse the call with +`-4` instead. See the [Dart page](generators/dart.md#known-limitations). + +### Synchronous calls that wait on Node.js callbacks (exploring) + +A synchronous Node.js call that blocks the JS thread while a producer thread +waits on a callback the JS thread must run would deadlock, so the addon +gives such a callback about a second to reach the JS thread and then fails +it with `-4`. A long synchronous call that legitimately overlaps callbacks +hits that guard. Making the call itself pump callbacks while it waits would +remove the heuristic. See the [Node.js page](generators/node.md#threading). ## Definitions @@ -54,17 +49,8 @@ off-thread call with `-4`, would turn that abort into an error. An IDL API is one document. Large APIs want to split by module, and a monorepo wants to reference another package's types. The plan is an `imports:` list resolved at parse time, with bare type names still unique -across the merged API, and `diff`, `validate`, the cache, and checksums -tracking every imported file. - -### Extraction from the compiled library (exploring) - -The CLI reads a Rust producer's API by parsing source, so it sees only -inline `#[weaveffi::module]`s in one file and can't expand macros. The macro -already lowers each module to the IR at compile time; embedding that IR in -the built library (a custom section or an exported symbol) would let -`weaveffi generate` read the exact API from the artifact, removing the -one-file limit and any chance of the parser and the compiler disagreeing. +across the merged API, and `diff`, `validate`, and contract tables covering +every imported file. ### Generic and trait-object interfaces (exploring) @@ -79,6 +65,13 @@ Producers pass time as `i64` with a documented unit. `duration` and `timestamp` primitives mapped to each language's types would remove the ambiguity; the open question is the representation. +### Custom types (exploring) + +A producer type that crosses as a builtin (a `Uuid` as `string`, a `Url` as +`string`, a fixed-point amount as `i64`) has to be converted by hand on both +sides today. A declared custom type with per-language conversion hooks would +let each binding expose the language's own type. + ## Targets and runtime ### Kotlin Multiplatform (exploring) @@ -94,6 +87,22 @@ a call is on the stack. A spawner that schedules on the JS event loop, or shared-memory builds with Web Workers, would lift both limits and let Emscripten mode support async functions and callback interfaces. +### Zero-copy blittable records (exploring) + +Every record crosses as a serialized value buffer. A record of fixed-size +scalars could instead cross as a `#[repr(C)]` struct passed by pointer, which +matters for hot paths that move many small records. The open questions are +how a record opts in, and how the contract hash and every target's layout +checks keep the two sides agreeing on the layout. + +### Poll-based async (exploring) + +Async functions complete through a callback the producer fires on its own +thread, and each binding hops back to its scheduler. A poll-based protocol, +where the consumer's runtime drives the future and the producer only wakes +it, would fit event-loop runtimes better and avoid that hop. It would be an +option next to the completion-callback ABI, not a replacement. + ### Per-language support packages (exploring) Every package inlines its helpers (codec, error types, object base), which @@ -108,27 +117,23 @@ consideration; inlining would stay the default. The workspace's tests run on Windows, Linux, and macOS, but the conformance harness runs only on Linux and macOS. Adding Windows lanes means making `conformance/run.sh` and the per-language scripts portable (or adding -PowerShell equivalents) and installing each toolchain on Windows runners. - -### Shared codec vectors (planned) - -Each language's `codec` conformance consumer asserts the same round-trip -values by hand. Moving those values into one data file that every consumer -reads (or into producer-side golden values) would shrink the consumers and -keep the eleven lanes from drifting apart. +PowerShell equivalents), installing each toolchain on Windows runners, and +prebuilding the Node.js addon and JNI shim for `windows-x64`, which +`weaveffi build` can't do yet. ## Toward 1.0 1.0 means the surfaces in [What 1.0 will cover](stability.md#what-10-will-cover) stop changing without a major release. It needs: -- ABI revision 3 stable for several releases, with the callback-return - allocator contract and vtable versioning settled so 1.0 doesn't need - revision 4; +- ABI revision 4 stable for several releases, so 1.0 doesn't need + revision 5; - every target passing the full conformance matrix on Linux, macOS, and Windows; - multi-file IDL and the deprecation policy in place; - a schema migration tool and more than one accepted schema version; +- the known limitations on each generator page resolved or documented as + permanent; - a review of every published crate's public API. ## Contributing diff --git a/docs/src/samples.md b/docs/src/samples.md index 6b2c1f4a..24e16a8b 100644 --- a/docs/src/samples.md +++ b/docs/src/samples.md @@ -1,86 +1,85 @@ # Samples -The `samples/` directory holds six Rust producers written with -`#[weaveffi::module]`. Each is a workspace crate built as a `cdylib` with the -`leak-check` feature on, has unit tests that call its exported symbols -directly, and is consumed by the [conformance harness](generators/README.md#conformance-lanes) -in all eleven languages. +The `samples/` directory holds three Rust producers written with +`#[weaveffi::module]`: a minimal one, a wire-format oracle, and a +feature-complete one. Each is a workspace crate built as a `cdylib` with the +`leak-check` feature on, has unit tests that call it directly, and is +consumed by the [conformance harness](generators/README.md#conformance-lanes) +in all eleven languages, one consumer per sample per language. ```bash -cargo build -p kvstore -weaveffi generate samples/kvstore/src/lib.rs -o generated +weaveffi generate samples/kvstore -o bindings ``` Each sample's crate name is its C prefix: `kvstore` exports -`kvstore_kv_Store_open`, and its bindings load `libkvstore` (or -`KVSTORE_LIBRARY`). The `weaveffi.toml` beside each crate supplies package +`kvstore_kv_Store_open`, and its bindings load `libkvstore` (the targets +that load it at run time also accept a path in `KVSTORE_LIBRARY`). `weaveffi generate` builds the crate and reads its API +from the library; the `weaveffi.toml` beside each crate supplies package metadata. ## calculator -The smallest producer: a `calculator` module with `add`, `mul`, `echo` -(a string round trip), and `div`, which throws the one-code `CalcError` -domain. Start here to see the shape of a producer and its tests. The -`c-producer-exports` lane also implements the calculator's generated header -by hand in C (`conformance/c/producer.c`) and checks that every required -runtime and checksum symbol is exported. +The getting-started example, small enough to read in a minute: a +`calculator` module with `add` (wrapping on overflow), `divide`, which +throws the one-code `CalcError` domain on a zero divisor, and `greet`, a +string in and a string out. The `c-producer-exports` lane also implements +the calculator's generated header by hand in C (`conformance/c/producer.c`) +and checks that every required runtime and contract symbol is exported. -## contacts - -An address book built around a `ContactBook` interface whose methods guard -their state with a `Mutex` and throw `ContactsError` codes. It shows records -(`Contact`, with an optional email), a C-style enum (`ContactType`), lists of -records, and both ways modules relate: - -- `contacts::groups` is nested under the `contacts` root, so it uses the - parent's `ContactBook` interface and `ContactType` enum, returns new - `ContactBook` objects, and inherits the parent's error domain. -- `directory` is a sibling root with its own `DirectoryError`. Across roots - only value types can be shared, so it takes the `Contact` record. - -## events +## codec -A publish/subscribe bus: an `EventBus` interface, a consumer-implemented -`Subscriber` callback interface, a `Message` record, and a `Delivery` enum. -It shows both callback styles: `route` returns -`Result` and receives consumer failures as values, -while `on_message` and `on_attached` (which hands the bus itself to the -consumer) return plain values. It also has an iterator over published -messages (`messages`), an async method (`publish_later`), an optional record -return (`last_message`), and a free function taking a callback -(`route_once`). The bus snapshots its subscribers before calling out and -never holds a lock across a callback. +The oracle for the [value-buffer format](reference/value-buffers.md), built +on shared test vectors. The producer holds a fixed table of values covering +every wire shape: each primitive at its extremes (with NaN, the infinities, +`-0.0`, and subnormals for floats), empty and non-empty strings and bytes +with non-ASCII text and interior NUL bytes, optionals, lists, maps keyed by +strings, integers, and a C-style enum, nested records, C-style and rich +enums, and objects inside buffers. Each vector is a variant of the `Vector` +rich enum, so every language's codec check is one loop: + +1. `vector_count()` returns how many vectors there are. +2. `vector(i)` returns vector `i` for the consumer to decode. An index past + the end fails with `CodecError::OutOfRange { index, count }`. +3. `check_vector(i, v)` takes the decoded value back, re-encoded by the + consumer, and returns `true` only if it's exactly vector `i` (floats keep + their sign, and any NaN matches NaN). +4. `vector_name(i)` and `describe_vector(v)` label and render a vector, so a + failing consumer can say what went wrong. + +A round trip can't catch a decoder and an encoder that make the same mistake, +so each consumer also builds a few vectors from literals and checks those, +and spot-checks decoded fields. The `echo_*` functions return their argument +through the direct ABI families (scalars by value, strings and bytes as +`(ptr, len)` runs, the C-style enum as an `int32_t`), and each consumer pushes +every primitive vector through the matching echo. A `Token` interface with +`sum_holder`, `primary_of`, and `same_primary` checks object identity and +reference counting through buffers. ## kvstore -The kitchen-sink reference: an in-memory key-value store that uses every -feature WeaveFFI supports. The `kv` module has a `Store` interface with a -throwing constructor, methods, and statics; a `KvError` domain; records with -optional, list, map, and bytes fields; `StoreInfo`, a record carrying `Store` -objects; methods that pass objects in every position (`share`, `fork`, -`larger` with `Store?` in and out, `describe`, `open_many` returning -`[Store]`, `total_count`); an `EvictionListener` callback interface; a -throwing iterator (`list_keys`); a cancellable async method (`compact`); a -deprecated method (`legacy_put`); and a nested `kv.stats` module that takes -the parent's `Store`. - -## async-demo - -Async functions in the `tasks` module: `run_task` (an async throwing function -returning a record), `run_batch` (an async list), `run_n_tasks` (for stress -tests of concurrent launches), and `wait`, a cancellable timer. Cancelling -`wait` completes the call with `-5` immediately and drops the pending timer; -`active_callbacks` returns to zero once every task body has finished or been -cancelled, which the lanes assert. The futures need no async runtime, so the -default spawner drives them. - -## codec - -A round-trip oracle for the [value-buffer format](reference/value-buffers.md). -`sample_*` functions return canonical values the consumer checks field by -field (producer encodes, consumer decodes); `verify_*` functions take the -same values back and fail with `CodecError::Mismatch` unless they decode -exactly (consumer encodes, producer decodes); `roundtrip_*` functions echo -scalars, 64-bit edge values, strings, bytes, optionals, maps, rich enums, and -records; and `describe_*` functions render what the producer saw. A `Token` -interface and a `Holder` record cover object tokens inside buffers. +The feature-complete producer: an in-memory key-value store whose API uses +every feature WeaveFFI supports, arranged so a consumer can check each with +concrete assertions. Time is a logical clock per store (`now`, advanced by +`tick`), so TTLs are deterministic. + +| Feature | Where | +|---|---| +| Interface with constructors, methods, and statics | `Store` (`open`, `new`; `put`, `get`, ...; `open_many`, `default_capacity`, ...) | +| Typed errors with payload fields | `KvError`: `KeyNotFound { key }`, `Expired { key, expired_at }`, `StoreFull { capacity }`, `InvalidPath`, `Rejected { key, reason }` | +| Records, optionals, lists, maps | `Entry` (bytes, an optional TTL, tags, metadata), `StoreInfo`, `Stats` (a map keyed by a C-style enum) | +| C-style and rich enums | `EntryKind`; `Change` (`Put`, `Removed`, `Cleared`) | +| Objects in every position | parameters, returns, `Store?` both ways (`larger`), lists (`open_many`, `total_count`), map values (`by_label`), record fields (`StoreInfo`), iterator elements (`partition`), an async result (`open_store`), and a callback's parameter and return (`Policy::route`) | +| Callback interfaces | `Listener` (retained by `subscribe`; void and direct returns), `Policy` (a record return, a throwing method whose typed error reaches the `put` caller, an object parameter and return), `Loader` (string, bytes, and optional-object returns; passed as an optional callback) | +| Callbacks from a producer thread | `compact` notifies listeners from the thread it runs on | +| Lazy iterators | `keys` (strings, throwing), `entries` (records), `partition` (objects) | +| Async and cancellation | `open_store` (an async free function returning an object), `compact` (cancellable; its background pause stops cooperatively, shown by `Store::active_jobs`), `get_many` (an async list), `summarize_all` | +| Nested modules | `kv.stats` uses the parent's `Store` and reports the parent's `KvError` | +| Sibling roots | `report` shares the `Entry` record and has its own `ReportError` | +| Deprecation | `Store::size` (use `count`) | + +A callback's failure surfaces according to its role: a `Listener` that fails +is unsubscribed while the store operation succeeds; a `Policy` or `Loader` +failure fails the call with the callback's code, message, and payload, so a +typed `KvError` a consumer raised reaches the original caller typed. The +store never holds a lock while a callback runs, so callbacks may call back +into it. diff --git a/docs/src/stability.md b/docs/src/stability.md index 6a78d1ad..9a3fc167 100644 --- a/docs/src/stability.md +++ b/docs/src/stability.md @@ -8,14 +8,13 @@ migrate to the current release. ## Three version numbers -- **The crate version** (`0.23.x`) is shared by every published crate - (`weaveffi`, `weaveffi-abi`, `weaveffi-macros`, `weaveffi-model`, - `weaveffi-gen`, `weaveffi-cli`) and by the CLI. Releases are cut from - Conventional Commits by release-plz. -- **The IDL schema version** (`0.10.0`) is the `version:` an IDL document +- **The crate version** (`0.24.x`) is shared by every published crate + (`weaveffi`, `weaveffi-macros`, `weaveffi-model`, `weaveffi-cli`) and by + the CLI. Releases are cut from Conventional Commits by release-plz. +- **The IDL schema version** (`0.11.0`) is the `version:` an IDL document declares. Pre-1.0, only the current version is accepted; a document declaring any other is rejected with `UnsupportedSchemaVersion`. -- **The C ABI revision** (`3`) is what `{prefix}_abi_version()` returns. It +- **The C ABI revision** (`4`) is what `{prefix}_abi_version()` returns. It changes only when the runtime surface or a calling convention changes incompatibly, independently of the other two. @@ -52,103 +51,213 @@ the definition that triggers it. Pin the CLI version and commit the generated output, so an upgrade is an explicit, reviewable change. Then gate CI on `weaveffi diff --check`, which regenerates in memory and compares with the committed directory without -writing anything or running hooks: +writing anything: ```yaml - name: Bindings are up to date run: | - cargo install weaveffi-cli --locked --version =0.23.0 + cargo install weaveffi-cli --locked --version =0.24.0 weaveffi validate --warn weaveffi diff --check ``` `diff --check` exits `0` when the output matches, `2` when files would -change, and `3` when files would be added or removed. +change, and `3` when files would be added or removed. For a Rust producer, +both commands build the crate first (see [Library Mode](guides/extract.md)), +so the job needs the Rust toolchain; pass `--library ` to reuse a +library an earlier step built. -## Migrating to schema 0.10 and ABI 3 +## Migrating to schema 0.11 and ABI 4 -This release renames every C symbol after the library and changes how -strings cross the boundary, so every consumer must be regenerated and every -hand-written producer or consumer updated. There are no compatibility shims. +This release makes names global, replaces the per-module checksums with +contract tables, lets callback interfaces return any value and report typed +errors, and reads a Rust producer's API from its built library instead of +its source. Every consumer must be regenerated, and every hand-written +producer or consumer updated. There are no compatibility shims. ### IDL -- Set `version: "0.10.0"`. -- Map keys can't be floats (`f32`, `f64`); key by an integer, `bool`, - `string`, or C-style enum. -- A qualified type reference (`a.b.T`) must spell the declaring module's - path exactly, and an unknown type name is always an error. -- `cancellable: true` on a synchronous callable is an error - (`CancellableNotAsync`). -- Every C identifier is checked across the whole API, including module path - flattening (`m.x_y` versus `m.x.y`) and async completion types - (`foo_callback`). `SymbolCollision` replaces `AbiSymbolCollision`. +Names are global: a declaration is identified by its bare name alone, and +modules only group declarations and namespace C symbols. + +- Set `version: "0.11.0"`. +- Type names (records, enums, interfaces, callback interfaces, and error + domains), free-function names, and error-code names must each be unique + across the whole API. Two modules can no longer both declare a function + `open`; rename one (`DuplicateFunctionName`). A free function also can't + share its name with an error domain. +- Qualified type references are removed: write `Contact`, not + `contacts.Contact` (`QualifiedTypeRef`). +- `usize`, `isize`, `u128`, `i128`, and `char` are reported as + `UnsupportedPrimitive`, with a suggested replacement. +- `DuplicateStructName`, `DuplicateEnumName`, `DuplicateInterfaceName`, + `DuplicateCallbackInterfaceName`, and `DuplicateErrorName` are gone: + `DuplicateTypeName` and `DuplicateErrorCodeName` report those once. +- A callback-interface method may return any type except an iterator or a + callback interface, and may set `throws: true` when an error domain is in + scope for its module. +- A callback-interface parameter may be optional (`Cb?`), as a top-level + parameter of a function or interface member. ### C ABI -- `{prefix}_abi_version()` returns `3`. -- Every symbol, type, constant, and macro carries the library's prefix: - `weaveffi_error` is `{prefix}_error`, `weaveffi_free_bytes` is - `{prefix}_free_bytes`, `WEAVEFFI_API` is `{PREFIX}_API`, and - `weaveffi_kv_Store_open` is `{prefix}_kv_Store_open`. The header is - `{library}.h` instead of `weaveffi.h`, and no `weaveffi_*` aliases remain. -- Strings cross as UTF-8 `(ptr, len)` everywhere: parameters, returns, - callback arguments, async results, and iterator elements. Returned strings - are freed with `{prefix}_free_bytes`; `free_string` is gone. -- Async launchers are named after the function, without the `_async` - suffix. -- New runtime code `-5` (cancelled). Cancel tokens are reference counted: - the launcher takes its own reference, cancelling completes the call with - `-5`, and the consumer may destroy its token at any time. -- Each top-level module exports `{prefix}_{module}_checksum()` and the - header defines `{PREFIX}_{MODULE}_CHECKSUM`. Generated consumers check - both the revision and every checksum at load time. -- New runtime symbol `{prefix}_debug_live`, which a hand-written producer - may implement as returning `0`. - -### Configuration and CLI - -- The C prefix and library name come from the library's identity. - `[global] c_prefix` and every per-target `prefix`, `c_prefix`, and - `input_basename` key are removed. For an IDL, set `[package] c_prefix` or - `[package] library`; for a Rust producer they're the crate's library name - and can't be set. -- Generated packages, modules, namespaces, and error types are named from - the identity; nothing is named after WeaveFFI. Loaders look for - `{library}` and honor `{PREFIX}_LIBRARY` instead of `WEAVEFFI_LIBRARY`. -- New `[project]` table (`input`, `out`, `targets`), so `weaveffi generate` - runs without arguments, and new `weaveffi init`. -- Cache records moved to `{out}/.weaveffi-cache/{target}.json`; generation - rewrites only changed files and deletes files it previously wrote but no - longer produces. -- `weaveffi diff` accepts `--target` and never runs hooks. -- `weaveffi extract --warn` is now `--lenient`. -- `weaveffi package` packages for the host platform unless `--platforms` is - given (it used to default to every platform). +- `{prefix}_abi_version()` returns `4`. +- Contract tables replace checksums. `{prefix}_{module}_checksum()` and + `{PREFIX}_{MODULE}_CHECKSUM` are gone; each top-level module exports + `const {prefix}_contract_entry* {prefix}_{module}_contract(size_t* out_len)`, + one `{id, hash}` entry per declaration, sorted by id. The header defines + `{PREFIX}_{MODULE}_CONTRACT`, `{PREFIX}_{MODULE}_CONTRACT_LEN`, and + `{prefix}_{module}_contract_check()`, which returns the id of the first + entry the library lacks or has with a different hash, or `0`. Adding a + declaration no longer breaks deployed bindings. A hand-written producer + returns the header's table (see + [Producers in other languages](reference/abi.md#producers-in-other-languages)). +- Every callback vtable starts with a header, `uint32_t size; uint32_t + flags; void (*free)(void*);`, then the methods in declaration order (`free` + used to come last). Set `size` to `sizeof` the vtable and `flags` to `0`. + A producer rejects a vtable smaller than its own with `-3`. +- Callback methods may return strings, bytes, buffers, and objects. A + string, bytes, or buffer return adds trailing `uint8_t** out_ptr, size_t* + out_len` slots holding a run the consumer allocates with `{prefix}_alloc`; + an object return is one strong reference. The producer adopts both. +- A callback method declared `throws` may report a positive code of its + domain, with the code's fields set by the new + `{prefix}_error_set_payload(err, ptr, len)` (which copies). Every other + callback failure still reaches the producer as `-4`. +- An optional callback parameter passes a null vtable for none. +- `{prefix}_alloc(size_t len)` is exported on every target (it was `wasm32` + only) and returns a zero-filled run, or null for `0`. `{prefix}_dealloc` is + removed: `{prefix}_free_bytes` frees both returned runs and `alloc` runs, + and is a no-op for length `0`. +- `{prefix}_debug_live(-1)` returns `1` when the producer counts and `0` + when it doesn't. +- An async call the executor can't start completes with `-1` and a message. +- A value buffer whose map repeats a key, and an iterator `_next` that + arrives while the same iterator is being advanced, fail with `-3`. ### Rust producers -- Symbols start with the crate's library name instead of `weaveffi`; a crate - `kvstore` exports `kvstore_kv_Store_open`. -- The runtime is exported with the `weaveffi::export_runtime!()` proc macro; - `weaveffi_abi::export_runtime!` is gone. -- A `#[weaveffi::error]` enum must implement `Display`, whose output is the - runtime error message. Doc comments remain the documented default. -- A callback-interface method may return `Result` - to receive consumer failures as values. A plain-return method's failure - is deferred only while a WeaveFFI call is active on the thread. -- A reference to a C-style enum, interface, or callback interface declared - in a different `#[weaveffi::module]` root is a compile error; nest the - modules under one root. -- A cancellable function no longer has to notice cancellation: the runtime - drops its future and completes with `-5`. -- The model and generator crates were consolidated: `weaveffi-ir`, - `weaveffi-core`, and `weaveffi-bridge` are now `weaveffi-model`, and the - eleven `weaveffi-gen-*` crates are now `weaveffi-gen`. The runtime's - Rust error type is `weaveffi_abi::FfiError`, and its string helpers work - on `(ptr, len)` runs (`lift_str`, `lower_string`). +- The crates are consolidated. `weaveffi-abi` is now the `weaveffi::abi` + module, and `weaveffi-gen` moved into `weaveffi-cli`, which is a library + as well as the `weaveffi` binary. Depend on `weaveffi` alone. +- Every callback-interface method must return `Result`; a plain `T` is a compile error. `ForeignError` + gains a `payload` field and `domain::()`, which decodes a declared + domain error. Mark a method `#[weaveffi::throws]` to let its consumer + report the domain in scope. +- The deferred-error API is removed: `ThunkScope`, `defer_foreign_error`, + `raise_foreign_error`, and `take_foreign_error` no longer exist, and a + callback failure never unwinds or waits for the thunk. To propagate one, + return `Result` from the exported function (which then + needs an error domain in scope) and use `?`. +- Accept an optional callback as `Option>`. +- `weaveffi::abi::decode_value` and `BufferValue::read_value` are `unsafe + fn`, because decoding an object token adopts a reference. A reader built + with `BufferReader::token_free` refuses tokens instead. +- C-style enums implement the runtime trait `weaveffi::abi::CEnum`, which + replaces the generated `__weaveffi_from_i32` and `__weaveffi_to_i32` + inherent methods. +- The macro validates its module tree with the CLI's validator and reports + each error on the offending item. A type from another module tree that + isn't a record or rich enum fails with "is not a WeaveFFI record or rich + enum". +- `#[cfg]` on an item applies to its thunks, metadata, and contract entries, + and non-generic type aliases in the tree (`pub type Id = u64;`) are + substituted. A `#[cfg]` on a member (a field, variant, or method) and an + out-of-line submodule (`mod x;`) are compile errors. +- The default async executor is a fixed pool of worker threads instead of a + thread per call. The `weaveffi` crate gains a `tokio` feature that runs + futures on the current Tokio runtime, or on one it creates; + `weaveffi::set_spawner` still overrides both. + +### CLI and configuration + +- **Library mode.** The CLI no longer reads Rust source. The macro embeds + the API in the library (one exported `{PREFIX}_META_{HASH}` static per + declaration, or the `weaveffi_meta` custom section on `wasm32`), and + `generate`, `validate`, `diff`, and `extract` build the crate with `cargo + build --lib` and read it from there, so generated bindings follow + `#[cfg]`. `--library ` reads an existing build instead, and + `--release` builds in release mode. See [Library Mode](guides/extract.md). +- **The input is the crate.** Set `[project] input = "."` (instead of + `"src/lib.rs"`), and pass a crate directory or its `Cargo.toml` on the + command line instead of a `.rs` file. +- **`weaveffi extract`** prints the IDL a built library embeds (`--library`, + or the project's crate) instead of parsing a source file. Its `--lenient` + flag is gone: what it prints always validates. +- **`generate --force` is gone,** along with the input-hash freshness check + it bypassed. Every run renders every selected target and rewrites only the + files whose contents changed. +- **`[global]` is gone,** with its `pre_generate` and `post_generate` hooks + and every `strip_module_prefix` key (per target, too). Run formatters as a + separate step. Targets that flatten module functions into one namespace use + the bare function name, which global names keep collision-free. +- **One `name` key per target.** `[generators.cpp] namespace`, + `swift.module_name`, `kotlin.package`, `node.package_name`, + `wasm.package_name`, `python.package_name`, `dotnet.namespace`, + `dart.package_name`, `go.module_path`, and `ruby.gem_name` all become + `name`. Ruby's `module_name` and Python's `import_name` stay. +- `[project] out` defaults to `bindings` instead of `generated`, and + generated files' headers no longer name the input file. +- `weaveffi dev` is new: it builds the debug library, generates, copies the + library into the generated Python, Node.js, and Ruby packages, and prints + how to point the other targets at it. +- `weaveffi build` is new, and `weaveffi package` writes installable + artifacts (wheels, npm tarballs, gems, an `XCFramework` SwiftPM package, a + `.nupkg`) instead of source trees. `package --build ` is gone: + `package` builds the project's producer itself, or reads + `--binaries `. The dist directory is `-o`, else `[package] dist`, + else `dist`. See [Packaging](guides/packaging.md). +- The `weaveffi-cli` library exposes `project::Project`, which locates a + project and generates it, for use from a `build.rs` (with an IDL input or + a previously built library). + +### Generated bindings + +Regenerate every target. Beyond the ABI changes, a few generated APIs +changed shape: + +- Every callback-interface method in every language may now return a value + and, when it's declared `throws`, raise the module's domain error. +- Kotlin maps `u8`, `u16`, `u32`, and `u64` to `UByte`, `UShort`, `UInt`, + and `ULong` instead of the signed types. +- Python ships one fully annotated module plus `py.typed`; the separate + `.pyi` stub is gone. +- The Node.js package loads a prebuilt addon when one matches the platform + and compiles it only as a fallback. +- Load failures name the declaration that's missing or changed, instead of + reporting a checksum mismatch. ### Samples -The `inventory` and `shapes` samples were removed; `contacts` covers nested -and sibling modules, and the `shapes` test fixture covers rich enums. +The six samples are now three: `calculator` (minimal), `codec` (the +value-buffer oracle, rebuilt around shared test vectors), and `kvstore` +(every feature). `contacts`, `events`, and `async-demo` were folded into +`kvstore` and removed, and every language's conformance consumers follow +suit: three lanes per language. See [Samples](samples.md). + +## Migrating to schema 0.10 and ABI 3 + +WeaveFFI 0.24.0 shipped schema 0.10 and ABI revision 3. Coming from an +older release, apply these changes first, then the ones above: + +- Every symbol, type, constant, and macro carries the library's prefix + (`{prefix}_error`, `{prefix}_free_bytes`, `{PREFIX}_API`, + `{prefix}_kv_Store_open`), the header is `{library}.h`, and generated + packages and loaders are named after the library's identity, never after + WeaveFFI. `[global] c_prefix` and every per-target prefix key are gone; an + IDL sets `[package] c_prefix` or `library`, and a Rust producer's prefix is + its crate's library name. +- Strings cross as UTF-8 `(ptr, len)` runs everywhere, and returned strings + are freed with `{prefix}_free_bytes`. +- Async launchers drop the `_async` suffix, cancel tokens are reference + counted, and runtime code `-5` means cancelled. +- Map keys can't be floats, and `cancellable: true` on a synchronous + callable is an error. +- The runtime is exported with `weaveffi::export_runtime!()`, and a + `#[weaveffi::error]` enum must implement `Display`. +- The `[project]` table and `weaveffi init` are new, and generation rewrites + only changed files and deletes stale ones. + +The [changelog](https://github.com/weavefoundry/weaveffi/blob/main/CHANGELOG.md) +lists every release. diff --git a/release-plz.toml b/release-plz.toml index 55f140de..3bed3881 100644 --- a/release-plz.toml +++ b/release-plz.toml @@ -39,10 +39,6 @@ commit_parsers = [ name = "weaveffi-model" version_group = "weaveffi" -[[package]] -name = "weaveffi-abi" -version_group = "weaveffi" - [[package]] name = "weaveffi-macros" version_group = "weaveffi" @@ -51,16 +47,12 @@ version_group = "weaveffi" name = "weaveffi" version_group = "weaveffi" -[[package]] -name = "weaveffi-gen" -version_group = "weaveffi" - [[package]] name = "weaveffi-cli" version_group = "weaveffi" changelog_update = true changelog_path = "CHANGELOG.md" -changelog_include = ["weaveffi-model", "weaveffi-abi", "weaveffi-macros", "weaveffi", "weaveffi-gen"] +changelog_include = ["weaveffi-model", "weaveffi-macros", "weaveffi"] git_tag_enable = true git_tag_name = "v{{ version }}" git_release_enable = true diff --git a/samples/async-demo/Cargo.toml b/samples/async-demo/Cargo.toml deleted file mode 100644 index 06d72635..00000000 --- a/samples/async-demo/Cargo.toml +++ /dev/null @@ -1,14 +0,0 @@ -[package] -name = "async-demo" -version = "0.1.0" -edition.workspace = true -publish = false - -[lib] -crate-type = ["cdylib", "rlib"] - -[dependencies] -weaveffi = { path = "../../crates/weaveffi", features = ["leak-check"] } - -[lints] -workspace = true diff --git a/samples/async-demo/src/lib.rs b/samples/async-demo/src/lib.rs deleted file mode 100644 index 73a3fcc6..00000000 --- a/samples/async-demo/src/lib.rs +++ /dev/null @@ -1,449 +0,0 @@ -//! Async-demo sample cdylib used to exercise WeaveFFI's `async: true` function -//! code generation across all targets. -//! -//! The producer writes plain `async fn`s; the `#[weaveffi::module]` expansion -//! emits a launcher for each (running the future to completion on a worker -//! thread, then firing the host completion callback exactly once). A small -//! RAII `ActiveGuard` keeps `active_callbacks` honest: it counts task bodies -//! that are in flight and returns to zero once every spawned body has -//! completed or been cancelled. -//! -//! `wait` is cancellable for real: it waits on a timer until its timeout, -//! and cancelling its token completes the call with the cancelled code (`-5`) -//! right away, dropping the pending timer. - -/// Async/await and cancellation demo across WeaveFFI's async-capable targets. -#[weaveffi::module] -pub mod tasks { - use std::future::Future; - use std::pin::Pin; - use std::sync::atomic::{AtomicI64, Ordering}; - use std::task::{Context, Poll}; - - /// The task module's error domain. - #[weaveffi::error] - #[derive(Debug)] - pub enum TaskError { - /// task name must not be empty - InvalidName = 1, - } - - impl std::fmt::Display for TaskError { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str("task name must not be empty") - } - } - - static NEXT_TASK_ID: AtomicI64 = AtomicI64::new(1); - static ACTIVE_CALLBACKS: AtomicI64 = AtomicI64::new(0); - - /// RAII counter for in-flight async task bodies: increments on construction - /// and decrements on drop. Because the `#[weaveffi::module]` expansion drops - /// the future (and thus this guard) just before invoking the completion - /// callback, `active_callbacks` is back to zero by the time a caller - /// observes the callback. - struct ActiveGuard; - - impl ActiveGuard { - fn new() -> Self { - ACTIVE_CALLBACKS.fetch_add(1, Ordering::SeqCst); - ActiveGuard - } - } - - impl Drop for ActiveGuard { - fn drop(&mut self) { - ACTIVE_CALLBACKS.fetch_sub(1, Ordering::SeqCst); - } - } - - fn next_id() -> i64 { - NEXT_TASK_ID.fetch_add(1, Ordering::Relaxed) - } - - /// The by-value result an async task completes with. - #[weaveffi::record] - #[derive(Debug, Clone)] - pub struct TaskResult { - /// The id assigned to the completed task. - pub id: i64, - /// A human-readable completion message. - pub value: String, - /// Whether the task succeeded. - pub success: bool, - } - - /// Run a single named task, completing with its `TaskResult`. An empty - /// name is rejected with [`TaskError::InvalidName`]. - #[weaveffi::export] - pub async fn run_task(name: String) -> Result { - let _guard = ActiveGuard::new(); - if name.is_empty() { - return Err(TaskError::InvalidName); - } - Ok(TaskResult { - id: next_id(), - value: format!("completed: {name}"), - success: true, - }) - } - - /// Run a batch of named tasks, completing with one `TaskResult` per name. - #[weaveffi::export] - pub async fn run_batch(names: Vec) -> Vec { - let _guard = ActiveGuard::new(); - names - .into_iter() - .map(|name| TaskResult { - id: next_id(), - value: format!("completed: {name}"), - success: true, - }) - .collect() - } - - /// A timer future that needs no async runtime: the first poll starts a - /// thread that sleeps until the deadline and then wakes the task. - struct Delay { - #[cfg(not(target_arch = "wasm32"))] - deadline: std::time::Instant, - waker: Option>>, - } - - impl Delay { - fn new(ms: u64) -> Self { - // `wasm32-unknown-unknown` has no clock (`Instant::now` traps), so - // there the delay completes on its first poll. - #[cfg(target_arch = "wasm32")] - let _ = ms; - Self { - #[cfg(not(target_arch = "wasm32"))] - deadline: std::time::Instant::now() + std::time::Duration::from_millis(ms), - waker: None, - } - } - } - - impl Future for Delay { - type Output = (); - - #[cfg(target_arch = "wasm32")] - fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> { - // `wasm32-unknown-unknown` has neither threads nor a clock to - // sleep on, so the timeout elapses at once there. - let _ = &self.waker; - Poll::Ready(()) - } - - #[cfg(not(target_arch = "wasm32"))] - fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> { - if std::time::Instant::now() >= self.deadline { - return Poll::Ready(()); - } - match &self.waker { - // Keep the waker current in case the executor moved the task. - Some(slot) => cx.waker().clone_into(&mut slot.lock().unwrap()), - None => { - let slot = std::sync::Arc::new(std::sync::Mutex::new(cx.waker().clone())); - let deadline = self.deadline; - let wake = std::sync::Arc::clone(&slot); - std::thread::spawn(move || { - std::thread::sleep( - deadline.saturating_duration_since(std::time::Instant::now()), - ); - wake.lock().unwrap().wake_by_ref(); - }); - self.waker = Some(slot); - } - } - Poll::Pending - } - } - - /// Wait `timeout_ms` milliseconds, then complete with the milliseconds - /// waited. Cancelling the token first completes the call with the - /// cancelled code instead: the runtime drops this future, timer and all. - #[weaveffi::export] - #[weaveffi::cancellable] - pub async fn wait(timeout_ms: i64, cancel: weaveffi::CancelToken) -> i64 { - let _guard = ActiveGuard::new(); - let _ = cancel; - let ms = u64::try_from(timeout_ms).unwrap_or(0); - Delay::new(ms).await; - timeout_ms.max(0) - } - - /// Complete immediately with `n`. Drives the async stress examples, which - /// verify the per-target wrapper pins the caller's context and callback for - /// the duration of the call. - #[weaveffi::export] - pub async fn run_n_tasks(n: i32) -> i32 { - let _guard = ActiveGuard::new(); - n - } - - /// The number of async task bodies currently in flight; returns to zero - /// once every outstanding task has completed. - #[weaveffi::export] - pub fn active_callbacks() -> i64 { - ACTIVE_CALLBACKS.load(Ordering::SeqCst) - } -} - -weaveffi::export_runtime!(); - -#[cfg(test)] -#[allow(unsafe_code)] -mod tests { - use crate::tasks::*; - use std::os::raw::c_void; - use std::sync::mpsc; - use std::time::{Duration, Instant}; - use weaveffi::abi::{self, FfiError}; - - type TaskCbMsg = (i32, Option); - type BatchCbMsg = (bool, Vec); - - /// The error code of a consumer-owned async error, which is released. - fn take_code(err: *mut FfiError) -> i32 { - if err.is_null() { - return 0; - } - let code = unsafe { (*err).code }; - unsafe { crate::async_demo_error_free(err) }; - code - } - - /// A buffered async result is an owned `(ptr, len)` value buffer: decode - /// it, then release it with `{prefix}_free_bytes`. - fn take_buffer(ptr: *const u8, len: usize) -> Option { - if ptr.is_null() { - return None; - } - let bytes = unsafe { std::slice::from_raw_parts(ptr, len) }; - let value = abi::decode_value::(bytes).expect("well-formed value buffer"); - unsafe { crate::async_demo_free_bytes(ptr.cast_mut(), len) }; - Some(value) - } - - extern "C" fn task_callback( - context: *mut c_void, - err: *mut FfiError, - result_ptr: *const u8, - result_len: usize, - ) { - let tx = unsafe { &*(context as *const mpsc::Sender) }; - let _ = tx.send((take_code(err), take_buffer(result_ptr, result_len))); - } - - extern "C" fn batch_callback( - context: *mut c_void, - err: *mut FfiError, - results_ptr: *const u8, - results_len: usize, - ) { - let tx = unsafe { &*(context as *const mpsc::Sender) }; - let had_error = take_code(err) != 0; - let results = take_buffer(results_ptr, results_len).unwrap_or_default(); - let _ = tx.send((had_error, results)); - } - - extern "C" fn i32_callback(context: *mut c_void, err: *mut FfiError, result: i32) { - let tx = unsafe { &*(context as *const mpsc::Sender<(i32, i32)>) }; - let _ = tx.send((take_code(err), result)); - } - - extern "C" fn i64_callback(context: *mut c_void, err: *mut FfiError, result: i64) { - let tx = unsafe { &*(context as *const mpsc::Sender<(i32, i64)>) }; - let _ = tx.send((take_code(err), result)); - } - - /// Intentionally leak a callback-context box. - /// - /// The completion callback runs on a worker thread, so a worker may still - /// be inside the callback's `send` when the test's `recv` returns (the - /// receiver unblocks as soon as the message is queued, before `send` - /// finishes). Reclaiming the box here would free the `Sender` out from - /// under that in-flight `send`, so the test leaks the context instead; - /// the OS reclaims the memory at process exit. - fn leak_ctx(ptr: *mut T) { - std::mem::forget(unsafe { Box::from_raw(ptr) }); - } - - fn run_task(name: &str) -> TaskCbMsg { - let (tx, rx) = mpsc::channel::(); - let tx_ptr = Box::into_raw(Box::new(tx)); - unsafe { - async_demo_tasks_run_task(name.as_ptr(), name.len(), task_callback, tx_ptr.cast()) - }; - let out = rx.recv_timeout(Duration::from_secs(5)).unwrap(); - leak_ctx(tx_ptr); - out - } - - #[test] - fn run_task_calls_callback() { - let (code, result) = run_task("test-task"); - assert_eq!(code, 0); - let r = result.expect("success path passes a result buffer"); - assert!(r.id > 0); - assert!(r.success); - assert!(r.value.contains("test-task")); - } - - #[test] - fn run_task_empty_name_reports_invalid_name() { - let (code, result) = run_task(""); - assert_eq!(code, 1, "TaskError::InvalidName's declared code"); - assert!(result.is_none(), "error path passes a null result buffer"); - } - - #[test] - fn run_task_invalid_name_reports_marshal_error_through_the_callback() { - let (tx, rx) = mpsc::channel::(); - let tx_ptr = Box::into_raw(Box::new(tx)); - // The launcher has no out_err slot, so an argument that fails to lift - // (a null pointer with a length) is reported through the completion - // callback with the reserved marshalling code. - unsafe { async_demo_tasks_run_task(std::ptr::null(), 3, task_callback, tx_ptr.cast()) }; - let (code, result) = rx.recv_timeout(Duration::from_secs(5)).unwrap(); - leak_ctx(tx_ptr); - assert_eq!(code, abi::MARSHAL_ERROR_CODE); - assert!(result.is_none()); - } - - fn run_batch(names: &[&str]) -> BatchCbMsg { - let (tx, rx) = mpsc::channel::(); - let tx_ptr = Box::into_raw(Box::new(tx)); - // The list-of-strings parameter is buffered: the launcher copies the - // bytes before returning, so the local buffer only needs to outlive - // the call. - let names = abi::encode_value(&names.iter().map(|n| n.to_string()).collect::>()); - unsafe { - async_demo_tasks_run_batch(names.as_ptr(), names.len(), batch_callback, tx_ptr.cast()) - }; - let out = rx.recv_timeout(Duration::from_secs(10)).unwrap(); - leak_ctx(tx_ptr); - out - } - - #[test] - fn run_batch_processes_sequentially() { - let (had_error, results) = run_batch(&["task-a", "task-b", "task-c"]); - assert!(!had_error); - assert_eq!(results.len(), 3); - assert!(results.iter().all(|r| r.id > 0 && r.success)); - assert!(results[0].value.contains("task-a")); - assert!(results[2].value.contains("task-c")); - } - - #[test] - fn run_batch_empty_names() { - let (had_error, results) = run_batch(&[]); - assert!(!had_error); - assert!(results.is_empty()); - } - - #[test] - fn task_result_buffer_round_trip() { - for result in [ - TaskResult { - id: 42, - value: "hello".to_string(), - success: true, - }, - TaskResult { - id: 1, - value: "fail".to_string(), - success: false, - }, - ] { - let back = abi::decode_value::(&abi::encode_value(&result)).unwrap(); - assert_eq!(back.id, result.id); - assert_eq!(back.value, result.value); - assert_eq!(back.success, result.success); - } - } - - #[test] - fn run_n_tasks_invokes_callback_with_n() { - let (tx, rx) = mpsc::channel::<(i32, i32)>(); - let tx_ptr = Box::into_raw(Box::new(tx)); - unsafe { async_demo_tasks_run_n_tasks(7, i32_callback, tx_ptr.cast()) }; - assert_eq!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), (0, 7)); - leak_ctx(tx_ptr); - } - - fn launch_wait(timeout_ms: i64, token: *mut abi::FfiCancelToken) -> mpsc::Receiver<(i32, i64)> { - let (tx, rx) = mpsc::channel::<(i32, i64)>(); - let tx_ptr = Box::into_raw(Box::new(tx)); - unsafe { async_demo_tasks_wait(timeout_ms, token, i64_callback, tx_ptr.cast()) }; - leak_ctx(tx_ptr); - rx - } - - #[test] - fn wait_completes_after_its_timeout() { - let token = crate::async_demo_cancel_token_create(); - let rx = launch_wait(20, token); - assert_eq!(rx.recv_timeout(Duration::from_secs(5)).unwrap(), (0, 20)); - unsafe { crate::async_demo_cancel_token_destroy(token) }; - } - - #[test] - fn cancelling_wait_completes_with_the_cancelled_code() { - let token = crate::async_demo_cancel_token_create(); - let start = Instant::now(); - let rx = launch_wait(60_000, token); - std::thread::sleep(Duration::from_millis(20)); - unsafe { - crate::async_demo_cancel_token_cancel(token); - crate::async_demo_cancel_token_destroy(token); - } - let (code, result) = rx.recv_timeout(Duration::from_secs(5)).unwrap(); - assert_eq!(code, abi::CANCELLED_ERROR_CODE); - assert_eq!(result, 0); - assert!( - start.elapsed() < Duration::from_secs(5), - "didn't wait for the timeout" - ); - assert!( - rx.recv_timeout(Duration::from_millis(100)).is_err(), - "completes once" - ); - } - - #[test] - fn a_token_cancelled_before_launch_never_runs_the_body() { - let token = crate::async_demo_cancel_token_create(); - unsafe { crate::async_demo_cancel_token_cancel(token) }; - let rx = launch_wait(60_000, token); - unsafe { crate::async_demo_cancel_token_destroy(token) }; - assert_eq!( - rx.recv_timeout(Duration::from_secs(5)).unwrap(), - (abi::CANCELLED_ERROR_CODE, 0) - ); - } - - #[test] - fn active_callbacks_returns_to_zero() { - let mut err = FfiError::default(); - let (tx, rx) = mpsc::channel::<(i32, i32)>(); - let tx_ptr = Box::into_raw(Box::new(tx)); - for i in 0..16 { - unsafe { async_demo_tasks_run_n_tasks(i, i32_callback, tx_ptr.cast()) }; - } - for _ in 0..16 { - rx.recv_timeout(Duration::from_secs(5)).unwrap(); - } - leak_ctx(tx_ptr); - - for _ in 0..50 { - if unsafe { async_demo_tasks_active_callbacks(&mut err) } == 0 { - break; - } - std::thread::sleep(Duration::from_millis(20)); - } - assert_eq!(unsafe { async_demo_tasks_active_callbacks(&mut err) }, 0); - } -} diff --git a/samples/async-demo/weaveffi.toml b/samples/async-demo/weaveffi.toml deleted file mode 100644 index 3f31d3c6..00000000 --- a/samples/async-demo/weaveffi.toml +++ /dev/null @@ -1,8 +0,0 @@ -[package] -name = "async-demo" -version = "1.0.0" -description = "Async/await and cancellation demo across WeaveFFI's async-capable targets." -license = "MIT" -authors = ["WeaveFoundry "] -homepage = "https://github.com/weavefoundry/weaveffi" -repository = "https://github.com/weavefoundry/weaveffi" diff --git a/samples/calculator/src/lib.rs b/samples/calculator/src/lib.rs index 3e287444..8f5cce99 100644 --- a/samples/calculator/src/lib.rs +++ b/samples/calculator/src/lib.rs @@ -1,15 +1,17 @@ -//! Calculator sample cdylib: the canonical minimal WeaveFFI producer. +//! Calculator sample cdylib: the smallest useful WeaveFFI producer. //! -//! Every exported function is plain, safe Rust. The `#[weaveffi::module]` -//! attribute reads the annotated items and generates the `#[no_mangle] -//! extern "C"` thunks that the stable C ABI (and every generated language -//! binding) calls, marshalling arguments and results through the runtime so -//! this file contains no `unsafe` glue. The `CalcError` domain shows the -//! smallest possible typed error surface: one code, one throwing function. +//! Three plain, safe Rust functions: `add`, `divide` (which fails with the +//! typed `CalcError` domain on a zero divisor), and `greet` (a string in and +//! a string out). The `#[weaveffi::module]` attribute generates the +//! `#[no_mangle] extern "C"` thunks that the stable C ABI (and every +//! generated binding) calls, so this file contains no `unsafe` glue. +//! +//! This is the example the README and the getting-started guide walk +//! through; `kvstore` is the feature-complete sample. #[weaveffi::module] pub mod calculator { - /// The calculator's error domain: the codes its throwing functions report. + /// The calculator's error domain. #[weaveffi::error] #[derive(Debug)] pub enum CalcError { @@ -19,37 +21,30 @@ pub mod calculator { impl std::fmt::Display for CalcError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { - Self::DivisionByZero => f.write_str("division by zero"), - } + f.write_str("division by zero") } } - /// Add two integers. + /// Add two integers, wrapping on overflow. #[weaveffi::export] pub fn add(a: i32, b: i32) -> i32 { - a + b - } - - /// Multiply two integers. - #[weaveffi::export] - pub fn mul(a: i32, b: i32) -> i32 { - a * b + a.wrapping_add(b) } - /// Divide two integers, failing on a zero divisor. + /// Divide `a` by `b`, rounding toward zero. Fails with + /// `DivisionByZero` when `b` is zero. #[weaveffi::export] - pub fn div(a: i32, b: i32) -> Result { + pub fn divide(a: i32, b: i32) -> Result { if b == 0 { return Err(CalcError::DivisionByZero); } - Ok(a / b) + Ok(a.wrapping_div(b)) } - /// Echo a string back to the caller. + /// Greet `name`: `"Hello, {name}!"`. #[weaveffi::export] - pub fn echo(s: String) -> String { - s + pub fn greet(name: String) -> String { + format!("Hello, {name}!") } } @@ -66,44 +61,56 @@ mod tests { } #[test] - fn add_and_mul() { + fn add_wraps() { let mut err = FfiError::default(); assert_eq!(unsafe { calculator_calculator_add(2, 40, &mut err) }, 42); assert_eq!(err.code, 0); - assert_eq!(unsafe { calculator_calculator_mul(6, 7, &mut err) }, 42); - assert_eq!(err.code, 0); + assert_eq!( + unsafe { calculator_calculator_add(i32::MAX, 1, &mut err) }, + i32::MIN + ); } #[test] - fn div_ok_path() { + fn divide_ok_path() { let mut err = FfiError::default(); - assert_eq!(unsafe { calculator_calculator_div(10, 2, &mut err) }, 5); + assert_eq!(unsafe { calculator_calculator_divide(-7, 2, &mut err) }, -3); assert_eq!(err.code, 0); + assert_eq!( + unsafe { calculator_calculator_divide(i32::MIN, -1, &mut err) }, + i32::MIN + ); } #[test] - fn div_by_zero_reports_domain_code() { + fn divide_by_zero_reports_domain_code() { let mut err = FfiError::default(); - let r = unsafe { calculator_calculator_div(1, 0, &mut err) }; + let r = unsafe { calculator_calculator_divide(1, 0, &mut err) }; assert_eq!(r, 0, "error path returns the zero sentinel"); assert_eq!(err.code, 1, "CalcError::DivisionByZero's declared code"); assert_eq!(message(&err), "division by zero"); } #[test] - fn echo_round_trips_utf8_with_interior_nul() { - let text = "h\u{e9}llo\0world"; + fn greet_round_trips_utf8() { + let name = "Wörld \u{1F980}"; let mut err = FfiError::default(); let mut len = 0usize; let ptr = - unsafe { calculator_calculator_echo(text.as_ptr(), text.len(), &mut len, &mut err) }; + unsafe { calculator_calculator_greet(name.as_ptr(), name.len(), &mut len, &mut err) }; assert_eq!(err.code, 0); - assert_eq!(unsafe { abi::lift_str(ptr, len) }, Some(text)); + assert_eq!( + unsafe { abi::lift_str(ptr, len) }, + Some("Hello, Wörld \u{1F980}!") + ); unsafe { abi::free_bytes(ptr.cast_mut(), len) }; } #[test] - fn checksum_is_exported() { - assert_ne!(calculator_calculator_checksum(), 0); + fn contract_table_is_exported() { + let mut len = 0usize; + let table = unsafe { calculator_calculator_contract(&mut len) }; + assert!(!table.is_null()); + assert_eq!(len, 4, "three functions and the error domain"); } } diff --git a/samples/calculator/weaveffi.toml b/samples/calculator/weaveffi.toml index c570522c..68fde909 100644 --- a/samples/calculator/weaveffi.toml +++ b/samples/calculator/weaveffi.toml @@ -1,7 +1,7 @@ [package] name = "calculator" version = "1.0.0" -description = "Arithmetic helpers exposed across every WeaveFFI target language." +description = "A minimal WeaveFFI producer: arithmetic with a typed error, and a greeting." license = "MIT" authors = ["WeaveFoundry "] homepage = "https://github.com/weavefoundry/weaveffi" diff --git a/samples/codec/src/lib.rs b/samples/codec/src/lib.rs index 5254c6c9..243e254f 100644 --- a/samples/codec/src/lib.rs +++ b/samples/codec/src/lib.rs @@ -1,23 +1,50 @@ -//! Codec sample cdylib: a round-trip oracle for the WeaveFFI value-buffer -//! protocol. +//! Codec sample cdylib: the wire oracle for the WeaveFFI value-buffer +//! protocol, built on shared test vectors. //! -//! Every generated binding ships its own encoder and decoder for records, rich -//! enums, optionals, lists, maps, and object tokens. This module gives the -//! conformance harness one producer that exercises every wire shape in both -//! directions, so a codec bug in any language shows up as a concrete mismatch: +//! Every generated binding ships its own encoder and decoder for records, +//! rich enums, optionals, lists, maps, and object tokens. This producer holds +//! one fixed table of test values, the *vectors*, covering every wire shape: +//! each primitive at its extremes (NaN, the infinities, `-0.0`, and +//! subnormals for floats), empty and non-empty strings and bytes with +//! non-ASCII text and interior NUL bytes, optionals present and absent, +//! lists, maps keyed by strings, integers, and C-style enums, nested records, +//! C-style and rich enums, and objects inside buffers. Each vector is one +//! variant of the [`codec::Vector`] rich enum. //! -//! * `sample_*` functions return a canonical fixture the consumer checks field -//! by field (producer encodes, consumer decodes); -//! * `verify_*` functions take the same fixture back and fail with -//! [`codec::CodecError::Mismatch`] unless it decodes to exactly the canonical -//! value (consumer encodes, producer decodes); -//! * `roundtrip_*` functions return their argument unchanged, which covers the -//! direct, string, and bytes families and the 64-bit edge values; -//! * `describe_*` functions render a value as text so a failing consumer can -//! print what the producer actually saw. +//! # The vector protocol //! -//! The producer itself is pure safe Rust; the `#[weaveffi::module]` expansion -//! supplies the `BufferValue` codecs it is checking the consumers against. +//! A consumer's codec check is one loop over the table: +//! +//! 1. `vector_count()` is the number of vectors; indexes run from `0`. +//! 2. `vector(i)` returns vector `i` (the producer encodes, the consumer +//! decodes). An index past the end fails with +//! [`codec::CodecError::OutOfRange`], whose payload carries the index and +//! the count. +//! 3. `check_vector(i, v)` takes the decoded value straight back (the +//! consumer re-encodes, the producer decodes) and returns `true` only when +//! it equals vector `i` exactly: integers bit for bit, floats with their +//! sign (any NaN matches NaN), strings and bytes byte for byte, optionals, +//! list order, map contents, enum variants, and the values of the objects +//! inside. A lossy decode or encode in either direction makes it `false`. +//! 4. `vector_name(i)` labels vector `i` and `describe_vector(v)` renders +//! any vector as text, so a failing consumer can print what it sent and +//! what the producer saw. +//! +//! A round trip alone can't catch a symmetric bug (a decoder and an encoder +//! that swap the same two fields), so a consumer also builds a few vectors +//! from literals and checks those, and spot-checks some decoded fields +//! against literals. +//! +//! Buffers aren't the only way values cross. The `echo_*` functions return +//! their argument unchanged through the direct ABI families (scalars by +//! value, strings and bytes as `(ptr, len)` runs, the C-style enum as an +//! `int32_t`), so a consumer feeds every primitive vector's value through the +//! matching echo too. The `Token` interface with `primary_of`, +//! `same_primary`, and `sum_holder` checks object identity and reference +//! counting through buffers. +//! +//! The producer itself is pure safe Rust; the `#[weaveffi::module]` +//! expansion supplies the codecs the consumers are checked against. /// Value-buffer round-trip oracle covering every wire shape. #[weaveffi::module] @@ -27,19 +54,30 @@ pub mod codec { /// The oracle's error domain. #[weaveffi::error] - #[derive(Debug)] + #[derive(Debug, PartialEq)] + #[repr(i32)] pub enum CodecError { - /// value does not match the canonical fixture - Mismatch = 1, + /// vector index out of range + OutOfRange { + /// The requested index. + index: u32, + /// The number of vectors. + count: u32, + } = 1, } impl std::fmt::Display for CodecError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str("value does not match the canonical fixture") + match self { + Self::OutOfRange { index, count } => { + write!(f, "vector {index} is out of range (count {count})") + } + } } } - /// A C-style enum (crosses by value, and as an `i32` inside buffers). + /// A C-style enum with sparse and negative discriminants (crosses by + /// value, and as an `i32` inside buffers). #[weaveffi::enumeration] #[repr(i32)] #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] @@ -50,6 +88,8 @@ pub mod codec { Green = 1, /// Blue. Blue = 7, + /// Below the visible range. + Infrared = -1, } /// Every fixed-width scalar the protocol defines. @@ -120,13 +160,13 @@ pub mod codec { #[weaveffi::record] #[derive(Clone, Debug, PartialEq)] pub struct Composite { - /// UTF-8 text, including non-ASCII. + /// UTF-8 text. pub name: String, /// Raw bytes. pub blob: Vec, - /// A present optional. + /// An optional integer. pub some_i64: Option, - /// An absent optional. + /// Another optional integer. pub none_i64: Option, /// An optional string. pub some_text: Option, @@ -134,17 +174,21 @@ pub mod codec { pub names: Vec, /// A list of lists. pub matrix: Vec>, - /// An empty list. - pub empty: Vec, + /// A list of floats. + pub floats: Vec, /// A string-keyed map. pub by_name: BTreeMap, /// An integer-keyed map with record values. pub by_id: BTreeMap, + /// A map keyed by a C-style enum. + pub by_color: BTreeMap, + /// A map keyed by `u64`, with `bool` values. + pub flags: BTreeMap, /// A nested record. pub scalars: Scalars, /// A rich enum. pub shape: Shape, - /// A list of rich enums, one of each variant. + /// A list of rich enums. pub shapes: Vec, /// An optional rich enum. pub maybe_shape: Option, @@ -156,27 +200,34 @@ pub mod codec { pub colors: Vec, } - /// An opaque object whose identity and value are checked through buffers. + /// An opaque object whose value is checked through buffers. #[weaveffi::interface] pub struct Token { value: i64, } + impl std::fmt::Debug for Token { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "Token({})", self.value) + } + } + impl Token { - /// Create a token. + /// Create a token carrying `value`. pub fn new(value: i64) -> Token { Token { value } } - /// The wrapped value. + /// The carried value. pub fn value(&self) -> i64 { self.value } } - /// Objects in every buffered position: a field, an optional, and a list. + /// Objects in every buffered position: a field, an optional, a list, + /// and a map value. #[weaveffi::record] - #[derive(Clone)] + #[derive(Clone, Debug)] pub struct Holder { /// A required object. pub primary: Arc, @@ -184,9 +235,142 @@ pub mod codec { pub spare: Option>, /// A list of objects. pub many: Vec>, + /// A map with object values. + pub by_name: BTreeMap>, + } + + /// One test vector: a value of one wire shape. Tags follow declaration + /// order (`Blank` is tag 0). + #[weaveffi::enumeration] + #[derive(Clone, Debug)] + // A vector is built, sent, and dropped; its size doesn't matter. + #[allow(clippy::large_enum_variant)] + pub enum Vector { + /// A unit variant. + Blank, + /// An `i8`. + I8 { + /// The value. + value: i8, + }, + /// A `u8`. + U8 { + /// The value. + value: u8, + }, + /// An `i16`. + I16 { + /// The value. + value: i16, + }, + /// A `u16`. + U16 { + /// The value. + value: u16, + }, + /// An `i32`. + I32 { + /// The value. + value: i32, + }, + /// A `u32`. + U32 { + /// The value. + value: u32, + }, + /// An `i64`. + I64 { + /// The value. + value: i64, + }, + /// A `u64`. + U64 { + /// The value. + value: u64, + }, + /// An `f32`. + F32 { + /// The value. + value: f32, + }, + /// An `f64`. + F64 { + /// The value. + value: f64, + }, + /// A `bool`. + Flag { + /// The value. + value: bool, + }, + /// A `string`. + Text { + /// The value. + value: String, + }, + /// A `bytes`. + Blob { + /// The value. + value: Vec, + }, + /// A C-style enum. + Hue { + /// The value. + value: Color, + }, + /// An `i64?`. + MaybeI64 { + /// The value. + value: Option, + }, + /// A `string?`. + MaybeText { + /// The value. + value: Option, + }, + /// A `[string]`. + Texts { + /// The value. + value: Vec, + }, + /// A `[[i32]]`. + Grid { + /// The value. + value: Vec>, + }, + /// A `{string: i64}`. + Counts { + /// The value. + value: BTreeMap, + }, + /// A rich enum. + Figure { + /// The value. + value: Shape, + }, + /// A list of rich enums. + Figures { + /// The value. + value: Vec, + }, + /// A record of every scalar. + AllScalars { + /// The value. + value: Scalars, + }, + /// A record of every composite shape. + Deep { + /// The value. + value: Composite, + }, + /// A record holding objects. + Objects { + /// The value. + value: Holder, + }, } - /// The canonical `Scalars` fixture. + /// The canonical `Scalars`: mixed signs and values past `2^53`. fn canonical_scalars() -> Scalars { Scalars { i8_value: -8, @@ -204,22 +388,68 @@ pub mod codec { } } - /// The canonical `Composite` fixture. - fn canonical_composite() -> Composite { - let scalars = canonical_scalars(); - let mut by_name = BTreeMap::new(); - by_name.insert("one".to_string(), 1); - by_name.insert("two".to_string(), 2); - by_name.insert("neg".to_string(), -3); - let mut by_id = BTreeMap::new(); - by_id.insert(-1, scalars.clone()); - by_id.insert( - 42, - Scalars { - flag: false, - ..scalars.clone() + /// Every minimum, and float specials. + fn minimum_scalars() -> Scalars { + Scalars { + i8_value: i8::MIN, + u8_value: u8::MIN, + i16_value: i16::MIN, + u16_value: u16::MIN, + i32_value: i32::MIN, + u32_value: u32::MIN, + i64_value: i64::MIN, + u64_value: u64::MIN, + f32_value: f32::NEG_INFINITY, + f64_value: f64::NAN, + flag: false, + color: Color::Infrared, + } + } + + /// Every field at its zero value. + fn zero_scalars() -> Scalars { + Scalars { + i8_value: 0, + u8_value: 0, + i16_value: 0, + u16_value: 0, + i32_value: 0, + u32_value: 0, + i64_value: 0, + u64_value: 0, + f32_value: 0.0, + f64_value: 0.0, + flag: false, + color: Color::Red, + } + } + + /// One shape of each variant. + fn every_shape() -> Vec { + vec![ + Shape::Empty, + Shape::Circle { radius: 2.5 }, + Shape::Rect { + width: -0.0, + height: f32::INFINITY, }, - ); + Shape::Labeled { + label: "✓ label".to_string(), + count: i32::MIN, + }, + Shape::Nested { + inner: canonical_scalars(), + note: Some("n".to_string()), + }, + Shape::Nested { + inner: minimum_scalars(), + note: None, + }, + ] + } + + /// The canonical `Composite`: every field populated. + fn canonical_composite() -> Composite { Composite { name: "héllo wörld ✓".to_string(), blob: vec![0, 1, 2, 253, 254, 255], @@ -228,182 +458,393 @@ pub mod codec { some_text: Some(String::new()), names: vec!["a".to_string(), String::new(), "ccc".to_string()], matrix: vec![vec![1, 2, 3], vec![], vec![-4]], - empty: vec![], - by_name, - by_id, - scalars: scalars.clone(), + floats: vec![ + f64::NAN, + f64::INFINITY, + f64::NEG_INFINITY, + -0.0, + 5e-324, + 0.1, + ], + by_name: BTreeMap::from([ + ("one".to_string(), 1), + ("two".to_string(), 2), + ("neg".to_string(), -3), + (String::new(), i64::MAX), + ]), + by_id: BTreeMap::from([ + (-1, canonical_scalars()), + (42, zero_scalars()), + (i32::MAX, minimum_scalars()), + ]), + by_color: BTreeMap::from([ + (Color::Infrared, "below".to_string()), + (Color::Blue, "sky".to_string()), + ]), + flags: BTreeMap::from([(0, false), (u64::MAX, true)]), + scalars: canonical_scalars(), shape: Shape::Labeled { label: "tag".to_string(), count: 3, }, - shapes: vec![ - Shape::Empty, - Shape::Circle { radius: 2.5 }, - Shape::Rect { - width: 1.0, - height: 0.5, - }, - Shape::Labeled { - label: String::new(), - count: -1, - }, - Shape::Nested { - inner: scalars, - note: Some("n".to_string()), - }, - ], + shapes: every_shape(), maybe_shape: Some(Shape::Nested { - inner: canonical_scalars(), + inner: zero_scalars(), note: None, }), maybe_list: Some(vec![9, 8]), sparse: vec![Some(true), None, Some(false)], - colors: vec![Color::Red, Color::Green, Color::Blue], + colors: vec![Color::Red, Color::Green, Color::Blue, Color::Infrared], } } - /// The canonical `Scalars` value. - #[weaveffi::export] - pub fn sample_scalars() -> Scalars { - canonical_scalars() + /// The sparse `Composite`: every collection empty, every optional absent + /// or present-but-empty. + fn sparse_composite() -> Composite { + Composite { + name: "nul\0inside \u{1F980}".to_string(), + blob: Vec::new(), + some_i64: Some(0), + none_i64: None, + some_text: None, + names: Vec::new(), + matrix: vec![Vec::new()], + floats: Vec::new(), + by_name: BTreeMap::new(), + by_id: BTreeMap::new(), + by_color: BTreeMap::new(), + flags: BTreeMap::new(), + scalars: zero_scalars(), + shape: Shape::Empty, + shapes: Vec::new(), + maybe_shape: None, + maybe_list: Some(Vec::new()), + sparse: vec![None, None], + colors: Vec::new(), + } } - /// Fail unless `value` is exactly the canonical `Scalars`. - #[weaveffi::export] - pub fn verify_scalars(value: &Scalars) -> Result { - if *value == canonical_scalars() { - Ok(true) - } else { - Err(CodecError::Mismatch) - } + fn token(value: i64) -> Arc { + Arc::new(Token::new(value)) } - /// The canonical `Composite` value. + /// Wrap a value in the `Vector` variant of its shape. + macro_rules! vector_from { + ($($t:ty => $variant:ident),* $(,)?) => { + $( + impl From<$t> for Vector { + fn from(value: $t) -> Self { + Vector::$variant { value } + } + } + )* + }; + } + + vector_from! { + i8 => I8, + u8 => U8, + i16 => I16, + u16 => U16, + i32 => I32, + u32 => U32, + i64 => I64, + u64 => U64, + f32 => F32, + f64 => F64, + bool => Flag, + String => Text, + Vec => Blob, + Color => Hue, + Option => MaybeI64, + Option => MaybeText, + Vec => Texts, + Vec> => Grid, + BTreeMap => Counts, + Shape => Figure, + Vec => Figures, + Scalars => AllScalars, + Composite => Deep, + Holder => Objects, + } + + fn text(s: &str) -> Vector { + Vector::from(s.to_string()) + } + + fn texts(items: &[&str]) -> Vector { + Vector::from(items.iter().map(|s| s.to_string()).collect::>()) + } + + /// The vector table, built fresh on every call so that object vectors + /// carry new tokens each time. + fn table() -> Vec<(&'static str, Vector)> { + vec![ + ("blank", Vector::Blank), + ("i8 min", i8::MIN.into()), + ("i8 max", i8::MAX.into()), + ("u8 zero", 0u8.into()), + ("u8 max", u8::MAX.into()), + ("i16 min", i16::MIN.into()), + ("i16 max", i16::MAX.into()), + ("u16 max", u16::MAX.into()), + ("i32 min", i32::MIN.into()), + ("i32 max", i32::MAX.into()), + ("u32 max", u32::MAX.into()), + ("u32 above i32", (1u32 << 31).into()), + ("i64 min", i64::MIN.into()), + ("i64 max", i64::MAX.into()), + ("i64 past 2^53", (-9_007_199_254_740_993i64).into()), + ("u64 max", u64::MAX.into()), + ("u64 2^63", (1u64 << 63).into()), + ("f32 1.5", 1.5f32.into()), + ("f32 -0", (-0.0f32).into()), + ("f32 nan", f32::NAN.into()), + ("f32 inf", f32::INFINITY.into()), + ("f32 -inf", f32::NEG_INFINITY.into()), + ("f32 max", f32::MAX.into()), + ("f32 min subnormal", 1e-45f32.into()), + ("f64 0.1", 0.1f64.into()), + ("f64 -0", (-0.0f64).into()), + ("f64 nan", f64::NAN.into()), + ("f64 inf", f64::INFINITY.into()), + ("f64 -inf", f64::NEG_INFINITY.into()), + ("f64 max", f64::MAX.into()), + ("f64 min subnormal", 5e-324f64.into()), + ("bool true", true.into()), + ("bool false", false.into()), + ("string empty", text("")), + ("string ascii", text("hello")), + ("string non-ascii", text("héllo wörld ✓ 日本語")), + ("string astral", text("\u{1F980} crab \u{1F600}")), + ("string interior nul", text("nul\0inside\0")), + ("bytes empty", Vec::::new().into()), + ("bytes every edge", vec![0u8, 1, 127, 128, 254, 255].into()), + ("enum red", Color::Red.into()), + ("enum green", Color::Green.into()), + ("enum blue", Color::Blue.into()), + ("enum infrared", Color::Infrared.into()), + ("optional absent", None::.into()), + ("optional zero", Some(0i64).into()), + ("optional min", Some(i64::MIN).into()), + ("optional string absent", None::.into()), + ("optional string empty", Some(String::new()).into()), + ("list empty", texts(&[])), + ("list of strings", texts(&["a", "", "ccc ✓"])), + ( + "list of lists", + vec![vec![1, 2, 3], vec![], vec![i32::MIN, i32::MAX]].into(), + ), + ("map empty", BTreeMap::::new().into()), + ( + "map of strings", + BTreeMap::from([ + (String::new(), i64::MAX), + ("héllo".to_string(), -1), + ("x".to_string(), 0), + ]) + .into(), + ), + ("shape empty", Shape::Empty.into()), + ("shape circle", Shape::Circle { radius: 2.5 }.into()), + ( + "shape rect", + Shape::Rect { + width: 1.0, + height: 0.5, + } + .into(), + ), + ( + "shape labeled", + Shape::Labeled { + label: "tag".to_string(), + count: 3, + } + .into(), + ), + ( + "shape nested", + Shape::Nested { + inner: canonical_scalars(), + note: Some("note".to_string()), + } + .into(), + ), + ("every shape", every_shape().into()), + ("scalars canonical", canonical_scalars().into()), + ("scalars minimum", minimum_scalars().into()), + ("scalars zero", zero_scalars().into()), + ("composite canonical", canonical_composite().into()), + ("composite sparse", sparse_composite().into()), + ( + "objects full", + Holder { + primary: token(10), + spare: Some(token(11)), + many: vec![token(12), token(13), token(i64::MIN)], + by_name: BTreeMap::from([ + ("a".to_string(), token(20)), + ("b".to_string(), token(21)), + ]), + } + .into(), + ), + ( + "objects sparse", + Holder { + primary: token(-1), + spare: None, + many: Vec::new(), + by_name: BTreeMap::new(), + } + .into(), + ), + ] + } + + /// Exact equality: the `Debug` rendering covers every field, prints + /// distinct floats differently (keeping the sign of zero, and every NaN as + /// `NaN`), and shows objects by value. + fn same(a: &Vector, b: &Vector) -> bool { + format!("{a:?}") == format!("{b:?}") + } + + /// The number of vectors. #[weaveffi::export] - pub fn sample_composite() -> Composite { - canonical_composite() + pub fn vector_count() -> u32 { + table().len() as u32 } - /// Fail unless `value` is exactly the canonical `Composite`. + /// Vector `index`. Fails with `OutOfRange` for an index + /// past the end. #[weaveffi::export] - pub fn verify_composite(value: &Composite) -> Result { - if *value == canonical_composite() { - Ok(true) - } else { - Err(CodecError::Mismatch) + pub fn vector(index: u32) -> Result { + let mut all = table(); + let count = all.len() as u32; + if index >= count { + return Err(CodecError::OutOfRange { index, count }); } + Ok(all.swap_remove(index as usize).1) } - /// Render a `Composite` as text (a debugging aid for failing consumers). + /// A short label for vector `index`. Fails with + /// `OutOfRange` for an index past the end. #[weaveffi::export] - pub fn describe_composite(value: &Composite) -> String { - format!("{value:?}") + pub fn vector_name(index: u32) -> Result { + let all = table(); + let count = all.len() as u32; + all.get(index as usize) + .map(|(name, _)| (*name).to_string()) + .ok_or(CodecError::OutOfRange { index, count }) + } + + /// Whether `value` equals vector `index` exactly (`false` for an index + /// past the end). + #[weaveffi::export] + pub fn check_vector(index: u32, value: Vector) -> bool { + table() + .get(index as usize) + .is_some_and(|(_, expected)| same(expected, &value)) } - /// Render a `Shape` as text. + /// Render a vector as text, as the producer sees it. #[weaveffi::export] - pub fn describe_shape(value: &Shape) -> String { + pub fn describe_vector(value: Vector) -> String { format!("{value:?}") } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_scalars(value: Scalars) -> Scalars { + pub fn echo_i8(value: i8) -> i8 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_composite(value: Composite) -> Composite { + pub fn echo_u8(value: u8) -> u8 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_shape(value: Shape) -> Shape { + pub fn echo_i16(value: i16) -> i16 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_shapes(value: Vec) -> Vec { + pub fn echo_u16(value: u16) -> u16 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_opt_i64(value: Option) -> Option { + pub fn echo_i32(value: i32) -> i32 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_map(value: BTreeMap) -> BTreeMap { + pub fn echo_u32(value: u32) -> u32 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_string(value: String) -> String { + pub fn echo_i64(value: i64) -> i64 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_bytes(value: Vec) -> Vec { + pub fn echo_u64(value: u64) -> u64 { value } - /// Return the argument unchanged (covers the full 64-bit range). + /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_i64(value: i64) -> i64 { + pub fn echo_f32(value: f32) -> f32 { value } - /// Return the argument unchanged (covers values above `2^63`). + /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_u64(value: u64) -> u64 { + pub fn echo_f64(value: f64) -> f64 { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_f64(value: f64) -> f64 { + pub fn echo_bool(value: bool) -> bool { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_bool(value: bool) -> bool { + pub fn echo_text(value: String) -> String { value } /// Return the argument unchanged. #[weaveffi::export] - pub fn roundtrip_color(value: Color) -> Color { + pub fn echo_blob(value: Vec) -> Vec { value } - /// Build a holder whose tokens carry `base`, `base + 1` (spare), and - /// `base + 2 ..= base + 4` (many). + /// Return the argument unchanged. #[weaveffi::export] - pub fn make_holder(base: i64, with_spare: bool) -> Holder { - Holder { - primary: Arc::new(Token::new(base)), - spare: with_spare.then(|| Arc::new(Token::new(base.wrapping_add(1)))), - many: (2..5) - .map(|i| Arc::new(Token::new(base.wrapping_add(i)))) - .collect(), - } + pub fn echo_color(value: Color) -> Color { + value } - /// Sum every token value inside `holder`, wrapping on overflow so that - /// consumer-built holders with extreme values can't trip a debug panic. + /// The sum of every token value inside `holder` (wrapping on overflow). #[weaveffi::export] pub fn sum_holder(holder: &Holder) -> i64 { std::iter::once(&holder.primary) .chain(holder.spare.iter()) .chain(holder.many.iter()) + .chain(holder.by_name.values()) .fold(0i64, |acc, t| acc.wrapping_add(t.value())) } @@ -413,7 +854,8 @@ pub mod codec { holder.primary } - /// Whether two holders share the same primary object. + /// Whether two holders share the same primary object (identity, not + /// value). #[weaveffi::export] pub fn same_primary(a: &Holder, b: &Holder) -> bool { Arc::ptr_eq(&a.primary, &b.primary) @@ -426,102 +868,158 @@ weaveffi::export_runtime!(); #[allow(unsafe_code)] mod tests { use crate::codec::*; + use std::collections::BTreeMap; use std::sync::Arc; use weaveffi::abi::{self, FfiError}; fn decode_and_free(ptr: *const u8, len: usize) -> T { assert!(!ptr.is_null()); let bytes = unsafe { std::slice::from_raw_parts(ptr, len) }; - let value = abi::decode_value::(bytes).expect("well-formed value buffer"); + let value = unsafe { abi::decode_value::(bytes) }.expect("well-formed value buffer"); unsafe { abi::free_bytes(ptr.cast_mut(), len) }; value } - #[test] - fn composite_round_trips_through_the_abi() { + fn fetch(index: u32) -> Vector { let mut err = FfiError::default(); let mut len = 0usize; - let ptr = unsafe { codec_codec_sample_composite(&mut len, &mut err) }; - let sample = decode_and_free::(ptr, len); - assert_eq!(sample.names.len(), 3); - assert_eq!(sample.by_id.len(), 2); + let ptr = unsafe { codec_codec_vector(index, &mut len, &mut err) }; + assert_eq!(err.code, 0); + decode_and_free(ptr, len) + } - let bytes = abi::encode_value(&sample); - assert!(unsafe { codec_codec_verify_composite(bytes.as_ptr(), bytes.len(), &mut err) }); + fn check(index: u32, value: &Vector) -> bool { + let mut err = FfiError::default(); + let bytes = abi::encode_value(value); + let ok = unsafe { codec_codec_check_vector(index, bytes.as_ptr(), bytes.len(), &mut err) }; assert_eq!(err.code, 0); + ok + } + + fn count() -> u32 { + let mut err = FfiError::default(); + unsafe { codec_codec_vector_count(&mut err) } + } + + #[test] + fn every_vector_round_trips() { + let n = count(); + assert!(n > 60, "the table covers every shape"); + for i in 0..n { + let v = fetch(i); + assert!(check(i, &v), "vector {i}: {v:?}"); + // Every vector is distinct from its neighbor. + let other = (i + 1) % n; + assert!(!check(other, &v), "vector {i} must not match {other}"); + } + } - let mut changed = sample.clone(); - changed.sparse[1] = Some(true); - let bytes = abi::encode_value(&changed); - assert!(!unsafe { codec_codec_verify_composite(bytes.as_ptr(), bytes.len(), &mut err) }); + #[test] + fn out_of_range_reports_payload() { + let n = count(); + let mut err = FfiError::default(); + let mut len = 0usize; + let ptr = unsafe { codec_codec_vector(n, &mut len, &mut err) }; + assert!(ptr.is_null()); assert_eq!(err.code, 1); assert_eq!( unsafe { err.message_str() }, - Some("value does not match the canonical fixture") + Some(format!("vector {n} is out of range (count {n})").as_str()) ); + let (index, total): (u32, u32) = { + let mut r = abi::BufferReader::token_free(err.payload()); + (r.read_u32().unwrap(), r.read_u32().unwrap()) + }; + assert_eq!((index, total), (n, n)); + unsafe { abi::error_clear(&mut err) }; + assert!(!check(n, &Vector::Blank)); } #[test] - fn scalars_round_trip() { - let mut err = FfiError::default(); - let mut len = 0usize; - let ptr = unsafe { codec_codec_sample_scalars(&mut len, &mut err) }; - let sample = decode_and_free::(ptr, len); - assert_eq!(sample.u64_value, u64::MAX); - let bytes = abi::encode_value(&sample); - let out = unsafe { - codec_codec_roundtrip_scalars(bytes.as_ptr(), bytes.len(), &mut len, &mut err) + fn nan_matches_any_nan_and_zero_keeps_its_sign() { + let n = count(); + let find = |name: &str| { + (0..n) + .find(|&i| { + let mut err = FfiError::default(); + let mut len = 0usize; + let p = unsafe { codec_codec_vector_name(i, &mut len, &mut err) }; + let s = unsafe { abi::lift_string(p, len) }.unwrap(); + unsafe { abi::free_bytes(p.cast_mut(), len) }; + s == name + }) + .unwrap() }; - assert_eq!(decode_and_free::(out, len), sample); - unsafe { - assert_eq!(codec_codec_roundtrip_i64(i64::MIN, &mut err), i64::MIN); - assert_eq!(codec_codec_roundtrip_u64(u64::MAX, &mut err), u64::MAX); - assert_eq!( - codec_codec_roundtrip_color(Color::Blue as i32, &mut err), - Color::Blue as i32 - ); - } + let nan = find("f64 nan"); + let payload_nan = f64::from_bits(0x7ff8_0000_0000_0001); + assert!(check(nan, &Vector::F64 { value: payload_nan })); + let neg_zero = find("f64 -0"); + assert!(check(neg_zero, &Vector::F64 { value: -0.0 })); + assert!(!check(neg_zero, &Vector::F64 { value: 0.0 })); + } + + #[test] + fn literal_vectors_match() { + let n = count(); + let scalars = Scalars { + i8_value: -8, + u8_value: 200, + i16_value: -16_000, + u16_value: 60_000, + i32_value: -2_000_000_000, + u32_value: 4_000_000_000, + i64_value: -9_007_199_254_740_993, + u64_value: u64::MAX, + f32_value: 1.5, + f64_value: -2.25e100, + flag: true, + color: Color::Blue, + }; + let v = Vector::AllScalars { value: scalars }; + assert!((0..n).any(|i| check(i, &v))); } #[test] - fn strings_and_bytes_round_trip_verbatim() { + fn echoes_are_identity() { let mut err = FfiError::default(); let mut len = 0usize; + unsafe { + assert_eq!(codec_codec_echo_i64(i64::MIN, &mut err), i64::MIN); + assert_eq!(codec_codec_echo_u64(u64::MAX, &mut err), u64::MAX); + assert_eq!(codec_codec_echo_u32(u32::MAX, &mut err), u32::MAX); + assert!(codec_codec_echo_f64(f64::NAN, &mut err).is_nan()); + assert_eq!( + codec_codec_echo_color(Color::Infrared as i32, &mut err), + Color::Infrared as i32 + ); + assert_eq!(codec_codec_echo_color(3, &mut err), 0); + assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); + abi::error_clear(&mut err); + } let text = "nul\0inside \u{1F980}"; - let out = - unsafe { codec_codec_roundtrip_string(text.as_ptr(), text.len(), &mut len, &mut err) }; + let out = unsafe { codec_codec_echo_text(text.as_ptr(), text.len(), &mut len, &mut err) }; assert_eq!(unsafe { abi::lift_str(out, len) }, Some(text)); unsafe { abi::free_bytes(out.cast_mut(), len) }; - - let data = [0u8, 255, 0, 7]; - let out = - unsafe { codec_codec_roundtrip_bytes(data.as_ptr(), data.len(), &mut len, &mut err) }; - assert_eq!(unsafe { abi::lift_bytes(out, len) }, Some(data.to_vec())); - unsafe { abi::free_bytes(out.cast_mut(), len) }; - - // The empty string is `(NULL, 0)` in both directions. - let out = unsafe { codec_codec_roundtrip_string(std::ptr::null(), 0, &mut len, &mut err) }; - assert!(out.is_null()); - assert_eq!(len, 0); - assert_eq!(err.code, 0); + let out = unsafe { codec_codec_echo_text(std::ptr::null(), 0, &mut len, &mut err) }; + assert!(out.is_null() && len == 0 && err.code == 0); } #[test] fn holders_carry_object_references() { let mut err = FfiError::default(); - let mut len = 0usize; - let ptr = unsafe { codec_codec_make_holder(10, true, &mut len, &mut err) }; - let holder = decode_and_free::(ptr, len); - assert_eq!(holder.primary.value(), 10); - assert_eq!(holder.spare.as_ref().unwrap().value(), 11); - assert_eq!(holder.many.len(), 3); + let holder = Holder { + primary: Arc::new(Token::new(10)), + spare: Some(Arc::new(Token::new(11))), + many: vec![Arc::new(Token::new(12))], + by_name: BTreeMap::from([("k".to_string(), Arc::new(Token::new(13)))]), + }; // Every encoding carries one fresh reference per token, and every // decode adopts it, so an encoded buffer is consumed exactly once. let bytes = abi::encode_value(&holder); assert_eq!( unsafe { codec_codec_sum_holder(bytes.as_ptr(), bytes.len(), &mut err) }, - 10 + 11 + 12 + 13 + 14 + 46 ); assert_eq!(Arc::strong_count(&holder.primary), 1); @@ -538,9 +1036,6 @@ mod tests { codec_codec_same_primary(a.as_ptr(), a.len(), b.as_ptr(), b.len(), &mut err) }); assert_eq!(Arc::strong_count(&holder.primary), 1); - - let ptr = unsafe { codec_codec_make_holder(0, false, &mut len, &mut err) }; - let without = decode_and_free::(ptr, len); - assert!(without.spare.is_none()); + assert_eq!(Arc::strong_count(&holder.by_name["k"]), 1); } } diff --git a/samples/codec/weaveffi.toml b/samples/codec/weaveffi.toml index 42ae2fad..1596f21c 100644 --- a/samples/codec/weaveffi.toml +++ b/samples/codec/weaveffi.toml @@ -11,16 +11,16 @@ repository = "https://github.com/weavefoundry/weaveffi" module_name = "Codec" [generators.dotnet] -namespace = "Codec" +name = "Codec" [generators.swift] -module_name = "Codec" +name = "Codec" [generators.cpp] -namespace = "codec" +name = "codec" [generators.dart] -package_name = "codec" +name = "codec" [generators.go] -module_path = "github.com/example/codec" +name = "github.com/example/codec" diff --git a/samples/contacts/Cargo.toml b/samples/contacts/Cargo.toml deleted file mode 100644 index 9805f46e..00000000 --- a/samples/contacts/Cargo.toml +++ /dev/null @@ -1,14 +0,0 @@ -[package] -name = "contacts" -version = "0.1.0" -edition.workspace = true -publish = false - -[lib] -crate-type = ["cdylib", "rlib"] - -[dependencies] -weaveffi = { path = "../../crates/weaveffi", features = ["leak-check"] } - -[lints] -workspace = true diff --git a/samples/contacts/src/lib.rs b/samples/contacts/src/lib.rs deleted file mode 100644 index 9a18732c..00000000 --- a/samples/contacts/src/lib.rs +++ /dev/null @@ -1,542 +0,0 @@ -//! Contacts sample: a small CRUD address book written as plain, safe Rust. -//! -//! `#[weaveffi::module]` reads the annotated items and generates the -//! `extern "C"` thunks that back the stable C ABI (and every generated -//! language binding). The address book is exported as an interface: each -//! `ContactBook` object owns its contacts directly (methods take `&self` and -//! guard the state with a `Mutex`, because the object is shared across the -//! FFI boundary), and fallible methods report typed `ContactsError` codes -//! through the ABI's error channel. The producer writes no `unsafe` glue. -//! -//! The sample also shows the two ways modules relate: -//! -//! * `contacts::groups` is nested under the `contacts` root, so it may use -//! any of the parent's types, including the `ContactType` C-style enum and -//! the `ContactBook` interface, and it inherits the parent's error domain. -//! * `directory` is a sibling root. A module tree only sees its own -//! declarations, so across roots only records and rich enums (value -//! types) may be shared; it uses the `Contact` record. - -#[weaveffi::module] -pub mod contacts { - use std::sync::atomic::{AtomicI64, Ordering}; - use std::sync::Mutex; - - /// The address book's error domain. - #[weaveffi::error] - #[derive(Debug)] - pub enum ContactsError { - /// name must not be empty - InvalidName = 1, - /// contact not found - NotFound = 2, - } - - impl std::fmt::Display for ContactsError { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str(match self { - Self::InvalidName => "name must not be empty", - Self::NotFound => "contact not found", - }) - } - } - - /// How a contact is classified. - #[weaveffi::enumeration] - #[repr(i32)] - #[derive(Clone, Copy, Debug, PartialEq, Eq)] - pub enum ContactType { - /// A personal contact. - Personal = 0, - /// A work contact. - Work = 1, - /// Any other classification. - Other = 2, - } - - /// A single address-book entry. - #[weaveffi::record] - #[derive(Clone, Debug)] - pub struct Contact { - /// Stable identifier assigned on creation. - pub id: i64, - /// Given name. - pub first_name: String, - /// Family name. - pub last_name: String, - /// Optional email address. - pub email: Option, - /// How the contact is classified. - pub contact_type: ContactType, - } - - /// An in-memory address book exported as an interface. Each book owns its - /// contacts and id counter directly; destroying the object (via the - /// generated destroy symbol) releases that state. - #[weaveffi::interface] - pub struct ContactBook { - contacts: Mutex>, - next_id: AtomicI64, - } - - impl Default for ContactBook { - fn default() -> Self { - Self::new() - } - } - - impl ContactBook { - /// Create an empty address book. - pub fn new() -> Self { - ContactBook { - contacts: Mutex::new(Vec::new()), - next_id: AtomicI64::new(1), - } - } - - /// Add a contact, returning the stored record with its assigned id. - /// An empty first or last name is rejected with - /// [`ContactsError::InvalidName`]. - pub fn add( - &self, - first_name: String, - last_name: String, - email: Option, - contact_type: ContactType, - ) -> Result { - if first_name.is_empty() || last_name.is_empty() { - return Err(ContactsError::InvalidName); - } - let id = self.next_id.fetch_add(1, Ordering::Relaxed); - let contact = Contact { - id, - first_name, - last_name, - email, - contact_type, - }; - self.contacts.lock().unwrap().push(contact.clone()); - Ok(contact) - } - - /// Look up a contact by id, failing with [`ContactsError::NotFound`] - /// when none exists. - pub fn get(&self, id: i64) -> Result { - self.contacts - .lock() - .unwrap() - .iter() - .find(|c| c.id == id) - .cloned() - .ok_or(ContactsError::NotFound) - } - - /// List every stored contact. - pub fn list(&self) -> Vec { - self.contacts.lock().unwrap().clone() - } - - /// Remove a contact by id, returning whether it existed. - pub fn remove(&self, id: i64) -> bool { - let mut contacts = self.contacts.lock().unwrap(); - let before = contacts.len(); - contacts.retain(|c| c.id != id); - contacts.len() < before - } - - /// Count the stored contacts. - pub fn count(&self) -> i32 { - self.contacts.lock().unwrap().len() as i32 - } - } - - /// Group queries over a book. Nested under the `contacts` root, so its - /// functions take and return the parent's `ContactType` enum and - /// `ContactBook` interface, and report the parent's `ContactsError`. - #[weaveffi::module] - pub mod groups { - use super::{Contact, ContactBook, ContactType, ContactsError}; - - /// Count the contacts of one type. - #[weaveffi::export] - pub fn count_of_type(book: &ContactBook, contact_type: ContactType) -> i32 { - book.list() - .iter() - .filter(|c| c.contact_type == contact_type) - .count() as i32 - } - - /// The most common contact type in a book (`Personal` when empty or - /// tied). - #[weaveffi::export] - pub fn dominant_type(book: &ContactBook) -> ContactType { - let count = |t| count_of_type(book, t); - [ContactType::Work, ContactType::Other] - .into_iter() - .filter(|t| count(*t) > count(ContactType::Personal)) - .max_by_key(|t| count(*t)) - .unwrap_or(ContactType::Personal) - } - - /// Copy every contact of one type into a new book. - #[weaveffi::export] - pub fn split_by_type(book: &ContactBook, contact_type: ContactType) -> ContactBook { - let out = ContactBook::new(); - for c in book - .list() - .into_iter() - .filter(|c| c.contact_type == contact_type) - { - // The names were validated when the contact was first added. - let _ = out.add(c.first_name, c.last_name, c.email, c.contact_type); - } - out - } - - /// The first contact of one type, failing with the parent domain's - /// [`ContactsError::NotFound`] when there is none. - #[weaveffi::export] - pub fn first_of_type( - book: &ContactBook, - contact_type: ContactType, - ) -> Result { - book.list() - .into_iter() - .find(|c| c.contact_type == contact_type) - .ok_or(ContactsError::NotFound) - } - } -} - -/// Directory formatting. A sibling root of `contacts`, so it shares only the -/// `Contact` record (a value type) with it. -#[weaveffi::module] -pub mod directory { - use super::contacts::Contact; - - /// The directory's error domain. - #[weaveffi::error] - #[derive(Debug)] - pub enum DirectoryError { - /// no contacts to list - Empty = 1, - } - - impl std::fmt::Display for DirectoryError { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str("no contacts to list") - } - } - - /// A contact formatted for display. - #[weaveffi::record] - #[derive(Clone, Debug, PartialEq)] - pub struct Card { - /// `Last, First`. - pub display_name: String, - /// The first letters of the first and last names. - pub initials: String, - /// Whether the contact has an email address. - pub has_email: bool, - } - - /// Format one contact as a card. - #[weaveffi::export] - pub fn card(contact: &Contact) -> Card { - let initial = |s: &str| s.chars().next().map(String::from).unwrap_or_default(); - Card { - display_name: format!("{}, {}", contact.last_name, contact.first_name), - initials: initial(&contact.first_name) + &initial(&contact.last_name), - has_email: contact.email.is_some(), - } - } - - /// Sort contacts by last name, then first name, failing with - /// [`DirectoryError::Empty`] for an empty list. - #[weaveffi::export] - pub fn sorted(mut contacts: Vec) -> Result, DirectoryError> { - if contacts.is_empty() { - return Err(DirectoryError::Empty); - } - contacts.sort_by(|a, b| (&a.last_name, &a.first_name).cmp(&(&b.last_name, &b.first_name))); - Ok(contacts) - } -} - -weaveffi::export_runtime!(); - -#[cfg(test)] -#[allow(unsafe_code)] -mod tests { - use super::contacts::{ContactBook, ContactType, ContactsError}; - - #[test] - fn create_and_get() { - let book = ContactBook::new(); - let added = book - .add( - "Alice".into(), - "Smith".into(), - Some("alice@example.com".into()), - ContactType::Work, - ) - .expect("valid contact"); - assert!(added.id > 0); - let c = book.get(added.id).expect("contact exists"); - assert_eq!(c.first_name, "Alice"); - assert_eq!(c.last_name, "Smith"); - assert_eq!(c.email.as_deref(), Some("alice@example.com")); - assert_eq!(c.contact_type, ContactType::Work); - } - - #[test] - fn create_without_email() { - let book = ContactBook::new(); - let added = book - .add("Bob".into(), "Jones".into(), None, ContactType::Personal) - .unwrap(); - assert_eq!(book.get(added.id).unwrap().email, None); - } - - #[test] - fn add_empty_name_is_invalid() { - let book = ContactBook::new(); - assert!(matches!( - book.add("".into(), "Smith".into(), None, ContactType::Personal), - Err(ContactsError::InvalidName) - )); - assert!(matches!( - book.add("Ada".into(), "".into(), None, ContactType::Personal), - Err(ContactsError::InvalidName) - )); - assert_eq!(book.count(), 0); - } - - #[test] - fn get_missing_is_not_found() { - let book = ContactBook::new(); - assert!(matches!(book.get(999), Err(ContactsError::NotFound))); - } - - #[test] - fn count_and_list() { - let book = ContactBook::new(); - assert_eq!(book.count(), 0); - book.add("A".into(), "B".into(), None, ContactType::Personal) - .unwrap(); - book.add("C".into(), "D".into(), None, ContactType::Work) - .unwrap(); - assert_eq!(book.count(), 2); - assert_eq!(book.list().len(), 2); - } - - #[test] - fn remove_deletes() { - let book = ContactBook::new(); - let added = book - .add("Del".into(), "Me".into(), None, ContactType::Other) - .unwrap(); - assert_eq!(book.count(), 1); - assert!(book.remove(added.id)); - assert_eq!(book.count(), 0); - assert!(!book.remove(added.id)); - } - - // A direct exercise of the generated C ABI thunks: construct a book - // through the interface constructor, drive its methods (including the - // typed error path), and decode the buffered `Contact` returns. Strings - // cross as `(ptr, len)`; the optional email and the records cross as - // value buffers. - mod ffi { - use super::super::contacts::groups::*; - use super::super::contacts::*; - use super::super::directory::*; - use weaveffi::abi::{self, FfiError}; - - fn decode(ptr: *const u8, len: usize) -> T { - assert!(!ptr.is_null()); - let bytes = unsafe { std::slice::from_raw_parts(ptr, len) }; - let value = abi::decode_value::(bytes).expect("well-formed value buffer"); - unsafe { abi::free_bytes(ptr.cast_mut(), len) }; - value - } - - fn add(book: *mut ContactBook, first: &str, last: &str, t: ContactType) -> Contact { - let mut err = FfiError::default(); - let email = abi::encode_value(&None::); - let mut len = 0usize; - let ptr = unsafe { - contacts_contacts_ContactBook_add( - book, - first.as_ptr(), - first.len(), - last.as_ptr(), - last.len(), - email.as_ptr(), - email.len(), - t as i32, - &mut len, - &mut err, - ) - }; - assert_eq!(err.code, 0); - decode(ptr, len) - } - - #[test] - fn book_surface() { - let mut err = FfiError::default(); - let book = unsafe { contacts_contacts_ContactBook_new(&mut err) }; - assert_eq!(err.code, 0); - assert!(!book.is_null()); - - let (first, last) = ("Zoe", "Quinn"); - let email = abi::encode_value(&Some("zoe@example.com".to_string())); - let mut len = 0usize; - let ptr = unsafe { - contacts_contacts_ContactBook_add( - book, - first.as_ptr(), - first.len(), - last.as_ptr(), - last.len(), - email.as_ptr(), - email.len(), - ContactType::Work as i32, - &mut len, - &mut err, - ) - }; - assert_eq!(err.code, 0); - let added: Contact = decode(ptr, len); - assert!(added.id > 0); - assert_eq!(added.first_name, "Zoe"); - assert_eq!(added.email.as_deref(), Some("zoe@example.com")); - - // An empty first name reports the InvalidName domain code. - let rejected = unsafe { - contacts_contacts_ContactBook_add( - book, - std::ptr::null(), - 0, - last.as_ptr(), - last.len(), - email.as_ptr(), - email.len(), - ContactType::Work as i32, - &mut len, - &mut err, - ) - }; - assert!(rejected.is_null()); - assert_eq!(err.code, 1); - assert_eq!(unsafe { err.message_str() }, Some("name must not be empty")); - - let ptr = - unsafe { contacts_contacts_ContactBook_get(book, added.id, &mut len, &mut err) }; - assert_eq!(err.code, 0); - let fetched: Contact = decode(ptr, len); - assert_eq!(fetched.contact_type, ContactType::Work); - - assert_eq!( - unsafe { contacts_contacts_ContactBook_count(book, &mut err) }, - 1 - ); - assert!(unsafe { contacts_contacts_ContactBook_remove(book, added.id, &mut err) }); - - // A missing id reports the NotFound domain code. - let missing = - unsafe { contacts_contacts_ContactBook_get(book, added.id, &mut len, &mut err) }; - assert!(missing.is_null()); - assert_eq!(err.code, 2); - - unsafe { contacts_contacts_ContactBook_destroy(book) }; - } - - #[test] - fn nested_module_uses_the_parent_enum_interface_and_errors() { - let mut err = FfiError::default(); - let book = unsafe { contacts_contacts_ContactBook_new(&mut err) }; - add(book, "Ann", "Lee", ContactType::Work); - add(book, "Bo", "Kim", ContactType::Work); - add(book, "Cy", "Ng", ContactType::Personal); - - let work = ContactType::Work as i32; - assert_eq!( - unsafe { contacts_contacts_groups_count_of_type(book, work, &mut err) }, - 2 - ); - assert_eq!( - unsafe { contacts_contacts_groups_dominant_type(book, &mut err) }, - work - ); - - let split = unsafe { contacts_contacts_groups_split_by_type(book, work, &mut err) }; - assert_eq!(err.code, 0); - assert_eq!( - unsafe { contacts_contacts_ContactBook_count(split, &mut err) }, - 2 - ); - unsafe { contacts_contacts_ContactBook_destroy(split) }; - - let mut len = 0usize; - let ptr = - unsafe { contacts_contacts_groups_first_of_type(book, work, &mut len, &mut err) }; - assert_eq!(decode::(ptr, len).first_name, "Ann"); - let other = ContactType::Other as i32; - let none = - unsafe { contacts_contacts_groups_first_of_type(book, other, &mut len, &mut err) }; - assert!(none.is_null()); - assert_eq!(err.code, 2, "the parent domain's NotFound code"); - - // An out-of-range discriminant is a marshalling failure. - assert_eq!( - unsafe { contacts_contacts_groups_count_of_type(book, 9, &mut err) }, - 0 - ); - assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); - unsafe { contacts_contacts_ContactBook_destroy(book) }; - } - - #[test] - fn sibling_root_shares_the_contact_record() { - let mut err = FfiError::default(); - let book = unsafe { contacts_contacts_ContactBook_new(&mut err) }; - let zoe = add(book, "Zoe", "Quinn", ContactType::Personal); - let ada = add(book, "Ada", "Byron", ContactType::Work); - unsafe { contacts_contacts_ContactBook_destroy(book) }; - - let bytes = abi::encode_value(&zoe); - let mut len = 0usize; - let ptr = - unsafe { contacts_directory_card(bytes.as_ptr(), bytes.len(), &mut len, &mut err) }; - assert_eq!( - decode::(ptr, len), - Card { - display_name: "Quinn, Zoe".into(), - initials: "ZQ".into(), - has_email: false, - } - ); - - let list = abi::encode_value(&vec![zoe, ada]); - let ptr = - unsafe { contacts_directory_sorted(list.as_ptr(), list.len(), &mut len, &mut err) }; - let sorted: Vec = decode(ptr, len); - assert_eq!(sorted[0].last_name, "Byron"); - - let empty = abi::encode_value(&Vec::::new()); - let none = unsafe { - contacts_directory_sorted(empty.as_ptr(), empty.len(), &mut len, &mut err) - }; - assert!(none.is_null()); - assert_eq!(err.code, 1); - assert_eq!(unsafe { err.message_str() }, Some("no contacts to list")); - } - - #[test] - fn each_root_exports_a_checksum() { - assert_ne!(contacts_contacts_checksum(), contacts_directory_checksum()); - } - } -} diff --git a/samples/contacts/weaveffi.toml b/samples/contacts/weaveffi.toml deleted file mode 100644 index b8752e58..00000000 --- a/samples/contacts/weaveffi.toml +++ /dev/null @@ -1,8 +0,0 @@ -[package] -name = "contacts" -version = "1.0.0" -description = "A small contacts book API demonstrating interfaces, enums, and typed errors." -license = "MIT" -authors = ["WeaveFoundry "] -homepage = "https://github.com/weavefoundry/weaveffi" -repository = "https://github.com/weavefoundry/weaveffi" diff --git a/samples/events/Cargo.toml b/samples/events/Cargo.toml deleted file mode 100644 index c2cd3ac4..00000000 --- a/samples/events/Cargo.toml +++ /dev/null @@ -1,14 +0,0 @@ -[package] -name = "events" -version = "0.1.0" -edition.workspace = true -publish = false - -[lib] -crate-type = ["cdylib", "rlib"] - -[dependencies] -weaveffi = { path = "../../crates/weaveffi", features = ["leak-check"] } - -[lints] -workspace = true diff --git a/samples/events/src/lib.rs b/samples/events/src/lib.rs deleted file mode 100644 index 48cbf785..00000000 --- a/samples/events/src/lib.rs +++ /dev/null @@ -1,434 +0,0 @@ -//! Events sample cdylib: a publish/subscribe bus built on a WeaveFFI -//! callback interface, a reference-counted object, and an iterator. -//! -//! The `#[weaveffi::module]` expansion emits exactly the ABI the WeaveFFI -//! generators bind to (see the generated `events.h`): a `Subscriber` vtable -//! the consumer implements, an `EventBus` object with `_clone`/`_destroy` -//! reference counting, and an opaque iterator with an -//! `int32_t next(iter, out_item, out_err)` contract. The conformance harness -//! binds the generated wrappers of every language against this library, so the -//! two must agree. -//! -//! The producer writes only safe Rust. The consumer's subscriber arrives as an -//! `Arc`; the bus retains it for as long as it likes and the -//! consumer's `free` entry fires when the last reference drops. -//! -//! The two callback styles are both on show. `route` returns -//! `Result`, so a consumer failure comes back as a -//! value; the bus chooses to propagate it, aborting the call with -//! `FOREIGN_ERROR_CODE`. `on_message` and `on_attached` return plain values, -//! so a consumer failure there aborts the call directly. Either way the bus -//! snapshots its subscriber list before calling out and never holds a lock -//! across a callback. - -/// A publish/subscribe event bus driven by a consumer-implemented subscriber. -#[weaveffi::module] -pub mod events { - use std::sync::{Arc, Mutex, PoisonError}; - - /// How a subscriber wants to be told about a message. - #[weaveffi::enumeration] - #[repr(i32)] - #[derive(Clone, Copy, Debug, PartialEq, Eq)] - pub enum Delivery { - /// Deliver the message. - Accept = 0, - /// Skip this subscriber for this message. - Skip = 1, - /// Deliver the message and stop delivering to later subscribers. - AcceptAndStop = 2, - } - - /// A published message as subscribers see it. - #[weaveffi::record] - #[derive(Clone, Debug, PartialEq)] - pub struct Message { - /// Monotonic sequence number, starting at 1. - pub seq: i64, - /// Topic the message was published under. - pub topic: String, - /// Message text. - pub text: String, - /// Free-form labels attached at publish time. - pub tags: Vec, - } - - /// A consumer-implemented subscriber. The bus asks `route` whether to - /// deliver each message and then calls `on_message` for accepted ones. - #[weaveffi::callback_interface] - pub trait Subscriber: Send + Sync { - /// Decide how the bus should treat `topic` for this subscriber. A - /// consumer failure is returned as an `Err`. - fn route(&self, topic: &str) -> Result; - /// Receive an accepted message. Returns the subscriber's running count - /// of received messages. - fn on_message(&self, message: &Message) -> i64; - /// Receive the bus itself (an object handed through a callback). The - /// consumer adopts the reference and may keep or drop it. - fn on_attached(&self, bus: Arc); - } - - /// A bus that retains its subscribers and logs every message. - #[weaveffi::interface] - pub struct EventBus { - subscribers: Mutex>>, - log: Mutex>, - } - - impl EventBus { - /// Create an empty bus. - pub fn new() -> Arc { - Arc::new(Self { - subscribers: Mutex::new(Vec::new()), - log: Mutex::new(Vec::new()), - }) - } - - /// Retain `subscriber` and tell it which bus it joined. Returns the - /// new subscriber count. - pub fn subscribe(self: Arc, subscriber: Arc) -> i64 { - subscriber.on_attached(Arc::clone(&self)); - let mut subs = self - .subscribers - .lock() - .unwrap_or_else(PoisonError::into_inner); - subs.push(subscriber); - subs.len() as i64 - } - - /// Publish `text` under `topic`, returning how many subscribers - /// accepted it. A subscriber failure aborts the call. - pub fn publish(&self, topic: String, text: String, tags: Vec) -> i64 { - let message = { - let mut log = self.log.lock().unwrap_or_else(PoisonError::into_inner); - let message = Message { - seq: log.len() as i64 + 1, - topic, - text, - tags, - }; - log.push(message.clone()); - message - }; - // Snapshot so no lock is held while the consumer runs. - let subs: Vec> = self - .subscribers - .lock() - .unwrap_or_else(PoisonError::into_inner) - .clone(); - let mut delivered = 0; - for sub in &subs { - let delivery = match sub.route(&message.topic) { - Ok(d) => d, - Err(e) => { - // Propagate the subscriber's failure to our caller, - // exactly as a plain-return method would have. - weaveffi::abi::raise_foreign_error(e); - return delivered; - } - }; - match delivery { - Delivery::Skip => {} - Delivery::Accept => { - sub.on_message(&message); - delivered += 1; - } - Delivery::AcceptAndStop => { - sub.on_message(&message); - delivered += 1; - break; - } - } - } - delivered - } - - /// Publish from a producer thread, resolving with the accepted count. - pub async fn publish_later(&self, topic: String, text: String) -> i64 { - self.publish(topic, text, Vec::new()) - } - - /// Number of retained subscribers. - pub fn subscriber_count(&self) -> i64 { - self.subscribers - .lock() - .unwrap_or_else(PoisonError::into_inner) - .len() as i64 - } - - /// Drop every subscriber; each consumer `free` entry runs when its - /// last reference goes away. - pub fn clear_subscribers(&self) { - self.subscribers - .lock() - .unwrap_or_else(PoisonError::into_inner) - .clear(); - } - - /// Stream the text of every message published so far, in order. - pub fn messages(&self) -> weaveffi::Iter { - let texts: Vec = self - .log - .lock() - .unwrap_or_else(PoisonError::into_inner) - .iter() - .map(|m| m.text.clone()) - .collect(); - weaveffi::Iter::new(texts) - } - - /// The most recent message, if any. - pub fn last_message(&self) -> Option { - self.log - .lock() - .unwrap_or_else(PoisonError::into_inner) - .last() - .cloned() - } - } - - /// Ask `subscriber` how it would route `topic` without a bus. A - /// subscriber failure fails the call. - #[weaveffi::export] - pub fn route_once(subscriber: Arc, topic: &str) -> Delivery { - subscriber.route(topic).unwrap_or_else(|e| { - weaveffi::abi::raise_foreign_error(e); - Delivery::Skip - }) - } -} - -weaveffi::export_runtime!(); - -#[cfg(test)] -#[allow(unsafe_code)] -mod tests { - use crate::events::*; - use std::os::raw::c_void; - use std::sync::atomic::{AtomicI64, AtomicUsize, Ordering}; - use std::sync::Arc; - use weaveffi::abi::{self, FfiError}; - - /// A consumer-side subscriber, exactly as a generated binding builds one: - /// a heap-allocated context and a process-wide vtable. - struct SubState { - received: AtomicI64, - attached: AtomicUsize, - skip_topic: String, - fail_topic: String, - freed: Arc, - } - - unsafe extern "C" fn route( - ctx: *mut c_void, - topic_ptr: *const u8, - topic_len: usize, - out_err: *mut FfiError, - ) -> i32 { - let state = unsafe { &*(ctx as *const SubState) }; - // String arguments are borrowed `(ptr, len)` runs for the call. - let topic = unsafe { abi::lift_str(topic_ptr, topic_len) }.unwrap(); - if topic == state.fail_topic { - let msg = c"subscriber rejected topic"; - unsafe { crate::events_error_set(out_err, abi::FOREIGN_ERROR_CODE, msg.as_ptr()) }; - return 0; - } - if topic == state.skip_topic { - Delivery::Skip as i32 - } else if topic == "stop" { - Delivery::AcceptAndStop as i32 - } else { - Delivery::Accept as i32 - } - } - - unsafe extern "C" fn on_message( - ctx: *mut c_void, - message_ptr: *const u8, - message_len: usize, - _out_err: *mut FfiError, - ) -> i64 { - let state = unsafe { &*(ctx as *const SubState) }; - let message: Message = - abi::decode_value(unsafe { std::slice::from_raw_parts(message_ptr, message_len) }) - .unwrap(); - assert!(message.seq >= 1); - state.received.fetch_add(1, Ordering::SeqCst) + 1 - } - - unsafe extern "C" fn on_attached(ctx: *mut c_void, bus: *mut EventBus, _err: *mut FfiError) { - let state = unsafe { &*(ctx as *const SubState) }; - state.attached.fetch_add(1, Ordering::SeqCst); - // The reference is ours: dropping it must not free the live bus. - unsafe { events_events_EventBus_destroy(bus) }; - } - - unsafe extern "C" fn free(ctx: *mut c_void) { - let state = unsafe { Box::from_raw(ctx as *mut SubState) }; - state.freed.fetch_add(1, Ordering::SeqCst); - } - - static VTABLE: events_events_Subscriber_vtable = events_events_Subscriber_vtable { - route, - on_message, - on_attached, - free, - }; - - fn new_sub(skip: &str, fail: &str, freed: &Arc) -> *mut c_void { - Box::into_raw(Box::new(SubState { - received: AtomicI64::new(0), - attached: AtomicUsize::new(0), - skip_topic: skip.to_string(), - fail_topic: fail.to_string(), - freed: Arc::clone(freed), - })) as *mut c_void - } - - fn new_bus() -> *mut EventBus { - let mut err = FfiError::default(); - let bus = unsafe { events_events_EventBus_new(&mut err) }; - assert!(!bus.is_null()); - bus - } - - fn subscribe(bus: *mut EventBus, sub: *mut c_void) -> i64 { - let mut err = FfiError::default(); - unsafe { events_events_EventBus_subscribe(bus, sub, &VTABLE, &mut err) } - } - - fn publish(bus: *mut EventBus, topic: &str, text: &str, err: &mut FfiError) -> i64 { - let tags = abi::encode_value(&vec!["a".to_string()]); - unsafe { - events_events_EventBus_publish( - bus, - topic.as_ptr(), - topic.len(), - text.as_ptr(), - text.len(), - tags.as_ptr(), - tags.len(), - err, - ) - } - } - - #[test] - fn subscribe_publish_and_iterate() { - let mut err = FfiError::default(); - let freed = Arc::new(AtomicUsize::new(0)); - let bus = new_bus(); - - let a = new_sub("quiet", "", &freed); - let b = new_sub("", "", &freed); - assert_eq!(subscribe(bus, a), 1); - assert_eq!(subscribe(bus, b), 2); - let a_state = unsafe { &*(a as *const SubState) }; - assert_eq!(a_state.attached.load(Ordering::SeqCst), 1); - - assert_eq!(publish(bus, "news", "hello", &mut err), 2); - assert_eq!(publish(bus, "quiet", "psst", &mut err), 1); - assert_eq!(publish(bus, "stop", "last", &mut err), 1); - assert_eq!(err.code, 0); - - let iter = unsafe { events_events_EventBus_messages(bus, &mut err) }; - let mut got = Vec::new(); - loop { - let mut item: *const u8 = std::ptr::null(); - let mut len = 0usize; - let has = unsafe { - events_events_EventBus_MessagesIterator_next(iter, &mut item, &mut len, &mut err) - }; - if has == 0 { - break; - } - got.push(unsafe { abi::lift_string(item, len) }.unwrap()); - unsafe { crate::events_free_bytes(item.cast_mut(), len) }; - } - unsafe { events_events_EventBus_MessagesIterator_destroy(iter) }; - assert_eq!(got, vec!["hello", "psst", "last"]); - - let mut len = 0usize; - let ptr = unsafe { events_events_EventBus_last_message(bus, &mut len, &mut err) }; - let last: Option = - abi::decode_value(unsafe { std::slice::from_raw_parts(ptr, len) }).unwrap(); - unsafe { crate::events_free_bytes(ptr.cast_mut(), len) }; - assert_eq!(last.unwrap().text, "last"); - - unsafe { events_events_EventBus_clear_subscribers(bus, &mut err) }; - assert_eq!( - freed.load(Ordering::SeqCst), - 2, - "free ran once per subscriber" - ); - unsafe { events_events_EventBus_destroy(bus) }; - } - - #[test] - fn a_failed_route_aborts_publish() { - let mut err = FfiError::default(); - let freed = Arc::new(AtomicUsize::new(0)); - let bus = new_bus(); - subscribe(bus, new_sub("", "boom", &freed)); - publish(bus, "boom", "x", &mut err); - assert_eq!(err.code, abi::FOREIGN_ERROR_CODE); - assert_eq!( - unsafe { err.message_str() }, - Some("subscriber rejected topic") - ); - - // The bus is still usable afterward. - assert_eq!(publish(bus, "ok", "y", &mut err), 1); - assert_eq!(err.code, 0); - unsafe { events_events_EventBus_destroy(bus) }; - assert_eq!( - freed.load(Ordering::SeqCst), - 1, - "destroying the bus frees its subscriber" - ); - } - - #[test] - fn route_once_does_not_retain() { - let mut err = FfiError::default(); - let freed = Arc::new(AtomicUsize::new(0)); - let topic = "quiet"; - let d = unsafe { - events_events_route_once( - new_sub("quiet", "", &freed), - &VTABLE, - topic.as_ptr(), - topic.len(), - &mut err, - ) - }; - assert_eq!(d, Delivery::Skip as i32); - assert_eq!(freed.load(Ordering::SeqCst), 1); - - let d = unsafe { - events_events_route_once( - new_sub("", "quiet", &freed), - &VTABLE, - topic.as_ptr(), - topic.len(), - &mut err, - ) - }; - assert_eq!(d, 0); - assert_eq!(err.code, abi::FOREIGN_ERROR_CODE); - } - - #[test] - fn reference_counting() { - let mut err = FfiError::default(); - let bus = new_bus(); - unsafe { - let again = events_events_EventBus_clone(bus); - assert_eq!(bus, again); - events_events_EventBus_destroy(bus); - assert_eq!(events_events_EventBus_subscriber_count(again, &mut err), 0); - events_events_EventBus_destroy(again); - events_events_EventBus_destroy(std::ptr::null_mut()); - } - } -} diff --git a/samples/events/weaveffi.toml b/samples/events/weaveffi.toml deleted file mode 100644 index bd94510e..00000000 --- a/samples/events/weaveffi.toml +++ /dev/null @@ -1,8 +0,0 @@ -[package] -name = "events" -version = "1.0.0" -description = "Publish/subscribe bus built on a callback interface, a shared object, and an iterator." -license = "MIT" -authors = ["WeaveFoundry "] -homepage = "https://github.com/weavefoundry/weaveffi" -repository = "https://github.com/weavefoundry/weaveffi" diff --git a/samples/kvstore/src/lib.rs b/samples/kvstore/src/lib.rs index 7c5cef4e..8020aa53 100644 --- a/samples/kvstore/src/lib.rs +++ b/samples/kvstore/src/lib.rs @@ -1,68 +1,150 @@ -//! Kvstore sample cdylib: a production-quality, in-memory key/value store that -//! exercises every IDL feature WeaveFFI supports through the -//! `#[weaveffi::module]` macro: a reference-counted interface with -//! constructors, methods, statics, and `_clone`/`_destroy` symbols, a typed -//! error domain (`#[weaveffi::error]`), a consumer-implemented callback -//! interface, optional/list/map/bytes record fields, records and lists that -//! carry objects, `Store?` in both directions, an iterator return, a -//! cancellable async method, deprecated and nested-submodule surface, all -//! over the C ABI. Records cross the boundary as value buffers: each -//! `#[weaveffi::record]` gets a generated `BufferValue` implementation instead -//! of per-field C accessors. +//! Kvstore sample cdylib: an in-memory key-value store that uses every +//! feature WeaveFFI supports, written as plain, safe Rust. //! -//! `Store` is exported as an interface, so each object owns its rich state -//! (its entries and the monotonic entry-id counter) directly. Methods take -//! `&self` and guard that state with a `Mutex` because the object is shared -//! across the FFI boundary; the last `kvstore_kv_Store_destroy` releases the -//! state with it. +//! The `kv` module tree is the feature-complete producer the conformance +//! harness runs in every language: +//! +//! * a reference-counted `Store` interface with fallible and infallible +//! constructors, methods, statics, and a deprecated method; +//! * the `KvError` domain, whose codes carry payload fields; +//! * records (`Entry`, `StoreInfo`), a C-style enum (`EntryKind`), and a rich +//! enum (`Change`), with optionals, lists, and maps (including maps keyed by +//! the C-style enum); +//! * `Store` objects in every position: parameters, returns, optionals, +//! lists, map values, record fields, iterator elements, an async result, +//! and both a parameter and the return of a callback method; +//! * three callback interfaces the consumer implements: `Listener` (retained, +//! and notified from a producer thread during compaction), `Policy` +//! (rich returns and a throwing method whose typed error, payload +//! included, reaches the original caller), and `Loader` (string, bytes, and +//! optional-object returns, passed as an optional parameter); +//! * lazy iterators of strings, records, and objects; +//! * async methods and functions, including a cancellable one that stops its +//! background work cooperatively, with a test hook (`Store::active_jobs`) +//! that shows it did; +//! * a nested `kv.stats` module that uses the parent's `Store` and inherits +//! the parent's error domain, and a sibling `report` root that shares the +//! `Entry` record. +//! +//! Time is a logical clock per store (`now`, advanced by `tick`), so TTLs are +//! deterministic: an entry put with `ttl_seconds: Some(t)` expires once the +//! clock reaches `now + t`. +//! +//! No lock is held while a consumer callback runs: each operation snapshots +//! the callbacks it needs, releases its locks, and then calls out, so a +//! callback may call back into the store. -/// An embedded key-value store API with TTLs, iteration, and async compaction. +/// An embedded key-value store with listeners, policies, read-through +/// loaders, iteration, and async compaction. #[weaveffi::module] pub mod kv { use std::collections::BTreeMap; - use std::sync::atomic::{AtomicI64, Ordering}; + use std::future::Future; + use std::pin::Pin; + use std::sync::atomic::{AtomicI64, AtomicU32, Ordering}; use std::sync::{Arc, Mutex, PoisonError}; - #[cfg(not(target_arch = "wasm32"))] - use std::time::{SystemTime, UNIX_EPOCH}; + use std::task::{Context, Poll}; - /// The store's error domain. Each variant's discriminant is the stable - /// ABI code a throwing method reports through `out_err`, its `Display` - /// output is the runtime message, and its doc comment is the documented - /// default message. + use weaveffi::{CancelToken, ErrorReport, ForeignError}; + + /// The store's error domain. Each code's fields travel as the error's + /// payload, so every binding raises a typed error carrying them. #[weaveffi::error] - #[derive(Debug)] + #[derive(Debug, Clone, PartialEq, Eq)] + #[repr(i32)] pub enum KvError { /// key not found - KeyNotFound = 1001, + KeyNotFound { + /// The key that was looked up. + key: String, + } = 1001, /// entry expired - Expired = 1002, - /// store has reached capacity - StoreFull = 1003, - /// I/O failure - IoError = 1004, + Expired { + /// The expired entry's key. + key: String, + /// The logical time at which it expired. + expired_at: i64, + } = 1002, + /// store is full + StoreFull { + /// The store's capacity. + capacity: u32, + } = 1003, + /// invalid path + InvalidPath = 1004, + /// write rejected by policy + Rejected { + /// The rejected key. + key: String, + /// Why the policy rejected it. + reason: String, + } = 1005, } impl std::fmt::Display for KvError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str(match self { - Self::KeyNotFound => "key not found", - Self::Expired => "entry expired", - Self::StoreFull => "store has reached capacity", - Self::IoError => "I/O failure", - }) + match self { + Self::KeyNotFound { key } => write!(f, "key not found: {key}"), + Self::Expired { key, expired_at } => { + write!(f, "entry {key} expired at {expired_at}") + } + Self::StoreFull { capacity } => write!(f, "store is full ({capacity} entries)"), + Self::InvalidPath => f.write_str("invalid path"), + Self::Rejected { key, reason } => write!(f, "write to {key} rejected: {reason}"), + } + } + } + + /// A store operation's failure: a `KvError`, or a consumer callback's + /// failure passed through unchanged (its code, message, and payload), so + /// a typed error a callback raised reaches the original caller typed. + #[derive(Debug)] + pub enum StoreError { + /// A failure of the store itself. + Kv(KvError), + /// A callback's failure. + Callback(ForeignError), + } + + impl From for StoreError { + fn from(e: KvError) -> Self { + Self::Kv(e) } } - /// The largest number of live entries one store will hold before `put` - /// rejects a new key with [`KvError::StoreFull`]. - const STORE_CAPACITY: usize = 1_000_000; + impl From for StoreError { + fn from(e: ForeignError) -> Self { + Self::Callback(e) + } + } + + impl ErrorReport for StoreError { + fn code(&self) -> i32 { + match self { + Self::Kv(e) => e.code(), + Self::Callback(e) => e.code(), + } + } + fn message(&self) -> String { + match self { + Self::Kv(e) => e.message(), + Self::Callback(e) => e.message(), + } + } + fn payload(&self) -> Vec { + match self { + Self::Kv(e) => e.payload(), + Self::Callback(e) => e.payload(), + } + } + } /// Persistence semantics applied to a stored entry. #[weaveffi::enumeration] #[repr(i32)] - #[derive(Clone, Copy, Debug, PartialEq, Eq)] + #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] pub enum EntryKind { - /// In-memory only; lost on close. + /// In-memory only. Volatile = 0, /// Flushed to durable storage. Persistent = 1, @@ -70,72 +152,60 @@ pub mod kv { Encrypted = 2, } - /// A single key-value entry persisted in the store. + /// A stored entry. #[weaveffi::record] - #[derive(Clone, Debug)] + #[derive(Clone, Debug, PartialEq)] pub struct Entry { - /// Stable monotonic identifier assigned on insert. - pub id: i64, - /// UTF-8 lookup key. + /// The entry's key. pub key: String, - /// Opaque binary payload. + /// The stored bytes. pub value: Vec, - /// Unix-timestamp seconds when the entry was created. - pub created_at: i64, - /// Optional unix-timestamp seconds at which the entry expires. + /// How the entry is persisted. + pub kind: EntryKind, + /// `1` on first insert, incremented by every later `put` of the key. + pub version: u32, + /// The logical time at which the entry expires, if it has a TTL. pub expires_at: Option, - /// Free-form labels attached to the entry. + /// Free-form labels (a policy may add some). pub tags: Vec, - /// Arbitrary string-valued metadata pairs. + /// String metadata (a loaded entry records its loader under + /// `source`). pub metadata: BTreeMap, } impl Entry { - /// Whether the entry's TTL has elapsed as of `now` (unix seconds). fn is_expired(&self, now: i64) -> bool { matches!(self.expires_at, Some(t) if t <= now) } } - #[cfg(not(target_arch = "wasm32"))] - fn now_unix_seconds() -> i64 { - SystemTime::now() - .duration_since(UNIX_EPOCH) - .map(|d| d.as_secs() as i64) - .unwrap_or(0) - } - - // `wasm32-unknown-unknown` has no wall clock; `SystemTime::now()` traps. Use - // a fixed epoch so TTL arithmetic stays deterministic and entries never - // appear spuriously expired when the bindings are exercised from JavaScript. - #[cfg(target_arch = "wasm32")] - fn now_unix_seconds() -> i64 { - 1_700_000_000 - } - - /// Why an entry left the store. + /// A change a listener is told about. #[weaveffi::enumeration] - #[repr(i32)] - #[derive(Clone, Copy, Debug, PartialEq, Eq)] - pub enum EvictionReason { - /// Removed by an explicit `delete`. - Deleted = 0, - /// Its TTL elapsed and a read evicted it. - Expired = 1, - } - - /// A consumer-implemented observer of evictions. One listener at a time - /// is attached to a store with [`Store::set_eviction_listener`]; the store - /// retains it until it's replaced, cleared, or the store is dropped. - #[weaveffi::callback_interface] - pub trait EvictionListener: Send + Sync { - /// An entry left the store. Returns whether the listener wants to keep - /// receiving notifications; `false` detaches it. - fn on_evict(&self, entry: &Entry, reason: EvictionReason) -> bool; - } - - /// A named view of a store, used to exercise objects inside records: the - /// `store` field carries a strong reference and `mirror` may be absent. + #[derive(Clone, Debug, PartialEq)] + pub enum Change { + /// An entry was stored. + Put { + /// The stored entry. + entry: Entry, + /// Whether it replaced an existing entry. + replaced: bool, + }, + /// An entry was removed. + Removed { + /// The removed key. + key: String, + /// `true` when it left because its TTL elapsed, `false` for an + /// explicit `delete`. + expired: bool, + }, + /// The store was cleared. + Cleared { + /// How many entries were removed. + count: u32, + }, + } + + /// A labeled view of a store: a record carrying objects. #[weaveffi::record] #[derive(Clone)] pub struct StoreInfo { @@ -143,116 +213,206 @@ pub mod kv { pub label: String, /// The described store. pub store: Arc, - /// An optional second store to compare against. + /// An optional second store. pub mirror: Option>, - /// Live entry count at the time of the snapshot. - pub count: i64, + /// The described store's live entry count at the time of the + /// snapshot. + pub count: u32, + } + + /// An observer of a store's changes, retained from `subscribe` until + /// `unsubscribe`, a failure, or the store's release. Changes made by + /// `compact` arrive on a producer thread. + #[weaveffi::callback_interface] + pub trait Listener: Send + Sync { + /// Whether the listener wants changes to `key` (`Cleared` changes + /// are always delivered). + fn accepts(&self, key: &str) -> Result; + + /// A change the listener accepted. A failure here (or in `accepts`) + /// unsubscribes the listener; the store operation still succeeds. + fn on_change(&self, change: &Change) -> Result<(), ForeignError>; + } + + /// A store's write policy, consulted by every `put` while installed + /// with `set_policy`. + #[weaveffi::callback_interface] + pub trait Policy: Send + Sync { + /// Admit an entry about to be stored, returning it as it should be + /// stored. The policy may change its value, kind, TTL, tags, and + /// metadata (its key and version are kept). Fail with + /// `Rejected` to veto the write: `put` then fails with that same + /// error. + #[weaveffi::throws] + fn admit(&self, entry: &Entry) -> Result; + + /// The store `key` belongs in: return `home` (the store `put` was + /// called on) to keep it there, or another store to redirect it. + fn route(&self, key: &str, home: Arc) -> Result, ForeignError>; + } + + /// A read-through source that `get_or_load` consults on a miss. + #[weaveffi::callback_interface] + pub trait Loader: Send + Sync { + /// The loader's name, recorded in a loaded entry's metadata under + /// `source`. + fn name(&self) -> Result; + + /// A store that may already hold `key`, consulted first, if any. + fn fallback(&self, key: &str) -> Result>, ForeignError>; + + /// The value for `key`. Fail with `KeyNotFound` naming + /// `key` when there's none. + #[weaveffi::throws] + fn load(&self, key: &str) -> Result, ForeignError>; + } + + /// The default capacity of a new store. + const DEFAULT_CAPACITY: u32 = 1_000_000; + + /// Background jobs (cooperative pauses) still running, across stores. + static ACTIVE_JOBS: AtomicU32 = AtomicU32::new(0); + + fn lock(m: &Mutex) -> std::sync::MutexGuard<'_, T> { + m.lock().unwrap_or_else(PoisonError::into_inner) } - /// An embedded key-value store owning its entries. Exported as an - /// interface: each object holds its own entry map and id counter behind a - /// `Mutex` (methods take `&self` because the object is shared across the - /// FFI boundary), and the last generated `destroy` releases the state. + /// An embedded key-value store. Each object owns its entries, logical + /// clock, listeners, and policy; the last release drops them (and frees + /// the consumer's callbacks). #[weaveffi::interface] pub struct Store { + path: String, entries: Mutex>, - next_entry_id: AtomicI64, - listener: Mutex>>, + clock: AtomicI64, + capacity: AtomicU32, + listeners: Mutex)>>, + next_listener: AtomicU32, + policy: Mutex>>, } impl Store { - /// Notify the attached listener (if any) outside every lock; detach it - /// when it asks to stop. - fn notify_eviction(&self, entry: &Entry, reason: EvictionReason) { - let listener = self - .listener - .lock() - .unwrap_or_else(PoisonError::into_inner) - .clone(); - if let Some(listener) = listener { - if !listener.on_evict(entry, reason) { - *self.listener.lock().unwrap_or_else(PoisonError::into_inner) = None; - } + fn with_path(path: String) -> Store { + Store { + path, + entries: Mutex::new(BTreeMap::new()), + clock: AtomicI64::new(0), + capacity: AtomicU32::new(DEFAULT_CAPACITY), + listeners: Mutex::new(Vec::new()), + next_listener: AtomicU32::new(1), + policy: Mutex::new(None), } } - } - impl Store { - /// Open (or create) a store backed by the given filesystem path. This - /// demo is purely in-memory, so the path is accepted but not used to - /// back the data; an empty path is rejected with - /// [`KvError::IoError`]. - pub fn open(path: String) -> Result { - if path.is_empty() { - return Err(KvError::IoError); + /// Tell every listener that accepts `change`'s key about it, with no + /// lock held, unsubscribing a listener that fails. + fn notify(&self, change: &Change) { + let key = match change { + Change::Put { entry, .. } => Some(entry.key.as_str()), + Change::Removed { key, .. } => Some(key.as_str()), + Change::Cleared { .. } => None, + }; + let listeners = lock(&self.listeners).clone(); + for (id, listener) in listeners { + let delivered = match key { + Some(k) => listener.accepts(k).and_then(|wanted| { + if wanted { + listener.on_change(change) + } else { + Ok(()) + } + }), + None => listener.on_change(change), + }; + if delivered.is_err() { + lock(&self.listeners).retain(|(i, _)| *i != id); + } } - Ok(Store { - entries: Mutex::new(BTreeMap::new()), - next_entry_id: AtomicI64::new(1), - listener: Mutex::new(None), - }) } - /// Attach `listener`, replacing any previous one (whose consumer - /// `free` then runs). - pub fn set_eviction_listener(&self, listener: Arc) { - *self.listener.lock().unwrap_or_else(PoisonError::into_inner) = Some(listener); + /// Store `entry` (whose key and version this sets), enforcing the + /// capacity, and notify listeners. + fn insert(&self, mut entry: Entry, key: String) -> Result { + let replaced = { + let mut entries = lock(&self.entries); + let previous = entries.get(&key).map(|e| e.version); + let capacity = self.capacity.load(Ordering::Relaxed); + if previous.is_none() && entries.len() >= capacity as usize { + return Err(KvError::StoreFull { capacity }); + } + entry.version = previous.map_or(1, |v| v.wrapping_add(1)); + entry.key = key.clone(); + entries.insert(key, entry.clone()); + previous.is_some() + }; + self.notify(&Change::Put { + entry: entry.clone(), + replaced, + }); + Ok(entry) } - /// Detach the current listener, if any. - pub fn clear_eviction_listener(&self) { - *self.listener.lock().unwrap_or_else(PoisonError::into_inner) = None; + /// The live entries whose keys start with `prefix`, in key order. + fn live(&self, prefix: Option<&str>) -> Vec { + let now = self.now(); + lock(&self.entries) + .values() + .filter(|e| !e.is_expired(now)) + .filter(|e| prefix.is_none_or(|p| e.key.starts_with(p))) + .cloned() + .collect() } - /// A second reference to this same store (the returned pointer equals - /// the receiver's; both must eventually be destroyed). - pub fn share(self: Arc) -> Arc { - self + fn copy_of(&self, entries: Vec, path: String) -> Store { + let store = Store::with_path(path); + store.clock.store(self.now(), Ordering::Relaxed); + *lock(&store.entries) = entries.into_iter().map(|e| (e.key.clone(), e)).collect(); + store } - /// A new store holding a copy of every live entry. - pub fn fork(&self) -> Arc { - let now = now_unix_seconds(); - let entries: BTreeMap = self - .entries - .lock() - .unwrap() - .iter() - .filter(|(_, e)| !e.is_expired(now)) - .map(|(k, e)| (k.clone(), e.clone())) - .collect(); - let next = self.next_entry_id.load(Ordering::Relaxed); - Arc::new(Store { - entries: Mutex::new(entries), - next_entry_id: AtomicI64::new(next), - listener: Mutex::new(None), - }) + /// Remove every expired entry and notify listeners; returns how many + /// were removed. + fn sweep(&self) -> u32 { + let now = self.now(); + let removed: Vec = { + let mut entries = lock(&self.entries); + let keys: Vec = entries + .values() + .filter(|e| e.is_expired(now)) + .map(|e| e.key.clone()) + .collect(); + for k in &keys { + entries.remove(k); + } + keys + }; + for key in &removed { + self.notify(&Change::Removed { + key: key.clone(), + expired: true, + }); + } + removed.len() as u32 } + } - /// Whichever of `self` and `other` holds more live entries, or `None` - /// when `other` is absent and `self` is empty. Exercises `Store?` as - /// both a parameter and a return. - pub fn larger(self: Arc, other: Option>) -> Option> { - match other { - Some(o) if o.count() > self.count() => Some(o), - Some(_) => Some(self), - None if self.count() > 0 => Some(self), - None => None, + impl Store { + /// Open a store at `path` (in memory; the path is only a label). + /// Fails with `InvalidPath` for an empty path. + pub fn open(path: String) -> Result { + if path.is_empty() { + return Err(KvError::InvalidPath); } + Ok(Store::with_path(path)) } - /// Snapshot this store into a record that carries the object itself. - pub fn describe(self: Arc, label: String, mirror: Option>) -> StoreInfo { - let count = self.count(); - StoreInfo { - label, - store: self, - mirror, - count, - } + /// Create an empty store with the path `memory`. + pub fn new() -> Store { + Store::with_path("memory".to_string()) } - /// Open one store per path. Exercises a list of objects as a return. + /// Open one store per path, in order. Fails with + /// `InvalidPath` if any path is empty. pub fn open_many(paths: Vec) -> Result>, KvError> { paths .into_iter() @@ -260,872 +420,505 @@ pub mod kv { .collect() } - /// Total live entries across `stores`. Exercises a list of objects as - /// a parameter and an object inside a record as a parameter. - pub fn total_count(stores: Vec>, extra: Option) -> i64 { - let base: i64 = stores.iter().map(|s| s.count()).sum(); - base + extra.map_or(0, |info| info.store.count()) + /// Index store infos by label (a later duplicate label wins). + pub fn by_label(infos: Vec) -> BTreeMap> { + infos.into_iter().map(|i| (i.label, i.store)).collect() } - /// Insert or replace a value, returning true on success. A new key is - /// rejected with [`KvError::StoreFull`] once the store holds - /// [`Store::default_capacity`] entries. + /// The total live entry count of `stores`, the values of `named`, + /// and `extra`'s store (if present). A store passed twice counts + /// twice. + pub fn total_count( + stores: Vec>, + named: BTreeMap>, + extra: Option, + ) -> u32 { + stores + .iter() + .chain(named.values()) + .chain(extra.as_ref().map(|i| &i.store)) + .map(|s| s.count()) + .sum() + } + + /// The capacity of a new store. + pub fn default_capacity() -> u32 { + DEFAULT_CAPACITY + } + + /// Background jobs still running across every store (a test hook for + /// cooperative cancellation: it returns to zero shortly after a + /// cancelled `compact` stops its pause). + pub fn active_jobs() -> u32 { + ACTIVE_JOBS.load(Ordering::SeqCst) + } + + /// The path the store was opened with. + pub fn path(&self) -> String { + self.path.clone() + } + + /// The store's logical time. + pub fn now(&self) -> i64 { + self.clock.load(Ordering::SeqCst) + } + + /// Advance the logical clock by `seconds`, returning the new time. + pub fn tick(&self, seconds: i64) -> i64 { + self.clock.fetch_add(seconds, Ordering::SeqCst) + seconds + } + + /// The most entries the store holds. + pub fn capacity(&self) -> u32 { + self.capacity.load(Ordering::Relaxed) + } + + /// Change the capacity. Existing entries stay; a new key is + /// rejected with `StoreFull` while the store is at or past + /// it. + pub fn set_capacity(&self, capacity: u32) { + self.capacity.store(capacity, Ordering::Relaxed); + } + + /// Store `value` under `key`, returning the stored entry. The + /// installed policy (if any) admits the entry first and may route it + /// to another store, where it's stored instead. Fails with + /// `StoreFull`, or with the policy's failure unchanged (such as + /// `Rejected`). pub fn put( - &self, + self: Arc, key: String, value: Vec, kind: EntryKind, ttl_seconds: Option, - ) -> Result { - // `kind` selects persistence semantics for a real backing store; - // this in-memory demo accepts it but does not surface it on the - // `Entry` record, so it is intentionally not retained. - let _ = kind; - let now = now_unix_seconds(); - let mut entries = self.entries.lock().unwrap(); - if entries.len() >= STORE_CAPACITY && !entries.contains_key(&key) { - return Err(KvError::StoreFull); - } - let expires_at = ttl_seconds.map(|t| now + t); - let entry_id = self.next_entry_id.fetch_add(1, Ordering::Relaxed); - entries.insert( - key.clone(), - Entry { - id: entry_id, - key, - value, - created_at: now, - expires_at, - tags: Vec::new(), - metadata: BTreeMap::new(), - }, - ); - Ok(true) + ) -> Result { + let mut entry = Entry { + key: key.clone(), + value, + kind, + version: 0, + expires_at: ttl_seconds.map(|t| self.now() + t), + tags: Vec::new(), + metadata: BTreeMap::new(), + }; + let policy = lock(&self.policy).clone(); + let target = match policy { + Some(policy) => { + entry = policy.admit(&entry)?; + policy.route(&key, Arc::clone(&self))? + } + None => self, + }; + Ok(target.insert(entry, key)?) } - /// Look up an entry by key; returns null if missing or expired (and - /// reports the matching [`KvError`] code through `out_err`). An - /// expired entry is evicted on read, notifying the eviction listener. - pub fn get(&self, key: String) -> Result, KvError> { - let now = now_unix_seconds(); - let (result, evicted) = { - let mut entries = self.entries.lock().unwrap(); + /// The live entry for `key`. Fails with `KeyNotFound`, or with + /// `Expired` for an entry whose TTL elapsed, which is + /// then removed (and listeners told). + pub fn get(&self, key: String) -> Result { + let now = self.now(); + let expired_at = { + let mut entries = lock(&self.entries); match entries.get(&key) { - Some(entry) if entry.is_expired(now) => { - let gone = entries.remove(&key); - (Err(KvError::Expired), gone) + None => return Err(KvError::KeyNotFound { key }), + Some(e) if !e.is_expired(now) => return Ok(e.clone()), + Some(e) => { + let at = e.expires_at.unwrap_or(now); + entries.remove(&key); + at } - Some(entry) => (Ok(Some(entry.clone())), None), - None => (Err(KvError::KeyNotFound), None), } }; - if let Some(entry) = evicted { - self.notify_eviction(&entry, EvictionReason::Expired); - } - result + self.notify(&Change::Removed { + key: key.clone(), + expired: true, + }); + Err(KvError::Expired { key, expired_at }) + } + + /// The live entry for `key`, if any (an expired entry stays until + /// `get` or `compact` removes it). + pub fn find(&self, key: String) -> Option { + let now = self.now(); + lock(&self.entries) + .get(&key) + .filter(|e| !e.is_expired(now)) + .cloned() } - /// Remove the entry for the given key, returning true if it existed. - /// A removed entry notifies the eviction listener. - pub fn delete(&self, key: String) -> Result { - let removed = self.entries.lock().unwrap().remove(&key); - match removed { - Some(entry) => { - self.notify_eviction(&entry, EvictionReason::Deleted); - Ok(true) + /// The live entry for `key`, else one from `loader`: its `fallback` + /// store's entry if that has one, otherwise a new `Volatile` entry + /// holding what `load` returns, with metadata `source` set to the + /// loader's name (either way it's stored here). Returns no entry + /// with no loader, or when `load` fails with `KeyNotFound` for + /// this same key; any other loader failure fails the call unchanged. + pub fn get_or_load( + &self, + key: String, + loader: Option>, + ) -> Result, StoreError> { + if let Some(entry) = self.find(key.clone()) { + return Ok(Some(entry)); + } + let Some(loader) = loader else { + return Ok(None); + }; + if let Some(entry) = loader.fallback(&key)?.and_then(|s| s.find(key.clone())) { + return Ok(Some(self.insert(entry, key)?)); + } + let value = match loader.load(&key) { + Ok(value) => value, + Err(e) => { + return match e.domain::() { + Some(KvError::KeyNotFound { key: missing }) if missing == key => Ok(None), + _ => Err(e.into()), + } } - None => Ok(false), + }; + let entry = Entry { + key: key.clone(), + value, + kind: EntryKind::Volatile, + version: 0, + expires_at: None, + tags: Vec::new(), + metadata: BTreeMap::from([("source".to_string(), loader.name()?)]), + }; + Ok(Some(self.insert(entry, key)?)) + } + + /// Remove `key`, returning whether it existed (listeners are told). + pub fn delete(&self, key: String) -> bool { + if lock(&self.entries).remove(&key).is_none() { + return false; } + self.notify(&Change::Removed { + key, + expired: false, + }); + true } - /// Stream every key, optionally filtered by a prefix. Expired entries - /// are skipped, and keys are yielded in sorted order (the backing map - /// is a `BTreeMap`). - pub fn list_keys(&self, prefix: Option) -> Result, KvError> { - let now = now_unix_seconds(); - let keys: Vec = self - .entries - .lock() - .unwrap() - .iter() - .filter(|(_, e)| !e.is_expired(now)) - .filter(|(k, _)| match &prefix { - Some(p) => k.starts_with(p), - None => true, - }) - .map(|(k, _)| k.clone()) - .collect(); - Ok(weaveffi::Iter::new(keys)) + /// Remove every entry, returning how many there were (listeners are + /// told once). + pub fn clear(&self) -> u32 { + let count = { + let mut entries = lock(&self.entries); + let n = entries.len() as u32; + entries.clear(); + n + }; + self.notify(&Change::Cleared { count }); + count } - /// Return the number of live (non-expired) entries in the store. - pub fn count(&self) -> i64 { - let now = now_unix_seconds(); - self.entries - .lock() - .unwrap() - .values() - .filter(|e| !e.is_expired(now)) - .count() as i64 + /// The number of live (unexpired) entries. + pub fn count(&self) -> u32 { + self.live(None).len() as u32 } - /// Drop every entry from the store. - pub fn clear(&self) { - self.entries.lock().unwrap().clear(); + /// The number of live entries. + #[deprecated(note = "use count()")] + pub fn size(&self) -> u32 { + self.count() } - /// Reclaim space asynchronously; returns the number of bytes - /// reclaimed. Cancelling the call's token completes it with the - /// cancelled code (the runtime drops the work); the check here only - /// covers a cancellation that lands as compaction starts. - #[weaveffi::cancellable] - pub async fn compact(&self, cancel: weaveffi::CancelToken) -> Result { - if cancel.is_cancelled() { - return Err(KvError::IoError); - } - let now = now_unix_seconds(); - let mut entries = self.entries.lock().unwrap(); - let expired: Vec = entries - .iter() - .filter(|(_, e)| e.is_expired(now)) - .map(|(k, _)| k.clone()) + /// The live keys in order, optionally only those starting with + /// `prefix`, pulled lazily from a snapshot. Fails with + /// `KeyNotFound` naming the prefix when a prefix matches + /// nothing. + pub fn keys(&self, prefix: Option) -> Result, KvError> { + let keys: Vec = self + .live(prefix.as_deref()) + .into_iter() + .map(|e| e.key) .collect(); - let mut reclaimed = 0i64; - for key in expired { - if let Some(entry) = entries.remove(&key) { - reclaimed += entry.value.len() as i64; - } + match prefix { + Some(p) if keys.is_empty() => Err(KvError::KeyNotFound { key: p }), + _ => Ok(weaveffi::Iter::new(keys)), } - Ok(reclaimed) } - /// Legacy single-shot put kept for compatibility. - #[deprecated(note = "use put() with explicit kind")] - pub fn legacy_put(&self, key: String, value: Vec) -> Result { - self.put(key, value, EntryKind::Volatile, None) + /// The live entries in key order, optionally only those whose keys + /// start with `prefix`. + pub fn entries(&self, prefix: Option) -> weaveffi::Iter { + weaveffi::Iter::new(self.live(prefix.as_deref())) } - /// The largest number of live entries one store will hold. - pub fn default_capacity() -> i64 { - STORE_CAPACITY as i64 + /// One new store per prefix, created lazily as the iterator is + /// pulled, holding copies of the live entries under that prefix + /// (with the prefix as its path). + pub fn partition(self: Arc, prefixes: Vec) -> weaveffi::Iter> { + weaveffi::Iter::new(prefixes.into_iter().map(move |p| { + let entries = self.live(Some(&p)); + Arc::new(self.copy_of(entries, p)) + })) } - } - /// Aggregate store-statistics surface, namespaced under `kv.stats`. - #[weaveffi::module] - pub mod stats { - use super::{KvError, Store}; - - /// Aggregate store statistics. - #[weaveffi::record] - #[derive(Clone, Debug)] - pub struct Stats { - /// Number of live entries. - pub total_entries: i64, - /// Sum of all value byte lengths. - pub total_bytes: i64, - /// Number of entries past their TTL but not yet evicted. - pub expired_entries: i64, + /// Subscribe `listener`, returning its subscription id. + pub fn subscribe(&self, listener: Arc) -> u32 { + let id = self.next_listener.fetch_add(1, Ordering::Relaxed); + lock(&self.listeners).push((id, listener)); + id } - /// Snapshot the current store statistics. Takes the parent module's - /// `Store` interface by reference across the module boundary. - #[weaveffi::export] - pub fn get_stats(store: &Store) -> Result { - let now = super::now_unix_seconds(); - let entries = store.entries.lock().unwrap(); - let total_entries = entries.len() as i64; - let total_bytes: i64 = entries.values().map(|e| e.value.len() as i64).sum(); - let expired_entries = entries.values().filter(|e| e.is_expired(now)).count() as i64; - Ok(Stats { - total_entries, - total_bytes, - expired_entries, - }) + /// Unsubscribe a listener by id, returning whether it was + /// subscribed (its consumer `free` runs once no call holds it). + pub fn unsubscribe(&self, id: u32) -> bool { + let removed = { + let mut listeners = lock(&self.listeners); + let at = listeners.iter().position(|(i, _)| *i == id); + at.map(|at| listeners.remove(at)) + }; + // Released here, with no lock held. + removed.is_some() } - } -} - -weaveffi::export_runtime!(); - -#[cfg(test)] -#[allow(unsafe_code)] -mod tests { - use crate::kv::stats::*; - use crate::kv::*; - use std::collections::BTreeMap; - use std::ffi::c_void; - use std::sync::atomic::{AtomicUsize, Ordering}; - use std::sync::{mpsc, Arc, Mutex}; - use std::time::Duration; - use weaveffi::abi::{self, FfiError}; - - /// Decode a buffered return and release the producer-owned bytes. - fn decode_and_free(ptr: *const u8, len: usize) -> T { - assert!(!ptr.is_null()); - let bytes = unsafe { std::slice::from_raw_parts(ptr, len) }; - let value = abi::decode_value::(bytes).expect("well-formed value buffer"); - unsafe { abi::free_bytes(ptr.cast_mut(), len) }; - value - } - - /// Copy a returned string and release it. - fn take_string(ptr: *const u8, len: usize) -> String { - let s = unsafe { abi::lift_string(ptr, len) }.expect("UTF-8"); - unsafe { abi::free_bytes(ptr.cast_mut(), len) }; - s - } - - fn message(err: &FfiError) -> &str { - unsafe { err.message_str() }.unwrap_or_default() - } - - // Stores are independent objects, but the tests share process-wide - // vtables and counters, so they run serialized for readable failures. - static TEST_MUTEX: Mutex<()> = Mutex::new(()); - fn setup() -> std::sync::MutexGuard<'static, ()> { - TEST_MUTEX - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner) - } - - fn open() -> *mut Store { - let mut err = FfiError::default(); - let path = "/tmp/kvstore-test"; - let s = unsafe { kvstore_kv_Store_open(path.as_ptr(), path.len(), &mut err) }; - assert_eq!(err.code, 0); - assert!(!s.is_null()); - s - } - - fn destroy(s: *mut Store) { - unsafe { kvstore_kv_Store_destroy(s) }; - } - - fn count(s: *const Store) -> i64 { - let mut err = FfiError::default(); - unsafe { kvstore_kv_Store_count(s, &mut err) } - } + /// The number of subscribed listeners. + pub fn listener_count(&self) -> u32 { + lock(&self.listeners).len() as u32 + } - /// `put` through the thunk; the optional TTL is buffered, so it's - /// encoded as `Option` and passed as a borrowed (ptr, len) pair. - fn put( - s: *mut Store, - key: &str, - value: &[u8], - kind: i32, - ttl: Option, - err: &mut FfiError, - ) -> bool { - let ttl = abi::encode_value(&ttl); - unsafe { - kvstore_kv_Store_put( - s, - key.as_ptr(), - key.len(), - value.as_ptr(), - value.len(), - kind, - ttl.as_ptr(), - ttl.len(), - err, - ) + /// Install a write policy, replacing (and releasing) any previous + /// one; passing none removes it. + pub fn set_policy(&self, policy: Option>) { + let previous = std::mem::replace(&mut *lock(&self.policy), policy); + drop(previous); } - } - fn put_simple(s: *mut Store, k: &str, v: &[u8]) { - let mut err = FfiError::default(); - assert!(put(s, k, v, EntryKind::Persistent as i32, None, &mut err)); - assert_eq!(err.code, 0); - } + /// Whether a policy is installed. + pub fn has_policy(&self) -> bool { + lock(&self.policy).is_some() + } - fn get(s: *mut Store, key: &str, err: &mut FfiError) -> Option { - let mut out_len = 0usize; - let ptr = unsafe { kvstore_kv_Store_get(s, key.as_ptr(), key.len(), &mut out_len, err) }; - if ptr.is_null() { - assert_ne!(err.code, 0); - return None; + /// Another reference to this same store. + pub fn share(self: Arc) -> Arc { + self } - decode_and_free::>(ptr, out_len) - } - fn delete(s: *mut Store, key: &str, err: &mut FfiError) -> bool { - unsafe { kvstore_kv_Store_delete(s, key.as_ptr(), key.len(), err) } - } + /// A new, independent store holding a copy of every live entry (with + /// the same path and clock, and no listeners or policy). + pub fn fork(&self) -> Arc { + Arc::new(self.copy_of(self.live(None), self.path.clone())) + } - fn keys(s: *mut Store, prefix: Option<&str>) -> Vec { - let mut err = FfiError::default(); - let prefix = abi::encode_value(&prefix.map(str::to_string)); - let iter = - unsafe { kvstore_kv_Store_list_keys(s, prefix.as_ptr(), prefix.len(), &mut err) }; - assert_eq!(err.code, 0); - assert!(!iter.is_null()); - let mut got = Vec::new(); - loop { - let mut item: *const u8 = std::ptr::null(); - let mut len = 0usize; - let has = unsafe { - kvstore_kv_Store_ListKeysIterator_next(iter, &mut item, &mut len, &mut err) - }; - assert_eq!(err.code, 0); - if has == 0 { - assert!(item.is_null()); - break; + /// Whichever of this store and `other` holds more live entries (this + /// one on a tie), or no store when `other` is absent and this store is + /// empty. + pub fn larger(self: Arc, other: Option>) -> Option> { + match other { + Some(o) if o.count() > self.count() => Some(o), + Some(_) => Some(self), + None if self.count() > 0 => Some(self), + None => None, } - got.push(take_string(item, len)); } - unsafe { kvstore_kv_Store_ListKeysIterator_destroy(iter) }; - got - } - - #[test] - fn open_destroy_lifecycle() { - let _g = setup(); - destroy(open()); - } - - #[test] - fn open_empty_path_reports_io_error() { - let _g = setup(); - let mut err = FfiError::default(); - // The fallible constructor rejects an empty path (here the canonical - // `(NULL, 0)` empty string) with the IoError domain code. - let s = unsafe { kvstore_kv_Store_open(std::ptr::null(), 0, &mut err) }; - assert!(s.is_null()); - assert_eq!(err.code, 1004, "KvError::IoError's declared code"); - assert_eq!(message(&err), "I/O failure"); - } - - #[test] - fn open_invalid_path_is_a_marshalling_error() { - let _g = setup(); - let mut err = FfiError::default(); - // A null pointer with a length, or bytes that aren't UTF-8, are - // rejected with the reserved marshalling code before `open` runs. - let s = unsafe { kvstore_kv_Store_open(std::ptr::null(), 4, &mut err) }; - assert!(s.is_null()); - assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); - let bad = [0xC0u8, 0x00]; - let s = unsafe { kvstore_kv_Store_open(bad.as_ptr(), bad.len(), &mut err) }; - assert!(s.is_null()); - assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); - } - - #[test] - fn default_capacity_static() { - let _g = setup(); - let mut err = FfiError::default(); - assert_eq!( - unsafe { kvstore_kv_Store_default_capacity(&mut err) }, - 1_000_000 - ); - assert_eq!(err.code, 0); - } - - #[test] - fn null_self_method_call_reports_error() { - let _g = setup(); - let mut err = FfiError::default(); - let n = unsafe { kvstore_kv_Store_count(std::ptr::null(), &mut err) }; - assert_eq!(n, 0); - assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); - } - - #[test] - fn put_and_get_roundtrip() { - let _g = setup(); - let s = open(); - put_simple(s, "alpha", b"hello"); - let mut err = FfiError::default(); - let e = get(s, "alpha", &mut err).expect("entry present"); - assert_eq!(err.code, 0); - assert_eq!(e.key, "alpha"); - assert_eq!(e.value, b"hello"); - assert!(e.id > 0); - destroy(s); - } - - #[test] - fn put_invalid_kind_errors() { - let _g = setup(); - let s = open(); - let mut err = FfiError::default(); - // An out-of-range `EntryKind` discriminant is rejected by the macro's - // enum lift with the reserved marshalling code. - assert!(!put(s, "k", b"", 999, None, &mut err)); - assert_eq!(err.code, abi::MARSHAL_ERROR_CODE); - destroy(s); - } - - #[test] - fn get_missing_key_returns_not_found() { - let _g = setup(); - let s = open(); - let mut err = FfiError::default(); - assert!(get(s, "nope", &mut err).is_none()); - assert_eq!(err.code, 1001, "KvError::KeyNotFound's declared code"); - assert_eq!(message(&err), "key not found"); - destroy(s); - } - #[test] - fn put_with_ttl_expires() { - let _g = setup(); - let s = open(); - let mut err = FfiError::default(); - assert!(put( - s, - "ttl", - b"x", - EntryKind::Volatile as i32, - Some(-1), - &mut err - )); - assert!(get(s, "ttl", &mut err).is_none()); - assert_eq!(err.code, 1002, "KvError::Expired's declared code"); - destroy(s); - } + /// Snapshot this store into a record that carries the store itself. + pub fn describe(self: Arc, label: String, mirror: Option>) -> StoreInfo { + let count = self.count(); + StoreInfo { + label, + store: self, + mirror, + count, + } + } - #[test] - fn delete_returns_existed() { - let _g = setup(); - let s = open(); - put_simple(s, "k", b"v"); - let mut err = FfiError::default(); - assert!(delete(s, "k", &mut err)); - assert_eq!(err.code, 0); - assert!(!delete(s, "k", &mut err)); - destroy(s); - } + /// Remove every expired entry, telling listeners from a producer + /// thread, and complete with how many were removed. First pauses for + /// `pause_ms` on a background thread that polls the cancel token + /// (counted by `active_jobs` while it runs). Cancelling + /// completes the call with the cancelled code at once; the + /// background pause then notices and stops. + #[weaveffi::cancellable] + pub async fn compact(&self, pause_ms: u32, cancel: CancelToken) -> u32 { + Pause::start(pause_ms, &cancel).await; + self.sweep() + } - #[test] - fn list_keys_iterates_in_order() { - let _g = setup(); - let s = open(); - put_simple(s, "alpha", b"1"); - put_simple(s, "beta", b"2"); - put_simple(s, "gamma", b"3"); - assert_eq!(keys(s, None), vec!["alpha", "beta", "gamma"]); - destroy(s); + /// The live entry for each key, in order (absent where there's + /// none), resolved on a producer thread. + pub async fn get_many(&self, keys: Vec) -> Vec> { + keys.into_iter().map(|k| self.find(k)).collect() + } } - #[test] - fn list_keys_with_prefix_filter() { - let _g = setup(); - let s = open(); - put_simple(s, "user.alice", b"1"); - put_simple(s, "user.bob", b"2"); - put_simple(s, "system.x", b"3"); - assert_eq!(keys(s, Some("user.")), vec!["user.alice", "user.bob"]); - destroy(s); + /// Open a store asynchronously, completing with the new object. Fails + /// with `InvalidPath` for an empty path. + #[weaveffi::export] + pub async fn open_store(path: String) -> Result, KvError> { + Store::open(path).map(Arc::new) } - #[test] - fn count_and_clear() { - let _g = setup(); - let s = open(); - let mut err = FfiError::default(); - assert_eq!(count(s), 0); - put_simple(s, "a", b"1"); - put_simple(s, "b", b"2"); - assert_eq!(count(s), 2); - unsafe { kvstore_kv_Store_clear(s, &mut err) }; - assert_eq!(err.code, 0); - assert_eq!(count(s), 0); - destroy(s); + impl Default for Store { + fn default() -> Self { + Store::new() + } } - #[test] - fn legacy_put_inserts_volatile() { - let _g = setup(); - let s = open(); - let mut err = FfiError::default(); - let (k, v) = ("legacy", b"v"); - // The generated thunk carries its own `#[allow(deprecated)]`, so - // calling it needs no opt-in here. - let ok = unsafe { - kvstore_kv_Store_legacy_put(s, k.as_ptr(), k.len(), v.as_ptr(), v.len(), &mut err) - }; - assert!(ok); - assert_eq!(count(s), 1); - destroy(s); + /// A pause that runs on a background thread polling a cancel token, so + /// a cancelled call stops its work instead of sleeping on. + struct Pause { + state: Option>, } - type Done = mpsc::Sender<(i32, i64)>; - - extern "C" fn on_compacted(context: *mut c_void, err: *mut FfiError, result: i64) { - let tx = unsafe { &*(context as *const Done) }; - let code = if err.is_null() { - 0 - } else { - let code = unsafe { (*err).code }; - unsafe { crate::kvstore_error_free(err) }; - code - }; - tx.send((code, result)).unwrap(); + #[derive(Default)] + struct PauseState { + done: std::sync::atomic::AtomicBool, + waker: Mutex>, } - fn compact(s: *mut Store, cancelled: bool) -> (i32, i64) { - let (tx, rx) = mpsc::channel::<(i32, i64)>(); - let tx_ptr = Box::into_raw(Box::new(tx)); - let token = crate::kvstore_cancel_token_create(); - unsafe { - if cancelled { - crate::kvstore_cancel_token_cancel(token); + impl Pause { + #[cfg(not(target_arch = "wasm32"))] + fn start(ms: u32, cancel: &CancelToken) -> Pause { + if ms == 0 { + return Pause { state: None }; } - kvstore_kv_Store_compact(s, token, on_compacted, tx_ptr.cast()); - // The launcher took its own reference, so the consumer may - // release the token right away. - crate::kvstore_cancel_token_destroy(token); + let state = Arc::new(PauseState::default()); + let shared = Arc::clone(&state); + let cancel = cancel.clone(); + let deadline = std::time::Instant::now() + std::time::Duration::from_millis(ms.into()); + ACTIVE_JOBS.fetch_add(1, Ordering::SeqCst); + std::thread::spawn(move || { + while !cancel.is_cancelled() && std::time::Instant::now() < deadline { + std::thread::sleep(std::time::Duration::from_millis(1)); + } + drop(cancel); + shared.done.store(true, Ordering::SeqCst); + if let Some(w) = lock(&shared.waker).take() { + w.wake(); + } + ACTIVE_JOBS.fetch_sub(1, Ordering::SeqCst); + }); + Pause { state: Some(state) } } - let out = rx.recv_timeout(Duration::from_secs(5)).unwrap(); - drop(unsafe { Box::from_raw(tx_ptr) }); - out - } - #[test] - fn compact_reclaims_expired_bytes() { - let _g = setup(); - let s = open(); - let mut err = FfiError::default(); - put( - s, - "dead", - b"hello", - EntryKind::Volatile as i32, - Some(-1), - &mut err, - ); - put( - s, - "alive", - b"x", - EntryKind::Persistent as i32, - None, - &mut err, - ); - assert_eq!(compact(s, false), (0, 5)); - destroy(s); + // `wasm32-unknown-unknown` has neither threads nor a clock, so the + // pause elapses at once there. + #[cfg(target_arch = "wasm32")] + fn start(ms: u32, cancel: &CancelToken) -> Pause { + let _ = (ms, cancel); + Pause { state: None } + } } - #[test] - fn compact_honors_cancel_token() { - let _g = setup(); - let s = open(); - assert_eq!(compact(s, true), (abi::CANCELLED_ERROR_CODE, 0)); - destroy(s); - } + impl Future for Pause { + type Output = (); - #[test] - fn get_stats_snapshots_state() { - let _g = setup(); - let s = open(); - put_simple(s, "a", b"hi"); - put_simple(s, "b", b"bye"); - let mut err = FfiError::default(); - let mut out_len: usize = 0; - let ptr = unsafe { kvstore_kv_stats_get_stats(s, &mut out_len, &mut err) }; - assert_eq!(err.code, 0); - let stats = decode_and_free::(ptr, out_len); - assert_eq!(stats.total_entries, 2); - assert_eq!(stats.total_bytes, 5); - assert_eq!(stats.expired_entries, 0); - destroy(s); + fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> { + let Some(state) = &self.state else { + return Poll::Ready(()); + }; + *lock(&state.waker) = Some(cx.waker().clone()); + if state.done.load(Ordering::SeqCst) { + Poll::Ready(()) + } else { + Poll::Pending + } + } } - #[test] - fn entry_buffer_round_trip() { - // The `Entry` record crosses the ABI as a value buffer; the macro - // implements `BufferValue`, so every field (including the optional, - // list, map, and bytes fields) round-trips through encode/decode. - let mut metadata = BTreeMap::new(); - metadata.insert("source".to_string(), "test".to_string()); - let entry = Entry { - id: 7, - key: "k".to_string(), - value: b"abc".to_vec(), - created_at: 123, - expires_at: Some(9999), - tags: vec!["hot".to_string()], - metadata, - }; - - let bytes = abi::encode_value(&entry); - let back = abi::decode_value::(&bytes).unwrap(); - assert_eq!(back.id, 7); - assert_eq!(back.key, "k"); - assert_eq!(back.value, b"abc"); - assert_eq!(back.created_at, 123); - assert_eq!(back.expires_at, Some(9999)); - assert_eq!(back.tags, vec!["hot"]); - assert_eq!( - back.metadata.get("source").map(String::as_str), - Some("test") - ); - } + /// Aggregate statistics, namespaced under `kv.stats`: functions that + /// take the parent module's `Store` and report its `KvError` domain. + #[weaveffi::module] + pub mod stats { + use std::collections::BTreeMap; + use std::sync::Arc; - #[test] - fn entry_buffer_round_trips_absent_expiry() { - let entry = Entry { - id: 1, - key: "x".to_string(), - value: Vec::new(), - created_at: 0, - expires_at: None, - tags: Vec::new(), - metadata: BTreeMap::new(), - }; - let bytes = abi::encode_value(&entry); - let back = abi::decode_value::(&bytes).unwrap(); - assert_eq!(back.expires_at, None); - assert!(back.value.is_empty()); - assert!(back.tags.is_empty()); - assert!(back.metadata.is_empty()); - } + use super::{EntryKind, KvError, Store}; - #[test] - fn stats_buffer_round_trip() { - let stats = Stats { - total_entries: 10, - total_bytes: 200, - expired_entries: 3, - }; - let bytes = abi::encode_value(&stats); - let back = abi::decode_value::(&bytes).unwrap(); - assert_eq!(back.total_entries, 10); - assert_eq!(back.total_bytes, 200); - assert_eq!(back.expired_entries, 3); - } + /// Aggregate statistics over a set of entries. + #[weaveffi::record] + #[derive(Clone, Debug, PartialEq, Default)] + pub struct Stats { + /// The number of live entries. + pub entries: u32, + /// The total size of their values in bytes. + pub bytes: u64, + /// Live entries per kind (kinds with none are absent). + pub by_kind: BTreeMap, + } - /// A consumer-side eviction listener, exactly as a generated binding - /// builds one: a heap context plus a process-wide vtable. - struct ListenerState { - evictions: Mutex>, - keep_after: usize, - freed: Arc, - } + fn add(stats: &mut Stats, store: &Store, prefix: Option<&str>) { + for e in store.live(prefix) { + stats.entries += 1; + stats.bytes += e.value.len() as u64; + *stats.by_kind.entry(e.kind).or_insert(0) += 1; + } + } - unsafe extern "C" fn on_evict( - ctx: *mut c_void, - entry_ptr: *const u8, - entry_len: usize, - reason: i32, - _out_err: *mut FfiError, - ) -> bool { - let state = unsafe { &*(ctx as *const ListenerState) }; - let entry: Entry = - abi::decode_value(unsafe { std::slice::from_raw_parts(entry_ptr, entry_len) }).unwrap(); - let mut seen = state.evictions.lock().unwrap(); - seen.push((entry.key, reason)); - seen.len() < state.keep_after - } + /// Statistics over `store`'s live entries, optionally only those + /// whose keys start with `prefix`. Fails with + /// `KeyNotFound` naming the prefix when it matches + /// nothing. + #[weaveffi::export] + pub fn summarize(store: &Store, prefix: Option) -> Result { + let mut stats = Stats::default(); + add(&mut stats, store, prefix.as_deref()); + match prefix { + Some(p) if stats.entries == 0 => Err(KvError::KeyNotFound { key: p }), + _ => Ok(stats), + } + } - unsafe extern "C" fn free_listener(ctx: *mut c_void) { - let state = unsafe { Box::from_raw(ctx as *mut ListenerState) }; - state.freed.fetch_add(1, Ordering::SeqCst); + /// Statistics over every store's live entries, computed on a + /// producer thread. + #[weaveffi::export] + pub async fn summarize_all(stores: Vec>) -> Stats { + let mut stats = Stats::default(); + for store in &stores { + add(&mut stats, store, None); + } + stats + } } +} - static LISTENER_VTABLE: kvstore_kv_EvictionListener_vtable = - kvstore_kv_EvictionListener_vtable { - on_evict, - free: free_listener, - }; - - fn new_listener(keep_after: usize, freed: &Arc) -> *mut c_void { - Box::into_raw(Box::new(ListenerState { - evictions: Mutex::new(Vec::new()), - keep_after, - freed: Arc::clone(freed), - })) - .cast() - } +/// Plain-text reports over entries. A sibling root of `kv`, so it shares +/// only the `Entry` record (a value type) with it, and has its own error +/// domain. +#[weaveffi::module] +pub mod report { + use super::kv::Entry; - #[test] - fn eviction_listener_sees_deletes_and_expiry_then_detaches() { - let _g = setup(); - let s = open(); - let freed = Arc::new(AtomicUsize::new(0)); - let mut err = FfiError::default(); - unsafe { - kvstore_kv_Store_set_eviction_listener( - s, - new_listener(2, &freed), - &LISTENER_VTABLE, - &mut err, - ) - }; - assert_eq!(err.code, 0); - - put_simple(s, "evict-me", b"v"); - assert!(delete(s, "evict-me", &mut err)); - - put( - s, - "expiring", - b"x", - EntryKind::Volatile as i32, - Some(-1), - &mut err, - ); - assert!(get(s, "expiring", &mut err).is_none()); - assert_eq!(err.code, 1002); - - // The second eviction returned `false`, so the store detached (and - // freed) the listener; a third eviction is not observed. - assert_eq!( - freed.load(Ordering::SeqCst), - 1, - "detached listener is freed" - ); - put_simple(s, "again", b"x"); - delete(s, "again", &mut err); - destroy(s); + /// The report module's error domain. + #[weaveffi::error] + #[derive(Debug)] + pub enum ReportError { + /// nothing to report + NothingToReport = 2001, } - #[test] - fn eviction_listener_is_retained_until_replaced() { - let _g = setup(); - let s = open(); - let freed = Arc::new(AtomicUsize::new(0)); - let mut err = FfiError::default(); - unsafe { - kvstore_kv_Store_set_eviction_listener( - s, - new_listener(usize::MAX, &freed), - &LISTENER_VTABLE, - &mut err, - ); - assert_eq!(freed.load(Ordering::SeqCst), 0); - kvstore_kv_Store_set_eviction_listener( - s, - new_listener(usize::MAX, &freed), - &LISTENER_VTABLE, - &mut err, - ); - assert_eq!( - freed.load(Ordering::SeqCst), - 1, - "replaced listener is freed" - ); - kvstore_kv_Store_clear_eviction_listener(s, &mut err); + impl std::fmt::Display for ReportError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("nothing to report") } - assert_eq!(freed.load(Ordering::SeqCst), 2); - destroy(s); - } - - #[test] - fn share_and_fork_reference_counting() { - let _g = setup(); - let s = open(); - put_simple(s, "a", b"1"); - let mut err = FfiError::default(); - - let shared = unsafe { kvstore_kv_Store_share(s, &mut err) }; - assert_eq!(shared, s, "share returns the same object"); - destroy(s); - assert_eq!(count(shared), 1, "still alive"); - - let forked = unsafe { kvstore_kv_Store_fork(shared, &mut err) }; - assert_ne!(forked, shared); - put_simple(forked, "b", b"2"); - assert_eq!(count(forked), 2); - assert_eq!(count(shared), 1); - - let cloned = unsafe { kvstore_kv_Store_clone(forked) }; - destroy(forked); - assert_eq!(count(cloned), 2); - destroy(cloned); - destroy(shared); } - #[test] - fn larger_handles_nullable_objects_both_ways() { - let _g = setup(); - let a = open(); - let b = open(); - put_simple(b, "x", b"1"); - let mut err = FfiError::default(); - unsafe { - assert!(kvstore_kv_Store_larger(a, std::ptr::null(), &mut err).is_null()); - let bigger = kvstore_kv_Store_larger(a, b, &mut err); - assert_eq!(bigger, b); - destroy(bigger); - let own = kvstore_kv_Store_larger(b, std::ptr::null(), &mut err); - assert_eq!(own, b); - destroy(own); + /// One line per entry, sorted by key: `"{key}: {n} bytes, {kind}"`, with + /// `, v{version}` appended when the version is above 1. Fails with + /// `NothingToReport` for an empty list. + #[weaveffi::export] + pub fn render_report(mut entries: Vec) -> Result, ReportError> { + if entries.is_empty() { + return Err(ReportError::NothingToReport); } - destroy(a); - destroy(b); + entries.sort_by(|a, b| a.key.cmp(&b.key)); + Ok(entries + .iter() + .map(|e| { + let mut line = format!("{}: {} bytes, {:?}", e.key, e.value.len(), e.kind); + if e.version > 1 { + line.push_str(&format!(", v{}", e.version)); + } + line + }) + .collect()) } +} - #[test] - fn objects_inside_records_and_lists() { - let _g = setup(); - let s = open(); - put_simple(s, "k", b"v"); - let mut err = FfiError::default(); - - let label = "primary"; - let mut out_len = 0usize; - let ptr = unsafe { - kvstore_kv_Store_describe( - s, - label.as_ptr(), - label.len(), - std::ptr::null(), - &mut out_len, - &mut err, - ) - }; - assert_eq!(err.code, 0); - let info = decode_and_free::(ptr, out_len); - assert_eq!(info.label, "primary"); - assert_eq!(info.count, 1); - assert!(info.mirror.is_none()); - // The record's object token carried its own reference. - assert_eq!(Arc::as_ptr(&info.store), s as *const Store); - assert_eq!(Arc::strong_count(&info.store), 2); - - let paths = abi::encode_value(&vec!["/a".to_string(), "/b".to_string()]); - let mut many_len = 0usize; - let many_ptr = unsafe { - kvstore_kv_Store_open_many(paths.as_ptr(), paths.len(), &mut many_len, &mut err) - }; - assert_eq!(err.code, 0); - let many = decode_and_free::>>(many_ptr, many_len); - assert_eq!(many.len(), 2); - put_simple(Arc::as_ptr(&many[0]).cast_mut(), "m", b"1"); - - // Objects written into a parameter buffer carry one reference each. - let stores = abi::encode_value(&many); - let extra = abi::encode_value(&Some(info.clone())); - let total = unsafe { - kvstore_kv_Store_total_count( - stores.as_ptr(), - stores.len(), - extra.as_ptr(), - extra.len(), - &mut err, - ) - }; - assert_eq!(total, 2); - drop(info); - drop(many); - destroy(s); - } +weaveffi::export_runtime!(); - #[test] - fn runtime_symbols_carry_the_crate_prefix() { - assert_eq!(crate::kvstore_abi_version(), weaveffi::abi::ABI_VERSION); - assert_ne!(crate::kv::kvstore_kv_checksum(), 0); - let t = crate::kvstore_cancel_token_create(); - assert!(!t.is_null()); - unsafe { - assert!(!crate::kvstore_cancel_token_is_cancelled(t)); - crate::kvstore_cancel_token_cancel(t); - assert!(crate::kvstore_cancel_token_is_cancelled(t)); - crate::kvstore_cancel_token_destroy(t); - } - } -} +#[cfg(test)] +mod tests; diff --git a/samples/kvstore/src/tests.rs b/samples/kvstore/src/tests.rs new file mode 100644 index 00000000..82b6c193 --- /dev/null +++ b/samples/kvstore/src/tests.rs @@ -0,0 +1,615 @@ +//! Producer-side tests: the store's behavior through its safe Rust API, plus +//! the ABI paths that the safe API can't reach (error payloads, a +//! consumer-built callback vtable, async completion and cancellation). + +#![allow(unsafe_code)] + +use crate::kv::stats::*; +use crate::kv::*; +use crate::report::*; +use std::collections::BTreeMap; +use std::ffi::c_void; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::{mpsc, Arc, Mutex}; +use std::time::Duration; +use weaveffi::abi::{self, ErrorReport, FfiError}; +use weaveffi::ForeignError; + +fn store() -> Arc { + Arc::new(Store::open("/tmp/kv".to_string()).unwrap()) +} + +fn put(s: &Arc, key: &str, value: &[u8]) -> Entry { + Arc::clone(s) + .put(key.to_string(), value.to_vec(), EntryKind::Persistent, None) + .unwrap_or_else(|e| panic!("put {key}: {}", e.message())) +} + +fn keys(s: &Store, prefix: Option<&str>) -> Result, KvError> { + let iter = s.keys(prefix.map(str::to_string))?; + Ok(collect(iter)) +} + +fn collect(iter: weaveffi::Iter) -> Vec { + iter.collect() +} + +#[test] +fn open_rejects_an_empty_path() { + assert_eq!(Store::open(String::new()).err(), Some(KvError::InvalidPath)); + assert_eq!(Store::new().path(), "memory"); +} + +#[test] +fn put_get_find_and_versions() { + let s = store(); + let first = put(&s, "alpha", b"one"); + assert_eq!(first.version, 1); + assert_eq!(first.kind, EntryKind::Persistent); + let second = put(&s, "alpha", b"two"); + assert_eq!(second.version, 2); + assert_eq!(s.get("alpha".to_string()).unwrap().value, b"two"); + assert_eq!(s.find("missing".to_string()), None); + assert_eq!( + s.get("missing".to_string()), + Err(KvError::KeyNotFound { + key: "missing".to_string() + }) + ); + assert!(s.delete("alpha".to_string())); + assert!(!s.delete("alpha".to_string())); +} + +#[test] +fn ttl_follows_the_logical_clock() { + let s = store(); + Arc::clone(&s) + .put("t".to_string(), vec![1], EntryKind::Volatile, Some(10)) + .unwrap(); + assert_eq!(s.tick(9), 9); + assert_eq!(s.count(), 1); + assert_eq!(s.tick(1), 10); + assert_eq!(s.count(), 0); + assert!(s.find("t".to_string()).is_none()); + assert_eq!( + s.get("t".to_string()), + Err(KvError::Expired { + key: "t".to_string(), + expired_at: 10 + }) + ); + // `get` removed it. + assert_eq!( + s.get("t".to_string()), + Err(KvError::KeyNotFound { + key: "t".to_string() + }) + ); +} + +#[test] +fn capacity_rejects_new_keys_only() { + let s = store(); + s.set_capacity(1); + put(&s, "a", b"1"); + put(&s, "a", b"2"); + let err = Arc::clone(&s) + .put("b".to_string(), vec![], EntryKind::Volatile, None) + .unwrap_err(); + assert_eq!(err.code(), 1003); + assert_eq!(err.message(), "store is full (1 entries)"); + assert_eq!(err.payload(), abi::encode_value(&1u32)); + assert_eq!(Store::default_capacity(), 1_000_000); +} + +#[test] +fn iterators_are_ordered_and_filtered() { + let s = store(); + for k in ["user.bob", "user.alice", "sys.x"] { + put(&s, k, k.as_bytes()); + } + assert_eq!( + keys(&s, None).unwrap(), + vec!["sys.x", "user.alice", "user.bob"] + ); + assert_eq!( + keys(&s, Some("user.")).unwrap(), + vec!["user.alice", "user.bob"] + ); + assert_eq!( + keys(&s, Some("none.")), + Err(KvError::KeyNotFound { + key: "none.".to_string() + }) + ); + let entries = collect(s.entries(Some("sys.".to_string()))); + assert_eq!(entries.len(), 1); + assert_eq!(entries[0].value, b"sys.x"); + + let parts = collect(Arc::clone(&s).partition(vec!["user.".to_string(), "zzz".to_string()])); + assert_eq!(parts.len(), 2); + assert_eq!(parts[0].path(), "user."); + assert_eq!(parts[0].count(), 2); + assert_eq!(parts[1].count(), 0); +} + +#[derive(Default)] +struct Recorder { + changes: Mutex>, + skip: Option, + fail_on: Option, + threads: Mutex>, +} + +impl Listener for Recorder { + fn accepts(&self, key: &str) -> Result { + if self.fail_on.as_deref() == Some(key) { + return Err(ForeignError::new(-4, "listener failed")); + } + Ok(self.skip.as_deref() != Some(key)) + } + + fn on_change(&self, change: &Change) -> Result<(), ForeignError> { + self.threads + .lock() + .unwrap() + .push(std::thread::current().id()); + self.changes.lock().unwrap().push(change.clone()); + Ok(()) + } +} + +#[test] +fn listeners_see_changes_and_detach_on_failure() { + let s = store(); + let rec = Arc::new(Recorder { + skip: Some("quiet".to_string()), + fail_on: Some("boom".to_string()), + ..Recorder::default() + }); + let id = s.subscribe(Arc::clone(&rec) as Arc); + assert_eq!(s.listener_count(), 1); + let entry = put(&s, "a", b"1"); + put(&s, "quiet", b"2"); + s.delete("a".to_string()); + assert_eq!(s.clear(), 1); + assert_eq!( + *rec.changes.lock().unwrap(), + vec![ + Change::Put { + entry, + replaced: false + }, + Change::Removed { + key: "a".to_string(), + expired: false + }, + Change::Cleared { count: 1 }, + ] + ); + put(&s, "boom", b"3"); + assert_eq!(s.listener_count(), 0, "a failing listener is detached"); + assert!(!s.unsubscribe(id)); +} + +struct Gatekeeper { + other: Arc, +} + +impl Policy for Gatekeeper { + fn admit(&self, entry: &Entry) -> Result { + if entry.key.starts_with("secret") { + let e = KvError::Rejected { + key: entry.key.clone(), + reason: "no secrets".to_string(), + }; + return Err(ForeignError { + code: e.code(), + message: e.message(), + payload: e.payload(), + }); + } + let mut admitted = entry.clone(); + admitted.tags.push("admitted".to_string()); + admitted.key = "ignored".to_string(); + Ok(admitted) + } + + fn route(&self, key: &str, home: Arc) -> Result, ForeignError> { + Ok(if key.starts_with("b/") { + Arc::clone(&self.other) + } else { + home + }) + } +} + +#[test] +fn policy_admits_routes_and_rejects() { + let s = store(); + let other = store(); + s.set_policy(Some(Arc::new(Gatekeeper { + other: Arc::clone(&other), + }))); + assert!(s.has_policy()); + let e = put(&s, "a", b"1"); + assert_eq!(e.key, "a", "the policy can't change the key"); + assert_eq!(e.tags, vec!["admitted"]); + put(&s, "b/x", b"2"); + assert_eq!((s.count(), other.count()), (1, 1)); + let err = Arc::clone(&s) + .put("secret".to_string(), vec![], EntryKind::Volatile, None) + .unwrap_err(); + assert_eq!(err.code(), 1005); + assert_eq!( + err.payload(), + KvError::Rejected { + key: "secret".to_string(), + reason: "no secrets".to_string() + } + .payload() + ); + s.set_policy(None); + assert!(!s.has_policy()); +} + +struct Shelf { + fallback: Option>, + missing_key: Option, + fail: bool, +} + +impl Loader for Shelf { + fn name(&self) -> Result { + Ok("shelf".to_string()) + } + + fn fallback(&self, _key: &str) -> Result>, ForeignError> { + Ok(self.fallback.clone()) + } + + fn load(&self, key: &str) -> Result, ForeignError> { + if self.fail { + return Err(ForeignError::new(-4, "shelf is broken")); + } + if let Some(missing) = &self.missing_key { + let e = KvError::KeyNotFound { + key: missing.clone(), + }; + return Err(ForeignError { + code: e.code(), + message: e.message(), + payload: e.payload(), + }); + } + Ok(format!("loaded {key}").into_bytes()) + } +} + +#[test] +fn loaders_fill_misses() { + let s = store(); + assert_eq!(s.get_or_load("k".to_string(), None).unwrap(), None); + + let shelf = |fallback, missing_key: Option<&str>, fail| -> Arc { + Arc::new(Shelf { + fallback, + missing_key: missing_key.map(str::to_string), + fail, + }) + }; + let loaded = s + .get_or_load("k".to_string(), Some(shelf(None, None, false))) + .unwrap() + .unwrap(); + assert_eq!(loaded.value, b"loaded k"); + assert_eq!(loaded.kind, EntryKind::Volatile); + assert_eq!(loaded.metadata["source"], "shelf"); + assert_eq!(s.count(), 1); + + let backup = store(); + put(&backup, "b", b"from backup"); + let copied = s + .get_or_load("b".to_string(), Some(shelf(Some(backup), None, false))) + .unwrap() + .unwrap(); + assert_eq!(copied.value, b"from backup"); + + assert_eq!( + s.get_or_load("x".to_string(), Some(shelf(None, Some("x"), false))) + .unwrap(), + None + ); + let err = s + .get_or_load("y".to_string(), Some(shelf(None, Some("z"), false))) + .unwrap_err(); + assert_eq!(err.code(), 1001); + assert_eq!(err.payload(), abi::encode_value(&"z".to_string())); + let err = s + .get_or_load("y".to_string(), Some(shelf(None, None, true))) + .unwrap_err(); + assert_eq!( + (err.code(), err.message()), + (-4, "shelf is broken".to_string()) + ); +} + +#[test] +fn object_graph() { + let s = store(); + put(&s, "k", b"v"); + let shared = Arc::clone(&s).share(); + assert!(Arc::ptr_eq(&shared, &s)); + let fork = s.fork(); + assert!(!Arc::ptr_eq(&fork, &s)); + put(&fork, "k2", b"v"); + assert_eq!((s.count(), fork.count()), (1, 2)); + + let empty = Arc::new(Store::new()); + assert!(Arc::clone(&empty).larger(None).is_none()); + assert!(Arc::ptr_eq( + &Arc::clone(&empty).larger(Some(Arc::clone(&fork))).unwrap(), + &fork + )); + + let info = Arc::clone(&s).describe("main".to_string(), Some(Arc::clone(&fork))); + assert_eq!(info.count, 1); + let many = Store::open_many(vec!["/a".to_string(), "/b".to_string()]).unwrap(); + assert_eq!(many[1].path(), "/b"); + assert_eq!( + Store::open_many(vec!["/a".to_string(), String::new()]).err(), + Some(KvError::InvalidPath) + ); + let named = Store::by_label(vec![info.clone()]); + assert!(Arc::ptr_eq(&named["main"], &s)); + assert_eq!(Store::total_count(many, named, Some(info)), 2); +} + +#[test] +fn stats_and_report() { + let s = store(); + put(&s, "a", b"12"); + Arc::clone(&s) + .put("b".to_string(), vec![1], EntryKind::Encrypted, None) + .unwrap(); + let stats = summarize(&s, None).unwrap(); + assert_eq!(stats.entries, 2); + assert_eq!(stats.bytes, 3); + assert_eq!( + stats.by_kind, + BTreeMap::from([(EntryKind::Persistent, 1), (EntryKind::Encrypted, 1)]) + ); + assert_eq!( + summarize(&s, Some("q".to_string())), + Err(KvError::KeyNotFound { + key: "q".to_string() + }) + ); + + put(&s, "a", b"123"); + let lines = render_report(collect(s.entries(None))).unwrap(); + assert_eq!( + lines, + vec!["a: 3 bytes, Persistent, v2", "b: 1 bytes, Encrypted"] + ); + assert!(matches!( + render_report(Vec::new()), + Err(ReportError::NothingToReport) + )); +} + +// ── ABI-level tests ──────────────────────────────────────────────────────── + +/// A consumer-side policy built exactly as a generated binding builds one: +/// a heap context plus a process-wide vtable. It rejects every write with a +/// typed error carrying a payload. +struct RejectAll { + freed: Arc, +} + +unsafe extern "C" fn reject_admit( + _ctx: *mut c_void, + entry_ptr: *const u8, + entry_len: usize, + _out_ptr: *mut *mut u8, + _out_len: *mut usize, + out_err: *mut FfiError, +) { + let entry: Entry = + unsafe { abi::decode_value(std::slice::from_raw_parts(entry_ptr, entry_len)) }.unwrap(); + let payload = KvError::Rejected { + key: entry.key, + reason: "read-only".to_string(), + } + .payload(); + unsafe { + crate::kvstore_error_set(out_err, 1005, c"rejected".as_ptr()); + crate::kvstore_error_set_payload(out_err, payload.as_ptr(), payload.len()); + } +} + +unsafe extern "C" fn reject_route( + _ctx: *mut c_void, + _key_ptr: *const u8, + _key_len: usize, + home: *mut Store, + _out_err: *mut FfiError, +) -> *mut Store { + home +} + +unsafe extern "C" fn reject_free(ctx: *mut c_void) { + let state = unsafe { Box::from_raw(ctx.cast::()) }; + state.freed.store(true, Ordering::SeqCst); +} + +static REJECT_VTABLE: kvstore_kv_Policy_vtable = kvstore_kv_Policy_vtable { + header: abi::VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free: reject_free, + }, + admit: reject_admit, + route: reject_route, +}; + +#[test] +fn a_typed_callback_error_reaches_the_caller_with_its_payload() { + let freed = Arc::new(AtomicBool::new(false)); + let mut err = FfiError::default(); + let path = "/tmp/abi"; + let s = unsafe { kvstore_kv_Store_open(path.as_ptr(), path.len(), &mut err) }; + assert_eq!(err.code, 0); + let ctx = Box::into_raw(Box::new(RejectAll { + freed: Arc::clone(&freed), + })); + unsafe { kvstore_kv_Store_set_policy(s, ctx.cast(), &REJECT_VTABLE, &mut err) }; + assert_eq!(err.code, 0); + + let (key, value) = ("k", b"v"); + let ttl = abi::encode_value(&None::); + let mut len = 0usize; + let out = unsafe { + kvstore_kv_Store_put( + s, + key.as_ptr(), + key.len(), + value.as_ptr(), + value.len(), + EntryKind::Volatile as i32, + ttl.as_ptr(), + ttl.len(), + &mut len, + &mut err, + ) + }; + assert!(out.is_null()); + assert_eq!(err.code, 1005); + assert_eq!(unsafe { err.message_str() }, Some("rejected")); + let mut r = abi::BufferReader::token_free(err.payload()); + assert_eq!(r.read_string().unwrap(), "k"); + assert_eq!(r.read_string().unwrap(), "read-only"); + unsafe { abi::error_clear(&mut err) }; + + assert!(!freed.load(Ordering::SeqCst)); + unsafe { kvstore_kv_Store_destroy(s) }; + assert!( + freed.load(Ordering::SeqCst), + "destroying the store frees its policy" + ); +} + +type Done = mpsc::Sender<(i32, u32)>; + +extern "C" fn on_compacted(context: *mut c_void, err: *mut FfiError, result: u32) { + let tx = unsafe { &*(context as *const Done) }.clone(); + let code = if err.is_null() { + 0 + } else { + let code = unsafe { (*err).code }; + unsafe { crate::kvstore_error_free(err) }; + code + }; + tx.send((code, result)).unwrap(); +} + +/// Launch `compact` through its thunk and wait for the completion. +fn compact(s: &Arc, pause_ms: u32, cancel_after: Option) -> (i32, u32) { + let (tx, rx) = mpsc::channel::<(i32, u32)>(); + let tx_ptr = Box::into_raw(Box::new(tx)); + let token = crate::kvstore_cancel_token_create(); + unsafe { + kvstore_kv_Store_compact(Arc::as_ptr(s), pause_ms, token, on_compacted, tx_ptr.cast()); + } + if let Some(delay) = cancel_after { + std::thread::sleep(delay); + unsafe { crate::kvstore_cancel_token_cancel(token) }; + } + unsafe { crate::kvstore_cancel_token_destroy(token) }; + let out = rx.recv_timeout(Duration::from_secs(5)).unwrap(); + drop(unsafe { Box::from_raw(tx_ptr) }); + out +} + +#[test] +fn compaction_notifies_from_a_producer_thread_and_cancels_cooperatively() { + let s = store(); + let rec = Arc::new(Recorder::default()); + s.subscribe(Arc::clone(&rec) as Arc); + Arc::clone(&s) + .put("old".to_string(), vec![1], EntryKind::Volatile, Some(1)) + .unwrap(); + put(&s, "new", b"2"); + s.tick(5); + rec.threads.lock().unwrap().clear(); + + assert_eq!(compact(&s, 0, None), (0, 1)); + let threads = rec.threads.lock().unwrap().clone(); + assert_eq!(threads.len(), 1); + assert_ne!(threads[0], std::thread::current().id()); + + // A long pause, cancelled: the call completes with the cancelled code + // at once, and the background pause stops soon after. + let started = std::time::Instant::now(); + assert_eq!( + compact(&s, 60_000, Some(Duration::from_millis(20))), + (abi::CANCELLED_ERROR_CODE, 0) + ); + assert!(started.elapsed() < Duration::from_secs(5)); + let stopped = (0..2000).any(|_| { + std::thread::sleep(Duration::from_millis(1)); + Store::active_jobs() == 0 + }); + assert!(stopped, "the cancelled pause stopped cooperatively"); +} + +#[test] +fn contract_tables_are_exported() { + assert_eq!(crate::kvstore_abi_version(), abi::ABI_VERSION); + let mut len = 0usize; + assert!(!unsafe { crate::kv::kvstore_kv_contract(&mut len) }.is_null()); + assert!(len > 40); + assert!(!unsafe { crate::report::kvstore_report_contract(&mut len) }.is_null()); + assert_eq!(len, 2); +} + +#[test] +fn listener_vtables_are_released_once() { + static FREED: AtomicUsize = AtomicUsize::new(0); + unsafe extern "C" fn free(_ctx: *mut c_void) { + FREED.fetch_add(1, Ordering::SeqCst); + } + unsafe extern "C" fn accepts( + _ctx: *mut c_void, + _key_ptr: *const u8, + _key_len: usize, + _out_err: *mut FfiError, + ) -> bool { + true + } + unsafe extern "C" fn on_change( + _ctx: *mut c_void, + _ptr: *const u8, + _len: usize, + out_err: *mut FfiError, + ) { + unsafe { crate::kvstore_error_set(out_err, -4, c"nope".as_ptr()) }; + } + static VTABLE: kvstore_kv_Listener_vtable = kvstore_kv_Listener_vtable { + header: abi::VtableHeader { + size: std::mem::size_of::() as u32, + flags: 0, + free, + }, + accepts, + on_change, + }; + let mut err = FfiError::default(); + let s = Arc::new(Store::new()); + let raw = Arc::as_ptr(&s); + let id = unsafe { kvstore_kv_Store_subscribe(raw, std::ptr::null_mut(), &VTABLE, &mut err) }; + assert_eq!((id, err.code), (1, 0)); + put(&s, "k", b"v"); + assert_eq!(s.listener_count(), 0); + assert_eq!(FREED.load(Ordering::SeqCst), 1); +} diff --git a/samples/kvstore/weaveffi.toml b/samples/kvstore/weaveffi.toml index cdfbff76..56786bec 100644 --- a/samples/kvstore/weaveffi.toml +++ b/samples/kvstore/weaveffi.toml @@ -1,17 +1,17 @@ # `weaveffi generate` (no arguments) from anywhere in this crate uses this. [project] -input = "src/lib.rs" +input = "." out = "bindings" # The package name, prefix, and library come from Cargo.toml (`kvstore`); # this table adds the published metadata. [package] version = "1.0.0" -description = "An embedded key-value store API with TTLs, iteration, and async compaction." +description = "An embedded key-value store with listeners, write policies, read-through loaders, iteration, and async compaction." license = "MIT" authors = ["WeaveFoundry "] homepage = "https://github.com/weavefoundry/weaveffi" repository = "https://github.com/weavefoundry/weaveffi" [generators.go] -module_path = "github.com/example/kvstore" +name = "github.com/example/kvstore" diff --git a/scripts/check-fixtures.sh b/scripts/check-fixtures.sh index 7bfd7e70..1456db54 100755 --- a/scripts/check-fixtures.sh +++ b/scripts/check-fixtures.sh @@ -11,6 +11,9 @@ # For each target it generates crates/weaveffi-cli/tests/fixtures/*.yml into a # scratch directory and runs scripts/fixtures/.sh , # which must exit non-zero on any compiler or type-checker error. +# +# A missing toolchain fails the check under CI (CI=true) and is reported as +# a skip otherwise (see scripts/fixtures/lib.sh). set -uo pipefail ROOT=$(cd "$(dirname "$0")/.." && pwd) cd "$ROOT" @@ -23,22 +26,29 @@ WEAVEFFI="${TARGET_DIR:-$ROOT/target}/debug/weaveffi" SCRATCH=$(mktemp -d) trap 'rm -rf "$SCRATCH"' EXIT FAILED="" +SKIPPED="" for fixture in crates/weaveffi-cli/tests/fixtures/*.yml; do name=$(basename "$fixture" .yml) for t in $TARGETS; do out="$SCRATCH/$name" - "$WEAVEFFI" generate "$fixture" -o "$out" --target "c,$t" --force >/dev/null || { + "$WEAVEFFI" generate "$fixture" -o "$out" --target "c,$t" >/dev/null || { FAILED="$FAILED $t:$name(generate)" continue } echo "==> $t: $name" - if ! bash "scripts/fixtures/$t.sh" "$out"; then - FAILED="$FAILED $t:$name" - fi + bash "scripts/fixtures/$t.sh" "$out" + case $? in + 0) ;; + 77) SKIPPED="$SKIPPED $t:$name" ;; + *) FAILED="$FAILED $t:$name" ;; + esac done done +if [ -n "$SKIPPED" ]; then + echo "fixture check skipped (missing tools):$SKIPPED" >&2 +fi if [ -n "$FAILED" ]; then echo "fixture check failed:$FAILED" >&2 exit 1 fi -echo "fixture check passed" +echo "fixture check passed${SKIPPED:+ (with skips)}" diff --git a/scripts/fixtures/c.sh b/scripts/fixtures/c.sh index 19f8cc18..54a42898 100755 --- a/scripts/fixtures/c.sh +++ b/scripts/fixtures/c.sh @@ -2,6 +2,8 @@ # Compile every generated C header (the ABI header and, when present, the # value-buffer helper header) as C11 and C++17 with warnings as errors. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require cc c++ dir=$1 for header in "$dir"/c/*.h; do name=$(basename "$header" .h) diff --git a/scripts/fixtures/cpp.sh b/scripts/fixtures/cpp.sh index 3eaf181f..b331e628 100644 --- a/scripts/fixtures/cpp.sh +++ b/scripts/fixtures/cpp.sh @@ -3,6 +3,8 @@ # ships, as C++17 with warnings as errors. The translation unit includes the # wrapper and then the C header directly, so including both must compile. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require "${CXX:-clang++}" dir=$1 hpp=$(ls "$dir"/cpp/*.hpp | head -1) h=$(ls "$dir"/cpp/*.h | head -1) diff --git a/scripts/fixtures/dart.sh b/scripts/fixtures/dart.sh index 07724587..bf3836c7 100644 --- a/scripts/fixtures/dart.sh +++ b/scripts/fixtures/dart.sh @@ -3,6 +3,8 @@ # warnings fatal. Resolving the `ffi` dependency uses the pub cache when it # can and the network otherwise. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require dart dir=$1 cd "$dir/dart" dart pub get --offline >/dev/null 2>&1 || dart pub get >/dev/null diff --git a/scripts/fixtures/dotnet.sh b/scripts/fixtures/dotnet.sh index 99aa9b35..6aead8c9 100644 --- a/scripts/fixtures/dotnet.sh +++ b/scripts/fixtures/dotnet.sh @@ -3,6 +3,8 @@ # targets net8.0; restore needs that reference pack (cached by any earlier # net8.0 restore, or downloaded from NuGet). set -euo pipefail +. "$(dirname "$0")/lib.sh" +require dotnet dir=$1 csproj=$(ls "$dir"/dotnet/*.csproj | head -1) dotnet build "$csproj" -warnaserror -nologo -v quiet \ diff --git a/scripts/fixtures/go.sh b/scripts/fixtures/go.sh index f1c2f067..9d745502 100755 --- a/scripts/fixtures/go.sh +++ b/scripts/fixtures/go.sh @@ -3,6 +3,8 @@ # C warnings as errors. No producer exists for a fixture, so the module links # against an empty stand-in library with unresolved symbols allowed. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require go gofmt cc dir=$1 mod="$dir/go" header=$(ls "$mod"/*.h | head -1) diff --git a/scripts/fixtures/js.sh b/scripts/fixtures/js.sh index 162e05bb..1910e2e6 100644 --- a/scripts/fixtures/js.sh +++ b/scripts/fixtures/js.sh @@ -2,7 +2,8 @@ # Shared by node.sh and wasm.sh: `js_check ...` runs # `tsc --noEmit --strict` on /index.d.ts and `node --check` on each # module. Without a `tsc` on PATH, TypeScript is installed once into -# ${WEAVEFFI_TSC_DIR:-$TMPDIR/weaveffi-typescript} (this needs the network). +# ${WEAVEFFI_TSC_DIR:-$TMPDIR/weaveffi-typescript} (this needs npm and the +# network). Callers source lib.sh first. tsc_bin() { if command -v tsc >/dev/null 2>&1; then @@ -21,6 +22,7 @@ js_check() { local dir=$1 shift local tsc + command -v tsc >/dev/null 2>&1 || require npm tsc=$(tsc_bin) "$tsc" --noEmit --strict --target es2022 --module es2022 \ --lib es2022,dom,esnext.disposable --types "" "$dir/index.d.ts" diff --git a/scripts/fixtures/kotlin.sh b/scripts/fixtures/kotlin.sh index 7755c989..0dd49fa6 100755 --- a/scripts/fixtures/kotlin.sh +++ b/scripts/fixtures/kotlin.sh @@ -3,6 +3,8 @@ # syntax-check the generated JNI shim against the JDK's JNI headers with # warnings as errors. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require java kotlinc cc dir=$1 kt="$dir/kotlin" @@ -11,7 +13,7 @@ if [ -z "$java_home" ] || [ ! -f "$java_home/include/jni.h" ]; then javabin=$(readlink -f "$(command -v java)" 2>/dev/null || true) [ -n "$javabin" ] && java_home=$(dirname "$(dirname "$javabin")") fi -[ -f "$java_home/include/jni.h" ] || { echo "jni.h not found (set JAVA_HOME)" >&2; exit 1; } +[ -f "$java_home/include/jni.h" ] || missing "jni.h not found (set JAVA_HOME)" case "$(uname -s)" in Darwin) jni_os="$java_home/include/darwin" ;; *) jni_os="$java_home/include/linux" ;; @@ -24,7 +26,7 @@ done kotlinc_bin=$(readlink -f "$(command -v kotlinc)" 2>/dev/null || command -v kotlinc) coroutines=$(ls "$(dirname "$kotlinc_bin")"/../libexec/lib/kotlinx-coroutines-core-jvm.jar \ "$(dirname "$kotlinc_bin")"/../lib/kotlinx-coroutines-core-jvm.jar 2>/dev/null | head -1 || true) -[ -n "$coroutines" ] || { echo "kotlinx-coroutines-core-jvm.jar not found" >&2; exit 1; } +[ -n "$coroutines" ] || missing "kotlinx-coroutines-core-jvm.jar not found next to kotlinc" rm -rf "$dir/kotlin_classes" find "$kt/src/main/kotlin" -name '*.kt' > "$dir/kotlin_sources.txt" kotlinc -Werror -cp "$coroutines" -d "$dir/kotlin_classes" @"$dir/kotlin_sources.txt" diff --git a/scripts/fixtures/lib.sh b/scripts/fixtures/lib.sh new file mode 100644 index 00000000..e47be546 --- /dev/null +++ b/scripts/fixtures/lib.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Shared by every scripts/fixtures/.sh check. +# +# `require ...` stops the check when a tool isn't on PATH, and +# `missing ` does the same for anything else a check needs (JDK +# headers, a node-gyp header cache). Under CI (CI=true), where the +# toolchains action installs everything, that's a failure. Elsewhere the +# check prints a skip note and exits with SKIPPED, which check-fixtures.sh +# reports as a skip rather than a pass. + +SKIPPED=77 + +missing() { + if [ "${CI:-}" = true ]; then + echo "error: $1" >&2 + exit 1 + fi + echo "skip: $1" >&2 + exit "$SKIPPED" +} + +require() { + local tool + for tool in "$@"; do + command -v "$tool" >/dev/null 2>&1 || missing "$tool not found" + done +} diff --git a/scripts/fixtures/node.sh b/scripts/fixtures/node.sh index 38e68aae..5c36e8ff 100644 --- a/scripts/fixtures/node.sh +++ b/scripts/fixtures/node.sh @@ -1,27 +1,32 @@ #!/usr/bin/env bash # Check the generated Node.js package: type-check index.d.ts with # `tsc --strict`, parse every ES module with `node --check`, and compile the -# N-API addon with warnings as errors against the cached node-gyp headers -# (skipped, with a note, when no headers are cached). +# N-API addon with warnings as errors against the Node.js headers (see +# lib.sh for what happens when there are none). set -euo pipefail dir=$1/node here=$(cd "$(dirname "$0")" && pwd) +. "$here/lib.sh" . "$here/js.sh" +require node cc js_check "$dir" index.js runtime.js +# The headers node-gyp caches, or the ones an official Node.js distribution +# (nvm, actions/setup-node) ships next to its binary. +version=$(node -p 'process.versions.node') +node_prefix=$(dirname "$(dirname "$(readlink -f "$(command -v node)" 2>/dev/null || command -v node)")") headers="" -for base in "$HOME/Library/Caches/node-gyp" "$HOME/.cache/node-gyp" "${LOCALAPPDATA:-/nonexistent}/node-gyp/Cache"; do - candidate="$base/$(node -p 'process.versions.node')/include/node" +for candidate in "$HOME/Library/Caches/node-gyp/$version/include/node" \ + "$HOME/.cache/node-gyp/$version/include/node" \ + "${LOCALAPPDATA:-/nonexistent}/node-gyp/Cache/$version/include/node" \ + "$node_prefix/include/node"; do if [ -f "$candidate/node_api.h" ]; then headers=$candidate break fi done -if [ -z "$headers" ]; then - echo "note: no node-gyp headers cached; skipping the addon compile check" >&2 - exit 0 -fi +[ -n "$headers" ] || missing "no Node.js $version headers found (npx node-gyp install)" for c in "$dir"/*_node.c; do cc -std=c11 -Wall -Wextra -Werror -fsyntax-only -I "$headers" -I "$dir" "$c" done diff --git a/scripts/fixtures/python.sh b/scripts/fixtures/python.sh index c49ba30f..36c3ccfc 100755 --- a/scripts/fixtures/python.sh +++ b/scripts/fixtures/python.sh @@ -1,9 +1,13 @@ #!/usr/bin/env bash -# Byte-compile every generated Python file, then type-check the package with -# mypy when it is available: the `.pyi` stub under --strict and the -# implementation module itself (which catches undefined names and bad calls -# into the runtime). Without mypy, only the byte-compile runs. +# Byte-compile every generated Python file, then type-check each package +# with mypy --strict: the implementation module is fully annotated (it ships +# `py.typed` and no separate stub), so this checks both the public surface +# consumers type against and the bodies (undefined names, bad calls into the +# runtime). A missing mypy is handled as lib.sh describes, after the +# byte-compile has run. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require python3 dir=$1 find "$dir/python" -name '*.py' -print0 | xargs -0 python3 -m py_compile if command -v mypy >/dev/null 2>&1; then @@ -11,11 +15,8 @@ if command -v mypy >/dev/null 2>&1; then elif python3 -c "import mypy" >/dev/null 2>&1; then mypy=(python3 -m mypy) else - echo "note: mypy not found; type checks skipped (pip install mypy)" - exit 0 + missing "mypy not found; type checks not run (pip install 'mypy<2')" fi for pkg in "$dir"/python/*/; do - name=$(basename "$pkg") - "${mypy[@]}" --strict --python-version 3.9 --no-incremental "$pkg$name.pyi" - "${mypy[@]}" --python-version 3.9 --no-incremental "$pkg$name.py" + "${mypy[@]}" --strict --python-version 3.9 --no-incremental "$pkg" done diff --git a/scripts/fixtures/ruby.sh b/scripts/fixtures/ruby.sh index 2a417f0e..25ed9de8 100644 --- a/scripts/fixtures/ruby.sh +++ b/scripts/fixtures/ruby.sh @@ -2,6 +2,8 @@ # Syntax-check every generated Ruby file with warnings enabled (any warning # fails), then build the gem from its generated gemspec. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require ruby gem dir=$(cd "$1" && pwd) status=0 while IFS= read -r file; do diff --git a/scripts/fixtures/swift.sh b/scripts/fixtures/swift.sh index d56e653e..d0d59b4a 100644 --- a/scripts/fixtures/swift.sh +++ b/scripts/fixtures/swift.sh @@ -3,6 +3,8 @@ # wrapper sources must compile against the bundled C module map with # warnings as errors, in both the Swift 5 and Swift 6 language modes. set -euo pipefail +. "$(dirname "$0")/lib.sh" +require swift swiftc dir=$1 pkg="$dir/swift" cdir=$(ls -d "$pkg"/Sources/C*/ | head -1) diff --git a/scripts/fixtures/wasm.sh b/scripts/fixtures/wasm.sh index 0ef74e78..2b7f469d 100644 --- a/scripts/fixtures/wasm.sh +++ b/scripts/fixtures/wasm.sh @@ -4,6 +4,8 @@ set -euo pipefail dir=$1/wasm here=$(cd "$(dirname "$0")" && pwd) +. "$here/lib.sh" . "$here/js.sh" +require node js_check "$dir" index.js runtime.js linear.js diff --git a/scripts/package-smoke.sh b/scripts/package-smoke.sh new file mode 100755 index 00000000..c72a0a4f --- /dev/null +++ b/scripts/package-smoke.sh @@ -0,0 +1,159 @@ +#!/usr/bin/env bash +# Install smoke test for `weaveffi package`. +# +# Builds samples/calculator for the host, generates its bindings, packages +# them, then installs each artifact with its ecosystem's own tool into a +# throwaway environment and calls `add(2, 3)` through it: +# +# python the wheel, pip-installed into a fresh venv +# node the per-platform and main npm tarballs, npm-installed into a +# temporary project (and checked to load the prebuilt addon, so +# nothing compiled at install time) +# swift the XCFramework archive unzipped beside the packaged SwiftPM +# package, used by path from a temporary executable package +# (macOS only) +# +# Usage: scripts/package-smoke.sh [target ...] +# (default: python node, plus swift on macOS) +# +# A missing tool fails under CI (CI=true) and skips that target otherwise. +set -euo pipefail +ROOT=$(cd "$(dirname "$0")/.." && pwd) +cd "$ROOT" + +if [ $# -gt 0 ]; then + TARGETS="$*" +elif [ "$(uname -s)" = Darwin ]; then + TARGETS="python node swift" +else + TARGETS="python node" +fi + +cargo build -q -p weaveffi-cli +TARGET_DIR=$(cargo metadata --format-version 1 --no-deps 2>/dev/null \ + | tr ',' '\n' | grep '"target_directory"' | head -1 \ + | sed 's/.*"target_directory":"//; s/"$//') +WEAVEFFI="${TARGET_DIR:-$ROOT/target}/debug/weaveffi" + +SCRATCH=$(mktemp -d) +trap 'rm -rf "$SCRATCH"' EXIT + +SKIPPED="" +FAILED="" + +# `have ...`: true when every tool is on PATH. A missing tool fails +# the run under CI and skips the current target otherwise. +have() { + local tool + for tool in "$@"; do + if ! command -v "$tool" >/dev/null 2>&1; then + if [ "${CI:-}" = true ]; then + echo "error: $tool not found" >&2 + exit 1 + fi + echo "skip: $tool not found" >&2 + return 1 + fi + done +} + +cat >"$SCRATCH/weaveffi.toml" < generate ($TARGET_LIST)" +"$WEAVEFFI" generate --config "$CONFIG" --target "$TARGET_LIST" +echo "==> package ($TARGET_LIST)" +"$WEAVEFFI" package --config "$CONFIG" --target "$TARGET_LIST" + +smoke_python() { + have python3 || return 77 + local wheel + wheel=$(ls "$DIST"/python/calculator-1.0.0-py3-none-*.whl) + python3 -m venv "$SCRATCH/venv" + "$SCRATCH/venv/bin/python" -m pip install --quiet --disable-pip-version-check "$wheel" + (cd "$SCRATCH" && venv/bin/python -c ' +import calculator +assert calculator.add(2, 3) == 5, calculator.add(2, 3) +print("python: calculator.add(2, 3) == 5") +') +} + +smoke_node() { + have node npm || return 77 + local app="$SCRATCH/node-app" + mkdir -p "$app" + printf '{"name":"smoke","version":"0.0.0","private":true,"type":"module"}\n' >"$app/package.json" + (cd "$app" && npm install --no-audit --no-fund --loglevel=error "$DIST"/node/*.tgz) + if [ -d "$app/node_modules/calculator/build" ]; then + echo "error: npm install compiled the addon instead of using the prebuilt one" >&2 + return 1 + fi + (cd "$app" && node --input-type=module -e ' +import { calculator } from "calculator"; +const sum = calculator.add(2, 3); +if (sum !== 5) throw new Error(`add(2, 3) returned ${sum}`); +console.log("node: calculator.add(2, 3) === 5"); +') +} + +smoke_swift() { + have swift unzip || return 77 + local pkg="$DIST/swift/Calculator" app="$SCRATCH/swift-app" + unzip -q -o "$DIST/swift/CCalculator.xcframework.zip" -d "$pkg" + mkdir -p "$app/Sources/Smoke" + cat >"$app/Package.swift" <"$app/Sources/Smoke/main.swift" <<'EOF' +import Calculator + +let sum = Calculator.add(a: 2, b: 3) +precondition(sum == 5, "add(2, 3) returned \(sum)") +print("swift: Calculator.add(a: 2, b: 3) == 5") +EOF + (cd "$app" && swift run --quiet) +} + +for t in $TARGETS; do + echo "==> $t" + set +e + ( set -e; "smoke_$t" ) + status=$? + set -e + case $status in + 0) ;; + 77) SKIPPED="$SKIPPED $t" ;; + *) FAILED="$FAILED $t" ;; + esac +done + +[ -n "$SKIPPED" ] && echo "skipped (missing toolchain):$SKIPPED" +if [ -n "$FAILED" ]; then + echo "FAILED:$FAILED" >&2 + exit 1 +fi +echo "package smoke passed for: $TARGETS" diff --git a/weaveffi.schema.json b/weaveffi.schema.json index b389c4f3..65a6287b 100644 --- a/weaveffi.schema.json +++ b/weaveffi.schema.json @@ -9,7 +9,7 @@ ], "properties": { "version": { - "description": "IR schema version this document targets (for example `0.10.0`). Validation rejects any value not listed in [`SUPPORTED_VERSIONS`].", + "description": "IR schema version this document targets (for example `0.11.0`). Validation rejects any value not listed in [`SUPPORTED_VERSIONS`].", "type": "string" }, "modules": { @@ -186,7 +186,7 @@ }, "TypeRef": { "title": "TypeRef", - "description": "Reference to a type. Encoded as a string with custom syntax: primitives (`i8`, `i16`, `i32`, `i64`, `u8`, `u16`, `u32`, `u64`, `f32`, `f64`, `bool`, `string`, `bytes`), iterators (`iter<{T}>`), lists (`[{T}]`), maps (`{{K:V}}`), optionals (`{T}?`), or any user-defined struct, enum, interface, or callback interface name.", + "description": "Reference to a type. Encoded as a string with custom syntax: primitives (`i8`, `i16`, `i32`, `i64`, `u8`, `u16`, `u32`, `u64`, `f32`, `f64`, `bool`, `string`, `bytes`), iterators (`iter<{T}>`), lists (`[{T}]`), maps (`{{K:V}}`), optionals (`{T}?`), or the bare (unqualified) name of a user-defined struct, enum, interface, or callback interface.", "type": "string" }, "InterfaceDef": { From 76fd5160f7e14a14ccdd211914d2e55ab87d657f Mon Sep 17 00:00:00 2001 From: Owen Carey <37121709+owenthcarey@users.noreply.github.com> Date: Thu, 8 Oct 2026 16:00:10 -0700 Subject: [PATCH 2/2] fix: make CI pass on Linux and with wrapped diagnostics - Define _GNU_SOURCE on Linux in the Node.js addon so libuv's headers compile under a strict -std=c11. - Find the Swift module beside its C shim instead of relying on ls order, which differs by locale. - Check that Swift listeners notify on the subscribing thread with pthread_equal instead of Thread.isMainThread, which is false for top-level async code on Linux. - Match CLI error messages with miette's line wrapping undone. --- conformance/swift/kvstore.swift | 6 +++++- conformance/swift/run.sh | 11 ++++++---- .../src/targets/node/runtime/addon.c | 5 +++++ crates/weaveffi-cli/tests/cli/extract.rs | 6 +++--- crates/weaveffi-cli/tests/cli/main.rs | 11 ++++++++++ crates/weaveffi-cli/tests/cli/package.rs | 20 +++++-------------- .../weaveffi-cli/tests/cli/project_config.rs | 2 +- ...cs_everywhere__docs_everywhere_node_c.snap | 5 +++++ .../node_edge_cases__edge_cases_node_c.snap | 5 +++++ ...ode_kitchen_sink__kitchen_sink_node_c.snap | 5 +++++ ...nested_modules__nested_modules_node_c.snap | 5 +++++ .../snapshots/node_shapes__shapes_node_c.snap | 5 +++++ 12 files changed, 62 insertions(+), 24 deletions(-) diff --git a/conformance/swift/kvstore.swift b/conformance/swift/kvstore.swift index 6e0659a6..a38bb9da 100644 --- a/conformance/swift/kvstore.swift +++ b/conformance/swift/kvstore.swift @@ -121,6 +121,10 @@ func open(_ path: String) -> Store { /// A thread-safe tally shared by the test and the callbacks it creates. final class Tally: @unchecked Sendable { + /// The thread that created the tally: synchronous calls must notify on + /// it. (`Thread.isMainThread` isn't reliable for this on Linux, where + /// top-level async code doesn't run on the main thread.) + let origin = pthread_self() private let lock = NSLock() private var counts: [String: Int] = [:] private var notes: [String: String] = [:] @@ -184,7 +188,7 @@ final class RecordingListener: Listener, @unchecked Sendable { } func onChange(change: Change) throws { - if !Thread.isMainThread { tally.add("offMain") } + if pthread_equal(pthread_self(), tally.origin) == 0 { tally.add("offMain") } switch change { case let .put(entry, replaced): tally.note("version", String(entry.version)) diff --git a/conformance/swift/run.sh b/conformance/swift/run.sh index 7a278fe4..9d93b6ea 100755 --- a/conformance/swift/run.sh +++ b/conformance/swift/run.sh @@ -15,10 +15,13 @@ swift_consumer() { local sample="$1" src="$2" local gen="$GENROOT/$sample/swift" local pkg="$OUT/swift-$sample" - local cdir mod - cdir=$(ls -d "$gen"/Sources/C*/ | head -1) - mod=$(basename "$cdir") - mod=${mod#C} + # The Swift module is the target whose C shim `C` sits beside it + # (`ls` order differs across locales, so don't rely on sorting). + local dir name mod="" + for dir in "$gen"/Sources/*/; do + name=$(basename "$dir") + if [ -d "$gen/Sources/C$name" ]; then mod=$name; fi + done rm -rf "$pkg" mkdir -p "$pkg/Sources/conformance" cp "$ROOT/conformance/swift/$src" "$pkg/Sources/conformance/main.swift" diff --git a/crates/weaveffi-cli/src/targets/node/runtime/addon.c b/crates/weaveffi-cli/src/targets/node/runtime/addon.c index 03b99b85..0f18e49c 100644 --- a/crates/weaveffi-cli/src/targets/node/runtime/addon.c +++ b/crates/weaveffi-cli/src/targets/node/runtime/addon.c @@ -1,3 +1,8 @@ +/* libuv's Unix headers use POSIX types (pthread_rwlock_t, struct addrinfo) + * that glibc hides under a strict -std=c11; ask for them explicitly. */ +#if defined(__linux__) && !defined(_GNU_SOURCE) +#define _GNU_SOURCE +#endif #ifndef NAPI_VERSION #define NAPI_VERSION 8 #endif diff --git a/crates/weaveffi-cli/tests/cli/extract.rs b/crates/weaveffi-cli/tests/cli/extract.rs index 22e39931..e230924c 100644 --- a/crates/weaveffi-cli/tests/cli/extract.rs +++ b/crates/weaveffi-cli/tests/cli/extract.rs @@ -126,7 +126,7 @@ fn extract_explains_what_it_cannot_read() { let idl = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/kitchen_sink.yml"); let output = weaveffi().arg("extract").arg(&idl).output().unwrap(); assert!(!output.status.success()); - let err = String::from_utf8_lossy(&output.stderr); + let err = crate::stderr(&output); assert!(err.contains("already an IDL"), "{err}"); // The CLI binary itself is a real executable with no metadata. @@ -137,7 +137,7 @@ fn extract_explains_what_it_cannot_read() { .output() .unwrap(); assert!(!output.status.success()); - let err = String::from_utf8_lossy(&output.stderr); + let err = crate::stderr(&output); assert!(err.contains("no WeaveFFI metadata"), "{err}"); assert!(err.contains("export_runtime!"), "{err}"); @@ -147,6 +147,6 @@ fn extract_explains_what_it_cannot_read() { .output() .unwrap(); assert!(!output.status.success()); - let err = String::from_utf8_lossy(&output.stderr); + let err = crate::stderr(&output); assert!(err.contains("pass the crate"), "{err}"); } diff --git a/crates/weaveffi-cli/tests/cli/main.rs b/crates/weaveffi-cli/tests/cli/main.rs index 9df62011..3c42ff62 100644 --- a/crates/weaveffi-cli/tests/cli/main.rs +++ b/crates/weaveffi-cli/tests/cli/main.rs @@ -14,3 +14,14 @@ mod no_silent_stubs; mod package; mod project_config; mod schema; + +/// A command's stderr with miette's wrapping undone: box-drawing gutters +/// removed and whitespace runs collapsed, so a message matches as one line +/// whatever width the report was wrapped to. +fn stderr(output: &std::process::Output) -> String { + String::from_utf8_lossy(&output.stderr) + .replace(['│', '×'], " ") + .split_whitespace() + .collect::>() + .join(" ") +} diff --git a/crates/weaveffi-cli/tests/cli/package.rs b/crates/weaveffi-cli/tests/cli/package.rs index f0162b15..1a96ba0f 100644 --- a/crates/weaveffi-cli/tests/cli/package.rs +++ b/crates/weaveffi-cli/tests/cli/package.rs @@ -65,16 +65,6 @@ fn weaveffi() -> assert_cmd::Command { assert_cmd::Command::cargo_bin("weaveffi").expect("binary not found") } -/// The command's stderr with miette's wrapping undone: box-drawing gutters -/// removed and whitespace runs collapsed, so messages match as one line. -fn stderr(output: &std::process::Output) -> String { - String::from_utf8_lossy(&output.stderr) - .replace(['│', '×'], " ") - .split_whitespace() - .collect::>() - .join(" ") -} - /// The `(path, bytes)` entries of a gzipped tarball. fn untar_gz(path: &Path) -> Vec<(String, Vec)> { let file = std::fs::File::open(path).unwrap_or_else(|e| panic!("{}: {e}", path.display())); @@ -195,7 +185,7 @@ fn package_reports_targets_with_no_platform() { .output() .unwrap(); assert!(!output.status.success()); - let err = stderr(&output); + let err = crate::stderr(&output); assert!(err.contains("no artifacts for wasm"), "{err}"); assert!( err.contains("none of the selected targets produced"), @@ -215,7 +205,7 @@ fn package_names_the_missing_platform_library() { .output() .unwrap(); assert!(!output.status.success()); - let err = stderr(&output); + let err = crate::stderr(&output); assert!(err.contains("libcalculator.so"), "{err}"); assert!( err.contains("weaveffi build --platforms android-arm64"), @@ -238,7 +228,7 @@ fn build_without_a_cargo_toml_explains_the_alternatives() { .output() .unwrap(); assert!(!output.status.success(), "{command} succeeded"); - let err = stderr(&output); + let err = crate::stderr(&output); assert!(err.contains("no Cargo.toml at"), "{command}: {err}"); assert!(err.contains("--binaries"), "{command}: {err}"); } @@ -257,7 +247,7 @@ fn build_rejects_unknown_and_foreign_platforms_before_compiling() { .output() .unwrap(); assert!(!output.status.success()); - let err = stderr(&output); + let err = crate::stderr(&output); assert!(err.contains("unknown platform `solaris-sparc`"), "{err}"); assert!(err.contains("darwin-arm64"), "{err}"); @@ -271,7 +261,7 @@ fn build_rejects_unknown_and_foreign_platforms_before_compiling() { .output() .unwrap(); assert!(!output.status.success()); - let err = stderr(&output); + let err = crate::stderr(&output); assert!(err.contains("can only be built on a"), "{err}"); assert!(!err.contains("Compiling"), "checks run before cargo: {err}"); } diff --git a/crates/weaveffi-cli/tests/cli/project_config.rs b/crates/weaveffi-cli/tests/cli/project_config.rs index b9883a28..a000130a 100644 --- a/crates/weaveffi-cli/tests/cli/project_config.rs +++ b/crates/weaveffi-cli/tests/cli/project_config.rs @@ -145,7 +145,7 @@ fn rust_source_input_points_at_the_crate() { .output() .unwrap(); assert!(!output.status.success()); - let stderr = String::from_utf8_lossy(&output.stderr); + let stderr = crate::stderr(&output); assert!(stderr.contains("pass the crate"), "{stderr}"); } diff --git a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap index e9463554..0c9ef81e 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_docs_everywhere__docs_everywhere_node_c.snap @@ -5,6 +5,11 @@ source: crates/weaveffi-cli/tests/snapshots.rs // DO NOT EDIT. Your changes will be overwritten. // To regenerate: weaveffi generate +/* libuv's Unix headers use POSIX types (pthread_rwlock_t, struct addrinfo) + * that glibc hides under a strict -std=c11; ask for them explicitly. */ +#if defined(__linux__) && !defined(_GNU_SOURCE) +#define _GNU_SOURCE +#endif #ifndef NAPI_VERSION #define NAPI_VERSION 8 #endif diff --git a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap index b0346180..78ff2d0c 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_edge_cases__edge_cases_node_c.snap @@ -5,6 +5,11 @@ source: crates/weaveffi-cli/tests/snapshots.rs // DO NOT EDIT. Your changes will be overwritten. // To regenerate: weaveffi generate +/* libuv's Unix headers use POSIX types (pthread_rwlock_t, struct addrinfo) + * that glibc hides under a strict -std=c11; ask for them explicitly. */ +#if defined(__linux__) && !defined(_GNU_SOURCE) +#define _GNU_SOURCE +#endif #ifndef NAPI_VERSION #define NAPI_VERSION 8 #endif diff --git a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap index 80c9012c..29c1081a 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_kitchen_sink__kitchen_sink_node_c.snap @@ -5,6 +5,11 @@ source: crates/weaveffi-cli/tests/snapshots.rs // DO NOT EDIT. Your changes will be overwritten. // To regenerate: weaveffi generate +/* libuv's Unix headers use POSIX types (pthread_rwlock_t, struct addrinfo) + * that glibc hides under a strict -std=c11; ask for them explicitly. */ +#if defined(__linux__) && !defined(_GNU_SOURCE) +#define _GNU_SOURCE +#endif #ifndef NAPI_VERSION #define NAPI_VERSION 8 #endif diff --git a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap index df1c3444..b20b1f76 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_nested_modules__nested_modules_node_c.snap @@ -5,6 +5,11 @@ source: crates/weaveffi-cli/tests/snapshots.rs // DO NOT EDIT. Your changes will be overwritten. // To regenerate: weaveffi generate +/* libuv's Unix headers use POSIX types (pthread_rwlock_t, struct addrinfo) + * that glibc hides under a strict -std=c11; ask for them explicitly. */ +#if defined(__linux__) && !defined(_GNU_SOURCE) +#define _GNU_SOURCE +#endif #ifndef NAPI_VERSION #define NAPI_VERSION 8 #endif diff --git a/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap b/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap index 11c4480f..5ac8e146 100644 --- a/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap +++ b/crates/weaveffi-cli/tests/snapshots/node_shapes__shapes_node_c.snap @@ -5,6 +5,11 @@ source: crates/weaveffi-cli/tests/snapshots.rs // DO NOT EDIT. Your changes will be overwritten. // To regenerate: weaveffi generate +/* libuv's Unix headers use POSIX types (pthread_rwlock_t, struct addrinfo) + * that glibc hides under a strict -std=c11; ask for them explicitly. */ +#if defined(__linux__) && !defined(_GNU_SOURCE) +#define _GNU_SOURCE +#endif #ifndef NAPI_VERSION #define NAPI_VERSION 8 #endif