diff --git a/.agents/notes/DECISIONS.md b/.agents/notes/DECISIONS.md deleted file mode 100644 index c67196e..0000000 --- a/.agents/notes/DECISIONS.md +++ /dev/null @@ -1,206 +0,0 @@ -# Decisions Log - Ladybug C# Binding - -Append-only record of decisions: the choice, the alternatives, and the rationale. -Keep this updated whenever a decision is made. - -## D1 - Package id and namespace: `LadybugDB` -- Alternatives: `Ladybug`, `Kuzu`-style, `Cloud.*`-style. -- Rationale: matches the project brand (`com.ladybugdb` Java group, `@ladybugdb/core` npm). Root namespace and NuGet PackageId both `LadybugDB`. - -## D2 - Type naming follows the Java binding -- `DataType` + `DataTypeId` (not `LogicalType`). Mirrors `com.ladybugdb` so cross-language users transfer knowledge. The managed value is exposed as a graph of CLR objects rather than a raw handle. - -## D3 - Target frameworks: `net10.0` (primary) + `netstandard2.0` (required reach) -- `LangVersion` = `latest` (C# 14) on all targets. `net10.0` is the modern, AOT-friendly primary; `netstandard2.0` is a hard requirement for broad/.NET Framework reach (per product direction). `net8.0` may be added later if useful. - -## D4 - Interop strategy is per-TFM -- `net7.0+`: source-generated `[LibraryImport]`, `nint`-free blittable structs, `StringMarshalling.Utf8`, and a `NativeLibrary.SetDllImportResolver` registered via `[ModuleInitializer]`. -- `netstandard2.0`: classic `[DllImport]` (Cdecl) with manual UTF-8 marshaling (no `LibraryImport`, no `UnmanagedType.LPUTF8Str`, no `NativeLibrary` available). Relies on the OS loader; on Windows the native file is `lbug_shared.dll`, which `DllImport("lbug_shared")` finds directly. -- Single canonical DllImport name `lbug_shared`. On modern runtimes the resolver remaps to `liblbug.so` / `liblbug.dylib` on Linux/macOS. - -## D5 - ABI correctness -- C `bool` (`_Bool`, 1 byte) is marshaled as `byte` in structs and `[return: MarshalAs(UnmanagedType.U1)] bool` for returns - never the default 4-byte .NET `bool`. -- `lbug_system_config` includes the macOS-only trailing `thread_qos` (`uint`) field always. This keeps the managed struct 56 bytes on every platform (on non-Apple it occupies what is otherwise tail padding), so the by-value struct ABI matches natively everywhere. -- `lbug_state` and `lbug_data_type_id` are marshaled as `int`-backed enums. - -## D6 - Owned C strings -- Functions returning `char*` are declared to return `IntPtr`; we copy to a managed `string` and free via `lbug_destroy_string` (helper `Native.TakeString`). A source-generated custom marshaller is a later refinement (Phase 2/3). - -## D7 - Value model -- Public API returns a fully-owned managed value (Rust-style), not live `lbug_value*` handles. Result rows are materialized eagerly to neutralize the documented flat-tuple reuse hazard. - -## D8 - Query failure surfaces as an exception (idiomatic .NET) -- `Connection.Query` throws `LadybugQueryException` when the query fails (covers both `lbug_state == Error` and `QueryResult.IsSuccess == false`), which is the common .NET idiom (cf. ADO.NET). `QueryResult.IsSuccess` / `GetErrorMessage()` remain available for callers who prefer to inspect. This is a deliberate, documented deviation from the raw two-level C contract. - -## D9 - Calling convention / architectures -- Targeting `x64` and `arm64` RIDs only (no `x86`), so the default platform calling convention is unambiguous; `[DllImport]` uses `CallingConvention.Cdecl` for the `netstandard2.0` path. - -## D10 - Disposal and thread-safety (no finalizers / no SafeHandle yet) -- Long-lived disposables (`Database`, `Connection`, `QueryResult`, `PreparedStatement`) use `Interlocked.Exchange` for retry-safe, idempotent `Dispose`; reads use `Volatile.Read`. -- `Connection` serializes `Query`/`Prepare`/`Execute`/`Dispose` with a private lock (cross-TFM `object` lock rather than `System.Threading.Lock`, which is net9+ only) to protect the engine's reused flat-tuple buffer. -- Deliberately NO finalizers and NO `SafeHandle`: native results/values hold engine resources tied to their parent connection/db, so non-deterministic finalization order risks use-after-free. Explicit `using`/`Dispose` is required (documented). A `SafeHandle`-based model remains possible future work but is not worth the ordering hazards now. - -## D11 - Test + solution tooling -- Solution uses the new `.slnx` format (default in the .NET 10 SDK). -- xUnit v2 (2.9.x) + `Xunit.SkippableFact` for graceful skipping when the native library is absent (xUnit v2 lacks `Assert.Skip`, which is v3-only). Tests build everywhere and skip rather than fail without native bits. - -## D12 - INT128 / DateOnly mapping differ by TFM -- `net7.0+` materializes INT128 as `System.Int128` and DATE as `DateOnly`; `netstandard2.0` uses `System.Numerics.BigInteger` and `DateTime` respectively (those BCL types are unavailable on ns2.0). `record` types are enabled on ns2.0 via an `IsExternalInit` polyfill. - -## D13 - ABI guard tests instead of a SafeHandle rewrite (P/Invoke review pass) -- Reviewed the whole interop surface against the `dotnet-pinvoke` skill checklist. Findings: signatures match `lbug.h` exactly (no `size_t`/C `long`/`wchar_t` anywhere, so no `CLong`/`nuint` needed); string encoding is explicit UTF-8 on both paths; memory ownership matches the library's own `lbug_destroy_string`/`lbug_destroy_blob`; no naked `bool` (all `UnmanagedType.U1` / `byte`); both TFMs build with 0 SYSLIB interop warnings. -- Added native-independent struct-layout tests (`StructLayoutTests`) asserting `Marshal.SizeOf`/`Marshal.OffsetOf` for every struct that crosses the boundary (the skill's "verify struct sizes match" step). Required `InternalsVisibleTo("LadybugDB.Tests")`. These 17 tests run (do not skip) even without the native library, locking the ABI down at build time. -- Re-affirmed D10: keep explicit `Dispose` over `SafeHandle`. The skill recommends `SafeHandle`, but its finalizer-driven, non-deterministic release reintroduces the parent/child ordering hazard (value -> tuple -> result -> connection -> database). The skill's own review guidance is "don't rewrite working code"; revisit only if a finalizer safety-net is later deemed worth the ordering risk. - -## D14 - Local Windows native build (MSVC + Ninja + pip CMake) -- Compiler: MSVC `14.51` (VS Community 2026) over Ninja. Clang 22 is also installed and supported by the - CMakeLists' Windows branch, but MSVC is the project's best-tuned Windows path, so it's the default. -- Build tooling: CMake `4.3.2` + Ninja `1.13.0` via `pip install --user`. CMake 4.x rejects the ancient - `cmake_minimum_required` of several vendored deps, so configure with `-DCMAKE_POLICY_VERSION_MINIMUM=3.5`. -- Build the C-API only: `-DBUILD_SHELL=OFF -DBUILD_SINGLE_FILE_HEADER=OFF -DBUILD_STATIC_LBUG=OFF -DBUILD_TESTS=OFF`, - target `lbug_shared` (~1046 steps, ~8 min, 18 MB DLL at `build//src/lbug_shared.dll`). -- Repeatable via `scripts/build-native-and-test.ps1` (imports vcvars64, puts cmake/ninja on PATH, - configures/builds/stages/tests). Verified end-to-end: 27/27 tests pass on win-x64. - -## D15 - CI build + publish (reuse precompiled-bin, multi-RID pack, trusted publishing) -- Cross-platform natives: rather than hand-roll a per-RID native matrix, the C# release workflow - (`.github/workflows/csharp-release.yml`) REUSES the repo's canonical `precompiled-bin-workflow.yml` - via `workflow_call`. That workflow already encodes the hard platform knowledge - manylinux_2_28 for - broad glibc compat, `-static-libstdc++`, macOS deployment target, vcvars on Windows - and already - emits shared libs for exactly the 5 RIDs we ship: `liblbug-{windows-x86_64, linux-x86_64, - linux-aarch64, osx-x86_64, osx-arm64}`. Trade-off: it also builds static libs + the CLI we ignore, - but DRY + known-good natives outweigh the extra build time. A lean shared-only matrix is the - fallback if decoupling is ever needed. -- RID mapping (artifact -> package path): windows-x86_64->win-x64 (`lbug_shared.dll`), - linux-x86_64->linux-x64 + linux-aarch64->linux-arm64 (`liblbug.so`), osx-x86_64->osx-x64 + - osx-arm64->osx-arm64 (`liblbug.dylib`). The `.so`/`.dylib` tarballs ship a symlink chain - (`liblbug.so` -> `.so.0` -> `.so.0.x.y`); we `cp -L` the real object into a single regular file, - because the binding loads it by path (the SONAME/install_name is irrelevant for a path load) and - NuGet does not preserve symlinks. The C API is exported on every platform (`LBUG_C_API` = - `extern "C" __attribute__((visibility("default")))` / `__declspec(dllexport)`), so P/Invoke resolves. -- Single multi-RID package: one job downloads all native artifacts, stages them into - `lib/runtimes//native/`, and packs once. The existing `nuget-package.props` glob already turns - that into `runtimes//native/*` in the `.nupkg`. The pack job ASSERTS all 5 natives + both - managed TFMs are present in the `.nupkg` so a silent path/glob regression fails the build instead of - shipping a managed-only package. -- Publish gate: before packing, the workflow runs the full xUnit suite on the linux-x64 runner against - the freshly built `liblbug.so` (manylinux build loads fine on newer-glibc ubuntu-latest). Publish - only happens on a `csharp-v*` tag; manual `workflow_dispatch` builds + packs + uploads the artifact - but does NOT publish. -- Versioning from the tag: `csharp-v1.2.3` -> `-p:Version=1.2.3`. No file edits; the package version is - decoupled from the engine's CMake version and from the main repo's GitHub-release pipeline (which - triggers on `release: created`, not on `csharp-v*` tags, so there is no collision). -- Trusted publishing (OIDC), not API keys: `NuGet/login@v1` + `id-token: write` + `environment: release`, - pushing with the short-lived token (`--skip-duplicate` keeps re-runs idempotent). One-time nuget.org - setup is required (policy: owner=LadybugDB, repo=ladybug, workflow=`csharp-release.yml`, env=`release`; - plus a `release` GitHub environment with secret `NUGET_USER` = nuget.org profile name). -- A separate lightweight `csharp-ci.yml` builds both TFMs + runs the managed/ABI suite (native - round-trips skip) + a pack smoke check on every PR/push touching the binding - fast feedback kept - out of the heavy release path. - -## D16 - Standalone repo + submodule; natives via upstream release download (supersedes D15's CI shape) -- Repo split: the binding lives in its own repo, `LadybugDB/ladybug-dotnet` (adopted into the LadybugDB - org from its initial personal home). It is developed as a git submodule - mounted at `tools/csharp_api` in `LadybugDB/ladybug`, exactly mirroring `tools/rust_api` -> - `ladybug-rust` and `tools/java_api` -> `ladybug-java` (see the monorepo `.gitmodules`). The repo - root == the old `tools/csharp_api/` contents (so `src/`, `test/`, `.github/`, `.agents/` sit at the - root). For now the submodule is wired in the monorepo LOCALLY ONLY and not pushed upstream. -- Native source CHANGED from D15: the release workflow no longer reuses the monorepo's - `precompiled-bin-workflow.yml`. A cross-repo reusable-workflow call does a plain `actions/checkout`, - which checks out the CALLER (this C# repo, which has no engine source) and would fail to build. - Instead, `release.yml` downloads the prebuilt `liblbug-*` assets from a `LadybugDB/ladybug` GitHub - Release (`gh release download`), pinned to `ENGINE_VERSION`. This is decoupled, fast, and version- - pinned - how a published binding consumes a released C lib in reality. `release-artifacts.yml` - upstream already attaches those assets to releases. -- Engine version pin: `ENGINE_VERSION` (env in `release.yml`, overridable via the `engine_version` - dispatch input) is the upstream tag the natives come from. The two repos are no longer one commit; - bumping it requires re-syncing the managed signatures/structs/enums against that release's - `lbug.h` and updating the ABI tests in the same change (also captured in `AGENTS.md`). -- Workflows renamed for the dedicated repo: `csharp-ci.yml` -> `.github/workflows/ci.yml` (managed-only - build/test/pack; path filters now repo-relative) and `csharp-release.yml` -> - `.github/workflows/release.yml` (download-natives -> stage -> linux-x64 gate -> pack -> assert -> - OIDC publish). Release tag is now `v*` (not `csharp-v*`); `v1.2.3` -> package version `1.2.3`. -- Trusted publishing now targets this repo: nuget.org policy owner=`LadybugDB`, repo=`ladybug-dotnet`, - workflow=`release.yml`, env=`release`. The repo now lives in the LadybugDB org; publishing the official - `LadybugDB` package id still needs the nuget.org policy wired up and package-id ownership on that account. -- The staging/`cp -L`/RID-mapping/package-content-assertion logic and the `LADYBUG_REQUIRE_NATIVE=1` - gate from D15 all carry over unchanged; only the source of the native artifacts differs. - -## D17 - Adopted into the LadybugDB org; native CI matrix -- The repo was adopted into the LadybugDB org as `LadybugDB/ladybug-dotnet` (no longer a personal repo). - The submodule `origin` and the monorepo `.gitmodules` URL were repointed to it; the trusted-publishing - policy owner is now `LadybugDB` and `RepositoryUrl` points at the org repo. -- `ci.yml` gained a `native-test` matrix (linux-x64 + win-x64) that downloads the prebuilt `liblbug-*` - for the pinned engine release and runs the full suite with `LADYBUG_REQUIRE_NATIVE=1`, so native - round-trips run on every push - not only in the release gate. - -## D18 - Split package family + Cake Frosting build (supersedes D16's single fat package) -- Packaging changed from one fat `LadybugDB` package (managed + all natives) to a **family**: managed-only - `LadybugDB`, one `LadybugDB.Native.` per RID, and a `LadybugDB.Native` meta-package depending on all - five. Consumers now reference TWO packages (`LadybugDB` + a native package). This is a deliberate, breaking - change to the consumption model: it lets an app pull only the platform(s) it needs (reference a single - `LadybugDB.Native.`) instead of carrying every platform's binary. The managed package has NO - dependency on the native packages - that decoupling is exactly what enables the slim, single-RID install. -- Naming: `LadybugDB.Native` (meta) + `LadybugDB.Native.{win-x64, linux-x64, linux-arm64, osx-x64, osx-arm64}`. - Each native package carries only `runtimes//native/*` plus an empty `lib/netstandard2.0/_._` marker - (so NuGet adds no compile reference) and uses `SuppressDependenciesWhenPacking` (no framework dependency). -- `nuget/nuget-package.props` no longer globs `lib/runtimes/**` into the managed package; `LadybugDB` is now - purely `lib/net10.0` + `lib/netstandard2.0` + README + symbols. -- Build tool: **Cake Frosting** (not Nuke), as a normal .NET console app under `cake/` - (`LadybugDB.Build.csproj`, `BuildContext.cs` which also hosts `Main`, `Tasks/`). The folder is named `cake/` (not - the conventional `build/`, which the monorepo root `.gitignore` reserves for build output) so the build - tool is obvious. Layout: `cake/` + `cake/common.props` + `cake/native/`. Tasks: Clean, Restore, BuildManaged, Test, - FetchNatives, PackManaged, PackRuntimes, PackNativeMeta, VerifyPackages, Pack, Default. -- Per-RID packaging is ONE parameterized template (`cake/native/LadybugDB.Native.Runtime.csproj`, packed - once per RID via `-p:NativeRid/-p:PackageId`) rather than five hand-authored nuspecs. The meta-package IS a - nuspec (`cake/native/LadybugDB.Native.nuspec`, a template) because a csproj can't cleanly declare NuGet - dependencies on packages that don't exist on a feed yet; version/commit tokens are substituted in C# at - pack time (avoids passing semicolon-laden `NuspecProperties` through MSBuild). All packing is `dotnet`-only - (no nuget.exe / mono). -- `BuildContext` is the single source of truth for the RID set and the RID->(release asset, library file) - mapping. `EnsureNativeStaged` skips when a native is already staged (e.g. built locally from source), - otherwise downloads the pinned engine asset via `gh` and extracts the canonical library (taking the real - shared object out of the `.so`/`.dylib` symlink chain, since NuGet doesn't preserve symlinks and the binding - loads by path). `VerifyPackages` asserts the managed assemblies, each per-RID payload, and the meta deps. -- Versioning: all packages take ONE release version from the `v*` tag (`--package-version`); `ENGINE_VERSION` - remains the internal pin for WHICH prebuilt natives to fetch, decoupled from the package version. (Alternative - - version native packages by engine version - rejected for a simpler single-version family.) -- Cake gotcha: the host reserves `--version` (prints Cake's own version), so the package version argument is - `--package-version`. The orchestrator project must exclude `native/**` from its compile items, or it picks up - the native projects' generated `AssemblyInfo.cs` and fails with duplicate-attribute errors. -- CI/release now invoke the pipeline (`dotnet run --project cake/... -- --target Test|Pack`) instead of inline - bash. The release `publish` job pushes all 7 packages via OIDC; the trusted-publishing policy / package-id - ownership on nuget.org must now cover every id (`LadybugDB`, `LadybugDB.Native`, and the five - `LadybugDB.Native.`), not just `LadybugDB`. - -## D19 - Package version tracks the upstream engine version with an -alpha.N dev suffix -- All 7 packages share ONE version (per D18). That version's BASE now equals the upstream engine version - (`ENGINE_VERSION` without the leading `v`, e.g. `v0.17.0` -> `0.17.0`), so the binding's released - version lines up with the engine it wraps. This refines D18's versioning bullet: the family is still - uniformly versioned, we just pick the engine version as the base instead of an arbitrary number. -- While the binding is in development the version carries a prerelease suffix (`alpha.1`, `alpha.2`, ...), - so the current default is `0.17.0-alpha.1`. `--prerelease ` overrides the suffix, `--prerelease ""` - cuts a stable build equal to the engine version, and `--package-version ` overrides the whole thing - (the release workflow passes the exact version from the `v*` git tag). -- Single source of truth: the version lives in ONE data file, `version.txt` at the binding root - (currently `0.17.0-alpha.1`) - not hardcoded in code or workflows. `BuildContext` reads it (engine - release = its base prefixed with `v`, e.g. `v0.17.0`; package version = base + suffix); the managed - `nuget-package.props` reads it (via an MSBuild `File.ReadAllText`) for a direct `dotnet pack`; and the - CI/release workflows derive from it (the `v*` tag overrides on release). Bumping the alpha (or moving - to a new engine) is a one-line edit to `version.txt` - no code or workflow change. This replaced the - earlier `DevelopmentPrerelease` C# constant and the `ENGINE_VERSION` workflow env (both removed); - `--engine-version` / `ENGINE_VERSION` survive only as optional per-run overrides. The old `0.0.0-dev` - / `0.0.1-alpha` placeholders are gone. - -## D20 - First package-family release published -- On 2026-05-31 the first published package-family release was cut from tag `v0.17.0-alpha.1`. - `release.yml` completed successfully: it gated on linux-x64 against the real engine, packed and verified - the family, then published all 7 NuGet package ids via trusted publishing. -- Published ids: `LadybugDB`, `LadybugDB.Native`, and `LadybugDB.Native.{win-x64, linux-x64, linux-arm64, - osx-x64, osx-arm64}`. This closes the Phase 4 build/package-publication track; future release work is - normal version bump + tag flow unless the native ABI or package layout changes. - -## Open (decide later) -- Timestamp representation: `DateTime` (UTC) for non-tz precisions vs `DateTimeOffset` for `TIMESTAMP_TZ` (Phase 2). -- Whether to expose write-side construction of unsigned / `UNION` / `ARRAY` values and full `INTERVAL` (months/days). -- `win-arm64` / `linux-musl` coverage (not produced by the upstream precompiled workflow today). diff --git a/.agents/notes/HANDOFF.md b/.agents/notes/HANDOFF.md deleted file mode 100644 index e8c8f04..0000000 --- a/.agents/notes/HANDOFF.md +++ /dev/null @@ -1,190 +0,0 @@ -# Handoff / Transfer Notes - Ladybug C# Binding - -Current state, how to build/test, and immediate next steps. Keep this current whenever work pauses. - -## Current state -- Standalone repo in the LadybugDB org: `LadybugDB/ladybug-dotnet`. Developed as the `tools/csharp_api` - submodule of `LadybugDB/ladybug` (mirrors `tools/rust_api`, `tools/java_api`). The submodule is wired - in the monorepo LOCALLY ONLY here (the monorepo gitlink is not pushed upstream yet). -- Phases 0-3 implemented. The solution builds for BOTH target frameworks (`net10.0` + `netstandard2.0`), - and `dotnet pack` produces a valid package. -- END-TO-END VALIDATED on Windows x64: native `lbug_shared.dll` built locally with MSVC + Ninja and - the full xUnit suite passes (28/28, 0 skipped) against the real engine - including from the submodule - checkout, which builds the parent engine via `../../`. -- The C# project does NOT build native code. It P/Invokes the Ladybug C API (`lbug.h`, in the parent - `LadybugDB/ladybug` repo) exported from the `lbug_shared` target. - -## What works (verified) -- Interop: `LibraryImport` (net7+) / `DllImport` (ns2.0) for ~90 C API functions, resolver, UTF-8 helpers. -- API: `Database`, `Connection`, `QueryResult`, `FlatTuple`, `Value` (full type system), - `PreparedStatement` (typed + object binding), `LadybugVersion`, graph types, `SystemConfig`. -- 28 xUnit tests: 17 ABI/struct-layout guards + 10 native round-trip tests (smoke, type-mapping, - prepared statements) + 1 native-required gate, all PASSING against `lbug_shared.dll`. - -## Local toolchain on this machine (verified 2026-05-29) -- Visual Studio Community 2026 (v18.6): MSVC `14.51.36231` (`cl.exe` x64) + Windows SDK `10.0.26100`. -- LLVM clang `22.1.6` at `C:\Program Files\LLVM\bin` (alternative compiler; not used for the build below). -- CMake `4.3.2` + Ninja `1.13.0` installed via `pip install --user cmake ninja` (in the Python - user Scripts dir; NOT on the default PATH). - -## Build the native lib + validate the wrapper (verified recipe) -The native build needs the MSVC environment (cl/INCLUDE/LIB) and cmake/ninja on PATH. The dotnet/C# -build does NOT need MSVC. Easiest path is the helper script: -```powershell -# from tools/csharp_api — builds lbug_shared.dll, copies it into lib/runtimes/win-x64/native, runs tests -pwsh -File scripts/build-native-and-test.ps1 -``` -What it does (and the manual equivalent): -```powershell -# 1. Enter the MSVC x64 env (imports vcvars64.bat) and put pip's cmake/ninja on PATH -# (see scripts/build-native-and-test.ps1 for the exact env import) -# 2. Configure once (CMake 4.x needs the policy floor for the vendored deps): -cmake -B build/release -G Ninja -DCMAKE_BUILD_TYPE=Release ` - -DBUILD_SHELL=OFF -DBUILD_SINGLE_FILE_HEADER=OFF -DBUILD_STATIC_LBUG=OFF -DBUILD_TESTS=OFF ` - -DCMAKE_POLICY_VERSION_MINIMUM=3.5 . -# 3. Build just the C-API shared lib (~8 min, 1046 steps -> build/release/src/lbug_shared.dll, 18 MB): -cmake --build build/release --target lbug_shared -# 4. Wire it into the binding and run the suite: -Copy-Item build/release/src/lbug_shared.dll tools/csharp_api/lib/runtimes/win-x64/native/ -Force -dotnet test tools/csharp_api/test/LadybugDB.Tests/LadybugDB.Tests.csproj -``` - -## Build / test / pack via the Cake Frosting pipeline (`cake/`) -All packaging is driven by the Cake Frosting build project under `cake/` (don't hand-run `dotnet pack`). -Use the `build.ps1` / `build.sh` bootstrap from `tools/csharp_api`: -```powershell -./build.ps1 --target Test # build both TFMs + stage host native + run suite -./build.ps1 --target Pack # full package family -> ./artifacts, verified -``` -- Versioning: the package version tracks the upstream engine version (`v0.17.0` -> `0.17.0`) with an - `-alpha.N` prerelease suffix while in development, so the default is `0.17.0-alpha.1`. It lives in ONE - place - `version.txt` at the binding root - which `BuildContext`, `nuget-package.props`, and both - workflows all read; bump the alpha (or the engine) there with no code change. Overrides: `--prerelease - alpha.2` (suffix), `--package-version ` (exact; the release workflow passes the git tag), and - `--prerelease ""` (stable build equal to the engine version). -- The binding now ships a FAMILY (see DECISIONS D18): managed-only `LadybugDB`, one - `LadybugDB.Native.` per RID, and the `LadybugDB.Native` meta-package. `Pack` stages every RID's - native (downloading from the pinned engine release when not already present), packs all 7, and - `VerifyPackages` asserts contents. -- VERIFIED locally end-to-end (2026-05-30, win-x64 host, engine `v0.17.0`): `--target Pack` produced - and verified all 7 packages; `--target Test` passed 28/28 with the native loaded. Confirmed layouts: - `LadybugDB` = `lib/net10.0` + `lib/netstandard2.0` (with XML docs) + `README.md`, no `runtimes/` and - zero dependencies; `LadybugDB.Native.` = `runtimes//native/` + `lib/netstandard2.0/_._` - and no dependencies; `LadybugDB.Native` = `_._` + dependencies on all five per-RID packages. -- RELEASED (2026-05-31): tag `v0.17.0-alpha.1` ran `release.yml` successfully and published all 7 - packages to NuGet: `LadybugDB`, `LadybugDB.Native`, and `LadybugDB.Native.{win-x64, linux-x64, - linux-arm64, osx-x64, osx-arm64}`. -- Cake arg notes: the package version is `--package-version` (the host reserves `--version`); - `--engine-version` overrides the pinned engine; `--commit` (or `GITHUB_SHA`) stamps the repository - metadata. Packages land in `tools/csharp_api/artifacts/` (gitignored). - -## Gotchas -- PowerShell mangles `-DCMAKE_POLICY_VERSION_MINIMUM=3.5` into `=3` when args aren't quoted; pass cmake - `-D` flags via an explicit `@()` array (the helper script does this). -- Importing `vcvars64.bat` sets `Platform=x64`, which makes subsequent `dotnet` builds output to - `bin/x64/...`. Harmless; tests still find the native lib. Use a fresh shell for pure C# work. -- The DLL links the dynamic MSVC CRT (`vcruntime140`, `msvcp140`, `ucrtbase`); present with VS here, - needs the VC++ redistributable on clean machines. - -## Layout -``` -tools/csharp_api/ - Directory.Build.props # shared build props (LangVersion, OS/arch detection, paths) - nuget/nuget-package.props # managed package metadata (managed-only; natives ship separately) - LadybugDB.slnx - build.ps1 / build.sh # bootstrap for the Cake Frosting pipeline - cake/ # Cake Frosting packaging pipeline (NOT in LadybugDB.slnx) - LadybugDB.Build.csproj # the build console app (Cake.Frosting) - BuildContext.cs # Main entry point + RID set, paths, EnsureNativeStaged() - Tasks/ # Clean, Restore, BuildManaged, Test, FetchNatives, - # PackManaged, PackRuntimes, PackNativeMeta, VerifyPackages, Pack, Default - common.props # shared NuGet metadata for the native packages - native/ # native packaging assets - LadybugDB.Native.Runtime.csproj # one template, packed once per RID - LadybugDB.Native.Meta.csproj # thin host to pack the meta nuspec via dotnet - LadybugDB.Native.nuspec # meta-package template ($version$/$commit$ tokens) - _._ # empty lib marker - src/LadybugDB/ # the binding (multi-target net10.0 + netstandard2.0) - Interop/ # Native (P/Invoke), per-TFM marshaling, structs, resolver - *.cs # Database, Connection, QueryResult, FlatTuple, Value, ... - test/LadybugDB.Tests/ # xUnit tests (net10.0) - lib/runtimes//native/ # native libs are staged here for packaging/tests (gitignored) - artifacts/ # produced .nupkg/.snupkg (gitignored) - download/ # cached engine release assets (gitignored) - .agents/notes/ # DECISIONS / HANDOFF / ROADMAP -``` - -## Native library requirement -- Tests and any real use need the native shared library at runtime, named per-OS: - - Windows: `lbug_shared.dll` · Linux: `liblbug.so` · macOS: `liblbug.dylib` -- The test project's `PlaceNativeLibrary` target copies `lib/runtimes//native/*` next to the test - output, where the DllImport resolver finds it. `lib/` is gitignored, so the DLL is a local artifact. -- Tests SKIP (not fail) when the native lib is absent (`TestEnvironment.NativeAvailable`). - -## Examples (`examples/`) -Two kinds of samples, kept apart on purpose: -- Database-usage samples are NuGet package CONSUMERS (they reference `LadybugDB` + `LadybugDB.Native` - from the published feed, version pinned via the `LadybugVersion` MSBuild property → `version.txt`): - - `quickstart/` - in-memory DB, create/insert/select. - - `demo-graph/` - User/City/Follows/LivesIn loaded from VENDORED CSVs under `demo-graph/data/` - (copied next to the binary) with `COPY`, then traversed. Self-contained; no `dataset/` dependency. - - `prepared-statements/` - prepared-statement reuse + `Connection.Execute(cypher, parameters)`. - - `result-values/` - how `QueryResult.Rows()` materializes each engine type into CLR objects. -- `native-loading/` is a packaging/DEPLOYMENT sample (bundled native NuGet vs. system `.deb`), not a - database-usage sample. Left as-is. -- `examples/README.md` is the index separating the two categories. -- VERIFIED 2026-05-31 on win-x64 against `v0.17.0-alpha.1`: all four database examples build and run - green. The only thing surfaced was an example-formatter assumption (NOT a binding bug): a `MAP` - materializes to `Dictionary` while a `STRUCT`/node/rel properties materialize to - `Dictionary`. The `result-values` formatter now handles both via non-generic - `IDictionary`, and the distinction is documented in its README. Note `LadybugVersion.Version` reports - the native engine version (`0.17.0`), which differs from the package version (`0.17.0-alpha.1`). -- Examples are NOT wired into CI yet (deferred until we settle how examples should behave when - `version.txt` is bumped ahead of a published package). - -## CI / CD (GitHub Actions, in the standalone repo) -Both workflows invoke the Cake pipeline via `dotnet run --project cake/LadybugDB.Build.csproj -- ...` -(natives come from upstream releases; the engine is never built here). `GH_TOKEN` is set so -`FetchNatives` can download the prebuilt assets. -- `.github/workflows/ci.yml` - PR/push validation, path-filtered to `src/**`, `test/**`, `cake/**`, - `nuget/**`, `**/*.props`, `LadybugDB.slnx`. A `test` matrix (linux-x64 + win-x64) runs `--target Test` - (stages the host native, runs the suite with no skips), and a `pack` job runs `--target Pack` - (version derived from `version.txt`) to build + verify the whole family without publishing. -- `.github/workflows/release.yml` - the release pipeline. Two jobs: - 1. `pack`: resolves the version (tag `v1.2.3` -> `1.2.3`) and engine version, runs `--target Test` - as the linux-x64 gate against the real engine, then `--target Pack` (which stages all 5 RIDs, packs - the 7 packages, and `VerifyPackages` asserts every package's contents). Uploads the artifacts. - 2. `publish` (only on `v*` tags): trusted publishing via `NuGet/login@v1` - (`id-token: write`) + `dotnet nuget push "artifacts/*.nupkg" --skip-duplicate` - now pushes ALL 7 - packages (the `.snupkg` symbols ride along with the managed package). -- The upstream engine release the natives are taken from defaults to the version's base (from - `version.txt`, or the `v*` tag on release); override per-run with `--engine-version` / `ENGINE_VERSION` - or the `engine_version` dispatch input. Keep it in sync with the managed ABI. - -### Releasing a version -```bash -git tag v0.1.0 # tag drives the package version (v1.2.3 -> 1.2.3) -git push origin v0.1.0 -``` -`workflow_dispatch` (with a `version` input) builds + packs + uploads the artifacts WITHOUT publishing - -use it to dry-run the full family build before tagging. - -### One-time setup before the first publish (DONE) -- nuget.org -> Account -> Trusted Publishing -> Add a policy for EACH package id (the publish job pushes - all 7): `LadybugDB`, `LadybugDB.Native`, and `LadybugDB.Native.{win-x64, linux-x64, linux-arm64, - osx-x64, osx-arm64}`. owner=`LadybugDB`, repo=`ladybug-dotnet`, workflow file=`release.yml`; leave the - policy's Environment blank for now (the publish job runs without a GitHub environment). -- Repo Settings -> Secrets and variables -> Actions: add repository secret `NUGET_USER` = the nuget.org - PROFILE name (not email) that owns/co-owns the package ids. (DONE.) A dedicated `release` environment - (required reviewers as an approval gate, with the policy's Environment set to match) can be added later - when we separate release vs test environments. -- All 7 package ids are owned/reserved and `v0.17.0-alpha.1` was published successfully. - -## Next steps -1. Keep the release process as-is for follow-up alphas: bump `version.txt` (for example, - `0.17.0-alpha.2`), merge through CI, then tag the same version (`v0.17.0-alpha.2`) when ready. -2. For a new upstream engine release, re-sync managed signatures/struct layouts/enums against that - release's `lbug.h`, update ABI tests, bump `version.txt`, and let the Cake pipeline fetch that engine's - prebuilt `liblbug-*` assets. -3. Phase 3 (remaining): expand the suite to mirror the Java + C API tests over `dataset/tinysnb`. -4. Phase 5 (optional): Native AOT validation, Arrow C Data interface, observability. diff --git a/.agents/notes/ROADMAP.md b/.agents/notes/ROADMAP.md deleted file mode 100644 index 2aa76f5..0000000 --- a/.agents/notes/ROADMAP.md +++ /dev/null @@ -1,74 +0,0 @@ -# Roadmap - Ladybug C# Binding - -Live status of the phased plan. Keep statuses current. - -Legend: [x] done and verified here. As of 2026-05-29 the native `lbug_shared.dll` is built locally -(MSVC + Ninja, win-x64) and the full suite passes end-to-end (28/28, 0 skipped), so the previous -"[~] awaiting native validation" markers are now resolved for win-x64. - -As of 2026-05-30 the binding was split into a standalone repo - now adopted into the LadybugDB org as -`LadybugDB/ladybug-dotnet` - and wired back into the monorepo as the `tools/csharp_api` submodule (local -only). The suite still passes 28/28 from the submodule, building `lbug_shared` against the parent engine -via `../../`. - -As of 2026-05-31 the first package-family release, `v0.17.0-alpha.1`, was published to NuGet: `LadybugDB`, -`LadybugDB.Native`, and all five per-RID native packages. - -Also as of 2026-05-31, four database-usage examples were added under `examples/` (quickstart, demo-graph, -prepared-statements, result-values) as package consumers of the published NuGet feed. All four build and -run green on win-x64 against `v0.17.0-alpha.1`. The only issue surfaced was an example-formatter -assumption, not a binding bug: a `MAP` materializes to `Dictionary` while a `STRUCT` -materializes to `Dictionary` (now documented). - -## Phase 0 - Scaffold [x] -- [x] `tools/csharp_api/` layout -- [x] `.agents/notes/` living docs (DECISIONS, HANDOFF, ROADMAP) -- [x] `Directory.Build.props`, `nuget/nuget-package.props` -- [x] `LadybugDB.slnx` + projects building (both target frameworks) - -## Phase 1 - Core happy path [x build] [x runtime: win-x64] -- [x] Interop layer (`Native`) + per-TFM marshaling (LibraryImport / DllImport) -- [x] `NativeLibrary` resolver (net7+) mapping `lbug_shared` -> `liblbug.{so,dylib}` -- [x] `Database`, `Connection`, `QueryResult`, `FlatTuple`, `Value` (scalars + string), `LadybugVersion` -- [x] Smoke tests compile; [~] skip until a native lib is present - -## Phase 2 - Full value/type model [x build] [x runtime: win-x64] -- [x] Temporal (date, 4 timestamp precisions + tz, interval), decimal, int128, uuid, blob -- [x] Nested: list/array -> object[], struct/union -> dictionary, map -> dictionary -- [x] Graph: `Node`, `Rel`, `RecursiveRel` -- [x] `_is_owned_by_cpp`-safe disposal; eager row materialization -- [~] Type-mapping tests written; validate Cypher/assertions against a native build - -## Phase 3 - Prepared statements + hardening [x build] [x runtime: win-x64] -- [x] `PreparedStatement` with typed + object `Bind` overloads; `Connection.Prepare/Execute` -- [x] Convenience `Connection.Execute(query, parameters)` -- [x] Two-level error model (throw on failure; `IsSuccess`/`GetErrorMessage` still available) -- [x] Per-connection serialization (lock) + retry-safe disposal (`Interlocked`) -- [~] Prepared-statement tests written; broader tinysnb-dataset suite still TODO (needs native lib) -- [x] P/Invoke review pass vs `dotnet-pinvoke` skill: signatures/strings/ownership/`bool` all clean, 0 SYSLIB warnings -- [x] ABI guard tests (`StructLayoutTests`, 17) assert struct sizes/offsets without needing the native lib - -## Phase 4 - Packaging + release [x] -- [x] Local native build recipe + `scripts/build-native-and-test.ps1` (win-x64, MSVC + Ninja) -- [x] win-x64 native lib wired into `lib/runtimes/win-x64/native/` and loaded by the tests -- [x] Repo split: standalone `LadybugDB/ladybug-dotnet` (in the LadybugDB org), wired as the `tools/csharp_api` submodule (local) -- [x] Repo adopted into the LadybugDB org as `LadybugDB/ladybug-dotnet` -- [x] **Split package family** (DECISIONS D18): managed-only `LadybugDB` + `LadybugDB.Native.` per - RID + `LadybugDB.Native` meta-package. Consumers reference `LadybugDB` plus a native package; - managed has no native dependency so single-RID slim installs work. -- [x] **Cake Frosting** build project under `cake/` (Clean/Restore/BuildManaged/Test/FetchNatives/ - PackManaged/PackRuntimes/PackNativeMeta/VerifyPackages/Pack), with `build.ps1`/`build.sh` bootstrap. -- [x] `nuget/nuget-package.props` stripped of native packing (managed-only). -- [x] Verified end-to-end locally (2026-05-30, win-x64, engine `v0.17.0`): `--target Pack` produced + - verified all 7 packages (correct layouts, meta deps, no stray references); `--target Test` 28/28. -- [x] CI/release rewired to drive the pipeline (`dotnet run --project cake/... -- --target Test|Pack`); - release `publish` pushes all 7 packages via OIDC; `ci.yml` runs the `test` matrix + a `pack` job. -- [x] One-time nuget.org trusted-publishing policy (one per package id) + `NUGET_USER` secret completed -- [x] First green release run on CI: `v0.17.0-alpha.1` tag built, tested, packed, and published all 7 packages -- [x] Ownership/reservation of all 7 package ids on nuget.org before the first real publish -- [ ] Optional: win-arm64 / linux-musl RIDs (not produced by the upstream precompiled workflow today) -- [x] Database-usage examples under `examples/` (quickstart, demo-graph, prepared-statements, - result-values) as NuGet package consumers; verified green on win-x64 (2026-05-31) - -## Phase 5 (optional) - Extras [pending] -- Native AOT validation, Arrow C Data interface, observability metrics. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d768daf..087f6ef 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -18,8 +18,10 @@ on: - 'cake/**' - 'nuget/**' - '**/*.props' + - 'version.txt' - 'LadybugDB.slnx' - '.github/workflows/ci.yml' + - '.github/workflows/release.yml' pull_request: paths: - 'src/**' @@ -27,8 +29,10 @@ on: - 'cake/**' - 'nuget/**' - '**/*.props' + - 'version.txt' - 'LadybugDB.slnx' - '.github/workflows/ci.yml' + - '.github/workflows/release.yml' workflow_dispatch: permissions: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d6213a2..130153c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -5,14 +5,14 @@ name: Release # trusted publishing (OIDC). # # Native engine libraries are NOT built here; the pipeline downloads prebuilt `liblbug-*` release -# assets from the upstream engine repo (LadybugDB/ladybug). The engine release defaults to the base -# of the version being built (the git tag, or version.txt for manual runs). +# assets from the upstream engine repo (LadybugDB/ladybug). The engine release defaults to the first +# three numeric segments of the package version (the git tag, or version.txt for manual runs). # # Triggers: # - push a tag like `v0.1.0` -> build, test, pack, and PUBLISH all packages to nuget.org # - manual `workflow_dispatch` -> build, test, and pack only (artifact uploaded; no publish) # -# One-time setup required before the first publish (see notes/HANDOFF.md): +# One-time setup required before publishing (see MAINTAINING.md): # - nuget.org trusted publishing policy covering every package id: LadybugDB, LadybugDB.Native, and # LadybugDB.Native.{win-x64, linux-x64, linux-arm64, osx-x64, osx-arm64} # (owner=LadybugDB, repo=ladybug-dotnet, workflow file=release.yml; environment left blank for now) @@ -29,7 +29,7 @@ on: required: false default: '' engine_version: - description: "LadybugDB/ladybug release tag to pull prebuilt native libs from (defaults to the version's base)." + description: "LadybugDB/ladybug release tag to pull prebuilt native libs from (defaults to the package version's first three numeric segments)." required: false default: '' @@ -71,7 +71,13 @@ jobs: set -euo pipefail VERSION="${{ steps.ver.outputs.version }}" ENGINE="${{ inputs.engine_version }}" - ENGINE="${ENGINE:-v${VERSION%%-*}}" # default: engine release matching the version's base + BASE="${VERSION%%[-+]*}" + IFS='.' read -r MAJOR MINOR PATCH _ <<< "$BASE" + if [[ ! "${MAJOR:-}" =~ ^[0-9]+$ || ! "${MINOR:-}" =~ ^[0-9]+$ || ! "${PATCH:-}" =~ ^[0-9]+$ ]]; then + echo "Package version '$VERSION' must start with three numeric engine segments." >&2 + exit 1 + fi + ENGINE="${ENGINE:-v${MAJOR}.${MINOR}.${PATCH}}" echo "engine=$ENGINE" >> "$GITHUB_OUTPUT" echo "Pulling native libs from LadybugDB/ladybug release: $ENGINE" diff --git a/AGENTS.md b/AGENTS.md index bf642cc..d3b2f55 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,11 +7,9 @@ live in that parent repo — reachable at `../../` when mounted, NOT in this rep enums/structs/signatures mirror that header exactly; if they drift, calls corrupt memory or crash at runtime, not at compile time. -**Working notes — read first, keep current.** This binding is an in-progress port driven by three living -docs under `.agents/notes/`: `ROADMAP.md` (phase status / what's left), `HANDOFF.md` (current state + -verified build recipe), `DECISIONS.md` (append-only ABI/design/repo-split log). They carry context across -agent sessions — start a task by reading them, and update the relevant one in the same change (full -descriptions under [Deeper docs](#deeper-docs)). +**Maintainer guide — read first, keep current.** Durable repository operations, versioning, release, +packaging, and ABI-update guidance lives in `MAINTAINING.md`. Start there for non-trivial changes and +update it when workflow or release policy changes. ## Boundaries @@ -45,7 +43,7 @@ pwsh -File scripts/build-native-and-test.ps1 Builds `lbug_shared.dll`, stages it into `lib/runtimes/win-x64/native/`, and runs the suite. The manual CMake recipe (other OSes, flags, the `-DCMAKE_POLICY_VERSION_MINIMUM=3.5` floor) is in -`.agents/notes/HANDOFF.md`. +`MAINTAINING.md`. ## Tests @@ -83,10 +81,11 @@ There is **no** binding generator (no ClangSharp). "Source-generated" refers onl ## Upstream coupling The binding targets the Ladybug engine C API in the separate `LadybugDB/ladybug` repo, pinned to one -release: the engine tag is the base of `version.txt` at the repo root (e.g. `0.17.0-alpha.1` -> `v0.17.0`), -overridable per build via `--engine-version` / `ENGINE_VERSION`. The two repos are no longer a single -commit: when moving to a new engine release, re-sync the managed signatures/structs/enums against that -release's `src/include/c_api/lbug.h` and update the ABI tests in the same change. +release: the engine tag defaults to the first three numeric segments of `version.txt` at the repo root +(e.g. `0.17.0.1` -> `v0.17.0`), overridable per build via `--engine-version` / `ENGINE_VERSION`. +The two repos are no longer a single commit: when moving to a new engine release, re-sync the managed +signatures/structs/enums against that release's `src/include/c_api/lbug.h` and update the ABI tests in +the same change. ## Packaging @@ -113,17 +112,15 @@ A **Cake Frosting** build project under `cake/` drives everything (don't hand-ru The release pipeline (`.github/workflows/release.yml`, tag `v*`) runs `--target Test` (linux-x64 gate against the real engine) then `--target Pack`, and publishes all 7 packages via OIDC. The engine release -the natives come from defaults to the version's base (`version.txt`, or the `v*` tag on release). Details -+ one-time nuget.org setup (the trusted publishing policy must now cover every package id): -`.agents/notes/HANDOFF.md`. +the natives come from defaults to the first three numeric segments of the package version (`version.txt`, +or the `v*` tag on release). Details + release checklist live in `MAINTAINING.md`. ## Deeper docs - `src/include/c_api/lbug.h` (in the parent `LadybugDB/ladybug` repo) — source of truth for every signature, struct, and enum. -- `.agents/notes/DECISIONS.md` — interop/ABI/repo-split decision log (numbered D-entries with rationale). -- `.agents/notes/HANDOFF.md` — verified native build recipe, gotchas, CI/release, how to cut a release. -- `.agents/notes/ROADMAP.md` — phased status. +- `MAINTAINING.md` — repository operations, versioning, packaging, release, native build, and ABI update +guidance. - `dotnet-pinvoke` skill (in the monorepo's `.agents/skills/`) — P/Invoke technique. C# style is enforced by the .NET SDK analyzers (the build is warning-clean); don't restate it. diff --git a/MAINTAINING.md b/MAINTAINING.md new file mode 100644 index 0000000..8c36507 --- /dev/null +++ b/MAINTAINING.md @@ -0,0 +1,176 @@ +# Maintaining LadybugDB for .NET + +This document is the durable maintainer guide for the `LadybugDB/ladybug-dotnet` repository. It replaces +the development-era working notes. + +## Repository Shape + +This repository contains the hand-written C# binding for the native Ladybug C API. It is also used as the +`tools/csharp_api` submodule inside the main `LadybugDB/ladybug` monorepo, where the engine source and +`src/include/c_api/lbug.h` are available at `../../`. + +Main areas: + +- `src/LadybugDB/` - managed API and P/Invoke interop. +- `test/LadybugDB.Tests/` - ABI guards and native round-trip tests. +- `cake/` - Cake Frosting build, test, native download, pack, and verification pipeline. +- `examples/` - runnable package-consumer examples. +- `lib/runtimes//native/` - locally staged native libraries; gitignored. +- `artifacts/` - generated NuGet packages; gitignored. +- `download/` - cached upstream native release assets; gitignored. + +## Versioning Policy + +All packages in the family share one package version: + +- `LadybugDB` +- `LadybugDB.Native` +- `LadybugDB.Native.win-x64` +- `LadybugDB.Native.linux-x64` +- `LadybugDB.Native.linux-arm64` +- `LadybugDB.Native.osx-x64` +- `LadybugDB.Native.osx-arm64` + +The first three numeric segments track the native Ladybug engine release. The optional fourth numeric +segment is the .NET binding/package revision for binding-only releases over the same engine. + +Examples: + +- `0.17.0` - first stable .NET package family for engine `v0.17.0`. +- `0.17.0.1` - binding/package-only release that still uses engine `v0.17.0`. +- `0.17.0.2` - another binding/package-only release over engine `v0.17.0`. +- `0.17.1` - first package family for engine `v0.17.1`. +- `0.18.0-preview.1` - preview package family for a future engine `v0.18.0`. + +`version.txt` is the default package-family version source. The build pipeline derives the native engine +tag from the first three numeric package-version segments unless explicitly overridden with +`--engine-version` or `ENGINE_VERSION`. + +## Build, Test, Pack + +From the repository root: + +```powershell +dotnet build LadybugDB.slnx -c Release +dotnet test test/LadybugDB.Tests/LadybugDB.Tests.csproj -c Release +``` + +Native round-trip tests skip when no native library is staged. To require a native load, set: + +```powershell +$env:LADYBUG_REQUIRE_NATIVE = '1' +dotnet test test/LadybugDB.Tests/LadybugDB.Tests.csproj -c Release +``` + +Use the Cake pipeline for package work: + +```powershell +./build.ps1 --target Test +./build.ps1 --target Pack +``` + +On non-Windows shells: + +```bash +./build.sh --target Test +./build.sh --target Pack +``` + +`Pack` downloads prebuilt native assets from `LadybugDB/ladybug` releases when they are not already staged +under `lib/runtimes//native/`, then verifies package contents. + +## Local Native Build + +When this repository is checked out as `tools/csharp_api` in the main `LadybugDB/ladybug` monorepo, the +Windows helper can build the native shared library from the parent engine tree and run the full suite: + +```powershell +pwsh -File scripts/build-native-and-test.ps1 +``` + +Manual Windows recipe, from the main monorepo root with MSVC, CMake, and Ninja available: + +```powershell +cmake -B build/release -G Ninja -DCMAKE_BUILD_TYPE=Release ` + -DBUILD_SHELL=OFF -DBUILD_SINGLE_FILE_HEADER=OFF -DBUILD_STATIC_LBUG=OFF -DBUILD_TESTS=OFF ` + -DCMAKE_POLICY_VERSION_MINIMUM=3.5 . +cmake --build build/release --target lbug_shared +Copy-Item build/release/src/lbug_shared.dll tools/csharp_api/lib/runtimes/win-x64/native/ -Force +dotnet test tools/csharp_api/test/LadybugDB.Tests/LadybugDB.Tests.csproj -c Release +``` + +PowerShell can mangle unquoted `-D` arguments; pass CMake flags as quoted strings or via an explicit +PowerShell array when scripting. + +## Release Flow + +1. Decide the package version and update `version.txt`. +2. Ensure the native engine release exists in `LadybugDB/ladybug` for the first three numeric package + version segments, or pass `--engine-version` / workflow `engine_version`. +3. Run local validation where practical: + + ```powershell + ./build.ps1 --target Test + ./build.ps1 --target Pack + ``` + +4. Merge through CI. +5. Tag the package version: + + ```bash + git tag v0.17.0.1 + git push origin v0.17.0.1 + ``` + +The release workflow gates on linux-x64 against the real engine, packs the full package family, verifies +contents, and publishes all packages to NuGet through trusted publishing. + +Manual `workflow_dispatch` builds and uploads artifacts without publishing. Use it for dry runs. + +## ABI Update Checklist + +The binding mirrors the C API in `LadybugDB/ladybug` exactly. ABI mistakes can compile cleanly and still +corrupt memory at runtime. + +When moving to a new engine release: + +1. Compare managed declarations against that release's `src/include/c_api/lbug.h`. +2. Update both interop declaration files: + - `src/LadybugDB/Interop/Native.LibraryImport.cs` + - `src/LadybugDB/Interop/Native.DllImport.cs` +3. Update structs/enums in `src/LadybugDB/Interop/NativeTypes.cs`. +4. Update or add ABI guard tests in `test/LadybugDB.Tests/StructLayoutTests.cs`. +5. Stage the matching native library and run the native round-trip tests. + +Rules that should not change without deliberate review: + +- Calling convention is Cdecl. +- C `bool` is one byte: use `byte` in structs and `[MarshalAs(UnmanagedType.U1)]` on bool returns. +- `lbug_system_config` includes the macOS-only trailing `thread_qos` field so the by-value struct layout + matches across platforms. +- Native strings/blobs are copied to managed memory and then freed with the matching native destroy + function. +- Result/tuple/value disposal must respect the native ownership flag. + +## Package Family + +`LadybugDB` is managed-only. Native libraries ship separately in one package per RID, and +`LadybugDB.Native` is a meta-package that depends on every per-RID native package. Consumers reference: + +- `LadybugDB` plus `LadybugDB.Native` for all supported platforms, or +- `LadybugDB` plus one `LadybugDB.Native.` package for a slim single-platform app. + +The shipped RIDs are `win-x64`, `linux-x64`, `linux-arm64`, `osx-x64`, and `osx-arm64`. + +## Examples + +`examples/` contains two categories: + +- Database-usage examples: quickstart, demo graph, prepared statements, and result/value materialization. + These consume published NuGet packages and share the example package version in + `examples/Directory.Build.props`. +- `native-loading/`: deployment/package-loading example showing bundled native NuGet vs. system-installed + native library behavior. + +Examples are not currently part of CI because their package-restore behavior depends on a published package +version being available. diff --git a/README.md b/README.md index 0f2b4db..48d13f3 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Official C# binding for the [Ladybug](https://github.com/ladybugdb/ladybug) embe It wraps the native Ladybug C API via P/Invoke and ships prebuilt native libraries for supported platforms, so you can run Cypher queries against an embedded graph database directly from .NET. -> Status: under construction. See `.agents/notes/ROADMAP.md` for progress. +> Current package family: `0.17.0.1`, built against the Ladybug `v0.17.0` engine. ## Target frameworks @@ -76,14 +76,18 @@ project under [`cake/`](cake) (run it with the `build.ps1` / `build.sh` bootstra ./build.sh --target Pack # build the full package family into ./artifacts ``` -The package version tracks the upstream engine version (e.g. `0.17.0`), with an `-alpha.N` prerelease -suffix while the binding is in development. It is defined once in `version.txt` at the repo root; override -it with `--package-version ` (the release workflow uses the git tag), or `--prerelease ""` for a stable -build that matches the engine version. +All packages in the family share one version. The first three numeric segments track the upstream engine +release, and the optional fourth segment is the .NET package revision for binding-only releases. For +example, package `0.17.0.1` wraps the Ladybug `v0.17.0` engine; a future binding-only fix over the same +engine would be `0.17.0.2`. Prerelease suffixes are reserved for preview builds. + +The package version is defined once in `version.txt` at the repo root. Override it with +`--package-version ` (the release workflow uses the git tag). The engine release defaults to the first +three numeric package-version segments; override with `--engine-version` when needed. - **`Pack`** stages the prebuilt `liblbug-*` assets for every shipped RID (downloaded from an upstream - `LadybugDB/ladybug` GitHub Release, pinned to the engine version from `version.txt`; override with - `--engine-version`), + `LadybugDB/ladybug` GitHub Release, pinned to the engine version derived from `version.txt`; override + with `--engine-version`), packs the managed `LadybugDB` package, one `LadybugDB.Native.` package per RID, and the `LadybugDB.Native` meta-package, then verifies every package's contents. - **CI / release** (`.github/workflows/`) invoke the same pipeline; the release workflow gates on the diff --git a/cake/BuildContext.cs b/cake/BuildContext.cs index 5a03c28..712e596 100644 --- a/cake/BuildContext.cs +++ b/cake/BuildContext.cs @@ -34,20 +34,21 @@ public BuildContext(ICakeContext context) : base(context) Root = FindBindingRoot(); - // version.txt at the binding root is the single source of truth for the version (e.g. - // "0.17.0-alpha.1"): its base drives both the engine release we pull natives from and the - // package version's base, and its prerelease suffix (alpha.1, alpha.2, ...) is the dev suffix. - // Bump it there - no code change. Overrides: --engine-version / ENGINE_VERSION pick a different - // engine release; --prerelease "" cuts a stable build; --package-version sets the version - // verbatim (the release workflow passes the git tag). 'version' is reserved by the Cake host. - (string baseVersion, string filePrerelease) = ReadVersion(Root); - EngineVersion = context.Argument("engine-version", - Environment.GetEnvironmentVariable("ENGINE_VERSION") ?? $"v{baseVersion}"); + // version.txt at the binding root is the single source of truth for the package family version + // (e.g. "0.17.0.1"). The first three numeric segments identify the upstream engine release + // (v0.17.0), while an optional fourth segment identifies a binding-only package revision. + // Overrides: --engine-version / ENGINE_VERSION pick a different engine release; --prerelease + // can add or replace a prerelease suffix; --package-version sets the package version verbatim + // (the release workflow passes the git tag). 'version' is reserved by the Cake host. + string fileVersion = ReadVersion(Root); + string versionBase = StripVersionSuffixes(fileVersion); + string filePrerelease = GetPrereleaseSuffix(fileVersion); string prerelease = context.Argument("prerelease", filePrerelease); - string engineBase = EngineVersion.TrimStart('v', 'V'); Version = context.HasArgument("package-version") ? context.Argument("package-version") - : prerelease.Length == 0 ? engineBase : $"{engineBase}-{prerelease}"; + : prerelease.Length == 0 ? versionBase : $"{versionBase}-{prerelease}"; + EngineVersion = context.Argument("engine-version", + Environment.GetEnvironmentVariable("ENGINE_VERSION") ?? $"v{DeriveEngineVersion(Version)}"); ManagedProject = Path.Combine(Root, "src", "LadybugDB", "LadybugDB.csproj"); TestProject = Path.Combine(Root, "test", "LadybugDB.Tests", "LadybugDB.Tests.csproj"); @@ -211,23 +212,55 @@ private static void ExtractFromTarGz(string archive, string dest) } /// - /// Reads the binding's version from version.txt at the root - the single source of truth for - /// the package version (e.g. "0.17.0-alpha.1"). Returns the base ("0.17.0") and the prerelease - /// suffix ("alpha.1", or "" when stable). Bump it there to advance the alpha or the engine; no code change. + /// Reads the binding package-family version from version.txt at the root. The first three + /// numeric segments track the native engine release, and an optional fourth segment tracks a + /// binding-only package revision. /// - private static (string Base, string Prerelease) ReadVersion(string root) + private static string ReadVersion(string root) { string path = Path.Combine(root, "version.txt"); if (!File.Exists(path)) { throw new CakeException( $"Version file not found at '{path}'. " + - $"It is the single source of truth for the package version (e.g. 0.17.0-alpha.1)."); + $"It is the single source of truth for the package version (e.g. 0.17.0.1)."); + } + + return File.ReadAllText(path).Trim(); + } + + private static string DeriveEngineVersion(string packageVersion) + { + string baseVersion = StripVersionSuffixes(packageVersion); + string[] parts = baseVersion.Split('.'); + if (parts.Length < 3 || parts.Take(3).Any(p => p.Length == 0 || !p.All(char.IsDigit))) + { + throw new CakeException( + $"Package version '{packageVersion}' must start with three numeric engine segments, " + + "for example 0.17.0 or 0.17.0.1."); + } + + return string.Join(".", parts.Take(3)); + } + + private static string StripVersionSuffixes(string version) + { + int dash = version.IndexOf('-'); + int plus = version.IndexOf('+'); + int suffix = dash < 0 ? plus : plus < 0 ? dash : Math.Min(dash, plus); + return suffix < 0 ? version : version[..suffix]; + } + + private static string GetPrereleaseSuffix(string version) + { + int dash = version.IndexOf('-'); + if (dash < 0) + { + return string.Empty; } - string raw = File.ReadAllText(path).Trim(); - int dash = raw.IndexOf('-'); - return dash < 0 ? (raw, string.Empty) : (raw[..dash], raw[(dash + 1)..]); + int plus = version.IndexOf('+', dash + 1); + return plus < 0 ? version[(dash + 1)..] : version[(dash + 1)..plus]; } private static string FindBindingRoot() diff --git a/examples/Directory.Build.props b/examples/Directory.Build.props new file mode 100644 index 0000000..7e99895 --- /dev/null +++ b/examples/Directory.Build.props @@ -0,0 +1,9 @@ + + + + + 0.17.0.1 + + + diff --git a/examples/README.md b/examples/README.md index dfaa26b..620213e 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,8 +1,8 @@ # LadybugDB C# examples Runnable samples for the published `LadybugDB` NuGet packages. They restore from -[nuget.org](https://www.nuget.org/packages/LadybugDB) and use the version pinned in each -project file (override with `dotnet run -p:LadybugVersion=`). +[nuget.org](https://www.nuget.org/packages/LadybugDB) and use the version pinned in +[`Directory.Build.props`](Directory.Build.props) (override with `dotnet run -p:LadybugVersion=`). Every project references two packages: diff --git a/examples/demo-graph/DemoGraph.csproj b/examples/demo-graph/DemoGraph.csproj index 8e7cec5..44f8942 100644 --- a/examples/demo-graph/DemoGraph.csproj +++ b/examples/demo-graph/DemoGraph.csproj @@ -6,8 +6,6 @@ enable enable LadybugDB.Examples.DemoGraph - - 0.17.0-alpha.1 diff --git a/examples/native-loading/app/ConsumerApp.csproj b/examples/native-loading/app/ConsumerApp.csproj index 6088a1f..e0f702a 100644 --- a/examples/native-loading/app/ConsumerApp.csproj +++ b/examples/native-loading/app/ConsumerApp.csproj @@ -8,15 +8,14 @@ - Scenario 2 (system): published with the default (IncludeNative unset) so the app carries ONLY the managed assembly and must find liblbug.so on the system. - LadybugVersion defaults to the value in ../../../version.txt; the Dockerfiles pass it - explicitly via -p:LadybugVersion= so the example is not pinned to one string. + LadybugVersion defaults in examples/Directory.Build.props; the Dockerfiles pass it + explicitly via -p:LadybugVersion= so the example restores the staged local packages. --> Exe net10.0 enable ConsumerApp - 0.17.0-alpha.1 diff --git a/examples/prepared-statements/PreparedStatements.csproj b/examples/prepared-statements/PreparedStatements.csproj index 899246a..1326adc 100644 --- a/examples/prepared-statements/PreparedStatements.csproj +++ b/examples/prepared-statements/PreparedStatements.csproj @@ -6,8 +6,6 @@ enable enable LadybugDB.Examples.PreparedStatements - - 0.17.0-alpha.1 diff --git a/examples/quickstart/Quickstart.csproj b/examples/quickstart/Quickstart.csproj index 4a8abcb..36770b6 100644 --- a/examples/quickstart/Quickstart.csproj +++ b/examples/quickstart/Quickstart.csproj @@ -6,8 +6,6 @@ enable enable LadybugDB.Examples.Quickstart - - 0.17.0-alpha.1 diff --git a/examples/result-values/ResultValues.csproj b/examples/result-values/ResultValues.csproj index 26cfb33..dc24d40 100644 --- a/examples/result-values/ResultValues.csproj +++ b/examples/result-values/ResultValues.csproj @@ -6,8 +6,6 @@ enable enable LadybugDB.Examples.ResultValues - - 0.17.0-alpha.1 diff --git a/nuget/nuget-package.props b/nuget/nuget-package.props index 19c6f22..1883fd9 100644 --- a/nuget/nuget-package.props +++ b/nuget/nuget-package.props @@ -2,7 +2,7 @@ - $([System.IO.File]::ReadAllText('$(CSharpDir)version.txt').Trim()) LadybugDB diff --git a/test/LadybugDB.Tests/TestEnvironment.cs b/test/LadybugDB.Tests/TestEnvironment.cs index 56d841c..f0fa06d 100644 --- a/test/LadybugDB.Tests/TestEnvironment.cs +++ b/test/LadybugDB.Tests/TestEnvironment.cs @@ -9,7 +9,7 @@ internal static class TestEnvironment { /// /// True when the native Ladybug library can be loaded. When false, tests skip rather than fail. - /// See .agents/notes/HANDOFF.md for how to provide the native library. + /// See MAINTAINING.md for how to provide the native library. /// public static readonly bool NativeAvailable = Probe(); diff --git a/version.txt b/version.txt index 22a956e..b558c6a 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -0.17.0-alpha.1 +0.17.0.1