From 5740c67cdfd2e1e21cd0ace2fbe3b7e89344a1d0 Mon Sep 17 00:00:00 2001 From: Patrick O'Reilly Date: Thu, 17 Sep 2026 02:55:57 -0700 Subject: [PATCH 1/2] changelog: fold the five accumulated fragments into [Unreleased] Mechanical run of tools/fold_changelog.py. Five fragments, ten bullets: the aarch64 portability pass and its CI job, the -msse2 removal, the multi-arch docker workflow and its capacity reporting, the REPL :help and README rewrites, the SemFiresOnlyAfterFlush deflake, and the aarch64 orlyc teardown hang. They accumulated across the arm arc, which is exactly the case the fragment directory exists for -- unique filenames cannot conflict, and nothing had to touch CHANGELOG.md while five PRs were in flight. --- CHANGELOG.md | 10 ++++++++++ changelog.d/548-arm64-portability.md | 4 ---- changelog.d/548-first-run-funnel.md | 2 -- changelog.d/548-multiarch-image.md | 2 -- changelog.d/551-sem-flush-race.md | 1 - changelog.d/554-fiber-tls-aarch64.md | 1 - 6 files changed, 10 insertions(+), 10 deletions(-) delete mode 100644 changelog.d/548-arm64-portability.md delete mode 100644 changelog.d/548-first-run-funnel.md delete mode 100644 changelog.d/548-multiarch-image.md delete mode 100644 changelog.d/551-sem-flush-race.md delete mode 100644 changelog.d/554-fiber-tls-aarch64.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 58a1e97b..aa98773f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,16 @@ The project does not currently cut numbered releases. Entries reference the orig ## [Unreleased] ### Fixed +- **Fixed**: `orlyc` no longer hangs on aarch64 while tearing down its embedded test server. A fiber's stack outlives the OS thread that last ran it — frames come from a cross-thread pool and are routinely re-latched onto a runner on a different thread — but GCC computes the thread pointer once per function (one hoisted `mrs x, tpidr_el0` on aarch64) and derives every `__thread` access from it, so `TFrame::Run()`'s read of `TRunner::LocalRunner` *after* the runnable returned yielded the previous thread's runner and `longjmp`ed onto that thread's scheduler stack. Two OS threads then executed one stack, the hijacked runner's `Run()` never returned, and `~TDurableManager` waited forever on `SchedulerExitedSem` inside `TServer::Shutdown()`. The three fiber thread-locals are now `TFiberSafeLocal` slots read through `noipa` accessors, so the thread pointer is re-read at every access; nothing cheaper works (`volatile`, `thread_local`, an `asm` memory clobber, `returns_twice`, and all four `-ftls-model` settings all still emit one loop-invariant `mrs`). x86-64 folds the TLS base into each access's addressing mode, which is why only arm ever showed it (#554). +- **Fixed**: `durable_manager.test`'s `SemFiresOnlyAfterFlush` no longer fails intermittently on loaded CI runners. It asserted that the durability sem has *not* fired immediately after `Save()` — sound only while less than the manager's write delay has actually elapsed, and at a 300ms delay a descheduled test thread could legitimately miss the window, so correct behaviour reported as a failure. The negative assertion is now guarded on measured elapsed time and the fixture runs at a 2s write delay, so the skip should be vanishingly rare and, when it happens, means "unprovable here" rather than "regression". The #277 property it guards is unchanged and still caught: a synchronous `Save()` push fires the sem at elapsed ≈ 0, inside the window, and fails the assertion (#551). +- **Changed**: the docker workflow builds the image on both `linux/amd64` and `linux/arm64` and, on a publish, stitches them into one multi-arch manifest — so `docker run ghcr.io/orlyatomics/orly` resolves natively on Apple Silicon instead of printing a platform-mismatch warning on the first command in the README. Each architecture is built on its own **native** runner (`ubuntu-24.04` / `ubuntu-24.04-arm`) rather than via buildx+QEMU: the image compiles the whole source tree (1753 build jobs, ~10 min natively), which under emulation would be hours and exceed the job timeout. The side benefit is that every existing gate — MCP smoke against the containerised server, in-container `orlyc` compile, REPL entrypoint mode — now runs **on the arm image too**, so the published arm64 artifact is proven to work rather than merely to build. The manifest job runs only when both legs pass, and asserts the published manifest advertises both platforms rather than assuming the stitch worked (#548). +- **Changed**: the docker image job reports `nproc` and `free -g` alongside the architecture, and takes its build parallelism and timeout from the matrix — 4 workers / 90 min on x86, 2 workers / 180 min on arm64. The first arm attempt was killed at the 90-minute cap still inside `jhm -c release`, while the same runner class completes `make debug` in 14 minutes (ci.yml). The difference is LTO, which is memory-hungry and largely serial at link time, so if the arm runner has less RAM per core then *fewer* concurrent LTO jobs is faster rather than slower. That is a hypothesis; the capacity report is what confirms or kills it, and not having it is why the first failure was uninformative (#548). +- **Changed**: the `orly-repl` `:help` now teaches the commutative field calls — `*<['hits']>::(int) += 1` and `*<['tags']>::({str}) |= {'blue'}` — with the leading `*` and `::(type)` annotation that the form requires, and a note that `+=` on an absent key folds from the monoid identity so there is nothing to create first. The help previously showed only assignment (`new <['k']> <- 1`) and read, so the one operation the project is pitched on was undiscoverable from the prompt: substituting `+=` into the documented write form fails with an internal compiler path (`orly/type/add_visitor.h, 50`) and the follow-up read returns `0`, which reads as a silently wrong write rather than a syntax error (#548). +- **Changed**: the README leads with the use case rather than the architecture. The concurrent-writer problem and a copy-pasteable 30-second REPL demo (both operators, with their real output) come first; the MCP server is promoted from a parenthetical to a first-class quickstart with a working `npx -y orly-mcp` client config; the 2019-dormancy status paragraph moves from above the fold to its own `## Project status` section near the end. The Docker section now states that the published image is `linux/amd64` and runs under emulation on Apple Silicon rather than implying it is native on "any host" (#548). +- **Changed**: the disk cache and fiber scheduler now use `__builtin_prefetch` instead of `_mm_prefetch`/``. Identical codegen on x86-64 — gcc lowers locality 3 and 2 to `prefetcht0`/`prefetcht1`, which is exactly what `_MM_HINT_T0`/`_MM_HINT_T1` emitted — but it also compiles on aarch64, where the SSE header does not exist. These were the only x86-specific constructs in the tree: there is no inline assembly anywhere, and the fiber fast path is `setjmp`/`longjmp` bootstrapped from `ucontext`, both POSIX, so there is no hand-rolled context switch to port. Verified with an explicit-instantiation probe, since every changed cache site lives inside `TCache` and `-fsyntax-only` on an includer does not compile class-template member bodies (#548). +- **Added**: an informational `aarch64 build` CI job (`ubuntu-24.04-arm`, `continue-on-error`) running `make debug` + `make test`. aarch64 has never been a supported platform, so this reports whether it could be — with evidence rather than opinion — without gating master. It matters because the published docker image is amd64-only, so Apple Silicon users hit a platform-mismatch warning on the first command in the README (#548). +- **Fixed**: dropped `-msse2` from `bootstrap.sh` (both seed compiles) and `root.jhm`. It was a 32-bit-x86-era leftover and a verified no-op on the only supported platform — `g++ -dM -E` emits a byte-identical macro set with and without it on x86-64, `__SSE2__` included — while being a hard `unrecognized command-line option` error on aarch64, which killed the bootstrap before a single translation unit compiled (#548). +- **Added**: aarch64 now builds clean and passes `make test` — 1753 build jobs and 203 test binaries, zero failures, on a native arm CI runner. The entire gap between "Linux only, x86-64" and a working arm build was two changes: `_mm_prefetch` → `__builtin_prefetch`, and dropping a `-msse2` that was a verified no-op on x86-64. No inline assembly exists anywhere in the tree and the fiber fast path is `setjmp`/`longjmp` over `ucontext`, both POSIX, so there was no context switch to port. `README.md` records this as "known to work, not yet released against" — the CI job stays informational rather than becoming a gate, because one green run is evidence and not a support commitment (#548). - **Fixed**: `~TClient` could deadlock forever in `IoThread.join()` when a server push raced destruction: the dispatch thread's #503 exception exit skipped the internal-pipe close that was the i/o thread's only shutdown signal. The i/o thread now polls the destruction semaphore directly (and closes its pipe end on the way out, which also unblocks a dispatch thread parked mid-`Read()`), and the dispatch thread closes the pipe on its exception path too. A second interleaving of the same race — a pending push request's context ref making the dispatch thread the client's LAST owner, so `~TClient` self-joined on it (terminate; previously a silent two-thread leak) — is fixed by detaching instead and having the dispatch loop check a self-destruction flag before touching the object again. Root cause of the `import_replication.test` CI wedge: a client destroyed right after a replication-ack notification storm, under scheduler starvation, hit the race roughly every other run (#537). - **Changed**: `import_replication.test` no longer dereferences RPC futures untimed: every `Try`/install/import/pov call now waits on the future's event fd with a deadline (240s) and fails the test with the RPC's name and the server log tails if no answer comes — the #537 wedge's client half. A recurrence in CI now reads as `RPC [write_val Try] unanswered after 240s` instead of a hung test (#537). - **Added**: `jhm --test` now enforces a per-test timeout (default 600s; `--test-timeout ` or config `test.timeout_secs`, 0 disables). A wedged test binary is SIGKILLed and reported as a named `TIMEOUT:` failure with its output echoed, instead of silently eating the rest of the CI job's 30-minute budget with the only evidence in the orphan-process list (#537). diff --git a/changelog.d/548-arm64-portability.md b/changelog.d/548-arm64-portability.md deleted file mode 100644 index 275ba2be..00000000 --- a/changelog.d/548-arm64-portability.md +++ /dev/null @@ -1,4 +0,0 @@ -- **Changed**: the disk cache and fiber scheduler now use `__builtin_prefetch` instead of `_mm_prefetch`/``. Identical codegen on x86-64 — gcc lowers locality 3 and 2 to `prefetcht0`/`prefetcht1`, which is exactly what `_MM_HINT_T0`/`_MM_HINT_T1` emitted — but it also compiles on aarch64, where the SSE header does not exist. These were the only x86-specific constructs in the tree: there is no inline assembly anywhere, and the fiber fast path is `setjmp`/`longjmp` bootstrapped from `ucontext`, both POSIX, so there is no hand-rolled context switch to port. Verified with an explicit-instantiation probe, since every changed cache site lives inside `TCache` and `-fsyntax-only` on an includer does not compile class-template member bodies (#548). -- **Added**: an informational `aarch64 build` CI job (`ubuntu-24.04-arm`, `continue-on-error`) running `make debug` + `make test`. aarch64 has never been a supported platform, so this reports whether it could be — with evidence rather than opinion — without gating master. It matters because the published docker image is amd64-only, so Apple Silicon users hit a platform-mismatch warning on the first command in the README (#548). -- **Fixed**: dropped `-msse2` from `bootstrap.sh` (both seed compiles) and `root.jhm`. It was a 32-bit-x86-era leftover and a verified no-op on the only supported platform — `g++ -dM -E` emits a byte-identical macro set with and without it on x86-64, `__SSE2__` included — while being a hard `unrecognized command-line option` error on aarch64, which killed the bootstrap before a single translation unit compiled (#548). -- **Added**: aarch64 now builds clean and passes `make test` — 1753 build jobs and 203 test binaries, zero failures, on a native arm CI runner. The entire gap between "Linux only, x86-64" and a working arm build was two changes: `_mm_prefetch` → `__builtin_prefetch`, and dropping a `-msse2` that was a verified no-op on x86-64. No inline assembly exists anywhere in the tree and the fiber fast path is `setjmp`/`longjmp` over `ucontext`, both POSIX, so there was no context switch to port. `README.md` records this as "known to work, not yet released against" — the CI job stays informational rather than becoming a gate, because one green run is evidence and not a support commitment (#548). diff --git a/changelog.d/548-first-run-funnel.md b/changelog.d/548-first-run-funnel.md deleted file mode 100644 index 9a12fee2..00000000 --- a/changelog.d/548-first-run-funnel.md +++ /dev/null @@ -1,2 +0,0 @@ -- **Changed**: the `orly-repl` `:help` now teaches the commutative field calls — `*<['hits']>::(int) += 1` and `*<['tags']>::({str}) |= {'blue'}` — with the leading `*` and `::(type)` annotation that the form requires, and a note that `+=` on an absent key folds from the monoid identity so there is nothing to create first. The help previously showed only assignment (`new <['k']> <- 1`) and read, so the one operation the project is pitched on was undiscoverable from the prompt: substituting `+=` into the documented write form fails with an internal compiler path (`orly/type/add_visitor.h, 50`) and the follow-up read returns `0`, which reads as a silently wrong write rather than a syntax error (#548). -- **Changed**: the README leads with the use case rather than the architecture. The concurrent-writer problem and a copy-pasteable 30-second REPL demo (both operators, with their real output) come first; the MCP server is promoted from a parenthetical to a first-class quickstart with a working `npx -y orly-mcp` client config; the 2019-dormancy status paragraph moves from above the fold to its own `## Project status` section near the end. The Docker section now states that the published image is `linux/amd64` and runs under emulation on Apple Silicon rather than implying it is native on "any host" (#548). diff --git a/changelog.d/548-multiarch-image.md b/changelog.d/548-multiarch-image.md deleted file mode 100644 index c43e213d..00000000 --- a/changelog.d/548-multiarch-image.md +++ /dev/null @@ -1,2 +0,0 @@ -- **Changed**: the docker workflow builds the image on both `linux/amd64` and `linux/arm64` and, on a publish, stitches them into one multi-arch manifest — so `docker run ghcr.io/orlyatomics/orly` resolves natively on Apple Silicon instead of printing a platform-mismatch warning on the first command in the README. Each architecture is built on its own **native** runner (`ubuntu-24.04` / `ubuntu-24.04-arm`) rather than via buildx+QEMU: the image compiles the whole source tree (1753 build jobs, ~10 min natively), which under emulation would be hours and exceed the job timeout. The side benefit is that every existing gate — MCP smoke against the containerised server, in-container `orlyc` compile, REPL entrypoint mode — now runs **on the arm image too**, so the published arm64 artifact is proven to work rather than merely to build. The manifest job runs only when both legs pass, and asserts the published manifest advertises both platforms rather than assuming the stitch worked (#548). -- **Changed**: the docker image job reports `nproc` and `free -g` alongside the architecture, and takes its build parallelism and timeout from the matrix — 4 workers / 90 min on x86, 2 workers / 180 min on arm64. The first arm attempt was killed at the 90-minute cap still inside `jhm -c release`, while the same runner class completes `make debug` in 14 minutes (ci.yml). The difference is LTO, which is memory-hungry and largely serial at link time, so if the arm runner has less RAM per core then *fewer* concurrent LTO jobs is faster rather than slower. That is a hypothesis; the capacity report is what confirms or kills it, and not having it is why the first failure was uninformative (#548). diff --git a/changelog.d/551-sem-flush-race.md b/changelog.d/551-sem-flush-race.md deleted file mode 100644 index e5a4ba31..00000000 --- a/changelog.d/551-sem-flush-race.md +++ /dev/null @@ -1 +0,0 @@ -- **Fixed**: `durable_manager.test`'s `SemFiresOnlyAfterFlush` no longer fails intermittently on loaded CI runners. It asserted that the durability sem has *not* fired immediately after `Save()` — sound only while less than the manager's write delay has actually elapsed, and at a 300ms delay a descheduled test thread could legitimately miss the window, so correct behaviour reported as a failure. The negative assertion is now guarded on measured elapsed time and the fixture runs at a 2s write delay, so the skip should be vanishingly rare and, when it happens, means "unprovable here" rather than "regression". The #277 property it guards is unchanged and still caught: a synchronous `Save()` push fires the sem at elapsed ≈ 0, inside the window, and fails the assertion (#551). diff --git a/changelog.d/554-fiber-tls-aarch64.md b/changelog.d/554-fiber-tls-aarch64.md deleted file mode 100644 index 5b1e9a10..00000000 --- a/changelog.d/554-fiber-tls-aarch64.md +++ /dev/null @@ -1 +0,0 @@ -- **Fixed**: `orlyc` no longer hangs on aarch64 while tearing down its embedded test server. A fiber's stack outlives the OS thread that last ran it — frames come from a cross-thread pool and are routinely re-latched onto a runner on a different thread — but GCC computes the thread pointer once per function (one hoisted `mrs x, tpidr_el0` on aarch64) and derives every `__thread` access from it, so `TFrame::Run()`'s read of `TRunner::LocalRunner` *after* the runnable returned yielded the previous thread's runner and `longjmp`ed onto that thread's scheduler stack. Two OS threads then executed one stack, the hijacked runner's `Run()` never returned, and `~TDurableManager` waited forever on `SchedulerExitedSem` inside `TServer::Shutdown()`. The three fiber thread-locals are now `TFiberSafeLocal` slots read through `noipa` accessors, so the thread pointer is re-read at every access; nothing cheaper works (`volatile`, `thread_local`, an `asm` memory clobber, `returns_twice`, and all four `-ftls-model` settings all still emit one loop-invariant `mrs`). x86-64 folds the TLS base into each access's addressing mode, which is why only arm ever showed it (#554). From 0b0f5eda8c6b686a893b65b438d1459b30a52678 Mon Sep 17 00:00:00 2001 From: Patrick O'Reilly Date: Thu, 17 Sep 2026 02:58:01 -0700 Subject: [PATCH 2/2] changelog: cut v0.1.0, and stop saying we do not cut releases The header line "The project does not currently cut numbered releases" was accurate and is not any more. Replace it with what the process actually is: semver with an explicit pre-1.0 caveat (the language, the wire protocol and the on-disk format can still move between minor versions), fragments in changelog.d/, folded into [Unreleased], moved under a version heading when a release is cut. Move the folded [Unreleased] block under `## [v0.1.0]` and leave an empty [Unreleased] above it. Nothing in the entries changes -- everything in v0.1.0 had already shipped to master. What the tag buys is an immutable image and a version string: today `:latest` is a moving tag and `make version` falls back to a bare commit hash, because `git describe --match 'v[0-9]*'` has nothing to find. 0.1.0 rather than 1.0.0 because it is what the project already publishes everywhere else -- @orlyatomics/orly, orly-mcp and orly-repl are all 0.1.0 on npm, and the one tag in the repo is clients/go/v0.1.0 -- and because 1.0.0 would claim a stability commitment that two gated RFCs (#218, #255) could still change out from under. The empty [Unreleased] keeps its `### Fixed` subheading deliberately: fold_changelog.py anchors on the literal "## [Unreleased]\n\n### Fixed\n", so dropping it would silently break the next fold. Checked by folding a probe fragment against this structure and confirming it lands in [Unreleased] rather than in the released section. --- CHANGELOG.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aa98773f..803a2be0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,10 +2,19 @@ All notable changes to Orly. -The project does not currently cut numbered releases. Entries reference the originating pull request and the closed issue where applicable. Format loosely follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). +Entries reference the originating pull request and the closed issue where applicable. Format loosely follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow [Semantic Versioning](https://semver.org/) with the pre-1.0 caveat that the orlyscript language, the wire protocol and the on-disk format may still change between minor versions. In-repo work lands as a fragment in `changelog.d/`; `tools/fold_changelog.py` folds them into `[Unreleased]`, and cutting a release moves that section under a version heading. ## [Unreleased] +### Fixed + +## [v0.1.0] — 2026-09-17 + +First numbered release. Everything below had already shipped to `master`; this draws a line under it so +there is an immutable image and a version string to refer to, rather than only a moving `:latest` and a +commit hash. The version matches the clients already published at `0.1.0` — `@orlyatomics/orly`, +`orly-mcp`, `orly-repl` on npm, and the `clients/go/v0.1.0` module tag. + ### Fixed - **Fixed**: `orlyc` no longer hangs on aarch64 while tearing down its embedded test server. A fiber's stack outlives the OS thread that last ran it — frames come from a cross-thread pool and are routinely re-latched onto a runner on a different thread — but GCC computes the thread pointer once per function (one hoisted `mrs x, tpidr_el0` on aarch64) and derives every `__thread` access from it, so `TFrame::Run()`'s read of `TRunner::LocalRunner` *after* the runnable returned yielded the previous thread's runner and `longjmp`ed onto that thread's scheduler stack. Two OS threads then executed one stack, the hijacked runner's `Run()` never returned, and `~TDurableManager` waited forever on `SchedulerExitedSem` inside `TServer::Shutdown()`. The three fiber thread-locals are now `TFiberSafeLocal` slots read through `noipa` accessors, so the thread pointer is re-read at every access; nothing cheaper works (`volatile`, `thread_local`, an `asm` memory clobber, `returns_twice`, and all four `-ftls-model` settings all still emit one loop-invariant `mrs`). x86-64 folds the TLS base into each access's addressing mode, which is why only arm ever showed it (#554). - **Fixed**: `durable_manager.test`'s `SemFiresOnlyAfterFlush` no longer fails intermittently on loaded CI runners. It asserted that the durability sem has *not* fired immediately after `Save()` — sound only while less than the manager's write delay has actually elapsed, and at a 300ms delay a descheduled test thread could legitimately miss the window, so correct behaviour reported as a failure. The negative assertion is now guarded on measured elapsed time and the fixture runs at a 2s write delay, so the skip should be vanishingly rare and, when it happens, means "unprovable here" rather than "regression". The #277 property it guards is unchanged and still caught: a synchronous `Save()` push fires the sem at elapsed ≈ 0, inside the window, and fails the assertion (#551).