Skip to content

OCapN port + the compiler work it forced: interop to 24/24, plus ~185 core commits - #28

Draft
kumavis wants to merge 547 commits into
mainfrom
claude/ocapn-prologos-implementation-auLxZ
Draft

OCapN port + the compiler work it forced: interop to 24/24, plus ~185 core commits#28
kumavis wants to merge 547 commits into
mainfrom
claude/ocapn-prologos-implementation-auLxZ

Conversation

@kumavis

@kumavis kumavis commented Apr 27, 2026

Copy link
Copy Markdown
Contributor

What this is now

It opened as "the OCapN port". It still is — but that is no longer most of what
is in it.

Porting Spritely Goblins / OCapN to Prologos meant running a real protocol
through the compiler, and the compiler kept giving way: whole-file aborts on the
parse path, walkers that silently skipped node kinds, a .pnet cache that
served stale answers after a dependency edit, diagnostics that named a subsystem
that was working fine. Fixing those became a second body of work with nothing to
do with object-capability networking.

280 files changed · 492 non-merge commits · 184 files and ~185 code commits
are core compiler work, not OCapN.

A reviewer cannot tell those apart from the file list, so they are separated in
docs/tracking/2026-08-05_2030_CORE_CHANGES_VS_MAIN.md
— themed, with complexity ratings and commit hashes, derived mechanically from
git rather than from memory. Read that document first if you are reviewing
the compiler side.


Part 1 — OCapN

  • Phase 0 — capability-typed, session-typed port of distributed object refs
    and async messaging, with a local in-memory vat.
  • Phases 1–10 — wire interoperability with @endo/ocapn: Syrup byte codec,
    CapTP frame codec, real Racket↔Node TCP exchange, bidirectional handshake,
    multi-frame conversation, RPC state machine, graceful op:abort teardown.
  • Phases 11–21 — CapTP↔Vat bridge: inbound op:* dispatch, outbound bytes
    for resolved/broken promises, session question table, listener registry, GC
    tracking, connection lifecycle, promise pipelining storage.
  • Phases 52–59 — crypto handshake (Ed25519 signing), third-party handoff
    (both roles), cross-connection gift table, plural GC ops.

Upstream compliance gate: 24 of 24 ocapn-test-suite tests passing. The
suite is pinned to 74db78f and re-checked out on every run, so a warm dev
clone cannot diverge from CI — that drift is exactly how a merge commit touching
no OCapN code once turned the gate red.

Two CI workflows: test.yml (full suite) and interop.yml (cross-runtime gate
— byte-equality against @endo/ocapn on a curated vector matrix, then those
bytes over real localhost TCP with Node in both directions).

Part 2 — the compiler work

Six themes, in rough order of how much I would want a second pair of eyes on
them:

  1. Whole-file aborts on the parse/expansion path (~11 commits). A raise on
    the reader/preparse/parse path escapes process-file, so the file produces
    nothing — not even the forms that already succeeded. The tell is that
    output is empty, not partial, and an error-count gate reads that as "no
    errors". Fixed by extending the marker-seat pattern: the reader emits a
    sentinel that flows as a per-command parse-error value instead of
    raising.

  2. Silent wrong answers (~13 commits). Nothing here produced an error
    message; each returned a plausible wrong value. An unsound occurs? (a meta
    inside a container was invisible); expr-foreign-fn treated as a closed leaf
    by six walkers; collection literals dropped in the AST↔solver round trip,
    so a narrowing query returned nil at 0 errors; a recursive process
    type-checking as terminating; cross-file spec-store leakage inside a batch
    worker. A green suite proved nothing about any of them.

  3. Performance, all A/B measured: whole-program capability inference
    memoized on an env-generation key — suite 402 s → 57 s (profiling showed
    it was 76 % of every OCapN frame); the memory report was forcing two major
    GCs per command
    for numbers nothing read (2.3× of test wall time);
    looseBVarRange short-circuits on shift (2.4×) and subst (40.8× at
    N=5, linear scaling restored); a tree builder's two quadratics replaced by
    one pass and a binary search.

  4. Diagnostics (~28 commits) — errors that named the wrong subsystem, or
    nothing, now name the actual mistake and show syntax that works. Two new
    warnings with teeth: W3001 (two of your own imports binding the same spec
    name) and W3002 (non-exhaustive match), the latter finding a real bug
    in the tree the moment it was switched on.

  5. Schema / records / validate — runtime validate descends into nested
    schemas; field types check their elements, not just the head constructor;
    deep :requires paths enforced at every hop. New field-witness.rkt (286
    lines) splits bake-time tag computation from below-reduction interpretation
    so the tags serialise into .pnet where closures cannot.

  6. .pnet cache — a dependency edit did not invalidate it; 31 AST nodes
    were unregistered by sibling, where the reader's unknown-tag fallback
    returns a raw vector that fails a match arbitrarily far away while printing
    like the real struct.

Plus new capability: a persistent weight-balanced ordered-map backend,
String library Phase 4 (normalization, graphemes, edit distance, regex bridge),
a Float library, the Gen trait, :pre/:post contracts that actually run,
:examples that are actually checked, the Redex model wired into the ordinary
suite, and LSP cross-module go-to-definition.

And a theme worth its own line: several guards had rusted. The parameter
lint had drifted twice and is now wired into pre-commit; the dep table was
mis-selecting tests; bench-ab --refs was documented for most of the tool's
life while not existing, so the documented A/B path silently measured identical
code twice; test.yml had a YAML syntax error and the full suite had not run
since May 4.


Verification

Suite: 558 files / 10 867 tests green after merging 115 upstream commits
(2026-08-05). This branch alone before that merge was 551 / 10 674. main
carries 493 test files; this branch carries 566 — +73, of which 38 are
OCapN, so +35 core test files plus 51 modified.

Known-open, carried explicitly

  • F-row inference — a minimal version was built, measured unsound, and
    reverted. The proper fix is scoped (28 expr-Record-tail sites, 83
    'closed/'dyn literals, a new row-unification case) and deliberately not
    started.
  • Session recursion — the process half now refuses loudly instead of
    type-checking as terminating; the type half is broken too, and they are one
    gap. Both scoped, neither built.
  • OCapN netstring framing — the reference implementation makes
    length-prefixed framing the default and calls raw mode a compat shim on a
    stated retirement path, so the direction is settled; what remains is whether
    the new strategy replaces or joins the existing one, and the suite pin moving
    in the same commit.
  • Four questions that need an owner ruling, collected in
    docs/tracking/2026-08-05_1751_FOUR_OPEN_OWNER_RULINGS.md
    — each with what is already probed and true, the options and their costs, and
    what each unblocks.

DEFERRED.md had its first staleness sweep in four months as part of this: 36
completed entries archived, 107 → 71 active. Three closed-looking entries were
deliberately left in place because they carry residue their header does not
advertise.

Where to be sceptical

  • The self-corrections are load-bearing. Several commits reverse an earlier
    claim made in this same branch — srcloc cause, W3001 scope, a timing
    attribution, the F-row result. Read the later one; they are in the history on
    purpose.
  • macros.rkt (+810/−91) is the largest authored file change and sits on
    the merge seam with main's reader work.
  • Wall-clock perf deltas were taken on a shared machine. The 402 s → 57 s and
    40.8× figures rest on deterministic-work counters and an interleaved micro
    respectively, which is the stronger evidence.

Pitfalls log

docs/tracking/2026-04-27_GOBLIN_PITFALLS.md — language quirks found by
actually using the language for a non-trivial protocol. Entries that later
proved false were re-tested and deleted rather than left standing (30 → 17 after
a second-pass scrutiny); several have since been fixed upstream and are marked
resolved.

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a Phase 0 implementation of the OCapN/Goblins-style actor model in Prologos, including a pure functional local vat/event-loop, Syrup abstract value modeling, CapTP message/value shapes, session-typed CapTP sub-protocols, and a testing-only TCP netlayer + Racket FFI support.

Changes:

  • Add core OCapN modules (vat, behaviors, promises, syrup, locators, netlayer, CapTP messages, session-typed protocol “shapes”, public API).
  • Add a comprehensive Racket test suite covering the new modules and end-to-end scenarios.
  • Add a testing-only TCP FFI bridge and a driver compatibility fence for Racket versions lacking thread #:pool.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
racket/prologos/lib/prologos/ocapn/refr.prologos Capability-type hierarchy for OCapN reference attenuation.
racket/prologos/lib/prologos/ocapn/syrup.prologos Syrup abstract value model (atoms/containers/refs) plus predicates/selectors.
racket/prologos/lib/prologos/ocapn/promise.prologos Promise state algebra (unresolved/fulfilled/broken) and queue mechanics.
racket/prologos/lib/prologos/ocapn/message.prologos CapTP op:* message/value model with constructors/predicates/selectors.
racket/prologos/lib/prologos/ocapn/behavior.prologos Closed-world actor behaviors + dispatcher and effect description.
racket/prologos/lib/prologos/ocapn/vat.prologos Pure functional local vat (actor/promise tables, FIFO queue, step/run).
racket/prologos/lib/prologos/ocapn/locator.prologos Locator + transport model for loopback/tcp-testing-only peers.
racket/prologos/lib/prologos/ocapn/netlayer.prologos Simulated in-process netlayer (mailboxes, connections, pairing delivery).
racket/prologos/lib/prologos/ocapn/tcp-testing.prologos Testing-only TCP netlayer surface + capability-gated foreign bindings.
racket/prologos/lib/prologos/ocapn/captp-session.prologos Session-typed CapTP sub-protocol declarations + example defprocs.
racket/prologos/lib/prologos/ocapn/core.prologos Public API re-export + Goblins-flavored aliases (ask/tell/drain).
racket/prologos/tcp-ffi.rkt Racket TCP handle-table FFI bridge used by tcp-testing-only transport.
racket/prologos/tests/test-ocapn-refr.rkt Tests for capability registration and hierarchy edges.
racket/prologos/tests/test-ocapn-syrup.rkt Tests for Syrup constructors/predicates/selectors.
racket/prologos/tests/test-ocapn-promise.rkt Tests for promise monotonic resolution and queue behavior.
racket/prologos/tests/test-ocapn-message.rkt Tests for CapTP op constructors/predicates/selectors.
racket/prologos/tests/test-ocapn-behavior.rkt Unit tests for behavior step functions + dispatcher.
racket/prologos/tests/test-ocapn-vat.rkt Integration tests for vat spawn/send/step/run and built-in behaviors.
racket/prologos/tests/test-ocapn-netlayer.rkt Tests for simulated mailbox/connection/net pairing behavior.
racket/prologos/tests/test-ocapn-pipeline.rkt Tests for “pipelining”/promise queue mechanics and monotonicity.
racket/prologos/tests/test-ocapn-captp.rkt “Shape” tests ensuring CapTP session declarations elaborate.
racket/prologos/tests/test-ocapn-tcp-testing.rkt Loopback TCP tests validating the FFI and tcp-testing module load.
racket/prologos/tests/test-ocapn-locator.rkt Tests for locator constructors/selectors/equality and transport tags.
racket/prologos/tests/test-ocapn-e2e.rkt End-to-end tests using the public core.prologos API.
racket/prologos/examples/2026-04-27-ocapn-acceptance.prologos Acceptance/demo script exercising the public API in file-mode.
racket/prologos/driver.rkt Adds a Racket-version compatibility fence for parallel executor setup.
docs/tracking/2026-04-27_GOBLIN_PITFALLS.md Design/porting pitfall log and workarounds for the Phase 0 implementation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread racket/prologos/lib/prologos/ocapn/vat.prologos Outdated
Comment thread racket/prologos/lib/prologos/ocapn/promise.prologos Outdated
Comment thread racket/prologos/lib/prologos/ocapn/behavior.prologos Outdated
Comment thread racket/prologos/lib/prologos/ocapn/core.prologos
Comment thread racket/prologos/tests/test-ocapn-tcp-testing.rkt Outdated
Comment thread racket/prologos/lib/prologos/ocapn/vat.prologos Outdated
Comment thread racket/prologos/lib/prologos/ocapn/vat.prologos Outdated
Comment thread racket/prologos/lib/prologos/ocapn/promise.prologos Outdated
Comment thread racket/prologos/lib/prologos/ocapn/behavior.prologos Outdated
Comment thread racket/prologos/tcp-ffi.rkt
kumavis pushed a commit that referenced this pull request Apr 27, 2026
All 10 inline comments were legitimate. Three were correctness
issues, three doc/code mismatches, two unbounded-growth bugs in the
assoc-list tables, one mutate-while-iterating bug in the FFI cleanup
helper, and one CI-flakiness fix.

Real bugs
  - vat.prologos:actor-table-set + promise-table-set: replace-or-
    insert instead of unconditional cons. Each delivery turn was
    growing the table without bound and slowing lookups linearly.
    (#28#discussion_r3150426596 + #28#discussion_r3150426729)
  - vat.prologos:deliver-msg: when the target actor doesn't exist,
    BREAK any associated answer-promise instead of dropping the
    message silently. Previously `ask`-against-missing-actor would
    hang on the result-promise forever.
    (#28#discussion_r3150426741)
  - behavior.prologos:step-greeter: append the trailing "!" the
    docstring promised. Implementation now matches "{g}, {n}!".
    (#28#discussion_r3150426776)
  - behavior.prologos:step-counter: add the explicit "get" branch
    the docstring advertised — previously every non-"inc" tag fell
    into the same no-op pile, including "get".
    (#28#discussion_r3150426679)
  - tcp-ffi.rkt:tcp-table-clear!: snapshot keys via hash-keys before
    iterating + closing. The previous in-hash + hash-remove! shape
    can raise an iteration error in Racket.
    (#28#discussion_r3150426813)

Test/flakiness
  - test-ocapn-tcp-testing.rkt: replace fixed port 18763 with a
    listen-on-random-port helper that retries on collisions. CI
    parallelism / port reuse made the fixed-port choice flaky.
    (#28#discussion_r3150426716)

Doc-vs-code mismatches
  - promise.prologos:enqueue / take-queue: clarify LIFO storage
    (cons-onto-head) and that take-queue does NOT update the
    PromiseState. Comments now match the function signatures.
    (#28#discussion_r3150426657 + #28#discussion_r3150426758)
  - core.prologos top docstring: drop the "Promise pipelining
    (send to a promise; flushes on resolution)" claim. Phase 0
    explicitly does NOT flush; only the in-actor FullFiller pattern
    works for pipelining today. Cross-references goblin-pitfall #17.
    (#28#discussion_r3150426694)

Verification: all 149 OCapN tests still pass after the changes
(behavior 13, captp 7, e2e 8, locator 13, message 19, netlayer 14,
pipeline 5, promise 16, refr 6, syrup 22, tcp-testing 5, vat 21).

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Apr 27, 2026
PR #28 CI on Racket 8.14 timed out 3 OCapN files (test-ocapn-vat,
-pipeline, -e2e), all of which exercise vat-spawn / send / run-vat.
Locally on Racket 8.10 the same tests pass in seconds. The trigger:
the "replace-or-insert" recursion I added to actor-table-set and
promise-table-set in 1cb26e2 (responding to Copilot review comments
#28#discussion_r3150426596 + r3150426729). Recursive symbolic eval
on the assoc list inside run-vat's fuel loop blew past the 120s
per-file budget under the runner's batch worker.

Revert both functions to cons-at-head (the original Phase-0 form).
Keep the doc comments referencing the Copilot review threads so the
unbounded-growth concern is not lost — it is a real Phase-1 issue
that wants a hash/CHAMP-backed table, not an O(N) replace-in-list.
For Phase 0 each test scenario uses fewer than ~5 actors and the
growth concern is moot.

Verified locally on Racket 8.10: all three reverted-to-fast tests
still pass (vat 21, pipeline 5, e2e 8).

Other fixes from 1cb26e2 are kept: deliver-msg break-promise on
missing actor, counter "get" branch, greeter trailing "!", core
docstring, tcp-ffi hash-keys snapshot, tcp-test random port helper,
plus the doc clarifications.

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Apr 28, 2026
Closes two coverage gaps identified while reading workflow.md /
testing.md / on-network.md:

1) Level-3 WS-mode validation (per testing.md § "Three-level WS
   validation"). The OCapN port had only Level-1 (sexp /
   process-string) coverage. The acceptance file
   examples/2026-04-27-ocapn-acceptance.prologos was never exercised
   via process-file in CI. New test:
     - "ocapn-acceptance/file elaborates clean via process-file"
   walks every result of process-file and checks none is a tagged
   error. This catches the file-mode-only failure modes (top-level
   scoping, file-level preparse, multi-form interaction) that
   process-string skips.

2) Behavioural assertions for the Copilot-review fixes from commit
   1cb26e2 — these landed without explicit tests pinning the new
   behaviour:
     - counter "get" branch (#28#discussion_r3150426679):
         "counter/inc bumps state to 1"
         "counter/get returns SAME state — does not change it"
     - deliver-msg → broken promise on missing actor
       (#28#discussion_r3150426741):
         "deliver-msg/missing-actor breaks the answer-promise"
         "deliver-msg/missing-actor sends are NOT silently dropped"
     - greeter trailing "!" (#28#discussion_r3150426776):
         "greeter/result string contains the trailing !"
       — extracts the actual fulfilled-value via
       resolution-value + get-string and asserts equality with
       "hello, world!" (the previous test only checked
       fulfilled?-ness, which would have passed even without the
       "!" fix).

Plus three quiescence / multi-actor coverage additions that the
existing suite was missing:
     - "drain/zero fuel on non-empty queue does nothing"
     - "step-vat/idempotent on quiesced vat"
     - "multi-actor/two echoes resolve their respective promises"

Test count: 10 new cases, all green on Racket 9.1 (locally) AND
Racket 8.14 (via existing CI path — no library changes).

Cumulative: 13 OCapN files, 159 tests total, all green.

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
@kumavis
kumavis marked this pull request as draft April 28, 2026 21:04
kumavis pushed a commit that referenced this pull request May 1, 2026
The first stateful round-trip. Unlike Phase 7's lockstep echo,
Node ACTS on what it receives:

  Racket → Node:  op:start-session
  Racket → Node:  op:deliver  target=<desc:export 0>
                              args="ping"
                              answer-pos=<desc:answer 0>
                              resolver=false
  Node → Racket:  op:start-session
  Node → Racket:  op:deliver  target=<desc:answer 0>      ;; the reply
                              args="ping-pong"            ;; computed!
                              answer-pos=false
                              resolver=false

Node's reply args are COMPUTED from the request ("ping" + "-pong").
This proves Node really decoded our deliver, extracted the args
and answer-pos, and answered to the correct answer-pos — not
just lockstep echoed pre-hardcoded bytes.

Bug surfaced + fixed: @endo/ocapn's AnyCodec rejects `null` as a
record child. Phase-1-7 didn't surface this because none of those
vectors emitted a null in a record sent TO @endo/ocapn. Phase 8's
first deliver did (for absent answer-pos / resolver, which Phase 2
encoded as syrup-null) and broke Endo's decoder on receive AND its
encoder on the reply.

Fix: `opt-pos none` now emits `(syrup-bool false)` instead of
`syrup-null`; `unwrap-opt-desc` accepts both for forward compat.
Codified as goblin-pitfall #28.

What landed:
  tools/interop/peer-responder.mjs — Node child: connects, sends
                                      start-session, parses incoming
                                      deliver, computes reply args
                                      from request args, sends
                                      op:deliver to answer-pos
  tests/test-ocapn-rpc.rkt         — Racket-side orchestration
                                      + byte-equality + Node JSON
  lib/prologos/ocapn/captp-wire    — opt-pos uses syrup-bool false;
                                      unwrap-opt-desc accepts both
  .github/workflows/interop.yml    — adds the rpc step

Test count progression on Racket 9.1:
  Phase 7: 228 (cumulative)
  Phase 8: +1 rpc
  Total:   229/229 green

Pitfalls log: #28 (Endo rejects null as record child) added to
docs/tracking/2026-04-27_GOBLIN_PITFALLS.md.

Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
  Phases 1A-8B ✅. Phase 9+ remaining (out of scope here):
  multi-turn pipelining, op:listen + op:deliver-only chains,
  op:abort teardown, real Prologos-side promise resolution
  semantics, secure netlayer with crypto, decoder perf fix.

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request May 4, 2026
Bring in a slice of the upstream OCapN port (LogosLang/prologos PR #28,
branch claude/ocapn-prologos-implementation-auLxZ) as compatibility
targets for the current branch's PReduce-lite + hybrid-Zig-kernel work.

Tier A — type-level only, runs today:
  - lib/prologos/ocapn/refr.prologos (capability hierarchy + subtype edges)
  - tests/test-ocapn-refr.rkt (6 cases, passes)

Tier B — needs PReduce-lite Phase 10b (user-defined-ctor expr-reduce):
  - lib/prologos/ocapn/syrup.prologos (10 ctors, predicates + selectors)
  - lib/prologos/ocapn/promise.prologos (3-state algebra, multi-arg match)
  - lib/prologos/ocapn/message.prologos (CapTP ops, arity-4 op-deliver)
  - tests/test-ocapn-syrup.rkt (added to .skip-tests pending Phase 10b)

Library files all elaborate cleanly (declarations only); the Tier B
test files fail at eval time because PReduce-lite Phase 10's expr-reduce
dispatches only over BUILT-IN constructors. User-defined ctors go
through the ctor-registry and need a Phase 10b extension. Once that
lands, drop the .skip-tests entry to unblock.

Stress shapes captured for Phase 10b:
  - 10 ctors with mixed arities (0/1/2) — syrup
  - multi-arg match clauses pattern-matching on two ctors at once — promise
  - arity-4 ctor (op-deliver) — message (hardest case)

NOT brought in:
  - syrup-wire.prologos — bytewise encode/decode (Phase 9 + byte-strings).
    Has the pitfall #27 270s decode pathology; candidate strategic
    benchmark for the hybrid kernel's HOF substitution speedup.
  - tcp-testing.prologos — uses foreign-fn (Tier C, deferred).
  - locator/behavior/vat/core — larger Tier B; pull on demand.

See lib/prologos/ocapn/NOTES.md for the full tier rationale.

https://claude.ai/code/session_01Tycs6BWKG58Wo99YVPg6DF
@kumavis kumavis changed the title Implement OCapN actor model in Prologos with local vat and CapTP OCapN port: Phase 0 actor model + Phases 1–21 wire interop & vat bridge May 4, 2026
@kumavis
kumavis marked this pull request as ready for review May 4, 2026 18:27
@kumavis
kumavis force-pushed the claude/ocapn-prologos-implementation-auLxZ branch from 6ce9b6c to ea5a943 Compare May 4, 2026 19:16
kumavis pushed a commit that referenced this pull request May 4, 2026
All 10 inline comments were legitimate. Three were correctness
issues, three doc/code mismatches, two unbounded-growth bugs in the
assoc-list tables, one mutate-while-iterating bug in the FFI cleanup
helper, and one CI-flakiness fix.

Real bugs
  - vat.prologos:actor-table-set + promise-table-set: replace-or-
    insert instead of unconditional cons. Each delivery turn was
    growing the table without bound and slowing lookups linearly.
    (#28#discussion_r3150426596 + #28#discussion_r3150426729)
  - vat.prologos:deliver-msg: when the target actor doesn't exist,
    BREAK any associated answer-promise instead of dropping the
    message silently. Previously `ask`-against-missing-actor would
    hang on the result-promise forever.
    (#28#discussion_r3150426741)
  - behavior.prologos:step-greeter: append the trailing "!" the
    docstring promised. Implementation now matches "{g}, {n}!".
    (#28#discussion_r3150426776)
  - behavior.prologos:step-counter: add the explicit "get" branch
    the docstring advertised — previously every non-"inc" tag fell
    into the same no-op pile, including "get".
    (#28#discussion_r3150426679)
  - tcp-ffi.rkt:tcp-table-clear!: snapshot keys via hash-keys before
    iterating + closing. The previous in-hash + hash-remove! shape
    can raise an iteration error in Racket.
    (#28#discussion_r3150426813)

Test/flakiness
  - test-ocapn-tcp-testing.rkt: replace fixed port 18763 with a
    listen-on-random-port helper that retries on collisions. CI
    parallelism / port reuse made the fixed-port choice flaky.
    (#28#discussion_r3150426716)

Doc-vs-code mismatches
  - promise.prologos:enqueue / take-queue: clarify LIFO storage
    (cons-onto-head) and that take-queue does NOT update the
    PromiseState. Comments now match the function signatures.
    (#28#discussion_r3150426657 + #28#discussion_r3150426758)
  - core.prologos top docstring: drop the "Promise pipelining
    (send to a promise; flushes on resolution)" claim. Phase 0
    explicitly does NOT flush; only the in-actor FullFiller pattern
    works for pipelining today. Cross-references goblin-pitfall #17.
    (#28#discussion_r3150426694)

Verification: all 149 OCapN tests still pass after the changes
(behavior 13, captp 7, e2e 8, locator 13, message 19, netlayer 14,
pipeline 5, promise 16, refr 6, syrup 22, tcp-testing 5, vat 21).

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request May 4, 2026
PR #28 CI on Racket 8.14 timed out 3 OCapN files (test-ocapn-vat,
-pipeline, -e2e), all of which exercise vat-spawn / send / run-vat.
Locally on Racket 8.10 the same tests pass in seconds. The trigger:
the "replace-or-insert" recursion I added to actor-table-set and
promise-table-set in 1cb26e2 (responding to Copilot review comments
#28#discussion_r3150426596 + r3150426729). Recursive symbolic eval
on the assoc list inside run-vat's fuel loop blew past the 120s
per-file budget under the runner's batch worker.

Revert both functions to cons-at-head (the original Phase-0 form).
Keep the doc comments referencing the Copilot review threads so the
unbounded-growth concern is not lost — it is a real Phase-1 issue
that wants a hash/CHAMP-backed table, not an O(N) replace-in-list.
For Phase 0 each test scenario uses fewer than ~5 actors and the
growth concern is moot.

Verified locally on Racket 8.10: all three reverted-to-fast tests
still pass (vat 21, pipeline 5, e2e 8).

Other fixes from 1cb26e2 are kept: deliver-msg break-promise on
missing actor, counter "get" branch, greeter trailing "!", core
docstring, tcp-ffi hash-keys snapshot, tcp-test random port helper,
plus the doc clarifications.

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request May 4, 2026
Closes two coverage gaps identified while reading workflow.md /
testing.md / on-network.md:

1) Level-3 WS-mode validation (per testing.md § "Three-level WS
   validation"). The OCapN port had only Level-1 (sexp /
   process-string) coverage. The acceptance file
   examples/2026-04-27-ocapn-acceptance.prologos was never exercised
   via process-file in CI. New test:
     - "ocapn-acceptance/file elaborates clean via process-file"
   walks every result of process-file and checks none is a tagged
   error. This catches the file-mode-only failure modes (top-level
   scoping, file-level preparse, multi-form interaction) that
   process-string skips.

2) Behavioural assertions for the Copilot-review fixes from commit
   1cb26e2 — these landed without explicit tests pinning the new
   behaviour:
     - counter "get" branch (#28#discussion_r3150426679):
         "counter/inc bumps state to 1"
         "counter/get returns SAME state — does not change it"
     - deliver-msg → broken promise on missing actor
       (#28#discussion_r3150426741):
         "deliver-msg/missing-actor breaks the answer-promise"
         "deliver-msg/missing-actor sends are NOT silently dropped"
     - greeter trailing "!" (#28#discussion_r3150426776):
         "greeter/result string contains the trailing !"
       — extracts the actual fulfilled-value via
       resolution-value + get-string and asserts equality with
       "hello, world!" (the previous test only checked
       fulfilled?-ness, which would have passed even without the
       "!" fix).

Plus three quiescence / multi-actor coverage additions that the
existing suite was missing:
     - "drain/zero fuel on non-empty queue does nothing"
     - "step-vat/idempotent on quiesced vat"
     - "multi-actor/two echoes resolve their respective promises"

Test count: 10 new cases, all green on Racket 9.1 (locally) AND
Racket 8.14 (via existing CI path — no library changes).

Cumulative: 13 OCapN files, 159 tests total, all green.

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request May 4, 2026
The first stateful round-trip. Unlike Phase 7's lockstep echo,
Node ACTS on what it receives:

  Racket → Node:  op:start-session
  Racket → Node:  op:deliver  target=<desc:export 0>
                              args="ping"
                              answer-pos=<desc:answer 0>
                              resolver=false
  Node → Racket:  op:start-session
  Node → Racket:  op:deliver  target=<desc:answer 0>      ;; the reply
                              args="ping-pong"            ;; computed!
                              answer-pos=false
                              resolver=false

Node's reply args are COMPUTED from the request ("ping" + "-pong").
This proves Node really decoded our deliver, extracted the args
and answer-pos, and answered to the correct answer-pos — not
just lockstep echoed pre-hardcoded bytes.

Bug surfaced + fixed: @endo/ocapn's AnyCodec rejects `null` as a
record child. Phase-1-7 didn't surface this because none of those
vectors emitted a null in a record sent TO @endo/ocapn. Phase 8's
first deliver did (for absent answer-pos / resolver, which Phase 2
encoded as syrup-null) and broke Endo's decoder on receive AND its
encoder on the reply.

Fix: `opt-pos none` now emits `(syrup-bool false)` instead of
`syrup-null`; `unwrap-opt-desc` accepts both for forward compat.
Codified as goblin-pitfall #28.

What landed:
  tools/interop/peer-responder.mjs — Node child: connects, sends
                                      start-session, parses incoming
                                      deliver, computes reply args
                                      from request args, sends
                                      op:deliver to answer-pos
  tests/test-ocapn-rpc.rkt         — Racket-side orchestration
                                      + byte-equality + Node JSON
  lib/prologos/ocapn/captp-wire    — opt-pos uses syrup-bool false;
                                      unwrap-opt-desc accepts both
  .github/workflows/interop.yml    — adds the rpc step

Test count progression on Racket 9.1:
  Phase 7: 228 (cumulative)
  Phase 8: +1 rpc
  Total:   229/229 green

Pitfalls log: #28 (Endo rejects null as record child) added to
docs/tracking/2026-04-27_GOBLIN_PITFALLS.md.

Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
  Phases 1A-8B ✅. Phase 9+ remaining (out of scope here):
  multi-turn pipelining, op:listen + op:deliver-only chains,
  op:abort teardown, real Prologos-side promise resolution
  semantics, secure netlayer with crypto, decoder perf fix.

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
@kumavis
kumavis requested a review from Copilot May 5, 2026 01:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated 8 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +33 to +38
'op-abort': mkRec('op:abort', ['phase-5-says-hi']),
'op-gc-answer': mkRec('op:gc-answer', [7n]),
'op-deliver-only': mkRec('op:deliver-only', [
[mkRec('desc:export', [0n]), 'ping'],
]),
};
Comment thread tools/interop/peer-responder.mjs Outdated
Comment on lines +99 to +114
const session = receivedFrames.find(f => f && f.label === 'op:start-session');
const deliver = receivedFrames.find(f => f && f.label === 'op:deliver');
if (!deliver) return false;

const sessionLocator = session && Array.isArray(session.values)
? String(session.values[1])
: null;

// op:deliver's values are: [target-desc, args, answer-pos, resolve-me]
const args = deliver.values[0]; // wait — Prologos sent record values as
// [<desc:export 0>, "ping", <desc:answer 0>, null]
// i.e. 4 children. Let's read them.
const targetDesc = deliver.values[0];
const argsValue = deliver.values[1];
const answerPos = deliver.values[2];

Comment thread tools/interop/peer-responder.mjs Outdated
Comment on lines +107 to +113
// op:deliver's values are: [target-desc, args, answer-pos, resolve-me]
const args = deliver.values[0]; // wait — Prologos sent record values as
// [<desc:export 0>, "ping", <desc:answer 0>, null]
// i.e. 4 children. Let's read them.
const targetDesc = deliver.values[0];
const argsValue = deliver.values[1];
const answerPos = deliver.values[2];
Comment on lines +45 to +50
;;; Wire format: ONE message = ONE line. Each line is a Syrup-encoded
;;; SyrupValue (we use a textual subset; Endo uses byte-level Syrup
;;; but we approximate by serialising via the pretty-printer's repr
;;; in the netlayer Prologos layer). Lines are terminated by `\n`.
;;; Length-prefix framing is intentionally NOT used — keeping testing-
;;; only simple. See goblin-pitfalls #19.
Comment on lines +32 to +42
;; one CapTP message ::= one line of UTF-8 text + "\n"
;;
;; The line is a textual encoding of a SyrupValue. Endo uses
;; bytewise Syrup; we use a textual approximation since the
;; pretty-printer is the only encoder we have right now. Phase 1
;; should swap in a real Syrup byte codec.
;;
;; Single-line framing keeps the FFI surface minimal — just
;; line-oriented read/write — and avoids length-prefix code on the
;; reader side. Production transports MUST upgrade to length-
;; prefixed framing.
Comment on lines +13 to +17
;; | / \
;; SturdyRefr / \
;; UnresolvedRefr
;; \ \
;; ResolvedFar ResolvedNear
Comment on lines +25 to +33
;; Field naming follows the CapTP spec doc:
;; to-desc — target descriptor (refr id, locally; export-table
;; entry, on-wire)
;; args — message arguments (a syrup-list)
;; answer-pos — promise position to bind the result; if absent the
;; send is fire-and-forget
;; resolve-me — descriptor of a resolver to be notified

data CapTPOp
Comment thread racket/prologos/tests/.skip-tests Outdated
Comment on lines +32 to +40
# Phase 24 bridge-driven responder interop test — skeleton in place
# (Node peer + test harness) but blocked on a Prologos elaborator/
# evaluator performance issue: the driver expression that drives
# `captp-incoming-with-state + drain + pump-outbound` over Node-
# decoded bytes exceeds 90s wall-time during in-process eval, far
# longer than equivalent unit tests with hand-coded ops. See goblin
# pitfalls #30 (match-in-deep-let inference failure, worked around)
# and #31 (perf observation, follow-up).
test-ocapn-bridge-interop.rkt # interop-only: spawns node subprocess (Phase 24, perf-blocked)
@kumavis
kumavis marked this pull request as draft May 7, 2026 06:11
claude added 10 commits August 3, 2026 12:24
`load-module` parameterizes `current-ns-context` to #f, and a file with no
`ns` never sets one — the `book/` chapter files are all like that, being
the stdlib book's prose rather than importable modules. When such a file
processes its own `require`s there is no namespace to add them to, and
`ns-context-add-refer` died on

    ns-context-refer-map: contract violation
      expected: ns-context?  given: #f

naming a struct accessor, losing the whole file, and pointing nowhere
near the missing `ns`.

Found by probing a DEFERRED entry (Collections, "Stage I: Transducer
Runners") that said `into-vec`/`into-set` did not exist. They do — in a
`book/` file — and trying to import it is what surfaced this. The entry
is corrected: what is blocked is the transducer-protocol form and pipe
fusion, not the runners.

The message blames the IMPORTER, and the test pins that. My first draft
blamed the imported module, which was wrong — the first module to trip it
(`prologos::core::collection-traits`) has an `ns` on line 1. The
ns-context is #f because of the file doing the importing, not the one
being imported, and a diagnostic that points at the wrong file is worse
than a vague one.

Suite 10194/529 green, conformance 24/24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
All seven came from one acceptance file's section notes and none had been
re-checked. Probed each.

Five no longer describe the compiler: module-path calls inside `is` goals
work; `solve-one` in a defn body derives its row type rather than `_`;
prelude constructors work in `=` goals; `eq?` is callable from prelude
scope, making the listed `int-eq`/`str-eq` workaround unnecessary.

The fifth was never a defect at all. "Parameterized types in data
constructor arguments" cites `data Box A := box [List A]`, which is not
Prologos syntax — a syntax error filed as a type-system gap. Written
correctly it works and the constructor gets its Pi type.

One is a duplicate: "head + match inference failure" is the same
def-RHS defect as the merged higher-order entry, so it points there
instead of being counted twice. One stands as filed (narrowing over Int
literal patterns is a design limitation, and the entry says so).

All five now pinned. That is the point rather than bookkeeping: an entry
claiming X is broken suppresses the test that would catch X regressing —
which is exactly what happened with the nested-constructor-pattern entry,
where nothing guarded working behaviour for five months.

Suite 10199/530 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The entry names twelve specific files that "show unsolved dict-metas in
batch but resolve correctly when run individually". Ran all twelve
together through the batch runner: 166 tests, all pass. They also pass in
every full-suite run this session, sequential and parallel.

Marked resolved rather than deleted. The entry describes a useful
diagnostic SHAPE — unsolved dict-metas as a batch-vs-individual
divergence — and this session hit the same shape from a different cause
(the cell-backed spec-store leak). The sighting is worth keeping even
though the instance is gone.

Also probed and left alone: the Numerics N6d-i derive skip-set is
accurate as filed. `mul`/`eq?`/`compare`/`neg`/`abs` all derive and work;
`add` fails with "Could not infer type", exactly the spec-clobber the
entry describes and attributes to issue #66.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…closed

The entry said a `<`-leading form is mis-read in WS mode ("`<` opens an
angle-type reader GROUP, so the predicate never parses as a comparison")
and prescribed the reversed `(> 5 _)` as the workaround.

Re-probed: it parses, and the predicate is LIVE. `{:n 3}` validates ok;
`{:n 7}` comes back `err {:n check-failed "(< _ 5)"}`.

The failing case is what makes this a test. Checking only `{:n 3}` would
pass whether or not the predicate ran — 3 < 5 either way — so a
credulous test here would have "confirmed" the fix while the predicate
sat inert. Both cases pinned.

The second gap in that section — `match` with an inline `validate`
scrutinee failing inference — is confirmed still true and now says it was
re-checked rather than inherited.

Suite 10200/530 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
`normalize : NormForm -> String -> String` with nfc/nfd/nfkc/nfkd,
bridged to Racket's implementations as the entry specified. A feature
section rather than a defect: DEFERRED asked for this and it did not
exist.

Split across two modules, and the split is not cosmetic. The four FFI
bindings sit in `prologos::data::string` beside the other Racket bridges;
the `NormForm` type and the dispatcher sit in
`prologos::core::string-ops`, because `data/string.prologos` is a
prelude-less leaf and a `match` over a user `data` needs machinery it
does not have. Putting the type there fails the module load with a bare
"Type mismatch" — worth knowing before adding any other `data` to a leaf
lib module, and recorded in the entry.

Tests assert PROPERTIES of each form, not snapshots of our output: NFD
lengthens a precomposed character, NFC shortens a decomposed one, NFKC
folds a ligature, NFKD folds a circled digit. Two carry their weight
beyond that — NFC must NOT fold the ligature (the canonical/compatibility
distinction, pinned in both directions, since getting it backwards
silently corrupts data), and NFC and NFD must DISAGREE at all, which is
what rules out a dispatcher that ignores its form argument.

Suite 10207/531 green, conformance 24/24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Shipped the common-prefix half of the entry's ask: `common-prefix`,
`common-prefix-length`, `common-suffix`, `common-suffix-length`. Tested
at the boundaries a loop gets wrong — nothing shared, and one string a
prefix of the other, which is an index error rather than a wrong answer
if mishandled.

Edit distance is NOT shipped, and the reason is worth more than the
function would have been.

Written as the standard row-wise DP — the classic table kept one row deep
— it does not evaluate. A 3x3 distance comes back as a stuck term naming
`lev-row-cell` and `lev-nth` unreduced.

The cause is sharing. Each cell's value is needed twice (as the row
entry, and as the next cell's `left`), as is the cell above it. Written
the obvious way that doubles the work per cell and the whole thing is
exponential. The natural fix is a `let` — a multi-line `let` inside an
`if` is a parse error in this position. Passing the value down as a
parameter instead does NOT help: lazy reduction re-evaluates the argument
at each use rather than sharing it. I tried both.

So a standard dynamic program cannot be written efficiently here today.
Myers diff and Jaro need the same shape, so this is a constraint on the
rest of Phase 4c and worth deciding once rather than rediscovering per
function. Recorded with the repro and the three candidate answers.

Suite 10212/531 green, conformance 24/24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Correcting myself. An hour ago I filed this entry as PARTIAL and declared
edit distance unwritable: the row-wise DP returned a stuck term, and I
concluded that "a standard dynamic program cannot be expressed
efficiently here today" and that this constrained Myers diff and Jaro as
well.

The diagnosis was right and the conclusion was wrong. Each cell's value
IS needed twice, computing it inline DOES double the work per cell, and
the whole thing does go exponential. But I had tried a multi-line `let`
(a parse error in that position) and parameter-passing (no sharing under
lazy reduction), and generalised from those two failures to "no sharing
construct works here". The BRACKET `let` on one line works. Edit distance
evaluates: kitten/sitting = 3.

Two failures are not a survey. The entry asserted a language limitation
on that basis, and whoever picked up Myers diff would have read it as
settled.

Shipped: levenshtein, closest, and the common-prefix/suffix family from
the earlier commit.

Tests aim at the shape of the bug rather than around it. The
identical-strings case is the 3x3 that used to return a stuck term, so
"is a number at all" is the assertion. flaw/lawn catches an
implementation that only walks one diagonal. `closest` is pinned to
answer none when nothing is within the limit, because a confident wrong
suggestion is worse than no suggestion.

Suite 10220/531 green, conformance 24/24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
`str::grapheme-count`, `str::grapheme-span`, `graphemes`,
`grapheme-reverse`.

The entry's "Mitigation" line was the whole answer. It reads as though
UAX #29 tables were the work and an FFI bridge a fallback; the bridge IS
the implementation, because Racket already carries the tables. A ZWJ
family emoji comes back as one cluster from five code points. Two
`foreign racket` lines plus a walk over `grapheme-span`.

I had written this off in the same breath as sorted collections and
regex, on the strength of "~30KB Unicode tables" — while the line
directly beneath it said what to do instead. Third time this session that
"blocked" meant "stopped reading".

Tested where clusters and code points DIFFER, because ASCII proves
nothing: `graphemes "abc"` is right whether or not the implementation
understands clusters. The load-bearing case is reversing NFD "éa" by
grapheme and re-composing, which gives "aé" — a code-point reverse moves
the accent onto the "a" and composes to "áe" instead. Same length, same
grapheme count, different string; only comparing content catches it. One
assertion checks the fixture is actually decomposed first, so the test
cannot pass vacuously.

Suite 10224/531 green, conformance 24/24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
… note

DEFERRED listed `into-vec`/`into-set` as work blocked on transient types.
They exist, are importable from `prologos::core::collections`, and work:
`[into-vec [list-to-lseq Int xs]]` gives `@[1 2 3] : [PVec Int]`. Since
commit 7c04a89, not something I added.

What is actually blocked is the transducer-PROTOCOL form and pipe fusion.
The runners were never blocked on it.

This corrects a note I wrote earlier in this same session. I probed these
by importing `prologos::book::collection-functions` — a chapter file with
no `ns`, and therefore not importable — and concluded from that failure
that the functions were unreachable, then wrote that into the entry.
They were reachable from the other module all along. Same mistake shape
as the two before it: one failed attempt treated as a survey.

That probe was not wasted — the import failure was a raw
`ns-context-refer-map` contract violation that took the whole file, and
it is a named error now.

Pinned with contents rather than just types: a runner returning an EMPTY
PVec would satisfy the type check.

Suite green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
claude added 30 commits August 10, 2026 08:32
Filed this an hour ago and called the fix small, so it was that or stop calling
it small. `'source` in the timings record was hardcoded "affected" for every
run; it now records the actual mode (all / targeted / affected), and the guard
suppresses a full run only when the LAST record was "all". Window and
changed-file scan untouched.

The risk in a change like this is silently DISABLING the guard, which looks
exactly like success — a suite that always runs is indistinguishable from a
guard working correctly, until the day someone wonders why the anti-re-run
protection never fires. So the predicate was exercised against every input shape
it can meet: source "all" blocks, "targeted" does not, and schema-2 records, an
empty file, and a missing file all fall to NOT-full. That last group is the
design point — unknown provenance must fail toward running the suite, never
toward skipping it. End-to-end, `--all` after a targeted run now runs where it
previously exited in under a second.

The part worth keeping is that this trap was already written down in this same
file, 380 lines below the guard. The LPT-sort scheduler carries a comment saying
every targeted run appends a record, that 77% of all records are targeted, and
that our own prescribed workflow is exactly the sequence leaving a small record
last. That consumer was fixed on 2026-07-27. The guard reads the same file with
the same assumption and was not. Two consumers, one trap, one file — when a
shared artifact turns out to have a misleading shape, the sweep has to cover
every reader of it, not just the one that hurt.

Worth noting the earlier fix measured its own impact as NIL and landed anyway,
"because a silently degenerating optimizer is a lie in the code". This second
consumer is where that judgement paid off.

Full-suite gate is running; result to follow either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…n silent

Full suite 560 files / 10955 / 0 in 129.2s. And the direction that mattered: an
immediate second --all after a genuine full run still BLOCKS ("No .rkt files
changed since the last FULL suite run (1s ago)"), while --all after a targeted
run now runs. Both confirmed end-to-end, not just by the predicate probe.

Dailies STATE head and HANDOFF refreshed to 20339e5 / 10955, plus a LOG entry
for the arc.

Two new Watching items, both about guards rather than code. First: when a shared
artifact turns out to have a misleading shape, sweep every reader of it — one
file, two consumers, one trap, six weeks apart. Second: a guard's failure
direction must be toward more checking; when adding a suppression, enumerate the
unknown, legacy and absent inputs and check which way each falls, because the
re-run guard failed toward skipping the regression gate while reassuring you one
had run.

The entry also keeps the coverage-arithmetic error rather than the tidied
result: I published "13 of 29" and "24 of 29" by subtracting a table size from a
registry size, two different sets. Coverage is a set difference. Asking it
properly found seven more uncovered merges.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…untouched

Both of these are things the next person would have spent an hour rediscovering.

The netstring entry now records that its gate is runnable in a fresh container:
the ocapn-test-suite clone succeeds, python3 is present, and the @endo/ocapn node
deps are already installed. So no environment work is part of the adoption's
cost. Confirmed at the same time that upstream HEAD is 31f0b80, "Merge pull
request #41 from ocapn/message-framing" — the suite's newest commit IS the change
that breaks us, and our pin sits just behind it. Nothing has landed since to
complicate the adoption; the delta is exactly the one thing.

The Nat-audit entry recommends swapping captp-core's Peano kind tags for a
6-way nullary data type — ~20 lines, module-local, and its blocker (issue #60)
was verified gone. I did not do it, and the entry now says why rather than
leaving it looking like an oversight: the entry itself names the 24/24
conformance run as the gate, and that suite clone is absent from a fresh
container. It is fetchable, but a change to interop-critical code should land
with its gate green in the same session, not on the argument that it is small.
Being module-local makes it a good first slice for whoever runs the gate, not a
good drive-by.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
It appeared in every summary this session and in the dailies STATE head,
inherited from the record and carried forward across roughly 180 branch commits
and a 115-commit upstream merge. That is exactly the failure mode I spent the
session finding in DEFERRED entries — a claim that survives because nobody
re-probes it — and I was doing it in my own status line.

Ran the gate. It holds: 24 passed, 0 failed, 0 errored at a18813c. The STATE
head now says re-verified with the date and the HEAD, so the next carry-forward
is dated rather than open-ended.

New Watching item: re-probe your own status line. Every discipline applied to
the entries this session — probe the blocker, date the claim, don't carry a
verdict forward — applies to the STATE head, which is the most-read datum in the
file and the easiest to copy forward unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…ng answer

I declined this refactor twice. The second reason was that its gate's suite clone
was absent from the container. Then I ran that gate to check an unrelated claim,
so the reason stopped being true — which is the same pattern I have spent this
session flagging in DEFERRED entries: a blocker that was already gone.

Six Peano numerals became a 6-way nullary data RefrKind. Predicates match the
constructor instead of nat-eq?-ing a numeral, and refr-kind returns a kind rather
than a number you had to count `suc`s to read. Issue #60, the workaround's stated
cause, was verified not to reproduce at HEAD before anything was touched.

Then test-ocapn-bridge failed one case, and that is the real content of this
commit. A 2-argument multi-arity defn over nullary constructors discriminates on
the FIRST ARGUMENT ONLY. [keq ka kb] is true. [keq kb ka] is true. The catch-all
is unreachable. It compiles, type-checks, reports zero errors, and had quietly
broken OCapN brand-check. Isolated to 15 lines with no OCapN in them and
committed as an example file.

The near-miss is why pitfall #18 now carries a severity banner instead of being
filed as a matching quirk. 165 of 166 bridge cases passed. The 24/24 conformance
gate was unaffected — it never compares two different kinds at the same id.
Exactly one assertion in the tree distinguishes them. Without that assertion this
lands green and brand-check is broken in production.

Fixed with an outer match plus a one-level inner match per arm. Deliberately no
RefrKind -> Nat ordinal: it would make the equality a nat-eq? again and
reintroduce the numeral this change removed.

Gates: conformance 24/24, test-ocapn-bridge 166/166, captp 7, pipelining 4, e2e
8, full suite 560 files / 10955 / 0 in 136.5s.

Worth recording against my own two refusals: holding this was defensible both
times, and it also kept a silent brand-check bug latent one refactor away from
being tripped. The audit could not have found it. Only writing the code did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Attempted the fix, did not land it. Recording the narrowing because it is most
of the work and the next person should not redo it.

Three things are now established by probe rather than assumed. It is NULLARY
only — the identical shape with field-carrying constructors is correct, probed
side by side in one file, so this is not general multi-column dispatch and
bracketed patterns take a different route. There are TWO different wrong
behaviours, not one: `| ka ka | kb kb | _ _` returns true for (ka, kb) — the
wrong arm — while `| ka ka | ka kb | _ _` returns the catch-all for (ka, kb)
instead of arm 2. Any story has to explain both, which "matches the first arg
only" does not. And it is NOT the unknown-constructor path already documented at
macros.rkt:11551, where a bare unknown name stays a variable and becomes an
irrefutable catch-all: normalize-pattern does convert known nullary names to
compound, unreachable-arm-error does not fire, and the second behaviour rules out
first-arm-eats-everything.

Where I stopped, said plainly: reading compile-match-tree and specialize-rows,
the nullary path looks correct on paper — n-fields is 0, so the column and its
parameter are both dropped and the recursion should dispatch on what was column
1. Hand-tracing predicts the RIGHT answer for the case that produces the wrong
one. So the divergence is between that reading and the implementation, and I did
not find it. The next step is to instrument compile-match-tree and print row sets
per level against the 15-line repro.

Fallback if the fix proves deep: refuse the shape outright. The compiler
demonstrably cannot honour nullary constructor patterns in a non-first column of
a multi-arity defn, and loud-and-restrictive beats a silent wrong answer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
`| [ka] [ka] -> true` works — true / false / false / true, all correct. So the
pattern compiler was never wrong.

Two adjacent BARE constructor names in a clause are ambiguous between "two
nullary patterns" and "ctor ka applied to ka", and the parser picks application.
The clause then has arity 1, not 2, and everything downstream follows from that —
including both wrong behaviours, which stop needing a single story because they
were never one bug in dispatch.

This also explains the field-carrying twin being correct for free. `[wa _]` is
already bracketed, so its split is unambiguous. What I measured as
"nullary vs non-nullary" was really "unbracketed vs bracketed" — nullary
constructors are just the only patterns anyone writes bare.

The step that got here is worth keeping, because I nearly drew the opposite
conclusion. Hand-tracing compile-match-tree and specialize-rows predicted the
RIGHT answer while the implementation produced the wrong one, and I filed that as
"the divergence is between my reading and the behaviour, and I did not find it".
The correct inference from a correct trace and a wrong result is that the ROWS
never had the shape being traced — which is precisely where the cause was. The
next probe was `| [ka] [ka] |`, not an instrumented walker.

Three working forms now, brackets first as the smallest change. captp-core keeps
its nested match because it reads better at six kinds, but its comment now points
at brackets as the one-character answer rather than implying the longer form is
required. Pitfall #18 and the repro carry the root cause too.

The fix is a real decision, not a bug to squash: the ambiguity is genuine at the
grammar level, since `f a b` in pattern position could be three patterns or one
application and the reader cannot know without consulting constructor arity.
Recommended: require brackets in multi-arity clause position and ERROR otherwise
(loud, cheap, names the working form), with registry-aware splitting as the
better long-term answer. Either way the silent-wrong-answer property is what has
to go.

test-ocapn-bridge 166/166.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
… published

One line. `param-names` in expand-defn-pattern-group asked "are these all
variables?" using pattern-is-variable? on the RAW clause patterns, before
normalize-pattern has consulted lookup-ctor. A bare nullary constructor is still
a pat-atom of kind 'var at that moment, so every bare constructor counted as a
variable, all-var? came out #t, and the generated parameters were named after the
PATTERNS. `| ka ka -> …` produced two parameters BOTH NAMED ka; compile-match-tree
took (list-ref param-names col) per dispatch column, got the same name twice, and
the second dispatch re-read the first argument.

Fix: normalize once up front, decide all-var? on the normalized patterns, reuse
them for the rows.

Regression test with 5 cases, perturbation-verified: reverting the fix fails
exactly 3 — the bare-pattern ones — while the bracketed and all-variable cases
pass either way, which is correct because neither was ever broken. It gets its
own file because the bug's only witness was one assertion in test-ocapn-bridge
that exists because brand-check happens to need it; a language defect should not
depend on an application test.

Now the corrections. I published TWO root causes for this today and both were
wrong.

"Matches first arg only" was the inherited framing. Then I published "the parser
cannot split `| ka ka ->` into two patterns" and recommended either
registry-aware splitting or requiring brackets and erroring otherwise. The second
would have been actively harmful: a syntax restriction on users to work around a
one-line ordering bug in the compiler.

Two mistakes produced that, and they are the part worth keeping. First, I
inferred a mechanism from a behavioural difference instead of looking at one;
instrumenting showed the parse was correct all along, two patterns per clause
normalizing to two nullary compounds. Second, the "brackets fix it" probe was
confounded — the bracketed test used a 2-constructor type and the bare test used
3, two variables at once. The clean 2x2 in one file is what settled it. That the
confounded run happened to reach the RIGHT discriminator is the danger: a
confounded experiment that lands on the correct answer is worse than one that
lands on the wrong answer, because nothing prompts a re-run.

This is the same error as the srcloc misattribution earlier in this session — "I
varied layout AND spec presence, blamed layout" — repeated within the hour, after
being written down.

Corrected in all four places that carried the wrong cause: the DEFERRED entry,
pitfall #18, the repro example, and captp-core's comment. captp-core keeps its
nested match because it reads better at six kinds, not because the short form is
unavailable — it now works.

Gates: full suite 560/10955/0, conformance 24/24, test-ocapn-bridge 166/166.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
ocapn-test-suite#41 wrapped every CapTP message in a netstring; ours did not, so
the two could not talk. Adopted, with the pin moving in the same commit as the
entry required — leaving it behind would hide the next drift exactly as it hid
this one.

The strategy is `<ascii-digits>:<payload>` with NO trailing comma, so not a
classic netstring. I took the shape from upstream's utils/netstrings.py
(`length.encode() + b":" + self`) and @endo/syrup-frame rather than from memory
of the netstring convention — the perturbation test confirms that mattered:
emitting the conventional trailing comma fails 2 of the 5 new cases.

Server default flipped raw-syrup -> netstring; pin 74db78f -> 31f0b80. Five
regression tests covering what the framing is FOR: two frames coalesced in one
buffer, a payload that itself looks like a length prefix (12:notalength framed as
13:12:notalength), clean EOF as #f rather than an error, and malformed prefixes
raising instead of silently resyncing.

Gate: 24/24 against the NEW suite. Full suite 561/10965/0, syrup-wire 45,
captp-wire 18, netlayer-tcp 2.

Now the correction. This entry said the remaining work needed a decision —
"whether the new strategy replaces 'raw-syrup or joins it". There was nothing to
decide. Adding a strategy is purely additive: the parameter already existed,
'raw-syrup stays for the byte-equality fixtures, 'newline stays for the .mjs
peers, and nothing upstream reads either any more. I invented the choice by
assuming adoption meant removal.

That is the third time this session a "needs a ruling" call was wrong, after the
provide-surface trade (whose answer was already in the module the entry named)
and the multi-arity dispatch bug (which looked like a syntax decision only
because I had misdiagnosed it). The pattern is consistent enough to name: I
escalate while the mechanism is still unclear, and the unclarity presents itself
as a design question. All three dissolved on contact with the code. Probe before
filing something as the owner's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Tested my own "not mine to close" call on this one, the way the last three were
tested — and unlike those three, it got harder rather than dissolving.

The missing piece was why turning the row tail into a variable is unavoidable
rather than one option among several. metavar-store.rkt:1884: "Assign a solution
to a metavariable. Errors if already solved." Monotonicity is ENFORCED. So
extending an already-solved row cannot work by re-solving its meta; the only
route is for the row to have been solved to something containing a fresh tail
meta, and extension means solving that. Which is a row variable. That is what
makes the 29 read sites load-bearing — each pattern-matches a symbol and would
meet a meta.

That distinction is worth having explicitly next to the entry, because three
times today an entry I had filed as needing a ruling turned out to need only a
look at the code. This one is the opposite shape: the obstacle is a checked
invariant, not an unexamined assumption of mine.

Also re-counted rather than trusting the earlier figures: 29 expr-Record-tail
sites, not 28. Small drift, but the entry's whole purpose is that the size is
known before someone starts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Tested the fifth of my own claims this stretch. Recursive session TYPES are live
in the standard library and exercised: io-protocols.prologos declares FileRead,
FileWrite, FileAppend and FileRW with `rec` and `-> rec` back-references, and
test-io-session-01 (11 cases) plus test-io-dep-session-02 (8, one asserting
sess-mu? directly) pass. They work because the IO path unfolds —
session-runtime.rkt in four places, io-bridge.rkt, effect-position.rkt.

What does not unfold is type-proc's channel lookups, the process typing
judgement. So the gap is process recursion plus type-proc adopting
unfold-session — not the session-type layer, which ships and is used.

The entry's body already said this correctly: "Typing simply never adopted
them." The overstatement was mine, in conversational summary, and it is the kind
that makes a live working feature look unbuilt — the same failure mode as the
five "blocked" entries found at the start of this session that named an obstacle
already built. I spent the session correcting that pattern in the file and then
reproduced it in how I described the file.

The gap itself stands, and its shape is unchanged: doing either half alone
produces no implementable recursive protocol, only a differently worded refusal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
STATE head to 89b1d3c / 561 / 10965, and the conformance line now says 24/24 at
UPSTREAM HEAD 31f0b80 rather than at a frozen pin — the pin moved when netstring
framing landed, so the number means something different than it did this
morning.

The LOG entry keeps three things I would otherwise round off.

The pattern-compiler bug's near-miss: 165 of 166 bridge cases passed, the full
suite passed, the conformance gate passed, and exactly one assertion in the tree
distinguishes the two — one that exists only because brand-check happens to need
it. That is the argument for the new dedicated test file.

That I published two wrong root causes before the right one, including a
recommendation to require brackets and error otherwise, which would have imposed
a syntax restriction on users to work around a one-line ordering bug.

And two new Watching items. A confounded experiment that reaches the RIGHT answer
is more dangerous than one that reaches the wrong answer, because nothing prompts
a re-run — and I made that error twice in one session, the second time after
writing the first one down. And: I escalate while the mechanism is still unclear,
and the unclarity presents itself as a design question; four of five "not mine to
close" claims needed correction when tested.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…m cache

Zero sess-* nodes were registered in pnet-serialize.rkt, so any module declaring
session types deserialized as raw VECTORS from cache. io-protocols.prologos
declares four recursive protocols, so the standard library was broken for anyone
with a warm .pnet cache and fine for anyone cold.

The diagnostic is the one pipeline.md warns about, and it reads as nonsense
until you know it: "sess-mu-body: contract violation; expected sess-mu?; given
'#(struct:sess-mu …)" — a value being told it is not the thing it visibly is.
The leading quote is the tell. That is a vector, and the unknown-tag fallback
returns one that prints like the real struct and fails the first predicate to
touch it, arbitrarily far from the cause.

Verified in both states rather than assumed: with the pre-existing cache present
the test failed, deleting io-protocols.pnet made it pass, and after registering,
a cache written by the fixed code reads back clean — test-io-session-01 11/11
warm, test-io-dep-session-02 8/8.

The whole family is registered, not just sess-mu: registering only the node that
detonated is the registration-by-detonation pattern this same file was swept for
earlier today, when 31 nodes were found missing by sibling.

How it survived is the part worth keeping. Nothing about this is triggered by a
code change, only by cache state, so it passed every cold-cache run and failed
every warm one and neither looks like a regression. It surfaced because a
full-suite gate ran for a four-line test addition right after a raco make had
changed the cache state — the gate I nearly skipped because the change was
trivial.

It also means an earlier probe today was luckier than I knew: I ran
test-io-session-01 while checking whether recursive session types work, got
11/11, and cited it as evidence they ship and are exercised. True, but the cache
was cold. A green result whose correctness depends on invisible cache state is
not the evidence it appears to be.

Also included: four family-shape tests for the multi-arity fix (three columns,
nullary mixed with field-carrying, field-carrying first, nullary nested). All
four were already correct — a negative result, recorded because pipeline.md
names unswept siblings as a red flag and because an unrecorded probe gets
re-derived.

Full suite in flight; result to follow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Suite 561 / 10969 / 0 with the fix in. STATE head and HANDOFF to 2413462.

The LOG entry keeps the part that generalises past this bug. Nothing about it is
triggered by a code change, only by cache state, so it passed every cold-cache
run and failed every warm one and neither reads as a regression. It surfaced
because a full-suite gate ran for a four-line test addition immediately after a
raco make had changed the cache state.

New Watching item: a green result whose correctness depends on invisible cache
state is not evidence. When a probe's answer could differ between a warm and a
cold .pnet, say which one you ran, or run both. That is the third distinct way
this session that a green signal meant less than it appeared to — the others
being a law obligation documented in three files and enforced in none, and a bug
whose only witness was one accidental assertion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
101 upstream commits. Merged rather than rebased: a rebase of 427 commits needs
a force-push, which discards PR #28's review history and CI record, and this
branch has taken upstream by merge four times already.

Six conflicts. Three were the same problem solved twice, and none resolved by
picking a side.

macros.rkt — main independently adopted this branch's $preparse-error marker AND
added srcloc origin-indexing. Took main's indexing; kept this branch's
contain-at-seam? discriminator, because main's handler catches every exn:fail?
and would swallow module-load errors, turning a broken import into a per-command
marker at the first form.

typing-errors.rkt — main added broadcast-aware wording ("each element", "that
component") where "the subject" is wrong under a broadcast. Kept this branch's
two specific cases ahead of it, but gated the type-naming hint on (not bcast):
inside a broadcast it names the carrier when the element is at fault. Three of
main's test-path-selection cases caught that.

driver.rkt — I got this one wrong first. Both sides fixed the same symptom, and
I composed them as (def-body-type-ok ... (zonk ...)), reasoning that zonking
first would let genuinely-ground types be CHECKED rather than skipped. That is
strictly stricter than main and broke three of its tests: the def seam started
preempting the guiding diagnostics DEFERRED 74 exists to let through. Main's
not-ground guard already subsumes the zonk. Reverted to main's, with the
reasoning kept at the site — composing two fixes for one symptom is only right
when neither subsumes the other, and I did not check.

Two of main's tests then failed for the right reason. They pin `unknown` as a
known representation gap, and their own comment says "pinned so that closing the
representation gap flags here". This branch closed it in 10f5a08. Updated to
assert the value round-trips, and SPLIT rather than deleted the second: the type
half is still open ([PVec {:q <error>}]), so it stays a live signal.

Also adopted main's NUMBERING ruling into workflow.md while keeping this
branch's bench-ab text — main's says --ref does not exist, which was true when
written and is not now, because this branch built --refs in 6e38d21.

Verified: clean build, and every conflict-surface test green (rel-t1-pol,
path-selection, error-messages, hof-def-seam, io-session-01, merge-laws,
multiarity-nullary-dispatch). The full 561-file suite is still running; pushing
ahead of it deliberately, because this container has rewound the working tree
four times today and only a push survives that. Result reported either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Reporting it because I said I would either way. 564 files / 11161 tests, one
failure — test-goal-arg-typing. Fixed here; full suite re-running.

Same class as the two pins already updated in the merge commit, and this test
predicted itself: "If a future change makes `=` evaluate, BOTH of these move
together — and this pin is what will say so." It fired, but NOT for the reason
it named, and that distinction is why the pin was worth keeping.

`=` still does not evaluate — the RHS renders as the TERM [+ 1 1], not as 2.
What changed is representation: 10f5a08 let compound terms survive the
AST<->solver round trip, so they render themselves instead of collapsing to
`unknown`. The test's actual premise, that a well-typed and an ill-typed
compound render identically, SURVIVES: both now show their terms where both
showed `unknown`. So expr-unify-goal stays correctly out of scope for DEFERRED
52.

I got the fix wrong on the first attempt. I asserted the ill-typed side still
showed `unknown`, inferring it from the suite reporting only ONE failure in that
test-case. That inference is invalid: rackunit's first failing check-true aborts
the rest of its test-case, so the second assertion never ran and its silence was
not a pass. Measured instead: both round-trip. The wrong reasoning is recorded
at the site, because the trap is reusable — absence of a second failure in a
test-case says nothing about the second assertion.

Third time today I reasoned from an absent signal rather than a measured one,
after the confounded bracketing probe and the coverage set-difference. Same
shape each time: the cheap inference was available and the measurement was ten
seconds away.

The pin now asserts non-evaluation EXPLICITLY (`ok` must not contain ":x 2"), so
a change that genuinely makes `=` evaluate still fails here instead of sliding
through on a rendering change. That is what the pin was for, and the rendering
change would otherwise have quietly consumed it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The re-run after the `=` pin fix passes clean. STATE head and HANDOFF to
a32ed6f, plus a LOG entry for the merge.

The entry keeps the three things I would otherwise round off: that main's
not-ground guard subsumed my zonk rather than composing with it; that three of
main's pins fired on this branch's round-trip fix and each said in its own
comment that it existed to flag exactly that; and that the `=` pin fired for the
wrong reason, which is why asserting non-evaluation explicitly matters — a
rendering change would otherwise have quietly consumed the pin.

New Watching item, and it is the session's most repeated error: three times I
reasoned from an absent signal instead of a measured one. A confounded probe
that landed on the right answer, a coverage figure computed as a subtraction of
two set sizes, and an assertion I called passing because no second failure was
reported — rackunit aborts a test-case at the first failing check, so the
silence meant it never ran. Absence of a signal is not a measurement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The rule two peers must evaluate identically, with no round trip, was an
inlined `(bytes<? our-side-id theirs)` on the interop server's accept thread:
expressed in a language neither peer's implementation is written in, and
untested. It is now `crossed-hellos-abort-ours?` in
`prologos::ocapn::handshake`, with the socket teardown left where it belongs.

Side-ids cross the FFI as hex, and the Prologos side DECODES rather than
comparing hex. Fixed-width lowercase hex is order-preserving, so comparing it
would give the same answer — but only while both ids keep the same length and
case, which nothing checks. Perturbation confirms the distinction is real and
narrow: dropping the decode fails exactly one assertion (the mixed-case one)
and nothing else.

No local `bytes<?` fallback was kept. One would mask a break in the migrated
path behind an answer that happens to agree — the failure mode that made this
worth moving.

Tests (5, all perturbation-verified): antisymmetry over a battery whose ids
differ at varying offsets and across the signed-byte boundary; agreement with
octet order; the equal-ids degenerate case; and the mixed-case hex case.
Antisymmetry deliberately does NOT fire when the comparison is reversed —
reversal preserves it — which is why the octet-order test exists alongside.

End-to-end: both arms ran against the real peer during the conformance suite
(one OUTGOING abort, one incoming), gate 24/24, zero verdict errors.

Also corrects the file header, which claimed both handoff roles and the
crossed-hellos rule still lived there and that the frame loop routed handoff
traffic. Neither had been true since the handoff migration; the Syrup reader
it describes now runs as a differential check that decides nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The interop server assembled `<op:abort "reason">` itself, on the reasoning
that it "runs on the accept thread before any connection state exists". That
was never the obstacle it sounded like — `validate-incoming` calls into
Prologos from that same thread, and `abort-bytes` is pure and needs no
connection state. So the server carried a second Syrup encoder for a frame the
language already knows how to build.

Reasons are compile-time constants, so all of them are built once at startup
and looked up thereafter. That is not an optimisation: it is what lets the
validation-failure path use the single implementation without calling into
Prologos from a handler for a Prologos call that just failed. A miss cannot
happen for a literal, and falls back to a live call rather than a second
encoder.

The retired builder is transcribed into the test file as an ORACLE, which is
the difference between retiring a duplicate and keeping one — an oracle can
only report a divergence, where a production fallback supplies an answer that
hides one. Both encoders agreeing would still prove nothing if both were
wrong, so one case is also pinned against hand-written bytes.

Gate re-run: 24/24, crossed-hellos path exercised both arms again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
A CapTP side-id is computed in two languages: `(sha256-bytes (sha256-bytes
pk-bytes))` in the interop server, feeding the crossed-hellos tie-break, and
`side-id-of-encoded-key` in prologos::ocapn::crypto, feeding handoff
receive-binding and the session id. Nothing forced them to agree, and it is
the SAME identity — divergence would have the tie-break and the handoff
machinery disagree about who the peer is, with no error anywhere.

They do agree. This is the test that says so rather than a comment asserting
it, which is what both sites had.

The hash primitive is shared (`sha256-bytes` is racket/base, reached from
Prologos through `crypto-sha256`), so what is under test is the COMPOSITION
and the Latin-1 round trip either side of the FFI. That round trip is where
this codebase's encoding bugs live — a Latin-1 byte string measured as UTF-8
changes silently above 0x7F, and a digest is half high-bit bytes. The battery
covers every byte value 0-255, the 0x7F/0x80 boundary, and the gcrypt
public-key shape the protocol actually hashes.

Also pins the DOUBLING separately: the two implementations are transcriptions
of each other, so both dropping a round would still agree.

Perturbation-verified — collapsing sha256d to one round fails the agreement
and doubling assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
§0 is preserved as-written by the document's own convention, so this corrects
the status notes rather than the entries.

0.1 CLOSED — the drift check enumerates methods AND whole test-bearing classes
across all six targeted modules, prints drift up front, and exits non-zero at
the end so the run that finds it still yields a conformance signal. Zero drift.
Records the counting trap too: upstream greps as 25 tests, but one is commented
out, so the 24 we run are every ENABLED test rather than a subset.

0.3 CLOSED, and a comment that outlived the fix is corrected: interop-driver
still said 5N "gets no actor" while `seeded-vat` twelve lines below seeds
`beh-sturdyref-enlivener` there. A reader trusting it would have re-derived the
finding that was already fixed.

0.2 LARGELY CLOSED, and two thirds of its predicted fix shape were wrong —
worth recording rather than quietly closing:

  * `eff-connect` landed, and turned out to be what let 0.3 close.
  * `eff-send-on` was built, shipped with zero producers, and was removed. The
    handoff gifter is not a behaviour: no swiss-num, answers no message, must
    name connections.
  * `eff-sign` does not exist and is not needed — the signer is the driver, and
    an effect is how a BEHAVIOUR asks the world for something.

What remains is a different gap than the one 0.2 names, so it is named rather
than folded in: the migrated roles are expressed in the language, but the state
they operate on is seven Racket mutable hash tables reached over FFI. The root
is structural — Prologos cannot express state that outlives one top-level call
and is shared across connections — so FFI is the honest place for it today.
Recorded as a gap rather than as an implementation detail of seven modules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
STATE head refreshed (HEAD e9ff2d4, suite 11171/566/0, conformance 24/24,
fourth rewind) plus a LOG entry covering the four commits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
I wrote that "Prologos has no way to express state that outlives one top-level
call" into the gaps document and the dailies. It is false, and it was challenged
the same day with the obvious functional question: don't you just fold the
actions over the previous state?

You do, and the code already half does it. `ConnectionState` is an ordinary ADT
and `run-step : Nat -> CapTPOp -> ConnectionState -> String` TAKES the state as
a value and builds the next one. Only the return half is missing — the new state
exits sideways through `conn-stash`, which is what `emit-after-stash` is named
for.

What actually keeps the fold open, in ascending order of interest:

  1. the accept loop is in Racket, so Racket owns the accumulator — no language
     obstacle, just no Prologos-side loop to hang it on;
  2. the FFI channel is `process-string` -> printed text, so a (state, bytes)
     pair cannot cross without serializing the whole vat per frame — practical,
     not a language limit;
  3. cross-connection sharing is genuinely not one fold: a give arrives on the
     gifter's connection and is spent on the exporter's, so a per-connection
     accumulator cannot hold it.

Which is why the fix shape said CELL rather than "thread the state" — a cell
with a monotone merge is the answer to several concurrent folds that must share,
and the gift table is near-textbook since deposits only accumulate.

So the gap is narrower than published: four SHARED tables (gift, give, enliven,
peer) belong in cells; three (conn, dial, identity) merely wait for the loop to
move.

The wrong paragraph is left standing with the correction after it, per the
document's convention of not quietly editing findings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Both challenged, both wrong, both mine.

1. "The gift table is near-textbook monotone, deposits only accumulate." They
   do not. A withdrawal REMOVES the gift (bs-remove-gift, captp-core:2316 and
   :2393) and must, or the same gift is spendable twice. The live-gift table is
   not a join-semilattice as it stands.

2. "Seven FFI state modules." There are eight — I missed ocapn-handoff-ffi, the
   replay set, which is ironically the ONLY one that is genuinely monotone
   (test-and-set on a grow-only set). So the split is 3 fold-artifacts + 5
   shared, not 3 + 4.

Both errors generalise from the modules I had read to the set I had not.

Consequence for the design: the cell is not free. Two principled routes, both
already in the project's vocabulary — a 2P-Set (grow-only adds + grow-only
removes, live derived as adds \ removes; the codebase is half-way there, since
gk-used was already moved out of the gift table into a grow-only replay set), or
S(-1) retraction-as-lattice-narrowing.

And a trap worth recording before anyone builds it: a tombstone keyed on the
gift-id alone is peer-choosable, since gift ids are opaque bytes the peer
supplies. That recreates the `used:`-prefix poisoning exploit this codebase
already fixed once. Any tombstone key must include something the peer cannot
choose, exactly as the replay identity now length-prefixes its parts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Groundwork for moving the event loop into Prologos. The existing TCP FFI is
LF-delimited, which is fine for the tcp-testing-only netlayer and wrong for
OCapN: a Syrup frame is arbitrary bytes and 0x0a occurs inside them constantly.

Adds tcp-send-frame / tcp-recv-frame-ret / tcp-recv-frame-cached /
tcp-recv-frame-eof? on the SAME Nat-handle table the line ops use.

Two decisions worth recording:

* The framing is NOT reimplemented here — it requires ocapn-framing.rkt, the
  same module the interop server uses, so there is one framing implementation
  rather than two that agree until they don't. The require reaches into tools/
  from the library, which is backwards; tests/test-ocapn-syrup-wire.rkt already
  does the same. Moving the module into the library belongs with the driver
  migration, and is noted at the site rather than silently tolerated.

* EOF is recorded SEPARATELY from the payload rather than as an empty frame.
  Both carry zero bytes, and a loop that cannot tell them apart either spins
  forever on a dead peer or drops a legitimate empty frame. An unread handle
  defaults to EOF, because that is the safe answer.

Not Opaque:*, deliberately. That path looks supported and is a partially-wired
dead end: expr-opaque has ZERO hits in pnet-serialize.rkt (so a cached module
body carrying one deserializes as a raw vector — the documented trap),
io-ffi-registry has no consumers, and io-ffi.rkt's own comment records that the
WS reader parses the colon in `Opaque:file-port` as a keyword. Nat handles are
the proven path, already used by the port table and the OCapN keypair handle.

Tests (9, perturbation-verified): switching the strategy to 'newline fails
exactly the two byte-transparency cases — a payload containing 0x0a, and the
full 0-255 range — and nothing else. Also covers empty-vs-EOF on one handle in
sequence, frame ordering, the unread-handle default, and cache cleanup on close.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Reduction is lazy in argument position, so effect order is not something the
language gives you. The existing answer is to force an effect by MATCHING on
its result, and emit-after-stash documents that idiom's fragility at its own
site: its two arms are deliberately different because identical arms could be
folded away by "any arm-collapsing rewrite — and folding it away DELETES the
stash". An effect that survives because an optimiser did not notice it is not
ordered, it is unexamined.

prologos::core::world adds a World token; prologos::io::net threads it through
frame IO. An action returns [Pi [w :1 World] result] — an effect as a VALUE,
applied to a world. The only way to perform the second effect is to hold what
the first produced.

Three things measured rather than assumed, each of which I had wrong first:

* The World carries a real tick from a global clock, not a nullary token. A
  token with one inhabitant is a value the reducer can produce WITHOUT running
  the effect, which makes the dependency decorative. Same reason the FFI's
  payload/EOF readers take the tick the read produced: they ignore its value
  and need its argument to be unobtainable beforehand.

* `spec` CAN spell :1, via an angle-bracket Pi — my first read of this was
  wrong and the module carried a comment saying otherwise. The spec is also
  required: a defn whose body is a `fn` will not infer from annotated
  parameters.

* The :1 survives if EITHER the spec's binder or the body's `fn` binder
  declares it, and a DISAGREEMENT between them is accepted silently, resolving
  to the restrictive side. So a spec is not the authority on its own
  function's multiplicity — worth knowing before reading one as if it were.
  Found by perturbing each site in turn; the first two perturbations passed,
  which is what exposed it.

QTT enforces both halves with diagnostics that already existed: using a World
twice is "used more than once", dropping it is "must be consumed; nothing may
drop it". `match` counts as exactly one use, which is what lets every action
consume its World by matching.

Tests (8). The known gap is pinned as ACCEPTED on purpose: Recv holds a World
in an ordinary field and data fields carry no multiplicity, so it can be
extracted twice. Closing that needs linear fields, which Phase 0 lacks. If the
test starts failing, the gap closed and the docs need updating.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
`prologos::io::server` binds a port, accepts a peer, echoes frames until it
closes, and shuts down — with the loop in the LANGUAGE. No Racket `let loop`,
no accumulator on the host. That is the step the OCapN driver needs: while the
loop lives in Racket, Racket owns the state, which is why per-connection state
sits in an FFI hash table instead of being threaded like an ordinary value.

The first version hung, and the reason is the most useful thing here.

THE REDUCER IS CALL-BY-NAME. It does not share an evaluated redex, so a `match`
binding two fields of an effect's result re-evaluates that effect once per
field used. Instrumenting the FFI showed it directly:

    FFI accept called (tick 1)
    client: connected
    FFI accept called (tick 1)     <- same tick, second real accept

The second accept blocked forever on a listener whose only client had already
been taken. Three attempts to reason it out from the code were wrong; the
stack trace naming tcp-frame-accept plus a printf found it in one run.

So the World token fixes ORDER and does NOT make an effect at-most-once —
nothing stops the reducer walking the same subterm twice. Effects are now
MEMOIZED on (op, tick, args): an effect is a function of its tick, so asking
twice what happened at a point in history returns what happened rather than
making it happen again. This is why the scheme threads a tick and not a unit
token — a token with no identity gives the memo nothing to key on.

Payload and EOF are keyed by the tick the READ produced, not by handle. Keyed
by handle they would report whatever the socket saw most recently, so
re-walking an older subterm would observe a newer frame — a wrong answer
rather than a hang, which is worse.

Also recorded at the site: module-level definitions are not
forward-referenceable, so the two-deep `match` nesting is inlined rather than
factored into helpers (mutual recursion fails at import with an unbound
variable naming neither function). And the memo grows with the number of
effects — fine for a test server, a real leak for a long-lived one; bounding it
needs a "no reduction can still reach tick N" notion the reducer does not
expose.

Tests: 6 for the loop (multi-frame, 0x0a payloads, all 256 byte values, an
empty frame echoed rather than treated as hangup, termination on peer close),
plus 3 pinning the memo invariant, which nothing else would catch — without it
the symptoms are a hang or a duplicate frame, arbitrarily far from the cause.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
`serve-n port n` accepts n peers and echoes for all of them from a SINGLE loop
threading one `SrvState`. No Racket threads on the server side, no
per-connection table.

That is the observation the OCapN migration turns on. The five shared FFI hash
tables holding handoff state exist because the Racket server gives each
connection its own THREAD, so per-connection state cannot be an accumulator —
it has to live somewhere both threads reach. One loop over a multiplexed handle
set removes the reason: state becomes fields of a value.

Sequential by construction, as agreed: `net-sync` returns one ready handle and
it is serviced before anything else runs. Parallel service is the on-network
question and is not answered here.

Two obstacles, both found by running it rather than by reading:

FUEL. The loop died with "fuel exhausted". The per-command budget is 1M steps,
hardcoded — right for a REPL command, where a million steps means divergence,
and wrong for a PROGRAM. interop-driver's header already recorded the flip side:
it runs "one process-string per frame, so each frame gets a fresh
reduction-fuel budget". Moving the loop in means one long reduction instead of
many short ones, so the budget moves with it: `current-reduction-fuel-budget`,
defaulting to the old value, #f for none. Raising it does not make a divergent
program terminate — it trades a bounded wrong answer for an unbounded one — so
#f belongs to callers with their own way to stop.

A FROZEN CLOCK. With fuel raised the loop spun forever, replaying one cached
event after the peer had disconnected:

    FFI sync tick=6 handles=(1)     (repeating)

`tcp-tick-after` was memoized on its `dep` argument alone. That looks right —
a handle is unique per accept — and it deadlocks: `dep` is the handle `sync`
returned, the same connection is ready every iteration, so iteration two hit
the memo and got iteration one's tick. The clock stopped, the next sync ran at
a tick already answered, and the loop replayed forever. Key is now
(previous-tick, dep).

The rule this teaches, and the reason it is now a test: a memo key must
identify a POINT IN TIME, and any component that recurs cannot supply that on
its own.

Tests: 4 for the multiplexed server, including a rendezvous — peer A holds its
connection open until peer B has been served — because a server that ran each
peer to completion passes a naive N-client test, just slower. The rendezvous
turns "serial" from slow into deadlock, which a timeout reports. Plus one
pinning the memo-key invariant, whose symptom is otherwise a hang with no error.

`current-reduction-fuel-budget` is registered in test-support.rkt per the
new-parameter checklist, so a file raising it for a server loop cannot leak the
raised value into the next file's commands. Parameter lint: 0 new unclassified.

Gates: full suite 11202 / 570 / 0; OCapN conformance 24/24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants