Skip to content

test(platform): engine conformance legs — OCC, shape-poke ordering, RLS identity - #218

Merged
prisis merged 9 commits into
feat/platform-abstraction-layerfrom
feat/engine-conformance-legs
Jul 28, 2026
Merged

prisis merged 9 commits into
feat/platform-abstraction-layerfrom
feat/engine-conformance-legs

Conversation

@prisis

@prisis prisis commented Jul 28, 2026 •

Copy link
Copy Markdown
Member

Completes plan 114 §5.4's engine legs, then closes the §5.3/§5.5 gaps an audit of the remaining workstreams turned up. Stacked on #190 — review that first.

@lunora/platform/conformance asserts what a host provides. This adds the other half: what the engine does with those primitives. The split is forced, not chosen — the engine legs need createShardCtxDb/relay-hub, which live in @lunora/shard-engine, and @lunora/platform is zero-dependency by contract, so importing the engine from there would invert the dependency and cycle. A host is proven only when it passes both.

Ten legs, run against both the platform reference host and real workerd through createShardPlatform:

Group What it pins
OCC (4) zero-changes CAS → CONFLICT/kind:"occ", no server retry, no partial write; the trigger-recursion ceiling as a distinct kind
Shape-poke (3) pokeStart → pokePart → pokeEnd under one poke id; delivery gated on the per-socket cohort memo; epoch disambiguation
RLS (3) the seed resolves under the subscriber's identity; identity-scoped shapes get per-socket pokes, never the cohort multicast

Every leg is mutation-verified — 11 mutations, each caught by its own leg. The two RLS gate mutations are opposite-direction (gate forced true and forced false, both caught), so neither leg passes on a gate that simply refuses everything. The two most load-bearing were re-verified through workerd specifically, not just the reference run.

Findings worth reviewer attention

The first OCC leg I wrote was wrong, and passing was how it hid. Provoking the conflict through ctx.db re-enters the trigger and trips the recursion ceiling before the CAS — so it asserted kind: "trigger" while the test name claimed occ. The clobber now goes through raw SQL (which doesn't re-fire triggers), and the recursion ceiling gets its own leg. The two kinds must stay distinct because the client's correct response differs: refetch-and-retry for one, and an infinite loop for the other.

ConformanceHost had no way to read back what a socket was sent. So no delivery guarantee was assertable at all — and the platform TCK's own send leg was asserting only that send didn't throw. A host that silently dropped every frame would have passed it while breaking every subscription. Added readFrames and strengthened that leg.

Related, and only visible once something read the buffer: ReferenceSocket.received is typed (string | ArrayBuffer)[], but the send path encoded unconditionally — the string arm was unreachable. Write-only state doesn't get type-checked by use.

I asserted the owner seed path could not reach RelayHost.sql, and was wrong. The owner persists its relay set in __lunora_relays. It now runs on the host under test rather than a double, because a host whose SQL can't carry that set drops its relays on every wake and silently stops fanning out. The stub earned its keep by being wrong loudly.

The real transport forced two changes the in-memory host had been hiding. readFrames is now awaitable — a send reaches its peer through an event, so only an in-memory host can answer synchronously, and reading eagerly on workerd reports an empty buffer for a frame that did leave the socket. And the client end of a WebSocketPair must be accepted after the host takes the server end; accepting it first marks the pair as used and state.acceptWebSocket refuses. That was the first workerd run: three poke legs failing while the other seven already passed.

Scheduler at-least-once + dead-letter — the contract had to grow

§5.4 lists this leg; it started out unassertable. SchedulerHost was {schedule, cancel, cron?} — enqueue and cancel only — so at-least-once was promised in the module docstring and stated nowhere a test could reach.

It now carries two optional members, both describing RPCs SchedulerDO already serves (/list, /dead, POST /dead/retry) rather than an invented surface, so there is a real Cloudflare implementation behind the contract and not only a reference one:

  • list → pending jobs as ScheduledJobStatus, which is where attempts lives. That field is what makes at-least-once observable: without it a caller cannot tell a job waiting between retries from one never dispatched.
  • deadLetter.list / deadLetter.requeue. Its absence is a real statement — a host without it can promise at-most-once only, because "dropped" and "delivered" look identical to every caller.

Scheduler gained dead/deadRetry to expose them. Six legs in total: attempts: 0 on a fresh job; the two listings disjoint in both directions (park and requeue); requeue restores a fresh budget; requeue of an unparked id is false; cancel answers truthfully on a second call; two identical schedules get independently cancellable ids.

ConformanceHost.simulateDeadLetter drives the transition, following simulateRecycle's convention. The retry policy is deliberately not in the contract — how many failures and what backoff before parking is host policy; only the observable outcome is contract-level. So §7 open question 2 (extract SchedulerDO host-neutrally) stays open, but is now an optimization rather than a prerequisite.

One of these legs was wrong until mutation testing said so: the requeue leg checked that a job returned to pending but never that it stopped being parked, so leaving the dead record behind survived. Fixed in its own commit.

Plan 114 §5.3 / §5.5 — the target selector was unreachable

An audit of the remaining workstreams found one gap wearing two hats. resolveDeployDriver(target) and runCodegen({ target }) both accepted a target; nothing supplied one, so options.target was always undefined and only the default was ever selectable. The machinery was complete and unreachable.

  • lunora.json gains a target key beside remote.
  • --target is declared on codegen, dev, prepare, deploy, and logs, with help text generated from the driver registry rather than a hand-kept string.
  • resolveProjectTarget(projectRoot, explicit) is the single resolution point — flag, then config, then default. One point on purpose: codegen tailors the emitted ctx.* surface to a target while deploy picks the driver that ships it, and resolving those separately lets them disagree, producing an app that builds cleanly and fails at runtime. The Vite plugin resolves the same way, so vite build and lunora deploy agree without configuring each.

Wiring it up surfaced two things that would have left the flag decorative:

  • The flag parsed and appeared in --help, but no CreateOptions map declared it and no execute wrapper forwarded it — the same declared-but-unreachable shape as the original finding, one layer down. Caught by running the built binary, not by reading the types.
  • codegen --target aws emitted the full Cloudflare surface un-gated, reported platform_unknown_target as a warning, and exited 0. That is the silent fallback the driver registry was built to prevent, arriving through the back door: codegen resolves no driver of its own, so nothing rejected the name. All five commands now go through resolveTargetOrThrow, so a typo fails identically everywhere — including one committed to lunora.json, which is where it would go unnoticed longest.

Default behaviour is unchanged: goldens byte-identical, 921 codegen tests green. Five mutations verified against the resolution order, the config-degradation path, and the validation.

(Unrelated: examples/blog/lunora/_generated/server.ts is stale — last regenerated in #202, one commit before vector emission landed in #203. Left alone rather than folded into this PR.)

Verification

@lunora/do 687 + 62 workerd (incl. the 10 engine legs through the composition root)
@lunora/shard-engine 638
@lunora/platform 36
@lunora/scheduler 113
@lunora/cli 792
@lunora/config 491
@lunora/codegen 921 (goldens byte-identical)
@lunora/vite 186
repo lint:types, lint:eslint, lint:prettier, lint:package-json, api:check

API snapshots updated for the new @lunora/shard-engine/conformance subpath, ConformanceHost.readFrames, the SchedulerHost delivery members, and the @lunora/config target helpers.

🤖 Generated with Claude Code

https://claude.ai/code/session_0137KKKmGRyB69XgLzMaFgSh

prisis added 6 commits July 28, 2026 08:05
…engine TCK

Adds plan 114 §5.4's poke leg: pokeStart → pokePart → pokeEnd under one poke
id, delivery gated on the per-socket cohort memo, and the resume case where a
re-delivered poke must not double-apply.

The seam is `RelayHost`, not `buildPokeFrames` — the framing is pure and
host-invariant, so asserting it per-host proves nothing. What varies is the
memo, which is keyed on the handle the host issues at accept; a host whose
handles aren't identity-stable misses every lookup silently.

Two gaps surfaced while wiring it up, both in the platform reference host:

- No way to read back what a socket was sent, so no delivery guarantee could
  be asserted at all. Added `ConformanceHost.readFrames` (optional — a host
  that can't observe its own outbound traffic is only partially proven) and
  used it to strengthen the platform TCK's send leg, which until now asserted
  only that `send` did not throw.
- `ReferenceSocket.received` is typed `(string | ArrayBuffer)[]` but the send
  path encoded unconditionally, making the string arm unreachable. Unnoticed
  because nothing read the buffer back.

Also corrects the OCC leg added in the previous commit: provoking the conflict
through `ctx.db` re-enters the trigger and trips the recursion ceiling first,
so it was asserting `kind: "trigger"` while claiming `occ`. The clobber now
goes through raw SQL, and the recursion ceiling gets its own leg — the two
kinds must stay distinct because refetch-and-retry is right for one and an
infinite loop for the other.

Six mutations verified: cursor check, epoch check, memo advance, frame order,
per-frame poke id, and the OCC zero-changes branch each fail a distinct leg.
…ngine TCK

Plan 114 §5.4's third engine leg — the `cb632cd7` guarantee that a live
subscription keeps evaluating under the socket's own verified identity.

The relay tier is where that is easiest to lose: a cohort multicast computes
one delta under the ANONYMOUS identity and ships it to every subscriber, which
is correct only while the shape is identity-blind. A shape whose `where` reads
a claim must get a per-socket proxy poke instead. Misclassifying one as uniform
multicasts one tenant's rows onto another tenant's socket, with no error
anywhere — so the three legs pin the seed identity, the scoped-shape routing,
and the identity-blind case that keeps the first two from passing on a gate
that simply refuses everything.

The owner's `sql` runs on the host under test rather than a double: it persists
the relay set in `__lunora_relays`, and a host whose SQL can't carry that set
drops its relays on every wake and silently stops fanning out. That surfaced
by asserting the seed path *couldn't* reach `RelayHost.sql` and being wrong.

Four mutations verified: the uniformity gate forced true and forced false (both
caught, so neither leg is vacuous), an anonymously-resolved seed, and a proxy
poke that loses its `targetConnectionId`.
Plan 114 §5.4's last engine leg was "scheduler at-least-once + dead-letter".
It is not expressible against the current contracts: `SchedulerHost` is
`{schedule, cancel, cron?}` — enqueue and cancel only, with no delivery
surface, attempt count, or dead-letter key. The retry/backoff/`dead:`
machinery lives in `SchedulerDO` (`@lunora/scheduler`), which is
Cloudflare-resident and not extracted.

So this adds the part that IS host-varying, and records the rest as blocked
rather than asserting a guarantee the contracts cannot carry:

- `cancel` must answer truthfully on a second call. A host returning `true`
  unconditionally lets a caller believe it stopped a job that is still
  pending, and a retry layer reading that answer stops retrying a delivery
  that never happened.
- Two identical schedules must get independently cancellable ids. A host
  keying jobs by payload silently drops the survivor: the caller enqueued
  twice, cancelled once, and is owed one delivery.

The plan now says which §5.4 legs are shipped, and that unblocking the
remainder means either growing `SchedulerHost` a delivery surface or
extracting `SchedulerDO` host-neutrally — which §7 open question 2 already
asks.

Both mutations verified: unconditional-success cancel and payload-keyed job
ids each fail their own leg.
…n workerd

Completes plan 114 §5.4's phase-1 gate: the engine TCK now runs on both hosts,
not just the reference one. Same 10 legs, a genuine `DurableObjectState` —
real SQLite, real hibernatable sockets, real `acceptWebSocket` tagging — built
through the composition root, so a wiring mistake in `createShardPlatform`
fails here rather than in production.

Two things the real transport forced, both of which the in-memory host had
been hiding:

- `readFrames` is now awaitable. A `send` reaches its peer through an event, so
  only an in-memory host can answer synchronously; reading eagerly on workerd
  reports an empty buffer for a frame that did leave the socket, which looks
  exactly like a delivery bug.
- The client end of a `WebSocketPair` must be accepted AFTER the host takes the
  server end. Accepting it first marks the pair as used and
  `state.acceptWebSocket` refuses — which is what the first run did, failing
  all three poke legs while the other seven already passed.

Verified load-bearing through workerd specifically, not just the reference run:
dropping the memo cursor check and forcing the RLS uniformity gate to `true`
each fail their leg here too.

API snapshots updated for the new `@lunora/shard-engine/conformance` subpath
and `ConformanceHost.readFrames`.
@coderabbitai

coderabbitai Bot commented Jul 28, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2)
  • main
  • alpha

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: be3a749c-cacd-4ed6-93bf-85e990835335

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/engine-conformance-legs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for following the naming conventions! 🙏

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for confirming the Contributor License Agreement! 🙏

prisis added 3 commits July 28, 2026 09:59
Plan 114 §5.4's scheduler leg was blocked because the contract had nowhere to
state the guarantee: `SchedulerHost` was `{schedule, cancel, cron?}` — no
delivery state, no dead-letter, so at-least-once was promised in the module
docstring and unassertable anywhere.

Adds three optional members, all describing RPCs `SchedulerDO` already serves
(`/list`, `/dead`, `POST /dead/retry`) rather than inventing a surface:

- `list` — pending jobs as `ScheduledJobStatus`, which carries `attempts`.
  That field is what makes at-least-once *observable*: without it a caller
  cannot tell a job waiting between retries from one never dispatched.
- `deadLetter.list` — jobs that exhausted their budget. Its absence is a real
  statement: a host without it cannot promise at-least-once, only at-most-once,
  because "dropped" and "delivered" look identical to every caller.
- `deadLetter.requeue` — return a parked job with a fresh budget.

The Cloudflare adapter wires straight through to those RPCs (`Scheduler` gains
`dead`/`deadRetry`), so the contract has a real implementation behind it and
not just a reference one.

Four TCK legs assert the invariants any host must uphold however it implements
retries: a pending job reports `attempts: 0`; the pending and dead-letter
listings are **disjoint** (a host reporting a parked job in both shows a
permanently-failed job as still on its way); requeue restores with a fresh
budget; requeue of an unparked id is `false`.

`ConformanceHost.simulateDeadLetter` drives the transition, following
`simulateRecycle`'s convention — the observable invariants are contract-level,
while how many failures it takes to get there is host policy the contract
deliberately does not fix.
Mutation testing found the gap: leaving the parked copy behind after a
requeue survived every leg. The requeue leg checked the job returned to
pending but never that it stopped being parked — so a host that recovered
into pending while keeping the dead record would break the same disjointness
the neighbouring leg pins, and be recoverable again into a second live job.
@prisis
prisis merged commit ba2fa4f into feat/platform-abstraction-layer Jul 28, 2026
6 checks passed
@prisis
prisis deleted the feat/engine-conformance-legs branch July 28, 2026 08:22
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using our GitHub Discussions tab for questions.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Sep 8, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant