diff --git a/proto/realtime/CHANGELOG.md b/proto/realtime/CHANGELOG.md new file mode 100644 index 0000000..9c2dc30 --- /dev/null +++ b/proto/realtime/CHANGELOG.md @@ -0,0 +1,150 @@ +# Changelog + +All notable changes to the Sitrep realtime protocol are documented here. +Versioning follows the compatibility policy in `SPEC.md` §15: the protocol's +own major/minor/patch version is independent of this repository's release +version. + +## 1.0.0 — initial frozen specification + +### Pre-freeze revisions (adversarial review, protocol owner rulings) + +Applied before the 1.0.0 freeze was published; the version stays 1.0.0. + +- Unified the viewer-facing reliable-event carrier as `delta`: `task.event` + and `message.event` are now uplink-only (source → server), and the server + re-emits every applied reliable event as a broadcast single-event + `delta{from, to, events}`, so every viewer-visible reliable event carries + its revision. Mandatory viewer connection sequence is now + hello → subscribe → resume, with explicit client rules for + `from_revision` <, ==, > the local revision. +- `resume` now always produces exactly one reply (never silence): + `last_revision: 0` or an unservable gap → `snapshot` (possibly empty); + `N == R > 0` → an explicit empty `delta{from: N, to: N, events: []}`. +- Added snapshot chunking (`part` from 1 + `final`, all chunks sharing one + `revision`; receiver applies only after the final chunk) and chained + catch-up deltas split on event boundaries; listed both in the framing + section's split mechanisms. +- `command` gained optional `task_id`/`automation_id` with a per-action + required-field matrix (pause/resume/stop → `task_id`; run_now → + `automation_id`), aligning with `POST /v2/tasks/:id/commands`. `params` + stays a reserved extension point with no v1 keys. +- Added the 14th message type `config.event`: a server-minted reliable + event for control-plane changes with no device_id/device_seq (its + identity is its revision slot), carried to viewers inside `delta`; + `snapshot` gained an `automations` array; authorization matrix and cost + table updated (persisted, revision++, no push). +- Envelope top level is now permanently strict: any unknown top-level + field is `malformed` and this rule can never be relaxed; the runtime + ignore-unknown-fields tolerance applies only inside `body`. +- Hardened the handshake: the client sends nothing between its offer and + the server's accept; pre-accept frames are answered with + `hello_required` and the connection closes; 10 s recommended offer and + accept timeouts; `hello_required` documented as server-only, with + retryable meaning retry-after-reconnect. +- Decoupled interest leases from connections: leases are per device, end + only by expiry or explicit `unsubscribe`, survive disconnects, and + support lazy expiry evaluation; `throttle`/`resume_rate` fire only on + the space lease count's 1↔0 edges; `interest.renew` wholesale-replaces + topics; `unsubscribe` is acked via `in_reply_to` like the other lease + operations; in v1 only the `metric` topic filters (reliable deltas are + always delivered to preserve revision continuity). +- Command relay rules: client-sent `origin: "server"` commands are always + rejected as unauthorized; the server preserves the original envelope + `ts` and `command_id` when relaying, validates TTL at relay time, and + the source re-validates against its local clock with ±30 s skew + allowance. +- The server now rejects any uplink event (`task.event`, `message.event`, + `metric.frame`) whose `body.device_id` does not match the connection's + authenticated identity. +- `snapshot.metrics` documented as a best-effort ephemeral cache (may be + empty or stale, outside revision accounting); snapshot is sent only as + a reply to `resume`. +- Added the deterministic event-folding section (task.event sequence → + task_state field by field; config.event → automation set; message + window truncation as normative behavior with the delta/snapshot history + asymmetry declared an accepted deviation). +- `ack.acked` defined as an exact enumeration scoped to the receiving + connection's device; `device_seq` scope pinned to (device, space). +- Added connection supersession: a device's new connection replaces its + old one, which is closed with the new error code `superseded`; acks and + targeted commands route to the newest connection; duplicate connections + do not double-count leases. +- `metric_sample` gained optional gauge/threshold geometry + (`target`/`min`/`max`/`alert_above`/`alert_below`); `internal_error` + messages must not leak implementation details; `message.event` now + composes the shared reliability `$def` like `task.event`. +- Fixtures: added config.event, empty delta, and `superseded` error + fixtures; a role-based invalid fixture (viewer sending an + `origin: "server"` command) checked by a new authorization layer in the + validator; scenarios updated to the hello → subscribe → resume order + with a live single-event delta; the revision-gap scenario now shows a + two-chunk snapshot; new `duplicate-connection-supersede` scenario. + +### Pre-freeze revisions, round 2 (Opus re-review rulings) + +- Connection-level delta gating: every connection (including a + superseding one) must complete hello → subscribe → resume before it is + delta-eligible; resume is now required per connection; the server sends + no live delta before that connection's resume reply (always the first + delta-family envelope on a connection); viewers defensively discard any + pre-reply delta, justified by in-order delivery rather than a causal + heuristic. Lease survival across supersession affects only + throttle-edge accounting (no spurious throttle/resume_rate) and never + confers delta eligibility; the local revision is always re-established + from the new connection's resume reply, with no cross-connection + inheritance. +- Chunked-snapshot server obligation: while chunks are in flight, all + other outbound envelopes on that connection (including acks) are + deferred until after the final chunk; only ping/pong may interleave, + and a viewer treats any other interleaved envelope as a malformed + sequence. +- Clients may only send hello{stage: offer}; a client-sent accept is + answered with hello_required and a close (matrix, validator rule, and + invalid fixture added). +- config.event minting and its revision increment are a single durable + transaction, giving duplicate-resistance equivalent to + (device_id, device_seq) dedup; HTTP control-plane retries cannot mint + duplicates. +- command field exclusivity enforced by schema: pause/resume/stop forbid + automation_id, run_now forbids task_id, throttle/resume_rate forbid + both (invalid fixtures added). +- Unified free-text caps in common $defs (free_text ≤ 2048 chars for + title/step/message/text, label_text ≤ 256 for labels/names), with the + framing section noting these caps make a single record exceeding a + frame impossible. +- Validator now asserts sequence invariants over scenario directories: + delta arithmetic, delta chaining, snapshot chunk-run integrity, and + per-device device_seq monotonicity — still zero dependencies beyond + the pinned ajv. + +### Initial content + +- Defines the complete v1 message set: `hello`, `resume`, `snapshot`, + `delta`, `ack`, `task.event`, `message.event`, `metric.frame`, + `config.event`, `subscribe`, `unsubscribe`, `interest.renew`, `command`, + `error`. +- Defines the generic envelope (`type`/`id`/`ts`/`body`) and the shared + `$defs` in `common.schema.json`. +- Defines reliable-event semantics: per-device `device_seq`, + `(device_id, device_seq)` deduplication, `ack`-driven resend queues. +- Defines `space_revision` accounting and the two resume flows (`delta` for + a small gap, `snapshot` for a large one or no prior state). +- Defines the interest lease (30-60 s, server-chosen duration) and the + `command{origin: server}` `throttle`/`resume_rate` notifications on the + space's 1↔0 active-lease transitions. +- Defines `command` for viewer-initiated `pause`/`resume`/`stop`/`run_now`, + with idempotency key, TTL, and role/origin validation. +- Defines version negotiation via `hello` offer/accept, the authentication + boundary (credential lives at the transport layer, never in an envelope), + and the per-message-type/per-role authorization matrix. +- Fixes all timestamps to integer Unix milliseconds, with a schema-level + minimum bound that rejects seconds-scale values. +- Defines the plain-text `ping`/`pong` heartbeat, decoupled from every other + piece of protocol state. +- Enumerates all 13 error codes with fixed `retryable`/`fatal` semantics. +- Publishes the server cost table (persisted / revision++ / outbox / + broadcast) per message type. +- Ships fixtures for every message type (valid and invalid) and five + end-to-end scenario sequences, plus a self-contained ajv-based validator + under `tools/`. diff --git a/proto/realtime/SPEC.md b/proto/realtime/SPEC.md new file mode 100644 index 0000000..e361423 --- /dev/null +++ b/proto/realtime/SPEC.md @@ -0,0 +1,1010 @@ +# Sitrep realtime protocol v1 + +Status: frozen for implementation. Version `1.0.0` — see `CHANGELOG.md`. + +## 0. Scope and relationship to other protocols + +This document specifies the **realtime synchronization protocol** between a +Sitrep source device (an Agent that executes and reports on tasks), a Sitrep +viewer device (a phone, tablet, or menu bar app watching a space), and a +server that relays and persists state for that space. + +This is a *different layer* from `proto/SPEC.md`, which describes the +line-oriented protocol a task process writes to stdout for consumption by the +local Agent. That layer is unaffected by this document. By the time a task +transition or metric sample reaches this protocol, the Agent has already +parsed it into a discrete domain event; this protocol is only concerned with +getting that event (or a metric sample, or a control instruction) from one +device to another, reliably or best-effort as appropriate, and keeping a +space's state convergent across every connected device. + +This protocol is **transport-independent**. It defines a message format and a +set of behavioral rules; it does not assume WebSockets, a particular cloud +provider, or any specific network topology. An implementation MAY carry this +protocol over a persistent bidirectional byte stream that supports discrete +text frames — a WebSocket, a raw TCP connection with framing, a WebRTC +DataChannel, or an equivalent. The rest of this document says "connection" +to mean whatever such channel the transport provides, and "frame" to mean one +discrete unit of data sent over it. + +## 1. Conformance language + +The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, +**SHOULD**, **SHOULD NOT**, **RECOMMENDED**, **MAY**, and **OPTIONAL** in this +document are to be interpreted as described in RFC 2119. Every implementation +(TS server, Go Agent, Swift app, and any future one) MUST satisfy every MUST +in this document; deviating from a SHOULD requires understanding and +accepting the consequences described here. + +## 2. Directory contents + +``` +proto/realtime/ +├── SPEC.md this document +├── common.schema.json shared $defs (timestamps, ids, task/metric/message shapes) +├── envelope.schema.json the generic envelope shape, common to every message type +├── messages/ one *.schema.json per message type, named after its `type` +├── fixtures/ +│ ├── valid/ one or more valid fixtures per message type +│ ├── invalid/ one fixture per constraint this spec calls out as rejectable +│ └── scenarios/ numbered multi-message flows with a README each +├── tools/ self-contained ajv-based fixture validator (see §16) +└── CHANGELOG.md +``` + +`common.schema.json` and `envelope.schema.json` are not sent on the wire by +themselves; every frame validates against exactly one file under `messages/`, +which itself composes `envelope.schema.json` via `$ref`. + +## 3. Envelope + +Every frame except the literal heartbeat text (§9.3) is a single UTF-8 JSON +object — an **envelope** — with exactly these top-level fields: + +| field | type | required | meaning | +|---|---|---|---| +| `type` | string | yes | one of the 14 message type names in §4 | +| `id` | string | yes | unique identifier for this transmission, assigned by the sender (RECOMMENDED: UUIDv4 or ULID) | +| `ts` | integer | yes | Unix epoch **milliseconds** at which the sender constructed/sent this envelope | +| `body` | object | yes | type-specific payload, see `messages/.schema.json` | + +No other top-level field is permitted, and this strictness is enforced at +runtime: a receiver MUST reject an envelope carrying any unknown top-level +field with `error{code: malformed}`. This rule is permanent — it MUST NOT +be relaxed by any future minor version (the runtime tolerance for unknown +fields in §15 applies **only inside `body`**). The strict top level is what +guarantees that **an envelope never carries an authentication credential** +(§10) — there is no `token`, `secret`, or similar field, at this level or +inside `body`, in any message type, and any attempt to smuggle one in at +the top level is mechanically rejected (see +`fixtures/invalid/envelope-carries-credential-field.json`). + +`id` identifies one transmission, not one logical event. When a reliable +event (§5) is retransmitted after a disconnect, it is sent in a **new** +envelope with a **new** `id`; the identity that matters for deduplication is +`(device_id, device_seq)` inside `body`, never the envelope `id`. `id` exists +for logging, and for correlating a control-plane request with its `ack`/ +`error` response via `body.in_reply_to`. + +`ts` is the send-time of the envelope itself. It is **not** the same as any +domain timestamp carried inside `body` (e.g. `task.event.body.occurred_at`): +a resent `task.event` keeps its original `occurred_at` but gets a fresh `ts` +each time it is put on the wire. + +### 3.1 Timestamps are always milliseconds + +**Every** timestamp field anywhere in this protocol — `envelope.ts`, every +`occurred_at`, every metric `ts`, every lease `expires_at` — is an integer +count of milliseconds since the Unix epoch. There is no field anywhere in +this protocol expressed in seconds, and none MAY be added in a +backward-compatible way (introducing a seconds-based timestamp field would be +a breaking change requiring a major version bump, precisely because this +codebase has previously suffered a seconds/milliseconds confusion bug). + +To make this mechanically checkable, every timestamp field's schema +(`common.schema.json#/$defs/unix_ms_timestamp`) sets `minimum: +1000000000000` (13 digits — 2001-09-09T01:46:40.000Z in milliseconds). A +plausible Unix seconds value is at most 10 digits (until the year 2286), so +any value that is actually in seconds fails this bound and is rejected by +schema validation rather than silently misinterpreted. See +`fixtures/invalid/message-event-timestamp-in-seconds.json`. + +## 4. Message types (complete set) + +| `type` | direction | reliable? (§5) | schema | +|---|---|---|---| +| `hello` | offer: client → server; accept: server → client | n/a (connection setup) | `messages/hello.schema.json` | +| `resume` | viewer → server | n/a | `messages/resume.schema.json` | +| `snapshot` | server → viewer (only as reply to `resume`) | n/a | `messages/snapshot.schema.json` | +| `delta` | server → viewer | n/a (carries reliable events, itself not acked) | `messages/delta.schema.json` | +| `ack` | server → source (event acks); server → viewer (control-plane acks); source → server (optional command-delivery acks) | n/a (is itself the ack) | `messages/ack.schema.json` | +| `task.event` | source → server (uplink only; reaches viewers wrapped in `delta`) | yes | `messages/task.event.schema.json` | +| `message.event` | source → server (uplink only; reaches viewers wrapped in `delta`) | yes | `messages/message.event.schema.json` | +| `metric.frame` | source → server → viewers | no (best-effort) | `messages/metric.frame.schema.json` | +| `config.event` | server → viewer only (server-minted; in v1 always carried inside `delta`, never standalone) | yes (server-minted, §5.5) | `messages/config.event.schema.json` | +| `subscribe` | viewer → server | n/a | `messages/subscribe.schema.json` | +| `unsubscribe` | viewer → server | n/a | `messages/unsubscribe.schema.json` | +| `interest.renew` | viewer → server | n/a | `messages/interest.renew.schema.json` | +| `command` | viewer → server → source, or server → source | delivery best-effort, effect idempotent (§8) | `messages/command.schema.json` | +| `error` | any → any | n/a | `messages/error.schema.json` | + +This is the complete set of 14 message types for protocol version 1. No +implementation may invent an additional type without a version negotiation +(§9) that both peers agree to; see §15 for how an unrecognized type is +handled in the meantime. + +**Viewer-facing reliable events travel only inside `delta`.** A viewer +never receives a bare `task.event`, `message.event`, or `config.event` +envelope: each time the server applies one reliable event it emits a +single-event `delta` (§6.2) to every viewer holding an active interest +lease. This is what gives every viewer-visible reliable event an attached +revision, so a viewer can always tell exactly where it stands. + +## 5. Reliable events + +The reliable events are `task.event` and `message.event` (device-uplinked, +§5.1–5.4) and `config.event` (server-minted, §5.5). Device-uplinked +reliable events work at-least-once: a source device MUST keep retrying +delivery until it is acknowledged, and the server MUST deduplicate so that +retries never apply twice. + +### 5.1 `device_seq` + +`device_seq` is scoped to one **(device, space)** pair: every source device +maintains **one** monotonically increasing counter per space it reports +into, starting at 1, **shared across both** `task.event` and +`message.event` (they are not counted separately; `config.event` does not +participate at all, §5.5). Within one (device, space) scope, a device: + +- MUST include `device_seq` in every `task.event` and `message.event` it + sends. +- MUST NOT reuse a `device_seq` value once used for a different event. +- MUST NOT send `device_seq` values out of increasing order. +- MAY skip values (e.g. 5 then 7) if it locally determines an event is + unrecoverably lost (e.g. it crashed mid-event and cannot reconstruct it); + the server MAY surface this to the source as an advisory + `error{code: sequence_gap}` (§13), but MUST still accept and apply the + event that arrived. + +### 5.2 Deduplication + +The server deduplicates reliable events on the pair `(device_id, +device_seq)`. Receiving the same pair twice (see +`fixtures/scenarios/duplicate-device-seq/`): + +- MUST NOT apply the event's effect a second time. +- MUST NOT increment `space_revision` (§6) a second time. +- MUST still respond with an `ack` covering that `device_seq`, so the + sender can retire it from its resend queue. + +This is why `id` (envelope-level) is deliberately independent of +`device_seq`: repeated transmissions of the same logical event are expected +and harmless, precisely because the identity the server keys on lives in +`body`, not in the transport-level envelope wrapper. + +### 5.3 Acknowledgement and resend + +- The server MUST send `ack{body.acked: [{device_id, device_seq}, ...]}` + once it has durably applied one or more reliable events. `acked` MAY + batch multiple `device_seq` values in a single `ack` envelope, but every + pair in one ack MUST carry the `device_id` of the connection's + authenticated device — an ack is always addressed to exactly one device + (and routed to that device's most recent connection, §9.4); a receiver + that finds a pair for any other device MUST treat the envelope as + `malformed`. +- `acked` is an **exact enumeration**, not cumulative coverage: + acknowledging `device_seq` 7 confirms event 7 only and says nothing about + events 1–6. A sender retires from its resend queue exactly the + `device_seq` values literally listed, no more. +- A source device MUST keep every sent `task.event`/`message.event` in a + local resend queue until it receives an `ack` that covers its + `device_seq`. +- A source device SHOULD use an increasing backoff between resend attempts + of the same unacked event while a connection is up, and MUST resend every + still-unacked event (oldest first) immediately after `hello`/`hello accept` + completes on a new connection (§5.4, "Agent reconnect replay flow"). + It does not send a `resume` for its own queue — `resume` is a viewer-only + message (§6.3). + +### 5.4 Agent reconnect replay flow (step by step) + +See `fixtures/scenarios/agent-reconnect-replay/` for the executable version +of this flow. + +1. Source device is connected, sends `task.event` for `device_seq` 10 and + 11. +2. Server acks `device_seq` 10; the ack for 11 never reaches the source + (connection drops, or the ack itself is lost in flight — the source + cannot distinguish these, and does not need to). +3. Connection drops. The source's local resend queue now holds one entry: + `device_seq` 11. +4. Source reconnects: sends `hello{stage: offer}`, receives + `hello{stage: accept}`. +5. Source immediately resends `device_seq` 11 in a fresh envelope (new + `id`, new `ts`, identical `body.occurred_at` and domain content). +6. Server applies it if it had not already done so, or recognizes the + duplicate if it had (§5.2) — either way it now sends `ack` for + `device_seq` 11. +7. Source's resend queue is now empty; steady-state resumes. + +The source does **not** need to know whether the server actually received +event 11 before the drop. Idempotent apply-by-`(device_id, device_seq)` +makes resending always safe. + +### 5.5 Server-minted reliable events: `config.event` + +Configuration changes (automation created/edited/removed) are initiated +through the HTTP control plane, where the server is the sole writer. The +server records each such change as a **`config.event`** — a reliable event +with special provenance rules: + +- Its body carries **no `device_id` and no `device_seq`**: since the server + mints it, there is no uplink to deduplicate. Its identity IS the + `space_revision` slot it occupies. +- Minting a `config.event` and incrementing `space_revision` MUST be a + **single durable transaction**: the event exists if and only if its + revision slot does. This gives `config.event` a duplicate-resistance + guarantee equivalent to the `(device_id, device_seq)` deduplication of + uplinked events — in particular, retries of the HTTP control-plane + operation that triggered the change MUST NOT produce duplicate + `config.event`s (the control plane deduplicates its own retries before + minting). +- It is never acked by anyone and never sits in a resend queue. +- Like every reliable event, it is persisted and increments + `space_revision` by exactly 1 (§6.1), and reaches viewers inside `delta` + (§6.2) — so a delta-following viewer and a snapshot-taking viewer + converge on the same automation state (`snapshot.body.automations` + carries the folded result). +- No client ever sends `config.event`; a client-sent one MUST be rejected + with `error{code: unauthorized}` (§10.1). In protocol version 1 the + server also never sends it as a standalone envelope — it appears only as + an entry in `delta.body.events` — but the type is registered and + schema'd (`messages/config.event.schema.json`) so the body shape is + pinned and the name is reserved. + +## 6. `space_revision` and state synchronization + +### 6.1 Definition + +Every space has one monotonically increasing integer counter, +`space_revision`, starting at 0. The server increments it by **exactly 1** +each time it durably applies one reliable event: a `task.event` or +`message.event` from a source device (§5.1–5.4), or a server-minted +`config.event` (§5.5). + +`metric.frame` and every control-plane message type (`hello`, `resume`, +`snapshot`, `delta`, `ack`, `subscribe`, `unsubscribe`, `interest.renew`, +`command`, `error`) never change `space_revision`. + +### 6.2 `delta` (catch-up and live) and `snapshot` + +`delta` is the **only** carrier of reliable events toward a viewer. It is +sent in two situations: + +- **Live**: each time the server applies one reliable event, taking the + space from revision `R-1` to `R`, it MUST send a single-event + `delta{from_revision: R-1, to_revision: R, events: [e]}` to every viewer + **connection that is delta-eligible**: it holds an active interest lease + (§7) AND has completed the full `hello → subscribe → resume` sequence, + including having been sent a **`snapshot` or `delta` reply** to its most + recent `resume` (§6.3). An `error` reply (e.g. `revision_unavailable`) + does NOT make the connection delta-eligible; eligibility begins only + with a successful snapshot-or-delta reply. The server MUST NOT send any + live delta on a connection before that reply. +- **Catch-up**: as the reply to `resume` (§6.3), covering the range + `(from_revision, to_revision]`. + +Rules, in both situations: + +- `to_revision - from_revision` MUST equal `events.length`, since each + event advances the revision by exactly 1. `events` MAY be empty only in + the explicit you-are-current reply (`from_revision == to_revision`, + §6.3). +- A server MUST send a catch-up `delta` only when it has retained every + reliable event in the range. If it cannot — because the requested + `last_revision` is older than its retention window, or because it has no + history at all for the space — it MUST send `snapshot` instead. This is + not an error condition. +- When a catch-up range would exceed the frame size limit (§11), the + server MUST split it **on event boundaries** into several consecutive + `delta`s whose ranges chain exactly: `d1.to_revision == + d2.from_revision`, and so on. Each chained delta is individually valid, + and the viewer applies them one by one under the §6.3 rules — no special + chunk-reassembly logic is needed for deltas. + +`snapshot` carries the full folded state (`tasks`, `metrics`, `messages`, +`automations`) of a space at one revision, and is sent **only** as a reply +to `resume` — nothing else triggers it. Because a full snapshot routinely +exceeds the frame limit, it is chunkable: + +- Every chunk carries the SAME `revision`; `part` numbers the chunks + consecutively from 1; `final: true` marks the last chunk. A single-chunk + snapshot is `part: 1, final: true`. +- Chunks of one snapshot MUST be sent consecutively, in order, on the same + connection. **While a chunked snapshot is in flight, the server MUST + defer every other outbound envelope on that connection — including + control-plane `ack`s — until after the `final` chunk**; only the + `ping`/`pong` heartbeat text frames (§9.3), which are not envelopes, may + interleave. A viewer that receives any non-`ping`/`pong` envelope + between chunks MUST treat it as a malformed sequence and MAY close the + connection and reconnect. +- The receiver concatenates the four arrays across chunks and MUST NOT + apply anything until the `final` chunk has arrived. If the connection + drops mid-snapshot, the partial chunks are discarded and the viewer + resumes afresh on its next connection. +- A snapshot MAY be empty (all four arrays empty) — e.g. the reply to + `resume{last_revision: 0}` on a brand-new space. +- `snapshot.body.metrics` is a **best-effort convenience cache**: metric + samples are not persisted and are outside `space_revision` accounting + (§12), so this array MAY be empty or stale — for instance after a server + restart — even while metrics are actively flowing. A viewer MUST treat + it as a hint and rely on subsequent `metric.frame` envelopes for truth. + +### 6.3 Client resume flow (step by step) + +The mandatory viewer connection sequence is **`hello` → `subscribe` → +`resume`**, and `resume` is **required on every connection**, not +optional: a connection — including a new connection that supersedes an +old one (§9.4) — becomes **delta-eligible** only once it has completed +all three steps in order and the server has sent a snapshot-or-delta +reply to its `resume` (§6.2; an `error` reply does not confer +eligibility). A +viewer MUST NOT send `resume` before it has sent `subscribe` on the same +connection; a server receiving them out of order MUST reject the early +`resume` with `error{code: malformed, retryable: true, fatal: false}`. +Within one connection the transport delivers messages in order (§11), +which this flow relies on. + +The gate has a server side and a defensive viewer side: + +- **Server gate**: the server MUST NOT send any live delta on a + connection until it has sent that connection's resume reply. The resume + reply is therefore always the **first** delta-family envelope (`delta` + or `snapshot`) the server sends on a connection. +- **Viewer defense**: a viewer MUST discard any `delta` received on a + connection before that connection's resume reply. This is provably + safe, not merely heuristic: frames on one connection are delivered in + order (§11), so a delta that arrives before the reply was necessarily + *sent* before the server computed the reply — the reply's + revision therefore already covers that delta's content. (This rule is + pure defense-in-depth against a non-conformant server; a conformant + server never triggers it.) + +Together these remove any window in which a viewer would have to evaluate +a delta against an uninitialized local revision: `C` (below) is defined +by the resume reply, and no delta is ever evaluated before it exists. + +`resume` MUST produce **exactly one reply** — never silence. The complete +decision table, for `resume{last_revision: N}` against the space's current +revision `R`: + +| condition | server reply | +|---|---| +| `N == 0` | `snapshot{revision: R, ...}` (chunked if needed; possibly empty) | +| `0 < N == R` | empty `delta{from_revision: N, to_revision: N, events: []}` — the explicit "you are already current" signal | +| `0 < N < R`, full range retained | `delta{from_revision: N, to_revision: R, events: [...]}` (split into chained deltas if needed, §6.2) | +| `0 < N < R`, range not fully retained | `snapshot{revision: R, ...}` (chunked if needed) | +| `N > R` | `error{code: revision_unavailable, retryable: true, fatal: false}`; the viewer MUST retry with `last_revision: 0` | + +The chunked-snapshot and chained-delta replies each count as the one +reply. + +After the reply, the viewer maintains a local current revision `C` +(initialized to the reply's `revision`/`to_revision`) and applies every +subsequently received `delta` by these rules: + +- `from_revision < C`: **discard silently.** The delta's content is + already covered by the resume reply or an earlier applied delta (the + same in-order argument as the pre-reply discard rule above: anything + the server sent with an older `from_revision` reflects state the viewer + has already absorbed). +- `from_revision == C`: **apply**, then set `C = to_revision`. +- `from_revision > C`: **revision gap** — one or more deltas were lost + (which a transport with in-order lossless frames should not produce, but + e.g. a server-side drop under pressure might). The viewer MUST send a + fresh `resume{last_revision: C}` and process the new reply under the + same table above. + +Worked example (executable versions: +`fixtures/scenarios/client-resume-delta/` and +`fixtures/scenarios/client-revision-gap-snapshot/`): + +1. Viewer connects: `hello{stage: offer, role: viewer}` / + `hello{stage: accept}`. +2. Viewer sends `subscribe`; server replies `ack{lease: {expires_at}}`. + The lease is now active, but this connection is **not yet + delta-eligible**: the server sends no live delta on it yet, even if + reliable events are being applied to the space right now. +3. Viewer sends `resume{last_revision: 126}`. +4. Server has revisions 127–128 retained → replies + `delta{from_revision: 126, to_revision: 128, events: [2 events]}` — + the first delta-family envelope on this connection. Viewer applies it, + `C = 128`; the connection is now delta-eligible. (Had the retention + window not reached back to 126, the reply would instead have been + `snapshot{revision: 128, part: 1, final: true, ...}` and the viewer + would discard its stale local state; had the viewer sent + `last_revision: 128`, the reply would have been the empty delta. Any + reliable events applied between steps 2 and 4 are simply included in + the reply itself — the reply is computed against the space's current + revision at reply time.) +5. A source reports progress; the server applies it (revision 129) and + sends `delta{from_revision: 128, to_revision: 129, events: [1 event]}` + to every delta-eligible lease-holding connection. `from_revision == C`, + so the viewer applies it, `C = 129`. +6. Any duplicate or late delta with `from_revision < 129` is discarded; + any future delta with `from_revision > C` triggers a fresh + `resume{last_revision: C}` (and while that re-resume is outstanding, + the same pre-reply discard rule applies to any delta that races the + new reply). + +### 6.4 Deterministic event folding + +Two viewers — one that replayed deltas, one that took a snapshot — MUST +converge on identical task and automation state. To guarantee that, the +folding of reliable events into state is fixed, field by field. Servers +MUST produce `snapshot` contents by these exact rules, and viewers MUST +apply delta-carried events by these same rules. + +**`task.event` sequence → `task_state`** (events applied in revision +order): + +| field | folding rule | +|---|---| +| `task_id`, `device_id` | from the event; constant across a run | +| `state` | `started`/`progress`/`step` → `running`; `done` → `done`; `failed` → `failed` | +| `title` | latest **non-empty** `title` seen; an event without `title` (or with an empty one) leaves it unchanged | +| `percent` | latest `percent` seen while running; on `done` → set to `100`; on `failed` → keep last value | +| `step` | latest `step` seen while running; on `done` or `failed` → **cleared** (absent) | +| `message` | from the `done`/`failed` event only; absent while running | +| `updated_at` | the `occurred_at` of the latest applied event — **never** the server's receive time | +| `display` | latest event that carried a `display` object wins wholesale (no per-key merge) | + +**`config.event` sequence → automation set**: `automation.upserted` +replaces the automation's entire state with the event's `automation` +object (create-or-replace, no per-field merge); `automation.removed` +deletes it. `snapshot.body.automations` is exactly the surviving set. + +**`message.event` sequence → message history**: append-only, ordered by +revision. The server keeps a bounded window: `snapshot.body.messages` +carries only the most recent `N` messages (implementations SHOULD document +their `N`; 200 is RECOMMENDED). **Truncation to this window is normative +behavior**, and the resulting asymmetry — a delta-replaying viewer may +hold older messages than a snapshot-taking viewer ever sees — is an +**accepted, expected deviation**: message history is a log, not folded +state, and the convergence guarantee above covers task and automation +state, not the depth of message history. + +## 7. Interest lease + +A source device MAY reduce the frequency and richness of what it publishes +when it knows nobody is watching. The interest lease is how a viewer tells +the server (and transitively, the server tells the source) that someone is +watching. + +Leases are held **per device**, not per connection, and their lifetime is +**decoupled from connection lifetime**: a lease ends in exactly two ways — +its `expires_at` passes without renewal, or the device sends an explicit +`unsubscribe`. A connection drop does NOT end the device's lease; a device +that reconnects within its lease window still counts as interested the +whole time. Because leases are persisted (§14), this also gives a +restarted server the basis to rebuild the space's interest count instead +of spuriously firing a throttle notification. + +- `subscribe` establishes (or wholly replaces) the sending device's lease. + The server MUST choose a lease duration between 30000 ms and 60000 ms + inclusive and communicate the absolute deadline as + `ack.body.lease.expires_at` (an absolute timestamp, not a duration, so + the viewer never needs to know or assume the exact constant the server + chose). 45000 ms is RECOMMENDED as a default when an implementation has + no other reason to pick a value. +- `interest.renew`, sent by the viewer before `expires_at`, extends the + lease; the server responds the same way, with a freshly computed + `expires_at`. `interest.renew` ALWAYS replaces the lease's `topics` + wholesale with the request's `topics` (omitted/empty meaning all) — it + never merges with or preserves the previous set. If the device holds no + active lease when `interest.renew` arrives (already lapsed, or never + established), the server MUST treat it identically to a fresh + `subscribe` using the request's `topics` — this avoids a race between + the viewer's local renewal timer and server-side expiry, and means + `interest.renew` never needs to fail with "no such lease". +- `unsubscribe` ends the lease immediately, without waiting for + `expires_at`, and is confirmed with `ack{in_reply_to}` like `subscribe` + and `interest.renew` (with no `lease` object, since none remains). +- A lease that is not renewed lapses silently at `expires_at`; no envelope + marks the lapse by itself. Expiry detection MAY be **lazy**: the server + is not required to run a timer to the exact `expires_at` instant — it + MAY evaluate lease expiry the next time it processes any state change or + interest operation for the space, as long as an expired lease is never + counted as active in that evaluation. +- `topics` scoping: in protocol version 1 only the `metric` topic has + filtering effect (a lease without it receives no `metric.frame` + forwarding). `task` and `message` are advisory: the server MUST always + deliver every reliable single-event `delta` to every active lease + holder regardless of topics, because withholding any reliable event + would break that viewer's revision continuity (§6.3) and force it into + a resume loop. + +A lease's survival across disconnects and connection supersession (§9.4) +affects **only** this section's throttle-edge accounting: in particular, +a supersession MUST NOT cause a `throttle`/`resume_rate` emission, since +the device's lease — and therefore the space's count — is unchanged by +it. Lease survival does NOT make any connection eligible to receive +deltas: delta eligibility is strictly per connection and requires that +connection's own completed `hello → subscribe → resume` sequence (§6.3). + +The server tracks **one lease-count per space**: the number of devices +currently holding an unexpired lease. A device with multiple simultaneous +connections (§9.4) contributes exactly once. On every transition of that +count: + +- **1 → 0** (the last active lease in the space just ended, by expiry or + `unsubscribe`): the server MUST send + `command{origin: "server", action: "throttle"}` (§8) to every source + device currently connected to the space. +- **0 → 1** (a `subscribe` or `interest.renew` just created the space's + first active lease): the server MUST send + `command{origin: "server", action: "resume_rate"}` to every source device + currently connected to the space. + +Both notifications omit `target_device_id` (broadcast to every source in +the space) unless an implementation has a reason to scope more narrowly. +`throttle` is advisory: a source SHOULD reduce its `metric.frame` cadence +while throttled (an implementation-defined lower rate; this document does +not fix one), but MUST continue sending `task.event` for lifecycle +transitions (`started`/`done`/`failed`) and every `message.event` at normal +priority regardless of throttle state — those are cheap, reliable, and +often user-facing (push notifications), so there is no cost benefit to +throttling them. + +See `fixtures/scenarios/interest-lease-expiry/` for the executable version +of this flow, including the two-viewer resume_rate case. + +## 8. `command`: reverse control + +`command` carries an instruction toward one or more source devices. It has +two distinct origins that share one envelope type: + +- `origin: "viewer"` — a human-initiated action, relayed by the server: + `action` is one of `pause`, `resume`, `stop`, `run_now`. The sender MUST + set `issued_by_device_id` to its own authenticated `device_id`; the + server MUST reject a mismatch with `error{code: unauthorized}` (§10.1) and + MUST NOT relay the command to any source in that case. +- `origin: "server"` — a lease-driven notification the server generates + itself (§7): `action` is one of `throttle`, `resume_rate`. **The server + MUST reject ANY client-sent command with `origin: "server"` with + `error{code: unauthorized}`, regardless of action** (see + `fixtures/invalid/role-viewer-command-origin-server.json`); additionally + the schema rejects a viewer-origin envelope that carries `throttle`/ + `resume_rate` (see + `fixtures/invalid/command-viewer-sends-throttle.json`). + +Per-action required-field matrix — the command names the exact object it +operates on, aligning with the HTTP control surface +(`POST /v2/tasks/:id/commands`): + +| `action` | `origin` | required object field | forbidden object field | +|---|---|---|---| +| `pause`, `resume`, `stop` | `viewer` | `task_id` (the task run to act on) | `automation_id` | +| `run_now` | `viewer` | `automation_id` (the automation to trigger) | `task_id` | +| `throttle`, `resume_rate` | `server` | none (device-level, not object-level) | both `task_id` and `automation_id` | + +The forbidden column is enforced by schema (`not`/`required` patterns in +`messages/command.schema.json`): a command carrying an object field its +action does not use is `malformed`, never silently tolerated. + +Other fields: + +- `command_id` is an idempotency key chosen by the sender (the viewer for + `origin: viewer`, the server for `origin: server`). A source device MUST + track `command_id`s it has already acted on and MUST NOT execute the + same `command_id` twice. +- `target_device_id`, when present, scopes the command to one source + device; when absent, it applies to every source device connected to the + space (used for `throttle`/`resume_rate`). +- `ttl_ms` bounds how long, from `envelope.ts`, the command remains + actionable. A device that would first act on a command after `ts + + ttl_ms` has passed MUST drop it without executing it, and SHOULD emit + `error{code: command_expired, retryable: false}`. +- `params` is a reserved extension point; **protocol version 1 defines no + keys in it**. Unlike every other object in this protocol, `params` + explicitly permits additional properties, which a receiver that does not + recognize them MUST ignore. + +Relay rules (viewer → server → source): + +- When relaying, the server preserves the **original** envelope `ts` and + `command_id` unchanged; only the envelope `id` is freshly assigned (it + identifies the new transmission, §3). TTL therefore measures from the + viewer's send time, not the relay time. +- The server validates the TTL **at relay time** and MUST NOT relay an + already-expired command (it SHOULD answer the viewer with + `error{code: command_expired}` instead). +- The receiving source re-validates the TTL against its **own local + clock**, allowing ±30 s of clock skew: the command is actionable while + the local clock reads within `[ts − 30000, ts + ttl_ms + 30000]`. In + particular a source MUST NOT reject a command merely because its `ts` + appears to lie slightly in the local future. + +The §10.1 authorization matrix governs only client-originated `command` +envelopes (i.e. `origin: "viewer"`); the matrix does not restrict what the +trusted server itself may send — the server is not a "role" in the +source/viewer sense, and messages it originates (like `throttle`) are not +subject to the client authorization checks in §10.1. + +A source device receiving a `command` MAY send back +`ack{body.in_reply_to: }` to confirm delivery, +but this is OPTIONAL: correctness does not depend on it, since +`command_id`-based idempotency (above) already makes re-delivery safe, and +the command's effect is independently observable through domain events +(next paragraph). + +A `command`'s *effect* is observed by everyone through ordinary domain +events: e.g. a successful `pause` surfaces as a subsequent `task.event` +reporting the affected task's new state. This protocol intentionally has no +separate "command result" message type — reusing the existing reliable +event stream for effects keeps the type list closed and avoids a second +notion of "did it work" alongside `ack`/`error`. + +## 9. `hello` and version negotiation + +### 9.1 Ordering and handshake timeouts + +`hello` MUST be the first envelope sent on every new connection, in **both** +directions: the client's `hello{stage: offer}` (client → server) and the +server's `hello{stage: accept}` (server → client). + +The handshake is strictly sequential: + +- A client may only ever send `hello{stage: "offer"}`; `stage: "accept"` + belongs exclusively to the server. If the server receives a + `hello{stage: "accept"}` from a client — at any point in the + connection's life — it MUST treat it as a handshake violation: reply + `error{code: hello_required, fatal: true}` and close the connection. +- After sending its offer, the client **MUST NOT send any other envelope + until it has received and validated the server's accept.** There is no + pipelining of `subscribe`/`resume`/events behind the offer. +- The server MUST answer any frame it receives before it has sent its + accept — including a second `hello` — with `error{code: hello_required, + fatal: true}` and then close the connection. (`hello_required` is only + ever sent by the server; a client never polices this.) +- The server MUST NOT send any envelope other than its accept (or an + `error`) before the handshake completes. + +Recommended timeouts: the server SHOULD close a connection that has sent +no offer within **10 s** of transport establishment; the client SHOULD +close and reconnect if no accept has arrived within **10 s** of sending +its offer. + +### 9.2 Offer / accept + +- The connecting device sends `hello{body.stage: "offer", device_id, role, + protocol_versions, capabilities?}`. `protocol_versions` lists every major + protocol version that device can speak, ascending, at least one entry. +- The server computes the intersection of the offer's `protocol_versions` + with its own supported set and selects the **maximum** version in that + intersection. + - If the intersection is non-empty: the server MUST reply + `hello{body.stage: "accept", protocol_version, session_id, + heartbeat_interval_ms, capabilities?}` using the selected version, and + the connection proceeds on that version. + - If the intersection is empty: the server MUST reply + `error{code: version_unsupported, retryable: false, fatal: true}` and + then close the connection. It MUST NOT send `hello{stage: accept}` in + this case. +- `session_id` is an opaque, per-connection identifier for logging and + observability. It is not a credential and is not required to resume + state — resuming is entirely keyed by `space_revision` (§6), not by + session continuity. +- `role` is the realtime-protocol role (`source` or `viewer`), which is + coarser than and distinct from the HTTP-layer roles (`owner`, `admin`, + `viewer`, `source`) used for pairing and REST access: an `owner` or + `admin` device opening a realtime connection to observe state always + presents `role: "viewer"` here. Only a device that executes and reports + tasks/metrics/messages presents `role: "source"`. + +### 9.3 Heartbeat + +Once `hello` has completed, either side MAY send a bare text frame +containing exactly the four ASCII bytes `ping` — not a JSON envelope, no +`type`/`id`/`ts`/`body` wrapper — at the interval named in +`hello{stage: accept}.heartbeat_interval_ms`. This is deliberately plain +text rather than JSON specifically so that a transport-layer intermediary +that does byte-level keepalive (a reverse proxy, load balancer, or any +other pass-through component) MAY answer it directly without parsing JSON +or waking any protocol/business logic above it. + +A recipient of a `ping` text frame MUST respond with a bare text frame +containing exactly the four ASCII bytes `pong`, as promptly as possible, +without forwarding it to application logic. A `pong` received without a +preceding `ping` MUST be ignored. This exchange MUST NOT reset or otherwise +interact with the interest lease (§7) or any resend timer (§5.3) — it is +a pure liveness probe, decoupled from every other piece of protocol state. +Either side MAY close the connection if it sends `ping` and observes no +`pong` within an implementation-defined timeout (2 × `heartbeat_interval_ms` +is RECOMMENDED). + +### 9.4 Connection supersession + +At most one connection per device is current. When a device completes +`hello` on a new connection while the server still holds an open connection +for the same authenticated `device_id`: + +- The server MUST accept the **new** connection. +- The server MUST close the **old** connection, first sending + `error{code: superseded, retryable: false, fatal: true}` on it. + `retryable: false` refers to that dying connection; the device's newer + connection is unaffected and simply continues. +- From that moment, ALL server-to-device traffic — `ack`s for the device's + reliable events, targeted `command`s, live deltas — MUST route to the + device's **most recent** connection. A resume reply is the exception: it + is always sent on the connection that carried the `resume` request, so a + reply still pending on a superseded connection dies with that connection + and is never forwarded (the new connection obtains its own reply by + running the full sequence below). +- Interest leases are unaffected: they are held per device (§7), so + supersession neither ends a lease nor double-counts it — a device + briefly holding two open connections contributes exactly one lease to + the space's count, and the supersession itself MUST NOT trigger a + `throttle`/`resume_rate` emission (§7). +- Lease survival does NOT carry delta eligibility across connections. + The new connection MUST run the full `hello → subscribe → resume` + sequence (§6.3) before it receives any delta — for a viewer, `resume` + is required on every connection — and the viewer's local revision `C` + is re-established from the **new** connection's resume reply. No + cross-connection inheritance of `C` (or of any in-flight snapshot + chunks or pending replies) is defined by this protocol. + +A client that receives `superseded` after itself opening a new connection +treats it as confirmation the new connection won. A client that receives +`superseded` without having opened a new connection SHOULD surface the +situation (its credential may be in use elsewhere) rather than silently +reconnecting in a loop. + +See `fixtures/scenarios/duplicate-connection-supersede/`. + +## 10. Authentication boundary + +The credential (a `st2__` token, or an equivalent a future +transport might use) is presented **once, at transport/connection +establishment** — e.g. as a WebSocket upgrade header or query parameter, or +whatever the concrete transport's native mechanism for out-of-band +credentials is. It is never repeated inside any envelope afterward. + +An envelope carries only `device_id` (in `hello` and event bodies) and, +where relevant, `issued_by_device_id`/`target_device_id` (in `command`) — +none of these are secrets; they identify a device, they do not +authenticate one. The server MUST bind `device_id` and `role` to the +identity it resolved from the connection-establishment credential and MUST +NOT trust a `device_id` or `role` value asserted inside a +`hello{stage: offer}` body if it conflicts with that resolved identity. + +The same binding applies to every event a source uplinks: the server MUST +reject any `task.event`, `message.event`, or `metric.frame` whose +`body.device_id` does not match the connection's authenticated device +identity, with `error{code: unauthorized}`. A source can only ever report +as itself. + +### 10.1 Authorization matrix + +The server MUST validate **every** incoming envelope's `type` against the +sending connection's `role`, not only once at connection time. A message +sent by a role not listed below MUST be rejected with +`error{code: unauthorized}`. + +| `type` | source may send | viewer may send | +|---|---|---| +| `hello` | yes (`stage: "offer"` only — a client-sent `stage: "accept"` is a handshake violation answered with `hello_required` + close, §9.1) | yes (`stage: "offer"` only, same rule) | +| `resume` | no | yes | +| `snapshot` | no (server-only) | no (server-only) | +| `delta` | no (server-only) | no (server-only) | +| `ack` | yes, OPTIONAL (may acknowledge receipt of a `command` it was sent; not required for correctness since `command_id` already makes execution idempotent, §8) | no (a viewer has no scenario in this protocol requiring it to send `ack`: the server always originates the acks a viewer receives for `subscribe`/`unsubscribe`/`interest.renew`, and a viewer never receives a reliable event it must acknowledge — catch-up after a gap uses `resume`/`delta`/`snapshot`, §6, not per-message acking) | +| `task.event` | yes (body.device_id must match the authenticated identity) | no | +| `message.event` | yes (body.device_id must match the authenticated identity) | no | +| `metric.frame` | yes (body.device_id must match the authenticated identity) | no | +| `config.event` | no (server-only) | no (server-only) | +| `subscribe` | no | yes | +| `unsubscribe` | no | yes | +| `interest.renew` | no | yes | +| `command` | no (only receives it) | yes, with `origin: "viewer"` only — a client-sent `origin: "server"` is always unauthorized regardless of action (§8) — and actions limited to `pause`/`resume`/`stop`/`run_now` | +| `error` | yes | yes | + +`snapshot`, `delta`, and `config.event` are server-to-client only; a +client that sends any of them MUST be rejected with +`error{code: unauthorized}`. This matrix governs client-originated +envelopes only — see §8 for why server-originated `command` envelopes are +not restricted by it. + +## 11. Transport framing limits + +- One frame carries exactly one JSON object (or the literal `ping`/`pong` + text, §9.3) — this protocol never batches multiple envelopes into one + frame. +- The transport MUST deliver frames of one connection **in order**. The + resume/delta interleaving rules (§6.3) and snapshot chunk reassembly + (§6.2) rely on this. +- A frame MUST NOT exceed **64 KiB** of UTF-8-encoded JSON. A sender that + would exceed this MUST split the content across multiple envelopes. + Every payload that can grow has a defined split mechanism: + - `snapshot` → chunks (`part`/`final`, same `revision`; §6.2), + - catch-up `delta` → consecutive chained deltas split on event + boundaries (`d1.to_revision == d2.from_revision`; §6.2), + - `metric.frame` → multiple frames (each within the item cap below), + - `ack.body.acked` → multiple `ack` envelopes. + A receiver that gets an oversized frame MUST reject it with + `error{code: frame_too_large, retryable: true}` and MUST NOT attempt to + parse it. +- Every free-text field is capped by the shared `$defs` in + `common.schema.json` (`free_text` ≤ 2048 characters for task title, + step, completion message, and message text; `label_text` ≤ 256 for + metric labels and automation names), and every id/value field has its + own cap. These caps guarantee that any **single** record — a + `task_state`, `message_record`, `metric_sample`, `automation_state`, or + one delta event — serializes far below the 64 KiB frame limit. A + single record can therefore never exceed a frame by itself, and the + split mechanisms above (snapshot chunks, chained deltas) always have a + valid record/event boundary to split on. +- Batch item caps: `metric.frame.body.metrics` MUST NOT exceed 64 entries; + `delta.body.events` and `ack.body.acked` have no protocol-fixed cap but + are bounded in practice by the 64 KiB frame limit. A receiver that gets a + batch exceeding an enforced cap MUST reject it with + `error{code: batch_too_large, retryable: true}`. +- A connection MUST NOT emit more than 10 `metric.frame` envelopes per + second, regardless of how many distinct metrics it is reporting (see also + the per-metric-id cadence rule in §12); excess frames MAY be rejected + with `error{code: rate_limited, retryable: true}`. + +## 12. `metric.frame` merge and ordering rules + +- A sender MUST merge multiple pending updates to the same `metric_id` + into a single sample (the latest one observed) before emitting a frame — + it MUST NOT emit two samples for the same `metric_id` within one frame. +- A sender MUST NOT emit a frame containing a given `metric_id` more often + than once per 500 ms (a 2 Hz ceiling per metric); it MAY combine several + different metrics that changed within that window into one frame. +- A receiver MUST discard any incoming sample whose `ts` is less than or + equal to the last-applied `ts` it has already accepted for that + `metric_id`, regardless of the order in which frames arrive at the + transport level. This makes stale/out-of-order frames harmless without + requiring in-order delivery from the transport. +- `metric.frame` is never acknowledged and never retransmitted: a dropped + frame is superseded by the sender's next one, and the protocol accepts + the resulting gap in the metric's visible history. + +## 13. Error codes + +Every `error` envelope's `body.retryable` and `body.fatal` fields are +authoritative and self-describing — an implementation never needs this +table to decide how to react to a specific error it receives. The table +below documents the fixed values every code in this version always carries, +for the benefit of implementers writing the code that produces them. + +For a code with `fatal: true` (the connection closes), `retryable: true` +means: **reconnect, then retry the logical operation on the new +connection** — never retry on the dying connection itself. + +| code | sent by | retryable | closes connection | when | +|---|---|---|---|---| +| `version_unsupported` | server | false | yes | `hello{stage: offer}`'s `protocol_versions` shares no version with the server's supported set (§9.2) | +| `hello_required` | server (only — a client never emits it) | true (after reconnect) | yes | any frame other than the offer arrived before the server sent its accept (§9.1) | +| `unauthenticated` | server | false | yes | the connection's credential became invalid mid-connection (e.g. revoked) | +| `unauthorized` | server | false | no | sender's role does not permit this `type` (§10.1); a client-sent `command` with `origin: "server"` (§8); a `command.issued_by_device_id` mismatch (§8); or an uplink event whose `body.device_id` does not match the authenticated identity (§10) | +| `malformed` | any | true | no | an envelope fails schema validation (including an unknown top-level envelope field, §3), or violates a protocol sequence rule other than the pre-hello case (e.g. `resume` before `subscribe`, §6.3; an `ack` pair for a foreign device, §5.3) | +| `rate_limited` | server | true | no | a per-connection frequency cap was exceeded (§11, §12) | +| `frame_too_large` | server | true | no | a single frame exceeded 64 KiB (§11) | +| `batch_too_large` | server | true | no | an array field exceeded its item cap (§11) | +| `revision_unavailable` | server | true | no | `resume.body.last_revision` is greater than the space's current `space_revision` (§6.3); the viewer retries with `last_revision: 0` | +| `command_expired` | any | false | no | a `command` was received or first actionable after its TTL window (§8) | +| `sequence_gap` | server | false | no | a source's `device_seq` skipped one or more values (§5.1); advisory only, the event that did arrive is still applied | +| `superseded` | server (only) | false (on that connection — the device's newer connection continues) | yes | the same device completed `hello` on a newer connection; this older one is being closed (§9.4) | +| `internal_error` | server | true | no | an unexpected server-side fault unrelated to the request's validity; its `message` MUST NOT contain stack traces, internal file paths, or other implementation details | + +Unknown fields anywhere in `body` MUST be ignored by every receiver in +every one of these cases and every other message type (§15) — `malformed` +is reserved for validation failures against fields this version of the +protocol *does* define, never for the mere presence of extra ones. + +## 14. Cost table + +This table is the basis for server-side cost/capacity review. "Persisted" +means the server durably stores the message or its effect beyond the +lifetime of the connection; "revision++" means it advances `space_revision`; +"outbox" means it may need delivery to a device that is not currently +connected (e.g. a push notification, or queued for a source that is +offline); "broadcast" means the server fans it out to every subscribed +viewer rather than replying only to the requester. + +| `type` | persisted | revision++ | outbox | broadcast | +|---|---|---|---|---| +| `hello` | no | no | no | no | +| `resume` | no | no | no | no | +| `snapshot` | no (computed on demand from persisted state; its `metrics` array is a best-effort cache outside revision accounting, §6.2) | no | no | no (unicast reply) | +| `delta` | no (computed from persisted event log) | no | no | yes (live single-event deltas fan out to every active lease holder, §6.2; catch-up deltas are unicast) | +| `ack` | no | no | no | no | +| `task.event` | yes | yes | yes (push/Live Activity update) | no as itself — re-emitted as a broadcast single-event `delta` | +| `message.event` | yes | yes | yes (push notification) | no as itself — re-emitted as a broadcast single-event `delta` | +| `metric.frame` | no | no | no | yes (best-effort, filtered by the lease's `metric` topic) | +| `config.event` | yes | yes | no (never triggers push) | no as itself — re-emitted as a broadcast single-event `delta` | +| `subscribe` | yes (lease record; survives disconnects, §7) | no | no | no | +| `unsubscribe` | yes (lease record removed) | no | no | no | +| `interest.renew` | yes (lease record updated) | no | no | no | +| `command` | yes (until delivered or expired) | no | yes (source may be offline) | no (targeted) | +| `error` | no | no | no | no | + +## 15. Compatibility policy + +- **Minor/patch changes** (this protocol's version stays `1`): additive + only. New optional **body** fields, new enum values in a field that is + documented as open to extension, or new `capabilities` flags. A receiver + MUST ignore fields **inside `body`** that it does not recognize — it + MUST NOT reject a message solely because of their presence. This + tolerance applies to `body` ONLY: the envelope's top level is + permanently strict (§3) — an unknown top-level field is always + `malformed`, in this and every future minor version, so no minor version + may add an envelope-level field. The body-level tolerance is a + **runtime requirement on implementations**; it is deliberately looser + than what the JSON Schemas in this directory enforce (most body schemas + here use `additionalProperties: false`). The schemas describe the + exact, frozen v1.0.0 conformance surface for this repository's fixtures + and validator (§16) — precision that is valuable for catching + implementation bugs against a known-good baseline — while production + parsers remain lenient toward body fields a future minor/patch version + might add. These are not in tension: the schemas are a conformance test + for *this* version, not a description of every byte a lenient parser + must accept. +- **Breaking changes** (removing a field, changing a field's meaning or + type, removing a message type, or tightening a previously-open + constraint) require a **major version bump**, a new entry in + `CHANGELOG.md`, and — because schema `$id`s are versioned by directory + (`.../realtime/v1/...`) — a new `v2/` schema namespace published + alongside `v1/` until every implementation has migrated. `hello`'s + version negotiation (§9.2) is exactly the mechanism that lets `v1` and + `v2` implementations coexist during that migration. +- **Unknown message types**: a receiver that gets an envelope whose `type` + it does not recognize MUST ignore that envelope (log it if useful) and + MUST NOT treat it as `malformed`. This lets a future protocol version add + a wholly new message type as an additive change from the perspective of + a peer that has not yet been upgraded, as long as that peer can safely + make no progress on it. + +## 16. Fixture validation + +`tools/validate.js` is a self-contained Node script (its own +`package.json`/`package-lock.json`, pinned `ajv@8.20.0`, no dependency on or +from `server/`, `daemon/`, or `apple/`) that: + +1. Loads `common.schema.json`, `envelope.schema.json`, and every + `messages/*.schema.json` into one Ajv (draft 2020-12) instance. +2. Asserts every fixture under `fixtures/valid/` and + `fixtures/scenarios/**/` validates against both the generic envelope + schema and the specific schema selected by its own `type` field. +3. Asserts every fixture under `fixtures/invalid/` is rejected — by one of + those two schemas, or by the client-authorization matrix (§10.1), + which the script mirrors (including the client-may-only-send-offer and + client-may-only-send-origin-viewer rules). +4. Checks semantic invariants JSON Schema cannot express: delta revision + arithmetic (`to_revision − from_revision == events.length`) on every + delta fixture, and — per scenario directory, in filename order — + consecutive-delta chaining, snapshot chunk runs (same revision, + consecutive parts from 1, nothing interleaved, `final: true` closing + the run), and per-device `device_seq` monotonicity (equal values + allowed, as retransmissions). +5. Exits non-zero if any fixture or invariant produces an unexpected + result. + +Fixtures come in two forms. A bare envelope object is schema-checked +only. A wrapper of the form +`{"sender_role": "source" | "viewer", "frame": {…}}` is schema-checked +AND checked against the §10.1 authorization matrix as if a client with +that role had sent the frame — this form exists for constraints that are +role-dependent rather than shape-dependent (e.g. +`fixtures/invalid/role-viewer-command-origin-server.json`, a perfectly +well-formed command that only the server may originate). The wrapper is a +test artifact, not a wire format. + +Run it with: + +```sh +cd proto/realtime/tools +npm install # once, materializes node_modules/ from the committed lockfile +npm run validate +``` diff --git a/proto/realtime/common.schema.json b/proto/realtime/common.schema.json new file mode 100644 index 0000000..5b55e31 --- /dev/null +++ b/proto/realtime/common.schema.json @@ -0,0 +1,163 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/common.schema.json", + "title": "Sitrep realtime protocol - shared definitions", + "description": "Reusable $defs referenced by envelope.schema.json and every schema under messages/. Not sent on the wire by itself.", + "$defs": { + "unix_ms_timestamp": { + "type": "integer", + "description": "Unix epoch time in milliseconds. MUST NOT be epoch seconds. The minimum bound is 1_000_000_000_000 (2001-09-09T01:46:40Z in milliseconds); any legitimate Sitrep timestamp is larger than this, so a seconds-scale value (10 digits, < 1e10) is always rejected by this bound. This is a deliberate guard against the historical seconds/milliseconds confusion in this codebase.", + "minimum": 1000000000000, + "maximum": 9999999999999 + }, + "envelope_id": { + "type": "string", + "description": "Opaque unique identifier for one envelope, assigned by the sender. RECOMMENDED: UUIDv4 or ULID. Retransmissions of the same application-level event get a NEW envelope_id each time; reliable-event deduplication uses (device_id, device_seq) in the body, never this field.", + "pattern": "^[A-Za-z0-9_-]{1,64}$" + }, + "device_id": { + "type": "string", + "description": "Stable identifier for one physical device/installation, assigned at pairing time and reused across reconnects. Opaque to the protocol.", + "pattern": "^[A-Za-z0-9_-]{1,128}$" + }, + "device_role": { + "type": "string", + "description": "The realtime-protocol role of a connecting device. This is distinct from and coarser than the HTTP-layer roles (owner/admin/viewer/source) used for pairing and REST access: an 'owner' or 'admin' device that opens a realtime connection to observe state always connects with device_role 'viewer'. Only a device that executes and reports tasks/metrics/messages connects as 'source'.", + "enum": ["source", "viewer"] + }, + "device_seq": { + "type": "integer", + "description": "Monotonically increasing counter scoped to one (device, space) pair: each device keeps one independent counter per space it reports into, shared across ALL device-originated reliable event types (task.event and message.event) it emits into that space. The first reliable event a device ever sends in a given space uses device_seq 1. Within one (device, space) scope a device MUST NOT reuse a device_seq value and MUST NOT emit device_seq values out of increasing order, but a gap (e.g. 5 then 7) is permitted and signals to the receiver that event 6 was lost and cannot be recovered. config.event does not participate in this counter (it is server-minted and identified by its revision instead).", + "minimum": 1 + }, + "space_revision": { + "type": "integer", + "description": "Monotonically increasing counter for one space. Starts at 0 (empty space). Incremented by exactly 1 each time the server durably applies one reliable event (task.event or message.event, including a message-carried config change) to the space. metric.frame and control-plane messages never change it.", + "minimum": 0 + }, + "protocol_version": { + "type": "integer", + "description": "A single protocol major version number understood by an implementation.", + "minimum": 1 + }, + "task_id": { + "type": "string", + "description": "Identifier of one task run, stable for the lifetime of that run.", + "minLength": 1, + "maxLength": 256 + }, + "metric_id": { + "type": "string", + "description": "Identifier of one metric series. Matches the id grammar from proto/SPEC.md's metric.update verb.", + "pattern": "^[a-z0-9_.-]{1,64}$" + }, + "message_level": { + "type": "string", + "enum": ["info", "warn", "error"] + }, + "free_text": { + "type": "string", + "description": "Unified cap for user-visible free text: task title, step, completion message, and message text. The cap (2048 characters, so at most ~8 KiB of UTF-8) guarantees any single record built from these fields serializes far below the 64 KiB frame limit (SPEC.md section 11) - a single record can never exceed a frame.", + "maxLength": 2048 + }, + "label_text": { + "type": "string", + "description": "Unified cap for short labels and names: metric label, automation name.", + "maxLength": 256 + }, + "task_kind": { + "type": "string", + "description": "Lifecycle phase this task.event body reports. Named distinctly from the proto/SPEC.md stdout verbs (task.start/progress/step/done/fail) because this is a post-processed sync-layer event, not a raw process line.", + "enum": ["started", "progress", "step", "done", "failed"] + }, + "display_hints": { + "type": "object", + "description": "Optional, purely cosmetic presentation hints carried alongside a task or metric. Renderers MUST ignore hint values they do not understand.", + "additionalProperties": false, + "properties": { + "icon": { "type": "string", "maxLength": 64 }, + "tint": { "type": "string", "maxLength": 32 }, + "template": { "type": "string", "maxLength": 32 } + } + }, + "task_state": { + "type": "object", + "description": "Full current state of one task, as carried in a snapshot.", + "additionalProperties": false, + "required": ["task_id", "device_id", "state", "updated_at"], + "properties": { + "task_id": { "$ref": "#/$defs/task_id" }, + "device_id": { "$ref": "#/$defs/device_id" }, + "title": { "$ref": "#/$defs/free_text" }, + "state": { "type": "string", "enum": ["running", "done", "failed"] }, + "percent": { "type": "integer", "minimum": 0, "maximum": 100 }, + "step": { "$ref": "#/$defs/free_text" }, + "message": { "$ref": "#/$defs/free_text" }, + "updated_at": { "$ref": "#/$defs/unix_ms_timestamp" }, + "display": { "$ref": "#/$defs/display_hints" } + } + }, + "metric_sample": { + "type": "object", + "description": "Latest known sample of one metric, as carried in a snapshot or a metric.frame. The optional geometry/threshold fields (target/min/max/alert_above/alert_below) mirror the metric-owned hints in proto/SPEC.md so a viewer can render a gauge/bar and threshold markers from realtime data alone; like `value`, they are strings.", + "additionalProperties": false, + "required": ["metric_id", "value", "ts"], + "properties": { + "metric_id": { "$ref": "#/$defs/metric_id" }, + "value": { "type": "string", "maxLength": 256 }, + "label": { "$ref": "#/$defs/label_text" }, + "ts": { "$ref": "#/$defs/unix_ms_timestamp" }, + "display": { "$ref": "#/$defs/display_hints" }, + "target": { "type": "string", "maxLength": 64 }, + "min": { "type": "string", "maxLength": 64 }, + "max": { "type": "string", "maxLength": 64 }, + "alert_above": { "type": "string", "maxLength": 64 }, + "alert_below": { "type": "string", "maxLength": 64 } + } + }, + "automation_state": { + "type": "object", + "description": "Full current state of one automation, as carried in snapshot.body.automations and in config.event bodies.", + "additionalProperties": false, + "required": ["automation_id", "name", "executor_kind", "schedule", "state"], + "properties": { + "automation_id": { "type": "string", "minLength": 1, "maxLength": 128 }, + "name": { "$ref": "#/$defs/label_text", "type": "string", "minLength": 1 }, + "executor_kind": { "type": "string", "enum": ["script", "agent", "hybrid"] }, + "schedule": { + "type": "object", + "additionalProperties": false, + "required": ["kind", "every_seconds"], + "properties": { + "kind": { "const": "interval" }, + "every_seconds": { "type": "integer", "minimum": 1 } + } + }, + "state": { "type": "string", "enum": ["active", "paused"] }, + "last_run_at": { "$ref": "#/$defs/unix_ms_timestamp" } + } + }, + "message_record": { + "type": "object", + "description": "Full content of one message, as carried in a snapshot.", + "additionalProperties": false, + "required": ["message_id", "device_id", "level", "text", "occurred_at"], + "properties": { + "message_id": { "type": "string", "minLength": 1, "maxLength": 128 }, + "device_id": { "$ref": "#/$defs/device_id" }, + "level": { "$ref": "#/$defs/message_level" }, + "text": { "$ref": "#/$defs/free_text" }, + "occurred_at": { "$ref": "#/$defs/unix_ms_timestamp" } + } + }, + "reliability": { + "type": "object", + "description": "Fields shared by every reliable event body (task.event, message.event). Composed with allOf by those schemas rather than duplicated.", + "required": ["device_id", "device_seq"], + "properties": { + "device_id": { "$ref": "#/$defs/device_id" }, + "device_seq": { "$ref": "#/$defs/device_seq" } + } + } + } +} diff --git a/proto/realtime/envelope.schema.json b/proto/realtime/envelope.schema.json new file mode 100644 index 0000000..481815c --- /dev/null +++ b/proto/realtime/envelope.schema.json @@ -0,0 +1,40 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/envelope.schema.json", + "title": "Sitrep realtime protocol - generic envelope", + "description": "Shape shared by every frame of the Sitrep realtime protocol. Every message type schema under messages/ composes this with `allOf` and narrows `type` to a single const and `body` to a type-specific shape. This generic form is used for a first-pass dispatch validation (see tools/validate.js) before the type-specific schema is applied. The envelope's top level is STRICT: exactly the four fields type/id/ts/body, and a receiver MUST reject an envelope carrying any other top-level field with error{code: malformed} - this strictness is permanent and will never be relaxed by a future minor version (it is the guard that keeps credentials and other out-of-band data out of the envelope). Runtime tolerance for unknown fields (SPEC.md section 15) applies only INSIDE `body`. The literal text frames `ping` and `pong` (see SPEC.md `Transport and heartbeat`) are NOT envelopes and are not described by this schema.", + "type": "object", + "required": ["type", "id", "ts", "body"], + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "description": "Dotted message type name. Receivers MUST ignore an envelope whose type they do not recognize (forward compatibility with additive future types) rather than raising `malformed`.", + "enum": [ + "hello", + "resume", + "snapshot", + "delta", + "ack", + "task.event", + "message.event", + "metric.frame", + "config.event", + "subscribe", + "unsubscribe", + "interest.renew", + "command", + "error" + ] + }, + "id": { "$ref": "common.schema.json#/$defs/envelope_id" }, + "ts": { + "$ref": "common.schema.json#/$defs/unix_ms_timestamp", + "description": "When the sender constructed/sent this envelope. Distinct from any domain timestamp (e.g. task.event.body.occurred_at) carried inside `body`." + }, + "body": { + "type": "object", + "description": "Type-specific payload. Never carries authentication credentials of any kind (see SPEC.md `Authentication boundary`)." + } + } +} diff --git a/proto/realtime/fixtures/invalid/ack-neither-acked-nor-in-reply-to.json b/proto/realtime/fixtures/invalid/ack-neither-acked-nor-in-reply-to.json new file mode 100644 index 0000000..d347e73 --- /dev/null +++ b/proto/realtime/fixtures/invalid/ack-neither-acked-nor-in-reply-to.json @@ -0,0 +1,6 @@ +{ + "type": "ack", + "id": "01H9Z0X8K3F0T3B0S0V0W0X2F0", + "ts": 1752480010500, + "body": {} +} diff --git a/proto/realtime/fixtures/invalid/command-pause-with-automation-id.json b/proto/realtime/fixtures/invalid/command-pause-with-automation-id.json new file mode 100644 index 0000000..5bc7c02 --- /dev/null +++ b/proto/realtime/fixtures/invalid/command-pause-with-automation-id.json @@ -0,0 +1,14 @@ +{ + "type": "command", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3H0", + "ts": 1752480020000, + "body": { + "command_id": "cmd-pause-extra-field", + "origin": "viewer", + "issued_by_device_id": "iphone-quintin-01", + "action": "pause", + "task_id": "run-2f9a", + "automation_id": "auto-disk-watch", + "ttl_ms": 30000 + } +} diff --git a/proto/realtime/fixtures/invalid/command-throttle-with-task-id.json b/proto/realtime/fixtures/invalid/command-throttle-with-task-id.json new file mode 100644 index 0000000..c66319b --- /dev/null +++ b/proto/realtime/fixtures/invalid/command-throttle-with-task-id.json @@ -0,0 +1,12 @@ +{ + "type": "command", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3H1", + "ts": 1752480060000, + "body": { + "command_id": "cmd-throttle-extra-field", + "origin": "server", + "action": "throttle", + "task_id": "run-2f9a", + "ttl_ms": 3600000 + } +} diff --git a/proto/realtime/fixtures/invalid/command-viewer-sends-throttle.json b/proto/realtime/fixtures/invalid/command-viewer-sends-throttle.json new file mode 100644 index 0000000..460f6e6 --- /dev/null +++ b/proto/realtime/fixtures/invalid/command-viewer-sends-throttle.json @@ -0,0 +1,12 @@ +{ + "type": "command", + "id": "01H9Z0X8K3F0T3B0S0V0W0X2D0", + "ts": 1752480020000, + "body": { + "command_id": "cmd-should-not-work", + "origin": "viewer", + "issued_by_device_id": "iphone-quintin-01", + "action": "throttle", + "ttl_ms": 30000 + } +} diff --git a/proto/realtime/fixtures/invalid/envelope-carries-credential-field.json b/proto/realtime/fixtures/invalid/envelope-carries-credential-field.json new file mode 100644 index 0000000..b1252fe --- /dev/null +++ b/proto/realtime/fixtures/invalid/envelope-carries-credential-field.json @@ -0,0 +1,12 @@ +{ + "type": "hello", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3D0", + "ts": 1752480000000, + "token": "st2_ab12cd_deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef", + "body": { + "stage": "offer", + "device_id": "mac-quintin-01", + "role": "source", + "protocol_versions": [1] + } +} diff --git a/proto/realtime/fixtures/invalid/error-missing-retryable-fatal.json b/proto/realtime/fixtures/invalid/error-missing-retryable-fatal.json new file mode 100644 index 0000000..e2591a7 --- /dev/null +++ b/proto/realtime/fixtures/invalid/error-missing-retryable-fatal.json @@ -0,0 +1,9 @@ +{ + "type": "error", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3C0", + "ts": 1752480070000, + "body": { + "code": "rate_limited", + "message": "metric.frame rate exceeded 10/s on this connection" + } +} diff --git a/proto/realtime/fixtures/invalid/hello-offer-empty-protocol-versions.json b/proto/realtime/fixtures/invalid/hello-offer-empty-protocol-versions.json new file mode 100644 index 0000000..5fa7ffd --- /dev/null +++ b/proto/realtime/fixtures/invalid/hello-offer-empty-protocol-versions.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "01H9Z0X8K3F0T3B0S0V0W0X2E0", + "ts": 1752480000000, + "body": { + "stage": "offer", + "device_id": "mac-quintin-01", + "role": "source", + "protocol_versions": [] + } +} diff --git a/proto/realtime/fixtures/invalid/message-event-timestamp-in-seconds.json b/proto/realtime/fixtures/invalid/message-event-timestamp-in-seconds.json new file mode 100644 index 0000000..86bfcec --- /dev/null +++ b/proto/realtime/fixtures/invalid/message-event-timestamp-in-seconds.json @@ -0,0 +1,13 @@ +{ + "type": "message.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X2B0", + "ts": 1752480006100, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 56, + "message_id": "mac-quintin-01:56", + "level": "info", + "text": "Archive 3 of 5 uploaded", + "occurred_at": 1752480006 + } +} diff --git a/proto/realtime/fixtures/invalid/metric-frame-oversized-value.json b/proto/realtime/fixtures/invalid/metric-frame-oversized-value.json new file mode 100644 index 0000000..e99126b --- /dev/null +++ b/proto/realtime/fixtures/invalid/metric-frame-oversized-value.json @@ -0,0 +1,15 @@ +{ + "type": "metric.frame", + "id": "01H9Z0X8K3F0T3B0S0V0W0X2C0", + "ts": 1752480009900, + "body": { + "device_id": "mac-quintin-01", + "metrics": [ + { + "metric_id": "cpu.load", + "value": "999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999", + "ts": 1752480009800 + } + ] + } +} diff --git a/proto/realtime/fixtures/invalid/resume-negative-revision.json b/proto/realtime/fixtures/invalid/resume-negative-revision.json new file mode 100644 index 0000000..d861695 --- /dev/null +++ b/proto/realtime/fixtures/invalid/resume-negative-revision.json @@ -0,0 +1,8 @@ +{ + "type": "resume", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3B0", + "ts": 1752480010000, + "body": { + "last_revision": -1 + } +} diff --git a/proto/realtime/fixtures/invalid/role-client-hello-accept.json b/proto/realtime/fixtures/invalid/role-client-hello-accept.json new file mode 100644 index 0000000..8dd4833 --- /dev/null +++ b/proto/realtime/fixtures/invalid/role-client-hello-accept.json @@ -0,0 +1,14 @@ +{ + "sender_role": "viewer", + "frame": { + "type": "hello", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3G0", + "ts": 1752480000010, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess-forged", + "heartbeat_interval_ms": 20000 + } + } +} diff --git a/proto/realtime/fixtures/invalid/role-viewer-command-origin-server.json b/proto/realtime/fixtures/invalid/role-viewer-command-origin-server.json new file mode 100644 index 0000000..9558fbf --- /dev/null +++ b/proto/realtime/fixtures/invalid/role-viewer-command-origin-server.json @@ -0,0 +1,14 @@ +{ + "sender_role": "viewer", + "frame": { + "type": "command", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3F0", + "ts": 1752480020000, + "body": { + "command_id": "cmd-forged-throttle", + "origin": "server", + "action": "throttle", + "ttl_ms": 3600000 + } + } +} diff --git a/proto/realtime/fixtures/invalid/subscribe-unknown-topic.json b/proto/realtime/fixtures/invalid/subscribe-unknown-topic.json new file mode 100644 index 0000000..26d9e4a --- /dev/null +++ b/proto/realtime/fixtures/invalid/subscribe-unknown-topic.json @@ -0,0 +1,8 @@ +{ + "type": "subscribe", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3A0", + "ts": 1752480010550, + "body": { + "topics": ["task", "automation"] + } +} diff --git a/proto/realtime/fixtures/invalid/task-event-missing-device-seq.json b/proto/realtime/fixtures/invalid/task-event-missing-device-seq.json new file mode 100644 index 0000000..23b6c3f --- /dev/null +++ b/proto/realtime/fixtures/invalid/task-event-missing-device-seq.json @@ -0,0 +1,12 @@ +{ + "type": "task.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X2A0", + "ts": 1752480004100, + "body": { + "device_id": "mac-quintin-01", + "task_id": "run-2f9a", + "kind": "progress", + "occurred_at": 1752480004000, + "percent": 40 + } +} diff --git a/proto/realtime/fixtures/invalid/task-event-progress-missing-percent.json b/proto/realtime/fixtures/invalid/task-event-progress-missing-percent.json new file mode 100644 index 0000000..0859b34 --- /dev/null +++ b/proto/realtime/fixtures/invalid/task-event-progress-missing-percent.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X3E0", + "ts": 1752480004100, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 55, + "task_id": "run-2f9a", + "kind": "progress", + "occurred_at": 1752480004000, + "step": "uploading archives" + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/01-source-hello-offer.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/01-source-hello-offer.json new file mode 100644 index 0000000..f81ca37 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/01-source-hello-offer.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "arr-env-01", + "ts": 1752481000000, + "body": { + "stage": "offer", + "device_id": "mac-quintin-01", + "role": "source", + "protocol_versions": [1] + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/02-server-hello-accept.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/02-server-hello-accept.json new file mode 100644 index 0000000..5962f8c --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/02-server-hello-accept.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "arr-env-02", + "ts": 1752481000010, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess-arr-1", + "heartbeat_interval_ms": 20000 + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/03-source-task-event-seq10.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/03-source-task-event-seq10.json new file mode 100644 index 0000000..a7cfe4b --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/03-source-task-event-seq10.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "arr-env-03", + "ts": 1752481001000, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 10, + "task_id": "run-77c1", + "kind": "started", + "occurred_at": 1752481000900, + "title": "Encode video" + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/04-source-task-event-seq11.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/04-source-task-event-seq11.json new file mode 100644 index 0000000..24dffd8 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/04-source-task-event-seq11.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "arr-env-04", + "ts": 1752481001500, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 11, + "task_id": "run-77c1", + "kind": "progress", + "occurred_at": 1752481001400, + "percent": 15 + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/05-server-ack-seq10.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/05-server-ack-seq10.json new file mode 100644 index 0000000..b161a27 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/05-server-ack-seq10.json @@ -0,0 +1,8 @@ +{ + "type": "ack", + "id": "arr-env-05", + "ts": 1752481001100, + "body": { + "acked": [{ "device_id": "mac-quintin-01", "device_seq": 10 }] + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/06-source-hello-offer-reconnect.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/06-source-hello-offer-reconnect.json new file mode 100644 index 0000000..ff60aa8 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/06-source-hello-offer-reconnect.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "arr-env-06", + "ts": 1752481010000, + "body": { + "stage": "offer", + "device_id": "mac-quintin-01", + "role": "source", + "protocol_versions": [1] + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/07-server-hello-accept-reconnect.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/07-server-hello-accept-reconnect.json new file mode 100644 index 0000000..5e1b7df --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/07-server-hello-accept-reconnect.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "arr-env-07", + "ts": 1752481010010, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess-arr-2", + "heartbeat_interval_ms": 20000 + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/08-source-task-event-seq11-resend.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/08-source-task-event-seq11-resend.json new file mode 100644 index 0000000..8db88d2 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/08-source-task-event-seq11-resend.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "arr-env-08", + "ts": 1752481010200, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 11, + "task_id": "run-77c1", + "kind": "progress", + "occurred_at": 1752481001400, + "percent": 15 + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/09-server-ack-seq11.json b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/09-server-ack-seq11.json new file mode 100644 index 0000000..42d888d --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/09-server-ack-seq11.json @@ -0,0 +1,8 @@ +{ + "type": "ack", + "id": "arr-env-09", + "ts": 1752481010250, + "body": { + "acked": [{ "device_id": "mac-quintin-01", "device_seq": 11 }] + } +} diff --git a/proto/realtime/fixtures/scenarios/agent-reconnect-replay/README.md b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/README.md new file mode 100644 index 0000000..813c0e9 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/agent-reconnect-replay/README.md @@ -0,0 +1,36 @@ +# Scenario: agent-reconnect-replay + +A source device (`mac-quintin-01`) loses its connection while it still has an +unacknowledged reliable event queued, reconnects, and replays it. This is the +"Agent 重连重放" flow required by SPEC.md's reliable-events section. + +## Message sequence + +| # | file | sender | note | +|---|------|--------|------| +| 1 | `01-source-hello-offer.json` | source | first envelope on the connection | +| 2 | `02-server-hello-accept.json` | server | negotiates protocol_version 1 | +| 3 | `03-source-task-event-seq10.json` | source | `device_seq: 10`, task started | +| 4 | `04-source-task-event-seq11.json` | source | `device_seq: 11`, progress 15% | +| 5 | `05-server-ack-seq10.json` | server | acks seq 10 only | +| — | *(connection drops here)* | | the ack for seq 11 never arrives at the source, whether because the server never sent it or because it was in flight when the socket died - the source cannot tell which, and does not need to | +| 6 | `06-source-hello-offer-reconnect.json` | source | fresh connection, hello is first again | +| 7 | `07-server-hello-accept-reconnect.json` | server | new `session_id`, same `protocol_version` | +| 8 | `08-source-task-event-seq11-resend.json` | source | **same `device_seq: 11`**, identical domain content, but a **new envelope `id`** (`arr-env-08`, not `arr-env-04`) | +| 9 | `09-server-ack-seq11.json` | server | acks seq 11 | + +## Expected behavior + +- The source's resend queue is keyed by its own unacked `device_seq` values, + not by connection or envelope id. On reconnect it replays every reliable + event at or after the oldest unacked `device_seq` (here, just 11) - it does + NOT need a `resume` message of its own; `resume` is a viewer-side message. +- The server deduplicates on `(device_id, device_seq)`. If the server had, in + fact, already durably applied `device_seq: 11` before the drop (i.e. only + its ack was lost), reapplying it on step 8 is a no-op for `space_revision` + (it is NOT incremented a second time) and step 9's ack is simply resent. + A conformant server implementation cannot tell steps 3-4 apart from a + first-time delivery vs. a replay by looking at the envelope alone, and it + does not need to: idempotent apply-by-`device_seq` makes both cases safe. +- Envelope `id` is intentionally different between step 4 and step 8: it is a + per-transmission identifier, not part of the reliable-event identity. diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/01-viewer-hello-offer.json b/proto/realtime/fixtures/scenarios/client-resume-delta/01-viewer-hello-offer.json new file mode 100644 index 0000000..be90a00 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/01-viewer-hello-offer.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "crd-env-01", + "ts": 1752482000000, + "body": { + "stage": "offer", + "device_id": "iphone-quintin-01", + "role": "viewer", + "protocol_versions": [1] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/02-server-hello-accept.json b/proto/realtime/fixtures/scenarios/client-resume-delta/02-server-hello-accept.json new file mode 100644 index 0000000..00cf25b --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/02-server-hello-accept.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "crd-env-02", + "ts": 1752482000010, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess-crd-1", + "heartbeat_interval_ms": 20000 + } +} diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/03-viewer-subscribe.json b/proto/realtime/fixtures/scenarios/client-resume-delta/03-viewer-subscribe.json new file mode 100644 index 0000000..c71ab18 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/03-viewer-subscribe.json @@ -0,0 +1,8 @@ +{ + "type": "subscribe", + "id": "crd-env-03", + "ts": 1752482000015, + "body": { + "topics": ["task", "metric", "message"] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/04-server-ack-subscribe.json b/proto/realtime/fixtures/scenarios/client-resume-delta/04-server-ack-subscribe.json new file mode 100644 index 0000000..50ad890 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/04-server-ack-subscribe.json @@ -0,0 +1,11 @@ +{ + "type": "ack", + "id": "crd-env-04", + "ts": 1752482000018, + "body": { + "in_reply_to": "crd-env-03", + "lease": { + "expires_at": 1752482045018 + } + } +} diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/05-viewer-resume.json b/proto/realtime/fixtures/scenarios/client-resume-delta/05-viewer-resume.json new file mode 100644 index 0000000..7fe8191 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/05-viewer-resume.json @@ -0,0 +1,8 @@ +{ + "type": "resume", + "id": "crd-env-05", + "ts": 1752482000020, + "body": { + "last_revision": 126 + } +} diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/06-server-delta-catchup.json b/proto/realtime/fixtures/scenarios/client-resume-delta/06-server-delta-catchup.json new file mode 100644 index 0000000..06c1e88 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/06-server-delta-catchup.json @@ -0,0 +1,34 @@ +{ + "type": "delta", + "id": "crd-env-06", + "ts": 1752482000030, + "body": { + "from_revision": 126, + "to_revision": 128, + "events": [ + { + "event_type": "task.event", + "event": { + "device_id": "mac-quintin-01", + "device_seq": 55, + "task_id": "run-2f9a", + "kind": "progress", + "occurred_at": 1752481999000, + "percent": 40, + "step": "uploading archives" + } + }, + { + "event_type": "message.event", + "event": { + "device_id": "mac-quintin-01", + "device_seq": 56, + "message_id": "mac-quintin-01:56", + "level": "info", + "text": "Archive 3 of 5 uploaded", + "occurred_at": 1752482000000 + } + } + ] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/07-server-delta-live.json b/proto/realtime/fixtures/scenarios/client-resume-delta/07-server-delta-live.json new file mode 100644 index 0000000..7afe754 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/07-server-delta-live.json @@ -0,0 +1,23 @@ +{ + "type": "delta", + "id": "crd-env-07", + "ts": 1752482005000, + "body": { + "from_revision": 128, + "to_revision": 129, + "events": [ + { + "event_type": "task.event", + "event": { + "device_id": "mac-quintin-01", + "device_seq": 57, + "task_id": "run-2f9a", + "kind": "progress", + "occurred_at": 1752482004900, + "percent": 60, + "step": "uploading archives" + } + } + ] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-resume-delta/README.md b/proto/realtime/fixtures/scenarios/client-resume-delta/README.md new file mode 100644 index 0000000..8891a40 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-resume-delta/README.md @@ -0,0 +1,47 @@ +# Scenario: client-resume-delta + +A viewer device (`iphone-quintin-01`) connects holding recent state, follows +the mandatory viewer connection sequence **hello -> subscribe -> resume**, +catches up with an incremental `delta`, and then receives a live +single-event `delta` - the only carrier of reliable events toward a viewer. +This is the "客户端 resume" flow of SPEC.md §6.3, in the case where the gap +is small enough for the server to serve from its retained event history. + +## Message sequence + +| # | file | sender | note | +|---|------|--------|------| +| 1 | `01-viewer-hello-offer.json` | viewer | role: viewer | +| 2 | `02-server-hello-accept.json` | server | viewer sends nothing else until this arrives | +| 3 | `03-viewer-subscribe.json` | viewer | establishes the device's interest lease BEFORE resume | +| 4 | `04-server-ack-subscribe.json` | server | lease `expires_at`; the connection is NOT yet delta-eligible | +| 5 | `05-viewer-resume.json` | viewer | `last_revision: 126` - the last revision this viewer fully applied; resume is REQUIRED on every connection | +| 6 | `06-server-delta-catchup.json` | server | the resume reply: `from_revision: 126, to_revision: 128`, the 2 reliable events applied in between; viewer's local revision C becomes 128 and the connection becomes delta-eligible - this is always the FIRST delta-family envelope on the connection | +| 7 | `07-server-delta-live.json` | server | live single-event delta `128 -> 129`, sent because a source just reported progress; `from_revision (128) == C`, so the viewer applies it and sets C = 129 | + +## Expected behavior + +- The full sequence hello -> subscribe -> resume is mandatory on EVERY + connection (including a superseding reconnect), and `resume` is not + optional: the server MUST NOT send any live delta on a connection until + it has sent that connection's resume reply (step 6). Reliable events + applied between steps 4 and 6 are simply included in the reply itself. +- Defensively, the viewer MUST discard any delta received before the + resume reply. This is provably safe: frames on one connection arrive in + order, so such a delta was sent before the server computed the reply, + and the reply's revision already covers its content. A conformant + server never triggers this rule. +- Viewer delta application rules (SPEC.md §6.3), driving every step above: + - `from_revision < C`: discard silently (already covered by the resume + reply or an earlier delta). + - `from_revision == C`: apply, set `C = to_revision`. + - `from_revision > C`: revision gap - the viewer MUST re-send `resume`. +- `to_revision - from_revision` MUST equal `events.length` in every delta + (2 in step 6, 1 in step 7): every reliable event advances `space_revision` + by exactly 1, so the arithmetic is exact, never approximate. +- The server MUST only serve the step-6 catch-up delta when it has retained + every reliable event in `(126, 128]`; compare with + `client-revision-gap-snapshot/`, which shows the snapshot fallback. +- Live reliable events always arrive as single-event deltas like step 7 - + a viewer never receives a bare `task.event`/`message.event`/`config.event` + envelope. diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/01-viewer-hello-offer.json b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/01-viewer-hello-offer.json new file mode 100644 index 0000000..a27ce74 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/01-viewer-hello-offer.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "crg-env-01", + "ts": 1752483000000, + "body": { + "stage": "offer", + "device_id": "ipad-quintin-01", + "role": "viewer", + "protocol_versions": [1] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/02-server-hello-accept.json b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/02-server-hello-accept.json new file mode 100644 index 0000000..ed57583 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/02-server-hello-accept.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "crg-env-02", + "ts": 1752483000010, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess-crg-1", + "heartbeat_interval_ms": 20000 + } +} diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/03-viewer-subscribe.json b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/03-viewer-subscribe.json new file mode 100644 index 0000000..2c5ef76 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/03-viewer-subscribe.json @@ -0,0 +1,8 @@ +{ + "type": "subscribe", + "id": "crg-env-03", + "ts": 1752483000015, + "body": { + "topics": ["task", "metric", "message"] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/04-server-ack-subscribe.json b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/04-server-ack-subscribe.json new file mode 100644 index 0000000..75ba139 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/04-server-ack-subscribe.json @@ -0,0 +1,11 @@ +{ + "type": "ack", + "id": "crg-env-04", + "ts": 1752483000018, + "body": { + "in_reply_to": "crg-env-03", + "lease": { + "expires_at": 1752483045018 + } + } +} diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/05-viewer-resume.json b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/05-viewer-resume.json new file mode 100644 index 0000000..f205b85 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/05-viewer-resume.json @@ -0,0 +1,8 @@ +{ + "type": "resume", + "id": "crg-env-05", + "ts": 1752483000020, + "body": { + "last_revision": 40 + } +} diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/06-server-snapshot-part1.json b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/06-server-snapshot-part1.json new file mode 100644 index 0000000..f888127 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/06-server-snapshot-part1.json @@ -0,0 +1,32 @@ +{ + "type": "snapshot", + "id": "crg-env-06", + "ts": 1752483000030, + "body": { + "revision": 150, + "part": 1, + "final": false, + "tasks": [ + { + "task_id": "run-2f9a", + "device_id": "mac-quintin-01", + "title": "Nightly backup", + "state": "done", + "percent": 100, + "updated_at": 1752482990000 + } + ], + "metrics": [], + "messages": [], + "automations": [ + { + "automation_id": "auto-disk-watch", + "name": "Disk space watch", + "executor_kind": "script", + "schedule": { "kind": "interval", "every_seconds": 300 }, + "state": "active", + "last_run_at": 1752482900000 + } + ] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/07-server-snapshot-part2-final.json b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/07-server-snapshot-part2-final.json new file mode 100644 index 0000000..480407f --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/07-server-snapshot-part2-final.json @@ -0,0 +1,24 @@ +{ + "type": "snapshot", + "id": "crg-env-07", + "ts": 1752483000040, + "body": { + "revision": 150, + "part": 2, + "final": true, + "tasks": [], + "metrics": [ + { "metric_id": "cpu.load", "value": "0.18", "ts": 1752483000000, "min": "0", "max": "8" } + ], + "messages": [ + { + "message_id": "mac-quintin-01:120", + "device_id": "mac-quintin-01", + "level": "info", + "text": "Backup completed for volume Data", + "occurred_at": 1752482990000 + } + ], + "automations": [] + } +} diff --git a/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/README.md b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/README.md new file mode 100644 index 0000000..5187157 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/client-revision-gap-snapshot/README.md @@ -0,0 +1,44 @@ +# Scenario: client-revision-gap-snapshot + +A viewer device (`ipad-quintin-01`) reconnects after being offline long +enough that the server can no longer serve it an incremental catch-up, so +the server falls back to a full `snapshot` - delivered here as **two +chunks** to demonstrate snapshot chunking under the frame-size limit. This +is the fallback half of the "客户端 resume" flow of SPEC.md §6.3. + +## Message sequence + +| # | file | sender | note | +|---|------|--------|------| +| 1 | `01-viewer-hello-offer.json` | viewer | | +| 2 | `02-server-hello-accept.json` | server | | +| 3 | `03-viewer-subscribe.json` | viewer | mandatory order: subscribe before resume | +| 4 | `04-server-ack-subscribe.json` | server | lease granted | +| 5 | `05-viewer-resume.json` | viewer | `last_revision: 40` | +| 6 | `06-server-snapshot-part1.json` | server | `revision: 150, part: 1, final: false` - tasks + automations | +| 7 | `07-server-snapshot-part2-final.json` | server | `revision: 150, part: 2, final: true` - metrics + messages | + +## Expected behavior + +- The server MUST send `snapshot` instead of `delta` whenever it cannot + produce the complete, gapless event range `(last_revision, current]` - + here because its retained event history does not go back as far as + revision 40. This is a normal outcome, not an error, and `resume` always + produces exactly one reply (a snapshot counts as one reply even when + chunked) - never silence. +- Chunking rules (SPEC.md §6.2): every chunk carries the SAME `revision` + (150); `part` counts from 1; chunks arrive consecutively and in order on + the same connection; the receiver concatenates the four arrays across + chunks and applies nothing until the `final: true` chunk arrives. A + single-chunk snapshot would be `part: 1, final: true`. +- `snapshot.metrics` is a best-effort cache and MAY be empty or stale + (e.g. after a server restart); the viewer refreshes metrics from + subsequent `metric.frame` envelopes. +- Contrast with `last_revision` *greater* than the server's current + `space_revision` (the viewer claims a revision the space never reached, + e.g. after the space was rebuilt): that case is answered with + `error{code: revision_unavailable}`, and the viewer retries with + `last_revision: 0`. This scenario does not exercise it. +- After applying the aggregated snapshot, the viewer's local revision + becomes 150 regardless of what it was before; a live delta with + `from_revision: 150` is then applicable. diff --git a/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/01-viewer-hello-offer-conn1.json b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/01-viewer-hello-offer-conn1.json new file mode 100644 index 0000000..d0d0e52 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/01-viewer-hello-offer-conn1.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "dcs-env-01", + "ts": 1752486000000, + "body": { + "stage": "offer", + "device_id": "iphone-quintin-01", + "role": "viewer", + "protocol_versions": [1] + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/02-server-hello-accept-conn1.json b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/02-server-hello-accept-conn1.json new file mode 100644 index 0000000..802a7b2 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/02-server-hello-accept-conn1.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "dcs-env-02", + "ts": 1752486000010, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess-dcs-1", + "heartbeat_interval_ms": 20000 + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/03-viewer-hello-offer-conn2.json b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/03-viewer-hello-offer-conn2.json new file mode 100644 index 0000000..1fadd7e --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/03-viewer-hello-offer-conn2.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "dcs-env-03", + "ts": 1752486030000, + "body": { + "stage": "offer", + "device_id": "iphone-quintin-01", + "role": "viewer", + "protocol_versions": [1] + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/04-server-hello-accept-conn2.json b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/04-server-hello-accept-conn2.json new file mode 100644 index 0000000..e10cc1b --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/04-server-hello-accept-conn2.json @@ -0,0 +1,11 @@ +{ + "type": "hello", + "id": "dcs-env-04", + "ts": 1752486030010, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess-dcs-2", + "heartbeat_interval_ms": 20000 + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/05-server-error-superseded-conn1.json b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/05-server-error-superseded-conn1.json new file mode 100644 index 0000000..625a9ca --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/05-server-error-superseded-conn1.json @@ -0,0 +1,11 @@ +{ + "type": "error", + "id": "dcs-env-05", + "ts": 1752486030020, + "body": { + "code": "superseded", + "message": "this device opened a newer connection; closing this one", + "retryable": false, + "fatal": true + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/README.md b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/README.md new file mode 100644 index 0000000..0698185 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-connection-supersede/README.md @@ -0,0 +1,44 @@ +# Scenario: duplicate-connection-supersede + +A device that already holds an open connection opens a second one (e.g. the +app relaunched before the old socket's death was detected, or the network +path changed). The server accepts the new connection and closes the old one +with `error{code: superseded}`. See SPEC.md §9.4. + +## Message sequence + +| # | file | sender | connection | note | +|---|------|--------|------------|------| +| 1 | `01-viewer-hello-offer-conn1.json` | viewer | C1 | device `iphone-quintin-01` | +| 2 | `02-server-hello-accept-conn1.json` | server | C1 | C1 is live | +| 3 | `03-viewer-hello-offer-conn2.json` | viewer | C2 | SAME device_id, new connection | +| 4 | `04-server-hello-accept-conn2.json` | server | C2 | server MUST accept the new connection | +| 5 | `05-server-error-superseded-conn1.json` | server | C1 | then closes C1 | + +## Expected behavior + +- On completing `hello` for a device that already has a live connection, + the server MUST accept the NEW connection and close the OLD one, sending + `error{code: superseded, retryable: false, fatal: true}` on the old + connection first. `retryable: false` means: do not retry on that (dying) + connection - the device's newer connection is unaffected and simply + continues. +- From the moment C2's hello completes, ALL server-to-device traffic - + `ack`s for the device's reliable events, targeted `command`s, resume + replies, live deltas - routes to C2, the device's most recent connection, + never to C1. +- Interest leases are held per DEVICE, not per connection (SPEC.md §7): + the supersession neither ends nor double-counts the device's lease, and + MUST NOT trigger a throttle/resume_rate emission (the count is + unchanged). A viewer briefly holding two open connections still + contributes exactly one lease to the space's count. +- Lease survival does NOT carry delta eligibility to C2: after step 4 the + viewer MUST still send `subscribe` and then `resume` on C2 before the + server sends it any delta, and its local revision C is re-established + from C2's resume reply. Nothing (C, in-flight snapshot chunks, pending + replies) is inherited from C1. +- A client that receives `superseded` on an old connection it still + considered live SHOULD treat it as confirmation that its newer connection + won; if it receives `superseded` unexpectedly (it did not itself open a + new connection), its credential may be in use elsewhere and it SHOULD + surface that to the user rather than silently reconnecting in a loop. diff --git a/proto/realtime/fixtures/scenarios/duplicate-device-seq/01-source-task-event-seq20-first.json b/proto/realtime/fixtures/scenarios/duplicate-device-seq/01-source-task-event-seq20-first.json new file mode 100644 index 0000000..16e8419 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-device-seq/01-source-task-event-seq20-first.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "dds-env-01", + "ts": 1752485000000, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 20, + "task_id": "run-9b31", + "kind": "done", + "occurred_at": 1752484999900, + "message": "Export finished" + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-device-seq/02-server-ack-seq20-first.json b/proto/realtime/fixtures/scenarios/duplicate-device-seq/02-server-ack-seq20-first.json new file mode 100644 index 0000000..0512b4d --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-device-seq/02-server-ack-seq20-first.json @@ -0,0 +1,8 @@ +{ + "type": "ack", + "id": "dds-env-02", + "ts": 1752485000050, + "body": { + "acked": [{ "device_id": "mac-quintin-01", "device_seq": 20 }] + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-device-seq/03-source-task-event-seq20-retry.json b/proto/realtime/fixtures/scenarios/duplicate-device-seq/03-source-task-event-seq20-retry.json new file mode 100644 index 0000000..e963655 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-device-seq/03-source-task-event-seq20-retry.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "dds-env-03", + "ts": 1752485003000, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 20, + "task_id": "run-9b31", + "kind": "done", + "occurred_at": 1752484999900, + "message": "Export finished" + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-device-seq/04-server-ack-seq20-retry.json b/proto/realtime/fixtures/scenarios/duplicate-device-seq/04-server-ack-seq20-retry.json new file mode 100644 index 0000000..c201447 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-device-seq/04-server-ack-seq20-retry.json @@ -0,0 +1,8 @@ +{ + "type": "ack", + "id": "dds-env-04", + "ts": 1752485003040, + "body": { + "acked": [{ "device_id": "mac-quintin-01", "device_seq": 20 }] + } +} diff --git a/proto/realtime/fixtures/scenarios/duplicate-device-seq/README.md b/proto/realtime/fixtures/scenarios/duplicate-device-seq/README.md new file mode 100644 index 0000000..cd4d6cc --- /dev/null +++ b/proto/realtime/fixtures/scenarios/duplicate-device-seq/README.md @@ -0,0 +1,32 @@ +# Scenario: duplicate-device-seq + +A source device sends a reliable event, its `ack` is lost in transit (not +absence of connectivity - just packet/frame loss), the source's own resend +timeout fires before it ever sees the ack, and it retransmits the identical +event. The server must treat this as a no-op duplicate, not a second event. + +## Message sequence + +| # | file | sender | note | +|---|------|--------|------| +| 1 | `01-source-task-event-seq20-first.json` | source | `device_seq: 20`, task done | +| 2 | `02-server-ack-seq20-first.json` | server | acks seq 20 - **this ack is lost in transit and never reaches the source** | +| 3 | `03-source-task-event-seq20-retry.json` | source | same `device_id`/`device_seq: 20`/domain content as step 1, but a new envelope `id` (`dds-env-03`) and a later `ts`, sent because the source's local resend timeout elapsed without seeing an ack | +| 4 | `04-server-ack-seq20-retry.json` | server | acks seq 20 again | + +## Expected behavior + +- The server's deduplication key is `(device_id, device_seq)` = + `(mac-quintin-01, 20)`, identical between steps 1 and 3. The server MUST + detect step 3 as an already-applied event and MUST NOT apply it again: + `space_revision` is incremented exactly once for this event, not twice. +- The server MUST still send the ack in step 4 even though it performed no + new work, so the source can stop retrying. Acking a duplicate is always + safe (idempotent) and never rejected as an error. +- This is the normal, expected behavior for at-least-once delivery. It is not + an error condition and produces no `error` envelope - contrast with the + invalid fixtures under `fixtures/invalid/`, which are all cases a + conformant server MUST reject. +- Envelope `id` and `ts` differ between steps 1 and 3, exactly as they do + between the two transmissions in `agent-reconnect-replay/`; only + `(device_id, device_seq)` inside `body` carries reliable-event identity. diff --git a/proto/realtime/fixtures/scenarios/interest-lease-expiry/01-viewer-subscribe.json b/proto/realtime/fixtures/scenarios/interest-lease-expiry/01-viewer-subscribe.json new file mode 100644 index 0000000..078d2f2 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/interest-lease-expiry/01-viewer-subscribe.json @@ -0,0 +1,8 @@ +{ + "type": "subscribe", + "id": "ile-env-01", + "ts": 1752484000000, + "body": { + "topics": ["task", "metric", "message"] + } +} diff --git a/proto/realtime/fixtures/scenarios/interest-lease-expiry/02-server-ack-subscribe.json b/proto/realtime/fixtures/scenarios/interest-lease-expiry/02-server-ack-subscribe.json new file mode 100644 index 0000000..41a114e --- /dev/null +++ b/proto/realtime/fixtures/scenarios/interest-lease-expiry/02-server-ack-subscribe.json @@ -0,0 +1,11 @@ +{ + "type": "ack", + "id": "ile-env-02", + "ts": 1752484000010, + "body": { + "in_reply_to": "ile-env-01", + "lease": { + "expires_at": 1752484045010 + } + } +} diff --git a/proto/realtime/fixtures/scenarios/interest-lease-expiry/03-server-command-throttle.json b/proto/realtime/fixtures/scenarios/interest-lease-expiry/03-server-command-throttle.json new file mode 100644 index 0000000..581f6f1 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/interest-lease-expiry/03-server-command-throttle.json @@ -0,0 +1,11 @@ +{ + "type": "command", + "id": "ile-env-03", + "ts": 1752484045010, + "body": { + "command_id": "cmd-throttle-ile-1", + "origin": "server", + "action": "throttle", + "ttl_ms": 3600000 + } +} diff --git a/proto/realtime/fixtures/scenarios/interest-lease-expiry/04-viewer2-subscribe.json b/proto/realtime/fixtures/scenarios/interest-lease-expiry/04-viewer2-subscribe.json new file mode 100644 index 0000000..ae43019 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/interest-lease-expiry/04-viewer2-subscribe.json @@ -0,0 +1,8 @@ +{ + "type": "subscribe", + "id": "ile-env-04", + "ts": 1752484200000, + "body": { + "topics": ["task", "metric", "message"] + } +} diff --git a/proto/realtime/fixtures/scenarios/interest-lease-expiry/05-server-ack-subscribe-viewer2.json b/proto/realtime/fixtures/scenarios/interest-lease-expiry/05-server-ack-subscribe-viewer2.json new file mode 100644 index 0000000..63189d6 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/interest-lease-expiry/05-server-ack-subscribe-viewer2.json @@ -0,0 +1,11 @@ +{ + "type": "ack", + "id": "ile-env-05", + "ts": 1752484200010, + "body": { + "in_reply_to": "ile-env-04", + "lease": { + "expires_at": 1752484245010 + } + } +} diff --git a/proto/realtime/fixtures/scenarios/interest-lease-expiry/06-server-command-resume-rate.json b/proto/realtime/fixtures/scenarios/interest-lease-expiry/06-server-command-resume-rate.json new file mode 100644 index 0000000..f176177 --- /dev/null +++ b/proto/realtime/fixtures/scenarios/interest-lease-expiry/06-server-command-resume-rate.json @@ -0,0 +1,11 @@ +{ + "type": "command", + "id": "ile-env-06", + "ts": 1752484200020, + "body": { + "command_id": "cmd-resume-rate-ile-1", + "origin": "server", + "action": "resume_rate", + "ttl_ms": 3600000 + } +} diff --git a/proto/realtime/fixtures/scenarios/interest-lease-expiry/README.md b/proto/realtime/fixtures/scenarios/interest-lease-expiry/README.md new file mode 100644 index 0000000..0ee01ce --- /dev/null +++ b/proto/realtime/fixtures/scenarios/interest-lease-expiry/README.md @@ -0,0 +1,54 @@ +# Scenario: interest-lease-expiry + +A viewer subscribes, its lease lapses without renewal while it is the space's +only interested viewer, and the server tells the source device(s) to +throttle. Later a second viewer subscribes and the server tells the source(s) +to resume full rate. This is the "服务端通知 source 降频" behavior required by +SPEC.md's interest-lease section (§7). + +## Message sequence + +| # | file | sender | note | +|---|------|--------|------| +| 1 | `01-viewer-subscribe.json` | viewer | first subscriber in the space (after its hello completes) | +| 2 | `02-server-ack-subscribe.json` | server | `lease.expires_at = ts + 45000` | +| — | *(45 seconds pass; the viewer sends no `interest.renew`)* | | no envelope marks the lapse itself; the server MAY detect it lazily, on the next state change it evaluates | +| 3 | `03-server-command-throttle.json` | server | `command{origin: server, action: throttle}`, sent to every connected source device in the space, because the space's lease count just went from 1 to 0 | +| 4 | `04-viewer2-subscribe.json` | a different viewer | | +| 5 | `05-server-ack-subscribe-viewer2.json` | server | fresh lease for the new viewer's device | +| 6 | `06-server-command-resume-rate.json` | server | `command{origin: server, action: resume_rate}`, sent because the lease count just went from 0 to 1 | + +## Expected behavior + +- Leases are held **per device**, not per connection, and are fully + decoupled from connection lifetime: a lease ends ONLY when `expires_at` + passes without renewal or when the device sends an explicit + `unsubscribe`. A connection drop does not end the device's lease - if + the first viewer had merely disconnected at second 10 and reconnected at + second 20, its lease (and the space's count) would have been unaffected. + This also lets a restarted server rebuild the count from persisted lease + records. +- Expiry evaluation MAY be lazy: the server is not required to fire a timer + at the exact `expires_at` instant; it may notice the lapse the next time + it evaluates the space's interest state. The 1→0 `throttle` notification + follows the (possibly lazy) detection. +- The server tracks one interest-lease-count per space (one entry per + device holding an unexpired lease). The `throttle`/`resume_rate` + notifications fire ONLY on the 1→0 and 0→1 transitions of that count, + never on every subscribe/unsubscribe/expiry individually - two devices + subscribed at once produce no command traffic when one of them + unsubscribes, since the count only drops from 2 to 1. A device with two + open connections still counts once (see `duplicate-connection-supersede/`). +- `throttle` and `resume_rate` commands always have `origin: "server"` and + can never be sent by a client: the server MUST reject a client-sent + command with origin "server" as unauthorized (see + `fixtures/invalid/role-viewer-command-origin-server.json`) and the schema + additionally rejects a viewer-origin envelope carrying these actions (see + `fixtures/invalid/command-viewer-sends-throttle.json`). +- `target_device_id` is omitted on both command envelopes here: they + broadcast to every source device currently connected to the space, not + just one. +- `throttle` is advisory: a source SHOULD reduce its `metric.frame` cadence + while throttled, but MUST continue sending `task.event` for lifecycle + transitions (started/done/failed) and all `message.event`s at normal + priority regardless of throttle state. diff --git a/proto/realtime/fixtures/valid/ack-events.json b/proto/realtime/fixtures/valid/ack-events.json new file mode 100644 index 0000000..0315076 --- /dev/null +++ b/proto/realtime/fixtures/valid/ack-events.json @@ -0,0 +1,11 @@ +{ + "type": "ack", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y5", + "ts": 1752480010500, + "body": { + "acked": [ + { "device_id": "mac-quintin-01", "device_seq": 55 }, + { "device_id": "mac-quintin-01", "device_seq": 56 } + ] + } +} diff --git a/proto/realtime/fixtures/valid/ack-subscribe.json b/proto/realtime/fixtures/valid/ack-subscribe.json new file mode 100644 index 0000000..0118ef4 --- /dev/null +++ b/proto/realtime/fixtures/valid/ack-subscribe.json @@ -0,0 +1,11 @@ +{ + "type": "ack", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y6", + "ts": 1752480010600, + "body": { + "in_reply_to": "01H9Z0X8K3F0T3B0S0V0W0X0Y7", + "lease": { + "expires_at": 1752480055600 + } + } +} diff --git a/proto/realtime/fixtures/valid/command-server-throttle.json b/proto/realtime/fixtures/valid/command-server-throttle.json new file mode 100644 index 0000000..02c756c --- /dev/null +++ b/proto/realtime/fixtures/valid/command-server-throttle.json @@ -0,0 +1,11 @@ +{ + "type": "command", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1E1", + "ts": 1752480060000, + "body": { + "command_id": "cmd-throttle-9c1", + "origin": "server", + "action": "throttle", + "ttl_ms": 3600000 + } +} diff --git a/proto/realtime/fixtures/valid/command-viewer-pause.json b/proto/realtime/fixtures/valid/command-viewer-pause.json new file mode 100644 index 0000000..73f5f21 --- /dev/null +++ b/proto/realtime/fixtures/valid/command-viewer-pause.json @@ -0,0 +1,15 @@ +{ + "type": "command", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1E0", + "ts": 1752480020000, + "body": { + "command_id": "cmd-8f2c1a", + "origin": "viewer", + "issued_by_device_id": "iphone-quintin-01", + "action": "pause", + "task_id": "run-2f9a", + "target_device_id": "mac-quintin-01", + "ttl_ms": 30000, + "params": {} + } +} diff --git a/proto/realtime/fixtures/valid/config-event.json b/proto/realtime/fixtures/valid/config-event.json new file mode 100644 index 0000000..f6d79f5 --- /dev/null +++ b/proto/realtime/fixtures/valid/config-event.json @@ -0,0 +1,18 @@ +{ + "type": "config.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1G0", + "ts": 1752480008100, + "body": { + "kind": "automation.upserted", + "automation_id": "auto-disk-watch", + "automation": { + "automation_id": "auto-disk-watch", + "name": "Disk space watch", + "executor_kind": "script", + "schedule": { "kind": "interval", "every_seconds": 300 }, + "state": "active", + "last_run_at": 1752480000000 + }, + "occurred_at": 1752480008000 + } +} diff --git a/proto/realtime/fixtures/valid/delta-empty.json b/proto/realtime/fixtures/valid/delta-empty.json new file mode 100644 index 0000000..c50c2f8 --- /dev/null +++ b/proto/realtime/fixtures/valid/delta-empty.json @@ -0,0 +1,10 @@ +{ + "type": "delta", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1G1", + "ts": 1752480010450, + "body": { + "from_revision": 128, + "to_revision": 128, + "events": [] + } +} diff --git a/proto/realtime/fixtures/valid/delta.json b/proto/realtime/fixtures/valid/delta.json new file mode 100644 index 0000000..4ed97ac --- /dev/null +++ b/proto/realtime/fixtures/valid/delta.json @@ -0,0 +1,49 @@ +{ + "type": "delta", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y4", + "ts": 1752480010400, + "body": { + "from_revision": 125, + "to_revision": 128, + "events": [ + { + "event_type": "task.event", + "event": { + "device_id": "mac-quintin-01", + "device_seq": 55, + "task_id": "run-2f9a", + "kind": "progress", + "occurred_at": 1752480004000, + "percent": 40, + "step": "uploading archives" + } + }, + { + "event_type": "message.event", + "event": { + "device_id": "mac-quintin-01", + "device_seq": 56, + "message_id": "mac-quintin-01:56", + "level": "info", + "text": "Archive 3 of 5 uploaded", + "occurred_at": 1752480006000 + } + }, + { + "event_type": "config.event", + "event": { + "kind": "automation.upserted", + "automation_id": "auto-disk-watch", + "automation": { + "automation_id": "auto-disk-watch", + "name": "Disk space watch", + "executor_kind": "script", + "schedule": { "kind": "interval", "every_seconds": 300 }, + "state": "active" + }, + "occurred_at": 1752480008000 + } + } + ] + } +} diff --git a/proto/realtime/fixtures/valid/error-superseded.json b/proto/realtime/fixtures/valid/error-superseded.json new file mode 100644 index 0000000..48abdd1 --- /dev/null +++ b/proto/realtime/fixtures/valid/error-superseded.json @@ -0,0 +1,11 @@ +{ + "type": "error", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1G2", + "ts": 1752480080000, + "body": { + "code": "superseded", + "message": "this device opened a newer connection; closing this one", + "retryable": false, + "fatal": true + } +} diff --git a/proto/realtime/fixtures/valid/error.json b/proto/realtime/fixtures/valid/error.json new file mode 100644 index 0000000..9cc20d8 --- /dev/null +++ b/proto/realtime/fixtures/valid/error.json @@ -0,0 +1,12 @@ +{ + "type": "error", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1F0", + "ts": 1752480070000, + "body": { + "code": "rate_limited", + "message": "metric.frame rate exceeded 10/s on this connection", + "in_reply_to": "01H9Z0X8K3F0T3B0S0V0W0X1C0", + "retryable": true, + "fatal": false + } +} diff --git a/proto/realtime/fixtures/valid/hello-accept.json b/proto/realtime/fixtures/valid/hello-accept.json new file mode 100644 index 0000000..240f6c3 --- /dev/null +++ b/proto/realtime/fixtures/valid/hello-accept.json @@ -0,0 +1,12 @@ +{ + "type": "hello", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y1", + "ts": 1752480000050, + "body": { + "stage": "accept", + "protocol_version": 1, + "session_id": "sess_7e1a9c", + "capabilities": [], + "heartbeat_interval_ms": 20000 + } +} diff --git a/proto/realtime/fixtures/valid/hello-offer.json b/proto/realtime/fixtures/valid/hello-offer.json new file mode 100644 index 0000000..acd63ad --- /dev/null +++ b/proto/realtime/fixtures/valid/hello-offer.json @@ -0,0 +1,12 @@ +{ + "type": "hello", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y0", + "ts": 1752480000000, + "body": { + "stage": "offer", + "device_id": "mac-quintin-01", + "role": "source", + "protocol_versions": [1], + "capabilities": [] + } +} diff --git a/proto/realtime/fixtures/valid/interest-renew.json b/proto/realtime/fixtures/valid/interest-renew.json new file mode 100644 index 0000000..27f0096 --- /dev/null +++ b/proto/realtime/fixtures/valid/interest-renew.json @@ -0,0 +1,8 @@ +{ + "type": "interest.renew", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1D1", + "ts": 1752480040000, + "body": { + "topics": ["task", "metric", "message"] + } +} diff --git a/proto/realtime/fixtures/valid/message-event.json b/proto/realtime/fixtures/valid/message-event.json new file mode 100644 index 0000000..17ead1e --- /dev/null +++ b/proto/realtime/fixtures/valid/message-event.json @@ -0,0 +1,13 @@ +{ + "type": "message.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1B0", + "ts": 1752480006100, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 56, + "message_id": "mac-quintin-01:56", + "level": "info", + "text": "Archive 3 of 5 uploaded", + "occurred_at": 1752480006000 + } +} diff --git a/proto/realtime/fixtures/valid/metric-frame.json b/proto/realtime/fixtures/valid/metric-frame.json new file mode 100644 index 0000000..92872b1 --- /dev/null +++ b/proto/realtime/fixtures/valid/metric-frame.json @@ -0,0 +1,12 @@ +{ + "type": "metric.frame", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1C0", + "ts": 1752480009900, + "body": { + "device_id": "mac-quintin-01", + "metrics": [ + { "metric_id": "cpu.load", "value": "0.62", "label": "CPU load", "ts": 1752480009800 }, + { "metric_id": "mem.used_gb", "value": "18.4", "ts": 1752480009800 } + ] + } +} diff --git a/proto/realtime/fixtures/valid/resume.json b/proto/realtime/fixtures/valid/resume.json new file mode 100644 index 0000000..d344df0 --- /dev/null +++ b/proto/realtime/fixtures/valid/resume.json @@ -0,0 +1,8 @@ +{ + "type": "resume", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y2", + "ts": 1752480010000, + "body": { + "last_revision": 128 + } +} diff --git a/proto/realtime/fixtures/valid/snapshot.json b/proto/realtime/fixtures/valid/snapshot.json new file mode 100644 index 0000000..8a5164b --- /dev/null +++ b/proto/realtime/fixtures/valid/snapshot.json @@ -0,0 +1,52 @@ +{ + "type": "snapshot", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y3", + "ts": 1752480010200, + "body": { + "revision": 128, + "part": 1, + "final": true, + "tasks": [ + { + "task_id": "run-2f9a", + "device_id": "mac-quintin-01", + "title": "Nightly backup", + "state": "running", + "percent": 42, + "step": "uploading archives", + "updated_at": 1752480005000, + "display": { "icon": "arrow.up.circle", "tint": "blue", "template": "progress" } + } + ], + "metrics": [ + { + "metric_id": "cpu.load", + "value": "0.62", + "label": "CPU load", + "ts": 1752480009800, + "min": "0", + "max": "8", + "alert_above": "6" + } + ], + "messages": [ + { + "message_id": "mac-quintin-01:41", + "device_id": "mac-quintin-01", + "level": "info", + "text": "Backup completed for volume Data", + "occurred_at": 1752479990000 + } + ], + "automations": [ + { + "automation_id": "auto-disk-watch", + "name": "Disk space watch", + "executor_kind": "script", + "schedule": { "kind": "interval", "every_seconds": 300 }, + "state": "active", + "last_run_at": 1752480000000 + } + ] + } +} diff --git a/proto/realtime/fixtures/valid/subscribe.json b/proto/realtime/fixtures/valid/subscribe.json new file mode 100644 index 0000000..7eadaff --- /dev/null +++ b/proto/realtime/fixtures/valid/subscribe.json @@ -0,0 +1,8 @@ +{ + "type": "subscribe", + "id": "01H9Z0X8K3F0T3B0S0V0W0X0Y7", + "ts": 1752480010550, + "body": { + "topics": ["task", "metric", "message"] + } +} diff --git a/proto/realtime/fixtures/valid/task-event-done.json b/proto/realtime/fixtures/valid/task-event-done.json new file mode 100644 index 0000000..2095ff7 --- /dev/null +++ b/proto/realtime/fixtures/valid/task-event-done.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1A2", + "ts": 1752480012000, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 58, + "task_id": "run-2f9a", + "kind": "done", + "occurred_at": 1752480011900, + "message": "Backup completed for volume Data" + } +} diff --git a/proto/realtime/fixtures/valid/task-event-failed.json b/proto/realtime/fixtures/valid/task-event-failed.json new file mode 100644 index 0000000..81826fb --- /dev/null +++ b/proto/realtime/fixtures/valid/task-event-failed.json @@ -0,0 +1,13 @@ +{ + "type": "task.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1A3", + "ts": 1752480013000, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 59, + "task_id": "run-2f9b", + "kind": "failed", + "occurred_at": 1752480012900, + "message": "Upload failed: network unreachable" + } +} diff --git a/proto/realtime/fixtures/valid/task-event-progress.json b/proto/realtime/fixtures/valid/task-event-progress.json new file mode 100644 index 0000000..7403ead --- /dev/null +++ b/proto/realtime/fixtures/valid/task-event-progress.json @@ -0,0 +1,14 @@ +{ + "type": "task.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1A1", + "ts": 1752480004100, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 55, + "task_id": "run-2f9a", + "kind": "progress", + "occurred_at": 1752480004000, + "percent": 40, + "step": "uploading archives" + } +} diff --git a/proto/realtime/fixtures/valid/task-event-started.json b/proto/realtime/fixtures/valid/task-event-started.json new file mode 100644 index 0000000..8060dd8 --- /dev/null +++ b/proto/realtime/fixtures/valid/task-event-started.json @@ -0,0 +1,14 @@ +{ + "type": "task.event", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1A0", + "ts": 1752480003000, + "body": { + "device_id": "mac-quintin-01", + "device_seq": 53, + "task_id": "run-2f9a", + "kind": "started", + "occurred_at": 1752480002900, + "title": "Nightly backup", + "display": { "icon": "arrow.up.circle", "tint": "blue", "template": "progress" } + } +} diff --git a/proto/realtime/fixtures/valid/unsubscribe.json b/proto/realtime/fixtures/valid/unsubscribe.json new file mode 100644 index 0000000..19ada92 --- /dev/null +++ b/proto/realtime/fixtures/valid/unsubscribe.json @@ -0,0 +1,6 @@ +{ + "type": "unsubscribe", + "id": "01H9Z0X8K3F0T3B0S0V0W0X1D0", + "ts": 1752480050000, + "body": {} +} diff --git a/proto/realtime/messages/ack.schema.json b/proto/realtime/messages/ack.schema.json new file mode 100644 index 0000000..09e0329 --- /dev/null +++ b/proto/realtime/messages/ack.schema.json @@ -0,0 +1,55 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/ack.schema.json", + "title": "ack", + "description": "Generic acknowledgment envelope, used for two distinct purposes that MAY be combined in one ack: (1) confirming receipt of one or more reliable events, identified by (device_id, device_seq) pairs, so the sender can prune its resend queue; (2) confirming a single control-plane request (hello is never acked this way; subscribe/unsubscribe/interest.renew/command are), identified by echoing that request's envelope id in `in_reply_to`, optionally carrying a renewed interest lease. At least one of `acked` or `in_reply_to` MUST be present. `acked` is an EXACT enumeration, not cumulative coverage: only the pairs literally listed are confirmed, and acknowledging seq N says nothing about any other seq. Every pair in one ack MUST carry the device_id of the connection's authenticated device (acks are routed to that device's most recent connection); a receiver that finds a pair for any other device MUST treat the envelope as malformed.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "ack" }, + "body": { + "type": "object", + "additionalProperties": false, + "anyOf": [ + { "required": ["acked"] }, + { "required": ["in_reply_to"] } + ], + "properties": { + "acked": { + "type": "array", + "description": "Reliable events being confirmed as durably applied - an exact enumeration (see top-level description); all pairs belong to the receiving connection's authenticated device. Safe to acknowledge the same (device_id, device_seq) more than once (idempotent).", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["device_id", "device_seq"], + "properties": { + "device_id": { "$ref": "../common.schema.json#/$defs/device_id" }, + "device_seq": { "$ref": "../common.schema.json#/$defs/device_seq" } + } + }, + "minItems": 1 + }, + "in_reply_to": { + "$ref": "../common.schema.json#/$defs/envelope_id", + "description": "The `id` of the subscribe/unsubscribe/interest.renew/command envelope this ack confirms." + }, + "lease": { + "type": "object", + "description": "Present when acking a subscribe or interest.renew: the interest lease the server just granted or extended.", + "additionalProperties": false, + "required": ["expires_at"], + "properties": { + "expires_at": { + "$ref": "../common.schema.json#/$defs/unix_ms_timestamp", + "description": "Absolute deadline by which the viewer must send interest.renew to keep the lease alive. Always between 30000 and 60000 ms after this ack's envelope.ts." + } + } + } + } + } + } + } + ] +} diff --git a/proto/realtime/messages/command.schema.json b/proto/realtime/messages/command.schema.json new file mode 100644 index 0000000..2831bc5 --- /dev/null +++ b/proto/realtime/messages/command.schema.json @@ -0,0 +1,90 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/command.schema.json", + "title": "command", + "description": "Reverse control instruction directed at one or more source devices. Two origins share this envelope type: 'viewer' (a human-initiated pause/resume/stop/run_now, relayed by the server) and 'server' (an interest-lease throttle/resume_rate notification the server generates itself; see SPEC.md 'Interest lease'). A client may only ever send origin 'viewer': the server MUST reject any client-sent command with origin 'server' with error code unauthorized, regardless of action. A viewer MUST set issued_by_device_id to its own authenticated device_id (the server MUST reject a mismatch with error code unauthorized). Per-action field exclusivity: pause/resume/stop require task_id and MUST NOT carry automation_id; run_now requires automation_id and MUST NOT carry task_id; throttle/resume_rate MUST carry neither. Relay rules: when the server relays a viewer command to a source it preserves the ORIGINAL envelope ts and command_id (only the envelope id is fresh), validates ttl at relay time, and the source re-validates ttl against its own local clock allowing +/- 30 s of clock skew. A source device deduplicates by command_id: receiving the same command_id twice MUST NOT execute the action twice. A command received or first actionable after envelope.ts + ttl_ms (with the skew allowance) MUST be dropped without executing it; the device that drops it SHOULD emit error{code: command_expired}.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "command" }, + "body": { + "type": "object", + "additionalProperties": false, + "required": ["command_id", "origin", "action", "ttl_ms"], + "properties": { + "command_id": { + "type": "string", + "description": "Idempotency key chosen by the sender. Preserved verbatim when the server relays the command.", + "minLength": 1, + "maxLength": 64 + }, + "origin": { + "type": "string", + "enum": ["viewer", "server"] + }, + "issued_by_device_id": { + "$ref": "../common.schema.json#/$defs/device_id", + "description": "Required and meaningful only when origin is 'viewer': the authenticated device_id of the viewer that requested this command." + }, + "action": { + "type": "string", + "enum": ["pause", "resume", "stop", "run_now", "throttle", "resume_rate"] + }, + "task_id": { + "$ref": "../common.schema.json#/$defs/task_id", + "description": "The task run this command applies to. Required for pause/resume/stop; aligns with the HTTP control surface (POST /v2/tasks/:id/commands)." + }, + "automation_id": { + "type": "string", + "description": "The automation this command applies to. Required for run_now.", + "minLength": 1, + "maxLength": 128 + }, + "target_device_id": { + "$ref": "../common.schema.json#/$defs/device_id", + "description": "The source device this command applies to. If absent, the command applies to every source device connected in the space (used for throttle/resume_rate broadcasts)." + }, + "ttl_ms": { + "type": "integer", + "description": "How long after envelope.ts this command remains actionable.", + "minimum": 1, + "maximum": 86400000 + }, + "params": { + "type": "object", + "description": "Reserved extension point; protocol version 1 defines NO keys here. Unlike other objects in this protocol, additional properties here are explicitly permitted and MUST be ignored by a receiver that does not recognize them.", + "default": {} + } + }, + "allOf": [ + { + "if": { "properties": { "action": { "enum": ["pause", "resume", "stop"] } } }, + "then": { + "required": ["origin", "issued_by_device_id", "task_id"], + "properties": { "origin": { "const": "viewer" } }, + "not": { "required": ["automation_id"] } + } + }, + { + "if": { "properties": { "action": { "const": "run_now" } } }, + "then": { + "required": ["origin", "issued_by_device_id", "automation_id"], + "properties": { "origin": { "const": "viewer" } }, + "not": { "required": ["task_id"] } + } + }, + { + "if": { "properties": { "action": { "enum": ["throttle", "resume_rate"] } } }, + "then": { + "properties": { "origin": { "const": "server" } }, + "not": { "anyOf": [{ "required": ["task_id"] }, { "required": ["automation_id"] }] } + } + } + ] + } + } + } + ] +} diff --git a/proto/realtime/messages/config.event.schema.json b/proto/realtime/messages/config.event.schema.json new file mode 100644 index 0000000..5683f53 --- /dev/null +++ b/proto/realtime/messages/config.event.schema.json @@ -0,0 +1,38 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/config.event.schema.json", + "title": "config.event", + "description": "Server-minted reliable event reporting a configuration change (automation created/edited/removed). Configuration changes are initiated through the HTTP control plane, so the server is the sole writer: no client ever sends config.event, and its body carries NO device_id and NO device_seq - its identity IS the space_revision it occupies, so it does not participate in (device_id, device_seq) deduplication and is never acked by anyone. It is persisted and increments space_revision by 1 like every reliable event. Direction is server -> viewer only; in protocol version 1 the server delivers it to viewers inside delta.body.events (the single-event live delta, like every other reliable event), so a standalone config.event envelope does not normally appear on the wire - this schema exists to define the body shape referenced by delta.schema.json and to reserve the type.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "config.event" }, + "body": { "$ref": "#/$defs/body" } + } + } + ], + "$defs": { + "body": { + "type": "object", + "additionalProperties": false, + "required": ["kind", "automation_id", "occurred_at"], + "properties": { + "kind": { + "type": "string", + "description": "'automation.upserted' covers both creation and any edit (the full new state is carried); 'automation.removed' deletes the automation.", + "enum": ["automation.upserted", "automation.removed"] + }, + "automation_id": { "type": "string", "minLength": 1, "maxLength": 128 }, + "automation": { + "$ref": "../common.schema.json#/$defs/automation_state", + "description": "Required when kind is 'automation.upserted'; absent when kind is 'automation.removed'." + }, + "occurred_at": { "$ref": "../common.schema.json#/$defs/unix_ms_timestamp" } + }, + "if": { "properties": { "kind": { "const": "automation.upserted" } } }, + "then": { "required": ["automation"] } + } + } +} diff --git a/proto/realtime/messages/delta.schema.json b/proto/realtime/messages/delta.schema.json new file mode 100644 index 0000000..062fb73 --- /dev/null +++ b/proto/realtime/messages/delta.schema.json @@ -0,0 +1,59 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/delta.schema.json", + "title": "delta", + "description": "The ONLY carrier of reliable events toward a viewer. Sent by the server in two situations: (1) as the catch-up reply to `resume`, covering the range (from_revision, to_revision]; when the range would exceed the frame limit the server splits it on event boundaries into several consecutive deltas whose from/to chain exactly (d1.to_revision == d2.from_revision); (2) live: each time the server applies one reliable event it sends a single-event delta{from_revision: R-1, to_revision: R, events:[e]} to every delta-eligible viewer connection - one that holds an active interest lease AND has completed hello -> subscribe -> resume including having been sent its resume reply. The server MUST NOT send any live delta on a connection before that connection's resume reply (the reply is always the first delta-family envelope on a connection), and a viewer MUST discard any delta received before the reply - provably safe because in-order delivery means the reply covers it. `to_revision` MUST equal `from_revision` plus events.length. A viewer tracking local revision C applies a delta only when from_revision == C (then sets C = to_revision); it MUST silently discard a delta with from_revision < C (already covered by its resume reply or an earlier delta) and MUST treat from_revision > C as a gap requiring a fresh `resume`. An empty delta (from_revision == to_revision, events: []) is the explicit 'you are already current' reply to resume when last_revision equals the space's current revision. See SPEC.md section 6.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "delta" }, + "body": { + "type": "object", + "additionalProperties": false, + "required": ["from_revision", "to_revision", "events"], + "properties": { + "from_revision": { "$ref": "../common.schema.json#/$defs/space_revision" }, + "to_revision": { "$ref": "../common.schema.json#/$defs/space_revision" }, + "events": { + "type": "array", + "description": "Ordered oldest-first. Each item is the body of one task.event, message.event, or config.event, tagged with its own `event_type` so the receiver can apply it exactly as it would the corresponding uplink event. MAY be empty (the you-are-current reply).", + "items": { + "oneOf": [ + { + "type": "object", + "required": ["event_type", "event"], + "additionalProperties": false, + "properties": { + "event_type": { "const": "task.event" }, + "event": { "$ref": "task.event.schema.json#/$defs/body" } + } + }, + { + "type": "object", + "required": ["event_type", "event"], + "additionalProperties": false, + "properties": { + "event_type": { "const": "message.event" }, + "event": { "$ref": "message.event.schema.json#/$defs/body" } + } + }, + { + "type": "object", + "required": ["event_type", "event"], + "additionalProperties": false, + "properties": { + "event_type": { "const": "config.event" }, + "event": { "$ref": "config.event.schema.json#/$defs/body" } + } + } + ] + } + } + } + } + } + } + ] +} diff --git a/proto/realtime/messages/error.schema.json b/proto/realtime/messages/error.schema.json new file mode 100644 index 0000000..fe36876 --- /dev/null +++ b/proto/realtime/messages/error.schema.json @@ -0,0 +1,57 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/error.schema.json", + "title": "error", + "description": "Diagnostic envelope. MAY be sent by any authenticated party (server, source, or viewer) about any other envelope; not restricted to server-to-client. `retryable` and `fatal` are authoritative and self-describing: a receiver MUST NOT need the SPEC.md error-code table to decide how to react, though that table documents the fixed values every code in this version always carries. See SPEC.md 'Error codes' for the full enumeration.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "error" }, + "body": { + "type": "object", + "additionalProperties": false, + "required": ["code", "message", "retryable", "fatal"], + "properties": { + "code": { + "type": "string", + "enum": [ + "version_unsupported", + "hello_required", + "unauthenticated", + "unauthorized", + "malformed", + "rate_limited", + "frame_too_large", + "batch_too_large", + "revision_unavailable", + "command_expired", + "sequence_gap", + "superseded", + "internal_error" + ] + }, + "message": { + "type": "string", + "description": "Human-readable, not machine-parsed. Implementations MUST branch only on `code`. For internal_error, this text MUST NOT contain stack traces, internal file paths, or other implementation details.", + "maxLength": 500 + }, + "in_reply_to": { + "$ref": "../common.schema.json#/$defs/envelope_id", + "description": "The id of the envelope this error concerns, when one is identifiable." + }, + "retryable": { + "type": "boolean", + "description": "True if resending the same logical request (typically with a new envelope id) can plausibly succeed without any other change. When fatal is also true, 'retry' means: reconnect first, then retry on the new connection." + }, + "fatal": { + "type": "boolean", + "description": "True if the sender will close the connection immediately after this envelope." + } + } + } + } + } + ] +} diff --git a/proto/realtime/messages/hello.schema.json b/proto/realtime/messages/hello.schema.json new file mode 100644 index 0000000..01650e1 --- /dev/null +++ b/proto/realtime/messages/hello.schema.json @@ -0,0 +1,74 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/hello.schema.json", + "title": "hello", + "description": "MUST be the first envelope sent on a new connection, in both directions. A connecting device sends a `stage: offer` hello (client -> server); the server responds with a `stage: accept` hello (server -> client) carrying the negotiated version. `stage: accept` belongs exclusively to the server: a client-sent accept, at any point in the connection's life, is a handshake violation the server MUST answer with error{code: hello_required} and a connection close. The client MUST NOT send ANY other envelope until it has received and validated the accept; the server MUST answer any frame received before it has sent its accept with error{code: hello_required} and close the connection. Recommended timeouts: the server SHOULD close a connection that sends no offer within 10 s of establishment; the client SHOULD close and reconnect if no accept arrives within 10 s of sending its offer.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "hello" }, + "body": { + "oneOf": [ + { "$ref": "#/$defs/offer" }, + { "$ref": "#/$defs/accept" } + ] + } + } + } + ], + "$defs": { + "offer": { + "type": "object", + "description": "Sent by the connecting device (source or viewer).", + "additionalProperties": false, + "required": ["stage", "device_id", "role", "protocol_versions"], + "properties": { + "stage": { "const": "offer" }, + "device_id": { "$ref": "../common.schema.json#/$defs/device_id" }, + "role": { "$ref": "../common.schema.json#/$defs/device_role" }, + "protocol_versions": { + "type": "array", + "description": "Every protocol major version this device can speak, ascending, no duplicates.", + "items": { "$ref": "../common.schema.json#/$defs/protocol_version" }, + "minItems": 1, + "uniqueItems": true + }, + "capabilities": { + "type": "array", + "description": "Optional feature flags this device understands. Unknown flags MUST be ignored by the peer.", + "items": { "type": "string", "maxLength": 64 }, + "default": [] + } + } + }, + "accept": { + "type": "object", + "description": "Sent by the server once it has chosen a protocol_version from the offer's intersection with its own supported set. If the intersection is empty the server MUST instead send `error` with code `version_unsupported` and close the connection; it MUST NOT send this envelope in that case.", + "additionalProperties": false, + "required": ["stage", "protocol_version", "session_id", "heartbeat_interval_ms"], + "properties": { + "stage": { "const": "accept" }, + "protocol_version": { "$ref": "../common.schema.json#/$defs/protocol_version" }, + "session_id": { + "type": "string", + "description": "Server-assigned identifier for this connection instance, for logging/observability only. Not a credential and not required for resume (see `resume`, which is keyed by space revision instead).", + "minLength": 1, + "maxLength": 64 + }, + "capabilities": { + "type": "array", + "items": { "type": "string", "maxLength": 64 }, + "default": [] + }, + "heartbeat_interval_ms": { + "type": "integer", + "description": "Interval at which the server will emit a bare `ping` text frame on this connection. Informative; the peer MAY also send its own `ping`s.", + "minimum": 1000, + "maximum": 300000 + } + } + } + } +} diff --git a/proto/realtime/messages/interest.renew.schema.json b/proto/realtime/messages/interest.renew.schema.json new file mode 100644 index 0000000..956c5b4 --- /dev/null +++ b/proto/realtime/messages/interest.renew.schema.json @@ -0,0 +1,16 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/interest.renew.schema.json", + "title": "interest.renew", + "description": "Sent by a viewer before its current lease's expires_at to keep it alive. The server responds with `ack{lease:{expires_at}}` carrying a freshly computed deadline (again 30000-60000 ms out). interest.renew ALWAYS replaces the lease's topics wholesale with the topics in this request (omitted/empty meaning all topics) - it never merges with or preserves the previous topics. If the device holds no active lease when interest.renew arrives (e.g. it already lapsed, or none was ever established), the server MUST treat this identically to a fresh `subscribe` with the given topics, rather than rejecting it: this avoids a race between the viewer's local renewal timer and server-side expiry.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "interest.renew" }, + "body": { "$ref": "subscribe.schema.json#/$defs/body" } + } + } + ] +} diff --git a/proto/realtime/messages/message.event.schema.json b/proto/realtime/messages/message.event.schema.json new file mode 100644 index 0000000..6e327c1 --- /dev/null +++ b/proto/realtime/messages/message.event.schema.json @@ -0,0 +1,38 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/message.event.schema.json", + "title": "message.event", + "description": "Reliable event reporting a single point-in-time message. UPLINK ONLY: sent by a source device to the server, never forwarded to viewers as a message.event envelope - the server re-emits it to subscribed viewers wrapped in a single-event `delta` (see SPEC.md section 6). Persisted, increments space_revision by 1, and MAY also trigger a push notification. Shares the same per-(device, space) device_seq counter as task.event (see common.schema.json#/$defs/device_seq) and uses the same (device_id, device_seq) deduplication and resend rules. The server MUST reject a message.event whose body.device_id does not match the connection's authenticated device identity (error code unauthorized).", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "message.event" }, + "body": { "$ref": "#/$defs/body" } + } + } + ], + "$defs": { + "body": { + "type": "object", + "allOf": [{ "$ref": "../common.schema.json#/$defs/reliability" }], + "additionalProperties": false, + "required": ["device_id", "device_seq", "message_id", "level", "text", "occurred_at"], + "properties": { + "device_id": { "$ref": "../common.schema.json#/$defs/device_id" }, + "device_seq": { "$ref": "../common.schema.json#/$defs/device_seq" }, + "message_id": { + "type": "string", + "description": "Unique id for this message. SHOULD be deterministically derivable from (device_id, device_seq) but MUST be sent explicitly rather than re-derived by the receiver, so the wire format stays self-contained.", + "minLength": 1, + "maxLength": 128 + }, + "level": { "$ref": "../common.schema.json#/$defs/message_level" }, + "text": { "$ref": "../common.schema.json#/$defs/free_text" }, + "occurred_at": { "$ref": "../common.schema.json#/$defs/unix_ms_timestamp" }, + "automation_id": { "type": "string", "maxLength": 128 } + } + } + } +} diff --git a/proto/realtime/messages/metric.frame.schema.json b/proto/realtime/messages/metric.frame.schema.json new file mode 100644 index 0000000..95309af --- /dev/null +++ b/proto/realtime/messages/metric.frame.schema.json @@ -0,0 +1,29 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/metric.frame.schema.json", + "title": "metric.frame", + "description": "Best-effort batch of metric samples from a source device. NOT persisted, NEVER acked, and MAY be dropped by the transport or the server under load; a lost frame is superseded by the next one and is never retransmitted. A sender MUST merge multiple pending updates to the same metric_id into a single sample (the latest one) before emitting a frame, and MUST NOT emit a frame containing a given metric_id more often than once per 500ms. A connection MUST NOT emit more than 10 metric.frame envelopes per second; the server MAY reject excess frames with error code rate_limited. A receiver MUST discard any sample whose ts is less than or equal to the last-applied ts it has already accepted for that metric_id (out-of-order/duplicate drop), independent of envelope arrival order. The server MUST reject a metric.frame whose body.device_id does not match the connection's authenticated device identity (error code unauthorized).", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "metric.frame" }, + "body": { + "type": "object", + "additionalProperties": false, + "required": ["device_id", "metrics"], + "properties": { + "device_id": { "$ref": "../common.schema.json#/$defs/device_id" }, + "metrics": { + "type": "array", + "items": { "$ref": "../common.schema.json#/$defs/metric_sample" }, + "minItems": 1, + "maxItems": 64 + } + } + } + } + } + ] +} diff --git a/proto/realtime/messages/resume.schema.json b/proto/realtime/messages/resume.schema.json new file mode 100644 index 0000000..dc36789 --- /dev/null +++ b/proto/realtime/messages/resume.schema.json @@ -0,0 +1,26 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/resume.schema.json", + "title": "resume", + "description": "Sent by a viewer device to ask the server to bring it up to date. The mandatory connection sequence for a viewer is hello -> subscribe -> resume, and resume is REQUIRED on every connection (including a superseding reconnect): a connection becomes delta-eligible only once the server has sent its resume reply, and the server sends no live delta on it before then. resume MUST NOT be sent before this connection's subscribe has been sent. resume MUST produce exactly one reply, never silence: last_revision 0, or a gap the server cannot serve incrementally -> a `snapshot` (possibly split into chunks, possibly empty); last_revision equal to the space's current revision (and > 0) -> an empty `delta{from_revision: N, to_revision: N, events: []}` as the explicit you-are-current signal; otherwise -> one or more chained `delta`s covering the full range. last_revision greater than the space's current revision -> error{code: revision_unavailable}. See SPEC.md section 6.3. A source device does not send resume for its own outbound queue: it simply re-sends any unacked task.event/message.event it holds (see SPEC.md 'Agent reconnect replay flow').", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "resume" }, + "body": { + "type": "object", + "additionalProperties": false, + "required": ["last_revision"], + "properties": { + "last_revision": { + "$ref": "../common.schema.json#/$defs/space_revision", + "description": "The space_revision this viewer last fully applied. 0 means the viewer holds no prior state and always yields a snapshot (which MAY be empty on a fresh space)." + } + } + } + } + } + ] +} diff --git a/proto/realtime/messages/snapshot.schema.json b/proto/realtime/messages/snapshot.schema.json new file mode 100644 index 0000000..526ca71 --- /dev/null +++ b/proto/realtime/messages/snapshot.schema.json @@ -0,0 +1,50 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/snapshot.schema.json", + "title": "snapshot", + "description": "Full current state of a space at a given revision, sent unicast to a single viewer, ONLY as a reply to `resume` (there is no other trigger). A snapshot MAY be split into multiple chunks to respect the frame size limit: every chunk carries the SAME `revision`, `part` numbers the chunks consecutively from 1, and `final: true` marks the last chunk. The receiver concatenates the tasks/metrics/messages/automations arrays across chunks and MUST NOT apply anything until the final chunk has arrived; a single-chunk snapshot is `part: 1, final: true`. While a chunked snapshot is in flight the server MUST defer every other outbound envelope on that connection (including control-plane acks) until after the final chunk - only ping/pong heartbeat text frames may interleave; a viewer receiving any other envelope between chunks treats it as a malformed sequence and MAY close and reconnect. A snapshot MAY be empty (all four arrays empty) - e.g. the reply to resume{last_revision: 0} on a fresh space. The `metrics` array is a best-effort convenience cache: metric samples are not persisted and not part of space_revision accounting, so this array MAY be empty or stale (e.g. after a server restart) even when metrics are actively flowing; a viewer MUST rely on subsequent metric.frame envelopes to refresh them. Never persisted itself and never increments space_revision.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "snapshot" }, + "body": { + "type": "object", + "additionalProperties": false, + "required": ["revision", "part", "final", "tasks", "metrics", "messages", "automations"], + "properties": { + "revision": { "$ref": "../common.schema.json#/$defs/space_revision" }, + "part": { + "type": "integer", + "description": "1-based chunk index. All chunks of one snapshot share the same revision and arrive consecutively, in order, on the same connection.", + "minimum": 1 + }, + "final": { + "type": "boolean", + "description": "True on the last chunk. The receiver applies the aggregated snapshot only after receiving the chunk with final: true." + }, + "tasks": { + "type": "array", + "items": { "$ref": "../common.schema.json#/$defs/task_state" } + }, + "metrics": { + "type": "array", + "description": "Best-effort, possibly empty or stale (see top-level description); refreshed by subsequent metric.frame envelopes.", + "items": { "$ref": "../common.schema.json#/$defs/metric_sample" } + }, + "messages": { + "type": "array", + "description": "Recent messages only. The server keeps a bounded window (the most recent N; implementations SHOULD document their N, e.g. 200); truncation to this window is normative behavior, see SPEC.md section 6.4.", + "items": { "$ref": "../common.schema.json#/$defs/message_record" } + }, + "automations": { + "type": "array", + "items": { "$ref": "../common.schema.json#/$defs/automation_state" } + } + } + } + } + } + ] +} diff --git a/proto/realtime/messages/subscribe.schema.json b/proto/realtime/messages/subscribe.schema.json new file mode 100644 index 0000000..792651b --- /dev/null +++ b/proto/realtime/messages/subscribe.schema.json @@ -0,0 +1,31 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/subscribe.schema.json", + "title": "subscribe", + "description": "Sent by a viewer to declare interest in a space, establishing (or replacing) that DEVICE's interest lease - leases are held per device, not per connection, survive a disconnect, and end only by expires_at passing or an explicit unsubscribe (see SPEC.md section 7). In the mandatory viewer connection sequence (hello -> subscribe -> resume) subscribe comes immediately after hello completes and before resume. The server MUST choose a lease duration between 30000 and 60000 ms inclusive and communicate the absolute deadline via `ack.body.lease.expires_at`. If this creates the space's first active lease (0 -> 1 transition of the space's lease count), the server MUST notify every connected source device with `command{origin: server, action: resume_rate}`. subscribe when the device already holds a lease replaces it entirely, including a full replacement of its topics.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "subscribe" }, + "body": { "$ref": "#/$defs/body" } + } + } + ], + "$defs": { + "body": { + "type": "object", + "additionalProperties": false, + "properties": { + "topics": { + "type": "array", + "description": "Interest declaration. Omitted or empty means all topics. In protocol version 1 only the 'metric' topic has filtering effect (omitting it stops metric.frame forwarding to this device); 'task' and 'message' are advisory - the server MUST always deliver every reliable single-event delta to an active lease holder regardless of topics, because withholding any reliable event would break the viewer's revision continuity and force a resume loop.", + "items": { "type": "string", "enum": ["task", "metric", "message"] }, + "uniqueItems": true, + "default": [] + } + } + } + } +} diff --git a/proto/realtime/messages/task.event.schema.json b/proto/realtime/messages/task.event.schema.json new file mode 100644 index 0000000..f8f4b06 --- /dev/null +++ b/proto/realtime/messages/task.event.schema.json @@ -0,0 +1,49 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/task.event.schema.json", + "title": "task.event", + "description": "Reliable event reporting a task lifecycle transition or progress update. UPLINK ONLY: sent by a source device to the server, never forwarded to viewers as a task.event envelope - the server re-emits it to subscribed viewers wrapped in a single-event `delta`, which carries the revision the viewer needs for alignment (see SPEC.md section 6). Persisted and increments space_revision by 1. Deduplicated by the server on (device_id, device_seq); the sender keeps retransmitting an unacked task.event (with its original device_seq, in a fresh envelope with a new envelope id) until it receives an `ack` that covers that device_seq. The server MUST reject a task.event whose body.device_id does not match the connection's authenticated device identity (error code unauthorized).", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "task.event" }, + "body": { "$ref": "#/$defs/body" } + } + } + ], + "$defs": { + "body": { + "type": "object", + "allOf": [{ "$ref": "../common.schema.json#/$defs/reliability" }], + "additionalProperties": false, + "required": ["device_id", "device_seq", "task_id", "kind", "occurred_at"], + "properties": { + "device_id": { "$ref": "../common.schema.json#/$defs/device_id" }, + "device_seq": { "$ref": "../common.schema.json#/$defs/device_seq" }, + "task_id": { "$ref": "../common.schema.json#/$defs/task_id" }, + "kind": { "$ref": "../common.schema.json#/$defs/task_kind" }, + "occurred_at": { + "$ref": "../common.schema.json#/$defs/unix_ms_timestamp", + "description": "When this transition actually happened on the source device, independent of envelope.ts (send time)." + }, + "title": { "$ref": "../common.schema.json#/$defs/free_text" }, + "percent": { + "type": "integer", + "description": "Present when kind is 'progress'.", + "minimum": 0, + "maximum": 100 + }, + "step": { "$ref": "../common.schema.json#/$defs/free_text" }, + "message": { + "$ref": "../common.schema.json#/$defs/free_text", + "description": "Present when kind is 'done' or 'failed'." + }, + "display": { "$ref": "../common.schema.json#/$defs/display_hints" } + }, + "if": { "properties": { "kind": { "const": "progress" } } }, + "then": { "required": ["percent"] } + } + } +} diff --git a/proto/realtime/messages/unsubscribe.schema.json b/proto/realtime/messages/unsubscribe.schema.json new file mode 100644 index 0000000..37aa4ce --- /dev/null +++ b/proto/realtime/messages/unsubscribe.schema.json @@ -0,0 +1,20 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schema.sitrep.dev/realtime/v1/messages/unsubscribe.schema.json", + "title": "unsubscribe", + "description": "Sent by a viewer to voluntarily end its device's interest lease before expiry - one of only two ways a lease ends (the other being expires_at passing; a connection drop does NOT end a lease, see SPEC.md section 7). The server MUST release the lease immediately on receipt (it MUST NOT wait for expires_at) and MUST confirm with `ack{in_reply_to: }`, exactly as it confirms subscribe and interest.renew (but with no `lease` object, since none remains). If this removes the space's last active lease (1 -> 0 transition), the server MUST notify every connected source device with `command{origin: server, action: throttle}`.", + "allOf": [ + { "$ref": "../envelope.schema.json" }, + { + "type": "object", + "properties": { + "type": { "const": "unsubscribe" }, + "body": { + "type": "object", + "additionalProperties": false, + "properties": {} + } + } + } + ] +} diff --git a/proto/realtime/tools/package-lock.json b/proto/realtime/tools/package-lock.json new file mode 100644 index 0000000..1685073 --- /dev/null +++ b/proto/realtime/tools/package-lock.json @@ -0,0 +1,68 @@ +{ + "name": "sitrep-realtime-protocol-tools", + "version": "1.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "sitrep-realtime-protocol-tools", + "version": "1.0.0", + "dependencies": { + "ajv": "8.20.0" + } + }, + "node_modules/ajv": { + "version": "8.20.0", + "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz", + "integrity": "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA==", + "license": "MIT", + "dependencies": { + "fast-deep-equal": "^3.1.3", + "fast-uri": "^3.0.1", + "json-schema-traverse": "^1.0.0", + "require-from-string": "^2.0.2" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/epoberezkin" + } + }, + "node_modules/fast-deep-equal": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz", + "integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==", + "license": "MIT" + }, + "node_modules/fast-uri": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.3.tgz", + "integrity": "sha512-i70LwGWUduXqzicKXWshooq+sWL1K3WUU5rKZNG/0i3a1OSoX3HqhH5WbWwTmqWfor4urUakGPiRQcleRZTwOg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fastify" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fastify" + } + ], + "license": "BSD-3-Clause" + }, + "node_modules/json-schema-traverse": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz", + "integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==", + "license": "MIT" + }, + "node_modules/require-from-string": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", + "integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + } + } +} diff --git a/proto/realtime/tools/package.json b/proto/realtime/tools/package.json new file mode 100644 index 0000000..2ae216a --- /dev/null +++ b/proto/realtime/tools/package.json @@ -0,0 +1,13 @@ +{ + "name": "sitrep-realtime-protocol-tools", + "version": "1.0.0", + "private": true, + "description": "Self-contained JSON Schema conformance checker for proto/realtime. Not published; not depended on by server/, daemon/, or apple/.", + "type": "module", + "scripts": { + "validate": "node validate.js" + }, + "dependencies": { + "ajv": "8.20.0" + } +} diff --git a/proto/realtime/tools/validate.js b/proto/realtime/tools/validate.js new file mode 100644 index 0000000..ff28f2d --- /dev/null +++ b/proto/realtime/tools/validate.js @@ -0,0 +1,289 @@ +#!/usr/bin/env node +// Self-contained conformance checker for proto/realtime. +// +// What it does: +// 1. Loads common.schema.json, envelope.schema.json, and every +// messages/*.schema.json into a single Ajv (draft 2020-12) instance. +// 2. Every file under fixtures/valid/**/*.json (including scenario +// subdirectories under fixtures/scenarios/) MUST validate against the +// generic envelope schema AND against the specific messages/.schema.json +// selected by its own `type` field. Any failure is a script failure. +// 3. Every file under fixtures/invalid/**/*.json MUST be rejected, either +// by one of those two schemas or by the client-authorization matrix +// (SPEC.md section 10.1). A fixture that passes every check is a script +// failure (it means the fixture no longer demonstrates the constraint +// its filename claims to). +// +// Fixture forms: +// - A bare envelope object: schema-checked only. +// - A wrapper {"sender_role": "source"|"viewer", "frame": {...}}: the +// frame is schema-checked AND checked against the authorization matrix +// as if a client with that role had sent it. This form exists for +// constraints that are role-dependent rather than shape-dependent +// (e.g. a viewer sending a command with origin "server", which is a +// perfectly well-formed frame when the server sends it). +// +// Usage: +// npm install (once, to materialize node_modules/ from package-lock.json) +// npm run validate +// or +// node validate.js +// +// Exits 0 with a summary on success, non-zero on any unexpected result. + +import { readFileSync, readdirSync, statSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import path from "node:path"; +import Ajv2020 from "ajv/dist/2020.js"; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const realtimeRoot = path.resolve(here, ".."); +const messagesDir = path.join(realtimeRoot, "messages"); +const fixturesDir = path.join(realtimeRoot, "fixtures"); + +const ajv = new Ajv2020({ + allErrors: true, + strict: true, + // anyOf branches like ack's `{"required": ["acked"]}` intentionally test + // for a property's presence without redeclaring its full shape (that shape + // already lives in the sibling `properties` at the same schema level) - + // this is standard practice, not a schema-authoring mistake, so it is + // exempted from strict mode's usually-helpful "did you forget the shape?" + // check. Every other strict-mode check stays on. + strictRequired: false, +}); + +function loadJson(p) { + return JSON.parse(readFileSync(p, "utf8")); +} + +function walk(dir) { + const out = []; + for (const entry of readdirSync(dir)) { + const full = path.join(dir, entry); + const st = statSync(full); + if (st.isDirectory()) out.push(...walk(full)); + else if (entry.endsWith(".json")) out.push(full); + } + return out; +} + +// --- 1. load schemas --------------------------------------------------- + +ajv.addSchema(loadJson(path.join(realtimeRoot, "common.schema.json"))); +ajv.addSchema(loadJson(path.join(realtimeRoot, "envelope.schema.json"))); + +const messageSchemaIds = new Map(); // envelope `type` value -> $id +for (const file of readdirSync(messagesDir).filter((f) => f.endsWith(".schema.json"))) { + const schema = loadJson(path.join(messagesDir, file)); + ajv.addSchema(schema); + // messages/.schema.json <-> envelope type of the same dotted name + const type = file.replace(/\.schema\.json$/, ""); + messageSchemaIds.set(type, schema.$id); +} + +const envelopeSchema = ajv.getSchema("https://schema.sitrep.dev/realtime/v1/envelope.schema.json"); + +// Client-side authorization matrix, mirroring SPEC.md section 10.1: the set +// of message types each client role may SEND. snapshot/delta/config.event +// are server-only in the client-send direction. +const CLIENT_SEND_ALLOWED = { + source: new Set(["hello", "ack", "task.event", "message.event", "metric.frame", "error"]), + viewer: new Set(["hello", "resume", "subscribe", "unsubscribe", "interest.renew", "command", "error"]), +}; + +function authorizationErrors(role, frame) { + const errors = []; + const allowed = CLIENT_SEND_ALLOWED[role]; + if (!allowed) { + errors.push(`[auth] unknown sender_role: ${JSON.stringify(role)}`); + return errors; + } + if (!allowed.has(frame?.type)) { + errors.push(`[auth] role "${role}" may not send ${JSON.stringify(frame?.type)}`); + } + if (frame?.type === "command" && frame?.body?.origin !== "viewer") { + errors.push(`[auth] a client may only send command with origin "viewer" (got ${JSON.stringify(frame?.body?.origin)})`); + } + if (frame?.type === "hello" && frame?.body?.stage !== "offer") { + errors.push(`[auth] a client may only send hello with stage "offer" (got ${JSON.stringify(frame?.body?.stage)})`); + } + return errors; +} + +function validateFrame(frame) { + const errors = []; + + const envelopeOk = envelopeSchema(frame); + if (!envelopeOk) { + errors.push(...(envelopeSchema.errors ?? []).map((e) => `[envelope] ${e.instancePath} ${e.message}`)); + } + + const type = frame && typeof frame === "object" ? frame.type : undefined; + const schemaId = messageSchemaIds.get(type); + if (!schemaId) { + errors.push(`[type] unknown or missing message type: ${JSON.stringify(type)}`); + return errors; + } + + const specific = ajv.getSchema(schemaId); + const specificOk = specific(frame); + if (!specificOk) { + errors.push(...(specific.errors ?? []).map((e) => `[${type}] ${e.instancePath} ${e.message}`)); + } + + // Semantic invariant JSON Schema cannot express: delta revision arithmetic. + if (type === "delta" && specificOk && envelopeOk) { + const b = frame.body; + if (b.to_revision - b.from_revision !== b.events.length) { + errors.push(`[delta] to_revision - from_revision (${b.to_revision - b.from_revision}) != events.length (${b.events.length})`); + } + } + + return errors; +} + +// --- scenario sequence invariants (SPEC.md sections 5.1, 6.2, 6.3) ------- +// +// Each scenario directory is an ordered message sequence (lexicographic +// filename order). Beyond per-frame schema validity, the sequence itself +// must uphold: +// - consecutive deltas chain exactly: d2.from_revision == d1.to_revision +// - snapshot chunks: consecutive, same revision, part numbers 1..n, +// final:false on all but the last, final:true closing the run, and no +// other envelope interleaved before the final chunk +// - device_seq values are non-decreasing per device across the sequence +// (equal values are legitimate retransmissions), covering both uplink +// event bodies and delta-embedded events +function scenarioInvariantErrors(files) { + const errors = []; + let lastDeltaTo = null; + let chunkRun = null; // { revision, nextPart } while a snapshot is unfinished + const lastSeqByDevice = new Map(); + + const noteSeq = (rel, device, seq) => { + const prev = lastSeqByDevice.get(device); + if (prev !== undefined && seq < prev) { + errors.push(`${rel}: device_seq ${seq} for ${device} decreases (previous ${prev})`); + } + lastSeqByDevice.set(device, Math.max(prev ?? 0, seq)); + }; + + for (const file of files) { + const rel = relative(file); + const { frame } = unwrap(loadJson(file)); + const type = frame?.type; + const body = frame?.body ?? {}; + + if (chunkRun && type !== "snapshot") { + errors.push(`${rel}: ${type} envelope interleaved into an unfinished snapshot chunk run (SPEC 6.2 forbids anything but ping/pong)`); + chunkRun = null; + } + + if (type === "delta") { + if (lastDeltaTo !== null && body.from_revision !== lastDeltaTo) { + errors.push(`${rel}: delta from_revision ${body.from_revision} does not chain from previous delta's to_revision ${lastDeltaTo}`); + } + lastDeltaTo = body.to_revision; + for (const item of body.events ?? []) { + const ev = item.event ?? {}; + if (ev.device_id !== undefined && ev.device_seq !== undefined) noteSeq(rel, ev.device_id, ev.device_seq); + } + } else if (type === "snapshot") { + if (!chunkRun) { + if (body.part !== 1) errors.push(`${rel}: snapshot chunk run starts at part ${body.part}, expected 1`); + chunkRun = { revision: body.revision, nextPart: body.part + 1 }; + } else { + if (body.revision !== chunkRun.revision) { + errors.push(`${rel}: snapshot chunk revision ${body.revision} differs from run revision ${chunkRun.revision}`); + } + if (body.part !== chunkRun.nextPart) { + errors.push(`${rel}: snapshot chunk part ${body.part}, expected ${chunkRun.nextPart}`); + } + chunkRun.nextPart = body.part + 1; + } + if (body.final === true) chunkRun = null; + } else if (type === "task.event" || type === "message.event") { + if (body.device_id !== undefined && body.device_seq !== undefined) noteSeq(rel, body.device_id, body.device_seq); + } + } + + if (chunkRun) { + errors.push(`scenario ends with an unfinished snapshot chunk run (no final: true chunk)`); + } + return errors; +} + +// --- 2 & 3. check fixtures ---------------------------------------------- + +let failures = 0; +let checked = 0; + +function relative(p) { + return path.relative(realtimeRoot, p); +} + +const validDir = path.join(fixturesDir, "valid"); +const scenariosDir = path.join(fixturesDir, "scenarios"); +const invalidDir = path.join(fixturesDir, "invalid"); + +function unwrap(doc) { + if (doc && typeof doc === "object" && "frame" in doc && "sender_role" in doc) { + return { frame: doc.frame, senderRole: doc.sender_role }; + } + return { frame: doc, senderRole: null }; +} + +function allErrors(doc) { + const { frame, senderRole } = unwrap(doc); + const errors = validateFrame(frame); + if (senderRole !== null) errors.push(...authorizationErrors(senderRole, frame)); + return errors; +} + +for (const file of [...walk(validDir), ...walk(scenariosDir)]) { + checked++; + const errors = allErrors(loadJson(file)); + if (errors.length > 0) { + failures++; + console.error(`FAIL (expected valid): ${relative(file)}`); + for (const e of errors) console.error(` ${e}`); + } else { + console.log(`ok ${relative(file)}`); + } +} + +for (const file of walk(invalidDir)) { + checked++; + const errors = allErrors(loadJson(file)); + if (errors.length === 0) { + failures++; + console.error(`FAIL (expected invalid, but it validated cleanly): ${relative(file)}`); + } else { + console.log(`ok ${relative(file)} (rejected as expected: ${errors[0]})`); + } +} + +// --- 4. scenario sequence invariants ------------------------------------- + +for (const scenario of readdirSync(scenariosDir).sort()) { + const dir = path.join(scenariosDir, scenario); + if (!statSync(dir).isDirectory()) continue; + const files = readdirSync(dir).filter((f) => f.endsWith(".json")).sort() + .map((f) => path.join(dir, f)); + checked++; + const errors = scenarioInvariantErrors(files); + if (errors.length > 0) { + failures++; + console.error(`FAIL (scenario invariants): fixtures/scenarios/${scenario}`); + for (const e of errors) console.error(` ${e}`); + } else { + console.log(`ok fixtures/scenarios/${scenario} (sequence invariants hold)`); + } +} + +console.log(""); +console.log(`${checked} fixture(s) checked, ${failures} unexpected result(s).`); +if (failures > 0) { + process.exit(1); +}