Install/update state schema (RFC D1)

[sehkone]

Install/update state schema (RFC D1)

Summary

review-database today stores per-node config (applied + draft) and a coarse config-reload Status. It has no notion of installed software: no version, no install/run lifecycle, no package-operation record. This effort adds that state additively, so that review (D2) and review-web (D3) have the types they need to install, update, and display packages on hosts.

The effort delivers, in dependency order:

1. a Lifecycle value type — the install/run state of a package on a host, distinct from and orthogonal to the existing config-reload Status — together with its first users, the installed_version, installed_commit, lifecycle and bound_addrs fields on Agent and ExternalService;
2. a core-component registry table keyed by (component, host) for host-fixed infrastructure (REView / aice-web-next / roxyd / bootroot) — these are not agents and not external services;
3. the operation_attempt ledger record — a durable, crash-safe record of a package operation, keyed uniquely by idempotency_key;
4. the ledger's secondary indexes, expiry sweep and retention rule, which are what make it usable as an orchestration guard;
5. the migration (migrate_0_46_to_0_47), the COMPATIBLE_VERSION_REQ bump to ">=0.47.0,<0.48.0", and registration of the two new column families — all in one slice;
6. a public format-version writer so a rolled-back REView can restore the VERSION markers together with a restored snapshot.

The source of truth is docs/rfcs/0001-install-update-state-schema.md in this repo. Each child issue restates the contract it needs and is implementable from its own text.

Children

• Add the Lifecycle type and install/update fields to Agent and ExternalService — Value types
• Add the core-component registry table — New tables
• Add the operation_attempt ledger record and idempotency-keyed CRUD — New tables
• Add operation_attempt secondary indexes, expiry sweep, and retention — New tables
• Migrate to DB format 0.47.0 and register the new column families — Migration and format bump
• Add a public format-version marker writer for rollback — Migration and format bump

Shared background

This repo only defines and persists types. Store layout, signature verification, GraphQL, and roxyd control belong to review (D2) and review-web (D3). Nothing here calls out to a host or a registrar.

No desired state. Install/update is an immediate imperative action, not a stored intent. There is no desired_version and no reconcile loop. Only actual state is stored, for display and for crash-safe resume. UpdateAvailable is computed by review (installed (version, commit) differs from the store's latest build) and is never stored here.

Build identity is (version, commit), never version alone. The same version may carry different commits — a pre-release rebuilt from a new commit, or a hotfix without a version bump. version is an opaque display label and is not required to be semver. Every place that records an installed or resolved build records both.

Where the data lives today. The public Node struct (src/tables/node.rs) exposes agents: Vec<Agent> and external_services: Vec<ExternalService>, but that is an assembled view. The persisted node record is the private Inner struct, which holds only the agent/external-service string keys. The Agent and ExternalService values live in their own column families, AGENTS ("agents") and EXTERNAL_SERVICES ("external services"), listed in MAP_NAMES in src/tables.rs. Anything that has to touch those values — notably the migration — walks those two column families, not the node records.

Multi-instance is already expressible. Agent.key is <instance>.<service> and is unique per node, so two piglet instances on one node are two rows, 001.piglet and 002.piglet, under the same node_id and the same kind. No schema change is needed for that. What it does invalidate is any assumption that a node holds at most one row per AgentKind; no code path may assume that.

v1 pins the instance number to 1. At most one instance per (component, host) is stored and the number is always 1. There is deliberately no counter, reservation table, or release rule — a second install for a (component, host) that already has a row is refused, not given another number. The schema nevertheless carries the instance dimension (operation_attempt.instance, the (host, target, instance) index) so a later allocator is an extension rather than a migration.

Column families are registered only with the format bump. StateDb::open auto-creates every CF named in MAP_NAMES, while migrate_data_dir returns early for a data dir that is already at a compatible version. Adding a new CF name to MAP_NAMES before COMPATIBLE_VERSION_REQ bumps would silently mutate a 0.46.0 data dir with a new CF and no version change. So the new-table issues implement their table modules without touching MAP_NAMES, and the CF registration lands in the same change as the version bump.

Do not cut a release from the middle of this effort. Between the first child merging and the format-bump child merging, main carries a changed persisted record shape for Agent and ExternalService while COMPATIBLE_VERSION_REQ and the crate version still read 0.46.0 — so a build from main in that window advertises a 0.46.0 compatibility it no longer has, and would fail to read a real 0.46.0 data dir. The window is deliberate: the RFC's own ordering requires the shapes to exist before the migration that converts them, and it is CI-safe because no existing test builds a persisted Agent / ExternalService fixture from bytes. But no release may be cut from inside it. The effort is releasable again only once Migrate to DB format 0.47.0 and register the new column families has merged.

Naming trap. operation_attempt is not aice-web-next's apply_attempts table, and neither replaces the other. apply_attempts tracks one operator's config-Apply click through the UI (which drafts it intended to dispatch, whether the browser-side run still holds the lock). operation_attempt tracks the package operation REView is executing on a host, survives a REView restart, and carries the compensation owed to bootroot. They sit on different sides of the API, key on different things, and have different lifetimes. Do not collapse them or drive one from the other.

Execution order

Issues in the same wave have no unmet dependencies among these children and can run in parallel.

• Wave 1: Add the Lifecycle type and install/update fields to Agent and ExternalService #825, Add the operation_attempt ledger record and idempotency-keyed CRUD #827, Protect operation attempt iteration #858
• Wave 2: Add the core-component registry table #826, Add operation_attempt secondary indexes, expiry sweep, and retention #828
• Wave 3: Migrate to DB format 0.47.0-alpha.2 and register column families #829
• Wave 4: Add a public format-version marker writer for rollback #830

graph TD
  issue825["#825 Add the `Lifecycle` type and install/update fields to `Agent` and `ExternalService` [phase: Value types]"]
  issue826["#826 Add the core-component registry table [phase: New tables]"]
  issue827["#827 Add the `operation_attempt` ledger record and idempotency-keyed CRUD [phase: New tables]"]
  issue828["#828 Add `operation_attempt` secondary indexes, expiry sweep, and retention [phase: New tables]"]
  issue829["#829 Migrate to DB format 0.47.0 and register the new column families [phase: Migration and format bump]"]
  issue830["#830 Add a public format-version marker writer for rollback [phase: Migration and format bump]"]
  issue858["#858 Protect operation attempt iteration [phase: unphased]"]
  issue825 --> issue826
  issue827 --> issue828
  issue825 --> issue829
  issue826 --> issue829
  issue827 --> issue829
  issue828 --> issue829
  issue829 --> issue830

Loading

[sehkone]

All six children of this umbrella, plus the one follow-up filed during the effort, are closed as completed with merged pull requests:

Child	PR	Merged
#825 Lifecycle type and install/update fields	#854 (5174bfafea)	2026-08-17
#826 core-component registry table	#856 (8091212217)	2026-08-18
#827 operation_attempt ledger and idempotency-keyed CRUD	#853 (bc3190ac5f)	2026-08-17
#828 secondary indexes, expiry sweep, and retention	#857 (6feedf3906)	2026-08-18
#829 format migration and column-family registration	#861 (34bb709bd6)	2026-08-18
#830 public format-version marker writer	#863 (1727d97776)	2026-08-22
#858 operation-attempt iteration guard (follow-up)	#860 (b81c117199)	2026-08-18

Two notes for the record:

• Migrate to DB format 0.47.0-alpha.2 and register column families #829 landed the format move as 0.47.0-alpha.1 → 0.47.0-alpha.2, following the repository's documented alpha-to-alpha convention: it extends the existing migrate_0_46_to_0_47 entry (requirement >=0.46.0,<0.47.0-alpha.2, target 0.47.0-alpha.2) rather than adding a separate alpha-step migration. The umbrella's literal "0.47.0" phrasing resolves to that convention, not to a final-release marker.
• The "do not cut a release from the middle of this effort" window described under Shared background closed when Migrate to DB format 0.47.0-alpha.2 and register column families #829 merged; from this tree's standpoint main is releasable again.

With every child delivered, this umbrella is complete.