Skip to content

Add the core-component registry table #826

Description

@sehkone

Add the core-component registry table

Context

review-database models per-node agents (src/tables/agent.rs) and external services (src/tables/external_service.rs). Neither fits the platform's own software: REView, aice-web-next, roxyd and bootroot are host-fixed infrastructure, not agents and not external services. roxyd runs on every host; REView and aice-web-next are singletons; bootroot is the installer-managed trust anchor and must never be offered for update through the UI.

This issue adds their registry: a table keyed by (component, host) recording what build is installed and what install/run state it is in.

The Lifecycle enum this issue uses is added by its dependency, in src/tables/lifecycle.rs: #[repr(u8)], variants NotInstalled = 0, Installing = 1, Running = 2, Stopped = 3, Failed = 4, Removing = 5, Unknown = u8::MAX.

Persist it the same way that dependency does, and for the same reason. The public record field is lifecycle: Lifecycle, but the private persisted Value struct stores a raw u8 holding the variant index (NotInstalled = 0 … Removing = 5, Unknown = 6). Putting the Lifecycle type directly in the persisted struct would make an unrecognized stored number fail the whole row's deserialization under bincode::DefaultOptions (invalid value: integer <n>, expected variant index 0 <= i < 7) rather than reading back as Unknown — which is exactly what the unknown-lifecycle test below checks. The raw-index form is byte-identical to what a derived-enum field writes, so this costs nothing on disk.

The dependency ships exactly one Lifecycle → index function and exactly one index → Lifecycle function, next to the enum in src/tables/lifecycle.rs; the read direction resolves every unrecognized number to Lifecycle::Unknown. Call those two from this table's ValueTrait::value() and FromKeyValue::from_key_value; the dependency named them Lifecycle::to_stored_index and Lifecycle::from_stored_index, and both are already pub(crate), so nothing in lifecycle.rs needs widening. Do not write a third mapping, and do not reach for FromPrimitive::from_u8 / ToPrimitive::to_u8: those map the #[repr(u8)] discriminant, not the stored variant index, so from_u8(6) is None while 6 is exactly what Unknown stores, and from_u8(255) returns Some(Unknown) for a number this encoding never writes.

Scope

Add src/tables/core_component.rs with a record type carrying:

  • component: String — the canonical package-id: review, aice-web-next, roxyd, or bootroot
  • host: String
  • installed_version: Option<String>
  • installed_commit: Option<String>
  • lifecycle: Lifecycle
  • installer_managed: bool

Implement UniqueKey, the crate's Value trait, and FromKeyValue, plus a Table<'_, CoreComponent> block with open, get(component, host), delete(component, host), update(old, new), and iteration — following the shape of src/tables/hosts.rs and src/tables/agent.rs. Declare the module in src/tables.rs and re-export the record type from src/tables.rs and src/lib.rs. The key encoder stays private to the module: get and delete take the pair as two &strs, so no caller outside this crate ever names the type, and src/tables/hosts.rs keeps its Key unexported for the same reason.

Build identity is (version, commit), never version alone: the same version may carry different commits (a pre-release rebuilt from a new commit, or a hotfix without a version bump). version is an opaque display label and is not required to be semver.

installer_managed is true for bootroot and marks the row as excluded from UI-driven update — bootroot is the trust anchor and is installer-managed. This crate stores the flag; enforcing it is review-web's job.

Why component is a String and not an enum

This is settled; do not substitute an enum. The project's coding standards prefer an enum over a String wherever a finite set of values is expected, and the four core components do look like such a set — so the choice needs its reason recorded rather than re-argued in review:

  • The package-id registry is owned outside this crate. component holds a canonical package-id, and that registry is defined by the platform's packaging layer, not by review-database. An enum here would fork it: the authoritative list would live in one place and a copy of it would live in this crate's persisted schema.
  • review and review-web already exchange package-ids as strings, so a String crosses both boundaries with no translation layer in either direction.
  • Adding a core component must not be a schema change. With an enum, a fifth component means editing this crate, and — because the variant would be persisted — a data migration and a format bump for what is really just a new row.
  • The cost is accepted deliberately: an invalid package-id is not rejected at the type level, and the component half of the key stays variable-length, which is what makes the collision-safe encoding below mandatory rather than optional.

Validating the package-id is out of scope here (see Out of scope); this table stores what it is given.

The composite key must be collision-safe

The (component, host) key is an unambiguously-encoded tuple, never a naive byte concatenation. Both fields are variable-length strings, so concatenating their bytes collides: ("ab", "c") and ("a", "bc") would map to the same key. The key MUST be a length-prefixed or otherwise serialized tuple — or a delimiter byte that is validated not to occur in either field, since component is a fixed package-id and host is a DNS label. Either encoding is acceptable, but the choice must be stated in a doc comment on the key encoder and covered by the collision test below. Neither field is ever empty, so the key has no empty-segment case.

No existing table in this crate is a precedent for this. Every composite key here is either fixed-width (hosts.rs: u32 + IpAddr) or a fixed-width prefix followed by a single trailing variable-length field (agent.rs / external_service.rs: node_id.to_be_bytes() then key.as_bytes()) — both are unambiguous for free, and neither shows how to join two variable-length strings. Copying that extend-the-bytes idiom straight across is precisely the bug this section exists to prevent.

Keep the encoding in one function used by every read and write path, so get, delete and unique_key cannot drift apart. The decoder that from_key_value uses must be its exact inverse and must live beside it.

v1 stores at most one row per (component, host)

v1 stores at most one instance per (component, host), and the instance number is always 1. There is deliberately no counter, no reservation table, and no release rule — the three things an allocator would need. A second install for a (component, host) that already has a row is refused, not resolved by picking another number. At this layer that means the insert path must fail on an existing key rather than overwrite it; the crate's Map::insert already bails with key already exists, so Table::insert gives this for free — do not route inserts around it.

Core components are single-instance, so unlike agents this table needs no instance dimension in its key. roxyd is per-host, so multiple rows share component = "roxyd" with distinct host; REView and aice-web-next are singletons with one row each.

Acceptance criteria

  • The record type exists with exactly the six fields above — component as a String, not an enum — and is re-exported from src/tables.rs and src/lib.rs.
  • The private persisted Value struct stores the lifecycle as a raw u8 variant index, not as a Lifecycle, and calls the dependency's two conversion functions rather than adding a second mapping or using FromPrimitive::from_u8.
  • The (component, host) key is produced by a single encoding function, documented as length-prefixed/serialized (or validated-delimiter), and used by every read and write path.
  • ("ab", "c") and ("a", "bc") map to distinct keys.
  • CRUD works: insert, get by (component, host), update, delete, and iterate.
  • Inserting a second row for a (component, host) that already exists is refused with an error; the existing row is unchanged.
  • Multiple roxyd rows with distinct host values coexist and are independently readable and deletable.
  • A bootroot row carries installer_managed = true and round-trips.
  • CHANGELOG.md gains an entry under ## [Unreleased].
  • cargo fmt -- --check --config group_imports=StdExternalCrate and cargo clippy --bins --tests --all-features -- -D warnings pass, and cargo test --all-features is green.

Constraints

  • Do not add this table's column-family name to MAP_NAMES, and do not add a Store::*_map() accessor for it. StateDb::open auto-creates every CF listed in MAP_NAMES, while migrate_data_dir returns early for a data dir already at a compatible version — so registering the CF before COMPATIBLE_VERSION_REQ bumps would silently add a CF to a 0.46.0 data dir with no version change. Registration and the Store accessor land with the format bump, in a later change. Define the CF name constant in src/tables.rs next to the other name constants, with the same pub(super) visibility they use (it is inert until it is listed), but leave MAP_NAMES and its array length alone. One consequence to expect: with no Store accessor calling it, open is reachable only from the tests below, so it needs an #[allow(dead_code)] — carrying a comment saying the registration and the accessor land with the format bump — for cargo clippy -D warnings to pass on a non-test build.
  • Because the CF is not registered yet, tests must open their own database. Use tempfile::tempdir() plus a raw rocksdb::OptimisticTransactionDB::open_cf with create_missing_column_families(true) over MAP_NAMES plus this table's CF name, then build the table over that handle with the crate-internal Map::open(...).map(Table::new) path. src/migration.rs's tests show the raw-open pattern. Hold a crate::test::acquire_db_permit() guard alongside the TempDir for as long as the database is open, as the other table tests do: it caps concurrent RocksDB instances so a parallel run does not exhaust file descriptors. Never write to a fixed path.
  • The tests must live in a #[cfg(test)] mod tests inside src/tables/core_component.rs. Table::new is private to the tables module, so it is reachable from this child module but not from tests/; an integration test would not compile.
  • Do not touch COMPATIBLE_VERSION_REQ, add a migration, or change Agent / ExternalService.
  • Do not implement any instance allocation, reservation, or release logic.
  • Follow the crate's tables::serialize / tables::deserialize helpers for the value encoding; do not introduce a second bincode configuration.
  • Visibility: start at the most restrictive that compiles, widening to pub only for the record type, its fields, and the table methods that review / review-web will call.
  • No unwrap() outside tests; no [] indexing into slices or Vecs.

Out of scope

  • Registering the column family, the Store accessor, MAP_NAMES, the version bump, and any migration.
  • The operation_attempt ledger.
  • Enforcing the installer_managed exclusion in an API surface — that is review-web's job; this crate only stores the flag.
  • Validating that component is one of the four known package-ids, or that host is a well-formed DNS name. The table stores what it is given; the only string property it relies on is the one the key encoding itself tests.
  • Deciding which core components exist beyond the four package-ids named above.

Test plan

  • Value round-trip: every field, including installed_version / installed_commit as None and as Some, and both installer_managed values.
  • Key collision: ("ab", "c") and ("a", "bc") produce distinct keys; both can be stored simultaneously and each get returns its own row.
  • Key round-trip: encoding then decoding a (component, host) pair returns the original pair for realistic values (("roxyd", "host-01.example.com")) and for the adversarial pairs above.
  • Three roxyd rows on three hosts coexist; each is independently gettable; deleting one leaves the other two intact.
  • A bootroot row stores and reads back installer_managed = true, and a review row reads back false.
  • Duplicate refusal: inserting a second row for an existing (component, host) returns an error and leaves the stored row unchanged — asserting that v1 refuses rather than allocating another instance.
  • lifecycle round-trips through every variant. Writing a row, overwriting its lifecycle byte with a number no variant uses (for example 9), and reading it back succeeds, yields Lifecycle::Unknown, and leaves component, host, installed_version, installed_commit and installer_managed intact — the row stays readable rather than failing to deserialize.

Dependencies

Depends on the Lifecycle enum issue. Part of the install/update state schema umbrella.

Pointers

  • src/tables/lifecycle.rs — added by the dependency: the Lifecycle enum and the two index conversion functions this table must call.
  • src/tables/agent.rsFromKeyValue / UniqueKey / ValueTrait and the Table::<Agent> CRUD block, including the module-level test setup; also the crate's closest composite-key idiom (node_id.to_be_bytes() then key.as_bytes()), which is unambiguous only because the variable-length half is last — read it as a pattern to adapt, not to copy.
  • src/tables/hosts.rs — a simple Table<'_, R> with a dedicated Key struct and a Key::new(..).to_bytes() / Key::from_be_bytes(..) pair; the structural model for a separate key encoder, though its key is fixed-width.
  • src/tables.rs — CF name constants, MAP_NAMES (leave alone), module declarations, the pub use block, Map, Table, Table::new (private to this module), serialize / deserialize, and Table::insert.
  • src/collections/map.rsMap::insert, which bails with key already exists; the duplicate-refusal criterion rests on it.
  • src/migration.rs tests — the raw OptimisticTransactionDB::open_cf pattern over a tempfile::tempdir().

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions