You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Probe the reserved audit store's capacity and report it on the endpoint
Context
The reserved audit store is built, the reserve behind it is real, and the mode that decides what stands behind it is settled. What is missing is measurement.
[registrar] in the bootroot-agent configuration carries audit_store_dir, audit_store_reserve_bytes (default 2147483648, 2 GiB), audit_store_low_water_bytes (default 536870912, 512 MiB) and audit_store_enforcement (filesystem by default, directory the explicit opt-out). They are declared in src/config.rs and checked by validate_registrar_settings in src/config/validation.rs, which already rejects a reserve above i64::MAX, a low-water value at or above the reserve, and an audit_record_dir that resolves outside the store. src/registrar/audit_store.rs owns the layout: records_dir() and openbao_dir() resolve <audit_store_dir>/records and <audit_store_dir>/openbao, both created root-owned and 0700. openbao/ does not stay that way by design — the OpenBao container's entrypoint chowns and chmods its bind-mount source on first start, and bootroot deliberately neither asserts nor repairs that directory afterwards, because a root-owned file in a directory the container cannot chown is a device that cannot write.
In filesystem mode bootroot init provisions a fully allocated loopback image of exactly audit_store_reserve_bytes carrying an ext4 filesystem, mounted at audit_store_dir through a generated systemd mount unit that is restored on boot, so a write past the reserve fails with ENOSPC on the reserve rather than filling the host's root filesystem; an endpoint-enabled daemon that finds the store is not a mount point at start keeps running but refuses every verb with a permanent registrar_unavailable carrying reason audit_unwritable. In directory mode the store is a plain directory sharing the host's root filesystem and the reserve is a budget with nothing behind it.
A hard ceiling tells nobody it is about to be reached. Both audit artifacts are fail-closed inputs to a live security argument: OpenBao's file audit device is mandatory and OpenBao fails requests it cannot audit, and the verb records are the only detection for an abuse whose signature is a rate. A reservation with no measurement turns a slow fill into a sudden outage on the one host that must not be restarted, and a look-back window that has quietly shrunk is invisible.
The record store's own signals exist already, and they stop at the bootroot host. scan_audit_store in src/registrar/audit/scan.rs reads the store without creating or modifying anything and returns intent_without_outcome, malformed_records and a derived retention_short flag together in one AuditScan. bootroot status calls it — through the spawn_blocking wrapper scan_audit_store_off_runtime — over audit_record_dir, with pub const AUDIT_SCAN_WINDOW: Duration = Duration::days(30), audit_max_retained_files and audit_min_retain_days, and prints the three values host-locally. Nothing carries them off the host. The daemon is not a control-plane protocol peer and has no channel of its own; the co-located registrar relays what the daemon exposes on the mint/deregister endpoint.
That relay already has both a container and a shared holder, so this issue plugs into them rather than building either. RegistrarHealth in src/registrar/endpoint/protocol.rs is carried on all three response shapes — mint success, deregister success and refusal — and today holds one member, limiter. src/daemon.rs builds one Arc<Mutex<RegistrarHealth>>, hands it to ProductionHandler::with_health, and refreshes it from refresh_registrar_health on the OpenBao rotation loop's existing maintenance callback, which fires every ROTATION_INTERVAL — one minute. The response encoder receives that one daemon-held snapshot and has no other source for it.
This issue adds a capacity measurement, an alarm over it, one new member on that existing container, and one new body of work on that existing tick. It provisions nothing, mounts nothing, adds no scheduler and creates no holder.
Scope
The capacity probe
A capacity probe reports two numbers for audit_store_dir: the store's usage in bytes, and the available bytes on the filesystem backing it.
Available bytes come from libc::fstatvfs on an open descriptor for the store root, not from a pathname statvfs (libc is already a direct dependency, so this needs no new crate): f_bavail × f_frsize, the space available to an unprivileged writer, not f_bfree, which counts blocks reserved for root and would report headroom the writers cannot actually use. The descriptor is the one described below, opened once per tick with O_NOFOLLOW in both enforcement modes; a pathname statvfs would follow a symbolic link at the store root, which this issue requires to fail the probe.
Usage is measured per enforcement mode, and this is spelled out because statvfs cannot answer it in one of them.statvfs describes a filesystem, not a subtree, so a single implementation is wrong in one mode or the other:
In filesystem mode the store is its own filesystem, so its usage is that filesystem's used space — (f_blocks − f_bfree) × f_frsize from the same fstatvfs call on the same descriptor. One syscall, exact, no walk.
In directory mode the store shares a filesystem with the rest of the host, where f_blocks − f_bfree measures everything on the host and not the store. Usage there is measured by walking audit_store_dir on the tick and summing allocated size — st_blocks × 512 — rather than apparent length, so a sparse or block-rounded file is not under-counted against a ceiling the kernel enforces in blocks. st_blocks is in fixed 512-byte units by POSIX, independent of the filesystem's f_frsize; give that unit a named file-top constant rather than a 512 at the call site.
The walk is affordable precisely because of what lives there: two subdirectories holding a bounded number of large append-only files — the record store caps itself at roughly 136 MiB across at most 17 files, and the OpenBao device is one active file plus its retained generations. It is a per-tick cost, never a per-request one. Do not walk in filesystem mode, where the syscall already answers it, and do not cache a stale usage across ticks to avoid the walk.
The probe is an abstraction, so the alarm's tests are portable. Define it as a small trait — usage bytes and available bytes for a path — with the real implementation behind it (fstatvfs on the root descriptor for available bytes in both modes, and for usage either that same call or a walk descending from that same descriptor, per the mode rule above) and a test implementation. The alarm and hysteresis logic are then driven against the trait over a tempfile::tempdir() with a synthetic reserve, so "the alarm fires before the reserve is consumed" is an ordinary cargo test needing no special filesystem, no root and no container. One test additionally exercises the real implementation against a tempdir to prove it reports plausible non-zero values. ReserveProbe and HostProbe in src/commands/audit_store/reserve.rs are the shape to copy, not the code to share — see below for why the two stay separate.
The root descriptor, and the descriptor-based directory-mode walk
The store root's open and the walk's mechanism are pinned here rather than left to the implementer, because part of the store is attacker-influenced by design: openbao/ is owned by the OpenBao container's uid, so every entry the container writes there — including entries a compromised container writes — is inside the subtree this walk sums.
The root descriptor is opened once per tick, by path, carrying O_NOFOLLOW — and it is opened in both enforcement modes. Open audit_store_dir itself with O_NOFOLLOW | O_DIRECTORY | O_RDONLY | O_CLOEXEC on its configured absolute path. That one descriptor is the whole probe's anchor: fstatvfs on it yields the available bytes and, in filesystem mode, the usage; its fstat establishes both the st_dev every entry below is compared against and the first (st_dev, st_ino) in the seen set; and in directory mode the walk descends from it. Binding both halves of headroom_bytes to one descriptor is what makes them describe one object — a pathname statvfs beside a separately opened walk root resolves the configured path twice, and nothing holds the two resolutions to the same directory. In filesystem mode nothing is enumerated, so the root is opened, fstat-ed, fstatvfs-ed and closed again without a single readdir on it. The open-directory operation the test seam below defines spans both openat and fdopendir, and it is that one operation which opens the root in both modes, so the root's stream is created and released rather than read; keeping a second, stream-less open for this mode would put the ownership transfer at two call sites instead of one, which is the thing that seam exists to prevent. A symbolic link at the store root is therefore an ELOOP from that open and a failed probe in either mode, leaving the previous state and measured_at in place, exactly as a substituted directory further down is. Without it, filesystem mode would have nothing that refuses such a link, and a pathname statvfs would quietly report the target's filesystem. That is not a new rule this issue invents: the store contract already refuses it. check_store_directory in src/registrar/audit_store.rs stats the path with symlink_metadata so a planted link is seen as a link rather than as whatever it points at, reporting it as PathFault::Symlink, and check_ancestors refuses one at every component above the store. Nor does an operator's own configured path make the object at it safe later: O_NOFOLLOW checks what is there at open time, the same time-of-check problem the descent rule exists for. It covers the final component only; the components above the store are the store contract's business, checked where the layout is created and verified, and this walk does not re-derive them. Any error from that open is a failed probe on the same terms — ELOOP for a planted link, EACCES, and ENOENT for a store path that is not there at all. The vanished-entry carve-out below is about entries a readdir listed, and does not reach the root.
Descend by descriptor, never by path. Each directory at or below audit_store_dir is opened with openat from its parent's descriptor using O_NOFOLLOW | O_DIRECTORY | O_RDONLY | O_CLOEXEC and enumerated with fdopendir/readdir on that same descriptor. fstat on that descriptor supplies both the (st_dev, st_ino) identity and the st_blocks the walk sums, so the object that is classified is the object that is traversed. A path-based lstat followed by a path-based read_dir cannot guarantee that: a directory can be replaced by a symlink between the two resolutions, and in directory mode the store shares the root filesystem, so the same-device rule would not exclude a link pointing at /. A followed link there would sum the whole root filesystem into used_bytes, drive headroom far negative, report exhausted and refuse the registrar verbs — a denial of service reachable from the subtree this issue already declares attacker-influenced.
Skip . and .. by name, before anything else.readdir returns both in every directory, and nothing filters them out the way std::fs::read_dir does — that convenience is what the raw enumeration above gives up, which is why measure_underlying never had to state this rule. Discard both as each entry is read, ahead of any fstatat, any openat and any accounting. .. is an ordinary directory on the same device, so neither O_NOFOLLOW nor the same-device rule refuses it: descending into it would climb out of audit_store_dir into its parent and over the whole filesystem from there, summing all of it into used_bytes — the same runaway the descent rule exists to prevent, reached with no attacker at all. . is caught eventually by the seen-inode set, but is skipped by name too: leaving an infinite descent to a dedup is not a rule a reader can check.
Entries that are not directories are never opened. They are classified with fstatat(parent_fd, name, AT_SYMLINK_NOFOLLOW) and counted from that. Each contributes whatever st_blocks its own fstatat reports — a FIFO, a socket or a device node holds no data blocks on the filesystems this store runs on and so reports zero, but the walk counts what it is told rather than assuming a class of entry contributes nothing. Nothing is opened or read for them, so a FIFO cannot block the walk and a symlink is never followed: a symlink contributes its own entry's allocated blocks and its target is not resolved, whether it points inside the store or out of it. Directories are opened because opening them is the only race-free way to enumerate them.
Distinguish end-of-directory from an error.readdir returns a null pointer for both, so set errno to zero before each call and read it back after: a null with errno still zero is the end of the stream, and a null with a nonzero errno is a failed probe. Without that, an I/O error mid-directory reads as a short directory, and the walk returns a smaller total for a store it could not finish reading — under-reporting usage on the one control whose whole job is to notice a store filling up, which is the same failure as silently summing an unreadable subdirectory as zero.
Stay on one device. Apply the same-device rule to the descriptor's fstat — an entry whose st_dev differs from audit_store_dir's is not counted and not descended into. Usage summed across a nested mount would be weighed against statvfs available bytes for a different filesystem, so the two halves of headroom_bytes would describe different devices.
Count each distinct (st_dev, st_ino) once, from the descriptor for a directory and from the fstatat for everything else. Two paths hard-linked to one file consume one file's blocks; that is what the kernel accounts for and what filesystem mode's statvfs usage reports for the same bytes, so counting per path would inflate usage and fire an alarm over space nobody used. The store holds a bounded number of files, so the seen-inode set is a small HashSet, not a memory concern.
Directories count their own blocks, audit_store_dir included. A directory that once held a great many entries keeps that allocation, the kernel charges it against the reserve, and filesystem mode's statvfs usage counts it — so omitting it here would make the two modes answer the same question differently over the same bytes.
A usage measurement that fails is a failed probe, on the same terms as a failed statvfs: the previous state and measured_at stand. An unreadable subdirectory in the middle of a walk must not be silently summed as zero, which would report a filling store as empty. The one carve-out is an entry that vanishes under the walk: an entry readdir listed whose openat or fstatat then returns ENOENT is skipped and the walk continues, because the OpenBao device and the record store both rotate generations away while the tick runs, and treating that ordinary race as a probe failure would strand the alarm on a stale reading every time a rotation lands on a tick. Every other error fails the probe — EACCES on a subdirectory above all, and ELOOP or ENOTDIR from an openat that met a substituted entry. A directory that became a symlink under an active walk is precisely the event a capacity measurement must not paper over with a number.
The unsafe this introduces is bounded, and its one ownership transfer is pinned exactly. These calls have no safe wrapper in the standard library, so the walk is the first unsafe FFI on this path in the library crate. Keep each block to the single call it wraps, precede it with a // SAFETY: comment stating the invariant that makes it sound, and give every descriptor exactly one owner at every point:
A successful openat yields a raw descriptor that is wrapped in std::os::fd::OwnedFd immediately, so the closing is structural rather than a thing to remember.
fdopendir takes ownership of the descriptor it is given, so hand it over with into_raw_fd(), never as_raw_fd().as_raw_fd() would leave the OwnedFd still owning a descriptor that closedir will also close. A double close is worse than a leak in a long-lived daemon: the second close lands on whatever descriptor has since been handed that number, so an unrelated file, socket or directory stream is closed under whichever task owns it, at a moment nothing correlates with this walk.
A null fdopendir return transfers nothing. The caller still owns the raw descriptor and must close it before propagating the failure — reconstructing the OwnedFd from it and letting the drop run keeps that structural too. Skipping it leaks one descriptor per failure, on a tick that fires every minute for the life of the process.
After a successful fdopendir, closedir is the only close. Do not also close the raw value, and reach the descriptor for the child openat and fstatat calls through dirfd(), which borrows rather than owns.
as_raw_fd() stays correct everywhere the callee only borrows — fstat on the directory, and openat/fstatat against it as a parent. The rule above is about the one call that takes ownership, not a ban on borrowing.
This is deliberately a second implementation, and the reason is recorded here so it does not read as an oversight.measure_underlying in src/commands/audit_store/reserve.rs already sums st_blocks × 512 over a probe trait, dedups (st_dev, st_ino) and refuses to cross a device boundary. It is pub(super) in the binary crate, so the daemon — which lives in the library crate — cannot call it. Do not lift it into the library and do not rewire the bootroot init preflight around a shared implementation. The two answer different questions under different rules: that one is path-based and runs once under an operator's init on a store nothing is writing to, and it fails on an arithmetic overflow, while this one runs unattended every minute against a live, partly attacker-writable subtree and saturates instead. Merging them would force one set of rules onto both.
The walk's test seam
Most of the walk is testable against a real tempfile::tempdir(), and is tested that way. Five of its required behaviours are not: each is a race, a privilege assumption or a process-global observation, and staging any of them on disk yields a test that passes for the wrong reason. Put the walk's syscall operations behind a small test-only seam and drive those five through it.
The seam is the walk's own operation set, not a general syscall mock: open a directory relative to a parent and return it ready to enumerate — the same operation with no parent descriptor is how the root is opened, read the next entry from an open directory, stat an open directory, stat an entry by name relative to a parent, and close a directory. The production implementation is the openat/fdopendir/readdir/fstat/fstatat/closedir sequence above. The open-directory operation deliberately spans both openat and fdopendir, so the ownership transfer between them has one implementation rather than one per call site. The test implementation wraps a real tempdir and can be told to fail one named operation, on one named path, with one named errno; for the open-directory operation it must also say which of the two syscalls fails, because the descriptor has a different owner on either side of the transfer.
What goes through it, and why on-disk staging does not work for these:
EACCES on a subdirectory. A chmod 0o000 fixture passes vacuously whenever the suite runs as root, which it does whenever the suite is run from a root shell — a container shell being the common case. The repository's existing workaround — assert_ne!(current_process_euid(), 0, ...) at the top of such a test, as src/registrar/internal/tests.rs does — turns that into a hard failure rather than a false pass, which is better but still leaves the required behaviour unproven wherever the assert fires.
An entry that vanishes between readdir and the openat/fstatat that follows it. Injecting ENOENT at exactly that call is deterministic; a second thread deleting a file while the walk runs is not, and a sleep hoping to land between two syscalls is the synchronisation the project rules forbid.
A directory replaced by a symlink between its enumeration and its descent. The openat is the only synchronisation point at which that substitution is observable, so injecting ELOOP/ENOTDIR there is what makes the test deterministic.
EIO from readdir, for the end-of-directory rule above. There is no portable way to make a real directory stream fail mid-read.
Descriptor hygiene, including the fdopendir failure path. The accounting counts the descriptor openat produced as well as the closedir that releases it, so a walk asserts opens and closes balance — over a walk that succeeded, over one that failed part-way through a subdirectory, and over one whose fdopendir failed after its openat had already succeeded. That last path is the one the ownership rules above exist for, and none of the other injection points reaches it: each of them fails before the transfer or after it, never inside it. Counting entries in /proc/self/fd instead would observe every other thread the test runner has in flight, so it is not an assertion that can hold under cargo test's default parallelism — and checking one descriptor number with fcntl afterwards is no better, since another thread can be handed that number between the close and the check.
What must not go through it. Everything a plain tempdir already makes deterministic is tested against the real filesystem, so the security properties are proven against real syscalls rather than against the double: the symlink, hard-link, sparse-file, special-file, nested-subdirectory and same-device cases, a symbolic link planted at the store root itself, the directories-count-their-own-blocks rule, the ./.. skip, and the real fstatvfs.
Headroom and its arithmetic
headroom_bytes = min(audit_store_reserve_bytes − used_bytes, filesystem_available_bytes). The min is the point: a reserve larger than the device's free space is not a reserve, and the alarm must fire on whichever bound binds first.
A filesystem-derived byte count formed from a block count and a block size saturates rather than wrapping or failing. That covers all three of them — f_bavail × f_frsize, filesystem mode's (f_blocks − f_bfree) × f_frsize, and each entry's st_blocks × 512. Compute each with u64::checked_mul and take u64::MAX on None; take the f_blocks − f_bfree difference with saturating_sub and accumulate the walk's total with saturating_add, on the same terms. No wider intermediate type, no as cast, and the probe does not fail. Saturating is what the min above makes correct: a filesystem reporting a block count whose product overflows 64 bits is reporting something no device holds, so saturating makes that term stop binding and the reserve term decides the headroom, which is the answer the operator wants. Failing the probe instead would turn an implausible-but-harmless filesystem report into unknown and drop the alarm entirely — the one outcome this issue exists to prevent. Wrapping is what is actually forbidden: a wrapped product can read as a small available figure and manufacture a spurious exhausted on a healthy store.
The three inputs are u64 and the result is i64, so the conversion is specified rather than left to chance. Configuration validation already bounds audit_store_reserve_bytes at i64::MAX, so that term converts exactly; used_bytes and filesystem_available_bytes are clamped to i64::MAX on conversion, and the subtraction is saturating. Use a checked conversion with an explicit clamp — no as cast anywhere between the u64 inputs and the i64 result.
The alarm state
state is defined exactly, including the negative-headroom case:
unknown — no capacity probe has succeeded yet (daemon just started, or every probe so far has failed). This is the only state in which used_bytes, headroom_bytes and measured_at are absent, and it exists so that "we have not measured" is never encoded as ok.
exhausted — the store is at or past its reserve.
low_water — the alarm is on but the reserve is not yet consumed.
ok — headroom is above the threshold, or above the clear threshold when an alarm is being cleared.
The rules are evaluated in this order, with previous the state the last successful probe recorded and threshold the configured audit_store_low_water_bytes:
If headroom_bytes <= 0, the state is exhausted. Zero counts as exhausted, not as low water: at zero headroom the next write is already past the reserve.
Otherwise, if previous is low_water or exhausted, the state is ok when headroom_bytes >= threshold + margin and low_water otherwise.
Otherwise — previous is unknown or ok — the state is ok when headroom_bytes > threshold and low_water otherwise.
Rule 2 is the hysteresis, and it exists so a store hovering at the threshold does not flap an operator's console. Rule 3 is what keeps the cold-start behaviour honest: from unknown, headroom of threshold + 1 is ok, because the first probe has no alarm to clear and the hysteresis margin would otherwise invent one. Four points fall out and are pinned by the acceptance criteria: the alarm is on at headroom_bytes == threshold (rule 3, so the configured number reads as "alarm at 512 MiB left" rather than "alarm just under"), off at threshold + 1 from a cold start (rule 3), held on the way back up below threshold + margin (rule 2), and cleared at threshold + margin (rule 2).
An exhausted store whose headroom jumps to at-or-above threshold + margin in one probe goes straight to ok by rule 2 — the clear condition is met, and nothing requires it to dwell a probe in low_water first. The clear threshold gates a return from an alarm state; it is not the general rule for reaching ok. There is no hysteresis between the two alarm states, so recovery is never held back, and a store that jumps from non-positive headroom to at-or-above the clear threshold in one tick is not hovering.
margin is defined exactly, because it is test-visible:margin = max(audit_store_low_water_bytes / 10, AUDIT_STORE_MIN_HYSTERESIS_MARGIN_BYTES), where the division is u64 integer division truncating toward zero and AUDIT_STORE_MIN_HYSTERESIS_MARGIN_BYTES is a file-top constant of 1048576 (1 MiB). The floor is not decoration: audit_store_low_water_bytes is operator-configurable, and on any value below 10 bytes the truncating division yields a margin of zero, which silently deletes the anti-flap rule the constant exists to provide. 1 MiB is the chosen floor because it is far larger than any single record this store can append — a record is bounded well under the 64 KiB minimum file bound audit_max_file_bytes enforces — so no single write can carry the store across the clear threshold and back. The clear comparison is performed in i64, with threshold + margin computed as a saturating add: a threshold near the reserve's own i64::MAX bound can push the sum past it.
A failed probe leaves the previous state and its previous measured_at in place — it does not reset the state to unknown, which would hide an alarm behind a probe failure.
One configuration rule this issue adds
validate_registrar_settings already rejects audit_store_low_water_bytes >= audit_store_reserve_bytes and audit_store_reserve_bytes > i64::MAX. Add the one missing bound: reject audit_store_low_water_bytes == 0 at load, with its own diagnostic naming the key. At zero the low_water band is empty, so the state machine would step straight from exhausted to ok and the alarm this issue exists to raise would never fire. Disabling the alarm is not an offered mode, and a value that silently disables it is worse than one that is refused. Together with the existing upper bound this gives 0 < audit_store_low_water_bytes < audit_store_reserve_bytes, which also settles representability: headroom never exceeds the reserve, the reserve is bounded at i64::MAX, so a threshold below it converts to i64 exactly and the rule-2 and rule-3 comparisons need no clamp. This is the only configuration change in scope; add no key, and change no other key's validation.
Running the probe and the scan without blocking the runtime
The maintenance callback run_rotation_loop_with_maintenance takes today is a synchronous FnMut() invoked inside the rotation loop's tokio::select! branch body, and all it does now is copy two atomic counters. A directory walk over the store and a scan that reads up to the record store's full ceiling are both blocking filesystem work and must not run there.
Change the maintenance seam to an async callback, awaited in the same branch body that already awaits rotation.run_pass(...). That body runs to completion once its timer branch is selected, so the awaits inside it are not cancelled part-way by the shutdown arm.
Run both filesystem operations under tokio::task::spawn_blocking and await each handle inside the tick. scan_audit_store_off_runtime already is exactly that wrapper for the record scan — call it rather than writing a second one. Give the capacity probe the equivalent. Awaiting the handles inside the tick is what keeps this clear of the orphan-task rule: nothing is spawned and dropped, and a join failure is a failed probe or a failed scan under the rules above rather than a silent gap.
The cadence is the existing one and stays ROTATION_INTERVAL. Do not add a second scheduler, a second interval or a second long-lived task. That interval is what bounds staleness: in a healthy daemon measured_at and records_measured_at are never more than about a minute old, and the documentation says so, because an operator reading a timestamp needs to know what "fresh" looks like.
The record-store signals
The three record signals are read, not re-derived. Call scan_audit_store — through scan_audit_store_off_runtime — with registrar.audit_record_dir, the tick's now, AUDIT_SCAN_WINDOW, registrar.audit_max_retained_files and registrar.audit_min_retain_days, exactly the five arguments bootroot status passes, and map the returned AuditScan's three fields onto the members below. Note the directory: the record scan reads audit_record_dir, which defaults to <audit_store_dir>/records, while the capacity walk reads audit_store_dir as a whole. Re-deriving any of the three signals here would put a second definition of "anomaly" and "shortfall" in the tree, and the two would disagree the first time either changed its window or its retention rule. (retention_short on the reader and retention_shortfall on the wire are one signal under two spellings, internal versus wire, not two signals.)
The window is AUDIT_SCAN_WINDOW and it is reused, not restated. It is already pub in src/registrar/audit/scan.rs — pub rather than pub(crate) because its other consumer, bootroot status, is in the binary crate — so pass it and widen nothing. Do not declare a second constant and do not write a 30-day duration into this issue's production code: two definitions is exactly how the relayed value and the host-local bootroot status line drift apart, and this issue's acceptance criteria require the two surfaces to agree. The bar is on a second code definition of the window and nothing else — the docs/ pages this issue updates name the 30-day window in prose because operators need it, and tests that age fixtures across the window are expected; those derive the age from AUDIT_SCAN_WINDOW rather than restating the number.
The scan runs on the health tick, not per response. The reader computes its counts by reading the store's files on demand rather than from an in-memory counter, so calling it inside the request path would make every successful mint and deregister read up to the store's full 136 MiB ceiling. It runs on the same tick as the capacity probe, and the response relays the last snapshot exactly as it relays the last probe result.
A failed scan leaves the previous values and records_measured_at in place, for the same reason a failed probe leaves the previous state in place: reporting zero anomalies and zero malformed lines because the scan itself failed would hide the alarm behind the failure. Those members are absent only before the first successful scan.
The health member
Extend the existing RegistrarHealth with exactly one new member, audit_capacity, appended after limiter so no existing member's serialized position changes, and carry it wherever the container is already carried — mint success, deregister success and refusal. Do not reshape the container, and do not read, write, reorder or otherwise touch limiter. That the container rides refusals is what makes this signal reachable at all. The store this issue measures is a fail-closed control: as it fills, invocations are refused — so a success-only container would stop carrying the low-water alarm in exactly the state the alarm exists to announce, and an operator would learn about the exhausted reserve from the enrollment outage instead.
Populate the existing shared holder; create none. Extend the RegistrarHealth the daemon constructs in src/daemon.rs with the new member's starting value — unknown, with enforcement, reserve_bytes and low_water_bytes read from the settings the daemon already has — and extend refresh_registrar_health to write the tick's probe and scan results into it. The new member's Default exists only to keep the container's derived Default (which tests use) compiling; production never serializes it, because the daemon builds the member from configuration before the handler can answer anything.
registrar_health.audit_capacity carries exactly these members:
state — an enum, not a bool and not a free string, with the four values and the exact rules above. Always present.
enforcement — an enum mirroring audit_store_enforcement, filesystem or directory. Always present. A console cannot otherwise tell a kernel-enforced reserve from a configured estimate, and the two warrant different operator responses to the same headroom number; shipping the alarm without the mode would let a directory-mode deployment read as protected. AuditStoreEnforcement in src/config.rs derives Deserialize and #[serde(rename_all = "snake_case")] but not Serialize — add Serialize to it rather than declaring a second enum, so the wire spelling and the configuration spelling cannot drift.
reserve_bytes — u64; the configured audit_store_reserve_bytes. Always present.
low_water_bytes — u64; the configured threshold, so a console can render the alarm without knowing the daemon's configuration. Always present.
used_bytes — u64; the store's measured usage. Present only when a probe has succeeded.
headroom_bytes — signed (i64); the min computed above. Signed, not unsigned: a store that has overrun its reserve has negative headroom, and an unsigned field would clamp that to zero and report the overrun as the healthiest possible value. Present only when a probe has succeeded.
measured_at — RFC 3339 timestamp in UTC; the probe's last successful run. Present only when a probe has succeeded. Without it an operator cannot tell a healthy signal from a stale one left by a probe that stopped running.
intent_without_outcome — u64 count of unpaired intent records over the scan window. Present only when a scan has succeeded.
malformed_records — u64 count of lines the scan could not parse, over the same window. Present only when a scan has succeeded. A malformed line is what a forged or parser-breaking record attempt looks like, and the record store's escaping rule exists precisely because attacker-influenced bytes reach that log by design. Relaying the anomaly and retention signals while dropping this one would leave the console blind to the single attack that rule defends against.
retention_shortfall — bool, the reader's retention_short. Present only when a scan has succeeded. true when the store is at its maximum retained generations and its oldest retained record is newer than now - audit_min_retain_days, i.e. the hard size ceiling is winning against the soft retention target. It is the signal that says the look-back window the whole detection argument rests on has quietly shrunk, so a design that surfaces capacity but not retention is reporting the less important of the two.
records_measured_at — RFC 3339 timestamp in UTC; the scan's last successful run. Present only when a scan has succeeded. It is a second timestamp because the capacity probe and the record scan fail independently — an unreadable record store fails the scan while statvfs still succeeds — and a single measured_at would let one half of the payload go arbitrarily stale with nothing on the wire to show it.
state describes the capacity half only: the record signals and records_measured_at come from the store scan, which succeeds or fails independently, so their presence is not governed by state and a response may carry a healthy ok alongside absent record signals or the reverse.
Follow the container's existing wire idioms. Optional members are omitted rather than serialized as null, and deserialize through the module's reject_null_option so an explicit null is refused rather than read as absence. Both timestamps are formatted the way material.expires_at already is — time's Rfc3339 formatter over a UTC OffsetDateTime, producing a Z string. The two enums serialize snake_case.
Populating the member changes the encoded success and refusal messages, so extend the golden serialization fixtures additively — mint-success.json, deregister-success.json, refusal-permanent.json, refusal-busy.json and refusal-unclassified.json under src/registrar/endpoint/fixtures/ all carry registrar_health — and extend docs/reference/registrar-wire-contract.md with the audit_capacity schema, beside the limiter paragraph it already documents. A stale fixture is a silent cross-repo break.
Document the alarm thresholds and their defaults, what each state value means, the hysteresis behaviour, and how to read the two timestamps and what bounds their staleness, in bothdocs/en/ and docs/ko/, on the existing pages, with no mkdocs.yml nav change.
Acceptance criteria
The low-water alarm fires before the reserve is consumed, computed as the min of the configured-budget headroom and the backing filesystem's available bytes; a test drives the capacity probe abstraction over a tempfile::tempdir() and asserts the alarm is on at headroom_bytes == audit_store_low_water_bytes, off at low_water + 1 from a cold start, does not clear while headroom sits between the threshold and low_water + margin, and clears at low_water + margin. A second test exercises the real fstatvfs-backed probe against a tempdir and asserts plausible non-zero values.
state follows the three ordered rules exactly; a test covers unknown before any probe, ok, low_water at and just below the inclusive threshold, exhausted at zero headroom, exhausted at negative headroom, the hysteresis-gated return to ok, the immediate exhausted → low_water transition on the first positive headroom below the clear threshold, the direct exhausted → ok transition when one probe reaches threshold + margin, and a failed probe leaving the previous state and measured_at intact.
The hysteresis margin is max(audit_store_low_water_bytes / 10, 1 MiB); a test asserts the floor applies for a small configured low-water value where the 10% term truncates to zero, that the 10% term applies at the default, and that a threshold near i64::MAX saturates the clear sum rather than wrapping it.
Headroom conversion is saturating, not wrapping; a test drives used_bytes and filesystem_available_bytes above i64::MAX and asserts the resulting state is neither a spurious ok nor a spurious exhausted. A separate test drives a statvfs result whose f_bavail × f_frsize overflows u64, one whose (f_blocks − f_bfree) × f_frsize overflows u64, and one walked entry whose st_blocks × 512 overflows u64, and asserts each saturates to u64::MAX and the probe still succeeds.
audit_store_low_water_bytes = 0 is rejected at configuration load with a diagnostic naming the key; a test asserts the rejection and that the existing reserve and upper-bound rules still reject what they rejected before.
The alarm and the intent-without-outcome count are readable from the endpoint's health response, not only from the bootroot host's filesystem.
They are carried in registrar_health.audit_capacity with the state, enforcement, reserve_bytes, low_water_bytes, used_bytes, headroom_bytes, measured_at, intent_without_outcome, malformed_records, retention_shortfall and records_measured_at members specified in Scope — state and enforcement enums, headroom_bytes signed, both timestamps RFC 3339 UTC, the three capacity measurement members absent exactly when state is unknown, and the four record members absent exactly before the first successful scan — leaving limiter byte-identical; a test asserts a response carrying both members round-trips and that adding this one changed no limiter byte.
An optional member is omitted rather than emitted as null, and an explicit null in a decoded payload is rejected rather than read as absence; a test covers both directions for one optional member of each type.
enforcement is always present and mirrors the configured mode, so a directory-mode deployment cannot be mistaken for an enforced reserve.
The pre-registrar audit_unwritable refusal path still serializes an empty registrar_health object ({}); a test asserts it gained no member.
The three record signals are read from scan_audit_store called with AUDIT_SCAN_WINDOW, audit_record_dir, audit_max_retained_files and audit_min_retain_days; a test asserts the relayed values equal the reader's own output for the same store and window, and this issue's production code declares no window value of its own — neither a second constant nor an inline 30-day duration at the call site. Prose in docs/ and fixture ages in tests are not covered by that bar; test fixtures derive their ages from AUDIT_SCAN_WINDOW.
The relayed malformed_records value is exact, not merely present: a test appends a known number of unparseable lines to the store, drives a health tick, and asserts the health response reports that count and that bootroot status reports the same one. A malformed line in a rotated generation the reader does not select — a surplus generation beyond audit_max_retained_files — is counted by neither. Do not assert anything about a malformed line's age: an unparseable line has no timestamp to age, and the reader deliberately selects the newest pre-window generation as a boundary file so that intents and outcomes match across the window's edge, so malformed content in that file is counted by design. An age-based assertion could only be satisfied by re-deriving the reader's selection rule here, which this issue forbids.
The retention shortfall reaches the console: a store forced into the reader's derived shortfall reports retention_shortfall = true on the health response and on bootroot status, and a healthy store reports false on both — so the host-local surface and the relayed one cannot disagree.
The scan runs on the health tick and not in the request path; a test asserts a mint response performs no store scan of its own and relays the last snapshot, and that a failed scan leaves the previous three values and records_measured_at unchanged rather than zeroing them.
The probe and the scan run on the daemon's existing rotation-loop maintenance tick at ROTATION_INTERVAL; no second scheduler, interval or long-lived task is added, both filesystem operations run under spawn_blocking, and every spawned handle is awaited inside the tick rather than dropped.
Usage is measured per enforcement mode: a test asserts filesystem mode derives it from the same fstatvfs call on the store root descriptor and performs no directory walk, while directory mode walks audit_store_dir and reports a value that tracks a file written into the store — and that available bytes come from f_bavail rather than f_bfree in both. A walk that cannot read part of the store fails the probe rather than summing the unreadable part as zero.
The directory-mode walk resolves no symlink and counts no file twice: a test builds a store holding a symlink to a large file outside it, a symlink to a directory, two hard links to one file, a FIFO, a sparse file and a nested real subdirectory, then asserts the reported usage equals the summed allocated blocks of the distinct real objects plus the link entries themselves — unmoved by the symlink targets' sizes, counting the hard-linked file's blocks once, and counting the sparse file by its blocks rather than its apparent length. The expected total is computed from each fixture's own reported st_blocks, never from an assumption that a class of entry contributes zero.
The walk's syscall operations sit behind a test-only seam that can fail one named operation on one named path with one named errno, and the five behaviours below are driven through it rather than staged on disk. Every other walk test runs against a real tempdir.
A symbolic link at audit_store_dir itself fails the probe in both enforcement modes and leaves the previous state and measured_at intact, rather than being followed: a real-filesystem test points the configured store path at a symlink to a directory holding known bytes and asserts none of them are reported and no new measured_at is stamped — including in filesystem mode, where there is no walk and the root open is the only step that can refuse the link.
The walk descends by descriptor: a seam-injected ELOOP/ENOTDIR at the openat that descends into an enumerated directory surfaces as a probe failure rather than being followed, and that walk contributes no usage at all — the previous state and measured_at stand. A companion real-filesystem test plants a symlink to a directory outside the store and asserts the reported usage is unmoved by what that target holds.
The walk never enumerates itself or its parent: a real-filesystem test builds the store as a subdirectory of a tempdir, writes a large file and a populated subdirectory beside it in that parent, and asserts the reported usage equals the store subtree's own total and is unmoved when those parent-side bytes grow — proving . and .. are neither counted nor traversed.
The walk distinguishes a race from a failure: a seam-injected ENOENT at the openat/fstatat following a readdir is skipped and the probe still succeeds, while a seam-injected EACCES on a subdirectory fails the probe and leaves the previous state and measured_at intact.
A null readdir return with a nonzero errno fails the probe rather than reading as end-of-directory; a seam-injected EIO mid-directory asserts it, and the probe does not report the short total it had accumulated.
Every unsafe block is minimal and carries a // SAFETY: comment stating its invariant, and no descriptor ever has two owners or none: the descriptor is handed to fdopendir with into_raw_fd() and not as_raw_fd(), and a null fdopendir return closes the descriptor its openat produced before propagating the failure. The seam's open and close counts balance for a walk that succeeded, for one that failed part-way through a subdirectory, and for one whose fdopendir failed after a successful openat.
measure_underlying in src/commands/audit_store/reserve.rs is unchanged and the bootroot init reserve preflight is not rewired.
The alarm thresholds, the state values, the hysteresis behaviour, the two timestamps and what bounds their staleness are documented in bothdocs/en/ and docs/ko/, with no mkdocs.yml nav change, and docs/reference/registrar-wire-contract.md documents the audit_capacity schema.
The CI check job's Rust and documentation gates pass as CI runs them: cargo fmt -- --check --config group_imports=StdExternalCrate, cargo clippy --all-targets -- -D warnings, cargo doc --no-deps --document-private-items with RUSTDOCFLAGS=-D warnings — the new module, its trait, its enums and the new health member all carrying rustdoc — markdownlint over the changed Markdown, and ./scripts/check-docs.sh for the docs/ changes.
The pre-push preflight has been run — or the carve-out below has been exercised — and the outcome is stated in the pull request body. This change touches daemon scheduling, configuration and the endpoint, so it is not eligible for the E2E exemption: run scripts/preflight/run-all.sh, and at minimum scripts/preflight/ci/e2e-matrix.sh, whose step 13 exercises bootroot init on an endpoint-enabled loopback host — the very mode this issue's filesystem-mode measurement reads. Where the environment cannot run the matrix — it cannot supply the passwordless sudo that matrix step 13 needs, or the matrix's own setup would tear down an unrelated live bootroot-* Compose project — run the matrix as far as it safely goes, which may be no arm at all, and say in the pull request body which arm ran and passed locally, which did not run, and why, leaving CI's Docker E2E jobs to gate the arms that did not run, as AGENTS.md's "CI Requirements" section provides. Do not weaken, skip, continue-on-error or delete a check to make a local run pass. (Exercised by Probe the audit store's capacity and report it on the endpoint (#774) #965: no arm could run on the implementing host without destroying an unrelated live bootroot-* stack; the pull request body reports which arms did not run and why, and CI's Docker E2E matrix was green on every arm, including registrar-internal-init.)
Constraints
Do not re-derive the anomaly, malformed-line or retention signals, and do not introduce a second lookback window for them. Call scan_audit_store and pass AUDIT_SCAN_WINDOW — a second definition of either the scan or the window makes the relayed values and the host-local bootroot status ones drift silently apart.
Do not call that reader from the request path. It reads the store's files on demand, so a per-response call puts a full-store read on every successful mint.
Do not add a second scheduler, a second interval or a second long-lived task. The capacity probe and the record scan run on the daemon's existing rotation-loop maintenance tick.
Do not run blocking filesystem work on a runtime worker, and do not spawn a task whose handle is then dropped.
Do not walk the store from the request path, and do not walk it at all in filesystem mode, where one fstatvfs call already answers usage exactly.
Resolve the configured store path exactly once per tick: the root open is that resolution, and available bytes come from fstatvfs on the descriptor it returns, never from a second pathname lookup. Never fstatat, openat or account for the . and .. entries readdir returns. Do not resolve a path a second time during the walk, do not follow a symlink, and do not cross a device boundary out of audit_store_dir. openbao/ is written by the container's uid, so an entry planted there must never be able to point this measurement at bytes outside the store. Directories are opened only to enumerate them; nothing else is opened and no walked entry's contents are read.
Do not change measure_underlying or the bootroot init reserve preflight, and do not lift a shared measurement into the library crate.
The walk's test seam carries no production behaviour. Production takes exactly one path through it — the real syscalls — with no branch, flag or environment read that a deployed daemon could take differently, and no failure injection reachable outside #[cfg(test)]. A seam that can change what a running daemon measures is a second implementation of the control, not a test aid.
Do not test the walk's real-filesystem behaviour through the seam. The symlink, hard-link, sparse-file, special-file, nested-subdirectory and same-device cases are staged on a real tempfile::tempdir(), because a double that does not follow symlinks proves nothing about the code that does the following.
Do not synchronise a walk test with a sleep or a second thread mutating the store mid-walk, and do not assert on a process-global descriptor count.
No descriptor may have two owners or none. fdopendir receives the descriptor by into_raw_fd(), never as_raw_fd(); after it succeeds closedir is the only close; when it returns null the caller closes the descriptor itself.
Do not add a health snapshot holder. Use the Arc<Mutex<RegistrarHealth>> the daemon already builds.
The health member must be additive. Do not reshape the registrar_health container, do not touch the limiter member's shape, contents or ordering, and do not add a member to the empty health object the pre-registrar audit_unwritable refusal path serializes.
Use checked conversions with an explicit clamp for the headroom arithmetic; no as cast between the u64 inputs and the i64 result, and none in the block-count products.
Do not encode "we have not measured" as ok, and do not let a failed probe or a failed scan reset a signal to a healthy-looking value. Absence before the first success and staleness afterwards are both visible on the wire by design.
Do not claim, in code comments, error text or documentation, that the reservation keeps OpenBao serving. It bounds the blast radius to the audit artifacts; OpenBao still fails on its own unwritable device once the reserve is full.
The only configuration change in scope is rejecting audit_store_low_water_bytes == 0. Add no configuration key, change no other key's validation, provision nothing, render no Compose override, and create or verify no mount.
No unwrap() in production code; no [] indexing; keep each unsafe block minimal and preceded by a // SAFETY: comment.
Out of scope
Provisioning the reserved store — the directory layout and its ownership, the location of audit_record_dir, the Compose override that puts the OpenBao audit device on the store, the loopback-backed filesystem and its systemd mount unit, the mount-point verification at daemon start, the audit_store_* keys themselves and the never-degrade-to-directory rule. All of it is merged and this issue only reads it.
Defining the record encoding, the rotation bounds, the anomaly scan itself, its store reader, AUDIT_SCAN_WINDOW or audit_record_dir. This issue calls the reader and reuses the constant as they already exist.
The certificates member of registrar_health and the certificate-lifetime reporting behind it. That member has its own owner; this issue adds audit_capacity and touches nothing else in the container.
The limiter member, the two token buckets, the rate_limit_* keys, the coalesced counted records and RegistrarBusy { retry_after_seconds } — all merged, all read-only here.
Rotating, pruning or capping the OpenBao audit device's own growth.
The control-plane side of rendering the relayed health values.
Test plan
Capacity tests: the alarm is on at headroom_bytes == audit_store_low_water_bytes and off at low_water + 1 from a cold start, honours the hysteresis margin on the way back, and clears at low_water + margin — driven through the probe abstraction over a tempfile::tempdir(); plus one test of the real fstatvfs-backed probe reporting plausible non-zero values.
state table test covering unknown, ok, low_water at and below the inclusive threshold, exhausted at zero and at negative headroom, all three transitions including exhausted → ok in one probe, and a failed probe preserving the previous state and measured_at.
Usage-measurement tests: directory mode over a tempdir holding files of known sizes reports their total and tracks a newly written file; filesystem mode takes its usage from the same fstatvfs result on the root descriptor and performs no walk; an unreadable subdirectory during a walk fails the probe rather than under-reporting; available bytes are f_bavail-derived.
Root-symlink test over a real tempdir, run in both enforcement modes: a configured audit_store_dir that is a symbolic link to a real directory fails the probe rather than being followed, preserving the previous state and measured_at; no seam injection is needed, since the fixture is deterministic and needs no privilege.
Walk entry-type tests over a tempdir store: a symlink to a large file outside the store and a symlink to a directory add only their own entries' size and are not traversed; two hard links to one file count its blocks once; a FIFO — mkfifo needs no privilege, where a device node would need root — is counted from its own fstatatst_blocks like any other non-directory entry, is neither opened nor traversed, and does not block the walk; a sparse file counts by allocated blocks, not apparent length; a nested real subdirectory on the same device is descended into and its own blocks counted.
Dot-entry test over a real tempdir: with a large file and a populated subdirectory in audit_store_dir's parent, the walk reports the store subtree's own blocks only and is unmoved when those parent-side bytes grow — . and .. are neither counted nor descended into.
Seam-injected walk-failure tests, one per errno and injection point: ELOOP/ENOTDIR at a descent openat (substitution) fails the probe and contributes no bytes; ENOENT at the openat/fstatat after a readdir (vanish) is skipped and the probe succeeds; EACCES on a subdirectory fails the probe and leaves the previous state and measured_at intact; EIO from readdir fails the probe rather than truncating the directory; a null fdopendir return after a successful openat fails the probe.
Descriptor-hygiene test through the seam: open and close counts balance for a successful walk, for one aborted by a failure injected before the ownership transfer, and for one aborted by an injected fdopendir failure after it — with no reliance on /proc/self/fd, on any process-global count, or on an fcntl check of a descriptor number another thread may have been handed.
Boundary-arithmetic tests: the margin floor applies where low_water / 10 truncates to zero; the clear sum saturates for a threshold near i64::MAX; usage and available values above i64::MAX clamp rather than wrap; f_bavail × f_frsize, (f_blocks − f_bfree) × f_frsize and st_blocks × 512 each saturate to u64::MAX rather than wrapping or failing the probe.
Configuration test: audit_store_low_water_bytes = 0 is rejected at load with a diagnostic naming the key, and the existing reserve and upper-bound rules are unchanged.
Endpoint test asserting registrar_health.audit_capacity carries every specified member with the specified types and presence rules, that a response carrying it alongside limiter round-trips, that no limiter byte changed, that an optional member is omitted rather than null, and that an explicit null is rejected on decode.
Enforcement-mode test asserting enforcement is always present and mirrors the configured mode in both filesystem and directory deployments, over an otherwise identical capacity payload.
Refusal-path test asserting the pre-registrar audit_unwritable response still serializes "registrar_health":{}.
Retention-shortfall relay test: a store forced into the reader's derived shortfall reports retention_shortfall = true on the health response and on bootroot status, and a healthy store reports false on both.
Malformed-record relay test: append a known number of unparseable lines, drive a health tick, and assert the exact malformed_records value on the health response, the same value on bootroot status, and that a malformed line in a surplus rotated generation beyond audit_max_retained_files — one the reader does not select — is counted by neither, with no assertion made about a malformed line's age.
Window-reuse test asserting the reader is called with AUDIT_SCAN_WINDOW and that this issue's production code carries no window value of its own; a fixture that must fall outside the window derives its age from that constant rather than hardcoding 30 days.
Scan-cadence tests: a mint response triggers no store scan and relays the last snapshot; a failed scan preserves the previous three record values and records_measured_at; before the first successful scan those four members are absent while the capacity members are present.
Golden serialization fixtures extended additively for all five response fixtures that carry the container — mint-success.json, deregister-success.json, refusal-permanent.json, refusal-busy.json, refusal-unclassified.json — asserting no limiter byte changed.
Run the bootroot status agreement tests with cargo test --bin bootroot: status lives in the binary crate, so cargo test --lib would silently skip them. The endpoint tests are Linux-only, since src/registrar/endpoint is gated on target_os = "linux". The probe's own module carries #[cfg(any(target_os = "linux", test))] — Linux-only in a build, because the daemon that drives the measurement is, and compiled under cfg(test) everywhere so an unconditional module is not dead code on every other target — so its tests run wherever the suite does.
Tests use tempfile::tempdir() and never a fixed path, and mutate no process environment.
Dependencies
Nothing this issue needs is outstanding. The reserved store, its four audit_store_* configuration keys and their validation, the kernel-enforced reserve behind filesystem mode, the record store's reader and its 30-day window constant, the registrar_health container, its placement on all three response shapes and the daemon-held snapshot that feeds it are all merged and in the default branch; this issue reads them and adds to them.
It is independent of the two remaining members of that container. Each of the three adds a disjoint member, so whichever lands last is purely additive.
src/config.rs — the [registrar] table holding the four audit_store_* keys this issue reads, and AuditStoreEnforcement, which needs Serialize added
src/config/validation.rs — validate_registrar_settings, where the reserve and low-water bounds live and where the zero rejection goes
src/registrar/audit_store.rs — records_dir(), openbao_dir(), layout_of() and the store's ownership and mode contract; the natural parent for a new capacity submodule, mirroring audit.rs and its audit/scan.rs child
src/registrar/audit/scan.rs — scan_audit_store, scan_audit_store_off_runtime, AuditScan and pub const AUDIT_SCAN_WINDOW
src/commands/status.rs — the five arguments bootroot status passes to the reader, and the host-local surface the relayed values must agree with
src/registrar/endpoint/protocol.rs — RegistrarHealth, the three response shapes that carry it, UnavailableRegistrarHealth, the reject_null_option module, and the fixture-backed golden serialization tests
src/registrar/endpoint/fixtures/ — the five response fixtures carrying registrar_health
src/daemon.rs — where the shared Arc<Mutex<RegistrarHealth>> is built, handed to ProductionHandler::with_health, and refreshed by refresh_registrar_health, and where the rotation loop's maintenance closure is passed in
src/registrar/openbao_audit.rs — run_rotation_loop_with_maintenance, its tokio::select! body and ROTATION_INTERVAL
src/commands/audit_store/reserve.rs — ReserveProbe, HostProbe::space (a pathname statvfs wrapper and its SAFETY comments), ST_BLOCKS_UNIT_BYTES and measure_underlying; read for the idioms, changed by nothing here
src/fast_poll.rs and src/hooks.rs — the OffsetDateTime::format(&Rfc3339) idiom
src/openbao.rs (verify_audit_file) — the mandatory device whose space this measures
docs/en/operations.md and docs/ko/operations.md — the audit-logging section and "The shared audit store" section
docs/en/configuration.md and docs/ko/configuration.md — where the audit_store_* keys and their defaults are documented
docs/reference/registrar-wire-contract.md — the registrar_health paragraph and the response member-order tables
src/registrar/internal/tests.rs — the repository's existing assert_ne!(current_process_euid(), 0, ...) guard on a chmod 0o000 unreadable-file test, for contrast with the seam this issue asks for
scripts/preflight/run-all.sh and scripts/preflight/ci/e2e-matrix.sh — the pre-push preflight
.github/workflows/ci.yml — the check job's exact fmt, clippy, rustdoc and docs invocations
docs/rfcs/0001-registrar-role-and-non-self-propagation.md §5.6 and §6
Probe the reserved audit store's capacity and report it on the endpoint
Context
The reserved audit store is built, the reserve behind it is real, and the mode that decides what stands behind it is settled. What is missing is measurement.
[registrar]in thebootroot-agentconfiguration carriesaudit_store_dir,audit_store_reserve_bytes(default2147483648, 2 GiB),audit_store_low_water_bytes(default536870912, 512 MiB) andaudit_store_enforcement(filesystemby default,directorythe explicit opt-out). They are declared insrc/config.rsand checked byvalidate_registrar_settingsinsrc/config/validation.rs, which already rejects a reserve abovei64::MAX, a low-water value at or above the reserve, and anaudit_record_dirthat resolves outside the store.src/registrar/audit_store.rsowns the layout:records_dir()andopenbao_dir()resolve<audit_store_dir>/recordsand<audit_store_dir>/openbao, both created root-owned and0700.openbao/does not stay that way by design — the OpenBao container's entrypoint chowns and chmods its bind-mount source on first start, and bootroot deliberately neither asserts nor repairs that directory afterwards, because a root-owned file in a directory the container cannot chown is a device that cannot write.In
filesystemmodebootroot initprovisions a fully allocated loopback image of exactlyaudit_store_reserve_bytescarrying an ext4 filesystem, mounted ataudit_store_dirthrough a generated systemd mount unit that is restored on boot, so a write past the reserve fails withENOSPCon the reserve rather than filling the host's root filesystem; an endpoint-enabled daemon that finds the store is not a mount point at start keeps running but refuses every verb with a permanentregistrar_unavailablecarrying reasonaudit_unwritable. Indirectorymode the store is a plain directory sharing the host's root filesystem and the reserve is a budget with nothing behind it.A hard ceiling tells nobody it is about to be reached. Both audit artifacts are fail-closed inputs to a live security argument: OpenBao's file audit device is mandatory and OpenBao fails requests it cannot audit, and the verb records are the only detection for an abuse whose signature is a rate. A reservation with no measurement turns a slow fill into a sudden outage on the one host that must not be restarted, and a look-back window that has quietly shrunk is invisible.
The record store's own signals exist already, and they stop at the bootroot host.
scan_audit_storeinsrc/registrar/audit/scan.rsreads the store without creating or modifying anything and returnsintent_without_outcome,malformed_recordsand a derivedretention_shortflag together in oneAuditScan.bootroot statuscalls it — through thespawn_blockingwrapperscan_audit_store_off_runtime— overaudit_record_dir, withpub const AUDIT_SCAN_WINDOW: Duration = Duration::days(30),audit_max_retained_filesandaudit_min_retain_days, and prints the three values host-locally. Nothing carries them off the host. The daemon is not a control-plane protocol peer and has no channel of its own; the co-located registrar relays what the daemon exposes on the mint/deregister endpoint.That relay already has both a container and a shared holder, so this issue plugs into them rather than building either.
RegistrarHealthinsrc/registrar/endpoint/protocol.rsis carried on all three response shapes — mint success, deregister success and refusal — and today holds one member,limiter.src/daemon.rsbuilds oneArc<Mutex<RegistrarHealth>>, hands it toProductionHandler::with_health, and refreshes it fromrefresh_registrar_healthon the OpenBao rotation loop's existing maintenance callback, which fires everyROTATION_INTERVAL— one minute. The response encoder receives that one daemon-held snapshot and has no other source for it.This issue adds a capacity measurement, an alarm over it, one new member on that existing container, and one new body of work on that existing tick. It provisions nothing, mounts nothing, adds no scheduler and creates no holder.
Scope
The capacity probe
audit_store_dir: the store's usage in bytes, and the available bytes on the filesystem backing it.libc::fstatvfson an open descriptor for the store root, not from a pathnamestatvfs(libcis already a direct dependency, so this needs no new crate):f_bavail × f_frsize, the space available to an unprivileged writer, notf_bfree, which counts blocks reserved for root and would report headroom the writers cannot actually use. The descriptor is the one described below, opened once per tick withO_NOFOLLOWin both enforcement modes; a pathnamestatvfswould follow a symbolic link at the store root, which this issue requires to fail the probe.statvfscannot answer it in one of them.statvfsdescribes a filesystem, not a subtree, so a single implementation is wrong in one mode or the other:filesystemmode the store is its own filesystem, so its usage is that filesystem's used space —(f_blocks − f_bfree) × f_frsizefrom the samefstatvfscall on the same descriptor. One syscall, exact, no walk.directorymode the store shares a filesystem with the rest of the host, wheref_blocks − f_bfreemeasures everything on the host and not the store. Usage there is measured by walkingaudit_store_diron the tick and summing allocated size —st_blocks × 512— rather than apparent length, so a sparse or block-rounded file is not under-counted against a ceiling the kernel enforces in blocks.st_blocksis in fixed 512-byte units by POSIX, independent of the filesystem'sf_frsize; give that unit a named file-top constant rather than a512at the call site.filesystemmode, where the syscall already answers it, and do not cache a stale usage across ticks to avoid the walk.fstatvfson the root descriptor for available bytes in both modes, and for usage either that same call or a walk descending from that same descriptor, per the mode rule above) and a test implementation. The alarm and hysteresis logic are then driven against the trait over atempfile::tempdir()with a synthetic reserve, so "the alarm fires before the reserve is consumed" is an ordinarycargo testneeding no special filesystem, no root and no container. One test additionally exercises the real implementation against a tempdir to prove it reports plausible non-zero values.ReserveProbeandHostProbeinsrc/commands/audit_store/reserve.rsare the shape to copy, not the code to share — see below for why the two stay separate.The root descriptor, and the descriptor-based
directory-mode walkThe store root's open and the walk's mechanism are pinned here rather than left to the implementer, because part of the store is attacker-influenced by design:
openbao/is owned by the OpenBao container's uid, so every entry the container writes there — including entries a compromised container writes — is inside the subtree this walk sums.O_NOFOLLOW— and it is opened in both enforcement modes. Openaudit_store_diritself withO_NOFOLLOW | O_DIRECTORY | O_RDONLY | O_CLOEXECon its configured absolute path. That one descriptor is the whole probe's anchor:fstatvfson it yields the available bytes and, infilesystemmode, the usage; itsfstatestablishes both thest_devevery entry below is compared against and the first(st_dev, st_ino)in the seen set; and indirectorymode the walk descends from it. Binding both halves ofheadroom_bytesto one descriptor is what makes them describe one object — a pathnamestatvfsbeside a separately opened walk root resolves the configured path twice, and nothing holds the two resolutions to the same directory. Infilesystemmode nothing is enumerated, so the root is opened,fstat-ed,fstatvfs-ed and closed again without a singlereaddiron it. The open-directory operation the test seam below defines spans bothopenatandfdopendir, and it is that one operation which opens the root in both modes, so the root's stream is created and released rather than read; keeping a second, stream-less open for this mode would put the ownership transfer at two call sites instead of one, which is the thing that seam exists to prevent. A symbolic link at the store root is therefore anELOOPfrom that open and a failed probe in either mode, leaving the previous state andmeasured_atin place, exactly as a substituted directory further down is. Without it,filesystemmode would have nothing that refuses such a link, and a pathnamestatvfswould quietly report the target's filesystem. That is not a new rule this issue invents: the store contract already refuses it.check_store_directoryinsrc/registrar/audit_store.rsstats the path withsymlink_metadataso a planted link is seen as a link rather than as whatever it points at, reporting it asPathFault::Symlink, andcheck_ancestorsrefuses one at every component above the store. Nor does an operator's own configured path make the object at it safe later:O_NOFOLLOWchecks what is there at open time, the same time-of-check problem the descent rule exists for. It covers the final component only; the components above the store are the store contract's business, checked where the layout is created and verified, and this walk does not re-derive them. Any error from that open is a failed probe on the same terms —ELOOPfor a planted link,EACCES, andENOENTfor a store path that is not there at all. The vanished-entry carve-out below is about entries areaddirlisted, and does not reach the root.audit_store_diris opened withopenatfrom its parent's descriptor usingO_NOFOLLOW | O_DIRECTORY | O_RDONLY | O_CLOEXECand enumerated withfdopendir/readdiron that same descriptor.fstaton that descriptor supplies both the(st_dev, st_ino)identity and thest_blocksthe walk sums, so the object that is classified is the object that is traversed. A path-basedlstatfollowed by a path-basedread_dircannot guarantee that: a directory can be replaced by a symlink between the two resolutions, and indirectorymode the store shares the root filesystem, so the same-device rule would not exclude a link pointing at/. A followed link there would sum the whole root filesystem intoused_bytes, drive headroom far negative, reportexhaustedand refuse the registrar verbs — a denial of service reachable from the subtree this issue already declares attacker-influenced..and..by name, before anything else.readdirreturns both in every directory, and nothing filters them out the waystd::fs::read_dirdoes — that convenience is what the raw enumeration above gives up, which is whymeasure_underlyingnever had to state this rule. Discard both as each entry is read, ahead of anyfstatat, anyopenatand any accounting...is an ordinary directory on the same device, so neitherO_NOFOLLOWnor the same-device rule refuses it: descending into it would climb out ofaudit_store_dirinto its parent and over the whole filesystem from there, summing all of it intoused_bytes— the same runaway the descent rule exists to prevent, reached with no attacker at all..is caught eventually by the seen-inode set, but is skipped by name too: leaving an infinite descent to a dedup is not a rule a reader can check.fstatat(parent_fd, name, AT_SYMLINK_NOFOLLOW)and counted from that. Each contributes whateverst_blocksits ownfstatatreports — a FIFO, a socket or a device node holds no data blocks on the filesystems this store runs on and so reports zero, but the walk counts what it is told rather than assuming a class of entry contributes nothing. Nothing is opened or read for them, so a FIFO cannot block the walk and a symlink is never followed: a symlink contributes its own entry's allocated blocks and its target is not resolved, whether it points inside the store or out of it. Directories are opened because opening them is the only race-free way to enumerate them.readdirreturns a null pointer for both, so seterrnoto zero before each call and read it back after: a null witherrnostill zero is the end of the stream, and a null with a nonzeroerrnois a failed probe. Without that, an I/O error mid-directory reads as a short directory, and the walk returns a smaller total for a store it could not finish reading — under-reporting usage on the one control whose whole job is to notice a store filling up, which is the same failure as silently summing an unreadable subdirectory as zero.fstat— an entry whosest_devdiffers fromaudit_store_dir's is not counted and not descended into. Usage summed across a nested mount would be weighed againststatvfsavailable bytes for a different filesystem, so the two halves ofheadroom_byteswould describe different devices.(st_dev, st_ino)once, from the descriptor for a directory and from thefstatatfor everything else. Two paths hard-linked to one file consume one file's blocks; that is what the kernel accounts for and whatfilesystemmode'sstatvfsusage reports for the same bytes, so counting per path would inflate usage and fire an alarm over space nobody used. The store holds a bounded number of files, so the seen-inode set is a smallHashSet, not a memory concern.audit_store_dirincluded. A directory that once held a great many entries keeps that allocation, the kernel charges it against the reserve, andfilesystemmode'sstatvfsusage counts it — so omitting it here would make the two modes answer the same question differently over the same bytes.statvfs: the previous state andmeasured_atstand. An unreadable subdirectory in the middle of a walk must not be silently summed as zero, which would report a filling store as empty. The one carve-out is an entry that vanishes under the walk: an entryreaddirlisted whoseopenatorfstatatthen returnsENOENTis skipped and the walk continues, because the OpenBao device and the record store both rotate generations away while the tick runs, and treating that ordinary race as a probe failure would strand the alarm on a stale reading every time a rotation lands on a tick. Every other error fails the probe —EACCESon a subdirectory above all, andELOOPorENOTDIRfrom anopenatthat met a substituted entry. A directory that became a symlink under an active walk is precisely the event a capacity measurement must not paper over with a number.unsafethis introduces is bounded, and its one ownership transfer is pinned exactly. These calls have no safe wrapper in the standard library, so the walk is the firstunsafeFFI on this path in the library crate. Keep each block to the single call it wraps, precede it with a// SAFETY:comment stating the invariant that makes it sound, and give every descriptor exactly one owner at every point:openatyields a raw descriptor that is wrapped instd::os::fd::OwnedFdimmediately, so the closing is structural rather than a thing to remember.fdopendirtakes ownership of the descriptor it is given, so hand it over withinto_raw_fd(), neveras_raw_fd().as_raw_fd()would leave theOwnedFdstill owning a descriptor thatclosedirwill also close. A double close is worse than a leak in a long-lived daemon: the second close lands on whatever descriptor has since been handed that number, so an unrelated file, socket or directory stream is closed under whichever task owns it, at a moment nothing correlates with this walk.fdopendirreturn transfers nothing. The caller still owns the raw descriptor and must close it before propagating the failure — reconstructing theOwnedFdfrom it and letting the drop run keeps that structural too. Skipping it leaks one descriptor per failure, on a tick that fires every minute for the life of the process.fdopendir,closediris the only close. Do not also close the raw value, and reach the descriptor for the childopenatandfstatatcalls throughdirfd(), which borrows rather than owns.as_raw_fd()stays correct everywhere the callee only borrows —fstaton the directory, andopenat/fstatatagainst it as a parent. The rule above is about the one call that takes ownership, not a ban on borrowing.measure_underlyinginsrc/commands/audit_store/reserve.rsalready sumsst_blocks × 512over a probe trait, dedups(st_dev, st_ino)and refuses to cross a device boundary. It ispub(super)in the binary crate, so the daemon — which lives in the library crate — cannot call it. Do not lift it into the library and do not rewire thebootroot initpreflight around a shared implementation. The two answer different questions under different rules: that one is path-based and runs once under an operator'siniton a store nothing is writing to, and it fails on an arithmetic overflow, while this one runs unattended every minute against a live, partly attacker-writable subtree and saturates instead. Merging them would force one set of rules onto both.The walk's test seam
Most of the walk is testable against a real
tempfile::tempdir(), and is tested that way. Five of its required behaviours are not: each is a race, a privilege assumption or a process-global observation, and staging any of them on disk yields a test that passes for the wrong reason. Put the walk's syscall operations behind a small test-only seam and drive those five through it.openat/fdopendir/readdir/fstat/fstatat/closedirsequence above. The open-directory operation deliberately spans bothopenatandfdopendir, so the ownership transfer between them has one implementation rather than one per call site. The test implementation wraps a real tempdir and can be told to fail one named operation, on one named path, with one namederrno; for the open-directory operation it must also say which of the two syscalls fails, because the descriptor has a different owner on either side of the transfer.EACCESon a subdirectory. Achmod 0o000fixture passes vacuously whenever the suite runs as root, which it does whenever the suite is run from a root shell — a container shell being the common case. The repository's existing workaround —assert_ne!(current_process_euid(), 0, ...)at the top of such a test, assrc/registrar/internal/tests.rsdoes — turns that into a hard failure rather than a false pass, which is better but still leaves the required behaviour unproven wherever the assert fires.readdirand theopenat/fstatatthat follows it. InjectingENOENTat exactly that call is deterministic; a second thread deleting a file while the walk runs is not, and asleephoping to land between two syscalls is the synchronisation the project rules forbid.openatis the only synchronisation point at which that substitution is observable, so injectingELOOP/ENOTDIRthere is what makes the test deterministic.EIOfromreaddir, for the end-of-directory rule above. There is no portable way to make a real directory stream fail mid-read.fdopendirfailure path. The accounting counts the descriptoropenatproduced as well as theclosedirthat releases it, so a walk asserts opens and closes balance — over a walk that succeeded, over one that failed part-way through a subdirectory, and over one whosefdopendirfailed after itsopenathad already succeeded. That last path is the one the ownership rules above exist for, and none of the other injection points reaches it: each of them fails before the transfer or after it, never inside it. Counting entries in/proc/self/fdinstead would observe every other thread the test runner has in flight, so it is not an assertion that can hold undercargo test's default parallelism — and checking one descriptor number withfcntlafterwards is no better, since another thread can be handed that number between the close and the check../..skip, and the realfstatvfs.Headroom and its arithmetic
headroom_bytes = min(audit_store_reserve_bytes − used_bytes, filesystem_available_bytes). Theminis the point: a reserve larger than the device's free space is not a reserve, and the alarm must fire on whichever bound binds first.f_bavail × f_frsize,filesystemmode's(f_blocks − f_bfree) × f_frsize, and each entry'sst_blocks × 512. Compute each withu64::checked_muland takeu64::MAXonNone; take thef_blocks − f_bfreedifference withsaturating_suband accumulate the walk's total withsaturating_add, on the same terms. No wider intermediate type, noascast, and the probe does not fail. Saturating is what theminabove makes correct: a filesystem reporting a block count whose product overflows 64 bits is reporting something no device holds, so saturating makes that term stop binding and the reserve term decides the headroom, which is the answer the operator wants. Failing the probe instead would turn an implausible-but-harmless filesystem report intounknownand drop the alarm entirely — the one outcome this issue exists to prevent. Wrapping is what is actually forbidden: a wrapped product can read as a small available figure and manufacture a spuriousexhaustedon a healthy store.u64and the result isi64, so the conversion is specified rather than left to chance. Configuration validation already boundsaudit_store_reserve_bytesati64::MAX, so that term converts exactly;used_bytesandfilesystem_available_bytesare clamped toi64::MAXon conversion, and the subtraction is saturating. Use a checked conversion with an explicit clamp — noascast anywhere between theu64inputs and thei64result.The alarm state
stateis defined exactly, including the negative-headroom case:unknown— no capacity probe has succeeded yet (daemon just started, or every probe so far has failed). This is the only state in whichused_bytes,headroom_bytesandmeasured_atare absent, and it exists so that "we have not measured" is never encoded asok.exhausted— the store is at or past its reserve.low_water— the alarm is on but the reserve is not yet consumed.ok— headroom is above the threshold, or above the clear threshold when an alarm is being cleared.The rules are evaluated in this order, with
previousthe state the last successful probe recorded andthresholdthe configuredaudit_store_low_water_bytes:headroom_bytes <= 0, the state isexhausted. Zero counts as exhausted, not as low water: at zero headroom the next write is already past the reserve.previousislow_waterorexhausted, the state isokwhenheadroom_bytes >= threshold + marginandlow_waterotherwise.previousisunknownorok— the state isokwhenheadroom_bytes > thresholdandlow_waterotherwise.Rule 2 is the hysteresis, and it exists so a store hovering at the threshold does not flap an operator's console. Rule 3 is what keeps the cold-start behaviour honest: from
unknown, headroom ofthreshold + 1isok, because the first probe has no alarm to clear and the hysteresis margin would otherwise invent one. Four points fall out and are pinned by the acceptance criteria: the alarm is on atheadroom_bytes == threshold(rule 3, so the configured number reads as "alarm at 512 MiB left" rather than "alarm just under"), off atthreshold + 1from a cold start (rule 3), held on the way back up belowthreshold + margin(rule 2), and cleared atthreshold + margin(rule 2).An
exhaustedstore whose headroom jumps to at-or-abovethreshold + marginin one probe goes straight tookby rule 2 — the clear condition is met, and nothing requires it to dwell a probe inlow_waterfirst. The clear threshold gates a return from an alarm state; it is not the general rule for reachingok. There is no hysteresis between the two alarm states, so recovery is never held back, and a store that jumps from non-positive headroom to at-or-above the clear threshold in one tick is not hovering.marginis defined exactly, because it is test-visible:margin = max(audit_store_low_water_bytes / 10, AUDIT_STORE_MIN_HYSTERESIS_MARGIN_BYTES), where the division isu64integer division truncating toward zero andAUDIT_STORE_MIN_HYSTERESIS_MARGIN_BYTESis a file-top constant of1048576(1 MiB). The floor is not decoration:audit_store_low_water_bytesis operator-configurable, and on any value below 10 bytes the truncating division yields a margin of zero, which silently deletes the anti-flap rule the constant exists to provide. 1 MiB is the chosen floor because it is far larger than any single record this store can append — a record is bounded well under the 64 KiB minimum file boundaudit_max_file_bytesenforces — so no single write can carry the store across the clear threshold and back. The clear comparison is performed ini64, withthreshold + margincomputed as a saturating add: a threshold near the reserve's owni64::MAXbound can push the sum past it.A failed probe leaves the previous state and its previous
measured_atin place — it does not reset the state tounknown, which would hide an alarm behind a probe failure.One configuration rule this issue adds
validate_registrar_settingsalready rejectsaudit_store_low_water_bytes >= audit_store_reserve_bytesandaudit_store_reserve_bytes > i64::MAX. Add the one missing bound: rejectaudit_store_low_water_bytes == 0at load, with its own diagnostic naming the key. At zero thelow_waterband is empty, so the state machine would step straight fromexhaustedtookand the alarm this issue exists to raise would never fire. Disabling the alarm is not an offered mode, and a value that silently disables it is worse than one that is refused. Together with the existing upper bound this gives0 < audit_store_low_water_bytes < audit_store_reserve_bytes, which also settles representability: headroom never exceeds the reserve, the reserve is bounded ati64::MAX, so a threshold below it converts toi64exactly and the rule-2 and rule-3 comparisons need no clamp. This is the only configuration change in scope; add no key, and change no other key's validation.Running the probe and the scan without blocking the runtime
The maintenance callback
run_rotation_loop_with_maintenancetakes today is a synchronousFnMut()invoked inside the rotation loop'stokio::select!branch body, and all it does now is copy two atomic counters. A directory walk over the store and a scan that reads up to the record store's full ceiling are both blocking filesystem work and must not run there.rotation.run_pass(...). That body runs to completion once its timer branch is selected, so the awaits inside it are not cancelled part-way by the shutdown arm.tokio::task::spawn_blockingand await each handle inside the tick.scan_audit_store_off_runtimealready is exactly that wrapper for the record scan — call it rather than writing a second one. Give the capacity probe the equivalent. Awaiting the handles inside the tick is what keeps this clear of the orphan-task rule: nothing is spawned and dropped, and a join failure is a failed probe or a failed scan under the rules above rather than a silent gap.ROTATION_INTERVAL. Do not add a second scheduler, a second interval or a second long-lived task. That interval is what bounds staleness: in a healthy daemonmeasured_atandrecords_measured_atare never more than about a minute old, and the documentation says so, because an operator reading a timestamp needs to know what "fresh" looks like.The record-store signals
scan_audit_store— throughscan_audit_store_off_runtime— withregistrar.audit_record_dir, the tick'snow,AUDIT_SCAN_WINDOW,registrar.audit_max_retained_filesandregistrar.audit_min_retain_days, exactly the five argumentsbootroot statuspasses, and map the returnedAuditScan's three fields onto the members below. Note the directory: the record scan readsaudit_record_dir, which defaults to<audit_store_dir>/records, while the capacity walk readsaudit_store_diras a whole. Re-deriving any of the three signals here would put a second definition of "anomaly" and "shortfall" in the tree, and the two would disagree the first time either changed its window or its retention rule. (retention_shorton the reader andretention_shortfallon the wire are one signal under two spellings, internal versus wire, not two signals.)AUDIT_SCAN_WINDOWand it is reused, not restated. It is alreadypubinsrc/registrar/audit/scan.rs—pubrather thanpub(crate)because its other consumer,bootroot status, is in the binary crate — so pass it and widen nothing. Do not declare a second constant and do not write a 30-day duration into this issue's production code: two definitions is exactly how the relayed value and the host-localbootroot statusline drift apart, and this issue's acceptance criteria require the two surfaces to agree. The bar is on a second code definition of the window and nothing else — thedocs/pages this issue updates name the 30-day window in prose because operators need it, and tests that age fixtures across the window are expected; those derive the age fromAUDIT_SCAN_WINDOWrather than restating the number.records_measured_atin place, for the same reason a failed probe leaves the previous state in place: reporting zero anomalies and zero malformed lines because the scan itself failed would hide the alarm behind the failure. Those members are absent only before the first successful scan.The health member
RegistrarHealthwith exactly one new member,audit_capacity, appended afterlimiterso no existing member's serialized position changes, and carry it wherever the container is already carried — mint success, deregister success and refusal. Do not reshape the container, and do not read, write, reorder or otherwise touchlimiter. That the container rides refusals is what makes this signal reachable at all. The store this issue measures is a fail-closed control: as it fills, invocations are refused — so a success-only container would stop carrying the low-water alarm in exactly the state the alarm exists to announce, and an operator would learn about the exhausted reserve from the enrollment outage instead.RegistrarHealththe daemon constructs insrc/daemon.rswith the new member's starting value —unknown, withenforcement,reserve_bytesandlow_water_bytesread from the settings the daemon already has — and extendrefresh_registrar_healthto write the tick's probe and scan results into it. The new member'sDefaultexists only to keep the container's derivedDefault(which tests use) compiling; production never serializes it, because the daemon builds the member from configuration before the handler can answer anything.registrar_health.audit_capacitycarries exactly these members:state— an enum, not a bool and not a free string, with the four values and the exact rules above. Always present.enforcement— an enum mirroringaudit_store_enforcement,filesystemordirectory. Always present. A console cannot otherwise tell a kernel-enforced reserve from a configured estimate, and the two warrant different operator responses to the same headroom number; shipping the alarm without the mode would let adirectory-mode deployment read as protected.AuditStoreEnforcementinsrc/config.rsderivesDeserializeand#[serde(rename_all = "snake_case")]but notSerialize— addSerializeto it rather than declaring a second enum, so the wire spelling and the configuration spelling cannot drift.reserve_bytes—u64; the configuredaudit_store_reserve_bytes. Always present.low_water_bytes—u64; the configured threshold, so a console can render the alarm without knowing the daemon's configuration. Always present.used_bytes—u64; the store's measured usage. Present only when a probe has succeeded.headroom_bytes— signed (i64); themincomputed above. Signed, not unsigned: a store that has overrun its reserve has negative headroom, and an unsigned field would clamp that to zero and report the overrun as the healthiest possible value. Present only when a probe has succeeded.measured_at— RFC 3339 timestamp in UTC; the probe's last successful run. Present only when a probe has succeeded. Without it an operator cannot tell a healthy signal from a stale one left by a probe that stopped running.intent_without_outcome—u64count of unpaired intent records over the scan window. Present only when a scan has succeeded.malformed_records—u64count of lines the scan could not parse, over the same window. Present only when a scan has succeeded. A malformed line is what a forged or parser-breaking record attempt looks like, and the record store's escaping rule exists precisely because attacker-influenced bytes reach that log by design. Relaying the anomaly and retention signals while dropping this one would leave the console blind to the single attack that rule defends against.retention_shortfall—bool, the reader'sretention_short. Present only when a scan has succeeded.truewhen the store is at its maximum retained generations and its oldest retained record is newer thannow - audit_min_retain_days, i.e. the hard size ceiling is winning against the soft retention target. It is the signal that says the look-back window the whole detection argument rests on has quietly shrunk, so a design that surfaces capacity but not retention is reporting the less important of the two.records_measured_at— RFC 3339 timestamp in UTC; the scan's last successful run. Present only when a scan has succeeded. It is a second timestamp because the capacity probe and the record scan fail independently — an unreadable record store fails the scan whilestatvfsstill succeeds — and a singlemeasured_atwould let one half of the payload go arbitrarily stale with nothing on the wire to show it.statedescribes the capacity half only: the record signals andrecords_measured_atcome from the store scan, which succeeds or fails independently, so their presence is not governed bystateand a response may carry a healthyokalongside absent record signals or the reverse.null, and deserialize through the module'sreject_null_optionso an explicitnullis refused rather than read as absence. Both timestamps are formatted the waymaterial.expires_atalready is —time'sRfc3339formatter over a UTCOffsetDateTime, producing aZstring. The two enums serializesnake_case.mint-success.json,deregister-success.json,refusal-permanent.json,refusal-busy.jsonandrefusal-unclassified.jsonundersrc/registrar/endpoint/fixtures/all carryregistrar_health— and extenddocs/reference/registrar-wire-contract.mdwith theaudit_capacityschema, beside thelimiterparagraph it already documents. A stale fixture is a silent cross-repo break.statevalue means, the hysteresis behaviour, and how to read the two timestamps and what bounds their staleness, in bothdocs/en/anddocs/ko/, on the existing pages, with nomkdocs.ymlnav change.Acceptance criteria
minof the configured-budget headroom and the backing filesystem's available bytes; a test drives the capacity probe abstraction over atempfile::tempdir()and asserts the alarm is on atheadroom_bytes == audit_store_low_water_bytes, off atlow_water + 1from a cold start, does not clear while headroom sits between the threshold andlow_water + margin, and clears atlow_water + margin. A second test exercises the realfstatvfs-backed probe against a tempdir and asserts plausible non-zero values.statefollows the three ordered rules exactly; a test coversunknownbefore any probe,ok,low_waterat and just below the inclusive threshold,exhaustedat zero headroom,exhaustedat negative headroom, the hysteresis-gated return took, the immediateexhausted→low_watertransition on the first positive headroom below the clear threshold, the directexhausted→oktransition when one probe reachesthreshold + margin, and a failed probe leaving the previous state andmeasured_atintact.max(audit_store_low_water_bytes / 10, 1 MiB); a test asserts the floor applies for a small configured low-water value where the 10% term truncates to zero, that the 10% term applies at the default, and that a threshold neari64::MAXsaturates the clear sum rather than wrapping it.used_bytesandfilesystem_available_bytesabovei64::MAXand asserts the resulting state is neither a spuriousoknor a spuriousexhausted. A separate test drives astatvfsresult whosef_bavail × f_frsizeoverflowsu64, one whose(f_blocks − f_bfree) × f_frsizeoverflowsu64, and one walked entry whosest_blocks × 512overflowsu64, and asserts each saturates tou64::MAXand the probe still succeeds.audit_store_low_water_bytes = 0is rejected at configuration load with a diagnostic naming the key; a test asserts the rejection and that the existing reserve and upper-bound rules still reject what they rejected before.registrar_health.audit_capacitywith thestate,enforcement,reserve_bytes,low_water_bytes,used_bytes,headroom_bytes,measured_at,intent_without_outcome,malformed_records,retention_shortfallandrecords_measured_atmembers specified in Scope —stateandenforcementenums,headroom_bytessigned, both timestamps RFC 3339 UTC, the three capacity measurement members absent exactly whenstateisunknown, and the four record members absent exactly before the first successful scan — leavinglimiterbyte-identical; a test asserts a response carrying both members round-trips and that adding this one changed nolimiterbyte.null, and an explicitnullin a decoded payload is rejected rather than read as absence; a test covers both directions for one optional member of each type.enforcementis always present and mirrors the configured mode, so adirectory-mode deployment cannot be mistaken for an enforced reserve.audit_unwritablerefusal path still serializes an emptyregistrar_healthobject ({}); a test asserts it gained no member.scan_audit_storecalled withAUDIT_SCAN_WINDOW,audit_record_dir,audit_max_retained_filesandaudit_min_retain_days; a test asserts the relayed values equal the reader's own output for the same store and window, and this issue's production code declares no window value of its own — neither a second constant nor an inline 30-day duration at the call site. Prose indocs/and fixture ages in tests are not covered by that bar; test fixtures derive their ages fromAUDIT_SCAN_WINDOW.malformed_recordsvalue is exact, not merely present: a test appends a known number of unparseable lines to the store, drives a health tick, and asserts the health response reports that count and thatbootroot statusreports the same one. A malformed line in a rotated generation the reader does not select — a surplus generation beyondaudit_max_retained_files— is counted by neither. Do not assert anything about a malformed line's age: an unparseable line has no timestamp to age, and the reader deliberately selects the newest pre-window generation as a boundary file so that intents and outcomes match across the window's edge, so malformed content in that file is counted by design. An age-based assertion could only be satisfied by re-deriving the reader's selection rule here, which this issue forbids.retention_shortfall = trueon the health response and onbootroot status, and a healthy store reportsfalseon both — so the host-local surface and the relayed one cannot disagree.records_measured_atunchanged rather than zeroing them.ROTATION_INTERVAL; no second scheduler, interval or long-lived task is added, both filesystem operations run underspawn_blocking, and every spawned handle is awaited inside the tick rather than dropped.filesystemmode derives it from the samefstatvfscall on the store root descriptor and performs no directory walk, whiledirectorymode walksaudit_store_dirand reports a value that tracks a file written into the store — and that available bytes come fromf_bavailrather thanf_bfreein both. A walk that cannot read part of the store fails the probe rather than summing the unreadable part as zero.directory-mode walk resolves no symlink and counts no file twice: a test builds a store holding a symlink to a large file outside it, a symlink to a directory, two hard links to one file, a FIFO, a sparse file and a nested real subdirectory, then asserts the reported usage equals the summed allocated blocks of the distinct real objects plus the link entries themselves — unmoved by the symlink targets' sizes, counting the hard-linked file's blocks once, and counting the sparse file by its blocks rather than its apparent length. The expected total is computed from each fixture's own reportedst_blocks, never from an assumption that a class of entry contributes zero.errno, and the five behaviours below are driven through it rather than staged on disk. Every other walk test runs against a real tempdir.audit_store_diritself fails the probe in both enforcement modes and leaves the previous state andmeasured_atintact, rather than being followed: a real-filesystem test points the configured store path at a symlink to a directory holding known bytes and asserts none of them are reported and no newmeasured_atis stamped — including infilesystemmode, where there is no walk and the root open is the only step that can refuse the link.ELOOP/ENOTDIRat theopenatthat descends into an enumerated directory surfaces as a probe failure rather than being followed, and that walk contributes no usage at all — the previous state andmeasured_atstand. A companion real-filesystem test plants a symlink to a directory outside the store and asserts the reported usage is unmoved by what that target holds..and..are neither counted nor traversed.ENOENTat theopenat/fstatatfollowing areaddiris skipped and the probe still succeeds, while a seam-injectedEACCESon a subdirectory fails the probe and leaves the previous state andmeasured_atintact.readdirreturn with a nonzeroerrnofails the probe rather than reading as end-of-directory; a seam-injectedEIOmid-directory asserts it, and the probe does not report the short total it had accumulated.unsafeblock is minimal and carries a// SAFETY:comment stating its invariant, and no descriptor ever has two owners or none: the descriptor is handed tofdopendirwithinto_raw_fd()and notas_raw_fd(), and a nullfdopendirreturn closes the descriptor itsopenatproduced before propagating the failure. The seam's open and close counts balance for a walk that succeeded, for one that failed part-way through a subdirectory, and for one whosefdopendirfailed after a successfulopenat.measure_underlyinginsrc/commands/audit_store/reserve.rsis unchanged and thebootroot initreserve preflight is not rewired.statevalues, the hysteresis behaviour, the two timestamps and what bounds their staleness are documented in bothdocs/en/anddocs/ko/, with nomkdocs.ymlnav change, anddocs/reference/registrar-wire-contract.mddocuments theaudit_capacityschema.checkjob's Rust and documentation gates pass as CI runs them:cargo fmt -- --check --config group_imports=StdExternalCrate,cargo clippy --all-targets -- -D warnings,cargo doc --no-deps --document-private-itemswithRUSTDOCFLAGS=-D warnings— the new module, its trait, its enums and the new health member all carrying rustdoc — markdownlint over the changed Markdown, and./scripts/check-docs.shfor thedocs/changes.scripts/preflight/run-all.sh, and at minimumscripts/preflight/ci/e2e-matrix.sh, whose step 13 exercisesbootroot initon an endpoint-enabled loopback host — the very mode this issue'sfilesystem-mode measurement reads. Where the environment cannot run the matrix — it cannot supply the passwordlesssudothat matrix step 13 needs, or the matrix's own setup would tear down an unrelated livebootroot-*Compose project — run the matrix as far as it safely goes, which may be no arm at all, and say in the pull request body which arm ran and passed locally, which did not run, and why, leaving CI'sDocker E2Ejobs to gate the arms that did not run, asAGENTS.md's "CI Requirements" section provides. Do not weaken, skip,continue-on-erroror delete a check to make a local run pass. (Exercised by Probe the audit store's capacity and report it on the endpoint (#774) #965: no arm could run on the implementing host without destroying an unrelated livebootroot-*stack; the pull request body reports which arms did not run and why, and CI'sDocker E2Ematrix was green on every arm, includingregistrar-internal-init.)Constraints
scan_audit_storeand passAUDIT_SCAN_WINDOW— a second definition of either the scan or the window makes the relayed values and the host-localbootroot statusones drift silently apart.filesystemmode, where onefstatvfscall already answers usage exactly.fstatvfson the descriptor it returns, never from a second pathname lookup. Neverfstatat,openator account for the.and..entriesreaddirreturns. Do not resolve a path a second time during the walk, do not follow a symlink, and do not cross a device boundary out ofaudit_store_dir.openbao/is written by the container's uid, so an entry planted there must never be able to point this measurement at bytes outside the store. Directories are opened only to enumerate them; nothing else is opened and no walked entry's contents are read.measure_underlyingor thebootroot initreserve preflight, and do not lift a shared measurement into the library crate.#[cfg(test)]. A seam that can change what a running daemon measures is a second implementation of the control, not a test aid.tempfile::tempdir(), because a double that does not follow symlinks proves nothing about the code that does the following.sleepor a second thread mutating the store mid-walk, and do not assert on a process-global descriptor count.fdopendirreceives the descriptor byinto_raw_fd(), neveras_raw_fd(); after it succeedsclosediris the only close; when it returns null the caller closes the descriptor itself.Arc<Mutex<RegistrarHealth>>the daemon already builds.registrar_healthcontainer, do not touch thelimitermember's shape, contents or ordering, and do not add a member to the empty health object the pre-registraraudit_unwritablerefusal path serializes.ascast between theu64inputs and thei64result, and none in the block-count products.ok, and do not let a failed probe or a failed scan reset a signal to a healthy-looking value. Absence before the first success and staleness afterwards are both visible on the wire by design.audit_store_low_water_bytes == 0. Add no configuration key, change no other key's validation, provision nothing, render no Compose override, and create or verify no mount.unwrap()in production code; no[]indexing; keep eachunsafeblock minimal and preceded by a// SAFETY:comment.Out of scope
audit_record_dir, the Compose override that puts the OpenBao audit device on the store, the loopback-backed filesystem and its systemd mount unit, the mount-point verification at daemon start, theaudit_store_*keys themselves and the never-degrade-to-directoryrule. All of it is merged and this issue only reads it.AUDIT_SCAN_WINDOWoraudit_record_dir. This issue calls the reader and reuses the constant as they already exist.certificatesmember ofregistrar_healthand the certificate-lifetime reporting behind it. That member has its own owner; this issue addsaudit_capacityand touches nothing else in the container.limitermember, the two token buckets, therate_limit_*keys, the coalesced counted records andRegistrarBusy { retry_after_seconds }— all merged, all read-only here.Test plan
headroom_bytes == audit_store_low_water_bytesand off atlow_water + 1from a cold start, honours the hysteresis margin on the way back, and clears atlow_water + margin— driven through the probe abstraction over atempfile::tempdir(); plus one test of the realfstatvfs-backed probe reporting plausible non-zero values.statetable test coveringunknown,ok,low_waterat and below the inclusive threshold,exhaustedat zero and at negative headroom, all three transitions includingexhausted→okin one probe, and a failed probe preserving the previous state andmeasured_at.directorymode over a tempdir holding files of known sizes reports their total and tracks a newly written file;filesystemmode takes its usage from the samefstatvfsresult on the root descriptor and performs no walk; an unreadable subdirectory during a walk fails the probe rather than under-reporting; available bytes aref_bavail-derived.audit_store_dirthat is a symbolic link to a real directory fails the probe rather than being followed, preserving the previous state andmeasured_at; no seam injection is needed, since the fixture is deterministic and needs no privilege.mkfifoneeds no privilege, where a device node would need root — is counted from its ownfstatatst_blockslike any other non-directory entry, is neither opened nor traversed, and does not block the walk; a sparse file counts by allocated blocks, not apparent length; a nested real subdirectory on the same device is descended into and its own blocks counted.audit_store_dir's parent, the walk reports the store subtree's own blocks only and is unmoved when those parent-side bytes grow —.and..are neither counted nor descended into.errnoand injection point:ELOOP/ENOTDIRat a descentopenat(substitution) fails the probe and contributes no bytes;ENOENTat theopenat/fstatatafter areaddir(vanish) is skipped and the probe succeeds;EACCESon a subdirectory fails the probe and leaves the previous state andmeasured_atintact;EIOfromreaddirfails the probe rather than truncating the directory; a nullfdopendirreturn after a successfulopenatfails the probe.fdopendirfailure after it — with no reliance on/proc/self/fd, on any process-global count, or on anfcntlcheck of a descriptor number another thread may have been handed.low_water / 10truncates to zero; the clear sum saturates for a threshold neari64::MAX; usage and available values abovei64::MAXclamp rather than wrap;f_bavail × f_frsize,(f_blocks − f_bfree) × f_frsizeandst_blocks × 512each saturate tou64::MAXrather than wrapping or failing the probe.audit_store_low_water_bytes = 0is rejected at load with a diagnostic naming the key, and the existing reserve and upper-bound rules are unchanged.registrar_health.audit_capacitycarries every specified member with the specified types and presence rules, that a response carrying it alongsidelimiterround-trips, that nolimiterbyte changed, that an optional member is omitted rather thannull, and that an explicitnullis rejected on decode.enforcementis always present and mirrors the configured mode in bothfilesystemanddirectorydeployments, over an otherwise identical capacity payload.audit_unwritableresponse still serializes"registrar_health":{}.retention_shortfall = trueon the health response and onbootroot status, and a healthy store reportsfalseon both.malformed_recordsvalue on the health response, the same value onbootroot status, and that a malformed line in a surplus rotated generation beyondaudit_max_retained_files— one the reader does not select — is counted by neither, with no assertion made about a malformed line's age.AUDIT_SCAN_WINDOWand that this issue's production code carries no window value of its own; a fixture that must fall outside the window derives its age from that constant rather than hardcoding 30 days.records_measured_at; before the first successful scan those four members are absent while the capacity members are present.mint-success.json,deregister-success.json,refusal-permanent.json,refusal-busy.json,refusal-unclassified.json— asserting nolimiterbyte changed.bootroot statusagreement tests withcargo test --bin bootroot:statuslives in the binary crate, socargo test --libwould silently skip them. The endpoint tests are Linux-only, sincesrc/registrar/endpointis gated ontarget_os = "linux". The probe's own module carries#[cfg(any(target_os = "linux", test))]— Linux-only in a build, because the daemon that drives the measurement is, and compiled undercfg(test)everywhere so an unconditional module is not dead code on every other target — so its tests run wherever the suite does.tempfile::tempdir()and never a fixed path, and mutate no process environment.Dependencies
Nothing this issue needs is outstanding. The reserved store, its four
audit_store_*configuration keys and their validation, the kernel-enforced reserve behindfilesystemmode, the record store's reader and its 30-day window constant, theregistrar_healthcontainer, its placement on all three response shapes and the daemon-held snapshot that feeds it are all merged and in the default branch; this issue reads them and adds to them.It is independent of the two remaining members of that container. Each of the three adds a disjoint member, so whichever lands last is purely additive.
Part of #775.
Pointers
src/config.rs— the[registrar]table holding the fouraudit_store_*keys this issue reads, andAuditStoreEnforcement, which needsSerializeaddedsrc/config/validation.rs—validate_registrar_settings, where the reserve and low-water bounds live and where the zero rejection goessrc/registrar/audit_store.rs—records_dir(),openbao_dir(),layout_of()and the store's ownership and mode contract; the natural parent for a newcapacitysubmodule, mirroringaudit.rsand itsaudit/scan.rschildsrc/registrar/audit/scan.rs—scan_audit_store,scan_audit_store_off_runtime,AuditScanandpub const AUDIT_SCAN_WINDOWsrc/commands/status.rs— the five argumentsbootroot statuspasses to the reader, and the host-local surface the relayed values must agree withsrc/registrar/endpoint/protocol.rs—RegistrarHealth, the three response shapes that carry it,UnavailableRegistrarHealth, thereject_null_optionmodule, and the fixture-backed golden serialization testssrc/registrar/endpoint/fixtures/— the five response fixtures carryingregistrar_healthsrc/daemon.rs— where the sharedArc<Mutex<RegistrarHealth>>is built, handed toProductionHandler::with_health, and refreshed byrefresh_registrar_health, and where the rotation loop's maintenance closure is passed insrc/registrar/openbao_audit.rs—run_rotation_loop_with_maintenance, itstokio::select!body andROTATION_INTERVALsrc/commands/audit_store/reserve.rs—ReserveProbe,HostProbe::space(a pathnamestatvfswrapper and itsSAFETYcomments),ST_BLOCKS_UNIT_BYTESandmeasure_underlying; read for the idioms, changed by nothing heresrc/fast_poll.rsandsrc/hooks.rs— theOffsetDateTime::format(&Rfc3339)idiomsrc/openbao.rs(verify_audit_file) — the mandatory device whose space this measuresdocs/en/operations.mdanddocs/ko/operations.md— the audit-logging section and "The shared audit store" sectiondocs/en/configuration.mdanddocs/ko/configuration.md— where theaudit_store_*keys and their defaults are documenteddocs/reference/registrar-wire-contract.md— theregistrar_healthparagraph and the response member-order tablessrc/registrar/internal/tests.rs— the repository's existingassert_ne!(current_process_euid(), 0, ...)guard on achmod 0o000unreadable-file test, for contrast with the seam this issue asks forscripts/preflight/run-all.shandscripts/preflight/ci/e2e-matrix.sh— the pre-push preflight.github/workflows/ci.yml— thecheckjob's exact fmt, clippy, rustdoc and docs invocationsdocs/rfcs/0001-registrar-role-and-non-self-propagation.md§5.6 and §6