feat(output): add transactional record spool - #50
Open
ronheichman wants to merge 12 commits into
Open
ronheichman wants to merge 12 commits into
ronheichman wants to merge 12 commits into
Conversation
Store complete records in a transactional queue and acknowledge them only after successful HTTP delivery. Built with Codex
The failure that corrupted the legacy append file was a write that could not grow its backing file: a short or failed append left a partial NDJSON line that the next record concatenated with. Assert the spool is immune to that trigger by capping RLIMIT_FSIZE in a child process (so the cap cannot disturb the test harness) and forcing a multi-megabyte Put to fail during bbolt's file growth. The store must stay byte-identical to its pre-Put state and remain usable once space is available, so a failed write can never leave a partial or glued record.
A hook appends one NDJSON record per process to the shared records file. On a full disk the kernel wrote what fit in the last block and then failed, leaving a record with no trailing newline; the next hook's record appended onto the same line, and the shipper delivered the glued pair as one line that ingestion rejected as invalid JSON, stalling the queue. Hold the existing file lock across the whole append and keep the file ending on a record boundary: repair a missing trailing newline before writing (append sinks now open read-write so the last byte can be read), and truncate a short or failed write back to the pre-write size. Either way the next record starts on its own line instead of concatenating onto a partial one.
An already-poisoned records file, written before the append-side fix, can still contain a line that is two records glued together by an interrupted append. Shipping it makes ingestion reject the whole batch, and the checkpoint never advances past it, so the queue stalls indefinitely. Skip a complete line that is not a single JSON object the same way an oversized record is skipped: ship any good records buffered before it, advance the checkpoint past it with a diagnostic, and continue. The record stays in the input file for inspection; only its HTTP delivery is skipped. The object check mirrors the spool sink.
spoolSink.Write and isShippableRecord both required a record to be a single JSON object, and the two open-coded the same check. Route the sink through isShippableRecord so the enqueue-time and ship-time validations cannot drift.
The installer emitted spool paths without applying the runtime collision guard, allowing hooks to be installed in a configuration that every callback rejected. Validate resolvable spool paths through the same path-identity check before writing agent settings.
adel-pplx
approved these changes
Sep 15, 2026
Record writes always sought to the reported file size. Linux FIFOs reject that operation with ESPIPE, so hooks lost both findings and enforcement records while failing open. Seek only regular files. This keeps regular-file rollback positioning, including Windows append handles, while allowing FIFO writes. Built with Perplexity Computer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ship --spool-filedelivery path2xxship --input-filesupport for existing deploymentsThe spool uses the existing bbolt dependency. Producers open it only for a short transaction, so the shipper never holds the database lock during HTTP delivery. Failed and ambiguous deliveries retain the selected records; ambiguous outcomes can duplicate a batch, so receivers must deduplicate by stable record ID.
Spool creation refuses final-component links or reparse points and unsafe Unix directory or file modes. The implementation also rejects spool/state collisions across symlinks and filesystem-equivalent paths, preserves owner write-only file outputs while repairing permissions, rejects installed sink paths that expand to relative locations, and retains Linux FIFO output compatibility.
Operators opt in with
--output spool --spool-file PATHand runnumbat ship --spool-file PATH --http-url URLseparately. Existing installs continue to default to file output. File and spool are alternative durable local streams; either can be combined with direct HTTP when duplicate delivery is intentional.There are no event-schema or wire-format changes.
Verification
go test ./...go test -race ./...go vet ./...golangci-lint v2.12.2 rungolangci-lint v2.12.2 fmt --diffgovulncheck v1.6.0 ./...go mod tidywith no module diff