Require explicit source scope and historical checks before no-change refreshes - #96
Merged
Merged
Conversation
4 tasks
obsessixnv
added a commit
to ElcanoTek/fleet
that referenced
this pull request
Sep 16, 2026
…#1518) ## What changed, and why Scheduled runs could lose completion evidence because their durable tool transcript contained the 4,000-byte UI preview. They could also report success after ignoring verifier findings, or dead-letter after a provider failure even though a preceding tool step had completed successfully. Record complete redacted tool results before display truncation and project enclosing status/version fields before deep metric profiles. Recheck verifier repairs, with at most three reviews; unresolved findings and malformed or unavailable verification prevent success. Resume retryable provider failures from a completed step's input, retaining tool results and usage instead of restarting the original tool sequence. Recovery remains suppressed if a tool started in the failed step or an earlier tool result indicated failure. Preserve iteration limits, remove provider-specific reasoning from replay, recover structured SSE status codes, and retain bounded redacted diagnostics. Authentication and validation errors stay terminal. ## How you verified it - Regression tests pass for complete evidence through the core/scheduled/verifier boundary, bounded repair reviews, malformed verdicts, completed-write fallback without duplicate execution, reasoning removal, preserved iteration limits, provider status classification, and redaction. - Release compilation, tagged vet, lint, the full Go suite, and the full race suite pass against isolated local PostgreSQL databases. A verbose database smoke test confirms PASS rather than SKIP. `make govulncheck` reports no reachable vulnerabilities. - `make ci-web` passes under Node 24: dependency audit, lint, typecheck, 1,639 tests across 153 files, and production build. Built-in guides and web copies are synchronized. - The live-browser CI failure was traced to the fake model replaying its worker script for the independent verifier. A new regression first reproduced invalid verdicts for all five scheduled/A2A completion fixtures; it now passes after adding opt-in non-streaming fixture responses. Both fake-model packages pass with `-race`; release compilation and the full lint gate pass on the fixture patch. - All CI checks pass on `51c930ba`, including the live scheduled-task browser journey, mocked browser suite, full Go race lane, CodeQL, Semgrep, and container vulnerability scan. - Existing contained-panic coverage continues to require suppression of uncertain tool replay. ## Scope and deviations The runtime remains general purpose. Source-scope and historical-correction rules live in the companion Pages and bundle changes: ElcanoTek/pages#96 and ElcanoTek/elcano-config#145. No customer schedules, source data, page schema, connector permissions, or network policy are changed. ADR-0065 narrows ADR-0035 only at a completed-step checkpoint. Verification now fails closed when configured; this can make an unavailable verifier fail a run that previously appeared successful. --- - [x] Title and behavior description are written for release notes - [x] Design note and operator guides updated - [x] ADR added for the completed-step recovery invariant - [x] Diff is scoped to reliable task completion and recovery
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Recurring refreshes could treat unconfigured source identifiers as exclusions and declare no change from matching dates or totals. Generated prompts now require explicit source scope, retain in-scope zero-metric rows, and check historical records or immutable provenance before recording
source_not_updated. Ambiguous scope or an incomplete comparison blocks the refresh.Existing saved prompts are unchanged. Regenerate the recurring prompt after deploying this change, with any intended exclusions stated in the source bindings.
Checklist
npm testpasses, including regression coverage for scope and historical corrections before branch selectionbash test/run-integration.shpasses; browser tests were not run locally because this changes prompt generation and has no browser behavior change