Repository navigation
Description
Observed after #144
Read-only production acceptance on November 12 checked all five scheduled runs since the context-limit fix at 58c03ad5b94ef99252c14285b3acc6c348df3522:
| Local check date | Outcome | Model calls | Settled cost |
|---|---|---|---|
| November 8 | evidence review rejected the revision: supported | 4 | $0.115918 |
| November 9 | evidence review rejected the revision: supported | 4 | $0.121854 |
| November 10 | posted | 4 | $0.117986 |
| November 11 | revision left a rejected item unchanged | 3 | $0.070122 |
| November 12 | posted | 4 | $0.114854 |
No context/JSON errors recurred in these checks. All 19 attempts settled; total $0.540734. No pending deliveries or unsettled attempts. Both successful posts used the semantic revision path. The three rejected workflows are not deliberate quiet/no-news decisions.
Diagnostic gap
stories.parse_review() reports the gate name supported, but the scheduled outcome does not retain which reason/change/public-claim item failed. require_rejected_items_repaired() detects unchanged rejected text and citations. Failed drafts and their structured reviewer verdicts are discarded; only sanitized status and the rejected input fingerprint remain. Accepted decisions are persisted only after stories.decide() succeeds.
Consequently the actual failed drafts cannot be inspected now. The logs establish rejection but cannot show whether it was a justified evidence/citation problem, ineffective revision, or reviewer false rejection. Ordinary supported flags are Boolean-only; there is no hidden explanatory rationale to recover.
Scope and acceptance
- Retain a bounded, versioned diagnostic record for editorial failures, linked to the input version and workflow. Distinguish stage and failure gate.
- Preserve the rejected item positions/fields, text and exact cited spans, the corresponding revised items, and both structured verdicts when available. Include only the bounded evidence/render projection needed to judge the specific failure. Missing initial/final stages remain explicitly absent.
- Keep records private and operator-readable with explicit count/size/age limits. Never copy credentials, hidden reasoning, unrestricted provider payloads or full prompts to logs, public issues or ops messages. Treat retained model/source strings as untrusted data.
- Make sanitized status identify the failed field/stage or diagnostic reference without leaking the retained content.
- Add offline regressions for post-revision support rejection, unchanged rejected items, absent final review, retention bounds and safe diagnostics. Preserve accepted-output behavior, exact-citation checks, completeness review, one-revision/four-call limit and the existing fail-closed result.
- Inspect captured examples before changing prompts, reviewer thresholds or models. A fresh model run is not a reconstruction of a discarded response.
This issue records the next diagnostic improvement. It does not authorize paid replay, relaxed review, production configuration changes or implementation in this status-review session. Related: #115, #110, #143, #97, #98.
Activity