Skip to content

Fix budgeted same-turn recovery for unknown and disabled tool names - #245

Closed
acoliver wants to merge 3 commits into
mainfrom
b3-issue-206
Closed

acoliver wants to merge 3 commits into
mainfrom
b3-issue-206

Conversation

@acoliver

@acoliver acoliver commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Closes #206.

Unknown and disabled tool names now produce ordered, charged failed tool results so a model can correct a syntactically valid naming mistake within the same turn. Disabled capabilities never execute. Malformed protocol, authorization failures, unsafe identifiers and configured-secret reflection remain fail-closed.

The change applies one shared tool-call budget to the original batch order, keeps live and persisted result identities aligned, and uses existing output bounds and scrubbing. Two new regressions failed before the fix by admitting three calls under a two-call budget. Ten deterministic regressions cover correction, disabled execution, mixed ordering, persistence, exhaustion and terminal rejection cases.

Implemented and gated by the actual Rust CLI using Astra. Exact pushed head: e481f67527adb4228a5838275a89b758d2530462; branch base: 44530155261e62b49065c8c240bc9a8715acf89d.

Validation at that head: 64 focused agent tests, seven persistence-validation tests, and the full suite (899 passed, no failures or ignored tests). All five literal gates passed: cargo fmt --all --check; cargo clippy --offline --locked --workspace --all-targets --all-features -- -D warnings; cargo test --offline --locked --workspace --all-targets --all-features; cargo xtask quality; cargo xtask compat. No vendor delta. Local worktree clean and equal to remote.

Raw commands, exits and pre-fix failures retained in worktree tmp/astra206/. Independent exact-head review and current-main GitHub CI are pending; this PR does not claim merge eligibility yet.

@acoliver

acoliver commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Updated delivery: final pushed HEAD c69462adf6e515e3c6a9f5fc453469c52b825ff5, integrated main b39bcef155ec4115ed004f901cd8468613c6a9c7.

The independent review found that persisted charged failures could accept unsafe or empty tool names. Repair 344fc6b27d2ff54cae043c7467754c8709832239 reuses the provider's nonempty ASCII identifier predicate and retains the existing 64-byte cap. All three added rejection regressions failed before the production fix. Valid charged unknown-name failures remain accepted; unsafe names and unknown successes remain rejected.

At the final committed head, focused session (33), agent (64), naming (10), and security integration (18) tests passed. All five literal gates passed: fmt, offline/locked workspace all-targets/all-features clippy with denied warnings, the matching full test command, cargo xtask quality, and cargo xtask compat. No net vendor delta. The first full-suite attempt hit the outer timeout and is preserved as interrupted, not counted as a pass; a detached rerun completed successfully at the same SHA.

Local and remote identities matched with a clean tree in the worker's final proof. Raw command/log/exit/head artifacts are in the implementation worktree's tmp/astra206-repair/; delivery metadata is branch-3 tmp/batch4/repair206-result.json. Independent bounded verification of the correction is pending. Approval and CI at the earlier head do not apply to this repaired head.

@acoliver

acoliver commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Final-head review delivery update for c69462adf6e515e3c6a9f5fc453469c52b825ff5: blocked on review-tool evidence, not admitted for merge.

The independent operator inspected the full issue delta, classified the four OCR comments as nonblocking and supplied a source-based approval. The completed OCR run covered 10/10 files at exit0 using zai-anthropic/glm-5.2, but its raw stderr contains:

[ocr]   ✘ file_read failed: invalid line range: start_line 320 is greater than end_line 240

The operator treated that error as internally recovered. This does not satisfy the assigned warning-free/tool-failure-free delivery requirement, so the orchestrator is not accepting or silently waiving that approval. An earlier foreground OCR attempt was also interrupted by the driver timeout; both runs are preserved. No further automatic review retry or merge has been scheduled.

Raw evidence and the separate admission decision are under branch-3 tmp/batch4/zai-review-results/issue206-pr245/, including orchestrator-admission.json. Existing local and CI passes do not override this review-delivery blocker.

@acoliver

acoliver commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Current-main integration is preserved locally at 49952e05d730f369d4ed0843e7f4f8c89eda8dff, with original PR head as a merge parent and main’s live/durable/executed-prefix/deadline contracts retained. Independent scoped integration verification completed with two compiled-wire findings in the retained summary caller: a budget-truncated answered tool batch is appended a second time, and nonempty tool-call preamble can bypass the required post-failure summary. Both also exist in pinned main/original source; they are not attributed as new merge regressions.

Native Sol is repairing the owning summary state/caller in a new isolated copy with exact captured-wire regressions. #305’s combined transcript integration will carry only the stable fix after source mapping, without overwriting the original candidate. Existing historical review-delivery limits and earlier deadline-load failure remain retained. No OCR restart, assertion weakening, provider fallback, force push or merge. Final findings follow-up, combined fidelity/package acceptance and exact-head CI remain pending.

@acoliver

acoliver commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Consolidated replacement draft #324 is published at 40dc91cd62ee2bdf8993b0b3c43e9ede6112707c. It preserves original #245 head c69462ad as an ancestor, integrates current main and only required shared/profile residuals, and combines related #305 agent/session transcript scope through its sole native owner. The six-file summary caller fix is carried with one transcript-hook reconciliation; original compiled assertions and all refs/evidence remain.

The single recovery findings-only follow-up resolved F1/F2 with14 independent wire cases but retained two unchanged strict elapsed-deadline failures. Full final #305 independent review, exact40dc package/authenticated fidelity and fresh CI are pending, so #324 remains draft/not merge-ready. Original review-delivery limitations are not silently waived.

Closing this old delivery as a mapped consolidation avoids two recovery/transcript PRs editing the same owning seams. No merge, force-push or branch deletion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dogfood: one unknown model tool name aborts the whole implementation turn

1 participant