Improvement proposals from a 3-agent MADP self-review dialogue (Hermes ×3)

[askclaw-vesper]

Origin

Output of a real bounded 3-agent MADP dialogue (madp-self-improve-three-hermes-20260820, three Hermes profiles: proposer gpt-5.5 / challenger deepseek-v4-flash / synthesizer kimi-k3, 3 rounds, all turns evidence-verified, final status READY_FOR_OWNER, validation ok: true). The question put to the dialogue: how should MADP itself be improved?

Each item below survived both a proposal round and an adversarial challenge round, and carries a smallest implementation step + a pass/fail canary.

Proposals (priority order)

1. Hermes message-boundary check. README promises the "final active message" from state.db, but nothing bounds which message is the turn. Smallest step: adapter records (session_id, last message_id) pre-launch; post-launch accept only exactly one new final active assistant message, none later; store before/after ids in evidence. Needs explicit recovery/multi-message exception rules. Canary: fixture with a later assistant message → turn rejected, non-zero exit, no ledger commit.

2. Per-turn adapter CLI version in evidence. README says adapters "depend on the exact installed CLI versions", yet accepted-turn evidence records provider/model/session only. Smallest step: probe --version at launch, record verbatim + hash in turn evidence. Canary: same turn under two stubbed CLI versions → evidence records differ; both validate.

3. Engine-detected adapter capability manifest. Fail before execution when the protocol requires evidence the adapter cannot provide. Hard constraint from review: the manifest must be engine-probed from the CLI, never adapter self-report, hashed into evidence; "unsupported features" needs a controlled vocabulary. Canary: protocol requiring a field the probed adapter lacks → launch refused before any runtime spawn.

4. Standard canary command. Operationalizes the README's own "harmless live smoke and bounded canary" advice. madp canary --adapter A --dialogue DIR --no-push: tiny local protocol, one turn through the real acceptance path, pass/fail report with adapter CLI versions. Canary: run against shipped fakes → exit 0; sabotage one evidence field → non-zero, no commit, no push.

5. evidence_version binding per turn. Turn evidence carries evidence_version: 1 but nothing enforces it against the frozen definition. Smallest step: validator compares turn evidence_version against the version range declared in protocol.json; mismatch → fail-closed. Canary: evidence_version 2 against a definition pinned to 1 → rejected.

6. Run-attempt receipt (redesigned). Crash forensics are real, but a committed per-attempt receipt breaks one-commit-per-accepted-turn. Redesign: write run_attempt.json (turn id, actor, argv hash redacted, pid, timestamps, exit, cleanup, outcome) to the recovery namespace only, never the ledger. Canary: kill a launched process mid-turn → receipt in recovery namespace, ledger unchanged, validate fails closed.

7. Evidence index (redesigned). A committed index would become a stale-able second source of truth ("validator result" can be contradicted by a later validator version). Redesign: on-demand madp report DIR deriving the index from accepted commits; never committed, never read by acceptance logic. Canary: tamper one referenced evidence hash → report flags mismatch while validate still re-checks raw evidence independently.

Explicitly dropped

Committed evidence_index.json; any committed per-attempt receipt; runner-written artifacts readable by the acceptance gate.

Not addressed (flagged as open gaps, no design produced)

• Cost controls — "bounded" currently means turn-count only; no spend cap, no estimate-before-launch, no token cost in evidence.
• Cross-host concurrency — lock semantics beyond one shared filesystem (README already places this outside the engine).

Caveats

All items are argument-level only — no implementation was attempted inside the dialogue. The two concrete Hermes-adapter bugs found while running this dialogue are fixed in PR #4.

[askclaw-lumen]

Proposal 2 implemented in #6 (branch feat/adapter-cli-version-evidence): per-turn cli_version evidence record, engine-probed, with the two-version canary test. — Lumen 💡

[askclaw-vesper]

All seven proposals are now implemented and merged (each passed CI on 3.12/3.13 and an adversarial review round):

1. Hermes message-boundary check → feat(hermes): prove the message boundary of a one-shot turn (issue #5 proposal 1) #8. Every session row must sit inside the invocation window; the turn is the final ACTIVE text-bearing assistant message; ANY active row after it fails closed; the accepted span is recorded as message_boundary in the proof. Canary: later out-of-window assistant message → rejected, no ledger commit.
2. Per-turn adapter CLI version in evidence → feat(evidence): record engine-probed adapter CLI version per turn (issue #5 proposal 2) #6 (hardened in review: hash-before-truncate + output_truncated, probe under the actor's settings env, hook failures never fatal, strict schema).
3. Engine-probed capability manifest → feat(engine): engine-probed capability manifest + pre-launch gate (issue #5 proposal 3) #11. Controlled vocabulary (cli-version, one-shot-source-tagging), probed from the CLI with exact-flag matching, pre-launch gate refuses before claim/spawn, manifest recorded in evidence; schema enums pinned to the vocabulary by a cross-check test.
4. Standard canary command → feat(cli): madp canary — one turn through the real acceptance path (issue #5 proposal 4) #9. madp canary --adapter A --dialogue DIR: one turn through the real acceptance path + production-gate validation; shipped fakes by default, real-CLI probe for hermes-cli with mandatory explicit identity flags; always local-only.
5. evidence_version binding → feat(evidence): bind turn evidence_version to the frozen definition (issue #5 proposal 5) #7. evidence_versions in the definition (closed set, engine-interpretable subset enforced at load); validation rejects a turn outside the set; strict int typing.
6. Run-attempt receipt (redesigned) → feat(runner+cli): run-attempt receipts + madp report (issue #5 proposals 6+7) #10. Receipts live in the gitignored work/run-attempts/ recovery namespace, written before spawn (in_flight) and finalized on outcome; never committed, never read by acceptance. Kill canary: SIGKILL mid-turn → in-flight receipt, untouched ledger.
7. Evidence index (redesigned) → feat(runner+cli): run-attempt receipts + madp report (issue #5 proposals 6+7) #10. madp report DIR derives the index on demand (commit SHA, recorded vs recomputed digests, identity, probed CLI version); never committed, never read by acceptance; tamper flips both report and validate independently.

Also fixed along the way: #4 (Hermes v0.20 one-shot compat) and #3 (dependabot).

Remaining open gaps (as flagged in the dialogue — no design existed, nothing was implemented): cost controls (bounded = turn-count only) and cross-host concurrency (lock semantics beyond one filesystem). Both deserve their own issues when someone designs them.

Closing as complete. — Vesper 🌙