Skip to content

feat(runtime): generation-fence authoritative artifacts - #27

Merged
jamespud merged 7 commits into
mainfrom
feat/artifact-generation-fencing
Oct 7, 2026
Merged

jamespud merged 7 commits into
mainfrom
feat/artifact-generation-fencing

Conversation

@jamespud

@jamespud jamespud commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Parent: #17 · T3 of 6 (#21) · Base 1beffac68efd2b5bf3599911cb2500a1a4f2f186

Summary

Generation-fence authoritative artifacts and checkpoints using the durable owner established by T2.

A positive-generation write is authoritative only when the repository can prove, in the same database transaction as the INSERT/Save:

job_id + case_id + RUNNING + worker_id + execution_generation + unexpired lease

and decision_case.execution_generation == execution_generation.

Generation 0 remains legacy/history compatibility.

S27 schema

Adds execution_generation BIGINT NOT NULL DEFAULT 0 to:

  • AgentRun
  • Evidence
  • Claim
  • Vote
  • DebateRound
  • Reflection
  • ToolCall
  • Resolution
  • Checkpoint

Reflection and ToolCall also gain direct case_id provenance. Legacy Reflection/ToolCall rows relationally backfill only their Case ID from AgentRun; generation remains 0 and is never inferred from old -aN- IDs.

Checkpoint durable identity becomes:

(run_id, execution_generation)

so a newer generation cannot silently restore or overwrite an older owner's snapshot.

Explicit owner propagation

The durable execution owner is a single runtime-only value object in domain/entity:

  • CaseID
  • JobID
  • WorkerID
  • ExecutionGeneration
  • JobAttempt (diagnostic only)
  • RunID where needed for runtime identity

RunManager constructs it after Claim and passes it explicitly through Orchestrator → Dispatcher → AgentLoop/checkpoint. It is not hidden in context.Value and is not persisted on DecisionCase.

A positive-generation durable run without the owner-aware orchestration capability fails closed.

Artifact write semantics

Production aggregate repository implements OwnedArtifactRepository.

Every owned Create verifies the active T2 owner and writes in the same transaction:

  • AgentRun
  • Evidence
  • Claim
  • Vote
  • DebateRound
  • Reflection
  • ToolCall
  • Checkpoint

Legacy sub-repository Create rejects positive-generation rows with ErrLeaseLost, preventing a bypass around the owner-aware API.

Owner loss propagates back to Orchestrator and terminates the durable attempt instead of being treated as a best-effort artifact failure.

ID namespace

Persisted execution artifact IDs now use the durable generation:

-g<execution_generation>-

instead of resettable ExecutionAttempt / -a<attempt>-.

ID text is diagnostic/stable identity only; authorization never parses the ID.

Read semantics

Repository keeps two explicit classes:

  • generation-scoped authoritative reads via OwnedArtifactRepository;
  • legacy/history ListByCase reads returning all generations with generation metadata.

DecisionService current views load the Case's current ExecutionGeneration and return only that generation for Evidence, Claims, Votes, AgentRuns and ToolCalls.

No-cleanup retries therefore remain correct even while old-generation rows remain physically present.

Checkpoint semantics

Logical checkpoint RunID remains stable across durable generations, while storage is generation-scoped.

  • generation N+1 never implicitly loads N;
  • stale N cannot save after N+1 owns the Case;
  • legacy RunID-only checkpoint APIs expose only generation 0;
  • positive-generation AgentLoop uses the owner-aware checkpoint path and fails closed if the capability is absent.

#11 integration

Resolution now persists ExecutionGeneration.

Atomic terminal reference verification and the explicit non-atomic test fallback require cited Vote/Evidence/Claim rows to match both:

  • Case ID
  • Resolution execution generation

Old generation rows cannot satisfy a newer Resolution merely because their IDs exist.

General Case status/terminal generation CAS is intentionally not included; that remains T4 / #22.

T3 / T5 boundary

Positive-generation retries no longer invoke the historical case-wide CleanupCaseArtifacts.

Correctness is now independent of cleanup. Old rows remain readable as history; T5 / #23 will convert the legacy cleaner into generation-scoped GC.

Real MySQL verification

The MySQL CI gate explicitly runs T3 suites proving:

  • stale gen1 writer after gen2 Claim → ErrLeaseLost, zero insert;
  • owner check + artifact INSERT share one transaction/linearization boundary;
  • current generation reads exclude older rows while history preserves them;
  • no-cleanup retry keeps gen1 + gen2 rows without mixing authority;
  • legacy generation 0 remains visible as history;
  • checkpoint gen1 cannot be restored or overwritten by gen2;
  • same-Case cross-generation Vote/Evidence/Claim references are rejected by terminal verification;
  • same-generation references commit successfully;
  • S27 scripted migration and fresh/AutoMigrate schema preserve generation/provenance and checkpoint composite key.

Verification

Final exact head: cb014244cd926762702a22576006b7b5d30d937a

GitHub Actions run 37589099246: success

  • Backend quality gate — pass
    • formatting
    • go test -race ./...
    • go vet ./...
  • Runtime conformance — pass
  • MySQL migration compatibility — pass
    • schema lifecycle
    • existing MySQL runtime invariants
    • T2 decision ownership generation invariants
    • T3 artifact generation fencing invariants
    • T3 adapter artifact-generation invariants
  • Frontend quality gate — pass
  • Operational scripts gate — pass

The iterative branch history was reduced to four logical commits. The rewritten final Git tree SHA is byte-identical to the previously validated content tree.

Commit structure

  1. 969846b — schema / artifact generation provenance
  2. f70f431 — runtime owner fencing + checkpoint generation scope
  3. e806615 — current decision views scoped to generation
  4. cb01424 — RED→GREEN tests + real MySQL CI gates

Non-goals

Not included:

  • ordinary Case status transition generation CAS;
  • terminal state CAS against active generation;
  • final-failure Case generation CAS;
  • generation-scoped cleanup/GC;
  • rollout/capability gate.

Those remain T4 #22, T5 #23, and T6 #24.

@jamespud
jamespud force-pushed the feat/artifact-generation-fencing branch from d293f0d to cb01424 Compare October 7, 2026 07:43
@jamespud
jamespud marked this pull request as ready for review October 7, 2026 07:53
persistArtifact only escalated ErrLeaseLost, so any other repository or storage error on an authoritative artifact write was logged and counted while the run continued. T3 widened the authoritative generation domain past the terminal reference fence (AgentRun, ToolCall, Reflection, DebateRound), so a case could still reach RESOLVED while claiming artifacts that never landed.

Route the durable owner through persistArtifact so any write error aborts the execution; the legacy generation-0 and best-effort paths keep the old log-and-count contract. A commit whose outcome is unknown is treated the same way: the generation is abandoned and a later one recovers, never guessed at.

Also stop flattening the failure cause to a string in fail(), so a caller can classify it with errors.Is instead of only by message.

Tests: unit classification of the durable/legacy split, durable in-memory orchestration failures for agent_run/evidence/claim/tool_call/vote, plus real-MySQL fault injection for an owner-verification lock timeout, an INSERT failure and an abandoned in-flight write against a resolving control.
S27 adds case_id to magi_agent_checkpoint, but the pre-S27 table recorded no Case at all: it stored only run_id plus snapshot state, and run_id is the logical working-memory identity rather than a declared reference to magi_agent_run. The migration comment now states that legacy rows keep case_id empty with execution_generation=0 instead of fabricating provenance, and that a reflection/tool-call row with a broken AgentRun relationship keeps unknown provenance rather than failing the upgrade.

CaseRepo.Delete now documents the three buckets it actually has: direct case_id provenance, the pre-S27 containment rule that removes a checkpoint whose run_id names one of the Case's runs, and the unknown-provenance rows it deliberately retains. The containment rule stays because it is an existing cleanup contract; it is a deletion rule, not a provenance claim, and the generation-scoped loader cannot see retained rows.

Tests: S27 upgrade assertions for unchanged checkpoint provenance, preserved orphan reflection/tool-call history and RunID collisions, plus the delete/retention contract on SQLite and real MySQL including generation-scoped loader isolation.
persistArtifact only fails closed when it receives the durable owner, so the second defence line is persistArtifactValue refusing a positive-generation entity that arrives without one. That branch had no coverage: a future authoritative write that forgets to thread the owner would silently fall back to the legacy Create path and only the guard would stop it.

The regression drives all seven authoritative artifact types through persistArtifact with a nil owner and a repository that does not implement OwnedArtifactRepository (its embedded interface is nil, so any storage call panics), and asserts ErrLeaseLost instead.
@jamespud
jamespud merged commit 0a3d6a0 into main Oct 7, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant