You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Every outcome path in the harness assumes the deliverable is a diff. There
is no way to express an item whose product is knowledge — an investigation, a
feasibility answer, an ADR, a comparison of three approaches.
What happens today if you try
Write the item anyway and the agent does the work well. Then:
It leaves the worktree clean, because findings are not a code change.
A completed investigation is indistinguishable from an agent that did
nothing.
If the agent writes its findings to a file to avoid that, it is now a diff, and
it meets the rest of the pipeline:
the checks run against the worktree — cargo test has no opinion on an ADR;
REVIEW_PROMPT asks the reviewer to judge "your diff against the item", so
it grades a document as though it were an implementation;
the definition of done is a pull request containing a change.
So the two available outcomes are "recorded as nothing" or "graded as code".
Why this matters beyond tidiness
It is the prerequisite for #181. Proposing a plan for an existing project
honestly requires investigating it first — reading the roadmap, checking what
is deployed, finding what the last release left open. If investigation cannot
be expressed as work, then plan generation is a single opaque model call rather
than something that can be split, checked, reviewed and retried like everything
else here.
It is also the shape of a large fraction of real engineering work. On rdpapp
the owner's own next package (P7.0) opens with "the first decision is the
attachment ADR: compare Node B's existing host-managed tailscaled, a managed
sidecar, and a tsnet/bridge approach". That is an item. The harness cannot
currently hold it.
What is actually needed
The interesting question is what "done" means when there is no diff, and it
should not be answered by loosening the gates:
A declared deliverable. The item states what it must produce — a written
answer, a recommendation with alternatives considered, a document at a named
path. Checks that cannot apply are skipped because the item says so, not
because they failed silently.
A review rubric for findings, not diffs. Are the claims cited? Were the
named alternatives actually examined? Is the recommendation supported by what
was found, or asserted? A reviewer told to judge a document against the
diff-rubric will reject it for being a document.
An outcome that is not no_target. A finished investigation is completed, and the findings are its result, in the same way a diff is.
Deliberately not in scope
Letting an agent decide for itself that an item is "really" an investigation
and skip its checks. The item declares its kind; the agent does not choose it.
Otherwise the first hard test failure becomes an essay about why the test was
wrong.
Every outcome path in the harness assumes the deliverable is a diff. There
is no way to express an item whose product is knowledge — an investigation, a
feasibility answer, an ADR, a comparison of three approaches.
What happens today if you try
Write the item anyway and the agent does the work well. Then:
session_executorsees a clean tree and recordsescalated / no_target — "the agent made no changes and left no reason"(An agent that correctly refuses an impossible item should be flagged as needing a person #174, shipped in An agent that says an item is impossible is asking for a person, not failing #175/An escalation lands in blocked, and does not read as a failed run #176).
nothing.
If the agent writes its findings to a file to avoid that, it is now a diff, and
it meets the rest of the pipeline:
cargo testhas no opinion on an ADR;REVIEW_PROMPTasks the reviewer to judge "your diff against the item", soit grades a document as though it were an implementation;
So the two available outcomes are "recorded as nothing" or "graded as code".
Why this matters beyond tidiness
It is the prerequisite for #181. Proposing a plan for an existing project
honestly requires investigating it first — reading the roadmap, checking what
is deployed, finding what the last release left open. If investigation cannot
be expressed as work, then plan generation is a single opaque model call rather
than something that can be split, checked, reviewed and retried like everything
else here.
It is also the shape of a large fraction of real engineering work. On
rdpappthe owner's own next package (P7.0) opens with "the first decision is the
attachment ADR: compare Node B's existing host-managed
tailscaled, a managedsidecar, and a tsnet/bridge approach". That is an item. The harness cannot
currently hold it.
What is actually needed
The interesting question is what "done" means when there is no diff, and it
should not be answered by loosening the gates:
answer, a recommendation with alternatives considered, a document at a named
path. Checks that cannot apply are skipped because the item says so, not
because they failed silently.
named alternatives actually examined? Is the recommendation supported by what
was found, or asserted? A reviewer told to judge a document against the
diff-rubric will reject it for being a document.
no_target. A finished investigation iscompleted, and the findings are its result, in the same way a diff is.Deliberately not in scope
Letting an agent decide for itself that an item is "really" an investigation
and skip its checks. The item declares its kind; the agent does not choose it.
Otherwise the first hard test failure becomes an essay about why the test was
wrong.
Related
needs this to be honest about how it reached its proposal.
fix there is right for what it covers; this is the case it cannot know about.