Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,77 @@ and sync it, exactly as though you had written it by hand.

---

## 0bb. Or point it at a project and state an objective

`inception` scopes a **new** project from prose. `survey` does the same for one
that already exists: you say what you want done, and the first run works out
what the items should be instead of being handed them.

```console
$ agent-harness survey "review and generate a plan to upgrade to Node v22" \
--work ./service --doc docs/roadmap.md \
--surveyor claude-sonnet-4-6 --endpoint $HARNESS_ENDPOINT --out PLAN.md
read 3 source(s): docs/roadmap.md, 412 tracked path(s), recent history
9 work item(s), 4 heading(s) skipped as narrative
blocking question: is the native addon in vendor/ still maintained upstream?
wrote PLAN.md
Review it, then: agent-harness adopt PLAN.md --project NAME --work ./service
```

**Name your roadmap with `--doc`.** Without it the harness guesses from a short
list (`docs/current-state.md`, `ROADMAP.md`, `README.md`, …) and stops at two.
A named file that is missing is *reported*, not skipped — the failure this
command exists to prevent is a confident plan built without the document that
states the project's direction.

**The gate is the harness's own parser.** The generated plan is read back by
`parse_plan`, the same function a hand-written plan goes through, so a plan the
queue would read differently from how it looks is caught here rather than three
commands later. If it cannot be read — no items, or duplicate ids, which cannot
each become one issue — nothing is written. `--force` overrides that, and means
executing a plan the harness has told you it does not understand.

**Nothing external happens.** No queue rows, no issues, no branches. Without
`--out` it prints the plan and writes nothing at all, which is the right
default for output whose entire purpose is to be argued with.

Blocking questions are reported and do **not** stop the file being written.
They are questions for you, and your answer decides — the same rule
`inception` applies at its approval gate.

### 0bb.1 Items that produce an answer rather than a change

Some work has no diff. "Compare these three approaches", "is this feasible",
"which of these is the cause" — the answer *is* the deliverable, and an item
like that used to leave a clean worktree and be recorded as
`escalated / no_target`: indistinguishable from an agent that did nothing.

An item can now say what it produces:

```markdown
### T1 — Choose the Tailnet attachment architecture

Compare a host-managed daemon, a managed sidecar, and a tsnet bridge. Say
which fits this deployment and what rules the others out.

deliverable: findings
```

`deliverable: code` is the default and means a diff, judged exactly as before.
`deliverable: findings` tells the agent to write its answer to
`.harness-findings.md` and change nothing else. The answer becomes the item's
result, the item completes, and the file is never committed.

**The plan declares this; the agent never chooses it.** Otherwise the first
hard test failure becomes an essay about why the test was wrong.

A findings item can still refuse — "this question cannot be answered from this
repository" is an escalation whatever the item was asked to produce — and one
that answers nothing at all lands in the same clean-tree path as any other
agent that did nothing, which is where it belongs.

---

## 0c. Or adopt a project that is already half-built

The common case is not a blank repository. It is a plan, a repository, some
Expand Down
115 changes: 115 additions & 0 deletions src/agent_harness/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -505,6 +505,7 @@ def _run(args: argparse.Namespace) -> int:
title=i.title,
brief=i.brief(),
depends_on=i.depends_on,
deliverable=i.deliverable,
project_id=args.project,
)
for i in plan.deduplicated()
Expand Down Expand Up @@ -859,6 +860,71 @@ def ask(prompt: str) -> str:
return ModelAssessor(ask)


def _survey(args: argparse.Namespace) -> int:
"""Read an existing project and propose a plan for a stated objective.

Writes nothing unless `--out` is given, and refuses to write a plan the
harness's own parser could not read. The parser is the gate on purpose: a
generated plan that the queue reads differently from how it looks is worse
than no plan, and catching it here costs one function call.
"""
from .executor import _text_of
from .model_client import ModelClient, Route
from .survey import SURVEYOR, survey

if not (args.work / ".git").exists():
print(f"{args.work} is not a git repository", file=sys.stderr)
return 2
if not args.surveyor:
print(
"--surveyor MODEL is required: this command's whole job is one model call",
file=sys.stderr,
)
return 2
if not args.endpoint:
print("--endpoint or $HARNESS_ENDPOINT is required", file=sys.stderr)
return 2

api_key = os.environ.get("HARNESS_API_KEY", "")
client = ModelClient(
roles={
SURVEYOR: Route(args.surveyor, args.endpoint, api_key=api_key, preset=args.preset),
},
transport=_http_transport(api_key),
)

def ask(prompt: str) -> str:
return str(_text_of(client.call(SURVEYOR, [{"role": "user", "content": prompt}]).body))

report = survey(
args.objective,
args.work,
ask=ask,
docs=list(args.doc),
name=args.name,
now=time.time(),
)
for line in report.lines():
print(line)

if not report.usable and not args.force:
print(
"refusing to write: the harness cannot read the plan it just generated. "
"Re-run, name the roadmap with --doc, or pass --force.",
file=sys.stderr,
)
return 1
if args.out is None:
print()
print(report.markdown)
print("(nothing was written; pass --out PATH to keep it)")
return 0
args.out.write_text(report.markdown)
print(f"wrote {args.out}")
print(f"Review it, then: agent-harness adopt {args.out} --project NAME --work {args.work}")
return 0


def _adopt(args: argparse.Namespace) -> int:
"""Inspect an existing project, then reconcile only with explicit approval.

Expand Down Expand Up @@ -1029,6 +1095,52 @@ def main(argv: list[str] | None = None) -> int:
help="where `export` writes its JSON (default: stdout)",
)

p_survey = sub.add_parser(
"survey", help="read an existing project and propose a plan for an objective"
)
p_survey.add_argument(
"objective",
help='what you want done, in prose: "review and generate a plan to upgrade to Node v22"',
)
p_survey.add_argument(
"--work", type=Path, default=Path("."), help="the existing repository to read"
)
p_survey.add_argument(
"--out",
type=Path,
default=None,
help="write the proposed plan here. Without this it is printed and nothing is "
"written, which is the safe default for a command whose whole output is a "
"proposal to argue with.",
)
p_survey.add_argument(
"--doc",
action="append",
default=[],
metavar="PATH",
help="a document that states this project's direction, relative to the "
"repository. Repeatable. Naming them beats the guesses: a plan built without "
"the roadmap is confident and wrong, and a named file that is missing is "
"reported rather than skipped silently.",
)
p_survey.add_argument("--name", default="Plan", help="title for the generated plan")
p_survey.add_argument("--surveyor", default="", help="model for the surveyor role")
p_survey.add_argument(
"--endpoint",
default=os.environ.get("HARNESS_ENDPOINT", ""),
help="model API base url (or $HARNESS_ENDPOINT)",
)
p_survey.add_argument(
"--preset", default="", help="route preset, as for run: claw-bay, chat-completions, generic"
)
p_survey.add_argument(
"--force",
action="store_true",
help="write the plan even when the harness's own parser could not read it. "
"The parser is the gate here; overriding it means executing a plan the "
"queue will read differently from how it looks.",
)

p_adopt = sub.add_parser(
"adopt", help="inspect an existing project and propose a reconciliation"
)
Expand Down Expand Up @@ -1410,6 +1522,9 @@ def main(argv: list[str] | None = None) -> int:
if args.command == "adopt":
return _adopt(args)

if args.command == "survey":
return _survey(args)

store = EventStore(args.db)

if args.command == "ingest":
Expand Down
8 changes: 7 additions & 1 deletion src/agent_harness/adoption.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
from .executor import Checks
from .github import MARKER, GitHub
from .outcomes import ESCALATE, RETRY
from .plan import ParsedPlan, WorkItem
from .plan import CODE, ParsedPlan, WorkItem
from .work import CLAIMED, DONE, PENDING, Project, WorkQueue, WorkRecord, revives

#: The lifecycle of one adoption (proposal §5.1). `rejected` and `revise` are
Expand Down Expand Up @@ -195,6 +195,10 @@ class AdoptionItem:
#: The brief the queue is holding, so the report can tell a re-sync that
#: changes nothing from one that rewrites the item.
queue_brief: str | None = None
#: What the plan says this item produces. Carried through rather than
#: defaulted, or a `deliverable: findings` item would be adopted as one
#: that must produce a diff (#182).
deliverable: str = CODE
proposed_state: str = PENDING
evidence: list[Evidence] = field(default_factory=list)
candidates: list[ExternalCandidate] = field(default_factory=list)
Expand Down Expand Up @@ -595,6 +599,7 @@ def _inspect_item(
depends_on=list(item.depends_on),
queue_state=existing.state if existing else None,
queue_brief=existing.brief if existing else None,
deliverable=item.deliverable,
candidates=list(candidates),
)

Expand Down Expand Up @@ -868,6 +873,7 @@ def reconcile(self, project_id: str, *, dry_run: bool = False) -> AdoptionReport
title=item.title,
brief=item.brief,
depends_on=item.depends_on,
deliverable=item.deliverable,
state=DONE if item.item_id in approved else PENDING,
)
for item in report.items
Expand Down
21 changes: 19 additions & 2 deletions src/agent_harness/inception.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,11 +193,22 @@ def parse_proposal(text: str, revision: int, now: float, feedback: str | None =
)


def render_plan(proposal: Proposal, name: str) -> str:
def render_plan(proposal: Proposal, name: str, *, phases_as_items: bool = True) -> str:
"""A proposal as a PLAN.md the existing parser can read.

Headings use the `### T1 — Title` shape the parser recognises, so the
generated plan goes through exactly the same path as a hand-written one.

**`phases_as_items` decides whether a phase heading is itself work.** A
heading of `## P0 Upgrade` matches the parser's item pattern, because `P0`
is a well-formed id, so by default each phase becomes an item as well as a
container. That is deliberate for `inception` — real hand-written plans do
track phases as issues, and a generated plan should behave like one.

It is wrong for a plan meant to be executed straight away. The phase item's
brief is the phase's *rationale* — "because we need the runtime current" —
which is not a specification, and an agent that claims it is being asked to
implement a reason. `survey` therefore passes False.
"""
out: list[str] = [f"# {name}", ""]
if proposal.goal:
Expand Down Expand Up @@ -231,7 +242,13 @@ def render_plan(proposal: Proposal, name: str) -> str:
out += ["## Work", ""]
for phase in proposal.phases:
title = phase.get("title") or phase.get("id") or "Phase"
out += [f"## {phase.get('id', '')} {title}".strip(), ""]
if phases_as_items:
out += [f"## {phase.get('id', '')} {title}".strip(), ""]
else:
# "Phase" first: the word cannot start an id, so the heading stays
# readable and stops matching the item pattern.
marker = f"Phase {phase['id']} — " if phase.get("id") else "Phase — "
out += [f"## {marker}{title}".rstrip(), ""]
if phase.get("why"):
out += [str(phase["why"]), ""]
for item in phase.get("items") or []:
Expand Down
29 changes: 28 additions & 1 deletion src/agent_harness/plan.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,24 @@
#: treating it as one would fill the backlog with the document's structure.
ID = r"[A-Z][A-Z0-9]{0,7}-?\d{1,4}(?:\.\d{1,3})*"

#: What an item produces. `code` is a diff, judged by the checks and a reviewer
#: reading that diff. `findings` is an answer \u2014 a feasibility verdict, a
#: comparison, a recommendation \u2014 where there may be nothing to commit, and
#: where an empty worktree is the expected shape rather than a failure.
#:
#: Two kinds, not a taxonomy. A third would need a third review rubric and a
#: third definition of done, and neither exists yet.
CODE = "code"
FINDINGS = "findings"
DELIVERABLES = (CODE, FINDINGS)

_SEP = r"[:.)\s\u2010-\u2015-]+"
_HEADING = re.compile(rf"^(#{{2,6}})\s+({ID}){_SEP}\s*(.+?)\s*$")
_PLAIN_HEADING = re.compile(r"^(#{1,6})\s+(.+?)\s*$")
_CHECKBOX = re.compile(rf"^\s*[-*]\s+\[( |x|X)\]\s+(?:({ID}){_SEP}\s*)?(.+?)\s*$")
_TABLE_ROW = re.compile(rf"^\|\s*({ID})\s*\|(.+)$")
_META = re.compile(
r"^\s*(labels?|milestone|phase|depends[ _-]?on|size|risk|verify)\s*:\s*(.+?)\s*$",
r"^\s*(labels?|milestone|phase|depends[ _-]?on|size|risk|verify|deliverable)\s*:\s*(.+?)\s*$",
re.IGNORECASE,
)
_TABLE_SEP = re.compile(r"^\|[\s:|-]+\|$")
Expand Down Expand Up @@ -122,6 +133,14 @@ class WorkItem:
verification: list[str] | None = None
#: Line number in the source plan, so a reader can find it again.
line: int = 0
#: What this item produces, from `deliverable:`. `code` is the default and
#: means a diff. `findings` means the answer *is* the output — "is X
#: feasible", "compare these three approaches" — and there may be nothing
#: to commit at all.
#:
#: **The item declares this; the agent never chooses it.** Otherwise the
#: first hard test failure becomes an essay about why the test was wrong.
deliverable: str = CODE

def brief(self) -> str:
"""What an agent is told to do. The title alone is rarely enough;
Expand Down Expand Up @@ -465,6 +484,14 @@ def _apply_metadata(item: WorkItem) -> None:
item.depends_on.extend(_split(value))
elif key in ("size", "risk"):
item.labels.append(f"{key}:{value.lower()}")
elif key == "deliverable":
kind = value.strip().lower()
if kind not in DELIVERABLES:
raise ValueError(
f"item {item.id} deliverable must be one of {', '.join(DELIVERABLES)}, "
f"not {value.strip()!r}"
)
item.deliverable = kind
elif key == "verify":
try:
argv = json.loads(value)
Expand Down
Loading
Loading