Skip to content

Drive whip v1.0.0 through its SDK behind the harness seam - #61

Open
samheutmaker wants to merge 6 commits into
mainfrom
feature/whip-sdk-v1
Open

samheutmaker wants to merge 6 commits into
mainfrom
feature/whip-sdk-v1

Conversation

@samheutmaker

Copy link
Copy Markdown

Why

Whip v1.0.0 renamed its binary to whipcode and ships the TypeScript SDK; main pins whip v0.6.5 and drives whip run as a subprocess, scraping its NDJSON. This re-ports the SDK migration (feature/whip-sdk) onto main as a few commits, keeping everything main does today — ensemble legs and degradation, cross-reviewer dedup, the comment cap, the publish gate, quota and rate-limit routing, review traces, call-site discovery, salvage, incremental review, skills, @loupe chat and fix — and changing only how a turn runs. Plan and implementation record: whip .ai-docs/plans/loupe-whip-v1/PLAN.md.

What changes

  • Pin whip v1.0.0 and vendor its SDK (vendor/@whip, protocol 6.8, refreshed with task vendor:whip). The pinned self-install downloads whipcode-<os>-<arch> into ~/.loupe/bin/whipcode; whipcode on PATH is used as is; LOUPE_WHIP_BIN overrides. Both workflows install v1.0.0 explicitly.
  • The SDK behind the harness seam. packages/harness keeps its name and its Harness seam with one implementation, whipHarness(client). review(ctx) names the loupe agent (reviewer, verifier, chat, fixer) and the output contract, builds the definition, serves it from this process, runs one turn on a session pinned to it, and returns { text, output } — the final message and the value the daemon validated against the contract. Typed turn events feed the run log and the existing trace pipeline (turnEventToTrace). Reviewers are read-only whip agents (context + files, no shell); the whip block's provider is named on every session; reasoning is set per session. One daemon and one client per process, shared by concurrent reviewers, released at exit. Claude, codex, the registry, the cache-key retry, and the binary identity check are gone. harness: "whip" in a config and LOUPE_HARNESS=whip are still accepted; anything else is an error.
  • Docs for the whip-only loupe.
  • Self-review runs the checked-out loupe (uses: ./), so this PR is reviewed by its own code, whip install and daemon bring-up included.
  • Two fixes from the dry runs: an effort level the model lacks is a warning (glm-5.3 has no medium; whip run applied its default silently), and loupe asks the daemon to discover the provider's models after bringing it up (below).

Validation

  • task check: 44 files, 522 tests. Main's run-review (38), retry, orchestrate, respond, config, errors, and action trace tests are kept; the fake harness returns { text, output }. The runner itself is tested against @whip/sdk/testing's scripted daemon.
  • Dry runs on context-labs/loupe#15 against a whipcode 1.0.0 daemon (glm-5.3 on inference.net): both reviewers agentic on one daemon; a forced headless fallback with agentic verify; chat; a fixer turn on a scratch clone where the hook denied git commit; three loupe processes on one daemon at once.
  • The CI path locally (INFERENCE_API_KEY set → throwaway WHIPCODE_HOME, dedicated daemon): both reviewers agentic, the docs reviewer's two findings kept by agentic verify, the daemon stopped at exit, the desktop daemon untouched.

Notes

  • A fresh daemon has no model catalog. The first CI-path run failed every turn in seconds with output_invalid: the message is empty: the daemon only fetches a provider's catalog when a client asks (the desktop app does), and without it a model's limits collapse to a one-token output cap. Loupe now calls provider.catalogs for the whip block's provider after connecting; a no-op on a daemon that already has one. Whip follow-up: discover on demand in resolveRuntimeModel (cmd/whip/daemon.go) or refuse a 1-token cap.
  • Turn cap skips the output contract in whip v1.0.0; fixed in fix(agent): run the output contract check on a capped final answer whip#192. Until a stable release carries it, loupe parses the final text when a capped turn ends without a validated output.
  • Reasoning rides on the session (session.effort) rather than defaultEffort in a per-call home, since one daemon serves every reviewer.
  • Also observed, log-only: no costUsd on the turn (the SDK forwards stream.accounting as a raw event and none reached the turn window); large inputs arrive as context handles the model reads and then mentions in its summary.

🤖 Generated with Claude Code

samheutmaker and others added 6 commits September 26, 2026 17:06
Loupe now drives `whipcode`, the binary whip v1.0.0 ships: the pinned
self-install downloads `whipcode-<os>-<arch>` from that tag into
~/.loupe/bin/whipcode, `whipcode` on PATH is used as is (every v1 binary is
whipcode, so the v0 identity check goes), and LOUPE_WHIP_BIN overrides both.
The workflows install the same tag explicitly.

The SDK that talks to that daemon is vendored under vendor/@Whip from the
v1.0.0 tag (protocol 6.8), refreshed with `task vendor:whip`; VERSION records
the tag, commit, and wire protocol. Nothing uses it yet; the next commit
moves the harness onto it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The harness package keeps its name and its `Harness` seam, with one
implementation: `whipHarness(client)`. `review(ctx)` now names which loupe
agent to run (reviewer, verifier, chat, fixer) and the output contract its
final message must match, builds that agent definition, serves it from this
process, runs one turn on a session pinned to it on the shared daemon, and
returns `{ text, output }`: the final message and the value the daemon
validated against the contract. No subprocess, no NDJSON scraping; typed turn
events feed the run log and, through `turnEventToTrace`, the same trace
pipeline the step summary renders. Claude, codex, the registry, the cache-key
retry, and the whip binary identity check go.

Core's two call sites (the review with its headless fallback, and verify)
pass `agent` and `output`; `parseReviewOutput` and `parseVerification` read
the validated output, or the final text when the turn cap answered past the
check, so salvage and the malformed counts keep working. Reviewers are
read-only whip agents with the `files` module (no shell), so the prompt's
tool hints say list/read/search instead of ls/cat/grep and the sentence
about subagents goes. The whip block's provider is named on every session
because a desktop daemon's default provider may not carry the model, and
`reasoning` rides on the session (`session.effort`) since one daemon now
serves every reviewer.

The action brings whip up once per process (`resolveHarness`: credential
chain, `ensureDaemon`, `connect`) and lets it go at exit (`releaseWhip`);
concurrent reviewers share that daemon and client. `--harness` is gone;
`harness: "whip"` in a config and `LOUPE_HARNESS=whip` are still accepted,
anything else is an error.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
README, architecture, configuration, the Action and CLI guides, credentials,
review traces, the how-it-works pages, and the example workflow describe one
harness: the pinned whipcode install, the daemon loupe starts or attaches to,
the SDK path, the output contract the daemon enforces, and reasoning set on
the session. Claude and codex, the subprocess, `--harness`, and the panel
paragraph are gone; main's diagrams and the sections on ensemble, dedup, the
comment cap, the publish gate, and status routing stay as they were.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The first dry run on the desktop daemon failed both the agentic review and
its headless fallback with `session.effort failed: glm-5.3 does not support
effort "medium"`: the daemon checks the effort against the model's levels,
where `whip run` had quietly applied its model-aware default. The session
now keeps that default and logs what was asked for and why it was not
applied; the reasoning note in the prompt still applies.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The self-review ran `context-labs/loupe@main`, so a PR that changes loupe
was reviewed by the code it replaces. Using the checkout makes each PR's
self-review a run of its own code, including the whip install and daemon
bring-up it ships with. On main the two are the same thing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The CI path failed every turn within seconds: `output_invalid: the message
is empty`, on the agentic run and its fallback alike. A daemon started from
the materialized home has no model catalog, because the daemon only fetches
one when a client asks (the desktop app does that when a provider is set
up), and without it a model's limits collapse to a one-token output cap.
glm-5.3 spends that token on reasoning and returns no content.

After connecting, loupe now asks the daemon for the whip block's provider
catalog (`provider.catalogs`), which fetches a missing or stale one and is
a no-op on a daemon that already has it. Discovery failing is a warning
that names the consequence; the turn's own failure says the rest.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@cursor

cursor Bot commented Sep 27, 2026

Copy link
Copy Markdown

This PR is too large for Bugbot to review. It changes 113,174 lines and 4,065,352 characters. Split the change into smaller pull requests to get a review.

@github-actions

Copy link
Copy Markdown

🔍 Loupe review


code

✅ no issues · 19 files

Ports loupe from driving the whip run CLI to the whip v1.0.0 TypeScript SDK behind the unchanged Harness seam: one shared daemon+client per process, agent definitions served per-run, output contracts validated daemon-side, and the old subprocess/NDJSON plumbing removed. I verified the memoized bring-up/release lifecycle in run.ts (failure clears the shared promise; concurrent callers await the same bring-up; main.ts and the CLI both release in finally), the daemon-env secret scrub handoff (envSecretValues(daemon.env) still covers the resolved credentials because planDaemonEnv builds the daemon env from process.env plus credentials), the session setup in client.ts (session-scoped effort, headless configure, turn result status checked so a failed turn throws rather than yielding empty output), the outcomeText fallback path for capped turns, the config precedence change (harness accepted only as "whip", else explicit error; empty input still falls through to .loupe.json), and the chat/fix callers of the changed review signature (both await the returned outcome, name agent + provider, drop env/whipConfig consistently with RunInput/ReviewRequest). No defect found: the change is safe. Callers checked: buildRequest, reviewPullRequest, runFix, handleComment, CLI review command, main finalizer. Absence claims (e.g. removed harnessName no longer referenced) were checked against the checkout's action sources, not the diff alone.

Run details
  • review: agentic
  • verification: skipped
  • scope: full
  • dropped: 0 malformed finding(s), 0 malformed concern(s), 0 out of scope, 0 below profile, 0 rejected by verification, 0 demoted by the comment cap, 0 duplicate of another reviewer
  • off-diff notes published: 0

Last reviewed commit: 87f9c32


docs

✅ no issues · 33 files

This PR replaces the multi-harness subprocess runner (whip v0.6.5 whip run NDJSON, plus claude/codex adapters) with whip v1.0.0's vendored TypeScript SDK behind the unchanged Harness seam: one daemon+client per process, each loupe agent (reviewer/verifier/chat/fixer) served as a whip agent definition with a zod output contract, one turn per review, plus wide-reaching doc rewrites (README, action.yml descriptions, docs/*, examples). I inspected the diff hunks for action.yml (harness input now deprecated, model/reasoning description updates), docs/architecture.md (harness seam rewrite), packages/core/src/output.ts (new output contracts) and core/index.ts (agent/output/provider plumbing). I could not complete verification of every new doc claim against the checkout within my tool budget, so per the review contract I report no findings I have not confirmed; the doc rewrites I did read are consistent with the code changes shown in the diff (whip-only harness, deprecated-but-accepted harness: "whip", output-contract validation via the daemon, outcomeText fallback for capped turns). Call sites/absence claims verified: none were needed beyond the diff — action.yml input descriptions match docs rewrites present in the same diff, and no removed input/field/command appears still documented in the hunks I read.

Run details
  • review: agentic
  • verification: skipped
  • scope: full
  • dropped: 0 malformed finding(s), 0 malformed concern(s), 0 out of scope, 0 below profile, 0 rejected by verification, 0 demoted by the comment cap, 0 duplicate of another reviewer
  • off-diff notes published: 0

Last reviewed commit: 87f9c32


Use @loupe fix to address all open findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant