Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,27 @@ The public site is intentionally static. It creates a scoped run request and kee
boundary as the CLI: product-path proof, proxy benchmark proof, and official scorer output must be
labeled separately. It does not collect tokens or repository credentials in the browser.

The portable CLI now includes the local intake layer that service uses first:

```bash
npx proofloop target --url https://your-app.example --write-runner-plan
npx proofloop target --dir . --write-runner-plan
```

`target` fetches the live URL or scans the codebase, recommends benchmark families with evidence,
detects any already-configured benchmark/browser scripts, writes
`.proofloop/target/latest-target-plan.json`, and can write a runnable
`.proofloop/runner/target.plan.json`. It does not invent official scores; missing adapters and
official scorer paths are recorded as blockers.

## Quickstart

```bash
npx proofloop init --agent auto --live # config + manifest + agent docs + scripts + live scaffold
npx proofloop doctor --json # setup checks and fix commands
npx proofloop manifest --dense # compact repo status for agents
npx proofloop ui contract --dense # stable selectors/actions/assertions
npx proofloop target --write-runner-plan # benchmark-family plan + runnable adapter discovery
npx proofloop prompt # kickoff prompt to paste into your coding agent
npx proofloop this-repo --goal "proofloop my latest updates" --write-runner-plan
npx proofloop runner run --plan proofloop.runner.json --budget-usd 100
Expand Down Expand Up @@ -130,6 +144,7 @@ script. With neither, it reports `no_gate` with exit code 2. An unconfigured gat
| `proofloop init --agent auto --live` | Add agent docs, manifest, package aliases, workflows, and rubrics. |
| `proofloop doctor [--json]` | Report node/git/agent readiness, manifest/docs/scripts, Playwright/browser readiness, GitHub workflow, UI contracts, and fix commands. |
| `proofloop manifest [--json\|--dense]` | Print project status: stack, commands, proof gates, workflows, UI contracts, blockers. |
| `proofloop target [--url <url>] [--write-runner-plan] [--json]` | Recommend benchmark families from a URL/codebase, detect configured adapters, and write target/runner plan receipts. |
| `proofloop docs agents --dense` | Print compact agent workflow instructions. |
| `proofloop ui contract\|component <id>` | Discover stable `data-testid` and `data-proofloop` selectors. |
| `proofloop template --list` / `proofloop template <id> --write` | List or write starter proof-loop templates. |
Expand Down Expand Up @@ -213,6 +228,12 @@ This is the external-orchestrator path for "proofloop my latest repo" style usag
execute that plan locally, but official benchmark meaning still belongs to your app-specific
scorers and receipts.

`proofloop target` is the next layer for "give ProofLoop a URL or codebase" usage. It matches known
families such as BankerToolBench/accounting, SpreadsheetBench, FinAuditing/FinMR, Finch,
WorkstreamBench, underwriting, research copilot, NodeAgent memory ingestion, and live-browser smoke
tests. It writes evidence and blockers so an agent or managed runner knows what adapter/scorer work
is still missing before it claims coverage.

The package does not pretend to know your app's official benchmark or browser flow by default. You
make that real by putting deterministic checks in `proofloop.config.json`: build, tests, Playwright
user flows, live deployment smoke checks, official scorers, or your own verifier. Proof Loop then
Expand Down
17 changes: 17 additions & 0 deletions dist/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ exports.runCli = runCli;
* proofloop tooluse <verify|init> expected-tool-use contracts
* proofloop ci install github write the GitHub Actions gate workflow
* proofloop prompt print the one-prompt kickoff
* proofloop target [--url <url>] [--write-runner-plan]
* proofloop this-repo [--goal ...] [--write-runner-plan] [--run]
* proofloop manifest|docs|template|workflow|ui|resume|report|charts|receipt|mcp
*
Expand All @@ -31,6 +32,7 @@ const receipts_1 = require("./receipts");
const mcp_1 = require("./mcp");
const project_1 = require("./project");
const runner_1 = require("./runner");
const targetPlan_1 = require("./targetPlan");
exports.MCP_SERVER_RUNNING = -999;
/** Parse `--flag`, `--flag value`, `--flag=value`, and positionals. */
function parseArgs(argv) {
Expand Down Expand Up @@ -94,6 +96,7 @@ function usage() {
" charts latest write local JSON/SVG proof charts",
" receipt verify --file <path> verify app-produced proof receipts",
" runner run|resume|status|report durable append-only task runner with budget and resume",
" target [--url <url>] [--write-runner-plan] recommend benchmark families and write target receipt",
" mcp start the optional read-only MCP server",
" prompt print the one-prompt kickoff",
" this-repo [--goal <text>] [--write-runner-plan] [--run]",
Expand Down Expand Up @@ -161,6 +164,8 @@ function runCli(argv) {
return runReceiptCommand(positional[1], options, root);
case "runner":
return runRunnerCommand(positional[1], options, root);
case "target":
return runTargetCommand(options, root);
case "mcp":
(0, mcp_1.startMcpServer)({ root });
return exports.MCP_SERVER_RUNNING;
Expand Down Expand Up @@ -331,6 +336,18 @@ async function runRunnerCommand(sub, options, root) {
});
return result.exitCode;
}
async function runTargetCommand(options, root) {
const result = await (0, targetPlan_1.runProofloopTarget)({
root,
...(str(options.url) !== undefined ? { url: str(options.url) } : {}),
...(str(options.out) !== undefined ? { outPath: str(options.out) } : {}),
writeRunnerPlan: options["write-runner-plan"] === true || options.runner === true,
json: options.json === true,
dense: options.dense === true,
...(num(options["timeout-ms"]) !== undefined ? { timeoutMs: num(options["timeout-ms"]) } : {}),
});
return result.exitCode;
}
function runHooksCommand(sub, options, root) {
switch (sub) {
case "install": {
Expand Down
1 change: 1 addition & 0 deletions dist/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,6 @@ export * from "./project";
export * from "./mcp";
export * from "./runner";
export * from "./layeredPlan";
export * from "./targetPlan";
export * from "./receipts";
export { runCli } from "./cli";
1 change: 1 addition & 0 deletions dist/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ __exportStar(require("./project"), exports);
__exportStar(require("./mcp"), exports);
__exportStar(require("./runner"), exports);
__exportStar(require("./layeredPlan"), exports);
__exportStar(require("./targetPlan"), exports);
__exportStar(require("./receipts"), exports);
var cli_1 = require("./cli");
Object.defineProperty(exports, "runCli", { enumerable: true, get: function () { return cli_1.runCli; } });
86 changes: 86 additions & 0 deletions dist/targetPlan.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import type { ProofloopRunnerPlan } from "./runner";
export type ProofloopTargetKind = "codebase" | "live-url" | "hybrid";
export type ProofloopBenchmarkFit = "strong" | "medium" | "weak";
export type ProofloopAdapterStatus = "configured" | "candidate" | "blocked";
export type ProofloopBenchmarkRecommendation = {
id: string;
title: string;
fit: ProofloopBenchmarkFit;
confidence: number;
adapterStatus: ProofloopAdapterStatus;
officialScoreStatus: "configured_command" | "requires_adapter" | "not_bundled";
evidence: string[];
configuredScripts: Array<{
name: string;
command: string;
}>;
notes: string[];
};
export type ProofloopTargetPlan = {
schema: "proofloop-target-plan-v1";
generatedAt: string;
target: {
kind: ProofloopTargetKind;
root?: string;
url?: string;
packageName?: string;
httpStatus?: number;
title?: string;
};
summary: {
recommendedFamilies: number;
configuredAdapters: number;
blockedFamilies: number;
liveUrlReachable: boolean | null;
officialScoreReady: boolean;
runnerPlanReady: boolean;
};
recommendations: ProofloopBenchmarkRecommendation[];
runnerPlan?: ProofloopRunnerPlan;
blocked: string[];
nextActions: string[];
honesty: string;
};
export type ProofloopTargetResult = {
exitCode: number;
plan: ProofloopTargetPlan;
planPath: string;
runnerPlanPath?: string;
};
export type ProofloopTargetOptions = {
root: string;
url?: string;
outPath?: string;
writeRunnerPlan?: boolean;
json?: boolean;
dense?: boolean;
timeoutMs?: number;
log?: (message: string) => void;
logError?: (message: string) => void;
};
type TargetSignals = {
root: string;
packageName?: string;
scripts: Record<string, string>;
text: string;
evidence: string[];
};
type UrlSignals = {
url: string;
ok: boolean;
status?: number;
title?: string;
text: string;
evidence: string[];
};
export declare function runProofloopTarget(options: ProofloopTargetOptions): Promise<ProofloopTargetResult>;
export declare function writeProofloopTargetPlan(options: ProofloopTargetOptions): Promise<ProofloopTargetResult>;
export declare function buildProofloopTargetPlan(args: {
root: string;
codebaseSignals?: TargetSignals;
urlSignals?: UrlSignals;
generatedAt?: string;
}): ProofloopTargetPlan;
export declare function classifyBenchmarkFamilies(textInput: string, scripts?: Record<string, string>, hasLiveUrl?: boolean, seedEvidence?: string[]): ProofloopBenchmarkRecommendation[];
export declare function formatProofloopTargetPlanDense(plan: ProofloopTargetPlan, planPath: string, runnerPlanPath?: string): string;
export {};
Loading
Loading