Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,15 @@ whichever key is present, or set explicitly with `--provider` /
| `openai` | `OPENAI_API_KEY` | `gpt-4o` |
| `gemini` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `gemini-2.0-flash` |
| `deepseek` | `DEEPSEEK_API_KEY` | `deepseek-chat` |
| `host` | none — uses an installed agentic CLI | — |

If no API key is set, GithubPill falls back to the **`host`** provider and
drives the agentic CLI already on your machine (`claude`, `opencode`, `codex`,
or `pi`) to do the reasoning, so it works with **no API key**. Set
`GITHUBPILL_AGENT` to choose one explicitly.

```bash
export ANTHROPIC_API_KEY=... # or OPENAI_API_KEY / GEMINI_API_KEY
export ANTHROPIC_API_KEY=... # or OPENAI_API_KEY / GEMINI_API_KEY / DEEPSEEK_API_KEY
export GITHUB_TOKEN=... # optional; raises GitHub rate limits
```

Expand Down Expand Up @@ -195,4 +201,4 @@ the Actions token to write, or you must pass a personal access token instead.

## License

MIT — see [LICENSE](./LICENSE).
MIT License
16 changes: 14 additions & 2 deletions skills/githubpill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Flags:
|---|---|
| `--deep` | clone top candidates and cite `path:LINE` evidence |
| `--explore` | explore a space instead of validating an idea |
| `--provider <id>` | LLM provider: `anthropic`, `openai`, `gemini`, `deepseek` |
| `--provider <id>` | LLM provider: `anthropic`, `openai`, `gemini`, `deepseek`, `host` |
| `--json` / `--html` | also write machine-readable / self-contained HTML reports |
| `--out <dir>` | output directory (default `githubpill-reports`) |
| `--sources github,npm,pypi,hackernews` | restrict sources |
Expand All @@ -54,13 +54,25 @@ Flags:

## Requirements

- One LLM API key — `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`,
An LLM, in one of two ways:

- **The host agent itself (no API key).** If no API key is set, the CLI falls
back to the `host` provider and drives the installed agentic CLI
(`claude`, `opencode`, `codex`, or `pi`) to do the reasoning. This is the
zero-config path: the user's existing agent does the LLM work.
- **An API key** — `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`,
`GEMINI_API_KEY` / `GOOGLE_API_KEY`, or `DEEPSEEK_API_KEY`. Auto-detected, or
set with `--provider`.

Also:

- `GITHUB_TOKEN` or `GH_TOKEN` — optional but strongly recommended; raises the
GitHub rate limits. Falls back to `gh auth token`.
- Node.js ≥ 20.

Query generation uses the same LLM when one is available; set
`GITHUBPILL_LLM_QUERIES=0` to use the built-in heuristic planner instead.

## Present the result

Reports land in `./githubpill-reports/` (one per idea or topic). Relay to the user:
Expand Down
3 changes: 3 additions & 0 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,14 @@ Providers and API keys (first key found selects the provider):
openai OPENAI_API_KEY default model ${DEFAULT_MODELS.openai}
gemini GEMINI_API_KEY or GOOGLE_API_KEY default model ${DEFAULT_MODELS.gemini}
deepseek DEEPSEEK_API_KEY default model ${DEFAULT_MODELS.deepseek}
host (no key) uses an installed agentic CLI: claude, opencode, codex, pi

Other environment:
GITHUB_TOKEN / GH_TOKEN Optional; raises GitHub rate limits (falls back to \`gh auth token\`)
GITHUBPILL_PROVIDER Force a provider instead of auto-detecting
GITHUBPILL_MODEL Override the model id
GITHUBPILL_AGENT Agentic CLI for the host provider (claude|opencode|codex|pi)
GITHUBPILL_LLM_QUERIES Set to 0 to use the heuristic query planner
GITHUBPILL_LOG silent | error | warn | info | debug

Examples:
Expand Down
18 changes: 16 additions & 2 deletions src/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,22 @@ describe("loadConfig", () => {
);
});

it("throws when no key is present", () => {
expect(() => loadConfig({}, noGh)).toThrow(ConfigError);
it("falls back to the host provider when no key is present", () => {
expect(loadConfig({}, noGh).llm.provider).toBe("host");
});

it("accepts an explicit host provider without a key", () => {
expect(loadConfig({ GITHUBPILL_PROVIDER: "host" }, noGh).llm.provider).toBe("host");
});

it("reads GITHUBPILL_AGENT for the host provider", () => {
const config = loadConfig({ GITHUBPILL_PROVIDER: "host", GITHUBPILL_AGENT: "opencode" }, noGh);
expect(config.llm.agent).toBe("opencode");
});

it("generates queries with the LLM by default, unless disabled", () => {
expect(loadConfig({ OPENAI_API_KEY: "k" }, noGh).llmQueries).toBe(true);
expect(loadConfig({ OPENAI_API_KEY: "k", GITHUBPILL_LLM_QUERIES: "0" }, noGh).llmQueries).toBe(false);
});

it("names the missing key when the provider is explicit", () => {
Expand Down
22 changes: 14 additions & 8 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ export const DEFAULT_MODELS: Record<ProviderId, string> = {
openai: "gpt-4o",
gemini: "gemini-2.0-flash",
deepseek: "deepseek-chat",
host: "host",
};

/** Env vars checked, in order, for each provider's API key. */
Expand All @@ -17,20 +18,24 @@ const API_KEY_ENV: Record<ProviderId, readonly string[]> = {
openai: ["OPENAI_API_KEY"],
gemini: ["GEMINI_API_KEY", "GOOGLE_API_KEY"],
deepseek: ["DEEPSEEK_API_KEY"],
host: [],
};

const BASE_URL_ENV: Record<ProviderId, string> = {
anthropic: "ANTHROPIC_BASE_URL",
openai: "OPENAI_BASE_URL",
gemini: "GEMINI_BASE_URL",
deepseek: "DEEPSEEK_BASE_URL",
host: "",
};

export interface LlmConfig {
provider: ProviderId;
apiKey: string;
model: string;
baseUrl?: string;
/** For the `host` provider: which agentic CLI to invoke. */
agent?: string;
maxTokens: number;
timeoutMs: number;
}
Expand All @@ -45,6 +50,8 @@ export interface Config {
concurrency: number;
requestTimeoutMs: number;
maxQueriesPerSource: number;
/** Generate search queries with the LLM instead of the heuristic planner. */
llmQueries: boolean;
/** Deep mode: how many top candidates to clone and inspect. */
deepCandidates: number;
cloneTimeoutMs: number;
Expand Down Expand Up @@ -83,12 +90,9 @@ function resolveProvider(env: NodeJS.ProcessEnv): ProviderId {
return requested as ProviderId;
}

const detected = PROVIDER_IDS.find((provider) => apiKeyFor(env, provider));
if (!detected) {
const keys = PROVIDER_IDS.flatMap((provider) => API_KEY_ENV[provider]).join(", ");
throw new ConfigError(`No LLM API key found. Set one of: ${keys}.`);
}
return detected;
// Prefer a real API key; fall back to the host agent, which needs none.
const detected = PROVIDER_IDS.find((provider) => provider !== "host" && apiKeyFor(env, provider));
return detected ?? "host";
}

function resolveSources(env: NodeJS.ProcessEnv): SourceId[] {
Expand Down Expand Up @@ -128,8 +132,8 @@ export interface ConfigDeps {

export function loadConfig(env: NodeJS.ProcessEnv = process.env, deps: ConfigDeps = {}): Config {
const provider = resolveProvider(env);
const apiKey = apiKeyFor(env, provider);
if (!apiKey) {
const apiKey = apiKeyFor(env, provider) ?? "";
if (provider !== "host" && !apiKey) {
throw new ConfigError(
`Provider "${provider}" is selected but no API key is set. Set ${API_KEY_ENV[provider].join(" or ")}.`,
);
Expand Down Expand Up @@ -163,13 +167,15 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env, deps: ConfigDep
maxTokens: limits.maxTokens,
timeoutMs: limits.requestTimeoutMs,
...(env[BASE_URL_ENV[provider]] ? { baseUrl: env[BASE_URL_ENV[provider]] as string } : {}),
...(env.GITHUBPILL_AGENT ? { agent: env.GITHUBPILL_AGENT } : {}),
},
sources: resolveSources(env),
perSourceLimit: limits.perSourceLimit,
maxCandidates: limits.maxCandidates,
concurrency: limits.concurrency,
requestTimeoutMs: limits.requestTimeoutMs,
maxQueriesPerSource: limits.maxQueriesPerSource,
llmQueries: env.GITHUBPILL_LLM_QUERIES !== "0",
deepCandidates: limits.deepCandidates,
cloneTimeoutMs: limits.cloneTimeoutMs,
deepMaxFiles: limits.deepMaxFiles,
Expand Down
1 change: 1 addition & 0 deletions src/explore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ export async function explore(options: ExploreOptions): Promise<ExploreResult> {
const retrieval = await retrieve({
idea: options.topic,
config: options.config,
llm: options.llm,
adapters: options.adapters,
onProgress: progress,
signal: options.signal,
Expand Down
1 change: 1 addition & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ export type { SearchOptions, SourceAdapter } from "./adapters/index.js";
export {
AnthropicClient,
GeminiClient,
HostClient,
OpenAICompatibleClient,
createLLMClient,
} from "./synthesis/providers/index.js";
Expand Down
28 changes: 27 additions & 1 deletion src/retrieval/query-plan.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { describe, expect, it } from "vitest";
import { extractPreservedTerms, extractProductNames, planQueries } from "./query-plan.js";
import { extractPreservedTerms, extractProductNames, planQueries, planQueriesWithLlm } from "./query-plan.js";
import { FakeLLM } from "../testing/fakes.js";

describe("planQueries", () => {
it("preserves acronyms and hyphenated jargon", () => {
Expand Down Expand Up @@ -34,3 +35,28 @@ describe("planQueries", () => {
expect(plan.sharpened.length).toBeLessThanOrEqual(200);
});
});

describe("planQueriesWithLlm", () => {
it("uses the model plan and keeps heuristic preserved terms", async () => {
const llm = new FakeLLM({
query_plan: {
sharpened: "An NDIS invoice validator",
keywords: ["ndis invoice checker"],
productNames: ["SomeProduct"],
topics: ["ndis"],
preservedTerms: [],
},
});

const plan = await planQueriesWithLlm("An NDIS invoice validator for providers", llm);

expect(plan.keywords).toEqual(["ndis invoice checker"]);
expect(plan.productNames).toEqual(["SomeProduct"]);
expect(plan.preservedTerms).toContain("NDIS");
});

it("falls back to the heuristic when the model fails", async () => {
const plan = await planQueriesWithLlm("a todo cli", new FakeLLM());
expect(plan.keywords.length).toBeGreaterThan(0);
});
});
42 changes: 42 additions & 0 deletions src/retrieval/query-plan.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { z } from "zod";
import type { LLMClient } from "../synthesis/providers/types.js";
import type { QueryPlan } from "../types.js";

const STOPWORDS = new Set(
Expand Down Expand Up @@ -140,3 +142,43 @@ export function planQueries(idea: string): QueryPlan {
preservedTerms,
};
}

const QueryPlanSchema = z.object({
sharpened: z.string().describe("One sentence restating the idea"),
keywords: z.array(z.string()).describe("Two to four word search phrases"),
productNames: z.array(z.string()).describe("Known product or project names in this space"),
topics: z.array(z.string()).describe("GitHub topic tags, lowercase and hyphenated"),
preservedTerms: z.array(z.string()).describe("Proper nouns and jargon that must survive verbatim"),
});

const QUERY_SYSTEM = [
"You write search queries for a prior-art tool.",
"Given a project idea, produce diverse queries that will surface existing projects in that space.",
"Cover literal phrasing, near synonyms, the user-facing outcome, adjacent domains, and the names of known products or projects.",
"Preserve proper nouns and jargon verbatim. Never invent projects.",
].join(" ");

/**
* Ask the LLM for a query plan. Falls back to the heuristic plan on any
* failure, so a flaky model never blocks retrieval.
*/
export async function planQueriesWithLlm(idea: string, llm: LLMClient): Promise<QueryPlan> {
const fallback = planQueries(idea);
try {
const output = await llm.completeStructured({
system: QUERY_SYSTEM,
prompt: `IDEA: ${idea}\n\nProduce the query plan.`,
schema: QueryPlanSchema,
schemaName: "query_plan",
});
return {
sharpened: output.sharpened || fallback.sharpened,
keywords: output.keywords.length > 0 ? output.keywords : fallback.keywords,
productNames: output.productNames.length > 0 ? output.productNames : fallback.productNames,
topics: output.topics.length > 0 ? output.topics : fallback.topics,
preservedTerms: [...new Set([...fallback.preservedTerms, ...output.preservedTerms])],
};
} catch {
return fallback;
}
}
10 changes: 8 additions & 2 deletions src/retrieval/retrieve.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
import type { Candidate, ProgressHandler, QueryPlan, SourceRun } from "../types.js";
import type { Config } from "../config.js";
import { createAdapters, type SourceAdapter } from "../adapters/index.js";
import { planQueries } from "./query-plan.js";
import { planQueries, planQueriesWithLlm } from "./query-plan.js";
import { fanout, type AdapterError } from "./fanout.js";
import { rankCandidates } from "./rank.js";
import { verifyCandidates } from "../verify/citations.js";
import type { LLMClient } from "../synthesis/providers/types.js";

export interface RetrieveOptions {
idea: string;
config: Config;
/** When present (and enabled in config), queries are LLM-generated. */
llm?: LLMClient;
/** Defaults to the adapters for `config.sources`. Inject for tests. */
adapters?: SourceAdapter[];
onProgress?: ProgressHandler;
Expand All @@ -35,7 +38,10 @@ export async function retrieve(options: RetrieveOptions): Promise<Retrieval> {
const progress = options.onProgress;

progress?.({ type: "stage", stage: "plan" });
const plan = planQueries(options.idea);
const plan =
options.llm && config.llmQueries
? await planQueriesWithLlm(options.idea, options.llm)
: planQueries(options.idea);

progress?.({ type: "stage", stage: "search" });
const { hits, runs, errors } = await fanout(adapters, plan, config, progress, options.signal);
Expand Down
52 changes: 52 additions & 0 deletions src/synthesis/providers/host.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import { describe, expect, it } from "vitest";
import { z } from "zod";
import { HostClient, resolveHost, type HostRunner } from "./host.js";

const schema = z.object({ answer: z.string() });
const options = { apiKey: "", model: "host", maxTokens: 10, timeoutMs: 1_000 };
const request = { system: "be terse", prompt: "answer", schema, schemaName: "emit" };

describe("HostClient", () => {
it("sends the schema and parses JSON out of fenced output", async () => {
let captured = "";
const runner: HostRunner = async (_command, args) => {
captured = args.join(" ");
return 'Here you go:\n```json\n{"answer":"ok"}\n```\n';
};

const client = new HostClient({ ...options, agent: "claude" }, runner);
const result = await client.completeStructured(request);

expect(client.provider).toBe("host");
expect(result).toEqual({ answer: "ok" });
expect(captured).toContain("JSON Schema");
expect(captured).toContain('"answer"');
});

it("extracts JSON embedded in prose", async () => {
const runner: HostRunner = async () => 'sure — {"answer":"ok"} hope that helps';
const client = new HostClient({ ...options, agent: "opencode" }, runner);
await expect(client.completeStructured(request)).resolves.toEqual({ answer: "ok" });
});

it("surfaces a host failure", async () => {
const runner: HostRunner = async () => {
throw new Error("boom");
};
const client = new HostClient({ ...options, agent: "claude" }, runner);
await expect(client.completeStructured(request)).rejects.toThrow(/boom/);
});
});

describe("resolveHost", () => {
it("uses a known command for a named agent", () => {
expect(resolveHost("claude").command).toBe("claude");
expect(resolveHost("codex").command).toBe("codex");
});

it("treats an unknown name as a custom command", () => {
const host = resolveHost("my-agent");
expect(host.command).toBe("my-agent");
expect(host.args("hello")).toEqual(["hello"]);
});
});
Loading
Loading