Add podtx nuggets: multi-provider LLM extraction of durable insights

[frarredondo]

Summary

Extract durable, non-obvious insights ("nuggets") from episode transcripts using an LLM, with the provider fully configurable — Anthropic, OpenAI, OpenRouter, and local LM Studio (OpenAI-compatible API), plus models.dev as the model catalog for validation and cost estimation — so extraction can run offline-first or against whichever hosted model the user prefers. Every quote is verified against the transcript and cited with a timestamp.

Status of referenced infrastructure (verified against main @ 1c094b9)

• No silent external calls + offline-friendly default: precedent already shipped in podtx summarize (Add podtx summarize for episode summaries and takeaways #8 → PR summarize: add openrouter/opencode/lmstudio backends with Keychain (fixes #34) #33, df7409b): default fake (offline extractive), LLM backends opt-in.
• Provider-layer precedent already shipped inside src/podtx/summarize.py (openrouter / opencode / lmstudio, Keychain lookup, OpenAI-compatible caller). Add podtx nuggets: multi-provider LLM extraction of durable insights #29 formalizes this into a shared package and adds anthropic + openai backends.
• Transcript shape supports mechanical quote verification: Transcript.text + Segment{start,end,text} (src/podtx/models.py).
• FTS5 search shipped (Add podtx search for full-text search across transcripts #4 → PR Add offline FTS5 search with incremental indexing (#4) #26): search_fts in src/podtx/db.py, podtx search command. Nugget indexing is additive.

Decisions (groomed 2026-08-30)

1. Shared provider package, built by this work. New src/podtx/providers/ package (see below) serving podtx nuggets. podtx summarize is not migrated in this effort (no churn to shipped, 100%-patch-covered code); migrating summarize onto the shared package is a separate follow-up ticket and non-blocking.
2. Settings naming mirrors summarize_* (not a generic llm_* prefix): nuggets_backend, nuggets_model, nuggets_base_url, nuggets_api_key_service, nuggets_api_key_account, nuggets_timeout, nuggets_max_input_chars — same TOML > env > CLI precedence as config.py, env prefix PODCAST_TRANSCRIBER_NUGGETS_*.
3. Auth follows house pattern. Extend podtx auth set to accept anthropic and openai (Keychain services podtx-anthropic / podtx-openai); env fallbacks ANTHROPIC_API_KEY / OPENAI_API_KEY / OPENROUTER_API_KEY; --api-key override. No key required for lmstudio. Never store keys in TOML.
4. Merge/cluster dedup mechanism specified: reuse the configured provider (prefer local lmstudio / inexpensive model) for pairwise semantic clustering; fallback to normalized quote/text overlap when no model is configured. Always disclose N episodes processed of M in scope.

Proposal

1. Pluggable LLM provider layer

src/podtx/providers/
  base.py        # Protocol: complete(system, prompt, *, schema=None) -> str | dict
  anthropic.py
  openai.py      # also serves any OpenAI-compatible base_url
  openrouter.py
  opencode.py    # shipped precedent in summarize; carry over
  lmstudio.py    # OpenAI-compatible, local base_url, no API key required
  catalog.py     # models.dev: model metadata (context window, pricing) for validation + cost estimates
  registry.py    # available_providers() / get_provider(name)

models.dev is a model metadata catalog, not an inference API — it slots in as the source for podtx models (list/validate models per provider) and for --dry-run cost estimates, cached locally so offline use still works.

Credential resolution per backend: Keychain (podtx-auth, service from auth set) > provider-standard env var > --api-key. No silent external calls is inherited from the summarize principle: a hosted provider is only used when explicitly configured; otherwise the command explains what to set.

podtx nuggets path/to/episode.json
podtx nuggets --feed corecursive-coding-stories --limit 10
podtx nuggets --feed corecursive-coding-stories --all --dry-run   # token + $ estimate, no calls
podtx nuggets --provider lmstudio --model qwen2.5-14b-instruct
podtx models --provider openrouter                                 # catalog listing via models.dev

2. Reproducible extraction prompt (scored rubric)

Replace free-form "extract the most surprising/actionable nuggets" with a scored rubric so results are consistent across providers/models. The prompt is versioned (prompt_version stamped into output) so results are attributable and reruns can detect staleness.

You are extracting durable, non-obvious wisdom from a podcast episode transcript for a software-engineering / technical-career audience.

Extract 3–7 nuggets. A nugget is one of: a memorable quote, a counterintuitive insight, a hard-won lesson, a mental model, or an operating principle.

Score each candidate 0–2 on every axis (silently; only output nuggets that clear the bar):

• Timelessness — still true/relevant in 5+ years? (0 = tied to a specific tool/version/news item, 2 = durable principle)
• Surprise — inverts a common assumption or reveals something non-obvious? (0 = restates conventional wisdom, 2 = genuinely counterintuitive)
• Evidence — backed by a concrete example, quote, or number? (0 = vague assertion, 2 = specific verbatim quote or data point)
• Actionability — could a listener apply or test this tomorrow? (0 = purely descriptive, 2 = a decision rule or practice)

Keep only nuggets scoring ≥5/8. Discard tool-version news, intro/outro chatter, sponsor reads, and guest self-promotion regardless of score.

Primary lens: software engineering and technical careers — architecture, debugging, incentives, team dynamics, hiring, career growth, technical decision-making. Tag each nugget [eng] for that lens, or [general] if it's broader wisdom that still clears the bar.

Per nugget, output:

• Insight (1–2 sentences, punchy, stated as a general principle — not "the guest said...")
• Context: guest name + episode
• Why it matters: 1 sentence, framed for a software engineer
• Quote: verbatim, <30 words, copied character-for-character from the transcript — never paraphrase as a quote; omit if none supports the insight
• Scores: T/S/E/A, e.g. T2 S2 E1 A1 = 6/8
• Tag: [eng] or [general]

Rank the episode's nuggets best-first by score.

3. Quote verification + timestamps (mechanical anti-hallucination check)

After extraction, verify each quote by (normalized) substring match against Transcript.text. A quote that doesn't match is dropped or demoted to an unquoted insight (E score capped at 1). Matched quotes are located in segments to attach [hh:mm:ss] timestamps — making every nugget citable and spot-checkable against the audio.

4. Long-transcript handling

Hour-long episodes run 12k–16k+ words (~20k tokens). Use the model's context window (from the models.dev catalog, or nuggets_max_input_chars/override for local models) to pick a strategy: single-pass when it fits, otherwise map-reduce — extract per chunk (split on segment boundaries with overlap), then merge with the same rubric.

5. Output format + storage

• Sidecar episode.nuggets.json (stable, documented schema: nuggets with scores, tags, quote, timestamp, char offsets; metadata with prompt_version, provider, model, token usage) and episode.nuggets.md for humans.
• Idempotent: skip episodes whose sidecar matches current prompt_version + model; --force re-extracts.
• Index nugget text into the existing FTS5 (search_fts) so podtx search can find insights, not just raw transcript.

6. Cross-episode merge/dedup pass

After per-episode extraction, an explicit podtx nuggets merge --feed … (or --merge) pass over all sidecars that:

• Clusters nuggets expressing the same underlying idea across guests/episodes (semantic via configured provider per Decision 4, with offline overlap fallback)
• Merges duplicates into one entry citing every source episode + timestamp, keeping the strongest quote and highest score
• Emits a corpus-level report: sampling disclosure (N episodes processed of M in scope) + "Best of Show" top 5–10 by score

Out of scope

Full RAG/chat-with-podcast interface; auto-publishing to a blog/newsletter; fine-tuning on extracted nuggets; speaker attribution beyond what metadata gives (that's #5 diarization, shipped in #32). Migrating podtx summarize onto src/podtx/providers/ is a tracked follow-up, not part of this issue.

Acceptance criteria

• podtx nuggets works against Anthropic, OpenAI, OpenRouter, and a local LM Studio server, selected via --provider/--model or config, with the same TOML > env > CLI precedence as transcription/summarize settings
• lmstudio path requires no API key; hosted providers are never called unless explicitly configured
• podtx auth set accepts anthropic and openai (Keychain services), alongside existing backends; keys never stored in TOML
• podtx models lists/validates models via the models.dev catalog; --dry-run prints token + cost estimate without any inference call
• Extraction applies the versioned 0–8 rubric; every emitted quote substring-matches the transcript and carries a timestamp
• Episodes exceeding the model's context window are chunked on segment boundaries and merged, not truncated silently
• Re-running skips episodes already extracted with the same prompt version + model; --force overrides
• Merge pass deduplicates same-idea nuggets across episodes into one multi-cited entry and discloses sample vs. corpus size
• Nuggets are searchable via the existing FTS5 podtx search
• Provider layer is unit-tested with a fake provider (no network in CI); schema-invalid model output gets one retry then a clear error
• Output schema, --help, and README documented

Revision log

• 2026-08-30: Reconciled with main — summarize (Add podtx summarize for episode summaries and takeaways #8) shipped in df7409b and its provider internals are precedent, not a dependency (Decision 1); settings renamed to nuggets_*; auth via extended podtx auth set; merge clustering mechanism specified. Original rubric/prompt, output schema, and acceptance criteria retained.

[frarredondo]

Groomed body to match main (2026-08-30). Key changes from the original spec:

• Add podtx summarize for episode summaries and takeaways #8/summarize is shipped (df7409b) and its OpenAI-compatible provider internals are taken as precedent — the shared src/podtx/providers/ package is built by this work; migrating summarize onto it is a separate, non-blocking follow-up.
• Settings renamed from generic llm_* to per-command nuggets_*, mirroring summarize_*.
• Auth aligned with house pattern: extend podtx auth set (anthropic/openai Keychain services) + env + --api-key; no TOML keys.
• Merge/dedup clustering mechanism specified: configured provider (local-first) with offline overlap fallback; sampling disclosure required.
• Rubric, quote-verification, chunking, output schema, FTS5 indexing, and acceptance criteria retained from the original proposal.

Additional context found while grooming: #8 was closed as shipped; its README-docs gap is tracked in #37.