Passport quality pass: 85/104 cleared, 9 oneLiners repaired#6
Merged
Conversation
…, clear manualReview on 85/104
Three-stage automated review pass on all 104 Wave-2 backfilled passports.
Stage 1 — seed:review detects quality issues
- Leaked oneLiners (JSON fragments, README-instruction fragments, Q&A fragments)
- Tautological oneLiners (self-referential to the slug)
- Short/vague oneLiners (<25 chars, random fragments like "mathew-johnson")
- Contradictory language tags (e.g. "typescript" on a package with only .py + pyproject.toml)
- Contradictory programmingLanguages (LLM-claimed langs that don't match source scan)
- Low confidence (<0.8 self-assessed)
Stage 2 — seed:autofix applies mechanical fixes
- 32 passports had contradictory tags removed (grounded in detected ecosystem:
.csproj / .sln → C#, pyproject.toml → Python, Cargo.toml → Rust, etc.)
- 33 passports had programmingLanguages corrected from actual source-file scan
(top-3 languages by .ts/.py/.rs/.cs file count, ecosystem-implied primary included)
- 85 passports cleared of ingest.manualReview (schema-clean + no contradictions +
confidence ≥ 0.8)
Stage 3 — seed:relm repairs the 9 broken oneLiners with hermes3:8b
- Sharper repair-focused prompt that starts from current state and forbids the
specific failure patterns (no "Yes,", "Go to", "Read the", no slug tautology,
no file-count hints, no ecosystem mentions)
- All 9 repaired cleanly on first try. Examples:
- nullout: "pip install nullout-mcp" → "MCP server that finds and safely
removes hazardous files on Windows using a scan-plan-delete workflow."
- control-room: "mathew-johnson" → "A local-first desktop app for managing
scripts, multi-step workflows, …"
- codeteam-suite: tautological → "A .NET-based CLI and library for
authoritative cryptographic verification of software packages."
Results
- 104/104 schema-valid
- 85 verified clean (manualReview=false)
- 19 still flagged for manual review — all at confidence=0.7 from the original
backfill, where hermes3:8b self-reported uncertainty. These are the genuine
review queue; mechanical gates can't resolve them.
New scripts + pnpm commands
- seed:review — emits scripts/review-report.json with issue categorization
- seed:autofix — reads the report, applies mechanical fixes, clears clean entries
- seed:relm — re-LLMs specific broken passports with a repair prompt
Regenerated derived artifacts: site/src/data/seeds.json, README tables, llms.txt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three-stage automated review pass on all 104 Wave-2 backfilled passports. Drops the manual-review queue from 104 to 19.
Stage 1 —
seed:reviewDetects: leaked oneLiners (JSON/README fragments), tautologies, short/vague entries, language-tag contradictions, ecosystem mismatches, low-confidence entries.
Stage 2 —
seed:autofixtechnical.programmingLanguagescorrected from actual source-file scaningest.manualReviewcleared (schema-clean + no contradictions + confidence ≥ 0.8)Stage 3 —
seed:relmRepaired 9 broken oneLiners with a sharp repair-focused hermes3:8b prompt. Examples:
pip install nullout-mcpmathew-johnsonA .NET-based CLI for the 'codeteam-suite' developer toolFinal state
manualReview: false)New scripts
pnpm seed:review— emitsscripts/review-report.jsonpnpm seed:autofix— mechanical fixes based on reportpnpm seed:relm— re-LLMs specific packages with repair prompt🤖 Generated with Claude Code