Repository navigation
Conversation
parseCatalog accepted any table row with a kebab-case first cell, so the Intent Routing table in CATALOG.md (harden, document, skip-frontend, skip-backend, security-only) was parsed as five extra agents and could be recommended. Only accept rows whose File column is agents/<id>.md.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
parseCatalog()inscripts/recommendation-engine.jstakes any markdown table row whose first cell is kebab-case. It doesn't limit itself to the Agent Catalog section and doesn't check the File column. So the five rows of Section 2: Intent Routing inskills/agent-registry/CATALOG.md(harden,document,skip-frontend,skip-backend,security-only) get parsed as agents. For those rows the "Primary Agents" column is read asspecialtyand "Secondary Agents" astaskTypes, which means they score against prompts like real agents do.Repro (on
main@ 30662e3)None of
harden,documentorsecurity-onlyhas anagents/<id>.md. Anything that loads the recommended agent's file gets a missing path.Fix
Only accept rows whose File column is
agents/<id>.md(backticks stripped). Every Agent Catalog row has that column, including the Custom Division row format thatagent-creatorwrites. No intent row has it.Verification (Node 20.20.2, same as CI)
tests/recommendation-engine.test.js:parseCatalog()ids must equal theagents/*.mdset, andhardenmust not be recommended. It fails onmainand passes with the fix.parseCatalog().lengthis now 49.npm run validate: PASS 316 / FAIL 0.npm run release:checkpassed. Checksums diff is clean (scripts/isn't in the published set).npm test: 1391 / 1391 pass.I ran 20 prompts (the fixture prompts plus 10 extra) through the engine before and after the change. Three things change: the phantom ids are gone, real agents take some of the freed slots, and in two prompts a real agent drops out. For
security audit of auth, the default output goes from 4 entries to 2 (engineering-security-engineer,project-management-project-shepherd), andwrite docs for the backendlosesengineering-senior-developer. Both drops come from existing behavior, not this change. When the shortlist is shorter thantopN, coordinator injection overwrites the last real candidate instead of appending, and the refill loop is capped atshortlist.length. You can see the same thing onmainwithreview auth, wheretesting-qa-verification-specialistis dropped. Before this change the phantomhardenrow happened to pad the shortlist and hid it in these two cases. I left it out to keep this PR to one fix, and can send it separately if that's useful.🤖 Generated with Claude Code