Populate related_skills graph (text-embedding-3-large) + numpy fast path#21
Merged
Conversation
…-embedding-3-large - related_map gains a numpy fast path (pure-Python O(n^2*d) does not scale to 5865 x 3072 vectors — 48min -> 49s); element-wise finite-guards drop degenerate embedding values; pure-Python kept as the numpy-free fallback (identical output, 17 tests green). numpy added to the [embed]/[embed-openai] extras. - Populated related_skills for all 5,865 skills (skills_index.json + kb_bundle.json) via the OpenAI text-embedding-3-large backend (the model Perspicacité uses). 0 dangling refs; check_related_skills gate OK; neighbours are semantically sound. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lfnothias
commented
Jun 26, 2026
lfnothias
left a comment
lfnothias
left a comment
Contributor
Author
There was a problem hiding this comment.
Auto-review — VERDICT: MERGEABLE ✅
Independently verified in the worktree:
related_skillspopulated for 5,865/5,865 skills in bothskills_index.jsonandkb_bundle.json; 0 dangling refs;check_related_skillsexit 0.related_mapnumpy path matches the pure-Python fallback (17/17 tests); finite-guards make it warning-free; 49 s vs the 48-min pure-Python stall.- Semantic spot-checks pass (NMR↔NMR, chromatography↔chromatography neighbours).
- Full suite 434 passed after a conflict-free merge of
dev(Alpha+Beta). numpy>=1.24added to the embed extras; no git/gh/network in the build path (embeddings via the injected/OpenAI backend only at CLI time).
lfnothias
added a commit
that referenced
this pull request
Jun 26, 2026
Propagates two post-#19 merges into the 8 packs (which weren't rebuilt after): - synthesize-meta-skill.md: the #7 --review-doi addition (PR #20) - kb_bundle.json: related_skills from the #1 embedding precompute (PR #21) claim-skill.md was already shipped to all packs in 2be335f (PR #19) and is unchanged here. Idempotent: re-running build_all_grounding.sh now yields no diff. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Populates the
related_skillssimilarity graph (the B4 machinery shipped empty in #18) using OpenAItext-embedding-3-large— the same model Perspicacité runs. Offdev.What landed
related_skills(top-8 cosine neighbours) inskills_index.json+kb_bundle.json. 0 dangling refs;check_related_skillsgate green. Neighbours are semantically sound (e.g. a 2D-NMR skill → NMR simulation / J-coupling skills; a 2D-TIC skill → chromatography-preprocessing skills).related_mapnumpy fast path — the pure-Python O(n²·d) ranker doesn't scale to 5,865 × 3,072-dim vectors (48 min → 49 s). Element-wise finite-guards drop degenerate embedding values; the pure-Python path is kept as the numpy-free fallback (identical output, 17 tests green).numpy>=1.24added to the[embed]/[embed-openai]extras.Notes
--backend openaipath (text-embedding-3-large); the embeddings are model-dependent, so a regeneration may shift a few borderline neighbours — expected for an embedding artifact (the gate enforces referential integrity, not byte-idempotence).