feat: stub-extraction guard — the roster/directory class - #3
Merged
Conversation
Found by the 2026-07-24 post-lift audit. A page with no prices, no
table and no wall gave every existing guard nothing to arm on, so a
content-bearing page could return its boilerplate and drop everything
else, silently:
sbir.gov/awards 10,698 visible -> 421 chars: only the
"official website of the United States
government" banner. Zero award data.
noaa SBIR archive 6,373 visible -> 361 chars: 1 of 3+ success
stories kept.
apex/our-team 1,994 visible -> 161 chars: a marketing
sentence; all four staff entries dropped.
nomic.ai/pricing 3,043 visible -> 259 chars: marketing copy,
zero prices (the price guard could not arm).
This is NOT the bare yield ratio measured and rejected in v0.1.2. That
design fires on neither real pricing failure (smith.ai 13%, quo 21%
retention) and false-positives on nav-heavy pages. Retention is half
the condition; the other half is an absolute stub-sized output, and
that floor is what separates the groups. The rejection reasoning is
preserved in the code comment so this cannot be "simplified" back.
Calibrated by replaying every content-bearing page in a live 162-page
cache: fires on 5 (3.1%), each verified a genuine omission. Silent on
the LinkedIn false-positive page (10.2%), heyrosie (14.0%), smith.ai
(13.2%), quo (20.7%) and eur-lex (5.2%, a legitimate terse result
list, which is why the floor must not rise past ~570).
Calibration note worth keeping: measure with a FRESH extract_content,
never a cached markdown column. Stored rows can carry
include_links=True, which inflated noaa to 614 chars and hid it as
"healthy" during the first pass.
Gate: ruff, mypy (16 files), 247 passed / 6 deselected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wynelson94
added a commit
that referenced
this pull request
Jul 24, 2026
The stub-extraction guard (#3) shipped with README text reading "Since v0.1.5", while pyproject.toml and __init__.py still said 0.1.4 and the CHANGELOG carried the guard under [Unreleased]. That is the same version-drift class PR #1 was opened to fix, reintroduced in the same day. Bumps both manifests, promotes [Unreleased] to [0.1.5], and adds the link ref pointing at 32858dd. The rest of the [Unreleased] block (MIT LICENSE, CI gate, synthetic FP fixture, UA correction) also shipped after 0.1.4, so it moves under 0.1.5 with the guard rather than lingering as unreleased. No git tags exist in this repo, so CHANGELOG links continue to point at shipping commits. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Found by the 2026-07-24 post-lift audit. Existing guards arm on prices, tables, or walls. A page with none of those gave them nothing to work with — so a content-bearing page could return its boilerplate and drop everything else, silently, with no warning line.
Four real cases sitting in the live cache right now:
sbir.gov/awardsidahoapexaccelerator.com/our-teamatlas.nomic.ai/pricingThree of the four are federal grant-research pages, which is where this tooling actually gets used.
This is not the rejected design
v0.1.2 measured and threw out a bare markdown/visible-text yield ratio: it fires on neither real pricing failure (smith.ai 13%, quo 21% retention) and false-positives on nav-heavy pages. That reasoning still holds and is preserved verbatim in the code comment so this can't be "simplified" back into it.
Retention is only half the condition here. The other half is an absolute stub-sized output, and that floor is what separates the two groups:
Verification
TestLowYieldcovers the fire case plus three distinct silence cases (nav-heavy-but-healthy, faithfully-short page, below-substance-floor).ruff check,mypy(16 files), 247 passed / 6 deselected (was 243).Calibration gotcha worth keeping (it's in the code comment and the test docstring): measure with a fresh
extract_content, never a cachedmarkdowncolumn. Stored rows can carryinclude_links=True, which inflated NOAA to 614 chars and hid it as "healthy" on my first pass — I had it in the silent column until the full-cache replay caught it.No real staff names or third-party page bytes are added as fixtures; tests are synthetic, matching the repo's existing posture.
🤖 Generated with Claude Code