Skip to content

fix: corrective classifier precision — reduce false positive rate #68

Description

@asvarnon

Problem

The corrective classifier matches ~45% of passages in validation (target: 10–25%). The negation-proximity heuristic (negation marker within 5 words of a triggering term + non-question filter) is too permissive for conversational transcript data.

Negation words (not, don't, shouldn't, etc.) appear naturally at high density in English conversation. Finding one within 5 words of any recurring term is statistically likely in any substantive passage — this produces false positives rather than genuine corrective patterns.

Evidence

Validation Run Passages Avg Length Corrective %
Post Pre-Filter (8 entries, 309 chars avg) 1,919 309 chars 19.1%
Post-#49 (10 entries, 3,037 chars avg) 120 3,037 chars 42.5%
Post-Passage Cap (10 entries, 413 chars avg) 745 413 chars 45.0%

The 19% → 45% jump correlates with 2 new large conversational entries, not passage length. The passage cap (PR #67) fixed decisive precision but had no effect on corrective.

Possible Approaches

  1. Tighter proximity window — Reduce from 5 words to 3. Risk: false negatives on legitimate patterns like "you should NOT use unwrap in library crates" (negation 3-4 words from term).

  2. Require correction-specific context — Not just negation near a term, but negation + correction pattern ("don't use", "should not", "never do", "stop using", "avoid"). This would filter out incidental negation ("this is not the only way", "not just for testing").

  3. Negation-term directionality — Require the negation to precede the term (most corrections say "don't do X", not "X is not"). Would reduce matches where negation follows the term incidentally.

  4. Sentence-level scope — Only check negation proximity within the same sentence as the triggering term, not across sentence boundaries within the passage.

  5. Combined approach — Sentence-scoped proximity (option 4) + correction-pattern requirement (option 2). Most restrictive but highest precision.

Priority

Low — does not block any current implementation. Should be addressed after the importance detection pipeline (#50) is complete and validated. The corrective category weight (1.5) is provisional until precision is acceptable.

Dependencies

Acceptance Criteria

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions