docs(observer): expand Test Failure Extraction and Analysis README section#418
Closed
ProtocolWarden wants to merge 2 commits into
Closed
docs(observer): expand Test Failure Extraction and Analysis README section#418ProtocolWarden wants to merge 2 commits into
ProtocolWarden wants to merge 2 commits into
Conversation
…ction Rewrites README.md:1038–1340 to cover all extraction capabilities introduced since the section was last updated. Adds four new subsections — Monitoring Extraction Health, Gaps and Edge Cases, Alert Integration, and Troubleshooting — documenting the extraction-health CLI command, ExtractionHealth fields (success_rate, message_quality_rate, gaps, edge_cases, low_quality_messages), alert thresholds, and diagnostic guidance. All code examples verified against the live implementation (cli.py, ExtractionHealth dataclass, FlakyTestAlertConfig). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…board-unblock console-repair)
Owner
Author
|
Needs human attention (reason= CI has not gone green after 20 checks (1 failing: audit: failure). Not merged (red CI) and not closed (work preserved) — needs a human to fix CI. |
Owner
Author
|
Closing: CONFLICTING with main (superseded by the observer changes in #417) and stuck in reviewer escalation, blocking the goal lane via OPEN_PR_GATE. The underlying docs task (c2c5588c) can be re-readied to regenerate cleanly against current main. — operator-driven cleanup |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
extraction-healthCLI, JSON/table examples, field reference table), Gaps and Edge Cases (all three sample lists with per-value meaning tables), Alert Integration (threshold table, channel routing, programmatic example), and Troubleshooting (message_quality_rateinterpretation table + diagnostic pattern)success_ratevs. qualitymessage_quality_rate) and reorders sections so health monitoring comes before queryingAccuracy verification
All examples were verified against the actual source before merging:
ExtractionHealthdataclass fields (query_flaky.py:107)cli.py:1079–1106character-for-characterhistoryobject shape matchescli.py:1058–1070includingtrend,weekly_trend,slope,anomalies,observations,recent,snapshots_prunedFlakyTestAlertConfig(flaky_test_alert_config.py:131–142)EXTRACTION_SUCCESS_RATE_LOWandMESSAGE_QUALITY_RATE_LOWFlakyTestAlertManager.check_message_quality_rate()signature and description format verifiedTimeRangeimport fromoperations_center.observer.queryconfirmed correctTest plan
test_extraction_health_queries.py,test_cli_extraction_health.py,test_assertion_extractor.py)🤖 Generated with Claude Code