Fix get_ts_synopsis catch-all triggered by bless trailer after PASS#5002
Merged
Conversation
Replace 'endswith("PASS") or re.search(r"\nPASS\n", ...)' with a
single 're.search(r"^PASS\r?$", comments, re.MULTILINE)' that handles
PASS at end-of-string, on its own line, and at start-of-string
uniformly, and tolerates CRLF line endings without weakening exactness.
Add tests pinning two guarantees:
* Only the verbatim token PASS marks success: variants like 'Pass',
'pass', 'PASSING', 'passed', ' PASS', and 'PASS extra' fall
through to the normal failure-detection logic.
* PASS on a CRLF-terminated line is still recognized, both standalone
and when followed by a bless trailer.
Convert the variants loop in test_get_ts_synopsis_pass_is_case_sensitive_and_exact
to pytest.mark.parametrize so each rejected variant ('Pass', 'pass',
'PASSING', 'passed', ' PASS', 'PASS extra') becomes an individually
reported test case. Failures now identify the offending variant by name
without relying on the per-assert message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes the
master: ERROR Could not interpret CPRNC outputmessage that has been appearing inTestStatus.logfor passing baseline comparisons across entire integration suites.Root cause
compare_baseline(CIME/hist_utils.py:592-598) appends a bless trailer after thePASStoken emitted by_compare_hists:get_ts_synopsisonly handled the case wherecommentsended exactly with"PASS", so once any trailer was present the function fell through every recognized pattern and landed on the catch-allERROR Could not interpret CPRNC output.This became a near-universal symptom after #4952 (
Preserve TestStatus and bless_log) addedBLESS_LOG_NAMEto the preserve list, so prior bless lines now persist across baseline regeneration and the trailer is appended on essentially every comparison.Fix
Replace the brittle
endswith("PASS")check with a single multiline regex that recognizes the verbatimPASStoken regardless of where it sits in the comments and tolerates CRLF line endings without weakening the exact-match guarantee:Tests
New doctest in
get_ts_synopsiscovering the realistic production string'Comparing hists\nPASS\n Most recent bless: sha:abc\n'.Four unit tests in
CIME/tests/test_unit_hist_utils.py:test_get_ts_synopsis_pass_at_end— PASS at end-of-string.test_get_ts_synopsis_pass_on_own_line_with_multiple_lines_after— PASS followed by a bless trailer.test_get_ts_synopsis_pass_is_case_sensitive_and_exact— pins that variants likePass,pass,PASSING,passed,PASS, andPASS extrado NOT short-circuit.test_get_ts_synopsis_pass_handles_crlf_line_endings— PASS on a CRLF-terminated line is recognized, both standalone and with a bless trailer.All hist_utils unit tests + doctests pass locally.
Closes Still seeing "Could not interpret CPRNC output" in TestStatus.log #4932
Checklist