Skip to content

Fix get_ts_synopsis catch-all triggered by bless trailer after PASS#5002

Merged
jgfouca merged 3 commits into
masterfrom
fix/4932-cprnc-output
Jun 24, 2026
Merged

Fix get_ts_synopsis catch-all triggered by bless trailer after PASS#5002
jgfouca merged 3 commits into
masterfrom
fix/4932-cprnc-output

Conversation

@jasonb5

@jasonb5 jasonb5 commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Description

Fixes the master: ERROR Could not interpret CPRNC output message that has been appearing in TestStatus.log for passing baseline comparisons across entire integration suites.

Root cause

compare_baseline (CIME/hist_utils.py:592-598) appends a bless trailer after the PASS token emitted by _compare_hists:

Comparing hists for case '...'
  ...matched...
PASS
  Most recent bless: <sha>

get_ts_synopsis only handled the case where comments ended exactly with "PASS", so once any trailer was present the function fell through every recognized pattern and landed on the catch-all ERROR Could not interpret CPRNC output.

This became a near-universal symptom after #4952 (Preserve TestStatus and bless_log) added BLESS_LOG_NAME to the preserve list, so prior bless lines now persist across baseline regeneration and the trailer is appended on essentially every comparison.

Fix

Replace the brittle endswith("PASS") check with a single multiline regex that recognizes the verbatim PASS token regardless of where it sits in the comments and tolerates CRLF line endings without weakening the exact-match guarantee:

# Comparison passed if a line consisting of exactly "PASS" appears.
# _compare_hists writes this token verbatim; any other casing (Pass, pass,
# PASSING, ...) is not a status marker and must fall through. The optional
# \r tolerates CRLF line endings without weakening the exact-match check.
if re.search(r"^PASS\r?$", comments, re.MULTILINE):
    return ""

Tests

  • New doctest in get_ts_synopsis covering the realistic production string 'Comparing hists\nPASS\n Most recent bless: sha:abc\n'.

  • Four unit tests in CIME/tests/test_unit_hist_utils.py:

    • test_get_ts_synopsis_pass_at_end — PASS at end-of-string.
    • test_get_ts_synopsis_pass_on_own_line_with_multiple_lines_after — PASS followed by a bless trailer.
    • test_get_ts_synopsis_pass_is_case_sensitive_and_exact — pins that variants like Pass, pass, PASSING, passed, PASS, and PASS extra do NOT short-circuit.
    • test_get_ts_synopsis_pass_handles_crlf_line_endings — PASS on a CRLF-terminated line is recognized, both standalone and with a bless trailer.
  • All hist_utils unit tests + doctests pass locally.

  • Closes Still seeing "Could not interpret CPRNC output" in TestStatus.log #4932

Checklist

  • My code follows the style guidelines of this project (black formatting)
  • I have performed a self-review of my own code
  • My changes generate no new warnings
  • I have added tests that exercise my feature/fix and existing tests continue to pass
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding additions and changes to the documentation

jasonb5 added 3 commits June 22, 2026 12:02
Replace 'endswith("PASS") or re.search(r"\nPASS\n", ...)' with a
single 're.search(r"^PASS\r?$", comments, re.MULTILINE)' that handles
PASS at end-of-string, on its own line, and at start-of-string
uniformly, and tolerates CRLF line endings without weakening exactness.

Add tests pinning two guarantees:
  * Only the verbatim token PASS marks success: variants like 'Pass',
    'pass', 'PASSING', 'passed', '  PASS', and 'PASS extra' fall
    through to the normal failure-detection logic.
  * PASS on a CRLF-terminated line is still recognized, both standalone
    and when followed by a bless trailer.
Convert the variants loop in test_get_ts_synopsis_pass_is_case_sensitive_and_exact
to pytest.mark.parametrize so each rejected variant ('Pass', 'pass',
'PASSING', 'passed', '  PASS', 'PASS extra') becomes an individually
reported test case. Failures now identify the offending variant by name
without relying on the per-assert message.
@jasonb5 jasonb5 requested a review from jgfouca June 24, 2026 16:53

@jgfouca jgfouca left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! thanks

@jgfouca jgfouca self-assigned this Jun 24, 2026
@jgfouca jgfouca merged commit 9575e12 into master Jun 24, 2026
9 checks passed
@jgfouca jgfouca deleted the fix/4932-cprnc-output branch June 24, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Still seeing "Could not interpret CPRNC output" in TestStatus.log

2 participants