Skip to content

Fix inconsistent lastTransitionTime format causing 2.6→2.7 upgrade failure - #279

Open
skashid1882 wants to merge 2 commits into
ansible:mainfrom
skashid1882:fix/normalize-lastTransitionTime-format-AAP-86911
Open

skashid1882 wants to merge 2 commits into
ansible:mainfrom
skashid1882:fix/normalize-lastTransitionTime-format-AAP-86911

Conversation

@skashid1882

Copy link
Copy Markdown

Summary

  • Adds a normalization step in the common role that detects and patches any lastTransitionTime values written in space-separated format (YYYY-MM-DD HH:MM:SS+00:00) to proper RFC3339 (YYYY-MM-DDTHH:MM:SS+00:00)
  • Runs early in each reconciliation so conditions that never re-transition (e.g. Routes-Ready) are fixed before CRD validation on upgrade
  • Prevents InstallComponentFailed errors during supported AAP 2.6 → 2.7 upgrades caused by the tightened date-time format constraint in the 2.7 CRD

Root Cause

The k8s_status module (operator_sdk.util) in certain code paths uses Python's str(datetime.now(timezone.utc)) which produces a space between date and time, rather than .isoformat() which produces the required T separator. Conditions whose status never flips (same True/False value) retain the original malformed timestamp indefinitely because the module only rewrites lastTransitionTime on actual transitions.

Changes

File Change
roles/common/tasks/normalize_condition_timestamps.yml New task: detects and patches malformed timestamps
roles/common/tasks/main.yml Includes normalization after fetching current CR state
CHANGES/86911.bugfix Changelog entry

Test Plan

  • Deploy AAP 2.4 operator, create a fresh AutomationHub CR, confirm conditions include space-separated timestamps
  • Apply this fix (upgrade to patched 2.6 operator), run reconciliation, confirm all lastTransitionTime values are now T-separated
  • Upgrade to AAP 2.7 — InstallPlan should succeed without CRD validation errors
  • Verify no regression on clusters where all timestamps are already RFC3339 compliant (normalization should be a no-op)

Fixes: AAP-86911

Made with Cursor

The k8s_status module in older operator_sdk.util versions occasionally
wrote status condition timestamps using Python's str(datetime) which
produces space-separated format ("YYYY-MM-DD HH:MM:SS+00:00") instead
of the RFC3339-mandated T-separated format ("YYYY-MM-DDTHH:MM:SS+00:00").

Conditions whose status never transitions (e.g. Routes-Ready once set)
retain the malformed value indefinitely. When upgrading from AAP 2.6 to
2.7, the tightened CRD validation rejects these values, blocking the
InstallPlan with "must be of type date-time".

This adds a normalization step early in reconciliation that detects and
patches any space-separated timestamps to proper RFC3339 format, ensuring
clean upgrades through supported paths.

Fixes: AAP-86911
Co-authored-by: Cursor <cursoragent@cursor.com>
@skashid1882
skashid1882 force-pushed the fix/normalize-lastTransitionTime-format-AAP-86911 branch from 46d5e63 to 79492a2 Compare August 21, 2026 12:51
The k8s_status module's merge_status_conditions() calls
has_condition_transitioned() which explicitly ignores lastTransitionTime
changes. This caused the normalization patch to be silently discarded
as a no-op. Using replace: true with the full status object bypasses
the merge logic entirely.

Fixes: AAP-86911
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant