Skip to content

ACM-42590 Contract/golden test framework for backend migration - #52

Open
Ginxo wants to merge 12 commits into
mainfrom
ACM-42590
Open

ACM-42590 Contract/golden test framework for backend migration#52
Ginxo wants to merge 12 commits into
mainfrom
ACM-42590

Conversation

@Ginxo

@Ginxo Ginxo commented Sep 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

NO NEED TO MERGE, JUST FOR THE RECORD

we need just agreement on this

📝 Summary

Ticket: ACM-42590 — Contract/golden test framework for backend migration
Epic: ACM-42568

Type of Change:

  • 🧪 Test-related
  • 📄 Docs

Executive summary

Isolated Go module in ACM-42568_go_migration/. Black-box contract tests against the Node backend today; same catalog gates the Go cutover later. Not wired into root package.json or CI.

Delivered

  • 73 YAML cases (REST, SSE, WebSocket) from backend/src/app.ts + frontend/plugin usage
  • Go runner: assert / compare / record, TLS-insecure, Bearer/cookie, /multicloud, normalization
  • run-catalog.sh — one command to run the full catalog (~2 min)
  • Colored summary at end: Executed / OK (green) / SOFT (yellow) / FAIL (red)
  • QUIRKS.md — Node behaviors Go should replicate vs. fix later

Gate rule: FAIL = 0 on mandatory cases. SOFT skips are optional upstreams missing on your hub (normal on dev).

How to run (npm run plugins already up)

Prerequisites: oc login, npm run plugins from repo root (Node on :4000), Go 1.22+.

You do not need OpenShift Console on :9000 — only the backend on port 4000.

cd ACM-42568_go_migration
./run-catalog.sh

Quick check: curl -sk https://localhost:4000/ping → 200.

Example summary (colors in terminal):

 Executed: 107
 OK: 84
 SOFT (skipped): 23
 FAIL: 0
 Mandatory cases passed. 23 optional case(s) skipped.
Result Meaning
OK (green) Mandatory case passed
SOFT (yellow) Optional case skipped — missing Search, Observability, cluster-proxy agent, OCM/ROSA, etc.
FAIL (red) Must be 0 for a green migration gate

Manual alternative (per-subtest log + summary):

cd ACM-42568_go_migration
export CONTRACT_BACKEND_URL=https://localhost:4000
export CONTRACT_TOKEN=$(oc whoami -t)
go test . -count=1 -timeout 15m -v -run TestCatalogAgainstBackend

NO_COLOR=1 disables ANSI colors.

Optional: through OCP Console proxy (:9000)

Only if you also run the full Console stack:

cd ACM-42568_go_migration
export CONTRACT_BACKEND_URL=http://localhost:9000
export CONTRACT_PATH_PREFIX=/api/proxy/plugin/mce/console/multicloud
export CONTRACT_TOKEN=$(oc whoami -t)
go test . -count=1 -timeout 15m -v -run TestCatalogAgainstBackend

Other modes

Variable Default Effect
CONTRACT_MODE assert Status, headers, JSON, SSE, WS
CONTRACT_COMPARE_URL — Diff REST cases against second backend
CONTRACT_RECORD=1 — Write captures to testdata/recorded/

See ACM-42568_go_migration/README.md and QUIRKS.md.

Relationship to child epic stories

All work is ACM-42590. Folder name references the epic; cases are not tagged with child ticket keys.

Child ticket Area Catalog
ACM-42592 Main API proxy 10-kube-proxy.yaml
ACM-42593 Stateless proxies 20-metrics-mcp-vm-serve.yaml + 60-websocket.yaml
ACM-42594 Static / serve 20-metrics-mcp-vm-serve.yaml (serve)
ACM-42595 OAuth 30-auth-cluster.yaml
ACM-42596 Auth + cluster-info 30-auth-cluster.yaml
ACM-42597 + ACM-42598 Informer + SSE 50-sse.yaml, watched-resources.yaml
ACM-42600 Aggregation 40-aggregate-search.yaml
ACM-42601 Search + WS 40-aggregate-search.yaml, 60-websocket.yaml
ACM-42602 Long-tail 70-long-tail.yaml

ACM-42589, ACM-42591, ACM-42599 — no dedicated file; covered indirectly (probes, negatives).

Soft tests (soft: true)

Optional for the gate. Runner skips (not fails) when status is in softStatuses or the request errors/times out (soft cases only).

Mandatory: kube proxy, SSE, auth, aggregation, negatives, probes.

Case ID Why soft Typical skip on dev hub
metrics-prometheus-query Thanos/Prometheus proxy 5xx if monitoring unreachable
metrics-observability-query ACM Observability addon 500 not installed
metrics-prometheus-query-range Same as Prometheus 5xx
mcp-hub-namespaces cluster-proxy-addon needs agent 502 no agent
vm-get-missing KubeVirt/CNV APIs 404/500
vm-get-snapshot-missing Same 404/500
vm-resource-usage CNV resource metrics 500
serve-plugin-manifest Static assets on webpack :3001/:3002, not Node public/ 404
search-graphql search-api in cluster 502/timeout
search-websocket WS relay to search-api 502/timeout
mcp-websocket-watch WS via cluster-proxy 502
placement-debug OCM placement debug CA 500/503
upgrade-risks Insights / pull-secret 4xx/5xx
rosa-* (11 routes) OCM API; test uses invalid SA 500

✅ Checklist

General

  • PR title follows the convention
  • Code builds and runs locally without errors
  • No console logs, commented-out code, or unnecessary files
  • All commits are meaningful and well-labeled
  • All new display strings are externalized for localization (English only)
  • (Nice to have) JSDoc comments added for new functions and interfaces

If Feature

  • UI/UX reviewed (if applicable)
  • All acceptance criteria met
  • Unit test coverage added or updated
  • Relevant documentation or comments included

🗒️ Notes for Reviewers

  • Module is outside frontend/, backend/, root package.json, and CI.
  • CONTRACT_COMPARE_URL is for Node-vs-Go diffs later.
  • SOFT skips on a minimal hub are expected and documented above.

Ginxo added 3 commits September 1, 2026 07:32
@Ginxo
Contract/golden test framework for backend migration
ef4c34c 
Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
@Ginxo
Merge branch 'main' into ACM-42590
7f5add5 
Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
@Ginxo
run-catalog.sh
b698b3f 
Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
@Ginxo

Ginxo commented Sep 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner Author

against current main
image

@Ginxo Ginxo changed the title Contract/golden test framework for backend migration ACM-42590 Contract/golden test framework for backend migration Sep 2, 2026
Ginxo and others added 4 commits September 2, 2026 10:05
@Ginxo @cursoragent
Add contract catalog preflight for hub alignment and TLS.
46a5b4b 
Fail fast on CLUSTER_API_URL mismatch, missing certs, or a dead /ping so catalog FAILs are not false negatives.

Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Ginxo @cursoragent
Complete watched-resources.yaml with selectors matching events.ts.
bb695d0 
Each of the 67 definitions keeps labelSelector, fieldSelector, polled, and forwardEventsToClients. A unit test diffs the YAML against backend-node definitions.

Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Ginxo @cursoragent
Add informer cache snapshot harness for ACM-42597.
2ae7302 
Normalize {apiVersion,kind,namespace,name}, exclude polled kinds, and skip when GET /debug/informer-snapshot is not wired yet. REST compare stays SSE-free until ACM-42598.

Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Ginxo @cursoragent
Document catalog layers and add GET /events/rbac SSE case.
4c8dfe8 
Clarify REST vs SSE compare limits, plugin hub/TLS failures, and the ACM-42597 snapshot gate. events-rbac-sse is soft so sidecar-only runs still skip 404.

Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Ginxo

Ginxo commented Sep 2, 2026

Copy link
Copy Markdown
Owner Author

new report against upstream/main after latest improvements

Backend: https://localhost:4000
Building test binary...
Running full catalog (TestCatalogAgainstBackend)...
PASS

════════════════════════════════════════════════════
 Contract test summary
════════════════════════════════════════════════════
 Executed: 109
 OK: 84
 SOFT (skipped): 25
 FAIL: 0
════════════════════════════════════════════════════
 Soft skips (optional upstream missing):
   metrics-observability-query — status 500 is in softStatuses
   metrics-observability-query/multicloud — status 500 is in softStatuses
   mcp-hub-namespaces — status 502 is in softStatuses
   mcp-hub-namespaces/multicloud — status 502 is in softStatuses
   vm-resource-usage — status 500 is in softStatuses
   vm-resource-usage/multicloud — status 500 is in softStatuses
   serve-plugin-manifest — status 404 is in softStatuses
   serve-plugin-manifest/multicloud — status 404 is in softStatuses
   events-rbac-sse — status 404 is in softStatuses
   events-rbac-sse/multicloud — status 404 is in softStatuses
   mcp-websocket-watch — websocket dial: websocket: bad handshake (status 502)
   placement-debug — status 500 is in softStatuses
   placement-debug/multicloud — status 500 is in softStatuses
   rosa-aws-account-ids — status 500 is in softStatuses
   rosa-aws-account-ids/multicloud — status 500 is in softStatuses
   rosa-billing-accounts — status 500 is in softStatuses
   rosa-oidc-configs — status 500 is in softStatuses
   rosa-regions — status 500 is in softStatuses
   rosa-cluster-name-check — status 500 is in softStatuses
   rosa-sts-role-arns — status 500 is in softStatuses
   rosa-vpcs — status 500 is in softStatuses
   rosa-sts-ocm-role — status 500 is in softStatuses
   rosa-sts-user-role — status 500 is in softStatuses
   rosa-openshift-versions — status 500 is in softStatuses
   rosa-machine-types — status 500 is in softStatuses
 Mandatory cases passed. 25 optional case(s) skipped.
════════════════════════════════════════════════════

Ginxo and others added 5 commits September 2, 2026 12:40
@Ginxo
updates (#56)
5385887
@Ginxo
40-aggregate-search.yaml ACM-42600
6f5f233 
Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
@Ginxo
Merge remote-tracking branch 'origin/ACM-42590' into ACM-42590
8c09691 
Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
@Ginxo
tests updated and improved
b2baf6e 
Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
@Ginxo
Merge remote-tracking branch 'origin/ACM-42590' into ACM-42590
ccbbe99 
Signed-off-by: Enrique Mingorance Cano <ginxaco@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Ginxo