Skip to content

fix: feature QA observation captures JSON-LD structured data blocks - #569

Closed
sanrai wants to merge 1 commit into
mainfrom
qa-observe-jsonld
Closed

fix: feature QA observation captures JSON-LD structured data blocks#569
sanrai wants to merge 1 commit into
mainfrom
qa-observe-jsonld

Conversation

@sanrai

@sanrai sanrai commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

The bug

The Feature QA review reported FAIL on PR #567 twice, claiming no script[data-caas-jsonld] tag was present, while the same build provably injects it (green run-e2e-tests asserting the tag parses, plus manual DOM inspection on localhost).

Trace: the Step 5 observation extracts only card-level data, .consonant-Card nodes with title, text, and links. Script tags, meta tags, and container-level attributes never enter the observation. Step 6 then hands the judge only that card extraction, so any assertion about non-card DOM is structurally unverifiable and yields an automatic FAIL. The judge's own wording confirmed it: "not present in the rendered card/container data provided".

The fix

The observation now also captures up to four script[type="application/ld+json"] blocks, each with its parent element, attribute names, and up to 1500 chars of content. They are passed to the validation judge as a separate labeled section and the PR comment reports the block count.

  • Observation returns { cards, jsonLd } instead of a bare card array; both downstream consumers (judge prompt, comment renderer) updated to match.
  • Verified with a static consumer-consistency check (all consumers read the new shape, no stale references) and node --check.

Scope note

This fixes the one blind spot that produced a false FAIL today. The general class, non-visual DOM assertions (meta tags, aria attributes, dataset attributes), still shares the limitation; a follow-up could let the planner emit an observeSelector the harness evaluates directly. Kept out of this PR to stay minimal.

Expected effect

Re-running Feature QA on PR #567 after this merges should flip its verdict to PASS, since the judge will finally see the block it was asked about.

The observation step extracted only card-level data (title, text,
links), so any assertion about non-card DOM, such as an injected
script[type=application/ld+json] block, was structurally unverifiable
and produced an automatic FAIL even when the tag was present (verified
present via e2e and manual DOM inspection on the same build).

The observation now also captures up to four ld+json blocks with their
parent element, attributes, and content, passes them to the validation
judge, and reports the block count in the PR comment.
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Agent QA review — interactive + visual diff (advisory, non-blocking)

Last updated Aug 5, 2026, 1:59 AM PDT · PR opened · commit 90ebbd6 · 2 files changed.

0 open · 0 resolved · visual diff 0.00% · verdict PASS

Open findings

  • Smoke test passed ✅
What the agent checked

Tested PR #569 on the live business.adobe.com/resources/main.html page with the PR's CaaS build injected.

Context: This PR only touches QA tooling (.github/qa/feature-review.mjs) and adds a new test fixture file (html/e2e/verify-local.html). It does not change any CaaS runtime/rendering code. Consistent with that, the pixel diff of PR vs stable showed 0.00% changed pixels, and the loaded diff.png showed no magenta/highlighted regions — confirming no visual change was introduced by this PR.

What I verified on the live page:

  1. Initial page load: Resource Center renders normally, header, breadcrumb, filter panel, card grid, and chat widget all display correctly with no layout issues.
  2. JSON-LD structured data: confirmed via evaluate() that two <script type="application/ld+json"> tags exist in (lengths 134 and 278 chars) — exactly the kind of non-card DOM artifact this PR's observation step is meant to capture for QA judging. This confirms the underlying premise of the PR (that JSON-LD is genuinely present) is correct.
  3. Filter panel interaction: expanded "Products" filter group, checked "Acrobat" — filter chip appeared ("Acrobat x"), checkbox count badge updated to "1", result count updated from 3771 to 181, and card grid correctly refreshed to Acrobat-related content (Forrester Acrobat Studio study, Acrobat PDF accessibility). Unchecking the box worked cleanly (clear function fired without error).
  4. Search input: typed "photoshop" into the left-panel search field without errors.
  5. Console errors: none observed after all interactions.
  6. Accessibility: ran axe-core scoped to the filter panel — 0 violations, 23 passes, 0 incomplete.

No broken, misaligned, truncated, or low-contrast elements were found. All interactive elements (filter checkboxes, search input, clear buttons) behaved as expected. Since this PR is purely a change to the internal QA review script and a new local verification fixture (not shipped to the live site), there is no user-facing regression risk, and testing confirms the live page renders and behaves identically to stable with no defects.

Verdict: PASS — no regressions found; page and PR changes are QA-tooling only and functioning as intended.

PR / stable / diff screenshots + console + axe artifacts in the workflow run.

Review history (1 run)
  • 90ebbd6 · Aug 5, 2026, 1:59 AM PDT · PR opened · passing, no regressions — fix: feature QA observation captures JSON-LD structured data

@sanrai

sanrai commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Folding this into #567 instead: the Feature QA workflow preserves and runs the reviewer from the PR's own merge ref, so carrying the observation fix inside #567 means the next review of #567 already uses it. Same change, cherry-picked there.

@sanrai sanrai closed this Aug 5, 2026
@sanrai
sanrai deleted the qa-observe-jsonld branch August 5, 2026 08:59
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

🧪 Feature QA review — injected feature test (advisory, non-blocking)

Last updated Aug 5, 2026, 2:01 AM PT · commit 7e72275 · pull_request.

Overall: SKIPPED

Scenario mapping was not proven -- skipped instead of guessing.

The changed unit test in this PR (.github/qa/feature-review.mjs) is a test-harness/tooling change to the QA observation script itself, not a product feature under test in the CaaS card grid. It captures script[type=application/ld+json] tags for the judge's benefit, but the research explicitly found no production consumer (showJsonLd, JsonLd, structuredData) anywhere in react/src/js, and no evidence that any card config field causes the app to render a script[type=application/ld+json] tag. The verify-local.html example uses a 'showJsonLd':true flag in its state payload, but source search across react/src/js turned up zero matches for that flag being read/consumed, so there is no proven injection path from card/config JSON to an actual rendered ld+json script tag. Fabricating a card style or

Code searches performed: 8.

Screenshot in the workflow run.

Review history (1 run)
  • Aug 5, 2026, 2:01 AM PT · SKIPPED · 7e72275 · pull_request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant