fix: feature QA observation captures JSON-LD structured data blocks - #569
fix: feature QA observation captures JSON-LD structured data blocks#569sanrai wants to merge 1 commit into
Conversation
The observation step extracted only card-level data (title, text, links), so any assertion about non-card DOM, such as an injected script[type=application/ld+json] block, was structurally unverifiable and produced an automatic FAIL even when the tag was present (verified present via e2e and manual DOM inspection on the same build). The observation now also captures up to four ld+json blocks with their parent element, attributes, and content, passes them to the validation judge, and reports the block count in the PR comment.
Agent QA review — interactive + visual diff (advisory, non-blocking)Last updated Aug 5, 2026, 1:59 AM PDT · PR opened · commit 0 open · 0 resolved · visual diff 0.00% · verdict PASS Open findings
What the agent checkedTested PR #569 on the live business.adobe.com/resources/main.html page with the PR's CaaS build injected. Context: This PR only touches QA tooling (.github/qa/feature-review.mjs) and adds a new test fixture file (html/e2e/verify-local.html). It does not change any CaaS runtime/rendering code. Consistent with that, the pixel diff of PR vs stable showed 0.00% changed pixels, and the loaded diff.png showed no magenta/highlighted regions — confirming no visual change was introduced by this PR. What I verified on the live page:
No broken, misaligned, truncated, or low-contrast elements were found. All interactive elements (filter checkboxes, search input, clear buttons) behaved as expected. Since this PR is purely a change to the internal QA review script and a new local verification fixture (not shipped to the live site), there is no user-facing regression risk, and testing confirms the live page renders and behaves identically to stable with no defects. Verdict: PASS — no regressions found; page and PR changes are QA-tooling only and functioning as intended. PR / stable / diff screenshots + console + axe artifacts in the workflow run. Review history (1 run)
|
🧪 Feature QA review — injected feature test (advisory, non-blocking)Last updated Aug 5, 2026, 2:01 AM PT · commit Overall: SKIPPED Scenario mapping was not proven -- skipped instead of guessing.
Code searches performed: 8. Screenshot in the workflow run. Review history (1 run)
|
The bug
The Feature QA review reported FAIL on PR #567 twice, claiming no
script[data-caas-jsonld]tag was present, while the same build provably injects it (greenrun-e2e-testsasserting the tag parses, plus manual DOM inspection on localhost).Trace: the Step 5 observation extracts only card-level data,
.consonant-Cardnodes withtitle,text, andlinks. Script tags, meta tags, and container-level attributes never enter the observation. Step 6 then hands the judge only that card extraction, so any assertion about non-card DOM is structurally unverifiable and yields an automatic FAIL. The judge's own wording confirmed it: "not present in the rendered card/container data provided".The fix
The observation now also captures up to four
script[type="application/ld+json"]blocks, each with its parent element, attribute names, and up to 1500 chars of content. They are passed to the validation judge as a separate labeled section and the PR comment reports the block count.{ cards, jsonLd }instead of a bare card array; both downstream consumers (judge prompt, comment renderer) updated to match.node --check.Scope note
This fixes the one blind spot that produced a false FAIL today. The general class, non-visual DOM assertions (meta tags, aria attributes, dataset attributes), still shares the limitation; a follow-up could let the planner emit an
observeSelectorthe harness evaluates directly. Kept out of this PR to stay minimal.Expected effect
Re-running Feature QA on PR #567 after this merges should flip its verdict to PASS, since the judge will finally see the block it was asked about.