diff --git a/README.md b/README.md index 50c202019a3..70a5f477cc8 100644 --- a/README.md +++ b/README.md @@ -90,19 +90,29 @@ Allure results, selected by test labels: | Dashboard | all results (aggregate) | | Meshery | `project == "Meshery"` | | Mesheryctl | `project == "mesheryctl"` | -| **Kubernetes Connections** | `epic == "Kubernetes Connections"`; results with no `epic` label also match via the Kubernetes `componentUnderTest` fallback (UI + CLI connection tests, grouped by `client`) | +| **Connection Lifecycle** | `testGroup == "Connection Lifecycle"` (Test Plan Test Group, col B); transitionally, results carrying **no** `testGroup` label also match the legacy `epic == "Kubernetes Connections"` / Kubernetes `componentUnderTest` fallback (UI + CLI connection tests, grouped by `client`) | | Extension: Remote Provider Layer5 Cloud | `project == "Layer5Cloud"` | | Extension: Kanvas | `project == "Kanvas"` | -The Kubernetes Connections report is a cross-client behavior lens: connection -tests are tagged at their source (UI Playwright specs, CLI converters) with -`epic="Kubernetes Connections"`, `componentUnderTest`, `testId` (`TC-`), and -`client` (`UI`|`CLI`), sourced from the Meshery Test Plan. Tagged tests still -appear in their `project` report; the Connections report is an additional view. -As a fallback, a result that carries no `epic` label at all is included when its -`componentUnderTest` matches Kubernetes, so results predating the `epic` -convention still appear; a result with a *different* `epic` value is not pulled -in by the component fallback. +The Connection Lifecycle report is a **Test-Group-keyed view**: each test is +tagged at its source (UI Playwright specs, CLI converters) with a `testGroup` +label whose value is the Meshery Test Plan "Latest" tab Test Group (column B). +The report filters on `testGroup == "Connection Lifecycle"`. This pattern +**generalizes to one filtered report per Test Group** - any Test Group can get +its own report by keying a plugin's filter on its `testGroup` value. Tagged +tests still appear in their `project` report; this is an additional +cross-client (`UI`|`CLI`) lens. + +Transitionally, a result that carries **no** `testGroup` label at all also +matches the legacy epic-based selector (`epic == "Kubernetes Connections"`, with +a Kubernetes `componentUnderTest` fallback for results carrying no `epic` label) +so the report stays populated with connection results emitted before the +`testGroup` label existed. A result that already carries a `testGroup` is +authoritative - it is never pulled in by the legacy fallback even if its +component looks like Kubernetes - so once every connection result carries +`testGroup`, the fallback can be removed with no change in membership. The +report is published at https://qa.meshery.io/connections/ (the plugin key is +kept stable so the URL is unchanged).

 

diff --git a/allurerc.mjs b/allurerc.mjs index 6fb14e42944..20cf787ec33 100644 --- a/allurerc.mjs +++ b/allurerc.mjs @@ -13,20 +13,41 @@ const PROJECTS = { const isProject = (labels, projectName) => labels.find(({ name, value }) => name === "project" && value === projectName); -// Behavior-scoped report keys. A Connections result is tagged at its source -// (UI Playwright specs and CLI converters) with epic="Kubernetes Connections", -// componentUnderTest (Test Plan col C), testId=TC- (Test Plan col A), and -// client (UI|CLI). See https://qa.meshery.io and the meshery test-tagging docs. +// Test-Group-keyed report selector. The general report key is the `testGroup` +// label, whose value is the Meshery Test Plan "Latest" tab Test Group (col B). +// Every Test Group can drive its own filtered report by keying on this label; +// "Connection Lifecycle" is the first consumer. Results are tagged at their +// source (UI Playwright specs and CLI converters) - see the meshery +// test-tagging docs and https://qa.meshery.io. +const isTestGroup = (labels, groupName) => + labels.some( + ({ name, value }) => name === "testGroup" && value === groupName, + ); + +const hasTestGroup = (labels) => labels.some(({ name }) => name === "testGroup"); + +// Single source of truth for the Connection Lifecycle Test Group value - used +// for both the report display name and the testGroup filter so the two cannot +// drift apart. +const CONNECTION_LIFECYCLE_GROUP = "Connection Lifecycle"; + +// --- Transitional epic-based fallback (remove once all connection results +// carry testGroup) ----------------------------------------------------------- +// Before this rename the connection report keyed on epic="Kubernetes +// Connections" plus a Kubernetes componentUnderTest fallback. Kept only so the +// report is not empty for results emitted before the testGroup label existed; +// drops once fresh runs of every connection test carry testGroup. const CONNECTIONS_EPIC = "Kubernetes Connections"; // Matches componentUnderTest values that denote Kubernetes connection behavior. // Used as a fallback selector when a result predates the epic label. const CONNECTION_COMPONENT_RE = /kubernetes/i; -// Select a result into the Connections report: prefer the explicit epic label. -// The componentUnderTest fallback applies ONLY to results that carry no epic -// label at all (tagged before the epic convention) - a result with a different -// epic value must not be pulled in just because its component is Kubernetes. +// Select a result into the connection report via the legacy epic label: prefer +// the explicit epic label. The componentUnderTest fallback applies ONLY to +// results that carry no epic label at all (tagged before the epic convention) - +// a result with a different epic value must not be pulled in just because its +// component is Kubernetes. const isConnectionBehavior = (labels) => { const hasEpic = labels.some(({ name }) => name === "epic"); if (hasEpic) { @@ -123,19 +144,37 @@ export default defineConfig({ groupBy: ["parentSuite", "suite", "subSuite"], }, }, - // Cross-client behavior report: aggregates Kubernetes Connection tests from - // BOTH the UI (project=Meshery) and CLI (project=mesheryctl) pools. It keys - // on the epic label, NOT project, so it is an additional lens - connection - // tests still appear in the Meshery and Mesheryctl reports above. + // Test-Group-keyed report: a filtered view over one shared Allure pool, + // selected by the `testGroup` label (Test Plan col B). This "Connection + // Lifecycle" report aggregates the connection tests from BOTH the UI + // (project=Meshery) and CLI (project=mesheryctl) pools - it keys on + // testGroup, NOT project, so it is an additional lens; those tests still + // appear in the Meshery and Mesheryctl reports above. The same pattern + // generalizes to one filtered report per Test Group: add a plugin whose + // filter is isTestGroup(labels, ""). + // + // The plugin key stays `connections` so the published report URL + // (https://qa.meshery.io/connections/) is unchanged; only the display name + // and filter change. + // + // Transitional: the isConnectionBehavior (epic) fallback keeps the report + // populated with connection results emitted before the testGroup label + // existed. It applies ONLY to results carrying no testGroup label at all - + // a result that already carries a testGroup (of any value) is authoritative, + // so a different-group result must not be pulled in via the legacy epic/ + // component heuristic. Drop the whole fallback once all connection results + // carry testGroup. connections: { import: "@allurereport/plugin-awesome", options: { - reportName: "Kubernetes Connections", + reportName: CONNECTION_LIFECYCLE_GROUP, singleFile: false, reportLanguage: "en", open: false, logo: "https://raw.githubusercontent.com/meshery-extensions/qa/refs/heads/master/.github/assets/images/meshery/icon-only/meshery-light-icon.svg", - filter: ({ labels }) => isConnectionBehavior(labels), + filter: ({ labels }) => + isTestGroup(labels, CONNECTION_LIFECYCLE_GROUP) || + (!hasTestGroup(labels) && isConnectionBehavior(labels)), // Group by client (UI vs CLI) first, then the suite hierarchy. The // awesome plugin (preciseTreeLabels) keeps only label names present on // at least one result, so results missing "client" fall back to the