feat: feature added failure intelligence

CHANGELOG.md

@@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 - **HTML report: tab and failure fixes** — the report always opens on the Dashboard (it no longer restores the last-open tab), failure details start collapsed, and the Failures tab no longer goes empty after you expand or collapse a class in Test Cases (both tabs shared one group id).
 
+- **Failure Intelligence: high-level triage and suggested actions** — each failure is classified as `Product bug`, `Test bug`, `Environment`, `Flaky` or `Unknown` (flaky keeps its cause, e.g. `Flaky (Locator not found)`), with a rule-based suggested action. Shown in the HTML report, as `failureType` / `suggestedAction` in the metrics JSON, and in the JUnit XML; opt-in AI analysis uses it as a baseline. No API key needed. (#138)
+
 - **AI provider is configurable** — `ai.provider` selects the backend for failure analysis and the run summary: `anthropic` (default, unchanged), `gemini`, or `openai` (any OpenAI-compatible `/chat/completions` endpoint — OpenAI, Azure, Groq, local Ollama via `ai.baseUrl`). `ai.model` now defaults per provider. Existing configs keep working with no change.
 
 - **AI run summary (opt-in)** — `ai.runSummary: true` adds one short paragraph to the HTML dashboard summarising the run's failures: which same-cause groups account for most of them, which look like product bugs versus test or environment problems, and what to check first. One API call per run, independent of `ai.failureAnalysis`; only grouped error messages and test names are sent (see the data note in the HTML report docs). Absent when disabled, when nothing failed, or on any error; the suite result is never affected. The text is exported as `aiRunSummary` in the metrics JSON.

docs-site/docs/changelog.md

@@ -11,6 +11,8 @@ All notable changes to Selenium Boot are documented here.
 
 ## [Unreleased]
 
+- **Failure Intelligence: high-level triage and suggested actions** — each failure is classified as `Product bug`, `Test bug`, `Environment`, `Flaky` or `Unknown` (flaky keeps its cause, e.g. `Flaky (Locator not found)`), with a rule-based suggested action. Shown in the HTML report, as `failureType` / `suggestedAction` in the metrics JSON, and in the JUnit XML; opt-in AI analysis uses it as a baseline. No API key needed. (#138)
+
 - **HTML report: tab and failure fixes** — the report always opens on the Dashboard (it no longer restores the last-open tab), failure details start collapsed, and the Failures tab no longer goes empty after you expand or collapse a class in Test Cases (both tabs shared one group id).
 
 - **AI provider is configurable** — `ai.provider` selects the backend for failure analysis and the run summary: `anthropic` (default, unchanged), `gemini`, or `openai` (any OpenAI-compatible `/chat/completions` endpoint — OpenAI, Azure, Groq, local Ollama via `ai.baseUrl`). `ai.model` now defaults per provider. Existing configs keep working with no change.

docs-site/docs/guides/failure-intelligence.md

@@ -0,0 +1,113 @@
+---
+description: "Failure Intelligence in Selenium Boot: know why a test failed and what to do, with zero setup and no API key."
+id: failure-intelligence
+title: Failure Intelligence
+sidebar_position: 12
+---
+
+# Failure Intelligence
+
+**Failure Intelligence** takes you from *"a test failed"* to *"I know why and what to do"*, working on the first run with zero configuration and no API key.
+
+---
+
+## Why Failure Intelligence?
+
+When automated test suites grow, diagnosing failures quickly becomes the biggest bottleneck:
+- **Alert fatigue**: 40 tests failing due to a single backend outage read as 40 independent bugs.
+- **Ambiguity**: Error messages like `TimeoutException` or `NoSuchElementException` don't distinguish between an application defect, a broken locator, or an environment timeout.
+- **Trial-and-error triage**: Engineers often have to reproduce the failure locally or search through massive stack traces to figure out what to fix.
+- **AI lock-in**: Most triage tools require commercial LLM API keys and send test code or stack traces to external servers.
+
+Selenium Boot's Failure Intelligence solves this by diagnosing failures **locally**, using rule-based classification, history awareness, and root-cause grouping.
+
+---
+
+## 1. High-level triage classification (`FailureType`)
+
+Every failed test is automatically classified into a high-level triage type:
+
+| Classification | Meaning | Typical Trigger |
+|---|---|---|
+| **`Product bug`** | The application behavior diverged from expectations (potential regression). | Assertion failure (`AssertionError`, `ComparisonFailure`), especially when newly failing. |
+| **`Test bug`** | An issue in test code, selectors, or test sequencing. | `LocatorException`, `NoSuchElementException`, `ElementNotInteractableException`, `StaleElementReferenceException`. |
+| **`Environment`** | Infrastructure, network, browser, or backend server outage. | `NoSuchSessionException`, `UnreachableBrowserException`, `net::ERR_*`, DNS failures, HTTP 5xx errors. |
+| **`Flaky`** | Non-deterministic test outcome due to timing or shared state. | Passed on retry, or marked `INTERMITTENT` based on run history. |
+| **`Unknown`** | Unrecognised or unexpected exception, or generic timeout. Raw stack trace is preserved. | `TimeoutException` with no locator or navigation cause, or any unmapped exception. The classifier never guesses. |
+
+Shown as a color-coded badge (`[Product bug]`, `[Test bug]`, `[Environment]`, `[Flaky]`) in the HTML report and exported as `failureType` in `selenium-boot-metrics.json`.
+
+---
+
+## 2. Granular category and evidence (`FailureCategory`)
+
+Below the high-level classification, each failure is tagged with a fine-grained category and the exact evidence extracted from the exception cause chain:
+
+| Category | Triggered by |
+|---|---|
+| `LOCATOR_NOT_FOUND` | `LocatorException`, `NoSuchElementException`, `NoSuchFrameException`, `NoSuchShadowRootException` anywhere in the cause chain. |
+| `ELEMENT_NOT_INTERACTABLE` | `ElementNotInteractableException`, `ElementClickInterceptedException`, `InvalidElementStateException`. |
+| `STALE_ELEMENT` | `StaleElementReferenceException` anywhere in the cause chain. |
+| `TIMEOUT` | A `TimeoutException` with no locator or navigation cause. |
+| `ASSERTION_MISMATCH` | Outermost error is an `AssertionError` (or TestNG/JUnit/`DbAssertException` equivalent), or soft assertions failed. |
+| `NAVIGATION_ERROR` | `net::ERR_*`, DNS failures, renderer timeouts, page load timeouts, or HTTP 5xx status codes in the message. |
+| `BROWSER_CRASH` | `NoSuchSessionException`, `UnreachableBrowserException`, "invalid session id", or browser process crash. |
+| `UNKNOWN` | None of the above. |
+
+---
+
+## 3. Actionable remediation ("What to do", No API Key)
+
+Failure Intelligence does not just report *why* a test failed — it provides prescriptive, context-aware remediation guidance:
+
+- **Locator not found**: Names the exact failing locator when available and recommends checking if the selector changed, or if the element is inside an iframe, shadow DOM, or delayed by async loading.
+- **Element not interactable**: Identifies whether the element is covered by an overlay/modal, disabled, or outside the viewport, recommending scroll or modal dismissal.
+- **Stale element**: Recommends using dynamic `$(...)` locators which auto-wait and re-resolve instead of holding cached `WebElement` references.
+- **Assertion mismatch**: Highlights expected vs. actual values. When newly failing, it advises verifying whether the application regressed or if requirements changed and test expectations need updating.
+- **Navigation error (HTTP 5xx)**: Instructs checking backend service health, database connectivity, and `execution.baseUrl`.
+- **Browser crash**: Suggests reviewing driver binary compatibility, memory limits, and container resource limits.
+- **Flaky**: Preserves the underlying failure category (e.g. `Flaky (Locator not found)`) so you know what caused the flake, and recommends replacing arbitrary thread sleeps with explicit `Locator` waits and isolating test state.
+
+The remediation guidance appears in an **Action** callout box in the HTML report and is exported as `suggestedAction` in metrics.
+
+---
+
+## 4. Root cause grouping ("N tests, same cause")
+
+When multiple tests fail for the same reason (for example, a shared login locator change or a backend service outage), Failure Intelligence groups them by a signature:
+- Failure category
+- Outermost exception class
+- Normalised error message (UUIDs, numbers, timestamps, and quoted strings replaced with placeholders)
+- Top application stack frame (page object or test method)
+
+The HTML report's **Failures** tab opens with a *"Same cause (N tests)"* section showing the largest failure groups first. One outage no longer reads as forty disconnected failures.
+
+---
+
+## 5. Retry and history status (`TestStatus`)
+
+Failures are compared against the test's recent history in `target/metrics-history/` (or `-Dseleniumboot.history.dir`):
+
+- **Newly failing**: Passed in all earlier runs in the window, but failed in this run. These are sorted to the top of the Failures tab as the most likely regressions.
+- **Consistently failing**: Failed in this run and in earlier runs without passing.
+- **Intermittent**: Alternates between passes and failures across recent runs.
+- **Passed on retry**: Failed on initial attempt, but recovered upon retry.
+
+---
+
+## 6. Optional AI enhancement (`ai.failureAnalysis`)
+
+When you configure an LLM API key (`ai.apiKey`), Selenium Boot passes the structured Failure Detail record (including rule-based classification, failing locator, console errors, self-healing events, and same-cause group context) to the model.
+
+Crucially:
+- **Tests in the same-cause group share one AI call**, so you make far fewer AI calls.
+- **AI output adheres to the framework's classification taxonomy**:
+  ```markdown
+  **Classification:** (Product Bug | Test Bug | Environment | Flaky)
+  **Root Cause:** (1-2 sentences explaining what likely went wrong)
+  **Suggested Fix:**
+  - (bullet 1)
+  - (bullet 2, if needed)
+  ```
+- **Privacy by default**: If no API key is provided, zero data leaves your machine. Classification and remediation work 100% locally.
+

docs-site/docs/reporting/html-report.md

@@ -59,22 +59,25 @@ Same detail panels as Test Cases tab, but only for failed tests. Detail panels a
 
 ---
 
-## Failure classification
+## Failure classification & intelligence
 
-Each failed test is tagged with a category and the evidence for it, worked out from the exception class chain and message the framework already records. It needs no AI, no API key and no network. The badge and evidence appear at the top of the failure's detail panel; the raw error and stack trace stay right below.
+Each failed test is tagged with a high-level triage classification (`Product bug`, `Test bug`, `Environment`, `Flaky`), a fine-grained category, and the evidence for it, worked out from the exception class chain and message the framework already records. It also provides actionable remediation advice ("Action: ...") right out of the box with no AI, no API key, and no network. The triage badge, category, and action callout appear at the top of the failure's detail panel; the raw error and stack trace stay right below.
 
-| Category | Triggered by |
-|---|---|
-| `LOCATOR_NOT_FOUND` | `LocatorException`, `NoSuchElementException`, `NoSuchFrameException` or `NoSuchShadowRootException` anywhere in the cause chain, including a `LocatorException` caused by a `TimeoutException` |
-| `ELEMENT_NOT_INTERACTABLE` | `ElementNotInteractableException`, `ElementClickInterceptedException`, `InvalidElementStateException` |
-| `STALE_ELEMENT` | `StaleElementReferenceException` anywhere in the cause chain |
-| `TIMEOUT` | a `TimeoutException` with no locator or navigation cause |
-| `ASSERTION_MISMATCH` | the outermost error is an `AssertionError` (or TestNG/JUnit/`DbAssertException` equivalent), or soft assertions failed |
-| `NAVIGATION_ERROR` | `net::ERR_*`, DNS failures, "Timed out receiving message from renderer", page load timeouts, HTTP 5xx in the message |
-| `BROWSER_CRASH` | `NoSuchSessionException`, `UnreachableBrowserException`, "invalid session id", page crash |
-| `UNKNOWN` | none of the above. The classifier never guesses |
-
-The category is also written to the metrics JSON as `failureCategory` and `failureEvidence`, and to the JUnit XML as the `type` of `<failure>`.
+| Classification | Category | Triggered by |
+|---|---|---|
+| `Product bug` | `ASSERTION_MISMATCH` | the outermost error is an `AssertionError` (or TestNG/JUnit/`DbAssertException` equivalent), or soft assertions failed |
+| `Test bug` | `LOCATOR_NOT_FOUND` | `LocatorException`, `NoSuchElementException`, `NoSuchFrameException` or `NoSuchShadowRootException` anywhere in the cause chain, including a `LocatorException` caused by a `TimeoutException` |
+| `Test bug` | `ELEMENT_NOT_INTERACTABLE` | `ElementNotInteractableException`, `ElementClickInterceptedException`, `InvalidElementStateException` |
+| `Test bug` | `STALE_ELEMENT` | `StaleElementReferenceException` anywhere in the cause chain |
+| `Unknown` | `TIMEOUT` | a `TimeoutException` with no locator or navigation cause |
+| `Environment` | `NAVIGATION_ERROR` | `net::ERR_*`, DNS failures, "Timed out receiving message from renderer", page load timeouts, HTTP 5xx in the message |
+| `Environment` | `BROWSER_CRASH` | `NoSuchSessionException`, `UnreachableBrowserException`, "invalid session id", page crash |
+| `Flaky (Category)` | Any | When retried and passed (`Passed on retry`) or history indicates `Intermittent` execution. Retains the underlying category (e.g. `Flaky (Locator not found)`) |
+| `Unknown` | `UNKNOWN` | none of the above. The classifier never guesses |
+
+The classification and action are also written to the metrics JSON as `failureType`, `failureCategory`, `failureEvidence`, and `suggestedAction`, and to the JUnit XML.
+
+For full architecture and design details, see [Failure Intelligence](../guides/failure-intelligence.md).
 
 ---
 

src/main/java/com/seleniumboot/ai/AiFailureAnalyzer.java

@@ -143,8 +143,12 @@ static String buildPrompt(
         com.seleniumboot.failure.FailureClassifier.Result classification =
                 com.seleniumboot.failure.FailureClassifier.classify(timing);
         sb.append("\n## Failure Classification\n");
+        sb.append("- Type:     ").append(classification.type().label()).append(" (").append(classification.type().name()).append(")\n");
         sb.append("- Category: ").append(classification.category().name()).append("\n");
         sb.append("- Evidence: ").append(classification.evidence()).append("\n");
+        if (classification.suggestedAction() != null && !classification.suggestedAction().isBlank()) {
+            sb.append("- Baseline recommendation: ").append(classification.suggestedAction()).append("\n");
+        }
 
         detail.appendPrompt(sb);
 
@@ -173,6 +177,7 @@ static String buildPrompt(
 
         sb.append("\n## Your Task\n");
         sb.append("Provide a concise failure analysis in this exact format:\n\n");
+        sb.append("**Classification:** (Product Bug | Test Bug | Environment | Flaky)\n\n");
         sb.append("**Root Cause:** (1-2 sentences explaining what likely went wrong)\n\n");
         sb.append("**Suggested Fix:**\n- (bullet 1)\n- (bullet 2, if needed)\n\n");
         sb.append("Be specific and actionable. Do not repeat the error message verbatim.");

src/main/java/com/seleniumboot/failure/FailureCategory.java

@@ -20,4 +20,36 @@ public enum FailureCategory {
     public String label() {
         return label;
     }
+
+    /** The default high-level {@link FailureType} associated with this category. */
+    public FailureType defaultType() {
+        return switch (this) {
+            case ASSERTION_MISMATCH -> FailureType.PRODUCT_BUG;
+            case LOCATOR_NOT_FOUND, ELEMENT_NOT_INTERACTABLE, STALE_ELEMENT -> FailureType.TEST_BUG;
+            case NAVIGATION_ERROR, BROWSER_CRASH -> FailureType.ENVIRONMENT;
+            case TIMEOUT, UNKNOWN -> FailureType.UNKNOWN;
+        };
+    }
+
+    /** Actionable remediation advice for this failure category without an API key. */
+    public String defaultAction() {
+        return switch (this) {
+            case LOCATOR_NOT_FOUND ->
+                "Check if the element selector has changed in the application, or if it renders inside an iframe, shadow DOM, or after an asynchronous animation.";
+            case ELEMENT_NOT_INTERACTABLE ->
+                "Element is present in the DOM but not clickable or visible. Check if obscured by an overlay/modal, disabled, or needs scrolling into view.";
+            case STALE_ELEMENT ->
+                "Element detached from DOM after retrieval. Use dynamic Locator actions $(...) which auto-wait and re-resolve instead of caching WebElement references.";
+            case TIMEOUT ->
+                "Operation timed out waiting for condition. Check if page response or network is unusually slow, increase timeouts.explicit if expected, or adjust wait condition.";
+            case ASSERTION_MISMATCH ->
+                "Assertion failed. Compare expected vs actual values; check if product functionality changed or test expectation is outdated.";
+            case NAVIGATION_ERROR ->
+                "Page navigation or network request failed. Verify target URL, server status (HTTP 5xx), DNS resolution, and execution.baseUrl configuration.";
+            case BROWSER_CRASH ->
+                "Browser process or driver session closed unexpectedly. Check driver binary compatibility, available container memory, and headless mode options.";
+            case UNKNOWN ->
+                "Unclassified failure. Review the root exception and application stack trace below for details.";
+        };
+    }
 }