diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index eba4af6..aef550a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -5,7 +5,7 @@ on: branches: [main] pull_request: schedule: - - cron: '0 0 1 1,7 *' + - cron: '0 0 * * 0' workflow_dispatch: inputs: env: @@ -50,7 +50,6 @@ jobs: runs-on: ubuntu-latest env: ENV: ${{ inputs.env || 'dev' }} - GITHUB_TOKEN: ${{ secrets.ARIA_GITHUB_TOKEN }} pact_do_not_track: true steps: - name: Checkout @@ -105,6 +104,7 @@ jobs: build/reports/test-duration-report.md build/reports/portfolio-metrics-v1.json build/reports/spotbugs/** + build/reports/pitest/** - name: Upload Pact contracts uses: actions/upload-artifact@v7 diff --git a/README.md b/README.md index eed1fc4..32fd96d 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,7 @@ $env:BOOKER_PASSWORD="" .\gradlew.bat liveTest -Denv=dev ``` -Scheduled CI runs `liveSmokeTest` weekly against the configured live environment; keep broader live regression runs manual unless the target APIs are controlled. `liveSmokeTest` uses a strict `live & smoke` tag expression, so regression-only live tests are excluded from the scheduled smoke path. +Scheduled CI runs `liveSmokeTest` weekly (Sunday 00:00 UTC) against the configured live environment; keep broader live regression runs manual unless the target APIs are controlled. `liveSmokeTest` uses a strict `live & smoke` tag expression, so regression-only live tests are excluded from the scheduled smoke path. Run the full quality gate used by CI: @@ -162,7 +162,7 @@ Response-time SLA thresholds are environment-configurable through `sla.responseT - CycloneDX SBOM generation in CI - Secret-sanitized failure diagnostics and structured test logs under `build/logs` -GitHub Actions also runs dependency review, OSV scanning, Gradle wrapper validation, Docker-backed container tests, scheduled live smoke tests, and uploads Allure, SpotBugs, test-result, log, SBOM, and OpenAPI coverage artifacts. +GitHub Actions also runs dependency review, OSV scanning, Gradle wrapper validation, Docker-backed container tests, scheduled live smoke tests, and uploads Allure, SpotBugs, PIT mutation, test-result, log, SBOM, and OpenAPI coverage artifacts. Failure-only logs and test-result bundles are clearly labeled as failure diagnostics. ## Runtime Metrics and CI Shape @@ -237,6 +237,8 @@ Do not distribute `.gradle/`, `.idea/`, or `build/` as part of the portfolio sou ## Documentation - [Portfolio Review Guide](docs/Portfolio_Review_Guide.md) +- [Current verification record](docs/evidence/latest-verification.md) +- [Enterprise adaptation](docs/enterprise-adaptation.md) - [Configuration Guide](docs/Configuration_Guide.md) - [Execution Guide](docs/Execution_Guide.md) - [Writing Tests](docs/Writing_Tests.md) diff --git a/docs/CONTRACT_STRATEGY.md b/docs/CONTRACT_STRATEGY.md index e4fba97..2c5a241 100644 --- a/docs/CONTRACT_STRATEGY.md +++ b/docs/CONTRACT_STRATEGY.md @@ -1,14 +1,14 @@ # Contract Strategy -ARIA uses four contract layers: +ARIA uses three contract layers: 1. JSON Schema assertions validate response shape in live endpoint tests. -2. Pact consumer tests (`BookingConsumerPactTest`, `GithubConsumerPactTest`) define deterministic provider expectations for core flows, and `OwnedProviderPactVerificationTest` replays the generated pacts through the real Pact provider verifier against the owned fixture. -3. `OwnedProviderStateContractTest` verifies provider-state-style behavior (mass-assignment rejection, ownership checks, role/expiry handling) that sits outside what a Pact interaction expresses. -4. `OpenApiRuntimeValidationTest` validates every documented operation's live response against its OpenAPI schema (status, media type, and full body-schema conformance via `swagger-request-validator`), and OpenAPI coverage mapping verifies every endpoint in the checked-in API subsets has mapped default-CI tests and complete request/response contracts. +2. Pact consumer tests define deterministic provider expectations for core flows. +3. The owned in-memory provider fixture verifies provider behavior for default-CI endpoint coverage and provider-state style contract checks. +4. OpenAPI coverage mapping verifies every endpoint in the checked-in API subsets has mapped default-CI tests and complete request/response contracts. Response DTOs for public third-party APIs intentionally tolerate unknown fields where the provider may add fields without a breaking change. GitHub response schemas are therefore permissive by policy and should not use `additionalProperties: false` unless a field subset is explicitly owned by ARIA. For owned APIs, use strict JSON Schema (`additionalProperties: false`), strict response DTOs, and avoid `@JsonIgnoreProperties(ignoreUnknown = true)` unless the API contract explicitly allows additive fields. -Pact provider verification runs against the owned in-JVM fixture by default, so it is part of the deterministic gate rather than a live, deployed-provider check. For a deployed owned provider, add a provider-verification job that downloads the pact artifacts or broker pacts, starts the real provider, and runs the Pact verifier before deployment. +Current third-party Pact tests are consumer-side checks and are published as CI artifacts. ARIA also includes `OwnedProviderPactVerificationTest`, which starts the owned fixture and verifies the provider states represented by the core Restful Booker consumer contracts. For a deployed owned provider, add a provider-verification job that downloads those artifacts or broker pacts, starts the real provider, and runs the Pact verifier before deployment. diff --git a/docs/enterprise-adaptation.md b/docs/enterprise-adaptation.md new file mode 100644 index 0000000..7a2ef09 --- /dev/null +++ b/docs/enterprise-adaptation.md @@ -0,0 +1,8 @@ +# Reference Target to Enterprise Adaptation + +| Reference implementation | Enterprise adaptation seam | Production concern not claimed here | +|---|---|---| +| Owned provider and Pact verification | Replace with a service-owned provider boundary and broker/version policy. | No production broker adoption is claimed. | +| Typed clients and service layer | Add product auth/token lifecycle, idempotency keys, and environment-owned test data. | Credentials, rate limits, and tenant isolation remain product-specific. | +| OpenAPI and JSON-schema checks | Bind endpoint coverage to the release contract and backward-compatibility policy. | The demo does not prove production contract governance. | +| HTTP diagnostics | Keep redaction policy and add trace/correlation IDs from the product platform. | No live PII/security posture is claimed beyond the repository policy. | diff --git a/docs/evidence/latest-verification.json b/docs/evidence/latest-verification.json new file mode 100644 index 0000000..f1e5d8b --- /dev/null +++ b/docs/evidence/latest-verification.json @@ -0,0 +1,15 @@ +{ + "schemaVersion": 1, + "repository": "aria-api-framework", + "evidenceState": "evidence-stale", + "evidenceClass": ["controlled", "scheduled-live"], + "verifiedRef": "main", + "verifiedSha": null, + "workflow": {"name": "ci.yml", "runId": null, "runUrl": null, "branch": "main", "event": null, "conclusion": null, "completedAt": null}, + "target": {"name": "owned/container-backed provider", "environment": "controlled"}, + "results": [], + "reportUrl": "https://qa-test-automation-frameworks.github.io/aria-api-framework/", + "artifactRetention": "CI retention plus published Allure history", + "limitationsUrl": "https://github.com/qa-test-automation-frameworks/aria-api-framework/blob/main/docs/known-issues.md", + "generatedAt": "2026-07-11T00:00:00Z" +} diff --git a/docs/evidence/latest-verification.md b/docs/evidence/latest-verification.md new file mode 100644 index 0000000..6f9d6dc --- /dev/null +++ b/docs/evidence/latest-verification.md @@ -0,0 +1,16 @@ +# Current Verification Record + +| Field | Value | +|---|---| +| Repository ref | `main` (refresh after the next weekly scheduled smoke) | +| Fast gate | `ci.yml` — deterministic Gradle quality, security, container, contract, and report gates | +| Full evidence | Weekly scheduled `live-smoke` job; broader live regression remains manual | +| Current state | `evidence-stale`; this record must be refreshed by the next weekly run | +| Target/environment | Owned/container-backed provider plus explicitly configured live smoke target | +| Evidence class | Controlled and scheduled-live | +| Report | [Allure report](https://qa-test-automation-frameworks.github.io/aria-api-framework/) | +| Known limitations | [Known issues](../known-issues.md) and [review guide](../Portfolio_Review_Guide.md) | + +The next record must include the exact SHA, workflow run URL, completion time, +deterministic/container/live scope, OpenAPI/Pact/mutation results, artifact links, +and any external-target limitation.