Repository navigation
W6: incident evaluation and reporting (IEC-CSO-EFR/DPR/EFI/IIR/OIR/FIR/AIR, VER-TFR-IRI/NRI) - #506
Closed
sam-aydlette wants to merge 2 commits into
Closed
sam-aydlette wants to merge 2 commits into
sam-aydlette wants to merge 2 commits into
Conversation
…/NRI triggers - data/incidents.json + schema + fail-closed loader (scripts/_incidents.py): every incident carries its FedRAMP reportability evaluation and reason; default N5 until an IEC-CSO-EFI estimate (checked against the rule's rubric). - scripts/build-incident-report.py renders Initial/Ongoing/Final reports in FedRAMP's incident report schema (validated before write) and the incident log; Class C clocks and the rating scale are read from the pinned rules. - policies/triage.rego incident_triggers: VER-TFR-IRI and VER-TFR-NRI, with thresholds parsed from the rule text (fail closed on a changed text). - Reconciliation invariant (m): an open incident's findings are in the VDR. - incident-reports.yml publishes unattended (publish-only; nothing is sent); bootstrap role scoped to .well-known/incident-*; the deploy also publishes; watchdog metric and alarm for overdue reports; freshness entry. - Dated exercise (2026-11-05, exercise: true): IIR -> 2x OIR -> FIR, plus a not-reportable evaluation; run end to end in tests. - FRR records: EFR, DPR, EFI, VER-TFR-IRI, VER-TFR-NRI Implemented; IIR, OIR, FIR, AIR Partially Implemented (publish-only; bootstrap apply pending). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s state the deployed system The deploy fails before applying anything unless AWS_INCIDENT_REPORTS_ROLE_ARN names the bootstrap incident-reports role and the incident-reports environment deploys only from main (scripts/check-incident-publisher.py; known-bad fixture with the variable missing). After the deploy job, the deploy calls incident-reports.yml (now also workflow_call) on its own role, passing only the CloudFront secret, so a green deploy proves the publisher publishes. IEC-CSO-IIR/OIR/FIR/AIR gaps now state only what holds after a successful deploy: published, not delivered to FedRAMP or agencies, by the operator's decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
Consolidated into #508. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SCN-Type: not-significant
SCN-Rationale: PROVISIONAL, Sam to confirm the type: adds a scoped IAM role and an incident-report workflow.
Changed
data/incidents.json(schemainfrastructure/schemas/incidents.schema.json, stdlib fail-closed loaderscripts/_incidents.pyfollowing_decisions.py). Each incident records detection, an IEC-CSO-EFR evaluation (time, reportable true/false, basis, reason, federal customer data considered), Incident Rating estimates (IEC-CSO-EFI; rating, customer effect, one or several agencies, reason, time) and the reports issued (Initial → Ongoing → Final, each with time and the items it sets), plusresolved_at/recovery_complete_atandexercise. A reportable incident is rated at the IEC-CSO-DPR default (N5, read from the rule text) until an estimate exists; an estimate whose rating is not the one the EFI bullets give for its effect and agency count is refused.scripts/_rules.py): Class C IIR/OIR/FIR clocks, the EFI rating rubric and the DPR default are read from the pinned corpus; a changed shape fails closed.scripts/build-incident-report.py: every report goes through_fedramp_schema.write_validatedagainst the vendored incident report schema. Ongoing and Final reports restate every earlier item and mark each one "updated" or "no update". The builder also writes the incident logincident-reports.jsonand.md(evaluations and reasons, rating history, each report with sha256, each clock with due/issued/on time, reports still due).--checkexits 1 when a report is overdue.policies/triage.regoincident_triggers/incident_errors): VER-TFR-IRI (IRV+LEV+PAIN>N3) and VER-TFR-NRI (NIRV+LEV+N5). The thresholds are parsed from the Class C rule text, and a changed text fails closed. A finding stops triggering once partially mitigated to N3 / N4 or below, and false positives and fully mitigated findings never trigger. Adopted rules are listed in the register policy (both adopted, perdocs/policies/ir-policy.md). A trigger that no unresolved incident names puts an Initial report due from the finding's first evaluation at the N5 clock. The VDR builder and its golden report are unchanged.reconcile.m(block) in the gate-mode register..github/workflows/incident-reports.ymlruns on push ofdata/incidents.jsonto main, on workflow_dispatch, twice daily, and after each successful nightly. Steps: verify the published VDR signature, build, sign (cosign keyless), redaction-gate, publish under.well-known/incident-*, invalidate, check freshness at the edge, then check the report clocks. A failure raises one reused GitHub issue.infrastructure/bootstrap/incident-reports.tf:s3:PutObjecton.well-known/incident-*pluscloudfront:CreateInvalidation. Trust is limited to theincident-reportsenvironment subject. It is classified in the trust-root group, has IIW text, and appears as boundary flow L.scripts/check-incident-publisher.py). It fails the deploy if:AWS_INCIDENT_REPORTS_ROLE_ARNis unset, is not an IAM role ARN, or names a role other thangithub-actions-incident-reports-oidc(the name is tied to the bootstrap file by a test; the ARN and account are never printed);incident-reportsenvironment does not exist (404), has no custom branch policy, or its branch policies are anything other than exactlymain.tests/fixtures/incidents/preflight-missing-variable.json(variable missing, environment correct) fails for the variable alone.incident-reports.ymlgainsworkflow_call. A new deploy-workflow jobincident-publisher(needsdeploy) calls it, so it assumes its own role from theincident-reportsenvironment, builds, signs, publishes and checks the clocks. If it fails, the run fails andnotifyraises the deployment-failure issue. OnlyCLOUDFRONT_DISTRIBUTION_IDis passed, notsecrets: inherit.IncidentReportsOverdue, computed from the published log'sobligations_dueon AWS's clock; a missing or unreadable log counts as 1. New alarmincident-report-overdue. The log has a freshness-config entry (24h, signed).docs/policies/ir-policy.md,docs/incident-response.md(reportability evaluation step before containment, and a new "Reporting FedRAMP Reportable Incidents (IEC)" section),docs/cr26-applicability.md,docs/runbooks/evidence-sla-alarm.md, CLAUDE.md map.exercise: true:tests/test_incidents.pyruns the exercise end to end on everymake check.Notification is "published only". Nothing is sent to FedRAMP or to any agency. The operator has not authorized sending, and this system has no certification and no agency customers. Every report, the log and the policy say so.
Effect
After a successful deploy (which requires the prerequisites under Needs human):
/.well-known/incident-reports.json(signed) and.md, plus four signed exercise reportsincident-INC-2026-001-0{1..4}-*.json, are served.Records (
python3 scripts/_frr.py, 154 rules), before → after: Implemented 6 → 11, Partially Implemented 93 → 97, Not Implemented 42 → 33, Not Applicable 13 → 13. The follow-up commit changes no statuses, only gap text.Verified
make checkpasses (ruff, mypy, pytest, opa fmt/check/test). Policy coverage is 95.7%, 49/49 policy tests pass.infrastructure/policyis clean.check-policy-annotations.py,check-exceptions.pyandcheck-wasm-safe.pypass.tests/test_incidents.py, the watchdog incident tests and the rego incident tests fail on the base, because the code does not exist there.Configure AWS credentialsandTerraform Applyin the deploy job and thatincident-publisheris wired intonotify. The new tests fail on the previous commit (the script does not exist there).secrets: inheriton the new job (least privilege), was fixed by passing only the CloudFront secret.AWS_INCIDENT_REPORTS_ROLE_ARNis not set and theincident-reportsenvironment does not exist.Residual
emitted_atand signatures show when publication happened.verify-published.shandcheck-manual-settings.pydo not yet cover the incident artifacts or the new environment.Needs human
SCN-Type: not-significantabove is provisional. The change adds a scoped IAM role, a new unattended publishing workflow and a deploy dependency on it, so it may be adaptive. Sam to confirm; no SCN record is added here.infrastructure/bootstrap/incident-reports.tffrom a tree synced to main.AWS_INCIDENT_REPORTS_ROLE_ARNfrom theincident_reports_role_arnoutput.incident-reports: no reviewer, deployment branches = custom, exactlymain.decided_by: Operator;reconcile.mmode block (recorded as proposed);Couldn't verify
incident-publisherjob were not run in Actions. Not run: the role assumption, publishing, cosign signing in CI, and the alarm in AWS.GITHUB_TOKEN(contents: read) can read/environments/incident-reportsand its branch policies was not observed. If it cannot, the preflight says so, and theincident-publisherjob still fails the deploy when the role cannot be assumed from that environment.concurrencygroup in a called workflow applies when it is called from the deploy is untested. The groups differ, so no deadlock is possible either way.make check-full/ reconcile against live artifacts (needs AWS) was not run.🤖 Generated with Claude Code