Skip to content

SCN: strict evaluation gate, append-only evaluation log, published notifications and deadline checks (SCN-*) - #504

Closed
sam-aydlette wants to merge 3 commits into
feat/frr-recordsfrom
feat/scn
Closed

sam-aydlette wants to merge 3 commits into
feat/frr-recordsfrom
feat/scn

Conversation

@sam-aydlette

@sam-aydlette sam-aydlette commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

SCN-Type: not-significant
SCN-Rationale: change-management process and compliance reporting only; no component, boundary, data flow or access change to the system.

Changed

  • SCN gate is strict (SCN-CSO-EVA). .github/workflows/scn-tag.yml now runs scripts/scn-evaluate.py (rules in scripts/_scn.py). Every PR description (or direct-push commit message) must carry SCN-Type: and a one-line SCN-Rationale:; adaptive, transformative and emergency changes also name an SCN-Record:. A missing tag fails; the silent default to routine-recurring is gone. New values not-significant and class-change; class-change always fails. A transformative change fails until its initial plan has been published 30 business days and its final plan 10 (read from the published index, fail-closed). On a push to main the merged PR's description is read via the API. The gate is stricter, never weaker; known-bad PR bodies are in tests/fixtures/scn/bad/.
  • Evaluation log (SCN-CSO-MAR). scripts/build-scn.py (new deploy step "Build Significant Change Notifications and the evaluation log") appends one entry per first-parent commit on main (commit, merge date, PR, type, rationale, record, who merged it) to /.well-known/scn-evaluations.json + .md. Append-only against the previously published copy: dropping or rewriting an entry, or a last entry that left main's history, fails the deploy. With no published copy (404) it starts at the commit that introduced the builder, so a lost copy is rebuilt from git. Free text is scrubbed of account IDs and run IDs; the build fails if a 9–11 digit number would be published.
  • Notifications (SCN-CSO-INF/HIS/HRM/ARI/NOM). Every register row is written in the vendored FedRAMP SCN schema through _fedramp_schema.write_validated and published inside /.well-known/significant-change-notifications.json (signed), with a Markdown view and an Atom feed (significant-change-notifications.atom, one entry per notice). Each notice carries its deadline and the date it was first published (carried forward from the previous index). Never pruned: dropping a notification or re-dating a notice fails the deploy. All three are signed, in the redaction-gate list, the publish step, the round-trip check, and freshness-config.json (deploy cadence, 720 h). The CPO lists them (package contents plus a significant_change_notifications section with mechanisms, access, sharing and the emergency procedure).
  • Deadlines. scripts/_bizdays.py: business days excluding weekends and computed US federal holidays (observed-day rules, including New Year's observed on Dec 31). Timeframes come from the vendored rules (timeframe_num), not code. Adaptive: 10 after finishing. Transformative: initial plan ≥30 before start, final plan ≥10 before, 5 after finishing, 5 after verification, documentation within 30. Emergency: retroactive record within 5 business days of the merge.
  • Scheduled check. New .github/workflows/scn-deadlines.yml (weekdays, public URLs only, no AWS) runs scripts/check-scn-deadlines.py and raises or updates one issue on anything overdue.
  • Register and templates. docs/scn/scn-register.csv gains date_started, verified_on, emergency, evaluation_prs, plan_final, tpr_human_validation, tpr_assessor_or_reason, sar_reference, docs_to_update, docs_updated_on, docs_update_not_required; the incorrect notify_by column ("No necessary parties") is removed. Six rows backfilled from their records: verification dates from their status text; SCN-2026-005 date_finished set to 2026-09-28 (its status records implementation and verification that day); SCN-2026-006 status reworded to "Implemented and verified 2026-10-27" as its record already says (clears the trust center's scn_unverified item); GitHub run IDs removed from SCN-2026-005's status. New docs/scn/evaluation-examples.md (one example per type, each checked against the gate) and docs/scn/TEMPLATE-transformative.md.
  • Prose (AI-drafted, for review): docs/scn/README.md, docs/policies/cm-policy.md (SCN paragraph; the "direct push to main is the default" sentence replaced by the PR flow), docs/incident-response.md emergency procedure (evaluation lines; retroactive record within 5 business days counted from the merge, previously from incident closure), and the two new docs above.

Notification is publication only. FedRAMP's rules name the method for each SCN notice as an update to the Certification Data; this change does that and sends nothing to FedRAMP or anyone else.

This affects every open PR. Once this is in a PR's base, its description needs the SCN-Type: / SCN-Rationale: lines (and SCN-Record: where applicable) or scn-tag fails. That includes #501 (feat/frr-records → main) and every Dependabot PR, whose descriptions must be edited by hand before merge.

Effect

After the deploy that ships this: /.well-known/significant-change-notifications.{json,md,atom} and /.well-known/scn-evaluations.{json,md} appear, signed; the CPO lists them. The six adaptive SCNs are published for the first time. SCN-2026-006's notice is on time if that deploy runs by 2026-11-10; the other five are published marked late (onTime: false, listed under lateNotices). The evaluation log's first entry is the main commit that brings in this builder.

Rule records (python3 scripts/_frr.py): Implemented 6 → 22, Partially Implemented 93 → 78, Not Implemented 42 → 41, Not Applicable 13 → 13.

Flipped to Implemented: SCN-CSO-EVA, SCN-CSO-MAR, SCN-CSO-INF, SCN-CSO-HIS, SCN-CSO-HRM, SCN-CSO-ARI, SCN-CSO-NOM, SCN-CSO-EMG, SCN-RTR-NNR, SCN-TRF-NIP, SCN-TRF-NFP, SCN-TRF-NAF, SCN-TRF-NAV, SCN-TRF-UPD, SCN-ADP-NTF; and SCN-TRF-TPR (Not Implemented before). Evidence is tests/test_scn.py tests, the scn-tag.yml, scn-deadlines.yml and deploy steps, and the new freshness-governed artifacts. Records describe the state after a successful deploy of this commit. SCN-ADP-NTF is Implemented on that basis. A successful deploy has published every notice: a build or publish failure fails the deploy, and the round-trip step compares the served index, feed and log; a test pins both. The scheduled check enforces the 10-business-day deadline. The record names the five historical late notices, SCN-2026-001 to SCN-2026-005, with their deadlines. It says SCN-2026-006 (due 2026-11-10) is on time only if the deploy runs by then, and the published index records which.

Verified

  • make check passes (1031 passed, 1 skipped; policy suites pass). regal lint, check-policy-annotations.py, check-exceptions.py, check-wasm-safe.py pass. tests/test_scn.py (56 tests) cannot import on the base branch (the modules are new) and passes here.
  • Dated exercise, 2026-11-05, local, clearly an exercise (no publish): build-scn.py over this branch built six schema-valid notifications (five marked late), the Atom feed and a one-entry evaluation log read from this branch's own commit message. The publication redaction gate passed on all five outputs. check-scn-deadlines.py over those outputs reported nothing overdue. Against the live site it reports the five past-due notices as unpublished, the expected state until the first deploy.
  • This description passes scripts/scn-evaluate.py.
  • CodeGuard ran over the diff. One finding fixed: the last commit read from the published log is now accepted only as a full SHA before it reaches git. Other checks: subprocess calls use argument lists, untrusted PR text reaches the workflow only through the environment, the Atom output is XML-escaped, URLs are fixed HTTPS, actions are pinned by SHA, workflow permissions are minimal.

Residual

  • The log reads PR descriptions at deploy time, not at merge, so an edit made between the merge and the deploy is what gets recorded. Once logged, an entry is frozen.
  • The previous published index and log are read over HTTPS without verifying their Sigstore bundles. A CloudFront copy up to 5 minutes stale could re-date a notice published by a deploy run inside that window.
  • Per-SCN JSON files are written and validated but published only embedded in the index: freshness-config paths are flat, so they are not published separately.
  • Changes merged before this gate mostly carry no evaluation. The log starts at this change and does not reconstruct them.
  • The scheduled deadline check fails every weekday from merge until the first deploy publishes the six notices.
  • The Secure Configuration Guide still describes the Basic Auth gate retired by SCN-2026-002. SCN-TRF-UPD covers only transformative changes, so this PR does not fix it.

Needs human

  • Apply: none to bootstrap or AWS. The main-stack deploy publishes the new artifacts after merge. The deploy step uses the job's existing GITHUB_TOKEN (it already has pull-requests: write).
  • Add the SCN lines to FedRAMP 20x Class C: every applicable rule Implemented or Not Applicable #501 and to open Dependabot PR descriptions, or they fail scn-tag.
  • Decide whether to tell FedRAMP that the feed exists. Nothing is sent today.
  • Confirm or change: this PR's own type (not-significant); counting the emergency 5 business days from the merge; SCN-2026-001's "upper edge" adaptive categorization, which is unchanged.
  • Review the AI-drafted prose listed above.

Couldn't verify

  • The deploy step end to end: gh api in the deploy job, the first publish, signing, and the round-trip. These run only on an approved deploy.
  • The scn-tag push-event path (fetching the merged PR's description) and the scn-deadlines issue job. Neither has run in Actions yet.

🤖 Generated with Claude Code

sam-aydlette and others added 3 commits November 5, 2026 14:31
…notifications and deadline checks

Every change now carries an explicit significant-change evaluation (SCN-Type,
SCN-Rationale, and SCN-Record for adaptive/transformative/emergency); a
missing tag fails instead of defaulting to routine-recurring, and class-change
blocks. The deploy appends every merged evaluation to a signed, append-only
log and publishes every register row in FedRAMP's SCN schema with a Markdown
view and an Atom feed, listed in the CPO. Business-day deadlines (US federal
holidays) are read from the vendored rules; a scheduled workflow fails on
anything overdue. Notification is publication only; nothing is sent.

SCN-Type: not-significant
SCN-Rationale: change-management process and compliance reporting only; no component, boundary, data flow or access change to the system.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ard)

SCN-Type: not-significant
SCN-Rationale: input validation in the new SCN log builder; no system component, boundary or access change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…torical notices

The record describes the system after a successful deploy of this commit: the
deploy builds and publishes every notice (any failure fails the deploy; the
round-trip check compares the served index and feed), so the mechanism is in
force. SCN-2026-001..005 were published after their deadlines and are named.

SCN-Type: not-significant
SCN-Rationale: rule-record text and a test only; no component, boundary, data flow or access change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sam-aydlette

Copy link
Copy Markdown
Owner Author

Consolidated into #508.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant